20 block-producers/witnesses with one in reserve is why our transactions are so fast. Ethereum is playing with the idea of doing something similar with their protocol which will be a hybrid of Proof of Stake (PoS) and Proof of Work (PoW).

EOS also uses the Delegated Proof of Stake (DPoS) protocol. Too many block-producers/witnesses would slow things down.

I notice that you are currently only voting 6 witnesses. Have you checked the witness list lately?