I agree with the above comment. It is basically a tradeoff. Having less votes makes it much easier to get enough witnesses to block consensus, but more difficult to get all 20 (Community could focus all votes on 1-4 candidates).

I do like the "witness downvoting" potential. That could make things more interesting.