Essentially one vote vetos all the other votes
If you look at how these large payouts are generated it is almost always the case that the bulk of the rewards comes from one or a few big accounts. Your notion that masses of minnows are all voting for a big reward and then one evil whale comes and takes it all away is not accurate.
RE: The Downside of Downvotes