It's the same whales (or different ones) who vote for most of the rewards in the first place. Somehow it is perfectly legitimate to get whale upvotes but not whale downvotes? I don't think so.