Let's be honest for a moment. In your downvote activity, you're not voting where you want the rewards to go. They are safe and sound in the reward pool and haven't gone anywhere. What you're doing is voting where you don't want them to go, and that is punitive

That's the same thing and no I don't see it as punitive. Voting where they don't go means they go to others. They don't get taken by the downvoter.