You state,

If you think about it, those rewards that get returned from downvotes are distributed equally with everyone else. The blockchain doesn't keep track of who downvoted what and give the downvoter the rewards it returns to the pool, in that case you may start to wonder if the motives are that but considering how much active stake there is in total (I think about 200m HP) it makes little sense for me as a 340k HP account to go out and downvote ~4 posts per day or however much my downvote mana would afford just so those $20 in rewards were to be redistributed to literally all other accounts receiving curation rewards.

This is not true. The rewards go back into the pool where the larger stakes get the highest percentage of it. It is NOT split equally.