When you downvote a post with no rewards and the account has higher rep than you, you have burned your downvote to no gain.
Your downvote only affects lower rep accounts, in the proportion that it has been downvoted.
Zeroing rewards does not impact reputation, only when the balance goes below zero.