It might not even be about bullying you. For all we know, that guy just had a bad day and your post came across as complaining and annoying to him so he smacked it with a downdoot. And since he knows he's got a heavy hammer to swing, he swings it.

r0nd0n, a well-known user, downvoted my wife's post just to be a goof on air. Expecting it to get used properly is like expecting reddit users to not use the upvote system as "I agree with this comment" or "I don't agree with this comment" buttons.