Your reputation goes down when someone with a higher reputation downvotes you and your post gets more downvotes than upvotes. Someone must not have liked something you posted.