Downvotes need to go in my opinion. It should be enough that people vote for content they want to vote for.