Upvotes and downvotes do have an effect on visibility. When you upvote someone you increase the visibility of that person's post and increase their share of the reward pool. When you downvote someone you reduce the visibility of their post and undue the votes of others. So you not only censor the person but you render other voters' vote worthless and hurt their curation reward value.

Also, how can you not be aware of the many flag wars that have occurred on Steem? There have been plenty of flag wars that are well known here, so its very unreasonable to make the claim that you have not witnessed it happen. If you truly have never seen it happen then it is more suggestive of a lack of experience on Steem than anything. Fulltimegeek vs themarkymark, berniesanders vs everybody and so on... The existence of flag wars is not up for debate, subjective, motivated downvoting is historic fact proved by the immutable blockchain.