You do have "control or influence" by punishing people for what you consider "inappropriate behavior".

That's your entire argument in favor of downvoting.

Downvoting gives you "control and influence".

Wouldn't unjustified downvotes (like "vegan wars") qualify as "personal attacks" and or "harassment"?

Would you consider downvoting unjustified downvoters?