So you're against the downvote wars as actions of the people involved rather than being against the downvote option altogether right?
On Youtube within the past year we saw the removal of showing the downvotes. The reason given was that it could emotionally/mentally anguish the creators if there was a targeted campaign against them from another creator with a sizable following. This is sort of like the whales situation here I'd say.
But in reality, Youtube seemed to do this because of cultural/political dissent. The current US presidents official White House channel was the first one to "test" this function before it was rolled out to public, and prior to that the videos had terrible ratios.
Additionally, you'd see similar ratios on movies pushing certain social/political agendas. One of their most powerful weapons via social-media is making people feel like they are in a minority on opinion, and seeing those downvotes made people realize their numbers.
I think it's always better to be able to see what's going on, but I see your point too about driving people away as votes on here are handled very differently than Youtube.