would u have eventually downvoted someone constantly talking shit about you with false information?

I absolutely would.

I never got downvoted for talking shit here, because I assume people realize in my own demented way that there are at least some kernel of logic and truth to my shit talking.

Or maybe it is because I don't go out of my way to constantly mention and harass people.

It is one of those.