As I explained before, I don't give the downvotes.
I have the power to stop them though, and if you show that you seriously change your ways I will do so in a heartbeat and happily ask the group to switch their focus to other self-voters. I can't tell you how exactly, that'd be up to you to figure out. Not giving a meager 3-4% to strangers while your friends and family receive 50-100% would definitely be a requirement, the exact details are your decision.

Support the community, and it will support you.