This is the right attitude. Downvoters are going to downvote. You don't have to like it, but you need to get used to it. If you aren't explaining that downvotes are a thing to people you onboard, then the failure is yours.