I think it is, but I don't go around downvoting those who disagree.

For any account large enough to do damage with a downvote, choosing not to upvote a self-voted post is a solid loss for an author (maybe even a comment saying "hey, I was going to vote you, but you self-voted" if they want to direct behavior), there is no need to negate the upvotes/curation of the other accounts that voted.

Plagiarism, advocating of violence, child porn, shit post spamming...these are things that need a downvote.