The downvotes that come after Xeldal's vote are meant to prevent him from gaining curation rewards but I'm not fully sure if they're bots (people run bots on accounts to negate some votes seen as exploitative) that calculate the downvote value based on the upvote value or if they're a downvote trail. You're not on any blacklist or anything like that and the voting patterns aren't anything you did or could have done. I'm not sure how Xeldal curates but I suspect it's either tag or pattern based. I can't guess how his bot works as I don't know him or how he builds things. Probably the easiest way to get out of these bots is to figure out what makes Xeldal upvote and simply avoid doing that. It's likely either a trail trigger or a tag.