The idea is that if you are doing this then you would presumably lose witness votes and not be able to make many more blocks.

The other witnesses would flag this block as a bad-block and reverse it I assume,
which again leads to a scenario of losing witness votes.

But again if this was the case that means the witness is breaking consensus.
So I'm still curious as to how this actually works.

Considering what happened when we booted up RCs I have to assume you're correct.
We all had negative RCs which would have been impossible with hard-enforced consensus.