Yes, good point, a quantum computer, if it existed, could indeed be viewed as a very powerful mining node, Bitcoin would then naturally adjust its difficulty but if the quantum computer was outperforming all other miners it could indeed become a security risk, if there were no other miners with quantum computers.