If the firmware isn't compromised, you can still lose all your money if you lose the device if someone happens to have the skill and $100 of electronic equipment to read the internal board.

Is it really worth someone's time to steal my Trezor when they have no idea how much money is on it? Also, they have to hack it before I realize it's missing. Tough sell. It would be a lot easier to get access to the seed phrase backup. No hacking required.