I learn this important information in the Banless podcast with the interview of the CTO of Ledger. If you don't use the recovery service, nothing will happen and you can still use your device as usual. The risk of the backdoor is already big I believe.