Let's say I put everything on a ledger, there is a risk of losing it all in case somehow it gets destroyed, stolen, doesn't work anymore, if ledger gets hacked, or whatever.

Centralized exchanges are a way for me to diversify that risk so there is no central point of failure. I don't like the idea of having a lot of your wealth in your house on a stick and a piece of paper. It's an issue that crypto still needs to solve somehow as I think the majority of people would feel safer if it was in custody of a 3rd party somehow.