Too Much fucking fees With Cryptocurrency

Hi I'm rich, I started participating in cryptocurrency last week. I'm fascinated by the whole idea of cryptocurrency. Decentralized
money system, No middle man, No fees and yet the biggest problem i had dealt with are those two thing. Exodus charge me a fee of $1.38 to sent $6 .50 worth of btc to my blockchain wallet account. Then i was Charge $0.36 to sent $1.00 worth of Bitcoin to my exchange account Poloniex. I decided to buy 3 XEM Coin and i wanted to withdraw my XEM to my wallet and the fee is 15 XEM worth about $4.26 USD. Why are the fees so high, I understand that I'm not a Big investor but it isn't right for such system to exist. Thank you if you Read This.