There will always be someone that follows for a few days or weeks the unfollows and/or mutes.
I used to let it bother me but it is just the way people are and how the blockchain works.
Never let it bother me anymore and I hope people get the idea from reading this post that it is really not a big deal if they do or don't, just be you and do what you do!