You are right about that... Personally, I do check people whom I follow and occasionally unfollow people who aren't active anymore... For those who followed me, it's on them to recheck...