Apart from performances before the records, that logic is very flawed though. To make that logic work, you literally need people to run/jump/swim/do their sport everyday to be considered the best everyday, which is stupid. What would happen is if there is an ongoing competition in the world for a given day, they will most likely be the 'best' in the world for that day since other athletes aren't competing, and probably aren't pushing themselves on that day. That is why we have world records. Other competitions still record the times of participants, so if they do beat a world record, they can submit those times there. Maybe in the future when we have accurate and infallible personal timers, only then can we truly take into consideration the best in any given day. But until then, we can just compare the best a person can do to the best ever recorded.

Thinking about people who can do things better back then is pointless, since we don't have any record. I would argue that our current records are faster than whatever their best was back then. The reason for that is specifically because of the Olympics. We have athletes whose work and life is in doing one specific sport. Usain Bolt eats, breathes, and lives running. The people back then didn't have that luxury. Their best bet against Usain would be runner messengers, but they are for long distances and would lose to Usain. I don't see anyone beating Phelps in swimming. For weights, they were going for function instead of max weight. I can go on for a lot of sports.