This fragmentation will lead to burnout. The only thing you have to do when the next great idea comes is writing it down. That way you can finish whatever you were doing without losing track. Every context switch drains mental energy you could spend on productive tasks.

So you need a task management system. It could be an easy notepad or a complex ERP based on your needs, but you have to use one. Pick one and use it every day to form the habit of checking. If you have a system and the habit of using it even when it's empty you can relax and just do the things and trust that everything is handled without this frustration.