Not to assume you haven't tried the following, here is what I'd try:

Point 3 should be solved by enabling IPNI. This is a database index and not a node traversal that happens before falling back to DHT.

Point 2: Can you move to NVMe drives?

Point 1: Break up the massive node into smaller ones, running a process for each. I gather that the Badger/RocksDB backend is what is recommended. Just have to try and see if that can work all on your server of if this is the start of necessitating more people hosting the nodes.

I see now how scaling is a serious issue. Any other aspects of it that are not solvable?