This is something that flew under the radar. xAI now has 100K GPUs operating as a super cluster. This eclipses the 33K that was thought to be the limit for coherence.
In this video I go through what this is and what it means. We know xAI is getting another 100K in GPUs from Nvidia in the first quarter. Will they be able to scale it to 200K?
▶️ 3Speak