It seems the race for top large language model (LLM) is getting tight.
While OpenAi jumped out to an early lead with ChatGPT, it is now less clear who is the top dog.
A couple weeks ago, Claude3 by Anthropic hit the market and it stunned. Prior to that, bragging rights when to OpenAI with ChatGPT 4.0. Many claimed that Claude3 was far superior based upon lining up the metrics.
While these two garnered attention, lurking in the weeds are Gemini (Google) and Grok (Elon Musk). Not getting much attention was Zuckerberg with Meta's Llama3.
That was until today.
Whether you prefer ChatGPT or Claude3, both are proprietary models. These are run by companies that have keep their platforms closed.
Llama3 is different. This is open source meaning anyone is free to download and adapt it. It is following the same concept as Linux, which differed from the closed UNIX framework.
Many question whether open source can rival its closed counterparts. On the other side, many feel it is only a matter of time before the open models catch, and surpass, everything else out there.
Open operates at a disadvantage to start. There tends to be fewer resources along with a late start to contend with. However, due to the fact so many can innovate on the software, we see how this can alter things completely. The different developers and teams add to the ecosystem, allowing it to close the gap rather quickly.
This might be what will occur with the Llama platform.
@taskmaster4450le/re-leothreads-kvhlxvdc
As the video above details, in a comparison of the specs, it appears that Llama3 is in the lead. In addition, it appears that it is positioning itself to be the open source standard, much in the same way Android took over smartphones.
It is interesting to watch Zuckerberg position himself as the "good guy". He is taking the opposite stance of Sam Altman, the new Mega-Tech demon. Altman is petitioning the US government to make all AI closed, banning open source.
He, along with others, asserts that this is to protect us from nefarious actors getting a hold of the software. Of course, the presumption is that he can be trusted.
This is going to be a very interesting fight, one that has huge implications.
In this race, nothing sits still.
OpenAI still has ChatGPT 5.0 in its pocket. A lot is being discussed about it although it is not released. Some are starting that it is AGI. It is also believed this was at the core of Altman's firing from ChatGPT, albeit a move that was quickly reversed.
A late arrival to the race is Elon Musk. At this moment, few are giving him much weight in this race. Grok is still lagging, with Grok 2.0 in testing. Even when it is released, likely next month, it is still behind what the others are doing.
There is a caveat worthy of mentioning. Grok is literally late to the game. xAI is roughly 1 year old, with its first model only hitting the market last fall.
So why is there a chance that Grok jumps into the lead?
The answer of this can be summed up by Zuckerberg and Musk unzipping their flies and pulling it out. This is nothing more than a measuring contest. In other words, who's is bigger?
So far, Zuckerberg is the biggest one out there.
As our largest model yet, training Llama 3.1 405B on over 15 trillion tokens was a major challenge. To enable training runs at this scale and achieve the results we have in a reasonable amount of time, we significantly optimized our full training stack and pushed our model training to over 16 thousand H100 GPUs, making the 405B the first Llama model trained at this scale.
It was no secret that Meta was a huge buyer of NVIDIA GPUs. We see how this model was trained on over 16K of them, the most of anything at this scale.
In a race where bigger is better, this appears to be impressive.
The only challenge is we saw an announcement by Musk and xAI that really puts this in perspective.
That certainly changes things.
We are in a race where compute plus data is the foundation. xAI has the processing power and, with X, more than 15 years worth of data.
During this announcement, Musk said Grok 3.0 would be ready by the end of the year.
In the LLM race, that is a lifetime. We can bet the ranch the other companies are not sitting back doing nothing. There will be progress from each one, with a few other releases that capture the attention of the community.
For the time being, Zuckerberg and Meta are the top dog.
We will see how long it lasts.
