At a high-profile AI event in London, Meta executives on Tuesday provided the first official confirmation and details about the release of Llama 3, the company's highly anticipated next open-source large-scale language model.
“Within the next month, actually less, hopefully in the short term, we hope to start rolling out the new next-generation Foundation models, the Lama 3,” announced Meta's president of global affairs, Nick Clegg. AI Day London, TechCrunch reports.
According to Clegg, the Lama 3 will go on sale later this year and will consist of “a number of different models with different capabilities, different versatility”.
Once launched, Lama 3 is expected to be the most advanced open source model, with Meta investing heavily in its development. The model was trained on 140 billion parameters, Meta says, more than twice the capacity of Llama. Meta CEO Mark Zuckerberg teased some technical details back in January.
“We're building massive compute infrastructure to support our future roadmap, including 350k H100s by the end of this year — and a total of 600k H100s of equivalent compute if you include other GPUs,” Zuckerberg said at the time. This amount of computing power is significantly greater than that used by OpenAI to train GPT-4, which is estimated to require around 25,000 GPUs in 90 to 100 days.
Zuckerberg also revealed that his AI assistant, Meta AI, is set to run on Lama 3.
Chris Cox, chief product officer, will integrate Llama 3 into the meta.
“Our plan is to have Llama 3 power many different products and experiences across our family of applications,” he said.
Open source strategy
The impact of Llama 3's release extends beyond Meta and demonstrates the company's philosophical commitment to developing as an open source model, as opposed to the apparently closed and proprietary approach of rivals like OpenAI with ChatGPT.
Meta aims to nurture an ecosystem of open AI development by opening up their language models and making the Llama family the basis for a variety of tools and applications created by third-party developers and researchers.
Yann LeCun, Meta's head of AI research, tweeted last month that “it's important to recognize that innovation always builds on the previous contributions of others.” This is why open research is so important: it makes the field faster for everyone.
This apparent ethos has already created a vibrant community rally around Lama. Some of the most advanced open source language models today, such as Mistral, Falcon, and Beluga, are built by fine-tuning the earlier Llama 2 base model. Several of these community models matched or outperformed GPT-3.5 on certain criteria.
The release of Llama-3 as another open source based model paves the way for a new generation of LLMs that will further raise the bar for quality and efficiency in AI.
Challenge the dominance of OpenAI
Lama 3's open-source premise poses a serious and multi-layered challenge to OpenAI's current market dominance and – by extension – to other proprietary models such as Cloud and Gemini.
The open source community will soon build on Llama 3 and iterate on their variants to quickly match or exceed the capabilities of GPT-4—just as they did with GPT-3.5. With low training cost shared among contributors, the open ecosystem can bypass OpenAI's proprietary model development that requires enormous computing resources and costs.
If open source offerings routinely find common ground with commercial offerings, enterprises may gravitate toward more accessible and cost-effective ecosystems like Llama instead of relying on and paying for OpenAI. Currently, GPT-4 is the most expensive model on the market in terms of price per token.
Additionally, the open source community grows stronger as more people participate in it. Meta benefits from having a huge community building on the model, fine-tuning it, developing new technologies, and improving it for free. This makes it easier for Meta to develop better versions of the model while also monetizing it with alternate programs by licensing it to commercial use for large industries.
In other words, continued inertia and network effects make it difficult for OpenAI's proprietary models to attract users and customers in the future.
Certainly, OpenAI currently holds a strong leadership in terms of profitability. Anthroponic can boast of having the best performing LLM in the AI space. But Llama 3 represents another strategic strike in the meta to grow the generative AI landscape.
Of course, much depends on Llama 3's real-world performance and adoption in the coming year. But the open source AI community is very active – and already loves Llama-2. Things are going to get really interesting over the next few months, especially with OpenAI's GPT-5 right around the corner.
Edited by Ryan Ozawa.
Stay on top of crypto news, get daily updates in your inbox.
