Curiosity about Meta's next big step has reached fever pitch in the race to dominate artificial intelligence. With Llama 2's generative text model—released in July—well established in the market, AI watchers are hungrily searching for signs of Llama 3.
If industry grumbling is to be believed, the tech titan's next open source success could arrive as early as 2024.
Meta hasn't officially confirmed the rumor, but Mark Zuckerberg recently shed some light on what the future of Meta's LLMs (large-scale language models) might be, starting with admitting that Llama 3 is in the works. However, he said the new base AI model is still on the back burner and the priority is fine-tuning Lama 2 to make it more user-friendly.
“We've trained Llama 2, and we've released it as an open source model, and now the priority is to build it into a set of consumer products…
“But yes, we are also working on future base models, and I have no new or news on this,” he continued. “I don't know exactly when it will be ready.”
While Meta has not officially confirmed the rumors, patterns in development cycles and significant hardware investments hint at an imminent launch. Llama 1 and Llama 2 have seen six-month gaps in training, and if this intensity continues, the new Llama 3 – which is expected to be equivalent to OpenAI's GPT-4 – could be launched in the first half of 2024.
Adding to the depth of the speculation, Reddit user Lamashil provided a comprehensive analysis of meta-historical model development cycles.
Llama 1 training lasted from July 2022 to January 2023, and a follow-on Llama 2 until July 2023, the user suggested, setting the stage for Llama 3 training from July 2023 to January 2024. Relentlessly pursuing AI excellence to showcase its next evolution that can stand shoulder to shoulder with GPT-4's capabilities.
Meanwhile, tech forums and social media are abuzz with discussions about how this new iteration will rejuvenate the competitive edge of the meta. The tech community has also pieced together a possible timeline from scraps of information.
Heard on Meta GenAI Social:
“We have the compute to train Llama 3 and 4. The plan is for Llama-3 to be as good as GPT-4.”
“Wow, if Llama-3 is as good as GPT-4, are you guys still open source?”
“Yes we will. Sorry guys line up.”
— jason (@agikoala) August 25, 2023
Add to that a bit of a Twitter cult: a discussion on “Meta GenAI” social was reported, later tweeted by OpenAI researcher Jason Wei. “We have the calculations to train Llama 3 and 4,” an unidentified source explained to Wei – who went on to confirm that it would be open source.
Meanwhile, the company's partnership with Dell offers enterprise users the Lama 2 on-premises—underscoring its commitment to managing and protecting personal data, a move that's both strategic and contemporary. As Meta prepares to take on giants like OpenAI and Google, this commitment is critical.
Meta is also introducing AI into many of its products, so it makes sense for the company to increase its stake in order not to be left behind. Llama 2 Meta AI and other services like Meta Chatbots, Meta Generator Services and Meta AI Glasses to name a few.
In the midst of this storm of speculation, Mark Zuckerberg's proposal to open source Lima 3 only drew attention and speculation. “We need a process to make this team red and secure,” Zuckerberg said during a recent podcast with computer scientist Lex Friedman.
Llama 2 is a multi-level architecture offering 7 billion, 13 billion and a solid 70 billion parameters, each suited for different levels of complexity and computing power. Parameters in LLM serve as neural constructs that describe the model's ability to understand and generate language, with the number of parameters often related to the model's complexity and potential output quality.
The AI powerhouse is trained on a vast corpus of 2 trillion tokens, enhancing its ability to explore and generate human-like text across a variety of topics and contexts.
Image courtesy of Meta
In the background, the hardware base is also being patched. As reported by Decrypt, Meta is stocking up on Nvidia H100s data center, one of the most powerful pieces of hardware for AI training – a clear sign that the wheels are in good motion.
However, for all the excitement and speculation, the truth is shrouded in corporate secrecy.
Meta's desire to compete in the AI space is largely shaped by the required training time, hardware investments, and demand for open source. The release of Lama 3 in 2024 is as eagerly awaited as it gets.
Edited by Ryan Ozawa.
Stay on top of crypto news, get daily updates in your inbox.
