Zuckerberg says Meta will need 10 times more computing power to train Llama 4 than Llama 3

Meta, which develops one among the biggest open-source language models called Llama, believes it will need significantly more computing power to train models in the long run.

Mark Zuckerberg said Tuesday during Meta’s second-quarter earnings conference call that the corporate will need 10 times more computing power to train Llama 4 than it did to train Llama 3. But he still wants Meta to expand its model training capabilities slightly than fall behind the competition.

(*3*) Zuckerberg said.

“It’s hard to predict how this will play out over generations. But at this point, I’d rather risk building capacity before it’s needed than too late, given the long lead times for new inference projects.”

Meta released Llama 3 with 80 billion parameters in April. The company released an improved version of the model last week, called Llama 3.1 405B, which had 405 billion parameters, making it Meta’s largest open-source model.

Meta CFO Susan Li also said the corporate is taking a look at various data center projects and constructing capability to train future AI models. She said Meta expects this investment to drive up capital spending in 2025.

Training large language models could be an expensive business. Meta’s capital expenditures rose nearly 33% to $8.5 billion in Q2 2024 from $6.4 billion a 12 months earlier, driven by investments in servers, data centers and network infrastructure.

According to the report from (*4*)informationOpenAI is spending $3 billion on training models and an extra $4 billion on discounted server rentals from Microsoft.

“As we scale generative AI training capabilities to evolve our underlying models, we will continue to build our infrastructure in a way that gives us flexibility in how we use it over time. This will allow us to redirect training capabilities to generative AI inference or to our core work of ranking and recommendation when we expect that to be more valuable,” Li said on the decision.

During the decision, Meta also discussed consumer usage of Meta AI, noting that India is the biggest marketplace for its chatbot. Li noted, nevertheless, that the corporate doesn’t expect Gen AI products to contribute significantly to revenue.