Managing such a huge number of chips to develop Llama 4 poses unique engineering challenges and can require significant amounts of energy. Meta executives on Wednesday sidestepped an analyst question about energy access constraints in some parts of the U.S. that are hampering companies’ efforts to develop more powerful AI.
By some estimates, a cluster of 100,000 H100 chips requires 150 megawatts of power. By comparison, America’s largest national laboratory supercomputer, El Capitan, requires 30 megawatts of power. Meta expects to spend up to $40 billion in capital on data centers and other infrastructure this year, an increase of more than 42 percent from 2023. The company expects its spending to rise even more sharply next year.
Meta’s total operating expenses have increased by about 9% this year. But overall sales (mostly from advertising) have soared more than 22%, and the company’s margins have widened further, generating even bigger profits despite pouring billions of dollars into its Llama efforts. I was able to get it.
Meanwhile, OpenAI, considered the current leader in cutting-edge AI development, is running out of money despite charging developers for access to its models. The company, which remains a nonprofit venture for now, said it is training GPT-5, a successor to the model that currently powers ChatGPT. OpenAI has said that GPT-5 will be larger than previous generations, but has not said anything about the computer cluster it is using for training. OpenAI also said that in addition to scale, GPT-5 will also incorporate other innovations, such as recently developed inference approaches.
CEO Sam Altman said GPT-5 will be a “significant advance” compared to previous generations. Last week, Altman responded to reports that OpenAI’s next frontier model would be released by December, writing about X: “Fake news is out of control.”
Google CEO Sundar Pichai said Tuesday that the latest version of the company’s Gemini family of generative AI models is in development.
Meta’s open approach to AI has sometimes been controversial. Some AI experts say having significantly more powerful AI models at their disposal could help criminals launch cyberattacks or automate the design of chemical and biological weapons. I’m concerned that it could be dangerous. Llama has been tweaked before release to limit cheating, but removing these restrictions is relatively easy.
Despite Google and OpenAI pushing proprietary systems, Zuckerberg remains bullish on his open source strategy. “Open source is clearly the most cost-effective, customizable, reliable, high-performance, and easy-to-use option available to developers,” he said Wednesday. “And we are proud that Llama is leading the way in this field.”
Zuckerberg added that Llama 4’s new features should enable broader functionality enhancements across metaservices. Currently, the signature product based on the Llama model is a ChatGPT-like chatbot known as Meta AI that is available on Facebook, Instagram, WhatsApp, and other apps.
According to Zuckerberg, more than 500 million people use MetaAI every month. Over time, Meta expects to generate revenue through advertising within this feature. “The range of queries that people use it for is going to grow, and the monetization opportunities will exist over time as we get there,” Meta’s chief financial officer Susan Lee said in a Wednesday call. spoke at the meeting. With the potential for revenue from advertising, Meta may be able to help others subsidize Llama.