Meta’s Chief AI scientist Yann LeCun (Julien De Rosa/AFP via Getty Images)

GP-who?

Just four companies are hoarding tens of billions of dollars worth of Nvidia GPU chips

Each Nvidia H100 can cost up to $40,000, and one big tech company has 350,000 of them.

7/25/24 1:26PM

Meta just announced the release of Llama 3.1, the latest iteration of their open source large language model. The long-awaited, jumbo-sized model has high scores on the same benchmarks that everyone else uses, and the company said it beats OpenAi’s ChatGPT 4o on some tests.

According to the research paper that accompanies the model release, the 405b parameter version of the model (the largest flavor) was trained using up to 16,000 of Nvidia’s popular H100 GPUs . The Nvidia H100 is one of the most expensive, and most coveted pieces of technology powering the current AI boom. Meta appears to have one of the largest hoards of the powerful GPUs.

Of course, the list of companies seeking such powerful chips for AI training is long, and likely includes most large technology companies today, but only a few companies have publicly crowed about how many H100s they have.

The H100 is estimated to cost between $20,000 and $40,000 meaning that Meta used up to $640 million worth of hardware to train the model. And that’s just a small slice of the Nvidia hardware Meta has been stockpiling. Earlier this year, Meta said that it was aiming to have a stash of 350,000 H100s in its AI training infrastructure – which adds up to over $10 billion worth of the specialized Nvidia chips.

Venture capital firm Andreesen Horowitz is reportedly hoarding more than 20,000 of the pricey GPUs, which it is renting out to AI startups in exchange for equity, according to The Information.

Tesla has also been collecting H100s. Musk said on an earnings call in April that Tesla wants to have between 35,000 and 85,000 H100s by the end of the year.

But Musk also needs H100s for X and his AI company xAI. This week, Musk boasted on X that xAI’s company’s training cluster is made up of 100,000 H100s.

A tweet from Elon Musk stating that xAI has 100,000 H100 GPUs. — Source: X @elonmusk https://x.com/elonmusk/status/1815325410667749760

Musk was recently sued by Tesla shareholders for allegedly re-directing 12,000 of the H100s intended for the car maker’s AI training infrastructure to xAI instead. When asked about this diversion in yesterday’s Tesla Q2 earnings call, Musk said that the GPUs were sent to xAI because “the Tesla data centers were full. There was no place to actually put them.”

The H100s are in such demand that people are being paid to sneak them into China, to bypass U.S. export controls. You can watch unboxing videos of these graphics cards, and there are even a few for sale on Amazon – including one for $34,749.95 (with free delivery).

OpenAI hasn’t said how many H100s they are sitting on, but The Information reports that the company rents a cluster of processors dedicated to training from Microsoft at a steep discount as part of Microsoft’s $10 billion investment in OpenAI. The training cluster reportedly has the power of 120,000 of Nvidia’s previous gen A100 GPUs, and will be spending $5 billion to rent more training clusters from Oracle over the next two years, according to The Information’s report. OpenAI does appear to have a special relationship with Nvidia — in April, Nvidia CEO Jensen Huang “hand-delivered” the first cluster of the company’s next generation H200 GPUs to co-founders Sam Altman and Greg Brockman.

A tweet by OpenAI’s Greg Brockman with a photo featuring Brockman, OpenAI CEO Sam Altman and Nvidia CEO Jensen Huang — Source: X @gbd https://x.com/gdb/status/1783234941842518414

Nvidia declined to comment for this story, and Meta, X, OpenAI, Tesla, and Andreessen Horowitz did not respond to requests for comment.

More Tech

See all Tech

$10B

Rani Molla

13h

Apple made $10.1 billion on App Store commissions in the US last year, according to data from Appfigures reported by TechCrunch. That figure has more than doubled since 2020. Last week, a federal judge has banned the 27% commission Apple charged developers for purchases made outside the App Store, meaning Apple could lose a big chunk of that revenue.

In the decision, the judge wrote that Apple, when considering what a no-commission option would be like, “anticipated that most large developers and potentially many medium and small developers would offer link-out purchases to their users,” a move that would have a revenue hit of “hundreds of millions to billions.” The exact figures are redacted.

Jon Keegan

5/8/25

Analysis: Big Tech is building thirsty AI data centers in highly “water stressed” areas

Big Tech is currently on a building spree, setting up massive power-hungry data centers all over the world to power the current AI frenzy.

The huge amounts of energy required to run these data centers has led companies like Microsoft, Elon Musk’s xAI, Meta, and Google to come up with some creative solutions for supplying that energy, including firing up long-closed nuclear power plants, hauling in dozens of portable (unregulated) gas turbines, and reopening gas- and-coal-fired power plants.

Many new data centers are building substantial renewable energy projects to stay green.

But it’s much harder to scale up another crucial resource needed for these data centers: water. A new analysis from Bloomberg looked at the places where large data centers are being built around the world, and found a troubling trend:

“Bloomberg News found that about two-thirds of new data centers built or in development since 2022 are in places already gripped by high levels of water stress. While these facilities are popping up all over the country, five states alone account for 72% of the new centers in high-stress areas.”

The analysis also found that the problem was especially severe in China, where more of the population lives near high water-stressed areas.

The AI Boom Is Draining Water From the Areas That Need It Most

Many new data centers are building substantial renewable energy projects to stay green.

“Bloomberg News found that about two-thirds of new data centers built or in development since 2022 are in places already gripped by high levels of water stress. While these facilities are popping up all over the country, five states alone account for 72% of the new centers in high-stress areas.”

The analysis also found that the problem was especially severe in China, where more of the population lives near high water-stressed areas.

Rani Molla

Blue likes green

5/8/25

Economists confirm what everyone already knew: Democrats buy electric vehicles

Tesla CEO Elon Musk’s political move to the right is likely alienating his core customer base.

Jon Keegan

5/8/25

OpenAI poaches Instacart CEO Fidji Simo to lead apps

OpenAI is growing fast. It’s building a $500 billion “Stargate” AI data center campus, and ChatGPT is breaking traffic records, even though it is too sycophantic.

It sounds like CEO Sam Altman is spread a little thin. He’s overseeing AI research, new infrastructure tools, and some very popular consumer apps.

To help shore up the app part of the business, OpenAI is poaching Instacart’s CEO, Fidji Simo, to be its “CEO of Applications.” Simo, who spent years as a high-profile exec at Facebook before moving to Instacart, has been an OpenAI board member for more than a year.

In a note to employees, Altman wrote:

“To strengthen our execution, I’m excited to announce Fidji Simo is joining as our CEO of Applications, reporting directly to me. I remain the CEO of OpenAI and will continue to directly oversee success across all pillars of OpenAI — Research, Compute, and Applications — ensuring we stay aligned and integrated across all areas. I will work closely with our board on making sure our non-profit has maximum positive impact.”

Simo will be transitioning to the new role over the next few months. She wrote:

“Joining OpenAI at this critical moment is an incredible privilege and responsibility. This organization has the potential of accelerating human potential at a pace never seen before and I am deeply committed to shaping these applications toward the public good.”

Instacart shares were down about 3% in early trading.

OpenAI Expands Leadership with Fidji Simo

In a note to employees, Altman wrote:

“To strengthen our execution, I’m excited to announce Fidji Simo is joining as our CEO of Applications, reporting directly to me. I remain the CEO of OpenAI and will continue to directly oversee success across all pillars of OpenAI — Research, Compute, and Applications — ensuring we stay aligned and integrated across all areas. I will work closely with our board on making sure our non-profit has maximum positive impact.”

Simo will be transitioning to the new role over the next few months. She wrote:

“Joining OpenAI at this critical moment is an incredible privilege and responsibility. This organization has the potential of accelerating human potential at a pace never seen before and I am deeply committed to shaping these applications toward the public good.”

Instacart shares were down about 3% in early trading.

Rani Molla

5/8/25

Apple defends its $20 billion Google deal by downplaying its importance

Google searches on iPhones are declining for the first time ever as people — and Apple — move to AI, so there’s no need to worry about our fusty old Google Search contract. That’s roughly the argument Apple Senior Vice President of Services Eddy Cue gave yesterday during a Google antitrust hearing, a clever defense of the company’s $20 billion deal with Google to make it the default browser on iPhones.

But as Bloomberg’s Mark Gurman points out, Apple has about 20 billion reasons to downplay the deal, which represents about 5% of its revenue and has been helping drive its lucrative services revenue.

“Cue’s disclosure of Apple’s plan to shift its browser to AI systems had a clear goal: downplaying the importance of the existing Google deal. If the industry has changed and there are now clear alternatives to Google, the judge may decide there’s no reason to upend the long-running agreement. Cue spent time praising these rival options, including Perplexity’s service in particular.
‘It is logical that Apple might highlight data points supporting the narrative that Google is not anticompetitive in search,’ Jefferies LLC analyst Brent Thill said in a note.
But Apple and Google have the opportunity to deepen their partnership around artificial intelligence — something Cue didn’t focus on. Google has already pivoted to its own Gemini AI system for search. When users make a query via Google, they are often first presented with an AI result. That’s true even on iPhones, iPads and Macs today.”

Apple, of course, has been discussing adding Google’s Gemini AI to iPhones this year.

Apple Tries to Save $20 Billion Google Search Deal By Saying It’s Unnecessary

“Cue’s disclosure of Apple’s plan to shift its browser to AI systems had a clear goal: downplaying the importance of the existing Google deal. If the industry has changed and there are now clear alternatives to Google, the judge may decide there’s no reason to upend the long-running agreement. Cue spent time praising these rival options, including Perplexity’s service in particular.
‘It is logical that Apple might highlight data points supporting the narrative that Google is not anticompetitive in search,’ Jefferies LLC analyst Brent Thill said in a note.
But Apple and Google have the opportunity to deepen their partnership around artificial intelligence — something Cue didn’t focus on. Google has already pivoted to its own Gemini AI system for search. When users make a query via Google, they are often first presented with an AI result. That’s true even on iPhones, iPads and Macs today.”

Apple, of course, has been discussing adding Google’s Gemini AI to iPhones this year.

Latest Stories

Sherwood Media, LLC produces fresh and unique perspectives on topical financial news and is a fully owned subsidiary of Robinhood Markets, Inc., and any views expressed here do not necessarily reflect the views of any other Robinhood affiliate, including Robinhood Markets, Inc., Robinhood Financial LLC, Robinhood Securities, LLC, Robinhood Crypto, LLC, or Robinhood Money, LLC.