Much has been written about xAI’s Colossus 1. The Memphis build belongs in the history books: the largest AI training cluster, erected from scratch in 122 days. With roughly 200,000 H100/H200s and ~30,000 GB200 NVL72, it remains, today, the largest fully operational, single-coherent cluster (setting apart Google, master of multi-datacenter-training).

However, Colossus 1’s ~300 MW looks modest next to the Gigawatt-scale clusters under construction by OpenAI, Meta and Anthropic. Their hyperscaler partners are happy to leverage their balance sheet and win the market by throwing dollars at it.

Was xAI’s prowess a one-time wonder? Today we will publicize some data from our industry leading datacenter model over the last year that is accessible to clients. This is the our same proprietary data that called the Oracle deals many months ahead of the announcement.

Short answer: no. xAI is still squarely in the frontier-AI race and is positioned to leapfrog most rivals again on compute. By our estimates, its total datacenter capacity for a single training cluster will surpass Meta Superintelligence and Anthropic by Q3 2025. The datacenter capacity will be ready for the GPUs to be moved in to create the largest single datacenter in the world, yet again. xAI has to raise the capital for those GPUs, but they have the allocations from Nvidia to have it fully training large scale models early next year.

Elon came up with a new genius trick to beat rivals at time-to-market. Colossus 2 will be an even more impressive achievement than xAI’s first cluster. Let’s dig in.

The first half of this report will dig into the Colossus 2 prowess. The second half will discuss Grok models, our mid-to-long term thoughts on xAI, and the unique RL method xAI is using that may lead them to leapfrog OpenAI, Anthropic, and Google.  

---

SemiAnalysis Is Hiring

We are seeking a highly motivated & skilled Member of Technical Staff to join our growing special projects engineering team. You will play a crucial role in developing industry leading gpu cloud benchmarks & evaluation framework.  Our gpu cloud evaluation frameworks is endorsed by many tier 1 & 2 frontier labs. You may be a good fit if you have the following experience:

• Demonstrated experience in ML frameworks such as PyTorch or JAX through professional experience, personal projects, or personal Substack blogs
• 1-2 years using GPU or TPU clusters and/or running a multi-tenant GPU cluster
• Past experience working at a hyperscaler or a GPU cloud (preferred)
• Solid understanding of SLURM, Kubernetes, NCCL & GPU Cloud industry
• Strong research skills and the ability to synthesize information from various sources to draw insights

Compensation is competitive & as part of the interview process, you’ll complete a paid coding challenge designed to reflect typical daily tasks at SemiAnalysis

---

Colossus 2: from zero to 200MW in six months

The Colossus 2 project was kicked off on March 7^th, 2025, when xAI acquired a 1m sqft warehouse in Memphis, and two adjacent sites totaling 100 acres. By August 22^nd, 2025, we count 119 air-cooled chillers on site, i.e. roughly 200MW of cooling capacity. That’s enough to power roughly 110k GB200 NVL72. And an Elon tweet shows some racks were already installed in July.

xAI built in six months what took 15 months for Oracle, Crusoe and OpenAI!

Looking closer at the picture above, readers familiar with our Datacenter Anatomy series might wonder where the power infrastructure is. The picture doesn’t show any onsite substation, nor does it show onsite turbines…  How is the datacenter powered?

Colossus 2: in Tennessee, Mississippi… or both?

Things get more confusing once factoring-in a Greater Memphis Chamber statement in May that no turbines would be sited in Memphis. They’re not lying.

Memphis and Tennessee have been getting a lot of pushback, so xAI’s genius move was to develop a Gigawatt-scale energy hub right across the border in Southaven, Mississippi. In mid-2025, the company acquired a former Duke Energy power plant in Southaven. Shortly after, Mississippi regulators granted xAI temporary approval to run gas turbines there for up to 12 months without a permit!

To transport and manage power generated by the Mississippi power plant, xAI is building infrastructure near Colossus 2. We show below the first deployment of Tesla Megapacks, as well as the Medium Voltage power lines connecting the two sites.

From 200MW to 1.1GW and Solaris Energy Infrastructure partnership

In Southaven, MS, xAI is moving at the pace of light. The discontinued power plant now sees seven 35MW turbines in operations.

To deploy faster than peers, xAI relies on rental turbine companies. NYSE-listed Solaris Energy Infrastructure owns a fleet of 600MW of gas turbines, of which ~400MW currently serve xAI. Musk’s firm weighs 67% of SEI’s 1700MW orderbook, i.e. 1,140MW. There are ~240MW on the Memphis Colossus 1 site, while the remaining 900MW will be owned by a Joint Venture owned at 50.1% by Solaris and 49.9% by xAI.

As shown below, ~460MW are currently installed and in operations/under construction.

The newly formed JV already spent $112 million in capital expenditures in Q2 2025. After a slow Q3, spending will pick up again in Q4 2025 and Q1 2026. Solaris expects to have over 1.1GW of fully operating turbines for xAI by Q2 2027. There remains ~425MW available for contracting, and we think xAI will likely pull the trigger to get to over 1.5GW of total gross power. Solaris also appears to temporarily lease power generation capacity from third parties to deliver faster:

During the second quarter, the Power Solutions segment generated revenue from approximately 600 megawatts of capacity, an increase of greater than 50% from the prior quarter. This increase was driven by increased demand from our customers, which we are meeting using a combination of new equipment deliveries as well as selective short-term sourcing of third-party power generation capacity.

Solaris Energy Infrastructure, Q2 2025

As such, xAI has figured out how to scale to >1GW from a power perspective. In terms of datacenter space, we see four options:

• Given a 40ft height, xAI could turn their 1mm sqft warehouse into a two-story datacenter, doubling the space. Given ultra-high density, 2mm sqft could be enough for >1GW.
• xAI could build a second, smaller facility in parcel 3
• They could acquire more land, possibly in Mississippi near the Southaven plant.
• Using non-standard layout of the datacenter, they could achieve >1GW as is.

Can xAI afford Colossus 2?

Further expansion will require ample funding. Required CapEx for Colossus 2 will be in the tens of billions of dollars, and xAI is yet to generate any meaningful external revenue, with the preponderance of the rumored 9 digit revenue ARR being inter-company transfers from X.com to xAI. We’ve forecasted xAI’s CapEx on Core Research, our institutional research service and are now closely tracking the ROIC of AI investments across the hyperscalers and AI labs in our new Tokenomics model.

Middle East – funding + datacenters, a winning combination?

To be clear, in typical xAI & Elon fashion, the company’s future is highly unpredictable. But given funding needs, we see a large-scale expansion in the Middle East as likely. Musk and the Middle have a long-time relationship:

• KSA’s Kingdom Holding Company (owned at 16.87% by the Public Investment Fund) owned and kept a $1.9B stake in Twitter when Musk took the company private in 2022. It also owned a $800M stake in xAI prior to the merger with X.
• UAE’s privately owned Vy Capital brought in $700M in 2022 to support Elon’s takeover of Twitter. It also invested in xAI’s Series C, alongside UAE state fund MGX.
• Qatar’s QIA also owned and kept a $375M stake in Twitter, and participated on xAI’s Series C.

According to the FT, xAI is preparing a new round in the tens of billions, valuing the company close to $200B, with Saudi’s PIF sovereign wealth fun playing a large role. This is challenging though because its hard for most investors to justify xAI as having a valuation higher than Anthropic.

We’ve heard of a raise as large as $40B. Given skyrocketing datacenter growth in the area, we see a two-way deal as likely, with xAI deploying that capital in a new large scale datacenter in the Kingdom.

We show below a likely location for xAI’s expansion, a large-scale planned located Saudi Arabia that recently broke ground. While still early, there is plenty of land and power to serve large-scale AI campuses.

For more details on the Middle East’s AI expansion, check out our May 2025 deep dive.

AI Arrives In The Middle East: US Strikes A Deal with UAE and KSA

Beyond external capital sources, Elon could generate capital internally. Since merging X.com with xAI to form X Holdings, we believe an ever growing piece of xAI’s revenues is inter-company transfers, i.e. calls to @Grok to answer questions or just X.com licensing the LLM technology for functionality such as search, ad recsys, or even content creation.

This is just money going from Elon’s right pocket to his left pocket. From what we can track externally, Ani was a huge boon for Grok app revenue but even growth in that revenue stream has flattened out in recent months. Ani needs more gacha to grow revenue further.

At the end of the day, Elon can get Tesla to invest more or take loans on more of his Tesla and SpaceX stocks to invest tens of billions into xAI. This will allow them to build Colossus 2. No one truly knows how levered Elon is already, but it is widely understood he can always sell and unlock a lot more of his dry powder into xAI. Elon will do everything he can to not lose to Sam Altman.

Let’s now discuss xAI as a business and whether we think the company has a shot at being a Frontier AI Lab and justifying a multi-hundred-bullion dollar valuation. We have some unique info and insights below and why they may have a shot at being first to AGI due to their different approach.

Does xAI have a shot at becoming a frontier lab?

To many power users (Pro and Plus), GPT5 was a disappointing release. But with closer inspection, the real release is focused on the vast majority of ChatGPT’s users, which is the 700m+ free userbase that is growing rapidly. Power users should be disappointed; this release wasn’t for them. The real consumer opportunity for OpenAI lies with the largest user base, and getting the unmonetized users who currently use ChatGPT infrequently in their day-to-day to indirectly pay is their largest opportunity.

Analysts focused on the model capabilities are missing the much larger context of network effects that ChatGPT is gaining quickly. In November of 2023, ChatGPT wasn’t even in the top 100 websites; now it is number 5. Larger than X/Twitter, Reddit, Whatsapp, Wikipedia, and quickly approaching Instagram, Facebook, Youtube and Google. On the top 10 list, every single property is much older than ChatGPT.com, and the sheer number of unmonetized users is staggering.

But that changes with GPT-5. OpenAI is laying the groundwork to monetize one of the largest and fastest-growing web properties in the world, and it all begins with the router.

The Router is the Release

On OpenAI’s release website, the second paragraph is about the “One United System,” specifically focused on the router. The wording is instructive.

GPT‑5 is a unified system with a smart, efficient model that answers most questions, a deeper reasoning model (GPT‑5 thinking) for harder problems, and a real‑time router that quickly decides which to use based on conversation type, complexity, tool needs, and your explicit intent (for example, if you say “think hard about this” in the prompt). The router is continuously trained on real signals, including when users switch models, preference rates for responses, and measured correctness, improving over time. Once usage limits are reached, a mini version of each model handles remaining queries. In the near future, we plan to integrate these capabilities into a single model.

The router serves multiple purposes on both the cost and performance side. On the cost side, routing users to mini versions of each model allows OpenAI to service users with lower costs. On the performance side, it will enable many users to use thinking aka CoT (Chain of Thought) reasoning for the first time. Over 99% of the free users have yet to interact with a thinking model like o3, and for the average user, ChatGPT just got a huge upgrade. The number of free users exposed to thinking models went up 7x in the first day and the number of paying users up nearly 3.5x.

But the router is clearly a feature to the new service, and can likely see improvements or changes over time. It will continuously learn on preference rates and OpenAI promises it will improve over time. It just takes a single additional attribute to begin the path to monetization: the commercial value of the query.  

We believe that the Router is the groundwork for the next leg of ChatGPT’s story, and that’s monetization of free users.

The Router is Preparing for Free Monetization, Sam’s Subtle Tone Shift

Centralizing the control of the free user experience allows for many more future monetization paths. And this monetization path is one that has been hinted at subtly for a while. It all starts with OpenAI’s decision to hire Fidji Simo as CEO of Applications in May. Let’s look at her background because it’s telling.

Fidji was at Ebay from 2007 to 2011, but her defining career was primarily at Facebook. She was Vice President and Head of Facebook, and she is known for having a superpower to monetize. She was critical in rolling out videos that autoplay, improving the Facebook feed, and monetizing mobile and gaming. She might be one of the most qualified individuals alive to turn high-intent internet properties into ad products, and now she’s at the fastest-growing internet property of the last decade that is unmonetized. It’s an obvious story.

What’s more is Sam Altman himself has had a very direct tone shift in the last year.

“I will disclose as a personal bias I hate ads. I think ads were important to give the early internet a business model. I’m not totally against them, but ads plus AI are uniquely unsettling to me. I kind of think of ads as a last resort as a business model.”

But in recent interviews his tone has now shifted. There is clearly a lot of thought happening about how to best monetize free users more recently. This likely is in conjunction with Simo’s hiring.

“I am not totally against it… if you compare us to social media or web search where you can kinda tell that you are being monetized… we would hate to ever modify anything in the stream of an LLM… maybe if you click on something in there that is going to be there we’d show anyway, we’ll get a bit of transaction revenue and it’s a flat thing for everything, maybe that could work. It’s clearly possible to be a good ad driven company but there are obviously issues to it.

Compared to the previous conversation, when he was dismissive, his most recent thoughts clearly shows that Sam Altman is thinking about monetizing free users. He mentions a take rate a potential affiliate model. And as a response, the interviewer (Andrew Mayne of OpenAI) literally says, “I would love to do all my purchasing through ChatGPT because often times I feel like I am not making the most informed decisions,” as a response in the conversation. This is likely the direction OpenAI is taking.

The Router release can now understand the intent of the user’s queries, and importantly, can decide how to respond. It only takes one additional step to decide whether the query is economically monetizable or not. Today we will make our case for how ChatGPT’s monetized free end state could look like an Agentic super-app for the consumer. This is only possible because of routing.

We believe that display ads are unlikely. Perplexity has tried this, and it doesn’t seem to be going that well. Instead of inserting a paid feature into the query, we believe it’s more likely that they will pursue a take-rate based model.

Now let’s discuss, because it looks like an Agentic Assistant is a way that could align Sam Altman’s vision of AI being helpful, as well as monetizing via a transaction take rate.

Agentic Advice and Purchasing, Supply and Demand Dynamics

Let’s talk about Agentic purchasing and compare it to a search query today. Because LLMs have a core feature that Search does not, and that is scaling marginal costs. This is fundamentally different than the world search grew up in. Let’s examine “Aggregation Theory” work of Ben Thompson, because the core feature was that most technology companies had zero marginal costs to an additional user. There were some fixed overheads for running the largest search engine, but the incremental cost of another query was virtually zero. Agents and LLMs kill this concept.

For the first time, the more you spend the better your result is because of CoT reasoning tokens and now marginal costs exist in software again. There is a somewhat direct relationship between more money, more compute, and a better answer. Nowhere is this clearer than in AI, in which you can spend variable costs to get a variably better answer or outcome.

So let’s apply marginal costs to a new purchasing experience. Let’s compare two queries, an information query and a commercial query.

1. A trivial information query like “Why is the sky blue?”
2. A highly commercial query, “What is the best DUI lawyer near me”

Before the router, there was no way for a query to be distinguished, and after the router, the first low-value query could be routed to a GPT5 mini model that can answer with 0 tool calls and no reasoning. This likely means serving this user is approaching the cost of a search query.

The search query on the other hand has a fixed cost. It would show a page ranking of websites, with a potential AI summary at the top. This is a fixed supply response to what could be a variably hard question. But now ChatGPT free (because of routing) can dynamically answer a harder question with a better answer, which is not how search is designed today. Below shows the value of a changing supply query to a harder question, the supply is fixed for Search.

So now let’s bring in the higher value query, the DUI Lawyer question. As you may know, this is an extremely valuable question. Today on search, this is one of the higher cost per click keywords, and it is plastered with ads. In a world of dynamic supply, ChatGPT can not only answer this question, it could realize this is a very valuable question and answer this question at the level of a human. It could throw $50 dollars of compute if there is a belief of high conversion, because that transaction is worth $1000s of dollars.

The router makes this possible. ChatGPT 5 could decide to allocate $50 to the query, create a plan, gather information about the incident, research local lawyers, consider who is likely to answer fastest, consider your budget, and then contact multiple lawyers on your behalf. It could even Agentically reach out to lawyers on behalf of the free user knowing that the conversion ratio of this query is even higher. This version of ChatGPT is highly helpful, aligns with the user’s query, and is a valuable referral to the seller of the services or goods. This wouldn’t be intrusive in the current format, and many free users would use this instantly.

This would apply to more than just services. Products that are highly likely to be purchased with agentic purchase would pay for referral fees, such as groceries, ecommerce purchases, flights and hotels. This could be a consumer SuperApp that would be an “agent” for day to day planning, purchases, and basic services.

The user wouldn’t pay via a cost of a subscription, but by transaction fees or ad take rates on purchase. The AI agent could still make the best response output but drive extremely high value business to a company almost instantly, in which the business would be willing to pay take rates on.

And what’s more you can see the glimpses of this in their model release notes. They highlighted Gmail and Google Calendar integration, as well as new tool use benchmarks for services such as Telecom, Retail, and Airlines. Imagine a world where very little customer acquisition cost is spent on ads, but rather asking a helpful AI assistant to set up the best internet plan in your neighborhood.

And while this feels like speculation, this is already happening. Instacart added this feature to let agents checkout products in January of this year. Fidji Simo was at Instacart when this was implemented, and now she just joined OpenAI as head of product. The wheels are already in motion, this will be the future of free usage at ChatGPT.

AI labs such as Anthropic and OpenAI are even paying startups hundreds of thousands of dollars to spin up replicas of popular sites like DoorDash and Amazon to RL agents on successfully completing end-to-end transactions. It is not a question of if, but when this capability happens.

You can see the future. Imagine a world where you ask for new dinner recipes for the week, and ChatGPT gives you multiple options and orders the cart for you to check out. The fee would be paid on the completion of the purchase, and Search is completely cut out of the picture. Every thing that can be researched or planned in an AI app could be purchased for companies that adopt partnerships quickly.

Companies would flock quickly to this large new purchasing habit. Booking flights, purchasing items, buying food, etc. If there is a connection to a website and payment information, this could all be monetized for “free” use of the application but at transaction.

And while we are far from this future, the router is the necessary step to begin the sorting of high and low compute, and eventually commercial intent queries. None of this is possible without a single unified interface routing dynamic responses to users.

This would fulfill a vision for non-intrusive ads by Sam Altman, as well as continue to enable ChatGPT to be a trusted advisor to users for free. ChatGPT will become an agent to help users make one of the most important decisions in their day-to-day lives: buying stuff. OpenAI and Shopify are already working on a checkout integration today.

From Today to Agentic Purchasing End State

It’s no secret that we are very far from that future today. It would likely take a few steps before a true product launch, but it starts with a Router and likely many partnerships on the other side as connectors. Maybe in the beginning, ChatGPT gets affiliate fees for the currently recommended items that lead to a purchase. This would look like a typical affiliate marketing deal, and would have hard-to-measure success rates and lower take rates.

As the model becomes more agentic, it would likely need plug in to the service’s system to be able to make reservations, book flights, or schedule appointments. It would likely need heavy partnership.

Let’s examine some of the partnerships they have already made today. So far OpenAI has partnered with:

Finance Companies: Stripe, Visa, PayPal

Consumer Companies: Mattel, Booking.com, Lowe’s

Enterprise Software: Salesforce, Intercom, Zendesk

Consumer Internet: Snapchat, Shopify, Instacart, Mercari

Every company that can and will shift to a cheaper customer acquisition cost will be excited to move sooner. There will be low overhead, as less customer service, advertising, marketing and other functions will be needed as ChatGPT collapses that entire purchasing funnel into a helpful assistant.

OpenAI is Creating a Consumer SuperApp

OpenAI has clearly crossed the chasm with its web-scale presence. And every single large internet-scale property with this many users has been monetized for “free” by ads. Agentic purchasing with take rates can begin now after the implementation of the router.

We now turn our focus onto the hyperscalers, who are facing increasing competition in the consumer space, and which smaller companies are already benefiting from the shift of monetizable queries away from search to AI.

OpenAI is firmly knocking on the door of technology giants Google and Meta and even Amazon. Previous scares about AI have been focused on search query volume, not being replaced in the ad tech stack. ChatGPT can compete with dominant platforms for its place in the ecosystem, and to date this is push into purchasing is the most concrete example of OpenAI coming for advertising at large. If they were to first launch an aggressive Agentic checkout solution before Meta or Google, this would be seen as huge competitive shots at both companies.

A reminder that if we are talking about pure usage, only one company is growing users at a meaningful rate. It’s OpenAI.

In some ways by completely bypassing the top of funnel of search or pushing ads, it creates a third space for purchasing. Social media time continues to climb as human-to-human interaction is still the primary time consumption, so the research portion of consumer journeys would more heavily impact Google, but there is opportunity for smaller players. Certain smaller players are actually winning here as shown by the data below

Meta’s shocking purchase of 49% of Scale AI at a ~$30B valuation shows that money is of no concern for the $100B annual cashflow ad machine. Despite seemingly unlimited resources, Meta has been falling behind foundation labs in model performance.

The real wake-up call came when Meta lost its lead in open-weight models to DeepSeek. That stirred the sleeping giant. Now in full Founder Mode, Mark Zuckerberg is personally leading Meta’s charge, identifying Meta’s two core shortcomings: Talent and Compute. As one of the last founders still running a tech behemoth, Mark doesn’t need SemiAnalysis to tell him to slow down stock buybacks to fund the future!

In addition to throwing money at the problem, he’s fundamentally rethinking Meta’s approach to GenAI. He’s starting a new “Superintelligence” team from scratch and personally poaching top AI talent with pay that makes top athlete pay look like chump change. The typical offer for the folks being poached for this team is $200 million over 4 years. That is 100x that of their peers. Furthermore, there have been some billion dollar offers that were not accepted by researcher/engineering leadership at OpenAI. While these offers aren’t all successful, Zuck is crushing the competitors by drastically increasing their cost per employee.

Perhaps even more iconic, Zuck threw his entire Datacenter playbook into the trash and is now building multi-billion-dollar GPU clusters in “Tents”!

As this report details, nothing is off the table. We unpack Meta’s unprecedented reinvention from Compute to Talent in the pursuit of Superintelligence as well as the story of how we got here. From Llama 3.0 open-sourced dominance to the epic fail of Llama 4 Behemoth, this Titan of AI is down but not out. In fact, we believe Meta’s ramp in training FLOPS will rival even that of OAI. The company is going from GPU-poor to GPU-filthy-rich on a per researcher basis.

Meta GenAI 1.0: AI Incrementalism

Compared to pure-play AI labs like OpenAI, companies like Meta and Google have followed an “AI Incrementalism” strategy by enhancing existing products with better recommendation systems and GenAI to improve ad targeting, content tagging, and internal tools. This has paid off handsomely in financial results, allowing Meta to shrug off Apple’s attempts at stopping them from tracking users with the release of their App Tracking Transparency (ATT) feature in iOS 14.5 (late 2021, early 2022).

While Meta is arguably more insulated from GenAI disruption than Google, LLM efforts at both companies have been somewhat disappointing. One reason is the allocation of capital primarily for the core business rather than pursuing superintelligence.

“Our CapEx growth this year is going toward both generative AI and core business needs with the majority of overall CapEx supporting the core.”

Source: Meta Q1 2025 earnings call, emphasis SemiAnalysis

GenAI acts as an extension to these tech giants’ businesses. They don’t have the same existential need to dominate new use cases like OpenAI in chatbots and Anthropic in coding APIs. This is readily apparent in the compute vs human capital allocation of a leading AI foundation lab like OpenAI compared to Meta. Zuck’s impact on competing offers and inflation of salary will hurt badly.

As a result, when measuring GenAI consumer app traction, Meta and Google meaningfully lag ChatGPT in its reach and engagement.

But that’s changing. Leveraging our proprietary Accelerator Industry Model and Datacenter Industry Model, we forecast a meaningful step up in GenAI investment from Meta in the coming years.

Meta GenAI 2.0 – Part 1, Re-Inventing the Datacenter Strategy (Again)

From Buildings to Tents

Just a year ago, Meta scrapped its decade-old “H”-shaped datacenter blueprint for a new AI-optimized design.

Now in 2025, Zuck decided to re-invent the strategy again. Inspired by xAI’s unprecedented time-to-market, Meta is embracing a datacenter design that prioritizes speed above all else. They’re already building more of them! Traditional datacenter and real estate investors, still somewhat reeling from xAI’s Memphis site and time to market, will be shocked yet again.

This design isn’t about beauty or redundancy. It’s about getting compute online fast! From prefabricated power and cooling modules to ultra-light structures, speed is of the essence as there is no backup generation (ie no diesel generators in sight).

Power currently uses nearby Meta on-site substations. Meta likely uses sophisticated workload management to maximize the utilization of every watt of power it gets from the grid. It might even need to shut down workloads in the hottest summer days.

The Prometheus 1GW AI Training Cluster – An “All Of The Above” Infrastructure Strategy

Meta is quietly building one of the world’s largest AI training clusters in Ohio. We have heard from sources within their infrastructure organization they are calling this cluster Prometheus. To beat rival AI labs, Meta has carried out an “all the above” infrastructure strategy:

• Self-build campus.
• Leasing from third parties.
• AI-optimized designs.
• Multi-datacenter-campus training.
• On-site, behind-the-meter natural gas generation.

Looking at the Prometheus training cluster below, we believe Meta is connecting all these sites with ultra-high-bandwidth networks all on one backend network powered by Arista 7808 Switches with Broadcom Jericho and Ramon ASICs.

By combining both self-build and leasing, Meta ramps faster. In fact they pre-leased more capacity second half of 2024 than any hyperscaler, mostly in Ohio.

Furthermore, when the local power grid couldn’t keep up, Meta went full Elon mode. With help from Williams, they’re building two 200MW on-site natural gas plants. The equipment breakdown for the first includes:

• 3* Solar Turbines’ Titan 250 turbines
• 9* PGM 130 turbines
• 3* Siemens Energy SGT400 turbines
• 15* CAT 3520 Reciprocating Engines

A future SemiAnalysis report will dig much deeper into these systems for the purpose of powering datacenters: cost, advantages & disadvantages, etc. This has signficant negative implications for a certain supplier of onsite natural gas solutions.

This total compute advantage for OpenAI is important as the advent of reinforcement learning means many large datacenters distributed around the US can be used asynchronously to contribute to improving model intelligence with post-training.

Not to be one-upped, Hyperion—Meta’s second frontier cluster—aims to erase that gap to OpenAI.

Beating Stargate at Scale: Meta’s Hyperion 2GW Cluster

While all eyes are on the high-profile Stargate datacenter in Abilene, Meta has been planning a response for over a year and making tremendous progress. The Louisiana cluster is set to be the world’s largest individual campus by the end of 2027, with over 1.5GW of IT power in phase 1. Sources tell us this is internally named Hyperion.

Meta broke ground at the end of 2024 and is currently actively working on both the power infrastructure, and the datacenter campus.

To be clear, Meta has many other datacenters under construction and ramping. A full list of Meta’s AI Datacenters, their expected completion dates, and power per building by quarter can be found in our Datacenter Industry Model.

Llama 4 Failure – From Open-Source Prince to Behemoth Pauper

Before we dive into the Superintelligence Talent race, we should take a look at how Meta found itself in this awkward position. After leading the open-source frontier with Llama 3, Meta now finds itself trailing China’s DeepSeek.

On a technical level, we believe the major contributors to the failed run were as follows:

1. Chunked attention
2. Expert choice routing
3. Pretraining data quality
4. Scaling strategy and coordination

Chunked Attention

Naively implemented, attention in large language models scales quadratically with token count. To address this, researchers introduced memory-efficient mechanisms. Meta chose chunked attention for Behemoth, and that may have been a mistake.

Standard Causal Attention: Imagine a series of expanding triangles fanning out from the top left corner, representing the attention size for every subsequent token. Double the tokens and the area of the triangle quadruples.

Chunked Attention breaks this triangle into fixed-size blocks. Each block resets attention to a new “first” token. With the efficiency of reduced memory, it enables even longer context. Meta felt they needed this to achieve long context, but the tradeoffs aren’t worth it. The first token in each block lacks access to prior context. While there are some global attention layers, that’s not enough as we expand on below.

Sliding Window Attention, used in other models, provides a smoother alternative: the attention window slides forward token-by-token. This maintains local continuity, even if long-range reasoning still requires multiple layers to propagate context.

Behemoth’s implementation of chunked attention chasing efficiency created blind spots, especially at block boundaries. This impacts the model’s ability to develop reasoning abilities as chain of thought exceeds one chunk in length. The model struggles to reason across longer ranges. While this may seem obvious in hindsight, we believe part of the problem was that Meta didn’t even have the proper long context evaluations or testing infrastructure set up to determine that chunked attention would not work for developing a reasoning model. Meta is very far behind on RL and internal evals, but the new poached employees will help close the reasoning gap massively.

Expert Choice Routing

Most modern LLMs use a Mixture of Experts architecture in which, between each model layer, a the token is routed to different experts based on a router. In modern MoE models, most are trained with token choice routing, i.e. the router provides a tensor of shape T x E (T being total tokens, E being number of experts in the MoE model) and a topK softmax is run on the E dimension producing a T by K tensor. This effectively means that the router is asked to choose K most probable experts for each of the tokens T, where K can be one or more experts. K is a hyperparameter that can be tuned by the researcher.

The advantage of this approach is that each token is guaranteed to be attended to by K experts, making sure the information value of each token is absorbed by the same number experts. The disadvantage is that certain experts can be disproportionately ‘popular’ with tokens while other experts are under-trained, resulting in an imbalance in the ‘intelligence’ of each expert. A known problem, many of the top labs have developed a fix with auxiliary-loss (or loss-less) load balancing. When training with EP (Expert Parallelism), this can lead to lower training MFU, as the model is spread across different GPU nodes, resulting in many more collectives (NCCL) running across the scale-out network (InfiniBand or RoCE) instead of the scale-up network (NVLink). This is a major motivation for NVIDIA’s NVL72 design, where the scale-up network expands beyond standard 8-way servers.

Expert choice routing, introduced by Google in 2022, flips the logic: experts choose the top-N tokens. Taking the same T x E tensor produced by the router, in expert choice routing a topN softmax is run on the T dimension which produces a E by N tensor. This effectively means each of the E experts has chosen the N highest probability tokens to be routed. The N hyperparameter can be tuned by the researcher but to compare it to a token choice routing approach, N = K * T / E.

When compared directly to token choice routing, expert choice routing guarantees that experts are being activated in a balanced manner, avoiding the performance degradations associated with imbalanced experts. To be clear: the router makes the choice in both cases. In token choice, the input to the router is the tokens, and it picks the experts. In expert choice, the input to the router is the experts, and it picks the tokens.

This equalizes the load for expert training and improves MFU across distributed hardware. Hyperscaler networks are purpose-built for this kind of parallelism, and we cover them extensively in our Networking model.

The disadvantages of this approach is inverse of the token choice architecture. Expert choice routing can result in certain ‘popular’ tokens being attended to by multiple experts. While this doesn’t produce the same training bottleneck that is present in token choice routing, it can result in degraded model generalization as the LLM now does not pay attention to all tokens equally. Furthermore, most EC models need to be finished with TC training to produce an effective LLM.

Inference is also another issue for EC. Inference is split in two steps: Prefill and Decode. In the Prefill stage, the user prompt is encoded and loaded into the KVCache. This step is Flop-bound. In the Decode step, the model calculates the attention and runs through the feed forward network of the model one token at a time one layer at a time.

Here, expert choice routing struggles as the experts can only choose from to 1 token x batch size per layer initially, resulting in each expert only given a very small set of tokens compared to when it was trained (an example training run would have 8k seqlen x 16 batch size = 128k tokens per pass). Being able to see the entire sequence as in training would break causality of modern autoregressive models hence the mix of EC and TC even for EC models.

Meta switched from expert choice to token choice routing partway through the run which is not unusual for EC models. However the performance drop from the switch resulted in a model that was meaningfully worse than a model fully trained on TC.

Data Quality: A Self-Inflicted Wound

Llama 3 405B was trained on 15T tokens and we believe Llama 4 Behemoth required substantially more tokens, 3-4x order of magnitude larger. Getting sufficiently high-quality data is a major bottleneck that Western hyperscalers cannot shortcut by copying the homework of other models’ outputs.

Prior to Llama 4 Behemoth, Meta had been using public data (like Common Crawl), but switched mid-run to an internal web crawler they built. While this is generally superior, it also backfired. The team struggled to clean and deduplicate the new data stream. The processes hadn’t been stress-tested at scale.

Furthermore, unlike all other leading AI labs including OpenAI and Deepseek, Meta does not utilize YouTube data. YouTube lecture transcripts and other videos are an incredible source for data and the company may have struggled to produce a multimodal model without the data.

Scaling Experiments

Beyond the above technical issues, the Llama 4 team also struggled to scale research experiments into a full-fledged training run. There were competing research directions and a lack of leadership to decide which direction was the most productive path forward. Certain model architecture choices did not have proper ablations but were thrown into the model. This led to poorly managed scaling ladders.

As an example of how hard scaling experiments can be, let’s look at OpenAI’s training of GPT 4.5. OAI’s internal code monorepo is very important for training their model as they need a validation dataset to measure perplexity against when doing training ablations that is known to be uncontaminated. While scaling GPT 4.5 training experiments, they were seeing promising developments in the model’s ability to generalize only to realize mid-run that parts of the monorepo were copied-pasted directly from publicly available data. The model was not generalizing but rather regurgitating memorized code from its training dataset! Large pretraining runs requires enormous amounts of diligence and preparation to effectively execute.

Despite all of these technical issues, not all was lost. Meta was still able to distill the logits into the smaller and more efficient pretrained Maverick and Scout models, bypassing some of the flawed architectural choices of the larger model. Distillation is far more efficient than reinforcement learning for smaller models. That said, these models are still bound by the limitations of their source: they aren’t best-in-class for their size.

Meta GenAI 2.0 Pt 2: Bridging the Talent Gap

With infrastructure revamp underway and the technical lessons absorbed, Meta’s GenAI 2.0 strategy now pivots to the next ingredient of superintelligence: talent.

Mark Zuckerberg understands the talent gap relative to leading AI labs and has taken over recruiting. He’s on a mission to build a small but extremely talent-dense team, casually offering signing bonuses in the tens of millions of dollars. The goal is to create a “flywheel effect”: top tier researchers join the adventure, bringing credibility and momentum to the project. It’s already working with recent high-profile hires including:

• Nat Friedman, former GitHub CEO
• Alex Wang, former Scale AI CEO
• Daniel Gross, who was the CEO and co-founder of SSI, Ilya Sustkever’s startup.

The recruiting pitch is powerful: unrivaled compute per researcher, a shot at building the best open-source model family, and access to over 2 billion Daily Active Users. The offers that generally range from $200M to $300M per researcher for 4 years also strengthens this pitch. As such Meta has acquired awesome talent from OpenAI, Anthropic, and many other firms.

M&A, Scale AI, etc

Zuckerberg reportedly made acquisition offers to both Thinking Machines and SSI, but was turned down. While some have noted that Zuckerberg “settled” for Scale AI, we do not think this is the case. As we discussed, core to many of the Llama 4 issues were data problems and the Scale acquisition is a direct move to address that.

Alex will bring many of the top engineers from Scale, especially the SEAL lab which specializes in evals that Meta so desperately needs. SEAL has developed one of the top benchmarks for Reasoning model evaluations, HLE (Humanity’s Last Exam). With Nat Friedman and Daniel Gross joining the team, Meta gains not just elite operators but tow of the most prolific and respected investors in the AI community. Meta has very strong product people at the top.

The More You Buy The More You Save: OBBB Edition

Zuck could not have picked a better time start this spending splurge. The One Big Beautfiul Bill has some tax goodies specific to hyperscalers that could massively accelerate the tax incentives to build now and go big. Superintelligence funded by the federal government is the modern Manhattan Project.