Much has been written about xAI’s Colossus 1. The Memphis build belongs in the history books: the largest AI training cluster, erected from scratch in 122 days. With roughly 200,000 H100/H200s and ~30,000 GB200 NVL72, it remains, today, the largest fully operational, single-coherent cluster (setting apart Google, master of multi-datacenter-training).

However, Colossus 1’s ~300 MW looks modest next to the Gigawatt-scale clusters under construction by OpenAI, Meta and Anthropic. Their hyperscaler partners are happy to leverage their balance sheet and win the market by throwing dollars at it.

Was xAI’s prowess a one-time wonder? Today we will publicize some data from our industry leading datacenter model over the last year that is accessible to clients. This is the our same proprietary data that called the Oracle deals many months ahead of the announcement.

Short answer: no. xAI is still squarely in the frontier-AI race and is positioned to leapfrog most rivals again on compute. By our estimates, its total datacenter capacity for a single training cluster will surpass Meta Superintelligence and Anthropic by Q3 2025. The datacenter capacity will be ready for the GPUs to be moved in to create the largest single datacenter in the world, yet again. xAI has to raise the capital for those GPUs, but they have the allocations from Nvidia to have it fully training large scale models early next year.

Elon came up with a new genius trick to beat rivals at time-to-market. Colossus 2 will be an even more impressive achievement than xAI’s first cluster. Let’s dig in.

The first half of this report will dig into the Colossus 2 prowess. The second half will discuss Grok models, our mid-to-long term thoughts on xAI, and the unique RL method xAI is using that may lead them to leapfrog OpenAI, Anthropic, and Google.  

---

SemiAnalysis Is Hiring

We are seeking a highly motivated & skilled Member of Technical Staff to join our growing special projects engineering team. You will play a crucial role in developing industry leading gpu cloud benchmarks & evaluation framework.  Our gpu cloud evaluation frameworks is endorsed by many tier 1 & 2 frontier labs. You may be a good fit if you have the following experience:

• Demonstrated experience in ML frameworks such as PyTorch or JAX through professional experience, personal projects, or personal Substack blogs
• 1-2 years using GPU or TPU clusters and/or running a multi-tenant GPU cluster
• Past experience working at a hyperscaler or a GPU cloud (preferred)
• Solid understanding of SLURM, Kubernetes, NCCL & GPU Cloud industry
• Strong research skills and the ability to synthesize information from various sources to draw insights

Compensation is competitive & as part of the interview process, you’ll complete a paid coding challenge designed to reflect typical daily tasks at SemiAnalysis

---

Colossus 2: from zero to 200MW in six months

The Colossus 2 project was kicked off on March 7^th, 2025, when xAI acquired a 1m sqft warehouse in Memphis, and two adjacent sites totaling 100 acres. By August 22^nd, 2025, we count 119 air-cooled chillers on site, i.e. roughly 200MW of cooling capacity. That’s enough to power roughly 110k GB200 NVL72. And an Elon tweet shows some racks were already installed in July.

xAI built in six months what took 15 months for Oracle, Crusoe and OpenAI!

Looking closer at the picture above, readers familiar with our Datacenter Anatomy series might wonder where the power infrastructure is. The picture doesn’t show any onsite substation, nor does it show onsite turbines…  How is the datacenter powered?

Colossus 2: in Tennessee, Mississippi… or both?

Things get more confusing once factoring-in a Greater Memphis Chamber statement in May that no turbines would be sited in Memphis. They’re not lying.

Memphis and Tennessee have been getting a lot of pushback, so xAI’s genius move was to develop a Gigawatt-scale energy hub right across the border in Southaven, Mississippi. In mid-2025, the company acquired a former Duke Energy power plant in Southaven. Shortly after, Mississippi regulators granted xAI temporary approval to run gas turbines there for up to 12 months without a permit!

To transport and manage power generated by the Mississippi power plant, xAI is building infrastructure near Colossus 2. We show below the first deployment of Tesla Megapacks, as well as the Medium Voltage power lines connecting the two sites.

From 200MW to 1.1GW and Solaris Energy Infrastructure partnership

In Southaven, MS, xAI is moving at the pace of light. The discontinued power plant now sees seven 35MW turbines in operations.

To deploy faster than peers, xAI relies on rental turbine companies. NYSE-listed Solaris Energy Infrastructure owns a fleet of 600MW of gas turbines, of which ~400MW currently serve xAI. Musk’s firm weighs 67% of SEI’s 1700MW orderbook, i.e. 1,140MW. There are ~240MW on the Memphis Colossus 1 site, while the remaining 900MW will be owned by a Joint Venture owned at 50.1% by Solaris and 49.9% by xAI.

As shown below, ~460MW are currently installed and in operations/under construction.

The newly formed JV already spent $112 million in capital expenditures in Q2 2025. After a slow Q3, spending will pick up again in Q4 2025 and Q1 2026. Solaris expects to have over 1.1GW of fully operating turbines for xAI by Q2 2027. There remains ~425MW available for contracting, and we think xAI will likely pull the trigger to get to over 1.5GW of total gross power. Solaris also appears to temporarily lease power generation capacity from third parties to deliver faster:

During the second quarter, the Power Solutions segment generated revenue from approximately 600 megawatts of capacity, an increase of greater than 50% from the prior quarter. This increase was driven by increased demand from our customers, which we are meeting using a combination of new equipment deliveries as well as selective short-term sourcing of third-party power generation capacity.

Solaris Energy Infrastructure, Q2 2025

As such, xAI has figured out how to scale to >1GW from a power perspective. In terms of datacenter space, we see four options:

• Given a 40ft height, xAI could turn their 1mm sqft warehouse into a two-story datacenter, doubling the space. Given ultra-high density, 2mm sqft could be enough for >1GW.
• xAI could build a second, smaller facility in parcel 3
• They could acquire more land, possibly in Mississippi near the Southaven plant.
• Using non-standard layout of the datacenter, they could achieve >1GW as is.

Can xAI afford Colossus 2?

Further expansion will require ample funding. Required CapEx for Colossus 2 will be in the tens of billions of dollars, and xAI is yet to generate any meaningful external revenue, with the preponderance of the rumored 9 digit revenue ARR being inter-company transfers from X.com to xAI. We’ve forecasted xAI’s CapEx on Core Research, our institutional research service and are now closely tracking the ROIC of AI investments across the hyperscalers and AI labs in our new Tokenomics model.

Middle East – funding + datacenters, a winning combination?

To be clear, in typical xAI & Elon fashion, the company’s future is highly unpredictable. But given funding needs, we see a large-scale expansion in the Middle East as likely. Musk and the Middle have a long-time relationship:

• KSA’s Kingdom Holding Company (owned at 16.87% by the Public Investment Fund) owned and kept a $1.9B stake in Twitter when Musk took the company private in 2022. It also owned a $800M stake in xAI prior to the merger with X.
• UAE’s privately owned Vy Capital brought in $700M in 2022 to support Elon’s takeover of Twitter. It also invested in xAI’s Series C, alongside UAE state fund MGX.
• Qatar’s QIA also owned and kept a $375M stake in Twitter, and participated on xAI’s Series C.

According to the FT, xAI is preparing a new round in the tens of billions, valuing the company close to $200B, with Saudi’s PIF sovereign wealth fun playing a large role. This is challenging though because its hard for most investors to justify xAI as having a valuation higher than Anthropic.

We’ve heard of a raise as large as $40B. Given skyrocketing datacenter growth in the area, we see a two-way deal as likely, with xAI deploying that capital in a new large scale datacenter in the Kingdom.

We show below a likely location for xAI’s expansion, a large-scale planned located Saudi Arabia that recently broke ground. While still early, there is plenty of land and power to serve large-scale AI campuses.

For more details on the Middle East’s AI expansion, check out our May 2025 deep dive.

AI Arrives In The Middle East: US Strikes A Deal with UAE and KSA

Beyond external capital sources, Elon could generate capital internally. Since merging X.com with xAI to form X Holdings, we believe an ever growing piece of xAI’s revenues is inter-company transfers, i.e. calls to @Grok to answer questions or just X.com licensing the LLM technology for functionality such as search, ad recsys, or even content creation.

This is just money going from Elon’s right pocket to his left pocket. From what we can track externally, Ani was a huge boon for Grok app revenue but even growth in that revenue stream has flattened out in recent months. Ani needs more gacha to grow revenue further.

At the end of the day, Elon can get Tesla to invest more or take loans on more of his Tesla and SpaceX stocks to invest tens of billions into xAI. This will allow them to build Colossus 2. No one truly knows how levered Elon is already, but it is widely understood he can always sell and unlock a lot more of his dry powder into xAI. Elon will do everything he can to not lose to Sam Altman.

Let’s now discuss xAI as a business and whether we think the company has a shot at being a Frontier AI Lab and justifying a multi-hundred-bullion dollar valuation. We have some unique info and insights below and why they may have a shot at being first to AGI due to their different approach.

Does xAI have a shot at becoming a frontier lab?

Compute is the lifeblood of AI. He who controls the spice controls the universe the compute will control the production of tokens and reap the benefits of AI. Without compute you do not have a seat at the table. The United States technology community is all in on compute and AI as the next platform and is now adding compute at a staggering pace.

There is competition, and it not only comes from companies but from countries, and the US government has placed a series of export controls to limit China’s rising compute. Today the US controls and is the undisputed leader in compute with more than 70% of the worlds deployed FLOPs. One way to stay ahead is to keep going full steam while hindering your competition. Limiting your competitor nation state from compute, which will limit them from intelligence, is the current policy to stay ahead in the AI race.

These moves have led to backlash, including with China cutting rare earth minerals and magnets off from the US. Secretary of Commerce Howard Lutnick says the resumption of Nvidia GPU sales to China were required to restart China’s shipments of their linchpin supply chain materials.

But constraints have also led to adaptation, and Chinese companies have adapted. High batch sizes and disaggregated serving are but two examples. Despite advances, in the case of DeepSeek, most of their tokens are still inferenced on western hardware. We wrote about this dynamic in our recent DeepSeek debrief. Training of DeepSeek’s next generation model was also delayed by the use of Huawei chips, as we also said in the debrief.

This is not a stable equilibrium. There are always moving pieces in the race for intelligence. Beijing plans for the long term and knows it must secure its own domestic compute destiny. There is an irony: in the 2010s China kicked out Google to enforce its Great Firewall and foster its domestic industry, this time the US government is withholding hardware technology so they cannot seize the lead in AI.

We believe that at China’s core, they want to control not only its internet and AI, but the hardware that supports it. From silicon to tokens, China seeks sovereignty over every layer of the stack, and given recent history will never want to be beholden to foreign powers. Enter Huawei.

China loves national champions, and in it’s characteristic capitalism, tends to funnel resources to a few national champions. Today’s champion is probably Nvidia’s greatest adversary: Huawei. We expect Huawei to be able to make millions of chips this year, and to be bottlenecked by HBM next year. Today we want to talk about Huawei, which can translate as “China’s achievement”.

All In with Huawei’s chips

Huawei is the key piece for China’s compute destiny. Huawei’s chip ecosystem is vertically integrated and capable network of tools, fabs, and design that allows it to express a full stack vision of hardware. This hardware is very impressive although less efficient then western hardware.

Our previous investigation into Huawei’s fab network is a great example of their broad reaching network.

Huawei wants to vertically integrate the entire manufacturing process. The goal is to own not just the manufacturing of the logic die, the brains of the chip, but also the memory and packaging. They have even created their own tool company, SiCarrier to copy foreign firms tools. Huawei has bought over $9B of tools to put in their own fabs as well as reverse engineer to replicate.

Their efforts are not to be underestimated. SiCarrier, for example, has recently raised $2.8B in funding. That money is going into building fabs dedicated for Huawei staffed by Huawei employees. Some reporting on these fabs, which we believe to be Huawei owned and operated, suggests that their combined production by next year could entirely exceed SMIC, the current leader and where production is currently outsourced. With Huawei ramping up their own efforts, allocation at SMIC can be freed up for other chips, including Cambricon. The Cambricon chip is itself popular among Chinese companies, especially ByteDance.

Huawei operating their own fabs will represent a material increase in Chinese production, but also in their ability to iterate, control, and improve on processes. Huawei and SMIC will directly work on increasing yields, refining R&D for the next node, and bolstering Chinese semiconductor manufacturing capability.

Currently, all high volume chip production is outsourced to SMIC, the leading Chinese pure-play foundry. This includes the Ascend series of accelerator chips along with the Kirin mobile processors. Yields are poor for SMIC’s 7 nm-class processes due to a combination of immaturity, export controls, and the inherent difficulty in yielding large die such as the Ascend. Thus a relatively low percentage of SMIC’s overall capacity is allocated to producing Ascend die, since smaller mobile processors just make better business sense at this point. But this can change quickly. Let’s discuss what is possible for Huawei….

Huawei Production Numbers and SMIC’s Ramp

Our data shows Huawei’s production is 507k Ascend units shipped in 2024, the majority of which are 910Bs, and 805k this year, 653k of those being 910C. The 910C is the more advanced version. This includes die made by TSMC and SMIC.

SMIC, hampered by export controls, struggled to get production off the ground. But Huawei got Ascend die fabricated at TSMC while SMIC ramped. This was a violation of export controls and Huawei ended up receiving more than 2.9M Ascend die, which can be used for both 910B and 910C. We detailed this here.

It is specifically this “Die Bank” of foreign chips from TSMC that gets them through 2024 and 2025. Without this Die Bank, Huawei’s Ascend production numbers would be much lower.

We expect that the TSMC die bank to run out within the next 9 months. SMIC, however, now has more than enough capacity to produce meaningful volumes of chips. We forecast that SMIC will no longer be the bottleneck for Ascend production, as they will have sufficiently ramped capacity by end of year.

The chart above shows a baseline scenario with a modest increase in SMIC capacity allocation for Ascend. It requires at most 20k wafers per month (wspm) of SMIC capacity to produce millions of Ascend die per month.

For reference, a conservative estimate of SMIC’s total advanced node capacity (7nm and below) is 45k wspm by end of 2025, increasing to 60k wspm in 2026 and 80k wspm in 2027. In addition, Huawei is building their own fabs, not all of which are export controlled, and collaborating with SMIC on process technology, so production could ramp higher for advanced process technologies.

If 100% of capacity was allocated to Ascend die, their production capability would be in the tens of millions per year. They are on track to be more than capable of supporting a large domestic demand for China-produced compute die.

Our forecasts above use a conservative estimate for both yield and its rate of improvement in the future. It’s very likely that SMIC can exceed these estimated as its 7nm nodes mature. Our estimates for yields are lower than TSMC, Intel, Samsung, ASE, Amkor, etc for front end wafers and packaging.

Indeed, yield is an important lever that can be pulled to increase production without giving up more allocation. With just small increases in yield beyond our forecast above, SMIC will be able to produce several million Ascend die at lower allocation amounts than they would otherwise. Every percentage counts and SMIC has their best engineers working on exactly this problem.

In other words, SMIC only needs small amounts, as low as single digit percentages, of allocation to produce more than a million die as soon as early next year. Production to a couple million is possible, more allocation is all that is needed. For the reasons we articulate above, we believe the reported number of 200k Ascend chips to be significantly off the mark.

While SMIC is expanding and is no longer the bottleneck and Ascend production, Huawei’s long-term ambitions include its own fab network buildout. We noted in our Fab Whack-A-Mole report:

Huawei is clearly taking full advantage to the tune of $7.3B of WFE expenditure in 2024, up 27% year-on-year. They’ve gone from effectively zero in 2022 to the 4^th largest WFE customer globally in two years.

Since we published that report in fall of 2024, the Huawei fab buildout has, if anything, accelerated. Their ecosystem of semiconductor-related shell companies is expanding. Massive new cleanroom buildouts are in progress. And improved domestic options and possible diversion of $30B+ of imports of wafer fabrication equipment means they are likely able to equip these fabs as well. The details are worthy of their own report, but suffice to say Huawei continues with a large, concentrated effort to own every vertical in the Ascend supply chain.

TSMC Access

Huawei currently makes many mobile chips, but there is zero reason to do so from a strategic geopolitical perspective. Oppo and Xiaomi currently fabricate mobile SoCs at TSMC. They are ramping up their own designs to decrease dependence on companies like MediaTek and Qualcomm.

Huawei can reduce their mobile SoC production while not consuming, in any meaningful volume, SMIC allocation. The continued access to TSMC for other Chinese entities decreases the pressure for SMIC to make mobile SoCs, meaning more can be allocated to AI chips.

Huawei and SMIC can ramp up AI production without having to worry about serving all the nation’s demand for mobile chips due to other companies’ access to TSMC.

Export Control Lag Benefits SMIC

Another part of SMIC’s expansion strategy is stockpiling considerable amounts of semiconductor tooling. The controls are released on an announced schedule and it’s usually easy to anticipate what will be included.

This is due to timing differences in controls that they can take advantage of. For example, the U.S. routinely exempts Japanese and Dutch companies from its equipment export controls. Ostensibly this is done because they are allied countries and have their own export control regimes and semiconductor industries.

The problem, however, is that when new export controls come out, Japan and the Netherlands do not immediately follow. In many instances, matching controls are delayed by 6 months or never. Chinese companies can rush order years’ worth of equipment to stockpile, while American vendors are shut out of the process.   Japanese WFE vendors are happy to sell into this gap with margins fattened by order expedites. Many key suppliers are seeing well above 40% revenue share from China:

In cases where controls do follow the US, Japan and the Netherlands do not have any matching controls on re-exporting. This means that it is possible for restricted tooling to reach China if it goes through a different country first.

In addition, the use of “advanced ICs” as the bar for restricting tools leaves room for issues. ASML’s NXT:1980 scanners are perfectly capable of 7nm-class logic, and likely beyond if economics (throughput and yield) can be ignored or subsidized. These are allowed into China and even to certain SMIC facilities.

Export controls, as noted, can be expanded in addition to very stringent enforcement of existing mechanisms. This includes tighter coordination with allies on timelines for matching controls and getting buy in on re-exporting.

We are broadly encouraged by the administration’s Action Plan. As an example, we are pleased to see controls on semiconductor subsystems being called out. This is a drum SemiAnalysis has been beating for a while and think it is the right direction. With that said, many subsystem firms that supply western players such as VAT Group in countries like Switzerland without controls will not be stopped from shipping critical chambers to China.

The Action Plan also included concrete suggestions around aligning protection measures globally, which we think if implemented, will greatly ameliorate the issue of alliance’s lags in export controls described above. The issue is that SMIC and CXMT are able to continue to expand production because the sanctions on them are flimsy. International co-operation, especially with partners in Korea, is key to get right and ensure that Chinese commercial bottlenecks are not eased. Korea manufactures a huge amount of memory and Samsung has historically supplied large amounts of memory into China, so tight alignment of goals and enforcement is critical.

The other part of the supply chain that is just as critical is memory. This is where we believe the key constraint lies.

HBM Is the Bottleneck

We believe HBM production is the bottleneck. China does too, which is why they have asked US officials to relax controls on HBM as part of the recent trade talks. What the ask omits is telling: it does not include more TSMC access or lithography tools. Beijing is specifically asking Washington to loosen HBM restrictions.

Much like how Huawei was able to stockpile TSMC logic wafer inventory, they were also able to stockpile HBM inventory. Samsung, due to their failures in entering the accelerator supply chain for Western chips, sold their product to Chinese customers that funneled this inventory to Huawei. This is why it is so critical to work with Korea on enforcement of memory controls moving forward.

Samsung alone has directly provided 11.4 million stacks of HBM to China, including a staggering 7M stacks in the 1-month gap between controls announcement and enforcement dates. When including other providers and methods of shipment, that is 13 million stacks of HBM.

Specifically, on December 2^nd 2024, the Bureau of Industry and Security (BIS) announced controls on anything more advanced than HBM2E. Full compliance was required on December 31^st 2024. Samsung exported as much as possible to China in that one quarter. This comprises the majority of China’s HBM. They were able to achieve this due to the US Government and media telecasting the restrictions for many months before they came out.

After the ruling, Samsung HBM still made it into China. We previously detailed how companies like CoAsia Electronics and Faraday supplied non-functional chips with HBM into China, though we believe due to our efforts in exposing this both privately in late January and publicly later that it has now stopped with revenue numbers returning to normal. This does not mean that HBM smuggling has stopped entirely, though. There could be other sources.

In sum, China has procured 13M HBM stacks which is sufficient for 1.6M Ascend 910C packages. Despite this, we expect that China will be bottlenecked by HBM by the end of the year as they run out of foreign HBM.

China can easily make more than 805k Huawei Ascends this year from TSMC and SMIC capacity, but they will not because they do not have enough HBM.

We expect SMIC’s production to be 1M 910Cs and almost half a million 910Bs this year, however, not all of them will be turned into ASICs as the HBM is not available. If some HBM enters through smuggling, then Huawei can produce more Ascend AI ASICs.

Without Foreign HBM, China has no Domestic AI Accelerator Industry

Without access to more foreign HBM, Huawei will not be able to fabricate even 1 million Huawei Ascend chips next year. They must rely entirely on domestic production which we will detail below. Nvidia and AMD have effectively 0 competition in China once these HBM banks run out.

The other option available to China is utilize slower GDDR and LPDDR memory, but this is not suitable for the leading language models with modern reinforcement learning techniques or for large scale inference deployments.

Domestic HBM Industry – CXMT

China’s main DRAM player, CXMT, has caught up with the west rapidly. This is due to a combination of extremely strong domestic engineering capabilities, poaching engineers from Samsung, SK Hynix, and Micron, as well as the leading tool vendors Applied Materials, Lam Research, and Tokyo Electron teaching them sub-processes.

CXMT is able to ship DDR5 memory, only a couple years behind SK Hynix, Micron, and Samsung, and is winding down profitable DDR4 production previously reserved for PCs and mobile. While they have not shipped much HBM, their roadmap is aggressive. By next year, their production capacity will rival that of Micron’s, though not fast enough to save Huawei Ascend’s production. In 2026, we expect them to be producing 257k WPM, which would be just under 15% of global DRAM production. In our estimates, this scales to 490k in 2030.

CXMT’s shift in focus and production ramp are driven in part by investments from the CCP. China’s “Big Fund III”, which started in May 2024, invested $2B into the company. CXMT is also expanding in Shanghai as well as Beijing, with HBM packaging subsidiaries being set up in the former.

Expecting, though ultimately averting, an entity listing by USG, CXMT stockpiled years of tools in 2024 and is likely still adding HBM-specific gear. CXMT is still not entity listed! While HBM was noted as a key focus in that salvo of controls, the previous administration failed to include China’s HBM champion. The Trump administration needs to solve this failure from the Biden administration immediately.

The stockpiling matters as advanced tools, like Hanmi’s TCB systems for HBM3, are restricted, but they are not that instrumental. Older systems can be run slower without impacting cost too much. More importantly, CXMT can still procure leading edge equipment for Through Silicon Via (TSV) formation – that is critical for making HBM -through Japanese suppliers. In addition, Chinese OSATs such as JCET and Tong Fu are also racing ahead with their R&D efforts and building capacity for the critical TSV and stacking processes to package front end HBM wafers from CXMT. This is unlike the Western memory incumbents who have vertically integrated these processes. This is a typical example of Chinese industrial development where multiple players are encouraged to develop domestic manufacturing capabilities: creating cutthroat competition that accelerates the speed of development. It is therefore critical to not just focus controls on CXMT, but the whole of China as critical process steps can always be outsourced.

Because front-end logic is currently not the binding constraint, Beijing’s main asks in the trade deal target HBM and the relaxation of controls relating to this set of tooling. Given the importance of HBM, it is critical to understand CXMT’s future production depending on various allocation scenarios.

CXMT Production Forecast

This can play out in several ways, depending on how much wafer capacity is given to HBM. China can easily produce tens of millions of stacks with less than 50% of wafer capacity. CXMT is at a bit over ~250,000 wafers a month of production capacity and is expected to reach 300,000 wafers by the end of the year.

Currently they have not built out the tooling required to convert standard DRAM production lines to HBM, but it is inevitable. Properly designed, targeted sanctions can slow this conversion massively.

Different scenarios lead to different amounts of Huawei Ascends produced. As noted, SMIC can produce the die needed to match the HBM.

To be clear, this could change. The rate of production for CXMT could increase if they continue stockpiling key tooling or significantly improve yields. The might of China’s manufacturing capability and capacity knows no bounds. This estimate is somewhat conservative, it is likely that CXMT is able to produce the significantly more capable HBM3e in 2026.

We believe CXMT will only be able to make ~2 million stacks of HBM next year, which is only sufficient for 250,000-300,000 Ascend 910C’s. Yields and capacity conversion will take some time to improve for CXMT to commit significant capacity.

If all leading edge logic die capacity could be paired with HBM, Huawei production of Ascend would grow from 805k this year to 1,175k in 2025. More importantly, production next year would grow from 300k to over 5 million Ascend 910C!

Our analysis indicates that export controls have been effective in constraining and limiting Chinese chip production capabilities. Assuming no smuggling, China will be able to make less Ascends next year, not more. CXMT is squeezed tight. Had controls not been present, the Ascend ramp would be fully realized, Chinese models would be served on Huawei Ascend at scale, and there would be enough compute capacity that advanced models like DeepSeek R2 and V4 would already be here. Not to mention, with more capacity, China would be better positioned to export their AI on their chips.

As such, it is absolutely critical to ensure the application, enforcement, and continued updating of export controls to prevent CXMT and related entities from ramping production. As mentioned, this includes not just CXMT but OSATs and subsidiaries they work with. Second, it is also important for the intelligence community to track and identify any instances of HBM smuggling, like the Faraday + CoAsia scheme which we disclosed privately in January and publicly reported on later.

By no means should HBM be allowed to be shipped into China. Production of AI chips over the next few years is heavily gated by CXMT’s ramp and foreign shipments of HBM.

There is very strong incentive to find ways to ship HBM into China. This is why enforcement of controls are so critical.

There is another pillar to China’s production strategy, separate from memory and logic. This pillar is about networking the chips together and where they are made.

Networking and Datacenter CPU

Chips do not exist in isolation. We have argued that the system matters more than the microarchitecture for years. Clusters are comprised of many tens of thousands of interconnected chips and how they are interconnected matters.

We detailed Huawei’s CloudMatrix 384, or CM384, system below.

We believe the networking equipment used, specifically the scale up switches, are being made at TSMC and not at SMIC through shell companies. We also think they are stockpiling this equipment.

We believe Huawei has also been able to manufacture their datacenter CPU at TSMC. This showcases that current controls aimed to limit Huawei’s access to TSMC are insufficient. The Ascend AI ASIC is 7nm, but because of poor screening, Huawei has been able to procure technology from the more advanced TSMC 5nm node.

By producing these chips at TSMC, this frees up capacity at SMIC, meaning more allocation can be given to the Ascend or Cambricon programs (HBM permitting). This, as mentioned earlier in the report, is a critical variable. If SMIC had to make the networking equipment, this would squeeze the wafer allocations towards Ascend significantly.

There is all the incentive in the world for China to offload as much production as possible to TSMC. They have used shell companies before and there is ample evidence they are still doing it. While some ideas around KYC and red flags have been implemented, we believe that existing frameworks are insufficient.

Nvidia and Huawei: Blackwell and the H20

The US government previously banned the Nvidia H20 chip from going into China, but recently indicated that Nvidia will be granted a license to be able to export the chip. As we have previously noted, the H20 is a capable chip with more memory than the banned the H100, though it has less much fewer FLOPs. Having more memory is helpful for inference performance.

Nvidia is poised to sell its existing inventory of the H20 and H20E (a variant with even more memory) at a minimum. This will amount to more than several hundreds of thousands of chips and billions of dollars of revenue.

There have also been reports of a more advanced version based on the Blackwell series. This is slated to have just as much memory as the H20 but will also contain significantly more FLOPs. Specifically, the B30A could have more than 10x the FLOPs than the H20, significantly above the export control threshold. The B30A could be positioned at half the price and have half the performance of the B300. The solution to this is rather simple: just buy 2x the chips for equivalent performance. Given how strong and weak scaling work in machine learning workloads, this is a valid path to remain competitve.

The B30A is argued as needed to be shipped to China due to decreased interest from China in the H20. A lot of this is due to government pressure of China’s major tech firms. While there is significant messaging that China has decreased interest in the H20, we do not believe this to be true.

Specifically, we do not believe this is reflective of China’s demand for compute or foreign chips. It is an incorrect policy being pushed from the top down which will be reversed as soon as Huawei runs out of HBM to produce more Ascend chips, or it could even be orchestrated brinkmanship to get approval to a more powerful chip.

This is on the tail of other news, like DeepSeek unable to get acceptable performance on Huawei’s chips. That is no coincidence. Export controls are working and the biggest blocker for more progress is compute.

China may be trying to psyop their way into getting the significantly more performant Blackwell chip. While the H20 and the H20E have much, much better software than the 910C, it is arguable that they are in the same league. However, it is unquestionable that the B30A would be in a league of its own. Given that the H20, H20E, and especially the B30A are good at inference, this also allows Chinese companies more compute to serve their models and applications.

It will allow them to export Chinese AI on hardware they own. This will increase the proliferation of Chinese AI and Chinese apps, inevitably taking market share away from American applications. The American AI stack is American AI on American chips, not just the chips.

The US needs to balance having China on the US AI stack and slowing the development of their own, while also limiting the quality and volumes that are shipped into China.

The H20 is allowed into China, but the progression of shipping more powerful chips into China should be closely guarded and watched considering China’s domestic capabilities.

The US should only move the quality of a chip shipped to China when it becomes clear China can ship something competitive to the H20E in significant volumes.

Regardless of the exact SKU, the export licenses have several implications discussed below.

Implications of Compute Diplomacy

First, it means that there will be more chips for top players like DeepSeek and Alibaba to use for Reinforcement Learning (RL). RL is a big driver of progress right now. The majority of RL compute is inference, which the H20 and especially the H20E, are good at. The B30A will have even better performance.

Players like DeepSeek had enough compute to keep improving – R1 received a major update with performance increases on a timeline that is like o1 -> o3. We firmly believe GPT-5, Claude 4, and Grok 4 are considerably ahead of R1 though, especially in agentic tasks.

However, problems in AI are solvable by two things: talent and compute. DeepSeek has always had the former, now they will have more of the latter. We expect DeepSeek’s rate of progress to materially increase as they receive substantial volumes of any chip that gets into China. DeepSeek has ambitions to release a multimodal model in V4, but scarce compute is slowing progress.

Alibaba (Qwen) and Moonshot (Kimi K2), have also yet to produce a large multimodal model, primarily focusing on text only due to compute limitations. They will release multimodal models this year, but they are still squarely behind OpenAI, Anthropic, and Google in many capabilties. If Blackwell GPU shipments come too early, that will accelerate China’s pace.

The second major implication of this is that China will now have more compute to serve and inference models to its population. Compute constraints affect user experience, with DeepSeek deliberately serving R1 at low speeds to users to preserve compute.

Poor user experience from the lack of compute has greatly limited China’s ability to deploy AI. As user experience improves, so will adoption, and the economic benefits from AI. It is easy see a future in which models are pre-trained via H800s, post-trained on H20s, then served to the population on Ascends and H20s. There is so much demand for AI powered products that should more capacity to serve these models exist, it would immediately be satisfied. One of the reasons Chinese models are open sourced is that it allows for other people to serve their models for them. With more capacity, models can be closed sourced and dependence on American providers cut off.

Claims that models trained on one chip must be inferenced on the same type are entirely false. Anthropic did different stages of research and training for Claude 4 on GPUs and TPUs. Additionally, Claude 4 inference is offered on Nvidia GPUs, Google TPUs, and Amazon Trainiums. This complexity is all while Anthropic is also the fastest growing AI company in terms of revenue with by far the most capable model for software engineering.

DeepSeek, Alibaba, Moonshot, etc primarily train their models on Nvidia chips and we do not think this will change anytime soon. If the Blackwell version ships, gains across all levels will be more pronounced.

China’s compute at a snapshot

With the H20 approved, this represents a material increase in the number of FLOPs and memory that China has access to. The 910C will be the first meaningful effort of indigenous production in China in terms of realized FLOPs and memory. We also expect some re-exportation into China by various bad actors, constituting meaningful volumes of H100 and low volumes of B200.

Should the Blackwell version ship, gains around FLOPs will be even more pronounced as the chip has significantly more FLOPs. China will end up having more FLOPs in that scenario, in addition to more memory.

This is not to mention that Chinese players, including major ones like ByteDance, can still access compute through renting in non-restricted countries. For example, ByteDance can still access top of the line Blackwell GPUs from providers like Oracle and Google. We have previously covered this relationship in detail. Malaysia, in no uncertain terms, has become a huge market for Nvidia. Export controls were also effective in ensuring that it remains that way, with Malaysia immediately distancing themselves from any effort around the Huawei Ascend.

Renting enables a path for Chinese dependence on Nvidia without Nvidia chips being in China, which is possible because the chips do not need to be in China. Bytedance’s Seed models were trained in the US on a US cloud.

Inspecting GPUs is much easier in Malaysia than in China. As such, it is an effective way to limit any unauthorized re-exporting by bad actors, as we believe methods like location tracking are technically intractable and easy to bypass. Note that renting will still allow China access to the capability of training advances models. If allowed to be used for serving the models, this allows China the chance to proliferate their AI through new and improved AI powered applications, taking market share away from American products. The critical difference, though, is that access can be cut off.

With that said, China will not allow their data to be shipped out of China in large volumes hence there is still significant need for domestic AI capabilities. Selling chips into China does not change China’s drive for total silicon self sovereignty and will only serve to help provide a buffer while local production meets demand.

It is worth remembering that China’s focus on silicon self sovereignty predates US export controls. The argument Blackwell needs to be sold into China is a false narrative as Huawei will soon run out of HBM. Selling chips into China only served to bail them out until domestic production ramps. The production ramp will not slow until self sufficiency is achieved. Shipments of Blackwell must be weighed with a close eye on production of domestic capabilties. If Huawei, Cambricon, and CXMT accelerate production faster than expected, then the US should raise the bar sooner.

Next, we dive into Nvidia’s future in China. This includes expected revenue from the H20 and newer China specific chips on the horizon. We also compare how much compute China has relative to the US how the is absolutely dominating China when it comes to both memory and FLOPs.

Two-and-a-half years ago, we flagged a looming “cloud crisis” at AWS. Today, the evidence has mounted. AWS is the crown jewel of the Amazon empire, generating ~60% of group profits, and dominating the lucrative Cloud Computing market. But it struggles to translate this strength into the new GPU/XPU Cloud era.

Microsoft Azure now leads the market on quarterly new cloud revenue, and the gap between Google Cloud and AWS has materially narrowed especially with Google’s big moves on the TPU that we’ve been posting about for over a month. Markets have noticed. Year-to-date, Amazon is the clear laggard among the four tech-and-AI titans as investors mark down the company most for losing momentum in AI.

Today, SemiAnalysis is back with another out-of-consensus call. While the market overplays the Cloud Crisis theme, we call for an AWS AI Resurgence. We laid out our thesis a month ago to our Core Research subscribers, forecasting an upcoming acceleration beyond 20% year-over-year growth by the end of 2025.

Amazon’s savior has a name: Anthropic. The startup has been the clear outperformer in the GenAI market in 2025, multiplying revenue fivefold year-to-date to reach $5B annualized.

To keep that trajectory, Anthropic is betting hard on Scaling Laws. While Dario’s startup draws fewer headlines than OpenAI, xAI and Meta Superintelligence, it isn’t shy about investment. AWS has well over a gigawatt of datacenter capacity in final stages of construction for its anchor customer. AWS is building datacenters faster than it ever has in its entire history. And there’s much more on the horizon.

To understand and forecast GPU/XPU power capacity by AI Lab broken down by Cloud Provider, we rely on our proprietary Datacenter Industry Model powered by real-time satellite imagery. Trusted by all hyperscalers, AI labs, and the world’s largest investors, it provides a quarterly building-by-building datacenter forecast for OpenAI, Anthropic, xAI, Meta Superintelligence, Google DeepMind, and more. Contact us for more information.

Trainium vs GPUs

While Amazon’s AI Datacenters are impressive in scale and speed, the design of individual building is unremarkable. Hyper-optimized for air-cooling, this blueprint is identical to 5-year-old traditional AWS Cloud datacenters.

What makes these facilities unique is their inside: they’ll host the world’s largest cluster of non-Nvidia AI chips, with just under a million Trainium2 in the largest campus. To understand everything about the Trainium2 system, read our December 2024 technical deep dive.

Trainium2 lags Nvidia’s systems in many ways, but it was pivotal to the multi-gigawatt AWS/Anthropic deal. Its memory bandwidth per TCO advantage perfectly fits into Anthropic’s aggressive Reinforcement Learning roadmap. Dario Amodei’s startup was heavily involved in the design process, and its influence on the Trainium roadmap only grows from here.

Put plainly: Trainium2 is converging toward an Anthropic custom-silicon program. This will enable Anthropic to be, alongside Google DeepMind, the only AI labs benefiting from tight hardware–software co-design in the near horizon.

This report will dig into all aspects of Amazon’s AI resurgence: the Anthropic partnership, datacenters, and Trainium. At the end of the report, we provide a longer-term outlook on Anthropic, AWS Bedrock and internal models, and explain why everything isn’t rosy.

First, a step back on why AWS has underperformed rival AI Clouds to date.

AWS GenAI underperformance

To understand the causes of Amazon’s underperformance in the GenAI era, we can analyze drivers of success in the GPU/XPU cloud market. In the most simplistic way, we see two primary customer groups for GPU/XPU capacity:

• Wholesale bare metal users: large-scale customers like OpenAI, Anthropic, ByteDance, and other hyperscalers.
• Managed SLURM/Kubernetes: Smaller customers such as startups, research institutes, and enterprise pilot projects.

Cloud Crisis and ClusterMAX underperformance

In the second category, our ClusterMax AI cloud rating is the best way to compare relative strengths and weaknesses. Platinum and gold-rated AI Clouds have seen more traction than others and boast higher-than-average pricing power. As such, the likes of CoreWeave, Oracle, Nebius, Crusoe and Azure have outperformed the market for multitenant GPU clusters – which require high performance and advanced software layers.

As predicted two years ago, key to Amazon’s underperformance is the use of custom networking fabric EFA. AWS’s success with ENA on the frontend network has not yet translated to EFA on the backend. EFA still lags behind other networking options on performance: NVIDIA’s InfiniBand and Spectrum-X, as well as RoCEv2 options from Cisco, Arista, and Juniper. Raw performance isnt the only metric, the user experience of EFA isn’t as good as InfiniBand & RoCEv2 either. That being said, with Amazon’s newest EFAv4 performance at real world msg sizes is improving, albeit still behind the competiton.

Amazon’s custom networking also reduces their time-to-market due to customization requirements of Nvidia systems. Other items like advanced passive and active automated weekly scheduled health check strategies aren’t as solid as gold & platinum-rated clouds.

Our upcoming ClusterMAXv2 rating will provide an update on all major cloud providers based on our proprietary testing. Stay tuned!

Searching for an anchor customer

More important to AWS’ XPU business growth is the ability to secure anchor customers – the market-makers in this first wave of GenAI demand. Scale, time-to-market, deep partnerships, and pricing are key to winning these accounts, more so than advanced software layers.

No firm better illustrates this than Microsoft. Azure’s AI outperformance over peers is entirely driven by its OpenAI partnership. As of Q2 2025 (June 2025), all of OpenAI’s >$10B cloud spending is booked by Azure.

Amazon understood early on the need for an anchor customer and invested $1.25B, expandable to $4B in Anthropic in September 2023. The partnership expanded in March 2024 with Anthropic committing to use Tranium and Inferentia chips. In November 2024, Amazon invested an additional $4B into Anthropic, with the latter naming AWS as its primary LLM training partner.

Anthropic’s outperformance, AWS underperformance?

Amazon’s bet has been the right one. Anthropic is the clear outperformer in 2025 in the GenAI market, with revenue surging from $1B to $5B annualized. In this context, AWS’ underperformance understandably frustrates investors, but they’re misunderstanding the composition of Anthropic’s spending on training and inference.

There are two clear reasons explaining why Amazon isn’t yet truly benefiting from its relationship with Anthropic:

1. As of Q2 2025, Anthropic’s cloud spending is over 2x smaller than that of OpenAI.
2. A large share of Anthropic’s spending is going to Google Cloud – one of Anthropic’s first major investors ($300M round late-2022) and preferred cloud partner in 2023 and 2024, before the expanded AWS deal.

Anthropic & AWS multi-gigawatt AI Training infrastructure

In particular, we believe that most of Anthropic’s skyrocketing inference needs are served by Google Cloud. Having the world’s best inference system (TPU) is a key competitive advantage.

The AWS infrastructure buildout is aimed at taking a chunk of this for its key customer will also focusing on training. While Anthropic makes less headlines than peers like OpenAI, xAI and Meta, is it all-in on the AGI race and isn’t planning to be shy on training spending. Anthropic leadership truly believes in Scaling for RL.

Their belief will materialize as early as this year. We show below three AWS campuses in final stages of construction, boasting over 1.3GW of IT capacity for the sole purpose of serving Anthropic’s training needs. The speed of construction is remarkable.

While these datacenters look built from the skies, we don’t think they are generating any meaningful revenue yet. Trainium has faced some yield issues on the assembly phase – fairly standard for a new system. We think the three large AWS campuses will meaningfully contribute to AWS’ top line by the end of 2025 and jack up growth above the 20% YoY threshold.

Anthropic isn’t stopping there. Its ~$13B funding round at a $183B valuation will provide capital to sign additional deals with AWS, Google, and others. AWS isn’t waiting standing still – they’re already breaking ground on upcoming GW-scale datacenters to capture this growth.

As explained earlier, these datacenters will primarily be filled with AWS’ custom chip Trainium. Given the sheer scale, we can’t understate how bold Anthropic’s bet is. Not only are they committing to spending tens of billions of dollars, they’re doing it on a largely unproven chip!

Let’s try to make some sense of their bet by digging into Trainium’s TCO and roadmap.

Trainium2 TCO analysis – how Anthropic’s big bet could pay off

Trainium2 supply chain signals are currently extremely strong. Our industry-leading AI Accelerator Model track both the package shipments and the system/rack shipments and they’ve surged since the beginning of the year. It provides quarterly volume forecasts for the 10+ SKUs comprising the Trainium2 and Trainium3 product family and calls out suppliers set to disproportionately benefit from specific SKUs. Contact us for more information.

Note this is chip production, rack production is lagged, but we also track it.

Competing with Nvidia and Google’s TPU is, of course, no small feat. While Google is rolling out its seventh generation TPU, Ironwood, Trainim2 is only Amazon’s third-generation AI Accelerator.

Chip specifications: Trainium2 inferior on all fronts, but…

A simple look at chip specifications shows Trainium as a clear laggard relative to Nvidia:

• Nvidia’s GB200 has a 3.85x FP16 FLOPs advantage, at 2500 TFLOP/s/chip vs Trainium2’s 667 TFLOP/s/chip. Note that spec sheet numbers are inflated compared to actual achievable FLOPs.
• In terms of Memory bandwidth, the gap narrows to 2.75x, at 8000GB/s/GPU vs 2900GB/s/Trn2

Evaluating scale-up network bandwidth is another key item. We’ve explained several times the importance of scale-up networks for reasoning model inference. Our deep-dive on Reinforcement Learning highlighted the similarities of RL with inference workloads, making memory bandwidth a crucial item to scale post-training.

• Nvidia’s GB200 NVL72 boasts an aggregate 576TB/s memory bandwidth across World Size.
• This is a 3.1x advantage relative to Trainium2’s (Teton2-PD-Ultra-3L SKU) 186 TB/s – with the caveat that it varies across SKUs.

While Trainium appears materially behind, the picture changes once we factor-in Total Cost of Ownership.

Trainium’s memory bandwidth per TCO advantage

In the table below, we incorporate TCO into our comparison. While Nvidia has a material head on a TCO per effective training PFLOP, Trainium2 is highly competitive on a TCO per million Tokens and TCO per TB/s of memory bandwidth.

And we don’t see Nvidia’s upcoming VR200 NVL144 change the picture relative to AWS’ Trainium3. To be clear, TCO has many other moving parts. AWS has other system-level architecture deployments that better fit some use cases. Later down the road, Nvidia’s Kyber rack will boast the world’s most advanced scale-up network architecture.

For a full understanding of the TCO of 50+ Nvidia SKU and a detailed TCO comparison with all AMD, Trainium and TPU SKUs, check out our AI Cloud TCO Model. The largest hyperscalers, Neoclouds and their financial sponsors rely on our model to time their investment decisions.

Anthropic is betting on hardware-software codesign

Trainium2’s memory bandwidth per TCO advantage is key to understanding Anthropic’s choice. While Nvidia’s chips and systems are better on most fronts, Trainium2 fits perfectly into Anthropic’s roadmap. They’re the most aggressive AI Lab on scaling post-training techniques like Reinforcement Learning. Their roadmap is more memory-bandwidth-bound than FLOPs bound. Our recent HBM report explains in-depth which AI workloads tend to be memory-bound.

Anthropic’s ramp will make it not only the only large external end-user of Trainium2, it’ll also be materially larger than Amazon’s internal needs (e.g. Bedrock, Alexa, etc). They’re now heavily involved in all Trainium design decisions and, essentially, use Amazon’s Annapurna Labs as a custom silicon partner! This makes Anthropic the only AI lab, alongside Google DeepMind, benefiting from tight hardware-software codesign.

Trainium’s roadmap: doubling down on systems

Amazon is rolling out a new system-level architecture for its anchor customer. Currently, the two systems deployed by AWS are Teton PD and Teton PD Ultra. Next year, the new Teton PDS and Teton Max are set to ship in large volumes. Our AI Accelerator Model provides the exact volumes and SKU-by-SKU breakdown on a quarterly basis.

The key difference is the introduction of an all-to-all scale-up network dubbed NeuronLinkv3. Trainium’s architecture is thus converging towards Nvidia’s NVL72 NVLink.

Four NeuronLinkv3 switch trays will be placed in the middle of the rack with 16 compute trays above and below split evenly. Certain supply chain vendors are set to benefit disproportionately, as highlighted two months ago on Core Research – our institutional research service trusted by the world’s largest hedge funds. That vendor is up 73% since our post. We see the introduction of PDS as an intermediate step in Trainium’s path to catch up with Nvidia. We also believe that Anthropic was heavily involved in the launch of this new system-level architecture.

Anthropic’s increased involvement in design decisions bodes well for future volumes. But they’re not giving up on TPUs and Nvidia GPUs either. Our Accelerator Model forecasts Amazon and Google Cloud’s chip purchases broken down by precise SKU, and our Datacenter model to understand which datacenter and cloud partners support Anthropic’s ramp. The TPU ramp for Anthropic in 2026 is huge, and their are unique aspects to their deal as we have been posting about for over a month.

Let’s now take a longer-term view and evaluate what the future of AWS might look like. Behind paywall, we discuss the following items:

• The outlook for Amazon’s key customer: Anthropic.
• Amazon’s GenAI business beyond Anthropic: Bedrock and internal LLM efforts.
• The Trainium ramp in 2026 & 2027, potential new external customers, and how it might impact Amazon’s financial profile in future years.

To many power users (Pro and Plus), GPT5 was a disappointing release. But with closer inspection, the real release is focused on the vast majority of ChatGPT’s users, which is the 700m+ free userbase that is growing rapidly. Power users should be disappointed; this release wasn’t for them. The real consumer opportunity for OpenAI lies with the largest user base, and getting the unmonetized users who currently use ChatGPT infrequently in their day-to-day to indirectly pay is their largest opportunity.

Analysts focused on the model capabilities are missing the much larger context of network effects that ChatGPT is gaining quickly. In November of 2023, ChatGPT wasn’t even in the top 100 websites; now it is number 5. Larger than X/Twitter, Reddit, Whatsapp, Wikipedia, and quickly approaching Instagram, Facebook, Youtube and Google. On the top 10 list, every single property is much older than ChatGPT.com, and the sheer number of unmonetized users is staggering.

But that changes with GPT-5. OpenAI is laying the groundwork to monetize one of the largest and fastest-growing web properties in the world, and it all begins with the router.

The Router is the Release

On OpenAI’s release website, the second paragraph is about the “One United System,” specifically focused on the router. The wording is instructive.

GPT‑5 is a unified system with a smart, efficient model that answers most questions, a deeper reasoning model (GPT‑5 thinking) for harder problems, and a real‑time router that quickly decides which to use based on conversation type, complexity, tool needs, and your explicit intent (for example, if you say “think hard about this” in the prompt). The router is continuously trained on real signals, including when users switch models, preference rates for responses, and measured correctness, improving over time. Once usage limits are reached, a mini version of each model handles remaining queries. In the near future, we plan to integrate these capabilities into a single model.

The router serves multiple purposes on both the cost and performance side. On the cost side, routing users to mini versions of each model allows OpenAI to service users with lower costs. On the performance side, it will enable many users to use thinking aka CoT (Chain of Thought) reasoning for the first time. Over 99% of the free users have yet to interact with a thinking model like o3, and for the average user, ChatGPT just got a huge upgrade. The number of free users exposed to thinking models went up 7x in the first day and the number of paying users up nearly 3.5x.

But the router is clearly a feature to the new service, and can likely see improvements or changes over time. It will continuously learn on preference rates and OpenAI promises it will improve over time. It just takes a single additional attribute to begin the path to monetization: the commercial value of the query.  

We believe that the Router is the groundwork for the next leg of ChatGPT’s story, and that’s monetization of free users.

The Router is Preparing for Free Monetization, Sam’s Subtle Tone Shift

Centralizing the control of the free user experience allows for many more future monetization paths. And this monetization path is one that has been hinted at subtly for a while. It all starts with OpenAI’s decision to hire Fidji Simo as CEO of Applications in May. Let’s look at her background because it’s telling.

Fidji was at Ebay from 2007 to 2011, but her defining career was primarily at Facebook. She was Vice President and Head of Facebook, and she is known for having a superpower to monetize. She was critical in rolling out videos that autoplay, improving the Facebook feed, and monetizing mobile and gaming. She might be one of the most qualified individuals alive to turn high-intent internet properties into ad products, and now she’s at the fastest-growing internet property of the last decade that is unmonetized. It’s an obvious story.

What’s more is Sam Altman himself has had a very direct tone shift in the last year.

“I will disclose as a personal bias I hate ads. I think ads were important to give the early internet a business model. I’m not totally against them, but ads plus AI are uniquely unsettling to me. I kind of think of ads as a last resort as a business model.”

But in recent interviews his tone has now shifted. There is clearly a lot of thought happening about how to best monetize free users more recently. This likely is in conjunction with Simo’s hiring.

“I am not totally against it… if you compare us to social media or web search where you can kinda tell that you are being monetized… we would hate to ever modify anything in the stream of an LLM… maybe if you click on something in there that is going to be there we’d show anyway, we’ll get a bit of transaction revenue and it’s a flat thing for everything, maybe that could work. It’s clearly possible to be a good ad driven company but there are obviously issues to it.

Compared to the previous conversation, when he was dismissive, his most recent thoughts clearly shows that Sam Altman is thinking about monetizing free users. He mentions a take rate a potential affiliate model. And as a response, the interviewer (Andrew Mayne of OpenAI) literally says, “I would love to do all my purchasing through ChatGPT because often times I feel like I am not making the most informed decisions,” as a response in the conversation. This is likely the direction OpenAI is taking.

The Router release can now understand the intent of the user’s queries, and importantly, can decide how to respond. It only takes one additional step to decide whether the query is economically monetizable or not. Today we will make our case for how ChatGPT’s monetized free end state could look like an Agentic super-app for the consumer. This is only possible because of routing.

We believe that display ads are unlikely. Perplexity has tried this, and it doesn’t seem to be going that well. Instead of inserting a paid feature into the query, we believe it’s more likely that they will pursue a take-rate based model.

Now let’s discuss, because it looks like an Agentic Assistant is a way that could align Sam Altman’s vision of AI being helpful, as well as monetizing via a transaction take rate.

Agentic Advice and Purchasing, Supply and Demand Dynamics

Let’s talk about Agentic purchasing and compare it to a search query today. Because LLMs have a core feature that Search does not, and that is scaling marginal costs. This is fundamentally different than the world search grew up in. Let’s examine “Aggregation Theory” work of Ben Thompson, because the core feature was that most technology companies had zero marginal costs to an additional user. There were some fixed overheads for running the largest search engine, but the incremental cost of another query was virtually zero. Agents and LLMs kill this concept.

For the first time, the more you spend the better your result is because of CoT reasoning tokens and now marginal costs exist in software again. There is a somewhat direct relationship between more money, more compute, and a better answer. Nowhere is this clearer than in AI, in which you can spend variable costs to get a variably better answer or outcome.

So let’s apply marginal costs to a new purchasing experience. Let’s compare two queries, an information query and a commercial query.

1. A trivial information query like “Why is the sky blue?”
2. A highly commercial query, “What is the best DUI lawyer near me”

Before the router, there was no way for a query to be distinguished, and after the router, the first low-value query could be routed to a GPT5 mini model that can answer with 0 tool calls and no reasoning. This likely means serving this user is approaching the cost of a search query.

The search query on the other hand has a fixed cost. It would show a page ranking of websites, with a potential AI summary at the top. This is a fixed supply response to what could be a variably hard question. But now ChatGPT free (because of routing) can dynamically answer a harder question with a better answer, which is not how search is designed today. Below shows the value of a changing supply query to a harder question, the supply is fixed for Search.

So now let’s bring in the higher value query, the DUI Lawyer question. As you may know, this is an extremely valuable question. Today on search, this is one of the higher cost per click keywords, and it is plastered with ads. In a world of dynamic supply, ChatGPT can not only answer this question, it could realize this is a very valuable question and answer this question at the level of a human. It could throw $50 dollars of compute if there is a belief of high conversion, because that transaction is worth $1000s of dollars.

The router makes this possible. ChatGPT 5 could decide to allocate $50 to the query, create a plan, gather information about the incident, research local lawyers, consider who is likely to answer fastest, consider your budget, and then contact multiple lawyers on your behalf. It could even Agentically reach out to lawyers on behalf of the free user knowing that the conversion ratio of this query is even higher. This version of ChatGPT is highly helpful, aligns with the user’s query, and is a valuable referral to the seller of the services or goods. This wouldn’t be intrusive in the current format, and many free users would use this instantly.

This would apply to more than just services. Products that are highly likely to be purchased with agentic purchase would pay for referral fees, such as groceries, ecommerce purchases, flights and hotels. This could be a consumer SuperApp that would be an “agent” for day to day planning, purchases, and basic services.

The user wouldn’t pay via a cost of a subscription, but by transaction fees or ad take rates on purchase. The AI agent could still make the best response output but drive extremely high value business to a company almost instantly, in which the business would be willing to pay take rates on.

And what’s more you can see the glimpses of this in their model release notes. They highlighted Gmail and Google Calendar integration, as well as new tool use benchmarks for services such as Telecom, Retail, and Airlines. Imagine a world where very little customer acquisition cost is spent on ads, but rather asking a helpful AI assistant to set up the best internet plan in your neighborhood.

And while this feels like speculation, this is already happening. Instacart added this feature to let agents checkout products in January of this year. Fidji Simo was at Instacart when this was implemented, and now she just joined OpenAI as head of product. The wheels are already in motion, this will be the future of free usage at ChatGPT.

AI labs such as Anthropic and OpenAI are even paying startups hundreds of thousands of dollars to spin up replicas of popular sites like DoorDash and Amazon to RL agents on successfully completing end-to-end transactions. It is not a question of if, but when this capability happens.

You can see the future. Imagine a world where you ask for new dinner recipes for the week, and ChatGPT gives you multiple options and orders the cart for you to check out. The fee would be paid on the completion of the purchase, and Search is completely cut out of the picture. Every thing that can be researched or planned in an AI app could be purchased for companies that adopt partnerships quickly.

Companies would flock quickly to this large new purchasing habit. Booking flights, purchasing items, buying food, etc. If there is a connection to a website and payment information, this could all be monetized for “free” use of the application but at transaction.

And while we are far from this future, the router is the necessary step to begin the sorting of high and low compute, and eventually commercial intent queries. None of this is possible without a single unified interface routing dynamic responses to users.

This would fulfill a vision for non-intrusive ads by Sam Altman, as well as continue to enable ChatGPT to be a trusted advisor to users for free. ChatGPT will become an agent to help users make one of the most important decisions in their day-to-day lives: buying stuff. OpenAI and Shopify are already working on a checkout integration today.

From Today to Agentic Purchasing End State

It’s no secret that we are very far from that future today. It would likely take a few steps before a true product launch, but it starts with a Router and likely many partnerships on the other side as connectors. Maybe in the beginning, ChatGPT gets affiliate fees for the currently recommended items that lead to a purchase. This would look like a typical affiliate marketing deal, and would have hard-to-measure success rates and lower take rates.

As the model becomes more agentic, it would likely need plug in to the service’s system to be able to make reservations, book flights, or schedule appointments. It would likely need heavy partnership.

Let’s examine some of the partnerships they have already made today. So far OpenAI has partnered with:

Finance Companies: Stripe, Visa, PayPal

Consumer Companies: Mattel, Booking.com, Lowe’s

Enterprise Software: Salesforce, Intercom, Zendesk

Consumer Internet: Snapchat, Shopify, Instacart, Mercari

Every company that can and will shift to a cheaper customer acquisition cost will be excited to move sooner. There will be low overhead, as less customer service, advertising, marketing and other functions will be needed as ChatGPT collapses that entire purchasing funnel into a helpful assistant.

OpenAI is Creating a Consumer SuperApp

OpenAI has clearly crossed the chasm with its web-scale presence. And every single large internet-scale property with this many users has been monetized for “free” by ads. Agentic purchasing with take rates can begin now after the implementation of the router.

We now turn our focus onto the hyperscalers, who are facing increasing competition in the consumer space, and which smaller companies are already benefiting from the shift of monetizable queries away from search to AI.

OpenAI is firmly knocking on the door of technology giants Google and Meta and even Amazon. Previous scares about AI have been focused on search query volume, not being replaced in the ad tech stack. ChatGPT can compete with dominant platforms for its place in the ecosystem, and to date this is push into purchasing is the most concrete example of OpenAI coming for advertising at large. If they were to first launch an aggressive Agentic checkout solution before Meta or Google, this would be seen as huge competitive shots at both companies.

A reminder that if we are talking about pure usage, only one company is growing users at a meaningful rate. It’s OpenAI.

In some ways by completely bypassing the top of funnel of search or pushing ads, it creates a third space for purchasing. Social media time continues to climb as human-to-human interaction is still the primary time consumption, so the research portion of consumer journeys would more heavily impact Google, but there is opportunity for smaller players. Certain smaller players are actually winning here as shown by the data below

Meta’s shocking purchase of 49% of Scale AI at a ~$30B valuation shows that money is of no concern for the $100B annual cashflow ad machine. Despite seemingly unlimited resources, Meta has been falling behind foundation labs in model performance.

The real wake-up call came when Meta lost its lead in open-weight models to DeepSeek. That stirred the sleeping giant. Now in full Founder Mode, Mark Zuckerberg is personally leading Meta’s charge, identifying Meta’s two core shortcomings: Talent and Compute. As one of the last founders still running a tech behemoth, Mark doesn’t need SemiAnalysis to tell him to slow down stock buybacks to fund the future!

In addition to throwing money at the problem, he’s fundamentally rethinking Meta’s approach to GenAI. He’s starting a new “Superintelligence” team from scratch and personally poaching top AI talent with pay that makes top athlete pay look like chump change. The typical offer for the folks being poached for this team is $200 million over 4 years. That is 100x that of their peers. Furthermore, there have been some billion dollar offers that were not accepted by researcher/engineering leadership at OpenAI. While these offers aren’t all successful, Zuck is crushing the competitors by drastically increasing their cost per employee.

Perhaps even more iconic, Zuck threw his entire Datacenter playbook into the trash and is now building multi-billion-dollar GPU clusters in “Tents”!

As this report details, nothing is off the table. We unpack Meta’s unprecedented reinvention from Compute to Talent in the pursuit of Superintelligence as well as the story of how we got here. From Llama 3.0 open-sourced dominance to the epic fail of Llama 4 Behemoth, this Titan of AI is down but not out. In fact, we believe Meta’s ramp in training FLOPS will rival even that of OAI. The company is going from GPU-poor to GPU-filthy-rich on a per researcher basis.

Meta GenAI 1.0: AI Incrementalism

Compared to pure-play AI labs like OpenAI, companies like Meta and Google have followed an “AI Incrementalism” strategy by enhancing existing products with better recommendation systems and GenAI to improve ad targeting, content tagging, and internal tools. This has paid off handsomely in financial results, allowing Meta to shrug off Apple’s attempts at stopping them from tracking users with the release of their App Tracking Transparency (ATT) feature in iOS 14.5 (late 2021, early 2022).

While Meta is arguably more insulated from GenAI disruption than Google, LLM efforts at both companies have been somewhat disappointing. One reason is the allocation of capital primarily for the core business rather than pursuing superintelligence.

“Our CapEx growth this year is going toward both generative AI and core business needs with the majority of overall CapEx supporting the core.”

Source: Meta Q1 2025 earnings call, emphasis SemiAnalysis

GenAI acts as an extension to these tech giants’ businesses. They don’t have the same existential need to dominate new use cases like OpenAI in chatbots and Anthropic in coding APIs. This is readily apparent in the compute vs human capital allocation of a leading AI foundation lab like OpenAI compared to Meta. Zuck’s impact on competing offers and inflation of salary will hurt badly.

As a result, when measuring GenAI consumer app traction, Meta and Google meaningfully lag ChatGPT in its reach and engagement.

But that’s changing. Leveraging our proprietary Accelerator Industry Model and Datacenter Industry Model, we forecast a meaningful step up in GenAI investment from Meta in the coming years.

Meta GenAI 2.0 – Part 1, Re-Inventing the Datacenter Strategy (Again)

From Buildings to Tents

Just a year ago, Meta scrapped its decade-old “H”-shaped datacenter blueprint for a new AI-optimized design.

Now in 2025, Zuck decided to re-invent the strategy again. Inspired by xAI’s unprecedented time-to-market, Meta is embracing a datacenter design that prioritizes speed above all else. They’re already building more of them! Traditional datacenter and real estate investors, still somewhat reeling from xAI’s Memphis site and time to market, will be shocked yet again.

This design isn’t about beauty or redundancy. It’s about getting compute online fast! From prefabricated power and cooling modules to ultra-light structures, speed is of the essence as there is no backup generation (ie no diesel generators in sight).

Power currently uses nearby Meta on-site substations. Meta likely uses sophisticated workload management to maximize the utilization of every watt of power it gets from the grid. It might even need to shut down workloads in the hottest summer days.

The Prometheus 1GW AI Training Cluster – An “All Of The Above” Infrastructure Strategy

Meta is quietly building one of the world’s largest AI training clusters in Ohio. We have heard from sources within their infrastructure organization they are calling this cluster Prometheus. To beat rival AI labs, Meta has carried out an “all the above” infrastructure strategy:

• Self-build campus.
• Leasing from third parties.
• AI-optimized designs.
• Multi-datacenter-campus training.
• On-site, behind-the-meter natural gas generation.

Looking at the Prometheus training cluster below, we believe Meta is connecting all these sites with ultra-high-bandwidth networks all on one backend network powered by Arista 7808 Switches with Broadcom Jericho and Ramon ASICs.

By combining both self-build and leasing, Meta ramps faster. In fact they pre-leased more capacity second half of 2024 than any hyperscaler, mostly in Ohio.

Furthermore, when the local power grid couldn’t keep up, Meta went full Elon mode. With help from Williams, they’re building two 200MW on-site natural gas plants. The equipment breakdown for the first includes:

• 3* Solar Turbines’ Titan 250 turbines
• 9* PGM 130 turbines
• 3* Siemens Energy SGT400 turbines
• 15* CAT 3520 Reciprocating Engines

A future SemiAnalysis report will dig much deeper into these systems for the purpose of powering datacenters: cost, advantages & disadvantages, etc. This has signficant negative implications for a certain supplier of onsite natural gas solutions.

This total compute advantage for OpenAI is important as the advent of reinforcement learning means many large datacenters distributed around the US can be used asynchronously to contribute to improving model intelligence with post-training.

Not to be one-upped, Hyperion—Meta’s second frontier cluster—aims to erase that gap to OpenAI.

Beating Stargate at Scale: Meta’s Hyperion 2GW Cluster

While all eyes are on the high-profile Stargate datacenter in Abilene, Meta has been planning a response for over a year and making tremendous progress. The Louisiana cluster is set to be the world’s largest individual campus by the end of 2027, with over 1.5GW of IT power in phase 1. Sources tell us this is internally named Hyperion.

Meta broke ground at the end of 2024 and is currently actively working on both the power infrastructure, and the datacenter campus.

To be clear, Meta has many other datacenters under construction and ramping. A full list of Meta’s AI Datacenters, their expected completion dates, and power per building by quarter can be found in our Datacenter Industry Model.

Llama 4 Failure – From Open-Source Prince to Behemoth Pauper

Before we dive into the Superintelligence Talent race, we should take a look at how Meta found itself in this awkward position. After leading the open-source frontier with Llama 3, Meta now finds itself trailing China’s DeepSeek.

On a technical level, we believe the major contributors to the failed run were as follows:

1. Chunked attention
2. Expert choice routing
3. Pretraining data quality
4. Scaling strategy and coordination

Chunked Attention

Naively implemented, attention in large language models scales quadratically with token count. To address this, researchers introduced memory-efficient mechanisms. Meta chose chunked attention for Behemoth, and that may have been a mistake.

Standard Causal Attention: Imagine a series of expanding triangles fanning out from the top left corner, representing the attention size for every subsequent token. Double the tokens and the area of the triangle quadruples.

Chunked Attention breaks this triangle into fixed-size blocks. Each block resets attention to a new “first” token. With the efficiency of reduced memory, it enables even longer context. Meta felt they needed this to achieve long context, but the tradeoffs aren’t worth it. The first token in each block lacks access to prior context. While there are some global attention layers, that’s not enough as we expand on below.

Sliding Window Attention, used in other models, provides a smoother alternative: the attention window slides forward token-by-token. This maintains local continuity, even if long-range reasoning still requires multiple layers to propagate context.

Behemoth’s implementation of chunked attention chasing efficiency created blind spots, especially at block boundaries. This impacts the model’s ability to develop reasoning abilities as chain of thought exceeds one chunk in length. The model struggles to reason across longer ranges. While this may seem obvious in hindsight, we believe part of the problem was that Meta didn’t even have the proper long context evaluations or testing infrastructure set up to determine that chunked attention would not work for developing a reasoning model. Meta is very far behind on RL and internal evals, but the new poached employees will help close the reasoning gap massively.

Expert Choice Routing

Most modern LLMs use a Mixture of Experts architecture in which, between each model layer, a the token is routed to different experts based on a router. In modern MoE models, most are trained with token choice routing, i.e. the router provides a tensor of shape T x E (T being total tokens, E being number of experts in the MoE model) and a topK softmax is run on the E dimension producing a T by K tensor. This effectively means that the router is asked to choose K most probable experts for each of the tokens T, where K can be one or more experts. K is a hyperparameter that can be tuned by the researcher.

The advantage of this approach is that each token is guaranteed to be attended to by K experts, making sure the information value of each token is absorbed by the same number experts. The disadvantage is that certain experts can be disproportionately ‘popular’ with tokens while other experts are under-trained, resulting in an imbalance in the ‘intelligence’ of each expert. A known problem, many of the top labs have developed a fix with auxiliary-loss (or loss-less) load balancing. When training with EP (Expert Parallelism), this can lead to lower training MFU, as the model is spread across different GPU nodes, resulting in many more collectives (NCCL) running across the scale-out network (InfiniBand or RoCE) instead of the scale-up network (NVLink). This is a major motivation for NVIDIA’s NVL72 design, where the scale-up network expands beyond standard 8-way servers.

Expert choice routing, introduced by Google in 2022, flips the logic: experts choose the top-N tokens. Taking the same T x E tensor produced by the router, in expert choice routing a topN softmax is run on the T dimension which produces a E by N tensor. This effectively means each of the E experts has chosen the N highest probability tokens to be routed. The N hyperparameter can be tuned by the researcher but to compare it to a token choice routing approach, N = K * T / E.

When compared directly to token choice routing, expert choice routing guarantees that experts are being activated in a balanced manner, avoiding the performance degradations associated with imbalanced experts. To be clear: the router makes the choice in both cases. In token choice, the input to the router is the tokens, and it picks the experts. In expert choice, the input to the router is the experts, and it picks the tokens.

This equalizes the load for expert training and improves MFU across distributed hardware. Hyperscaler networks are purpose-built for this kind of parallelism, and we cover them extensively in our Networking model.

The disadvantages of this approach is inverse of the token choice architecture. Expert choice routing can result in certain ‘popular’ tokens being attended to by multiple experts. While this doesn’t produce the same training bottleneck that is present in token choice routing, it can result in degraded model generalization as the LLM now does not pay attention to all tokens equally. Furthermore, most EC models need to be finished with TC training to produce an effective LLM.

Inference is also another issue for EC. Inference is split in two steps: Prefill and Decode. In the Prefill stage, the user prompt is encoded and loaded into the KVCache. This step is Flop-bound. In the Decode step, the model calculates the attention and runs through the feed forward network of the model one token at a time one layer at a time.

Here, expert choice routing struggles as the experts can only choose from to 1 token x batch size per layer initially, resulting in each expert only given a very small set of tokens compared to when it was trained (an example training run would have 8k seqlen x 16 batch size = 128k tokens per pass). Being able to see the entire sequence as in training would break causality of modern autoregressive models hence the mix of EC and TC even for EC models.

Meta switched from expert choice to token choice routing partway through the run which is not unusual for EC models. However the performance drop from the switch resulted in a model that was meaningfully worse than a model fully trained on TC.

Data Quality: A Self-Inflicted Wound

Llama 3 405B was trained on 15T tokens and we believe Llama 4 Behemoth required substantially more tokens, 3-4x order of magnitude larger. Getting sufficiently high-quality data is a major bottleneck that Western hyperscalers cannot shortcut by copying the homework of other models’ outputs.

Prior to Llama 4 Behemoth, Meta had been using public data (like Common Crawl), but switched mid-run to an internal web crawler they built. While this is generally superior, it also backfired. The team struggled to clean and deduplicate the new data stream. The processes hadn’t been stress-tested at scale.

Furthermore, unlike all other leading AI labs including OpenAI and Deepseek, Meta does not utilize YouTube data. YouTube lecture transcripts and other videos are an incredible source for data and the company may have struggled to produce a multimodal model without the data.

Scaling Experiments

Beyond the above technical issues, the Llama 4 team also struggled to scale research experiments into a full-fledged training run. There were competing research directions and a lack of leadership to decide which direction was the most productive path forward. Certain model architecture choices did not have proper ablations but were thrown into the model. This led to poorly managed scaling ladders.

As an example of how hard scaling experiments can be, let’s look at OpenAI’s training of GPT 4.5. OAI’s internal code monorepo is very important for training their model as they need a validation dataset to measure perplexity against when doing training ablations that is known to be uncontaminated. While scaling GPT 4.5 training experiments, they were seeing promising developments in the model’s ability to generalize only to realize mid-run that parts of the monorepo were copied-pasted directly from publicly available data. The model was not generalizing but rather regurgitating memorized code from its training dataset! Large pretraining runs requires enormous amounts of diligence and preparation to effectively execute.

Despite all of these technical issues, not all was lost. Meta was still able to distill the logits into the smaller and more efficient pretrained Maverick and Scout models, bypassing some of the flawed architectural choices of the larger model. Distillation is far more efficient than reinforcement learning for smaller models. That said, these models are still bound by the limitations of their source: they aren’t best-in-class for their size.

Meta GenAI 2.0 Pt 2: Bridging the Talent Gap

With infrastructure revamp underway and the technical lessons absorbed, Meta’s GenAI 2.0 strategy now pivots to the next ingredient of superintelligence: talent.

Mark Zuckerberg understands the talent gap relative to leading AI labs and has taken over recruiting. He’s on a mission to build a small but extremely talent-dense team, casually offering signing bonuses in the tens of millions of dollars. The goal is to create a “flywheel effect”: top tier researchers join the adventure, bringing credibility and momentum to the project. It’s already working with recent high-profile hires including:

• Nat Friedman, former GitHub CEO
• Alex Wang, former Scale AI CEO
• Daniel Gross, who was the CEO and co-founder of SSI, Ilya Sustkever’s startup.

The recruiting pitch is powerful: unrivaled compute per researcher, a shot at building the best open-source model family, and access to over 2 billion Daily Active Users. The offers that generally range from $200M to $300M per researcher for 4 years also strengthens this pitch. As such Meta has acquired awesome talent from OpenAI, Anthropic, and many other firms.

M&A, Scale AI, etc

Zuckerberg reportedly made acquisition offers to both Thinking Machines and SSI, but was turned down. While some have noted that Zuckerberg “settled” for Scale AI, we do not think this is the case. As we discussed, core to many of the Llama 4 issues were data problems and the Scale acquisition is a direct move to address that.

Alex will bring many of the top engineers from Scale, especially the SEAL lab which specializes in evals that Meta so desperately needs. SEAL has developed one of the top benchmarks for Reasoning model evaluations, HLE (Humanity’s Last Exam). With Nat Friedman and Daniel Gross joining the team, Meta gains not just elite operators but tow of the most prolific and respected investors in the AI community. Meta has very strong product people at the top.

The More You Buy The More You Save: OBBB Edition

Zuck could not have picked a better time start this spending splurge. The One Big Beautfiul Bill has some tax goodies specific to hyperscalers that could massively accelerate the tax incentives to build now and go big. Superintelligence funded by the federal government is the modern Manhattan Project.