Moonshot AI became the latest AI company to discover that launching a popular model is only half the battle. Less than two days after releasing Kimi K3, the company stopped accepting new subscribers after demand exhausted its available GPU capacity. Existing users will keep access while Moonshot expands its infrastructure and reopens subscriptions in batches.

Inference demand outpaces supply

The incident emphasizes how demand is outpacing available infrastructure. As AI models take on longer, more coding and agentic workloads, companies are finding they need more inference capacity than they anticipated.

“Kimi K3 has received far more love than we expected,” the official Moonshot account writes on X. “Over the past 48 hours, demand has pushed close to the limits of our current capacity. We’re adding capacity as fast as we can and will reopen new subscription spots in batches.”

Kimi K3 has received far more love than we expected, and our GPUs are feeling it.

Over the past 48 hours, demand has pushed close to the limits of our current capacity. To protect the experience of existing subscribers, we're temporarily pausing new subscriptions and…

— Kimi.ai (@Kimi_Moonshot) July 19, 2026

“We’re adding capacity as fast as we can and will reopen new subscription spots in batches.”

For infrastructure engineers and developers, the resulting capacity crunch is a bold indication of why companies from OpenAI to Anthropic to Moonshot are rationing access instead of selling unlimited usage.

Open weights, closed capacity

At 2.8 trillion parameters, Kimi K3 is one of the largest open-weight models slated for release — Moonshot has scheduled the public weight drop for July 27. In Arena.ai’s Frontend Code Arena, K3 topped both OpenAI’s GPT-5.6 Sol and Anthropic’s Claude Fable 5. On the broader Artificial Analysis Intelligence Index, it trails both, scoring 57 to Fable 5’s 60 and Sol’s 59. That doesn’t make it any easier to run.

Open weights let anyone deploy the model, but whoever hosts it still has to pay the inference bill. Coding activities tend to tie up GPU resources far longer than a typical chatbot interaction, making it harder to keep latency low as more developers pile on.

“Agent tasks are not one-off question answering; they continuously generate, read, and process tokens during ongoing tasks,” wrote Citigroup semiconductor analyst Peter Lee in a research note.

Lee argued that as developers build longer agentic workflows, lower inference costs are quickly “re-converted into higher total resource consumption,” shifting the bottleneck from compute to server memory.

Moonshot’s subscription pause is a sign that keeping enough inference capacity online once developers start using it at scale might be just as hard as building the model.

“Agent tasks are not one-off question answering; they continuously generate, read, and process tokens during ongoing tasks.”

China’s chip constraints compound crunch

For a company like Moonshot, this general industry bottleneck is compounded by regional infrastructure realities. Unlike traditional software companies, AI developers typically rent much of this computing power from cloud providers such as Alibaba Cloud, Tencent Cloud, and Huawei Cloud rather than owning extensive data-center infrastructure themselves.

The capacity crunch illustrates the mounting challenge facing Chinese AI developers as US export controls continue to restrict access to leading chip provider Nvidia’s most advanced AI chips. As a result, companies such as Moonshot depend on a combination of older chips and domestically produced alternatives. These constraints have forced Chinese developers to concentrate strongly on software tuning and more efficient use of computing resources to narrow the performance gap with US rivals.

Token economics under pressure

The scramble for computing power has fueled a data-center construction boom across China. Alibaba has committed more than $53 billion to AI and cloud infrastructure over three years, while ByteDance is reportedly considering spending as much as $70 billion this year on AI data centers and related infrastructure.

AI companies typically charge customers based on the number of tokens, or units of text, processed by a model, making token prices a key measure of operating costs.

According to Bernstein Research, Moonshot charges $3 per million input tokens and $15 per million output tokens for Kimi K3. This makes it about 40% cheaper than Anthropic’s Opus 4.8 and roughly 70% cheaper than Claude Fable 5.

“A world where there are only [two to three] dominant frontier labs with 90 percent inference margins is net negative for every other layer while being awesome for those [two to three] labs,” wrote Atreides Management founder Gavin Baker on X. Baker argued that models like Kimi K3, Grok 4.5, and Muse 1.1 could shift value away from the model layer and toward chipmakers, cloud providers, and the software companies building the infrastructure that serves AI models.

For developers, Moonshot’s subscription freeze serves as an architectural warning. The era of assuming infinite, cheap API access is ending.

“A world where there are only [two to three] dominant frontier labs with 90 percent inference margins is net negative for every other layer while being awesome for those [two to three] labs.”

The post Moonshot launched Kimi K3. Then demand shut down subscriptions in 48 hours. appeared first on The New Stack.

Frontier model pre-training has converged on mixture of experts (MoE), which is fundamentally changing what limits large-scale AI training. As compute per token...

Frontier model pre-training has converged on mixture of experts (MoE), which is fundamentally changing what limits large-scale AI training. As compute per token falls, communication increasingly determines how efficiently models scale across thousands of GPUs. NVIDIA GB300 NVL72 set a world record for pre-training DeepSeek-V3 671B at 1,648 TFLOPs per GPU, showing how advances across the entire AI…

Source

After weeks of extending temporary access while bringing additional inference capacity online, Anthropic has finalized Claude Fable 5 subscriptions.

In a Friday announcement on X, the company said Claude Fable 5 will become a permanent part of Max and Team Premium subscriptions beginning Monday, July 20, at 50% of each plan’s usage limits. Pro and Team Standard users will transition to using the model through usage credits and will receive a one-time $100 credit.

An Anthropic spokesperson tells The New Stack, “Friday’s announcement means that Fable 5 at 50% of usage limits is now a standard, permanent part of Max and Team Premium plans.”

The changes bring an end to weeks of temporary extensions that left subscribers wondering whether Fable 5 would remain part of their subscription. Rather than introducing a new usage policy, Anthropic is making permanent the same access level subscribers have had since Fable 5 was redeployed on July 1.

Capacity drove the timeline

The move gives a glimpse into the economics of serving frontier AI models. Although Anthropic repeatedly extended temporary access over the past several weeks, the company says making Fable 5 a permanent part of Max and Team Premium subscriptions was always the goal.

An Anthropic spokesperson tells The New Stack, “Friday’s announcement means that Fable 5 at 50% of usage limits is now a standard, permanent part of Max and Team Premium plans. This was always our intention with Fable access (see more in our original launch blog on June 9). We planned to expand Fable 5 access in stages as we brought additional inference capacity online, with the ultimate goal of making it a standard part of subscription plans.”

Fable 5 usage limits explained

Anthropic has not explained exactly how it calculates the 50% limit or how many prompts that will translate to in practice. Developers using Fable 5 for heavy workflows are likely to reach the limit sooner than lighter users.

The company has said demand for Claude Fable 5 was difficult to predict and that broader access depended on bringing more capacity online. Reuters reported earlier this week that the company is in talks with Meta over a potential compute lease worth up to $10 billion, which could give Anthropic access to Meta’s AI infrastructure for as long as two years. 

Meta compute deal looms

Thariq Shihipar, an Anthropic Developer Relations team member, posted on X, crediting the company’s employees with getting the necessary capacity online.

“This was due to a heroic effort by many people at Anthropic working sometimes literally around the clock,” Shihipar wrote on X. “It was not at all clear that we’d be able to do this in time, and so proud of everyone who made it happen. Enjoy Fable.”

“This was due to a heroic effort by many people at Anthropic working sometimes literally around the clock. It was not at all clear that we’d be able to do this in time, and so proud of everyone who made it happen. Enjoy Fable.”

Claude expands into India

The revised policy reflects broader changes across the AI industry. Frontier models are becoming expensive, which means AI companies are forced to balance compute costs with subscription pricing. OpenAI recently introduced GPT-5.6 Sol, while Chinese AI companies have released increasingly capable open-weight models, including Moonshot AI’s Kimi K3. Although Anthropic has not linked today’s pricing decision to competitive pressure, both model capability and the cost of serving those models have become key competitive factors.

…both model capability and the cost of serving those models have become key competitive factors.

Alongside the subscription changes, Anthropic is also formalizing its presence in one of its fastest-growing markets by introducing local pricing for Claude in India.

Indian customers can now pay for Claude in rupees instead of U.S. dollars, with GST included in the listed prices. That removes foreign currency conversions and other cross-border payment costs that previously came with subscribing. Claude Pro is now priced at Rs. 2,000 per month with annual billing or Rs. 2,399 on a month-to-month plan (previously $20 USD per month).

Industry reports have identified India as Claude’s second-largest user base after the United States. Together, the announcements show Anthropic expanding Claude in two ways: by making subscriptions easier to purchase in a major growth market while locking in an access policy determined by the compute required to serve Fable 5.

The post Anthropic employees worked “literally around the clock” to keep Fable 5 from disappearing appeared first on The New Stack.

What if autonomous coding AI agents could push your vision reasoning models above 90% accuracy with almost no manual effort? When adapting vision reasoning...

What if autonomous coding AI agents could push your vision reasoning models above 90% accuracy with almost no manual effort? When adapting vision reasoning models to production video tasks, developers often lose days to data formatting, container setup, training scripts, baseline evaluation, and hyperparameter sweeps before they even know whether post-training improves accuracy.

Source

Large language model (LLM) training workloads increasingly run into GPU memory limits before compute is fully used. Model weights, gradients, optimizer states,...

Large language model (LLM) training workloads increasingly run into GPU memory limits before compute is fully used. Model weights, gradients, optimizer states, communication buffers, and intermediate activations all compete for GPU high-bandwidth memory (HBM). As model size, sequence length, and batch size grow, HBM capacity often becomes the primary scaling bottleneck. This post explains how…

Source

AI performance comes down to three dimensions:  Accuracy: How well the model reasons and produces outputs Throughput: How many tokens per second a...

AI performance comes down to three dimensions: Deployments must balance all three: High accuracy is wasted if responses are slow, and raw throughput means little if each user’s experience is laggy. Practical systems therefore optimize accuracy, throughput, and interactivity together. This post focuses on throughput and interactivity, and how model-design choices shape both without…

Source

While the household-name frontier models race forward with version numbers and context windows of at least a million tokens, a new breed of upstart data science specialists is pushing the context window into double figures. 

Subquadratic debuted a 12-million-token window in May of this year, and Silicon Valley and South Africa-based Refiant launched its 10-million-token context-window model, Protea, on Wednesday. It’s a move that may signal the long-context AI race is now on.

Model inefficiency & workarounds are commonplace

But more context windows alone are not enough. This is because even the most capable models have a few hundred thousand tokens in working memory, which can force workarounds to compensate for what the model can’t access.

Refiant co-founder Dr. Viroshan Naicker tells The New Stack that he believes modern LLMs “fail to be organically efficient at an elemental level” and that his organization’s approach mimics how systems in nature, from ant colonies to beehives, find efficient solutions to complex problems.

“This is no case of pseudoscientific puff; nature is the most computationally efficient system that we know, and many algorithms used in science are nature-inspired,” Naicker says. “This is a road well-traveled in science. There are multiple teams globally working in this particular (nature-inspired) direction, trying to bridge the gap between AI inference as we know it and the energy efficiency of natural systems.”

“Fish and birds coordinate their movements to converge on the mathematically shortest, most efficient routes — honeybees, fireflies and bacteria are also programmed to use degrees of swarm-style optimization.”

What can the birds & the bees teach us about AI?

Did Naicker just mention ant colonies and honeybees?

Yes, because Refiant uses swarm-style optimization. It’s seen in ant colonies, which initially move randomly until a food source is detected, after which they leave a pheromone trail for other ants to optimize their journeys. Fish and birds also coordinate their movements to converge on the mathematically shortest, most efficient routes. Honeybees, fireflies, and bacteria are also programmed to use degrees of swarm-style optimization.

Naiker, along with his co-founders, Siddharth Gutta and Mathew Haswell, form a team with experience spanning quantum mathematics, traditional finance, and commercial scaling. Applying swarm-style optimization to data in Protea means inference is performed through a combination of compression and context management.

“From our perspective, we are also advancing a technology which provides context-specific inference models grounded in data,” Naicker clarifies. “We think this has value for reducing model hallucinations, replacing RAG, and constructing better, more reliable, agentic workflows. This adds a layer of trust in sensitive application scenarios, rather like an added insurance, rather than taking it away.”

Just how much is 10 million tokens?

The Refiant team describes 10 million tokens as equivalent to 7.5 million words in a single conversation (and we know from Anthropic’s own benchmarks this year that Claude has a 1-million context window), or five years of a user’s emails, 83 novels, or 830 podcast episodes, all held in active memory at the same time.

The team claims Protea is capable of working on entire enterprise codebases or decades of clinical trial data — datasets that previously had to be broken apart and fed to models in fragments — so they can be processed in a single pass with full fidelity. Engineers on Protea also submit that they can successfully tackle the “lost in the middle” problem — a limitation of million-plus-token windows, where models stay accurate at the start and end of the context but lose the thread of everything buried in between.

Refiant first applied these techniques to model compression, shrinking OpenAI’s GPT-OSS-120B so it could run on a MacBook Pro with 18GB of RAM. 

“Rather than publishing benchmarks, we’re inviting users to run the models and try them out.”

The Protea series is open and live, and Refiant is inviting teams to stress-test the context window across different industries and use cases. But should we trust sensitive hould enterprise data archives to a completely unproven startup founded only one year ago?

Bring-your-own-cloud, a possible progression

“We adhere to data management best practices, processes and compliance requirements,” Naiker confirms. “This is reasonable for a startup at our particular stage. Privacy and data sovereignty are important values for us, and we are actively exploring edge, self-hosted, and bring-your-own-cloud data models.”

But a 10 million-token context window is big. Won’t that fall short when Protea starts to suffer from massive latency spikes when processing a full dataset? Naiker agrees that “latency is a core issue with long-context inference models,” but in the tests his company has run, it has delivered inference at a reasonable latency, even with large token windows.

“We have internal reports and tests that validate the technology, including Ruler, MRCR and Babilong, but we aren’t asking anyone to take our word on this. Rather than publishing benchmarks, we’re inviting users to run the models and try them out,” adds Naiker.

What comes next, a 100-million context window?

Although the technology industry is littered with apocryphal statements and Bill Gates almost certainly never said “64K ought to be enough for anyone” in real life, we have to ask ourselves today whether we’ll be laughing about those “silly little” 10 million token context windows by the end of the decade.

It may not take that long. Internally, Refiant maintains that it has already demonstrated a working prototype with a 100-million-context window and is exploring how best to benchmark and productionize it at that scale in the future.

Coming next, then, as Dr. Evil from Austin Powers would say, the one-hundred-billion-context window, right?

The post “Nature is the most computationally efficient system we know”: How Refiant used swarm optimization to build a 10-million-token AI model appeared first on The New Stack.

Power can account for 40% of the operating expenses (OpEx) to run an AI factory. Each watt can be spent on overhead, data ingestion, training, or generating...

Power can account for 40% of the operating expenses (OpEx) to run an AI factory. Each watt can be spent on overhead, data ingestion, training, or generating tokens for customers. And most sites are capped at a fixed power level provided by a regional provider. Under these conditions, performance per watt becomes a key efficiency metric that directly translates to token costs.

Source

Transformer architectures are the backbone of many modern large language and generative AI models. As these models grow in size, training runs consume more GPU...

Transformer architectures are the backbone of many modern large language and generative AI models. As these models grow in size, training runs consume more GPU hours and more engineering iteration time. Accelerating transformers is therefore not just a performance optimization, but directly affects how quickly teams can experiment and how large a model they can afford to train.

Source

Foundation models are reshaping computational biology. Pretrained on massive corpora of protein or genomic sequences, models such as ESM2 (a protein language...

Foundation models are reshaping computational biology. Pretrained on massive corpora of protein or genomic sequences, models such as ESM2 (a protein language model) and Evo 2 (a DNA language model) capture statistical regularities of biological sequences. These transfer well to a wide range of downstream tasks, including structure prediction, variant effect, and functional annotation.

Source