Nvidia and Palantir announced Thursday that they’re working together to bring “sovereign AI to critical supply chains,” kicking off initially with Nvidia’s own sprawling supply chain.
The news builds on a partnership that kicked off last October, when the duo said they would combine Nvidia’s AI computing and models with Palantir’s software to help companies use AI to make complex operational decisions. Then in June, they expanded that effort into sovereign AI, allowing organizations to run and customize Nvidia’s AI models inside tightly controlled environments while keeping sensitive data and model weights under their own control.
Now, they’re applying that technology inside Nvidia itself, where they say a smaller, fine-tuned model is already outperforming a far larger one.
The companies have fine-tuned Nvidia’s 30-billion-parameter Nemotron 3.5 Lightning model on decisions made by Nvidia’s supply-chain operations team. Palantir’s Foundry and Artificial Intelligence Platform (AIP) bring together the data behind those decisions, while its Ontology acts as a live map connecting components, factories, capacity and production commitments. Nvidia’s cuOpt software, meanwhile, works out how to distribute scarce parts, with Nemotron weighing the wider context and recommending what planners should do.
They then plan to “extend the learnings from Nvidia’s deployment” to companies in other sectors, including manufacturing, energy, healthcare, automotive and aerospace. Palantir’s own customers will be able to build versions tailored to their own supply chains by training Nemotron on their proprietary data using Foundry and AIP, then run the resulting system on-premises or through cloud and colocation providers.
So, in effect, Nvidia and Palantir are putting the sovereign AI partnership they outlined in June into practice inside Nvidia, while using that deployment as a proving ground for an architecture other companies can adapt to their own use cases.
As the world’s most valuable public company at a $5.4 trillion market cap, there’s good reason for Nvidia to start close to home. Its supply chain spans millions of parts, thousands of suppliers, and a global network of manufacturing partners, with the company saying a single Vera Rubin rack alone contains some 1.3 million parts. Those components have to arrive in the right place at the right time: if one part is missing, assembly can stall while everything else that arrived sits waiting.
“Supply chains are the operating system of the physical economy, and AI factories are among the most complex systems ever built.
Jensen Huang
And that complexity is what Nvidia founder and CEO Jensen Huang says makes supply chains a natural target for the technology. From chips and memory to manufacturing, networking, power and cooling, he argues that building modern AI systems increasingly depends on coordinating an enormous web of companies and components.
“Supply chains are the operating system of the physical economy, and AI factories are among the most complex systems ever built,” Huang says in a statement.
Palantir co-founder and CEO Alex Karp goes further, arguing that Nvidia’s operations provide an unusually demanding environment in which to put the companies’ approach to the test.
“Nvidia has arguably the most valuable, intricate, and complex supply chain in the world.”
“Nvidia has arguably the most valuable, intricate, and complex supply chain in the world,” Karp adds in a separate statement.
Nvidia has long been positioning itself at the center of the open-model debate. In July, Huang even used his first-ever post on X to promote an industry letter lobbying Washington to support frontier open-weight models, arguing that they give companies and countries more control over their AI infrastructure.
Then in early September, Nvidia swooped in with a $12.9 billion deal for Hugging Face, the so-called “GitHub for AI models.” Amid concerns that ownership by the world’s dominant AI chipmaker could undermine Hugging Face’s neutrality, Huang pledged that it would remain open, continue hosting models from across the industry and support hardware beyond Nvidia’s own.
Nemotron is central to Nvidia’s own open-model push. The name dates back to 2023, when Nvidia released its first Nemotron-3 8B models for enterprises to customize and fine-tune. Those early models were downloadable through Hugging Face and Nvidia’s NGC catalog, although access was gated and governed by Nvidia’s own community license. So they were customizable, and their weights were available, but the much broader “open model” positioning Nvidia uses today came later.
The current Nemotron 3 series arrived back in December, initially spanning Nano, Super and Ultra models aimed at different agentic AI jobs. Nvidia now publishes weights and, for many of the models, training data and recipes so developers can customize themselves. Nemotron 3.5 Lightning, released in August, is the 30B model Nvidia and Palantir have fine-tuned for this supply-chain deployment.
That openness is also at the heart of the whole sovereignty pitch: companies can adapt Nemotron using proprietary data while keeping that data, the model weights, and inference inside their own environment.
Nvidia’s own deployment gives outsiders a result to chew on. It says the fine-tuned 30B Lightning scored 86.7% accuracy on its supply-allocation task, versus 55.5% for the 550B Nemotron 3 Ultra—a model roughly 18 times its size.
In a technical blog post published on Thursday alongside the main announcement, Nvidia solutions architects Nell Barber, Rana Haber, and Aastha Jhunjhunwala note that the result shows how far specialization can go. On a tightly defined allocation task, the 30B model outperformed a general-purpose model more than an order of magnitude larger.
“This doesn’t mean the smaller model is more capable overall. Its gains are concentrated in the domain it was post-trained on.”
“This doesn’t mean the smaller model is more capable overall,” they add. “Its gains are concentrated in the domain it was post-trained on. Future production risk forecasting remained difficult despite fine-tuning. Specialization improved the decision task but failed to solve every prediction problem attached to it.
For companies considering Nvidia’s blueprint, the more interesting takeaway may be this: a smaller open model, trained on business specifics, can sometimes be more useful than reaching for the biggest model available.
The post Nvidia and Palantir fine-tune a 30B Nemotron model for Nvidia’s supply chain. It beats a model 18 times its size. appeared first on The New Stack.
Update: As of 6:30 p.m. Eastern on Friday, September 4, GPT-6 Astra was available on ChatGPT for all paying users with plans that include access to it.
Thibault Sottiaux, a leading member of the technical staff at OpenAI, posted on his X account, “OK nevermind, the team and Astra did a good job and our systems are more scalable than we anticipated. Astra is now rolled out to all Plus and Business users too. Hope you have a blast and let us know how it goes!.”
Earlier in the day Friday, OpenAI CEO Sam Altman posted on X, “GPT-6 Astra is now available to all Pro, Enterprise, and Business Premium users in Work/Codex, and is available in the API. We will start rollout to Plus and Business users next. Thank you for the patience.”
Users who pay for the $8/month ChatGPT Go plan do not and will not have access to GPT-6 Astra or GPT-5.6, according to the company’s pricing tier documentation.
Sottiaux posted on Thursday evening after OpenAI announced the debut of Astra but before it released it to users: “We are starting to release GPT-6 Astra, and we are doing it as carefully and quickly as possible.”
OK nevermind, the team and Astra did a good job and our systems are more scalable than we anticipated.
— Tibo (@thsottiaux) September 4, 2026
Astra is now rolled out to all Plus and Business users too. Hope you have a blast and let us know how it goes!
Our original story, published at 12:24 p.m. Eastern on Friday, continues below:
OpenAI launched GPT-6 Astra on Thursday, but many developers hoping to give it a test drive are still waiting for access.
Hours after the announcement, OpenAI CEO Sam Altman apologized for what he called a “messy rollout,” acknowledging that broad access to Astra had not yet begun for either API customers or ChatGPT subscribers.
“First, sorry for the messy rollout,”
“First, sorry for the messy rollout,” Altman wrote in a post on X. He added that OpenAI expected to begin the broader rollout “in the near future,” starting with ChatGPT Pro subscribers.
first, sorry for the messy rollout.
— Sam Altman (@sama) September 4, 2026
second, when we screw up, we try to make it right.
third, we should be able to begin broad rollout to API customers and chatgpt subscribers in the near future. as usual we will start with pro subscribers. https://t.co/nKOhW18CDK
For developers, that creates an unusual bind. OpenAI has already published Astra’s API documentation and pricing, including its 1.05 million-token context window, support for up to 128,000 output tokens, and standard API rates of $10 per million input tokens and $50 per million output tokens. But the endpoint itself is still rolling out, and OpenAI has yet to explain specifically what went wrong with the rollout. So while Astra’s benchmark results were impressive, developers and independent reviewers who want to evaluate those claims against their own workloads are still left waiting.
OpenAI’s engineering lead for Codex, Thibault Sottiaux, offered a little more detail in an X post of his own about what is happening behind the scenes. He said the rollout will take “a few days” to complete and that OpenAI is bringing new systems and additional compute online as it expands access.
“We are starting to release GPT-6 Astra, and we are doing it as carefully and quickly as possible,” Sottiaux wrote. He added that “many novel systems will operate at scale for the first time” during the rollout and that OpenAI is “bringing a lot of compute up.”
“We are starting to release GPT-6 Astra, and we are doing it as carefully and quickly as possible,”
While this does not establish that compute capacity caused the delay, it does offer some insight into what OpenAI is dealing with as it expands access. The company’s original announcement said Astra would initially be available to a limited group of organizations, with ChatGPT Plus, Pro, Business, and Enterprise users, the OpenAI API, Microsoft Azure, and AWS Bedrock expected to follow “over the coming days.”
Developers who have secured early API access are already encountering surprises beyond the rollout itself. As The New Stack reported this week, Astra’s API introduces a new class of safety-triggered interruptions that look like timeouts but aren’t — which could make a real difference for developers building production workflows around the model.
While users wait, OpenAI is trying to make up for at least part of the delay with something it calls a “banked reset.”
Sottiaux said paid ChatGPT subscribers will receive one banked reset for every day they remain without Astra access, beginning September 3, adding that the “team is moving mountains to give access as fast as we can.”
“Team is moving mountains to give access as fast as we can.”
OpenAI has used banked resets before with ChatGPT Work and Codex, giving users a way to replenish their usage after hitting a limit. They aren’t additional API credits or a permanent increase in usage limits but resets users can save until they need them.
What’s different with Astra is how OpenAI is handing them out: one for every day a paying ChatGPT subscriber remains without access. So far, OpenAI hasn’t announced anything similar for developers waiting to use Astra through the API.
The gesture fits a broader pattern of OpenAI experimenting with how it charges for AI: the company recently announced an outcome-based pricing model that would bill only when the model produces a correct result; another sign that its pricing strategy is still very much in flux.
OpenAI says Astra will roll out to Plus, Pro, Business and Enterprise users, along with the OpenAI API, Microsoft Azure and AWS Bedrock, “over the coming days,” with Pro subscribers first. Sottiaux said the rollout should take a few days to complete.
For developers, the biggest unanswered question is when broad API access will arrive and whether it will come with tighter usage limits. Astra’s persistent-agent capabilities also make the wait more significant for developers.
OpenAI did not immediately respond to The New Stack’s questions about the rollout problems, banked resets, or API timeline.
The post “Sorry for the messy rollout”: OpenAI launches GPT-6 Astra to most paying users a day after its unveiling appeared first on The New Stack.
Nvidia has confirmed that it’s agreed to acquire Hugging Face in a mammoth $12.9 billion deal that will bring one of the AI industry’s most important open-model platforms under the auspices of the world’s dominant AI chipmaker — and, with a market cap of well over $5 trillion, the most valuable company on Earth.
When reports first emerged in August that Nvidia was lining up a gargantuan bid for what has often been described as the “GitHub of AI Models,” concerns quickly surfaced over what the deal could mean for the platform’s openness and hardware neutrality. As The New Stack reported at the time, Hugging Face’s value lies partly in giving developers a neutral place to find and deploy open models across Nvidia, AMD, Intel and cloud accelerators — raising the question of what happens if that platform is owned by just one of them.
That, ultimately, is why Nvidia founder, president and CEO Jensen Huang is going to great lengths to allay fears that Hugging Face could become a vehicle for steering developers toward Nvidia’s own hardware and software stack.
“Nvidia compute will not be required to build on or deploy through Hugging Face.”
In the official announcement on Thursday, Huang makes a series of explicit promises around neutrality, noting that Hugging Face would “remain an open platform for the entire AI ecosystem,” continue to support multiple clouds and accelerators, while stressing that “Nvidia compute will not be required to build on or deploy through Hugging Face.”
“Hugging Face will continue to support open source and open weight models from across the ecosystem, from every model builder,” Huang writes. “It will continue to support multi-cloud and multi-accelerator development and deployment, so builders can use the hardware and infrastructure that best fit their work.”
Indeed, it’s clear Nvidia had clocked the concerns around the impending acquisition. The word “open” appears no fewer than 19 times in Huang’s relatively short announcement, underlining just how central that reassurance is to Nvidia’s pitch for the deal.
Huang also points to a recent open letter on open weights that he co-signed alongside executives and researchers from across the AI industry, including those from Hugging Face. The letter argued that open-weight models are critical to broadening access to AI, strengthening competition and giving developers more control over how models are deployed and adapted.
For what it’s worth, Nvidia has been pushing hard into open-weight AI itself, including through its Nemotron models and a broader effort to make frontier-class models easier to run locally. Hugging Face co-founder and CEO Clément Delangue went so far as to call Nvidia the “King of American open-source AI” a few months ago, pointing to its growing collection of public models, datasets and Spaces on Hugging Face.
In the wake of the announcement on Thursday, Delangue doubled down on that position in a fresh post announcing the deal, arguing that Hugging Face has reached the point where keeping open-source AI competitive would require substantially more resources.
“It needs more compute, more support, more collaboration and more visibility.” – Hugging Face CEO Clément Delangue
“10 years after starting Hugging Face, open-source AI is at an inflection point,” Delangue writes on LinkedIn. “Thanks to the community, we’ve shown that it can be a complement, and even an alternative, to closed-source APIs. But for it to happen at larger scale, it needs more compute, more support, more collaboration and more visibility.”
He adds that Nvidia has committed to backing Hugging Face while keeping the platform open, independent and compute-agnostic, with the founders and existing team staying on.
“Together, we think we can make open source the default way to build AI,” Delangue writes, setting out a goal of helping 100 million AI builders “own their intelligence rather than rent it.”
However, there is some historical precedent for taking such assurances with a degree of caution. When Microsoft bought GitHub for $7.5 billion in 2018, it similarly promised that the platform would remain independent and open. GitHub largely retained that openness, though Microsoft’s later use of public GitHub code to help train the proprietary, paid Copilot service sparked a backlash among factions of the open-source community.
In truth, that obvious comparison may actually understate Nvidia’s challenge. In recent analysis for Forbes, technology analyst Janakiram MSV argues that neutrality at Hugging Face has a hardware dimension that GitHub never had to contend with. Hugging Face maintains integrations spanning AWS Trainium and Inferentia, Google TPUs, Intel Gaudi, AMD Instinct and other accelerators. Under Nvidia ownership, continued support for those rival chips becomes a real test of just how neutral the platform will remain in the long run.
As for the acquisition itself, well, it’s not over the line quite yet. In a filing with the US Securities and Exchange Commission (SEC), Nvidia notes that it expects the deal to close in the first half of 2027, subject to customary closing conditions and regulatory approvals. Given Nvidia’s lofty position in AI infrastructure and Hugging Face’s role as a major distribution point for open models, those approvals are unlikely to be a mere formality. Nvidia’s filing also flags the possibility that future regulation around open-source AI could affect Hugging Face’s operations or increase compliance costs.
“Hugging Face [will] continue to permit model makers, developers, and users to upload and download models and datasets of their choosing and to support other silicon vendors.”
Notably, Nvidia also uses its SEC filing to reaffirm its neutrality commitments, stating that under the commitment, “Hugging Face would continue to permit model makers, developers, and users to upload and download models and datasets of their choosing and to support other silicon vendors.”
Of the roughly $12.9 billion headline price, about $11.9 billion will go to Hugging Face stockholders, with up to $1 billion earmarked for equity-based retention awards for employees joining Nvidia.
And while the headline price is usually rounded to $12.9 billion, the actual figure is an oddly specific $12,930,300,000. That’s no accident either, as Hugging Face co-founder Thomas Wolf alludes to in a LinkedIn post. For those still in the dark, 129303 is the decimal Unicode value for the 🤗 emoji, while #129303 is a green color code nodding to Nvidia’s branding.
A neat little Easter egg buried inside what can only be described as one of the biggest AI deals of the year.
The post “Hugging Face will remain an open platform”: Nvidia strikes $12.9B deal for the ‘GitHub of AI’ appeared first on The New Stack.
A 438-billion-parameter reasoning model isn’t an obvious choice when speed is a priority. Multiverse Computing is betting that compression can make it one.
On Wednesday, the Spanish company launched Quasar 438B, its first large-scale model and a system designed specifically for coding and enterprise agents. Quasar achieves a score of 43 on Artificial Analysis’ Intelligence Index and 69.3 on Terminal-Bench v2.1, while Artificial Analysis currently records its output speed at approximately 183 tokens per second. Multiverse is positioning Quasar as the European model with the highest score on the Intelligence Index, outperforming Mistral Medium 3.5 (30) and NVIDIA Nemotron 3 Ultra (38).
Multiverse is betting that a 438B-parameter model can be fast and inexpensive enough for agents that repeatedly reason, call tools, and check the results.
That puts Quasar in an interesting middle ground. It doesn’t match the strongest models on coding performance, but Multiverse is betting that a 438B-parameter model can be fast and inexpensive enough for agents that repeatedly reason, call tools, and check the results.
The model features a context window of one million tokens, is available in English and Spanish, and can be accessed via the Multiverse CompactifAI API.
Multiverse built CompactifAI to shrink large AI models so they need less memory and compute to run. The company says it can reduce model size by 80% to 95% with only a small loss in accuracy, but it hasn’t disclosed how much Quasar was compressed or which model it started with.
In July, Multiverse announced a $570 million Series C to expand its library of compressed models and commercialize the technology. Quasar is the biggest test of that approach so far.
Multiverse hasn’t said what hardware is required to run Quasar or how much the compression reduces its memory and compute needs. That matters for agents, which may repeatedly call the model and other tools before finishing a task.
Multiverse hasn’t said what hardware it takes to run Quasar or how much the compression cuts its memory and compute needs.
In Multiverse’s comparison, Quasar’s Terminal-Bench v2.1 score of 69.3 places it ahead of Mistral Medium 3.5 but still well behind the best frontier systems, while Claude Opus 5 achieves the highest score on that benchmark at 89.1.
Multiverse is pitching Quasar for software engineering, technical copilots, research, and workflow automation. Its 1-million-token context window gives agents room to work with large codebases and hold onto information as a task progresses, although processing more context also requires more compute. That can be especially important in coding, where code that passes every test can still trip up the next AI agent if it loses track of what came before.
Artificial Analysis found that Quasar starts responding in about 1.1 seconds and can produce a 500-token response, including reasoning, in around 15.3 seconds. Those numbers are fast, but an agent also has to wait for tools, process growing context, and make repeated model calls over the course of a task. The agent tooling layer itself is still catching up to what these models need.
Those numbers are fast, but an agent also has to wait for tools, process growing context, and make repeated model calls over the course of a task.
Quasar is proprietary and only available through Multiverse’s API, so developers can’t inspect the weights or run it on their own hardware. For now, that also makes it difficult to know whether the speed Multiverse is reporting will carry over to everyday agent use.
Quasar also arrives as European AI companies are building more of their own model and compute infrastructure instead of relying on U.S. hyperscalers. Multiverse is taking a different route, using compression to make a 400B-plus model cheaper and faster to run. The next step is to see how that holds up against benchmarks.
The post Multiverse says its 438B model is fast enough for AI agents. The benchmarks tell a more complicated story. appeared first on The New Stack.
This post is the third in a series on AI model co-design. It explores how to accelerate LLM inference while maintaining accuracy using speculative decoding and...This post is the third in a series on AI model co-design. It explores how to accelerate LLM inference while maintaining accuracy using speculative decoding and offers five guidelines for selecting draft length and draft mechanism across the Pareto frontier. For a discussion of how model design choices impact both throughput and interactivity without sacrificing accuracy, see AI Model Co…
Alibaba this week unveiled Qwen3.8-Flash, an open-weight, multimodal Mixture-of-Experts (MoE) model.
Hot on the heels of Qwen 3.8 Max, which arrived at the start of the month, this 125-billion-parameter AI model is offered as a prelude to Qwen 4. It is positioned as both a performance and value-for-money play. As such, it is claimed to have “superior capabilities in coding and office tasks” and an optimal balance among capability, latency, and cost.
“In this release, we are opening the weights of Qwen3.8-Flash-Next, a multimodal MoE model that also serves as an early preview of the architecture used in Qwen4,” confirmed Alibaba in its release blog.
By releasing the architectural changes in Qwen3.8-Flash early, the organization hopes the community will examine the mechanics, constructs, and components within and start road-testing them before the full Qwen4 model family is built on top of them.
By stating that Qwen3.8-Flash paves the way for Qwen4, Alibaba is showcasing (and, importantly, openly sharing) design forms that it will carry into its subsequent models, so that developers can start building (or at least planning) their next codebases early.
Specifically then, Alibaba has stated that Qwen3.8-Flash “plays the same role” that Qwen3-Next played for Qwen3.5 i.e. by which the company means that Qwen3.8-Flash introduced developers to the company’s hybrid Gated DeltaNet + Gated Attention design (computational components that enable AI models to balance long-context efficiency with contextual focus and attention during inference and training), which was then used across the Qwen3.5, Qwen3.6, Qwen3.7 and Qwen3.8 series.
“Qwen3.8-Flash-Next upgrades the model systematically along four aspects – attention, residual, embedding and optimization – improving model capability while further optimizing computational efficiency, model capacity and training stability,” stated Alibaba.
When benchmarked on agentic coding, long-horizon agent tasks and multimodal intelligence, Qwen3.8-Flash appears to perform respectably against models including DeepSeek-V4-Flash and Claude-Opus-4.6 across SWE-bench Pro (agentic coding), CoWorkBench (long-horizon office work), Toolathlon Verified (real-world tool use), MathVision (visual maths problem-solving), AndroidWorld (agentic mobile use) and ERQA (embodied intelligence).
“Qwen3.8-Flash-Next upgrades the model systematically along four aspects – attention, residual, embedding and optimization – improving model capability while further optimizing computational efficiency, model capacity and training stability.”
Tested on agentic coding using SWE-bench Pro, Qwen3.8-Flash-Next scores 62.5, compared to 61.7 on Qwen3.8-27B, 55 on Qwen3.7-Plus, 56.0 on DeepSeek-V4-Flash-0731, and 53.4 on Claude-Opus-4.6 (Max).
Importantly, Qwen3.8-Flash requires only what Alibaba details as “around one-ninth of the training resources,” while delivering superior performance. The company says that this means Qwen3.8-Flash “significantly reduces” both training and inference costs compared with Qwen3.7-Plus, a model three times its size.
For the sake of nomenclature, Qwen3.8-Flash-Next is the open-weight research-frontier model available to developers on both the Hugging Face AI developer hub and Alibaba’s ModelScope community portal. Built on the same underlying architecture, Qwen3.8-Flash is the production version of the model, offered via the QwenCloud API with 1 million tokens by default and official built-in tools.
Qwen3.8-Flash-Next features a 125B-parameter main model, supplemented by an additional 51B N-gram embeddings, with 6B parameters activated per token. It natively supports 262,144 tokens of context and can be extended to 1,000,000 tokens with YaRN.
As noted, Qwen3.8-Flash introduces architectural extensions across attention mechanisms, residual connections, embeddings, and optimization. Its hybrid attention architecture combines Gated DeltaNet (GDN), which compresses historical information, with Qwen Sparse Attention (QSA), a novel design that uses a lightweight compressed indexer to select relevant context, substantially reducing attention costs for long sequences.
Alibaba has explained that the Gated Residual (GR) mechanism expands data pathways between layers while strengthening cross-layer information flow and training stability. The N-gram Embedding technique scales model capacity with minimal additional computation, while the Muon Optimizer enhances the efficiency of large-scale model training.
“I’d love it if local LLMs actually could replace remote ones today, and I have no reason to lie about my experience either…it’s just not there (yet) for local professional software development.”
In terms of developer reaction, it’s been a mixed bag so far.
Mechatronics engineer Alok posts on X, saying he thinks the Video RAM barrier (i.e., the need for physical, high-speed GPU-based memory for LLM memory management in the face of model quantization that aims to enable better long-context inference) is now officially dead.
“I just ran Qwen3.8-Flash-Next (MoE) 125B A6B with a 250,000 context window on a single 24GB RTX 4090 – 21 tokens/sec decode. 364 t/s prefill – no mtp. No dflash. No KV cache quantization! We are running datacenter models on consumer hardware,” enthused Alok.
Multi-disciplined software developer Embedding Shapes is less happy.
They post on Hacker News using some colorful language to describe how models keep [insert expletive]-ing up very basic things before saying, “I’d love it if local LLMs actually could replace remote ones today, and I have no reason to lie about my experience either. But I too got hopeful reading the sentiment on the Internet about Qwen 3.8, but it’s just not there (yet) for local professional software development.”
Qwen3.8-Flash can be accessed via API Model Studio and Qwen Cloud, Alibaba’s AI-native cloud platform. Pricing per 1 million tokens is US$0.16 for input and US$0.47 (or 3 RMB for developers inside China) for output.
The model is also available on QwenWork, Alibaba’s workplace AI agent platform, where it runs a redesigned “standard mode” that cuts token consumption per task by 75% and “roughly doubles generation speed” compared with the current mode.
Alibaba has claimed that this brings flagship-level capabilities within reach of everyday workloads and that developers can run this model on hardware that they likely already own.
Alibaba has not confirmed a firm launch date for Qwen4. Still, a casual web search on the topic yields a range of commentators and market watchers who broadly agree it will arrive before the end of the year, possibly as early as September.
Conjecture in this space explores whether the next model family could be optimized for complex 3D coding and design tasks in advanced spatial modeling. Others think that the Mixture-of-Experts (MoE) architecture will be extended and that deeper native multimodal processing capabilities might be featured.
Alibaba was contacted for broader comment on this story but declined to engage.
Qwen, in traditional Chinese, 通義千問 (pronounced Tōngyì qiān wèn), translates to “a thousand questions on general meaning” in English. Use that in your local pub quiz this weekend.
The post Alibaba just released Qwen3.8-Flash: “An early preview of the architecture in Qwen4” appeared first on The New Stack.
Growing datasets and public benchmarks are making it harder to tell whether a model is being tested on something it hasn’t seen before.
On Thursday, Google DeepMind showed off what the company calls the first double-blind evaluation of a proprietary frontier-class AI model. The setup keeps Gemini’s model weights hidden from the evaluators while the test questions are hidden from Google.
The pilot tested Gemini 2.5 Flash Lite against private benchmarks from MLCommons and the Singapore AI Safety Institute, but rather than using the results to tout a new Gemini score, Google is focusing on how the tests were run, with neither side having access to the other’s data.
Google’s technical report cites earlier research that found signs of benchmark leakage in about half of the 31 models tested, and another study published this year that found contamination can inflate scores, particularly for larger models. This dynamic has already raised questions about where certain post-training coding gains actually come from.
Keeping benchmarks private would seem like the obvious answer, except closed models make that difficult. With a closed model, evaluators usually have to run their questions through the provider’s API, which means the company can see a test that was supposed to stay private, whereas running it independently would require the provider to hand over its model weights. Google’s setup is meant to give both sides another option.
Keeping benchmarks private would seem like the obvious answer, except closed models make that difficult.
The pilot is using Google Cloud Confidential Space together with an NVIDIA H100 Confidential GPU and Intel TDX host memory encryption.
Google DeepMind provides Gemini with its weights and inference code, while the evaluator offers its benchmark prompts and evaluation code; these are then transmitted over encrypted connections into the enclave, where the evaluation takes place without either party gaining access to the other’s protected assets.
The model weights are stored in hardware-encrypted GPU memory, and the evaluation prompts are kept in encrypted host memory as well. After the test is complete, the evaluator is given the permitted results, and the temporary environment can then be destroyed.
Before either side sends over its private data, remote attestation verifies that the enclave is running the software they agreed on.
The enclave protects what’s stored in memory, but the code running inside it can still create problems. If an application has unrestricted network access, for example, it could send sensitive information somewhere else.
OpenMined’s PySyft handles that part of the process by allowing Google and the evaluator to approve the code in advance and to block sensitive parts of the evaluation from making external connections.
That process takes some work, although compute doesn’t appear to be the problem. The paper puts the overhead at less than 5% and instead points to legal agreements and code reviews between organizations as some of the bigger hurdles to running these evaluations.
The researchers eventually want to make attestation much less hands-on, comparing the goal to the HTTPS lock icon in a browser, where the security checks happen without the user having to deal with the underlying hashes and keys.
Google still has a hand in verification because, although the Confidential Space guest OS is open source and its build process has been externally validated, individual builds rely on private signing keys and can’t be independently reproduced. At the same time, Google’s own services are used to sign and verify the attestation report.
The paper puts the overhead at less than 5% and points instead to the legal agreements and code reviews between organizations as some of the bigger hurdles to running these evaluations.
So while the system reduces how much the two sides need to trust each other, it doesn’t remove trust entirely from the equation. Some of it simply moves elsewhere, including to the hardware itself. There is still some trust involved in the hardware itself, including the assumption that the cloud provider and the hardware maker aren’t working together to circumvent the protections.
Then there’s the benchmark itself. MLCommons points out that it still needs to be carefully managed, no matter how well the questions are protected.
The pilot ran Gemini 2.5 Flash Lite on a single H100 80GB Confidential GPU. Still, the researchers are already looking at clusters of H100 and B200 GPUs connected via encrypted links to eventually evaluate models that are too large to fit on a single GPU.
If this approach catches on, developers could have another way to look beyond the benchmark score, with some proof that the company behind the model didn’t see the test before the results came out.
If this approach catches on, a benchmark score could offer some proof that the company behind the model didn’t get to see the test first.
The post Google found a way to test Gemini without seeing the questions appeared first on The New Stack.
NVIDIA Groq 3 LPX is the interactive AI inference accelerator for the NVIDIA Vera Rubin platform. At the core of the platform is NVIDIA Vera Rubin NVL72, the...NVIDIA Groq 3 LPX is the interactive AI inference accelerator for the NVIDIA Vera Rubin platform. At the core of the platform is NVIDIA Vera Rubin NVL72, the most versatile machine ever built, delivering high throughput and interactivity across the widest range of AI workloads—from small to large models, both open and closed. Groq 3 LPX, when paired with Vera Rubin NVL72, extends the platform’s…
I’m Matt Burns, Chief Content Officer at Insight Media Group. Each week, I round up the most important AI developments, explaining what they mean for people and organizations putting this technology to work. The thesis is simple: workers who learn to use AI will define the next era of their industries, and this newsletter is here to help you be one of them.
Model triage is becoming one of the most important skills for the AI-native developer. I’ve argued all summer that the people getting the most out of frontier models are the ones disciplined enough not to run the best model by default. On Wednesday, Stripe and Ramp validated that idea 70 minutes apart: Stripe bought OpenRouter, and Ramp released its internal router.
Bloomberg puts the OpenRouter price tag above $7 billion, and Axios says it’s more than $8 billion in cash and stock. Stripe has not released the terms, so the details remain fuzzy.
While the acquisition made headlines, the architecture is the story. For the last couple of years, picking a model was something written into an application, a string in a config file, and swapping models took some work. A router changes that workflow. Stripe bought the layer, and Ramp built it. Both are betting their existing relationships give them a unique wedge to own this critical layer in the new AI stack.
OpenRouter is an endpoint serving more than 400 models from over 80 providers, processing more than 10 trillion tokens a day. Andrej Karpathy calls it the transfer switch for AI. Ramp’s Router does the same job on a smaller catalog and claims roughly 40% lower cost for the same output.
Both are betting the model name in a codebase is a liability. They’re mostly right. Back in June, I pointed to Mitchell Hashimoto, who found a standard coding task that cost about $1.50 on GPT-5.5 and roughly $9 on Claude Fable, with both producing equally acceptable results. A router automates that triage, making decisions on every request rather than only on those a developer explicitly configures.
This is becoming a large problem and a large opportunity. Our own Amanda Caswell reported this week that Anthropic’s /claude-api skill was burning about 200,000 tokens before answering a single question, and that loading its reference docs on demand instead of up front cut that to roughly 25,000. Hafiz Hassan wrote for us last week about why AI pipelines cost 10x more than the demo, and every culprit on his list is an engineering decision: system prompts resent every turn, whole conversation histories appended, oversized RAG chunks, raw JSON dumped into context. It’s a great practical guide, and none of the items Hassan identifies are procurement problems.
The token bill is generated by your code, which is why the tools to control it are arriving there as well.
Stripe is attacking the problem from the bottom up, through developers. The company’s investor letter, leaked Wednesday by Eric Newcomer, makes the argument directly: “Up until now, every developer has needed a straightforward and reliable way to manage their revenue pipeline, and serving this need gave rise to Stripe. Going forward, however, every developer will also need a straightforward and reliable way to manage their intelligence pipeline.”
Stripe wants to control AI spending through the long tail of developers. Ramp wants to control it through its existing relationship with finance.
Stripe has built this product before. Its payments business hides dozens of local payment methods behind a single API, routing each transaction to the payment method most likely to convert. The AI version is the same idea applied to models instead of payment networks.
Ramp is attacking it from the top down, through finance. The company bought the router.com domain and says its customers already buy quadrillions of tokens a month through Ramp. Founder Veeral Patel’s launch post pitches the service simply: “Monitor and control your AI bill across every provider.” Adam Wazzan sums up Ramp’s strategy better than I can: “when a CFO ships a product for CTOs.”
Stripe wants to control AI spending through the long tail of developers. Ramp wants to control it through its existing relationship with finance. Both are chasing what is rapidly becoming one of the largest line items in corporate technology budgets: tokens.
On X, Kabir Goel pushes back on Stripe’s framing. Routing tokens is a way to spend less, while Stripe’s other products are designed to help businesses make more.
“Stripe is just not where teams go to understand how much they’re spending,” he writes. “That’s pretty squarely Ramp territory.”
He has a point about where teams look today. Whether that’s still true three years from now is exactly what Stripe just spent billions betting against.
Whichever router you point at decides which model writes your code, and not every router is disinterested. Our own Paul Sawers flagged the problem in July when he covered the first wave of Cursor’s, Ramp’s, and Meta’s routers. Cursor backs Grok and Composer. Meta is building Muse Spark. Both have reasons to send work to their own models, and Paul quoted developer Elvis Saravia asking whether routing logic ought to be open source rather than a vendor’s private judgment call.
Stripe and Ramp do not sell models. OpenRouter CEO Alex Atallah says as much in Stripe’s own announcement: Developers “need a neutral layer to orchestrate and manage them all.” Investor Gavin Baker frames the opportunity the same way, arguing that Stripe can become the neutral infrastructure layer for AI, just as it became the neutral infrastructure layer for payments.
Neutrality isn’t free, though. Stripe takes a percentage of token spend, and Ramp wants your spending relationship, so “free through 2026” is a customer acquisition strategy with an expiration date.
The obvious objection is that vendor motives are the wrong thing to worry about, and routing quality is what really matters. That’s fair, and Towards Data Science published one of the best practical examples I’ve read. Pratik Rupareliya describes a routing layer that cut a support agent’s inference bill by 40% but also broke the product. A classifier sent “simple” queries to a cheaper model, but some of those “simple” queries were actually fraud investigations.
The cheaper model answered them confidently and incorrectly. Customers stopped using the agent, churn rose above baseline in month four, and retention costs were four to five times higher than the savings. It took three months to surface and another month to identify the cause. His fix was per-tier quality monitoring combined with an uncertainty-routed cascade, which ultimately settled at 35% savings without sacrificing quality. It’s an excellent article to read before diving into model routing.
So instrument the routing. Log what the router picks on every request and break out your quality metrics by the model that served them. Ramp Router reportedly records the model, provider, tier, tokens, latency, cost, and fallback attempts for every call. Stripe OpenRouter rankings have been a public version of that telemetry for years. You can get similar visibility with either approach.
Right now, the model is becoming an implementation detail. The competition is shifting to the layer that decides which model gets the job. Stripe and Ramp are betting that developers won’t care what sits behind the endpoint, so long as the bill is lower and the results are good enough.
The post Forget the model wars, Stripe and Ramp just started the router wars appeared first on The New Stack.
Modern vision-language models (VLMs) can support tasks such as visual question answering, captioning, and image-text reasoning. In practice, however, the data...Modern vision-language models (VLMs) can support tasks such as visual question answering, captioning, and image-text reasoning. In practice, however, the data needed to adapt these models may be distributed across institutions or organizations that cannot centralize their raw records. Federated learning provides a way to coordinate training across these data-local sites. For VLMs…
Teams customize their models to hit their targets for latency, speed, memory, and compute. With the open NVIDIA Nemotron family of models, developers can find...Teams customize their models to hit their targets for latency, speed, memory, and compute. With the open NVIDIA Nemotron family of models, developers can find the right-sized model for their needs. The new Nemotron 3.5 Lightning NVFP4 checkpoint, for example, preserves accuracy while unlocking up to 4x faster throughput. It’s compressed down to 22 GB from the 66 GB full precision checkpoint…
Anthropic announced it will embed invisible watermarks into text generated by new Claude models, including output produced through its API, coding tools and cloud partners. For developers, the mark offers another way to trace where AI-generated text or code might have come from, but it is not strong enough to prove its origin.
Laying out the plan in a support document, Anthropic said Claude models launched in the EU on or after Aug. 2, 2026, will include machine-readable marking from release. The company is working to add support to older models as well.
The marks will apply worldwide across supported Claude products, including the Claude API, Claude Code, Claude Cowork and Claude Tag. Text generated through AWS, Google Cloud or Microsoft Foundry will also carry the watermark when those platforms use a supported model. Because the mark is added at the model level, it follows the output into applications built on top of Claude — the same applications that are already reshaping how enterprises deploy AI infrastructure — although Anthropic cautions that some platforms and features may not support every type of mark.
The change follows the Aug. 2 start of Article 50’s transparency requirements under the EU AI Act, which require providers of generative AI systems to make synthetic output detectable in a machine-readable format. Anthropic signed the accompanying Code of Practice as a provider of both generative AI models and systems. OpenAI, Google, Meta, Microsoft and Mistral are among the other model providers that have committed to the code.
Yet, Anthropic is handling text and files differently. Text gets a watermark hidden in the words themselves, while supported files such as SVGs, PNGs and JPGs receive a digital signature using the C2PA standard. The file metadata can show that Claude processed an asset and whether the metadata has been altered.
“Because the watermark is part of the text, it will travel with the text when it’s copied and pasted elsewhere, and may persist through some editing,” Anthropic said.
“Because the watermark is part of the text, it will travel with the text when it’s copied and pasted elsewhere, and may persist through some editing.”
Anthropic has not explained how its text watermark works or said whether Claude uses KGW, a semantic version or another method. The company has also not shared any figures showing whether watermarking affects latency or adds to inference costs — a gap that matters for teams already wrestling with the hidden costs of agentic AI workflows.
Alex Cui, CTO and co-founder of AI detection company GPTZero, wrote in a technical explainer on X that watermarking systems fast enough to run on a streaming frontier model often follow the same general approach. One such technique, known as the KGW method, changes the probabilities the model uses when selecting its next token.
A language model normally calculates a probability for every token that could appear next. In a simplified watermarking system, a secret key and the preceding tokens are used to generate a hash, which divides the candidate tokens into two groups, often described as green and red. The model then slightly increases the probability of selecting one of the green tokens.
A detector with the same key can use the preceding text to reconstruct which tokens would have been favored at each position. A passage containing an unusually high number of those choices may carry the watermark.
Cui wrote that more advanced approaches can derive the watermark from the meaning of nearby text rather than an exact sequence of tokens, which may help the signal survive some paraphrasing because replacing a word does not always change the surrounding context.
“Their watermark needs to work token-by-token because they are streaming their text to users,” Cui wrote. “Many watermark methods plan sentences or paragraphs at a time, or change the text after it’s entirely written, in order to make their watermark robust to paraphrasers.”
Anthropic has not confirmed that Claude uses any of these methods, but streaming limits the techniques available because the model must add the signal while generating its response rather than rewriting a completed passage afterward.
Code presents a different problem because the model has fewer valid choices. Words can often be swapped or sentences rewritten without changing their meaning, but seemingly minor changes can break working code. That challenge intensifies as the AI coding era matures and more production code flows through model-assisted pipelines.
“There are some texts, like code, that cannot be arbitrarily changed; otherwise the code will break,” Cui wrote. “In those cases, the watermark needs to selectively change words in parts of the text that can tolerate synonyms,” such as variable names.
Code may also be difficult to track through a normal development workflow. Anthropic has not published tests showing how well its watermark survives those changes, so teams do not yet know whether a Claude-generated patch will remain detectable after passing through a pull request.
“In my testing, the watermarks don’t survive intense paraphrasing, especially if you combine word choice and syntax attacks.”
The same issue comes up when applications change Claude’s output before showing it to a user or committing it to a repository. Summarizing it with another model, translating it, splitting it into smaller sections, turning it into structured data or mixing it with database content could all make the watermark harder to detect.
Anthropic acknowledges this limitation. Editing, paraphrasing, translating or combining the response with other text may weaken or remove the watermark, while short excerpts may not contain enough of the signal to detect.
Cui wrote that determined users can attack a watermark by changing both the vocabulary and the structure of a passage.
“In my testing, the watermarks don’t survive intense paraphrasing, especially if you combine word choice and syntax attacks,” he wrote. Cui added that free paraphrasing tools he tested were able to bypass Google DeepMind’s SynthID text watermark.
Research supports those concerns. The “Watermarks in the Sand” paper found that, under defined assumptions, attackers can remove watermarks without severely damaging the quality of the content. The absence of a watermark does not show that Claude had no role in creating the content. The response may have come from an older model, may be too short to carry a detectable signal or may have been changed somewhere in an application pipeline. It may also have passed through a platform or feature that does not support that type of mark.
Finding a watermark does not prove authorship either. Claude may have proofread, translated or reformatted material written by a person. Anthropic says a detected mark means only that the content “may have been processed by Claude,” not that Claude created the underlying work.
“If Anthropic releases the watermark detector publicly, I think they defeat their own watermark. People find reliable watermark-removal strategies by testing against Anthropic.”
Anthropic plans to give users and third parties a way to detect its marks, but it has not said whether that will take the form of a local tool, a detection API or access limited to selected organizations. A public detector would be easier for developers to add to their applications, but it would also allow someone trying to remove a watermark to keep editing and checking the text until the signal disappears.
“If Anthropic releases the watermark detector publicly, I think they defeat their own watermark,” Cui wrote. “People find reliable watermark-removal strategies by testing against Anthropic.”
The secret keys behind the watermark create another challenge. A leaked key could make it easier to remove the mark or imitate it in text that Claude did not produce. The scenario echoes what happened when provenance attestations were turned into camouflage — a trust signal that was supposed to increase confidence instead became an attack surface.
“To avoid a large blast damage from this, you need to have a couple secret keys in rotation,” Cui wrote.
Key rotation would require detectors to recognize marks created with both current and retired keys, including those embedded in content generated months or years earlier. Anthropic has not explained how it plans to handle that history.
For developers, Claude’s watermark is best treated as another clue, not a replacement for audit logs or provenance tracking. Applications that need to show where an artifact came from can record the model ID, prompt version, response time and a hash of the original output, then log any changes made before it reaches a user or is committed to a repository.
The watermarking announcement arrives as Anthropic navigates deeper questions about what its models do in the wild. Recent incidents have exposed gaps between lab safety evaluations and real-world containment, and the company has publicly backed calls for the most powerful AI labs to slow down. Watermarking fits into that posture — a transparency mechanism rather than a safety guarantee — but its practical value depends on technical details Anthropic has not yet shared.
Anthropic tells customers to determine how Article 50 applies to their own products and says more technical documentation is coming. But until Anthropic shares those details, teams do not know how they will detect the marks, how key rotation will work or how well the watermark will survive common changes to code and application output.
The post Anthropic’s watermark survives copy-paste, but not the real dev workflow appeared first on The New Stack.
At the start of 2025, investors wondered whether Google could keep pace with OpenAI. By December, Alphabet was completing its best year on the stock market since 2009, helped by growing confidence in Gemini and Google’s broader AI strategy.
Much of that work came out of DeepMind, the British AI lab Google acquired in 2014 for about £400 million ($659 million in 2014). Now, Google is changing its leadership, while four of its best-known engineers are leaving to start an automated research lab.
Google announced Wednesday that DeepMind founder Demis Hassabis will step away from the lab’s day-to-day operations to become chair of Google DeepMind and chief scientist of Alphabet. Koray Kavukcuoglu, DeepMind’s chief technology officer and Google’s chief AI architect, will take control of Gemini model development, frontier AI research, the Gemini app and its developer teams.
At the same time, Jeff Dean, Sanjay Ghemawat, Oriol Vinyals and Quoc Le are also leaving Google to start Discovery Loop. As a public-benefit corporation, the company wants to use AI to automate scientific and engineering research. Google will remain involved as a founding investor and cloud provider. Just one person will now oversee everything from research to products.
In a January interview for CNBC’s podcast The Tech Download, Hassabis described DeepMind as the “engine room” of Google’s AI efforts. Hassabis said DeepMind develops Google’s core AI technology before it is distributed across the company’s products.
To get new AI into products faster, Google had to do more than update its models. DeepMind spent years reworking Google’s infrastructure so its AI work could move out the door quicker. That infrastructure push has extended to silicon, too — Google recently bet its inference future on a chip built for one model, a sign of how tightly the company is coupling hardware to its Gemini roadmap.
Hassabis said Google never struggled to invent new technology. After all, its researchers came up with the transformer architecture, which became the backbone of large language models. The problem was turning that research into products fast enough.
And moving faster is exactly what Google did. Hassabis pointed to Gemini 2.5, which landed in March 2025, as a turning point. Gemini 3 came out in November and put Google back in the mix with OpenAI and Anthropic — both of which have been making aggressive moves of their own to capture developer share.
By January, Hassabis said he and Google CEO Sundar Pichai were speaking almost every day, sometimes adjusting product plans and research roadmaps daily.
Kavukcuoglu has been at DeepMind for 13 years, started the deep learning team, and worked on projects like WaveNet and DQN. Now, model research, the Gemini app, and developer products are all on his plate. The flagship version of Gemini 4 remains unreleased after a planned June launch, according to Reuters. Hassabis confirmed the model’s name in his note to employees, saying Google was making progress on Gemini 4.
Dean joined Google in 1999 and helped create Google Brain before becoming a technical co-lead for Gemini. Working with Ghemawat, he developed systems including MapReduce, Bigtable, and Spanner. Dean was one of the primary designers of TensorFlow, while Ghemawat worked on the infrastructure behind Google Search and several generations of its distributed computing systems.
Vinyals and Le made foundational contributions to deep learning and Gemini. Along with former OpenAI chief scientist Ilya Sutskever, they co-authored the influential 2014 paper that introduced sequence-to-sequence learning with neural networks. Their new company plans to use that experience to work on machine learning itself, at least to start.
“We are building AI solutions that can automatically solve important problems in machine learning, science, and engineering,” Discovery Loop says on its website.
“We are building AI solutions that can automatically solve important problems in machine learning, science, and engineering.”
The idea that AI can automate scientific discovery is no longer speculative. OpenAI’s Astra recently proved 10 long-standing math and science theorems for about $2,000 in token costs — a data point that suggests Discovery Loop is entering a space where early results are already landing.
Discovery Loop is built around the idea that AI can automate the entire experimental cycle. At Y Combinator‘s Startup School in July, Dean said the system could run the entire process, from proposing and carrying out an experiment to evaluating the results and deciding what to test next.
Ghemawat told Wired that Google’s systems were designed to support products such as Search, advertising and large consumer applications. Discovery Loop wants to build specialized infrastructure around research instead.
“We wanted to build something differently than how things are built at Google right now,” he said.
“We wanted to build something differently than how things are built at Google right now.”
Rather than cut ties with the departing engineers, Google is investing in Discovery Loop and has signed a cloud partnership to provide the startup with computing capacity. That arrangement gives Discovery Loop access to the infrastructure needed to run large numbers of experiments without first building its own data centers. It gives Google a stake in anything the new company discovers and another major AI workload for Google Cloud.
Rather than cut ties with the departing engineers, Google is investing in Discovery Loop and has signed a cloud partnership to provide the startup with computing capacity.
Google has already lost Gemini co-leader and transformer co-author Noam Shazeer to OpenAI and AlphaFold researcher John Jumper to Anthropic. Worth noting, Alphabet shares fell more than 5% following Wednesday’s announcement.
Hassabis, meanwhile, will focus more of his attention on long-term AGI strategy and Isomorphic Labs, the drug-discovery company spun out of DeepMind.
In his message to employees, he said AGI now feels “close at hand” and that he wants more time to influence what happens next — a sentiment that carries extra weight as some of the most powerful AI labs face growing pressure to slow down.
The post Google’s four AI departures: “We wanted to build something differently” appeared first on The New Stack.
DeepSeek has launched DeepSeek-V4-Flash-0731, delivering a significant boost in agent performance without changing the model’s core architecture.
Following an announcement last week, the company made the update available as a public beta through DeepSeek’s API, and the open weights were published on Hugging Face under the MIT license later the same day.
Although the model itself hasn’t changed, DeepSeek says additional post-training is responsible for the performance gains, showing that meaningful improvements don’t always require a larger model.
🚀 DeepSeek-V4-Flash Official API is now LIVE in public beta!
— DeepSeek (@deepseek_ai) July 31, 2026
🔷 We’ve massively upgraded its Agent capabilities—benchmark scores are now far surpassing the V4-Pro-Preview. Check out the massive performance leap below! 👇
🔷 The official V4-Flash now natively supports the… pic.twitter.com/NUzOyxza2f
DeepSeek’s decision to release the production-ready weights under a permissive license gives organizations much more control over how they deploy and customize the model.
Although the model itself hasn’t changed, DeepSeek says additional post-training is responsible for the performance gains, showing that meaningful improvements don’t always require a larger model.
DeepSeek says V4-Flash-0731 uses the same architecture as the preview release, with 284 billion total parameters and 13 billion activated parameters per token.
This is much smaller than V4-Pro, which has 1.6 trillion total parameters and 49 billion activated parameters. For companies running agents at scale, the activated-parameter gap translates directly into inference cost — though model price alone doesn’t tell the full story.
Even though it is still the smaller model, DeepSeek says the updated Flash version now beats the earlier V4-Pro preview on several agent-focused benchmarks.
Even though it is still the smaller model, DeepSeek says the updated Flash version now beats the earlier V4-Pro preview on several agent-focused benchmarks.
The company reported 82.7 on Terminal-Bench 2.1, 54.4 on DeepSWE, and 70.3 on Toolathlon-Verified.
Early independent testing by Artificial Analysis found a lower Terminal-Bench 2.1 score of 79%, which suggests that DeepSeek’s reported numbers may not always match independent results.
DeepSeek also shared results from several internal tests, though they have not yet been independently verified. If those results hold up, they add to growing evidence that companies can get more performance out of existing models through post-training instead of simply making them larger.
The MIT license means organizations aren’t limited to using DeepSeek through its hosted API. The release adds to a trend towards open-weight models closing the gap with proprietary alternatives. That flexibility is paired with support for tools many developers already use. V4-Flash now supports the Responses API for building AI agents and multi-step workflows, and DeepSeek has published instructions for integrating the model into Codex-based development workflows.
For teams already using OpenAI-style APIs, that lowers the barrier to trying another model because they can evaluate it without making major changes to their existing setup.
The V4 technical report also covers inference improvements, like speculative decoding with DeepSeek’s DSpark framework, which are designed to make serving more efficient. When combined with self-hosted deployments, these features give infrastructure teams more ways to adjust performance for their own production needs.
This release reflects that companies are now finding new ways to improve model effectiveness without making models larger, and they’re competing on how those models are delivered. While many AI vendors focus on hosted APIs, DeepSeek continues to publish downloadable weights that organizations can run on their own infrastructure. Support for familiar API formats also makes it easier for teams to test open-weight models without revamping present workflows.
Companies are finding new ways to improve model effectiveness without making models larger, and they’re increasingly competing on how those models are delivered.
The post DeepSeek’s smaller model just outperformed its own flagship appeared first on The New Stack.
OpenAI has lowered API prices for two GPT-5.6 models only three weeks after their launch. On Thursday, the company announced that GPT-5.6 Luna is now 80% cheaper and GPT-5.6 Terra is 20% cheaper, while the price for its main reasoning model, GPT-5.6 Sol, stays the same.
“Major price cuts today,” OpenAI CEO Sam Altman writes in a post on X published on Thursday. “We want to offer the best price/intelligence tradeoff at every level.”
“We want to offer the best price/intelligence tradeoff at every level.”
Luna now costs $0.20 for a million input tokens and $1.20 for a million output tokens, down from $1 and $6. Terra is priced at $2 per million input tokens and $12 per million output tokens, reduced from $2.50 and $15. Sol’s price stays at $5 per million input tokens and $30 per million output tokens.
Developers using Luna do not need to change their processes, but their inference costs will go down. High-volume tasks will now be much cheaper to run, without requiring any code updates or model changes.
major price cuts today:
— Sam Altman (@sama) July 30, 2026
*80% drop for GPT-5.6 Luna, now $0.20 per million input tokens and $1.20 per million output
*20% drop for GPT-5.6 Terra, to $2/$12
*GPT-5.6 Sol gets Fast mode in the API, up to 2.5x the speed for 2x the price, same intelligence pic.twitter.com/erC6u4VoDR
This timing is unusual because AI vendors usually keep prices steady for several months after launching a new model family. OpenAI cut prices less than a month after GPT-5.6 became available on July 9.
…serving costs can be more important than small differences in benchmark performance between models.
The company says these price cuts were possible because of improvements to the infrastructure behind GPT-5.6, which lets the company offer “substantially more intelligence per dollar.”
These infrastructure upgrades were expected, however. A day before the price announcement, OpenAI shared an engineering overview that explained optimizations across the inference stack for Codex and ChatGPT Work.
OpenAI engineers rewrote the production GPU kernels, cutting serving costs by about 20%. They also redesigned Sol’s speculative decoding system, making token generation over 15% more efficient. The company updated its agent runtime as well, reducing repeated prompt computation by using prompt caching more during multi-step workflows.
Lately, developers are paying more attention to inference costs since agents often make dozens or even hundreds of model calls to finish a single task. For these workloads, serving costs can be more important than small differences in benchmark performance between models.
The elephant in the room is that the competition has intensified from overseas. Lower-cost open-weight models from Chinese AI companies like Moonshot are pushing commercial providers to show not just better performance, but also better pricing for production use. OpenAI and Anthropic know that leaning on performance just isn’t an option anymore, which is pushing them to match Chinese prices.
The issue here is that most of those steps don’t need a model like Sol, and Chinese labs have figured out how to pack better capabilities into efficient models; a helpful option for companies running through billions of tokens a day.
The ability to send the easy tasks to open models and save the pricey APIs for the tough stuff makes a difference; OpenAI is banking on its 80% price cut on Luna to narrow that gap. Suddenly, switching to self-hosted models doesn’t look worth the hassle.
These pricing changes come as both OpenAI and Anthropic keep adjusting the economics of their newest model families — a dynamic that played out across three companies in a single week earlier this month. Earlier this week, OpenAI raised GPT-5.6 Sol usage limits for ChatGPT Work and Codex after finding that long coding sessions used up allowances faster than expected. Anthropic has also made pricing changes and added premium inference tiers as enterprise customers move bigger agentic workloads into production.
This announcement highlights a trend in the industry for infrastructure. Now, every percentage point of serving efficiency can lead directly to lower API prices, turning cost optimization into a competitive advantage instead of just an engineering goal.
The post OpenAI slashes API costs amid rising global competition appeared first on The New Stack.
Moonshot AI has released the open weights for Kimi K3 on Hugging Face, giving developers access to one of the largest open-weight language models yet. The Monday release follows a wave of overwhelming demand that forced Moonshot to temporarily pause new API subscriptions. Now, organizations with the necessary hardware can deploy K3 themselves.
In its documentation, Moonshot describes the model as being built for “long-horizon coding and end-to-end knowledge work.” Another notable detail is that Kimi K3 uses an OpenAI-compatible API. Because teams can try the model without rebuilding their existing integrations, switching to K3 could be as simple as changing the endpoint and model name.
For engineers who have already built around OpenAI-compatible SDKs, that makes it much easier to evaluate K3 alongside existing commercial models. Taken together with the one-million-token context window, it’s clear the company is targeting engineering teams that already build around models like Claude Fable 5 and OpenAI’s GPT-5.6 Sol. While K3 is openly available, running it is another matter.
While K3 is openly available, running it is another matter.
The model uses a 2.8-trillion-parameter Mixture-of-Experts (MoE) architecture and ships in the hardware-friendly MXFP4 format. The weights alone occupy roughly 1.4 TB of storage, and practical self-hosted deployments require a distributed GPU environment — realistically eight or more servers equipped with eight NVIDIA H100 or B200 accelerators each.
That changes the conversation around open-weight AI. As The New Stack recently noted, the case for ownable models has grown stronger after Anthropic’s Fable 5 was pulled offline by a Commerce Department directive, a warning that access is not ownership.
Releasing the model weights and technical report of Kimi K3.
— Kimi.ai (@Kimi_Moonshot) July 27, 2026
Kimi K3 is our most capable model: a 2.8T MoE model with native visual understanding and a 1M-token context window.
New model architecture: 2.5x the intelligence per unit of compute, not just more params.
Alongside… pic.twitter.com/Yz5uWeMbIm
Instead of paying recurring API costs to OpenAI or Anthropic, organizations trade those operating expenses for significant investments in GPUs, networking, storage, power and operational expertise. That benefit is control.
For organizations operating under strict regulatory requirements, the trade-off may justify the infrastructure investment. For many others, managed APIs will potentially remain the more economical option. The economics of open-weight models at enterprise scale remain an active area of debate across the industry.
Moonshot positions K3 as a frontier-class model capable of competing with OpenAI’s GPT-5.6 Sol and Anthropic’s Claude Fable 5 on a variety of public benchmarks.
The developer community is already taking notice of these coding capabilities. As MindStudio recently noted, “If you want to understand why developers are paying attention to Kimi K3, the benchmark to look at is SWE-bench Verified… For most of its history, SWE-bench has been dominated by proprietary models.”
The company’s own documentation is notably candid about its ongoing limitations. K3 always runs with reasoning enabled and defaults to its highest reasoning-effort setting, though Moonshot has since added lower-effort tiers. It may also behave too proactively when prompts are ambiguous. Moonshot also cautions that switching models within an ongoing conversation can reduce response quality.
That type of transparency is refreshing, but it additionally reinforces that benchmark scores shouldn’t drive deployment decisions. Early hands-on comparisons, such as The New Stack‘s Fable 5 vs. K3 coding match-up, suggest K3 can match Fable 5 on programming tasks at roughly a third of the cost, but runs about four times slower.
Organizations evaluating K3 still need to test it against their own workloads. But early community sentiment shows promise; open-source developers are already successfully utilizing K3 for complex, system-level tasks like porting the Godot game engine to WebGPU.
Morningstar senior equity analyst Malik Ahmed Khan echoed that overall wariness about benchmarks. “While K3 constitutes progress, we’d hesitate to ascribe it near-parity with American frontier models, such as Fable 5, in actual tasks,” Khan writes in a research note published before the release of the model weights on Monday.
K3 also arrives under growing geopolitical scrutiny. Anthropic and U.S. officials have accused Moonshot AI of distilling outputs from American frontier models during training. Anthropic’s Head of Public Policy Sarah Heck characterized the practice as intellectual property theft, while White House Office of Science and Technology Policy Director Michael Kratsios publicly alleged Moonshot relied on Anthropic’s models during development.
Moonshot has denied the allegations. Huang Zhenxin, Moonshot’s head of enterprise business, told Chinese state media that K3’s performance gains stem from architectural enhancements — specifically Kimi Delta Attention and Attention Residuals — not distillation. Some industry analysts have also questioned whether the timeline supports large-scale distillation, noting that Fable 5 had only been publicly available since July 1 before K3 appeared on July 16.
Whether those claims are ultimately substantiated or not, they bring another consideration for enterprise buyers. Beyond performance and infrastructure costs, organizations evaluating K3 may also have to consider future compliance, procurement, and regulatory risks.
K3 matters because of where it’s aimed. Moonshot isn’t building another consumer chatbot; its documentation makes clear this model was built for enterprise coding agents, heavy knowledge work, and production systems. The fact that demand blew past Moonshot’s GPU capacity within 48 hours says it all: at this scale, infrastructure pressure is guaranteed, whether you’re making API calls or hosting the weights yourself.
The post Moonshot opens Kimi K3 weights — but few can run it appeared first on The New Stack.
NVIDIA Ising Calibration is an open source vision language model (VLM) designed to interpret diagnostic outputs from quantum processors and determine how they...NVIDIA Ising Calibration is an open source vision language model (VLM) designed to interpret diagnostic outputs from quantum processors and determine how they should be tuned to continue operating. This post introduces the latest model release, NVIDIA Ising Calibration 1.5, which advances AI-based QPU calibration by analyzing unfamiliar diagnostic results without prior training examples.
Nvidia CEO Jensen Huang used his first post on X to share a public letter backing frontier open-weight models. Signed by Microsoft, Meta, Hugging Face, and 22 other organizations, the letter argues that open models improve security, encourage faster innovation, and give countries and enterprises more control over their AI infrastructure. The timing of the Friday post is notable, with more organizations choosing to run open-weight models inside their own environments rather than relying entirely on hosted services.
“Open weights let every organization match the right model to the right job at the right cost, reserving frontier-scale capability for genuine frontier problems and running efficient, specialized models everywhere else.”
The letter draws a comparison between open-weight AI and open-source software, arguing that broad access has driven decades of software innovation. It says the same principle applies to AI. Instead of sending every request to a hosted model, organizations can download an open-weight model, run it on their own infrastructure, customize it for a specific workload, and keep sensitive data behind their own firewall.
Nvidia has already been moving in this direction — its leaders have described a future where local and frontier models split the work by cost, speed, and control. As the coalition argues, “Open weights let every organization match the right model to the right job at the right cost, reserving frontier-scale capability for genuine frontier problems and running efficient, specialized models everywhere else.”
For my first post, I’m sharing a letter @NVIDIA signed on why open models matter.
— Jensen Huang (@JensenHuang) July 24, 2026
AI will transform every industry, power every company, and be built by every country.
Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty.… pic.twitter.com/t02bi51N4C
One isn’t necessarily replacing the other, and many enterprise teams already use both, choosing between them based on cost, performance, compliance requirements, and where the workload needs to run.
The letter underscores this reality for infrastructure teams, noting, “As organizations invest in AI, they want to know that they will not become locked into a single provider or lose the knowledge and capabilities they build over time.” That vendor lock-in concern is already reshaping deals — Anaconda’s recent acquisition of Kilo was framed explicitly around enterprises’ reluctance to depend on a single AI provider.
The timing is notable. Washington is weighing new restrictions on some Chinese AI models, including Moonshot AI’s Kimi K3, even though the Trump administration’s AI Action Plan described open models as a strategic advantage for the United States.
That split extends to the AI industry itself, though critics differ in their specific concerns. OpenAI’s Dean Ball has publicly warned about the broad economic and regulatory risks associated with Chinese open-weight models. Anthropic’s Sarah Heck, meanwhile, has specifically backed the White House’s allegations regarding intellectual property theft through distillation.
“Relying solely on closed models is not inherently safe: they can be breached, misused, or fail in ways that outsiders cannot detect. And concentrating advanced AI capabilities behind a small number of closed models compounds that risk.”
The companies that signed the letter argue that continuing to develop frontier open models is part of maintaining U.S. leadership in AI. They argue that security through obscurity is fundamentally flawed, stating: “Relying solely on closed models is not inherently safe: they can be breached, misused, or fail in ways that outsiders cannot detect. And concentrating advanced AI capabilities behind a small number of closed models compounds that risk.”
Palantir and Nvidia have already operationalized this argument, building an engine that runs open Nemotron models inside air-gapped government networks.
Distillation, one of AI’s most contentious topics, was also addressed. Widely used by researchers and model builders to create systems that are cheaper to run and easier to deploy, the letter states that it should be treated as a legitimate research technique rather than intellectual property theft.
The issue has taken on new significance after the White House accused China’s Moonshot AI of specifically distilling Anthropic’s Fable model to build Kimi K3 — an allegation the company has denied.
Distillation is one way teams build smaller models that can run on private infrastructure, edge devices, or lower-cost GPU clusters. Kimi K3’s launch showed just how much GPU capacity these deployments demand — Moonshot had to freeze new subscriptions within 48 hours after demand overwhelmed its available inference capacity. Any new restrictions on that process could affect how enterprise AI systems are built and deployed.
“As organizations invest in AI, they want to know that they will not become locked into a single provider or lose the knowledge and capabilities they build over time.”
Every enterprise that chooses to self-host, fine-tune, or customize an open-weight model needs infrastructure to run it. Nvidia also has clear business reasons for supporting a future in which both proprietary and open-weight models thrive.
The company supplies the hardware powering both hosted frontier models and enterprise-owned deployments. Organizations that use a mix of commercial APIs and self-hosted open models ultimately expand demand for AI infrastructure across the board — a dynamic that is already redirecting enterprise IT budgets toward AI hardware at the expense of traditional software spending.
The post Jensen Huang made his first X post. He used it to lobby Washington about open-weight AI. appeared first on The New Stack.
The AI industry has largely focused on language-based approaches, using transformers trained on massive datasets to predict words or fill in missing information.
As AI expands into more structured fields, however, the limitations of text-generation models are becoming clearer. Nvidia is pursuing a different approach.
The company just dropped JEPA-DNA on Hugging Face. It’s a genomic foundation model that adds a latent-space prediction objective alongside MLM.
This release is a win for hybrid architectures that go beyond purely generative training. It’s the kind of shift Executive Chairman of AMI Labs, formerly Meta’s Chief AI Scientist, Yann LeCun, has been pushing for years. He’s championed predictive architectures as a general alternative to next-token prediction, which researchers are now applying to biology.
Conventional genomic base models have historically mirrored NLP models, relying purely on MLM, masking parts of a DNA sequence and forcing the model to guess the missing literal tokens. This approach favors local token reconstruction and teaches the model the basic “syntax” of the sequence, but it frequently struggles to grasp the wider functional “meaning.”
JEPA-DNA changes the paradigm. The newly released checkpoint, JEPA-DNA-DNABERT2, serves as a model-agnostic continual pre-training framework. It couples standard token-level DNA language modeling with JEPA, adding a second learning objective.
Instead of forcing the architecture to reconstruct missing tokens, JEPA-DNA supervises the model’s global sequence embedding in a latent space.
Instead of forcing the architecture to reconstruct missing tokens, JEPA-DNA supervises the model’s global sequence embedding in a latent space. It predicts the functional representation of masked genomic segments rather than their literal, character-by-character makeup.
Token prediction is still part of the training process, but it is no longer the model’s only learning objective. The model also learns the basic structure of the data, proving that predictive architectures can produce representations that are easier to work with for biological tasks without sacrificing generative capabilities.
Token prediction is still part of the training process, but it is no longer the model’s only learning objective.
The model builds on DNABERT-2, a 117 million-parameter model developed by Zhihan Zhou and collaborators. Nvidia layers its continual pre-training approach on top of that architecture, letting the model learn from both token-level predictions and latent-space representations.
Nvidia has released the model globally for non-commercial research. According to the company, the model is meant to support research workflows, including feature extraction, linear probing, continual pre-training experiments, and zero-shot scoring of DNA sequence changes. It isn’t a diagnostic tool or a clinically validated medical product.
DNA contains patterns and relationships that aren’t captured by sequence prediction. JEPA-DNA supplements masked-token prediction with a latent-space objective intended to capture broader sequence-level information.
Ultimately, models like JEPA point to a new path for AI that combines next-token prediction with other ways of learning and helping models build a deeper understanding of complex systems.
The post Nvidia’s new DNA model learns what token prediction misses appeared first on The New Stack.
Login