Microsoft is turning Fabric, its integrated data platform, into the place enterprise agents go to learn how a company works, whether that’s what happened last quarter or what’s happening on the factory floor right now.

On Tuesday, at its FabCon and SQLCon data conference in Barcelona, Spain, Microsoft outlined the next step in its vision for how the products that make up Fabric can simplify all of this information gathering.

Microsoft, for example, described how Fabric IQ, the context layer inside Fabric, now feeds Microsoft 365 Copilot by default. Outside agents can query it over MCP. Power BI will soon be able to turn the same definitions into apps. And ontologies, which add business rules to the mix, moved further into preview.

As Microsoft Fabric CTO Amir Netz said in a press briefing after the keynote, “Agents are a very, very strange animal, and I always like to say it’s like Drew Barrymore from 50 First Dates. Every time they open their eyes, they forget everything that happened before.

“They don’t know where they are, so the first thing we have to do is tell them where they are. You are now working for Microsoft. You are now working for Wells Fargo. You are now working for Emirates.”

“Agents are a very, very strange animal, and I always like to say it’s like Drew Barrymore from 50 First Dates. Every time they open their eyes, they forget everything that happened before.”

Customers tend to find this out the hard way. Yitzhak Kesselman, the corporate vice president who runs Fabric IQ, tells The New Stack that companies unify their data, run models on it, and then look at the answers.

“Customers that are more advanced on their journey have their own evals for their agent,” he says, adding that what they see sometimes isn’t what they expected. “Then [customers] will understand: ‘Okay, now I need to create the context for my agents.'”

Kesselman says he met with more than 320 companies last year, and the pressure to get there comes from the business side, which asks to “‘show me the value of those agents,'” he says, “‘the before and after.'”

Arun Ulag, Microsoft’s executive vice president for Azure Data, said in the keynote that coding agents work because they have “the code, the repos, the change history, the specs, the tests.” But outside of coding, most enterprises have nothing comparable, he argued.

Fabric IQ is one of four pieces of what Microsoft calls Microsoft IQ (this is Microsoft, after all, so there’s always a lot of different names and products involved).

Work IQ covers email, Teams, and SharePoint. Foundry IQ covers documents and manuals. Web IQ covers the public internet.

Fabric IQ, Ulag said, “focuses on the state of your business and how your business actually runs.” His announcement blog post describes it as combining “unified data from OneLake, trusted metrics from Power BI semantic models, and operational context from ontologies and real-time intelligence.”

Getting the data in

“Everyone wants to jump to AI, but then they realize, ‘Oh, I need data for it,'” Kesselman says. “They need data from their business applications, their structured data. They need to have real-time data.”

OneLake, Fabric’s storage layer, is at the core of all of this, and for the system to provide context, enterprises have to feed it data from across all the first-party and third-party services they use.

Fabric’s shortcuts and mirroring, which connect or replicate external sources into OneLake, are free of charge, and Netz said storage under management in OneLake is growing 300% year over year.

What’s new this week at FabCon and SQLCon Barcelona 2026

On Tuesday, the company announced a two-way integration with Salesforce Data Cloud 360, mirroring from Google BigQuery in general availability, a ClickHouse workload that runs that engine directly against OneLake, and dbt’s Fusion engine, due in Fabric in the next couple of weeks.

Simply copying data won’t satisfy security teams, though. Mirrored security roles, which reach public preview in the coming weeks, bring permissions along with the data. Access roles defined in Snowflake, for example, will show up in Fabric with the same members and the same table permissions.

As Microsoft’s Shireen Bahadur, who demonstrated the feature in the keynote, put it, “We’re not just solely replicating security rules. We’re actually actively enforcing and preserving those roles.”

“We’re not just solely replicating security rules. We’re actually actively enforcing and preserving those roles.”

OneLake isn’t a one-way street. Enterprises can also pull data out and use it in other tools.

OneLake data, after all, is stored in open formats and has open APIs, and Netz said on stage that any engine that reads them can use it. “If you want to use this data that we help people to get for free with our competitors’ tools, go for it. I’m not happy about it, but go for it,” he said.

IQ sharing, now in preview, lets an organization share governed tables, files, Markdown agent instructions, and RDF ontologies with another tenant without copying, with an expiration date. That’s something Fabric users have wanted for a while, and the keynote announcement drew plenty of applause from the thousands of data professionals in attendance.

Past, present, and future

“Not only what we have in the past, not only what’s happening in the present. We also want the agents to understand what we want to happen in the future,” Netz said in the keynote.

The past is semantic models. A semantic model is the layer under every Power BI report that defines how a metric like revenue is calculated, how entities relate, and which tables supply the numbers. Netz said Power BI users have already created 22 million, and Ulag called them “the core of Fabric IQ.”

The present is real-time intelligence, Fabric’s streaming and event stack, which Kesselman also runs. He recalls two CIOs telling him the same thing that day in Paris. “‘There is no AI without RTI;’ there’s no AI without real-time intelligence,” he says. “If you don’t have the streaming data… data that is fresh, you will run your LLMs on data which is like hours or days old.”

“‘There is no AI without RTI;’ there’s no AI without real-time intelligence.”

A signal on its own isn’t enough, though: “You want to take the signal that you get now… the event that comes now, and look historically, okay, is it an anomaly or not?” Kesselman says. That’s why the keynote showed batch copy jobs and event streams feeding each other, so an agent can judge what just happened against what usually happens.

The future is Fabric Planning, which went generally available in July and got a performance and feature update this week. It’s Microsoft’s tool for budgets, forecasts, and targets, the numbers a business wants to hit rather than the ones it has already recorded.

A plan is a Fabric item like a lakehouse or a notebook, and it borrows its measures from the same semantic models the reports use, so revenue means the same thing in the forecast as it does in the dashboard. In the demo, a change to the inputs cascaded through a model of about 13 million cells.

What makes these work is ontologies: Formal descriptions of the things a business deals with, how they relate, and the rules for handling them. Netz called them “semantic models plus plus” on stage, and in a later press briefing, he used an airline to explain the plus.

“If you are an airline, you have planes, you have pilots, you have ground crews, you have airports, you have luggage,” he said. Those are the entities.

The relationships between them “are way more than data relationships,” he said. “It’s not only to say, ‘Oh, I have a match with a foreign key and a primary key between a pilot and a plane,’ but it could be a policy relationship. Which pilot can fly which plane based on their certification, or is the pilot allowed to fly this plane right now based on the rest hours in the last 24 hours?”

But ontologies don’t only define nouns; they also define verbs. “I can ground a plane. I can redirect the plane. I can assign a plane a gate.”

Creating those ontologies, which have to map the various terms that a company might use for the same thing to a single entity, can be extremely time-consuming. But since they are so core to this project, and to allowing agents to reason over data, Microsoft built a tool to generate them automatically.

Putting it to work

Fabric IQ in Microsoft 365 Copilot Chat and Cowork, the mode for delegating multistep tasks that Microsoft built on the technology behind Anthropic’s Claude Cowork, is generally available as of Tuesday. It answers business questions from Power BI semantic models and reports; it’s on by default for Fabric and Power BI customers, and Microsoft says it doesn’t consume additional AI tokens.

But business users don’t just want to chat with these models. In this age of vibecoding, they also want to generate applications that use all of this data.

Power BI Desktop is getting an app-creation experience in preview in the coming weeks. Users start from a semantic model, describe the app, and then have Copilot generate and publish it to Fabric.

Unlike with a report, Ulag wrote, these apps “can accept inputs, write back data, preserve shared state, and support operational workflows.”

They’re Fabric Apps built on Rayfin, the open-source SDK Microsoft introduced at Build. Power BI Pro and Premium Per User customers get them with a Fabric database of up to 1GB per app at no additional cost.

Netz confirmed in the briefing that “there’s no core difference” between the two, and Ulag added, “What gets created is a Fabric app, right? And that’s it.”

For agents outside Copilot, Fabric IQ MCP is generally available now, with six read-only tools for finding semantic models and reports, reading their schemas, and running DAX queries against them.

Ontology MCP tools, currently in preview, expose ontology definitions and queries. Fabric data agents can now, also in preview, use an ontology as their context source. And the context reaches Microsoft Foundry, Copilot Studio, and GitHub Copilot through the same layer.

Developers can drive the new data engineering agent from their own tools.

Built on the Osmos technology Microsoft acquired in January, it takes on long-running work like migrations and ETL, and can be started and steered from GitHub Copilot, VS Code, Codex, and Claude Code.

Frontier models will write that code, Netz said on stage, and “it won’t fail,” but you don’t know whether the results are right.

Everyone has a context layer now

This year, Databricks shipped Genie Ontology, Snowflake shipped a Horizon Context Layer, Google shipped a Knowledge Catalog, and Salesforce shipped a headless Data 360, which it describes as a context service. Databricks CEO Ali Ghodsi summed up the pitch at his own conference in June as AI having a context problem rather than an intelligence problem.

Microsoft’s version turns on by default inside Microsoft 365 Copilot and starts from 22 million semantic models. Microsoft also said this week it will contribute to Apache Ossie, the vendor-neutral semantic metadata standard Snowflake started, and that it wants DAX, Power BI’s formula language, recognized as an Ossie query language.

When Microsoft made the context argument at Build in June, the pieces that let an agent act on that context were mostly on a roadmap. Now the answers are generally available, and the actions are in preview.

Asked how Fabric copes when the user is no longer one person running a dashboard but a fleet of agents, Kesselman points to observability.

“Fabric really allows you to have this kind of holy grail of combination of both kinds of system data, how the agent runs, but also the business data,” he says, so a company can check whether an agent did what it was supposed to do and what that did to the business. “There’s more in this area that will come in the next few months.”

The post Microsoft Fabric is where AI agents learn how the business works appeared first on The New Stack.

If you rely too heavily on one AI provider, switching can mean rebuilding workloads and moving data while you face untangling integrations. A new vendor-neutral group aims to help organizations keep their options open. That’s the goal of the Eclipse Foundation, which announced the formation of the Sovereign AI Foundation on Wednesday.

The new body will serve as a vendor-neutral initiative to help member organizations manage AI platform lock-in, monitor cyber risk, and maintain control of their data. The group has been active on Eclipse’s project site since July.

An initial community of 17 organizations comes together to advance AI sovereignty through open-source collaboration.

The foundation’s mission is to help organizations “navigate AI dependencies” (as in dependence on any single AI service, not control over where application dependencies get tied to a particular model’s DNA… although, by extension, that too), where dependence can limit a software engineering team’s ability to control where data is processed, choose which technologies to use, or change direction as costs, requirements, and provider terms evolve.

A core rationale for AI sovereignty control

Executive director at the Eclipse Foundation, Mike Milinkovich, tells The New Stack that the risk of not adopting an open approach to AI sovereignty control depends on how deeply AI is embedded in an organization and how difficult the underlying systems are to change.

“The deeper the dependency, the greater the potential risk,” Milinkovich says. “But however embedded the underlying systems are, the costs show up in predictable places: higher prices that make switching difficult, and the challenge of rebuilding integrations, moving data and workloads, documenting systems after the fact, compliance delays, and operational disruption.”

He underlines his point and says part of the Sovereign AI Foundation’s work will be to “compare real-world experiences” and build stronger evidence around those risks.

“The deeper the dependency, the greater the potential risk. But however embedded the underlying systems are, the costs show up in predictable places.”

The Sovereign AI Foundation will work to give members an open forum to understand AI dependencies, compare experiences, and develop practical guidance for decisions about data, models, infrastructure, and operations.

Can we move beyond the AI hype-cycle now?

Acknowledging what he calls the pre-IPO hype cycle around the promotion of frontier models, Milinkovich calls for a concerted focus on where the real value in AI lies: enterprise and industrial applications that deliver measurable business value, without vendor lock-in.

“AI will not deliver on its potential if it is based entirely on a dependence on a relatively small number of providers for critical AI models, platforms, and infrastructure,” Milinkovich insists. “As AI becomes part of the systems organizations rely on every day, that dependence can leave them with too little visibility into where their data goes, which models use it, or how easily they can switch when costs, terms, or requirements change.”

Choice, control &  freedom of action

He asks us to understand how far the Sovereign AI Foundation will go to find practical ways to “preserve choice and control,” with a vision of delivering open-source and open-weight solutions that allow freedom of action.

As admirable and philanthropic as this sounds — and coming from a Brussels-based foundation whose founding members are mostly European — is there any America-first idealism here, given that open-weight Chinese models are also openly available?

“This is not about China or any single provider. The same questions apply to AI technologies from the United States, Europe, Asia, or anywhere else,” confirms Milinkovich. “Closed models can limit visibility, while access to model weights alone does not provide the transparency, control, or the independent governance that organizations may need. The real question for our members is whether they can understand, adapt, and, if necessary, replace the systems they depend on.”

17 founding members

As noted above, this initiative launches with 17 participating organizations spanning technology, industry, research, and the open source community: France’s CEA LIST,  Thales and Kentyou, Germany’s EclipseSource, Open Elements, TypeFox, Vector Informatik, and Bosch, Italy’s Engineering Ingegneria Informatica and Eurotech, Sweden’s Ericsson, India’s Infosys, Belgium’s KU Leuven, Japan’s Renesas Electronics, Malaysia’s The IO Foundation… and then there’s North Carolina’s Red Hat, and the University of York in the UK.

The Sovereign AI Foundation focuses on five areas:

• Mapping real-world use cases: Building a shared view of how members deploy, evaluate, and pilot open-source AI.
• Assessing technologies and trends: Monitoring developments across models, evaluation frameworks, inference infrastructure, and regulation.
• Developing practical resources: Producing whitepapers, blueprints, landscape analyses, and reference materials.
• Building a peer community: Connecting organizations at every stage of open source AI adoption.
• Translating research into action: Turning outcomes from Eclipse Foundation research initiatives, including EU-funded projects, into insights that members can apply.

A mission to tackle data control, cyber safety, or lock-in?

There’s a lot to unpack here in an organization only just out of the blocks; it begs the question whether the Sovereign AI Foundation will focus on data control challenges and efficiency, cyber safety, or platform lock-in challenges.

“It’s all three, and they reinforce one another,” Milinkovich says. “It is an efficiency issue if moving to a better or less expensive model requires rebuilding the workload. It is a security issue if organizations lack the visibility needed to assess risk. And it is a lock-in and resilience issue if critical operations become so tied to one provider that changing direction is costly or disruptive.”

“The need here is clearest wherever AI is used in systems with long lifecycles, demanding safety and security requirements, or sensitive data.”

He explains that “the specific needs vary”, so a manufacturer may need AI to keep operating when connectivity is limited, while a public agency may need to explain how an AI-supported decision was made.

“The need here is clearest wherever AI is used in systems with long lifecycles, demanding safety and security requirements, or sensitive data. That includes automotive, manufacturing, telecommunications, aerospace and defense, energy, and the public sector,” adds Milinkovich.

Interest will extend well beyond Europe

He confirms that the foundation’s initial participants do “reflect that mix”, as companies including Bosch, Vector, Renesas, Ericsson, and Thales put sovereignty high on the agenda. But he qualifies that dependence on critical AI technologies is a global concern, “so we expect interest well beyond Europe” in the weeks and months ahead.

Milinkovich offers a selection of still-forming use cases that he thinks are already solid. 

• EclipseSource is working with industrial customers on AI-enabled development tools like Eclipse Theia AI, where teams need flexibility over models and deployment.
• The Open VSX Registry provides vendor-neutral access to thousands of extensions for a growing range of open source and AI-enabled development tools.
• Eurotech is using Eclipse-governed projects to bring AI to the edge, where systems interact with physical equipment and must remain secure, reliable, and maintainable for years.
• Eclipse LMOS, originally developed at Deutsche Telekom, supports production multi-agent systems at significant scale and underpins multiple global deployments today.

“Across these examples, the common approach is modularity: keeping models, infrastructure, and applications separate enough that one can change without rebuilding everything. Our job is to capture what works and turn it into guidance others can use,” affirmed Milinkovich.

The Sovereign AI Foundation team says that alongside the 17 founding members, additional organizations are in the process of joining, and it invites eligible Eclipse Foundation member organizations to participate.

The post Eclipse wants companies to be free to switch AI providers. Today, doing so can mean a costly rebuild. appeared first on The New Stack.

Parallel work, model-family isolation, reversible changes, and GPU-backed validation shaped an open source project designed around coding agents NVIDIA TensorRT...

Parallel work, model-family isolation, reversible changes, and GPU-backed validation shaped an open source project designed around coding agents NVIDIA TensorRT Model Connect is an open source collection of AI model reference implementations in C++, built on top of NVIDIA TensorRT. It began with a practical question: could the performance of the NVIDIA inference stack be made accessible to…

Source

Fireworks Research launched Ember-1 on September 23 as a research preview. Built on Moonshot’s open-weight model, Kimi K3, it claims it matches Kimi K3’s quality using roughly 40% fewer tokens. Fireworks says it “learned to cut unnecessary reasoning while keeping the thinking that matters.” Reasoning tokens bill as output, so a model that thinks less should cost less.

There’s a catch, though. On OpenRouter, Ember costs $3 per million input tokens and $15 per million output tokens. Fireworks charges the same for Kimi K3, but other providers sell Kimi for as little as $1 per million input tokens and $9 per million output tokens.

I ran both models through OpenRouter on Fireworks, so both were billed at $3 and $15, and I also calculated what Kimi would have cost at the lower price.

It doesn’t matter how much cheaper a model is if it isn’t accurate. I wanted to know whether Ember’s shorter thinking holds up. If it cuts reasoning it actually needs, accuracy should slip first on the hardest problems. So I built tests that get progressively harder and ran each one five times, the same consistency check I’ve been adding to my recent testing.

The tests

I called both models through OpenRouter with identical prompts and their default reasoning settings. I routed both to Fireworks to keep the speed comparison fair. Each test ran five times per model, and I logged reasoning tokens separately from the rest of the output.

• Logic puzzles – Three puzzles of increasing size, with 4, 5, and 7 engineers and one solution each. Bigger puzzles need longer chains of reasoning, so cutting the thinking should hurt first.
• Deploy scheduling – Twelve services with dependencies, one deploy per team at a time, and two blackout windows. The model has to find the fastest possible rollout: 17 hours.
• Probability – Five questions about a retry system whose server flips between healthy and degraded, plus a circuit breaker. Every answer is an exact fraction, which I checked against a simulation of 2 million requests.

I confirmed every answer key with two independent methods before either model saw it. I’ve included the prompts at the end of this article for anyone who wants to replicate these tests on their system.

Logic puzzles

Both models solved all three puzzles on all five runs.

Ember averaged 13,630 reasoning tokens, 3 minutes 46 seconds, and $0.27 per run. Kimi averaged 16,679 reasoning tokens, 12 minutes 26 seconds, and $0.34. That’s 18% fewer reasoning tokens for Ember. Though not included in the marketing claims, Ember was much faster than Kimi. Kimi’s slowest run took nearly 20 minutes.

Deploy scheduling 

Both models found the 17-hour schedule every time, and every schedule passed my checker.

Ember averaged 6,543 reasoning tokens, 1 minute 29 seconds, and $0.10. Kimi averaged 7,792 reasoning tokens, 4 minutes 46 seconds, and $0.13. That’s 16% fewer reasoning tokens and significantly faster speed.

Probability

This test produced the only miss. Kimi answered all five questions correctly on every run. Ember got them all right four times. On the fifth, it made a small arithmetic slip on the first question, 0.94619 instead of 0.94629, and that error carried into two other answers.

It also saved the most. It averaged 6,242 reasoning tokens, 1 minute 47 seconds, and $0.13. Kimi averaged 9,682 reasoning tokens, 6 minutes 48 seconds, and $0.19. This was the first time Ember came close to its marketing claim of 40% fewer tokens. Once again, Ember was significantly faster.

Results

Test (5 runs each)	Ember-1	Kimi K3
Logic puzzles	5/5 perfect, 3:46, 13,630 reasoning / 17,766 total out, $0.27	5/5 perfect, 12:26, 16,679 reasoning / 22,553 total out, $0.34
Deploy scheduling	5/5 perfect, 1:29, 6,543 reasoning / 6,822 total out, $0.10	5/5 perfect, 4:46, 7,792 reasoning / 8,365 total out, $0.13
Probability	4/5 perfect, 1:47, 6,242 reasoning / 8,365 total out, $0.13	5/5 perfect, 6:48, 9,682 reasoning / 12,381 total out, $0.19
Perfect runs	14 of 15	15 of 15
Total cost, both on Fireworks ($3/$15)	$2.48	$3.26
Total cost, Kimi at cheapest price ($1/$9)	$2.48 (no cheaper provider)	$1.96

Kimi K3 had 15 perfect runs out of 15, and Ember-1 had 14. Ember-1’s one miss was an arithmetic slip on the probability test; it doesn’t seem too significant to me since it was one out of 15.

Something I noticed during my testing that wasn’t in the marketing claims was that Ember-1 finished each set of tests 3.4 times faster than Kimi K3. As for the reduced reasoning token usage, Ember-1 used 23% fewer reasoning tokens. This helped keep costs down. In total, it cost $2.48 compared to Kimi K3’s $3.26 on Fireworks, which makes it 24% cheaper. 

One caveat is that Kimi K3’s cost depends on the provider. At its lowest listed price of $1 per million input tokens and $9 per million output tokens, the same runs would have cost $1.96, less than Ember-1.

What do I think?

Ember-1 is about as accurate as Kimi K3 and uses far fewer reasoning tokens. The lower costs on Ember-1 on this test aren’t as important because a user who wants to use Kimi K3 can route to a cheaper provider on OpenRouter, which brings the price down below Ember-1’s.

One thing I know about Kimi K3 is that it’s slow. Cheap and slow. My speed numbers come from Fireworks’ standard endpoint, though, and I didn’t test the cheaper providers. If you have time and want to pay less, Kimi K3 wins. If you want almost identical results to Kimi K3 at a much faster rate, use Ember-1. 

The prompts

Each prompt ends with a fixed answer format so grading can be automatic.

Logic puzzles

Solve all three logic puzzles below. Each has exactly one solution.

PUZZLE SMALL: 4 engineers (Ava, Bo, Cleo, Dev) each own exactly one server. Each server has a rack position (1, 2, 3, 4, numbered left to right), an operating system (Debian, Alpine, Fedora, Ubuntu), a role (cache, queue, proxy, db). No two servers share any of these values.

Clues:

If the server in rack 3 runs Fedora, then the Debian server is in rack 3.

The server in rack 1 is Dev’s.

Exactly one of these is true: the server in rack 4 runs Ubuntu, or the Alpine server is not Bo’s.

Cleo is the proxy.

The db server is Dev’s.

Ava runs Alpine.

The queue server is not Ava’s.

Bo is in rack 3.

Bo and the Ubuntu server are in neighboring racks.

PUZZLE MEDIUM: 5 engineers (Ava, Bo, Cleo, Dev, Eun) each own exactly one server. Each server has a rack position (1, 2, 3, 4, 5, numbered left to right), an operating system (Debian, Alpine, Fedora, Ubuntu, Arch), a role (cache, queue, proxy, db, build), a replication data center (Oslo, Lima, Pune, Accra, Perth). No two servers share any of these values.

Clues:

Cleo is the db.

Exactly one of these is true: the cache server runs Arch, or the server replicated to Perth is the proxy.

The server in rack 5 runs Fedora.

Exactly one of these is true: the server in rack 2 is not the proxy, or Cleo runs Debian.

Exactly one of these is true: the server replicated to Pune is the db, or Dev is in rack 3.

The server in rack 1 is the db.

The proxy server is in rack 5.

The server in rack 4 runs Ubuntu.

The cache server and the server replicated to Pune are in neighboring racks.

Ava is the cache.

The server replicated to Accra is the queue.

The server replicated to Lima is not in rack 3.

Eun runs Alpine.

The queue server is not in rack 2.

The Fedora server is not Dev’s.

The server in rack 2 is not replicated to Lima.

PUZZLE LARGE: 7 engineers (Ava, Bo, Cleo, Dev, Eun, Finn, Gia) each own exactly one server. Each server has a rack position (1, 2, 3, 4, 5, 6, 7, numbered left to right), an operating system (Debian, Alpine, Fedora, Ubuntu, Arch, Rocky, NixOS), a role (cache, queue, proxy, db, build, metrics, auth), a replication data center (Oslo, Lima, Pune, Accra, Perth, Quito, Riga). No two servers share any of these values.

Clues:

The build server is Cleo’s.

The proxy server does not run Fedora.

The server replicated to Quito is Finn’s.

The server in rack 6 is replicated to Lima.

The Alpine server is exactly 4 racks to the right of the auth server.

The server replicated to Oslo is Dev’s.

The Debian server is replicated to Quito.

The server replicated to Accra is not the build.

The Rocky server is exactly 1 rack to the right of Gia.

Exactly one of these is true: the Ubuntu server and the server replicated to Pune are in neighboring racks, or the server replicated to Oslo runs Fedora.

The server replicated to Oslo does not run Fedora.

Ava runs Rocky.

Exactly one of these is true: the proxy server is somewhere to the left of the server replicated to Riga, or the Debian server is in rack 7.

The auth server and Bo are in neighboring racks.

The metrics server is Gia’s.

The Fedora server is somewhere to the left of the server replicated to Pune.

Exactly one of these is true: the server replicated to Accra is in rack 2, or Finn is the queue.

If the Debian server is Finn’s, then the server in rack 7 does not run Rocky.

The NixOS server is not replicated to Perth.

The server in rack 7 is replicated to Accra.

The server replicated to Pune is the db.

At the end of your response, give each solution as a block, one line per engineer, in the order the engineers are listed in that puzzle:

SMALL:

<name> | <rack> | <os> | <role>

MEDIUM:

<name> | <rack> | <os> | <role> | <dc>

LARGE:

<name> | <rack> | <os> | <role> | <dc>

---

Deploy scheduling

You are planning a production rollout of 12 services. Time is measured in whole hours from hour 0.

Services (owning team, deploy duration in hours):

auth: team A, 3 hours

billing: team B, 4 hours

catalog: team C, 2 hours

search: team C, 3 hours

cart: team B, 2 hours

checkout: team B, 3 hours

payments: team A, 4 hours

notify: team C, 2 hours

ledger: team A, 2 hours

gateway: team A, 3 hours

reports: team B, 3 hours

inventory: team C, 4 hours

Rules:

Dependencies. A service may start deploying only after every service it depends on has finished deploying:

gateway depends on auth

payments depends on auth

search depends on catalog

cart depends on catalog

cart depends on inventory

checkout depends on cart

checkout depends on payments

ledger depends on billing

ledger depends on payments

notify depends on checkout

reports depends on ledger

gateway depends on search

notify depends on gateway

Each team can deploy only one of its services at a time.

Different teams can deploy at the same time.

Blackout windows. No deploy may be in progress at any time during hours 9 to 11 or hours 17 to 19. A deploy may end exactly at hour 9 or 17, and may start exactly at hour 11 or 19. A deploy cannot pause and resume.

Each deploy runs from its start hour for its full duration without interruption.

What is the earliest hour by which all 12 services can be finished? Give a schedule that achieves it.

At the end of your response, give your answer in exactly this format:

MAKESPAN: <hour>

<service>: <start hour>

(one line per service, all 12 services)

---

Probability

A client calls a payment API and retries on failure.

The server is in one of two states on each attempt, Healthy or Degraded.

On the first attempt, the server is Healthy with probability 4/5 and Degraded with probability 1/5.

Between consecutive attempts, the state changes like this: from Healthy, it stays Healthy with probability 3/4 and becomes Degraded with probability 1/4. From Degraded, it stays Degraded with probability 2/3 and becomes Healthy with probability 1/3.

An attempt succeeds with probability 9/10 if the server is Healthy and 2/5 if it is Degraded, independently of everything else given the state.

The client makes at most 4 attempts and stops as soon as one succeeds.

Circuit breaker: if two consecutive attempts both hit a Degraded server and both fail, the client stops immediately and makes no more attempts.

Answer these questions with exact fractions in lowest terms:

Q1. What is the probability that the request eventually succeeds?

Q2. What is the expected number of attempts the client makes?

Q3. Given that the request succeeds, what is the probability that it succeeded on exactly the second attempt?

Q4. What is the probability that the circuit breaker ends the request early, before the client has used all 4 attempts?

Q5. Given that the request succeeds, what is the probability that the first attempt failed?

At the end of your response, give exactly five lines in this format:

Q1: <fraction>

Q2: <fraction>

Q3: <fraction>

Q4: <fraction>

Q5: <fraction>

The post Ember-1 vs. Kimi K3: Nearly identical results at 3.4 times the speed appeared first on The New Stack.

Radiology AI has made remarkable strides in detecting abnormalities across chest X-rays, pathology slides, and 2D scans. Yet one of the most clinically rich and...

Radiology AI has made remarkable strides in detecting abnormalities across chest X-rays, pathology slides, and 2D scans. Yet one of the most clinically rich and data-dense modalities—the 3D computed tomography (CT) scan—remains largely underserved by modern vision language models (VLMs). Frontier general-purpose models perform poorly on volumetric imaging, and most open medical AI models lack the…

Source

Anthropic wants developers to use Claude and its family of tools to handle complete coding tasks. The company introduced Claude Opus 5.5 on Tuesday to span more of the software application development lifecycle: from design specification creation through debugging to code generation and testing.

As the first release in a new family of Claude 5.5 models, Anthropic says that Claude Opus 5.5 performs at the level of Claude Fable 5.1 on “most work” tasks and costs around 40% less to run than Opus 5. 

GitHub chief product officer Mario Rodriguez is quoted in Anthropic’s launch announcement on exactly where software engineers sit today with frontier models for code automation. He said that developers want agents.

“In our testing [of Claude Opus 5.5] across GitHub Copilot CLI and VS Code, Claude Opus 5.5 used among the fewest tokens and steps we measured. In VS Code, it solved more terminal tasks than Opus 5 in less than half the steps. More than making individual tasks more efficient, it’s making developers’ bigger projects more achievable,” says Rodriguez.

Where does Claude Opus 5.5 get its power from?

Anthropic explains that the Claude 5.5 family’s expanded full-lifecycle capabilities were developed under an established set of practices.

These practice elements include extensive alignment testing (model evaluation processes put in place to make sure actions and outputs closely match human values and intended goals), pre-release evaluation by outside organizations, and safeguards for high-risk areas such as cybersecurity and biology.

The organization says that on its most comprehensive alignment test, Opus 5.5 is the strongest-performing model tested to date, with particular improvements in several behaviors that contributed to recent cybersecurity incidents (e.g., biased reasoning, attempting to escape a sandbox, and others).

Independent SRE & AI reliability architect Akash Thakur tells The New Stack that Anthropic’s work getting its model to complete whole coding tasks is impressive, but “getting it to know when it hasn’t” is the harder problem developers need to think about.

“Models working at the level of Claude Opus 5.5 are genuinely good at breaking a project into small, finishable pieces — and that’s the real unlock, because momentum on software comes from finishing things, not starting them,” Thakur says. “…But ‘completed’ and ‘correct’ aren’t the same thing. The task that looks done and completed is exactly the one that often costs the team later, so the win isn’t removing the human — it’s moving them from writing the code to verifying it.”

“Models working at the level of Claude Opus 5.5 are genuinely good at breaking a project into small, finishable pieces …but ‘completed’ and ‘correct’ aren’t the same thing.”

Suggesting that we are now witnessing a “higher bar for more capable models”, Anthropic said that models that could fully automate AI research itself should meet a higher safety standard. The company noted that as AI becomes more capable, public policy should play a larger role in making sure these systems are safe. Its recent work with Accenture is offered as an example of how Anthropic is building the infrastructure to support this.

Sprawling jobs: codebase-wide migrations & audits

Getting more specific, Anthropic has claimed Opus 5.5 is “particularly good” at long, sprawling jobs like codebase-wide migrations and audits. An early tester said it audited and fixed a 200,000-line codebase in under three hours, while Opus 5 took over 20 hours and used 2.5x as many tokens. 

“In an internal test, we asked Opus 5.5 and Fable 5.1 to translate HAProxy, widely used software that balances web traffic loads across servers, from C into Rust. Both rewrites passed nearly all of HAProxy’s own regression tests, but Opus 5.5 finished in 9.5 hours compared to 12 for Fable 5.1, and cost 51% less,” said Anthropic in a press statement.

Field CTO for the EMEA region at Coder, Eric Paulsen, tells The New Stack that he is happy to hear Claude Opus 5.5 is becoming more holistically capable, but he’s “not surprised” because AI is “eating the software delivery chain end to end” today.

“Despite the efficiencies showcased in Opus 5.5, the moment an agent can reliably finish real work, the question stops being whether the model is good enough and becomes where you’re letting it run,” Paulsen says. 

“Kicking off a Claude Code session on a laptop that can be compromised, stolen, or simply run out of compute is not an engineering environment; capable agents need dedicated, governed infrastructure with the same guardrails, secrets handling, and observability a developer would demand of any other production workload. As we’ve seen here with Anthropic’s own work, firms need to underline testing and safeguard procedures for launches of this kind,” he added. 

Anthropic assures users that external testing has been conducted and that the model was tested before release by Frontier Design and METR. When established safeguards for Opus 5.5 intervene, requests “fall back to another model transparently,” meaning developers may not see which model actually handled a given call.

In cybersecurity, users can identify and fix bugs in their code, but most cybersecurity tasks will be re-routed to Opus 4.8. Requests flagged by biology and frontier LLM development classifiers will be re-routed to Opus 5. Vetted organizations can apply to Anthropic’s Life Sciences Verification Program to use Opus 5.5 for biology research, and the company says it will expand its Cyber Verification Program in the coming weeks.

The developer’s terminal prompt has fundamentally changed

HasData co-founder, Sergey Ermakovich, tells The New Stack that his work as a web scraping specialist for data pipelines and AI means he sees zen-like, one-brick-at-a-time logic in what Anthropic has done. 

“The biggest change — and it’s a trend that will have driven Anthropic’s design and development aspirations for Claude Opus 5.5 — is that the terminal is no longer just a place where a developer pastes generated code,” Ermakovich says. “The terminal now becomes part of the model’s workspace.”

Because a model can run commands, inspect failures, modify files, and verify the result, Ermakovich suggests it can “close the loop” instead of handing unfinished work back to an engineer. “That makes small complete tasks much more valuable. Fix one failing test, update one dependency, migrate one endpoint, verify it, then move to the next task,” he adds.

“The terminal is no longer just a place where a developer pastes generated code — the terminal now becomes part of the model’s workspace.”

Commenting on the development of this model as part of Anthropic’s approved corporate messaging, John Ruelas, staff software engineer at Ramp, said that verbose, hard-to-follow output has been his “biggest frustration” with frontier models, but “Claude Opus 5.5 fixes it” for him.

He noted that it writes like a good colleague and follows his company’s writing rules. A design spec came out usable with very minimal edits, and when it rewrote one of his prompts, he preferred its version to his own. When it optimized the team’s test suite, he could follow its reasoning easily and “shipped the change with confidence.”

We’re so done with the autocomplete era

Vice president of AI strategy at Abbyy, Maxime Vermeir, tells The New Stack that the more lifecycle-wide Claude Opus 5.5 features on offer show that “we’re done with the autocomplete era” for basic code automation tools.

“Anthropic’s elevation here reflects the fact that it used to be thought of as marvelous if AI could complete a developer’s next line of code, but today the expectation is that you hand it a whole Jira ticket and that it gets done,” Vermeir says. “But despite the power on show with Claude Opus 5.5, the question remains as to how much these newer models will actually understand what ‘done’ means, as often it seems they have a desire to keep burning tokens by offering you yet another thing it didn’t quite do right.”

Opus 5.5 also “communicates more naturally” than prior models, with early testers finding its writing clearer and easier to follow, making it a better work partner over long sessions.

Costed out lower than Opus 5, Opus 5.5 is priced at $4 per million input tokens and $20 per million output tokens (20% less than Opus 5), and Anthropic has cut cache read prices by 60% for token-billed usage. Opus 5.5 also needs fewer tokens for higher-quality work and generates output more than 30% faster than Opus 5.

The post Claude Opus 5.5 wants to finish your coding tasks, not just start them appeared first on The New Stack.

AI agents spend a ridiculous amount of compute generating text nobody actually needs. The decisions an agent makes along the way don’t require a written answer and yet, agents still send them to generative models, wait for an answer while burning through tokens and then parse that output back. The overhead is already drawing scrutiny — OpenAI’s own researchers recently disclosed spending $7,000 a day running agent workloads.

Kev, a new family of open decision models built on Qwen 3.5, takes a different approach and skips the generation entirely.

Developer Jared Palmer released a new generation of Kev on Sunday, with 0.8 billion, 4 billion, and 9 billion parameter models built on Qwen 3.5. Kev is prefill-only, processing the state, questions, and candidates in a single forward pass before reading the decisions from a pointer head without an autoregressive decoding loop.

Kev, a new family of open decision models built on Qwen 3.5, takes a different approach and skips the generation entirely.

Decisions without generated text

Kev supports three decision types: Noul for yes/no, Choice for selecting among candidates, and Score for ordered levels, mirroring TypeSafe’s System One API. Developers provide the state and questions, and the pointer head returns probabilities across the available candidates.

For a tool-routing decision, the output could look like this:

search: 0.82

database: 0.13

calculator: 0.05

Kev can still choose the wrong tool, but because it scores only the candidates it’s given, it can’t introduce an option that isn’t on the list.

Routing, safety checks, escalation, and ranking can then move to the decision layer, leaving larger reasoning models to handle the open-ended work.

Kev can still choose the wrong tool, but because it scores only the candidates it’s given, it can’t introduce an option that isn’t on the list.

Batching choices, one pass

Multiple decisions can also be made against the same state in a single forward pass, with a block-causal attention mask isolating the questions while the pointer head scores each set of candidates independently.

Palmer’s documentation shows the 4B model processing three questions in 277 milliseconds in bf16 on an M5, although without a controlled comparison against Qwen generating equivalent answers on the same hardware, the result doesn’t establish how much faster the approach is in practice.

The ability to evaluate several decisions against the same context could become more useful as agent loops grow more complex, but skipping generation doesn’t make the resulting decisions inherently better.

Calibration limits and tradeoffs

The largest model, Kev-9B, reached 83.7% accuracy on the project’s locked out-of-domain test, according to Palmer’s model card. That’s a developer-reported benchmark, and Palmer documents some limitations alongside it.

The probabilities Kev returns don’t always reflect how confident developers should be in the result. Palmer found that temperature calibration can drift on unseen source distributions, a problem for agents that use probability thresholds to decide whether to execute an action or escalate it, since even a high-probability choice can still be wrong.

Fine-tuning also changes some of the capabilities inherited from the underlying model. Palmer’s evaluations show declines on general-knowledge and arithmetic tests, particularly among the smaller models. That’s consistent with Kev’s more specialized role alongside a general-purpose model, although its performance in dynamic agent environments will also depend on how well it handles tools, choices, and labels it never encountered during training — and debugging agent failures often points to infrastructure rather than the model itself.

The approach predates Kev. TypeSafe introduced Jev earlier this month as part of its System One platform, using the same Noul, Choice and Score primitives, and Kev implements its /v1/systemone request and response format so applications built against the API can point to a local Kev server instead.

Open weights, open training

The biggest difference is that Palmer released Kev under Apache 2.0 with the model weights, training code, and evaluation tooling, giving developers the option to run and train it on their own infrastructure. Jev’s weights and training data aren’t public, however, which makes direct performance comparisons difficult because differences between the models can’t be isolated to architecture, size, or training.

For applications that make only a handful of bounded decisions, constrained decoding on a model that’s already running may be simpler than adding another model to the stack. Agent loops can make those decisions constantly, however, moving through routing, ranking, safety checks, tool selection, and escalation before generating much user-facing text. It’s a pattern showing up across model architectures — stripping out unnecessary computation when the task doesn’t require it.

When those steps only require a choice or probability, Kev can handle the decision directly while leaving open-ended reasoning and final responses to the larger generative model.

When those steps only require a choice or probability, Kev can handle the decision directly while leaving open-ended reasoning and final responses to the larger generative model.

The post Your AI agent is burning tokens on choices that don’t need words appeared first on The New Stack.

The 1.58 in a 1.58-bit language model sounds like a hard limit, but Intel researchers pushed a ternary model below it by changing how its weights are stored rather than changing the model itself.

Their new BITCOS format compressed one checkpoint to 1.485 bits per weight and improved decoding throughput by as much as 18% on CPUs and 27% on GPUs.

The key is that the familiar 1.58-bit figure assumes a model uses its three possible weight values equally, while real ternary models contain far more zeros than that calculation accounts for. BITCOS stores the location and sign of each nonzero weight separately, allowing zeros to take up less space without retraining the model or altering its output — the equivalent of packing the same contents into a smaller box.

Where the 1.58-bit figure comes from

Ternary models use only three weight values — -1, 0, and +1 — and 1.58 bits is the theoretical minimum needed to represent three equally likely options. That number is cleaner than the reality of storing the weights, where the standard approach fits five ternary values into an eight-bit byte for an average of 1.6 bits each. Models commonly store weights in blocks of 128; however, this leaves the final byte partly unused and pushes the actual rate to 1.625 bits per weight.

When Intel’s researchers measured the distribution of weights across 29 checkpoints from seven ternary model families, they found that zeros accounted for between 29.7% and 51.5% of the weights. In 26 of those checkpoints, there were enough zeros for BITCOS to beat five-trit packing.

The sparsest was a ternary version of Qwen3-1.7B produced with CAT-Q post-training quantization, where 51.48% of the weights were zero, and BITCOS brought the storage cost down to 1.485 bits per weight.

Models commonly store weights in blocks of 128; however, this leaves the final byte partly unused and pushes the actual rate to 1.625 bits per weight.

How zeros save space

BITCOS stands for “BITmap and COmpacted Signs” and divides a model’s weights into two streams. The first assigns one bit to every weight to record whether it is zero or nonzero, while the second assigns a sign bit only to nonzero weights.

A positive or negative weight therefore consumes two bits, but a zero needs only the presence bit because it has no sign to record.

If z is the proportion of zero weights, BITCOS uses 2 − z bits per weight, dropping from 1.6 bits at 40% zeros to 1.485 bits at 51.5%. Because it changes only the storage format, unpacking restores the original -1, 0 and +1 values without affecting accuracy.

BITCOS becomes smaller than five-trit packing once more than 37.5% of a model’s weights are zero, a threshold reached by 26 of the 29 checkpoints Intel examined.

BITCOS becomes smaller than five-trit packing once more than 37.5% of a model’s weights are zero, a threshold reached by 26 of the 29 checkpoints Intel examined.

Making smaller weights run faster

Built for token-by-token decoding with small batch sizes, the format reduces the weight data moving through memory. Intel developed separate unpacking kernels for AVX-512 and AVX2 CPUs as well as Xe2 GPUs, joining other efforts to fit compressed models into faster inference pipelines for AI agents.

On AVX-512 hardware, the kernel uses the presence bitmap as a mask and pdep to scatter the compacted sign bits across the nonzero weight positions. Because Xe2 GPUs lack an equivalent instruction, Intel implemented the same operation with a 2KB lookup table.

Benchmarks across five systems

Compared with the 2-bit kernels, BITCOS ran 10% to 18% faster on the 64-core Xeon server and 2% to 15% faster on the 24-core Core Ultra 9. Performance improved by 9% to 22% on the integrated Arc 140V and by 2% to 27% on the discrete Arc Pro B70. These results measure decoding after the model has loaded, separate from efforts to cut GPU inference cold starts from minutes to seconds.

The smaller format did not win everywhere

On the eight-core Lunar Lake CPU, Intel’s fixed 2-bit kernel beat BITCOS on every model because the system had enough bandwidth to make unpacking the bottleneck. BITCOS remained faster on the GPUs, although decoding overhead limited the gains. Computer scientist and AI infrastructure author Chip Huyen has made the same point about inference more generally, arguing that the right optimization depends on whether compute, memory or bandwidth is holding back the workload.

On the eight-core Lunar Lake CPU, Intel’s fixed 2-bit kernel beat BITCOS on every model because the system had enough bandwidth to make unpacking the bottleneck.

Format limits and open questions

The paper has not been peer-reviewed; all five test systems used Intel hardware, and the end-to-end benchmarks covered seven models at batch size one. Intel has yet to test the format on Nvidia, AMD, or Arm hardware.

The post Intel squeezed a 1.58-bit LLM down to 1.485 bits without changing a single weight appeared first on The New Stack.

Mixture of experts (MoE) has become one of the defining architectural trends in large-scale AI model training. DeepSeek, Qwen, and Mixtral are examples of MoE...

Mixture of experts (MoE) has become one of the defining architectural trends in large-scale AI model training. DeepSeek, Qwen, and Mixtral are examples of MoE models that match or exceed the performance of dense model counterparts at a fraction of the training compute. MoE models provide efficient training through conditional computation. Instead of one dense feed-forward network (FFN) shared…

Source

Everyone knows the open-weight model pitch by now: companies can download the weights, customize them, run them on infrastructure of their choosing, and retain far greater control over where their data is processed — often at a much lower cost than using proprietary models.

Moreover, open-weight models are now thought to trail the leading frontier models by only around four to five months. Nvidia, the world’s most valuable company, is betting heavily on that future. In early September, it agreed to acquire Hugging Face — the sprawling “GitHub for AI” that hosts more than three million models — for $12.9 billion, while pledging to keep the platform open to different models, clouds and computing providers. And on Thursday, Nvidia detailed how Nvidia is using its own open-weight Nemotron model to manage its vast global supply chain in partnership with Palantir.

That power also comes with serious security questions. OpenAI president Greg Brockman recently warned that increasingly capable open-weight models — pointing specifically to China’s GLM-5.3 — could “significantly accelerate the threat landscape” as models with advanced cyber capabilities become freely downloadable and modifiable.

But for businesses accessing those models through third-party services, there is another concern closer to home: where their own data goes when they use those models, particularly when the model originated in China.

China and the open-weight factor

Hugging Face data from February showed models from Chinese developers accounted for 41% of downloads in the preceding 12 months, ahead of the US at 36.5%. Over on OpenRouter, meanwhile, open-weight models now account for around 60% of tokens consumed by US-originating requests, with the company noting that Chinese models constitute the majority.

And that’s why OpenRouter is now giving companies a way to put a geographic fence around that traffic. The AI model marketplace has officially launched US in-region routing into general availability for business and enterprise customers, promising that requests sent through its US endpoint are decrypted, processed and served entirely inside the country — or rejected if that can’t be done.

The feature itself had been quietly available in some form before now, with OpenRouter updating its documentation in early August to say US in-region routing was available to enterprise customers by request. It’s also worth noting that this is in addition to European in-region routing, which it says has been available since October 2025.

Started in early 2023 by former OpenSea CTO Alex Atallah, OpenRouter serves as an interface to the crowded AI model market, with developers able to switch between hundreds of models from myriad providers via a single API. Payments giant Stripe recently announced plans to acquire the company in a reported $8 billion deal, while a slew of other companies including Cursor, Ramp, and Meta, are also building their own model routers.

The reason why model routers are such hot property right now is largely down to economics. Developers have traditionally hard-coded applications to send everything to the same model, while a model router can instead make that choice request by request, sending easier jobs to cheaper models while reserving the pricier frontier systems for the work that actually needs them.

That intermediary role is also what makes OpenRouter’s new residency controls possible: it already decides which provider serves each request, and can now restrict that choice to provider endpoints operating in the US.

Keeping Chinese models inside the US

In a blog post announcing the new feature on Wednesday, Cailee Moberg, who works on OpenRouter’s product team, notes that while US-developed models from Nvidia and Thinking Machines are contributing to the broader open-weight model boom, Chinese models dominate usage and raise tough questions for companies concerned about their data.

“Models from Chinese labs are still most of the [open-weight model] volume, and procurement approval for those models can be difficult.”

“Models from Chinese labs are still most of the [open-weight model] volume, and procurement approval for those models can be difficult,” Moberg writes.

In its 2026 State of AI in the Enterprise report, Deloitte concluded that sovereign AI was on the rise, noting that 77% of companies “now factor country of origin into their vendor selection,” while nearly 60% construct their AI stacks “primarily with local vendors.”

And this at least partly explains why OpenRouter is now offering in-region routing for US customers. Moberg points to DeepSeek V4 Pro, Kimi K3 and GLM 5.2 as specific examples. All three are available through US In-Region Routing because Baseten, Fireworks and Azure serve them from US data centers. Companies could already keep these models inside the US by self-hosting them or using a US provider directly; OpenRouter’s new routing gives its own customers that residency guarantee without having to manage those deployments themselves.

OpenRouter maintains a live list of models eligible for US in-region routing, ranging from proprietary frontier models from OpenAI and Anthropic to open-weight models from the major Chinese labs.

“In-Region Routing allows teams with data residency requirements to get the price and performance gains from Chinese open-weight models,” Moberg continues. “When a US or EU provider hosts a model, requests go to that provider and the lab is not involved.”

“In-Region Routing allows teams with data residency requirements to get the price and performance gains from Chinese open-weight models.”

The technical change happens at the routing layer. With OpenRouter’s standard global endpoint, a request can be served by an eligible provider operating in any region, so even using a model from a US company does not guarantee that the request itself is processed in the US. With us.openrouter.ai, the request is decrypted on OpenRouter infrastructure inside the US and the pool of providers is filtered to endpoints OpenRouter has approved as operating there.

If no compliant US provider can serve the requested model, OpenRouter returns a 404 error. Companies can also enforce the regional restriction through OpenRouter’s Guardrails at the workspace, team or API-key level, while tools that would send prompt data outside the US are disabled on the regional endpoint.

So while none of this ultimately changes where the DeepSeek, Kimi or GLM models are developed, in-region routing alters which copies of those models its US customers can be routed to, and where their prompts are handled along the way.

The post Chinese AI models dominate OpenRouter’s US token consumption. It can now guarantee that traffic stays entirely in the US. appeared first on The New Stack.

Enterprise AI company Cohere announced North Small Translate last week, a mixture-of-experts (MOE) open-weight machine translation model that works across 50 languages.

Developers can download the weights for noncommercial use under CC BY-NC 4.0. Cohere offers commercially licensed deployment through Model Vault, which is a Cohere-managed inference environment. Cohere positions the model as part of its sovereign AI strategy, aimed at organizations that want greater control over where their models run and how their data is handled.

North Small Translate builds on Cohere’s multilingual and translation lineage, which includes its Tiny Aya and Command A Translate model families. The company claims North Small Translate outperforms “similarly sized open-weight models” under 1T parameters, as well as API-based translation models in various dimensions of machine translation on average. 

Cohere co-founder Nick Frosst tells The New Stack that the model’s efficiency draws from the fact that it is non-reasoning, i.e., it relies on learned statistical patterns without a step-by-step logic process, which means it uses fewer tokens.

Machine translation is still broken for most of the world’s languages

“We spent nine years scaling an architecture invented to fix translation, and machine translation is still broken for most of the world’s languages,” Frosst says. “General-purpose models get you most of the way and then stop. The next phase of enterprise AI in this space is smaller, more specialized, and runs inside your own walls.”

“…machine translation is still broken for most of the world’s languages.”

In Cohere’s reported evaluation using WMT26 benchmarks, the company states that North Small Translate leads with a WMT26 All Languages benchmark score of 83.60, compared with 81.56 for Qwen 3.5 397B A17B, 76.50 for GLM 5.2 FP8, 81.37 for DeepL NextGen, 79.46 for Gemma 4 31B (on), and 68.20 for Google Translate. 

With its mixture-of-experts architecture and 218 billion total parameters, with 25 billion active. Cohere points to North Small Translate’s smaller compute & memory footprint than other models. Some model-to-model comparisons in this space aren’t fully substantiable, since not every vendor discloses parameter counts.

With current solutions, long documents start to fall apart

“Machine translation allows documents to be translated from one language to another automatically. With current solutions, long documents start to fall apart,” Frosst says. “Google Translate scores 21.3 on our long-context test, Gemma 4 31B 19.4; we score 48.9. That’s [for example] a safety manual that reads fine on page one… and has drifted by page ten. The other risk is where the text goes. Once you push HR policies or regulated documents through a third-party API, that data has left your building, and necessarily that means your control over it is diminished.”

“The risk [in machine translation] is where the text goes. Once you push HR policies or regulated documents through a third-party API, that data has left your building and necessarily that means your control over it is diminished.”

Explaining why the model offers “stronger translation performance” across complex enterprise translation tasks, Frosst says the model can support work spanning “a high volume” of sensitive documents. 

As well as its 50 languages (32 ‘high-resource’ languages + 18 others), the Cohere team explains that the model also supports translation-workflow-focused capabilities, such as structured translations (i.e., Markdown or JSON documents), instruction following (i.e., recommended tone & format), and terminology guides (i.e., providing specific vocabulary to use in the translation), all as part of the model.

“North Small Translate works with a multi-pass workflow,” explains Frosst. “The model translates, reviews its own output, finds errors, and fixes them – and this is the same loop we used in training. We ship both because standard is one pass and built for volume, while the agentic [version] spends more tokens for 84.36 against 83.60 on WMT26. That difference ends up being worth it when the document is a contract or a safety procedure, for instance, but in other cases you’d rather optimize for efficiency.”

“The model translates, reviews its own output, finds errors and fixes them.”

Model ‘steerability’ drives suggesting language tone and formatting

This model uses the same architecture as prior Cohere models but improves performance through post-training advances, including reinforcement learning and new datasets, specifically for machine translation tasks.

Frosst concludes that, across the translation model marketplace, generative machine translation models offer the highest quality and steerability (i.e., suggesting tone, formatting, etc.) but typically cost much more than Neural Machine Translation (NMT) models commonly used in commercial use cases. 

North Small Translate was developed in partnership with RWS, an AI solutions company pioneering in language technology and services. Collaboration with RWS, specifically with its Language Weaver research and science teams along with its language experts, helped shape the model’s real-world translation performance throughout development. 

As noted above, developers can access the weights free of charge for non-commercial use in three quantizations. There is also a Hugging Face Space and an API for those who lack the required hardware. 

The post “Machine translation is still broken for most of the world’s languages”: Cohere builds non-reasoning for a reason appeared first on The New Stack.

This week, Cohere released North Small Translate 1.0 under a CC BY-NC 4.0 license: the weights are there to download, evaluate and study, but not to run in production without a commercial agreement.

It’s an interesting choice from the Canadian foundation model company, which has built its pitch around AI sovereignty for regulated industries and describes this release as part of a mission “to make sovereign AI a technological reality.” Sovereignty there means control over where the model runs and who sees the data. A commercial license keeps that promise intact. It stops short of independence from Cohere. Enterprises keep their data and their infrastructure. They don’t get to fork the model, build a product on it, or keep running it if the terms change at renewal.

Open weights, except for commercial production

North Small Translate is an open-weights mixture-of-experts model built for machine translation across over 50 languages and locale variants. It has 218 billion total parameters, with 25 billion active parameters and a 16,000-token context window.

Not all users have the same access to those weights.

Per Cohere, the model is designed to give researchers, developers, and enterprises “flexible ways to evaluate and deploy machine translation while retaining control over their data and infrastructure.”

That’s an appealing description for organizations keen on pursuing sovereign AI. But the open-weight release comes with an important caveat: Not all users get the same rights to take advantage of those weights.

North Small Translate is available today on Cohere’s free tier through the Chat V2 API. For those who intend to use the model weights for non-commercial use, the FP8 weights are available on Hugging Face under the CC BY-NC 4.0 license.

But if enterprises want to put them into production, then a different set of terms applies. They’ll have to purchase a commercial license and deploy North Small Translate through Model Vault, Cohere’s fully managed inference platform.

Cohere’s not the only one drawing a line around open-weight use

Other AI companies are starting to attach more conditions to their open-weight models, too.

Last month, Chinese AI lab Z.ai released the weights for its flagship GLM-5.3 model on Hugging Face. But like the Canadian AI company, it also changed its licensing terms depending on who is deploying the model — a departure from its previous approach. While GLM-5.2 shipped under the permissive MIT license, GLM-5.3 adds new requirements for certain commercial users.

Cohere, for its part, has been similarly mum about why it made North Small Translate’s open weights noncommercial.

These requirements apply only to companies with aggregate revenue over $10 billion over 12 consecutive months. Additionally, if these companies want to host GLM-5.3 or its derivative works for commercial purposes, they have to first pass the Chinese lab’s security review.

Z.ai didn’t explicitly spell out why it decided to make such an about-face for GLM-5.3, which is especially puzzling given that its predecessor shipped under MIT without any commercial stipulations. Cohere, for its part, has been similarly mum about why it made North Small Translate’s open weights non-commercial.

Sovereign deployment, with restrictions

The Canadian company’s decision to make North Small Translate available as open weights but gate commercial use is a head-scratcher, given its history of selling sovereign AI to enterprises.

In fact, in June, it pitched North Mini Code, its first coding model, as a response to developers demanding the same sovereignty guarantees that regulated industries have long required.

Unlike North Small Translate, though, this open-weight model was released under an Apache 2.0 license from the get-go — without any comparable restrictions for commercial users.

Clearly, Cohere is going in a different direction with its latest open-weight release, emerging as another example of AI companies putting tighter terms around increasingly capable open-weight models.

The post Cohere’s new translation model is open weights — but not for commercial use appeared first on The New Stack.

The multi-modal model could help astronauts navigate craters, investigate ancient lava, and search for ice, as the US plans for a long-term lunar presence.

Nvidia and Palantir announced Thursday that they’re working together to bring “sovereign AI to critical supply chains,” kicking off initially with Nvidia’s own sprawling supply chain.

The news builds on a partnership that kicked off last October, when the duo said they would combine Nvidia’s AI computing and models with Palantir’s software to help companies use AI to make complex operational decisions. Then in June, they expanded that effort into sovereign AI, allowing organizations to run and customize Nvidia’s AI models inside tightly controlled environments while keeping sensitive data and model weights under their own control.

Now, they’re applying that technology inside Nvidia itself, where they say a smaller, fine-tuned model is already outperforming a far larger one.

A proving ground for sovereign AI

The companies have fine-tuned Nvidia’s 30-billion-parameter Nemotron 3.5 Lightning model on decisions made by Nvidia’s supply-chain operations team. Palantir’s Foundry and Artificial Intelligence Platform (AIP) bring together the data behind those decisions, while its Ontology acts as a live map connecting components, factories, capacity and production commitments. Nvidia’s cuOpt software, meanwhile, works out how to distribute scarce parts, with Nemotron weighing the wider context and recommending what planners should do.

They then plan to “extend the learnings from Nvidia’s deployment” to companies in other sectors, including manufacturing, energy, healthcare, automotive and aerospace. Palantir’s own customers will be able to build versions tailored to their own supply chains by training Nemotron on their proprietary data using Foundry and AIP, then run the resulting system on-premises or through cloud and colocation providers.

So, in effect, Nvidia and Palantir are putting the sovereign AI partnership they outlined in June into practice inside Nvidia, while using that deployment as a proving ground for an architecture other companies can adapt to their own use cases.

Nvidia as a test case

As the world’s most valuable public company at a $5.4 trillion market cap, there’s good reason for Nvidia to start close to home. Its supply chain spans millions of parts, thousands of suppliers, and a global network of manufacturing partners, with the company saying a single Vera Rubin rack alone contains some 1.3 million parts. Those components have to arrive in the right place at the right time: if one part is missing, assembly can stall while everything else that arrived sits waiting.

“Supply chains are the operating system of the physical economy, and AI factories are among the most complex systems ever built.

Jensen Huang

And that complexity is what Nvidia founder and CEO Jensen Huang says makes supply chains a natural target for the technology. From chips and memory to manufacturing, networking, power and cooling, he argues that building modern AI systems increasingly depends on coordinating an enormous web of companies and components.

“Supply chains are the operating system of the physical economy, and AI factories are among the most complex systems ever built,” Huang says in a statement.

Palantir co-founder and CEO Alex Karp goes further, arguing that Nvidia’s operations provide an unusually demanding environment in which to put the companies’ approach to the test.

“Nvidia has arguably the most valuable, intricate, and complex supply chain in the world.”

“Nvidia has arguably the most valuable, intricate, and complex supply chain in the world,” Karp adds in a separate statement.

The sovereignty selling point

Nvidia has long been positioning itself at the center of the open-model debate. In July, Huang even used his first-ever post on X to promote an industry letter lobbying Washington to support frontier open-weight models, arguing that they give companies and countries more control over their AI infrastructure.

Then in early September, Nvidia swooped in with a $12.9 billion deal for Hugging Face, the so-called “GitHub for AI models.” Amid concerns that ownership by the world’s dominant AI chipmaker could undermine Hugging Face’s neutrality, Huang pledged that it would remain open, continue hosting models from across the industry and support hardware beyond Nvidia’s own.

Nemotron is central to Nvidia’s own open-model push. The name dates back to 2023, when Nvidia released its first Nemotron-3 8B models for enterprises to customize and fine-tune. Those early models were downloadable through Hugging Face and Nvidia’s NGC catalog, although access was gated and governed by Nvidia’s own community license. So they were customizable, and their weights were available, but the much broader “open model” positioning Nvidia uses today came later.

The current Nemotron 3 series arrived back in December, initially spanning Nano, Super and Ultra models aimed at different agentic AI jobs. Nvidia now publishes weights and, for many of the models, training data and recipes so developers can customize themselves. Nemotron 3.5 Lightning, released in August, is the 30B model Nvidia and Palantir have fine-tuned for this supply-chain deployment.

That openness is also at the heart of the whole sovereignty pitch: companies can adapt Nemotron using proprietary data while keeping that data, the model weights, and inference inside their own environment.

Specialization over size

Nvidia’s own deployment gives outsiders a result to chew on. It says the fine-tuned 30B Lightning scored 86.7% accuracy on its supply-allocation task, versus 55.5% for the 550B Nemotron 3 Ultra—a model roughly 18 times its size.

In a technical blog post published on Thursday alongside the main announcement, Nvidia solutions architects Nell Barber, Rana Haber, and Aastha Jhunjhunwala note that the result shows how far specialization can go. On a tightly defined allocation task, the 30B model outperformed a general-purpose model more than an order of magnitude larger.

“This doesn’t mean the smaller model is more capable overall. Its gains are concentrated in the domain it was post-trained on.”

“This doesn’t mean the smaller model is more capable overall,” they add. “Its gains are concentrated in the domain it was post-trained on. Future production risk forecasting remained difficult despite fine-tuning. Specialization improved the decision task but failed to solve every prediction problem attached to it.

For companies considering Nvidia’s blueprint, the more interesting takeaway may be this: a smaller open model, trained on business specifics, can sometimes be more useful than reaching for the biggest model available.

The post Nvidia and Palantir fine-tune a 30B Nemotron model for Nvidia’s supply chain. It beats a model 18 times its size. appeared first on The New Stack.

Update: As of 6:30 p.m. Eastern on Friday, September 4, GPT-6 Astra was available on ChatGPT for all paying users with plans that include access to it.

Thibault Sottiaux, a leading member of the technical staff at OpenAI, posted on his X account, “OK nevermind, the team and Astra did a good job and our systems are more scalable than we anticipated. Astra is now rolled out to all Plus and Business users too. Hope you have a blast and let us know how it goes!.”

Earlier in the day Friday, OpenAI CEO Sam Altman posted on X, “GPT-6 Astra is now available to all Pro, Enterprise, and Business Premium users in Work/Codex, and is available in the API. We will start rollout to Plus and Business users next. Thank you for the patience.”

Users who pay for the $8/month ChatGPT Go plan do not and will not have access to GPT-6 Astra or GPT-5.6, according to the company’s pricing tier documentation.

Sottiaux posted on Thursday evening after OpenAI announced the debut of Astra but before it released it to users: “We are starting to release GPT-6 Astra, and we are doing it as carefully and quickly as possible.”

OK nevermind, the team and Astra did a good job and our systems are more scalable than we anticipated.

Astra is now rolled out to all Plus and Business users too. Hope you have a blast and let us know how it goes!

— Tibo (@thsottiaux) September 4, 2026

---

Our original story, published at 12:24 p.m. Eastern on Friday, continues below:

OpenAI launched GPT-6 Astra on Thursday, but many developers hoping to give it a test drive are still waiting for access.

Hours after the announcement, OpenAI CEO Sam Altman apologized for what he called a “messy rollout,” acknowledging that broad access to Astra had not yet begun for either API customers or ChatGPT subscribers.

“First, sorry for the messy rollout,”

“First, sorry for the messy rollout,” Altman wrote in a post on X. He added that OpenAI expected to begin the broader rollout “in the near future,” starting with ChatGPT Pro subscribers.

first, sorry for the messy rollout.

second, when we screw up, we try to make it right.

third, we should be able to begin broad rollout to API customers and chatgpt subscribers in the near future. as usual we will start with pro subscribers. https://t.co/nKOhW18CDK

— Sam Altman (@sama) September 4, 2026

For developers, that creates an unusual bind. OpenAI has already published Astra’s API documentation and pricing, including its 1.05 million-token context window, support for up to 128,000 output tokens, and standard API rates of $10 per million input tokens and $50 per million output tokens. But the endpoint itself is still rolling out, and OpenAI has yet to explain specifically what went wrong with the rollout. So while Astra’s benchmark results were impressive, developers and independent reviewers who want to evaluate those claims against their own workloads are still left waiting.

Why developers are waiting

OpenAI’s engineering lead for Codex, Thibault Sottiaux, offered a little more detail in an X post of his own about what is happening behind the scenes. He said the rollout will take “a few days” to complete and that OpenAI is bringing new systems and additional compute online as it expands access.

“We are starting to release GPT-6 Astra, and we are doing it as carefully and quickly as possible,” Sottiaux wrote. He added that “many novel systems will operate at scale for the first time” during the rollout and that OpenAI is “bringing a lot of compute up.”

“We are starting to release GPT-6 Astra, and we are doing it as carefully and quickly as possible,”

While this does not establish that compute capacity caused the delay, it does offer some insight into what OpenAI is dealing with as it expands access. The company’s original announcement said Astra would initially be available to a limited group of organizations, with ChatGPT Plus, Pro, Business, and Enterprise users, the OpenAI API, Microsoft Azure, and AWS Bedrock expected to follow “over the coming days.”

Developers who have secured early API access are already encountering surprises beyond the rollout itself. As The New Stack reported this week, Astra’s API introduces a new class of safety-triggered interruptions that look like timeouts but aren’t — which could make a real difference for developers building production workflows around the model.

Banked resets explained briefly

While users wait, OpenAI is trying to make up for at least part of the delay with something it calls a “banked reset.”

Sottiaux said paid ChatGPT subscribers will receive one banked reset for every day they remain without Astra access, beginning September 3, adding that the “team is moving mountains to give access as fast as we can.”

“Team is moving mountains to give access as fast as we can.”

OpenAI has used banked resets before with ChatGPT Work and Codex, giving users a way to replenish their usage after hitting a limit. They aren’t additional API credits or a permanent increase in usage limits but resets users can save until they need them.

What’s different with Astra is how OpenAI is handing them out: one for every day a paying ChatGPT subscriber remains without access. So far, OpenAI hasn’t announced anything similar for developers waiting to use Astra through the API.

The gesture fits a broader pattern of OpenAI experimenting with how it charges for AI: the company recently announced an outcome-based pricing model that would bill only when the model produces a correct result; another sign that its pricing strategy is still very much in flux.

Compute capacity complicates rollout

OpenAI says Astra will roll out to Plus, Pro, Business and Enterprise users, along with the OpenAI API, Microsoft Azure and AWS Bedrock, “over the coming days,” with Pro subscribers first. Sottiaux said the rollout should take a few days to complete.

For developers, the biggest unanswered question is when broad API access will arrive and whether it will come with tighter usage limits. Astra’s persistent-agent capabilities also make the wait more significant for developers.

OpenAI did not immediately respond to The New Stack’s questions about the rollout problems, banked resets, or API timeline.

The post “Sorry for the messy rollout”: OpenAI launches GPT-6 Astra to most paying users a day after its unveiling appeared first on The New Stack.

Nvidia has confirmed that it’s agreed to acquire Hugging Face in a mammoth $12.9 billion deal that will bring one of the AI industry’s most important open-model platforms under the auspices of the world’s dominant AI chipmaker — and, with a market cap of well over $5 trillion, the most valuable company on Earth.

When reports first emerged in August that Nvidia was lining up a gargantuan bid for what has often been described as the “GitHub of AI Models,” concerns quickly surfaced over what the deal could mean for the platform’s openness and hardware neutrality. As The New Stack reported at the time, Hugging Face’s value lies partly in giving developers a neutral place to find and deploy open models across Nvidia, AMD, Intel and cloud accelerators — raising the question of what happens if that platform is owned by just one of them.

Keeping Hugging Face open

That, ultimately, is why Nvidia founder, president and CEO Jensen Huang is going to great lengths to allay fears that Hugging Face could become a vehicle for steering developers toward Nvidia’s own hardware and software stack.

“Nvidia compute will not be required to build on or deploy through Hugging Face.”

In the official announcement on Thursday, Huang makes a series of explicit promises around neutrality, noting that Hugging Face would “remain an open platform for the entire AI ecosystem,” continue to support multiple clouds and accelerators, while stressing that “Nvidia compute will not be required to build on or deploy through Hugging Face.”

“Hugging Face will continue to support open source and open weight models from across the ecosystem, from every model builder,” Huang writes. “It will continue to support multi-cloud and multi-accelerator development and deployment, so builders can use the hardware and infrastructure that best fit their work.”

Indeed, it’s clear Nvidia had clocked the concerns around the impending acquisition. The word “open” appears no fewer than 19 times in Huang’s relatively short announcement, underlining just how central that reassurance is to Nvidia’s pitch for the deal.

Huang also points to a recent open letter on open weights that he co-signed alongside executives and researchers from across the AI industry, including those from Hugging Face. The letter argued that open-weight models are critical to broadening access to AI, strengthening competition and giving developers more control over how models are deployed and adapted.

For what it’s worth, Nvidia has been pushing hard into open-weight AI itself, including through its Nemotron models and a broader effort to make frontier-class models easier to run locally. Hugging Face co-founder and CEO Clément Delangue went so far as to call Nvidia the “King of American open-source AI” a few months ago, pointing to its growing collection of public models, datasets and Spaces on Hugging Face.

In the wake of the announcement on Thursday, Delangue doubled down on that position in a fresh post announcing the deal, arguing that Hugging Face has reached the point where keeping open-source AI competitive would require substantially more resources.

“It needs more compute, more support, more collaboration and more visibility.” – Hugging Face CEO Clément Delangue

“10 years after starting Hugging Face, open-source AI is at an inflection point,” Delangue writes on LinkedIn. “Thanks to the community, we’ve shown that it can be a complement, and even an alternative, to closed-source APIs. But for it to happen at larger scale, it needs more compute, more support, more collaboration and more visibility.”

He adds that Nvidia has committed to backing Hugging Face while keeping the platform open, independent and compute-agnostic, with the founders and existing team staying on.

“Together, we think we can make open source the default way to build AI,” Delangue writes, setting out a goal of helping 100 million AI builders “own their intelligence rather than rent it.”

GitHub as a historical precedent

However, there is some historical precedent for taking such assurances with a degree of caution. When Microsoft bought GitHub for $7.5 billion in 2018, it similarly promised that the platform would remain independent and open. GitHub largely retained that openness, though Microsoft’s later use of public GitHub code to help train the proprietary, paid Copilot service sparked a backlash among factions of the open-source community.

In truth, that obvious comparison may actually understate Nvidia’s challenge. In recent analysis for Forbes, technology analyst Janakiram MSV argues that neutrality at Hugging Face has a hardware dimension that GitHub never had to contend with. Hugging Face maintains integrations spanning AWS Trainium and Inferentia, Google TPUs, Intel Gaudi, AMD Instinct and other accelerators. Under Nvidia ownership, continued support for those rival chips becomes a real test of just how neutral the platform will remain in the long run.

Not a done deal yet

As for the acquisition itself, well, it’s not over the line quite yet. In a filing with the US Securities and Exchange Commission (SEC), Nvidia notes that it expects the deal to close in the first half of 2027, subject to customary closing conditions and regulatory approvals. Given Nvidia’s lofty position in AI infrastructure and Hugging Face’s role as a major distribution point for open models, those approvals are unlikely to be a mere formality. Nvidia’s filing also flags the possibility that future regulation around open-source AI could affect Hugging Face’s operations or increase compliance costs.

“Hugging Face [will] continue to permit model makers, developers, and users to upload and download models and datasets of their choosing and to support other silicon vendors.”

Notably, Nvidia also uses its SEC filing to reaffirm its neutrality commitments, stating that under the commitment, “Hugging Face would continue to permit model makers, developers, and users to upload and download models and datasets of their choosing and to support other silicon vendors.”

Of the roughly $12.9 billion headline price, about $11.9 billion will go to Hugging Face stockholders, with up to $1 billion earmarked for equity-based retention awards for employees joining Nvidia.

And while the headline price is usually rounded to $12.9 billion, the actual figure is an oddly specific $12,930,300,000. That’s no accident either, as Hugging Face co-founder Thomas Wolf alludes to in a LinkedIn post. For those still in the dark, 129303 is the decimal Unicode value for the 🤗 emoji, while #129303 is a green color code nodding to Nvidia’s branding.

A neat little Easter egg buried inside what can only be described as one of the biggest AI deals of the year.

The post “Hugging Face will remain an open platform”: Nvidia strikes $12.9B deal for the ‘GitHub of AI’ appeared first on The New Stack.

A 438-billion-parameter reasoning model isn’t an obvious choice when speed is a priority. Multiverse Computing is betting that compression can make it one. 

On Wednesday, the Spanish company launched Quasar 438B, its first large-scale model and a system designed specifically for coding and enterprise agents. Quasar achieves a score of 43 on Artificial Analysis’ Intelligence Index and 69.3 on Terminal-Bench v2.1, while Artificial Analysis currently records its output speed at approximately 183 tokens per second. Multiverse is positioning Quasar as the European model with the highest score on the Intelligence Index, outperforming Mistral Medium 3.5 (30) and NVIDIA Nemotron 3 Ultra (38).

Multiverse is betting that a 438B-parameter model can be fast and inexpensive enough for agents that repeatedly reason, call tools, and check the results.

That puts Quasar in an interesting middle ground. It doesn’t match the strongest models on coding performance, but Multiverse is betting that a 438B-parameter model can be fast and inexpensive enough for agents that repeatedly reason, call tools, and check the results.

The model features a context window of one million tokens, is available in English and Spanish, and can be accessed via the Multiverse CompactifAI API.

Compression claims, missing details 

Multiverse built CompactifAI to shrink large AI models so they need less memory and compute to run. The company says it can reduce model size by 80% to 95% with only a small loss in accuracy, but it hasn’t disclosed how much Quasar was compressed or which model it started with.

In July, Multiverse announced a $570 million Series C to expand its library of compressed models and commercialize the technology. Quasar is the biggest test of that approach so far.

Multiverse hasn’t said what hardware is required to run Quasar or how much the compression reduces its memory and compute needs. That matters for agents, which may repeatedly call the model and other tools before finishing a task.

Multiverse hasn’t said what hardware it takes to run Quasar or how much the compression cuts its memory and compute needs.

Coding benchmarks show tradeoffs 

In Multiverse’s comparison, Quasar’s Terminal-Bench v2.1 score of 69.3 places it ahead of Mistral Medium 3.5 but still well behind the best frontier systems, while Claude Opus 5 achieves the highest score on that benchmark at 89.1.

Multiverse is pitching Quasar for software engineering, technical copilots, research, and workflow automation. Its 1-million-token context window gives agents room to work with large codebases and hold onto information as a task progresses, although processing more context also requires more compute. That can be especially important in coding, where code that passes every test can still trip up the next AI agent if it loses track of what came before.

Agent latency beyond throughput 

Artificial Analysis found that Quasar starts responding in about 1.1 seconds and can produce a 500-token response, including reasoning, in around 15.3 seconds. Those numbers are fast, but an agent also has to wait for tools, process growing context, and make repeated model calls over the course of a task. The agent tooling layer itself is still catching up to what these models need.

Those numbers are fast, but an agent also has to wait for tools, process growing context, and make repeated model calls over the course of a task.

Proprietary model, open questions 

Quasar is proprietary and only available through Multiverse’s API, so developers can’t inspect the weights or run it on their own hardware. For now, that also makes it difficult to know whether the speed Multiverse is reporting will carry over to everyday agent use.

Quasar also arrives as European AI companies are building more of their own model and compute infrastructure instead of relying on U.S. hyperscalers. Multiverse is taking a different route, using compression to make a 400B-plus model cheaper and faster to run. The next step is to see how that holds up against benchmarks.

The post Multiverse says its 438B model is fast enough for AI agents. The benchmarks tell a more complicated story. appeared first on The New Stack.

This post is the third in a series on AI model co-design. It explores how to accelerate LLM inference while maintaining accuracy using speculative decoding and...

This post is the third in a series on AI model co-design. It explores how to accelerate LLM inference while maintaining accuracy using speculative decoding and offers five guidelines for selecting draft length and draft mechanism across the Pareto frontier. For a discussion of how model design choices impact both throughput and interactivity without sacrificing accuracy, see AI Model Co…

Source

Alibaba this week unveiled Qwen3.8-Flash, an open-weight, multimodal Mixture-of-Experts (MoE) model. 

Hot on the heels of Qwen 3.8 Max, which arrived at the start of the month, this 125-billion-parameter AI model is offered as a prelude to Qwen 4. It is positioned as both a performance and value-for-money play. As such, it is claimed to have “superior capabilities in coding and office tasks” and an optimal balance among capability, latency, and cost.

“In this release, we are opening the weights of Qwen3.8-Flash-Next, a multimodal MoE model that also serves as an early preview of the architecture used in Qwen4,” confirmed Alibaba in its release blog.

How and why does Alibaba offer early access precursor models?

By releasing the architectural changes in Qwen3.8-Flash early, the organization hopes the community will examine the mechanics, constructs, and components within and start road-testing them before the full Qwen4 model family is built on top of them.

By stating that Qwen3.8-Flash paves the way for Qwen4, Alibaba is showcasing (and, importantly, openly sharing) design forms that it will carry into its subsequent models, so that developers can start building (or at least planning) their next codebases early. 

Specifically then, Alibaba has stated that Qwen3.8-Flash “plays the same role” that Qwen3-Next played for Qwen3.5 i.e. by which the company means that Qwen3.8-Flash introduced developers to the company’s hybrid Gated DeltaNet + Gated Attention design (computational components that enable AI models to balance long-context efficiency with contextual focus and attention during inference and training), which was then used across the Qwen3.5, Qwen3.6, Qwen3.7 and Qwen3.8 series.

“Qwen3.8-Flash-Next upgrades the model systematically along four aspects – attention, residual, embedding and optimization –  improving model capability while further optimizing computational efficiency, model capacity and training stability,” stated Alibaba.

Benchmark scores against rival models

When benchmarked on agentic coding, long-horizon agent tasks and multimodal intelligence, Qwen3.8-Flash appears to perform respectably against models including DeepSeek-V4-Flash and Claude-Opus-4.6 across SWE-bench Pro (agentic coding), CoWorkBench (long-horizon office work), Toolathlon Verified (real-world tool use), MathVision (visual maths problem-solving), AndroidWorld (agentic mobile use) and ERQA (embodied intelligence).

“Qwen3.8-Flash-Next upgrades the model systematically along four aspects – attention, residual, embedding and optimization – improving model capability while further optimizing computational efficiency, model capacity and training stability.”

Tested on agentic coding using SWE-bench Pro, Qwen3.8-Flash-Next scores 62.5, compared to 61.7 on Qwen3.8-27B, 55 on Qwen3.7-Plus, 56.0 on DeepSeek-V4-Flash-0731, and 53.4 on Claude-Opus-4.6 (Max).

Importantly, Qwen3.8-Flash requires only what Alibaba details as “around one-ninth of the training resources,” while delivering superior performance. The company says that this means Qwen3.8-Flash “significantly reduces” both training and inference costs compared with Qwen3.7-Plus, a model three times its size. 

What’s the difference between Qwen3.8-Flash-Next and Qwen3.8-Flash?

For the sake of nomenclature, Qwen3.8-Flash-Next is the open-weight research-frontier model available to developers on both the Hugging Face AI developer hub and Alibaba’s ModelScope community portal. Built on the same underlying architecture, Qwen3.8-Flash is the production version of the model, offered via the QwenCloud API with 1 million tokens by default and official built-in tools. 

Qwen3.8-Flash-Next features a 125B-parameter main model, supplemented by an additional 51B N-gram embeddings, with 6B parameters activated per token. It natively supports 262,144 tokens of context and can be extended to 1,000,000 tokens with YaRN.

What architectural updates have happened?

As noted, Qwen3.8-Flash introduces architectural extensions across attention mechanisms, residual connections, embeddings, and optimization. Its hybrid attention architecture combines Gated DeltaNet (GDN), which compresses historical information, with Qwen Sparse Attention (QSA), a novel design that uses a lightweight compressed indexer to select relevant context, substantially reducing attention costs for long sequences.

Alibaba has explained that the Gated Residual (GR) mechanism expands data pathways between layers while strengthening cross-layer information flow and training stability. The N-gram Embedding technique scales model capacity with minimal additional computation, while the Muon Optimizer enhances the efficiency of large-scale model training.

“I’d love it if local LLMs actually could replace remote ones today, and I have no reason to lie about my experience either…it’s just not there (yet) for local professional software development.”

What do developers think of Qwen3.8-Flash?

In terms of developer reaction, it’s been a mixed bag so far. 

Mechatronics engineer Alok posts on X, saying he thinks the Video RAM barrier (i.e., the need for physical, high-speed GPU-based memory for LLM memory management in the face of model quantization that aims to enable better long-context inference) is now officially dead. 

“I just ran Qwen3.8-Flash-Next (MoE) 125B A6B with a  250,000 context window on a single 24GB RTX 4090 – 21 tokens/sec decode. 364 t/s prefill – no mtp. No dflash. No KV cache quantization! We are running datacenter models on consumer hardware,” enthused Alok.

Multi-disciplined software developer Embedding Shapes is less happy.

They post on Hacker News using some colorful language to describe how models keep [insert expletive]-ing up very basic things before saying, “I’d love it if local LLMs actually could replace remote ones today, and I have no reason to lie about my experience either. But I too got hopeful reading the sentiment on the Internet about Qwen 3.8, but it’s just not there (yet) for local professional software development.”

Pricing and access

Qwen3.8-Flash can be accessed via API Model Studio and Qwen Cloud, Alibaba’s AI-native cloud platform. Pricing per 1 million tokens is US$0.16 for input and US$0.47 (or 3 RMB for developers inside China) for output.

The model is also available on QwenWork, Alibaba’s workplace AI agent platform, where it runs a redesigned “standard mode” that cuts token consumption per task by 75% and “roughly doubles generation speed” compared with the current mode.

Alibaba has claimed that this brings flagship-level capabilities within reach of everyday workloads and that developers can run this model on hardware that they likely already own.

When is Alibaba’s Qwen4 scheduled for launch?

Alibaba has not confirmed a firm launch date for Qwen4. Still, a casual web search on the topic yields a range of commentators and market watchers who broadly agree it will arrive before the end of the year, possibly as early as September.

Conjecture in this space explores whether the next model family could be optimized for complex 3D coding and design tasks in advanced spatial modeling. Others think that the Mixture-of-Experts (MoE) architecture will be extended and that deeper native multimodal processing capabilities might be featured.

Alibaba was contacted for broader comment on this story but declined to engage.

Qwen, in traditional Chinese, 通義千問 (pronounced Tōngyì qiān wèn), translates to “a thousand questions on general meaning” in English. Use that in your local pub quiz this weekend.

The post Alibaba just released Qwen3.8-Flash: “An early preview of the architecture in Qwen4” appeared first on The New Stack.

Growing datasets and public benchmarks are making it harder to tell whether a model is being tested on something it hasn’t seen before.

On Thursday, Google DeepMind showed off what the company calls the first double-blind evaluation of a proprietary frontier-class AI model. The setup keeps Gemini’s model weights hidden from the evaluators while the test questions are hidden from Google.

The pilot tested Gemini 2.5 Flash Lite against private benchmarks from MLCommons and the Singapore AI Safety Institute, but rather than using the results to tout a new Gemini score, Google is focusing on how the tests were run, with neither side having access to the other’s data.

Benchmark leakage inflates scores

Google’s technical report cites earlier research that found signs of benchmark leakage in about half of the 31 models tested, and another study published this year that found contamination can inflate scores, particularly for larger models. This dynamic has already raised questions about where certain post-training coding gains actually come from.

Keeping benchmarks private would seem like the obvious answer, except closed models make that difficult. With a closed model, evaluators usually have to run their questions through the provider’s API, which means the company can see a test that was supposed to stay private, whereas running it independently would require the provider to hand over its model weights. Google’s setup is meant to give both sides another option.

Keeping benchmarks private would seem like the obvious answer, except closed models make that difficult.

How the enclave works

The pilot is using Google Cloud Confidential Space together with an NVIDIA H100 Confidential GPU and Intel TDX host memory encryption.

Google DeepMind provides Gemini with its weights and inference code, while the evaluator offers its benchmark prompts and evaluation code; these are then transmitted over encrypted connections into the enclave, where the evaluation takes place without either party gaining access to the other’s protected assets.

The model weights are stored in hardware-encrypted GPU memory, and the evaluation prompts are kept in encrypted host memory as well. After the test is complete, the evaluator is given the permitted results, and the temporary environment can then be destroyed.

Before either side sends over its private data, remote attestation verifies that the enclave is running the software they agreed on.

Code controls beyond encryption

The enclave protects what’s stored in memory, but the code running inside it can still create problems. If an application has unrestricted network access, for example, it could send sensitive information somewhere else.

OpenMined’s PySyft handles that part of the process by allowing Google and the evaluator to approve the code in advance and to block sensitive parts of the evaluation from making external connections.

That process takes some work, although compute doesn’t appear to be the problem. The paper puts the overhead at less than 5% and instead points to legal agreements and code reviews between organizations as some of the bigger hurdles to running these evaluations.

The researchers eventually want to make attestation much less hands-on, comparing the goal to the HTTPS lock icon in a browser, where the security checks happen without the user having to deal with the underlying hashes and keys.

Google still has a hand in verification because, although the Confidential Space guest OS is open source and its build process has been externally validated, individual builds rely on private signing keys and can’t be independently reproduced. At the same time, Google’s own services are used to sign and verify the attestation report.

The paper puts the overhead at less than 5% and points instead to the legal agreements and code reviews between organizations as some of the bigger hurdles to running these evaluations.

Scaling past one GPU

So while the system reduces how much the two sides need to trust each other, it doesn’t remove trust entirely from the equation. Some of it simply moves elsewhere, including to the hardware itself. There is still some trust involved in the hardware itself, including the assumption that the cloud provider and the hardware maker aren’t working together to circumvent the protections.

Then there’s the benchmark itself. MLCommons points out that it still needs to be carefully managed, no matter how well the questions are protected.

The pilot ran Gemini 2.5 Flash Lite on a single H100 80GB Confidential GPU. Still, the researchers are already looking at clusters of H100 and B200 GPUs connected via encrypted links to eventually evaluate models that are too large to fit on a single GPU.

If this approach catches on, developers could have another way to look beyond the benchmark score, with some proof that the company behind the model didn’t see the test before the results came out.

If this approach catches on, a benchmark score could offer some proof that the company behind the model didn’t get to see the test first.

The post Google found a way to test Gemini without seeing the questions appeared first on The New Stack.