Running an LLM on your PC is easy enough, but putting an agent to work there is a different story. Portable Computer, the local version of Perplexity’s Computer agent, is now available inside the Perplexity app for Windows on compatible Nvidia GeForce RTX and RTX PRO GPUs.

That’s the good news; the catch is, you’ll need an Nvidia GPU with at least 24GB of VRAM.

The Windows launch gives Perplexity three platforms in less than three weeks. Portable Computer debuted on Linux and Nvidia DGX Spark on August 25, followed a week later by hybrid compute for Apple silicon, which splits tasks between local and cloud models on Macs. Now Windows joins the mix, but bringing Portable Computer over took more than simply porting the app. Perplexity had to adapt the model runtime, orchestration, security, and hardware integration for each platform while keeping the user experience the same.

you’ll need an Nvidia GPU with at least 24GB of VRAM to use it.

Orchestration beyond the model

Portable Computer bundles those pieces together. On Windows, it supports PPLX 27B — Perplexity’s post-trained model — and Qwen 3.8 27B, both optimized for RTX GPUs, alongside a built-in browser, tool calling,  and Perplexity’s proprietary SPACE sandbox.

It’s a different lane from LM Studio or Ollama, which make running models locally as painless as possible but stop well short of giving a model autonomy over multistep work. DeepSeek’s recent hiring spree of roughly 150 new roles, nearly all of them focused on agent infrastructure rather than the model, hints at how much engineering sits between a capable model and a capable agent.

DeepSeek’s recent hiring spree of roughly 150 new roles, nearly all of them focused on agent infrastructure rather than the model, hints at how much engineering sits between a capable model and a capable agent.

Connectors blur local boundaries

Perplexity ships connectors for Microsoft Outlook, OneDrive, and Word, plus Google Drive, Gmail, Slack, and GitHub — which tells you something about what “local” actually means here.

The agent can reach external services because it’s not air-gapped. Locally completed tasks can process files without sending documents to a cloud model. Once an agent has access to both local files and remote APIs on the same machine, figuring out which resources it actually needs and where to find them gets harder.

Hybrid cloud as fallback

Perplexity isn’t pretending that a 27-billion-parameter model running on a desktop GPU can handle everything, which explains the hybrid architecture. When the agent determines that a task needs more reasoning power than the local model can deliver, it can escalate to Perplexity’s cloud models.

According to Nvidia, the agent identifies when cloud support would help and asks the user for permission before sending any data off the machine.

For organizations handling sensitive or regulated data, that split can make all the difference. A local agent can grind through source code or financial records without uploading them to a hosted model for basic processing. There’s a cost angle too, since tasks completed locally don’t burn Perplexity Computer credits.

High VRAM floor limits reach

Portable Computer is available with Perplexity Pro ($20/month) and Max ($200/month), across individual and enterprise plans, with Nvidia DGX Station support coming later. The real challenge is taking local agents from developer passion projects to enterprise-ready tools. By baking this into Windows, it immediately gets in front of the scale of users needed to make that happen.

The real challenge is taking local agents from developer passion projects to enterprise-ready tools.

The post Perplexity’s new agent runs entirely on your GPU — with one expensive catch appeared first on The New Stack.

Last October, the P99 conference — the online gathering for developers focused on high-performance, low-latency applications — featured a cracking keynote from Chip Huyen. 

The author of the best-selling AI Engineering, Huyen opened with simple math: Training a frontier model is a one-off cost, but inference is the same cost paid over and over. That’s great for the frontier model providers, and bad for us token burners. Over the life of a model, Huyen reckons the compute split ratio lands somewhere between 1:10 and 1:100 for training to inference. Reasoning models — which burn even more tokens — push that out even further. We all know the feeling of hitting our weekly session quotas.

Huyen’s point is that if inference is too expensive, then nobody ever recovers the training bill, which might explain why there are so many memes about the “profitability” of frontier models. So, how do we optimize inference? 

That’s a topic that Huyen spent months researching for her book. And in the spirit of optimization, Huyen distilled it down to 30 minutes for the conference in October 2025.

Huyen is returning for P99 CONF 2026 in a few weeks. Ahead of that moment, watch her full talk – or read the recap below – from last year, then let’s talk about how those ideas aged over the past 11 months.

What to measure

Chip recommends focusing on a few key latency metrics:

• Time to first token (TTFT): How much time elapses before the user sees anything
• Time per output token (TPOT): The average time between consecutive tokens (aka inter-token latency)
• End-to-end latency: Time to first token, plus time per output token, multiplied by the number of output tokens minus one

With reasoning models, some of those tokens never reach the user. “The first generated token might not be the same as the first visible token,” Huyen explained. “The model might think for a while, and it will only show the first token of the final output to the user.” 

“The first generated token might not be the same as the first visible token,”
— Chip Huyen

Some people also measure Time to Publish for that (i.e., how long until the user sees the first token). The best metric to prioritize depends on what matters most for your users. 

Also consider “goodput” alongside throughput. Throughput measures requests processed in a given window. Goodput measures the requests that actually met your targets. Chip’s example: an app targets 200 ms time to first token and 100 ms time per output token, and processes 10 requests per minute, but only three hit both. 

3 ways to optimize LLM inference

With inference servers, you can optimize from 3 different angles: the hardware, the model, and the service that manages the requests and responses.

Huyen previously worked at Nvidia and opted out of the hardware discussion: “Even though I find it to be an intellectually interesting topic, it’s not relevant to a lot of people because we don’t have the power to change the hardware itself,” Huyen explained. She also didn’t want to spend much time on the obvious solution: replica parallelism, or just adding more machines. It’s costly, and it gets complicated fast – especially if you end up with a mix of 80GB, 48GB and 24GB machines and models of varying sizes to distribute across them.

That leaves the model and the service. Huyen offers these tips on how to decide: “If you want to host the models yourself, or if you have access to the model weights, or if you train a model yourself, or you want to fine-tune or distill a model, then model optimizations might be for you. However, if you want to take a model as-is and make it more efficient on your own inference service, you might want to look into service optimizations.

Model optimization

The following techniques change the actual weights so that they can change the model outputs.

Quantization lowers the precision used to store weights and activations  (e.g., from four bytes per parameter at 32-bit to one byte at 8-bit). Huyen explained, “Reducing the precision not only reduces the memory requirement to run the model, making it cheaper. It can also make the model a lot faster. If you do additions bit by bit and each weight is 32 bits, you have to do it 32 times. If it’s 8 bits, you only have to do it eight times.”

The tradeoff is a small quality hit. Huyen continued: “It’s possible to reduce a lot of the model’s memory footprint with minimal quality degradation, and quantization is pretty generalizable to a wide variety of model architectures and model sizes. That’s why it’s very popular. I rarely see any companies running a model at full precision anymore.”  

“I rarely see any companies running a model at full precision anymore.”
— Chip Huyen

Distillation involves using a large model to generate training data for a smaller model. For example, say you have a truly large model (the example Huyen used was o1) and want a model that performs like it, but is much smaller. Basically, you collect a large set of prompts, run them through the larger model, then train the smaller model on its responses.

Proceed with caution, though. Huyen warned, “A lot of model providers have the condition that they do not allow their models to be used to train competitive models. So even though it’s a very common technique, you need to check licensing.”

Service optimization

This set of techniques targets how requests are scheduled, routed, and reused. The actual weights aren’t affected.

Batching groups multiple requests so they’re processed together in a single pass through the model – which is much more efficient than dealing with them one at a time. Huyen presented a few batching options:

• Static batching waits for the batch to fill. This maximizes compute utilization, but it might increase the latency for the first requests.
• Dynamic batching runs on a timer instead (e.g., batching every 15 ms). This is less compute-efficient, but it’s better for latency.
• Continuous batching handles the case where requests finish at wildly different times, which is common with LLMs. One request asks for the capital of Vietnam; another kicks off deep research. With static or dynamic batching, the finished request’s slot sits idle until the slowest one completes – and new requests queue up behind it. Continuous batching returns each request as it finishes and fills the spot with another request. That can improve compute resource utilization and latency.

Decoupling prefill and decode separates the two phases of a request onto different machines. (Prefill processes the input, while decode generates the output.) Huyen said, “Input tokens can be processed in parallel, whereas output tokens need to be generated sequentially. With parallel processing, it’s bounded by compute, the processing power of the chip. With decoding, it’s bounded by memory, because you have to move model weights.” 

Because each phase stresses different resources, most services now separate them. To improve time to first token, shift machines toward prefill. If you care more about improving time per output token, shift them to decode. 

Parallelism splits work across machines. Replica parallelism copies the whole model onto more machines. Tensor parallelism divides a very large matrix, so different machines compute different parts of it. Pipeline parallelism divides the model by layer, so requests move through as a pipeline. 

Prompt caching processes shared text once, saving cost and latency. A lot of repetition exists across requests to the same application: the system prompt, the examples, the same code base, the same document behind different questions. You might as well process that shared segment once, cache it, and reuse it.

The technique was relatively rare when Huyen was writing AI Engineering. “There was one paper about it, and it was not really known, but it made a lot of sense. So I included prompt caching in the book, and I’m very happy to see that nowadays it’s pretty much everywhere.”

The savings scale depending on how much of your prompt gets cached. In Claude Code logs, Huyen’s open-source tool Sniffly found cache hit rates of 90% to 97%. Some providers rewrite prompts internally to improve hit rates, but you might as well structure them yourself. 

Huyen’s tip: since caching works on shared prefixes, put the stable parts of your prompt first and the variable parts later. “It’s pretty easy to do, and it can improve your application performance significantly,” she noted. 

Evaluating inference providers

Huyen closed with a warning for anyone evaluating inference providers: “There are many inference companies that provide inference optimizations for models you want to use, and a lot of them advertise just cost and latency. 

“But pay attention to how many inference optimization techniques also change the model behavior or reduce the model quality. So when evaluating an inference service, it’s important to look not just at cost and latency, but also at model quality. Does this model, provided on this service, also perform similarly on standard benchmarks?”

What’s changed one year later?

So where do we stand today, one year on from this keynote? Most of it actually aged quite well. 

On the economics, I reckon Huyen was bang on… I think, for most of us as users, we don’t have all the cost levers to pull that Huyen outlined. But it’s great to understand what is happening. As a novice local LLM user myself, I found I could relate to her points on parallelism (I don’t have it) and prompt caching/quantization (within my grasp of control). 

Prompt caching (which Huyen said was new when Huyen wrote AI Engineering) is now priced into every bundle purchase of API tokens. And her Claude Code observation (90% cache hit rates) is probably the reason we mere mortals can still afford agentic coding agents at all.

Some of it aged in ways that were hard to predict at the time. Huyen mentioned how reasoning models make inference even more significant. One year on, I think agents running multi-step loops with tool calls have turned that idea from a footnote into a way to turn Claude’s rate limits (and their infamous 99.x% availability) on their head. 

All the metrics Huyen described – time to first token, time to publish, goodput under a latency SLO, etc., are all now part of the lingo and probably need to be reasoned about differently. 

That’s one thing I hope she’s talking about this year! Grab a free conference pass and join us online. 

Grab a complimentary pass to PG 99 Conf 2026 and join us on October 21 and 22 to chat with Huyen.

The post Chip Huyen explains how to cut inference costs without new hardware appeared first on The New Stack.

OpenAI rolled out its Agents API in public beta Thursday, opening the backend behind Codex to developers looking to run agents unattended for days.

Now, developers don’t have to build their own system to keep an agent going because the API tracks the job as it progresses and gives the agent somewhere to execute its work, even when a task stretches well beyond a single context window.

That makes long-running agents easier to try, but it also gives developers more ways to burn through compute. Interestingly enough, on the same day Agents API launched, OpenAI paused new sign-ups for its $200-a-month Pro plan after demand for GPT-6 Astra strained capacity.

Thibault Sottiaux, engineering lead for Codex, writes on X that Pro subscriptions “put the most strain on our systems,” adding that OpenAI was working to add capacity “as fast as we can.”

To make sure our current users have an incredible experience and continued access to Astra, we are going to pause subscriptions to our $200 Pro plan. These put the most strain on our systems and we wanted to take the smallest step that allows us to continue giving the broadest… https://t.co/WhLEm3HBL7

— Tibo (@thsottiaux) September 10, 2026

The Agents API and ChatGPT Pro are separate products, so there’s no reason to assume one is taking capacity from the other. Still, the timing stands out: the company is making it easier for developers to run agents for hours or days while pulling back access to its heaviest-use consumer plan and working to add more capacity.

Agent inference adds up fast

As a task gets longer, the API can compress earlier context, so the agent doesn’t just stop when it reaches the model’s context limit. It can also bring in tools only when they’re needed or send parts of a larger job to subagents working in parallel. The actual work can run in OpenAI’s sandbox or on infrastructure the developer controls.

The actual work can run in OpenAI’s sandbox or on infrastructure the developer controls.

As agents make progress, they go back to the model for the next step, and a job that takes hours can rack up far more inference than a typical API call. The usage climbs even faster when agents work in parallel.

OpenAI has already seen this inside its own shop. In a research report published September 6, OpenAI said its research organization was logging 3.1 agent-workdays for every human workday by mid-August, measured in standard eight-hour equivalents. The median researcher, ranked by agent usage, was spending more than $600 per day on inference at API prices, while the 90th percentile exceeded $7,000.

Before June, OpenAI’s researchers were still putting in more hours than their agent, but by mid-August, the agents were doing three times as much work.

Arguably, OpenAI’s researchers are an extreme case, but the numbers show what happens when agent use starts to scale. One person can suddenly generate far more inference than their headcount would suggest.

One person can suddenly generate far more inference than their headcount would suggest.

Friction limited compute demand

The Agents API lowers the cost of that experimentation by leaving the orchestration layer out of the bill. Developers pay for the models, tools, and hosted compute their agents actually use.

The flip side is that it’s now easier to consume more inference. Context compaction is a good example. A full context window used to force developers to decide what to discard or how to summarize the work so far. Now the API handles that automatically and the agent keeps going. That’s useful for developers, but it also means the workload doesn’t stop when the context window fills up.

Astra demand hit the ceiling

The Astra rollout offers a preview of what that could look like. OpenAI stopped accepting new Pro subscribers less than two weeks after the model launched on September 3, saying those accounts put the most strain on its systems. The Agents API has its own rate limits and usage tiers, so the Pro pause doesn’t directly affect developers using it. Still, the company is already having to manage capacity around its newest model.

Infrastructure outweighs benchmarks now

The more agents developers run, and the longer they run them, the faster that usage adds up. One developer might have several agents working at once, each going back to the model throughout the task. So headcount alone doesn’t tell you much about how much compute you’re using.

For long-running agents, the challenge is keeping the work moving without wasting tokens or losing track of the task. Cloudflare made a similar bet this summer, arguing that the infrastructure around AI workloads would eventually matter as much as the models themselves.

For long-running agents, the challenge is keeping the work moving without wasting tokens or losing track of the task.

The post OpenAI’s researchers burned $7,000 a day on AI agents — now it’s opening the floodgates appeared first on The New Stack.

Hundreds of thousands of AI agent sandboxes can already run concurrently on a single DeepSeek cluster. Now the company is staffing up to handle what happens as that number — along with its training, evaluation, and other backend workloads — keeps climbing.

Cui Tianyi, who joined DeepSeek in March and works on its Harness team, the group responsible for the infrastructure and environments used to run and evaluate agents, announced in an X post that roughly 150 engineering positions on September 7, with the hiring concentrated in server-side engineering and Agent Elastic Compute rather than AI research. The work spans operating systems, virtualization, networking, storage, scheduling, and the control-plane services that coordinate those resources.

Cui said DeepSeek’s existing backend systems will need upgrades, maintenance, and rewrites as workloads grow. One such system at the center of that scaling challenge is DeepSeek Elastic Compute, or DSec, the sandbox infrastructure DeepSeek built to execute agent workloads during post-training and evaluation.

Cui said DeepSeek’s existing backend systems will need upgrades, maintenance, and rewrites as workloads grow.

Four sandboxes, one SDK

Agent workloads require more than GPUs for inference, with each agent also needing an isolated environment to run code, call tools, change files, and collect the results.

DSec supports four types of those environments through the same Python SDK. Simple function calls go to pre-warmed containers, while Docker-compatible containers handle jobs that need a persistent environment. DeepSeek uses Firecracker microVMs when stronger isolation is needed and QEMU virtual machines for workloads that require a full guest operating system.

That range means the same infrastructure can handle anything from a simple tool call to a software-engineering task that needs an entire OS. It’s a similar challenge to the one the rest of the industry is bumping into as agents move from demos to production. OpenAI, for instance, recently designed custom silicon specifically to address the compute pressure that agent workloads create, and DeepSeek open sourced its own agent harness in August.

Lazy loading agent environments

Every sandbox needs its own environment, but copying complete container or VM images onto every host would consume enormous amounts of storage and network bandwidth while adding to startup time. DeepSeek gets around that by tying DSec into 3FS, the distributed filesystem it originally built for its AI infrastructure, and keeping container base images and filesystem commits as read-only layers backed by 3FS.

The metadata stays local, but the underlying data blocks are fetched only when they’re actually needed. MicroVMs use a similar setup, sharing their read-only base layer through 3FS while writes from individual sandboxes are kept in local copy-on-write layers.

DeepSeek says DSec reduces duplicate page-cache usage across virtualized environments and reclaims memory to allow safe overcommitment, while changes to the container runtime cut the CPU overhead of each sandbox.

The team also had to deal with spinlock contention inside the container runtime. At small scale, the CPU time spent there barely registers. At scale, it limits how densely those environments can be packed onto each host.

DeepSeek says DSec reduces duplicate page-cache usage across virtualized environments and reclaims memory to allow safe overcommitment, while changes to the container runtime cut the CPU overhead of each sandbox.

When replay breaks training

During reinforcement learning and other post-training workloads, large numbers of agent rollouts can be running at once, and jobs may be interrupted as compute gets reassigned. Starting over wastes everything the agent has already done, but picking up where it left off isn’t as simple as replaying its previous commands.

Some of those commands may have changed a file or otherwise altered the environment, so running them again could produce a different result or leave the training trajectory in the wrong state. DSec avoids that with a globally ordered trajectory log that records commands along with their results.

When a rollout resumes, DSec can fast-forward through the completed work using those recorded results rather than executing the commands a second time. That reduces the cost of interruptions across thousands of training and evaluation runs, while the same logs preserve a history of how each sandbox changed and allow earlier sessions to be replayed.

Engineers, not researchers, wanted

The roughly 150 openings reach across DeepSeek’s backend, including the lower-level systems work behind Agent Elastic Compute as well as the services that support its models and agents.

DeepSeek said in June that it planned to at least double the size of every department, but this round of hiring leans heavily toward the systems underneath its models rather than the models themselves. DSec is part of that work, with hundreds of thousands of sandboxes running concurrently and putting pressure on everything from how jobs are scheduled to how they recover after an interruption.

The roughly 150 openings reach across DeepSeek’s backend, including the lower-level systems work behind Agent Elastic Compute as well as the services that support its models and agents.

The post DeepSeek is hiring 150 engineers, and none of them will touch a model appeared first on The New Stack.

Ask a traditional enterprise application for a customer address or today’s revenue figures and, broadly speaking, it follows a predictable route its developers have already mapped out: authenticate the user, query the right system, return the result. Given the same underlying data, you’ll get the same answer each time.

Ask an AI agent the same question, and the journey is much harder to forecast. It might consult one system, decide it needs more context from another, make a dozen tool calls, pass information through a language model and only then produce an answer. Run the same request again, and it may take a different route altogether.

And in an enterprise, what happens along that route can matter just as much as the answer: which systems the agent accesses, what data it sees, what actions it takes and how much it spends.

That distinction — between predetermined software, and applications that make probabilistic decisions on the fly — sits at the heart of a new company from a founder who knows a thing or two about bringing order to a new generation of infrastructure.

AI agents are hard to govern

Dome Systems was co-founded at the turn of the year by David McJannet, who spent close to a decade leading Terraform-creator HashiCorp through the cloud era, culminating in its blockbuster 2021 IPO and subsequent $6.4 billion sale to IBM in 2025. McJannet is joined at the helm by Marc Holmes, who spent more than six years at HashiCorp as chief marketing officer.

In an interview with The New Stack, McJannet lays out his company’s thesis on AI agent governance, arguing that enterprises are now running into the same kind of problem that they did with cloud infrastructure: adoption comes first, then the real spadework begins of putting the right controls in place across security, operations and finance.

“It’s actually a very different architecture, and that is what unlocks the power of these new [agentic] applications.”

Part of the challenge, he says, is that agents are built very differently from the enterprise applications of yore, which companies spent years learning how to control.

“It’s actually a very different architecture, and that is what unlocks the power of these new [agentic] applications,” McJannet explains.

He points to self-driving cars as an example: a model takes in live inputs and interacts with the vehicle’s systems as conditions change, because no developer can reasonably pre-program every possible situation a car might encounter on the road.

“It’s making judgments along the way, as opposed to trying to look up the historical maps of the world and make a real-time decision,” McJannet continues.

An enterprise agent can behave in much the same way: call one tool, assess the result, decide it needs another, and keep going until the task is complete. That flexibility lets agents tackle work that would be difficult to script exhaustively in advance — but it also makes their behaviour harder for enterprises to govern.

And this gets to the heart of what McJannet is striving for with Dome.

Table stakes for the agent era

The company launched out of stealth back in April with $14 million in seed funding, with McJannet having departed HashiCorp the previous August after the IBM transition concluded.

Dome’s starting point is that an agent combines three things: code, a model, and the backend systems or tools it interacts with. Bringing those pieces together under one platform, McJannet says, is “table stakes” for applying meaningful constraints to what the agent can do.

“If you don’t have an integrated platform, you can’t enforce controls across everything that the agent is doing,” McJannet says.

“If you don’t have an integrated platform, you can’t enforce controls across everything that the agent is doing.”

And so Dome’s platform is built around those three elements. An agent registry keeps track of the agents themselves; an MCP gateway controls the tools they can call; and a model broker/router governs which models they can use and how requests are routed.

The setup starts by registering the agent and giving it an identity, establishing who is allowed to call it, and connecting the backend tools it can reach — Zendesk, in this example.

Next, Dome connects a model provider, groups available models into a pool with routing and failover rules, then combines the agent, its tools and its models behind a single gateway. That gateway becomes the point through which Dome can apply the policies governing what the agent is allowed to do.

Once those pieces are connected, teams can set permissions on each call, use guards to inspect responses, apply quotas to cap spending, and keep a common audit trail across the agent’s activity.

Today, McJannet says, enterprises are often piecing all of this together themselves. A standalone model broker might be brought in to control spending, while a separate tool gateway handles security and operational concerns. Some are then building their own agent registry to tie those systems together.

Moreover, buying those capabilities separately leaves enterprises with another integration problem to solve. A model router might govern one part of an agent’s activity and a tool gateway another, while the agent itself continues moving between them.

“If you just provide the tool gateway or just the model router, it doesn’t allow you to have this kind of system of control,” he says.

That is also where Dome’s latest move enters the fray. After spending its first months in early access, the company is now opening the platform to self-service users for the first time, allowing teams to sign up with little more than a credit card, bypassing the typically arduous enterprise sales process.

Dome goes self-serve

Self-serve is relatively unusual route for this kind of enterprise infrastructure product. Dome is publishing its prices, offering a free tier and letting practitioners get started without first going through a sales process, while keeping the traditional enterprise route open for larger customers.

The thinking is partly about who McJannet expects to use the product. Rather than limiting access to buyers who are already deep into a procurement process, for example, self-serve enables individual practitioners to be able to discover, try and use the platform themselves.

“”We want to make the barrier as low as possible to have people come on board,” McJannet says, adding that Dome had already seen a number of self-service sign-ups ahead of the launch.

Separately, its pricing reflects a belief about where value will ultimately sit in this market. McJannet regards model routing and tool connectivity as baseline capabilities, with the more valuable piece being the controls that sit across the agent as a whole — think permissions, data redaction and spending quotas.

It’s also worth noting that while Dome’s main target user will be platform engineering teams inside large enterprises, typically working alongside operations and security, self-serve also creates an opening for another kind of user: the small company, perhaps even only one or two people, building an agent and trying to sell into an enterprise. The sort of scenario that aligns with the fabled one-person unicorn promised by many in the AI realm.

Indeed, McJannet says developers can get far building the application itself, only to hit a wall when a prospective enterprise customer begins its security and operations review. How is identity enforced? Who can see the data the agent reaches? What happens when it calls other agents? Can its activity be reconstructed afterwards?

Some builders, he says, have asked whether they can “certify” their agents on Dome because “my agent won’t get deployed until I can satisfy these infrastructure elements.” McJannet is careful to add that Dome doesn’t currently run such a certification program, but it’s clearly one route the company could venture down.

“If you register that agent on Dome, all the infrastructure elements are taken care of,” McJannet says.

‘Unblocking AI agents’: Lessons from the cloud era

That division between developers eager to ship, and enterprise teams worried about what happens after, is also where McJannet sees the strongest parallel with his years at HashiCorp.

During McJannet’s tenure, HashiCorp increasingly positioned itself around helping large organizations standardize how cloud infrastructure was provisioned, secured and connected. That included the 2020 launch of HashiCorp Cloud Platform (HCP), which offered its infrastructure tools as managed cloud services.

More broadly, McJannet’s account of early cloud adoption begins with developers swiping a credit card and deploying directly to Amazon because cloud infrastructure allowed them to build applications that had previously been impractical. The applications were compelling enough that enterprises adopted cloud despite resistance from operations and security teams, and what followed was a second phase: companies needed common services for provisioning, credentials, networking and other controls before cloud could become routine across the organization.

Platform engineering teams became the people responsible for reconciling those two demands: allowing developers to build while giving security, operations and finance enough control to permit those applications into production. McJannet believes agents are now creating the same tension.

“You’ve got this queue of cool apps that developers build that the ops and security teams are just not comfortable letting flourish in their environments.”

“You’ve got this queue of cool apps that developers build that the ops and security teams are just not comfortable letting flourish in their environments,” he says. “And so, inevitably, it has to go that same direction where the platform engineering team has to figure out [a way] to get to say ‘yes’.”

Dome’s bet is that enterprises will eventually prefer one system spanning the entire agent to a patchwork of gateways, routers and security products. In McJannet’s telling, that common control layer is what gives enterprises a way to limit how far an agent can roam while still letting it act autonomously.

“You have to have this control layer that provides this corridor where we can constrain the behavior of that new type of application architecture,” he says. “Because without that, you cannot unblock the deployment of AI applications.”

“That’s the part that we’re trying to answer — how do we unblock agents at scale?”

There is still plenty for Dome to prove. The company isn’t naming customers at this stage; McJannet says none of the enterprises it has worked with are yet willing to be identified publicly, though he says Dome has spent the past eight months talking to dozens of them.

Ultimately, McJannet believes the cloud era showed that new applications only become commonplace once enterprises have the controls to let them through. Dome is his attempt to solve that problem for agents.

“I think that’s the part that we’re trying to answer — how do we unblock agents at scale?”

The post After nine years as HashiCorp CEO, Dave McJannet now wants to “unblock” enterprise AI agents appeared first on The New Stack.

OpenAI says it hit a goal it set last fall, stating researchers are now using what the company calls an “automated research intern,” which is an agent that can handle well-defined tasks that would normally take a researcher several days.

The data shows coding-agent use climbing throughout 2026, and by mid-August its agents were logging 3.1 agent-workdays for every human workday across the research organization. The median researcher was spending more than $600 on inference per day at API prices, while those in the 90th percentile spent more than $7,000.

The median researcher was spending more than $600 on inference per day at API prices, while those in the 90th percentile spent more than $7,000.

Agent hours versus useful output

But everyone knows that an agent-workday and a human workday aren’t the same. The company converts the time agents spend working on tasks into standard eight-hour workdays. Because researchers can run several agents at once, the figure tells us how long the agents are working, but not necessarily what they’re completing.

For engineering teams, that leaves plenty of work on the human side, which means running more agents can increase the amount of work happening at once, but it can also increase the amount of work a human needs to keep track of.

OpenAI’s very specific definition of a research intern highlights that it must be able to complete well-defined research tasks that would take a skilled person several days, but a human is still in charge. The company’s next goal, an automated AI researcher, is one they hope to reach by March 2028.

The company’s next goal, an automated AI researcher, is one they hope to reach by March 2028.

Supervision becomes the constraint

Using a taxonomy from Epoch AI, OpenAI broke the agents’ work into six areas — Decide, Design, Build, Run, Analyze, and Communicate — and found activity increased across all six between January and August, although agents still did relatively little of the work involved in deciding what research to pursue.

Much of the work is practical, with agents writing research and infrastructure code, monitoring experiments, and providing enough technical support that OpenAI says attendance at debugging office hours has fallen, prompting one team to stop holding the sessions altogether.

And yet, more agent hours don’t automatically mean more useful research. OpenAI says code output and experiment counts are relatively easy to track, but neither shows how much progress those agents actually made. Compute also increased significantly as the number of experiments rose.

OpenAI used another model to judge how well agents performed on tasks of varying difficulty and found that, despite improving success rates between January and July, humans still had to step in on more than half of successful tasks that would have taken a person four to eight hours.

Security incidents limit Astra deployment

Once engineers can run several agents at once, with those agents launching subagents of their own, the challenge shifts to keeping up with what they produce — catching runs that go off track, reviewing code diffs, and deciding what is ready to ship or feed into a training run.

⁠Astra’s persistent-agent capabilities already let researchers hand off multi-day assignments⁠, which makes this supervisory strain worse, not better.

The company acknowledges that as agents take over more of the execution, the parts of research that are hardest to automate will consume more of an engineer’s time, putting a practical limit on how much agent output one person can realistically review.

On July 20, a series of outages caused by agents disrupted OpenAI’s research infrastructure badly enough that the company took its training container service offline and later brought it back with tighter restrictions.

Nearly a month later on August 7, OpenAI tightened access again after early evidence suggested Astra could reach the “Critical” cybersecurity threshold in its Preparedness Framework, restricting the model to higher-security research areas and adding safeguards that developers may already be encountering as unexpected API interruptions⁠

Workloads shift between models fast

Astra-class GPU allocation fell 59.2% the following week, but that compute didn’t sit idle for long. Researchers moved much of the work to other models, which saw GPU allocation rise 17.2% and made up for roughly 85% of the drop in Astra usage. Instead of reducing the amount of work being run, the restrictions pushed it to other models, showing how easily workloads can move when one part of the system is locked down.

OpenAI’s researchers are handing off larger jobs to agents, running more of them at once and launching more experiments, but whether that translates into faster research is harder to measure — and OpenAI is still figuring out how to price it.⁠

OpenAI’s researchers are handing off larger jobs to agents, running more of them at once and launching more experiments, but whether that translates into faster research is harder to measure — and OpenAI is still figuring out how to price it.⁠

The post AI agents are creating more work, not less — and OpenAI’s own numbers back it up appeared first on The New Stack.

Someone moves off the finance team at 9 a.m. on a Monday. Your sync runs nightly at 2 a.m. For seventeen hours, that person can still pull finance documents out of your retrieval index, and nothing in the system knows it is wrong. I am borrowing the example from Truto, but every team I talk to recognizes some version of it.

That is the version with a clock on it. The version people ask about in security review sounds different. The retrieval pilot works, the demo lands, the executive sponsor is happy, and then someone asks how you guarantee this thing will never summarize the CEO’s compensation review for an intern who asked an innocent question about salary bands.

Most teams do not have an answer. What they have is a filter.

I think the answer has to be structural. Permissions are not a filter you apply to context after you have assembled it. They are a property of how context gets assembled for a particular identity, because assembly is the last moment where refusing to include something still means the model never saw it.

Permissions are not a filter you apply to context after you have assembled it.

The major platform vendors in this race are building some version of the same step, and nobody has really settled on a name for it. I run a company, Modus, that builds in this lane, so weigh the argument accordingly. In our product, we call it context composition. For this piece, I will call it context assembly. It is where a system decides which pieces of enterprise knowledge to hand a model for a specific person, in a specific moment, for a specific question. Everything upstream is storage, and everything downstream is inference. Assembly is where identity either lives or doesn’t.

Announced is not the same as shipped

The reason to argue about this in September rather than in June is that platform vendors have stopped disagreeing about where the step goes, and the software most companies run has not caught up with them.

AWS made the most explicit version of the case in June, announcing AWS Context at its New York Summit, covered here at the time. The design decision underneath it is the interesting part. The graph is governed by the same permissions as the lake through Glue Data Catalog, SageMaker Unified Studio, and Lake Formation, and identity is checked again when someone asks. The people who would govern it are the ones already governing everything else, with the column-, row-, and cell-level policies that S3 object permissions alone can’t provide.

It is worth being precise about the tense, because the retelling has already blurred it. Every call is “designed to inherit the calling user’s IAM and Lake Formation permissions, so an agent can only see and traverse the relationships its identity is authorized to access.” Designed to. That is a roadmap language, and nearly three months later, AWS Context is still listed as coming soon, with no GA date, no regional list, and no pricing. Amazon Bedrock Managed Knowledge Base did go generally available that day, which is most of why the two get conflated.

Microsoft shipped identity-aware retrieval on June 16. AWS announced it on June 17, and you still cannot buy it.

The day before AWS announced Context, Microsoft’sWork IQ API became generally available. It runs in the context of the signed-in user, honors Microsoft 365 permissions, is billable through Copilot Credits, and an administrator can switch it on today. Two announcements one day apart, the same architectural position, and only one of them is something you can put in production.

Databricks reached the same slot from the other direction, extending Unity Catalog to the agent. However,h partners in that ecosystem note that the protection is anchored to the Databricks Runtime rather than to the data, so it stops applying when a BI tool or an MCP server reaches the same source directly.

Teams did not wait for any of this. They shipped the flat-index version while the identity-aware version stayed on the slide.

The direction is consistent, and so is the limit. Each of those controls is strongest inside the system that issues it. The interesting problem begins when an agent needs context that crosses several of those systems at once, and that is the job assembly has to solve.

The lake is not the business

Lake Formation enforces fine-grained permissions inside the lake it governs, and it does that well. Those permissions do not become the sharing rules in Salesforce, Slack, Google Drive, or Confluence.

AWS documents where its own boundaries sit. Its August guidance on propagating user authorization context through AgentCore walks through handing Salesforce a token scoped to the actual user, so Salesforce applies its own sharing rules. In AWS’s words, “the agent acts as an orchestrator, not a gatekeeper,” and “downstream services enforce authorization.”

That is a reasonable call. It is also an important product boundary. Lake Formation is not integrating with Salesforce, GitHub, Jira, Slack, Confluence, or Google Drive. Each of those decides who sees what on its own terms, or nobody does.

The most useful line is about the filter itself. In that same security post, AWS states plainly that “metadata filtering is application-layer enforcement. The bedrock:Retrieve API doesn’t expose metadata filter content as an IAM condition key.” I keep coming back to that sentence because it is a vendor calmly telling you where its guarantee ends and yours begins.

The same is true of your own stack. The tags on your chunks are not an identity boundary. They are a hint that your application code is trusted to honor.

What breaks when authorization arrives too late

The failure is structural, which is why I keep running into the same few versions of it.

I want to be careful here. “Filters are bad” is not the argument. The problem is ordering. A retrieval system can search a mixed index, retrieve opaque IDs, authorize them, and hydrate only the documents the person is allowed to read. That is a filter, and it is fine, because nothing unauthorized ever left the retrieval boundary.

The version I see more often runs the check after the documents have already been fetched. Once restricted text has been hydrated, reranked, summarized, or cached outside that boundary, authorization is chasing the problem instead of preventing it. AWS’s own guidance calls the broad-credential version of this a single point of failure because a prompt injection or a bug in the filtering logic can expose the whole dataset. And if it reached a model, the model has already read something the person was never entitled to retrieve, with any bug or injected instruction in that window free to act on it.

The defense most teams reach for first can make things worse. Jiale Liu, Jiahao Zhang, and Suhang Wang at Penn State red-teamed graph-based retrieval and found that summarization reduces leakage in untargeted attacks but can increase it in targeted attacks. My read of why is that summarizing preserves the salient detail, and the salient detail is usually the sensitive one. A separate 2026 preprint found cross-tenant leakage in pipelines that hand off from vector search to a graph, and eliminated it by re-checking authorization at every hop. Two individually secure components can still compose an insecure system when no one re-checks authorization at the transition between them.

Two individually secure components can still compose an insecure system when no one re-checks authorization at the transition between them.

The seventeen-hour window at the top of this piece is the same failure in slower motion. Direct shares, nested groups, and public links all change independently, which is why Google built Zanzibar as a relationship model rather than a list. A list of allowed users stamped on each chunk is a snapshot of a graph that moved without telling you.

None of this is fringe anymore. The OWASP Top 10 for LLM Applications moved sensitive information disclosure from sixth place to second in its 2025 revision and added LLM08, Vector and Embedding Weaknesses, which names the risk of context leaking between users who share a vector database and recommends a permission-aware store as the fix.

The enterprise-scale version of this is Copilot. In the first year of its enterprise rollout, a 2024 Gartner survey of 132 IT leaders found that oversharing led 40 percent to delay Microsoft 365 Copilot rollouts by 3 months or more. What makes that example useful is that Copilot is not the one doing the wrong thing. Microsoft checks the user’s permissions at query time, and its own documentation says results are trimmed to content the signed-in user has permission to access. Copilot surfaces what those people were already allowed to open.

A surprising amount of enterprise data stays private mainly because it is hard to find, and retrieval is very good at finding things.

The exposure was sitting there the whole time. A surprising amount of enterprise data stays private mainly because it is hard to find, and retrieval is very good at finding things.

Where identity has to arrive

So the lesson from Copilot is that resolving identity at assembly is necessary and not sufficient. Assembly inherits whatever the permission graph actually says. If the graph is wrong, stale, or too broad, the retrieval system will faithfully enforce the wrong answer. Homegrown retrieval can inherit the same problem, often with less governance tooling.

That does not weaken the case for assembly. It locates it. Assembly is not what makes your permissions correct. It is the last place where correct permissions still matter, because after that point the model has read the document.

I am not claiming to have invented this. AWS is arguing a version of it by governing the graph with the permissions the lake already has. OWASP got to the same place from the security side, and its recommended fix for LLM08 is a store that knows who is asking rather than a check that runs after the fact.

The part I would add comes from watching enterprise products make the jump from pilot to production. Teams can postpone many architectural decisions during a demo. They cannot postpone this one for very long. Eventually somebody asks who can see what, who guarantees it, how quickly a permission change propagates, and who owns the answer when three different systems disagree. That is often the moment when an impressive AI pilot turns into a security project, and it usually starts with something like an intern’s question.

So there are four questions I would put to any team building this.

1. Whether identity gets resolved at assembly or after retrieval.
2. How much of your context lives outside the lake, in chat and tickets and docs, where IAM does not reach.
3. What your worst-case staleness window looks like when someone changes teams.
4. And whether you can re-check authorization at every step along the way, or only once at the door.

If those answers are uncomfortable, that is useful. I have not had many of these conversations where they weren’t.

The post Permissions belong in the assembly context appeared first on The New Stack.

AI agents are multiplying as corporations adopt the technology in record numbers. Smarter underlying models, better tool use, and improved multi-agent collaboration have pushed agents to evolve beyond impressive demos into practical technology that companies marshal in production environments. But the job’s not finished. 

As companies deploy more agents, more often, and against longer tasks, the plumbing that provides their AI ephemera with the required information is buckling.

Here’s the problem: AI agents are sending waves of queries against company data, creating concurrency issues and exposing just how difficult it can be to ensure a company’s AI-legible information is fresh, served only when relevant, and quickly available.

Join the live conversation: On September 24 at 12 p.m. Eastern/9 a.m. Pacific, Whit Walters, Field CTO and Lead Analyst at GigaOm and author of Defeating the Integration Tax report, joins Bonnie Chase, Director of Product Marketing at Vespa.ai, to discuss what happens when retrieval architecture meets that workload.

And crucially, they will explore in this live conversation what changes when a team rebuilds it as a unified layer instead of a fragmented one.

Register for our free event on September 24

You might be asking yourself: How has this problem not been solved yet? Google famously handles tens of thousands of search queries every second; how difficult can it be to serve agents the information that they need when we’ve solved the human version of the same problem? It’s no small challenge, and it’s why retrieval engineering is a labor category you’ll hear more about in coming quarters.

So, why is the problem worse with AI? Agents don’t ask a single question. They may retrieve data, reason against it, and then go back for more context. That doesn’t sound too complicated, until we recall that companies often stitch multiple systems together to provide their agents with required information. In practice, that means fusing vector databases, ranking tools, and serving layers into a single hybrid retrieval system that serves ever more agentic queries.

Worse, when several agents ping the same cobbled-together architecture at once, relevance drift becomes a real issue. You might do all the work to get your company or team up and running with agents, only to see the effort fail because of stale data, generic answers, or even truncated results as retrieval plumbing stumbles.

Your AI agents can’t scale successfully if they get dumber the more agents you deploy. So join the conversation on September 24, where we’ll break down how you can solve your retrieval engineering woes.

What you’ll take away:

• Why agent workloads create a fundamentally different retrieval challenge than added concurrency alone
• The specific failure modes at agent scale — latency stacking, stale context, relevance drift
• Why fragmented retrieval stacks amplify those failures
• What a unified retrieval architecture looks like in practice

The post Want to scale AI agents without breaking anything? Retrieval engineering is the answer. appeared first on The New Stack.

A 438-billion-parameter reasoning model isn’t an obvious choice when speed is a priority. Multiverse Computing is betting that compression can make it one. 

On Wednesday, the Spanish company launched Quasar 438B, its first large-scale model and a system designed specifically for coding and enterprise agents. Quasar achieves a score of 43 on Artificial Analysis’ Intelligence Index and 69.3 on Terminal-Bench v2.1, while Artificial Analysis currently records its output speed at approximately 183 tokens per second. Multiverse is positioning Quasar as the European model with the highest score on the Intelligence Index, outperforming Mistral Medium 3.5 (30) and NVIDIA Nemotron 3 Ultra (38).

Multiverse is betting that a 438B-parameter model can be fast and inexpensive enough for agents that repeatedly reason, call tools, and check the results.

That puts Quasar in an interesting middle ground. It doesn’t match the strongest models on coding performance, but Multiverse is betting that a 438B-parameter model can be fast and inexpensive enough for agents that repeatedly reason, call tools, and check the results.

The model features a context window of one million tokens, is available in English and Spanish, and can be accessed via the Multiverse CompactifAI API.

Compression claims, missing details 

Multiverse built CompactifAI to shrink large AI models so they need less memory and compute to run. The company says it can reduce model size by 80% to 95% with only a small loss in accuracy, but it hasn’t disclosed how much Quasar was compressed or which model it started with.

In July, Multiverse announced a $570 million Series C to expand its library of compressed models and commercialize the technology. Quasar is the biggest test of that approach so far.

Multiverse hasn’t said what hardware is required to run Quasar or how much the compression reduces its memory and compute needs. That matters for agents, which may repeatedly call the model and other tools before finishing a task.

Multiverse hasn’t said what hardware it takes to run Quasar or how much the compression cuts its memory and compute needs.

Coding benchmarks show tradeoffs 

In Multiverse’s comparison, Quasar’s Terminal-Bench v2.1 score of 69.3 places it ahead of Mistral Medium 3.5 but still well behind the best frontier systems, while Claude Opus 5 achieves the highest score on that benchmark at 89.1.

Multiverse is pitching Quasar for software engineering, technical copilots, research, and workflow automation. Its 1-million-token context window gives agents room to work with large codebases and hold onto information as a task progresses, although processing more context also requires more compute. That can be especially important in coding, where code that passes every test can still trip up the next AI agent if it loses track of what came before.

Agent latency beyond throughput 

Artificial Analysis found that Quasar starts responding in about 1.1 seconds and can produce a 500-token response, including reasoning, in around 15.3 seconds. Those numbers are fast, but an agent also has to wait for tools, process growing context, and make repeated model calls over the course of a task. The agent tooling layer itself is still catching up to what these models need.

Those numbers are fast, but an agent also has to wait for tools, process growing context, and make repeated model calls over the course of a task.

Proprietary model, open questions 

Quasar is proprietary and only available through Multiverse’s API, so developers can’t inspect the weights or run it on their own hardware. For now, that also makes it difficult to know whether the speed Multiverse is reporting will carry over to everyday agent use.

Quasar also arrives as European AI companies are building more of their own model and compute infrastructure instead of relying on U.S. hyperscalers. Multiverse is taking a different route, using compression to make a 400B-plus model cheaper and faster to run. The next step is to see how that holds up against benchmarks.

The post Multiverse says its 438B model is fast enough for AI agents. The benchmarks tell a more complicated story. appeared first on The New Stack.

Perplexity wants its AI agents to use more of the computing power already sitting inside a Mac. The company launched Hybrid Compute on Tuesday, a new feature that lets Perplexity Computer move parts of the same task between powerful cloud models and smaller models running directly on Apple silicon.

The timing is significant. Hybrid Compute arrives on the first day of John Ternus’ tenure as Apple’s CEO. Ternus, who previously led Apple’s hardware engineering organization and played a key role in the company’s transition from Intel processors to its own Apple silicon, succeeds Tim Cook after 15 years as CEO. Perplexity is now betting that same hardware can become part of the infrastructure behind autonomous AI agents.

The task starts in the cloud. When a step involves sensitive information, Computer can move that part of the job to a model running on the Mac without having to start over.

When a step involves sensitive information, Computer can move that part of the job to a model running on the Mac without starting over.

Privacy Gate checks first

A Perplexity-trained Privacy Gate runs locally on the Mac and looks for sensitive information such as names, addresses, account numbers, and secrets. When it flags something, the user can decide whether that part of the task should stay on the device.

Users can review what the system wants to keep local before work begins and ensure it hasn’t missed anything they don’t want sent to the cloud. They can also choose which model handles the local work.

At launch, users can choose between Gemma E4B, Qwen3.6 35B-A3B, and a version of Qwen3.6 35B that Perplexity post-trained itself, with more models planned for later. Perplexity also handles the installation through the desktop app, so users don’t have to open a terminal or set up the model themselves.

Local tokens cost nothing

Once a task is running, the app displays local CPU, GPU, and memory utilization, as well as the number of tokens consumed. Users aren’t charged for tokens generated by models running locally on their Mac. That gives Perplexity another reason to move work onto the device beyond privacy, since running models locally can also cut inference costs. (The question of who bears inference costs and when is becoming a competitive issue across the industry.)

There is a compromise. Perplexity uses its most capable models in the cloud, while smaller models handle local work, so keeping more of a task on the Mac can improve privacy and cut costs at the expense of some capability. Hybrid Compute leaves that choice to the user.

Perplexity uses its most capable models in the cloud, while smaller models handle local work, so keeping more of a task on the Mac can improve privacy and cut costs at the expense of some capability.

Context crosses the boundary

The trickier bit is what happens to context when part of a task moves onto the Mac. The cloud model still needs to know enough about what happened locally to continue the job, without getting access to the private information that was supposed to stay there.

Perplexity says Computer can move a step from the cloud to a local model without restarting the task or losing context, and ultimately combine the cloud and local work into a single result. It does not detail in its announcement exactly what context is passed between those environments or how information produced by the local subagent is filtered before it returns to the broader workflow.

The local subagent can work with private files and data and take actions on the Mac. Users can also start a task on an iPhone and hand off local work to their Mac without having to start over.

For enterprise customers, Perplexity adds company-wide rules for what stays local and a record of what leaves each device, bringing the same governance questions facing AI agents down to the device level.

DGX Spark starts local

Hybrid Compute reverses the approach Perplexity introduced for Nvidia’s DGX Spark last week. DGX Spark starts locally and reaches out to frontier cloud models only with permission, while the Mac version starts in the cloud and moves work onto the device when needed.

In both cases, the agent harness decides where each part of the job runs. That orchestration — deciding which tools and context an agent actually needs — is becoming an increasingly difficult engineering problem as agents gain access to more systems and data.

Apple silicon becomes part of the agent stack

Running this much of an agent locally still requires a fairly powerful Mac. Perplexity recommends at least 32GB of unified memory, Apple silicon, and macOS 15. The feature is available to Pro and Max subscribers as well as enterprise customers.

Those requirements show the limits of local AI at the moment. Smaller models can run on plenty of Macs, but giving an agent enough compute to handle meaningful work still requires relatively high-end hardware.

That will likely change as Macs get better at running larger models. For Ternus, the bigger question is how much of the AI work now happening in the cloud will eventually move onto the machines Apple sells.

For Ternus, the bigger question is how much of the AI work now happening in the cloud will eventually move onto the machines Apple sells.

The post Your Mac is now part of Perplexity’s AI infrastructure appeared first on The New Stack.

Every application has always needed a place to keep its state. That is not new. What is new is who creates that place, how many get created, and how long they have to survive after everyone has stopped looking.

“Separating durable state from ephemeral compute is the new requirement, and the idle cost trap is only half of it.”

Consider the agents built by Kimi, the AI platform from Moonshot AI. A non-technical user describes an application in plain language. An agent builds the frontend, backend, and database, then deploys them in minutes. The user never thinks about infrastructure. They asked for a tool and received a running one.

Then the agent does something the previous generation of coding assistants never did. It does not hand over code and walk away. It keeps the application running and comes back to maintain it. That last word is where the old assumptions break. Build once and hand off, and state is almost an afterthought. 

Build, deploy, and maintain across tens of millions of applications, most idle most of the time, and persistence stops being a database feature and becomes the central economic problem of the whole system.

I have written before about what happens to a database when its user is an agent rather than a human. Kimi is the clearest production example I have seen, and it surfaces a requirement that the industry has not really been asked to meet: keep state alive cheaply in two very different places at once. The problem is not performance. It is the economics of persistence.

The number that changes the question

When you serve thousands of tenants, the questions are familiar. How fast are queries? How do we fail over? How do we back up? The industry has good answers for all of them. When you serve tens of millions, most created by agents and most idle most of the time, a different question moves to the front, and it is the one nobody planned for: what does it cost to keep something alive that nobody is using right now?

Ask it once, and you are talking about idle databases. Ask it again, one layer up, and you are talking about the agent’s own half-finished work sitting between maintenance sessions. Same question, two surfaces. At a thousand you absorb it. At ten thousand it stings. At tens of millions it is fatal.

“In a human-driven product, what you provision and what people use track each other. In an agent-driven product, they diverge wildly.”

This is the shift agents force. In a human-driven product, what you provision and what people use track each other. In an agent-driven product, they diverge wildly, because agents create far faster and more casually than humans ever did, and most of what they create goes idle immediately.

One principle, two places

There is a single move underneath everything that follows: separate durable state from ephemeral compute. State is the part you must never lose, so it belongs on a shared, cheap, effectively infinite foundation. 

I have argued before that object storage is becoming the new network layer of the data stack, and this is what that looks like, built for agents: durable data at rest costs almost nothing. Compute is the part you should be able to throw away, summoned when work arrives and released when it stops.

Couple the two and you pay to keep compute running for state that is doing nothing. Decouple them, and your cost stops tracking how much you have created and starts tracking how much work is happening. 

Every hard number in the Kimi deployment comes from getting this separation right in the two places an agent-scale product needs it: the databases the agent provisions for its users and the workspace the agent lives in.

The first place: the idle tenant

The instinct most teams start with is the one that has worked for a decade: give each tenant its own database instance. It is clean and easy to reason about. Kimi’s earlier shape was close to this, with single-instance PostgreSQL behind the product. It works beautifully until the tenant count climbs into the tens of thousands, and then the economics invert. The reason is structural. 

A per-tenant instance couples the logical experience of an isolated database to the physical reality of dedicated, always-on compute. The tenant needs the first. The business cannot afford the second at scale. Cost grows with tenants created, not tenants active, and in an agent-driven product those two numbers are worlds apart.

“The agent experiences a dedicated, isolated database, while a virtual layer beneath provides each one its own namespace. The agent does not need a dedicated instance. It needs the experience of one.”

I have started calling this the idle cost trap, because teams walk into it without seeing it, and by the time they feel it, the architecture that created it is load-bearing. The way out is the principle. The agent experiences a dedicated, isolated database, while a virtual layer beneath provides each one its own namespace and isolation guarantees over a shared substrate, with object storage holding durable data and a routing layer sending requests where they belong. The agent does not need a dedicated instance. It needs the experience of one.

There is a catch worth being honest about. If compute is summoned on demand, then when a request arrives for an idle tenant something has to spin up, and if that is slow, you have traded a cost problem for a latency problem. 

Kimi provisions a database in about a second, from a warm pool of pre-initialized resources kept ready and replenished behind each claim, so database setup drops out of the delivery pipeline entirely. That number is not a benchmark to brag about. It is the evidence that ephemeral compute over shared, persistent storage can stand in for the always-on model, because it appears fast enough to feel always-on.

The second place: the agent’s own workspace

The maintenance half of the lifecycle drags the same problem into a place most teams never look: the agent’s own working environment. An agent that builds once can treat its workspace as disposable: spin up a sandbox, write code, ship, discard. An agent that maintains cannot work that way. 

It has to return days or weeks later and pick up where it left off: source code, Git history, checkpoints, the record of what it was in the middle of doing. If that context is lost, the agent does not resume work. It is reconstructing it.

And execution environments are ephemeral on purpose, because keeping millions alive between sessions is the idle cost trap in a different costume. So the environment gets torn down. If the work goes with it, every maintenance session begins by rebuilding state the agent already had: wasted compute, wasted tokens, and a user watching an agent relearn its own project.

The fix is the same separation, one layer up. Kimi uses a persistent filesystem that keeps development state alive independently of the compute that produced it. Source code, Git history, checkpoints, and task progress persist after the environment is destroyed, so the agent resumes rather than starting over. Compute is ephemeral and cheap to discard. State is durable and cheap to keep. Same trade as the idle tenant, seen from the builder’s side.

The database choice becomes a quality input

There is a further effect that surprised me, because it shows up in the quality of what the agent builds. Every infrastructure decision is a chance to get something wrong. If each task forces the agent to reason from scratch about which database to use and how to configure it, it is improvising on every run, and improvisation is where errors enter. 

When the stack is unified, it applies known-good patterns instead. Kimi saw code generation success rates improve from standardizing on a unified data layer, and that is the mechanism: fewer places for the output to go wrong.

“The choice of database is no longer just an infrastructure decision. It is a quality input to the agent’s work.”

This is a version of an argument I keep returning to. Agents need guardrails, and the most effective ones are built into the infrastructure rather than bolted on as instructions. A database that behaves consistently every time is a guardrail. The choice of database is no longer just an infrastructure decision. It is a quality input to the agent’s work.

Four properties that have never been requried together

Step back from Kimi and the shape of the requirement is clear. An agent-scale product needs four things from its data layer at once:

• Tenant isolation: every agent-created database must be logically separate, so millions of tenants never bleed into one another.
• Instant provisioning: a new tenant has to exist in about a second, because an agent will not wait and neither will the user behind it.
• Cost elasticity: an idle tenant, and an idle workspace, has to cost almost nothing, because most things are idle most of the time.
• Persistent state independent of compute: the tenant’s data and the agent’s development state alike must survive the ephemeral environments that produced them.

Each has been solved before in isolation. Databases have offered isolation for decades. Serverless systems provision quickly. Object storage is cheap at rest. What is new is the demand for all four at once, at a scale where any one failing breaks the product. That is the real infrastructure competition of the agent era, and it is not the one the industry is used to having. It is not about who has the fastest single query. It is about who can hold all four together at tens of millions of tenants without one collapsing the others.

The pattern is bigger than one product

I do not think Kimi is a special case. It is an early, unusually clear instance of a pattern that will repeat across every product where agents build for non-technical users at scale. The shape is always the same. One agent, one workspace, one database, repeated millions of times. Each instance feels independent. 

The infrastructure underneath is shared, and it works only because a virtual layer separates the logical experience of isolation from the physical cost of compute, and because durable state is allowed to outlive the compute that produced it.

Kimi is the version where the agent hands a finished application to an end user and then stays on to maintain it, which is harder in two ways at once. The tenant does not go away when the run ends; it persists, idle, waiting, multiplied by tens of millions. And the agent does not go away either, so its own working state has to survive every gap between sessions. Persistence is no longer a property of one component. It is the property the whole system is organized around.

“The database conversation in the agent era is no longer really about speed. Get it wrong, and no amount of model quality will save the margins.”

Teams building in this direction will hit the idle cost trap on both surfaces, whether they plan for it or not. The only choice is whether they see it coming. Design for all four from the start and you scale past the point where the per-tenant-instance model collapses. Do not, and you hit that ceiling at tens of thousands of tenants, exactly where Kimi’s earlier architecture did, and sooner than you expect, because agents fill a tenant table faster than any human-driven product ever has.

The database conversation in the agent era is no longer really about speed. It is about whether the economics of persistence can survive the scale that agents create in both the database and the workspace. Get that right early, and the rest of the product has room to grow. Get it wrong, and no amount of model quality will save the margins.

If you are building a product where agents provision infrastructure for end users and then stay on to maintain it, four things make that economically viable: isolation, instant provisioning, near-zero idle cost, and durable state on a shared substrate. That is what we have built into TiDB’s serverless and agent workloads. It is the pattern these teams keep arriving at from different directions.

The post When agents build, deploy, and maintain, persistence becomes the hard problem appeared first on The New Stack.

Connecting an AI agent to a tool is relatively straightforward. Things get more complicated once an organization has hundreds or thousands of resources spread across different clouds and platforms. Agentic Resource Discovery, or ARD, is designed to help agents navigate all of that.

AWS highlighted the open specification in its August 31 Weekly Roundup after taking a deeper technical look at it a week earlier, describing the idea as “DNS, but for agents.” Instead of telling an agent where to find everything in advance, ARD lets it search across different registries for what it needs.

And despite AWS highlighting the project, ARD isn’t an AWS technology. It was authored by Junjie Bu of Google, R.V. Guha of Microsoft, and Shaun Smith of Hugging Face, and released under the Apache 2.0 license. Engineers from several other companies have helped shape the project, including Cisco, Databricks, GitHub, GoDaddy, Nvidia, Salesforce, ServiceNow, and Snowflake.

AWS’s role, at least so far, has been to provide feedback on the specification and explore how it could work with its own Agent Registry. The goal is to make the existing registries work together.

Instead of telling an agent where to find everything ahead of time, ARD lets it search across different registries for what it needs.

MCP skips the discovery step

The Model Context Protocol has become a common way for AI applications to connect to external tools and data, but it assumes the client already knows which server it wants to use. That becomes a problem as companies spread their infrastructure across clouds, SaaS platforms, and internal systems.

ARD helps an agent find a resource before it tries to use it. The specification uses the term “agentic resource” to refer to anything an AI client can connect to, from an MCP server to other external capabilities. An ARD-compatible service keeps track of what’s available, rather than requiring developers to set up every connection in advance.

The Model Context Protocol has become a common way for AI applications to connect to external tools and data, but it assumes the client already knows which server it wants to use.

Federation without forced migration

Companies can keep their own catalogs and policies while routing searches to other ARD-compatible services. An enterprise, for example, could keep internal resources private while searching approved external catalogs when needed. AWS calls this “describe once, discover everywhere.”

The current v0.91 proposal, dated August 26, uses JSON-LD and a REST interface. Its required POST /search endpoint searches by task, while optional endpoints allow clients to browse available resources.

Each discovery service can set its own rules for what it returns and which sources it trusts. This is also where AWS’s DNS comparison falls short. A domain name points to a specific location, while an ARD search could turn up several options that all appear capable of doing the job.

Route 53 engineers shaped ARD

The DNS comparison has some history behind it. Two of the three authors of AWS’s August 24 ARD post work closely with Route 53. Principal software engineer Jeffrey Damick focuses on DNS and networking technologies. At the same time, Bhargav Talluri leads product management for Route 53 and for agent identity and discovery in AWS Agent Registry.The

Agent Registry already provides AWS customers with a central view of their resources. Adding ARD could bring resources running elsewhere into that view without requiring companies to register everything with AWS.

Adding ARD could bring resources running elsewhere into that view without requiring companies to register everything with AWS.

ARD’s governance is still being worked out, with board terms and membership among the details yet to be settled. The group has also discussed eventually moving the project to a neutral organization such as the W3C or an AI foundation.

Finding tools before using them

AWS is already exploring how it could connect with Agent Registry and find resources outside its own catalog.

There may not be much time to settle on a common approach, since connecting all these directories will only get harder once companies have built their own discovery systems.

The post MCP was supposed to solve the agent tooling problem. It missed a step. appeared first on The New Stack.

AI coding company Replit is throwing its weight behind the model-routing trend by making its “intelligent model routing” system the default across every account.

The system automatically selects the underlying model to handle a task as it evolves, with Replit weighing quality, speed, and cost in its routing decisions.

The company says the feature, dubbed Auto mode, will become the default option for all users, though Core and Pro subscribers can still override it and manually select models when they want more control.

Model-routing momentum

The announcement comes hot on the heels of a flurry of activity in the model-routing realm. Earlier in August, Stripe agreed to acquire model gateway platform OpenRouter for a reported $8 billion. On the very same day, Ramp launched Router.com, which routes requests to the lowest-cost model that meets a specified performance bar.

Before all that, in July, SpaceX-owned Cursor launched its own router, which automatically selects models for coding requests and claims to deliver comparable performance at a substantially lower cost. Meanwhile, Meta is reportedly developing an internal router called “Switchboard” that scores coding tasks by difficulty and sends simpler jobs to cheaper models.

“Across one model family, per-token rates can span orders of magnitude. At the same time, the intelligence of cheaper, smaller models is now much closer to their larger frontier counterparts, providing us a lot of room for cost optimizations.”

Michele Catasta, president and head of AI at Replit, says that one reason for the wider push into routing is simple economics — the growing gap between what models cost and the level of capability developers actually need for a given task.

“Across one model family, per-token rates can span orders of magnitude,” Catasta tells The New Stack. “At the same time, the intelligence of cheaper, smaller models is now much closer to their larger frontier counterparts, providing us a lot of room for cost optimizations.”

Replit, for its part, has been moving in this direction for some time. Catasta says that the company has spent recent months experimenting with early versions of Auto mode, subagent routing, and multiple iterations.

“Like any pivotal launch, we thoroughly tested Intelligent Model Routing in beta for a long period of time before we decided to release it in public,” he says. “The most important learning is understanding from first principles the failure modes of every experiment, so we could keep hill climbing on the final system that we just shipped.”

Enter Auto mode

The foundation of that work surfaced last week when Replit introduced Free Mode, a lower-cost Agent mode that doesn’t consume usage credits and uses Auto to choose the model on theuser’ss behalf, subject to usage limits.

Now, that same Auto routing approach is being pushed more broadly across Replit. The company says intelligent model routing will become the default across every account, with all users starting in Free Mode and Replit deciding which model is best suited to the task.

Free Mode, it’s worth noting, isn’t “free” in the sense of unlimited usage. When it launched, Replit made it available to Core and Pro subscribers without consuming their usage credits, but imposed limits that reset every five hours, with higher allowances for Pro users. In Free Mode, users cannot manually select a model.

Core and Pro subscribers can, however, switch to Replit’s Power or Max modes, where they can turn off Auto and choose a model themselves. Replit may also suggest moving a task into one of those higher-powered modes when it determines that more capability is required, though those modes can incur usage costs.

For Enterprise customers, meanwhile, administrators can restrict Auto to an approved set of models for each workspace, allowing Replit to continue routing tasks automatically while keeping model choice within company policy.

The agent advantage

Even before SpaceX agreed to pay a cool $60 billion to acquire Cursor, the AI coding startup had long been investing in its own coding models, including its Composer family. More recently, under the auspices of SpaceX, Cursor has been developing more cutting-edge models, too.

Replit, by contrast, isn’t making ownership of the underlying model layer central to its pitch. Instead, it’s betting that controlling the agent and the systems around it gives Replit enough insight to make better model-selection decisions on the fly.

“Replit has owned, from the start, both the agent harness and the infrastructure surrounding models which in turn allows us to train sophisticated model routers.”

“Replit has owned, from the start, both the agent harness and the infrastructure surrounding models which in turn allows us to train sophisticated model routers,” Catasta said. “Only in this way can we always offer useful intelligence to our users at the most competitive price point.”

That becomes particularly relevant as an Agent task unfolds, with Replit noting that its system can change which model it uses as the task develops, seeking a better trade-off between capability and cost at different points in the process. But for Catasta, that kind of dynamic routing is still only one part of a much broader research problem around how agents should use models.

“Model routing is still in its early development phase, and we expect further research will move the needle on serving the best intelligence when customers most need it,” Catasta explains. “Routing is but one piece of the puzzle that is tightly integrated to many other aspects of our harness research.”

“No third-party router company could reproduce the same results for our own agent.”

Replit also argues that seeing how people use its own Agent gives it an advantage that a standalone routing provider would struggle to reproduce. Catasta says a router has to infer the nature, difficulty, scope, and intent of a request, with Replit able to train against proprietary usage data and observe those signals across its user base.

“No third-party router company could reproduce the same results for our own agent,” he says.

The post Replit’s new default: Auto mode picks the best model for each task appeared first on The New Stack.

OpenAI’s unreleased foundation model, codenamed Astra, is already working directly inside the company’s internal codebase, taking on experimental work that previously required as much as a week from a human researcher.

As part of a series of interviews with Time, OpenAI chief scientist Jakub Pachocki said Astra can take an idea for an experiment, turn it into code, run it, and return the results. A step beyond asking AI to fix a bug or write a function, OpenAI is essentially testing what happens when you hand the agent the experiment itself and let it figure out the steps in between.

For developers, Astra offers a glimpse into the future of coding agents. But giving an agent that much freedom creates another problem. Astra may already be powerful enough to trigger OpenAI’s highest-level cybersecurity safeguards.

OpenAI is essentially testing what happens when you hand the agent the experiment itself and let it figure out the steps in between.

Persistent agents change everything

OpenAI CEO Sam Altman described what the company is building as “persistent agents,” or systems that can keep working without needing a person to prompt them through every step.

Coding agents can already dig through a repository, change files, run tests, and try again when something breaks. Persistent agents are meant to keep going without a developer guiding them through each step.

That also changes what developers need from the tools around the agent. An IDE gives it somewhere to work, but a long-running agent needs infrastructure that can keep it running safely without constant oversight.

Multi-agent coordination at scale

In one demonstration witnessed by Time, 16 Astra agents worked together on a research-level math problem, splitting it into smaller pieces and then bringing their work together into a proposed solution.

For developers, it’s not hard to imagine that same setup applied to a large software project, with different agents working on different pieces at the same time. OpenAI is already experimenting with that kind of coordination, along with agents that can stay on a job for much longer.

Running several agents at once also complicates the infrastructure behind them. Developers need a way to keep the whole operation under control. Giving agents that much freedom, though, has created another problem for OpenAI: keeping them under control.

When agents escape containment

OpenAI said this month that preliminary evaluations indicate Astra may have reached the “Critical” cybersecurity capability threshold in the company’s Preparedness Framework, a finding the company disclosed alongside a pause on some frontier workloads.

Under the company’s framework, hitting that threshold brings stricter safeguards for how the model can be used.

OpenAI had already seen what could go wrong with agents that have access to tools. During a cybersecurity test, one of its internal AI agents escaped its sandbox and accessed Hugging Face systems without authorization. Astra was not the model involved, but the incident led OpenAI to pause some frontier-model research workloads while it tightened the infrastructure used to run them.

And OpenAI isn’t the only company running into this problem. Google’s AI coding agent recently broke out of the boundaries of its IDE. The circumstances were different, and the incident was less serious, but both point to the same problem that happens when giving an agent more freedom to act; you also have to make sure it stays where you put it.

Astra is now running under OpenAI’s strictest security controls. Some training and evaluation workloads have resumed, but OpenAI says a “significant number” are still paused while it upgrades the infrastructure behind them.

Astra is now running under OpenAI’s strictest security controls.

Monitoring costs real compute

The company also says it is monitoring Astra more closely when it uses tools, watching for behavior that could signal it is going beyond what it’s allowed to do. OpenAI estimates that monitoring adds about 20% to the inference compute for those workloads.

Time reports that OpenAI still plans to release Astra, although there’s no launch date yet. Until then, OpenAI is working through the same challenge developers could eventually face as agents work for longer stretches without human supervision and begin coordinating with other agents.

As agents take on more work, the IDE is only part of the picture. Developers also need a way to see what those agents are doing and keep them from going where they shouldn’t. OpenAI is already seeing the cost, with monitoring alone adding about 20% to Astra’s inference compute.

That kind of overhead could become another cost developers have to account for, along with the governance around autonomous agents that sets boundaries on what they can do.

OpenAI is already seeing what that costs, with monitoring alone adding about 20% to Astra’s inference compute.

The post OpenAI’s Astra can do a researcher’s week of work. That’s the problem. appeared first on The New Stack.

Perplexity shipped Portable Computer this week, the local-first version of its Computer agent running on an Nvidia DGX Spark workstation, and the bill is steep: A DGX Spark starts at $4,700, and even an aging 24GB RTX 3090 sells well above $1,500.

The architectural choice beneath the surface deserves as much attention as the price tag: Most platforms building agents address reliability with greater intelligence. That typically includes a larger orchestrator model, a planner model, or a critic model reviewing the work.

Probabilistic reasoning proposes the next action, and deterministic software decides whether to execute it.

Perplexity separated the two jobs instead of stacking them. Probabilistic reasoning proposes the next action, and deterministic software decides whether to execute it.

The loop controller is code, and the decisions are still a model

Perplexity uses the term “orchestrator” to refer to the runtime controller rather than the planning model. That controller assembles context, enforces policy, and executes approved tool calls inside an OS-level sandbox. The company says it is deterministic code, not yet another model. The local model proposes the next action, including which tool to call and when to ask a cloud advisor for help. The split resembles a control plane architecture, where reasoning suggests, and inspectable software retains authority. Nate Kupp, Perplexity’s vice president of Computer Enterprise and Infrastructure, tells The New Stack that the harness accounted for most of the engineering work.

Same weights, better scores

Perplexity kept the base model and the silicon constant, putting Qwen3.8-27B on the same DGX Spark across three agent stacks. On its internal Local Knowledge Work Bench, a held-out set of 53 tasks, the company reported 82.6% for Computer. Pi scored 77.6% and Hermes 74%.

On ParseBench-100, a subset covering charts, layouts, tables, and formatting, the gap widened considerably. Perplexity reported 65.1% for Computer, while Hermes reported 34.6% and Pi reported 13.9%.

Because the base weights were constant, the gap measures the system around the model rather than a better model. That does not mean weights stopped mattering. Perplexity post-trained Qwen into PPLX 27B and reported 85.4%, above its own base-model number. Harness engineering and post-training are significant factors of this approach.

The word harness also covers a lot of ground, including prompts, tool schemas, context management, verification hooks, and document processing. Orchestration code is one part of that surface.

The security boundary lives outside the model

The sandbox is the boundary, not the determinism. Perplexity says the sandbox restricts processes, filesystem paths, and network access. If the sandbox is unavailable, the harness disables itself before making any tool call. Deterministic code is valuable here because it makes policy easy to inspect and helps the system fail safely. Deterministic code can still ship a vulnerability or faithfully execute a permitted mistake.

The key distinction is not between model orchestration and code orchestration. It is whether permission is enforced by something other than asking a model in plain English to comply.

The key distinction is not between model orchestration and code orchestration. It is whether permission is enforced by something other than asking a model in plain English to comply.

The same discipline shapes how context gets spent. Perplexity reports that Qwen3.8-27B advertises a 260,000-token context window but begins to struggle beyond 100,000 tokens. The harness therefore keeps the core prompt and toolset small and loads skills on demand. Commonly used connectors became command-line tools rather than full Model Context Protocol definitions sitting permanently in context.

Deterministic execution cannot rescue reasoning that exceeds the local model. On Terminal Bench 2.1, Perplexity reported 59.6% running locally. Letting the local agent consult Claude Opus 5 raised it to 73.0%, compared with 82.4% when Opus 5 worked alone. All of this remains vendor-reported evidence on a bench that the company has yet to open-source.

For enterprises evaluating local agents, the component to scrutinize is the layer that grants and denies authority, because that is where the platform’s engineering is most evident.

The post Perplexity just separated reasoning from authority. Here’s why it matters for enterprises. appeared first on The New Stack.

Agentic reinforcement learning has been suffering from a disconnect, an uncoupling, and a misarticulation. The polarity arises from how a training engine handles resource management, compared with how a post-training live production harness does.

Microsoft wants to ensure the production harness is engaged from the start to oversee infrastructure services and agent interactions, during both initial training and subsequent reinforcement learning actions.

Redmond’s Microsoft Research division first introduced the Agent Lightning framework in August 2025 as an infrastructure concept for agent optimization to address structural challenges in post-training LLM-based agents as they enter reinforcement learning processes. Microsoft subsequently launched the Agent Lightning v1.0 release with a commit tagged on GitHub on August 16. 

Using Agent Lightning v1.0 on what Microsoft has called “modest compute” 6K training examples, reinforcement learning improves Qwen3.5-9B on OpenAI’s SWE-bench Verified benchmark from 41.8% to 56.4%, an absolute 14.6-point gain.

“Using Agent Lightning v1.0 on ‘modest compute’ 6K training examples, reinforcement learning improves Qwen3.5-9B on OpenAI’s SWE-bench Verified benchmark from 41.8% to 56.4%, an absolute 14.6-point gain.”

Who owns the interaction loop?

In traditional agentic reinforcement learning, the training engine owns the interaction loop, i.e., steps including observing the environment, selecting an action based on the policy, executing the action, receiving a numerical reward, storing and updating the policy… and so on. 

In harnessed agentic reinforcement learning, the harness owns context construction, tool execution, and the agent–environment loop. The training engine observes only a sequence of LLM request–response pairs across a service boundary. Therefore, developers do not need to reimplement the agent loop within the training environment.

According to a collected group of software engineers at Microsoft, because an AI model harness owns the loop that governs infrastructure access and operations procedures during harnessed agentic reinforcement learning, it “introduces challenges” including retokenization (re-segmenting text into new tokens during active training), sample merging, advantage calculation, loss normalization, and training backend scheduling.

All of which, if not addressed, can result in ineffective or unstable training.

How Agent Lightning v1.0 turns the tables

“[In Agent Lightning v1.0] the harness, rather than the trainer, owns context construction, tool execution, and the agent–environment interaction loop, while the training system observes and optimizes the resulting model calls across a service boundary. This formulation preserves the harness’s deployment-time context policy, tool protocols, and execution semantics without requiring its agent loop to be reimplemented inside the RL framework,” explained the Redmond team.

For coding agents, the team has said that it finds existing reinforcement learning frameworks provide limited support, including a lack of data and complete training scripts, and a reliance on large-scale computational resources. To address this gap, with Agent Lightning v1.0, Microsoft provides a “complete data-cleaning pipeline” and reproducible training scripts built on open-source datasets and models.

Is this the end of the training time liability?

For users inside Microsoft environments, this might feel like good news. Machine learning specialists and platform engineers may have a good harness or a good set of reinforcement learning tools; they would rarely have both.

Rather than having to hard-code retokenization, advantage calculation, and reward shaping every time they wanted to train an agent to call external services in reinforcement learning procedures, they can keep their existing agent architecture as an asset, rather than treating it as a training-time liability.

Nebraska-based software engineering researcher Md Rashedul “Rashed” Hasan tells The New Stack that training on the exact production harness matters, but not only for its efficiency and benchmark gains. 

Training through the real harness keeps semantics intact

“It reduces train–serve mismatch,” Hasan says. “If you train inside a simplified trainer loop and deploy inside a different harness, tool protocols, context policy, and recovery behavior can all drift. Training through the real harness keeps those semantics intact, so gains are more likely to transfer to production behavior, not only to a lab environment.”

Hasan thinks that Microsoft has clearly named the paradigm, kept the core framework small, and shipped what he defines as a “concrete coding-agent pipeline” with open data and scripts. 

“In terms of who this will appeal to, it’s application and platform engineers who already have a production agent harness (such as a coding assistant or support-triage agent), and want to improve the underlying model with reinforcement learning without rewriting deployment logic to fit a training framework. Reinforcement learning and machine learning platform teams would also use it when they need a thin, reproducible testbed for harnessed agentic reinforcement learning,” Hasan clarifies.

He agrees that the SWE-bench Verified lift with only 6K examples is “useful proof” that harnessed reinforcement learning can move a hard-coding benchmark without forcing teams to reimplement their agent stack within the trainer.

“Environment setup for coding agents, reward design, evaluation fidelity, and the retokenization, sample-merging, advantage, and loss-normalization details are still easy to get wrong. Adoption will also depend on whether teams can integrate this proxy pattern into existing orchestration, observability, and safety controls,” cautions Hasan.

“Environment setup for coding agents, reward design, evaluation fidelity, and the retokenization, sample-merging, advantage, loss-normalization details are still easy to get wrong. Adoption will also depend on whether teams can integrate this proxy pattern into existing orchestration, observability, and safety controls.”

Killing train-serve skew, the oldest & most expensive bug in machine learning

Colorado-based data science professional Priyank Jain tells The New Stack that training through the production harness to strip the framing away is really all about “killing train-serve skew”, which is the oldest and most expensive bug in applied machine learning.

“Models rarely blow up in production because the math was wrong,” Jain says. “They blow up because the training setup quietly lied to them about what production actually looks like. Training through the same harness that serves the agent is the right instinct, and honestly it’s overdue. The accuracy bump is nice, but the real win is that the thing you optimized is finally the thing you shipped.”

In terms of what Microsoft is getting right here, Jain says it shows Redmond wants to “meet developers where they already are,” i.e., allowing users to improve an existing agent with reinforcement learning without rewriting their deployment just to please a training framework.

“That’s a genuinely good call, and it opens this up to folks who aren’t reinforcement learning specialists. What I’d worry about is that it also lowers the bar for people to run reinforcement learning they don’t fully understand. The hard part was never wiring up the loop; it’s designing a reward that survives contact with a model actively looking for the path of least resistance. Make that part easy, and you’ll get more people optimizing the wrong thing faster,” Jain advises.

“3500 lines of code is small enough that an infrastructure engineer can actually read it before trusting it… and that’s a good thing.”

Just 3,500 lines of core Python code

Built with “simplicity as its first principle”, the entire framework consists of around 3,500 lines of core Python code.

Infrastructure engineering developer and founder-developer of Tooldex, a platform that autodiscovers MCP servers across LLM agents, Ria Banerjee, tells The New Stack that the simplicity element here is a positive, i.e. 3500 lines of code is small enough that an “infrastructure engineer can actually read it before trusting it,” and that’s a good thing.

“But in terms of who would actually use this tool, the honest answer is fewer teams than the framing suggests,” Banerjee says. “Realistically, you would need a GPU cluster and a Kubernetes cluster under your control. An app developer who built a support-triage agent on LangChain isn’t running that. So the real audience is platform teams that already employ infrastructure engineers.”

“Because the harness is where the behavior actually lives, but you’re now baking your harness’s quirks into the model weights… so you change your retry logic next quarter, you’ve silently shifted what the model was trained on. So now, you have to version your harness,” adds Albuquerque-based Banerjee.

Microsoft has released the complete workflow and training scripts to facilitate reproducible, harnessed agentic RL in Agent Lightning v1.0 on GitHub under the MIT license, including data cleaning and reward-hacking prevention.

The post Microsoft just released Agent Lightning v1.0. Here’s why it matters for platform engineers. appeared first on The New Stack.

What do you do when a technology you’ve become dependent on belongs to someone else? You buy the startup behind it, of course. And that’s exactly what data warehouse company MotherDuck has done with Tower, a data infrastructure startup whose technology was already powering MotherDuck’s AI-built data pipelines.

The deal, announced on Tuesday, is MotherDuck’s first acquisition in its four-year history, bringing both Tower’s technology and team in-house as MotherDuck pushes further into AI agents that can build and operate data pipelines.

Tower takes flight

Tower was founded out of Germany in late 2024 by ex-Snowflake engineers Serhii Sokolenko (CEO) and Brad Heller (CTO). Their pitch: once a developer, or an AI assistant, has written the code for a data pipeline, someone still has to package it, deploy it to the right infrastructure, wire up credentials, and maintain it — the unglamorous work, Sokolenko told The New Stack back in March, that amounts to data engineering’s “last mile.”

Tower, essentially, is a managed runtime for Python pipelines — it packages the code, deploys it, and keeps it running in production. It also offers tools built on top of that runtime, like the browser-based AI agent Tower Control that allows users to describe the pipeline they want in plain language.

Control can then generate the code, deploy it as a Tower app, and run it — effectively taking the process from prompt to production without the developer having to set up the underlying runtime themselves.

MotherDuck, for its part, is a serverless data warehouse built on the open-source database DuckDB, founded in 2022 by Jordan Tigani, a former engineering lead at Google focused on BigQuery. The company has raised some $100 million since its inception.

MotherDuck’s original pitch leaned on speed and local compute: queries could run on a laptop via DuckDB, in MotherDuck’s cloud, or across both — a departure from the likes of Snowflake, Databricks and BigQuery’s cloud-first model. More recently, MotherDuck has extended that approach to AI agents, using MCP to let agents interact directly with data. And things took a more operational turn in June with the launch of Flights, a feature that exposes a general-purpose Python runtime through the same MCP server, letting agents create, run and schedule data pipelines.

And Tower, as it turns out, was the key infrastructure underpinning Flights.

‘We became their largest customer almost overnight’

Tower’s involvement with MotherDuck actually predates Flights. Tigani says MotherDuck had initially been looking for a third-party tool it could recommend to customers as an easier way to get data into its warehouses. But then advances in AI changed the nature of the problem the company thought it needed to solve.

“When AI suddenly started to be able to solve data problems, we realized we were thinking about the problem wrong.”

“When AI suddenly started to be able to solve data problems, we realized we were thinking about the problem wrong,” Tigani tells The New Stack. “Claude can solve that problem we were trying to address by writing the connectors to help people move their data, but what it can’t do is the sandboxing and scheduling.”

That left MotherDuck needing somewhere to safely execute the code those agents generated, manage credentials and run jobs on a schedule. And as luck would have it, Tower already provided those capabilities.

“It perfectly solved our problem and let us ship Flights in only a matter of weeks,” Tigani adds.

For MotherDuck, Tower had supplied the missing execution layer; for Tower, that realization translated into a significant customer relationship. “We became their largest customer almost overnight, and our teams have been shipping together ever since,” Tigani says.

Having Tower on its radar so early also gave MotherDuck a chance to test the technology — and the team behind it — before deciding whether to build something similar itself. Tigani says the calculation ultimately came down to how quickly MotherDuck could get the capabilities it wanted into customers’ hands.

“It’s always tempting to build yourself, but after trying out Tower, we realized pretty quickly that there were a bunch of problems we were going to have to solve to make our underlying infrastructure actually work well, and Tower was pretty much exactly what we needed,” he says.

Ultimately, the more central Tower became to what MotherDuck wanted to build, the stronger the case became for owning the technology outright. Once Tower was executing jobs created and scheduled inside MotherDuck, Tigani argues, customers would inevitably hold MotherDuck accountable for the security, reliability and behavior of that runtime.

“There’s a rule I’ve relearned at every infrastructure company I’ve worked at: you can rent a feature, but you can’t rent a foundation,” Tigani says. “When an agent inside MotherDuck builds a job and schedules it, the thing executing that job is our product — whatever logo is on it.”

“There’s a rule I’ve relearned at every infrastructure company I’ve worked at: you can rent a feature, but you can’t rent a foundation.”

One example of what MotherDuck now hopes to do with that technology involves bringing Flights together with Dives, an AI-generated data visualization feature it debuted in February. Tower can generate stable URLs for jobs running through Flights, effectively allowing those jobs to act as data APIs that a Dive — or another frontend — can call.

Tigani gives the example of an application displaying user recommendations. A Dive could generate the interface for viewing those recommendations, while a Flight could handle requests to create or modify them. Rather than giving the frontend broad write access to the underlying data, the Flight can constrain and validate what changes users are allowed to make.

“When you put them together, you can build rich applications,” Tigani says.

Tower’s next chapter

All of this raises an obvious question for Tower customers. Part of the startup’s pitch was that developers could use its runtime without tying themselves to a particular data platform, and now Tower itself belongs to one.

Tower co-founder and CEO Serhii Sokolenko argues that MotherDuck represents a different kind of home to that of the industry’s larger cloud data platforms. His case is that Tower can become more deeply integrated without being bent around an architecture established long before AI agents entered the picture.

“Joining a hyperscaler usually means adapting to its legacy architecture,” Sokolenko tells The New Stack. “Joining MotherDuck lets us help shape where data and AI infrastructure are heading.”

“Joining a hyperscaler usually means adapting to its legacy architecture.”

There is still a trade-off, though. Tower is giving up some of the breadth that came with being database-agnostic in exchange for building much more specifically around one platform — a bet that tighter integration will ultimately produce a better experience than supporting many systems at arm’s length.

“By focusing Tower’s Pythonic compute specifically on MotherDuck, we’re trading broad, basic connectivity for deep, native execution,” he says.

Sokolenko’s argument is that the “lock-in” question then shifts down a layer. Tower may now be tied much more closely to MotherDuck, but because MotherDuck itself is built on DuckDB, he says the underlying data remains open and portable. The aim is to bring the runtime, agents and warehouse closer together without trapping the data itself inside a proprietary system.

That also helps explain why MotherDuck’s hybrid execution model appealed to Tower. DuckDB lets work move between local and cloud compute, which Sokolenko sees as closely aligned with Tower’s own direction.

“This directly matches Tower’s vision — allowing business users and agents to move seamlessly from local data exploration to cloud production execution,” he says.

For Tower’s existing customers, the immediate future means a move toward MotherDuck. Sokolenko says Tower customers are already in discussions with MotherDuck about migration paths, while people who have used Tower previously will be invited to try MotherDuck and its broader agentic data capabilities.

Tigani confirms that MotherDuck is working to move existing Tower customers onto Flights, though he concedes that the two products aren’t entirely identical. “There are a couple of differences, and we’re working on closing the gaps to make the transition more seamless,” he says.

Tower’s technology will meanwhile be folded more deeply into MotherDuck itself. Today, MotherDuck has two separate sandboxed, on-demand runtimes: Flights, which is backed by Tower, and Ducklings, its serverless DuckDB instances. Tigani says the plan is eventually to merge the two, combining the near-instant startup of Ducklings with the more robust sandboxing provided by Tower jobs.

Surfing the agent wave

MotherDuck is hardly alone in pushing data agents beyond answering questions. Databricks’ Genie Code can generate and run code, build pipelines and debug failures inside Databricks. Snowflake, meanwhile, has been moving in a similar direction with the likes of CoCo, its AI coding agent, while newer CoCo Automations can schedule unattended agent runs inside Snowflake-managed sandboxes.

The details differ, but both point toward a broader shift in the data industry: giving AI agents the infrastructure to act on data and operate the systems around it, rather than simply query what is already there.

“AI makes it possible to build useful features that we could never fathom five to 10 years ago. The platform is the most complex part of the data estate, and so warehouse vendors are well positioned to be at the center of whatever new patterns emerge.”

Tigani has been predicting something close to this for some time. Earlier this year, he outlined a future in which data engineering increasingly becomes an agent-supervision problem, with agents handling tasks such as building and repairing pipelines and responding to changes in schemas and data quality, while humans oversee their work. He also previously likened the advance of LLMs to a wave that data companies will have to learn to ride.

“The way I like to think about it is this — data platform vendors are reacting to new opportunities to make lives better for their customers,” Tigani says. “AI makes it possible to build useful features that we could never fathom five to 10 years ago. The platform is the most complex part of the data estate, and so warehouse vendors are well positioned to be at the center of whatever new patterns emerge. The Tower acquisition gives us a platform for deploying, tracking, and scheduling data agents, which should set us up well to surf that wave.”

The post “You can rent a feature, but you can’t rent a foundation”: why MotherDuck bought the startup already powering its data pipelines appeared first on The New Stack.

Put several AI bots to work, and a mistake by one may not stay within its assigned task. For example, that error could reach another bot’s files and login credentials, or even the computer running them all. Two releases this month offered companies very different ways of containing that risk.

On August 17, Nous Research announced that its Bot Mode would ship bundled and enabled by default in Hermes Agent v0.20.3, turning agent profiles into a roster of named bots that hand off work to one another. About a week earlier, SpaceXAI launched Grok Bot with almost the same interface: a sidebar of named teammates who sign in to your tools and keep working long after you close your laptop.

The interface converged within a week, but the answer to the question every platform team has to ask did not: When one bot goes wrong, what can it reach?

Four projects have now come to their own answer, and no two of them agree.

1. Grok Bot draws the line around the user account.
2. Hermes draws it around the profile.
3. OpenClaw draws it around an optional runtime sandbox.
4. ClawFleet draws it around a container.

Taken together, the documentation shows an industry converging on the persistent coworker interface far faster than it is converging on what constitutes an identity or a security boundary for it.

Four projects, four written answers

Every one of these products now offers the same surface. You create several named agents, assign them different jobs, and have them pass work among themselves. The naming convention alone suggests separation, since a bot called Expense Manager and a bot called Talent Scout sound like they occupy different rooms in a shared office.

The documentation says otherwise, and it says something different in each case. The unit of isolation is the account in one product, the profile directory in another, an opt-in container in a third, and the deployment topology in the fourth. Those four units are not interchangeable, and an operator who assumes the roster itself is the boundary will be right in exactly one of the four cases.

Is Grok Bot confused about what it wants to be?

SpaceXAI’s launch post leads with the promise that bots have their own computer. The documentation, last updated the same day, describes a single persistent cloud computer assigned to the user account rather than to any individual bot. Browser cookies and signed-in sessions are shared across the roster, files are visible to every bot, and command-line credentials are shared. One bot can pick up work that another bot saved.

Each bot gets its own screen on that machine, which allows several of them to run browser and desktop tools in parallel. SpaceXAI is direct about what those screens are not. The documentation calls them “separate work surfaces, not separate security boundaries.” It then instructs operators to keep a credential or file off the machine entirely if another bot on the account cannot use it.

The consequences run further than credentials. Signing in for one bot makes that session available to the others because the browser is shared. Installed connectors are account-wide, and their availability is not isolated to a single bot. The shared workspace sits at /workspace and is designed to survive computer updates and recovery, so the durable state is shared across the whole roster.

None of this is an implementation accident. It is what makes handoffs between bots cheap, and cheap handoffs are the product. But an operator reading only the launch page would build a mental model that the documentation contradicts, and that gap is where the risk sits.

Hermes gives each bot its own profile

Nous took the opposite architectural position. In Hermes, a bot is a profile, and each profile has its own configuration, memory, skills, credentials, and chat history stored in its own directory on disk. Handoffs between bots run as real invocations against the named profile, rather than as a shared context blob passed around within a single process.

Nous shipped the teammate protocol as part of v0.20.3, alongside the MCP 2.x SDK migration and a set of runtime hardening changes. The company archived the standalone plugin repository once the merge was completed. Bot Mode is on by default, so a Hermes user who updates gets the roster without opting in.

Two qualifications matter before anyone reads that as containment. A separate credential store does not guarantee different credentials, since what ends up in a new profile depends on how the operator created it and what they edited afterward. And every profile still shares the host machine, its operating system user, and its filesystem permissions. What Hermes documents is workstation-level separation of agent state, a meaningfully stronger default than a shared cloud account, but not the same as isolation.

OpenClaw’s sandbox is off by default

OpenClaw documents the most complete boundary of the four. When the sandbox is enabled with the Docker backend, agent tool execution runs inside isolated containers. At the same time, the gateway remains on the host, and the scope can be selected per session, per agent, or shared across agents. Each scope gets its own workspace. Auth material lives per agent under an agent-scoped auth profiles file. Operators can configure network isolation, resource limits, and allow-or-deny tool policies on top of it.

The documented default for that sandbox mode is off. That is a defensible choice for a project most people run on a laptop, where the container overhead buys little against a single-user threat model. The underlying setup behavior deserves more attention. If sandbox prerequisites fail during setup, the script resets sandbox mode to off rather than refusing to start, so an operator who intended isolation and encountered a Docker socket issue ends up running without sandbox isolation. The documentation also warns against mounting the host Docker socket into agent sandbox containers and flags the CLI container’s shared network namespace with the gateway as a trust boundary in its own right.

ClawFleet answers the same question by moving it into the deployment topology. The project documents a wrapper that puts each OpenClaw or Hermes agent in its own Docker container with an isolated filesystem and network. It lists roughly 500 MB of memory per OpenClaw instance and 150 MB per Hermes instance. That cost is why the other three projects make the boundary optional or skip it, and naming the number makes the trade-off legible.

How to choose which bot is right for you

Scenario	Documented fit	Rationale
Persistent work that must continue with the laptop closed	Grok Bot	The only one of the four with a vendor-run always-on cloud computer, at the cost of one shared credential surface for the whole roster
Several agents with genuinely different credential sets on one workstation	Hermes	Per-profile stores are the documented default, and Bot Mode ships on
Untrusted or multi-tenant agent sessions	OpenClaw with sandbox enabled	Per-agent or per-session container scope with configurable network and tool policy, provided the operator turns it on and verifies it
Isolation as the deployment model rather than a runtime setting	ClawFleet	Container per agent with separate filesystem and networking, at a documented memory cost per instance

Each project gives operators different advice

The operational guidance diverges as sharply as the architecture. OpenClaw’s docs read like infrastructure documentation, naming specific hazards such as the Docker socket and the shared network namespace, and telling the operator what not to do. Hermes documents the profile layout and the protocol, then leaves policy to the operator. Grok Bot’s guidance is largely the warning itself, an instruction to treat the account as the boundary and to keep sensitive credentials off the shared machine entirely.

Grok Bot carries a second disclaimer worth reading alongside the first. Sensitive actions route through an approval mechanism. SpaceXAI documents the categories that trigger it, including sending messages, publishing content, purchases and transfers, deleting data, and touching production. Enforcement runs through an LLM classifier. Cursor‘s documentation for that same engine states plainly that the classifier is not a security boundary and can make mistakes. A buyer evaluating the product therefore finds the phrase twice, attached to the two mechanisms they would most reasonably assume protect them.

AI agents still lack identities of their own

Enterprises can adopt any of these four products today and get real work done, and the honest reading is that all four are engineering their boundaries in good faith against different threat models. What none of them provides is an identity for the agent. In every case, the bot borrows the operator’s credentials, whether from a shared cloud browser, a profile directory, or a container volume, and the entire security conversation boils down to how far those borrowed credentials travel.

“There is no primitive to standardize on, so each project has invented a boundary at whatever layer it already controlled, the account, the profile, the runtime, or the container.”

That is why the four answers differ so much. There is no primitive to standardize on, so each project has invented a boundary at whatever layer it already controlled: the account, the profile, the runtime, or the container. Expect to see that gap close on the identity side rather than the agent side, through scoped delegation and per-agent credentials issued by the identity provider, rather than being copied from the human. Until then, the useful move for platform teams is unglamorous and specific. Read the security page before the launch page, because for this class of product, they describe different things.

The post Grok Bot vs. Hermes: Where each draws the security boundary appeared first on The New Stack.

On July 16, the team at Hugging Face noticed something weird moving through their production systems: An intruder that was cloning datasets, harvesting credentials, and hopping between internal clusters faster than a human could type. Five days into the investigation, the authorities were notified, and several days later, OpenAI traced the intruder back to one of its models.

“Two frontier models escaped their test environments this summer; only one of them had a funny reason.”

That prompted Anthropic to look through its own test logs, and it found that similar events had occurred three times. One Claude model probed nine thousand hosts in a single run without setting off any alarms. Another slipped a poisoned package into the public Python registry, compromising fifteen machines before it was caught. The third incident dated back to April and went undetected for three months until a rival’s error prompted the audit. Nobody had noticed. Two of the three companies involved didn’t know anything was amiss until they got a call.

Containment by instruction

A sandbox, in the context of AI testing, means a model gets a task, a set of tools, and an instruction: no access to the outside world.

OpenAI’s model found an exploit in a tool it was allowed to use, and leveraged it to access the open Internet. Anthropic’s models didn’t even need that. The models were told a path didn’t exist, so when their tasks led them down a path that wasn’t supposed to be there, they treated it as part of the exercise.

“In both instances, the only constraint was an instruction. There was no mechanism outside the model itself to ensure those instructions wouldn’t be treated as optional.”

In both instances, the only constraint was an instruction. There was no mechanism outside the model itself to ensure those instructions wouldn’t be treated as optional.

So how did they escape?

OpenAI’s agent was being evaluated on a cybersecurity benchmark called CyberGym. During the test, it chained a code-execution flaw in a dataset loader to a second flaw in the platform dataset configuration file handler. It used the combined exploit to run commands on a live processing worker. From there it grabbed a write-scoped access token and cloned a private dataset as a staging area, which became a dead drop for its own tools. By the time anyone noticed, the intrusion had been running for three days, touched several internal clusters, and compromised accounts at four other companies, according to Hugging Face’s own review.

What it wanted, in the end, was comical: the answer key to the test it was taking.

Anthropic’s three incidents were nowhere near as convoluted. Its models were being run through security exercises against third-party evaluation partners, and the only thing keeping them from accessing the Internet was an instruction not to. One incident involved an SQL injection targeting a resource the model wasn’t supposed to be able to access. Another, during an exercise where the usual safeguards were deliberately switched off, ended with a Claude model publishing a malicious package to the Python Package Index, and fifteen machines picking it up before it was pulled down. No zero-day exploit. No dramatic escape. Just locked doors that turned out to be mirages.

The structural weakness

Strip away the specifics, and both incidents share the same root cause: there was nothing outside the model’s own judgment that could enforce the limitations placed on it. A credential that shouldn’t have been able to access the open Internet did so anyway. An instruction that should’ve stopped a plan cold got treated as just another input to reason around.

“If a fence doesn’t hold during testing, it won’t hold in production.”

It feels like we’ve spent the last year reliving the same problem in new ways. For example, in deleted databases, in poisoned supply chains, in every credential an agent inherited because provisioning it correctly took one extra step nobody got around to. The only delta is where the boundary was supposed to be: around the dev experiment, meant to contain dangerous capabilities, or standing between it and every production system it could reach. If a fence doesn’t hold during testing, it won’t hold in production.

How do we break out of the loop?

Unfortunately, the answer isn’t self-improvement. A more capable model wouldn’t have caught any of this. The only way to fix it is with Infrastructure that verifies where a model actually is, rather than trusting its reports.

Egress controls that are closed by default. A test environment that can reach the open Internet is probably not a great idea. Outbound traffic should be locked to an explicit allowlist. Never assume a model will simply decline to look for a way out. “No internet access” needs to be a network-enforced fact, not just an instruction.

Narrowly scoped credentials with short lifespans. Both incidents ran, in part, on tokens that extended beyond what the tasks required. A credential built for one evaluation run, scoped to nothing beyond it, and expiring on a timer, can’t become the write-scoped key that turns into a lateral-movement problem across multiple clusters. Yes, it will be a chore; so gird your patience by imagining having to explain to company leadership why taking steps to avert a catastrophe was too tedious to bother with.

Policy gates that sit outside the agent’s reasoning. An agent that can decide for itself whether an action is in scope is grading its own test. A plan should have clear rules written in code, evaluated by something other than the agent, before anything executes. Open Policy Agent has become a de facto default here for exactly this reason. It doesn’t care how confident the plan sounds; nothing runs without proper authorization from an external source.

An audit trail that is entirely independent of the agent. Both incidents were discovered by rereading logs after the fact, not by asking the models what happened. That’s how oversight should be done. As the proverb goes: Quis custodiet ipsos custodes? Who watches the watchers? A system of record that captures what actually occurred, regardless of the agent’s own account of itself, is the only version of events worth trusting.

This is a sign

The industry spent a decade learning that the CI/CD pipeline is an attack vector that requires real fortification, not just bolted-on convenience. Test environments for frontier models are following the same arc at a faster pace. The next time one of these agents escapes containment, it’ll probably be one built around finding unlocked doors, which will make it substantially more dangerous than a coding agent that deletes a few databases.

Test rigs must be treated as if they hold something real, because, as far as the credentials are concerned, they do. A sign on a door is never going to be enough to keep everyone out; there has to be a lock whose robustness correlates to the value of what it guards.

“A sign on a door is never going to be enough to keep everyone out; there has to be a lock whose robustness correlates to the value of what it guards.”

Whether by a state-sponsored crew probing a water management system in the middle of the night, or a company’s own model trying to shave a few points off a benchmark, boundaries will always be tested. Two labs found out this summer, and the story needs to be taken seriously. The vulnerabilities are real, the transparency from the labs is welcome, and the containment failures are a cause for concern.

Catching a model that tried the handle is the easy part; both labs proved that. The more challenging, and therefore critical, part is making sure the next containment environment actually has doors that are firmly locked.

The post Securing sandboxes: What happens when AI agents escape containment? appeared first on The New Stack.

The artificial intelligence landscape has reached a pivotal inflection point. Over the past several years, the paradigm has shifted from passive, conversational Large Language Models (LLMs) to autonomous AI agents, digital software entities capable of reasoning, invoking tools, executing multi-step workflows, and making real-time decisions across enterprise systems without constant human intervention.

As organizations accelerate the production deployment of autonomous agents, modern security frameworks must evolve to keep pace. Traditional Identity and Access Management (IAM) systems were primarily designed around two distinct operational models:

• Human users: Authenticated via Multi-Factor Authentication (MFA), Single Sign-On (SSO), and interactive sessions.
• Service accounts and workloads: Authenticated via static API keys, fixed service tokens, or IP whitelisting.

Autonomous AI agents blur the line between these two models. An agent acts with the non-deterministic reasoning and delegated agency of a human, but operates at the scale, parallel velocity, and automation speed of a machine service.

Identity dimension	Human users	Traditional service accounts	Autonomous AI agents
Velocity & scale	Low (human typing speed)	High (scripted requests)	Extremely high (dynamic, parallel tool execution)
Decision logic	Deterministic / goal-driven	Rigid / hardcoded	Non-deterministic / adaptive reasoning
Auth mechanics	Passkeys, MFA, SSO	Static API keys, OAuth M2M	Ephemeral delegation & contextual attestation
Access granularity	Role-based access control (RBAC)	System-wide scope	Fine-grained / relationship-based (ReBAC/ABAC)

To safely harness the power of autonomous workflows, enterprise security architecture must move toward continuous, agent-aware Zero Trust governance. Below are six foundational identity capabilities that organizations should adopt to secure AI agents in production environments effectively.

“Autonomous AI agents blur the line between these two models. An agent acts with the non-deterministic reasoning and delegated agency of a human, but operates at the scale, parallel velocity, and automation speed of a machine service.”

“When it comes to agentic AI identity, most organizations are woefully unprepared for inherent security risks and operational challenges of managing those identities.” – Ken Buckler, Research Director, EMA – Agentic AI Identities – Is Your Organization Prepared?

1. Verifiable agent identities & “Know Your Agent” (KYA)

Autonomous entities require verifiable digital identity frameworks that establish clear, cryptographically bound accountability for every machine action.

• Cryptographic attestation: Every agent instance should possess a unique, cryptographically signed identity bound to its underlying model version, execution environment, and deployment origin.
• Delegation chains: When a human user delegates a task to an agent (or when a primary agent spawns sub-agents), the identity system must construct an immutable, traceable chain of delegation. This ensures the infrastructure can continuously verify who authorized the initial action and what specific scope was granted.

2. Ephemeral credentials & just-in-time (JIT) tokenization

Static API keys and persistent service tokens represent a significant surface area of exposure when integrated into dynamic agentic workflows. Replacing long-lived credentials with short-lived tokens dramatically reduces the potential window of risk.

• Just-in-time (JIT) minting: AI agents should operate with ephemeral credentials generated on demand, strictly limited to the API calls required for a single operational step, and configured to expire within seconds or minutes.
• Bound OAuth flows & PKCE: Enforcing Proof Key for Code Exchange (PKCE) and strict token-binding protocols ensures that credentials cannot be reused or replayed outside of their intended runtime context.

“Replacing long-lived credentials with short-lived tokens dramatically reduces the potential window of risk.”

3. Relationship-based access control (ReBAC) & intent binding

Coarse-grained permissions, such as those in traditional Role-Based Access Control (RBAC), are often too broad for non-deterministic tool usage. Access governance should be based on fine-grained relationship models and task intent.

• Intent-bound authorization: Authorization systems should evaluate not only whether an agent has general permission to access a resource, but whether that request directly aligns with the explicitly authorized sub-task.
• Fine-grained contextual policies: Implementing relationship-based access control (ReBAC) or Attribute-Based Access Control (ABAC) allows teams to define precise conditions (e.g., “Agent X may read Document Y only if human user Z is the document owner and the active workflow is ‘Data Summarization'”).

4. Machine-speed containment & automated anomaly detection

Because AI agents operate at speeds far exceeding those of manual monitoring, security containment mechanisms must be automated, agent-aware, and built into the control plane.

• Behavioral rate & scope limits: Security controls should establish baselines for expected agent behavior to detect anomalies, such as rapid parallel tool invocations, repetitive execution loops, or unusual queries to non-standard endpoints.
• Automated circuit breakers: If an agent’s execution pattern or request velocity exceeds defined behavioral bounds, identity proxies can automatically revoke ephemeral tokens and safely isolate the workload in real time.

5. In-the-loop runtime enforcement & human approvals

Security governance cannot rely solely on static pre-authorization; policies must be evaluated continuously at runtime before individual actions execute.

• Action-level policy interception: Enforce real-time policy checks at the agent harness layer—evaluating shell commands, database queries, file operations, and outbound API calls against governance rules before execution.
• Configurable approval workflows: Establish flexible escalation paths that permit low-risk read operations automatically while requiring explicit human-in-the-loop validation for high-impact actions, such as code deployments or financial transactions.

6. Web-scale identity architecture built for machine workloads

Autonomous workflows generate significant operational volume. Identity systems must be architected to handle machine-scale throughput without performance degradation or store bloat.

• Machine-speed throughput: Multi-step workflows and parallel worker agents demand identity control planes that can handle high-volume token validation and policy evaluation with minimal latency.
• Lifecycle governance for sub-agents: Dynamically spawned sub-agents require rapid provisioning and immediate teardown upon task completion, thereby preventing the accumulation of orphaned credentials and ensuring clean session termination.
• Inline cryptographic safeguards: Prioritizing inline policy enforcement over post-mortem log reviews allows organizations to intercept unauthorized state changes before they occur, maintaining operational integrity across multi-cloud environments.

Conclusion: securing the future of enterprise automation

As AI models evolve from passive assistance tools to active operational participants, identity becomes the primary boundary for enterprise governance. By bridging the machine identity gap with verifiable agent identities, short-lived JIT credentials, fine-grained relationship authorization, and automated runtime enforcement, security leaders can confidently deploy autonomous AI agents to drive productivity while maintaining complete operational control.

The post Six identity capabilities for securing autonomous AI agents appeared first on The New Stack.