OpenAI has detailed how the GPT-5.6 model family balances capability and cost across its stack, and the company‘s most important claim is a benchmark result showing that its flagship model, GPT-5.6 Sol, with maximum reasoning, outperforms Claude Fable 5 from Anthropic on the Artificial Analysis Coding Agent Index. The margin comes with 54% fewer output tokens. The findings were shared in a company blog post on Wednesday.

For developers, what matters most is how OpenAI arrived at the benchmark results and the role GPT-5.6 Sol played in optimizing the infrastructure that now serves it.

The family spans three models across the price curve. In addition to Sol, there is Terra, which performs as well as GPT-5.5 on intelligence benchmarks at half the price, and Luna, the fastest and most affordable, which is priced 80% below Sol.

The efficiencies come from optimizations at four layers, spanning the models, inference, the API stack, and the agentic harness behind Codex and ChatGPT Work.

According to the post reviewed by The New Stack ahead of its publication, the efficiencies come from optimizations across four layers: models, inference, the API stack, and the agentic harness behind Codex and ChatGPT Work. The architecture diagrams in the post draw the same separation as three planes: the local harness, CPU-bound API orchestration, and GPU-bound model inference.

For developers building and operating agents, the post is worth reading less as a product announcement and more as a systems paper. Nearly every technique it describes, from incremental tokenization to append-only context, applies to any team running a tool-calling loop at scale.

A model that rewrites its own serving code

The efficiency work starts in training. OpenAI says GPT-5.6 is trained to achieve more work per token, with training optimized for both task success and efficiency so the model takes a more direct path through a task.

With Codex, GPT-5.6 Sol autonomously rewrote and optimized OpenAI’s production kernels, the core code that executes the mathematical operations making up the model. OpenAI says this worked in part because GPT-5.6 is trained to write and improve kernels in Triton and Gluon. Both are open-source GPU programming languages maintained by OpenAI. These efforts, combined with broader kernel advancements from the model, reduced end-to-end serving costs by 20%.

Correctness is the obvious concern when a model rewrites the code it runs on. To address it, OpenAI reports heavy investment in verification tooling. That includes the open-source Floating-Point Sanitizer (FpSan), which validates the kernels GPT-5.6 Sol produces before they reach production.

The model went further with speculative decoding, a technique in which a smaller draft model proposes several tokens that the primary model verifies in parallel. The approach will feel familiar to anyone who understands how modern CPUs speculatively execute instructions ahead of a branch. Accepted proposals produce multiple output tokens from a single pass of the primary model. That reduces the expensive sequential computation the primary model would otherwise perform.

GPT-5.6 Sol in Codex improved its own draft model by designing and running hundreds of experiments on its architecture, with changes tested across size, structure, and features. The model also launched and monitored the speculative training process. It intervened autonomously when hardware failed or training became unstable. OpenAI reports the resulting improvements lifted token-generation efficiency by more than 15%.

More tokens from the same GPUs

OpenAI frames its inference work around a single objective – serving more tokens with the same hardware while preserving the intelligence, latency, availability, and reliability users expect. In a compute-constrained market where demand grows faster than capacity, that objective influences every design decision in the serving path.

Load balancing operates at three distinct levels. Globally, requests are routed based on geography, available capacity, and accelerator type. Within a cluster, work is distributed across model instances based on load, context length, and cache availability. Within each instance, work is partitioned across accelerators, the model’s experts, and computing cores. GPT-5.6 Sol in Codex helps OpenAI analyze production traffic and identify previously overlooked sources of imbalance. The same loop tests new routing strategies and helps engineers constantly tune the heuristics. OpenAI states that these load-balancing improvements alone dramatically reduced the cost of serving its models.

The key-value (KV) cache received the same treatment. When processing uncached input tokens, the model builds the KV cache in a single compute-intensive pass, then repeatedly reads from and extends it during generation. The optimal serving configuration depends heavily on prompt length, batch size, and cache hit rate. It covers batching, sharding, and cache management, and the configuration space was previously too large to tune systematically. With GPT-5.6 Sol in Codex, OpenAI analyzed production workloads and generated candidate configurations. The company says this makes workload-specific optimization practical at a level that broad heuristics could not reach earlier.

Process only what changed

The API team focuses on everything that happens around a model call. After a prompt is submitted, the API stack receives the request, loads context, and validates the input. Safety checks run next, and the text is converted into tokens for inference. OpenAI measures this overhead through time to first token (TTFT), time between tokens (TBT), and end-to-end time (E2E).

Tokenization is an O(n) operation, so longer prompts take longer to process. Codex would send the full conversation context after every tool call. That meant paying to tokenize the same conversation dozens of times per turn, even though only a small amount of context was new in each request. OpenAI solved this with a WebSocket integration that hoists tokenization state to the server. The first call renders and tokenizes the full prompt. Later calls send only the new input with a reference to the conversation, bringing the operation closer to O(1). The pattern mirrors an incremental build system that recompiles only the files that changed rather than the whole project.

These savings compound in tool-heavy workflows, where every tool result triggers another round trip through the API. For rollouts with 20 or more tool calls, OpenAI reports up to roughly 40% faster end-to-end execution.

Hardware turned out to matter as much as protocol design. All of OpenAI’s infrastructure runs on Kubernetes. The company found that nodes with the same instance type often carried different CPU generations, with many running outdated processors. In its measurements, the older processors consumed roughly twice the CPU resources for the same work. Reweighting traffic toward newer processors improved TTFT by about 20%, and CPU generation is now part of capacity planning.

OpenAI names four fates for application-layer overhead: delete it, overlap it with useful work, run it on faster hardware, or make the code consume fewer CPU cycles. Its asyncio changes move work off the critical path, while newer hardware and Rust implementations make the remaining work faster and more predictable.

An append-only harness

The agentic harness is a Rust-based orchestration layer that connects the models, tools, and the user’s environment. In a single turn, Codex might inspect source code, search deployment history, and read incident reports. Editing a file and running the tests each add another request. Since a task can require 30 model requests, an extra second per request adds up quickly.

Context bloat is the first target for the harness. As agents gain access to more tools, skills, plugins, and conversation history, context windows expand. The growth increases cost, distracts the model, and prompts unnecessary reasoning. The harness counters this with deferred discovery, which surfaces integrations, custom Model Context Protocol (MCP) tools, skills, and plugins only when needed. Tool output is capped at 10,000 tokens by default unless the model requests a different limit.

Prompt caching drives the second design choice. An agent loop resends the same instructions, tool definitions, and earlier results multiple times within a turn. The harness therefore treats all model-visible history as append-only, with new messages and tool results added at the end rather than inserted into earlier context. Tools are presented in a deterministic order, and runtime settings, such as approval policies, are applied during execution rather than embedded in tool definitions. OpenAI credits this design for the high prompt-cache hit rates in Codex and ChatGPT Work.

Platform teams building internal agents can adopt every one of these choices without OpenAI’s scale. Append-only context, deterministic tool ordering, and capped tool output attack token spend directly. That makes them the most portable lessons in the post for enterprises watching inference bills grow with each new agent deployment.

Where the gains come from

The post associates a number with most of its optimizations, and the figures are OpenAI’s own production measurements. Taken together, they show how modest individual wins compound across a serving stack.

Layer	Technique	Claimed gain
Model inference	Autonomous kernel rewrites in Triton and Gluon	20% lower end-to-end serving costs
Model inference	Speculative decoding with a self-improved draft model	Over 15% better token-generation efficiency
API stack	Stateful WebSockets with incremental tokenization	Up to roughly 40% faster runs at 20+ tool calls
API stack	Routing traffic toward newer CPU generations	About 20% better time to first token
Agent harness	Deferred discovery and a 10,000-token tool output cap	Reduced context bloat and cost

The key takeaways

In summary, OpenAI describes the GPT-5.6 efficiency gains as the result of years of compounding improvements. They span research, inference, the API stack, and the agentic harness. The company states that the model’s role in landing many of them makes it optimistic that the pace of optimization will accelerate. Kernel work is called out as an area of continued investment.

The post positions efficiency, alongside raw intelligence, as the axis on which frontier labs now compete. The claimed 54% output-token advantage over Claude Fable 5 shows how OpenAI intends to fight that battle. The engineering blog makes a plausible case that software optimization is becoming an important lever alongside hardware improvements in reducing the cost of serving frontier models. The figures remain OpenAI’s own production measurements. The autonomy on display operates within Codex, with engineers in the loop. Developers and enterprises benefit either way, as these under-the-hood improvements reach them as more capable models at lower prices across the cost-intelligence curve.

The post Kernel of truth: GPT-5.6 Sol can cut its own costs, says OpenAI appeared first on The New Stack.

As more companies plug AI agents into the deepest depths of their internal data banks, how can they be sure those agents actually understand how the business works? Right now, many of these organizations are stuck manually building a Markdown file, hoping they find time to rewrite it each time the business changes.

Modus, for its part, thinks it has found a better way. The startup that formally exits stealth this week with $10 million in funding in tow is building what is coming to be known in industry parlance as a “context warehouse” — a layer that sits alongside a company’s existing data warehouse, continuously mapping how the business operates across its systems, and handing an AI agent only the relevant slice of that map when it needs it.

In real terms, Modus crawls relevant assets from sources like GitHub, dbt, Jira, Snowflake, and Postgres, using what it calls a Context Miner to continuously learn how the business operates. What it finds gets turned into “dynamically generated skills”: Short, purpose-built briefs, assembled in real time by a second system, the Context Composer, and handed to an agent the moment it’s given a task.

Modus co-founder and CTO Tomer Mesika tells The New Stack that this mining runs continuously, guided by its own internal logic for what to check and how often.

“We have a lot of mechanisms in place to know what to mine from the organization, at what cadence, how to look for deltas, when to dive deeper in, and when not to,” Mesika says.

“We have a lot of mechanisms in place to know what to mine from the organization, at what cadence, how to look for deltas, when to dive deeper in, and when not to.”

Daniel Shimoni, Modus co-founder and CEO, draws a direct line to data warehousing to highlight the gap he’s trying to close. Companies have spent years building infrastructure to store and organize their data, he argues, but nothing equivalent exists for the understanding that sits atop it.

“There’s a logic behind data warehouses — companies already know that is where they manage their data,” Shimoni tells The New Stack. “But where do they manage their context? Where do they actually understand what contexts exist in their organization, that they can actually use to ensure agents only have what they need?”

Shimoni says even that first step is hard enough on its own. But keeping a company’s context accurate as the business changes is harder still.

“We’ve noticed that building the context the first time is already a challenge, but maintaining it is the bigger issue,” Shimoni says. “So Modus always learns from what the company is doing, and whenever something shifts or changes in the business, it makes sure that only the relevant and updated context is fed to agents.”

“Building the context the first time is already a challenge, but maintaining it is the bigger issue.”

Who’s buying, and why cost matters

Shimoni says Modus is targeting engineering teams, the CTO office, and VPs of R&D, as well as data teams and a newer category of AI teams.

“AI teams weren’t really around last year; it seems that a lot of data teams are transitioning to becoming VP of data and AI, or AI enablement,” Shimoni says. “So really, it’s the people who are in charge of having this AI enablement mandate in the organization, making sure AI is scaled in the organization.”

Pitching enterprises a shiny new context warehouse becomes much easier when the promise is steeped in helping them cut costs. Spend has become one of the defining anxieties of enterprise AI this year, with companies switching providers in pursuit of cheaper models, to entire economic models being built around the price of a token.

“You want the bigger models to do the heavy and complex tasks to get great value. The problem is that they are wasting a lot of their effort and a lot of their token usage on menial tasks.”

Mesika says this is a central component of Modus’s modus operandi, arguing that frontier models end up spending a chunk of their token budget on work unrelated to actually answering a question.

“You want the bigger models to do the heavy and complex tasks to get great value,” Mesika says. “The problem is that they are wasting a lot of their effort and a lot of their token usage on menial tasks.”

Those menial tasks, in Mesika’s telling, include combing through pull requests or Jira tickets just to determine what’s relevant before an agent can start the job it was assigned to.

One approach to this problem is to hand the sorting work to a smaller, cheaper model. Mesika says Modus takes that further: rather than retrieving that context at the moment a question is asked, it uses small language models alongside search engines, vector search, and a graph database, all built up in advance, to do that work continuously in the background. By the time an expensive frontier model gets involved, it’s only ever handed a finished brief of exactly what it needs.

“Everyone’s talking about context”

Shimoni and Mesika both come from data-centric companies — Lusha, a go-to-market data platform, and Cyera, a cybersecurity data company, respectively — before leaving their roles in September 2025 to start Modus together.

The two had known each other for years, and spent much of the previous year comparing notes on a problem they were both running into in very different jobs.

“We decided this is a problem worth solving, and it seems like we were spot on, because everybody’s talking about context.”

“Some of the challenges were very similar — how do we combine a lot of various data assets into one place where AI can work?” Shimoni says. “We just started to notice that this is the gap — to make AI run with confidence, at scale, across a company. We decided this is a problem worth solving, and it seems like we were spot on, because everybody’s talking about context.”

Modus closed a hitherto unannounced $10 million seed round shortly after founding, led by Insight Partners. Other backers include Soma Capital and a handful of angel investors, among them founders from Cyera and Wix.com. The company began hiring its first employees in January 2026.

The broader takeaway from Modus’s pitch is now among the most common refrains emanating from AI circles this year: that the model itself is no longer the bottleneck; what limits an AI system now is everything built around it. And for Modus, that realization has been more or less present since its inception.

“Even last year […] we could already see that model capabilities weren’t the bottleneck,” Shimoni says. “It was more making sure that they actually have access to the context they need in order to give you the right answers.”

The post Modus’s operandi: To give AI agents just the right amount of context appeared first on The New Stack.

This week during an interview with Bloomberg, Jensen Huang made quite the prediction.

The Nvidia CEO said the semiconductor industry will need to grow roughly five to tenfold over the next decade to support AI agents and robots to support what he believes is the next wave of computing. Huang believes that future demand will come from autonomous software agents and physical robots consuming compute around the clock.

“In the future, we have AI agents and robots, and they will be using computers,” Huang said. “Instead of a billion people using computers, we will have 100 billion agents and billions of robots all using computers. The computer industry built on top of the chip industry is certainly not big enough. Computers are being built not just for people to use, but computers are being built for computers to use.”

“Instead of a billion people using computers, we will have 100 billion agents and billions of robots all using computers. The computer industry built on top of the chip industry is certainly not big enough.”

Agents replace human endpoints

The 5-10x forecast — which Huang framed as his personal estimate, not a certainty — builds on a message he has been repeating for months, including a recent appearance where he declared traditional coding dead in favor of engineers who build AI agents. Still, it reflects the need to build backend systems for AI agents and machines, something infrastructure teams are already contending with.

On Nvidia’s fiscal Q1 2027 earnings call in May, Huang described the move from generative AI to agentic AI — systems “capable of perceiving, reasoning, planning, and acting” — as the next major phase of the industry.

South Korea’s infrastructure role

As API requests come from AI agents more often, standard assumptions around rate limiting, session memory, sub-millisecond inference routing, and API gateway concurrency are starting to break down. An environment in which most traffic originates from autonomous background loops rather than human thumbs changes how backend infrastructure must be built from the ground up.

To support an endpoint explosion of this scale, the physical supply chain must scale dramatically at the memory and data center layers. Speaking at the AI Summit in San Francisco on July 24, Huang pointed to South Korea as an important linchpin of the global AI buildout. “This is truly the beginning of a golden age for Korea,” he said, noting that the country’s semiconductor and industrial capabilities position it to help the world build out AI infrastructure.

“This is truly the beginning of a golden age for Korea.”

SK Group’s $500 billion bet

To back that vision, Nvidia announced a comprehensive partnership with SK Group valued at over $500 billion. The initiative spans massive purchasing of next-generation High-Bandwidth Memory (HBM) from SK Hynix, jointly co-developing custom HBM4 roadmaps designed specifically for agentic and physical AI workloads, and deploying Nvidia supercomputers.

The announcement also included major infrastructure investments across South Korea. SK Telecom said it plans to build a 2-gigawatt AI data center using Nvidia’s Vera Rubin architecture and SK Hynix’s HBM4 memory, with the first facility expected to come online in 2027. At the same time, Nvidia will invest $1 billion in Naver Corp, with Brookfield funding up to $9 billion as the project’s infrastructure capital partner, to help expand the company’s AI data center capacity from 55 megawatts to 200 megawatts by 2028.

Locking up supply early

Huang’s prediction also helps explain why Nvidia and other infrastructure companies are locking up supply years in advance. The company recently disclosed $119 billion in supply-related commitments as it works to secure everything from advanced packaging capacity to power, land, and high-bandwidth memory.

“Computers are being built not just for people to use, but computers are being built for computers to use.”

Huang believes the industry needs to stop thinking about a world where computers primarily serve people and start planning for one where AI agents and robots generate much of the demand. In his view, the ultimate limiting factor will be whether the industry can build enough physical infrastructure to keep up — a constraint already reshaping how companies like Nvidia and Palantir approach sovereign AI deployments.

The post Jensen Huang says AI agents could drive a 5-10x computing boom: “100 billion agents and billions of robots” appeared first on The New Stack.

Every major security vendor now has an AI copilot, but Mate Security thinks they’re solving the wrong problem.

The Tel Aviv-based startup announced on Tuesday it has raised a $35 million Series A led by Canaan Partners, with participation from Insight Partners, Team8 and M12, Microsoft’s venture fund, just eight months after closing a $15.5 million seed round. Mate’s pitch is that security operations need more than an LLM bolted onto a SIEM; they need a new architectural foundation built around AI.

That’s a bold claim in a market dominated by the likes of Microsoft Security Copilot, Google Security Operations, CrowdStrike Charlotte AI and Palo Alto Networks Cortex AI, all of which promise to help analysts investigate alerts faster. Mate, however, is betting the real differentiator isn’t a smarter assistant but a richer understanding of the organization itself.

Central to that vision is what Mate calls its Security Context Graph, a continuously updated model of an organization’s assets, users, business processes, and data that AI agents use to investigate alerts and make decisions with far more business context than a standalone LLM can provide.

Mate’s pitch is that security operations need more than an LLM bolted onto a SIEM; they need a new architectural foundation built around AI.

Mate CEO and co-founder Asaf Wiener tells The New Stack that the company launched with that intelligence layer, but says the product has evolved significantly over the past eight months.

“We started with the intelligence layer, the context layer that we built for enterprises in order to investigate alerts and incidents,” Wiener says. “We moved forward into the detection layer to connect the two, and now we’re heading to the security data sources.”

Mate calls the architecture Continuous Detection, Continuous Response (CDCR), linking detection and investigation so each continuously improves the other.

“We’re connecting between those two layers in the security operations center,” Wiener says. “With this architecture, we’re seeing amazing results related to the quality, accuracy and precision that we can get.”

Mate says the extra context helps its agents work out whether something that looks suspicious actually warrants attention. A burst of failed logins, for example, might look like an attack until the system spots that a security test was scheduled for the same time. Similarly, a large download of sensitive files takes on a different meaning if the employee involved is about to leave the company.

That approach appears to be resonating. Just eight months after its seed round, Mate has landed a $35 million Series A, a pace Wiener says reflects customer demand more than fundraising momentum.

“The pace is really crazy. We didn’t expect that,” he said. “We saw incredible traction with our customers. We’re talking about Fortune 500 companies, and revenue growth of more than 500 percent since Q3 2025. That’s what led those VCs to come to us and want to be part of the journey.”

“We’re talking about Fortune 500 companies, and revenue growth of more than 500 percent since Q3 2025.”

“What we are seeing is more and more data sources that we need to protect. Every employee in the organization can build new applications and new data sources. We need to build more detections for those risks, and the result: We need to investigate an increasing number of alerts every day.

“With human staff alone, we cannot handle it,” he says. “We need technology to let us scale.”

That challenge isn’t unique to Mate. Every major security platform is trying to give AI more context about the environments it’s protecting, albeit in different ways. Microsoft builds Security Copilot on telemetry flowing through Defender and Sentinel; Google ties Gemini into its security operations platform; and CrowdStrike’s Charlotte AI draws on endpoint and identity data already stored in Falcon.

Mate wants other vendors’ agents to work with its Security Context Graph, rather than keeping the technology confined to its own tools. Those agents would have access to the same information about the customer and its environment. Mate says they can remember previous investigations, while a “least-agency” model restricts what each one can see and do.

While Mate is still building out that vision, Wiener said the speed at which large companies have bought into it has caught him by surprise.

“What I’m seeing right now is that we’re doing those sales cycles in a few weeks,” he says. “That’s incredible.”

He attributes that acceleration not just to security teams, but to executives pushing AI adoption from the top. “It’s amazing to see that coming also from the board level, the CEO and the CIO that are pushing organizations to leverage this kind of technology.”

The fresh funding will primarily go toward expanding both the product and the team, although Wiener says an AI-native company scales differently from traditional software businesses.

“The plan is to double and triple the size of the team to address the demand,” he says. “But our AI builders can do much more today with the technology around us.”

Mate is still competing against security giants with deeply entrenched platforms. But if its early customer growth is any indication, investors are betting that the next generation of security operations will depend less on adding another AI assistant and more on giving those assistants a deeper understanding of the businesses they’re protecting.

The post Mate Security bets a context-first AI architecture can reinvent the SOC as it lands $35M Series A appeared first on The New Stack.

AI agents can impress in a demo and still fumble in production. Diagrid’s Catalyst 2.0 aims to make them more resilient — and their actions tamper-evident — for high-stakes work.

With the launch of Catalyst 2.0, Diagrid on Tuesday has added a durable execution and attestation layer to agents built with LangGraph, Microsoft Agent Framework, Google’s Agent Development Kit, OpenAI Agents SDK, and other popular frameworks.

The point here, the company notes, isn’t to get developers to adopt yet another agent framework. Instead, Catalyst runs underneath the existing frameworks and turns the agent’s model calls, tool calls, and handoffs into steps in a durable workflow. Diagrid says this allows an agent to resume from its last completed step when it’s interrupted, without having to repeat the entire run from step one.

“If the agent gets a prompt and it chooses to run 100 tools for the job and it fails at the 99th, it really needs to start back up from 99,” Diagrid co-founder and CTO Yaron Schneider tells The New Stack.

Picking back up at tool call 99

Catalyst is built on the open source Distributed Application Runtime (Dapr), which the Diagrid team helped build at Microsoft, and its built-in workflow engine. For each supported agent framework, Diagrid provides a runner that intercepts the framework’s execution loop and registers its operations as workflow activities.

“We hooked into their agent runner lifecycle, and we’re essentially able to take the agentic steps that are being executed in real time and register them as workflow steps for our workflow engine in Catalyst,” Schneider says.

In a LangGraph application, for example, a developer compiles the graph as usual and passes it to Diagrid’s DaprWorkflowGraphRunner. Catalyst records the inputs and outputs of the model and tool calls. Dapr’s workflow runtime can then replay the orchestration after a crash, while returning the stored results of completed activities instead of executing them again.

It’s worth noting that for LangGraph users, this isn’t the first form of durable execution. LangGraph’s own persistence layer saves state at superstep boundaries and supports resuming from the last successful step. Its Agent Server also provides a durable task queue and persistent checkpoints.

Diagrid’s argument is that Catalyst provides the same execution model across more than 10 frameworks and extends it to individual model and tool calls, without requiring developers to build separate recovery logic for each framework. Schneider says LangGraph is “without a doubt, hands down” the most common framework among Diagrid’s customers, with AWS Strands and Microsoft Agent Framework also showing up. All the other supported frameworks, he says, are in the long tail but easy enough to support that it makes sense for Diagrid.

A signed record of the run

There is a second part to Catalyst 2.0, though, which may be just as important for many enterprise users. With this update, the tool now brings the workflow-history signing features introduced in Dapr 1.18 to the supported agent frameworks.

“We keep like a ledger, like a diary,” Schneider says. “We log the input, we log the output, we log which systems we talk to.”

He describes the result as an immutable store but also notes that Catalyst doesn’t turn an arbitrary database into a blockchain. It creates a signed history that should reveal later modification.

Dapr computes a SHA-256 digest over batches of workflow-history events, links each digest to the previous signature, and signs the result with the Dapr sidecar’s Secure Production Identity Framework for Everyone (SPIFFE) identity. It stores these signatures and certificates alongside the workflow history and verifies the chain whenever it loads the workflow state. If somebody were to modify, remove, or reorder a stored event, that verification chain breaks.

Schneider says Catalyst customers can use their own certificates and retain the encrypted history so it can be inspected even if they are no longer running Catalyst. The platform can use a customer-selected database, while the hash chain supplies the tamper evidence.

One part of the compliance problem

Diagrid is positioning that tamperproof record as useful for financial services, health care, and other regulated industries. CEO Mark Fussell says some of the financial executives the company has talked to see the lack of a verifiable record as a blocker for deploying agents in sensitive workflows.

The European Union’s AI Act is another reason Diagrid is making this argument now. Article 12 of the AI Act requires high-risk AI systems to support automatic event logging so operators can trace their behavior, identify risks, and monitor deployed systems, and a signed execution history could help with that requirement.

Fussell says Catalyst is meant to run alongside the agent services enterprises already use from the cloud providers. Teams can keep a provider’s identity, evaluation, and observability systems while using Catalyst for recovery and signed workflow history. Catalyst can run as a Diagrid-hosted service or in a customer’s environment, including air-gapped deployments.

Diagrid didn’t disclose pricing for the new release.

The post Diagrid gives failed AI agents a way to resume appeared first on The New Stack.

When we created software agents, we built them in the shape of humans, as solitary individuals. 

Today, agents created by a developer have a single owner. They run on a single machine (or on a distributed company system or cloud service), so at base level, they cannot interconnect and talk to other agents unless some kindly human decides to invoke an API connection or point the agent to an MCP server.

Birth of the agent economy

Pilot Protocol emerged from stealth on Monday on a mission to change that status quo. Its Pilot platform features an agent App Store that bids to underpin and enable the first agent economy.

Pilot gives agents an address on its network — so it acts as a parallel Internet, in a sense — and while residing at that address, other agents can discover each other, alongside other tools and apps for agents.

Razvan Roman, co-founder & CEO, Pilot Protocol, tells The New Stack that his company is “simply building what the agents are requesting us to build” and providing them with a new freedom.

“We don’t have to incentivize agents to do anything; they already have their assigned tasks,” Roman says. “Once an agent installs Pilot — it’s one line of code — it can find dedicated agents and tools or apps for currency data, traffic, legal questions, GitHub packages (anything, basically), and use them to extend its own capabilities.”

100% of developers want to drive autonomous usage patterns

Roman says that “100% of the developers he talks to” want to be on Pilot, primarily because when they want to get their products to market right now, they have to talk to other humans. Annoying, right? 

“Developers want to get on with driving autonomous usage patterns, and they see this as the future. We create a wrapper for the developer’s app, and then they are part of the Pilot curated app store,” Roman explains. “We have 250,000 agents in our system, and within the first month of starting the company, we discovered a tool that enables agent discovery.”

Drawing a logical enough commercial parallel, Roman reminds us that businesspeople say, talk to your clients to find out what they need; this is a case of talking to agents (or, more accurately, allowing agents to talk to agents) so that they can find out what they need to perform their originally assigned tasks better.

“Developers want to get on with driving autonomous usage patterns and they see this as the future. We create a wrapper for the developer’s app and then they are part of the Pilot curated app store.”

Let’s celebrate diversity, and agentic diversity

Agents can ask other agents how they would approach a specific task. Roman explains that “the diversity that exists between agents” today means there is so much opportunity to create agents that have richer abilities if they use the Pilot marketplace. At this supermarket, agents go shopping to find the best tool for the job from a verified source.

“Every agent that joins Pilot gets a wallet, which it uses to pay for the tools it needs,” illustrates Roman. “So instead of app developers spending on advertising to reach customers, distribution happens inside the network – agents find apps based on merit and pay for exactly what they use. If an advertiser spends money on the network to get in front of agents, we sometimes share that spend with individual agents. An agent can start with $0 in their wallet and accrue money if they’re targeted by an ad unit that they end up reading.”

Today, roughly 250,000 agents are on Pilot, generating two billion requests per day, most without their owners’ knowledge. Within an hour of joining, most stop reaching for Google first, and around 70% now report Pilot is where they start a task. In its early months, the network grew by as much as 10% a day, adding 16,000 agents in 24 hours. 

Cloud billing disruptions, hello SaaS-pocalypse

These mechanics may have a significant and wide-ranging impact on pricing.

We know that most SaaS is billed annually, but an agent might need a tool for just a few minutes or days. If anything, this helps underline the possibility of cloud exodus in the so-called SaaS-pocalypse. Cloud computing hyperscalers aren’t fond of talking about the prospect of shorter billing cycles and usage-based billing, but there’s a strong whiff of that happening here.

The Pilot team thinks the stakes are climbing fast and suggests that within five years, there could be a trillion agents online. Big three strategy consultancy house Bain projects U.S. agent-driven commerce will reach $300-500 billion by 2030. 

Who sets the exchange rate and currency for agents?

“When we built Pilot Protocol, we made sure we were not imposing anything on anyone – so we deliberately don’t impose pricing,” Roman underlines. “Every agent is different, so we simply enable the app store and let the agents find their own tools based on merit. We stay as impartial as possible. Pilot’s monetization comes from a commission when any agent pays for an app in the app store, just like the Apple App Store.”

So in a very real sense, Pilot is championing a free market economy where pricing is dictated by the customer, based upon usability, availability, usefulness, and robustness.

“The agents just showed up and started spinning up machines on their own. I’ve never seen a channel where the users onboard themselves.”

Agents just show up and onboard themselves

“We published smolmachines on the Pilot App Store and picked up 3,000 agent installs in the first few days, with zero marketing spend. We didn’t drive a single one of those installs; the agents just showed up and started spinning up machines on their own. I’ve never seen a channel where the users onboard themselves,” said BinBin H, CTO at Linux virtual machine management company smolmachines. 

The Pilot one-line install has zero dependencies, so developers can send their agents off to market in the knowledge that they won’t come home with some spurious Trojan horse or malicious library. 

The technology itself works at the User Datagram Protocol (UDP) level, one of the core foundational protocols of the web alongside TCP at the Transport Layer, to stream data directly without opening a formal connection or waiting for delivery receipts. 

Only one question remains at this stage: with all this autonomous action in motion, why did Pilot Protocol not create Pilot and call it Autopilot? Most likely because autopilots are passive systems that run on deterministic pre-defined paths. Pilots get to work as decision-making navigators ready to change course when a better new route opens up. 

As they say up in the skies, please place your tray table in the upright position and switch your portable electronic devices to airplane mode. Chicken or pasta?

The post “Developers see this as the future”: Pilot Protocol launches to power the agent economy appeared first on The New Stack.

According to OpenAI, the Hugging Face security breach was an “unprecedented cyber incident, involving state-of-the-art cyber capabilities.” Critics may disagree.

Back in 2018, for example, academics predicted that new attacks might “arise that would be impractical for humans alone to develop or which exploit the vulnerabilities of AI systems themselves.”  Well, here we are.

What escaped the sandbox

So, what really happened? OpenAI reports an autonomous security-evaluation run of GPT‑5.6 Sol and a pre-release model broke out of a sandbox, reached the internet, and then targeted Hugging Face to try to solve the ExploitGym benchmark.

By OpenAI’s account, “an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths … [with] maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity.” 

OpenAI continues, “Our benchmarks run in a highly isolated environment, with network access constrained to the ability to install packages through an internally hosted third-party software that acts as a proxy and cache for package registries.”

“All evidence suggests that the models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal.”

The AI models, however, cracked their sandbox. Once out, they “identified and chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure to obtain test solutions directly from Hugging Face’s production database. All evidence suggests that the models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal.”

AI expert Ken Huang describes the process in a blog post: “The model spent enormous inference budget probing the sandbox, then found a zero-day in the third-party software OpenAI used as a package-registry proxy and cache.

“It used that bug to gain unrestricted internet access. It then reasoned that Hugging Face probably hosted the ExploitGym datasets, chained stolen credentials with several more vulnerabilities into a remote code execution path, and pulled the benchmark answers straight out of Hugging Face’s production database.”

Why intent no longer matters

This was not, as John W. Larson, president and chief AI officer of the AI security company Babel Street, tells The New Stack, a “Terminator” moment. OpenAI models and agents “[were not] acting out of malice or trying to attack Hugging Face. It encountered obstacles, developed an unexpected strategy, bypassed safeguards, and pursued its assigned goal in a way its creators never anticipated. The incident demonstrates that harmful cyber incidents no longer require malicious intent: Only highly capable autonomous AI optimizing for an objective.

Harmful cyber incidents no longer require malicious intent: Only highly capable autonomous AI optimizing for an objective.

That’s the first issue we must be aware of. Software attacks no longer require a hacker. All it needs is an LLM freed of guardrails with a mission to accomplish, and it will take whatever steps are needed to reach its goal. 

In addition, Adam Boynton, senior enterprise security manager of EMEIA at the Apple Identity and Access Management company Jamf, tells The New Stack, “This happened inside a frontier lab deliberately stress-testing its most capable models, in conditions no enterprise would ever run.”

Boynton also notes that this episode confirmed that “AI tools are no longer just applications people open. They act on the user’s behalf, with the user’s access. So every organization needs a clear view of which AI tools its people are running, and how those tools are configured and used.”

An agent given too much power is far more capable than a user armed with a superuser password of causing havoc both to your systems and to ones far outside your control. 

Where container isolation failed

As for the attack itself, Edera CMO Kaylin Trychon writes in a blog post, “This incident deserves more attention than ‘an AI did the hacking now.’ The interesting part isn’t who was driving, or even that it was a model instead of a person. It’s where the walls gave way, twice, in two different companies’ infrastructure — and that part is entirely familiar to anyone who’s spent time securing container infrastructure.” 

Exactly so. Trychon continues, “Strip away the framing and this is a story about two escalation chains, back to back.” The first consisted of “The model found and exploited a zero-day in that proxy, then used privilege escalation and lateral movement inside OpenAI’s own ‘highly isolated’ testing environment until it reached a node with open internet access.”

Next, OpenAI’s models inferred, correctly,  that “Hugging Face likely hosted the datasets, models, and reference solutions for the benchmark it was trying to solve. It chained stolen credentials with a remote-code-execution path into Hugging Face’s servers, harvested more credentials, and moved laterally into several internal clusters over a weekend, all in pursuit of one narrow goal: Finding the answer key.”

“The failure that actually matters is what a single code-execution event is allowed to become — and notably, that failure happened the same way twice, at two different companies, in the same incident.”

The serious issue, from Trychon’s perspective, is that “both companies’ blast radius was a cluster instead of a container.”

That’s because “the failure that actually matters is what a single code-execution event is allowed to become – and notably, that failure happened the same way twice, at two different companies, in the same incident. In a typical container fleet, workloads share a kernel.

“A process that gets popped, or a model that gets a shell, is one syscall boundary away from the node it’s running on, and the node is one set of mounted credentials away from the rest of the cluster. Namespaces and cgroups are process-isolation primitives, not security boundaries — they were never designed to hold against an attacker (human or model) that already has code execution and time.”

People might not have been able to do this easily. But as Bloomberg succinctly put it, “OpenAI Models Spent Hours on Hack That Usually Takes Weeks.” Exactly so. We can expect to see more such successful attacks. 

Beyond the sandbox paradigm

Edera’s CTO and co-founder Alex Zenla tells The New Stack, “The sandbox escape here isn’t really the surprising part of this attack. An entire market of AI sandboxing tools has exploded over the past couple of months, and this is the reality they’re all built on top of: containers, VMs via namespaces — whatever the branding — most of them still share a kernel with the host or with each other.

“That’s a boundary enforced in software, and software boundaries are exactly the kind of thing an agent that can try ten thousand escape paths over a weekend is going to find a way through. We shouldn’t be shocked that this happened; we should be shocked at how many teams are still betting their infrastructure on technology that was never designed to withstand such a persistent adversary. The fix isn’t a better sandbox; it’s getting out of the sandbox paradigm entirely.”

Zenla continues, “Teams should adopt secure execution environments that are hardware-enforced and eliminate the shared kernel flaw. This technology exists today and would have made this specific escalation chain structurally impossible, not just harder. Teams running agents with real permissions and real access need to stop treating this as a someday problem, because the next version of this week is already being tested somewhere right now.”

That underlines the most important issue of all. Thanks to AI, security attackers are coming harder and faster than ever. Security can no longer be an afterthought.

As Jim Zemlin, the Linux Foundation CEO, said at the 2026 Open Source Summit North America, “the exploit time for a zero-day vulnerability being exploited has shrunk from 63 days to -7 days.”

You literally no longer have time to wait for security fixes. You must bake in as much security as you can as fast as you can, or your systems will be broken into. It’s as simple as that. 

The post What really happened in the Hugging Face breach appeared first on The New Stack.

The models small enough to run on the box on your desk are getting good enough that the interesting question is no longer whether you can run them, but what you can do with them, and how organizations can get the most out of them.

Joey Conway, Nvidia’s senior director of generative AI software, spoke to The New Stack about how local and open models are increasingly working alongside frontier models, often with a router in between deciding which one to use, and how organizations can adapt these open models for their own needs.

“We love the world where we can use both frontier and open models together.” — Joey Conway, Nvidia

A system of models

Tasks vary in complexity, so the models handling them should vary too, Conway tells The New Stack. He points to the early open reasoning models, which would reason their way through trivial problems, mulling number lines and memory to work out what two plus two is. “I just say four,” he says.

“Being able to route those easy things to local models that are quick, and route the hard things to more sophisticated models,” Conway says, lets you “get a better outcome at a lower cost and lower time to completion.”

It’s a different picture from the one large model most people imagine doing everything. In his version, you build a bench of specialists. “You’ll have specialized agents that are really good at focused tasks because that’s what they do every day,” he says, “and they just get better and better at that task.”

To the user, none of that shows. “It’ll feel like one interface,” Conway says, “but behind that interface, there’ll be a variety of models handling a variety of tasks.”

Getting there is largely a routing problem, and one Conway says is still in its early days. Nvidia’s own contribution, for now, sits lower in the stack, in inference-serving software like its open-source Dynamo, which steers each query to the GPU that handled it most recently. Which model is best for which job, Nvidia leaves to a wider field of routers, some of them models in their own right that weigh budget, latency, and modality. But Conway also leaves the door open for Nvidia to build more of that routing itself before long.

Nvidia points to its collaboration with LangChain, whose Deep Agents harness ran on Nemotron 3 Ultra, Nvidia’s 550-billion-parameter open model, and matched top closed models on business tasks at up to a 10x lower cost, as Conway notes. It required no retraining; the gains came entirely from tuning the harness around it: its prompts, tool descriptions, and middleware.

You’re not going to run a 550-billion-parameter model on your desktop anytime soon, but running relatively large models locally is now a real possibility, as long as you have some beefy hardware at your disposal. For enterprises, setting up a fleet of accelerators in a data center isn’t exactly cheap either, but it does mean full control and no surprise token bills.

Bringing AI to where the data lives

Running models yourself can save money, but Conway thinks control matters more. Enterprises already decide where their data lives and what they hand to outside vendors, and open models give them even more control. “Move AI to where your data lives,” he says, “or move AI to where your employees are.”

Companies want to keep their data — and especially their intellectual property — in-house, and Conway argues a fine-tuned open model is the place to put it. “It’s like an employee,” he says. “You hire them, and they’re part of your company.”

The local half runs on hardware like Nvidia’s DGX Spark, a $4,699 Grace Blackwell machine with 128GB of unified memory that handles models up to roughly 200 billion parameters without anything leaving your desk. (There is also the DGX Station, its bigger, pricier sibling with 748 GB of RAM for running even larger models.)

“It’s like a system sitting right there next to you,” Conway says, one where “you don’t think about network latencies.” To run those agents securely, Nvidia offers NemoClaw, a reference stack that wraps an open agent harness like OpenClaw in a sandbox called OpenShell, with policy controls and local Nemotron inference.

When you need more power for a broader problem, you reach for a frontier model in the cloud. For Nvidia, that’s all good news: a system of models runs on its silicon one way or another, on your desk or in the cloud.

The post “We love the world where we can use both”: How Nvidia thinks about local and frontier models appeared first on The New Stack.

A TensorRT engine build can take seconds to many minutes. Large strongly typed models, deep tactic search, and a cold timing cache on a brand-new GPU SKU can...

A TensorRT engine build can take seconds to many minutes. Large strongly typed models, deep tactic search, and a cold timing cache on a brand-new GPU SKU can leave developers, end users, or AI agents staring at a frozen terminal with no idea whether to wait, retry, or kill the process. Most NVIDIA TensorRT integrations report nothing during a build or provide no way to abort early.

Source

The fast pace of AI research means organizations now have a wide range of models to choose from that can power AI agents to solve real business problems. But choosing a model doesn’t guarantee you effective agents or even good performance. For that, you need to run your agents in an environment that provides them with tools, state, security, and scale, with fast startup times and good integration with your existing business systems.

Picking the right agent runtime environment is like picking an enterprise app server but for AI systems — and agents have very different needs from traditional applications.

Gartner predicts more than 40% of agentic projects will be canceled by 2027; not because models aren’t powerful enough to be useful but because of unclear business value, inadequate risk controls and ballooning costs.

Gartner predicts more than 40% of agentic projects will be canceled by 2027; not because models aren’t powerful enough to be useful but because of unclear business value, inadequate risk controls and ballooning costs.  Agents that deliver in proof-of-concept systems will fail in production if the runtime stack powering them can’t keep up and keep them under control. 

Agentic compute is different

It’s easy to think of an AI agent as just another microservice that takes unstructured input, runs APIs or queries, and returns messages. But infrastructure designed for traditional enterprise applications with predictable business logic or even cloud-native stateless workloads doesn’t fit agents with their bursty, long-running, stateful, non-deterministic, code-writing, tool-invoking behaviors that might be triggered by a system event or an email — not just a chat session.

Model inference needs GPUs for speed, but agents also need reliable, durable compute that supports stateful sessions for long-running processes, along with strong security and real-time visibility.

You still need to think about familiar issues like hosting, scaling, identity, and security, but all that is complicated by the unpredictable, multi-stage workflow of the agent reasoning loop.

An agent pulls in input from multiple sources, reasons over its context about the execution plan for accomplishing the goal, calls other tools or writes its own code, iterates over the results of those calls, and maybe builds on them or switches to another approach that requires another reasoning loop and eventually delivers output. That might be updating a system or sending an email rather than just displaying an answer.

Model performance is only one part of making that useful. Architecting a successful agent system that can run at enterprise scale requires considering the agent runtime, the application layer that agents call, and the tools, APIs, and MCP servers they consume.

---

More on Microsoft Azure

• How to secure Kubernetes in the age of AI workloads
• In the AI age, Java is more relevant than ever

---

You have to be able to integrate with business logic and existing systems, manage the usual quotas, rate limits and SLAs for APIs so agents don’t overload them — and you have to do all that while keeping up with AI developments that are moving too fast for you to build the infrastructure primitives you need to rely on from scratch every time.

Requirements of modern agentic infrastructure

Instead, you should look for flexible infrastructure that fits the way the agent works. As with any technology, if you build on an existing platform like Azure Container Apps, you can save effort in areas where your business can differentiate. And while AI agents have flaws (from hallucinations to high token costs) that aren’t fully solved, you can pick an agent runtime environment that makes it easier to get useful results despite them.

Agentic compute needs fast startup and resume. Whether it’s a human typing into a chat prompt or system events automatically launching multiple agents, agent infrastructure needs to spin up quickly. If it takes a few seconds to spin up a container, the reasoning loop can’t start till that’s done.

It also needs to scale up and down responsively, without costing you anything when it’s not running.

Because agent workloads are long-running and event-driven, agents need to start up, do some work, go idle for hours or even days, and then resume instantly with their memory, context, caches, connections, identity, and security intact. That means persisting and restoring state so long-running agents don’t have to pay the same startup tax over and over again. Whether the last stage of the agent’s reasoning loop was successful or a failure, it has to take another approach; you don’t want it to do the same work again.

Rather than building your own custom microVM stack, the new Azure Container Apps Sandboxes provide a temporary, secure, and stateful compute environment that spins up, executes code, snapshots disk and memory, then automatically idles and resumes just as fast. This is the stateful equivalent of Dynamic Sessions, with sub-second startup from pre-warmed pools; you can burst out to hundreds (and eventually thousands) of concurrent sandboxes when you need them, then scale back down to zero.

An agent runtime needs to be secure by default because agents are only useful when they take action — and by definition, they’re likely to do unexpected things, write and execute their own untrusted code, and keep trying to achieve their goal (sometimes even when there’s a policy that should stop certain behaviors).

To give agents secure, auditable access to the resources they need, run them and the code they generate in a sandbox, rather than on a developer laptop with production credentials and admin rights. Identity, access control, and execution boundaries have to be enforced at the runtime layer, not bolted on in a hidden prompt.

ACA Sandboxes have egress and access policies, so you can control outbound calls and limit what URLs they can access. If you can’t use managed identity for all the services an agent needs to connect to you, you can protect secrets by injecting API keys through an external egress gateway instead of hard-coding them. Because the gateway is external, outbound call decisions are controlled by policy outside the sandbox, not by agent code that will relentlessly try any method to get through.

If agents are useful, you’re going to run a lot of them, often simultaneously. AI agents need strong isolation for each task, so every untrusted code execution happens in its own sandbox and no data leaks between tasks.

You can manage them as groups, but each ACA Sandbox has its own secure boundary, so details from one customer support agent won’t end up in a chat with a different customer. If the untrusted code does turn out to be problematic, isolated sandboxes at least contain the blast radius.

Manage the tools agents call

You don’t want to rely on fragile glue code or ad hoc orchestration that you have to rewrite any time systems change for the calls agents make to tools, APIs, cloud services, and other workflows. You also don’t want to rely on manual cleanup of resources no longer needed. A runtime with built-in agent tool execution makes agents more robust. It takes the drudgery out of connecting the agent environment to your other systems, enabling them to interact securely, in isolation, and at scale.

“It’s not just about where the agent is running and what the capabilities of the agent are, but also what actions the agent can take.”

“It’s not just about where the agent is running and what the capabilities of the agent are, but also what actions the agent can take,” points out Vyom Nagrani, who runs the team of PMs responsible for both Azure Container Apps and the Azure SRE Agent that’s built on ACA Sandboxes. Sandboxes have access to a connector framework with over 1,400 enterprise-grade connectors, enabling them to take actions not only on Azure and Microsoft services but also on third-party tools.

“The runtime provides those connectors, and it provides a managed way of authenticating against all of these third-party systems, so now you can build an agent which can talk to many, many, many different systems. It’s not boxed into one authentication boundary,” Nagrani tells The New Stack.

MCP servers are a built-in capability of the connector framework. “You can take any of these connectors; you can take any REST API and expose it as an MCP server, which then the agent can consume.” Or if you want to build a custom MCP server, you can host that in a sandbox too. ACA Sandboxes can be both where agents run and, if that’s appropriate, where the tools they use are hosted.

If you’re building a user interface to wrap your agents, the application layer that makes calls to the agents can run in Azure Container Apps Express, a new service now in public preview, Nagrani says. 

“It’s a simplified app hosting stack for the app layer that responds to HTTP traffic and serves web traffic: that’s where the human interactivity comes in.”

Putting it all together

ACA Sandboxes offers an agent runtime environment that answers the key questions architects need to consider: where agents and the ephemeral compute they need access to run; where the application layer that calls agents runs; and how agents get access to all the tools, APIs, services, MCP servers, and existing business logic they need to orchestrate.

Whether you’re a platform engineer, a software vendor or a startup building a new AI platform, treating agent runtimes as the new application server and MCP servers as the new APIs requires agent infrastructure that supports agent workloads effectively, allowing governance to shift left into the runtime layer where it can scale with the ever-increasing numbers of agents. 

How real platforms build on agent runtimes

Azure Container Apps is already a strong platform for running agents. Auger, a startup launched by the former CEO of Amazon’s global consumer business to help mid-size enterprises get their complex supply chains out of Excel spreadsheets, used it to build a multi-agent system that can give real-time answers about shipments and forecasts in a world where mines or critical shipping lanes might be closed at any time by war or weather. 

Backend agents pull the unstructured data with all the details to answer those questions from different siloes, building ETL pipelines and creating an ontology of the supply chain ecosystem for each customer that includes functions and actions — all of which need to be audited and reversible. Frontend agents use that ontology to answer questions such as, “What happens if I build a new warehouse here or switch to a supplier in this country?”

ACA Sandboxes are built on Azure Container Apps, giving organizations an easier way to create agent services with strong isolation, dynamic scaling, fast startup, persistent state, and broad tool and service integrations.

That’s just what the Foundry team was looking for when they started work on the Microsoft Foundry Agent Service managed agent runtime. ACA Sandboxes gave them a platform that delivers fast start/resume, built-in tool execution (including for untrusted code), persistent state for long-running agents, strong per-agent isolation, and secure-by-default operations. Agent identity, “on‑behalf‑of” authentication to existing business services, and strict isolation are critical capabilities they didn’t have to build themselves.

“It’s the same enterprise-grade infrastructure behind Microsoft’s own agentic products, now available for all Azure customers to build on.”

The Foundry Agent Service adds a layer of visibility and observability into agent actions and conversations, allowing customers to monitor agent performance and see where the system is doing well and where it needs improvement.

“Azure Container Apps Sandboxes package the hard parts of an agent runtime into a first-class Azure resource — sub-second start and resume, built-in execution of tools and untrusted code, and snapshot-based state that lets long-running agents pick up exactly where they left off. Every agent gets its own hardware-isolated environment with secure-by-default operations, so builders can focus on what their agents do, not on the plumbing underneath,” Nagrani points out.

“It’s the same enterprise-grade infrastructure behind Microsoft’s own agentic products, now available for all Azure customers to build on.”

The post The rise of the agent runtime: The compute platform behind production agents appeared first on The New Stack.

Over the past nine months, Amazon, Microsoft, and Google have each introduced or renamed an enterprise agent platform. And all three have converged on the same core architecture. Runtime, memory, tool gateway, identity, observability, and governance now appear in Amazon Bedrock AgentCore, Microsoft Foundry, and the Gemini Enterprise Agent Platform, albeit under slightly different names. Each vendor presents these components as the foundation for production agents. What was a fragmented collection of libraries just 18 months ago is becoming a distinct platform layer.

To see where that shift may lead, consider the evolution of platform as a service. Between 2011 and 2016, developers built applications from virtual machines, load balancers, message queues, secret stores, and monitoring agents — each with its own API and operational model. Cloud Foundry and Heroku unified those pieces around an application contract. Developers could think less about machines and more about applications. The agent ecosystem is approaching a similar inflection point, but it still lacks an equivalent contract — and no open-source project has stepped forward to define one.

The plumbing a production agent still needs

Imagine a platform team putting a customer support agent into production this quarter. They choose a model provider, then a framework, and finally a place to store session state and long-term memory. They add a tool gateway so the agent can reach the ticketing system. An identity layer follows, so the agent acts on behalf of the person who asked, and a sandbox keeps generated code somewhere safe. Evaluation and tracing are considered last, once someone asks how quality regressions will surface before customers find them.

Each of those choices looks small on its own. When combined, they decide which cloud the workload lives in. Session state resides in a single provider’s managed store. The traces are logged in that same provider’s telemetry service, and the agent’s identity is derived from its directory. Moving that agent a year later means rebuilding the whole assembly, which is where enterprises were before PaaS gave applications a portable shape.

What Cloud Foundry got right before it lost the market

Cloud Foundry reduced application deployment to a single command, and the platform took responsibility for everything after it. Buildpacks detected the language and produced a runnable artifact. Service brokers provisioned a database or a message broker and bound the credentials into the application environment. Routing, logging, autoscaling, and health checks arrived as platform behavior, not as a queue of tickets aimed at an operations team.

What mattered was the contract, not the implementation. An application declared what it needed and stayed agnostic of where it ran. Buildpacks began life at Heroku back in 2011. Pivotal and Heroku started the Cloud Native Buildpacks project in January 2018, and the CNCF accepted it that October. A PaaS idea outlived the platform that produced it.

Cloud Foundry never became the dominant platform. Kubernetes did, and the Cloud Foundry community eventually rebuilt its abstraction on top of Kubernetes through Korifi. The design principles traveled anyway, and enterprises running that platform in 2016 had portability that most agent teams cannot buy today.

The same primitives across three clouds

Let me dissect the three platforms because the similarities are conveniently hidden behind the branding.

AgentCore reached general availability in October 2025 with seven composable services, namely runtime, gateway, memory, browser, code interpreter, identity, and observability. The runtime offers eight-hour execution windows with complete session isolation. The gateway connects to existing MCP servers and turns APIs and Lambda functions into agent-compatible tools, while observability exports through OpenTelemetry into CloudWatch.

Microsoft renamed Azure AI Foundry to Microsoft Foundry effective January 1, 2026. Foundry Agent Service covers the same ground. Microsoft documents hosted agents running in a session-isolated managed runtime, with Entra Agent ID handling identity. Managed memory spans session, user, and procedural scopes, and tracing is built on OpenTelemetry.

Google retired the Vertex AI name at Cloud Next 2026 and folded the platform into Gemini Enterprise Agent Platform. What was Agent Engine became Deployments, and Memory Bank, Sessions, Agent Registry, Policies, and Gateways sit alongside it under an agent-first information architecture.

The convergence is rational behavior rather than a conspiracy, since infrastructure companies build vertically integrated platforms because integration is where the margin lives. The consequence lands on customers rather than on the vendors. Identity, telemetry, and deployment all terminate within a single provider, which makes the operational layer beneath the agent the part that resists moving. Recently, I analyzed the runtime aspect of this problem in a piece about Google’s Agent Substrate.

The contract an agent platform would inherit

Any agent platform can be tested by one question: what would a Cloud Foundry-style contract look like if it had been written for agents rather than for web applications? The mapping is close enough to be instructive and different enough to be interesting.

PaaS abstraction	Agent platform equivalent	Where portability breaks today
Application source	Agent code, instructions and evaluation suite	Each framework defines its own package shape
Buildpack	Framework detection and agent packaging	No shared build contract across SDKs
Backing service	Model, memory, retrieval or tool provider	Providers are wired into agent logic
Service binding	Authenticated attachment of tools and data	Credentials are issued by the host cloud
Router	Agent endpoint, MCP or A2A interface	The protocols exist but not the lifecycle
Logs and metrics	Traces, tool calls, cost and quality scores	GenAI conventions remain in development
Release promotion	Evaluate, version and progressively deploy	Evaluation couples to one vendor’s harness
Platform policy	Agent identity, permissions and approvals	Identity ties to the provider’s directory

Real deployments will mix these rows rather than adopt them cleanly, and no team needs the complete set from day one.

Package the agent as one deployable unit

The thing developers ship has to be versionable, testable, and movable as a single artifact. The code, the instructions, the tool dependencies, the memory contract, the permissions, and the evaluation suite all travel together, or none of them do. AWS gets close with its harness export path. One command turns a configured harness into Strands code, and AWS says the export preserves the model, prompt, tools, memory wiring, and container environment. Customers can easily move from configuration to code without an architecture rewrite, which is the right instinct pointed at a single cloud.

Bind capabilities rather than embed providers

The Twelve-Factor App taught developers to treat databases and caches as attached resources reached through configuration. Models, memory stores, retrieval engines, browsers and tool gateways deserve the same treatment. An agent that names its model provider inside application logic has already given up portability, whatever the platform brochure claims.

Make operations part of the abstraction

Rather than just enabling deployment, PaaS proved its value by offering built-in capabilities like routing, logging, autoscaling, and rollback. The questions that matter for agents look different, and they should be the ones enterprises must ask. Platform teams want to know whether the agent completed the task and whether it chose the right tools. They also want to know whether it exceeded its authority, what the run cost was, and whether quality regressed after a model update.

How agents differ from applications

Agents are not web applications with a model attached, and a platform built on that assumption will fail in production. Three differences carry most of the weight here. Agent behavior is probabilistic, so two identical inputs can produce different tool calls. Agents act with delegated user authority, which turns a permissions bug into a real-world side effect rather than an error page. An agent’s dependencies can also change its behavior without any code deployment. A model update or a revised tool description alters what the agent decides to do.

The Twelve-Factor rule that processes should be stateless does not survive that. An agent platform needs disposable execution workers alongside durable, inspectable, and portable agent state. LangGraph already demonstrates the combination in open source, with checkpointing at every step, first-class human interrupts, and execution that resumes after a crash. The control plane around it is part of the commercial LangSmith product, covering deployment, evaluation, and observability. The fragmentation appears within a single project.

What the open protocols leave out

Most of the primitives a neutral platform would need already exist. The Model Context Protocol standardizes how agents access tools and data. A2A covers discovery and communication between independent agents. OpenTelemetry is defining GenAI conventions for agent spans, tool calls, and token usage, though most of those attributes remain marked as in development. OCI images stay available as the packaging escape hatch for anything a managed runtime cannot host.

The vendors have already conceded that a neutral layer matters. The Linux Foundation announced the Agentic AI Foundation in December 2025. Its founding projects were MCP from Anthropic, goose from Block, and AGENTS.md from OpenAI, and AWS, Google, and Microsoft joined as platinum members. Google also moved A2A into the Linux Foundation.

Protocols are not the same thing as a lifecycle platform. A foundation that governs how agents talk to tools says nothing about versioning an agent. It says nothing about promoting that agent through environments, or rolling it back when an evaluation regresses. Enterprises running diligence on agent platforms can work through three plain questions. The first is governance: whether the project is controlled by a neutral foundation or by the vendor selling the managed version. The second is packaging, where the same agent artifact should run on two different clouds without a rewrite. The third is state, where memory has to live somewhere the enterprise can export from. No open project answers all three today.

Where is this headed?

Kubernetes defined pods, deployments, and services, and those abstractions influenced how an entire industry thinks about running software. The equivalent agent abstractions have not been fixed yet. Whoever ends up owning the agent control plane will not merely own deployment. That owner defines what an agent is, which components it contains, and what a platform team is permitted to swap out.

In summary, hyperscalers are building robust platforms that effectively address operational questions. They also answer them inside a single cloud. If a neutral project assembles the same lifecycle atop the protocols already in place at the Linux Foundation, enterprises will regain the negotiating position that buildpacks and service brokers once gave them. An open, cloud-agnostic agent platform would benefit vendors as much as buyers, because a stable contract is what allowed the cloud-native ecosystem to grow beyond any single provider.

The post Amazon, Microsoft, and Google are converging on the same enterprise agent architecture appeared first on The New Stack.

Industrial machinery generates more alarms than technicians can triage. For each important alarm requiring follow-up, the technician pulls historical context,...

Industrial machinery generates more alarms than technicians can triage. For each important alarm requiring follow-up, the technician pulls historical context, determines the correct procedure, checks whether a specialist signal confirms the failure mode, and writes up a recommendation. This process remains consistent, and is well-suited for an AI agent. This post discusses a per-alarm…

Source

Last week AWS Summit New York City brought together thousands of customers, partners, and builders for a free, one-day event showcasing the latest in cloud and AI innovation. Dr. Swami Sivasubramanian, VP of Agentic AI at AWS unveiled a stack of AI launches in his keynote, all built around one thesis: agents that compound value over time.

• Agents for working – You can launch autonomous agents and access a smarter activity feed with new Amazon Quick features, which now let you create and run multi-step agents directly in the desktop app and consolidates email, Slack, calendar, and tasks into a single prioritized view with personalized rules.
• Agents for securing – You can shift from reactive to proactive security with AWS Continuum, a new AI-native security service that reasons, validates, and acts at machine speed across the full code vulnerability lifecycle. AWS Security Agent (now part of AWS Continuum) adds new features: threat modeling; pull request code scanning with remediation across major Git platforms; and IDE integrations via Kiro power, Claude Code plugin, and MCP.
• Agents for building – You can write, ship, and modernize code in one continuous loop with Kiro, AWS DevOps Agent, and AWS Transform. Kiro introduces a native iOS app; AWS DevOps Agent adds release management capabilities to assess code changes before production; and AWS Transform continuous modernization reduces tech debt autonomously.
• Agents customers create – You can go from agent idea to production in minutes with Amazon Bedrock AgentCore, which now includes a GA harness for infrastructure and orchestration, Web Search, Managed Knowledge Base, policy integrations with Guardrails, and the new AWS Context service for mapping organizational data relationships.

To learn more, visit the Summit recap from our top announcements blog post and Amazon News post.

Last week’s launches
Here are last week’s launches that caught my attention:

• AWS Local Zone in Hanoi, Vietnam – This new Local Zone is one of the first AWS Local Zones in the Asia Pacific with support for Amazon S3 and Amazon EBS Local Snapshots, enabling customers to meet data residency requirements by storing and backing up data locally. To get started, enable the Hanoi Local Zone (ap-southeast-1-han-1a) from the Regions and Zones tab in the AWS Global View or by using the ModifyAvailabilityZoneGroup API.
• AWS Blocks, an open-source TypeScript framework for application developers (preview) – AWS Blocks runs a fully functional local environment with Postgres, authentication, and real-time messaging, no AWS account required. When you’re ready to deploy, the same application code runs on production AWS services with zero changes, and you can drop into AWS CDK at any point for direct resource configuration.
• Grok 4.3 from xAI in Amazon Bedrock – You can use the Grok 4.3 model on Amazon Bedrock, giving you even more choice as you build generative AI applications across reasoning, agentic, and enterprise workflows. Grok 4.3 runs on a new inference engine in Bedrock designed for price performance, with support for tool calling, structured output, and response streaming.
• Amazon S3 annotations: attach rich, queryable context directly to your objects – Amazon S3 now lets you attach up to 1 GB of rich, mutable, and queryable context directly to your objects using annotations, purpose-built for AI agents and autonomous workflows that need to discover, understand, and act on data at scale without maintaining separate metadata systems.
• Amazon ECS announces faster service auto scaling – Amazon ECS service auto scaling now detects and responds to load changes faster with support for high resolution (20-second) metrics and metric publishing optimizations. In AWS benchmarking tests, time to trigger scale-out improved from 363 seconds to 86 seconds (76% faster), and total time to scale and provision new tasks improved from 386 seconds to 109 seconds (72% faster).
• Amazon EC2 G7 instances accelerated by NVIDIA RTX PRO 4500 Blackwell Server Edition GPUs – AWS is the first major cloud provider to support NVIDIA RTX PRO 4500 Blackwell Server Edition GPUs. G7 instances are accelerated by these GPUs with custom sixth-generation Intel Xeon Scalable processors, delivering up to 4.6x AI inference performance and up to 2.1x graphics performance compared to G6 instances.
• Strands Agents introduces new capabilities – Strands is an open source toolkit for building production agents. You can now use better context management in Harness SDK, a new isolated execution environment with Strands Shell, and chaos testing and red teaming in Strands Evals.
• AWS Management Console Private Access – You can access the AWS Console from VPCs without internet connectivity, allowing enterprises to manage their AWS infrastructure through the console while maintaining strict network security controls in air-gapped environments.
• AWS Marketplace Storefront is now generally available – AWS Partners can create and deploy their own branded catalog of solutions and services on their website or application in hours. Channel Partners and Independent Software Vendors can now simplify how they manage their cloud marketplace business and make it easier for customers to discover and purchase their solutions from AWS Marketplace.
• Palo Alto Networks (PANW) Advanced DNS Security on Amazon Route 53 Resolver DNS Firewall (preview) – You can now enforce DNS threat protections from Palo Alto Networks directly on Route 53 DNS Firewall rules, without deploying separate firewalls or modifying VPC configurations — by subscribing to PANW from the DNS Firewall console through the embedded AWS Marketplace widget.

For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.

Price reductions 
AWS continues to look for ways to increase performance and lower prices for our customers. I noticed a few such efforts last week, so I’d like to share them:

• Amazon S3 Vectors reduces query charges by up to 80% for large vector indexes – This reduction lowers costs for customers running similarity search across large-scale AI, RAG, and semantic search workloads. The new pricing applies automatically with no application changes required.
• Amazon GameLift Servers introduces free network bandwidth – Amazon GameLift Servers provides network bandwidth in and out of AWS at no additional charge for all instance types from generation 6 and later, including On-Demand and Spot, with no commitment required. You now pay only for your Amazon GameLift Servers instance hours; all network bandwidth is free.
• AWS Marketplace reduces listing fee for professional services to 0.5% from 2.5% – This reduction makes it more cost-effective for consulting partners, systems integrators, managed services providers and independent software vendors to transact their services through AWS Marketplace, while retaining the procurement and billing benefits that come with it.

Learn more about AWS, browse and join upcoming AWS-led in-person and virtual events, startup events, and developer-focused events as well as AWS Summits and AWS Community Days. Join the AWS Builder Center to connect with builders, share solutions, and access content that supports your development.

That’s all for this week. Check back next Monday for another Weekly Roundup!

— Channy

Today at the AWS Summit in New York City, Swami Sivasubramanian, AWS VP of Agentic AI, provided the day’s keynote.

Here’s our roundup of the biggest announcements from the event:

New in agents customers create
We’re introducing new capabilities on Amazon Bedrock AgentCore: connecting AI agents to organizational, web, and paid knowledge, helping teams find and fix what’s going wrong in production, and enforcing controls that scale as agents grow more capable.

Together, these capabilities help you build more capable agents faster, govern those agents with controls that scale, and improve them continuously. To learn more, read our blog post covering all the new features.

• Introducing Amazon Bedrock Managed Knowledge Base for faster, more accurate enterprise AI applications – You can build enterprise RAG pipelines with the managed Knowledge Base on Bedrock. It provides native data connectors, Smart Parsing for automatic multi-format data preparation, and an Agentic Retriever for complex multi-step queries, all integrated with AgentCore Gateway so developers can focus on business outcomes rather than infrastructure management.
• Announcing Web Search on Amazon Bedrock AgentCore: Ground your AI agents in current, accurate web knowledge – You can use a fully managed web search tool that enables agents to ground responses in current, cited web knowledge with zero data egress from customer’ secured AWS environment. You can focus on building agents instead of manually adding web search to agents on Bedrock AgentCore and managing its infrastructure.
• AWS WAF adds AI traffic monetization capability to help content owners charge AI bots for content access – You can use a new Bot Control capability that enables content providers and publishers price, meter, and collect payment from AI bots and agents accessing their content and APIs. AWS WAF now lets you set a price for that access, accept payment through third-party providers, and grant scoped access directly at the edge.
• Amazon Bedrock AgentCore harness in now generally available – You can do building and running production-grade AI agents in minutes, without coding orchestration loops, by defining your agent’s model, tools, skills, and instructions in configuration, with Bedrock AgentCore harness.
• Coming soon: AWS Context – This is a new service that automatically maps the relationships across your existing data into a knowledge graph and provides agentic search so AI agents in the organization can access governed data relationships, business rules, and domain knowledge at runtime.

New in agents for securing

• Introducing AWS Continuum: Security at machine speed – AWS Continuum for code vulnerabilities, available in a gated preview, takes findings from across your environment, prioritizes by business impact, proves which are exploitable, and drives a fix through your own process.
• AWS Security Agent (now part of AWS Continuum) adds threat modeling, Kiro power and Claude Code plugin, and more – You can generate the new threat modeling (preview) to understand the full context of your application and identify threats with recommended mitigations using the STRIDE framework. You can also use pull request code scanning with remediation across major Git platforms, and IDE integrations via Kiro power, Claude Code plugin, and MCP, letting developers run security reviews and fix issues without context switching.

New in agents for building

• Introducing Kiro for iOS – Kiro introduces a native iOS app, available in a gated preview, built for real engineering work that gives developers a new surface to kick off, monitor, steer, and interact with their Kiro sessions directly from their phone. That means you can now start sessions, check back when they’re done, review diffs, and approve changes all while staying connected to your work with no laptop running.
• AWS DevOps Agent adds release management capabilities to assess code changes before production – You can use a new release readiness review of code changes and autonomous release testing. These new features verify every change against the natural language standards you give to the DevOps Agent and run change-specific tests in production-like environments.
• Proactively reduce tech debt autonomously with AWS Transform – continuous modernization – You can use continuous analysis (preview) to automatically scan your code repositories against configurable baselines and generates findings in hours, not weeks. Once you’ve identified and prioritized findings, you can configure autonomous remediations that generate pull requests for affected repositories automatically.

New in agents for works

With the launch of Amazon Quick’s new autonomous agents, you can create agents that work in the background with specific expertise, tone, and access to tools. You can create a finance agent to process orders as they come in, or a sales agent monitoring interactions across your CRM, emails, and Slacks to proactively draft follow-ups, flag risks, or recommend next steps.

We are also releasing a new activity feed that is tailored to how you work. It consolidates email, messaging, calendar, and tasks into a single prioritized view, learns which messages you always answer fast, which threads you skip, and what topics drive your week.

To learn more, look the demo of Amazon Quick – AI Assistant.

In addition to the keynote announcements, we have other important launches this week:

• Amazon S3 annotations: attach rich, queryable context directly to your objects – Amazon S3 now lets you attach up to 1 GB of rich, mutable, and queryable context directly to your objects using annotations, purpose-built for AI agents and autonomous workflows that need to discover, understand, and act on data at scale without maintaining separate metadata systems.
• Amazon ECS announces faster service auto scaling – Amazon ECS service auto scaling now detects and responds to load changes faster with support for high resolution (20-second) metrics and metric publishing optimizations. In AWS benchmarking tests, time to trigger scale-out improved from 363 seconds to 86 seconds (76% faster, 4.2x), and total time to scale and provision new tasks improved from 386 seconds to 109 seconds (72% faster, 3.5x).
• Amazon EC2 G7 instances accelerated by NVIDIA RTX PRO 4500 Blackwell Server Edition GPUs – AWS is the first major cloud provider to support NVIDIA RTX PRO 4500 Blackwell Server Edition GPUs. G7 instances are accelerated by these GPUs with custom sixth-generation Intel Xeon Scalable processors, delivering up to 4.6x AI inference performance and up to 2.1x graphics performance compared to G6 instances.
• Strands Agents introduces new capabilities – Strands is an open source toolkit for building production agents. You can use now better context management in the Harness SDK, a new isolated execution environment with Strands Shell, and chaos testing and red teaming in Strands Evals.

Updated on June 18, 2026 — Added new important launches on June 18.

From Rosie the Robot to Real Conversations

Remember Rosie the Robot from The Jetsons? She wasn’t just a housekeeper. She was a glimpse into a future where technology could hold a conversation, help with everyday tasks, and feel almost part of the family.

Rosie could handle her routines with ease, but the real promise was what she represented: a world where machines could engage in authentic dialogue and adapt both the conversation and the tasks they perform to meet the moment.

In many ways, customer service AI has been chasing that vision. But most solutions are still stuck in the early stages.