Multi-tenancy has moved in one direction for 60 years: the tenant keeps getting smaller. Mainframe time-sharing carved a single machine into slices so an organization’s departments could share it, and the tenant was the org. Virtualization gave each team its own fleet of virtual machines, and the tenant became the team. Containers and Kubernetes namespaces shrank it again, until a platform team could hand every developer an isolated environment on a shared cluster.
That last step, an environment per developer, became the target state of platform engineering in the 2020s. A namespace per developer, capacity planned by seat, golden paths sized to headcount. Underneath all of it sits one assumption: a person produces one stream of work at a time, so isolating people isolates work.
Coding agents broke that assumption. A developer running five agent sessions has five changes in flight at once, each needing its own working version of the system. Anthropic’s engineers, building a C compiler with a fleet of parallel agents, ran nearly 2,000 Claude Code sessions across two weeks. Cursor’s documentation tells developers to run as many agents as you want in parallel. None of those concurrent workstreams is a person.
“The tenant has shrunk one more time. It is no longer the developer. It is the change.”
The tenant has shrunk one more time. It is no longer the developer (or even the agent). It is the change.
Capacity planning by seat worked because changes arrived at human pace, roughly one per developer at a time. That denominator is gone. A Microsoft study of command-line coding agent adoption found that developers merged roughly 24% more pull requests over four months, and merged pull requests understate the pressure. Every change that reaches merge is preceded by iterations and abandoned attempts, and each of those also needed somewhere to run.
Run the seat math against the change math. A 50-developer organization where each engineer supervises a few agent sessions has hundreds of changes in some stage of validation on a busy day. Each one wants data it can migrate and write to without asking permission, its own view of shared message topics, and a running version of the services it touched. That is the demand of a 300-person (or more) engineering org on a 50-tenant platform.
Every layer built on the person-tenant assumption misprices this. A per-developer namespace hands one tenant slot to what is now five concurrent workstreams. Shared staging serializes all of them into a single queue. Seat-based capacity plans budget for the number of employees while the bill tracks the number of changes in flight.
The tempting candidate for the new tenant is the agent, and it is the wrong one. Agents are interchangeable workers. Two agents can collaborate on one change, one agent can rotate through five changes, and a crashed agent gets replaced mid-task without anything downstream noticing. Give each agent its own environment, and you have repeated the old mistake at a new scale: isolating workers when the thing that must not leak is work.
“Give each agent its own environment, and you have repeated the old mistake at a new scale: isolating workers when the thing that must not leak is work.”
The durable unit is the change. It comes into existence when work on it starts. It accumulates state that no other tenant should see: a schema migration, test writes, new versions of one or two services, the messages it produced during validation. It needs to observe a version of the system that includes its own edits and nobody else’s. And it is torn down when it merges or is abandoned, taking all of that state with it.
Naming the change as the tenant turns a vague scaling problem into a design target, because change-level tenancy has three requirements that person-level tenancy never had to meet:
The discipline these requirements call for is not new. Anyone operating a multi-tenant production service already knows the rules: tenants share the substrate, each tenant privately owns only what makes it distinct, creating a tenant is self-service and cheap, and a tenant’s resources are reclaimed the moment it leaves. Nobody stands up a private copy of the product per customer, and nobody files a ticket to onboard one.
Those same organizations run pre-production on the opposite rules. Environments are provisioned by ticket or by seat, capacity is planned per person, and isolation is achieved by duplicating the stack when it is achieved at all. The multi-tenancy playbook that runs the product has never been applied to the platform that builds the product.
Change-level tenancy is that playbook, applied. Treat every change as a tenant of the development platform, and the three requirements stop being novel. They are the standard properties of any competently run multi-tenant system.
A SaaS tenant owns its data and configuration, never a copy of the application. A change tenant is sized the same way. It owns the one or two services it modified and an isolated database branch it can migrate and write against, and nothing else. Everything the change did not touch resolves against one shared stable environment, continuously deployed from main, so every tenant validates against real, current dependencies without owning a copy of them.
A footprint that small makes tenant creation nearly free, and creation cost is what decides whether the model scales to agent demand. Isolated data no longer requires copying a database: Neon and Xata create copy-on-write branches in seconds regardless of dataset size, consuming storage only for the data that diverges. The runtime side costs one deployment, because starting the modified services is all that is left to do. A tenant that costs one deployment can be created hundreds of times a day.
Multi-tenant platforms scale because nobody provisions tenants by hand. Signup creates the tenant, cancellation removes it, and no operator sits in the loop. Change tenants need the same contract. The tenant comes into existence when work on the change starts and disappears when the change merges or is abandoned, with no ticket at the front and no cleanup script at the back.
Offboarding is the half that platform teams underestimate. At person scale, an orphaned environment was a minor waste found in a quarterly cleanup. At change scale, orphans accumulate as fast as agents abandon experiments, and the leak outgrows the cleanup.
Automatic offboarding also keeps the accounting accurate. When tenants are created and destroyed by the change’s own lifecycle events, the number of live tenants equals the number of changes in flight, and platform capacity becomes a quantity you can measure and plan against instead of a pile of environments nobody is sure anyone still uses.
The practical shift for platform teams starts with measurement. Count changes in flight at peak, not seats: open pull requests with activity in the last day is a fine proxy, and for most teams the number is already several times headcount. Then price the marginal tenant: What does one more concurrent change cost in dollars and in minutes of setup? If the answer is a full environment and tens of minutes, the platform is still doing person-level tenancy.
Those two numbers expose where the old assumptions live. Namespace quotas sized per developer, staging booked by team calendar, database seeds refreshed nightly for everybody at once: each is a seat-denominated policy waiting to fail under change-denominated load. The fix in every case is the same three requirements: near-free creation, isolation sized to the change, lifecycle bound to the change, applied to whichever part of the platform still assumes the tenant is a person.
Every previous definition of the tenant named a person or a group of people, and that held because only people produced changes. A platform could equate one seat with one workstream, plan capacity from the hiring plan, and keep a human in the provisioning loop. Coding agents break all three of those properties at once: one person now operates several concurrent workstreams, those workstreams are created and abandoned at machine pace, and no human is positioned to provision or clean up each one.
“The change is the only unit of isolation that stays stable when the workers become software.”
That is why the software development lifecycle (SDLC) needs its tenant redefined around the change rather than around whoever, or whatever, wrote the code: the change is the only unit of isolation that stays stable when the workers become software.
Organizations that keep person-sized tenancy will watch agent-generated changes queue behind infrastructure built for a fraction of the load. The ones that re-platform around the change will convert agent throughput into merged work. If you’re exploring the second path, that is exactly what we built Signadot to support.
The post Per-developer environments were the goal. Agents moved the goalposts. appeared first on The New Stack.
Most organizations that know they should sign their images still don’t. Not because they disagree, but because the path to doing it well has been too long. The result is a delivery pipeline built on trust that nobody can verify.
Unsigned container images create an open door for attackers at every stage of the delivery pipeline. Malicious images masquerade as legitimate packages, waiting to be pulled by an unsuspecting team. Compromised CI/CD pipelines silently inject tampered artifacts into production builds with no cryptographic evidence of modification. Stolen credentials let a bad actor impersonate a trusted publisher. Even within a single organization, inconsistent practice means some teams sign while others skip the step entirely, leaving gaps in the chain of trust that nobody has mapped. Compounding all of it is base image inheritance. Every container image inherits the security posture of its parent, so one compromised base image can propagate across dozens of downstream services before anyone notices.
“Scanning is fundamentally reactive. One tells you what is inside. The other tells you whether you can trust it.”
Scanning is fundamentally reactive. It answers, “what vulnerabilities exist in this image?” It cannot answer the question that matters more as artifacts get harder to inspect: “who built this, and has it been modified since it left the build system?” That is the domain of cryptographic signing, which provides proactive provenance. The two are complementary, not interchangeable. One tells you what is inside. The other tells you whether you can trust it.
The workloads have changed faster than the tooling. Model weights, training datasets, inference runtimes, and agent tooling increasingly ship as OCI artifacts. A pickled PyTorch checkpoint itself has no CVE to match against. Safer serialization formats like .safetensors remove the code execution path, but they say nothing about who produced the weights or whether they’re the ones you meant to load. There is no vulnerability database for a set of trained weights, and the CVE and SCA based scanning that registries run has nothing to compare them to.
This is not theoretical. In February 2024, JFrog researchers found a malicious PyTorch model on Hugging Face that opened a reverse shell the moment it loaded, abusing pickle’s __reduce__ hook to execute arbitrary code on torch.load(). Their analysis surfaced roughly 100 models on the hub carrying genuinely malicious payloads. No CVE fired, because there was nothing for a CVE to describe. The malice lived in the serialized weights. Model-specific scanning has since appeared to close that gap. Hugging Face runs ClamAV plus a pickle import scan on every file pushed to the Hub, statically disassembling the pickle’s opcode stream to flag dangerous imports. While they help, they are also already being evaded. In February 2025, ReversingLabs described nullifAI, two models that slipped past picklescan by compressing with 7z instead of ZIP and by corrupting the pickle stream immediately after the payload ran, so static analysis errored out on a file whose reverse shell would have already run. Hugging Face removed the models inside 24 hours and patched picklescan. That is the shape of the problem. Pattern matching scanners are a line that keeps moving, and each one answers whether a file resembles something known to be bad. None of them answers where the file came from.
“A tampered application image defaces a page. A tampered AI model artifact corrupts predictions at scale.”
AI is widening the attack surface in the same motion. Coding assistants suggest dependencies that never pass a human threat model, and that code gets containerized and shipped faster than review can keep up. The blast radius changed too. A tampered application image defaces a page. A tampered AI model artifact corrupts predictions at scale, poisons recommendations served to millions, or in the agentic case takes actions in production: API calls, tool invocations, spend. And when you consume a pre-trained model, you inherit every upstream decision about its training data and its security with zero visibility into any of them. Provenance stopped being a question about your application code. It became a question about the model, the agent, and the tooling that carries them.
But signing is not a checkbox. It is a chain. It only works if every link holds.
Operating the registry at the scale of Amazon ECR has taught us something that shaped how we think about supply chain security. Most teams don’t verify images. They verify addresses. An admission policy allows images from your registry account, push credentials belong to the pipeline rather than to people, and a scanner blocks critical CVEs. That stops a lot of attacks. What it can’t do is tell a good image from a bad one once it’s inside the boundary, because registry provenance is a claim about location, not origin. Anything that can write to the repository produces an image that looks legitimate: a leaked CI token, a misconfigured cross-account role, a compromised build step. Digest pinning tells you that you got the bytes you asked for, not that those were the right bytes to ask for.
Every container image passes through a registry before it runs. It is the last system in the path that sees every artifact, knows who pushed it, and controls who can pull it. It already holds identity context, already enforces access policy, and already stores the metadata that describes what an image contains. The hard part of image signing is doing it consistently across every team and every pipeline without slowing anyone down. The registry is the only layer that can make it invisible.
“The hard part of image signing is doing it consistently across every team. The registry is the only layer that can make it invisible.”
Signing does not make forgery impossible. An attacker who fully compromises a trusted signing identity, stealing both the credential and the permission to sign, can produce a validly signed malicious image that passes verification. What signing does is shrink the attack surface. Without it, tampering anywhere in the path works, because nothing downstream checks. With signing and enforcement, none of it works unless the attacker compromises one narrowly scoped signer, and that rogue signature is an auditable event tied to an identity instead of an anonymous overwrite. Revoke the identity and the whole fleet stops trusting it in one change. Signing turns an invisible, unbounded problem into a scoped, attributable, revocable one.
Signing is a three-step process:
Sign: Generate a signature at build or push time, binding the image digest to a verifiable identity. The hard question is custody: who holds the private key, and how is it rotated and protected?
Verify: At pull time, and critically before the workload is admitted, check the signature against a trust policy which is a declared list of the identities you trust to have signed what you are about to run.
Enforce: A Kubernetes admission controller like Kyverno blocks any image not signed by a trusted identity from ever running. Signing without enforcement changes nothing.
Enabling signing comes with operational cost. Engineers had to install and configure client-side tooling like Notation CLI or Cosign, then own their signing keys, certificates, rotation schedules, and revocation lists, then build custom automation to wire signing into every pipeline. Across an enterprise with thousands of uniquely configured pipelines, that rollout took weeks to months. What we wanted to know was whether the registry itself could absorb the cost, so that signing could become a property of pushing an image rather than a project each team takes on. The answer to that question became Amazon ECR Managed Signing.
The mechanics are deliberately boring, which took some doing. You create a registry level signing configuration with up to ten rules, each pairing a signing profile with repository filters, and every matching push gets signed from then on.
Managed Signing answers the custody question by not giving you the keys. You configure a signing profile in AWS Signer, which pins the signing algorithm, a validity period, and the identity that appears in the signature. Signer keeps the certificate and the private key. This means no signing key ever sits in a repo, a runner, or a build log. Validity defaults to 135 months, so signatures won’t expire on you. Revocation is what you’ll actually use when you find out a build was compromised.
Then what gets signed, which is narrower than people assume. Signer signs a small Notary payload whose targetArtifact describes the image manifest: media type, digest, size. Not the image bytes directly. Because the signed material is content addressed, verification becomes a statement about exact bytes. The signature itself lands in the same repository as a detached OCI artifact, typed application/vnd.cncf.notary.signature, with a subject descriptor pointing at the image manifest digest. One image can carry signatures from several profiles as your trust requirements change.
Signing happens asynchronously, which keeps Signer off the push path. A synchronous call would turn an availability dip or a throttle into a failed docker push for a developer, and it would put signing latency in front of every pipeline. The push commits first, and ECR calls SignPayload after.
Verification and enforcement happen downstream, and the trust policy is where the whole design becomes legible. Your cluster operator writes it and imports it with notation policy import. It’s a short reviewable file:
Login
{
"version": "1.0",
"trustPolicies": [
{
"name": "aws-signer-tp",
"registryScopes": ["*"],
"signatureVerification": { "level": "strict" },
"trustStores": ["signingAuthority:aws-signer-ts"],
"trustedIdentities": [
"arn:aws:signer:us-east-1:111122223333:/signing-profiles/platform_images"
]
}
]
}
That policy says a workload runs only if it carries a signature chaining to the AWS Signer root and produced by that specific profile. Admission does the work in order: resolve the reference to a digest, fetch the signature via OCI Referrers API, validate the envelope against its embedded certificate chain, walk that chain to the root in the trust store, check the signing identity against trustedIdentities, and check revocation. Revoking a profile makes verification fail wherever that profile is trusted. New admissions stop immediately and running pods pick it up when they’re next rescheduled. On EKS you get there with Gatekeeper and Ratify, or with Kyverno. Both paths use the AWS Signer plugin. Every link is checkable by the cluster itself, from the artifact plus a root certificate without asking the verifier to trust the registry it pulled from, or the pipeline that pushed.
Vulnerability scanning answers a question that mattered in the application era: what is broken inside this image? The AI era asks a harder one that scanning was never built to answer. Can you prove where this came from, and that no one touched it?
The cryptography was never the hard part. Making it the path of least resistance was. Sign, verify, and enforce, and let the registry carry the tax so your teams don’t have to.
To explore what’s referenced here, see Amazon ECR managed signing and signature verification on Amazon EKS.
The post Your container images are unsigned. In the AI era, that’s a ticking time bomb. appeared first on The New Stack.
Mistral AI, the French AI company that built its reputation releasing open-weight models, wants companies to use its infrastructure even when they pick a model other than Mistral.
The company said Tuesday that it will begin hosting third-party open models, starting with GLM-5.2 from China’s Z.ai. The model will run on the same infrastructure as Mistral’s own models, with access to its regional processing controls and new priority service tier.
The company wants to give enterprises one place to run different open models, without forcing them to start over every time they switch.
Mistral’s regional endpoints are now largely available in Europe and the United States. Its Priority Tier, which puts eligible requests ahead of standard traffic and comes with a 99.5% uptime service-level agreement, is in public preview.
The company wants to give enterprises one place to run different open models, without forcing them to start over every time they switch.
The first is GLM-5.2, a model from Z.ai with a 1 million-token context window. Mistral lists coding and long-context agentic work among its main uses. GLM-5.2 is available through the company’s API as zai-glm-5-2 and costs $1.40 per million input tokens, $4.40 per million output tokens, and $0.14 per million cached input tokens.
A team might use GLM-5.2 for coding, Mistral Medium for work involving images and text, and Small for cheaper, everyday requests. Using the same API doesn’t make the models interchangeable. Each model has its own quirks, so teams will still need to test it before putting it into production.
And while GLM-5.2 has open weights, using it through Mistral is still a hosted service. Mistral decides which version is available and runs the infrastructure behind it.
Each model has its own quirks, so teams will still need to test it before putting it into production.
With Mistral’s regional inference service, developers can decide whether their requests are processed in Europe or the United States by changing the API endpoint. The regular endpoint comes without that guarantee. Mistral says keeping inference closer to users could reduce latency and help companies meet data-location requirements, although it adds 10% to the cost of every input, output and cached token.
Mistral says some account and usage data may still leave the selected region. Information may also be shared with outside companies under the safeguards in its Trust Center.
Companies working with financial, health or government data will still have some homework to do. The prompts may stay in Europe, but teams also need to find out what Mistral logs, where that information is stored, who can see it, and what gets through to outside companies. The Microsoft-Mistral sovereign compute partnership addresses some of these questions for Azure customers, but teams running workloads directly on Mistral’s own endpoints face a different set of guarantees.
The regional endpoints don’t support everything yet and, depending on the region, some models are missing. Developers can use function calling, but not Agents, Batch or the Files API. That indicates moving an existing application to the EU endpoint may take more than swapping out the base URL. If the application depends on an unsupported feature, Mistral’s regional processing guarantee no longer covers the workload.
Mistral’s Priority Tier puts eligible API calls into a queue ahead of Standard Tier requests when infrastructure is busy. Customers must arrange access with Mistral and agree on custom limits for each model. Once that is set up, developers can add service tier: auto to a completion request. Leave the field out, and the request goes through Standard Tier.
A request only gets prioritized when the organization has an active entitlement, the selected model is covered, the request falls inside its custom rate limit, and Mistral has capacity for that model in that region. Miss one of those conditions and the request can drop to Standard Tier.
Mistral includes the tier that served the request in the response’s usage object. Developers can record that field to find out how often requests are being downgraded, rather than assuming every call marked auto received priority treatment. If latency suddenly climbs, the team needs to know whether the model slowed down or requests quietly fell back to the standard queue.
The company is gathering a group of European businesses and institutions willing to make multi-year compute commitments. ASML, Amadeus, Capgemini, Caisse des Dépôts and CMA CGM were named in the announcement, but the company didn’t say how much capacity any of them committed to buy. Mistral calls the resulting allocations European Compute Units, or ECUs. Customers will be able to apply those units across Mistral Compute products as their needs change.
For customers, it is a bet made years in advance. They may know they will need AI compute, but not which models they will use, how large those models will be, or whether the workload will be inference, fine-tuning or something that has not been productized yet. Letting ECUs move across Mistral’s products is supposed to leave room for that uncertainty.
Mistral has not said how much compute an ECU buys, what it will cost, whether customers can carry over unused capacity or what happens if the infrastructure is not ready on time. But it’s clear that Mistral needs customers to keep using its infrastructure, whichever model they choose. That same bet — that the infrastructure layer matters more than any single model — is one that a number of enterprises are already making. GLM-5.2 is the first sign of how that could work.
Mistral needs customers to keep using its infrastructure, whichever model they choose.
The post Five European companies just agreed to buy AI compute that doesn’t exist yet appeared first on The New Stack.
AI data centers in space sound great, but practically speaking, they may be next to impossible.
For tech bros, it sounds great. Two of the buzziest tech giants, SpaceX and NVIDIA, are partnering together to bring AI data centers into space using the just-announced Starmind AI1 satellite.
These 30-meter-tall satellites with a 75-meter solar-array wingspan will contain the latest NVIDIA Vera CPUs and Rubin GPUs. These will live in a Low Earth Orbit (LEO) of about 600 kilometers. For networking, it will use Starlink’s laser links. SpaceX says the first AI1 spacecraft will perform localized AI computing in orbit and relay results to Earth via Starlink.
According to SpaceX, AI1 is designed around a compute payload drawing up to 250 kW at peak and 175 kW on average. It will be solar-powered, unlike its Earth-bound competitors, which frequently require the construction of new power plants.
Starmind is not simply a conventional NVIDIA AI cluster launched into orbit. The effort hinges on integrating high-density accelerator hardware with a spacecraft platform capable of generating power, rejecting waste heat, surviving radiation, maintaining laser communications, and being produced in large quantities. None of that is easy.
Once in orbit, which will require SpaceX’s still-not-ready-for-prime-time Starship rockets to launch the estimated 2.3-metric-ton satellites, the satellites will work together.
Eventually, to reach SpaceX’s goal of a million (that’s not a typo, that’s a million) Starmind satellites, the two companies will need to design a standard model spacecraft. These will be built in SpaceX’s 11-million-square-foot manufacturing campus, Gigasat Factory, which is still under construction in Bastrop County, Texas.
This AI-in-space proposal is the most ambitious yet of SpaceX CEO Elon Musk’s dream of placing energy-intensive AI infrastructure in orbit. There, these satellites won’t need to compete for land, electrical-grid capacity, or water with increasingly contentious terrestrial data center buildouts.
However, SpaceX glosses over the technical issues of turning this vision into reality.
Let’s start with the biggest headache: Cooling.
Contrary to what you may think from bad science-fiction movies, the vacuum of space is not cold per se. Whether the surface of an object is hot or cold depends entirely on whether it’s facing the sun. Those on the sun side will heat up, while those away from the sun will eventually cool down toward the 3 Kelvin background of deep space.
The keyword is “eventually.” You can’t simply use convection, cooling towers, or evaporative cooling to carry away heat. The heat must radiate away as infrared radiation, and that’s a very slow process.
The physics creates a direct trade-off between computing power, radiator area, spacecraft mass, and operating temperature. A system running hundreds of kilowatts of AI hardware must reject nearly all of that power as waste heat. Liquid cooling can carry heat away from chips, but it does not eliminate the requirement for extensive radiator surfaces.
As NASA has found, “satellites experience harsh environments in orbit,” ranging from about 393 Kelvin in full sun (248 degrees Fahrenheit) all the way down to ~3 Kelvin (-454 degrees Fahrenheit).
To cool down the Starmind satellites, each will have a deployable liquid radiator system measuring 160 square meters. What liquid? We don’t know yet. Hugh Lewis, a professor of astronautics at the University of Birmingham, expects it to use ammonia, which is already used on the International Space Station (ISS). Whether this will reliably scale to data-center-class AI deployments with their enormous heat remains to be seen.
Another issue is its networking. The architecture depends heavily on Starlink’s optical inter-satellite links. SpaceX says AI1 satellites will use high-speed laser links to communicate with other spacecraft and send AI results to Earth via the Starlink network.
Starlink’s published technology specifications describe mini laser terminals operating at up to 25 Gbps across distances as long as 4,000 kilometers, while SpaceX cites roughly 25-millisecond latency for its customer service.
Those figures suggest a potentially useful network for distributing inference results, transmitting model updates, connecting orbital sensors to compute nodes, and avoiding some reliance on ground-station passes. But they do not establish that a satellite constellation can function like the tightly coupled networking fabric of a terrestrial AI supercomputer.
We won’t be seeing large-scale machine learning and training in space. This requires huge, predictable bandwidth and very low latency for GPU-to-GPU communications. An orbital network would also face physical propagation delays, laser-link acquisition and handoffs, routing across a moving constellation, and limits on available capacity per spacecraft.
Another issue, according to Doug Mohney, a long-time space influencer, is debris. “One bad day, a piece of random junk hits one satellite, which fragments into multiple pieces of shrapnel, which hits another satellite and so on and so on until you get a Kessler event that turns the selective orbit into a roaming cloud of debris.”
A Kessler event is when one satellite breaks up, and its fragments hit another, and so on until an area of LEO is filled with wreckage rather than viable satellites.
Adding insult to injury, a Kessler event may not happen by accident. Mohney also observes that space warfare is a real threat: “A bad actor such as Russia, China, Iran, or North Korea could use kinetic (unrandom junk!) means to target one or more satellites, resulting in space debris.” Or, “One good nuclear weapon uses an electromagnetic pulse to get rid of all of them at once. Both Russia and China (and the US) already have anti-satellite weapons (ASAT) programs. North Korea could have ASAT, but a nuke would ensure mass destruction of orbital capability.”
If that sounds crazy, keep in mind that Starlink satellites are already being used by Ukraine, and Russia has been trying to block their transmissions. There have also been credible reports of Russia developing ASAT weapons specifically designed to knock Starlink satellites out of the sky. Larger and more fragile Starmind satellites would be far more vulnerable.
Mohney also worries about the “known unknown” of space weather.
“A Solar flare that hit the Earth along the lines of the 1859 Carrington Event, the largest recorded solar storm, would take out orbital electronics of all satellites.” This, in turn, as uncontrolled satellites drift from their orbit, might cause a Kessler event. Lesser events have already pushed LEO satellites out of space. For example, a February 2022 geomagnetic storm forced thirty-eight newly launched Starlink satellites out of orbit.
There are also business concerns. For all the obstacles that new and expanded ground-based AI data centers face, the energy analytics firm Wood Mackenzie believes “A hypothetical 1 GW orbital data center would cost an estimated $170 billion, more than three times the equivalent terrestrial facility, with launch and satellite costs accounting for approximately 60% of that total. To bring orbital costs to parity with terrestrial alternatives would require a 70% reduction.”
The company thinks that might be possible, but Robert Liew, Wood Mackenzie Research Director, observes, “That gap does not close without sustained and dramatic progress on launch costs. We forecast US$ 9 trillion of terrestrial data center investment between now and 2040. That is where capital goes first. Orbital data centers are a serious long-term proposition, but right now they remain a bet on the cost curve.”
For now, SpaceX has offered a broad technical vision and a hardware partnership with NVIDIA, but few of the operational metrics that would establish commercial viability. The real test will be whether SpaceX Starship becomes a practical launch vehicle and can overcome its cooling and safety issues. Then, the AI1 must also show enough usable compute per kilogram, kilowatt, square meter of radiator, and dollar of launch cost to outperform or complement ground-based AI infrastructure. I don’t see this happening anytime soon.
The post Why space is actually a terrible place to cool a data center appeared first on The New Stack.
During AI DevCon in London this summer, Lamis Mukta, member of technical staff at Anthropic, hosted a stage presentation session entitled ‘Learning while you sleep, beyond memory to dreaming’.
Mukta set out to examine where state-of-the-art memory management sits today in a world where (as she put it) “context is often orthogonal to the model intelligence” at hand.
“The newest model we’ve just released isn’t going to go out of the box and know exactly what it takes to succeed in your organization and what tasks you want it to do,” said Mukta. “It’s like agents [initially] not knowing their way around a codebase or knowing enough about your own user preferences.”
To steer agentic services the right way, systems obviously need access to memory to create a context window.
Providing a brief history of Anthropic memory management, Mukta said that traditional approaches made use of CLAUDE.md, a file that Claude reads at the start of every conversation (that includes Bash commands, code style, and workflow rules) to give Claude persistent context that it can’t infer from code alone.
Effective to a degree, this technique becomes hard to manage over time, especially when a file with very important preferences gets very, very long.
“So a second avenue that we investigated was memory tools, and this is interesting because it leans into the idea of what happens if we let agents autonomously manage their own memory systems? We let them decide when they read, when they write, and when they update memories,” explained Mukta.
This process happens in-band i.e. within the context of a session. When dovetailed with so-called progressive disclosure, the agent only looks at the light metadata at Layer 1, before reaching for full content and original source files in Layers 2 and 3, respectively, so that the system doesn’t overload the model’s context.
“The way I like to think about it is as if I’d had a bookshelf in my room, and every time someone talks to me, I can kind of scan and look at my list of books and see if any of the titles might be relevant to the conversation, and then pick that off the shelf and read it when I need to,” explained Mukta.
But the bottleneck here is that we’re still driven by humans and agents working together i.e. we’re still being quite opinionated about what things need skills. The additional problem here is that memories can go stale and become irrelevant to an organization’s needs. Add the fact that a memory file may be written incorrectly or even maliciously injected and you can see why a lot of guardrails need to be in place.
“We introduced the concept of dreaming, which is a process that runs asynchronously in batch with its own allocated resources, to ensure that memories themselves are effective, up to date, and [so we can] help the agents learn over time.”
“So we introduced the concept of dreaming, which is a process that runs asynchronously in batch with its own allocated resources, to ensure that memories themselves are effective, up to date, and [so we can] help the agents learn over time,” explained Mukta. “[This process allows us] to consolidate memory and cut things that are no longer relevant, add things that agents are missing, and clean up and organize memory systems.”
In Anthropic’s world of slumber, dreaming is an out-of-band asynchronous process which the organization says solves the in-band limitation, where agents must split effort between completing and executing tasks, while also concurrently curating memory for their future selves. Dreaming spots recurring failure patterns where agents are consistently failing (wrong units, missing topics, broken tool configs, stylistic tics like overused em dashes), and proposes memory-store updates, again for human review, but hopefully at a more effecient level.
This architecture underpins Anthropic’s Managed Agents memory and API approach at this level, so has the frontier model company won over developers?
Staff software engineer, cloud architect and independent researcher in AI agent systems, Jayakumar Ramalingam, tells The New Stack that “dreaming is useful, but it also creates a dangerous promotion path” i.e. one that leads from repeated mistakes to persistent policy.
“A bad answer normally dies with the session; a bad memory can influence thousands of future sessions. Human review sounds reassuring, but at fleet scale it can easily become a rubber stamp for recommendations nobody has time to reconstruct,” Ramalingam says.
“The industry has spent too much time treating memory as a context window problem when it is really a state management problem.”
He insists that every proposed memory should “carry provenance, evidence and an expiration condition”, and not just exist as a pattern that recurred often enough to look real. Otherwise, he thinks that dreaming may help agents remember more while making organizations forget why the memory was trusted.
“Anthropic is getting one important thing right: its agent memory should look more like versioned infrastructure than artificial cognition. The industry has spent too much time treating memory as a context window problem when it is really a state management problem,” underlines Ramalingam.
His point is – if an agent cannot show who changed a memory, why it changed and how to roll it back, it does not have production memory, so it becomes an unaudited configuration file with an AI attached.
Enterprise AI architect and founder of Besk Tech, Vladimir Beskorovainyi, tells The New Stack that “dreaming is the right instinct aimed at the wrong evidence”, because the failures it catches (wrong units, broken tool configs, too many em dashes etc) are all visible on the surface of a transcript.
“The failure that actually costs you is an agent reaching for the wrong tool for a reason that looked perfectly defensible at the time,” Beskorovainyi says. “In the systems I run in production, the log records the decision rather than the API call, and that is the only reason a review pass like this finds anything worth finding.”
“When the ‘lately’ factor quietly becomes true. That leaves us at a point where versioning tells us what changed and when, not what is correct.”
He points to what he calls “a worse problem underneath the agent’s decision” i.e. if updates are proposed from recent batches, the memory store drifts towards whatever the agent fleet happened to do lately, and so the “lately” factor quietly becomes true. That leaves us at a point where versioning tells us what changed and when, not what is correct.
“The industry spent two years insisting that memory meant embeddings, and Anthropic solved it with a filesystem and grep [a Linux command that searches for patterns in files] and that is the most interesting decision in this whole discussion,” insists Beskorovainyi.
He says the reason it matters is legibility. A memory store a developer can open and read is a memory store an engineer can audit, and (he insists) “no vector database has ever offered that”, while everything else in the architecture (the versioning, the hashes, the tiered permissions), is ordinary distributed systems engineering we have known how to do for decades.
Founder of autonomous AI penetration testing company Penetrify, Viktor Bulanek, tells The New Stack that when the industry spent two years convinced that agent memory was a vector database problem, and Anthropic shipped grep, that was a useful thing.
“In terms of what Anthropic is getting right… a memory store you can cat, diff and code review is one you can actually operate, whereas nobody has ever successfully debugged an embedding that quietly ranked the wrong chunk third,” Bulanek says.
“Anthropic’s approach to dreaming is the clever part and also the part that worries me most, because it points an automated writer at session transcripts, and transcripts are full of content the agent did not author.”
He thinks that the versioning matters here far more than the auditability framing suggests and reminds us that “rollback is not a compliance feature”; it is the undo button for a poisoned memory a software engineer discovers three weeks after it was written, which is the incident every serious agent deployment is going to have eventually.
“But to add balance here, Anthropic’s approach to dreaming is the clever part and also the part that worries me most, because it points an automated writer at session transcripts, and transcripts are full of content the agent did not author,” Bulanek cautions.
“Anthropic is right that human review is the answer, but bulk review of proposed diffs is exactly the control that decays fastest once the suggestions are mostly good. The other gap is that nothing in this architecture says when a stored fact stops being true. Versioning tells you what changed, it does not tell you what rotted, and a confident note about a system that was refactored last month is worse than no memory at all,” he advises.
Bulanek’s work sees him run autonomous agents in production that perform penetration testing and run for hours unsupervised with real credentials against live systems, so memory for his team is both an operational cost and a security boundary at the same time.
Co-founder and CTO of Noah Labs, Berk Yilmaz, tells The New Stack that the Anthropic way of doing things has “an endearing lack of flair to it” in his view.
“Everyone wants memory to feel like the newest incarnation of machine intelligence, and their pitch goes something like: just give it a filesystem, versioning, searchability, and don’t let a thousand processes stamp all over each other,” Yilmaz says. “This is closer to how production AI should be done. While we have spent a long time improving models, the supporting infrastructure has not kept up, failing in incredibly prosaic engineering ways.”
Yilmaz is behind a company that develops an AI-native IDE for government and regulated systems, built for air-gapped environments and legacy codebases. He reminds us that once a memory decision is made on which past behavior should become future behavior, memory itself ceases to be inert.
“A hallucination that dies after a single session is a pain in the neck, but a hallucination that outlives a thousand sessions is infrastructure. The same thing applies to security; if an attack succeeds in writing to memory, it has become persistent. Provenance becomes absolutely critical here, how was the system taught this, where did it learn it from, who certified it, and can I undo it? In enterprise AI, sometimes forgetting is a safety measure,” adds Yilmaz.
AI, product & data science leader and former Meta employee, Kerstin Frailey, tells The New Stack that at face value, dreaming (for her money) “certainly sounds like it has the potential to blow up AI bills” right now.
“A cynic would say this is designed to fill the revenue hole left by tokenmaxxing before Anthropic’s IPO,” Frailey says. “An optimist would hope for a beautifully thrifty design. A pragmatist would remember that Anthropic gets paid for usage, not efficiency. A skilled practitioner would run incremental pilots, aggressively monitor costs, and routinely test for measurable improvements.”
“As a nice bonus, dreaming offers potential system improvement, too. But its familiar predecessors – garbage collection and storage compaction – are comparatively deterministic and controlled.”
She continues and notes that dreaming offers cleanup and consolidation, which she defines as a “reasonable development” for any system that constantly generates new files.
“As a nice bonus, it offers potential system improvement, too. But its familiar predecessors – garbage collection and storage compaction – are comparatively deterministic and controlled. Unlike its namesake or those analogues, dreaming appears neither cheap nor efficient: pay an AI to do the work once, then pay AIs to regularly review, revise, and restructure it,” she adds.
Dreaming as part of Anthropic’s Managed Agents memory and API approach isn’t alone. The notion of AI model dreaming (or automatic out-of-band background memory consolidation if we’re being formal about things) is also being popularised by OpenAI for ChatGPT, in stateful agent coding platform Letta and elsewhere.
The bottom line here may be a realization that, in AI modeling terms at least, memory is actually maintenance.
The post Anthropic gave agents the ability to dream. Then developers woke up. appeared first on The New Stack.
The AI boom is an oddly miserable bubble. Despite interesting tech, huge new companies, and products with global reach, AI has attracted legions of detractors.
Some have valid complaints, like seeing their roles automated, or the value of human art being pressured by machine generation. Other complaints have had less staying power.
It was once in vogue to argue that AI companies would run out of data, and thus their models would stop improving. False. Some of the same voices argued that AI lacked a use case and was thus little more than a fancy toy high on its own hype. Incorrect.
Later, the argument shifted to AI being too expensive to use, an incredible flip from the AI has no real use argument. This is being proved false, as low-cost models from China now face both low-cost, closed-source AI models from OpenAI and new, open models from Meta. Agents were too brittle to start; now they are hacking the world. You get the idea.
Lately, I’ve read criticism about the AI boom from a financial perspective. Namely, that the major cloud players (AWS, Google Cloud, Azure) are spending too much money on AI infra. Surely we can’t use all that compute, the argument goes, and thus hyperscalers are torching their nest egg and investor goodwill at the same time.
I wanted to put the contention to the test, so I pulled together data from Amazon, Alphabet, and Microsoft’s cloud groups (here) to peel back the onion a little. Here’s what I found: Growth is accelerating, hyperscaler profitability scales with scale, and hyperscaler capex efficiency is improving.
This is an excerpt from Cautious Optimism, a modestly upbeat publication focused on technology, business, and power. Read more about the concern of hyperscaler cost on Cautious Optimism.
The post How I learned to stop worrying and love hyperscaler capex appeared first on The New Stack.
Anthropic announced it will embed invisible watermarks into text generated by new Claude models, including output produced through its API, coding tools and cloud partners. For developers, the mark offers another way to trace where AI-generated text or code might have come from, but it is not strong enough to prove its origin.
Laying out the plan in a support document, Anthropic said Claude models launched in the EU on or after Aug. 2, 2026, will include machine-readable marking from release. The company is working to add support to older models as well.
The marks will apply worldwide across supported Claude products, including the Claude API, Claude Code, Claude Cowork and Claude Tag. Text generated through AWS, Google Cloud or Microsoft Foundry will also carry the watermark when those platforms use a supported model. Because the mark is added at the model level, it follows the output into applications built on top of Claude — the same applications that are already reshaping how enterprises deploy AI infrastructure — although Anthropic cautions that some platforms and features may not support every type of mark.
The change follows the Aug. 2 start of Article 50’s transparency requirements under the EU AI Act, which require providers of generative AI systems to make synthetic output detectable in a machine-readable format. Anthropic signed the accompanying Code of Practice as a provider of both generative AI models and systems. OpenAI, Google, Meta, Microsoft and Mistral are among the other model providers that have committed to the code.
Yet, Anthropic is handling text and files differently. Text gets a watermark hidden in the words themselves, while supported files such as SVGs, PNGs and JPGs receive a digital signature using the C2PA standard. The file metadata can show that Claude processed an asset and whether the metadata has been altered.
“Because the watermark is part of the text, it will travel with the text when it’s copied and pasted elsewhere, and may persist through some editing,” Anthropic said.
“Because the watermark is part of the text, it will travel with the text when it’s copied and pasted elsewhere, and may persist through some editing.”
Anthropic has not explained how its text watermark works or said whether Claude uses KGW, a semantic version or another method. The company has also not shared any figures showing whether watermarking affects latency or adds to inference costs — a gap that matters for teams already wrestling with the hidden costs of agentic AI workflows.
Alex Cui, CTO and co-founder of AI detection company GPTZero, wrote in a technical explainer on X that watermarking systems fast enough to run on a streaming frontier model often follow the same general approach. One such technique, known as the KGW method, changes the probabilities the model uses when selecting its next token.
A language model normally calculates a probability for every token that could appear next. In a simplified watermarking system, a secret key and the preceding tokens are used to generate a hash, which divides the candidate tokens into two groups, often described as green and red. The model then slightly increases the probability of selecting one of the green tokens.
A detector with the same key can use the preceding text to reconstruct which tokens would have been favored at each position. A passage containing an unusually high number of those choices may carry the watermark.
Cui wrote that more advanced approaches can derive the watermark from the meaning of nearby text rather than an exact sequence of tokens, which may help the signal survive some paraphrasing because replacing a word does not always change the surrounding context.
“Their watermark needs to work token-by-token because they are streaming their text to users,” Cui wrote. “Many watermark methods plan sentences or paragraphs at a time, or change the text after it’s entirely written, in order to make their watermark robust to paraphrasers.”
Anthropic has not confirmed that Claude uses any of these methods, but streaming limits the techniques available because the model must add the signal while generating its response rather than rewriting a completed passage afterward.
Code presents a different problem because the model has fewer valid choices. Words can often be swapped or sentences rewritten without changing their meaning, but seemingly minor changes can break working code. That challenge intensifies as the AI coding era matures and more production code flows through model-assisted pipelines.
“There are some texts, like code, that cannot be arbitrarily changed; otherwise the code will break,” Cui wrote. “In those cases, the watermark needs to selectively change words in parts of the text that can tolerate synonyms,” such as variable names.
Code may also be difficult to track through a normal development workflow. Anthropic has not published tests showing how well its watermark survives those changes, so teams do not yet know whether a Claude-generated patch will remain detectable after passing through a pull request.
“In my testing, the watermarks don’t survive intense paraphrasing, especially if you combine word choice and syntax attacks.”
The same issue comes up when applications change Claude’s output before showing it to a user or committing it to a repository. Summarizing it with another model, translating it, splitting it into smaller sections, turning it into structured data or mixing it with database content could all make the watermark harder to detect.
Anthropic acknowledges this limitation. Editing, paraphrasing, translating or combining the response with other text may weaken or remove the watermark, while short excerpts may not contain enough of the signal to detect.
Cui wrote that determined users can attack a watermark by changing both the vocabulary and the structure of a passage.
“In my testing, the watermarks don’t survive intense paraphrasing, especially if you combine word choice and syntax attacks,” he wrote. Cui added that free paraphrasing tools he tested were able to bypass Google DeepMind’s SynthID text watermark.
Research supports those concerns. The “Watermarks in the Sand” paper found that, under defined assumptions, attackers can remove watermarks without severely damaging the quality of the content. The absence of a watermark does not show that Claude had no role in creating the content. The response may have come from an older model, may be too short to carry a detectable signal or may have been changed somewhere in an application pipeline. It may also have passed through a platform or feature that does not support that type of mark.
Finding a watermark does not prove authorship either. Claude may have proofread, translated or reformatted material written by a person. Anthropic says a detected mark means only that the content “may have been processed by Claude,” not that Claude created the underlying work.
“If Anthropic releases the watermark detector publicly, I think they defeat their own watermark. People find reliable watermark-removal strategies by testing against Anthropic.”
Anthropic plans to give users and third parties a way to detect its marks, but it has not said whether that will take the form of a local tool, a detection API or access limited to selected organizations. A public detector would be easier for developers to add to their applications, but it would also allow someone trying to remove a watermark to keep editing and checking the text until the signal disappears.
“If Anthropic releases the watermark detector publicly, I think they defeat their own watermark,” Cui wrote. “People find reliable watermark-removal strategies by testing against Anthropic.”
The secret keys behind the watermark create another challenge. A leaked key could make it easier to remove the mark or imitate it in text that Claude did not produce. The scenario echoes what happened when provenance attestations were turned into camouflage — a trust signal that was supposed to increase confidence instead became an attack surface.
“To avoid a large blast damage from this, you need to have a couple secret keys in rotation,” Cui wrote.
Key rotation would require detectors to recognize marks created with both current and retired keys, including those embedded in content generated months or years earlier. Anthropic has not explained how it plans to handle that history.
For developers, Claude’s watermark is best treated as another clue, not a replacement for audit logs or provenance tracking. Applications that need to show where an artifact came from can record the model ID, prompt version, response time and a hash of the original output, then log any changes made before it reaches a user or is committed to a repository.
The watermarking announcement arrives as Anthropic navigates deeper questions about what its models do in the wild. Recent incidents have exposed gaps between lab safety evaluations and real-world containment, and the company has publicly backed calls for the most powerful AI labs to slow down. Watermarking fits into that posture — a transparency mechanism rather than a safety guarantee — but its practical value depends on technical details Anthropic has not yet shared.
Anthropic tells customers to determine how Article 50 applies to their own products and says more technical documentation is coming. But until Anthropic shares those details, teams do not know how they will detect the marks, how key rotation will work or how well the watermark will survive common changes to code and application output.
The post Anthropic’s watermark survives copy-paste, but not the real dev workflow appeared first on The New Stack.
Databricks on Tuesday announced that it’s acquiring Electric, the startup behind the WASM-based Postgres project PGlite and the Electric sync engine, as agentic applications change how developers use databases.
The Electric team will join Neon, the serverless Postgres company Databricks acquired for about $1 billion last year and the foundation of its Lakebase database service.
The companies didn’t disclose the terms of the deal.
PGlite is a complete Postgres database in WebAssembly (WASM). It runs in the browser, a Node.js process, or inside the kind of sandboxes agents use to execute code. It supports dynamic extension loading, including pgvector, the preferred Postgres vector extension.
According to the companies, PGlite has grown from 1 million to 13 million weekly downloads over the last year.
It’s the Electric sync engine that is core to Databrick’s interest in Electric, though. This engine keeps a central Postgres database that can then be synced in near real-time with browser tabs, mobile apps, or agents. As Databricks notes, this is the multiplayer model of Figma or Google Docs, but applied to Postgres and the agents that use it.
The Neon team, in its own announcement, notes that “complex problems like conflict resolution, partial replication, and reconnection logic make real-time sync difficult to build from scratch.” Hence why Databricks likely acquired Electric instead of trying to build this from scratch itself.
As for the future of Electric, the company’s founders James Arthur and Valter Balegas write that “everything we’ve previously open sourced stays open source.” This covers the sync engine, PGlite, Durable Streams, and TanStack DB.
What doesn’t survive the deal, however, is Electric’s hosted service. “Electric Cloud is winding down,” the founders. “Cloud users will need to self-host or move to another provider.”
The deal also extends a string of database acquisitions for Databricks that includes Neon itself and, more recently, the transactional processing startup Mooncake.
As the Databricks team argues, traditional non-agentic applications share one database among many clients, and that database is the most permanent piece of the stack. But agent workloads change this.
In a recent post on how agentic development changes databases, Databricks’ Ippokratis Pandis, Nikita Shamgunov, and Reynold Xin write that agents now create roughly four times more databases than human users do on Lakebase. They also stress that the average project now carries about 10 database branches, and that some projects run more than 500 branch iterations deep.
For some types of applications on Lakebase, the average database compute is now alive for under 10 seconds.
Agents, as it turns out, like to branch databases the way they branch code, a pattern Neon built its architecture around.
In practice, a coding agent spins up a sandbox, instantiates PGlite inside it, builds and tests against the database, and then either throws the whole thing away or syncs the result with — in the Databricks context — a Lakebase branch. Because Lakebase separates storage from compute and keeps its data in Postgres page formats on object storage, creating that branch is a relatively cheap copy-on-write metadata operation.
“As coding agents drive the cost of creation to zero,” the Neon team writes, “the number of applications explodes, and most of them are small.” A database server, even a serverless one that scales to zero, imposes a floor on what the smallest viable app costs to run. “You can’t have an age of abundance if every app requires a fixed minimum of compute,” the post argues.
It’s worth noting that PGlite didn’t start at Electric. Instead, it began as an experiment by Neon co-founder Stas Kelvich, who compiled Postgres to WASM to see whether it could run client-side. Electric picked the work up and turned it into a production project. “That repo became the basis of PGlite,” Arthur and Balegas write.
As Databricks’ announcement notes, this now “reunites two halves of the same idea.”
The post Databricks acquires Electric to give every AI agent its own Postgres database appeared first on The New Stack.
Mistral AI wants to turn European AI sovereignty from a talking point into a product — one with a service-level agreement attached.
The French artificial intelligence company announced Tuesday a three-part expansion of its infrastructure business: regional inference endpoints that let customers choose whether their AI workloads run in Europe or the United States, a new "Priority Tier" backed by an uptime guarantee for mission-critical deployments, and a coalition of European enterprises making multi-year compute commitments that Mistral says will underwrite 200 megawatts of infrastructure across Europe by the end of 2027 — and a full gigawatt by the end of 2030.
In a move that may raise eyebrows among sovereignty purists, the company also said it will begin hosting third-party open models on its platform, starting with GLM-5.2 from Z.ai, the Chinese AI lab formerly known as Zhipu.
Taken together, the announcements mark a decisive shift in how Mistral positions itself. The company that built its reputation training open-weight language models is now selling something closer to critical infrastructure: assured capacity, regional control, and contractual reliability for enterprises and governments that want frontier AI without surrendering control over where it runs.
"When we spoke in June, the story was around how Mistral was building a full-stack AI offering," Timothée Lacroix, Mistral's co-founder and chief technology officer, told VentureBeat in an exclusive interview ahead of the announcement. "Today, the announcement is about strengthening one part of this infrastructure, which is the inference part."
That one part, it turns out, comes with a price tag measured in the tens of billions of dollars.
The headline numbers deserve scrutiny, because they imply staggering capital requirements. Mistral currently operates less than 200 megawatts of capacity, according to the company. Details shared with VentureBeat show the near-term buildout resting on three sites: a 44-megawatt facility near Paris that became operational in the second quarter of this year, a 23-megawatt facility in Sweden built in partnership with EcoDataCenter using renewable energy and advanced cooling, and a 10-megawatt site in Les Ulis, France, that came online in the third quarter.
Getting from there to one gigawatt by 2030 is a different order of magnitude. Independent estimates suggest just how different: research firm Epoch AI calculates that a typical one-gigawatt AI data center requires roughly $38 billion in upfront capital expenditure, with servers and GPUs — not buildings or land — consuming the majority of the cost. Goldman Sachs Research pegs next-generation AI facilities at $15 million to $20 million per megawatt before accounting for the chips inside them.
Lacroix did not dispute the scale of the challenge. The investment required for a gigawatt of capacity "is a large investment that requires also a lot of scaling and revenue behind it," he said.
The urgency, in his telling, comes from a supply crunch that is about to get worse. "More and more, and especially around 2027 and 2028, we see that the demand for AI compute is exceeding what the market has to offer, especially in Europe," Lacroix said. McKinsey has estimated that meeting global AI demand could require $5.2 trillion in data-center capital expenditure by 2030 — and Europe, by most analyses, is starting from behind.
A company valued at a fraction of its American rivals cannot close that gap with venture capital alone. Which explains the most consequential — and most unusual — piece of Tuesday's announcement.
Mistral is assembling what it calls an anchor group of enterprises whose long-term commitments will collectively finance infrastructure none of them could justify alone. Those commitments convert into "European Compute Units," or ECUs — a claim on Mistral-built capacity over multiple years that participants can spend on inference, training, model adaptation, or other AI workloads as their needs evolve.
If that structure sounds more like a power-purchase agreement than a cloud contract, that appears to be the point. Data-center financing increasingly resembles large infrastructure projects — gigawatts, substations, energy agreements — rather than traditional technology spending, and lenders want demand locked in before capital gets deployed. Mistral raised €830 million ($962 million) in debt earlier this year to fund its data center near Paris, TechCrunch reported in March, and pre-committed enterprise demand is exactly what makes that kind of financing repeatable at ten times the scale.
Lacroix was unusually direct about the mechanics. "The entire point of compute units is to have commitment," he said. "The goal is to have customers commit for around five years, or at least a long time." Asked what happens if a customer wants out early, he didn't soften the answer: "There is no getting out."
What makes a five-year, no-exit commitment palatable, he argued, is flexibility in how the capacity gets consumed. "Typically this can be spent on raw inference that you then feed through any other AI stack. It can be spent on raw compute as managed Kubernetes, and it can be spent at the very top with our full AI offering," he said. "My hope is that they will use it with our full-stack services and will love it."
The anchor group already includes some of Europe's industrial heavyweights. Amadeus CEO Luis Maroto said in a statement that "capacity, deployment control, and operating continuity become increasingly important for all enterprises." ASML chief Christophe Fouquet — whose company led Mistral's $13.4 billion (€11.7 billion) Series C last year — called building European AI capacity one of the few industrial endeavors that "will matter more to Europe's next generation," while Capgemini's Aiman Ezzat framed it as "a question of who shapes the future of European industry." CMA CGM chairman Rodolphe Saadé said the shipping group's Mistral deployment is "already under way among thousands of employees."
Commitments of that duration only make sense, of course, if the sovereignty being purchased is real. On that question, Mistral's announcement contains an asterisk worth reading closely.
The centerpiece product is Mistral Regional Endpoints, now generally available, which let customers pin inference and its associated processing to Europe or the U.S. Alongside it, the new Priority Tier — in public preview — offers committed service levels, custom rate limits, and an uptime SLA for mission-critical workloads.
Mistral claims it is the only European AI lab offering both a choice of processing region and an SLA-backed service tier, and Lacroix said a third option is coming: an endpoint "that stays on Mistral-controlled infrastructure, so on Mistral compute" — for customers who want their inference not just in Europe, but off hyperscaler hardware entirely.
Then comes the fine print. Mistral's own materials note that in-region inference remains subject to "limited, safeguarded transfers" to sub-processors that may sit outside the chosen region. Pressed on what actually leaves Europe, Lacroix pointed to the connective tissue of modern AI applications: tool calls.
"There are some tool services, like some tool calls, that might be hosted in places where we don't fully control this," he said, citing web search as an example. "A few of our web-search providers might not all be in Europe, and in that case, we need to potentially gate that capability."
His answer to the compliance question — would this satisfy a European bank or a defense ministry? — was that gating is the feature, not the bug. Capabilities that cannot be sourced in-region can be switched off entirely, restricted to certain users or workspaces, or, given sufficient demand, rebuilt with European providers. "Any capabilities that we don't find a provider for in Europe — if it needs to be done in Europe, we'll find some way to implement it or find ways to address it," Lacroix said.
For enterprise buyers, that is a more honest framing than most sovereignty marketing offers: full regional control is available, but the moment an AI agent reaches out to the open web, sovereignty becomes a configuration decision rather than a default. The same pragmatism runs through the announcement's most surprising line item.
A French national champion — one that has partnered with the French army and positioned itself as Europe's answer to American AI dependence — hosting a Chinese lab's model invites an obvious question. Lacroix's answer was disarmingly matter-of-fact.
"It's a great model. Everyone loves it. It's open weight, so there was no good reason for us not to do it, really," he said, noting that Mistral's own stack is already built on open-source software like Kubernetes.
On security vetting, he argued that open weights fundamentally change the risk calculus. "The risks in taking a new model, at the layer of the weights, are — at least in my opinion — rather limited," Lacroix said. "We checked basically all of the safety and compliance evals that we have. We'll control that model, its outputs, and what it does the same way we do any of our models. We have the same inputs and outputs and monitoring capabilities over all of it."
The strategic logic is worth unpacking. By hosting third-party open models under European regional controls and the same SLAs as its own, Mistral is repositioning itself from model vendor to sovereign distribution layer — the trusted intermediary through which any open model, regardless of origin, can be consumed by a regulated European enterprise that could never call a Chinese API directly. It is the "model garden" playbook the hyperscalers run with Bedrock and Vertex, executed on European soil with European guarantees.
Customers appear to be reading it that way. "Mistral allows us to run open models under strict regional controls and service commitments, making it easy for us to maintain data residency and compliance requirements," Matan Griberg, CEO of AI software-engineering company Factory, said in a statement.
Lacroix stressed the move is not a retreat from frontier training: the model Mistral had in training as of June "is still training, and we're still very excited about it," he said. But openness to rivals' models signals where the company now believes its moat lies — not in any single model, but in the infrastructure underneath all of them. Which makes its relationship with the world's most powerful infrastructure company all the more interesting.
Hovering over every sovereignty claim is Mistral's deepening relationship with Microsoft. In July, the two companies announced a multibillion-dollar expansion of their partnership under which Microsoft will rent capacity from Mistral's European data centers to serve its own cloud and AI demand, while adding Mistral Medium 3.5 and OCR 4 to Microsoft Foundry, bringing Medium 3.5 to Copilot Studio, and enabling Mistral models on Azure Local for disconnected, customer-controlled environments. Mistral CEO Arthur Mensch told The Wall Street Journal at the time that two-thirds of Mistral's customers already work with Microsoft.
How does a company selling independence from U.S. hyperscalers square taking one on as its largest tenant? Lacroix described Microsoft not as a patron but as an anchor customer that de-risks the buildout.
"It allows us to scale different parts of the business differently by building infrastructure with Microsoft as a customer," he said. "We can scale that team, we can scale our infrastructure, and make sure that we can then, on the side of it, also build for ourselves and for our customers." He compared the arrangement to the neocloud playbook — companies that built businesses supplying capacity to the hyperscalers themselves. "As that part of our business resembles that of neoclouds, we're following the same thing."
It is a genuinely clever inversion: rather than renting American infrastructure, Mistral is renting infrastructure to one of America's largest companies, using Microsoft's demand to finance capacity that also serves European sovereignty customers. But the independence has limits no contract can engineer away — the GPUs filling Mistral's European data centers come overwhelmingly from Nvidia and other American chipmakers, as SiliconANGLE noted in its coverage of the July deal.
Asked directly why a customer should choose Mistral over an EU region on AWS or Azure, Lacroix gave two answers. "The simplest possible answer is capacity. There is more demand than supply right now, and so it adds another option," he said. The second cuts closer to the pitch: "We are a European provider, and on the region that would be Mistral compute, we are fully independent. That's a truly differentiated offering than all of the hyperscalers or pure inference companies can provide."
There has always been a tension at the heart of Mistral's business: its best-known models are free to download, and open models have historically been difficult to monetize through APIs. Asked how free weights fund a gigawatt buildout, Lacroix offered the clearest articulation yet of the company's thesis — that the economics of self-hosting are collapsing under the weight of the models themselves.
"When the models were smaller, and we were before the explosion of agentic AI, it was doable for enterprises to host their own — up to, let's say, 100-billion-parameter dense models — on their premises," he said. "More and more, with models going into the trillion or more parameters, with the current hardware, and with the increasing amount of tokens that need to be processed, it becomes harder."
His conclusion was blunt: "I don't see how, with the current trend of model size and growth of agentic tokens, we keep the full inference on-prem. To me, that is why we think we're going to monetize our cloud inference." Inference, he noted, is particularly well suited to the cloud because it "does not need to hold any data" and can be encrypted in transit.
In other words: open weights get Mistral into the enterprise, and the physics of trillion-parameter agentic workloads brings the inference — and the revenue — back to Mistral's data centers. The thesis will get an expensive test. Mistral has raised roughly $4 billion to date, according to PitchBook data — a fraction of the war chests assembled by OpenAI and Anthropic — and Bloomberg reported in June that the company is in talks to raise about €3 billion at a roughly €20 billion valuation, nearly double its Series C mark. The revenue behind the buildout will have to come from exactly the enterprises Tuesday's announcement is courting.
And Europe, in Mistral's telling, is only the first market for what it is selling. Asked whether the framework could be replicated in the Middle East, Asia, or anywhere else anxious about AI dependence, Lacroix didn't hedge: "It's completely right. We're starting this in Europe because it's also an easier part of the world for us to scale into, especially in the infrastructure. But we definitely want to extend this, depending on customer demand." Every layer of the stack, he said, "can be controlled, changed, replaced depending on where we operate and what the requirements are — that's pretty much where we excel."
That is the wager underneath the SLAs, the compute units, and the Chinese model flying a European flag: in a world where the U.S. and China dominate frontier AI, the durable business is selling everyone else control. To fund it, Mistral is asking Europe's largest enterprises to sign five-year contracts with no exit — while making a bigger, longer commitment of its own. A gigawatt, after all, is a promise measured in decades. For Mistral, too, there is no getting out.
When the image is the bottleneck: Machine learning changed what a container image looks like. A typical application ships in a few hundred MB and starts in seconds. A modern ML inference image carries a deep-learning framework, the CUDA stack, and sometimes the model weights. These images roughly reach 20 to 30 GB, with some even higher. On the GPU and accelerated instances these run on, pulling one of those images takes several minutes before the application can serve its first request: minutes during which provisioned accelerators are ready to process real work but waiting for images to be pulled.
We hit this bottleneck on a production ML platform running on Amazon EKS. The team needed pods ready within two minutes, but image pull alone consumed several minutes. Each pod pulled a roughly 30 GB container image on top of loading model data from a shared filesystem. The images were rebuilt on a regular cadence, so worker nodes faced cold pulls with no usable local cache. While the image was pulled, accelerators sat idle, autoscaling lagged demand, and request queues built up.
The natural first suspect for these large image pull times was the network or the registry. After all, 30 GB is a lot of data. However, profiling the image pull path showed neither was the bottleneck on accelerated instances with 100 to 400 Gbps of network bandwidth available. The real constraint was how the software used the hardware already available.
“The real constraint was how the software used the hardware already available.”
By rethinking the pull pipeline to leverage the network bandwidth, storage throughput, and compute these instances already had, we got those multi-minute pulls down to seconds. The improvements are available by default on EKS Auto Mode today, and we contributed the core changes upstream to containerd and the SOCI snapshotter. This is the story of how we dove into the internals of the image pull path, identified where time was being lost, and rebuilt those stages. It starts where we started: understanding what a large container image looks like and how it gets onto a node.
A container image is not one file. It is a stack of layers plus a small JSON manifest listing them. Each layer is a tar archive of part of the filesystem (the base OS in one, CUDA libraries in another, your code in a third), gzip-compressed. For each layer, the manifest records a digest, a SHA-256 hash of the compressed bytes, so the node can prove it received exactly what was published. When the container runs, these layers are stacked and mounted together into a single unified filesystem.
Here is what real ML images look like when measured directly from their registry manifests:
| Image | Compressed Size | Layers | Largest Layer |
| AWS DJL LMI 21.0 Inference (cu129) | 16.5 GB | 29 | 9.5 GB |
| AWS PyTorch Training NeuronX 2.7.0 | 12.8 GB | 21 | 3.9 GB |
| AWS SageMaker Distribution 4.2.1 GPU | 10.5 GB | 28 | 9.4 GB |
Notice that layers within an image are not roughly equal in size. A single layer can account for more than half the total image, with individual layers often reaching 9 GB or larger, while the remaining layers are comparatively small. This size disparity has direct consequences for how long a pull takes, as the next sections explain.
Getting each of these layers onto a node involves six stages, and traditionally containerd, the industry-standard container runtime, performed most of these operations sequentially.
For each layer, containerd performs six operations in sequence. The first three are the download phase: fetch the compressed bytes from the registry over a single HTTP connection, verify the bytes by computing their SHA-256 and comparing it against the manifest digest, and write the compressed blob to local disk.
The next three are the unpack phase: decompress the gzip archive, verify the decompressed content by computing a second SHA-256, and extract the files into the local snapshot directory managed by a containerd snapshotter, the component that stores and serves the on-disk representation of each layer. By default, containerd downloads up to three layers in parallel, but each layer uses a single connection, and unpacking remains strictly sequential across layers.
Two details are worth noting. Both SHA-256 checks mentioned above are mandated by the Open Container Initiative (OCI) specification, and on a multi-gigabyte layer, each of these hashes requires real work. The first is computed over the compressed bytes and proves the download was not corrupted or tampered with.
The second is computed over the decompressed content. It gives the runtime a stable fingerprint of the layer’s actual filesystem data, which is what allows it to recognize shared layers across images and avoid redundant unpacking. Decompression is another deceptively expensive operation on large layers: gzip often triples a layer’s size on expansion, and because each block depends on the previous one, it runs on a single core. At the same time, the rest of the instance sits idle.
Downloading layers can be parallelized, but the unpack sequence runs layer by layer. Layer two cannot begin until layer one finishes all six stages. The result is that at any given moment during a pull, the node is bottlenecked on only one resource: network bandwidth during download, CPU during decompression and hash verification, or disk throughput during extraction. The other resources sit idle, waiting their turn in the pipeline.
“Because layers are not equally sized, the single largest layer becomes the long pole in the pipeline.”
Because layers are not equally sized, the single largest layer becomes the long pole in the pipeline. A 10 GB layer that takes a minute to decompress on one core holds up the entire image, even if the other layers finish in seconds. Total pull time is effectively bounded by that one dominant layer moving through all six stages.
There are well-known approaches to reducing image pull times. Each works around the bottleneck differently, whether by altering images, caching them, or relying on assistance from other parts of the stack.
Image size reduction: The most direct approach is to make images smaller through multi-stage builds, distroless base images, and stripping unused packages. For ML workloads, this hits hard limits. The GPU software stack alone (PyTorch, cuDNN, CUDA) imposes a compressed floor of roughly 3 to 4 GB that no build optimization can remove. Beyond that, ML images are assembled across organizational boundaries: a platform team provides the OS and drivers, a frameworks team adds deep-learning libraries, and researchers contribute application code and model weights. No single team controls the final artifact; model weights frequently end up baked in because serving frameworks expect local paths, and multi-stage builds yield only single-digit percentage savings on images whose bulk is irreducible.
Image caching (pre-pulling): Cache images on nodes by snapshotting image content into a volume and mounting it at provisioning time, so subsequent launches skip the pull entirely. This works for stable images but adds a dedicated pipeline stage for each image version. For ML workloads, the challenge goes beyond frequent rebuilds: a common pattern is a single pod consuming an entire node, so nodes scale in and out with each scheduling decision and every new node faces the full cold pull. This limits pre-caching to workloads with long-lived, static node pools.
“The GPU software stack alone (PyTorch, cuDNN, CUDA) imposes a compressed floor of roughly 3 to 4 GB that no build optimization can remove.”
Registry-side optimization: Some approaches serve image content remotely rather than pulling it to the node. Alibaba’s DADI, for example, presents container images as remote block devices that the node mounts on demand without a discrete pull step. This eliminates startup latency for workloads that access only a fraction of their image. Still, the container depends on the network throughout its lifetime and requires purpose-built serving infrastructure that may not be portable across providers.
Lazy loading: Instead of pulling the whole image up front, start the container immediately and fetch file content on demand. Projects in this space include eStargz and Nydus (which require converting the image to a new format) and AWS’s SOCI (Seekable OCI), which adds a seekable index alongside the unmodified image. These techniques work well when containers touch only a fraction of their data at startup. Still, ML images densely access the framework, CUDA libraries, and model weights before serving the first request, so nearly all the data ends up being fetched anyway. For these workloads, the pull pipeline itself needs to be faster.
Peer-to-peer distribution within the cluster: Tools like Dragonfly (a CNCF graduated project) and Spegel turn nodes that already have an image into seeders for nodes that need it, reducing registry egress and accelerating rolling deployments. For ML workloads, though, the first node still faces the full cold pull, and when images are rebuilt frequently, no node has the new version cached yet. P2P distribution complements rather than replaces improvements to the pull pipeline itself.
Out of the six stages in the pull pipeline, we focused on two: downloading the layer blob from the registry, and unpacking the layers on the node. These are where the most time is spent and where the serialization cost is highest.
Since SOCI was already an open-source containerd snapshotter plugin with the plumbing to intercept and customize the pull path, it served as a staging ground where we could develop and validate these changes before contributing them upstream to containerd.
containerd traditionally uses a single HTTP connection to download each layer. We identified that splitting a layer into fixed-size chunks and fetching them concurrently over separate connections using HTTP range requests was significantly faster. At the same time, the containerd community also independently added parallel chunked download support in containerd 2.1, and we built on the same principle in the SOCI snapshotter.
However, there is one significant difference that allows us to keep the runtime’s memory footprint constant regardless of image size. The difference is where chunks live between arrival and final assembly. Our implementation writes each chunk directly to the local disk the moment it arrives, while containerd holds it in memory as the layer is assembled. This means the runtime’s memory stays flat whether you are pulling a 1 GB layer or a 15 GB layer, which matters on GPU nodes where system memory is shared with model weights and CUDA contexts.
Once download completes in seconds rather than minutes, the compressed layer hash that previously hid behind it becomes visible. Because the layer is now a complete file on disk rather than a stream consumed once, integrity verification and unpacking can proceed at the same time.
After downloading, containerd decompresses and extracts each layer one at a time. This sequencing exists because in some filesystem backends, a later layer can overwrite files from an earlier one, so the order matters. The overlay snapshotter, which is the default on EKS and most Kubernetes clusters, sidesteps this constraint by keeping each layer in its own separate directory. The kernel mounts them together into one unified view only when the container starts. Because each layer extracts to an independent directory, unpacking one layer does not need to depend on another to finish.
With that established, we built an unpack path that decompresses and extracts all layers concurrently. The snapshotter detects whether the backend actually requires ordering and falls back to sequential if it does. For images with multiple large layers, total unpack time goes from the sum of all layers to roughly the time of the single largest one. We contributed this parallel unpack capability upstream to containerd v2.2, so the improvement is available to the broader community.
With chunked parallel download, a large layer that previously took over a minute on a single connection finishes in single-digit seconds. When you then unpack all layers concurrently rather than sequentially, the full pipeline for a large ML image compresses from several minutes to well under a minute on instances with fast local NVMe storage. The machine spends its available compute and bandwidth actively pulling rather than waiting on serial stages. On larger images and instances with faster storage, the gains are more pronounced because the gap between available hardware capacity and what a single connection can use is wider.
The two changes above address download and layer sequencing, but decompression of a single large layer remains serial. On a 64 vCPU machine, decompressing a layer that expands to 18 GB means a lot of compute sitting idle. Two opportunities stand out:
Parallel decompression within a single layer. Libraries like rapidgzip can locate block boundaries and inflate blocks across cores in parallel.
Parallelizable integrity verification. Today, the layer hash requires a single sequential read over the entire compressed blob after all bytes have landed. A tree-structured hash like BLAKE3 would allow computing the layer digest from independently hashed chunks so that verification could run in parallel with download rather than as a separate pass afterward.
Addressing these opportunities will squeeze out the last remaining serial stages in the pull path, bringing total pull time closer to what the raw hardware is capable of delivering.
This image pull optimization is enabled by default on G/P/Trn instances with EKS Auto Mode. You can also benefit from this on other node types through one of two mechanisms:
EKS AL2023 and Bottlerocket AMIs that ship with containerd 2.2 do not enable this feature by default, but you can set the containerd config explicitly as shown below:
AL2023: add the containerd config through the nodeadm NodeConfig in user data:
apiVersion: node.eks.aws/v1alpha1
kind: NodeConfig
spec:
containerd:
config: |
[plugins.'io.containerd.transfer.v1.local']
max_concurrent_downloads = 20
concurrent_layer_fetch_buffer = 16777216
max_concurrent_unpacks = 5
Bottlerocket (K8s 1.36): Bottlerocket generates the same containerd config from its settings API, so set the equivalent keys in user data:
[settings.container-runtime] max-concurrent-downloads = 20 concurrent-download-chunk-size = 16777216 max-concurrent-unpacks = 5
SOCI snapshotter is bundled in the optimized EKS AMIs (AL2023 and Bottlerocket).
Bottlerocket: enable SOCI through EC2 user data:
[settings.container-runtime] snapshotter = "soci" [settings.container-runtime-plugins.soci-snapshotter.parallel-pull-unpack] max-concurrent-downloads = 20 concurrent-download-chunk-size = "16mb" max-concurrent-unpacks-per-image = 5
Tune chunk size and concurrency under [settings.container-runtime-plugins.soci-snapshotter.parallel-pull-unpack]; the right values depend on your instance type and images.
AL2023: enable SOCI through the nodeadm FastImagePull feature gate, which switches image pulls to SOCI’s parallel-pull-unpack mode, and you can override the SOCI tuning parameters through user-data:
apiVersion: node.eks.aws/v1alpha1
kind: NodeConfig
spec:
featureGates:
FastImagePull: true
The image pull problem was never really about the registry, and it was never about needing faster hardware. The network, the storage, and the CPU were always there. When containerd’s pull pipeline took shape, images were measured in hundreds of megabytes, and the sequential approach served that world well.
“The image pull problem was never really about the registry, and it was never about needing faster hardware.”
As AI and ML workloads pushed images past 20 GB, the gap between available hardware throughput and what the pull path was utilizing became impossible to ignore. For workloads running on the overlay snapshotter with high-bandwidth instances and fast local storage, parallelizing download and unpack closes most of that gap today. Other snapshotters and storage backends may have different constraints, and opportunities like parallel decompression and parallelizable integrity verification remain open.
We have been contributing these changes upstream because this is where they belong: in the runtime itself, available to everyone by default rather than locked behind additional software. Some of that work has already landed in containerd 2.2, and more is in progress.
If you are working on container runtimes, image formats, or compression tooling and any of the open problems described here interest you, we would welcome collaboration. The faster we can collectively close the remaining serial stages, the sooner multi-gigabyte images stop being a deployment bottleneck for the entire ecosystem.
The post Pulling multi-gigabyte container images in seconds on Amazon EKS appeared first on The New Stack.
“AI kill switch” entered the public conversation because it gives people a simple way to talk about a complex fear.
As AI systems become more autonomous and harder to evaluate with familiar operating assumptions, a clearly defined intervention capability sounds reassuring. If something starts behaving in a way that creates unacceptable risk, people want confidence that someone has both the authority and the mechanism to stop it. It’s the “kill switch.”
Recent reporting around OpenAI models escaping a sandboxed testing environment and reaching Hugging Face gave that concern a concrete example. CNBC reports that the incident helped trigger a bipartisan bill requiring certain AI companies to maintain the ability to shut down, throttle, or suspend their models, with the Department of Homeland Security given authority to order a slowdown or shutdown in cases involving potential catastrophic harm.
The political reaction is understandable. When a new category of risk surfaces, especially one the public does not yet know how to evaluate, leaders look for a way to make an abstract concern into something actionable. In this case, that language has formed around shutdown authority.
People who operate large environments tend to hear a different question underneath the policy language. If a shutdown order arrives, what exactly gets shut down?
In a modern production environment, answering that question usually means tracing more than one system. An AI-enabled service may depend on endpoints, APIs, cloud resources, identity systems, package registries, data pipelines, workflow automation, logging tools, and downstream applications that act on model output.
Some of those dependencies may belong to different teams. Others may sit outside the company entirely. A few may have started as experiments and later become part of a production path without receiving the same scrutiny as the original architecture. By the time the service is important enough to raise governance concerns, it may no longer resemble a bounded application with a single owner and a clean operating surface.
Writing shutdown authority into legislation is far simpler than carrying that decision through a production estate shaped by years of migrations, exceptions, integrations, acquisitions, temporary fixes, and team-level decisions. That implementation gap is where the issue becomes most relevant to infrastructure teams.
For the last several years, much of the AI safety conversation has focused on acceptable use, privacy, model behavior, and human-in-the-loop oversight. Those topics still deserve attention, especially as organizations formalize where AI may be used, which data can be shared, and how employees should evaluate generated output.
“Writing shutdown authority into legislation is far simpler than carrying that decision through a production estate shaped by years of migrations, exceptions, integrations, acquisitions, temporary fixes, and team-level decisions.”
As AI moves deeper into production workflows, the discussion also needs to involve a more practical concern: when an AI-enabled system creates unacceptable risk, can the organization understand the affected environment well enough to constrain it quickly, consistently, and with evidence?
The phrase “kill switch” may drive the public discussion, but the practical answer lives in the systems surrounding the AI capability.
Emergency stops are the kind of control most people picture when they hear the phrase “kill switch.” They make sense in physical systems. Manufacturing equipment, industrial machinery, and certain safety-critical devices can be designed with direct shutdown mechanisms. Software estates already stretch that metaphor, and enterprise AI stretches it further.
The model may be the most visible part of the discussion, although it is rarely the full surface area. An AI assistant used in software delivery might have access to repositories, CI/CD tools, artifact stores, ticketing systems, secrets, test environments, and deployment workflows. An AI agent used in IT operations might read telemetry, recommend remediation, open change requests, call automation scripts, or modify infrastructure through approved orchestration paths.
Stopping one part of that chain can leave other paths untouched. Turning off a service may not revoke the credentials it uses. Suspending inference may leave downstream systems acting on outdated outputs. Interrupting the wrong dependency can create a separate service incident while the original risk remains only partly contained. Anyone who has worked through a security incident, emergency patch cycle, or major outage knows how quickly a clean decision turns into a sequence of technical tradeoffs.
A credible response plan must account for the system as it exists now, not as it looked during an architecture review months earlier. Infrastructure teams bring useful skepticism to that exercise because they are used to tracing scope, access, ownership, dependencies, and verification paths under pressure. They also know that many environments contain a gap between documented intent and production behavior.
Once AI is embedded in business workflows, those operational details become part of the governance conversation. They expose the places where policy language has moved faster than the infrastructure knowledge needed to make policy executable.
Much of the public conversation assumes organizations know where AI is running, which is a generous assumption in many enterprise environments.
AI can enter an enterprise through obvious channels, such as internally approved model providers or purpose-built applications. It also arrives through less visible paths. A SaaS product adds an AI feature. A development team experiments with an open source model. A hosted API gets attached to an internal tool. A vendor introduces an AI capability inside software the company already approved. Over time, the line between an “AI system” and a system that happens to use AI becomes harder to define.
This is where the kill switch metaphor starts to show its limits. If the relevant systems are discovered during the response, the team is already behind. Dependency questions, business impact, access paths, and evidence collection all become harder when the basic inventory is still being assembled.
“If the relevant systems are discovered during the response, the team is already behind.”
Infrastructure teams have seen versions of this problem before. During incident response, a service thought to be isolated turns out to have undocumented consumers. During a cloud migration, a supposedly unused integration is suddenly linked to a business process. During an audit, ownership records, configuration data, and actual operating conditions refuse to line up cleanly. AI adds a new category of concern, but the underlying visibility problem is familiar.
The challenge quickly expands beyond the model itself. Teams need to understand which systems call external models, where generated content influences workflows, which accounts and automation paths sit between a recommendation and an action, and how third-party AI capabilities have found their way into the environment.
Most discussions start with how to stop risky AI behavior. In many environments, the more revealing question comes earlier: can the organization produce a reliable picture of where AI touches the estate, which systems depend on it, and which workflows would keep moving if access changed?
Most infrastructure teams know that containment is less a single action than a set of operating conditions. The difficult work happens before the incident, when teams decide what should change if risk reaches a level that requires intervention.
Under ordinary circumstances, a service operates with a defined set of permissions, connections, dependencies, and logging requirements. Under restricted operation, selected assumptions change while investigators preserve evidence and determine whether the risk has been contained. A team might turn off endpoints, suspend integrations, limit external network access, revoke or rotate credentials, increase logging, or isolate workloads while the situation is being investigated.
The details vary because environments vary. That is exactly why generic answers tend to fall apart. A useful containment plan must match the systems it is intended to govern, including the dependencies that surround them and the business processes that rely on them.
Inventory sounds mundane until a response effort depends on it. In my work with infrastructure and compliance teams, I’ve repeatedly seen organizations struggle to maintain an accurate picture of their environments as cloud resources, Kubernetes clusters, SaaS services, and AI projects multiply faster than governance processes can track them during normal operations, which creates audit and support headaches. During a containment event, those same blind spots slow investigations, complicate dependency analysis, and make evidence harder to produce.
Governance efforts often begin with documentation. Committees are formed to define responsibilities, agree on escalation paths, and establish a common language for discussing risk before an incident forces the issue.
The conversation shifts once someone asks whether the control can be demonstrated. A document can describe who has authority to suspend an AI-enabled service. Still, it cannot turn off an integration, revoke a credential, increase logging, or prove that a set of systems entered a restricted condition. A risk register can identify a containment scenario, although it cannot document which nodes changed, when they changed, and whether they remain aligned with the required configuration.
Security, compliance, and infrastructure teams know this gap well. It appears in patching programs, configuration baselines, incident response exercises, supply chain reviews, and disaster recovery planning. Written controls tend to be cleaner than the real-world environments they describe. Production systems reflect years of accumulated decisions, exceptions, migrations, temporary fixes, acquired assets, and workarounds that may outlive the original reasons they existed.
“Written controls tend to be cleaner than the real-world environments they describe.”
AI increases urgency because some systems are becoming more autonomous and more connected to business workflows. It also adds outside pressure. When an incident becomes visible enough to prompt legislative action, boards and customers start asking sharper questions. A company may be able to point to an AI policy, but boards, customers, and regulators eventually want to understand exactly how that policy translates into action.
If leadership declares an AI-enabled workflow should be restricted, the discussion moves quickly from oversight to execution. Teams need to know where to intervene, which systems are affected, who owns the required changes, how completion will be verified, and what evidence remains once the response is over.
A vague answer may pass during experimental stages but becomes much harder to defend once AI is embedded in production services, regulated workflows, customer-facing systems, or environments connected to critical business operations.
A federal shutdown authority, if enacted, would place legal pressure on a narrow class of powerful AI providers. It would not remove the implementation burden for organizations that adopt, integrate, fine-tune, host, or embed AI systems inside their own environments.
Even if a major AI provider can throttle or suspend a model, each enterprise still must understand its own exposure in the context of its applications, workflows, dependencies, and operating assumptions. Which applications depend on that model? Which workflows fail open or fail closed if access is restricted? Which internal systems contain cached outputs, calculated decisions, or agent-created changes? Which business processes need manual fallback when an AI service is unavailable?
Policy debates often treat AI control as if the decisive action happens at the model layer. Sometimes it will; however, in many business environments, the risk will live in the connections that surround the model. A hosted AI service may be suspended while local workflows, scripts, integrations, and access tokens continue following their last known configuration.
“A serious AI containment strategy has more in common with mature infrastructure management than with an emergency stop button.”
A serious AI containment strategy has more in common with mature infrastructure management than with an emergency stop button. It requires an up-to-date inventory of AI-adjacent systems, a map of dependencies and access paths, predefined restricted conditions for high-risk services, tested procedures for applying those conditions, and evidence that changes were enforced. Ownership also needs to be clear, since fast action becomes difficult when authority is scattered across teams.
The work is less exotic than the public conversation can make it sound. AI-enabled systems still need to be managed as production systems with real dependencies and business impact.
Spend enough time running infrastructure, and you develop a complicated relationship with documentation. Most organizations have diagrams, inventories, and governance processes, and all of them serve a purpose. The challenge is that production systems keep evolving long after those artifacts are created. Acquisitions introduce systems that do not fit cleanly into existing models. Applications gain integrations nobody anticipated during the original design process. Temporary exceptions become permanent. Cloud resources intended to live for a week are still running after a year.
Most of this happens for defensible reasons, usually in support of uptime mandates, delivery pressure, customer needs, or business continuity SLAs. The result is that operational knowledge becomes dispersed across people (some of whom will inevitably have moved on), tickets, runbooks, monitoring systems, and memory rather than living neatly in one place.
Infrastructure teams spend their days tracing dependencies, untangling ownership questions, and figuring out how systems behave outside a design review. Bringing that perspective into AI governance conversations early can prevent containment plans from depending on assumptions that did not translate into production. It also helps organizations understand the difference between disabling a model, restricting access to a service, isolating a workload, and preserving evidence during an investigation.
Scale complicates things further. A manual action that works effectively for ten systems may fail across hundreds or thousands. A change one expert can perform during business hours may become fragile if that person is unavailable when an event occurs. A runbook that looks adequate in a tabletop exercise may not survive a live environment where dependencies have changed, and the current ownership is unclear.
The phrase “kill switch” will probably remain part of the public debate because it is simple, memorable, and familiar. Practitioners do not have to accept the metaphor literally to leverage the attention it creates. They can redirect the conversation toward more useful questions: what restricted operation would mean for a given service, which dependencies would have to change, which controls can be applied reliably, which steps remain manual, and how the organization would prove the response worked.
These questions are less dramatic than a big red button, but their answers are also most likely to improve readiness.
The Hugging Face incident gave the industry a vivid story, and Washington responded with the language of shutdown authority. That reaction is understandable. Leaders want mechanisms that sound equal to the risk, especially when the public conversation moves faster than the technical details can be explained.
By the time an organization begins thinking about containment, much of the hard work should already be done. Teams should already understand what is running, who owns it, what depends on it, and how changes will ripple through the environment.
AI can reduce certain workflow bottlenecks, but it also exposes weak inventory, unclear ownership, and brittle operating assumptions faster than many teams are prepared to handle. A future incident will not pause while teams locate assets, clarify ownership, identify credentials, or discover that a service dependency was never documented.
The current debate may be framed around new kill switches. For most organizations, the more useful work starts with building and maintaining an accurate picture of the systems, dependencies, and workflows that already exist across the estate.
The post The “AI kill switch” assumes you know what you are trying to shut down appeared first on The New Stack.
Cloudflare launched its CloudflareOS open-source AI workspace platform this week, promising every employee a secure workspace equipped with AI tools and access to internal company systems.
Positioned significantly beyond the notion of legacy virtual desktop infrastructure (VDI) services, which delivered the same fixed applications through a remote screen — and even past the dynamic application delivery, app masking and streaming of modern VDI iterations — this is an essentially more dynamic way of working with internal company tools, documents and systems.
Cloudflare’s CloudflareOS makes its apps and services accessible through secure connection points that verify every user and every agentic request or connection point before access is granted.
The technology proposition here is built on the fundamental truth that the typical enterprise AI tool knows a great deal about the world, but almost nothing about how a specific company operates, the shape of its internal systems, approval processes, or the ways teams actually get work done.
That means every new work session starts from zero, with employees re-explaining context the AI should already know. But how can new business context-aware agentic access freedoms be granted securely?
Rita Koslov, VP for developers & AI at Cloudflare, tells The New Stack that powering up modern agent use cases means “data is often leaving controlled systems en masse” for the first time.
“It used to be the case that, for example, people asked analytics questions in the data warehouse where the organization had control,” Koslov says. “Now, employees are asking for API keys for their own tools, agents, etc. This creates a new class of security problems that Cloudflare OS helps to solve.”
Cloudflare has built what we can call capability-based access, which the company promises beats handing agents raw API keys outright.
“API keys give agents broad access to systems; a capability-access-based approach lets us grant one specific resource, then record exactly what the agent observed, and verify that anyone who sees its work is also allowed to access the source,” underlines Koslov.
Cloudflare OS enables an agent to create documents, slides, spreadsheets, workflows, other agents – or entirely new full-stack applications – all tailored to an employee’s work. What it creates can remain connected to live data sources, be modified and shared safely, and be used directly by both people and agents.
“API keys give agents broad access to systems; a capability-access-based approach lets us grant one specific resource, record exactly what the agent observed, and verify that anyone who sees its work is also allowed to access the source.”
In terms of how developers and systems operations professionals should react to this offering, Koslov suggests that “the difficult problem is not generating an app” today. Instead, the real challenge is safely running thousands (or millions) of dynamically generated apps, each with persisted state and controlled access.
“Cloudflare OS uses Dynamic Workers, which provide lightweight isolated runtimes to load each app’s code on demand, and Durable Objects Facets to give it isolated SQLite storage under the platform’s supervision. Outbound networking is disabled by default, and Gatekeepers expose only the resources explicitly granted by the users,” Koslov says. “Dynamic Workers and Durable Objects Facets were invented because doing this was previously not possible.”
For completeness here – and once again a Cloudflare original technology service – a Gatekeeper is a service-specific Worker that sits between Cloudflare OS and an external service to interpret and understand the service’s API, its resources, and the operations that can be performed on them.
Koslov confirms that she knows how badly things can skew out of control in unmanaged environments.
“We know this from our own experience talking to other companies on all accounts. They’ve shared instances of internal data copied into AI tools that IT did not know were in use, AI keys embedded into agent-built applications, and even data being shared internally to people who ordinarily wouldn’t have access (or even publicly),” she adds.
Building a tailored alternative is no small project; a platform with proper security and real integration into internal systems can take years to develop and cost millions to maintain. In the meantime, employees find workarounds, IT loses track of which AI tools are running and who is using them, and costs pile up, often with little to show for it.
CloudflareOS starts from a different premise: a company captures its knowledge, processes, and ways of working once in a form AI can actually execute, and that knowledge travels with every employee’s workspace from day one.
“Captured business ‘context’ in this case can include company terminology, policies, operating procedures, product documentation, technical standards, sales processes, templates, and established ways of performing recurring work,” confirms Koslov.
CloudflareOS started as the platform Cloudflare built to run its own workforce. Thousands of Cloudflare employees across every team use it daily to perform research, create documents connected to live data, automate repetitive tasks, and build working apps for their day-to-day jobs.
That same platform is now available to any organization as open-source software. Because it’s open source and runs in a company’s own Cloudflare account, organizations own what they build on it.
The platform itself works on any AI model and controls cost. Through Cloudflare AI Gateway, organizations can use any AI model provider, so they’re not locked into one vendor. Administrators see exactly what’s being spent, broken down by person, team, or app. They can set spending budgets, rate limits, or route routine tasks to smaller, more affordable models where a top-tier model isn’t needed.
Cautiously upbeat about the wider story playing out here, enterprise AI architect and founder of Besk Tech, Vladimir Beskorovainyi, tells The New Stack that, traditionally, the industry is pricing these platforms by the token, “and that is the wrong meter entirely” in his view.
“In this example with Cloudflare OS, what a company actually buys here is the obligation to write down how an AI-powered business process really works, and then keep that description true as the business shifts underneath it,” Beskorovainyi says. “The model is the commodity part. What costs real money is the curated context, and nobody budgets for the fact that it starts decaying the day it is written, which is exactly what decides whether any of this survives contact with production.”
“Cost broken down by person, team and app is the first time I have seen a vendor treat spend as an engineering signal rather than an invoice, and sending routine work to a smaller model is the obvious next step that most enterprises still fail to take.
Beskorovainyi insists that the organizations that win in this game will “not necessarily be the ones running the best model”; they will be the ones that could “already answer in writing what their own approval process is”, way before an agent ever asked.
“Cost broken down by person, team and app is the first time I have seen a vendor treat spend as an engineering signal rather than an invoice, and sending routine work to a smaller model is the obvious next step that most enterprises still fail to take,” advises Beskorovainyi.
He clarifies his point and explains that the qualification here is that “owning your own context is not the same thing as your context being any good”, and so open source tooling and community connections plus an organization’s own account settle who holds the context file.
“Neither tells us whether what is recorded and logged in the context file is still true this quarter. That work stays with the customer permanently, and it is where I expect most of these deployments to come apart, not in anything Cloudflare has built,” Beskorovainyi adds.
Matthew Prince, co-founder and CEO of Cloudflare has said that his team built Cloudflare OS, “because nothing else did what we needed”, and so now, any company can start from where it took the organization’s internal software engineering function years to get to.
The apparent appeal here must come down to the dynamic nature of Cloudflare OS and its ability to work with and apply AI tools at a custom-engineered business context-aware level with zero trust by default. The platform can turn any output into a working app with its own isolated database, real-time capabilities, and access controls – once agan, that’s not legacy virtual desktop is it?
The bottom line from Cloudflare is that employees can use any app on Cloudflare OS directly, or adapt it for their own needs so that it’s a case of “no developer required”, or at least until the next integration task needs to be shouldered, or the big thing comes along, or both.
The post Why AI tools know nothing about your company — until now appeared first on The New Stack.
Mergers, acquisitions, and the steady churn of business and technology initiatives are creating something nobody asked for: Duplicate infrastructure and expertise.
Here’s the typical split: A platform engineering team that owns cloud-native and Kubernetes workloads. Meanwhile, traditional IT holds the keys to virtual machine (VM) workloads. Two teams. Two domains. One budget. And the costs keep going up.
Even organizations that talk about standardizing on Kubernetes still have a substantial VM footprint. For many teams, this coexistence isn’t a temporary transition state. It’s the operating model.
On-premises, this split forces two separate environments. Each environment includes networking, servers, and storage. Such duplication can be structurally less cost-efficient than consolidation. VM-based mission-critical workloads aren’t going away anytime soon.
It’s not like teams don’t want to modernize. They absolutely do. But it’s not as simple as just picking between old-school VMs or diving into Kubernetes. What’s really happened is these two worlds have grown up on their own.
That kind of split often leads to extra infrastructure, more people doing the same jobs, slower projects, mixed-up governance, and budgets that keep ballooning. And when you’re on-prem or working at the edge, running two separate setups for networking, compute, storage, and playbooks just doesn’t make sense anymore.
To make matters worse, “just rewrite” bares its fangs on the modernization initiative. Finance and executive leadership see two teams running two tech stacks. It’s only natural that they reach for the obvious fix: Pick one team’s platform, with no technology consideration, migrate everything to it, and watch the added cost disappear from the executive briefing slide and move to the CFO’s budget spreadsheet.
Over time, we learned from our customers that “rewrite it” isn’t a modernization strategy. Rather, it’s a budget, risk, and timeline strategy all at once. In many cases, rewrite it doesn’t make sense financially. The tech industry loves the idea of re-platforming and re-architecting legacy applications. Even then, such a move only returns your enterprise to square one and functional parity. The more realistic path is to keep mission-critical applications as-is when scaling out cloud-native platforms to deliver new value.
Moving everything to Kubernetes/cloud initiatives won’t prevent two platforms either. Such initiatives often stall because some workloads don’t fit or take far longer than planned.
The economics of rewrites don’t disappear just because AI accelerates software delivery. AI can compress the time it takes to write code. However, writing code was never the expensive part of a rewrite. The costs that dominate many rewrite budgets are judgment costs, and those remain stubbornly human.
Start with architecture. Organizations still need software engineering expertise to design the target system. That design problem has gotten harder, not easier. Cloud-native applications built on microservices for horizontal scaling bear little structural resemblance to the traditional enterprise applications they replace. Someone has to make those translation decisions and then spend the time directing the AI on what to build. That direction time is a real line item.
Validation is the next cost that survives. When customers or employees depend on a piece of software, even small behavioral changes are disruptive, making it non-negotiable to prove feature parity. Testing and validating that parity remains heavily human work. AI can generate test cases. It can’t tell you which broken workflow will cost you a customer.
Then comes the data. Teams must migrate and adapt data to the new system, and that work almost always surfaces complexities nobody scoped, including undocumented dependencies and format assumptions baked into decades of records. No amount of generation speed on the code side makes the data side move faster.
The rewrite math changes shape with AI. It doesn’t shrink to zero. The spend shifts from writing software to decision-making, verification, and migration.
The rewrite math changes shape with AI. It doesn’t shrink to zero. The spend shifts from writing software to decision-making, verification, and migration.
We see the same pattern repeat with rewrites among our customers. They keep mission-critical systems running as they are. Then they build new value with cloud-native applications in parallel. Modernizing selectively only when it’s truly worth it.
The gap we see isn’t philosophical — VMs versus containers — it’s operational. The tooling, workflows, and skills that define VM and cloud-native operations differ. If platform teams can’t deliver these services at the expected velocity, developers will blame the platform. When developers are accustomed to provisioning core services in minutes, any friction in on-prem or edge environments is perceived as the platform adding friction or slowing delivery.
The gap we see isn’t philosophical — VMs versus containers — it’s operational.
Historically, day-to-day operations in VM environments are UI-driven. Cloud-native environments are much more command-line interface (CLI) driven, where APIs, config files, and the terminal are the center of gravity. That gap becomes both an organizational and technical constraint. Moving from UI-driven operations to deep command-line interface (CLI)/config workflows isn’t a natural step without a significant shift in the team’s capabilities.
The operational gap shows up quickly in data services. Cloud-native workloads don’t just need compute. They need databases, object storage, file, and block services delivered at cloud-like speed. And despite the myth that containers are stateless, the reality is that most meaningful workloads have state somewhere as data, logs, metrics, or dependencies that must be handled consistently.
Another notable gap is that storage consumption differs:
The public cloud, by shaping cloud-native expectations, further contributes to the gap. Developers can click to get a database, such as Amazon Relational Database Service (RDS), and object storage, such as Amazon Simple Storage Service (S3), is just there. Developers expect this level of self-service simplicity when these platforms are extended beyond the public cloud, which isn’t always something platform teams are prepared for.
Today, edge and disconnected environments, such as air-gapped computing, have moved from niche use cases to mainstream constraints. When connectivity is intermittent or when latency matters, platform assumptions change. In these environments, reliability isn’t an IT metric. It’s a business outcome. Even minutes of downtime can cause major financial loss. It’s also a sign that data gravity is driving more pragmatic architectural conversations about the growing need to locate compute and data services closer to where data is generated.
AI raises the stakes further. If you’re collecting data at the edge, shipping it away for processing and pulling results back can be too slow and too expensive.
Before we’d bet on any platform, we’d ask a basic question: Can a single team operate both VM and Kubernetes environments without duplicating the entire organization? We’d insist on consistent governance: security controls and role-based access control (RBAC) should not fracture just because workloads are deployed differently.
We’d also look for cloud-like data services — object, file, block, and database capabilities — delivered quickly enough to keep developers moving toward their delivery targets, and designed to scale easily as application usage expands.
Then we would evaluate whether the platform helps reduce on-prem duplication. If it forces parallel networking, storage, and operational runbooks, the cost structure won’t improve.
Finally, we’d scrutinize lifecycle operations, including patching, upgrades, and maintenance, because “heroic” weekend work isn’t a sustainable strategy.
We use “dual native” to reject the binary choice. Enterprises need platforms that are both VM-native and container-native. Some workloads benefit from the operational efficiency of virtualization. Others are sensitive to latency or specialized hardware and are better served on bare metal. A one-size-fits-all mandate creates friction on both sides.
Dual native platform architecture isn’t just integration. It’s the one operational model that treats VMs and containers as first-class citizens. Teams no longer have to pick one architecture or stitch together separate stacks. In this model, organizations can keep mission-critical VM workloads running while building and scaling new cloud-native applications. Teams can maintain consistent management, governance, lifecycle operations, and cloud-like data services across VMs and bare metal servers across globally distributed infrastructure.
Nutanix Kubernetes Platform (NKP) solution with NKP Metal, which extends the Nutanix operating model and the NKP solution, supports Kubernetes deployments directly on bare-metal infrastructure. This solution provides unified Kubernetes operations, shared data services, centralized visibility, and automated bare-metal lifecycle management to support a dual native platform architecture.
Our approach with NKP starts with the premise that VM and bare-metal Kubernetes should operate under a consistent model rather than be split into separate toolchains and teams. To that end, a major focus has been on unified data services across deployment targets so the storage layer doesn’t become the breaking point when workloads span VMs and bare metal. We also purposefully centralize day-to-day operations and visibility across VMs and Containers in NKP so teams aren’t forced to manage two worlds with two separate management planes.
NKP Metal addresses lifecycle management, one of the biggest challenges of running bare metal at scale, including host OS setup, patching, and upgrades without resorting to late-night or holiday/weekend manual maintenance windows.
Some things we know with confidence. VM workloads aren’t disappearing — the coexistence of VMs and containers will remain the operating model for many enterprises well into the next decade. Edge and AI workloads will likely continue to pull compute toward where data is generated, and budget pressure on duplicated infrastructure will likely only intensify.
What we don’t know is the pace. How quickly enterprises consolidate two platform teams into one depends on skills, internal politics, and licensing decisions, which vary widely from one enterprise to the next. Nobody can credibly predict a timeline there.
What we think is coming: AI inference at the edge will make bare metal a first-class deployment target rather than a special case, and platform teams will be judged less on which architecture they picked and more on whether developers can self-serve their own infrastructure, including data services, without opening a ticket.
The path forward is about building an operational foundation that accepts reality where VMs, containers, and bare metal coexist under a unified model.
The path forward is about building an operational foundation that accepts reality where VMs, containers, and bare metal coexist under a unified model. Enterprises that will thrive in this future are those adopting dual-native approaches that are ready for whatever comes next.
The post “Just rewrite it”: What platform teams really think about modernization appeared first on The New Stack.
Consider a platform team managing a shared GPU cluster with a mix of B200s, H100s, and recently added B300s. Every Monday morning, the on-call engineer finds a queue of pending jobs from the weekend. Training workloads are stuck because they landed on H100s and triggered Out-Of-Memory (OOM) errors. Inference jobs sit idle because the small MIG (Multi-Instance GPU) slices are exhausted, even though larger slices sit empty right next to them.
Their fix? A 200-line Bash script running every 30 minutes to reconfigure MIG profiles, reschedule stuck jobs, and send a Slack alert when it succeeds, or a PagerDuty alert when it fails.
Here is what was actually broken: Kubernetes treated every GPU as an identical unit. The resource limit nvidia.com/gpu: 1 was the extent of its awareness. The scheduler had no idea if it was handing a pod a 192GB B200 or an 80GB H100. A training job requiring 150GB of VRAM would land on an H100 and immediately OOM, while B200 nodes sat completely idle nearby.
“Kubernetes treated every GPU as an identical unit.”
The industry’s accepted “fix” relied heavily on node labels, taints, tolerations, and separate node pools per GPU type. Every workload manifest hardcoded hardware assumptions. Adding a single new GPU generation meant updating 40 different Helm charts.
MIG made this worse. MIG slices a single GPU into smaller, isolated partitions, each with dedicated memory and compute. Instead of one inference job monopolizing a B200, you can run seven smaller jobs on the same card.
In theory, this sounds great. But when you enable MIG in Kubernetes, each profile becomes a separate, rigid resource type (e.g., nvidia.com/mig-1g.10gb, nvidia.com/mig-3g.40gb). There is no fallback logic. You cannot instruct a job to “try a small slice first, and use a large one if nothing else is free.” When small slices run out, jobs sit pending, while large slices go to waste.
“When small slices run out, jobs sit pending, while large slices go to waste.”
This inefficiency was accepted as the cost of running GPU workloads on Kubernetes. Then, Kubernetes 1.34 shipped.
Kubernetes 1.34 introduced Dynamic Resource Allocation (DRA), fundamentally changing the scheduling model. GPU drivers now publish structured data. Instead of requesting a generic nvidia.com/gpu: 1, workloads can express explicit intent using Common Expression Language (CEL):
“Give me an H100 or better with at least 40GB memory.”
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
name: h100-or-better-40gb
spec:
spec:
devices:
requests:
- name: gpu
deviceClassName: gpu.nvidia.com
count: 1
selectors:
- cel:
# Attribute names are illustrative.
# Your NVIDIA DRA driver must expose these fields.
expression: >
device.attributes["gpu.nvidia.com"].memory >= quantity("40Gi") &&
device.attributes["gpu.nvidia.com"].generation in ["H100", "B200", "B300"]
---
apiVersion: batch/v1
kind: Job
metadata:
name: training-job-h100-or-better
spec:
template:
spec:
restartPolicy: Never
resourceClaims:
- name: gpu
source:
resourceClaimTemplateName: h100-or-better-40gb
containers:
- name: trainer
image: nvcr.io/nvidia/pytorch:24.12-py3
command: ["python", "train.py"]
resources:
claims:
- name: gpu
“Give me a MIG slice: small if available, medium if not, or full GPU if needed.”
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
name: mig-prefer-small-then-medium-then-full
spec:
spec:
devices:
requests:
- name: gpu
deviceClassName: gpu.nvidia.com
count: 1
selectors:
- cel:
# Prefer any acceptable MIG profile or full GPU
expression: >
device.attributes["gpu.nvidia.com"].profile in [
"mig-1g.10gb",
"mig-2g.20gb",
"mig-3g.40gb",
"full-gpu"
]
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: inference-service-flexible-gpu
spec:
replicas: 2
selector:
matchLabels:
app: inference-service
template:
metadata:
labels:
app: inference-service
spec:
resourceClaims:
- name: gpu
source:
resourceClaimTemplateName: mig-prefer-small-then-medium-then-full
containers:
- name: inference
image: nvcr.io/nvidia/tritonserver:24.12-py3
args: ["tritonserver", "--model-repository=/models"]
resources:
claims:
- name: gpu
“Give me 4 GPUs that are NVLink-connected.”
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
name: four-nvlink-connected-gpus
spec:
spec:
devices:
requests:
- name: gpus
deviceClassName: gpu.nvidia.com
count: 4
selectors:
- cel:
# Attribute names are illustrative.
# Some NVIDIA DRA setups may model this through ComputeDomains.
expression: >
device.attributes["gpu.nvidia.com"].fabric == "nvlink"
constraints:
- requests: ["gpus"]
matchAttribute: "gpu.nvidia.com/nvlinkDomain"
---
apiVersion: batch/v1
kind: Job
metadata:
name: distributed-training-nvlink
spec:
template:
spec:
restartPolicy: Never
resourceClaims:
- name: gpus
source:
resourceClaimTemplateName: four-nvlink-connected-gpus
containers:
- name: trainer
image: nvcr.io/nvidia/pytorch:24.12-py3
command:
- torchrun
- --nproc_per_node=4
- train.py
resources:
claims:
- name: gpus
This architecture requires one manifest. It eliminates fragile node selectors and the need to duplicate job definitions for every new hardware generation. As GPU clusters become increasingly heterogeneous, mixing H100s, B200s, B300s, and whatever silicon drops next, the old integer-based scheduling model breaks down. DRA represents Kubernetes finally maturing to support the nuanced realities of production AI workloads.
The post Say goodbye to K8s GPU pain: How DRA changes everything appeared first on The New Stack.
A developer supervising four coding agents has four changes in flight at once, each in its own git worktree. That isn’t an exotic setup anymore: Anthropic’s documentation now treats a worktree per session as the default way to run agents in parallel, and what was an expert workflow two years ago is the recommended starting point today.
The branches themselves aren’t new. Git made them cheap 20 years ago so developers could isolate changes and work on several things at once, but in practice a developer switched between branches and shipped one change at a time. That kept everything below the code layer singular: one continuous integration (CI) queue, one staging environment, one database everyone tested against. The number of changes contending for those shared resources was capped by headcount, and before agents, only larger teams ever hit the cap.
“Coding agents removed the cap. The branch can no longer stop at the code layer.”
Coding agents removed the cap. Those four branches are no longer something one developer rotates through. They are four active changes moving toward merge in parallel. The gap becomes unworkable: branching is free at the code layer and missing everywhere below it. Each change needs to exist all the way down the stack, not as a diff in a directory but as a running, testable version of the system. The branch can no longer stop at the code layer.
Code branches in milliseconds. A worktree gives each agent a private copy of the repository for the cost of a checkout, and 10 agents can work side by side without seeing each other’s edits.
The output shows up downstream. Telemetry from Faros AI across more than 10,000 developers found that teams with high AI adoption merge 98% more pull requests while review time grows 91%. Nothing downstream of code generation was sized for that arrival rate.
Then each change needs to run. There is one staging cluster, one seeded database, one message queue, one set of dependent services, and every branch that reaches this floor stops being parallel. Four agents produce four candidate changes in an afternoon, and all four line up behind the same shared environment to find out whether they work.
The queue is more expensive than it looks, because agents don’t wait well. An agent blocked on an environment either sits idle holding a stale view of the system or plows ahead validating against mocks, and the developer supervising it context-switches away. By the time the shared environment frees up, the cheap part of the work has to be partially redone.
The bottleneck isn’t code generation, and it isn’t review capacity alone. It’s the first shared resource a change touches, because a branch that can’t run is a branch that can’t be trusted.
“The bottleneck isn’t code generation, and it isn’t review capacity alone. It’s the first shared resource a change touches.”
The way out is to stop treating branching as something git does and start treating it as something every layer does. Branch-based development names the pattern: each layer of the stack offers a cheap, instant, disposable branch primitive, so a change can exist end to end without duplicating anything it didn’t touch.
The mechanic is the one git established, and everyone has been living on for two decades: branches are cheap because they share everything unchanged and carry only the delta. The rest of the stack has been relearning that idea layer by layer ever since — share by default, isolate what changed.
Naming the pattern matters because each layer discovered it separately and called it something different. Worktrees, pipeline caching, preview deploys, database branching, and environment sandboxing sound like five unrelated features. They’re the same idea applied at five layers, and seeing that changes what you ask of the layers that lack it.
CI absorbed the lesson a decade ago. Every branch gets its own pipeline run on a shared runner pool, with build caches doing the copy-on-write work of reusing unchanged artifacts. Nobody provisions a build system per branch, and nobody queues behind a single global build anymore.
The front end followed. On Vercel, every push to a non-production branch gets its own preview deployment by default; Netlify works the same way, and the branch itself is one immutable build plus routing on shared hosting infrastructure. Reviewers stopped asking whether a change works on someone’s laptop, because the change is already running somewhere.
Both cases have the same shape: the expensive machinery is shared, the branch is thin, and creating one is cheap enough that nobody thinks about it. That’s what a layer feels like once it has a branch primitive.
Each of these primitives also changed behavior once it arrived. Per-branch CI made it normal to run the full test suite on every push instead of nightly. Preview deploys made it normal for a product manager to click through a change before merge. Cheap branches don’t just remove a queue; they raise the bar for what gets checked before merge.
Databases carry state, so conventional wisdom said branching would never work there. Then Neon, PlanetScale and Xata shipped it anyway, and Neon’s documentation now makes the parallel explicit: branch your data the same way you branch your code.
A database branch is a copy-on-write view over shared storage pages, created in seconds regardless of how large the database is. Schema migrations and risky data changes get validated against production-shaped data instead of a stale seed script, and the branch disappears when the work merges.
“If the layer with the most state can hand out branches in seconds, statelessness was never the real requirement.”
The data layer matters to this story because it removed the best excuse. If the layer with the most state can hand out branches in seconds, statelessness was never the real requirement. Whatever is still unbranched is unbranched by choice.
The microservices runtime resisted longest because it looks nothing like a file tree. It has live traffic, a service graph and dozens of moving dependencies, and the naive branch, a full copy of the environment, is so expensive that most teams concluded branching did not apply here.
The copy-on-write move works anyway. Run one shared, stable version of the system that is continuously deployed from main. For each change, deploy only the services the change touches as a lightweight ephemeral environment, and route each test request through the changed services while everything else falls through to the shared stable versions. The environment branch costs roughly what the changed services cost, which is why one can exist for every change an agent produces.
Routing is the part that sounds exotic and isn’t. A request tagged with a label gets steered to the changed service versions at each hop, propagated through the call chain the same way trace context already flows through most instrumented systems. The shared stable environment plays the role of main, the changed services are the delta, and the label is the pointer that assembles a coherent view of the system per request.
This isn’t a hypothetical architecture. Uber built SLATE to give each developer an ephemeral environment routed against shared production-grade dependencies because contention over staging could not keep up with its developer count.
Put the layers together and a different development model appears. An agent picks up a task, and the change gets a worktree, a pipeline run, a preview, a data branch, and a running environment from the start. Validation stops being the scarce resource that serializes everything upstream of it.
Teams are already composing the lower layers. Bitso, a crypto exchange with 250-plus engineers, pairs an environment branch with a database branch for each change, so the runtime delta and the data delta travel together and shared staging stays out of the critical path.
That end-to-end branch is what the phrase agent-native software development lifecycle should mean. Not agents wired into yesterday’s pipeline, but a stack where any change, human or machine, can exist at every layer for as long as validation takes and disappear afterward.
The payoff compounds with agent count. When the branch primitive at every layer is a delta over something shared, validation concurrency scales with cluster capacity instead of with budget, and the number of changes a team can prove correct per day rises with the number it can generate. That is the ratio that decides whether agent adoption shows up as shipped software or as a longer queue.
The audit is cheap to run. Follow one change from worktree to validated and note the first layer where it waits on something shared. That’s where your stack stops branching.
For most teams, the answer is the runtime, and if it’s yours, Signadot is a practical place to start.
The post Anthropic recommends a git worktree per agent. Your runtime infra makes that a problem. appeared first on The New Stack.
At the start of 2025, investors wondered whether Google could keep pace with OpenAI. By December, Alphabet was completing its best year on the stock market since 2009, helped by growing confidence in Gemini and Google’s broader AI strategy.
Much of that work came out of DeepMind, the British AI lab Google acquired in 2014 for about £400 million ($659 million in 2014). Now, Google is changing its leadership, while four of its best-known engineers are leaving to start an automated research lab.
Google announced Wednesday that DeepMind founder Demis Hassabis will step away from the lab’s day-to-day operations to become chair of Google DeepMind and chief scientist of Alphabet. Koray Kavukcuoglu, DeepMind’s chief technology officer and Google’s chief AI architect, will take control of Gemini model development, frontier AI research, the Gemini app and its developer teams.
At the same time, Jeff Dean, Sanjay Ghemawat, Oriol Vinyals and Quoc Le are also leaving Google to start Discovery Loop. As a public-benefit corporation, the company wants to use AI to automate scientific and engineering research. Google will remain involved as a founding investor and cloud provider. Just one person will now oversee everything from research to products.
In a January interview for CNBC’s podcast The Tech Download, Hassabis described DeepMind as the “engine room” of Google’s AI efforts. Hassabis said DeepMind develops Google’s core AI technology before it is distributed across the company’s products.
To get new AI into products faster, Google had to do more than update its models. DeepMind spent years reworking Google’s infrastructure so its AI work could move out the door quicker. That infrastructure push has extended to silicon, too — Google recently bet its inference future on a chip built for one model, a sign of how tightly the company is coupling hardware to its Gemini roadmap.
Hassabis said Google never struggled to invent new technology. After all, its researchers came up with the transformer architecture, which became the backbone of large language models. The problem was turning that research into products fast enough.
And moving faster is exactly what Google did. Hassabis pointed to Gemini 2.5, which landed in March 2025, as a turning point. Gemini 3 came out in November and put Google back in the mix with OpenAI and Anthropic — both of which have been making aggressive moves of their own to capture developer share.
By January, Hassabis said he and Google CEO Sundar Pichai were speaking almost every day, sometimes adjusting product plans and research roadmaps daily.
Kavukcuoglu has been at DeepMind for 13 years, started the deep learning team, and worked on projects like WaveNet and DQN. Now, model research, the Gemini app, and developer products are all on his plate. The flagship version of Gemini 4 remains unreleased after a planned June launch, according to Reuters. Hassabis confirmed the model’s name in his note to employees, saying Google was making progress on Gemini 4.
Dean joined Google in 1999 and helped create Google Brain before becoming a technical co-lead for Gemini. Working with Ghemawat, he developed systems including MapReduce, Bigtable, and Spanner. Dean was one of the primary designers of TensorFlow, while Ghemawat worked on the infrastructure behind Google Search and several generations of its distributed computing systems.
Vinyals and Le made foundational contributions to deep learning and Gemini. Along with former OpenAI chief scientist Ilya Sutskever, they co-authored the influential 2014 paper that introduced sequence-to-sequence learning with neural networks. Their new company plans to use that experience to work on machine learning itself, at least to start.
“We are building AI solutions that can automatically solve important problems in machine learning, science, and engineering,” Discovery Loop says on its website.
“We are building AI solutions that can automatically solve important problems in machine learning, science, and engineering.”
The idea that AI can automate scientific discovery is no longer speculative. OpenAI’s Astra recently proved 10 long-standing math and science theorems for about $2,000 in token costs — a data point that suggests Discovery Loop is entering a space where early results are already landing.
Discovery Loop is built around the idea that AI can automate the entire experimental cycle. At Y Combinator‘s Startup School in July, Dean said the system could run the entire process, from proposing and carrying out an experiment to evaluating the results and deciding what to test next.
Ghemawat told Wired that Google’s systems were designed to support products such as Search, advertising and large consumer applications. Discovery Loop wants to build specialized infrastructure around research instead.
“We wanted to build something differently than how things are built at Google right now,” he said.
“We wanted to build something differently than how things are built at Google right now.”
Rather than cut ties with the departing engineers, Google is investing in Discovery Loop and has signed a cloud partnership to provide the startup with computing capacity. That arrangement gives Discovery Loop access to the infrastructure needed to run large numbers of experiments without first building its own data centers. It gives Google a stake in anything the new company discovers and another major AI workload for Google Cloud.
Rather than cut ties with the departing engineers, Google is investing in Discovery Loop and has signed a cloud partnership to provide the startup with computing capacity.
Google has already lost Gemini co-leader and transformer co-author Noam Shazeer to OpenAI and AlphaFold researcher John Jumper to Anthropic. Worth noting, Alphabet shares fell more than 5% following Wednesday’s announcement.
Hassabis, meanwhile, will focus more of his attention on long-term AGI strategy and Isomorphic Labs, the drug-discovery company spun out of DeepMind.
In his message to employees, he said AGI now feels “close at hand” and that he wants more time to influence what happens next — a sentiment that carries extra weight as some of the most powerful AI labs face growing pressure to slow down.
The post Google’s four AI departures: “We wanted to build something differently” appeared first on The New Stack.
Presented by Tata Communications
Continuous inference, agent-to-agent communication, and real-time data pipelines are generating unpredictable, always-on traffic that legacy architectures were never built to support. As AI moves from pilot project to operational backbone, the network is emerging as a critical control layer that determines performance, reliability, and cost.
The shift is forcing organizations to question assumptions that have held for decades. Legacy systems were static and rigid, and lacked the ability to manage network demand efficiently or dynamically, while AI-ready networks need to adapt in real time. A study by Cisco notes that 80% of executives believe their company’s competitive survival will depend on agentic AI, and consumer usage of AI is already prevalent and accelerating. This is driving a fundamental shift in how traffic is generated, distributed, and experienced, with implications for service providers and enterprises that manage large-scale networks.
This infrastructure gap is a global concern. A recent Bloomberg study, "The Future-Ready Enterprise," commissioned by Tata Communications, found that while 3 in 4 leaders consider AI a board-level priority, nearly two-thirds (65%) of enterprises continue to operate on transitional or legacy infrastructure. This disconnect between ambition and reality is a primary obstacle to realizing value from AI investments.
The performance bar has also moved by an order of magnitude. Traditional business applications could tolerate 100 to 500 milliseconds of latency, while mission-critical AI workloads now require latency below 10 milliseconds.
"This isn't just an incremental improvement," says Kapil, Vice President, Global Network Services at Tata Communications. "It's a completely different performance paradigm that breaks traditional network design assumptions, where such extreme low latency was never a primary consideration."
That gap between what legacy infrastructure can deliver and what AI demands turns network performance into a direct driver of AI reliability and cost. Treating the network as a best-effort transport layer introduces risk that many organizations only discover once a deployment underperforms in production. A model built for real-time fraud detection or supply chain optimization becomes worthless the moment network congestion delays the data it depends on, and Kapil notes that every millisecond of that delay can carry a direct financial or operational cost.
"Relying on a 'best-effort' network turns multi-million-dollar AI stack investments into a high-stakes gamble, where performance is left to chance," Kapil says.
He adds that businesses often underestimate the complexity of using the public internet as a global enterprise network. Performance may look acceptable within a single country, but once data starts crossing borders or connecting to international cloud platforms, the lack of end-to-end control becomes an operational barrier.
Complexity compounds as AI components spread across cloud, edge, and enterprise environments. Organizations often focus on compute power and data infrastructure while overlooking the network fabric that connects them. That blind spot often surfaces as a performance bottleneck created by high-frequency east-west traffic moving between GPUs.
Distribution also widens the surface enterprises have to defend. Applications, users, and partner ecosystems are now spread across cloud, SaaS, edge, and device environments, and Kapil notes that AI-driven malicious bots account for roughly 37 percent of online traffic, making it increasingly difficult to distinguish legitimate users from automated threats. Many enterprises have responded by layering on siloed tools, which has produced fragmentation, inconsistent security, and a lack of unified visibility rather than a coherent defense.
"SASE helps mitigate these risks by converging networking and security into a unified, cloud-delivered architecture," Kapil says. "This convergence is enabling consistent policy enforcement across cloud, on-premises, and edge environments, while supplying the scalability and proximity needed to secure real-time AI-driven interactions."
Closing that gap requires organizations to gain far greater visibility into how AI traffic moves across distributed environments and the ability to direct workloads accordingly. Kapil says that demands a different approach to network management.
"Leaders must realize that the network is no longer passive 'plumbing.' It must be managed as an active, intelligent platform foundational to the entire AI stack," he says. "That platform requires real-time observability into how and where AI traffic flows, paired with the control to orchestrate workloads across the most efficient and secure path available."
It's the difference between merely connecting systems and unlocking new capability, for instance a seamless shopping experience during a peak sales period or a global sports broadcast streamed without buffering.
This intelligence also changes how infrastructure teams spend their day. The network itself is now software-defined and API-driven rather than fixed by hardware configuration, which Kapil says shifts infrastructure teams away from reacting to outages and toward designing the systems that prevent them.
"Instead of manually re-routing traffic during an outage, the team must define the rules, policies, and business outcomes for an intelligent fabric," Kapil says. "The network itself then executes those policies automatically and autonomously."
Tata Communications is putting this principle into practice with its recently launched IZO Data Centre Dynamic Connectivity. The software-defined platform creates a “self-healing, intelligent network” using deterministic multi-path routing to reroute traffic automatically in seconds during a disruption.
The company says the platform transforms resilience from a reactive process into an autonomous capability, providing the predictable, low-latency performance mission-critical AI applications require while reducing operational costs by up to 30%.
Delivering on that intelligence in practice means giving mission-critical workloads dedicated capacity rather than having them compete for it. Reaching that level of consistency also requires enterprises to define performance far more precisely than they have in the past. It's the shift from vague goals like "high performance" toward deterministic performance criteria where an organization commits to a guaranteed service level, such as latency for a specific workload not exceeding 10 milliseconds 99.999% of the time, for instance.
That same demand for predictability extends into capacity planning. As AI workloads become larger and more dynamic, networking infrastructure must be able to absorb rapid shifts in demand without sacrificing performance or efficiency.
"Without dynamic scalability, enterprises are forced into a false choice: either risk performance-killing congestion or engage in massive, inefficient overprovisioning of their network 'just in case.' This is incredibly expensive and unsustainable," Kapil says.
Building this foundation for the world's most demanding AI workloads is already underway. For example, Tata Communications is collaborating with Amazon Web Services (AWS) to build one of India’s largestAI-ready networks. This high-capacity, resilient network will connect major AWS infrastructure locations in Mumbai, Hyderabad, and Chennai, providing the ultra-low latency backbone needed to accelerate generative AI adoption and cloud innovation across the country.
He points to a consumption-based model, where software allows bandwidth and network functions to scale instantly with demand, as the operational alternative, since it lets organizations pay only for what they use while still protecting performance during spikes.
CIOs and infrastructure leaders need to reframe the network, not thinking of it as a cost center but as something closer to an insurance policy for an organization's broader AI investment portfolio. An intelligent network de-risks those investments in three ways:
enabling dynamic scalability that removes the need for overprovisioning
strengthening security and governance through the visibility needed to protect data and models
and providing a flexible, programmable foundation that can absorb future compute demands without a full architectural overhaul.
Getting there does not require enterprises to start from scratch.
Choosing a partner with a proven track record is critical. Tata Communications was recently named a Leader in the Gartner Magic Quadrant for Global WAN Services for the 13th consecutive year, reflecting its completeness of vision and ability to execute. That recognition reflects continued investment in areas such as SASE capabilities for AI-driven security and high-capacity 800G services designed for AI-scale infrastructure.
"We recommend a phased approach that begins with assessing the current state of the network and identifying inefficiencies, then prioritizing upgrades in areas such as AI-ready technologies, seamless data exchange, and advanced security solutions," Kapil says. "Treating the network as a business enabler rather than overhead gives organizations the scalable, secure, and resilient infrastructure the AI economy will continue to demand."
Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. For more information, contact sales@venturebeat.com.
