Not long ago, I rented an SUV from a well-known car rental company. Within hours of an employee scanning my driver's license, a high-resolution scan of my ID was available for sale on the dark web.
An exposé published Tuesday by KrebsOnSecurity reports that my license was one of more than 153 million that were available through Nexus, the name of the new ID theft service. Like other driver's licenses available there—including some belonging to journalist Brian Krebs, his mother, an FBI assistant director, and several security researchers—my license was purported to include multiple image files showing both the front and back of the ID. Besides a basic image scan, the files also captured the images in the infrared and ultraviolet spectrums. Presumably, the additional formats may allow cloned-based counterfeit IDs to pass hologram tests.
Growing by the day
Besides advertising the availability of driver's licenses, Nexus offered to sell a bevy of other forms of ID. They included:
The accounts Anthropic flagged were card-billed, self-serve accounts, which is the population no corporate identity provider governs, and no admin console can sign out. Session-cookie replay bypasses SSO as thoroughly as it bypasses 2FA. What SSO provides here is revocation and visibility, not prevention. The company disclosed the campaign in notification emails to affected users, named six stealer families, signed the accounts out, stripped the saved payment methods, and refunded the charges it found.
The burned usage is the small loss. What those sessions could reach is the exposure, and none of it sat behind an identity controlled by an enterprise.
Anthropic told affected users that a bad actor was using common infostealer malware to lift Claude login sessions off their computers and then replaying them to burn the accounts' usage, according to the notification an affected user posted to Reddit and BleepingComputer reported on August 30.
It named Vidar, LummaC2, StealC, RedLine and Acreed on Windows and Atomic Stealer on a small number of Macs, and it described general-purpose malware that copies browser login cookies along with saved passwords. "Your Claude session was likely one of the many things it collected," the email said.
A session cookie is the proof that a login already happened
The attack chain runs in one direction, from an infected machine through a stolen cookie past a checkpoint that never fires, and into everything the account can reach.
Signing the accounts out worked because a replayed cookie dies with the session it copies.
Two-factor authentication guards the login page. The site then hands the browser a cookie so the user stays signed in, and an attacker who copies that cookie and replays it looks to the server like the person who already passed the check. Help Net Security described the mechanism on August 31 as session theft becoming the new credential theft.
Anthropic spotted the theft in the usage meter. Limits were refilled and drained while the owner was away from Claude, the company wrote.
One Redditor who received the notification traced the infection to a pirated game, per BleepingComputer. That is one machine, and Anthropic has not said what the others ran.
Anthropic's notification gave no count. The company had not responded by publication to VentureBeat's questions on how many accounts were affected, whether any Team or Enterprise seats behind SSO were among them, or whether the replayed sessions reached conversation history or connected apps rather than usage alone.
Removing a saved card and refunding charges point to directly billed, self-serve accounts that authenticate through Anthropic's own login rather than a corporate identity provider. Those include personal subscriptions. Team and self-serve Enterprise organizations can also be card-billed, so the deduction is strong rather than closed.
Bugcrowd CEO Dave Gerry told Axios in early August that his company sent employees nearly a dozen emails saying the OpenClaw agent was not allowed on corporate networks, and employees kept trying to download it anyway. A personal Claude subscription on a managed laptop is the same reflex, and it comes with a card on file. LayerX data in Akamai's enterprise AI risk report found 47% of enterprise AI conversations run through personal identities, with Claude at 61%.
The pirated game is one vector. In July, attackers hosted a spoofed Claude download page on the claude.ai domain itself through a public Artifact, and a sponsored Bing ad sent employees searching for "Claude Desktop app" straight to it. Huntress documented the campaign, named FakeAgent, after SectopRAT compromised employees at 29 organizations in two days. The artifact collected roughly 7,100 downloads before Anthropic removed it. A separate campaign pushed a fake Claude installer through a spoofed download site earlier in the year, per Malwarebytes. The vector is not piracy. It is enterprise employees searching for the official app on their work machines.
Refunds cover the usage. Nothing covers the connectors
A replayed session inherits everything the legitimate one could reach, and Anthropic has not said whether these did. On a Claude account, that means the conversation history, the files uploaded into projects, and any connectors the owner authorized. Anthropic's help center states that connectors let Claude retrieve data and take actions inside connected services and that Claude inherits each person's permissions from the connected service. Read and search operations run without approval. Write actions, including send, reply, forward, share, move, and trash, are approval-gated by default. The exfiltration path is the one that is open. Google Workspace connectors are available to individual Claude accounts, so a personal Pro subscription can hold a live authorization into a Gmail inbox or a Drive folder.
If that inbox is the work inbox, the attacker holding the replayed cookie has a read path into it that the corporate identity provider evaluated once, at the moment the employee clicked allow, and rarely again. On a personal plan, the employee owns that grant. No Claude tenant administrator can sign that account out, and the Workspace or Entra administrator who can pull the underlying grant rarely knows it exists.
Adam Meyers, CrowdStrike's senior vice president of counter adversary operations, put numbers to the market in an August 6 Axios interview. Criminals have been buying and reselling stolen ChatGPT, Claude and Gemini credentials since ChatGPT took off in late 2022, fed by infostealer malware. CrowdStrike's 2026 Threat Hunting Report documents one LLMjacking campaign that pushed nearly 200,000 API requests through a compromised cloud account's AI model access in two minutes.
Meyers drew the line in a July briefing on the report. LLMjacking, in his framing, is stealing the credentials, and cost harvesting is what the buyer does next, manipulating AI resources that belong to the victim "in order to conduct operations and generate massive bills as a byproduct of that," he said. "So think of this as LLM coin mining."
One architect refused to build the same exposure into his product
Tom Kleinpeter, co-founder and chief architect at Common Room, described in written answers to VentureBeat why he held his company's AI agent integrations back through the summer of 2025.
"We rejected local MCP servers early, full stop. That path meant storing a long-lived API key or token on someone's machine. Steal that credential, and you can impersonate the user, pull their data, or do anything else the token allows, indefinitely, until someone notices and manually revokes it. We weren't willing to ship that."
Common Room shipped its first agent integration in October 2025 with Okta'sAuth0 handling authentication, separate read and write scopes, and writes off by default, per Kleinpeter.
An AI coding agent working on Common Room's own system proposed caching access tokens in plain text in Redis to cut down on repeated authentication calls, he wrote. It worked, and it would have parked live credentials in shared infrastructure had a human reviewer not caught it before it shipped.
Asked what was acceptable in 2024 and a liability now, he named one thing. "Long-lived, broadly scoped API keys. Those made sense when one human operated one trusted system and stayed in the loop. Agents now run across laptops and multiple clients, often with no human watching in real time."
Okta gave agents governed identities the same week Claude users lost their cookies
Okta made Agent SSO generally available on August 24, registering AI agents as first-class identities in Universal Directory and issuing short-lived, identity-governed tokens in place of stored credentials, according to the company's announcement. The release names Claude as its example of an agent a security team can now govern natively.
Six days later, Anthropic was signing users out because six stealer families had copied the humans' Claude cookies. The agents got governed identities. The people using Claude on their own cards did not.
That 3% has a reason, Kayne McGladrey, author of the forthcoming "Cyber Risk is a Myth" and a senior IEEE member, told VentureBeat during a July interview. "It's only those well-resourced companies that are above the poverty line that have met all the prerequisites," he said.
The prerequisites he named are the same controls most enterprises still treat as hygiene, not strategic investment.
"If they don't have their defenses in order, like attack surface management or blast radius containment or basic MFA, that would not be a useful capability or a meaningful spend."
Anthropic's position deserves its hearing. The company told users it has no reason to believe the malware is related to Claude, installed through Claude, or tied to anything they did with Claude, and it warned that signing out stops the stolen sessions while leaving the malware in place to steal the next login.
Both hold, and they are the last thing a provider can do, because the infected device belongs to the customer. On a work laptop, the device belongs to the enterprise, and the control that catches Vidar or LummaC2 before it reads a cookie jar is endpoint detection, the control in this story the security team already runs.
The profession's gap is rarely a missing control anymore, in McGladrey's framing. "I think we've got technical solutions for nearly all of the things that could go wrong, what we don't have is a way of prioritizing those," he argued.
The endpoint team owns the machine. The identity team owns an SSO the account never touched, and the AI governance lead wrote a policy the employee routed around the day the card went on file.
Each of those owners is paid to close a different gap. "Engineering is comped on getting product out the door quickly, your internal audit team is comped on checking boxes to meet your compliance goals, and security is comped and sometimes penalized on a lack of incidents," he argued. "People aren't doing the wrong thing either. They're doing what pays their bills on an ongoing basis."
What security leaders need to do next
Add AI accounts to the infostealer response playbook. When an endpoint alert names a stealer family, treat every AI service session on that machine as compromised, revoke what the enterprise tenant lets you revoke, and have the employee sign out of personal accounts until the machine is clean.
Warn users that the notification itself is now a phishing template. Help Net Security flagged copycat phishing impersonating Anthropic using this campaign as pretext. If the notification lands in a user's inbox, the next email that looks like it may not be from Anthropic.
Count the personal subscriptions on managed devices. Browser telemetry, CASB logs, and expense reports surface the sessions and the payments.
Stop personal AI accounts from holding OAuth grants into corporate Google Workspace or Microsoft 365. Both platforms let administrators restrict third-party app authorization. Use that gate so a work inbox can only be attached from a tenant the security team can revoke.
Revoke the OAuth grants Claude already holds, not just the Claude session. Signing out of Claude invalidates the stolen session but does not revoke the Google or Microsoft grant Claude was already authorized to use. Check Google's third-party app authorizations and Microsoft's enterprise application consents for live grants the sign-out left behind.
Move the heavy users onto the organization-managed tenant. On Team and Enterprise plans, an owner decides whether connectors can be enabled at all.
Put session binding on the renewal agenda. Google shipped Device Bound Session Credentials in Chrome 146 on Windows in April and turned it on by default for Google accounts and Workspace Individual accounts in May, binding each session to a private key in the device's TPM so a copied cookie cannot be refreshed anywhere else. It covers Chrome on Windows only so far, so the Mac victims in this campaign sit outside it. Ask Anthropic and OpenAI for parity and Google for a coverage date before the next contract signs.
Anthropic sent its notification to individuals. The laptop the cookie came from belongs to whoever manages it, and Vidar and LummaC2 will be back for the next login on the same machine.
Hackers carried out a supply chain attack that installed malware on networks using an unusual technique: hijacking a chunk of Internet space where cloud management software used by hosting providers, data centers, and other large infrastructure companies is updated.
In a well-coordinated operation, the unknown attackers exploited weaknesses in the routing security setup of hosting provider Hetzner Online and the process for attaining valid TLS certificates. The lapses allowed the attackers to successfully perform a BGP (Border Gateway Protocol) hijacking to obtain control over IP addresses assigned to Softaculous. The company, based in the United Arab Emirates, is the maker of a platform for installing and managing Web software and is the developer of Virtualizor, a management platform for virtualized environments.
Softaculous used the IPs to issue updates and host a client and billing site. With control over the hijacked space, the attacker was now using the addresses to push malware masquerading as updates to unsuspecting users.
Egiziago Cioffi is the IT and Enterprise Architect and CEO of SynSphere Italia, a Microsoft partner based in Milan. He built an agent himself. He wrote the indexing job, configured the Azure OpenAI retrieval pipeline, connected it to SharePoint, and watched it pass every evaluation his team ran.
His Azure OpenAI email assistant auto-resolves about 60% of inbound customer email, Cioffi told VentureBeat in written responses to our interview questions. The evaluation scores were clean, and the unit tests passed. None of them asked the question that mattered.
Cioffi ran a low-privilege account against the same questions a high-privilege account had already put to the assistant. The outputs did not match. The assistant returned SharePoint content the requesting user could not have opened in SharePoint on their own. The logs told a different story than the evaluation scores.
Cioffi's retrieval logs are the evidence for this specific production failure. What follows is independent data showing the failure class is not isolated.
In many production RAG deployments, the agent answers with the indexer's permissions, not the requester's
Azure AI Search has shipped native document-level ACL trimming via Entra-based tokens since preview in May 2025, and SharePoint ACL sync followed in a later preview. The capability exists; however, it does not exist everywhere it needs to.
The SharePoint ACL preview can now ingest site-group metadata via the spg: prefix in the 2026-05-01-preview API. However, only Entra-backed principals are documented as reliably enforced at query time. The preview runs through the REST API and preview SDKs and does not cover all agent deployment paths. Azure OpenAI On Your Data, for example, supports document-level access via Azure AI Search security filters, but Microsoft's own documentation states that if the permitted-groups field is not mapped, document-level access is disabled.
That is a fail-open default in a first-party path. Custom RAG pipelines that bypass Azure AI Search entirely still index under a broadly privileged service account with no query-time entitlement check unless the developer builds one. Cioffi's deployment took the custom-pipeline path.
Across production agents at scale, 91% of successful attacks ended in silent data exfiltration
Straiker's red team ran more than 1,700 successful exploit attempts against production agents and published the results in its inaugural STAR Labs Threat Report in July. The 91% figure from their research measures all successful attacks on productivity agents that ended in data exfiltration without detection. It is a measure of what happened after an exploit succeeded, not a measure of how many deployments fail to enforce retrieval-time entitlements specifically.
Across the productivity agents in scope, 91% of successful attacks ended in silent data exfiltration, with the report noting no malware had been required. There was also no lateral movement through the network. The agent returned all the data it could reach. Straiker's report does not break out which of those successes trace to entitlement failures specifically versus prompt injection, tool abuse, or other attack classes.
Working independently, the U.K.'s AI Security Institute documented 19 unsanctioned agent actions from a July 25 to 28 cyber evaluation. The UKASI published its incident report on August 4of this year. The evaluation deliberately ran with cyber classifiers disabled and internet access enabled. What the UKASI report demonstrates is agents acting outside the scope their deployers intended, in a permissive test environment, with no reliable mechanism to catch the deviation before it causes damage. It is a containment failure, not a retrieval-entitlement failure, and the overlap with the Cioffi incident is the shared absence of a runtime scope check rather than an identical mechanism.
Why evaluations miss this and why the native fix did not reach Cioffi's deployment
The evaluations Cioffi's team ran were designed to test whether the agent answers correctly. They check factual accuracy, relevance, and task completion. They do not ask whose permissions the retrieval pipeline uses when it fetches the source material, because that question is not in the evaluation framework.
Azure AI Search is currently shipping the retrieval-time entitlement check at the platform level. The query-time ACL trimming validates the caller's Entra token, extracts user and group claims, and returns only documents whose synchronized permission metadata grants the caller access. For deployments that use Azure AI Search with the SharePoint indexer and Entra-backed principals, the control exists natively. Cioffi's deployment did not use this path. His custom Azure OpenAI retrieval pipeline bypassed the native trimming layer, which is how the gap survived every evaluation his team ran.
From the attacker's side, this is a broken access control. Adriel Desautels, founder and CEO of Netragard, told VentureBeat in written responses that the failure reduces to a structural collapse of authorization boundaries. "If the NHI credentials usually have broad authorization and can read high privilege data then that is then stored in their index," Desautels wrote. "If an app does not enforce identity-aware retrieval, then a 'normal' user with lower permissions can query the app and access otherwise restricted data. This collapses authorization boundaries down to the lowest privilege level with search capability."
That gap is what Cioffi's low-privilege test exposed. The assistant's context window contained SharePoint content the low-privilege account could not have retrieved through SharePoint directly. The evaluation had passed. The retrieval permission boundary had not been enforced.
Desautels put the evaluation blind spot in operational terms. "Agents tend to run a single, long-lived, non-human identity that holds a wide range of permissions that it might need for any task it is ever asked to complete," he wrote. "Evaluations also don't often cover prompts, outputs, transcripts, memory, and logs where it can be read or hijacked through injected content. That mismatch is what most current evaluations get wrong."
Cioffi's filter narrowed the assistant's retrieval scope. It still resolves roughly 60% of email
Cioffi's fix did not require a new identity platform. He moved the entitlement decision into the retrieval path itself, adding a query-path filter that checks the requesting user's SharePoint permissions before the model sees a chunk. The filter runs at query time, not at index time. Content the user could not open in SharePoint does not enter the model's context window.
The control narrowed what the assistant could reach. The assistant still auto-resolves roughly 60% of inbound email with the filter live, Cioffi told VentureBeat. He did not provide a before-the-filter auto-resolution figure for comparison. The qualitative tradeoff he described is that some content the assistant previously used to answer questions is now excluded because the requesting user's permissions do not reach it. That is the price of enforcing the boundary.
The question of whether retrieval-time entitlement filtering is worth the narrowed retrieval scope does not have a single answer. It depends on the sensitivity of the indexed content, the permission variance across the user population, and whether the deployment can tolerate unanswered queries when the filter blocks a chunk the model needs. What Cioffi's incident demonstrates is that the gap exists in custom Azure OpenAI pipelines, that answer-quality evaluations do not catch it, and that a query-path filter closes it at a trade-off the builder can describe.
Identity governance platforms address a different layer. Both controls are needed
CrowdStrike announced its $740 million acquisition of SGNL on January 8, 2026, and closed the deal on February 20, 2026. Palo Alto Networks announced its $25 billion acquisition of CyberArk in July 2025 and closed the deal on February 11, 2026. Both deals closed the same month, establishing identity security as a platform pillar at two of the largest security vendors in the world.
Identity governance platforms focus on which service accounts exist, what they can reach, and when their tokens expire. They govern the lifecycle of the credentials that power AI agents. That layer matters. What it does not govern is the retrieval permission boundary. That is the moment a correctly scoped service account retrieves content on behalf of a user who holds fewer permissions than the indexing job does.
Every credential in the chain is legitimate. The service account is clean and properly managed. The knowledge base is correctly indexed. A low-privilege user queries the assistant, and it answers from the full indexed scope. Nothing flags the retrieval because no credential was misused.
Cioffi's filter is a control at the retrieval permission boundary layer specifically. Azure AI Search's native ACL trimming addresses the same layer for deployments that use it. Neither replaces identity governance. A production deployment that wants to close both the credential lifecycle gap and the retrieval-time entitlement gap needs controls at both layers.
One question and one test, any security team can run
Ask whose permissions each AI retrieval system uses when it fetches content.
If the deployment uses Azure AI Search with the SharePoint indexer and Entra-backed principals, verify that query-time ACL trimming is enabled and that the user population does not depend on SharePoint site groups. If the deployment uses a custom retrieval pipeline, the entitlement check may not exist at all.
Start by proving the answer from a low-privilege account. Run the same question a high-privilege account has already put to the assistant. Compare the outputs against what the low-privilege account can access through the underlying system directly.
Desautels confirmed that this is where a red team would start. "The first test would likely target the gaps between data and instructions, and the gaps between the user's identity and the assistant's own credentials," he wrote. "We'd attempt to plant an instruction within content that we think the assistant will ingest as data. We'd have that content direct a side-effectful, privileged action that the attacking user is not authorized to perform." A failing result, in Desautels' assessment, is "the successful or even partial execution of our injected commands."
If the assistant returns more than the account's direct access would allow, the retrieval permission boundary is not enforced at query time. That test costs two accounts and thirty minutes. It produces a result an evaluation score cannot replicate.
Cioffi built the agent on a custom Azure OpenAI pipeline that bypassed the native ACL trimming layer. He ran every evaluation his team had. He found the gap in his own logs after all of them passed. The evaluation tested whether the agent answered correctly. It did not test whose permissions the agent was using. Run the two-account comparison before the next deployment goes live. Thirty minutes tells you which side of the line you are on.
AI is changing the pace of cybersecurity. Agentic systems can coordinate work and pursue complex objectives over long horizons. Security teams are beginning to...
AI is changing the pace of cybersecurity. Agentic systems can coordinate work and pursue complex objectives over long horizons. Security teams are beginning to apply agents across security operations, but many implementations remain anchored to existing alerts, predefined workflows, and known attack behaviors. The harder problem is identifying what defenses miss and turning those gaps into…
As online services get better at blocking malicious traffic, the attackers and scammers behind them have been forced to find new ways to reach their targets. The alternative of choice is now what are known as residential proxy networks. These systems funnel millions of home Internet connections into a unified network, and the proxy operators allow attackers to route their malicious traffic through these connections for a fee. The online services see only IP addresses with good reputations and geolocations that don’t stand out.
More often than not, the home users have no idea that their connections are being used to facilitate crime and occasionally even nation-state attacks. Users who do know often don’t care much. In exchange for leasing out part of their unlimited bandwidth to others, many get free movie and TV show streaming. Several less tech-savvy people I know who own such digital media players have told me, after I explain how the media players piggyback off their connections, that the bonanza of content is worth it. They find the tangible benefits outweigh the abstract harm they pose.
Infecting already compromised devices
Research published Monday brings the threat into much clearer view. Security firm Plume cataloged a vast ecosystem of malware that preys squarely on users of SuperBox, just one of many media players offering pirated content. These malicious apps can be surreptitiously installed by remote attackers even when the devices are positioned behind a router. While Monday’s deep-dive analysis focused exclusively on SuperBox, Plume warned that dozens of similar streaming devices pose precisely the same threat.
Identity and permissions are no longer enough to secure enterprise AI agents. They govern what an agent can reach, not how it behaves once it starts working on its own, and an autonomous agent can turn legitimate access of enterprise data into unintended action in seconds. That gap is pushing enterprise AI security from just governing access toward a layered approach that includes governing execution, says Heather Ceylan, chief information security officer at Box.
"Access controls and permissions are the foundation, but the challenge is they were designed for humans," Ceylan says. "Permissions are still the foundation, but you have to think about how the agents get their permissions scoped as well."
Access controls were built for a slower, more forgiving world, where a human employee with lingering permissions to a decade-old folder will rarely go looking for any data inside it, or even remember they have access at all. But an agent will explore all of its permissions, operating at a scale no person can match and surfacing forgotten misconfigurations and stale permissions far faster than a human ever could. Identity and scoped permissions remain the first layer of any credible defense, however, because agents raise the stakes, clean access hygiene is more valuable than ever.
A steady drip of incidents in recent months underlines the potential for major damage, where models have slipped the sandboxes they were supposed to stay contained within, reached systems they weren't scoped to touch, or read content they were not permitted to access. While the specifics vary, the overarching threat remains the same: an agent found a path through the data that it could reach, and took it.
Access controls alone can't secure autonomous AI agents
Complications crop up when agents are handed broad standing permissions to match broad workflows. An agent might legitimately need to call fifty tools across twenty different actions over the life of a task, reading and writing folders that span every department in the company. But granting all of that access at once inflates the blast radius of possible damage to data and content if a single step goes wrong. A stronger permissions model provides access only in the moments that it's needed, Ceylan says.
"You need permissions that change based on what the agent has been asked to do, when it needs to take that action," she explains. "If it's taking one step and only needs two tools, it should be scoped to only those two. When you narrow permissions to the task in front of the agent, the number of ways any given step can misfire shrinks with it."
From governing access to governing execution
Tight security is also no longer tied to whether an agent has access to specific data, but whether it should take a specific action on a specific step, which is the difference between a standing access grant and a bounded one. In other words, an agent may hold legitimate rights to a finance folder but still have no business writing four thousand of those files to a new location, even if it's told to do so.
"An employee with access to payroll data they were never meant to keep could be instructed to pull the payroll records and write them to a public shared folder, publishing the entire company's compensation in a single move," she says. "Every access check passed, but the behavior still has catastrophic consequences."
Prompts alone can’t reliably govern agent behavior because instructions can change, agents can absorb injected instructions, or they can be steered by files they read along the way. Durable controls need to live one level down, with the tool calls themselves, as well as the content those calls act on. Deciding in advance what an agent is permitted to execute, regardless of how its prompt is manipulated, keeps behavior inside fixed bounds that no clever input can talk it out of.
Legacy content platforms weren't built for AI agents
Most of what an agent touches inside a company is unstructured content, including contract, policies, customer records — the accumulated files that make up the overwhelming majority of corporate data. Much of it still sits in systems that were designed for human filing habits: network drives, aging ECM platforms, and many SaaS tools. Those systems were built to ask the question, does this person have permission? and they answer it with folder-level access that hasn't been audited for years.
The legacy platforms don't have metadata for an agent to reason over or classification for an enforcement layer to act on, and their logs aren't detailed enough to show what an agent read. Bolting an AI connector onto that stack doesn't fix any of it, it just hands agents the same blind spots, only at machine speeds, Ceylan says.
"Every agent action eventually resolves to content," Ceylan says. "If the content layer can't tell you what it's holding, who it belongs to, and what should never leave it, there's nothing underneath your controls."
How to decide which AI actions need human approval
Two years ago, everyone assumed security would always mean including humans in the loop. Building and living with agents has changed that assumption, Ceylan says, and Box now sorts actions into three tiers: fully autonomous actions, monitored actions and high-risk actions that require human approval.
Fully autonomous, reserved for actions that are reversible, bounded, logged, and free of untrusted input, and where a mistake doesn't cost too much.
Monitored actions, once a team has built enough confidence in an agent and paired it with alerting and rollback that can catch and undo a problem in flight.
Irreversible, high-risk actions that are always routed through a person — for instance, when an agent that wants to delete a large number of files or wipe the primary folder in a structure.
Each team must calibrate the lines between these layers to match its own risk tolerance.
Box’s approach is to put controls in the platform rather than the workflow, enforcing protections such as data classification, labeling and expiration without requiring a human checkpoint for every action.
"The right configuration should be enforced at the outset, instead of blocking an action at the end," Ceylan says.
Building trust in AI agents through behavior over time
An organization can make an access decision once, but trusting an agent's behavior should only happen over time, observing how the agent runs, collaborates, and builds on the outputs of other agents. Published frameworks tend to lag the technology, or are too abstract to implement; instead Box works from a smaller set of principles: tightly scoped agent identities and actions, clear expectations about what can be rolled back, three approval tiers, and a way for teams to test and iterate quickly.
"The sanctioned path has to be the fast path, because when teams aren't given a safe way to experiment, they tend to route around the controls entirely," Ceylan says. "A security leader's job is to offer a way to move quickly without stepping outside the guardrails."
Sustaining trust in an agent depends on being able to see how it actually behaves. But traditional monitoring tools weren’t built for the job; the behavioral baselines that underpin user and entity behavior analytics were calibrated to human activity, and suspicious agent behavior doesn’t necessarily look like human activity.
And while establishing a useful agent baseline begins with logging, many agents start out as experiments, so their actions never make it into logging infrastructure. Plus, important signals are rarely individual easily flagged actions, but chains of activities across systems, as one agent's output becomes another's input, while the detections for that pattern are still being designed.
Governing agents, then, requires visibility not just into what they can access, but what they actually do, and that visibility has to live where the content does.
Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. For more information, contact sales@venturebeat.com.
Enterprise AI has entered a new era. Organizations are rapidly moving beyond assistants that answer questions to autonomous agents capable of reasoning, invoking tools, accessing enterprise applications, coordinating with other agents, and completing multi-step business workflows with minimal human intervention.
This shift represents a fundamental change in how software operates. Traditional applications execute predefined logic written by developers. AI agents, however, dynamically determine how to achieve an objective. They decide which tools to use, which APIs to call, what information to retrieve, and how to sequence actions based on context. That flexibility unlocks enormous business value, but it also introduces a new class of security risks.
Much of today's AI security discussion focuses on prompt injection, model vulnerabilities, and data leakage. These are important concerns, but they represent only part of the challenge. Once an AI agent has successfully authenticated and begins acting autonomously, traditional security controls provide very little visibility into whether it continues to operate safely.
This is where enterprises need to adopt a new security mindset: runtime trust.
Authentication establishes identity, not trust
Enterprise security has traditionally relied on three foundational questions: Who are you, what can you access, and what actions are you authorized to perform. Identity providers, multi-factor authentication (MFA), role-based access control, and zero trust architectures answer these questions effectively for human users and conventional applications, and NIST's zero trust guidance remains a solid reference point for how those principles are meant to work (NIST SP 800-207).
AI agents introduce a different problem. An AI agent may legitimately authenticate using an enterprise identity, receive valid API credentials, and be granted access to systems like Microsoft 365, ServiceNow, Salesforce, or GitHub. From an identity perspective, everything appears correct. The real challenge begins after authentication: During execution, the agent continuously reasons, interprets objectives, invokes tools, retrieves information, and adapts its behavior based on new context, and security teams must determine whether those actions remain aligned with the user's intent and organizational policy. Authentication verifies who an AI agent is. Runtime trust continuously verifies what it is doing.
Enterprise AI is becoming an autonomous workforce
Modern AI agents increasingly interact with large language models (LLMs), Model Context Protocol (MCP) servers, retrieval-augmented generation (RAG) systems, vector databases, enterprise APIs, SaaS platforms, and internal knowledge repositories, as well as other AI agents. This interconnected ecosystem enables sophisticated automation but dramatically expands the attack surface: A single compromised tool, poisoned knowledge source, overly permissive API, or manipulated prompt can influence downstream decisions across an entire workflow, and unlike traditional software, these risks evolve during execution rather than being fixed at deployment.
That expanding surface is exactly what a handful of runtime threats exploit.
Goal drift happens when an agent begins with a legitimate objective but gradually deviates from the user's original intent while attempting to optimize outcomes. An agent tasked with preparing a customer report, for instance, might autonomously retrieve unrelated confidential information because it incorrectly determines that additional context would improve the response.
Excessive tool invocation is what happens when autonomous agents with access to numerous enterprise tools call unnecessary APIs, modify configurations, access sensitive repositories, or perform administrative actions simply because the model believes those actions are useful, absent any runtime controls to stop it.
Memory poisoning exploits the persistent memory that improves personalization: Attackers can intentionally insert misleading instructions into long-term memory or retrieval systems, causing future decisions to be influenced by malicious or outdated information.
Context manipulation takes advantage of how heavily LLMs depend on context: If attackers influence retrieved documents, system prompts, conversation history, or external data sources, they can indirectly steer autonomous behavior without ever compromising the underlying model. MITRE's ATLAS framework catalogs this kind of adversarial behavior against AI systems in useful detail.
Multi-agent amplification emerges as organizations deploy specialized AI agents that collaborate: If one agent behaves incorrectly, downstream agents may trust and amplify those actions, creating cascading failures across enterprise workflows.
Introducing runtime trust
Runtime trust extends security beyond authentication by continuously validating AI behavior throughout execution. Rather than assuming authenticated agents remain trustworthy indefinitely, it continuously evaluates whether autonomous decisions remain aligned with organizational policy. A runtime trust architecture rests on several complementary capabilities.
Intent validation evaluates, before executing sensitive actions, whether proposed behavior still matches the user's original objective: Is this action necessary? Is it expected? Does it exceed the requested scope? Would a reasonable human perform the same action?
Behavioral monitoring observes tool usage, API activity, reasoning patterns, execution frequency, delegated actions, and abnormal workflows, so unexpected behavior becomes immediately visible rather than remaining hidden inside model reasoning.
Policy enforcement means enterprise policies govern what AI agents can do, not merely what they can access — blocking financial transactions above approval thresholds, preventing privilege modifications, restricting administrative operations, limiting sensitive data retrieval, and requiring approval for high-risk actions. These controls function much like application firewalls for autonomous decision-making.
Least-privilege execution means AI agents receive only the capabilities necessary for the current task. Instead of granting permanent access to dozens of enterprise tools, organizations should dynamically issue short-lived permissions based on runtime context, an approach that OWASP's guidance for agentic applications increasingly emphasizes (OWASP GenAI Security Project).
Human oversight recognizes that not every decision should be autonomous — high-impact operations, including financial approvals, identity changes, regulatory actions, or customer-impacting decisions, should require explicit human confirmation before execution.
Protecting the enterprise AI ecosystem
Runtime trust also extends beyond individual agents. As MCP adoption accelerates, enterprises should verify trusted servers, authenticated tools, approved capabilities, monitored interactions, and policy enforcement. RAG knowledge repositories require document integrity, source validation, access control, retrieval auditing, and poisoning detection. Persistent AI memory should implement lifecycle management, expiration policies, integrity verification, access logging, and sensitive data protection.
Building operational visibility
One of the biggest challenges in enterprise AI is observability. Security teams need visibility into why an agent selected particular tools, which data influenced its decisions, how it reached its conclusions, what actions it executed, whether policies were triggered, and which safeguards prevented unsafe behavior. Runtime logging, audit trails, and behavioral analytics are becoming essential components of enterprise AI operations, not optional add-ons.
A practical roadmap
Organizations do not need to rebuild existing security programs. Instead, they should extend them by incorporating runtime trust into existing governance processes. Practical first steps include inventorying AI agents and their capabilities, applying least-privilege access to tools and APIs, classifying high-risk autonomous actions, implementing runtime policy enforcement, monitoring behavioral anomalies continuously, protecting memory and RAG data sources, requiring human approval for critical operations, and integrating AI runtime telemetry into existing SOC workflows.
Looking ahead
Enterprise AI will continue evolving toward increasingly autonomous systems capable of collaborating, planning, and executing complex business processes. Security strategies must evolve alongside them. The question is no longer whether an AI agent successfully authenticated. The more important question is whether it continues to behave safely throughout its entire lifecycle. Organizations that adopt continuous runtime governance today will be significantly better positioned to deploy autonomous AI responsibly, reduce operational risk, and build the confidence necessary for large-scale enterprise AI adoption.
The future of AI security will not be defined solely by stronger models or better authentication. It will be defined by our ability to establish, measure, and continuously verify trust while intelligent systems are making decisions in real time.
There is a clear repeating trend in agent deployments: The gateway is the first control teams reach for, but it is the one they are least ready to run. This is because gateways sit on top of identity and attribution layers that are mostly not there.
The first layer of risk is not hypothetical. In June,CISA added a LiteLLM flaw to its Known Exploited Vulnerabilities catalog after attackers were caught abusing it in the wild. The bug ran commands on the host through the gateway itself, and chained with a second flaw it required no credentials. It was one of seven common vulnerabilities and exposures (CVEs) disclosed in that single AI gateway in a month. This is the layer many enterprises reach for first to secure their AI agents.
When considering secure agent architecture, gateway controls should not be the first control. They should be the fifth.
Most models on the maturity of agent security describe the controls a company will need in the future. They tend to miss, from my experience, the more difficult problem of describing the brownfield scenario: In what order should these controls be layered in conjunction with an identity and access management system that is already in place?
If the control plane is unaware of which agent is acting, who delegated the work, what task the agent is to perform, and what credentials are being used, then the context is incomplete. A gateway may block clear policy violations, but will struggle to distinguish a justified action from one that is technically permissible but operationally inappropriate.
The pattern of failure is clear when sequencing these controls for agent production deployments: Enforcement is taken early, while the identity and attribution context it depends on has yet to be developed. Agent security functions as a dependency chain, with each control depending on context generated upstream.
The wrong starting point
Think about routing agent traffic via a new runtime gateway. A finance-reconciliation agent tries to alter a record in production. The gateway authenticates the user token and checks the API call. What it can't observe is that the request is agent-initiated, that the agent is executing a more limited function, or that the request is part of a tool chain invoked by an untrusted artifact.
The credential is valid. The API call is permissible. The action contradicts the purpose of the delegation. The gateway is there, but its set of supports seems absent, so a costly control is applied to a very small part of the whole picture.
Limiting an agent's privileges to those of the human principal is useful so the agent does not exceed the person it serves. However, having a privilege ceiling does not create separate attribution. Twenty agents might operate under a single person's permissions and still need unique identities, audit logs, behavior profiles, and revocation paths.
Dependency-gated deployment
I call this process dependency-gated deployment. Upstream exit tests must be satisfied before any downstream control is considered operationally complete. Concurrent development of downstream controls is permissible.
Here are the six gates, and the proof that they work:
Gate
Control
Operational proof it works
1
Agent inventory and accountable ownership
Every production agent has a named owner, purpose, approved tools, and lifecycle state
2
Distinct agent identity plus delegation context
The system can identify the agent, its owner, and the principal it is acting for
3
Task-scoped, short-lived credentials
A compromised agent cannot reach resources unrelated to its assigned task
4
Attributable telemetry
A completed task can be reconstructed from initiation to downstream effect
5
Runtime action enforcement
Policy decisions incorporate agent, principal, task, and action context, not just token validity
6
Behavioral baselines and cross-system kill path
The agent's effective authority can be stopped everywhere it reaches
The six dependency gates for the agent security controls. Each control is contextualized by the gates above it. From the author's analysis of production agent deployments.
Start with the agents you can actually name
To begin, recognize the production agents in open-source frameworks, cloud offerings, SaaS services, and developer tools. For each, record the owner, responsibility, lifecycle stage, allowed tools, data domains, and sources of credentials.
Bypass this step, and the organization will lose the first hour of incident response while they figure out what should have been obvious. The inventory identifies the asset that every control thereafter governs.
An agent needs its own identity, but it cannot lose the human behind it
An agent should not be buried in a developer token, a shared service account, or a human session. Simply knowing the caller is an agent is not sufficient. The control plane requires additional delegation context: Who delegated the work, what specific task the agent was instructed to execute, and which resources the agent needs the authority to access. Identity specifies which actor placed the call. Delegation is the answer to whose authority it acts, and for what reason.
Once that connection is cut off, the downstream logs attribute the reconciliation agent to the employee whose token it borrowed, and every action it takes is attributed to someone who did not start it.
Shrink authority before you inspect behavior
Once an agent can be identified, capabilities should be limited. Access restrictions should be time-bound to the task and limited to the tools and resources required to perform the task. This can be implemented using identity access management (IAM) features such as workload identity, token exchange, conditional access, and time-bound entitlements which the organization already possesses.
With regard to the2026 Teleport study involving 205 security leaders, the access scope surpasses the predictive capacity of industry, maturity, or self-assurance concerning predicting AI-related incidents. For example, organizations with over-privileged AI reported a 76% incident rate, whereas AI incidents occurred in 17% of organizations under the least privilege. This indicates that access scope in the dependency chain is more important than context-aware runtime enforcement.
The primary principle is monotonic delegation. Every transfer of responsibility must preserve or diminish authority; under no circumstances should it increase authority. For the reconciliation agent, this means an agent who can view one ledger as opposed to one who inherits the employee's access to all systems the employee can access.
Fix attribution before automating enforcement
Most audit stacks can capture what resource was accessed and which credential allowed access. In the agent deployments I have reviewed, this is the most commonly missed gate. Prior to utilizing an adaptive runtime policy, link any relevant tool invocation to the agent identity, initiating principal, task id, parent action, and outcome. After doing so, examine the telemetry: For one completed task, see if you can track down the initiator, the agent who executed it, the authority under which the action was taken, the tools utilized, and the outcome. In regulated environments, oversight that is not attributed cannot be justified.
Now the gateway earns its keep
The gateway can use registered identities, explicit delegation, scoped credentials, and attributable telemetry to question if this agent is authorized to perform this action, for this principal, within this task, involving this resource. Although the user's credentials may provision write access to the finance-reconciliation agent, the gateway has situational context and so determines that it is out of scope. This is control's point of greatest value. The most stringent controls should be applied at irreversible boundaries — payments, access policy changes, deletions, modifications of the production environment, and data exports.
Detection and the kill path come last
Behavioral baselines are developed last because distinguishable and attributable agent activity must be established to set a standard. Then, security teams are able to identify anomalous patterns of tool usage, unexpected cross-domain access, and deviations from their assigned tasks. Containment is more than just the disabling of a single directory object: A proper kill path entails disabling the agent's identity, invalidation of active and derived credentials, blockage of tool activation, termination of active tasks, and isolation of the workload that contains the agent.
Start without replacing your IAM
Designing a whole new identity program is unnecessary. If the existing identity provider doesn't treat agents as native object types, begin with an authoritative registry linked to the existing workload identities. Following this, extend agent and task identifiers as trusted execution contexts, implement short-lived credentials to mitigate inherited privileges, and include those identifiers in tool-call logs for subsequent gateway ingestion. The dependency model remains unchanged as vendor support matures.
Control gaps are measurable. InOkta's 2026 survey, only 34% of executives said their organization always applies the same level of security rigor to its agentic workforce as to its human workforce. The last control from the chain cannot be applied first to close that gap.
What to do in the next 30 days
Begin with 10 production agents. For each one, identify the owner, purpose, approved tools, and credentials. By now, you should have the beginnings of an agent registry and perhaps your first insights on governance.
Test attribution. Find out if IAM and logging can tell each agent apart from the human or service that delegated the task. If this kind of differentiation is not possible, a gateway would be operating without any visibility.
Reconstruct one completed agent task within an action chain, from start to finish, including downstream effects. Wherever the chain breaks is where your deployment falls short.
Adding downstream enforcement ahead of required context breaks agent security. Maturity models describe the destination. A build order gets you there without breaking production along the way.
Nik Kale is a principal engineer specializing in enterprise AI platforms and security.
Autonomous systems that can reason, make their own decisions, and execute actions across an environment introduce a category of risk that application-level controls were never built to contain. Treating that risk as a single problem produces incomplete architectures, says Oscar Wahlberg, senior director of product management at Nutanix.
"The guardrails to catch a malicious prompt won't stop an agent from hallucinating and doing something it never should have done, like accidentally deleting databases or leaking sensitive data with a credential it was granted but then uses for something entirely different," Wahlberg says. "That's the central problem as enterprises move autonomous agents out of experimentation and into production."
Once an agentic system is granted execution privileges across the data center, the security posture has to scale into a defense-in-depth architecture spanning infrastructure, storage, compute, networking, and a governing control plane. Each layer addresses a distinct category of risk, rather than duplicating the same controls across the stack. No single security control or vendor can provide that protection on its own. Defense-in-depth depends on those layers working together.
By dividing the responsibilities across the stack and adhering to zero trust segmentation, organizations can create a secure framework that improves their overall posture. Understanding which risks belong in each layer is what turns the principle of defense-in-depth into a practical security framework, with three layers that each have a distinct responsibility.
Infrastructure layer: Establishing trust where AI agents run
The infrastructure layer’s foundational responsibility is establishing a root of trust that answers a simple question: who is operating in the environment? That trusted identity becomes the prerequisite for every security control above it. Before an organization can trust what an agent does, it first has to trust the integrity of the environment where the agent runs. When an agent requests permission to execute an operation, the system must be able to verify that the request came from the legitimate agent — not something impersonating it.
Delivering that kind of assurance depends on technologies that root trust in the hardware itself, including platform attestation, confidential computing, and secure boot, alongside controls that prevent unauthorized access both within a server and beyond it. For regulated industries such as financial services, this layer provides the ability to isolate AI production workloads so that neither the agent nor the environment can operate outside its assigned scope. That mitigates risks including model and runtime tampering, supply chain compromise, and unauthorized access to sensitive AI workloads.
Network layer: Governing how AI agents communicate
Once agents begin communicating with other agents, APIs, applications, and enterprise systems, they generate a level of concurrency and dynamic communication that traditional static network configurations were never designed to handle. An agent configured to call APIs, query data sources, and spin up additional agents without constraint creates a sprawling web of east-west traffic that becomes very difficult to reason about, and that complexity can easily mask lateral movement or data exfiltration when the right network security layers are not in place.
"We should treat AI agents as a new class of network identity, and make sure that an agent can only talk to other agents or data sources where it's explicitly allowed to do so," Wahlberg says. "That means moving away from rigid static rules toward dynamic policy enforcement."
Nutanix's solution is Agent Gateway, part of the Nutanix Agentic AI solution. It's a unified, governed layer that is designed to provide cost control and governance capabilities to help manage autonomous agent users. Coupled with agents grounded in zero trust segmentation and using capabilities like Nutanix Flow for micro segmentation and integrating with networking vendors, including its integration into the Cisco Secure AI Factory, Agent Gateway helps enterprises govern interactions across agents, models, data sources, and enterprise applications.
The network layer governs lateral movement, data exfiltration, and gates the agent's network interactions. A zero trust framework with access blocked by default and scalable interaction monitoring is important for agents since they can exhibit unreliable behavior. The Nutanix software integration with Cisco UCS servers and Cisco AI PODs delivers the turnkey physical infrastructure (compute, storage, and networking) that the AI factory runs on.
Control plane layer: Governing what AI agents are permitted to do
The control plane is the brains of the operation, providing a central point for managing agent permissions, tool access, resource consumption, and runtime visibility. What matters most is having a single place where policies can be enforced consistently rather than reinvented for every agent, Wahlberg says.
"Agent Gateway acts as a universal endpoint for different models and tools, so an IT team can configure their agents to talk to this single control point," he explains.
The centralized AI gateway enables the admin to observe, audit, and control access to models as well as MCP tools protecting data and gating privileged access. This layer is designed to help mitigate risks such as privilege misuse, runaway agents, unauthorized tool usage, data leakage, and the excessive model consumption that can lead to increased token consumption when agents get stuck in runtime loops. And it depends on treating governance as a runtime control system rather than a compliance afterthought.
Why one-size-fits-all security fails agentic AI environments
The biggest architectural mistake enterprises make is assuming a single security model can be stretched across every layer of an AI stack. When an organization tries to solve for hardware-level trust with application-level software, or leans on static legacy network rules to manage dynamic agents, it builds an architecture that either blocks the agentic system from doing its job or leaves critical doors wide open. One-size-fits-all thinking tends to produce significant performance penalties and operational friction.
"By failing to assign specific responsibilities to the appropriate layers, enterprises end up with blind spots in governance," Wahlberg says. "They might secure the model output but miss that there's data leakage between agents, or they might secure the network but lack the control plane visibility to understand that they're wildly burning tokens because the agents are stuck in some kind of runtime loop."
Focusing exclusively on the model leaves the largest gaps of all, because a guardrail that catches a malicious prompt does nothing to stop a hallucinating agent from misusing a legitimate credential. Embedding security across the full stack helps ensure that even when a model level threat slips past the initial filters, the agent remains constrained by hardware rooted trust, network isolation, and access controls at the agent layer.
How Intel, Cisco, and Nutanix build defense-in-depth together
The three-way partnership from the three companies demonstrates how the layered architecture comes together in practice as a well-governed, enterprise-grade AI Cloud. Intel supplies the computer to run agentic workloads and secures the execution environment through hardware-rooted trust and confidential computing, while also driving costs down through their accelerators. Intel Xeon 6 processors with built-in AMX accelerate AI inference efficiently without relying exclusively on expensive GPUs.
Cisco wraps the environment in a secure fabric that governs communication between agents and enterprise tools, while Nutanix provides the software platform, minimizing architectural silos, and the central control plane that enforces permissions, delivers visibility and cost governance, and ties the architecture together into a defense-in-depth solution that lets enterprises scale agentic AI.
Of the three layers, enterprises currently underestimate the control plane the most, Wahlberg says. A true control plane extends far beyond initial deployment to simplify Day 2 operations, he explains, giving IT teams the continuous observability, and strict token governance required to keep autonomous agents secure and cost-effective in production.
"Apart from model and tool selection, governing the agent deployments and their access to models and business tools in a tightly integrated full stack platform will be important for the success of AI projects," he says, pointing to a near future in which organizations move from a handful of AI use cases to thousands of agents working autonomously to drive the business.
Technology leaders should prioritize building a centralized governance layer today that can manage agent identities, tool permissions, and token budgets in real time, because that control point is what builds the operational muscle to scale safely.
"You can't build an AI system without getting into a lot of complex decisions," he explains. "And you need a control plane that talks across multiple vendors and infrastructures to help you solve for those defense-in-depth strategies."
Learn more about the Nutanix Agentic AI solution here.
Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. For more information, contact sales@venturebeat.com.
Authorities in Australia said Wednesday that they arrested two men accused of participating in cybercrimes for TeamPCP, a prolific group of hackers that, over nine months, has carried out a relentless series of supply-chain attacks that infected more than 1,000 organizations worldwide.
In a statement, the Australian Federal Police said the two men were arrested and charged with 14 offenses. The statement said the men were members of TeamPCP, which by the authorities’ count, compromised more than 1,000 organizations worldwide. The statement didn’t identify the men, except to say they lived in the Western Australian towns of Cottesloe and Mandurah. KrebsOnSecurity, citing a lengthy investigation, provided what it reports to be both defendants' names, along with an extensive background of their lives and the mistakes that led to their downfall.
The hacks that keep on hacking
TeamPCP has vexed law enforcement officials and security personnel around the world since it emerged in December. The group is best known for a sustained series of supply-chain attacks that laced open source software with malware that self-propagated from one package to another. The viral infections worked by targeting organizations’ CI/CD pipelines, which are used to rapidly develop, update, and deploy software.
Growing datasets and public benchmarks are making it harder to tell whether a model is being tested on something it hasn’t seen before.
On Thursday, Google DeepMind showed off what the company calls the first double-blind evaluation of a proprietary frontier-class AI model. The setup keeps Gemini’s model weights hidden from the evaluators while the test questions are hidden from Google.
The pilot tested Gemini 2.5 Flash Lite against private benchmarks from MLCommons and the Singapore AI Safety Institute, but rather than using the results to tout a new Gemini score, Google is focusing on how the tests were run, with neither side having access to the other’s data.
Benchmark leakage inflates scores
Google’s technical report cites earlier research that found signs of benchmark leakage in about half of the 31 models tested, and another study published this year that found contamination can inflate scores, particularly for larger models. This dynamic has already raised questions about where certain post-training coding gains actually come from.
Keeping benchmarks private would seem like the obvious answer, except closed models make that difficult. With a closed model, evaluators usually have to run their questions through the provider’s API, which means the company can see a test that was supposed to stay private, whereas running it independently would require the provider to hand over its model weights. Google’s setup is meant to give both sides another option.
Keeping benchmarks private would seem like the obvious answer, except closed models make that difficult.
How the enclave works
The pilot is using Google Cloud Confidential Space together with an NVIDIA H100 Confidential GPU and Intel TDX host memory encryption.
Google DeepMind provides Gemini with its weights and inference code, while the evaluator offers its benchmark prompts and evaluation code; these are then transmitted over encrypted connections into the enclave, where the evaluation takes place without either party gaining access to the other’s protected assets.
The model weights are stored in hardware-encrypted GPU memory, and the evaluation prompts are kept in encrypted host memory as well. After the test is complete, the evaluator is given the permitted results, and the temporary environment can then be destroyed.
Before either side sends over its private data, remote attestation verifies that the enclave is running the software they agreed on.
Code controls beyond encryption
The enclave protects what’s stored in memory, but the code running inside it can still create problems. If an application has unrestricted network access, for example, it could send sensitive information somewhere else.
OpenMined’s PySyft handles that part of the process by allowing Google and the evaluator to approve the code in advance and to block sensitive parts of the evaluation from making external connections.
That process takes some work, although compute doesn’t appear to be the problem. The paper puts the overhead at less than 5% and instead points to legal agreements and code reviews between organizations as some of the bigger hurdles to running these evaluations.
The researchers eventually want to make attestation much less hands-on, comparing the goal to the HTTPS lock icon in a browser, where the security checks happen without the user having to deal with the underlying hashes and keys.
Google still has a hand in verification because, although the Confidential Space guest OS is open source and its build process has been externally validated, individual builds rely on private signing keys and can’t be independently reproduced. At the same time, Google’s own services are used to sign and verify the attestation report.
The paper puts the overhead at less than 5% and points instead to the legal agreements and code reviews between organizations as some of the bigger hurdles to running these evaluations.
Scaling past one GPU
So while the system reduces how much the two sides need to trust each other, it doesn’t remove trust entirely from the equation. Some of it simply moves elsewhere, including to the hardware itself. There is still some trust involved in the hardware itself, including the assumption that the cloud provider and the hardware maker aren’t working together to circumvent the protections.
Then there’s the benchmark itself. MLCommons points out that it still needs to be carefully managed, no matter how well the questions are protected.
The pilot ran Gemini 2.5 Flash Lite on a single H100 80GB Confidential GPU. Still, the researchers are already looking at clusters of H100 and B200 GPUs connected via encrypted links to eventually evaluate models that are too large to fit on a single GPU.
If this approach catches on, developers could have another way to look beyond the benchmark score, with some proof that the company behind the model didn’t see the test before the results came out.
If this approach catches on, a benchmark score could offer some proof that the company behind the model didn’t get to see the test first.
OpenAI, together with more than 100 companies including Microsoft, Google, Anthropic, Deutsche Telekom, and SAP, has published an open letter on AI-powered cyber defense. The coalition warns of increasingly sophisticated AI attacks on critical infrastructure such as hospitals and water treatment plants and calls for swift action while defenders still have the upper hand.
After an affair with a fellow police officer ended, a Georgia cop used Flock to track her movements—and those of a man whose vehicle often showed up near hers, internal investigation records show.
Some of the world's largest tech companies and AI startups have come together to decry the current state of cybersecurity and to advertise a new solution that they say can ward off a new generation of cyber threats.
Around 1,200 isolated OpenAI agents organized themselves into a collective through an internal package registry during a safety test, broke into Hugging Face systems, and eventually attacked OpenAI's own infrastructure. Their multi-day deception effort targeted an automated evaluator that never existed. OpenAI calls the incident a "warning shot," and the investigation had to be carried out largely by one of the involved models itself because no alternative was available.
Login
