Growing datasets and public benchmarks are making it harder to tell whether a model is being tested on something it hasn’t seen before.
On Thursday, Google DeepMind showed off what the company calls the first double-blind evaluation of a proprietary frontier-class AI model. The setup keeps Gemini’s model weights hidden from the evaluators while the test questions are hidden from Google.
The pilot tested Gemini 2.5 Flash Lite against private benchmarks from MLCommons and the Singapore AI Safety Institute, but rather than using the results to tout a new Gemini score, Google is focusing on how the tests were run, with neither side having access to the other’s data.
Benchmark leakage inflates scores
Google’s technical report cites earlier research that found signs of benchmark leakage in about half of the 31 models tested, and another study published this year that found contamination can inflate scores, particularly for larger models. This dynamic has already raised questions about where certain post-training coding gains actually come from.
Keeping benchmarks private would seem like the obvious answer, except closed models make that difficult. With a closed model, evaluators usually have to run their questions through the provider’s API, which means the company can see a test that was supposed to stay private, whereas running it independently would require the provider to hand over its model weights. Google’s setup is meant to give both sides another option.
Keeping benchmarks private would seem like the obvious answer, except closed models make that difficult.
How the enclave works
The pilot is using Google Cloud Confidential Space together with an NVIDIA H100 Confidential GPU and Intel TDX host memory encryption.
Google DeepMind provides Gemini with its weights and inference code, while the evaluator offers its benchmark prompts and evaluation code; these are then transmitted over encrypted connections into the enclave, where the evaluation takes place without either party gaining access to the other’s protected assets.
The model weights are stored in hardware-encrypted GPU memory, and the evaluation prompts are kept in encrypted host memory as well. After the test is complete, the evaluator is given the permitted results, and the temporary environment can then be destroyed.
Before either side sends over its private data, remote attestation verifies that the enclave is running the software they agreed on.
Code controls beyond encryption
The enclave protects what’s stored in memory, but the code running inside it can still create problems. If an application has unrestricted network access, for example, it could send sensitive information somewhere else.
OpenMined’s PySyft handles that part of the process by allowing Google and the evaluator to approve the code in advance and to block sensitive parts of the evaluation from making external connections.
That process takes some work, although compute doesn’t appear to be the problem. The paper puts the overhead at less than 5% and instead points to legal agreements and code reviews between organizations as some of the bigger hurdles to running these evaluations.
The researchers eventually want to make attestation much less hands-on, comparing the goal to the HTTPS lock icon in a browser, where the security checks happen without the user having to deal with the underlying hashes and keys.
Google still has a hand in verification because, although the Confidential Space guest OS is open source and its build process has been externally validated, individual builds rely on private signing keys and can’t be independently reproduced. At the same time, Google’s own services are used to sign and verify the attestation report.
The paper puts the overhead at less than 5% and points instead to the legal agreements and code reviews between organizations as some of the bigger hurdles to running these evaluations.
Scaling past one GPU
So while the system reduces how much the two sides need to trust each other, it doesn’t remove trust entirely from the equation. Some of it simply moves elsewhere, including to the hardware itself. There is still some trust involved in the hardware itself, including the assumption that the cloud provider and the hardware maker aren’t working together to circumvent the protections.
Then there’s the benchmark itself. MLCommons points out that it still needs to be carefully managed, no matter how well the questions are protected.
The pilot ran Gemini 2.5 Flash Lite on a single H100 80GB Confidential GPU. Still, the researchers are already looking at clusters of H100 and B200 GPUs connected via encrypted links to eventually evaluate models that are too large to fit on a single GPU.
If this approach catches on, developers could have another way to look beyond the benchmark score, with some proof that the company behind the model didn’t see the test before the results came out.
If this approach catches on, a benchmark score could offer some proof that the company behind the model didn’t get to see the test first.
OpenAI, together with more than 100 companies including Microsoft, Google, Anthropic, Deutsche Telekom, and SAP, has published an open letter on AI-powered cyber defense. The coalition warns of increasingly sophisticated AI attacks on critical infrastructure such as hospitals and water treatment plants and calls for swift action while defenders still have the upper hand.
After an affair with a fellow police officer ended, a Georgia cop used Flock to track her movements—and those of a man whose vehicle often showed up near hers, internal investigation records show.
Some of the world's largest tech companies and AI startups have come together to decry the current state of cybersecurity and to advertise a new solution that they say can ward off a new generation of cyber threats.
Around 1,200 isolated OpenAI agents organized themselves into a collective through an internal package registry during a safety test, broke into Hugging Face systems, and eventually attacked OpenAI's own infrastructure. Their multi-day deception effort targeted an automated evaluator that never existed. OpenAI calls the incident a "warning shot," and the investigation had to be carried out largely by one of the involved models itself because no alternative was available.
Documentation files on more than 100 websites are referencing potentially dangerous executable content that gets installed automatically when visited by many AI agents. A few dozen companies, some of them Fortune 500s, are among those that executed proof-of-concept code. At least one misconfigured site is directing visitors, human or AI, to live malware.
The potentially dangerous content is in llms.txt and llms-full.txt files, an emerging convention websites employ to provide machine-readable summaries of the site’s content and its high-level structure. These files are the AI equivalent of the robots.txt standard that instructs search engines how to index the site's content. Google Lighthouse, a tool for helping web developers, has more here. Correctly configured llms.txt and llms-full.txt files for Cloudflare are here and here.
How the researchers found it
Researchers at a stealth startup in Israel scanned 6,214 live domains belonging to defense contractors, Fortune 500, and Big Tech companies. Of the 8,265 llms.txt and llms-full.txt files they found (many sites hosted both an llms.txt and an llms-full.txt file), 120 of them, each on a different site, pointed to one or more code packages or domain names that weren’t registered. To test what happens when an AI agent processes such files, the researchers registered a handful of the unclaimed names and hosted packages that caused any machine executing them to reach out to their server. Within an hour, the researchers received a phone-home response from a Fortune 500 company. Over time, they got a few dozen more, some from more Fortune 500 companies and others from startups. Their beacon also recorded the chain of parent processes that spawned each install, ultimately revealing that coding agents, including Claude, OpenAI's Codex, and Nous Research's Hermes, were involved. Anthropic, OpenAI, and Nous Research did not respond to requests for comment by the time of publication.
A recap of all the incidents involving LLMs made by Anthropic, Meta, and OpenAI, which went rogue and attacked real companies and individuals on the internet.
The OpenAI agents involved in last month’s incursion into Hugging Face were trained so heavily on winning a competition that they pursued a relentless campaign to cheat, a new report documented. In the process, and without authorization, they created an improvised message board to hatch a plan that ultimately landed them squarely inside the latter company’s network.
Over the course of May and June, OpenAI gave the agents what the company described as “impossible tasks” to complete on the benchmarking framework ExploitGym. The internal test was designed to test how the agents would respond. To get a full understanding of the agent capabilities, company engineers disabled safety guardrails that normally are in place to prevent the sort of hacks that eventually hit Hugging Face and one other undisclosed organization. The stymied agents’ training made them so focused on winning that they performed tasks they were never explicitly instructed to follow.
Cheaters gonna cheat
The first step was creating a message board that allowed the agents to pass notes to each other. OpenAI hadn’t provided any such platform, so the agents repurposed a platform called Artifactory, which OpenAI was using in internal testing of several unreleased hacking agents. OpenAI was using Artifactory as one of the measures to prevent the agents from egressing its isolated sandboxes and accessing the Internet, while at the same time simulating a real-world hacking environment.
Visa's open-source security harness now finds the vulnerability, writes the fix, and turns an adversarial panel on its own patch before any human reviews it. The whole loop ships on by default. A plain scan of the Visa Vulnerability Agentic Harness runs all 11 stages and edits source files in the target repo unless the operator caps it at detection.
The announcement Thursday pairs the release with an expansion of the Visa Consulting & Analytics advisory practice. Visa is shipping that default 18 days after Tenet Security demonstrated GhostJacking on the DEF CON 34 main stage, an attack chain in which an agent read an attacker's payload out of a log file and rewrote DNS with a valid credential. Two days earlier, Steve Wilson, Chief AI and Product Officer at Exabeam and project co-lead for the OWASP Top 10 for LLM Applications, made the case in VentureBeat for the opposite default. "The first thing I'd do is put an authorization gate outside the model," Wilson said in written responses. "The agent can propose the exact DNS change, but it cannot grant itself the authority to make it."
The bottleneck moved, so Visa moved the pipeline
Rajat Taneja, Visa's president of technology, rejects the premise that the default is a risk decision and calls it the product. "The bottleneck has moved," Taneja told VentureBeat in an exclusive interview. "AI is finding vulnerabilities faster than humans can in the history of our technology industry. The new bottleneck is fixing and proving we have fixed things."
VVAH grew out of Visa's participation in Anthropic's Project Glasswing, where the company aimed Claude Mythos at the network behind billions of daily transactions and watched the model chain minor weaknesses into working exploits, a hunt VentureBeat covered in July. "VVAH initially was completely only using Mythos, and that's when all of us, as part of Project Glasswing, realized the power of this new class of models that does semantic reasoning," Taneja said.
The harness went to GitHub in June and has climbed from 595 stars and 97 forks on July 20 to more than 2,300 stars and 300 forks as of August 25, with a clone-to-visitor ratio Taneja put near 9%. "We have got some very high-profile companies that have started using this harness," he said.
Why give it away? Taneja's answer starts with Visa's technology DNA and a harness built "to protect Visa and our ecosystem." The reason he leaned on hardest was obligation, "to do good by doing right" for "companies who may not have the same level of investments or knowledge in cybersecurity."
Contribution runs one way. The repo states it is not currently accepting external code contributions, so the harness that edits adopters' source takes no code into its own.
Thursday's release extends the pipeline past the report. "We're going from discover, verify, and report, and then fix it, to discover it, verify it, remediate it, validate it, and iterate it," Taneja said. "If a fix doesn't negate the exploit, then there should be a structured, automated feedback that preserves the learnings from the first run and then enhances it." Underneath that loop, the release refactors scanning around an abstract syntax tree call graph that maps subroutine calls and the traversal paths an attacker could reach. Taneja argued the change cuts token counts while delivering "better reasoning, context, and better exploitability analysis." On top sits MTTA observability across the stages, what he called a window pane, plus real-time progress views. "A pretty good step function," he said of the release.
One metric, three definitions
Mean Time to Adapt, the metric Visa invented alongside the harness, gets a shorter definition in this release. The short form is the time between discovery and resolution of attack paths, with some resolutions, Visa claims, shrinking from weeks to hours. Visa published a wider construction in June, and the Project Glasswing white paper tracks MTTA along three dimensions that include inventory freshness, exploitable paths per release, and validation cycle time. The repo carries a third, elapsed time from AI-discovered exploitability to a validated fix in production. Board slides will quote the shortest interval. Ask for all three, because a resolution count that skips validation is what MTTA was invented to replace.
Taneja ranks MTTA as "the most strategically important metric" because it shifts the focus from scanning to how fast an enterprise adapts. His shorthand is blunter. "It's not the finding. It's the fixing that matters," he said.
The default and the gate
Wilson's argument went past naming the gate. "We have to remember that security rules written inside prompts may shape the model's behavior, but they are still suggestions to the model, not enforceable security controls," he wrote. He also priced the control honestly. "The tradeoff is that the agent loses the ability to improvise arbitrary, high-impact infrastructure changes on its own, while retaining autonomous investigation and routine, bounded remediation," Wilson said.
The harness ships no approval step between patch and edited file. Where the human sits was the first question VentureBeat put to Visa in writing.
The company's own June white paper sets the bar. "AI agents are identities" sits among its 12 non-negotiable practices, requiring scoped permissions, least privilege, audit trails, and IAM governance for every agent that modifies a system. VVAH's shipped default is that agent.
"A lot of the traditional systems that are used today are basically signal providers," Taneja told VentureBeat. "They are telemetry, and then it's a lot of human analysis, and your SOC and your security and incident response teams doing a lot of the heavy lifting when they respond," and that, he said, cannot work at this scale. He pointed to the Hugging Face incident and "other frontier models escaping sandboxes to do things more autonomously" as the preview. "We have seen the trailer of this movie," Taneja said, and "every company in the world should prepare and rethink their architecture."
What the harness automates is the adversarial step. Before a fix counts as validated, the panel scores whether the patch negates the exploit, Taneja's test for done, with failed fixes feeding the next attempt, the iterate step Taneja described. Stage 11 itself runs read-only, per the README, and VVAH does not compile, build, or run tests against the patched tree. Taneja calls that wrapper "the governance architecture on top of that," and chaining findings into working exploits takes threat modeling and business context, which is why he argued "the harness with a model is far more effective than somebody using the model by itself."
Visa answers the gate question
VentureBeat put its questions to Visa in writing after the interview, and the answers arrived before publication. On why remediation ships on, the response repeated the bottleneck argument, then narrowed the scope. "VVAH is meant for authorized operators running against code they own, and in a controlled environment," the company said in written responses.
The approval question drew the most specific answer. "VVAH is a harness, not a merge tool," Visa wrote. "Stage 10 writes candidate fixes to a working copy of the repo. Stage 11 then runs an adversarial validation panel that scores each fix and returns one of three verdicts: validated, validation failed or needs review. None of these bypasses your normal build, test, and code review flow." Humans, the company wrote, are "the gate in three places. Before running the tool. When reviewing the patches. And before anything gets merged." "The final call on any fix stays with the security and engineering teams. In an enterprise, trust and auditability are not optional. The default flow is built around that."
Set beside Wilson's standard, the architecture lands close to his line and the sequence does not. Wilson's gate clears an action before it happens. The default's human gates open before the run and after the write. The attack is the automated part, and the three human gates sit outside the model, the boundary Wilson drew. "Our goal is to help security teams work at AI speed, not to replace them," Visa wrote. "VVAH does the repetitive parts. It finds issues, tests whether they are real, and proposes fixes. Before a fix gets to a human, an adversarial validation panel at stage eleven tries to break it. That way the human is spending time on decisions that need judgment, not on triaging noise."
Client zero got direct confirmation. "VVAH runs against Visa code today," Visa wrote, and Taneja had volunteered the posture on the call. "We designed this and we were using it for ourselves, and we were client zero," he said, adding "only when we saw the impact and the positive effect of what we were finding, we said every company would need this." What adopters value, Visa says, is context. VVAH pulls in CMDB data, threat models, and business risk, and where "most tools stop at findings," it "tries to answer, 'which of these should you fix first, given how your business runs.'"
Model choice becomes a per-stage decision
Multi-model orchestration is the other substantive change. "Mythos has a very high recall, but the Opus model has very high precision," Taneja said. "On stage one I want to use this model. On stage two I want to use this model," is how Taneja framed the per-stage setup, with newer GPT releases in the ensemble and open-weight models where pricing stings, all through configuration rather than code changes. "The whole is greater than the sum of the parts," as he put it. The harness was model-agnostic from day one, he added, and the evolution moved that choice into configuration, with prompt tuning and caching shared underneath. One boundary moved. In June, applying a fix required Anthropic backends, and OpenAI-compatible backends ran report-only. The current README extends remediation and validation to OpenAI-compatible and open-weight models through a shared model-agnostic runtime, with no single provider as a hard dependency, and the default routing for both stages stays Anthropic.
That flexibility lands on a market already churning. VentureBeat's Q2 2026 Pulse research found 59% of enterprises plan to adopt or switch agent security tooling within the year, and 82% still rely on provider-native controls as the primary layer. Visa said Thursday it is contributing VVAH to Nvidia's Open Secure AI Alliance as a model-agnostic framework and collaborating in Project Lightwell, the $5 billion IBM and Red Hat effort to harden open-source components.
Before turning fix mode on
Decision
What to establish first
Run posture
Start with --stop-after s9 and read the SARIF output before any run that can write to source files.
Approval gate
Map Visa's three human gates onto the pipeline, at run, at patch review, and at merge, and name who holds each.
Validation scope
Stage 11 verdicts score the fix. Build, test, and code review stay in the team's own flow, per Visa, so keep an exploit re-test before merge.
Repository scope
The tool runs with elevated privilege, per its own README. Fence which repos the harness can reach, and run scans in an ephemeral environment with scoped credentials, no production secrets, and network limited to the target repo and model endpoint. Write access to production code is the GhostJacking exposure class, an agent acting on data it read. Per the README's egress warning, any role routed through the SDK, OpenAI, or DeepAgents backends sends prompt data to that provider's endpoint.
Model roles
Assign models per stage deliberately. Recall and precision differ by model, per Taneja, the fix stages carry the highest blast radius, and the README states precision and recall figures are not yet published, so measure your own.
Consulting is the other half of Thursday's announcement. Visa Consulting & Analytics is adding executive workshops, a VVAH-informed maturity assessment scored on a NIST one-to-five scale, and a cyber risk prioritization roadmap. "We were getting a lot of calls. Hey, can you help?" Taneja said, and the practice "became very important to handhold and help those who are using it." Carl Rutstein, global head of Visa Consulting & Analytics, framed it the same way. "Finding vulnerabilities is no longer the hardest part. Speed to remediation is the new battleground."
As enterprises give AI agents more autonomy — the ability to plan, decide, and act across systems without a human approving each step — a hard question moves to the center of every architecture review: When an agent tries to complete an action that it was never authorized to do, what actually stops it?
These are your agents, running on your models, touching your data in your infrastructure — and the responsibility for what they do sits with you. That responsibility can’t be met in hindsight or with a set of abstract policies that live on paper but not in practice. Agents need rules in the context of the moment, because they don’t exercise overriding judgment of their own actions.
Consider a simple rule: Never open the car door. Followed literally, an agent could never get in or out of the car at all. But if you change the context (the car has just crashed, there’s a fire, someone is hurt and needs to get out), then the rule you actually want is the opposite. Context in the moment is everything. We are asking agents to do intelligent things; that requires intelligent rules.
The instinct is to add guardrails around the agent: instructions, policies, and monitoring layered above the model. Those mechanisms matter, but they share a structural limit: The car-door rule is plausible right up until the moment you actually have to decide whether to open the door. Controls at the agent layer are only as reliable as the agent’s output is predictable, and autonomy is precisely the property that makes that output hard to predict. Governance that depends on reviewing an action before it happens cannot keep pace with a system that acts in milliseconds, across many systems at once.
Governance has to become executable, and enforced where agents actually do their work: at the operational data layer, in the context, and exactly at the moment it is happening.
The data layer is the enforcement point
Agents create value by touching data. They query it, retrieve it, transform it, and increasingly act on it. A policy that says an agent should not reach a certain class of data is meaningful only if the system can deny that access at the moment the agent requests it. Additionally, a principle that says AI must be auditable is meaningful only if the organization can reconstruct what the agent did, what data it touched, which user it acted for, and what resulted. When governance lives at the data layer, it holds regardless of how the agent was built or how it behaves, because the control is a property of the database itself, not a promise made by the agent.
Agent behavior may be probabilistic. Governance cannot be
The enterprise should not rely on a model choosing to follow policy. The policy has to be enforced by the system. That is the difference between hoping an actor stays in bounds and constructing bounds it cannot cross to begin with.
The controls that make this real are ones many enterprises already run at the data layer: role- and attribute-based access, row- and column-level security, classification and masking, policy as code, and complete audit trails.
What agents change is not the mechanism, but who the mechanism has to recognize. Identity management has to treat the agent as a principal in its own right, with its own identity and a purpose declared when the session opens.
Once purpose is bound to identity, the policy engine can evaluate it the same way it evaluates role or department today, and the record of what happened can capture not just who acted and what they touched, but what they declared they were there to do.
In practice, this resolves into nine controls, grouped under three imperatives:
Enforce it
Role- and attribute-based access control enforced at query time, for agents as well as users
Dynamic column masking driven by the same policy path
Agent identity as a first-class principal, with declared purpose bound at session start and the acting user preserved
See it and prove it
Classification and tagging that drives policy
Session-level audit logging that records which agent acted, for which user, and under what declared purpose
Lineage across pipelines, so a result can be traced back to the request that produced it
Unify and harden
Centralized, portable policy management
Encryption at rest and in transit
Consistent enforcement across on-prem, cloud, and sovereign or air-gapped environments
“Declared purpose is what makes the difference. It becomes an attribute the access layer already understands, evaluated in the same policy path as role and row-level security. The enforcement mechanism does not change. What changes is that the agent's purpose is part of what it evaluates, and part of what the record proves afterward,” says Priyanka Jain, VP, product management, data & AI governance, EDB.
Wherever you are in your AI adoption journey, enforcement at the data layer is what lets you move faster rather than slower. The controls are already in the database. The difference is that agents now have to pass through them.
A digital leash, not a locked door
The goal is not to stop agents from doing useful work. It is to define how far an agent can go, what it can touch, what it can change, what requires escalation, and how the organization can reconstruct events if something goes wrong. Governed this way, agents are identified, scoped, monitored, and auditable. The enterprise can adopt them faster, because security, risk, and leadership teams trust the operating model underneath.
Open, sovereign, and enforceable at the source
Built on open source Postgres, this open foundation keeps enterprises in control of where their data lives, who can reach it, and under what policy, without ceding governance to a layer they don’t own or can’t inspect. For regulated industries, that combination of data sovereignty and source-level enforcement isn’t a nice-to-have; it’s the precondition for putting agents into production at all.
Agentic systems will keep getting more capable and more autonomous. That is a reason to be deliberate about where control lives, not a reason to slow down. The enterprises that enforce governance at the data layer can move aggressively on AI, because the thing protecting their data is more than just wishful thinking.
EDB Postgres AI is an open, enterprise-grade sovereign data and AI platform that unifies transactional, analytical, and AI workloads — with governance enforced where the data lives. For the full framework, see EDB’s white paperGoverning Agentic AI at Enterprise Speed.
Max Romanenko is Chief Technology Officer at EDB.
Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. For more information, contact sales@venturebeat.com.
Login
