OpenAI’s unreleased foundation model, codenamed Astra, is already working directly inside the company’s internal codebase, taking on experimental work that previously required as much as a week from a human researcher.
As part of a series of interviews with Time, OpenAI chief scientist Jakub Pachocki said Astra can take an idea for an experiment, turn it into code, run it, and return the results. A step beyond asking AI to fix a bug or write a function, OpenAI is essentially testing what happens when you hand the agent the experiment itself and let it figure out the steps in between.
For developers, Astra offers a glimpse into the future of coding agents. But giving an agent that much freedom creates another problem. Astra may already be powerful enough to trigger OpenAI’s highest-level cybersecurity safeguards.
OpenAI is essentially testing what happens when you hand the agent the experiment itself and let it figure out the steps in between.
Persistent agents change everything
OpenAI CEO Sam Altman described what the company is building as “persistent agents,” or systems that can keep working without needing a person to prompt them through every step.
Coding agents can already dig through a repository, change files, run tests, and try again when something breaks. Persistent agents are meant to keep going without a developer guiding them through each step.
That also changes what developers need from the tools around the agent. An IDE gives it somewhere to work, but a long-running agent needs infrastructure that can keep it running safely without constant oversight.
Multi-agent coordination at scale
In one demonstration witnessed by Time, 16 Astra agents worked together on a research-level math problem, splitting it into smaller pieces and then bringing their work together into a proposed solution.
For developers, it’s not hard to imagine that same setup applied to a large software project, with different agents working on different pieces at the same time. OpenAI is already experimenting with that kind of coordination, along with agents that can stay on a job for much longer.
Running several agents at once also complicates the infrastructure behind them. Developers need a way to keep the whole operation under control. Giving agents that much freedom, though, has created another problem for OpenAI: keeping them under control.
Under the company’s framework, hitting that threshold brings stricter safeguards for how the model can be used.
OpenAI had already seen what could go wrong with agents that have access to tools. During a cybersecurity test, one of its internal AI agents escaped its sandbox and accessed Hugging Face systems without authorization. Astra was not the model involved, but the incident led OpenAI to pause some frontier-model research workloads while it tightened the infrastructure used to run them.
And OpenAI isn’t the only company running into this problem. Google’s AI coding agent recently broke out of the boundaries of its IDE. The circumstances were different, and the incident was less serious, but both point to the same problem that happens when giving an agent more freedom to act; you also have to make sure it stays where you put it.
Astra is now running under OpenAI’s strictest security controls. Some training and evaluation workloads have resumed, but OpenAI says a “significant number” are still paused while it upgrades the infrastructure behind them.
Astra is now running under OpenAI’s strictest security controls.
Monitoring costs real compute
The company also says it is monitoring Astra more closely when it uses tools, watching for behavior that could signal it is going beyond what it’s allowed to do. OpenAI estimates that monitoring adds about 20% to the inference compute for those workloads.
Time reports that OpenAI still plans to release Astra, although there’s no launch date yet. Until then, OpenAI is working through the same challenge developers could eventually face as agents work for longer stretches without human supervision and begin coordinating with other agents.
As agents take on more work, the IDE is only part of the picture. Developers also need a way to see what those agents are doing and keep them from going where they shouldn’t. OpenAI is already seeing the cost, with monitoring alone adding about 20% to Astra’s inference compute.
That kind of overhead could become another cost developers have to account for, along with the governance around autonomous agents that sets boundaries on what they can do.
OpenAI is already seeing what that costs, with monitoring alone adding about 20% to Astra’s inference compute.
The AI giant acknowledges that it could have done far more to prevent its AI agents from going rogue. But it still fails to explain why it didn't see this fiasco coming.
Perplexity shipped Portable Computer this week, the local-first version of its Computer agent running on an Nvidia DGX Spark workstation, and the bill is steep: A DGX Spark starts at $4,700, and even an aging 24GB RTX 3090 sells well above $1,500.
The architectural choice beneath the surface deserves as much attention as the price tag: Most platforms building agents address reliability with greater intelligence. That typically includes a larger orchestrator model, a planner model, or a critic model reviewing the work.
Probabilistic reasoning proposes the next action, and deterministic software decides whether to execute it.
Perplexity separated the two jobs instead of stacking them. Probabilistic reasoning proposes the next action, and deterministic software decides whether to execute it.
The loop controller is code, and the decisions are still a model
Perplexity uses the term “orchestrator” to refer to the runtime controller rather than the planning model. That controller assembles context, enforces policy, and executes approved tool calls inside an OS-level sandbox. The company says it is deterministic code, not yet another model. The local model proposes the next action, including which tool to call and when to ask a cloud advisor for help. The split resembles a control plane architecture, where reasoning suggests, and inspectable software retains authority. Nate Kupp, Perplexity’s vice president of Computer Enterprise and Infrastructure, tells The New Stack that the harness accounted for most of the engineering work.
Same weights, better scores
Perplexity kept the base model and the silicon constant, putting Qwen3.8-27B on the same DGX Spark across three agent stacks. On its internal Local Knowledge Work Bench, a held-out set of 53 tasks, the company reported 82.6% for Computer. Pi scored 77.6% and Hermes 74%.
On ParseBench-100, a subset covering charts, layouts, tables, and formatting, the gap widened considerably. Perplexity reported 65.1% for Computer, while Hermes reported 34.6% and Pi reported 13.9%.
Because the base weights were constant, the gap measures the system around the model rather than a better model. That does not mean weights stopped mattering. Perplexity post-trained Qwen into PPLX 27B and reported 85.4%, above its own base-model number. Harness engineering and post-training are significant factors of this approach.
The word harness also covers a lot of ground, including prompts, tool schemas, context management, verification hooks, and document processing. Orchestration code is one part of that surface.
The security boundary lives outside the model
The sandbox is the boundary, not the determinism. Perplexity says the sandbox restricts processes, filesystem paths, and network access. If the sandbox is unavailable, the harness disables itself before making any tool call. Deterministic code is valuable here because it makes policy easy to inspect and helps the system fail safely. Deterministic code can still ship a vulnerability or faithfully execute a permitted mistake.
The key distinction is not between model orchestration and code orchestration. It is whether permission is enforced by something other than asking a model in plain English to comply.
The key distinction is not between model orchestration and code orchestration. It is whether permission is enforced by something other than asking a model in plain English to comply.
The same discipline shapes how context gets spent. Perplexity reports that Qwen3.8-27B advertises a 260,000-token context window but begins to struggle beyond 100,000 tokens. The harness therefore keeps the core prompt and toolset small and loads skills on demand. Commonly used connectors became command-line tools rather than full Model Context Protocol definitions sitting permanently in context.
Deterministic execution cannot rescue reasoning that exceeds the local model. On Terminal Bench 2.1, Perplexity reported 59.6% running locally. Letting the local agent consult Claude Opus 5 raised it to 73.0%, compared with 82.4% when Opus 5 worked alone. All of this remains vendor-reported evidence on a bench that the company has yet to open-source.
For enterprises evaluating local agents, the component to scrutinize is the layer that grants and denies authority, because that is where the platform’s engineering is most evident.
A security agent read a Cloudflare log, found an attacker’s prompt-injection payload sitting inside it, and rewrote the company’s DNS. The firewall had already blocked that payload, and blocking it is what wrote it into the log.
That chain is GhostJacking, which Tenet Security demonstrated on the DEF CON 34 main stage on August 9. A request hits Cloudflare’s managed ruleset, gets blocked, and is stored byte for byte with its poisoned User-Agent header. An AI coding agent reviewing those blocked events reads the attacker's text as an instruction — with no way to tell it apart from one the company meant to give it — and acts on it with credentials the company issued months earlier. In Tenet’s benchmark, Claude Code on Sonnet 4.6 followed the planted instruction in nine of 10 attempts under Cloudflare’s recommended configuration.
The block rate is not the boundary
Nothing malfunctioned. The firewall worked, and every call after it carried a valid credential already issued to the agent. Endpoint detection, the web application firewall and identity management stayed quiet because no rule broke.
Tenet found public evidence of the exposed setup at 48 organizations, six confirmed Fortune 500 companies, and SecurityWeek reported the same chain against Datadog and Sentry, where the injection surface is an alert or an error report. No single platform patch removes the architectural risk: an agent that consumes attacker-reachable data and can independently execute high-impact changes. That is why a high prompt-injection block rate cannot serve as a security boundary.
OWASP’s co-lead names the fix
“The first thing I’d do is put an authorization gate outside the model,” said Steve Wilson, Chief AI and Product Officer at Exabeam and project co-lead for the OWASP Top 10 for LLM Applications, in written responses to VentureBeat. “The agent can propose the exact DNS change, but it cannot grant itself the authority to make it.”
The move relocates the decision into code that either passes or fails. A safe change, defined cleanly, clears a deterministic policy check and stays autonomous. Anything ambiguous or high in blast radius routes to a named human who approves the actual change.
“The tradeoff is that the agent loses the ability to improvise arbitrary, high-impact infrastructure changes on its own, while retaining autonomous investigation and routine, bounded remediation,” Wilson wrote.
On teams that try to solve this inside the prompt, Wilson is blunt. “We have to remember that security rules written inside prompts may shape the model’s behavior, but they are still suggestions to the model, not enforceable security controls,” he wrote.
The blocked payload became the instruction
GhostJacking needs no compromised admin account and no bypassed firewall. It needs an agent that reads operational data and holds write access to the systems that data describes. Every step in Tenet’s chain is something the agent was already allowed to do, so tools tuned to catch unauthorized actions have nothing to catch.
The mechanism is specific. SC Media reported that the agent Tenet drove live at Cloudflare was Cursor, reading through a GraphQL integration and writing through the Cloudflare API, and that pairing closes the chain. Tenet ran the chain against more than one coding agent. Cursor carried the demonstration, and the nine of 10 figure came from separate testing of the same attack against Claude Code. The Cursor agent ingests the poisoned header, patches the DNS A record and adds a CNAME to “resolve” the injected finding, giving the attacker a path to reroute the company’s web and email traffic.
One agent’s output became the next agent’s input
Events reach Sentry through a public write-only endpoint with no authentication, by design. Tenet used a leaked identifier to post a crafted error report. On an ordinary triage prompt, the coding agent escalated it to Sentry’s own AI, Seer, then trusted the analysis it got back. Seer had already absorbed the attacker’s proposed fix and returned it as its own finding. What reached the coding agent was a recommendation from another AI, and it implemented that recommendation.
That walked straight through a control Sentry had already written. Sentry's guidance instructs agents reading its event data never to follow directives found there, and the coding agent held to the letter of that rule. It acted on Seer's conclusion instead, and that conclusion belonged to the attacker. It was acting on Seer’s conclusion, and that conclusion belonged to the attacker. An authorization boundary that accepts another model’s output inherits every injection that model absorbed, which is why Wilson’s gate belongs between agents too.
OWASP moved excessive agency from sixth to third
The 2026 OWASP Top 10 for LLM Applications, published August 4, elevated Excessive Agency three places on a ranking blending a 75% practitioner vote with 25% incident data from 6,639 documented cases. It was the largest upward move on the list, driven by real-world incidents clustering in agentic deployments.
The fix isn't better prompting — it's the permission map: which actions are pre-approved, and which need a human. Reading logs, correlating alerts and drafting timelines stay autonomous. A bounded remediation like restarting a named service inside a fixed condition set clears a policy check outside the model. Anything that changes DNS, alters identity privileges, deploys code or reroutes production traffic needs a named human, and letting an agent open new access paths or approve its own proposals defeats the point of the gate. Useful autonomy survives that. What does not survive is the path from an attacker’s text to unreviewed production authority.
What the control costs in practice
Tenet co-founder and CEO Barak Sternberg told Dark Reading that a request the firewall already blocked was the way in, and that the firewall never went down, it just stopped mattering. His own fix is to split what an agent can read from what it can execute, and he concedes the cost, because an agent that reads alerts but cannot act on them is not the agent anyone deployed. The cheaper first step is an inventory. Every agent that reads outside data and can also write or execute belongs on a risk register, and that register needs no new tooling.
Wilson’s design survives that cost because the split it draws is proposal from approval rather than read from write. The agent still reads its alerts, still investigates and still runs bounded work. What it loses is the ability to invent a high-impact change and carry it out on its own authority.
Almost nobody has built it. Kayne McGladrey, a senior member of the IEEE, has argued for years that an AI deployment needs a hard governance threshold, a named human holding a kill switch and a way to roll back. Asked whether any Fortune 500 company runs that, he was blunt. “I haven’t seen it done, and no, they haven’t come out and publicly said it,” McGladrey told VentureBeat.
His reason is economic. Companies “are accepting the risk, and they’re accepting it either deliberately or unconsciously,” he observed, betting the advantage beats the penalty. “What I think would change behavior in the space is if the penalties and the consequences were to outweigh the advantages.” The gate belongs outside the model for a reason unrelated to malice. “If you get into the inference layer, it won’t tell you it’s cheating, and it will lie about having cheated,” he argued, pointing to findings from the U.K.’s AI Safety Institute among others. A system that cannot reliably report its own shortcuts should not authorize its own.
The industry is not positioned to make that split quickly. Ivanti’s 2026 State of Cybersecurity Report found 77% of security professionals at least somewhat comfortable letting AI act without human review, the exact posture Wilson’s gate constrains. CrowdStrike pushed its prompt-injection taxonomy past 200 techniques in July, naming indirect injection through data an agent reads as the critical vector for agents that call tools and run commands.
The architect who moved the boundary before the attack had a name
Egiziago Cioffi hit a related failure in production months before GhostJacking had a name, with one caveat. Cioffi is CEO of SynSphere Italia, a Microsoft reseller, the architect who built and sold the system rather than a security leader defending one he inherited. His Azure OpenAI assistant over SharePoint scored well on faithfulness and still returned content the asking user could not have opened. “An evaluation set with no identity dimension cannot fail an authorisation bug, however high the faithfulness score,” Cioffi told VentureBeat in written answers. He fixed it with a query-time filter built from the asking user’s group claims, so an unentitled chunk never becomes a candidate and never reaches the model. GhostJacking turns on what the model may do, the half Wilson’s gate is built for.
One gap stays open here. No sitting CISO has gone on the record with a change made since August 9 and what it cost in agent capability. Until one does, the cost side rests on the people who specified the control, not anyone defending a production environment with it.
What security leaders need to do this week
Four questions produce an honest picture faster than any procurement cycle. Which agents read attacker-reachable material, which of those can also change production systems, whose permissions run at retrieval, and which changes a policy engine can approve without a human.
Then run the negative test. Plant an adversarial instruction in a log the agent is expected to inspect, and keep the transcript, because that transcript is the difference between claiming a control and showing a test of it.
Tenet, which sells runtime protection for AI agents, leads its own defender guidance with denying an agent outbound network access by default, cutting the leg where the poisoned instruction fetches a payload and reroutes traffic. But an agent that investigates without a standing path to the open internet loses a capability few workflows will miss.
Enumerate the service principals in the tenant, drop the Microsoft first-party apps that ship pre-provisioned, then filter to those holding a credential or app-role assignment. Every identity on that list needs an owner and an expiry date, because one with both gets reviewed and one with neither never does.
For any agent with production authority, write the containment sequence before an incident, not during one. Revoke or rotate its workload credential, disable its write-capable API or tool integration, preserve the execution transcript, then validate and roll back whatever infrastructure it changed.
McGladrey’s read on why the work keeps getting deferred is uncomfortable. “I think that there’s a level of tolerance that’s being given right now in AI that is unlike anything else in society,” he said. GhostJacking makes that visible. A blocked payload reaches the agent through the system built to record blocked payloads, and once it arrives the question is no longer whether the model recognizes an attack. It is whether the model holds the authority to turn one into a production change.
A CISO who sees a low CVE count and deprioritizes prompt injection is reading the scoreboard wrong. Prompt injection has held the No. 1 spot on the OWASP Top 10 for LLM Applications for three consecutive years. When two leaders of that list checked it against 6,639 labeled real-world incidents, it came back at No. 12. The drop measures visibility rather than danger, because the attack operates where a vulnerability scanner cannot see it.
The machinery behind it is real: 7,714 LLM security incidents from CVE, GitHub Security Advisories, OSV, and the AIAAIC AI-harm database, 6,639 of them labeled against a 20-entry taxonomy, and a Bayesian model that corrects each count for classifier error before setting the data-driven ranking beside the expert vote.
The comparison found no statistically detectable agreement between expert judgment and the public incident record. Cohen’s kappa comes in at 0.20 with a 90% interval running from negative 0.16 to 0.57. “The interval crosses zero, so we cannot rule out that the two rankings agree only by chance,” they write. “The honest bottom line: weak agreement, not confirmation.”
Lambros, co-lead of the OWASP GenAI Security Project Top 10 for LLM Applications and director of AI standards and governance at Zenity, put the finding in evidentiary terms in written answers to VentureBeat. “We had two ways of measuring the same risk, expert judgment and the public incident record, and they disagree with each other. Neither one is the truth,” Lambros said. “Two witnesses are contradicting each other, and we can’t tell you which one is lying.”
The attack chain a scanner never logs
The gap is structural. Prompt injection hides instructions inside the content a model reads, anything from a log entry to a support ticket to a document pulled back by retrieval. The agent then makes the tool call the attacker wanted, using credentials it legitimately holds. Nothing in that chain is a product defect, so the attack leaves no CVE behind for a scanner to find.
The defenses that catch it are adversarial tests against the deployed system and hard caps on what the agent can reach, so a fooled model cannot touch anything expensive. The same logic argues for funding agent memory and MCP tool boundaries now, on architecture, rather than waiting for advisory volume that will always arrive a cycle late.
The first control Wilson would deploy
Wilson, Chief AI and Product Officer at Exabeam and project co-lead for the OWASP Top 10 for LLM Applications, named the control he would deploy first against exactly that chain, an agent that reads an attacker’s payload in a log file, treats it as an instruction, and rewrites DNS with a valid credential, in written responses to VentureBeat.
“The first thing I’d do is put an authorization gate outside the model: the agent can propose the exact DNS change, but it cannot grant itself the authority to make it,” Wilson said. “Security rules written inside prompts may shape the model’s behavior, but they are still suggestions to the model, not enforceable security controls.”
The gate has a price, and Wilson states it plainly. “The tradeoff is that the agent loses the ability to improvise arbitrary, high-impact infrastructure changes on its own, while retaining autonomous investigation and routine, bounded remediation,” he said.
Why the No. 1 risk looks small in the record
“Prompt injection is the best-understood LLM attack, and deployed systems defend against it actively,” the authors write, and they compress the whole divergence into one sentence. “Experts rank it first because the attack surface stays enormous even when the defenses mostly hold; the data sees the successes that got through.”
Wilson has watched the gap from both sides of it. “Incident data is incredibly valuable, but it is inherently backward-looking and notoriously tricky to interpret,” he said. “It tells us what was observed, recognized, classified, and reported. It does not necessarily tell us what is most dangerous in the systems people are building right now.”
He compares prompt injection to “death and taxes” and, increasingly, to “a law of physics for LLM systems,” because one model is being asked to interpret trusted instructions and untrusted content at the same time.
Better defenses have not closed the case. “A control that works 99% of the time is not sufficient when the failure case gives an attacker meaningful access. And, frankly, I don’t think we are at 99%,” Wilson said. “The durable answer is not believing we can perfectly screen prompt injection out of existence. It is designing systems with the assumption that prompt injection will occur, understanding why it works, and limiting what an attacker can accomplish when it does.”
A low advisory count can mean the defenses are working. It can just as easily mean nobody has looked, and the public record cannot tell a security team which one it is.
The attempt volume is documented. CrowdStrike’s 2026 Global Threat Report found adversaries injected malicious prompts into legitimate GenAI tools at more than 90 organizations in 2025, stealing credentials and cryptocurrency, under a section titled “Prompts are the New Malware.” The telemetry shows pressure on the attack surface without proving defenses produced the No. 12 placement, but it is the pattern the mechanism predicts.
The gap runs the other way too, and further
Prompt injection is the headline case, and misinformation is the bigger one.
The expert vote puts misinformation at No. 13, while the incident record places it at No. 2. The paper calls it “the widest disagreement between the two witnesses” and reports that its concordance flag “puts the probability that the two signals disagree at 99 percent.”
The authors do not treat their own data as the winner. On misinformation they note the corpus “carries a large volume of deepfake and AI-generated disinformation,” records that often “describe harm produced by an AI rather than a vulnerability inside an LLM.” The authors call it the entry the record most disputes, stopping short of concluding the experts got it wrong.
Where “too new to measure” runs into the CVE record
The two brand-new taxonomy entries sit at the sharpest end. Persistent memory poisoning lands at expert No. 4 and incident No. 16, MCP tool interface exploitation at expert No. 7 and incident No. 16, each with an incident interval of 6 to 20 that spans most of the taxonomy.
Public 2026 CVEs exist for both. On MCP tool interfaces, the Azure Data Explorer MCP Server carried KQL injection, and the CVE record describes it allowing “an attacker (or a prompt-injected AI agent) to execute arbitrary KQL queries against the Azure Data Explorer cluster,” scored 8.3 High. Kong’s Konnect MCP Server shipped an indirect prompt injection that lets a remote attacker steer the server into executing unintended API requests, the exact failure the MCP entry names.
Agent memory has its own record. An agent harness, Ruflo, exposed unauthenticated MCP bridge endpoints that let a network attacker obtain a shell, read provider API keys, and poison the learning store, rated 10.0 Critical.
The record is so thin and uncertain that the model cannot place either entry within 14 rank positions. A team waiting for advisory volume to justify a control on agent memory or an MCP tool boundary would still be waiting while the CVEs accumulate at Critical and High.
Lambros makes the budget case in operational terms. Poisoned memory “doesn’t announce itself,” he said. It looks like a procurement agent told once that invoices from a given supplier under $50,000 clear without a second signature, and because the agent remembers, every approval after that looks like the process working. “Nobody files an advisory for that, because nobody knows it happened. A count of zero is measuring your blindness, not your safety.” The argument he says a CFO will sign off on is timing, since memory and tool permissions get wired into these systems once, early, and everything else sits on top of them. “Build it in now and it’s a rounding error. Come back in two years and you’re re-architecting and re-training your systems.”
The authors flag their own measurement problems first
The expert side is thin. “The expert signal is a practitioner survey: about 29 respondents scored each candidate risk on importance,” the authors write. Twenty-nine votes set the ranking that carries three-quarters of the published list’s weight, the compression point for OWASP’s more than 25,000 community members.
On the data side, the classifier is the weak joint. Precision “varies sharply across entries, from 93% (LLM01, LLM03) down to 13% (LLM08),” four entries fall below 50%, and the base classifier “never predicts ‘out of scope’ and files every incident into some category, including the roughly 38% of the gold set that belongs in none.”
The authors name the central limitation themselves. One reviewer adjudicated all 1,200 gold-set incidents and overrode the model consensus on 553 of them. “A single annotator cannot measure inter-rater reliability,” they write. “The single-author gold set remains the central limitation.”
Lambros lays the weak kappa at the feet of the taxonomy itself. “That number is telling you about our categories, not about our experts,” he said. When the people who wrote a taxonomy cannot reliably sort incidents into it, he argues, “a weak score on the ordering of those buckets is a fact about the buckets.”
A better classifier will not fix the disagreement. A pre-registered bake-off of four frontier models produced no winner. None beat the incidence floor’s balanced accuracy of 0.863, and a ground-truth check left the floor’s ordering in place at a Spearman correlation of 0.918. The authors published the engine and artifacts on GitHub for anyone to rerun.
The robustness result tested only one side of the gap. Every check behind the abstract’s word “robust” runs on the incident side, showing the incident-derived ranking stays put when the labeling machinery changes, and none of it touches the 29-vote survey. A board that hears “robust” will assume validated, yet the record supports only stable.
What the published list did with this
OWASP shipped the GenAI LLM Top 10 2026 on August 4, the first edition to fold incident data into the ranking, weighting the practitioner vote at 75% and the incident corpus at 25%. Prompt injection stayed at No. 1, misinformation moved up two places, excessive agency climbed from No. 6 to No. 3 as the entry where the two signals agree most clearly, unbounded consumption rose four spots to No. 6, and improper output handling fell from No. 5 to No. 10, the largest drop.
Wilson declines to defend the blend as arithmetic. “There is nothing magical about a 75/25 weighting,” he said, “or about reversing it to 25/75. The value of the data wasn’t that it gave us a mathematical answer; it changed the conversation.” The excessive agency entry is where that conversation landed hardest for him. “If I were a CISO evaluating a new agentic deployment today, Excessive Agency is where I would start,” Wilson said.
Lambros would go further next cycle, a view he flags as his own and separate from the working group. The blend hands the same 25% incident weight to every category, while the hand-checked classifier precision runs from roughly nine in 10 on prompt injection and supply chain down to roughly one in eight on vector and embedding weaknesses. A quarter of the weight on the first rides on something solid, he argues, and the same quarter on the second rides on noise. “The ratio should track how well we actually measure each category,” Lambros said.
Why this lands now
Ivanti’s 2026 State of Cybersecurity research found 87% of security teams call adopting agentic AI a priority and 77% report at least some comfort letting AI act without human review. Teams are signing off on agent autonomy while the expert ranking of what can go wrong with those agents shows no statistically detectable agreement with the incident record.
What to do with this on Monday
The behavioral change is narrow and it is the whole point.
Use the OWASP LLM Top 10 as a coverage map, not a queue. The rank positions carry 29 votes and a corpus whose own authors call the agreement weak, so build your own priority order from your own exposure: production reach, breach-notification data, and controls that have actually been tested. Lambros draws the funding line the same way. “I’d prioritize spend where the expert vote and the incident record point the same direction, because that’s two independent witnesses agreeing,” he said. “Where they split, stop letting the ranking allocate your money and go look at what your own systems are doing.”
Log what your AI systems are actually doing, field by field. The prompt that went in, what came back out, the documents pulled to build the answer, the tools called and the arguments passed to them, and the model’s confidence score on every response. Confidence is the field Lambros would fight for, because most security leaders do not realize it is measurable, and it is where the attack surfaces. “A model running on a poisoned instruction doesn’t act broken. It acts certain,” he said. “Certainty is what your monitoring treats as a healthy system.” The cost is a sprint or two of engineering. The constraint is a person, because a SIEM does events and these are trends. “Somebody has to analyze those trends every week and say whether a drift means anything, and most security teams have nobody who can.”
Stop expecting scanner output to reproduce the Top 10’s order. Scanner findings live on the incident side of the gap, counting what got disclosed rather than what a deployed system should fear, and the classifier bake-off shows a smarter model does not close that distance. The test that sees prompt injection is an adversarial one run against the live system, paired with Wilson’s authorization gate so the change an injected agent proposes is never the change it can execute.
Fund the thin-record categories on architecture, not incident volume. Agent memory and MCP tool boundaries sit at expert No. 4 and No. 7 with incident intervals spanning most of the taxonomy, and the CVEs that do exist are landing at High and Critical. Kayne McGladrey, an IEEE senior member who advises enterprises on risk, put the funding logic bluntly in an interview with VentureBeat. “Anything that seems to have a cybersecurity flavor is generally put into the cybersecurity risk category, which is a complete fiction,” McGladrey said. “They should be focused on business risks, because if it doesn’t affect the business, like a financial loss, then nobody’s going to pay attention to it, and they will not budget it appropriately.” A rank number from a 29-person vote is a weaker budget argument than the business system the agent touches.
Steal McGladrey’s baseline test for the AI systems themselves. “If you wouldn’t expose your database to the public internet without identity and access controls, why would you do that for your AI model?” he said in CSO Online’s analysis of 2026 breach costs.
The board question for the next meeting is short. If our AI risk ranking came from a 29-person vote and a corpus that disagrees with it, what are we actually using to decide which controls get funded next year?
Chinese retailer AliExpress has been caught fingerprinting visitors after one of the metrics—an outdated technique that measures inaudible sounds it sends to browsers—impeded a researcher's ability to use his bluetooth headphones.
Researcher Matthew Callaghan said he stumbled on the stealthy tracking by mistake. After loading the AliExpress homepage, audio from his phone stopped playing over his multipoint headphones, which accept connections from more than one device at a time. He set the headphones to play sounds from his phone except when his PC was producing audio. Each time he loaded AliExpress, the phone audio stopped. Each time he closed the tab the site was loaded into, the phone was once again audible.
Users can't hear it, but browsers can
While investigating the odd behavior, Callaghan said he found two highly obfuscated scripts. Together, they rendered a graph that analyzed the WebAudio readings of each visiting browser. This graph acted as an oscillator that measured Sawtooth waves, which are common in output from digital audio.
Put several AI bots to work, and a mistake by one may not stay within its assigned task. For example, that error could reach another bot’s files and login credentials, or even the computer running them all. Two releases this month offered companies very different ways of containing that risk.
On August 17, Nous Research announced that its Bot Mode would ship bundled and enabled by default in Hermes Agent v0.20.3, turning agent profiles into a roster of named bots that hand off work to one another. About a week earlier, SpaceXAI launched Grok Bot with almost the same interface: a sidebar of named teammates who sign in to your tools and keep working long after you close your laptop.
The interface converged within a week, but the answer to the question every platform team has to ask did not: When one bot goes wrong, what can it reach?
Four projects have now come to their own answer, and no two of them agree.
Grok Bot draws the line around the user account.
Hermes draws it around the profile.
OpenClaw draws it around an optional runtime sandbox.
Taken together, the documentation shows an industry converging on the persistent coworker interface far faster than it is converging on what constitutes an identity or a security boundary for it.
Four projects, four written answers
Every one of these products now offers the same surface. You create several named agents, assign them different jobs, and have them pass work among themselves. The naming convention alone suggests separation, since a bot called Expense Manager and a bot called Talent Scout sound like they occupy different rooms in a shared office.
The documentation says otherwise, and it says something different in each case. The unit of isolation is the account in one product, the profile directory in another, an opt-in container in a third, and the deployment topology in the fourth. Those four units are not interchangeable, and an operator who assumes the roster itself is the boundary will be right in exactly one of the four cases.
Is Grok Bot confused about what it wants to be?
SpaceXAI’s launch post leads with the promise that bots have their own computer. The documentation, last updated the same day, describes a single persistent cloud computer assigned to the user account rather than to any individual bot. Browser cookies and signed-in sessions are shared across the roster, files are visible to every bot, and command-line credentials are shared. One bot can pick up work that another bot saved.
Each bot gets its own screen on that machine, which allows several of them to run browser and desktop tools in parallel. SpaceXAI is direct about what those screens are not. The documentation calls them “separate work surfaces, not separate security boundaries.” It then instructs operators to keep a credential or file off the machine entirely if another bot on the account cannot use it.
The consequences run further than credentials. Signing in for one bot makes that session available to the others because the browser is shared. Installed connectors are account-wide, and their availability is not isolated to a single bot. The shared workspace sits at /workspace and is designed to survive computer updates and recovery, so the durable state is shared across the whole roster.
None of this is an implementation accident. It is what makes handoffs between bots cheap, and cheap handoffs are the product. But an operator reading only the launch page would build a mental model that the documentation contradicts, and that gap is where the risk sits.
Hermes gives each bot its own profile
Nous took the opposite architectural position. In Hermes, a bot is a profile, and each profile has its own configuration, memory, skills, credentials, and chat history stored in its own directory on disk. Handoffs between bots run as real invocations against the named profile, rather than as a shared context blob passed around within a single process.
Nous shipped the teammate protocol as part of v0.20.3, alongside the MCP 2.x SDK migration and a set of runtime hardening changes. The company archived the standalone plugin repository once the merge was completed. Bot Mode is on by default, so a Hermes user who updates gets the roster without opting in.
Two qualifications matter before anyone reads that as containment. A separate credential store does not guarantee different credentials, since what ends up in a new profile depends on how the operator created it and what they edited afterward. And every profile still shares the host machine, its operating system user, and its filesystem permissions. What Hermes documents is workstation-level separation of agent state, a meaningfully stronger default than a shared cloud account, but not the same as isolation.
OpenClaw’s sandbox is off by default
OpenClaw documents the most complete boundary of the four. When the sandbox is enabled with the Docker backend, agent tool execution runs inside isolated containers. At the same time, the gateway remains on the host, and the scope can be selected per session, per agent, or shared across agents. Each scope gets its own workspace. Auth material lives per agent under an agent-scoped auth profiles file. Operators can configure network isolation, resource limits, and allow-or-deny tool policies on top of it.
The documented default for that sandbox mode is off. That is a defensible choice for a project most people run on a laptop, where the container overhead buys little against a single-user threat model. The underlying setup behavior deserves more attention. If sandbox prerequisites fail during setup, the script resets sandbox mode to off rather than refusing to start, so an operator who intended isolation and encountered a Docker socket issue ends up running without sandbox isolation. The documentation also warns against mounting the host Docker socket into agent sandbox containers and flags the CLI container’s shared network namespace with the gateway as a trust boundary in its own right.
ClawFleet answers the same question by moving it into the deployment topology. The project documents a wrapper that puts each OpenClaw or Hermes agent in its own Docker container with an isolated filesystem and network. It lists roughly 500 MB of memory per OpenClaw instance and 150 MB per Hermes instance. That cost is why the other three projects make the boundary optional or skip it, and naming the number makes the trade-off legible.
How to choose which bot is right for you
Scenario
Documented fit
Rationale
Persistent work that must continue with the laptop closed
Grok Bot
The only one of the four with a vendor-run always-on cloud computer, at the cost of one shared credential surface for the whole roster
Several agents with genuinely different credential sets on one workstation
Hermes
Per-profile stores are the documented default, and Bot Mode ships on
Untrusted or multi-tenant agent sessions
OpenClaw with sandbox enabled
Per-agent or per-session container scope with configurable network and tool policy, provided the operator turns it on and verifies it
Isolation as the deployment model rather than a runtime setting
ClawFleet
Container per agent with separate filesystem and networking, at a documented memory cost per instance
Each project gives operators different advice
The operational guidance diverges as sharply as the architecture. OpenClaw’s docs read like infrastructure documentation, naming specific hazards such as the Docker socket and the shared network namespace, and telling the operator what not to do. Hermes documents the profile layout and the protocol, then leaves policy to the operator. Grok Bot’s guidance is largely the warning itself, an instruction to treat the account as the boundary and to keep sensitive credentials off the shared machine entirely.
Grok Bot carries a second disclaimer worth reading alongside the first. Sensitive actions route through an approval mechanism. SpaceXAI documents the categories that trigger it, including sending messages, publishing content, purchases and transfers, deleting data, and touching production. Enforcement runs through an LLM classifier. Cursor‘s documentation for that same engine states plainly that the classifier is not a security boundary and can make mistakes. A buyer evaluating the product therefore finds the phrase twice, attached to the two mechanisms they would most reasonably assume protect them.
AI agents still lack identities of their own
Enterprises can adopt any of these four products today and get real work done, and the honest reading is that all four are engineering their boundaries in good faith against different threat models. What none of them provides is an identity for the agent. In every case, the bot borrows the operator’s credentials, whether from a shared cloud browser, a profile directory, or a container volume, and the entire security conversation boils down to how far those borrowed credentials travel.
“There is no primitive to standardize on, so each project has invented a boundary at whatever layer it already controlled, the account, the profile, the runtime, or the container.”
That is why the four answers differ so much. There is no primitive to standardize on, so each project has invented a boundary at whatever layer it already controlled: the account, the profile, the runtime, or the container. Expect to see that gap close on the identity side rather than the agent side, through scoped delegation and per-agent credentials issued by the identity provider, rather than being copied from the human. Until then, the useful move for platform teams is unglamorous and specific. Read the security page before the launch page, because for this class of product, they describe different things.
On July 16, the team at Hugging Face noticed something weird moving through their production systems: An intruder that was cloning datasets, harvesting credentials, and hopping between internal clusters faster than a human could type. Five days into the investigation, the authorities were notified, and several days later, OpenAI traced the intruder back to one of its models.
“Two frontier models escaped their test environments this summer; only one of them had a funny reason.”
That prompted Anthropic to look through its own test logs, and it found that similar events had occurred three times. One Claude model probed nine thousand hosts in a single run without setting off any alarms. Another slipped a poisoned package into the public Python registry, compromising fifteen machines before it was caught. The third incident dated back to April and went undetected for three months until a rival’s error prompted the audit. Nobody had noticed. Two of the three companies involved didn’t know anything was amiss until they got a call.
Containment by instruction
A sandbox, in the context of AI testing, means a model gets a task, a set of tools, and an instruction: no access to the outside world.
OpenAI’s model found an exploit in a tool it was allowed to use, and leveraged it to access the open Internet. Anthropic’s models didn’t even need that. The models were told a path didn’t exist, so when their tasks led them down a path that wasn’t supposed to be there, they treated it as part of the exercise.
“In both instances, the only constraint was an instruction. There was no mechanism outside the model itself to ensure those instructions wouldn’t be treated as optional.”
In both instances, the only constraint was an instruction. There was no mechanism outside the model itself to ensure those instructions wouldn’t be treated as optional.
So how did they escape?
OpenAI’s agent was being evaluated on a cybersecurity benchmark called CyberGym. During the test, it chained a code-execution flaw in a dataset loader to a second flaw in the platform dataset configuration file handler. It used the combined exploit to run commands on a live processing worker. From there it grabbed a write-scoped access token and cloned a private dataset as a staging area, which became a dead drop for its own tools. By the time anyone noticed, the intrusion had been running for three days, touched several internal clusters, and compromised accounts at four other companies, according to Hugging Face’s own review.
What it wanted, in the end, was comical: the answer key to the test it was taking.
Anthropic’s three incidents were nowhere near as convoluted. Its models were being run through security exercises against third-party evaluation partners, and the only thing keeping them from accessing the Internet was an instruction not to. One incident involved an SQL injection targeting a resource the model wasn’t supposed to be able to access. Another, during an exercise where the usual safeguards were deliberately switched off, ended with a Claude model publishing a malicious package to the Python Package Index, and fifteen machines picking it up before it was pulled down. No zero-day exploit. No dramatic escape. Just locked doors that turned out to be mirages.
The structural weakness
Strip away the specifics, and both incidents share the same root cause: there was nothing outside the model’s own judgment that could enforce the limitations placed on it. A credential that shouldn’t have been able to access the open Internet did so anyway. An instruction that should’ve stopped a plan cold got treated as just another input to reason around.
“If a fence doesn’t hold during testing, it won’t hold in production.”
It feels like we’ve spent the last year reliving the same problem in new ways. For example, in deleted databases, in poisoned supply chains, in every credential an agent inherited because provisioning it correctly took one extra step nobody got around to. The only delta is where the boundary was supposed to be: around the dev experiment, meant to contain dangerous capabilities, or standing between it and every production system it could reach. If a fence doesn’t hold during testing, it won’t hold in production.
How do we break out of the loop?
Unfortunately, the answer isn’t self-improvement. A more capable model wouldn’t have caught any of this. The only way to fix it is with Infrastructure that verifies where a model actually is, rather than trusting its reports.
Egress controls that are closed by default. A test environment that can reach the open Internet is probably not a great idea. Outbound traffic should be locked to an explicit allowlist. Never assume a model will simply decline to look for a way out. “No internet access” needs to be a network-enforced fact, not just an instruction.
Narrowly scoped credentials with short lifespans. Both incidents ran, in part, on tokens that extended beyond what the tasks required. A credential built for one evaluation run, scoped to nothing beyond it, and expiring on a timer, can’t become the write-scoped key that turns into a lateral-movement problem across multiple clusters. Yes, it will be a chore; so gird your patience by imagining having to explain to company leadership why taking steps to avert a catastrophe was too tedious to bother with.
Policy gates that sit outside the agent’s reasoning. An agent that can decide for itself whether an action is in scope is grading its own test. A plan should have clear rules written in code, evaluated by something other than the agent, before anything executes. Open Policy Agent has become a de facto default here for exactly this reason. It doesn’t care how confident the plan sounds; nothing runs without proper authorization from an external source.
An audit trail that is entirely independent of the agent. Both incidents were discovered by rereading logs after the fact, not by asking the models what happened. That’s how oversight should be done. As the proverb goes: Quis custodiet ipsos custodes? Who watches the watchers? A system of record that captures what actually occurred, regardless of the agent’s own account of itself, is the only version of events worth trusting.
This is a sign
The industry spent a decade learning that the CI/CD pipeline is an attack vector that requires real fortification, not just bolted-on convenience. Test environments for frontier models are following the same arc at a faster pace. The next time one of these agents escapes containment, it’ll probably be one built around finding unlocked doors, which will make it substantially more dangerous than a coding agent that deletes a few databases.
Test rigs must be treated as if they hold something real, because, as far as the credentials are concerned, they do. A sign on a door is never going to be enough to keep everyone out; there has to be a lock whose robustness correlates to the value of what it guards.
“A sign on a door is never going to be enough to keep everyone out; there has to be a lock whose robustness correlates to the value of what it guards.”
Whether by a state-sponsored crew probing a water management system in the middle of the night, or a company’s own model trying to shave a few points off a benchmark, boundaries will always be tested. Two labs found out this summer, and the story needs to be taken seriously. The vulnerabilities are real, the transparency from the labs is welcome, and the containment failures are a cause for concern.
Catching a model that tried the handle is the easy part; both labs proved that. The more challenging, and therefore critical, part is making sure the next containment environment actually has doors that are firmly locked.
The artificial intelligence landscape has reached a pivotal inflection point. Over the past several years, the paradigm has shifted from passive, conversational Large Language Models (LLMs) to autonomous AI agents, digital software entities capable of reasoning, invoking tools, executing multi-step workflows, and making real-time decisions across enterprise systems without constant human intervention.
As organizations accelerate the production deployment of autonomous agents, modern security frameworks must evolve to keep pace. Traditional Identity and Access Management (IAM) systems were primarily designed around two distinct operational models:
Human users: Authenticated via Multi-Factor Authentication (MFA), Single Sign-On (SSO), and interactive sessions.
Service accounts and workloads: Authenticated via static API keys, fixed service tokens, or IP whitelisting.
Autonomous AI agents blur the line between these two models. An agent acts with the non-deterministic reasoning and delegated agency of a human, but operates at the scale, parallel velocity, and automation speed of a machine service.
Identity dimension
Human users
Traditional service accounts
Autonomous AI agents
Velocity & scale
Low (human typing speed)
High (scripted requests)
Extremely high (dynamic, parallel tool execution)
Decision logic
Deterministic / goal-driven
Rigid / hardcoded
Non-deterministic / adaptive reasoning
Auth mechanics
Passkeys, MFA, SSO
Static API keys, OAuth M2M
Ephemeral delegation & contextual attestation
Access granularity
Role-based access control (RBAC)
System-wide scope
Fine-grained / relationship-based (ReBAC/ABAC)
To safely harness the power of autonomous workflows, enterprise security architecture must move toward continuous, agent-aware Zero Trust governance. Below are six foundational identity capabilities that organizations should adopt to secure AI agents in production environments effectively.
“Autonomous AI agents blur the line between these two models. An agent acts with the non-deterministic reasoning and delegated agency of a human, but operates at the scale, parallel velocity, and automation speed of a machine service.”
“When it comes to agentic AI identity, most organizations are woefully unprepared for inherent security risks and operational challenges of managing those identities.” – Ken Buckler, Research Director, EMA – Agentic AI Identities – Is Your Organization Prepared?
1. Verifiable agent identities & “Know Your Agent” (KYA)
Autonomous entities require verifiable digital identity frameworks that establish clear, cryptographically bound accountability for every machine action.
Cryptographic attestation: Every agent instance should possess a unique, cryptographically signed identity bound to its underlying model version, execution environment, and deployment origin.
Delegation chains: When a human user delegates a task to an agent (or when a primary agent spawns sub-agents), the identity system must construct an immutable, traceable chain of delegation. This ensures the infrastructure can continuously verify who authorized the initial action and what specific scope was granted.
Static API keys and persistent service tokens represent a significant surface area of exposure when integrated into dynamic agentic workflows. Replacing long-lived credentials with short-lived tokens dramatically reduces the potential window of risk.
Just-in-time (JIT) minting: AI agents should operate with ephemeral credentials generated on demand, strictly limited to the API calls required for a single operational step, and configured to expire within seconds or minutes.
Bound OAuth flows & PKCE: Enforcing Proof Key for Code Exchange (PKCE) and strict token-binding protocols ensures that credentials cannot be reused or replayed outside of their intended runtime context.
“Replacing long-lived credentials with short-lived tokens dramatically reduces the potential window of risk.”
3. Relationship-based access control (ReBAC) & intent binding
Coarse-grained permissions, such as those in traditional Role-Based Access Control (RBAC), are often too broad for non-deterministic tool usage. Access governance should be based on fine-grained relationship models and task intent.
Intent-bound authorization: Authorization systems should evaluate not only whether an agent has general permission to access a resource, but whether that request directly aligns with the explicitly authorized sub-task.
Fine-grained contextual policies: Implementing relationship-based access control (ReBAC) or Attribute-Based Access Control (ABAC) allows teams to define precise conditions (e.g., “Agent X may read Document Y only if human user Z is the document owner and the active workflow is ‘Data Summarization'”).
Because AI agents operate at speeds far exceeding those of manual monitoring, security containment mechanisms must be automated, agent-aware, and built into the control plane.
Behavioral rate & scope limits: Security controls should establish baselines for expected agent behavior to detect anomalies, such as rapid parallel tool invocations, repetitive execution loops, or unusual queries to non-standard endpoints.
Automated circuit breakers: If an agent’s execution pattern or request velocity exceeds defined behavioral bounds, identity proxies can automatically revoke ephemeral tokens and safely isolate the workload in real time.
5. In-the-loop runtime enforcement & human approvals
Security governance cannot rely solely on static pre-authorization; policies must be evaluated continuously at runtime before individual actions execute.
Action-level policy interception: Enforce real-time policy checks at the agent harness layer—evaluating shell commands, database queries, file operations, and outbound API calls against governance rules before execution.
Configurable approval workflows: Establish flexible escalation paths that permit low-risk read operations automatically while requiring explicit human-in-the-loop validation for high-impact actions, such as code deployments or financial transactions.
6. Web-scale identity architecture built for machine workloads
Autonomous workflows generate significant operational volume. Identity systems must be architected to handle machine-scale throughput without performance degradation or store bloat.
Machine-speed throughput: Multi-step workflows and parallel worker agents demand identity control planes that can handle high-volume token validation and policy evaluation with minimal latency.
Lifecycle governance for sub-agents: Dynamically spawned sub-agents require rapid provisioning and immediate teardown upon task completion, thereby preventing the accumulation of orphaned credentials and ensuring clean session termination.
Inline cryptographic safeguards: Prioritizing inline policy enforcement over post-mortem log reviews allows organizations to intercept unauthorized state changes before they occur, maintaining operational integrity across multi-cloud environments.
Conclusion: securing the future of enterprise automation
As AI models evolve from passive assistance tools to active operational participants, identity becomes the primary boundary for enterprise governance. By bridging the machine identity gap with verifiable agent identities, short-lived JIT credentials, fine-grained relationship authorization, and automated runtime enforcement, security leaders can confidently deploy autonomous AI agents to drive productivity while maintaining complete operational control.
A frontier language model is only one component of an AI agent. The surrounding agent system—often called a harness—determines how the model receives...
A frontier language model is only one component of an AI agent. The surrounding agent system—often called a harness—determines how the model receives context, uses tools, maintains state, responds to feedback, recovers from failure, and sustains progress over long-running tasks. The challenge is how to build the agent architecture that makes frontier language models work reliably on extended…
As AI agents become more capable and operate over longer horizons, building security and trust into the applications they power becomes increasingly important....
As AI agents become more capable and operate over longer horizons, building security and trust into the applications they power becomes increasingly important. Drawing on work with NVIDIA OpenShell, agent developers, open-source projects, and partners across the ecosystem, AI safety and security teams at NVIDIA offer their perspective on the emerging agent stack—including the role of each layer…
Earlier this week, researchers outlined an attack that used a secret input provided by Microsoft 365 Copilot for enterprise to cause the AI assistant to exfiltrate a password present in the user’s inbox. Now, a separate team has devised a similar attack against Grok. The new data theft hack employs a deceptively simple trick to force the Elon Musk-owned large language model to steal user chats and other personal information. At the time this post went live, the assistant continued to cough up the data, despite xAI being informed of it in June.
The lesson from both this week’s episodes—and the countless other ones that have come before it—is that LLMs are incapable of solving the root causes for prompt injections, the most severe vulnerability classes they’re most prone to. That leaves AI developers with no other option but to build a guardrail that steers the model away from the harmful actions. As I noted in Tuesday’s story, the approach is tantamount to a road traffic safety engineer erecting a protective rail around a dangerous bend rather than banking the curve.
Cryptographic Context Injection in the house
Prompt injections exploit LLMs' training to comply with user requests whenever possible. Attackers can capitalize on the predilection by smuggling harmful instructions into emails or webpages the assistant is instructed to summarize. Because LLMs can’t reliably distinguish between content in an email sent by an untrusted party and user instructions entered directly into a prompt, the overly solicitous LLM faithfully follows them. To date, Grok and other LLMs' only recourse is to create guardrails that flag suspicious instructions and forbid them from being executed.
Short answer: The strongest general-purpose MDR services are well documented. Forrester’s Q1 2025 Wave evaluated the ten most significant vendors across 21 criteria and named CrowdStrike, Expel and Red Canary as Leaders, with eSentire and Binary Defense as Strong Performers. None of that evaluation tells you which service can monitor a programmable logic controller without […]
What are the best AI avatar generators? The honest answer for an enterprise buyer is that the best platform is rarely the one with the most convincing face. It is the one that can pass a security review, produce a defensible record of consent for every likeness it generates, and tell you who inside your […]
Flock’s surveillance cameras have already sparked outrage. WIRED reconstructed its next-generation AI system, already in use by some police, to confirm it goes much further than tracking license plates.
The ChatGPT maker says its upcoming Astra model may have reached “critical” cyber capabilities, prompting it to halt a significant number of training runs while it tightens internal safeguards.
It’s not every day that attackers can force a frontier AI model to cough up user passwords and other sensitive data without user confirmation. That’s exactly what researchers recently did to Microsoft 365 Copilot for enterprise. Even more unusual is the source they tapped to discover the critical vulnerability that made their exploit possible. Rather than employing reverse engineering or other traditional vulnerability-hunting methods, they asked Copilot. The LLM assistant readily complied.
Researchers at security firm Varonis knew they wanted to create an exploit that would exfiltrate user data when a user did nothing more than click on a link. Like most AI assistants today, Copilot steadfastly refused and made clear that sensitive prompts like that require explicit user consent in the form of a gesture, such as pressing a return key or other key. In response, the researchers peppered Copilot with questions about the guardrails that required user confirmation before the assistant could execute powerful commands.
Loose lips sink ships
The dialog was like a game of 20 questions. Each answer provided a new clue that divulged information about the complex safety mechanism. Why was auto-execution impossible, they asked. What URL structures and deep links were involved? What happens when a page is loaded with input already in the prompt field? Each answer provided a deeper view into the guardrail and its limits. Eventually, Copilot provided a stunning Microsoft trade secret—an undocumented prompt parameter that completely bypassed the requirement for user consent.
As robots become more intelligent, autonomous and connected, much of the attention naturally falls on increasingly powerful AI models. But underneath those models is an increasingly complex collection of sensors, processors, control systems and security hardware that must operate reliably – often within strict limits on power consumption, size and heat. Lattice Semiconductor is one […]
Login
