The race to make AI inference cheaper is pushing chip design beyond general-purpose accelerators. We’re now moving toward silicon that is tailored to individual models. And Google’s reported “Frozen v2” project suggests the Gemini AI model is part of the movement.

First reported by The Information, the unannounced chip would reportedly hardwire parts of Gemini’s architecture while leaving its weights updatable. That compromise could give Google much of the efficiency of model-specific silicon without making the hardware obsolete every time Gemini changes.

A spokesperson for Google tells The New Stack, “Our teams are constantly researching and experimenting with new innovations to deliver maximum performance and efficiency for our users and customers.”

A spokesperson for Google tells The New Stack, “Our teams are constantly researching and experimenting with new innovations to deliver maximum performance and efficiency for our users and customers. While not every project moves into production, this rigorous exploration is central to our full stack approach. By co-designing our hardware and software from the ground up, we ensure our systems are integrated and highly optimized for real-world workloads.”

According to the reporting, Google hopes the chip will help relieve the AI compute crunch that’s made it harder for cloud providers to keep up with demand, while also making Gemini much cheaper and more efficient to serve. Internal projections reportedly estimate the design could deliver six to ten times more tokens per watt than Google’s current generation of AI chips.

If the project moves forward, it would represent a different approach to AI infrastructure because Google would be building hardware specifically for Gemini. For developers, that’s an early indication that future AI systems may be designed with much tighter integration between the model and the hardware beneath it. 

Specialized silicon replaces flexibility

Currently, the vast majority of AI inferences run on Nvidia GPUs or Google’s own Tensor Processing Units (TPUs). Because these chips are intended to accommodate a wide variety of AI models, they carry a high degree of processing load.

There’s precedent for this kind of shift. Bitcoin mining followed a similar path, starting with CPUs, then GPUs, before ultimately settling on ASICs. AI inference may be headed in the same direction. Training still benefits from the flexibility of GPUs, but once a model reaches production, the priority shifts to serving as many requests as possible while using less power. 

Training still benefits from the flexibility of GPUs, but once a model reaches production, the priority shifts to serving as many requests as possible while using less power.

Competitors hardwire their own

Google isn’t the only company looking beyond general-purpose GPUs for AI inference. As inference becomes a larger share of AI workloads, more companies are experimenting with specialized hardware designed to improve performance while using less power. Even Nvidia, whose GPUs dominate the AI market, has invested heavily in inference, striking a reported $20 billion deal with Groq last year to license the startup’s technology.

Among the more ambitious efforts is Canadian startup Taalas. The company has demonstrated a chip with an entire 8-billion-parameter Llama model embedded directly into the silicon. By keeping the model on the chip instead of constantly moving data back and forth from external memory, Taalas says it can dramatically speed up inference, claiming throughput of roughly 17,000 tokens per second. Meanwhile, other companies are optimizing the spatial distance between memory and compute to avoid hardware lock-in.

d-Matrix’s new Corsair platform uses an SRAM-based in-memory compute architecture rather than relying on standard high-bandwidth memory (HBM) packaging. Similarly, SambaNova is deploying custom dataflow technology with a three-tier memory architecture to maximize tokens per watt for advanced workflows.

Google’s Frozen v2 sits uniquely in this landscape — borrowing the hyper-efficiency of hardwired architectures like Taalas, but retaining just enough flexibility to remain viable across multiple product cycles.

Freezing architecture, not weights

The “Frozen” name reportedly comes from the idea of permanently etching part of Gemini’s design into the chip itself. According to The Information, Google’s engineers have spent years trying to find the right balance between efficiency and flexibility.

An earlier concept, led by Google DeepMind Chief Scientist Jeff Dean, reportedly would have embedded Gemini’s model weights directly into the silicon. That idea was eventually abandoned because it would have tied the hardware to a single version of the model, dramatically limiting its useful life as Gemini continued to evolve.

Frozen v2 reportedly takes a different approach. Rather than locking in the weights, the chip would hardwire parts of Gemini’s underlying design while still allowing the weights to be updated over time. That would let Google continue improving the model without needing to replace the hardware every time Gemini receives a new version.

Rather than locking in the weights, the chip would hardwire parts of Gemini’s underlying design while still allowing the weights to be updated over time.

Cheaper inference reaches developers

By moving some of Gemini’s execution into the chip itself, Google could reduce some of the overhead that comes with running models on more general-purpose hardware. That could translate into lower latency, especially for applications that depend on near real-time responses. 

Google’s reported goal of delivering six to ten times more tokens per watt is ultimately about efficiency, and those savings could eventually make their way to enterprise teams building on Gemini, through lower API costs or more available capacity. While Google hasn’t said how it would pass those gains along, reducing the cost of inference has become a priority across the industry.

The post Google just bet its inference future on a chip built for one model appeared first on The New Stack.

Rapidly falling launch costs are making space more accessible than ever. But new research suggests the economics are improving even faster than most people realize, potentially opening the door to entirely new industries beyond Earth.

For most of the space age, the cost of getting material into space was so vast that only the most well-heeled governments and corporations could participate. In 1960, getting a kilogram of payload into orbit would have cost you more than $87,000 (in 2024 US dollars).

But according to researchers at the University of Cambridge, that figure had collapsed 96 percent to $3,868 by 2025. The team’s modeling suggests this trend will continue apace for at least the next few decades, with prices forecast to hit just $1,569 by 2030 and as little as $273 by 2040.

The rapid decline in prices is thanks to a well-established economic principle known as Wright’s Law, which holds that technologies get predictably cheaper as cumulative production grows. The Cambridge team says the trends seen in launch costs could soon make a host of possibilities previously confined to science fiction commercially viable, including orbital solar power, asteroid mining, and space-based manufacturing.

“Space is no longer a science-fiction fantasy or a purely scientific pursuit, it is becoming a marketplace,” Alessio Terzi, who led the study, said in a press release. “Rapidly falling launch costs could open the way to space colonization and commercial activity far beyond low Earth orbit.”

To conduct their study, published in PNAS Nexus,the researchers assembled a massive dataset of rocket launches covering over 4,400 flights by more than 330 different rocket designs from 1960 to 2025. For each launch, they estimated the “unit flyaway cost,” or the total cost to manufacture, maintain, and launch the vehicles, excluding research and development investments.

They then checked how this data stacked up against Wright’s Law, which predicts that every time production volumes double the cost should fall by a fixed percentage. This is known as a technology’s “learning curve” as the reduction in costs is attributed to an industry getting better at producing the technology with experience.

The researchers found space launches obey the law almost perfectly, with every doubling of payload sent to orbit shaving 21.2 percent off the average cost per kilogram. More importantly, this represents a particularly steep learning curve compared to previous technologies.

Solar panels are often held up as the poster boy for learning curves, with prices falling 99.8 percent between 1975 and 2023. But while solar power’s total price reduction is higher than that achieved by launch vehicles, the technology got there by scaling deployment far more. When accounting for total production, solar’s learning curve lags launch costs at 20.2 percent.

The researchers also compared launch costs to another revolution in transport. Steamships transformed our ability to ship goods like wheat and cotton around the world in the 19th century. They found that steamship costs only fell 15.5 percent with each doubling of cargo.

“The cost of space launch technology is now falling faster than during one of history’s greatest transport revolutions,” said Terzi. “Steamships cut costs through explosive growth in global trade. Space technology, by contrast, has achieved even steeper declines at a far smaller scale. This suggests there is plenty of scope for further cost reductions and the industry may now be on the cusp of a comparable economic boom.”

There are, of course, caveats. The researchers note that the industry’s progress is inextricably tied to the fate of a single company. SpaceX already accounts for roughly 80 percent of payload reaching orbit. If the company successfully scales up its reusable, heavy-lift Starship vehicle it could massively reduce costs.

But a company with a stranglehold on the global launch market may be tempted to take advantage of its monopolistic position. This may also push foreign governments and companies away from relying on SpaceX even if it’s the cheapest option.

There’s also the danger that as costs fall and launching material into space becomes more accessible, low Earth orbit could quickly become clogged with debris that makes it increasingly difficult to reach orbit safely.

If these challenges can be sidestepped, the implications of such rapidly falling costs could be profound. The researchers suggest that everything from zero-gravity research and orbital tourism to factories churning out fiber-optic cables and 3D-bioprinted organs could become financially viable.

The post Spaceflight Nears Its Steamship Era appeared first on SingularityHub.

Enterprise AI is facing an ROI paradox. While throwing more compute at the strongest foundation model works well in product experiments, the costs become unbearable when the product is deployed in production.

A new paper from researchers at Writer provides a solution that is accessible to engineering teams. The study takes a systematic look at optimizing the different components of the orchestration layer that wraps around the foundation model, aka the AI harness. 

By optimizing the harness, the researchers show dramatic reductions in tokens per task, a drop in cost-per-successful-task by up to 61%, and quality that holds steady, all without changing the underlying foundation model.

Because the harness is fully under the developer's control and requires no model fine-tuning, engineering teams can apply these findings to build highly cost-efficient AI applications.

The ROI crisis of tokenmaxxing

The current state of AI engineering is plagued by "tokenmaxxing," an industry trend where developers rely on massive context windows and brute-force token consumption as a substitute for good system design. 

Rather than engineering elegant workflows, developers have imported a reflex from traditional software development: generate, run, fail, stuff the error and more context back into the window, and retry. 

"Teams tokenmaxx because it's the cheapest fix in the moment, and because it's literally how most engineers work today," Waseem AlShikh, CTO and co-founder of Writer, told VentureBeat. Because this approach succeeds often enough on coding tasks, it has become the default reflex for every other agentic workload. The danger is that per-token price drops mask the underlying inefficiency. 

"Your invoice is tokens-per-task times price-per-token, and most teams only watch the second number," AlShikh said. "In agentic workloads, tokens-per-task compounds — every loop iteration re-transmits the growing context — and it compounds faster than prices fall. The price cut becomes an anesthetic. It masks the fact that the loop itself is bleeding."

Tokenmaxxing leads to several enterprise failure modes. Teams route simple tasks to premium frontier models by default. They use the LLM as a lazy search index, stuffing the context window with raw documents instead of retrieving exact answers. Most destructively, they build unconstrained agentic loops that spiral out of control when the model encounters an error. Because output tokens cost significantly more than input tokens across all major model providers, inefficient task execution acts as a silent budget killer.

The industry has introduced several efficiency techniques to curb these costs, but they largely fall short because they treat the model in isolation: 

• Prompt compression condenses input text to save space, but ignores how the system sequences those inputs across complex workflows. 

• Budgeted reasoning caps the computational steps a model can take, which often degrades output quality if the workflow isn't intelligently routed. 

• Terse coding forces models to output minimal code to save output tokens, but does nothing to solve inefficient tool calling. 

• Speculative decoding uses a smaller draft model to speed up a larger model's text generation, optimizing inference speed while failing to address bloated agent architectures.

These efforts fail because they optimize the engine while ignoring the transmission. They do not look at the orchestration layer, leaving underlying architectural inefficiencies unresolved.

Unpacking the harness: the levers of efficiency

The harness is the orchestration layer that routes, formats, and turns the underlying LLM into a working system.

The core levers of harness optimization include system prompt caching, interaction history compaction, tool management, retrieval strategies, and error management. These are the most accessible intervention points for engineering teams looking to improve AI performance. 

As the Writer researchers note in the study: “If the harness is the layer that composes model calls into work, it is also the layer that sets the price of work.”

Historically, developers have treated the harness as disposable glue code designed simply to connect an API to a user interface. The study signals that the harness must now be treated as a first-class object: a primary software artifact that requires its own testing, versioning, and rigorous design. 

For enterprises, this reframes the "own-versus-rent" decision. 

"Enterprises spend months on model evaluations and then rent their orchestration off the shelf — which means they're optimizing the smaller lever and outsourcing the bigger one," AlShikh said. "Whoever owns the harness owns your unit economics, and an open framework tuned for demos is not tuned for your invoice." 

Inside the experiments

To isolate the impact of the orchestration layer, the researchers ran experiments on six foundation models spanning multiple vendors and weight classes: Claude Sonnet 4.6, Gemini 3.1, Gemini Flash 3.5, Qwen 3.6, GLM 5.1, and Writer’s own model, Palmyra X6. 

Their experiments compared a frozen, conventional production agent loop against the finished Writer Agent Harness on the same 22 locked enterprise tasks, spanning capabilities like grounding and retrieval, multi-step workflows, tool use, and content generation. By holding the models and tasks constant, they could isolate the effects of the orchestration layer itself.

The optimized harness drove a significant drop in costs, cutting the blended cost per task by 41%, from 21 cents to 12 cents. This was largely achieved by slashing token consumption, with the number of tokens per task falling 38%, from 14.2k to 8.8k.

The harness is designed to delegate tasks like search to specialized sub-agents. A sub-agent receives only the tool and the specific query it needs, retrieves the exact data, and returns a capped, clean summary to the main agent — keeping the primary context window from filling up with raw search results.

Task success rates held steady even as token use fell — moving from 78% to 81%, a gain the researchers describe as directional rather than statistically significant at their sample size, meaning quality didn't suffer even as costs dropped.

End-to-end task latency also dropped significantly, reducing the median wall-clock time by 44%, from 48 seconds to 27 seconds, due to prompt caching and the elimination of dead-end reasoning loops.

However, the researchers also found limits to multi-agent orchestration. Smaller models like Gemini Flash 3.5 and Qwen 3.6 scored well below a usable reliability threshold on sub-agent delegation tasks (0.45 and 0.42, respectively) — the capability simply isn't dependable yet on lighter-weight models.

Sub-agent orchestration only crossed a usable reliability threshold on the two strongest models tested: Writer's own Palmyra X6 (0.86) and Claude Sonnet 4.6 (0.85).

The developer’s playbook: actionable takeaways and tradeoffs

The findings from the study translate into a playbook for enterprise developers building agentic workflows at scale. The first step is to implement what AlShikh calls the "Two-Zone Prompt" and "Context Offloading."

Structure for system prompt caching (The Two-Zone Prompt): Modern LLM APIs offer prompt caching, but developers must structure their payloads correctly to trigger it. Developers must separate the "stable zone" from the "volatile zone." Place static, unchanging elements (e.g., core rules, large tool schemas, and standard operating procedures) at the top of the prompt. Dynamic elements, such as the specific user query or recent conversational task state, must be appended at the bottom. This ordering allows the harness to reuse the cached prefix across hundreds of calls. "That single separation makes prompt caching actually work and stops you from re-paying for the same instructions on every one of an agent's thirty steps," AlShikh said.

Manage context with Context Offloading: Avoid context stuffing, where every turn of a loop is appended into a monolithic prompt until the window maxes out. Instead, move history and intermediate artifacts out of the window into retrievable storage, and pull back only what the current step needs. If possible, delegate tasks to single-purpose sub-agents to avoid context bloat. As AlShikh points out, "the biggest line item in agent spend isn't reasoning — it's re-sending things the model has already seen."

Build resilient loops and redefine KPIs: Unmanaged agent loops drain API budgets rapidly. Teams must begin tracking Completions Per Million tokens (CPM) to understand their true task costs, but the harness itself must contain physical guardrails. "The core principle is that you never ask the model to police its own spending," AlShikh said. "The fence has to live below the model, in code, on your side of the API." This requires three hard checks:

• Hard per-task token budgets: The run terminates when the budget is spent, no exceptions.

• Generation fencing: Caps on steps, tool calls, and recursion depth to stop non-converging agents. 

• Failure-spend governance: Cap what a run can spend after its first failed validation so a failing task doesn't become your most expensive task.

Avoid unnecessary complexity: Optimizing the orchestration layer comes with engineering overhead. If you're in the prototyping and exploration stage, that overhead isn't justified — iterate fast with a strong model and a light harness. Once you're scaling to millions of requests a day, the savings from harness optimization become substantial.

However, teams must be aware of "harness leverage." Adding structural scaffolding requires the model to hold and obey that context. If a model is too small, it will spend its limited capacity parsing the scaffolding instead of doing the task, causing accuracy to drop and tokens to rise. The rule for adding complex orchestration features is strictly mathematical: "If a feature adds more coordination tokens than it removes task tokens for that specific model, cut it," AlShikh said. "Nothing in the harness is free."

The future of the enterprise harness

The era of tokenmaxxing and treating context windows like bottomless buckets is coming to an end. Throwing more compute at poorly designed systems is not a viable strategy for companies that need to demonstrate a return on their AI investments. 

As foundation models evolve to absorb planning, tool selection, and multi-step reasoning natively into their weights, the role of the harness will shift from compensating for model weakness to enforcing enterprise policy.

"What never moves into the model is the 'allowed': budgets, permissions, data boundaries, audit trails, deterministic kill-switches," AlShikh said. "Five years from now, the harness will be thinner but more important. There will be less scaffolding and more governance. However capable the model gets, someone external to it still has to define what it may spend, see, and touch. That layer belongs to the enterprise, and it should never be rented."

Native access to data lakes and a new data retrieval engine highlight the latest release as the vendor competes for market share amid a crowded field.

A single AI agent conversation can look flawless scored on its own and still point to a broken product. That gap is driving a shift in how enterprises evaluate agents, away from scoring individual traces and toward comparing cohorts of users against a baseline.

At VB Transform 2026, Harrison Chase, CEO of LangChain; Hui Zhang, CTO and co-founder of Conviva; and Emmanuel Turlay, director of engineering at CoreWeave, described that shift, along with a parallel move toward cheaper, narrower judge models.

Agent-as-judge — judging one AI agent's output with another — hasn't replaced LLM-as-judge, which Chase said remains the default. The larger tension, Zhang said, is between automated judging, whether by LLM or agent, and human review.

"You have scalable but ungrounded, whether it's agents as judge or LLMs as judge, you grade the outcome, you grade the work. It still is very difficult to ground it and then you use humans and that's just not scalable," Zhang said. "The whole industry is facing this, which poison you want to pick."

Evaluation criteria now function as the product spec

That gap — a conversation that scores well but still signals a broken product — is what teams try to close by building an exhaustive evaluation suite before they ship anything. Chase said that doesn't work.

"We sometimes see teams that have almost eval paralysis," Chase said. "They're like, this is an eval set, I can't launch it. The best teams launch and then iterate."

Chase framed evaluation criteria as a living specification, not a one-time test suite: a product requirements document — the standard software-development spec for what an application should do. "Evals are like the new PRD," he said. "They define what your agent should and shouldn't do."

Turlay described hitting the same failure from a different angle. "I was trying to reach 100% coverage for my tests, and I still had bugs in production," he said — a test suite that looked complete but still missed what mattered, the same gap Chase was describing with evals.

Broad, always-on monitoring, he said, catches more real failures than an exhaustive pre-launch test suite. Teams should set up wide online checks first, use those to identify failure classes as they occur, then build a targeted offline evaluation set around the problems that surface.

Why scoring traces one at a time is a mistake

Even a well-built evaluation process can still score the wrong thing. Zhang's objection is to how most teams run evaluation: sampling traces, whether 50 of them or a full population, scoring each in isolation. That approach misses a signal that only shows up when comparing cohorts of users against a baseline, a method Zhang calls contrastive analysis.

Zhang illustrated it with a retail example: a shopper asks an agent for a running shoe ahead of a half marathon, the agent asks qualifying questions, and the shopper buys a shoe. Scored individually, that interaction looks fine. But the clarification ratio, how many follow-up questions an agent asks before completing a task, came in three times higher than baseline for that shoe category across the full user population. A second metric, how often shoppers finished their purchase outside the conversation, was five times higher than baseline for the same category.

Neither number is visible from a single trace. Both point to a debuggable, category-specific problem. Zhang said the industry also lacks a second data source: what happens before, between and after the conversation, not just the trace itself.

Sizing the judge to the job

Once contrastive analysis flags which category is actually broken, the next problem is what watches for it going forward — and at what cost. Turlay's rule was to start with the most capable model available to prove a task is solvable, then work down. If it can't be done with a top-tier model, he said, it won't work with a smaller one. Once a pattern proves viable, teams can sample a fraction of traffic instead of judging every interaction, and move simpler tasks like binary classification to smaller open source models.

LangChain took that further, fine-tuning its own model to detect when a user believes the agent made a mistake, a signal Chase calls perceived error. "The model we fine-tuned was a Qwen model," he said, referring to Alibaba's open source family. Combining hand labeling with distillation, the result performed well. "Same as [Claude]Sonnet, for, depending on how we served it, either 10 to 100x cost reduction," Chase said.

Not every guardrail needs a model. Chase pointed to Claude Code's own guardrails as proof: regexes, the common programming technique for finding and validating patterns in code. "A lot of the guardrails they had were just regexes," he said. "They weren't small LLMs, they were just regexes."

LLM-as-judge doesn't mean human-in-the-loop disappears

The bigger question is whether using LLM as a judge removes the need for a human in the loop.

Turlay pointed to accountability, drawing on his prior work at a self-driving car company. His team compressed data intake and retraining into a two-week cycle for shipping a new model to the car. Even then, someone still had to sign off.

"I felt confident on behalf of the company to say this model should go into the car," he said. The same logic extends to legal, finance and healthcare. "Before we can remove a human to say, I endorse this and I take responsibility legally for it, it's going to be a while before agents can do that on their own."

Zhang agreed a human has to remain the guardian on corner cases, even as automation eventually runs at a scale that beats individual human accuracy — machines can see more at the pattern level.

Chase went further: that human check isn't just a safety net. "Human in the loop is really important for building trust in how these agentic systems work, and also really important for memory and learning from systems," he said. "There has to be interactions in order for the system to learn."

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

Over the weekend, several current and former advisors to President Donald Trump on AI publicly lobbed insults at the country’s leading AI companies. David Sacks, the president’s AI and crypto “czar” until March, branded Anthropic’s models as “lobotomized” and “woke.” Emil Michael, a top Pentagon official, called OpenAI’s new head of strategic futures a “supreme village idiot.”

It began because no one can agree on what to do about Kimi, a free, open source model that Chinese AI company Moonshot launched last week. It appears to rival the intelligence of models from OpenAI and Anthropic, which are very much not free. 

Kimi and other Chinese models like it pose a real problem for Trump. And they’re dividing the top AI strategists in his orbit into factions. Every time a new smart, free model from China like Kimi gets released, US companies see less reason to fork out money to access models from Anthropic or OpenAI. Given that enthusiasm for these and other AI companies is driving an outsized share of economic growth, China’s AI models create both economic and political problems for the president. They are “a threat for an administration that really doesn’t want more economic bad news,” Anton Leicht, a fellow at the Carnegie Endowment, wrote on X. They’ve already rattled US stocks. 

What is Trump to do? First, consider that this is all happening just a week after New York imposed the country’s first state ban on new data centers. There is growing distrust of AI companies, and I imagine a not-insignificant share of Americans would have little sympathy for OpenAI or Anthropic as they fend off cheaper competitors, and would say it’s not the government’s job to protect their interests.

On this point, they’d see a sliver of agreement (and really just a sliver) with David Sacks, who on July 19 criticized top AI companies that “want the government to eliminate their open source competition.” He has also argued that Chinese AI models have become popular because they come with fewer restrictions on how people can use them (putting aside the built-in state censorship). 

Sacks, however, is out of a job. He no longer has a formal role advising Trump, and his position that more open AI is better has been largely replaced in the administration by one that sees a larger role for government intervention. The thinking behind this view is that because AI models have gotten strong enough to pose threats to national security, the government must control how they’re used. 

This position has fueled the new White House review process that aims to vet AI models’ security before they’re released. Dean Ball, a former Trump AI advisor who now works for OpenAI, criticized it over the weekend as a “de facto licensing regime for frontier AI.” Ball predicted Trump may solve his Chinese open source problem with a bit of soft power, perhaps by making US companies afraid to use models like Kimi. That drew a response from Michael, who, with Secretary of Defense Pete Hegseth, has been the agency’s main liaison with AI companies. Michael called Ball the AI industry’s “supreme village idiot,” bristling at the suggestion that the government would quietly strong-arm companies rather than, as Michael put it, go through “the democratic process not some Deep State scheme.”

Left out of the conversation has been how a model like Kimi got so good in the first place. For much of the Biden administration and even the beginning of Trump’s second administration, keeping China from getting top chips was a priority. Those export controls have loosened—Trump made the controversial decision to allow Nvidia to sell more chips to China, in exchange for the US government taking a cut—and the government has alleged that some chip smuggling has taken place. But China nonetheless has limited computing power, and it’s not clear what chips the company behind Kimi used to train the model. 

It’s possible that the process involved some distillation, a practice in which AI models are trained on the outputs of existing AI models. OpenAI and Anthropic have long complained that Chinese AI companies do this, and they have requested government help to put a stop to it. In April, they got it, when the Trump administration announced a series of efforts to curb the practice.  

But Kimi is out there and free, and it is nearly as good as the Anthropic model the US government deemed so powerful that it was briefly shut down because it threatened national security. The weekend’s sparring suggests many in Trump’s orbit see that as a wake-up call. But nobody can agree on what for.

Over the past nine months, Amazon, Microsoft, and Google have each introduced or renamed an enterprise agent platform. And all three have converged on the same core architecture. Runtime, memory, tool gateway, identity, observability, and governance now appear in Amazon Bedrock AgentCore, Microsoft Foundry, and the Gemini Enterprise Agent Platform, albeit under slightly different names. Each vendor presents these components as the foundation for production agents. What was a fragmented collection of libraries just 18 months ago is becoming a distinct platform layer.

To see where that shift may lead, consider the evolution of platform as a service. Between 2011 and 2016, developers built applications from virtual machines, load balancers, message queues, secret stores, and monitoring agents — each with its own API and operational model. Cloud Foundry and Heroku unified those pieces around an application contract. Developers could think less about machines and more about applications. The agent ecosystem is approaching a similar inflection point, but it still lacks an equivalent contract — and no open-source project has stepped forward to define one.

The plumbing a production agent still needs

Imagine a platform team putting a customer support agent into production this quarter. They choose a model provider, then a framework, and finally a place to store session state and long-term memory. They add a tool gateway so the agent can reach the ticketing system. An identity layer follows, so the agent acts on behalf of the person who asked, and a sandbox keeps generated code somewhere safe. Evaluation and tracing are considered last, once someone asks how quality regressions will surface before customers find them.

Each of those choices looks small on its own. When combined, they decide which cloud the workload lives in. Session state resides in a single provider’s managed store. The traces are logged in that same provider’s telemetry service, and the agent’s identity is derived from its directory. Moving that agent a year later means rebuilding the whole assembly, which is where enterprises were before PaaS gave applications a portable shape.

What Cloud Foundry got right before it lost the market

Cloud Foundry reduced application deployment to a single command, and the platform took responsibility for everything after it. Buildpacks detected the language and produced a runnable artifact. Service brokers provisioned a database or a message broker and bound the credentials into the application environment. Routing, logging, autoscaling, and health checks arrived as platform behavior, not as a queue of tickets aimed at an operations team.

What mattered was the contract, not the implementation. An application declared what it needed and stayed agnostic of where it ran. Buildpacks began life at Heroku back in 2011. Pivotal and Heroku started the Cloud Native Buildpacks project in January 2018, and the CNCF accepted it that October. A PaaS idea outlived the platform that produced it.

Cloud Foundry never became the dominant platform. Kubernetes did, and the Cloud Foundry community eventually rebuilt its abstraction on top of Kubernetes through Korifi. The design principles traveled anyway, and enterprises running that platform in 2016 had portability that most agent teams cannot buy today.

The same primitives across three clouds

Let me dissect the three platforms because the similarities are conveniently hidden behind the branding.

AgentCore reached general availability in October 2025 with seven composable services, namely runtime, gateway, memory, browser, code interpreter, identity, and observability. The runtime offers eight-hour execution windows with complete session isolation. The gateway connects to existing MCP servers and turns APIs and Lambda functions into agent-compatible tools, while observability exports through OpenTelemetry into CloudWatch.

Microsoft renamed Azure AI Foundry to Microsoft Foundry effective January 1, 2026. Foundry Agent Service covers the same ground. Microsoft documents hosted agents running in a session-isolated managed runtime, with Entra Agent ID handling identity. Managed memory spans session, user, and procedural scopes, and tracing is built on OpenTelemetry.

Google retired the Vertex AI name at Cloud Next 2026 and folded the platform into Gemini Enterprise Agent Platform. What was Agent Engine became Deployments, and Memory Bank, Sessions, Agent Registry, Policies, and Gateways sit alongside it under an agent-first information architecture.

The convergence is rational behavior rather than a conspiracy, since infrastructure companies build vertically integrated platforms because integration is where the margin lives. The consequence lands on customers rather than on the vendors. Identity, telemetry, and deployment all terminate within a single provider, which makes the operational layer beneath the agent the part that resists moving. Recently, I analyzed the runtime aspect of this problem in a piece about Google’s Agent Substrate.

The contract an agent platform would inherit

Any agent platform can be tested by one question: what would a Cloud Foundry-style contract look like if it had been written for agents rather than for web applications? The mapping is close enough to be instructive and different enough to be interesting.

PaaS abstraction	Agent platform equivalent	Where portability breaks today
Application source	Agent code, instructions and evaluation suite	Each framework defines its own package shape
Buildpack	Framework detection and agent packaging	No shared build contract across SDKs
Backing service	Model, memory, retrieval or tool provider	Providers are wired into agent logic
Service binding	Authenticated attachment of tools and data	Credentials are issued by the host cloud
Router	Agent endpoint, MCP or A2A interface	The protocols exist but not the lifecycle
Logs and metrics	Traces, tool calls, cost and quality scores	GenAI conventions remain in development
Release promotion	Evaluate, version and progressively deploy	Evaluation couples to one vendor’s harness
Platform policy	Agent identity, permissions and approvals	Identity ties to the provider’s directory

Real deployments will mix these rows rather than adopt them cleanly, and no team needs the complete set from day one.

Package the agent as one deployable unit

The thing developers ship has to be versionable, testable, and movable as a single artifact. The code, the instructions, the tool dependencies, the memory contract, the permissions, and the evaluation suite all travel together, or none of them do. AWS gets close with its harness export path. One command turns a configured harness into Strands code, and AWS says the export preserves the model, prompt, tools, memory wiring, and container environment. Customers can easily move from configuration to code without an architecture rewrite, which is the right instinct pointed at a single cloud.

Bind capabilities rather than embed providers

The Twelve-Factor App taught developers to treat databases and caches as attached resources reached through configuration. Models, memory stores, retrieval engines, browsers and tool gateways deserve the same treatment. An agent that names its model provider inside application logic has already given up portability, whatever the platform brochure claims.

Make operations part of the abstraction

Rather than just enabling deployment, PaaS proved its value by offering built-in capabilities like routing, logging, autoscaling, and rollback. The questions that matter for agents look different, and they should be the ones enterprises must ask. Platform teams want to know whether the agent completed the task and whether it chose the right tools. They also want to know whether it exceeded its authority, what the run cost was, and whether quality regressed after a model update.

How agents differ from applications

Agents are not web applications with a model attached, and a platform built on that assumption will fail in production. Three differences carry most of the weight here. Agent behavior is probabilistic, so two identical inputs can produce different tool calls. Agents act with delegated user authority, which turns a permissions bug into a real-world side effect rather than an error page. An agent’s dependencies can also change its behavior without any code deployment. A model update or a revised tool description alters what the agent decides to do.

The Twelve-Factor rule that processes should be stateless does not survive that. An agent platform needs disposable execution workers alongside durable, inspectable, and portable agent state. LangGraph already demonstrates the combination in open source, with checkpointing at every step, first-class human interrupts, and execution that resumes after a crash. The control plane around it is part of the commercial LangSmith product, covering deployment, evaluation, and observability. The fragmentation appears within a single project.

What the open protocols leave out

Most of the primitives a neutral platform would need already exist. The Model Context Protocol standardizes how agents access tools and data. A2A covers discovery and communication between independent agents. OpenTelemetry is defining GenAI conventions for agent spans, tool calls, and token usage, though most of those attributes remain marked as in development. OCI images stay available as the packaging escape hatch for anything a managed runtime cannot host.

The vendors have already conceded that a neutral layer matters. The Linux Foundation announced the Agentic AI Foundation in December 2025. Its founding projects were MCP from Anthropic, goose from Block, and AGENTS.md from OpenAI, and AWS, Google, and Microsoft joined as platinum members. Google also moved A2A into the Linux Foundation.

Protocols are not the same thing as a lifecycle platform. A foundation that governs how agents talk to tools says nothing about versioning an agent. It says nothing about promoting that agent through environments, or rolling it back when an evaluation regresses. Enterprises running diligence on agent platforms can work through three plain questions. The first is governance: whether the project is controlled by a neutral foundation or by the vendor selling the managed version. The second is packaging, where the same agent artifact should run on two different clouds without a rewrite. The third is state, where memory has to live somewhere the enterprise can export from. No open project answers all three today.

Where is this headed?

Kubernetes defined pods, deployments, and services, and those abstractions influenced how an entire industry thinks about running software. The equivalent agent abstractions have not been fixed yet. Whoever ends up owning the agent control plane will not merely own deployment. That owner defines what an agent is, which components it contains, and what a platform team is permitted to swap out.

In summary, hyperscalers are building robust platforms that effectively address operational questions. They also answer them inside a single cloud. If a neutral project assembles the same lifecycle atop the protocols already in place at the Linux Foundation, enterprises will regain the negotiating position that buildpacks and service brokers once gave them. An open, cloud-agnostic agent platform would benefit vendors as much as buyers, because a stable contract is what allowed the cloud-native ecosystem to grow beyond any single provider.

The post Amazon, Microsoft, and Google are converging on the same enterprise agent architecture appeared first on The New Stack.

Zillow, the real estate technology company, doesn't get one conversation with its customers. They move from a phone screen to a loan officer to a real estate agent, sometimes over months or years, and expect the context to follow them. A single chatbot could never carry that thread.

At VB Transform 2026, Zillow SVP of Engineering Toby Roberts and Glean co-founder and CEO Arvind Jain described how they built AI architecture meant to carry context across that entire journey — and why context, not raw data, turned out to be the harder problem to solve. Zillow's products touch roughly 80% of U.S. real estate transactions each year, and the company has been using AI long before ChatGPT existed.

"We pretty quickly identified that we were going to need a persistent context layer that was going to meet our customers and the professionals wherever they were," Roberts said.

Data was never the hard part

Roberts said Zillow's AI effort started where most enterprise AI efforts start, with the data itself.

"We started with a large push around making sure our data did have the right foundation," Roberts said. That meant a data mesh approach, clear data lineage and a governance structure with permissions and identity attached to the data itself.

None of that turned out to be the hard problem. The hard problem was building something that remembered where a customer was in their journey and carried that forward, no matter which surface they showed up on next.

"This context layer has to live to be able to support you where you are at any given point in your journey," Roberts said. Zillow chose to own that layer itself rather than depend on a single external chat interface, a decision Roberts said the team reached quickly once it looked at the shape of a real transaction rather than a single conversation.

Why Zillow built its own architecture, and where Glean fits into it

Zillow built its own harness rather than route customers through a single model API. The team drew on 20 years of machine learning history behind products like Zestimate, leaning into smaller, task-specific fine-tuned models instead of one general-purpose model.

Internally, that harness runs alongside Glean. Roberts said Zillow now has thousands of Glean agents in production, handling repetitive tasks with tens of thousands of executions across the company. Glean's pitch, per Jain, is centralizing that integration work once, through the Glean MCP gateway, rather than letting finance, legal and marketing each rebuild their own connections to the same systems.

That centralization is also a cost lever. Jain pointed to two mechanisms: model routing, which sends most tasks to smaller, cheaper models instead of defaulting to frontier models, and precomputed context, which avoids an agent burning tokens assembling its own context from scratch.

"Claude is also very slow because the first part of assembling that context actually takes forever," Jain said. Routing that request through Glean instead, he said, can cut token consumption by as much as half.

What Zillow and Glean's approach means for enterprises

Across data, cost and permissions, the session offered a few practical takeaways for enterprises building agentic AI on their own systems.

Build the measurement baseline before the AI push, not after. Roberts said Zillow's ability to credibly attribute a 40% increase in shipped code to AI adoption rests on a DORA metrics baseline the team put in place years earlier, not on the AI rollout itself.

Centralize context once instead of letting every team rebuild it. Jain's core argument for Glean's platform is that duplicated integration work across finance, legal and marketing teams is a hidden cost most enterprises haven't accounted for.

Don't assume permission inheritance is enough for regulated data. Even with a permissions-aware context platform in place, Zillow layered hard rules and a standing compliance check on top for its most sensitive categories, rather than trusting the architecture to handle it automatically.

Treat context as a cost lever, not just a capability. Model routing and precomputed context were the two mechanisms Jain pointed to for cutting AI spend, both aimed at reducing wasted token consumption rather than adding new capability.

"Models by themselves are not enough to bring automation with AI inside your enterprise," Jain said. "You do have to connect it with your enterprise context."

After weeks of extending temporary access while bringing additional inference capacity online, Anthropic has finalized Claude Fable 5 subscriptions.

In a Friday announcement on X, the company said Claude Fable 5 will become a permanent part of Max and Team Premium subscriptions beginning Monday, July 20, at 50% of each plan’s usage limits. Pro and Team Standard users will transition to using the model through usage credits and will receive a one-time $100 credit.

An Anthropic spokesperson tells The New Stack, “Friday’s announcement means that Fable 5 at 50% of usage limits is now a standard, permanent part of Max and Team Premium plans.”

The changes bring an end to weeks of temporary extensions that left subscribers wondering whether Fable 5 would remain part of their subscription. Rather than introducing a new usage policy, Anthropic is making permanent the same access level subscribers have had since Fable 5 was redeployed on July 1.

Capacity drove the timeline

The move gives a glimpse into the economics of serving frontier AI models. Although Anthropic repeatedly extended temporary access over the past several weeks, the company says making Fable 5 a permanent part of Max and Team Premium subscriptions was always the goal.

An Anthropic spokesperson tells The New Stack, “Friday’s announcement means that Fable 5 at 50% of usage limits is now a standard, permanent part of Max and Team Premium plans. This was always our intention with Fable access (see more in our original launch blog on June 9). We planned to expand Fable 5 access in stages as we brought additional inference capacity online, with the ultimate goal of making it a standard part of subscription plans.”

Fable 5 usage limits explained

Anthropic has not explained exactly how it calculates the 50% limit or how many prompts that will translate to in practice. Developers using Fable 5 for heavy workflows are likely to reach the limit sooner than lighter users.

The company has said demand for Claude Fable 5 was difficult to predict and that broader access depended on bringing more capacity online. Reuters reported earlier this week that the company is in talks with Meta over a potential compute lease worth up to $10 billion, which could give Anthropic access to Meta’s AI infrastructure for as long as two years. 

Meta compute deal looms

Thariq Shihipar, an Anthropic Developer Relations team member, posted on X, crediting the company’s employees with getting the necessary capacity online.

“This was due to a heroic effort by many people at Anthropic working sometimes literally around the clock,” Shihipar wrote on X. “It was not at all clear that we’d be able to do this in time, and so proud of everyone who made it happen. Enjoy Fable.”

“This was due to a heroic effort by many people at Anthropic working sometimes literally around the clock. It was not at all clear that we’d be able to do this in time, and so proud of everyone who made it happen. Enjoy Fable.”

Claude expands into India

The revised policy reflects broader changes across the AI industry. Frontier models are becoming expensive, which means AI companies are forced to balance compute costs with subscription pricing. OpenAI recently introduced GPT-5.6 Sol, while Chinese AI companies have released increasingly capable open-weight models, including Moonshot AI’s Kimi K3. Although Anthropic has not linked today’s pricing decision to competitive pressure, both model capability and the cost of serving those models have become key competitive factors.

…both model capability and the cost of serving those models have become key competitive factors.

Alongside the subscription changes, Anthropic is also formalizing its presence in one of its fastest-growing markets by introducing local pricing for Claude in India.

Indian customers can now pay for Claude in rupees instead of U.S. dollars, with GST included in the listed prices. That removes foreign currency conversions and other cross-border payment costs that previously came with subscribing. Claude Pro is now priced at Rs. 2,000 per month with annual billing or Rs. 2,399 on a month-to-month plan (previously $20 USD per month).

Industry reports have identified India as Claude’s second-largest user base after the United States. Together, the announcements show Anthropic expanding Claude in two ways: by making subscriptions easier to purchase in a major growth market while locking in an access policy determined by the compute required to serve Fable 5.

The post Anthropic employees worked “literally around the clock” to keep Fable 5 from disappearing appeared first on The New Stack.

OpenAI shares lessons from deploying long-running AI models, highlighting new safety risks, observed failures, and improved safeguards through iterative deployment.

Ray 2.55 introduces official, first-class support for Google Cloud TPUs, enabling developers to run distributed Python workloads on Google's accelerators using the familiar Ray task-and-actor APIs. To handle the strict networking requirement of keeping multi-host TPU "slices" together over their Inter-Chip Interconnect (ICI), the KubeRay Operator on GKE automatically provisions and labels the underlying hardware layout. Ray Core utilizes these labels via its slice_placement_group() primitive to atomically reserve complete slices, allowing developers to deploy jobs through KubeRay, Ray Train, or Ray Serve simply by declaring a hardware topology (like "4x4") without writing custom placement code.

Hugging Face’s incident response team first turned to frontier AI models to analyze a breach of the company’s production infrastructure, and the models refused to help. Commercial safety guardrails built to stop attackers blocked every forensic query because they treated the IR team’s real exploit data the same way they would treat a live attack.

The attacker, an autonomous AI agent running the campaign end to end, moved laterally across the Hugging Face infrastructure for a weekend, undetected and unstopped.

Security leaders are quick to recognize the pattern and diagnose what went wrong. “I’ve seen versions of this during red-team exercises and internal security testing, but this is one of the first high-profile examples where it materially affected real incident response,” said Merritt Baer, senior adviser to Andesite, G2I, and AppOmni and former Deputy CISO at AWS.

None of this is unique to Hugging Face, Baer said. “Commercial frontier models optimize for preventing misuse. They generally have no cryptographic or organizational way to determine whether the person asking ‘analyze this malware’ is an incident responder or the malware author.”

A malicious dataset opened two code-execution paths

On July 16, Hugging Face disclosed that an autonomous AI agent system had compromised its production infrastructure, gaining unauthorized access to a limited set of internal datasets and several service credentials. The company verified that its software supply chain was clean and found no evidence of tampering with public models, datasets, or Spaces.

Hugging Face is still assessing whether the intrusion touched any partner or customer data and says it will contact affected parties directly as required. But few enterprise threat models had accounted for the attacker Hugging Face faced. An autonomous agent ran the entire campaign from initial access through credential harvesting. No human guided it.

The entry point was a malicious dataset. When the data-processing pipeline ingested it, the dataset triggered code execution in two ways, through a remote-code loader and through a template-injection flaw in dataset configuration files. No admission gate screened the file before it reached a processing worker, the compute job assigned to each dataset. Initial access came through the data pipeline itself, a route that worked because most enterprise security teams treat the data feeding their pipelines as trusted input rather than an attack surface.

Worker isolation did not hold. From that first foothold, the agent broke out onto the node running the workload, then harvested cloud and cluster credentials scoped broadly enough to reach multiple internal clusters, all of it inside a single weekend. Hugging Face described the agent framework as executing thousands of individual actions through a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services. The disclosure adds that the framework appeared to be built on an agentic security-research harness, which would put tooling designed for red-team work behind a live intrusion.

Why the defenders’ queries looked like attacks

Investigators reconstructed more than 17,000 recorded events using AI-driven analysis agents of their own.

First attempts at the log analysis ran on frontier models behind commercial APIs. Defenders’ steps included submitting real attack commands, exploit payloads, and command-and-control artifacts for classification, but safety guardrails blocked the requests outright.

Baer traced the block to the prompts themselves. “The same prompts that are most valuable during an active intrusion, shell commands, exploit chains, credential dumps, persistence mechanisms, lateral movement, are exactly the prompts most likely to trigger safety systems,” she told VentureBeat. “As AI becomes embedded in security operations, this becomes an operational resilience issue rather than merely a model policy issue.”

The forensic analysis finished on GLM 5.2

GLM 5.2, an open-weight model deployed on Hugging Face’s own infrastructure, took the job the commercial APIs refused. No attacker data left the company’s environment. “This experience points to a gap worth planning for,” the company wrote in its disclosure. Hugging Face does not know which model powered the agents. It could have been a jailbroken hosted model or an open-weight model running without restrictions. Either way, the disclosure continued, “the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried.” Hugging Face drew that line itself, writing that the experience is not an argument against safety measures on hosted models and that it is sharing the feedback with the providers concerned.

What authenticated trust changes

The industry, Baer argued, needs to move past treating AI safety as a content moderation problem. “Security operations require something different. Authenticated trust.” Instead of asking whether anyone should receive an answer, the question becomes whether an authenticated security team, operating under enterprise controls, should receive it. “The model shouldn’t only understand what is being asked. It should understand who is asking, why, and under what governance.”

“Organizations already build contingency plans for cloud outages, identity provider failures, or EDR failures,” Baer wrote. “AI assistants are becoming another dependency.”

Her advice on IR playbooks was blunt. “A mature incident response plan should assume that during a severe incident, commercial AI APIs may refuse requests, API rate limits may become unavailable, internet connectivity may be impaired, and data governance rules may prohibit uploading forensic evidence externally.” The lesson, she wrote in her emailed answers, “isn’t ‘don’t use commercial models.’ It’s ‘don’t make them a single point of failure.’”

AI-enabled attacks rose 89% year-over-year

Autonomous AI-driven attacks are not limited to AI platforms. CrowdStrike’s 2026 Global Threat Report documented AI-enabled adversary operations increasing by 89% year over year, with average breakout times falling to 29 minutes. Enterprises running AI workloads in production with agentic access to their pipelines face similar exposure.

Six control domains determined the blast radius and recovery speed at Hugging Face. Each one maps to a concrete action security leaders can take before the next autonomous-agent breach arrives.

AI Pipeline Breach Response Playbook

Control Domain	What Broke	Monday Action
Dataset admission controls	Two code-execution paths were exploited. No admission gate validated the dataset before it reached a processing worker. The data pipeline became the initial access infrastructure.	Require sandbox execution and static analysis of all datasets before they reach workers. Block remote-code loaders and template-injection paths by default. Audit for any path granting code execution to untrusted content. Report to the board as a supply-chain risk.
Worker-to-node privilege boundaries	Worker isolation failed to prevent escalation to the node. The agent gained cluster credentials because the workload-infrastructure boundary was never enforced at container runtime.	Enforce hard privilege boundaries between workers and nodes. Deploy container runtime security to prevent workload escape. Audit whether workers can reach node-level APIs or credential stores. Include in the next penetration test scope.
Credential exposure	Cloud and cluster credentials harvested after node access. The scope was broad enough for lateral movement across multiple clusters over a weekend.	Rotate credentials on a scheduled cadence and after any anomaly alert. Scope to the minimum cluster and service. Deploy monitoring that flags access from unexpected nodes at machine speed. Map blast radius for board reporting.
Machine-speed detection	Thousands of actions through short-lived sandboxes with self-migrating C2. AI-assisted anomaly detection surfaced the campaign after a weekend of lateral movement, per the disclosure.	Calibrate detection for machine-speed patterns. Ensure high-severity alerts page responders in minutes, regardless of time. Audit SIEM rules for detecting thousands of short-lived executions within a single hour.
Private AI forensic capacity	Commercial APIs blocked forensic analysis. Guardrails screened query content, never analyst identity. Investigation ran on GLM 5.2 privately.	Deploy a capable open-weight model on private infrastructure before an incident. Test against real forensic workflows. Ensure IR playbook includes fallback for when commercial APIs refuse. Document gap for cyber insurance.
Autonomous-agent threat modeling	The campaign matched the forecast agentic-attacker scenario, but no threat model had operationalized it. LLM powering the agent is still unknown.	Add autonomous AI agents as a distinct adversary class with machine-speed decision cycles. Run tabletop at agent speed. Present results to the board as evidence that timelines need recalibration. Include in the cyber insurance application.

The board question is operational resilience

“The question for directors is simple. What happens if one of our critical security tools becomes unavailable during the exact moment we need it most?” Baer framed that as operational resilience, not AI policy.

She would have boards take that framing straight to management and press for specifics. “Have we actually exercised that fallback during tabletop exercises? How quickly can we switch during an incident?” Procurement needs to change alongside governance, starting with the questions buyers ask. Security teams evaluating AI vendors should ask about their process for authenticated incident responders, whether enterprise customers receive different handling during verified incidents, and whether models can be deployed privately. “Those questions belong alongside uptime, privacy, and compliance,” Baer said.

“The biggest takeaway isn’t that safety guardrails are ‘bad.’ They’re doing what they were designed to do,” she argued.

Her larger point is that the threat model itself has changed. “For decades, defenders had better tools than attackers because they operated inside trusted enterprise environments. With foundation models, both sides increasingly use the same capabilities, but one side is constrained by enterprise governance, policy, compliance, and safety controls, while the adversary simply downloads an uncensored open-weight model and keeps going. That’s a new kind of asymmetry,” she added. “The organizations that handle it best won’t necessarily be the ones with the most powerful AI. They’ll be the ones that architect AI as a resilient security capability rather than a single cloud service.”

Hugging Face has contained the intrusion, rebuilt compromised nodes, rotated credentials, and reported the incident to law enforcement. The company recommends that all users rotate access tokens and review recent account activity. Mid-incident, Hugging Face found out whether its own AI tooling would be available, and the first answer was no. Security leaders running AI in production should find out in incident response planning instead, before an autonomous agent forces the test.

On Thursday July 16th, Moonshot AI released their latest flagship model Kimi K3. K3 is a 2.8T parameter MoE model which will have its weights released on July 27th. Much of this article follows as a reflection on the state of the ecosystem, under the assumption that Moonshot keeps their promise of the weights release date. This is a more extreme view of the equilibrium, and many of the results end up in a middle ground if the state of affairs is that China has similarly powerful, but closed models (i.e. K3 is never released).

The key fact is that either the open-to-closed or American-to-Chinese model performance gap has been reduced from the debated 6-9 months to something shorter, say 3-5 months.

From the release materials, it is clear that K3 is a true frontier model. It will be the closest open models have been to the frontier since DeepSeek R1. DeepSeek R1 was a different story. This was a Chinese lab being extremely quick to pivot to reasoning models and release one faster than many American companies. Kimi K3 an example of a Chinese lab executing on scaling the known areas: data, algorithms, architecture, tools, environments, etc.

Kimi K3 comes in at #2 overall on the Vals AI index, #3 overall on Artificial Analysis’s Intelligence Index (only beaten by Claude Fable and GPT-5.6 Sol Max while being cheaper), #1 overall in Frontend Code Arena, and more impressive results. Moonshot AI is going toe to toe with Anthropic and OpenAI with far, far fewer resources.

It is clearly the strongest open model ever released. It should be clear looking at this model that if adversarial distillation from the closed frontier models in the U.S. contributed, it is at most to a relatively small degree. AI observers who followed the distillation panic and came away with the wrong conclusion that Chinese AI labs are only producing good models due to IP theft are in for an awakening – that Chinese companies are extremely good at building models in the same way the leading American companies are. Moonshot AI is solving many of the same problems that folks at OpenAI or Anthropic are solving. I’m confident there will be more distillation discussion, and pressure, but the evidence is now out that Chinese companies can do more than just fast following.

Meeting some of the core Kimi team on my trip to China, it was clear to me that they had incredible culture, some would say aura, and a freedom to express it – within the constraints of a GPU-limited environment. Where building models is so much of a scaling game, much of the ability to build a good model still comes down individual execution, motivation, and expression. Having visited them, this result is less surprising. Having visited many AI companies, very few have a culture that you can immediately pick up like this.

At the same time, China’s AI adoption trends started later than those in the U.S. So, while all the Chinese labs have way less compute than their counterparts in the U.S., more of it can certainly go to training. When I joked around about how much compute an average researcher at OpenAI could have – say a few thousand H100 equivalent machines – the researchers at Kimi were shocked. The org chart and approach to building the Kimi models surely reflect this, but it is difficult to tease out what this looks like without substantial proprietary information.

The state of affairs on peak model performance is roughly as follows:

1. Anthropic – Claude Fable 5

2. OpenAI – GPT 5.6 Sol

3. Moonshot AI – Kimi K3 (open weights*)

4. SpaceXAI – Grok 4.5

5. Zhipu (Z.ai) – GLM 5.2 (open weights)

6. Meta – Muse Spark 1.1

7. DeepMind – Gemini Flash 3.5

8. Alibaba – Qwen 3.7 Max (3.8 announced, also to be open-weights, when writing)

It is astonishing to see DeepMind, and some of the other American giants this low. In many ways, the X AI team deserves more credit. A visual summary from Artificial Analysis is below:

This release and other recent events have caused a major change in direction for the most likely outcomes in the balance between open and closed models. I’ll unpack them individually.

In many ways, it feels like the start of a new era. An era with much more competition, but also a much higher need for coordination, as we rollout incredibly powerful technologies around the world.

Share

1. China’s recommits to open-source AI – showing a different read on near-term risks

Many people started following China’s AI scene relatively recently, so they can reach the conclusion that releasing models openly is their core strategy. In fact, I think most labs have a core strategy far closer to Anthropic or OpenAI – build the best intelligence possible. Having followed and engaged with the Chinese labs for years now, the best explanation for their original turn to releasing their models openly is practicality. They needed to release the models openly to get adoption, attention, and feedback (especially in the high-value, Bay Area market).

For a long time, there had been very limited policy in China explaining the role of open-source AI, and what could be the “country-level strategy.” To my knowledge, no senior leaders had commented on open-source AI publicly. This changed this week too, as Xi Jinping gave a keynote address at the World AI Conference (WAIC), and very directly committed the future of China’s AI ecosystem to open-source and global diffusion. This commitment to the status quo, the same week as the announcement of the strongest open-weight model to date, is a clear mark in the early history of modern AI.

This comes during a time period where many potential paths forward have been discussed for the Chinese AI industry – Will they stay open? Can they keep up with the American labs in scaling? Is there a growing revenue market in China? With these, the focus has been on China’s risk tolerance, the companies’ ability to monetize, and any closely related reason for a company to stop releasing their best models openly.

In tying Xi’s commitment in time to a very strong model, China has implicitly commented on its risk tolerance with respect to releasing open-weight models. For the time being, it is a read into the perceived risks of topics like strong cybersecurity capabilities (or bio-dangers) within the Chinese system.

The simplest explanation is that China’s government is definitely following potential risks from the models closely – likely with more technical scope than the US government’s vibe regulation – and would take action if it measured risk. The simple explanation is that they do not find current frontier models to have meaningful risk.

At the same time, China’s economic decision makers think having AI adoption is good, so they can make profits on the industry later – after growing distribution (as China has done for cars, solar, advanced manufacturing, and many areas in recent history).

These can seem somewhat shocking, in an American AI media landscape that has gone through months of hype and fearmongering over the Claude Mythos model. This surprise should be excellent grounding – the world does not have a unanimous agreement with the narratives about AI that we hear most in the U.S.

2. Open models as the economic Achilles heel of frontier labs

Many of the narrators guiding the discussion on AI have clear incentives to depress the perceived capabilities of the best, open AI models. Dean Ball – who is personally supportive of open models, but now works at OpenAI – had a widely commented on post with some reflections on Kimi, where he said the following on open models. It is important to understand the statement, as it focuses the role of open models in the economic side of the AI buildout. Dean says:1

3. Open-weight models are inherently decelerationist, and I’m continually surprised to see the so-called “accelerationists” so excited about open-weight models.

Explaining why open-weight models are a form of decelerationism is important to understanding the coming world order. He is right.

Open models are decelerationist economically for the frontier labs, which will slow the net investment and capex rollout for AI. This is due to the fact that strong open-weight AI models massively reduce the margin potential for the closed labs. This has two effects. First, the AI labs have fewer profits to re-invest into future models. Second, the market sees the terminal value of these companies as being lower, so they will kneecap future fundraising rounds. These together will slow timelines to the most transformative AI models, but I do not see them as strong enough effects to stop OpenAI and Anthropic from being a few of the top valued companies in the world.

These, to me, are a net good for society. As open-weight models are accelerationist for AI diffusion across the economy by having the entry price for intelligence at a certain level of performance be lower. Open models also encourage customization. The thing is that this type of diffusion is by its nature far slower than the frontier AI labs products, who sell tools used directly by developers. The potential for open models is for nearly every business to use them to craft domain-specific agents. This economic diffusion takes an extremely long time! I’ve described this as open-weight models being on a much slower starting, but potentially bigger exponential. The problem is, if closed models get too far ahead in raw capabilities, this ability to customize can be moot.

The combination of increased diffusion and decreased concentration of power in the AI labs I see to be very positive for the AI transition. It gives us more time to figure out the hard problems of new capabilities and lets more stakeholders impact the story – any one company is very likely to have issues with controlling the world’s most important technology safely.

It is, of course, important to me in this world for the best models to still be made by the U.S. companies, which will allow the US to control the trajectory of the technology and its values. I also expect this to be the case, as the U.S. has larger capital markets that are willing to invest in AI (and a growing share of profits), but it is not a given.

3. China’s efficiency advantage

Kimi’s launch blog has some technical details that confirm the sort of improvements that are supplying the consistent model improvements we feel. To select one:

Kimi K3 is built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), two architectural updates designed to improve how information flows across sequence length and model depth. We have also scaled up Mixture of Experts (MoE) sparsity, effectively activating 16 out of 896 experts when paired with a Stable LatentMoE framework. Together with refined training and data recipes, these structural changes yield an approximate 2.5× improvement in overall scaling efficiency compared to Kimi K2, allowing the model to convert compute into intelligence more effectively.

Training efficiency really adds up. They will result in continued, incredible steps for the models.

As an aside, tracing the path of this particular innovation through the ecosystem is an interesting example. Kimi Delta Attention (KDA) was introduced in the Kimi Linear paper, which is similar to the Gated DeltaNet used for Olmo Hybrid (my last Olmo model while at Ai2). Qwen’s latest models switched to a related architecture and the recent Nemotron models also are hybrid (but still closer to Mamba than Gated DeltaNet). It’s awesome to see new architecture ideas like these, which were heavily progressed by academia, get so quickly translated into frontier-scale models. Gated Delta Networks were introduced in late 2024, building on ideas from Mamba. By mid 2026, they’re in frontier models.

I chose to focus on this example, partially because the Kimi team put a cool number to innovations between models, but primarily to give space to a broader discussion of China’s resource efficiency.

It is becoming clear that the Chinese labs are far more capital efficient. In a world where scaling laws dictate that intelligence is proportional to effective capital – which buys compute, data, & talent – that may be the greatest strength your AI industry could ever have. There are many possible explanations for why this is the case, such as Chinese researchers being paid less while being more effective at LLM research puzzles, but we will probably never get such specific reasons.

Since writing my notes on China, I’m hearing more about an emerging data industry in China (far behind the billion dollar budgets of Anthropic for data) and that Chinese labs have access to meaningful training compute (by skirting export controls). Chinese companies do not have the same inference demand (until recently, as Moonshot AI had to pause new subscriptions for access to their K3 model - while the API is still live), so much more of their compute could go to training. These areas impinge heavily on the truth of the ability of the labs, but we have very limited measurement into them.

The facts on the ground are that these Chinese labs have raised orders of magnitude less capital than any slice of the American AI ecosystem. The most direct comparisons are to OpenAI and Anthropic, who have slightly better public models. Others, such as Google and Meta have the largest cash flows in the history of business, and are behind on building models. As for American neolabs, the picture is even more competitive – Thinking Machines released their first model recently, Inkling, which is strong but not in the same class as Kimi K3.

These American companies with more resources could still catch up, but you need to strongly weigh the public measurements we have of model quality and not resort to hope – which often reflects a bias. If the Chinese labs do have a latent advantage, they could continue to utilize that to build even stronger models than all the competitors! Many outcomes are plausible and K3 should increase most people’s probability that China can outright lead in AI capabilities in the near future on the back of more efficient training efforts – even if it’s not your most likely predicted outcome.

A big contributor to the capital efficiency is likely in the approach, where American labs are spending meaningful energy in pushing the frontier in dramatic, big steps, and the Chinese labs are more focused on catching up — this catch-up is cheaper. Just as the student model can outperform the teacher in distillation generally (not limited to the adversarial distillation of the Chinese labs), an approach of “trying to catch up” rather than “invent the next paradigm” could lead to stronger models.

4. A growing ecosystem of frontier, open models

The weekend after the Kimi K3 release, while writing this and discussing the events broadly, Alibaba announced that a 2.4 trillion parameter Qwen 3.8 model is coming soon with open-weights. Historically, Alibaba has kept their largest models as API-only offerings via their cloud business, so this is another big vibe shift opening the doors to the next chapter of the open model economy. Even if the model is behind Kimi K3 on benchmarks, it signifies that Chinese companies may not only be maintaining the status quo for their open model strategy, but leaning further into it.

If this Qwen 3.8 model releases soon, i.e. before the next Gemini model, it could push Google to the 8th position on the leaderboard of labs with the smartest models – a list that China has been climbing. There are other rumors of more strong Chinese models soon, with DeepSeek V4 expected to graduate out of it’s “preview” version.

Leave a comment

5. The very beginning of a long story of frontier open-weight policy

I think that if Claude Mythos was released as an open-weight model today, the negative outcomes would be relatively minor. This is a somewhat challenging opinion to hold, as we have very limited public cybersecurity evaluations and it is a complicated ecosystem (and because I trust many people at Anthropic). I still stand by it. The risks have been over-hyped.

The problem is that this will not always be the case for the strongest AI models. Far stronger models are coming — and with them increased risks — so it is an incredibly safe equilibrium for the best models to be accessed in a controlled, closed manner several months ahead of similar open-weight models.

Open-weight models which are very controllable by the user will always be coming — you cannot effectively ban digital products, especially from bad actors — as AI training has proven globally accessible longer than many analysts expected.

Still, as I write this, the government continues to flirt with more measures aimed at restricting open-weight models in the U.S. The latest is from Axios:

Behind the scenes: The Commerce Department last year considered adding multiple Chinese AI labs to its “Entity List,” which would effectively cut off U.S. access without a license, a source close to the administration told Axios.

• The National Security Agency and White House Office of the National Cyber Director also considered putting out an advisory on Chinese AI lab threats last year, practically discouraging U.S. companies from using their tech, the source said.

• The White House considered implementing an executive order saying U.S. companies could only host Chinese models if they could guarantee security and take liability if it were breached, the source added.

• Commerce last summer also circulated draft rules within the administration leveraging its authorities to secure domestic supply chains to target Chinese open-source models, another source close to the administration said.

This would leave the U.S. in a very asymmetric state where the best models in the U.S. have guardrails on cybersecurity tasks, but global actors have access to great Chinese open-weight models to probe our defenses. This is one of many examples where banning open-weight models is not only harms the free markets of AI but also makes the ecosystem less safe in the short-term. There are other very bad outcomes, such as slowing the diffusion of AI applications and AI research, as I discussed above.

These equilibriums are very hard to maintain, especially as AI tools accelerate progress in the models, but it is important to maintain this status quo between open and closed. Having a model that is truly alone at the frontier in capabilities — something like Mythos when it was announced — also be open-weight poses serious risks as we go into the unknown of capabilities. Models are going to progress very fast and it is increasingly hard to measure their total capabilities.

We are then stuck in a world where we are trying to thread the needle on open models. It’s reasonable to not want something so powerful to be diffused globally in an instant, but meanwhile the makers of the models are incentivized to hype their capabilities, and their competitors are incentivized to hype their risks. It all comes down to careful measurement and proactive hardening of society to risk vectors.

This careening train of policy debates, model releases, and raucous reactions is only going to continue from today. We’ve been on a train of rapid progress, where all the key ideas of how AI should play out are tested, since the release of Claude Opus 4.5 last December, which sent us down the agentic pathway. The key to making good decisions here is evaluation capabilities, independent of the companies with the largest financial stakes. One of many actions needed then, as we enter the AGI era of AI governance, is an Operation Warp Speed style approach of bootstrapping state capacity (and other independent actors) that can evaluate models accurately, and study emerging risks.

Conclusion: The wake-up call

Open-weight models, by accelerating the diffusion of capabilities, are a massive escalation in the good and the potential bad of AI. For now, the bad side of frontier language models has been largely hypothetical, but that will not always remain the case.

Having open weight models be slightly behind the closed frontier is our natural buffer to mitigate the risks. The key point is that we must collectively act to mitigate potential harms as they appear, and whether open-weight models are 3 or 6 or 9 months behind, that is still a very short timeline. If we regulate open-weight models heavy-handedly, I suspect much of the world will be lulled into thinking we no longer need to act. All we would’ve done is slightly delayed the inevitable — open models will continue to cross all the key capability thresholds eventually and regardless of legality.

Understanding and benefiting from this open-closed dance must be a collective action from the AI community across all sectors of power and influence over the coming years.

Kimi K3 is a watershed moment because frontier open-weight models are now real. Many hypotheses will be tested on where risks of open-weight models truly land — I suspect it’ll be narrower than many expect, and many risks of AI will still be proliferated by closed and “safer” APIs. The evaluation of these risks will evolve in time with an acceleration of AI’s integration in our economy. We cannot get one without the other, and we will continue to get both.

With this, 2025 was when open models started to be taken more seriously — especially when China leaped ahead with such a clear lead — as people realized that it would not be a unipolar world, with only American, closed AI labs determining the trajectory. 2026 is when those previously discussed, potential risks and accelerations due to truly frontier, open-weight models landed.

The demand for AI continues to accelerate. Workloads are getting larger, models are becoming more complex, and there is mounting pressure to deploy AI compute...

The demand for AI continues to accelerate. Workloads are getting larger, models are becoming more complex, and there is mounting pressure to deploy AI compute infrastructure faster than ever. AI factories—data center-scale systems that continuously convert data and energy into intelligence—are being deployed to meet this insatiable demand. This AI factory approach to the data center has…

Source

Developers building 3D, design, simulation, robotics, and industrial digital twin applications need ways to bring physical AI capabilities into the tools and...

Developers building 3D, design, simulation, robotics, and industrial digital twin applications need ways to bring physical AI capabilities into the tools and services they already use. Many of these workflows already depend on OpenUSD scenes, simulation-ready (SimReady) assets, Blender-based workflows, CAD pipelines, or domain-specific app stacks. The challenge is how to provide applications and…

Source

Presented by JumpCloud

---

The organizations losing confidence in AI are the ones most likely to get it right.

Six months ago, 40% of IT leaders described their organizations as mature in AI deployment. Today that number is 23%. Before you read that as a setback, consider what it actually reflects.

We recently surveyed 800 IT leaders across the U.S. and U.K. for our Q3 2026 trends report, and the data tells a consistent story: the organizations revising their self-assessment downward are overwhelmingly the ones that have moved AI agents from pilots into production. They’re not losing faith in AI. They’re running into the problems that only show up when agents are doing real work in real systems, and they’re being honest about what they found.

That kind of honesty is harder to come by than it sounds, and it matters more than the confidence number itself.

Deployment was the easy part

84% of organizations plan to expand AI use in IT operations over the next 6 to 24 months, so the drop in confidence isn’t a retreat. What it reflects is a more accurate picture of what production actually requires.

In a pilot, an AI agent does one thing in a controlled setting. In production, it accesses real systems, makes decisions that affect real workflows, and operates continuously, often without a human in the loop. The governance infrastructure that entails is materially different from what it took to get the pilot working. Most organizations built enough to ship. Fewer built enough to scale.

The IT leaders revising their self-assessment are confronting questions they didn’t have to ask at the pilot stage: Can we see every agent running in our environment? Do we know what each one can access? If an agent behaved unexpectedly last week, how long would it take to find out? For most organizations, at least one of those answers is uncomfortable.

The gap between perception and reality is where risk accumulates

The graphic above captures the structural problem. Across confidence, governance, and autonomy, the same pattern holds: deployment is moving faster than the controls built around it.

The organizations that have closed this gap share specific characteristics. They’ve consolidated their IT environments rather than adding tools to solve each new problem, because every additional platform creates another place where agent identity, access, and accountability can go unmanaged. They treat AI agents as governed identities rather than tolerated shadow processes. And they measure what AI actually produces, not just what it deploys.

The payoff is tangible. Organizations in the top tier of our maturity model are five times more likely to report no barriers to expanding their AI agents than the average organization. They are not more cautious about AI. They are more confident in it, because they built the foundation that makes confidence earned rather than assumed.

The governance gap has a specific shape

The hardest problem in enterprise AI right now is not capability. It is accountability, and the data makes the specific failure point clear: non-human identity governance is the least adopted AI security practice we measured, in place at just 21% of organizations.

Non-human identities now outnumber human users in 83% of organizations, and that population is growing fast. Yet most of those identities exist without the governance structures that every human employee has as a matter of course: no formal record, no named owner, no defined scope of access, no offboarding process when their purpose expires. They keep running. They keep accessing systems. They keep accumulating permissions. We call these Zombie Agents, and they are the service account problem of the AI era, operating at machine speed and in every department.

The accountability gap is where real risk lives. When a human employee takes an action, there is an implicit accountability chain. When an autonomous agent takes an action, that chain breaks unless it has been deliberately engineered. Most organizations have not yet engineered it, and the gap between the autonomy agents are being granted and the oversight structures in place to manage them is widening every month.

What the confidence drop is actually telling us

When AI maturity confidence was uniformly high across the market, that was worth worrying about. It meant most organizations hadn’t yet run into the hard parts. A selective drop, concentrated among organizations actively running agents in production, means the market is developing a more accurate picture of what AI operations genuinely require.

The organizations recalibrating are doing the work that makes long-term AI adoption possible: building identity infrastructure that covers agents alongside humans and devices, unifying the environments where governance needs to apply, and measuring outcomes rather than just counting deployments. They haven’t lowered their ambitions for AI. They have raised their standards for what it means to run it responsibly.

84% of organizations plan to expand AI use over the next two years. The ones that will do it well are honest enough, right now, to admit what they haven’t yet built.

JumpCloud’s Q3 2026 AI Readiness Research report (n=800 IT leaders, U.S. + U.K.) is available here. The report covers AI agent deployment stages, identity governance gaps, IT unification benchmarks, and budget realism across mid-market and enterprise organizations.

Rajat Bhargava is CEO and Co-founder at JumpCloud.

---

Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. For more information, contact sales@venturebeat.com.

Nature Machine Intelligence, Published online: 20 July 2026; doi:10.1038/s42256-026-01274-0

Li et al. show that recurrent neural networks optimized for free recall discover diverse, human-like memory strategies beyond classical temporal context models, with top models using an index-based mechanism resembling the memory palace technique.

The next time you apply for a job, AI may screen your résumé before any human sees it. But there’s good reason to question whether AI will judge you fairly. Researchers already know that LLMs pick up human biases from their training data. New research suggests that LLMs can also develop their own biases from experience—and stereotype job applicants more than humans do. As AI companies race to build agentic models that remember the tiniest details about users, they may be handing them ammunition for forming those biases. 

Researchers at Princeton University and the University of Chicago ran LLMs, including ChatGPT, Claude, and Gemini, through a simulated hiring game, adapted from a psychology study that explored how humans can form stereotypes. Each model was told it had been hired as a consultant by the mayor of a fictional city and was then asked to help hire people for 20 jobs, including doctors, lawyers, child-care aides, and janitors. Candidates came from four fictional ethnic groups: Tufa, Aima, Reku, and Weki. 

In each round, there was a new job opening and four candidates, one from each group. After the model hired a candidate, it learned whether they succeeded at their job and moved onto the next round. The model was told to make as many successful hires as possible over 40 rounds. Unbeknownst to the models, all candidates were equally likely to succeed at every job.

The models quickly started segregating candidates from different groups into different jobs on the basis of early observations of hiring outcomes. For example, when a model was told an Aima had failed as a doctor, a job considered to require high levels of warmth and competence, it veered away from hiring all Aimas as doctors. Instead, it started hiring Aimas as janitors, which the model classified as being less warm and competent than doctors. Newer models with higher reasoning capabilities, such as OpenAI’s o3 and DeepSeek’s R1, showed stronger biases.

The models were even more likely to stereotype people by demographic group than the human participants in the original study. On the study’s segregation scale, where 2 means every group has been completely confined to its own job niche, human participants scored 0.84. The models scored roughly 65% higher, with OpenAI’s reasoning model o3 scoring 1.83, close to the maximum possible.

That’s because LLMs “really are eager to create generalizations from limited data,” says Ryan Liu, a PhD student at Princeton University and a coauthor of the study, which was published in a paper at ICML in Seoul in July. “That’s literally a lot of what they’re optimized for.”

Every decision-maker, human or machine, faces a trade-off between sticking with what worked before and trying something new that might work better—a phenomenon psychologists call the “exploration-exploitation dilemma.” It’s like choosing between a new restaurant and your reliable favorite. 

Because LLMs are trained on math, coding, and science problems—tasks that reward generalizing from just a few examples—they can settle on a hunch too early. And the same instinct that helps LLMs crack logic puzzles also makes them quick to stereotype. When LLMs rush to generalize in social settings, “that’s when things tend to go wrong,” says Liu. OpenAI and Anthropic did not respond to requests for comment.

The finding is especially relevant now that chatbots are gaining improved memory and personalization features, says Angelina Wang, a computer scientist at Cornell University who did not work on the study. When a chatbot draws on its previous conversation history, it can “over-index on the same kinds of behaviors it’s experienced before” and form biases, she says.

Simply having chatbots remember less isn’t a fix, though, because users want chatbots to remember what they say. “We still are trying to figure out just the right amount that isn’t too much or too little,” says Wang.

Telling the model to be fair didn’t change its behavior much. “Either it can’t put these values into action or that process is being submerged under the tendency to try to optimize for the goal of getting the most correct hires,” says Liu. But promising the models an additional bonus for diverse hiring made them far less biased. The trick, then, is to design goals that “incorporate desirable social values in order to make the large language model act in socially desirable ways,” says Liu.

The models also became less biased when they were told more personal information about individuals. In another experiment in the same study, the researchers asked the models to resettle members of different ethnic groups in cities across Canada. When the models were told personal information relevant to the ability to adapt to a new city, such as age and education, they were less likely to segregate people by their ethnicity. But when they were given irrelevant information, such as hair color and tattoo shape, the models largely fell back to sorting people by their ethnicity again. 

To what extent AI systems will stereotype job applicants in the real world is still an open question. While the models in the experiment immediately learned whether they’d made successful hires, a model screening résumés in the real world doesn’t get an instant report card. Companies can take a long time to find out whether a new hire is any good, if they ever do.

As LLMs learn from experience to make decisions about who gets hired, who gets a loan, or who gets parole, the biases we should worry about may include ones no human ever taught them. “These novel biases—they’re sort of ever present,” says Liu.

The enterprise technology ecosystem is caught in a costly cycle. Over the past two years, millions of dollars have been funneled into generative AI pilots, yet many of these initiatives stall out before ever reaching a live production environment.

When a project fails, the immediate instinct of technical leadership is often to blame the model: The context window was too restrictive, the latency was too high, or the reasoning capabilities simply were not there.

But as data engineers building the scaffolding for these systems, we often see a different reality: The model receives the blame, but the pipeline usually contains the root cause. Production gen AI rarely fails because of model limitations alone. More often, it fails because the enterprise data foundation underneath it is fundamentally unready.

This is what I call the 'Cleanup Trap': The false belief that an organization can pipe fragmented, inconsistent, and ungoverned legacy data into a large language model (LLM) orchestrator and simply “clean it up” or patch it at the retrieval layer.

The mirage of the retrieval layer

In a standard retrieval-augmented generation (RAG) architecture, the retrieval layer is tasked with pulling relevant business context to ground the model’s responses. Because modern frameworks make it simple to stand up a vector database and a basic embedding pipeline, leadership often assumes that the data engineering problem is solved.

It is not.

When an embedding model receives raw, unvalidated data directly from operational silos, the resulting vector space inherits the structural noise, duplicate records, and conflicting states present in the source systems.

If the core data pipeline suffers from silent degradation — schema drift, missing fields, delayed change-data-capture (CDC) synchronization — that degradation cascades directly into the vector store. An AI model cannot accurately synthesize customer intelligence if the data pipeline behind it is serving stale, contradictory profiles across disparate storage layers.

No amount of prompt engineering, semantic reranking, or vector hyperparameter tuning can compensate for a broken ingestion pipeline. If the foundation is compromised, the downstream application will hallucinate, expose unauthorized context, or fail to deliver deterministic value.

Shifting from ad-hoc patching to programmatic guardrails

To break out of the 'Cleanup Trap,' enterprise data teams must stop treating data quality as a post-processing step. They need to treat data readiness for AI with the same rigor they bring to traditional transaction processing.

This requires a deliberate architectural shift toward zero-trust data ingestion, structured validation frameworks, and automated anomaly detection before data ever reaches an AI orchestration layer.

1. Harden the ingestion pipeline

Data quality checks cannot exist as a nightly batch afterthought. If an enterprise AI application relies on real-time data to assist users, validation must happen inline.

Teams should implement explicit schema validation checks at the earliest ingestion point, such as the streaming ingress layer or the bronze landing layer of a medallion architecture. If an upstream operational database mutates a schema without warning, the pipeline should quarantine anomalous payloads rather than allowing corrupted metadata to pollute downstream AI contexts.

2. Use multi-tiered algorithmic validation

Static row-count validation rules are insufficient for AI readiness. True data health requires a multi-tiered approach.

This means pairing structural verification — null checks, type conformance, and schema validation — with statistical profiling to monitor for data drift. Tracking metric deviations across feature distributions helps ensure that historical context remains stable over time.

If a pipeline suddenly processes an unexpected spike in empty string variables or structurally deviant fields, automated alerts should trigger an immediate pause before vector database updates continue.

3. Decouple security and compliancemfrom the model

An LLM should never be the arbiter of data access control. Trying to enforce row-level security or personal data filtering through system prompts is a compliance risk.

Security must be managed within the data infrastructure tier. Enterprise data foundations should enforce strict access controls, tokenization of sensitive identifiers, and rigorous lineage tracing before information is indexed into vector stores or passed into an agent’s context window.

Technical alignment: A pragmatic blueprint

For technology leaders mapping their infrastructure roadmaps, AI readiness requires evaluating data pipelines against a strict operational checklist.

• Can you trace a flawed AI response back to the exact pipeline execution, source record, and transformation step that produced it?

• Does your data lake architecture have a programmatic mechanism to segment and quarantine corrupted or non-compliant data before it reaches production feature stores?

• Are your operational systems and AI-facing vector databases tightly synchronized, or are your agents making automated decisions based on outdated snapshots?

These questions matter because production AI is not just a model deployment problem. It is a data reliability problem.

Building for the production era

The honeymoon phase of gen AI experimentation is ending. Enterprise leaders are demanding measurable, predictable, and secure business outcomes from their AI investments.

If an organization wants to transition from isolated, impressive-looking demos to resilient, production-grade AI systems, it must redirect its focus. Stop looking exclusively at the model tier.

The real competitive differentiator is not only the LLM an organization chooses. It is the engineering discipline, data governance, and pipeline resilience of the infrastructure built to feed it.

In the production era of AI, data engineering is no longer a backend function. It is the control plane for enterprise intelligence.

Naveen Ayalla is a senior data engineer.

Apache Spark 4.2 launched last week, and it signals an expansion of Spark’s decade-plus role at the center of enterprise data processing

With new features for AI workloads, including governed metrics, vector retrieval primitives, real-time processing, improved Python support and native geospatial analytics, Spark 4.2 builds on a recent history of new AI and streaming features, reflecting how many engineering teams use the platform today. The release builds on Spark’s traditional role as a data processing engine by adding more of the capabilities needed to support production AI applications.

The launch introduced several features that enable developers to do more without leaving the platform, and for teams already using Spark, that could mean fewer systems to manage.

Governed metrics prevent conflicts

One team’s definition of a business metric isn’t always the same as another’s. Over time, those differences can lead to conflicting reports and uncertainty about which number to trust.

That becomes even more problematic when AI applications start consuming the same enterprise data that analysts and business intelligence tools use. If different teams define the same metric differently, AI systems can produce inconsistent results for the same question.

Spark 4.2 introduces governed metric views to address that issue. Organizations can define a business metric once and reuse that definition across applications. A metric view makes dimensions and measures first-class objects that Spark understands, so the engine can preserve the intended aggregation semantics regardless of who or what is querying it.

Organizations can define a business metric once and reuse that definition across applications.

Vector search goes native

One of the more significant additions is native vector search, which reduces the need to move data between Spark and a separate vector database.

Spark 4.2 introduces vector distance and similarity functions, vector normalization, vector aggregation, and NEAREST BY, a new SQL operator for top-K similarity searches. By bringing vector search into Spark, developers can keep more of their retrieval pipeline on the same platform.

By bringing vector search into Spark, developers can keep more of their retrieval pipeline in the same platform.

Python interoperability gets easier

Spark 4.2 makes it easier to move data between Spark and Arrow-native tools. With support for the Arrow C Data Interface and the PyCapsule protocol, Spark DataFrames can be passed directly to tools like Polars and DuckDB without copying or serializing the underlying data, as long as both sides support the standards.

Python also gets a few other updates. PySpark has been expanded; Arrow-optimized UDF execution is now the default, and Python Data Sources now include built-in time and memory profiling to help developers troubleshoot custom connectors.

Spark Connect makes the engine callable by agents

Spark Connect, which separates the client from the Spark server via a gRPC- and Arrow-based protocol, receives several updates in 4.2. The key idea is that a client builds a logical plan, the server handles analysis, optimization, and execution, and the results come back as Arrow batches. The client doesn’t need a full Spark runtime or a colocated JVM.

This update includes several changes to Spark Connect, the project’s client-server interface. AI applications can send processing requests to a remote Spark cluster while the work continues to run inside Spark. The release improves RDD API compatibility, error handling, and status reporting along that path.

Streaming powers real-time AI

Streaming gets several updates in Spark 4.2, including Auto CDC and Real-Time Mode. Many AI applications depend on continuously updated data rather than scheduled batch jobs. Auto CDC brings first-class change data capture to Spark Declarative Pipelines, handling the merge logic for keeping target tables current as source data changes — something that previously required hand-written, error-prone code. The new CHANGES SQL clause allows teams to retrieve data changes through a single SQL interface.

Spark 4.2 also adds built-in GEOMETRY and GEOGRAPHY types along with ST_* functions for location-aware analytics, without requiring external spatial extensions. For teams doing anything with location data — logistics, real estate, IoT — this removes another reason to move data out of Spark.

The bigger picture

Spark 4.2 brings more of the AI and data stack into the platform itself. Features that once depended on separate tools can now be handled directly in Spark.

For teams that currently use Spark for ETL and then hand data off to other systems for retrieval, governance, or real-time processing, this release starts to blur that line. As more AI applications run directly on operational data, Spark is becoming part of the serving layer rather than simply preparing data for it.

Spark is becoming part of the serving layer rather than simply preparing data for it.

The post Spark 4.2 has a feature that could retire your vector database appeared first on The New Stack.