Most teams building retrieval augmented generation (RAG) systems for high stakes classification make the same architectural bet: Route every ambiguous case straight to the language model and trust the retrieved context to sort it out. This works fine in a demo. It falls apart the moment the system has to survive an audit, a regulator, or a compliance officer asking why a specific decision was made six months ago.

I have spent the last year building RAG based classification systems in regulated enterprise settings, where the cost of a wrong answer is not a bad chatbot reply. A decision has to hold up to scrutiny long after the model produced it. This environment forces a different design philosophy than most AI engineering content assumes.

Here is what changes when you cannot afford to be probabilistic about everything, and how a cascade architecture solves it.

The invisible cost of an all LLM pipeline

The appeal of routing everything through a large language model (LLM) is obvious: Fewer moving parts, faster iteration, the model handles unanticipated edge cases. The problem shows up later, in three places.

First, auditability. "The model decided based on retrieved context" is not an acceptable answer. You need a decision path a human can reconstruct without rerunning inference and hoping for the same output.

Second, cost at scale. If your system processes tens of thousands of cases a day and every one hits an LLM call with several retrieved documents in context, your inference bill and latency both scale with volume in a way that rule based logic does not.

Third, and least discussed, model drift on the easy cases. LLMs are excellent at nuanced judgment calls. They are inconsistent, in ways that are hard to detect, on cases that should have a deterministic answer. A clear structured match against known criteria should never depend on a language model's mood.

The cascade approach

The fix: Stop treating the LLM as the front line and start treating it as the escalation path. In practice this means a three stage pipeline.

Stage one is deterministic. Exact matches, structured field comparisons, and anything with a clear rule get resolved here with no model call at all. This stage should clear the majority of volume, often more than half depending on your data quality, and every decision is fully explainable because it is a lookup, not an inference.

Stage two is where retrieval earns its keep. For cases that survive stage one — and I mean survive as in they were not clearly resolved — you build a retrieval layer that pulls the specific evidence relevant to the ambiguity: Prior reviewer decisions on similar cases, contextual documents that explain an apparent conflict, or historical precedent that clarifies an edge case. The retrieval step matters more than the generation step here. If you retrieve the wrong context, even the best language model in the world will produce a confident, well reasoned, wrong answer.

Stage three is the LLM call, and it should only see the residue that stages one and two could not resolve. This is the part people skip when they design their first version, and it is the single biggest lever for both cost and quality. In one system I worked on, routing only the genuinely ambiguous 10 to 15% of cases to the LLM cut inference cost by roughly 6X compared to an all LLM baseline, while improving consistency on the deterministic majority to effectively perfect.

Designing the prompt for asymmetric risk

Once a case reaches the LLM stage, most teams default to a neutral prompt: "Assess whether this case should be approved or flagged." That framing is wrong for high stakes classification because the cost of the two error types is not symmetric. Missing something that genuinely needed attention can mean real harm downstream. Incorrectly flagging something that was fine costs a reviewer's time and a delay. Those two outcomes are rarely equally bad, yet a neutral prompt asks the model to treat them as if they were.

An asymmetric risk prompt makes that tradeoff explicit to the model rather than letting it guess at your risk tolerance. Concretely, this means instructing the model to treat uncertainty as a reason to escalate rather than clear, providing calibrated examples of both error types with their consequences spelled out, and asking for a confidence score alongside the classification rather than a binary answer. The confidence score becomes your second cascade point: Anything below a certain threshold goes to a human reviewer instead of being auto resolved, no matter what the model's classification says.

This sounds like a small prompt engineering detail. In practice it is the difference between a system that reduces reviewer workload and one that quietly increases risk while looking like it is working.

Evaluating a system like this properly

Standard RAG evaluation metrics were not built with this use case in mind, and using them without adaptation will give you a false sense of confidence. A few adjustments that matter.

Retrieval quality needs to be measured separately from final classification accuracy. A system can have excellent retrieval ranking scores and still make bad final decisions if the generation step misweights the evidence. Track them independently.

Your evaluation set needs deliberate oversampling of the cases that reach stage three, since that is where your system's judgment actually gets tested. If your eval set mirrors your production distribution, it will be dominated by the deterministic cases your cascade already handles well, and you will be blind to exactly the failures that matter most.

LLM as judge evaluation works for this domain but only if the judge prompt encodes the same asymmetric risk framing as your production prompt. A judge that treats both error types equally will systematically favor the wrong tradeoff when you are tuning your system.

Finally, build a feedback loop from confirmed outcomes back into your retrieval corpus. When a human reviewer overturns a model decision, that case and its correct resolution should become retrievable context for future similar cases. Without this, your system's handling of ambiguous cases never improves, it just keeps making the same category of mistake at the same rate.

The broader lesson

The instinct to reach for the most capable model for every decision is understandable, but in domains where wrong answers have real consequences, the more valuable engineering work is deciding what should never touch the model at all. Cascade architecture is not a workaround for LLM limitations. It is what a mature RAG system looks like once you have actually had to defend its decisions to someone whose job is to find the flaw in your logic.

If you are building AI systems for any regulated or high stakes domain, the question worth asking before you write a single prompt is not "How do I get the model to handle this well." It is "Which parts of this decision should never have been the model's job in the first place."

Vineet Vijay is a Lead AI and machine learning engineer.

DeepSeek's V4 Flash has topped model leaderboards and been hailed by developers as a "total monster" since its rollout. But in real-world testing, it completed just 53.8% of a batch of complex agent tasks.

Composio ran the model through eight different agent harnesses, including Claude Code, Codex, and OpenCode, on 30 deliberately difficult, multi-step tasks spanning live tools like Gmail, GitHub, Slack, and Google Sheets. Of 240 total runs, 129 passed — and only six of the 30 workflows were completed successfully by every harness tested.

The gap illustrates why orchestration, not raw model capability, may decide whether the model succeeds in enterprise settings: the same model produced substantially different results depending on the harness, tool configuration, caching behavior, retries, and provider stack it ran on.

DeepSeek said it will be hiking the prices for V4 Flash and Pro, models that have quickly become favorites among developers building coding assistants and agents.

While it seems the move might undercut its very appeal — strikingly capable models at ultra-low pricing that frontier providers simply can't match — it also moves the story beyond the now-clichéd "cheap Chinese model" narrative, as early use cases emerge and enterprises figure out where different models fit into their tech stacks and what workflows they should be aimed at.

"Insane" adoption numbers as DeepSeek flips the cost structure

DeepSeek rolled out V4 Flash to public beta July 31, and made V4 Pro generally available on August 13. The 284-billion-parameter Flash is built for volume and speed, the 1.6 trillion-parameter Pro for more complex workflows.

Both models have flexible reasoning capabilities (low, high, max) and ‘thinking modes’ applying chain-of-thought (CoT) reasoning to improve answer accuracy.

Users were immediately impressed by Flash’s capabilities. It has dominated OpenRouter's usage leaderboard since its rollout, currently the most-used model on the platform by weekly token volume.

“The adoption numbers of the initial DeepSeek V4 Flash were insane,” ML researcher Nathan Lambert posted to X, adding that the new version "scored the same as GLM 5.2," making it a "total monster" that will be used extensively.

DeepSeek switching the cost model adds an interesting dimension.

V4 API rates are going up by as much as 1,100% depending on the model, token type and time of use. The new pricing structure:

• Flash will be 22 cents per million input tokens and 66 cents per million output tokens off-peak; and 44 cents per million input tokens and $1.32 per million output tokens at peak. This represents a 57% to 371% increase.

• Pro will be 66 cents per million input tokens and $1.98 per million output tokens off-peak; and $1.32 per million input tokens and $3.96 per million output tokens at peak. This shows a 51% to 355% jump.

• Cache hits, meanwhile (when models reuse prompts rather than starting from scratch), are going up between 52% and 1,100%.

DeepSeek says offering 50% lower off-peak usage is intended to encourage “more flexible workload scheduling." Seventeen of every 24 hours stay at half price, and the new structure actually prices the company's home market the highest.

"This is not a simple price rise," said Sanchit vir Gogia of Greyhound Research. "It is a pricing architecture that makes the timing of inference an economic variable."

Work that can wait — such as batch evaluation, synthetic-data generation, and overnight development runs — moves into the cheap hours; interactive agents and live operations cannot. Gogia said irritation among developers and enterprises is genuine and vocal, and that DeepSeek's past low pricing doesn't obligate it to stay cheap forever.

At first glance, it does look like a “suicidal move from a platform still looking for credibility against more established AI model vendors,” said tech analyst Carmi Levy. The increases will certainly eat into DeepSeek's price advantage and force customers to weigh concerns around the company's Chinese origins more heavily.

Still, DeepSeek remains far cheaper by all pricing measures relative to competing models from OpenAI, Anthropic, Google, Cohere, xAI, and others, he said.

So, while the move will force DeepSeek to emphasize performance and security over cost, it hardly wipes out its already-notable price-performance advantage, and still gives customers ample wiggle room to justify its use for specific workloads, Levy said. The math will just have to be more tightly calculated.

“The advantage will likely erode over time as DeepSeek inevitably continues to align pricing with market realities, but for now it's still easy to make the business case,” Levy said.

Where can DeepSeek Flash fit into enterprise environments?

Adoption inside enterprises remains an open question due to cost, capability, reliability, data governance, security, and other factors.

One use case is batch processing, Levy said. This kind of work is typically routine and repetitive rather than demanding top-tier intelligence, so it makes sense to use a cheaper, more efficient model. “It’s a high-performance inference engine that enterprises can consider using for point solution workloads rather than as a wholesale replacement for the incumbent offerings,” Levy said.

Partial adoption will likely involve isolated, non-sensitive workloads with clearly-defined success metrics, strict oversight, permissions controls, and fallback models in case of failures, he noted. Broader deployment will require DeepSeek and its hosting partners to demonstrate strong reliability, security, privacy, auditability, and deployment options. As it adjusts price structures based on demand, DeepSeek also must retain a large enough price-performance advantage to justify any risk, he said.

Expect unsanctioned, smaller-scale use in backroom labs and contained test environments as IT teams get familiar with the new model and figure out when and how to bring it to senior leadership for budget approval.

“DeepSeek has built a well-earned reputation as a global disruptor,” he said, “and it’s clear that its march to broader enterprise adoption will continue to gather momentum.”

Testing DeepSeek in multi-tool workflows

While many use cases are still in the experimental phases, Meta software engineer Naman Ahuja offers one that could translate directly into enterprise environments. In a project unrelated to his employer, he built a home-automation agent with DeepSeek V4 Flash to explore how a lower-cost model performs as the reasoning/orchestration layer for a real multi-tool workflow.

When he leaves home, an agent coordinates several actions across otherwise separate systems: Such as setting a thermostat to “away” to reduce unnecessary energy use, arming a Ring security system, closing and locking doors.

“What interested me was not simply whether the model could understand a command, but whether it could translate intent into a sequence of actions across multiple tools where reliability matters,” Ahuja said.

The biggest lesson was that once a model can take actions, reliability matters as much as intelligence. The system needs structured tool outputs, verification that actions actually succeeded, retry/failure handling, and clear boundaries around what the model is allowed to do.

In the case of enterprise, “the architecture is similar.” Home devices change to ticketing systems, databases, CRM platforms, or infrastructure APIs. The most useful agents will likely orchestrate repetitive workflows across multiple systems, with scoped permissions, auditability, observability, and human approval for higher-risk actions.

“Many valuable AI agents will not be chatbots; they will be background agents coordinating APIs, infrastructure, and business systems in response to events,” he said.

Enterprises need tangible use cases

But Flash's API is still in public beta, Gogia pointed out, and there is not yet an evidence trail of settled enterprise adoption, real-world deployments, and named customers.

“The benchmark story is looser than its retelling, the portfolio story is newer than it looks, and the economics have moved into the system around the model,” he said. Developer mainstreaming is proven; enterprise standardization is not. “The model is mainstream by traffic and still unproven by contract.”

DeepSeek's own integration guidance is an important consideration, he said: Its documentation for at least one popular agent environment states that built-in V4 entries are not sufficient for reliable operation without compatibility overrides.

“Which is a vendor telling the market, accurately, that benchmark performance is not a proxy for production readiness,” Gogia said. “A model can score beautifully and still misbehave once tools, credentials, and state enter the room.”

The serving layer behaves no differently: the same open weights run by different hosts show visible differences in throughput and uptime. "Choosing Flash therefore answers one procurement question and opens three more: Who serves it, where it runs, and which controls surround it," Gogia said.

Prepare for a multi-model future

DeepSeek offers a nuanced case for a multi-model future. V4 Flash is being deployed as the high-volume worker inside diverse estates, Gogia noted: It handles routine generation, retrieval, and background automation, while more difficult or sensitive tasks go elsewhere.

“The question is whether its performance is sufficient for the real-world workflows enterprises actually run, not whether it tops every benchmark,” he said. Enterprises must determine which combination of model, harness, and provider completes the work safely at the lowest cost.

Adam Dalloul, CEO and founder of EmpirioLabs AI, pointed out that bigger isn’t always better; workflows should be task-dependent. For example, his team at EmpirioLabs AI — which hosts 100-plus models on one API, including DeepSeek V4 Flash — were recently working on translating its site into different languages, and there was no need for a large model like GPT 5.6 Sol or Opus 5 to complete the task.

“This is where subagents come in handy,” he said.

His recommended approach: Spawn cheaper subagents and adapt per task. For instance, use Flash variants for day-to-day work, and Pro variants when you need something more powerful. “It depends on the nature of your application.”

Many companies are pivoting towards their own internal benchmarks to route models effectively, Dalloul noted. For example, his team has a workflow that puts a model through various gates and instructions. This helps them identify the model with the speed and accuracy required for the task.

In another example, one of his enterprise clients exclusively wanted access to DeepSeek V4 Flash. They had tested a variety of models and V4 Flash was the only one that met their criteria for speed, cost, and an “appropriate intelligence threshold.”

Meta’s Ahuja agreed that smaller, more efficient models can handle frequent, well-defined agentic tasks, while more expensive frontier models can be reserved for “ambiguous, difficult, or higher-risk decisions.” The relevant metric increasingly becomes cost per successfully completed workflow rather than simply cost per token.

The trade-off, however, is that cheap inference does not automatically mean cheap or safe automation, he said. Once an AI system can take actions, reliability, verification, permissions, failure handling, and security become much more important.

“A failed text response is inconvenient; a failed action in an operational workflow can have real consequences,” Ahuja said.

Making a publishable video no longer requires a full crew, a suite of desktop NLEs, and a week of versioning. AI can now take a brief, a product still, or a script and return motion in minutes. The bottleneck moved. It is no longer “Can we generate a clip?” It is “Can the tool carry […]

A predictive maintenance system can flag rising vibration, unusual temperature patterns, or a combination of signals that suggests a component is likely to fail. That may be technically impressive, but it does not reduce downtime by itself. Someone still has to decide whether the signal matters, how urgent it is, what action should follow, and […]

There is a step in the development process for large language model (LLM)-assisted tooling that most teams skip because it's tedious, time-consuming, and doesn't produce results visible to end users: Verifying that what the model is saying is actually correct. Not fluent, not coherent, not topically relevant — correct in the sense of accurately identifying the right answer to the specific problem the tool was built to solve.

The gap between "this output sounds right to me" and "this output is verifiably correct" is where most LLM-assisted enterprise tools fail quietly. They pass internal review because the output sounds right. They fail in production because those people weren't reviewing against ground truth — they were reviewing against their intuition about what a good answer looks like.

This distinction matters more as LLM-assisted tools move from productivity accessories to components that influence real business decisions. If your AI-assisted tool is shaping how an analyst investigates a data quality issue, how a compliance reviewer decides whether to escalate a flagged record, or how an operations team triages a validation failure — the accuracy of its output has real consequences. "Seems reasonable" is not an adequate evaluation standard for that.

What qualitative evaluation actually catches

The standard evaluation approach for LLM output in enterprise tooling is qualitative: A sample of outputs is reviewed by someone with domain knowledge, judged against a mental model of what a good answer looks like, and the prompt is adjusted if too many outputs seem off.

This catches a specific class of problems: Outputs that are obviously wrong, poorly formatted, or off-topic. These are real issues worth catching. They're also the easy ones.

What qualitative evaluation consistently misses is the class of outputs that are wrong in ways that are difficult to see without checking against something external. An explanation that confidently identifies the wrong root cause, in language that sounds authoritative, based on reasoning that sounds plausible — this passes qualitative review. It fails the moment someone with the right context checks it against what actually happened.

In a system whose value proposition depends on accuracy, "sounds plausible" is not the same as "correct." The two can diverge significantly, and qualitative review won't tell you when they have.

What an actual eval harness looks like

The alternative is building an evaluation harness that scores model output against labeled ground truth — a set of cases where the correct answer is known, against which you can measure accuracy rather than coherence.

I built this while developing a root-cause explainer for data migration drift: A tool that takes a detected drift event and generates a ranked explanation of what most likely caused it. The first prototype produced fluent, specific-sounding explanations that passed qualitative review. When I tested it against cases where I already knew the root cause, the explanation was wrong often enough to matter.

The eval harness I built works in three parts.

First, a synthetic ground truth dataset: Cases where the correct answer is known by construction. This meant introducing specific, controlled causes into a test pipeline — schema changes, transformation logic bugs, source system behavioral shifts — recording exactly what I introduced, and running the model against the resulting drift events. The correct answer for each case was the cause I had deliberately introduced.

Getting the synthetic scenarios realistic enough to be useful required more care than I expected. Early versions were too clean — the drift signal was obvious in ways that real production drift events aren't. Adding realistic noise, overlapping signals, and cases where multiple plausible causes were present simultaneously was what made the synthetic set actually predictive of real-world performance.

Second, a scoring function that evaluates ranked output. Binary correct/incorrect isn't sufficient when the model produces a ranked list of likely causes rather than a single answer. An explanation that correctly identifies the root cause as the third most likely candidate is meaningfully different from one that identifies it as the most likely. The scoring function evaluated two dimensions: Presence — did the correct answer appear in the output at all — and rank — how prominently was it featured relative to incorrect candidates. These were combined into a weighted score that rewarded both finding the right answer and ranking it appropriately.

Third, systematic evaluation across the full synthetic dataset rather than spot-checking. Running the harness across the complete set reveals patterns that spot-checking misses: Which categories of problem the model handles reliably, which it consistently gets wrong, and which combinations of signals produce the highest rate of confident incorrect explanations.

What the evaluation revealed

The results were more informative than any qualitative review could have been.

Schema change scenarios scored well — the model was reliable at identifying upstream schema changes when the evidence was present and distinctive. Transformation logic bugs were harder — the model consistently identified the right general category but misattributed the specific change that caused the problem, particularly when multiple changes had been made close together. Overlapping-signal scenarios were the hardest — cases where two different causes occurred close in time produced the highest rate of confidently wrong explanations.

That last finding is the one that qualitative review would never have surfaced. The model's expressed confidence didn't correlate with its accuracy — it was most confident in the cases where it was most wrong. Without the eval harness measuring against ground truth, that pattern would have been invisible.

The practical implication for enterprise AI deployment

For teams deploying LLM-assisted tools in enterprise contexts — particularly tools that influence how people investigate problems, triage alerts, or make routing decisions — the eval harness question to answer before production deployment is: Have we measured accuracy against cases where we know the right answer, or have we only reviewed whether the outputs seem reasonable?

If the answer is the latter, the tool has been tested for fluency and coherence but not for correctness. Those are different properties. For tools that shape business decisions, correctness is the one that matters.

Building the synthetic ground truth dataset is the hard part and the part most worth investing in. It forces you to define precisely what "correct" means for your specific use case — which turns out to be a useful exercise independent of the evaluation itself. The scoring function and the harness infrastructure are relatively straightforward once you have that definition. Without it, you're measuring something other than what you're trying to guarantee.

Arun Mishra is an enterprise architect.

Agent frameworks for developers are still at an early stage, with the likes of Vercel’s eve and Fred Schott’s Flue — both launched this year — setting the early template.

Schott is the creator of the web framework Astro, which led to his company being acquired by Cloudflare in January. He’s just released version 2 of Flue, its first stable release, which has as its foundation React-style “Agent Hooks.”

In Flue, an agent is represented by a JavaScript function. This function “re-renders on every turn,” meaning before every model call.

The addition of hooks came after Schott realized that React’s composability would be a great fit for agent development.

“I originally tweeted that we were building the Astro for agents or the Next.js for agents,” he told us. “But then I realized: maybe no one has even built the React for agents.”

Editor’s Note: we last talked about the React for Agents with Bret Taylor, CEO of Sierra and Chairman of OpenAI:

“We’re still trying to figure out who the reactive agents are and the jury is still out… We’re sort of in the jQuery era of agents, not the react era.”

Hooks are authored in TypeScript. According to the Flue 2 launch post, they “let you build dynamic agents that can manage their own state, listen to agent lifecycle events, and even attach different resources and capabilities dynamically to enhance themselves at runtime.”

There are 16 built-in hooks in Flue 2, including useSkill(), useTool(), useSubagent(). You can also add custom hooks.

What hooks open up for developers is that they make an agent much more dynamic, by allowing its configuration to change as a conversation or workflow progresses. Schott said this is needed to build “real support bots, real triage bots,” because they can’t be fully configured in advance. The agent can’t just be static — it has to adapt in real-time to what the user wants or the situation demands.

Agent hooks bring those capabilities to Flue. For example, a support agent might bring in an account management tool after first verifying a user.

File based magic is an antipattern

Schott’s thinking about how to build an agent framework has evolved rapidly since he publicly launched Flue 1 in early May. Initially, he wanted to take existing web framework concepts and apply them to his new agent framework. He uses file-based routing as an example.

“So we kind of naively ported that over to Flue, thinking — great, well, I’ll put your five agents in these five files, and that’ll be the five routes that they expose. But for a lot of people building with Flue, especially the bigger customers, their whole company is one agent. They don’t care about routing. There’s one agent.”

So after the first Flue users showed these early patterns, composability became front of mind for Schott. That led him back to React.

“As you can see from the Flue 2 API, we’re taking it more from React [...] than we are from Astro or Next.js — where it’s less about routing and these website concepts and more about, at its base level, how do you compose an agent on many different things?”

Flue’s central proposition: agents need a harness

A key concept in Flue is that an agent must have a harness — meaning that it’s in an environment where it has access to the context and capabilities needed to accomplish various tasks.

“Instead of you and your code driving the LLM and telling it what to do with scripts, you’re putting the agent into this harness, and it is able to drive itself and work through problems,” explained Schott.

Flue is built on top of Pi, an open source minimal harness. Essentially, Flue is an opinionated take on Pi — adding features that Schott thinks are helpful to developers building agents. For example: hosted agents in Flue 2 are now built with Vite, an open source build tool.

Indeed, Schott likens Pi’s role to the foundational role that Vite now plays beneath Astro.

“I think Pi can serve that role, where it’s the right abstraction — it doesn’t do too much, but it gives the right APIs that then we can go and say, well, let’s have an opinionated take on this that does more.”

Building on Pi meant committing to having a built-in agent harness.

“Our early bet was that the harness is actually not a feature, but it’s fundamental to what you think an agent is,” Schott said. “There is no agent without a harness.”

Building Flue agents with coding agents

The Flue project began earlier this year within the Astro repository, as an issue-triage system. At first, it was an LLM-driven script or workflow reviewing issues. But then, explained Schott, it gained the ability to take actions in the repo.

“It started to transition from just automation in a repo to wanting to take the Claude Code experience, make it headless, make it hostable and run it in the cloud.”

So that’s when the idea of a harness as anchor emerged. Indeed, in his v1 launch post in early May, Schott described Flue as “like Claude Code, but 100% headless and programmable.”

I myself tested out Flue using Claude Code, which guided me through setting up my first Flue agent. And Schott confirmed this is how many developers use Flue.

“We very much are building for them,” he said, regarding AI coding agents. “Our whole onboarding flow is that, you know, pass this prompt to your agent, it’s gonna guide you through it. All of our docs have markdown support.”

Where Flue fits in the agent development stack

The closest comparison to Flue is Vercel’s eve, which also treats the harness as foundational. Vercel and Cloudflare have been known to beef in public, but Schott is generous in his opinion of eve.

“Eve, I think, is the most directly competitive,” Schott said. “It came around at the same time, so it had that same take that a harness is built-in.”

Schott also referenced what he called the “OG agent frameworks,” which came before Flue and so weren’t created with a harness as the central concept. He listed Vercel’s AI SDK, Cloudflare’s Agents SDK, and Mastra (developed by the same team that built Gatsby, a web framework predating Astro).

While these “OG agent frameworks” are all adding harnesses now, Schott considers that an added feature — whereas Flue and eve both have built-in harnesses.

I asked where Flue sits compared to emerging “meta-harnesses,” like Databricks’ Omnigent and perhaps even the self-improving Exo harness.

Note: we’re also publishing our interview with Exo coauthor Alex Krentsel this weekend; it’s worth a watch and has a bonus discussion on OpenClaw architecture!

Schott rightly noted that there’s confusion about what the term meta-harness even means at this early stage. Regardless, he thinks having one API for working across all harnesses would muddle the story for Flue. His framework specifically defines how skills work in Flue, how subagents work, and so on. As he put it, “the framework [Flue] and the harness are very intertwined.”

He personally finds the meta-harness discussion fascinating, and has played with Exo, but says it’s “a different interest scenario that isn’t really related to hosted agents.”

The Cloudflare connection

Throughout the interview, Schott referenced being able to take advantage of his employer Cloudflare’s tooling and infrastructure. But he was also very clear that Flue is an “open source framework for every host,” as he put it, and he wants it to stay that way.

“The best tools are the ones that float above the host,” he said. “That opens the door for the most developer adoption and the most innovation.”

Host portability is one of Flue’s defining principles — and perhaps that’s where the fundamental difference to Vercel’s eve is. While eve can also be self-hosted, it is optimized to take advantage of Vercel’s many features. Of course that’s a known playbook of Vercel, which does the same thing with Next.js.

All that said, Vercel itself has shown that a Flue agent can be deployed on Vercel. So the two companies can play nice together.

I also mentioned LangChain’s new Managed Deep Agents offering as an example of hosted agent platforms coming onto the market. However, Schott said a managed agents product is not currently on Flue’s roadmap.

“It’s so early for us, we’re just focused on building the best harness,” he said.

Links to find Flue and Fred online; Richard is at @ricmac. This is a new written interview series we are trying out for subscribers — let us know your feedback!

The post This Week’s Awesome Tech Stories From Around the Web (Through August 15) appeared first on SingularityHub.

Multi-tenancy has moved in one direction for 60 years: the tenant keeps getting smaller. Mainframe time-sharing carved a single machine into slices so an organization’s departments could share it, and the tenant was the org. Virtualization gave each team its own fleet of virtual machines, and the tenant became the team. Containers and Kubernetes namespaces shrank it again, until a platform team could hand every developer an isolated environment on a shared cluster.

That last step, an environment per developer, became the target state of platform engineering in the 2020s. A namespace per developer, capacity planned by seat, golden paths sized to headcount. Underneath all of it sits one assumption: a person produces one stream of work at a time, so isolating people isolates work.

Coding agents broke that assumption. A developer running five agent sessions has five changes in flight at once, each needing its own working version of the system. Anthropic’s engineers, building a C compiler with a fleet of parallel agents, ran nearly 2,000 Claude Code sessions across two weeks. Cursor’s documentation tells developers to run as many agents as you want in parallel. None of those concurrent workstreams is a person.

“The tenant has shrunk one more time. It is no longer the developer. It is the change.”

The tenant has shrunk one more time. It is no longer the developer (or even the agent). It is the change.

Tenancy demand scales with changes in flight, not headcount

Capacity planning by seat worked because changes arrived at human pace, roughly one per developer at a time. That denominator is gone. A Microsoft study of command-line coding agent adoption found that developers merged roughly 24% more pull requests over four months, and merged pull requests understate the pressure. Every change that reaches merge is preceded by iterations and abandoned attempts, and each of those also needed somewhere to run.

Run the seat math against the change math. A 50-developer organization where each engineer supervises a few agent sessions has hundreds of changes in some stage of validation on a busy day. Each one wants data it can migrate and write to without asking permission, its own view of shared message topics, and a running version of the services it touched. That is the demand of a 300-person (or more) engineering org on a 50-tenant platform. 

Every layer built on the person-tenant assumption misprices this. A per-developer namespace hands one tenant slot to what is now five concurrent workstreams. Shared staging serializes all of them into a single queue. Seat-based capacity plans budget for the number of employees while the bill tracks the number of changes in flight.

The new tenant is the change, not the agent

The tempting candidate for the new tenant is the agent, and it is the wrong one. Agents are interchangeable workers. Two agents can collaborate on one change, one agent can rotate through five changes, and a crashed agent gets replaced mid-task without anything downstream noticing. Give each agent its own environment, and you have repeated the old mistake at a new scale: isolating workers when the thing that must not leak is work.

“Give each agent its own environment, and you have repeated the old mistake at a new scale: isolating workers when the thing that must not leak is work.”

The durable unit is the change. It comes into existence when work on it starts. It accumulates state that no other tenant should see: a schema migration, test writes, new versions of one or two services, the messages it produced during validation. It needs to observe a version of the system that includes its own edits and nobody else’s. And it is torn down when it merges or is abandoned, taking all of that state with it.

Naming the change as the tenant turns a vague scaling problem into a design target, because change-level tenancy has three requirements that person-level tenancy never had to meet:

• Creating a tenant must be near free.
• Isolation must cover only what changed.
• The tenant’s lifecycle must be bound to the change itself, not to a ticket or a timer.

Platform teams already run this playbook in production

The discipline these requirements call for is not new. Anyone operating a multi-tenant production service already knows the rules: tenants share the substrate, each tenant privately owns only what makes it distinct, creating a tenant is self-service and cheap, and a tenant’s resources are reclaimed the moment it leaves. Nobody stands up a private copy of the product per customer, and nobody files a ticket to onboard one.

Those same organizations run pre-production on the opposite rules. Environments are provisioned by ticket or by seat, capacity is planned per person, and isolation is achieved by duplicating the stack when it is achieved at all. The multi-tenancy playbook that runs the product has never been applied to the platform that builds the product.

Change-level tenancy is that playbook, applied. Treat every change as a tenant of the development platform, and the three requirements stop being novel. They are the standard properties of any competently run multi-tenant system.

The tenant owns what changed and shares everything else

A SaaS tenant owns its data and configuration, never a copy of the application. A change tenant is sized the same way. It owns the one or two services it modified and an isolated database branch it can migrate and write against, and nothing else. Everything the change did not touch resolves against one shared stable environment, continuously deployed from main, so every tenant validates against real, current dependencies without owning a copy of them.

A footprint that small makes tenant creation nearly free, and creation cost is what decides whether the model scales to agent demand. Isolated data no longer requires copying a database: Neon and Xata create copy-on-write branches in seconds regardless of dataset size, consuming storage only for the data that diverges. The runtime side costs one deployment, because starting the modified services is all that is left to do. A tenant that costs one deployment can be created hundreds of times a day.

Tenants onboard and offboard themselves

Multi-tenant platforms scale because nobody provisions tenants by hand. Signup creates the tenant, cancellation removes it, and no operator sits in the loop. Change tenants need the same contract. The tenant comes into existence when work on the change starts and disappears when the change merges or is abandoned, with no ticket at the front and no cleanup script at the back.

Offboarding is the half that platform teams underestimate. At person scale, an orphaned environment was a minor waste found in a quarterly cleanup. At change scale, orphans accumulate as fast as agents abandon experiments, and the leak outgrows the cleanup.

Automatic offboarding also keeps the accounting accurate. When tenants are created and destroyed by the change’s own lifecycle events, the number of live tenants equals the number of changes in flight, and platform capacity becomes a quantity you can measure and plan against instead of a pile of environments nobody is sure anyone still uses.

Re-measure the platform in changes, not seats

The practical shift for platform teams starts with measurement. Count changes in flight at peak, not seats: open pull requests with activity in the last day is a fine proxy, and for most teams the number is already several times headcount. Then price the marginal tenant: What does one more concurrent change cost in dollars and in minutes of setup? If the answer is a full environment and tens of minutes, the platform is still doing person-level tenancy.

Those two numbers expose where the old assumptions live. Namespace quotas sized per developer, staging booked by team calendar, database seeds refreshed nightly for everybody at once: each is a seat-denominated policy waiting to fail under change-denominated load. The fix in every case is the same three requirements: near-free creation, isolation sized to the change, lifecycle bound to the change, applied to whichever part of the platform still assumes the tenant is a person.

Change-level tenancy is the prerequisite for an agent-native SDLC

Every previous definition of the tenant named a person or a group of people, and that held because only people produced changes. A platform could equate one seat with one workstream, plan capacity from the hiring plan, and keep a human in the provisioning loop. Coding agents break all three of those properties at once: one person now operates several concurrent workstreams, those workstreams are created and abandoned at machine pace, and no human is positioned to provision or clean up each one.

“The change is the only unit of isolation that stays stable when the workers become software.”

That is why the software development lifecycle (SDLC) needs its tenant redefined around the change rather than around whoever, or whatever, wrote the code: the change is the only unit of isolation that stays stable when the workers become software. 

Organizations that keep person-sized tenancy will watch agent-generated changes queue behind infrastructure built for a fraction of the load. The ones that re-platform around the change will convert agent throughput into merged work. If you’re exploring the second path, that is exactly what we built Signadot to support.

The post Per-developer environments were the goal. Agents moved the goalposts. appeared first on The New Stack.

Substack recently launched its AI detector feature in the UI, which is super interesting.

Separately, lots of people asked me about interesting local do-it-yourself LLM projects as demos to show what small language models (SLMs) are capable of.

Putting one and one together, I thought it would be interesting to show how an AI detector can be implemented. I will also use it as a verifier to train a small language model to produce text that avoids detection. This is a small educational project for studying the limitations of AI detectors and exploring a verifier-based LLM application beyond regular reasoning models trained on math and code.

So, as mentioned above, the intended goal of this tutorial is to explain how AI detectors work by building (a simple) one.

In practice, such a detector can be used to filter out spammy content, but also to potentially improve your personal writing without turning it into AI-generated text. For example, if you wrote a lengthy article and want to improve spelling and grammar, it is tempting (and actually useful) to use a grammar checker to polish it and improve readability. There are different services for that, including general-purpose LLMs like ChatGPT. However, this also runs the risk that these tools turn your writing, even though it’s still your own writing, into something that is then overpolished and now sounds like AI and gets flagged as spammy content.

For example, with an AI checker, one could say, “Fix my grammar while ensuring that my text still scores 0% AI-generated.”

Anyway, while we are building a fully functional checker here, the goal is to explain 1) how AI checkers (can) work and 2) use this as a case study for a more general topic on how to build a scorer or verifier that can be used with LLMs.

Disclaimer: AI checkers are essentially a cat-and-mouse game. AI checkers may learn to detect a certain pattern that is indicative of AI-generated content. Then, the next LLM may incidentally or deliberately not exhibit that pattern and avoid detection. The AI checker then has to be updated to detect said LLM, and so forth. Plus, it’s also likely to encounter false positives (human written text flagged as AI-generated), but more on that later.

Project goals

There are several goals of this project. The overarching goal is, of course, to illustrate how AI detectors work and show an applied end-to-end LLM project including evaluation, training, and local deployment for real-world use.

The outcome of this is an AI-detector API that can be used by humans and agents, and a user-friendly UI.

Method overview

Here, we are going to develop a method similar to Pangram models, which, as far as I know, are behind Substack AI detection feature.

I wrote a short article about AI-text detection a while back in 2023: What Are the Different Approaches for Detecting Content Generated by LLMs Such As ChatGPT? And How Do They Work and Differ?

In essence, there are different ways to detect AI-written text, from supervised classifiers and perturbation-based probability tests to perplexity measures and watermarking.

In this tutorial, we will build a model that returns a 0-100 score. It’s essentially a classifier with an estimated probability score. The probability score will denote how likely a text is AI-generated according to the classifier. (Or, to be precise the score is the classifier’s estimated probability for the AI-generated class based on its training distribution. However, we shouldn’t interpreted it as a general probability that the text was written by AI.)

For this, we are going to fine-tune a DistilBERT classifier (similar to what I described in one of my early Substack articles, Finetuning Large Language Models), but more details on that later when we get to that stage.

Read more

When Twitch announced that streamers could opt out, thousands of users questioned why their content was being used to train AI models in the first place.

The United States Patent and Trademark Office (USPTO or Office) is seeking information to consider regarding the selection of locations for one or more community outreach offices (COOs) in the agency's Southwest Region, which comprises Arizona, Texas, New Mexico, Oklahoma, Kansas, and Missouri.

The Food and Drug Administration (FDA, the Agency, or we) is issuing a final order reclassifying in situ hybridization (ISH) test systems indicated for use with a corresponding approved oncology therapeutic product (product codes NYQ, MVD, OWE, and PNK), all postamendments class III (premarket approval) devices, into class II (special controls), subject to premarket notification. FDA is also establishing a new device classification regulation, along with the special controls that are necessary to provide a reasonable assurance of safety and effectiveness of this device type.

Chinese AI startup Z.ai, known internationally for its growing lineup of powerful, largely open source GLM series of language models, today released GLM-5.3 with substantial gains in long-horizon coding and a more consequential — and potentially sensitive — jump in cybersecurity capabilities.

Already, GLM-5.3's cyber capabilities have found a "potentially serious vulnerability in Cursor," the AI coding startup recently acquired by SpaceX, according to z.ai developer advocate Lou, posting on X. VentureBeat also tagged Cursor for confirmation on X and is awaiting response.

GLM-5.3 is available initially only through the company's GLM Coding Plan and ZCode coding environment, while API access and open weights are coming later, "once safety evaluation and hardening are complete," according to the company.

Z.ai says it plans to release weights approximately two weeks after launch.

For enterprise developers, the notable part of the release is not simply another round of benchmark improvements. Z.ai says GLM-5.3 uses the same base model as GLM-5.2, with the improvements coming entirely from scaling post-training across more environments, more diverse tasks and additional reinforcement-learning compute.

That makes GLM-5.3 something of a test of how far a frontier-scale base model can be pushed without another expensive pretraining cycle.

“Scaling post-training is all we did for GLM-5.3,” Z.ai wrote in its technical announcement.

The results suggest considerable headroom. But they have also produced an unusual problem for an open-model developer: according to Z.ai, cybersecurity capabilities improved faster than anticipated as training scaled, particularly as tasks progressed from vulnerability identification toward constructing complete exploitation chains.

Reuters reported Friday that Z.ai is also introducing controls around some of the model's more advanced capabilities, including a “trusted access” approach for sensitive functionality.

A large jump in coding without another base model

GLM-5.3 builds on the 743-billion-parameter-scale base model behind GLM-5.2 rather than replacing it. Z.ai instead expanded the post-training system it had already assembled around long-horizon reinforcement learning.

Those environments increasingly resemble complete engineering jobs rather than isolated programming exercises.

Z.ai describes scenarios in which an agent receives access to codebases, documentation, compute clusters, storage systems and experimental results, then has to diagnose problems, modify systems, run experiments and demonstrate a measurable improvement while preserving correctness. Some tasks are designed to approximate several days of work for an experienced engineer.

The approach produced sizable generation-over-generation improvements on Z.ai's reported evaluations.

GLM-5.3 jumps from 4.6 to 28.3 on Terminal-Bench 3.0, from 46.2 to 66.9 on DeepSWE v1.1, and from 26.2 to 48.2 on AutomationBench. On Agents' Last Exam CLI, it improves from 23.8 to 28.5.

The model does not dominate every frontier competitor. Z.ai's own benchmark table shows GPT-5.6 Sol at 34.6 and Claude Fable 5 at 33.7 on Terminal-Bench 3.0, compared with GLM-5.3's 28.3. On DeepSWE v1.1, GLM-5.3 scores 66.9, compared with 72.7 for GPT-5.6 Sol and 69.7 for Fable 5.

But Z.ai is also emphasizing efficiency rather than benchmark position alone.

On its private Z.ai Code Bench, GLM-5.3 reaches a 34.5% result at its Max reasoning setting while consuming roughly 75,000 output tokens per task. GLM-5.2 reaches 23.4% while consuming approximately 96,000. At High effort, GLM-5.3 reaches 31.4% at roughly 50,000 output tokens, compared with Z.ai's reported 29.5% for Claude Opus 4.8 using 120,000.

Because Code Bench is Z.ai's own private evaluation, those comparisons should be treated as company-reported results rather than independent measurements. Still, reducing token consumption while improving task completion is operationally important for enterprises deploying coding agents, where long-running loops can make inference cost and latency compound quickly.

Cyber capabilities developed faster than Z.ai expected

The more unusual development is cybersecurity.

Z.ai introduced vulnerability-discovery environments into GLM-5.3's post-training mix expecting the model to improve at finding software flaws. Instead, the company says capability began progressing further along the exploitation chain.

“As we scaled post-training, cyber capability developed faster than we expected,” Z.ai wrote.

On CyberGym, which tests vulnerability discovery and validation against source code, GLM-5.3 scores 84.5%, compared with 77.2% for GLM-5.2. That also edges Z.ai's reported scores for GPT-5.6 Sol at 83.6% and Mythos 5 at 83.8%.

The advantage does not extend across the entire exploitation stack. GLM-5.3 scores 54.4% on ExploitBench, more than twice GLM-5.2's 24.4%, but remains well behind the 76.5% Z.ai reports for GPT-5.6 Sol and 78% for Mythos 5.

Similarly, on ExploitGym, GLM-5.3 completes 105 tasks under a normalized two-hour budget and 130 under six hours, up from 29 and 39 for GLM-5.2. Fable 5 reaches 181 and 247, while GPT-5.6 Sol reaches 216 and 293.

The direction of travel may matter more than the leaderboard position.

Z.ai says work with security teams in China has resulted in 2,436 vulnerability findings across 269 projects after expert review, screening and deduplication. Its disclosure ledger lists 1,097 as critical or high severity, with 53 publicly disclosed and 2,383 still under embargo at the time of the release.

That creates a tension increasingly facing frontier model providers: the same long-horizon agent capabilities that make models more useful for software engineering can also make them more capable security researchers — and potentially more capable offensive operators.

GLM-5.3 also requires developers to change how they call the model

Developers migrating existing GLM applications should pay attention to a breaking API behavior.

GLM-5.3 supports three reasoning-effort levels — low, high and max — with max the default and Z.ai's recommended setting for coding. But unlike previous releases, thinking cannot be disabled.

Applications currently sending thinking.type: "disabled" must change the value to enabled and specify a reasoning effort before switching the model identifier to GLM-5.3. Otherwise, Z.ai says the request will fail.

That makes GLM-5.3 an actual migration rather than simply a model-name substitution for some production applications.

From GLM-4.5 to GLM-5.3: Z.ai's rapid push into agentic engineering

GLM-5.3 is the latest step in a rapid shift by Z.ai — formerly known as Zhipu AI — toward coding agents and long-running autonomous engineering workloads.

GLM-4.5, released in July 2025, established much of that direction. The 355-billion-parameter mixture-of-experts model was designed to combine reasoning, coding and agent capabilities, while the smaller GLM-4.5-Air offered 106 billion total parameters. Z.ai released the models with open weights and emphasized integration with agent frameworks.

GLM-4.6 followed in September, expanding context from 128,000 to 200,000 tokens and targeting coding, tool use and agent workflows in environments including Claude Code, Cline, Roo Code and Kilo Code. Z.ai also began placing greater emphasis on token efficiency in real-world coding evaluations rather than benchmark performance alone.

The larger architectural jump came with GLM-5 in February 2026. Z.ai scaled the model from GLM-4.5's 355 billion parameters to 744 billion, with 40 billion active parameters, and increased pretraining data to 28.5 trillion tokens. It also introduced its “slime” asynchronous reinforcement-learning infrastructure and explicitly repositioned the GLM family around “agentic engineering” and long-horizon tasks.

By June, GLM-5.2 had turned that strategy into a more direct enterprise proposition. The 753-billion-parameter model arrived with a stable 1-million-token context window, open weights under an MIT license and support across more than 20 coding environments. It also introduced IndexShare, which reuses an indexer across sparse-attention layers to reduce the computational burden of very long contexts.

GLM-5.2 was priced at $1.40 per million API input tokens and $4.40 per million output tokens, with cached input priced substantially lower, positioning Z.ai as both a technical and pricing competitor to proprietary frontier labs.

Z.ai's ambitions have been expanding outside model development as well. Reuters reported last month that Zhipu AI raised roughly HK$31.4 billion, or about $4 billion, through a Hong Kong share sale, with proceeds intended for areas including research and development, computing infrastructure, talent and business expansion.

Taken together, the releases show a consistent progression: GLM-4.5 unified reasoning, coding and agents; GLM-5 substantially scaled the foundation model; GLM-5.2 attacked long-context and long-horizon engineering; and GLM-5.3 is now attempting to extract substantially more capability from that same foundation through post-training.

Pricing, ZCode and availability

GLM-5.3 is available now through Z.ai's GLM Coding Plan and ZCode.

ZCode is the company's own coding-agent environment and supports long-running “Goal” tasks that plan, implement, test and verify work. It also offers remote control of running tasks and is available on macOS, Windows and Linux.

Individual GLM Coding Plans currently start at a listed promotional price of $12.60 per month for Lite with 10,000 credits per week. Pro is listed at $56 per month with six times Lite usage, while Max costs $117.60 per month with 14 times Lite usage. Team Standard and Premium seats are listed at $88 and $188 per user per month, respectively.

Z.ai has also moved the Coding Plan to a points-based quota system that separately accounts for input, cached-input and output tokens. Calls outside the company's weekday peak period consume 50% of the normal points.

The company has not yet provided general GLM-5.3 API pricing in the supplied launch materials, making total production API cost difficult to compare directly with GLM-5.2 or competing frontier models until staged API access arrives.

That staged release may ultimately be the most important part of GLM-5.3.

Z.ai spent the past year pushing an open-model strategy centered on permissive weights, low-cost inference and compatibility with existing coding-agent ecosystems. GLM-5.3 demonstrates what happens when that strategy succeeds perhaps too well in one sensitive domain: better autonomous engineering also means better autonomous security research.

The result is a model that advances Z.ai's coding ambitions while forcing the company to confront the same capability-versus-access tradeoff facing the largest closed frontier labs.

For enterprise developers, GLM-5.3 is therefore worth watching for two reasons. Its coding results provide another indication that increasingly capable agents can emerge from better post-training and environments without continuously rebuilding the underlying foundation model. Its cybersecurity results show why deciding how those agents are distributed may become just as important as deciding how they are trained.

The following article originally appeared on Addy Osmani’s blog site and is being republished here with the author’s permission. Technical debt lives in your code. Cognitive debt lives in your head. Intent debt lives in the artifacts you may never have written: the goals, constraints, and rationale for why the system is the way it […]

Housekeeping: I’m traveling so cannot make a voiceover for this post. EDIT — I added a bullet point 5 on the Chinese data industry after sending the email out.

Today, Z.ai announced their GLM-5.3 model, currently only available in the coding plan, coming soon to their API and in two weeks’ time to Hugging Face (open weights). This model looks exceptional, with a somewhat astounding increase in scores. On many benchmarks the model has surpassed Moonshot AI’s Kimi K3 and on some it’s surpassed Claude Fable 5 or GPT-5.6-Sol.

Here’s a more complete comparison:

This puts the model more or less at the frontier of agentic coding benchmarks, with only ~750B parameters – a third of Kimi K3! The Z.ai blog post is rather straightforward, and starts with a bold sentence:

Scaling post-training is all we did for GLM-5.3.

GLM-5.3 is the same base model as GLM-5.2 with substantially extended post-training. To risk a broad oversimplification, Z.ai seems to have a strength in post-training when compared to Kimi, which is more of a pretraining masterpiece. Following this release there have been a lot of discussions wondering how China can keep up so well? How can such a small model be matching the leading public American models? Are these results real?

Subscribe now

The simplest explanation is that Z.ai is very good at what they do – it’s worth recalling that they’ve been working on this line of models longer than almost anyone in the industry. Here’s a brief history of the GLM models.

• Zhipu AI Founded – 2019

• GLM (General Language Model) — March 2021 — released by THUDM, Tsinghua University’s Data Mining / Knowledge Engineering group. Weights

• GLM-130B — August 2022 — Scaled version. Technical report for GLM-130B through GLM-4 — Weights

• ChatGLM — March 14, 2023 — first chat version. Weights

• ChatGLM2 — June 25, 2023 — Weights

• ChatGLM3 — October 27, 2023 — Weights

• GLM-4 — January 16, 2024 — rebranded as just GLM; open-weight GLM-4-9B followed in June. Weights

• GLM-5 — February 11, 2026 — latest major generation. Weights

GLM 5.2, released on June 22 of this year, was a big deal – weeks after the release, I regularly heard from AI researchers I know who still used the model due to its speed (some deploy the model on internal clusters for faster speeds than public offerings) and simplicity (as a model with no rollbacks, etc., when working on frontier AI systems). GLM-5.2 altogether stood up to the hype.

I’ve been going through some of the same denial myself, thinking “how do they keep doing this? Surely the models aren’t as good as they look.” There’s something a bit off-putting with how the American companies have such a commanding resource lead, but can’t seem to pull away in capabilities. The common answer is distillation, which I’ve written at length about, but I deem not to be the major factor. On that note, there was a recent paper that showed simple methods for extracting the reasoning traces from frontier models – this is the sort of thing that Chinese labs could definitely use at scale. I’m confused why the labs in the U.S. haven’t patched this behavior faster; instead they’re running to the government asking for policy help. It doesn’t add up for me.

Z.ai’s blog is direct and matches with an RL-dominated training regime. They say they used “more environments, more diverse tasks, and more compute spent training on them.” One does not simply “distill” RL environments, infrastructure to run them at scale, or algorithms to mix them together effectively.

So, how do the Chinese labs do it if not distillation? Are they benchmaxxing? An accepted definition of benchmaxxing is focusing the model on the test sets, such that the real-world performance meaningfully differs from the on-paper scores. The determining factors are much more big picture than technical (yes, the technical details definitely matter, but are harder to differentiate from lab to lab):

1. The time to release for Z.ai is likely days, not months as with OpenAI or Anthropic. It is very, very likely that OpenAI and Anthropic have far better internal models than Z.ai and Moonshot AI. Still, these American companies tend to take months to release their models to the public, which massively flatters the Chinese labs in adoption decisions at the frontier. To put it simply – the Chinese labs use all the time that American labs do pre-release testing to keep hillclimbing on benchmarks (SpaceXAI is likely far closer to the Chinese labs here). With the pace of progress being so fast, this is likely the largest determining factor of why Chinese labs stay at the frontier. This, so far, has been economically acceptable for the American labs, as they’ve still had massive demand for their models.

   
   As model self-improvement loops ramp up within the labs building LLMs, if any of these feedback loops require user data, this faster release cycle could massively favor the Chinese labs, giving their offerings longer lifespans before the next vastly superior model comes out, undercutting demand for their models.

   
   These are very clearly the race dynamics that many in the industry worry about. With so many labs building frontier models in the envelope of leading capabilities, it is hard to see this abating in the near future.

2. Yes, Z.ai probably cares slightly more about public benchmarks than OpenAI or Anthropic. These benchmarks, e.g. scoring highly on the Artificial Analysis Intelligence Index, or similar aggregators, have a very direct impact on their stock price. They in many ways need to do this to keep raising capital and maintain team morale, as being the scrappy underdog matching American giants is a wonderful story.
   
   Subtle benchmaxxing does not need to come out of desperation or any similar pressures. It’s the industry standard across a remarkable number of labs. Many companies’ data acquisition strategy is to buy data on the benchmarks they’re behind on.

3. Z.ai is not benchmaxxing to the point where GLM-5.3 is fried (at least not intentionally, and they’ll check for it). Every lab is dealing with the rough edges of scaling RL right now. Anthropic’s Opus 5 and Sonnet 5 models have very mixed reputations, despite the incredible benchmark scores. Everyone in the industry is in the same boat, so some model weights end up being easier to use than others, but the benchmark scores in their release blogs are the real deal.

4. GLM-5.3 is likely a narrower model than Claude Fable or GPT Sol. When GPT-5.2 was released, it had mixed reviews outside of agentic coding. At the same time, OpenAI and Anthropic support very large businesses with countless use-cases for their models. This is a benefit of being a company earlier in their adoption curve – you can target the most valuable use-cases. Within post-training, caring about a bit less will make assembling the final model far easier.
   
   I’m overstating this a bit, as Z.ai reportedly reached $1B of ARR on the back of a strong on-premises deployment business.

   
   Similarly, the flagship GLM models have not had visual capabilities. Being text-only definitely helps Z.ai get more competitive scores, but it is a more competitive space. On the other side of things are models like Inkling-Small, which is designed to be omnimodal.

5. (ADDED) The RL data industry is taking off in China. Many sources and rumor-mills we’re following have been mentioning how the data industry is taking off in China — very much driven by American data companies selling to Chinese model labs. This could look like Chinese labs buying many of the same RL environments that are used by American frontier labs, and releasing the downstream RL’d model sooner. We still have large error bars on the scale and impact of this market, but it is certainly becoming important.

6. Z.ai is an extremely skilled LLM organization – one that is likely far more compute efficient than OpenAI / Anthropic. This needs repeating. These folks are very good at what they do. The company has very close ties to Tsinghua University, which is home to many of the best Chinese computer scientists. This abundant, eager talent pool is as central to their success as it is for any Western counterpart.

Altogether, it seems like a perfectly good strategy they’re executing with the GLM line of models. Congrats on the release! I’m excited for the weights to be out so I can do more extended testing (I tend to use American open-weight inference services like Fireworks or Baseten).

Leave a comment

This is another step towards the inevitable proliferation of very strong cyber capabilities across the economy. Z.ai has acknowledged this, saying:

GLM-5.3 is our most capable model to date for cybersecurity tasks. It delivers substantial improvements in vulnerability discovery, exploit analysis, and complex multistep security tasks. These capabilities can help defenders identify weaknesses earlier, validate risks, and accelerate remediation.

They also create clear dual-use risks. We are therefore taking a staged approach to release. Selected security partners will first evaluate GLM-5.3 in controlled settings. Broader access and API availability will follow. Once the necessary safety evaluations and release preparations are complete, we will publish GLM-5.3’s complete model weights.

They go on to acknowledge how they’re monitoring inference on their platforms via a request classifier and chain of thought monitoring (on top of model alignment). The devil is in the details here, and it is unclear the level of execution every AI lab will have here. The capability diffusion is determined by the lowest common denominator.

At the end of the day, this type of safety barely matters when true open-weights are coming. If not GLM-5.3, then another model. The size of the models with these capabilities is reducing over time, becoming easier to modify and deploy (potentially without safeguards). Z.ai does some of the right things, including pushing for more vulnerability discovery and proactive management, but any single company is far from being able to handle this on their own.

We need industrial-scale guidance led by the government or industry coalitions to immediately prepare for this transition across all software.

The Unitree G1 has found online fame as a relatively affordable robot that can charm a crowd. But can it ever hold down a real job?

This article was originally published on Undark. Read the original article.

In June, a community of mostly mathematicians released the Leiden Declaration on Artificial Intelligence and Mathematics, an articulation of the values they hope to preserve as automated systems are integrated into the practice of developing mathematical proofs. This is necessary because some frontier AI systems have shown striking capabilities for solving certain advanced mathematics problems, though independent tests show that AI still has important limits.

Although I’m not a member of the pure mathematics community in any strict sense, much of the declaration’s message resonated with me and was relevant to my own research interests as a computational biologist.

I’m encouraged that the mathematics community decided to take a stand on the issue and that it has been successful in organizing a large number of eminent mathematicians to sign the document. The Leiden Declaration, which originated at a conference held at Leiden University in the Netherlands, should spawn proper copycats, because what is true for mathematics is true for virtually every field that calls itself as a science. The inventions of mathematics percolate into the algorithms and statistical methods that help scientists design experiments, build simulations, and analyze data, from sociology to statistical physics and beyond.

I argue that biological fields should consider something of the sort, because the kinds of knowledge that biology generates and predicts are uniquely vulnerable to subversion and mischaracterization by artificial intelligence.

The conversation in the mathematics community has been illuminating, in that the declaration is a coordinated response to the powers and risks of AI, whose acceleration has felt like a Thanos snap, changing the universe in an instant. And part of the reason that mathematicians felt the effects so immediately is tied to the manner in which their research is conducted: A mathematical proof, in principle, is transparent and independently verifiable, and no proprietary equipment is (generally) required to check it.

The Leiden Declaration should spawn proper copycats, because what is true for mathematics is true for virtually every field that calls itself as a science.

As the Leiden Declaration notes, automated techniques now present mathematics with a new forgery problem: Because mathematical truths are fixed and verifiable, one can identify a counterfeit formalism by comparing it to the genuine proof. Biology, however, offers no such guarantee; our so-called “truths” are often noisy and context-dependent, making it nearly impossible to define what an authentic version should even look like. Some of the most widely appreciated biological principles (such as Mendel’s laws of genetic inheritance) are better described as powerful but limited in scope, and with well-characterized exceptions that don’t undermine the laws but refine their application. This is true for many biological theories. Boundary conditions, edge cases, and noise are not bugs but features of how the natural world works.

For example, a mutation that confers drug resistance to a virus with one genetic background may have a much weaker, neutral, or even harmful effect in another, because its impact depends strongly on the surrounding genetic context. This phenomenon, which biologists call epistasis, is not an exotic edge case but a powerful force across the biosphere in shaping the relationship between an organism’s genes and its expressed characteristics. And epistasis is just one of many examples of context dependence in biological systems, in which a finding that holds true in a dish falls apart in a body or has an effect in a mouse model but not in a primate.

When it comes to AI, the mathematician fears producing a counterfeit solution. But the biologist often cannot say, even acting in the fullest good faith, what the authentic version is supposed to look like.

Despite the differences between mathematics and biology, the life sciences should consider embarking on an exercise that is at least analogous to the Leiden Declaration. If nothing else, a biology version could borrow its structure and ambition. We should insist that researchers disclose their use of automated tools, that they are responsible for the veracity of their findings, that credit and accountability belong to people rather than to systems, and that early-career scientists be protected from incentives that prioritize high-volume output over genuine scientific insight.

A biology declaration should adopt these ideas and others, and emphasize additional provisions that are important to the field: the validation of AI-generated hypotheses against results from the wet lab, the management of training data drawn from the biological commons, and the heightened scrutiny owed to any model whose outputs will eventually touch a patient or an ecosystem. The last point is crucial: In the biomedical realm, AI’s missteps and triumphs will manifest in living bodies, with all of the associated corporeal, emotional, ethical, and legal consequences.

We should insist that researchers disclose their use of automated tools, that they are responsible for the veracity of their findings, that credit and accountability belong to people rather than to systems.

A biological Leiden Declaration must appreciate the nature of the data and observation in biology, and other particulars of the field. But the most important feature for responsibly managing the relationship between AI and living systems involves the durability of what is produced.

A mathematical declaration can aspire for permanence because verified proofs can remain valid across centuries. Biological understandings, on the other hand, tend to shift over time, sometimes rapidly. A policy hastily calibrated to the models of this summer might already be miscalibrated by winter. A declaration written in the hope of lasting a decade could risk spending much of that decade catching up.

If we are to orchestrate a responsible treatise for artificial intelligence in the life sciences, it should be adaptive: versioned, dated, revisited on a published schedule, and amended in the open by the very community it claims to represent. And because different subfields of biology have unique challenges—cardiology versus forest ecology, for instance—perhaps we need multiple declarations (but not too many).

The Leiden Declaration incorporates some of these elements, clarifying that its content reflects AI technologies and mathematical practice as of May 2026 and that updates on the document will be shared. Biology should make that feature part of the central architecture, wiring the process of revision into the document, so that updating it becomes a positive action rather than a confession of failure.

Fortunately, life scientists are well equipped for this task. We have long understood that structures unable to change with their environments rarely endure. It would therefore be a strange betrayal of our discipline to write a declaration that forgets this basic principle. Our policies for technological change must keep the dynamism of living systems at the center of how we imagine the future of biology.

The post Biology Needs an AI Declaration appeared first on SingularityHub.

Dutch officials have warned that a high-severity macOS vulnerability that allows attackers to execute malicious code is under active exploitation.

“The NCSC has received a notification indicating that active abuse of this vulnerability has been observed on multiple systems on which port 5900 was accessible from the Internet,” the Netherlands National Cyber Security Centrum warned earlier this week. “In all these cases, root had been accessed on the affected system and a Monero crypto miner had been placed.”

Do you know if your screen sharing is on?

The vulnerability, tracked as CVE-2026-65400, received a patch from Apple last week for macOS Tahoe, Sequoia, and Sonoma. The vulnerability, with a severity rating of 7.1 out of 10, stems from a bug in the macOS screen sharing capability, which allows a remote party to view the screen and control the keyboard and mouse while a machine is turned on. A flaw in the “state management,” which keeps track of preceding events, user interactions, variables, and other system states, is the underlying cause.

Read full article

Comments

© Getty Images

After its cloud storage provider went defunct, a PBS affiliate decided to sue a data center provider to regain access to 50TB of TV shows, videos, and other data dating back 70 years.

As reported this week by Current, a trade newspaper covering public broadcasting, St. Louis affiliate Nine PBS filed a lawsuit against Iron Mountain Data Centers on July 28, seeking access to the data. In the litigation filed in Denver District Court, Nine PBS says that its cloud storage provider, Open Source Storage (OSS), used one of Iron Mountain’s Denver data centers to store the channel’s data. However, OSS is being unresponsive, and Nine PBS says Iron Mountain has refused to release its data.

The data in question includes the station’s coverage of the COVID-19 pandemic, East St. Louis’ history, The Great Flood of 1993, and over 11,000 files, The Denver Post reported in July. The lawsuit claims that “most” of the data is “unique and irreplaceable,” according to the Post.

Read full article

Comments

© Getty

Experts note that the benefits of going public ultimately outweigh those of staying private, even with the vendor seemingly able to raise unlimited financing.