Participatory AI initiatives are meant to bring together diverse stakeholders to design and oversee AI systems that are fair and trustworthy. However, a recent review of 80 participatory AI initiatives revealed that, rather than providing participants with genuine decision-making power, the vast majority consult specific communities on narrow implementation details. 

Meanwhile, every major AI lab now conducts some form of public consultation, the EU AI Act requires stakeholder involvement, and the OECD AI Principles regard stakeholder engagement as a fundamental element of trustworthy AI. Participation in AI governance has become more popular than ever. But an uncomfortable pattern is emerging: the more we discuss participation, the less we discuss meaningful engagement, real influence and power.

Many participatory approaches primarily involve communities during the early stages of the AI system lifecycle: design, data collection, and model development, while later stages, such as deployment, monitoring and system evolution, attract far less attention. During my fieldwork across Kenya, Malawi, and the Philippines, a troubling question keeps surfacing: once the participatory design phase concludes, who truly governs the AI system? Even highly participatory processes often dissolve once the system is launched. Governance then shifts back to those who built or commissioned the system. Communities involved in shaping the design often have little influence over how systems evolve to meet ongoing community needs or how they expand beyond their initial scope.

The old lesson we keep relearning

Back in 1969, urban planner Sherry Arnstein published a deceptively simple insight: not all participation redistributes power. Her “ladder of citizen participation” described eight levels, from manipulation at the bottom to citizen control at the top. She called the middle rungs “tokenism”: processes that perform inclusion without actually transferring authority. Arnstein was writing about urban planning in American cities, but her framework resonates powerfully in today’s AI landscape.

Over fifty years later, AI researchers are revisiting this lesson. Recent studies have introduced the concept of “participation washing”: the act of claiming inclusion without the necessary redistribution of power. Other researchers have documented how participatory rhetoric often conceals the ongoing centralisation of decision-making authority.

These findings paint a consistent picture. Participatory AI, despite methodological progress, largely remains at Arnstein’s “consultation” and “informing” levels rather than reaching the “partnership” or “delegated power” levels. The reasons for this are understandable. Genuine participation is costly, slow, and fosters accountability relationships that complicate rapid development and deployment cycles. Organisations optimising for scale naturally minimise governance complexity. However, this means participatory AI often reproduces the very power imbalances it seeks to address.

The real divide: Methods versus infrastructure

Despite all this, the field has made genuine progress in developing methods to meaningfully involve people in AI design. The repository of tools and metrics on the OECD.AI Policy Observatory showcases participatory frameworks that now guide humanitarian and public sector AI initiatives globally. Methods have become increasingly sophisticated, moving beyond superficial consultation towards authentic co-design. What remains underdeveloped is the infrastructure for ongoing governance after deployment.

Think of it this way. Methods answer: “How do we involve people in design?” Infrastructure addresses: “How do people exercise authority after deployment?”

This distinction matters because AI systems, as the OECD definition emphasises, are adaptive. They are never “finished.” They evolve continuously through new training data, model updates, and deployment expansions. Each evolution requires governance decisions.

AI also relies on collective data. These are not tools that individuals choose to use; rather, they are infrastructure that processes communal information and makes decisions that affect entire populations. Without proper governance, participation during the design phase can inadvertently legitimise systems that centralise power. “We consulted the community” becomes a justification for deployment, even if communities no longer hold any authority over the system’s future.

What commons governance looks like in practice

If the challenge is institutional, what institutional forms could address it? One promising approach draws on principles from natural resource commons. Economist Elinor Ostrom received the Nobel Prize for demonstrating that communities can effectively manage shared resources such as fisheries, forests and irrigation systems. The application of commons governance to knowledge and digital resources has been extensively developed in later work, from Hess and Ostrom’s study of knowledge commons to comprehensive frameworks for analysing knowledge commons governance across various institutional contexts.

Commons governance models have several features that matter for AI: collective ownership, where communities hold rights to the resource; participatory decision making, where rules for usage and development are established through community processes; value sharing, where benefits are returned to the community; and continuous stewardship, where governance continues as long as the system operates.

The work I contribute to involves exploring whether these principles can guide AI system development in resource-limited settings. In Malawi, our research team at NYU and our local partners are creating a voice-based crisis reporting system designed to ensure that community governance councils oversee data practices, model updates, and deployment decisions. In the Philippines, we are collaborating with Kalinga State University on an Indigenous Knowledge Data Collaborative that allows communities to control how their traditional knowledge is digitised and used. The CARE Principles for Indigenous Data Governance, emphasising Collective benefit, Authority to control, Responsibility, and Ethics, offer a powerful framework here. These principles emerged from broader movements around Indigenous data sovereignty and represent a significant way to reshape data governance around community authority rather than external oversight.

While the OECD AI Principles mention data trusts as a mechanism for ethical data sharing, commons models take this logic further. Data trusts typically delegate authority to trustees acting on a community’s behalf. Commons models place community decision-making at the centre. The community does not transfer control to a benevolent intermediary; it retains authority itself.

Initiatives like Mozilla Common Voice and various Indigenous data-sovereignty movements are exploring this path worldwide, yet critical questions persist. Can commons governance operate effectively in resource-limited humanitarian settings? What occurs when commons governance moves too slowly for operational needs? Our fieldwork aims to address these exact questions.

What we are learning and what presents challenges

I want to be frank about the limitations of this work. These are early-stage experiments, not established models. However, they reveal important tensions that the wider field needs to address. If participatory AI governance is to go beyond words, we must be honest about what makes it challenging.

Power redistribution is costly. Governance meetings involve travel expenses, translation services, and participant compensation. In our work in Malawi, these costs are significant and surpass typical AI development budgets. However, this should be viewed not just as an expense but as an investment in risk mitigation. Without this investment, systems in complex environments risk rejection, non-adoption and even obsolescence.

Democracy is slow, and crises are urgent. Emergency responses demand quick decisions. Community governance takes time. The challenge is to differentiate between decisions that truly require rapid centralised action and those that only seem urgent to technical implementers.

“Community” is not uniform. Gender, age, and ethnicity influence who takes part and whose voice carries weight, and commons governance does not automatically resolve representation issues. However, it does make them visible and demands specific mechanisms to address them.

Sustainability remains uncertain because meaningful participation requires ongoing resources; governance cannot be an afterthought funded by short-term grants. It must be incorporated into budgets from the outset and treated as operational expenditure. International development finance models will need to adapt to support this. A system that works brilliantly for two years and then collapses because governance funding runs out is not a success.

What this means for AI policy

These experiments demonstrate methods to strengthen international frameworks for stakeholder engagement.

First, we need clearer distinctions between engagement levels. Consultative engagement involves gathering input to inform others’ decisions. Governance authority means stakeholders exercise binding power over system operation. Frameworks often conflate these two very different concepts. When a government or company says it has “engaged stakeholders,” does that mean it sought feedback or that it shared power? Making this distinction explicit helps implementers understand what they are truly committing to, and helps communities know what to expect.

Second, resource models need to change. Governance infrastructure, including ongoing meetings, capacity building, and conflict resolution, should be seen as a way to protect assets rather than as overhead. Current assessments of AI pilots rarely differentiate between technical and governance factors when a project fails. This complicates the process of allocating resources effectively. Funding structures must recognise that governance is the mechanism that maintains an AI system’s viability and trust over time.

Third, we need governance metrics. The OECD effectively monitors AI policy implementation. But we also need ways to measure the quality of power distribution. Who makes binding decisions about how a system evolves? Just as “technical debt” builds up when code is rushed, “governance debt” accumulates when engagement is overlooked. Eventually, the interest is paid in the form of lost trust or system failure. Recent work on AI accountability frameworks suggests promising directions for creating such assessments.

Finally, community data ownership has an unclear legal status in most jurisdictions. Cross-jurisdictional research on legal frameworks that support collective governance would be very valuable. Experiments such as New Zealand’s Māori data sovereignty frameworks and Barcelona’s data cooperatives offer promising models to learn from. The goal is not to impose universal frameworks but to document what works, what does not, and where the legal gaps are most severe.

Community governance authority as infrastructure

The participatory turn in AI governance represents genuine progress. However, involving people during design without engaging them in governance risks becoming a sophisticated form of consultation that maintains existing power structures. This is exactly what Arnstein warned against more than fifty years ago.

Moving from participation to power involves treating community governance authority as infrastructure: something that requires investment, upkeep, and institutional backing comparable to the technical systems it oversees. It means funding that supports governance alongside technical development. It means metrics that evaluate power distribution, not just engagement processes. And it means honest recognition that meaningful participation is slower and more costly than consultation, but also more likely to produce systems that communities genuinely trust and use over the long term.

Eighty participatory AI initiatives were reviewed, and most of them never moved beyond consultation. That finding should concern anyone who values participation. If we are serious about closing the gap between engagement and authority, we must start building the governance infrastructures to do it. AI remains in its early stages, and there is much we still need to understand. But one thing is becoming clear: if AI is considered infrastructure, it requires governance infrastructure. Otherwise, there is no trustworthy AI.

The international community has made significant progress in defining what responsible AI looks like. The next step is investing in the unglamorous, difficult, and necessary work of establishing governance structures to uphold these standards. This involves supporting communities not only as participants in design but also as equal partners in decision-making.

The post To be truly participative, stakeholder involvement should follow an AI system’s entire lifecycle appeared first on OECD.AI.

Fast takeoff, the singularity, and recursive self-improvement (RSI) are all top of mind in AI circles these days. There are elements of truth to them in what’s happening in the AI industry. Two, maybe three, labs are consolidating as an oligopoly with access to the best AI models (and the resources to build the next ones). The AI tools of today are abruptly transforming engineering and research jobs.

AI research is becoming much easier in many ways. The technical problems that need to be solved to scale training large language models even further are formidable. Super-human coding assistants making these approachable is breaking a lot of former claims of what building these things entailed. Together this is setting us up for a year (or more) of rapid progress at the cutting edge of AI.

We’re also at a time where language models are already extremely good. They’re in fact good enough for plenty of extremely valuable knowledge-work tasks. Language models taking another big step is hard to imagine — it’s unclear which tasks they’re going to master this year outside of code and CLI-based computer-use. There will be some new ones! These capabilities unlock new styles of working that’ll send more ripples through the economy.

These dramatic changes almost make it seem like a foregone conclusion that language models can then just keep accelerating progress on their own. The popular language for this is a recursive self-improvement loop. Early writing on the topic dates back to the 2000s, such as the blog post entirely on the topic from 2008:

Recursion is the sort of thing that happens when you hand the AI the object-level problem of “redesign your own cognitive algorithms”.

And slightly earlier, in 2007, Yudkowsky also defined the related idea of a Seed AI in Levels of Organization in General Intelligence:

A seed AI is an AI designed for self-understanding, self-modification, and recursive self-improvement. This has implications both for the functional architectures needed to achieve primitive intelligence, and for the later development of the AI if and when its holonic self-understanding begins to improve. Seed AI is not a workaround that avoids the challenge of general intelligence by bootstrapping from an unintelligent core; seed AI only begins to yield benefits once there is some degree of available intelligence to be utilized. The later consequences of seed AI (such as true recursive self-improvement) only show up after the AI has achieved significant holonic understanding and general intelligence.

It’s reasonable to think we’re at the start here, with how general and useful today’s models are.

Generally, RSI can be summarized as when AI can improve itself, the improved version can improve even more efficiently, creating a closed amplification loop that leads to an intelligence explosion, often referred to as the singularity. There are a few assumptions in this. For RSI to occur, it needs to be that:

1. The loop is closed. Models can keep improving on themselves and beget more models.

2. The loop is self-amplifying. The next models will yield even bigger improvements than the current ones.

3. The loop continues to run without losing efficiency. There are not added pieces of friction that make the exponential knee-capped as an early sigmoid.

While I agree that momentous, socially destabilizing changes are coming in the next few years from sustained AI improvements, I expect the trend line of progress to be more linear than exponential when we reflect back. Instead of recursive self-improvement, it will be lossy self-improvement (LSI) – the models become core to the development loop but friction breaks down all the core assumptions of RSI. The more compute and agents you throw at a problem, the more loss and repetition shows up.

I’m still a believer that the complexity brake on advanced systems will be a strong counterbalance to the reality that AI models are getting substantially better at every narrow task we need to compose together in making a leading AI model. I quoted this previously in April of 2025 in response to AI 2027.

Microsoft co-founder Paul Allen argued the opposite of accelerating returns, the complexity brake: the more progress science makes towards understanding intelligence, the more difficult it becomes to make additional progress. A study of the number of patents shows that human creativity does not show accelerating returns, but in fact, as suggested by Joseph Tainter in his The Collapse of Complex Societies, a law of diminishing returns. The number of patents per thousand peaked in the period from 1850 to 1900, and has been declining since. The growth of complexity eventually becomes self-limiting, and leads to a widespread “general systems collapse”.

There are plenty of examples in how models are already trained, the deep intuitions we need to get them right, and the organizations that build them that show where the losses will come from. Building leading language models is incredibly complex, and only becoming more-so. There are a few core frictions in my mind.

1. Automatable research is too narrow

First, it is clear that language models this year will already be useful tools at optimizing localized tasks like lowering the test loss of a model. Andrey Karpathy recently launched his autoresearch that popularized doing just this. This allows AI agents to play directly on GPUs to target tasks like lowering the loss on the test set. This approach works in narrow domains, i.e. one general test loss or one overall reward. The problem is that there’s a long-standing gap between an on-paper more accurate model and models that users find more productive. The most provocative case is for pretraining, which was discussed more at length around scaling laws. Scaling laws show us that the loss will continue going down, but we don’t know if that’ll be economically more valuable.

In post-training, reinforcement learning algorithms are at least more directly tied to specific performance gains as most RL training environments can be used directly as an evaluation. Still, I worry about generalization and tying back to models that are better at the specific task of improving themselves. It’s a big leap from models get better at some things to that necessarily translating to models that are better at building themselves and designing experiments. We’ve seen many AI capabilities sort of saturate at certain levels of human taste, such as writing quality. AI research is a bit different here, as there is a very high ceiling to climb up to. Where models mostly saturate on writing because there’s inherent tension in preferences, models will saturate on research because the search space and optimization target is too wide.

The early benchmarks for measuring this sort of ability all fall prey to the same problem – narrow scope. Agents will do well at optimizing single metrics, but the leap required to navigate many metrics at once is a very different skill set. That is actually what the best researchers do — they make many scalable ideas work together.

The most related benchmark we have to measure this is PostTrainBench, which is quite fun, but progress will very rapidly get distorted on this. Over 90% of the challenge in doing post-training well is getting the last 1-3% of performance, especially without cooking the model in out-of-domain tasks. Post-training a general, leading model is extremely complex, and only getting more complex.

I could go on and on about this. Another example is from during my Ph.D. (2017-2022), when there was immense hype around a field called “AutoML” which aimed to use techniques like Bayesian Optimization to find new architectures and parameters for models. The hype never translated into changing my job. Language models will do more than this, but not enough to take jobs away from top AI researchers any time soon. The core currency of researchers is still intuition and managing complexity, rather than specific optimization and implementation.

2. Diminishing returns of more AI agents in parallel

The biggest problem for rapid improvement in AI is that even though we’ll have 10,000 remote workers in a datacenter, it’ll be nearly impossible to channel all of them at one problem. Inherently, especially when the models are still so similar, they’re sampling from the same distribution of solutions and capabilities while being bottlenecked by human supervision. Adding more agents will have a strict saturation in the amount of marginal performance that can be added – the intuition of the best few researchers (and time to run experiments) will be the final bottleneck.

A common idea to illustrate this is Amdahl’s law, which is taken from computer architecture and shows that a given task can only generate a fixed speedup proportional to how much can be parallelized and how many parallel workers exist. An illustration is below:

In AI this should be relatively easier to convey, as the low-level operating details of computers are fairly mysterious. Consider an AI researcher on the transition from writing code by hand to using AI autocomplete assistance to now using autonomous coding agents. These are all massive gains. Let us continue. Now this researcher uses 3-4 agents working on different sub-tasks or approaches to the problem at hand. This is still a large gain. Now consider this single researcher trying to organize 30-40 agents with tasks to do every day. Some people can get more value out of this scale, but not many.

How many people do you think could come up with 300-400 tasks for AI agents every day? Not many. This problem will hit the AI models soon enough as well.

3. Resource bottlenecks and politics

Fundamentally, all the AI companies are walking a fine line of acquiring substantial capital, converting new compute resources to revenue via sufficient demand, and repeating the process all-the-while spending an extreme amount on research. With the scale of resources here, there will always be political bottlenecks on who gets resources and what gets bet on. In this layer, research leadership sits above the AIs and the researchers. Even as models continue to improve, this source of friction will never get removed. It isn’t a substantial friction, but the AI models are fundamentally operating in organizations where humans are the bottleneck on resources.

The early scale of improvements with language models is local optimizations, where the resources used cost <$1M per day. With my other views on the frictions of AI, this is on its own a very minor impact on the rate of improvement, but for those with worries of fast take-off, RSI, and loss of control to AIs, it should be obvious that billions of dollars of compute resources for research are unlikely to be totally isolated for end-to-end experimentation of AI models.

Share

The conclusion here is that because we’re at the early stages of using AI assistance, autonomously and at scale for AI-development, we’re collectively discovering the ways that AI can help us massively. We’re all applying these tools to capture the low-hanging fruit we see and our jobs are literally changing to be higher paced and more productive. The problem is that all of these axes have clear human, political, or technical complexity bottlenecks.

The bottom of every sigmoid feels like an exponential. We’ve ridden multiple exponentials in the era of language models, in 2023 we scaled to huge models and GPT-4 felt like magic, by 2025 we added inference-time scaling with o1 and reasoning models — they let us “solve” math and coding, now we’re going to take a big step by polishing the entire AI workflow (all the while scaling training compute massively). 2026 will feel like a huge step, but it doesn’t have a fundamental change convincing me that progress will begin to take off.

This could still cross the colloquial threshold for AGI, which is a drop-in replacement for most remote workers, which would be an incredible milestone. Much of the challenge in the debate of if we hit AGI in the coming years is that AI models are jagged and smart in different ways than humans, so they won’t look like drop-in replacements for remote workers, but in many cases just using AI will be far more effective than trying to work with a human. It’s reshaping what jobs are.

Let us consider the scenarios we’re working through.

1. Engineering is becoming automated today. Humans are way more productive, models can scale through complex infrastructure deployments much faster, run with higher GPU utilization, etc. Infrastructure gains become fixed improvements in the rate and scale of experimentation, the fundamental units of progress in AI.

2. Basic AI model research and optimization will be automated. The AI models are expanding in scope – they transition from writing kernels to deciding on architectures. This is moving from improving the experimentation toolkit to running minor experiments themselves. Configs, hyperparameters, etc. become the domain of the AI assistants.

These are both real. The problem is that a third era doesn’t have a simple scale to jump to. Where the AI models can create knowledge by synthesis and execution, the next jump requires harnessing thousands of agents or having models make more novel discoveries – like unlocking the next paradigm after inference time scaling. The improvements downstream of AI are going to make the industry supercharged at hill climbing, but I worry that this won’t bring paradigm shifts that are needed for new categories of AI – continual learning, world models, whatever your drug of choice is.

All together, the models are becoming core to the development loop and that’s worth being excited (and worried) about. The models are performing self-improvement. They’re not transforming the approach. We are scaling up the compute we spend on our own research practices and tools. There are diminishing returns. Agents are going to start being autonomous entities we work with. They feel like a cross between a genius and a 5 year old. We will be in this era of lossy self-improvement (LSI) for a few years, but it is not enough for a fast takeoff.

I had originally planned to write about DeepSeek V4. Since it still hasn’t been released, I used the time to work on something that had been on my list for a while, namely, collecting, organizing, and refining the different LLM architectures I have covered over the past few years.

So, over the last two weeks, I turned that effort into an LLM architecture gallery (with 45 entries at the time of this writing), which combines material from earlier articles with several important architectures I had not documented yet. Each entry comes with a visual model card, and I plan to keep the gallery updated regularly.

You can find the gallery here: https://sebastianraschka.com/llm-architecture-gallery/

After I shared the initial version, a few readers also asked whether there would be a poster version. So, there is now a poster version via Redbubble. I ordered the Medium size (26.9 x 23.4 in) to check how it looks in print, and the result is sharp and clear. That said, some of the smallest text elements are already quite small at that size, so I would not recommend the smaller versions if you intend to have everything readable.

Alongside the gallery, I was/am also working on short explainers for a few core LLM concepts.

So, in this article, I thought it would be interesting to recap all the recent attention variants that have been developed and used in prominent open-weight architectures in recent years.

My goal is to make the collection useful both as a reference and as a lightweight learning resource. I hope you find it useful and educational!

1. Multi-Head Attention (MHA)

Self-attention lets each token look at the other visible tokens in the sequence, assign them weights, and use those weights to build a new context-aware representation of the input.

Multi-head attention (MHA) is the standard transformer version of that idea. It runs several self-attention heads in parallel with different learned projections, then combines their outputs into one richer representation.

The sections below start with a whirlwind tour of explaining self-attention to explain MHA. It’s more meant as a quick overview to set the stage for related attention concepts like grouped-query attention, sliding window attention, and so on. If you are interested in a longer, more detailed self-attention coverage, you might like my longer Understanding and Coding Self-Attention, Multi-Head Attention, Causal-Attention, and Cross-Attention in LLMs article.

EXAMPLE ARCHITECTURES

GPT-2, OLMo 2 7B, and OLMo 3 7B

1.2 Historical Tidbits And Why Attention Was Invented

Attention predates transformers and MHA. Its immediate background is encoder-decoder RNNs for translation.

In those older systems, an encoder RNN would read the source sentence token by token and compress it into a sequence of hidden states, or in the simplest version into one final state. Then the decoder RNN had to generate the target sentence from that limited summary. This worked for short and simple cases, but it created an obvious bottleneck once the relevant information for the next output word lived somewhere else in the input sentence.

In short, the limitation is that the hidden state can’t store infinitely much information or context, and sometimes it would be useful to just refer back to the full input sequence.

The translation example below shows one of the limitations of this idea. For instance, a sentence can preserve many locally reasonable word choices and still fail as a translation when the model treats the problem too much like a word-by-word mapping. (The top panel shows an exaggerated example where we translate the sentence word by word; obviously, the grammar in the resulting sentence is wrong.) In reality, the correct next word depends on sentence-level structure and on which earlier source words matter at that step. Of course, this could still be translated fine with an RNN, but it would struggle with longer sequences or knowledge retrieval tasks because the hidden state can only store so much information as mentioned earlier.

The next figure shows that change more directly. When the decoder is producing an output token, it should not be limited to one compressed memory path. It should be able to reach back to the more relevant input tokens directly.

Transformers keep that core idea from the aforementioned attention-modified RNN but remove the recurrence. In the classic Attention Is All You Need paper, attention becomes the main sequence-processing mechanism itself (instead of being just part of an RNN encoder-decoder.)

In transformers, that mechanism is called self-attention, where each token in the sequence computes weights over all other tokens and uses them to mix information from those tokens into a new representation. Multi-head attention is the same mechanism run several times in parallel.

1.3 The Masked Attention Matrix

For a sequence of T tokens, attention needs one row of weights per token, so overall we get a T x T matrix.

Each row answers a simple question. When updating this token, how much should each visible token matter? In a decoder-only LLM, future positions are masked out, which is why the upper-right part of the matrix is grayed out in the figure below.

Self-attention is fundamentally about learning these token-to-token weight patterns, under a causal mask, and then using them to build context-aware token representations.

1.4 Self-Attention Internals

The next figure shows how the transformer computes the attention matrix (A) from the input embeddings X, which is then used to produce the transformed inputs (Z).

Here Q, K, and V stand for queries, keys, and values. The query for a token represents what that token is looking for, the key represents what each token makes available for matching, and the value represents the information that gets mixed into the output once the attention weights have been computed.

The steps are as follows:

• Wq, Wk, and Wv are weight matrices that project the input embeddings into Q, K, and V

• QK^T produces the raw token-to-token relevance scores

• softmax converts those scores into the normalized attention matrix A that we discussed in the previous section

• A is applied to V to produce the output matrix Z

Note that the attention matrix is not a separate hand-written object. It emerges from Q, K, and softmax.

The next figure shows the same concept as the previous figure but the attention matrix computation is hidden inside the “scaled-dot-product attention” box, and we perform the computation only for one input token instead of all input tokens. This is to show a compact form of self-attention with a single head before extending this to multi-head attention in the next section.

1.5 From One Head To Multi-Head Attention

One set of Wq/Wk/Wv matrices gives us one attention head, which means one attention matrix and one output matrix Z. (This concept was illustrated in the previous section.)

Multi-head attention simply runs several of these heads in parallel with different learned projection matrices.

This is useful because different heads can specialize in different token relationships. One head might focus on short local dependencies, another on broader semantic links, and another on positional or syntactic structure.

2. Grouped-Query Attention (GQA)

Grouped-query attention is an attention variant derived from standard MHA. It was introduced in the 2023 paper GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints by Joshua Ainslie and colleagues.

Instead of giving every query head its own keys and values, it lets several query heads share the same key-value projections, which makes KV caching much cheaper (primarily as a memory reduction) without changing the overall decoder recipe very much.

EXAMPLE ARCHITECTURES

Dense: Llama 3 8B, Qwen3 4B, Gemma 3 27B, Mistral Small 3.1 24B, SmolLM3 3B, and Tiny Aya 3.35B.
Sparse (Mixture-of-Experts): Llama 4 Maverick, Qwen3 235B-A22B, Step 3.5 Flash 196B, and Sarvam 30B.

2.1 Why GQA Became Popular

In my architecture comparison article, I framed GQA as the new standard replacement for classic multi-head attention (MHA). The reason is that standard MHA gives every head its own keys and values, which is more optimal from a modeling perspective but expensive once we have to keep all of that state in the KV cache during inference.

In GQA, we keep a larger set of query heads, but we reduce the number of key-value heads and let multiple queries share them. That lowers both parameter count and KV-cache traffic without making drastic implementation changes like multi-head latent attention (MLA), which will be discussed later.

In practice, that made and keeps it a very popular choice for labs that wanted something cheaper than MHA but simpler to implement than newer compression-heavy alternatives like MLA.

2.2 GQA Memory Savings

GQA results in big savings in KV storage, since the fewer key-value heads we keep per layer, the less cached state we need per token. That is why GQA becomes more useful as sequence length grows.

GQA is also a spectrum. If we reduce all the way down to one shared K/V group, we are effectively in multi-query attention territory, which is even cheaper but can hurt modeling quality more noticeably. The sweet spot is usually somewhere in between multi-query attention (1 shared group) and MHA (where K/V groups are equal to the number of queries), where the cache savings are large but the modeling degradation relative to MHA stays modest.

2.3 Why GQA Still Matters In 2026

More advanced variants such as MLA are becoming popular because they can offer better modeling performance at the same KV efficiency levels (e.g., as discussed in the ablation studies of the DeepSeek-V2 paper), but they also involve a more complicated implementation and a more complicated attention stack.

GQA remains appealing because it is robust, easier to implement, and also easier to train (since there are fewer hyperparameter tunings necessary, based on my experience).

That is why some of the newer releases still stay deliberately classic here. E.g., in my Spring Architectures article, I mentioned that MiniMax M2.5 and Nanbeige 4.1 as models that remained very classic, using only grouped-query attention without piling on other efficiency tricks. Sarvam is a particularly useful comparison point as well: the 30B model keeps classic GQA, while the 105B version switches to MLA.

3. Multi-Head Latent Attention (MLA)

The motivation behind Multi-head Latent Attention (MLA) is similar to Grouped-Query Attention (GQA). Both are solutions for reducing KV-cache memory requirements. The difference between GQA and MLA is that MLA shrinks the cache by compressing what gets stored rather than by reducing how many K/Vs are stored by sharing heads.

MLA, originally proposed in the DeepSeek-V2 paper, became such a defining DeepSeek-era idea (especially after DeepSeek-V3 and R1). It is more complicated to implement than GQA, more complicated to serve, but nowadays also often more compelling once model size and context length get large enough that cache traffic starts to dominate, because at the same rate of memory reduction, it could maintain better modeling performance (more on that later).

EXAMPLE ARCHITECTURES

DeepSeek V3, Kimi K2, GLM-5, Ling 2.5, Mistral Large 3, and Sarvam 105B

3.1 Compression, Not Sharing

Instead of caching full-resolution key and value tensors as in MHA and GQA, MLA stores a latent representation and reconstructs the usable state when needed. Essentially, it is a cache compression strategy embedded inside attention, as illustrated in the previous figure.

The figure below shows the savings compared to regular MHA.

3.2 MLA Ablation Studies

The DeepSeek-V2 paper provided some ablations where GQA looked worse than MHA in terms of modeling performance, while MLA held up much better and could even outperform MHA when tuned carefully. That is a much stronger justification than “it (also) saves memory.”

In other words, MLA is a preferable attention mechanism for DeepSeek not just because it was efficient, but because it looked like a quality-preserving efficiency move at large scale. (But colleagues also told me that MLA only works well at a certain size. For smaller models, let’s say <100B, GQA seems to work better, or, is at least easier to tune and get right.)

Below is again the comparison between GQA in 30B Sarvam versus MLA in 105B Sarvam.

3.3 How MLA Spread After DeepSeek

Once DeepSeek V3/R1, V3.1 etc. normalized the design after its introduction in V2, it started showing up in a second wave of architectures. Kimi K2 kept the DeepSeek recipe and scaled it up. GLM-5 adopted MLA together with DeepSeek Sparse Attention (from DeepSeek V3.2). Ling 2.5 paired MLA with a linear-attention hybrid. Sarvam released two models where the 30B model stayed with classic GQA and the 105B model switched to MLA.

That last pair is particularly useful as it puts the technical-complexity discussion aside. I.e., the Sarvam team implemented both variants and deliberately chose to then use GQA for one variant and MLA for the other. So, in a sense, that makes MLA feel less like a theoretical alternative and more like a concrete architectural upgrade path once a family scales up.

4. Sliding Window Attention (SWA)

Sliding window attention reduces the memory and compute cost of long-context inference by limiting how many previous tokens each position can attend to. Instead of attending to the entire prefix, each token only attends to a fixed window of recent tokens around its position. Because attention is restricted to a local token neighborhood, this mechanism is often referred to as local attention.

Some architectures combine these local layers with occasional global attention layers so that information can still propagate across the entire sequence.

EXAMPLE ARCHITECTURES

Gemma 3 27B, OLMo 3 32B, Xiaomi MiMo-V2-Flash, Arcee Trinity, Step 3.5 Flash, and Tiny Aya

4.1 Gemma 3 As A Reference Point

Gemma 3 is still one of the clearest recent SWA examples because it is easy to compare against Gemma 2. Gemma 2 already used a hybrid attention setup with a 1:1 ratio between local and global layers and a 4096-token window. Gemma 3 pushed this further to a 5:1 ratio and reduced the window size to 1024.

The key finding was not that local attention is cheaper, because that was already known. Here, the more interesting takeaway from the Gemma 3 ablation study was that using this more aggressively seemed to hurt modeling performance only slightly.

4.2 The Ratio And Window Size

In practice, saying that a model “uses SWA” does not mean it relies on SWA alone. What usually matters are the local-to-global layer pattern and the attention window size. For example:

• Gemma 3 and Xiaomi use a 5:1 local-to-global pattern.

• OLMo 3 and Arcee Trinity use a 3:1 pattern.

• Xiaomi also uses a window size of 128, which is much smaller, and therefore more aggressive, than Gemma’s 1024.

SWA is essentially a knob that can be tuned more or less aggressively.

4.3 Combining SWA with GQA

SWA often appears together with GQA because the two ideas address different parts of the same inference problem. SWA reduces how much context a local layer has to consider. GQA reduces how much key-value state each token contributes to the cache.

That is why many recent dense models use both rather than treating them as alternatives. Gemma 3 is again a good reference point here, since it combines sliding window attention with grouped-query attention in the same architecture.

5. DeepSeek Sparse Attention (DSA)

DeepSeek Sparse Attention is one of the architectural changes that appeared in the DeepSeek V3.2 line and later showed up again in GLM-5.

Specifically, DeepSeek V3.2 combines it with Multi-head Latent Attention (MLA), and GLM-5 adopts the same pair for the same general reason, namely, reducing inference cost when context lengths get large.

EXAMPLE ARCHITECTURES

DeepSeek V3.2 and GLM-5

5.1 Changes Relative To Sliding-Window Attention

In sliding-window attention, the current token does not attend to the full prefix but only to a fixed local window. This is the same broad idea behind DeepSeek Sparse Attention, where each token also only attends to a subset of previous tokens.

However, the selected tokens are not determined by a fixed-width local window. Instead, DeepSeek Sparse Attention uses a learned sparse pattern. In short, it uses an indexer-plus-selector setup, where a lightning indexer computes relevance scores, and a token selector keeps only a smaller set of high-scoring past positions.

The way the subset of tokens is selected is the main difference from sliding-window attention. Sliding-window attention hard-codes locality. DeepSeek Sparse Attention still limits attention to a subset, but it lets the model decide which prior tokens are worth revisiting.

5.2 DeepSeek Sparse Attention and MLA

DeepSeek V3.2 uses both Multi-head Latent Attention (MLA) and DeepSeek Sparse Attention. MLA reduces KV-cache cost by compressing what gets stored. DeepSeek Sparse Attention reduces how much of the prior context the model has to revisit. Put differently, one optimizes the cache representation, the other optimizes the attention pattern on top of it.

The sparse pattern is not random. The first stage is a lightning indexer that scores previous tokens for each new query token. It uses MLA’s compressed token representations and computes a learned similarity score over the prior context, so the model can rank which earlier positions are worth revisiting.

The second stage is a token selector. It keeps only a smaller high-scoring subset, for example, a top-k set of past positions, and turns that subset into the sparse attention mask. So the main point is that DeepSeek Sparse Attention does not hard-code the sparsity pattern. It learns which past tokens to keep.

DeepSeek Sparse Attention is relatively new and relatively complicated to implement, which is why it has not been so widely adopted as Grouped-Query Attention (GQA) yet.

6. Gated Attention

Gated attention is best understood as a modified full-attention block rather than as a separate attention family.

It usually appears inside hybrid stacks that still keep an occasional full-attention layer for exact content retrieval, but add a few stability-oriented changes on top of an otherwise familiar scaled dot-product attention block.

6.1 Where Gated Attention Appears

The Qwen3-Next and Qwen3.5 architectures show that recent hybrids (covered in the next section) do not replace attention everywhere. Instead, they replace most attention layers with a cheaper alternative and keep a smaller number of full-attention layers in the stack.

Those remaining full-attention layers are where gated attention typically appears. Qwen3-Next and Qwen3.5 use it together with Gated DeltaNet in a 3:1 pattern.

But hybrid architectures aside, Trinity uses a related gating idea in a more conventional attention stack, as shown in the previous figure above.

6.2 Gated Attention Relative To Standard Attention

The gated attention block in Qwen-style hybrids or Trinity (not a hybrid) is essentially standard scaled-dot-product attention with a few changes on top. In the original Gated Attention paper, those changes are presented as a way to make the retained full-attention layers behave more predictably inside a hybrid stack.

The block still looks like standard (full) attention, but it adds:

1. an output gate that scales the attention result before it is added back to the residual,

2. a zero-centered QK-Norm variant instead of standard RMSNorm for q and k,

3. partial RoPE.

These are not changes on the scale of MLA or linear attention but merely stability and control changes applied to an otherwise familiar attention block.

Note that the figure above also includes Gated DeltaNet, which we will cover in the next section below.

7. Hybrid Attention

Hybrid attention is a broader design pattern rather than a specific, single mechanism. The overall idea is to keep a transformer-like stack, but replace most of the expensive full-attention layers with cheaper linear or state-space sequence modules.

The motivation is long-context efficiency. Full attention grows quadratically with sequence length, so once models move to contexts like 128k, 256k, or 1M tokens, attention memory and compute become expensive enough that using cheaper sequence modules in most layers while keeping only a smaller number of heavier retrieval layers starts making more sense. (Note that this comes with a bit of a modeling performance trade-off, though.)

In Qwen3-Next, this pattern appears as a 3:1 mix of Gated DeltaNet and Gated Attention blocks. Gated DeltaNet is also closely related to Mamba-2 (see the Gated Delta Networks: Improving Mamba2 with Delta Rule paper, for instance), and the mechanism can be read as a DeltaNet-style fast-weight update combined with Mamba-style gating. Later architectures keep the same overall idea but swap in other lightweight sequence mixers, such as Kimi Delta Attention, Lightning Attention, or standard Mamba-2.

7.1 Gated DeltaNet in Qwen3-Next

To my knowledge, the first prominent example of a close-to-flagship LLM with hybrid attention was Qwen3-Next in 2025, which does not remove attention completely but mixes three Gated DeltaNet blocks with one Gated Attention block.

Here, lightweight Gated DeltaNet blocks do most of the long-context work and keep memory growth much flatter than full attention. The heavier gated-attention layer remains because DeltaNet is less exact at content-based retrieval.

Inside a Gated DeltaNet block, the model computes query, key, and value vectors together with two learned gates (α, β). Rather than forming the usual token-to-token attention matrix, it writes to a small fast-weight memory using a delta-rule update. In rough terms, the memory stores a compressed running summary of past information, while the gates control how much new information is added and how much previous state is retained.

That makes Gated DeltaNet a linear-attention or recurrent-style mechanism rather than just another tweak to MHA. Relative to Mamba-2, the close connection is that both belong to the linear-time gated sequence-model family, but Gated DeltaNet uses a DeltaNet-style fast-weight memory update instead of the Mamba state-space update.

Qwen3.5 moves the former Qwen3-Next hybrid into Qwen’s main flagship series, which is an interesting move. This basically signals that the hybrid strategy is a success and that we may see more models with this architecture in the future.

7.2 Kimi Linear And Modified Delta Attention

Kimi Linear keeps the same broad transformer skeleton and the same 3:1 pattern, but it changes both halves of the recipe.

On the lightweight side, Kimi Delta Attention is a refinement of Gated DeltaNet. Where Qwen3-Next uses a scalar gate per head to control memory decay, Kimi uses channel-wise gating, which gives finer control over the memory update. On the heavier side, Kimi replaces Qwen3-Next’s gated-attention layers with gated MLA layers.

So, it’s still the same broader pattern as in Qwen3-Next and Qwen3.5, but both ingredients (slightly) change. I.e., most layers are still handled by a cheaper linear-style mechanism, and periodic heavier layers still remain for stronger retrieval.

7.3 Ling 2.5 And Lightning Attention

Ling 2.5 shows another swap on the lightweight side. Instead of Gated DeltaNet, Ling uses a slightly simpler recurrent linear attention variant called Lightning Attention. On the heavier side, it keeps MLA from DeepSeek.

Most sequence mixing happens in the cheaper linear-attention blocks, while a smaller number of heavier layers remain to preserve stronger retrieval. The difference is that the specific lightweight mechanism is now Lightning Attention rather than DeltaNet or Kimi Delta Attention.

Ling 2.5 is aimed more at long-context efficiency than at absolute benchmark leadership. According to the Ling team, it was reported as substantially faster than Kimi K2 at 32k tokens, which is the practical payoff these hybrids are aiming for.

Nemotron And Mamba-2

Nemotron pushes the pattern further away from the transformer baseline. Nemotron 3 Nano is a Mamba-Transformer hybrid that interleaves Mamba-2 sequence-modeling blocks with sparse MoE layers and uses self-attention only in a small subset of layers.

This is a more extreme version of the same basic tradeoff discussed above. Here, the lightweight sequence module is a Mamba-2 state-space block rather than a DeltaNet-style fast-weight update, but the basic tradeoff is similar.

The larger Nemotron 3 Super keeps the Mamba-2 hybrid attention approach and adds other efficiency-oriented changes such as latent MoE and shared-weight multi-token prediction (MTP) for speculative decoding.

Conclusion

Of course, there are many more (mostly niche) attention variants throughout the literature that I haven’t covered here. The focus of this article was on those that are currently used in state-of-the-art (open-weight) models.

In particular, I am looking forward to (1) seeing the brand new Mamba-3 layers getting integrated into the aforementioned hybrid architectures (replacing Gated DeltaNet) and (2) attention residuals being used in general.

In practice, you may also wonder what the “best” architecture is at the moment. This is hard to answer, as there are no public experiments that train different architectures on the same training data etc.

Hence, we can currently only answer what the best (trained) model choice is for a given problem. In my opinion, hybrid architectures are still a novelty, and the main selling point is mainly (long-context) efficiency versus just modeling performance. Hence, I think they are a great candidate for agent contexts (like OpenClaw).

Personally, I think the problem with hybrid architectures is also that the inference stacks are not quite as optimized, yet, and I find that I get better tok/sec throughput when running LLMs locally using more classic setups like GPT-OSS with grouped-query attention.

Anyways, I am curious to see what DeepSeek V4 has in store, since DeepSeek has been quite the reliable trend-setter in the recent 2 years.

by Dan Shipper in Chain of Thought

Was this newsletter forwarded to you? Sign up to get it in your inbox.

At 4 a.m. on the day after we launched our agent-native document editor, Proof, I watched yet another Codex agent try to revive our server.

Over 4,000 documents had been created since launch, but the app had been mysteriously crashing all day. This left users with crucial documents that they couldn’t access, and me with egg on my face.

I hadn’t slept for almost 24 hours, and all I could do was nervously munch trail mix as Codex investigated yet another bug buried deep in a codebase that I didn’t understand. It felt less like programming and more like being the dumbest participant at a math Olympiad. Needless to say, I was reconsidering my life choices.

Today, almost a week later, Proof is more or less stable. And I’ve learned a lot about both building and launching a purely vibe coded app. Perhaps more importantly, I’ve also learned what happens once that app goes live—and then goes down.

My current opinion is this: If you can vibe code it, you can vibe fix it. You just might not be able to fix it quickly.

Software engineering is changing rapidly as a discipline. The days of typing code into a computer manually seem to be over, and the current conversation on X is around “zero-human startups.” My experience with Proof, though, is a good reality check.

It demonstrates both what is truly possible with vibe coded apps, and where human engineers will continue to be critical now and in the future.

What’s possible at the edge

I’ve been writing about how AI is changing programming for a few years now, and my experience with Proof confirms a lot of my thoughts...

Become a paid subscriber to Every to unlock this piece and learn about:

1. What it took to bring a crashing, vibe coded app back from the brink
2. The specific failure modes coding models keep hitting
3. Why allocation is the new key skill for human engineers

Click here to read the full post

Want the full text of all articles in RSS? Become a subscriber, or learn more.

Among the various tools available to policymakers, regulatory sandboxes have gained considerable prominence in the AI governance landscape because they enable supervised innovation testing under controlled conditions and within limited timeframes. This can help to identify risks early, foster regulatory learning and refine regulatory requirements before they are applied at scale.

As AI regulatory sandboxes expand across jurisdictions and sectors, common design principles, recurring challenges and opportunities for greater effectiveness and policy coherence are emerging. As this happens, institutional co-operation and knowledge sharing are more important for ensuring coherent and effective regulatory experimentation both nationally and across borders.

In November 2025, the OECD webinar “AI Sandboxes: Sharing knowledge for success” brought together government officials, regulators and policy experts from seven countries to discuss the design and implementation of AI regulatory sandboxes. Here are the event’s key takeaways.

What is an ‘AI regulatory sandbox’?

Although there is no universally accepted definition, a regulatory sandbox generally offers temporary regulatory flexibility or waivers, allowing innovative products, services, or business models to be tested under controlled conditions and regulatory oversight. This approach promotes responsible experimentation and innovation while protecting the public interest.

To cite a few examples, Singapore’s AI healthcare sandbox offers guidelines for synthetic data to minimise privacy risks while allowing realistic testing. In the UK and other countries, AI-powered innovations in financial services are being tested under supervision that helps to prevent consumer harms such as biased scoring and automated decision-making. 

In AI, this approach aligns with the OECD AI Principles – specifically Principle 2.3, which encourages governments to promote experimentation to enable the safe testing and scaling of AI systems. Similarly, the Recommendation of the Council for Agile Regulatory Governance to Harness Innovation urges governments to facilitate greater experimentation, testing, and trialling to stimulate innovation under regulatory supervision.

AI regulatory sandboxes are valuable for testing new technologies and rules in a safe, controlled way before full rollout. They offer less benefit if risks are low or if rules are already well established, but can be useful for compliance and learning in more complex regulatory environments. Decisions to utilise sandboxes should follow clear criteria to ensure efforts are appropriately targeted. Generally, initiatives with high innovation potential, substantial risks, and opportunities for regulatory discovery and improvement (including by removing barriers to beneficial innovation) should be prioritised. Key regulators, industry actors and other relevant stakeholders should be involved in this process.

In July 2023, the OECD published the policy paper Regulatory Sandboxes in artificial intelligence. Building on lessons from fintech, the report highlights the benefits of AI sandboxes, including accelerating market entry, improving regulatory understanding, and stimulating investment. It also explains why adapting the traditional sandbox model to AI presents unique technical and governance challenges. As a cross-sectoral technology, AI covers multiple legal, ethical, and technical domains, requiring strong coordination among several regulatory authorities.

Since the paper’s release, the use of AI sandboxes has accelerated. By February 2025, the Datasphere Initiative identified over 60 sandboxes worldwide related to AI, data, and technology. Furthermore, key regulatory and policy frameworks, including the European Union’s AI Act and America’s AI Action Plan, view regulatory sandboxes as essential tools for fostering AI innovation and ensuring the safe development and adoption of AI.

Insights shared during the webinar by experts from Spain, Thailand, Luxembourg, Brazil, Korea, Israel and Singapore offer valuable lessons on how different jurisdictions design and operate AI sandboxes, highlighting what works, where challenges arise, and how approaches vary across contexts. For example, Spain provides appropriate, tailored guidance to ensure effectiveness and facilitate regulatory compliance further down the line. Brazil’s sequenced approach includes capacity-building to enable participants to contribute to effective experimentation and evaluation.

>> REVISIT THE WEBINAR AND RELATED PUBLICATIONS <<

Six insights about AI regulatory sandboxes from around the globe

1. AI sandboxes are not uniform

• According to the Datasphere Initiative, three primary types of sandboxes are emerging worldwide, especially within the context of AI. Regulatory sandboxes: Collaborative processes where regulators work with innovators to test innovations under regulatory supervision.
• Operational sandboxes: Testing environments and infrastructure where data can be hosted and accessed in controlled conditions.
• Hybrid models: Combining regulatory oversight with operational capabilities, sometimes offering infrastructure and operational spaces for testing and experimentation (e.g., “supercharged sandbox” in the UK).

These models intervene at different phases of the policy and regulatory lifecycle. Some are employed before formal regulation to identify gaps and suggest necessary updates. Others operate during the development process, supporting iterative regulatory design. Some focus on helping understand legal obligations and ensure regulatory compliance, such as under the EU AI Act. Sector-specific sandboxes are also common, with countries adopting different approaches depending on regulatory priorities and institutional settings. Across these models, regulatory waivers are frequently used to enable experimentation under regulatory supervision. 

Several experimentation-related initiatives, such as regulatory testbeds, living labs, or policy prototyping, share certain features and objectives with regulatory sandboxes. What truly distinguishes sandboxes is that they are the most institutionalised form of regulatory experimentation, usually led by regulators and integrated with regulatory supervision.

2. Coordination is essential

AI does not always fit neatly within existing sectoral, jurisdictional, or administrative boundaries. Its development and deployment span multiple regulatory domains, making effective coordination crucial. Luxembourg’s approach demonstrates this well, showing that AI sandboxes are more than testing spaces—they are platforms for regulatory collaboration and coordination. Luxembourg’s model brings together 11 authorities and innovation actors to align priorities and prevent fragmentation, emphasising the need for skilled project management alongside legal and technical expertise. 

Specific stakeholders within the AI ecosystem pursue different objectives: data protection authorities concentrate on privacy, cybersecurity authorities on resilience, and innovators on efficiency and speed. They also offer different kinds of expertise. Sandboxes can offer a neutral space to reconcile these priorities, fostering trust and mutual understanding. To achieve this, managing the expectations of involved parties and clearly defining the objectives of a sandbox are particularly important steps. 

In Thailand, a multi-faceted approach to AI regulatory sandboxing shows how balancing safety and flexibility depends on agile cooperation between sectoral regulators and industry. This approach integrates three complementary pathways: in the short term, fostering AI deployment where existing rules already allow it; in the medium term, establishing sector-specific sandboxes to manage domain-specific risks and opportunities; and eventually, developing system-wide sandboxes, including for government use, to test cross-cutting applications. Together, these mechanisms help promote AI-driven innovation within current legal frameworks while leveraging testing and experimentation to better understand the implications of emerging AI applications. Effective coordination is essential to prevent duplication of effort, regulatory gaps or conflicting rules.

Sandboxes can also play a valuable role in involving expert and academic communities in the development of AI regulation, with countries such as Spain, Luxembourg, and Brazil benefiting from such expertise at multiple stages of sandbox design and operation.

3. From policy to practice, and back again

AI sandboxes are increasingly used to bridge the gap between regulatory frameworks and real-world implementation. For example, Spain’s regulatory sandbox pilot translates the EU AI Act’s requirements for high-risk AI applications into practical compliance steps, enabling early identification of gaps and clarifying obligations for deployers. In December 2025, the Spanish AI Supervision Agency (AESIA) published a series of introductory and technical resources, developed from insights gathered during the regulatory sandbox pilot, that demonstrate how sandboxes can support evidence-based compliance guidance. Luxembourg’s AI sandbox, in turn, acts as a coordination platform to ensure lessons learned on overlapping obligations feed into the domestic operationalisation of the EU AI Act and related future guidance.

In July 2025, Singapore launched its Global AI Assurance Sandbox, building on insights from a previous pilot phase, to create a testing environment where creators or deployers of GenAI applications can have their applications evaluated by expert technical testers. Key risk aspects examined during testing include hallucination, undesirable content, data leakage, and vulnerability to adversarial prompts, with the findings informing policy guidance. Brazil’s Regulatory Sandbox on AI and Data Protection also exemplifies this trend. It aims to promote transparency, privacy by design and responsible innovation in AI systems that handle personal data, using structured experimentation to help innovators achieve regulatory compliance and assist regulators in understanding how rules work in practice and where adjustments may be necessary.

Simultaneously, AI sandboxes continue to shape future regulatory frameworks. In Thailand, sector-specific sandboxes for digital payments, digital assets, banking and insurance are expected to help regulators understand real-world AI applications and prepare for system-wide governance. As sandboxes move regulation from theory to practice and back to policy, they create an iterative loop that can strengthen trust and adaptability in AI governance. To achieve this, sandboxes should generate insights to inform better regulation. This, in turn, requires consistent reporting, sharing of results, and the establishment of feedback loops across sectors and countries to boost compliance and policy development.

Nevertheless, translating sandbox results into regulatory improvements remains challenging, even in countries like Korea, which has considerable experience conducting regulatory experiments across sectors.

4. Incentives matter

Participation in AI sandboxes is not automatic. Clear and well-designed incentives are essential for both innovators and regulators. Israel, for instance, has introduced a government fund that provides financial support, legal counselling and mentorship for regulators launching AI sandboxes, while also offering grants to participating firms. Similarly, Singapore reduces testing-related barriers to GenAI adoption through practical guidance and access to specialised testing partners.

These models recognise a fundamental challenge: AI experimentation is resource-intensive and needs to focus on areas where it matters most. Without targeted support, regulators may struggle to operate sandboxes, and companies might be hesitant to participate. Furthermore, when offering incentives, authorities should encourage a diverse mix of participants—small firms, big players, different sectors, and different AI applications—to ensure that sandbox insights are both representative and robust. The complex nature of regulatory sandboxes themselves may also pose challenges for some applicants or even participants. In this context, Brazil outlined a three-stage execution framework, starting with capacity building for selected participants undertaken by a partner university, before advancing to the experimentation and evaluation phases. 

5. The growing need for interoperability and cross-border collaboration

As AI systems operate across borders, there is a growing need for AI sandboxes to extend beyond national borders. Without international coordination, firms may engage in ‘jurisdiction hopping’, seeking the most permissive regulatory environments. Interoperability between sandboxes is thus becoming a governance necessity. Cross-border collaboration is also crucial for international regulatory cooperation. In Brazil’s case, preparatory work to develop the sandbox included international consultations on the experimental methodology. This approach enables the benefit from international practices and standards and facilitates the sharing of experiences in later stages of the project. 

Cross-border sandboxes have already proven their worth. Singapore’s Global AI Assurance Pilot, for example, involved 17 AI deployers from nine countries collaborating with 16 specialised testers from the US, UK and Europe. Use cases include summarisation, chatbots to AI applications in healthcare, finance and human resource management. These cross-border tests allowed regulators and companies to understand how AI performs in different legal, cultural and technical environments. For example, a chatbot that safely managed English queries inadvertently leaked confidential information when prompted in Mandarin, demonstrating the importance of multilingual testing. 

6. AI sandboxes come with challenges of their own

AI sandboxes face several challenges. Regulators often encounter capacity limitations and lack the technical expertise or project management skills necessary to supervise complex AI systems. Fragmentation and coordination issues also arise, as AI spans multiple sectors and necessitates collaboration among numerous authorities, including across borders. 

Designing suitable requirements and safeguards for sandbox frameworks can be challenging, especially when multiple regulatory regimes are involved, as demonstrated by Brazil. Deciding the appropriate level of transparency, human oversight, and data governance can be particularly difficult when firms seek waivers to speed up testing.

At the same time, Korea’s experience demonstrates that although safety and consumer protection rules are vital, excessively strict requirements may deter participation, especially among SMEs, and hinder experimentation. Safeguards should therefore be proportionate to and aligned with the risks posed by the technology, as overly complex procedures can undermine the agility required in regulatory sandboxes in a rapidly evolving AI landscape.

Measuring the impact of AI sandboxes is also difficult. Without clear metrics, sandboxes risk becoming isolated experiments rather than influential policy tools. Ideally, impact should be monitored across various areas, such as faster time-to-market for compliant AI systems, increased regulatory clarity and coherence (including through less fragmentation), and tangible updates to laws and standards shaped by sandbox insights. 

Furthermore, as highlighted in a 2024 OECD policy paper, there are potential limitations concerning legality, feasibility, resources, and equity. Regulatory experiments should adhere to constitutional norms, including those concerning equal treatment.

A vital element for responsible innovation and public trust?

As a relatively new regulatory tool designed to address a rapidly evolving general-purpose technology, AI sandboxes raise significant questions about their role. Some of the questions raised during the online workshop include:

• What role might civil society play in AI sandboxing?
• How can public institutions build the expertise required to supervise regulatory sandboxes involving frontier-level AI systems?
• How can sandboxes balance flexibility with protecting long-term societal values (e.g., what types of safeguards should be in place regarding regulatory exemptions)?
• What measures, such as reporting and documentation requirements, talent management, and capacity building, are necessary to ensure transparency and build trust in AI regulatory sandboxes? 

As AI governance develops and these questions are addressed, AI sandboxes hold the potential to become key tools for promoting responsible innovation, enhancing governance and building public trust in AI systems across the globe. 

The OECD is well placed to advance these objectives by facilitating the systematic exchange of knowledge and expertise, and by developing standardised, comparable frameworks for measuring outcomes. It can also use its convening role to support alignment on guidance for the targeting, design and implementation of AI regulatory sandboxes. By grounding this work in empirical evidence and practical experience, the OECD can help strengthen the overall evidence base and inform more effective policy approaches.

The authors would like to thank Natalie Cohen, Lucia Russo, Guillermo Hernandez, Xavier Pearson, Viktor Samek and John Leo Tarver for their contributions to this piece.

The post Why AI Sandboxes matter for responsible innovation and public trust appeared first on OECD.AI.

I’m a little late to this model review, but that has given me more time to think about the axes that matter for agents. Traditional benchmarks reduce model performance to a single score of correctness – they always have because that was simple, easy to quickly use to gauge performance, and so on. This is also advice that I give to people trying to build great benchmarks – it needs to reduce to one number that is interpretable. This is likely still going to be true in a year or two, and benchmarks for agents will be better, but for the time being it doesn’t really map to what we feel because agentic tasks are all about a mix of correctness, ease of use, speed, and cost. Eventually benchmarks will individually address these.

Where GPT 5.4 feels like another incremental model on some on-paper benchmarks, in practice it feels like a meaningful step in all four of those traits. GPT 5.4 in Codex, always on fast mode and high or extra-high effort, is the first OpenAI agent that feels like it can do a lot of random things you can throw at it.

I haven’t been particularly deep in software engineering over the last few months, so most of my working with agents has been smaller projects (not totally one-off, but small enough where I’ve built the entire thing and manage the design over weeks), data analysis, and research tasks. When you embrace being agent-native, this style of work entails a lot of regular APIs, background packages (like installing and managing LateX binaries, ffmpeg, multimedia conversion tools, etc), git operations, file management, search etc. Prior to GPT 5.4, I always churned off of OpenAI’s agents due to a death by a thousand cuts. It felt like rage quits. I’d feel like I was getting into GPT 5.2 Codex, but it would fail on a git operation and have me (or Claude) need to reset it. Those hard edges are no longer there.

The other subtle change in GPT 5.4’s approachability – the biggest reason I think OpenAI is much more back in the agent wars – is that it just feels a bit more “right.” I classify this differently to the routine tasks I discussed above, and it has to do with how the product (i.e. the model harness) presents the model outputs, requests, and all that to you the user. It has to do with how easy it is to dive in. This has always been Claude’s biggest strength in its astronomical growth. Not only has Claude been immensely useful, but it has a charm and entertainment value to it that’ll make new people stick around. GPT 5.4 has a bit of that, but the underlying model strengths of Claude still leave it feeling warmer.

Where Claude is a super smart model, with character, a turn of phrase in a debate, and sometimes forgetting something, OpenAI’s models in Codex feel meticulous, slightly cold, but deeply mechanical. I’d use Claude for things I need more of an opinion on and GPT 5.4 to churn through an overwhelmingly specific TODO list. The instruction following of GPT 5.4 is so precise that I need to learn to interact with the models differently after spending so much time with Claude. Claude, in some domains, you come to see has an excellent model for your intent. GPT 5.4 just does what you say to do. These are very different philosophies of “what will make the best model for an agent”, Claude will likely appeal to the newcomers, but GPT 5.4 will likely appeal to the master agent coordinator that wants to unleash their AI army on distributed tasks.

Share

Outside of charm, and dare I say taste, a lot of the usability factors are actually better on OpenAI’s half of the world. The Codex app is compelling – I don’t always use it, but sometimes I totally love it. I suspect substantial innovation is coming in what these apps look like. Personally, I expect them to eventually look like Slack (when multiple agents need to talk to eachother, under my watch).

OpenAI also natively offers fast mode for their models with a subscription and very large rate limits. I’ve been on the $100/month Claude plan and $200/month ChatGPT plan for quite some time. I’ve never been remotely close to my Codex limits with fast mode and xhigh reasoning effort, where I hit my Claude limits from time to time. There’s definitely a modeling reason to this – most of OpenAI’s release blogs showcase each iterative model being substantially more concise in the number of tokens it takes to get peak benchmark performance. This is a measure of reasoning efficiency. This 2D (or more) benchmark picture is exactly where the world is going.

Here’s a plot from Cursor, which sadly doesn’t have all the GPT 5.4 reasoning efforts, but it confirms this point in a third party evaluation. What is missing across model families is the speed and price (a proxy for total compute used) to get there.

The final benefit of GPT 5.4, and OpenAI’s agentic models in general for that matter, is much better context management. In using them regularly now I feel like I’ve never hit the context wall or context anxiety point. The reasoning efficiency I suspect is the case above just lets the model do way more with its initially empty context window. Then, when GPT 5.4 does compact, it’s been less noticeable.

The one problem I’ve been having with both Claude Opus 4.6 and GPT 5.4 is a light forgetfulness. If you give the models multiple TODOs in a single message outside of planning mode, I find them often dropping them. Sometimes it feels like the models glitch and try to solve a previous problem rather than the recent ones. I’m not sure what in the model or the harness is the exact cause, but sometimes I like to queue up a few messages as I see the model working on something, to refine the task, but currently this tends to be a pretty risky outcome except in the simplest use-cases.

These days I’ve been using both GPT and Claude extensively, mostly based on my mood, and have been getting more done than ever. Having a GPT 5.4 Pro integration directly with Codex, e.g. like \ultrathink, would be a big differentiator for OpenAI. Those models have been incredible.

All in, I see GPT 5.4 as an agentic model that brings a ton more simple usability and “agentness” to the very strong software foundation of GPT 5.3 Codex. It’s a big step, and I’m unbelievably excited for which of these two companies releases an update next. On paper, listing the strengths of GPT 5.4 across better top end coding performance, better speed, better context management, better rate limits, it’s a testament to how nuanced choosing a model is. I genuinely still enjoy Claude a bit more for ways that’ll never show up on benchmarks. This makes me type claude into my terminal at the start of my day, rather than codex.