Thibault Sottiaux, who leads core products at OpenAI, believes that today’s version of Codex will seem outdated before the year ends.
Sottiauxposted on X late Monday, “Given some of the results I’m seeing recently, it’s pretty clear Codex is a good harness.” He continued, “But it will seem primitive in 2-3 months and we’re about to go through another major evolution in how we use AI at the frontier.” He also said, “The next generation of models need more than your laptop.”
“It will seem primitive in 2-3 months and we’re about to go through another major evolution in how we use AI at the frontier.”
Given some of the results I'm seeing recently, it's pretty clear Codex is a good harness.
But it will seem primitive in 2-3 months and we're about to go through another major evolution in how we use AI at the frontier. The next generation of models need more than your laptop.
Sottiaux did not share details about OpenAI’s plans for the coming months. However, his comments are timely since the company is already working to move Codex beyond tasks limited to a developer’s computer. Since launching a new GPT-5 model for Codex in early July and surpassing 8 million users shortly after, the product has been evolving quickly.
Ona fills the infrastructure gap
In June, OpenAI said it plans to buy Ona, a company that creates secure cloud development environments. OpenAI called this deal part of the “next phase of Codex,” where agents can keep working in a customer’s cloud even after the laptop that started the job is closed.
“The next generation of models need more than your laptop.”
Codex currently uses cloud infrastructure, but it might still need the developer’s laptop to access projects and run tools. If the laptop goes offline, the agent may lose what it needs to keep working.
OpenAI has already tested this approach. In an experiment published in February, Codex worked for about 25 hours straight, used around 13 million tokens, and generated about 30,000 lines of code while building a design tool from scratch. Alibaba has pushed even further — its Qwen3.8-Max agent recently coded autonomously for 16 days, producing 265 commits with zero human help. Ona could help solve this problem.
The company, which used to be called Gitpod, creates cloud environments that can be set up with the tools and dependencies needed for each project. OpenAI said Ona has helped 2 million developers use these environments.
Agents need persistent workspaces
If the acquisition goes through, Ona’s technology would let Codex have a permanent workspace in a customer’s cloud. Agents could get the context and tools they need for a task without relying on an active session on a local machine.
OpenAI says companies will still decide how Codex works in their cloud environments, including what sensitive systems it can access. The deal is not final yet, so OpenAI and Ona are still separate companies.
It is not clear if Sottiaux’s prediction is truly related to Ona. Although the acquisition shows OpenAI is looking beyond just the model, because for Codex to work on its own, it needs an environment that stays online even when the developer’s laptop is off.
Unfortunately, moving the execution environment to the cloud solves one problem but creates many new ones.
Security risks grow with access
Letting a coding agent have full access to a company’s network or a developer’s credentials is undoubtedly risky. OpenAI said Ona’s customer-controlled model will let agents work inside an organization’s own cloud, while OpenAI provides the model and orchestration. Even if the model gets stronger and can handle more complex tasks, it still needs a secure place to run commands, save its progress, and interact with other systems.
Developers can assign tasks like refactoring, upgrading dependencies, or investigating bugs to the agent and let it work remotely. They can track its progress, check terminal output, and step in if a human decision is needed. When the agent finishes, users can review the pull request and see which tests were run.
Agent environments will use computing resources along with CI/CD systems.
Managing a new agent layer
This change means there is a new type of infrastructure to manage. Anthropic is already moving on this front — its acqui-hire of Mendral is aimed at automating CI/CD tasks like flaky tests and dependency reviews directly inside its platform. Agents will need their own identities and access rules, and their actions will need to be logged, reviewed, and linked back to them, just like with human developers and current automation.
Sottiaux’s prediction certainly has provoked curiosity. Two or three months is a very short time for a product to become “primitive.”
Automation and robotics have always been about combining reliable hardware with increasingly capable software. Over the last few years, the software half of that equation has shifted decisively toward artificial intelligence. Vision systems that read labels and detect defects, natural-language interfaces that let operators query a line in plain English, predictive models that flag a […]
DeepSeek has launched DeepSeek-V4-Flash-0731, delivering a significant boost in agent performance without changing the model’s core architecture.
Following an announcement last week, the company made the update available as a public beta through DeepSeek’s API, and the open weights were published on Hugging Face under the MIT license later the same day.
Although the model itself hasn’t changed, DeepSeek says additional post-training is responsible for the performance gains, showing that meaningful improvements don’t always require a larger model.
🚀 DeepSeek-V4-Flash Official API is now LIVE in public beta!
🔷 We’ve massively upgraded its Agent capabilities—benchmark scores are now far surpassing the V4-Pro-Preview. Check out the massive performance leap below! 👇 🔷 The official V4-Flash now natively supports the… pic.twitter.com/NUzOyxza2f
DeepSeek’s decision to release the production-ready weights under a permissive license gives organizations much more control over how they deploy and customize the model.
Although the model itself hasn’t changed, DeepSeek says additional post-training is responsible for the performance gains, showing that meaningful improvements don’t always require a larger model.
Same architecture, better results
DeepSeek says V4-Flash-0731 uses the same architecture as the preview release, with 284 billion total parameters and 13 billion activated parameters per token.
This is much smaller than V4-Pro, which has 1.6 trillion total parameters and 49 billion activated parameters. For companies running agents at scale, the activated-parameter gap translates directly into inference cost — though model price alone doesn’t tell the full story.
Even though it is still the smaller model, DeepSeek says the updated Flash version now beats the earlier V4-Pro preview on several agent-focused benchmarks.
Even though it is still the smaller model, DeepSeek says the updated Flash version now beats the earlier V4-Pro preview on several agent-focused benchmarks.
The company reported 82.7 on Terminal-Bench 2.1, 54.4 on DeepSWE, and 70.3 on Toolathlon-Verified.
Benchmark claims under scrutiny
Early independent testing by Artificial Analysis found a lower Terminal-Bench 2.1 score of 79%, which suggests that DeepSeek’s reported numbers may not always match independent results.
DeepSeek also shared results from several internal tests, though they have not yet been independently verified. If those results hold up, they add to growing evidence that companies can get more performance out of existing models through post-training instead of simply making them larger.
Open weights, full control
The MIT license means organizations aren’t limited to using DeepSeek through its hosted API. The release adds to a trend towards open-weight models closing the gap with proprietary alternatives. That flexibility is paired with support for tools many developers already use. V4-Flash now supports the Responses API for building AI agents and multi-step workflows, and DeepSeek has published instructions for integrating the model into Codex-based development workflows.
Familiar APIs, lower switching costs
For teams already using OpenAI-style APIs, that lowers the barrier to trying another model because they can evaluate it without making major changes to their existing setup.
The V4 technical report also covers inference improvements, like speculative decoding with DeepSeek’s DSpark framework, which are designed to make serving more efficient. When combined with self-hosted deployments, these features give infrastructure teams more ways to adjust performance for their own production needs.
This release reflects that companies are now finding new ways to improve model effectiveness without making models larger, and they’re competing on how those models are delivered. While many AI vendors focus on hosted APIs, DeepSeek continues to publish downloadable weights that organizations can run on their own infrastructure. Support for familiar API formats also makes it easier for teams to test open-weight models without revamping present workflows.
Companies are finding new ways to improve model effectiveness without making models larger, and they’re increasingly competing on how those models are delivered.
Cloud platform company Nscale announced this week a definitive agreement to acquire AI workload scaling specialist Anyscale, in a move that signals a new test of whether cloud-neutral AI software can stay neutral once it is paired with a GPU neocloud.
The purchase coalesces Nscale’s infrastructure capabilities, which span control systems that oversee GPUs, datacenters, power consumption, and the application layer where AI services themselves are executed, with Anyscale’s software layer for scaling AI workloads across data processing, training, inference, and reinforcement learning.
Argued by Nscale to be the coming together of “two highly complementary companies”, Nscale scooping up Anyscale could be a fundamental change in the resulting business model.
Is this the start of GPU neocloud lock-in?
It’s important to remember that Nscale is a GPU neocloud (a specialized cloud provider running bare-metal GPUs and infrastructure optimized for AI and machine learning workloads), meaning that it runs its own GPU-rich datacenters and its own software stack. At the same time, Anyscale is an independent cloud-neutral software orchestration multi-cloud control plane that works with any cloud hyperscaler… but now owned by a single neocloud.
That doesn’t sound quite so much like cloud-neutrality and agnosticism; it sounds more like a vertically integrated AI cloud provider proposition.
Chief product officer at Nscale, Dan Bathurst, tells The New Stack that the Anyscale platform “continues to be its own brand and product,” and that includes working with bring-your-own-cloud deployments on AWS, GCP, Azure, and the other clouds.
“Where we want to win is on performance, not on any sort of vendor lock-in or forcing of someone to choose Nscale as the infrastructure provider.”
“But what really changes — or how it’s changing — is that customers now also get this first-party option, where they can have Anyscale running on Nscale fleet as a full-stack, highly-optimized solution. Where we want to win is on performance, not on any sort of vendor lock-in or forcing of someone to choose Nscale as the infrastructure provider,” Bathurst says.
He insists that it is in Nscale’s interest to ensure that it is making it easy for software engineering teams to get the outcomes they want with the workloads that they’re trying to run.
“For us, the existing commitments will carry forward, so Nscale’s value really is meeting instances where the compute already lives,” he says. “Where we want to win is on performance, not on any sort of vendor lock-in or forcing of someone to choose Nscale as the infrastructure provider.”
Neutrality on the platform layer, differentiation on the infrastructure layer
Bathurst invites users to think of it as “neutrality on the platform layer, but differentiation on the infrastructure layer” because the combination of the two organizations is a full-stack play.
“The differentiation comes from the fact that Nscale is fully vertically integrated with Anyscale. Therefore, if users want that first-party option, they can choose Anyscale and get the most optimized solution because, obviously, we’re designing, optimizing, and co-engineering every layer of that stack from power to the datacenter through to the application. It’s quite a unique proposition, but it’s not something we are going to force upon any customer,” confirms Bathurst.
Not everyone is convinced by the company’s pledge to maintain an agnostic and neutral open house. Sanjeev Mohan, principal analyst, SanjMo and former Gartner research VP for data and analytics, tells The New Stack that Anyscale “stops being a neutral player” the moment its best features and most optimal pricing land on Nscale first.
“The software will still run anywhere, but ‘runs anywhere’ and ‘runs best somewhere’ are different things, and buyers will feel the gap in performance and cost. At that point, neutrality is a label.”
Runs anywhere, but… runs best somewhere
“The software will still run anywhere, but ‘runs anywhere’ and ‘runs best somewhere’ are different things, and buyers will feel the gap in performance and cost. At that point, neutrality is a label,” says Mohan.
He agrees that integrating software and compute will produce measurable cost, performance and reliability gains. Defining this as “the strongest part of the deal”, Mohan explains that with Nscale controlling both the silicon and Anyscale’s control plane, it can tune scheduling, memory, and networking together in ways the compute-neutral Anyscale never could.
Anyscale commercial support for Ray
Anyscale was founded by the creators of Ray, an open source project that provides a distributed computing framework designed to scale Python workloads across any infrastructure into live production application jobs and services.
Ray was donated to the PyTorch Foundation in 2025. Anyscale continues to provide its commercially supported services for Ray, which include a “no DevOps” route to 100% managed cloud infrastructure and serverless autoscaling, making it simpler to create, deploy, and monitor machine learning workflows in production.
Anyscale supports data processing, model training, batch inference, and LLMs across public and private cloud environments. As open source as this all feels, are we still edging towards narrower proprietary channels, or the possible threat of deeper application and data service dependencies that developers will ultimately have to wrangle around?
“I don’t think so, primarily because the way that the platform works, it’s designed to orchestrate across various different clouds and different infrastructure. It’s like a heterogeneous distributed compute platform. So the platform’s always gonna remain multi-cloud,” confirms Nscale’s Bathurst.
Pricing permutations and hyperscalers hearsay
Pressed on any forthcoming pricing changes or likely reactions from the major cloud hyperscalers in relation to Nscale now being a credible alternative, Bathurst and team were (perhaps understandably one day after an acquisition deal announcement) politely tight-lipped.
More voluble is always-affable analyst Mohan, who says that, “Every optimization that only shows up on Nscale hardware is a dependency. So, an argument can be made either way. Standalone orchestration software and independent tooling vendors are getting absorbed into whoever owns the GPUs, because the economics only work when you control both. Expect more of it,” Mohan underlines.
He explains that Nscale “now becomes a real specialist cloud services provider alternative,” i.e., not a general-purpose one like AWS, Azure and Google Cloud with their plethora of managed services, from databases and data warehousing to container orchestration through to AI/ML pipeline technology. However, he does see space for Nscale to become a strong player in raw training and inference at scale.
From cryptocurrency to cloud contender
London, UK-based Nscale was established in 2024 from what was originally a cryptocurrency mining business.
As suggested, Anyscale will retain its brand name as part of the Nscale family, and the company has restated its stance that customers are “free to choose the cloud infrastructure on which they run their AI workloads” today.
The company’s initial press statement said that “over time” users will gain the additional option of running the Anyscale software layer on Nscale’s full-stack AI platform.
The first full-stack AI hyperscaler?
“Companies are moving beyond simply using AI to actually building their own. Doing that well requires the software and the infrastructure it runs on to be designed together,” says Keerti Melkote, CEO of Anyscale in the press release announcing the acquisition.
Melkote has defined the combination of Anyscale’s platform — built on Ray — with Nscale’s datacenter, compute and AI cloud services as the “first full-stack AI hyperscaler,” i.e., one that runs any AI workload at greater scale, so more software engineering teams can build and own their AI applications and services.
With this acquisition and the fusion of Nscale with Anyscale’s software layer, the organization will aim to widen its customer base. Existing work sees the company working in verticals from healthcare to e-commerce to robotics. It says its full stack offering will help companies speed up image and document processing, fine-tune LLMs on their proprietary data, and deploy AI agents in-house using open-source models.
The transaction is subject to closing conditions and regulatory approvals and is expected to close in the second half of 2026. Financial terms of the transaction were not disclosed, although Reuters reports a source stating that the deal price is “about $1.65 billion”, according to a person familiar with the deal.
AWS, Google Cloud and Microsoft Azure representatives were all contacted and invited to comment on this story.
Blockchain is gradually finding a more pragmatic place in the digital economy. After years of bold predictions, the focus has shifted from “revolution” to practical application. Today, companies are asking a simpler question: in which cases does distributed infrastructure genuinely help solve real operational challenges? Most often, the answer involves situations where multiple parties need […]
A property manager calls about a 1958 mixed-use building downtown. The utility wants time-of-use metering installed by next quarter, the tenants want their AC to keep running through the switchover, and nobody on the ownership side wants to hear the words “full rewire”. This is, more or less, how most smart grid retrofit jobs start. […]
OpenAI has lowered API prices for two GPT-5.6 models only three weeks after their launch. On Thursday, the company announced that GPT-5.6 Luna is now 80% cheaper and GPT-5.6 Terra is 20% cheaper, while the price for its main reasoning model, GPT-5.6 Sol, stays the same.
“Major price cuts today,” OpenAI CEO Sam Altman writes in a post on X published on Thursday. “We want to offer the best price/intelligence tradeoff at every level.”
“We want to offer the best price/intelligence tradeoff at every level.”
Luna now costs $0.20 for a million input tokens and $1.20 for a million output tokens, down from $1 and $6. Terra is priced at $2 per million input tokens and $12 per million output tokens, reduced from $2.50 and $15. Sol’s price stays at $5 per million input tokens and $30 per million output tokens.
Developers using Luna do not need to change their processes, but their inference costs will go down. High-volume tasks will now be much cheaper to run, without requiring any code updates or model changes.
major price cuts today:
*80% drop for GPT-5.6 Luna, now $0.20 per million input tokens and $1.20 per million output *20% drop for GPT-5.6 Terra, to $2/$12 *GPT-5.6 Sol gets Fast mode in the API, up to 2.5x the speed for 2x the price, same intelligence pic.twitter.com/erC6u4VoDR
This timing is unusual because AI vendors usually keep prices steady for several months after launching a new model family. OpenAI cut prices less than a month after GPT-5.6 became available on July 9.
…serving costs can be more important than small differences in benchmark performance between models.
Infrastructure gains drive savings
The company says these price cuts were possible because of improvements to the infrastructure behind GPT-5.6, which lets the company offer “substantially more intelligence per dollar.”
OpenAI engineers rewrote the production GPU kernels, cutting serving costs by about 20%. They also redesigned Sol’s speculative decoding system, making token generation over 15% more efficient. The company updated its agent runtime as well, reducing repeated prompt computation by using prompt caching more during multi-step workflows.
The elephant in the room is that the competition has intensified from overseas. Lower-cost open-weight models from Chinese AI companies like Moonshot are pushing commercial providers to show not just better performance, but also better pricing for production use. OpenAI and Anthropic know that leaning on performance just isn’t an option anymore, which is pushing them to match Chinese prices.
The issue here is that most of those steps don’t need a model like Sol, and Chinese labs have figured out how to pack better capabilities into efficient models; a helpful option for companies running through billions of tokens a day.
The ability to send the easy tasks to open models and save the pricey APIs for the tough stuff makes a difference; OpenAI is banking on its 80% price cut on Luna to narrow that gap. Suddenly, switching to self-hosted models doesn’t look worth the hassle.
This announcement highlights a trend in the industry for infrastructure. Now, every percentage point of serving efficiency can lead directly to lower API prices, turning cost optimization into a competitive advantage instead of just an engineering goal.
Global demand for digitalization solutions in industry and the energy sector continues to grow. COPA-DATA is addressing this demand with its zenon software platform. Key drivers include industrial digitalization, the need for resilient supply chains, and the sustainable expansion of the energy sector. Skills shortages in these industries and the drive to remain competitive in […]
Fast-growing startup moves headquarters to Hazelwood Green and opens expanded manufacturing operations at Mill 19, creating over 500 local jobs Hellbender, a physical AI infrastructure company powering intelligent systems at the edge, today announced it has moved its global headquarters to the Roundhouse at Hazelwood Green and will open expanded manufacturing operations at nearby Mill […]
Tennant’s 2026 Sustainability Report details progress on renewable electricity, product emissions and technologies designed to reduce water, energy and chemical use Commercial cleaning could be one of the most overlooked opportunities for businesses to reduce their environmental footprint, according to new data released by Tennant Company. Publishing its 2026 Sustainability Report, the global cleaning technology […]
Qualcomm has announced that it has completed its acquisition of Modular Inc, an innovator in AI-native software infrastructure. Modular’s software platform gives developers a unified way to optimize and deploy generative and agentic AI workloads across heterogenous computing systems. Combined with Qualcomm Technologies’ leadership in high-performance, energy-efficient compute, Modular strengthens the company’s ability to deliver […]
OpenAI has detailed how the GPT-5.6 model family balances capability and cost across its stack, and the company‘s most important claim is a benchmark result showing that its flagship model, GPT-5.6 Sol, with maximum reasoning, outperforms Claude Fable 5 from Anthropic on the Artificial Analysis Coding Agent Index. The margin comes with 54% fewer output tokens. The findings were shared in a company blog post on Wednesday.
For developers, what matters most is how OpenAI arrived at the benchmark results and the role GPT-5.6 Sol played in optimizing the infrastructure that now serves it.
The family spans three models across the price curve. In addition to Sol, there is Terra, which performs as well as GPT-5.5 on intelligence benchmarks at half the price, and Luna, the fastest and most affordable, which is priced 80% below Sol.
The efficiencies come from optimizations at four layers, spanning the models, inference, the API stack, and the agentic harness behind Codex and ChatGPT Work.
According to the post reviewed by The New Stack ahead of its publication, the efficiencies come from optimizations across four layers: models, inference, the API stack, and the agentic harness behind Codex and ChatGPT Work. The architecture diagrams in the post draw the same separation as three planes: the local harness, CPU-bound API orchestration, and GPU-bound model inference.
Source: OpenAI
For developers building and operating agents, the post is worth reading less as a product announcement and more as a systems paper. Nearly every technique it describes, from incremental tokenization to append-only context, applies to any team running a tool-calling loop at scale.
A model that rewrites its own serving code
The efficiency work starts in training. OpenAI says GPT-5.6 is trained to achieve more work per token, with training optimized for both task success and efficiency so the model takes a more direct path through a task.
With Codex, GPT-5.6 Sol autonomously rewrote and optimized OpenAI’s production kernels, the core code that executes the mathematical operations making up the model. OpenAI says this worked in part because GPT-5.6 is trained to write and improve kernels in Triton and Gluon. Both are open-source GPU programming languages maintained by OpenAI. These efforts, combined with broader kernel advancements from the model, reduced end-to-end serving costs by 20%.
Correctness is the obvious concern when a model rewrites the code it runs on. To address it, OpenAI reports heavy investment in verification tooling. That includes the open-source Floating-Point Sanitizer (FpSan), which validates the kernels GPT-5.6 Sol produces before they reach production.
The model went further with speculative decoding, a technique in which a smaller draft model proposes several tokens that the primary model verifies in parallel. The approach will feel familiar to anyone who understands how modern CPUs speculatively execute instructions ahead of a branch. Accepted proposals produce multiple output tokens from a single pass of the primary model. That reduces the expensive sequential computation the primary model would otherwise perform.
GPT-5.6 Sol in Codex improved its own draft model by designing and running hundreds of experiments on its architecture, with changes tested across size, structure, and features. The model also launched and monitored the speculative training process. It intervened autonomously when hardware failed or training became unstable. OpenAI reports the resulting improvements lifted token-generation efficiency by more than 15%.
More tokens from the same GPUs
OpenAI frames its inference work around a single objective – serving more tokens with the same hardware while preserving the intelligence, latency, availability, and reliability users expect. In a compute-constrained market where demand grows faster than capacity, that objective influences every design decision in the serving path.
Load balancing operates at three distinct levels. Globally, requests are routed based on geography, available capacity, and accelerator type. Within a cluster, work is distributed across model instances based on load, context length, and cache availability. Within each instance, work is partitioned across accelerators, the model’s experts, and computing cores. GPT-5.6 Sol in Codex helps OpenAI analyze production traffic and identify previously overlooked sources of imbalance. The same loop tests new routing strategies and helps engineers constantly tune the heuristics. OpenAI states that these load-balancing improvements alone dramatically reduced the cost of serving its models.
The key-value (KV) cache received the same treatment. When processing uncached input tokens, the model builds the KV cache in a single compute-intensive pass, then repeatedly reads from and extends it during generation. The optimal serving configuration depends heavily on prompt length, batch size, and cache hit rate. It covers batching, sharding, and cache management, and the configuration space was previously too large to tune systematically. With GPT-5.6 Sol in Codex, OpenAI analyzed production workloads and generated candidate configurations. The company says this makes workload-specific optimization practical at a level that broad heuristics could not reach earlier.
Process only what changed
The API team focuses on everything that happens around a model call. After a prompt is submitted, the API stack receives the request, loads context, and validates the input. Safety checks run next, and the text is converted into tokens for inference. OpenAI measures this overhead through time to first token (TTFT), time between tokens (TBT), and end-to-end time (E2E).
Tokenization is an O(n) operation, so longer prompts take longer to process. Codex would send the full conversation context after every tool call. That meant paying to tokenize the same conversation dozens of times per turn, even though only a small amount of context was new in each request. OpenAI solved this with a WebSocket integration that hoists tokenization state to the server. The first call renders and tokenizes the full prompt. Later calls send only the new input with a reference to the conversation, bringing the operation closer to O(1). The pattern mirrors an incremental build system that recompiles only the files that changed rather than the whole project.
These savings compound in tool-heavy workflows, where every tool result triggers another round trip through the API. For rollouts with 20 or more tool calls, OpenAI reports up to roughly 40% faster end-to-end execution.
Hardware turned out to matter as much as protocol design. All of OpenAI’s infrastructure runs on Kubernetes. The company found that nodes with the same instance type often carried different CPU generations, with many running outdated processors. In its measurements, the older processors consumed roughly twice the CPU resources for the same work. Reweighting traffic toward newer processors improved TTFT by about 20%, and CPU generation is now part of capacity planning.
OpenAI names four fates for application-layer overhead: delete it, overlap it with useful work, run it on faster hardware, or make the code consume fewer CPU cycles. Its asyncio changes move work off the critical path, while newer hardware and Rust implementations make the remaining work faster and more predictable.
An append-only harness
The agentic harness is a Rust-based orchestration layer that connects the models, tools, and the user’s environment. In a single turn, Codex might inspect source code, search deployment history, and read incident reports. Editing a file and running the tests each add another request. Since a task can require 30 model requests, an extra second per request adds up quickly.
Context bloat is the first target for the harness. As agents gain access to more tools, skills, plugins, and conversation history, context windows expand. The growth increases cost, distracts the model, and prompts unnecessary reasoning. The harness counters this with deferred discovery, which surfaces integrations, custom Model Context Protocol (MCP) tools, skills, and plugins only when needed. Tool output is capped at 10,000 tokens by default unless the model requests a different limit.
Prompt caching drives the second design choice. An agent loop resends the same instructions, tool definitions, and earlier results multiple times within a turn. The harness therefore treats all model-visible history as append-only, with new messages and tool results added at the end rather than inserted into earlier context. Tools are presented in a deterministic order, and runtime settings, such as approval policies, are applied during execution rather than embedded in tool definitions. OpenAI credits this design for the high prompt-cache hit rates in Codex and ChatGPT Work.
Source: OpenAI
Platform teams building internal agents can adopt every one of these choices without OpenAI’s scale. Append-only context, deterministic tool ordering, and capped tool output attack token spend directly. That makes them the most portable lessons in the post for enterprises watching inference bills grow with each new agent deployment.
Where the gains come from
The post associates a number with most of its optimizations, and the figures are OpenAI’s own production measurements. Taken together, they show how modest individual wins compound across a serving stack.
Layer
Technique
Claimed gain
Model inference
Autonomous kernel rewrites in Triton and Gluon
20% lower end-to-end serving costs
Model inference
Speculative decoding with a self-improved draft model
Over 15% better token-generation efficiency
API stack
Stateful WebSockets with incremental tokenization
Up to roughly 40% faster runs at 20+ tool calls
API stack
Routing traffic toward newer CPU generations
About 20% better time to first token
Agent harness
Deferred discovery and a 10,000-token tool output cap
Reduced context bloat and cost
The key takeaways
In summary, OpenAI describes the GPT-5.6 efficiency gains as the result of years of compounding improvements. They span research, inference, the API stack, and the agentic harness. The company states that the model’s role in landing many of them makes it optimistic that the pace of optimization will accelerate. Kernel work is called out as an area of continued investment.
The post positions efficiency, alongside raw intelligence, as the axis on which frontier labs now compete. The claimed 54% output-token advantage over Claude Fable 5 shows how OpenAI intends to fight that battle. The engineering blog makes a plausible case that software optimization is becoming an important lever alongside hardware improvements in reducing the cost of serving frontier models. The figures remain OpenAI’s own production measurements. The autonomy on display operates within Codex, with engineers in the loop. Developers and enterprises benefit either way, as these under-the-hood improvements reach them as more capable models at lower prices across the cost-intelligence curve.
As more companies plug AI agents into the deepest depths of their internal data banks, how can they be sure those agents actually understand how the business works? Right now, many of these organizations are stuck manually building a Markdown file, hoping they find time to rewrite it each time the business changes.
Modus, for its part, thinks it has found a better way. The startup that formally exits stealth this week with $10 million in funding in tow is building what is coming to be known in industry parlance as a “context warehouse” — a layer that sits alongside a company’s existing data warehouse, continuously mapping how the business operates across its systems, and handing an AI agent only the relevant slice of that map when it needs it.
In real terms, Modus crawls relevant assets from sources like GitHub, dbt, Jira, Snowflake, and Postgres, using what it calls a Context Miner to continuously learn how the business operates. What it finds gets turned into “dynamically generated skills”: Short, purpose-built briefs, assembled in real time by a second system, the Context Composer, and handed to an agent the moment it’s given a task.
Modus co-founder and CTO Tomer Mesika tells The New Stack that this mining runs continuously, guided by its own internal logic for what to check and how often.
“We have a lot of mechanisms in place to know what to mine from the organization, at what cadence, how to look for deltas, when to dive deeper in, and when not to,” Mesika says.
“We have a lot of mechanisms in place to know what to mine from the organization, at what cadence, how to look for deltas, when to dive deeper in, and when not to.”
Daniel Shimoni, Modus co-founder and CEO, draws a direct line to data warehousing to highlight the gap he’s trying to close. Companies have spent years building infrastructure to store and organize their data, he argues, but nothing equivalent exists for the understanding that sits atop it.
“There’s a logic behind data warehouses — companies already know that is where they manage their data,” Shimoni tells The New Stack. “But where do they manage their context? Where do they actually understand what contexts exist in their organization, that they can actually use to ensure agents only have what they need?”
Modus founders Tomer Mesika (CTO) and Daniel Shimoni (CEO).
Shimoni says even that first step is hard enough on its own. But keeping a company’s context accurate as the business changes is harder still.
“We’ve noticed that building the context the first time is already a challenge, but maintaining it is the bigger issue,” Shimoni says. “So Modus always learns from what the company is doing, and whenever something shifts or changes in the business, it makes sure that only the relevant and updated context is fed to agents.”
“Building the context the first time is already a challenge, but maintaining it is the bigger issue.”
Who’s buying, and why cost matters
Shimoni says Modus is targeting engineering teams, the CTO office, and VPs of R&D, as well as data teams and a newer category of AI teams.
“AI teams weren’t really around last year; it seems that a lot of data teams are transitioning to becoming VP of data and AI, or AI enablement,” Shimoni says. “So really, it’s the people who are in charge of having this AI enablement mandate in the organization, making sure AI is scaled in the organization.”
“You want the bigger models to do the heavy and complex tasks to get great value. The problem is that they are wasting a lot of their effort and a lot of their token usage on menial tasks.”
Mesika says this is a central component of Modus’s modusoperandi, arguing that frontier models end up spending a chunk of their token budget on work unrelated to actually answering a question.
“You want the bigger models to do the heavy and complex tasks to get great value,” Mesika says. “The problem is that they are wasting a lot of their effort and a lot of their token usage on menial tasks.”
Those menial tasks, in Mesika’s telling, include combing through pull requests or Jira tickets just to determine what’s relevant before an agent can start the job it was assigned to.
One approach to this problem is to hand the sorting work to a smaller, cheaper model. Mesika says Modus takes that further: rather than retrieving that context at the moment a question is asked, it uses small language models alongside search engines, vector search, and a graph database, all built up in advance, to do that work continuously in the background. By the time an expensive frontier model gets involved, it’s only ever handed a finished brief of exactly what it needs.
Modus dashboard
“Everyone’s talking about context”
Shimoni and Mesika both come from data-centric companies — Lusha, a go-to-market data platform, and Cyera, a cybersecurity data company, respectively — before leaving their roles in September 2025 to start Modus together.
The two had known each other for years, and spent much of the previous year comparing notes on a problem they were both running into in very different jobs.
“We decided this is a problem worth solving, and it seems like we were spot on, because everybody’s talking about context.”
“Some of the challenges were very similar — how do we combine a lot of various data assets into one place where AI can work?” Shimoni says. “We just started to notice that this is the gap — to make AI run with confidence, at scale, across a company. We decided this is a problem worth solving, and it seems like we were spot on, because everybody’s talking about context.”
Modus closed a hitherto unannounced $10 million seed round shortly after founding, led by Insight Partners. Other backers include Soma Capital and a handful of angel investors, among them founders from Cyera and Wix.com. The company began hiring its first employees in January 2026.
The broader takeaway from Modus’s pitch is now among the most common refrains emanating from AI circles this year: that the model itself is no longer the bottleneck; what limits an AI system now is everything built around it. And for Modus, that realization has been more or less present since its inception.
“Even last year […] we could already see that model capabilities weren’t the bottleneck,” Shimoni says. “It was more making sure that they actually have access to the context they need in order to give you the right answers.”
Bright Machines wants to solve one of the least glamorous but most consequential problems in the AI buildout: what happens to quality data when a human being has to touch the production line.
The San Francisco-based manufacturer announced today the Hybrid BRC (Bright Robotic Cell), an expansion of its Bright Factory platform that lets human operators step inside a sensor-monitored robotic cell to perform prescribed assembly steps — without breaking the digital record that tracks every server from its first screw to its shipping label.
It sounds like an incremental hardware update. It isn't. The Hybrid BRC is a direct answer to a structural weakness in high-stakes electronics manufacturing — one that CEO Sviat Dulianinov quantified in stark terms in an exclusive interview with VentureBeat.
"If you assemble modern AI servers starting with manual operations, your initial yield — first-pass yield — can be as low as 20%," Dulianinov said. "Then you gradually ramp up and scale, and it can reach the 60s, 65% or so."
When a single AI server can cost hundreds of thousands of dollars, and hyperscalers are burning billions waiting for infrastructure they can't deploy fast enough, that number is the whole story. The Hybrid BRC is Bright Machines' attempt to keep human hands in the loop without letting human error back in the door.
Why manual assembly steps create a black hole in production data
Modern automated assembly lines generate a continuous stream of production data — torque values, placement coordinates, component serial numbers, inspection images. That "data thread" is what lets a manufacturer prove a server was built correctly and, when something fails in the field months later, trace the failure back to a specific station, step, or part.
But automated lines inevitably need manual intervention, and until now manufacturers had two bad options when that happened: stop the line entirely, or pull in-process units off to a separate manual workstation that sits outside the monitored data flow. The first choice kills throughput. The second punches a hole in the production record at precisely the moment when human error is most likely to occur.
The Hybrid BRC eliminates that tradeoff, the company says. The cell incorporates guarded access doors and safety panels directly into the production line. When an operator opens the doors, the robotic arm deactivates, and on-screen instructions guide the operator through each assembly step while the cell's sensor array — cameras, force feedback, and tooling sensors — continues monitoring for incorrect installs, missed steps, and wrong components, applying the same quality checks used during full automation. The traceability record persists at the serial-number level from start to finish.
The yield gap between humans and robots in AI server assembly
The economics driving the design become clear when Dulianinov's manual-assembly figures are set against what automation delivers. "At robotic operations, yield-per-station level is usually more than 98% with our technology, and even at the line level, we usually get to 97.5%, 97.7% or so," he said.
First-pass yield measures the percentage of units that come off the line correct the first time, without rework. The gap between a 20% manual ramp and a 98% automated station isn't a rounding error — it's the difference between profitability and disaster on hardware this expensive.
That math explains the company's design philosophy for the Hybrid BRC, which treats the human operator as an escape valve for exceptions rather than a substitute for automation. "The more human stations you introduce, the more you increase the risk of lower yields driving the overall yield down," Dulianinov said. "That's why we prefer to start at least with 50% automation, and then move to at least 80%." Speed follows a similar pattern: "On the line level, robots can be faster than humans from like 50 to 100%" in throughput terms, he said.
How server assembly became the hidden bottleneck of the AI infrastructure race
The AI infrastructure conversation usually revolves around chip supply, power availability, and data center construction. Dulianinov argues that assembly — the unglamorous work of turning chips and motherboards into racked, tested, deployable compute — is a quietly enormous drag on deployment timelines.
"When you have the chips and you have the motherboards, you want to be as fast as possible to deploy that in the data center," he said, describing greenfield deployments where power and buildings already exist. Getting hardware built, tested, and often rebuilt when quality falls short "could be months," he said. "With more technology used for this, as our tech, we believe that we can cut it by at least a third."
A company executive on the call added an anecdotal but telling data point: the servers Bright Machines produces are "flying out into production" rather than sitting stacked in warehouses awaiting deployment — evidence that assembly capacity, not just chips or power, gates hyperscaler timelines. The stakes are asymmetric, the executive noted, because the largest hyperscalers lose millions of dollars per day when servers fail or arrive late. That is why customers are less interested in buying boxes than in buying assurance — and why an unbroken data thread has become a product in its own right.
Inside the secretive customer base already running hybrid production lines
The Hybrid BRC is not vaporware. Dulianinov said the company already operates a number of the hybrid lines in the U.S. and has "built more than 10,000 compute nodes" through the new stations. This year, he said, Bright Machines plans to manufacture "more than half a gigawatt of compute capacity."
Who's buying? Don't ask. "We cannot unfortunately name customers. That's the toughest part of our job," Dulianinov said. "They're pretty secretive because, as you can imagine, everything data center related is IP related."
He did offer growth figures: customers grew "more than 3x this year" versus the prior year, driven by what he called the intersection of "physical AI, AI infrastructure buildout, and onshoring." The demand is spilling into real estate — the company is moving from its 16th Street San Francisco offices to a Burlingame space this fall that executives described as three to four times larger. Overall, the company says it has deployed more than 130 microfactories across 10-plus countries, served more than 60 customers, and produced more than 300,000 servers.
What separates Bright Machines from Tulip, Instrumental, and contract manufacturing giants
Asked how the Hybrid BRC's traceability claims stack up against operator-guidance and inspection software vendors like Tulip and Instrumental, Dulianinov drew a sharp line around business models.
"Tulip is just a company that does interface for operators. Instrumental, they focus on inspection. It's just pieces of the puzzle," he said. "We, as a technology-enabled manufacturer, we actually run this whole operation... We put our lines, put our software, put our data on the floor, our people, and run it from the beginning to the end."
The right comparison set, he argued, is contract manufacturing giants like Flex, Jabil, and Foxconn — companies that own the full production process but historically built it on manual labor that generates little data. Bright Machines' differentiation, he said, is that robot data, sensor data, and now human-station data all flow through one orchestration layer into a single environment the company calls Bright Insights.
That positioning is notable given the company's origins. Bright Machines was carved out of contract manufacturer Flex eight years ago, and its history has had turbulence: the company planned to go public in 2021 via a SPAC merger at a reported $1.6 billion valuation, according to contemporaneous reporting by The Wall Street Journal and CFO Dive, before the deal fell through. It rebounded in June 2024 with a $126 million Series C — $106 million in equity led by funds managed by BlackRock with participation from Nvidia, Microsoft, Eclipse, Jabil, and Shinhan Securities, plus $20 million in venture debt from J.P. Morgan — bringing its total raised past $400 million, per the company's announcement at the time.
Who owns the production data — and how workers feel about being monitored
For technical decision makers, two governance questions loom over any system that instruments human work this closely, and Dulianinov addressed both directly.
On data ownership, he drew a clean boundary: "Everything related to the customer and inspection of their devices and parts obviously would be protected and owned by the customer." Process and robotics data, he said, stays with Bright Machines to fuel continuous improvement across its platform.
On worker surveillance, he pushed back on the framing. High-IP electronics floors — especially those touching aerospace, defense, or government workloads — already prohibit workers from carrying personal electronics, he noted. "People who know those floors, they know that this is part of the game," he said, adding that employees "actually appreciate" the traceability because it underpins the security mission: "If you build a data center for the government, and then you build servers somewhere in China, you cannot guarantee how exactly it was built and what component was put there." In his telling, the monitoring isn't about watching workers — it's about being able to prove, component by component, that American-built AI infrastructure is what it claims to be.
The onshoring bet: rebuilding American manufacturing without 3 million workers
The Hybrid BRC's modular design carries strategic weight beyond quality assurance. Because the cells are software-defined and snap together like building blocks, Bright Machines says it can retool lines for new hardware generations in days or weeks rather than months — "we can introduce it within a day" for minor design changes within a product family, Dulianinov said, though a jump from air cooling to liquid cooling remains "a big jump." In an industry where new chip architectures now arrive on a roughly annual cadence, changeover speed is arguably as valuable as yield; a production line that takes six months to retool is obsolete before it amortizes.
But Dulianinov's closing argument was about labor arithmetic, not machinery. "We need to build in the U.S., and you don't have 3 million people to bring up manufacturing in the U.S.," he said, referencing the massive workforces of Shenzhen-scale electronics plants. "So you need to solve it with AI software and robots, and that's our thesis... It's not just robots on the floor — it's also creating jobs. All the robots, and some people on the floor."
Lior Susan, founder and CEO of Eclipse and chairman and co-founder of Bright Machines, framed the announcement in the same terms: "The future of manufacturing isn't choosing between automation and flexibility — it's combining both in the same digital production environment."
For all the talk of gigawatts and yield curves, the Hybrid BRC amounts to an admission wrapped in an innovation: even in the most automated factories on Earth, humans still have to open the door and reach inside. Bright Machines' wager is that the winners of the AI infrastructure race won't be the manufacturers who eliminate the human hand — but the ones who never lose sight of it.
Neura Robotics has partnered with RWTH Aachen University to establish a new Neura Gym in Germany as part of its expanding global network of Physical AI training centres. The new facility, known as Neura Gym RWTH Aachen, is scheduled to open by the end of 2026 at the university’s Hightech Campus Melaten. It will form […]
Passengers travelling from London Gatwick Airport this summer can experience the future of hassle-free airport parking with the launch of a unique robotic parking service – the first of its kind at a UK airport. Putting an end to parking stress, the new system – delivered in partnership with Stanley Robotics – allows passengers to […]
The US government has taken one of its most significant steps yet to regulate advanced robotics, adding foreign-produced humanoids, quadrupeds and other mobile robots to the Federal Communications Commission‘s “Covered List” of technologies considered to pose an unacceptable risk to national security. At first glance, the decision appears surprising. The FCC is best known for […]
Update follows determinations by executive branch agencies that these devices threaten national security The United States Federal Communications Commission updated its “Covered List” to include two new categories of devices – “advanced robotic devices” (defined as mobile robots, such as humanoids and quadrupeds) and, separately, connected power inverters produced in foreign countries. The action follows […]
This week during an interview with Bloomberg, Jensen Huang made quite the prediction.
The Nvidia CEO said the semiconductor industry will need to grow roughly five to tenfold over the next decade to support AI agents and robots to support what he believes is the next wave of computing. Huang believes that future demand will come from autonomous software agents and physical robots consuming compute around the clock.
“In the future, we have AI agents and robots, and they will be using computers,” Huang said. “Instead of a billion people using computers, we will have 100 billion agents and billions of robots all using computers. The computer industry built on top of the chip industry is certainly not big enough. Computers are being built not just for people to use, but computers are being built for computers to use.”
“Instead of a billion people using computers, we will have 100 billion agents and billions of robots all using computers. The computer industry built on top of the chip industry is certainly not big enough.”
Agents replace human endpoints
The 5-10x forecast — which Huang framed as his personal estimate, not a certainty — builds on a message he has been repeating for months, including a recent appearance where he declared traditional coding dead in favor of engineers who build AI agents. Still, it reflects the need to build backend systems for AI agents and machines, something infrastructure teams are already contending with.
On Nvidia’s fiscal Q1 2027 earnings call in May, Huang described the move from generative AI to agentic AI — systems “capable of perceiving, reasoning, planning, and acting” — as the next major phase of the industry.
South Korea’s infrastructure role
As API requests come from AI agents more often, standard assumptions around rate limiting, session memory, sub-millisecond inference routing, and API gateway concurrency are starting to break down. An environment in which most traffic originates from autonomous background loops rather than human thumbs changes how backend infrastructure must be built from the ground up.
To support an endpoint explosion of this scale, the physical supply chain must scale dramatically at the memory and data center layers. Speaking at the AI Summit in San Francisco on July 24, Huang pointed to South Korea as an important linchpin of the global AI buildout. “This is truly the beginning of a golden age for Korea,” he said, noting that the country’s semiconductor and industrial capabilities position it to help the world build out AI infrastructure.
“This is truly the beginning of a golden age for Korea.”
SK Group’s $500 billion bet
To back that vision, Nvidia announced a comprehensive partnership with SK Group valued at over $500 billion. The initiative spans massive purchasing of next-generation High-Bandwidth Memory (HBM) from SK Hynix, jointly co-developing custom HBM4 roadmaps designed specifically for agentic and physical AI workloads, and deploying Nvidia supercomputers.
The announcement also included major infrastructure investments across South Korea. SK Telecom said it plans to build a 2-gigawatt AI data center using Nvidia’s Vera Rubin architecture and SK Hynix’s HBM4 memory, with the first facility expected to come online in 2027. At the same time, Nvidia will invest $1 billion in Naver Corp, with Brookfield funding up to $9 billion as the project’s infrastructure capital partner, to help expand the company’s AI data center capacity from 55 megawatts to 200 megawatts by 2028.
Locking up supply early
Huang’s prediction also helps explain why Nvidia and other infrastructure companies are locking up supply years in advance. The company recently disclosed $119 billion in supply-related commitments as it works to secure everything from advanced packaging capacity to power, land, and high-bandwidth memory.
“Computers are being built not just for people to use, but computers are being built for computers to use.”
Huang believes the industry needs to stop thinking about a world where computers primarily serve people and start planning for one where AI agents and robots generate much of the demand. In his view, the ultimate limiting factor will be whether the industry can build enough physical infrastructure to keep up — a constraint already reshaping how companies like Nvidia and Palantir approach sovereign AI deployments.
Every major security vendor now has an AI copilot, but Mate Security thinks they’re solving the wrong problem.
The Tel Aviv-based startup announced on Tuesday it has raised a $35 million Series A led by Canaan Partners, with participation from Insight Partners, Team8 and M12, Microsoft’s venture fund, just eight months after closing a $15.5 million seed round. Mate’s pitch is that security operations need more than an LLM bolted onto a SIEM; they need a new architectural foundation built around AI.
That’s a bold claim in a market dominated by the likes of Microsoft Security Copilot, Google Security Operations, CrowdStrike Charlotte AI and Palo Alto Networks Cortex AI, all of which promise to help analysts investigate alerts faster. Mate, however, is betting the real differentiator isn’t a smarter assistant but a richer understanding of the organization itself.
Central to that vision is what Mate calls its Security Context Graph, a continuously updated model of an organization’s assets, users, business processes, and data that AI agents use to investigate alerts and make decisions with far more business context than a standalone LLM can provide.
Mate’s pitch is that security operations need more than an LLM bolted onto a SIEM; they need a new architectural foundation built around AI.
Mate CEO and co-founder Asaf Wiener tells The New Stack that the company launched with that intelligence layer, but says the product has evolved significantly over the past eight months.
“We started with the intelligence layer, the context layer that we built for enterprises in order to investigate alerts and incidents,” Wiener says. “We moved forward into the detection layer to connect the two, and now we’re heading to the security data sources.”
Mate calls the architecture Continuous Detection, Continuous Response (CDCR), linking detection and investigation so each continuously improves the other.
“We’re connecting between those two layers in the security operations center,” Wiener says. “With this architecture, we’re seeing amazing results related to the quality, accuracy and precision that we can get.”
Mate says the extra context helps its agents work out whether something that looks suspicious actually warrants attention. A burst of failed logins, for example, might look like an attack until the system spots that a security test was scheduled for the same time. Similarly, a large download of sensitive files takes on a different meaning if the employee involved is about to leave the company.
That approach appears to be resonating. Just eight months after its seed round, Mate has landed a $35 million Series A, a pace Wiener says reflects customer demand more than fundraising momentum.
“The pace is really crazy. We didn’t expect that,” he said. “We saw incredible traction with our customers. We’re talking about Fortune 500 companies, and revenue growth of more than 500 percent since Q3 2025. That’s what led those VCs to come to us and want to be part of the journey.”
“We’re talking about Fortune 500 companies, and revenue growth of more than 500 percent since Q3 2025.”
“What we are seeing is more and more data sources that we need to protect. Every employee in the organization can build new applications and new data sources. We need to build more detections for those risks, and the result: We need to investigate an increasing number of alerts every day.
“With human staff alone, we cannot handle it,” he says. “We need technology to let us scale.”
That challenge isn’t unique to Mate. Every major security platform is trying to give AI more context about the environments it’s protecting, albeit in different ways. Microsoft builds Security Copilot on telemetry flowing through Defender and Sentinel; Google ties Gemini into its security operations platform; and CrowdStrike’s Charlotte AI draws on endpoint and identity data already stored in Falcon.
Mate wants other vendors’ agents to work with its Security Context Graph, rather than keeping the technology confined to its own tools. Those agents would have access to the same information about the customer and its environment. Mate says they can remember previous investigations, while a “least-agency” model restricts what each one can see and do.
While Mate is still building out that vision, Wiener said the speed at which large companies have bought into it has caught him by surprise.
“What I’m seeing right now is that we’re doing those sales cycles in a few weeks,” he says. “That’s incredible.”
He attributes that acceleration not just to security teams, but to executives pushing AI adoption from the top. “It’s amazing to see that coming also from the board level, the CEO and the CIO that are pushing organizations to leverage this kind of technology.”
The fresh funding will primarily go toward expanding both the product and the team, although Wiener says an AI-native company scales differently from traditional software businesses.
“The plan is to double and triple the size of the team to address the demand,” he says. “But our AI builders can do much more today with the technology around us.”
Mate is still competing against security giants with deeply entrenched platforms. But if its early customer growth is any indication, investors are betting that the next generation of security operations will depend less on adding another AI assistant and more on giving those assistants a deeper understanding of the businesses they’re protecting.
Login
