Grid puzzles that people solve almost perfectly show where language and vision models still break
The post Spatial Reasoning in AI Models, Taken Apart appeared first on Towards Data Science.
Every enterprise company is currently trying to figure out how to have its AI tools work with its data. Meanwhile, every major enterprise vendor that works with those enterprises (think SAP, Microsoft, Salesforce, Workday, etc.) is making a claim on becoming the central hub for managing and securing that data.
But what about the companies that want to do it themselves?
Nvidia, on Wednesday, announced how it wants to provide enterprise AI agents with the data they need to make sense of company data — all without competing with the data platforms that already store it.
At its GTC Berlin conference Wednesday, the company added Agent Knowledge Foundation to its Agent Toolkit. This new collection of open-source components bundles the company’s Nemotron Retriever models, a new Auto-Ontology library, the Kumo family of models for structured data, and its CUDA-X data processing libraries.
With this system, Nvidia says, the data can stay where it is and remain managed under the access rules these companies already have in place.
Today, as Adel El Hallak, Nvidia’s vice president of product for agentic AI, tells The New Stack, enterprises may be able to connect an agent to a vendor like Snowflake, but that doesn’t help the agent understand the overall enterprise context.
“I wish it was as easy as connected to my Snowflake system, connected to my email exchange, connected to my knowledge base, and then all of a sudden, magically, the AI understands all the data. But the reality is different,” he says.
Large language models are “great at predicting next tokens,” El Hallak argues, but when presented with a single spreadsheet that has “a whole bunch of tabs,” the agent will have a hard time finding something specific. “It’s gonna struggle,” he says.
“In the world of agents, you want the most accurate embedding models or reranking models,” El Hallak says, even if that adds a bit of latency to each lookup. The agent then gets “much more pertinent information. So the number of lookups at the end, the overall latency actually goes down.”
Nvidia says its Nemotron Retriever embedding model raised search relevance by 65% in tests on an open source OpenSearch setup, needing fewer search steps and adding hardly any latency. SAP’s fine-tuned version of the reranker ran nearly three times faster, Nvidia says, and ServiceNow got better reranking quality with close to 25% less latency.
“RAG was awesome, but is it good enough?” El Hallak asks. Knowledge graphs, he says, represent concepts and the relationships between them better than retrieval-augmented generation, which pulls in chunks of text based on similarity.
Nvidia’s Auto-Ontology library builds those graphs from a company’s own tables and maps raw data to business concepts like customers, products, and orders. Its listing in Nvidia’s NGC software catalog shows a REST API and a Model Context Protocol (MCP) server that agents can use to query it.
“You and I, it’s easy for us to understand. A tier is the same thing as a plan when you’re looking at a mobile carrier,” El Hallak says, but LLMs “mess those up.”
Nvidia bought Illumex, an Israeli startup that automated the creation of semantic layers and ontologies for enterprise data, for a reported $75 million in February. At the time, an Nvidia spokesperson told Globes that the team’s ontology expertise would help it build “libraries and models that will enable the adoption of Agentic AI in organizations.”
Kumo, the startup Nvidia bought in June in a deal The Information valued at more than $400 million, built foundation models for relational data. Nvidia has since released several Kumo models as open weights on Hugging Face, including Kumo Tabular in September.
El Hallak says those models replace the per-task pipelines data teams used to build “for every classification, for every churn, for every sentiment analysis.”
“The whole point of the ontology … is to bring harmonization as you create those knowledge graphs, so there’s [consistency] across the tables, and then you could use Kumo to drive predictive analytics,” he says.
“I’m not looking to compete with, for example, a Snowflake. On the contrary, I want to partner with them,” El Hallak says.
Nvidia’s own supply chain team uses Agent Knowledge Foundation through Palantir’s Foundry platform, building on earlier work between the two companies. Databricks is pairing a Kumo anomaly detection model with Genie Ontology, the context layer it introduced in June, and Snowflake customers can run Kumo models in Snowflake Notebooks.
“I’m not thinking about ACLs and data access and all those things. That’s why I need the partners,” El Hallak says, referring to access control lists and other security features that its enterprise partners offer.
Not every vendor is taking that route. SAP closed its acquisition of Prior Labs, the company behind the TabPFN tabular models, in July and committed more than €1 billion to it over four years. It now offers TabPFN-3.5 Plus in SAP AI Core alongside its own RPT models.
Nvidia, meanwhile, distributes its models through Hugging Face, which it agreed to buy for $12.9 billion in September. “We just build little weights and models, and we publish them on Hugging Face,” El Hallak says.
That leaves Nvidia building the models, while the likes of Snowflake and Databricks keep control over the data and the permissions around it.
The post Nvidia bets open models can give enterprise AI agents the context they’re missing appeared first on The New Stack.
Beam is a new Mac app that takes a window running on one Mac and puts it on another, where it behaves like a locally installed app. Why? Agents.
Serial founder Hiten Shah, who co-founded Crazy Egg and KISSmetrics, built it with Sam Asante under a company called Noats. It launched on Product Hunt on Tuesday and is aimed at the headless Mac Mini sitting in a closet running AI agents.
The pitch is great: Instead of opening Screen Sharing and steering a full remote desktop, a user picks a single window on the other machine, say a Terminal session or a browser window an agent is working in, and it opens alongside everything else on the Mac in front of them. It gets its own Dock icon and can be called using the Command+Tab switcher. Resizing the window resizes the window on the other Mac. Keyboard shortcuts work, files can be dragged across, and the app’s audio can play locally. The app itself never leaves the original machine. When one window isn’t enough, Beam can pull over a full display or the whole Mac.
Shah frames it around his own setup in his Product Hunt launch post, describing Macs running agents and local models “with work continuing even when nobody’s sitting in front of them.” Getting to one of those machines usually means Screen Sharing and a second desktop to navigate. Beam is meant to cut that down to the one window that matters.
Beam plays into the exploding ecosystem of Macs powering AI systems, so the app runs only on macOS. And both Macs need Apple Silicon, macOS Sonoma or later, and Beam installed. The machines find each other on the same Wi-Fi network, or anywhere through Tailscale.
The company says windows, keystrokes, clipboard, and audio travel directly between Macs, encrypted, and never pass through its servers. It also says hidden windows and password managers never show up in the window picker.
For devices without Beam, a share link opens a single window in any browser, view-only unless control is granted, and expires after anywhere from an hour to a week.
Asante said on Product Hunt that native iOS and Android apps are on the way.
Pricing depends on the setup. Hosting a Mac and joining a share link are free. Using your own Mac is free for 20 minutes a day, then $12 a month, $84 a year, or $199 for a lifetime license.
Beam follows where agents’ hardware is growing. The Mac Mini is becoming the default home for always-on agents, driven initially by OpenClaw, but others like Perplexity followed. Apple credited agentic workloads on its April earnings call.
The New Stack‘s Janakiram MSV caught the trend early and covered the shift in May, writing, “the Mac mini just became infrastructure.”
Beam isn’t the first to this problem. I used Astropad’s Workbench for a few weeks, and it let me remotely access agents on my desktop and through iOS apps. Workbench bets on mobile devices, while Beam bets that the unit of remote access should be the window, not the screen. Jump Desktop, Screens and Apple’s own Screen Sharing all cover the basics too.
Still, the problem is real and growing. The Mac stashed in a closet or on a rack is becoming a normal part of a developer’s setup. Beam’s answer is to stop treating it like a second desktop at all.
The post Beam pulls single app windows off the Mac Mini running your agents appeared first on The New Stack.
Towards Spec-Driven Test Automation, Part 4
The post Skill or MCP Server? Ask What Stops the LLM From Disobeying appeared first on Towards Data Science.
Turn scattered requirements into deployable agents
The post What's in the Box? Distinguishing Acceptable Traces from Unacceptable Ones appeared first on Towards Data Science.
A million-token output window changes what you can hand a model in one go, and the benchmarks and the bill show who it actually suits.
The post Welcome to Google Gemini 4 Argon appeared first on Towards Data Science.
Not content with challenging the major frontier AI labs with lower-cost models, Chinese AI company DeepSeek is setting its sights on the search API market.
On Saturday, the company announced it has opened up its internally developed search engine to outside developers through two new APIs, Search and Fetch, allowing them to retrieve live web search results and webpage content for use in their own apps and agents.
Both services cost $0.75 per 1,000 successful requests, and developers can use them independently of DeepSeek’s AI models.
The launch also came alongside DeepSeek-V4.1-Flash-1017, a minor update to the V4.1 Flash model the company released in September.
Large language models (LLMs) are trained on data collected up to a certain point in time, meaning they can’t always answer questions about recent events, updated documentation, or any other kind of dynamic information. By supporting live web search, models can retrieve current information from the internet, which supplements their responses.
For developers building AI agents, this is particularly important. A coding agent, for example, might need to consult the latest software documentation, investigate a recently disclosed vulnerability, or otherwise check that a particular library has been updated.
Until now, developers using DeepSeek’s models had several ways to provide this functionality. They could connect their applications to third-party search services, or use DeepSeek’s existing search functionality through its model API.
Indeed, when DeepSeek open-sourced its agent harness in August, it included web search among its capabilities. The Harness supported interchangeable search providers, including Exa, Perplexity and DeepSeek itself. However, accessing DeepSeek’s native search through the Harness required an additional model request, with each search running as a separate call to a DeepSeek model, according to the Harness documentation, introducing extra latency and token costs.
The new APIs are designed to simplify that process. Developers can now call DeepSeek’s Search endpoint directly to retrieve ranked web results, including titles, URLs, snippets and summaries, while the Fetch endpoint retrieves the contents of individual webpages in Markdown format.
“Neither requires a separate model request, meaning developers can integrate DeepSeek’s search capabilities into applications powered by competing AI models, too.”
Neither requires a separate model request, meaning developers can integrate DeepSeek’s search capabilities into applications powered by competing AI models, too.
For much of the past two years, DeepSeek has relied on outside providers to power web search, even as it was building its own.
For context, in March 2025, Chinese business publication National Business Daily reported that DeepSeek had integrated the search API of Bocha, a Chinese AI search startup based in Hangzhou. Bocha’s CTO, Weng Rouying, noted that DeepSeek had been using its service prior to the chatbot’s surge in popularity earlier that year.
DeepSeek’s own February 2025 privacy policy, meanwhile, also acknowledged its reliance on third-party search services, naming Microsoft’s Bing as an example of an external API provider.
“Behind the scenes, the company has also been laying the groundwork for its own search infrastructure.”
Behind the scenes, the company has also been laying the groundwork for its own search infrastructure. In January this year, Bloomberg reported that DeepSeek was recruiting engineers to build a multilingual AI search engine, according to job ads listed by the company. The planned capabilities reportedly extended beyond text searches to include images and audio.
Fast-forward some nine months, and the company says its internally developed search engine, dubbed DeepSeek Search, already handles hundreds of millions of requests daily, and maintains an index containing hundreds of billions of webpages.
According to its October 17 changelog, the engine supports multilingual searches and uses filtering mechanisms designed to identify search spam, low-quality AI-generated material, and attempts to manipulate search rankings.
DeepSeek has disclosed relatively little about the underlying architecture, or how it builds and maintains its index. It’s also not clear whether the company has entirely replaced its external search suppliers across the board.
Building a search engine is no trivial undertaking. It requires the infrastructure to crawl webpages, maintain an index of their contents and rank results, while continuously updating that information as the web changes. For years, that meant developers who wanted raw search results for their applications largely turned to Google and Microsoft, the two companies with web-scale indexes of their own.
Now, however, a growing number of companies are investing in their own search infrastructure, increasingly with AI applications and agents squarely in mind. Brave, for example, operates an independent index of more than 40 billion webpages, which it says is maintained through its own crawler and other discovery mechanisms. AI search company Exa, meanwhile, has also developed its own crawling and indexing infrastructure.
Perplexity has similarly invested in proprietary search infrastructure, including building its own crawling and indexing systems. Another competitor, Parallel, says it has built its search infrastructure from the ground up, including the crawling, indexing, query processing and ranking components.
Cloudflare is also entering the market from a different angle. Earlier this month, the company launched a Web Search API through its AI Gateway, allowing developers to access search results from serveral providers through a single interface. The service, currently in beta, uses Ceramic as its default provider and charges customers at each provider’s published API rates, without additional markup.
“DeepSeek, for its part, is competing aggressively against the incumbents on price.”
DeepSeek, for its part, is competing aggressively against the incumbents on price, although it’s not strictly the cheapest option available. Ceramic.ai, which provides keyword-based web search for AI agents, charges $0.25 per 1,000 queries on its pay-as-you-go plan.
Still, DeepSeek’s $0.75 per 1,000 successful searches compares favorably with many others, including Parallel’s Turbo and Fast search services and Perplexity’s Fast Search API, both of which charge $1 per 1,000 searches. Meanwhile, Exa charges $4 per 1,000 Instant Search requests, while Brave charges $5 per 1,000 searches. Other notable players include Tavily, a search API provider for AI agents that European AI infrastructure company Nebius acquired in February. Tavily charges $8 per 1,000 basic searches on its pay-as-you-go plan.
It’s worth noting that the various services differ in terms of retrieval capabilities, result limits and performance, so the prices aren’t necessarily entirely comparable. But they provide some useful benchmarks for DeepSeek’s move into the search infrastructure space.
The emergence of these alternatives also comes as Google and Microsoft have withdrawn longstanding search API offerings. Microsoft retired its Bing Search APIs in August 2025, while Google has closed its Custom Search JSON API to new customers, with existing users required to migrate by January 1, 2027. Both companies continue to provide search functionality through their respective AI platforms, including Grounding with Google Search for Gemini and Grounding with Bing Search for Microsoft Foundry agents. These services allow AI models operating within their respective platforms to retrieve web information and generate answers with citations.
Standalone APIs such as DeepSeek’s give developers direct access to search results, which they can use with AI models and applications of their choosing.
The announcement has sparked considerable discussion among developers about pricing and whether the new APIs offer much beyond existing search integrations. On Reddit, one user welcomed the cost in particular. “I don’t think any other search provider comes close to the price that DeepSeek is offering,” they write.
Others questioned what the announcement would ultimately change for users already accessing web search through third-party tools. One Reddit user notes that they are already using DeepSeek with the Hermes AI agent, which could search the web. Another user responded that those who had already configured their own search tools might see little immediate benefit from the new APIs.
A similar discussion emerged on Hacker News, where one developer questions the need for paid search APIs when free, self-hosted alternatives such as SearXNG were available. Another commenter argues that businesses have additional requirements, citing “reliable latency, data processing guarantees” and the potential risk of exposing sensitive company information through search queries.
For businesses weighing DeepSeek’s APIs, that last point carries an extra dimension. DeepSeek’s privacy policy states that it collects, processes and stores personal data in China, and that it generally keeps user inputs for as long as an account remains open. The Search and Fetch documentation, however, doesn’t say how long search queries specifically are retained.
The discussions highlight the differing needs of individual developers and businesses, particularly when it comes to cost, reliability, and handling large numbers of requests. And while DeepSeek’s pricing is certainly competitive, there are some practical restrictions worth considering.
According to its API documentation, Search and Fetch requests are each capped at five per second per account, although individual searches can return up to 50 results without increasing the request price. DeepSeek also only bills for requests that return results, so failed calls, empty searches, and timeouts are free.
DeepSeek is temporarily waiving Search and Fetch charges for server-side tool calls made through DeepSeek Harness, Claude Code, and Codex when using its API. That is very much a temporary arrangement, however, one that ends on October 31, when the standard per-request charges take effect. Direct calls to the standalone APIs are already billable, according to DeepSeek’s announcement.
The post DeepSeek made its name with cheap AI models. Now it’s undercutting search API rivals, too. appeared first on The New Stack.
A new benchmark called Taste-Bench tests how well AI agents make strategic choices during long-running tasks. Even the best model, GPT-5.6 Sol, picks the right path on just under 60 percent of simple binary questions. Thinking longer doesn't help.
The article AI agents have bad taste in long-term decisions and a new benchmark proves it appeared first on The Decoder.
Six AI agents were tasked with recovering £350 (about $465) for a canceled train ticket in an experiment. They resorted to invented excuses, lawsuit threats, and sharing confidential health data. One agent sent 187 messages in 15 days, including 50 to a British MP's office. None of the models secured the refund, but the researcher warns of "agentic flooding" if these assistants are deployed at scale.
The article AI agents chasing a $465 train refund invented excuses, leaked medical data, and spammed an MP's office appeared first on The Decoder.
Vercel is giving developers more options for running their backends on its platform.
At its Ship conference in San Francisco on Thursday, the company introduced Daemons, a new service for continuously running background workers, and made its Python support generally available.
It also made Drives, its persistent storage for AI agents, generally available.
Daemons, which is now in public beta, lets developers deploy a container and specify how many instances should run in each region. Vercel makes sure those processes keep running and restarts them if they exit or crash. At launch, it supports pull-based workloads that allow workers to fetch jobs from a queue or database and then process them as they arrive.
Vercel CTO Malte Ubl tells The New Stack that the company’s current push in terms of going further down the stack.
“Let’s look at what people are doing, and let’s make sure that a larger chunk of them could just only use Vercel as their infrastructure provider,” he says.
The company already offers queues and durable workflows, while its Services feature lets developers deploy frontends and backends together.
These features cover request-driven work, Ubl says, but not everything a sophisticated backend does. “What really doesn’t fit well into that bucket is what you do find in sophisticated backend workloads,” he says, “where you want to do essentially processing on some kind of queue,” he says.
He uses Vercel’s own deployment pipeline as an example for that kind of workload.
“When you call our API to make a deployment, we don’t do that deployment in that process that handles your request, because we can’t actually do an infinite number of concurrent deployments,” Ubl says. GitHub rate limits are one reason to back off, and Vercel’s existing compute primitives didn’t give developers “good control over the amount of concurrency” for that kind of worker.
With Daemons, “they come up, they start saying, here I am, give me work, and then they just process as it goes,” he says.
The company’s example for using Daemons combines a Next.js frontend with a Python API, as well as a containerized video processor. The container packages Python dependencies and the FFmpeg media framework, and then pulls jobs from Vercel Queues to generate playback files and thumbnails.
Transcoding is a typical case, Ubl says, with “extremely CPU-bound workloads” where “I just need computers that I want to always be 100% CPU.”
That also puts Daemons at the opposite end of the spectrum from Vercel’s sandboxes. “They’re almost the opposite of temporary,” he says of Daemons. “For temporary, you have sandboxes,” he says. Daemons stay on but aren’t invoked by requests from the internet. They reach out for work instead.
Developers can deploy through Git or the Vercel CLI. Vercel builds the container image and stores it in its Container Registry.
The company itself runs a good chunk of its control plane API on Vercel. That accounts for about a billion requests a month, Ubl says, and Vercel is now starting to move many of its asynchronous workloads onto Daemons.
Vercel didn’t invent the model, he acknowledges, “we ship it because people very much want this.”
If you’re deep in the Vercel ecosystem, you may wonder how all of this relates to Vercel Functions.
Vercel says 5,000 teams have made 150,000 container deployments since container support entered public beta. Containers let developers package applications with their own runtimes and system libraries, expanding the range of software they can deploy.
The trade-off here is who keeps the image current, Uble argues. With a managed runtime for Node, Bun, or Python, Vercel applies a security release without the customer redeploying, Ubl says. With a container, “you do have to redeploy. It’s ultimately your responsibility,” he says. “From our point of view, it’s just bytes.”
Python support, meanwhile, is now generally available for frameworks including FastAPI, Flask, and Django. Applications run on standard CPython with PyPI dependencies, while Vercel handles framework detection, builds, and scaling. The company says 150,000 teams already run Python backends on its platform, including engineers at OpenAI, Polymarket, and Exa.
Vercel Drives is a great example of how agents are changing the requirements for backend services like storage. Vercel’s Drives service separates an agent’s files from the sandbox executing its code. An agent can mount the drive in a new sandbox, change runtimes, and continue working with the files it already created.
“I wouldn’t claim it’s an industry first, but it’s certainly an instance of a new type of storage product,” Ubl says. He describes it as a hybrid of an EBS-style networked drive and object storage, something he doesn’t think AWS offers.
Writes always land on local NVMe disk and stream asynchronously to object storage, he says, and reads come from the local disk when the data is already there — or stream in block by block.
“You have a very large number of independent disks, which, because they’re just files in S3, are arbitrarily cheap to operate,” he says.
Drives versus snapshots
Many sandbox providers, Vercel included, offer snapshots, Ubl says, but he argues drives are the better model.
“The problem with snapshots is they get really big, because you ultimately snapshot all kinds of stuff that is the same every time,” he says. “In practice, what agents really do is there’s some directory, I don’t know, /home/agent. That’s where everything is. Everything else they don’t really care about.”
Mounting that one directory means every agent can boot from the same cached base image, he says. “It’s cheaper for them, it’s faster. There’s essentially no downside.”
Vercel isn’t removing snapshots, Ubl adds, but it is encouraging customers to switch.
Drives lets those agents reuse each other’s work. One can prepare a dataset while others mount read-only copies to analyze it or generate reports, saving their results separately. Agents can also fork a drive to test different approaches in independent writable workspaces. Those forks don’t merge back into the original.
In practice, Ubl says, that means telling the agent where to write. “When you use a sandbox, you always tell your agent where it should write things,” he says, typically a root directory such as /workspace. “Essentially every prompt will start with that sentence, and then that’s just what you mount into your durable drive, and that’s it.”
Drives and forking are generally available. The service is available on all Vercel plans, with default capacities of 1 TiB, or 1 GiB on Hobby, and configurations up to 16 TiB.
For a team already deploying its frontend on Vercel, running a worker or Python API there becomes an easier decision. Each additional backend service it can keep on the platform is one fewer deployment system its developers have to maintain.
The post Vercel adds always-on workers to run more of your backend appeared first on The New Stack.
A hands-on Jev repo comparison of routing accuracy, latency, and output reliability
The post Jev Your Engines: Seven Open Decision Models vs. a Generative LLM appeared first on Towards Data Science.
On BrowseComp-Plus, GPT-4.1 answered 93.49% of the questions correctly when it was given the supporting documents directly. When an agent used the same model but relied on BM25 to search for evidence to pass to the model, accuracy fell to 14.58%.
That difference matters because a system that gets the answer right 93% of the time is far more useful than one that gets it right only 15% of the time.
An agent can look excellent in a controlled demo. The corpus is clean, the task is familiar, and the relevant material is easy to find. Production is messier. Documents may be duplicated, outdated drafts can remain searchable, and teams may use different terminology for the same thing. Several versions of a policy can coexist, while the task itself may unfold in ways the evaluation set never covered.
“More model intelligence does not compensate for evidence the agent never retrieved.”
Under those conditions, the agent may retrieve incomplete, stale, or simply wrong material. More model intelligence does not compensate for evidence the agent never retrieved.
This is where retrieval starts to determine what the model can actually do.
BrowseComp-Plus provides a useful way to separate these effects because it evaluates agents against a fixed corpus rather than a changing web-search service. The benchmark includes 100,195 documents and 830 research questions, each with human-verified supporting documents.
One evaluation gives the model all labeled positive documents directly. The paper calls this the oracle setting. Another requires the agent to search the corpus and gather the material it needs.
The difference is large. GPT-4.1 scores 93.49% with the labeled supporting documents supplied and 14.58% when the agent searches with the BM25 baseline. Qwen3-32B moves from 83.25% to 3.49% under the same two conditions.
The oracle result shows how well the model can answer when the required material is already in front of it. For system design, comparing retrieval systems is even more useful. The reasoning model stayed the same, while better retrieval improved the final answer rate and reduced the number of search calls.
“The lesson is practical: before changing the model, examine what reaches it. A capable model working from poor retrieval results still has poor material to reason over.”
The lesson is practical: before changing the model, examine what reaches it. A capable model working from poor retrieval results still has poor material to reason over.
Enterprises can build agents around proprietary information that competitors do not have, from product and customer data to internal knowledge and operational records. That advantage depends on whether the agent can find the right information and rank it well, while also keeping up with changes in the underlying data.
In its Search as Code research, Perplexity describes tasks that can trigger hundreds or thousands of retrieval operations within a few minutes. Search becomes a sustained part of the agent’s workflow rather than an occasional lookup.
Traditional search usually returns one ranked result set per query. An agent can decompose a task into multiple searches, fan them out in parallel, combine the results, and issue follow-up searches as it learns more.
At that scale, fan-out becomes a normal part of the workload. If an agent runs ten searches in parallel and needs all ten results before it can continue, the slowest search sets the pace for that step. If the agent repeats that pattern several times during the task, tail latency matters even more than it does for a single interactive query.
The same multiplication shows up in capacity. One user task can generate hundreds of search operations, and many agents may be doing that at once. Measured in user requests, the service may look lightly loaded while the retrieval layer is processing a much larger volume of searches.
When one task fans out into many searches, retrieval quality matters even more. People judge search results using context they have built up over time. They know which sources they trust and can often recognize when something looks stale, unreliable, or out of place. An agent usually starts with far less background and has to build its understanding from the evidence it retrieves during the task. It is often on its first day on the job. If one of those early results is wrong, the next search may be wrong too, and the rest of the task can drift from there.
“People judge search results using context they have built up over time. An agent usually starts with far less background and has to build its understanding from the evidence it retrieves during the task. It is often on its first day on the job.”
A simple model shows why repeated retrieval steps raise the stakes. Suppose a workflow requires five retrieval steps, each with a 90% probability of supplying what the agent needs, and assume there are no retries or recovery. The probability that all five succeed is 0.9^5, or about 59%. Real agents can retry, use redundant evidence, branch, and recover, so this is an illustration rather than a measured agent success rate. It shows how small per-step weaknesses can accumulate when later work depends on earlier retrievals.
Agentic search therefore asks the retrieval layer to handle more queries for each user task, more searches running in parallel, and retrieval steps whose results influence later work. As those patterns scale, capacity, tail latency, and retrieval quality all become more important.
Web search spent decades adapting to human search behavior.
As Jon Bratseth writes in Your agent wants to search like a 2010 quant, people are often “lazy and clueless” when they search. The wording is blunt, but the point is straightforward: people tend to enter short queries without knowing much about the corpus or the search system behind it.
Search engines evolved around those habits. Short, underspecified queries became the norm, and much of the complexity moved behind the interface, where the search system inferred intent and returned a ranked list.
An agent can use a much richer interface. It can search for exact terms, construct filters, specify dates, group results, choose among ranking strategies, and issue follow-up searches based on what it has already found. Jon gives examples ranging from proximity search for names in a legal case to semantic search over high-quality sources and grouping events by month. In an agentic workflow, those searches can be chained together as the agent narrows, expands, and verifies its answer.
That means the retrieval layer receives more than a short text query. It may receive several kinds of input: lexical terms, query embeddings, structured filters, or combinations of them. An agent-facing interface should make those choices explicit so the agent can use the right retrieval method for the task instead of collapsing every request into one text string.
Evaluation needs to reflect the same shift. A retriever tested mostly on short, human-written questions may behave differently when an agent generates its own filters, search terms, and follow-up queries. Retrieval quality should therefore be measured on the queries the agent actually sends in production, not only on the questions in the evaluation set.
Before changing the model or retuning retrieval, inspect those production queries first.
Traditional search often exposes a narrow contract: send a query, let the server run a retrieval pipeline, and receive a ranked result list.
That works well when the caller only needs the output. An agent may need more control over how retrieval and ranking are carried out at runtime, depending on what the task requires.
Vespa exposes that control through its query language and rank profiles. The caller can combine lexical or semantic retrieval, structured filters, ranking choices, and grouping in the same request, affecting how candidates are selected, scored, and returned.
The interface matters because the agent can only use the controls it can reach. A system may support filtering, ranking, aggregation, or several retrieval methods internally, but those capabilities do not help if the agent cannot invoke them when the task calls for them.
A fixed pipeline is still a good fit for many well-defined applications. Agentic workflows become more demanding when the task changes from one step to the next and the agent needs different retrieval behavior at runtime.
The goal is to expose the controls the agent needs while keeping authorization, validation, budgets, and other safeguards on the trusted server side.
An engine can expose filters, lexical matching, nearest-neighbor search, grouping, multiple rank profiles, reranking, and query-time controls. Those capabilities only become useful to the agent when the interface explains what is available and when to use it. If rank profiles, filterable fields, or grouping operations are not described, the agent may default to a free-text query and push more of the work into later reasoning or tool calls.
One way to handle this is to put a query-translation model between the agent and the retrieval system. The translator takes the agent’s request and converts it into the engine’s query syntax. This can be useful when an application wants query construction centralized behind a stable interface.
When the caller is already an agent, a separate translator often inserts a specialized model between the agent and the retrieval system. The agent already has the task, prior results, and constraints in context; the translator must reconstruct that intent from a narrower request and convert it into a query. This extra handoff adds latency and another opportunity to lose or distort what the agent meant, making retrieval harder and less reliable.
Another approach is to describe the retrieval interface directly to the agent: which fields are available and what they mean, and what rank profiles are provided.
We call that an agent-ready schema: a description of the fields, controls, and retrieval options the agent can use, along with the application-specific meaning behind them. Vespa has described work on convenience tooling in this direction, and much of that guidance can be generated from the application schema itself.
A search schema already contains much of the information an agent needs: field names and types, which fields can be searched or used as filters, available rank profiles, and configured tensors and models.
What it lacks is a description of what the fields mean and how they could be used. A field called tier may represent an account plan, a service level, or something else entirely. A timestamp such as published may mean first publication, last modification, or ingestion time. A rank profile named exact tells the agent very little about the kinds of queries it was designed for.
Those meanings can be added as explanations in the schema. That gives the agent more than names and types; it gives it enough context to choose the right fields, filters, and ranking options for the task.
Schema-to-Skill is the proposed generation step: combine the Vespa application schema, engine guidance, and application-specific knowledge into a short Markdown skill the agent can consult when constructing queries. The schema provides the structure, engine guidance explains how to use the available capabilities safely, and application-specific knowledge adds the vocabulary and meaning that cannot be inferred from the schema alone.
The agent uses that skill to choose among the retrieval strategies the Vespa application exposes. Vespa still does the actual retrieval work over the corpus, including matching, filtering, ranking, and aggregation. The agent decides what to ask for; the retrieval system executes it.
Better models can help, but agents still fail when they search for the wrong evidence, cannot express the right retrieval strategy, or do not understand the search system they are calling.
“Better models can help, but agents still fail when they search for the wrong evidence, cannot express the right retrieval strategy, or do not understand the search system they are calling.”
Agentic workloads make those weaknesses more costly because retrieval happens throughout the task. Each search can affect what the agent sees next, how much context it carries, and whether it needs to search again. At scale, retrieval quality, tail latency, and query volume become part of the task itself.
That puts more pressure on the retrieval interface. Agents need access to the filters, ranking strategies, grouping, and other controls the task requires, along with enough guidance to use them correctly. An agent-ready schema makes those capabilities explicit, and Schema-to-Skill is one way to turn that structure, engine guidance, and application-specific knowledge into something the agent can use when constructing queries.
Before upgrading the model, inspect the searches the agent is issuing, the evidence those searches return, and whether the retrieval system gives the agent both the controls and the guidance it needs. Better retrieval can improve the answer, reduce unnecessary work, and make the rest of the agentic system more effective.
The post Agentic AI doesn’t have a model problem. It has a retrieval problem. appeared first on The New Stack.
Pasting in the whole chat history cost 20 to 31 times more than looking up notes, and it did no better on the tasks.
The post I Tested Four Ways to Give an AI Assistant a Memory. The Obvious One Was the Worst Value. appeared first on Towards Data Science.
Some numbers are hard to picture until you sit with them for a moment. Last week we published the Prime Day 2026 metrics, and the one that stuck with me is 2.3 quadrillion: the daily requests Amazon ElastiCache peaked at during the event. Prime Day ran back in June, so this post always lands as a retrospective, and that gap is part of what makes it worth reading. By the time the numbers reach you, the only evidence that any of it happened is that nothing went wrong.
The rest of the week pulled in a different direction: smaller, cheaper, and more constrained. Claude Haiku 5.5 arrived on Amazon Bedrock at roughly 75 percent less than Haiku 4.5, Amazon Nova 2.5 Sonic made voice agents quicker on their feet, and Strands Box showed up to put real boundaries around what an agent is allowed to do. Now, let’s get into this week’s AWS news…
Headlines
All the numbers: Amazon Prime Day 2026 powered by AWS is the latest entry in a tradition that goes back to 2016, and the scale is still difficult to hold in your head. Over the four days of Prime Day, Amazon DynamoDB processed more than 59 trillion requests while peaking at 192 million requests per second, and kept single-digit millisecond responses throughout. AWS Lambda handled over 2.3 trillion invocations per day. Amazon EBS peaked above 24.8 trillion I/O operations, moving more than an exabyte of data daily. Amazon ECS launched an average of 158.3 million tasks per day on AWS Fargate, a 47.7 percent jump over last year, and AWS Graviton powered up to 49 percent of the Amazon EC2 compute that Amazon.com used.
The number I keep coming back to, though, isn’t a throughput record. The team ran more than 44,000 AWS Fault Injection Service experiments ahead of the event, more than six times what they ran in 2025. Those are deliberate failures injected on purpose, months in advance, so that the trillions of requests above turn into an ordinary shopping afternoon. If you’re preparing for your own peak moment, AWS Countdown Premium is built for exactly that kind of planning.
Last week’s launches
Here are some launches and updates from this past week that caught my attention:
For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.
Other AWS news
Here are some additional posts and resources that you might find interesting:
Upcoming AWS events
Check your calendar and sign up for upcoming AWS events:
Join the AWS Builder Center to connect with builders, share solutions, and access content that supports your development. Browse here for upcoming AWS-led in-person and virtual events and developer-focused events.
That’s all for this week. Check back next Monday for another Weekly Roundup!
– Daniel Abib
This post is part of our Weekly Roundup series. Check back each week for a quick roundup of interesting news and announcements from AWS!
Last week, we announced a public preview of Amazon Bedrock Managed Agents powered by OpenAI, built on a customized version of OpenAI’s Agents API engineered to be AWS-native and integrated with AWS resources. You can now build agents optimized for OpenAI models that run entirely inside AWS with the identities, permissions, and governance controls you already use.
You can choose an execution environment: self-hosted compute to use an existing development machine, container, or compute environment or Amazon Bedrock AgentCore Runtime, for managed runtime sessions and configurable storage in your AWS account. To learn more, visit the Amazon Bedrock documentation.
In addition, we are adding new frontier models on Amazon Bedrock to expand your model choices:
Last week’s launches
Here are some launches that got my attention:
For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.
AWS service availability updates
When the availability of an AWS service or feature changes, we provide customers guidance in AWS Product Lifecycle Changes on available alternatives and support for migration so that disruptions to your operations are minimized. The following lifecycle changes were updated on September 29, 2026.
Services moving to Maintenance (no longer accessible to new customers starting October 29, 2026):
Services entering Sunset:
Services reaching End of Support (as of September 29, 2026):
We understand that changes in availability can impact your operations. For specific guidance, consult the relevant service documentation or contact AWS Support.
Other AWS news
Here are some additional projects and news items you may find interesting:
For a full list of AWS blog posts, be sure to keep an eye on the AWS Blogs page.
Learn more about AWS, browse and join upcoming AWS-led in-person and virtual events, startup events, and developer-focused events, including upcoming AWS re:Invent and AWS Community Days. Join the AWS Builder Center to connect with builders, share solutions, and access content that supports your development.
That is all for this week. Check back next Monday for another Weekly Roundup!
— Channy
For the first time, researchers have tracked down an uncontrolled fleet of Chinese AI agents in the wild and communicated with them via a web mailbox. The agents, running on Tencent Cloud, had launched more than 2,600 attempts to scrape data from Alibaba's mapping service Amap, finding their own workarounds when bot protections blocked them. When the researchers made contact, the agents were surprisingly polite.
The article Researchers tracked down a rogue fleet of Chinese AI agents in the wild and actually talked to it appeared first on The Decoder.
Login