AI retrieval has moved well beyond embeddings and vector search. Early retrieval architectures focused primarily on semantic similarity. Still, production AI applications increasingly demand more from the retrieval layer: combining keyword matching, semantic retrieval, ranking, and real-time signals within a single request path.

Vector databases solved an important problem by making semantic retrieval practical. But production AI systems increasingly require more than retrieval alone. Customer-facing applications such as search, recommendations, and RAG must retrieve, filter, and rank results in real time while serving large user populations under tight latency constraints. 

As systems evolve toward conversational, research-oriented, and agentic workflows, retrieval performance, ranking quality, and architectural simplicity become increasingly important to maintaining relevance at scale.

In recently published research commissioned by Vespa, GigaOm explores how AI search platforms are evolving as organizations move beyond standalone vector search toward more integrated retrieval and ranking architectures. Rather than focusing purely on model quality, the report examines the operational and architectural trade-offs that emerge as AI workloads move into production.

GigaOm’s findings

AI retrieval architectures have become more fragmented over time. What begins as a straightforward search stack often evolves into a collection of loosely coupled systems: lexical search, vector retrieval, feature serving, reranking, synchronization pipelines, and model infrastructure. 

“What begins as a straightforward search stack often evolves into a collection of loosely coupled systems.”

GigaOm’s view is that the operational overhead of connecting and maintaining these layers is becoming a limiting factor in itself, slowing iteration cycles and making every relevance improvement dependent on coordinated changes across multiple systems.

One of the more interesting findings in the report is that consolidation is not framed primarily as a procurement exercise but as an engineering and systems design decision. GigaOm argues that teams increasingly pay for fragmentation through duplicated data movement, synchronization logic, operational maintenance, and cross-system tuning. 

The hidden cost is not simply infrastructure spend but the engineering effort required to keep retrieval pipelines aligned, rather than improving ranking quality, personalization, and user-facing AI capabilities.

“The hidden cost is not simply infrastructure spend but the engineering effort required to keep retrieval pipelines aligned.”

The report also suggests that platform convergence matters because modern retrieval workloads increasingly combine keyword search, vector retrieval, real-time features, and ML-based ranking in the same request path. 

GigaOm highlights architectures that bring these stages closer together to reduce latency, improve data freshness, and simplify experimentation, while acknowledging trade-offs such as concentration risk and migration complexity. 

Rather than recommending wholesale replacement, the report advocates a phased adoption approach, beginning with ranking and validation on production workloads before progressively consolidating retrieval capabilities.

Download a copy of the report.

The post AI retrieval at scale is becoming a systems problem, not a tooling problem appeared first on The New Stack.

Platform engineers are some of the most resourceful people in IT. Give them a problem, and they’ll automate their way toward a solution. But what happens when the automation itself becomes the problem?

This is the quiet crisis hiding inside many organizations today. In the race to reduce toil, teams have built what amounts to a mountain of automation. Scripts, layered on blueprints, layered on orchestration workflows, layered on tooling, APIs, GitOps, and infrastructure and label it a “platform.” This isn’t a platform; it’s complexity dressed up in a better outfit.

Trading one problem for another

Here’s the dirty secret about using automation to build your own platform stack: you don’t actually eliminate complexity; you just become responsible for it in a new way.

It starts well enough. You automate a painful workflow, ship it, and you move on to the next fire. But automation doesn’t maintain itself. Over time, the team that wrote it moves on. As they do, the context behind why it was built fades.

Nobody quite remembers what the edge cases were, why that snowflake script was written that way, or the original problem it solved. And when it breaks–and it will break–you’re not debugging an application. You’ll be performing an archaeological excavation of your own infrastructure, to decipher the intent of the team that originally constructed the automation.

When scripts outlive their authors

So you do what engineers do: you automate around it. You add new automation on top of old to address the gaps. Now you’re managing two mountains of automation instead of one. And here’s the impact that rarely makes it into the original business case – the platform team doesn’t get to walk away when it’s “done” because it’s never done.

These engineers can’t be reassigned without your platform decaying beneath the applications and services that make up the business’s backbone. You need a robust team to manage this indefinitely just to keep the lights on. We’re simply trading software costs for people costs. And often, you end up spending more to produce something less scalable and capable than what you had from day one.

“Automation may mask complexity but does not eliminate it, and mountains of automation makes diagnosis and repair exponentially harder when things go sideways.”

This is the real trap. Automation, at its best, is a productivity multiplier. At its worst, it’s lipstick on a pig; all the ceremonies of agility without the true benefits. Automation may mask complexity but does not eliminate it, and mountains of automation makes diagnosis and repair exponentially harder when things go sideways.

What a pre-engineered PaaS actually does

A true Platform as a Service (PaaS) isn’t a collection of automation. It’s a pre-engineered system where the underlying plumbing, services, security, and resilience are already integrated before you ever install and consume it. This is the ‘batteries included’ model where the platform is ready to use on Day 1 based on best practices and proven architectures. This kind of integration out-of-the-box is one of the things that makes a platform trustworthy and predictable at scale.

For example, an integrated platform includes ‘how’ applications are built and deployed is pre-wired out of the box and works consistently across application types. One of the more unique things Tanzu Platform does is build deployment packages, including the base image for developers. This means that when a security problem comes out–like Copy Fail or the flood of AI-discovered vulnerabilities–the platform engineer can rebuild and redeploy apps very quickly without reaching back into the software delivery lifecycle (SDLC). They simply “restage’ the application using a single command.

The consistency of deployment packages and base images also enables developer velocity in the every day. When every application is built, packaged, and deployed the same way, developers stop re-solving the same infrastructure problems, and start focusing on the code that actually matters. This is a distinction worth drawing clearly–assembling capable open source tooling like Terraform, ArgoCD, Kubernetes, cert-manager, OpenBao, and Istio gives you powerful building blocks, but it doesn’t give you a platform.

You still own the integration, automation, the opinions, lifecycle management, and the operational model that ties them together. A pre-engineered PaaS handles the myriad of decisions for you. With a PaaS, onboarding a new team or new application isn’t a one-off integration project; it’s a repeatable, predictable process. Standardization is a core outcome of a pre-engineered PaaS, not a side effect, and it’s precisely what makes it possible to push changes faster and with more confidence, regardless of the team, language, or application type.

Security built into the platform

Security works the same way. When your team makes the decision to pull from open-source components and stitch them together, you now own every security gap between them – data at rest, data in flight, and its running state. This seems like a worthwhile investment at first, but as recent research suggests, AI-assisted attacks are on the rise, and platform teams won’t be able to keep up with surging security vulnerabilities in the build-it-yourself model.

A pre-engineered PaaS standardizes the governance and compliance posture across the board. Updates, patches, and fixes from a trusted, first-party vendor mean you’re not reinventing governance from scratch each time you add a new component. The PaaS can help you uniformly and at scale apply the changes in a cascading fashion rather than relying on custom automation.

The calculus starts to shift when you see deployment, security, and onboarding issues in aggregate. Achieving all of this with custom automation means spanning organizational silos, coordinating teams, sustaining headcount, and continuously funding work that never actually finishes.

And worse yet, this work has no meaningful competitive advantage for your organization. Your competitors are working to solve the same problem, but you’re just burning more resources to accomplish the same end result. You own the problem entirely rather than leaning on a trusted, proven vendor. Embracing a PaaS allows you to focus on higher-value, differentiating initiatives for the business.

AI is the forcing function

The conversation around PaaS is urgent again, and AI is why. Code generation can speed up your development cycles, building and pushing features faster, but production delays will persist if you’re still deploying at the same speed as before. 

To avoid eroding the benefits of code generation, you need to deploy applications nearly as fast as they can be coded with AI. This requires streamlining each step of the path to production. In an era where more organizations are exploring the use of autonomous agents, they need a platform that doesn’t take weeks to rotate credentials, days to provision a database, or require access to JIRA’s MCP Server to accomplish their goals.

“To avoid eroding the benefits of code generation, you need to deploy applications nearly as fast as they can be coded with AI.”

The pace of AI innovation itself compounds the problem. Whether it’s shadow AI use, MCP servers, agentic harnesses, this week’s new foundation models, or whatever emerges next, the landscape is evolving fast enough that what’s bleeding-edge today may be table stakes in six months. When you build your own platform, you’re on the hook to evaluate each layer of these new technologies, determine how they fit into your stack, and then integrate them yourself, on top of everything else your platform team is already managing to keep the business going.

Organizations running VMware Tanzu Platform receive those innovations. With Tanzu Platform 10.4, for example, customers gained MCP Gateway, an expanded service marketplace where developers can publish their own MCP servers and services for cross-organization consumption, as well as an agent buildpack to streamline and secure the deployment of agentic AI applications.

These are capabilities that would have taken a DIY team months to evaluate, build, and harden. And this isn’t unique to AI. Every release brings new capabilities across the platform that customers simply inherit, without the integration tax. That’s the compounding return on your investment in a pre-engineered PaaS. The platform keeps moving forward, and so do you.

VMware customers: You’re closer than you think

Platform engineers are uniquely equipped for this moment. They have the pattern recognition, they have seen the technology waves before, they bring their hard-won operational instincts, and the critical thinking skills to know when a system is genuinely resilient versus cleverly disguised complexity.

That’s exactly why their role in curating and operating this next generation of PaaS matters more than ever. Platform Engineers applying these battle-tested experiences using a PaaS will shortcut the tedious automation pitfalls, jumping straight to safely delivering Agentic AI and GenAI application services.

If you’ve already built your stack on VMware Cloud Foundation you’re already in a position to add the value of a true PaaS. VMware Tanzu Platform layers a pre-engineered PaaS on top of your existing infrastructure APIs, running alongside your VMs and containerized workloads, without ripping out what you’ve built and budgeting for alternatives.

With an incremental step, you can leverage what you know and trust, and layer on the only private cloud PaaS for agents that offers direct integration with Private AI services.

The post The DIY platform trap that’s burning out engineering teams appeared first on The New Stack.