Kubernetes teams automate deployments without thinking about it. CI/CD pipelines fire dozens of times a day, autoscaling adjusts replicas in the background, rollback is muscle memory. But there is one category of automation where that confidence vanishes: letting a system change CPU and memory requests on a running workload without a human reviewing it first. 

And as AI inference lands on Kubernetes at scale, that hesitation is becoming hard to ignore, and increasingly expensive.

Why teams trust automation for change but not for constraint

We surveyed 321 Kubernetes practitioners at enterprise organizations earlier this year. The headline finding is one most practitioners will recognize immediately: 82% report high or complete trust in automated delivery controls. But 71% still require human review before applying resource optimization recommendations. Only 27% allow CPU and memory changes to be auto-applied, even within guardrails.

“Deploying code feels additive… rightsizing feels subtractive because you are removing safety margin from a running service, and the failure mode is fundamentally different.”

Those numbers describe a specific asymmetry. The same engineers who deploy to production dozens of times a day without hesitation slow down the moment automation wants to adjust resource allocation. And the survey data make it clear why. Deploying code feels additive. You are shipping new value, the rollback path is well understood, and if something breaks you usually see it right away. Meanwhile, rightsizing feels subtractive because you are removing safety margin from a running service, and the failure mode is fundamentally different.

As one practitioner in the survey put it: “Automated right-sizing carries a unique risk because it directly impacts the underlying stability of the application runtime. Unlike a code deployment that follows a tested path, resource changes alter the invisible contract between the workload and the scheduler.”

When you change resource requests, you change how Kubernetes schedules, prioritizes, and allocates resources. Those effects are not visible the way a code change is. You can’t trace them through a deployment pipeline. And you might not discover that something went wrong until two weeks later, when a traffic spike hits a threshold that didn’t exist at the old values. By that point, three other things have changed too, and proving causation is nearly impossible. The people responsible for those workloads are the same people who get paged at 2 a.m., and they know this.

Why AI workloads raise the stakes

That trust gap existed before inference workloads showed up. What’s changed is the cost of not closing it.

For a long time, teams could absorb the cost of manual oversight. They knew their workloads, had intuition for where the safe boundaries were, and the inefficiency of over-provisioning was a price worth paying for stability. GPU-accelerated inference workloads change that math. GPU compute is significantly more expensive per hour than CPU. The cost of over-provisioning is no longer a rounding error you can absorb quietly. And the workload behavior is less familiar, as inference jobs are bursty in ways teams haven’t built intuition for, traffic patterns shift as models are updated and usage changes, and the resource dimensions involved differ from what teams have spent years learning to tune.

That unfamiliarity compounds with scale. Rightsizing isn’t a one-lever problem the way horizontal scaling is. It involves, at minimum, CPU and memory requests and potentially limits for both, with four dimensions per workload, multiplied across hundreds or thousands of workloads per cluster. The survey data indicates that manual optimization breaks down at around 250 changes a day. Inference workloads push teams past that threshold faster than anything they’ve managed before, because the resource decisions are more frequent and the cost of getting them wrong is higher.

The economic case for automated rightsizing has never been stronger. The organization’s willingness to delegate hasn’t caught up because teams are being asked to trust automation with workloads they don’t yet have a track record with.

What the survey says about closing the gap

When we asked practitioners what would actually increase their trust in optimization automation, 48% said visibility and transparency into how decisions are made, 25% wanted proven guardrails, and 23% needed instant rollback.

Nobody asked for full manual control and very few asked for blind autonomy. What they described is automation that earns trust in stages, and that’s consistent with how the teams furthest along in their automation journey actually got there. They didn’t start with production. They started with a single namespace in a dev environment, observed the system’s behavior, compared recommendations with outcomes, and gradually expanded the scope. Different environments remained at different levels of automation maturity simultaneously, and that was intentional. Production carried more scrutiny than dev.

CI/CD followed the same curve, and the timeline is easy to forget. Most organizations took years to get from running their first automated pipeline to trusting it with production deploys without manual approval on every commit. Kubernetes resource automation is earlier in that same process, and AI workloads are extending the timeline because teams are building trust from scratch with a workload category that doesn’t yet have a track record.

Why automation design matters as much as capability

Some automation architectures deliver meaningful value only with full delegation. The system needs complete control to function the way it was designed to. That’s a form of forced autonomy, and it creates an adoption problem because it asks for exactly the level of trust that most organizations haven’t built yet. Force generally doesn’t work. Teams that feel pushed into a level of delegation they aren’t comfortable with tend to pull back entirely after the first incident.

The alternative is what I’d describe as adaptive autonomy: designing the system to work at every stage of the trust curve. A team still evaluating gets useful recommendations in read-only mode. A team ready to act but wanting boundaries can run guardrailed execution within limits they define. As confidence grows, the system handles more decisions autonomously while humans manage exceptions. And for environments where the track record supports it, closed-loop optimization runs in the background and becomes boring, which is the goal. Each stage is a legitimate operating mode, not a stepping stone you have to rush through.

That design distinction matters more with AI workloads than it ever did with traditional services, precisely because the trust-building process is starting from zero on workloads where the cost of getting it wrong is highest.

“Trust takes a long time to build and a single production incident to undermine.”

The other piece that makes this sustainable is rollout safety. Trust takes a long time to build and a single production incident to undermine. Start with the workloads showing the most headroom between requests and actual usage. Make changes incrementally, small enough that a bad outcome stays contained. Rollback needs to be fast and tied to the health signals the team already monitors. And start with opt-in, not opt-out. Let the teams willing to go first build a track record that others can look at.

The broader pattern

The 71% figure is sometimes read as resistance to automation. I think it’s a more accurate picture of how operational trust actually forms: conditional, earned over time, and moving at different speeds depending on what’s at stake. AI workloads are raising those stakes significantly, which means the path to trusted automation matters more now than it did when the cost of caution was just some unused CPU.

“Most of what gets written about Kubernetes optimization focuses on tooling capability, and the tooling is capable. The harder problem is the human one.”

Most of what gets written about Kubernetes optimization focuses on tooling capability, and the tooling is capable. The harder problem is the human one. If your team is managing AI inference workloads on Kubernetes and your optimization tooling is sitting in read-only mode, the question worth asking isn’t whether to trust the system. It’s whether the system is designed to let you build that trust gradually, starting where the stakes are low and expanding as the evidence supports it, on workloads where getting it wrong costs more than it ever has before.

The post Kubernetes teams trust automation to ship code but not to touch CPU, and AI is raising the stakes appeared first on The New Stack.

For the past two years, the conversation about AI-assisted software development has been dominated by speed. A new GitLab survey of more than 1,500 developers and technology leaders found that 60% say AI coding ROI has already exceeded expectations, and 78% report their teams are writing and committing code faster since adopting AI tools. 

But speed without control is a liability.

Most organizations have pursued agentic engineering by adding AI coding tools on top of their existing infrastructure. Coding agents are delivering speed, but that speed isn’t showing up across the full software lifecycle: Only 21% of respondents report productivity gains beyond code generation itself.

“Speed without control is a liability.”

The infrastructure problem runs deeper. Git backends, toolchains, and governance frameworks were built for human-scale concurrency. Agents operate at machine scale, and that mismatch shows up fast. Platform reliability breaks down with millions of agent sessions hitting the same backend, security exposure widens as agents touch dependencies at scale, and cost overruns mount as agents consume tokens inefficiently on infrastructure that wasn’t built for them.

Agentic adoption outpaced governance

The adoption curve for AI coding tools outpaced the development of required guardrails, with 80% of organizations saying they adopted AI tools faster than they developed policies to govern them, and 82% reporting that AI-generated code risks creating a new form of technical debt that their organizations are not prepared to manage.

In practice, that means platform reliability challenges under agent load, security and compliance exposure that widens as agents touch dependencies at volume, and agents operating with artificial confidence because they lack full context. Only 28% of organizations say their software development lifecycle tools are fully integrated with shared data and workflows, which means most teams are trying to govern agent actions across a toolchain that was never designed for them.

Agentic engineering needs agentic infrastructure

Agentic engineering requires two things: agentic coding and agentic infrastructure. Most organizations have the first but lack the second.

Agentic infrastructure spans four areas: the execution layer, the context layer, the governance layer, and the orchestration layer working together.

The first is machine-scale execution. Git backends, CI/CD pipelines, and deployment systems were designed for human-paced development. In the agentic era, they need to handle millions of agent sessions without breaking. When a production incident occurs, the path from symptom back to origin should take minutes, not days.

The second is context that travels with code. As Bastian Stahmer, Business Owner of Vehicle Software Development Platform at Mercedes-Benz, put it on a panel recently, “An agent can only be as good as the context and semantics fed to it.” A context graph connecting code, work items, pipelines, security findings, and production signals is what makes agents genuinely useful at scale and keeps artificial confidence in check.

“An agent can only be as good as the context and semantics fed to it.”

The third is governance built into the flow. Agent actions need to be tied to an identity, logged against a policy, and provable to a reviewer. Low-risk changes move fast, while higher-risk changes trigger review. For Mercedes, operating under automotive regulatory standards that require full traceability and human accountability, GitLab is the control plane where that accountability lives.

The fourth is orchestration. Execution, context, and governance are only as effective as the system coordinating them. The orchestration layer coordinates agent actions across the full software lifecycle according to the policies teams define, determining which agents run, in what order, and how failures and handoffs are managed. Without it, agentic infrastructure is a set of independent capabilities rather than a working system.

What’s next

The next phase of AI in software will focus less on generating code and more on governing it, according to 85% of respondents. That shift reflects how enterprises are maturing their thinking about AI, from a productivity tool to a foundational capability that needs to be trusted, traced, and maintained at scale.

When governance is built into the platform, speed and control are no longer in tension. Traceability becomes a competitive advantage. Context becomes institutional memory. And the codebase, rather than accumulating invisible risk, becomes an asset that grows more reliable over time.

The post GitLab just surveyed 1,500 developers. Here’s why it matters for your codebase. appeared first on The New Stack.

For most of its 30-year history, Postgres has been viewed as a transactional database. Organizations trust it with customer records, financial transactions, and countless other operational workloads. Its reputation was built on reliability, strong transactional guarantees, and a vibrant open-source community that has spent decades refining the database without compromising its foundations.

However, some of the most important innovations in the Postgres ecosystem today have little to do with storing data. They have to do with reducing the need to move it around.

“Some of the most important innovations in the Postgres ecosystem today have little to do with storing data. They have to do with reducing the need to move it around.”

Database innovation has historically focused on performance, scalability, and reliability. Increasingly, the harder problem is interoperability: how operational data can be shared across analytical systems, AI applications, and downstream services without creating yet another pipeline or copy.

Why Postgres keeps showing up

The reality of modern software architecture is that data rarely stays in one place. Information created in operational systems quickly finds its way into warehouses, search platforms, machine learning environments, and AI applications. Every new system solves a legitimate business problem, but it also creates another destination for data and often another copy to maintain.

The costs of this approach extend beyond infrastructure spending alone. Every additional copy introduces latency, creates another potential source of inconsistency, and increases the operational burden of keeping systems synchronized. Many organizations now spend as much effort moving data as they do storing it.

“Many organizations now spend as much effort moving data as they do storing it.”

For many businesses, Postgres serves as the system of record for customer interactions, transactions, application state, and other business-critical information. As organizations expand their analytical, machine learning, and AI capabilities, they are not looking to create another source of truth; rather, they’re looking for better ways to work with the one they already trust.

That shift is changing how Postgres fits into modern architecture. Historically, Postgres was viewed primarily as the place where operational data originated before being copied into downstream systems. Increasingly, organizations want those systems to work more seamlessly with operational data while reducing the pipelines, copies, and synchronization processes required to support them.

Technologies such as logical replication, change data capture, and foreign data wrappers have helped Postgres participate more directly in larger data ecosystems. As a result, organizations are no longer asking only whether Postgres can store their data. They’re instead asking how easily it can connect to everything around it.

That shift, from evaluating databases primarily on storage and performance to evaluating them on interoperability, may be one of the most important changes happening in the Postgres ecosystem today.

AI is exposing old problems

The recent focus on AI has brought renewed attention to data movement. AI didn’t create the problem. If anything, it exposed a limitation that has been quietly growing for years. For decades, organizations built architectures around the idea that data would move between systems through pipelines and periodic synchronization. That model worked because most analytical workloads could tolerate some degree of delay.

AI is changing those expectations. Many AI applications depend on access to current operational context. The challenge is not that organizations lack data. In many cases, they already have it. The challenge is that the data is spread across multiple systems, each with its own copy, latency profile, and synchronization process.

“AI is forcing organizations to confront a broader question: How many copies of the same data are actually necessary? The answer increasingly appears to be fewer than most architectures maintain today.”

As a result, AI is forcing organizations to confront a broader question: How many copies of the same data are actually necessary? The answer increasingly appears to be fewer than most architectures maintain today. As expectations around freshness rise, reducing unnecessary data movement becomes just as important as accelerating it. The underlying challenge is not new. AI has simply made it harder to ignore.

What’s next

The database industry spent decades solving storage. Databases became more reliable, storage became cheaper, and infrastructure became dramatically easier to operate. The next challenge is not where data lives, but how easily it can be shared across systems without introducing additional pipelines, copies, and synchronization overhead. Increasingly, the goal is not simply moving data faster. It is reducing unnecessary movement altogether.

Postgres has a habit of outlasting predictions about its replacement. For years, members of the community have joked that every year is “the year of Postgres.” The joke works because it keeps turning out to be true. 

Three decades after its creation, Postgres continues to adapt to new workloads, new architectural patterns, and new ways of building applications.

That longevity is not an accident. Enterprises continue to rely on Postgres because it provides a stable and trusted foundation for operational data.  While that foundation is unlikely to change, the scope of what organizations expect Postgres to do will continue to expand.  

As new workloads continue to emerge, much of the innovation will come through extensions that expand Postgres’s capabilities without sacrificing the stability that made it successful. In that sense, the future of Postgres may not be about reinventing the database itself, but continuously expanding what can be built on top of it.

The post The database storage problem is solved. Here’s what comes next. appeared first on The New Stack.

A recent GigaOm CxO Decision Brief explores how AI retrieval architectures are evolving beyond flat vector databases as organizations combine semantic search, ranking, personalization, and machine learning inference in production systems.

Vector search changed the AI infrastructure landscape by making semantic retrieval practical at scale. By converting text, images, and user behavior into embeddings, organizations could move beyond exact keyword matching and retrieve information based on meaning. But production AI systems rarely stop at vector similarity.

A real-world query often requires multiple signals to be evaluated simultaneously. Semantic relevance may be one factor, but so are structured attributes, business rules, personalization signals, freshness, access controls, recommendation logic, and machine-learned ranking models. As organizations move from AI experimentation to production-scale applications, the challenge is no longer simply finding similar items. It is in combining all of the signals that matter while maintaining low latency and operational simplicity. This is where tensors are attracting increasing attention.

While vectors represent information as a single dimension of numerical values, tensors provide a more general framework for representing and operating on complex, multi-dimensional data structures. They offer more control in how relevance is computed, allowing dense embeddings, sparse features, metadata, and model outputs to be evaluated together within a unified retrieval and ranking process. For organizations building large-scale retrieval systems, this raises an important architectural question: is a flat vector store sufficient, or does the next generation of AI applications require something more expressive?

“Tensors provide a more general framework for representing and operating on complex, multi-dimensional data structures.”

A new GigaOm CxO Decision Brief, “The Tensor Advantage in AI Search,” explores this question in depth.

Among the findings:

• Production AI systems increasingly depend on combining semantic, lexical, behavioral, and business signals rather than relying on vector similarity alone.
• Architectural fragmentation between vector databases, search engines, rerankers, and feature stores introduces latency, operational complexity, and synchronization challenges that become more significant as workloads scale.
• Emerging retrieval models, including multi-vector and late-interaction approaches, place new demands on infrastructure that were not anticipated when first-generation vector databases were designed.
• Tensor-native architectures provide an alternative approach by treating multidimensional data structures as first-class citizens rather than forcing them into simpler vector abstractions.

The paper also examines the infrastructure, operational, and organizational implications of these architectural choices, including benchmark data, deployment considerations, and the trade-offs engineering leaders should evaluate when planning future AI retrieval systems.

“Retrieval is evolving from a nearest-neighbor problem into a ranking and decision-making problem.”

As AI applications become more sophisticated, retrieval is evolving from a nearest-neighbor problem into a ranking and decision-making problem. Understanding the role tensors play in that transition may be one of the most important architectural discussions facing engineering leaders today.

Download the GigaOm CxO Decision Brief to explore the findings in full.

The post Why AI retrieval and ranking need more than vector search appeared first on The New Stack.

“Keep readers reading” is the not-so-simple goal of Medium’s recommendations system. To predict what’s most likely to appeal to a particular reader at any given time, Medium continuously processes user activity signals (stories read, recommendations shown, follows, likes, etc.). It then immediately correlates that with the steady stream of new articles, which is estimated at millions per month.   

Smart models and good inference logic are required, but that’s not enough. The data must be stored and retrieved quickly enough to remain relevant while the user is browsing. That’s the job of Medium’s feature store. And getting the data model right started to matter a lot as they scaled to 1M operations per second.

Andréas Saudemont, Medium Principal Software Engineer, recently walked through how the team identified the problem and what they built to fix it. If you’d rather watch than read, you have two options: Watch a short version from Monster Scale Summit or an extended follow-up webinar

The feature store and its role in Medium’s recommendation system

The feature store ties it all together, ingesting user activity and internal events and feeding them to the ML models that power recommendations. It’s what enables customization like the “For You” feed that greets logged-in users.

Each feature is a property of an entity, usually a user or a story. Some are simple and static, like whether a user holds a paid membership. Others capture interaction history: which stories a user has read, what content they’ve recently been shown, etc.

The following diagram shows a highly simplified view of the Medium feature store architecture:

The problem with a relational features data model

When they built their feature store years ago, Medium used relational features for cross-entity relationships. Unlike regular features, a relational feature can have multiple values for a given entity ID. Each value is defined by a relation ID (the ID of the related entity) and a timestamp recording when the event occurred.

For example, a “story users have read” feature is attached to the story entity type. It relates to the user entity type, and its values indicate whether/when a given user has read that story. 

Andréas shared the following schema diagram to explain the concept:

Features sit at the center, each attached to an entity type and defined by name, version, and data type. Non-relational features are simply a feature, an entity ID, and a value. Relational features add a relation ID mapping to another entity type, plus the value itself and a timestamp.

This approach proved suboptimal from a data modeling perspective. Since relational features link two entity types, the data ends up split between two tables: one for the entity IDs and one for the values. That means you can’t get both in a single query. The first query retrieves only entity IDs (not their associated values) and relies on ALLOW FILTERING. A second query then runs for each entity ID to fetch its value. “If we have 1000 entity IDs for which we want to fetch values, then we have to run 1000 queries to fetch these values,” Andréas said.

Overrelying on ALLOW FILTERING made things worse. “This is bad,” Andréas said, referring to monitoring data showing that 90% of rows read via these queries were simply discarded. “This is just data that we don’t need. ALLOW_FILTERING should be an escape hatch, not our design pattern.”

“ALLOW_FILTERING should be an escape hatch, not our design pattern.”

The list feature model

So they reinvented their data model and shifted to a list-based feature model. Instead of splitting data across two tables, everything for a given entity lives in one place and is retrieved in a single query.

Like other features, a list feature is defined by its entity type, name, and optional version. What’s different is the value. While a non-relational feature has a single value, such as true or false, a list feature’s value is a collection of items, each containing a value and a timestamp. Item values can be of any data type; the feature store doesn’t enforce consistency within a list.

For example, consider a user’s reading history. The entity is user, the feature name is reading history, the TTL is 6 months. After that TTL is reached, the data is automatically dropped by the database (since older history isn’t useful for recommendations). The list for a given user is a collection of story IDs and the timestamps at which they were read. The same story can appear multiple times, and multiple items can share the same timestamp.

A range of operations need to be supported. Create List and Delete List operations run at most a few times per day. Remove List Items with Value, which lets a reader scrub a specific story from their history so it stops influencing recommendations, runs at 1k-10k per second. Add List Items is higher still: every story read and every thumbnail shown to a user generates an event. Get List Items is the top, at 100k-1M operations per second.

“The Add List Items, and even more the Get List Items operations, are really the reasons why we need an efficient data store.”

“The Add List Items, and even more the Get List Items operations, are really the reasons why we need an efficient data store,” Andréas said.

Multiple items, one timestamp

Beyond raw efficiency, the new data model also had to support multiple items with the same timestamp. When Medium shows a user four story thumbnails simultaneously, all four presentation events share the same timestamp, but have distinct story IDs. If this isn’t handled correctly, primary key collisions occur.

The team’s solution was a single list_items table that stores everything.

The partition key combines feature_key and entity_id, keeping all items for a given list together. All of user 123’s reading history is stored in one partition, retrieved in one query. The clustering key concatenates each item’s timestamp with an MD5 hash of its value. The hash is what makes same-timestamp items with distinct values possible. 

Relying on MD5 hashes for uniqueness raises its own set of questions, but in practice, the team hasn’t seen collisions. “The values that we are storing are sufficiently distinct, especially when you add the timestamp into the equation,” Andréas said. The table’s clustering order is set to descending so ScyllaDB can optimize for the typical read pattern (most recent N items) rather than leaving the application to sort afterward.

TTL to control storage costs

Storage cost is controlled entirely through ScyllaDB’s native TTL, with no cleanup logic required. Every row expires automatically based on its own timestamp plus the feature’s TTL duration. “We don’t have anything to do regarding that,” Andréas said. “Any row for which the TTL is expired will be considered deleted by ScyllaDB.” 

Storage plateaus for a steady write rate. When a feature is retired, its data drains away on its own. “That’s super useful for controlling our storage and usage costs.”

Implementing the list operations

Add List Items is a logged batch of INSERTs with atomicity guaranteed: all items land or none do. Each row carries its own TTL calculated from its timestamp, so older items expire sooner. Since items almost always carry a current timestamp, new entries append to the top of the partition, which is exactly where reads will look first.

Get List Items runs as a single-partition SELECT with a minimum timestamp and a row limit. “We run the query on a single partition,” Andréas said. “That’s the maximum efficiency that we can have.” The clustering key handles filtering and ordering directly. Post-processing is not required.

Remove List Items with Value is the one operation that couldn’t be reduced to a single query. Because value isn’t part of the primary key, a direct filter isn’t feasible.

A local secondary index built specifically for this case first finds the matching item keys, then a batch DELETE removes them by primary keys.

“Using an index is really faster than a scan because the query is highly selective,” Andréas explained. “We have very few items in a given list that have the same values compared to the total number of items in a list. And thanks to the current structure, using a local secondary index is faster than a global index.”

Andréas shared another example. Starting with the original table partition, the goal is to delete all items with the value “storyC.” Using the local secondary index, the system first identifies the two rows containing that value. It then issues two DELETE statements using the item keys from those rows, which removes them from the list. The final operation, removing all list items, is even more straightforward.

“We can just drop the partition,” Andréas said, “and ScyllaDB does its magic. It just deletes all the rows for that partition, which means that it deletes all the items for the given list. And bonus point: it’s atomic. It’s either completing successfully or not changing anything at all.

ScyllaDB vs. DynamoDB performance

Medium implemented the list operations on top of both ScyllaDB and DynamoDB. The main goal was to benchmark how both databases compared on their actual production data. “Conceptually they are very close,” Andréas noted, “but they have significant differences in how they operate.”

For AddListItems, P50 latencies were low with both databases: ScyllaDB came in under 1.5ms, DynamoDB under 5ms. “DynamoDB is extremely fast, not as fast as ScyllaDB, but extremely fast at sub 5ms latency,” Andréas commented. Things got more interesting at the P95 and P99 latencies. ScyllaDB held steady at around 5-6 ms P95s, while DynamoDB ranged from 13-45 ms. ScyllaDB’s P99s were steady single-digit milliseconds, while DynamoDB’s ranged from 40- 120 ms.

It was a similar story for GetListItems. At P50, ScyllaDB clocked in at 1 ms, DynamoDB at around 3.5 ms. At P95, ScyllaDB held around 5-6 ms while DynamoDB spiked from 30 – 60ms. And at P99, ScyllaDB remained at ~30ms while DynamoDB ranged from 70 ms all the way up to 220 ms.

“ScyllaDB is very fast, with very predictable performance, and that’s super important for us.”

One caveat: DynamoDB was running without an extra caching layer. “We expect that could have a significant impact for DynamoDB because of the high cache hit rate that we are seeing on the list,” Andréas said. “But we don’t have the data yet, so we cannot compare them.” His verdict for now: “ScyllaDB is very fast, with very predictable performance, and that’s super important for us.”

Key takeaways

One pleasant side effect of getting the data model right: Medium is now eager to use ScyllaDB for additional feature store workloads. Before, they were holding back because they didn’t want to build on the shaky relational feature foundation.

Reflecting on the path to this point, Andréas left the audience with this parting advice:

“If you have a suboptimal data model, you will have queries that are slow, that will scale badly. And most likely, you won’t be able to optimize that data model. You will have to define a new data model that will be better. So take time to think about your data model before you start the implementation, because once you have production data using your suboptimal data model, it’s too late.”

The post When your data model is the bottleneck: lessons from Medium’s feature store appeared first on The New Stack.

AI retrieval has moved well beyond embeddings and vector search. Early retrieval architectures focused primarily on semantic similarity. Still, production AI applications increasingly demand more from the retrieval layer: combining keyword matching, semantic retrieval, ranking, and real-time signals within a single request path.

Vector databases solved an important problem by making semantic retrieval practical. But production AI systems increasingly require more than retrieval alone. Customer-facing applications such as search, recommendations, and RAG must retrieve, filter, and rank results in real time while serving large user populations under tight latency constraints. 

As systems evolve toward conversational, research-oriented, and agentic workflows, retrieval performance, ranking quality, and architectural simplicity become increasingly important to maintaining relevance at scale.

In recently published research commissioned by Vespa, GigaOm explores how AI search platforms are evolving as organizations move beyond standalone vector search toward more integrated retrieval and ranking architectures. Rather than focusing purely on model quality, the report examines the operational and architectural trade-offs that emerge as AI workloads move into production.

GigaOm’s findings

AI retrieval architectures have become more fragmented over time. What begins as a straightforward search stack often evolves into a collection of loosely coupled systems: lexical search, vector retrieval, feature serving, reranking, synchronization pipelines, and model infrastructure. 

“What begins as a straightforward search stack often evolves into a collection of loosely coupled systems.”

GigaOm’s view is that the operational overhead of connecting and maintaining these layers is becoming a limiting factor in itself, slowing iteration cycles and making every relevance improvement dependent on coordinated changes across multiple systems.

One of the more interesting findings in the report is that consolidation is not framed primarily as a procurement exercise but as an engineering and systems design decision. GigaOm argues that teams increasingly pay for fragmentation through duplicated data movement, synchronization logic, operational maintenance, and cross-system tuning. 

The hidden cost is not simply infrastructure spend but the engineering effort required to keep retrieval pipelines aligned, rather than improving ranking quality, personalization, and user-facing AI capabilities.

“The hidden cost is not simply infrastructure spend but the engineering effort required to keep retrieval pipelines aligned.”

The report also suggests that platform convergence matters because modern retrieval workloads increasingly combine keyword search, vector retrieval, real-time features, and ML-based ranking in the same request path. 

GigaOm highlights architectures that bring these stages closer together to reduce latency, improve data freshness, and simplify experimentation, while acknowledging trade-offs such as concentration risk and migration complexity. 

Rather than recommending wholesale replacement, the report advocates a phased adoption approach, beginning with ranking and validation on production workloads before progressively consolidating retrieval capabilities.

Download a copy of the report.

The post AI retrieval at scale is becoming a systems problem, not a tooling problem appeared first on The New Stack.

Platform engineers are some of the most resourceful people in IT. Give them a problem, and they’ll automate their way toward a solution. But what happens when the automation itself becomes the problem?

This is the quiet crisis hiding inside many organizations today. In the race to reduce toil, teams have built what amounts to a mountain of automation. Scripts, layered on blueprints, layered on orchestration workflows, layered on tooling, APIs, GitOps, and infrastructure and label it a “platform.” This isn’t a platform; it’s complexity dressed up in a better outfit.

Trading one problem for another

Here’s the dirty secret about using automation to build your own platform stack: you don’t actually eliminate complexity; you just become responsible for it in a new way.

It starts well enough. You automate a painful workflow, ship it, and you move on to the next fire. But automation doesn’t maintain itself. Over time, the team that wrote it moves on. As they do, the context behind why it was built fades.

Nobody quite remembers what the edge cases were, why that snowflake script was written that way, or the original problem it solved. And when it breaks–and it will break–you’re not debugging an application. You’ll be performing an archaeological excavation of your own infrastructure, to decipher the intent of the team that originally constructed the automation.

When scripts outlive their authors

So you do what engineers do: you automate around it. You add new automation on top of old to address the gaps. Now you’re managing two mountains of automation instead of one. And here’s the impact that rarely makes it into the original business case – the platform team doesn’t get to walk away when it’s “done” because it’s never done.

These engineers can’t be reassigned without your platform decaying beneath the applications and services that make up the business’s backbone. You need a robust team to manage this indefinitely just to keep the lights on. We’re simply trading software costs for people costs. And often, you end up spending more to produce something less scalable and capable than what you had from day one.

“Automation may mask complexity but does not eliminate it, and mountains of automation makes diagnosis and repair exponentially harder when things go sideways.”

This is the real trap. Automation, at its best, is a productivity multiplier. At its worst, it’s lipstick on a pig; all the ceremonies of agility without the true benefits. Automation may mask complexity but does not eliminate it, and mountains of automation makes diagnosis and repair exponentially harder when things go sideways.

What a pre-engineered PaaS actually does

A true Platform as a Service (PaaS) isn’t a collection of automation. It’s a pre-engineered system where the underlying plumbing, services, security, and resilience are already integrated before you ever install and consume it. This is the ‘batteries included’ model where the platform is ready to use on Day 1 based on best practices and proven architectures. This kind of integration out-of-the-box is one of the things that makes a platform trustworthy and predictable at scale.

For example, an integrated platform includes ‘how’ applications are built and deployed is pre-wired out of the box and works consistently across application types. One of the more unique things Tanzu Platform does is build deployment packages, including the base image for developers. This means that when a security problem comes out–like Copy Fail or the flood of AI-discovered vulnerabilities–the platform engineer can rebuild and redeploy apps very quickly without reaching back into the software delivery lifecycle (SDLC). They simply “restage’ the application using a single command.

The consistency of deployment packages and base images also enables developer velocity in the every day. When every application is built, packaged, and deployed the same way, developers stop re-solving the same infrastructure problems, and start focusing on the code that actually matters. This is a distinction worth drawing clearly–assembling capable open source tooling like Terraform, ArgoCD, Kubernetes, cert-manager, OpenBao, and Istio gives you powerful building blocks, but it doesn’t give you a platform.

You still own the integration, automation, the opinions, lifecycle management, and the operational model that ties them together. A pre-engineered PaaS handles the myriad of decisions for you. With a PaaS, onboarding a new team or new application isn’t a one-off integration project; it’s a repeatable, predictable process. Standardization is a core outcome of a pre-engineered PaaS, not a side effect, and it’s precisely what makes it possible to push changes faster and with more confidence, regardless of the team, language, or application type.

Security built into the platform

Security works the same way. When your team makes the decision to pull from open-source components and stitch them together, you now own every security gap between them – data at rest, data in flight, and its running state. This seems like a worthwhile investment at first, but as recent research suggests, AI-assisted attacks are on the rise, and platform teams won’t be able to keep up with surging security vulnerabilities in the build-it-yourself model.

A pre-engineered PaaS standardizes the governance and compliance posture across the board. Updates, patches, and fixes from a trusted, first-party vendor mean you’re not reinventing governance from scratch each time you add a new component. The PaaS can help you uniformly and at scale apply the changes in a cascading fashion rather than relying on custom automation.

The calculus starts to shift when you see deployment, security, and onboarding issues in aggregate. Achieving all of this with custom automation means spanning organizational silos, coordinating teams, sustaining headcount, and continuously funding work that never actually finishes.

And worse yet, this work has no meaningful competitive advantage for your organization. Your competitors are working to solve the same problem, but you’re just burning more resources to accomplish the same end result. You own the problem entirely rather than leaning on a trusted, proven vendor. Embracing a PaaS allows you to focus on higher-value, differentiating initiatives for the business.

AI is the forcing function

The conversation around PaaS is urgent again, and AI is why. Code generation can speed up your development cycles, building and pushing features faster, but production delays will persist if you’re still deploying at the same speed as before. 

To avoid eroding the benefits of code generation, you need to deploy applications nearly as fast as they can be coded with AI. This requires streamlining each step of the path to production. In an era where more organizations are exploring the use of autonomous agents, they need a platform that doesn’t take weeks to rotate credentials, days to provision a database, or require access to JIRA’s MCP Server to accomplish their goals.

“To avoid eroding the benefits of code generation, you need to deploy applications nearly as fast as they can be coded with AI.”

The pace of AI innovation itself compounds the problem. Whether it’s shadow AI use, MCP servers, agentic harnesses, this week’s new foundation models, or whatever emerges next, the landscape is evolving fast enough that what’s bleeding-edge today may be table stakes in six months. When you build your own platform, you’re on the hook to evaluate each layer of these new technologies, determine how they fit into your stack, and then integrate them yourself, on top of everything else your platform team is already managing to keep the business going.

Organizations running VMware Tanzu Platform receive those innovations. With Tanzu Platform 10.4, for example, customers gained MCP Gateway, an expanded service marketplace where developers can publish their own MCP servers and services for cross-organization consumption, as well as an agent buildpack to streamline and secure the deployment of agentic AI applications.

These are capabilities that would have taken a DIY team months to evaluate, build, and harden. And this isn’t unique to AI. Every release brings new capabilities across the platform that customers simply inherit, without the integration tax. That’s the compounding return on your investment in a pre-engineered PaaS. The platform keeps moving forward, and so do you.

VMware customers: You’re closer than you think

Platform engineers are uniquely equipped for this moment. They have the pattern recognition, they have seen the technology waves before, they bring their hard-won operational instincts, and the critical thinking skills to know when a system is genuinely resilient versus cleverly disguised complexity.

That’s exactly why their role in curating and operating this next generation of PaaS matters more than ever. Platform Engineers applying these battle-tested experiences using a PaaS will shortcut the tedious automation pitfalls, jumping straight to safely delivering Agentic AI and GenAI application services.

If you’ve already built your stack on VMware Cloud Foundation you’re already in a position to add the value of a true PaaS. VMware Tanzu Platform layers a pre-engineered PaaS on top of your existing infrastructure APIs, running alongside your VMs and containerized workloads, without ripping out what you’ve built and budgeting for alternatives.

With an incremental step, you can leverage what you know and trust, and layer on the only private cloud PaaS for agents that offers direct integration with Private AI services.

The post The DIY platform trap that’s burning out engineering teams appeared first on The New Stack.

The transition from traditional web applications to agentic ecosystems is more than a change in the UI; it is a fundamental shift in the internet’s threat model. We are moving from a world where “bad input creates bad data” to one where “bad input creates bad actions.” As AI agents evolve from simple chatbots to autonomous conductors capable of calling APIs, reading sensitive files, and sending emails, our legacy security models are cracking under the pressure.

If you are building or deploying AI agents today, you are likely sitting on an IAM problem in disguise, considering that agents are outnumbering humans 144:1. In a recent global Enterprise Management Associates (EMA) survey on agentic, 95% of participants were in production or limited pilot programs using AI agents. Here is how to navigate the shift from human-centric security to the Agent IAM era.

1. What’s the problem? (The identity vacuum)

The core problem is that AI agents currently operate in an Identity Vacuum. In most production environments, agents are given ambient, inherited access. They run as service accounts with broad permissions or, worse, inherit the full permissions of the human user who triggered them.

This creates three critical vulnerabilities:

• The Action-Based Threat Model: Unlike traditional apps, agents “do” things. If an LLM is tricked via prompt injection, it doesn’t just display a wrong answer; it executes a malicious tool call. 80% report seeing apps act outside of intended boundaries.
• The RAG Attack Surface: Retrieval-Augmented Generation (RAG) systems are vulnerable to indirect prompt injection. If an agent fetches a document containing malicious instructions, that document becomes the new “master” of the agent, overriding developer guardrails.
• Non-Human Identity (NHI) Explosion: We are seeing a massive surge in APIs, services, and autonomous agents that lack a centralized source of truth for identity. 39% report unauthorized access incidents with agents, and most teams have no way to revoke an individual agent’s access without breaking the entire service.

2. Why does it matter? (The widening remediation gap)

Anthropic’s Claude Mythos discovery recently highlighted the stakes. The model identified thousands of zero-day vulnerabilities across major OSs and browsers, including bugs that had survived 20+ years of human review.

This matters because AI is now a force multiplier for vulnerability discovery. While AI can find bugs at machine speed, humans still remediate them at a “human pace” (meetings, backlogs, patch cycles). 

“While AI can find bugs at machine speed, humans still remediate them at a ‘human pace.'”

If your IAM infrastructure is homegrown or unmanaged open source, you cannot patch fast enough to keep up with an AI-powered attacker. Identity is the most exposed layer because it is the control plane; if the agent’s identity is compromised, the entire infrastructure is open for lateral movement. SailPoint research reports 33% have seen agents inappropriately handle restricted data.

3. How do I fix the problem? (The agentic IAM blueprint)

Fixing agentic security requires moving the guardrails from the LLM prompt to the infrastructure. You cannot talk an agent into being secure; you must authorize it to be secure. Compounding the agentic problem, the majority of EMA survey participants do not believe their IAM solutions are ready:

• 62% state not ready for agentic resiliency
• 49% claim not ready for agentic compliance
• 62% report not ready for agentic scale
• 59% disclose not ready for agentic security

“You cannot talk an agent into being secure; you must authorize it to be secure.”

Treat agents as first-class identities

Agents must be treated as first-class non-human Identities. This means:

• Authentication: Agents should authenticate against an Identity Provider using scoped credentials.
• Short-lived tokens: Use OAuth2 to issue tokens that are interaction-scoped. If an agent is compromised, the token expires quickly, limiting the window of exploitation.
• Relationship-based access control (ReBAC): Use a graph-based permission model to define exactly what an agent can touch.

Align retrieval with authorization

In RAG systems, the “view” permission must match the “retrieval” permission. Before an agent fetches a document to place in its context window, the system must check: Does this specific Agent ID have permission to view this Document ID? If not, the document is never retrieved, preventing the agent from ever seeing and being influenced by malicious payloads.

Engineers as conductors

Shift your engineering mindset. Stop trying to hard-code every agent action. Instead, act as a conductor, orchestrating agents through Policy as Code. Use tools to visualize these complex permission chains so you can see exactly how an agent’s relationships resolve to ALLOW or DENY.

4. “Gotcha” problems & how to avoid them

Even with a solid plan, several hidden costs and technical traps often emerge:

• The inherited access trap:
  • Problem: Developers often give agents Admin rights to simplify development.
  • Fix: Implement Least Privilege Access from day one. If an agent only needs to read Marketing docs, don’t give it access to the whole S3 bucket.
• The feedback loop delay:
  • Problem: As you add security layers, agent latency increases, leading users to bypass security for speed.
  • Fix: Use high-performance permission engines that can resolve complex queries in milliseconds, ensuring security doesn’t buffer the user experience.
• The ghost agent problem:
  • Problem: Agents are created for a task, the task ends, but the credentials remain active.
  • Fix: Implement automated lifecycle management. Use Token Chain Revocation so that if a parent orchestrator agent is flagged, all child agent tokens are instantly invalidated.
• Visual blindness:
  • Problem: Permission models for hundreds of agents become too complex to hold in a human brain.
  • Fix: Use visualization tools to audit your models. If you can’t see the graph, you can’t secure the graph.

Summary: Identity is where you start

Security is a process, not a product. While LLM guardrails and prompt hardening are important, they are easily bypassed. The only hard boundary that stays firm in the face of an autonomous agent is the Authorization Boundary.

Treat your agents as identities, scope their world with ReBAC, and ensure your IAM stack is professionally managed to keep up with the AI-driven pace of discovery. The future of the internet is agentic; make sure your security is too.

The post The agentic identity crisis: Why your security isn’t ready for the AI revolution appeared first on The New Stack.

Much of the conversation around AI coding is still centered on how fast machines can produce code. But code volume is not the same thing as software progress. As teams rely on agents for larger units of work, the harder question is whether they can build a repeatable system to steer, check, and correct machine-produced code before it creates downstream risk.

One useful way to think about that system is through the Agent Centric Development Cycle (AC/DC) framework. At its core, AC/DC defines four stages that govern how agentic development actually works at scale: Guide, Generate, Verify, Solve. Of those stages, Generate, the act of AI agents producing code, gets most of the market attention. But in practice, the framework stands or falls on the strength of the layers around it. If Guide is weak, agents start from the wrong assumptions. If Verify is weak, errors compound invisibly. If Solve is weak, teams inherit a growing queue of problems with no scalable way to address them.

Why verification has moved to the center

For years, modern software delivery was organized around a human pace of work. Developers wrote code in relatively small increments. Teammates reviewed it. The pipeline validated it. Problems were usually caught after the code had already been authored, but before they grew too large to understand.

Agentic development changes those conditions. Instead of a few hundred lines shaped through continuous human interaction, teams may now receive thousands of lines created in longer reasoning loops across multiple files and layers of the stack. At that scale, traditional review practices start to strain. The burden of understanding change rises much faster than the speed of generation.

“If organizations continue to treat verification as a late-stage checkpoint, they will discover that code generation has outpaced their ability to establish trust.”

That creates a governance problem. If organizations continue to treat verification as a late-stage checkpoint, they will discover that code generation has outpaced their ability to establish trust. This is where many teams will feel the first real friction in AI-assisted development: not at the moment of creation, but when they are asked to approve, merge, and maintain what was created.

Guide: Give agents boundaries, not just prompts

The first requirement in an agentic workflow is guidance. Not generic prompt advice, but structured context.

Agents need to understand more than the task in front of them. They need to understand the environment in which that task sits: architectural boundaries, engineering standards, compliance expectations, naming conventions, and the practical constraints that rarely live in a single document. Without that, an agent can produce something that appears correct locally while still being wrong for the broader system.

This is one of the central misconceptions in current AI tooling discussions. Many teams assume stronger models will naturally reduce the need for explicit guidance. In reality, the opposite is often true. The more work delegated to agents, the more important it becomes to define the terrain clearly. Guidance is what reduces avoidable drift before it enters the codebase.

In that sense, the “Guide” stage is not just preparation. It is the first layer of control.

Verify: The layer that turns speed into trust

Verification is where agentic development becomes either manageable or fragile.

AI systems often fail in ways that are hard to spot early: hidden logic flaws, reliability problems, security issues, or maintainability costs that only emerge later. Because these models are probabilistic and context-sensitive, verification cannot be a cursory review step. It has to be a core function of the development cycle.

That means verification has to happen in two places: inside the working loop while the agent is still generating, and again after the agent believes it has finished. The first catches mistakes early and helps steer the next step. The second tests whether the output actually satisfies functional, non-functional, and organizational requirements. 

This changes the role of feedback. Instead of surfacing issues only after a large pull request lands on a human reviewer, verification becomes an active part of shaping the work.

It also needs to be explainable and repeatable. Deterministic analysis, security checks, complexity analysis, and testing create evidence. They show what was checked, what passed, what failed, and why. In enterprise settings, that transparency is the basis for accountability.

“Code quality, in other words, is no longer just a maintainability concern. It is starting to look like an AI infrastructure efficiency variable.”

Code quality increasingly affects the economics of AI-assisted development. In a controlled study Sonar conducted using matched repository pairs with the same external behavior, architecture, dependencies, and test coverage, agents working in the higher-quality codebases used about 7% fewer input tokens, 8% fewer output tokens, and 11% less reasoning effort on average. They also re-read files 34% less often, a useful signal that clearer code reduces uncertainty and enables agents to commit edits more confidently. Code quality, in other words, is no longer just a maintainability concern. It is starting to look like an AI infrastructure efficiency variable.

Solve: Close the loop instead of growing the backlog

A verification layer is only useful if it leads to action.

That is why Solve matters so much in an AC/DC model. When issues are identified, the process needs a systematic way to remediate them, re-check the fixes, and learn from the results. Otherwise, verification becomes a reporting mechanism rather than an operational one. This is especially important in environments where AI is increasing the total volume of code under review. Without a remediation mechanism, every detection system eventually becomes a backlog generator. 

Solve is what prevents that failure mode. It turns findings into an iterative loop. Fixes are proposed, rechecked, and fed back into the next cycle so the system improves over time. In mature workflows, this means developers spend less energy chasing repetitive issues and more energy on architecture, judgment, and higher-order design decisions.

The real shift

The practical takeaway is simple. In an agentic development model, the primary challenge is no longer writing code; it is creating a system that makes generated code trustworthy.

Teams still need strong models and useful tooling, but the real differentiator is everything that surrounds generation: the quality of the context agents receive, the strength of the verification layer, and the ability to remediate issues quickly enough to keep pace with machine output.

The organizations that adapt fastest will not be the ones generating the most code. They will be the ones who can consistently turn that code into software that is understandable, governable, and production-ready.

“In the age of software agents, the real advantage will not come from generation alone. It will come from building the discipline around it.”

In the age of software agents, the real advantage will not come from generation alone. It will come from building the discipline around it.

The post How the AC/DC framework helps teams govern AI coding agents appeared first on The New Stack.