We’re more than 20 years past the launch of AWS, the starter gun for the shift of compute and storage from on-prem racks to the cloud. 

The rapid growth of AWS and competing services like Azure and Google Cloud underscores how many companies have made the jump from controlling their own infrastructure to renting capacity from hyperscale public clouds.

For the major providers, the public cloud has proved an incredible business. Amazon’s cloud service generated nearly 60% of its first-quarter operating profit, for example. For cloud customers, however, the tides may be turning.

Think back to the early days of the public cloud. Azure and AWS scrapped for market share, offering price cuts to entice workloads to their centralized silicon. The situation has evolved over the ensuing decades. Today, cloud costs are material and rising, prompting some companies to question whether being cloud-first is the best path forward.

Cloud bills are expanding due to increased usage of hyperscaler infrastructure, yes, but also because many customers today use the cloud for everything, rather than for what it is best suited for.

n the latest episode of The New Stack podcast, Byron Dill, Director of Solutions Engineering at Summit, tells us that shared compute and storage have their place in the modern IT mix, but that many companies would do well to segment their workloads and move some of that work back on-prem. (Think lower costs and simpler management of high-risk data.)

The argument echoes what we’ve seen recently in the AI realm. Many companies quickly adopted AI technology, only to be surprised later by the bills they incurred. The public cloud is a similar frog-boiler, albeit on a slightly longer timeframe.

In both cases — AI and the public cloud — companies have learned that a product once pitched as a way to reduce spend can evolve into the opposite without careful management. Summit, which offers managed private clouds to enterprise customers, thinks that some corporate workloads should be removed from the cloud and moved in-house.

What will that cost? How long does it take to move? And which industries are most primed to benefit from their own private cloud? We get into it all in this episode.

The post Public cloud vs. on-prem: Summit on where each workload belongs appeared first on The New Stack.

Kubernetes teams automate deployments without thinking about it. CI/CD pipelines fire dozens of times a day, autoscaling adjusts replicas in the background, rollback is muscle memory. But there is one category of automation where that confidence vanishes: letting a system change CPU and memory requests on a running workload without a human reviewing it first. 

And as AI inference lands on Kubernetes at scale, that hesitation is becoming hard to ignore, and increasingly expensive.

Why teams trust automation for change but not for constraint

We surveyed 321 Kubernetes practitioners at enterprise organizations earlier this year. The headline finding is one most practitioners will recognize immediately: 82% report high or complete trust in automated delivery controls. But 71% still require human review before applying resource optimization recommendations. Only 27% allow CPU and memory changes to be auto-applied, even within guardrails.

“Deploying code feels additive… rightsizing feels subtractive because you are removing safety margin from a running service, and the failure mode is fundamentally different.”

Those numbers describe a specific asymmetry. The same engineers who deploy to production dozens of times a day without hesitation slow down the moment automation wants to adjust resource allocation. And the survey data make it clear why. Deploying code feels additive. You are shipping new value, the rollback path is well understood, and if something breaks you usually see it right away. Meanwhile, rightsizing feels subtractive because you are removing safety margin from a running service, and the failure mode is fundamentally different.

As one practitioner in the survey put it: “Automated right-sizing carries a unique risk because it directly impacts the underlying stability of the application runtime. Unlike a code deployment that follows a tested path, resource changes alter the invisible contract between the workload and the scheduler.”

When you change resource requests, you change how Kubernetes schedules, prioritizes, and allocates resources. Those effects are not visible the way a code change is. You can’t trace them through a deployment pipeline. And you might not discover that something went wrong until two weeks later, when a traffic spike hits a threshold that didn’t exist at the old values. By that point, three other things have changed too, and proving causation is nearly impossible. The people responsible for those workloads are the same people who get paged at 2 a.m., and they know this.

Why AI workloads raise the stakes

That trust gap existed before inference workloads showed up. What’s changed is the cost of not closing it.

For a long time, teams could absorb the cost of manual oversight. They knew their workloads, had intuition for where the safe boundaries were, and the inefficiency of over-provisioning was a price worth paying for stability. GPU-accelerated inference workloads change that math. GPU compute is significantly more expensive per hour than CPU. The cost of over-provisioning is no longer a rounding error you can absorb quietly. And the workload behavior is less familiar, as inference jobs are bursty in ways teams haven’t built intuition for, traffic patterns shift as models are updated and usage changes, and the resource dimensions involved differ from what teams have spent years learning to tune.

That unfamiliarity compounds with scale. Rightsizing isn’t a one-lever problem the way horizontal scaling is. It involves, at minimum, CPU and memory requests and potentially limits for both, with four dimensions per workload, multiplied across hundreds or thousands of workloads per cluster. The survey data indicates that manual optimization breaks down at around 250 changes a day. Inference workloads push teams past that threshold faster than anything they’ve managed before, because the resource decisions are more frequent and the cost of getting them wrong is higher.

The economic case for automated rightsizing has never been stronger. The organization’s willingness to delegate hasn’t caught up because teams are being asked to trust automation with workloads they don’t yet have a track record with.

What the survey says about closing the gap

When we asked practitioners what would actually increase their trust in optimization automation, 48% said visibility and transparency into how decisions are made, 25% wanted proven guardrails, and 23% needed instant rollback.

Nobody asked for full manual control and very few asked for blind autonomy. What they described is automation that earns trust in stages, and that’s consistent with how the teams furthest along in their automation journey actually got there. They didn’t start with production. They started with a single namespace in a dev environment, observed the system’s behavior, compared recommendations with outcomes, and gradually expanded the scope. Different environments remained at different levels of automation maturity simultaneously, and that was intentional. Production carried more scrutiny than dev.

CI/CD followed the same curve, and the timeline is easy to forget. Most organizations took years to get from running their first automated pipeline to trusting it with production deploys without manual approval on every commit. Kubernetes resource automation is earlier in that same process, and AI workloads are extending the timeline because teams are building trust from scratch with a workload category that doesn’t yet have a track record.

Why automation design matters as much as capability

Some automation architectures deliver meaningful value only with full delegation. The system needs complete control to function the way it was designed to. That’s a form of forced autonomy, and it creates an adoption problem because it asks for exactly the level of trust that most organizations haven’t built yet. Force generally doesn’t work. Teams that feel pushed into a level of delegation they aren’t comfortable with tend to pull back entirely after the first incident.

The alternative is what I’d describe as adaptive autonomy: designing the system to work at every stage of the trust curve. A team still evaluating gets useful recommendations in read-only mode. A team ready to act but wanting boundaries can run guardrailed execution within limits they define. As confidence grows, the system handles more decisions autonomously while humans manage exceptions. And for environments where the track record supports it, closed-loop optimization runs in the background and becomes boring, which is the goal. Each stage is a legitimate operating mode, not a stepping stone you have to rush through.

That design distinction matters more with AI workloads than it ever did with traditional services, precisely because the trust-building process is starting from zero on workloads where the cost of getting it wrong is highest.

“Trust takes a long time to build and a single production incident to undermine.”

The other piece that makes this sustainable is rollout safety. Trust takes a long time to build and a single production incident to undermine. Start with the workloads showing the most headroom between requests and actual usage. Make changes incrementally, small enough that a bad outcome stays contained. Rollback needs to be fast and tied to the health signals the team already monitors. And start with opt-in, not opt-out. Let the teams willing to go first build a track record that others can look at.

The broader pattern

The 71% figure is sometimes read as resistance to automation. I think it’s a more accurate picture of how operational trust actually forms: conditional, earned over time, and moving at different speeds depending on what’s at stake. AI workloads are raising those stakes significantly, which means the path to trusted automation matters more now than it did when the cost of caution was just some unused CPU.

“Most of what gets written about Kubernetes optimization focuses on tooling capability, and the tooling is capable. The harder problem is the human one.”

Most of what gets written about Kubernetes optimization focuses on tooling capability, and the tooling is capable. The harder problem is the human one. If your team is managing AI inference workloads on Kubernetes and your optimization tooling is sitting in read-only mode, the question worth asking isn’t whether to trust the system. It’s whether the system is designed to let you build that trust gradually, starting where the stakes are low and expanding as the evidence supports it, on workloads where getting it wrong costs more than it ever has before.

The post Kubernetes teams trust automation to ship code but not to touch CPU, and AI is raising the stakes appeared first on The New Stack.

Memory has replaced compute as a primary constraint for modern tech teams. A perfect storm of hardware architecture limitations, semiconductor supply chain uncertainty, and changing software licensing models has left enterprises confronting increasingly memory-constrained environments. All while high-bandwidth AI workloads overload the production chain’s ability to provide sufficient memory — and when your AI token bill is starting to cost more than your salary bill.

Over the past year, the cost of high-bandwidth memory (HBM) and dynamic random access memory (DRAM) has increased by an unprecedented 170%, with some virtualization subscriptions more than doubling in price.

All this adds up to a demand for enterprises to shift from the previous buy-all-you-can mindset to a data-driven optimization strategy.

Fortunately, AI isn’t only part of the problem. When AI is applied to memory economics in modern virtualization, it becomes a vital part of the solution. 

Bharath Ram, director of product management at Hewlett-Packard Enterprise (HPE), explains it to The New Stack this way: “There’s a component shortage. Today the prices have increased. So customers are looking at ways to save and optimize the existing footprint, so that they can run their workloads on whatever and not have to procure anything new.”

By switching focus from gobbling up every bit of memory your organization can grab to optimizing your workloads and their placement across multi-cloud and hybrid-cloud environments, enterprises cannot only speed up modernization but also shorten decision-making cycle time by up to 80%, all while cutting costs by up to 50%. Read on for how to transition your enterprise from guesswork to smarter IT.

Tech has to confront its waste problem

Most enterprises operate with significant over-provisioning driven by limited visibility and risk avoidance.

These same companies are standing up legacy applications that become less efficient over time, including a significant number of zombie services running without any use. 

On top of this, most AI workloads rely on advanced memory technologies, which has led chip manufacturers to shift production priorities from DDR4 to DDR5 RAM, further reducing DDR4 supply. This impacts the whole industry, with even a personal computer costing 15% to 30% more than last year.

Add to this volatile DRAM pricing and higher core densities, and it’s clear that even non-technical leadership is worried about memory efficiency. Tech giants Microsoft, Google, Amazon, and Meta are buying up as many AI chips as they can, which is triggering even more shortages and price pressure across the supply chain. And thus more enterprises are hoarding more infrastructure and memory. 

And this overbuying isn’t limited to memory. Companies are now also buying infrastructure like servers even before they need them, too, Ram reflects, “because the cost is so exorbitant, the quote that you might have today might not be the same price that you’re quoted for the same infrastructure tomorrow. That’s how we’re seeing the market right now. It’s very volatile.” 

But it might all be ok. HPE estimates that between 20% and 40% of infrastructure is overprovisioned today. Which is an opportunity for efficiency — not only in these limited resources but also in faster, more secure workloads. 

Enterprises are more capable than ever to optimize the use of what they’ve got today, especially before they go searching for more RAM that will cost significantly more.

It all starts with understanding

So much of this waste persists because enterprise infrastructure is obscured — no one really knows what does what with which data, or which services rely on it. 

The same thing that holds companies back from doing anything more than lift-and-shift to the cloud is usually what keeps them from unlocking memory efficiency. There’s simply too little visibility across most enterprises’ complex, hybrid and multi-cloud distributed systems. Which has left organizations guessing and then rounding way up for over a decade now.

“It’s a combination of over-provisioning and not understanding underlying infrastructure. Because many of them are doing public cloud-based provisioning and self-service, where you don’t know what the underlying infrastructure is and you have admins leveraging whatever there is in terms of their service capabilities,” Ram explains. “One piece is memory shortage, and the other is understanding what’s been deployed and rightsizing it.”

To break these over-provisioning bad habits, any change has to be grounded in reality. The first step is to gather and analyze real usage data, using a tool like HPE CloudPhysics to establish a factual baseline that separates real cost drivers from those years of assumptions. 

This allows enterprises to:

• Understand their virtualization footprint and licensing exposure.
• See their workload initialization and efficiency.
• Identify true cost drivers before taking action.

You cannot right-size until you have real-time monitoring of how many hosts have how many VMs, and which are on and off.

Predictive, not reactive provisioning

Once an enterprise has a single source of truth for its complex distributed systems, it can explore what to deploy, where, when, and how.

“An application like SAP HANA is highly memory-intensive and highly latency-intensive. It’s not like this algorithm is optimized to pivot between hot and cold memory tiering” for cost reduction, Ram explains, without risking the application performance, akin to how, when older PCs had limited amounts of memory and, once that ran out, the computer would swap the program from running in memory to disk, slowing way down. 

Part of the modern solution, Ram argues, is that companies “can over-provision with what they already have. They don’t have to buy any new memory,” because of better shared resources available to all the virtual machines managed by a single host. 

“For example, a host with 64GB of physical memory may have more memory allocated across VMs than physically available,” he explains. “In practice, not all VMs consume their full allocation simultaneously, allowing unused capacity to be dynamically reassigned where needed.”

Memory ballooning, which, Ram says, is nothing new, but something desperately needed in the market right now. Version 9.0 of Morpheus, due out this summer, will feature a more modern sort of memory oversubscription, which, HPE explains, allows administrators to oversubscribe physical memory across VMs on a host, enabling higher VM density and more efficient use. This is particularly useful for testing and development environments, virtual desktop infrastructure, and workloads with variable memory demands.

Shift to architectural efficiency 

Eventually, once you’ve optimized and rightsized every memory allocation, it’s time to shift your workloads to a new platform to improve hardware efficiency. 

“The final step is increasing workload density per server, especially as per-core software licensing becomes more expensive,” Ram explains. He says this is best achieved using a virtualization solution with an open-source hypervisor, which can improve utilization now and help organizations shift toward per-socket licensing models. For suitable applications, that modernization may also include moving to containerized deployment, while in-memory deduplication reduces redundant data structures in RAM, improving memory efficiency.

“Not everybody can keep running on existing hardware forever. At some point, some organizations will need to move to a new platform to improve hardware efficiency. But higher workload density still brings added benefits,” he continues, especially at a time when even the biggest tech companies are overbuying infrastructure, driving up costs and tightening capacity.

Per-socket licensing saves more

And if you do go for a hardware refresh with HPE’s Morpheus Software, then you can unlock a different kind of subscription model, which charges per socket or CPU licensing, where multiple cores can share one socket. Some early results indicate that this can deliver up to 90% in savings.

In the end, it all starts with that baseline. Take the free cloud visibility assessment to see where your organization stands.

The post How AI is solving the memory crunch it created appeared first on The New Stack.

To hear Yasmin Rajabi, chief operating officer at CloudBolt, tell it, there’s an imbalance in how we view automation. We’re happy to automate decisions that result in more productivity and processes — but what about when it comes to turning the dial to the left? For some reason, there’s hesitation there.

“Trust is super-high when it comes to traditional automation, but there’s still a lot of caution when it comes to right-sizing,” Rajabi tells The New Stack. “The same engineers who are deploying multiple times a day through CI/CD aren’t questioning [automation] anymore, but when it comes to delegating right-sizing to the machine, the bar to earn that trust is much higher.”

The data reveal why this imbalance might exist: When faced with the pressure to remain always-on, a higher cloud bill from over-provisioning seems worth the cost. But now that GPU-heavy AI workloads have sent cloud bills soaring, right-sizing automated processes has become a priority for 89% of organizations, according to the March 2026 CloudBolt Research Report.

And yet, 71% of Kubernetes engineers respond that they still require human review for resource optimization, with only 27% allowing CPU and memory changes to be auto-applied. So while the data shows it’s a priority, that motivation hasn’t shown up in the workflows.

“When it comes to delegating right-sizing to the machine, the bar to earn that trust is much higher.”

The New Stack will sit down with Rajabi and Reid Vandewiele, product lead at StormForge, at 9 a.m. Pacific/5 p.m. BST on Wednesday, June 24 to discuss the urgency of this right-sizing gap — especially when it comes to Kubernetes workloads for AI.

Join us live to not only learn how to measure your organization’s automation maturity, but to develop this trust over time, with strategic CPU throttling, out-of-memory (OOM) behavior, and, of course, rollback patterns. 

Register to join this conversation

Right-sizing is a multi-dimensional problem, Rajabi explains, spanning increasingly complex workloads in increasingly complex environments, so that when something goes wrong, it feels almost impossible to reverse. For this automation to work, this trust has to be built not just across teams but scaled across your organization.

“It takes a long time to build up trust in an automation solution, and it’s very fast to eliminate or significantly undermine that trust,” Rajabi warns. “It takes one production incident to take an application team from being willing to entertain automated resourcing to absolutely not, ‘not on my application, we’re special’.”

Join us June 24 to learn how to gain insight into how much your AI workloads actually cost, and to adopt a plan that takes the guesswork out of your provisioning.

The post The fix for soaring AI cloud bills exists — so why won’t we trust it? appeared first on The New Stack.