Microsoft is positioning the new Surface Laptop Ultra as an alternative to Apple’s MacBook Pros for creatives and developers — and especially AI developers.
Built on Nvidia’s N1X-based RTX Spark platform, with a Blackwell RTX GPU with up to 6,144 cores, a CPU with up to 20 cores, up to 128GB of unified memory, and a price that undercuts Apple, it’s a tempting alternative if you want to run local AI models and also want to have a powerful machine for editing videos and gaming on the go.
This is The New Stack, so I’m leaving the gaming benchmarks and creator use cases to others. My focus in testing the unit was on running AI workloads.
I’ve been using the Surface Laptop Ultra in its high-end $4,300 GPU and CPU configuration with 64GB of RAM and a 1TB SSD for the last week, including while on the road.
Typically, I use a Mac Studio with the M4 Max chip at home, which happily runs most local models I throw at it (as long as they fit into the 48 GB of RAM). I installed all the usual tools I use to run local models on the laptop. Think Unsloth Studio, Ollama, Hermes Agent, etc.
At no point did I feel like I was making a compromise in running those on the laptop. Indeed, if anything, the Blackwell GPU was generally a bit faster than my Mac Studio.
Since the new Nemotron 3.5 Bolt model launched earlier this week, I also took that one for a spin. It answered queries at an impressive 56 to 60 tokens per second — and sometimes more. That makes it a great model for running in Hermes Agent or OpenCode, for example. It’s a heavily quantized model with 9B active parameters, so you would expect it to be fast, of course.
Yet, while the laptop has the unified memory to run it, the dense Qwen 3.8 27B at 8-bit quantization never achieved more than 8 or 9 tokens per second in Unsloth Studio. That’s not quite usable, but with these dense models, that’s virtually always due to memory bandwidth. But bringing it down to a 4-bit quantization brought the speed up to a respectable 25 to 28 tokens per second.
Using the 8-bit Gemma 4 26B-A4B shows how the mixture-of-experts technique is really what you need to run virtually any model locally at reasonable speeds. For this model, the Surface Laptop Ultra generally hit around 40 tokens per second.
I don’t have an M5-based MacBook or Mac Studio at hand, but Tom’s Hardware reports that the Ultra generally outperforms the M5 Max in a MacBook Pro in AI use cases.
All of those previous numbers were with the laptop on its 140W charger. Microsoft says it tuned the system so that running on battery would still give users most of the full performance. Microsoft claims its benchmarks showed that the laptop retained 99.6% of its performance when running on battery, but at least for the AI workloads I tested, that drop was far more noticeable.
The 4-bit Qwen 3.8 model, for example, dropped from its 25-28 tokens per second to around 18 tokens per second on battery. For Nemotron 3.5 Bolt, the performance dropped to around 42 to 45 tokens per second. Overall, it seems fair to expect around a 20% to 25% loss in performance when on battery when running AI workloads (and games show a very similar drop).
I did not run any formal battery life tests. Microsoft says users can expect 12 hours of active web usage. If you are consistently running local AI workloads, though, I wouldn’t expect more than a few hours of battery life based on my observations.
Having 64GB of RAM puts you in a bit of an odd situation given that most smaller models right now are in the 27B to 30B league. The extra RAM gives you plenty of headroom for the KV cache and other applications, but there aren’t a lot of models in the 32-48GB range, for example, that the machine could also handle.
Going for 128GB of RAM brings the price of the Surface Laptop Ultra to $5,900
Unsurprisingly, all of the compute power (Microsoft says you get up to 1 petaflop of FP4 AI performance) creates a bit of heat.
Nobody is going to argue that the Surface Laptop Ultra is whisper-quiet when you run AI applications. The noise its dual fans make isn’t extreme, but enough to get a bit on your nerves while you are working. That’s especially noticeable when Hermes or your favorite coding agent is chugging along for a while on a more complex query.
There is a reason the Surface RTX Spark Dev Box, which is built on top of the same N1X platform, has a thousand air vents, after all (which will also likely allow it to push its performance beyond the N1X in the laptop).
Outside of that, the laptop is very quiet, though, and the fans remain off for most day-to-day use cases. It also quiets down very fast once the GPU load drops again.
I would feel remiss if I didn’t highlight a few other things I found while testing the Ultra. This is, after all, Microsoft’s most ambitious laptop yet.
The post Running local AI models on Microsoft’s Surface Laptop Ultra: What we found appeared first on The New Stack.
Q.ANT, a startup out of Stuttgart, Germany, builds processors that use light instead of electricity to do some of the math behind AI. The company pitches them as a way to run AI on a fraction of the power today’s chips need.
Now developers can start writing software for those chips without owning one. Q.ANT pushed a free, open-source software kit to GitHub this week that lets developers build and test programs on a normal computer, then run them on the real chips once they get access.
This is a move out of Nvidia’s playbook. Nvidia owes its lead in AI as much to CUDA, the software developers use to program its GPUs, as it does to the chips themselves.
But with Q.ANT, the catch is the hardware. Q.ANT’s chips are running at a few research computing centers, and everyone else has to wait “the coming months” for cloud access through German provider IONOS or an on-site server from Q.ANT.
The kit, called the Q.ANT Native Computing Toolkit, is free on GitHub under a license that allows commercial use. Developers can work in Python or C. The key piece is a simulator that mimics the chip on a regular computer, with no Q.ANT drivers required.
What can it do today? The AI tools in this first version focus on running models that have already been trained. The examples read handwritten numbers, identify objects in photos and outline shapes in images. Training still happens on regular CPUs and GPUs.
The pitch for photonic computing is power. AI chips burn a lot of energy moving data back and forth between memory and the processor. Q.ANT’s chips do part of the math with light, specifically wave-shaped functions similar to a cosine, which regular chips calculate digitally. Q.ANT says AI models built around those functions get better results with fewer parameters, the settings a model learns during training. Fewer parameters means a smaller model, less data to move and less power. The kit includes examples comparing a standard model with one built Q.ANT’s way. Those comparisons are the company’s own.
“An ecosystem isn’t created by hardware alone. It emerges when the software layer is open and others can build on it,” said Michael Förtsch, Q.ANT’s founder and CEO. He calls the release the “Linux moment” of photonic computing.
Q.ANT is betting light can do the math itself. Lightmatter, one of the best-known companies in the field, now puts its focus on Passage, which uses light to move data between chips. The idea of light-based AI isn’t new, either. TNS covered MIT’s photonic processor for building optical neural networks back in 2017.
Q.ANT raised €62 million in July 2025 in a round led by Cherry Ventures, UVC Partners and imec.xpand. In March, it said its second-generation chips were running at the Leibniz Supercomputing Centre near Munich. The results it published from there compare the new chip with its old one: more than 50 times faster at the kind of math that does most of the work in AI models, and six times less energy on typical jobs, by the company’s numbers. Its bigger claims, like up to 30 times better energy efficiency, don’t say what they’re measured against.
Good software alone won’t carry a new chip. Nvidia has been building CUDA for nearly 20 years and is still adding to it, including deeper native Python support last year. Graphcore, the British AI chip startup, had its own software kit and still ended up being sold to SoftBank in 2024.
Q.ANT calls this the first openly available software kit for programming a photonic processor. That depends on how you count. Xanadu has offered free, open software for its light-based quantum computers since 2018. For now, developers can play with the simulator. What they can’t do yet is test Q.ANT’s power-saving claims on their own models. That has to wait until the chips open up.
The post Q.ANT gives away the software for its light-powered AI chips in a CUDA-style bet on developers appeared first on The New Stack.
The 1.58 in a 1.58-bit language model sounds like a hard limit, but Intel researchers pushed a ternary model below it by changing how its weights are stored rather than changing the model itself.
Their new BITCOS format compressed one checkpoint to 1.485 bits per weight and improved decoding throughput by as much as 18% on CPUs and 27% on GPUs.
The key is that the familiar 1.58-bit figure assumes a model uses its three possible weight values equally, while real ternary models contain far more zeros than that calculation accounts for. BITCOS stores the location and sign of each nonzero weight separately, allowing zeros to take up less space without retraining the model or altering its output — the equivalent of packing the same contents into a smaller box.
Ternary models use only three weight values — -1, 0, and +1 — and 1.58 bits is the theoretical minimum needed to represent three equally likely options. That number is cleaner than the reality of storing the weights, where the standard approach fits five ternary values into an eight-bit byte for an average of 1.6 bits each. Models commonly store weights in blocks of 128; however, this leaves the final byte partly unused and pushes the actual rate to 1.625 bits per weight.
When Intel’s researchers measured the distribution of weights across 29 checkpoints from seven ternary model families, they found that zeros accounted for between 29.7% and 51.5% of the weights. In 26 of those checkpoints, there were enough zeros for BITCOS to beat five-trit packing.
The sparsest was a ternary version of Qwen3-1.7B produced with CAT-Q post-training quantization, where 51.48% of the weights were zero, and BITCOS brought the storage cost down to 1.485 bits per weight.
Models commonly store weights in blocks of 128; however, this leaves the final byte partly unused and pushes the actual rate to 1.625 bits per weight.
BITCOS stands for “BITmap and COmpacted Signs” and divides a model’s weights into two streams. The first assigns one bit to every weight to record whether it is zero or nonzero, while the second assigns a sign bit only to nonzero weights.
A positive or negative weight therefore consumes two bits, but a zero needs only the presence bit because it has no sign to record.
If z is the proportion of zero weights, BITCOS uses 2 − z bits per weight, dropping from 1.6 bits at 40% zeros to 1.485 bits at 51.5%. Because it changes only the storage format, unpacking restores the original -1, 0 and +1 values without affecting accuracy.
BITCOS becomes smaller than five-trit packing once more than 37.5% of a model’s weights are zero, a threshold reached by 26 of the 29 checkpoints Intel examined.
BITCOS becomes smaller than five-trit packing once more than 37.5% of a model’s weights are zero, a threshold reached by 26 of the 29 checkpoints Intel examined.
Built for token-by-token decoding with small batch sizes, the format reduces the weight data moving through memory. Intel developed separate unpacking kernels for AVX-512 and AVX2 CPUs as well as Xe2 GPUs, joining other efforts to fit compressed models into faster inference pipelines for AI agents.
On AVX-512 hardware, the kernel uses the presence bitmap as a mask and pdep to scatter the compacted sign bits across the nonzero weight positions. Because Xe2 GPUs lack an equivalent instruction, Intel implemented the same operation with a 2KB lookup table.
Compared with the 2-bit kernels, BITCOS ran 10% to 18% faster on the 64-core Xeon server and 2% to 15% faster on the 24-core Core Ultra 9. Performance improved by 9% to 22% on the integrated Arc 140V and by 2% to 27% on the discrete Arc Pro B70. These results measure decoding after the model has loaded, separate from efforts to cut GPU inference cold starts from minutes to seconds.
On the eight-core Lunar Lake CPU, Intel’s fixed 2-bit kernel beat BITCOS on every model because the system had enough bandwidth to make unpacking the bottleneck. BITCOS remained faster on the GPUs, although decoding overhead limited the gains. Computer scientist and AI infrastructure author Chip Huyen has made the same point about inference more generally, arguing that the right optimization depends on whether compute, memory or bandwidth is holding back the workload.
On the eight-core Lunar Lake CPU, Intel’s fixed 2-bit kernel beat BITCOS on every model because the system had enough bandwidth to make unpacking the bottleneck.
The paper has not been peer-reviewed; all five test systems used Intel hardware, and the end-to-end benchmarks covered seven models at batch size one. Intel has yet to test the format on Nvidia, AMD, or Arm hardware.
The post Intel squeezed a 1.58-bit LLM down to 1.485 bits without changing a single weight appeared first on The New Stack.
Running an LLM on your PC is easy enough, but putting an agent to work there is a different story. Portable Computer, the local version of Perplexity’s Computer agent, is now available inside the Perplexity app for Windows on compatible Nvidia GeForce RTX and RTX PRO GPUs.
That’s the good news; the catch is, you’ll need an Nvidia GPU with at least 24GB of VRAM.
The Windows launch gives Perplexity three platforms in less than three weeks. Portable Computer debuted on Linux and Nvidia DGX Spark on August 25, followed a week later by hybrid compute for Apple silicon, which splits tasks between local and cloud models on Macs. Now Windows joins the mix, but bringing Portable Computer over took more than simply porting the app. Perplexity had to adapt the model runtime, orchestration, security, and hardware integration for each platform while keeping the user experience the same.
you’ll need an Nvidia GPU with at least 24GB of VRAM to use it.
Portable Computer bundles those pieces together. On Windows, it supports PPLX 27B — Perplexity’s post-trained model — and Qwen 3.8 27B, both optimized for RTX GPUs, alongside a built-in browser, tool calling, and Perplexity’s proprietary SPACE sandbox.
It’s a different lane from LM Studio or Ollama, which make running models locally as painless as possible but stop well short of giving a model autonomy over multistep work. DeepSeek’s recent hiring spree of roughly 150 new roles, nearly all of them focused on agent infrastructure rather than the model, hints at how much engineering sits between a capable model and a capable agent.
DeepSeek’s recent hiring spree of roughly 150 new roles, nearly all of them focused on agent infrastructure rather than the model, hints at how much engineering sits between a capable model and a capable agent.
Perplexity ships connectors for Microsoft Outlook, OneDrive, and Word, plus Google Drive, Gmail, Slack, and GitHub — which tells you something about what “local” actually means here.
The agent can reach external services because it’s not air-gapped. Locally completed tasks can process files without sending documents to a cloud model. Once an agent has access to both local files and remote APIs on the same machine, figuring out which resources it actually needs and where to find them gets harder.
Perplexity isn’t pretending that a 27-billion-parameter model running on a desktop GPU can handle everything, which explains the hybrid architecture. When the agent determines that a task needs more reasoning power than the local model can deliver, it can escalate to Perplexity’s cloud models.
According to Nvidia, the agent identifies when cloud support would help and asks the user for permission before sending any data off the machine.
For organizations handling sensitive or regulated data, that split can make all the difference. A local agent can grind through source code or financial records without uploading them to a hosted model for basic processing. There’s a cost angle too, since tasks completed locally don’t burn Perplexity Computer credits.
Portable Computer is available with Perplexity Pro ($20/month) and Max ($200/month), across individual and enterprise plans, with Nvidia DGX Station support coming later. The real challenge is taking local agents from developer passion projects to enterprise-ready tools. By baking this into Windows, it immediately gets in front of the scale of users needed to make that happen.
The real challenge is taking local agents from developer passion projects to enterprise-ready tools.
The post Perplexity’s new agent runs entirely on your GPU — with one expensive catch appeared first on The New Stack.
Perplexity wants its AI agents to use more of the computing power already sitting inside a Mac. The company launched Hybrid Compute on Tuesday, a new feature that lets Perplexity Computer move parts of the same task between powerful cloud models and smaller models running directly on Apple silicon.
The timing is significant. Hybrid Compute arrives on the first day of John Ternus’ tenure as Apple’s CEO. Ternus, who previously led Apple’s hardware engineering organization and played a key role in the company’s transition from Intel processors to its own Apple silicon, succeeds Tim Cook after 15 years as CEO. Perplexity is now betting that same hardware can become part of the infrastructure behind autonomous AI agents.
The task starts in the cloud. When a step involves sensitive information, Computer can move that part of the job to a model running on the Mac without having to start over.
When a step involves sensitive information, Computer can move that part of the job to a model running on the Mac without starting over.
A Perplexity-trained Privacy Gate runs locally on the Mac and looks for sensitive information such as names, addresses, account numbers, and secrets. When it flags something, the user can decide whether that part of the task should stay on the device.
Users can review what the system wants to keep local before work begins and ensure it hasn’t missed anything they don’t want sent to the cloud. They can also choose which model handles the local work.
At launch, users can choose between Gemma E4B, Qwen3.6 35B-A3B, and a version of Qwen3.6 35B that Perplexity post-trained itself, with more models planned for later. Perplexity also handles the installation through the desktop app, so users don’t have to open a terminal or set up the model themselves.
Once a task is running, the app displays local CPU, GPU, and memory utilization, as well as the number of tokens consumed. Users aren’t charged for tokens generated by models running locally on their Mac. That gives Perplexity another reason to move work onto the device beyond privacy, since running models locally can also cut inference costs. (The question of who bears inference costs and when is becoming a competitive issue across the industry.)
There is a compromise. Perplexity uses its most capable models in the cloud, while smaller models handle local work, so keeping more of a task on the Mac can improve privacy and cut costs at the expense of some capability. Hybrid Compute leaves that choice to the user.
Perplexity uses its most capable models in the cloud, while smaller models handle local work, so keeping more of a task on the Mac can improve privacy and cut costs at the expense of some capability.
The trickier bit is what happens to context when part of a task moves onto the Mac. The cloud model still needs to know enough about what happened locally to continue the job, without getting access to the private information that was supposed to stay there.
Perplexity says Computer can move a step from the cloud to a local model without restarting the task or losing context, and ultimately combine the cloud and local work into a single result. It does not detail in its announcement exactly what context is passed between those environments or how information produced by the local subagent is filtered before it returns to the broader workflow.
The local subagent can work with private files and data and take actions on the Mac. Users can also start a task on an iPhone and hand off local work to their Mac without having to start over.
For enterprise customers, Perplexity adds company-wide rules for what stays local and a record of what leaves each device, bringing the same governance questions facing AI agents down to the device level.
Hybrid Compute reverses the approach Perplexity introduced for Nvidia’s DGX Spark last week. DGX Spark starts locally and reaches out to frontier cloud models only with permission, while the Mac version starts in the cloud and moves work onto the device when needed.
In both cases, the agent harness decides where each part of the job runs. That orchestration — deciding which tools and context an agent actually needs — is becoming an increasingly difficult engineering problem as agents gain access to more systems and data.
Running this much of an agent locally still requires a fairly powerful Mac. Perplexity recommends at least 32GB of unified memory, Apple silicon, and macOS 15. The feature is available to Pro and Max subscribers as well as enterprise customers.
Those requirements show the limits of local AI at the moment. Smaller models can run on plenty of Macs, but giving an agent enough compute to handle meaningful work still requires relatively high-end hardware.
That will likely change as Macs get better at running larger models. For Ternus, the bigger question is how much of the AI work now happening in the cloud will eventually move onto the machines Apple sells.
For Ternus, the bigger question is how much of the AI work now happening in the cloud will eventually move onto the machines Apple sells.
The post Your Mac is now part of Perplexity’s AI infrastructure appeared first on The New Stack.
SpaceX and Nvidia say they are adapting the Vera Rubin NVL72 rack-scale AI platform for orbital use, with SpaceX targeting a first launch in the fourth quarter of 2027.
The dream of an AI data center in space lives on in SpaceX and Nvidia’s August 24 announcements that the platform for Low Earth Orbit (LEO) Starmind AI satellites will be based on the Vera Rubin NVL72 chip family and architecture.
This proposed system would form the computing core of SpaceXAI’s first-generation Starmind AI satellite and extend Nvidia’s architecture from terrestrial AI data centers into space.
SpaceX CEO Elon Musk posted on X the same day, “SpaceX, in partnership with Nvidia, has designed a space-optimized Vera Rubin NVL72 system for launch to orbit in Q4 next year, with significant scale in 2028.”
SpaceX, in partnership with Nvidia, has designed a space-optimized Vera Rubin NVL72 system for launch to orbit in Q4 next year, with significant scale in 2028 https://t.co/qdDq8YBkzl
— Elon Musk (@elonmusk) August 24, 2026
Musk’s post came after he said during SpaceX’s Q2 earnings call, “Going forward, we’ve decided to build exclusively on Nvidia because we think the Vera Rubin architecture is the best architecture.” Musk continued, “This is not some sort of far-future, distant thing; we expect to start launching these next year. We think the design of the NVL72 VR computer is a much better design than, say, having a standard rack -style design. So we expect to deploy this on the ground as well as in orbit, because we think it’s going to be a radical simplification of the standard NVL72 rack. It will cost less. It will be more effective. If we’re going to put it in space, why not want to put it on the ground? I think that’s going to be pretty cool.”
On Earth, the Vera Rubin NVL72 is Nvidia’s rack-scale AI design that combines 72 Rubin GPUs and 36 Vera CPUs, alongside high-speed networking components such as ConnectX-9 SuperNICs. Nvidia says SpaceXAI’s planned Starmind satellite will be based on an optimized version of that system.
A conventional NVL72 rack assumes gravity, technicians, stable grid power, a building-scale liquid loop and frequent replacement of failed parts. Orbit removes each of these assumptions.
The idea is more ambitious than putting a conventional edge-AI accelerator aboard a spacecraft. Nvidia and SpaceXAI are proposing to bring a modified architecture used in AI data centers into orbit, while altering it for orbital operational requirements.
Getting that working in orbit, though, is easier said than done.
As Curtis Pyke, founder of Kingy AI, writes, “A conventional NVL72 rack assumes gravity, technicians, stable grid power, a building-scale liquid loop and frequent replacement of failed parts. Orbit removes each of these assumptions.“
“Cooling is unforgiving. Space is cold, but vacuum does not carry heat away through convection.”
In particular, Pyke continues, “Cooling is unforgiving. Space is cold, but vacuum does not carry heat away through convection. Heat must travel from the chips to the radiator surfaces and then leave as infrared radiation. SpaceX says AI1 can avoid chillers, cooling towers and fans and reduce cooling overhead by an order of magnitude.”
SpaceX explains that AI1 would instead use closed-loop liquid cooling inside the spacecraft and large deployable radiators to send heat directly to space as infrared radiation. While the claimed reduction is physically plausible in principle, there’s no proof yet that these AI satellites’ cooling systems can deliver.
Another major problem that remains unaddressed is how to make the orbital rack radiation-tolerant. Making Vera Rubin NVL72 radiation-tolerant means far more than putting an ordinary NVL72 rack in a shielded satellite enclosure. It would require a system-level redesign of its GPUs, CPUs, memory, networking, power, cooling, firmware, and operations around a specified orbit and mission life.
LEO orbit is not benign. NASA cites typical trapped-particle dose rates of 100 to 1,000 rad(Si) per year for low-inclination LEO spacecraft below 500 km. That level of radiation is not an immediate death sentence for electronics, but over a multiyear mission it will cause cumulative degradation. Radiation-qualified space hardware can deal with that. Commercial Off-The-Shelf (COTS) electronics are another matter. A true radiation-hardened Rubin GPU would also require design changes at the transistor and circuit levels.
Even were Nvidia to make such a chip, for a high-density AI system such as the SpaceX design, the concern isn’t simply whether one processor survives a 5- or 10-year dose. The satellite contains numerous radiation-sensitive elements, such as GPU logic, SRAM caches, register files, system memory, and memory controllers. With thousands of cores and billions of memory storage cells, the aggregate fault rate — not the behavior of an individual component — drives the design.
The most realistic near-term answer would be a radiation-tolerant, fault-managed Rubin-derived orbital system, not a fully radiation-hardened NVL72 in the traditional military-space sense. It could use selected commercial Nvidia parts, substantial shielding, ECC and data integrity mechanisms, redundant controllers and power paths, aggressive fault detection, software recovery, and reduced-performance operating modes.
The post SpaceX designed an orbital Vera Rubin. Radiation comes next. appeared first on The New Stack.
Nvidia has reportedly agreed to buy Hugging Face for $12.9 billion, putting one of the biggest names in AI hardware in charge of a platform developers rely on to find and run open models.
The Information first reported the deal Wednesday, citing a person familiar with the agreement. Nvidia and Hugging Face had not publicly confirmed it as of publication.
Hugging Face doesn’t push developers toward one chipmaker, which is what makes the acquisition interesting. Its Optimum libraries work with Nvidia’s TensorRT-LLM and also support hardware from AMD, Intel, and AWS. Projects such as Optimum AMD and Optimum Intel let developers run Transformers and Diffusers models on non-Nvidia hardware.
The company also plays a role in what happens after a developer chooses a model, including how easily they can get it running on the hardware they want to use. The problem for Nvidia might be this: It is buying a platform whose value depends on openness and hardware neutrality, but if the purchase means Nvidia hardware is favored, that value might diminish.
The company also plays a role in what happens after a developer chooses a model, including how easily they can get it running on the hardware they want to use.
Hugging Face has expanded well beyond file hosting. With Inference Endpoints, developers can deploy a model from the Hub while Hugging Face handles the underlying infrastructure.
Those hosted deployments can run on AWS, Microsoft Azure or Google Cloud, but most of the GPU options Hugging Face lists are Nvidia chips, including the T4, L4 and A100. That gives developers a wider choice of hardware through Hugging Face’s open-source libraries than through its hosted services.
If the deal goes through, Nvidia would own both sides of that experience.
NIM (Nvidia Inference Microservices) already works with models hosted on Hugging Face. Developers can point NIM to an hf:// repository path and pull the model directly from the Hub.
Owning Hugging Face would give Nvidia more room to bring NIM and CUDA-optimized containers directly into the deployment experience. Nvidia hasn’t announced plans to make NIM the default, and support for AMD and Intel could remain exactly where it is.
Owning Hugging Face would give Nvidia more room to bring NIM and CUDA-optimized containers directly into the deployment experience.
The bigger question is what happens over time. Nvidia could provide earlier support for new models on its own hardware or make deployment easier. At the same time, AMD, Intel, and AWS may have to reconsider how much engineering work they want to contribute to integrations maintained within a competitor-owned platform. Some of that work could eventually move elsewhere.
For developers, the difference may come down to which path requires less work. A competing chip doesn’t have to disappear from Hugging Face to become less appealing if an Nvidia model deployment takes fewer steps. We’ve seen a similar fight over the layers between AI models and the developers using them, with Cloudflare building more of that infrastructure itself.
That tension also helps explain why Hugging Face could be worth considerably more to Nvidia than its revenue alone would suggest.
Nvidia has been expanding its own Nemotron family of open models while investing heavily across the AI ecosystem. At the same time, some of its biggest customers are working to reduce their dependence on Nvidia hardware.
Google has its TPUs, AWS has Trainium, and Microsoft has Maia. OpenAI and Anthropic are also developing their own AI server chips. The push extends beyond the hyperscalers. Earlier this month, five European companies committed to purchasing AI compute built around non-Nvidia hardware that hasn’t been manufactured yet — a sign that the appetite for alternative accelerators is strong enough to attract forward contracts.
OpenAI this week published results from its new Jalapeño accelerator, which showed 1.5 to 1.9 times more work per watt while cutting end-to-end latency by up to 3.6 times on large open-weight models — although the chip has not yet been deployed at anything approaching Nvidia’s scale. A strong open-model ecosystem gives Nvidia a counterweight to that trend.
Open models are often expected to run in very different environments, and Hugging Face helps developers make that possible. A model found on the Hub might end up running on Nvidia hardware, an AMD GPU, or a cloud accelerator.
That flexibility is part of what Nvidia would be buying. Pushing Hugging Face too heavily toward its own hardware could make the platform less useful to developers who rely on it to work across different systems.
Interest in those models is also growing. Models from companies including DeepSeek, Moonshot AI and Z.ai have narrowed the gap with proprietary systems. At the same time, Hugging Face CEO Clément Delangue told The Information in June that the company had doubled its number of paying subscribers during the first six months of 2026. Delangue later said the company was “close to profitability.”
The Information puts Hugging Face’s annualized revenue at about $150 million. Against a $12.9 billion price tag, that’s a multiple of roughly 86.
So Nvidia would be paying for much more than Hugging Face’s current business. It would be buying a place developers already turn to when they want to work with open models, including models that don’t have to run on Nvidia hardware.
It would be buying a place developers already turn to when they want to work with open models, including models that don’t have to run on Nvidia hardware.
Buying Hugging Face wouldn’t give Nvidia control over everything developers find there. Libraries such as transformers and diffusers are open source, and models on the Hub remain subject to their own licenses. Openly licensed models can still be hosted elsewhere, while the underlying libraries can be forked.
Much harder to recreate is the community Hugging Face has built around them. Developers already know where to look for models and have built workflows around the Hub and its integrations.
The post Nvidia’s $12.9B Hugging Face deal has an open-source problem appeared first on The New Stack.
When OpenAI unveiled Jalapeño, its first custom inference chip, in June, the company made some big promises. The chip, developed with Broadcom, was built from scratch for large language model inference, with OpenAI saying early testing showed substantially better performance per watt than existing accelerators. At the time, though, OpenAI didn’t release the detailed performance results to back that up.
On Tuesday, OpenAI published its first results from working Jalapeño silicon across GPT-OSS 120B, DeepSeek R1 and Kimi K2.5. The results show what OpenAI was aiming for with Jalapeño: higher throughput without the longer response times that can come with it.
“Agents need to complete many steps in sequence, so delays can compound across an entire task.”
An agent may call a model over and over as it works through a task, using tools and deciding what to do next based on the results, which means a delay that barely registers during a single inference can become much more noticeable when it happens repeatedly over the course of a longer task.
“Agents need to complete many steps in sequence, so delays can compound across an entire task,” OpenAI said.
Jalapeño was designed with those delays in mind. Different parts of running a large language model place different demands on the hardware, with the initial prompt requiring more compute and the response generation putting more pressure on memory bandwidth. Every time data has to move between cores and chips, that can add even more waiting.
Jalapeño takes a different approach, cutting down on that waiting without optimizing one part of the process at the expense of another.
“Agents need to complete many steps in sequence so that delays can compound across an entire task,” OpenAI said.
That helps explain some of the choices OpenAI made with Jalapeño. Running a large language model puts different demands on the hardware at different points: processing the initial prompt requires a lot of compute, while generating the response token by token relies more heavily on memory bandwidth. There’s also time lost whenever data has to move between cores and chips, leaving parts of the system waiting for what they need.
The idea is to reduce that waiting without optimizing one part of the process at the expense of another. Model state, including the KV cache used while generating a response, can be kept local, while Jalapeño’s networking allows more of the workload to stay within the same connected system. That means less time spent moving data around as the workload shifts between compute and memory.
OpenAI put Jalapeño through InferenceX, SemiAnalysis’ public benchmark for AI inference, using GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. Across the three models, Jalapeño handled 1.5 to 1.9 times more work per watt while cutting end-to-end latency by 1.7 to 3.6 times. On highly interactive workloads, OpenAI says it was 2.1 to 4.1 times faster than the systems it compared against.
The differences become particularly large when Jalapeño is compared at the previous best time-between-tokens operating point. OpenAI reported between 8.6 and 104.3 times more work per watt, depending on the model.
OpenAI based the power-efficiency comparisons on each accelerator’s published power rating. Jalapeño is rated at 700 watts, although the company says it never drew more than 550 watts during these tests. The bigger point is that OpenAI isn’t trying to improve throughput at the expense of response time, which is often the tradeoff with inference.
Batching more work can make infrastructure more efficient, but it can also mean making an individual user wait longer. OpenAI’s argument with Jalapeño is that an inference system increasingly needs to be good at both — particularly as the company continues cutting the cost of API access while also needing to keep interactive workloads responsive.
The bigger point is that OpenAI isn’t trying to improve throughput at the expense of response time, which is often the tradeoff with inference.
OpenAI used its own models throughout Jalapeño’s development, helping the hardware team move from initial design to tapeout in nine months by exploring implementations and shortening design, measurement, and verification cycles. The work didn’t stop once the chip was built.
The company says AI-generated implementations of selected GPT-OSS attention and mixture-of-experts blocks ran 1.5 to 1.8 times faster than versions written by its own experts. That doesn’t mean the entire model ran that much faster, but it does show what OpenAI is trying to do with Jalapeño: make the chip straightforward enough for AI, not just humans, to program and optimize.
Engineers describe work using local tensors, explicit communication and predictable synchronization, giving AI a way to help determine how that work should be mapped, placed and scheduled across the system. That could make it faster to adapt the chip as new models come along, although OpenAI says each new model family still requires its own kernels and optimizations.
Using Codex with GPT-Astra and earlier OpenAI models, the hardware team brought three open-weight models that weren’t part of Jalapeño’s original production plan to high performance within two months. That fits with OpenAI’s broader plans for Codex, which the company has said is still early in its development, and shows how it could eventually play a role well beyond writing code.
OpenAI plans to start using Jalapeño in its own infrastructure by the end of the year, and it’s already working on the next two generations. The company will continue to use accelerators from Nvidia and other partners, but building its own chips gives OpenAI more control over how the hardware evolves alongside its models.
OpenAI plans to start using Jalapeño in its own infrastructure by the end of the year, and it’s already working on the next two generations.
The post OpenAI’s Jalapeño chip tackles a problem AI agents make worse appeared first on The New Stack.
Login