IBM has made Spyre a first-class PyTorch backend, giving developers the same workflows across devices.

Misumi Americas has launched Misumi Ventures, a $50 million early-stage investment fund targeting robotics, automation, advanced hardware, physical AI and industrial software companies. The San Francisco Bay Area-based fund plans to build a portfolio of 20 to 30 seed and early-stage companies, typically investing between $500,000 and $1.5 million initially and reserving capital for follow-on […]

We compiled a list of all the hottest AI wearables, including Muse Charm, Bee, Friend, Plaud, and more.

At PyTorch Conference North America 2026 this week, Google open-sourced TorchTPU, a native PyTorch stack for Google TPUs, now available on GitHub. AWS also launched TorchNeuron in public beta, its open-source native PyTorch backend for AWS Trainium, also available on GitHub.

Both releases seem aimed at making it easier for developers to move beyond Nvidia GPUs, whose CUDA software ecosystem has long dominated AI development. But while Google and AWS’s new offerings do ease some of the challenges of switching hardware, getting top-tier performance still isn’t straightforward. 

For developers eager for more computing options, it’s promising but somewhat disappointing.

Google wants to make TPUs more PyTorch-friendly

Google put TorchTPU in preview with select customers in April (PyTorch says Google’s been working on the project since mid-2025), teasing a simpler way to run PyTorch workloads on Google’s AI chips. PyTorch support has long been an awkward dance, so the new backend could make it easier to run existing models on TPUs.

Google says it wants to make its TPU software ecosystem more open and developer-friendly for the PyTorch community, as it works to make the chips a more practical option for developers already using PyTorch. 

Developed in collaboration with the PyTorch team at Meta, TorchTPU allows developers to run existing PyTorch workloads without JAX dependencies in the runtime. This way, rather than requiring extensive code changes, developers can migrate existing workloads with only “trivial code changes,” according to PyTorch. 

Specifically, developers can use eager execution to debug models and torch.compile to optimize TPU utilization and reduce memory overhead. 

The new PyTorch stack from Google and Meta comes alongside a slew of other products and integrations announced at the conference, which Google says are “designed to help you run PyTorch effectively on Google Cloud or in your own infrastructure.” Google also announced integrations involving Hugging Face, vLLM, SGLang, llm-d, TorchTitan, Ray and TorchRec, while contributing LiteRT to the PyTorch ecosystem. 

AWS wants in, too

AWS’s release extends the same PyTorch-native approach to its own AI chips.

Like TorchTPU, TorchNeuron gives developers a way to natively experiment and build models with PyTorch — this time for Trainium, AWS’s portfolio of AI chips designed for deep learning training and inference workloads. The public beta supports Trainium2 and Trainium3 instances. 

AWS says moving a model to Trainium can be as simple as switching to .to(‘neuron’) for eager execution. Compiled models also need the Neuron backend selected, either as the default or in a torch.compile call.

TorchNeuron also integrates with Neuron Kernel Interface (NKI) — a bare-metal language and compiler for programming AWS Trainium and Inferentia NeuronDevices — so developers can create custom operators and optimize performance on Trainium. AWS and PyTorch have published more detailed guidance on the backend’s execution modes and performance tradeoffs.

Both are getting closer, but not there yet

While TorchTPU and TorchNeuron are notable steps forward in opening up more hardware choices for PyTorch developers, neither is a silver bullet.

PyTorch says a simple code change can get models running on TPUs, but peak performance can require TPU-specific optimization.

For distributed workloads, PyTorch says TorchTPU’s default Skinny Collectives approach can incur an end-to-end performance penalty of roughly 30–40% compared with an optimized approach because it prevents the XLA compiler from overlapping computation and communication.

For workloads seeking peak performance, PyTorch recommends additional tuning and production-grade stacks: 

“Of course, for production workloads seeking peak performance we would absolutely recommend pursuing an optimal approach based on Pallas kernels and production-grade stacks such as vLLM-TorchTPU, SGLang-TorchTPU, or TorchTitan-TorchTPU, as examples.”

TorchNeuron is in public beta. In a separate blog post, AWS says teams can try Trainium with a configuration change instead of a code or model rewrite “in most cases,” leaving open which workloads need more changes.

In a case study, AWS describes how its Generative AI Innovation Center worked with AI research and technology company i14 to adapt its Laplace 3B model for Trainium2. The implementation required three changes: 1) using native PyTorch for the GLA attention kernel; 2) redesigning how Muon communicates; 3) compiling the sequential backward pass.

The releases give PyTorch developers more options beyond Nvidia GPUs. For now, moving existing code is easier than getting peak performance on unfamiliar chips. 

The post Google and AWS open new paths for PyTorch beyond Nvidia appeared first on The New Stack.

Blackstone’s AI head on why the PE giant isn't panicked by market jitters.

Google has joined Amazon in releasing a developer tool that makes it easy for PyTorch users to move their apps off Nvidia GPUs.

DeepSeek has open-sourced the inference engine it uses to serve its models in production, a step it said last year wasn’t yet practical.

Dubbed DeepInfer, the framework is designed to let developers run DeepSeek’s V4.1 Flash model on their own infrastructure, with many of the same optimizations the company uses behind its API. Support for future DeepSeek models is also planned.

“Running a model efficiently is a different matter altogether.”

DeepSeek has, of course, long made its model weights available for developers to download and run using third-party software such as vLLM. But running a model efficiently is a different matter altogether. The software that manages requests, distributes work across chips, and generates responses can have a significant impact on speed, throughput and computing costs. And so DeepInfer essentially gives developers access to DeepSeek’s own approach to those challenges.

“We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.”

‘Making AI more affordable’

Taking to X on Tuesday, DeepSeek said it was making its internal inference technology available to outside developers in an effort to bring down the expense of running large AI models.

“We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable,” the company writes.

🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!

Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more…

— DeepSeek (@deepseek_ai) October 20, 2026

Released under the Apache 2.0 license, DeepInfer includes optimized computing kernels and communication libraries, including DeepSeek’s previously open-sourced FlashMLA, DeepGEMM, and DeepEP, alongside mechanisms for distributing inference across multiple chips. It also supports speculative decoding, a technique that speeds up response generation by predicting and verifying multiple tokens at once, and asynchronous scheduling to reduce the time processors spend waiting for new work. The framework also supports structured outputs, allowing developers to specify how model responses should be formatted.

Alongside DeepInfer, the company is also open-sourcing deepseek-recipe, a companion library that handles how API requests are converted into model inputs, and how the resulting output is processed into responses. Available for Rust and Python, it’s designed to help developers reproduce the behavior of DeepSeek’s official API when running their own deployments. The library was quietly released in September, with an updated version published on Tuesday.

The announcement comes just days after DeepSeek released an updated version of V4.1 Flash, called DeepSeek-V4.1-Flash-1017, touting improvements in coding performance and efficiency. That release coincided with the company making its in-house web search technology available to developers through new Search and Fetch APIs, as per The New Stack‘s report on Monday.

Why DeepSeek held back in 2025

DeepInfer goes further than DeepSeek’s previous plans to share its inference technology. Back in April 2025, the company published a document outlining its plans for its inference engine, explaining that it had considered releasing the entire system before deciding to contribute individual components and improvements to existing projects instead.

At the time, DeepSeek identified several obstacles. Its inference software had evolved from an early version of vLLM into a highly customized system, and the company questioned whether its relatively small research team could maintain another major open-source project. There were also practical difficulties in making the software usable outside DeepSeek’s own infrastructure.

“The engine is tightly coupled with our internal infrastructure, including cluster management tools, making it impractical for public deployment without significant modifications,” the company wrote at the time.

So what’s changed now, exactly? Well, DeepSeek hasn’t explained how it overcame those earlier difficulties, although the scope of the new release offers some clues. The DeepInfer documentation makes clear that the publicly available software currently supports offline inference, with “the core features needed for serving at scale,” as the company puts it. However, several components needed to operate a complete production service are still missing. These include a KV cache management system, infrastructure for separating different stages of inference across machines, and load balancing across replicas.

In other words, DeepSeek has opened up much of the engine responsible for running its models, while leaving developers to supply some of the surrounding infrastructure required to operate a full-fledged service. That narrower release may help explain how the company has made its technology available despite the challenges it identified last year.

Nvidia and Huawei, side by side

While the release of DeepSeek’s inference engine is significant enough in its own right, arguably the more interesting tidbit is the hardware supported from day one.

“Arguably the more interesting tidbit is the hardware supported from day one.”

Nvidia is included, as expected, though support is restricted to its SM100 architecture, used in Blackwell data center GPUs. Notably, the other supported platform is Huawei’s Ascend 950 NPU series, putting the Chinese chipmaker alongside Nvidia in DeepSeek’s list of compatible hardware. Older Nvidia architectures, including Hopper, are conspicuously absent.

This also ties back to DeepSeek’s published open-source plans from 2025. While the company was reluctant to release its entire inference engine at the time, it committed to working with hardware manufacturers ahead of future model launches, with the aim of making its models available across different chip architectures from day one.

US export restrictions have complicated Chinese companies’ access to Nvidia’s most advanced chips, adding another dimension to DeepSeek’s growing ties with Huawei. In September, Bloomberg reported that DeepSeek was planning to deploy at least 160,000 Huawei Ascend 950DT chips at a new data center in Inner Mongolia, specifically to run its models. The company has largely relied on Nvidia for training, although The Information reported in September that DeepSeek also plans to increase its use of Huawei chips for training, with deliveries expected as early as the fourth quarter.

Later that month, Reuters reported that DeepSeek and Huawei were collaborating on software tools intended to make Ascend processors easier to program, further underlining the company’s investment in Chinese AI hardware.

The Huawei inclusion has already attracted attention among the wider developer community. In a discussion on Reddit, one commenter singled out the two supported hardware platforms as a highlight of the news, while another pointed to DeepSeek’s origins in quantitative finance as a possible explanation for its technical expertise.

“Once again Deepseek shows how good high-frequency traders are at optimizing performance… On Ascend NPUs nonetheless!!,” the commenter wrote.

Another user seemingly poked fun at Anthropic’s Claude Opus 5.5, following reports that requests involving kernel programming for Huawei’s AI chips could trigger safeguards that redirect certain advanced AI development tasks to an older model.

“Don’t let Opus 5.5 see this! It will delete the Ascend support,” the user joked.

It is worth noting that feature support differs between the Nvidia and Huawei implementations. DeepInfer’s batch-invariant inference and deterministic sampling features, which help ensure reproducible outputs under specified conditions, are available only on Nvidia hardware. DeepSeek’s published performance measurements also use Nvidia GPUs, with four SM100 GPUs used for prefill and 64 for decoding, leaving open the question of how the framework performs on Huawei’s chips by comparison.

For developers without access to either chip family, the immediate value may lie in studying the newly available code and adapting its optimizations to other inference engines. For now, DeepSeek has given the wider open-source community considerably more of its inference technology to experiment with.

The post DeepSeek open-sources the inference engine behind its API. And Huawei gets equal billing with Nvidia. appeared first on The New Stack.

'Dataflow Architectures for AI' is a 6-course specialization from IBM Research that compares leading accelerator architectures through hands-on labs.

Counterfeiters are getting better at replicating brands' marks and passing off dupes, known as "superfakes," as genuine items. A startup called Veritas wants to tackle the problem with its own chip that verifies goods.

Microsoft is positioning the new Surface Laptop Ultra as an alternative to Apple’s MacBook Pros for creatives and developers — and especially AI developers.

Built on Nvidia’s N1X-based RTX Spark platform, with a Blackwell RTX GPU with up to 6,144 cores, a CPU with up to 20 cores, up to 128GB of unified memory, and a price that undercuts Apple, it’s a tempting alternative if you want to run local AI models and also want to have a powerful machine for editing videos and gaming on the go.

This is The New Stack, so I’m leaving the gaming benchmarks and creator use cases to others. My focus in testing the unit was on running AI workloads.

I’ve been using the Surface Laptop Ultra in its high-end $4,300 GPU and CPU configuration with 64GB of RAM and a 1TB SSD for the last week, including while on the road.

Typically, I use a Mac Studio with the M4 Max chip at home, which happily runs most local models I throw at it (as long as they fit into the 48 GB of RAM). I installed all the usual tools I use to run local models on the laptop. Think Unsloth Studio, Ollama, Hermes Agent, etc.

At no point did I feel like I was making a compromise in running those on the laptop. Indeed, if anything, the Blackwell GPU was generally a bit faster than my Mac Studio.

Fast local inference (with the right models)

Since the new Nemotron 3.5 Bolt model launched earlier this week, I also took that one for a spin. It answered queries at an impressive 56 to 60 tokens per second — and sometimes more. That makes it a great model for running in Hermes Agent or OpenCode, for example. It’s a heavily quantized model with 9B active parameters, so you would expect it to be fast, of course.

Yet, while the laptop has the unified memory to run it, the dense Qwen 3.8 27B at 8-bit quantization never achieved more than 8 or 9 tokens per second in Unsloth Studio. That’s not quite usable, but with these dense models, that’s virtually always due to memory bandwidth. But bringing it down to a 4-bit quantization brought the speed up to a respectable 25 to 28 tokens per second.

Using the 8-bit Gemma 4 26B-A4B shows how the mixture-of-experts technique is really what you need to run virtually any model locally at reasonable speeds. For this model, the Surface Laptop Ultra generally hit around 40 tokens per second.

I don’t have an M5-based MacBook or Mac Studio at hand, but Tom’s Hardware reports that the Ultra generally outperforms the M5 Max in a MacBook Pro in AI use cases.

Performance drops when you unplug

All of those previous numbers were with the laptop on its 140W charger. Microsoft says it tuned the system so that running on battery would still give users most of the full performance. Microsoft claims its benchmarks showed that the laptop retained 99.6% of its performance when running on battery, but at least for the AI workloads I tested, that drop was far more noticeable.

The 4-bit Qwen 3.8 model, for example, dropped from its 25-28 tokens per second to around 18 tokens per second on battery. For Nemotron 3.5 Bolt, the performance dropped to around 42 to 45 tokens per second. Overall, it seems fair to expect around a 20% to 25% loss in performance when on battery when running AI workloads (and games show a very similar drop).

I did not run any formal battery life tests. Microsoft says users can expect 12 hours of active web usage. If you are consistently running local AI workloads, though, I wouldn’t expect more than a few hours of battery life based on my observations.

The RAM questions

Having 64GB of RAM puts you in a bit of an odd situation given that most smaller models right now are in the 27B to 30B league. The extra RAM gives you plenty of headroom for the KV cache and other applications, but there aren’t a lot of models in the 32-48GB range, for example, that the machine could also handle.

Going for 128GB of RAM brings the price of the Surface Laptop Ultra to $5,900

The trade-off for all that power: heat and noise

Unsurprisingly, all of the compute power (Microsoft says you get up to 1 petaflop of FP4 AI performance) creates a bit of heat.

Nobody is going to argue that the Surface Laptop Ultra is whisper-quiet when you run AI applications. The noise its dual fans make isn’t extreme, but enough to get a bit on your nerves while you are working. That’s especially noticeable when Hermes or your favorite coding agent is chugging along for a while on a more complex query.

There is a reason the Surface RTX Spark Dev Box, which is built on top of the same N1X platform, has a thousand air vents, after all (which will also likely allow it to push its performance beyond the N1X in the laptop).

Outside of that, the laptop is very quiet, though, and the fans remain off for most day-to-day use cases. It also quiets down very fast once the GPU load drops again.

A few more observations

I would feel remiss if I didn’t highlight a few other things I found while testing the Ultra. This is, after all, Microsoft’s most ambitious laptop yet.

• The touchpad is huge and has haptic feedback. There are only a few applications that support it so far, including DaVinci Resolve, for example. Having that feedback makes editing faster, as you can feel when the playhead snaps into place, for example. I want more touchpads with that feature.
• Whatever Microsoft did with the speakers works wonders. There are side vents near the front of the laptop that create a surprisingly wide stereo field, something you will definitely notice in games.
• The new magnetic USB-C charging connector is nice. It feels a lot like the magnetic Connect on other Surface devices. I just worry I’ll lose the cable sooner rather than later.
• Microsoft is not positioning this as a gaming PC, but with that GPU, people will try to play a few games, and so did I. Gaming on a Windows ARM machine is still in its early days, but thanks to Microsoft’s emulation, it generally works without too much overhead.
  Games that rely more on the GPU than the CPU will fare better, but I was able to play Cyberpunk 2077 in 2K with ray tracing turned on at around 60fps. With frame generation and a few settings tweaks, even 4K gaming is an option.
• With ray tracing off, the remaster of The Witcher 3 also happily ran at 60fps in 4K.
• But since the 15-inch, 3:2 touchscreen (3270×2180) is essentially a better 3K screen, you wouldn’t want to run a game in 4K anyway unless you plug in an external monitor.
• The screen looks great and can get very bright, but it is also highly reflective. That’s often distracting.
• Overall, Windows on ARM feels a lot more solid now than it did a year ago. Surely there are some applications and games that won’t run (or run well), but I haven’t come across any issues. Increasingly, developers (like OpenCode) offer Windows ARM64 versions of the desktop apps.
• Having two USB-C, one USB-A, and an HDMI port is such a joy if you have any older accessories. Add the headphone jack and SD card reader, and you don’t even need to carry a USB-C hub.
• I’ve gotten used to my MacBook Air. The Laptop Ultra is heavy at 4.41lbs (2kg).
• The SSD is replaceable. There are four screws underneath the laptop, and you don’t void your warrenty of you open the machine up. Apple should pay attention.

The post Running local AI models on Microsoft’s Surface Laptop Ultra: What we found appeared first on The New Stack.

Speaking at TechCrunch Disrupt, Amazon devices chief Panos Panay described a future where ambient AI devices work in the background, anticipating users' needs instead of waiting for commands.

Amazon-owned Bee has launched its first new AI devices since its acquisition, including an updated wearable and a credit-card-sized recorder that attaches to smartphones. The devices can summarize conversations, create to-do lists, and integrate with Alexa and Amazon shopping, with prices starting at $74.99.

A federal judge ruled that Apple can access OpenAI's interview materials involving former Apple employees, as well as private messages and testimony from key figures in the case.

Flox's edge devices can identify animals and use species-specific sounds to drive them away from critical infrastructure.

Open models and llm-d enabled IBM Research to explore quantum error correction hardware at 11.5× lower inference cost than an all-proprietary configuration.

One of the core promises of AI is that the more it knows about a person, the better it will be able to assist them with tasks and make appropriate suggestions, which, if you'll believe the tech companies, requires apps and devices to listen, watch, or record users' activity all the time and gather all possible context.

Over the past couple of years, we’ve seen multiple reports that Apple was working on a new smart home product, but now it seems the product has a name and a launch date.

For six years, Danu founder Amy Ma has been working on a better way to sort recyclable waste.

Upside Parts, a Massachusetts-based custom manufacturing company, released an analysis of more than 30,000 3D-printed parts produced across hundreds of customer orders, looking at how robotics companies use on-demand 3D printing. Robotics customers accounted for 82.3 percent of total part volume. The analysis comes as robotics production is expanding across the US, increasing the need […]

Natura’s $99 Interface smart ring lets you summon AI agents with the press of a finger to complete tasks, capture thoughts, and control devices — while doubling as a health tracker.