❌

Normal view

Speculative Decoding on CPUs: Nearly 4x Faster Token Generation with DFlash

24 August 2026 at 13:30

Speculative decoding can turn underused CPU compute into faster token generation, without changing the model's output. In our vLLM tests, DFlash delivered 3.92x the autoregressive throughput with Qwen3.5-9B on Intel Xeon 6 at concurrency 1. We break down where the speedup comes from, explain the acceptance metrics, and show what determines whether speculation pays off.

The post Speculative Decoding on CPUs: Nearly 4x Faster Token Generation with DFlash appeared first on Towards Data Science.

AI Agents Don’t Need More Context β€” They Need Typed Context

24 August 2026 at 12:00

AI agents don’t just have a context problemβ€”they have a context typing problem. When instructions, memory, retrieved evidence, and tool outputs are flattened into one string, their semantic boundaries can disappear. I built a lightweight, zero-dependency Python runtime that keeps those boundaries explicit, tracks provenance, and rejects invalid context transformations before they reach the model. This article walks through the implementation, tests, and what this approach doesβ€”and does notβ€”guarantee.

The post AI Agents Don’t Need More Context β€” They Need Typed Context appeared first on Towards Data Science.

Webwright: Why AI Web Agents Should Write Code, Not Click

17 August 2026 at 16:30

For years, web agents have worked one click at a timeβ€”and often fallen apart on long tasks. Microsoft Research’s Webwright makes a different bet: give the model a terminal and let it write the program instead. On long-horizon tasks, the same GPT-5.4 model jumps from 33.5% to 60.1% success. And instead of leaving behind a click trace, it leaves something you can actually use again: a command-line tool.

The post Webwright: Why AI Web Agents Should Write Code, Not Click appeared first on Towards Data Science.

Can a Local LLM Run My AI Assistant?

11 August 2026 at 12:00

I replayed the same 27 real production tasks through two local models, one hardware upgrade apart, to find out what it actually takes to replace Claude as the brain behind a 90-tool personal agent.

The post Can a Local LLM Run My AI Assistant? appeared first on Towards Data Science.

❌