METR will publish its own assessments of Anthropic’s risks, incidents, and practices, and the frontier AI model maker has given up the right to block them.

A hands-on Jev repo comparison of routing accuracy, latency, and output reliability

The post Jev Your Engines: Seven Open Decision Models vs. a Generative LLM appeared first on Towards Data Science.

Pasting in the whole chat history cost 20 to 31 times more than looking up notes, and it did no better on the tasks.

The post I Tested Four Ways to Give an AI Assistant a Memory. The Obvious One Was the Worst Value. appeared first on Towards Data Science.

Real repositories, seeded agent decisions: what 50,400 runs show about wasted turns, patch churn, and which harness rules help

The post Coding Agents Can Solve the Task — But How Much Work Did They Waste? appeared first on Towards Data Science.

Three vision AI models ignored the pretty image, yet GPT-4o invented answers and Claude kept answering B.

The post I Built a Trap for AI Image Judges. They Walked Into a Different One. appeared first on Towards Data Science.

Mapping structural variance against execution outputs to estimate coding agent reliability without ground truth.

The post The Consistency Quadrant: A Visual Guide to LLM Reliability appeared first on Towards Data Science.

Towards Spec-Driven Test Automation: Part 3

The post A Google Team Measured Half of My Argument, and Left the Other Half Open appeared first on Towards Data Science.

Benchmarking the impact of fewer, larger files across three SQL workloads

The post I Compacted 1,000 Apache Iceberg Files Into 6. Here’s What Happened to Query Performance. appeared first on Towards Data Science.

Here’s how to solve it exactly.

The post Building Fair Evaluation Sets Is a Combinatorial Problem appeared first on Towards Data Science.

Retrieval is not evidence. How to build AI that proves its own claims.

The post Beyond RAGs: Building Actually Truthful AI Harnesses appeared first on Towards Data Science.

A small adversarial test set that catches the retrieval failures your evaluation set never will

The post Break Your Own RAG Pipeline Before Users Do appeared first on Towards Data Science.

Why the skill-inflation panic is aimed at the wrong thing, and what it costs to make agent knowledge a build artifact instead of a file.

The post From Static to Dynamic Skills: A Different Model for Agent Knowledge appeared first on Towards Data Science.

A real Weave project that regression-tests three OpenAI models against the exact reply format your app depends on.

The post One Capital Letter Was Silently Breaking My AI Support Bot, and It Wasn't in the New Model appeared first on Towards Data Science.