I replayed correct model decisions through four deliberately broken browser actions, and a simple page check missed half of the wrong clicks, which shows that noticing a fault and recovering from it are different skills.

The post An AI Agent Can Spot a Mistake and Still Not Fix It appeared first on Towards Data Science.

I let an AI agent search again whenever it chose. It found some papers plain search had missed, lost some plain search had found, and gained too little to trust.

The post Does Agentic RAG Actually Retrieve Better Evidence? I Tested It on 298 Questions appeared first on Towards Data Science.

I put wrong answers in front of 200 exam questions and read what a small model’s reasoning said when it fell for them.

The post Same Answer, Different Reason: What a Reasoning Model Says When It Follows a Planted Hint appeared first on Towards Data Science.

Grid puzzles that people solve almost perfectly show where language and vision models still break

The post Spatial Reasoning in AI Models, Taken Apart appeared first on Towards Data Science.

Turn scattered requirements into deployable agents

The post What's in the Box? Distinguishing Acceptable Traces from Unacceptable Ones appeared first on Towards Data Science.

Ten quantization guides, and not one threshold-dependent metric between them — what that gap hides, and the two numbers that close it.

The post The Number Your Quantization Gate Doesn't Have appeared first on Towards Data Science.

The same three workloads show a much stronger JIT, but it still doesn't help every program

The post I Reran My Python 3.14 JIT Benchmarks on Python 3.15 appeared first on Towards Data Science.

A hands-on Jev repo comparison of routing accuracy, latency, and output reliability

The post Jev Your Engines: Seven Open Decision Models vs. a Generative LLM appeared first on Towards Data Science.

Pasting in the whole chat history cost 20 to 31 times more than looking up notes, and it did no better on the tasks.

The post I Tested Four Ways to Give an AI Assistant a Memory. The Obvious One Was the Worst Value. appeared first on Towards Data Science.

Real repositories, seeded agent decisions: what 50,400 runs show about wasted turns, patch churn, and which harness rules help

The post Coding Agents Can Solve the Task — But How Much Work Did They Waste? appeared first on Towards Data Science.

Three vision AI models ignored the pretty image, yet GPT-4o invented answers and Claude kept answering B.

The post I Built a Trap for AI Image Judges. They Walked Into a Different One. appeared first on Towards Data Science.

Mapping structural variance against execution outputs to estimate coding agent reliability without ground truth.

The post The Consistency Quadrant: A Visual Guide to LLM Reliability appeared first on Towards Data Science.

Towards Spec-Driven Test Automation: Part 3

The post A Google Team Measured Half of My Argument, and Left the Other Half Open appeared first on Towards Data Science.

Benchmarking the impact of fewer, larger files across three SQL workloads

The post I Compacted 1,000 Apache Iceberg Files Into 6. Here’s What Happened to Query Performance. appeared first on Towards Data Science.

Here’s how to solve it exactly.

The post Building Fair Evaluation Sets Is a Combinatorial Problem appeared first on Towards Data Science.

Retrieval is not evidence. How to build AI that proves its own claims.

The post Beyond RAGs: Building Actually Truthful AI Harnesses appeared first on Towards Data Science.

A small adversarial test set that catches the retrieval failures your evaluation set never will

The post Break Your Own RAG Pipeline Before Users Do appeared first on Towards Data Science.

Why the skill-inflation panic is aimed at the wrong thing, and what it costs to make agent knowledge a build artifact instead of a file.

The post From Static to Dynamic Skills: A Different Model for Agent Knowledge appeared first on Towards Data Science.

A real Weave project that regression-tests three OpenAI models against the exact reply format your app depends on.

The post One Capital Letter Was Silently Breaking My AI Support Bot, and It Wasn't in the New Model appeared first on Towards Data Science.

I built a prompt dependency graph that separates everything a component can reach from the smaller set that actually needs targeted evaluation.

The post Changing One Prompt Can Affect 50 Others — I Built a Prompt Dependency Graph to Find What Needs Retesting appeared first on Towards Data Science.