Evals and Observability · Agent evals

Benchmarks and Their Limits

Video + articles

Video

Big Techday 26: LLM benchmarks in the time of agents - Florian Brand, Prime Intellect

TNG Technology Consulting GmbH · 50 min · June 3, 2026

Articles

Eval awareness in Claude Opus 4.6's BrowseComp performance

Anthropic Engineering · Anthropic · March 6, 2026

Quantifying infrastructure noise in agentic coding evals

Anthropic Engineering · Anthropic · February 5, 2026

Why SWE-bench Verified no longer measures frontier coding capabilities

OpenAI

Supplementary

Evals Are Broken, Use Them Anyway — Ara Khan, Cline

AI Engineer · 19 min · June 6, 2026

Benchmarks: The Good, the Bad, and the Ugly — Ali Khial, G2i

AI Engineer · 13 min · July 31, 2026

DeepSWE: A Contamination-Resistant Coding Benchmark — James Shi, Datacurve

AI Engineer · 18 min · July 26, 2026