Evals and Observability · Building evals

Error Analysis and Failure Taxonomies

Video + articles

Video

How to Automate AI Evals (Correctly)

Hamel Husain · 27 min · July 3, 2026

Articles

A Field Guide to Rapidly Improving AI Products

hamel.dev · Hamel Husain · March 24, 2025

Evals Skills for Coding Agents

hamel.dev · Hamel Husain · March 3, 2026

Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences

arXiv · Shreya Shankar, J.D. Zamfirescu-Pereira, Björn Hartmann, Aditya G. Parameswaran, Ian Arawjo · April 18, 2024

Supplementary

Paired Error Analysis With AI Agents

Hamel Husain · 26 min · April 27, 2026

Improving Agents is a Data Mining Problem — Vivek Trivedy, LangChain

AI Engineer · 20 min · August 12, 2026