01Command Line0/8

Shell, Quoting, and Expansion
Redirection, Pipes, and Exit Codes
Processes and Job Control
Environment Variables and PATH
Package Managers and Toolchains
Search and Text Processing
Git Objects, Branches, and Diffs
Git Worktrees, Reflog, and Recovery

02Model Behavior0/7

Tokens and Tokenization
Context Windows
Sampling Parameters
Reasoning and Test-Time Compute
Why Models Hallucinate
Prompting, RAG, and Fine-Tuning
Model Selection and Cost

03How Claude Code Works0/11

The Agentic Loop
The System Prompt and What Loads
The Built-In Tools
Context Management and Compaction
The Permission Model
Auto Mode and the Classifier
Settings and Precedence
CLAUDE.md, Rules, and Memory
Sessions, Checkpoints, and Worktrees
Headless Mode and the CLI
Cost and Prompt Caching

04Customization0/8

Agent Skills
Writing a Skill
Slash Commands
Hooks
Writing a Hook
Subagents
Plugins and Marketplaces
Choosing Between Skill, Subagent, Command, and MCP

What MCP Is
Tools, Resources, and Prompts
Transports and the Stateless Model
MCP in Claude Code
The API MCP Connector
Building a Server
Tool Design for Agents
When Not to Use MCP

06Building on the API0/9

The Messages API
Tool Use
Writing the Agentic Loop
Server Tools
Structured Outputs
Extended Thinking and Effort
Prompt Caching
Message Batches
The Claude Agent SDK

07Context and Retrieval0/9

Context Engineering
Compaction, Notes, and External Memory
Embeddings
Chunking
Hybrid Search
Reranking
Contextual Retrieval
Agentic Search and Long Context
Evaluating Retrieval

08Orchestration0/7

Orchestration Topologies
Orchestrator and Workers
When Multi-Agent Helps and When It Hurts
Passing State Between Agents
Harness Design
Long-Horizon Agents
Multi-Agent Failure Modes

09Evals and Observability0/8

Eval-Driven Development
Error Analysis and Failure Taxonomies
Graders and Eval Types
LLM-as-Judge
Statistics for Evals
Evaluating Agents
Benchmarks and Their Limits
Tracing and Observability

The Agent Threat Model
Prompt Injection
The Lethal Trifecta
Exfiltration Channels
Tool Permissioning and Least Privilege
Human Approval and Its Limits
Sandboxing
Supply Chain Risk in Skills, Plugins, and MCP Servers
Vetting a Third-Party Skill or Server
Secrets and Agent Identity

11Deployment and Governance0/7

Enterprise Administration
Managed Settings and Policy
Gateways and Cloud Routing
Cost Controls and Usage Analytics
Data Retention and Zero Data Retention
Compliance Posture
AI Governance Frameworks

Adoption Program Models
Enablement and Internal Libraries
Measuring Adoption and Impact
The Productivity Evidence
Change Management for Knowledge Workers
Teaching Methods That Work

Extended lessons

Curriculum/Evals and Observability/Statistics for Evals

Evals and Observability · Judging

Statistics for Evals

Video + articles

Video

Dr. Sida Wang: Measuring all the noises of agentic LLM Evals

AI Agent Frontier · 1 hr · March 23, 2026

Articles

Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations

arXiv · Evan Miller (Anthropic) · November 1, 2024

Stats for LLM Evals — Resources

statsforevals.com

Measuring all the noises of LLM Evals

arXiv · Sida Wang · December 24, 2025

Supplementary

Computer Use at the Edge of the Statistical Precipice — Pierluca D'Oro, Programma Labs

AI Engineer · 17 min · August 14, 2026

Previous

LLM-as-Judge

Evals and Observability · AI Engineer · 41 min

Up next

Evaluating Agents

Evals and Observability · AI Engineer · 20 min