Test AI Systems
For engineers putting an AI feature into production who cannot yet tell whether it works.
Traditional testing assumes the same input gives the same output. AI systems break that assumption, so the instruments change: eval sets instead of assertions, sampled judgement instead of pass or fail, and monitoring that watches quality drift rather than uptime. The discipline is the same. The tools are not.
Start here
Build Your First Eval Set — the closest thing to a unit test for a model
LLM-as-Judge for Subjective Criteria — grading output no assertion can check
Continuous Evaluation in Production — keeping quality visible after launch
Then go deeper
AI Evaluation & Monitoring runs ten parts, from a first eval set through to continuous production evaluation. Building AI Agents runs ten more, from a single-tool agent through to guardrails and safety.