https://jamesm.blog/tags/eval/ Recent content in Eval on jamesm.blog https://jamesm.blog/papermod-cover.png https://jamesm.blog/papermod-cover.png Hugo en Sun, 14 Jun 2026 09:00:00 +0100 https://jamesm.blog/ai/evaluating-agents-in-production-trajectory-metrics/ Sun, 14 Jun 2026 09:00:00 +0100 https://jamesm.blog/ai/evaluating-agents-in-production-trajectory-metrics/ Endpoint scoring certifies answers, not behaviour. A practitioner guide to step-level agent evaluation - trajectory rubrics, replay harnesses, and regression suites for multi-step production agents. https://jamesm.blog/ai/ai-evals-are-broken/ Fri, 12 Jun 2026 19:00:00 +0100 https://jamesm.blog/ai/ai-evals-are-broken/ A grounded look at why the public benchmark numbers you see on AI model launches in 2026 are increasingly disconnected from real-world capability - saturation, contamination, gaming, and what's replacing them as the basis for actual decisions.