# Evals Skills for Coding Agents

DevFeed: [Evals Skills for Coding Agents](<https://devfeed.tech/articles/evals-skills-for-coding-agents-18790.md>)

Original publisher: [Read original article](<https://hamel.dev/blog/posts/evals-skills/>)

Author: Shreya Shankar

Published: 2026-03-02T08:00:00Z

Content type: release

Language: en

Sources: [Hamel Husain](<https://devfeed.tech/sources/hamel-husain.md>)

Topics: [AI-assisted coding](<https://devfeed.tech/topics/ai-assisted-coding.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [coding](<https://devfeed.tech/topics/coding.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-evals](<https://devfeed.tech/tags/ai-evals.md>), [audit](<https://devfeed.tech/tags/audit.md>), [coding-agents](<https://devfeed.tech/tags/coding-agents.md>), [evals](<https://devfeed.tech/tags/evals.md>), [review](<https://devfeed.tech/tags/review.md>), [skills](<https://devfeed.tech/tags/skills.md>), [traces](<https://devfeed.tech/tags/traces.md>)

## AI overview

Shreya Shankar and the author publish evals skills for AI product evaluations. The collection routes users to skills for auditing eval pipelines, discovering errors from traces, generating synthetic test inputs, designing and validating LLM judges, and evaluating RAG quality.

## Source excerpt

Today, Shreya Shankar and I are publishing evals skills, a set of skills for AI product evals1. Eval tools often get in the way. They nudge you toward generic off-the-shelf metrics and fully automated evals before you've looked at your data. These skills help you avoid common mistakes we've seen helping 50+ companies and teaching students in our AI Evals course. Why skills for evals There are many easily avoidable footguns in evals. These skills help you avoid them. evals-start is the entry point. It looks at your situation and routes you to the right skill. Most of the time it will send you to one of these two: eval-audit, if you already have an eval pipeline. It inspects your setup and recommends next steps. 2 error-discovery, if you have traces but haven't analyzed them yet. It builds a customized annotation interface and helps you sample traces intelligently. Shreya does a live walkthrough of using this skill here. The skills Install the skills: npx skills add https://github.com/ai-evals-course/evals-skills Then give your agent this prompt: Run the evals-start skill from the evals plugin and follow the skill it picks. If it picks eval-audit, investigate each diagnostic area using a separate subagent in parallel, then synthesize the findings into a single report. If you're experienced with evals, skip the router and pick the skill you need: Skill What it does evals-start Entry point. Routes to the skill that matches your situation eval-audit Audit an eval pipeline and surface problems with prioritized severity error-discovery Build a review app, select diverse samples, and organize your notes into failure modes generate-synthetic-data Create diverse synthetic test inputs using dimension-based tuple generation write-judge-prompt Design LLM-as-Judge evaluators for subjective quality criteria validate-evaluator Calibrate LLM judges against human labels using data splits, TPR/TNR, and bias correction evaluate-rag Evaluate retrieval and generation quality in RAG pipel