The eval-writing skill

skill-lens init gives you the structure of a suite. It cannot tell you which cases this skill needs, or what a red case means. That judgment ships as an Agent Skill.

What it does

Given a skill to evaluate, it reads the SKILL.md for its claims, proposes a case list and confirms it with you, scaffolds and fills in the suite, runs it, and triages the failures — distinguishing an eval that is wrong from a skill that is wrong, and proposing changes to your SKILL.md rather than making them silently. It also audits suites you already have: missing negative controls, cases that assert nothing, rubric entries no evidence could support.

Installing it

The skill lives in skills/writing-skill-evals/. Copy or symlink it into the skills directory your agent reads:

git clone https://github.com/EmadMokhtar/skill-evaluator
ln -s "$PWD/skill-evaluator/skills/writing-skill-evals" ~/.claude/skills/writing-skill-evals

Then ask for it by name, or describe the task — "write evals for my order-support skill".

Using it

It expects skill-lens on PATH:

uv tool install "skill-lens[pydantic-ai]"

A checkout works as well — uv sync there, and uv run skill-lens then works from inside it, as does running it in a project that already depends on it. Everything the skill writes is an ordinary eval file: nothing about the suite depends on the skill afterwards.