Skip to content

Contributing

Development

uv sync
uv run pytest                  # test suite
uv run ruff check .            # lint
uv run ruff format --check .   # formatting (as CI runs it)
uv audit --preview-features audit --locked   # known vulnerabilities in uv.lock
uv run skill-lens list ./examples

.python-version pins the interpreter to 3.13, so uv sync uses the same one CI does — no per-job setup step, and no drift between your machine and the pipeline. The package itself still supports >=3.11.4 (requires-python — the patch matters, because tarfile's filter= keyword, which --baseline previous uses to extract a previous bundle safely, only exists from 3.11.4); CI does not currently exercise that floor.

Tests marked integration hit real provider APIs and are deselected by default; run them with uv run pytest -m integration. Tests marked cassette replay recorded provider traffic — zero cost, no key needed, and selected by default. Everything else passes offline with no API spend. uv run skill-lens run ./examples needs the pydantic-ai runner (see Running against a real agent); the shipped examples now assert real model behavior, so list is what dogfoods discovery for free.

test_a_baseline_run_reaches_the_provider_without_the_skill_name in tests/test_cassettes.py proves the baseline no-name-leak rule survives the wire, but its cassette has not been recorded yet — no API key was available when it was written. A missing cassette skips rather than fails, by design, so a fresh clone is never red for this; whoever has a key next should record it with:

uv run pytest tests/test_cassettes.py --record-mode=once

Development is test-driven: write the failing test first, then the implementation.

Documentation

uv sync --group docs
uv run mkdocs serve            # live preview at http://127.0.0.1:8000
uv run mkdocs build --strict   # as CI runs it
uv run pytest tests/test_docs.py

Documentation ships with the change, not as a follow-up. Two CI jobs enforce it: docs builds the site with --strict and runs tests/test_docs.py with the docs group installed, and docs-freshness fails a PR that changes src/skill_lens/** without touching docs/, README.md, ARCHITECTURE.md or mkdocs.yml. When a change genuinely needs no documentation — a pure refactor, a dependency bump — add the no-docs-needed label to the PR.

tests/test_docs.py is the precise half of the same idea. It asserts that:

  • every command, flag, config key, EvalCase field and assertion kind appears in the docs;
  • every page is in the nav, and linked from README.md's Documentation table or the home page's "Where to go next" table;
  • no relative link is dead, and no #anchor names a heading that does not exist. Renaming a heading means updating every link to it;
  • the install instructions are written once, in docs/snippets/install.md, which pages include with --8<-- "docs/snippets/install.md". README.md cannot include a file, so its copy of the command is compared with the snippet's;
  • ARCHITECTURE.md and CLAUDE.md state the three protocols as src/ defines them.

Heading ids are worked out the way Python-Markdown does it. With the docs group installed, the same file also checks that rule against the real renderer; that is why the docs job runs it.

tests/test_troubleshooting.py does the same for Troubleshooting. Every message quoted there is listed in its QUOTED table with the module that emits it, and the test fails if either side changes. A new message on that page needs a row there.

Conventional Commits are required

Every commit message and every pull request title must follow Conventional Commits:

<type>[optional scope][!]: <description>

Types: feat, fix, docs, refactor, test, perf, build, ci, chore, style, revert. Use the imperative mood, lowercase, no trailing period.

This is not stylistic. cz bump derives the next version and the changelog from commit history, so a non-conforming message silently breaks the release. Because PRs are squash-merged, the PR title becomes the commit on main — so the title is what release automation actually reads.

Install the hooks once per clone so bad messages are rejected before they land, and the dependency audit runs before every push:

uv run pre-commit install --hook-type commit-msg --hook-type pre-push

The audit hook runs at push time rather than commit time because it needs the network; see Security for what it checks and what to do when it finds something.

CI enforces the same rules on every PR: the title is checked with cz check, and every commit on the branch with scripts/check_commits.py. Two docs-only commits predating the convention are exempted in scripts/legacy-commits.txt; that list should only ever shrink.

One-time repository settings

These settings live in the GitHub UI, not in this repo, so they are easy to miss when standing up a fork or a new instance of this project:

  • Settings -> Code security -> Private vulnerability reporting = enabled. SECURITY.md and the Security page send reporters to the private advisory form; with the setting off, that form does not exist and the only remaining path is a public issue.

  • Settings -> Code security -> Dependabot alerts and Dependabot security updates = enabled. .github/dependabot.yml turns on version updates by itself; alert-driven security updates — a pull request the day an advisory is published, rather than on the weekly schedule — are a separate switch.

  • Settings -> Pages -> Source = "GitHub Actions". Without it, docs.yml's build job succeeds but deploy fails with an opaque error, and every published docs link (including the ones in README.md) stays dead.

  • Copilot code review enabled on the repository. Without it, .github/copilot-instructions.md and .github/instructions/* are never read, so Copilot's PR reviews ignore this project's conventions.