SKILL.md. Writing one that measurably improves an agent is a craft with learnable rules — this guide teaches them, with a before/after example for each.
A skill’s worth is its lift (the with-vs-without improvement): how much better your agent scores with the skill loaded than without it. Everything below serves that number — making it real, making it big, and knowing when it’s gone.
The one rule
A skill lifts only when it supplies knowledge the model can’t infer on its own. Generic advice is not that knowledge. A capable model already knows to “write clean SQL” or “be thorough” — a skill restating it measures roughly zero. What lifts is the specific, unguessable stuff: your company’s field conventions, a spec’s exact token list, a new API the model hasn’t seen. A pattern we see constantly (numbers illustrative — run your own benchmark):- A skill saying “write correct, well-structured SQL” → the model already passes every case without it. Lift: ~0.
- The same schema plus four house conventions the model cannot guess (“active user =
deleted_at IS NULL”, “revenue =net_amounton completed orders only”, “fiscal year starts Feb 1”) → the model fails without the skill and passes with it. Lift: large.
One corollary you never violate: never game an eval. If honest iteration shows no lift, the model is genuinely good at this — re-scope the skill or drop the idea. A manufactured number is worse than no skill.
Step 1: Classify your skill
Before writing a line of the body, declare the skill’s classification. Two orthogonal axes decide what evals the skill needs and how long it will live.The two axes
Run this litmus, in order:Could the base model do this task at all? If no →capability(you’re teaching a missing ability). If yes, but you want a specific FORM →preference(you’re steering the shape). Then: is that knowledge public or private? A published standard the base may one day learn →public. Internal house knowledge the base will never absorb →private.
skill-type: capability— the base model genuinely can’t do the task yet: parse a particular PDF layout, call an internal API, know an internal DB schema. The skill teaches a missing ability.skill-type: preference— the base model can do the task; the skill only steers the form to a chosen shape: a PR-description format, a commit convention, a review checklist, a naming scheme. The skill encodes a chosen process or style.skill-scope: public— a standard the base may absorb as models improve (RFC 9457, Conventional Commits, BEM).skill-scope: private— internal knowledge the base will never learn from training data (your DB schema, your PR format).
Only
(capability, public) expires. A private ability or a chosen house form stays useful indefinitely; only public capability decays, because a public capability is exactly the thing a future base model can pick up. That one quadrant is why the platform re-benchmarks on a monthly cadence — when the base catches up, with ≈ without, and the skill is flagged to retire.
Legacy
skill-type: model-gap | proprietary | convention frontmatter still parses — it maps to (capability, public), (capability, private), and (preference, public) respectively — so existing skills keep working without an edit.Choosing a preference skill to author
A good preference skill encodes a process or format many teams would recognize — a PR-description structure, a code-review checklist, a commit convention, a docstring style. Avoid obscure standards picked only because the base model doesn’t default to them: a skill that lifts solely because it teaches a rule almost nobody uses is closer to gaming the benchmark than teaching something useful (see Step 5). Name and frame skills generically —pr-description, code-review-checklist, commit-format — and do not tie a skill to a fictional company (no “Acme PR format”). A preference skill should read as a normal, reusable skill any team could install, not a private artifact dressed up as a public one. (Genuinely internal house style is fine — that’s a private-scope skill; just don’t invent a fake company to make a public standard look proprietary.)
Edge cases — classify by where the graded knowledge lives:
- A house style guide built on a public standard: if your eval grades the public standard’s rules, it’s
(preference, public); if it grades your private additions, it’sprivatescope —capabilityorpreferencedepending on whether the base could produce them unaided at all. - Framework-migration rules are
(capability, public)and simply retire when models catch up — they don’t turn into preferences.
skill-type: preference, skill-scope: public. Litmus checked: the model can do the task, we’re steering the form, and the standard is public. It needs conformance cases plus a re-test on model upgrades, and it retires only if a future model emits feat: add csv export unprompted.”
Automatic or on-demand?
Theinvocation field decides who fires the skill:
any takes the stricter rule from each row. Default to model; choose user for workflows a person runs intentionally (release procedures, cleanup scripts) where automatic triggering would only be noise. Outcome evals are required either way — invocation mode never waives lift.
Claude Code spells on-demand as
disable-model-invocation: true. DecimalAI treats that as equivalent to invocation: user and round-trips both spellings on import and export.Step 2: Write the description as the trigger
Trigger problems cause most skill failures in the wild — and rewriting the description alone fixes most of them. The description is not documentation. It is the firing mechanism, and two separate steps read it before your skill can act:- Retrieval — the router embeds your description and matches it against the user’s query. If it isn’t retrieved, the skill can never fire. It needs the concrete nouns and verbs users actually type.
- Menu selection — retrieved skills are shown to the model as name + description rows, and the model picks one. Your row has to beat its neighbors.
- State the WHAT and the WHEN, in third person: what the skill does, and the situations where it applies.
- Name the artifacts and verbs users type (“git commit message”, “docx”, “migration”) — not category words (“documents”, “helper”, “workflow”).
- Add negative scope: one “Do NOT use for …” clause naming the nearest thing this skill must not hijack.
- Don’t keyword-stuff. Every extra topic word widens retrieval and raises false fires — and both are measured by your trigger cases (Step 4).
description: Helps with documents
After: description: Create, edit, and analyze .docx files — tracked changes, comments, formatting, text extraction. Use when the user works with Word documents. Do NOT use for PDFs or plain-text/markdown files.
Before: description: Formats commits nicely
After: description: Format git commit messages to the Conventional Commits spec (type, scope, imperative subject). Use when writing or rewriting a commit message. Do NOT use for PR descriptions, changelogs, or code review comments.
For invocation: user skills, write for a human scanning a list instead: lead with the outcome (“Clean up stale worktrees and prune merged branches”).
Step 3: Write the body
The body is the knowledge, stated so the model acts on it. Four rules.Directives, not essays
Models follow instructions; they skim past background prose. Give the reason behind a rule in one clause when it helps generalization — not a paragraph. Before: “The Interactions API is generally recommended for multi-turn chat scenarios because it handles session state automatically and is the direction the SDK is moving.” After: “Always useclient.interactions.create() for chat. Never use the legacy generate_content API (it drops session state).”
The highest-value section in any body is worked examples: one before→after pair per rule cluster, showing the model’s wrong default and the conforming output. A 5-line example beats 5 paragraphs of explanation.
Constraints, not steps
Describe the outcome and the constraints, not a step-by-step path — a rigid script strips the model’s ability to recover from surprises. If exact, fragile steps are genuinely required, write ascripts/ program instead of prose: code is deterministic; prose pretending to be code is neither.
Before: “Step 1: Read config.json. Step 2: Extract the port. Step 3: Edit line 4. Step 4: Save the file.”
After: “Set the database port in config.json to 5432. The file must still parse. Always run the tests before opening the pull request.”
Exception: on-demand (invocation: user) playbooks may be procedural — the user explicitly asked for that workflow.
Put depth in references/, exact steps in scripts/
A skill is a folder, loaded in three layers with three different costs:
Placement rules: keep
references/ files one level deep, give any reference over ~100 lines a table of contents, and make the body point at each reference with when-to-open guidance (“per-language edge cases: references/edge-cases.md”) — an unmentioned reference is never read.
Before: one 480-line SKILL.md — 30 rules, a 200-line per-country exceptions table, and a
three-paragraph essay on why formatting matters.
After: a ~110-line body carrying the 30 rules and two worked examples; the exceptions table
moved to references/exceptions.md with a pointer (“per-country exceptions:
references/exceptions.md”); the essay deleted — it failed the no-op test below.
Length is a cost
The body is injected into your agent’s context on every activation — every line is paid for in tokens, and the A/B benchmark measures that cost. There is no minimum length and no “sweet spot”: a 60-line skill with real knowledge lifts fine, and past roughly a thousand lines a skill tends to measure as a no-op because bloat buries the directives. The editing tool is the no-op test: remove the line — does output change? If not, delete it. Before: “Be thorough and precise. Write clean, high-quality, maintainable commit messages that your teammates will appreciate.” (removing this changes nothing → delete) After: the line is gone; the exact type-token list it was crowding stays.Step 4: Build the eval suite
No skill ships on vibes. Two manual runs is not testing — bad skills don’t crash, they quietly corrupt outputs. The suite has two independent parts: graded cases (does the skill change behavior?) and trigger cases (does it fire at the right times?). See skillevaluation for theeval.yaml format and runner.
Compose the golden prompts
Start with 10–20 prompts and extend from failures — every production bug becomes a permanent case. Compose deliberately, three kinds:- Happy path — the primary intended workflows, one per major rule in the skill, so every check gets cases where it matters. Twenty-two paraphrases of one prompt is not coverage.
- Near-miss negatives — inputs where the model’s wrong default is most tempting, plus easy cases the model already handles. A suite made only of the model’s worst cases flatters the skill; the easy passes honestly dilute the lift.
- Production-shaped — prompts with the mess of real use: surrounding context, irrelevant detail, imperfect phrasing.
If a program can check it, make it a validator
Every check in a case is one of exactly two kinds — never blended:validators— code decides. Shell commands run against the response; the exit code is the verdict. Regex, parsers, executing the emitted SQL and diffing rows. Deterministic, free, milliseconds. Put everything a program can check here.expectations— an LLM (large language model) judge decides. Qualitative properties only: tone, faithfulness, judgment calls.
The expectation litmus
For every expectation, ask: “would a generic ‘be careful and well-structured’ instruction also pass this?” If yes, it’s testing general competence, not your skill’s knowledge — rewrite it into a specific, binary, observable property only the skill supplies. Before: “The commit message is clear and well-formatted.” After: “The subject begins with a lowercase type token from the closed set (feat, fix, docs, …) — not ‘feature’, ‘add’, or ‘new’ — followed by a colon and a single space.”Trigger cases — automatic skills only
Available from skillevaluation 0.3.0 (the
should_trigger field).- 3
should_trigger: true— phrased from three different angles (different vocabulary, different intent framing, different surface form), so recall isn’t one lucky phrasing. - 3
should_trigger: falsenear-misses — one adjacent topic, one same-keywords-different-intent, one generic task. Far-away negatives (“write a poem”) teach nothing; the near-miss is the test.
invocation: user) skills skip this section entirely.
Repeat runs — average out the luck
Model behavior is probabilistic; one run per case measures luck. Run the suite with--runs 3: the whole suite executes three independent times and the per-case results are averaged by mean, so the headline pass-rate’s expected value doesn’t depend on the count — more runs just tighten the confidence. Repetition is a runner flag, not a per-case field, so no single case can be weighted more than another. (This replaced the old per-case trials/pass^k knob in skillevaluation 0.6.0.)
Step 5: Keep the number honest
The benchmark catches many problems automatically. These are the ones only you can catch. Never leak the rule into a test prompt — including by paraphrase or by example. A graded prompt must not name, state, paraphrase, or show a sample of the conforming output; otherwise you’re measuring the prompt, not the skill. Before (leak): “Write a Conventional Commits message for the CSV export feature.” After: “I added an endpoint that lets users export their data as CSV. Write a one-line git commit message.” Keep case prompts disjoint from the body’s worked examples too — a shared example is a leak by another route. Don’t stack the deck. A suite composed only of the model’s worst cases flatters the skill. Include the easy cases it already passes; they keep the lift honest. The with-skill arm must never score worse. If loading the skill drops any score below the no-skill baseline, the skill is actively hurting — fix it or cut the offending rule. Low-quality skills measurably reduce accuracy; “mostly helps” is not good enough. Teach real conventions, not invented ones. Every graded token should be something a practitioner following the convention without your skill would produce. A made-up tag vocabulary scores spectacular lift (the model fails every case without it, by construction) and teaches nothing anyone asked for. Big lift from invented rules is gaming, and only the author can catch it. One behavior per skill. If the topic has parallel variants (per-venue templates, per-language style guides), scope to one variant or split into one skill per variant — cases graded against a different variant become noise while the headline stays high. Honest drops beat no-lift skills. If the model already does it, drop the idea and note why. A healthy authoring pipeline drops a meaningful fraction of its candidates.Lifecycle: ablation and retirement
Skills ablation means testing your agent with and without the skill loaded — the same eval suite, run in both arms. It’s how lift is measured at birth, and how retirement is detected later. The schedule follows the quadrant you declared in Step 1:
Reading an ablation (numbers illustrative — run your own):
- with 94% / without 32% → +62 lift: real knowledge, keep it loaded.
- with 96% / without 95% → ~0 lift: the model absorbed it — retire the skill; every activation now costs tokens for nothing.
stability: deprecated so consumers see the state before you do). The registry shows each published skill’s re-verification history (“Verified on ⟨model⟩ — re-tested ⟨date⟩”), so consumers can see whether a skill’s lift is fresh or stale — see Community Registry.
Self-check before you publish
skill-type,skill-scope, andinvocationdeclared, and you can say why (Step 1).- Description states what + when + “Do NOT use for…”, third person, no keyword stuffing (Step 2).
- Every rule is a directive; zero passive essays (Step 3).
- Constraints, not steps — or a script where steps must be exact (Step 3).
- Worked examples cover every rule cluster: wrong default → conforming output (Step 3).
- Every line survives the no-op test; depth moved into
references/(Step 3). - Anything that must prove lift is in the body, not references (Step 3).
- Graded set composed: happy path + near-miss negatives + production-shaped (Step 4).
- Automatic skills: at least 6 trigger cases (3 fire / 3 near-miss no-fire), outside the graded set (Step 4).
- Everything code can check is a validator; expectations are qualitative-only and pass the litmus (Step 4).
--runs 3— the verdict you act on is a mean over repeats, not a single lucky run (Step 4).- No prompt leaks the rule; the lift comes from real, adopted knowledge (Step 5).
- You know what this skill’s ablation should show a year from now (Lifecycle).
Related
Skills guide
The SKILL.md format, discovery, routing, sync, and observability.
skillevaluation
The eval.yaml format and the open A/B runner that measures lift.
Community Registry
Publish gates, badges, trigger health, and re-verification history.
SkillScore
How published skills are scored from benchmark, live, rating, and adoption evidence.