The four signals
A SkillScore blends up to four independent kinds of proof:Benchmark
The skill is run on example tasks with it vs. without it. Did it produce lift (the with-vs-without improvement)?
Live eval
On real production runs that were evaluated, did the skill pass?
AI rating
An AI judge scores the quality of real production outputs.
Adoption
Is the skill actually being used — across more than one org?
A skill with no evidence yet shows New rather than a number — not a failure, just nothing measured.
How the four signals combine
Each signal gets a fixed weight, and the weights of whichever signals are present are scaled back up to 100% — a missing signal never counts as a zero:
Example: a skill with only benchmark + live eval evidence scores 50/50 from those two (0.32 and 0.32 rescaled). A skill with exactly one signal shows its score marked provisional (zero signals shows “New”, no score).
Two design choices worth knowing:
- Adoption can’t run away. The adoption signal grows logarithmically (going from 5 to 50 uses matters; 5,000 to 5,050 barely does) and is capped by org diversity — heavy usage concentrated in a single org is pinned down. Breadth is what counts, not raw volume. When the quality signals are present, adoption is the smallest major weight — but note the rescaling above: a skill with few signals leans harder on whichever it has, which is exactly why such scores are marked provisional.
- Efficiency is a separate axis, not part of the score. Turn/token savings at equal correctness are tracked and shown, but deliberately not blended in — a “fast but wrong” skill can’t buy headline score with cheapness.
Where the evidence comes from
The score is always computed from the evidence attached to that specific skill. What differs is whose runs count and who can see the result:Forking starts a fresh score
When you fork a skill you get an independent, editable copy. Because you can change it, it does not inherit the original’s score — it builds its own from your runs. The original keeps its public score.
Public reputation vs. your results
For a public skill your team uses, you’ll see two complementary numbers. They answer different questions, so we keep them separate:SkillScore (public)
“Is this proven to work, across everyone?” The cross-org reputation — use it to decide whether to adopt a skill.
Your results
“Is it working for us?” Your team’s own pass rate, usage, and ratings on the skill — use it to catch a version that regressed for your workload, then pin a version or fork your own.
How it’s calculated
A background job regularly reads each skill’s benchmark runs, evaluated traces, AI ratings, and usage, computes the 0–100, and stores it with a per-signal breakdown. Public skills are scored from anonymized cross-org evidence; private and forked skills are scored from your team’s data only and kept private to your org. Only verified benchmark runs feed the benchmark signal — runs the platform executed itself, or publisher-executed runs the platform re-graded and spot-checked. Self-reported numbers that never pass verification never do.On the leaderboard
The registry leaderboard ranks skills by SkillScore. A skill needs at least one signal to appear — and the more corroborating signals it has, the more its score can be trusted.Related
Public Registry
Browse and install skills ranked by SkillScore.
skillevaluation
The A/B benchmark that produces the strongest SkillScore signal.
Authoring Skills
Write a skill that earns real lift — and prove it honestly.