The four signals
A SkillScore blends up to four independent kinds of proof:Benchmark
The skill is run on example tasks with it vs. without it. Did it produce lift (the with-vs-without improvement)?
Live eval
On real production runs that were evaluated, did the skill pass? (Today most of those runs come from our own agent fleet — see below.)
AI rating
An AI judge scores the quality of real production outputs.
Adoption
Is the skill actually being used — across more than one org? (Today most of that usage is our own agent fleet — see below.)
Where the production evidence comes from right now. DecimalAI runs its own fleet of automated agents that exercise registry skills against production — real executions, really evaluated, and that is what currently generates almost all of the live-eval and adoption evidence on public skills. It is deliberate bootstrapping: a registry cannot wait for outside traffic to tell it which skills work. What it means for you is precise — those signals say the skill held up when it was actually run, not lots of independent teams chose it. The benchmark signal is unaffected: lift comes from with-vs-without A/B runs on the skill’s own eval cases. (The AI-rating signal is not producing evidence at the moment, so no public score currently rests on it.)
A skill with no evidence yet shows New rather than a number — not a failure, just nothing measured.
How the four signals combine
Each signal gets a fixed weight, and the weights of whichever signals are present are scaled back up to 100% — a missing signal never counts as a zero:
Example: a skill with only benchmark + live eval evidence scores 50/50 from those two (0.32 and 0.32 rescaled). A skill with exactly one signal shows its score marked provisional (zero signals shows “New”, no score).
Two design choices worth knowing:
- Adoption can’t run away. The adoption signal grows logarithmically (going from 5 to 50 uses matters; 5,000 to 5,050 barely does) and is capped by org diversity — heavy usage concentrated in a single org is pinned down. Breadth is what counts, not raw volume. When the quality signals are present, adoption is the smallest major weight — but note the rescaling above: a skill with few signals leans harder on whichever it has, which is exactly why such scores are marked provisional.
- Efficiency is a separate axis, not part of the score. Turn/token savings at equal correctness are tracked and shown, but deliberately not blended in — a “fast but wrong” skill can’t buy headline score with cheapness.
Where the evidence comes from
The score is always computed from the evidence attached to that specific skill. What differs is whose runs count and who can see the result:Forking starts a fresh score
When you fork a skill you get an independent, editable copy. Because you can change it, it does not inherit the original’s score — it builds its own from your runs. The original keeps its public score.
Public reputation vs. your results
For a public skill your team uses, you’ll see two complementary numbers. They answer different questions, so we keep them separate:SkillScore (public)
“Has this held up wherever it has been run?” The cross-org reputation — use it to decide whether a skill is worth trying. Read it as evidence the skill works, not as evidence that many teams picked it: today most of the runs behind it are our own agent fleet’s.
Your results
“Is it working for us?” Your team’s own pass rate, usage, and ratings on the skill — use it to catch a version that regressed for your workload, then pin a version or fork your own.
How it’s calculated
A background job regularly reads each skill’s benchmark runs, evaluated traces, AI ratings, and usage, computes the 0–100, and stores it with a per-signal breakdown. Public skills are scored from aggregated cross-org evidence; private and forked skills are scored from your team’s data only and kept private to your org. Only verified benchmark runs feed the benchmark signal — runs the platform executed itself, or publisher-executed runs the platform re-graded and spot-checked. Self-reported numbers that never pass verification never do.On the leaderboard
The registry leaderboard ranks skills by SkillScore. A skill needs at least one signal to appear — and the more corroborating signals it has, the more its score can be trusted. Most skills on the board today have exactly one, so the board marks those rows provisional: a single line of evidence, not a settled result. A 100 with one signal and a 100 with four look the same until you read the marker.Related
Public Registry
Browse and install skills ranked by SkillScore.
skillevaluation
The A/B benchmark that produces the strongest SkillScore signal.
Authoring Skills
Write a skill that earns real lift — and prove it honestly.