Failure-first harvest: run a skill's eval cases on the BASE model to see if the base already passes
Answer “should this skill exist?” before benchmarking/publishing it.
Runs the BASE model alone (no skill body) over the skill’s latest version’s eval
cases and returns a verdict — build (the base fails, so the skill has room to
lift), skip (the base already passes >= 80%, so the skill is unlikely to
help), or invalid_errors (too many provider errors to judge) — plus the base
pass-rate and per-case detail.
Calls the LLM once per eval case, so it is gated (owner-only) and METERED against the org’s monthly benchmark-run allowance (HTTP 429 when exhausted) — the usage limit that kept it off the customer surface until now. NB: runs synchronously; for large suites move it to a background job (follow-up).