> ## Documentation Index
> Fetch the complete documentation index at: https://docs.decimal.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Skills Observability

> Tutorial: Track skill effectiveness from discovery to production analytics.

This tutorial shows how DecimalAI tracks your agent's skills — from initial discovery through activation detection and effectiveness analysis. By the end, you'll know which skills are working and which need improvement.

## The Scenario

You have a coding assistant with 3 skills: `code-review`, `sql-optimizer`, and `deploy-checklist`. You want to know:

* Which skills are being used most?
* Which version of `code-review` performs best?
* Should you keep `deploy-checklist` or retire it?

***

<Steps>
  <Step title="Auto-Discover Skills">
    Your skills live in `.claude/skills/` as SKILL.md files:

    ```
    .claude/skills/
    ├── code-review/
    │   └── SKILL.md
    ├── sql-optimizer/
    │   └── SKILL.md
    └── deploy-checklist/
        └── SKILL.md
    ```

    Initialize DecimalAI — skills are discovered automatically:

    ```python theme={null}
    import decimalai
    decimalai.init(api_key="dai_sk_...")

    from decimalai.openai_agents import install
    install()
    # ✓ Discovered 3 skills: code-review, sql-optimizer, deploy-checklist
    # ✓ Pushed to platform (synced hashes)
    # ✓ Registered in manifest as skill_registry
    ```

    No configuration needed. The SDK scans standard directories, parses SKILL.md frontmatter, and registers the skill registry.
  </Step>

  <Step title="Run Your Agent">
    Your agent handles requests as normal. For each trace, the SDK detects which skills were activated by comparing the LLM's rendered prompt against known skill bodies.

    ```python theme={null}
    # User asks: "Review this PR for security issues"
    # → Agent loads code-review skill
    # → Trace records: active_skills: [{"name": "code-review", "hash": "a1b2c3"}]

    # User asks: "Optimize this SQL query"
    # → Agent loads sql-optimizer skill
    # → Trace records: active_skills: [{"name": "sql-optimizer", "hash": "d4e5f6"}]
    ```

    <Note>
      No extra code needed — activation detection is automatic across all framework integrations.
    </Note>
  </Step>

  <Step title="View Skill Analytics">
    After accumulating traces, check the Skills page in the dashboard. Each skill shows:

    * **Activation count**: How many traces used this skill
    * **Pass rate**: % of skill traces that passed evaluators
    * **Effectiveness**: Composite score
    * **Trend**: Improving, stable, or declining

    Example data after 1 week:

    | Skill              | Activations | Pass rate | Trend           |
    | ------------------ | ----------- | --------- | --------------- |
    | `code-review`      | 145         | 88%       | ↗ trending up   |
    | `sql-optimizer`    | 67          | 92%       | → stable        |
    | `deploy-checklist` | 3           | 33%       | ↘ trending down |

    <Warning>
      `deploy-checklist` is barely used and performs poorly. Consider retiring it or rewriting the instructions.
    </Warning>
  </Step>

  <Step title="Improve a Skill">
    Edit the `code-review` SKILL.md to add better instructions:

    ```markdown theme={null}
    ---
    name: code-review
    description: Reviews code for security vulnerabilities and bugs
    ---

    # Code Review (Updated)

    ## Step 1: Security Scan
    Check for these specific vulnerabilities:
    1. SQL injection — look for string concatenation in queries
    2. XSS — check for unescaped user input in HTML templates
    3. CSRF — verify token validation on state-changing endpoints
    ```

    Restart your agent. The SDK detects the content hash changed, creates **v2**, and registers a new manifest version.
  </Step>

  <Step title="Compare Versions">
    After v2 collects traces, compare the two versions:

    | Version          | Traces | Pass rate | Delta |
    | ---------------- | ------ | --------- | ----- |
    | `code-review` v1 | 145    | 88%       | —     |
    | `code-review` v2 | 52     | 94%       | ↗ +6% |

    The improved instructions are catching more security issues. DecimalAI's comparison includes statistical significance testing — so you know the improvement is real, not noise.
  </Step>
</Steps>

***

## Smart Routing (Bonus)

<Accordion title="Use smart routing for large skill sets">
  When you have many skills, use smart routing to select the best ones for each query:

  ```python theme={null}
  from decimalai.skill_router import SkillRouter

  router = SkillRouter(api_key="dai_sk_...", strategy="auto")

  prompt_fragment = router.get_menu_prompt(
      query="Review this PR for security issues"
  )
  # → Returns code-review (high relevance + high pass rate)
  # → Does NOT return deploy-checklist (low relevance + low pass rate)
  ```

  Smart routing combines semantic similarity with historical effectiveness — skills that perform well on similar queries get boosted.
</Accordion>

***

<Note>
  **Key takeaway:** DecimalAI turns skills from "static instructions" into **observable, measurable, improvable components**. You can see exactly which skills contribute to good outputs, which need improvement, and which should be retired — all backed by production data, not guesswork.
</Note>

## You've done it

<Check>Auto-discovered skills from `.claude/skills/` in production traces</Check>
<Check>Inspected per-skill activation counts and effectiveness scores</Check>
<Check>Used smart routing to select the best skills for each query</Check>

## Next Steps

<CardGroup cols={2}>
  <Card title="Skills Guide" icon="puzzle-piece" href="/guides/skills">
    Versioning, forking, publishing to the public registry.
  </Card>

  <Card title="Skills Registry" icon="store" href="https://app.decimal.ai/skills">
    Browse community skills ranked by SkillScore.
  </Card>
</CardGroup>