Skills - DecimalAI

Skills are structured, reusable instructions that tell AI agents how to perform specific tasks. DecimalAI adds observability on top: tracking which skills activate, how effective they are, and how they change over time. This is a long guide. Jump to what you need:

What's a skill?

The SKILL.md format and frontmatter fields.

Get skills in

Auto-discovery from disk, SDK sync, or platform-first authoring.

Routing & sync

Smart routing, bidirectional sync, skill-specific evaluators.

Registry

Publish to the public registry with SkillScore effectiveness.

Agents

Per-agent skill assignment, dashboard, manifest tracking.

Playground

Test skill edits against real production traces before saving.

Benchmark (eval.yaml)

A/B benchmark the skill via the open skillevaluation spec.

What Is a Skill?

A skill is a directory containing a SKILL.md file and optional supporting files:

code-review/
├── SKILL.md              ← The main instruction file
├── scripts/
│   └── scan.py           ← Helper script the agent can execute
└── references/
    └── owasp-top-10.md   ← Reference material the agent can read

Skills follow the open agentskills.io specification. When an AI agent (Claude Code, Cursor, Copilot, etc.) encounters a matching task, it reads the SKILL.md and follows the instructions.

Skills are not tools. Tools are functions the agent can call (search, calculate, etc.). Skills are instructions that tell the agent how to approach a task — they’re composable behavior units.

SKILL.md Format

Every skill has YAML frontmatter and a markdown body:

---
name: code-review
description: Reviews code for security vulnerabilities and bugs
license: MIT
allowed-tools:
  - bash
  - python
---

# Code Review

When asked to review code, follow this process:

## Step 1: Security Scan
Check for:
1. SQL injection vulnerabilities
2. Cross-site scripting (XSS)
3. Unvalidated input

## Step 2: Output Format
Present findings as a table:
| Issue | Severity | File | Line |

Field	Required	Description
`name`	✅	Unique identifier (lowercase, hyphens)
`description`	✅	Short description (used for routing and registry)
`license`		MIT, Apache-2.0, etc.
`allowed-tools`		Tools the skill may invoke
`category`		Grouping category
`stability`		`stable`, `experimental`, or `deprecated`

Getting Skills Into DecimalAI

There are two separate questions, and it helps to keep them apart:

How does a skill get into your workspace? Three sources, covered below: auto-discovery from disk, install from the public registry, or manual creation.
How does a skill reach a running agent? Once it’s in your workspace, pick one delivery path per runtime: hosted routing via the Skill Router (your SDK agent asks the platform which skills to load on each query — this is the path that measures effectiveness and powers the leaderboard) or disk export (DecimalAI writes the open-format SKILL.md into the directories Claude Code / Cursor / etc. already scan, and the runtime loads it itself).

Running both delivery paths for the same runtime double-injects the skill into the system prompt. Pick one. See the Skill Router — Router vs disk auto-loading section for the full decision matrix; the SDK logs a one-shot warning when it detects this configuration.

1. Auto-Discovery (Bring Your Own)

If you already have SKILL.md files, DecimalAI discovers them automatically:

import decimalai
decimalai.init(api_key="dai_sk_...")

from decimalai.openai_agents import install
install()  # auto-discovers SKILL.md files from disk

The SDK scans standard directories:

Location	Scope
`.agents/skills/`	Universal (Cursor, Copilot, Cline, Warp)
`.claude/skills/`	Claude Code
`.windsurf/skills/`	Windsurf
`.continue/skills/`	Continue
`~/.claude/skills/`	Global / personal

Your files are never modified or moved. DecimalAI acts as a passive observer — it reads SKILL.md files to build the registry but never alters them.

2. From the Public Registry

Browse community skills and install them:

from decimalai.skill_router import SkillRouter

router = SkillRouter(api_key="dai_sk_...")

# Install "pdf" skill for Claude Code and Cursor
result = router.install("pdf", agents=["claude-code", "cursor"])

This forks the skill into your org, writes SKILL.md + attachments to disk, and creates a lockfile at .decimal/skills.lock.

3. Manual Creation

Create skills directly via the SDK or API:

router.create_skill(
    name="my-custom-skill",
    description="Does something specific",
    body_markdown="# My Skill\n\nInstructions...",
    category="automation",
)

How Skills Are Tracked

Auto-Versioning

Every time you edit a SKILL.md and restart your app, the SDK:

Re-discovers the file
Computes a content hash
If the hash changed → creates a new version automatically

No manual tagging needed. The dashboard shows version history with SkillScore trends:

Version	Date	Pass rate	Trend
v3 (current)	Apr 17	88%	↗ Improved by 4%
v2	Apr 10	84%	↗ Improved by 12%
v1	Apr 1	72%	—

Activation Detection

The SDK detects which skills were activated by comparing the LLM’s rendered prompt against known skill bodies. This works automatically across all frameworks — no code changes needed. Each trace includes active_skills metadata:

{
  "trace_id": "abc-123",
  "active_skills": [
    {"name": "code-review", "hash": "a1b2c3d4e5f6"}
  ]
}

Effectiveness Scoring

The platform correlates skill activations with trace evals to compute:

Pass rate: % of traces with the skill that evaluated as “pass”
Effectiveness: The skill’s SkillScore — a 0–100 quality composite from benchmark lift, live eval pass rates, and AI-judge quality
Trend: Improvement or regression over time

How activation detection works internally

The SDK uses a three-tier approach:

Prompt-diff matching — Compares the rendered LLM prompt against known skill bodies using fuzzy content matching
Span attribute detection — Checks for decimal.active_skills span attributes (OTel frameworks)
Explicit declaration — trace.log_skill_activation(name="code-review") for manual control

Priority: explicit > span attributes > prompt-diff. Multiple tiers can be active simultaneously.

Skill Routing

When your agent needs to select which skills to load, DecimalAI offers two routing strategies: Dump all skill names + descriptions into the system prompt and let the LLM decide:

router = SkillRouter(api_key="dai_sk_...")

menu = router.get_menu()
system_prompt += menu["prompt_fragment"]

Best for agents with < 20 skills.

Smart Routing (Semantic)

For larger skill sets, smart routing uses semantic search + performance-weighted re-ranking:

router = SkillRouter(api_key="dai_sk_...", strategy="auto")

prompt_fragment = router.get_menu_prompt(
    query="Review this PR for security issues"
)
system_prompt += prompt_fragment

This embeds the user query, matches against skill descriptions via cosine similarity, then re-ranks by SkillScore. Skills with higher measured quality on similar queries get boosted.

get_menu_prompt(query=...) returns just the ready-to-inject prompt string. If you want the ranked skills plus the fragment together, call router.smart_route(query=...) — same semantic + effectiveness ranking, but it returns the full result dict (skills, prompt_fragment, strategy). The pricing page and Skill Router reference use smart_route(); both are real.

Bidirectional Sync

Skills sync automatically between your local files and the platform:

from decimalai.openai_agents import install
install(agent_name="claude-code")
# ↑ pushes local skills TO platform
# ↑ pulls platform-only skills TO disk
# ↑ updates local skills that changed on the platform

Disk State	Platform State	Action
SKILL.md exists	Same hash	Nothing (in sync)
SKILL.md exists	Different hash	Platform wins — disk updated
SKILL.md exists	Doesn’t exist	Push disk → platform
No SKILL.md	Skill exists	Pull platform → disk

All sync operations run in background threads and never block startup. However, if the platform is unreachable, the SDK falls back to local-only mode — skills are still discovered from disk.

Community Registry

Browse, install, fork, and publish skills in the public registry. The full lifecycle — discover, install, receive upstream updates, publish your own — is covered in its own guide:

Community Registry guide

Browse without signing up · install in one call · receive upstream updates · publish your own with effectiveness data attached.

Agent Skill Assignment

Once a skill is in your organization (created manually or installed from the registry), you need to assign it to specific agents. This gives you fine-grained control over which agents use which skills.

Assigning from the Skill Settings Tab

Navigate to any skill detail page → Settings tab → Agent Assignments section:

Select an agent from the dropdown and click “Assign”
Remove an assignment by clicking the “Remove” button next to an assigned agent
Each assignment shows the version mode: Latest (auto-updates) or Pinned (locked to a specific version)

Assigning from the Agent Skills Tab

Navigate to any agent → Skills tab:

Click ”+ Add Skill” in the table header → a picker modal shows all org skills not yet assigned
Search, multi-select, and click “Assign” to batch-assign skills
Click “Browse Registry →” to discover and install new skills from the public registry

How Assignment Works at Runtime

When an agent runs, the Skill Router checks which skills are subscribed to that agent:

# The SDK handles this automatically:
# 1. Fetch subscribed skills for this agent
# 2. Build the skill menu (prompt fragment)
# 3. Inject into the system prompt
# 4. Track which skills activate per trace

Skills assigned to an agent are injected into its prompt context at runtime. The platform then tracks which skills actually activated on each trace for analytics.

Agent Skills Dashboard

The Skills tab on each agent’s dashboard provides full observability into how skills perform for that specific agent.

Summary Cards

Four metrics at the top:

Metric	Description
Skills Used	Number of distinct skills that have activated on this agent’s traces
Total Activations	Cumulative skill activation count across all traces (30d)
Avg Pass Rate	Mean eval pass rate across all skills
Avg Effectiveness	Mean SkillScore across all skills — a quality composite, not volume

Activation Timeline (30d Heatmap)

A GitHub-style heatmap grid showing daily activation intensity for each skill over the last 30 days:

Rows = skills (top 6 by effectiveness)
Columns = days (30 cells, left = 30 days ago, right = today)
Color intensity = activation volume (darker green = more activations)
Hover = exact date, activation count, and pass rate

This lets you quickly spot:

Which skills are used most frequently
Whether a skill’s usage is increasing or declining
Days with unusually high or low activity

Skill Insights

Auto-generated insight cards based on the data:

🏆 Top Skill — The skill with the highest SkillScore, shown with its pass rate and usage count
📈 Improving — Skills whose pass rate is trending upward over the last 15 days
⚠️ Degrading — Skills whose pass rate is declining, with a suggestion to review the latest version

Leaderboard Table

All skills ranked by effectiveness with columns:

Column	Description
Uses	Activation count for this agent (30d)
% Evals Passed	Percentage of evals that passed on traces using this skill
% Trace Success	Percentage of traces that completed without errors
Effectiveness	Composite score with color-coded bar (green ≥70%, amber ≥40%, red <40%)
Trend	Direction indicator: ↑ improving, → stable, ↓ degrading

Skills and Manifests

Skills are tracked as a skill_registry surface in your agent’s version manifest. When the skill registry changes, a new manifest version is registered:

What Changes	New Manifest?	Why
Skill registry updated (add/remove/edit)	✅ Yes	Different available behaviors
Different skills activated per task	❌ No	Same agent, different runtime path
Skill content edited (body changed)	✅ Yes	Available instructions changed

Skill changes follow the same compatibility policy as other surfaces (keep/repair/replay/drop). See Manifests & Versioning for details.

Testing Skill Changes in the Playground

Before saving a skill edit, test it against real production traces in the Playground — open a skill detail page, click “Test in Playground”, edit the body, and run it side-by-side against the original output. See the Playground guide for the full procedure.

Writing Effective Skills

Do

✅ Be specific: “Check for SQL injection by looking for string concatenation in queries”
✅ Include examples: Show expected inputs and outputs
✅ Use structured output: Define tables, JSON schemas, or templates
✅ Break into steps: Numbered instructions for clarity
✅ Reference attachments: See scripts/scan.py for the scanning utility

Don’t

❌ Be vague: “Review the code” — review for what?
❌ Duplicate: If two skills overlap, merge them
❌ Include secrets: No API keys, credentials, or tokens
❌ Use absolute paths: Use relative paths from the skill directory

Next Steps

Skills Observability tutorial

See per-skill activation counts, pass rates, and smart routing in action.

Skills API

REST reference for create, sync, fork, publish, version diff.

Skills & Data Pipeline

Conceptual model — skills vs tools, activation tracking, effectiveness.

Public registry

Browse community skills ranked by SkillScore — measured quality from live activations, not install counts.

What's a skill?

Get skills in

Routing & sync

Registry

Agents

Playground

Benchmark (eval.yaml)

​What Is a Skill?

​SKILL.md Format

​Getting Skills Into DecimalAI

​1. Auto-Discovery (Bring Your Own)

​2. From the Public Registry

​3. Manual Creation

​How Skills Are Tracked

​Auto-Versioning

​Activation Detection

​Effectiveness Scoring

​Skill Routing

​Menu Injection (Simple)

​Smart Routing (Semantic)

​Bidirectional Sync

​Community Registry

Community Registry guide

​Agent Skill Assignment

​Assigning from the Skill Settings Tab

​Assigning from the Agent Skills Tab

​How Assignment Works at Runtime

​Agent Skills Dashboard

​Summary Cards

​Activation Timeline (30d Heatmap)

​Skill Insights

​Leaderboard Table

​Skills and Manifests

​Testing Skill Changes in the Playground

​Writing Effective Skills

​Do

​Don’t

​Next Steps

Skills Observability tutorial

Skills API

Skills & Data Pipeline

Public registry

What Is a Skill?

SKILL.md Format

Getting Skills Into DecimalAI

1. Auto-Discovery (Bring Your Own)

2. From the Public Registry

3. Manual Creation

How Skills Are Tracked

Auto-Versioning

Activation Detection

Effectiveness Scoring

Skill Routing

Menu Injection (Simple)

Smart Routing (Semantic)

Bidirectional Sync

Community Registry

Agent Skill Assignment

Assigning from the Skill Settings Tab

Assigning from the Agent Skills Tab

How Assignment Works at Runtime

Agent Skills Dashboard

Summary Cards

Activation Timeline (30d Heatmap)

Skill Insights

Leaderboard Table

Skills and Manifests

Testing Skill Changes in the Playground

Writing Effective Skills

Do

Don’t

Next Steps