Skip to main content

See it first — 2 minutes, no waiting for your own data

The impact report and the skills leaderboard are most convincing on real data — so both ship with a one-command sandbox that seeds a realistic agent and trace corpus into your workspace. See the payoff before you instrument anything.
DecimalAI requires Python 3.10 or newer. On Python 3.9 and older, pip silently installs an outdated release that lacks the demo command.
Pick the door that matches your job — or run both:

For engineers

Catch regressions before they ship.
Seeds a v1→v2 agent change and links you straight to the impact report — which production traces the change would break, which may behave differently, and which are unaffected.

For prompt engineers

Find skills that actually work.
Seeds three skills with real, varied effectiveness and links you to the ranked registry — per-model pass rates and cross-org activation, not download counts.
No account yet? Browse the public skill registry right now — no signup — to see skills ranked by production effectiveness.
Once a demo makes the value land, pick the track that matches what you came for.

Here for skills instead?

The rest of this page wires up the regression capability — what most teams start with. The skills workflow is a separate, shorter track (no GitHub Action needed):
1

Browse the registry

Find skills ranked by SkillScore in the public registry — no signup.
2

Try a skill with no account

skills add needs no API key or signup for a public skill:
Writes the registry’s own files for playwright-cli to .agents/skills/playwright-cli/, records the skill in .decimal/skills.toml, and pins the exact bytes in .decimal/skills.lock. Commit all three, and decimalai skills install --frozen reproduces the folder in CI (Skills on disk has the details). Point your framework’s skill loader at .agents/skills/; Claude Code loads it with claude --plugin-dir "$(git rev-parse --show-toplevel)/.agents". For a loose copy with the skill’s eval.yaml test suite instead, decimalai skills pull playwright-cli — the Install a Skill tutorial walks this exact skill end-to-end, and every keyless route is in Use skills without the SDK.
3

Prove one helps

A/B-benchmark a skill with skillevaluation (pip install "skillevaluation[runner]") to measure its lift on your own cases.
4

Install it

Add it to your workspace with Install on the skill’s page (or router.use("<name>") in the SDK) — a link that tracks the author’s updates and lets hosted routing offer the skill per turn. For runtimes that load SKILL.md from disk, decimalai skills add from step 2 is the same skill as files, and decimalai skills update moves it forward. See Vocabulary for Install vs Fork vs Export.

Wire DecimalAI to your own agent

No agent code yet? Build an agent creates one in the dashboard and decimalai init <agent-name> writes the code for it. This guide is for adding DecimalAI to an agent you already have.
You’ll do five things in this guide:
1

Install the SDK

1 minute.
2

Send your first trace with decimalai init

1 minute.
3

View your first trace in the dashboard

1 minute.
4

Instrument your agent, so the traces come from real runs

3 minutes.
5

Add the GitHub Action so every PR gets a manifest impact report

5 minutes.
By the end, your team’s next agent change will get an automatic structural impact analysis on the PR — without you writing any eval cases. Here’s the kind of report that lands on the PR:

Run this in Colab

Run the SDK portion interactively — no local setup required.

1. Install the SDK

The base package covers no-framework tracing, the CLI, and the Skill Router. Framework integrations install as extras — e.g. pip install "decimalai[langchain]" — shown per-framework in step 4. Each extra floors its framework at the latest release (SDK 0.14.0+), and two of them ask more of your environment:
  • [adk], [pydantic-ai] and [all] need Python 3.11+. The core package and the other extras install on 3.10.
  • [evals] is not part of [all], and can’t share an environment with [openai], [openai-agents] or [pydantic-ai]: litellm pins openai<3, and those extras need openai>=3.
pip reports neither conflict. It quietly installs decimalai 0.13.9 instead, the newest release whose requirements still resolve. pip show decimalai says which release you got; a floor on the SDK itself, such as pip install "decimalai[adk]>=0.14", makes pip fail with an error instead of falling back.
Checkpoint: decimalai --version prints a version number. If the demo command is missing, you’re on Python < 3.10 and pip silently installed an outdated release.

2. Send Your First Trace

One command puts a trace in your workspace — no agent code, nothing instrumented yet. It needs a DecimalAI API key — and you already have one. Signing up mints a default key for you automatically, and the first time the app loads after signup it shows a one-time green banner, “Your SDK API key is ready”, with the plaintext key and a Copy key button. The banner follows your session, so it appears on whichever page you land on, not just the home page. Copy the key there and put it in DECIMAL_API_KEY:
That banner is the only time the key is ever displayed — copying or dismissing it clears it for good, and the server keeps only a hash. If you missed it, don’t hunt for it: mint a replacement at Settings → API keys (see Creating a key). That tab is also where you create additional keys — one per CI job, per environment, per service. The POST /api-keys endpoint can mint those too, but it authenticates with an existing key, so it can’t be your very first one. Then run:
It checks your API key, tests connectivity, and sends a test trace — all in one command. Once the third check prints, your workspace has a trace in it; step 3 is just opening the dashboard to look at it.
Checkpoint: decimalai init prints all three green checks:
The test trace is filed under an agent named decimalai-init-test, so you’ll see that name in your agent list.A ✗ takes one of three shapes, and each names its own fix.No key found — nothing in DECIMAL_API_KEY or --api-key:
The CLI still prints the old label. The tab is now called API keys and has its own URL: app.decimal.ai/settings/api-keys.
Invalid key — the server answered and rejected it (wrong, truncated, expired, or revoked). Regenerate it at Settings → API keys:
Network problem — DNS failure, connection refused, or timeout. The server never answered, so the key may be fine — check --base-url and your connectivity:
Any other HTTP status prints ✗ Server returned HTTP <status> — the base URL answered but isn’t a DecimalAI backend. A DecimalAI bare auto-init failed warning above the checks is the import-time auto-init hitting the same problem the ✗ line diagnoses; fix the ✗ and it goes away.

3. View Your Traces

Open the Traces page in the dashboard. Your first trace should appear within seconds. Each trace is auto-tagged with the of the agent that produced it — this is what powers the regression check in step 5.
The DecimalAI Traces page: stat cards for trace count, error rate, tokens, estimated cost and p95 latency, above a table of demo agent traces showing ID, agent, status, eval verdict, tokens, cost, duration and input.

The Traces page, filterable by agent and by manifest version. Shown here after decimalai init plus the two decimalai demo seeds.

Checkpoint: the Traces page shows a row for the trace decimalai init just sent. Nothing after 30 seconds? Work through the silent no-ops below.
Skills (SKILL.md files) in your project’s directories (.claude/skills/, .agents/skills/, …) are discovered wherever the SDK reads disk: a @decimalai.trace run lists them in its manifest, and the LangChain and OpenAI Agents adapters upload them when installed without the skill loader. With their skill loader on, as in step 4’s LangChain and OpenAI Agents tabs and the init() framework flags from SDK 0.14.0, those adapters take skills from the platform only and read nothing from disk: push local files with decimalai skills sync .claude/skills, or pass init(skill_authority="harness") to keep the adapters’ sync. Personal directories like ~/.claude/skills/ are deliberately opt-in (include_global=True) so private skills don’t leak into your org’s registry, and a SKILL.md that carries the registry’s identity stamp — anything installed or copied from the registry — is never uploaded. See Skills.

4. Instrument Your Agent

That trace came from decimalai init, not from your agent. This step is what makes the traces real — one row per actual run, instead of a single test ping.
Calling openai / anthropic / google.genai directly, no agent framework in between? This is the shortest path — and it gets full skill routing:
build_prompt_fragment stamps the routing decision and the offered skill names onto the active trace automatically — no extra logging calls. On a network failure it returns ("", None) so prompt assembly never blocks.
Prefer zero manual logging? decimalai.init(openai=True) auto-traces every raw OpenAI SDK call — it drives an OpenInference instrumentor, so install both: pip install "decimalai[openai]" openinference-instrumentation-openai. anthropic=True / google=True work the same way via openinference-instrumentation-anthropic / openinference-instrumentation-google-genai (no extra for those — install the instrumentor package directly). Don’t combine a provider flag with a framework flag that already traces the same provider.On these raw rails, wrap each run in agent_run() so a multi-call tool loop lands in one trace instead of one trace per call — the instrumentor can’t see a run boundary you haven’t declared.
Auto-detection depth varies by framework. LangChain and OpenAI Agents extract full tool schemas (OpenAI Agents reads them from the running Agent, or at install with instrument(agent=...)); LlamaIndex / CrewAI extract tool names only. See the capability matrix before deciding which integration to commit to. AutoGen / AG2 is not an integration: init(autogen=True) installs the generic exporter and warns — see Generic OpenTelemetry.
Checkpoint: run your agent once. Startup logs show DecimalAI SDK initialized: base_url=https://api.decimal.ai ... with no auto-init failed or not installed warnings. Traces flush in the background and at process exit, and the Traces page shows a row for your run, tagged with a manifest hash, and the input/output you sent. Nothing after 30 seconds? Work through the silent no-ops below — the most common cause is an integration flag whose package isn’t installed.
Now wire DecimalAI into your CI so every PR gets a manifest impact report. This is the most-used capability for engineering teams.
What this step assumes, honestly:
  • An importable agent factory. The CI script imports and calls one function that constructs your agent. If construction is spread across a script, extract a build_agent() first.
  • A baseline builds on the first run. The Action’s first run finds no baseline manifest, records your current manifest as the baseline, and exits green — real diffs start on your second PR.
  • Impact counts come from your ingested traces. A workspace that just finished step 4 has a handful of traces, so early reports will honestly say few or zero traces are affected. The report earns its weight over days as production tracing accumulates volume.
Three things, all copy-pasteable below: a tiny scripts/init_for_decimal.py that calls your agent factory, a .github/workflows/decimal.yml that runs it under DECIMALAI_MODE=manifest_only, and your DECIMAL_API_KEY in GitHub Secrets. Here’s what runs on every PR: 1. Add scripts/init_for_decimal.py — it calls your existing agent factory, then registers the manifest as the PR’s candidate and writes its ID where the Action’s next step will look for it. In manifest_only mode the SDK reads tools, prompts, and models from the runtime objects, without any LLM calls:
scripts/init_for_decimal.py
If your factory doesn’t return a LangChain/LangGraph object, drop chain= and pass the components yourself — tools=[...], prompts={...}, models={...} — the same arguments register_manifest() takes. Without one or the other, the run registers an empty manifest. 2. Add .github/workflows/decimal.yml:
.github/workflows/decimal.yml
Prefer to see the output first? Omit api-key, agent-name and the manifest step and the Action runs in fixture mode: it renders a sample report (the seeded demo agent) and posts the same comment, labelled as sample data, with no key and no signup. 3. Add the DECIMAL_API_KEY secret in Settings → Secrets and variables → Actions → New repository secret, with the value from Settings → API keys in DecimalAI. That’s the whole setup. Once a baseline exists and traces have accumulated, each PR gets a comment like this within ~30 seconds:
Checkpoint: open a trivial PR. The Agent Regression Check check runs green, and the PR gets a comment. On the very first run the comment says “First run for this agent. Recorded the current manifest as the baseline.” — that’s the expected day-one state, not a failure. Impact counts like the example above appear from the second PR onward, sized by how many traces you’ve ingested.
Full setup, troubleshooting, severity thresholds, override behavior, and alerting are in the Regression Check Guide.

If something looks wrong: the six silent no-ops

The SDK is deliberately fail-open: a misconfiguration degrades quietly instead of crashing your agent. These are the six places that bites, in the order people hit them.
Browsing or previewing a skill on the public registry doesn’t make it routable — you have to adopt it into your org first. Either way works — Install linked (on the skill’s page, or router.use("name") in the SDK) creates a linked pointer that tracks the author’s updates, and Fork editable (on the skill’s page, or router.fork("name")) creates an editable copy you own. The Skill Router offers both. If you use per-agent assignments, also assign it to the requesting agent. Until you install or fork it, the skill never appears in your agent’s menu, with no warning anywhere.
By default build_prompt_fragment injects one-line menu rows (name + description). The skill’s actual instructions reach the model only if you pass inject_body=True to the SkillRouter (smart-routed queries), or enable the load_skill tool so the model can pull bodies on demand. Menu rows count as offered — usage panels show rung-labeled counts for them, and activation isn’t measurable for bare prompt-injection usage. If a skill “isn’t working,” check whether its body ever actually reached the model.
An adapter’s instrument() leaves the skill loader off unless you pass enable_skill_loader=True, and so does an init() framework flag given skills=False, or run with DECIMALAI_SKILLS=0 in the environment — good traces, zero skills, no warning. From SDK 0.14.0 the flags themselves (init(langchain=True), init(openai_agents=True), init(adk=True), init(crewai=True)) and DECIMAL_AUTO_TRACE turn the loader on. Through 0.13.9 they traced with it off, so on an older SDK either upgrade or call the adapter’s instrument(agent_name=..., enable_skill_loader=True). With the loader on, openai_agents and pydantic_ai also register the load_skill tool (kill switch: init(load_skill_tool=False) or DECIMALAI_LOAD_SKILL_TOOL=0). The langchain, anthropic, adk and crewai adapters have no tool loop, so they put the menu into the prompt, plus the top skill’s full body when the platform ranked the result by this turn’s message or the skill is the agent’s only one (init(inject_skill_body=False) keeps it to the menu); enable_load_skill_tool is accepted there but dormant.
decimalai.init(langchain=True) (and every other framework flag) logs a warning and continues untraced when the framework package isn’t importable — your agent runs, nothing is recorded. The same applies to DECIMAL_AUTO_TRACE (auto-init warns and skips) and to the raw-provider flags (openai=True etc.), which soft-skip when the matching OpenInference instrumentor is absent. Fix: for a framework flag, install the matching extra, e.g. pip install "decimalai[langchain]"; for a raw-provider flag, install the instrumentor package the startup warning names, e.g. pip install openinference-instrumentation-openai (the [openai] extra covers only the provider SDK, not the instrumentor). Then re-check startup logs.
Adapters only inject skills into prompt shapes they can safely rewrite. On langchain, the loader rewrites a string, a message list and a prompt template’s output; any other input shape passes through unchanged. The agent runs normally — with zero skills injected and no error raised.
The regression check needs two things before it can say anything real: a baseline manifest (recorded automatically on the Action’s first run, or by decimalai.init() running in production) and ingested traces to measure blast radius against. A fresh workspace legitimately reports “first run — no baseline” and then near-zero affected traces. That’s honesty, not breakage — reports gain weight over days as production trace volume accumulates.

Next Steps

Regression Check Guide

Full configuration, troubleshooting, and severity tuning for the GitHub Action.

Manifests Guide

What manifests capture, how diffs work, and the compatibility policy model.

Concepts

How traces, manifests, evals, and datasets connect.

Training Pipeline

End-to-end: trace → evaluate → fine-tune.