Skip to main content
The fastest way to understand DecimalAI is to see its two headline outputs on real-looking data. Both ship as one-command sandboxes that seed your workspace and link you straight to the result:
  1. decimalai demo regression — what a risky agent change does to production traffic (the impact report)
  2. decimalai demo skills — the skills registry, ranked by measured effectiveness
Everything seeded is prefixed [Demo] and fully removable. No agent code, no framework setup, no LLM API keys.

​
Prerequisites

  • Python 3.10+ (python --version — older Pythons silently install an outdated SDK)
  • A DecimalAI API key from app.decimal.ai/settings
1

Run the regression demo

In ~30 seconds this seeds a demo support agent with two manifest versions and a production trace corpus, then runs the regression check:
The two manifest hashes are shown as <hash> above because the real values differ per seed — don’t expect yours to match anyone else’s. The v1→v2 diff contains the three change types you’ll actually ship: a model swap, a tool rename + removal, and a prompt rewrite.
traces: 10 and Traces analyzed: 120 mean different things. 10 counts the curated v1 showcase traces the banner highlights — the click-into examples. The seed also writes a matching v2 set plus roughly 100 volume-history traces so the check runs against realistic traffic; the dashboard agent and the regression check both see all 120.
You should now see: a Verdict: line, then an “Open the impact report” line with an https://app.decimal.ai/agents/.../impact-reports/<id> link — as in the sample output above. If the command errors with demo being an unknown command, you’re on Python < 3.10 and pip silently installed an outdated release — upgrade Python and reinstall.
2

Read the impact report

Open the printed link. The report answers the question every reviewer has — what does this change do to traffic we’ve already served?
  • 🔴 HIGH IMPACT — traces that called the removed tool. They will break.
  • 🟡 MEDIUM IMPACT — traces touched by the model swap / prompt rewrite. Outputs may differ; structural analysis can’t predict direction.
  • 🟢 LOW IMPACT — traces that never touched a changed surface.
Below the severity bands, each affected trace gets a keep / repair / replay / drop verdict — the same classification that keeps training datasets valid as your agent evolves.This is exactly what the GitHub Action posts on every PR — computed against your production traces instead of seeded ones.
3

Run the skills demo

Seeds three workspace-scoped skills with deliberately varied effectiveness and runs the stats recompute. They’re visible only to your own workspace — nothing lands in the public registry:
The three seeded skills land at different SkillScores by design (roughly 93 / 79 / 50):The high performer is verified and tops the ranking; the weak one sits at the bottom — because SkillScore is computed from live eval pass rates and AI-judge quality, not install counts.
You should now see: three [Demo] -prefixed skills on your Skills page, ranked code-reviewer > sql-optimizer > flaky-summarizer. If the list is empty, the seed didn’t reach your workspace — check that DECIMAL_API_KEY is set in this shell and belongs to the workspace you’re viewing.
4

Clean up (optional)

Removes all [Demo] -prefixed agents, manifests, traces, and skills. Exact-prefix matched — anything you created yourself is untouched. Re-running either demo also resets first by default, so you always land in a clean state.

​
What just happened

No agent was executed and no LLM was called — the regression check is a structural query against the trace store, which is why it runs in seconds and costs nothing. The same mechanism, pointed at skills, produces the registry’s effectiveness ranking.

​
If something looks wrong

Demo failures are usually loud (auth errors print immediately). It’s the next step — wiring DecimalAI to your own agent — where the SDK’s fail-open design can quietly do less than you think: The SDK is deliberately fail-open: a misconfiguration degrades quietly instead of crashing your agent. These are the six places that bites, in the order people hit them.
Browsing or previewing a skill on the public registry doesn’t make it routable — you have to adopt it into your org first. Either way works — Use (router.use("name") or the Use button) creates a linked pointer that tracks the author’s updates, and Fork (router.fork("name"), or router.install("name") to also write it to disk, or the Fork button) creates an editable copy you own. The Skill Router offers both. If you use per-agent assignments, also assign it to the requesting agent. Until you Use or Fork it, the skill never appears in your agent’s menu, with no warning anywhere.
By default build_prompt_fragment injects one-line menu rows (name + description). The skill’s actual instructions reach the model only if you pass inject_body=True to the SkillRouter (smart-routed queries), or enable the load_skill tool so the model can pull bodies on demand. Menu rows count as offered — usage panels show rung-labeled counts for them, and activation isn’t measurable for bare prompt-injection usage. If a skill “isn’t working,” check whether its body ever actually reached the model.
The on-demand body loader is opt-in: pass enable_skill_loader=True to install() on the openai_agents or pydantic_ai adapters. On the anthropic adapter there is no tool loop — enable_skill_loader=True there injects the skill menu into system (the offered rung only); add init(inject_skill_body=True) if you want full bodies delivered, and note enable_load_skill_tool is accepted but dormant. Kill-switch: init(load_skill_tool=False) or DECIMALAI_LOAD_SKILL_TOOL=0.
decimalai.init(langchain=True) (and every other framework flag) logs a warning and continues untraced when the framework package isn’t importable — your agent runs, nothing is recorded. The same applies to DECIMAL_AUTO_TRACE (auto-init warns and skips) and to the raw-provider flags (openai=True etc.), which soft-skip when the matching OpenInference instrumentor is absent. Fix: for a framework flag, install the matching extra, e.g. pip install "decimalai[langchain]"; for a raw-provider flag, install the instrumentor package the startup warning names, e.g. pip install openinference-instrumentation-openai (the [openai] extra covers only the provider SDK, not the instrumentor). Then re-check startup logs.
Adapters only inject skills into prompt shapes they can safely rewrite. On openai_agents, an agent whose instructions is a user-supplied callable is left untouched; on langchain, a prebuilt PromptValue (or any unrecognized prompt shape) passes through unchanged. The agent runs normally — with zero skills injected and no error raised.
The regression check needs two things before it can say anything real: a baseline manifest (recorded automatically on the Action’s first run, or by decimalai.init() running in production) and ingested traces to measure blast radius against. A fresh workspace legitimately reports “first run — no baseline” and then near-zero affected traces. That’s honesty, not breakage — reports gain weight over days as production trace volume accumulates.

​
Do it with your own agent

Quickstart

Instrument your agent and get your first real trace in ~5 minutes.

Regression Check on every PR

Wire the GitHub Action so this report appears on your next pull request.

Skills guide

Auto-discover your SKILL.md files and measure them on your traffic.

Run the manual loop in Colab

Prefer code? Build the v1→v2 loop yourself in a live notebook.