Skip to main content
Those tools detect regressions by running your eval suite on a new agent version. That works only if you’ve already written eval cases — most teams haven’t, and the ones they have are usually stale.DecimalAI works differently. Your production traces are tagged with the manifest they ran under. When you propose a manifest change, we identify which traces depended on what’s changing and tell you the structural blast radius — no eval suite required.The tools are complementary. Use both if you want behavioral verification on the eval surface; use DecimalAI alone if you don’t have an eval suite yet. See Why DecimalAI? for a detailed comparison.
No. Pre-deploy regression checks run entirely against our trace store — we don’t run your agent, so we don’t need your LLM credentials.The Playground does need an LLM key when you click “Run”, but it’s BYOK — you paste your own OpenAI, Gemini, or Anthropic key into Settings and it stays there. We never proxy or store outbound LLM traffic on your behalf.
No, and we don’t recommend that. DecimalAI runs alongside LangSmith, Braintrust, Langfuse, Phoenix, etc. Pipe traces into both — we add the manifest layer underneath. The integrations work via OpenTelemetry, so most tools coexist cleanly.The exception: if you’re using one of those tools only for regression detection and not for trace search/eval/etc., you may be able to drop it after adopting DecimalAI’s regression check.
For each trace: agent name, input/output text, LLM call messages (prompts and completions), tool calls (name + arguments + results), token counts, latency, cost estimate, the manifest hash, and any eval scores you push.See Security for retention periods, encryption details, and PII handling.
Not today. The hosted platform — trace store, registry, dashboard, and the verified-benchmark runner — is a cloud service, and self-hosting it isn’t offered yet.What is on your side of the fence: every client that runs on your machines is open source (SDK, CLI, GitHub Action), every artifact is an open format (SKILL.md, eval.yaml, manifests), and everything is exportable — so your files, data, and tooling exit cleanly. Trust & Exit states the full dependency surface plainly. If self-hosting would be decisive for your team, tell us at hello@decimal.ai — demand shapes the roadmap.
The SDK requires Python 3.10+ (requires-python = ">=3.10").On Python 3.9 or older, pip install decimalai silently resolves to an outdated release that predates the demo command and several framework integrations. If decimalai demo reports an unknown command, check python --version first.
Yes — that’s the recommended first step. With just an API key:
Each command seeds realistic demo data into your workspace (prefixed [Demo]) and prints a link straight to the result. decimalai demo reset removes it all. No agent code, no framework setup, no LLM keys.
First-class:
  • LangChain / LangGraph
  • OpenAI Agents SDK
  • Claude Agent SDK / Claude Code
Supported:
  • Pydantic AI
  • Google ADK
  • LlamaIndex (v0.10.20+)
Community (via OTel):
  • AutoGen / AG2
  • CrewAI
What each tier captures is in the capability comparison. Any framework that emits OpenTelemetry GenAI spans works through decimalai.init(otel=True). For custom Python code without a framework, use @decimalai.trace() directly. See Tracing for setup details.
Traces from any provider are accepted — the SDK stores model name + provider as free-form strings. Cost estimation works out of the box for OpenAI, Anthropic, Google, Mistral, and Cohere model families.The Playground supports OpenAI, Gemini, and Anthropic. LLM-judge evaluators run on Gemini with an automatic OpenAI fallback.
Not a tracing SDK yet — the Python SDK is the only first-party instrumentation client. But TypeScript projects aren’t locked out — two paths, both live today:
  • REST API — everything the Python SDK does goes through the same documented REST surface, callable from any language, today. See Authentication.
  • MCP server — decimalai-mcp gives any MCP client (Claude Code, Cursor, Claude Desktop) registry search, full skill detail, and the leaderboard as read-only tools.
A first-party TypeScript SDK is under consideration. If it matters to you, please open an issue on decimal-labs/decimalai-python — usage signals shape priority.
Workspace admins can delete a workspace from Settings → Workspaces — this purges all traces, manifests, datasets, and skills owned by that workspace.For per-user data deletion (GDPR), the dedicated endpoint is on the roadmap. For now, email hello@decimal.ai with the user IDs to purge and we’ll run it manually.
Production runs on Google Cloud Platform (Cloud Run + Cloud SQL for PostgreSQL) in us-central1. Enterprise customers can request a different GCP region; see Security for details.
For 5xx errors or unexpected behavior, include the X-Request-ID response header — it lets us look up the failing request in logs.

​
Skills

A tool is a function the agent can call — search, run SQL, create a ticket. A skill is instructions: a SKILL.md file that tells the agent how to approach a task (your refund policy, a commit-message convention, an extraction schema). Tools execute; skills shape behavior. A skill may reference tools, but it never executes anything itself.See the Skills guide for the format and Skills & Data Pipeline for the conceptual model.
They’re the first axis of the two-axis classification. Capability = the base model genuinely can’t do the task; the skill teaches a missing ability. Preference = the model can do it, but the skill steers the form (your PR format, a citation style). The second axis is scope: public (a standard future models may absorb) vs. private (house knowledge they never will).As a consumer, the combination tells you what to check: only Capability · Public skills expire, so look at their re-verified date; for Preference skills, look at trigger health instead — see choosing skills from the registry.
Lift is the with-vs-without improvement: how much better an agent scores with the skill loaded than without it. A skills ablation is how it’s measured — the same eval suite run twice, once with the skill and once without (skillevaluation is the open spec + runner for this). Lift is the registry’s strongest quality signal, and publishing a skill requires a completed benchmark run.
Every published skill goes through SkillSafety: a deterministic static scan (secrets, remote-code execution, exfiltration, hidden unicode), an AI security review that reads intent, and a content review. The three fold into one public band — Passed / Caution / Blocked / Not yet reviewed. A Blocked skill still appears in browse carrying a Blocked chip, so you can see it exists and why it’s unsafe — but it can’t be installed or forked, never charts or gets recommended, its raw URL 404s, and the runtime refuses to load it. Prohibited-content blocks and takedowns are hidden outright. See How skills are vetted.Honest caveat: a Passed band means nothing known tripped — it’s a layer, not a guarantee. What scanning can’t catch covers the defense-in-depth you should add as a consumer.
Yes — private is the default. A skill you create is visible only inside your org (with personal and workspace visibility for narrower scoping); it becomes public only if you explicitly publish it, which is gated. And when you install a public registry skill, it’s forked into your org as an independent copy — your edits (your policies, your voice) stay yours and never flow upstream. See the registry guide.
Only one quadrant expires: Capability · Public — a skill teaching something future base models will absorb. The platform re-benchmarks these on a monthly cadence; when with ≈ without, the base model has caught up and the skill is surfaced as a retire-candidate (every activation then costs tokens for nothing). Private knowledge and house-style preferences don’t decay this way. See Lifecycle: ablation and retirement.