> ## Documentation Index
> Fetch the complete documentation index at: https://docs.decimal.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Catch agent regressions before they ship

> The manifest-aware platform for agent change management — catch regressions before deploy, share proven skills, and keep your training data valid as your agent evolves.

Every agent team ships changes weekly. Every change risks breaking something. Most teams find out only after their users do.

**DecimalAI tracks your agent's structural identity — tools, prompts, models, skills — as a versioned manifest, then uses your production traffic as the test set when you propose a change.** You don't write eval cases. You don't hand over your API keys. We don't run your agent.

```mermaid theme={null}
%%{init: {'theme':'base','themeVariables':{'primaryColor':'#f5f5f4','primaryBorderColor':'#a8a29e','primaryTextColor':'#44403c','lineColor':'#a8a29e'}}}%%
flowchart LR
    A[Agent change<br/>opened as a PR] --> B[Manifest diff<br/>tools · prompts · models · skills]
    B --> C[Query your<br/>production traces]
    C --> D{Impact report}
    D --> E[HIGH<br/>will break]
    D --> F[MEDIUM<br/>may differ]
    D --> G[LOW<br/>unaffected]
    classDef high fill:#fee2e2,stroke:#ef4444,color:#991b1b;
    classDef med fill:#fef9c3,stroke:#eab308,color:#854d0e;
    classDef low fill:#dcfce7,stroke:#16a34a,color:#166534;
    class E high
    class F med
    class G low
```

Here's what the check posts on your next PR — structural impact, computed before the change deploys:

<Columns cols={2}>
  ```
  🔍 Decimal Manifest Impact — support-agent

    🔴 HIGH IMPACT — 247 traces will break (called the removed `compare_competitors` tool)
    🟡 MEDIUM IMPACT — 501 traces may behave differently
    🟢 LOW IMPACT — 1,254 traces unaffected

    Verdict: Review before merging.
  ```

  ```mermaid theme={null}
  %%{init: {'theme':'base','themeVariables':{'primaryColor':'#f5f5f4','primaryBorderColor':'#a8a29e','primaryTextColor':'#44403c','lineColor':'#a8a29e'}}}%%
  pie showData
      title Impact across 2,002 traces
      "HIGH IMPACT" : 247
      "MEDIUM IMPACT" : 501
      "LOW IMPACT" : 1254
  ```
</Columns>

<Tip>
  **See it in action:** Follow the [2-Minute Demo](/tutorials/two-minute-demo).
</Tip>

<Card title="Run this in Colab" icon="play" horizontal href="https://colab.research.google.com/github/decimal-labs/decimalai-python/blob/main/examples/quickstart/quickstart.ipynb">
  Live notebook, no setup — just paste your API key.
</Card>

## Try either demo in 2 minutes

Two of the three capabilities ship with a one-command demo — seeded realistic data in your workspace, so the payoff lands before you instrument your own agent.

```bash theme={null}
pip install decimalai
export DECIMAL_API_KEY="dai_sk_..."   # from app.decimal.ai/settings
```

<CardGroup cols={2}>
  <Card title="For engineers — catch regressions" icon="shield-check">
    ```bash theme={null}
    decimalai demo regression
    ```

    Links straight to the impact report: which production traces your next change would break.
  </Card>

  <Card title="For prompt engineers — find skills that work" icon="puzzle-piece">
    ```bash theme={null}
    decimalai demo skills
    ```

    Links to the registry ranked by real production effectiveness — or [browse it now, no signup](https://app.decimal.ai/skills).
  </Card>
</CardGroup>

## What you can do with DecimalAI

Three capabilities, one foundation. Regression checks, the skills registry, and training-data validation each stand alone — but they compound on one thing: **manifest-aware versioning**, the structural fingerprint of your agent.

<CardGroup cols={3}>
  <Card title="Catch regressions before you ship" icon="shield-check" href="/guides/regression-check">
    Open a PR with an agent change; get a structural impact report — which production traces will break, may differ, or are unaffected — before it deploys.
  </Card>

  <Card title="Discover & share proven skills" icon="puzzle-piece" href="/guides/skills">
    Install skills from the registry ranked by real production-effectiveness data, then measure how they perform on your own traffic.
  </Card>

  <Card title="Keep training data valid" icon="database" href="/guides/datasets">
    As your agent evolves, traces are auto-classified keep / repair / replay / drop against the manifest diff — so your training set stays clean.
  </Card>
</CardGroup>

## Why "manifest-aware" matters

Other observability platforms (LangSmith, Braintrust, Langfuse, Weave) detect regressions by running your **eval suite** on the new version. That works only if you've written eval cases — which most teams haven't, and the ones they have are usually stale.

DecimalAI works differently. Your production traces are tagged with the manifest they ran under. When you propose a manifest change, we identify which traces depended on what's changing and tell you the structural blast radius — no eval suite required.

|                                    | Eval-driven (other tools)                   | Manifest-aware (DecimalAI)                                          |
| ---------------------------------- | ------------------------------------------- | ------------------------------------------------------------------- |
| Requires writing eval cases        | ✅ Yes                                       | ❌ No — production traffic IS the test set                           |
| Requires running your agent in CI  | ✅ Yes                                       | ❌ No — pure trace-store query                                       |
| Knows the blast radius of a change | ❌ No                                        | ✅ Yes — identifies exactly which traces touched the changed surface |
| Catches removed-tool regressions   | ⚠️ Only if eval coverage exists             | ✅ Yes, structurally                                                 |
| Cost per check                     | \$\$ (full eval suite, possibly LLM-graded) | \<\$0.001 (database query)                                          |

## What DecimalAI does NOT do (yet)

We're honest about the boundaries:

| Boundary                                   | What we do today                                                                                                                                                                                                             | Deferred / how to cover                                                                                                                                                                                                             |
| ------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Running your full agent in CI**          | Pre-deploy regression detection is structural impact analysis — we tell you what's at risk based on the manifest diff, without executing your agent.                                                                         | Full end-to-end agent replay is deferred. For changes where structure can only say "everything may be affected" (large prompt rewrites), use a careful canary deploy and our [post-deploy bisect](/guides/post-deploy-bisect) view. |
| **Behavioral verification of model swaps** | Model-swap call-replay ships now (preview): for each affected trace we re-issue one recorded model call against the candidate model and diff the outputs. Defaults to mock (no spend); `mode=real` does same-provider swaps. | Cross-provider swaps and replaying the full agent (not just a single recorded call) are deferred.                                                                                                                                   |
| **Holding your LLM API keys**              | Pre-deploy structural analysis runs entirely against our trace store — no keys required.                                                                                                                                     | `mode=real` call-replay needs a key for the same provider; mock replay needs none.                                                                                                                                                  |
| **Replacing your observability tool**      | DecimalAI runs alongside LangSmith, Braintrust, Langfuse, etc. Pipe your traces in via the SDK; we add the manifest layer underneath.                                                                                        | Not a goal — keep your existing tracing in place.                                                                                                                                                                                   |

## Supported Frameworks

| Framework                 | Integration               | Manifest Fidelity               |
| ------------------------- | ------------------------- | ------------------------------- |
| **LangChain / LangGraph** | `decimalai.langchain`     | Full (tools, models, prompts)   |
| **OpenAI Agents SDK**     | `decimalai.openai_agents` | Full (with Agent introspection) |
| **LlamaIndex**            | `decimalai.llamaindex`    | Full (span-based)               |
| **AutoGen / AG2**         | `decimalai.autogen`       | Via OTEL                        |
| **CrewAI, Haystack, ADK** | `decimalai.otel`          | Model + tool names              |
| **Any Python agent**      | `@decimalai.trace`        | Configurable                    |

## Next Steps

### Start here

<CardGroup cols={2}>
  <Card title="Quickstart" icon="rocket" href="/quickstart">
    Install the SDK, get your first trace, and add the GitHub Action in under 10 minutes.
  </Card>

  <Card title="2-Minute Demo" icon="play" href="/tutorials/two-minute-demo">
    One command each: a live impact report and the ranked skills registry, on seeded data.
  </Card>
</CardGroup>

### By role

<CardGroup cols={3}>
  <Card title="Engineering teams" icon="shield-check" href="/for-engineers">
    Catch agent regressions on every PR with manifest-aware impact analysis.
  </Card>

  <Card title="Prompt engineers" icon="puzzle-piece" href="/guides/skills">
    Track skills, browse the registry, and measure effectiveness with production data.
  </Card>

  <Card title="ML teams" icon="database" href="/guides/datasets">
    Build versioned SFT datasets that stay valid as your agent evolves.
  </Card>
</CardGroup>

### Reference

<CardGroup cols={2}>
  <Card title="Concepts" icon="book-open" href="/concepts">
    Every term and system explained, with diagrams.
  </Card>

  <Card title="Manifests Guide" icon="layer-group" href="/guides/manifests">
    How automatic version tracking and compatibility scoring work.
  </Card>

  <Card title="Why DecimalAI?" icon="circle-question" href="/guides/why-decimalai">
    Detailed comparison with LangSmith, Braintrust, Langfuse.
  </Card>

  <Card title="API Reference" icon="code" href="/api-reference/authentication">
    Every REST endpoint with examples and schemas.
  </Card>
</CardGroup>