> ## Documentation Index
> Fetch the complete documentation index at: https://docs.decimal.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Why DecimalAI?

> How DecimalAI compares to LangSmith, Braintrust, Langfuse — and why manifest-aware agent change management is a different category.

Every observability platform can trace your agent and run evaluations. DecimalAI does something none of them do: **it tracks your agent's structural identity — and uses that to catch regressions, measure skills, and keep training data valid.**

We call this **manifest-aware agent change management.** It's a different category from observability.

## The Problem Every Team Hits

You ship an agent. It has tools, prompts, a model, maybe some skills. Then a developer:

1. Renames `search_docs` → `search_knowledge_base` (tool registry change)
2. Updates the system prompt to include a new persona (prompt stack change)
3. Removes `compare_competitors` (tool removal)

**Three things break. No other platform catches all three.**

| Platform      | Regressions?                 | Skills affected?                          | Training data stale?         |
| ------------- | ---------------------------- | ----------------------------------------- | ---------------------------- |
| LangSmith     | ⚠️ "Run your evals"          | ❌                                         | ❌                            |
| Braintrust    | ⚠️ "Run your evals"          | ❌                                         | ❌                            |
| Langfuse      | ⚠️ Visible in traces, manual | ❌                                         | ❌                            |
| **DecimalAI** | **✅ 247 traces HIGH IMPACT** | **✅ 3 skills reference the removed tool** | **✅ 1,800 traces are stale** |

<Note>Illustrative figures. Actual counts depend on your trace volume and how the change touches each surface. See the [canonical Impact Report example](#the-roi-of-manifest-awareness) for a representative breakdown.</Note>

## The Structural Differentiator: Manifest-Aware Detection

Other tools detect regressions by running your **eval suite**. That requires writing eval cases (most teams haven't), maintaining them (they go stale silently), running the agent in CI (slow, costly, non-deterministic), and paying for LLM-graded judgment.

DecimalAI detects regressions by **diffing the manifest and querying the trace store.** No eval cases. No agent execution. No LLM API keys. Cost per check: \<\$0.001.

|                                    | Eval-driven (other tools)       | Manifest-aware (DecimalAI)                                    |
| ---------------------------------- | ------------------------------- | ------------------------------------------------------------- |
| Requires writing eval cases        | ✅ Yes                           | ❌ No — production traffic is the test set                     |
| Requires running your agent in CI  | ✅ Yes                           | ❌ No — pure database query                                    |
| Knows the blast radius of a change | ❌ Runs all evals every time     | ✅ Identifies exactly which traces touched the changed surface |
| Catches removed-tool regressions   | ⚠️ Only if eval coverage exists | ✅ Structurally                                                |
| Cost per check                     | \$\$ (LLM-graded evals)         | \<\$0.001                                                     |

### Regression testing, reframed

"Regression testing" usually means *eval-based* regression testing: keep a golden set of cases, re-grade them after every change, and watch the scores. It answers **"did quality drop?"** — but only for the cases you thought to write, only after you run the agent, and only if the eval set is still current.

DecimalAI's regression check answers a different, earlier question: **"what did this change structurally touch, and which production traces are affected?"** It runs on the manifest diff *before* deploy, with no agent execution and no graded judgment. Severity is reported as **HIGH / MEDIUM / LOW IMPACT** per trace — a representative diff lands at **247 HIGH / 501 MEDIUM / 1,254 LOW across 2,002 traces**.

The two are complementary, not competing. Eval-based testing measures *quality* on a curated set; manifest-aware regression measures *blast radius* on real traffic. Most teams have the second gap, not the first — which is why DecimalAI leads with it.

<Note>IMPACT (HIGH / MEDIUM / LOW) answers "was this trace structurally touched?" — a separate axis from the compatibility verdict (keep / repair / flag / replay / drop), which answers "what should I do with the trace for training?"</Note>

## Feature Comparison

| Capability                             | DecimalAI             | LangSmith                       | Braintrust | Langfuse                        |
| -------------------------------------- | --------------------- | ------------------------------- | ---------- | ------------------------------- |
| Trace collection                       | ✅                     | ✅                               | ✅          | ✅                               |
| LLM evaluations                        | ✅                     | ✅                               | ✅          | ✅                               |
| Prompt playground                      | ✅ BYOK                | ✅                               | ✅          | ✅                               |
| Datasets / fine-tuning                 | ✅                     | ✅                               | ✅          | Partial                         |
| **Manifest versioning**                | ✅ Auto-detect         | ❌                               | ❌          | ❌                               |
| **Pre-deploy regression check (CI)**   | ✅ GitHub Action       | ❌                               | ❌          | ❌                               |
| **Compatibility scoring**              | ✅ Per-trace           | ❌                               | ❌          | ❌                               |
| **Mechanical trace repair**            | ✅ Zero LLM cost       | ❌                               | ❌          | ❌                               |
| **Skills effectiveness tracking**      | ✅ Pass rates + trends | ❌ (Hub stores, doesn't measure) | ❌          | ❌ (Prompts, no activation data) |
| **Performance-weighted skill routing** | ✅ Self-improving      | ❌                               | ❌          | ❌                               |
| **Session-aware replay**               | ✅ DPO pairs           | ❌                               | ❌          | ❌                               |
| **Multi-agent topology**               | ✅ Drift detection     | ❌                               | ❌          | ❌                               |
| Self-hostable                          | ✅                     | ❌ (cloud only)                  | ❌          | ✅                               |
| Pricing                                | Free tier + usage     | Per-trace                       | Per-eval   | Free tier                       |

## Where Each Tool Excels

Every tool here is good at something. The honest read:

| Tool           | Best for                                                                                                                                                 | Weakest at                                                                              |
| -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
| **LangSmith**  | Teams deeply embedded in the LangChain ecosystem; strong hub for prompt management and LangGraph tracing                                                 | Manifest-aware regression detection, skill effectiveness measurement, dataset lifecycle |
| **Braintrust** | Prompt evaluation and A/B testing; strong scoring framework with human-in-the-loop grading                                                               | Production tracing depth, manifest tracking, skill observability                        |
| **Langfuse**   | Self-hosted, open-source observability; excellent community and integrations                                                                             | Regression detection, training-data management, automated change workflows              |
| **DecimalAI**  | Knowing **what your agent is** (its manifest), not just what it does — and using that to catch regressions, measure skills, and keep training data valid | Out-of-the-box prompt-hub breadth; deep LangChain-specific tooling                      |

DecimalAI's edge comes from the structural identity layer:

* **Catch regressions** without writing eval cases (GitHub Action on every PR)
* **Measure skills** with production effectiveness data (pass rates, activation trends)
* **Keep training data valid** when the agent changes (auto-classify + repair)

## The ROI of Manifest Awareness

The clearest proof is the artifact itself. When a manifest change lands, DecimalAI produces an **Impact Report** — every affected trace bucketed by IMPACT severity, with a per-trace compatibility verdict (keep / repair / flag / replay / drop):

### Scenario: 10-Agent Production System

<Note>Illustrative figures for a representative team — your numbers will vary with update cadence, trace volume, and how often changes touch high-traffic surfaces.</Note>

| Metric                              | Without DecimalAI           | With DecimalAI          |
| ----------------------------------- | --------------------------- | ----------------------- |
| Agent updates per month             | 15                          | 15                      |
| Regression check method             | Write + maintain eval cases | Automated manifest diff |
| Time to detect regressions          | Hours (after deploy)        | Seconds (before deploy) |
| Manual audit time per update        | 4–8 hours                   | 0 (automated)           |
| Stale traces in training data       | Unknown (est. 20-40%)       | 0%                      |
| Fine-tune quality regression rate   | \~25% after agent changes   | \<5%                    |
| **Monthly engineering hours saved** | —                           | **60-120 hours**        |

## When to Use DecimalAI

DecimalAI is the right choice if:

<Check>You ship agent changes regularly and want to catch regressions before deploy</Check>
<Check>You use skills/instructions and want to know which ones actually work</Check>
<Check>You fine-tune models and need version-aware training data</Check>
<Check>Your agent's tools, prompts, or models change frequently</Check>
<Check>You run multi-agent systems and need to track cross-agent drift</Check>

DecimalAI may not be the best fit if:

* You only need basic LLM tracing with no change management workflow → Consider Langfuse
* You're exclusively in the LangChain ecosystem and need hub features → Consider LangSmith
* You only need evaluation scoring without production tracing → Consider Braintrust

## Getting Started

Pick the path that matches your immediate need:

<CardGroup cols={3}>
  <Card title="Catch regressions" icon="shield-check" href="/guides/regression-check">
    Most common entry point. Manifest impact analysis on every PR — no eval cases required.
  </Card>

  <Card title="Track skills" icon="puzzle-piece" href="/guides/skills">
    Effectiveness analytics, smart routing, public registry with SkillScore.
  </Card>

  <Card title="Build training data" icon="database" href="/guides/datasets">
    Versioned SFT datasets that stay valid as the agent evolves.
  </Card>
</CardGroup>

If you're migrating from a different tool, the [migrations guide](/migrations) maps concepts side-by-side.