Skip to main content
DecimalAI automatically tracks changes to your agent’s configuration across deployments. When a change is detected, a new manifest version is registered and a compatibility analysis shows how existing traces are affected.

What Is a Manifest?

A manifest is a snapshot of your agent’s contract — the tools, model, prompts, subagents, and output schema that define how your agent behaves:
Manifest v3
├── Tools: search, calculator, web_browser
├── Model: gpt-4o (temperature: 0.7)
├── Prompts: "You are a helpful research assistant..."
├── Subagents: fact-checker, summarizer
└── Skills: code-review, sql-optimizer
Every time your agent runs, the SDK extracts this contract, hashes it, and compares to the previous version. If something changed, a new version is registered automatically.

How Auto-Detection Works

Detection is framework-specific. Each integration extracts the maximum available information:
FrameworkInit flagToolsModelPromptsSchema Depth
LangChainlangchain=True✅ Auto + full schema✅ Full config✅ System/userFull
OpenAI Agents (install(agent=...))openai_agents=True + explicit install✅ Full JSON schema✅ Name✅ InstructionsFull
OpenAI Agents (auto)openai_agents=TrueNames only✅ Full configNames
LlamaIndexllamaindex=TrueNames only (OTel)✅ Full configNames
CrewAIcrewai=TrueNames only (OTel)✅ Full configNames
AutoGen / AG2autogen=TrueNames only (OTel)✅ Full configNames
Generic OTelotel=TrueNames only✅ Full configNames
Generic + @tool decorator(no flag)✅ From type hints✅ FullFull
Explicit register_manifest()(no flag)✅ Whatever you passFull
This is the canonical capability matrix. The Quickstart shows code for each framework but doesn’t repeat the matrix — come back here if you want to know how deep auto-detection goes for your framework before committing to it.
Two calling forms — when to use which:
  • Flag form (decimalai.init(api_key=..., langchain=True)) auto-installs the integration. Use this by default — it’s what the Quickstart shows.
  • Explicit install(...) form is for when you need to pass advanced arguments — e.g., an Agent object for OpenAI Agents to capture full schemas, or prompts={...} to override dynamic prompts in LangChain. Call decimalai.init(api_key=...) first, then install(...).
Don’t pass the framework flag and call install() separately — pick one form per integration.
import decimalai

# Default — flag form auto-installs the integration
decimalai.init(api_key="your-api-key", langchain=True)

# Use LangChain as normal — everything is auto-detected
Tools are extracted from callbacks, model config from on_chat_model_start, and prompts from system messages.
Gotcha: Dynamic prompts (RAG chunks, today’s date) cause false drift. Drop into the explicit form to override:
decimalai.init(api_key="your-api-key")
from decimalai.langchain import install
install(prompts={"system": "Your static template"})

Manifest Hashing

Manifest hashing is deterministic and noise-resistant:
RuleWhat it doesEffect on drift
Per-surface hashingTools, prompts, models, subagents, skills, and output schema each get their own hash; the overall hash is the hash of all surface hashes sorted alphabeticallyIsolates where drift came from
Temperature quantizationSmall changes (0.70 → 0.71) are quantized to the nearest 0.1No drift for tweaks; 0.7 → 0.8 will trigger a new manifest
Runtime settings excludedmax_retries, timeout, and operational settings are stripped before hashingThese never cause drift
Tool order invariantTools are sorted by name before hashingReordering doesn’t create false drift
Hash dedupIf the same hash already exists, no new version is createdDeploying the same config to multiple pods won’t create duplicates

Compatibility Scoring

When a new manifest is detected, DecimalAI classifies every existing trace against the new config:
VerdictMeaningExample
KeepTrace is fully compatibleModel temperature changed slightly
RepairTrace can be adapted mechanicallyTool schema added an optional field
ReplayTrace should be re-run with the new agentSystem prompt changed significantly
DropTrace is incompatible — exclude from training dataTool was removed entirely
The classification considers 10 surfaces, each with its own severity:
SurfaceMinor (keep)Moderate (flag)Major (drop)
prompt_stackWhitespace changeContent changedComplete rewrite
model_runtimeTemperature tweakModel changedProvider changed
tool_registrySchema field addedSchema field changedTool removed
skill_registrySkill addedSkill content changedSkill removed
workflowParameter changeHandoff addedStep removed
subagentsSub-agent addedSub-agent reconfiguredSub-agent removed
output_contractOptional field addedField type changedRequired field removed
guardrailsThreshold relaxedRule changedRule removed
context_configWindow grewRetrieval source changedSource removed
environmentNon-semantic var addedValue changedVar removed

Repair: Fix Traces Mechanically

Traces classified as repair can be fixed with a deterministic data migration — no LLM calls, no re-running your agent. The platform rewrites tool call data inside the trace records so they conform to the new manifest.

What Repair Does

Rule TypeWhat It FixesExample
tool_renameRewrites tool name in trace recordscheck_inventory(...)lookup_stock(...)
param_renameRenames parameter in tool call argsproduct_iditem_id
param_removeStrips deprecated parameterRemoves legacy_flag from args
param_add_defaultAdds new optional parameterAdds region: null
After repair, the trace’s manifest_id is updated to point to the new manifest — it’s now “compatible” with v2.

Repair Flow

1

Preview

See what rules would be applied, with before/after examples:
curl -X POST https://api.decimal.ai/api/v1/repair/preview \
  -H "Authorization: Bearer $API_KEY" \
  -d '{"old_manifest_id": "...", "new_manifest_id": "...", "sample_size": 5}'
Response shows each rule with confidence level and sample trace diffs.
2

Approve

Review the rules. You can apply all rules or select specific ones:
  • Apply All — repairs every eligible trace with all generated rules
  • Selective — pick which rules to apply (e.g., approve the tool rename but skip the param removal)
3

Apply

# Apply all rules
curl -X POST https://api.decimal.ai/api/v1/repair/apply \
  -H "Authorization: Bearer $API_KEY" \
  -d '{"old_manifest_id": "...", "new_manifest_id": "..."}'

# Or apply selectively
curl -X POST https://api.decimal.ai/api/v1/repair/apply-selective \
  -H "Authorization: Bearer $API_KEY" \
  -d '{"old_manifest_id": "...", "new_manifest_id": "...", "approved_rule_indices": [0, 2]}'
Returns a batch ID with repaired/failed/skipped counts.

Dashboard Quickstart

In the dashboard, the Impact Report banner appears automatically when your agent has 2+ manifest versions. From the banner you can:
  1. Click “Repair All” to preview repair rules (before/after examples for each affected trace)
  2. Review the rules, then click “Apply Repairs” to execute
  3. Or click “Repair & Build” for a combined flow: repair all eligible traces, then build a clean SFT dataset from keep + repaired traces — one click, end to end

REST API

import httpx

base = "https://api.decimal.ai/api/v1"
headers = {"Authorization": "Bearer dai_sk_..."}

# Preview repair rules — returns the proposed transforms with sample diffs
rules = httpx.post(
    f"{base}/repair/preview",
    headers=headers,
    json={"old_manifest_id": "...", "new_manifest_id": "...", "sample_size": 5},
).json()

# Apply all repairs
result = httpx.post(
    f"{base}/repair/apply",
    headers=headers,
    json={"old_manifest_id": "...", "new_manifest_id": "..."},
).json()

# Apply selectively (approve specific rule indices only)
result = httpx.post(
    f"{base}/repair/apply-selective",
    headers=headers,
    json={
        "old_manifest_id": "...",
        "new_manifest_id": "...",
        "approved_rule_indices": [0, 2],
    },
).json()
Repair vs Replay: Repair is instant (deterministic DB rewrite, zero LLM cost). Replay re-runs the full agent on original prompts (minutes/hours, full LLM cost). Use repair for schema changes; use replay when the agent’s behavior changed.

Compatibility Policies

How aggressively traces are classified when a manifest changes (strict / default / permissive presets, per-surface overrides, distillation mode, impact preview) is covered in its own guide:

Compatibility Policies

Preset reference, per-surface overrides, impact preview before applying, and the distillation override for fine-tuning workflows.

What Happens When a Manifest Changes

1

New version is registered

Auto-incremented (v1 → v2 → v3).
2

Compatibility report is generated

Each trace is classified as keep/repair/replay/drop.
3

Dashboard shows Impact Report

Visual breakdown with progress bars and action buttons.
4

Sidebar shows notification badge

Pulsing dot alerts you to drift.
5

Toast notification appears

Slide-in alert with “View Impact →” link.
If you roll back to a previous configuration, DecimalAI detects the matching hash and re-activates the old manifest instead of creating a duplicate. Your version history stays clean.

The Manifest Timeline

The agent dashboard shows a timeline of all manifest versions with:
  • Version label (v1, v2, v3, or custom labels)
  • Date registered
  • Diff view — which surfaces changed between versions
  • Trace count — how many traces were recorded under each version
  • Compatibility summary — keep/repair/replay/drop counts
Click any two versions to see a side-by-side diff highlighting exactly what changed: tools added/removed, prompt text differences, model configuration changes.

When Manifests Are Registered

FrameworkTiming
LangChainOn first trace completion (lazy)
OpenAI Agents (with Agent)Immediately at install() time
OpenAI Agents (without Agent)On first trace completion
OTelOn first trace batch export
Generic (explicit)When register_manifest() is called
Generic (auto-deduce)On first trace completion

Best Practices

  1. Use static prompts. Dynamic content (RAG chunks, dates) in system prompts causes false drift. Pass dynamic content as user messages instead.
  2. Use @decimalai.tool for full schema tracking. The decorator generates JSON schemas from Python type hints — enabling schema-level drift detection, not just name-level.
  3. Start with the default policy. The balanced preset works for most teams. Switch to strict when you need training data purity guarantees.
  4. Label important versions. Use version_label="v2.1-prod" when deploying significant changes so you can find them in the timeline.

Next Steps

Regression Check

Get manifest impact analysis as a PR comment on every change.

Manifests API

REST reference for register, list, get, diff.

Versioning & Compatibility

Conceptual model — manifest hash, component verdict, severity, repair.

Compatibility Policies

Tune how the engine maps severity to verdicts.

Troubleshooting

False drift, or the action says “no manifest”? Common fixes.