Skip to main content
DecimalAI captures full execution traces from your AI agents — including LLM calls, tool usage, token counts, latency, and errors. Traces are the foundation for evaluation, dataset building, and manifest tracking.

​
Supported Frameworks

DecimalAI provides first-class integration with the most popular agent frameworks. Pick yours and add two lines of code:
FrameworkFlagHow it works
LangChain / LangGraphlangchain=TrueNative callback handler
OpenAI Agents SDKopenai_agents=TrueNative tracing hook
LlamaIndexllamaindex=TrueNative SpanHandler on LlamaIndex’s dispatcher
CrewAIcrewai=TrueOpenTelemetry (CrewAI emits standard GenAI spans)
AutoGen / AG2autogen=TrueOpenTelemetry (AutoGen emits standard GenAI spans)
Any OTEL frameworkotel=TrueGeneric OpenTelemetry span exporter
Any Python function@decimalai.trace()Decorator-based manual tracing
CrewAI and AutoGen use the OpenTelemetry GenAI semantic conventions under the hood. The crewai=True and autogen=True flags are the same as otel=True — they’re convenience aliases so you don’t need to know what “OTEL” means.

​
Quick Setup

import decimalai

# Pick your framework — only one flag needed
decimalai.init(
    api_key="dai_sk_...",
    langchain=True,       # or openai_agents, llamaindex, crewai, autogen
)

# Run your agent as normal — traces are captured automatically

​
Zero-Code Setup (Environment Variables)

You can instrument without changing any code: 
export DECIMAL_API_KEY="dai_sk_..."
export DECIMAL_AUTO_TRACE=langchain    # or openai-agents, llamaindex, crewai, autogen

python my_agent.py

​
What Gets Captured

Each trace contains:
  • User input — the query or task that triggered the agent
  • Final output — the agent’s response
  • LLM calls — model name, provider, prompt/completion tokens, latency, temperature
  • Tool calls — function name, arguments, results
  • Spans — hierarchical execution steps (retrieval, chain, agent, etc.)
  • Status — success or error, with error messages
  • Timestamps — start/end times for precise latency measurement
  • Active skills — which skills were engaged during execution (auto-detected)

​
Trace Schema

Every trace is represented as a RunTrace with these core fields:
FieldTypeDescription
idUUIDUnique trace identifier
agent_namestringThe agent that produced this trace
statussuccess / errorOverall execution status
user_input_previewstringTruncated user input
final_output_previewstringTruncated agent output
llm_callsLlmCallRecord[]All LLM API calls made
spansTraceSpan[]Hierarchical execution spans
started_at / ended_atdatetimeTrace timing
source_typestringproduction, replay, experiment
manifest_idUUIDThe manifest version active when this trace was recorded
active_skillsstring[]Skills detected as active in this trace

​
LlmCallRecord

Each LLM call within a trace records:
FieldTypeDescription
model_namestringe.g., gpt-4o, claude-sonnet-4-6, gemini-2.5-pro
providerstringopenai, anthropic, google, etc.
input_tokensintPrompt tokens
output_tokensintCompletion tokens
latency_msintWall-clock time for this call
tool_calls_jsonlistTool calls made by the model
finish_reasonstringstop, tool_calls, length, etc.

​
Manual Tracing

For custom frameworks or non-LLM workflows, use the @decimalai.trace() decorator: 
import decimalai

decimalai.init(api_key="dai_sk_...")

@decimalai.trace(agent_name="my-custom-agent")
def process_request(query: str):
    # Your custom logic here
    result = call_llm(query)
    return result

process_request("What is the weather?")

​
Viewing Traces

Once traces are flowing, view them in the dashboard:
  1. Traces page (/traces) — all traces across all agents
  2. Agent dashboard → Traces tab — filtered to a specific agent
  3. Trace detail — click any trace to see the full execution tree with spans, LLM calls, and tool invocations

​
Search & Filtering

The Traces page supports filtering by:
FilterExample
Agentsupport-agent, research-bot
Statussuccess, error
Date rangeLast 24h, last 7 days, custom
Manifest versionv1, v2, v3
Eval verdictpass, fail, review

​
Cost Tracking

DecimalAI estimates the cost of each trace based on the LLM calls made. Costs are calculated from model pricing tables and displayed per-trace and aggregated on the dashboard. This helps you:
  • Identify expensive queries
  • Compare cost across agent versions
  • Set budgets for evaluation and auto-scoring

​
Playground

From any trace detail page, click the “Open in Playground” button on an LLM call to debug and iterate on prompts using real production data. The Playground opens with:
  • System message pre-populated from the original call (editable)
  • User message pre-populated (editable)
  • Model and temperature pre-filled from the original call
  • Original output shown on the left for comparison
Edit the prompt, click “Run”, and the new output appears side-by-side with the original. This lets you:
  • Test prompt variations without re-running the full agent
  • Compare outputs across different model parameters
  • Debug specific LLM calls in isolation
Prerequisite: Configure your LLM API key in Settings → Credentials first. The Playground uses your own key (BYOK) — DecimalAI does not subsidize LLM calls. If no key is configured, you’ll see a prompt to add one.
You can also test skill changes in the Playground — see Testing Skill Changes.

​
Next Steps

Evaluations

Auto-score traces with built-in checks or LLM-as-judge.

Manifests

Detect agent configuration changes automatically.

Datasets

Build training data from your best traces.

Troubleshooting

Traces not showing up, or missing a verdict? Common fixes.