Quick decision: replace or coexist?
Coexist (recommended for most)
Send the same traces to both tools. DecimalAI adds the manifest layer; your existing tool keeps doing what it does. Switching cost: ~10 minutes.
Migrate fully
Replace your existing tool entirely. Larger change — see the per-tool sections below for what features map cleanly and what doesn’t.
LangSmith → DecimalAI
| LangSmith concept | DecimalAI equivalent | Notes |
|---|---|---|
| Run | Trace | Same atomic unit — input + LLM calls + output. |
| Project | Agent | LangSmith projects are bag-of-runs; DecimalAI agents are named entities with manifest history. Set agent_name in decimalai.init(). |
| Dataset | Dataset | Both formats supported. Pull a LangSmith dataset and pipe rows into POST /api/v1/datasets/{id}/build. |
| Annotation | Eval Score | LangSmith feedback maps to a DecimalAI eval score via source="langsmith". Use decimalai.push_langsmith_scores(...). |
| Evaluator | Evaluator | Both pre-built and custom evaluators supported. The signature is similar — see Evaluations. |
| Regression test | Experiment | LangSmith uses run-on-eval-set for regression; DecimalAI uses manifest impact for pre-deploy + experiments for post-deploy verification. |
Braintrust → DecimalAI
| Braintrust concept | DecimalAI equivalent | Notes |
|---|---|---|
| Span / Experiment Run | Trace | Identical model. |
| Project | Agent | Same as LangSmith. |
| Eval function | Custom evaluator | Wrap your Braintrust scorer in @decimalai.eval(). |
| Dataset | Dataset | Use decimalai.pull_dataset() to round-trip. |
| Regression check | Manifest impact + Experiment | Braintrust runs evals on a new model version; DecimalAI’s regression check is structural (no eval suite needed) — add experiments on top for behavioral verification. |
Langfuse → DecimalAI
| Langfuse concept | DecimalAI equivalent | Notes |
|---|---|---|
| Trace | Trace | Same model. |
| Observation | Span or LLM Call | Langfuse “observations” map to DecimalAI spans (span_type=tool/retriever/other) or LLM calls (span_type=llm). |
| Generation | LLM Call | Direct mapping with full fidelity. |
| Score | Eval Score | Push via decimalai.push_custom_scores(...). |
| Session | Session | Same model; share session_id across traces. |
| Prompt management | (Not built-in) | DecimalAI tracks prompts as manifest components but doesn’t have prompt-management UI. Continue using Langfuse for prompt registries. |
Phoenix / Arize → DecimalAI
Both Phoenix and Arize emit OpenTelemetry GenAI spans. DecimalAI consumes OTel directly:| Phoenix concept | DecimalAI equivalent |
|---|---|
| Span | Span (1:1) |
| Dataset | Dataset |
| Annotation | Eval Score |
Bulk import of historical traces
If you have a JSONL backup from any of the above tools, use either the CLI or the REST API:- Count at half rate against your
traces_ingestedquota - Get tagged with
source_type="imported" - Are deduplicated by
trace_id— re-running is safe
What doesn’t migrate cleanly
Be honest about gaps:- Prompt registries. DecimalAI tracks prompts inside manifests but isn’t a dedicated prompt-management tool. If you use LangSmith Prompts or Langfuse Prompt Management for non-engineers, keep using it.
- Tracing UI feature parity. Our trace viewer covers the common case (timeline tree + LLM call inspection) but doesn’t yet have feature parity with mature observability tools for things like service-level dependency graphs or distributed tracing across non-LLM services.
- Pre-existing eval datasets. Datasets transfer (we accept JSONL of any shape), but the eval definitions don’t — you’ll re-implement custom evaluators.