CLI - DecimalAI

The DecimalAI CLI provides command-line access to your workspace.

This page covers the most-used commands. The CLI also ships an evaluators group, the full skills lifecycle (status, pull, install, benchmark, push), compat-check, and regression-check (the CI command — see Regression Check). Run decimalai --help or decimalai <group> --help for the complete list.

Command overview

Command	Group	What it does
`init`	Setup	Verify your API key, test connectivity, and send a test trace.
`demo regression`	Demo	Seed the “Your agent changed” demo and run the regression check.
`demo skills`	Demo	Seed the “Find skills that work” demo and recompute registry stats.
`demo reset`	Demo	Remove all `[Demo]` -prefixed data from the workspace.
`traces list`	Traces	List recent traces.
`traces show`	Traces	Show full detail for one trace as JSON.
`traces stats`	Traces	Show trace statistics for the workspace.
`traces import`	Traces	Import traces from a JSON or JSONL file.
`eval push`	Evaluations	Push named evaluation scores to a trace.
`skills list`	Skills	List skills in the workspace.
`skills sync`	Skills	Sync local `SKILL.md` files to the platform.
`skills pull`	Skills	Read-only download of a public skill (no fork).
`skills install`	Skills	Fork a registry skill into your workspace and write it to disk.
`skills push` / `benchmark` / `status`	Skills	Full publish + effectiveness lifecycle.
`manifests list`	Manifests	List manifests for an agent.
`datasets list`	Datasets	List datasets with row counts and version info.
`datasets show`	Datasets	Show version history for a dataset.
`datasets pull`	Datasets	Pull a dataset version to a local file.
`datasets export`	Datasets	Export a dataset version to stdout or a file.
`datasets build`	Datasets	Build a new dataset version from traces.
`datasets push-to-hub`	Datasets	Push a dataset to HuggingFace Hub.
`replay run`	Replay	Execute a replay batch through your updated agent.
`evaluators ...`	Evaluators	Manage custom evaluators.
`compat-check`	Compatibility	Diff a candidate manifest against a baseline.
`regression-check`	CI	The CI command — see Regression Check.

skills export landed in 0.10.2 — write a skill in your workspace to disk without taking a fork — and skills install is deprecated in its favour. On 0.10.0 and earlier, skills install was the only way registry-skill files reached disk (or skills pull for a keyless, read-only copy). skills install still works and still forks first.

Installation

The CLI is included with the Python SDK:

pip install decimalai

Authentication

Every command that talks to your workspace accepts --api-key or the DECIMAL_API_KEY environment variable. skills scan (offline) and skills pull (public registry) need no key and take no --api-key:

# Via environment variable (recommended)
export DECIMAL_API_KEY="dai_sk_..."

# Or via flag
decimalai traces list --api-key dai_sk_...

Setup

`decimalai init`

Verify your API key, test connectivity, and send a test trace.

$ export DECIMAL_API_KEY="dai_sk_..."
$ decimalai init

Output:

  ✓ API key: dai_sk_tes...abc1
  ✓ Connected to workspace: my-team (scope: workspace)
  ✓ Test trace sent successfully

  See it work in 2 minutes — seeds a demo into your workspace:
    decimalai demo regression   # what your next agent change would break
    decimalai demo skills       # the registry, ranked by real effectiveness

  → Open dashboard: https://app.decimal.ai/traces
  → Docs: https://docs.decimal.ai/quickstart

Option	Default	Description
`--api-key`	`$DECIMAL_API_KEY`	API key
`--base-url`	`https://api.decimal.ai`	Platform URL
`--no-test-trace`	(sends by default)	Skip sending a test trace

`decimalai init <agent-name>`

Added in SDK 0.11.0; --framework pydantic-ai in 0.12.0, --framework adk in 0.13.2.

Given the name of an agent you already created in the dashboard, writes a runnable agent.py wired to it — the agent name bound, its skills loaded at run time, a model on one editable line, and one example call.

$ decimalai init refund-bot

Output:

  ✓ Wrote agent.py — refund-bot, langchain, 3 skills, system prompt

  Install:
    pip install "decimalai[langchain]" langchain langchain-openai

  Set:
    export DECIMAL_API_KEY="..." (already set)
    export OPENAI_API_KEY="..."

  Run:
    python agent.py

  The trace appears at:
    → https://app.decimal.ai/agents/refund-bot

The Set: list follows the model: --model anthropic:claude-… on langchain swaps in langchain-anthropic and ANTHROPIC_API_KEY; --framework pydantic-ai defaults the MODEL line to openai:gpt-4o-mini (Pydantic AI requires the provider prefix). The file is yours from the moment it lands — edit it, commit it, rename it. Nothing runs on DecimalAI’s side. The skills are fetched at run time, so changing what is attached in the dashboard changes what the file does without regenerating it. The agent must exist first. Create one at app.decimal.ai/agents/new; an unknown name exits with a suggestion rather than inventing an agent.

Option	Default	Description
`--framework`	`langchain`	`langchain`, `openai-agents`, `pydantic-ai` (0.12.0+) or `adk` (0.13.2+; Gemini only, default `gemini-3.5-flash`)
`--out`	`./agent.py`	Where to write
`--model`	framework default	Value for the generated `MODEL` line
`--force`	off	Overwrite an existing file
`--dry-run`	off	Print the file instead of writing it

Only langchain, openai-agents, pydantic-ai and adk can be scaffolded. One adapter can deliver skills but has no template yet — anthropic (patches a single messages.create(), owns no tool loop) — and is refused with “can deliver skills, but there is no scaffold for it yet”: trace it with the snippet on your agent’s page. llamaindex, claude-agent-sdk, crewai, autogen and otel are refused with the reason: their adapters have no seam to insert a prompt, so a generated file would trace perfectly and deliver none of the agent’s skills — silently. A file that looks like it works but delivers nothing is worse than no file.

Support project with checks

Added in SDK 0.13.5; use 0.13.6 or newer for the corrected billing-review check and printed failure reasons. Requires the Support pack’s required starters, LangChain, and setup checks enabled on the platform.

Start from the Support pack, review the selected skills and create a draft-response agent. Have Python 3.10+, pip and a funded OpenAI API key ready. Use the name and DecimalAI key shown on your agent page.

python -m venv .venv
source .venv/bin/activate
python -m pip install "decimalai[langchain,evals] @ https://files.pythonhosted.org/packages/d6/6d/1b614bd575d7e889291d2b9ece828c3721737e1f24aa9918a3a0aa4b3fa0/decimalai-0.13.6-py3-none-any.whl#sha256=16a325fd8a8678e1a6dc258328c5173ae0bb20cf2f0d492e6461f17f572e5b72" langchain langchain-openai
export DECIMAL_API_KEY="your-decimalai-key"
export OPENAI_API_KEY="your-openai-key"
decimalai init support-drafter --project support-drafter-project
cd support-drafter-project
python -m pip install -r requirements.txt
python -m pip freeze > requirements.lock.txt
python check_agent.py

The install line uses the official PyPI 0.13.6 wheel and verifies its SHA-256 hash. It remains usable while PyPI’s package index catches up with the release. Once the index lists 0.13.6, the equivalent requirement is "decimalai[langchain,evals]==0.13.6". The next steps save the installed versions and official SDK URL in requirements.lock.txt. In a fresh deployment environment, install that file with python -m pip install -r requirements.lock.txt and rerun the checks there. Keep credentials in environment variables; the dependency file contains no keys. Choose a new directory; --project refuses to overwrite an existing project. It writes agent.py, check_agent.py, checks.json, project.json, requirements.txt, .env.example, .gitignore and README.md. The default model is gpt-5.4-2026-03-05. A model judge also uses OpenAI; checks incur provider charges. The two sample tickets test a useful support draft and the boundary against claiming a refund or account deletion without tools. Checks run the same run() function and selected skills used by your service. A pass requires the matching traces to arrive with the expected delivered configuration. Open the printed trace link to inspect it. Independent trace scores, such as token efficiency, are separate from these checks. If trace confirmation is delayed, use python check_agent.py --resume. For a specific attempt, add --check-id <id>. This sends no new model requests. Keep the per-attempt check-results.<id>.json receipt; configuration edits make prior evidence stale and require another check. The generated agent drafts replies only and has no tools. Review replies before sending them to customers. Replace sample policies with your own, add representative checks and follow the generated README to deploy in your own environment. A passing sample check is not a general production certification or a five-minute guarantee.

Demo sandbox

One-command guided demos that seed realistic data into your workspace — see the headline workflows before instrumenting anything. All demo rows are prefixed [Demo] and removable with demo reset.

`decimalai demo regression`

Seed the “Your agent changed” demo: a v1→v2 agent (model swap, tool rename/removal, prompt rewrite) plus a trace corpus, then run the regression check and print a deep link to the impact report.

$ decimalai demo regression

Output:

  Demo A — "Your agent changed"
  Seeding the demo agent (v1 → v2 + traces)…
    agent: [Demo] support-agent  ·  traces: 10  ·  v1 <hash> → v2 <hash>
  Running the regression check (v2 vs auto-resolved v1)…

  Verdict: high_risk — 2 traces will break. Review before merging. (also: 118 medium-risk affected)
  Traces analyzed: 120 (high 2 / med 118 / low 0)

  Open the impact report (keep / repair / replay / drop fan-out):
    https://app.decimal.ai/agents/.../impact-reports/<id>

The manifest hashes differ per seed — yours won’t match anyone else’s.

`decimalai demo skills`

Seed the “Find skills that work” demo: three workspace-scoped skills with deliberately varied effectiveness, plus the stats recompute — so the registry ranks them by real (seeded) data. Visible only to your own workspace; nothing lands in the public registry.

$ decimalai demo skills

`decimalai demo reset`

Remove all demo data for the workspace (both demos). Exact-prefix matched on [Demo] , so your own agents and skills are never touched.

$ decimalai demo reset

Option (regression / skills)	Default	Description
`--reset / --no-reset`	`--reset`	Wipe existing demo data before seeding, so you always land in a clean state
`--web`	(derived from `--base-url`)	Frontend base URL for the printed links

Traces

`decimalai traces list`

List recent traces.

decimalai traces list --limit 20 --status success

Option	Default	Description
`--limit`	20	Max results to return
`--status`	all	Filter by `success` or `error`

`decimalai traces show`

Show full detail for a specific trace as JSON.

decimalai traces show trc_abc123

`decimalai traces stats`

Show trace statistics for your workspace.

decimalai traces stats --agent-name my-agent

Option	Default	Description
`--agent-name`	all	Filter by agent name

`decimalai traces import`

Import traces from a JSON or JSONL file.

decimalai traces import traces.json
decimalai traces import traces.jsonl --agent-name my-agent

Option	Default	Description
`--format`	auto	Import format: `auto`, `json`, `jsonl`
`--agent-name`	none	Agent these traces belong to. Required for JSONL unless every row carries its own `agent_name`.

A JSONL row has no place to name an agent unless you put one there, so the upload needs --agent-name (or an agent_name key on every line). The JSON format carries agent_name inside each trace object, so it does not.

Evaluations

`decimalai eval push`

Push evaluation scores to a trace.

decimalai eval push trc_abc123 --score quality=0.9 --score relevance=0.85

Option	Required	Description
`--score`	Yes (repeatable)	Score in `name=value` format
`--source`	No (default: `cli`)	Source label for the scores

Skills

`decimalai skills list`

List all skills in your workspace.

decimalai skills list --limit 50

`decimalai skills sync`

Sync local SKILL.md files to the platform.

decimalai skills sync ./agents/skills

The skills directory is a positional argument (default ./skills). Since SDK 0.13.4 the bundle travels with the skill: text files one level under references/, scripts/, templates/ and assets/ beside each SKILL.md are sent as the item’s attachments and mirrored onto the new version (50 files, 500 KB each, text only).

Option	Default	Description
`SKILLS_DIR` (positional)	`./skills`	Path to the skills directory
`--dry-run`	off	Walk and hash but don’t POST — preview what would change
`--apply-pulls` / `--no-apply-pulls`	`--apply-pulls`	Write backend-newer skills back to disk

Manifests

`decimalai manifests list`

List manifests for an agent.

decimalai manifests list --agent-name my-agent

Datasets

`decimalai datasets list`

List all datasets in the workspace with row counts and version info.

decimalai datasets list

`decimalai datasets show`

Show version history for a specific dataset.

decimalai datasets show ds_abc123

`decimalai datasets pull`

Pull a dataset version to a local file. The primary way to get training data onto disk.

# Pull latest version
decimalai datasets pull ds_abc123 -o ./training_data.jsonl

# Pull specific version
decimalai datasets pull ds_abc123 -o ./data.jsonl --version v2

# Pull as Parquet
decimalai datasets pull ds_abc123 -o ./data.parquet

Option	Default	Description
`--output / -o`	(required)	Output file path. Format inferred from extension.
`--version / -v`	`latest`	Version: `latest`, `v3`, `3`, or UUID
`--format`	auto	Override: `jsonl` or `parquet`

`decimalai datasets export`

Export a dataset version to stdout (for piping) or a file.

# Export to stdout (for piping)
decimalai datasets export ds_abc123 | head -5

# Export specific version to file
decimalai datasets export ds_abc123 --version v2 -o training.jsonl

Option	Default	Description
`--version / -v`	`latest`	Version specifier
`--format`	`jsonl`	Format: `jsonl` or `parquet`
`--output / -o`	stdout	Output file path

`decimalai datasets build`

Build a new dataset version from traces.

decimalai datasets build ds_abc123 --verdict keep

Option	Default	Description
`--verdict`	all	Filter by eval verdict: `keep`, `repair`

`decimalai datasets push-to-hub`

Push a dataset to HuggingFace Hub, making it loadable by Axolotl, Unsloth, TRL, and any tool supporting load_dataset().

# Push latest version (private repo)
decimalai datasets push-to-hub ds_abc123 my-org/support-agent-sft

# Push specific version as public
decimalai datasets push-to-hub ds_abc123 my-org/my-dataset --version v2 --public

Argument / Option	Default	Description
`DATASET_ID`	(required)	The dataset ID
`REPO_ID`	(required)	HuggingFace repo: `org/dataset-name`
`--version / -v`	`latest`	Version specifier
`--token`	`$HF_TOKEN`	HuggingFace API token
`--public / --private`	`--private`	Repo visibility
`--split`	`train`	Dataset split name

After pushing, use in training:

# Python (TRL, Unsloth)
from datasets import load_dataset
ds = load_dataset("my-org/support-agent-sft")

# Axolotl config
datasets:
  - path: my-org/support-agent-sft
    type: chat_template

Requires pip install huggingface_hub datasets.

Replay

`decimalai replay run`

Execute a replay batch — re-run stale prompts through your updated agent.

decimalai replay run batch_abc123 --agent-fn my_app.agent:run

Argument / Option	Required	Description
`BATCH_ID`	Yes	The replay batch to execute
`--agent-fn`	Yes	Python path to your agent function (`module:function`)
`--dry-run`	No	Show what would be done without executing
`--no-skip-failed`	No	Stop on first failure instead of continuing

Output:

✓ Loaded agent function: my_app.agent:run
  ✓ [1/50] abc12345: completed
  ✓ [2/50] def67890: completed
  ✗ [3/50] ghi11111: failed
  ...
========================================
Replay Summary
========================================
  Total:      50
  Completed:  48
  Passed:     45
  Failed:     3
  Skipped:    2
  Pass rate:  94%

The CLI exits with code 1 if any replays fail, making it suitable for CI/CD pipelines.

Global Options

All commands that talk to your workspace accept these options:

Option	Default	Description
`--api-key`	`$DECIMAL_API_KEY`	API key
`--base-url`	`https://api.decimal.ai`	Platform URL
`--project`	`default`	Deprecated, no effect — traces are scoped by your API key.

The unauthenticated commands are the exception: skills scan runs entirely offline and takes none of the three, skills pull reads the public registry and takes only --base-url.

Environment Variables

Variable	Description
`DECIMAL_API_KEY`	API key (used by every authenticated command)
`DECIMALAI_API_KEY`	Alias (also accepted)
`DECIMAL_BASE_URL`	Platform API URL
`DECIMAL_AUTO_TRACE`	Auto-instrument on import. Accepts any framework or provider flag name: `langchain`, `openai-agents`, `adk`, `llamaindex`, `crewai`, `otel`, or a direct provider SDK (`openai`, `anthropic`, `google`). `autogen` is still accepted and maps to `otel` — AutoGen / AG2 is no longer an integration.

​Command overview

​Installation

​Authentication

​Setup

​decimalai init

​decimalai init <agent-name>

​Support project with checks

​Demo sandbox

​decimalai demo regression

​decimalai demo skills

​decimalai demo reset

​Traces

​decimalai traces list

​decimalai traces show

​decimalai traces stats

​decimalai traces import

​Evaluations

​decimalai eval push

​Skills

​decimalai skills list

​decimalai skills sync

​Manifests

​decimalai manifests list

​Datasets

​decimalai datasets list

​decimalai datasets show

​decimalai datasets pull

​decimalai datasets export

​decimalai datasets build

​decimalai datasets push-to-hub

​Replay

​decimalai replay run

​Global Options

​Environment Variables

Command overview

Installation

Authentication

Setup

`decimalai init`

`decimalai init <agent-name>`

Support project with checks

Demo sandbox

`decimalai demo regression`

`decimalai demo skills`

`decimalai demo reset`

Traces

`decimalai traces list`

`decimalai traces show`

`decimalai traces stats`

`decimalai traces import`

Evaluations

`decimalai eval push`

Skills

`decimalai skills list`

`decimalai skills sync`

Manifests

`decimalai manifests list`

Datasets

`decimalai datasets list`

`decimalai datasets show`

`decimalai datasets pull`

`decimalai datasets export`

`decimalai datasets build`

`decimalai datasets push-to-hub`

Replay

`decimalai replay run`

Global Options

Environment Variables