CLI - DecimalAI

The DecimalAI CLI provides command-line access to your workspace.

This page covers the most-used commands. The CLI also ships an evaluators group, the full skills lifecycle (status, pull, install, benchmark, push, verify, attest, spotcheck), compat-check, and regression-check (the CI command — see Regression Check). Run decimalai --help or decimalai <group> --help for the complete list.

Command overview

Command	Group	What it does
`init`	Setup	Verify your API key, test connectivity, and send a test trace.
`demo regression`	Demo	Seed the “Your agent changed” demo and run the regression check.
`demo skills`	Demo	Seed the “Find skills that work” demo and recompute registry stats.
`demo reset`	Demo	Remove all `[Demo]` -prefixed data from the workspace.
`traces list`	Traces	List recent traces.
`traces show`	Traces	Show full detail for one trace as JSON.
`traces stats`	Traces	Show trace statistics for the workspace.
`traces import`	Traces	Import traces from a JSON or JSONL file.
`eval push`	Evaluations	Push named evaluation scores to a trace.
`skills list`	Skills	List skills in the workspace.
`skills sync`	Skills	Sync local `SKILL.md` files to the platform.
`skills pull`	Skills	Read-only download of a public skill (no fork).
`skills install`	Skills	Fork a skill and write `SKILL.md` to disk.
`skills push` / `benchmark` / `verify` / `attest` / `spotcheck` / `status`	Skills	Full publish + effectiveness lifecycle.
`manifests list`	Manifests	List manifests for an agent.
`datasets list`	Datasets	List datasets with row counts and version info.
`datasets show`	Datasets	Show version history for a dataset.
`datasets pull`	Datasets	Pull a dataset version to a local file.
`datasets export`	Datasets	Export a dataset version to stdout or a file.
`datasets build`	Datasets	Build a new dataset version from traces.
`datasets push-to-hub`	Datasets	Push a dataset to HuggingFace Hub.
`replay run`	Replay	Execute a replay batch through your updated agent.
`evaluators ...`	Evaluators	Manage custom evaluators.
`compat-check`	Compatibility	Diff a candidate manifest against a baseline.
`regression-check`	CI	The CI command — see Regression Check.

Installation

The CLI is included with the Python SDK:

pip install decimalai

Authentication

Every command that talks to your workspace accepts --api-key or the DECIMAL_API_KEY environment variable. skills scan (offline) and skills pull (public registry) need no key and take no --api-key:

# Via environment variable (recommended)
export DECIMAL_API_KEY="dai_sk_..."

# Or via flag
decimalai traces list --api-key dai_sk_...

Setup

`decimalai init`

Verify your API key, test connectivity, and send a test trace.

$ export DECIMAL_API_KEY="dai_sk_..."
$ decimalai init

Output:

  ✓ API key: dai_sk_tes...abc1
  ✓ Connected to workspace: my-team (scope: workspace)
  ✓ Test trace sent successfully

  See it work in 2 minutes — seeds a demo into your workspace:
    decimalai demo regression   # what your next agent change would break
    decimalai demo skills       # the registry, ranked by real effectiveness

  → Open dashboard: https://app.decimal.ai/traces
  → Docs: https://docs.decimal.ai/quickstart

Option	Default	Description
`--api-key`	`$DECIMAL_API_KEY`	API key
`--base-url`	`https://api.decimal.ai`	Platform URL
`--no-test-trace`	(sends by default)	Skip sending a test trace

Demo sandbox

One-command guided demos that seed realistic data into your workspace — see the headline workflows before instrumenting anything. All demo rows are prefixed [Demo] and removable with demo reset.

`decimalai demo regression`

Seed the “Your agent changed” demo: a v1→v2 agent (model swap, tool rename/removal, prompt rewrite) plus a trace corpus, then run the regression check and print a deep link to the impact report.

$ decimalai demo regression

Output:

  Demo A — "Your agent changed"
  Seeding the demo agent (v1 → v2 + traces)…
    agent: [Demo] support-agent  ·  traces: 10  ·  v1 <hash> → v2 <hash>
  Running the regression check (v2 vs auto-resolved v1)…

  Verdict: high_risk — 2 traces will break. Review before merging. (also: 118 medium-risk affected)
  Traces analyzed: 120 (high 2 / med 118 / low 0)

  Open the impact report (keep / repair / replay / drop fan-out):
    https://app.decimal.ai/agents/.../impact-reports/<id>

The manifest hashes differ per seed — yours won’t match anyone else’s.

`decimalai demo skills`

Seed the “Find skills that work” demo: three workspace-scoped skills with deliberately varied effectiveness, plus the stats recompute — so the registry ranks them by real (seeded) data. Visible only to your own workspace; nothing lands in the public registry.

$ decimalai demo skills

`decimalai demo reset`

Remove all demo data for the workspace (both demos). Exact-prefix matched on [Demo] , so your own agents and skills are never touched.

$ decimalai demo reset

Option (regression / skills)	Default	Description
`--reset / --no-reset`	`--reset`	Wipe existing demo data before seeding, so you always land in a clean state
`--web`	(derived from `--base-url`)	Frontend base URL for the printed links

Traces

`decimalai traces list`

List recent traces.

decimalai traces list --limit 20 --status success

Option	Default	Description
`--limit`	20	Max results to return
`--status`	all	Filter by `success` or `error`

`decimalai traces show`

Show full detail for a specific trace as JSON.

decimalai traces show trc_abc123

`decimalai traces stats`

Show trace statistics for your workspace.

decimalai traces stats --agent-name my-agent

Option	Default	Description
`--agent-name`	all	Filter by agent name

`decimalai traces import`

Import traces from a JSON or JSONL file.

decimalai traces import traces.json
decimalai traces import traces.jsonl --format jsonl

Option	Default	Description
`--format`	auto	Import format: `auto`, `json`, `jsonl`

Evaluations

`decimalai eval push`

Push evaluation scores to a trace.

decimalai eval push trc_abc123 --score quality=0.9 --score relevance=0.85

Option	Required	Description
`--score`	Yes (repeatable)	Score in `name=value` format
`--source`	No (default: `cli`)	Source label for the scores

Skills

`decimalai skills list`

List all skills in your workspace.

decimalai skills list --limit 50

`decimalai skills sync`

Sync local SKILL.md files to the platform.

decimalai skills sync ./agents/skills

The skills directory is a positional argument (default ./skills).

Option	Default	Description
`SKILLS_DIR` (positional)	`./skills`	Path to the skills directory
`--dry-run`	off	Walk and hash but don’t POST — preview what would change
`--apply-pulls` / `--no-apply-pulls`	`--apply-pulls`	Write backend-newer skills back to disk

Manifests

`decimalai manifests list`

List manifests for an agent.

decimalai manifests list --agent-name my-agent

Datasets

`decimalai datasets list`

List all datasets in the workspace with row counts and version info.

decimalai datasets list

`decimalai datasets show`

Show version history for a specific dataset.

decimalai datasets show ds_abc123

`decimalai datasets pull`

Pull a dataset version to a local file. The primary way to get training data onto disk.

# Pull latest version
decimalai datasets pull ds_abc123 -o ./training_data.jsonl

# Pull specific version
decimalai datasets pull ds_abc123 -o ./data.jsonl --version v2

# Pull as Parquet
decimalai datasets pull ds_abc123 -o ./data.parquet

Option	Default	Description
`--output / -o`	(required)	Output file path. Format inferred from extension.
`--version / -v`	`latest`	Version: `latest`, `v3`, `3`, or UUID
`--format`	auto	Override: `jsonl` or `parquet`

`decimalai datasets export`

Export a dataset version to stdout (for piping) or a file.

# Export to stdout (for piping)
decimalai datasets export ds_abc123 | head -5

# Export specific version to file
decimalai datasets export ds_abc123 --version v2 -o training.jsonl

Option	Default	Description
`--version / -v`	`latest`	Version specifier
`--format`	`jsonl`	Format: `jsonl` or `parquet`
`--output / -o`	stdout	Output file path

`decimalai datasets build`

Build a new dataset version from traces.

decimalai datasets build ds_abc123 --verdict keep

Option	Default	Description
`--verdict`	all	Filter by eval verdict: `keep`, `repair`

`decimalai datasets push-to-hub`

Push a dataset to HuggingFace Hub, making it loadable by Axolotl, Unsloth, TRL, and any tool supporting load_dataset().

# Push latest version (private repo)
decimalai datasets push-to-hub ds_abc123 my-org/support-agent-sft

# Push specific version as public
decimalai datasets push-to-hub ds_abc123 my-org/my-dataset --version v2 --public

Argument / Option	Default	Description
`DATASET_ID`	(required)	The dataset ID
`REPO_ID`	(required)	HuggingFace repo: `org/dataset-name`
`--version / -v`	`latest`	Version specifier
`--token`	`$HF_TOKEN`	HuggingFace API token
`--public / --private`	`--private`	Repo visibility
`--split`	`train`	Dataset split name

After pushing, use in training:

# Python (TRL, Unsloth)
from datasets import load_dataset
ds = load_dataset("my-org/support-agent-sft")

# Axolotl config
datasets:
  - path: my-org/support-agent-sft
    type: chat_template

Requires pip install huggingface_hub datasets.

Replay

`decimalai replay run`

Execute a replay batch — re-run stale prompts through your updated agent.

decimalai replay run batch_abc123 --agent-fn my_app.agent:run

Argument / Option	Required	Description
`BATCH_ID`	Yes	The replay batch to execute
`--agent-fn`	Yes	Python path to your agent function (`module:function`)
`--dry-run`	No	Show what would be done without executing
`--no-skip-failed`	No	Stop on first failure instead of continuing

Output:

✓ Loaded agent function: my_app.agent:run
  ✓ [1/50] abc12345: completed
  ✓ [2/50] def67890: completed
  ✗ [3/50] ghi11111: failed
  ...
========================================
Replay Summary
========================================
  Total:      50
  Completed:  48
  Passed:     45
  Failed:     3
  Skipped:    2
  Pass rate:  94%

The CLI exits with code 1 if any replays fail, making it suitable for CI/CD pipelines.

Global Options

All commands that talk to your workspace accept these options:

Option	Default	Description
`--api-key`	`$DECIMAL_API_KEY`	API key
`--base-url`	`https://api.decimal.ai`	Platform URL
`--project`	`default`	Project name

The unauthenticated commands are the exception: skills scan runs entirely offline and takes none of the three, skills pull reads the public registry and takes only --base-url. init takes --api-key and --base-url, but has no --project.

Environment Variables

Variable	Description
`DECIMAL_API_KEY`	API key (used by every authenticated command)
`DECIMALAI_API_KEY`	Alias (also accepted)
`DECIMAL_BASE_URL`	Platform API URL
`DECIMAL_AUTO_TRACE`	Auto-instrument on import. Accepts any framework or provider flag name: `langchain`, `openai-agents`, `adk`, `llamaindex`, `crewai`, `autogen`, `otel`, or a direct provider SDK (`openai`, `anthropic`, `google`).

​Command overview

​Installation

​Authentication

​Setup

​decimalai init

​Demo sandbox

​decimalai demo regression

​decimalai demo skills

​decimalai demo reset

​Traces

​decimalai traces list

​decimalai traces show

​decimalai traces stats

​decimalai traces import

​Evaluations

​decimalai eval push

​Skills

​decimalai skills list

​decimalai skills sync

​Manifests

​decimalai manifests list

​Datasets

​decimalai datasets list

​decimalai datasets show

​decimalai datasets pull

​decimalai datasets export

​decimalai datasets build

​decimalai datasets push-to-hub

​Replay

​decimalai replay run

​Global Options

​Environment Variables

Command overview

Installation

Authentication

Setup

`decimalai init`

Demo sandbox

`decimalai demo regression`

`decimalai demo skills`

`decimalai demo reset`

Traces

`decimalai traces list`

`decimalai traces show`

`decimalai traces stats`

`decimalai traces import`

Evaluations

`decimalai eval push`

Skills

`decimalai skills list`

`decimalai skills sync`

Manifests

`decimalai manifests list`

Datasets

`decimalai datasets list`

`decimalai datasets show`

`decimalai datasets pull`

`decimalai datasets export`

`decimalai datasets build`

`decimalai datasets push-to-hub`

Replay

`decimalai replay run`

Global Options

Environment Variables