Playground uses your own LLM API keys (BYOK — Bring Your Own Key). DecimalAI doesn’t subsidize LLM calls — you pay your provider directly. Configure keys in Settings → Credentials.
Playground vs. skillevaluation. The Playground is for interactive, single-trace exploration — re-run one trace, eyeball the side-by-side, iterate by hand. To measure whether a skill actually helps across a batch of cases (a with-skill vs. without-skill A/B that produces a SkillScore), use the skillevaluation benchmark instead — see Skills. Use the Playground to form a hypothesis; use a benchmark to prove it.
Getting Started
Navigate to the Playground page from the sidebar, or open it contextually:- From a trace: Click “Open in Playground” on any trace detail page
- From a skill: Click “Test in Playground” on any skill detail page
- Direct URL:
/playgroundor/playground?skill=code-review
Three Modes
The Playground runs in one of three modes — pick the one that matches what you’re iterating on.- Import from Trace
- Skill Testing
- Scratch Pad
Re-run a production trace with modifications:
- Select an agent from the dropdown
- Select a trace — the system prompt and user message auto-populate
- The original output appears on the right for comparison
- Edit the system prompt or user message
- Choose a model and temperature
- Click Run (or press
⌘+Enter) - Compare the new output against the original side-by-side
Model Selection
The Playground page supports multiple LLM providers:
Select a provider and model from the dropdowns. If no API key is configured for the selected provider, you’ll see a friendly error with a link to Settings.
All three providers run directly in the Playground with your own keys (BYOK) — set each provider’s key in Settings.
gemini-3.5-flash is the default model.Temperature Control
Adjust the temperature slider (0.0–2.0) to control output randomness:Comparing Outputs
When importing from a trace or testing a skill, the page shows a side-by-side comparison:
This makes it easy to spot differences and judge whether your changes improved the output.
Saving Skill Changes
In Skill Testing mode, after running a modified skill body:- If the output improves, click “Save as New Version”
- This creates a new
SkillVersionwith your edited body - The version is automatically tracked in the skill’s version history
- All future activations of this skill use the updated body
Keyboard Shortcuts
Workflow Examples
Debugging a Bad Output
1
Find the trace
Find a trace with a poor output in the Traces page.
2
Open in Playground
Click “Open in Playground” on the trace detail page.
3
Refine the prompt
Edit the system prompt to add more specific instructions.
4
Iterate
Run → compare → iterate until the output improves.
5
Apply the fix
Apply the improved prompt to your agent’s configuration.
Hand-Tuning a Skill Body
1
Open the skill
Open a skill → “Test in Playground”.
2
Pick a few traces
Select 3–5 recent traces from the trace dropdown to sanity-check your edit against real conversations.
3
Edit and compare
For each trace: edit the skill body → run → compare side-by-side.
4
Save a new version
When satisfied, click “Save as New Version”.
5
Confirm it helps
Monitor the new version’s effectiveness on the Skills dashboard. To confirm the edit actually helps across a batch — not just on the handful you eyeballed — run a skillevaluation benchmark.
Testing Different Models
1
Set up the prompt
Import a trace or write a scratch prompt.
2
Run the first model
Run with
gpt-4o → note the output.3
Run a second model
Switch to
claude-sonnet-4-6 → run again.4
Choose the best fit
Compare outputs to choose the best model for your use case.
Next Steps
Tracing
Open any production trace in the playground to iterate on it.
Skills
Test skill changes in the playground before saving a new version.
Evaluations
Score playground runs with the same evaluators used in production.