Skip to main content
Since Part 2, every run of order-support has been traced: what it was asked, the model calls, the tools it called, the skills it loaded, and the prompt and skill versions it ran on. In this part you read those traces, compare the same question before and after a change, and read the agent’s versions.

​
1. Make two runs to look at

Run main.py twice, once with each question:
Open the agent’s Traces tab at app.decimal.ai/agents/order-support. The two runs are at the top; each takes a few seconds to arrive.

​
2. Read one trace

Open the double-charge run. The trace page shows:
  • The question and the reply.
  • The model calls and tool calls, in order: the lookup_order call with ORD-1002 and what it returned, and the load_skill call.
  • Prompt: the source (decimalai, since the agent has a saved prompt), the version it ran on, and a link to the other traces on that version.
  • Recorded skills, three rungs:
    • Offered: the skills on the menu the model saw.
    • Delivered: the skills whose full text the run received, with the version of each.
    • Loaded: the skills the model actually asked for with load_skill.
Open the account-deletion run. Its Loaded rung names account-deletion-verification-policy: same agent, same code, a different skill, because the question called for it. A run can be offered skills and load none. Ask “Hi, what can you do?” and the trace shows eight skills Offered and nothing Loaded. That’s normal: the model loads a skill only when the question calls for one.

​
3. Compare before and after

Part 3 changed two things: the prompt (the pack’s starter, then your blanks) and one skill (your copy of billing-dispute-resolution). Each trace records which versions it ran on, so you can line up the same question before and after.
  • The prompt. On the Traces tab, the Prompt column shows each run’s prompt version, and the prompt-version filter shows the runs on one version. Or start from the Prompt tab: each saved version links to Traces on this version. Pick a double-charge run on v1 and one on v3, and read the two replies side by side.
  • The skill. On each run’s trace page, the Delivered rung shows the version of billing-dispute-resolution the run received: the registry’s version before you forked it, your copy’s after. That tells you which instructions the reply followed.
Model output varies from run to run, so one pair of runs shows what changed, not how often. Counting that is Part 5.

​
4. Read the agent’s versions

The Versions tab lists the agent’s setup versions: each change to its skills, tools or model is a new one. Applying the Support pack in Part 3 shows there as one setup version, with the note “Applied the Support pack”, holding the added skills. Change review compares two versions, and sorts recent traces by whether the change could affect them: use it when you change a tool, the model or a skill. A prompt edit on its own does not make a new version, so to see what a prompt edit changed, use the prompt-version filter from step 3. Which prompt a run used has the details.

​
Where to go from here

  • Tracing: every field a trace records.
  • Skills: how skills are offered, delivered and loaded, and the usage numbers built from them.

Next: 5. Measure it

Run eight cases through order-support on two setups, check each run, and compare the counts.