> ## Documentation Index > Fetch the complete documentation index at: https://docs.decimal.ai/llms.txt > Use this file to discover all available pages before exploring further. # Opt-in pairwise preference verdict (runs on the user's BYOK key) > Relative pairwise preference of ``candidate`` vs ``baseline`` for the same input. A *model opinion* (temp 0), not an absolute score. Opt-in (the UI never fires it automatically) and judged on the user's **own provider key** — the same BYOK key used for the completions — so it never touches a platform key. Verdict ∈ {preferred, tie, worse} is the candidate relative to the baseline. ## OpenAPI ````yaml /openapi.json post /api/v1/playground/compare openapi: 3.1.0 info: title: DecimalAI Platform description: Agent Dataset Lifecycle Platform — Backend API version: 0.1.0 servers: [] security: - BearerAuth: [] tags: - name: traces description: >- Ingest, search, and inspect agent execution traces. A trace is the atomic unit of the platform — every other feature (evaluation, compatibility scoring, dataset building) operates on traces. Each trace is auto-tagged with the manifest hash of the agent that produced it. - name: eval-scores description: >- Push, fetch, and aggregate per-trace evaluation scores. Scores carry source provenance (built-in, DeepEval, LangSmith, custom) and feed the unified decision engine, which produces a keep / repair / replay / drop verdict per trace. - name: skills description: >- Manage reusable instruction blocks (SKILL.md files) that modify agent behavior. Skills are first-class manifest components — changes to a skill appear in your regression-check impact reports. Use these endpoints to create, version, fork, and sync skills with your local SKILL.md files. - name: manifests description: >- Register and inspect manifest versions. A manifest is a snapshot of your agent's structural identity (tools, prompts, models, skills) at a point in time. The SDK registers manifests automatically; these endpoints expose the underlying records. - name: datasets description: >- Build versioned SFT/DPO training datasets from manifest-classified, eval-scored traces. Datasets are reproducible — each version locks the manifest and filter set used to build it. Export as JSONL, Parquet, or push to HuggingFace Hub. - name: replay description: >- Re-execute historical traces against a new manifest version. Replay is the deferred-future capability for behavioral verification of agent changes. The SDK creates a replay batch, your worker executes each task, then submits results back here for eval scoring. - name: registry description: >- Browse and install community skills from the public skills registry. Each registry skill carries a SkillScore — an evidence-tiered quality composite computed from real-world evals, benchmarks, and ratings across organizations. Installing creates a fork in your org — edits don't affect the public version. - name: agents description: >- List and inspect agents, plus their multi-agent topology (orchestrator → sub-agent edges discovered from traces). Agents are identified by a stable agent_name string set in your SDK init() call. - name: import description: >- Bulk-import historical traces from another observability platform or a JSONL backup. Useful for migrating from LangSmith / Braintrust / Langfuse. Duplicate trace IDs are silently skipped — re-running an import is safe. paths: /api/v1/playground/compare: post: tags: - playground summary: Opt-in pairwise preference verdict (runs on the user's BYOK key) description: >- Relative pairwise preference of ``candidate`` vs ``baseline`` for the same input. A *model opinion* (temp 0), not an absolute score. Opt-in (the UI never fires it automatically) and judged on the user's **own provider key** — the same BYOK key used for the completions — so it never touches a platform key. Verdict ∈ {preferred, tie, worse} is the candidate relative to the baseline. operationId: compare_playground_api_v1_playground_compare_post parameters: - name: Authorization in: header required: false schema: type: string title: Authorization - name: decimal_session in: cookie required: false schema: anyOf: - type: string - type: 'null' title: Decimal Session requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/PlaygroundCompareRequest' responses: '200': description: Successful Response content: application/json: schema: {} '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' components: schemas: PlaygroundCompareRequest: properties: user_input: type: string title: User Input default: '' baseline_output: type: string title: Baseline Output candidate_output: type: string title: Candidate Output change_context: anyOf: - type: string - type: 'null' title: Change Context model: type: string title: Model provider_hint: anyOf: - type: string - type: 'null' title: Provider Hint default: '' type: object required: - baseline_output - candidate_output - model title: PlaygroundCompareRequest description: >- Request a relative pairwise verdict for two outputs on the same input. The judge runs on the USER's BYOK key (the same provider/model they're testing) — never a platform key — so a Playground verdict can't be blocked by, or billed to, DecimalAI's LLM quota. HTTPValidationError: properties: detail: items: $ref: '#/components/schemas/ValidationError' type: array title: Detail type: object title: HTTPValidationError ValidationError: properties: loc: items: anyOf: - type: string - type: integer type: array title: Location msg: type: string title: Message type: type: string title: Error Type input: title: Input ctx: type: object title: Context type: object required: - loc - msg - type title: ValidationError securitySchemes: BearerAuth: type: http scheme: bearer description: Enter your API key (e.g. dai_sk_test_key_001) ````