> ## Documentation Index
> Fetch the complete documentation index at: https://docs.decimal.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Receive LangSmith evaluation results via webhook

> Receive LangSmith evaluation results via webhook.

Expected payload format:
{
    "trace_id": "...",
    "scores": [
        {"key": "correctness", "score": 0.9, "comment": "Good response"},
        {"key": "helpfulness", "score": 0.85}
    ]
}

Verification: X-Webhook-Secret header must match DECIMAL_LANGSMITH_WEBHOOK_SECRET env.



## OpenAPI

````yaml /openapi.json post /api/v1/webhooks/langsmith
openapi: 3.1.0
info:
  title: DecimalAI Platform
  description: Agent Dataset Lifecycle Platform — Backend API
  version: 0.1.0
servers: []
security:
  - BearerAuth: []
tags:
  - name: traces
    description: >-
      Ingest, search, and inspect agent execution traces. A trace is the atomic
      unit of the platform — every other feature (evaluation, compatibility
      scoring, dataset building) operates on traces. Each trace is auto-tagged
      with the manifest hash of the agent that produced it.
  - name: eval-scores
    description: >-
      Push, fetch, and aggregate per-trace evaluation scores. Scores carry
      source provenance (built-in, DeepEval, LangSmith, custom) and feed the
      unified decision engine, which produces a keep / repair / replay / drop
      verdict per trace.
  - name: skills
    description: >-
      Manage reusable instruction blocks (SKILL.md files) that modify agent
      behavior. Skills are first-class manifest components — changes to a skill
      appear in your regression-check impact reports. Use these endpoints to
      create, version, fork, and sync skills with your local SKILL.md files.
  - name: manifests
    description: >-
      Register and inspect manifest versions. A manifest is a snapshot of your
      agent's structural identity (tools, prompts, models, skills) at a point in
      time. The SDK registers manifests automatically; these endpoints expose
      the underlying records.
  - name: datasets
    description: >-
      Build versioned SFT/DPO training datasets from manifest-classified,
      eval-scored traces. Datasets are reproducible — each version locks the
      manifest and filter set used to build it. Export as JSONL, Parquet, or
      push to HuggingFace Hub.
  - name: replay
    description: >-
      Re-execute historical traces against a new manifest version. Replay is the
      deferred-future capability for behavioral verification of agent changes.
      The SDK creates a replay batch, your worker executes each task, then
      submits results back here for eval scoring.
  - name: registry
    description: >-
      Browse and install community skills from the public skills registry. Each
      registry skill carries a SkillScore — an evidence-tiered quality composite
      computed from real-world evals, benchmarks, and ratings across
      organizations. Installing creates a fork in your org — edits don't affect
      the public version.
  - name: agents
    description: >-
      List and inspect agents, plus their multi-agent topology (orchestrator →
      sub-agent edges discovered from traces). Agents are identified by a stable
      agent_name string set in your SDK init() call.
  - name: import
    description: >-
      Bulk-import historical traces from another observability platform or a
      JSONL backup. Useful for migrating from LangSmith / Braintrust / Langfuse.
      Duplicate trace IDs are silently skipped — re-running an import is safe.
paths:
  /api/v1/webhooks/langsmith:
    post:
      tags:
        - webhooks
      summary: Receive LangSmith evaluation results via webhook
      description: >-
        Receive LangSmith evaluation results via webhook.


        Expected payload format:

        {
            "trace_id": "...",
            "scores": [
                {"key": "correctness", "score": 0.9, "comment": "Good response"},
                {"key": "helpfulness", "score": 0.85}
            ]
        }


        Verification: X-Webhook-Secret header must match
        DECIMAL_LANGSMITH_WEBHOOK_SECRET env.
      operationId: langsmith_webhook_api_v1_webhooks_langsmith_post
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/LangSmithWebhookPayload'
        required: true
      responses:
        '200':
          description: Successful Response
          content:
            application/json:
              schema: {}
        '401':
          description: Invalid or missing webhook signature
        '422':
          description: Malformed JSON payload
components:
  schemas:
    LangSmithWebhookPayload:
      description: Payload shape for POST /api/v1/webhooks/langsmith.
      properties:
        run_id:
          description: LangSmith run ID (UUID)
          title: Run Id
          type: string
        feedback:
          items:
            $ref: '#/components/schemas/LangSmithFeedback'
          title: Feedback
          type: array
      required:
        - run_id
        - feedback
      title: LangSmithWebhookPayload
      type: object
    LangSmithFeedback:
      properties:
        key:
          description: Feedback name, e.g., "correctness"
          title: Key
          type: string
        score:
          anyOf:
            - type: number
            - type: 'null'
          default: null
          title: Score
        value:
          anyOf:
            - {}
            - type: 'null'
          default: null
          title: Value
        comment:
          anyOf:
            - type: string
            - type: 'null'
          default: null
          title: Comment
      required:
        - key
      title: LangSmithFeedback
      type: object
  securitySchemes:
    BearerAuth:
      type: http
      scheme: bearer
      description: Enter your API key (e.g. dai_sk_test_key_001)

````