Scanning locally and in CI

The SkillSafety publish gate isn’t a black box you only meet when a publish fails. Its first tier — the deterministic static scan — is a CLI you can run yourself, before you ever push. Same rules, same code, same verdicts. It’s free, needs no API key, and never sends your skill anywhere: the scan is a pure-text pass over the files on your disk. Run it as a pre-commit hook, in a Makefile, or as a required CI check so a skill is caught on the pull request instead of at publish time.

# The whole thing — scan a directory of skills, no account, no network
decimalai skills scan ./skills

The same-engine guarantee

The local scanner is not a lookalike. The decimalai CLI imports and runs the exact Tier-1 scanner the publish gate runs — the same regex/AST-free rule set, versioned together — so a clean local run means the gate’s static tier will agree. Two things follow from that:

What runs locally is Tier-1 only. The static scan catches prompt-injection phrasing, remote-code-execution and reverse-shell patterns, destructive commands, committed secrets, data-exfiltration and agent-snooping, cloud-metadata/SSRF, hidden-unicode and homoglyph tricks, over-broad allowed-tools, and the rest of the deterministic checks. Secrets it finds are redacted, never echoed.
The LLM tiers stay server-side. The AI security review (Tier-2 intent judge) and content-safety review (Tier-3) need a model and run only on the platform. Exercise them with Deep Review on a private draft, or by hitting the publish gate itself.

Local is advisory — and deliberately never looser than the gate. The CLI judges context from the skill’s local files and frontmatter alone. It doesn’t call the registry, so it lacks the server-side signals — benchmark history, a published verdict, category reputation — that the gate uses to downgrade a borderline finding. That makes a local run stricter, not weaker: it can raise a Caution the gate would settle to Passed, but it will never pass a skill the gate would Block. Treat a local Blocked as a hard stop; treat a local pass as necessary, not sufficient.

Scan locally

Point the scanner at any directory; it recurses for SKILL.md files. The same command ships in both the DecimalAI CLI and the open-source skillevaluation runner — identical code, so pick whichever is already installed.

# Included with the Python SDK
pip install decimalai

# Human-readable report, grouped by severity (default)
decimalai skills scan ./skills

# Fail only on Blocked — the gate's own threshold (default)
decimalai skills scan ./skills --fail-on blocked

# Be stricter than the gate: fail on Caution too
decimalai skills scan ./skills --fail-on caution

# The open runner — no DecimalAI account, no API key
pip install "skillevaluation[runner]"

# Same engine, same verdicts
skillevaluation scan ./skills

# Emit machine-readable findings for a custom gate
skillevaluation scan ./skills --format json > findings.json

Output formats

--format controls how findings are rendered — the detection is identical across all four:

`--format`	Output	Reach for it when
`text`	Colorized, grouped by severity (default)	Reading results in a terminal
`json`	A findings array with per-finding fix hints	Feeding a script, dashboard, or custom gate
`github`	GitHub Actions workflow commands (`::error` / `::warning`)	Inline annotations on a pull request
`sarif`	SARIF 2.1.0	GitHub code scanning, or any SARIF viewer

Exit codes

--fail-on sets the band at or above which the command exits non-zero. Combined with the exit code, that’s all a CI gate needs:

Exit	Meaning
`0`	Clean — nothing at or above the `--fail-on` band
`1`	One or more skills at or above the `--fail-on` band
`2`	The scan itself couldn’t run (bad path, unreadable file, invalid flags)

--fail-on accepts blocked (default), caution (stricter — fail on either band), or never (always exit 0, report only). Distinguish 1 from 2 in scripts: 1 is a real finding you should act on; 2 is an operational error you should fix before trusting the result.

Wire decimalai skills scan ./skills --fail-on blocked into a pre-commit hook or make lint. Every finding comes back with a structured, per-finding fix — so a local Blocked tells you both what tripped and what to change, before the diff ever leaves your branch.

In CI (GitHub Actions)

Two things worth doing on every pull request that touches a skill: annotate the diff inline so a reviewer sees findings in context, and (optionally) upload SARIF so Blocked findings become tracked code-scanning alerts. This workflow does both, and needs no secrets — the scan is fully local to the runner.

# .github/workflows/skill-scan.yml
name: Skill safety scan

on:
  pull_request:
  push:
    branches: [main]

permissions:
  contents: read
  security-events: write   # required to upload SARIF to code scanning

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install the scanner
        run: pip install decimalai

      # Inline ::error (Blocked) / ::warning (Caution) annotations on the
      # changed lines. --fail-on blocked → exit 1 fails the check.
      - name: Scan skills
        run: decimalai skills scan ./skills --format github --fail-on blocked

      # Optional: also emit SARIF for tracked code-scanning alerts.
      # --fail-on never so this step always produces a file (exit 0);
      # the gating decision stays with the annotation step above.
      - name: Scan skills (SARIF)
        if: always()
        run: decimalai skills scan ./skills --format sarif --fail-on never > skills.sarif

      - name: Upload SARIF
        if: always()
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: skills.sarif

How the pieces fit:

--format github makes the scanner print GitHub workflow commands, so a Blocked finding surfaces as a red ::error and a Caution as a yellow ::warning, pinned to the offending line in the PR’s Files changed view — no log-scrolling.
The gating step and the SARIF step are separate on purpose. The annotation step (--fail-on blocked) owns the pass/fail decision. The SARIF step runs with --fail-on never and if: always() so it produces a file and uploads even when the gate fails — otherwise a Blocked finding would abort the job before it was ever recorded as an alert.
security-events: write is required for upload-sarif; contents: read covers the checkout. If you only want inline annotations and no code-scanning tab, drop the last two steps and the security-events permission.

This is the same binary you run locally, so a green CI check and a green laptop run mean the same thing. To also gate on the LLM tiers before a PR merges, run Deep Review on the draft (metered; the publish gate is always free) rather than expecting the local scan to cover them.

Defense in depth: run an ensemble

The SkillSafety static scanner is one detector with one rule set. Static scanners each catch a slightly different slice, so the strongest CI posture is an ensemble — run more than one and act on the union. Two independent open scanners pair well as additional steps:

Scanner	Adds
NVIDIA SkillSpector	An independent detection engine for agent-skill risks — a second opinion on the same files.
Cisco skill-scanner	A separate rule set and heuristics, catching patterns a single scanner can miss.

Both are third-party projects, not maintained by DecimalAI — add them as extra workflow steps alongside decimalai skills scan, each writing its own annotations or SARIF. More detectors mean more redundancy: a genuinely malicious skill has to slip past all of them, not just one.

skillevaluation

The open runner that ships the same scan command — and the A/B benchmark beside it.

Community Registry

Where the publish gate lives — the server-side counterpart to the local scan.

Security

DecimalAI’s platform security posture and data handling.

Get Started

Foundation

Ship Safely

Improve with skills

Trust & Safety

Data & Training

Operate

Core Concepts

Platform

Scanning locally and in CI

The same-engine guarantee

Scan locally

Output formats

Exit codes

In CI (GitHub Actions)

Defense in depth: run an ensemble

skillevaluation

Community Registry

Security

​The same-engine guarantee

​Scan locally

​Output formats

​Exit codes

​In CI (GitHub Actions)

​Defense in depth: run an ensemble

​Related

skillevaluation

Community Registry

Security

The same-engine guarantee

Scan locally

Output formats

Exit codes

In CI (GitHub Actions)

Defense in depth: run an ensemble

Related