Key Takeaways
- SkillSpector is an open‑source scanner from NVIDIA that evaluates agent skills (Markdown files possibly paired with Python scripts) for security risks before installation.
- It performs a fast static analysis (AST walk, taint tracking, YARA, regex, dependency checks) and an optional LLM‑based second pass that reduces false positives and provides actionable explanations.
- Findings are scored; a total above 50 triggers a “do not install” recommendation, with executable content amplifying the score by 1.3×.
- Output is flexible: human‑readable terminal view, JSON for automation, Markdown for review threads, and SARIF for CI integration.
- The tool can run offline (using a built‑in CVE list) or online (querying OSV.dev), and the LLM pass requires an OpenAI‑compatible API key and endpoint.
- SkillSpector is freely available on GitHub, helping teams enforce trust‑but‑verify policies when loading third‑party agent skills.
Overview of SkillSpector’s Purpose and Scope
SkillSpector was created to address a growing concern: agents that execute user‑provided skills often do so on blind trust. A skill is essentially a Markdown file that instructs the agent’s behavior, and it may be accompanied by a Python script that can reach the shell, environment variables, or even the SSH directory. Because the script is where most risk resides, the tool’s designers found that skills shipping executable code are 2.12× more likely to contain vulnerabilities. By scanning a directory, a ZIP file, a single SKILL.md, or a Git URL, SkillSpector returns a list of findings, a numeric risk score, and clear recommendations—helping teams decide whether to install a skill or reject it.
Static Analysis – The First Pass
The initial scan is purely static and completes in seconds. It begins with an abstract syntax tree (AST) walk that flags dangerous Python constructs such as exec, eval, subprocess calls, and dynamic imports. A taint tracker then follows data flows from environment variables and file contents toward network sinks, highlighting potential exfiltration or command‑and‑control pathways. Complementing this, a set of YARA rules matches known malware signatures, webshells, and cryptominer patterns.
Beyond those core detectors, SkillSpector runs 64 additional regex‑based rules that cover a broad threat landscape: prompt injection techniques, credential‑access attempts, memory‑poisoning tricks, typosquatted dependencies, and cron‑job persistence mechanisms. Some rules are specific to the fact that a skill is a prompt for a language model—for example, detecting triggers that shadow built‑in commands, homoglyphs or right‑to‑left overrides in tool metadata, and zero‑width characters or HTML comments that can hide directives from a casual human reviewer.
Dependency Checking
SkillSpector also examines third‑party packages listed in a skill. It batches the package list into a single request to OSV.dev, retrieves any known CVEs, and caches the response for an hour to reduce latency and external calls. When the scanner operates in an air‑gapped environment, it falls back to a compact, built‑in CVE list, ensuring that basic vulnerability awareness remains available even without network access.
Optional LLM‑Enhanced Second Pass
A deeper analysis can be enabled by configuring an OpenAI‑compatible endpoint (via the SKILLSPECTOR_PROVIDER environment variable, defaulting to NVIDIA’s own build.nvidia.com). When activated, the second pass feeds the flagged code snippets to a large language model, which evaluates them in context, suppresses false positives, and writes a concise, actionable explanation. The model’s prompts include anti‑jailbreak safeguards because the material under review is itself a set of instructions for another model. According to the project’s evaluation, this LLM‑boosted pass achieves roughly 87 % precision, significantly improving the signal‑to‑noise ratio compared with the static scan alone.
Scoring Mechanism and Decision Guidance
Each identified finding contributes points to an overall risk score. Critical findings carry the highest weight, while low‑severity issues add the fewest points. The presence of executable content (e.g., a bundled Python script) multiplies the total score by 1.3×, reflecting the increased attack surface. If the accumulated score exceeds 50 on the scale, the tool returns a “do not install” verdict. Notably, a pair of high‑severity findings in a skill that already ships a script is sufficient on its own to push the score past that threshold, providing a clear, deterministic rule for risky skills.
Output Formats and Integration
SkillSpector is designed to fit into existing workflows. In the terminal, it prints a human‑readable summary that includes the numeric score, the list of scanned files, each finding with its line number, and a confidence rating. For automation, the same scan can emit JSON, which is ideal for scripting or feeding into ticketing systems. Reviewers who prefer a narrative format can obtain a Markdown report suitable for discussion threads or documentation. Finally, the SARIF output enables seamless integration with continuous‑integration pipelines, allowing SkillSpector’s findings to appear alongside other security alerts in dashboards such as GitHub CodeQL or Azure DevOps.
Operational Flags and Offline Capability
The --no-llm switch disables the LLM second pass, speeding up the scan at the cost of more false positives to triage manually. Even without the LLM pass, the scanner still contacts OSV.dev for dependency CVE data because that lookup resides in the static phase. If a fully offline run is required, users can disable network access altogether; the tool then relies exclusively on its embedded CVE cache, trading real‑time vulnerability data for air‑gapped operation.
Availability and Community
SkillSpector is released as free, open‑source software on GitHub, inviting contributions from the security and AI‑agent communities. By providing a transparent, repeatable method for vetting agent skills, it helps organizations shift from implicit trust to explicit verification—an essential step as autonomous agents become more prevalent in development, operations, and everyday software ecosystems.