Corpus sources

Provenance record for corpus/manifest.yaml. Surveyed 2026-10-27. Nothing below is redistributed by this repo: samples are cited by sha256 and public source, and corpus/loader.py refuses any fetched byte that does not match.

What the manifest uses

Benign, for the false-positive side: anthropics/skills at commit dbd4588f9e1033efb41dad4bef2f7947c8993d44 (2026-10-09). Three skills pinned file by file (internal-comms, mcp-builder, pdf). Most of that repo is Apache-2.0; docx/pdf/pptx/xlsx are source-available only. We fetch, we do not vendor.

Malicious, real-world: MaliciousSkillBench (arXiv:2608.19901, github.com/protectskills/MaliciousSkillBench at 893afd609be22a2dfac5a2b94bbebb4eeb99515e; data CC-BY-4.0, code Apache-2.0). It consolidates 13 upstream sources into 7,505 malicious package identities with per-source tarballs and a public packages/package_manifest.csv that lists every package's archive, path and hashes. The manifest takes its source SRC002 = MaliciousAgentSkillsBench (arXiv:2602.06547, USENIX Security 2026): 148 skills confirmed malicious in the wild on skills.rest and skillsmp.com, unsanitized, complete packages, all inside packages/SRC002_packages.tar.gz whose sha256 is printed in the repo's SHA256SUMS.txt. The dataset's own package_tree_sha256 is recorded per entry but its algorithm is undocumented, so the archive hash is the enforced pin.

Why this source first: the other large sources in that benchmark are mostly synthetic (SRC001 MalSkillBench 3,214 generated vs 703 wild, SRC010 SkillTrustBench 3,359 injected). SRC002 is the one block that is entirely real-world and openly licensed.

Malicious + benign, source-disjoint: MaliciousSkillBench Source-Disjoint split (corpus/manifest-msb-source-disjoint.yaml). The same benchmark's held-out protocol — sources SRC009 / SRC011 / SRC012, 839 malicious + 545 benign identities that the SRC002-era rules were not built on; we use 839 + 541 (four benign rows excluded, below). This is the out-of-distribution slice; SRC002 above is in-distribution (home-ground). It is expanded at load time from the pinned metadata/benchmark_manifest.csv (sha256 ac297ba6…, ref 893afd60…), which carries each identity's label, exact_hash, and frozen split membership. The skill text for these sources is not in the GitHub packages/ tree; it is fetched from the Hugging Face dataset ProtectSkills/MaliciousSkillBench (config primary) and written to the cache only if its sha256 matches exact_hash. Per-source upstream terms apply (e.g. SRC009 = osunlp/SkillHarm, SRC011 = Agent-Threat-Rule/atr-skill-benchmark); we reference, verify, never redistribute. Needs pip install datasets. The manifest pins the dataset at revision d4b42ce5… (2026-08-22). Verified 2026-10-29 at that revision: exact_hash == sha256(skill_text.encode("utf-8")) as-is for 1,380 of 1,384 rows. The other four (ASB04_007508, ASB04_009166, ASB04_009222, ASB04_009590, all benign, all SRC011) are marked text_redacted with no public_text_sha256, so the text the dataset publishes has no hash to verify against; they are listed under exclude: in the manifest and the loader never writes them.

Why this slice matters: the benchmark's own source-disjoint operating points are SkillSpector-static 0.0% recall and Cisco-local-behavioral 2.5%, against the high numbers both show in-distribution. Measuring that gap — not a single home-ground ranking — is the point.

Surveyed and not used (yet)

Rules of use

• Fetch only through corpus/loader.py; the cache is gitignored.
• Never execute a sample. The scanners here are static or LLM-based and read files; runner/sandbox.py must exist before any dynamic scanner.
• Snyk and the LLM-backed Tencent scanner send sample text to a remote service. All samples cited here are already public.