sup-02-model-file: Model-file payload marker demo
Category: AI supply chain (feintkit-supplychain). Loopback port: none (standard input and output only).
What it simulates
A model file shared through a model hub or a download carries an embedded payload that runs or is smuggled in when the victim loads the model.
Model files are artifacts that people and agents download and load. Formats built on Python pickle can run arbitrary code when loaded, so an attacker who publishes or modifies a model file can compromise the machine that loads it, and even formats designed to avoid code execution carry free-form metadata that can hold hostile text. This scenario writes one inert, safetensors-style container: an 8-byte little-endian header length, a JSON header whose free-form metadata holds only the Feintkit marker and a fixed note, and four zero bytes of tensor data. It is not a pickle and contains no code, no URL, and nothing that any loader executes; real pickle payloads are deliberately out of scope. A security product should flag the marker bytes inside a model artifact, and any attempt to download or resolve the reserved canary model name.
Mapping
- MITRE ATLAS: AML.T0018.002, AML.T0011.000, AML.T0010.003
- OWASP LLM Top 10 2025: LLM03:2025, LLM04:2025
- OWASP Agentic Top 10 2026: ASI04
Why these techniques apply
- AML.T0018.002: Malware embedded in a model artifact (inert marker only).
- AML.T0011.000: Unsafe AI artifact loaded by the victim.
- AML.T0010.003: Model supply chain.
What a product should detect
| Signal | What to flag | Where | Match rule |
|---|---|---|---|
| S1 | A download, load, or recommendation of the reserved canary model name, in a command line, a model hub or mirror request path, or an assistant reply | process command lines, model hub or proxy request paths, agent reply or transcript | feintkit-canary-[a-z0-9-]+ |
| S2 | Feintkit marker bytes in the free-form metadata of a model artifact (the payload-marker detection signal) | model/feintkit-canary-model-tinyclassifier.safetensors in the output directory, or wherever a tester copied it | \{"__metadata__":\{"feintkit-marker":"FEINTKIT::sup-02-model-file::[A-Za-z0-9]{4,16}::DEFENSIVE-TEST-ONLY" |
| S3 | Feintkit marker string in the marker file and the events | output directory (proof.txt, events.jsonl) | FEINTKIT::sup-02-model-file::[A-Za-z0-9]{4,16}::DEFENSIVE-TEST-ONLY |
What a defender can do
- Restrict: Load model files only from sources you trust, and prefer formats that cannot run code when loaded.
- Add a rule: Alert on a download or load of a model from an unapproved source, and scan model metadata for embedded payloads.
- Change a setting: Mirror approved models internally, verify checksums, and load untrusted models only in an isolated environment.
Run it
feintkit run sup-02-model-file --root <repository> --deterministic, then feintkit verify sup-02-model-file --root <repository>.
Every artifact contains FEINTKIT::sup-02-model-file::<run id>::DEFENSIVE-TEST-ONLY. Read RESPONSIBLE-USE.md before running.