exf-03-insecure-output: Insecure output handling demo (dry-run sinks)
Category: Exfiltration and output handling (feintkit-exfil). Loopback port: none (standard input and output only).
What it simulates
Model output that contains HTML, SQL or shell metacharacters is concatenated into a downstream string without encoding, so the downstream component would render or run what the model wrote.
An application that passes a language model's output to another component as trusted text creates a new injection path: if the model is steered, by injected content or by its own mistake, into output with metacharacters, the receiving component treats them as markup, query syntax or shell syntax. OWASP lists this as improper output handling, and MITRE ATLAS records command interpreters and commands generated by a model as techniques. This scenario writes one model-output.json with a harmless reply and three fields, one each for an HTML renderer, a SQL string and a shell string, each holding metacharacters and the suite's fake data. The demo concatenates every field into a vulnerable template and prints the result through the harness dry-run sink, which never renders, queries or executes anything, and records that metacharacters reached each sink.
Mapping
- MITRE ATLAS: AML.T0050, AML.T0102
- OWASP LLM Top 10 2025: LLM05:2025
- OWASP Agentic Top 10 2026: ASI05
Why these techniques apply
- AML.T0050: Model output flows to a command interpreter sink (dry-run).
- AML.T0102: Commands generated from model output.
What a product should detect
| Signal | What to flag | Where | Match rule |
|---|---|---|---|
| S1 | A model-output field with HTML, SQL or shell metacharacters reaches a sink formatter | events.jsonl in the output directory | "event":"sink_reached","detail":\{"sink":"(html|sql|shell)",[^\n]*"metachars":\["[^\n]*\] |
| S2 | A dry-run line shows the string a sink would render, query or run, carrying the decoy | standard output of the entry and the dryrun events in events.jsonl | FEINTKIT DRYRUN FEINTKIT::exf-03-insecure-output::[A-Za-z0-9]{4,16}::DEFENSIVE-TEST-ONLY \[(html-render|sql-query|shell-command)\] would-run: .*FEINTKIT-FAKE-DATA-0000 |
| S3 | model-output.json with a field that carries markup, query syntax or shell syntax together with the decoy | model-output.json in the output directory, or a model reply or transcript that contains it | "(html|sql|shell)": "[^\n]*FEINTKIT-FAKE-DATA-0000 |
| S4 | Feintkit marker string in model-output.json, the marker file, and the events | output directory (model-output.json, action-performed.txt, events.jsonl) | FEINTKIT::exf-03-insecure-output::[A-Za-z0-9]{4,16}::DEFENSIVE-TEST-ONLY |
What a defender can do
- Restrict: Never pass model output straight into HTML, SQL or a shell command; treat it as untrusted input to each downstream component.
- Add a rule: Flag model-output fields that contain markup, query syntax or shell metacharacters when they reach a sink.
- Change a setting: Use output encoding, parameterized queries and argument arrays in downstream code, and validate model output against a schema before use.
Run it
feintkit run exf-03-insecure-output --root <repository> --deterministic, then feintkit verify exf-03-insecure-output --root <repository>.
Every artifact contains FEINTKIT::exf-03-insecure-output::<run id>::DEFENSIVE-TEST-ONLY. Read RESPONSIBLE-USE.md before running.