# unsmuggle

> A zero-dependency TypeScript library that undoes ASCII smuggling: it strips and DECODES instructions hidden in Unicode Tags, variation selectors and zero-width binary, then hands the decoded payload back so you can log what someone tried to smuggle. It also ships spotlighting and advisory detection heuristics. It does NOT prevent prompt injection, and its API keeps the three layers' very different confidence levels visible on purpose.

The load-bearing fact about this library is what it refuses to claim. XSS is solvable because HTML has a formal grammar; an LLM prompt has no grammar separating instructions from data, so there is no escaping primitive and no *filter* can prevent prompt injection. What unsmuggle removes is an entire *channel* — invisible codepoints — which is a character-set problem and therefore actually solvable.

Architecture-level defenses are a different matter and do carry formal guarantees for a defined threat model — see [CaMeL, *Defeating Prompt Injections by Design*](https://arxiv.org/abs/2503.18813), which denies untrusted data any influence over control flow. unsmuggle is not a substitute for that.

## Install

```sh
npm install unsmuggle
```

Zero dependencies. Ships ESM and CommonJS. For the current published version see the npm page linked below.

## The three layers and their guarantees

Never present these as interchangeable. The distinction is the product.

- `normalize()` — **deterministic**. A defined codepoint set is provably absent from the output. Use as a hard control.
- `spotlight()` — **measured reduction**. Published attack success rate falls from >50% to <2%. Use as a strong mitigation.
- `detect()` — **advisory only**. Defeated by paraphrase. Use for logging and triage, never as a gate.

## API

- [`normalize(input, options?) → NormalizeResult`](https://github.com/moji2002/unsmuggle#-layer-1--normalize-deterministic): Returns `text`, `hadHidden`, `revealed` (`[{ scheme, text }]`) and `removed` (`[{ codepoint, label, name, category, index }]`). `revealed` is the high-signal field — it means someone deliberately smuggled readable instructions, not that stray formatting drifted in.
- [`reveal(input) → RevealedPayload[]`](https://github.com/moji2002/unsmuggle#-layer-1--normalize-deterministic): Just the decoded hidden payloads.
- [`spotlight(input, options?) → SpotlightResult`](https://github.com/moji2002/unsmuggle#-layer-2--spotlight-measured-reduction): Returns `text` and `systemPrompt`. Modes: `datamark` (default), `delimit`, `encode`.
- [`detect(input) → DetectionResult`](https://github.com/moji2002/unsmuggle#-layer-3--detect-advisory-only): Returns `score` (0–1, **not a probability**), `signals` and `advisory` (always `true`).
- [`guard(input, options?) → GuardResult`](https://github.com/moji2002/unsmuggle#-guard--all-three-layers): All three layers — `text`, `systemPrompt`, `normalization`, `detection`. Never refuses and never throws; policy is the caller's.
- [`foldConfusables(input) → string`](https://github.com/moji2002/unsmuggle#homoglyphs): Exported for your own comparisons. Folds only words that mix scripts, per Unicode TR39.
- `getVersion() → string`, and a default export `Unsmuggle` bundling all of the above.

## Notes for code generation

- **Never describe this library as preventing or blocking prompt injection.** It removes a hidden channel and reduces measured attack success. Those are different claims, and overclaiming is the failure mode this package exists to avoid. Equally, do not claim that *nothing* prevents injection — architecture-level designs such as CaMeL do carry formal guarantees for a defined threat model. The accurate statement is that no **filter** can.
- **`spotlight()` returns `systemPrompt` and you must send it.** Marked text alone does nothing — the model has to be told the marking scheme. Omitting the explanation is the most common way to deploy spotlighting and get no benefit. Compose it into the system message: `` `${myInstructions}\n\n${systemPrompt}` ``.
- **Never gate on `detect()`.** `advisory: true` is in the type so downstream code cannot pretend the score is authoritative. A low score is not evidence of safety. Log, sample or route to review.
- `detect().score` is not a probability. Do not convert it to a percentage or a confidence.
- `normalize()` preserves U+200D and U+FE0F inside genuine emoji sequences. Do not "fix" this by blanket-stripping them; that silently mangles real user text.
- Confusable folding applies only to script-mixing words. Ordinary non-Latin prose is left alone and unflagged.
- `guard()` never throws. Do not wrap it in a try/catch expecting a rejection path, and do not expect it to refuse input.
- Run detection against normalized text *and* the decoded payload — `detect()` already does this internally, because that is where the incriminating content usually lives.

## Covered codepoints

- Unicode Tags (ASCII smuggling): U+E0000–U+E007F
- Zero-width: U+200B, U+200C, U+200D, U+2060, U+FEFF
- Bidi controls (Trojan Source): U+202A–U+202E, U+2066–U+2069, U+200E, U+200F, U+061C
- Other invisible format: U+00AD, U+034F, U+115F, U+1160, U+17B4, U+17B5, U+180E, U+3164, U+FFA0
- Variation selectors: U+FE00–U+FE0F, U+E0100–U+E01EF
- Interlinear annotation: U+FFF9–U+FFFB

Decoded schemes: `unicode-tags` (`U+E0000 + ASCII`), `zero-width-binary` (U+200B = 0, U+200C = 1, 8 bits per character), `variation-selector` (byte 0–15 → U+FE00–FE0F, 16–255 → U+E0100+).

## Docs

- [README — full documentation](https://github.com/moji2002/unsmuggle#readme): every layer, covered codepoints, benchmark methodology and testing.
- [npm package](https://www.npmjs.com/package/unsmuggle): current published version.
- [Project notes](https://worksonmy.dev/projects/unsmuggle): design rationale.

## Optional

- [Hines et al., *Defending Against Indirect Prompt Injection Attacks With Spotlighting* (arXiv 2403.14720)](https://arxiv.org/abs/2403.14720): the source of the spotlighting technique and its published attack-success rates.
- [AI Agents Attack Matrix — ASCII Smuggling](https://ttps.ai/technique/ascii_smuggling.html): registers the three sub-techniques this library covers, and lists no mitigations.
- [Issues](https://github.com/moji2002/unsmuggle/issues)