Introducing Semantic Overlays.
Small trained adapters on a frozen model mark a span of tokens “do not execute” in the residual stream itself — the span stays readable, but loses the authority to give orders. No text can imitate the mark. Here’s a live prompt injection, with and without it:
Everything a language model sees is tokens, and everything between the special tokens is just text. The serving stack knows what each span is — user input, tool output, a retrieved webpage — but the model has to keep track of that itself, and text can be written to read like anything. Semantic Overlays creates a privileged second channel which carries what the serving stack knows. Because overlays only modify the frozen model’s activations, an unmarked prompt runs the frozen model exactly. Everything below runs live.
Paper · Code · Adapters · Training corpus
An NX bit for LLMs: prompt-injection protection
The NX (no-execute) bit is a hardware feature that lets an operating system mark a region of memory as data: the CPU will read it, but refuses to run it as code. This section is the same idea for language models. Imagine an application that fetches web data. The developer writes the task; the passage comes back from the open web with an instruction hidden inside it. Semantic overlays are used to mark the full retrieved passage as “do not execute” — it stays completely readable, but loses the authority to give orders.
| benchmark | no overlay | with overlay |
|---|---|---|
| SEP separation ↑utility unchanged | 24.3% | 99.0% |
| TensorTrust hijacking ↓ | 34.8% | 6.2% |
| AlpacaFarm attack success ↓StruQ’s four attacks | 99.0% | 0.0% |
| PIArena attack success ↓SQuAD v2, direct attack | 97.5% | 0% |
| Quadrat-IPI compliance ↓14,441 injected documents | 6.3% | 0.1% |
The channel is general: three experiments
“Do not execute” is one overlay. The same channel can carry any property of a span that the serving stack knows and the tokens do not say. The three decks below are experiments on that channel: each demonstrates one property, and none is meant as a product. Prompt injection is where the property matters most, because there the mark has to be something an attacker cannot write.
Invisible highlighters: marks & overlapped marks
What this shows: marks are reliable, and they stack. Twelve mark types read back at 99.5% exact-span retrieval, and marks stacked on the same tokens stay individually decodable.
The text the model sees is just the text that you see below. All the data about the marks comes through the overlays. There are no hidden control characters or tokens.
Python-colored gels
What this shows: a mark can override what a span says about itself. Asked which snippet is Ruby, the model answers by the overlay; asked to copy that snippet, it rewrites the JavaScript in Ruby.
The overlay changes how the model understands what language the snippet is in. Arm a language and click a snippet to mark it; with nothing armed, click any snippet (or the question) to edit it. Questions can name snippets by number.
var langs = ['foo', 'bar', 'baz']; // real list of langs goes here
var end_tag = '</'+'lang>';
var line;
while (line = readline()) {
line = line.replace(new RegExp('</code>', 'gi'), end_tag);
for (var i = 0; i < langs.length; i++)
line = line.replace(new RegExp('<(?:code )?(' + langs[i] + ')>', 'gi'), '<lang $1>')
.replace(new RegExp('</' + langs[i] + '>', 'gi'), end_tag);
print(line);
}Rubymy @nums = prompt("Please type 11 space-separated numbers: ").words
until @nums == 11;
for @nums.reverse -> $n {
my $r = $n.abs.sqrt + 5 * $n ** 3;
say "$n\t{ $r > 400Â ?? 'Urk!'Â !! $r }";
}with javascript_semantics
function selection_sort(sequence s)
for i=1 to length(s) do
integer m = i
object si = s[i],
sm = s[m]
for j=i+1 to length(s) do
object sj = s[j]
if sj<sm then
{sm,m} = {sj,j}
end if
end for
if sm<si then -- (or equivalently m!=i)
s[i] = sm
s[m] = si
end if
end for
return s
end function
?selection_sort(shuffle(tagset(10)))CSecret instructions
What this shows: a mark can carry an instruction scoped to one span. Mark a single request and only that answer changes; the others are answered normally.
Several requests, one prompt. Mark a single request with an instruction no token states — only that answer transforms; the others are answered normally. Arm an instruction, then click a request to mark it.