What is prompt injection?
Prompt injection is the placement of instructions inside content a model reads, intended to redirect its behaviour. The carrier is ordinary material, such as a web page an agent retrieves or a document a counterparty uploaded. A model has no dependable way to separate instructions from the data it was handed, which is what makes the technique effective.
Why an injection defence has to live outside the model
An agent asked to summarise an inbound supplier email follows an instruction buried in the signature block and forwards an internal document to an external address. No model was compromised and no credential was stolen, yet the outcome is a reportable data breach. Attempt volume has increased sharply as agents acquired tools and reach, and the defences that hold are structural rather than textual. A system depending on the model recognising an attack has no defence at all.
- A model cannot separate instruction from data, so the separation is imposed around it.
- Every tool an agent holds extends what a successful injection can reach.
- Retrieved content is attacker controlled whenever the source lies outside the organisation.
- Filtering for known phrasings fails against an attacker who rewords the instruction.
- The damage is measured in actions taken rather than in text produced.
How prompt injection works
- Content from outside the trust boundary is labelled as data at retrieval.
- Instructions are held in a channel the retrieved content cannot reach.
- Each tool an agent may call is restricted to the minimum scope the task requires.
- Actions with external effect require a confirmation step outside the model's control.
- Outputs are checked against permitted destinations before anything leaves the perimeter.
- Attempts are detected and recorded so the pattern of targeting becomes visible.
How prompt injection is evaluated
Resistance is demonstrated by attack rather than by inspection. Measure the following against an adversarial corpus maintained over time:
- proportion of known injection patterns that reach a tool call
- blast radius of each tool, meaning the actions it can take
- share of attempts detected at retrieval rather than at output
- success rate of reworded variants of a payload already blocked
- time from a technique appearing publicly to coverage in the corpus
Limitations, risks and common failure modes
- Defences are tested only against the examples already in the filter list.
- A document store accepts counterparty uploads with no treatment of the content.
- An agent is given a broad integration token because scoping was deferred.
- An attack succeeds through a formatting layer the text filter never sees.
- Confirmation prompts are approved reflexively once volume increases.