LLMs process instructions and data through the same channel with no built-in separation. OWASP's LLM Top 10 has held prompt injection at #1 for two consecutive editions. Here is why — and the layered defense that actually works.
Prompt injection has held the top spot on the OWASP Top 10 for LLM Applications for two consecutive editions. The reason is structural, not incidental. LLMs process instructions and the data they are working on through the same channel, with no built-in separation between the two. When a model reads “here is an email, summarise it” followed by the actual email text, it has no hard boundary telling it that everything after a certain point is data, not commands. If the email contains something that looks like an instruction, the model cannot reliably tell the difference.
OWASP is direct about the consequence: you cannot patch your way out of prompt injection. It exploits how LLMs fundamentally work, not a specific bug you can fix once. As anything agentic gets wired to these models, email access, file-system access, API credentials, the ceiling on what a successful injection can achieve rises sharply.
A user explicitly types a malicious instruction — the classic pattern is asking a chatbot to disregard its prior instructions and reveal internal configuration. This is the version most people picture, and it is the easier of the two to defend against, because the input comes directly from a user you can apply scrutiny to.
The attacker types nothing into your chat. Instead, they embed malicious instructions inside content the model will later process on your behalf: a webpage, a document, an email, a support ticket, a Slack message. When your AI assistant reads that content — to summarise, triage, or review — it can encounter those embedded instructions and follow them as if they came from you. From the model’s perspective, text is text.
This is precisely what happened in the documented Slack AI incident: malicious instructions hidden in ordinary Slack content caused the AI integration to exfiltrate information it should not have accessed. The attack surface is not the model — it is the model’s pipeline of untrusted content.
Anthropic’s own published system card for Claude Opus 4.5 provides one of the more concrete, quantified pictures of real-world injection resistance in an agentic coding environment:
Two things worth taking from that data. First, single-attempt defenses genuinely work most of the time — a 4.7% success rate at one attempt means a casual attacker hits a wall more than 95% of the time. Second, an attacker who can make many attempts — an automated, repeated attack against a production system — sees success rates climb to 63% at a hundred tries. That is the actual argument for defense in depth and monitoring, not just a single good filter.
The ceiling on impact scales directly with what the compromised model is allowed to do. A chatbot with no tool access and no memory of sensitive data has a low ceiling on what a successful injection can achieve. An agent with email access, file-system access, and API credentials has a very different ceiling. This is why prevention has to focus as much on limiting what a model can do as on stopping it from being tricked in the first place.
Instruction override
Model ignores original guidance and follows attacker instructions instead.
Data exfiltration
Sensitive content the model can access — system prompt, documents, connected-account data — gets surfaced to the attacker.
Unauthorised tool calls
In an agentic system, a successful injection can trigger real actions: send email, modify files, call APIs — using the agent's own legitimate permissions.
System prompt leakage
Attacker extracts the hidden instructions that shape the application's AI behaviour, enabling more targeted follow-on attacks.
Downstream chain corruption
In a multi-step workflow, a single injected instruction early in the chain quietly corrupts subsequent steps that trust earlier outputs.
No single control solves this. OWASP’s own guidance is explicit that mitigation requires layering multiple defenses, because none of them is individually sufficient against a motivated, repeated attack. Each technique below reduces risk — stacked together, they reduce it substantially.
Structure prompts so untrusted content — a fetched webpage, an uploaded document, an email body — is visibly delimited as data to be processed, not blended into the instruction stream. This is the direct analogue to parameterised queries preventing SQL injection: separate the command channel from the data channel wherever the architecture allows it.
Define expected output formats and behavioral boundaries in the system prompt itself. A model instructed to only ever output a specific structured format has less room for an injected instruction to hijack the interaction into open-ended behaviour.
This is the highest-leverage structural defense for anything agentic. Give a model or agent only the specific tools and access it needs for its actual task — nothing broader “just in case.” An agent that can only read a specific document has a low ceiling even if successfully injected. An agent with broad file-system, email, and API access has a much higher one. Same principle as row-level security in a database, applied to agent permissions.
For anything with real consequence — sending an email, making a purchase, deleting data, executing code against production — insert an explicit human-in-the-loop checkpoint rather than letting an agent act autonomously. This is the single most reliable backstop against an injection that clears every earlier filter.
Scan for known attack patterns and known-bad output signatures on both sides. This catches what you have seen before — not novel attacks, which is exactly why it is one layer among several rather than a complete solution.
For any RAG-connected system, assess context relevance and groundedness as a way of catching outputs that have drifted from what the retrieved content actually supports — a useful signal that something embedded in a retrieved document has pulled the model off-track.
Conduct regular red-teaming and breach simulations — not a one-time pre-launch audit, but a recurring practice. The attack-success-rate data makes the argument: repeated attempts matter, and a single pre-launch test does not model a production attacker who gets a hundred tries. Open-source tooling (Promptfoo, Garak) exists specifically for this.
Grounding a model in retrieved data (RAG) or fine-tuning it on your own data does not close the prompt injection gap. Both are still vulnerable to malicious content smuggled into whatever they process. Defense in depth remains necessary regardless of whether you have also invested in RAG or fine-tuning for other reasons. These are separate problems.
Prompt injection is not a theoretical concern for a future AI product — it is a present concern for anything already wiring AI to external content. Four categories where the risk is immediate:
The attack surface is not the model — it is every piece of untrusted content you let the model touch, and everything you let it do with what it finds there.
OWASP Foundation · owasp.org
The industry-standard risk list for LLM deployments, with prompt injection ranked #1. Essential reading before shipping any LLM-powered feature.
Perez & Ribeiro · arXiv 2022
The first systematic study of prompt injection as an attack class, establishing the taxonomy that the security community now uses.
Ross Anderson · Wiley
The most comprehensive textbook on building secure systems. The chapters on protocol failures translate directly to LLM input/output design.
Continue the conversation
If this changed how you think about it — or you think I'm wrong — I want to know.
Corrections, disagreements, and applications all welcome. Replies go directly to Chris.
Get in touch →THE SAME CHANNEL
~6-8 min1× · Two speakers · tap to play