iSearch

What is prompt injection?

Prompt injection is an attack in which text that a language model reads makes it follow instructions that neither the user nor the system intended.

A language model receives instructions and content in the same stream of text. It has no reliable boundary between what it should do and what it should merely read, and that is the weakness being exploited.

In direct prompt injection, the user types the instructions themselves, for instance to get round the rules in the system prompt. In indirect prompt injection, the instructions are hidden in an email, a web page or a document that the model reads while carrying out a task.

NIST describes both forms in its overview of attacks on AI systems. In December 2025 the UK's NCSC judged that prompt injection may never be fully mitigated in the way SQL injection was.

How it works

Here is an example. An AI agent is asked to summarise the day's email. One message contains a line in white text asking the model to send the latest contracts to an outside address. To the model the line looks like any other instruction, and if the agent may send email it can do so.

The attacker needs no access to the system. It is enough to place text where the model will read it, in a shared file, on a web page or in a form.

Why it matters

The risk grows with what the model is allowed to do. An assistant that only answers can give a wrong answer that someone then checks. An agent using tool calling can carry out an action before anyone has time to react, and what leaks is often exactly what the model already had access to.

What to keep in mind

Filters and clear instructions reduce the risk without removing it. What holds are limits outside the model, such as least privilege, approvals and logging.

Test the system deliberately. Red teaming shows how the model responds to attempts to trick it before an outsider tries.

What it means for you

Assume a model can be tricked at some point. Limit what it can reach and do, and have a person approve anything that cannot be undone.

Common questions

Can prompt injection affect an AI that only answers questions?

Yes. It can make the model give misleading answers or repeat things it should not. The harm is often greater when the model can also take actions.

Can the attacks be filtered out?

Filters catch many attempts but far from all. Protection therefore also needs to rest on what the model may reach and do.

How does prompt injection relate to data leakage?

A successful attack can make the model pass data on to the wrong recipient. Read more about data leakage.

Sources: NIST AI 100-2 E2025 on attacks on AINCSC on prompt injection (2025)

All terms Updated