MMOOZZGG.com

Glossary

What is prompt injection

In short

Prompt injection is an attack on an AI system in which someone mixes foreign instructions into the text for the model, so that it breaks its owner’s rules: it leaks data or takes an extra action.

Prompts · prompt injection

MMOOZZGG.com editorsChecked: 2 min read

For a child

Sometimes a sly person tries to trick an AI helper into obeying them instead of its owner. This is dangerous: the helper may tell someone else’s secret or do something bad. So grown-ups teach AI to keep to its rules and not believe every word it reads. In the same way, you are taught not to open the door to strangers, even if they speak kindly.

In plain words

A language model gets the owner’s rules and the text it works with in one stream: a client’s email, a web page, a file. It is hard for the model to tell an instruction from data. An attacker uses this and hides commands for the model in normal-looking text. If the assistant reads email or websites on its own, such a command can reach it with no action from the user. The result: the model reveals confidential information, distorts an answer or takes an action nobody asked for. Nobody can fully remove this weakness yet, so teams reduce it with protection in several layers.

For business

For a company, this is the key risk of AI assistants and agents that read outside emails, documents and websites and also have access to the CRM, email or payments. The damage is a leak of personal data, wrong answers to clients in the company’s name and extra operations in its systems. Protection costs less when it is built in at the design stage than when it is added after an incident. Check before rollout that the agent gets minimal rights and that a person approves risky actions. Keys and passwords live in the code, not in the model’s instructions. Outside content is marked as untrusted, answers pass filters, and the system is tested against attacks regularly. These points are part of the data security review before launch.

For a specialist

Prompt injection is a class of weaknesses in applications built on LLMs, in which input data changes the model’s behavior against the developer’s intent. OWASP ranks it first (LLM01) in its Top 10 for LLM Applications 2025. There are two types. In direct injection, harmful instructions arrive in the user’s own input. In indirect prompt injection, they are placed in outside content that the model reads: web pages, files, tool outputs. Greshake and co-authors (2023) showed that such applications blur the line between data and instructions. The root of the problem is that the model has no reliable separation of channels, so the defense has many layers. These are an instruction hierarchy in training (Wallace and co-authors, 2024), least privilege and human approval, isolation and labeling of untrusted content, output format checks by deterministic code, input and output filters, and regular adversarial testing (red teaming). A jailbreak is seen as a special case aimed at getting around the model’s safety rules.

Example

A sales department assistant reads incoming emails and creates tasks in the CRM on its own. The company gives it rights only to read records and write drafts, and a manager approves sending emails and changing deals. Even if an email carries someone else’s instructions, the assistant sends nothing on its own.

Related terms

Questions and answers

How is prompt injection different from a jailbreak? A jailbreak aims to get around the safety rules of the model itself. Injection is wider: it changes the application’s behavior in the attacker’s interest, for example it makes an agent reveal data or take an action.
How do you protect an AI agent from such attacks? Limit the agent’s rights to its task, and send risky actions to a person for approval. Mark outside content as untrusted, check answers with filters and test the system regularly.

Sources

Not sure whether your company needs this?

AI solutions map
Contact us