Glossary
What is AI data leakage?
In short
AI data leakage is a case in which company or customer data reaches outsiders because of a neural network: through employees’ prompts, model answers, or connected systems.
Security and law · data leakage
MMOOZZGG.com editorsChecked: 1 min read
For a child
Imagine you told a friend a secret, and the friend wrote it in a shared notebook that the whole class reads. Your friend did not mean any harm, they just did not think. A computer helper is sometimes like that notebook: what you write in it, someone else may see. So adults agree on which secrets must not be told to it, and they watch who can read its notes.
In plain words
Most often it happens with no hacking at all. An employee copies a contract, a customer table, or code into a public chatbot to finish work faster. This data goes to another company’s servers, may be stored there, and sometimes ends up in the training of future models. It can also happen another way: the company’s AI assistant is connected to documents and by mistake shows a person something they should not see. Protection starts with simple rules: what may be sent to a neural network, what may not, and which services to use.
For business
The consequences are measured in money: fines for personal data breaches, loss of trade secrets, and customer claims. The sources of risk are public services that employees use without permission, assistants with too much access to files, and integrations that give the model more fields than it needs. The measures, from cheapest to most costly, are: an AI use policy and employee training, corporate plans that opt out of training on your data, masking personal data before it is sent, and your own model in a closed network for highly sensitive data. Start with an inventory: which services employees already use and what goes into them.
For a specialist
OWASP lists sensitive information disclosure among the key risks of LLM applications, item LLM02 in the 2025 edition: personal, financial, and medical data, business data, credentials, and legal documents can appear in the model’s output. The channels are data sent to an outside API, prompts stored by the vendor, memorization of training data, overly broad assistant access in RAG and agent systems, and prompt injection through connected documents. Carlini et al. (2020) showed that word-for-word fragments of the training set, including names, phone numbers, and email addresses, can be extracted from GPT-2, and that larger models are more vulnerable than smaller ones. OWASP measures: cleaning and masking data before training and sending, the principle of least privilege, access control for sources, differential privacy and federated learning, clear retention rules, and a user’s right to opt out of training on their data. Bans in the system prompt are not reliable on their own.
Example
A manager pasted a CRM export with customer phone numbers into a public chatbot to get a summary. After that, the company set up corporate access to the model with phone number masking and banned uploading exports to personal accounts.
Related terms
Questions and answers
Sources
Not sure whether your company needs this?
AI solutions map