AI data breach: your questions answered

It is a compromise that exposes data used by or produced by AI tools, such as prompts, chat logs, outputs or training data. OWASP lists personal information, financial and health records, confidential business data and credentials as data an LLM application can disclose[4]. The legal test is the same as for other incidents: whether personal data was exposed.

If it is a personal data breach, UK GDPR requires notification not later than 72 hours after becoming aware of it, unless it is unlikely to result in a risk to people's rights and freedoms[1]. If you report late you must give the reasons for the delay[1].

Yes. The controller must document any personal data breach, including its facts, effects and the remedial action taken[1]. Keep the reasoning for deciding that it was unlikely to result in a risk.

It is AI use that the organisation has not sanctioned or does not control, such as staff using public chatbots for work. IBM's 2025 study found that high levels of shadow AI added USD 670,000 to the average breach cost[5].

OpenAI attributed a March 2023 incident to a bug in redis-py, an open-source library. It exposed some users' chat queries and payment details for about 1.2% of ChatGPT Plus subscribers active in a nine-hour window[7]. OpenAI said full credit card numbers were not exposed[7].

Wiz reported a publicly accessible ClickHouse database with no authentication that held over a million log entries, including chat history and API keys[6]. Wiz said it disclosed the issue and DeepSeek secured it promptly[6].

OWASP describes training-data inversion attacks and personal information leakage as examples of sensitive information disclosure by LLMs[4]. It advises scrubbing sensitive content before it enters training data and not relying on system prompt restrictions alone[4].

Build security in from the start, clarify who is accountable for AI security, have a response plan for serious AI incidents and ask suppliers the right security questions[3]. The NCSC's guidelines for secure AI system development cover design, development, deployment and operation[2].

Samsung banned generative AI on company devices and networks but described the ban as temporary while it reviewed safer ways to use it[8]. The alternative is a written rule on what data may be entered, plus approved tools. See the AI use policy guide.