Actions
What to do now about AI data breach risk
Start by finding out where AI touches your data, then set rules, restrict access and make sure your breach response covers AI. The steps below are in the order most organisations can complete them.
Last reviewed against legislation.gov.uk, National Cyber Security Centre and OWASP GenAI Security Project. Sources
Where to begin
Knowledge before controls
You cannot protect data flows you have not found. The NCSC's advice to leaders is to understand their data, model and machine learning software supply chains and to ask suppliers the right security questions [3]. That includes tools staff adopted without telling IT, which the IBM study calls shadow AI [5].
Work through the actions below, then use the guides for detail on reporting, containment and prevention.
Who owns what
Decisions to make
Four decisions need a named person before anything else.
Accountability
The NCSC says to clarify who is accountable for AI security and to understand the worst case, operationally and for reputation, if an AI tool fails [3].
Reportability
Someone must be able to judge quickly whether a breach is reportable, because the UK GDPR deadline runs from when you become aware of it [1].
Supplier contact
Know who to call at each AI supplier, and what their incident notification process is [3].
Record keeping
The controller must document each personal data breach, its effects and the action taken [1].
A short check
Questions for your next leadership meeting
The NCSC suggests leaders ask technical and security staff whether security factors into decisions to use AI products and how AI risks fit into existing governance [3].
Check these as well:
- Does every AI tool in use appear on a list that IT keeps?
- Is there a written rule on what staff may enter into an AI tool?
- Can anyone reach prompt logs or chat history from the internet?
- Does the incident plan mention AI tools and AI suppliers?
- Is there a named person who decides whether to report to the ICO?
Plan for it
Build AI into the incident plan
The NCSC says to have a response plan for serious AI-related security incidents [3]. Use the containment guide as a starting point and test it with a realistic case, such as a supplier telling you that your prompts were visible to others, as happened in the ChatGPT incident [7].
What to do
Six steps, in order
-
Step 1
Find where AI touches your data[3]
List every AI tool and service in use, who uses it and what data goes in, including tools staff chose themselves. The NCSC tells leaders to understand their data, model and software supply chains.
Who: IT, security and data protection leads
-
Step 2
Set a clear rule on what may be entered[4]
Say in writing which kinds of data must never go into an AI tool and tell staff why. OWASP advises teaching users not to submit sensitive information and publishing clear data retention policies.
Who: Directors, HR and line managers
-
Step 3
Restrict access to AI data stores and logs[5]
Apply least privilege to prompt logs, chat history, vector stores and API keys, and check nothing is reachable from the internet without authentication. In the IBM study 97% of breached organisations with an AI incident lacked proper AI access controls.
Who: IT and cloud teams
-
Step 4
Prepare the breach response[1]
Add AI incidents to your response plan, name who decides whether a breach is reportable and keep a record of every decision. The 72 hour clock starts when you become aware of a breach.
Who: Data protection officer and incident lead
-
Step 5
Ask AI suppliers the security questions[3]
Ask how your data is stored, whether it is used for training, who can see it and how incidents are notified. The NCSC says leaders should ask suppliers the right security questions.
Who: Procurement and security leads
-
Step 6
Record breach decisions, including the ones you do not report[1]
UK GDPR requires the controller to document any personal data breach, its effects and the action taken, even where it is unlikely to result in a risk.
Who: Data protection officer
Keep reading
Read next
Timeline
The public AI data exposures and the UK guidance that followed, in date order and sourced, from the March 2023 ChatGPT bug to the 2025 IBM research.
Read the guide
Shadow AI
How staff use of unapproved AI tools leaks company and personal data, what IBM and Samsung's experience show, and how to find and reduce it.
Read the guide
Exposed databases
What Wiz found in an open DeepSeek database, why it matters for any AI service and the checks UK teams can make on their own AI data stores.
Read the guideHow CyPro can help
Need a hand working through this?
Finding out whether an incident reaches you is one job. Checking your own systems, tightening access and briefing the board are several more. CyPro's responders and consultants can take any of them on.
Incident response
If you may already be compromised, our responders contain it, preserve the evidence, find out what was touched and get you running again safely.
More on cypro.co.uk
24/7 security monitoring
Our SOC analysts watch your systems day and night for the signs of an attack in progress: unexpected processes, new outbound connections, logins that do not fit.
More on cypro.co.uk
Fractional CISO for the board
A senior security lead who advises the board on what to fix first, what to accept and what to tell customers, and owns the decision record.
More on cypro.co.uk
Penetration testing
A focused test of your systems and the controls around them, so you know they hold before an attacker finds out for you.
More on cypro.co.uk
Hardening and segmentation
Our consultants put the controls in place that limit the damage when something is missed: firewall rules, network isolation and tighter admin access.
More on cypro.co.uk
Vulnerability scanning
Find every internet-facing system you run, including the forgotten ones, and see which are still on a vulnerable build.
See the scanning service
Talk to a specialist
Worried this affects you?
If data has leaked through an AI tool or an exposed AI database, the first hours decide how much is lost and what you must report. An incident response team helps you contain it, work out what was exposed and meet your notification duties.