Last week I ran a poll for the data protection community. The question was simple. When you complete a DPIA for AI systems, what are you actually assessing, the technology or the processing activity it enables. Seventy one people voted. Four per cent said the technology. Thirty per cent said the processing activity. Sixty one per cent said honestly, a bit of both. Two per cent were not sure, and I liked that answer too.

I promised an article setting out my own thinking. Here it is, along with what UK GDPR actually says, why the bit of both instinct is closer to correct practice than it sounds, and where that instinct can still send a DPIA off course if the scope is not set properly.

What is a DPIA for AI systems?

A data protection impact assessment, or DPIA, is a structured process required under UK GDPR Article 35 whenever a type of processing is likely to result in high risk processing that affects the rights and freedoms of individuals. It identifies risks before processing begins and records the measures taken to reduce them. It is a process applied to an activity, not a review of a product.

For an AI system, that means the DPIA sits around whatever the system is being used to do. Screening, scoring, drafting, redacting or summarising personal data are all processing activities. The model behind them is the tool, not the subject of the assessment. Building data protection by design into this stage, rather than after deployment, is what keeps the assessment meaningful.

What does UK GDPR actually require a DPIA to assess?

Article 35 asks for a description of the envisaged processing operations and their purposes, an assessment of necessity and proportionality, and an assessment of the risks to individuals. Nowhere does the legislation name the software, the vendor or the model architecture as the thing being assessed.

The ICO frames the same requirement as describing the nature, scope, context and purposes of the processing. 

• Nature covers what you plan to do with personal data, from collection through to retention and security.
• Scope covers the volume and sensitivity of data and the number of people affected.
• Context covers your relationship with those individuals and what they would reasonably expect.
• Purpose covers why you are doing it and what outcome you expect for them.

None of those four elements is the AI system. All four are about the activity the system supports.

This is exactly the point Simon Howarth raised in the comments under the poll. A DPIA has one function, to identify and help mitigate the risks of the project to the people affected by it, not to review a system in isolation. Legally, he is right.

Why did 61 per cent say it is a bit of both?

The poll reached 1,721 impressions and drew 71 votes from a practitioner audience. A third of respondents were senior, and Data Protection Officer was the single most common job title. This was not a general audience guessing. It was working data protection people describing what they actually do.

Sixty one per cent chose bit of both. That is not confusion about the law. It is experience talking. Anyone who has written a DPIA for a machine learning tool knows the processing activity cannot be assessed honestly without opening up the system that performs it.

Is a DPIA about the AI system or the processing activity?

Legally, a DPIA assesses the processing activity, not the AI system on its own. Practically, the ICOs guidance on the accountability and governance implications of AI treats the characteristics of the system, its data, its accuracy and its behaviour, as essential components of that same processing assessment, not as a separate exercise sitting beside it.

This is where a lot of AI accountability advice gets muddled. The ICOs guidance sets out what a DPIA for an AI system should include: a systematic description of the processing activity, including the data flows and the stages at which the AI produces effects on individuals. It also expects an explanation of any relevant variation or margin of error in how the system performs, because that variation affects the fairness of the processing itself.

So the technology is in scope. It is in scope because of what it does to the processing, not because it deserves its own separate audit.

Why does the AI system still matter inside a processing focused DPIA?

A processing led DPIA still has to examine training data choices, model accuracy, security controls and the potential for bias, because each of these changes how the processing affects individuals. This is, in effect, an algorithmic risk assessment carried out inside the processing DPIA rather than as a document of its own. The ICO specifically expects assessment of allocative harms, where a decision denies someone an opportunity or a resource, and representational harms, where a system reinforces a stereotype about a group.

Machine learning systems can reproduce discrimination that already exists in historic data. A recruitment tool trained on ten years of hiring decisions will happily inherit the patterns of that decade unless someone checks for them. That check belongs inside the DPIA for the recruitment processing activity itself. It does not need a separate, standalone assessment of the system as a product.

Trade offs matter too. A model tuned for higher statistical accuracy sometimes needs more personal data, which pulls against data minimisation. Weighing that trade off is squarely a DPIA question, and nobody can answer it without understanding how the system actually works.

What goes wrong when you DPIA the system instead of the activity?

Get the scope of a DPIA for AI systems wrong and the mistake compounds fast. Scoping a DPIA around an AI system rather than around each processing activity it supports tends to undercount the assessments an organisation actually needs. It also produces documents too generic to catch the risks specific to each use, such as who the data subjects are, what decision is made about them, and what happens if that decision is wrong.

Most organisations do not buy an AI system for a single purpose. A language model procured for drafting internal reports can quietly end up supporting a customer chatbot, redacting personal data in subject access requests, and summarising HR case files within the year. Each of those is a distinct processing activity, with different data subjects, different purposes and a different risk profile. One assessment of the system cannot responsibly cover all three.

Does a DPIA for an AI system end once it goes live?

No. A DPIA for an AI system is a living record, not a one off form completed before launch. UK GDPR expects ongoing review, and AI systems change behaviour over time as models are retrained, prompts are adjusted, or the same tool is pointed at a new data set, so the original assessment can go stale quickly.

Set a review trigger rather than a review date. A meaningful trigger might be a model update that changes how outputs are generated, a new data source feeding the system, or evidence from monitoring that the error rate has shifted. Any of these can move risk in ways a calendar reminder alone would miss.

This is another reason the processing focused approach holds up better than a system focused one. A processing activity has a defined purpose you can check against, month by month. A system, left to its own devices, keeps evolving, and there is no obvious point at which reviewing the software rather than the activity would tell you whether people are still being treated fairly.

Build monitoring into the DPIA itself. Record who checks model performance, how often, and what threshold would trigger a fresh assessment. That turns the document from a compliance record into a working safeguard, which is the outcome UK GDPR is actually asking for.

How should you scope a DPIA for a new AI system in practice?

Start by listing every distinct purpose the AI system will be used for across the organisation, then run the ICO guidance on high risk screening against each purpose separately. UK GDPR attaches the high risk test to the processing, not to the tool that happens to be doing it.

• List every purpose the system will be put to, not only the one that justified the original purchase.
• Screen each purpose against the ICO guidance on high risk criteria on its own terms.
• For each purpose that needs a DPIA, describe the nature, scope, context and purpose of that specific processing.
• Fold in the relevant system characteristics for that processing: the training data, the accuracy metrics, the security controls, and the points where a human reviews the output.
• Revisit the DPIA whenever the system is put to a new purpose. A new purpose is a new processing activity, not a minor system update.

How does the ProvePrivacy platform help you scope AI DPIAs correctly?

If you want a fuller walkthrough of AI governance readiness, our guide on DPIA software for AI systems covers the wider picture. Our DPIA knowledge base article is also a good reference for the screening test itself.

The ProvePrivacy platform is built around processing activities rather than around individual tools, which matches how UK GDPR actually asks organisations to think, and embeds data protection by design into the workflow itself. It lets a data protection team record every purpose an AI system supports as its own entry, link several activities back to a single shared system, and see at a glance where risk assessments are missing.

When a new purpose is added for an existing system, the ProvePrivacy platform prompts a fresh screening question rather than assuming the earlier DPIA still applies. That keeps assessments specific to real risk, and keeps the record ready to show a regulator exactly what was considered, and when.

So was the poll right?

The poll was right in substance and imprecise in law. Sixty one per cent of respondents were describing good practice, not the legal test itself. UK GDPR asks you to assess the processing activity. A processing activity built on AI cannot be assessed honestly without examining the system performing it, which is why practice and law arrive at the same place from different directions.

Simon Howarth was also right, and his challenge is the one worth keeping in mind. Start with the project and the people it affects. Bring the system in because it changes their risk, not because it is new or interesting. Scope the DPIA for AI systems to the purpose, and the technology will find its proper, supporting place inside it.

Frequently asked questions

Does a DPIA assess the AI system or the processing activity?
UK GDPR Article 35 requires an assessment of the processing activity, described through its nature, scope, context and purpose. The AI system is examined only for how its characteristics, such as training data and accuracy, affect that processing.

Do you need a separate DPIA for every use of the same AI system?
Yes, where each use is a distinct processing activity with its own purpose and risk profile. A single system level DPIA will usually miss risks that are specific to one particular use.

What does the ICO expect a DPIA for an AI system to include?
The ICO expects a description of the processing activity and data flows, the stages where the AI affects individuals, and an explanation of any margin of error in system performance that could affect fairness.

Does a DPIA need to be updated after an AI system goes live?
Yes. UK GDPR expects ongoing review, and AI systems change over time through retraining and new data sources, so a DPIA should include a monitoring plan with clear triggers for reassessment rather than a single review date.

Sources

• ICO, Data Protection Impact Assessments (DPIAs): ico.org.uk
• ICO, When do we need to do a DPIA: ico.org.uk
• ICO, How do we do a DPIA: ico.org.uk
• ICO, What are the accountability and governance implications of AI: ico.org.uk
• UK GDPR, Article 35, Data protection impact assessment: uk-gdpr.org