ISO Compliance Insights & Best Practices
CART
ISO Compliance Insights & Best Practices
AI impact assessment monitoring is the work that begins when the assessment is signed. An impact assessment predicts how a system could affect people, but predictions are made before real users, real data and real-world messiness arrive. Only monitoring after deployment shows whether the impacts you expected are the ones that actually occur, and whether new ones have appeared.
This guide explains how to plan AI impact assessment monitoring: what to measure, where feedback comes from, which triggers should prompt a review, how to assign responsibilities and how to keep the assessment record current. It is general guidance that follows the spirit of ISO/IEC 42005, and it should be adapted to your systems and to the regulations that apply.
Impact assessments are based on assumptions about the data, users, context and controls. Those assumptions weaken with time. Data drifts, users find new ways to use the system, the deployment expands to new groups and regulations change. If the assessment is not revisited, it becomes a comforting record of a system that no longer exists.
Standards and regulators expect ongoing attention. ISO/IEC 42005 describes impact assessment as a process that includes monitoring and review, as set out in ISO/IEC 42005 on AI system impact assessment, and ISO/IEC 42001 expects assessments to be repeated after significant change. AI impact assessment monitoring shows that the organization is still watching the effects on people, not only the technical performance of the model.
Go back to the impacts identified in the assessment and ask how each one would show up in the real world. Choose indicators that reflect impact on people, not only model metrics. For a hiring tool, monitor selection rates by group as well as accuracy. For a triage tool, monitor waiting times and outcomes for different patient groups. For a chatbot, monitor harmful outputs and complaints.
Include indicators for the safeguards you relied on: are human reviewers really overriding, are appeals being answered on time, are notices reaching people? Link each indicator to a threshold that would signal concern. See AI impact assessment severity rating for how to weigh what you find.
Numbers do not capture everything. Set up channels for users, staff, affected individuals and their representatives to report problems: contact forms, hotlines, appeal routes, user forums and periodic surveys. Make them easy to use and accessible to people with different needs.
Where the people affected are not your users, such as job applicants or benefit claimants, consider outreach through advocacy groups or community organizations. Their experience may reveal harms invisible in the data. Record the feedback received and how it was handled. Our guide to AI impact assessment stakeholders explains how to identify them.
| Signal source | What it tells you | Example indicator |
|---|---|---|
| System performance data | Accuracy and error patterns across groups | Error rate gap between groups |
| User and reviewer feedback | Problems seen in use | Override rate, complaints about outputs |
| Affected people and advocates | Real-world harm | Appeals, reported unfair outcomes |
| Incident and near-miss reports | Failures and misuse | Number and severity of AI incidents |
| External change | New law, technology or context | Regulatory updates, new user groups |
Do not rely on calendar reviews alone. Define events that trigger an unscheduled reassessment: significant model updates or retraining, new data sources, extension to new user groups or purposes, indicator thresholds being crossed, serious incidents, complaints of unfair treatment, and changes in law or guidance. Connect the triggers to change management, so a project team cannot quietly change a system without prompting a review.
Also set a fixed cycle, for example annually for higher-risk systems and every two years for lower-risk ones. Record the outcome of each review, even when nothing changes. Our AI model drift guide describes technical signals that often justify a review.
Someone must own the monitoring. Usually the system owner is accountable, with data science or operations collecting indicators, privacy and legal advising, and a governance forum receiving reports. Name people and write down what each will do and how often.
Include the supplier where a third party provides the model. Ask what monitoring data and change notices they will supply, and build the requirements into the contract. See third-party AI impact assessment for supplier questions. A gap in supplier information is itself a finding to record.
Review indicators at a regular rhythm, monthly for high-risk systems and quarterly for others. Compare results with thresholds, look at trends and segment the data by relevant groups. Investigate anomalies rather than accepting them: a sudden fall in appeals may mean the system is fairer, or that people no longer know how to appeal.
When a threshold is crossed, act: investigate, adjust the system, strengthen safeguards, retrain reviewers, notify affected people or suspend the feature. Feed serious cases into incident processes, as described in AI incident management.
Update the impact assessment whenever monitoring shows a change in impacts, likelihood, severity or safeguard effectiveness. Keep earlier versions and a change log with dates and reasons. Update the related records too: the AI risk register, the data protection impact assessment where personal data is involved and the model inventory.
Free AI impact assessment (ISO 42005)
Who could this AI system affect, and how?
Screen the system against sensitive and prohibited uses, describe it, check the safeguards for fairness, transparency and oversight, and rate its impacts on people and society from 26 scenarios with ISO 42001 Annex A measures. Free, with findings.
Start the free AI impact assessment → or View premium report sample
Record positive findings as well. If monitoring shows that a safeguard works well, that is useful evidence for the next review and for stakeholders who ask how you know the system is safe.
Report monitoring results to the governance forum on a schedule, showing the indicators, trends, incidents, actions and changes to the assessment. Highlight anything that exceeds tolerance and the response. Senior leaders should see whether impacts are moving in the right direction.
Consider what to share externally. Some organizations publish summaries of impact assessment results and monitoring outcomes, especially in the public sector. Even where you do not publish, be ready to explain the approach to regulators, customers and affected people in plain language.
Frequent errors include monitoring only technical metrics, ignoring feedback from affected people, having no thresholds, relying only on annual reviews, failing to assign an owner, not connecting model changes to review, losing the link between monitoring and the assessment and reporting only good news. Another is monitoring the average when harm falls on a specific group.
Avoid these by tying indicators to identified impacts, disaggregating results, setting triggers, assigning owners and reporting candidly.
Look across your portfolio, not just at individual systems. If the same kind of impact keeps appearing, such as poor performance for one language group or a weak appeal process, fix it in the standard template, the procurement checklist or the development guidance. Share anonymised lessons between teams, and use them to sharpen screening questions for new projects. Over time, monitoring becomes a source of organizational learning rather than a series of isolated checks.
A housing agency uses a model to prioritise repair requests. The assessment identified a risk that residents with limited English would be disadvantaged. After launch, the agency monitors response times by language group, appeals and complaints, and reviewer overrides. In month four, response times for one language group are longer by 30 percent, crossing the threshold.
The team investigates, finds that free-text descriptions in that language are poorly understood by the model and adds a translation step and human review of low-confidence cases. The assessment is updated, and monitoring confirms the gap closes. The record shows how the issue was found, addressed and verified.
Not every system needs intensive monitoring. Scale effort to the impact rating: high-impact systems need frequent, detailed monitoring and active feedback channels, while low-impact systems may need periodic checks and incident reporting only. Document the reasoning, so you can show that the approach was considered.
Reuse existing processes where possible. Many indicators already exist in operations, customer service and quality dashboards, and adding an impact lens to them is far cheaper than building new systems. Start with a small set of well-chosen measures, and expand as the programme matures.
If you want a report and workbook that carry screening, impacts, safeguards, monitoring measures and review dates in one place, the AI Impact Assessment Report and Workbook provides a structured layout built around ISO/IEC 42005. Whatever tool you use, effective AI impact assessment monitoring watches the real-world effects on people, acts when they change and keeps the record honest.
Because the assessment rests on assumptions that may not hold in practice. Monitoring shows whether expected impacts occur and whether new ones appear.
Indicators linked to the identified impacts, including outcomes by group, complaints, appeals, overrides, incidents and the effectiveness of safeguards.
Significant model changes, new data or user groups, new purposes, crossed thresholds, serious incidents, complaints of unfair treatment and changes in law or guidance.
The accountable system owner, supported by data science, privacy, legal and operations, with reports to a governance forum.
At least annually for higher-risk systems, and more often where indicators or triggers require it. Lower-risk systems may be reviewed less frequently.
September 30, 2026
