ISO Compliance Insights & Best Practices
CART
ISO Compliance Insights & Best Practices
An AI impact assessment in healthcare has to answer a question that most other sectors do not face so directly: could this system harm a patient? Tools that read scans, prioritise emergency calls, predict deterioration, recommend treatment or schedule staff all touch clinical outcomes, some directly and some indirectly. Errors can cost health or life, and biased performance can widen existing health inequalities.
This guide explains how to run an AI impact assessment in healthcare: how to scope the system, which impacts to consider, how to test for performance across populations, how to handle health data and consent, how to design clinician oversight and how to monitor after deployment. It is general guidance, not medical or legal advice, and health technologies are heavily regulated, so involve clinical safety and regulatory specialists.
Healthcare combines high stakes, vulnerable people, sensitive data and complex workflows. A small error rate that is acceptable in a retail recommendation engine is not acceptable in cancer screening. Patients often cannot judge or challenge what a system does, and clinicians may trust a tool more than its evidence supports.
Many healthcare AI products are also regulated as medical devices, with clinical evaluation and safety standards. An impact assessment does not replace those requirements. It complements them by looking at the wider effects on patients, staff and the health system, including fairness and access. The WHO guidance at the WHO guidance on ethics and governance of artificial intelligence for health sets out ethical principles for AI in health, such as protecting autonomy, promoting well-being and safety, ensuring transparency, fostering accountability and promoting inclusiveness.
Describe what the tool does, in which clinical pathway, for whom and with what degree of autonomy. Is it advisory, or does it act? Who uses it, clinicians or patients? What data does it need? What decision does it inform, and what happens if it is wrong?
Screen the system to confirm the depth of assessment needed; see AI impact assessment screening. Record the regulatory status: is it a medical device, and what evidence supports its intended use? Note the intended population and the settings it was validated in, since performance often falls outside them.
Free AI impact assessment (ISO 42005)
Who could this AI system affect, and how?
Screen the system against sensitive and prohibited uses, describe it, check the safeguards for fairness, transparency and oversight, and rate its impacts on people and society from 26 scenarios with ISO 42001 Annex A measures. Free, with findings.
Start the free AI impact assessment → or View premium report sample
List impacts on patients: missed or delayed diagnosis, unnecessary treatment, false reassurance, loss of privacy, reduced human contact and unequal access. For staff: changes in workload, skills, autonomy, alert fatigue, moral distress and liability concerns. For the system: costs, capacity and trust.
Include groups likely to be affected differently: older people, children, women, ethnic minorities, people with disabilities, those with rare conditions and patients with limited digital access. Involve patients, carers and clinicians in the assessment; see AI impact assessment stakeholders.
| Type of tool | Example | Main impact concern |
|---|---|---|
| Diagnostic support | Image analysis for cancer screening | Missed or false findings, unequal accuracy |
| Triage and prioritisation | Emergency call or waiting list ranking | Delays for under-recognised groups |
| Prediction | Risk of deterioration or readmission | Overreliance, alert fatigue |
| Generative tools | Clinical note drafting or patient chat | Errors, fabricated content, privacy |
| Operational | Bed and staff scheduling | Workload, indirect effects on care |
Overall accuracy hides differences. Ask for evidence of performance across age, sex, ethnicity, skin tone, comorbidities, device types and care settings. Compare the validation population with your own patients. Where evidence is missing, run local validation before deployment.
Look at both false negatives and false positives. In screening, a missed case may be catastrophic, while excess false positives may cause anxiety, unnecessary tests and cost. Document the trade-off chosen and why. Use fairness testing methods described in AI bias testing, adapted to clinical contexts.
Health data is special category data under data protection law, and processing needs a lawful basis and an additional condition. Check how the data for training and operation was obtained, whether patients were informed, and whether secondary use is lawful and expected. Complete a DPIA where required; see DPIAs for AI systems.
Tell patients when AI is involved in their care, in plain language, and explain how they can ask questions or opt out where appropriate. Transparency builds trust and supports informed consent. Record how you decide what to disclose, and how you handle patients who object.
Clinicians need to be able to understand, question and override the tool. Provide training on the tool’s purpose, limits and known failure modes. Design the workflow so the clinician has time and information to review, avoid interfaces that encourage blind acceptance and make clear who is accountable for the final decision.
Watch for automation bias and alert fatigue. Track overrides and near misses. See human oversight of AI for design principles. Involve the clinical safety officer or equivalent, and follow local clinical risk management standards, which may require a hazard log and safety case.
Match safeguards to impacts: local validation, subgroup testing, clinician training, fallback procedures when the tool is unavailable, alert thresholds, audit of outputs, patient information, data protection controls and incident reporting. Assign owners and dates.
Rate the residual impact using a consistent scale, such as that in AI impact assessment severity rating. In healthcare, irreversibility and vulnerability weigh heavily. If the residual impact is not acceptable, narrow the use, add controls or do not deploy.
Clinical performance changes as populations, devices, protocols and data change. Monitor outcomes and performance by group, override and complaint rates, near misses and adverse events. Compare them with the validation results. Set thresholds for review and processes for rapid suspension if safety concerns arise.
Report to clinical governance committees, and integrate with existing patient safety incident systems; see AI incident management. Keep the assessment current through the approach in AI impact assessment monitoring.
Frequent errors include relying on vendor performance claims without local validation, ignoring subgroup performance, treating the assessment as a data protection exercise only, giving clinicians no training, failing to plan for tool downtime, neglecting patient information and never monitoring after go-live. Another is assuming regulatory clearance answers all questions about local use.
Avoid these by involving clinicians and patients, testing locally, planning fallbacks and monitoring outcomes.
Before buying a clinical AI tool, ask the vendor for its intended use, clinical evidence, populations and settings studied, known limitations, update policy and post-market monitoring. Ask how it handles data, where data is stored and whether data is used to train other models. Require notification of significant changes, and agree who is responsible for incidents. Weak answers are findings to record and to weigh in the decision to buy.
A hospital trust plans to use an AI tool that flags chest X-rays with possible lung nodules. The assessment scopes the tool as advisory for radiologists. Vendor evidence is strong overall but thin for patients with certain chest conditions and for images from one older scanner type used at a satellite clinic.
The trust runs local validation, finds lower sensitivity on the older scanner, restricts use to newer devices at first, trains radiologists on the tool’s limits, informs patients and sets up monthly monitoring of flags, overrides and missed findings by subgroup. Residual impact is accepted by the clinical safety committee with a review in six months.
Not all healthcare AI is diagnostic. Tools that draft notes, summarise records, answer patient queries or automate coding are growing fast. Their risks include fabricated content, leakage of sensitive data, subtle errors that enter records and overreliance. Require clinician review of any content that enters the record, restrict sensitive data in prompts and test outputs for accuracy.
Include administrative tools in your assessment process at a proportionate depth. A scheduling tool needs less analysis than a clinical decision support tool, but patients can still be harmed by systematic delays or exclusion.
If you want a report and workbook with screening, impacts on patients and staff, safeguards, measures and review in one place, the AI Impact Assessment Report and Workbook provides a structured layout built around ISO/IEC 42005. Whatever tool you use, a thorough AI impact assessment in healthcare puts patient safety first, tests for fairness and keeps watching after launch.
Regulatory approval covers safety and performance for the intended purpose, but an impact assessment still helps examine local use, fairness, staff and patient effects and data protection.
Harm from wrong outputs, especially missed diagnoses, combined with overreliance by clinicians and unequal performance across patient groups.
Transparency is good practice and often expected or required. Explain in plain language how AI is used in their care and how they can ask questions.
Yes, where possible. Performance can differ in your population, devices and workflows, so local validation is a key safeguard.
Track performance and outcomes by group, overrides, near misses and adverse events, with thresholds for review and a process for rapid suspension.
September 30, 2026
