Third-Party AI Impact Assessment Guide 2026

A third-party AI impact assessment examines how an AI system that you buy or license, rather than build, could affect the people it touches in your organization’s hands. Most organizations now use AI supplied by others: a resume screening feature in an HR platform, a fraud score from a payments provider, a chatbot from a software vendor, or a foundation model behind an API. You cannot see inside these systems, yet you remain accountable for what they do in your processes.

This guide explains how to scope and carry out a third-party AI impact assessment, what to ask suppliers, how to handle missing information, which contract terms help, and how to keep the assessment current as vendors change their models.

Why a third-party AI impact assessment is needed

ISO/IEC 42001:2023 asks an organization to assess the impact of AI systems on individuals and society, and its Annex A includes a group of controls on third-party and customer relationships, covering the allocation of responsibilities and the handling of suppliers. You can view the standard’s listing at ISO/IEC 42001 on iso.org. Our guide to the ISO 42001 impact assessment covers the core requirement.

Under the EU AI Act, providers of high-risk systems must give deployers instructions for use and information needed to use the system properly, and deployers have their own duties, such as using the system according to the instructions and assigning human oversight. Buying a system does not transfer the deployer’s responsibility. The impact of a tool depends heavily on how and where you use it, so the supplier cannot do the assessment for you.

What is different when you do not control the model

With an in-house system you can inspect data, change the model and test freely. With a vendor system you usually cannot. Three differences follow. Information is limited and shaped by the supplier’s interests. Changes happen on the supplier’s schedule, sometimes without notice. And your controls sit mostly at the edges: what you feed in, what you do with outputs, and who reviews decisions. The assessment should therefore focus on your deployment context and on the evidence you can obtain.

Question	In-house system	Third-party system
Who knows the training data?	You	Supplier, often only in summary
Who tests for bias?	You	Supplier, plus your own testing on your data
Who controls updates?	You	Supplier, often automatically
Where are your controls?	Throughout the life cycle	Inputs, outputs, oversight and contract

Scoping the third-party AI impact assessment

Start with an inventory of all AI features you use, including ones embedded in ordinary software. Ask each business team what tools they use and check supplier release notes for new AI features. Then screen each item using the same questions you apply to internal systems: does it make or influence decisions about people, does it use sensitive data, does it affect access to services, and how much human oversight exists? Our guide to AI impact assessment screening provides a set of questions. Only systems that pass the screen need the full assessment, but record why the others did not.

Shadow AI and free tools

Staff often use public AI tools without approval, pasting internal or personal data into them. Include this in the scope by asking teams, reviewing web and network logs where lawful, and providing approved alternatives. An outright ban tends to push use out of sight, while a clear policy with approved tools brings it back into the assessment process.

What to ask suppliers

Send a focused questionnaire rather than a generic security form. The most useful questions cover the following areas, and the answers become part of your record.

1. Intended use and limits. What is the system designed for, and what uses does the supplier advise against?
2. Data and training. What kinds of data were used, whether customer data is used for training, and how it is protected.
3. Testing. What accuracy, robustness and fairness testing has been done, on which populations and with what results.
4. Oversight and explainability. What information does the system give to reviewers, and can decisions be explained?
5. Change management. How are model changes announced, and can you delay or opt out of updates?
6. Incidents. How are AI-specific failures reported and handled?
7. Sub-suppliers. Which foundation models or services does the supplier depend on?
8. Certifications. Does the supplier hold ISO/IEC 42001 or similar assurance, and what does the scope cover?

Handling gaps in supplier information

Suppliers often decline to share details, citing confidentiality. Do not treat silence as reassurance. Record what was asked, what was received and what was refused, then reflect the gap in your rating. A system used for a high-impact decision with no evidence of testing should rate higher than one with published test results. Possible responses include running your own tests on your data, limiting the use to lower-impact decisions, adding human review, or choosing a different supplier. Escalate the choice to whoever can accept the residual risk.

Testing in your own context

Evidence gathered from the supplier is only half of a sound third-party AI impact assessment; the other half comes from what you observe yourself, and it deserves equal weight when you rate the result.

Whatever the supplier says, the behavior in your process is what matters. Run a pilot on representative data before launch, and compare outputs against known correct answers. For decisions about people, check error rates and outcomes across relevant groups where the law allows you to do so. Test what happens with unusual inputs, such as accents, unusual names, low-quality scans or non-standard formats. Keep the results with the assessment. Our article on AI impact assessment stakeholders explains who to involve in reviewing results, including those affected.

Contract terms that support the assessment

You can only manage what you can see, so put visibility into the contract. Useful terms include the duty to notify material model changes in advance, the right to receive updated documentation and test results, commitments on data use and training, incident notification, audit or assessment rights, cooperation with regulatory requests and clear allocation of responsibility. For SaaS tools, see our guide to SaaS vendor risk assessment, and for the wider process see the third-party risk management framework.

Rating and recording the third-party AI impact assessment

Use the same scales as for internal systems: severity, likelihood and reversibility, with inherent and residual ratings. Rate the impact in your deployment, not the supplier’s marketing case. Record the affected parties, the decisions influenced, the safeguards you control and the assumptions about the supplier’s controls. Link the outcome to your vendor risk tiering, so that high-impact AI suppliers get closer oversight. See our guide to vendor risk tiering for how to classify them.

A short worked example

A bank licenses a transaction-monitoring tool with an AI scoring feature. Screening shows that the scores influence account restrictions, so a full assessment is carried out. The supplier provides a technical summary and validation results but declines to share training data details. The bank runs a pilot on twelve months of its own alerts, finds a higher false-positive rate for customers with irregular income patterns, and adds a human review step and a customer route to contest restrictions. The contract is amended to require notice of model changes. The residual impact is rated moderate and accepted by the head of financial crime with a review in six months.

Keeping the third-party AI impact assessment current

Vendors change models often, so review triggers matter: supplier release notices, new features, changes of sub-supplier, incidents, complaints and changes to your own use. Ask relationship owners to report changes, and review high-impact suppliers at least annually. Track a few measures, such as the share of AI suppliers assessed, the number with unresolved evidence gaps and the time to reassess after a model change.

Using a ready structure

To avoid building the forms yourself, the AI Impact Assessment Report and Workbook provides a structured report with screening, impacts on people and society, safeguards and a working register that you can use for vendor systems as well. Whichever format you choose, make the third-party AI impact assessment a routine step in procurement, not an afterthought once the tool is live.

Third-party AI impact assessment FAQ

Do I need to assess AI that a vendor supplies?

Yes, where it affects people in your processes. You remain accountable for how you use the system, and the impact depends on your deployment context.

What if the supplier will not share details?

Record what you asked and what you received, reflect the gap in your rating, test the system on your own data and consider limiting the use, adding human review or choosing another supplier.

Is a supplier’s ISO 42001 certificate enough?

It is useful assurance about the supplier’s management system, but it does not assess the impact of a specific system in your deployment. Check its scope and still assess your use.

How often should I reassess vendor AI?

Reassess when the supplier changes the model, when your use changes, and after incidents. High-impact systems should also be reviewed at least annually.

Who should own it?

The business owner of the tool should own the assessment, with support from procurement, security, privacy, legal and the AI governance lead.