AI Supply Chain Security: Models, Datasets and the AI-BOM

AI supply chain security is about everything your AI system contains that you did not build: pre-trained and foundation models, public and purchased datasets, labelling work, ML libraries, notebook tools, MLOps platforms and hosted model APIs. In conventional software the supply chain delivers code you can inspect. In AI it also delivers data and learned parameters, whose behaviour you cannot fully inspect and whose origins are often layered several suppliers deep.

This guide covers the AI-specific supply chain threats ISO/IEC 27090 identifies, the controls it recommends, the model file risk that is easy to overlook, and how to keep a usable record of what your AI systems are made of.

What this guide covers

• Why AI supply chain security is different
• The AI supply chain attack vectors
• The six core AI supply chain security controls
• Model files: the risk hiding in plain sight
• AI supply chain security in practice: admitting an open-weight model
• Contracts and suppliers
• Provenance and the AI bill of materials
• AI supply chain security and regulation
• Frequently asked questions
• Putting AI supply chain security in place

Why AI supply chain security is different

ISO 27090 makes three observations that set AI apart.

• Data and models are supply. Pre-trained models and training, test and validation datasets can carry malicious manipulations, backdoors or poisoned examples, planted deliberately upstream.
• Development tools touch the crown jewels. Notebook environments, MLOps platforms and training frameworks run with access to training data, model parameters and feature pipelines, so a compromised tool reaches exactly what an attacker wants.
• Provenance is tangled. A model may be fine-tuned from a base model, trained on a mix of sources, labelled by a contractor and served through a third party. Each contributor can introduce a weakness that shows up only in the final system’s behaviour, where conventional testing may not find it.

The AI supply chain attack vectors

A manipulated model supplied by a third party can keep its unwanted behaviour after you fine-tune it. ISO 27090 calls this a transfer learning attack, and it is why fine-tuning someone else’s model does not make it yours from a security point of view.

The six core AI supply chain security controls

ISO 27090 recommends comprehensive AI supply chain security built from six measures. In our words, with how to apply each:

1. Verify provenance of data and models before first use: who produced it, from what, under what licence. Run a written intake assessment for every external model and dataset.
2. Require cryptographic signing of datasets and models, or at least published hashes, and verify them on every download, not just the first.
3. Use verified repositories. Pull models and data only from approved sources, and mirror admitted artefacts internally so production never depends on an external download.
4. Scan AI dependencies regularly. Put ML libraries and frameworks in the software composition analysis scope, and scan model files for malicious code.
5. Test supplied components for unexpected behaviour. Behavioural, backdoor and robustness tests before admission, and red teaming of supplied models used in high-risk systems.
6. Monitor model performance in operation for signs of compromise. A drift that appears after a supplier update is a question to ask the supplier.

These extend, not replace, conventional supplier controls such as those in ISO/IEC 27002 for supplier relationships, supplier agreements and the ICT supply chain.

Model files: the risk hiding in plain sight

ISO 27090’s Annex B makes a point that is easy to miss: a model weight file can carry ordinary malware. Backdoors and vulnerabilities can be used to embed code in a weight file that runs when the model is loaded. That makes downloading a model from a public hub much closer to running an unknown executable than to downloading a document.

The standard’s mitigations are practical:

• Use safe weight formats, such as safetensors, that store tensors only and cannot execute code on load.
• Scan model files for malicious code patterns and dangerous operations before admission.
• Verify provenance through checksums and signatures.
• Use trusted model repositories, and monitor at runtime for anomalous behaviour that could indicate a triggered backdoor.

A simple rule follows: no model file in an unsafe format runs on a workstation or training cluster unless it has been converted or explicitly approved.

AI supply chain security in practice: admitting an open-weight model

A team wants to fine-tune an open-weight model from a public hub for an internal classification task. A sound intake looks like this:

1. Identify the publisher. Confirm the model comes from the organisation it claims to, through the hub’s verified account or the publisher’s own site, not a lookalike.
2. Read the model card critically. Note the training data sources, licence, evaluation and known limitations. Missing training data information is a risk factor to record, not a gap to ignore.
3. Check integrity. Compare the downloaded files against the published hashes or signature.
4. Check the format. If the weights are in a format that can execute code on load, convert them to a safe format in an isolated environment, or reject them.
5. Scan. Run a model-file scanner and include the model’s library requirements in dependency scanning.
6. Test behaviour. Run evaluation, robustness and backdoor tests proportionate to how the model will be used.
7. Admit and record. Mirror the approved artefact internally, add it to the AI-BOM with its hash and intake reference, and only then let the team fine-tune it.

For a low-risk use the process is short. Skipping it is how a trojaned model ends up in production with nobody able to say where it came from, which is precisely what AI supply chain security is meant to prevent.

Contracts and suppliers

The standard’s Annex B adds supplier-side measures: put supply chain security into contracts with the parties who provide training data, pre-trained models and development work; educate suppliers on security; take particular care over exchanged specifications such as model cards; and red-team supplied models. Useful contract terms include:

• disclosure of training data origin, base models, licences and material pre-processing;
• signed or hashed deliverables in safe file formats;
• notice before the model, data or version behind a service changes;
• incident notification, including suspected poisoning;
• a right to test and red-team, and flow-down to subcontractors.

Provenance and the AI bill of materials

ISO 27090 recommends a provenance framework that tracks every critical AI component: training and inference data, architecture, software dependencies, configuration and data pipelines, and, for generative AI, prompts and fine-tuning data. Provenance records should be cryptographically protected, kept traceable through development, training, deployment and retraining, and retained for audit and incident review. Where feasible, standard formats such as those from the Coalition for Content Provenance and Authenticity (C2PA) or JPEG Trust help records interoperate.

In practice that record is an AI bill of materials (AI-BOM), our term for an SBOM extended to AI: every model, dataset, library, prompt template and tool a system contains, with origin, version, hash, licence and the intake assessment that admitted it. When a supplier announces a problem, the AI-BOM tells you in minutes which systems are affected.

AI supply chain security and regulation

EU AI Act Article 15(5) requires high-risk AI systems to resist attacks on pre-trained components used in training, which it calls model poisoning. A provider cannot evidence that without information and assurances from its suppliers. Our guide to EU AI Act Article 15 covers the rest of the article; on the data side of the same threat, see the data poisoning attack guide.

Frequently asked questions

Does AI supply chain security apply if we only call a hosted model API?

Yes. You depend on the provider’s model, data and change process. Intake assessment, contract terms on change notification and incidents, and monitoring for unexplained behaviour changes all apply.

Is fine-tuning a supplied model enough to remove a backdoor?

Not reliably. ISO 27090 notes that a manipulated model can keep its unwanted behaviour through further training. Treat supplied models as untrusted until tested.

What is the difference between an SBOM and an AI-BOM?

An SBOM lists software components. An AI-BOM adds the models, datasets, prompts and AI services, along with their provenance, which an SBOM does not capture.

Which document should we write first?

The intake assessment. It forces the provenance, integrity and documentation questions for every new model or dataset, and its records become the first entries in your AI-BOM.

Putting AI supply chain security in place

The order that works is intake first, then repositories and file formats, then contracts, then the AI-BOM and monitoring. For the full list of AI threats, start with the ISO 27090 overview.

Our ISO 27090 Toolkit includes an AI Supply Chain Security Procedure, a Third-Party Model and Dataset Intake Assessment, a contract-ready AI Supplier Security Schedule, a Model File and Backdoor Security Standard and an AI Provenance and AI-BOM Register, organised on the clauses of ISO/IEC 27090.