Data Poisoning Attacks: Detection and Defence Under ISO 27090

A data poisoning attack changes what a model learns by changing what it learns from. The attacker inserts, alters, mislabels or reorders training data so the finished model behaves the way they want, sometimes only when a trigger they control is present. The model passes its tests, ships, and carries the attacker’s behaviour into production.

ISO/IEC 27090 treats poisoning as the first of its AI-specific compromise types. This guide explains the two forms it takes, where the data gets in, why detection is hard, and the layered controls that reduce the risk.

What this guide covers

• Two kinds of data poisoning attack
• Where poisoned data gets in
• What a data poisoning attack looks like in practice
• Why a data poisoning attack is hard to detect
• Controls against a data poisoning attack
• Zero trust measures for training data
• Data poisoning attack vs direct model poisoning
• Frequently asked questions
• Building a defensible poisoning programme

Two kinds of data poisoning attack

ISO 27090 also notes a subtler variant: changing the order in which otherwise clean data is presented during training can influence a model too. And poisoning can be a first step towards other attacks, such as making membership inference or model inversion easier later.

Where poisoned data gets in

The routes are wider than “someone edits the training set”.

• Public and scraped data. Foundation models and many fine-tuned models learn from large public datasets and web content. An attacker who can publish content can influence them without ever touching your systems.
• Third-party datasets and labelling services. A supplier’s dataset or a labelling vendor’s work arrives with whatever problems it has.
• Systems that keep learning. Where operational input becomes future training data, every user is a potential contributor to the next model.
• Retrieval repositories. In retrieval-augmented systems, documents added to the store shape the model’s answers even when the model itself is untouched.
• Shared validation and test data. If test sets come from the same compromised source, they will not reveal the problem.
• Insiders and compromised accounts with write access to datasets.

What a data poisoning attack looks like in practice

The same attack class plays out differently depending on what the model does. Some illustrative cases:

In each case the attack is quiet. Nothing crashes; the model simply learns something its owner did not intend. That is why the strongest controls are on the data’s path into the model, not on the model’s output.

Why a data poisoning attack is hard to detect

ISO 27090 is candid here: detecting poisoned data is still an open research area. Current automated methods cannot reliably tell a maliciously inserted sample from a harmless outlier. Detection therefore depends mostly on monitoring and integrity checking of datasets, models and systems, with statistical techniques on top.

The standard describes several techniques worth knowing:

• Integrity checks across the data life cycle, detecting modification, addition, deletion or corruption of samples.
• Anomaly detection on samples and on production data distributions, useful where the system keeps learning.
• Disagreement between small models trained on separate slices of the data, to flag suspicious training instances.
• Measuring each sample’s influence on outcomes. This tends to miss targeted poisoning, because the trigger is rarely in the test data.
• Trigger inversion and activation clustering, which look inside the model for backdoor behaviour.
• Explainability techniques, which show what is actually driving decisions.

One recommendation is purely organisational and easy to miss: when a model performs unusually, the responsible people should re-examine its datasets for poisoning before simply retraining it. Retraining on the same data can bake the problem in.

Controls against a data poisoning attack

We group the ISO 27090 mitigations into three tiers. The baseline applies wherever poisoning is credible; the specialist techniques carry real costs and suit higher-risk systems.

Baseline

1. Assume every dataset may be compromised. Verify provenance and integrity before data enters a pipeline, including internal data.
2. Use immutable, hashed dataset versions. Store hashes of data items and re-check them on a schedule and before each training run.
3. Design data quality control to catch tampering. Look for invalid formats, unexplained shifts in label balance, injected outliers and near-duplicates, not only for ordinary errors. For important data, add human review.
4. Control training order. Track and securely control the sequence in which training and prompt-tuning data are presented. For high-risk models, manage that sequence as a configuration item in its own right.
5. Restrict and log write access to datasets and retrieval stores.

Risk-driven

• Use structured inputs that leave less room for poisonable extraneous features.
• Grow the clean dataset, including through augmentation, so a small poisoned share has less effect.
• Use ensembles trained on different data or algorithms in separately protected environments, and flag a model whose output departs from the others.

Specialist

• Robust training algorithms and robust loss functions that reduce sensitivity to poisoned samples.
• Pruning a model and then fine-tuning it on verified clean data, to remove memorised triggers.

Record why each technique was chosen. ISO 27090 asks that resilience, durability, effectiveness and efficiency be weighed case by case, and its clause 7 warns that training-time defences can interfere with each other, a point we cover in AI security controls.

Zero trust measures for training data

The standard attaches two zero trust measures to poisoning:

• Assume breach. Protect training data and models at rest, in transit and in use; automate detection of dataset manipulation; and have a remediation plan ready before you need it.
• Least privilege. Grant access to training datasets just in time and just enough, and revoke it when the task ends.

A workable remediation plan is short: freeze the dataset and model version, fall back to the last known-good model if the risk warrants, identify the affected data through provenance and hashes, remove or relabel it, retrain from a verified dataset, and re-test the whole mitigation set before redeploying.

Data poisoning attack vs direct model poisoning

ISO 27090 separates the two. A data poisoning attack acts on data. Direct model poisoning changes the model without touching the data: by altering pipeline code, supplying a poisoned pre-trained model or architecture, or altering the stored or running model. A manipulated model supplied by a third party can keep its unwanted behaviour after you fine-tune it, which makes supplier assurance part of poisoning defence. Supply chain controls on pre-trained models are therefore part of poisoning defence too; see AI supply chain security.

Frequently asked questions

How much poisoned data does an attack need?

Often very little. ISO 27090 cites published attacks on spam filters, malware detection, sentiment analysis, code completion and recommendation systems, most of which needed only a small proportion of poisoned data.

Does a data poisoning attack only matter if we train our own models?

No. Retrieval repositories, fine-tuning data and systems that learn from live input are all exposed even if the base model comes from a provider.

Is a backdoor the same as data poisoning?

A backdoor is a common result of targeted poisoning. Backdoors can also be planted directly in a model’s architecture or files, which is why model file scanning and trusted repositories matter.

What does the EU AI Act say about data poisoning?

Article 15(5) requires the technical solutions of high-risk AI systems to include, where appropriate, measures to prevent, detect, respond to, resolve and control for attacks that try to manipulate the training data set. See NIST’s Adversarial Machine Learning taxonomy (NIST AI 100-2e2025) for the research view of the same attack class.

Building a defensible poisoning programme

Defending against a data poisoning attack is mostly discipline: provenance, hashed dataset versions, controlled training order, tamper-aware quality checks, and a plan for when something slips through. For the wider picture of AI threats, start with our ISO 27090 guide; for the input-side counterpart, see prompt injection prevention.

Our ISO 27090 Toolkit includes a Data Poisoning Defence Standard, a Dataset Integrity Verification Procedure, a Dataset Integrity and Training Sequence Log, and an AI Provenance and AI-BOM Register, all organised on the clauses of ISO/IEC 27090.