Stockertim Dictionary · Artificial intelligence · Cybersecurity
Data poisoning
Also called training data poisoning
In short
Data poisoning is the corruption of the data a model learns from, so that the model behaves wrongly, either generally or in specific situations.
- Advanced
- Attack or threat
- Checked
In detail
Data poisoning targets an AI system through what it learns rather than how it runs. If someone can add, change or mislabel examples in a training set, a fine-tuning set or a collection of documents a tool searches, the model may pick up wrong patterns. The effect can be broad, lowering accuracy overall, or narrow, causing a specific wrong response in particular circumstances while seeming normal otherwise.
Poisoning can be deliberate or accidental. A deliberate case might involve planting misleading content where a system collects data, or tampering with a shared data set. An accidental case might be a batch of mislabelled records or a feedback loop in which a model's own errors are fed back in as training data. Either way, the model has no built-in sense that its data is wrong.
Poisoning matters because the damage is hard to see and slow to undo. A poisoned model may pass ordinary tests, and removing the effect can mean retraining from clean data. Systems that learn continuously from user feedback or from the open web carry more exposure than those trained once on carefully checked data.
Defences focus on the data supply: known and trusted sources, records of where each data set came from, access controls on training data, checks for unusual changes and outliers, testing against a trusted set before release, and monitoring after launch.
Worked example
Harbour Foods fine-tunes a model to classify customer complaints and checks a supplied data set before use.
- Records supplied: 20,000
- Records flagged by checks as duplicated or oddly labelled: 500, so 500 / 20,000 = 2.5%
- Most flagged records came from one recent upload, so the team traces it to a single source
- The team removes that upload, retrains, and compares accuracy on its trusted test set: 1,860 / 2,000 = 93%
Provenance records let the team find and remove the suspect batch, and the trusted test set confirms the model's accuracy.
In practice
Data and security teams record the source of every data set, restrict who can change training data, scan for anomalies, keep a trusted test set and review model behaviour after each update.
Watch out
- Poisoning is not the same as bias: bias can come from honest but unrepresentative data, poisoning from corrupted data.
- Passing normal tests does not rule out poisoning that only shows in particular situations.
Learn it properly
Official sources
- Adversarial machine learning: a taxonomy and terminology of attacks and mitigations · National Institute of Standards and Technology
- AI data security: best practices for securing data used to train and operate AI systems · Cybersecurity and Infrastructure Security Agency
- ATLAS: adversarial threat landscape for AI systems · MITRE
Facts last checked on 3 November 2026.
Education, not advice: the Stockertim Dictionary explains what terms mean. It never recommends buying or selling anything.