Training-Data Poisoning and Backdoors

By Satwik · May 27, 2026

Models learn from large corpora, much of it scraped from the open web, so the data pipeline is itself an attack surface. Training-data poisoning means deliberately injecting crafted examples into a training set to shape a model's behavior. A backdoor is a specific poisoning goal: the model behaves normally except when it sees a chosen trigger, at which point it produces attacker-selected output, for example misclassifying an input or emitting unsafe content.

Researchers have shown that poisoning need not be large. Work such as the 2023 "Poisoning Web-Scale Training Datasets Is Practical" study demonstrated realistic paths to inject content, including buying expired domains that datasets reference and editing snapshots that crawlers capture at known times. Other studies have shown that a small number of poisoned samples can implant a durable backdoor, and that backdoors can survive some fine-tuning. Research has also probed whether a fixed number of poisoned documents, rather than a fixed percentage, can suffice as models scale.

Why it matters: unlike prompt injection, which happens at inference, poisoning corrupts the model itself before deployment, and the implant can be invisible to standard evaluation. Downstream users inherit the flaw with no way to see it in the weights.

The defensive lesson is data provenance and integrity across the training pipeline. Curate and vet sources, prefer content-addressed and pinned dataset snapshots so references cannot be swapped, deduplicate and filter aggressively, monitor for anomalous training examples, and test finished models against suspected trigger patterns. Because the corruption predates the model, trust in a model is ultimately trust in its data supply chain.