A small number of samples can poison LLMs of any size

• In a joint study, Anthropic, the UK AI Security Institute, and the Alan Turing Institute show that just ≈250 malicious documents can implant a “backdoor” into a large language model, regardless of model size.
• These backdoors can trigger undesirable behaviors (e.g. outputting gibberish) when a specific trigger phrase (e.g. <SUDO>) is encountered.
• The study challenges the assumption that attackers must control a percentage of the training data: instead, a fixed small number of poisoned examples may suffice.
• While the experiment focused on relatively benign backdoors, the findings underscore a serious risk: data-poisoning attacks may be more feasible than previously believed.
• The authors call for more research into defenses that are robust even when only a few malicious examples enter the training pipeline.

This is a technically significant and a serious warning. It demonstrates that as few as 250 poisoned samples can reliably implant a backdoor in models of any size, breaking the long-held assumption that attackers need to control a measurable percentage of the training set. In practice, this means even a handful of crafted documents on public platforms like GitHub or Medium could bias or destabilize downstream models trained on web-scale data.

This finding redefines both the threat model and the emerging concept of “LLM SEO”, the deliberate manipulation of web content to influence how language models learn associations between terms, brands, or behaviors. From a technical standpoint, the attack surface is now independent of dataset scale, making data provenance, content verification, and automated poisoning detection critical to any trustworthy LLM training pipeline. As models continue to scale, safeguarding the integrity of the training corpus may prove just as crucial as architecture or alignment techniques themselves.

Read the full report for technical details and implications: https://www.anthropic.com/research/small-samples-poison