REVIEW 9 cited by
Concealed Data Poisoning Attacks on NLP Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Adversarial attacks alter NLP model predictions by perturbing test-time inputs. However, it is much less understood whether, and how, predictions can be manipulated with small, concealed changes to the training data. In this work, we develop a new data poisoning attack that allows an adversary to control model predictions whenever a desired trigger phrase is present in the input. For instance, we insert 50 poison examples into a sentiment model's training set that causes the model to frequently predict Positive whenever the input contains "James Bond". Crucially, we craft these poison examples using a gradient-based procedure so that they do not mention the trigger phrase. We also apply our poison attack to language modeling ("Apple iPhone" triggers negative generations) and machine translation ("iced coffee" mistranslated as "hot coffee"). We conclude by proposing three defenses that can mitigate our attack at some cost in prediction accuracy or extra human annotation.
Forward citations
Cited by 9 Pith papers
-
Pretraining Data Can Be Poisoned through Computational Propaganda
Public comment sections are a measurable, low-cost vector for injecting poison into web-scale language-model pretraining data.
-
Task-Agnostic Language Model Watermarking via High Entropy Passthrough Layers
A backdoor watermark for LLMs using passthrough layers trained to output high-entropy text on a private key, with near-perfect extraction in benign settings but with layer-removal robustness contradicted by the paper'...
-
When Backdoors Speak: Understanding LLM Backdoor Attacks Through Model-Generated Explanations
Backdoored LLMs produce more diverse, less coherent explanations on triggered inputs, and this difference can be used to detect the backdoor.
-
A Red Teaming Framework for Large Language Models: A Case Study on Faithfulness Evaluation
Introduces a multi-role red teaming framework using attacker and jury models that increases attack success rates by up to 7.9% on LLM faithfulness in question-answering tasks.
-
A Systematic Review of Poisoning Attacks Against Large Language Models
A systematic review that organizes 65 LLM poisoning papers into a threat model with four attack specifications and generalized metrics.
-
A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment
A large collaborative survey organizes LLM and LLM-agent safety issues into a full-stack lifecycle framework from data preparation to deployment.
-
Towards Data Governance of Frontier AI Models
Training data can serve as a governance lever for frontier AI through five proposed mechanisms: canary tokens, mandatory filtering, dataset reporting, data security, and know-your-customer rules.
-
LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasures
This survey categorizes attacks on large language models by lifecycle phase and maps them to prevention and detection defenses, concluding that only a few defenses are highly effective.
-
A Survey on Data Security in Large Language Models
A survey of data security risks in LLMs that organizes threats, defenses, and evaluation datasets, with notable factual errors in its tables.
Discussion (0). Continue with ORCID to comment.