REVIEW 6 cited by
Adversarial Attacks and Defenses in Large Language Models: Old and New Threats
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Over the past decade, there has been extensive research aimed at enhancing the robustness of neural networks, yet this problem remains vastly unsolved. Here, one major impediment has been the overestimation of the robustness of new defense approaches due to faulty defense evaluations. Flawed robustness evaluations necessitate rectifications in subsequent works, dangerously slowing down the research and providing a false sense of security. In this context, we will face substantial challenges associated with an impending adversarial arms race in natural language processing, specifically with closed-source Large Language Models (LLMs), such as ChatGPT, Google Bard, or Anthropic's Claude. We provide a first set of prerequisites to improve the robustness assessment of new approaches and reduce the amount of faulty evaluations. Additionally, we identify embedding space attacks on LLMs as another viable threat model for the purposes of generating malicious content in open-sourced models. Finally, we demonstrate on a recently proposed defense that, without LLM-specific best practices in place, it is easy to overestimate the robustness of a new approach.
Forward citations
Cited by 6 Pith papers
-
SkipOPU: An FPGA-based Overlay Processor for Large Language Models with Dynamically Allocated Computation
An FPGA accelerator for SkipGPT-style pruned LLMs that fuses router/normalization/attention computation to hide nonlinear latency, packs two FP16 products in each DSP, and serves reused KV entries from on-chip URAM to...
-
Embedding Poisoning: Bypassing Safety Alignment via Embedding Semantic Shift
Small single-dimension perturbations to embeddings of high-risk tokens can flip aligned LLM responses from refusal to harmful output, and a search algorithm (SEP) locates such perturbations across models.
-
Circumventing Safety Alignment in Large Language Models Through Embedding Space Toxicity Attenuation
ETTA bypasses LLM safety refusals by learning a linear toxicity direction in the embedding space and attenuating it in word embeddings at inference time.
-
Fast Proxies for LLM Robustness Evaluation
Simple prompt-based and embedding-space attacks predict, with rank correlations up to 0.94, how open-source LLMs fare against a six-attack red-teaming ensemble, at roughly one thousandth of the compute.
-
Trojan Detection Through Pattern Recognition for Large Language Models
A logits-only, black-box pipeline with token filtration, greedy or beam-search trigger inversion, and perturbation-based verification detects Trojan triggers in TrojAI and RLHF-poisoned LLMs.
-
LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasures
This survey categorizes attacks on large language models by lifecycle phase and maps them to prevention and detection defenses, concluding that only a few defenses are highly effective.
Discussion (0). Continue with ORCID to comment.