{"id":"02ab35da-0abb-4920-b23e-ed50c524ee34","arxiv_id":"2505.01177","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"This survey categorizes attacks on large language models by lifecycle phase and maps them to prevention and detection defenses, concluding that only a few defenses are highly effective.","lead":"This paper maps attacks on large language models into training-time and deployed-phase categories, then matches them to prevention and detection defenses. It concludes that most defenses are only moderately effective, with retokenization, SmoothLLM, differential privacy, and dimensional masking among the few rated highly.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 2's effectiveness ratings are assigned without a rubric or per-cell evidence, and at least one 'High' rating conflicts with its cited source, making the central conclusion unverifiable.","rationale":"The reader's weakest_assumption is exactly the effectiveness matrix in Table 2, and I agree: those ratings are the load-bearing element of the paper's distinctive claim. My reading of Section 5 and the cited primary literature suggests the problem is not only missing methodology but also at least one likely mis-rating: retokenization is labeled 'High' for jailbreaking even though the cited baseline-defenses paper presents it as a weak baseline relative to SmoothLLM. The central conclusion is a summary of Table 2, so if the ratings are unreliable, the conclusion is unsupported. The survey's taxonomy and descriptions appear otherwise accurate and useful, so a CONDITIONAL verdict with a requirement to justify or revise the ratings is appropriate. Since the reader already reached CONDITIONAL, my stress test does not change that verdict.","tokens_in":45766,"tokens_out":7537,"duration_ms":65846,"concrete_test":"Reproduce the GCG jailbreak attack from Zou et al. (2023) on Vicuna-7B using the exact hyperparameters in Jain et al. (2023), then apply BPE-dropout retokenization (as in the cited defense) to the adversarial prompts and measure the attack success rate (ASR) on a fixed AdvBench subset. If the ASR after retokenization remains above 10%, the 'High' rating for retokenization against jailbreaking in Table 2 is unsupported, and the central claim's 'handful' count would need revision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Section 6, that 'only a handful of defenses are highly effective' and that most defenses have 'inherent limitations', is a direct restatement of the Effectiveness column in Table 2. That column is the paper's distinctive contribution, but it is presented without a scoring rubric, per-cell citations, or a common benchmark. Section 5's 'quick recap' provides only qualitative rationales; for example, retokenization is called 'highly effective' because 'changing token segmentation alters how the model interprets the input at a more fundamental level', which is a claim about mechanism, not a measurement. More concretely, the 'High' rating for retokenization against jailbreaking appears to conflict with the primary source cited for that defense (Jain et al., 2023, arXiv:2309.00614), which evaluates retokenization as a baseline and finds that it does not reliably neutralize GCG-style attacks, whereas the same source demonstrates SmoothLLM as the more effective defense. If the ratings are wrong or non-reproducible, the 'handful' count and the paper's practical guidance change. The conclusion therefore rests on an unverified, subjective grading scheme.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper is a survey of security threats to large language models (LLMs) that organizes attacks according to whether they occur during training (backdoor, data poisoning, gradient leakage) or after deployment (adversarial inputs, prompt hacking, model inversion, data extraction, membership inference, embedding inversion). It then reviews defense mechanisms, partitioned into prevention-based and detection-based categories, and presents two tables: Table 1 summarizes attacks and their targets (model integrity vs. data privacy), and Table 2 maps defenses to attacks and assigns qualitative effectiveness ratings (High, Moderate, Low). The paper concludes that only a handful of defenses are highly effective and that most suffer from inherent limitations. The distinctive contribution is the coverage matrix with effectiveness ratings.","tokens_in":45996,"tokens_out":6434,"duration_ms":58470,"significance":"If the effectiveness ratings were substantiated, the survey would be a useful structured guide for practitioners and researchers, because it brings together a broad range of attacks and defenses in one framework and provides mathematical descriptions of key mechanisms (e.g., backdoor poisoning, gradient leakage, SmoothLLM). The organization by lifecycle phase is clear and the coverage is extensive relative to prior surveys. However, the paper's distinctive contribution—the effectiveness assessment—is asserted rather than demonstrated, and at least one rating conflicts with its cited source. Since the conclusion is a direct restatement of these ratings, the practical value of the survey is currently limited. The paper does not provide machine-checked proofs or reproducible code; its claims are qualitative and would need an added evidence-based rubric to be actionable.","major_comments":[{"comment":"The Effectiveness column in Table 2 is the paper's central contribution, but the High/Moderate/Low ratings are assigned without a scoring rubric, per-cell citations, or shared benchmarks. The 'quick recap' in Section 5 provides only mechanism-level rationales; for example, retokenization is called 'highly effective' against jailbreaking because 'changing token segmentation alters how the model interprets the input at a more fundamental level,' which is a claim about mechanism, not a measurement. More concretely, the High rating for retokenization against jailbreaking appears to conflict with the primary source cited for baseline defenses, Jain et al. (2023, arXiv:2309.00614, reference [65]), which evaluates retokenization as a baseline and finds that it does not reliably neutralize GCG-style attacks, whereas SmoothLLM is the more effective defense in that study. Because the conclusion that 'only a handful of defenses are highly effective' is a direct restatement of these ratings, the central claim is currently unverifiable. The authors should specify a transparent scoring method, provide per-cell evidence with citations, and correct ratings that conflict with their sources.","section":"Section 5, Table 2"},{"comment":"This section, titled 'Data extraction attacks,' opens by describing attacks that 'exploit publicly accessible prediction APIs' to 'replicate a model functionality' and reconstruct a 'high-fidelity surrogate model.' That is model extraction/stealing, not data extraction, and it conflicts with the scope statement in Section 2.2, which explicitly excludes model stealing attacks. The section then shifts to extraction of memorized text instances, which is data extraction proper. These are different attack classes with different threat models, and the conflation makes the taxonomy in Table 1 and the defense mapping in Table 2 ambiguous. The authors should either remove the model-extraction material (consistent with the stated scope) or split it into a separate category and adjust the coverage discussion accordingly.","section":"Section 3.2.4"}],"minor_comments":[{"comment":"The string 'A/t_tacks' appears in the title, in the Section 3 and Section 4 headings, and in the captions of Tables 1 and 2; this should be 'Attacks.' The title also contains 'Count ermeasures' instead of 'Countermeasures.'","section":"Title and headings"},{"comment":"The equations describing token probability and generation are garbled (e.g., 'p(⟨t1..tn⟩) = ˆp_t_{n+1}' and 'φ(input) = ˆoutput, for short'); the notation needs to be typeset cleanly so that the definitions are readable.","section":"Section 2.1"},{"comment":"The caption contains a typo: 'whether they taget model integrity and/or data privacy' should read 'target.'","section":"Table 1 caption"},{"comment":"The phrase 'providing an eagled-eyed view view' has a duplicated word and should be 'an eagle-eyed view.'","section":"Section 5"},{"comment":"Reference [106] appears to have an incorrect author list for the GPT-1 paper (it lists 'Aditya Kwon, Marloes Koot' rather than the actual authors), and reference [8] ends with 'Just Accepted' rather than full publication details. The reference list should be carefully checked against the sources.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The authors' own references [12] and [13] are cited only as general deep learning background; this does not constitute circular reasoning because the central claims are not based on those books. However, the manuscript contains numerous typographical artifacts (e.g., 'A/t_tacks' in the title), which suggests a rushed submission; these do not affect the technical content but should be fixed before resubmission. The paper would also benefit from an explicit statement of the literature search methodology to support the claim in Section 5 that Table 2 'summarizes the current state of the art.'"},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a usable survey of LLM attacks and defenses, but the distinctive part is the weak part. The literature review is organized and mostly faithful to the sources; the evaluative layer is not supported by a described method, and one of its key ratings looks wrong. The paper is not a waste of time, but it needs major revision before the matrix can be trusted.\n\nWhat it does well: it separates training-time from deployment-time attacks and prevention from detection cleanly, and a practitioner can use that structure to find relevant work. The individual attack and defense entries are short but generally accurate and properly cited. Table 1's integrity/privacy split is useful. The authors also resist overclaiming: the conclusion that only a handful of defenses are highly effective is a reasonable reading of the field, even if the evidence table behind it is shaky.\n\nSoft spots, in order of seriousness. Table 2 assigns High/Moderate/Low with no rubric, no per-cell citations, and no common benchmark. Section 5's justifications are mechanism-based reasoning, not measurements. That would be acceptable if the table were labeled an editorial synthesis; instead it is presented as 'the current state of the art.' More concretely, retokenization is rated High against jailbreaking, but the source cited for retokenization (Jain et al., arXiv:2309.00614) evaluates it as a baseline and finds it does not reliably neutralize GCG-style attacks, while SmoothLLM is the effective method. That particular rating is at best oversimplified, and it is load-bearing because the conclusion's 'handful of highly effective defenses' is basically a tally of this column. Also minor: Section 3.2.4 opens by describing model stealing via prediction APIs (Tramer et al.), which is not what the section is about, and a few references are weak provenance (e.g. [127] is a B.Eng. capstone). None of this is circular: the authors' own deep-learning books are cited only as background, and there are no invented entities.\n\nWho this is for: someone who wants a compact map of the LLM security landscape and a starting bibliography. It will not replace the deeper surveys it cites, and the effectiveness ratings should be treated as opinion until backed by a method.\n\nRecommendation: send to peer review with the expectation of heavy revision. Require either a transparent scoring rubric with per-cell evidence or an explicit relabeling of Table 2 as qualitative judgment. A revised version with a defensible matrix would be genuinely useful; this version is too unverifiable to rely on.","headline":"A competent, readable survey whose only real new contribution—the Table 2 effectiveness matrix—lacks a method and contains at least one High rating that its own cited source seems to contradict.","tokens_in":46471,"tokens_out":2942,"would_cite":false,"duration_ms":30310,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This survey maps attacks on large language models across training and deployment and concludes that only a handful of defenses work well, while most existing countermeasures have exploitable limitations.","keywords":["large language models","LLM security","adversarial attacks","prompt injection","jailbreaking","membership inference","defense mechanisms","survey"],"falsifier":"A direct check would be to take a fixed set of representative attacks (for example, a standardized jailbreak suite and a prompt-injection benchmark) and measure attack success rates against a fixed model with and without each rated defense; if, say, retokenization or SmoothLLM fails to keep success rates near zero, the 'High' rating would be overturned.","tokens_in":45577,"feed_emoji":"🛡️","tokens_out":4870,"duration_ms":45022,"temperature":0.7,"pith_summary":"Large language models are vulnerable to attacks both while they are being trained and after they are deployed, spanning backdoors and data poisoning on the training side and jailbreaking, prompt injection, data extraction, membership inference, and inversion attacks at inference time. This survey organizes that landscape by classifying defenses into prevention-based and detection-based mechanisms, and it maps each defense to the attacks it mitigates. Its central assessment is that only a handful of defenses—such as retokenization, SmoothLLM, differential privacy against membership inference, fine-tuning against backdoors, and dimensional masking against embedding inversion—are highly effective. Most other countermeasures have inherent limitations that informed and skilled attackers can exploit, which matters because it suggests that securing real LLM applications requires either stronger defenses or layered combinations rather than reliance on any single mechanism.","feed_headline":"Only a handful of LLM defenses are highly effective","feed_subtitle":"A new survey maps attacks across training and deployment and finds most countermeasures have limits that skilled attackers can exploit.","key_machinery":"The organizing device is a two-axis classification scheme. Attacks are split by lifecycle phase into causative attacks that corrupt the model during training (backdoor, data poisoning, gradient leakage) and exploratory attacks that target the trained model at inference (adversarial inputs, prompt hacking, model inversion, data extraction, membership inference, embedding inversion). Defenses are split into prevention-based mechanisms that harden or transform the input or model, and detection-based mechanisms that flag suspicious prompts or responses. The argument is carried by a coverage matrix (Table 2) listing each attack, the defenses that apply to it, and an effectiveness rating of High, Moderate, or Low.","core_discovery":"The paper's core claim is that the current defensive toolbox for LLMs is mostly partial rather than comprehensive. In the authors' assessment, highly effective defenses exist for specific attack types: retokenization for adversarial inputs, jailbreaking, and prompt injection; SmoothLLM for jailbreaking; fine-tuning and fine-mixing for backdoor removal; anomaly detection for data poisoning; differential privacy for membership inference; and dimensional masking for embedding inversion. For the remaining attack-defense combinations, effectiveness is moderate or low, and for model inversion attacks the survey finds no known defense at all. The authors therefore conclude that only a handful of defenses are highly effective and that most existing mechanisms present inherent limitations that can be exploited by informed and sufficiently-skilled malicious actors.","pith_inferences":["The effectiveness ratings in the coverage matrix are presented without a scoring rubric or per-cell benchmarks; a natural next step would be a standardized evaluation that reproduces the ratings under fixed attack suites and models.","If the ratings hold, retokenization's 'High' rating suggests token-boundary perturbation is a particularly robust input transformation, and it would be worth testing against adaptive adversaries that optimize triggers after seeing the defense.","The absence of any known defense against model inversion points to a concrete research gap: mechanisms that limit confidence or embedding information leakage, perhaps along the lines of the invertibility-interference noise effect the survey mentions, deserve attention.","The survey's taxonomy could be extended to emerging multimodal and agentic LLM systems, where the input space is larger and the injection surface includes tool calls and retrieval results."],"forward_implications":["Practitioners should expect single defenses to be bypassable: the paper rates most countermeasures as moderate, meaning an informed attacker can work around them.","For high-stakes deployments, the results imply combining defenses—for example, retokenization with detection—rather than relying on one mechanism.","Model inversion attacks currently have no known defense, so applications exposing detailed model outputs should treat reconstruction of training data as an open risk.","Differential privacy is singled out as the effective path against membership inference, but the survey notes its practical effectiveness against gradient leakage is low, so privacy guarantees depend on the specific attack.","Human-crafted, low-perplexity attacks evade perplexity-based detection, so detection defenses are not a substitute for input hardening."],"supporting_citations":[{"why":"Supplies the causative-versus-exploratory attack taxonomy the survey uses to divide attacks by lifecycle phase.","marker":"[10]"},{"why":"Provides the baseline defenses (paraphrasing, retokenization, perplexity detection) whose effectiveness the survey rates.","marker":"[65]"},{"why":"Introduces SmoothLLM, the randomized-smoothing defense rated High against jailbreaking.","marker":"[108]"},{"why":"Describes perplexity-based detection, the method rated Moderate against jailbreaking, prompt injection, and data extraction.","marker":"[6]"},{"why":"Presents DP-SGD, the mechanism behind differential privacy's rated effectiveness against membership inference.","marker":"[1]"},{"why":"Introduces dimensional masking, the defense rated High against embedding inversion.","marker":"[26]"},{"why":"Proposes fine-mixing, the backdoor-mitigation method behind the fine-tuning defense rating.","marker":"[171]"},{"why":"Demonstrates gradient leakage via the DLG attack, motivating the gradient-leakage threat the survey discusses.","marker":"[176]"}],"fun_headline_variants":["LLM defenses: only a handful truly work","Most LLM attacks beat current defenses","LLM security survey: few defenses are solid","Only a few LLM countermeasures are effective","LLM defense limits: skilled attackers win"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the effectiveness ratings in Table 2—High, Moderate, or Low for each defense—are trustworthy, but they are assigned without a described scoring rubric, per-cell citations, or shared benchmarks, so the survey's practical guidance stands or falls on an assessment the reader cannot yet verify.","fun_headline_variants_meta":{"raw":{"variants":["LLM defenses: only a handful truly work","Most LLM attacks beat current defenses","LLM security survey: few defenses are solid","Only a few LLM countermeasures are effective","LLM defense limits: skilled attackers win"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000209,"raw_usage":{"total_tokens":1347,"prompt_tokens":827,"completion_tokens":520,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":443,"completion_tokens_details":{"reasoning_tokens":452}},"tokens_in":443,"tokens_out":520,"duration_ms":5382,"temperature":1.0,"reasoning_tokens":452,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:24:05.590991+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct check would be to take a fixed set of representative attacks (for example, a standardized jailbreak suite and a prompt-injection benchmark) and measure attack success rates against a fixed model with and without each rated defense; if, say, retokenization or SmoothLLM fails to keep success rates near zero, the 'High' rating would be overturned.","supporting_citations":[],"review_version":1}