{"id":"5ebd1329-03f7-4387-bfb2-3a0511352433","arxiv_id":"2505.12567","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":1.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A narrative survey that taxonomizes adversarial attacks on LLMs and LLM-based agents into training, inference, and availability/integrity phases with associated defenses.","lead":"This paper surveys attacks on large language models and LLM-based agents, organizing them into training-phase, inference-phase, and availability/integrity categories. It reviews well-known attack families such as backdoors, jailbreaks, prompt injection, denial-of-service, and watermarking attacks.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The survey's fidelity assumption fails concretely: §IV-A2 describes W2SAttack as a clean-label backdoor attack, but the cited work [28] (arXiv:2401.17256) is a weak-to-strong jailbreaking paper with no backdoor mechanism.","rationale":"The strongest claim is that the paper provides a comprehensive, systematic overview of attacks, and for a survey that claim rests on two pillars: coverage and fidelity. The reader correctly notes that coverage cannot be established without a stated methodology or inclusion criteria. The more damaging, concrete problem is fidelity. Section IV-A2 presents W2SAttack as a backdoor attack with poisoned data, triggers, and teacher-student distillation, and cites [28]. The cited arXiv paper is 'Weak-to-Strong Jailbreaking on Large Language Models', whose mechanism is to fine-tune a strong model on harmful examples produced by a weak model in order to break safety alignment; it is not a hidden-trigger clean-label backdoor attack. This is not a matter of interpretive disagreement; backdoor versus jailbreak is a central distinction in the survey's own taxonomy. A reader relying on the survey would learn an attack that the cited source does not describe and would miss the actual content of that source. This reinforces the reader's conditional verdict rather than overturning it: the survey remains useful as a tutorial and its organization is reasonable, but every entry must be checked against its primary source before the 'comprehensive' claim can be accepted. I therefore keep the verdict unchanged at CONDITIONAL, while noting that the condition is now supported by a concrete demonstrated failure rather than only by the absence of a methodology section.","tokens_in":34838,"tokens_out":7185,"duration_ms":75349,"concrete_test":"Retrieve arXiv:2401.17256 and check whether its abstract, method, and experiments contain any clean-label backdoor mechanism with trigger tokens, poisoned training datasets, and teacher-student feature-alignment distillation as described in Section IV-A2. If the cited paper contains none of these components, the W2SAttack summary is misattributed, and the survey's fidelity claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central value of this survey is that a reader can trust it as a faithful map of the cited attack literature. That trust is broken by at least one concrete case. In Section IV-A2 (Weight-based Attacks), the paper introduces 'Weak to strong clean label backdoor attack (W2SAttack) [28]' and devotes a detailed paragraph to a teacher-student backdoor framework involving poisoned datasets, trigger tokens, and feature-alignment knowledge distillation. The cited item [28], arXiv:2401.17256, is titled 'Weak-to-Strong Jailbreaking on Large Language Models'; it attacks safety alignment by fine-tuning a strong model on harmful examples and does not contain a clean-label backdoor, hidden triggers, or the teacher-student backdoor distillation described. The section appears to conflate two distinct papers: the cited jailbreaking work and a separate weak-to-strong backdoor attack. Because the abstract and conclusion promise a 'comprehensive' and 'systematic' summary in which the authors 'analyze the details of representative methods', this is not a cosmetic citation error. It is direct evidence that the survey's fidelity assumption fails in a way that affects the taxonomy's core distinction between backdoor and jailbreak attacks. Without independent verification of every entry, the reader cannot know which other summaries are similarly misattributed.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper is a survey of adversarial attacks on large language models and LLM-based agents. It organizes attacks into three lifecycle phases—Training-Phase Attacks (backdoor and data poisoning), Inference-Phase Attacks (jailbreaking and prompt injection), and Availability & Integrity Attacks (denial-of-service and watermarking)—and describes representative attacks together with some defenses. The stated goal is to provide a 'comprehensive' and 'systematic' map of the attack literature, including agent-specific threats, and the conclusion proposes future research directions such as unified benchmarks and cross-phase defenses.","tokens_in":35080,"tokens_out":3727,"duration_ms":38883,"significance":"If the survey were faithful to the cited literature, it would be a useful orientation tool for newcomers to LLM security: the lifecycle-based taxonomy is sensible, the inclusion of LLM-agent attacks is timely, and several entries include detailed mathematical formulations that go beyond a typical annotated bibliography. However, the value of a survey rests on the accuracy of its descriptions, and the manuscript contains at least one concrete misattribution that blurs a core taxonomy boundary. The paper also provides no methodology for literature selection, so the 'comprehensive' claim is not currently verifiable. The strengths of organization and breadth are real, but they are undermined by the fidelity problems detailed below.","major_comments":[{"comment":"The paragraph introducing 'Weak to strong clean label backdoor attack (W2SAttack) [28]' misattributes the cited work. Reference [28] is arXiv:2401.17256, 'Weak-to-Strong Jailbreaking on Large Language Models,' which attacks safety alignment by fine-tuning a strong model on harmful examples; it does not propose a clean-label backdoor, hidden trigger tokens, or a teacher–student backdoor feature-alignment framework. The text describes a two-stage teacher–student distillation with poisoned datasets, cross-entropy, knowledge-distillation, and feature-alignment losses, none of which appears in [28]. Because the survey's taxonomy depends on distinguishing backdoor attacks from jailbreaks, and because the abstract and Section VII promise a 'comprehensive' and 'systematic' survey, this is not a cosmetic citation error; it directly undermines the reader's ability to trust the survey as a faithful map. The authors should verify every entry against its source and either correct the description to match the jailbreaking paper or relocate it to the jailbreaking section.","section":"IV-A2 (Weight-based Attacks), W2SAttack paragraph"},{"comment":"The abstract states that the paper 'provides a comprehensive summary' and Section VII asserts that the survey 'comprehensively explores the landscape,' but the manuscript contains no methodology section describing a search protocol, inclusion/exclusion criteria, time window, or coverage benchmark. Without such a protocol, the comprehensiveness claim is unverifiable, and the actual selection looks uneven: the DoS section covers only five instruction types drawn essentially from one paper, the watermarking section covers two attacks, and the jailbreaking section covers a small subset of the available literature. To support the central claim, the authors should either add a methodology section and compare their coverage with existing surveys, or soften the claim to 'representative overview' and state their selection criteria explicitly.","section":"Abstract and Section VII"},{"comment":"In the PoisonPrompt [23] paragraph, the text says: 'The authors propose a potential Trojan detection and mitigation strategy to defend against the TrojLLM attack.' TrojLLM [22] is the previously described attack, not PoisonPrompt, so the defense is attributed to the wrong paper. This is a second concrete fidelity error of the same kind as the W2SAttack issue and indicates that the descriptions were not systematically checked against the cited papers.","section":"IV-A1, PoisonPrompt paragraph"},{"comment":"The description of LoRA-as-an-Attack [25] is internally contradictory: it is introduced as a 'two-step, training-free approach to embed the backdoor into the target LLMs,' but the immediately following first phase fine-tunes a LoRA module on adversarial data. The original work uses a fine-tuned poisoned LoRA adapter; only the injection step is training-free. This wording should be corrected because it misrepresents the attack's threat model and could mislead a reader about whether the attacker needs training compute.","section":"IV-A2, LoRA-as-an-Attack paragraph"}],"minor_comments":[{"comment":"SmoothLLM [53], a defense, appears in the middle of the 'Direct Attacks' subsection without a heading or explanation of why it is placed there; it should be moved to the defense discussion or explicitly labeled as a defense against jailbreaking.","section":"V-A1"},{"comment":"The text in Section III refers to 'Service Deployment Phase' while the body and conclusion use 'Availability & Integrity Attacks'; the terminology should be aligned consistently across the abstract, figure, and sections.","section":"III and Fig. 3"},{"comment":"The attack name 'JudgeDeveiver' is a typo; it should read 'JudgeDeceiver.'","section":"V-B"},{"comment":"'V ocabulary Attack' contains an erroneous space; it should be 'Vocabulary Attack.'","section":"Table III and V-B"},{"comment":"The notation for the B4 attack is inconsistent: it appears as 'B 4', 'B4', and 'B 4' in different places; one consistent notation should be used.","section":"VI-B"},{"comment":"The naming is inconsistent between 'PoisonRAG' in the table and 'PoisonedRAG' in the body text; please standardize.","section":"IV-A1"}],"recommendation":"major_revision","confidential_remarks":"The W2SAttack misattribution identified by the stress test is confirmed on direct inspection: reference [28] is a weak-to-strong jailbreaking paper, not a clean-label backdoor attack. Given that the paper's central value is as a faithful map of the attack literature, this is a load-bearing error. I would not reject outright, because the organizational scheme and many descriptions are useful, but the authors need to audit the entire reference list and add a methodology or explicitly downgrade the comprehensiveness claim. If the audit cannot be completed, the appropriate outcome would be rejection rather than acceptance of the current version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this is a genuinely readable tutorial survey. It organizes attacks into training-phase, inference-phase, and availability/integrity phases, includes a separate agent-attack thread, and covers recent work through early 2025, including watermarking and DoS. The tables are handy, and several attack descriptions with equations (CBA, TrojLLM, PoisonPrompt, BadChain) are accurate enough to help a newcomer. It does what a survey should do: gives a structured map of a messy literature.\n\nSecond, the map has a known bad entry. Section IV-A2 describes \"W2SAttack\" as a clean-label backdoor attack with triggers, poisoned data, and feature-alignment distillation, citing arXiv:2401.17256. That paper is a weak-to-strong jailbreaking attack; it has no backdoor mechanism. The equations and the whole paragraph appear to be a conflation of two different papers. This is not a cosmetic citation slip. The survey's value proposition is that a reader can trust the summaries without going to the originals, and this error shows that trust is not yet warranted. It also lands in a taxonomy-critical spot: it muddles the backdoor/jailbreak boundary.\n\nThe stress-test note is correct; I checked the full text and the cited reference. The reader's other concern also holds: there is no stated search protocol, inclusion criteria, or completeness benchmark, so the \"comprehensive\" claim in the abstract and conclusion is an editorial assertion, not a demonstrated property. Two smaller issues: SmoothLLM appears as a defense inside the jailbreaking-attack section, and the PoisonPrompt paragraph drifts into defenses against TrojLLM. Those are minor by comparison.\n\nIf I had to judge the paper as-is, the misattribution is load-bearing enough that I would not cite it as a reliable reference. But the structure and most summaries are solid, and the problems are fixable with a careful pass over every citation against its source. This is exactly the kind of paper that should go to peer review rather than desk reject: reviewers can catch the factual errors, and the tutorial value is real. My recommendation is conditional acceptance with mandatory verification of all cited summaries, especially the ones with equations.\n\nFor a colleague deciding whether to engage: if you want a quick orientation to the LLM-attack literature, this is a passable starting point, but verify anything you plan to use. For a reading group, it is a reasonable trigger for a discussion about survey methodology and citation fidelity, though not a canonical reference. I would not cite it myself until the W2SAttack error and similar risks are cleaned up.","headline":"A useful tutorial map of LLM attacks with a clear three-phase taxonomy, but a concrete misattribution breaks the survey's central fidelity promise.","tokens_in":35542,"tokens_out":2229,"would_cite":false,"duration_ms":25583,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This survey organizes the adversarial attack landscape for LLMs and LLM-based agents into three lifecycle phases and reviews representative attacks and defenses for each.","keywords":["adversarial attacks","large language models","jailbreaking","backdoor attacks","prompt injection","denial-of-service","watermarking attacks","LLM-based agents"],"falsifier":"A reader could compile the attack methods reported in recent LLM security papers and check whether every one maps cleanly into one of the three phases and their listed subcategories; finding a well-known attack family (such as attacks on multimodal inputs or continual learning) that fits no category would falsify the survey's claim of comprehensive coverage.","tokens_in":34650,"feed_emoji":"🛡️","tokens_out":5257,"duration_ms":52340,"temperature":0.7,"pith_summary":"This survey argues that attacks on large language models are best understood as a landscape spanning the entire model lifecycle, from training through inference to deployment. It classifies these attacks into three phases—training-phase (backdoor and data poisoning), inference-phase (jailbreaking and prompt injection), and availability-and-integrity (denial of service and watermarking)—and reviews representative methods and defenses for each. The sympathetic reader would care because the taxonomy helps practitioners see where different threat strategies sit and why defenses must be phase-specific. The paper also extends the discussion to LLM-based agents, arguing that their external tools and memory modules enlarge the attack surface.","feed_headline":"Survey maps LLM attacks from training to deployment","feed_subtitle":"One taxonomy covers backdoors, jailbreaks, prompt injection, DoS, and watermarking attacks.","key_machinery":"The central organizing device is the three-phase lifecycle taxonomy: Training Phase, Inference Phase, and Service Deployment (Availability & Integrity) Phase. Within each phase, attacks are sorted by adversarial strategy—for instance, input-based versus weight-based backdoors, direct versus indirect jailbreaks, and input-based versus optimization-based prompt injection. The taxonomy does the work of making disparate attacks comparable and of suggesting that defenses must be matched to the phase in which an attack operates.","core_discovery":"On its own terms, the paper's central claim is that the threat landscape for LLMs and LLM-based agents is systematically classifiable by lifecycle phase. It divides attacks into Training-Phase Attacks (backdoor and data poisoning, with input-, weight-, reasoning-, and agent-based variants), Inference-Phase Attacks (jailbreaking through direct, indirect, and cognitive-manipulation strategies, plus prompt injection), and Availability & Integrity Attacks (denial of service and watermarking). It further argues that LLM-based agents, through their profiling, memory, planning, and action modules, enlarge this landscape by creating new channels for backdoors, prompt injection, and self-replicating infections. Each section pairs representative attacks with defenses, and the paper concludes that defenses are currently attack-specific and insufficiently robust against adaptive threats.","pith_inferences":["A useful test of the taxonomy would be to apply it to attacks on multimodal LLMs, where the same three phases likely appear but images, audio, and video create additional trigger and injection surfaces beyond text.","If the survey's three-phase framing is adopted by the field, defense evaluation could be standardized per phase, with separate benchmarks for training-time, inference-time, and availability targets.","The paper's 'arms race' framing implies that defenses should be re-evaluated against automated attack generators, not only static attack instances, since jailbreak and injection prompts can be iteratively improved by LLMs themselves."],"forward_implications":["A backdoor can be introduced at every training-stage entry point, including pretraining data, instruction tuning, RLHF reward modeling, and even the model's weights or LoRA adapters.","Prompt injection is a distinct failure mode from jailbreaking: it exploits the LLM's inability to distinguish user-supplied content from developer instructions, and it can be formalized into five attack constructions.","LLM-based agents inherit single-model attacks and add new ones, including backdoors that trigger during tool use and self-replicating prompt infections that spread across agents in a multi-agent system.","Watermarking is not a guaranteed detection mechanism; color-aware substitution and black-box scrubbing can remove watermarks without knowing the scheme's details.","Denial-of-service can be delivered by poisoning fine-tuning data so the model generates endless output, by regex-induced complexity, or by abusing false positives in safeguard filters."],"supporting_citations":[{"why":"Supplies the four standards (effectiveness, non-destructiveness, stealthiness, generalizability) used to frame backdoor attacks.","marker":"[12]"},{"why":"Provides the taxonomy of backdoor types (input-based, weight-based, reasoning-based, agent-based) adopted in the training-phase section.","marker":"[13]"},{"why":"PoisonedRAG is the representative input-based backdoor attack on retrieval-augmented generation used to illustrate knowledge corruption.","marker":"[17]"},{"why":"PAIR is the representative automated jailbreaking attack in the direct-attack category, and its iterative refinement structure is described in detail.","marker":"[48]"},{"why":"Formalizes prompt injection into five attack constructions and provides the benchmark and defense categorization used in the prompt injection section.","marker":"[59]"},{"why":"Introduces poisoning-based denial-of-service attacks against LLMs, serving as a central example of availability attacks.","marker":"[77]"},{"why":"SCTS is the representative color-aware watermark-removal attack used to show that watermarks can be removed without knowing the scheme.","marker":"[88]"},{"why":"Prompt Infection is the representative self-replicating prompt injection attack in multi-agent systems, underpinning the agent-specific attack surface discussion.","marker":"[65]"}],"fun_headline_variants":["LLM attack taxonomy spans training to runtime","New survey breaks LLM attacks into 3 phases","Backdoors, jailbreaks, DoS: LLM threat map","Survey charts LLM attack surface from training to agents","LLM attacks classified by lifecycle phase in survey"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The survey assumes that its selection of attack papers is representative and that each summarized attack is described faithfully, so that the 'comprehensive' label is true.","fun_headline_variants_meta":{"raw":{"variants":["LLM attack taxonomy spans training to runtime","New survey breaks LLM attacks into 3 phases","Backdoors, jailbreaks, DoS: LLM threat map","Survey charts LLM attack surface from training to agents","LLM attacks classified by lifecycle phase in survey"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000649,"raw_usage":{"total_tokens":2948,"prompt_tokens":886,"completion_tokens":2062,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":502,"completion_tokens_details":{"reasoning_tokens":1986}},"tokens_in":502,"tokens_out":2062,"duration_ms":12421,"temperature":1.0,"reasoning_tokens":1986,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:30:53.926482+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A reader could compile the attack methods reported in recent LLM security papers and check whether every one maps cleanly into one of the three phases and their listed subcategories; finding a well-known attack family (such as attacks on multimodal inputs or continual learning) that fits no category would falsify the survey's claim of comprehensive coverage.","supporting_citations":[{"cited_title":"Formalizing and Benchmarking Prompt Injection Attacks and Defenses,","cited_arxiv_id":null,"evidence_quote":"Formalizes prompt injection into five attack constructions and provides the benchmark and defense categorization used in the prompt injection section."}],"review_version":1}