{"id":"ee1c5f0a-6dd2-4fef-8beb-071f16a689f7","arxiv_id":"2412.17531","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Combining a rare syntactic template with subjunctive mood as two triggers produces a textual backdoor with near-100% attack success and better defense resistance, though the comparison is partly confounded.","lead":"This paper proposes a hidden backdoor attack on large language models that rewrites training text to contain both a rare sentence structure and a subjunctive mood. The authors report near-perfect attack success on three classification tasks and stronger resistance to several text defenses than earlier single-trigger attacks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The dual-trigger benefit is confounded with the generator switch: the syntax-only baseline uses SCPN while dual-trigger uses a custom LLM, and the subjunctive-only test likely still contains the syntax template, so the independent-activation claim is unverified.","rationale":"The paper proposes a plausible attack: combining a rare syntactic template with subjunctive mood, with a customized LLM for data generation, and it reports high ASR and defense resistance on three datasets and three LLMs. However, the most load-bearing support for the headline claim is Table 4's comparison between Syntactic and Dual-Trigger, and that comparison varies two factors at once: the trigger set and the poisoned-data generator. The syntax baseline uses SCPN, while the dual-trigger condition uses a LoRA-tuned LLM, and Table 6 shows the two generators produce data of very different quality. Higher-quality, semantically faithful paraphrases can themselves improve ASR and CACC, so the improvement cannot be attributed to the subjunctive trigger without a same-generator syntax-only condition. The subjunctive-only experiment in Table 5 is intended to show independent activation, but those test samples come from the same dual-trigger-fine-tuned LLM, and no parse-based verification ensures the syntax template is absent. Given the examples in Table 9, contamination is likely. These issues are fixable: a matched ablation and a syntax-presence check would settle whether the second trigger contributes anything. The reader's conditional verdict is appropriate, and this stress-test does not identify a reason to change it.","tokens_in":17413,"tokens_out":6043,"duration_ms":58639,"concrete_test":"Run a matched same-generator ablation: using the same Qwen2-72B-Chat LoRA generator, prompt it (a) to rewrite each sample into S(SBAR)(,)(NP)(VP)(.) with no subjunctive mood, (b) to rewrite into subjunctive mood without that template, and (c) both. Train the same three victim models with the same poisoning rates and report ASR/CACC for all three conditions. Additionally, parse 200 samples from condition (b) and from the paper's Table 5 test sets with Stanford Parser or Benepar and report the fraction matching S(SBAR)(,)(NP)(VP)(.). If condition (a) matches dual-trigger ASR, or if condition (b) mostly contains the template, the paper's dual-trigger and independent-activation claims are unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central comparative claim in Section 4.2 (Table 4) is that Dual-Trigger 'significantly outperforms' the syntax-based method. But the syntax-only condition is generated with SCPN while the dual-trigger condition is generated with a LoRA-tuned Qwen2-72B-Chat model (Section 3.2.3). The generator is therefore varied together with the trigger set. The quality gap is large: in Table 6, SCPN-based Syntactic poison data have SSA 0.049-0.168 versus 0.744-0.895 for Dual-Trigger and roughly 3-13x more grammatical errors. A model trained on low-quality, semantically shifted paraphrases may simply learn the task worse or learn a distorted trigger distribution; the ASR gain in Table 4 can be caused by the generator upgrade alone. The subjunctive-only test in Section 4.3 (Table 5) does not resolve this: those samples are produced by the same customized LLM, which was fine-tuned using dual-trigger input-output pairs, and no parse check is reported to confirm that the S(SBAR)(,)(NP)(VP)(.) template is absent. Since the Table 9 examples all contain initial 'If/Were/Suppose' conditional clauses, the 'subjunctive-only' test set likely still carries the first-layer syntax trigger. Consequently, neither the superiority of the second trigger nor the independence of the two triggers is established by the experiments as reported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a dual-trigger invisible textual backdoor attack for large language models, combining the syntactic template S(SBAR)(,)(NP)(VP)(.) with subjunctive mood as two independent abstract triggers. Poisoned data are generated by a LoRA-fine-tuned Qwen2-72B-Chat model rather than by the SCPN network used in prior syntactic attacks. The authors evaluate on SST-2, OLID, and AG's News, attacking Qwen2-72B-Instruct, Llama3-8B-Instruct, and Llama3.2-3B-Instruct, and compare with BadNet, InsertSent, and a syntactic baseline. They report near-100% ASR, high clean accuracy, better robustness than baselines under three defenses, and higher-quality poisoned data in terms of perplexity, grammar, and semantic similarity, and they claim the two trigger layers can be activated independently.","tokens_in":17687,"tokens_out":4281,"duration_ms":40293,"significance":"If the central comparative claim were established, the idea of combining two abstract linguistic features as independently activatable backdoor triggers would be a useful contribution to the textual-backdoor literature, and the customized-LLM data-generation pipeline is a practical engineering contribution. The paper's strengths include the use of public datasets and victim models, a multi-metric poisoned-data quality evaluation, evaluation against several defense strategies, and the public release of code and data on GitHub. However, the headline comparison between the dual-trigger method and the syntactic baseline is currently confounded, and the evidence for independent activation of the mood trigger is incomplete; both issues are fixable with additional experiments. For this reason the paper needs major revision before the central claims can be accepted.","major_comments":[{"comment":"The claim that 'the dual-trigger attack method using dual triggers significantly outperforms the syntax-based method' is confounded by a simultaneous change in the poisoned-data generator. The Syntactic baseline is generated with SCPN, while Dual-Trigger uses a LoRA-fine-tuned Qwen2-72B-Chat model. Table 6 shows that the SCPN-generated syntax-poisoned data are of much lower quality (SSA 0.049-0.168, GEN 1541-1844) than the customized-LLM data (SSA 0.744-0.895, GEN 89-1079). The ASR improvement in Table 4 could therefore be caused by the generator upgrade alone, and the contribution of the second trigger (subjunctive mood) is not isolated. Please add a syntax-only condition generated with the same customized LLM, or a dual-trigger condition generated with SCPN, and report the ASR/CACC for those conditions.","section":"Section 3.2.3, Section 4.2 (Table 4)"},{"comment":"The 'subjunctive-only' test does not establish that the two triggers can be activated independently. The subjunctive-only test samples are produced by the same customized LLM, which was fine-tuned on dual-trigger input-output pairs, and no parse-tree or rule-based check is reported to confirm that the S(SBAR)(,)(NP)(VP)(.) syntactic template is absent. Since all examples in Table 9 begin with an 'If/Were/Suppose' conditional clause, they already realize the S(SBAR)(,)(NP)(VP)(.) structure, so the subjunctive-only set likely still contains the first-layer trigger. Please provide a parse audit of the subjunctive-only test set and, ideally, a human-verified subset that contains subjunctive mood without the target syntactic template.","section":"Section 4.3 (Table 5)"},{"comment":"The claim that Dual-Trigger 'significantly outperforms' Syntactic is not supported by all reported cells. For example, on OLID with Llama3.2-3B-It, Syntactic achieves ASR 99.52 while Dual-Trigger achieves 96.51, and on AG's News both methods are at or near 99.9-100. The caption mentions bold and underlined formatting for significance, but the table does not show such formatting, and no significance tests, variances, or confidence intervals are reported. Please clarify the exact comparison statistic and provide significance testing or error bars over multiple runs.","section":"Section 4.2 (Table 4)"}],"minor_comments":[{"comment":"In the Back-translation Paraphrasing block, the BadNet entry '56.67(43.33)' appears to be missing a minus sign; it should read 56.67(-43.33) to match the other entries.","section":"Table 8"},{"comment":"The sentence 'for the AG's News dataset, the optimal poisoning rates of the three models are determined to be 5%, 10%, and 10%, respectively' is ambiguous about which rates correspond to which of the three victim models. Please specify the mapping or present the rates in a table.","section":"Section 4.1.5"},{"comment":"The description of the customized LLM omits several implementation details needed for reproducibility, such as the LoRA rank, alpha, and target modules, the prompt/instruction template used for transformation, and the exact generation parameters (temperature, top-p, etc.). Please include these in an appendix or supplementary material.","section":"Section 3.2.3"},{"comment":"The introduction states that 'all the known textual backdoor attack methods use a single-trigger as their trigger mechanism'; this is a strong claim and should be softened or supported with a more systematic citation of recent multi-trigger or multi-style attacks, if any exist.","section":"Section 1"}],"recommendation":"major_revision","confidential_remarks":"The main issue is the generator confound in the central comparison and the lack of a parse check for the subjunctive-only test set. These are load-bearing but fixable within the manuscript's scope: adding a syntax-only LLM-generated condition and a parse-audited subjunctive-only condition would resolve both concerns. If the authors can also provide significance testing or multiple runs, the revision would be much stronger. The public code/data availability is a point in their favor."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper proposes a dual-trigger textual backdoor: a rare syntactic template plus subjunctive mood, with poisoned data generated by a LoRA-tuned Qwen2-72B. The combination is new relative to the single-trigger literature, and the high ASR numbers on three datasets and three models suggest the attack works. The data-quality evaluation (SSA, PPL, LanguageTool) is a genuine attempt to measure stealth, and they ship code and data. Credit where due: the idea of pairing two grammatical features as independently activatable triggers is worth exploring, and the LLM-based generator is a sensible upgrade over SCPN.\n\nThe soft spots are real. The core comparison in Table 4 is confounded: the Syntactic baseline uses SCPN-generated poison data, while Dual-Trigger uses the custom LLM. Table 6 shows SCPN's SSA is 0.05-0.17, while Dual-Trigger's is 0.74-0.90. So the ASR gain could simply be better data, not the added second trigger. A syntax-only condition with the same LLM generator is missing. The subjunctive-only test in Table 5 is also not convincing: those samples come from the same LLM fine-tuned on dual-trigger pairs, and every example in Table 9 starts with If/Were/Suppose, which fits the SBAR template. Without a parse check, the independence claim is unverified. Additionally, the claim that dual-trigger significantly outperforms syntax is not consistent with their own Table 4: on OLID with Llama3.2-3B, Syntactic reaches 99.52 ASR vs 96.51 for Dual-Trigger. No repeated-seed variance is reported anywhere.\n\nThis is a solid, incremental paper with a load-bearing experimental-control problem. It deserves a serious referee, but the referee should require a syntax-only ablation with the same generator, a parse-level check that subjunctive-only samples lack the syntax template, and error bars. If those come back clean, the paper would be a useful contribution to the backdoor-attack literature. As-is, the central comparative claim is not fully supported.\n\nRecommendation: send to peer review, conditional on fixing the confound.","headline":"Dual-trigger backdoor idea is new and worth exploring, but the key comparison is confounded by a generator switch and the independent-trigger test is likely contaminated, so the central claims need an ablation before they can be trusted.","tokens_in":18252,"tokens_out":2424,"would_cite":false,"duration_ms":22015,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Combining a rare sentence structure with subjunctive mood lets a backdoor hit near-100% attack success on language models while staying invisible.","keywords":["textual backdoor attack","dual-trigger","abstract text features","syntactic trigger","subjunctive mood","poisoned dataset generation","backdoor defense","large language models"],"falsifier":"Re-run the dual-trigger pipeline with a syntax-only condition generated by the same fine-tuned LLM, i.e., sentences following S(SBAR)(,)(NP)(VP)(.) without subjunctive mood, on the same datasets and victim models. If the syntax-only LLM condition matches the dual-trigger attack success rate, the claim that the mood layer is responsible for the improvement is falsified; if it falls short, the claim is supported.","tokens_in":17176,"feed_emoji":"🎭","tokens_out":8200,"duration_ms":74313,"temperature":0.7,"pith_summary":"This paper tries to show that a textual backdoor in a large language model can be made both more effective and more covert by arming it with two independent grammar-based triggers instead of one. The proposed dual trigger combines a rare sentence template, the constituency parse structure S(SBAR)(,)(NP)(VP)(.), with the subjunctive mood, so that a poisoned model reacts to either feature alone. On three text-classification tasks and three instruction-tuned LLMs, the authors report attack success rates near 100 percent, clearly above a syntax-only baseline and comparable to insertion-based attacks, with clean accuracy nearly unchanged. They also report that the poisoned text scores better on automated quality checks, and that the attack loses far less effectiveness under word-removal and sentence-paraphrasing defenses than the baselines. If these results hold, trigger multiplicity is a viable route to backdoors that are both invisible and resilient.","feed_headline":"Dual-trigger backdoor hits near-100% attack success on LLMs","feed_subtitle":"Pairing a rare sentence shape with subjunctive mood beats syntax-only attacks and survives text defenses.","key_machinery":"The carrying object is the dual trigger: a layered combination of a low-frequency constituency parse template, S(SBAR)(,)(NP)(VP)(.), and the subjunctive mood, with a fine-tuned LLM as the poison-data generator. The template provides a rare structural marker the model seldom encounters in clean text, while the subjunctive provides a semantic-grammatical marker that slots naturally into that template, keeping the sentence fluent. The generator is equally load-bearing: the paper argues that only a meaning-preserving, grammatically fluent generator can make abstract dual triggers work at scale, and it supports that claim by measuring poisoned-data quality through semantic similarity, perplexity, and grammar-error counts. Together the two trigger layers are meant to make the backdoor activatable by either feature independently, which is what the authors credit for the attack's flexibility and resilience.","core_discovery":"The paper's central claim is that two grammatical abstractions can be stacked into one trigger mechanism, like a double landmine: the victim model learns to associate either the rare syntactic template or the subjunctive mood with the attacker's target label, and at inference time either trigger activates the backdoor. The syntax template is chosen because it is one of the least frequent common parse structures, making it distinctive; the subjunctive mood is chosen because its clause patterns fit naturally inside that template, so rewriting a sentence into subjunctive form preserves fluency and meaning. To produce such sentences at scale, the authors replace a generic paraphrase generator with a fine-tuned LLM that rewrites clean samples into subjunctive-mood versions of the template, and they report that this poisoned data has lower perplexity, fewer grammatical errors, and higher semantic similarity to the originals than existing methods. The experiments are presented as evidence that the dual-trigger model reaches near-perfect attack success while keeping clean accuracy close to the benign model, and that it survives word-removal and sentence-level defenses better than its single-trigger counterparts.","pith_inferences":["An untested extension is whether the mood layer alone, without the syntax template, would already produce high attack success; the paper only evaluates the mood layer as one half of the combined trigger.","Because the syntax-only comparison is generated by a different, older generator, the reported gap may partly reflect generator quality rather than the second trigger; a same-generator syntax-only arm would settle this.","The claimed resilience to word-removal defenses likely holds because the attack has no rare token to delete; a defense that models mood statistics rather than rare words could target the second layer and is not tested here.","If the mechanism generalizes, other grammatical pairs (e.g., voice combined with a rare tense, or negation combined with passive structure) should yield similar dual-trigger backdoors; the paper demonstrates only syntax-plus-subjunctive."],"forward_implications":["A backdoor can be built so that either of two unrelated linguistic features triggers it; removing or paraphrasing one trigger no longer disables the attack.","Abstract-feature backdoors can close the attack-success gap with insertion-based methods, reaching near-100% ASR without inserting any detectable token.","Poisoned-data quality can be quantified and compared through perplexity, grammar-error counts, and semantic similarity, and a fine-tuned LLM generator can dominate an older paraphrase-network generator on those metrics.","Word-removal defenses that hunt for rare inserted tokens will degrade insertion attacks far more than this dual-trigger attack.","A small set of hand-crafted sentence-transformation examples is enough to fine-tune a generator that mass-produces poisoned samples for a new task."],"supporting_citations":[{"why":"Supplies the syntax-trigger baseline, the choice of the rare syntactic template, and the two sentence-level defenses used for comparison.","marker":"[15]"},{"why":"Supplies the sentence-insertion baseline and the ASR/CACC evaluation conventions the paper follows.","marker":"[13]"},{"why":"Supplies the word-insertion (BadNet-style) baseline adapted for text attacks.","marker":"[14]"},{"why":"Supplies the word-removal defense whose impact on attack success rates is central to the robustness claim.","marker":"[26]"},{"why":"Supplies the earlier paraphrase-network generator that the paper's LLM-based poisoned-data generator is compared against on quality metrics.","marker":"[29]"},{"why":"Establishes abstract-feature triggers (tense and voice) and frames the semantic-preservation problem that the dual-trigger method addresses.","marker":"[12]"},{"why":"Provides the parsing tool used to measure syntax-template frequencies and select the rare template.","marker":"[28]"}],"fun_headline_variants":["Dual-trigger backdoor: syntax plus mood hits 100% success","Invisible dual-trigger backdoor beats defenses at 99% success","Double landmine backdoor: syntax or mood triggers attack","Dual-trigger backdoor: syntax + subjunctive = 100% attack"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper attributes the attack's gain over syntax-only attacks to the newly added mood trigger, but the two conditions also use different poison-data generators, so that causal attribution has not been isolated.","fun_headline_variants_meta":{"raw":{"variants":["Dual-trigger backdoor: syntax plus mood hits 100% success","Invisible dual-trigger backdoor beats defenses at 99% success","Double landmine backdoor: syntax or mood triggers attack","Dual-trigger backdoor: syntax + subjunctive = 100% attack"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000432,"raw_usage":{"total_tokens":2253,"prompt_tokens":1047,"completion_tokens":1206,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":663,"completion_tokens_details":{"reasoning_tokens":1124}},"tokens_in":663,"tokens_out":1206,"duration_ms":8422,"temperature":1.0,"reasoning_tokens":1124,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T05:27:15.935024+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the dual-trigger pipeline with a syntax-only condition generated by the same fine-tuned LLM, i.e., sentences following S(SBAR)(,)(NP)(VP)(.) without subjunctive mood, on the same datasets and victim models. If the syntax-only LLM condition matches the dual-trigger attack success rate, the claim that the mood layer is responsible for the improvement is falsified; if it falls short, the claim is supported.","supporting_citations":[{"cited_title":"IEEE Access 7, 138872–138878 (2019)","cited_arxiv_id":null,"evidence_quote":"Supplies the sentence-insertion baseline and the ASR/CACC evaluation conventions the paper follows."},{"cited_title":"In: Proceedings of the 37th Annual Computer Security Applications Conference, pp","cited_arxiv_id":null,"evidence_quote":"Establishes abstract-feature triggers (tense and voice) and frames the semantic-preservation problem that the dual-trigger method addresses."},{"cited_title":"In: Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp","cited_arxiv_id":null,"evidence_quote":"Provides the parsing tool used to measure syntax-template frequencies and select the rare template."}],"review_version":1}