{"id":"ab13cb1c-52bf-4791-b275-898fd68206f4","arxiv_id":"2412.12192","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Adversative in-context examples ('Sure... However...') defend many LLMs against prefilling jailbreaks but cause over-refusal, so the defense trades safety for helpfulness.","lead":"The paper tests whether giving an LLM a few example answers that start with agreement but then say 'however' and refuse can block prefilling jailbreak attacks, where the attack already begins the model's response with 'Sure' or 'Yes'. It finds the trick sharply lowers attack success on most open models, but makes models refuse too many benign questions, so there is no free lunch.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Adv and Refusal demonstrations differ in both the adversative 'However' and the affirmative lead-in 'Sure, here is how...'; the claimed adversative mechanism is not isolated.","rationale":"The strongest empirical effect—large ASR drops with Adv demonstrations—appears real and is consistent across models and benchmarks. My concern is about the causal attribution. The paper compares Adv demonstrations that contain both an affirmative lead-in and an adversative 'However' against Refusal demonstrations that contain neither. The prefilling attack itself supplies the affirmative prefix, so the Adv demonstrations may simply be a better template for continuing after that prefix with a refusal, independent of 'However.' The paper does not include an ablation that removes the adversative conjunction while keeping the lead-in, nor one that keeps the conjunction without the lead-in. Without this, the abstract's 'by employing adversative sentence structures' is not established. This is not an accusation; it is a standard confound in demonstration-based intervention studies. The proposed check is cheap and decisive. If the non-adversative lead-in condition performs like Adv, the paper should be revised to claim post-affirmative refusal placement rather than adversative structure. The reader's leakage concern is related but distinct; I partially agree because both concerns center on the role of demonstration content versus structure. The missing Table 4 also weakens the 'no free lunch' over-defense claim, but the central defense-effectiveness claim is supported by Tables 1, 2, 4-7, so I would not reject; the verdict remains conditional pending the ablation and missing data.","tokens_in":14078,"tokens_out":8469,"duration_ms":75080,"concrete_test":"Re-run the AdvBench k=6 experiments on Vicuna-7b and Llama3.1-8b with four demonstration sets: (1) the original Adv template; (2) 'Sure, here is how to do it. I cannot assist with this question.' (no 'However'); (3) 'I cannot assist with this question. However, let me explain why.' (adversative but no affirmative lead-in); (4) the Refusal baseline. Compare rule-based and model-based ASR. If (2) ≈ (1) and (3) ≈ (4), the adversative conjunction is not the active ingredient and the central claim should be revised to post-affirmative refusal placement.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Table 1 and Appendix B, the Adv/Adv-mul demonstrations begin with an affirmative lead-in ('Sure, here is how to do it...') followed by an adversative conjunction ('However') and a refusal, whereas the Refusal baseline is merely 'No, I cannot answer.' The prefilling attack injects affirmative tokens such as 'Sure, here is...' at the start of the response. The Adv demonstrations therefore match the attack's opening template, potentially teaching the model to continue after an affirmative prefix with a refusal; the 'However' may be incidental. The paper's central claim—that adversative sentence structures are the active ingredient—is not supported because the two demonstration families differ in two variables simultaneously. This is load-bearing: if a non-adversative lead-in such as 'Sure, here is how to do it. I cannot assist with this question.' yields the same ASR reduction, the defense is not about adversative structure but about post-affirmative refusal placement. The paper itself concedes in Section 6 that effectiveness is sensitive to textual similarity between demonstrations and input queries, which is consistent with the lead-in/content explanation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes using in-context learning (ICL) demonstrations that contain adversative sentence structures (e.g., \"Sure, here is how to do it. However, I cannot assist...\") to defend open-weight LLMs against prefilling jailbreak attacks, where the attacker pre-fills affirmative tokens. Across three benchmarks (AdvBench, JailBench, SorryBench) and nine models, the authors report large reductions in attack success rate (ASR) for their Adv and Adv-mul variants relative to a no-defense baseline and a refusal-demonstration baseline, for both rule-based and model-based ASR. They also study safety alignment, combined attacks, number of demonstrations, and over-defense. The paper concludes that ICL with adversative demonstrations is effective but causes over-defense, and that effectiveness is sensitive to textual similarity between demonstrations and queries.","tokens_in":14298,"tokens_out":4897,"duration_ms":39345,"significance":"If the result holds, the empirical finding that simple prompt-only demonstrations can neutralize prefilling attacks without fine-tuning is practically valuable, as prefilling attacks remain a known weakness of open-weight models. The study's breadth is a strength: it evaluates nine models, three benchmarks, two evaluation metrics, multiple prefilling token counts, and combined jailbreak attacks, with no fitted parameters. The paper also honestly documents over-defense and the sensitivity to textual similarity. However, the central mechanistic claim that the adversative conjunction is the active ingredient is not isolated from the affirmative lead-in, and the \"robust across model sizes and complex attacks\" claim is stronger than the data show. With additional ablations and toned-down claims, this could be a solid empirical contribution.","major_comments":[{"comment":"The comparison that supports the paper's central claim is confounded. In Table 1 and Appendix B, the Adv and Adv-mul demonstrations begin with an affirmative lead-in (\"Sure, here is how to do it...\") followed by \"However\" and a refusal, while the Refusal baseline is only \"No, I cannot answer.\" These two demonstration families differ in both the lead-in and the adversative conjunction. Because the prefilling attack itself injects an affirmative prefix such as \"Sure, here is...\", the Adv demonstrations may simply teach the model to continue after an affirmative prefix with a refusal; the \"However\" may be incidental. The paper's conclusion that \"adversative sentence structures\" are the active ingredient is therefore not established. An ablation is needed, for instance comparing \"Sure, here is how to do it. I cannot assist with this question.\" (lead-in without adversative conjunction) and \"However, I cannot assist with this question.\" (adversative without affirmative lead-in) against the current Adv template. Section 6's admission that effectiveness is sensitive to textual similarity is consistent with the lead-in/content explanation, not uniquely with the adversative explanation.","section":"§5.1, Appendix B"},{"comment":"Section 5.5 refers to \"Table 4\" that summarizes over-defense results, but the manuscript as provided contains no Table 4; only Figure 4 is present. The over-defense claim is an explicit contribution of the paper (and the basis for the \"no free lunch\" conclusion), so the missing table with the benign-query refusal rates and harmful-query ASRs is load-bearing evidence that must be included or explicitly referenced to an appendix.","section":"§5.5"},{"comment":"The abstract and Section 5.1 claim that adversative demonstrations provide \"robust defense across various model sizes and complex jailbreak attacks.\" This is contradicted by the paper's own data. In Table 1, Mistral-7B-v01 remains mostly vulnerable (Adv rule-based ASR 83.7, Adv-mul 60.8; model-based 84.8/63.1). In Table 2, combining prefilling with AIM, Evil Confidant, or Refusal Suppression raises ASR substantially; for example, Llama3.1-8b Adv-mul goes from 17 (prefill only) to 44 (Prefill+AIM), 79 (Prefill+EC), and 76 (Prefill+RS) on rule-based ASR, and Vicuna-7b reaches 68/75 on Prefill+EC. The paper's more careful statement that adversative demonstrations are \"relatively more effective\" than refusal demonstrations is supported, but the \"robust\" characterization is an overclaim that should be revised.","section":"Abstract, §5.1, §5.3"},{"comment":"Appendix A describes generating harmful demonstration questions with Wizard-13b and filtering with GPT-4o-mini, using ten categories and examples that resemble JailBench taxonomy items. The paper does not report any overlap or near-duplicate analysis between these generated demonstrations and the test queries in AdvBench, JailBench, and SorryBench used in Section 4. Since Section 6 attributes effectiveness partly to textual similarity between demonstrations and queries, the lack of a leakage analysis leaves open the possibility that the measured ASR reductions for some models reflect memorization or retrieval of near-identical test content rather than a general ICL capability. The authors should either demonstrate disjointness or quantify how many test queries are near-duplicates of demonstration questions.","section":"Appendix A, §4"}],"minor_comments":[{"comment":"The caption states \"The utilized benchmarks are AdvBench and Jailbench\" even though SorryBench results appear in the same table; the caption should list all three benchmarks.","section":"Table 1 caption"},{"comment":"The caption contains a typo: \"safty\" should be \"safety.\"","section":"Figure 2 caption"},{"comment":"The example for \"Harassment/Discrimination\" is duplicated in the list of category examples; one duplicate should be removed.","section":"Appendix A"},{"comment":"The reference for \"Anil et al., Many-shot jailbreaking\" lacks a year and venue, and the entry for \"Onel AJ\" is inconsistently formatted; these should be brought into citation style.","section":"References"},{"comment":"Table 3 lists Llama2-70b and Llama3.1-70b among evaluated LLMs, but these models do not appear in the main results tables; the authors should either report their results or clarify why they were excluded.","section":"Table 3 and Table 1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is marked \"Work in progress\" in the footnote, which is unusual for a formal submission. The missing Table 4 and the lack of code or artifacts make verification difficult; a public artifact with the demonstration-generation pipeline and overlap analysis would substantially strengthen the paper. The central confound between the affirmative lead-in and the adversative conjunction is the main substantive issue and will require an ablation study rather than text changes alone."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper reports a real, useful empirical phenomenon. A handful of in-context demonstrations that begin with an affirmative lead-in and then say \"However, I cannot assist\" reduce prefilling jailbreak attack success on most of the nine open-weight models tested, often dramatically. That is worth knowing about. But the central mechanistic claim—that adversative sentence structures are the active ingredient—is not actually tested, because the Adv demonstrations differ from the Refusal baseline in two variables at once: the \"Sure, here is how...\" lead-in and the \"However\". The authors' own Section 6 concedes that effectiveness is sensitive to textual similarity between demonstrations and input queries, which is exactly what you would expect if the lead-in, not the adversative conjunction, is doing the work.\n\nWhat is genuinely new: prior work used adversative response templates only in fine-tuning (Qi et al. 2024a); showing the same idea works as ICL across models and benchmarks, with systematic ablations on demonstration count, prefilled token count, and combined attacks, is a solid new application. The main tables are consistent: Adv and Adv-mul beat Refusal almost everywhere, and the \"no free lunch\" point—that the defense causes over-refusal and degrades under combined attacks—is honest and useful.\n\nThe soft spots: (1) the confound above is load-bearing; a proper control would swap the lead-in to something like \"Sure, here is how to do it. I cannot assist with this question.\" If that also works, the story is about post-affirmative refusal, not adversative structure. (2) The over-defense section cites \"Table 4\" but that table is missing from the manuscript. Since that table is the only evidence for the over-defense claim, it needs to appear. (3) Mistral-7B-v01 is a clear outlier (Adv-mul ASR still 60–85%), so \"robust across model sizes\" is overstated. (4) No error bars, code, or data; and no leakage check between the generated demonstrations and the test benchmarks. Those are minor-to-moderate concerns, not fatal ones.\n\nBottom line: the empirical effect is probably real and worth publishing as a defense study, with the mechanism claim softened or controlled. The paper deserves peer review; a careful referee should ask for the missing table, a lead-in control, and artifacts. Whoever reads it will learn something about prefilling attacks and a cheap prompt-only mitigation, even if the exact reason it works remains open.","headline":"A real empirical effect—ICL with affirmative lead-ins followed by 'However, I cannot assist' sharply cuts prefilling jailbreak success on most open-weight models—but the paper's 'adversative structure' mechanism is confounded with the lead-in itself, and the missing over-defense table needs to reappear.","tokens_in":14826,"tokens_out":2581,"would_cite":true,"duration_ms":22819,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that in-context learning with adversative demonstrations — 'Sure, here is how to do it.","keywords":["jailbreak attacks","prefilling attack","in-context learning","LLM safety","adversative demonstrations","attack success rate","over-defensiveness","safety alignment"],"falsifier":"Run the same evaluation but build the demonstration pool by sampling harmful queries directly from the test benchmarks (e.g., use AdvBench queries as demonstrations for JailBench tests). If ASR remains near zero, the defense is robust to demonstration novelty; if ASR rises sharply, the reported defense depends on the demonstration pool being disjoint from the test queries. A cheaper check is to compute the maximum n-gram overlap between the generated demonstrations and the three benchmarks; high overlap confirms the leakage risk.","tokens_in":13906,"feed_emoji":"🛡️","tokens_out":7843,"duration_ms":58036,"temperature":0.7,"pith_summary":"The paper tries to show that a simple, compute-free defense works against a stubborn class of jailbreak: when the attacker pre-fills the model's response with affirmative tokens like 'Sure, here is how to do it,' the model tends to continue along that path. The authors' claim is that including demonstrations with adversative structures — 'Sure... However, I cannot assist' — in the prompt teaches the model to refuse despite the affirmative prefix, across most tested open-weight LLMs and benchmarks. A sympathetic reader should care because prefilling attacks currently defeat standard safety alignment, and ICL is a cheap alternative that requires no fine-tuning. The paper also shows the defense has a cost: it makes models over-refuse benign queries, and the over-refusal appears independent of model size.","feed_headline":"Adversative prompts block prefilling jailbreaks across open LLMs","feed_subtitle":"A single 'Sure... However' example in the prompt, no fine-tuning, drops attack success on most open-weight LLMs.","key_machinery":"The mechanism is the adversative sentence structure in the ICL demonstrations: a template response that starts with affirmative continuation ('Sure, here is how to do it...') and then pivots with a contrastive conjunction ('However') into a refusal. The paper operationalizes this as 'Adv' (a single adversative response) and 'Adv-mul' (a pool of multiple versions with the 'However' placed at different positions, randomly sampled per query). This structure does the argument's work by making the model's continuation after the prefilled affirmative tokens switch into refusal, whereas a plain refusal demonstration only models the 'no' answer and fails to override the affirmative prefix.","core_discovery":"The central claim is that ICL with adversative demonstrations is an effective defense against prefilling jailbreak attacks. Concretely, on AdvBench, JailBench, and SorryBench, replacing plain refusal demonstrations (e.g., 'No, I cannot answer') with adversative ones (e.g., 'Sure, here is how to do it. However, I reconsider and cannot assist with this question.') drops attack success rates from high baselines to near zero on most of the nine evaluated open-weight models, including Falcon-7b/11b, Llama-2-7b/13b, Llama-3.1-8b, Llama-3.2-3b, and Vicuna-7b/13b; Mistral-7B-v01 is the main exception. The paper further shows that safety alignment (comparing Wizard-Vicuna-13B with Vicuna-13B) does not mitigate the prefilling attack, that more demonstrations and larger models strengthen the defense, and that combined attacks (prefill plus AIM, Evil Confidant, or Refusal Suppression) degrade it but still leave adversative ICL relatively more effective than refusal ICL. The 'no free lunch' is the over-defensiveness: the same demonstrations cause LLMs to refuse benign queries, and this behavior appears independent of model size.","pith_inferences":["The authors hint that the defense works because 'However' pivots are rare in pretraining data; this could be tested by varying the pivot word (e.g., 'Yet', 'But') and by moving the conjunction to different positions within the demonstration.","Because the demonstration pool was generated independently of the evaluation benchmarks, a test-set-overlap check is the natural next experiment: if generated demonstrations are semantically near any AdvBench, JailBench, or SorryBench query, the near-zero ASR may partly reflect retrieval rather than a general refusal skill.","The over-defensiveness result implies a calibration design: adversative ICL could be switched on only for inputs resembling the harmful-question distribution, preserving benign utility while blocking prefilled completions.","Extending the same prompt-level method to closed commercial LLMs (the paper evaluates only open-weight models) would show whether the adversative template transfers, and would give system-prompt writers a zero-training defence to try."],"forward_implications":["Safety alignment alone does not stop prefilling attacks; adding adversative ICL demonstrations does, on most tested open-weight LLMs and across three benchmarks.","Increasing the number of adversative demonstrations (up to about eight) and using larger model sizes further reduces attack success, while plain refusal demonstrations remain ineffective.","Combining prefilling with other jailbreak techniques (AIM, Evil Confidant, Refusal Suppression) weakens but does not erase the defense; adversative ICL still beats refusal ICL in those combined settings.","The same demonstrations cause over-defensiveness: LLMs refuse benign queries more often, and this trade-off holds across model sizes, so deployment needs a selectivity mechanism."],"supporting_citations":[{"why":"Introduces the prefilling jailbreak attack and the observation that safety alignment is shallow (only a few tokens deep); the paper's problem definition and baseline vulnerability rest on this.","marker":"Qi et al. (2024a)"},{"why":"Provides the AdvBench benchmark and the rule-based ASR evaluation method used to measure attack success.","marker":"Zou et al. (2023)"},{"why":"Provides JailBench, one of the three evaluation benchmarks, and the ten-category taxonomy used for demonstration generation.","marker":"Chao et al. (2024)"},{"why":"Provides SorryBench and the model-based ASR judge, the second evaluation metric.","marker":"Xie et al. (2024)"},{"why":"Supplies the attack taxonomy (AIM, Evil Confidant, Refusal Suppression) used in the combined-attack experiments.","marker":"Wei et al. (2024)"},{"why":"Prior work on ICL demonstrations for jailbreak defense; the paper shows its refusal-style demonstrations fail against prefilling, motivating the adversative variant.","marker":"Wei et al. (2023)"},{"why":"Safety alignment via RLHF, the incumbent defense the paper shows is insufficient against prefilling.","marker":"Bai et al. (2022)"}],"fun_headline_variants":["ICL adversative demos stop prefilling jailbreaks, but over-refuse","No free lunch: ICL stops prefilling attacks but over-defends","Adversative ICL blocks prefilling jailbreaks, costs over-refusal","Prefilling jailbreak? One adversative demo in prompt kills it","Sure... However: ICL defense against prefilling attacks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The demonstrations used for ICL were generated separately from the test queries, and the paper assumes they are semantically disjoint from the harmful queries in AdvBench, JailBench, and SorryBench; if the demonstration pool overlaps with or retrievably resembles the test set, the measured defense could be test-set memorization rather than a general learned refusal pattern.","fun_headline_variants_meta":{"raw":{"variants":["ICL adversative demos stop prefilling jailbreaks, but over-refuse","No free lunch: ICL stops prefilling attacks but over-defends","Adversative ICL blocks prefilling jailbreaks, costs over-refusal","Prefilling jailbreak? One adversative demo in prompt kills it","Sure... However: ICL defense against prefilling attacks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000194,"raw_usage":{"total_tokens":1407,"prompt_tokens":1053,"completion_tokens":354,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":669,"completion_tokens_details":{"reasoning_tokens":259}},"tokens_in":669,"tokens_out":354,"duration_ms":3640,"temperature":1.0,"reasoning_tokens":259,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:46:00.703498+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same evaluation but build the demonstration pool by sampling harmful queries directly from the test benchmarks (e.g., use AdvBench queries as demonstrations for JailBench tests). If ASR remains near zero, the defense is robust to demonstration novelty; if ASR rises sharply, the reported defense depends on the demonstration pool being disjoint from the test queries. A cheaper check is to compute the maximum n-gram overlap between the generated demonstrations and the three benchmarks; high overlap confirms the leakage risk.","supporting_citations":[],"review_version":1}