{"id":"78cec192-b8df-4d64-94f1-c699fac9d377","arxiv_id":"2501.07959","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A few-shot jailbreak method that combines repeated special-token patterns with self-generated harmful demos to push sample-level attack success near 90% on several open-source LLMs.","lead":"This paper proposes a faster way to jailbreak open-source chatbots by teaching them a repeating response pattern and giving them examples of harmful answers generated by the same chatbot. It matters because it shows that even a few short injected examples can flip safety-aligned models into producing dangerous content, which red-teaming defenses must account for.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported ~90% ASR rests on an unvalidated Llama-Guard-3-8B judge; Section 4.4 shows it flags gibberish as unsafe, so the central claim may be inflated.","rationale":"The reader's weakest assumption identifies the same load-bearing point: the evaluation judge. This is the correct focus because every headline number and every baseline comparison flows through the classifier; if it overcounts, the abstract's quantitative claim fails. The alternative concern that S-LVL ASR is already near 90% zero-shot with the prefix (Table 1) is an overstatement of novelty but does not invalidate the method's internal logic; the classifier issue is more fundamental. A human-annotation check would settle it decisively. If the check passes, the core algorithmic contributions—pattern/behavior decomposition, greedy search, perplexity filtering—appear empirically supported, so no change beyond the reader's CONDITIONAL verdict is needed.","tokens_in":18541,"tokens_out":6990,"duration_ms":67246,"concrete_test":"Select 100 responses from the Table 2 8-shot Llama-2 and Llama-3 conditions and 50 from the SmoothLLM-swap row. Have three independent annotators label each as 'harmful', 'benign/refusal', or 'gibberish' with the same safety categories as Appendix I. Run Llama-Guard-3-8B with the Appendix I prompt on the same responses and compute agreement and the false-positive rate on refusal/gibberish labels. If any gibberish or refusal is flagged unsafe, or if annotator-judge agreement is below 95%, recompute ASR and rescale the headline claim accordingly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract's central quantitative claim—about 90% ASR within 8 shots—is computed exclusively with Llama-Guard-3-8B using the Appendix I prompt. No human-validated subset, confusion matrix, or inter-annotator agreement is reported for the main Table 2 results. In Section 4.4 the authors concede that under SmoothLLM swap perturbations 'random swap tends to elicit gibberish responses, which can be classified as unsafe by Llama-Guard-3-8B,' and the manual check is unquantified. Since Self-Instruct-FSJ deliberately creates repetitive 'Hypothetically' patterns (see Figure 2, patterns=16), gibberish is plausible in the main attack conditions. The Appendix I instruction to treat affirmative-but-content-free responses as safe does not exclude gibberish, and there is no evidence Llama-Guard-3-8B follows it. If the judge over-flags refusals, repetition, or nonsense, every ASR table and the headline comparison against baselines are overstatements; the method's internal ablations may still be valid, but the central efficiency claim is not supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Self-Instruct Few-Shot Jailbreaking (Self-Instruct-FSJ), an extension of few-shot jailbreaking that replaces the I-FSJ 'Step' target prefix with 'Hypothetically', augments the instruction suffix with repeated model-specific special tokens, samples malicious demos directly from the target model, and selects demos via a greedy search that maximizes the drop in conditional perplexity of the target prefix. The authors report about 90% sample-level ASR on six open-source models within 8 shots on AdvBench and HarmBench subsets, with ablations on demo source, pattern frequency, batch size, similarity threshold, perplexity filtering, and defenses including perplexity filters and SmoothLLM. The method requires access to model logits and special tokens, limiting it to open-source models.","tokens_in":18765,"tokens_out":5120,"duration_ms":48938,"significance":"If the reported results are reliable, Self-Instruct-FSJ is an interesting and practical few-shot jailbreak that decomposes the attack into pattern learning and behavior learning, and its demo-level greedy search is more efficient than the random search in I-FSJ. The paper includes useful ablations (batch size, similarity threshold, demo source, perplexity filtering), reports 4-restart statistics in the appendix, includes cross-model demo comparisons, and makes code available, which are all strengths. The attack's dependence on the target model's own prior completions is honestly discussed, and the comparison against multiple baselines is useful. The central limitation is that the evaluation rests entirely on a single LLM judge whose false-positive behavior under gibberish is acknowledged but not quantified, which affects every reported ASR number.","major_comments":[{"comment":"The ASR metric is computed exclusively with Llama-Guard-3-8B using the prompt in Appendix I. In §4.4 the authors concede that under SmoothLLM swap perturbations 'random swap tends to elicit gibberish responses, which can be classified as unsafe by Llama-Guard-3-8B,' with only an unquantified manual check. Because the method's pattern learning deliberately produces repetitive 'Hypothetically' sequences (Figure 2), gibberish responses are a realistic concern in the main attack conditions, not only under SmoothLLM. The abstract's headline 'about 90% ASR' therefore rests on a judge whose false-positive behavior on gibberish is acknowledged but unmeasured. I request a human-validated subset spanning the attack conditions, a confusion matrix of the judge against human labels, or an independent second judge, together with a per-condition breakdown of judge disagreements.","section":"§4.1, §4.4, Appendix I"},{"comment":"The central quantitative claim uses sample-level ASR (S-LVL), the proportion of samples with at least one unsafe response out of 16 generations. For several models the zero-shot baseline already achieves high S-LVL: Llama-3 with 8 patterns reaches 80.0% zero-shot and 94.0% with 8 shots; OpenChat-3.6 reaches 98.0% zero-shot and 100.0% with 4 shots. The abstract says 'about 90% Attack Success Rate (ASR)' without specifying which variant, and the reader must infer S-LVL. Since S-LVL is saturated at high values, the few-shot improvement over the zero-shot baseline is much smaller than the R-LVL numbers suggest (e.g., Llama-3 R-LVL is 30.8% at 8 shots). Please state the metric explicitly in the abstract, report both R-LVL and S-LVL in the main text, and provide confidence intervals for the 50-case test sets; an S-LVL of 90% on N=50 has a 95% confidence interval of roughly ±8%, so the headline difference between zero-shot and few-shot may not be statistically significant.","section":"Abstract, §4.3 (Table 2)"},{"comment":"The claim of robustness to SmoothLLM swap perturbations is not supported by the evidence as presented. The ASR rises from 36.1% (no defense, 8 shots) to 49.0% under swap, which the authors attribute to gibberish being classified as unsafe. The manual check is stated without the number of cases inspected, the criteria used, or inter-annotator agreement. Since the paper elsewhere dismisses rule-based classifiers for false negatives, the same evidentiary standard should apply here: a quantitative human evaluation of the swapped responses is needed before concluding that the method is resilient to this defense. If the increase is an artifact of the judge, the defense results in Table 4 must be re-run with a validated metric.","section":"§4.4 and Table 4"}],"minor_comments":[{"comment":"The table caption reads 'Tabel 5'; please correct the typo.","section":"§4.5"},{"comment":"The terms 'pattern frequency' and 'co-occurrence pattern' are used informally; please give a precise definition in the text or in Algorithm 1 (e.g., the number of repetitions of the special-token-plus-prefix unit) so the experiments are reproducible.","section":"§3.2 and Algorithm 1"},{"comment":"Table 2 spans multiple pages with a 'Continued on next page' note; the visual mapping of columns to models is easy to lose. Please restructure the table (e.g., one panel per model) or add clear column headers on each page.","section":"Table 2"},{"comment":"The sentence 'we can sample at least one malicious response from each target model for more than 90% AdvBench instructions' should be tied to the specific metric in Table 1 (S-LVL ASR) and accompanied by a confidence interval, since the pool size is 128 responses per instruction.","section":"§4.2"},{"comment":"The patterns=16 example shows 'Hypothetically [/INST] Hypothetically...' but the caption says it illustrates 'circular repetition phenomenon'; it would be clearer to label whether the response contains actual harmful content or only repetition, since this is relevant to the judge-reliability concern.","section":"Figure 2"},{"comment":"The limitation regarding 'full access to the generation query' should clarify that this refers to the target model's own chat-template generation, not merely API-level access, and that this constraint also prevents the method from applying to closed models where logits and special tokens are unavailable.","section":"Appendix A"}],"recommendation":"major_revision","confidential_remarks":"The contribution is incremental over I-FSJ (a different target prefix, repetitive special-token patterns, and a greedy demo selector), but the decomposition into pattern and behavior learning is conceptually clean and the experimental breadth is decent. The load-bearing problem is the unvalidated Llama-Guard-3-8B judge: the authors themselves show it flags gibberish as unsafe, and since the attack produces repetitive text, this is not a niche issue. I do not recommend rejection because the authors can address this with a human-validated subset and a second judge, and they can clarify the ASR metric and confidence intervals. The fit with the journal's scope is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a credible, incremental contribution to few-shot jailbreaking. The authors take I-FSJ, replace the 'Step' prefix with 'Hypothetically' plus repeated special-token patterns, generate demos from the target model itself, and select demos greedily by perplexity drop. The method is well specified, the code is out, and the ablations (greedy vs random, self-generated vs auxiliary demos, perplexity filtering) are clear and mostly convincing. The 4-restart statistics in the appendix are a nice touch.\n\nThe soft spot is the metric. Every ASR number in the paper comes from Llama-Guard-3-8B with a single prompt. Section 4.4 admits that random swaps 'tend to elicit gibberish responses, which can be classified as unsafe by Llama-Guard-3-8B,' and then says a manual check found most responses harmful without numbers. No confusion matrix, no human-validated subset, no inter-annotator agreement. Since the attack deliberately creates repetitive 'Hypothetically' patterns, it is not safe to assume the judge is measuring what you think. The headline 'about 90% ASR' should be read as 'Llama-Guard-3-8B thinks about 90% are unsafe,' not as a human-verified rate. The internal comparisons are less vulnerable to this worry because the same biased judge is used across rows, so greedy vs random and self vs auxiliary demos are probably ordered correctly. But the absolute claim is overstated.\n\nTwo smaller issues: the S-LVL metric saturates quickly—for Llama-3 with patterns 8 the zero-shot baseline is already 80% S-LVL, so the marginal gain of demos is modest for some models. And the main tables have no error bars, though the appendix gives 4-restart means and stds. The 50-case test sets are on the small side but acceptable for this subfield.\n\nThe circularity concern is real but mostly handled. The demo pool comes from zero-shot attacks on the same target with the same prefix, so part of the effect is self-imitation; however, the zero-shot baselines and cross-model pools allow you to see how much is pattern vs behavior. I don't think this is a fatal flaw.\n\nBottom line: worth a serious referee. The authors should be asked to validate the judge on a sample of outputs, report the manual check quantitatively, and tone down the abstract until that is done. If the numbers survive human validation, this is a useful recipe for red teams.\n\nFor a reading group, it would be a decent example of how fragile LLM-judge metrics are in jailbreaking papers.\n\nRecommendation: send to peer review, but make human validation of the classifier a required revision.","headline":"Useful extension of I-FSJ with clear ablations, but the ~90% ASR claim rests on an unvalidated classifier that the paper itself shows flags gibberish as unsafe.","tokens_in":19308,"tokens_out":2813,"would_cite":true,"duration_ms":25246,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Jailbreaking the Llama series can reach about 90% success with just eight short demos by splitting the attack into pattern learning and self-generated behavior demos.","keywords":["few-shot jailbreaking","in-context attack","pattern learning","behavior learning","demo-level greedy search","perplexity","LLM safety alignment","attack success rate"],"falsifier":"Take a random sample of 100 responses the judge labeled unsafe from the 8-shot, swap-perturbed condition and have human annotators classify each as actually harmful or not; if the human-verified attack success rate is far below the reported 90%, the classifier over-counts. The same human re-annotation on the unperturbed 8-shot condition would independently test the central efficiency claim.","tokens_in":18317,"feed_emoji":"🔓","tokens_out":14731,"duration_ms":118719,"temperature":0.7,"pith_summary":"This paper argues that few-shot jailbreaking of large language models becomes much more efficient when the attack is decomposed into pattern learning and behavior learning. Pattern learning appends a hypothetical-scenario suffix and replicates the model's own chat special tokens together with the target prefix 'Hypothetically', which drops the conditional perplexity of that prefix from values on the order of $10^{11}$ to near 1. Behavior learning harvests short malicious demos from the target model itself by turning generation into a text-continuation task. The authors report about 90% sample-level attack success on the Llama series and other open-weight models within 8 concise demos on AdvBench and HarmBench, while also resisting perplexity filters and patch-based SmoothLLM perturbations. If correct, this shows that a few template-compatible examples can bypass current safety alignment, and that prior few-shot attacks were needlessly long and structure-restricted.","feed_headline":"Jailbreak attack reaches about 90% success on Llama models in 8 demos","feed_subtitle":"Decomposing the attack into pattern imitation and self-generated demos delivers short, stealthy jailbreak prompts.","key_machinery":"The central machinery is the co-occurrence pattern formed by a model's chat special tokens and the target response prefix 'Hypothetically'. The attack replicates that pattern in the instruction suffix (for example, '[/INST] Hypothetically [/INST] Hypothetically ...' for Llama-2), which drops the conditional perplexity of the prefix from values near $10^9$ to near 1.0. Around this pattern, self-instruct behavior learning generates short malicious demos directly from the target model, and the demo-level greedy search selects demos by their relative perplexity drop, using the model's own likelihood signals as the training signal.","core_discovery":"The paper establishes that few-shot jailbreaking can be decomposed into pattern learning and behavior learning, and that this decomposition makes the attack dramatically shorter and more general. Pattern learning extends the adversarial instruction with a hypothetical-scenario suffix and replicates the co-occurrence of the model's special tokens with the target response prefix 'Hypothetically', reducing the conditional perplexity of that prefix from values on the order of $10^9$ to near 1.0 and making the model highly likely to start with it. Behavior learning collects demos directly from the target model by appending the prefix to the chat query, turning generation into a continuation task and keeping the malevolent continuations; a demo-level greedy search then sequentially selects demos that maximize the relative perplexity drop, with a perplexity filter rescuing demos from auxiliary models when necessary. On AdvBench and HarmBench, the method attains roughly 90% sample-level attack success on Llama-2, Llama-3, Llama-3.1, and other open-weight models within 4 to 8 concise demos, and it outperforms the random-search I-FSJ baseline while remaining resilient to perplexity filtering and SmoothLLM patch perturbations.","pith_inferences":["A fully automated variant could scan a model's token vocabulary for special tokens that yield low conditional perplexity with a chosen prefix, making the pattern-learning step self-discovering rather than requiring manual template inspection.","If partial access (e.g., output logits) is available for a closed-source model, the demo-level greedy search could still run with an assistant model seeding the demo pool, suggesting the attack could in principle be adapted to API-only targets, though the paper does not demonstrate this.","The SmoothLLM swap results, where gibberish is classified as unsafe by the judge, imply that the reported robustness numbers may partly reflect judge over-sensitivity rather than actual harmful generation; re-scoring with human annotation would clarify the practical threat."],"forward_implications":["If the reported success rates hold, open-weight safety-aligned models can be jailbroken with fewer than eight short demos, substantially shorter than the 32-shot context I-FSJ required for Llama-3.","The attack works with natural user instructions and a generalized 'Hypothetically' prefix, so it transfers across response structures rather than relying on the stepwise 'Step' format of I-FSJ.","Because demos are sampled from the target model itself, the method is self-sustaining: a single model provides both the attack template and the examples, without needing a weaker auxiliary model.","Perplexity filters that flag high-perplexity adversarial prompts do not catch this attack, since the extended instructions stay within the natural language perplexity range.","Demos synthesized from AdvBench transfer to HarmBench test cases, indicating the attack generalizes beyond its source distribution."],"supporting_citations":[{"why":"Supplies the I-FSJ baseline and the heuristic few-shot jailbreaking framework with special-token injection and demo-level random search that this work extends.","marker":"[52]"},{"why":"Introduces in-context attack / few-shot jailbreaking with harmful demos, the attack family this paper generalizes.","marker":"[41]"},{"why":"Provides the AdvBench harmful-behaviors benchmark used for demo synthesis and evaluation, and the GCG baseline.","marker":"[55]"},{"why":"Provides the HarmBench benchmark used to test generalization of the attack.","marker":"[28]"},{"why":"Llama-Guard-3-8B, the LLM classifier that produces every reported attack success rate.","marker":"[18]"},{"why":"SmoothLLM, the perturbation defense the paper evaluates against.","marker":"[34]"},{"why":"Proposes the perplexity-based attack detector used as a jailbreaking defense that the attack is shown to evade.","marker":"[2]"},{"why":"Defines perplexity, the heuristic that drives demo-level greedy search and demo filtering.","marker":"[20]"}],"fun_headline_variants":["Jailbreak LLMs in 4-8 demos via pattern + behavior decomposition","Self-instruct jailbreak: 90% success on open models in 4-8 demos","Pattern + behavior decomposition shrinks jailbreak to 4-8 demos","90% jailbreak success on open LLMs with just 4-8 demos","Decompose attack into pattern and behavior for 90% jailbreak in 8 demos"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline attack success rates all depend on Llama-Guard-3-8B classifying a response as unsafe, and the paper's own section 4.4 notes that under SmoothLLM swap perturbations gibberish responses are counted as unsafe; if that judge over-counts refusals or nonsense, the 90% figure is unsupported.","fun_headline_variants_meta":{"raw":{"variants":["Jailbreak LLMs in 4-8 demos via pattern + behavior decomposition","Self-instruct jailbreak: 90% success on open models in 4-8 demos","Pattern + behavior decomposition shrinks jailbreak to 4-8 demos","90% jailbreak success on open LLMs with just 4-8 demos","Decompose attack into pattern and behavior for 90% jailbreak in 8 demos"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001607,"raw_usage":{"total_tokens":6431,"prompt_tokens":1005,"completion_tokens":5426,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":621,"completion_tokens_details":{"reasoning_tokens":5312}},"tokens_in":621,"tokens_out":5426,"duration_ms":33934,"temperature":1.0,"reasoning_tokens":5312,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:29:40.935022+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a random sample of 100 responses the judge labeled unsafe from the 8-shot, swap-perturbed condition and have human annotators classify each as actually harmful or not; if the human-verified attack success rate is far below the reported 90%, the classifier over-counts. The same human re-annotation on the unperturbed 8-shot condition would independently test the central efficiency claim.","supporting_citations":[{"cited_title":"Perplexity—a measure of the difficulty of speech recognition tasks","cited_arxiv_id":null,"evidence_quote":"Defines perplexity, the heuristic that drives demo-level greedy search and demo filtering."}],"review_version":1}