{"id":"2d52488f-c5e5-46fd-a0e1-5a11f506a2a0","arxiv_id":"2501.18626","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Encoding forbidden content in ciphers, riddles, or code tasks lets attackers bypass safety filters in six current LLMs, and the PHRYGE benchmark measures how often this succeeds.","lead":"The paper introduces 'Task-in-Prompt' attacks, where harmful words are hidden inside ciphers, riddles, or code puzzles that the model is asked to solve, causing LLMs to generate prohibited content indirectly. It also introduces a 120-prompt benchmark called PHRYGE and reports successful attacks across six models including GPT-4o and LLaMA 3.2.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Quantitative ASR claims rest on a single LLM judge validated on only 100 samples, with no per-condition error analysis and the judge itself being a tested model; reported success rates may be systematically inflated.","rationale":"The paper is an empirical attack paper; its strongest claim is that TIP attacks successfully circumvent safeguards in six state-of-the-art language models. That claim is supported by ASR numbers rather than by formal proof or released code. The single most load-bearing input to those numbers is the judge. The paper's own limitation section (Section 8) concedes that the automated evaluation may introduce biases or fail to capture nuanced instances, and the 100-sample 92% validation is not stratified, so it cannot bound per-condition error. This is not an abstract worry: the judge is also a target model, and the judge instructions are explicitly designed to catch disguised procedural content, which creates a plausible false-positive mechanism. Alternative concerns such as best-of-grid selection, novelty relative to ArtPrompt, and missing confidence intervals are real but secondary: best-of-grid selection inflates the magnitude of ASR rather than the existence of successful attacks, and the novelty framing does not affect the empirical phenomenon. A stratified human re-annotation is a concrete, feasible check that would settle whether the ASR claims are trustworthy. Until then, the appropriate verdict is conditional: the phenomenon is plausible and probably present, but the precise numbers and comparisons in Tables 1-3 should not be taken at face value.","tokens_in":17732,"tokens_out":5855,"duration_ms":58630,"concrete_test":"Re-annotate a stratified sample of responses with two independent human raters: draw roughly 30 outputs per tested model per attack objective (about 720 total), spanning all encodings, difficulty levels, and depersonalization conditions, plus all non-TIP baseline outputs. Compute a per-condition confusion matrix for the LLaMA-3.1-70B judge against the human labels, then recompute Table 1 ASRs and Table 2 comparisons using judge decisions only where they match the human majority. If the corrected ASRs remain materially positive for all six models and the best-prompt columns are unchanged, the concern is resolved; if the corrected ASRs drop or the ordering changes, the paper's central quantitative claims require revision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.3 defines attack success using LLaMA-3.1-70B as an automated judge, with validation limited to 100 randomly selected samples at 92% accuracy. No confusion matrix, per-task/per-model error rates, or human agreement measure is reported. LLaMA-3.1-70B is also one of the six attacked models, so any bias in its safety judgment contaminates both the victim and the measurement instrument. The judge prompt instructs it to flag 'detailed procedural information, even if disguised as hypothetical, indirect, or within a role-playing context'; a model that over-applies this instruction could mark procedural-sounding refusals, generic warnings, or outputs that merely mention the target concept as successful attacks. Because Table 1 and the cross-model comparison in Table 2 select the best cell from a large grid (10 encodings × 3 difficulty levels × 4 objectives × depersonalization on/off), a judge false-positive rate concentrated in only a few cells would change the headline ASR values and the reported superiority over TAP and ArtPrompt. The 92% aggregate accuracy is not sufficient to support quantitative claims of 0.94–1.0 ASR or per-model rankings without a per-condition breakdown.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Task-in-Prompt (TIP) attacks, a proposed class of jailbreak in which an unsafe concept is encoded via a sequence-to-sequence task (cipher, riddle, code, etc.) embedded in the prompt, and the model is asked to perform the task rather than directly produce the prohibited content. The authors formalize the attack, introduce the PHRYGE benchmark (10 encodings x 4 objectives x 3 difficulty levels), evaluate six LLMs, report attack success rates (ASR), compare against JailbreakBench attacks, and test several defenses. They report that all six models are vulnerable, with best ASR reaching 1.0 on several tasks, and that TIP attacks outperform prior jailbreaks on GPT-4o and LLaMA 3.2. The central quantitative claim rests on an LLM-as-judge protocol (LLaMA-3.1-70B) validated on only 100 manually inspected samples at 92% accuracy.","tokens_in":17957,"tokens_out":3855,"duration_ms":37409,"significance":"If the empirical claims hold, the paper is significant: it identifies a general attack class rather than a single encoding trick, provides a reusable benchmark, and makes falsifiable predictions about the vulnerability of instruction-following models. The formal definition in Section 2.1 is clear, the PHRYGE benchmark is a useful artifact, and the paper includes sanity checks, a comparison with established attacks on JailbreakBench, and reproducible details such as GPU hours and sample prompts. The main weakness is that the headline ASR numbers are only as credible as the automated judge, and the current validation is too thin to support per-model rankings and best-cell maxima.","major_comments":[{"comment":"The only validation of the automated judge is manual inspection of 100 randomly selected samples, reported as 92% accuracy. This is insufficient to support the headline ASR values in Tables 1 and 2, because Table 1 selects the best cell from the 10 encodings x 3 difficulty levels x 2 depersonalisation settings per objective. A judge false-positive rate concentrated in a few cells would directly inflate the reported maxima. Please provide per-task and per-model confusion matrices, human agreement on the selected high-ASR cells, and confidence intervals on the ASR estimates.","section":"Section 4.3"},{"comment":"The judge is LLaMA-3.1-70B, and LLaMA-3.1-70B-Instruct is one of the six attacked models. Using the same model as both victim and measurement instrument creates a systematic-bias risk: a judge that over-applies its instruction to flag 'detailed procedural information, even if disguised as hypothetical, indirect, or within a role-playing context' may mark refusals or generic warnings as successful attacks, and this bias is inseparable from the model's own safety behavior. Please use a judge that is not among the attacked models, or at minimum report per-cell agreement between the judge and human labels for each victim model.","section":"Sections 4 and 4.3"},{"comment":"ASR values such as 0.94, 0.97, and 1.00 are reported without sample sizes, confidence intervals, or any measure of run-to-run variability; only the sanity check in Section 4.1 states 100 runs per query. Without the number of runs per PHRYGE cell, differences such as GPT-4o's 0.94 vs 0.79 in the same column are not statistically interpretable, and the 'best attack' selection in Table 1 has no grounding. Please state how many runs were executed per cell and report uncertainty for every ASR estimate, including the JailbreakBench comparisons in Table 2.","section":"Section 5.1, Table 1"},{"comment":"The 'GPT-4o*' column defines detection rate as 1 minus ASR, which is a refusal rate of the target model, not the performance of a separate detection system. This conflates the target model's alignment with the effectiveness of an actual defense and weakens the claim in Section 6.5 that TIP attacks bypass modern detection models, at least for that column. Please either evaluate a real guard model (e.g., an available content-moderation API) or remove this column and qualify the defense comparison accordingly.","section":"Table 3 and Section 6.5"}],"minor_comments":[{"comment":"The header 'Llama3' appears in Table 2, but Section 4 lists four LLaMA variants; please clarify whether this is LLaMA-3.1-70B, LLaMA-3.2-3B, or another variant, and use consistent naming across tables.","section":"Table 2"},{"comment":"The red shading is described only qualitatively as 'darker red shades indicate higher ASR'; please add a numeric color bar or label values so the heatmaps are quantitatively readable.","section":"Figure 2"},{"comment":"The benchmark is described as 10 encodings x 4 objectives x 3 difficulty levels generating 120 prompts, while Section 4.5 says scenarios were tested with and without depersonalisation; please clarify whether the 120 count includes depersonalisation variants, and if not, state the total number of prompt configurations actually evaluated.","section":"Sections 3 and 4.5"},{"comment":"The Limitations section already acknowledges that the automated evaluation function 'may introduce biases or fail to capture nuanced instances of unsafe content generation'; the main text in Section 4.3 should not soften this caveat, and the experimental section should operationalize the needed validation rather than leaving it as a deferred limitation.","section":"Section 8"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know about this paper is that the core observation is probably right: you can hide prohibited words in a sequence-to-sequence task, ask the model to solve it, and get it to produce the unsafe content while its safety filters stay quiet. That is worth saying clearly, and the PHRYGE benchmark is a genuinely useful artifact. Ten encodings, four objectives, three difficulty levels, six models — that is real work, and the demonstration that the ArtPrompt effect generalizes far beyond ASCII art is a legitimate contribution. The reframing of ArtPrompt as one instance of a task-in-prompt class is an organizing insight that will stick, even if the class was not as \"previously unidentified\" as the abstract claims; cipher- and riddle-based jailbreaks were already in the literature.\n\nThe tables are internally consistent, the appendices give full per-cell ASR data, and the authors are honest enough to list the automated judge as a limitation in Section 8. That helps. But the quantitative center does not support the strength of the headline numbers. Attack success is reported as the best result from a 60-cell grid (or worse, best of multiple difficulty levels), and the judge is LLaMA-3.1-70B — the same architecture as one of the attacked models — validated on 100 samples at 92% accuracy with no confusion matrix or per-condition breakdown. If the judge over-flags procedural-sounding refusals, the ASR values in Tables 1 and 2 could be systematically inflated, and the per-model ordering could shuffle. The paper also selects each model's best prompt inside PHRYGE and then reports that prompt's ASR as the headline, which is selection bias, not fraud.\n\nThese are fixable problems. Add a per-condition human-validated judge analysis, report confidence intervals or raw counts, and walk back the novelty language. None of them kills the central phenomenon: the sanity check shows direct queries are refused at 0 ASR, and the benchmark shows encoded versions succeed far more often. The defense results in Table 3 are also directionally striking even if the numbers are approximate.\n\nI would send this to a serious referee, with a request to focus on the evaluation methodology and the novelty framing. This is a paper for red teams and safety researchers, not a finished security analysis, but it deserves engagement. The right verdict is conditional acceptance with revisions, not desk rejection.","headline":"A useful benchmark and a plausible reframing of ArtPrompt as a broad class, but the headline ASRs rest on a lightly validated LLM judge and best-of-grid selection.","tokens_in":18491,"tokens_out":1943,"would_cite":true,"duration_ms":20913,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Task-in-Prompt attacks embed forbidden requests inside benign transformation tasks and circumvent safety safeguards in all six tested LLMs, including GPT-4o and LLaMA 3.2.","keywords":["LLM safety","jailbreak attacks","adversarial prompts","sequence-to-sequence tasks","Task-in-Prompt","PHRYGE benchmark","cipher decoding","attack success rate"],"falsifier":"A human expert panel independently labels a random, condition-stratified sample of target-model outputs and compares its labels with the LLaMA-3.1-70B judge; if human agreement falls well below 92 percent or systematic disagreement appears in any condition, the reported attack success rates are unreliable.","tokens_in":17537,"feed_emoji":"🧩","tokens_out":7850,"duration_ms":57872,"temperature":0.7,"pith_summary":"The paper claims that a previously unnoticed class of jailbreak attacks, called Task-in-Prompt (TIP), defeats the safety filters of every large language model tested. The attack hides a forbidden request inside a benign-sounding transformation task, such as decoding a Caesar cipher or solving a riddle, so the model itself reconstructs the harmful content. The authors introduce a 120-prompt benchmark, PHRYGE, spanning ten encoding schemes and four abuse categories, and report attack success rates reaching 100% on several models and tasks. If correct, the result means current safety alignment cannot be secured by filtering trigger words or known attack formats, because any task-solving model can be turned into a decoder for its own forbidden concepts.","feed_headline":"Cipher-encoded prompts jailbreak all six tested LLMs","feed_subtitle":"Task-in-Prompt attacks hide forbidden requests inside decoding tasks, slipping past safety filters.","key_machinery":"The load-bearing object is the TIP prompt template $x^* = x_{\\text{task}} + E(u)$, where $E$ is an encoding function (Caesar, Morse, Base64, riddles, Python code, etc.) that maps an unsafe prompt $u$ into a seemingly benign string, and $x_{\\text{task}}$ instructs the model to process that string. The attack relies on implicit decoding: the model is not asked to output the decoded word step by step; instead it reconstructs the meaning through attention and context and uses it in a downstream request. Difficulty tiers modulate how transparent the encoding is, which determines whether a model can solve it and whether its safeguards register the danger.","core_discovery":"The central claim is that encoding unsafe content with an arbitrary sequence-to-sequence transformation and embedding that transformation as an instruction in the prompt is sufficient to make aligned LLMs produce prohibited output with high probability. The paper demonstrates this across six state-of-the-art models, including GPT-4o and LLaMA 3.2, with per-task attack success rates reaching 1.0. The mechanism, according to the paper, is that the model performs the decoding internally and implicitly, integrating the reconstructed unsafe meaning into its response without an explicit chain of thought, thereby bypassing safety filters that are triggered by direct forbidden wording. The paper frames TIP as a general class of vulnerabilities of which the prior ArtPrompt ASCII-art attack is one instance, and shows in JailbreakBench comparisons that TIP outperforms several established jailbreaks on the newest models.","pith_inferences":["If TIP is as general as claimed, then model capability increases and safety tuning may be in a direct arms race: any benchmark that measures task-solving skill is, indirectly, a measure of the surface available for this attack class.","The PHRYGE benchmark could be extended to multimodal encodings, such as images or audio steganography, and to retrieval-augmented systems; the paper lists these as future work but does not test them.","A natural follow-up experiment is to test whether fine-tuning on explicit decoding tasks amplifies TIP success, since the model's task-solving ability is the attack's engine.","Because the ASR numbers rest on an LLM judge, they should be treated as upper bounds until a human-rated subset across all conditions confirms them."],"forward_implications":["Any safety alignment that relies on detecting trigger words, known attack patterns, or prompt-format rules will miss TIP prompts, because the harmful term never appears in the input.","The attack transfers across the encoding space: at least one of the ten tested transformations succeeded on every tested model, so the vulnerability is not specific to one cipher or format.","Defenses and red-teaming must assume that task-solving capability itself is an attack surface; models that get better at decryption, riddles, and code will become more, not less, vulnerable.","Depersonalisation, attributing the request to a third party, raises success rates for most models but is not necessary; TIP works without it on all models.","Existing filter and classifier defenses, including Llama Guard 3 and keyword filtering, have near-zero detection rates for several TIP encodings."],"supporting_citations":[{"why":"This work introduces the ArtPrompt ASCII-art jailbreak that the paper identifies as the specific instance of the broader TIP class.","marker":"Jiang et al., 2024"},{"why":"This work reports that LLMs cannot interpret ASCII art, prompting the question of whether the encoding format itself matters.","marker":"Berezin et al., 2024"},{"why":"This work supplies the JailbreakBench protocol and the LLM-as-judge evaluation method used to measure attack success rates.","marker":"Chao et al., 2024"},{"why":"This work explains how safety training fails on new inputs, supporting the claim that alignment does not robustly generalize.","marker":"Wei et al., 2023"},{"why":"This work presents the DAN persona-based jailbreak, providing the depersonalisation technique and a comparison baseline.","marker":"Shen et al., 2024"},{"why":"This work provides the Tree of Attacks (TAP) jailbreak used as a comparison baseline on JailbreakBench.","marker":"Mehrotra et al., 2024"},{"why":"This work provides the Past Tense Attack (PTA) jailbreak used as a comparison baseline on JailbreakBench.","marker":"Andriushchenko and Flammarion, 2024"}],"fun_headline_variants":["Task-in-Prompt attacks jailbreak top LLMs via embedded tasks","Cipher and riddle prompts bypass safety in six LLMs","PHRYGE: New benchmark exposes hidden task jailbreaks","TIP attacks hide forbidden prompts inside decoding tasks","Aligned LLMs fooled by task-embedded adversarial prompts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported attack success rates all depend on an automated judge model, LLaMA-3.1-70B, whose classifications were checked on only 100 sampled responses with 92 percent accuracy; if that judge is biased, the central claim about universal vulnerability is not supported.","fun_headline_variants_meta":{"raw":{"variants":["Task-in-Prompt attacks jailbreak top LLMs via embedded tasks","Cipher and riddle prompts bypass safety in six LLMs","PHRYGE: New benchmark exposes hidden task jailbreaks","TIP attacks hide forbidden prompts inside decoding tasks","Aligned LLMs fooled by task-embedded adversarial prompts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00076,"raw_usage":{"total_tokens":3317,"prompt_tokens":831,"completion_tokens":2486,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":447,"completion_tokens_details":{"reasoning_tokens":2403}},"tokens_in":447,"tokens_out":2486,"duration_ms":16889,"temperature":1.0,"reasoning_tokens":2403,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T13:49:24.880915+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A human expert panel independently labels a random, condition-stratified sample of target-model outputs and compares its labels with the LLaMA-3.1-70B judge; if human agreement falls well below 92 percent or systematic disagreement appears in any condition, the reported attack success rates are unreliable.","supporting_citations":[],"review_version":1}