{"id":"1ddbdc68-7fb9-479f-9049-627d7640880b","arxiv_id":"2506.00548","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"Con Instruction embeds harmful textual instructions into adversarial images or audio by aligning their representations, achieving successful jailbreaks on several vision- and audio-language models.","lead":"This paper shows that adversarial images or audio clips, optimized to match a harmful instruction in a model's internal embedding space, can make multimodal AI assistants produce dangerous content despite safety training. The result matters because it exposes a gap where safety alignment protects text inputs but not non-textual instruction channels, and it offers a concrete benchmark for defense research.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim rests on an untested sufficiency assumption: low embedding distance to a text instruction guarantees the LLM will decode and obey it.","rationale":"This is the single most load-bearing concern because every headline number is produced by an optimization whose objective is only embedding distance in Section 3.1, Algorithm 1; the language model's decoding of the instruction is never directly optimized or verified. The five-model evaluation, the human-validated ARC framework with 88% agreement, and the defense experiments are real evidence that the attack works on the tested models, and I am not disputing that empirical finding. However, the abstract's strong 'universal' framing requires that the embedding alignment is semantically sufficient, not merely that this particular optimizer sometimes produces effective inputs. The oracle-insertion control cleanly separates the mechanism from optimization artifacts and would settle the concern. If it passes, the paper's strongest claim is supported and the conditional verdict can be relaxed; if it fails, the contribution is a set of strong case studies rather than a universal mechanism. The reader's conditional verdict already reflects a need for more robustness evidence, and I agree with that, so no verdict change is needed beyond keeping the paper conditional until the sufficiency check is reported.","tokens_in":18580,"tokens_out":11633,"duration_ms":121341,"concrete_test":"Run an oracle-insertion control on LLaVA-13B and Qwen-Audio: for each of 100 target instructions, start from the optimized adversarial sample, run the victim encoder and fusion layer, then replace the last N_inst projected non-textual embeddings with exactly T(Inst), the target instruction's token embeddings, leaving the preceding tokens and the text suffix unchanged. Measure ARCr/ARCa with the paper's ARC judge. If the oracle insertion reproduces the reported ASR within about 10 percentage points, the embedding-alignment mechanism is sufficient; if ASR drops substantially, Con Instruction's success is not explained by embedding alignment and the universality claim must be weakened. As a secondary check, recompute ASR while varying tau over +/-0.15 around each reported value to test sensitivity.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption is that minimizing D(F(E(X_adv))[-N:], T(Inst)) in the fusion space is sufficient for the LLM to decode and obey the instruction. Algorithm 1 stops purely on this embedding-space distance (tau), with no check that the optimized tokens are semantically legible in context. The paper's empirical ASR is consistent with the assumption, but it does not establish it. There is no reported distribution of final D, no plot of ASR versus D, and no control in which the exact target token embeddings are inserted at the image/audio-token positions while leaving the rest of the prompt fixed. Without such a control, high ASR on five models could arise from optimization side effects, e.g., global perturbation of all 576 visual tokens, position-specific cues from where the aligned tokens are placed, or accidental interaction with the appended text suffix (Hypo/Anti), rather than from faithful semantic transfer of the instruction in the shared embedding space. The per-model hand-set tau (0.60/0.75/0.85) reinforces the problem: because D combines Euclidean and cosine terms, tau is not scale-invariant, so 'low distance' does not have a consistent semantic meaning across architectures. If the sufficiency assumption fails for other fusion layers, or after small architectural changes, the 'universal' claim collapses even though the tested models remain vulnerable. The concern is not that the results are fabricated; it is that the central mechanism and its generality are under-determined by the reported evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Con Instruction, a gray-box jailbreak attack against multimodal large language models (MLLMs). The method initializes an image or audio snippet from random noise and iteratively optimizes it so that its embeddings after the encoder and fusion module align with the token embeddings of a target malicious text instruction, using a combination of Euclidean distance and cosine similarity. The generated non-textual adversarial example, optionally paired with benign text prompts (Agree, Anti, Hypo), is shown to elicit harmful responses from LLaVA-1.5 (7B/13B), InternVL (13B/34B), Qwen-VL, and Qwen-Audio on AdvBench and SafeBench. The paper also introduces the Attack Response Categorization (ARC) framework with an LLM judge validated against human annotations (88% agreement), and evaluates several defense strategies.","tokens_in":18871,"tokens_out":7116,"duration_ms":65367,"significance":"If the results hold, this is a significant contribution to multimodal safety research. The attack demonstrates that non-textual modalities can carry arbitrary malicious instructions in a way that bypasses text-focused safety alignment, including audio, which previous OCR-based attacks do not cover. The ARC evaluation framework is a practical improvement over refusal-string matching and existing judge-based metrics. The paper releases code, includes human-validated evaluation, and reports extensive ablations across model families and modalities. These strengths make the empirical claims credible and the work likely to influence future defense research.","major_comments":[{"comment":"The central claim that minimizing the distance D(H_Inst, H_adv) in the fusion space makes the MLLM decode and obey the target instruction is not directly validated. The paper reports no distribution of final D values, no analysis of ASR versus D, and no control experiment in which the target instruction's text token embeddings are inserted directly at the non-textual token positions while the rest of the prompt is held fixed. Such a control would test whether the embedding space is semantically shared and whether the optimization is transferring instruction content rather than producing a generic safety-disabling perturbation. Without it, the observed high ASR could be partly attributable to optimization side effects (e.g., global perturbation of all visual tokens or position-specific cues) rather than faithful transfer of the instruction. This matters for the 'universal' claim across architectures. Please add (a) a control with direct embedding insertion, (b) reporting of final D values per model, and (c) an ASR-versus-D analysis.","section":"Section 3.1, Algorithm 1"},{"comment":"The distance function D is underspecified. The paper states that 'combining Euclidean distance and cosine similarity often yields better results' but does not provide the exact formula or the relative weights of the two terms. Table 4 shows qualitative improvement but no numerical definition. Because the stopping criterion in Algorithm 1 is L > tau, and tau is hand-tuned per model (0.60/0.75/0.85), the threshold is not scale-invariant and has no consistent semantic meaning across models. Please specify D exactly (including any normalization and weights), report the achieved final D values, and describe how tau was selected (e.g., the number of optimization steps or examples used for tuning).","section":"Section 3.1 (distance function and threshold)"},{"comment":"The success metric is the max over n=5 sampled responses (Eq. 2). The headline figures in the abstract (81.3% and 86.6% on LLaVA-v1.5 13B) correspond to this max-of-5 metric, not to a single response. While n=5 is disclosed in Section 4.1, the abstract and conclusions do not state this, which overstates the per-query attack reliability. Please report single-response ASR alongside the max-of-5 metric in Table 3 (or at least in an appendix table), and explicitly mention the max-of-5 definition in the abstract.","section":"Eq. (2), Table 3, Abstract"}],"minor_comments":[{"comment":"The text mentions 'ESCO', but the method is called ECSO (Gou et al., 2025); please correct the acronym.","section":"Section 4.2, Finding A1"},{"comment":"The notation for the non-textual encoder is inconsistent: the text says 'I(·)' while Algorithm 1 uses 'E(·)'. Please unify the notation.","section":"Section 3.1"},{"comment":"There are several typos: 'utlized' should be 'utilized', 'Guassian' should be 'Gaussian', and 'detecor' should be 'detector'.","section":"Appendix A"},{"comment":"In the sentence preceding Table 4, 'additon' should be 'addition'.","section":"Table 4"},{"comment":"The display of Equation (2) is malformed ('n max i=1'); it should be formatted as a proper max over i=1..n.","section":"Eq. (2)"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical contribution, but the mechanism claim is stronger than the evidence supports. The missing direct embedding control and the underspecified loss/threshold are fixable with additional experiments and reporting. I suggest encouraging the authors to add the control and to clearly separate the empirical attack effectiveness from the proposed semantic-transfer explanation. The max-of-5 reporting should also be made transparent in the abstract."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth a serious referee. The core idea—optimizing an image or audio snippet to match the embedding of a target text instruction—is a variant of existing embedding-alignment attacks, but the audio-language jailbreak, the ARC evaluation framework, and the systematic comparison across five models are real contributions. The evaluation is careful: ARC is human-validated with 88% agreement against GPT-4, and the code is released. That counts for a lot in this area.\n\nThe main practical concern is that the sufficiency of the embedding-distance criterion is not directly tested. The stress-test note asks for a control where exact target token embeddings are inserted at the image/audio positions; that control is missing, and the per-model threshold tau (0.60–0.85) makes 'low distance' hard to interpret across architectures. I don't think this is fatal. The ARCr numbers show the model consistently responds on-topic to the adversarial input, and the final-token experiment (Table 6) suggests the embedding alignment is conveying the intended meaning rather than causing confusion. Still, the authors should report the distribution of final distances and, if possible, an ASR-vs-distance curve to tighten the mechanism claim.\n\nTwo smaller issues: the weighting of the Euclidean and cosine terms in the joint loss is undisclosed, which is a reproducibility gap, and the max-of-5 success metric inflates ASR. The latter is standard in the jailbreak literature, so I'd note it but not call it a flaw; reporting mean success rates as well would be a quick fix.\n\nOverall, the central empirical claim holds: this attack jailbreaks multiple MLLMs across vision and audio, and the defense analysis is a useful addition. The paper is for the adversarial ML and AI safety community. I'd bring it to a reading group and cite it myself. It deserves a serious referee; with the threshold sensitivity analysis and loss-weight disclosure addressed in revision, it should be accepted.","headline":"A solid, well-evaluated jailbreak paper for MLLMs with a genuinely new audio result and a useful evaluation framework; the main soft spots are methodological details to tighten in revision.","tokens_in":19390,"tokens_out":1931,"would_cite":true,"duration_ms":19129,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a malicious instruction can be hidden entirely inside an image or audio clip by optimizing the non-textual input's embedding to match the instruction's text embedding in the fusion space, and that multimodal LLMs…","keywords":["jailbreak attack","multimodal large language models","adversarial images","adversarial audio","embedding-space alignment","gray-box attack","attack response categorization","safety alignment"],"falsifier":"Measure the embedding distance after optimization: if a large fraction of successful attacks have final distance above the preset threshold $\\tau$, the attack's claimed mechanism (minimizing $D$ below $\\tau$) is not doing the work.","tokens_in":18403,"feed_emoji":"⚠️","tokens_out":7766,"duration_ms":66163,"temperature":0.7,"pith_summary":"Con Instruction is a gray-box jailbreak attack that hides a malicious textual instruction inside an image or an audio clip rather than in the prompt text. The method starts from random noise and iteratively adjusts the non-textual input so that its embedding in the model's fusion space closely matches the token embeddings of the target instruction; when the image or audio is then fed to a multimodal LLM, the model decodes and follows the embedded instruction. On LLaVA-v1.5 (13B) the attack reaches 81.3% and 86.6% attack success on AdvBench and SafeBench, and it transfers across vision- and audio-language models including InternVL, Qwen-VL, and Qwen-Audio. Because no training data or preprocessing of the textual instruction is required, the same scheme works for arbitrary malicious prompts. The paper also introduces an Attack Response Categorization (ARC) framework that grades responses by quality and relevance, arguing that refusal-string matching overestimates attack success.","feed_headline":"Optimized images and audio jailbreak multimodal LLMs up to 87%","feed_subtitle":"Attacks built by matching embeddings to a target instruction slip past text-only safety filters.","key_machinery":"The carrying mechanism is embedding-space alignment in the fusion layer. For a target instruction $\\text{Inst}$, the attack computes $H_{\\text{Inst}} = T(\\text{Inst})$, the text token embeddings; initializes the adversarial sample $X_{\\text{adv}}$ as Gaussian noise; and minimizes $L = D(H_{\\text{Inst}}, H_{\\text{adv}})$ with $H_{\\text{adv}} = F(E(X_{\\text{adv}}))$, where $E$ is the vision or audio encoder, $F$ is the multimodal fusion module, and $D$ combines L2 distance with cosine similarity. Optimization selects the last $N_{\\text{inst}}$ visual or audio embeddings to match the $N_{\\text{inst}}$ token embeddings, an empirical choice reported to give the best attack rate. The attack operates in a gray-box setting: it needs access to the encoder, the token-embedding layer, and the fusion module, but not the full language model.","core_discovery":"The central claim is that multimodal LLMs' joint embedding space can be used as a carrier for instructions: an adversarial image or audio snippet optimized to be close to a target text prompt in that space is sufficient to make the model produce harmful content, even when the text-only version of the instruction is refused. The authors show this with a two-stage attack: Stage I optimizes the non-textual sample under a combined Euclidean-plus-cosine distance between its fused embedding and the target instruction's token embeddings, using the last $N_{\\text{inst}}$ embeddings of the image; Stage II pairs the sample with an empty string or benign helper texts (Agree, Anti, Hypo) that amplify recognition and success rates. They further claim that existing evaluation metrics such as refusal matching and StrongREJECT misclassify responses, and that their four-class ARC with an LLM judge agrees with human annotation at roughly 88%.","pith_inferences":["An implication left implicit is that the same embedding-alignment recipe could be adapted to black-box models by training a surrogate encoder and fusion module, in which case the gray-box requirement would become a practical rather than conceptual barrier.","The paper's t-SNE observation that adversarial activations mix with safe queries suggests a detection hook: a defense could exploit activation geometry rather than response text, though no such defense is constructed here.","The last-$N_{\\text{inst}}$ embedding heuristic implies that token-position alignment matters; a testable extension would randomize which visual embeddings are matched to instruction tokens to see whether position is load-bearing for the attack's success."],"forward_implications":["If Con Instruction is correct, text-only safety filters are insufficient because harmful instructions can arrive entirely through images or audio.","The same adversarial recipe works across vision and audio models, so the safety gap is not specific to one encoder or one modality.","Pairing the non-textual sample with benign helper text such as Hypo amplifies attack success, showing that safety alignment is weaker for non-textual input than for equivalent text.","Larger multimodal models can be more vulnerable than smaller ones to this attack, even when they are more robust to text-only jailbreaks.","Existing defenses vary widely: MLLM-Protector and heavy input noise reduce success substantially, while self-assessment and VLGuard fine-tuning still leave nontrivial attack success rates."],"supporting_citations":[{"why":"Supplies AdvBench, the 520-behavior malicious instruction set used for evaluation, and the refusal-matching convention the paper argues is insufficient.","marker":"(Zou et al., 2023)"},{"why":"Supplies SafeBench's 500 harmful questions and the FigStep typographic baseline that Con Instruction is compared against.","marker":"(Gong et al., 2023)"},{"why":"Defines LLaVA-1.5, the primary victim vision-language model whose encoder, fusion layer, and token embeddings the attack uses.","marker":"(Liu et al., 2024a)"},{"why":"Provides Qwen-VL-Chat, a vision-language victim model tested in the paper.","marker":"(Bai et al., 2023)"},{"why":"Provides Qwen-Audio-Chat, the audio-language model that demonstrates the attack's transfer to the audio modality.","marker":"(Chu et al., 2023)"},{"why":"Provides InternVL-Chat, a second vision-language victim family tested in the experiments.","marker":"(Chen et al., 2024)"},{"why":"StrongREJECT is the LLM-judge evaluation baseline that the paper shows overestimates attack success and motivates the ARC framework.","marker":"(Souly et al., 2024)"},{"why":"Supplies VLGuard, the adversarial fine-tuning defense that the paper tests and finds insufficient against Con Instruction.","marker":"(Zong et al., 2024)"}],"fun_headline_variants":["Embedding-matched images and audio jailbreak MLLMs up to 87%","Non-textual adversarial samples bypass MLLM safety filters","Images and audio alone trigger harmful outputs in multimodal LLMs","87% success: embedding attack on LLaVA and other MLLMs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The attack collapses if closeness in the fusion embedding space does not reliably make the language model decode the target instruction; if that fails, minimizing the distance would produce noise that is near the text instruction in embedding distance but semantically unreadable by the model.","fun_headline_variants_meta":{"raw":{"variants":["Embedding-matched images and audio jailbreak MLLMs up to 87%","Non-textual adversarial samples bypass MLLM safety filters","Images and audio alone trigger harmful outputs in multimodal LLMs","87% success: embedding attack on LLaVA and other MLLMs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000558,"raw_usage":{"total_tokens":2669,"prompt_tokens":979,"completion_tokens":1690,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":595,"completion_tokens_details":{"reasoning_tokens":1614}},"tokens_in":595,"tokens_out":1690,"duration_ms":11990,"temperature":1.0,"reasoning_tokens":1614,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:02:49.474113+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the embedding distance after optimization: if a large fraction of successful attacks have final distance above the preset threshold $\\tau$, the attack's claimed mechanism (minimizing $D$ below $\\tau$) is not doing the work.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies VLGuard, the adversarial fine-tuning defense that the paper tests and finds insufficient against Con Instruction."}],"review_version":1}