{"id":"ad4b25f2-6fe9-493a-b9c0-3fa07dbfd594","arxiv_id":"2412.15614","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Suffix injection plus PGD image perturbations raises attack success on LLaVA 1.5, with the text suffix contributing most of the gain.","lead":"This report describes a top-ranked attack on LLaVA 1.5 in the ICML 2024 TiFA challenge, combining text suffix injection with projected gradient descent image perturbations. A generalist reader should care because it shows simple, known attacks still fool multimodal AI systems, and that text shortcuts can dominate visual evidence.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The H1/H2 'attack targets' are chosen from GPT-4o/manual pseudo-labels; if any such label marks the correct option as wrong, the reported +32.83/+7.29 gains count correct answers as successful attacks, so the 'fooling' claim is unproven without label verification.","rationale":"The reader's weakest assumption identifies pseudo-label correctness, and I agree that this is the single most load-bearing concern. The paper's headline result is an attack success rate that depends on what counts as an 'incorrect' option. Since ground-truth labels are withheld, the authors construct their own targets with GPT-4o and manual verification. If those labels are wrong, the attack is not necessarily fooling the model; it may be reinforcing a correct answer. The paper explicitly acknowledges this risk in the Limitations section, which makes the concern in-scope and self-flagged. The alternative concern about the 'w/o attack' baseline being copied from Blackfyre is real but less decisive: both the with-attack and without-attack numbers in Table 1 appear to come from the official challenge evaluation, and the authors provide evidence (similarity 1.0) that the baseline inputs were clean. The H3 local-vs-official discrepancy is a separate evaluation-protocol issue, but it does not directly invalidate the H1/H2 official deltas. In contrast, a pseudo-label error would directly contaminate the core metric: an attack that causes the model to output the true correct answer should not be counted as a successful attack. Because the official success metric is not specified in the report, we cannot rule out that the system itself treats the participant-specified target as the definition of 'undesirable', which would make the evaluation circular. The proposed test—checking pseudo-label accuracy against hidden ground truth and inspecting the official metric—would settle whether the reported gains actually measure model fooling. Until that check is run, the conditional verdict is appropriate; no change to the reader's verdict is needed.","tokens_in":4923,"tokens_out":8364,"duration_ms":76387,"concrete_test":"Obtain the hidden ground-truth labels for the TiFA H1/H2 instances from the challenge organizers (or reconstruct them via the official evaluation script) and compute the agreement between the paper's GPT-4o + manual pseudo-labels and the true labels. Report label accuracy and recompute Table 1's attack success rate restricted to instances where the selected target is verified incorrect. Also inspect the official evaluation metric: if success is defined as outputting the attacker-chosen option regardless of correctness, the H1/H2 results are not evidence of fooling. If the label accuracy is below ~100% or the metric is circular, the +32.83/+7.29 deltas should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim (Table 1) is that appending a pseudo-labeled 'incorrect' option plus PGD noise raises official attack success rate by +32.83 (Helpful) and +7.29 (Honest). The target of each H1/H2 attack is an option that GPT-4o and the authors judged incorrect (Sec. 2.3), because ground-truth labels were not provided. The paper itself lists this as a limitation (Sec. 3): both GPT-4o and human annotators may introduce labeling biases. If a pseudo-label is wrong, then the 'incorrect' target is actually the correct answer, so the attack pressure pushes LLaVA toward the right response. The reported success rate would then count correct answers as 'successful attacks', inflating the measured gains by an unknown amount. This concern is load-bearing because the entire claim that the model is fooled rests on the correctness of the pseudo-labels; the PGD/suffix mechanism could still work, but the evidence for it would be contaminated. The report does not define the official success metric for H1/H2, so it is not known whether the system counts output of the attacker-specified option as success regardless of ground truth, which would make the metric circular.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes the authors' top-ranked solution for the ICML 2024 TiFA workshop MLLM attack challenge, targeting the LLaVA-1.5 (13B) model. For the Helpful (H1) and Honest (H2) multiple-choice dimensions, the authors use GPT-4o with manual verification to pseudo-label the correct option, randomly select one incorrect option, append that option's text to the query as a suffix, and additionally apply a vanilla PGD attack to the image with an adaptive perturbation radius. For the Harmless (H3) dimension, they enumerate harmful texts from an external corpus and append them to the query. The paper reports attack success rates in Table 1, with gains of +32.83 on Helpful, +7.29 on Honest, and +1.05 on Harmless, and concludes that suffix injection and PGD can easily fool an MLLM. Section 3 acknowledges limitations concerning pseudo-label bias, prompt misalignment, and the labor-intensive labeling process.","tokens_in":5234,"tokens_out":3785,"duration_ms":36097,"significance":"If the headline numbers are reliable, the paper provides a clear demonstration that simple text-side manipulation (literally appending a chosen answer) combined with a standard white-box PGD image perturbation can substantially change LLaVA-1.5's multiple-choice outputs. The ablations in Figure 1, showing that longer suffixes increase attack success, are a useful empirical contribution. The paper is also candid in stating its limitations, which is commendable. However, the report does not ship code, seeds, per-instance results, or error bars, and the official success metric for H1/H2 is not precisely defined. The central quantitative claims therefore remain plausible but not fully established in this document.","major_comments":[{"comment":"The attack target for every H1/H2 instance is an option that GPT-4o and the authors judged to be incorrect, but the paper does not provide verification statistics (e.g., per-instance agreement, number of corrected labels) and Section 3 explicitly acknowledges that both GPT-4o and humans may introduce labeling biases. Because the reported success rate depends entirely on whether the chosen target is truly wrong, the authors must either report a verified-label subset with the attack success recomputed on it, or provide ground-truth-verified labels for all instances. They should also state clearly whether the official H1/H2 metric counts output of the attacker-specified option as success regardless of ground truth, or compares against the hidden ground-truth label; the former would make the metric circular, while the latter would mean label errors deflate rather than inflate the official numbers.","section":"Sec. 2.3 and Sec. 3"},{"comment":"The 'without attack' row is copied from the Blackfyre team's results, and the paper itself notes that these numbers 'might not be entirely accurate.' Since every reported improvement is a difference against this baseline, an inaccurate baseline directly invalidates the headline gains. The authors should obtain the official no-attack scores for their own test split or rerun a no-attack baseline under the same evaluation protocol, and report per-seed or per-instance variability.","section":"Table 1"},{"comment":"The stated objective in Eq. (2), min dist(t, t') subject to similarity constraints, does not match Algorithm 1, which performs PGD with a cross-entropy loss to maximize the likelihood of the attacker-chosen incorrect text t'. These are different objectives: one minimizes semantic distance between clean and adversarial outputs, the other maximizes the probability of a specific target. This inconsistency prevents exact reproduction and should be resolved by rewriting Eq. (2) to describe the actual targeted adversarial objective used in the algorithm.","section":"Eq. (2) and Algorithm 1"},{"comment":"For the Harmless dimension, the authors report that their own evaluation achieved an 80% attack success rate while the official system evaluation gave 39.27%, a discrepancy they attribute to prompt misalignment. This means the H3 result in Table 1 (+1.05) is not obtained under the authors' own controlled protocol, and the paper does not provide enough information (e.g., the set of system prompts tried, or an analysis of which lexical or stylistic differences caused the gap) to determine whether the reported improvement is meaningful. Without closing this local-versus-official gap, the H3 conclusion is not interpretable.","section":"Sec. 2.4 and Table 1"}],"minor_comments":[{"comment":"The caption says the table compares adaptive epsilon and fixed epsilon, but the rows labeled ✗ and ✓ are not explained; the reader cannot tell which row corresponds to adaptive epsilon and which to fixed epsilon, or what the numbers in parentheses (4/255, 16/255) mean. Please clarify the table structure and define all symbols.","section":"Table 2"},{"comment":"The y-axes in Figure 1 are annotated with '-22.14' and '-4.64', but the later text describes these as improvements of 22.14% and 4.64%. The sign convention should be made explicit or the annotations changed to match the text.","section":"Fig. 1"},{"comment":"The algorithm description contains several underspecified details: the initialization of t' inside the repeat loop, the conditions under which the Adaptive epsilon and Text Perturbation flags are set, and the exact loss used for the PGD update (e.g., cross-entropy with the target text as the label). Adding these details would improve reproducibility.","section":"Algorithm 1"},{"comment":"The text says the selected incorrect label is appended using the 'longest sub-sentence rule,' while Algorithm 1 appends the longest sub-sentence while the BERT similarity exceeds beta_q. It should be stated whether the suffix is the full incorrect option or a truncated sub-sentence, and how the truncation length is chosen.","section":"Sec. 1 and Alg. 1"},{"comment":"There are minor typographical issues, such as inconsistent spacing in 'LLaV A' in the abstract and Section 1, and the reference for the Visual-Adversarial-Examples harmful corpus is given only as a URL rather than a formal citation. Please clean these up.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"This is a short workshop-style technical report, and the review should be calibrated accordingly. The main risk is that the headline numbers in Table 1 are not independently reproducible from the manuscript: the baseline is borrowed from another team, the H3 local/official gap is unexplained, and the pseudo-label verification procedure is not quantified. These issues are fixable within the scope of a revised report if the authors release code, define the official metric precisely, and provide verification statistics. I do not see a need for rejection, but the central claim is not yet established as stated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe takeaway: this is a competition write-up that shows a simple suffix injection—appending the text of an incorrect multiple-choice option to the query—is the main reason LLaVA-1.5 fails on the TiFA benchmark, with PGD image noise adding little. That is a real, if unsurprising, illustration of language bias in an MLLM, and the paper is honest about its limits.\n\nWhat it does well: it runs ablations on suffix length and on adaptive vs fixed epsilon, and it clearly separates targeted (H1/H2) from untargeted (H3) attacks. The authors explicitly flag the pseudo-label risk and the prompt misalignment problem for H3. The writing is direct and the algorithm is easy to follow.\n\nThe soft spots are real but not fatal. The biggest one is the label correctness issue: attack targets are chosen from GPT-4o pseudo-labels that the authors manually verified. If any of those labels is wrong, the +32.83 Helpful gain could include cases where the model answered correctly and the attack was counted as success. Manual verification reduces the risk, but with only ~300 instances and random selection among incorrect options, even a few bad labels could move the headline number. The paper itself lists this as a limitation.\n\nRelatedly, the success metric for H1/H2 is undefined in the report. If 'attack success' means 'model outputs the attacker-chosen option,' then injecting that exact text as a suffix makes the metric partly circular—the model can succeed by copy-paste. That is still a valid attack vector and demonstrates language bias, but it should be framed that way rather than as gradient-based fooling.\n\nThe other issues are minor for a workshop report: no code, no error bars, and the 'w/o attack' baseline is copied from another team. The local-vs-official H3 gap (80% vs 39%) is large and unexplained beyond 'prompt misalignment,' which is unsatisfying but not unreasonable.\n\nWho this is for: people working on adversarial robustness of multimodal LLMs, or participants in similar challenges, will want to read this. It is not a breakthrough, but it provides benchmark-specific evidence for a known phenomenon. I would accept it for peer review if the authors provide code and precisely define the evaluation metric; as it stands, the quantitative claims are not reproducible.\n\nMy recommendation: engage with it, but ask for the missing artifacts. The core qualitative finding—text attacks dominate image attacks on this benchmark—likely holds.","headline":"A competition write-up that credibly shows simple suffix injection is the main driver of attack success on LLaVA-1.5 for the TiFA benchmark, but the reported numbers need code and a clearer metric before they can be trusted.","tokens_in":5769,"tokens_out":2956,"would_cite":false,"duration_ms":25610,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper shows that appending an incorrect answer option to the text and adding small PGD image noise is enough to make LLaVA-1.5 output the wrong answer, with attack success rising from 48.09 to 80.92 on the TiFA Helpful track.","keywords":["adversarial attacks","multimodal large language models","LLaVA-1.5","suffix injection","projected gradient descent","TiFA benchmark","prompt misalignment"],"falsifier":"Re-run the attack on a subset of instances where the correct option is independently validated by multiple human annotators or by a trusted source, and drop any instance where the pseudo-label disagrees; if the attack success gain on the fully verified subset is much smaller than the reported 80.92 percent, or if LLaVA-1.5 frequently picks the pseudo-labeled wrong options even on clean images, the claimed attack strength is partly an artifact of label noise.","tokens_in":4766,"feed_emoji":"🎯","tokens_out":6748,"duration_ms":53099,"temperature":0.7,"pith_summary":"This technical report describes the top-ranked solution to the 2024 TiFA MLLM attack challenge: take a multiple-choice question, append the text of an option that is known to be wrong (pseudo-labeled by GPT-4o and manually verified), and then add small gradient-based image noise via projected gradient descent (PGD). Applied to the fixed LLaVA-1.5 (13B) model, the combination raises attack success from 48.09 to 80.92 on the Helpful dimension and from 62.25 to 69.54 on Honesty, while Harmlessness improves only slightly, from 38.22 to 39.27. The authors' central claim is that these two simple techniques are sufficient to fool a large multimodal model, and that the text suffix is the more powerful component. The report also argues the result exposes a language bias in multimodal models, which tend to follow the text shortcut and neglect visual evidence.","feed_headline":"A wrong-answer suffix plus PGD noise fools LLaVA-1.5","feed_subtitle":"Tiny text and image edits lift attack success on Helpful questions from 48 to 81 percent.","key_machinery":"The machinery is a two-stage adversarial pipeline. Stage one is suffix injection: choose an incorrect option from GPT-4o pseudo-labels, then append its longest sub-sentence to the query so that the modified text stays within a BERT cosine-similarity threshold of 0.9. Stage two is vanilla PGD on the image: starting from an adaptively chosen perturbation radius from {32, 16, 8, 4, 2, 1}/255, the attack maximizes cross-entropy loss on the LLaVA-1.5 output over up to 1,000 steps while checking a ResNet cosine-similarity threshold of 0.9. The adaptive epsilon search matters because LLaVA-1.5 and ResNet preprocess images differently, so a fixed radius often violates the visual constraint. The suffix does the heavy lifting; PGD supplies an additional, smaller push.","core_discovery":"On the paper's own terms, the discovery is that a targeted attack on a multimodal large language model does not need elaborate alignment-breaking mechanisms: appending the text of an incorrect option to the query, under a BERT cosine-similarity constraint of 0.9, and then running vanilla PGD on the image, under a ResNet cosine-similarity constraint of 0.9, reliably makes LLaVA-1.5 output the attacker-chosen wrong answer. The authors report total attack-success gains of +32.83 on Helpful, +7.29 on Honest, and +1.05 on Harmless relative to no attack. They additionally observe that the suffix alone contributes most of the gain (a 22.14 percentage-point improvement over image attack alone on Helpful questions) and that longer suffixes keep helping. For Harmless questions, free-form safety output resists this recipe; the authors instead append external harmful content and attribute the remaining gap to prompt misalignment between their evaluation and the challenge system's evaluation.","pith_inferences":["A testable extension: if the authors' language-bias explanation is right, the same suffix recipe should transfer to other instruction-tuned MLLMs that share LLaVA-style training; running the suffix without PGD against a different open-source model would check whether the wrong-option preference persists.","A useful ablation the report does not run is a fully blank or black image with the same suffix: if the model still picks the wrong option, that would isolate the text shortcut from the PGD contribution.","The reported discrepancy between local and system evaluation for Harmless suggests future challenge designs should standardize prompts before declaring a defense robust; this is an implication about benchmarking, not about model safety."],"forward_implications":["If the claim holds, then defending a multimodal model requires treating the text channel as at least as important as the image channel; filtering or constraining text suffixes could block most of this attack.","A fixed model with frozen weights can be steered toward a specified wrong answer using only query text plus image noise, which matters for real-world deployment of MLLMs in multiple-choice settings.","The small Harmless gain implies safety-aligned free-form output is more resistant to this recipe, but the authors' reported 80 percent local success versus 39.27 percent system success warns that evaluation-protocol differences, not model robustness, can dominate measured safety.","Constraining attacks to be perceptually similar (cosine similarity above 0.9 in both modalities) does not prevent the attack; stealth and success coexist."],"supporting_citations":[{"why":"Supplies the suffix-injection method: appending adversarial text to an aligned model's prompt.","marker":"(Zou et al., 2023)"},{"why":"Supplies the PGD attack used to perturb images in the second stage.","marker":"(Madry et al., 2018)"},{"why":"Defines the victim model LLaVA-1.5 whose weights remain fixed during the attack.","marker":"(Liu et al., 2024b)"},{"why":"Provides the evidence the authors cite for language bias and visual neglect in multimodal models.","marker":"(Liu et al., 2024a)"},{"why":"Provides BERT, used to measure text similarity and constrain the suffix injection.","marker":"(Devlin et al., 2019)"},{"why":"Provides ResNet50, used to measure image similarity and constrain the PGD noise.","marker":"(He et al., 2016)"},{"why":"Provides the MD-Judge safety evaluator used for the Harmless dimension and for selecting unsafe options.","marker":"(Li et al., 2024)"}],"fun_headline_variants":["Wrong-answer suffix and PGD noise fool LLaVA-1.5","Suffix injection + PGD boosts LLaVA attack success","Pithy attack: wrong suffix and tiny image noise fool LLaVA","LLaVA-1.5 falls for wrong-answer suffix plus PGD","Simple combo: suffix and PGD trick LLaVA-1.5"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole attack depends on the pseudo-labels being correct: the authors ask GPT-4o for the right answer and then manually verify it, and a randomly chosen different option is treated as truly wrong; if a pseudo-label is wrong, the attack target is actually the correct answer and the reported success numbers would no longer mean the model was fooled.","fun_headline_variants_meta":{"raw":{"variants":["Wrong-answer suffix and PGD noise fool LLaVA-1.5","Suffix injection + PGD boosts LLaVA attack success","Pithy attack: wrong suffix and tiny image noise fool LLaVA","LLaVA-1.5 falls for wrong-answer suffix plus PGD","Simple combo: suffix and PGD trick LLaVA-1.5"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00085,"raw_usage":{"total_tokens":3649,"prompt_tokens":851,"completion_tokens":2798,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":467,"completion_tokens_details":{"reasoning_tokens":2698}},"tokens_in":467,"tokens_out":2798,"duration_ms":17127,"temperature":1.0,"reasoning_tokens":2698,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:15:39.543528+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the attack on a subset of instances where the correct option is independently validated by multiple human annotators or by a trusted source, and drop any instance where the pseudo-label disagrees; if the attack success gain on the fully verified subset is much smaller than the reported 80.92 percent, or if LLaVA-1.5 frequently picks the pseudo-labeled wrong options even on clean images, the claimed attack strength is partly an artifact of label noise.","supporting_citations":[{"cited_title":"Z., and Fredrikson, M","cited_arxiv_id":null,"evidence_quote":"Supplies the suffix-injection method: appending adversarial text to an aligned model's prompt."},{"cited_title":"Towards deep learning models resistant to adversarial attacks","cited_arxiv_id":null,"evidence_quote":"Supplies the PGD attack used to perturb images in the second stage."},{"cited_title":"BERT: pre-training of deep bidirectional transformers for language understanding","cited_arxiv_id":null,"evidence_quote":"Provides BERT, used to measure text similarity and constrain the suffix injection."}],"review_version":1}