Stage-wise Attention-Guided Region Sequencing for Adversarial Attacks on Large Vision-Language Models

Boryeong Cho; Jaehyun Kwak; Nam Cao; Segyu Lee; Se-Young Yun; Sumyeong Ahn

REVIEW 4 major objections 7 minor 7 references

An open-source LVLM's attention map can schedule black-box adversarial updates that beat random cropping across ten vision-language models.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 04:37 UTC pith:NND4ZIPS

load-bearing objection A plausible attention-guided attack with real but often small gains over random-crop baselines; the headline 'state-of-the-art' needs error bars and a direct test of the fixed-map sequencing assumption. the 4 major comments →

arxiv 2602.04356 v2 pith:NND4ZIPS submitted 2026-02-04 cs.CV

Stage-wise Attention-Guided Region Sequencing for Adversarial Attacks on Large Vision-Language Models

Jaehyun Kwak , Nam Cao , Boryeong Cho , Segyu Lee , Sumyeong Ahn , Se-Young Yun This is my paper

classification cs.CV

keywords adversarial attacksvision-language modelscross-modal attentionattention redistributionL-infinity perturbationblack-box attackstage-wise schedulingperturbation budget allocation

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Under a fixed per-pixel L∞ budget, the paper argues, attack success is a scheduling problem: perturbations should go first to image regions that most influence the target-caption similarity loss, then to regions that become influential as attention shifts. It establishes two empirical regularities in vision-language models: regional attention scores are positively correlated with adversarial loss sensitivity, and attacking a high-attention hotspot suppresses attention there while redirecting it toward the next-most-salient regions. On that basis it proposes SAGA, a black-box attack that extracts one attention map from an open-source model before optimization and uses it to define a stage-wise sequence of crop regions, expanding from concentrated hotspots to the full image. Across five closed-source and five open-source target LVLMs, SAGA reports the best attack success rate on every target and the smallest ℓ1/ℓ2 perturbation norms. The significance is that a single transferable attention signal, not gradients or attention from the target, can allocate a constrained budget to the pixels that matter.

Core claim

Central claim: regional attention values in an LVLM's cross-modal attention map predict adversarial loss sensitivity, and perturbing a high-attention hotspot suppresses its attention while redirecting mass to the next-salient regions; therefore an L∞-bounded attack should be scheduled as a stage-wise expansion from dominant hotspots outward. SAGA instantiates this with one attention map extracted once from an open-source LVLM, ten stages of top-k hotspot crops constrained by IoU, and surrogate image-text contrastive cosine-similarity optimization inside each crop under a global 16/255 bound, never touching target parameters, gradients, or attention. Reported results: best attack success rate

What carries the argument

The load-bearing object is a cross-modal attention map M extracted once from an open-source LVLM by prompting it to caption the source image; from that map the attack precomputes a stage-wise set of hotspots. At each of N=10 stages, the top-k=3 attention regions at an expanding area ratio are selected with an IoU threshold of 0.3 to remove redundancy, giving k×N crop regions that span from the most concentrated attention areas to the full image. Optimization then runs 300 iterations of a surrogate cosine-similarity attack, evenly split across those regions; at each iteration a random sub-crop within the current hotspot is updated under the global L∞ bound. The paper's own analysis uses Pears

Load-bearing premise

The load-bearing premise is that one attention map extracted once from an open-source LVLM—as the paper's own limitation section acknowledges, the method needs such a model—remains a valid guide to which regions become influential over the full 300-epoch attack; if early perturbation reshuffles saliency away from that precomputed order, the stage-wise schedule's rationale fails.

What would settle it

Run SAGA while recomputing the attention map from the same open extractor every stage, and compare attack success and attention-order statistics against the fixed-map version. If attack success does not improve and the rank correlation between precomputed hotspots and recomputed loss-sensitivity hotspots stays high, the fixed map is sufficient; if the correlation drops sharply during early stages or the adaptive version gains substantially, the static-map assumption, and with it the proposed scheduling mechanism, is falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

A single attention map from one open-source model is enough to attack closed-source targets with no access to their parameters, gradients, or attention maps.
The ordering of region updates matters: targeting coldspots instead of hotspots lowers attack success even though the same crop-level optimization is used.
Budget is used structurally, not by magnitude: SAGA ends with a lower pixel-saturation ratio yet higher attack success, implying imperceptibility and effectiveness do not trade off in the same way as in random-crop attacks.
Attention shifts under a successful SAGA attack are systematic rather than stochastic, scaling with similarity score, which distinguishes it from random cropping.
Attention extractors from the same model family as the target give slight extra gains, but cross-family attention still beats the baselines, implying the vulnerability signal transfers across architectures.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If the correlation result is causal rather than merely correlational, any saliency signal that predicts gradient sensitivity—not just cross-modal attention—could be substituted into the same scheduling framework; a cheaper saliency predictor might replicate SAGA's gains.
Because SAGA never recomputes attention during the 300-epoch run, a natural adaptive extension would recompute or re-rank hotspots when attention redistribution saturates; the paper's success with a static map suggests such an adaptive version might do even better, but the paper does not test this.
The attention-redistribution observation also suggests a defensive angle left implicit: a monitor that detects attention mass flowing from attacked hotspots to subsequent regions could flag images under active stage-wise attack, and training that regularizes attention dynamics could blunt this class of attacks.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

A plausible attention-guided attack with real but often small gains over random-crop baselines; the headline 'state-of-the-art' needs error bars and a direct test of the fixed-map sequencing assumption.

read the letter

SAGA is a legitimate incremental advance over M-Attack/FOA: it replaces stochastic whole-image cropping with attention-ranked stage-wise hotspots, and it contributes two new empirical observations — attention scores correlate with adversarial loss sensitivity, and attacking hotspots shifts attention toward the next salient regions. The method is simple, the surrogate setup matches the baselines, and the tables show consistent if sometimes thin gains. On Gemini models the 43% relative improvement is substantial; on several open-source targets the margins are 2-3 points and could vanish with error bars.

That brings me to the main soft spots. First, no confidence intervals or repeated seeds anywhere. With 1,000 samples, a 2-4% ASR gap is within noise, and the text repeatedly says 'consistently best' without qualification. Second, the attention extractor and layer are chosen using the same CLIP loss and the same Qwen3-VL family that generates the target captions — that is a mild self-referential loop, and the correlation analysis is only 50 image-text pairs. Third, and most load-bearing, the fixed attention map is never recomputed during the 300-epoch attack. The claim that attention redistributes toward the precomputed next hotspots is supported only by a 5-epoch experiment, a correlational JS-divergence plot, and one qualitative figure. There is no ablation that shuffles the hotspot order, and no ablation that recomputes attention periodically. So the stage-wise scheduling story is plausible but not directly established — the gains might come mostly from simply prioritizing high-attention regions early, not from the specific ordering.

There are also smaller packaging issues: the pseudocode in Algorithm 1 computes updates on a crop but never writes them back into the full image, the GitHub URL in the abstract differs from the one in the text, and Table 2 calls the target 'Llama 4 Maverick' while the text calls it 'LLaVA-4 Maverick'. These are easy fixes but suggest carelessness.

Net: the idea is worth taking seriously, the empirical core is defensible, and the paper deserves a serious referee. The path to acceptance is not deep rethinking but better evidence: error bars, a hotspot-order ablation, and ideally a version that recomputes attention every few epochs. With those, the SOTA claim would be much more credible. I'd engage with it and would cite it as a relevant baseline in the meantime.

Referee Report

4 major / 7 minor

Summary. The paper proposes SAGA, a black-box targeted attack on large vision-language models (LVLMs) under the L-infinity perturbation budget. SAGA extracts a single cross-modal attention map from an open-source LVLM (Qwen3-VL-8B), precomputes 30 stage-wise hotspots of increasing area, and runs surrogate-based PGD inside the current hotspot for a fixed number of iterations, never recomputing attention during the attack. The paper claims state-of-the-art attack success rates and best imperceptibility across ten closed- and open-source LVLMs. The central mechanistic claims are (i) regional attention scores positively correlate with adversarial loss sensitivity, and (ii) attacking high-attention regions redistributes attention toward subsequently salient regions, motivating the fixed stage-wise schedule.

Significance. The core idea — using cross-modal attention as a scheduling signal for localized perturbations — is original and practically relevant. The paper ships source code, includes several ablations (hotspot vs. coldspot, attention-extractor choice), and makes falsifiable predictions about attention redistribution. If the central claims hold, the method would be a strong alternative to stochastic cropping-based attacks. However, the main SOTA claim is currently underpowered: the primary tables report single-run numbers without error bars, the attention extractor/layer are selected on the same loss and data distribution used in the final evaluation, and the stage-wise sequencing mechanism lacks a direct ablation. These issues are fixable within the manuscript's scope, but they currently make the headline conclusion conditional.

major comments (4)

[§4.2, Tables 1–2] The main comparison tables report only single-run ASR and AvgSim values with no error bars or repeated-seed statistics. Several margins over the strongest baseline are small — e.g., LLaVA-1.5-7B: 0.67 vs. 0.65; Qwen3-VL-30B: 0.55 vs. 0.53 — and with n=1000 images the standard error of a proportion is about 0.015, so these differences are within roughly 1–2 standard errors. The claim that SAGA "consistently achieves the best performance" requires confidence intervals, multiple random-seed runs (especially for the random crop in Algorithm 1), or a paired bootstrap. This is load-bearing for the SOTA claim.
[Appendix C.2] The attention extractor (Qwen3-VL-8B) and the layer (e.g., layer 29 for Qwen3-VL-8B) are selected as the ones with the highest attention–loss correlation measured on the same CLIP-based cosine loss and the same data distribution (NIPS 2017 images, MSCOCO-generated captions) used in the final attack evaluation. The selection set is very small (10 images × 5 texts = 50 pairs), and no correction is made for the multiple layers tested. This constitutes a form of selection on the evaluation distribution; the reported advantage over random-crop baselines may be partly inflated by this selection. The analysis should use a held-out validation set for extractor/layer choice or report results averaged over plausible layers/extractors.
[§3, Algorithm 1] Algorithm 1 computes the attention map once (line 2) and never recomputes it during the 300-epoch attack. The method's rationale is that attacking a hotspot redirects attention to the next precomputed hotspot, so the fixed schedule tracks the evolving saliency. The presented evidence is indirect: Figure 3 is a 5-epoch local analysis on a small set; Figure 8 measures only total JS divergence, not whether attention moves into the specific precomputed next hotspots; Figure 9 is a single qualitative example. No ablation compares the proposed order with, e.g., a shuffled/reversed ordering of the same 30 hotspots or a variant that periodically recomputes attention. Without such a control, the superiority of the stage-wise sequencing over simply concentrating on the top hotspots is not established; the current gains could be attributable to hotspot selection alone.
[§4.1] The target captions are generated with Qwen3-VL-8B-Instruct, the default attention extractor is Qwen3-VL-8B, and two of the five open-source targets are Qwen3-VL-30B and Qwen3-VL-235B. The same-family overlap between extractor and two target models is a potential confound for the transferability claim. The observed gains on the Qwen targets are small (e.g., Qwen3-VL-30B: 0.55 vs. 0.53; Qwen3-VL-235B: 0.65 vs. 0.62). The paper should either provide a cross-family analysis for all reported targets (e.g., using LLaVA-7B as extractor on Qwen targets and vice versa in the main tables) or discuss explicitly how the family overlap affects the conclusions.

minor comments (7)

[Throughout] Inconsistent model naming: "Qwen3-VL-245B" in Figure 15 vs. "Qwen3-VL-235B" in Tables 1–2; "LLaVA-4 Maverick" vs. "Llama 4 Maverick" in Table 2; "Qwen-3-VL-30B" vs. "Qwen3-VL-30B" in the text. Please unify.
[§2.2] The correlation analysis says "a single gradient optimization step" is applied to each crop, but the step size and the number of CLIP surrogate models used are not specified. This detail is needed for reproducibility.
[Appendix C.2] The evaluation set of 10 source images and 5 target texts (50 pairs) is very small for selecting a layer across dozens of layers; the paper should state this as a limitation or provide a more extensive analysis.
[§2.3, Figure 3] Figure 3 appears to be a qualitative illustration; the quantitative evidence is in Appendix C.3. The main text should make this explicit and cross-reference the appendix to avoid overstating the strength of the observation.
[Algorithm 1, line 9] The operation "RandomCrop(R_{n,j})" is undefined. Does it sample a sub-region inside the hotspot or treat the entire hotspot as the crop? Please clarify.
[§1 and §4.1] The term "black-box" should be qualified: the method is black-box with respect to the target models, but it uses white-box surrogate CLIP models for gradient computation. This is standard in transfer attacks, but the text should be explicit.
[Eq. (2)] The expression min(|δ_p|/ε, 1) is not a norm; consider renaming the quantity "per-pixel saturation fraction" or "budget saturation ratio" with a clear definition.

Circularity Check

0 steps flagged

No load-bearing circularity; SAGA's core results are measured against external benchmarks and independent baselines.

full rationale

The paper's derivation chain is not circular. SAGA's two motivating observations (attention–loss correlation and attention redistribution) are empirical measurements on evaluation sets, and the algorithm instantiates them by precomputing stage-wise hotspot regions from a single attention map. The central SOTA claim is validated on ten target LVLMs against external baselines (M-Attack, FOA-Attack, etc.), not derived from the same data used to fit the attention extractor. Although the attention extractor (Qwen3-VL-8B) and its layer are selected after the correlation analysis in Section 2.2/Appendix C.2, this is a hyperparameter choice based on a proxy, and the reported ASR/imperceptibility numbers are independent measurements; the 'Attention Extractor Model Ablation' only confirms the correlation ranking, and it is not load-bearing for the main comparison. Target texts are generated with Qwen3-VL-8B-Instruct, but attention maps are extracted by captioning the source image, not by using target texts, so no definitional loop exists. The limitation in Appendix A (dependence on an open-source LVLM for attention) is an explicit assumption, not a circular justification. The fixed attention map's possible invalidity after attention redistribution is a correctness/robustness concern, not circularity: the paper provides direct (Fig. 9) and correlational (Fig. 8) evidence for redistribution, and even if that evidence is weak, the reported gains are empirical rather than implied by construction. No self-citations are load-bearing; the reference list contains no prior work by the present authors. Consequently, no specific circular step can be exhibited under the required standard.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 0 invented entities

Most of the attack's hyperparameters (N, k, τ, E, ε) are hand-chosen; the attention extractor and layer are selected on a small correlation set using the same loss and target distribution as the final evaluation, and target captions are generated by the same model family as the default extractor. These are the main owed assumptions.

free parameters (7)

Number of stages N = 10
Hand-chosen; divides the attack into 10 progressively expanding hotspot stages. No sensitivity analysis is reported, so the optimality of N=10 on the tested models is not established.
Hotspots per stage k = 3
Hand-chosen; selects top-k high-attention regions per stage with IoU threshold τ=0.3. Ablations do not vary k or τ.
IoU threshold τ = 0.3
Hand-chosen to reduce spatial redundancy among selected hotspots; no ablation.
Attention layer per extractor = 17 (LLaVA-7B), 6 (LLaVA-13B), 29 (Qwen3-VL-8B)
Selected as the layer with highest Pearson/Spearman correlation on a 50-pair, 1000-sample analysis; this is fitting the extractor to the attack-loss proxy.
Default attention extractor = Qwen3-VL-8B
Chosen because it had the highest measured attention–loss correlation; ablations confirm ordering, but this selection was made on the same evaluation distribution rather than an independent validation split.
Perturbation budget ε and step size = 16/255, 1/255
Follows prior work; standard but hand-set.
Total iterations E = 300
Hand-chosen; budget split evenly among N*k hotspots. No convergence analysis.

axioms (5)

domain assumption Cross-modal attention maps extracted from a single open-source LVLM transfer across target models, including closed-source LVLMs.
SAGA assumes the Qwen3-VL-8B attention hotspots are valid vulnerability guides for all ten targets; this is central to the black-box claim and is only empirically supported by the correlation/ablation results.
domain assumption The static, precomputed hotspot sequence remains valid for the entire attack trajectory; attention redistribution follows the initial saliency ordering.
Algorithm 1 extracts attention once and never recomputes it; the stage-wise expansion is based on the initial ranking, so the method depends on the stability of this ordering under perturbation. Figure 8 provides indirect evidence but does not prove that the precomputed order matches the actual redistribution.
domain assumption Average attention over all heads at the selected layer is a meaningful proxy for adversarial loss sensitivity.
The correlation analysis assumes that the selected layer's mean attention map, normalized over vision tokens, captures the regions whose perturbation moves the CLIP cosine loss most; this is the empirical basis for hotspot selection.
domain assumption The gpt-oss-20b judge scores are valid semantic similarity measurements, and the 0.5 threshold defines attack success correctly.
The headline ASR numbers are entirely produced by this judge; no human validation or calibration against alternative judges is provided.
domain assumption Random cropping inside each hotspot with a global L∞ projection is sufficient to optimize the local cosine-similarity objective.
The algorithm uses random subregion crops rather than solving for an optimal mask; this is inherited from M-Attack/FOA-Attack and is a heuristic.

pith-pipeline@v1.3.0-alltime-deepseek · 16886 in / 13184 out tokens · 129909 ms · 2026-08-03T04:37:52.116747+00:00 · methodology

0 comments

read the original abstract

Targeted adversarial attacks on Large Vision-Language Models (LVLMs) test whether small image perturbations can steer model responses toward attacker-specified content. Under the standard L-infinity constraint, targeted attacks become a regional perturbation budget allocation problem: attack success depends not only on the perturbation objective, but also on which regions receive updates and in what order. Existing localized attacks improve over global perturbations but rely on stochastic spatial sampling, often updating weakly influential regions. We address this limitation through an attention-based analysis showing that cross-modal attention identifies adversarially sensitive regions and that perturbing high-attention hotspots induces predictable redistribution toward subsequent salient regions. These findings motivate attention-guided region sequencing, which begins from dominant hotspots and progressively moves the update support toward next-salient regions. Based on these principles, we propose Stage-wise Attention-Guided Attack (SAGA), a black-box region-sequencing framework that uses a fixed attention map from an open-source LVLM to guide perturbation updates without accessing target-model parameters, gradients, or attention maps. Across ten closed-source and open-source LVLMs, SAGA achieves state-of-the-art attack success rates and the best overall imperceptibility. The source code is available at https://github.com/jaehyun-kwak/SAGA.

Figures

Figures reproduced from arXiv: 2602.04356 by Boryeong Cho, Jaehyun Kwak, Nam Cao, Segyu Lee, Se-Young Yun, Sumyeong Ahn.

**Figure 1.** Figure 1: Example captioning responses from commercial and open-source LVLMs to images attacked using our method. Lyu et al., 2024), jailbreaking (Niu et al., 2024; Shayegani et al., 2024), and adversarial attacks (Zhao et al., 2023; Dong et al., 2023). Among these threats, we focus on adversarial attacks, which offer a controlled way to evaluate the robustness of LVLMs by examining how carefully crafted input pert… view at source ↗

**Figure 2.** Figure 2: All LVLMs show a statistically significant positive correlation between attention score and adversarial loss change, indicating larger loss changes for highattention regions. Decrease Decrease Increase Decrease Top 10% Attention Top 20% Attention (a) Attention Map (b) Random Crop (c) Attention-guided Crop [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 4.** Figure 4: Overview of SAGA. Given a source image, we first extract an attention map using an open-source LVLM and pre-compute stage-wise attention hotspots before the attack. At each stage, we crop the image with the current hotspot and optimize the adversarial perturbation within the cropped region. We progressively expand the attacked region across stages, guiding the optimization toward newly emerging high-attent… view at source ↗

**Figure 5.** Figure 5: Hotspot vs. coldspot comparison on Qwen3-VL-235BA22B-Instruct and Gemini-2.5-Flash models. ensemble multiple CLIP surrogate models during optimization, we also evaluate an ensemble attention variant obtained by averaging attention maps across all three models. We conduct this ablation on five closed-source target models (Gemini-2.5-Flash, Gemini-3-Pro, GPT-4.1, GPT-5 Mini, and Grok 4 Fast) with results s… view at source ↗

**Figure 8.** Figure 8: The attention shift is quantified using Jensen–Shannon (JS) divergence between attention maps before and after the attack. M-Attack and FOA-Attack exhibit no consistent relationship between attention shift and attack similarity. In contrast, SAGA shows a clear increasing trend, where higher attack similarity is associated with larger attention shifts, reflecting structured redistribution of attention dur… view at source ↗

**Figure 9.** Figure 9: Visualization of attention maps from attacked images every 60 epochs for M-Attack, FOA-Attack, and SAGA. 5. Related Work 5.1. Large Vision–Language Models Large Vision–Language Models (LVLMs) extend large language models by incorporating visual modalities, enabling joint reasoning over images and text. Trained on large-scale image–text data, LVLMs have demonstrated strong performance on multimodal tasks … view at source ↗

**Figure 10.** Figure 10: Layer-wise correlation between regional attention scores and adversarial loss sensitivity. Markers with crosses indicate layers where the correlation is not statistically significant (p > 0.05). We conduct a correlation analysis on three open-source LVLMs: LLaVA-7B, LLaVA-13B, and Qwen3-VL-8B. The goal of this experiment is to examine the relationship between regional cross-modal attention scores and adve… view at source ↗

**Figure 11.** Figure 11: Comparison of attention changes in the top 10% attention regions when applying random crop attacks and hotspot-based attacks. Nearby (H . H . ) Coldspot (H . H . ) 0.0004 0.0002 0.0000 0.0002 0.0004 Attention Where Does the Removed Attention Go? Random Crop Hotspot [PITH_FULL_IMAGE:figures/full_fig_p013_11.png] view at source ↗

**Figure 13.** Figure 13: Evaluation prompt for LLM-as-judge. D.3. Qualitative Examples of Attacked Images E. More Case Studies We present additional examples of responses in [PITH_FULL_IMAGE:figures/full_fig_p014_13.png] view at source ↗

**Figure 14.** Figure 14: Qualitative comparison of adversarial images generated by M-Attack, FOA-Attack, and SAGA. We provide additional qualitative results using the SAGA attacked image on commercial models in [PITH_FULL_IMAGE:figures/full_fig_p015_14.png] view at source ↗

**Figure 15.** Figure 15: Qualitative case studies comparing M-Attack, FOA-Attack, and SAGA across multiple target models. Each panel illustrates the generated adversarial images and their corresponding model-generated responses. The similarity score measures each method’s effectiveness on its respective model. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_15.png] view at source ↗

**Figure 16.** Figure 16: Qualitative case studies of SAGA against commercial models: (a) Gemini-3 Pro, (b) GPT-5.2 Pro, (c) Grok-4.1 Thinking, and (d) Qwen-3 Max Thinking. Each panel displays the model response to an adversarial image generated with the target text: “Two giraffes stand near a beige building, surrounded by rocks and lush green trees in a zoo-like enclosure.” All models were prompted with “Describe this image, no l… view at source ↗

discussion (0)

Reference graph

Works this paper leans on

7 extracted references · 1 linked inside Pith

[1]

**Main Subject Consistency:** If both descriptions refer to the same key subject or object (e.g., a person, food, an event), they should receive a higher similarity score
[2]

Describe this image

URL https://openreview.net/forum? id=chanJGoa7f. Niu, Z., Ren, H., Gao, X., Hua, G., and Jin, R. Jail- breaking Attack against Multimodal Large Language Model, February 2024. URL http://arxiv.org/ abs/2402.02309. arXiv:2402.02309 [cs]. Pathade, C. Invisible Injections: Exploiting Vision- Language Models Through Steganographic Prompt Em- bedding, July 2025...

Pith/arXiv arXiv 2024
[3]

Focus on **whether both descriptions fundamentally describe the same thing.**

**Ignore Fine-Grained Details:** Do not penalize differences in **phrasing, sentence structure, or minor variations in detail**. Focus on **whether both descriptions fundamentally describe the same thing.**
[4]

**Relevant Description**: If the descriptions are related to the same context or topic, they should also contribute to a higher similarity score
[5]

Susan's Sweets,

**Similarity Score Range:** - **1.0**: Nearly identical in meaning. - **0.8-0.9**: Same subject, with highly related descriptions. - **0.7-0.8**: Same subject, core meaning aligned, even if some details differ. - **0.5-0.7**: Same subject but different perspectives or missing details. - **0.3-0.5**: Related but not highly similar (same general theme but d...
[6]

**Partial Matches:** If one description contains extra information but does not contradict the other, they should still have a high similarity score
[2025]

URL https://openreview.net/forum? id=9xXjWwAoUF. Lin, J. Divergence measures based on the shannon entropy. IEEE Transactions on Information Theory, 37(1):145– 151, 1991. doi: 10.1109/18.61115. Liu, H., Li, C., Wu, Q., and Lee, Y . J. Visual instruction tuning. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.),Advances in ...

doi:10.1109/18.61115 1991

[1] [1]

**Main Subject Consistency:** If both descriptions refer to the same key subject or object (e.g., a person, food, an event), they should receive a higher similarity score

[2] [2]

Describe this image

URL https://openreview.net/forum? id=chanJGoa7f. Niu, Z., Ren, H., Gao, X., Hua, G., and Jin, R. Jail- breaking Attack against Multimodal Large Language Model, February 2024. URL http://arxiv.org/ abs/2402.02309. arXiv:2402.02309 [cs]. Pathade, C. Invisible Injections: Exploiting Vision- Language Models Through Steganographic Prompt Em- bedding, July 2025...

Pith/arXiv arXiv 2024

[3] [3]

Focus on **whether both descriptions fundamentally describe the same thing.**

**Ignore Fine-Grained Details:** Do not penalize differences in **phrasing, sentence structure, or minor variations in detail**. Focus on **whether both descriptions fundamentally describe the same thing.**

[4] [4]

**Relevant Description**: If the descriptions are related to the same context or topic, they should also contribute to a higher similarity score

[5] [5]

Susan's Sweets,

**Similarity Score Range:** - **1.0**: Nearly identical in meaning. - **0.8-0.9**: Same subject, with highly related descriptions. - **0.7-0.8**: Same subject, core meaning aligned, even if some details differ. - **0.5-0.7**: Same subject but different perspectives or missing details. - **0.3-0.5**: Related but not highly similar (same general theme but d...

[6] [6]

**Partial Matches:** If one description contains extra information but does not contradict the other, they should still have a high similarity score

[7] [2025]

URL https://openreview.net/forum? id=9xXjWwAoUF. Lin, J. Divergence measures based on the shannon entropy. IEEE Transactions on Information Theory, 37(1):145– 151, 1991. doi: 10.1109/18.61115. Liu, H., Li, C., Wu, Q., and Lee, Y . J. Visual instruction tuning. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.),Advances in ...

doi:10.1109/18.61115 1991