{"id":"28bc851f-5f37-4acc-9437-0cdf86249e7f","arxiv_id":"2607.07288","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":8,"one_line_summary":"Edge-placed QR-inspired structured patches reduce infrared CLIP classifier accuracy from 98.67% to 0.70% and transfer to black-box captioning and VQA models.","lead":"This paper shows that small QR-code-like patches placed at the edges of infrared images can fool vision-language models into misclassifying objects, generating wrong captions, and answering questions incorrectly. The finding matters because it reveals that AI systems analyzing thermal imagery can be attacked without directly covering the target object.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"Transfer claims for captioning rest on unvalidated GPT-5.4 scoring with no error bars; InfraQR's captioning advantage over baselines is marginal in 2/6 settings and could be within noise.","rationale":"The reader correctly identified the GPT-5.4 evaluation as unvalidated in the rationale, but placed the primary weakest_assumption on probe-based placement and benchmark size. The probe concern is less load-bearing because suboptimal placement would underestimate attack strength, not threaten the central claim. The benchmark size concern is valid for generalization but the classification result is so extreme that statistical significance is not in doubt for the white-box setting. The more load-bearing concern is that the captioning transfer claims — where InfraQR's advantage is marginal — depend entirely on an unvalidated automated judge with no error bars. This doesn't overturn the core vulnerability finding (which is well-supported by classification and VQA), but it means the CONDITIONAL verdict is appropriate: the paper should validate its evaluation protocol and report uncertainty before the transfer claims can be fully accepted. The VQA results are more convincing due to larger margins and more constrained evaluation. Overall, the reader's CONDITIONAL verdict with MODERATE confidence is appropriate; my concern refines the rationale but doesn't change the verdict.","tokens_in":14642,"tokens_out":4903,"duration_ms":326271,"concrete_test":"Select a random subset of 50 clean/attacked caption pairs from Table 2 (across InfraQR and AdvIC). Have 3 independent human annotators score semantic consistency on the same 0-100 scale used by GPT-5.4. Compute Spearman correlation between mean human scores and GPT-5.4 scores. Additionally, bootstrap-resample the 300-image captioning results 1000 times to obtain 95% confidence intervals for each method's drop. If the correlation is below 0.7 or the InfraQR-vs-AdvIC confidence intervals overlap in the 2 settings where AdvIC wins, the captioning transfer advantage claim should be softened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim has two pillars: (1) white-box classification degradation and (2) black-box transfer to captioning/VQA. Pillar (1) is strongly supported — the 98.67%→0.70% result on OpenAI CLIP is objective and dramatic. Pillar (2) is weaker. For captioning (Table 2), InfraQR produces the largest drop in only 4 of 6 settings; in LLaVA-1.6 and InstructBLIP, AdvIC actually outperforms InfraQR (33.45 vs 33.03 and 8.24 vs 7.50 drops respectively). These margins are small and no error bars or significance tests are reported on a 300-image benchmark. Crucially, the entire captioning evaluation depends on GPT-5.4 as the sole judge of 'semantic consistency,' with no human validation, no alternative metrics (BLEU, CIDEr, CLIPScore), and no inter-annotator agreement reported. If GPT-5.4 systematically scores terse or unusual adversarial captions as less consistent (even when semantically equivalent), the captioning drops could be inflated. The VQA evaluation is less fragile because answer correctness is more constrained (yes/no, counting), and InfraQR's VQA advantage is larger and consistent across all 6 settings. The concern is therefore specifically about the captioning transfer claims and the precision of comparative statements, not about the core vulnerability existence.","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"The manuscript presents InfraQR, a QR-inspired structured adversarial patch attack targeting infrared vision-language models. Unlike prior infrared adversarial work that focuses on object detectors and places perturbations on the target object, InfraQR places a compact structured patch along image boundaries. The patch uses fixed finder-style anchors and learnable grid cells optimized through surrogate CLIP-style encoders with a binary regularization term. The authors evaluate InfraQR on three tasks: (1) white-box classification on four CLIP-style infrared classifiers, (2) black-box transfer to six captioning models, and (3) question-answer-aware VQA on six black-box generative models. On a 300-image infrared benchmark, InfraQR reduces OpenAI CLIP accuracy from 98.67% to 0.70% and produces competitive or superior degradation compared to three baselines (HCB, AdvIC, AdvGrid) across transfer settings. The central claim is that infrared VLMs are vulnerable to peripheral structured perturbations without direct object occlusion.","tokens_in":15077,"tokens_out":1446,"duration_ms":258979,"significance":"The paper addresses a genuine gap: prior infrared adversarial work targets detectors, while VLM robustness studies focus on RGB images. The finding that edge-placed structured patches can degrade not only surrogate classification but also downstream captioning and VQA behavior is non-trivial and motivates broader robustness evaluation. The white-box classification results are strong and clearly presented. The multi-task evaluation framework (classification, caption transfer, QA-aware VQA) is a reasonable design for studying cross-task vulnerability. The QA-aware extension (Section 3.5) that constructs question-conditioned semantic targets without accessing the evaluated VQA models is a thoughtful design choice. The ablation study (Section 4.5) covers key parameters including iteration budget, patch size, learning rate, and binary regularization weight.","major_comments":[{"comment":"Table 2 (caption transfer): InfraQR produces the largest semantic-consistency drop in only 4 of 6 settings. In the remaining 2 (LLaVA-1.6 under OpenAI CLIP: 33.03 vs AdvIC's 33.45; InstructBLIP under EVA-CLIP: 7.50 vs AdvIC's 8.24), AdvIC outperforms InfraQR. These margins are small, and no error bars, confidence intervals, or significance tests are reported on a 300-image benchmark. The manuscript should either (a) report variance or significance to support the comparative superiority claims, or (b) soften the language to acknowledge that InfraQR is competitive rather than strictly superior in caption transfer. As stated, the claim in Section 4.3 that InfraQR 'consistently causes a stronger or competitive degradation' is acceptable, but the abstract's implication of consistent superiority across tasks is slightly overstated for the captioning setting.","section":null},{"comment":"Section 4.3 and Table 2: The entire captioning evaluation depends on GPT-5.4 as the sole judge of semantic consistency, with no human validation, no alternative metrics (e.g., BLEU, CIDEr, CLIPScore), and no inter-annotator agreement reported. If GPT-5.4 systematically scores terse or unusual adversarial captions as less consistent even when semantically equivalent, the captioning drops could be inflated or distorted. The VQA evaluation (Table 3) is less fragile because answer correctness is more constrained (yes/no, counting, etc.), and InfraQR's advantage there is larger and consistent across all 6 settings. The authors should add at least one standard captioning metric or a small human validation subset to corroborate the GPT-5.4 scoring for the captioning claims.","section":null},{"comment":"Section 4.1: The 300-image benchmark (30 categories, 10 images each) is small. While the white-box classification results are dramatic enough that scale is less concerning there, the transfer claims to captioning and VQA models would be more convincing with a larger evaluation set. The VQA subset (295 image-question pairs, 236 test) is also modest. The authors should discuss this limitation explicitly and ideally provide results on a larger subset or cross-dataset validation to support generalization claims.","section":null}],"minor_comments":[{"comment":"The abstract mentions 'GPT-5.4-based evaluation' but the model name 'GPT-5.4' appears unusual; the reference [20] cites 'GPT-5.4 model' from 2026. If this is a placeholder or internal name, it should be clarified for readers.","section":null},{"comment":"Figure 1 is dense and the text labels are small; the flow from 'Edge Candidate Locations' through 'Attack Optimization' to 'Evaluation' could be clearer with larger annotations or a simplified sub-panel.","section":null},{"comment":"Section 3.4: The probe-based placement search uses rho_probe=0.5 (Eq. 15). The assumption that a neutral gray patch identifies the optimal location for an adversarially optimized structured patch is not validated. A brief ablation comparing probe-based selection vs. random or exhaustive placement would strengthen this design choice.","section":null},{"comment":"Table 1: The 'Drop' column for EVA-CLIP under InfraQR reports 35.93, but 98.33 - 62.40 = 35.93, which checks out. However, the HCB drop for EVA-CLIP is listed as 0.68 (98.33 - 97.65 = 0.68), which is correct but unusually small; a brief note on why EVA-CLIP is more robust would add context.","section":null},{"comment":"Section 3.2: The choice of G=21 grid resolution with three 7x7 anchors is stated but not justified. Why 21 specifically? A brief note on how this was selected (e.g., pilot experiments) would help reproducibility.","section":null},{"comment":"The manuscript uses 'GPT-5.4' for both caption scoring and VQA judging. It would help to specify the exact prompting protocol used for GPT-5.4 evaluation (e.g., the rubric shown in Figure 3C) in the main text or supplementary material for reproducibility.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The stress-test concern about GPT-5.4 scoring and marginal captioning advantages is valid and lands on a real weakness, but it does not undermine the core vulnerability claim. The white-box classification results are strong and the VQA transfer results are consistent. The captioning transfer is the weakest pillar, but the paper's language is mostly hedged ('competitive') except in the abstract. I recommend minor revision with the addition of variance/significance reporting and at least one corroborating captioning metric. The small benchmark size is a limitation but not a blocker given the dramatic classification results and consistent VQA transfer."},"author_rebuttal":null,"desk_editor":{"model":"glm-5.2","letter":"Quick take: I think this paper identifies a real vulnerability class — peripheral structured perturbations can disrupt infrared vision-language models without touching the target object. The white-box classification results are dramatic and the VQA transfer is convincing. The captioning transfer claims are softer than they look because the evaluation rests entirely on GPT-5.4 as judge with no error bars on a 300-image benchmark, and InfraQR's advantage over baselines in captioning is marginal in 2 of 6 settings. I agree with the reader's conditional verdict and the stress-test concern, which lands accurately on the paper as written. The core vulnerability claim holds; the precision of the comparative captioning statements does not. I'd send this to review with a request for code release, larger benchmarks, and at minimum one alternative captioning metric alongside GPT-5.4.","headline":"Edge-placed QR-inspired patches sharply degrade infrared VLMs; captioning transfer claims rest on unvalidated GPT-5.4 scoring with no error bars.","tokens_in":15664,"tokens_out":246,"would_cite":false,"duration_ms":135534,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Edge-Placed QR-Like Patches Crash Infrared Vision-Language Models","keywords":["adversarial patch attack","infrared vision-language models","QR-inspired structured perturbation","edge placement","CLIP","visual question answering","cross-task transferability","binary regularization"],"falsifier":"If infrared VLMs that use visually different architectures (not CLIP-family encoders) prove resistant to the same edge-placed structured patches, or if the cross-task transfer to captioning and VQA fails to replicate on larger and more diverse infrared benchmarks, the claim of a general peripheral-perturbation vulnerability would be weakened.","tokens_in":14867,"feed_emoji":"🌡️","tokens_out":1175,"duration_ms":166865,"temperature":0.7,"pith_summary":"This paper sets out to prove that infrared vision-language models — systems that connect thermal imagery to language understanding — can be broken not by attacking the object in the image, but by placing a small, structured, QR-code-like patch along the image's outer edge. The patch uses fixed corner anchors (mimicking QR finder patterns) and a grid of learnable cells pushed toward black-or-white values, optimized through a surrogate CLIP-style image-text model to flip the model's semantic ranking. The central claim is that this peripheral perturbation, spatially separated from the main visual evidence, still propagates through the model's global image-text representation and degrades downstream language outputs. On a 300-image infrared benchmark, the attack drops OpenAI CLIP classification accuracy from 98.67% to 0.70%, and the same adversarial images transfer to unseen captioning and visual question answering models, causing semantic degradation in generated captions and more error-prone VQA answers. The paper argues that image boundaries should not be treated as semantically irrelevant for infrared VLMs, and that the vulnerability is distinct from simple object occlusion.","feed_headline":"Edge Patch Drops Infrared CLIP Accuracy from 98.67% to 0.70%","feed_subtitle":"A QR-like structured patch placed at the image boundary — not on the object — crashes infrared vision-language models and transfers to black","key_machinery":"The attack combines four components: (1) a QR-inspired grid parameterization with fixed 7×7 finder-style anchors at three corners and learnable sigmoid-activated cells elsewhere, regularized toward binary values; (2) a discrete edge-placement search using a neutral gray probe patch to find the boundary location that most reduces ground-truth semantic similarity in the surrogate; (3) a classification-guided ranking loss that minimizes the gap between the ground-truth class similarity and the strongest competing class similarity; and (4) a question-answer-aware extension that constructs semantic text targets from question-reference-answer triplets, allowing the surrogate to provide gradients V","core_discovery":"The paper identifies that a compact, near-binary structured patch placed at an image edge — not on the target object — can be optimized through a surrogate CLIP-style encoder to sharply degrade infrared vision-language representations, and that this degradation transfers across tasks (classification to captioning to VQA) and across models (white-box surrogate to black-box targets). The mechanism is a ranking loss: the patch is tuned to push the ground-truth class similarity below the strongest competing class similarity in the surrogate's embedding space, and the resulting image-level bias propagates to downstream generative models that share similar visual encoding foundations.","pith_inferences":["If the shared-encoder transfer pathway holds generally, then any infrared VLM using a CLIP-family visual backbone would be vulnerable to the same surrogate-guided edge-patch strategy, regardless of the language model attached downstream.","The binary regularization producing near-binary patches suggests these patterns could potentially be rendered on physical displays or thermal materials at image boundaries, though the paper explicitly does not claim physical realizability — this is an untested extension.","The probe-based placement search using a neutral gray patch may underestimate the attack's potential; a jointly optimized placement could reveal even stronger vulnerabilities, meaning the reported degradation might be a lower bound on the true risk.","The finding that EVA-CLIP remains more robust than other encoders under all attacks hints that encoder architecture or training scale may confer differential resistance to structured peripheral perturbations, which could inform defensive design."],"forward_implications":["Infrared VLM security audits should evaluate peripheral perturbations, not just object-attached patches, since edge regions can carry enough representational weight to flip global image-text rankings.","The cross-task transfer from classification-guided optimization to captioning and VQA degradation suggests that shared visual encoders create a transfer pathway that attackers can exploit without access to the downstream generative model.","Defenses for infrared VLMs may need to consider the spatial distribution of representational sensitivity across the entire image canvas, including boundary regions typically treated as low-importance background.","The QR-inspired structure raises the question of whether other recognizable visual priors (barcodes, fiducial markers, structured noise templates) could similarly serve as effective adversarial carriers in thermal imagery."],"fun_headline_variants":["QR-like edge patch crashes infrared CLIP accuracy to 0.70%","Structured edge patch degrades infrared vision-language models across tasks","Edge-placed patch breaks infrared CLIP and transfers to black-box VQA","Compact edge patch undermines infrared vision-language model robustness","Near-binary edge patch collapses infrared CLIP and degrades downstream VQA"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The paper assumes that a 300-image benchmark with 30 categories adequately represents infrared VLM robustness, and that a neutral gray probe patch can identify the optimal boundary location for the adversarial patch. If the benchmark is too narrow or the probe-based placement is suboptimal, the reported attack strength could be an underestimate — but the core vulnerability claim would still hold.","fun_headline_variants_meta":{"raw":{"variants":["QR-like edge patch crashes infrared CLIP accuracy to 0.70%","Structured edge patch degrades infrared vision-language models across tasks","Edge-placed patch breaks infrared CLIP and transfers to black-box VQA","Compact edge patch undermines infrared vision-language model robustness","Near-binary edge patch collapses infrared CLIP and degrades downstream VQA"]},"model":"glm-5.2","effort":"low","cost_usd":0.0,"raw_usage":{"total_tokens":697,"prompt_tokens":608,"completion_tokens":89,"prompt_tokens_details":null},"tokens_in":608,"tokens_out":89,"duration_ms":46968,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-09T15:06:45.539847+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If infrared VLMs that use visually different architectures (not CLIP-family encoders) prove resistant to the same edge-placed structured patches, or if the cross-task transfer to captioning and VQA fails to replicate on larger and more diverse infrared benchmarks, the claim of a general peripheral-perturbation vulnerability would be weakened.","supporting_citations":[],"review_version":1}