{"id":"f916eb63-2767-4861-a942-9b956179c063","arxiv_id":"2608.05983","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A single image perturbation trained on text-image pairs disrupts SAM3 concept segmentation, cutting average mask AP from 59.43 to 18.73 across five benchmarks.","lead":"This paper presents Universal Concept Disruption (UCD), an adversarial image perturbation that makes SAM3, an open-vocabulary image segmentation model, fail to find objects described in text prompts. It is the first attack designed for SAM3's concept-aware design, and it transfers across datasets and model versions.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"UCD's claimed consistent advantage over GRAT rests on 100-image subsets with no error bars; the PhraseCut gap is 0.57 mask AP and may be sampling noise.","rationale":"The central claim is empirical: a single 8/255 perturbation trained on 100 images outperforms all adapted baselines on five datasets. The most load-bearing assumption is that the reported 100-image, three-seed evaluation is stable enough to rank methods. The UCD-vs-GRAT margin is 0.57 AP on PhraseCut and 1.32 on LVIS; with no per-seed spread or confidence intervals, these margins are within plausible sampling noise. I do not see a fatal flaw in the loss design: Equations (5)-(11) are internally coherent, the ablations show each term contributes, and the transfer to SAM3.1 and to video without re-optimization are independent supporting signals. Equation (2) is less load-bearing than it may appear because Lscore is computed on the model's actual output scores, so even if the multiplicative form is approximate, backpropagation still suppresses whatever final scores the released model uses. The paper also does not release code or the trained perturbation, which compounds the difficulty of verifying the exact evaluation procedure. The reader already conditioned acceptance on reporting error bars or larger evaluation sets; this stress-test agrees with that condition rather than moving to a harsher verdict.","tokens_in":11727,"tokens_out":6479,"duration_ms":67305,"concrete_test":"Rerun the Table 1 comparison on ten independent 100-image subsets per dataset (or on the full validation splits) and report per-subset mask AP for UCD and GRAT plus the paired UCD-minus-GRAT difference with a bootstrap 95% confidence interval. If the PhraseCut difference (currently 0.57 AP favoring UCD) changes sign in any subset, or if the confidence interval includes zero, the 'consistently outperforms' claim is unsupported; if the difference stays positive and separated from zero across all repeats, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table 1's headline comparison is computed from three random 100-image subsets per dataset, and only the averages are reported: no per-seed values, standard deviations, or confidence intervals are given. The central claim that UCD 'consistently outperforms' all adapted baselines therefore depends on margins that are small relative to expected sampling noise on two of the five datasets. On PhraseCut, UCD achieves 9.44 mask AP versus GRAT's 10.01, a difference of 0.57 AP points; on LVIS, the difference is 1.32 points (30.51 vs. 31.83). Mask AP is an instance-level statistic over roughly one hundred images, so subset-to-subset variation of several AP points is typical; with only three repeats, a 0.57-point gap is not distinguishable from noise. Because all methods are evaluated on identical subsets, paired differences could mitigate this, but no paired statistics are reported. If the PhraseCut or LVIS ranking flips on independent subsets, the paper's 'consistently outperforms' claim fails even though the average drop from clean remains large. The concern is about the strength of the comparative evidence, not the existence of an attack effect.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Universal Concept Disruption (UCD), a universal adversarial perturbation for SAM3's open-vocabulary concept segmentation. UCD is trained on (image, noun-phrase) pairs with an untargeted objective that combines suppression of the presence-gated final concept score, mask area collapse, clean-mask Dice disruption, and divergence of prompt-shared image features. The method is evaluated on five datasets against SAM/SAM2-style and general universal-perturbation baselines under a matched protocol, with ablations of each loss component, transfer experiments to SAM3.1 and to SAM3 video inference, and a defense-oriented evaluation of prompt ensembling, head fine-tuning, and temporal filtering. The headline result is a reduction of average mask AP from 59.43 to 18.73 and average cgF1 from 50.32 to 20.49 across the five datasets.","tokens_in":11982,"tokens_out":6185,"duration_ms":57570,"significance":"If the comparative claim holds, UCD is the first attack specifically targeting SAM3's presence-gated concept segmentation, and the paper's identification of the presence token as a universal attack surface is a useful conceptual contribution. The experimental design has real strengths: baselines are optimized under the same budget and evaluation pipeline, the perturbation is evaluated on held-out test images disjoint from the training set, ablations isolate each loss term, and the transfer tests to SAM3.1 and video are performed without re-optimization. The defense-oriented evaluation is also honest about the clean-performance cost of the tested mitigations. There is no evidence of circularity: the loss does not reduce to the reported metrics by construction, and the baselines are optimized under the same budget. The main weakness is statistical: on two datasets the margin over the strongest baseline is small relative to expected sampling variation, and no error bars or paired statistics are reported.","major_comments":[{"comment":"The central comparative claim that UCD 'consistently outperforms' all baselines is not statistically supported as reported. Each evaluation uses three random 100-image subsets and only the averages are given; no per-seed values, standard deviations, or confidence intervals are reported. On PhraseCut the UCD/GRAT margin is 0.57 mask AP (9.44 vs 10.01), and on LVIS it is 1.32 points (30.51 vs 31.83); the LVIS cgF1 margin is only 0.31 points (23.57 vs 23.88). These gaps are small relative to expected subset-to-subset variation of instance-level AP over 100 images, so the ranking on these datasets is plausibly sampling noise. Because all methods are evaluated on identical subsets, paired differences or confidence intervals would address this concern; please report per-seed values together with paired statistics, or increase the evaluation subset size.","section":"Experimental Setup / Table 1"},{"comment":"The paper states that all baselines are adapted to the same universal-perturbation setting but does not describe how any baseline is adapted. For example, DarkSAM and UAP-SAM2 are originally designed for geometric-prompt or video settings, and it is unclear how they are converted to a single bounded image perturbation trained on text prompts. Without a precise description of each baseline's objective, prompt form, and training schedule, the claim of a 'matched evaluation protocol' cannot be verified. Please provide these details for every baseline, or release code.","section":"Comparison Study / Experimental Setup"},{"comment":"The output-level attack design rests on Eq. (2), which states that the final concept score is the product of the sigmoid of the query logit and the sigmoid of the presence logit. The paper describes this as a 'simplified image inference path,' but it does not verify that the released SAM3 model implements exactly this global multiplication. If the actual combination differs (for example, an additive or learned fusion), the presence-suppression and score-targeting terms may be attacking an abstraction rather than the real decision. Please verify Eq. (2) against the released code or via a direct activation probe, and report the result.","section":"Methodology, Eq. (2)"}],"minor_comments":[{"comment":"Only the average Box AP across five datasets is reported; per-dataset box AP values are absent and should be added for completeness.","section":"Table 1"},{"comment":"The paper should state explicitly whether the three random 100-image evaluation subsets are identical across all methods and whether the reported averages are computed over the same image sets, so that paired comparisons are meaningful.","section":"Experimental Setup"},{"comment":"The paper notes that cgF1 is not applicable when a sampled set contains only positive prompts, but it does not specify how negative prompts are sampled for SACo-Gold and LVIS or whether the same negative-prompt set is used for every method; please clarify.","section":"Experimental Setup / Datasets and Metrics"},{"comment":"Several passages in the provided manuscript text have missing spaces between words (for example, 'Inthispaper'), which should be corrected in the final version.","section":"Abstract and Introduction"}],"recommendation":"major_revision","confidential_remarks":"The main technical concern is statistical: the headline comparative claim needs error bars or paired statistics, and the baseline adaptation procedure needs to be documented. These are fixable within the manuscript's scope. I did not independently verify the existence of the many 2025/2026 preprints cited in the reference list."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The new thing here is real: UCD is the first attack designed for SAM3's presence-gated concept segmentation, and it treats the model as a concept-grounding system rather than a mask decoder. The multi-level loss (score suppression, area collapse, dice disruption, feature divergence) is a sensible decomposition, and the ablations show each term earns its place, with feature divergence doing the heavy lifting for transfer. The matched baseline protocol is a genuine effort at fairness, and the transfer results to SAM3.1 and to video inference are useful evidence that the attack surface is not an artifact of one checkpoint. The defense-oriented section is unusually honest: prompt ensembling and temporal filtering only appear to help by crushing clean performance, and head fine-tuning leaves a large residual drop even under a fixed attack.\n\nThe main soft spot is exactly what the stress-test note flags. The headline comparison rests on three random 100-image subsets per dataset, with only averages reported. On PhraseCut the UCD advantage over GRAT is 0.57 mask AP; on LVIS it is 1.32. Those margins are within normal subset-to-subset variation, and without paired statistics or confidence intervals the claim that UCD \"consistently outperforms\" all baselines is not supported on those two datasets. The average drop from clean (59.43 to 18.73) is so large that the attack clearly works; the uncertainty is only about whether UCD beats GRAT everywhere, not about the existence of a strong attack. The paper also does not release code or the trained perturbation, which makes exact reproduction impossible and amplifies the sampling concern. The method's reliance on Eq. (2) as the exact inference path is a real assumption, but the paper describes the released SAM3 that way, so I do not count it heavily against the work.\n\nWho should read this: anyone working on adversarial robustness of open-vocabulary or concept-based segmentation. The paper is a competent, well-structured empirical study with a novel target. It deserves a serious referee, but the referee should demand error bars or larger evaluation sets on the per-dataset comparisons and at least a promise of code release. I would not desk-reject it; I would send it to review with the sampling issue as the main requested revision.","headline":"First attack on SAM3's presence-gated concept segmentation, with solid ablations and honest defense analysis, but the evaluation sampling is too thin to support the claim of consistent superiority over GRAT on every dataset.","tokens_in":12490,"tokens_out":1432,"would_cite":true,"duration_ms":15841,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Universal perturbation drops SAM3 mask AP from 59.4 to 18.7","keywords":["adversarial attack","universal adversarial perturbation","SAM3","concept segmentation","presence token","open-vocabulary segmentation","transferability","mask AP"],"falsifier":"Re-run the matched protocol with, say, ten evaluation seeds per dataset and report per-seed ranges; if on PhraseCut the best GRAT seed beats the worst UCD seed in mask AP, or if the perturbation fails to transfer when a checkpoint's presence gate is computed per-candidate, the paper's central advantage claim would not hold.","tokens_in":11533,"feed_emoji":"🎯","tokens_out":4789,"duration_ms":41003,"temperature":0.7,"pith_summary":"The paper claims that SAM3's presence-gated concept segmentation has a new universal attack surface: one bounded image perturbation, trained on image–noun-phrase pairs, can make present concepts disappear or their masks become spatially useless across datasets. If true, this means SAM3's integrated concept-grounding design—unlike earlier promptable SAM versions—can be defeated by a single reusable perturbation without per-image or per-concept optimization. The authors support this with evaluations on five benchmarks, showing average mask AP falls from 59.43 to 18.73 and average concept-grounding F1 from 50.32 to 20.49, with the perturbation transferring to SAM3.1 and to video inference. The paper also tests three defenses and finds prompt ensembling, head fine-tuning, and temporal filtering offer only limited or costly recovery.","feed_headline":"Universal attack drops SAM3 mask AP from 59.4 to 18.7","feed_subtitle":"One 8/255 perturbation also transfers to SAM3.1 and video; defenses recover accuracy only by hurting clean segmentation.","key_machinery":"The central object is the presence-gated final concept score $s_i = \\sigma(\\ell_i)\\cdot\\sigma(p)$, where $\\ell_i$ is the query logit for candidate $i$ and $p$ is a prompt-level presence logit that gates all candidates. Because post-processing keeps only candidates above a threshold on this product, suppressing the presence signal alone can discard geometrically valid masks. UCD's machinery is a four-term objective—score suppression, mask area collapse, Dice disruption against clean masks, and negative feature divergence—optimized jointly into one $\\ell^\\infty$-bounded perturbation $\\delta$ via projected Adam.","core_discovery":"The central discovery is that SAM3's presence token—a prompt-level existence logit that globally multiplies every candidate's score—is the load-bearing weakness. UCD optimizes a single universal perturbation that simultaneously activates the same text-conditioned path used at inference by training on real image–noun-phrase pairs, maximizes divergence in the prompt-shared visual backbone to encourage cross-concept and cross-dataset transfer, and suppresses the final presence-gated scores while collapsing or displacing retained masks through area and Dice losses. The result is that a single 8/255-bounded perturbation, trained on 100 images, disrupts concept segmentation on five datasets more than adapted existing attacks, and it transfers to a newer SAM3 checkpoint and to video without retraining.","pith_inferences":["The presence-gated score product is a general design pattern in open-vocabulary segmentation; UCD's success suggests that any model multiplying per-candidate confidence by a global existence estimate inherits a similar single-point attack surface, so the finding may extend beyond SAM3.","Since feature divergence is the component whose removal most weakens the attack, prompt-shared backbone features may be the most transferable carrier; this hints that input-space universal perturbations trained to maximize feature divergence could generalize to other vision-language grounding models.","The defense results imply that robustification should target the presence decision itself, for example by making the presence gate less sensitive to small perturbations, rather than pruning low-confidence predictions, which trades away clean accuracy.","A testable extension: training UCD on negative prompts as well to activate absent concepts could create a combined attack that both erases present concepts and hallucinates absent ones; the paper explicitly leaves that threat model out."],"forward_implications":["A single reusable perturbation can degrade SAM3 across five datasets with different prompt types (categories, referring expressions, phrases, open-vocabulary), implying the attack surface is not specific to one benchmark or prompt style.","The same image-trained perturbation transfers to SAM3.1 without re-optimization, implying prompt-shared visual features and presence-gated scoring are stable across the SAM3 family.","Injecting the perturbation into only the first frame of a video propagates through SAM3's memory to later clean frames, with tail-frame drops up to 10.36 mask AP in the 14/1 setting.","Prompt ensembling reduces the measured attack drop only by also reducing clean performance; head fine-tuning preserves clean utility but UCD retrained against it still leaves performance below clean, implying lightweight defenses do not restore robustness."],"supporting_citations":[{"why":"Introduces SAM3 and its presence-gated concept segmentation, defining the attack target and the SACo training data.","marker":"Carion et al. 2025"},{"why":"Establishes the universal-perturbation setting that UCD adopts and that all baselines are adapted to.","marker":"Moosavi-Dezfooli et al. 2017"},{"why":"Provides UAP-SAM2, the SAM2-specific universal attack baseline that UCD must beat in the comparison.","marker":"Zhou et al. 2026"},{"why":"Contributes CPA, one of the strongest adapted baselines that UCD outperforms across datasets.","marker":"Long et al. 2025"},{"why":"Contributes GRAT, the closest baseline UCD outperforms across datasets in the matched protocol.","marker":"Xia et al. 2024"},{"why":"Provides UAPGD, the generic universal adversarial attack baseline adapted to SAM3.","marker":"Deng and Karam 2020"},{"why":"Defines the original SAM promptable segmentation paradigm that SAM/SAM2-style baselines build on.","marker":"Kirillov et al. 2023"},{"why":"Defines SAM2 video segmentation, the setting used for UCD's image-to-video transfer experiments.","marker":"Ravi et al. 2025"}],"fun_headline_variants":["Single universal attack wrecks SAM3 concept masks","UCD: one perturbation collapses SAM3 mask AP to 18.7","Universal concept disruption transfers to SAM3.1 and video","One 8/255 perturbation breaks SAM3 across five datasets","SAM3's presence token is its Achilles heel: UCD attack"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison assumes that three randomly drawn 100-image evaluation subsets per dataset give stable estimates of attack strength, and that SAM3's released inference really computes each final score as a global presence probability multiplied into every candidate.","fun_headline_variants_meta":{"raw":{"variants":["Single universal attack wrecks SAM3 concept masks","UCD: one perturbation collapses SAM3 mask AP to 18.7","Universal concept disruption transfers to SAM3.1 and video","One 8/255 perturbation breaks SAM3 across five datasets","SAM3's presence token is its Achilles heel: UCD attack"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00019,"raw_usage":{"total_tokens":1334,"prompt_tokens":935,"completion_tokens":399,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":551,"completion_tokens_details":{"reasoning_tokens":313}},"tokens_in":551,"tokens_out":399,"duration_ms":4646,"temperature":1.0,"reasoning_tokens":313,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T19:53:07.121564+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the matched protocol with, say, ten evaluation seeds per dataset and report per-seed ranges; if on PhraseCut the best GRAT seed beats the worst UCD seed in mask AP, or if the perturbation fails to transfer when a checkpoint's presence gate is computed per-candidate, the paper's central advantage claim would not hold.","supporting_citations":[],"review_version":1}