{"id":"573dbaa8-3a97-46ad-b9c6-178d66b6a24b","arxiv_id":"2607.09164","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"low","formal_verification":"none","parameter_count":5,"one_line_summary":"SEAMS optimizes a compact soft mask so a frozen model’s class score, CLS embedding, or tokens stay nearly the same when non-mask pixels are replaced by self-generated distractor and blur.","lead":"SEAMS finds the smallest set of image pixels that still keep a frozen vision model’s chosen output almost unchanged. It is a practical post-hoc saliency tool that works across ViTs and ConvNets without extra datasets or architecture-specific tricks.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The three-way self-composite may leak low-frequency context, so low preservation MSE need not mean the retained pixels alone are sufficient.","rationale":"The Reader correctly isolates the composite-fill premise as the weakest assumption. The paper’s own Supp. F quantifies the gap and acknowledges that grey-fill metrics are conservative lower bounds precisely because the masks were trained with blur context present. That admission does not invalidate the method, but it does make the strongest claim (“sufficient pixels”) conditional on a fill strategy that is never fully ablated against a pure information-removal baseline during training. No internal contradiction appears in the reported numbers, and the modular-target and cross-architecture results remain interesting; the contribution is still accept-shaped once the fill dependence is either closed by the grey-fill re-optimisation test or more carefully scoped in the claims. Hence the verdict stays CONDITIONAL and agreement with the Reader is full.","tokens_in":19604,"tokens_out":594,"duration_ms":7755,"concrete_test":"Re-optimise the same ImageNet subset with the training composite forced to pure grey (or pure black) fill for non-salient pixels, keeping all other hyperparameters fixed; recompute Table 1 ins/del AUCs and the ascending-deletion AUCs of Table 2. If the new masks remain compact and the AUCs stay within ~5% of the published numbers, the leakage concern is largely dispelled; a large drop or collapse of sparsity would confirm that blur/distractor context was doing essential work.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that optimising a soft mask under the three-way composite of Eq. (1) (M⋅x + M''⋅x_dist + N⋅x_blur, with M''=min(1-M,α)) yields compact regions that are truly sufficient for the frozen target g. That claim rests on the premise that low MSE under this composite equals sufficiency of the retained pixels. Supp. F and Table 5 show a large train/eval gap: grey-fill cos/p_masked is substantially lower than blur-fill (e.g., CLS cos 0.740\to0.918; spaniel p 0.34\to0.93). The paper itself notes that masks rely on the blur fill’s low-frequency context. Consequently, the reported insertion/deletion and pixel-deletion curves (which use grey or mean fill) may overstate how much of the model behaviour is carried by the high-M pixels alone; the optimisation can exploit residual structure that the evaluation protocol later removes. This is the same soft spot the Reader flagged, and it is load-bearing for the sufficiency interpretation.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes SEAMS, a post-hoc sufficiency saliency method that optimises a continuous soft mask so a frozen differentiable model output (class probability, CLS embedding, or dense tokens) is preserved under a three-way composite built only from the query image (original content, self-augmented distractor, and heavy blur). Masks are parameterised by unconstrained logits transformed via softplus, mean-normalisation, and clipping with a learnable budget α, and trained with an MSE preservation loss plus an α penalty, without auxiliary distractor datasets or differentiable top-k. Experiments on ViT and ConvNeXt families report competitive insertion/deletion AUCs, strong ascending pixel-deletion faithfulness versus DAVE/LeGrad and gradient baselines, multi-target explanations from one pipeline, low cross-architecture mask agreement at similar fidelity, and a qualitative transfer demo on private ROP fundus images.","tokens_in":20034,"tokens_out":1474,"duration_ms":29914,"significance":"If the sufficiency interpretation is made rigorous, the work is a useful and practical contribution to visual explanation: a single, architecture-agnostic optimisation loop that switches among object-level, class-conditioned, and token-level targets, needs no external reference set, and yields compact nearly-binary masks competitive on standard faithfulness benchmarks. The cross-architecture finding—that different backbones can preserve similar outputs with largely disjoint pixel sets—is a concrete, falsifiable observation that sensitivity methods do not naturally surface. Strengths include modular targets, a simple normalise-and-clip sparsity mechanism, insertion/deletion and ascending-deletion results that beat strong baselines (Tables 1–2, Fig. 4), and an explicit train/eval fill analysis in Supp. F. The main scientific value is therefore conditional on clarifying what “sufficient” means under the self-composite versus pure information removal.","major_comments":[{"comment":"§3 Composite image / Eq. (1) and Supp. F / Table 5: the central sufficiency claim is that low preservation MSE under the three-way self-composite means the retained pixels are enough for model behaviour. Supp. F shows a large train/eval gap when non-salient regions are grey-filled instead of blur-filled (e.g. mean CLS cos 0.740→0.918; spaniel p_masked 0.34→0.93). The paper itself states that masks rely on the blur fill’s low-frequency context. Because the optimisation objective (Eq. 2) always trains against blur+distractor, absolute “sufficiency” of high-M pixels alone is not established by the training loss. Main-text claims and the title should either (i) report primary preservation metrics under grey/mean fill as the default, (ii) include an ablation that optimises under grey fill, or (iii) reframe results as composite-conditioned sufficiency with the gap quantified in the main paper—","section":null},{"comment":"§4 Faithfulness / Table 1 vs. pixel-deletion protocol (Fig. 4, Table 2): insertion/deletion and ascending deletion use grey or channel-mean replacement and still favour SEAMS, which supports ranking quality. However, SEAMS masks for the deletion benchmark are optimised with fewer steps (T=500) and n_copies=3 median aggregation, while main qualitative/ins-del settings use T=2000 and different targets. The paper should state whether the same mask generation protocol is used for all faithfulness tables, and report ins/del AUCs under a fixed, fully specified protocol (including fill, T, and aggregation) so that the large ascending-deletion gains cannot be attributed to protocol mismatch with baselines.","section":null},{"comment":"Abstract and §4 claim masks are “stable across random initialisations,” but the main text and reported tables give no quantitative stability metric (e.g. IoU/cos across seeds, variance of α or of deletion AUC). Either add a short multi-seed study on a fixed image set or soften the claim. Without that evidence the stability statement is unsupported relative to the rest of the evaluation.","section":null},{"comment":"§4 Transfer to medical imaging / Fig. 7 (and Supp. C): the ROP demo is only qualitative on private data with no insertion/deletion, preservation MSE, or clinical agreement metric, and no public subset or synthetic proxy. As written it cannot support a generalisation claim beyond “the optimiser runs and produces sparse maps.” Either add quantitative faithfulness on the ROP model (even on a held-out private split summarised by AUC/cos) or move the section to a brief qualitative appendix and avoid implying validated clinical transfer in the abstract/conclusions.","section":null}],"minor_comments":[{"comment":"Main text cites “Figure 19” for qualitative comparisons before later figures (e.g. Fig. 5); figure numbering appears out of order relative to the supplemental block. Renumber so main-text figures are sequential and self-contained.","section":null},{"comment":"§3: M'' = min(1−M, α) is easy to misread as a second soft mask rather than a capped residual. A one-sentence intuition (why the cap equals α) would help readers who skip the partition-of-unity algebra.","section":null},{"comment":"Hyperparameters λ_α=2.0, α_0=0.25, T=2000, D=4 are fixed for all main experiments; a short sensitivity plot (e.g. λ_α ∈ {0.5,2,8} on a small image set) would strengthen the “simple fixed pipeline” claim.","section":null},{"comment":"Related work: Vision DiffMask and Extremal Perturbations are cited; a clearer one-paragraph contrast on modular targets and self-only composites would help position novelty without overstating uniqueness of soft-mask optimisation.","section":null},{"comment":"Typos/notation: “desribed” (Supp. G), inconsistent spacing in “Fig. 1.Sufficiency”, and mixed use of g vs f for the encoder/target in places (e.g. Algorithm 1 uses f while §3 uses g).","section":null},{"comment":"Limitations correctly note iterative cost (~40s/image); stating wall-clock and GPU model in the main experimental setup (not only Supp. A) would aid reproducibility comparisons with single-pass attribution.","section":null}],"recommendation":"major_revision","confidential_remarks":"The technical core is solid enough for a methods venue after revision; the load-bearing issue is interpretive overclaim of pure pixel sufficiency under a blur-leaking composite, not a broken algorithm. Private ROP results should not be allowed to carry generalisation weight without numbers. Fit is appropriate for a vision/XAI journal if the sufficiency framing is tightened."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"This is a solid methods paper in a crowded XAI lane. What is actually new is not the idea of sufficiency or continuous masks (SIS, Meaningful/Extremal Perturbations, Vision DiffMask, I-GOS already exist), but a clean, modular pipeline: normalise-and-clip soft masks with a learnable budget, a query-only three-way composite (original + heavy self-aug distractor + blur), and the ability to swap the preserved target (class prob, CLS, tokens) without changing anything else. No auxiliary distractor set, no top-k relaxation, no architecture-specific attribution. That engineering package is real and usable.\n\nThey do the work well. Insertion/deletion gaps are large (Table 1). Ascending pixel-deletion curves beat DAVE and LeGrad on ViT-B/16 and DeiT-III-B/16 (Table 2, Fig. 4), with the near-flat early curve that you expect from a sparsity-constrained method. Cross-architecture IoU is low while fidelity stays comparable (Table 3, Fig. 6)—that architecture-dependence observation is useful and hard to get from pure sensitivity maps. Multi-target panels and the multi-instance contrast with DAVE are clear. Math is elementary and honest; citations cover the right prior art. Supp. F even reports the grey-vs-blur fill gap instead of hiding it.\n\nThe soft spot the stress-test flags is real and load-bearing for the strongest sufficiency language. Optimisation trains against blur+distractor fill; grey-fill cos/p drops a lot (e.g. CLS 0.74\to0.92, spaniel 0.34\to0.93). So low MSE under Eq. (1) does not strictly mean “these pixels alone are enough.” The insertion/deletion and ascending-deletion numbers still use grey/mean fill and remain competitive, so the ranking evidence is not empty—but the interpretation should be “sufficient under this composite,” not pure pixel sufficiency. Other caveats are ordinary: iterative cost, private ROP data, no code release, free hyperparameters (λ_α, α0, T, D, blur/HeavyAug). None of those invent a contradiction.\n\nWho it is for: people who build or debug post-hoc visual explanations and want a single pipeline that can target embeddings as well as logits. It deserves a serious referee. I would engage with it, cite the modular setup and the architecture-dependence result if I am writing in this area, and push for code plus a clearer statement of the fill dependence.","headline":"Clean modular sufficiency saliency with competitive deletion numbers; the three-way self-composite is a real soft spot for the sufficiency claim, but not a collapse of the paper.","tokens_in":20603,"tokens_out":620,"would_cite":true,"duration_ms":7976,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Saliency maps that keep only the pixels needed to preserve a model's output beat sensitivity maps on faithfulness.","keywords":["sufficiency saliency","soft-mask optimisation","preservation MSE","visual explanation","Vision Transformers","insertion-deletion","model-agnostic"],"falsifier":"Re-evaluate the same optimised masks with a pure grey (or pure noise) fill that removes all non-mask context: if preservation fidelity collapses while insertion/deletion rankings stay high only under the training fill, the sufficiency claim fails for that composite.","tokens_in":20549,"feed_emoji":"🖼️","tokens_out":564,"duration_ms":4780,"temperature":0.7,"pith_summary":"Most visual explanations measure how sensitive a model is to tiny input changes. This paper argues that the more useful question is which pixels are enough to keep the model’s behaviour the same. SEAMS finds a compact soft mask by optimising so that a frozen model output—class probability, CLS embedding, or patch tokens—stays close under a three-way composite built only from the query image: kept pixels, a self-augmented distractor, and heavy blur. The same pipeline, with no architecture-specific attribution rules or external distractor sets, produces object-level, class-conditioned, and token-level maps. The masks are sparse, competitive on insertion/deletion and ascending pixel-deletion tests, transfer to medical fundus images, and show that different backbones often select different sufficient pixels while matching preservation fidelity.","feed_headline":"Which pixels are enough to keep a model’s answer?","feed_subtitle":"SEAMS finds compact soft masks that preserve class scores, embeddings, or tokens without extra data.","key_machinery":"SEAMS: unconstrained logits turned into a soft mask by softplus, mean-normalisation with learnable budget α, and clip; the mask drives a three-way composite (original, heavyAug distractor, blur) whose augmented forward pass is matched in MSE to the fixed original target.","core_discovery":"A single soft-mask optimisation that minimises MSE preservation of a chosen differentiable model output, under a learnable budget and a three-way composite made only from the query image, yields compact sufficiency saliency that is competitive on faithfulness benchmarks and modular across targets and architectures.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["SEAMS: soft masks find the fewest pixels that keep model outputs intact","Which pixels suffice? Soft-mask MSE preserves class scores and tokens","Compact soft masks from the query image alone preserve model behaviour","SEAMS optimises sufficiency saliency under a learnable pixel budget","Architecture-dependent evidence revealed by MSE-preserving soft masks"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"That low preservation error under the paper’s self-made blur-and-distractor fill really means the kept pixels are enough for the model, rather than the fill still leaking usable context.","fun_headline_variants_meta":{"raw":{"variants":["SEAMS: soft masks find the fewest pixels that keep model outputs intact","Which pixels suffice? Soft-mask MSE preserves class scores and tokens","Compact soft masks from the query image alone preserve model behaviour","SEAMS optimises sufficiency saliency under a learnable pixel budget","Architecture-dependent evidence revealed by MSE-preserving soft masks"]},"model":"grok-4.5","effort":"low","cost_usd":0.005168,"raw_usage":{"total_tokens":1384,"prompt_tokens":736,"num_sources_used":0,"completion_tokens":91,"cost_in_usd_ticks":51680000,"prompt_tokens_details":{"text_tokens":736,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":557,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":736,"tokens_out":91,"duration_ms":4325,"temperature":1.0,"reasoning_tokens":557,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-13T04:58:28.119988+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Re-evaluate the same optimised masks with a pure grey (or pure noise) fill that removes all non-mask context: if preservation fidelity collapses while insertion/deletion rankings stay high only under the training fill, the sufficiency claim fails for that composite.","supporting_citations":[],"review_version":1}