{"id":"f99745b5-277f-41fb-b838-ed0197fefd10","arxiv_id":"2507.04388","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"CoIBA shares a single learnable damping ratio across multiple bottleneck layers to produce vision transformer attributions that outperform single-layer IBA on faithfulness benchmarks.","lead":"Researchers extend the information bottleneck attribution method for vision transformers by inserting noise bottlenecks at multiple layers and sharing a single damping ratio across them. The resulting attribution maps score higher on faithfulness benchmarks such as insertion/deletion and ROAD.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The key bound in Eq. (7) depends on an unproven and false-in-general monotonicity of I[Zl-1;Zl]; when it fails, CoIBA's 'guarantee' of per-layer necessity does not follow from the simplified objective Eq. (8).","rationale":"The reader's weakest assumption is exactly the monotonicity behind Eq. (7), and I agree with that diagnosis. The paper's own Fig. 9(a) measures only the first-layer term I[Z1;R1], so it provides no empirical evidence for the per-layer sequence. The failure of Eq. (7) would not necessarily invalidate the empirical comparisons—CoIBA may still be a useful multi-layer IBA heuristic—but it would remove the central 'guarantee' that distinguishes the method from single-layer IBA. The experiments are extensive and include a sanity check, FunnyBirds ground truth, and multiple faithfulness metrics, which supports the empirical side. No new verdict is needed beyond the reader's CONDITIONAL: the paper should be accepted only if the monotonicity is proven under stated conditions or empirically verified with the proposed test; otherwise the theoretical claim must be weakened. I therefore leave the reader's verdict unchanged.","tokens_in":39087,"tokens_out":8328,"duration_ms":88859,"concrete_test":"Run the released CoIBA code on ViT-B-16/224 with the reported hyperparameters and compute, on 1,000 ImageNet validation images, the variational estimates û_l = E_{Z_{l−1}}[D_KL(P(Z_l|Z_{l−1}) || Q(Z_l))] from Eq. (6) for every targeted layer l = 1,...,9. If the sequence û_l is not non-increasing, or if any later-layer estimate exceeds û_1, Eq. (7) is violated on the trained model and Eq. (8) is not a valid upper bound for the layer-wise compression terms. As an analytical check, compute I[Z1;Z2] for the two-step chain X → Z1 = X, Z2 = 10Z1 + ε; it exceeds I[X;Z1], directly refuting the unqualified noise-injection monotonicity claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central theoretical claim—that CoIBA bounds the information discarded in every targeted layer—rests on Eq. (7): I[R1;Z1] ≥ (1/L) Σ_l I[Zl−1;Zl]. This reduction from Eq. (4) to Eq. (8) requires the sequence I[Zl−1;Zl] to be non-increasing, which Sec. 3.4 asserts from 'iterated noise-injection' without proof or empirical check. The assertion is not generally true. With Zl = λR′_l + (1−λ)ε_l and R′_l a deterministic function of Zl−1, adding independent noise increases H(Zl|Zl−1), but H(Zl) can also increase if the intervening transformer blocks amplify the signal; I[Zl−1;Zl] = H(Zl) − H(Zl|Zl−1) can therefore rise. A minimal counterexample: Z1 = X, R′_2 = 10Z1, Z2 = R′_2 + ε with small ε gives I[Z1;Z2] ≈ H(Z1)+log 10 > I[X;Z1]. ViT residual and LayerNorm blocks are not information-contractive, so nothing in the architecture enforces the needed ordering. Moreover, even if Eq. (7) held, minimizing βI[R1;Z1] bounds only the average of the layer-wise compression terms and does not individually bound each I[Zl−1;Zl], so the Abstract's 'guarantees ... in every targeted layer' is stronger than the inequality. The empirical results may remain valid as a heuristic, but the stated theoretical guarantee is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CoIBA, a feature attribution method for vision transformers that inserts information bottlenecks into multiple layers and shares a single per-token damping ratio across them. The central claim is that minimizing a variational upper bound on the first-layer bottleneck mutual information compresses the information in every targeted layer, so that low-scored attributions are provably unnecessary for the decision. The experimental section reports substantial improvements over IBA and other ViT attribution baselines on insertion/deletion, ROAD, FunnyBirds, sensitivity-N, and sanity checks across ViT, DeiT, Swin, CLIP-pretrained, and self-supervised models.","tokens_in":39459,"tokens_out":4299,"duration_ms":50228,"significance":"If the theoretical claim were established, CoIBA would be a notable contribution: it promises a single-forward-pass attribution map that reflects evidence distributed across many layers while retaining the information-bottleneck-style guarantee of IBA. The empirical study is unusually broad, spanning multiple architectures, pretraining strategies, datasets, and evaluation protocols, and the authors provide open-source code. The method is likely practically useful as a heuristic. However, the load-bearing theoretical reduction from Eq. (4) to Eq. (8) is not proven, and the statement that CoIBA guarantees per-layer necessity is stronger than the derived inequality. The paper therefore needs major work before its central claim can be accepted.","major_comments":[{"comment":"The reduction from the layer-wise objective (4) to the simplified objective (8) rests on the asserted monotonicity I[Zl; Zl+1] ≤ I[Zl−1; Zl]. This is stated as following from 'iterated noise-injection' but no proof or empirical check is provided. The assertion is not true in general: with Zl = λR'_l + (1−λ)ε_l and R'_l a deterministic function of Zl−1, adding noise increases the conditional entropy H(Zl|Zl−1), but H(Zl) may also increase if intervening transformer blocks amplify the signal, so I[Zl−1; Zl] = H(Zl) − H(Zl|Zl−1) can rise. A simple scaling example (e.g., R'_2 = c Z1 with c ≫ 1 and small noise) yields I[Z1; Z2] > I[R1; Z1]. Since ViT residual and LayerNorm blocks are not information-contractive, nothing in the architecture enforces the needed ordering. Thus Eq. (7), and consequently the claimed guarantee, is not established. The supplementary derivation in App. A bounds only a single-layer KL term and does not address monotonicity across layers.","section":"Sec. 3.4, Eq. (7)"},{"comment":"Even if Eq. (7) were true, it is an average bound: I[R1; Z1] ≥ (1/L) Σ_l I[Zl−1; Zl]. Minimizing β I[R1; Z1] bounds the sum of the layer-wise compression terms, not each term individually. The Abstract and Conclusion claim that CoIBA 'guarantees that the discarded activation is unnecessary in every targeted layer.' From the average bound one can only conclude that the average compression is controlled, not that every layer's discarded information is small. A per-layer upper bound of the form I[Zl−1; Zl] ≤ c I[R1; Z1] with a constant c is needed, and no such bound is derived. This gap is load-bearing because the paper's central theoretical selling point is per-layer relevance.","section":"Sec. 3.4, Eq. (7) and Abstract"},{"comment":"The hyperparameters s (departure layer) and e (arrival layer) are selected on the basis of insertion/deletion scores on the same datasets used for final evaluation (Table B, Fig. 8), and the final reported numbers come from the best-performing settings. The trade-off β is also chosen using validation-set behavior (Supp. Table G). This creates a selection-on-evaluation-metric risk: part of the reported advantage may reflect tuning to the benchmark rather than a genuine property of the method. The authors should either use a separate validation split for hyperparameter selection, report results for a prespecified default, or present the sensitivity of the headline tables to the chosen configuration. They should also report error bars or multiple seeds, since all tables are point estimates.","section":"Sec. 4.1 and Supp. C.2, Tables A and B"}],"minor_comments":[{"comment":"Use semicolon notation consistently in the mutual information terms: I[Rl; Zl] rather than I[Rl, Zl].","section":"Sec. 3.1, Eq. (2)"},{"comment":"The phrase 'demonstrate our insistence' should be 'demonstrate our claim' or 'demonstrate our motivation'.","section":"Sec. 3.2"},{"comment":"The arrow notation in the table captions is easy to misread. Please state explicitly, e.g., 'higher is better for insertion and LeRF; lower is better for deletion and MoRF.'","section":"Tables 1 and 2"},{"comment":"The term 'universal damping ratio' is used for a per-token vector shared across layers; 'shared' or 'common' would be less ambiguous than 'universal', which suggests a single scalar.","section":"Sec. 3.3"},{"comment":"There is a stray word 'CannonRacket' immediately after the Abstract; it appears to be an editing artifact and should be removed.","section":"Page 1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The useful part of this paper is the empirical package; the advertised guarantee is not. CoIBA extends IBA to several bottleneck layers at once, sharing one per-token damping ratio and adding a variational upper bound so you don't have to search per-layer betas. The experiments are genuinely broad—insertion/deletion, ROAD, FunnyBirds, difficulty-aware splits, sanity checks, and a long list of ViT/DeiT/Swin variants—and CoIBA beats the baselines by consistent, nontrivial margins. Code is released. That is a real contribution to the interpretability toolkit, and I'd rather see this line of work continue than ignore it.\n\nThe soft spot is the theory, and it is load-bearing. The reduction from Eq. (4) to Eq. (8) needs I[Z_{l-1};Zl] to be non-increasing in l. That is asserted in Sec. 3.4 as 'iterated noise-injection,' with no proof and no empirical check. It is not true in general: if the layer amplifies the signal, H(Zl) can grow faster than the conditional entropy term, so the mutual information can rise. The stress-test's counterexample (Z1=X, R2'=10Z1) is correct in spirit, and nothing in ViT's residual or LayerNorm structure forces contraction. So Eq. (7) does not follow. And even if it did, beta I[R1;Z1] only bounds the average of the layer-wise compression terms, not each one individually. The Abstract's 'guarantees in every targeted layer' is therefore stronger than anything the objective can deliver.\n\nA few smaller issues: beta, start/end layers are chosen using the same insertion/deletion metrics used for the headline results, no error bars are reported, and InputIBA is missing from the comparison. Those are secondary and fixable.\n\nNet: I'd treat CoIBA as a promising multi-layer IBA heuristic with solid empirical support, not as a method with a proven per-layer information guarantee. This is for practitioners who want better ViT attribution maps now; theorists will be frustrated by the unproven bound. The right venue is a peer-reviewed conference with a serious referee who asks for a proof or an empirical check of monotonicity, or a rewritten claim that drops 'guarantees.' I'd accept it for review, but the theoretical section needs major work before publication.","headline":"A solid empirical multi-layer IBA extension whose advertised per-layer guarantee rests on an unproved and generally false monotonicity assumption.","tokens_in":39942,"tokens_out":3035,"would_cite":true,"duration_ms":35686,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that attribution for vision transformers is more faithful when an information bottleneck compresses several layers at once with one shared damping ratio, guaranteeing that discarded activations are unnecessary in every…","keywords":["feature attribution","information bottleneck","vision transformer","interpretability","mutual information","variational bound","insertion/deletion","faithfulness"],"falsifier":"Estimate the mutual information I[Zl−1;Zl] for each consecutive pair of CoIBA's bottleneck layers on a trained ViT-B/16, using the paper's own KL upper bound or a separate estimator, and check whether the sequence is non-increasing. A single violation, where some later pair exceeds an earlier pair, refutes the key inequality and the guarantee that discarded activations are unnecessary in every targeted layer.","tokens_in":38895,"feed_emoji":"🔍","tokens_out":4920,"duration_ms":49256,"temperature":0.7,"pith_summary":"The paper tries to establish that attribution maps for vision transformers become more faithful when the information bottleneck is applied simultaneously to multiple layers rather than to one chosen layer, and that this can be done with a single shared per-token damping ratio. Existing single-layer information bottleneck attribution must be run once per layer, and no single layer dominates in explanation quality. CoIBA instead inserts bottlenecks into a range of layers and optimizes one universal damping ratio, so information needed at any targeted layer can compensate for over-compression in another. The central guarantee is that any activation discarded by the attribution is unnecessary in every targeted layer for the decision. If correct, this removes the heuristic search over layers while keeping computational cost comparable to a single information bottleneck run.","feed_headline":"A single shared ratio makes ViT attributions faithful across layers","feed_subtitle":"CoIBA compresses many transformer layers at once, so the explanation captures evidence no single-layer method can see.","key_machinery":"The central object is the universal damping ratio λ = sigmoid(α) ∈ [0,1]^{P×1}, shared across all L bottleneck-inserted layers, with bottleneck variables Zl = λR′l + (1−λ)εl. Because λ is per-patch and channel-uniform, it measures token importance rather than per-neuron importance. The argument rides on the variational upper bound: assuming the mutual-information sequence I[Zl−1;Zl] is non-increasing under iterated noise injection, the inequality I[R1;Z1] ≥ (1/L) Σ I[Zl−1;Zl] bounds the average layer-wise compression by the first-layer compression, so the simplified objective max I[ZL;Y] − βI[R1;Z1] bounds every layer's compression with a single β. This upper bound is what converts the multi-layer objective into a tractable single-term objective and underpins the guarantee that discarded activations are unnecessary in every targeted layer.","core_discovery":"CoIBA claims that the relevant evidence for a vision transformer's decision is distributed across layers, and that a single-layer information bottleneck necessarily discards part of that evidence. By inserting bottleneck layers into a contiguous range of layers and sharing one universal damping ratio (one value per image patch, uniform across channels), the method estimates comprehensive relevant information; the shared ratio lets deeper layers' retained information compensate for over-compression in earlier layers. The objective replaces the sum of layer-wise compression terms with a single upper bound, leaving only one trade-off hyperparameter. With this bound, CoIBA guarantees that every activation it discards is unnecessary in every targeted layer, and the reported experiments show improved insertion/deletion, ROAD, and FunnyBirds scores over IBA and propagation-based baselines across ViT, DeiT, and Swin models.","pith_inferences":["The monotonicity assumption could be tested directly; if it fails only mildly, the method might still work in practice but the theoretical guarantee would need to be restated as approximate.","The universal per-token damping ratio effectively defines a token-level importance score, so λ itself could be repurposed as a coarse attribution map or used to select layers for deeper analysis.","The same multi-layer sharing scheme could apply to other transformer-based predictors, such as text or multimodal models, where evidence also accumulates across layers.","CoIBA's channel-uniform perturbation suggests a design principle for patch-based transformers: token-level masking can be more faithful than channel-level masking in information-bottleneck attribution."],"forward_implications":["Attribution maps from CoIBA reflect evidence across a range of layers, so explanations no longer depend on a heuristic choice of a single bottleneck layer.","The variational upper bound removes the need to tune per-layer trade-off parameters; one universal damping ratio and one β suffice.","Because the compression term is bounded by first-layer information, the computation stays close to a single information bottleneck run despite compressing many layers.","If the guarantee holds, low-scored image regions can be treated as unnecessary in every targeted layer, strengthening the case for using the maps in failure diagnosis.","The reported insertion/deletion, ROAD, and FunnyBirds scores improve across ViT, DeiT, and Swin variants, including low-confidence (hard) samples."],"supporting_citations":[{"why":"Supplies the single-layer information bottleneck attribution method that CoIBA extends and compares against, including the bottleneck formulation and the per-layer guarantee.","marker":"[32]"},{"why":"Provides the deep variational information bottleneck framework and reparameterization trick used to make the mutual-information terms tractable.","marker":"[3]"},{"why":"Previous information-bottleneck attribution in the input domain, used to motivate the discussion of overestimation in earlier layers that CoIBA addresses.","marker":"[42]"},{"why":"FunnyBirds dataset with ground-truth part annotations, used to measure completeness, correctness, and contrastivity of attribution maps.","marker":"[19]"},{"why":"RISE insertion/deletion protocol, the primary evaluation of feature-importance faithfulness.","marker":"[28]"},{"why":"ROAD evaluation that addresses mask-shape leakage, used as a main correctness benchmark.","marker":"[30]"},{"why":"Chefer-LRP, a transformer interpretability baseline compared throughout the experiments.","marker":"[10]"},{"why":"The vision transformer architecture on which CoIBA is implemented and evaluated.","marker":"[14]"}],"fun_headline_variants":["Shared bottleneck ratio boosts ViT attribution faithfulness across layers","One ratio for all layers: CoIBA makes ViT explanations faithful","Cross-layer information bottleneck improves vision transformer attributions","CoIBA: sharing one ratio to capture all evidence in ViT explanations","Universal ratio ties ViT layers for faithful attribution maps"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The guarantee rests on the unproven claim that injecting noise repeatedly makes the mutual information between consecutive bottleneck representations non-increasing, I[Zl;Zl+1] ≤ I[Zl−1;Zl]; if that monotonicity fails, the inequality does not bound the layer-wise compression terms and the simplified objective no longer delivers the per-layer guarantee.","fun_headline_variants_meta":{"raw":{"variants":["Shared bottleneck ratio boosts ViT attribution faithfulness across layers","One ratio for all layers: CoIBA makes ViT explanations faithful","Cross-layer information bottleneck improves vision transformer attributions","CoIBA: sharing one ratio to capture all evidence in ViT explanations","Universal ratio ties ViT layers for faithful attribution maps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00068,"raw_usage":{"total_tokens":3067,"prompt_tokens":900,"completion_tokens":2167,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":516,"completion_tokens_details":{"reasoning_tokens":2095}},"tokens_in":516,"tokens_out":2167,"duration_ms":15913,"temperature":1.0,"reasoning_tokens":2095,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:48:30.608113+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Estimate the mutual information I[Zl−1;Zl] for each consecutive pair of CoIBA's bottleneck layers on a trained ViT-B/16, using the paper's own KL upper bound or a separate estimator, and check whether the sequence is non-increasing. A single violation, where some later pair exceeds an earlier pair, refutes the key inequality and the guarantee that discarded activations are unnecessary in every targeted layer.","supporting_citations":[{"cited_title":"Restricting the flow: Information bottlenecks for attri- bution","cited_arxiv_id":null,"evidence_quote":"Supplies the single-layer information bottleneck attribution method that CoIBA extends and compares against, including the bottleneck formulation and the per-layer guarantee."},{"cited_title":"Deep variational information bottleneck","cited_arxiv_id":null,"evidence_quote":"Provides the deep variational information bottleneck framework and reparameterization trick used to make the mutual-information terms tractable."},{"cited_title":"Fine-grained neural net- work explanation by identifying input features with predic- tive information","cited_arxiv_id":null,"evidence_quote":"Previous information-bottleneck attribution in the input domain, used to motivate the discussion of overestimation in earlier layers that CoIBA addresses."},{"cited_title":"Fun- nybirds: A synthetic vision dataset for a part-based analysis of explainable ai methods","cited_arxiv_id":null,"evidence_quote":"FunnyBirds dataset with ground-truth part annotations, used to measure completeness, correctness, and contrastivity of attribution maps."},{"cited_title":"Rise: Random- ized input sampling for explanation of black-box models","cited_arxiv_id":null,"evidence_quote":"RISE insertion/deletion protocol, the primary evaluation of feature-importance faithfulness."},{"cited_title":"A consistent and efficient eval- uation strategy for attribution methods","cited_arxiv_id":null,"evidence_quote":"ROAD evaluation that addresses mask-shape leakage, used as a main correctness benchmark."},{"cited_title":"Transformer inter- pretability beyond attention visualization","cited_arxiv_id":null,"evidence_quote":"Chefer-LRP, a transformer interpretability baseline compared throughout the experiments."},{"cited_title":"An image is worth 16x16 words: Transformers for image recognition at scale","cited_arxiv_id":null,"evidence_quote":"The vision transformer architecture on which CoIBA is implemented and evaluated."}],"review_version":1}