{"id":"46b9399d-2567-4c7b-b26c-0a7b5d47b220","arxiv_id":"2502.01467","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"An unfolding fusion network uses integrated-gradient attribution weights and attention maps from a co-trained segmenter to improve infrared-visible fusion and downstream semantic segmentation.","lead":"UAAFusion uses attribution analysis to let a semantic segmentation network guide how infrared and visible photos are fused, and reports top or near-top fusion and segmentation scores on three public benchmarks. The method is a candidate recipe for making task-guided fusion more transparent, though the attribution-specific benefit is not fully isolated.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Ablations report only fusion metrics, never segmentation mIoU; the attribution mechanism is never isolated from Lseg, so the claimed segmentation gain is not causally supported.","rationale":"The reader's weakest assumption is exactly the isolability of the attribution mechanism from Lseg. My analysis agrees: the ablation protocol in Tab. III is the natural place to establish this, but it only measures fusion quality. The paper's own Exp. VIII modifies Lseg but still reports only fusion metrics, so even that partial check is not connected to the headline segmentation result. The full-model mIoU of 64.55 is competitive, but without a segmentation ablation the central causal claim is unverified rather than disproven. I do not see an internal inconsistency that invalidates the method; the inaccuracy in the completeness claim around Eq. (12) is a minor theoretical blemish. The evidential gap is addressable with additional experiments, so the conditional verdict remains appropriate.","tokens_in":23405,"tokens_out":6942,"duration_ms":62265,"concrete_test":"Retrain the full model and the following ablations on FMB, then evaluate SegFormer mIoU on the FMB test set: (a) full model; (b) no attribution attention (Exp. I); (c) IG→Grad (Exp. II); (d) Lint with fixed w1=w2=0.5 (no attribution weighting); (e) Lseg removed entirely; (f) Lseg=CE(If) only (Exp. VIII). If (d) or (e) produce mIoU within ~1 point of the full model's 64.55, the attribution analysis is not the driver of the segmentation gain. At minimum, Tab. III should be extended with an mIoU column for every existing ablation row.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Sec. IV-F (Tab. III) ablates the attribution attention, the IG computation, the intensity loss, the gradient loss, the memory modules, and the composition of Lseg, but every row is evaluated only on fusion metrics (EN, SF, CC, VIF, Qabf, SSIM). The central claim—that attribution analysis makes fused images more useful for semantic segmentation—rests on the single full-model mIoU of 64.55 in Tab. II. No ablation reports mIoU, and no ablation removes the attribution-weighted intensity loss while keeping Lseg, nor removes Lseg entirely. Consequently, the observed segmentation improvement could be caused by the mere presence of the cross-entropy term Lseg (multi-task training with a segmentation network), by the auxiliary DeeplabV3+ providing a better feature space, or by the attribution attention, without the attribution weights w1,w2 (Eq. 8) playing any role. The attribution-specific contribution is therefore entangled with other components. A secondary gap: Eq. (8) requires per-class pixel sets (i,j)∈c, but the paper never states whether these are ground-truth labels or the segmentation network's predictions at inference; since w1,w2 are used inside each unfolding stage (Eqs. 3-4), test-time behavior is underspecified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes UAAFusion, a deep-unfolding network for infrared-visible image fusion in which semantic segmentation is intended to guide the fusion process through attribution analysis. The fused image is constructed by unfolding an optimization problem whose data-fidelity weights are per-class attribution weights computed from integrated gradients of a segmentation network, and each unfolding stage includes an attribution attention map. A memory augmentation module is added to improve cross-stage information flow. The method is trained jointly with an auxiliary DeeplabV3+ segmentation network using intensity, gradient, and segmentation losses. Experiments on FMB, MSRS, and RoadScene report competitive fusion metrics and the highest mean IoU (64.55) on the FMB semantic segmentation benchmark among the compared methods.","tokens_in":23664,"tokens_out":5443,"duration_ms":54233,"significance":"If the central claim is supported, the paper offers a useful way to inject downstream-task information into image fusion while retaining the interpretability of model-based unfolding. The manuscript has notable strengths: the code is released; comparisons cover three datasets and several recent fusion methods; and the parameter study in Sec. IV-G and the ablation matrix in Table III show a systematic exploration of the proposed components. The empirical segmentation result is the strongest point in favor of the method. However, the evidence that the attribution mechanism specifically causes the segmentation improvement is currently incomplete, and one of the formal claims about integrated gradients is inaccurate. These issues are fixable but require additional experiments and a revised derivation.","major_comments":[{"comment":"The text states that the approximate attribution map A(γ)_i satisfies the completeness condition Σ_i A(γ)_i = DS(I_f^(k)) − DS(I_f^(0)). Eq. (12) is a right-endpoint Riemann sum with k terms over the stage outputs I_f^(j), and DS is a deep network, not an affine function along the path. The equality holds for the exact integral or for a piecewise-linear network along the path, but neither condition is stated or justified. Please either prove the claimed equality under explicit conditions, or rephrase the claim as an approximation and remove the exactness statement.","section":"Sec. III-B2, Eq. (12)"},{"comment":"All ablation rows in Table III report only fusion metrics (EN, SF, CC, VIF, Qabf, SSIM); none report the downstream segmentation mIoU. Consequently, the central claim that attribution analysis improves fused-image usefulness for semantic segmentation is not causally supported. No ablation removes Lseg completely, and no ablation removes the attribution-weighted intensity loss while keeping Lseg; Exp. VIII only changes the composition of Lseg, and every row retains a segmentation loss. Please report segmentation mIoU for the ablation variants, and add ablations that isolate the attribution weights w1, w2 (for example, fixed equal weights with Lseg retained) and that remove Lseg entirely.","section":"Sec. IV-F, Table III"},{"comment":"The sets (i,j) ∈ c used in Eq. (8) are never defined operationally. During training, are these sets obtained from ground-truth semantic labels or from the segmentation network's predictions? At test time, ground truth is unavailable, yet w1 and w2 are used inside each unfolding stage through Eq. (3). The paper must specify the label source used to compute the attribution weights, state whether the same computation is used at training and inference, and describe any thresholding, since otherwise the test-time behavior of the method is underspecified.","section":"Sec. III-B1, Eq. (8)"}],"minor_comments":[{"comment":"Eq. (7) is a discrete Riemann approximation to the integral in Eq. (6), but the text does not explicitly say so; please state that M steps yield an approximation and report the step size convention.","section":"Sec. III-B1, Eqs. (6)-(7)"},{"comment":"The path function in Eq. (11) is defined for l ∈ {0, 1, ..., k}, but for l = k the interval is [1, (k+1)/k] and I_f^(k+1) is not defined; the range should be l ∈ {0, 1, ..., k−1}.","section":"Sec. III-B2, Eq. (11)"},{"comment":"The text says UAAFusion 'consistently outperforms' the compared methods, but Table I shows several metrics where it is not the best, including VIF on FMB and MSRS and SSIM on RoadScene; please soften the claim to reflect the actual rankings.","section":"Sec. IV-B and Table I"},{"comment":"In the ConvLSTM equations, the symbol ⊙ is used for both element-wise multiplication and the convolution of the weights with inputs; please distinguish the convolution operator from element-wise multiplication to avoid ambiguity.","section":"Sec. III-D, Eq. (15)"},{"comment":"The notation DS is introduced without definition; it should be defined as the relevant segmentation network score function and connected to the Score notation used in Eqs. (5) and (9).","section":"Sec. III-B2, Eq. (12)"}],"recommendation":"major_revision","confidential_remarks":"The paper's main empirical result and code release are positive, but the causal attribution claim needs stronger evidence. In particular, the ablations in Sec. IV-F must report segmentation mIoU and must isolate the attribution weights from the mere presence of the segmentation loss. The completeness claim in Eq. (12) also needs correction. These are load-bearing but addressable within the manuscript's scope, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is worth a look: using integrated gradients over the output of each unfolding stage to derive per-class fusion weights and an attention map, then using those to guide the fusion network. That combination is new, as far as the cited work goes, and it is a genuine attempt to make the fusion process interpretable with respect to a downstream task.\n\nWhat the paper does well: the path function in Eq. (11)-(12) is a reasonable adaptation of IG to the unfolding setting, and the memory augmentation across stages is a sensible addition. The experiments cover three datasets with nine baselines, the code is public, and Fig. 3 shows the attribution weights behaving as intended—infrared gets higher weight for people, visible for vehicles. The writing is clear, and the related work is adequate.\n\nThe most serious gap is the ablation design. Table III changes components but only reports fusion metrics. The one number that supports the paper's reason to exist—segmentation mIoU—is only given for the full model. We never see whether removing the attribution weights w1,w2 (while keeping Lseg) drops mIoU. Without that, the segmentation gain could come from the cross-entropy term Lseg alone, or from the auxiliary DeeplabV3+ providing a better feature space. The attribution-specific mechanism is entangled with other components. This is the load-bearing flaw holding back a clean acceptance.\n\nThere are two smaller issues. First, Eq. (12) is a right-endpoint Riemann sum; the claim that it satisfies the IG completeness condition is only exact in the limit or for linear DS. It is a minor theoretical error since the attention map is just a spatial weight, but it should be corrected. Second, Eq. (8) uses per-class pixel sets (i,j)∈c, and the paper never states whether these are ground-truth labels or the segmentation network's predictions at inference. If ground truth is used at test time, that is a leak. If predictions are used, the authors should say so and show results under that protocol.\n\nI also note that the mIoU gain over the next best method is 1.55 points, and no error bars or significance tests are reported. That could be within run-to-run noise. Still, these are addressable issues. The architecture is coherent, the method is plausible, and the empirical results are competitive. This paper deserves a serious referee, but the referee should push for the missing mIoU ablation and the Eq. (8)/(12) clarifications.\n\nWho is this for? People working on infrared-visible fusion and task-guided fusion will get value from the attribution-path idea and the attention mechanism. It is not a field-reorganizing result, but it is a solid refinement with a novel component. My recommendation: send it to peer review, and require a revised version that isolates the attribution contribution.","headline":"Novel attribution-guided fusion network, but the central claim that attribution drives segmentation gain is not isolated from the segmentation loss in the ablations.","tokens_in":24193,"tokens_out":3443,"would_cite":true,"duration_ms":31604,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Attribution analysis lets semantic segmentation steer image fusion, and the result tops FMB segmentation among compared methods.","keywords":["infrared-visible image fusion","semantic segmentation","attribution analysis","integrated gradients","algorithm unfolding","attribution attention","memory augmentation","deep unfolding network"],"falsifier":"Train a control network with the same architecture and losses but set the attribution weights to constants, say $w_1=w_2=0.5$, and replace the attribution attention map with a constant map while keeping the segmentation loss $L_{seg}$; if the control's mean IoU on FMB matches 64.55, the attribution mechanism is not what drives the reported improvement.","tokens_in":23220,"feed_emoji":"🎯","tokens_out":3997,"duration_ms":33884,"temperature":0.7,"pith_summary":"The paper sets out to make multi-modal image fusion serve a downstream semantic segmentation task, rather than only produce visually pleasing images. Its claim is that attribution analysis—measuring which pixels of the infrared and visible inputs drive the segmentation network's decisions—can dynamically steer the fusion process. The fused images should then retain the source information that segmentation actually uses, improving both fusion quality and segmentation accuracy. The authors report that their UAAFusion network reaches the highest mean IoU (64.55) among compared methods on the FMB benchmark while staying competitive on standard fusion metrics across FMB, MSRS, and RoadScene.","feed_headline":"Attribution-guided fusion tops segmentation benchmark","feed_subtitle":"UAAFusion uses attribution maps from a segmentation network to steer infrared-visible fusion, beating nine baselines on FMB.","key_machinery":"The load-bearing objects are the attribution-derived fusion weights $w_1$, $w_2$ (Eq. 8), class-wise normalized integrated-gradient scores measuring each source image's contribution to segmentation, and the attribution attention map $A$ (Eq. 12), an integrated-gradient map over the fused image computed along a path that interpolates between adjacent unfolding-stage outputs. The path function (Eq. 11) is the device that adapts attribution analysis to fusion, since a fused image has two references rather than one. The unfolding network, built by unrolling a gradient-projection iteration into $K$ stages, supplies the intermediate fused images that define this path, and a memory augmentation module carries short- and long-term features across stages. The same weights $w_1$ and $w_2$ appear both in the network update and in the intensity loss, so the segmentation signal enters twice: once as a loss and once as a spatial attention mechanism.","core_discovery":"UAAFusion claims that semantic segmentation can provide direct, dynamic guidance to fusion through attribution analysis, instead of merely contributing a fixed auxiliary loss. The segmentation network produces class-wise attribution scores for the two source images, computed with integrated gradients along a path from a zero image, and these scores are normalized into per-class fusion weights $w_1$ and $w_2$. These weights control how much each source contributes to the fused image. At each unfolding stage, a second attribution computation over the fused image, using a path built from the stage outputs, yields an attribution attention map $A$ that emphasizes pixels and regions the segmentation relies on. With these components, the fused image is optimized to be faithful to the source images where they are useful and to concentrate detail where it matters for segmentation. Empirically, the method reports the best mean IoU (64.55) on FMB and best or second-best fusion metrics on FMB, MSRS, and RoadScene.","pith_inferences":["The attribution-specific contribution is not fully isolated: the paper's ablations never remove the segmentation loss $L_{seg}$, so part of the reported gain may come from training the fusion network with cross-entropy rather than from attribution weighting.","A stronger test would compare against a variant with equal weights $w_1=w_2=0.5$ and no attribution attention while keeping $L_{seg}$; if mean IoU does not drop, attribution analysis is not the active ingredient.","The path function in Eq. (11) is a natural target for theoretical analysis: as unfolding stages converge, the attribution map approaches the integrated gradient along the full trajectory, linking fusion to standard integrated-gradient axioms.","Applying the same design to object detection or depth estimation would test whether attribution steering transfers across downstream tasks."],"forward_implications":["Fusion methods can be designed around downstream task usefulness rather than only visual metrics, with downstream improvement measured directly.","The attribution weights provide a per-class, per-pixel account of which modality contributed to each region, making fusion decisions more interpretable.","The unfolding structure ties the network architecture to an explicit optimization objective, so each network stage corresponds to an iteration of a solvable model.","The same attribution-steering design could extend to other dense prediction tasks by computing attribution against the relevant task network."],"supporting_citations":[{"why":"Integrated gradients, the attribution method used for both the class-wise weights and the attention map.","marker":"[43]"},{"why":"Algorithm unfolding from fast sparse coding, the basis for converting iterative optimization steps into network stages.","marker":"[44]"},{"why":"The FMB benchmark and dataset supplying training pairs and the segmentation evaluation protocol.","marker":"[22]"},{"why":"The semantic-supervised dual-discriminator idea from which the class-wise weighting of source images is adapted.","marker":"[31]"},{"why":"Gradient-weighted class activation mapping for semantic segmentation, used to define per-class attribution scores.","marker":"[36]"},{"why":"Memory-augmented deep unfolding, the source of the ConvLSTM short- and long-term memory module.","marker":"[49]"},{"why":"Prior algorithm-unfolding fusion work that motivates the gradient-projection unfolding for infrared-visible fusion.","marker":"[69]"},{"why":"Path consistency condition that the attribution attention map is designed to satisfy.","marker":"[93]"},{"why":"DeeplabV3+, the auxiliary segmentation network used during fusion training.","marker":"[94]"}],"fun_headline_variants":["Attribution-guided unfolding fusion tops FMB benchmark","Segmentation-driven fusion via attribution analysis","UAAFusion: attribution maps direct fusion for segmentation","Deep unfolding fusion improves with attribution guidance"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central assumption is that the observed segmentation gains come from the attribution mechanism itself, not merely from training the fusion network with a segmentation loss; the paper's ablations never remove $L_{seg}$ or the attribution-weighted intensity loss, so the attribution-specific contribution is not isolated.","fun_headline_variants_meta":{"raw":{"variants":["Attribution-guided unfolding fusion tops FMB benchmark","Segmentation-driven fusion via attribution analysis","UAAFusion: attribution maps direct fusion for segmentation","Deep unfolding fusion improves with attribution guidance"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000362,"raw_usage":{"total_tokens":1979,"prompt_tokens":998,"completion_tokens":981,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":614,"completion_tokens_details":{"reasoning_tokens":925}},"tokens_in":614,"tokens_out":981,"duration_ms":8355,"temperature":1.0,"reasoning_tokens":925,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T15:11:41.701062+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a control network with the same architecture and losses but set the attribution weights to constants, say $w_1=w_2=0.5$, and replace the attribution attention map with a constant map while keeping the segmentation loss $L_{seg}$; if the control's mean IoU on FMB matches 64.55, the attribution mechanism is not what drives the reported improvement.","supporting_citations":[{"cited_title":"Axiomatic attribution for deep networks,","cited_arxiv_id":null,"evidence_quote":"Integrated gradients, the attribution method used for both the class-wise weights and the attention map."},{"cited_title":"Learning fast approximations of sparse coding,","cited_arxiv_id":null,"evidence_quote":"Algorithm unfolding from fast sparse coding, the basis for converting iterative optimization steps into network stages."},{"cited_title":"Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and Segmentation","cited_arxiv_id":"2308.02097","evidence_quote":"The FMB benchmark and dataset supplying training pairs and the segmentation evaluation protocol."},{"cited_title":"Semantic-supervised infrared and visible image fusion via a dual-discriminator generative adversarial network,","cited_arxiv_id":null,"evidence_quote":"The semantic-supervised dual-discriminator idea from which the class-wise weighting of source images is adapted."},{"cited_title":"Towards interpretable semantic segmentation via gradient-weighted class activation mapping (student abstract),","cited_arxiv_id":null,"evidence_quote":"Gradient-weighted class activation mapping for semantic segmentation, used to define per-class attribution scores."},{"cited_title":"Memory- augmented deep unfolding network for guided image super-resolution,","cited_arxiv_id":null,"evidence_quote":"Memory-augmented deep unfolding, the source of the ConvLSTM short- and long-term memory module."},{"cited_title":"Efficient and model-based infrared and visible image fusion via algorithm unrolling,","cited_arxiv_id":null,"evidence_quote":"Prior algorithm-unfolding fusion work that motivates the gradient-projection unfolding for infrared-visible fusion."},{"cited_title":"Paths and consistency in additive cost sharing,","cited_arxiv_id":null,"evidence_quote":"Path consistency condition that the attribution attention map is designed to satisfy."},{"cited_title":"Encoder- decoder with atrous separable convolution for semantic image segmen- tation,","cited_arxiv_id":null,"evidence_quote":"DeeplabV3+, the auxiliary segmentation network used during fusion training."}],"review_version":1}