Pith. sign in

REVIEW 3 major objections 5 minor 101 references

Deep Unfolding Multi-modal Image Fusion Network via Attribution Analysis

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Pith's one-line read Attribution analysis lets semantic segmentation steer image fusion, and the result tops FMB segmentation among compared methods.

desk verdict Novel attribution-guided fusion network, but the central claim that attribution drives segmentation gain is not isolated from the segmentation loss in the ablations. read the letter →

arxiv 2502.01467 v1 pith:BS7LBANN submitted 2025-02-03 cs.CV

classification cs.CV
keywords infrared-visibleimagefusionsemanticsegmentationattributionanalysisintegratedgradientsalgorithmunfoldingattentionmemoryaugmentationdeepnetwork
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to make multi-modal image fusion serve a downstream semantic segmentation task, rather than only produce visually pleasing images. Its claim is that attribution analysis—measuring which pixels of the infrared and visible inputs drive the segmentation network's decisions—can dynamically steer the fusion process. The fused images should then retain the source information that segmentation actually uses, improving both fusion quality and segmentation accuracy. The authors report that their UAAFusion network reaches the highest mean IoU (64.55) among compared methods on the FMB benchmark while staying competitive on standard fusion metrics across FMB, MSRS, and RoadScene.

What carries the argument

The load-bearing objects are the attribution-derived fusion weights $w_1$, $w_2$ (Eq. 8), class-wise normalized integrated-gradient scores measuring each source image's contribution to segmentation, and the attribution attention map $A$ (Eq. 12), an integrated-gradient map over the fused image computed along a path that interpolates between adjacent unfolding-stage outputs. The path function (Eq. 11) is the device that adapts attribution analysis to fusion, since a fused image has two references rather than one. The unfolding network, built by unrolling a gradient-projection iteration into $K$ stages, supplies the intermediate fused images that define this path, and a memory augmentation module carries short- and long-term features across stages. The same weights $w_1$ and $w_2$ appear both in the network update and in the intensity loss, so the segmentation signal enters twice: once as a loss and once as a spatial attention mechanism.

What would settle it

Train a control network with the same architecture and losses but set the attribution weights to constants, say $w_1=w_2=0.5$, and replace the attribution attention map with a constant map while keeping the segmentation loss $L_{seg}$; if the control's mean IoU on FMB matches 64.55, the attribution mechanism is not what drives the reported improvement.

Watch

Extended reading notes

Core claim

UAAFusion claims that semantic segmentation can provide direct, dynamic guidance to fusion through attribution analysis, instead of merely contributing a fixed auxiliary loss. The segmentation network produces class-wise attribution scores for the two source images, computed with integrated gradients along a path from a zero image, and these scores are normalized into per-class fusion weights $w_1$ and $w_2$. These weights control how much each source contributes to the fused image. At each unfolding stage, a second attribution computation over the fused image, using a path built from the stage outputs, yields an attribution attention map $A$ that emphasizes pixels and regions the segmentation relies on. With these components, the fused image is optimized to be faithful to the source images where they are useful and to concentrate detail where it matters for segmentation. Empirically, the method reports the best mean IoU (64.55) on FMB and best or second-best fusion metrics on FMB, MSRS, and RoadScene.

Load-bearing premise

The central assumption is that the observed segmentation gains come from the attribution mechanism itself, not merely from training the fusion network with a segmentation loss; the paper's ablations never remove $L_{seg}$ or the attribution-weighted intensity loss, so the attribution-specific contribution is not isolated.

Editorial extensions

If this is right

  • Fusion methods can be designed around downstream task usefulness rather than only visual metrics, with downstream improvement measured directly.
  • The attribution weights provide a per-class, per-pixel account of which modality contributed to each region, making fusion decisions more interpretable.
  • The unfolding structure ties the network architecture to an explicit optimization objective, so each network stage corresponds to an iteration of a solvable model.
  • The same attribution-steering design could extend to other dense prediction tasks by computing attribution against the relevant task network.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The attribution-specific contribution is not fully isolated: the paper's ablations never remove the segmentation loss $L_{seg}$, so part of the reported gain may come from training the fusion network with cross-entropy rather than from attribution weighting.
  • A stronger test would compare against a variant with equal weights $w_1=w_2=0.5$ and no attribution attention while keeping $L_{seg}$; if mean IoU does not drop, attribution analysis is not the active ingredient.
  • The path function in Eq. (11) is a natural target for theoretical analysis: as unfolding stages converge, the attribution map approaches the integrated gradient along the full trajectory, linking fusion to standard integrated-gradient axioms.
  • Applying the same design to object detection or depth estimation would test whether attribution steering transfers across downstream tasks.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes UAAFusion, a deep-unfolding network for infrared-visible image fusion in which semantic segmentation is intended to guide the fusion process through attribution analysis. The fused image is constructed by unfolding an optimization problem whose data-fidelity weights are per-class attribution weights computed from integrated gradients of a segmentation network, and each unfolding stage includes an attribution attention map. A memory augmentation module is added to improve cross-stage information flow. The method is trained jointly with an auxiliary DeeplabV3+ segmentation network using intensity, gradient, and segmentation losses. Experiments on FMB, MSRS, and RoadScene report competitive fusion metrics and the highest mean IoU (64.55) on the FMB semantic segmentation benchmark among the compared methods.

Significance. If the central claim is supported, the paper offers a useful way to inject downstream-task information into image fusion while retaining the interpretability of model-based unfolding. The manuscript has notable strengths: the code is released; comparisons cover three datasets and several recent fusion methods; and the parameter study in Sec. IV-G and the ablation matrix in Table III show a systematic exploration of the proposed components. The empirical segmentation result is the strongest point in favor of the method. However, the evidence that the attribution mechanism specifically causes the segmentation improvement is currently incomplete, and one of the formal claims about integrated gradients is inaccurate. These issues are fixable but require additional experiments and a revised derivation.

major comments (3)
  1. [Sec. III-B2, Eq. (12)] The text states that the approximate attribution map A(γ)_i satisfies the completeness condition Σ_i A(γ)_i = DS(I_f^(k)) − DS(I_f^(0)). Eq. (12) is a right-endpoint Riemann sum with k terms over the stage outputs I_f^(j), and DS is a deep network, not an affine function along the path. The equality holds for the exact integral or for a piecewise-linear network along the path, but neither condition is stated or justified. Please either prove the claimed equality under explicit conditions, or rephrase the claim as an approximation and remove the exactness statement.
  2. [Sec. IV-F, Table III] All ablation rows in Table III report only fusion metrics (EN, SF, CC, VIF, Qabf, SSIM); none report the downstream segmentation mIoU. Consequently, the central claim that attribution analysis improves fused-image usefulness for semantic segmentation is not causally supported. No ablation removes Lseg completely, and no ablation removes the attribution-weighted intensity loss while keeping Lseg; Exp. VIII only changes the composition of Lseg, and every row retains a segmentation loss. Please report segmentation mIoU for the ablation variants, and add ablations that isolate the attribution weights w1, w2 (for example, fixed equal weights with Lseg retained) and that remove Lseg entirely.
  3. [Sec. III-B1, Eq. (8)] The sets (i,j) ∈ c used in Eq. (8) are never defined operationally. During training, are these sets obtained from ground-truth semantic labels or from the segmentation network's predictions? At test time, ground truth is unavailable, yet w1 and w2 are used inside each unfolding stage through Eq. (3). The paper must specify the label source used to compute the attribution weights, state whether the same computation is used at training and inference, and describe any thresholding, since otherwise the test-time behavior of the method is underspecified.
minor comments (5)
  1. [Sec. III-B1, Eqs. (6)-(7)] Eq. (7) is a discrete Riemann approximation to the integral in Eq. (6), but the text does not explicitly say so; please state that M steps yield an approximation and report the step size convention.
  2. [Sec. III-B2, Eq. (11)] The path function in Eq. (11) is defined for l ∈ {0, 1, ..., k}, but for l = k the interval is [1, (k+1)/k] and I_f^(k+1) is not defined; the range should be l ∈ {0, 1, ..., k−1}.
  3. [Sec. IV-B and Table I] The text says UAAFusion 'consistently outperforms' the compared methods, but Table I shows several metrics where it is not the best, including VIF on FMB and MSRS and SSIM on RoadScene; please soften the claim to reflect the actual rankings.
  4. [Sec. III-D, Eq. (15)] In the ConvLSTM equations, the symbol ⊙ is used for both element-wise multiplication and the convolution of the weights with inputs; please distinguish the convolution operator from element-wise multiplication to avoid ambiguity.
  5. [Sec. III-B2, Eq. (12)] The notation DS is introduced without definition; it should be defined as the relevant segmentation network score function and connected to the Score notation used in Eqs. (5) and (9).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: attribution weights are computed, not fitted, and the central segmentation claim is tested with a separately trained SegFormer.

full rationale

The paper's derivation chain is self-contained. The attribution weights w1, w2 in Eq. (8) are computed by integrated-gradients scoring through an auxiliary segmentation network (Eqs. (5)-(7)); they are not fitted parameters that encode the FMB mIoU or the evaluation benchmark. The fusion objective in Eq. (1) and the loss in Eq. (17) use these weights, but the downstream claim is assessed in Sec. IV-E by retraining a separate SegFormer on the fused images, so no target result is fed back into the derivation. The path function in Eqs. (11)-(12) is a construction from the unfolding stages and satisfies the integrated-gradients completeness condition; it is not imported from an author-specific uniqueness or ansatz theorem. Self-citations such as [69] and [71] are ordinary algorithm-unfolding technique citations and are not load-bearing for the central attribution claim. The ablation study in Sec. IV-F does not isolate the attribution weights from the Lseg term and reports only fusion metrics, but that is an experimental identification gap, not an equation-level circularity; under the hard rules, no specific reduction of a prediction to its inputs can be quoted, so the appropriate finding is no circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim relies on several domain assumptions about attribution transfer and path choice, plus a set of manually chosen hyperparameters. No new physical or mathematical entities are introduced; the attribution path function is a methodological construct rather than an invented entity.

free parameters (5)
  • lambda (Lgrad weight) = 1
    Chosen by preliminary optimization studies; balances gradient loss in Eq. (16), directly affects texture preservation.
  • mu (Lseg weight) = 0.1
    Chosen by preliminary optimization; controls how strongly segmentation supervision influences fusion training.
  • number of unfolding stages K = 5
    Selected from a parameter study (Tab. IV); K=5 gives best performance, though the variations across K=4 to K=8 are small.
  • sampling steps M for attribution weights = 5
    Set in Sec. IV-A for computing Eq. (7); the paper does not report a sensitivity study for this value.
  • initial step size rho_k = 0.01
    Initialization for the learnable step size in Eq. (2), chosen without a reported sensitivity analysis.
assumptions (5)
  • standard math Integrated-gradients completeness: the sum of attribution equals the difference in network output between baseline and input.
    Invoked in Eqs. (6), (10), and (12) following Sundararajan et al. [43]; the paper relies on this property to interpret attribution weights and attention maps.
  • domain assumption Gradients of segmentation class scores with respect to input pixels indicate which pixels are beneficial for segmentation.
    Used in Eqs. (5)-(8) to define w1 and w2; if this mapping is unreliable, the attribution-weighted fusion objective would not guide fusion toward segmentation-friendly content.
  • domain assumption Attribution signals from the co-trained DeeplabV3+ segmenter transfer to the independently trained SegFormer benchmark.
    Attribution weights and attention maps are computed from DeeplabV3+ (Sec. III-B), while downstream evaluation uses a separately trained SegFormer (Sec. IV-E); this transfer is assumed, not demonstrated.
  • domain assumption The chosen linear paths (zero-to-source, and stage-to-stage fused outputs) fairly represent the image transition for integrated gradients.
    Attribution weights and attention maps depend on the path in Eqs. (6) and (11); a different path could change the weights and the resulting fused images.
  • domain assumption A CNN can represent the proximal operator prox_h in Eq. (2), so the unfolding network genuinely minimizes Eq. (1).
    The proximal operator is learned from data rather than derived; the link between the network and the optimization objective is architectural, not exact.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deep Unfolding Multi-modal Image Fusion Network via Attribution Analysis." pith.science (2026). https://pith.science/paper/BS7LBANN

@misc{pith2026250201467,
  author       = {Pith},
  title        = {Pith review of: Deep Unfolding Multi-modal Image Fusion Network via Attribution Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BS7LBANN}},
  note         = {Machine review of arXiv:2502.01467}
}
read the original abstract

Multi-modal image fusion synthesizes information from multiple sources into a single image, facilitating downstream tasks such as semantic segmentation. Current approaches primarily focus on acquiring informative fusion images at the visual display stratum through intricate mappings. Although some approaches attempt to jointly optimize image fusion and downstream tasks, these efforts often lack direct guidance or interaction, serving only to assist with a predefined fusion loss. To address this, we propose an ``Unfolding Attribution Analysis Fusion network'' (UAAFusion), using attribution analysis to tailor fused images more effectively for semantic segmentation, enhancing the interaction between the fusion and segmentation. Specifically, we utilize attribution analysis techniques to explore the contributions of semantic regions in the source images to task discrimination. At the same time, our fusion algorithm incorporates more beneficial features from the source images, thereby allowing the segmentation to guide the fusion process. Our method constructs a model-driven unfolding network that uses optimization objectives derived from attribution analysis, with an attribution fusion loss calculated from the current state of the segmentation network. We also develop a new pathway function for attribution analysis, specifically tailored to the fusion tasks in our unfolding network. An attribution attention mechanism is integrated at each network stage, allowing the fusion network to prioritize areas and pixels crucial for high-level recognition tasks. Additionally, to mitigate the information loss in traditional unfolding networks, a memory augmentation module is incorporated into our network to improve the information flow across various network layers. Extensive experiments demonstrate our method's superiority in image fusion and applicability to semantic segmentation.

Figures

Figures reproduced from arXiv: 2502.01467 by the authors.

Figure 1
Figure 1. Illustration of UAAFusion. UAAFusion consists of multiple stages, each stage contains three parts, [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Comparison of fusion results of different methods. The cases in the figure are “00122” and “00633” in FMB dataset, “00706N” in MSRS dataset [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Visualization of attribution weights for ”00524”, “00703” and [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Visualization of the attribution attention map for the total attention and some semantic classes. The cases in the figure are “00122” and “00997” in [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Visual comparison of semantic segmentation task for “01046” and “01439” in FMB dataset. [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

101 extracted references · 72 canonical work pages

  1. [1]

    Datfuse: Infrared and visible image fusion via dual attention transformer,

    W. Tang, F. He, Y . Liu, Y . Duan, and T. Si, “Datfuse: Infrared and visible image fusion via dual attention transformer,” IEEE Transactions on Circuits and Systems for Video Technology , vol. 33, no. 7, pp. 3159–3172, 2023

  2. [2]

    Drone-based rgb-infrared cross-modality vehicle detection via uncertainty-aware learning,

    Y . Sun, B. Cao, P. Zhu, and Q. Hu, “Drone-based rgb-infrared cross-modality vehicle detection via uncertainty-aware learning,” IEEE Transactions on Circuits and Systems for Video Technology , vol. 32, no. 10, pp. 6700–6713, 2022

  3. [3]

    Cross-modal transformers for infrared and visible image fusion,

    S. Park, A. G. Vien, and C. Lee, “Cross-modal transformers for infrared and visible image fusion,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 34, no. 2, pp. 770–785, 2023

  4. [4]

    Learning a deep multi-scale feature ensemble and an edge-attention guidance for image fusion,

    J. Liu, X. Fan, J. Jiang, R. Liu, and Z. Luo, “Learning a deep multi-scale feature ensemble and an edge-attention guidance for image fusion,”IEEE Transactions on Circuits and Systems for Video Technology , vol. 32, no. 1, pp. 105–119, 2021

  5. [5]

    Tufusion: A transformer-based universal fusion algorithm for multimodal images,

    Y . Zhao, Q. Zheng, P. Zhu, X. Zhang, and W. Ma, “Tufusion: A transformer-based universal fusion algorithm for multimodal images,” IEEE Transactions on Circuits and Systems for Video Technology , 2023

  6. [6]

    Ccafusion: cross-modal coordinate attention network for infrared and visible image fusion,

    X. Li, Y . Li, H. Chen, Y . Peng, and P. Pan, “Ccafusion: cross-modal coordinate attention network for infrared and visible image fusion,” IEEE Transactions on Circuits and Systems for Video Technology , 2023

  7. [7]

    Cddfuse: Correlation-driven dual-branch feature decomposition for multi-modality image fusion,

    Z. Zhao, H. Bai, J. Zhang, Y . Zhang, S. Xu, Z. Lin, R. Timofte, and L. Van Gool, “Cddfuse: Correlation-driven dual-branch feature decomposition for multi-modality image fusion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Computer Vision Foundation / IEEE, June 2023, pp. 5906– 5916

  8. [8]

    Infrared and visible image fusion via texture conditional generative adversarial network,

    Y . Yang, J. Liu, S. Huang, W. Wan, W. Wen, and J. Guan, “Infrared and visible image fusion via texture conditional generative adversarial network,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 12, pp. 4771–4783, 2021

Show all 101 references
  1. [9]

    A task-guided, implicitly- searched and metainitialized deep model for image fusion,

    R. Liu, Z. Liu, J. Liu, X. Fan, and Z. Luo, “A task-guided, implicitly- searched and metainitialized deep model for image fusion,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024

  2. [10]

    Paif: Perception- aware infrared-visible image fusion for attack-tolerant semantic seg- mentation,

    Z. Liu, J. Liu, B. Zhang, L. Ma, X. Fan, and R. Liu, “Paif: Perception- aware infrared-visible image fusion for attack-tolerant semantic seg- mentation,” ACM MM, 2023

  3. [11]

    Infrared and visible image fusion methods and applications: A survey,

    J. Ma, Y . Ma, and C. Li, “Infrared and visible image fusion methods and applications: A survey,” Information Fusion, vol. 45, pp. 153–178, 2019

  4. [12]

    A new multi-focus image fusion algorithm and its efficient implementation,

    S. Liu, J. Chen, and S. Rahardja, “A new multi-focus image fusion algorithm and its efficient implementation,” IEEE Transactions on Circuits and Systems for Video Technology , vol. 30, no. 5, pp. 1374– 1384, 2020

  5. [13]

    Dynamic instance normalization for arbitrary style transfer,

    Y . Jing, X. Liu, Y . Ding, X. Wang, E. Ding, M. Song, and S. Wen, “Dynamic instance normalization for arbitrary style transfer,” in Pro- ceedings of the AAAI conference on artificial intelligence (AAAI) , 2020, pp. 4369–4376

  6. [14]

    Towards reducing severe defocus spread effects for multi-focus image fusion via an optimization based strategy,

    S. Xu, L. Ji, Z. Wang, P. Li, K. Sun, C. Zhang, and J. Zhang, “Towards reducing severe defocus spread effects for multi-focus image fusion via an optimization based strategy,” IEEE Transactions Computational Imaging, vol. 6, pp. 1561–1570, 2020

  7. [15]

    HAM-MFN: hyperspectral and multispectral image multiscale fusion network with RAP loss,

    S. Xu, O. Amira, J. Liu, C. Zhang, J. Zhang, and G. Li, “HAM-MFN: hyperspectral and multispectral image multiscale fusion network with RAP loss,” IEEE Transactions on Geoscience and Remote Sensing , vol. 58, no. 7, pp. 4618–4628, 2020

  8. [16]

    Ivfusenet: Fusion of infrared and visible light images for depth prediction,

    Y . Li, H. Zhao, Z. Hu, Q. Wang, and Y . Chen, “Ivfusenet: Fusion of infrared and visible light images for depth prediction,” Information Fusion, vol. 58, pp. 1–12, 2020

  9. [17]

    Regularized restoration using image fusion for digital auto-focusing,

    V . Maik, D. Cho, J. Shin, and J. K. Paik, “Regularized restoration using image fusion for digital auto-focusing,” IEEE Transactions on Circuits and Systems for Video Technology , vol. 17, no. 10, pp. 1360–1369, 2007

  10. [18]

    Infrared and visible image fusion via gradient transfer and total variation minimization,

    J. Ma, C. Chen, C. Li, and J. Huang, “Infrared and visible image fusion via gradient transfer and total variation minimization,” Information Fusion, vol. 31, pp. 100–109, 2016

  11. [19]

    Ar in VR: simulating infrared augmented vision,

    F. Lahoud and S. S ¨usstrunk, “Ar in VR: simulating infrared augmented vision,” in IEEE International Conference on Image Processing (ICIP) . IEEE, 2018, pp. 3893–3897. 13

  12. [20]

    Mfnet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes,

    Q. Ha, K. Watanabe, T. Karasawa, Y . Ushiku, and T. Harada, “Mfnet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes,” in IROS. IEEE, 2017, pp. 5108–5115

  13. [21]

    Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network,

    L. Tang, J. Yuan, and J. Ma, “Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network,” Information Fusion, vol. 82, pp. 28–42, 2022

  14. [22]

    Multi-interactive feature learning and a full-time multi- modality benchmark for image fusion and segmentation,

    J. Liu, Z. Liu, G. Wu, L. Ma, R. Liu, W. Zhong, Z. Luo, and X. Fan, “Multi-interactive feature learning and a full-time multi- modality benchmark for image fusion and segmentation,” CoRR, vol. abs/2308.02097, 2023

  15. [23]

    Object segmentation- assisted inter prediction for versatile video coding,

    Z. Li, Z. Yuan, L. Li, D. Liu, X. Tang, and F. Wu, “Object segmentation- assisted inter prediction for versatile video coding,” arXiv preprint arXiv:2403.11694, 2024

  16. [24]

    Pedestrian detection with unsupervised multispectral feature learning using deep neural networks,

    Y . Cao, D. Guan, W. Huang, J. Yang, Y . Cao, and Y . Qiao, “Pedestrian detection with unsupervised multispectral feature learning using deep neural networks,” Information Fusion, vol. 46, pp. 206–217, 2019

  17. [25]

    Target-aware dual adversarial learning and a multi-scenario multi- modality benchmark to fuse infrared and visible for object detection,

    J. Liu, X. Fan, Z. Huang, G. Wu, R. Liu, W. Zhong, and Z. Luo, “Target-aware dual adversarial learning and a multi-scenario multi- modality benchmark to fuse infrared and visible for object detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Re...

  18. [26]

    Cross-modal ranking with soft consistency and noisy labels for robust RGB-T tracking,

    C. Li, C. Zhu, Y . Huang, J. Tang, and L. Wang, “Cross-modal ranking with soft consistency and noisy labels for robust RGB-T tracking,” in ECCV (13) , ser. Lecture Notes in Computer Science, vol. 11217. Springer, 2018, pp. 831–847

  19. [27]

    Cross- modality person re-identification with shared-specific feature transfer,

    Y . Lu, Y . Wu, B. Liu, T. Zhang, B. Li, Q. Chu, and N. Yu, “Cross- modality person re-identification with shared-specific feature transfer,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Computer Vision Foundation / IEEE, 2020, ...

  20. [28]

    Task-driven super res- olution: Object detection in low-resolution images,

    M. Haris, G. Shakhnarovich, and N. Ukita, “Task-driven super res- olution: Object detection in low-resolution images,” in ICONIP (5) , ser. Communications in Computer and Information Science, vol. 1516. Springer, 2021, pp. 387–395

  21. [29]

    Does haze removal help cnn-based image classification?

    Y . Pei, Y . Huang, Q. Zou, Y . Lu, and S. Wang, “Does haze removal help cnn-based image classification?” in ECCV (10), ser. Lecture Notes in Computer Science, vol. 11214. Springer, 2018, pp. 697–712

  22. [30]

    Single image deraining: A comprehensive benchmark analysis,

    S. Li, I. B. Araujo, W. Ren, Z. Wang, E. K. Tokuda, R. H. Junior, R. M. C. Junior, J. Zhang, X. Guo, and X. Cao, “Single image deraining: A comprehensive benchmark analysis,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Computer...

  23. [31]

    Semantic-supervised infrared and visible image fusion via a dual-discriminator generative adversarial network,

    H. Zhou, W. Wu, Y . Zhang, J. Ma, and H. Ling, “Semantic-supervised infrared and visible image fusion via a dual-discriminator generative adversarial network,” IEEE Transactions on Multimedia , vol. 25, pp. 635–648, 2023

  24. [32]

    Rfn-nest: An end-to-end residual fusion network for infrared and visible images,

    H. Li, X. Wu, and J. Kittler, “Rfn-nest: An end-to-end residual fusion network for infrared and visible images,” Information Fusion, vol. 73, pp. 72–86, 2021

  25. [33]

    Densefuse: A fusion approach to infrared and visible images,

    H. Li and X. Wu, “Densefuse: A fusion approach to infrared and visible images,” IEEE Transactions on Image Processing , vol. 28, no. 5, pp. 2614–2623, 2019

  26. [34]

    IFCNN: A general image fusion framework based on convolutional neural network,

    Y . Zhang, Y . Liu, P. Sun, H. Yan, X. Zhao, and L. Zhang, “IFCNN: A general image fusion framework based on convolutional neural network,” Information Fusion, vol. 54, pp. 99–118, 2020

  27. [35]

    Metafusion: Infrared and visible image fusion via meta-feature embedding from object detection,

    W. Zhao, S. Xie, F. Zhao, Y . He, and H. Lu, “Metafusion: Infrared and visible image fusion via meta-feature embedding from object detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 13 955–13 965

  28. [36]

    Towards interpretable semantic segmentation via gradient-weighted class activation mapping (student abstract),

    K. Vinogradova, A. Dibrov, and G. Myers, “Towards interpretable semantic segmentation via gradient-weighted class activation mapping (student abstract),” in Proceedings of the AAAI conference on artificial intelligence (AAAI), 2020, pp. 13 943–13 944

  29. [37]

    Explaining YOLO: leveraging grad-cam to explain object detections,

    A. Kirchknopf, D. Slijepcevic, I. Wunderlich, M. Breiter, J. Traxler, and M. Zeppelzauer, “Explaining YOLO: leveraging grad-cam to explain object detections,” CoRR, vol. abs/2211.12108, 2022

  30. [38]

    Interpreting the latent space of gans for semantic face editing,

    Y . Shen, J. Gu, X. Tang, and B. Zhou, “Interpreting the latent space of gans for semantic face editing,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Computer Vision Foundation / IEEE, 2020, pp. 9240–9249

  31. [39]

    GAN dissection: Visualizing and understanding generative adversarial networks,

    D. Bau, J. Zhu, H. Strobelt, B. Zhou, J. B. Tenenbaum, W. T. Freeman, and A. Torralba, “GAN dissection: Visualizing and understanding generative adversarial networks,” in Proceedings of theInternational Conference on Learning Representations (ICLR) . OpenReview.net, 2019

  32. [40]

    Deep inside convolu- tional networks: Visualising image classification models and saliency maps,

    K. Simonyan, A. Vedaldi, and A. Zisserman, “Deep inside convolu- tional networks: Visualising image classification models and saliency maps,” in Proceedings of the International Conference on Learning Representations (ICLR) Workshop, 2014

  33. [41]

    Striving for simplicity: The all convolutional net,

    J. T. Springenberg, A. Dosovitskiy, T. Brox, and M. A. Riedmiller, “Striving for simplicity: The all convolutional net,” in Proceedings of the International Conference on Learning Representations (ICLR) Workshop, 2015

  34. [42]

    Visualizing and understanding convo- lutional networks,

    M. D. Zeiler and R. Fergus, “Visualizing and understanding convo- lutional networks,” in Proceedings of the European Conference on Computer Vision (ECCV) , ser. Lecture Notes in Computer Science, vol

  35. [43]

    Axiomatic attribution for deep networks,

    M. Sundararajan, A. Taly, and Q. Yan, “Axiomatic attribution for deep networks,” in ICML, ser. Proceedings of the European Conference on Computer Vision (ECCV), vol. 70. PMLR, 2017, pp. 3319–3328

  36. [44]

    Learning fast approximations of sparse coding,

    K. Gregor and Y . LeCun, “Learning fast approximations of sparse coding,” in Proceedings of the International Conference on Machine Learning (ICML) , J. F ¨urnkranz and T. Joachims, Eds. Omnipress, 2010, pp. 399–406

  37. [45]

    Admm-csnet: A deep learning approach for image compressive sensing,

    Y . Yang, J. Sun, H. Li, and Z. Xu, “Admm-csnet: A deep learning approach for image compressive sensing,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 42, no. 3, pp. 521–538, 2020

  38. [46]

    Efficient and interpretable deep blind image deblurring via algorithm unrolling,

    Y . Li, M. Tofighi, J. Geng, V . Monga, and Y . C. Eldar, “Efficient and interpretable deep blind image deblurring via algorithm unrolling,” IEEE Transactions on Image Processing , vol. 6, pp. 666–681, 2020

  39. [47]

    Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing,

    V . Monga, Y . Li, and Y . C. Eldar, “Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing,” IEEE Signal Processing Magazine, vol. 38, no. 2, pp. 18–44, 2021

  40. [48]

    Model meets deep learning in image inverse problems,

    N. Wang and J. Sun, “Model meets deep learning in image inverse problems,” Learning, vol. 2, no. 9, p. 10, 2020

  41. [49]

    Memory- augmented deep unfolding network for guided image super-resolution,

    M. Zhou, K. Yan, J. Pan, W. Ren, Q. Xie, and X. Cao, “Memory- augmented deep unfolding network for guided image super-resolution,” International Journal of Computer Vision , vol. 131, no. 1, pp. 215–242, 2023

  42. [50]

    Memory-augmented deep unfolding network for compressive sensing,

    J. Song, B. Chen, and J. Zhang, “Memory-augmented deep unfolding network for compressive sensing,” in Proceedings of the ACM Inter- national Conference on Multimedia (ACM MM) . ACM, 2021, pp. 4249–4258

  43. [51]

    Region level based multi-focus image fusion using quaternion wavelet and normalized cut,

    Y . Liu, J. Jin, Q. Wang, Y . Shen, and X. Dong, “Region level based multi-focus image fusion using quaternion wavelet and normalized cut,” Signal Processing, vol. 97, pp. 9–30, 2014

  44. [52]

    Structure tensor and nonsubsampled shearlet transform based algorithm for CT and MRI image fusion,

    X. Liu, W. Mei, and H. Du, “Structure tensor and nonsubsampled shearlet transform based algorithm for CT and MRI image fusion,” Neurocomputing, vol. 235, pp. 131–139, 2017

  45. [53]

    Infrared and visible image fusion based on target-enhanced multiscale transform decomposition,

    J. Chen, X. Li, L. Luo, X. Mei, and J. Ma, “Infrared and visible image fusion based on target-enhanced multiscale transform decomposition,” Information Sciences, vol. 508, pp. 64–78, 2020

  46. [54]

    Mdlatlrr: A novel decomposition method for infrared and visible image fusion,

    H. Li, X. Wu, and J. Kittler, “Mdlatlrr: A novel decomposition method for infrared and visible image fusion,” IEEE Transactions on Image Processing, vol. 29, pp. 4733–4746, 2020

  47. [55]

    Image fusion with convolutional sparse representation,

    Y . Liu, X. Chen, R. K. Ward, and Z. J. Wang, “Image fusion with convolutional sparse representation,” IEEE Signal Processing Letters , vol. 23, no. 12, pp. 1882–1886, 2016

  48. [56]

    Region- based multimodal image fusion using ICA bases,

    N. Cvejic, J. J. Lewis, D. R. Bull, and C. N. Canagarajah, “Region- based multimodal image fusion using ICA bases,” in IEEE International Conference on Image Processing (ICIP) . IEEE, 2006, pp. 1801–1804

  49. [57]

    Image fusion of visible and thermal images for fruit detection,

    D. Bulanon, T. Burks, and V . Alchanatis, “Image fusion of visible and thermal images for fruit detection,” Biosystems Engineering, vol. 103, no. 1, pp. 12 – 22, 2009

  50. [58]

    Infrared and visual image fusion method based on discrete cosine transform and local spatial frequency in discrete stationary wavelet transform domain,

    X. Jin, Q. Jiang, S. Yao, D. Zhou, R. Nie, S.-J. Lee, and K. He, “Infrared and visual image fusion method based on discrete cosine transform and local spatial frequency in discrete stationary wavelet transform domain,” Infrared Physics & Technology , vol. 88, pp. 1 – 12, 2018

  51. [59]

    Multifocus image fusion scheme based on discrete cosine transform and spatial frequency,

    V . E, M. K, and S. B. R, “Multifocus image fusion scheme based on discrete cosine transform and spatial frequency,” Multimedia Tools and Applications, vol. 78, no. 13, pp. 17 573–17 587, 2019

  52. [60]

    A fusion algorithm for infrared and visible images based on adaptive dual-channel unit-linking pcnn in nsct domain,

    T. Xiang, L. Yan, and R. Gao, “A fusion algorithm for infrared and visible images based on adaptive dual-channel unit-linking pcnn in nsct domain,” Infrared Physics & Technology , vol. 69, pp. 53 – 61, 2015

  53. [61]

    The multiscale directional bilateral filter and its application to multisensor image fusion,

    J. Hu and S. Li, “The multiscale directional bilateral filter and its application to multisensor image fusion,” Information Fusion, vol. 13, no. 3, pp. 196–206, 2012

  54. [62]

    Image fusion based on non-negative matrix factorization,

    J. Zhang, L. Wei, Q. Miao, and Y . J. Wang, “Image fusion based on non-negative matrix factorization,” in IEEE International Conference on Image Processing (ICIP) . IEEE, 2004, pp. 973–976. 14

  55. [63]

    Fusiondn: A unified densely connected network for image fusion,

    H. Xu, J. Ma, Z. Le, J. Jiang, and X. Guo, “Fusiondn: A unified densely connected network for image fusion,” in Proceedings of the AAAI conference on artificial intelligence (AAAI) . Proceedings of the AAAI conference on artificial intelligence (AAAI) Press, 2020, pp. 12 484–12 491

  56. [64]

    Nestfuse: An infrared and visible image fusion architecture based on nest connection and spatial/channel attention models,

    H. Li, X. Wu, and T. S. Durrani, “Nestfuse: An infrared and visible image fusion architecture based on nest connection and spatial/channel attention models,” IEEE Transactions on Instrumentation and Measure- ment, vol. 69, no. 12, pp. 9645–9656, 2020

  57. [65]

    Didfuse: Deep image decomposition for infrared and visible image fusion,

    Z. Zhao, S. Xu, C. Zhang, J. Liu, J. Zhang, and P. Li, “Didfuse: Deep image decomposition for infrared and visible image fusion,” in Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI). ijcai.org, 2020, pp. 970–976

  58. [66]

    Fusiongan: A generative adversarial network for infrared and visible image fusion,

    J. Ma, W. Yu, P. Liang, C. Li, and J. Jiang, “Fusiongan: A generative adversarial network for infrared and visible image fusion,” Information Fusion, vol. 48, pp. 11–26, 2019

  59. [67]

    Infrared and visible image fusion via detail preserving adversarial learning,

    J. Ma, P. Liang, W. Yu, C. Chen, X. Guo, J. Wu, and J. Jiang, “Infrared and visible image fusion via detail preserving adversarial learning,” Information Fusion, vol. 54, pp. 85–98, 2020

  60. [68]

    Ddcgan: A dual- discriminator conditional generative adversarial network for multi- resolution image fusion,

    J. Ma, H. Xu, J. Jiang, X. Mei, and X. S. Zhang, “Ddcgan: A dual- discriminator conditional generative adversarial network for multi- resolution image fusion,” IEEE Transactions on Image Processing , vol. 29, pp. 4980–4995, 2020

  61. [69]

    Efficient and model-based infrared and visible image fusion via algorithm unrolling,

    Z. Zhao, S. Xu, J. Zhang, C. Liang, C. Zhang, and J. Liu, “Efficient and model-based infrared and visible image fusion via algorithm unrolling,” IEEE Transactions on Circuits and Systems for Video Technology , vol. 32, no. 3, pp. 1186–1196, 2022

  62. [70]

    Deep convolutional neural network for multi-modal image restoration and fusion,

    X. Deng and P. L. Dragotti, “Deep convolutional neural network for multi-modal image restoration and fusion,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 43, no. 10, pp. 3333– 3348, 2021

  63. [71]

    Deep convolutional sparse coding networks for interpretable image fusion,

    Z. Zhao, J. Zhang, H. Bai, Y . Wang, Y . Cui, L. Deng, K. Sun, C. Zhang, J. Liu, and S. Xu, “Deep convolutional sparse coding networks for interpretable image fusion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops . Compu...

  64. [72]

    U2fusion: A unified unsupervised image fusion network,

    H. Xu, J. Ma, J. Jiang, X. Guo, and H. Ling, “U2fusion: A unified unsupervised image fusion network,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 44, no. 1, pp. 502–518, 2022

  65. [73]

    Sdnet: A versatile squeeze-and-decomposition network for real-time image fusion,

    H. Zhang and J. Ma, “Sdnet: A versatile squeeze-and-decomposition network for real-time image fusion,” International Journal of Computer Vision, vol. 129, no. 10, pp. 2761–2785, 2021

  66. [74]

    Fusion from decomposition: A self- supervised decomposition approach for image fusion,

    P. Liang, J. Jiang, X. Liu, and J. Ma, “Fusion from decomposition: A self- supervised decomposition approach for image fusion,” in Proceedings of the European Conference on Computer Vision (ECCV) , ser. Lecture Notes in Computer Science, vol. 13678. Springer, 2022, pp. 719–735

  67. [75]

    Reconet: Recurrent correction network for fast and efficient multi-modality image fusion,

    Z. Huang, J. Liu, X. Fan, R. Liu, W. Zhong, and Z. Luo, “Reconet: Recurrent correction network for fast and efficient multi-modality image fusion,” in Proceedings of the European Conference on Computer Vision (ECCV), ser. Lecture Notes in Computer Science, vol. 13678. Springer...

  68. [76]

    Unsupervised misaligned infrared and visible image fusion via cross-modality image generation and registration,

    D. Wang, J. Liu, X. Fan, and R. Liu, “Unsupervised misaligned infrared and visible image fusion via cross-modality image generation and registration,” in Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI) . ijcai.org, 2022, pp. 3508–3515

  69. [77]

    Rfnet: Unsupervised network for mutually reinforcing multi-modal image registration and fusion,

    H. Xu, J. Ma, J. Yuan, Z. Le, and W. Liu, “Rfnet: Unsupervised network for mutually reinforcing multi-modal image registration and fusion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Computer Vision Foundation / IEEE, 2022, pp...

  70. [78]

    Discrete cosine trans- form network for guided depth map super-resolution,

    Z. Zhao, J. Zhang, S. Xu, Z. Lin, and H. Pfister, “Discrete cosine trans- form network for guided depth map super-resolution,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 5687–5697

  71. [79]

    Deep coupled ista network for multi- modal image super-resolution,

    X. Deng and P. L. Dragotti, “Deep coupled ista network for multi- modal image super-resolution,” IEEE Transactions on Image Processing, vol. 29, pp. 1683–1698, 2019

  72. [80]

    Multimodal deep unfolding for guided image super-resolution,

    I. Marivani, E. Tsiligianni, B. Cornelis, and N. Deligiannis, “Multimodal deep unfolding for guided image super-resolution,” IEEE Transactions on Image Processing , vol. 29, pp. 8443–8456, 2020

  73. [81]

    Memory- augmented deep conditional unfolding network for pan-sharpening,

    G. Yang, M. Zhou, K. Yan, A. Liu, X. Fu, and F. Wang, “Memory- augmented deep conditional unfolding network for pan-sharpening,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 1788–1797

  74. [82]

    Deep unfolding network for image super-resolution,

    K. Zhang, L. V . Gool, and R. Timofte, “Deep unfolding network for image super-resolution,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2020, pp. 3214– 3223

  75. [83]

    Learning deep CNN denoiser prior for image restoration,

    K. Zhang, W. Zuo, S. Gu, and L. Zhang, “Learning deep CNN denoiser prior for image restoration,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2017, pp. 2808– 2817

  76. [84]

    Denoising prior driven deep neural network for image restoration,

    W. Dong, P. Wang, W. Yin, G. Shi, F. Wu, and X. Lu, “Denoising prior driven deep neural network for image restoration,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 41, no. 10, pp. 2305–2318, 2019

  77. [85]

    Learning deep priors for image dehazing,

    Y . Liu, J. Pan, J. S. J. Ren, and Z. Su, “Learning deep priors for image dehazing,” in ICCV. IEEE Computer Society, 2019, pp. 2492–2500

  78. [86]

    Deep gradient projection networks for pan-sharpening,

    S. Xu, J. Zhang, Z. Zhao, K. Sun, J. Liu, and C. Zhang, “Deep gradient projection networks for pan-sharpening,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 1366–1375

  79. [87]

    Lrrnet: A novel representation learning guided fusion network for infrared and visible images,

    H. Li, T. Xu, X.-J. Wu, J. Lu, and J. Kittler, “Lrrnet: A novel representation learning guided fusion network for infrared and visible images,” IEEE transactions on pattern analysis and machine intelligence, 2023

  80. [88]

    Learning deep features for discriminative localization,

    B. Zhou, A. Khosla, `A. Lapedriza, A. Oliva, and A. Torralba, “Learning deep features for discriminative localization,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Computer Vision Foundation / IEEE, 2016, pp. 2921–2929

  81. [89]

    Grad-cam: Visual explanations from deep networks via gradient-based localization,

    R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV) . IEEE Computer Society, 2017, pp. 618–626

  82. [90]

    Robust tumor localization with pyramid grad-cam,

    S. Lee, J. Lee, J. Lee, C. Park, and S. Yoon, “Robust tumor localization with pyramid grad-cam,” CoRR, vol. abs/1805.11393, 2018

  83. [91]

    How to explain individual classification decisions,

    D. Baehrens, T. Schroeter, S. Harmeling, M. Kawanabe, K. Hansen, and K. M ¨uller, “How to explain individual classification decisions,” Journal of Machine Learning Research , vol. 11, pp. 1803–1831, 2010

  84. [92]

    Interpreting super-resolution networks with local attribution maps,

    J. Gu and C. Dong, “Interpreting super-resolution networks with local attribution maps,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Computer Vision Foundation / IEEE, 2021, pp. 9199–9208

  85. [93]

    Paths and consistency in additive cost sharing,

    E. J. Friedman, “Paths and consistency in additive cost sharing,” International Journal of Game Theory , vol. 32, no. 4, pp. 501–518, 2004

  86. [94]

    Encoder- decoder with atrous separable convolution for semantic image segmen- tation,

    L. Chen, Y . Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder- decoder with atrous separable convolution for semantic image segmen- tation,” in Proceedings of the European Conference on Computer Vision (ECCV), ser. Lecture Notes in Computer Science, vol. 11211. Springer, ...

  87. [95]

    Piafusion: A progressive infrared and visible image fusion network based on illumination aware,

    L. Tang, J. Yuan, H. Zhang, X. Jiang, and J. Ma, “Piafusion: A progressive infrared and visible image fusion network based on illumination aware,” Infromation Fusion, vol. 83-84, pp. 79–92, 2022

  88. [96]

    Ddfm: Denoising diffusion model for multi-modality image fusion,

    Z. Zhao, H. Bai, Y . Zhu, J. Zhang, S. Xu, Y . Zhang, K. Zhang, D. Meng, R. Timofte, and L. Van Gool, “Ddfm: Denoising diffusion model for multi-modality image fusion,” arXiv e-prints, pp. arXiv–2303, 2023

  89. [97]

    Murf: Mutually reinforcing multi-modal image registration and fusion,

    H. Xu, J. Yuan, and J. Ma, “Murf: Mutually reinforcing multi-modal image registration and fusion,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2023

  90. [98]

    Diff-if: Multi- modality image fusion via diffusion model with fusion knowledge prior,

    X. Yi, L. Tang, H. Zhang, H. Xu, and J. Ma, “Diff-if: Multi- modality image fusion via diffusion model with fusion knowledge prior,” Information Fusion, vol. 110, p. 102450, 2024

  91. [99]

    Text-if: Leveraging semantic text guidance for degradation-aware and interactive image fusion,

    X. Yi, H. Xu, H. Zhang, L. Tang, and J. Ma, “Text-if: Leveraging semantic text guidance for degradation-aware and interactive image fusion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2024, pp. 27 026–27 035

  92. [100]

    Segformer: Simple and efficient design for semantic segmentation with transformers,

    E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “Segformer: Simple and efficient design for semantic segmentation with transformers,” Advances in neural information processing systems , vol. 34, pp. 12 077–12 090, 2021. Haowen Bai is pursuing a Ph.D. degree i...

  93. [8689]

    Springer, 2014, pp. 818–833

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.