Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Task-driven Image Fusion with Learnable Fusion Loss

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a fusion loss generated by a neural network and updated by the downstream task loss via meta-learning yields fused images with lower task loss and better fusion metrics than fixed-loss fusion methods.

desk verdict A plausible extension of ReFusion that learns fusion loss weights from downstream task loss; the experiments support the idea, but the one-step surrogate assumption is unvalidated and the theoretical analysis is shaky. read the letter →

arxiv 2412.03240 v2 pith:TGQK763Y submitted 2024-12-04 cs.CV

Haowen Bai , Jiangshe Zhang , Zixiang Zhao , Yichen Wu , Lilun Deng , Yukun Cui , Tao Feng , Shuang Xu This is my paper · ORCID
classification cs.CV
keywords imagefusionlearnablelossmeta-learninginfrared-visiblesemanticsegmentationobjectdetectiontask-drivenbileveloptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TDFusion replaces the hand-designed loss used to train a multi-modal image fusion network with a fusion loss generated by a small neural network, and then tunes that generator so that fused images trained under its loss minimize the downstream task's loss. The paper's claim is that this task-learned loss adapts to what semantic segmentation or object detection actually needs, and that it improves both fusion quality and task accuracy compared with seven fixed-loss fusion methods on four infrared-visible datasets. Training needs no ground-truth fused image and no manually tuned fusion objective; the downstream task loss is the only supervision for the loss generator. If the claim holds, fusion becomes a task-adaptable component rather than a generic preprocessing step.

What carries the argument

The central machinery is the loss generation module $G$, a small neural network that outputs pixel-wise intensity weights $w_a, w_b$ for the fusion loss, trained through a bilevel meta-learning loop in the style of model-agnostic meta-learning. The inner update (Eq. 4) takes one gradient step of a clone of the fusion network under the generated loss $L_f$, while a clone of the task network takes one step under $L_t$; the outer update (Eq. 6) then back-propagates the task loss of the clone-fused image through the inner update to move $G$. The load-bearing identity is Eq. 7, which expresses $\partial L_t/\partial \theta_G$ as the task-loss gradient times the second-order coupling between fusion parameters and generator parameters; that coupling is what lets the downstream task reshape the fusion loss itself.

What would settle it

Run TDFusion with the inner update unrolled for two or three steps instead of one and compare the final task loss and fusion metrics; if the multi-step version gives different or worse results, the single-step clone was not a faithful proxy for the converged fusion network, and the outer update was optimizing the generator against a surrogate.

Watch

Extended reading notes

Core claim

TDFusion's learnable fusion loss has an intensity term and a gradient term: $L_f = L^{\mathrm{int}}_f + \alpha L^{\mathrm{grad}}_f$, where the pixel-wise intensity weights $\{w_a, w_b\} = G(I_a, I_b)$ are produced by a loss generation module and satisfy $w^{ij}_a + w^{ij}_b = 1$, and the gradient term keeps the stronger of the two source gradients at each location. The loss generation module is trained by a meta-learning loop: an inner step clones the fusion and task networks and updates each clone once using $L_f$ and the task loss $L_t$; an outer step computes the task loss of the image produced by the updated fusion clone on a separate meta-test batch and back-propagates through the inner update to update $G$. The stated objective is to minimize the task loss of fused images after optimizing the fusion module with the fusion loss. The paper reports that this scheme yields better fusion metrics and better segmentation and detection results than seven baselines on MSRS, FMB, M3FD, and LLVIP.

Load-bearing premise

The load-bearing premise is that one gradient step of the fusion network under the generated loss predicts how the fully trained fusion network will respond to that loss; if the one-step clone is not faithful, the loss generator is being optimized against a surrogate that does not represent real fusion training.

Editorial extensions

If this is right

  • Fusion networks for a given task can be trained with no ground-truth fused image, because the only supervision is the task loss of the fused output.
  • The same framework transfers to different fusion architectures and different task networks, since the generated loss depends only on the source images and the downstream task loss.
  • The learned fusion weights reveal task-specific modality preferences: segmentation favors boundary and texture information, while detection favors edge and contrast information, including infrared-bright regions.
  • On the four tested datasets, TDFusion reports higher fusion metrics (for example SCD, QAB/F, and SSIM) and higher downstream metrics (mIoU and mAP) than the seven compared fixed-loss methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Not tested in the paper: because the inner loop is a single gradient step, the learned loss is only guaranteed to help that one-step clone; unrolling several inner steps and checking whether task loss still falls would reveal whether the generator exploits the surrogate rather than helping the converged fusion network.
  • Not tested in the paper: the learned weights $w_a, w_b$ could serve as an interpretability map of what a task uses from each modality, potentially transferring to new scenes or new tasks without retraining.
  • Not tested in the paper: training solely against one task loss could make the fused image discard information that other tasks need, so a task-trained fused image may generalize poorly to an unseen task; evaluating TDFusion outputs on a task different from the one used in training would test this.
  • Not tested in the paper: the framework is architecture-agnostic in principle, so it should extend to other fusion settings, such as medical or multi-exposure fusion, with only the task network and its loss swapped.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes TDFusion, a task-driven infrared-visible image fusion framework in which the fusion loss itself is generated by a neural network (loss generation module G) and is updated by the downstream task loss through a MAML-style meta-learning loop. The fusion loss is a weighted intensity term plus a gradient term, and the weights are produced by G. In the inner update, clones of the fusion and task networks are advanced one gradient step on the fusion loss and task loss, respectively; in the outer update, G is updated using the task loss of the clone's fused output on a meta-test batch. The fusion network is then trained on the full dataset with the generated loss. Experiments on MSRS, FMB, M3FD, and LLVIP compare against seven fusion methods, using fusion metrics plus downstream semantic segmentation (SegFormer) and object detection (YOLOv8). The central claim is that TDFusion's learnable, task-guided fusion loss yields fused images with lower downstream task loss and better fusion metrics than fixed-loss baselines.

Significance. If the claims hold, TDFusion provides a principled way to replace hand-crafted fusion losses with a task-adaptive loss, which is a useful step for making fusion training explicitly optimize downstream objectives. The framework is architecture-agnostic and the code is released, which supports reproducibility. The empirical comparison spans four datasets and two high-level tasks, with visualizations of the learned loss weights that show task-dependent behavior. However, the significance is tempered by two issues: the meta-learning objective is trained against a one-step surrogate of the fusion network without validation that this surrogate represents the multi-step trained fusion network used at inference, and the reported margins are often small with no error bars. The theoretical analysis in Sec. 3.7 contains an incorrect chain-rule derivation that currently weakens the paper's formal contribution.

major comments (3)
  1. [Sec. 3.4.1, Eqs. (4)-(6); Algorithm 1] The loss generation module G is optimized against a one-step clone F' obtained by a single gradient descent step on the fusion loss Lf (Eq. (4)), and the outer update (Eq. (6)) backpropagates the task loss through this clone. However, the fusion network actually deployed is trained for many gradient steps with Lf (Eq. (8), N iterations per epoch in Algorithm 1). The paper never validates that the one-step clone's task-loss response is predictive of the final multi-step trained fusion network. This is an unstated faithfulness assumption: G could be rewarded for producing a loss that yields a large one-step improvement (e.g., by an ill-conditioned initial gradient) while not improving the converged fusion output. Please add an ablation that varies the number of inner steps (K=1 vs. K>1) or otherwise checks the correlation between the one-step clone's task loss and the task loss of the fully trained fusion network.
  2. [Sec. 3.7, Eq. (11)] The chain-rule expansion in Eq. (11) is mathematically incorrect. The derivative of (I_a - F_thetaF)^2 with respect to thetaF is -2(I_a - F_thetaF) * partial F / partial thetaF, not (I_a - partial F/partial thetaF) * (I_a - partial F/partial thetaF) as written. Moreover, the gradient term alpha Lgrad_f from Eq. (1) is dropped from the derivation even though it is part of Lf, and the notation 'partial G(I_a,I_b;thetaG)/thetaG' is missing the denominator 'partial thetaG'. As written, Eq. (11) does not constitute a valid derivation of the claimed inner-product interpretation. This section should be corrected or removed, since the conclusion states that the theoretical analysis explains how the task loss guides the fusion loss.
  3. [Sec. 4, Tables 1-2] No error bars, multiple seeds, or significance tests are reported for any of the quantitative results. Several reported margins are very small: e.g., MSRS SSIM is 0.70 tied with EMMA, M3FD SSIM is 0.72 tied with DCINN, and LLVIP AP50 is 95.00 versus 94.92 for DCINN. Without variance information, the claim that TDFusion 'outperforms' baselines on most metrics is not statistically supported. Please report results over at least three runs with standard deviations, or provide a significance test for the headline comparisons in Tables 1 and 2.
minor comments (5)
  1. [Sec. 3.6] The text 'After applying Sof tmax(·)' contains a formatting typo; it should read 'Softmax'.
  2. [Eq. (11)] In Eq. (11), the term 'partial G(I_a,I_b;thetaG)/thetaG' should be 'partial G(I_a,I_b;thetaG)/partial thetaG', and the symbol G is used both for the loss generation module and for the inner product, which is confusing.
  3. [Abstract and Sec. 1] The statement that 'TDFusion's training relies entirely on the downstream task loss' is an overstatement: the loss in Eq. (1) includes a fixed gradient term alpha Lgrad_f and a manually defined intensity structure, with only the weights {w_a,w_b} being learned. Please rephrase to describe the supervision source more precisely.
  4. [Table 2] The object-detection metrics are named inconsistently: 'mAP50' is used for M3FD while 'AP50' is used for LLVIP. Please use a consistent naming convention and define both.
  5. [Sec. 4.4] The cross-dataset evaluation on MSRS is described as an evaluation, but Fig. 5 reports only visualizations of learned weights; no quantitative transferability results are given. Please clarify that this is a qualitative illustration, or add numeric results.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity; the task-loss-guided fusion-loss objective is the method's explicit training target, and the reported gains come from held-out evaluations against external baselines.

full rationale

The paper's central claim is that a learnable fusion loss, trained by the downstream task loss, improves fusion and downstream-task performance. This is not a prediction derived from an input; it is the explicit optimization objective in Eq. (6), which minimizes the task loss of a one-step-updated clone of the fusion network. The subsequent fusion training in Eq. (8) applies the learned loss, and the experiments measure fusion and task metrics on held-out test splits across four datasets, comparing against seven external baselines. Thus the empirical results are not forced by construction: the one-step inner loop is a surrogate, and a gap between the one-step clone and the full fusion training could invalidate the method, but that is an unvalidated faithfulness assumption rather than circularity. The theoretical analysis in Eq. (11) is a restatement of the gradient of the outer objective, not an independent result that presupposes its conclusion. Self-citations, such as ReFusion [5] for the fusion network architecture and CDDFuse [83] for the standard gradient-loss form, are ancillary and are not load-bearing for the central claim. One overstatement exists: the loss also contains a fixed gradient term and softmax normalization, so training does not rely 'entirely' on the downstream task loss; however, this is a correctness or framing issue, not circular reasoning.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

TDFusion introduces no new physical entities. Its central claim rests on three hyperparameters (α, M, L) chosen by hand without sensitivity analysis, and on three domain assumptions: one-step inner update fidelity, expressiveness of the intensity/gradient loss family, and validity of task-loss supervision. The learned weights {wa, wb} are outputs of the proposed loss generation module and are not counted as hidden free parameters here.

free parameters (3)
  • α (gradient loss weight) = 1
    Hand-set in Eq. (1) with no sensitivity analysis; it controls the balance between the learned intensity term and the fixed gradient term and is used in every experiment.
  • M (inner/outer iterations per epoch) = 200
    Chosen in Sec. 4.1 without ablation; it controls how much the loss generation module is trained relative to the fusion network.
  • L (total training epochs) = 50
    Chosen in Sec. 4.1 without ablation; all reported results use this schedule.
assumptions (4)
  • domain assumption A single inner gradient step on the generated fusion loss is a faithful surrogate for the fusion network's full learned response to that loss.
    Eq. (4) defines θF′ after one step and Eq. (6) judges G on that clone; the paper does not validate that one-step behavior matches converged behavior.
  • domain assumption The intensity-plus-gradient loss form with per-pixel softmax weights is expressive enough to encode task preferences.
    Eqs. (1)-(3) fix the loss family; G only outputs per-pixel intensity weights, so any task-specific adaptation must be representable in those weights.
  • domain assumption Downstream task loss is a valid training signal for fusion in the absence of fusion ground truth.
    Sec. 3.4 supervises G entirely through Lt of fused images; no fusion ground truth is used.
  • domain assumption Second-order gradients through the inner update are numerically usable without special stabilization.
    Eq. (7) requires ∂²Lf/(∂θF∂θG); no clipping, damping, or gradient normalization is described.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Task-driven Image Fusion with Learnable Fusion Loss." pith.science (2026). https://pith.science/paper/TGQK763Y

@misc{pith2026241203240,
  author       = {Pith},
  title        = {Pith review of: Task-driven Image Fusion with Learnable Fusion Loss},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TGQK763Y}},
  note         = {Machine review of arXiv:2412.03240}
}
read the original abstract

Multi-modal image fusion aggregates information from multiple sensor sources, achieving superior visual quality and perceptual features compared to single-source images, often improving downstream tasks. However, current fusion methods for downstream tasks still use predefined fusion objectives that potentially mismatch the downstream tasks, limiting adaptive guidance and reducing model flexibility. To address this, we propose Task-driven Image Fusion (TDFusion), a fusion framework incorporating a learnable fusion loss guided by task loss. Specifically, our fusion loss includes learnable parameters modeled by a neural network called the loss generation module. This module is supervised by the downstream task loss in a meta-learning manner. The learning objective is to minimize the task loss of fused images after optimizing the fusion module with the fusion loss. Iterative updates between the fusion module and the loss module ensure that the fusion network evolves toward minimizing task loss, guiding the fusion process toward the task objectives. TDFusion's training relies entirely on the downstream task loss, making it adaptable to any specific task. It can be applied to any architecture of fusion and task networks. Experiments demonstrate TDFusion's performance through fusion experiments conducted on four different datasets, in addition to evaluations on semantic segmentation and object detection tasks.

Figures

Figures reproduced from arXiv: 2412.03240 by the authors.

Figure 1
Figure 1. The TDFusion workflow alternates between training the loss generation module and the fusion module. Training of the loss [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Visual comparison of fusion results. The cases are “01258N” in MSRS dataset, “00122” in FMB dataset, “00449” in M3FD [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Visual comparison for Semantic Segmentation. The cases are “00726N” in MSRS dataset and “01438” in FMB dataset. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Visual comparison for Object Detection. The cases are “02236” in M3FD dataset and “210145” in LLVIP dataset. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Visualisation of learnable loss for downstream tasks. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Neural Video Compression with Context Modulation

    eess.IV 2025-05 conditional novelty 6.0 of 10

    DCMVC modulates the propagated temporal context with an additional oriented context from the reference frame, reporting 10.1 percent bitrate savings over DCVC-FM and 22.7 percent over VVC on standard test sets.

Reference graph

Works this paper leans on

88 extracted references · 72 canonical work pages · cited by 1 Pith paper

  1. [1]

    Image fusion in remote sensing: An overview and meta-analysis

    Hessah Albanwan, Rongjun Qin, and Yang Tang. Image fusion in remote sensing: An overview and meta-analysis. Photogrammetric Engineering & Remote Sensing , 90(12): 755–775, 2024. 1

    2024

  2. [2]

    Antreas Antoniou and Amos J. Storkey. Learning to learn by self-critique. In Proceedings of the Advances in Neural In- formation Processing Systems (NeurIPS), pages 9936–9946,

  3. [3]

    Simulta- neous automatic picking and manual picking refinement for first-break

    Haowen Bai, Zixiang Zhao, Jiangshe Zhang, Yukun Cui, Chunxia Zhang, Zhenbo Guo, and Yongjun Wang. Simulta- neous automatic picking and manual picking refinement for first-break. IEEE Transactions on Geoscience and Remote Sensing, 2024. 1

    2024

  4. [4]

    Deep unfolding multi-modal image fusion network via attri- bution analysis

    Haowen Bai, Zixiang Zhao, Jiangshe Zhang, Baisong Jiang, Lilun Deng, Yukun Cui, Shuang Xu, and Chunxia Zhang. Deep unfolding multi-modal image fusion network via attri- bution analysis. IEEE Transactions on Circuits and Systems for Video Technology, 2024. 2

    2024

  5. [5]

    Refusion: Learning image fusion from reconstruction with learnable loss via meta-learning

    Haowen Bai, Zixiang Zhao, Jiangshe Zhang, Yichen Wu, Lilun Deng, Yukun Cui, Baisong Jiang, and Shuang Xu. Refusion: Learning image fusion from reconstruction with learnable loss via meta-learning. International Journal of Computer Vision, pages 1–21, 2024. 3, 5

    2024

  6. [6]

    Retinex-MEF: Retinex-based Glare Effects Aware Unsupervised Multi-Exposure Image Fusion

    Haowen Bai, Jiangshe Zhang, Zixiang Zhao, Lilun Deng, Yukun Cui, and Shuang Xu. Retinex-mef: Retinex-based glare effects aware unsupervised multi-exposure image fu- sion. arXiv preprint arXiv:2503.07235, 2025. 2

    work page Pith review arXiv 2025

  7. [7]

    Meta-learning with task- adaptive loss function for few-shot learning

    Sungyong Baik, Janghoon Choi, Heewon Kim, Dohee Cho, Jaesik Min, and Kyoung Mu Lee. Meta-learning with task- adaptive loss function for few-shot learning. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 9445–9454. IEEE, 2021. 2

    2021

  8. [8]

    Pedestrian detection with un- supervised multispectral feature learning using deep neural networks

    Yanpeng Cao, Dayan Guan, Weilin Huang, Jiangxin Yang, Yanlong Cao, and Yu Qiao. Pedestrian detection with un- supervised multispectral feature learning using deep neural networks. Information Fusion, 46:206–217, 2019. 1

    2019

Show all 88 references
  1. [9]

    Segformer: Simple and efficient design for semantic segmentation with transformers

    Bo Cheng, Xiang Li, Yujie Wei, Cheng Huang, Xiaoyong Zhang, Yandong Jiang, Tianyu Zhang, Na Xu, Shuai Yu, Xinxin Zhan, et al. Segformer: Simple and efficient design for semantic segmentation with transformers. InProceedings of the IEEE/CVF International Conference on Computer ...

    2021

  2. [10]

    Deep convolutional neural network for multi-modal image restoration and fusion

    Xin Deng and Pier Luigi Dragotti. Deep convolutional neural network for multi-modal image restoration and fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(10):3333–3348, 2021. 2

    2021

  3. [11]

    Model- agnostic meta-learning for fast adaptation of deep networks

    Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model- agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the International conference on machine learning (ICML), pages 1126–1135, 2017. 2

    2017

  4. [12]

    Online meta-learning

    Chelsea Finn, Aravind Rajeswaran, Sham Kakade, and Sergey Levine. Online meta-learning. In Proceedings of the International conference on machine learning (ICML) , pages 1920–1930, 2019. 2

    1920

  5. [13]

    Mfnet: Towards real-time se- mantic segmentation for autonomous vehicles with multi- spectral scenes

    Qishen Ha, Kohei Watanabe, Takumi Karasawa, Yoshitaka Ushiku, and Tatsuya Harada. Mfnet: Towards real-time se- mantic segmentation for autonomous vehicles with multi- spectral scenes. In IROS, pages 5108–5115. IEEE, 2017. 1

    2017

  6. [14]

    Task-driven super resolution: Object detection in low-resolution images

    Muhammad Haris, Greg Shakhnarovich, and Norimichi Ukita. Task-driven super resolution: Object detection in low-resolution images. In Neural Information Processing: 28th International Conference (ICONIP) , pages 387–395. Springer, 2021. 1

    2021

  7. [15]

    Stadie, Filip Wolski, Jonathan Ho, and Pieter Abbeel

    Rein Houthooft, Yuhua Chen, Phillip Isola, Bradly C. Stadie, Filip Wolski, Jonathan Ho, and Pieter Abbeel. Evolved pol- icy gradients. In Proceedings of the Advances in Neural In- formation Processing Systems (NeurIPS), pages 5405–5414,

  8. [16]

    Reconet: Recurrent correction network for fast and efficient multi-modality image fusion

    Zhanbo Huang, Jinyuan Liu, Xin Fan, Risheng Liu, Wei Zhong, and Zhongxuan Luo. Reconet: Recurrent correction network for fast and efficient multi-modality image fusion. In Proceedings of the European Conference on Computer Vi- sion (ECCV), pages 539–555. Springer, 2022. 2

    2022

  9. [17]

    Llvip: A visible-infrared paired dataset for low-light vision

    Xinyu Jia, Chuang Zhu, Minzhen Li, Wenqi Tang, and Wenli Zhou. Llvip: A visible-infrared paired dataset for low-light vision. In Proceedings of the IEEE/CVF international con- ference on computer vision, pages 3496–3504, 2021. 6, 7

    2021

  10. [18]

    Dynamic instance normalization for arbitrary style transfer

    Yongcheng Jing, Xiao Liu, Yukang Ding, Xinchao Wang, Errui Ding, Mingli Song, and Shilei Wen. Dynamic instance normalization for arbitrary style transfer. In Proceedings of the AAAI conference on artificial intelligence (AAAI), pages 4369–4376, 2020. 1

    2020

  11. [19]

    Ultralytics YOLOv8

    Glenn Jocher. Ultralytics YOLOv8. https://github. com/ultralytics, 2023. 4, 5, 7

    2023

  12. [20]

    Cross-modal ranking with soft consistency and noisy labels for robust RGB-T tracking

    Chenglong Li, Chengli Zhu, Yan Huang, Jin Tang, and Liang Wang. Cross-modal ranking with soft consistency and noisy labels for robust RGB-T tracking. In Proceedings of the Eu- ropean Conference on Computer Vision (ECCV), pages 831–

  13. [21]

    Densefuse: A fusion approach to infrared and visible images

    Hui Li and Xiao-Jun Wu. Densefuse: A fusion approach to infrared and visible images. IEEE Transactions on Image Processing, 28(5):2614–2623, 2019. 1, 2

    2019

  14. [22]

    Different input resolutions and arbitrary output resolu- tion: A meta learning-based deep framework for infrared and visible image fusion

    Huafeng Li, Yueliang Cen, Yu Liu, Xun Chen, and Zhengtao Yu. Different input resolutions and arbitrary output resolu- tion: A meta learning-based deep framework for infrared and visible image fusion. IEEE Transactions on Image Process- ing, 30:4070–4083, 2021. 3

    2021

  15. [23]

    Rfn-nest: An end-to- end residual fusion network for infrared and visible images

    Hui Li, Xiao-Jun Wu, and Josef Kittler. Rfn-nest: An end-to- end residual fusion network for infrared and visible images. Information Fusion, 73:72–86, 2021. 1, 2

    2021

  16. [24]

    Tokuda, Roberto Hirata Junior, Roberto Mar- condes Cesar Junior, Jiawan Zhang, Xiaojie Guo, and Xi- aochun Cao

    Siyuan Li, Iago Breno Araujo, Wenqi Ren, Zhangyang Wang, Eric K. Tokuda, Roberto Hirata Junior, Roberto Mar- condes Cesar Junior, Jiawan Zhang, Xiaojie Guo, and Xi- aochun Cao. Single image deraining: A comprehensive benchmark analysis. In Proceedings of the IEEE/CVF Conferenc...

    2019

  17. [25]

    Ccafusion: cross-modal coordinate attention network for infrared and visible image fusion

    Xiaoling Li, Yanfeng Li, Houjin Chen, Yahui Peng, and Pan Pan. Ccafusion: cross-modal coordinate attention network for infrared and visible image fusion. IEEE Transactions on Circuits and Systems for Video Technology, 2023. 1

    2023

  18. [26]

    Meta- sgd: Learning to learn quickly for few-shot learning

    Zhenguo Li, Fengwei Zhou, Fei Chen, and Hang Li. Meta- sgd: Learning to learn quickly for few-shot learning. arXiv preprint arXiv:1707.09835, 2017. 2

    2017 arXiv

  19. [27]

    Ustc-td: A test dataset and bench- mark for image and video coding in 2020s

    Zhuoyuan Li, Junqi Liao, Chuanbo Tang, Haotian Zhang, Yuqi Li, Yifan Bian, Xihua Sheng, Xinmin Feng, Yao Li, Changsheng Gao, et al. Ustc-td: A test dataset and bench- mark for image and video coding in 2020s. arXiv preprint arXiv:2409.08481, 2024. 1

    2024 arXiv

  20. [28]

    Object segmentation-assisted inter prediction for versatile video coding

    Zhuoyuan Li, Zikun Yuan, Li Li, Dong Liu, Xiaohu Tang, and Feng Wu. Object segmentation-assisted inter prediction for versatile video coding. IEEE Transactions on Broadcast- ing, 2024. 1

    2024

  21. [29]

    Fusion from decomposition: A self-supervised decomposi- tion approach for image fusion

    Pengwei Liang, Junjun Jiang, Xianming Liu, and Jiayi Ma. Fusion from decomposition: A self-supervised decomposi- tion approach for image fusion. In Proceedings of the Euro- pean Conference on Computer Vision (ECCV) , pages 719–

  22. [30]

    Darts: Differentiable architecture search

    Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09055, 2018. 2

    2018 arXiv

  23. [31]

    Learning a deep multi-scale feature ensemble and an edge-attention guidance for image fusion

    Jinyuan Liu, Xin Fan, Ji Jiang, Risheng Liu, and Zhongx- uan Luo. Learning a deep multi-scale feature ensemble and an edge-attention guidance for image fusion. IEEE Transac- tions on Circuits and Systems for Video Technology , 32(1): 105–119, 2021. 1

    2021

  24. [32]

    Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection

    Jinyuan Liu, Xin Fan, Zhanbo Huang, Guanyao Wu, Risheng Liu, Wei Zhong, and Zhongxuan Luo. Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection. In Proceedings of the IEEE/CVF Conference on Compu...

  25. [33]

    Attention-guided global-local adversarial learning for detail- preserving multi-exposure image fusion

    Jinyuan Liu, Jingjie Shang, Risheng Liu, and Xin Fan. Attention-guided global-local adversarial learning for detail- preserving multi-exposure image fusion. IEEE Transactions on Circuits and Systems for Video Technology, 32(8):5026– 5040, 2022. 2

    2022

  26. [34]

    Multi-interactive feature learning and a full-time multi-modality benchmark for image fusion and segmentation

    Jinyuan Liu, Zhu Liu, Guanyao Wu, Long Ma, Risheng Liu, Wei Zhong, Zhongxuan Luo, and Xin Fan. Multi-interactive feature learning and a full-time multi-modality benchmark for image fusion and segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vis...

    2023

  27. [35]

    Holoco: Holistic and local con- trastive learning network for multi-exposure image fusion

    Jinyuan Liu, Guanyao Wu, Junsheng Luan, Zhiying Jiang, Risheng Liu, and Xin Fan. Holoco: Holistic and local con- trastive learning network for multi-exposure image fusion. Information Fusion, 95:237–249, 2023. 2

    2023

  28. [36]

    Coconet: Coupled con- trastive learning network with multi-level feature ensemble for multi-modality image fusion

    Jinyuan Liu, Runjia Lin, Guanyao Wu, Risheng Liu, Zhongxuan Luo, and Xin Fan. Coconet: Coupled con- trastive learning network with multi-level feature ensemble for multi-modality image fusion. International Journal of Computer Vision, 132(5):1748–1775, 2024. 2

    2024

  29. [37]

    Infrared and visible im- age fusion: From data compatibility to task adaption

    Jinyuan Liu, Guanyao Wu, Zhu Liu, Di Wang, Zhiying Jiang, Long Ma, Wei Zhong, and Xin Fan. Infrared and visible im- age fusion: From data compatibility to task adaption. IEEE Transactions on Pattern Analysis and Machine Intelligence,

  30. [38]

    Search- ing a hierarchically aggregated fusion architecture for fast multi-modality image fusion

    Risheng Liu, Zhu Liu, Jinyuan Liu, and Xin Fan. Search- ing a hierarchically aggregated fusion architecture for fast multi-modality image fusion. In Proceedings of the ACM International Conference on Multimedia (ACM MM), pages 1600–1608. ACM, 2021. 3

    2021

  31. [39]

    Learning with nested scene modeling and cooper- ative architecture search for low-light vision

    Risheng Liu, Long Ma, Tengyu Ma, Xin Fan, and Zhongx- uan Luo. Learning with nested scene modeling and cooper- ative architecture search for low-light vision. IEEE Transac- tions on Pattern Analysis and Machine Intelligence , 45(5): 5953–5969, 2022. 1

    2022

  32. [40]

    A task-guided, implicitly-searched and metainitialized deep model for image fusion

    Risheng Liu, Zhu Liu, Jinyuan Liu, Xin Fan, and Zhongxuan Luo. A task-guided, implicitly-searched and metainitialized deep model for image fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 1, 2, 3, 6, 7

    2024

  33. [41]

    A new multi-focus image fusion algorithm and its efficient imple- mentation

    ShuMin Liu, Jiajia Chen, and Susanto Rahardja. A new multi-focus image fusion algorithm and its efficient imple- mentation. IEEE Transactions on Circuits and Systems for Video Technology, 30(5):1374–1384, 2020. 1

    2020

  34. [42]

    Bi-level dynamic learning for jointly multi- modality image fusion and beyond

    Zhu Liu, Jinyuan Liu, Guanyao Wu, Long Ma, Xin Fan, and Risheng Liu. Bi-level dynamic learning for jointly multi- modality image fusion and beyond. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelli- gence (IJCAI), pages 1240–1248, 2023. 2

    2023

  35. [43]

    Paif: Perception-aware infrared-visible im- age fusion for attack-tolerant semantic segmentation

    Zhu Liu, Jinyuan Liu, Benzhuang Zhang, Long Ma, Xin Fan, and Risheng Liu. Paif: Perception-aware infrared-visible im- age fusion for attack-tolerant semantic segmentation. InPro- ceedings of the 31st ACM international conference on multi- media, pages 3706–3714, 2023

    2023

  36. [44]

    Searching a compact architecture for robust multi-exposure image fusion

    Zhu Liu, Jinyuan Liu, Guanyao Wu, Zihang Chen, Xin Fan, and Risheng Liu. Searching a compact architecture for robust multi-exposure image fusion. IEEE Transactions on Circuits and Systems for Video Technology, 34(7):6224–6237, 2024. 2

    2024

  37. [45]

    Infrared and visible im- age fusion methods and applications: A survey

    Jiayi Ma, Yong Ma, and Chang Li. Infrared and visible im- age fusion methods and applications: A survey. Information Fusion, 45:153–178, 2019. 1

    2019

  38. [46]

    Fusiongan: A generative adversarial network for in- frared and visible image fusion

    Jiayi Ma, Wei Yu, Pengwei Liang, Chang Li, and Junjun Jiang. Fusiongan: A generative adversarial network for in- frared and visible image fusion. Information Fusion, 48:11– 26, 2019. 2

    2019

  39. [47]

    Infrared and visible image fusion via detail preserving adversarial learning

    Jiayi Ma, Pengwei Liang, Wei Yu, Chen Chen, Xiaojie Guo, Jia Wu, and Junjun Jiang. Infrared and visible image fusion via detail preserving adversarial learning. Information Fu- sion, 54:85–98, 2020

    2020

  40. [48]

    Ddcgan: A dual-discriminator con- ditional generative adversarial network for multi-resolution image fusion

    Jiayi Ma, Han Xu, Junjun Jiang, Xiaoguang Mei, and Xiao- Ping (Steven) Zhang. Ddcgan: A dual-discriminator con- ditional generative adversarial network for multi-resolution image fusion. IEEE Transactions on Image Processing, 29: 4980–4995, 2020. 2

    2020

  41. [49]

    Toward fast, flexible, and robust low-light image enhancement

    Long Ma, Tengyu Ma, Risheng Liu, Xin Fan, and Zhongx- uan Luo. Toward fast, flexible, and robust low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5637–5646, 2022. 1

    2022

  42. [50]

    On first-order meta-learning algorithms

    Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02999, 2018. 2

    2018 arXiv

  43. [51]

    Cross-modal transformers for infrared and visible image fusion

    Seonghyun Park, An Gia Vien, and Chul Lee. Cross-modal transformers for infrared and visible image fusion. IEEE Transactions on Circuits and Systems for Video Technology, 34(2):770–785, 2023. 1

    2023

  44. [52]

    Does haze removal help cnn-based image classifica- tion? In Proceedings of the European conference on com- puter vision (ECCV), pages 682–697, 2018

    Yanting Pei, Yaping Huang, Qi Zou, Yuhang Lu, and Song Wang. Does haze removal help cnn-based image classifica- tion? In Proceedings of the European conference on com- puter vision (ECCV), pages 682–697, 2018. 1

    2018

  45. [53]

    Ground- truth free meta-learning for deep compressive sampling

    Xinran Qin, Yuhui Quan, Tongyao Pang, and Hui Ji. Ground- truth free meta-learning for deep compressive sampling. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR) , pages 9947–9956,

  46. [54]

    Learning to reweight examples for robust deep learning

    Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urta- sun. Learning to reweight examples for robust deep learning. In Proceedings of the International conference on machine learning (ICML), pages 4334–4343, 2018. 2

    2018

  47. [55]

    Meta-weight-net: Learning an explicit mapping for sample weighting

    Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Zongben Xu, and Deyu Meng. Meta-weight-net: Learning an explicit mapping for sample weighting. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2019. 2

    2019

  48. [56]

    Image fusion in the loop of high-level vision tasks: A semantic-aware real- time infrared and visible image fusion network

    Linfeng Tang, Jiteng Yuan, and Jiayi Ma. Image fusion in the loop of high-level vision tasks: A semantic-aware real- time infrared and visible image fusion network. Information Fusion, 82:28–42, 2022. 1, 2

    2022

  49. [57]

    Image fusion in the loop of high-level vision tasks: A semantic-aware real- time infrared and visible image fusion network

    Linfeng Tang, Jiteng Yuan, and Jiayi Ma. Image fusion in the loop of high-level vision tasks: A semantic-aware real- time infrared and visible image fusion network. Information Fusion, 82:28–42, 2022. 4

    2022

  50. [58]

    Piafusion: A progressive infrared and visible im- age fusion network based on illumination aware.Infromation Fusion, 83-84:79–92, 2022

    Linfeng Tang, Jiteng Yuan, Hao Zhang, Xingyu Jiang, and Jiayi Ma. Piafusion: A progressive infrared and visible im- age fusion network based on illumination aware.Infromation Fusion, 83-84:79–92, 2022. 6, 7

    2022

  51. [59]

    Datfuse: Infrared and visible image fusion via dual atten- tion transformer

    Wei Tang, Fazhi He, Yu Liu, Yansong Duan, and Tongzhen Si. Datfuse: Infrared and visible image fusion via dual atten- tion transformer. IEEE Transactions on Circuits and Systems for Video Technology, 33(7):3159–3172, 2023. 1

    2023

  52. [60]

    Unsuper- vised misaligned infrared and visible image fusion via cross- modality image generation and registration

    Di Wang, Jinyuan Liu, Xin Fan, and Risheng Liu. Unsuper- vised misaligned infrared and visible image fusion via cross- modality image generation and registration. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI), pages 3508–3515...

    2022

  53. [61]

    A general paradigm with detail-preserving conditional in- vertible network for image fusion

    Wu Wang, Liang-Jian Deng, Ran Ran, and Gemine Vivone. A general paradigm with detail-preserving conditional in- vertible network for image fusion. International Journal of Computer Vision, 132(4):1029–1054, 2024. 6, 7

    2024

  54. [62]

    Event-based video super- resolution via state space models

    Zeyu Xiao and Xinchao Wang. Event-based video super- resolution via state space models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. 1

    2025

  55. [63]

    Asymmetric event-guided video super- resolution

    Zeyu Xiao, Dachun Kai, Yueyi Zhang, Xiaoyan Sun, and Zhiwei Xiong. Asymmetric event-guided video super- resolution. In Proceedings of the ACM International Con- ference on Multimedia (ACM MM), pages 2409–2418, 2024. 1

    2024

  56. [64]

    Event-adapted video super- resolution

    Zeyu Xiao, Dachun Kai, Yueyi Zhang, Zheng-Jun Zha, Xi- aoyan Sun, and Zhiwei Xiong. Event-adapted video super- resolution. In Proceedings of the European Conference on Computer Vision (ECCV), pages 217–235. Springer, 2024. 1

    2024

  57. [65]

    Fusiondn: A unified densely connected network for image fusion

    Han Xu, Jiayi Ma, Zhuliang Le, Junjun Jiang, and Xiaojie Guo. Fusiondn: A unified densely connected network for image fusion. In Proceedings of the AAAI conference on artificial intelligence (AAAI), pages 12484–12491, 2020. 2

    2020

  58. [67]

    U2fusion: A unified unsupervised image fusion net- work

    Han Xu, Jiayi Ma, Junjun Jiang, Xiaojie Guo, and Haibin Ling. U2fusion: A unified unsupervised image fusion net- work. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(1):502–518, 2022. 2

    2022

  59. [68]

    Rfnet: Unsupervised network for mutually reinforcing multi- modal image registration and fusion

    Han Xu, Jiayi Ma, Jiteng Yuan, Zhuliang Le, and Wei Liu. Rfnet: Unsupervised network for mutually reinforcing multi- modal image registration and fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19647–19656, 2022. 2

    2022

  60. [69]

    Murf: Mutually re- inforcing multi-modal image registration and fusion

    Han Xu, Jiteng Yuan, and Jiayi Ma. Murf: Mutually re- inforcing multi-modal image registration and fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence,

  61. [70]

    HAM-MFN: hyperspec- tral and multispectral image multiscale fusion network with RAP loss

    Shuang Xu, Ouafa Amira, Junmin Liu, Chun-Xia Zhang, Jiangshe Zhang, and Guanghai Li. HAM-MFN: hyperspec- tral and multispectral image multiscale fusion network with RAP loss. IEEE Transactions on Geoscience and Remote Sensing, 58(7):4618–4628, 2020. 1

    2020

  62. [71]

    Towards reducing se- vere defocus spread effects for multi-focus image fusion via an optimization based strategy

    Shuang Xu, Lizhen Ji, Zhe Wang, Pengfei Li, Kai Sun, Chunxia Zhang, and Jiangshe Zhang. Towards reducing se- vere defocus spread effects for multi-focus image fusion via an optimization based strategy. IEEE Transactions Compu- tational Imaging, 6:1561–1570, 2020. 1

    2020

  63. [72]

    Infrared and visible im- age fusion via texture conditional generative adversarial net- work

    Yong Yang, Jiaxiang Liu, Shuying Huang, Weiguo Wan, Wenying Wen, and Juwei Guan. Infrared and visible im- age fusion via texture conditional generative adversarial net- work. IEEE Transactions on Circuits and Systems for Video Technology, 31(12):4771–4783, 2021. 1

    2021

  64. [73]

    Diff-if: Multi-modality image fusion via diffusion model with fusion knowledge prior

    Xunpeng Yi, Linfeng Tang, Hao Zhang, Han Xu, and Ji- ayi Ma. Diff-if: Multi-modality image fusion via diffusion model with fusion knowledge prior. Information Fusion , 110:102450, 2024. 2

    2024

  65. [74]

    Restormer: Efficient transformer for high-resolution image restoration

    Syed Waqas Zamir, Aditya Arora, Salman Khan, Mu- nawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 572...

    2022

  66. [75]

    Sdnet: A versatile squeeze-and- decomposition network for real-time image fusion

    Hao Zhang and Jiayi Ma. Sdnet: A versatile squeeze-and- decomposition network for real-time image fusion. Inter- national Journal of Computer Vision , 129(10):2761–2785,

  67. [76]

    Mrfs: Mutually reinforcing image fusion and segmenta- tion

    Hao Zhang, Xuhui Zuo, Jie Jiang, Chunchao Guo, and Jiayi Ma. Mrfs: Mutually reinforcing image fusion and segmenta- tion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 26974– 26983, 2024. 1, 2, 3, 6, 7

    2024

  68. [77]

    IFCNN: A general image fusion framework based on convolutional neural network

    Yu Zhang, Yu Liu, Peng Sun, Han Yan, Xiaolin Zhao, and Li Zhang. IFCNN: A general image fusion framework based on convolutional neural network. Information Fusion, 54: 99–118, 2020. 1, 2

    2020

  69. [78]

    Metafusion: Infrared and visible image fusion via meta- feature embedding from object detection

    Wenda Zhao, Shigeng Xie, Fan Zhao, You He, and Huchuan Lu. Metafusion: Infrared and visible image fusion via meta- feature embedding from object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13955–13965, 2023. 1, 2, 3

    2023

  70. [79]

    Tufusion: A transformer-based universal fusion algorithm for multimodal images

    Yangyang Zhao, Qingchun Zheng, Peihao Zhu, Xu Zhang, and Wenpeng Ma. Tufusion: A transformer-based universal fusion algorithm for multimodal images. IEEE Transactions on Circuits and Systems for Video Technology, 2023. 1

    2023

  71. [80]

    Didfuse: Deep image de- composition for infrared and visible image fusion

    Zixiang Zhao, Shuang Xu, Chunxia Zhang, Junmin Liu, Jiangshe Zhang, and Pengfei Li. Didfuse: Deep image de- composition for infrared and visible image fusion. In Pro- ceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI), pages 970–976, 2020. 2

    2020

  72. [81]

    Efficient and model- based infrared and visible image fusion via algorithm un- rolling

    Zixiang Zhao, Shuang Xu, Jiangshe Zhang, Chengyang Liang, Chunxia Zhang, and Junmin Liu. Efficient and model- based infrared and visible image fusion via algorithm un- rolling. IEEE Transactions on Circuits and Systems for Video Technology, 32(3):1186–1196, 2022. 2

    2022

  73. [82]

    Cddfuse: Correlation-driven dual-branch feature decompo- sition for multi-modality image fusion

    Zixiang Zhao, Haowen Bai, Jiangshe Zhang, Yulun Zhang, Shuang Xu, Zudi Lin, Radu Timofte, and Luc Van Gool. Cddfuse: Correlation-driven dual-branch feature decompo- sition for multi-modality image fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern...

    2023

  74. [83]

    Cddfuse: Correlation-driven dual-branch feature decompo- sition for multi-modality image fusion

    Zixiang Zhao, Haowen Bai, Jiangshe Zhang, Yulun Zhang, Shuang Xu, Zudi Lin, Radu Timofte, and Luc Van Gool. Cddfuse: Correlation-driven dual-branch feature decompo- sition for multi-modality image fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern...

    2023

  75. [84]

    DDFM: denoising dif- fusion model for multi-modality image fusion

    Zixiang Zhao, Haowen Bai, Yuanzhi Zhu, Jiangshe Zhang, Shuang Xu, Yulun Zhang, Kai Zhang, Deyu Meng, Radu Timofte, and Luc Van Gool. DDFM: denoising dif- fusion model for multi-modality image fusion. CoRR, abs/2303.06840, 2023. 2

    2023 arXiv

  76. [85]

    Deep convolutional sparse cod- ing networks for interpretable image fusion

    Zixiang Zhao, Jiang-She Zhang, Haowen Bai, Yicheng Wang, Yukun Cui, Lilun Deng, Kai Sun, Chunxia Zhang, Junmin Liu, and Shuang Xu. Deep convolutional sparse cod- ing networks for interpretable image fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- ...

  77. [86]

    Equivariant multi-modality image fusion

    Zixiang Zhao, Haowen Bai, Jiangshe Zhang, Yulun Zhang, Kai Zhang, Shuang Xu, Dongdong Chen, Radu Timofte, and Luc Van Gool. Equivariant multi-modality image fusion. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 25912–25921,

  78. [87]

    Semantic-supervised infrared and visible image fusion via a dual-discriminator generative adversarial net- work

    Huabing Zhou, Wei Wu, Yanduo Zhang, Jiayi Ma, and Haibin Ling. Semantic-supervised infrared and visible image fusion via a dual-discriminator generative adversarial net- work. IEEE Transactions on Multimedia, 25:635–648, 2023. 1, 2

    2023

  79. [88]

    Probing synergistic high-order in- teraction for multi-modal image fusion

    Man Zhou, Naishan Zheng, Xuanhua He, Danfeng Hong, and Jocelyn Chanussot. Probing synergistic high-order in- teraction for multi-modal image fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 2

    2024

  80. [89]

    Task- customized mixture of adapters for general image fusion

    Pengfei Zhu, Yang Sun, Bing Cao, and Qinghua Hu. Task- customized mixture of adapters for general image fusion. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 7099–7108, 2024. 1, 2, 3

    2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.