Pith. sign in

REVIEW 4 major objections 6 minor 69 references

DM-FNet: Unified multimodal medical image fusion via diffusion process-trained encoder-decoder

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A two-stage diffusion-trained network fuses MRI-CT, MRI-PET, and MRI-SPECT images with a single set of parameters.

desk verdict Useful idea with a real methodological flaw: the SOTA claim rests on hyperparameters chosen on the same test sets used for the final comparison, so it needs a validation split before the numbers can be trusted. read the letter →

arxiv 2506.15218 v1 pith:3CMJQX5K submitted 2025-06-18 cs.CV

Dan He , Weisheng Li , Guofen Wang , Yuping Huang , Shiqiang Liu This is my paper · ORCID
classification cs.CV
keywords medicalimagefusiondiffusionmodelencoder-decoderattentionmechanismmultiscaleunifiedreconstructionmultimodalimaging
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a two-stage network, DM-FNet, can fuse structurally and functionally different medical image pairs (MRI-CT, MRI-PET, MRI-SPECT) with one set of parameters. The first stage trains a UNet through a denoising diffusion process, so the encoder learns fine anatomical detail through progressive denoising. The second stage feeds the fusion network the same source images corrupted by Gaussian noise at several time steps, then combines multiscale, cross-modal features with attention-guided fusion modules. On three medical test sets, the paper reports that this design outperforms nine previous fusion methods on most of nine quality metrics, and that the same model transfers to phase-contrast and green-fluorescent-protein fusion without fine-tuning. The appeal is a single unified fusion model rather than task-specific weights.

What carries the argument

The load-bearing mechanism is the diffusion-trained UNet encoder used as a fixed feature extractor. In Stage I, a denoising diffusion probabilistic model trains the UNet to predict the previous denoising step, forcing it to represent multilevel image structure; in Stage II the fusion network receives source images with Gaussian noise added at time steps 5, 10, and 20, so the same encoder sees each image at three noise levels. The attention-guided multimodal feature fusion module (AMFF) computes spatial, channel, and pixel attention maps that weight each modality's features, and the multiscale feature fusion module (MSFF) hierarchically combines the five encoder scales. The hybrid loss, which combines an intensity term, a maximum-local-standard-deviation SSIM term, and a Sobel gradient term, drives the final trade-off between luminance and detail.

What would settle it

Retrain the Stage I encoder with the same UNet and a plain reconstruction loss (no diffusion schedule) while keeping Stage II identical, and compare on the three test sets; if the non-diffusion encoder matches or exceeds DM-FNet on the nine metrics, the diffusion process itself is not the load-bearing component.

Watch

Extended reading notes

Core claim

The central claim is that diffusion-process training can serve as a feature-extraction engine for image fusion, not as a slow generative sampler. DM-FNet first trains a UNet to reconstruct clean medical images by removing Gaussian noise step by step; that training forces the network to encode the fine structures that noise masks, such as bone edges, soft tissue boundaries, and tumor margins. The fusion stage then corrupts each source image at time steps 5, 10, and 20, encodes those noisy versions with the frozen diffusion-trained UNet, and merges the resulting multilevel features using three modules: a feature-fusion step that combines the different time-step encodings, an attention-guided multimodal feature fusion module (AMFF) that blends spatial, channel, and pixel attention across the two modalities, and a multiscale feature fusion module (MSFF) that integrates the five UNet scales. A hybrid loss combining intensity, SSIM, and gradient terms balances brightness and detail. The paper reports that this unified architecture achieves the best or second-best scores on most of nine metrics across MRI-CT, MRI-PET, and MRI-SPECT, and that removing the diffusion process or either fusion module measurably degrades results.

Load-bearing premise

The paper's central assumption is that feeding the fusion network source images corrupted with Gaussian noise at time steps 5, 10, and 20 produces features from the diffusion-trained UNet that are genuinely more informative than features from clean inputs or from a conventionally trained encoder.

Editorial extensions

If this is right

  • One trained parameter set suffices for MRI-CT, MRI-PET, and MRI-SPECT fusion, so new clinical fusion tasks need not retrain a dedicated model.
  • Because the diffusion forward process runs only in training, fusion at test time avoids the slow iterative sampling of generative diffusion fusers such as DDFM or FusionDiff.
  • The same fusion network extends to infrared-visible fusion after fine-tuning and to PC-GFP fusion without fine-tuning, indicating that the diffusion-trained features generalize across modality types.
  • Removing the diffusion process or either fusion module lowers most metrics, implying each component contributes to the reported quality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One testable extension is whether the multi-time-step noise schedule acts purely as data augmentation: a simpler stochastic noise augmentation without the diffusion reconstruction objective might match DM-FNet if the diffusion objective itself is not essential.
  • Feeding noisy inputs at inference time is unusual; comparing DM-FNet against the same fusion network fed only clean source images would isolate whether the noise corruption, rather than the diffusion-trained weights, drives the reported gain.
  • Because nothing in the method is medical-specific except the training data, the same design could transfer to other fusion domains such as multi-focus, multi-exposure, or remote-sensing fusion where one unified model is desirable.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes DM-FNet, a two-stage framework for multimodal medical image fusion. Stage I trains a UNet denoiser with a diffusion process for image reconstruction; Stage II feeds the trained UNet with source images corrupted by Gaussian noise at three selected time steps (t=5, 10, 20) and uses attention-based multimodal feature fusion (AMFF) and multiscale feature fusion (MSFF) to combine encoder-decoder features and produce the fused image. A hybrid loss with intensity, SSIM, and gradient terms is used. Experiments on MRI-CT, MRI-PET, and MRI-SPECT datasets compare against nine methods; the authors also report generalization to PC-GFP without fine-tuning and to infrared-visible fusion after fine-tuning. The code is publicly available.

Significance. If the evaluation were clean, the contribution would be meaningful: the paper introduces a nontrivial two-stage design that uses diffusion training as a feature extractor and demonstrates a single parameter set across three medical fusion tasks, with public code and ablations. The multi-time-step feature fusion and the module design are clearly described, and the extension to PC-GFP without fine-tuning is a good generalization test. However, the current evidence for the central claim is weakened by evaluation-protocol issues that must be resolved before the reported improvements can be attributed to the method rather than to selection on the evaluation target.

major comments (4)
  1. [Section IV-A/B (Tables I and II)] The hyperparameter analysis in Section IV-B selects the intensity/SSIM weights (α=1.5, β=0.5) and the time-step combination (t1=5, t2=10, t3=20) by averaging metrics over the three test sets (Tables I and II), and then Tables V and VII report final comparisons computed on the same test sets. Because the test sets were used to choose the configuration, the reported margins over the nine baselines may reflect selection on the evaluation target rather than a genuine advantage. A held-out validation split or nested cross-validation is required before the state-of-the-art claim can be assessed.
  2. [Section IV-B (Table III)] The ablation labeled "w/o Dif." removes the diffusion process in Stage I and simultaneously removes the multi-time-step noisy inputs in Stage II ("the fusion module of different noise conditions was removed"). This conflates the effect of diffusion-based training with the effect of the multi-time-step input strategy. To attribute the improvement to diffusion-process training, the paper should compare against a control in which a conventionally trained UNet is fed the same multi-time-step noisy inputs, or in which the diffusion-trained UNet is used with a single time step.
  3. [Section IV-C (Tables V and VII)] The central comparison rests on point estimates over a single train/test split with only 90 training pairs and 50 test pairs per modality. Several winning margins are small (e.g., Table V: Proposed SCD 1.508 vs Cloud 1.378 and VIFF 0.600 vs 0.570; Table VII: Proposed Q_W 0.925 vs SHIP 0.926, where SHIP is slightly higher). No error bars, confidence intervals, or significance tests are reported, so the "outperforming" claim is not yet robust. At minimum, the authors should report results over multiple seeds or provide statistical tests.
  4. [Section III-C and IV-A] The forward diffusion process is defined with a total of T steps and a variance schedule α_t (Eqs. (1)-(2)), and Stage II uses t=5, 10, 20, but the paper never reports the value of T or the specific schedule used. Because the diffusion-trained UNet is the foundation of the feature extractor, the absence of these values prevents reproduction of Stage I and should be corrected.
minor comments (6)
  1. [Section III-E] The section heading "Loss fuction" should read "Loss function."
  2. [Section IV-A and throughout] The acronym "SOAT" appears where "SOTA" (state of the art) is intended; please correct it throughout, including "nine SOAT methods" in the experimental sections and figure captions.
  3. [Section IV-A and Section IV-C] The comparative method is introduced as "TL-SR" in Section IV-A but is referred to as "TLSR" in later sections and tables; please unify the naming.
  4. [Section IV-B (paragraph on the diffusion process)] The sentence "Most of the evaluation metrics across all test sets in Table V were also noticeably reduced" appears to refer to Table III, not Table V; please correct the cross-reference.
  5. [Section IV-B (Table IV discussion)] The statement "removing any module results in a significant performance drop" is not fully supported by Table IV: for MRI-CT, "w/o MSFF" yields higher AG (9.237 vs 9.000) and higher Q_AB/F (0.586 vs 0.571), and "w/o AMFF" yields higher Q_AB/F (0.588 vs 0.571). Please qualify this claim or explain how these increases are consistent with the stated conclusion.
  6. [Equation (4) and color-space descriptions] The notation IY_{B,t} introduces a superscript Y that is not defined when I_B is CT (grayscale). Clarify how the YCbCr luminance channel is used for CT as well as for PET/SPECT inputs.

Circularity Check

1 steps flagged · score 4.0 of 10

Final comparison rests on hyperparameters selected on the same test sets; the SOTA claim is partly a selection artifact.

  1. fitted input called prediction [Section IV-A (Implementation details) and Section IV-B (Tables I and II); final comparisons in Section IV-C (Tables V and VII)]
    "The three test sets each contained 50 pairs of images for MRI-CT, MRI-PET, and MRI-SPECT. ... Table I: Objective evaluation for different values of α and β (averaged over three test sets). ... After considering the visual effects and quantitative analysis, we choose α=1.5 and β=0.5. ... Table II: Objective evaluation for different time-step combinations (averaged over three test sets). ... the combination of (5, 10, 20) yielding the best results."

    The final SOTA claims (Tables V/VII, Figs. 7/10) are computed on the same 50-pair test sets whose metrics were used to select the configuration: α, β, and t1=5,t2=10,t3=20 were chosen because they gave the best/red values in Tables I and II, which are explicitly 'averaged over three test sets.' Thus the later 'prediction' that the chosen configuration outperforms baselines on those test sets is not an independent evaluation; it is the output of a selection procedure that tuned the model on the evaluation target. A held-out validation split or nested cross-validation is needed before the comparison can be read as evidence for the claimed advantage.

full rationale

The core derivation is not self-definitional: the fusion result is produced by a network trained with explicit intensity, gradient, and SSIM losses, and no predicted variable is defined as a fitted parameter. The diffusion feature extractor and attention modules are empirical design choices supported by ablations. Self-citations (LRFNet, Cloud, DSAGAN, MMIF-INet, DFENet) are baselines or related work and are not load-bearing for the derivation; no uniqueness theorem or ansatz is smuggled via citation. The main circular component is the test-set hyperparameter selection: Tables I and II are used to fix α, β, and the time steps, and the same test sets then produce the final comparison tables. This makes the reported advantage partially reflect selection on the evaluation target rather than an independent prediction. A separate validation split would remove this circular component; with the current protocol the SOTA claim is not fully independently assessed.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical entities or forces. The free parameters are loss weights and time-step selections, all chosen empirically on the test sets. The axioms are standard diffusion model assumptions plus two domain-specific modeling premises that are load-bearing for the method's design.

free parameters (3)
  • alpha (intensity loss weight) = 1.5
    Selected based on highest averaged objective metrics over the three test sets, as shown in Table I.
  • beta (SSIM loss weight) = 0.5
    Selected alongside alpha using test-set metrics in Table I.
  • Diffusion time steps for fusion inputs = 5, 10, 20
    The combination (5, 10, 20) was chosen after comparing seven time-step settings on the test sets, as shown in Table II.
assumptions (3)
  • standard math The Gaussian diffusion forward process (Eqs. 1-2) and the UNet reverse process (Eq. 3) are valid for medical images.
    Taken from the standard DDPM formulation [46]; the paper relies on this without modification.
  • ad hoc to paper Diffusion features extracted at time steps 5, 10, and 20 capture complementary detailed information useful for fusion.
    This is the central design premise of Stage II, supported only by ablations in Section IV-B and not by a theoretical argument.
  • domain assumption The ideal fused image's intensity and gradient equal the per-pixel maximum of the source images (Eqs. 15 and 18).
    This max-based loss is a strong modeling choice that biases the fused image toward high-brightness regions; it is not derived from any clinical or perceptual principle.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DM-FNet: Unified multimodal medical image fusion via diffusion process-trained encoder-decoder." pith.science (2026). https://pith.science/paper/3CMJQX5K

@misc{pith2026250615218,
  author       = {Pith},
  title        = {Pith review of: DM-FNet: Unified multimodal medical image fusion via diffusion process-trained encoder-decoder},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3CMJQX5K}},
  note         = {Machine review of arXiv:2506.15218}
}
read the original abstract

Multimodal medical image fusion (MMIF) extracts the most meaningful information from multiple source images, enabling a more comprehensive and accurate diagnosis. Achieving high-quality fusion results requires a careful balance of brightness, color, contrast, and detail; this ensures that the fused images effectively display relevant anatomical structures and reflect the functional status of the tissues. However, existing MMIF methods have limited capacity to capture detailed features during conventional training and suffer from insufficient cross-modal feature interaction, leading to suboptimal fused image quality. To address these issues, this study proposes a two-stage diffusion model-based fusion network (DM-FNet) to achieve unified MMIF. In Stage I, a diffusion process trains UNet for image reconstruction. UNet captures detailed information through progressive denoising and represents multilevel data, providing a rich set of feature representations for the subsequent fusion network. In Stage II, noisy images at various steps are input into the fusion network to enhance the model's feature recognition capability. Three key fusion modules are also integrated to process medical images from different modalities adaptively. Ultimately, the robust network structure and a hybrid loss function are integrated to harmonize the fused image's brightness, color, contrast, and detail, enhancing its quality and information density. The experimental results across various medical image types demonstrate that the proposed method performs exceptionally well regarding objective evaluation metrics. The fused image preserves appropriate brightness, a comprehensive distribution of radioactive tracers, rich textures, and clear edges. The code is available at https://github.com/HeDan-11/DM-FNet.

Figures

Figures reproduced from arXiv: 2506.15218 by the authors.

Figure 1
Figure 1. Performance comparison of DM-FNet with existing methods. (a) DM-FNet performs well on nine objective eval￾uation metrics. (b) The fusion results demonstrate that DM￾FNet successfully achieves unified fusion in all three datasets; this includes preserving the cranial region in the MRI and CT image fusion, as well as preserving appropriate color and luminance in the MRI and functional image fusion. resonance imaging (… view at source ↗
Figure 2
Figure 2. Overall architecture of the diffusion model-based fusion network (DM-FNet). In Stage I, UNet is trained to reconstruct images through the diffusion process. In Stage II, the fusion network comprises three components: feature fusion under different noisy images, attention-guided multimodal feature fusion (AMFF), and multiscale feature fusion (MSFF). This analysis aims to achieve unified fusion while balancing the bri… view at source ↗
Figure 3
Figure 3. Network structure of the AMFF. f i B,t = φ3(φ1([H Di B,t1 , ..., HDi B,tn ])) (7) where φ3 is a 3 × 3 conv and where φ1 is a 1 × 1 conv. After the above steps, we obtain feature maps at different scales in different modes. Attention-guided multimodal feature fusion (AMFF). We introduce an AMFF module [54], designated A, designed to extract and fuse key information from different modalities in a hierarchical and fine… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Fusion results for different values of α and β. TABLE I: Objective evaluation for different values of α and β (averaged over three test sets). Red: best, blue: second best. Experiments SF SD AG QW SCD VIFF QAB/F MSSSIM FMI WT Only Lint 27.933 77.543 7.165 0.805 1.445 0…
Figure 5
Figure 5. Figure 5: Visualization analysis without diffusion process. fusion of multimodal features via the addition operation, whereas ”w/o MSFF” involves the use of only the features from the final layer. Table IV shows that removing any module results in a significant performance drop.…
Figure 6
Figure 6. Figure 6: Visual comparison of DM-FNet with 9 SOAT methods for MRI and CT image fusion. The regions are enlarged for a more intuitive comparison, using close-ups and pseudocolor maps for better observation [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Quantitative comparison of DM-FNet with 9 SOAT methods for MRI and CT image fusion. The legend on the right displays the average scores for different methods. TABLE V: Quantitative comparison of DM-FNet with 9 SOAT methods for MRI-PET and MRI-SPECT image fusion. The up…
Figure 8
Figure 8. Figure 8: Visual comparison of DM-FNet with 9 SOAT methods for MRI-PET and MRI-SPECT image fusion. For a more intuitive comparison, the regions are enlarged as close-ups. fusion, EMFusion, SDNet, MATR, and CDDFuse exhibited incomplete color recovery and poorly bright regions. Ms…
Figure 9
Figure 9. Figure 9: Visual comparison of DM-FNet with 9 SOAT methods for PC and GFP image fusion. For a more intuitive comparison, the regions are enlarged as close-ups [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Quantitative comparison of DM-FNet with 9 SOAT methods for PC and GFP image fusion. The legend on the right displays the average scores for different methods. high-quality fusion results and relatively efficient fusion time of DM-FNet make it a highly effective and re…
Figure 11
Figure 11. Figure 11 [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 67 canonical work pages

  1. [1]

    Liu, and Yansong Duan

    Wei Tang, Fazhi He, Y . Liu, and Yansong Duan. Matr: Multimodal medical image fusion via multiscale adaptive transformer.IEEE Trans- actions on Image Processing, 31:5134–5149, 2022

    work page 2022

  2. [2]

    Lrfnet: A real-time medical image fusion method guided by detail information.Computers in biology and medicine, 173:108381, 2024

    Dan He, Weisheng Li, Guofen Wang, Yuping Huang, and Shiqiang Liu. Lrfnet: A real-time medical image fusion method guided by detail information.Computers in biology and medicine, 173:108381, 2024. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 13

    work page 2024

  3. [3]

    On the use of the sos metaheuristic algorithm in hybrid image fusion methods to achieve optimum spectral fidelity.International Journal of Remote Sensing, 41:3993 – 4021, 2020

    Cigdem Serifoglu Yilmaz, V olkan Murat Yılmaz, and Oguz G ¨ung¨or. On the use of the sos metaheuristic algorithm in hybrid image fusion methods to achieve optimum spectral fidelity.International Journal of Remote Sensing, 41:3993 – 4021, 2020

    work page 2020

  4. [4]

    Joint image fusion and denoising via three-layer decomposition and sparse representation

    Xiaosong Li, Fuqiang Zhou, and Haishu Tan. Joint image fusion and denoising via three-layer decomposition and sparse representation. Knowl. Based Syst., 224:107087, 2021

    work page 2021

  5. [5]

    F-darts: Foveated differentiable architecture search based multimodal medical image fusion.IEEE Transactions on Medical Imaging, 42:3348–3361, 2023

    Shaozhuang Ye, Tuo Wang, Mingyue Ding, and Xuming Zhang. F-darts: Foveated differentiable architecture search based multimodal medical image fusion.IEEE Transactions on Medical Imaging, 42:3348–3361, 2023

    work page 2023

  6. [6]

    Ifsepr: A general framework for image fusion based on separate representation learning.IEEE Transactions on Multimedia, 25:608–623, 2023

    Xiaoqing Luo, Yuanhao Gao, Anqi Wang, Zhancheng Zhang, and Xiaojun Wu. Ifsepr: A general framework for image fusion based on separate representation learning.IEEE Transactions on Multimedia, 25:608–623, 2023

    work page 2023

  7. [7]

    Multimodal medical image fusion based on multichannel coupled neural p systems and max-cloud models in spectral total variation domain

    Guofen Wang, Weisheng Li, Xinbo Gao, Bin Xiao, and Jiao Du. Multimodal medical image fusion based on multichannel coupled neural p systems and max-cloud models in spectral total variation domain. Neurocomputing, 480:61–75, 2022

    work page 2022

  8. [8]

    Gaurav Bhatnagar, Q. M. Jonathan Wu, and Zheng Liu. Directive contrast based multimodal medical image fusion in nsct domain.IEEE Transactions on Multimedia, 15(5):1014–1024, 2013

    work page 2013

Show all 69 references
  1. [9]

    A novel approach based on grasshopper optimization algorithm for medical image fusion.Expert Syst

    Phu-Hung Dinh. A novel approach based on grasshopper optimization algorithm for medical image fusion.Expert Syst. Appl., 171:114576, 2021

    2021

  2. [10]

    A new approach to medical image fusion based on the improved extended difference-of-gaussians combined with the coati optimization algorithm.Biomed

    Thi-Hong-Ha Le, Phu-Hung Dinh, Van-Hieu Vu, and Long Giang Nguyen. A new approach to medical image fusion based on the improved extended difference-of-gaussians combined with the coati optimization algorithm.Biomed. Signal Process. Control., 93:106175, 2024

    2024

  3. [11]

    Densefuse: A fusion approach to infrared and visible images.IEEE TIP, 28:2614–2623, 2018

    Hui Li and Xiaojun Wu. Densefuse: A fusion approach to infrared and visible images.IEEE TIP, 28:2614–2623, 2018

    2018

  4. [12]

    Ifcnn: A general image fusion framework based on convolutional neural network.Inf

    Yu Zhang, Yu Liu, Peng Sun, Han Yan, Xiaolin Zhao, and Li Zhang. Ifcnn: A general image fusion framework based on convolutional neural network.Inf. Fusion, 54:99–118, 2020

    2020

  5. [13]

    Dfenet: A dual-branch feature enhanced network integrating transform- ers and convolutional feature learning for multimodal medical image fusion.Biomed

    Weisheng Li, Yin Zhang, Guofen Wang, Yuping Huang, and Ruyue Li. Dfenet: A dual-branch feature enhanced network integrating transform- ers and convolutional feature learning for multimodal medical image fusion.Biomed. Signal Process. Control., 80:104402, 2023

    2023

  6. [14]

    Fu, Baiqing He, Jie Yang, Jianpeng Liu, Aijia Ouyang, and Ya Wang

    J. Fu, Baiqing He, Jie Yang, Jianpeng Liu, Aijia Ouyang, and Ya Wang. Cdrnet: Cascaded dense residual network for grayscale and pseudocolor medical image fusion.Computer methods and programs in biomedicine, 234:107506, 2023

    2023

  7. [15]

    Multi-level difference information replenishment for medical image fusion.Applied Intelligence, 53:4579–4591, 2022

    Luping Chen, Xue Wang, Yan Zhu, and Rencan Nie. Multi-level difference information replenishment for medical image fusion.Applied Intelligence, 53:4579–4591, 2022

    2022

  8. [16]

    Mactfusion: Lightweight cross transformer for adaptive multimodal medical image fusion.IEEE journal of biomedical and health informatics, PP, 2024

    Xinyu Xie, Xiaozhi Zhang, Xinglong Tang, Jiaxi Zhao, Dongping Xiong, Lijun Ouyang, Bin Yang, Hong Zhou, Bingo Wing-Kuen Ling, and Kok-Lay Teo. Mactfusion: Lightweight cross transformer for adaptive multimodal medical image fusion.IEEE journal of biomedical and health informati...

    2024

  9. [17]

    Liu, Yansong Duan, and Tongzhen Si

    Wei Tang, Fazhi He, Y . Liu, Yansong Duan, and Tongzhen Si. Datfuse: Infrared and visible image fusion via dual attention transformer.IEEE Transactions on Circuits and Systems for Video Technology, 33:3159– 3172, 2023

    2023

  10. [18]

    Wei Tang, Fazhi He, and Y . Liu. Ydtr: Infrared and visible image fusion via y-shape dynamic transformer.IEEE Transactions on Multimedia, 25:5413–5428, 2023

    2023

  11. [19]

    Xinyu Xie, Xiaozhi Zhang, Shengcheng Ye, Dongping Xiong, Lijun Ouyang, Bin Yang, Hong Zhou, and Yaping Wan. Mrscfusion: Joint residual swin transformer and multiscale cnn for unsupervised multi- modal medical image fusion.IEEE Transactions on Instrumentation and Measurement, 7...

    2023

  12. [20]

    Fusionmamba: Dynamic feature enhancement for multimodal image fusion with mamba.ArXiv, abs/2404.09498, 2024

    Xinyu Xie, Yawen Cui, Chio in Ieong, Tao Tan, Xiaozhi Zhang, Xubin Zheng, and Zitong Yu. Fusionmamba: Dynamic feature enhancement for multimodal image fusion with mamba.ArXiv, abs/2404.09498, 2024

    2024 arXiv

  13. [21]

    Dsagan: A generative adversarial network based on dual-stream attention mechanism for anatomical and functional image fusion.Inf

    Jun Fu, Weisheng Li, Jiao Du, and Liming Xu. Dsagan: A generative adversarial network based on dual-stream attention mechanism for anatomical and functional image fusion.Inf. Sci., 576:484–506, 2021

    2021

  14. [22]

    Ddcgan: A dual-discriminator conditional generative adversarial network for multi-resolution image fusion.IEEE Transactions on Image Processing, 29:4980–4995, 2020

    Jiayi Ma, Han Xu, Junjun Jiang, Xiaoguang Mei, and Xiao-Ping Zhang. Ddcgan: A dual-discriminator conditional generative adversarial network for multi-resolution image fusion.IEEE Transactions on Image Processing, 29:4980–4995, 2020

    2020

  15. [23]

    Unified gradient- and intensity-discriminator generative adversarial net- work for image fusion.Inf

    Huabing Zhou, Jilei Hou, Yanduo Zhang, Jiayi Ma, and Haibin Ling. Unified gradient- and intensity-discriminator generative adversarial net- work for image fusion.Inf. Fusion, 88:184–201, 2022

    2022

  16. [24]

    Glioma segmentation-oriented multi-modal mr image fusion with adversarial learning.IEEE CAA J

    Yu Liu, Yu Shi, Fuhao Mu, Juan Cheng, and Xun Chen. Glioma segmentation-oriented multi-modal mr image fusion with adversarial learning.IEEE CAA J. Autom. Sinica, 9:1528–1531, 2022

    2022

  17. [25]

    Coconet: Coupled contrastive learning network with multi-level feature ensemble for multi-modality image fusion.IJCV ., 132:1748– 1775, 2022

    Jinyuan Liu, Runji Lin, Guanyao Wu, Risheng Liu, Zhongxuan Luo, and Xin Fan. Coconet: Coupled contrastive learning network with multi-level feature ensemble for multi-modality image fusion.IJCV ., 132:1748– 1775, 2022

    2022

  18. [26]

    Self-supervised fusion for multi-modal medical images via contrastive auto-encoding and convolutional information exchange.IEEE Computational Intelligence Magazine, 18:68–80, 2023

    Ying Zhang, Ren qi Nie, Jinde Cao, and Chaozhen Ma. Self-supervised fusion for multi-modal medical images via contrastive auto-encoding and convolutional information exchange.IEEE Computational Intelligence Magazine, 18:68–80, 2023

    2023

  19. [27]

    Zixiang Zhao, Hao Bai, Jiangshe Zhang, Yulun Zhang, Shuang Xu, Zudi Lin, Radu Timofte, and Luc Van Gool. Cddfuse: Correlation-driven dual- branch feature decomposition for multi-modality image fusion.2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p...

    2023

  20. [28]

    A general paradigm with detail-preserving conditional invertible network for image fusion.International Journal of Computer Vision, 132(4):1029–1054, 2024

    Wu Wang, Liang-Jian Deng, Ran Ran, and Gemine Vivone. A general paradigm with detail-preserving conditional invertible network for image fusion.International Journal of Computer Vision, 132(4):1029–1054, 2024

    2024

  21. [29]

    Mmif-inet: Multimodal medical image fusion by invertible network.Inf

    Dan He, Weisheng Li, Guofen Wang, Yuping Huang, and Shiqiang Liu. Mmif-inet: Multimodal medical image fusion by invertible network.Inf. Fusion, 114:102666, 2024

    2024

  22. [30]

    An efficient approach to medical image fusion based on optimization and transfer learning with vgg19.Biomed

    Oanh Cuong Do, Chi Mai Luong, Phu-Hung Dinh, and Giang Son Tran. An efficient approach to medical image fusion based on optimization and transfer learning with vgg19.Biomed. Signal Process. Control., 87:105370, 2024

    2024

  23. [31]

    Mif-btf-mrn: Medical image fusion based on the bilateral texture filter and transfer learning with the resnet-101 network

    Phu-Hung Dinh. Mif-btf-mrn: Medical image fusion based on the bilateral texture filter and transfer learning with the resnet-101 network. Biomed. Signal Process. Control., 100:106976, 2025

    2025

  24. [32]

    Task-customized mixture of adapters for general image fusion.2024 IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), pages 7099–7108, 2024

    Pengfei Zhu, Yang Sun, Bing Cao, and Qinghua Hu. Task-customized mixture of adapters for general image fusion.2024 IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), pages 7099–7108, 2024

    2024

  25. [33]

    Zhang, Ren qi Nie, Jinde Cao, Chaozhen Ma, and Chengchao Wang

    Y . Zhang, Ren qi Nie, Jinde Cao, Chaozhen Ma, and Chengchao Wang. Ss-ssan: a self-supervised subspace attentional network for multi-modal medical image fusion.Artificial Intelligence Review, 56:421 – 443, 2023

    2023

  26. [34]

    Emfusion: An unsupervised enhanced medical image fusion network.Inf

    Han Xu and Jiayi Ma. Emfusion: An unsupervised enhanced medical image fusion network.Inf. Fusion, 76:177–186, 2021

    2021

  27. [35]

    Gesenet: A general semantic-guided network with couple mask ensemble for medical image fusion.IEEE transactions on neural networks and learning systems, PP, 2023

    Jiawei Li, Jinyuan Liu, Shihua Zhou, Qian Zhang, and Nikola Kirilov Kasabov. Gesenet: A general semantic-guided network with couple mask ensemble for medical image fusion.IEEE transactions on neural networks and learning systems, PP, 2023

    2023

  28. [36]

    Sdnet: A versatile squeeze-and-decomposition network for real-time image fusion.International Journal of Computer Vision, 129:2761 – 2785, 2021

    Hao Zhang and Jiayi Ma. Sdnet: A versatile squeeze-and-decomposition network for real-time image fusion.International Journal of Computer Vision, 129:2761 – 2785, 2021

    2021

  29. [37]

    Zhang, Deyu Meng, Radu Timofte, and Luc Van Gool

    Zixiang Zhao, Hao Bai, Yuanzhi Zhu, Jiangshe Zhang, Shuang Xu, Yulun Zhang, K. Zhang, Deyu Meng, Radu Timofte, and Luc Van Gool. Ddfm: Denoising diffusion model for multi-modality image fusion.2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 8048–8059, 2023

    2023

  30. [38]

    Fusiondiff: Multi-focus image fusion using denoising diffusion probabilistic models.Expert Syst

    Mining Li, Ronghao Pei, Tianyou Zheng, Yang Zhang, and Weiwei Fu. Fusiondiff: Multi-focus image fusion using denoising diffusion probabilistic models.Expert Syst. Appl., 238:121664, 2023

    2023

  31. [39]

    Dif-fusion: Toward high color fidelity in infrared and visible image fusion with diffusion models.IEEE Transactions on Image Processing, 32:5705– 5720, 2023

    Jun Yue, Leyuan Fang, Shaobo Xia, Yue Deng, and Jiayi Ma. Dif-fusion: Toward high color fidelity in infrared and visible image fusion with diffusion models.IEEE Transactions on Image Processing, 32:5705– 5720, 2023

    2023

  32. [40]

    Swinfusion: Cross-domain long-range learning for general image fusion via swin transformer.IEEE/CAA Journal of Automatica Sinica, 9:1200–1217, 2022

    Jiayi Ma, Linfeng Tang, Fan Fan, Jun Huang, Xiaoguang Mei, and Yong Ma. Swinfusion: Cross-domain long-range learning for general image fusion via swin transformer.IEEE/CAA Journal of Automatica Sinica, 9:1200–1217, 2022

    2022

  33. [41]

    Jinyu Wen, Fei wei Qin, Jiao Du, Mei Fang, Xinhua Wei, C. L. Philip Chen, and Ping Li. Msgfusion: Medical semantic guided two-branch network for multimodal brain image fusion.IEEE Transactions on Multimedia, 26:944–957, 2024

    2024

  34. [42]

    Ct and mri medical image fusion using noise-removal and contrast enhancement scheme with convolutional neural network.Entropy, 24, 2022

    Jameel Ahmed Bhutto, Lianfang Tian, Qiliang Du, Zheng-Chun Sun, Yu Lubin, and Muhammad Faizan Tahir. Ct and mri medical image fusion using noise-removal and contrast enhancement scheme with convolutional neural network.Entropy, 24, 2022

    2022

  35. [43]

    Fatfusion: A functional–anatomical transformer for medical image fusion.Information Processing & Management, 61(4):103687, 2024

    Wei Tang and Fazhi He. Fatfusion: A functional–anatomical transformer for medical image fusion.Information Processing & Management, 61(4):103687, 2024

    2024

  36. [44]

    Jane Wang, and Xun Chen

    Yu Liu, Chen Yu, Juan Cheng, Z. Jane Wang, and Xun Chen. Mm-net: A mixformer-based multi-scale network for anatomical and functional image fusion.IEEE Transactions on Image Processing, 33:2197–2212, 2024

    2024

  37. [45]

    An improved hybrid JOURNAL OF LATEX CLASS FILES, VOL

    Yanyu Liu, Yongsheng Zang, Dongming Zhou, Jinde Cao, Rencan Nie, Ruichao Hou, Zhaisheng Ding, and Jiatian Mei. An improved hybrid JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 14 network with a transformer module for medical image fusion.IEEE Journal of Biomedical ...

    2021

  38. [46]

    Jonathan Ho, Ajay Jain, and P. Abbeel. Denoising diffusion probabilistic models.ArXiv, abs/2006.11239, 2020

    2006 arXiv

  39. [47]

    Diffusion- based network for unsupervised landmark detection.Knowl

    Tao Wu, Kai Wang, Chuanming Tang, and Jianlin Zhang. Diffusion- based network for unsupervised landmark detection.Knowl. Based Syst., 292:111627, 2024

    2024

  40. [48]

    Corrdiff: Corrective diffusion model for accurate mri brain tumor segmentation.IEEE Journal of Biomedical and Health Informatics, 28:1587–1598, 2024

    Wenqing Li, Wenhui Huang, and Yuanjie Zheng. Corrdiff: Corrective diffusion model for accurate mri brain tumor segmentation.IEEE Journal of Biomedical and Health Informatics, 28:1587–1598, 2024

    2024

  41. [49]

    Mikolaj Czerkawski and Christos Tachtatzis. Exploring the capability of text-to-image diffusion models with structural edge guidance for multispectral satellite image inpainting.IEEE Geoscience and Remote Sensing Letters, 21:1–5, 2023

    2023

  42. [50]

    Yang, and Chaoyue Wang

    Zuopeng Yang, Tianshu Chu, Xin Lin, Erdun Gao, Daqing Liu, J. Yang, and Chaoyue Wang. Eliminating contextual prior bias for semantic image editing via dual-cycle diffusion.IEEE Transactions on Circuits and Systems for Video Technology, 34:1316–1320, 2023

    2023

  43. [51]

    Acdmsr: Accelerated conditional diffusion models for single image super-resolution.IEEE Transactions on Broadcasting, 70:492–504, 2023

    Axi Niu, Trung Xuan Pham, Kang Zhang, Jinqiu Sun, Yu Zhu, Qingsen Yan, Inso Kweon, and Yanning Zhang. Acdmsr: Accelerated conditional diffusion models for single image super-resolution.IEEE Transactions on Broadcasting, 70:492–504, 2023

    2023

  44. [52]

    Hanqun Cao, Cheng Tan, Zhangyang Gao, Yilun Xu, Guangyong Chen, Pheng-Ann Heng, and Stan Z. Li. A survey on generative diffusion models.IEEE Transactions on Knowledge and Data Engineering, 36:2814–2830, 2022

    2022

  45. [53]

    Fleet, and Mohammad Norouzi

    Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J. Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45:4713–4726, 2021

    2021

  46. [54]

    Dea-net: Single image dehazing based on detail-enhanced convolution and content-guided attention.IEEE Transactions on Image Processing, 33:1002–1015, 2023

    Zixuan Chen, Zewei He, and Zhe ming Lu. Dea-net: Single image dehazing based on detail-enhanced convolution and content-guided attention.IEEE Transactions on Image Processing, 33:1002–1015, 2023

    2023

  47. [55]

    M4fnet: Multimodal medical image fusion network via multi-receptive-field and multi-scale feature integration.Computers in biology and medicine, 159:106923, 2023

    Zhaisheng Ding, Haiyan Li, Yi Guo, Dongming Zhou, Yanyu Liu, and Shidong Xie. M4fnet: Multimodal medical image fusion network via multi-receptive-field and multi-scale feature integration.Computers in biology and medicine, 159:106923, 2023

    2023

  48. [56]

    Eskicioglu and Paul S

    Ahmet M. Eskicioglu and Paul S. Fisher. Image quality measures and their performance.IEEE Trans. Commun., 43:2959–2965, 1995

    1995

  49. [57]

    In-fibre bragg grating sensors.Measurement Science and Technology, 8:355 – 375, 1997

    Yun jiang Rao. In-fibre bragg grating sensors.Measurement Science and Technology, 8:355 – 375, 1997

    1997

  50. [58]

    Detail preserved fusion of visible and infrared images using regional saliency extraction and multi-scale image decomposition.Optics Com- munications, 341:199–209, 2015

    Guangmang Cui, Huajun Feng, Zhi hai Xu, Qi Li, and Yue ting Chen. Detail preserved fusion of visible and infrared images using regional saliency extraction and multi-scale image decomposition.Optics Com- munications, 341:199–209, 2015

    2015

  51. [59]

    Gemma Piella and Henk J. A. M. Heijmans. A new quality metric for image fusion.Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429), 3:III–173, 2003

    2003

  52. [60]

    A new image quality metric for image fusion: The sum of the correlations of differences.Aeu- international Journal of Electronics and Communications, 69:1890– 1896, 2015

    Veysel Aslantas ¸ and Emre Bendes. A new image quality metric for image fusion: The sum of the correlations of differences.Aeu- international Journal of Electronics and Communications, 69:1890– 1896, 2015

    2015

  53. [61]

    A new image fusion performance metric based on visual information fidelity.Inf

    Yu Han, Yunze Cai, Yin Cao, and Xiaoming Xu. A new image fusion performance metric based on visual information fidelity.Inf. Fusion, 14:127–135, 2013

    2013

  54. [62]

    Xydeas and Vladimir S

    Costas S. Xydeas and Vladimir S. Petrovic. Objective image fusion performance measure.Electronics Letters, 36:308–309, 2000

    2000

  55. [63]

    Simoncelli, and Alan Conrad Bovik

    Zhou Wang, Eero P. Simoncelli, and Alan Conrad Bovik. Multiscale structural similarity for image quality assessment.The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, 2:1398– 1402 V ol.2, 2003

    2003

  56. [64]

    A non- reference image fusion metric based on mutual information of image features.Comput

    Mohammad Haghighat, Ali Aghagolzadeh, and Hadi Seyedarabi. A non- reference image fusion metric based on mutual information of image features.Comput. Electr. Eng., 37:744–756, 2011

    2011

  57. [65]

    Lrrnet: A novel representation learning guided fusion network for infrared and visible images.IEEE Trans

    Hui Li, Tianyang Xu, Xiaojun Wu, Jiwen Lu, and Josef Kittler. Lrrnet: A novel representation learning guided fusion network for infrared and visible images.IEEE Trans. Pattern Anal. Mach. Intell, 45:11040– 11052, 2023

    2023

  58. [66]

    Multi-interactive feature learning and a full-time multi-modality benchmark for image fusion and segmentation

    Jinyuan Liu, Zhu Liu, Guanyao Wu, Long Ma, Risheng Liu, Wei Zhong, Zhongxuan Luo, and Xin-Yue Fan. Multi-interactive feature learning and a full-time multi-modality benchmark for image fusion and segmentation. InICCV, pages 8081–8090, 2023

    2023

  59. [67]

    Correlation-guided discriminative cross-modality features network for infrared and visible image fusion.IEEE Transactions on Instrumentation and Measurement, 2023

    Zhao Cai, Yong Ma, Jun Huang, Xiaoguang Mei, and Fan Fan. Correlation-guided discriminative cross-modality features network for infrared and visible image fusion.IEEE Transactions on Instrumentation and Measurement, 2023

    2023

  60. [68]

    Probing synergistic high-order interaction in infrared and visible image fusion

    Naishan Zheng, Man Zhou, Jie Huang, Junming Hou, Haoying Li, Yuan Xu, and Feng Zhao. Probing synergistic high-order interaction in infrared and visible image fusion. InCVPR, pages 26374–26385. IEEE, 2024

    2024

  61. [69]

    Mmdrfuse: Distilled mini-model with dynamic refresh for multi- modality image fusion

    Yanglin Deng, Tianyang Xu, Chunyang Cheng, Xiao-Jun Wu, and Josef Kittler. Mmdrfuse: Distilled mini-model with dynamic refresh for multi- modality image fusion. InACM MM, 2024. Dan Hereceived the B.S. degree from Hengyang Normal University, China, in 2020 and the M.S. degree f...

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.