Pith. sign in

REVIEW 4 major objections 4 minor 111 references

FlexiD-Fuse: Flexible number of inputs multi-modal medical image fusion based on diffusion model

T0 review · 4 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FlexiD-Fuse claims that a single diffusion model fuses two or three medical image modalities under one set of weights, by turning fusion into a maximum-likelihood problem with an EM correction inside each denoising step.

desk verdict A legitimate two-modal DDFM extension with a fast Mamba backbone and extensive experiments, but the tri-modal and 'flexible number' claims rest on an ad hoc add-the-third-image construction. read the letter →

arxiv 2509.09456 v1 pith:ISFHAX2J submitted 2025-09-11 cs.CV

Yushen Xu , Xiaosong Li , Yuchun Wang , Xiaoqi Cheng , Huafeng Li , Haishu Tan This is my paper
classification cs.CV
keywords multi-modalmedicalimagefusiondiffusionmodelflexiblenumberofinputsexpectation-maximizationhierarchicalBayesianmodelingMambastatespaceimaging
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FlexiD-Fuse sets out to remove the fixed-input constraint in multi-modal medical image fusion: the authors claim that one diffusion model, trained once, can fuse either two or three medical images (CT, MRI, PET, SPECT) end-to-end under the same set of weights, and report the best scores among the compared methods on most of nine fusion metrics for both settings. The mechanism is to stop treating the input count as part of the network and instead treat fusion as likelihood correction: a lightweight Mamba-based diffusion denoiser supplies a prior over plausible medical images, and an expectation-maximization module inside each denoising step adjusts the prediction so that the final image carries information from all sources. The load-bearing device is the additive fusion identity f = x + img2 + img3 (with img3 = 0 in two-modal mode), which turns the fusion objective into a single-variable L1-plus-total-variation maximum-likelihood problem with closed-form FFT updates. If the claim is right, a clinician could select two or three modalities per patient without retraining or the loss-prone two-stage cascade currently used for tri-modal fusion, and the same weights would extend to infrared-visible, multi-exposure, and multi-focus fusion.

What carries the argument

The load-bearing device is the additive fusion identity f = x + img2 + img3 (Eq. 31), paired with the substitution y = img1 - img2 (Eq. 20). It rewrites the switching loss, min_f (||f - img1 - img3||_1 + phi||f - img2 - img3||_1), into a one-variable maximum-likelihood problem, min_x (||y - x||_1 + phi||x||_1); the number of inputs enters only through the term added back at the end. Inference is carried by an EM module interleaved with the diffusion sampler: the E-step computes per-pixel confidence weights from the current prediction, and the M-step solves half-quadratic-splitting subproblems via closed-form FFT updates, then adds img2 + img3 back. The Diffusion Fusion Mamba (DFM), an ultra-

What would settle it

Fuse one fixed triple of source images six times, once for each permutation of the roles img1, img2, img3. If the fused outputs differ materially, the result depends on input ordering, not on the set of modalities, and the additive fusion model (Eq. 31) fails as a general prior. A lighter check: the paper's equations imply that tri-modal fusion with an all-zero img3 must exactly reproduce two-modal fusion; any deviation would reveal hidden input-count dependence.

Watch

Extended reading notes

Core claim

A fusion network need not know its input count in advance: FlexiD-Fuse fuses two- or three-modal medical images end-to-end with one set of weights, by re-casting diffusion fusion as maximum-likelihood estimation and running an EM correction inside every reverse-sampling step. The correction uses the additive identity f = x + img2 + img3 (img3 = 0 for two inputs), where x solves an L1-plus-total-variation problem with closed-form FFT updates. The paper reports best scores on most of nine metrics for both settings on the Harvard dataset, and the same weights transfer to infrared-visible, multi-exposure, and multi-focus fusion. The abstract claims independence from input number; the paper's own

Load-bearing premise

The tri-modal capability rests on the additive equation f = x + img2 + img3: for three inputs the loss simultaneously wants f approximately img1 + img3 and f approximately img2 + img3, so the method succeeds only to the extent both additive targets are valid descriptions of the same fusion, and the output can vary with which input is labeled img3.

Editorial extensions

If this is right

  • One pretrained denoiser serves both two-modal and tri-modal fusion under identical weights, eliminating per-count retraining and the information-losing two-stage cascade used to extend two-modal models to three inputs.
  • The flexibility is nearly free: the EM correction uses closed-form FFT updates, so handling a variable number of inputs adds no learned parameters to the network.
  • The same weights transfer across tasks: the paper demonstrates infrared-visible, multi-exposure, and multi-focus (including three-source) fusion without retraining, supporting the claim that the mechanism is modality-agnostic.
  • Because the correction re-derives the fusion target from the sources at every diffusion step, the fused image is generated without any ground-truth fusion image, removing a standard obstacle for diffusion-based fusion.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The additive identity is order-sensitive: permuting which input plays img1, img2, or img3 changes y = img1 - img2 and hence changes the fused output. A true any-number fusion should be permutation-invariant, so fusing the same triple in all six orderings is a direct, untested implication.
  • The math does not obviously stop at three inputs: the same E- and M-steps would work for four or more modalities with f = x + (sum of all but one input). The paper's stated limitation to two- and three-modal switching looks like an implementation choice rather than a formal ceiling, and would be easy to test.
  • The L1 intensity losses tacitly assume the source modalities share a comparable intensity scale; the paper normalizes images to [-1, 1] before fusion, so that normalization is likely load-bearing for the additive correction even though the paper does not discuss it.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. FlexiD-Fuse proposes a diffusion-based framework for two-modal and tri-modal medical image fusion with a single pretrained denoising model. At inference, the method interleaves standard DDPM sampling with an EM module that corrects the denoised estimate using the source images. The central claim is that this corrector is independent of the number of input modalities, so the same weights can fuse two or three images. The paper reports SOTA metrics on Harvard two- and tri-modal medical fusion and on extended infrared-visible, multi-exposure, and multi-focus experiments, and it releases code.

Significance. If the flexible-input claim were correct, the paper would be a meaningful advance: most existing fusion methods are fixed-input, and a single diffusion prior plus a likelihood corrector that accepts variable numbers of modalities would be practically valuable. The manuscript also has real strengths: a broad experimental campaign across six datasets, comparisons with many recent methods, a lightweight Mamba-based diffusion backbone (DFM), and a public code release. However, the core derivation of the tri-modal case is, by the paper's own substitution, two-modal EM followed by an additive third image. The headline claim therefore rests on a step that is neither justified nor permutation-invariant, and the reported tri-modal gains are consistent with a trivial additive baseline. The significance of the flexible-input contribution is consequently not established.

major comments (4)
  1. [§3.4, Eq. (18)–(20), (31); Algorithm 1 lines 11–18] The central construction is, by the paper's own substitution, two-modal EM plus an additive third image. Setting x=f−img2−img3 and y=img1−img2 turns Eq. (18) into Eq. (20), whose solution depends only on img1 and img2; Eq. (31) then returns x+img2+img3. In Algorithm 1 the E-step (line 12) also uses only y=img1−img2. Thus the tri-modal case does not perform a three-way likelihood correction; it is DDFM-style two-modal EM followed by an unchanged addition of img3. The abstract's claim that the method fuses inputs 'independently of the number of input images' is therefore not supported by the derivation.
  2. [§3.4, Eq. (18)] Eq. (18) is introduced without derivation and is not a natural symmetric extension of the two-modal loss. For three modalities a symmetric objective would depend on all pairwise differences or treat all images equally; Eq. (18) only couples img1 and img2 through their difference and forces f to approximate img1+img3 and img2+img3. Because these two targets conflict whenever img1≠img2, the minimizer is an arbitrary weighted compromise, and the result is permutation-dependent: swapping img1 and img2 changes x and hence changes f. No permutation-invariance test or theoretical justification is provided. This is load-bearing because the 'flexible number of inputs' claim requires the output to be independent of input ordering, at least up to the fusion semantics.
  3. [Conclusion (last paragraph) and abstract] The abstract states the method handles 'flexible quantities' of inputs and works 'independently of the number of input images,' but the Conclusion explicitly limits the method to two- and three-modal fusion: 'it is not yet capable of handling fusion tasks involving more than three modalities.' The extended experiments also only use two or three sources. The paper should at minimum restrict all claims to n=2,3. This is not a minor wording issue because the 'arbitrary number' claim is part of the advertised contribution.
  4. [Table 7 and §4.6] The ablation does not test the key assumption. Since Eq. (31) defines the tri-modal output as x+img2+img3, the proposed tri-modal method is exactly the trivial baseline 'run the two-modal algorithm on (img1,img2) and add img3'. Table 7 compares against a two-stage fusion (Config 2) but not against this baseline, nor against f=EM(img1,img3)+img2 or f=EM(img2,img3)+img1. Without these controls, the reported tri-modal gains cannot be attributed to a principled three-way fusion; they are consistent with the additive construction. The authors should add these baselines and a permutation analysis.
minor comments (4)
  1. [§3.2, Eq. (6)] The expectation notation in Eq. (6) is garbled: 'EtEx0 Ext|x0' is not readable as a standard expectation over (t, x0, xt), and the integrand mixes ∇_x_t with ∇_f. Please rewrite in standard notation.
  2. [§3.4.2, Eq. (30)] The x-update is written as x = (2m^2⊙y+ηk)⊘(2m^2+2n^2+η)^{-1}; as written this multiplies by the inverse, which is wrong. It should be division by (2m^2+2n^2+η). The code presumably uses the correct form, but the printed formula is a mathematical error in a core update.
  3. [§4.1.1 and §4.3] The text says 'nine metrics' but only eight are listed (SSIM, MI, PSNR, VIF, QS, QCV, FSIM, QNCIE). Similarly, §4.3 says '9 SOTA methods' but Table 2 lists eleven two-modal methods. Please correct these inconsistencies.
  4. [§4.4, Table 3] For the proposed method, Time-3M equals Time-2M (9.850 s in both cases). This is consistent with Eq. (31) but should be explained; otherwise it reinforces the impression that the third modality is not processed through the EM likelihood at all.

Circularity Check

1 steps flagged · score 6.0 of 10

Tri-modal fusion is by construction two-modal EM plus img3: Eq. 18's loss is rearranged via x=f−img2−img3 into the two-modal problem, and Eq. 31 adds img2+img3 back, so the third modality is only added, not fused.

  1. self definitional [Section 3.4, Eqs. 18-20 and 31; Algorithm 1 lines 11-18]
    "min f {∥f−img1−img3∥1 + φ∥f−img2−img3∥1} ... by performing a simple variable substitution x=f−img2−img3 and y=img1−img2, we derive the optimization formula: min x (∥y−x∥1 + φ∥x∥1) ... The final estimation of f is: f̂=x+img2+img3 (31). If 2-mode fusion, img3=0 (32)."

    The tri-modal loss (Eq. 18) is itself the assumption that the fused image equals img1+img3 and img2+img3. The substitution x=f−img2−img3, y=img1−img2 turns this into the two-modal problem min_x ||y−x||+φ||x||, which does not involve img3 at all. The final reconstruction f̂=x+img2+img3 then guarantees that the three-modal output is exactly the two-modal EM solution on (img1,img2) plus the raw third image img3. So the paper's central claim of processing two- and three-modal inputs 'independently of the number of input images' is true only by construction: n=3 is defined as n=2 plus an additive image. The derivation does not fuse img3 through the likelihood; it appends it. The output also depends on which modality is labelled img3, since no permutation symmetry is imposed. This is a self-defi

full rationale

The two-modal subcase (img3=0) is an independent DDFM-style EM formulation and is evaluated against external baselines; that part is not circular. However, the paper's headline contribution—flexible two/tri-modal fusion under the same weight—is obtained by defining the tri-modal output as x+img2+img3. The loss in Eq. 18 is chosen so that the variable substitution x=f−img2−img3 reduces to the same two-modal likelihood term, and Eq. 31 simply adds img2+img3 back. Therefore the tri-modal 'prediction' is an algebraic identity of the input loss, not a derived property. The paper acknowledges it only handles up to three modalities, which is consistent with this additive construction. Because the central flexible-number claim reduces by construction while the two-modal core retains independent content, the appropriate circularity score is 6.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced. The main extra assumptions are the hand-chosen fusion loss, the additive parametrization of the tri-modal output, and the latent-variable distributional model. Together these define the method's behavior but are not independently evidenced.

free parameters (4)
  • eta (penalty weight in Eq. 28) = 0.1
    Set by hand in Section 4.1.2; controls the strength of the half-quadratic splitting penalty terms in the M-step.
  • phi (weight on the second L1 term, written as psi in Eq. 21) = 0.5
    Set by hand in Section 4.1.2; balances the two modality fidelity terms in Eq. 18/20.
  • initial gamma and rho (latent variance hyperparameters) = unspecified
    Appear in the E-step Eqs. 23-24 and are updated by Eq. 33; their initial values are not reported, so the inference result depends on an unreported choice.
  • T (number of diffusion steps) = 100
    Set in Section 4.1.2; a hand-chosen sampling budget that affects output quality but is not fitted.
assumptions (4)
  • domain assumption The conditional score decomposes as an unconditional score plus a likelihood term via Bayes, as in Eq. 10 and validated in DPS.
    Invoked in Section 3.2 to turn the multi-condition diffusion problem into a prior-plus-likelihood form; the approximation is taken from the cited DPS paper and is not re-validated for multi-modality fusion.
  • ad hoc to paper The fusion objective is the ad hoc L1 loss in Eq. 18, min_f ||f-img1-img3||_1 + phi||f-img2-img3||_1.
    Introduced in Section 3.4 without derivation from imaging physics or a generative model; it directly defines the maximum-likelihood problem and thus controls all downstream EM steps.
  • ad hoc to paper The fused image is parametrized as f = x + img2 + img3 (Eq. 31), i.e., the third modality is added unchanged to the EM-corrected two-modal result.
    Algorithm 1 computes x from img1-img2 only, then adds img2+img3; this makes the output depend on which input is designated img3 and is not a joint tri-modal fusion rule.
  • domain assumption x follows a Laplace distribution and the log-likelihood has the Gaussian latent-variable form in Eq. 21, with m and n as latent variances.
    Needed to apply EM in Section 3.4; the distributional assumption is stated but not checked against data.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FlexiD-Fuse: Flexible number of inputs multi-modal medical image fusion based on diffusion model." pith.science (2026). https://pith.science/paper/ISFHAX2J

@misc{pith2026250909456,
  author       = {Pith},
  title        = {Pith review of: FlexiD-Fuse: Flexible number of inputs multi-modal medical image fusion based on diffusion model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ISFHAX2J}},
  note         = {Machine review of arXiv:2509.09456}
}
read the original abstract

Different modalities of medical images provide unique physiological and anatomical information for diseases. Multi-modal medical image fusion integrates useful information from different complementary medical images with different modalities, producing a fused image that comprehensively and objectively reflects lesion characteristics to assist doctors in clinical diagnosis. However, existing fusion methods can only handle a fixed number of modality inputs, such as accepting only two-modal or tri-modal inputs, and cannot directly process varying input quantities, which hinders their application in clinical settings. To tackle this issue, we introduce FlexiD-Fuse, a diffusion-based image fusion network designed to accommodate flexible quantities of input modalities. It can end-to-end process two-modal and tri-modal medical image fusion under the same weight. FlexiD-Fuse transforms the diffusion fusion problem, which supports only fixed-condition inputs, into a maximum likelihood estimation problem based on the diffusion process and hierarchical Bayesian modeling. By incorporating the Expectation-Maximization algorithm into the diffusion sampling iteration process, FlexiD-Fuse can generate high-quality fused images with cross-modal information from source images, independently of the number of input images. We compared the latest two and tri-modal medical image fusion methods, tested them on Harvard datasets, and evaluated them using nine popular metrics. The experimental results show that our method achieves the best performance in medical image fusion with varying inputs. Meanwhile, we conducted extensive extension experiments on infrared-visible, multi-exposure, and multi-focus image fusion tasks with arbitrary numbers, and compared them with the perspective SOTA methods. The results of the extension experiments consistently demonstrate the effectiveness and superiority of our method.

Figures

Figures reproduced from arXiv: 2509.09456 by the authors.

Figure 1
Figure 1. A comparison of results with the most advanced methods in [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Fusion strategies for different numbers of modalities: (a) The existing two-stage fusion process; (b) The proposed FlexiD-Fuse fusion process: first, fused features ft are generated through the DFM model, and then the likelihood correction algorithm is applied to adjust the DFM output using information from the source images. 3.2. Score-based Diffusion Process This paper employs a score-based diffusion process for i… view at source ↗
Figure 3
Figure 3. The DFM architecture is illustrated. First, the input image is divided into multiple patches and projected into patch tokens. Subsequently, [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Flowchart of FlexiD-Fuse computed in one iteration. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 3
Figure 3. Figure 3: Unlike Vim, DF-Mamba adopts unidirectional se￾quence modeling to make the model more lightweight and improve processing speed. The timestep t and the token sequence xl−1 are simultaneously input into the DFM block. The timestep t undergoes two layers of MLP for dimensi…
Figure 5
Figure 5. Figure 5: Results of Tri-modal Medical Image Fusion.(a): PET/ [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Results of Two-modal Medical Image Fusion.(a): MRI-PET Fusion, (b): MRI-SPECT Fusion, (c):CT-MRI Fusion [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Fusion results of different methods on infrared and visible image datasets, (a): MSRS, (b): LLVIP, (c): M3FD. proposed model effectively integrates visible light tex￾ture information and infrared thermal information, high￾lighting background details that are invisible …
Figure 8
Figure 8. Figure 8: Fusion results of different multi-exposure methods on the SICE dataset (Top: Spring, Bottom: Winter). fusion algorithms, including BHFMEF (ACM MM’23) [99], HoloCo (Information Fusion’23) [100], HSDS￾MEF (AAAI’24) [101], MERF (ACM Trans.Graph’23) [102], and SAMF (ICASSP…
Figure 9
Figure 9. Figure 9: (a) Fusion results of various methods on the MFI-WHU Dataset, with Source1 as the foreground-focused image and Source2 as the [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

111 extracted references · 8 linked inside Pith

  1. [1]

    C. Qu, T. Zhang, H. Qiao, Y . Tang, A. L. Yuille, Z. Zhou, et al., Abdomenatlas-8k: Annotating 8,000 ct volumes for multi-organ segmentation in three weeks, Advances in Neural Information Processing Systems 36 (2024)

    2024

  2. [2]

    Y . Jie, X. Li, F. Zhou, H. Tan, et al., Medical image fusion based on extended difference-of- gaussians and edge-preserving, Expert Systems with Applications 227 (2023) 120301

    2023

  3. [3]

    Salim, Y

    M. Salim, Y . Liu, M. Sorkhei, D. Ntoula, T. Foukakis, I. Fredriksson, Y . Wang, M. Ek- lund, H. Azizpour, K. Smith, et al., Ai- based selection of individuals for supplemental mri in population-based breast cancer screen- ing: the randomized screentrustmri trial, Nature Medicine 30 (9) (2024) 2623–2630

    2024

  4. [4]

    Huang, W

    Z. Huang, W. Li, Y . Wu, L. Yang, Y . Dong, Y . Yang, H. Zheng, D. Liang, M. Wang, Z. Hu, Accurate whole-brain image enhancement for low-dose integrated pet/mr imaging through spatial brain transformation, IEEE Journal of Biomedical and Health Informatics (2024)

    2024

  5. [5]

    Jin, L.-J

    Y . Jin, L.-J. Meng, Exploration of coincidence detection of cascade photons to enhance pre- clinical multi-radionuclide spect imaging, IEEE transactions on medical imaging (2024)

    2024

  6. [6]

    Diwakar, P

    M. Diwakar, P. Singh, A. Shankar, Multi- modal medical image fusion framework using co-occurrence filter and local extrema in nsst do- main, Biomedical Signal Processing and Control 68 (2021) 102788

    2021

  7. [7]

    Y . Li, M. E. H. Daho, P.-H. Conze, R. Zegh- lache, H. Le Boité, R. Tadayoni, B. Coch- ener, M. Lamard, G. Quellec, A review of deep learning-based information fusion techniques for multimodal medical image classification, Com- puters in Biology and Medicine (2024) 108635

    2024

  8. [9]

    R. Qin, Z. Wang, L. Jiang, K. Qiao, J. Hai, J. Chen, J. Xu, D. Shi, B. Yan, Fine-grained lung cancer classification from pet and ct images based on multidimensional attention mechanism, Complexity 2020 (1) (2020) 6153657. 19

    2020

Show all 111 references
  1. [10]

    P. Wu, Z. Wang, B. Zheng, H. Li, F. E. Alsaadi, N. Zeng, Aggn: Attention-based glioma grad- ing network with multi-scale feature extraction and multi-modal information fusion, Computers in biology and medicine 152 (2023) 106457

    2023

  2. [11]

    Dhaundiyal, A

    R. Dhaundiyal, A. Tripathi, K. Joshi, M. Di- wakar, P. Singh, Clustering based multi-modality medical image fusion, in: Journal of Physics: Conference Series, V ol. 1478, IOP Publishing, 2020, p. 012024

    2020

  3. [12]

    Diwakar, P

    M. Diwakar, P. Singh, A. Shankar, S. R. Nayak, J. Nayak, S. Vimal, R. Singh, D. Sisodia, Di- rective clustering contrast-based multi-modality medical image fusion for smart healthcare sys- tem, Network Modeling Analysis in Health In- formatics and Bioinformatics 11 (1) (2022) 15

    2022

  4. [13]

    Y . Jie, F. Zhou, H. Tan, G. Wang, X. Cheng, X. Li, Tri-modal medical image fusion based on adaptive energy choosing scheme and sparse rep- resentation, Measurement 204 (2022) 112038

    2022

  5. [14]

    Y . Liu, X. Chen, A. Liu, R. K. Ward, Z. J. Wang, Recent advances in sparse representation based medical image fusion, IEEE Instrumentation & Measurement Magazine 24 (2) (2021) 45–53

    2021

  6. [15]

    Zhang, A

    X. Zhang, A. Liu, G. Yang, Y . Liu, X. Chen, Simfusion: A semantic information-guided modality-specific fusion network for mr images, Information Fusion 112 (2024) 102560

    2024

  7. [16]

    Y . Jie, X. Li, T. Tan, L. Yang, M. Wang, Multi- modality image fusion using fuzzy set theory and compensation dictionary learning, Optics & Laser Technology 181 (2025) 112001

    2025

  8. [17]

    X. Li, W. Wan, F. Zhou, X. Cheng, Y . Jie, H. Tan, Medical image fusion based on sparse represen- tation and neighbor energy activity, Biomedical Signal Processing and Control 80 (2023) 104353

    2023

  9. [18]

    X. Li, F. Zhou, H. Tan, W. Zhang, C. Zhao, Mul- timodal medical image fusion based on joint bi- lateral filter and local gradient energy, Informa- tion Sciences 569 (2021) 302–325

    2021

  10. [19]

    Kalage, P

    D. Kalage, P. Gupta, A. Gulati, K. P. Reddy, K. Sharma, A. Thakur, T. D. Yadav, V . Gupta, L. Kaman, R. Nada, et al., Contrast enhanced ct versus mri for accurate diagnosis of wall- thickening type gallbladder cancer, Journal of Clinical and Experimental Hepatology 14 (5) (202...

    2024

  11. [20]

    X. Li, S. Ma, J. Xu, J. Tang, S. He, F. Guo, Tran- siam: Aggregating multi-modal visual features with locality for medical image segmentation, Expert Systems with Applications 237 (2024) 121574

    2024

  12. [21]

    Z. Guo, X. Li, H. Huang, N. Guo, Q. Li, Deep learning-based image segmentation on multi- modal medical imaging, IEEE Transactions on Radiation and Plasma Medical Sciences 3 (2) (2019) 162–169

    2019

  13. [22]

    Karthik, H

    A. Karthik, H. S. Hamatta, S. Patthi, C. Krubakaran, A. K. Pradhan, V . Racha- pudi, M. Shuaib, A. Rajaram, Ensemble-based multimodal medical imaging fusion for tumor segmentation, Biomedical Signal Processing and Control 96 (2024) 106550

    2024

  14. [23]

    Li, X.-J

    H. Li, X.-J. Wu, Densefuse: A fusion approach to infrared and visible images, IEEE Transactions on Image Processing 28 (5) (2018) 2614–2623

    2018

  15. [24]

    Y . Jie, Y . Xu, X. Li, H. Tan, Tsjnet: A multi- modality target and semantic awareness joint- driven image fusion network, arXiv preprint arXiv:2402.01212 (2024)

    2024

  16. [25]

    J. Liu, G. Wu, Z. Liu, D. Wang, Z. Jiang, L. Ma, W. Zhong, X. Fan, Infrared and visible image fu- sion: From data compatibility to task adaption, IEEE Transactions on Pattern Analysis and Ma- chine Intelligence (2024)

    2024

  17. [26]

    H. Li, Y . Cen, Y . Liu, X. Chen, Z. Yu, Differ- ent input resolutions and arbitrary output resolu- tion: A meta learning-based deep framework for infrared and visible image fusion, IEEE Transac- tions on Image Processing 30 (2021) 4070–4083

    2021

  18. [27]

    H. Li, Z. Yang, Y . Zhang, W. Jia, Z. Yu, Y . Liu, Mulfs-cap: Multimodal fusion-supervised cross- modality alignment perception for unregistered infrared-visible image fusion, IEEE Transactions on Pattern Analysis and Machine Intelligence (2025)

    2025

  19. [28]

    Z. Zhao, H. Bai, J. Zhang, Y . Zhang, K. Zhang, S. Xu, D. Chen, R. Timofte, L. Van Gool, Equiv- ariant multi-modality image fusion, in: Pro- ceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, 2024, pp. 25912–25921. 20

    2024

  20. [29]

    H. Xu, J. Yuan, J. Ma, Murf: Mutually reinforc- ing multi-modal image registration and fusion, IEEE transactions on pattern analysis and ma- chine intelligence 45 (10) (2023) 12148–12166

    2023

  21. [30]

    H. Li, J. Zhao, J. Li, Z. Yu, G. Lu, Feature dynamic alignment and refinement for infrared– visible image fusion: Translation robust fusion, Information Fusion 95 (2023) 26–41

    2023

  22. [32]

    J. Ma, L. Tang, F. Fan, J. Huang, X. Mei, Y . Ma, Swinfusion: Cross-domain long-range learning for general image fusion via swin transformer, IEEE/CAA Journal of Automatica Sinica 9 (7) (2022) 1200–1217

    2022

  23. [33]

    W. Tang, F. He, Y . Liu, Ydtr: Infrared and vis- ible image fusion via y-shape dynamic trans- former, IEEE Transactions on Multimedia 25 (2022) 5413–5428

    2022

  24. [34]

    J. Li, Y . Wang, X. Ning, W. He, W. Cai, Fefdm-transformer: Dual-channel multi-stage transformer-based encoding and fusion mode for infrared-visible images, Expert Systems with Applications (2025) 127229

    2025

  25. [35]

    H. Li, J. Liu, Y . Zhang, Y . Liu, A deep learning framework for infrared and visible image fusion without strict registration, International Journal of Computer Vision 132 (5) (2024) 1625–1644

    2024

  26. [36]

    A. Dong, L. Wang, J. Liu, G. Lv, G. Zhao, J. Cheng, Mfifusion: An infrared and visible im- age enhanced fusion network based on multi- level feature injection, Pattern Recognition 152 (2024) 110445

    2024

  27. [37]

    J. Ma, W. Yu, P. Liang, C. Li, J. Jiang, Fusion- gan: A generative adversarial network for in- frared and visible image fusion, Information fu- sion 48 (2019) 11–26

    2019

  28. [38]

    J. Ma, H. Xu, J. Jiang, X. Mei, X.-P. Zhang, Dd- cgan: A dual-discriminator conditional genera- tive adversarial network for multi-resolution im- age fusion, IEEE Transactions on Image Process- ing 29 (2020) 4980–4995

    2020

  29. [39]

    M. Das, D. Gupta, A. Bakde, An end-to- end content-aware generative adversarial net- work based method for multimodal medical im- age fusion, in: Data Analytics for Intelligent Sys- tems: Techniques and Solutions, IOP Publishing Bristol, UK, 2024, pp. 7–1

    2024

  30. [40]

    W. Zhao, S. Xie, F. Zhao, Y . He, H. Lu, Meta- fusion: Infrared and visible image fusion via meta-feature embedding from object detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 13955–13965

    2023

  31. [41]

    J. Yue, L. Fang, S. Xia, Y . Deng, J. Ma, Dif- fusion: Towards high color fidelity in infrared and visible image fusion with diffusion models, IEEE Transactions on Image Processing (2023)

    2023

  32. [42]

    Y . Xu, X. Li, Y . Jie, H. Tan, Simultaneous tri- modal medical image fusion and super-resolution using conditional diffusion model, in: Interna- tional Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2024, pp. 635–645

    2024

  33. [43]

    X. Yi, L. Tang, H. Zhang, H. Xu, J. Ma, Diff-if: Multi-modality image fusion via diffusion model with fusion knowledge prior, Information Fusion 110 (2024) 102450

    2024

  34. [44]

    Y . Jie, X. Li, F. Zhou, T. Ye, Tri-modal medi- cal image fusion and denoising based on bitonicx filtering, IEEE Transactions on Instrumentation and Measurement 72 (2023) 1–15

    2023

  35. [45]

    Huang, X

    J. Huang, X. Li, H. Tan, X. Cheng, Genera- tive adversarial network for trimodal medical im- age fusion using primitive relationship reasoning, IEEE Journal of Biomedical and Health Infor- matics (2024)

    2024

  36. [46]

    J. Ho, A. Jain, P. Abbeel, Denoising diffu- sion probabilistic models, Advances in neural in- formation processing systems 33 (2020) 6840– 6851

    2020

  37. [47]

    Zhang, A

    L. Zhang, A. Rao, M. Agrawala, Adding con- ditional control to text-to-image diffusion mod- els, in: Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, 2023, pp. 3836–3847

    2023

  38. [48]

    Zhang, L

    H. Zhang, L. Cao, J. Ma, Text-difuse: An in- teractive multi-modal image fusion framework 21 based on text-modulated diffusion model, arXiv preprint arXiv:2410.23905 (2024)

    2024 arXiv

  39. [49]

    B. Yang, Z. Jiang, D. Pan, H. Yu, G. Gui, W. Gui, Lfdt-fusion: a latent feature-guided dif- fusion transformer model for general image fu- sion, Information Fusion 113 (2025) 102639

    2025

  40. [50]

    Z. Zhao, H. Bai, Y . Zhu, J. Zhang, S. Xu, Y . Zhang, K. Zhang, D. Meng, R. Timofte, L. Van Gool, Ddfm: denoising diffusion model for multi-modality image fusion, in: Proceed- ings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 8082–8093

    2023

  41. [51]

    Y . Shi, Y . Liu, J. Cheng, Z. J. Wang, X. Chen, Vdmufusion: A versatile diffusion model-based unsupervised framework for image fusion, IEEE Transactions on Image Processing (2024)

    2024

  42. [52]

    Peebles, S

    W. Peebles, S. Xie, Scalable diffusion mod- els with transformers, in: Proceedings of the IEEE/CVF International Conference on Com- puter Vision, 2023, pp. 4195–4205

    2023

  43. [53]

    V . T. Hu, S. A. Baumann, M. Gui, O. Grebenkova, P. Ma, J. Schusterbauer, B. Ommer, Zigma: A dit-style zigzag mamba diffusion model, in: ECCV , 2024

    2024

  44. [54]

    X. Li, X. Li, T. Ye, X. Cheng, W. Liu, H. Tan, Bridging the gap between multi-focus and multi- modal: a focused integration framework for multi-modal image fusion, in: Proceedings of the IEEE/CVF winter conference on applications of computer vision, 2024, pp. 1628–1637

    2024

  45. [55]

    J. Li, X. Li, X. Li, D. Han, H. Tan, Z. Hou, P. Yi, Multi-focus image fusion based on mul- tiscale fuzzy quality assessment, Digital Signal Processing 153 (2024) 104592

    2024

  46. [56]

    J. Liu, X. Fan, J. Jiang, R. Liu, Z. Luo, Learn- ing a deep multi-scale feature ensemble and an edge-attention guidance for image fusion, IEEE Transactions on Circuits and Systems for Video Technology 32 (1) (2021) 105–119

    2021

  47. [57]

    X. Li, H. Tan, F. Zhou, G. Wang, X. Li, Infrared and visible image fusion based on domain trans- form filtering and sparse representation, Infrared Physics & Technology 131 (2023) 104701

    2023

  48. [58]

    X. Li, F. Zhou, H. Tan, Joint image fusion and de- noising via three-layer decomposition and sparse representation, Knowledge-Based Systems 224 (2021) 107087

    2021

  49. [59]

    Z. Zhu, H. Yin, Y . Chai, Y . Li, G. Qi, A novel multi-modality image fusion method based on image decomposition and sparse representation, Information Sciences 432 (2018) 516–529

    2018

  50. [60]

    X. Luo, G. Fu, J. Yang, Y . Cao, Y . Cao, Multi- modal image fusion via deep laplacian pyra- mid hybrid network, IEEE Transactions on Cir- cuits and Systems for Video Technology 33 (12) (2023) 7354–7369

    2023

  51. [61]

    J. Chen, X. Li, L. Luo, X. Mei, J. Ma, In- frared and visible image fusion based on target- enhanced multiscale transform decomposition, Information Sciences 508 (2020) 64–78

    2020

  52. [62]

    Pajares, J

    G. Pajares, J. M. De La Cruz, A wavelet-based image fusion tutorial, Pattern recognition 37 (9) (2004) 1855–1872

    2004

  53. [63]

    Z. Zhao, H. Bai, J. Zhang, Y . Zhang, S. Xu, Z. Lin, R. Timofte, L. Van Gool, Cddfuse: Correlation-driven dual-branch feature decompo- sition for multi-modality image fusion, in: Pro- ceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, 2023, pp. 5906–5916

    2023

  54. [64]

    H. Xu, J. Ma, J. Jiang, X. Guo, H. Ling, U2fusion: A unified unsupervised image fusion network, IEEE Transactions on Pattern Analysis and Machine Intelligence 44 (1) (2020) 502–518

    2020

  55. [65]

    M. Chen, Z. Chen, Y . Xi, X. Qiao, X. Chen, Q. Huang, Multimodal fusion network for detect- ing hyperplastic parathyroid glands in spect/ct images, IEEE Journal of Biomedical and Health Informatics 27 (3) (2022) 1524–1534

    2022

  56. [66]

    H. Li, D. Wang, Y . Huang, Y . Zhang, Z. Yu, Gen- eration and recombination for multifocus image fusion with free number of inputs, IEEE Trans- actions on Circuits and Systems for Video Tech- nology (2023)

    2023

  57. [67]

    Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Ku- mar, S. Ermon, B. Poole, Score-based generative modeling through stochastic differential equa- tions, arXiv preprint arXiv:2011.13456 (2020)

    2011 arXiv

  58. [68]

    B. D. Anderson, Reverse-time diffusion equation models, Stochastic Processes and their Applica- tions 12 (3) (1982) 313–326. 22

    1982

  59. [69]

    Chung, J

    H. Chung, J. Kim, M. T. Mccann, M. L. Klasky, J. C. Ye, Diffusion posterior sampling for general noisy inverse problems, arXiv preprint arXiv:2209.14687 (2022)

    2022 arXiv

  60. [70]

    OpenAI, Sora: Creating video from text (2024)

    2024

  61. [71]

    A. Gu, T. Dao, Mamba: Linear-time sequence modeling with selective state spaces, arXiv preprint arXiv:2312.00752 (2023)

    2023 arXiv

  62. [72]

    L. Zhu, B. Liao, Q. Zhang, X. Wang, W. Liu, X. Wang, Vision mamba: Efficient visual repre- sentation learning with bidirectional state space model, arXiv preprint arXiv:2401.09417 (2024)

    2024 arXiv

  63. [73]

    Dosovitskiy, An image is worth 16x16 words: Transformers for image recognition at scale, arXiv preprint arXiv:2010.11929 (2020)

    A. Dosovitskiy, An image is worth 16x16 words: Transformers for image recognition at scale, arXiv preprint arXiv:2010.11929 (2020)

    2010 arXiv

  64. [74]

    Gu, Modeling Sequences with Structured State Spaces, Stanford University, 2023

    A. Gu, Modeling Sequences with Structured State Spaces, Stanford University, 2023

    2023

  65. [75]

    A. Gu, I. Johnson, A. Timalsina, A. Rudra, C. Ré, How to train your hippo: State space models with generalized orthogonal basis projections, arXiv preprint arXiv:2206.12037 (2022)

    2022 arXiv

  66. [76]

    A. Gu, K. Goel, A. Gupta, C. Ré, On the pa- rameterization and initialization of diagonal state space models, Advances in Neural Information Processing Systems 35 (2022) 35971–35983

    2022

  67. [77]

    J. Wang, W. Zhu, P. Wang, X. Yu, L. Liu, M. Omar, R. Hamid, Selective structured state- spaces for long-form video understanding, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 6387–6397

    2023

  68. [78]

    Z. Zhao, S. Xu, C. Zhang, J. Liu, J. Zhang, Bayesian fusion for infrared and visible images, Signal Processing 177 (2020) 107734

    2020

  69. [79]

    J. Ma, C. Chen, C. Li, J. Huang, Infrared and vis- ible image fusion via gradient transfer and total variation minimization, Information Fusion 31 (2016) 100–109

    2016

  70. [80]

    L. Tang, J. Yuan, J. Ma, Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion net- work, Information Fusion 82 (2022) 28–42

    2022

  71. [81]

    Summers, Harvard whole brain atlas: www

    D. Summers, Harvard whole brain atlas: www. med. harvard. edu/aanlib/home. html, Journal of Neurology, Neurosurgery & Psychiatry 74 (3) (2003) 288–288

    2003

  72. [82]

    Piella, H

    G. Piella, H. Heijmans, A new quality metric for image fusion, in: Proceedings 2003 interna- tional conference on image processing (Cat. No. 03CH37429), V ol. 3, IEEE, 2003, pp. III–173

    2003

  73. [83]

    H. Chen, P. K. Varshney, A human perception inspired quality metric for image fusion based on regional information, Information fusion 8 (2) (2007) 193–207

    2007

  74. [84]

    Zhang, L

    L. Zhang, L. Zhang, X. Mou, D. Zhang, Fsim: A feature similarity index for image quality as- sessment, IEEE transactions on Image Process- ing 20 (8) (2011) 2378–2386

    2011

  75. [85]

    Q. Wang, Y . Shen, J. Jin, Performance evaluation of image fusion techniques, Image fusion: algo- rithms and applications 19 (2008) 469–492

    2008

  76. [86]

    Zheng, M

    N. Zheng, M. Zhou, J. Huang, J. Hou, H. Li, Y . Xu, F. Zhao, Probing synergistic high-order interaction in infrared and visible image fusion, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 26384–26395

    2024

  77. [87]

    P. Mu, G. Wu, J. Liu, Y . Zhang, X. Fan, R. Liu, Learning to search a lightweight generalized net- work for medical image fusion, IEEE Transac- tions on Circuits and Systems for Video Technol- ogy (2023)

    2023

  78. [88]

    J. Li, J. Liu, S. Zhou, Q. Zhang, N. K. Kasabov, Gesenet: A general semantic-guided network with couple mask ensemble for medical image fusion, IEEE Transactions on Neural Networks and Learning Systems (2023)

    2023

  79. [89]

    J. Liu, R. Lin, G. Wu, R. Liu, Z. Luo, X. Fan, Coconet: Coupled contrastive learning network with multi-level feature ensemble for multi- modality image fusion, International Journal of Computer Vision 132 (5) (2024) 1748–1775

    2024

  80. [90]

    Huang, X

    J. Huang, X. Li, H. Tan, L. Yang, G. Wang, P. Yi, Dednet: Infrared and visible image fusion with noise removal by decomposition-driven network, Measurement (2024) 115092. 23

    2024

  81. [91]

    Z. Ruan, J. Wan, G. Xiao, Z. Tang, J. Ma, Seman- tic attention-based heterogeneous feature aggre- gation network for image fusion, Pattern Recog- nition 155 (2024) 110728

    2024

  82. [92]

    Huang, X

    J. Huang, X. Li, T. Tan, X. Li, T. Ye, Mma-unet: A multi-modal asymmetric unet architecture for infrared and visible image fusion, arXiv preprint arXiv:2404.17747 (2024)

    2024 arXiv

  83. [93]

    X. Yi, H. Xu, H. Zhang, L. Tang, J. Ma, Text-if: Leveraging semantic text guidance for degradation-aware and interactive image fusion, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 27026–27035

    2024

  84. [94]

    L. Tang, J. Yuan, H. Zhang, X. Jiang, J. Ma, Pia- fusion: A progressive infrared and visible image fusion network based on illumination aware, In- formation Fusion 83 (2022) 79–92

    2022

  85. [95]

    X. Jia, C. Zhu, M. Li, W. Tang, W. Zhou, Llvip: A visible-infrared paired dataset for low-light vi- sion, in: Proceedings of the IEEE/CVF interna- tional conference on computer vision, 2021, pp. 3496–3504

    2021

  86. [96]

    J. Liu, X. Fan, Z. Huang, G. Wu, R. Liu, W. Zhong, Z. Luo, Target-aware dual adversar- ial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for ob- ject detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco...

    2022

  87. [97]

    H. Li, Z. Yang, Y . Zhang, D. Tao, Z. Yu, Single- image hdr reconstruction assisted ghost suppres- sion and detail preservation network for multi- exposure hdr imaging, IEEE Transactions on Computational Imaging (2024)

    2024

  88. [98]

    Jiang, B

    Z. Jiang, B. Yang, D. Pan, H. Yu, W. Gui, Irw-mef: Informative random walk for multi- exposure image fusion, Expert Systems with Ap- plications (2025) 127147

    2025

  89. [99]

    P. Mu, Z. Du, J. Liu, C. Bai, Little strokes fell great oaks: Boosting the hierarchical features for multi-exposure image fusion, in: Proceedings of the 31st ACM International Conference on Mul- timedia, 2023, pp. 2985–2993

    2023

  90. [100]

    J. Liu, G. Wu, J. Luan, Z. Jiang, R. Liu, X. Fan, Holoco: Holistic and local contrastive learning network for multi-exposure image fusion, Infor- mation Fusion 95 (2023) 237–249

    2023

  91. [101]

    G. Wu, H. Fu, J. Liu, L. Ma, X. Fan, R. Liu, Hybrid-supervised dual-search: Leveraging au- tomatic learning for loss-free multi-exposure im- age fusion, in: Proceedings of the AAAI Confer- ence on Artificial Intelligence, V ol. 38, 2024, pp. 5985–5993

    2024

  92. [102]

    Reiser, R

    C. Reiser, R. Szeliski, D. Verbin, P. Srinivasan, B. Mildenhall, A. Geiger, J. Barron, P. Hedman, Merf: Memory-efficient radiance fields for real- time view synthesis in unbounded scenes, ACM Transactions on Graphics (TOG) 42 (4) (2023) 1–12

    2023

  93. [103]

    X. Li, X. Li, H. Tan, J. Li, Samf: small-area- aware multi-focus image fusion for object de- tection, in: ICASSP 2024-2024 IEEE Interna- tional Conference on Acoustics, Speech and Sig- nal Processing (ICASSP), IEEE, 2024, pp. 3845– 3849

    2024

  94. [104]

    X. Li, F. Zhou, H. Tan, Y . Chen, W. Zuo, Multi- focus image fusion based on nonsubsampled con- tourlet transform and residual removal, Signal Processing 184 (2021) 108062

    2021

  95. [105]

    H. Li, M. Yuan, J. Li, Y . Liu, G. Lu, Y . Xu, Z. Yu, D. Zhang, Focus affinity perception and super- resolution embedding for multifocus image fu- sion, IEEE Transactions on Neural Networks and Learning Systems (2024)

    2024

  96. [106]

    Jiang, S

    S. Jiang, S. Yu, Refined multi-focus image fu- sion using multi-scale neural network with sp- swin autoencoder-based matting, Expert Systems with Applications (2025) 126980

    2025

  97. [107]

    X. Wang, L. Fang, J. Zhao, Z. Pan, H. Li, Y . Li, Mmae: A universal image fusion method via mask attention mechanism, Pattern Recognition 158 (2025) 111041

    2025

  98. [108]

    Zhang, Z

    H. Zhang, Z. Le, Z. Shao, H. Xu, J. Ma, Mff-gan: An unsupervised generative adversarial network with adaptive and gradient joint constraints for multi-focus image fusion, Information Fusion 66 (2021) 40–53

    2021

  99. [109]

    Nejati, S

    M. Nejati, S. Samavi, S. Shirani, Multi-focus im- age fusion using dictionary-based sparse repre- sentation, Information Fusion 25 (2015) 72–84. 24

    2015

  100. [110]

    Cheng, T

    C. Cheng, T. Xu, X.-J. Wu, Mufusion: A gen- eral unsupervised image fusion network based on memory unit, Information Fusion 92 (2023) 80– 92

    2023

  101. [111]

    M. Li, R. Pei, T. Zheng, Y . Zhang, W. Fu, Fu- siondiff: Multi-focus image fusion using denois- ing diffusion probabilistic models, Expert Sys- tems with Applications 238 (2024) 121664

    2024

  102. [112]

    B. Cao, Y . Sun, P. Zhu, Q. Hu, Multi-modal gated mixture of local-to-global experts for dynamic image fusion, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 23555–23564

    2023

  103. [113]

    H. Zhai, Y . Ouyang, N. Luo, L. Chen, Z. Zeng, Msi-dtrans: A multi-focus image fusion us- ing multilayer semantic interaction and dynamic transformer, Displays 85 (2024) 102837. 25

    2024

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.