Pith. sign in

REVIEW 5 major objections 7 minor 123 references

FS-Diff unifies multimodal image fusion and super-resolution into a single conditional diffusion process that recovers high-resolution fused images from low-resolution, unevenly blurred source pairs at 2x–8x magnification.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 19:05 UTC pith:HRFAWH7M

load-bearing objection A serious architecture for joint fusion+SR, but the empirical claims are currently undermined by inconsistent metric reporting and a degradation setup that doesn't match the paper's own motivating scenario. the 5 major comments →

arxiv 2509.09427 v1 pith:HRFAWH7M submitted 2025-09-11 cs.CV

FS-Diff: Semantic guidance and clarity-aware simultaneous multimodal image fusion and super-resolution

Yuchan Jie , Yushen Xu , Xiaosong Li , Fuqiang Zhou , Jianming Lv , Huafeng Li This is my paper
classification cs.CV
keywords image fusionsuper-resolutiondiffusion modelsemantic guidanceclarity-aware CLIPbidirectional Mambamultimodal benchmarkaerial imaging
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that simultaneous fusion and super-resolution of multimodal images can be done in one generative pass, rather than in separate steps. FS-Diff starts from pure Gaussian noise and iteratively denoises conditioned on the low-resolution source images and semantic embeddings extracted by a clarity-aware CLIP. The key is that when one input is blurred, the model uses the clear image's semantics; when both are blurred, it takes the maximum of both semantics. Experiments on infrared-visible, medical, and multifocus datasets at 2x, 4x, and 8x report better fidelity and downstream detection/segmentation than stepwise super-resolution-then-fusion baselines. The paper also introduces an aerial-view multimodal benchmark (AVMS) with 859 aligned image pairs and 3,821 annotated targets.

Core claim

FS-Diff treats joint fusion and super-resolution as a conditional generation problem. The forward process adds Gaussian noise to a high-resolution ground-truth fused image; the reverse process learns to remove that noise conditioned on the low-resolution source images and on semantic features. The semantic features come from a proposed clarity-sensing CLIP (CA-CLIP) that judges which input is clearer and selects the most informative semantics. A bidirectional feature Mamba block builds a global joint representation of the multimodal inputs. The paper reports that this one-stage approach outperforms two-stage methods (super-resolving then fusing) on VIF, Qabf, SSIM, and LPIPS metrics across m

What carries the argument

The load-bearing component is the conditional denoising diffusion model. The target is a high-resolution fused image F0, produced at training time by CDD; during inference the model iterates from Gaussian noise to the fused output using a modified U-Net that predicts the noise at each timestep. Conditions are injected in two ways: a bidirectional feature Mamba (BFM) concatenates the two source images and the noisy image into a global joint representation, and a clarity-aware CLIP (CA-CLIP) provides semantic embeddings via cross-attention. The clarity-sensing selection rule (use the clear image's semantics when one is blurred, take the element-wise maximum when both are blurred) is what adapt

Load-bearing premise

The low-resolution inputs are simulated by bicubic downsampling of sharp high-resolution images, so the model's 'clarity' judgement and denoising may not transfer to real long-range aerial images with motion blur, atmospheric scattering, and sensor noise.

What would settle it

Take the trained FS-Diff and run it on real aerial visible-infrared pairs captured from a drone at 5–20 m altitude with natural motion blur and haze, without bicubic preprocessing. If the fusion quality (VIF, LPIPS, and detection mAP on the AVMS annotations) drops much more than on bicubic-degraded AVMS images, the central practical claim fails. Equivalently, if replacing the bicubic training data with a blind degradation model changes the rankings among methods, the reported advantage may be an artifact of the degradation assumption.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • A single trained model can replace the standard pipeline of separate super-resolution and fusion, avoiding error accumulation between stages.
  • Fused outputs with recovered semantics should improve downstream tasks such as object detection and semantic segmentation on low-resolution aerial imagery.
  • The method extends beyond infrared-visible fusion to medical (MRI-PET, MRI-SPECT) and multifocus fusion, suggesting a general recipe for modality-agnostic joint fusion and super-resolution.
  • The new AVMS benchmark provides aligned visible-infrared aerial pairs with varying clarity and object annotations, enabling training and evaluation of long-range fusion systems.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The clarity-sensing selection rule is a form of adaptive fusion that could be generalized: a learned soft weighting of semantics might handle partial blur in both images more gracefully than the hard maximum.
  • Because the model is trained on bicubic downsampling, its performance under real atmospheric degradation is untested; a natural extension is to train with a degradation pool (motion blur, haze, noise) or use a blind-degradation model.
  • The diffusion process is computationally heavy (about 74 seconds per 128x128 image) relative to stepwise baselines; distillation or latent-space diffusion would be the natural next step to make this practical.
  • The AVMS dataset, with per-object annotations, could serve as a testbed for evaluating fusion methods by their effect on detection rather than only pixel-level metrics.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. FS-Diff proposes a unified conditional diffusion framework for simultaneous multimodal image fusion and super-resolution. The method initializes the fused high-resolution output as Gaussian noise and iteratively denoises using a modified U-Net conditioned on the low-resolution visible/infrared pair, a bidirectional feature Mamba (BFM) global representation, and semantic embeddings from a clarity-aware CLIP variant (CA-CLIP). A clarity sensing mechanism (CLSE) decides whether to use the clear image's embedding or the maximum cross-image embedding when both inputs are blurred. The paper also introduces an aerial-view multimodal benchmark (AVMS) with 859 aligned pairs, reports fusion and super-resolution results at 2x/4x/8x on public and new datasets, and evaluates detection and segmentation on the fused outputs. The central claim is that FS-Diff outperforms state-of-the-art methods at multiple magnifications while recovering richer details and semantics.

Significance. If the central claim is correct, FS-Diff is a meaningful step toward joint fusion and super-resolution in a single generative model, with a plausible mechanism for clarity-adaptive semantic conditioning. The AVMS dataset is a useful contribution for aerial multimodal fusion, and the code release is a strength. However, the quantitative evidence in the paper does not currently support the claimed superiority, and several metric/table inconsistencies make the evaluation unreliable. The core DDPM formulation is standard; the novelty lies in the BFM and CLSE conditioning. The benchmark dataset and code availability are valuable regardless of the metric issues, but the paper's main performance claims need to be substantiated with corrected and consistent evaluations.

major comments (5)
  1. [Abstract; Tables 1, 2, 3, 9] The abstract and conclusion claim that FS-Diff 'outperforms the state-of-the-art methods at multiple magnifications.' The tables do not support a blanket claim. For example, Table 1 (MSRS, both blurred, x8): FS-Diff PSNR is 15.858 vs. DDFM+SR3's 17.407, and MSE is 2632.360 vs. 1770.990; FS-Diff also loses VIF on LLVIP IR-blurred x2 (0.607 vs. CDD+SR3's 0.650, Table 2). In Table 3 (MRI-PET), FS-Diff's MSE is 3194.768, worse than U2Fusion's 1874.329. FS-Diff wins some perceptual metrics (VIF, Q_AB/F, LPIPS) in many rows, but the paper's stated conclusion of general superiority is contradicted by the reported PSNR/MSE results. The text should be revised to report strengths and weaknesses precisely rather than claiming overall superiority.
  2. [Tables 1, 2, 3, 9] Reported SSIM values exceed the theoretical maximum of 1.0 in many rows: e.g., Table 1 lists SSIM 1.139, 1.165, 1.277, 1.344, 1.377; Table 3 lists 1.377 and 1.476; Table 9 lists 1.797 and 1.941. Since SSIM is defined as a similarity index bounded above by 1, these values indicate either a nonstandard metric implementation, a computation error, or a mismatch between the metric and the images being compared. This casts doubt on the reliability of every SSIM comparison in the paper and requires correction or a clear explanation.
  3. [Table 4 vs. Table 1] The ablation table and the main results table report inconsistent numbers for the same configuration. Table 4 gives FS-Diff on MSRS (both blurred, x8) VIF = 0.737, Q_AB/F = 0.669, PSNR = 16.170; Table 1 for the identical setting gives VIF = 0.486, Q_AB/F = 0.481, PSNR = 15.858. These are materially different. If the ablation was run on a different subset or with a different protocol, that must be stated; otherwise the quantitative results are internally inconsistent and the reader cannot trust either table.
  4. [Sections 4 and 5.1; Table 6] The motivating applications are long-distance UAV imaging with motion blur, atmospheric scattering, and sensor noise, but all training and test low-resolution inputs are synthesized by bicubic downsampling (Section 5.1, 'downsampled from the HR source image via a bicubic approach'; Section 4, AVMS pairs 'downsampled via bicubic interpolation'). The only robustness test, Table 6, uses synthetic fog on AVMS. No real-captured degraded pairs or a realistic degradation model are evaluated. The paper's claim that FS-Diff addresses real long-distance imagery is therefore unverified; the limitation paragraph in Section 7 mentions only model complexity and does not acknowledge this gap. A real-capture or realistic-degradation experiment is needed before the practical-applicability claim can be sustained.
  5. [Section 5.4.2; Eq. (12)] The CLSE semantic selection rule in Eq. (12) is validated post hoc in Section 5.4.2 on the A VMS test set that is also used for the final comparisons. Table 5 compares several semantic selections on the same benchmark and then selects 'max semantics' for the both-blurred case. This is benchmark fitting: the rule is chosen based on test-set performance rather than on a held-out validation set. Since CLSE is a core contribution, the design choice should be justified by training/validation evidence or an independent test set, not by the final evaluation set used to report the method's performance.
minor comments (7)
  1. [Abstract] The abstract says the AVMS benchmark covers '600 pairs of images,' while the full text and contributions state 859 pairs. The number should be harmonized.
  2. [Section 3.2] The notation for image dimensions is confusing: x ∈ R^{3HW}, y ∈ R^{HW}, F0 ∈ R^{3HW}. This appears to use a nonstandard flattened size rather than H×W×C; please use conventional tensor notation such as R^{H×W×3}.
  3. [Table 5] The header includes a typo 'PSRN' and the column order is inconsistent with the row values (some rows show seven numeric entries while the header lists six columns). Please reformat and verify all entries.
  4. [Algorithm 2 and Eq. (18)] Algorithm 2 uses z for the inference noise, while Eq. (18) uses ε_t; it should be stated explicitly that these are the same random variable, or the notation should be unified.
  5. [Section 5.3 caption] The caption mentions '×8 magnification (162→128²)'; the '162' appears to be a typo for '16²' or should be omitted.
  6. [Table 10] The computational complexity table lists 'Diff-IF+SR3', but Diff-IF is not among the methods described in Section 5.1's comparison list. Clarify whether this method was used in the main comparisons and why it appears only here.
  7. [Section 6] The complexity table reports FLOPs for SR3 as 91681.6 G, which is about three orders of magnitude larger than typical SR3 values; this may be a unit error (G vs. M) and should be checked.

Circularity Check

1 steps flagged

Post-hoc selection of the CLSE semantic rule on the AVMS test set creates a partial circularity; the core diffusion pipeline is otherwise self-contained.

specific steps
  1. fitted input called prediction [Sec. 5.4.2, Table 5; Eq. (12); Sec. 5.1 (AVMS train/test split)]
    "As shown in Table 5 and Figure 9, for a single blurred image, choosing the clear semantics and, for two blurred images, selecting the maximum two-modal semantics can effectively enhance fusion performance."

    Eq. (12) hard-codes this selection rule (clear-image embedding when one input is clear; max(ec1,ec2) when both are blurred). Table 5 chooses among the alternatives ('VI clear+VI semantics', 'IR clear+IR semantics', 'Blurred IR &VI+joint semantics', 'Blurred IR &VI+max semantics', etc.) by ranking them on the AVMS dataset at scale 8. The AVMS split is fixed in Sec. 5.1 as training 689 / test 170, and Tables 1, 2, and 4 report 'Our FS-Diff' on the same AVMS test set. Thus the claim that the rule 'effectively enhance[s] fusion performance' is an evaluation on the very benchmark used to pick the rule; the final AVMS numbers are not independent confirmation of the CLSE choice, but the direct result of selecting the best-performing option on that test set.

full rationale

The main derivation chain is not circular. FS-Diff uses the standard DDPM objective (Eqs. 8-18), the ground-truth HR fused image F0 is obtained externally from CDD [31], and the conditioning inputs are the LR multimodal images plus semantic embeddings. The BFM and CA-CLIP components are adopted from non-overlapping prior work ([32], [84]), not from the present authors' own theorems. The bicubic-only degradation model is a correctness/generalization concern, not a circularity. The one genuine circular element is the validation of the CLSE semantic selection rule (Eq. 12) on the same AVMS test set used to choose that rule. This makes the AVMS-based evidence for that specific design choice partly fitted to the benchmark. However, this affects only the semantic-selection component and the AVMS numbers; the method is also evaluated on MSRS, M3FD, LLVIP, medical, and multifocus datasets, and the core diffusion formulation retains independent content. Therefore the overall circularity is moderate rather than severe.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

No physical invented entities are introduced. The load-bearing assumptions are the CDD-generated training target, the bicubic degradation model, and the transferability of CLIP features to heavily downsampled inputs. The method's hyperparameters are standard choices, not fitted to resolve a derivation.

free parameters (3)
  • Diffusion schedule length T = 4000
    Chosen per improved DDPM [116] in Section 5.1; controls the number of denoising iterations and trades quality against inference time.
  • Training schedule (steps, batch, learning rate) = 800k steps, batch 80, LR 1e-4
    Hand-chosen hyperparameters listed in Section 5.1; they define the reported model but are not shown to be critical.
  • Super-resolution scale factors = 2, 4, 8
    The three magnifications define the task; all LR input data is synthesized with bicubic downsampling at these scales.
axioms (4)
  • domain assumption The HR fused image F0 produced by CDD is a suitable ground truth for training the joint fusion+SR network.
    Section 5.1 states 'The HR fusion image was obtained via CDD', so the network is trained to imitate CDD's fusion output; the fusion quality ceiling is therefore CDD's.
  • domain assumption Low-resolution degradation is equivalent to bicubic downsampling of aligned HR images.
    Section 5.1 and Section 4 create LR inputs 'via a bicubic approach'/'downsampled via bicubic interpolation', omitting motion blur, atmospheric turbulence, and sensor noise.
  • domain assumption CLIP/DA-CLIP pretrained features remain semantically useful for heavily downsampled 16x16 inputs.
    The CLSE mechanism and e_ci embeddings are extracted from pretrained CLIP/DA-CLIP (Section 3.2.2); no evidence is provided that these features are stable at 8x degradation.
  • standard math The reverse diffusion posterior follows the standard DDPM parameterization.
    Equations 13-18 follow Ho et al. [30], used as standard background without modification.

pith-pipeline@v1.3.0-alltime-deepseek · 36723 in / 13564 out tokens · 154905 ms · 2026-08-04T19:05:49.896415+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of FS-Diff: Semantic guidance and clarity-aware simultaneous multimodal image fusion and super-resolution." pith.science (2026). https://pith.science/paper/HRFAWH7M

@misc{pith2026250909427,
  author       = {Pith},
  title        = {Pith review of: FS-Diff: Semantic guidance and clarity-aware simultaneous multimodal image fusion and super-resolution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HRFAWH7M}},
  note         = {Machine review of arXiv:2509.09427}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

As an influential information fusion and low-level vision technique, image fusion integrates complementary information from source images to yield an informative fused image. A few attempts have been made in recent years to jointly realize image fusion and super-resolution. However, in real-world applications such as military reconnaissance and long-range detection missions, the target and background structures in multimodal images are easily corrupted, with low resolution and weak semantic information, which leads to suboptimal results in current fusion techniques. In response, we propose FS-Diff, a semantic guidance and clarity-aware joint image fusion and super-resolution method. FS-Diff unifies image fusion and super-resolution as a conditional generation problem. It leverages semantic guidance from the proposed clarity sensing mechanism for adaptive low-resolution perception and cross-modal feature extraction. Specifically, we initialize the desired fused result as pure Gaussian noise and introduce the bidirectional feature Mamba to extract the global features of the multimodal images. Moreover, utilizing the source images and semantics as conditions, we implement a random iterative denoising process via a modified U-Net network. This network istrained for denoising at multiple noise levels to produce high-resolution fusion results with cross-modal features and abundant semantic information. We also construct a powerful aerial view multiscene (AVMS) benchmark covering 600 pairs of images. Extensive joint image fusion and super-resolution experiments on six public and our AVMS datasets demonstrated that FS-Diff outperforms the state-of-the-art methods at multiple magnifications and can recover richer details and semantics in the fused images. The code is available at https://github.com/XylonXu01/FS-Diff.

Figures

Figures reproduced from arXiv: 2509.09427 by Fuqiang Zhou, Huafeng Li, Jianming Lv, Xiaosong Li, Yuchan Jie, Yushen Xu.

Figure 1
Figure 1. Figure 1: Our FS-Diff demonstrates the outstanding fusion performance of joint image fusion and super-resolution with a scaling factor of 8 in five public and our AVMS datasets. The first four boxes show the VI-IR dataset, demonstrating three fusion scenarios: both VI and IR blurred, VI blurred, and IR blurred. The remaining two boxes present the multimodal medical dataset, where both images are blurred. Generally, … view at source ↗
Figure 2
Figure 2. Figure 2: The framework of FS-Diff. Ft and F0 represent the noisy output image with t timesteps and the ground truth. The forward diffusion process q(Ft |Ft−1) adds Gaussian noise to F0, and the backward diffusion process pθ(Ft−1|Ft , x, y) iteratively denoise. The CLSE mechanism dynamically selects semantic extraction strategies for multimodal images with inconsistent resolution. In Case 1 (single-image blur), CA-C… view at source ↗
Figure 3
Figure 3. Figure 3: The overview of the BFM. 3.2.2. CA-CLIP and the CLSE mechanism This section introduces the CA-CLIP and the CLSE mechanism. By leveraging CLIP’s [83] pre-trained cross￾modal alignment capability, our framework enables precise semantic extraction from heterogeneous input modalities with varying clarity. Moreover, this integration enhances the generalizability of the proposed method, enabling its adaptation t… view at source ↗
Figure 4
Figure 4. Figure 4: (a) Image pair acquisition, (b) post-processing, and (c) scene distribution for AVMS. [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Viewable distribution of scenes in our AVMS dataset. The first line is the visible images, and the second line is the corresponding infrared [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Fusion and super-resolution (scale:8, 4 and 2) results for di [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Fusion and super-resolution (scale:8) results for FS-Di [PITH_FULL_IMAGE:figures/full_fig_p018_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Visual comparison of ablation studies on CA-CLIP and CLSE Mechanism. [PITH_FULL_IMAGE:figures/full_fig_p020_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: The impact of content embeddings from images in di [PITH_FULL_IMAGE:figures/full_fig_p021_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Visualization of the effect of CLSE’s clarity judgment on the unseen foggy and low resolution (scale:8) AVMS dataset. For quantitative analysis, from [PITH_FULL_IMAGE:figures/full_fig_p021_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Qualitative comparisons of different fusion and super-resolution (scale: 8) detection results on the LLVIP and AVMS datasets. The area circled by the red ellipse indicates the missed detection of the comparison methods compared with our FS-Diff. separate processes of fusion and super-resolution lead to the loss of semantic information and the accumulation of errors, causing the comparison methods to fail … view at source ↗
Figure 12
Figure 12. Figure 12: Qualitative comparisons of different fusion and super-resolution (scale: 8) segmentation results on the MSRS and AVMS datasets. The red regions represent the missing segmentations of the comparison methods compared with our FS-Diff [PITH_FULL_IMAGE:figures/full_fig_p023_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Visual Comparison of SR3-only super-resolution and the proposed method for simultaneous multi-source image fusion and super [PITH_FULL_IMAGE:figures/full_fig_p024_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Fusion and super-resolution (scale: 8) results for di [PITH_FULL_IMAGE:figures/full_fig_p025_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

123 extracted references · 10 linked inside Pith

  1. [1]

    H. Xu, J. Yuan, J. Ma, Murf: Mutually reinforcing multi-modal image registration and fusion, IEEE transactions on pattern analysis and machine intelligence (2023)

    2023

  2. [2]

    Z. Zhao, H. Bai, Y . Zhu, J. Zhang, S. Xu, Y . Zhang, K. Zhang, D. Meng, R. Timofte, L. Van Gool, Ddfm: denoising diffusion model for multi-modality image fusion, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 8082–8093

    2023

  3. [3]

    W. Zhao, S. Xie, F. Zhao, Y . He, H. Lu, Metafusion: Infrared and visible image fusion via meta-feature em- bedding from object detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 13955–13965

    2023

  4. [4]

    C. He, K. Li, G. Xu, Y . Zhang, R. Hu, Z. Guo, X. Li, Degradation-resistant unfolding network for heterogeneous image fusion, in: Proceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 12611– 12621

    2023

  5. [5]

    Li, X.-J

    H. Li, X.-J. Wu, Crossfuse: A novel cross attention mechanism based infrared and visible image fusion ap- proach, Information Fusion 103 (2024) 102147

    2024

  6. [6]

    L. Tang, Y . Deng, Y . Ma, J. Huang, J. Ma, Superfusion: A versatile image registration and fusion network with semantic awareness, IEEE/CAA Journal of Automatica Sinica 9 (12) (2022) 2121–2137

    2022

  7. [7]

    S. M. Z. S. Z. Ariffin, N. Jamil, P. N. M. A. Rahman, Can thermal and visible image fusion improves ear recognition?, in: 2017 8th International Conference on Information Technology (ICIT), IEEE, 2017, pp. 780– 784

    2017

  8. [8]

    R. Liu, Z. Liu, J. Liu, X. Fan, Searching a hierarchically aggregated fusion architecture for fast multi-modality image fusion, in: Proceedings of the 29th ACM International Conference on Multimedia, 2021, pp. 1600–1608

    2021

  9. [9]

    C. He, K. Li, Y . Zhang, L. Tang, Y . Zhang, Z. Guo, X. Li, Camouflaged object detection with feature decom- position and edge reconstruction, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 22046–22055

    2023

  10. [10]

    Li, X.-J

    H. Li, X.-J. Wu, J. Kittler, Mdlatlrr: A novel decomposition method for infrared and visible image fusion, IEEE Transactions on Image Processing 29 (2020) 4733–4746

    2020

  11. [11]

    Y . Jie, X. Li, H. Tan, F. Zhou, G. Wang, Multi-modal medical image fusion via multi-dictionary and truncated huber filtering, Biomedical Signal Processing and Control 88 (2024) 105671

    2024

  12. [12]

    Y . Jie, X. Li, F. Zhou, H. Tan, et al., Medical image fusion based on extended difference-of-gaussians and edge-preserving, Expert Systems with Applications 227 (2023) 120301

    2023

  13. [13]

    Zhang, Y

    X. Zhang, Y . Demiris, Visible and infrared image fusion using deep learning, IEEE Transactions on Pattern Analysis and Machine Intelligence (2023)

    2023

  14. [14]

    Karim, G

    S. Karim, G. Tong, J. Li, A. Qadir, U. Farooq, Y . Yu, Current advances and future perspectives of image fusion: A comprehensive review, Information Fusion 90 (2023) 185–217

    2023

  15. [15]

    H. Li, Y . Cen, Y . Liu, X. Chen, Z. Yu, Different input resolutions and arbitrary output resolution: A meta learning-based deep framework for infrared and visible image fusion, IEEE Transactions on Image Processing 30 (2021) 4070–4083

    2021

  16. [16]

    W. Xiao, Y . Zhang, H. Wang, F. Li, H. Jin, Heterogeneous knowledge distillation for simultaneous infrared- visible image fusion and super-resolution, IEEE Transactions on Instrumentation and Measurement 71 (2022) 1–15. 27

    2022

  17. [17]

    H. Yin, S. Li, L. Fang, Simultaneous image fusion and super-resolution using sparse representation, Information Fusion 14 (3) (2013) 229–240

    2013

  18. [18]

    H. Li, M. Yuan, J. Li, Y . Liu, G. Lu, Y . Xu, Z. Yu, D. Zhang, Focus affinity perception and super-resolution embedding for multifocus image fusion, IEEE Transactions on Neural Networks and Learning Systems (2024)

    2024

  19. [19]

    Y . Sun, B. Cao, P. Zhu, Q. Hu, Drone-based rgb-infrared cross-modality vehicle detection via uncertainty-aware learning, IEEE Transactions on Circuits and Systems for Video Technology 32 (10) (2022) 6700–6713

    2022

  20. [20]

    Huang, J

    B. Huang, J. Li, J. Chen, G. Wang, J. Zhao, T. Xu, Anti-uav410: A thermal infrared benchmark and cus- tomized scheme for tracking drones in the wild, IEEE Transactions on Pattern Analysis and Machine Intelli- gence (2023)

    2023

  21. [21]

    Razakarivony, F

    S. Razakarivony, F. Jurie, Vehicle detection in aerial imagery: A small target detection benchmark, Journal of Visual Communication and Image Representation 34 (2016) 187–203

    2016

  22. [22]

    Toet, The tno multiband image data collection, Data in brief 15 (2017) 249–251

    A. Toet, The tno multiband image data collection, Data in brief 15 (2017) 249–251

    2017

  23. [23]

    H. Xu, J. Ma, J. Jiang, X. Guo, H. Ling, U2fusion: A unified unsupervised image fusion network, IEEE Transactions on Pattern Analysis and Machine Intelligence 44 (1) (2020) 502–518

    2020

  24. [24]

    X. Jia, C. Zhu, M. Li, W. Tang, W. Zhou, Llvip: A visible-infrared paired dataset for low-light vision, in: Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 3496–3504

    2021

  25. [25]

    J. Liu, X. Fan, Z. Huang, G. Wu, R. Liu, W. Zhong, Z. Luo, Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 5802–5811

    2022

  26. [26]

    L. Tang, J. Yuan, H. Zhang, X. Jiang, J. Ma, Piafusion: A progressive infrared and visible image fusion network based on illumination aware, Information Fusion 83 (2022) 79–92

    2022

  27. [27]

    Fu, X.-J

    Y . Fu, X.-J. Wu, T. Durrani, Image fusion based on generative adversarial network consistent with perception, Information Fusion 72 (2021) 110–125

    2021

  28. [28]

    J. Ma, H. Xu, J. Jiang, X. Mei, X.-P. Zhang, Ddcgan: A dual-discriminator conditional generative adversarial network for multi-resolution image fusion, IEEE Transactions on Image Processing 29 (2020) 4980–4995

    2020

  29. [29]

    J. Gui, Z. Sun, Y . Wen, D. Tao, J. Ye, A review on generative adversarial networks: Algorithms, theory, and applications, IEEE transactions on knowledge and data engineering 35 (4) (2021) 3313–3332

    2021

  30. [30]

    J. Ho, A. Jain, P. Abbeel, Denoising diffusion probabilistic models, Advances in neural information processing systems 33 (2020) 6840–6851

    2020

  31. [31]

    Z. Zhao, H. Bai, J. Zhang, Y . Zhang, S. Xu, Z. Lin, R. Timofte, L. Van Gool, Cddfuse: Correlation-driven dual- branch feature decomposition for multi-modality image fusion, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 5906–5916

    2023

  32. [32]

    L. Zhu, B. Liao, Q. Zhang, X. Wang, W. Liu, X. Wang, Vision mamba: Efficient visual representation learning with bidirectional state space model, arXiv preprint arXiv:2401.09417 (2024)

    Pith/arXiv arXiv 2024

  33. [33]

    Ronneberger, P

    O. Ronneberger, P. Fischer, T. Brox, U-net: Convolutional networks for biomedical image segmentation, in: Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, Springer, 2015, pp. 234–241

    2015

  34. [34]

    X. Li, F. Zhou, H. Tan, Y . Chen, W. Zuo, Multi-focus image fusion based on nonsubsampled contourlet trans- form and residual removal, Signal Processing 184 (2021) 108062. 28

    2021

  35. [35]

    Treece, Real image denoising with a locally-adaptive bitonic filter, IEEE Transactions on Image Processing 31 (2022) 3151–3165

    G. Treece, Real image denoising with a locally-adaptive bitonic filter, IEEE Transactions on Image Processing 31 (2022) 3151–3165

    2022

  36. [36]

    K. He, J. Sun, Fast guided filter, arXiv preprint arXiv:1505.00996 (2015)

    Pith/arXiv arXiv 2015

  37. [37]

    Y . Jie, F. Zhou, H. Tan, G. Wang, X. Cheng, X. Li, Tri-modal medical image fusion based on adaptive energy choosing scheme and sparse representation, Measurement 204 (2022) 112038

    2022

  38. [38]

    X. Li, W. Wan, F. Zhou, X. Cheng, Y . Jie, H. Tan, Medical image fusion based on sparse representation and neighbor energy activity, Biomedical Signal Processing and Control 80 (2023) 104353

    2023

  39. [39]

    X. Li, F. Zhou, H. Tan, Joint image fusion and denoising via three-layer decomposition and sparse representa- tion, Knowledge-Based Systems 224 (2021) 107087

    2021

  40. [40]

    Zhang, Y

    Q. Zhang, Y . Liu, R. S. Blum, J. Han, D. Tao, Sparse representation based multi-sensor image fusion for multi-focus and multi-modality images: A review, Information Fusion 40 (2018) 57–75

    2018

  41. [41]

    Y . Liu, X. Chen, A. Liu, R. K. Ward, Z. J. Wang, Recent advances in sparse representation based medical image fusion, IEEE Instrumentation & Measurement Magazine 24 (2) (2021) 45–53

    2021

  42. [42]

    Aydemir, L

    B. Aydemir, L. Hoffstetter, T. Zhang, M. Salzmann, S. Süsstrunk, Tempsal-uncovering temporal information for deep saliency prediction, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 6461–6470

    2023

  43. [43]

    Downes, S

    J. Downes, S. Saltwick, A. Chen, Saliency driven imagery preprocessing for efficient compression-industrial paper, in: Proceedings of the 31st ACM International Conference on Advances in Geographic Information Systems, 2023, pp. 1–11

    2023

  44. [44]

    Y . Su, J. Deng, R. Sun, G. Lin, H. Su, Q. Wu, A unified transformer framework for group-based segmentation: Co-segmentation, co-saliency detection and video salient object detection, IEEE Transactions on Multimedia (2023)

    2023

  45. [45]

    J. Ma, L. Tang, M. Xu, H. Zhang, G. Xiao, Stdfusionnet: An infrared and visible image fusion network based on salient target detection, IEEE Transactions on Instrumentation and Measurement 70 (2021) 1–13

    2021

  46. [46]

    J. Liu, R. Dian, S. Li, H. Liu, Sgfusion: A saliency guided deep-learning framework for pixel-level image fusion, Information Fusion 91 (2023) 205–214

    2023

  47. [47]

    D. Wang, J. Liu, R. Liu, X. Fan, An interactively reinforced paradigm for joint infrared-visible image fusion and saliency object detection, Information Fusion 98 (2023) 101828

    2023

  48. [48]

    J. Liu, X. Fan, J. Jiang, R. Liu, Z. Luo, Learning a deep multi-scale feature ensemble and an edge-attention guidance for image fusion, IEEE Transactions on Circuits and Systems for Video Technology 32 (1) (2021) 105–119

    2021

  49. [49]

    Liang, J

    P. Liang, J. Jiang, X. Liu, J. Ma, Fusion from decomposition: A self-supervised decomposition approach for image fusion, in: European Conference on Computer Vision, Springer, 2022, pp. 719–735

    2022

  50. [50]

    Z. Liu, J. Liu, G. Wu, L. Ma, X. Fan, R. Liu, Bi-level dynamic learning for jointly multi-modality image fusion and beyond, arXiv preprint arXiv:2305.06720 (2023)

    Pith/arXiv arXiv 2023

  51. [51]

    B. Cao, Y . Sun, P. Zhu, Q. Hu, Multi-modal gated mixture of local-to-global experts for dynamic image fusion, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 23555–23564

    2023

  52. [52]

    Y . Guan, R. Xu, M. Yao, L. Wang, Z. Xiong, Mutual-guided dynamic network for image fusion, in: Proceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 1779–1788. 29

    2023

  53. [53]

    H. Xu, J. Ma, Z. Le, J. Jiang, X. Guo, Fusiondn: A unified densely connected network for image fusion, in: Proceedings of the AAAI conference on artificial intelligence, V ol. 34, 2020, pp. 12484–12491

    2020

  54. [54]

    L. Tang, X. Xiang, H. Zhang, M. Gong, J. Ma, Divfusion: Darkness-free infrared and visible image fusion, Information Fusion 91 (2023) 477–493

    2023

  55. [55]

    Huang, J

    Z. Huang, J. Liu, X. Fan, R. Liu, W. Zhong, Z. Luo, Reconet: Recurrent correction network for fast and efficient multi-modality image fusion, in: European conference on computer Vision, Springer, 2022, pp. 539–555

    2022

  56. [56]

    D. Wang, J. Liu, X. Fan, R. Liu, Unsupervised misaligned infrared and visible image fusion via cross-modality image generation and registration, arXiv preprint arXiv:2205.11876 (2022)

    Pith/arXiv arXiv 2022

  57. [57]

    H. Li, J. Liu, Y . Zhang, Y . Liu, A deep learning framework for infrared and visible image fusion without strict registration, International Journal of Computer Vision 132 (5) (2024) 1625–1644

    2024

  58. [58]

    H. Li, J. Zhao, J. Li, Z. Yu, G. Lu, Feature dynamic alignment and refinement for infrared–visible image fusion: Translation robust fusion, Information Fusion 95 (2023) 26–41

    2023

  59. [59]

    Zhang, H

    H. Zhang, H. Xu, X. Tian, J. Jiang, J. Ma, Image fusion meets deep learning: A survey and perspective, Information Fusion 76 (2021) 323–336

    2021

  60. [60]

    L. Tang, H. Zhang, H. Xu, J. Ma, Rethinking the necessity of image fusion in high-level vision tasks: A practical infrared and visible image fusion network based on progressive semantic injection and scene fidelity, Information Fusion 99 (2023) 101870

    2023

  61. [61]

    Z. Zhao, S. Xu, C. Zhang, J. Liu, P. Li, J. Zhang, Didfuse: Deep image decomposition for infrared and visible image fusion, arXiv preprint arXiv:2003.09210 (2020)

    Pith/arXiv arXiv 2003

  62. [62]

    Li, X.-J

    H. Li, X.-J. Wu, J. Kittler, Rfn-nest: An end-to-end residual fusion network for infrared and visible images, Information Fusion 73 (2021) 72–86

    2021

  63. [63]

    M. Zhou, J. Huang, N. Zheng, C. Li, Learned image reasoning prior penetrates deep unfolding network for panchromatic and multi-spectral image fusion, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 12398–12407

    2023

  64. [64]

    H. Li, T. Xu, X.-J. Wu, J. Lu, J. Kittler, Lrrnet: A novel representation learning guided fusion network for infrared and visible images, IEEE transactions on pattern analysis and machine intelligence (2023)

    2023

  65. [65]

    Z. Zhao, S. Xu, J. Zhang, C. Liang, C. Zhang, J. Liu, Efficient and model-based infrared and visible image fusion via algorithm unrolling, IEEE Transactions on Circuits and Systems for Video Technology 32 (3) (2021) 1186–1196

    2021

  66. [66]

    H. Zhou, W. Wu, Y . Zhang, J. Ma, H. Ling, Semantic-supervised infrared and visible image fusion via a dual- discriminator generative adversarial network, IEEE Transactions on Multimedia 25 (2021) 635–648

    2021

  67. [67]

    Y . Gao, S. Ma, J. Liu, Dcdr-gan: A densely connected disentangled representation generative adversarial net- work for infrared and visible image fusion, IEEE Transactions on Circuits and Systems for Video Technology 33 (2) (2022) 549–561

    2022

  68. [68]

    J. Ma, H. Zhang, Z. Shao, P. Liang, H. Xu, Ganmcc: A generative adversarial network with multiclassification constraints for infrared and visible image fusion, IEEE Transactions on Instrumentation and Measurement 70 (2020) 1–14

    2020

  69. [69]

    Y . Rao, D. Wu, M. Han, T. Wang, Y . Yang, T. Lei, C. Zhou, H. Bai, L. Xing, At-gan: A generative adversarial network with attention and transition for infrared and visible image fusion, Information Fusion 92 (2023) 336– 349. 30

    2023

  70. [70]

    J. Ma, W. Yu, P. Liang, C. Li, J. Jiang, Fusiongan: A generative adversarial network for infrared and visible image fusion, Information fusion 48 (2019) 11–26

    2019

  71. [71]

    Li, X.-J

    H. Li, X.-J. Wu, Densefuse: A fusion approach to infrared and visible images, IEEE Transactions on Image Processing 28 (5) (2018) 2614–2623

    2018

  72. [72]

    Zhang, H

    H. Zhang, H. Xu, Y . Xiao, X. Guo, J. Ma, Rethinking the image fusion: A fast unified image fusion network based on proportional maintenance of gradient and intensity, in: Proceedings of the AAAI conference on artificial intelligence, V ol. 34, 2020, pp. 12797–12804

    2020

  73. [73]

    J. Liu, R. Lin, G. Wu, R. Liu, Z. Luo, X. Fan, Coconet: Coupled contrastive learning network with multi-level feature ensemble for multi-modality image fusion, International Journal of Computer Vision 132 (5) (2024) 1748–1775

    2024

  74. [74]

    X. Yi, H. Xu, H. Zhang, L. Tang, J. Ma, Text-if: Leveraging semantic text guidance for degradation-aware and interactive image fusion, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 27026–27035

    2024

  75. [75]

    Zhang, L

    J. Zhang, L. Jiao, W. Ma, F. Liu, X. Liu, L. Li, P. Chen, S. Yang, Transformer based conditional gan for multimodal image fusion, IEEE Transactions on Multimedia (2023)

    2023

  76. [76]

    Y . Sun, B. Cao, P. Zhu, Q. Hu, Detfusion: A detection-driven infrared and visible image fusion network, in: Proceedings of the 30th ACM international conference on multimedia, 2022, pp. 4003–4011

    2022

  77. [77]

    L. Tang, J. Yuan, J. Ma, Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network, Information Fusion 82 (2022) 28–42

    2022

  78. [78]

    J. Liu, Z. Liu, G. Wu, L. Ma, R. Liu, W. Zhong, Z. Luo, X. Fan, Multi-interactive feature learning and a full-time multi-modality benchmark for image fusion and segmentation, in: Proceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 8115–8124

    2023

  79. [79]

    J. Liu, G. Wu, Z. Liu, L. Ma, R. Liu, X. Fan, Where elegance meets precision: Towards a compact, automatic, and flexible framework for multi-modality image fusion and applications, in: IJCAI, 2024

    2024

  80. [80]

    Zhang, X

    H. Zhang, X. Zuo, J. Jiang, C. Guo, J. Ma, Mrfs: Mutually reinforcing image fusion and segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 26974– 26983

    2024

Showing first 80 references.