Pith. sign in

REVIEW 2 major objections 4 minor 63 references

Infrared and visible images from uncalibrated sensors can be aligned implicitly by letting their latent tokens interact through shared self-attention during diffusion denoising, eliminating explicit registration or warping.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-07-31 23:01 UTC pith:32A35TLX

load-bearing objection Well-engineered unified latent diffusion for IR SR + fusion, but the fusion comparison is circular (target = Mask-Difuser output) and the real-mobile calibration-free claim rests on qualitative evidence only. the 2 major comments →

arxiv 2607.24110 v1 pith:32A35TLX submitted 2026-07-27 cs.CV cs.LGphysics.optics

BeyondFusion: Self-Aligned Latent Diffusion for Calibration-Free Infrared Super-Resolution and Infrared-Visible Fusion

Minchong Chen , Xiaoyun Yuan , Minyu Cao , Jianing Zhang , Jun Zhang , Shuyang Liu , Xiaokang Yang This is my paper
classification cs.CV cs.LGphysics.optics
keywords infrared super-resolutioninfrared-visible fusionlatent diffusionself-attentioncross-modal alignmentcalibration-freemisalignment augmentationmobile imaging
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

BeyondFusion argues that cross-sensor alignment need not be a separate geometric preprocessing step for infrared-visible imaging. It represents infrared and visible inputs as latent tokens and lets their correspondence emerge from content-aware attention inside a latent diffusion process, rather than from a predicted warp. This one mechanism supports two output tasks—visible-guided infrared super-resolution and infrared-visible fusion—as different readouts of the same generative model. The paper demonstrates this on public benchmarks, simulated misalignments, and a real smartphone-mounted infrared camera, including gains in downstream pedestrian detection. The significance is a calibration-free path to multimodal imaging under realistic hardware constraints.

Core claim

The paper's central claim is that a latent diffusion model can learn cross-modal correspondence implicitly through shared self-attention. The CMSA module folds the two modalities' latents into one sequence before the existing self-attention layers, so queries, keys, and values from both modalities attend to each other; no alignment matrix, deformation field, or warp is predicted. Synthetic geometric perturbations during training prevent overfitting to fixed pixel neighbors. The paper reports competitiveness on aligned inputs, robustness at 64×64 infrared resolution and under misalignment, a small joint-training tradeoff, and improved zero-shot pedestrian detection relative to methods that as

What carries the argument

The Cross-Modal Self-Aligning (CMSA) module is the load-bearing mechanism. Eq. (2) reshapes the folded latent z_l (batch × two modalities × channels × height × width) into a single sequence of visible and infrared tokens, runs the existing self-attention operation, and reshapes back. Because both modalities share the same query/key/value projections, attention weights encode content-adaptive correspondences that can move visible structure to infrared locations and vice versa, without a geometric model. The companion Misalignment Augmentation module applies controlled shifts, scales, rotations, and perspective warps during training, which the paper argues is sufficient to discourage reliance

Load-bearing premise

The claim that the method is calibration-free for real sensors rests on the assumption that synthetic geometric perturbations applied during training are sufficient to teach the latent attention to handle the actual, device-dependent misalignments of real infrared-visible hardware; the evidence on real mobile captures is qualitative for fusion and no-reference-only for super-resolution, so this transfer from synthetic augmentation to real misalignment is not quantitatively es

What would settle it

Collect a dataset of real infrared-visible pairs where the two sensors' relative pose and timing offset can be mechanically varied over a known range, with ground-truth correspondences. If the method's PSNR/SSIM (or downstream detection mAP) degrades sharply as the real misalignment magnitude increases beyond the geometric perturbations used in training—while an explicit-registration baseline degrades more gracefully—then the 'calibration-free' generalization claim would be falsified. A simpler check: evaluate the trained model on real mobile captures with a full-reference protocol (e.g., usin

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If correct, infrared-visible camera systems can abandon explicit calibration and registration for downstream tasks, eliminating a fragile pipeline stage whose errors produce ghosting and duplicated edges.
  • The same latent process can produce both a super-resolved infrared image and a fused RGB image; joint training costs only a small per-task performance drop, so a single model can serve both purposes.
  • Fusion quality degrades only mildly as infrared resolution drops from 512×512 to 64×64 even under misalignment, suggesting the method tolerates the large resolution gaps typical of compact infrared sensors.
  • Fused outputs from the uncalibrated setting improve zero-shot pedestrian detection, indicating the implicit alignment preserves task-relevant thermal and structural cues.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The token-interaction principle is a general recipe for multimodal low-level vision: any set of heterogeneous observations (different resolutions, frame times, or spectral bands) could be fused by letting their latents attend within one diffusion process, with no task-specific alignment network.
  • The robustness claim is bounded by the augmentation family; a natural test is to stress the model with misalignments far outside the training range (e.g., large rotation or strong perspective) and measure when fusion quality breaks down.
  • Because the correspondence is learned rather than geometric, the approach may extend to non-rigid or temporally varying misalignment (atmospheric wobble, hand-held shake, moving objects) where parametric warps are inadequate, if the training augmentation covers those variations.
  • A practical deployment would need to check whether the latent attention remains stable when visible cues are misleading (e.g., strong glare or reflections), since attention can distribute weight based on appearance rather than physical correspondence.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes BeyondFusion, a unified latent diffusion framework for visible-guided infrared super-resolution (SR) and infrared-visible fusion (IVF) under uncalibrated mobile imaging. The method folds visible and infrared latents into the batch dimension and modifies the U-Net self-attention layers (CMSA) to process a joint token sequence, thereby avoiding explicit registration or geometric warping. Training uses misalignment-aware augmentation and one-step SD-Turbo with LoRA. Experiments cover public benchmarks, simulated misalignment, a real mobile infrared-visible system, and downstream pedestrian detection, with separate and jointly trained variants of the model.

Significance. The high-level idea—treating self-attention as a shared cross-modal workspace and decoding two task outputs from one denoising process—is interesting and potentially useful for practical multimodal imaging. The SR evaluation is fairly standard: synthetic degradations on public benchmarks, a no-reference real-mobile test, and qualitative comparisons. The paper also includes ablations and a downstream detection experiment. However, the central fusion claim is compromised by a circular evaluation: the fusion supervision in Eq. (6) is generated by Mask-Difuser, which is also one of the main comparison baselines. In addition, the 'calibration-free' claim rests on synthetic-to-real transfer that is not quantitatively validated on real mobile captures. These issues must be addressed before the paper's core contributions can be accepted.

major comments (2)
  1. [Eq. (6) and Tables III, VI] The fusion training target I_fusion in Eq. (6) is generated by Mask-Difuser [16]. Mask-Difuser then appears as a comparison baseline in the aligned-fusion evaluation (Table III) and in the downstream detection experiment (Table VI). This makes the reported fusion superiority over Mask-Difuser circular: BeyondFusion is trained, in part, to reproduce Mask-Difuser's output, so it is a distillate/refinement of that baseline rather than an independent method. Please remove Mask-Difuser from the baseline comparisons, or retrain/evaluate with an independent fusion target (e.g., a non-diffusion fusion method or a manually constructed target), and revise the claims accordingly.
  2. [Sec. III-D, Sec. IV-C-3, Table II] The 'calibration-free' claim is load-bearing and depends on the assumption that the synthetic perturbation family (shift, scale, rotation, perspective) transfers to real sensor misalignment. Section III-D explicitly states that these perturbations 'are not intended to reproduce the exact distribution of real sensor misalignment' and asserts sufficiency without evidence. The real-mobile fusion evaluation (Fig. 7) is qualitative only, and the mobile SR evaluation (Table II) relies solely on no-reference metrics MUSIQ/MANIQA. There is no quantitative demonstration that the method generalizes to the real parallax, FOV mismatch, and temporal offset of the mobile system. Please add quantitative real-mobile evaluation, such as detection mAP on mobile captures, correspondence/alignment error, or a perceptual study with statistical testing.
minor comments (4)
  1. [Table III] Some table entries are missing spaces between numbers, e.g., '1.8115.124' and '1.7604.723' should be '1.811 5.124' and '1.760 4.723'. Please proofread all tables for typographical errors.
  2. [Eq. (1) and Fig. 1] The notation in Eq. (1) uses 'fold' and later 'unfold', but the figure labels (z_mm^k, etc.) are inconsistent with the equation. Align the notation between the main text and the figure for clarity.
  3. [Sec. IV-A-5] The evaluation metric section cites [62] for MANIQA, but the reference appears to describe TopIQ. Please provide the correct MANIQA reference or clarify.
  4. [Sec. IV-F] The ablation study reports one configuration set, but does not discuss sensitivity to the misalignment augmentation ranges or the guidance scale omega=1.1. Adding a small sensitivity analysis would strengthen the robustness claims.

Circularity Check

1 steps flagged

IVF evaluation is circular: the fusion training target is Mask-Difuser's output, and Mask-Difuser is then used as a comparison baseline.

specific steps
  1. fitted input called prediction [Sec. IV-A-3, Eq. (6); Sec. IV-A-4; Table III]
    "For image fusion, let I_fusion denote the fused supervision generated by Mask-Difuser [16] ... L_int = ||Y(Ifusion_pred)-Y(I_fusion)||_1, L_color = ||C(Ifusion_pred)-C(I_vis)||_1 (6). ... For infrared-visible image fusion, we compare with CDDFuse [15], Mask-Difuser [16], and C-OPDR [20]."

    The fusion training target I_fusion is the output of Mask-Difuser, and Mask-Difuser is then one of the baselines in Table III (and Table VI). The reported 'outperformance' over Mask-Difuser is therefore a student-vs-teacher comparison, not an independent benchmark. The model is explicitly optimized to reproduce Mask-Difuser's fused images (with additional intensity/color losses), so beating the teacher on fusion metrics does not demonstrate that the proposed cross-modal mechanism produces better fusion than the baseline; it only shows the student can refine the teacher's own output. This makes the IVF evaluation circular for the claim of superiority over Mask-Difuser.

full rationale

The only load-bearing circularity is in the infrared-visible fusion evaluation: Eq. (6) defines the fusion supervision as Mask-Difuser's output, and Mask-Difuser is also a comparison baseline, so that particular comparison is contaminated. The infrared super-resolution results are not circular: Eq. (5) supervises against genuine high-resolution infrared ground truth, so the SR claims stand on independent data. The paper's own Sec. III-D limitation statement — 'These perturbations are not intended to reproduce the exact distribution of real sensor misalignment... synthetic perturbations are sufficient' — is a real generalization weakness for the 'calibration-free' claim, but it is a correctness/transfer risk, not circularity. No load-bearing self-citation of the authors' prior 3M-TI work is used to justify the main mechanism. The central CMSA idea and the SR validation are independent; the circularity is confined to the fusion readout. Score 6 reflects partial circularity: one of the two headline tasks has an evaluation that reduces to a teacher-student comparison rather than an independent benchmark.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 0 invented entities

The framework rests on the SD-Turbo latent prior, the assumption that token-level attention can substitute for geometric alignment, and — most problematically — a fusion training target produced by Mask-Difuser, a method that also appears as a comparison baseline. These assumptions are partly acknowledged in the paper but are not independently validated.

free parameters (7)
  • λ_sr_L2 = 10 = 10
    Loss weight for SR L2 term, chosen by hand (Sec. IV-A-3).
  • λ_fu_L2 = 2 = 2
    Loss weight for fusion L2 term, chosen by hand (Sec. IV-A-3).
  • CFG guidance scale ω = 1.1
    Guidance scale used in all inference runs, chosen by hand (Eq. 8).
  • LoRA rank (U-Net / VAE decoder) = 16 / 4
    Rank of LoRA adaptation, not swept; capacity parameter.
  • Positive:negative CFG sample ratio = 7:3
    Ratio of positive to negative branches during CFG training.
  • Gaussian noise added to LR infrared = not reported
    The degradation simulation adds Gaussian noise, but the standard deviation is not specified; this affects the SR condition distribution.
  • Misalignment augmentation ranges = not reported
    Translation, scale, rotation, and perspective perturbation ranges are not quantified; these ranges determine the calibration-free robustness claim.
axioms (5)
  • domain assumption A pretrained SD-Turbo one-step latent diffusion model provides a valid latent workspace for cross-modal interaction
    The method relies on the generative prior and latent geometry of SD-Turbo; if the latent space cannot support cross-modal semantic correspondences, the central mechanism fails.
  • domain assumption Self-attention over concatenated visible+IR tokens yields content-adaptive correspondence without explicit alignment
    This is the core CMSA hypothesis (Eq. 2); the paper validates it empirically but does not prove or analyze when it holds.
  • ad hoc to paper Mask-Difuser-generated fusion images are valid supervised targets for training a fusion model
    Eq. (6) defines I_fusion via Mask-Difuser; if those targets are biased or erroneous, the fusion readout inherits the bias. This also creates a circular comparison with Mask-Difuser as a baseline.
  • domain assumption The VAE encoder/decoder of SD-Turbo handles single-channel IR duplicated to 3 channels without losing thermal information
    IR images are duplicated into three channels to match the VAE input; no analysis of the spectral mismatch is provided.
  • domain assumption Synthetic misalignment augmentation transfers to real sensor misalignment
    Sec. III-D explicitly says perturbations are not intended to reproduce the real distribution; the real mobile validation is qualitative and limited.

pith-pipeline@v1.3.0-alltime-deepseek · 17389 in / 16914 out tokens · 142856 ms · 2026-07-31T23:01:16.374754+00:00 · methodology

0 comments
read the original abstract

Mobile infrared-visible imaging typically pairs a compact infrared sensor with a high-resolution visible camera for complementary perception. While cross-sensor misalignment caused by different optics, viewpoints, fields of view, and exposure timings hinders practical deployment. In this paper, we propose BeyondFusion, a unified latent diffusion framework for calibration-free visible-guided infrared super-resolution and infrared-visible fusion tasks. The proposed framework supports both task-specific training and joint training where two tasks are optimized and executed as two readouts of the same generative process. Instead of relying on explicit registration or geometric warping, BeyondFusion introduces a cross-modal self-aligning (CMSA) module into the denoising U-Net. CMSA reorganizes infrared and visible latent tokens into a shared attention space to learn content-adaptive cross-modal correspondence during the denoising process. Together with misalignment augmentation module, the model is facilitated to exploit visible structural and semantic cues while preserving thermal consistency, enabling high-frequency infrared reconstruction and informative fused-image generation under uncalibrated conditions. Extensive experiments on public benchmarks and a mobile infrared-visible imaging system show strong performance across aligned inputs, low-resolution infrared observations, synthetic misalignments, and real mobile captures with unsynchronized sensors. Ablation studies, unified training analysis, and downstream pedestrian detection further validate the effectiveness of BeyondFusion for calibration-free multimodal imaging.

Figures

Figures reproduced from arXiv: 2607.24110 by Jianing Zhang, Jun Zhang, Minchong Chen, Minyu Cao, Shuyang Liu, Xiaokang Yang, Xiaoyun Yuan.

Figure 1
Figure 1. Figure 1: Overview of the BeyondFusion architecture. BeyondFusion encodes visible and infrared inputs into a latent diffusion space, reshapes their latent [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Mobile infrared-visible imaging system. infrared-visible image fusion, whether the latent interaction remains robust to resolution gaps and uncalibrated inputs, and whether the fused outputs benefit downstream perception. A. Experimental Setup 1) Datasets and Preprocessing: For infrared super￾resolution, we use IRVI [51], LLVIP [52], M3FD [53], and the PBVS 2025 TISR Challenge Track 2 [54], yielding 10,922… view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative comparison for infrared image super-resolution on the test set. BeyondFusion reconstructs sharper structures and more faithful infrared [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative infrared super-resolution comparison on the real-world mobile dataset. BeyondFusion shows strong generalization to uncalibrated captures, [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative comparison under simulated unregistered inputs (zoom [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparison on our test set (zoom in for details). [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative infrared-visible image fusion comparison on our real [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Precision-recall curve comparison for pedestrian detection with IoU [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Visualization of comparison against CDDFuse and Mask-Difuser, [PITH_FULL_IMAGE:figures/full_fig_p011_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Visualization of comparison against C-OPDR, where green bounding [PITH_FULL_IMAGE:figures/full_fig_p011_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

63 extracted references · 6 linked inside Pith

  1. [1]

    Dcevo: Discriminative cross-dimensional evolutionary learning for infrared and visible image fusion,

    J. Liu, B. Zhang, Q. Mei, X. Li, Y . Zou, Z. Jiang, L. Ma, R. Liu, and X. Fan, “Dcevo: Discriminative cross-dimensional evolutionary learning for infrared and visible image fusion,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 2226–2235

    2025

  2. [2]

    Infrared and visible image fusion methods and applications: A survey,

    J. Ma, Y . Ma, and C. Li, “Infrared and visible image fusion methods and applications: A survey,”Information fusion, vol. 45, pp. 153–178, 2019

    2019

  3. [3]

    Divfusion: Darkness-free infrared and visible image fusion,

    L. Tang, X. Xiang, H. Zhang, M. Gong, and J. Ma, “Divfusion: Darkness-free infrared and visible image fusion,”Information Fusion, vol. 91, pp. 477–493, 2023

    2023

  4. [4]

    Deep depth estimation from thermal image,

    U. Shin, J. Park, and I. S. Kweon, “Deep depth estimation from thermal image,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 1043–1053

    2023

  5. [5]

    What happened 3 seconds ago? inferring the past with thermal imaging,

    Z. Tang, W. Ye, W.-C. Ma, and H. Zhao, “What happened 3 seconds ago? inferring the past with thermal imaging,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 17 111–17 120

    2023

  6. [6]

    Projecting trackable thermal patterns for dynamic computer vision,

    M. Sheinin, A. C. Sankaranarayanan, and S. G. Narasimhan, “Projecting trackable thermal patterns for dynamic computer vision,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, 2024, pp. 25 223–25 232

    2024

  7. [7]

    Deep learning for image super- resolution: A survey,

    Z. Wang, J. Chen, and S. C. Hoi, “Deep learning for image super- resolution: A survey,”IEEE transactions on pattern analysis and ma- chine intelligence, vol. 43, no. 10, pp. 3365–3387, 2020

    2020

  8. [8]

    Hyperspectral image super-resolution with optimized rgb guidance,

    Y . Fu, T. Zhang, Y . Zheng, D. Zhang, and H. Huang, “Hyperspectral image super-resolution with optimized rgb guidance,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 11 661–11 670

    2019

  9. [9]

    Corefusion: Contrastive regularized fusion for guided thermal super-resolution,

    A. Kasliwal, P. Seth, S. Rallabandi, and S. Singhal, “Corefusion: Contrastive regularized fusion for guided thermal super-resolution,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 507–514

    2023

  10. [10]

    Swinfusr: an image fusion-inspired model for rgb-guided thermal image super-resolution,

    C. Arnold, P. Jouvet, and L. Seoud, “Swinfusr: an image fusion-inspired model for rgb-guided thermal image super-resolution,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 3027–3036

    2024

  11. [11]

    Ifcnn: A general image fusion framework based on convolutional neural network,

    Y . Zhang, Y . Liu, P. Sun, H. Yan, X. Zhao, and L. Zhang, “Ifcnn: A general image fusion framework based on convolutional neural network,” Information Fusion, vol. 54, pp. 99–118, 2020

    2020

  12. [12]

    U2fusion: A unified unsupervised image fusion network,

    H. Xu, J. Ma, J. Jiang, X. Guo, and H. Ling, “U2fusion: A unified unsupervised image fusion network,”IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 1, pp. 502–518, 2020

    2020

  13. [13]

    Fusiongan: A generative adversarial network for infrared and visible image fusion,

    J. Ma, W. Yu, P. Liang, C. Li, and J. Jiang, “Fusiongan: A generative adversarial network for infrared and visible image fusion,”Information fusion, vol. 48, pp. 11–26, 2019

    2019

  14. [14]

    Swinfusion: Cross-domain long-range learning for general image fusion via swin transformer,

    J. Ma, L. Tang, F. Fan, J. Huang, X. Mei, and Y . Ma, “Swinfusion: Cross-domain long-range learning for general image fusion via swin transformer,”IEEE/CAA Journal of Automatica Sinica, vol. 9, no. 7, pp. 1200–1217, 2022

    2022

  15. [15]

    Cddfuse: Correlation-driven dual-branch feature decomposition for multi-modality image fusion,

    Z. Zhao, H. Bai, J. Zhang, Y . Zhang, S. Xu, Z. Lin, R. Timofte, and L. Van Gool, “Cddfuse: Correlation-driven dual-branch feature decomposition for multi-modality image fusion,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 5906–5916

    2023

  16. [16]

    Mask-difuser: A masked diffusion model for unified unsupervised image fusion,

    L. Tang, C. Li, and J. Ma, “Mask-difuser: A masked diffusion model for unified unsupervised image fusion,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    2025

  17. [17]

    Dif-fusion: Toward high color fidelity in infrared and visible image fusion with diffusion models,

    J. Yue, L. Fang, S. Xia, Y . Deng, and J. Ma, “Dif-fusion: Toward high color fidelity in infrared and visible image fusion with diffusion models,” IEEE Transactions on Image Processing, vol. 32, pp. 5705–5720, 2023

    2023

  18. [18]

    Towards unified semantic and controllable image fusion: A diffusion transformer approach,

    J. Li, C. Jiang, J. Jiang, P. Liang, J. Ma, and L. Nie, “Towards unified semantic and controllable image fusion: A diffusion transformer approach,”IEEE transactions on pattern analysis and machine intelli- gence, 2025

    2025

  19. [19]

    Mulfs-cap: Multimodal fusion-supervised cross-modality alignment perception for unregistered infrared-visible image fusion,

    H. Li, Z. Yang, Y . Zhang, W. Jia, Z. Yu, and Y . Liu, “Mulfs-cap: Multimodal fusion-supervised cross-modality alignment perception for unregistered infrared-visible image fusion,”IEEE Transactions on Pat- tern Analysis and Machine Intelligence, vol. 47, no. 5, pp. 3673–3690, 2025

    2025

  20. [20]

    Improving misaligned multi-modality image fusion with one-stage progressive dense registra- tion,

    D. Wang, J. Liu, L. Ma, R. Liu, and X. Fan, “Improving misaligned multi-modality image fusion with one-stage progressive dense registra- tion,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 34, no. 11, pp. 10 944–10 958, 2024

    2024

  21. [21]

    Unsupervised misaligned infrared and visible image fusion via cross-modality image generation and registration,

    D. Wang, J. Liu, X. Fan, and R. Liu, “Unsupervised misaligned infrared and visible image fusion via cross-modality image generation and registration,”arXiv preprint arXiv:2205.11876, 2022

    Pith/arXiv arXiv 2022

  22. [22]

    Adversarial diffusion distillation,

    A. Sauer, D. Lorenz, A. Blattmann, and R. Rombach, “Adversarial diffusion distillation,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 87–103

    2024

  23. [23]

    3m-ti: High-quality mobile thermal imaging via calibration-free multi-camera cross-modal diffusion,

    M. Chen, X. Yuan, J. Wan, J. Zhang, and J. Zhang, “3m-ti: High-quality mobile thermal imaging via calibration-free multi-camera cross-modal diffusion,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026, pp. 5659–5669

    2026

  24. [24]

    Crossnet: An end-to-end reference-based super resolution network using cross-scale warping,

    H. Zheng, M. Ji, H. Wang, Y . Liu, and L. Fang, “Crossnet: An end-to-end reference-based super resolution network using cross-scale warping,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 88–104

    2018

  25. [25]

    Crossnet++: Cross-scale large-parallax warping for reference-based super-resolution,

    Y . Tan, H. Zheng, Y . Zhu, X. Yuan, X. Lin, D. Brady, and L. Fang, “Crossnet++: Cross-scale large-parallax warping for reference-based super-resolution,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 12, pp. 4291–4305, 2020

    2020

  26. [26]

    Image super-resolution by neural texture transfer,

    Z. Zhang, Z. Wang, Z. Lin, and H. Qi, “Image super-resolution by neural texture transfer,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 7982–7991

    2019

  27. [27]

    Engram-driven videography,

    L. Fang, M. Ji, X. Yuan, J. He, J. Zhang, Y . Zhu, T. Zheng, L. Liu, B. Wang, and Q. Dai, “Engram-driven videography,”Engineering, vol. 25, pp. 101–109, 2023

    2023

  28. [28]

    Unaligned rgb guided hyperspectral image super-resolution with spatial-spectral concordance: Y . zhang et al

    Y . Zhang, Z. Lai, T. Zhang, Y . Fu, and C. Zhou, “Unaligned rgb guided hyperspectral image super-resolution with spatial-spectral concordance: Y . zhang et al.”International Journal of Computer Vision, pp. 1–21, 2025

    2025

  29. [29]

    Swinpaste: A swin transformer-based framework for rgb-guided thermal image super-resolution,

    H. Zhong, Y . Wang, and S. Zhao, “Swinpaste: A swin transformer-based framework for rgb-guided thermal image super-resolution,” inProceed- ings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 4589–4594

    2025

  30. [30]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,”Advances in neural information processing systems, vol. 30, 2017

    2017

  31. [31]

    Thermal uav image super-resolution guided by multiple visible cues,

    Z. Zhao, Y . Zhang, C. Li, Y . Xiao, and J. Tang, “Thermal uav image super-resolution guided by multiple visible cues,”IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp. 1–14, 2023

    2023

  32. [32]

    Multi-scale feature fusion using channel transformers for guided thermal image super resolution,

    R. S. Puttagunta, B. Kathariya, Z. Li, and G. York, “Multi-scale feature fusion using channel transformers for guided thermal image super resolution,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 3086–3095

    2024

  33. [33]

    Highlight what you want: Weakly-supervised instance-level controllable infrared- visible image fusion,

    Z. Wang, J. Zhang, H. Song, M. Ge, J. Wang, and H. Duan, “Highlight what you want: Weakly-supervised instance-level controllable infrared- visible image fusion,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2025, pp. 12 637–12 647

    2025

  34. [34]

    Cdtfusion: Crossing domain and task for infrared and visible image fusion,

    W. Zhao, W. Wang, H. Wang, Y . He, and H. Lu, “Cdtfusion: Crossing domain and task for infrared and visible image fusion,”IEEE Transac- tions on Pattern Analysis and Machine Intelligence, 2025

    2025

  35. [35]

    Freefusion: Infrared and visible image fusion via cross reconstruction learning,

    W. Zhao, H. Cui, H. Wang, Y . He, and H. Lu, “Freefusion: Infrared and visible image fusion via cross reconstruction learning,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    2025

  36. [36]

    A fine- grained semantic-driven pedestrian identity-aware method for visible and infrared image fusion,

    X. Liu, H. Huo, H. Cao, R. Wang, X. Dong, S. Pang, and J. Li, “A fine- grained semantic-driven pedestrian identity-aware method for visible and infrared image fusion,”Information Fusion, p. 104453, 2026

    2026

  37. [37]

    Mddpfuse: Multi-driven dynamic perception network for infrared and visible image fusion via data guidance and semantic injection,

    M. Wang, Y . Pan, Z. Zhao, Z. Li, and S. Yao, “Mddpfuse: Multi-driven dynamic perception network for infrared and visible image fusion via data guidance and semantic injection,”Knowledge-Based Systems, p. 114027, 2025

    2025

  38. [38]

    Spdfusion: A semantic prior knowledge-driven method for infrared and visible image fusion,

    Q. Xiao, H. Jin, H. Su, Y . Zhang, Z. Xiao, and B. Wang, “Spdfusion: A semantic prior knowledge-driven method for infrared and visible image fusion,”IEEE Transactions on Multimedia, vol. 27, pp. 1691–1705, 2024. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 13

    2024

  39. [39]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems, vol. 33, pp. 6840– 6851, 2020

    2020

  40. [40]

    Taming generative diffusion model for task-oriented infrared imaging,

    T. Ma, Z. Dai, Y . Diao, G. An, L. Ma, J. Liu, and R. Liu, “Taming generative diffusion model for task-oriented infrared imaging,” inPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026, pp. 30 843–30 853

    2026

  41. [41]

    Dreamsr: Towards ultra-high-resolution image super-resolution via a receptive- field enhanced diffusion transformer,

    Q. Dong, H. Dong, M. Chen, R. Zhang, and Y . Wang, “Dreamsr: Towards ultra-high-resolution image super-resolution via a receptive- field enhanced diffusion transformer,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026, pp. 38 258–38 269

    2026

  42. [42]

    Degradation-modeled multipath diffusion for tunable metalens photography,

    J. Zhang, J. Zhu, F. Ji, X. Yang, and X. Yuan, “Degradation-modeled multipath diffusion for tunable metalens photography,”arXiv preprint arXiv:2506.22753, 2025

    Pith/arXiv arXiv 2025

  43. [43]

    Seesr: Towards semantics-aware real-world image super-resolution,

    R. Wu, T. Yang, L. Sun, Z. Zhang, S. Li, and L. Zhang, “Seesr: Towards semantics-aware real-world image super-resolution,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 25 456–25 467

    2024

  44. [44]

    Sinsr: diffusion-based image super- resolution in a single step,

    Y . Wang, W. Yang, X. Chen, Y . Wang, L. Guo, L.-P. Chau, Z. Liu, Y . Qiao, A. C. Kot, and B. Wen, “Sinsr: diffusion-based image super- resolution in a single step,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 25 796–25 805

    2024

  45. [45]

    One-step effective diffusion network for real-world image super-resolution,

    R. Wu, L. Sun, Z. Ma, and L. Zhang, “One-step effective diffusion network for real-world image super-resolution,”Advances in Neural Information Processing Systems, vol. 37, pp. 92 529–92 553, 2024

    2024

  46. [46]

    Difiisr: A diffusion model with gradient guidance for infrared image super-resolution,

    X. Li, Z. Wang, Y . Zou, Z. Chen, J. Ma, Z. Jiang, L. Ma, and J. Liu, “Difiisr: A diffusion model with gradient guidance for infrared image super-resolution,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 7534–7544

    2025

  47. [47]

    Ds2d: Decoupling feature guidance with state-space diffusion for infrared- visible image fusion,

    Z. Wu, X. Liu, Z. Li, S. Wang, P. Fan, F. Wu, and G. Xiao, “Ds2d: Decoupling feature guidance with state-space diffusion for infrared- visible image fusion,”Pattern Recognition, p. 114011, 2026

    2026

  48. [48]

    Recofuse: Ultra-robust image fusion via restorative multi-modal diffusion reciprocal coupling,

    H. Zhang, S. Yang, L. Tang, X. Yi, and J. Ma, “Recofuse: Ultra-robust image fusion via restorative multi-modal diffusion reciprocal coupling,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026, pp. 33 838–33 847

    2026

  49. [49]

    Lora: Low-rank adaptation of large language models

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chenet al., “Lora: Low-rank adaptation of large language models.” ICLR, vol. 1, no. 2, p. 3, 2022

    2022

  50. [50]

    One- step image translation with text-to-image models,

    G. Parmar, T. Park, S. Narasimhan, and J.-Y . Zhu, “One- step image translation with text-to-image models,”arXiv preprint arXiv:2403.12036, 2024

    Pith/arXiv arXiv 2024

  51. [51]

    I2v-gan: Unpaired infrared-to-visible video translation,

    S. Li, B. Han, Z. Yu, C. H. Liu, K. Chen, and S. Wang, “I2v-gan: Unpaired infrared-to-visible video translation,” inProceedings of the 29th ACM international conference on multimedia, 2021, pp. 3061– 3069

    2021

  52. [52]

    Llvip: A visible-infrared paired dataset for low-light vision,

    X. Jia, C. Zhu, M. Li, W. Tang, and W. Zhou, “Llvip: A visible-infrared paired dataset for low-light vision,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 3496–3504

    2021

  53. [53]

    Target-aware dual adversarial learning and a multi-scenario multi- modality benchmark to fuse infrared and visible for object detection,

    J. Liu, X. Fan, Z. Huang, G. Wu, R. Liu, W. Zhong, and Z. Luo, “Target-aware dual adversarial learning and a multi-scenario multi- modality benchmark to fuse infrared and visible for object detection,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 5802–5811

    2022

  54. [54]

    Thermal image super-resolution challenge results-pbvs 2025,

    R. E. Rivadeneira, A. D. Sappa, R. Hammoud, J. Rao, H. Zhong, Y . Wang, S. Zhao, Z. Zhong, Y .-H. Li, S. Wang, Q. Shen, H. Wang, and X. Zhang, “Thermal image super-resolution challenge results-pbvs 2025,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2025, pp. 4630–4639

    2025

  55. [55]

    Msrs: Multi-spectral road scenarios for practical infrared and visible image fusion,

    L. Tang, J. Yuan, H. Zhang, X. Jiang, and J. Ma, “Msrs: Multi-spectral road scenarios for practical infrared and visible image fusion,” 2022

    2022

  56. [56]

    Videofusion: A spatio-temporal collaborative net- work for multi-modal video fusion and restoration,

    L. Tang, Y . Wang, M. Gong, Z. Li, Y . Deng, X. Yi, C. Li, H. Zhang, H. Xu, and J. Ma, “Videofusion: A spatio-temporal collaborative net- work for multi-modal video fusion and restoration,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026

    2026

  57. [57]

    Classifier-free diffusion guidance,

    J. Ho and T. Salimans, “Classifier-free diffusion guidance,”arXiv preprint arXiv:2207.12598, 2022

    Pith/arXiv arXiv 2022

  58. [58]

    Adam: A method for stochastic optimization,

    D. P. Kingma, “Adam: A method for stochastic optimization,”arXiv preprint arXiv:1412.6980, 2014

    Pith/arXiv arXiv 2014

  59. [59]

    Contourlet residual for prompt learning enhanced infrared image super-resolution,

    X. Li, J. Liu, Z. Chen, Y . Zou, L. Ma, X. Fan, and R. Liu, “Contourlet residual for prompt learning enhanced infrared image super-resolution,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 270– 288

    2024

  60. [60]

    Image quality assessment: from error visibility to structural similarity,

    Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,”IEEE transactions on image processing, vol. 13, no. 4, pp. 600–612, 2004

    2004

  61. [61]

    The unreasonable effectiveness of deep features as a perceptual metric,

    R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 586–595

    2018

  62. [62]

    Topiq: A top-down approach from semantics to distortions for image quality assessment,

    C. Chen, J. Mo, J. Hou, H. Wu, L. Liao, W. Sun, Q. Yan, and W. Lin, “Topiq: A top-down approach from semantics to distortions for image quality assessment,”IEEE Transactions on Image Processing, vol. 33, pp. 2404–2418, 2024

    2024

  63. [63]

    Grounded sam: Assembling open-world models for diverse visual tasks,

    T. Ren, S. Liu, A. Zeng, J. Lin, K. Li, H. Cao, J. Chen, X. Huang, Y . Chen, F. Yanet al., “Grounded sam: Assembling open-world models for diverse visual tasks,”arXiv preprint arXiv:2401.14159, 2024

    Pith/arXiv arXiv 2024