Pith. sign in

REVIEW 3 major objections 5 minor 66 references

Robust Self-Supervised Cross-Modal Super-Resolution against Real-World Misaligned Observations

T0 review · 3 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A self-supervised model jointly aligns and super-resolves misaligned RGB-depth and RGB-NIR pairs in a single online pass, with no training data or ground truth.

desk verdict A plausible self-supervised cross-modal SR method whose ×2 results are solid, but whose real-world ×4 SOTA claim rests on no-reference metrics and an unconstrained high-frequency band. read the letter →

arxiv 2602.18822 v3 pith:AMNR2XKT submitted 2026-02-21 cs.CV

Xiaoyu Dong , Jiahuan Li , Ziteng Cui , Naoto Yokoya This is my paper · ORCID
classification cs.CV
keywords self-supervisedsuper-resolutioncross-modalmisalignedmulti-modaldataonlineoptimizationfeaturetranslationreference-basedfilteringRGB-guideddepthRGB-NIR
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that cross-modal super-resolution on real-world data—where a low-resolution depth or near-infrared image is enhanced using a high-resolution RGB guide that is spatially misaligned—can be solved entirely online, with no training dataset, no ground-truth super-resolution, and no separate alignment step. It introduces RobSelf, which jointly optimizes a misalignment-aware feature translator and a content-aware reference filter for each test pair. The translator warps the guide feature and is weakly supervised by the low-resolution source, producing an aligned guide feature; the filter then uses that feature only as a reference to compute kernels that enhance the source's own pixels, so redundant guide content is not injected. On synthesized and newly collected real-world RGB-depth and RGB-NIR data with inherent sensor misalignment, viewpoint changes, and object motion, RobSelf reports lower error than supervised and prior self-supervised baselines, and runs up to 15.3× faster than prior self-supervised methods. The significance, if the claims hold, is that misaligned multi-sensor systems can be super-resolved without calibration or paired training data.

What carries the argument

The load-bearing mechanism is the joint weakly-supervised, misalignment-aware translation formulation. The translator is not trained to align the guide to the source directly (no alignment ground truth exists); instead, it is trained per test pair to produce an HR image whose downsampled version matches the LR source. This single regression loss forces the warped guide feature to be source-consistent while staying in HR space. The second component, a content-aware reference filter, uses a gradient-derived importance map to select large or small correlation-based kernels and applies them to the source's own pixels; the aligned guide feature serves only as the reference for kernel weights, whi

What would settle it

Run RobSelf-De on a real-world misaligned RGB-depth pair from the collected data at ×4, and capture a true high-resolution depth map of the same scene with the Azure Kinect's higher-resolution depth mode (or a second aligned depth sensor) to serve as ground truth. If the SR output's high-frequency structures that were added beyond the LR source do not match the true depth edges (measured, for example, by edge-location precision or RMSE computed only on high-gradient pixels), then the unconstrained high-frequency content is not source-consistent and the central fidelity claim collapses.

Watch

Extended reading notes

Core claim

RobSelf's central claim is that a single self-supervised architecture can jointly solve unsupervised cross-modal alignment and super-resolution in one online optimization, without pre-alignment or external training. A translator warps the HR guide feature under a weakly-supervised loss: the downsampled translation must match the LR source, yielding an aligned guide feature. A content-aware reference filter uses a gradient importance map to choose large or small correlation-based kernels, enhancing each source pixel from its own neighbors with the aligned guide pixel as reference. The paper reports lower RMSE/DSS/NIQE than supervised and prior self-supervised baselines on three tasks and show

Load-bearing premise

The load-bearing premise is that requiring the downsampled translation prediction to match the low-resolution source is enough to make the translator produce a guide feature that is source-consistent at high frequencies; because the loss only sees the downsampled output, any high-frequency content the translator invents is never checked, and the filter can inherit those errors.

Editorial extensions

If this is right

  • If RobSelf's reported results hold, cross-modal SR no longer requires per-domain training sets or ground-truth SR pairs, so a new modality pair can be super-resolved by online optimization alone.
  • Alignment and SR are optimized jointly in a single loss, so errors are not compounded across a two-stage pre-align-then-SR pipeline; the paper shows this is more robust to viewpoint variation and object motion.
  • The runtime speedup (up to 15.3× over P2P, at least 2.56× over MMSR and SSGNet on the NIR task) suggests self-supervised SR can move closer to practical use, although the reported per-pair times (64–122 s on an A100) are still far from real-time.
  • The translator's ability to synthesize guide structures that are missing due to occlusion or field-of-view differences implies the model can draw on context beyond the visible guide, which the filter can exploit for enhancement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The weak-supervision loss checks only downsampled consistency, so the translator's high-frequency output—explicitly described as redundant content discarded in downsampling—is never verified for fidelity. If that content is hallucinated, the filter's reference kernels could amplify false details; this is a testable risk in the ×4 real-world experiments, which lack ground truth.
  • The gradient-based importance mechanism is a general idea: any guided restoration task (pan-sharpening, medical fusion, guided depth completion) could use a content-aware reference filter to avoid injecting redundant guide content, provided an aligned reference feature can be obtained.
  • A natural extension would be to add a high-frequency consistency loss (e.g., patch-based or adversarial supervision at HR, or a cycle-consistency term on the translation) to constrain the unverified high-frequency content; the paper does not include such a term.
  • The reported speedup suggests that with a better initialization or fewer iterations, online self-supervised SR could approach interactive rates, but the paper does not investigate early stopping or initialization strategies.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes RobSelf, a self-supervised cross-modal super-resolution method for misaligned low-resolution source and high-resolution guide images. RobSelf jointly optimizes, online per test pair, a misalignment-aware feature translator (with deformable-conv and resampling variants) and a content-aware reference filter, using only a consistency loss (Eq. 1) between downsampled predictions and the LR source. The authors report state-of-the-art results on synthesized misaligned RGB-guided depth SR (×4/×8), real-world RGB-depth SR (×2/×4), and real-world RGB-NIR SR (×2/×4), and claim up to 15.3× faster runtime than prior self-supervised methods. The real-world ×4 evaluations use no-reference metrics (DSS, NIQE) because no ground truth exists.

Significance. If the claims hold, the contribution is significant: a single self-supervised framework that handles complex real-world misalignment without training data, ground truth, or pre-alignment would make cross-modal SR practical in many sensor-fusion settings. The paper includes strengths that should be acknowledged: it ships two well-specified variants, collects real-world RGB-depth and RGB-NIR datasets with multiple misalignment types, and shows large RMSE improvements over prior methods on synthesized and real-world ×2 tasks where ground truth exists. The runtime comparison is also favorable. However, the headline real-world ×4 claims rest on no-reference metrics and on a supervision loss that leaves the high-frequency band unconstrained, so the 'high fidelity' claim is not yet secured by the reported evidence.

major comments (3)
  1. [§3.1, Eq. (1); §5.5] The central self-supervised objective supervises both predictions only through f_down (average pooling), so all high-frequency content above the downsampling cutoff is unconstrained. Sec. 5.5 explicitly states that redundant content in F_aligned_guide 'is discarded in the downsampling before regression.' Since F_aligned_guide is used as the reference for the content-aware filter weights (Eqs. (5)–(6)), any high-frequency misalignment or hallucinated structure in the guide can directly bias the SR output. The real-world ×4 results in Tables 2 and 3 are evaluated with no-reference DSS/NIQE, which reward sharpness/edge statistics but do not verify fidelity to the true scene. The claim of 'high resolution and high fidelity' for ×4 is therefore not established. Please provide a fidelity check on ×4 (e.g., a held-out subset with ground truth, or a synthesized experiment where DSS/NIQE are comp
  2. [§5.1–5.4, Tables 1–3] The method has several task-dependent hyperparameters (estimator level i, threshold scaling η, kernel sizes {m,n}) that are re-tuned for each dataset and SR factor (i=3/4/5; {m,n}={7,5},{13,7},{3,3}). The manuscript says sensitivity analyses are in the supplementary material, but the main text reports no error bars, confidence intervals, or significance tests. Given that some margins are narrow (e.g., Table 3 ×2 RMSE of 3.09 vs 3.12; Table 2 ×2 RMSE of 2.23 vs 2.18), the SOTA claim is not robust as presented. Please report variance across test groups and a main-text summary of sensitivity to the key hyperparameters.
  3. [§5.3–5.4, Tables 2–3] The real-world datasets have no ×4 ground truth, so the ×4 comparisons rely entirely on no-reference DSS/NIQE values. These metrics are not designed to assess cross-modal fidelity and can reward hallucinated edges or textures. To support the claim that the large DSS/NIQE margins reflect faithful enhancement rather than artifacts, the paper should validate the metrics on a subset with known ground truth, or provide a user study or downstream-task evaluation. Please also report per-group variability, since the collected data include simple and complex cases with different misalignment types.
minor comments (5)
  1. [Author block] Affiliation contains a typo: 'Univsrsity of Tokyo' should be 'University of Tokyo'.
  2. [Eq. (1)] The notation L_sr and L_trans is not explicitly defined before the equation; please state that the first term corresponds to the SR prediction and the second to the translation prediction.
  3. [Tables 1–3] The footnote 'Params vary with SR factors (×4/×8) due to level i' is unclear. Please give the exact parameter counts or a table in the main text, rather than only in the supplementary.
  4. [§5.6, Fig. 11] The observation that the translator can 'synthesize' missing guide structures is intriguing but also highlights the hallucination risk discussed in the major comments. Please add a limitation discussion here or in the conclusions.
  5. [§5.5] The sentence about 'redundant content ... discarded in the downsampling before regression' is a load-bearing limitation. It should be presented as a limitation with an explicit discussion of its implications for high-frequency fidelity, not only as a design justification.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the self-supervised loss is an explicit online objective, the SR output is generated from source features with guide-derived kernel weights, and the central comparisons are against external baselines.

full rationale

RobSelf's derivation chain is not circular. The only training signal, Eq. (1), is an explicit self-supervised consistency objective: both I_SR_pred and I_Trans_pred are regressed to the LR source after average pooling. This is an optimization target, not a disguised reuse of the target output. The SR prediction is produced by a convex combination of source features (Eq. 6) with weights computed from correlations between source neighbors and the translated guide reference (Eq. 5); hence I_SR_pred is not set equal to the input, the guide, or the translation. The translator's F_aligned_guide is an intermediate feature rather than a fitted parameter renamed as a prediction, and the model's novelty (misalignment-aware translator + content-aware filter) is tested against external baselines (P2P, CMSR, DCTNet, SSGNet, SGNet, DORNet, etc.) with ground-truth RMSE on synthesized misaligned depth SR and on real-world x2 depth/NIR SR. The self-citation to MMSR [11] for the consistency loss and correlation-kernel weights is not load-bearing: the loss is written out fully and the same prior method appears as a baseline that RobSelf outperforms. The real-world x4 results are evaluated with no-reference DSS/NIQE rather than GT, and Sec. 5.5 admits that high-frequency 'redundant content' is 'discarded in the downsampling before regression'; this is an underdetermination/fidelity concern for the x4 claims, not an equation-level circularity. Therefore no step reduces to its own inputs by construction.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method introduces no new physical entities. Its main burden is the set of modeling assumptions behind the self-supervised loss: the low-res source is treated as the downsampled truth, the guide is assumed to share alignable structure with the source, and unconstrained high-frequency content in the aligned guide is assumed to be either useful or safely discarded. These are plausible but not independently verified, and the paper provides no external benchmark for the alignment itself.

free parameters (5)
  • Misalignment estimator level i = 4 (synth ×4), 5 (synth ×8), 3/4 (real depth ×2/×4), 4/5 (real NIR ×2/×4)
    Controls the number of downsampling stages in the deformation estimator; re-selected per task in Secs 5.2–5.4, affecting the model's ability to capture large vs local misalignments.
  • Filter kernel sizes {m, n} = {7,5} (depth ×2/×4), {13,7} (synth ×8), {3,3} (NIR)
    Chosen per task in Secs 5.2–5.4; larger kernels used at higher SR factors; no sensitivity analysis in the main text.
  • Threshold scaling factor η = 0.7 (depth tasks), none (NIR)
    Separates 'important' from 'less important' pixels in M_imp; hand-set per task, with analyses only in the supplementary.
  • Loss weight λ = 1
    Balances SR and translation losses in Eq. (1); set without reported tuning.
  • Online optimization iterations = 1000
    Fixed per-image-pair optimization following [11]; affects runtime and performance but is not reported as tuned.
assumptions (5)
  • domain assumption The LR source is exactly a downsampled version (average pooling) of the desired HR source, so self-supervision through f_down is valid.
    Used in Eq. (1) for both the SR and translation losses; standard SR degradation assumption but unverified on real data.
  • domain assumption The HR guide and LR source observe the same scene and share sufficient structure across modalities, so a dense deformation field plus translation can align them.
    Core premise of the translator; if the modalities are too disparate, the translated guide cannot provide reliable aligned references.
  • domain assumption Redundant or unalignable high-frequency guide content can be safely discarded by downsampling before regression, leaving enough source-consistent structure for SR.
    Explicitly stated in Sec. 5.5; this is the premise that lets the translator be trained with only the low-res consistency loss, but it also means high-frequency errors are unpenalized.
  • domain assumption The dense deformation field can be estimated from a single image pair without alignment ground truth via the translation objective.
    The translator has no alignment supervision; it relies entirely on the weakly supervised translation loss to infer displacement.
  • domain assumption No-reference metrics DSS and NIQE are adequate proxies for SR quality and fidelity when ground truth is absent.
    Used in Tables 2 and 3 for ×4 real-world evaluation; these metrics can reward sharp artifacts and cannot certify that added details are real.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Robust Self-Supervised Cross-Modal Super-Resolution against Real-World Misaligned Observations." pith.science (2026). https://pith.science/paper/AMNR2XKT

@misc{pith2026260218822,
  author       = {Pith},
  title        = {Pith review of: Robust Self-Supervised Cross-Modal Super-Resolution against Real-World Misaligned Observations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AMNR2XKT}},
  note         = {Machine review of arXiv:2602.18822}
}
abstract

Cross-modal super-resolution (SR) on real-world misaligned data is challenging, as only unlabeled low-resolution (LR) source and high-resolution (HR) guide images with complex spatial misalignment are available. Previous methods either rely on simulated training data or adopt suboptimal alignment strategies that overlook cross-modal dependencies, limiting their practical performance. To address these issues, we propose RobSelf, a self-supervised model that jointly optimizes a misalignment-aware feature translator and a content-aware reference filter online. The translator resolves unsupervised cross-modal and cross-resolution alignment via weakly-supervised, misalignment-aware translation, yielding an aligned guide feature. Guided by this feature, the filter performs reference-based discriminative self-enhancement on the source, enabling SR prediction with high resolution and high fidelity. Experiments on synthesized data and collected real-world data demonstrate that RobSelf achieves state-of-the-art performance, outperforming existing self-supervised and supervised methods. Moreover, it achieves superior efficiency, being up to 15.3$\times$ faster than prior self-supervised methods.

Figures

Figures reproduced from arXiv: 2602.18822 by the authors.

Figure 1
Figure 1. Real-world misaligned RGB-guided depth SR (×4). Our model achieves state￾of-the-art performance, requiring no training data, ground-truth supervision, or pre￾alignment. (a) LR source; (b) HR guide; (c) pre-aligned guide by MINIMA [36]; (d) SS￾GNet [40] + pre-alignment; (e) SGNet [48] + pre-alignment; (f) RobSelf-Re (Ours). arXiv:2602.18822v2 [cs.CV] 6 Mar 2026 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. RobSelf is supervised by the LR source. Within this framework, the transla￾tor is jointly optimized under weak supervision to map Fguide into I Trans pred that mimics the source, while yielding F Aligned guide . Guided by F Aligned guide , the filter performs reference￾based discriminative self-enhancement on Fsource to obtain F Enhanced source , from which I SR pred is generated. RobSelf has two variants depending … view at source ↗
Figure 3
Figure 3. Misalignment-aware feature translator. Each encoder layer downsamples by ×2; each decoder layer upsamples by ×2 [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Examples from our collected real-world misaligned data. LR sources are overlaid on HR guides for better visualization of misalignments [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Synthesized misaligned RGB-guided depth SR (×8). Image patches are shown due to space limit. *Pre-alignment applied. iterations following [11], without data augmentation or translator pre-training. We adopt the Adam optimizer [22] with an initial learning rate of 0.002…
Figure 7
Figure 7. Figure 7: Real-World misaligned RGB-guided depth SR (upper: ×2; lower: ×4). Image patches are shown due to space limit. *Pre-alignment applied. {7, 5}, respectively. We compare with four supervised methods (DORNet [46], DCNAS [66], SGNet [48], DCTNet [64]) and four self-supervis…
Figure 8
Figure 8. Figure 8: Real-world misaligned RGB-guided NIR SR (×2). Image patches are shown due to space limit. *Pre-alignment applied. ness and generalizability on real-world data with complex, large misalignments, owing to the joint weakly-supervised, misalignment-aware translation formul…
Figure 9
Figure 9. Figure 9: Effectiveness of our translator and filter [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Guide alignment performance: our translator vs. modules in [52]. Features by RobSelf-De for real-world misaligned RGB-guided depth (left) and NIR (right) SR. HR Guide LR Source Fguide Fsource F Aligned guide FEnhanced source ITrans pred I SR pred [PITH_FULL_IMAGE:fig…
Figure 11
Figure 11. Figure 11: Our translator can “synthesize” missing guide structures. See the right part of the square pot. Visualizations by RobSelf-De for real-world NIR SR (×2). and avoiding redundancy effects. Together, our techniques provide an effective inductive bias for self-supervised c…
Figure 12
Figure 12. Figure 12: Performance on well-aligned data: synthesized RGB-guided depth SR (×4) [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 2 linked inside Pith

  1. [1]

    arXiv preprint arXiv:1812.09276 (2018)

    Almasri, F., Debeir, O.: Multimodal sensor fusion in single thermal image super- resolution. arXiv preprint arXiv:1812.09276 (2018)

    arXiv 2018

  2. [2]

    In: CVPR (2020)

    Arar, M., Ginger, Y., Danon, D., Bermano, A.H., Cohen-Or, D.: Unsupervised multi-modal image registration via geometry preserving image-to-image transla- tion. In: CVPR (2020)

    2020

  3. [3]

    In: CVPR (2022)

    Bhatnagar, B.L., Xie, X., Petrov, I., Sminchisescu, C., Theobalt, C., Pons-Moll, G.: Behave: Dataset and method for tracking human object interactions. In: CVPR (2022)

    2022

  4. [4]

    In: CVPR (2011)

    Brown, M., Süsstrunk, S.: Multi-spectral SIFT for scene category recognition. In: CVPR (2011)

    2011

  5. [5]

    In: CVPR (2023)

    Cao, S.Y., Zhang, R., Luo, L., Yu, B., Sheng, Z., Li, J., Shen, H.L.: Recurrent homography estimation using homography-guided image warping and focus trans- former. In: CVPR (2023)

    2023

  6. [6]

    In: ICCV (2017)

    Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., Wei, Y.: Deformable convo- lutional networks. In: ICCV (2017)

    2017

  7. [7]

    IEEE TPAMI (2024)

    Deng, X., Liu, E., Gao, C., Li, S., Gu, S., Xu, M.: CrossHomo: Cross-modality and cross-resolution homography estimation. IEEE TPAMI (2024)

    2024

  8. [8]

    In: ICCV (2025)

    Do, J., Kim, S., Youk, G., Lee, J., Kim, M.: Pan-crafter: Learning modality- consistent alignment for PAN-sharpening. In: ICCV (2025)

    2025

Show all 66 references
  1. [9]

    In: ICCV (2023)

    Dong, J., Pan, J., Yang, Z., Tang, J.: Multi-scale residual low-pass filter network for image deblurring. In: ICCV (2023)

    2023

  2. [10]

    In: AAAI (2024)

    Dong, W., Xu, Y., Qu, J., Hou, S.: Learning multi-modal cross-scale deformable transformer network for unregistered hyperspectral image super-resolution. In: AAAI (2024)

    2024

  3. [11]

    In: ECCV (2022)

    Dong, X., Yokoya, N., Wang, L., Uezato, T.: Learning mutual modulation for self- supervised cross-modal super-resolution. In: ECCV (2022)

    2022

  4. [12]

    In: CVPR (2024)

    Duan, Y., Wu, X., Deng, H., Deng, L.J.: Content-adaptive non-local convolution for remote sensing pansharpening. In: CVPR (2024)

    2024

  5. [13]

    Prentice- Hall, Inc

    Gonzalez, R.C., Woods, R.E.: Digital Image Processing (3rd Edition). Prentice- Hall, Inc. (2006)

    2006

  6. [14]

    In: ECCV (2024)

    Guo,M.:Unsupervisedmulti-modalmedicalimageregistrationviainvertibletrans- lation. In: ECCV (2024)

    2024

  7. [15]

    IEEE TIP (2021)

    Gupta, H., Mitra, K.: Toward unaligned guided thermal super-resolution. IEEE TIP (2021)

    2021

  8. [16]

    In: CVPR (2021)

    He, L., Zhu, H., Li, F., Bai, H., Cong, R., Zhang, C., Lin, C., Liu, M., Zhao, Y.: Towards fast and accurate real-world depth super-resolution: Benchmark dataset and baseline. In: CVPR (2021)

    2021

  9. [17]

    In: CVPR (2007)

    Hirschmuller, H., Scharstein, D.: Evaluation of cost functions for stereo matching. In: CVPR (2007)

    2007

  10. [18]

    IEEE SPL (2020)

    Huang, Y., Li, L., Zhu, H., Hu, B.: Blind quality index of depth images based on structural statistics for view synthesis. IEEE SPL (2020)

    2020

  11. [19]

    In: AAAI (2025)

    Kang, J.,Cai, Q.,Tan,R.,Liu,Y.,Liu,Z.:C2PD: Continuity-constrained pixelwise deformation for guided depth super-resolution. In: AAAI (2025)

    2025

  12. [20]

    IJCV (2021)

    Kim, B., Ponce, J., Ham, B.: Deformable kernel networks for joint image filtering. IJCV (2021)

    2021

  13. [21]

    In: CVPR (2025)

    Kim, J., Baek, S.H.: Pixel-aligned RGB-NIR stereo imaging and dataset for robot vision. In: CVPR (2025)

    2025

  14. [22]

    In: ICLR (2015) 16 X

    Kingma, D., Ba, J.: Adam: A method for stochastic optimization. In: ICLR (2015) 16 X. Dong et al

    2015

  15. [23]

    In: ECCV (2012)

    Kohli, P., Silberman, N., Hoiem, D., Fergus, R.: Indoor segmentation and support inference from RGBD images. In: ECCV (2012)

    2012

  16. [24]

    In: CVPR (2023)

    Kong, L., Qi, X.S., Shen, Q., Wang, J., Zhang, J., Hu, Y., Zhou, Q.: Indescribable multi-modal spatial evaluator. In: CVPR (2023)

    2023

  17. [25]

    IEEE TNNLS (2024)

    Lai, Z., Fu, Y., Zhang, J.: Hyperspectral image super resolution with real unaligned RGB guidance. IEEE TNNLS (2024)

    2024

  18. [26]

    IEEE TPAMI (2025)

    Li, H., Yang, Z., Zhang, Y., Jia, W., Yu, Z., Liu, Y.: MulFS-CAP: Multimodal fusion-supervised cross-modality alignment perception for unregistered infrared- visible image fusion. IEEE TPAMI (2025)

    2025

  19. [27]

    In: CVPR (2025)

    Li, R., Okunev, M., Guo, Z., Duong, A.H., Richardt, C., O’Toole, M., Tompkin, J.: Time of the flight of the gaussians: Optimizing depth indirectly in dynamic radiance fields. In: CVPR (2025)

    2025

  20. [28]

    In: CVPR (2025)

    Lincetto, F., Agresti, G., Rossi, M., Zanuttigh, P.: Multimodalstudio: A heteroge- neous sensor dataset and framework for neural rendering across multiple imaging modalities. In: CVPR (2025)

    2025

  21. [29]

    In: NeurIPS (2018)

    Liu, D., Wen, B., Fan, Y., Loy, C.C., Huang, T.S.: Non-local recurrent network for image restoration. In: NeurIPS (2018)

    2018

  22. [30]

    In: ICCV (2019)

    Lutio, R.d., D’Aronco, S., Wegner, J.D., Schindler, K.: Guided super-resolution as pixel-to-pixel transformation. In: ICCV (2019)

    2019

  23. [31]

    In: CVPR (2021)

    Mei, Y., Fan, Y., Zhou, Y.: Image super-resolution with non-local sparse attention. In: CVPR (2021)

    2021

  24. [32]

    In: CVPR (2023)

    Metzger, N., Daudt, R.C., Schindler, K.: Guided depth super-resolution by deep anisotropic diffusion. In: CVPR (2023)

    2023

  25. [33]

    In: CVPR (2024)

    Mok, T.C.W., Li, Z., Bai, Y., Zhang, J., Liu, W., Zhou, Y.J., Yan, K., Jin, D., Shi, Y., Yin, X., Lu, L., Zhang, L.: Modality-agnostic structural image representa- tion learning for deformable multi-modality medical image registration. In: CVPR (2024)

    2024

  26. [34]

    In: ICCV (2019)

    Qiu, D., Pang, J., Sun, W., Yang, C.: Deep end-to-end alignment and refinement for time-of-flight RGB-D module. In: ICCV (2019)

    2019

  27. [35]

    IEEE TIP (2025)

    Qu, J., Wu, X., Dong, W., Cui, J., Li, Y.: IR&ArF: Toward deep interpretable arbitrary resolution fusion of unregistered hyperspectral and multispectral images. IEEE TIP (2025)

    2025

  28. [36]

    In: CVPR (2025)

    Ren,J.,Jiang,X.,Li,Z.,Liang,D.,Zhou,X.,Bai,X.:MINIMA:Modalityinvariant image matching. In: CVPR (2025)

    2025

  29. [37]

    In: CVPR (2007)

    Scharstein, D., Pal, C.: Learning conditional random fields for stereo. In: CVPR (2007)

    2007

  30. [38]

    In: CVPR (2021)

    Shacht, G., Danon, D., Fogel, S., Cohen-Or, D.: Single pair cross-modality super resolution. In: CVPR (2021)

    2021

  31. [39]

    IEEE TGRS (2023)

    Shi,L.,Zhao,R.,Pan,B.,Zou,Z.,Shi,Z.:Unsupervisedmultimodalremotesensing image registration via domain adaptation. IEEE TGRS (2023)

    2023

  32. [40]

    In: AAAI (2023)

    Shin, J., Shin, S., Jeon, H.G.: Task-specific scene structure representations. In: AAAI (2023)

    2023

  33. [41]

    In: NeurIPS (2020)

    Song, L., Li, Y., Jiang, Z., Li, Z., Zhang, X., Sun, H., Sun, J., Zheng, N.: Rethinking learnable tree filter for generic feature transform. In: NeurIPS (2020)

    2020

  34. [42]

    In: NeurIPS (2019)

    Song, L., Li, Y., Li, Z., Yu, G., Sun, H., Sun, J., Zheng, N.: Learnable tree filter for structure-preserving feature transform. In: NeurIPS (2019)

    2019

  35. [43]

    In: NeurIPS (2024)

    Song, S., Lew, J., Jang, H., Yoon, S.: Unsupervised homography estimation on multimodal image pair via alternating optimization. In: NeurIPS (2024)

    2024

  36. [44]

    In: CVPR (2022)

    Veksler, O., Boykov, Y.: Sparse non-local CRF. In: CVPR (2022)

    2022

  37. [45]

    In: CVPR (2018) RobSelf 17

    Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: CVPR (2018) RobSelf 17

    2018

  38. [46]

    Wang, Z., Yan, Z., Pan, J., Gao, G., Zhang, K., Yang, J.: DORNet: A degradation orientedandregularizednetworkforblinddepthsuper-resolution.In:CVPR(2025)

    2025

  39. [47]

    Wang, Z., Yan, Z., Wu, Y., Gao, G., Li, X., Yango, J.: Multi-order matching networkforalignment-freedepthsuper-resolution.arXivpreprintarXiv:2511.16361 (2026)

    2026 arXiv

  40. [48]

    In: AAAI (2024)

    Wang, Z., Yan, Z., Yang, J.: SGNet: Structure guided network via gradient- frequency awareness for depth map super-resolution. In: AAAI (2024)

    2024

  41. [49]

    In: CVPR (2018)

    Wu, H., Zheng, S., Zhang, J., Huang, K.: Fast end-to-end trainable guided filter. In: CVPR (2018)

    2018

  42. [50]

    In: WACV (2025)

    Xia, Z.X., Fadadu, S., Shi, Y., Foucard, L.: Robust long-range perception against sensor misalignment in autonomous vehicles. In: WACV (2025)

    2025

  43. [51]

    In: ICCV (2025)

    Xiang, J., Zhu, X., Wang, X., Wang, Y., Zhang, H., Guo, F., Yang, X.: DEPTHOR: Depth enhancement from a practical light-weight dToF sensor and RGB image. In: ICCV (2025)

    2025

  44. [52]

    IEEE TPAMI (2023)

    Xu, H., Yuan, J., Ma, J.: MURF: Mutually reinforcing multi-modal image regis- tration and fusion. IEEE TPAMI (2023)

    2023

  45. [53]

    IEEE TMI (2022)

    Xuan, K., Xiang, L., Huang, X., Zhang, L., Liao, S., Shen, D., Wang, Q.: Multi- modal MRI reconstruction assisted with spatial alignment network. IEEE TMI (2022)

    2022

  46. [54]

    In: ICCV (2025)

    Yan, Z., Wang, Z., Dong, H., Li, J., Yang, J., Lee, G.H.: Ducos: Duality constrained depth super-resolution via foundation model. In: ICCV (2025)

    2025

  47. [55]

    In: NeurIPS (2025)

    Yin,B.W.,Cao,J.L.,Zhang,X.,Chen,Y.,Cheng,M.M.,Hou,Q.:OmniSegmentor: A flexible multi-modal learning framework for semantic segmentation. In: NeurIPS (2025)

    2025

  48. [56]

    In: AAAI (2023)

    Yuan, J., Jiang, H., Li, X., Qian, J., Li, J., Yang, J.: Structure flow-guided network for real depth super-resolution. In: AAAI (2023)

    2023

  49. [57]

    In: ICML (2024)

    Zhang, K., Ma, J.: Sparse-to-dense multimodal image registration via multi-task learning. In: ICML (2024)

    2024

  50. [58]

    IEEE TIP (2015)

    Zhang, L., Zhang, L., Bovik, A.C.: A feature-enriched completely blind image qual- ity evaluator. IEEE TIP (2015)

    2015

  51. [59]

    In: ECCV (2024)

    Zhang, R., Ma, J., Cao, S.Y., Luo, L., Yu, B., Chen, S.J., Li, J., Shen, H.L.: SCPNet: Unsupervised cross-modal homography estimation via intra-modal self- supervised learning. In: ECCV (2024)

    2024

  52. [60]

    Information Fusion (2025)

    Zhang, S., Dong, J., Ma, Y., Cai, H., Wang, M., Li, Y., Kabika, T.B., Li, X., Hou, W.: CDF-DSR: Learning continuous depth field for self-supervised RGB-guided depth map super resolution. Information Fusion (2025)

    2025

  53. [61]

    IJCV (2025)

    Zhang, Y., Lai, Z., Zhang, T., Fu, Y., Zhou, C.: Unaligned RGB guided hyperspec- tral image super-resolution with spatial-spectral concordance. IJCV (2025)

    2025

  54. [62]

    In: CVPR (2021)

    Zhao, Y., Huang, X., Zhang, Z.: Deep lucas-kanade homography for multimodal image alignment. In: CVPR (2021)

    2021

  55. [63]

    In: ICCV (2023)

    Zhao, Z., Zhang, J., Gu, X., Tan, C., Xu, S., Zhang, Y., Timofte, R., Van Gool, L.: Spherical space feature decomposition for guided depth map super-resolution. In: ICCV (2023)

    2023

  56. [64]

    In: CVPR (2022)

    Zhao, Z., Zhang, J., Xu, S., Lin, Z., Pfister, H.: Discrete cosine transform network for guided depth map super-resolution. In: CVPR (2022)

    2022

  57. [65]

    In: CVPR (2020)

    Zheng, B., Yuan, S., Slabaugh, G., Leonardis, A.: Image demoireing with learnable bandpass filters. In: CVPR (2020)

    2020

  58. [66]

    IEEE TPAMI (2025)

    Zhong, Z., Liu, X., Jiang, J., Zhao, D., Wang, S.: Dual-level cross-modality neural architecture search for guided image super-resolution. IEEE TPAMI (2025)

    2025

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.