Pith. sign in

REVIEW 3 major objections 7 minor 66 references

Diff$^2$I2P: Differentiable Image-to-Point Cloud Registration with Diffusion Prior

T0 review · 3 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A diffusion prior distilled into cross-modal features lifts image-to-point-cloud registration recall on 7-Scenes from 75.8% to 83.0%.

desk verdict A clever training-time diffusion distillation for I2P registration, but inconsistent metric thresholds between text and supplement must be fixed before the headline numbers are trusted. read the letter →

arxiv 2507.06651 v1 pith:MHQUVFQD submitted 2025-07-09 cs.CV

Juncheng Mu , Chengwei Ren , Weixiang Zhang , Liang Pan , Xiao-Ping Zhang , Yue Gao This is my paper
classification cs.CV
keywords image-to-pointcloudregistrationcross-modalcorrespondencescoredistillationsamplingdiffusionpriordeformabletuningdifferentiablePnPmodalitygap
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the persistent accuracy gap in image-to-point-cloud registration comes from the modality gap between 2D texture and 3D geometry, and that a pretrained depth-conditioned diffusion model can supply a training signal that closes it. To use that signal, the authors make the whole registration pipeline differentiable end to end: a Deformable Correspondence Tuning module estimates point offsets so gradients can pass through correspondence formation, and a differentiable PnP solver carries them to the transformation. A new Control-Side Score Distillation loss then scores the predicted transformation by how well the projected depth map conditions the pretrained diffusion model, so badly aligned pairs pay a high loss. Trained this way, the feature backbones learn cross-modal features that match more reliably, and at inference the diffusion model is discarded, so runtime and memory stay at the level of the non-diffusion baseline.

What carries the argument

The load-bearing pieces are two. Control-Side Score Distillation (CSD) is a score-distillation objective that evaluates a predicted transformation by feeding the projected, densified depth map as the control condition to a pretrained depth-conditioned latent diffusion model; the residual between predicted and sampled noise, masked to the overlapping region, is backpropagated into the registration network with the size-matched Jacobian treated as the identity. Deformable Correspondence Tuning (DCT) is the differentiable bridge around non-differentiable top-k matching: a shared MLP takes the concatenated point and pixel features plus their coordinates and predicts 3D offsets, and the adjusted correspondences are passed to BPnP, a differentiable PnP solver whose gradients come from the implicit function theorem. Together they make the whole training loop differentiable and give the diffusion prior a channel through which to guide feature learning.

What would settle it

Train Diff2I2P identically but replace the CSD noise residual with a random target, for example the score model's output for an unrelated image-depth pair; if registration recall stays near 83.0% the diffusion prior is not the source of the gain, whereas if it collapses the CSD signal is carrying the improvement.

Watch

Extended reading notes

Core claim

The central claim is that score distillation from a depth-conditioned diffusion model can be turned into a direct, fully differentiable supervision for image-to-point-cloud registration. The paper introduces Control-Side Score Distillation (CSD), which treats a well-aligned image-depth pair as one that the pretrained ControlNet can denoise cleanly: the predicted transform projects the point cloud to a depth map, the depth map conditions the diffusion model on the input image, and the noise-prediction residual becomes a loss on the transformation. Because the top-k correspondence step and PnP-RANSAC would block gradients, the paper adds a Deformable Correspondence Tuning (DCT) module that predicts 3D point offsets from concatenated image-point features, and replaces RANSAC with the differentiable BPnP solver. With this path, the CSD gradient flows back into the feature backbones, so the diffusion prior is distilled into the cross-modal features themselves rather than used at inference. The paper reports that this formulation beats 2D3D-MATR on 7-Scenes by 7.2 percentage points in registration recall and sets a new high on RGB-D Scenes V2, while discarding the diffusion model at test time.

Load-bearing premise

Two premises share the load: that treating the size-matched Jacobian in the CSD gradient as the identity still yields a useful optimization direction, and that gradients flowing through the DCT offsets and BPnP are sufficient to update the feature backbones.

Editorial extensions

If this is right

  • If CSD is doing the work claimed, a pretrained cross-modal diffusion model can supervise registration without a ground-truth transformation loss, avoiding the gradient explosions the paper says a direct MSE on transformations produces.
  • The diffusion model is only a training-time teacher: at inference the same backbones run with speed and memory equal to the 2D3D-MATR baseline, so the method inherits accuracy gains without paying diffusion inference cost.
  • The DCT module gives a template for making other non-differentiable matching stages trainable: predict coordinate offsets with an MLP and let a differentiable solver carry the gradient.
  • Better cross-modal features should transfer across scenes because the network is scene-agnostic; the KITTI results in the supplementary material suggest the gain is not confined to indoor RGB-D benchmarks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the CSD gradient is an approximation with a dropped Jacobian, so its success may depend on the pretrained diffusion model being a good scorer of alignment; a weaker or domain-mismatched diffusion prior could erode the gain.
  • Editorial extension: because CSD supervises the transformation rather than the correspondences directly, it should in principle reward any global alignment that makes the depth conditioning coherent, which could help in low-overlap or repetitive-texture scenes where pointwise matching is ambiguous.
  • Editorial extension: a testable extension would be to swap the depth-conditioned diffusion model for a different conditional generator (for example, a semantic or surface-normal condition) and see whether registration recall tracks conditioning quality; if it does, CSD is a general distillation mechanism, not a depth-specific trick.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes Diff2I2P, an image-to-point cloud registration method that combines a coarse-to-fine matching pipeline (based on 2D3D-MATR) with a Deformable Correspondence Tuning (DCT) module and a differentiable BPnP solver, and trains the network with a Control-Side Score Distillation (CSD) loss derived from a pretrained depth-conditioned diffusion model. The core claims are that CSD provides a training signal bridging the image/point modality gap, that the whole pipeline is fully differentiable, and that the method outperforms state-of-the-art I2P registration baselines, in particular raising registration recall on 7-Scenes from 75.8% (2D3D-MATR) to 83.0% while keeping inference time comparable. The paper includes experiments on 7-Scenes, RGB-D Scenes V2, and KITTI, plus ablations and comparisons with FreeReg.

Significance. Should the empirical results hold, the paper makes a useful practical contribution: it shows how a large pretrained diffusion model can be used as a training-time supervisor for cross-modal feature learning, with no diffusion-model overhead at inference (0.074 s vs 0.072 s for 2D3D-MATR in Table 4). The reported 7.2-point improvement in registration recall on 7-Scenes is substantial, and the evaluations on multiple benchmarks, the KITTI extension, and the noise-robustness experiments in the supplementary material strengthen the empirical case. The method is clearly described and the architecture figures are helpful. However, the evaluation-protocol inconsistency and the overclaim of full differentiability currently limit confidence; the paper's central empirical claim is plausible but not yet fully verified.

major comments (3)
  1. [Section 4.1 and Supplementary Section A, Eqs. (11)-(12)] The same metrics are defined with conflicting thresholds in the main text and the supplementary. The main text states that Inlier Ratio (IR) counts correspondences within 5 cm and that Feature Matching Recall (FMR) uses an inlier-ratio threshold of 10%, while the supplementary defines tau1 = 10 cm and tau2 = 5%. Because IR and FMR are reported in Tables 1, 2, and 3 and in the supplementary comparisons, and because no evaluation code is released, a reader cannot determine which protocol generated the tables. If the two threshold sets were applied inconsistently, the claimed consistent improvements in IR/FMR could be artifacts of metric mismatch. The headline RR threshold (10 cm) is consistent, so the 7.2-point RR gain is not directly invalidated, but the supporting evidence must be pinned down before the empirical claims can be accepted.
  2. [Section 3.2 and Abstract] The paper advertises a 'fully differentiable' framework in the title and abstract, but Section 3.2 describes obtaining initial correspondences by 'selecting the point-pixel pairs with topk similarity.' The top-k selection is a hard, non-differentiable operation; gradients to the feature backbones flow only through the DCT offset prediction and the BPnP solver, not through the correspondence selection itself. The mapping from backbone parameters to the loss is therefore piecewise differentiable at best. Please replace 'fully differentiable' with a precise description such as 'end-to-end trainable via a differentiable correspondence refinement path,' and state explicitly whether any gradient signal reaches the matching scores used for the top-k selection.
  3. [Section 4.4, Table 3 (c) vs (e)] The CSD loss contributes only +1.7 RR (81.3 to 83.0), +0.2 IR, and +0.5 FMR over the no-CSD configuration, while the paper credits the diffusion prior with bridging the modality gap. Because the DCT module cannot be ablated during training, the 7.2-point RR gain over 2D3D-MATR is not isolated to the diffusion prior; it may be largely due to DCT and differentiable end-to-end training. Please provide an ablation that separates the effect of the diffusion prior, for example by replacing CSD with a stabilized geometric loss, and report variance or multiple seeds so that the +1.7 RR is shown to be significant.
minor comments (7)
  1. [Abstract and Section 4.2] The phrase 'consistently outperforms' is too strong in view of Table 1: on Stairs the proposed method has FMR 55.4 vs 58.1 for 2D3D-MATR, and IR ties at 18.1. Please qualify the claim, for example as 'outperforms on the mean and on most scenes.'
  2. [Various] There are several typographical errors: 'takle' in Section 3.1, 'sorely' in Section 1, 'leanrning' in Section 3.4, and 'RreeReg' in Supplementary Section C.3.
  3. [Table 3] The checkmark notation in Table 3 is ambiguous without a caption explaining each column and row; please add explicit column labels and row descriptions so that the reader can map each row to the loss configuration described in the text.
  4. [Equation (9)] Equation (9) uses Delta P_i for the offset while the text and Equation (2) use Delta p_i; please unify the notation.
  5. [Equation (8)] Equation (8) introduces x in the gradient expression without defining it; it should be the image latent z used in Equation (5).
  6. [Supplementary Section A] The text around Equation (11) swaps the roles of I and P, referring to 'the 3D point set I and the image pixel set P,' which conflicts with the main text notation where I is the image and P is the point cloud; please fix.
  7. [Equation (7)] The omission of the size-matched Jacobian is justified only by the statement that 'our implementation finds that this omission leads to an effective optimization.' Since this approximation is central to the CSD gradient, a small quantitative comparison or convergence analysis would strengthen the derivation.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the CSD prior is external, the losses are supervised by public ground truth, and the flagged IR/FMR threshold conflict is a reporting issue rather than a circular step.

full rationale

The paper's central training signal, the CSD gradient in Eq. (8), is computed from the noise residual of a pretrained depth-conditioned diffusion model (ControlNet/Stable Diffusion), whose weights are external to this paper; the gradient is not defined in terms of the paper's own outputs, and no fitted parameter of the method is embedded in the diffusion prior. The metric learning loss Lm follows the published 2D3D-MATR formulation with external ground-truth correspondences, and the offset loss Lo in Eq. (9) is supervised by the ground-truth transformation; neither reduces to the reported registration recall numbers by construction. The registration metrics (RR, IR, FMR) are evaluated against public benchmarks (7-Scenes, RGB-D Scenes V2, KITTI) with published baselines, so the headline 83.0% versus 75.8% RR gain is an externally comparable empirical claim. The only self-referential elements are ablations comparing loss configurations, which is standard practice and not circular. I do flag, for correctness rather than circularity, that Section 4.1 defines IR with a 5 cm threshold and FMR with a 10% inlier-ratio threshold, while supplementary Eqs. (11) and (12) define tau1 = 10 cm and tau2 = 5%; this inconsistency must be resolved before the reported IR/FMR numbers can be reproduced, but it does not make the derivation circular. No load-bearing self-citation or ansatz-smuggling-via-citation was found.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several hand-chosen hyperparameters (loss weights, densification kernels, topk count) and on two unvalidated modeling assumptions: the identity approximation of the CSD Jacobian and the sufficiency of DCT-only gradients. No new physical entities are introduced.

free parameters (5)
  • loss weights alpha, beta, gamma
    Combination weights for metric loss Lm, offset loss Lo, and CSD loss LCSD in Sec. 3.4; chosen by hand, not swept or justified.
  • offset regularization coefficient mu
    Regularization strength in Eq. (10), hand-chosen.
  • depth densification morphology parameters = kernel 7x7, thresholds 0.1 and 15.0, blur kernels 5
    Dilation, erosion, and blur sizes in Supplementary D.2, chosen to make sparse depth usable by ControlNet; these affect the CSD loss landscape.
  • topk correspondence count
    Number of initial point-pixel pairs selected in Section 3.2; threshold not given.
  • CSD timestep distribution
    Timestep t is sampled according to SDS, but schedule and weighting function w(t) are not specified for this application.
assumptions (4)
  • ad hoc to paper Omitted size-matched Jacobian can be treated as identity without corrupting optimization (Eq. 7).
    The paper states 'Our implementation finds that this omission leads to an effective optimization', an empirical assertion without theoretical support.
  • domain assumption Pretrained depth-conditioned ControlNet generalizes to synthetic densified depth maps produced by morphology operations.
    The CSD loss assumes the diffusion model's noise prediction responds sensibly to the projected and densified depth D from Eq. (4), which may differ from its training distribution.
  • ad hoc to paper Top-k correspondence selection can be left non-differentiable while DCT offsets still provide sufficient gradient signal to backbones.
    The 'fully differentiable' pipeline (Sec. 3.2) relies on this, but no gradient flows through the selection step itself.
  • domain assumption Overlap mask m excludes empty regions without introducing bias into the distillation loss.
    Eq. (5) masks out empty depth regions; this assumes those regions carry no useful alignment information.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Diff$^2$I2P: Differentiable Image-to-Point Cloud Registration with Diffusion Prior." pith.science (2026). https://pith.science/paper/MHQUVFQD

@misc{pith2026250706651,
  author       = {Pith},
  title        = {Pith review of: Diff$^2$I2P: Differentiable Image-to-Point Cloud Registration with Diffusion Prior},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MHQUVFQD}},
  note         = {Machine review of arXiv:2507.06651}
}
abstract

Learning cross-modal correspondences is essential for image-to-point cloud (I2P) registration. Existing methods achieve this mostly by utilizing metric learning to enforce feature alignment across modalities, disregarding the inherent modality gap between image and point data. Consequently, this paradigm struggles to ensure accurate cross-modal correspondences. To this end, inspired by the cross-modal generation success of recent large diffusion models, we propose Diff$^2$I2P, a fully Differentiable I2P registration framework, leveraging a novel and effective Diffusion prior for bridging the modality gap. Specifically, we propose a Control-Side Score Distillation (CSD) technique to distill knowledge from a depth-conditioned diffusion model to directly optimize the predicted transformation. However, the gradients on the transformation fail to backpropagate onto the cross-modal features due to the non-differentiability of correspondence retrieval and PnP solver. To this end, we further propose a Deformable Correspondence Tuning (DCT) module to estimate the correspondences in a differentiable way, followed by the transformation estimation using a differentiable PnP solver. With these two designs, the Diffusion model serves as a strong prior to guide the cross-modal feature learning of image and point cloud for forming robust correspondences, which significantly improves the registration. Extensive experimental results demonstrate that Diff$^2$I2P consistently outperforms SoTA I2P registration methods, achieving over 7% improvement in registration recall on the 7-Scenes benchmark.

Figures

Figures reproduced from arXiv: 2507.06651 by the authors.

Figure 1
Figure 1. Our proposed Control-Side Score Distillation (CSD) effectively promotes the learning of cross-modal features by intro￾ducing a novel Diffusion prior to bridging the modality gap. To illustrate this, we select a query point from the chair leg in the point cloud, where the features are primarily geometry-dominated and challenging for the image backbone to capture effectively. Its ground-truth corresponding pixel is ma… view at source ↗
Figure 2
Figure 2. The illustration of CSD leveraging diffusion prior to [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Pipeline of Diff2 I2P. We construct a fully differentiable pipeline for cross-modal registration. Given the input image and point cloud, we first perform feature matching to obtain the initial correspondences. Next, the Deformable Correspondence Tuning module takes their features as input and predicts point cloud coordinate offsets, ensuring the differentiability meanwhile refining the correspondences. Subsequently,… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Illustration of Control-Side Score Distillation. We employ a depth-conditioned Diffusion model [ [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Qualitative results on the 7-Scenes dataset. The red lines indicate erroneous correspondences (3D distance greater than 5 cm), [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Illustration of the DCT for refining correspondences. (a) [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: The network architecture of our proposed Diff2 [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 9
Figure 9. Figure 9: Detailed comparison with FreeReg [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 8
Figure 8. Figure 8: Detailed comparison with Diff-Reg. E.5. Detailed comparison with FreeReg Owing to space constraints, we present a simple comparison with FreeReg in the main paper. Additional detailed results on 7-Scenes and RGB-D Scenes V2 are provided in [PITH_FULL_IMAGE:figures/ful…
Figure 10
Figure 10. Figure 10: Correspondence visualizations on 7-scenes [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Correspondence visualizations on RGB-D Scenes V2. [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

66 extracted references · 58 canonical work pages

  1. [1]

    D3feat: Joint learning of dense detection and description of 3D local features

    Xuyang Bai, Zixin Luo, Lei Zhou, Hongbo Fu, Long Quan, and Chiew-Lan Tai. D3feat: Joint learning of dense detection and description of 3D local features. In CVPR, pages 6359– 6367, 2020. 1, 3

    work page 2020

  2. [2]

    PointDSC: Ro- bust point cloud registration using deep spatial consistency

    Xuyang Bai, Zixin Luo, Lei Zhou, Hongkai Chen, Lei Li, Zeyu Hu, Hongbo Fu, and Chiew-Lan Tai. PointDSC: Ro- bust point cloud registration using deep spatial consistency. In CVPR, pages 15859–15869, 2021. 3

    work page 2021

  3. [3]

    End-to-end learnable geometric vision by backpropa- gating pnp optimization

    Bo Chen, Alvaro Parra, Jiewei Cao, Nan Li, and Tat-Jun Chin. End-to-end learnable geometric vision by backpropa- gating pnp optimization. In CVPR, pages 8100–8109, 2020. 2, 3, 4

    work page 2020

  4. [4]

    Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation

    Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation. In ICCV, pages 22246– 22256, 2023. 3

    work page 2023

  5. [5]

    Fully convolutional geometric features

    Christopher Choy, Jaesik Park, and Vladlen Koltun. Fully convolutional geometric features. In ICCV, pages 8958– 8966, 2019. 3, 6, 7

    work page 2019

  6. [6]

    Deep global registration

    Christopher Choy, Wei Dong, and Vladlen Koltun. Deep global registration. In CVPR, pages 2514–2523, 2020. 3

    work page 2020

  7. [7]

    Deformable convolutional networks

    Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen Wei. Deformable convolutional networks. In ICCV, pages 764–773, 2017. 4

    work page 2017

  8. [8]

    Superpoint: Self-supervised interest point detection and description

    Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabi- novich. Superpoint: Self-supervised interest point detection and description. In CVPR, pages 224–236, 2018. 2

    work page 2018

Show all 66 references
  1. [9]

    Model globally, match locally: Efficient and robust 3d object recognition

    Bertram Drost, Markus Ulrich, Nassir Navab, and Slobodan Ilic. Model globally, match locally: Efficient and robust 3d object recognition. In CVPR, pages 998–1005. Ieee, 2010. 3

    2010

  2. [10]

    D2-net: A trainable cnn for joint description and detection of local features

    Mihai Dusmanu, Ignacio Rocco, Tomas Pajdla, Marc Polle- feys, Josef Sivic, Akihiko Torii, and Torsten Sattler. D2-net: A trainable cnn for joint description and detection of local features. In CVPR, pages 8092–8101, 2019. 2

    2019

  3. [11]

    2d3d-matchnet: Learning to match keypoints across 2d image and 3d point cloud

    Mengdan Feng, Sixing Hu, Marcelo H Ang, and Gim Hee Lee. 2d3d-matchnet: Learning to match keypoints across 2d image and 3d point cloud. In IEEE international conference on robotics and automation, pages 4790–4796. IEEE, 2019. 1, 3

    2019

  4. [12]

    Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Communications of the ACM, 24(6):381–395, 1981

    Martin A Fischler and Robert C Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Communications of the ACM, 24(6):381–395, 1981. 1, 3

    1981

  5. [13]

    Real-time rgb-d camera relocalization

    Ben Glocker, Shahram Izadi, Jamie Shotton, and Antonio Criminisi. Real-time rgb-d camera relocalization. In 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 173–179. IEEE, 2013. 2, 6, 7, 3, 5

    2013

  6. [14]

    The perfect match: 3D point cloud matching with smoothed densities

    Zan Gojcic, Caifa Zhou, Jan Wegner, and Andreas Wieser. The perfect match: 3D point cloud matching with smoothed densities. In CVPR, pages 5545–5554, 2019. 1, 3

    2019

  7. [15]

    Multiple view ge- ometry in computer vision

    Richard Hartley and Andrew Zisserman. Multiple view ge- ometry in computer vision . Cambridge university press,

  8. [16]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770–778, 2016. 2, 4, 6

    2016

  9. [17]

    Densely connected convolutional net- works

    Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kil- ian Q Weinberger. Densely connected convolutional net- works. In CVPR, pages 4700–4708, 2017. 2

    2017

  10. [18]

    Predator: Registration of 3D point clouds with low overlap

    Shengyu Huang, Zan Gojcic, Mikhail Usvyatsov, Andreas Wieser, and Konrad Schindler. Predator: Registration of 3D point clouds with low overlap. In CVPR, pages 4267–4276,

  11. [19]

    Contrastive learning of features between images and lidar

    Peng Jiang and Srikanth Saripalli. Contrastive learning of features between images and lidar. In 2022 IEEE 18th Inter- national Conference on Automation Science and Engineer- ing (CASE), pages 411–417. IEEE, 2022. 1

    2022

  12. [20]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 4

    2013 arXiv

  13. [21]

    Unsupervised fea- ture learning for 3d scene labeling

    Kevin Lai, Liefeng Bo, and Dieter Fox. Unsupervised fea- ture learning for 3d scene labeling. In IEEE international conference on robotics and automation , pages 3050–3057. IEEE, 2014. 2, 6, 7, 5

    2014

  14. [22]

    Patchmatch-based neighborhood consensus for se- mantic correspondence

    Jae Yong Lee, Joseph DeGol, Victor Fragoso, and Sudipta N Sinha. Patchmatch-based neighborhood consensus for se- mantic correspondence. In CVPR, pages 13153–13163,

  15. [23]

    Epnp: An accurate o(n) solution to the pnp problem

    Vincent Lepetit, Francesc Moreno-Noguer, and Pascal Fua. Epnp: An accurate o(n) solution to the pnp problem. IJCV, 81:155–166, 2009. 1, 2, 3

    2009

  16. [24]

    Deepi2p: Image-to-point cloud registration via deep classification

    Jiaxin Li and Gim Hee Lee. Deepi2p: Image-to-point cloud registration via deep classification. In CVPR, pages 15960– 15969, 2021. 1, 3

    2021

  17. [25]

    2d3d-matr: 2d-3d matching transformer for detection-free registration between images and point clouds

    Minhao Li, Zheng Qin, Zhirui Gao, Renjiao Yi, Chenyang Zhu, Yulan Guo, and Kai Xu. 2d3d-matr: 2d-3d matching transformer for detection-free registration between images and point clouds. In ICCV, pages 14128–14138, 2023. 1, 3, 4, 5, 6, 7, 8, 2

    2023

  18. [26]

    Dual- resolution correspondence networks

    Xinghui Li, Kai Han, Shuda Li, and Victor Prisacariu. Dual- resolution correspondence networks. NeurIPS, 33:17346– 17357, 2020. 1, 3

    2020

  19. [27]

    Magic3d: High-resolution text-to-3d content creation

    Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3d: High-resolution text-to-3d content creation. In CVPR, pages 300–309, 2023. 3

    2023

  20. [28]

    Feature pyramid networks for object detection

    Tsung-Yi Lin, Piotr Doll ´ar, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In CVPR, pages 2117–2125,

  21. [29]

    Object recognition from local scale-invariant features

    David G Lowe. Object recognition from local scale-invariant features. In ICCV, pages 1150–1157. Ieee, 1999. 2

    1999

  22. [30]

    Aslfeat: Learning local features of accurate shape and lo- calization

    Zixin Luo, Lei Zhou, Xuyang Bai, Hongkai Chen, Jiahui Zhang, Yao Yao, Shiwei Li, Tian Fang, and Long Quan. Aslfeat: Learning local features of accurate shape and lo- calization. In CVPR, pages 6589–6598, 2020. 2

    2020

  23. [31]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM , 65(1):99–106, 2021. 3 9

    2021

  24. [32]

    Colorpcr: Color point cloud registration with multi-stage geometric- color fusion

    Juncheng Mu, Lin Bie, Shaoyi Du, and Yue Gao. Colorpcr: Color point cloud registration with multi-stage geometric- color fusion. In CVPR, pages 21061–21070, 2024. 3

    2024

  25. [33]

    Lf-net: Learning local features from images

    Yuki Ono, Eduard Trulls, Pascal Fua, and Kwang Moo Yi. Lf-net: Learning local features from images. NeurIPS, 31,

  26. [34]

    Lcd: Learned cross-domain descriptors for 2d-3d matching

    Quang-Hieu Pham, Mikaela Angelina Uy, Binh-Son Hua, Duc Thanh Nguyen, Gemma Roig, and Sai-Kit Yeung. Lcd: Learned cross-domain descriptors for 2d-3d matching. In AAAI, pages 11856–11864, 2020. 1

    2020

  27. [35]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022. 3, 4, 5

    2022 arXiv

  28. [36]

    PointNet: Deep learning on point sets for 3D classification and segmentation

    Charles Ruizhongtai Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. PointNet: Deep learning on point sets for 3D classification and segmentation. InCVPR, pages 652– 660, 2017. 2, 3

    2017

  29. [37]

    PointNet++: Deep hierarchical feature learning on point sets in a metric space

    Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. PointNet++: Deep hierarchical feature learning on point sets in a metric space. NeurIPS, 30, 2017. 2, 3

    2017

  30. [38]

    Geometric transformer for fast and ro- bust point cloud registration

    Zheng Qin, Hao Yu, Changjian Wang, Yulan Guo, Yuxing Peng, and Kai Xu. Geometric transformer for fast and ro- bust point cloud registration. In CVPR, pages 11143–11152,

  31. [39]

    Corri2p: Deep image-to-point cloud registration via dense correspondence

    Siyu Ren, Yiming Zeng, Junhui Hou, and Xiaodong Chen. Corri2p: Deep image-to-point cloud registration via dense correspondence. IEEE Transactions on Circuits and Systems for Video Technology, 33(3):1198–1208, 2022. 1, 3

    2022

  32. [40]

    R2d2: Reliable and repeatable detec- tor and descriptor

    Jerome Revaud, Cesar De Souza, Martin Humenberger, and Philippe Weinzaepfel. R2d2: Reliable and repeatable detec- tor and descriptor. NeurIPS, 32, 2019. 2

    2019

  33. [41]

    Neighbourhood con- sensus networks

    Ignacio Rocco, Mircea Cimpoi, Relja Arandjelovi ´c, Akihiko Torii, Tomas Pajdla, and Josef Sivic. Neighbourhood con- sensus networks. NeurIPS, 31, 2018. 1, 3

    2018

  34. [42]

    Efficient neighbourhood consensus networks via submanifold sparse convolutions

    Ignacio Rocco, Relja Arandjelovi ´c, and Josef Sivic. Efficient neighbourhood consensus networks via submanifold sparse convolutions. In ECCV. Springer, 2020. 1, 3

    2020

  35. [43]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, pages 10684– 10695, 2022. 2, 3, 5

    2022

  36. [44]

    Orb: An efficient alternative to sift or surf

    Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary Bradski. Orb: An efficient alternative to sift or surf. InICCV, pages 2564–2571. Ieee, 2011. 2

    2011

  37. [45]

    Fast point feature histograms (fpfh) for 3d registration

    Radu Bogdan Rusu, Nico Blodow, and Michael Beetz. Fast point feature histograms (fpfh) for 3d registration. In IEEE international conference on robotics and automation, pages 3212–3217. IEEE, 2009. 3

    2009

  38. [46]

    Superglue: Learning feature matching with graph neural networks

    Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superglue: Learning feature matching with graph neural networks. InCVPR, pages 4938– 4947, 2020. 2

    2020

  39. [47]

    Loftr: Detector-free local feature matching with transformers

    Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xiaowei Zhou. Loftr: Detector-free local feature matching with transformers. In CVPR, pages 8922–8931, 2021. 1, 3

    2021

  40. [48]

    Dreamgaussian: Generative gaussian splatting for effi- cient 3d content creation

    Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for effi- cient 3d content creation. arXiv preprint arXiv:2309.16653,

    arXiv

  41. [49]

    KpConv: Flexible and deformable con- volution for point clouds

    Hugues Thomas, Charles Ruizhongtai Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, Franc ¸ois Goulette, and Leonidas J Guibas. KpConv: Flexible and deformable con- volution for point clouds. In ICCV, pages 6411–6420, 2019. 2, 3, 4, 6

    2019

  42. [50]

    Bundle adjustment—a mod- ern synthesis

    Bill Triggs, Philip F McLauchlan, Richard I Hartley, and Andrew W Fitzgibbon. Bundle adjustment—a mod- ern synthesis. In Vision Algorithms: Theory and Prac- tice: International Workshop on Vision Algorithms Corfu, Greece, September 21–22, 1999 Proceedings , pages 298–

    1999

  43. [51]

    Sparsity invariant cnns

    Jonas Uhrig, Nick Schneider, Lukas Schneider, Uwe Franke, Thomas Brox, and Andreas Geiger. Sparsity invariant cnns. In 2017 international conference on 3D Vision (3DV), pages 11–20. IEEE, 2017. 6, 2, 3

    2017

  44. [52]

    P2-net: Joint description and detection of local features for pixel and point matching

    Bing Wang, Changhao Chen, Zhaopeng Cui, Jie Qin, Chris Xiaoxuan Lu, Zhengdi Yu, Peijun Zhao, Zhen Dong, Fan Zhu, Niki Trigoni, et al. P2-net: Joint description and detection of local features for pixel and point matching. In ICCV, pages 16004–16013, 2021. 1, 6, 7

    2021

  45. [53]

    Freereg: Image- to-point cloud registration leveraging pretrained diffusion models and monocular depth estimators

    Haiping Wang, Yuan Liu, Bing Wang, Yujing Sun, Zhen Dong, Wenping Wang, and Bisheng Yang. Freereg: Image- to-point cloud registration leveraging pretrained diffusion models and monocular depth estimators. arXiv preprint arXiv:2310.03420, 2023. 3, 6, 8, 2, 5

    arXiv

  46. [54]

    Diff-reg: Diffusion model in doubly stochastic matrix space for registration problem

    Qianliang Wu, Haobo Jiang, Lei Luo, Jun Li, Yaqing Ding, Jin Xie, and Jian Yang. Diff-reg: Diffusion model in doubly stochastic matrix space for registration problem. In ECCV, pages 160–178. Springer, 2024. 3, 5

    2024

  47. [55]

    Efficient deformable convnets: Rethink- ing dynamic and sparse operator for vision applications

    Yuwen Xiong, Zhiqi Li, Yuntao Chen, Feng Wang, Xizhou Zhu, Jiapeng Luo, Wenhai Wang, Tong Lu, Hongsheng Li, Yu Qiao, et al. Efficient deformable convnets: Rethink- ing dynamic and sparse operator for vision applications. In CVPR, pages 5652–5661, 2024. 4

    2024

  48. [56]

    Sanet: Scene agnostic net- work for camera localization

    Luwei Yang, Ziqian Bai, Chengzhou Tang, Honghua Li, Ya- sutaka Furukawa, and Ping Tan. Sanet: Scene agnostic net- work for camera localization. In ICCV, pages 42–51, 2019. 2

    2019

  49. [57]

    Lift: Learned invariant feature transform

    Kwang Moo Yi, Eduard Trulls, Vincent Lepetit, and Pascal Fua. Lift: Learned invariant feature transform. In ECCV, pages 467–483. Springer, 2016. 2

    2016

  50. [58]

    CofiNet: Reliable coarse-to-fine correspondences for robust pointcloud registration

    Hao Yu, Fu Li, Mahdi Saleh, Benjamin Busam, and Slobo- dan Ilic. CofiNet: Reliable coarse-to-fine correspondences for robust pointcloud registration. NeurIPS, 34:23872– 23884, 2021. 1, 3

    2021

  51. [59]

    PEAL: Prior-embedded explicit attention learn- ing for low-overlap point cloud registration

    Junle Yu, Luwei Ren, Yu Zhang, Wenhui Zhou, Lili Lin, and Guojun Dai. PEAL: Prior-embedded explicit attention learn- ing for low-overlap point cloud registration. In CVPR, pages 17702–17711, 2023. 3

    2023

  52. [60]

    3DMatch: Learning local geometric descriptors from RGB-D recon- structions

    Andy Zeng, Shuran Song, Matthias Nießner, Matthew Fisher, Jianxiong Xiao, and Thomas Funkhouser. 3DMatch: Learning local geometric descriptors from RGB-D recon- structions. In CVPR, pages 1802–1811, 2017. 1, 3

    2017

  53. [61]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, pages 3836–3847, 2023. 2, 3, 4, 5 10

    2023

  54. [62]

    PCR-CG: Point cloud registration via deep explicit color and geometry

    Yu Zhang, Junle Yu, Xiaolin Huang, Wenhui Zhou, and Ji Hou. PCR-CG: Point cloud registration via deep explicit color and geometry. In ECCV, pages 443–459. Springer,

  55. [63]

    Differentiable registration of images and lidar point clouds with voxelpoint-to-pixel matching

    Junsheng Zhou, Baorui Ma, Wenyuan Zhang, Yi Fang, Yu- Shen Liu, and Zhizhong Han. Differentiable registration of images and lidar point clouds with voxelpoint-to-pixel matching. NeurIPS, 36, 2024. 1, 3, 2

    2024

  56. [64]

    Patch2pix: Epipolar-guided pixel-level correspondences

    Qunjie Zhou, Torsten Sattler, and Laura Leal-Taixe. Patch2pix: Epipolar-guided pixel-level correspondences. In CVPR, pages 4669–4678, 2021. 1, 3

    2021

  57. [65]

    De- formable convnets v2: More deformable, better results

    Xizhou Zhu, Han Hu, Stephen Lin, and Jifeng Dai. De- formable convnets v2: More deformable, better results. In CVPR, pages 9308–9316, 2019. 4

    2019

  58. [66]

    Deformable detr: Deformable trans- formers for end-to-end object detection

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable trans- formers for end-to-end object detection. arXiv preprint arXiv:2010.04159, 2020. 4 11 Diff2I2P: Differentiable Image-to-Point Cloud Registration with Diffusion Prior Suppl...

    2010 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.