Pith. sign in

REVIEW 10 cited by

Learning from 2D: Contrastive Pixel-to-Point Knowledge Transfer for 3D Pretraining

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2104.04687 v3 pith:EG6SOGE6 submitted 2021-04-10 cs.CV

Yueh-Cheng Liu , Yu-Kai Huang , Hung-Yueh Chiang , Hung-Ting Su , Zhe-Yu Liu , Chin-Tang Chen , Ching-Yu Tseng , Winston H. Hsu This is my paper · ORCID
classification cs.CV
keywords networksknowledgepretrainingtransfercontrastivedatadatasetsfeature
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Most 3D neural networks are trained from scratch owing to the lack of large-scale labeled 3D datasets. In this paper, we present a novel 3D pretraining method by leveraging 2D networks learned from rich 2D datasets. We propose the contrastive pixel-to-point knowledge transfer to effectively utilize the 2D information by mapping the pixel-level and point-level features into the same embedding space. Due to the heterogeneous nature between 2D and 3D networks, we introduce the back-projection function to align the features between 2D and 3D to make the transfer possible. Additionally, we devise an upsampling feature projection layer to increase the spatial resolution of high-level 2D feature maps, which enables learning fine-grained 3D representations. With a pretrained 2D network, the proposed pretraining process requires no additional 2D or 3D labeled data, further alleviating the expensive 3D data annotation cost. To the best of our knowledge, we are the first to exploit existing 2D trained weights to pretrain 3D deep neural networks. Our intensive experiments show that the 3D models pretrained with 2D knowledge boost the performances of 3D networks across various real-world 3D downstream tasks.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TOLiD: Bridging the Architecture Gap in Vision Foundation Model to LiDAR Pretraining via Token Lifting for Distillation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Coupling a LiDAR backbone to a same-family student ViT and distilling via frustum-pooled/attended patch tokens yields stronger frozen LiDAR features and cross-sensor transfer than direct VFM-to-3D distillation.

  2. Beyond One Shot, Beyond One Perspective: Cross-View and Long-Horizon Distillation for Better LiDAR Representations

    cs.CV 2025-07 conditional novelty 6.0 of 10

    LiMA distills long-term multi-camera image features into LiDAR backbones and reports consistent gains on segmentation and detection benchmarks.

  3. 2D-3D Attention and Entropy for Pose Robust 2D Facial Recognition

    cs.CV 2025-05 reject novelty 6.0 of 10

    A 2D-3D domain adaptation framework with shared attention and a joint entropy regularizer improves profile-view face recognition, though the entropy loss is mis-specified.

  4. The Devil is in the Details: Simple Remedies for Image-to-LiDAR Representation Learning

    cs.CV 2025-01 conditional novelty 6.0 of 10

    Using Cartesian voxelization and a positive-pair mining module to exploit unsynced LiDAR-image data improves image-to-LiDAR representation learning, setting new state-of-the-art on nuScenes segmentation and KITTI detection.

  5. TiGDistill-BEV: Multi-view BEV 3D Object Detection via Target Inner-Geometry Learning Distillation

    cs.CV 2024-12 conditional novelty 5.0 of 10

    A LiDAR-to-camera distillation method that supervises relative depth inside object foregrounds and distills BEV feature relationships to boost camera-only 3D object detection.

  6. Vernata: Self-Supervised Learning of LiDAR Point Representations

    cs.CV 2026-08 conditional novelty 4.0 of 10

    Vernata extends Sonata with sparse-view augmentation, a Sinkhorn-Knopp memory bank, and high-resolution 2D-to-3D distillation, improving LiDAR semantic segmentation mIoU by 5.9 to 7.3 points over baselines.

  7. Domain Adaptation-Based Crossmodal Knowledge Distillation for 3D Semantic Segmentation

    cs.CV 2025-08 conditional novelty 4.0 of 10

    A 3D self-calibrated convolution module plus feature and semantic distillation lets a LiDAR network learn from 2D image teachers without 3D labels.

  8. Multi-modal Multi-task Pre-training for Improved Point Cloud Understanding

    cs.CV 2025-07 reject novelty 4.0 of 10

    MMPT combines three existing self-supervised tasks for point cloud pre-training and reports improved results across several benchmarks.

  9. High-quality Pseudo-labeling for Point Cloud Segmentation with Scene-level Annotation

    cs.CV 2025-06 conditional novelty 4.0 of 10

    A scene-level annotated point cloud segmentation framework combining contrastive 2D-3D feature alignment with region-point consistency improves pseudo-label quality and achieves SOTA on ScanNet and S3DIS.

  10. Foundational Models for 3D Point Clouds: A Survey and Outlook

    cs.CV 2025-01 conditional novelty 4.0 of 10

    A structured review of methods that build or adapt 2D foundation models and LLMs for 3D point cloud tasks, with a proposed taxonomy and curated paper list.

Pith tools