Pith. sign in

REVIEW 5 cited by

Learning from 2D: Contrastive Pixel-to-Point Knowledge Transfer for 3D Pretraining

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2104.04687 v3 pith:EG6SOGE6 submitted 2021-04-10 cs.CV

Yueh-Cheng Liu , Yu-Kai Huang , Hung-Yueh Chiang , Hung-Ting Su , Zhe-Yu Liu , Chin-Tang Chen , Ching-Yu Tseng , Winston H. Hsu This is my paper
classification cs.CV
keywords networksknowledgepretrainingtransfercontrastivedatadatasetsfeature
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Most 3D neural networks are trained from scratch owing to the lack of large-scale labeled 3D datasets. In this paper, we present a novel 3D pretraining method by leveraging 2D networks learned from rich 2D datasets. We propose the contrastive pixel-to-point knowledge transfer to effectively utilize the 2D information by mapping the pixel-level and point-level features into the same embedding space. Due to the heterogeneous nature between 2D and 3D networks, we introduce the back-projection function to align the features between 2D and 3D to make the transfer possible. Additionally, we devise an upsampling feature projection layer to increase the spatial resolution of high-level 2D feature maps, which enables learning fine-grained 3D representations. With a pretrained 2D network, the proposed pretraining process requires no additional 2D or 3D labeled data, further alleviating the expensive 3D data annotation cost. To the best of our knowledge, we are the first to exploit existing 2D trained weights to pretrain 3D deep neural networks. Our intensive experiments show that the 3D models pretrained with 2D knowledge boost the performances of 3D networks across various real-world 3D downstream tasks.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TOLiD: Bridging the Architecture Gap in Vision Foundation Model to LiDAR Pretraining via Token Lifting for Distillation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Coupling a LiDAR backbone to a same-family student ViT and distilling via frustum-pooled/attended patch tokens yields stronger frozen LiDAR features and cross-sensor transfer than direct VFM-to-3D distillation.

  2. Beyond One Shot, Beyond One Perspective: Cross-View and Long-Horizon Distillation for Better LiDAR Representations

    cs.CV 2025-07 conditional novelty 6.0 of 10

    LiMA distills long-term multi-camera image features into LiDAR backbones and reports consistent gains on segmentation and detection benchmarks.

  3. Domain Adaptation-Based Crossmodal Knowledge Distillation for 3D Semantic Segmentation

    cs.CV 2025-08 conditional novelty 4.0 of 10

    A 3D self-calibrated convolution module plus feature and semantic distillation lets a LiDAR network learn from 2D image teachers without 3D labels.

  4. Multi-modal Multi-task Pre-training for Improved Point Cloud Understanding

    cs.CV 2025-07 reject novelty 4.0 of 10

    MMPT combines three existing self-supervised tasks for point cloud pre-training and reports improved results across several benchmarks.

  5. High-quality Pseudo-labeling for Point Cloud Segmentation with Scene-level Annotation

    cs.CV 2025-06 conditional novelty 4.0 of 10

    A scene-level annotated point cloud segmentation framework combining contrastive 2D-3D feature alignment with region-point consistency improves pseudo-label quality and achieves SOTA on ScanNet and S3DIS.

Pith tools