Pith. sign in

REVIEW 4 major objections 5 minor 72 references

LeAP: Consistent multi-domain 3D labeling using Foundation Models

T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Pith's one-line read LeAP generates high-quality 3D semantic labels from unlabeled image–LiDAR pairs and a class list, with no manual annotation, and the labels boost downstream segmentation mIoU by up to 34.2 points.

desk verdict A practical pseudo-labeling pipeline that works, with a real but not fatal statistical flaw in the Bayesian fusion; worth refereeing. read the letter →

arxiv 2502.03901 v1 pith:5P4AKIXQ submitted 2025-02-06 cs.CV cs.RO

classification cs.CVcs.RO
keywords 3Dsemanticsegmentationpseudo-labelingvisionfoundationmodelsopen-vocabularyBayesianfusionpointcloudsdomainadaptationLiDAR
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that pairs of unlabeled camera images and LiDAR scans, together with a user-supplied list of classes, are enough to produce high-quality 3D semantic labels for almost any environment. It does this by prompting open-vocabulary 2D foundation models to produce per-pixel soft class probabilities, projecting those labels onto 3D points, and fusing them across time into a sparse voxel grid using a Bayesian update. A 3D consistency network trained on the most confident voxel labels then sharpens the result. On automotive and synthetic drone data the method outperforms a pre-trained cross-domain segmentation baseline, and models fine-tuned on the generated labels improve mIoU by up to 34.2 points on the UAV domain.

What carries the argument

The central machinery is a sparse voxel grid storing a class-probability distribution per voxel, updated by Bayes' rule (Eq. 1) as each new projected 2D label arrives. This makes repeated observations of the same spatial region combine into a statistically grounded estimate, filters out per-frame projection noise, and allows retro-active labeling of points outside the current camera frustum. A distance-weighted k-nearest-neighbor average smooths the grid, and a 3D Consistency Network trained on the most confident voxel labels provides a complementary 3D-aware signal that is fused back into the grid with a temperature-weighted Bayesian update.

What would settle it

Measure pseudo-label accuracy as a function of the number of frames viewing a given voxel on objects that a VFM systematically mislabels; if accuracy drops or confidence in the wrong class grows with more views, the independence assumption is broken. For a moving cyclist consistently split into person and bicycle, the Bayesian update should be observed to assign higher probability to person or bicycle rather than to cyclist.

Watch

Extended reading notes

Core claim

LeAP's central claim is that one can generate high-quality 3D semantic labels for any set of classes in any domain without manual annotation, using only paired camera images and LiDAR plus a class list. The method combines two 2D foundation models — Grounding DINO for open-vocabulary region proposals and SAM for per-pixel masks — into per-pixel soft class probabilities, then projects them onto 3D points and accumulates them in a sparse voxel grid via Bayesian updating. The voxel grid gives spatio-temporal consistency, labels points never seen by the camera, and enables a self-trained 3D consistency network to refine the labels. The authors report that the resulting pseudo-labels beat a pre-trained cross-domain segmentation baseline on both SemanticKITTI and their synthetic AgriUAV dataset, and that fine-tuning a segmentation model on these labels raises mIoU by up to 34.2 on the aerial domain.

Load-bearing premise

The Bayesian update treats every projected 2D label for a voxel as an independent observation, so the whole consistency gain rests on the assumption that the 2D foundation models' errors are not systematically correlated across frames.

Editorial extensions

If this is right

  • New domains can obtain 3D semantic labels without any human annotation, needing only unlabeled image–LiDAR data and a class list.
  • Voxel-based fusion labels over six times more points than point-wise 2D-to-3D projection alone, including points never observed by the camera.
  • Fine-tuning a segmentation model on LeAP labels narrows the domain gap, improving mIoU by 11.5 on automotive and 34.2 on the synthetic aerial domain.
  • Fusing the self-trained 3D network's predictions with the camera-only labels yields higher pseudo-label quality than either source alone, approaching oracle-level performance in the aerial domain.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the independence assumption in Eq. 1 fails under systematic VFM errors, the Bayesian update could reinforce wrong labels; the paper's limitation section concedes self-training cannot correct such systematic errors.
  • The voxel representation is sensor-agnostic, so the same pipeline could plausibly fuse labels from radar, sonar, or multi-camera rigs without architectural changes.
  • A stronger test of the method would be to use it to bootstrap a segmentation model in a genuinely new domain without any ground truth, then measure human correction effort rather than mIoU against a fixed label set.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces LeAP, an automatic 3D pseudo-labeling pipeline that uses 2D vision foundation models (Grounding DINO and SAM) to generate per-pixel soft class probabilities from images, projects these onto LiDAR points, and accumulates them into a sparse voxel grid using a Bayesian update (Eq. 1), followed by k-NN smoothing. A 3D Consistency Network (3D-CN) is trained on the most confident camera-derived voxel labels and its predictions are fused back into the grid. The authors evaluate label quality on SemanticKITTI and on a synthetic AirSim UAV dataset (AgriUAV) that they introduce, reporting that LeAP outperforms a pre-trained source-domain baseline and that fine-tuning on LeAP labels improves target-domain mIoU by up to 34.2 points. They also show that the 3D-CN improves fused label quality over camera-only labels.

Significance. If the reported results are robust, LeAP addresses a real bottleneck: creating 3D semantic labels for novel domains without manual annotation. The pipeline is described in enough detail to be plausible, the SemanticKITTI evaluation uses real data with external ground truth, and the introduction of a synthetic UAV dataset is a useful resource. The paper provides comparisons against a source-domain pre-trained baseline and shows consistent gains from voxel fusion and the 3D-CN. However, the central technical claims rest on a questionable statistical foundation in Eq. 1 and on a self-training step whose complementary-information gain is not fully demonstrated. The lack of uncertainty estimates and the presence of per-dataset hyperparameter tuning further limit the strength of the 'domain-agnostic, no manual labeling' claim.

major comments (4)
  1. [Section III-B, Eq. (1)] The Bayesian update treats P(pi|Xk), the per-frame softmax output of the VFM, as a likelihood term in a repeated product. These are posterior class probabilities conditioned on the same model and are highly correlated across frames, so the product does not represent independent evidence. Consequently, systematic VFM errors (e.g., splitting a cyclist into person and bicycle, as the authors themselves note in Section IV-E) become increasingly confident in the voxel distribution instead of being averaged out. This is not merely a theoretical caveat: the 3D-CN in Section III-C selects supervision from the most confident voxel labels, so any overconfidence produced by Eq. 1 directly shapes the pseudo-ground truth. To support the claim that the update is 'statistically grounded', please either derive a proper likelihood model (e.g., calibrating VFM outputs as likelihoods) or compare Eq. 1 against simpler aggregators (averaging, max-pooling, or log-odds with a correlation penalty) and show that the gains in Tables I and II persist. At minimum, report the calibration of the fused probabilities against empirical accuracy.
  2. [Section III-C and Tables I-II] The 3D-CN is a single-round self-training bootstrap: it is trained on the 20% most confident camera pseudo-labels per class and its predictions are fused back into the same voxel grid. The paper distinguishes this from iterative self-training, but the supervision and the fused signal originate from the same initial pseudo-labels, so the claim that the network provides 'complementary information' is not established. The large improvements from the 3D-CN (Table I: mIoU 48.9 to 57.6; Table II UAV: 33.4 to 47.1) could partly reflect the network memorizing its own training labels rather than adding independent 3D cues. Please report the ground-truth quality of the 3D-CN's raw predictions before fusion, show per-class improvements for classes that already had high camera-only IoU, and specify how the 20% threshold and the fusion temperature were selected (ideally without using validation ground truth).
  3. [Section IV-D, Tables I and II] All quantitative results appear to come from single runs, with no error bars or repeated seeds reported. The difference between one and two 3D-CN iterations in Table I is 0.5 mIoU on SemanticKITTI and 0.7 mIoU on AgriUAV, which is likely within random seed noise. Since the headline claims include a '34.2 mIoU increase' and a '3.7x improvement' (Sections IV-G and V), please provide at least 3-5 runs for the trained models (the 3D-CN and the fine-tuned WaffleIron) and report mean and standard deviation, or otherwise justify that the reported differences are statistically significant.
  4. [Section IV-A, IV-B] The only non-automotive domain is represented by the synthetic AirSim dataset AgriUAV; there is no real-world non-automotive evaluation. Additionally, the Grounding DINO similarity threshold is tuned separately for each dataset (0.25 for SemanticKITTI, 0.2 for AgriUAV), and the text says this was necessary because higher values 'resulted in very few masks' on the synthetic images. This per-dataset hyperparameter tuning weakens the claims of being 'domain agnostic' and requiring 'no manual labeling', and the synthetic-only UAV evidence limits the generality of the multi-domain conclusion. Please either add a real non-automotive experiment or revise the claims to specify that the UAV results are synthetic-only, and report the sensitivity of the results to the threshold hyperparameter.
minor comments (5)
  1. [Section III-B] The phrase 'camera fustrum' should be 'camera frustum'.
  2. [Section IV-A] The abbreviation 'UA V' appears throughout the paper and should be 'UAV'.
  3. [Figure 3] The figure label '3D-CNC' appears to be a typo; the text consistently uses '3D-CN'.
  4. [Figure 4 caption] The word 'consistancy' in the caption should be 'consistency'.
  5. [Table II head] The header entry 'Wf.Iron [70]' should be spelled 'WaffleIron [70]' for consistency with the text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims are evaluated against external ground truth, and the 3D-CN bootstrap is empirical self-training rather than a definitional reduction.

full rationale

LeAP's derivation chain is: 2D VFMs produce per-pixel soft labels; projection and Bayesian voxel fusion (Eq. 1) aggregate them into 3D pseudo-labels; a 3D-CN is trained on the most confident subset of those pseudo-labels and its output is fused back into the voxel grid. No step defines the predicted quantity in terms of the target quantity. Label quality is measured against external ground truth on SemanticKITTI and AgriUAV (Table I), and domain adaptation is measured by fine-tuning an independent WaffleIron backbone on LeAP labels and evaluating on held-out validation sets (Table II). The only self-referential component is the 3D-CN in Section III-C, which is a one-iteration self-training procedure: it trains a 3D network on camera-only pseudo-labels and then fuses its predictions back. This is a genuine bootstrap, but the fused result is evaluated against external ground truth, and improvement is an empirical observation rather than a consequence of the method's definitions. The paper's own Limitations section concedes that self-training cannot correct systematic errors, which further indicates that the authors do not claim the 3D-CN adds information by construction. The self-citations in the reference list ([7], [8], [69]) are background or related-work citations and are not load-bearing for the main claim. The Bayesian update in Eq. 1 assumes per-frame VFM probabilities behave like conditionally independent likelihoods; that is a statistical correctness concern (and a potential risk), not a circularity, because the final labels are still validated against external ground truth. Overall, the paper's headline results do not reduce to its inputs by construction.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

LeAP is an empirical pipeline rather than a derivation, so the ledger tracks the modeling assumptions and hand-set hyperparameters. The main load-bearing assumptions are that 2D VFM probabilities are calibrated and conditionally independent across views, and that a self-trained 3D network can add complementary signal. No parameters are fitted to the final mIoU numbers, but several design choices, including voxel size, k, supervision fraction, temperature, and per-dataset thresholds, are chosen by hand and not ablated.

free parameters (6)
  • Grounding DINO region similarity threshold = 0.25 (SemanticKITTI), 0.2 (AgriUAV)
    Section IV-B: filters low-confidence region proposals; adjusted per dataset because higher values produced too few masks on synthetic images.
  • Voxel size = 0.2 m
    Section IV-B: chosen for the sparse voxel grid; no ablation reported.
  • KNN smoothing neighbors k = 9
    Section IV-B: number of neighbors used in distance-weighted voxel smoothing; fixed without ablation.
  • 3D-CN supervision fraction = 20% most confident labels per class
    Section III-C and IV-B: used to select reliable pseudo-ground-truth labels for training the 3D network.
  • 3D-CN fusion temperature = not reported
    Section III-C: temperature weights the reliability of 3D-CN predictions in the Bayesian fusion; the value is not given.
  • Grounding DINO Q and T = Q=900, T=256
    Section IV-B: query regions and text tokens for the detector.
assumptions (6)
  • domain assumption Per-pixel probabilities from Grounding DINO and SAM are calibrated enough to be used as likelihoods.
    Section III-A: softmax outputs are used as class probabilities and inserted into the Bayesian update without calibration.
  • domain assumption Observations of the same voxel across time are conditionally independent given the class.
    Section III-B, Eq. 1: the update multiplies per-frame probabilities; correlated VFM errors are not modeled.
  • ad hoc to paper Points inside a 2D mask are close in 3D, so the largest distance cluster from the camera is the true object.
    Section III-B: heuristic filtering step to remove projected background points; no independent validation.
  • ad hoc to paper The 20% most confident pseudo-labels per class are reliable enough to train a 3D segmentation network.
    Section III-C: used to supervise the 3D-CN; label noise and selection bias are not analyzed.
  • ad hoc to paper A 3D network trained on the same camera pseudo-labels provides complementary information when fused back into the labels.
    Section III-C and IV-G: the improvement is attributed to complementarity, but complementarity is not measured directly.
  • standard math Bayes rule and sparse voxel hashing apply as in SemanticFusion.
    Section III-B: Eq. 1 and the voxel grid follow prior work [54], [65], [66].

how reviews work

0 comments
Cite this review

Pith. "Pith review of LeAP: Consistent multi-domain 3D labeling using Foundation Models." pith.science (2026). https://pith.science/paper/5P4AKIXQ

@misc{pith2026250203901,
  author       = {Pith},
  title        = {Pith review of: LeAP: Consistent multi-domain 3D labeling using Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5P4AKIXQ}},
  note         = {Machine review of arXiv:2502.03901}
}
read the original abstract

Availability of datasets is a strong driver for research on 3D semantic understanding, and whilst obtaining unlabeled 3D point cloud data is straightforward, manually annotating this data with semantic labels is time-consuming and costly. Recently, Vision Foundation Models (VFMs) enable open-set semantic segmentation on camera images, potentially aiding automatic labeling. However,VFMs for 3D data have been limited to adaptations of 2D models, which can introduce inconsistencies to 3D labels. This work introduces Label Any Pointcloud (LeAP), leveraging 2D VFMs to automatically label 3D data with any set of classes in any kind of application whilst ensuring label consistency. Using a Bayesian update, point labels are combined into voxels to improve spatio-temporal consistency. A novel 3D Consistency Network (3D-CN) exploits 3D information to further improve label quality. Through various experiments, we show that our method can generate high-quality 3D semantic labels across diverse fields without any manual labeling. Further, models adapted to new domains using our labels show up to a 34.2 mIoU increase in semantic segmentation tasks.

Figures

Figures reproduced from arXiv: 2502.03901 by the authors.

Figure 1
Figure 1. Overview of our LeAP automatic labeling method. ( [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The process of generating 2D pseudo-labels. Using unlabeled images and a list of classes, we use Grounding Dino [ [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The process of generating 3D pseudo-labels. Point clouds are [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Qualitative results of our pseudo-labeling pipeline. Frames of SemanticKITTI [ [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

72 extracted references · 32 canonical work pages

  1. [1]

    nuScenes: A Multimodal Dataset for Autonomous Driving,

    H. Caesar, V . Bankiti, A. H. Lang, S. V ora, V . E. Liong, Q. Xu, A. Krishnan, Y . Pan, G. Baldan, and O. Beijbom, “nuScenes: A Multimodal Dataset for Autonomous Driving,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Seattle, W A, USA: IEEE, June 2020, pp. 11 618–11 628. [Online]. Available: https://ieeexplore.ieee.org/...

  2. [2]

    Scalability in Perception for Autonomous Driving: Waymo Open Dataset,

    P. Sun, H. Kretzschmar, X. Dotiwalla, A. Chouard, V . Patnaik, P. Tsui, J. Guo, Y . Zhou, Y . Chai, B. Caine, V . Vasudevan, W. Han, J. Ngiam, H. Zhao, A. Timofeev, S. Ettinger, M. Krivokon, A. Gao, A. Joshi, Y . Zhang, J. Shlens, Z. Chen, and D. Anguelov, “Scalability in Perception for Autonomous Driving: Waymo Open Dataset,” in 2020 IEEE/CVF Conference ...

  3. [3]

    Are we ready for autonomous driving? The KITTI vision benchmark suite,

    A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? The KITTI vision benchmark suite,” in 2012 IEEE Conference on Computer Vision and Pattern Recognition . Providence, RI: IEEE, June 2012, pp. 3354–3361. [Online]. Available: http://ieeexplore.ieee.org/document/6248074/

  4. [4]

    SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences,

    J. Behley, M. Garbade, A. Milioto, J. Quenzel, S. Behnke, C. Stachniss, and J. Gall, “SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Seoul, Korea (South): IEEE, Oct. 2019, pp. 9296–9306, arXiv:1904.01416 [cs]. [Online]. Available: https://ieeexplore.ieee....

  5. [5]

    KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D,

    Y . Liao, J. Xie, and A. Geiger, “KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 45, no. 3, pp. 3292–3310, Mar. 2023, arXiv:2109.13410 [cs]. [Online]. Available: https://ieeexplore.ieee.org/document/9786676/

  6. [6]

    PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm,

    H. Zhu, H. Yang, X. Wu, D. Huang, S. Zhang, X. He, T. He, H. Zhao, C. Shen, Y . Qiao, and W. Ouyang, “PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm,” Oct. 2023, arXiv:2310.08586 [cs]. [Online]. Available: http://arxiv.org/abs/2310.08586

  7. [7]

    Offline Tracking with Object Permanence

    X. Liu and H. Caesar, “Offline Tracking with Object Permanence,” May 2024, arXiv:2310.01288 [cs]. [Online]. Available: http://arxiv. org/abs/2310.01288

  8. [8]

    Towards learning-based planning:The nuPlan benchmark for real-world autonomous driving,

    N. Karnchanachari, D. Geromichalos, K. S. Tan, N. Li, C. Eriksen, S. Yaghoubi, N. Mehdipour, G. Bernasconi, W. K. Fong, Y . Guo, and H. Caesar, “Towards learning-based planning:The nuPlan benchmark for real-world autonomous driving,” Mar. 2024, arXiv:2403.04133 [cs]. [Online]. Available: http://arxiv.org/abs/2403.04133

Show all 72 references
  1. [9]

    Learning Transferable Visual Models From Natural Language Supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning Transferable Visual Models From Natural Language Supervision,” Feb. 2021, arXiv:2103.00020 [cs]. [Online]. Available: http://ar...

  2. [10]

    Segment Anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, P. Doll ´ar, and R. Girshick, “Segment Anything,” in 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Paris, France: IEEE, Oct. 2023, pp. 3992–400...

  3. [11]

    Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data,

    L. Yang, B. Kang, Z. Huang, X. Xu, J. Feng, and H. Zhao, “Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data,” Jan. 2024, arXiv:2401.10891 [cs]. [Online]. Available: http://arxiv.org/abs/2401.10891

  4. [12]

    PointCLIP: Point Cloud Understanding by CLIP,

    R. Zhang, Z. Guo, W. Zhang, K. Li, X. Miao, B. Cui, Y . Qiao, P. Gao, and H. Li, “PointCLIP: Point Cloud Understanding by CLIP,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, LA, USA: IEEE, June 2022, pp. 8542–8552. [Online]. Avail...

  5. [13]

    OpenScene: 3D Scene Understanding with Open V ocabularies,

    S. Peng, K. Genova, C. Jiang, A. Tagliasacchi, M. Pollefeys, and T. Funkhouser, “OpenScene: 3D Scene Understanding with Open V ocabularies,” in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Vancouver, BC, Canada: IEEE, June 2023, pp. 815–824, arX...

  6. [14]

    CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP,

    R. Chen, Y . Liu, L. Kong, X. Zhu, Y . Ma, Y . Li, Y . Hou, Y . Qiao, and W. Wang, “CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP,” in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Vancouver, BC, Canada: IEEE, June 2023, pp. ...

  7. [15]

    OVO: Open-V ocabulary Occupancy,

    Z. Tan, Z. Dong, C. Zhang, W. Zhang, H. Ji, and H. Li, “OVO: Open-V ocabulary Occupancy,” June 2023, arXiv:2305.16133 [cs]. [Online]. Available: http://arxiv.org/abs/2305.16133

  8. [16]

    POP-3D: Open-V ocabulary 3D Occupancy Prediction from Images,

    A. V obecky, O. Sim ´eoni, D. Hurych, S. Gidaris, A. Bursuc, P. P ´erez, and J. Sivic, “POP-3D: Open-V ocabulary 3D Occupancy Prediction from Images,” Jan. 2024, arXiv:2401.09413 [cs]. [Online]. Available: http://arxiv.org/abs/2401.09413

  9. [17]

    LidarCLIP or: How I Learned to Talk to Point Clouds,

    G. Hess, A. Tonderski, C. Petersson, K. ˚Astr¨om, and L. Svensson, “LidarCLIP or: How I Learned to Talk to Point Clouds,” in 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Waikoloa, HI, USA: IEEE, Jan. 2024, pp. 7423–7432. [Online]. Available: https...

  10. [18]

    Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Training,

    X. Wu, Z. Tian, X. Wen, B. Peng, X. Liu, K. Yu, and H. Zhao, “Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Training,” Aug. 2023, arXiv:2308.09718 [cs]. [Online]. Available: http://arxiv.org/abs/2308.09718

  11. [19]

    Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection,

    S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, C. Li, J. Yang, H. Su, J. Zhu, and L. Zhang, “Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection,” Mar. 2023, arXiv:2303.05499 [cs]. [Online]. Available: http://arxiv.org/abs/2303.05499

  12. [20]

    V oxNet: A 3D Convolutional Neural Network for real-time object recognition,

    D. Maturana and S. Scherer, “V oxNet: A 3D Convolutional Neural Network for real-time object recognition,” in 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . Hamburg, Germany: IEEE, Sept. 2015, pp. 922–928. [Online]. Available: http://ieeexplo...

  13. [21]

    3D Semantic Segmentation with Submanifold Sparse Convolutional Networks,

    B. Graham, M. Engelcke, and L. V . D. Maaten, “3D Semantic Segmentation with Submanifold Sparse Convolutional Networks,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, June 2018, pp. 9224–9232, arXiv:1711.10275 [cs]. [Onl...

  14. [22]

    Searching Efficient 3D Architectures with Sparse Point- V oxel Convolution,

    H. Tang, Z. Liu, S. Zhao, Y . Lin, J. Lin, H. Wang, and S. Han, “Searching Efficient 3D Architectures with Sparse Point- V oxel Convolution,” Computer Vision – ECCV 2020 , vol. 12373, pp. 685–702, 2020, arXiv:2007.16100 [cs]. [Online]. Available: https://link.springer.com/10.1...

  15. [23]

    Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation,

    X. Zhu, H. Zhou, T. Wang, F. Hong, Y . Ma, W. Li, H. Li, and D. Lin, “Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Nashville, TN, USA: IEEE, June 2021, pp. 9934–994...

  16. [24]

    SV ASeg: Sparse V oxel-Based Attention for 3D LiDAR Point Cloud Semantic Segmentation,

    L. Zhao, S. Xu, L. Liu, D. Ming, and W. Tao, “SV ASeg: Sparse V oxel-Based Attention for 3D LiDAR Point Cloud Semantic Segmentation,” Remote Sensing , vol. 14, no. 18, p. 4471, Sept. 2022. [Online]. Available: https://www.mdpi.com/2072-4292/14/18/4471

  17. [25]

    Spherical Transformer for LiDAR-Based 3D Recognition,

    X. Lai, Y . Chen, F. Lu, J. Liu, and J. Jia, “Spherical Transformer for LiDAR-Based 3D Recognition,” in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Vancouver, BC, Canada: IEEE, June 2023, pp. 17 545–17 555, arXiv:2303.12766 [cs]. [Online]. Avai...

  18. [26]

    OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Prediction,

    Y . Zhang, Z. Zhu, and D. Du, “OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Prediction,” in 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Paris, France: IEEE, Oct. 2023, pp. 9399–9409, arXiv:2304.05316 [cs]. [Online]. Available: ...

  19. [27]

    MonoScene: Monocular 3D Semantic Scene Completion,

    A.-Q. Cao and R. De Charette, “MonoScene: Monocular 3D Semantic Scene Completion,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . New Orleans, LA, USA: IEEE, June 2022, pp. 3981–3991, arXiv:2112.00726 [cs]. [Online]. Available: https://ieeexplo...

  20. [28]

    SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud,

    B. Wu, X. Zhou, S. Zhao, X. Yue, and K. Keutzer, “SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud,” Sept. 2018, arXiv:1809.08495 [cs]. [Online]. Available: http: //arxiv.org/abs/1809.08495

  21. [29]

    RangeNet ++: Fast and Accurate LiDAR Semantic Segmentation,

    A. Milioto, I. Vizzo, J. Behley, and C. Stachniss, “RangeNet ++: Fast and Accurate LiDAR Semantic Segmentation,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . Macau, China: IEEE, Nov. 2019, pp. 4213–4220. [Online]. Available: https://ieee...

  22. [30]

    SalsaNext: Fast, 7 Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving,

    T. Cortinhal, G. Tzelepis, and E. E. Aksoy, “SalsaNext: Fast, 7 Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving,” July 2020, arXiv:2003.03653 [cs]. [Online]. Available: http://arxiv.org/abs/2003.03653

  23. [31]

    PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation,

    Y . Zhang, Z. Zhou, P. David, X. Yue, Z. Xi, B. Gong, and H. Foroosh, “PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Seattle, W A, USA: IEEE, June ...

  24. [32]

    Rethinking Range View Representation for LiDAR Segmentation,

    L. Kong, Y . Liu, R. Chen, Y . Ma, X. Zhu, Y . Li, Y . Hou, Y . Qiao, and Z. Liu, “Rethinking Range View Representation for LiDAR Segmentation,” in 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Paris, France: IEEE, Oct. 2023, pp. 228–240, arXiv:2303.05367 ...

  25. [33]

    LMSCNet: Lightweight Multiscale 3D Semantic Completion,

    L. Roldao, R. De Charette, and A. Verroust-Blondet, “LMSCNet: Lightweight Multiscale 3D Semantic Completion,” in 2020 International Conference on 3D Vision (3DV) . Fukuoka, Japan: IEEE, Nov. 2020, pp. 111–119, arXiv:2008.10559 [cs]. [Online]. Available: https://ieeexplore.ieee...

  26. [34]

    S3CNet: A Sparse Semantic Scene Completion Network for LiDAR Point Clouds,

    R. Cheng, C. Agia, Y . Ren, X. Li, and L. Bingbing, “S3CNet: A Sparse Semantic Scene Completion Network for LiDAR Point Clouds,” Dec. 2020, arXiv:2012.09242 [cs]. [Online]. Available: http://arxiv.org/abs/2012.09242

  27. [35]

    PointOcc: Cylindrical Tri-Perspective View for Point-based 3D Semantic Occupancy Prediction,

    S. Zuo, W. Zheng, Y . Huang, J. Zhou, and J. Lu, “PointOcc: Cylindrical Tri-Perspective View for Point-based 3D Semantic Occupancy Prediction,” Aug. 2023, arXiv:2308.16896 [cs]. [Online]. Available: http://arxiv.org/abs/2308.16896

  28. [36]

    Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction,

    Y . Huang, W. Zheng, Y . Zhang, J. Zhou, and J. Lu, “Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction,” in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Vancouver, BC, Canada: IEEE, June 2023, pp. 9223–9232, arXiv:2302.07817...

  29. [37]

    PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space,

    C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space,” June 2017, arXiv:1706.02413 [cs]. [Online]. Available: http://arxiv.org/abs/1706.02413

  30. [38]

    PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation,

    R. Q. Charles, H. Su, M. Kaichun, and L. J. Guibas, “PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Honolulu, HI: IEEE, July 2017, pp. 77–85, arXiv:1612.00593 [cs]. [Onl...

  31. [39]

    KPConv: Flexible and Deformable Convolution for Point Clouds,

    H. Thomas, C. R. Qi, J.-E. Deschaud, B. Marcotegui, F. Goulette, and L. J. Guibas, “KPConv: Flexible and Deformable Convolution for Point Clouds,” Aug. 2019, arXiv:1904.08889 [cs]. [Online]. Available: http://arxiv.org/abs/1904.08889

  32. [40]

    RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds,

    Q. Hu, B. Yang, L. Xie, S. Rosa, Y . Guo, Z. Wang, N. Trigoni, and A. Markham, “RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, W A, USA: IEEE, June 2020, pp. 11 1...

  33. [41]

    Point Transformer V2: Grouped Vector Attention and Partition-based Pooling,

    X. Wu, Y . Lao, L. Jiang, X. Liu, and H. Zhao, “Point Transformer V2: Grouped Vector Attention and Partition-based Pooling,” Oct. 2022, arXiv:2210.05666 [cs]. [Online]. Available: http://arxiv.org/abs/2210.05666

  34. [42]

    (AF) 2 -S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Network,

    R. Cheng, R. Razani, E. Taghavi, E. Li, and B. Liu, “(AF) 2 -S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Network,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Nashville, TN, USA: IEEE, Jun...

  35. [43]

    RPVNet: A Deep and Efficient Range-Point-V oxel Fusion Network for LiDAR Point Cloud Segmentation,

    J. Xu, R. Zhang, J. Dou, Y . Zhu, J. Sun, and S. Pu, “RPVNet: A Deep and Efficient Range-Point-V oxel Fusion Network for LiDAR Point Cloud Segmentation,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Montreal, QC, Canada: IEEE, Oct. 2021, pp. 16 004–16 ...

  36. [44]

    LidarMultiNet: Towards a Unified Multi-Task Network for LiDAR Perception,

    D. Ye, Z. Zhou, W. Chen, Y . Xie, Y . Wang, P. Wang, and H. Foroosh, “LidarMultiNet: Towards a Unified Multi-Task Network for LiDAR Perception,” Mar. 2023, arXiv:2209.09385 [cs]. [Online]. Available: http://arxiv.org/abs/2209.09385

  37. [45]

    LiDAR-Camera Continuous Fusion in V oxelized Grid for Semantic Scene Completion,

    Z. Lu, B. Cao, and Q. Hu, “LiDAR-Camera Continuous Fusion in V oxelized Grid for Semantic Scene Completion,” IEEE Transactions on Circuits and Systems for Video Technology , pp. 1–1, 2024. [Online]. Available: https://ieeexplore.ieee.org/document/10613892/

  38. [46]

    GrowSP: Unsupervised Semantic Segmentation of 3D Point Clouds,

    Z. Zhang, B. Yang, B. Wang, and B. Li, “GrowSP: Unsupervised Semantic Segmentation of 3D Point Clouds,” in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Vancouver, BC, Canada: IEEE, June 2023, pp. 17 619–17 629, arXiv:2305.16404 [cs]. [Online]. A...

  39. [47]

    U3DS3: Unsuper- vised 3D Semantic Scene Segmentation,

    J. Liu, Z. Yu, T. P. Breckon, and H. P. H. Shum, “U3DS3: Unsuper- vised 3D Semantic Scene Segmentation,” arXiv:2311.06018 [cs]

  40. [48]

    PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding,

    S. Xie, J. Gu, D. Guo, C. R. Qi, L. J. Guibas, and O. Litany, “PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding,” Nov. 2020, arXiv:2007.10985 [cs]. [Online]. Available: http://arxiv.org/abs/2007.10985

  41. [49]

    Self-Supervised Pretraining of 3D Features on any Point-Cloud,

    Z. Zhang, R. Girdhar, A. Joulin, and I. Misra, “Self-Supervised Pretraining of 3D Features on any Point-Cloud,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Montreal, QC, Canada: IEEE, Oct. 2021, pp. 10 232–10 243. [Online]. Available: https://ieeexplo...

  42. [50]

    SegContrast: 3D Point Cloud Feature Representation Learning Through Self-Supervised Segment Discrimination,

    L. Nunes, R. Marcuzzi, X. Chen, J. Behley, and C. Stachniss, “SegContrast: 3D Point Cloud Feature Representation Learning Through Self-Supervised Segment Discrimination,” IEEE Robotics and Automation Letters , vol. 7, no. 2, pp. 2116–2123, Apr. 2022. [Online]. Available: https...

  43. [51]

    S4C: Self-Supervised Semantic Scene Completion With Neural Fields,

    A. Hayler, F. Wimbauer, D. Muhle, C. Rupprecht, and D. Cremers, “S4C: Self-Supervised Semantic Scene Completion With Neural Fields,” in 2024 International Conference on 3D Vision (3DV). Davos, Switzerland: IEEE, Mar. 2024, pp. 409–420, arXiv:2310.07522 [cs]. [Online]. Availabl...

  44. [52]

    OccNeRF: Self-Supervised Multi-Camera Occupancy Prediction with Neural Radiance Fields,

    C. Zhang, J. Yan, Y . Wei, J. Li, L. Liu, Y . Tang, Y . Duan, and J. Lu, “OccNeRF: Self-Supervised Multi-Camera Occupancy Prediction with Neural Radiance Fields,” Dec. 2023, arXiv:2312.09243 [cs]. [Online]. Available: http://arxiv.org/abs/2312.09243

  45. [53]

    Learning 3D Semantic Segmentation with only 2D Image Supervision,

    K. Genova, X. Yin, A. Kundu, C. Pantofaru, F. Cole, A. Sud, B. Brewington, B. Shucker, and T. Funkhouser, “Learning 3D Semantic Segmentation with only 2D Image Supervision,” in 2021 International Conference on 3D Vision (3DV) . London, United Kingdom: IEEE, Dec. 2021, pp. 361–...

  46. [54]

    Real-time multi- modal semantic fusion on unmanned aerial vehicles with label propagation for cross-domain adaptation,

    S. Bultmann, J. Quenzel, and S. Behnke, “Real-time multi- modal semantic fusion on unmanned aerial vehicles with label propagation for cross-domain adaptation,” Robotics and Autonomous Systems, vol. 159, p. 104286, Jan. 2023. [Online]. Available: https://linkinghub.elsevier.co...

  47. [55]

    Image-to-Lidar Self-Supervised Distillation for Autonomous Driving Data,

    C. Sautier, G. Puy, S. Gidaris, A. Boulch, A. Bursuc, and R. Marlet, “Image-to-Lidar Self-Supervised Distillation for Autonomous Driving Data,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . New Orleans, LA, USA: IEEE, June 2022, pp. 9881–9891....

  48. [56]

    Self-Supervised Image-to-Point Distillation via Semantically Tolerant Contrastive Loss,

    A. Mahmoud, J. S. K. Hu, T. Kuai, A. Harakeh, L. Paull, and S. L. Waslander, “Self-Supervised Image-to-Point Distillation via Semantically Tolerant Contrastive Loss,” in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Vancouver, BC, Canada: IEEE, J...

  49. [57]

    Segment Any Point Cloud Sequences by Distilling Vision Foundation Models,

    Y . Liu, L. Kong, J. Cen, R. Chen, W. Zhang, L. Pan, K. Chen, and Z. Liu, “Segment Any Point Cloud Sequences by Distilling Vision Foundation Models,” Oct. 2023, arXiv:2306.09347 [cs]. [Online]. Available: http://arxiv.org/abs/2306.09347

  50. [58]

    PointPainting: Sequential Fusion for 3D Object Detection,

    S. V ora, A. H. Lang, B. Helou, and O. Beijbom, “PointPainting: Sequential Fusion for 3D Object Detection,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, W A, USA: IEEE, June 2020, pp. 4603– 4611, arXiv:1911.10150 [cs, eess, stat]. [On...

  51. [59]

    360° from a Single Camera: A Few-Shot Approach for LiDAR Segmentation,

    L. Reichardt, N. Ebert, and O. Wasenm ¨uller, “360° from a Single Camera: A Few-Shot Approach for LiDAR Segmentation,” in 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) . Paris, France: IEEE, Oct. 2023, pp. 1067–1075, arXiv:2309.06197 [cs]. [Online...

  52. [60]

    Shelf-Supervised 8 Multi-Modal Pre-Training for 3D Object Detection,

    M. Khurana, N. Peri, D. Ramanan, and J. Hays, “Shelf-Supervised 8 Multi-Modal Pre-Training for 3D Object Detection,” June 2024, arXiv:2406.10115 [cs]. [Online]. Available: http://arxiv.org/abs/2406. 10115

  53. [61]

    SAM3D: zero-shot 3D object detection via the segment anything model,

    D. Zhang, D. Liang, H. Yang, Z. Zou, X. Ye, Z. Liu, and X. Bai, “SAM3D: zero-shot 3D object detection via the segment anything model,” Science China Information Sciences , vol. 67, no. 4, p. 149101, Mar. 2024, arXiv:2306.02245 [cs, eess]. [Online]. Available: http://arxiv.org/...

  54. [62]

    Unsupervised 3D Perception with 2D Vision- Language Distillation for Autonomous Driving,

    M. Najibi, J. Ji, Y . Zhou, C. R. Qi, X. Yan, S. Ettinger, and D. Anguelov, “Unsupervised 3D Perception with 2D Vision- Language Distillation for Autonomous Driving,” in 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Paris, France: IEEE, Oct. 2023, pp. 8568...

  55. [63]

    OpenAnnotate3D: Open-V ocabulary Auto-Labeling System for Multi-modal 3D Data,

    Y . Zhou, L. Cai, X. Cheng, Z. Gan, X. Xue, and W. Ding, “OpenAnnotate3D: Open-V ocabulary Auto-Labeling System for Multi-modal 3D Data,” in 2024 IEEE International Conference on Robotics and Automation (ICRA) . Yokohama, Japan: IEEE, May 2024, pp. 9086–9092. [Online]. Availab...

  56. [64]

    OpenAnnotate2: Multi-Modal Auto-Annotating for Autonomous Driving,

    Y . Zhou, L. Cai, X. Cheng, Q. Zhang, X. Xue, W. Ding, and J. Pu, “OpenAnnotate2: Multi-Modal Auto-Annotating for Autonomous Driving,” IEEE Transactions on Intelligent V ehicles , pp. 1–13, 2024. [Online]. Available: https://ieeexplore.ieee.org/document/10480248/

  57. [65]

    SemanticFusion: Dense 3D semantic mapping with convolutional neural networks,

    J. McCormac, A. Handa, A. Davison, and S. Leutenegger, “SemanticFusion: Dense 3D semantic mapping with convolutional neural networks,” in 2017 IEEE International Conference on Robotics and Automation (ICRA) . Singapore, Singapore: IEEE, May 2017, pp. 4628–4635, arXiv:1609.0513...

  58. [66]

    Real-time 3D reconstruction at scale using voxel hashing,

    M. Nießner, M. Zollh ¨ofer, S. Izadi, and M. Stamminger, “Real-time 3D reconstruction at scale using voxel hashing,” ACM Transactions on Graphics, vol. 32, no. 6, pp. 1–11, Nov. 2013. [Online]. Available: https://dl.acm.org/doi/10.1145/2508363.2508374

  59. [67]

    Distilling the Knowledge in a Neural Network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the Knowledge in a Neural Network,” Mar. 2015, arXiv:1503.02531 [cs, stat]. [Online]. Available: http://arxiv.org/abs/1503.02531

  60. [68]

    Learning to Detect Mobile Objects from LiDAR Scans Without Labels,

    Y . You, K. Luo, C. P. Phoo, W.-L. Chao, W. Sun, B. Hariharan, M. Campbell, and K. Q. Weinberger, “Learning to Detect Mobile Objects from LiDAR Scans Without Labels,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . New Orleans, LA, USA: IEEE, Ju...

  61. [69]

    Label-Efficient 3D Object Detection For Road-Side Units,

    M.-Q. Dao, H. Caesar, J. S. Berrio, M. Shan, S. Worrall, V . Fr ´emont, and E. Malis, “Label-Efficient 3D Object Detection For Road-Side Units,” Apr. 2024, arXiv:2404.06256 [cs]. [Online]. Available: http://arxiv.org/abs/2404.06256

  62. [70]

    Using a Waffle Iron for Automotive Point Cloud Semantic Segmentation,

    G. Puy, A. Boulch, and R. Marlet, “Using a Waffle Iron for Automotive Point Cloud Semantic Segmentation,” in 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Paris, France: IEEE, Oct. 2023, pp. 3356–3366, arXiv:2301.10100 [cs]. [Online]. Available: https://ie...

  63. [71]

    AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles,

    S. Shah, D. Dey, C. Lovett, and A. Kapoor, “AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles,” July 2017, arXiv:1705.05065 [cs]. [Online]. Available: http://arxiv.org/abs/1705. 05065

  64. [72]

    Unsupervised Domain Adaptation in LiDAR Semantic Segmentation with Self-Supervision and Gated Adapters,

    M. Rochan, S. Aich, E. R. Corral-Soto, A. Nabatchian, and B. Liu, “Unsupervised Domain Adaptation in LiDAR Semantic Segmentation with Self-Supervision and Gated Adapters,” in 2022 International Conference on Robotics and Automation (ICRA) . Philadelphia, PA, USA: IEEE, May 202...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.