Pith. sign in

REVIEW 2 major objections 4 minor 54 references

VisionPAD: A Vision-Centric Pre-training Paradigm for Autonomous Driving

T0 review · 2 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper shows that image-only pre-training with a 3D-GS decoder and two self-supervised losses—voxel velocity estimation and photometric consistency—outperforms LiDAR-supervised rendering pre-training on 3D detection, occupancy, and…

desk verdict Solid pre-training recipe with 3D-GS and photometric consistency, but the velocity-estimation mechanism as described cannot explain its own ablation gains. read the letter →

arxiv 2411.14716 v2 pith:TNBCEZW3 submitted 2024-11-22 cs.CV cs.LGcs.RO

Haiming Zhang , Wending Zhou , Yiyao Zhu , Xu Yan , Jiantao Gao , Dongfeng Bai , Yingjie Cai , Bingbing Liu
show 2 more authors
Shuguang Cui Zhen Li
This is my paper · ORCID
classification cs.CVcs.LGcs.RO
keywords self-supervisedpre-training3DGaussianSplattingvision-centricautonomousdrivingphotometricconsistencyvoxelvelocityestimationoccupancypredictionobjectdetectionmapsegmentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

VisionPAD tries to show that pre-training for vision-centric autonomous driving can be done with nothing but multi-frame, multi-view camera images, removing the field's reliance on LiDAR depth supervision. It replaces volumetric neural rendering with an anchor-based 3D Gaussian Splatting decoder that renders full-resolution images and depth maps from voxel features, then adds two image-only self-supervised tasks: warping voxels to adjacent frames to learn velocity, and a photometric consistency loss that re-projects neighbor views through the rendered depth. On nuScenes, pre-training this way improves 3D object detection by +2.5 mAP, semantic occupancy prediction by +4.5 mIoU, and map segmentation by +4.1 IoU over the re-implemented UniPAD pre-training baseline. The paper argues this makes camera-based pre-training cheaper, faster, and more scalable.

What carries the argument

The load-bearing object is the anchor-based 3D Gaussian Splatting decoder: each voxel center predicts a set of Gaussian primitives (position offset, opacity, scale, rotation, spherical-harmonic coefficients) via MLPs, and differentiable splat rasterization renders full multi-view images and alpha-blended depth maps. This decoder enables all three losses: the RGB reconstruction loss on the current frame, the velocity-guided warping loss on adjacent frames, and the photometric consistency loss that uses the rendered depth as the geometric bridge for re-projection. Compared with NeRF-style ray sampling, 3D-GS renders the whole image at once, which the paper says is why photometric consistency can provide dense supervision efficiently.

What would settle it

Measure the rendered depth maps against LiDAR on held-out nuScenes frames: if the rendered depths show high error (e.g., relative error greater than 10%) in static scenes while the photometric loss is low, the loss is being minimized without metric geometry, contradicting the paper's attribution of the gains to geometric learning.

Watch

Extended reading notes

Core claim

The paper's central claim is that a 3D-GS-based decoder supervised purely by RGB images can pre-train voxel representations for autonomous driving better than volume rendering that uses explicit LiDAR depth. Specifically, the photometric consistency term—re-projecting source frames into the target view using the Gaussian-rendered depth map and minimizing a perceptual plus L1 loss—is the biggest single contributor, adding +2.4 NDS and +4.4 mAP over the model without it, while the self-supervised voxel velocity estimation adds the motion cue (+1.2 mAP) and the opacity-based Gaussian filtering improves efficiency with a small gain. The authors also show that when UniPAD is restricted to image supervision it degrades performance, whereas VisionPAD improves it, attributing the difference to full-image splat rendering and the two auxiliary tasks.

Load-bearing premise

The method's weakest point is the assumption that the depths rendered by the Gaussian decoder are trustworthy enough for the photometric consistency loss to teach real geometry rather than being minimized by a degenerate, appearance-only arrangement of Gaussians.

Editorial extensions

If this is right

  • Camera-only pre-training can be scaled to any fleet with surround cameras, since no LiDAR sweeps or depth labels are needed during pre-training.
  • The same 3D-GS decoder and photometric-consistency loss should transfer to other voxel-based perception backbones; the paper shows an example with BEVDet for occupancy, where pre-training raises mIoU from 39.3% to 42.0%.
  • Pre-training becomes a data-efficiency lever: with only 25-50% of fine-tuning labels, the relative gains over the baseline grow to roughly +6 mAP, so the method may reduce annotation cost.
  • Because 3D-GS rendering uses about 93% less memory and 92% less latency than the volume-rendering baseline at the same resolution, large-scale pre-training becomes more practical.
  • The voxel-velocity head produces a motion signal that separates dynamic from static voxels, which should benefit downstream tracking and motion forecasting as well as detection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the authors do not run: validate the learned representation's metric quality by probing rendered depths against LiDAR on the pre-training set; this would test whether photometric consistency yields genuine geometry or a view-synthesis shortcut.
  • The method's design suggests a route to world models: the velocity-guided voxel warping could be iterated over longer horizons to predict future occupancy or BEV features, connecting pre-training to prediction and planning.
  • Because photometric consistency contributes most of the gain, combining 3D-GS pre-training with weak depth priors (e.g., pretrained monocular depth) could push further—though this reintroduces some supervision the paper aims to remove.
  • The opacity-based Gaussian filtering is a pragmatic inductive bias; one could test whether adaptive pruning of Gaussians during fine-tuning changes transfer, or whether the velocity head also helps when fine-tuned, not just pre-trained.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. VisionPAD proposes a self-supervised pre-training method for vision-centric autonomous driving that replaces volume rendering with a 3D Gaussian Splatting (3D-GS) decoder, adds a self-supervised voxel velocity estimation head, and applies a multi-frame photometric consistency loss, using only RGB images as supervision. On nuScenes, the paper reports gains over UVTR/UniPAD baselines on 3D detection (e.g., +2.5 mAP in the CS setting of Table 1), semantic occupancy prediction (approximately +5.4 mIoU in Table 2), and map segmentation (+5.4 IoU in Table 3), together with a large reduction in decoder memory and latency (Table 6). The paper includes ablations of the proposed components, data-efficiency experiments, and an efficiency comparison.

Significance. The empirical contribution is potentially significant: image-only pre-training with a 3D-GS decoder is considerably cheaper than NeRF-based UniPAD and still yields consistent improvements across three downstream tasks. The photometric consistency ablation (Model D in Table 4) indicates that cross-frame image supervision carries substantial signal, and the data-efficiency results in Fig. 4 are a strength. However, the stated gradient rule for the velocity head is internally inconsistent with the attributed downstream gains, and the rendered depths used for photometric consistency are not quantitatively validated. These issues bear directly on the claimed mechanisms, though they do not by themselves invalidate the full-model empirical result.

major comments (2)
  1. [Sec. 3.4, Table 4] The paper states in Sec. 3.4 that during backpropagation of the velocity estimation loss only the parameters of the velocity head are updated, and the velocity head is not part of the downstream UVTR/BEVDet models. Under this rule, L_vel cannot change the voxel features, image backbone, lifting network, or 3D-GS decoder, so the pre-trained weights should be identical with or without the velocity head; the +1.2 mAP improvement of Model C over Model B in Table 4 is therefore unexplained. Please either specify the actual gradient path (e.g., gradients reaching V_t through the GridSample operator), correct the description, or add a control experiment that removes the velocity head after pre-training and shows the gain persists.
  2. [Sec. 3.5, Table 4] The photometric consistency loss uses rendered depths D_t from the 3D-GS decoder as the geometric link for re-projecting adjacent frames. The paper does not quantitatively evaluate these rendered depths (e.g., against LiDAR or SfM), so it is not established that the +2.4 NDS and +4.4 mAP gain attributed to P.C. in Table 4 comes from improved geometry rather than from a degenerate depth solution that happens to reduce the photometric error. Reporting depth error metrics or an explicit geometric consistency check would make the mechanism credible.
minor comments (4)
  1. [Abstract, Introduction, Tables 2-3] The performance gains are reported inconsistently: the abstract says +5.4 mIoU occupancy and +5.4 lane IoU, while the introduction says +4.5 mIoU and +4.1 IoU; Table 2 shows +5.3 mIoU over UVTR and +4.4 mIoU over UniPAD, and Table 3 shows +5.4 lane IoU. Please harmonize these numbers.
  2. [Sec. 3.3] The Gaussian filtering rule uses a tanh activation and discards Gaussians with predicted opacity less than 0; please state how the surviving tanh outputs are mapped into the [0,1] opacity values used in Eq. (3), since tanh outputs can be negative.
  3. [Appendix A.2] Algorithm 1 uses inconsistent notation for the predicted flow (F_t in the comment and F_f in the formula); please unify the notation.
  4. [Sec. 3.6, Eq. (8)] The loss weights are fixed at 0.5, 1, and 1 without a sensitivity analysis; a small ablation over these weights would increase confidence that the reported improvements are not sensitive to the specific choices.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: VisionPAD's pre-training objectives are external-image reconstruction and consistency losses; the main flagged issue is an attribution gap, not a circular step.

full rationale

VisionPAD is an empirical pre-training paper; its headline gains are benchmark results, not conclusions derived from an equation. The three pre-training terms in Eq. (8) — L_img, L_vel, and L_pc — are standard reconstruction and consistency losses supervised by held-out multi-view images of adjacent frames plus known ego poses. None of Eqs. (5)–(8) collapses into another by construction: L_pc uses the rendered depth D_t = 3DGS(V_t, K_t, T_t) as a sampling map, but the supervision signal is the source image I_t′, which is external to the rendered target; this is the standard self-supervised photometric-consistency objective, not a tautology. The 3D-GS decoder and anchor MLPs are adopted from external work (Kerbl et al.; Charatan et al.), and the only author self-citations ([43], [50]) are contextual survey or prior-work mentions and are not load-bearing. The one substantive concern is an attribution gap, not circularity: Sec. 3.4 states that “during backpropagation, only the parameters of the velocity head are updated,” and that head is not part of the downstream detectors, so the +1.2 mAP / +3.2 NDS ablation gain credited to V.V.Est. in Table 4 is mechanistically unexplained unless unintended gradients reach V_t. That is a correctness and implementation risk and should not be scored as circular. Likewise, the unvalidated quality of the rendered depths feeding L_pc is an empirical risk, not a definitional reduction. Verdict: no significant circularity; score 2 reflects the minor non-load-bearing self-citation and the flagged attribution gap rather than any circular derivation.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method is empirical; no new physical entities or theoretical parameters are introduced. The main free choices are loss weights and Gaussian primitive bounds. The domain assumptions concern pose accuracy, the informativeness of 3D-GS rendering, and transferability of self-supervised objectives.

free parameters (3)
  • Loss weights omega1, omega2, omega3 = 0.5, 1, 1
    Hand-chosen weights in Eq. (8) for the total pre-training loss; no sensitivity analysis is shown, and downstream performance depends on this weighting.
  • Gaussian offset scaling factor s = 0.25
    Empirically set in Eq. (11) (Appendix B); the authors report that 0.5 gives no obvious change, but the value is still chosen by hand.
  • Gaussian scale bounds [sl, su] = [0.1, 0.5]
    Learnable scale range in Eq. (12); the alternative [0.2, 0.8] slightly hurts performance, so the chosen bounds are tuned.
assumptions (4)
  • domain assumption nuScenes provides accurate camera poses, intrinsics, and extrinsics for cross-frame warping and photometric re-projection.
    Used in Sec. 3.4 and Eq. (5); inaccurate poses would corrupt velocity-guided warping and the photometric consistency loss.
  • domain assumption 3D Gaussian Splatting, with anchor-based prediction, renders images and depth maps informative enough to drive representation learning.
    The entire pre-training relies on this differentiable rendering proxy; no quantitative depth evaluation against LiDAR is provided.
  • domain assumption Image reconstruction and photometric consistency objectives transfer to downstream 3D detection, occupancy, and map segmentation.
    The pre-training-to-fine-tuning transfer is assumed and validated only empirically on nuScenes.
  • standard math Alpha-blending and Gaussian projection equations from prior work are correct.
    Eqs. (1)-(4) are background from 3D-GS [16] and NeRF [27]; not re-derived in this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VisionPAD: A Vision-Centric Pre-training Paradigm for Autonomous Driving." pith.science (2026). https://pith.science/paper/TNBCEZW3

@misc{pith2026241114716,
  author       = {Pith},
  title        = {Pith review of: VisionPAD: A Vision-Centric Pre-training Paradigm for Autonomous Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TNBCEZW3}},
  note         = {Machine review of arXiv:2411.14716}
}
read the original abstract

This paper introduces VisionPAD, a novel self-supervised pre-training paradigm designed for vision-centric algorithms in autonomous driving. In contrast to previous approaches that employ neural rendering with explicit depth supervision, VisionPAD utilizes more efficient 3D Gaussian Splatting to reconstruct multi-view representations using only images as supervision. Specifically, we introduce a self-supervised method for voxel velocity estimation. By warping voxels to adjacent frames and supervising the rendered outputs, the model effectively learns motion cues in the sequential data. Furthermore, we adopt a multi-frame photometric consistency approach to enhance geometric perception. It projects adjacent frames to the current frame based on rendered depths and relative poses, boosting the 3D geometric representation through pure image supervision. Extensive experiments on autonomous driving datasets demonstrate that VisionPAD significantly improves performance in 3D object detection, occupancy prediction and map segmentation, surpassing state-of-the-art pre-training strategies by a considerable margin.

Figures

Figures reproduced from arXiv: 2411.14716 by the authors.

Figure 1
Figure 1. Comparison with existing methods. (a) UniPAD em￾ploys volume rendering to reconstruct the multi-view depth maps and images of the current frame, using explicit depth maps for su￾pervision. (b) In contrast, our proposed VisionPAD leverages only multi-frame, multi-view images for supervision, effectively learn￾ing motion and geometric representations through voxel velocity estimation and photometric consistency loss. … view at source ↗
Figure 2
Figure 2. Overall pipeline of VisionPAD. Taking a vision-centric perception model as the backbone, VisionPAD leverages multi-frame, multi-view images as input, generating explicit voxel representations. After that, a 3D Gaussian Splatting (3DGS) Decoder reconstructs multi-view images from the voxel features. After that, velocity-guided voxel warp is applied to warp current frame voxel features to adjacent frames, enabling sel… view at source ↗
Figure 3
Figure 3. Self-supervise velocity estimation. Current voxel fea￾tures are warped to the adjacent frame. Subsequently, multi-view images are rendered using the 3DGS Decoder and supervised by images captured in that frame. methodology presented in [6]: D(p) = X i∈K diαi i Y−1 j=1 (1 − αj ), (4) where di is the distance from the i-th Gaussian to the cam￾era. Unlike volume rendering [27], 3D-GS enables efficient rendering via spl… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparison of 3D object detection between VisionPAD (top) and UniPAD (bottom) on nuScenes val set. Each predicted object instance is illustrated by a unique colored 3D bounding box. VisionPAD demonstrably mitigates both false positive and false negative det…
Figure 6
Figure 6. Figure 6: Rendering results. Leveraging multi-view images as supervision, VisionPAD demonstrates compelling depth and im￾age reconstruction after pre-training. training, VisionPAD exhibits promising depth and image reconstruction capabilities. 5. Conclusion This paper introduces…
Figure 7
Figure 7. Figure 7: Visualization of 3D object detection and 3D semantic occupancy prediction results. Each detected object instance is depicted by [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 36 canonical work pages

  1. [1]

    ALSO: automotive lidar self- supervision by occupancy estimation

    Alexandre Boulch, Corentin Sautier, Bj ¨orn Michele, Gilles Puy, and Renaud Marlet. ALSO: automotive lidar self- supervision by occupancy estimation. In CVPR, 2023. 2

    work page 2023

  2. [2]

    Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Gi- ancarlo Baldan, and Oscar Beijbom

    Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Gi- ancarlo Baldan, and Oscar Beijbom. nuscenes: A multi- modal dataset for autonomous driving. In CVPR, 2020. 1, 2, 5

    work page 2020

  3. [3]

    Gaussianbev: 3d gaussian representation meets percep- tion models for bev segmentation

    Florian Chabot, Nicolas Granger, and Guillaume Lapouge. Gaussianbev: 3d gaussian representation meets percep- tion models for bev segmentation. arXiv preprint arXiv: 2407.14108, 2024. 3

    arXiv 2024

  4. [4]

    pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction

    David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann. pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19457–19467, 2024. 4

    work page 2024

  5. [5]

    Periodic vibration gaussian: Dynamic urban scene reconstruction and real-time rendering

    Yurui Chen, Chun Gu, Junzhe Jiang, Xiatian Zhu, and Li Zhang. Periodic vibration gaussian: Dynamic urban scene reconstruction and real-time rendering. arXiv preprint arXiv:2311.18561, 2023. 2

    arXiv 2023

  6. [6]

    Gaussianpro: 3d gaussian splatting with progressive propagation

    Kai Cheng, Xiaoxiao Long, Kaizhi Yang, Yao Yao, Wei Yin, Yuexin Ma, Wenping Wang, and Xuejin Chen. Gaussianpro: 3d gaussian splatting with progressive propagation. InForty- first International Conference on Machine Learning, 2024. 4

    2024

  7. [7]

    MMDetection3D: Open- MMLab next-generation platform for general 3D object detection

    MMDetection3D Contributors. MMDetection3D: Open- MMLab next-generation platform for general 3D object detection. https://github.com/open- mmlab/ mmdetection3d, 2020. 5

    2020

  8. [8]

    Simon Doll, Richard Schulz, Lukas Schneider, Viviane Ben- zin, Markus Enzweiler, and Hendrik P. A. Lensch. Spa- tialdetr: Robust scalable transformer-based 3d object de- tection from multi-view camera images with global cross- sensor attention. In ECCV, 2022. 5

    work page 2022

Show all 54 references
  1. [9]

    Gaussianocc: Fully self-supervised and effi- cient 3d occupancy estimation with gaussian splatting.arXiv preprint arXiv: 2408.11447, 2024

    Wanshui Gan, Fang Liu, Hongbin Xu, Ningkai Mo, and Naoto Yokoya. Gaussianocc: Fully self-supervised and effi- cient 3d occupancy estimation with gaussian splatting.arXiv preprint arXiv: 2408.11447, 2024. 3

    2024 arXiv

  2. [10]

    Digging into self-supervised monocular depth estimation

    Cl ´ement Godard, Oisin Mac Aodha, Michael Firman, and Gabriel J Brostow. Digging into self-supervised monocular depth estimation. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 3828–3838,

  3. [11]

    Planning-oriented autonomous driv- ing

    Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, Lewei Lu, Xiaosong Jia, Qiang Liu, Jifeng Dai, Yu Qiao, and Hongyang Li. Planning-oriented autonomous driv- ing. In CVPR, 2023. 2, 6, 7

    2023

  4. [12]

    Bevdet: High-performance multi-camera 3d object detection in bird-eye-view

    Junjie Huang, Guan Huang, Zheng Zhu, and Dalong Du. Bevdet: High-performance multi-camera 3d object detection in bird-eye-view. CoRR, abs/2112.11790, 2021. 1, 3, 6

    2021 arXiv

  5. [13]

    Tri-perspective view for vision- based 3d semantic occupancy prediction

    Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. Tri-perspective view for vision- based 3d semantic occupancy prediction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9223–9232, 2023. 6, 1

    2023

  6. [14]

    Gaussianformer: Scene as gaussians for vision-based 3d semantic occupancy prediction

    Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. Gaussianformer: Scene as gaussians for vision-based 3d semantic occupancy prediction. arXiv preprint arXiv:2405.17429, 2024. 3

    2024 arXiv

  7. [15]

    Nerf-mae: Masked autoencoders for self-supervised 3d representation learning for neural radiance fields

    Muhammad Zubair Irshad, Sergey Zakharov, Vitor Guizilini, Adrien Gaidon, Zsolt Kira, and Rares Ambrus. Nerf-mae: Masked autoencoders for self-supervised 3d representation learning for neural radiance fields. In European Conference on Computer Vision, pages 434–453. Springer, 2024. 2

    2024

  8. [16]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1,

  9. [17]

    Maeli: Masked autoencoder for large-scale lidar point clouds

    Georg Krispel, David Schinagl, Christian Fruhwirth- Reisinger, Horst Possegger, and Horst Bischof. Maeli: Masked autoencoder for large-scale lidar point clouds. In Proceedings of the IEEE/CVF Winter Conference on Appli- cations of Computer Vision, pages 3383–3392, 2024. 2

    2024

  10. [18]

    Unifying voxel-based representation with transformer for 3d object detection

    Yanwei Li, Yilun Chen, Xiaojuan Qi, Zeming Li, Jian Sun, and Jiaya Jia. Unifying voxel-based representation with transformer for 3d object detection. In NeurIPS, 2022. 5, 6, 7, 8, 3

    2022

  11. [19]

    Bevdepth: Acquisition of reliable depth for multi-view 3d object detec- tion

    Yinhao Li, Zheng Ge, Guanyi Yu, Jinrong Yang, Zengran Wang, Yukang Shi, Jianjian Sun, and Zeming Li. Bevdepth: Acquisition of reliable depth for multi-view 3d object detec- tion. In AAAI, 2023. 1

    2023

  12. [20]

    Simipu: Simple 2d image and 3d point cloud un- supervised pre-training for spatial-aware visual representa- tions

    Zhenyu Li, Zehui Chen, Ang Li, Liangji Fang, Qinhong Jiang, Xianming Liu, Junjun Jiang, Bolei Zhou, and Hang Zhao. Simipu: Simple 2d image and 3d point cloud un- supervised pre-training for spatial-aware visual representa- tions. In Proceedings of the AAAI Conference on Artifi...

    2022

  13. [21]

    Bevformer: 9 Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers

    Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chong- hao Sima, Tong Lu, Yu Qiao, and Jifeng Dai. Bevformer: 9 Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers. In ECCV, 2022. 1, 3, 5, 6

    2022

  14. [22]

    Fb-occ: 3d occupancy prediction based on forward-backward view transformation

    Zhiqi Li, Zhiding Yu, David Austin, Mingsheng Fang, Shiyi Lan, Jan Kautz, and Jose M Alvarez. Fb-occ: 3d occupancy prediction based on forward-backward view transformation. CVPRW, 2023. 6, 1

    2023

  15. [23]

    Fully sparse 3d occupancy prediction

    Haisong Liu, Yang Chen, Haiguang Wang, Zetong Yang, Tianyu Li, Jia Zeng, Li Chen, Hongyang Li, and Limin Wang. Fully sparse 3d occupancy prediction. In ECCV,

  16. [24]

    PETR: position embedding transformation for multi-view 3d object detection

    Yingfei Liu, Tiancai Wang, Xiangyu Zhang, and Jian Sun. PETR: position embedding transformation for multi-view 3d object detection. In ECCV, 2022. 5

    2022

  17. [25]

    A convnet for the 2020s

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feicht- enhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In CVPR, 2022. 5

    2022

  18. [26]

    Learning ego 3d representation as ray tracing

    Jiachen Lu, Zheyuan Zhou, Xiatian Zhu, Hang Xu, and Li Zhang. Learning ego 3d representation as ray tracing. In ECCV, 2022. 5

    2022

  19. [27]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. In ECCV, 2020. 2, 3, 4

    2020

  20. [28]

    V oxel-mae: Masked autoencoders for pre-training large-scale point clouds

    Chen Min, Dawei Zhao, Liang Xiao, Yiming Nie, and Bin Dai. V oxel-mae: Masked autoencoders for pre-training large-scale point clouds. CoRR, abs/2206.09900, 2022. 2

    2022 arXiv

  21. [29]

    Occupancy-mae: Self-supervised pre-training large- scale lidar point clouds with masked occupancy autoen- coders

    Chen Min, Liang Xiao, Dawei Zhao, Yiming Nie, and Bin Dai. Occupancy-mae: Self-supervised pre-training large- scale lidar point clouds with masked occupancy autoen- coders. IEEE Transactions on Intelligent Vehicles, 2023. 1, 2

    2023

  22. [30]

    Segcontrast: 3d point cloud feature representation learning through self-supervised seg- ment discrimination

    Lucas Nunes, Rodrigo Marcuzzi, Xieyuanli Chen, Jens Behley, and Cyrill Stachniss. Segcontrast: 3d point cloud feature representation learning through self-supervised seg- ment discrimination. IEEE Robotics and Automation Letters, 7(2):2116–2123, 2022. 2

    2022

  23. [31]

    Renderocc: Vision-centric 3d occupancy pre- diction with 2d rendering supervision

    Mingjie Pan, Jiaming Liu, Renrui Zhang, Peixiang Huang, Xiaoqi Li, Hongwei Xie, Bing Wang, Li Liu, and Shang- hang Zhang. Renderocc: Vision-centric 3d occupancy pre- diction with 2d rendering supervision. In 2024 IEEE Inter- national Conference on Robotics and Automation (ICRA...

    2024

  24. [32]

    Is pseudo-lidar needed for monocular 3d object detection? In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 3142–3152,

    Dennis Park, Rares Ambrus, Vitor Guizilini, Jie Li, and Adrien Gaidon. Is pseudo-lidar needed for monocular 3d object detection? In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 3142–3152,

  25. [33]

    BEVContrast: Self-supervision in bev space for automotive lidar point clouds

    Corentin Sautier, Gilles Puy, Alexandre Boulch, Renaud Marlet, and Vincent Lepetit. BEVContrast: Self-supervision in bev space for automotive lidar point clouds. In Interna- tional Conference on 3D Vision (3DV), 2024. 1, 2

    2024

  26. [34]

    3dppe: 3d point positional encoding for multi-camera 3d ob- ject detection transformers

    Changyong Shu, JIajun Deng, Fisher Yu, and Yifan Liu. 3dppe: 3d point positional encoding for multi-camera 3d ob- ject detection transformers. In ICCV, 2023. 5

    2023

  27. [35]

    Sparseocc: Re- thinking sparse latent representation for vision-based seman- tic occupancy prediction

    Pin Tang, Zhongdao Wang, Guoqing Wang, Jilai Zheng, Xi- angxuan Ren, Bailan Feng, and Chao Ma. Sparseocc: Re- thinking sparse latent representation for vision-based seman- tic occupancy prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogn...

    2024

  28. [36]

    Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving

    Xiaoyu Tian, Tao Jiang, Longfei Yun, Yucheng Mao, Huitong Yang, Yue Wang, Yilun Wang, and Hang Zhao. Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving. Advances in Neural Information Processing Systems, 36, 2024. 5, 1

    2024

  29. [37]

    Scene as occupancy

    Wenwen Tong, Chonghao Sima, Tai Wang, Li Chen, Silei Wu, Hanming Deng, Yi Gu, Lewei Lu, Ping Luo, Dahua Lin, et al. Scene as occupancy. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8406– 8415, 2023. 1, 2

    2023

  30. [38]

    Opus: occupancy prediction using a sparse set

    Jiabao Wang, Zhaojiang Liu, Qiang Meng, Liujiang Yan, Ke Wang, Jie Yang, Wei Liu, Qibin Hou, and Mingming Cheng. Opus: occupancy prediction using a sparse set. In Advances in Neural Information Processing Systems, 2024. 6

    2024

  31. [39]

    Fcos3d: Fully convolutional one-stage monocular 3d object detection

    Tai Wang, Xinge Zhu, Jiangmiao Pang, and Dahua Lin. Fcos3d: Fully convolutional one-stage monocular 3d object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 913–922, 2021. 1, 2

    2021

  32. [40]

    Openoccupancy: A large scale benchmark for surrounding semantic occupancy perception

    Xiaofeng Wang, Zheng Zhu, Wenbo Xu, Yunpeng Zhang, Yi Wei, Xu Chi, Yun Ye, Dalong Du, Jiwen Lu, and Xin- gang Wang. Openoccupancy: A large scale benchmark for surrounding semantic occupancy perception. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sio...

    2023

  33. [41]

    Cross modal transformer via coordinates encoding for 3d object dectection

    Junjie Yan, Yingfei Liu, Jianjian Sun, Fan Jia, Shuailin Li, Tiancai Wang, and Xiangyu Zhang. Cross modal transformer via coordinates encoding for 3d object dectection. In ICCV,

  34. [42]

    Spot: Scalable 3d pre-training via occu- pancy prediction for autonomous driving

    Xiangchao Yan, Runjian Chen, Bo Zhang, Jiakang Yuan, Xinyu Cai, Botian Shi, Wenqi Shao, Junchi Yan, Ping Luo, and Yu Qiao. Spot: Scalable 3d pre-training via occu- pancy prediction for autonomous driving. arXiv preprint arXiv:2309.10527, 2023. 1, 2

    2023 arXiv

  35. [43]

    Forging vision foundation models for autonomous driving: Challenges, methodologies, and opportunities

    Xu Yan, Haiming Zhang, Yingjie Cai, Jingming Guo, We- ichao Qiu, Bin Gao, Kaiqiang Zhou, Yue Zhao, Huan Jin, Jiantao Gao, Zhen Li, Lihui Jiang, Wei Zhang, Hongbo Zhang, Dengxin Dai, and Bingbing Liu. Forging vision foundation models for autonomous driving: Challenges, methodol...

    2024 arXiv

  36. [44]

    Street gaussians for modeling dynamic ur- ban scenes

    Yunzhi Yan, Haotong Lin, Chenxu Zhou, Weijie Wang, Haiyang Sun, Kun Zhan, Xianpeng Lang, Xiaowei Zhou, and Sida Peng. Street gaussians for modeling dynamic ur- ban scenes. arXiv preprint arXiv:2401.01339, 2024. 2

    2024 arXiv

  37. [45]

    Bevformer v2: Adapting modern image backbones to bird’s-eye-view recognition via perspective supervision

    Chenyu Yang, Yuntao Chen, Hao Tian, Chenxin Tao, Xizhou Zhu, Zhaoxiang Zhang, Gao Huang, Hongyang Li, Yu Qiao, Lewei Lu, Jie Zhou, and Jifeng Dai. Bevformer v2: Adapting modern image backbones to bird’s-eye-view recognition via perspective supervision. In CVPR, 2023. 5

    2023

  38. [46]

    Unipad: A universal pre-training paradigm for autonomous driving

    Honghui Yang, Sha Zhang, Di Huang, Xiaoyang Wu, Haoyi Zhu, Tong He, Shixiang Tang, Hengshuang Zhao, Qibo Qiu, Binbin Lin, et al. Unipad: A universal pre-training paradigm for autonomous driving. In Proceedings of the IEEE/CVF 10 Conference on Computer Vision and Pattern Recogn...

    2024

  39. [47]

    Visual point cloud forecasting enables scalable autonomous driving

    Zetong Yang, Li Chen, Yanan Sun, and Hongyang Li. Visual point cloud forecasting enables scalable autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 2

    2024

  40. [48]

    Ad-pt: Autonomous driving pre-training with large-scale point cloud dataset

    Jiakang Yuan, Bo Zhang, Xiangchao Yan, Botian Shi, Tao Chen, Yikang Li, and Yu Qiao. Ad-pt: Autonomous driving pre-training with large-scale point cloud dataset. Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  41. [49]

    Bevworld: A multimodal world model for au- tonomous driving via unified bev latent space.arXiv preprint arXiv:2407.05679, 2024

    Zhang Yumeng, Gong Shi, Xiong Kaixin, Ye Xiaoqing, Tan Xiao, Wang Fan, Huang Jizhou, Wu Hua, and Wang Haifeng. Bevworld: A multimodal world model for au- tonomous driving via unified bev latent space.arXiv preprint arXiv:2407.05679, 2024. 2

    2024 arXiv

  42. [50]

    Radocc: Learning cross-modality occupancy knowledge through ren- dering assisted distillation

    Haiming Zhang, Xu Yan, Dongfeng Bai, Jiantao Gao, Pan Wang, Bingbing Liu, Shuguang Cui, and Zhen Li. Radocc: Learning cross-modality occupancy knowledge through ren- dering assisted distillation. AAAI Conference on Artificial Intelligence, 2023. 1

    2023

  43. [51]

    Hugs: Holistic urban 3d scene understanding via gaus- sian splatting

    Hongyu Zhou, Jiahao Shao, Lu Xu, Dongfeng Bai, Weichao Qiu, Bingbing Liu, Yue Wang, Andreas Geiger, and Yiyi Liao. Hugs: Holistic urban 3d scene understanding via gaus- sian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages ...

    2024

  44. [52]

    Drivinggaussian: Composite gaussian splatting for surrounding dynamic au- tonomous driving scenes

    Xiaoyu Zhou, Zhiwei Lin, Xiaojun Shan, Yongtao Wang, Deqing Sun, and Ming-Hsuan Yang. Drivinggaussian: Composite gaussian splatting for surrounding dynamic au- tonomous driving scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages...

    2024

  45. [53]

    Class-balanced grouping and sampling for point cloud 3d object detection

    Benjin Zhu, Zhengkai Jiang, Xiangxin Zhou, Zeming Li, and Gang Yu. Class-balanced grouping and sampling for point cloud 3d object detection. arXiv preprint arXiv:1908.09492,

    1908 arXiv

  46. [54]

    Mim4d: Masked modeling with multi-view video for autonomous driving representation learning

    Jialv Zou, Bencheng Liao, Qian Zhang, Wenyu Liu, and Xinggang Wang. Mim4d: Masked modeling with multi-view video for autonomous driving representation learning. arXiv preprint arXiv:2403.08760, 2024. 2 11 Appendix A. Additional Implementation Details In this section, we elabor...

    2024 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.