Pith. sign in

REVIEW 5 major objections 5 minor 61 references

ODG: Occupancy Prediction Using Dual Gaussians

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ODG represents a driving scene as two separate sets of 3D Gaussians, one static and one dynamic, and reports new state-of-the-art occupancy prediction results on both Occ3D benchmarks.

desk verdict The dual static/dynamic Gaussian architecture is a genuine contribution, but the headline SOTA claim is confounded by an unmatched 100-epoch training schedule versus OPUS. read the letter →

arxiv 2506.09417 v2 pith:3WKJNW3P submitted 2025-06-11 cs.CV

Yunxiao Shi , Yinhao Zhu , Shizhong Han , Jisoo Jeong , Amin Ansari , Hong Cai , Fatih Porikli This is my paper
classification cs.CV
keywords 3DoccupancypredictionGaussiansplattingsparsequeriesstatic-dynamicdecompositioncoarse-to-finerefinementautonomousdrivingcamera-onlyperceptionOcc3Dbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ODG claims that camera-only 3D occupancy prediction improves when the scene is represented not by one set of sparse queries but by two: one set of 3D Gaussians dedicated to static background and another, augmented with bounding-box and velocity attributes, dedicated to dynamic agents. The paper argues that a single query set spreads capacity too thinly across objects that behave very differently, and that a coarse-to-fine hierarchy of transformer layers lets the model afford far more Gaussians than prior single-transformer designs. It also uses 3D Gaussian splatting to render depth and semantic maps at every stage, supervising those renders with LiDAR-projected labels to inject pixel-level alignment. On Occ3D-nuScenes and Occ3D-Waymo, ODG reports new state-of-the-art mIoU of 38.18 and 21.35, surpassing the previous best by 1.98 and 2.35 points respectively while keeping inference cost competitive. The core claim a sympathetic reader should take away is that static/dynamic decomposition plus multi-stage rendering supervision is the right way to scale sparse occupancy prediction.

What carries the argument

The load-bearing mechanism is the dual Gaussian query representation: two sets of 3D Gaussians, where static queries carry $\{\mu,s,r,\sigma\}$ and dynamic queries append the box-velocity attribute $b=[l,w,h,\theta,v_x,v_y,v_z]$. A hierarchical Gaussian transformer $T_\ell$ refines Gaussian means through coarse-to-fine stages, DaS attention exchanges information between static and dynamic queries, and 3D Gaussian splatting renders depth and semantic maps per camera for multi-stage supervision. These pieces work together to let a sparse set predictor afford a much larger Gaussian count and to make the non-mean Gaussian properties learn from pixel-level 2D labels rather than only from 3D occupancy.

What would settle it

Re-run ODG on Occ3D-nuScenes with the rendering loss term $L_r$ disabled and compare against the full model; if the mIoU drop is not close to the reported +0.69 and RayIoU drop is not close to +0.70, the paper's attribution of accuracy to rendering supervision is falsified. A second check is to replace LiDAR-projected labels with monocular predicted depth and semantics: if performance holds, the dependence on LiDAR-derived labels is not essential.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that driving scenes admit a universal static/dynamic decomposition, and encoding it directly into the query structure pays off. ODG initializes two sets of Gaussian queries, static $\{g^s_{i,k}\}$ and dynamic $\{g^d_{j,k}\}$, where dynamic queries append the box-velocity attribute $b=[l,w,h,\theta,v_x,v_y,v_z]$ to the standard Gaussian parameters $\{\mu,s,r,\sigma\}$. A Dynamic-and-Static (DaS) attention concatenates both query families before self-attention, and a hierarchy of transformer layers refines Gaussian means coarse-to-fine, with $K_{\ell-1}<K_\ell$ Gaussians per query at each stage. The occupancy loss supervises Gaussian means and class scores with Chamfer distance and focal loss, while the rendering loss supervises depth and semantic maps at every stage. The paper's claim is that this combination, not any single component, produces the reported state-of-the-art results, with the largest per-class gains concentrated on dynamic classes such as Bus, Car, Construction Vehicle, Motorcycle, and Truck.

Load-bearing premise

The paper's weakest load-bearing assumption is that the size, orientation, and opacity of each Gaussian can be learned well enough from pixel-level depth and semantic rendering alone, since the occupancy loss supervises only Gaussian positions and class labels; if that rendering signal is too weak or drifts, the claimed gains from splatting supervision would collapse.

Editorial extensions

If this is right

  • If the reported numbers hold, sparse occupancy predictors no longer need dense grids to reach state-of-the-art accuracy on Occ3D-nuScenes and Occ3D-Waymo.
  • Dedicating a separate query set to dynamic agents lifts per-class mIoU for exactly the classes that matter for downstream planning, with ODG-L ahead of OPUS-L by 4.13 mIoU averaged over Bus, Car, Construction Vehicle, Motorcycle, and Truck.
  • Multi-stage rendering supervision buys accuracy without inference cost, because rendering is disabled at test time; the ablation attributes +0.69 mIoU and +0.70 RayIoU to this module.
  • The coarse-to-fine hierarchy removes the single-transformer query-count bottleneck, allowing model capacity to scale with the number of Gaussians while the tiny variant still runs at 20.1 FPS.
  • On Occ3D-Waymo, the largest margins over prior work appear in Vehicle, Bicyclist, and Pedestrian, consistent with dynamic-agent modeling being the decisive factor.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's limitation note points to a natural next step it marks as expensive but unexplored: supervise scale, rotation, and opacity not only through rendering but also by aggregating nearby Gaussians into occupancy and comparing with 3D ground truth.
  • Because dynamic queries already predict velocity and box attributes, the same representation could be extended to jointly output object tracks or short-horizon motion forecasts without changing the architecture.
  • Rendering supervision currently relies on LiDAR-projected depth and semantic labels; a testable extension is to replace those labels with depth and semantics from a monocular prediction network, preserving pixel-level alignment in LiDAR-free settings.
  • If the static/dynamic decomposition is as universal in driving scenes as the paper claims, the dual-query design should transfer with minimal modification to other sparse 3D perception tasks such as panoptic occupancy or online mapping.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes ODG, a camera-based 3D semantic occupancy prediction method that represents the scene with two sets of sparse Gaussian queries, one for static background and one for dynamic agents. Dynamic queries additionally carry 3D box attributes and a velocity vector, and the two query sets interact through a self-attention mechanism over their concatenated features. Gaussians are predicted in a hierarchical coarse-to-fine transformer, with supervision from Chamfer distance and focal loss on occupied voxel centers and class scores, box losses from dynamic queries, and a rendering loss that uses 3D Gaussian splatting to render depth and semantic maps from the predicted Gaussians. Experiments on Occ3D-nuScenes and Occ3D-Waymo report state-of-the-art mIoU and RayIoU numbers, and ablations study motion compensation, query attention, and rendering supervision. The paper is clearly written and the design is well motivated.

Significance. The dual static/dynamic decomposition is a sensible inductive bias for driving scenes, and the coarse-to-fine Gaussian growth is a plausible extension of sparse-query occupancy models. The component-wise ablations on public benchmarks give initial support for each design choice, and the paper honestly discloses in the Limitations section that scale, rotation, and opacity are optimized only through the rendering loss. If the headline gains survive a controlled training-budget comparison and an ablation of the extra supervision signals, ODG would be a meaningful advance for efficient camera-based occupancy prediction. However, the current manuscript does not yet establish that the proposed architecture, rather than longer training and additional 3D box and LiDAR supervision, is responsible for the reported state-of-the-art margins.

major comments (5)
  1. [Sec. 4.1, Tables 1/3/4] The headline SOTA comparisons are not matched in training budget. ODG is trained with a global batch size of 8 for 100 epochs, while the ablations in Sec. 4.4, including the complete ODG-T row in Tab. 4, use 24 epochs. At 24 epochs ODG-T reaches 32.82 mIoU, which is below OPUS-T's 33.2 mIoU in Tab. 1. The training schedules for the OPUS entries in Tabs. 1 and 3 are not disclosed, so the relative contributions of architecture versus longer training to the claimed +2.34 and +1.98 mIoU margins cannot be assessed. Please match schedules, report OPUS numbers under the identical 100-epoch protocol, or provide an ablation that shows the 24-epoch ODG/OPUS comparison; also disclose the OPUS training schedules used in the tables.
  2. [Sec. 3.5, Eqs. (17) and (19)] The method adds two supervision signals, a 3D box loss and a rendering loss using LiDAR-projected depth and semantic labels, that are not used by OPUS. While the rendering supervision itself is ablated in Tab. 4 (+0.69 mIoU), the box loss Lbox is never ablated, so its contribution to the reported gains is unknown. To establish that the dual-Gaussian/hierarchical design, rather than the additional annotations, drives the improvement, please include an ablation without Lbox and, if possible, an ablation that uses the same set of labels as OPUS.
  3. [Sec. 4.1, Table 1] The efficiency claim is not supported by the reported FPS numbers. ODG-L runs at 4.9 FPS while OPUS-L runs at 7.2 FPS, and ODG-T runs at 20.1 FPS versus 22.4 FPS for OPUS-T; both ODG variants are slower. Reporting parameter counts, FLOPs, and memory usage would give a more informative comparison, since FPS alone is hardware- and implementation-dependent. Please either revise the 'low inference cost' claim or provide a more complete efficiency evaluation.
  4. [Sec. 3.2 and Sec. 4.1 (Implementation Details)] The coarse-to-fine growth schedule K_l is never specified. Eq. (4) defines K_l as the number of Gaussians per query at stage ℓ, and Eqs. (9)-(10) assert K_{ℓ-1} < K_ℓ, but the actual values (e.g., K_1 through K_6) and the mechanism that expands the Gaussians between layers are absent from the implementation details. Without this schedule, the hierarchical capacity of the model cannot be reproduced or compared with prior single-stage methods. Please provide the K_l values for all stages and the rule that maps the transformer layers to the increasing number of Gaussians.
  5. [Sec. 4.4] All ablations and the final model are single runs with no variance information. Several differences are small relative to typical seed-to-seed variation (e.g., the mIoU difference between Cross Attn and Self Concat Attn in Tab. 5a is 0.18, and the RayIoU difference in Tab. 5b is 0.5). Adding standard deviations over at least three seeds would make the component-level claims more convincing.
minor comments (5)
  1. [Eq. (4)] In the definition of Gs and Gd, the subscript K_l is used, but K_l is stage-dependent; please clarify that the notation applies per stage ℓ.
  2. [Eqs. (9)-(10)] The layer index ℓ is used both for transformer layers and for stages; clarify the relationship between the L transformer layers and the K_l growth schedule, e.g., whether each layer corresponds to exactly one stage.
  3. [Sec. 3.5, Eq. (16)] The loss includes a Chamfer distance term for the initial Gaussian means G:µ,0, but Sec. 3.2 states that Gaussian means are initialized from U[0,1]; please explain how the initial means are obtained or decoded before the first layer.
  4. [Tab. 5b] The abbreviations 'Ego Comp.' and 'Dyn. Comp.' are not defined in the table caption; please define them as ego-motion compensation and dynamic-object motion compensation.
  5. [Sec. 4.1] The implementation details list the values of S, D, and L but not K_l; please include the exact K_l values to make the coarse-to-fine design reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ODG is an empirical systems paper whose claims are evaluated on public benchmarks, not derived from its inputs.

full rationale

The paper's contributions are architectural and empirical. The dual Gaussian query design, hierarchical coarse-to-fine transformer, and multi-stage rendering supervision are implemented and tested against external ground truth on Occ3D-nuScenes and Occ3D-Waymo. The losses in Eqs. 16-19 supervise means, class scores, boxes, and rendered depth/semantics using occupancy labels, box labels, and LiDAR-projected depth/semantic maps; none of these quantities is fitted to a subset and then relabeled as a prediction of the same quantity. The rendering loss in Eq. 19 is an auxiliary external supervision signal, not a derivation from the occupancy loss, and the ablation quantifies its contribution empirically rather than assuming it. The only self-citation is Ref. [42] (H3O) appearing in a related-work list of heterogeneous-supervision approaches; it is context-only and not load-bearing, and no uniqueness theorem or prior result by the authors is invoked to force the design. The admitted limitation that Gaussian scale, rotation, and opacity are optimized only through rendering loss is an honest statement about optimization coverage, not evidence that the occupancy prediction reduces to those labels by construction. The potential training-schedule mismatch between 24-epoch ablations and 100-epoch main results is a comparison-fairness and correctness concern, not a circularity reduction, and therefore does not affect the circularity score.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim relies on domain assumptions about scene decomposition, the correctness of 3DGS rendering, and one arm of supervision omitting key Gaussian parameters. No new physical or conceptual entities are introduced beyond the architectural design.

free parameters (4)
  • loss weights λ3d and λ = λ3d=0.2; λ=0.05 for stages 1 and 6, 0.01 otherwise
    Hand-set to balance box, occupancy, and rendering losses; no sensitivity analysis reported.
  • query counts S and D = S=500, D=100 (tiny); S=4000, D=800 (large)
    Hand-chosen capacity allocation; no ablation across counts is reported.
  • transformer layers L = L=6
    Hand-chosen depth; no ablation on number of layers.
  • Gaussians per query per stage K_l = not reported
    The coarse-to-fine growth schedule is load-bearing but never specified in the paper, hurting reproducibility.
assumptions (4)
  • domain assumption Driving scenes decompose into static and dynamic parts
    Motivates the dual-query design (Sec. 3.2); if this categorization were not universal, the query split would not be generally beneficial.
  • standard math 3D Gaussian Splatting rasterization correctly renders depth and semantics from predicted Gaussians
    Borrowed from [27]; assumed correct as per the cited work, and used in Eq. 11.
  • standard math Hungarian matching gives stable assignment between predictions and ground truth
    Used in Eq. 17 and Sec. 3.5; standard set-prediction practice, but assignment details are omitted.
  • ad hoc to paper Non-mean Gaussian properties are adequately optimized by the rendering loss
    Stated in the Limitations: only mean and class scores get occupancy supervision; scale, rotation, and opacity rely solely on rendering gradients.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ODG: Occupancy Prediction Using Dual Gaussians." pith.science (2026). https://pith.science/paper/3WKJNW3P

@misc{pith2026250609417,
  author       = {Pith},
  title        = {Pith review of: ODG: Occupancy Prediction Using Dual Gaussians},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3WKJNW3P}},
  note         = {Machine review of arXiv:2506.09417}
}
read the original abstract

Occupancy prediction infers fine-grained 3D geometry and semantics from camera images of the surrounding environment, making it a critical perception task for autonomous driving. Existing methods either adopt dense grids as scene representation, which is difficult to scale to high resolution, or learn the entire scene using a single set of sparse queries, which is insufficient to handle the various object characteristics. In this paper, we present ODG, a hierarchical dual sparse Gaussian representation to effectively capture complex scene dynamics. Building upon the observation that driving scenes can be universally decomposed into static and dynamic counterparts, we define dual Gaussian queries to better model the diverse scene objects. We utilize a hierarchical Gaussian transformer to predict the occupied voxel centers and semantic classes along with the Gaussian parameters. Leveraging the real-time rendering capability of 3D Gaussian Splatting, we also impose rendering supervision with available depth and semantic map annotations injecting pixel-level alignment to boost occupancy learning. Extensive experiments on the Occ3D-nuScenes and Occ3D-Waymo benchmarks demonstrate our proposed method sets new state-of-the-art results while maintaining low inference cost.

Figures

Figures reproduced from arXiv: 2506.09417 by the authors.

Figure 1
Figure 1. Overview of proposed ODG, where we model the dynamic and static elements of the scene with two separate sets of Gaussian queries. Dual-query attention aggregates information across dynamic and static queries. These queries are then decoded into 3D Gaussians, 3D points, as well as 3D bounding boxes (from dynamic queries only), which are supervised by ground-truth depth and semantic maps through rendering, ground-trut… view at source ↗
Figure 2
Figure 2. Visualization of ODG prediction on the Occ3D-nuScenes [ [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Visualization of ODG prediction on the Occ3D-Waymo [48, 43] validation set. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

61 extracted references · 23 canonical work pages

  1. [1]

    Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields

    Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. InProceedings of the IEEE/CVF international conference on computer vision, pages 5855–5864, 2021

    2021

  2. [2]

    Mip-nerf 360: Unbounded anti-aliased neural radiance fields

    Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5470–5479, 2022

    2022

  3. [3]

    Semantickitti: A dataset for semantic scene understanding of lidar sequences

    Jens Behley, Martin Garbade, Andres Milioto, Jan Quenzel, Sven Behnke, Cyrill Stachniss, and Jurgen Gall. Semantickitti: A dataset for semantic scene understanding of lidar sequences. InProceedings of the IEEE/CVF international conference on computer vision, pages 9297–9307, 2019

    work page 2019

  4. [4]

    Gaussianflowocc: Sparse and weakly supervised occupancy estimation using gaussian splatting and temporal flow.arXiv preprint arXiv:2502.17288, 2025

    Simon Boeder, Fabian Gigengack, and Benjamin Risse. Gaussianflowocc: Sparse and weakly supervised occupancy estimation using gaussian splatting and temporal flow.arXiv preprint arXiv:2502.17288, 2025

    arXiv 2025

  5. [5]

    Occflownet: Towards self-supervised occupancy estimation via differentiable rendering and occupancy flow.IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2025

    Simon Boeder, Fabian Gigengack, and Benjamin Risse. Occflownet: Towards self-supervised occupancy estimation via differentiable rendering and occupancy flow.IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2025

    work page 2025

  6. [6]

    nuscenes: A multimodal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621–11631, 2020

    2020

  7. [7]

    Monoscene: Monocular 3d semantic scene completion

    Anh-Quan Cao and Raoul De Charette. Monoscene: Monocular 3d semantic scene completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3991–4001, 2022

    2022

  8. [8]

    Scenerf: Self-supervised monocular 3d scene reconstruction with radiance fields

    Anh-Quan Cao and Raoul de Charette. Scenerf: Self-supervised monocular 3d scene reconstruction with radiance fields. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 9387–9398, 2023

    work page 2023

Show all 61 references
  1. [9]

    End-to-end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. InEuropean conference on computer vision, pages 213–229. Springer, 2020

    2020

  2. [10]

    Gaussrender: Learning 3d occupancy with gaussian rendering.arXiv preprint arXiv:2502.05040, 2025

    Loick Chambon, Eloi Zablocki, Alexandre Boulch, Mickael Chen, and Matthieu Cord. Gaussrender: Learning 3d occupancy with gaussian rendering.arXiv preprint arXiv:2502.05040, 2025

    2025 arXiv

  3. [11]

    pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction

    David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann. pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19457–19467, 2024

    2024

  4. [12]

    Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images

    Yuedong Chen, Haofei Xu, Chuanxia Zheng, Bohan Zhuang, Marc Pollefeys, Andreas Geiger, Tat-Jen Cham, and Jianfei Cai. Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images. In European Conference on Computer Vision, pages 370–386. Springer, 2024

    2024

  5. [13]

    Openscene: The largest up-to-date 3d occupancy prediction benchmark in autonomous driving.https://github.com/OpenDriveLab/OpenScene, 2023

    OpenScene Contributors. Openscene: The largest up-to-date 3d occupancy prediction benchmark in autonomous driving.https://github.com/OpenDriveLab/OpenScene, 2023

    2023

  6. [14]

    An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:...

    2010 arXiv

  7. [15]

    A point set generation network for 3d object reconstruction from a single image

    Haoqiang Fan, Hao Su, and Leonidas J Guibas. A point set generation network for 3d object reconstruction from a single image. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 605–613, 2017

    2017

  8. [16]

    Vision meets robotics: The kitti dataset.The International Journal of Robotics Research, 32(11):1231–1237, 2013

    Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset.The International Journal of Robotics Research, 32(11):1231–1237, 2013

    2013

  9. [17]

    Are we ready for autonomous driving? the kitti vision benchmark suite

    Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In2012 IEEE conference on computer vision and pattern recognition, pages 3354–3361. IEEE, 2012

    2012

  10. [18]

    Sugar: Surface-aligned gaussian splatting for efficient 3d mesh reconstruction and high-quality mesh rendering

    Antoine Guédon and Vincent Lepetit. Sugar: Surface-aligned gaussian splatting for efficient 3d mesh reconstruction and high-quality mesh rendering. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5354–5363, 2024. 10

    2024

  11. [19]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  12. [20]

    Metric3d v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation.arXiv preprint arXiv:2404.15506, 2024

    Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3d v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation.arXiv preprint arXiv:2404.15506, 2024

    2024 arXiv

  13. [21]

    Planning-oriented autonomous driving

    Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al. Planning-oriented autonomous driving. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17853–17862, 2023

    2023

  14. [22]

    Bevdet: High-performance multi-camera 3d object detection in bird-eye-view.arXiv preprint arXiv:2112.11790, 2021

    Junjie Huang, Guan Huang, Zheng Zhu, Yun Ye, and Dalong Du. Bevdet: High-performance multi-camera 3d object detection in bird-eye-view.arXiv preprint arXiv:2112.11790, 2021

    2021 arXiv

  15. [23]

    Selfocc: Self-supervised vision- based 3d occupancy prediction

    Yuanhui Huang, Wenzhao Zheng, Borui Zhang, Jie Zhou, and Jiwen Lu. Selfocc: Self-supervised vision- based 3d occupancy prediction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19946–19956, 2024

    2024

  16. [24]

    Tri-perspective view for vision-based 3d semantic occupancy prediction

    Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. Tri-perspective view for vision-based 3d semantic occupancy prediction. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9223–9232, 2023

    2023

  17. [25]

    Gaussianformer: Scene as gaussians for vision-based 3d semantic occupancy prediction

    Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. Gaussianformer: Scene as gaussians for vision-based 3d semantic occupancy prediction. InEuropean Conference on Computer Vision, pages 376–393. Springer, 2024

    2024

  18. [26]

    Gausstr: Foundation model-aligned gaussian transformer for self-supervised 3d spatial understanding.The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    Haoyi Jiang, Liu Liu, Tianheng Cheng, Xinjie Wang, Tianwei Lin, Zhizhong Su, Wenyu Liu, and Xinggang Wang. Gausstr: Foundation model-aligned gaussian transformer for self-supervised 3d spatial understanding.The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CV...

    2025

  19. [27]

    3d gaussian splatting for real-time radiance field rendering.ACM Trans

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Trans. Graph., 42(4):139–1, 2023

    2023

  20. [28]

    A hierarchical 3d gaussian representation for real-time rendering of very large datasets.ACM Transactions on Graphics (TOG), 43(4):1–15, 2024

    Bernhard Kerbl, Andreas Meuleman, Georgios Kopanas, Michael Wimmer, Alexandre Lanvin, and George Drettakis. A hierarchical 3d gaussian representation for real-time rendering of very large datasets.ACM Transactions on Graphics (TOG), 43(4):1–15, 2024

    2024

  21. [29]

    The hungarian method for the assignment problem.Naval research logistics quarterly, 2(1-2):83–97, 1955

    Harold W Kuhn. The hungarian method for the assignment problem.Naval research logistics quarterly, 2(1-2):83–97, 1955

    1955

  22. [30]

    Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal trans- formers

    Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chonghao Sima, Tong Lu, Yu Qiao, and Jifeng Dai. Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal trans- formers. InEuropean conference on computer vision, pages 1–18. Springer, 2022

    2022

  23. [31]

    Focal loss for dense object detection

    Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. InProceedings of the IEEE International Conference on Computer Vision (ICCV), pages 2999–3007, 2017

    2017

  24. [32]

    Sparse4d: Multi-view 3d object detection with sparse spatial-temporal fusion.arXiv preprint arXiv:2211.10581, 2022

    Xuewu Lin, Tianwei Lin, Zixiang Pei, Lichao Huang, and Zhizhong Su. Sparse4d: Multi-view 3d object detection with sparse spatial-temporal fusion.arXiv preprint arXiv:2211.10581, 2022

    2022 arXiv

  25. [33]

    Sparsebev: High-performance sparse 3d object detection from multi-camera videos

    Haisong Liu, Yao Teng, Tao Lu, Haiguang Wang, and Limin Wang. Sparsebev: High-performance sparse 3d object detection from multi-camera videos. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 18580–18590, 2023

    2023

  26. [34]

    Fully sparse 3d panoptic occupancy prediction

    Haisong Liu, Haiguang Wang, Yang Chen, Zetong Yang, Jia Zeng, Li Chen, and Limin Wang. Fully sparse 3d panoptic occupancy prediction. InProceedings of the European Confernece on Computer Vision, 2024

    2024

  27. [35]

    Let occ flow: Self-supervised 3d occupancy flow prediction.The Conference on Robot Learning (CoRL), 2024

    Yili Liu, Linzhan Mou, Xuan Yu, Chenrui Han, Sitong Mao, Rong Xiong, and Yue Wang. Let occ flow: Self-supervised 3d occupancy flow prediction.The Conference on Robot Learning (CoRL), 2024

    2024

  28. [36]

    Petr: Position embedding transformation for multi-view 3d object detection.arXiv preprint arXiv:2203.05625, 2022

    Yingfei Liu, Tiancai Wang, Xiangyu Zhang, and Jian Sun. Petr: Position embedding transformation for multi-view 3d object detection.arXiv preprint arXiv:2203.05625, 2022

    2022 arXiv

  29. [37]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    I Loshchilov. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  30. [38]

    Sgdr: Stochastic gradient descent with warm restarts.arXiv preprint arXiv:1608.03983, 2016

    Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts.arXiv preprint arXiv:1608.03983, 2016. 11

    2016 arXiv

  31. [39]

    Nerf: Representing scenes as neural radiance fields for view synthesis.Communications of the ACM, 65(1):99–106, 2021

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis.Communications of the ACM, 65(1):99–106, 2021

    2021

  32. [40]

    Renderocc: Vision-centric 3d occupancy prediction with 2d rendering supervision

    Mingjie Pan, Jiaming Liu, Renrui Zhang, Peixiang Huang, Xiaoqi Li, Hongwei Xie, Bing Wang, Li Liu, and Shanghang Zhang. Renderocc: Vision-centric 3d occupancy prediction with 2d rendering supervision. In2024 IEEE International Conference on Robotics and Automation (ICRA), page...

    2024

  33. [41]

    Pytorch: An imperative style, high-performance deep learning library.arXiv preprint arXiv:1912.01703, 2019

    A Paszke. Pytorch: An imperative style, high-performance deep learning library.arXiv preprint arXiv:1912.01703, 2019

    1912 arXiv

  34. [42]

    H3o: Hyper-efficient 3d occupancy prediction with heterogeneous supervision.IEEE International Conference on Robotics and Automation (ICRA), 2025

    Yunxiao Shi, Hong Cai, Amin Ansari, and Fatih Porikli. H3o: Hyper-efficient 3d occupancy prediction with heterogeneous supervision.IEEE International Conference on Robotics and Automation (ICRA), 2025

    2025

  35. [43]

    Scalability in perception for autonomous driving: Waymo open dataset

    Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. InProceedings of the IEEE/CVF conference on computer vis...

    2020

  36. [44]

    Splatter image: Ultra-fast single-view 3d reconstruction

    Stanislaw Szymanowicz, Chrisitian Rupprecht, and Andrea Vedaldi. Splatter image: Ultra-fast single-view 3d reconstruction. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10208–10217, 2024

    2024

  37. [45]

    Sparseocc: Rethinking sparse latent representation for vision-based semantic occupancy prediction

    Pin Tang, Zhongdao Wang, Guoqing Wang, Jilai Zheng, Xiangxuan Ren, Bailan Feng, and Chao Ma. Sparseocc: Rethinking sparse latent representation for vision-based semantic occupancy prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,...

    2024

  38. [46]

    State of the art on neural rendering

    Ayush Tewari, Ohad Fried, Justus Thies, Vincent Sitzmann, Stephen Lombardi, Kalyan Sunkavalli, Ricardo Martin-Brualla, Tomas Simon, Jason Saragih, Matthias Nießner, et al. State of the art on neural rendering. InComputer Graphics F orum, volume 39, pages 701–727. Wiley Online ...

    2020

  39. [47]

    Advances in neural rendering

    Ayush Tewari, Justus Thies, Ben Mildenhall, Pratul Srinivasan, Edgar Tretschk, Wang Yifan, Christoph Lassner, Vincent Sitzmann, Ricardo Martin-Brualla, Stephen Lombardi, et al. Advances in neural rendering. InComputer Graphics F orum, volume 41, pages 703–735. Wiley Online Lib...

    2022

  40. [48]

    Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving.Advances in Neural Information Processing Systems, 36, 2024

    Xiaoyu Tian, Tao Jiang, Longfei Yun, Yucheng Mao, Huitong Yang, Yue Wang, Yilun Wang, and Hang Zhao. Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving.Advances in Neural Information Processing Systems, 36, 2024

    2024

  41. [49]

    Scene as occupancy

    Wenwen Tong, Chonghao Sima, Tai Wang, Li Chen, Silei Wu, Hanming Deng, Yi Gu, Lewei Lu, Ping Luo, Dahua Lin, et al. Scene as occupancy. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 8406–8415, 2023

    2023

  42. [50]

    Attention is all you need.Advances in Neural Information Processing Systems, 2017

    A Vaswani. Attention is all you need.Advances in Neural Information Processing Systems, 2017

    2017

  43. [51]

    Opus: Occupancy prediction using a sparse set

    Jiabao Wang, Zhaojiang Liu, Qiang Meng, Liujiang Yan, Ke Wang, Jie Yang, Wei Liu, Qibin Hou, and Mingming Cheng. Opus: Occupancy prediction using a sparse set. InAdvances in Neural Information Processing Systems, 2024

    2024

  44. [52]

    Exploring object-centric temporal modeling for efficient multi-view 3d object detection

    Shihao Wang, Yingfei Liu, Tiancai Wang, Ying Li, and Xiangyu Zhang. Exploring object-centric temporal modeling for efficient multi-view 3d object detection. InProceedings of the IEEE/CVF international conference on computer vision, pages 3621–3631, 2023

    2023

  45. [53]

    Openoccupancy: A large scale benchmark for surrounding semantic occupancy perception

    Xiaofeng Wang, Zheng Zhu, Wenbo Xu, Yunpeng Zhang, Yi Wei, Xu Chi, Yun Ye, Dalong Du, Jiwen Lu, and Xingang Wang. Openoccupancy: A large scale benchmark for surrounding semantic occupancy perception. InProceedings of the IEEE/CVF International Conference on Computer Vision, pa...

    2023

  46. [54]

    Detr3d: 3d object detection from multi-view images via 3d-to-2d queries

    Yue Wang, Vitor Campagnolo Guizilini, Tianyuan Zhang, Yilun Wang, Hang Zhao, and Justin Solomon. Detr3d: 3d object detection from multi-view images via 3d-to-2d queries. InConference on Robot Learning, pages 180–191. PMLR, 2022

    2022

  47. [55]

    Surroundocc: Multi- camera 3d occupancy prediction for autonomous driving

    Yi Wei, Linqing Zhao, Wenzhao Zheng, Zheng Zhu, Jie Zhou, and Jiwen Lu. Surroundocc: Multi- camera 3d occupancy prediction for autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 21729–21740, 2023. 12

    2023

  48. [56]

    Metric3d: Towards zero-shot metric 3d prediction from a single image

    Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai, Gang Yu, Kaixuan Wang, Xiaozhi Chen, and Chunhua Shen. Metric3d: Towards zero-shot metric 3d prediction from a single image. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 9043–9053, 2023

    2023

  49. [57]

    Mip-splatting: Alias-free 3d gaussian splatting

    Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. Mip-splatting: Alias-free 3d gaussian splatting. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19447–19456, 2024

    2024

  50. [58]

    Flashocc: Fast and memory-efficient occupancy prediction via channel-to-height plugin

    Zichen Yu, Changyong Shu, Jiajun Deng, Kangjie Lu, Zongdai Liu, Jiangyong Yu, Dawei Yang, Hui Li, and Yan Chen. Flashocc: Fast and memory-efficient occupancy prediction via channel-to-height plugin. arXiv preprint arXiv:2311.12058, 2023

    2023 arXiv

  51. [59]

    Occnerf: Self-supervised multi-camera occupancy prediction with neural radiance fields.arXiv preprint arXiv:2312.09243, 2023

    Chubin Zhang, Juncheng Yan, Yi Wei, Jiaxin Li, Li Liu, Yansong Tang, Yueqi Duan, and Jiwen Lu. Occnerf: Self-supervised multi-camera occupancy prediction with neural radiance fields.arXiv preprint arXiv:2312.09243, 2023

    2023 arXiv

  52. [60]

    Occformer: Dual-path transformer for vision-based 3d semantic occupancy prediction

    Yunpeng Zhang, Zheng Zhu, and Dalong Du. Occformer: Dual-path transformer for vision-based 3d semantic occupancy prediction. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 9433–9443, 2023

    2023

  53. [61]

    nucraft: Crafting high resolution 3d semantic occupancy for unified 3d scene understanding

    Benjin Zhu, Zhe Wang, and Hongsheng Li. nucraft: Crafting high resolution 3d semantic occupancy for unified 3d scene understanding. InEuropean Conference on Computer Vision, pages 125–141. Springer, 2024. 13

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.