Pith. sign in

REVIEW 4 major objections 6 minor 62 references

VoxelSplat: Dynamic Gaussian Splatting as an Effective Loss for Occupancy and Flow Prediction

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Camera-only self-driving perception improves by rendering predicted voxels as dynamic 3D Gaussians and supervising them with 2D labels, at zero inference cost.

desk verdict A genuinely new training-time Gaussian-splatting loss with credible gains, but an unresolved ambiguity about the 2D flow supervision could undercut the self-supervised claim. read the letter →

arxiv 2506.05563 v1 pith:M26BLYZN submitted 2025-06-05 cs.CV

Ziyue Zhu , Shenlong Wang , Jin Xie , Jiang-jiang Liu , Jingdong Wang , Jian Yang This is my paper
classification cs.CV
keywords 3DGaussianSplattingoccupancypredictionsceneflowautonomousdrivingrenderingloss2Dsupervisionsemantictraining-timeregularization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a major bottleneck in camera-based 3D occupancy prediction is the mismatch between 3D voxel supervision and the 2D image evidence the model actually sees, especially under occlusion and for fast-moving objects. The proposed remedy, VoxelSplat, turns the model's predicted voxel semantics and scene flow into a set of 3D semantic Gaussians at training time, moves the dynamic ones by the predicted flow, and splats them into camera views to be checked against 2D ground-truth labels from the current and next frames. The authors claim this training-only rendering loss improves occupancy and flow accuracy across three different base architectures, with no extra cost at inference. If correct, the paper establishes that explicit dynamic Gaussian rendering is a broadly useful training signal for occupancy perception, not just a rendering technique.

What carries the argument

The central object is the dynamic semantic Gaussian: a 3D Gaussian ellipsoid whose center is a sampled occupied voxel, whose appearance is replaced by semantic logits, and whose center is displaced by the predicted scene flow to represent the object's future position. These Gaussians are rasterized with the standard alpha-composited splatting of 3D Gaussian Splatting, giving a differentiable 2D projection of the 3D prediction. The mechanism that carries the argument is the rendering loss comparing this projection with 2D labels produced by projecting the same ground-truth voxels, split into static and dynamic parts, through the same camera geometry.

What would settle it

A controlled ablation that keeps the weighted sampling and static/dynamic split unchanged but replaces the rendered Gaussian 2D loss with an equivalent per-pixel 2D loss computed on the same GT-visible locations (e.g., by projecting GT voxels directly and supervising the occupancy network's own ray-traced depth and semantics) would settle whether the Gaussian splatting mechanism itself, rather than the reweighted sampling and extra 2D labels, produces the reported gains. If the simplified loss matches the RayIoU and mA VE numbers, the Gaussian rendering is not the operative ingredient. Additionally, checking whether the 2D labels in Sec. 3.3 use ground-truth or predicted scene flow would resolve the circularity concern.

Watch

Extended reading notes

Core claim

VoxelSplat demonstrates that dynamic 3D Gaussian splatting can serve as an effective auxiliary training loss for camera-based occupancy and scene-flow prediction. The method samples Gaussian centers from ground-truth occupied voxels inside the camera-visible region, decodes per-Gaussian shape, opacity, semantic logits, and flow, then renders static and dynamic Gaussians separately into 2D views. Semantic cross-entropy and depth L1 losses over the rendered views, applied to both the current frame and a future frame obtained by moving Gaussians along the predicted flow, provide dense 2D supervision that improves the learned 3D representation. The authors report consistent gains over three baselines, with FB-Occ improving by 3.4 RayIoU, 3.1 mIoU, and 0.202 mA VE, and state that the rendering branch is discarded at inference.

Load-bearing premise

The method assumes that supervising only the ground-truth occupied voxels that are visible to the cameras, through 2D projections of those same voxels, fixes the model's real errors (false positives, occluded regions, fast objects) rather than merely concentrating training on already-labeled regions; it also assumes the online 2D labels are generated with ground-truth flow, not predicted flow, so the scene-flow supervision is not circular.

Editorial extensions

If this is right

  • Occupancy networks trained with VoxelSplat converge faster on both occupancy and flow losses, and the final accuracy is higher, as shown by the paper's loss-curve experiments.
  • Because the rendering branch is training-only, any existing occupancy model can adopt the loss without changing its inference-time latency or memory footprint.
  • The method's speed-weighted and class-weighted point sampling rebalances sparse dynamic classes, improving predictions for pedestrians, bicycles, and motorcycles more than for static classes.
  • Gaussian-based 2D supervision outperforms NeRF-style volume rendering as an auxiliary loss for occupancy, since it concentrates samples on occupied space rather than empty space along rays.
  • The gained spatial-temporal consistency from rendering the same Gaussians in two frames helps the model learn scene flow from 2D labels of adjacent frames, even though 3D flow labels are still required for the final loss.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same rendering-loss recipe could be extended to occupancy forecasting: render Gaussians displaced by predicted multi-step flows into future views and supervise with future 2D labels, giving a free self-supervised signal for temporal prediction.
  • A testable extension would be to apply the loss to open-vocabulary occupancy models by rendering per-Gaussian CLIP-style embeddings instead of fixed semantic logits, allowing 2D language-supervised training.
  • The reported gains may partly stem from the weighted sampling of already-labeled GT locations rather than from the Gaussian representation itself; an ablation that decouples 'where to sample' from 'how to render' would clarify the mechanism.
  • If the online 2D labels are derived from ground-truth flow rather than predicted flow, then the method is semi-supervised for flow only in the sense of using 2D projections, and applying it to unlabeled datasets would require a flow estimate from somewhere else.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes VoxelSplat, a training-time auxiliary loss for camera-based 3D semantic occupancy and scene flow prediction. During training, semantic 3D Gaussians are decoded from sampled occupied voxels of a backbone occupancy model, split into static and dynamic subsets, with dynamic centers moved by predicted scene flow, and rendered into 2D semantic and depth maps that are supervised by 2D labels generated from ground-truth voxels projected into adjacent-frame camera views. These 2D rendering losses are added to the original 3D losses, while at inference only the base occupancy model is used. Experiments on nuScenes/OpenOcc show consistent improvements over three baselines (BEVDet-Occ, FB-Occ, SparseOcc) in RayIoU, mIoU, and mA VE, with ablations for Gaussian versus volume rendering, multi-frame supervision, static/dynamic decomposition, and weighted sampling.

Significance. If the mechanism works as intended, the paper's contribution is notable: a plug-and-play training loss that integrates with multiple occupancy architectures without inference overhead, and it reports sizable improvements, e.g., +3.4 RayIoU and +3.1 mIoU over FB-Occ in Table 1 and +2.9 to +3.6 RayIoU across three backbones in Table 3. The comparison with volume rendering (Table 4, versions A vs B) and the explicit motion modeling through dynamic Gaussians are interesting and potentially reusable ideas. The main caveat is that the flow-supervision mechanism must be disambiguated: if the online labels are moved by the predicted flow, the dynamic 2D losses are locally invariant to flow translations and cannot explain the reported mA VE gains; and the abstract's 'self-supervised' wording conflicts with the conclusion's admission that ground-truth 3D flow is required.

major comments (4)
  1. [Sec. 3.3 (Online Label Generation)] The sentence 'The dynamic ones are duplicated and moved according to the scene flow' does not specify whether 'the scene flow' is the predicted flow or ground-truth flow. This ambiguity is load-bearing: in Sec. 3.2 the dynamic Gaussian centers are updated as mu + Delta_x using predicted flow, so if the same Delta_x also moves the ground-truth voxels that generate the 2D labels, then the rendered dynamic Gaussians and their labels translate together, making the semantic and depth rendering losses locally invariant to the predicted flow and unable to explain the mA VE improvement from 0.545 to 0.303 in Table 1 or from 0.487 to 0.353 in Table 4, version E. If ground-truth flow is used instead, the mechanism is coherent but the 'self-supervised' wording in the abstract is inaccurate. Please state explicitly which flow is used and, if necessary, report a variant that separates the two cases.
  2. [Sec. 1, Sec. 3.2, Sec. 5] The abstract claims scene flow is learned 'in a self-supervised manner using the labels of adjacent frames,' but Sec. 3.2 states that the 3D scene flow prediction is supervised with 3D annotations, Sec. 3.3 defines a 3D flow loss L3D, and the Conclusion says 'our model still requires ground truth 3D scene flow.' These statements are irreconcilable as written. Please clarify the precise role of 2D labels versus 3D flow labels in the flow supervision, and soften or qualify the self-supervised claim accordingly.
  3. [Sec. 4.3, Table 4] The row references in the ablation text do not match Table 4. The text says 'In version E of Tab. 4, this approach leads to a significant performance drop,' but version E shows an improvement over version C (RayIoU 34.3 vs 34.1, mA VE 0.353 vs 0.487); the drop is in version D (RayIoU 31.4, mA VE 0.792). Since Table 4 is the core evidence for the component-wise claims, please correct the references and specify exactly which configuration (e.g., decomposition applied directly to the 3D loss without 2D loss) row D represents.
  4. [Sec. 4.1, Sec. 4.3, Fig. 5, Table 4] The empirical evaluation is reported exclusively on the nuScenes validation split, without error bars or repeated-seed statistics, and the sampling temperature t=0.5 used in the final configuration is selected from the same validation split (Fig. 5, 'Based on this analysis, we set t=0.5 in Version F'). This makes the magnitude of the headline gains (e.g., +3.4 RayIoU and +0.202 mA VE in Table 1) hard to judge and risks optimistic selection. Please report test-set numbers or at least standard deviations over multiple runs, and describe the validation-selection protocol.
minor comments (6)
  1. [Eq. (3)] In Eq. (3), the denominator uses P(N_{p,q}) with p,q as both the class label and the summation index; this notation is confusing and should be changed to P(N_{i,j}).
  2. [Sec. 3.3] There is a duplicated word: 'With the the efficient 3DGS tools' should be 'With the efficient 3DGS tools.'
  3. [Table 2 caption] The caption contains the typo 'OpenOccuancy' (should be 'OpenOccupancy').
  4. [Table 4 caption] The caption contains 'adjust frames,' which should be 'adjacent frames.'
  5. [Table 1] The parenthetical +0.202 mA VE improvement should be explicitly tied to the FB-Occ-flow baseline (0.505) rather than the BEVDet-Occ-flow row (0.545), which is the immediately preceding row, to avoid misreading.
  6. [Throughout] The spaced forms 'V oxelSplat,' 'V oxels,' and 'V ox' appear to be LaTeX rendering artifacts; please ensure the final PDF renders these words correctly.

Circularity Check

1 steps flagged · score 1.0 of 10

No confirmed circularity; one unresolved ambiguity in Sec. 3.3 could make the self-supervised flow loss self-referential if 'the scene flow' is the predicted flow.

  1. self definitional [Sec. 3.3, 'Online Label Generation'; loss defined in Eq. (5)]
    "The dynamic ones are duplicated and moved according to the scene flow. With the the efficient 3DGS tools [18], 2D labels are generated by projecting the static and dynamic voxels into camera views."

    Sec. 3.2 defines future dynamic Gaussians by 'we update the gaussians' centers with µ + ∆x' using predicted flow, and Eq. (5) computes CE/depth losses against the 2D labels. If 'the scene flow' in Online Label Generation is the same predicted flow, then both the rendered future Gaussian and its supervision label are translated by the same ∆x; the loss compares two objects that always co-move, so its gradient w.r.t. ∆x is (to first order) zero and the 2D loss cannot be the source of the claimed self-supervised flow learning. The target is then defined in terms of the prediction, the signature of a self-definitional loss. If ground-truth flow is meant, the mechanism is non-circular but the 'self-supervised' wording is inaccurate.

full rationale

The core derivation is otherwise self-contained: the 2D semantic/depth labels are projections of external OpenOcc/3D ground-truth voxels, the occupancy benchmarks (RayIoU, mIoU) are independent, no load-bearing self-citation or imported uniqueness theorem appears, and flow prediction also receives direct supervision from ground-truth 3D scene flow (L3D), so the reported gains do not reduce to fitted parameters or to the method's own outputs. The single flagged concern is the ambiguous 'moved according to the scene flow' in Sec. 3.3: under the predicted-flow reading the 2D flow signal collapses into a tautology, while under the ground-truth-flow reading it is coherent but not self-supervised. Because the text does not disambiguate and the code is not released, this is recorded as a potential circularity rather than a confirmed one; it does not by itself invalidate the external 3D-supervised flow results, but it weakens the paper's mechanistic claim that the rendering loss drives the mA VE gain.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No physical entities are postulated. The 3D semantic Gaussians are an internal, training-only representation; the paper claims no independent existence for them beyond the rendered outputs shown in the supplementary. The empirical claim rests mainly on hyperparameters (t, N, Q) and on the assumption that GT-voxel-derived 2D labels and differentiable splatting provide useful, unbiased gradients.

free parameters (3)
  • Sampling temperature t = 0.5
    Hyperparameter in the weighted sampling function of Eq. (3); selected from a validation sweep in Fig. 5 and fixed for all main experiments.
  • Number of sampled Gaussian centers N = 100000
    Number of points sampled per scene per batch (Sec. 4.1); no ablation is shown for this quantity.
  • Number of speed classes Q = 6
    Dynamic voxels are divided into 6 speed classes for weighted sampling (Sec. 4.1); this choice is not ablated.
assumptions (4)
  • domain assumption Differentiable Gaussian rasterization (Kerbl et al. [18]) provides usable and stable gradients through semantic-logit and depth rendering.
    The entire auxiliary loss relies on gradients through splatting improving voxel features, as used in Secs. 3.2 and 3.3.
  • domain assumption 2D labels built by projecting ground-truth voxels into camera views are accurate and aligned with the rendered Gaussians.
    Sec. 3.3 'Online Label Generation' assumes voxelization, projection, and splatting introduce no significant misalignment that would bias the rendering loss.
  • domain assumption Object motion is approximately linear over the frame interval, so updating Gaussian centers by mu plus Delta-x, with Delta-x equal to flow times time interval, describes the next frame.
    Sec. 3.2 updates Gaussian centers this way; nonlinear or rotational motion is not modeled.
  • domain assumption OpenOcc and Occ3D ground-truth annotations are reliable enough to serve as both 3D targets and as sources for 2D pseudo-labels.
    All experiments use these annotations (Sec. 4.1); annotation errors would propagate to both the 3D and 2D supervision signals.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VoxelSplat: Dynamic Gaussian Splatting as an Effective Loss for Occupancy and Flow Prediction." pith.science (2026). https://pith.science/paper/M26BLYZN

@misc{pith2026250605563,
  author       = {Pith},
  title        = {Pith review of: VoxelSplat: Dynamic Gaussian Splatting as an Effective Loss for Occupancy and Flow Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M26BLYZN}},
  note         = {Machine review of arXiv:2506.05563}
}
read the original abstract

Recent advancements in camera-based occupancy prediction have focused on the simultaneous prediction of 3D semantics and scene flow, a task that presents significant challenges due to specific difficulties, e.g., occlusions and unbalanced dynamic environments. In this paper, we analyze these challenges and their underlying causes. To address them, we propose a novel regularization framework called VoxelSplat. This framework leverages recent developments in 3D Gaussian Splatting to enhance model performance in two key ways: (i) Enhanced Semantics Supervision through 2D Projection: During training, our method decodes sparse semantic 3D Gaussians from 3D representations and projects them onto the 2D camera view. This provides additional supervision signals in the camera-visible space, allowing 2D labels to improve the learning of 3D semantics. (ii) Scene Flow Learning: Our framework uses the predicted scene flow to model the motion of Gaussians, and is thus able to learn the scene flow of moving objects in a self-supervised manner using the labels of adjacent frames. Our method can be seamlessly integrated into various existing occupancy models, enhancing performance without increasing inference time. Extensive experiments on benchmark datasets demonstrate the effectiveness of VoxelSplat in improving the accuracy of both semantic occupancy and scene flow estimation. The project page and codes are available at https://zzy816.github.io/VoxelSplat-Demo/.

Figures

Figures reproduced from arXiv: 2506.05563 by the authors.

Figure 1
Figure 1. During training, our method additionally predicts 3D [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overview of our framework: (1) Employing an occupancy model integrated with our flow decoder to predict occupancy and [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The qualitative comparison of our occupancy prediction with other methods is presented. We highlight the regions where our [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: We present a qualitative comparison of our occupancy prediction with the ground truth. We use a color scale to represent the [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: The effect of sampling function hyperparameter [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 7
Figure 7. Figure 7: The comparison of loss curves with and without our [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: The visualization results of rendering semantics and depths on the validation dataset [ [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: We provide a series of videos in the supplementary material, which demonstrate the predicted occupancy and flow from different [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

62 extracted references · 33 canonical work pages

  1. [1]

    Per-gaussian embedding-based deformation for deformable 3d gaussian splatting.arXiv preprint arXiv:2404.03613, 2024

    Jeongmin Bae, Seoha Kim, Youngsik Yun, Hahyun Lee, Gun Bang, and Youngjung Uh. Per-gaussian embedding-based deformation for deformable 3d gaussian splatting.arXiv preprint arXiv:2404.03613, 2024. 2

    arXiv 2024

  2. [2]

    The lov´asz-softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks

    Maxim Berman, Amal Rannen Triki, and Matthew B Blaschko. The lov´asz-softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 4413–4421,

    work page

  3. [3]

    A generalization of algebraic surface draw- ing.ACM transactions on graphics (TOG), 1(3):235–256,

    James F Blinn. A generalization of algebraic surface draw- ing.ACM transactions on graphics (TOG), 1(3):235–256,

    work page

  4. [4]

    nuscenes: A multi- modal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Gi- ancarlo Baldan, and Oscar Beijbom. nuscenes: A multi- modal dataset for autonomous driving. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621–11631, 2020. 2, 5, 7, 1

    work page 2020

  5. [5]

    Omnire: Omni ur- ban scene reconstruction.arXiv preprint arXiv:2408.16760,

    Ziyu Chen, Jiawei Yang, Jiahui Huang, Riccardo de Lutio, Janick Martinez Esturo, Boris Ivanovic, Or Litany, Zan Go- jcic, Sanja Fidler, Marco Pavone, et al. Omnire: Omni ur- ban scene reconstruction.arXiv preprint arXiv:2408.16760,

    arXiv

  6. [6]

    Dream- scene4d: Dynamic multi-object scene generation from monocular videos.arXiv preprint arXiv:2405.02280, 2024

    Wen-Hsuan Chu, Lei Ke, and Katerina Fragkiadaki. Dream- scene4d: Dynamic multi-object scene generation from monocular videos.arXiv preprint arXiv:2405.02280, 2024. 2

    arXiv 2024

  7. [7]

    4d gaussian splatting: Towards efficient novel view synthesis for dynamic scenes

    Yuanxing Duan, Fangyin Wei, Qiyu Dai, Yuhang He, Wen- zheng Chen, and Baoquan Chen. 4d gaussian splatting: Towards efficient novel view synthesis for dynamic scenes. arXiv preprint arXiv:2402.03307, 2024. 2

    arXiv 2024

  8. [8]

    A New Split Algorithm for 3D Gaussian Splatting

    Qiyuan Feng, Gengchen Cao, Haoxiang Chen, Tai-Jiang Mu, Ralph R Martin, and Shi-Min Hu. A new split algorithm for 3d gaussian splatting.arXiv preprint arXiv:2403.09143,

    work page Pith review arXiv

Show all 62 references
  1. [9]

    Gaussianocc: Fully self-supervised and effi- cient 3d occupancy estimation with gaussian splatting.arXiv preprint arXiv:2408.11447, 2024

    Wanshui Gan, Fang Liu, Hongbin Xu, Ningkai Mo, and Naoto Yokoya. Gaussianocc: Fully self-supervised and effi- cient 3d occupancy estimation with gaussian splatting.arXiv preprint arXiv:2408.11447, 2024. 2

    2024 arXiv

  2. [10]

    Gaussianflow: Splatting gaussian dynamics for 4d content creation.arXiv preprint arXiv:2403.12365, 2024

    Quankai Gao, Qiangeng Xu, Zhe Cao, Ben Mildenhall, Wen- chao Ma, Le Chen, Danhang Tang, and Ulrich Neumann. Gaussianflow: Splatting gaussian dynamics for 4d content creation.arXiv preprint arXiv:2403.12365, 2024. 2

    2024 arXiv

  3. [11]

    Motion-aware 3d gaussian splatting for efficient dynamic scene reconstruction.arXiv preprint arXiv:2403.11447, 2024

    Zhiyang Guo, Wengang Zhou, Li Li, Min Wang, and Houqiang Li. Motion-aware 3d gaussian splatting for efficient dynamic scene reconstruction.arXiv preprint arXiv:2403.11447, 2024. 2

    2024 arXiv

  4. [12]

    Bevdet: High-performance multi-camera 3d object de- tection in bird-eye-view.arXiv preprint arXiv:2112.11790,

    Junjie Huang, Guan Huang, Zheng Zhu, Yun Ye, and Dalong Du. Bevdet: High-performance multi-camera 3d object de- tection in bird-eye-view.arXiv preprint arXiv:2112.11790,

    arXiv

  5. [13]

    Selfocc: Self-supervised vision-based 3d oc- cupancy prediction.arXiv preprint arXiv:2311.12754, 2023

    Yuanhui Huang, Wenzhao Zheng, Borui Zhang, Jie Zhou, and Jiwen Lu. Selfocc: Self-supervised vision-based 3d oc- cupancy prediction.arXiv preprint arXiv:2311.12754, 2023. 2

    2023 arXiv

  6. [14]

    Tri-perspective view for vision-based 3d se- mantic occupancy prediction

    Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. Tri-perspective view for vision-based 3d se- mantic occupancy prediction. InCVPR, pages 9223–9232,

  7. [15]

    Gaussianformer: Scene as gaussians for vision-based 3d semantic occupancy prediction.arXiv preprint arXiv:2405.17429, 2024

    Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. Gaussianformer: Scene as gaussians for vision-based 3d semantic occupancy prediction.arXiv preprint arXiv:2405.17429, 2024. 5, 6

    2024 arXiv

  8. [16]

    Sc-gs: Sparse-controlled gaussian splatting for editable dynamic scenes

    Yi-Hua Huang, Yang-Tian Sun, Ziyi Yang, Xiaoyang Lyu, Yan-Pei Cao, and Xiaojuan Qi. Sc-gs: Sparse-controlled gaussian splatting for editable dynamic scenes. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4220–4230, 2024. 2

    2024

  9. [17]

    A compact dynamic 3d gaussian representation for real-time dynamic view synthesis

    Kai Katsumata, Duc Minh V o, and Hideki Nakayama. A compact dynamic 3d gaussian representation for real-time dynamic view synthesis. InEuropean Conference on Com- puter Vision, pages 394–412. Springer, 2025. 2

    2025

  10. [18]

    3d gaussian splatting for real-time radiance field rendering.ACM TOG, 42(4):1–14, 2023

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM TOG, 42(4):1–14, 2023. 2, 3, 4

    2023

  11. [19]

    Dgd: Dynamic 3d gaussians distillation.arXiv preprint arXiv:2405.19321, 2024

    Isaac Labe, Noam Issachar, Itai Lang, and Sagie Benaim. Dgd: Dynamic 3d gaussians distillation.arXiv preprint arXiv:2405.19321, 2024. 2

    2024 arXiv

  12. [20]

    Fully explicit dynamic gaussian splat- ting.arXiv preprint arXiv:2410.15629, 2024

    Junoh Lee, Chang-Yeon Won, Hyunjun Jung, Inhwan Bae, and Hae-Gon Jeon. Fully explicit dynamic gaussian splat- ting.arXiv preprint arXiv:2410.15629, 2024. 2

    2024 arXiv

  13. [21]

    V oxformer: Sparse voxel transformer for camera- based 3d semantic scene completion

    Yiming Li, Zhiding Yu, Christopher Choy, Chaowei Xiao, Jose M Alvarez, Sanja Fidler, Chen Feng, and Anima Anand- kumar. V oxformer: Sparse voxel transformer for camera- based 3d semantic scene completion. InCVPR, pages 9087– 9098, 2023. 1, 2

    2023

  14. [22]

    Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers

    Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chong- hao Sima, Tong Lu, Yu Qiao, and Jifeng Dai. Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers. InECCV, pages 1–

  15. [23]

    2, 3, 4, 5, 6

    Springer, 2022. 2, 3, 4, 5, 6

    2022

  16. [24]

    Fb-occ: 3d occupancy prediction based on forward-backward view transformation

    Zhiqi Li, Zhiding Yu, David Austin, Mingsheng Fang, Shiyi Lan, Jan Kautz, and Jose M Alvarez. Fb-occ: 3d occupancy prediction based on forward-backward view transformation. arXiv preprint arXiv:2307.01492, 2023. 1, 2, 4, 5, 6, 7

    2023 arXiv

  17. [25]

    Gaussian-flow: 4d reconstruction with dynamic 3d gaus- sian particle

    Youtian Lin, Zuozhuo Dai, Siyu Zhu, and Yao Yao. Gaussian-flow: 4d reconstruction with dynamic 3d gaus- sian particle. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21136– 21145, 2024. 2

    2024

  18. [26]

    Fully sparse 3d panop- tic occupancy prediction.arXiv preprint arXiv:2312.17118,

    Haisong Liu, Haiguang Wang, Yang Chen, Zetong Yang, Jia Zeng, Li Chen, and Limin Wang. Fully sparse 3d panop- tic occupancy prediction.arXiv preprint arXiv:2312.17118,

    arXiv

  19. [27]

    Surroundsdf: Implicit 3d scene understanding based on signed distance field.arXiv preprint arXiv:2403.14366, 2024

    Lizhe Liu, Bohua Wang, Hongwei Xie, Daqi Liu, Li Liu, Zhiqiang Tian, Kuiyuan Yang, and Bing Wang. Surroundsdf: Implicit 3d scene understanding based on signed distance field.arXiv preprint arXiv:2403.14366, 2024. 2

    2024 arXiv

  20. [28]

    Modgs: Dy- namic gaussian splatting from causually-captured monocular videos.arXiv preprint arXiv:2406.00434, 2024

    Qingming Liu, Yuan Liu, Jiepeng Wang, Xianqiang Lv, Peng Wang, Wenping Wang, and Junhui Hou. Modgs: Dy- namic gaussian splatting from causually-captured monocular videos.arXiv preprint arXiv:2406.00434, 2024. 2

    2024 arXiv

  21. [29]

    3d geometry-aware deformable gaussian splatting for dynamic view synthesis

    Zhicheng Lu, Xiang Guo, Le Hui, Tianrui Chen, Min Yang, Xiao Tang, Feng Zhu, and Yuchao Dai. 3d geometry-aware deformable gaussian splatting for dynamic view synthesis. arXiv preprint arXiv:2404.06270, 2024

    2024 arXiv

  22. [30]

    Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis.arXiv preprint arXiv:2308.09713, 2023

    Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis.arXiv preprint arXiv:2308.09713, 2023. 2

    2023 arXiv

  23. [31]

    3dgsr: Implicit surface reconstruction with 3d gaussian splat- ting.arXiv preprint arXiv:2404.00409, 2024

    Xiaoyang Lyu, Yang-Tian Sun, Yi-Hua Huang, Xiuzhe Wu, Ziyi Yang, Yilun Chen, Jiangmiao Pang, and Xiaojuan Qi. 3dgsr: Implicit surface reconstruction with 3d gaussian splat- ting.arXiv preprint arXiv:2404.00409, 2024. 2

    2024 arXiv

  24. [32]

    Cam4docc: Benchmark for camera-only 4d occupancy fore- casting in autonomous driving applications

    Junyi Ma, Xieyuanli Chen, Jiawei Huang, Jingyi Xu, Zhen Luo, Jintao Xu, Weihao Gu, Rui Ai, and Hesheng Wang. Cam4docc: Benchmark for camera-only 4d occupancy fore- casting in autonomous driving applications. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- ...

    2024

  25. [33]

    Cotr: Compact occupancy transformer for vision-based 3d occupancy prediction

    Qihang Ma, Xin Tan, Yanyun Qu, Lizhuang Ma, Zhizhong Zhang, and Yuan Xie. Cotr: Compact occupancy transformer for vision-based 3d occupancy prediction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19936–19945, 2024. 2

    2024

  26. [34]

    Occdepth: A depth-aware method for 3d semantic scene completion

    Ruihang Miao, Weizhou Liu, Mingrui Chen, Zheng Gong, Weixin Xu, Chen Hu, and Shuchang Zhou. Occdepth: A depth-aware method for 3d semantic scene completion. arXiv preprint arXiv:2302.13540, 2023. 2

    2023 arXiv

  27. [35]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis.Communications of the ACM, 65(1):99–106, 2021

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis.Communications of the ACM, 65(1):99–106, 2021. 7, 8

    2021

  28. [36]

    Renderocc: Vision-centric 3d occupancy prediction with 2d rendering supervision.arXiv preprint arXiv:2309.09502, 2023

    Mingjie Pan, Jiaming Liu, Renrui Zhang, Peixiang Huang, Xiaoqi Li, Li Liu, and Shanghang Zhang. Renderocc: Vision-centric 3d occupancy prediction with 2d rendering supervision.arXiv preprint arXiv:2309.09502, 2023. 1, 2, 5, 6, 7

    2023 arXiv

  29. [37]

    Learning occupancy for monocular 3d object detection

    Liang Peng, Junkai Xu, Haoran Cheng, Zheng Yang, Xi- aopei Wu, Wei Qian, Wenxiao Wang, Boxi Wu, and Deng Cai. Learning occupancy for monocular 3d object detection. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10281–10292, 2024. 2

    2024

  30. [38]

    Scene as occupancy.Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, 2023

    Chonghao Sima, Wenwen Tong, Tai Wang, Li Chen, Silei Wu, Hanming Deng, Yi Gu, Lewei Lu, Ping Luo, Dahua Lin, and Hongyang Li. Scene as occupancy.Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, 2023. 1, 5, 6, 7, 8

    2023

  31. [39]

    Col- laborative semantic occupancy prediction with hybrid fea- ture fusion in connected automated vehicles

    Rui Song, Chenwei Liang, Hu Cao, Zhiran Yan, Walter Zim- mer, Markus Gross, Andreas Festag, and Alois Knoll. Col- laborative semantic occupancy prediction with hybrid fea- ture fusion in connected automated vehicles. InProceedings of the IEEE/CVF Conference on Computer Vision ...

    2024

  32. [40]

    Dreamgaussian: Generative gaussian splatting for effi- cient 3d content creation.arXiv preprint arXiv:2309.16653,

    Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for effi- cient 3d content creation.arXiv preprint arXiv:2309.16653,

    arXiv

  33. [41]

    Sparseocc: Re- thinking sparse latent representation for vision-based seman- tic occupancy prediction

    Pin Tang, Zhongdao Wang, Guoqing Wang, Jilai Zheng, Xi- angxuan Ren, Bailan Feng, and Chao Ma. Sparseocc: Re- thinking sparse latent representation for vision-based seman- tic occupancy prediction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni...

    2024

  34. [42]

    Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving.NeurIPS, 36, 2024

    Xiaoyu Tian, Tao Jiang, Longfei Yun, Yucheng Mao, Huitong Yang, Yue Wang, Yilun Wang, and Hang Zhao. Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving.NeurIPS, 36, 2024. 1, 2, 3, 5, 6

    2024

  35. [43]

    Scene as occupancy

    Wenwen Tong, Chonghao Sima, Tai Wang, Li Chen, Silei Wu, Hanming Deng, Yi Gu, Lewei Lu, Ping Luo, Dahua Lin, et al. Scene as occupancy. InICCV, pages 8406–8415, 2023. 2

    2023

  36. [44]

    Pop-3d: Open-vocabulary 3d occupancy prediction from im- ages.Advances in Neural Information Processing Systems, 36, 2024

    Antonin V obecky, Oriane Sim ´eoni, David Hurych, Spyri- don Gidaris, Andrei Bursuc, Patrick P ´erez, and Josef Sivic. Pop-3d: Open-vocabulary 3d occupancy prediction from im- ages.Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  37. [45]

    Openoccupancy: A large scale benchmark for sur- rounding semantic occupancy perception

    Xiaofeng Wang, Zheng Zhu, Wenbo Xu, Yunpeng Zhang, Yi Wei, Xu Chi, Yun Ye, Dalong Du, Jiwen Lu, and Xingang Wang. Openoccupancy: A large scale benchmark for sur- rounding semantic occupancy perception. InICCV, pages 17850–17859, 2023. 1, 2, 3, 6

    2023

  38. [46]

    Panoocc: Unified occupancy representation for camera-based 3d panoptic segmentation.arXiv preprint arXiv:2306.10013, 2023

    Yuqi Wang, Yuntao Chen, Xingyu Liao, Lue Fan, and Zhaox- iang Zhang. Panoocc: Unified occupancy representation for camera-based 3d panoptic segmentation.arXiv preprint arXiv:2306.10013, 2023. 2, 3

    2023 arXiv

  39. [47]

    Omni-scene: omni- gaussian representation for ego-centric sparse-view scene re- construction

    Dongxu Wei, Zhiqi Li, and Peidong Liu. Omni-scene: omni- gaussian representation for ego-centric sparse-view scene re- construction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. 2

    2025

  40. [48]

    Surroundocc: Multi-camera 3d occu- pancy prediction for autonomous driving

    Yi Wei, Linqing Zhao, Wenzhao Zheng, Zheng Zhu, Jie Zhou, and Jiwen Lu. Surroundocc: Multi-camera 3d occu- pancy prediction for autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 21729–21740, 2023. 2, 5, 6

    2023

  41. [49]

    Deep height decoupling for pre- cise vision-based 3d occupancy prediction.arXiv preprint arXiv:2409.07972, 2024

    Yuan Wu, Zhiqiang Yan, Zhengxue Wang, Xiang Li, Le Hui, and Jian Yang. Deep height decoupling for pre- cise vision-based 3d occupancy prediction.arXiv preprint arXiv:2409.07972, 2024. 2

    2024 arXiv

  42. [50]

    Street gaussians for modeling dynamic ur- ban scenes.arXiv preprint arXiv:2401.01339, 2024

    Yunzhi Yan, Haotong Lin, Chenxu Zhou, Weijie Wang, Haiyang Sun, Kun Zhan, Xianpeng Lang, Xiaowei Zhou, and Sida Peng. Street gaussians for modeling dynamic ur- ban scenes.arXiv preprint arXiv:2401.01339, 2024. 2

    2024 arXiv

  43. [51]

    Rignet: Repetitive image guided network for depth completion

    Zhiqiang Yan, Kun Wang, Xiang Li, Zhenyu Zhang, Jun Li, and Jian Yang. Rignet: Repetitive image guided network for depth completion. InEuropean Conference on Computer Vision, pages 214–230. Springer, 2022. 2

    2022

  44. [52]

    Tri- perspective view decomposition for geometry-aware depth completion

    Zhiqiang Yan, Yuankai Lin, Kun Wang, Yupeng Zheng, Yufei Wang, Zhenyu Zhang, Jun Li, and Jian Yang. Tri- perspective view decomposition for geometry-aware depth completion. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4874– 4884, 2024. 2

    2024

  45. [53]

    Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction.arXiv preprint arXiv:2309.13101, 2023

    Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction.arXiv preprint arXiv:2309.13101, 2023. 2

    2023 arXiv

  46. [54]

    Gaussian- dreamer: Fast generation from text to 3d gaussian splatting with point cloud priors.arXiv preprint arXiv:2310.08529,

    Taoran Yi, Jiemin Fang, Guanjun Wu, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Qi Tian, and Xinggang Wang. Gaussian- dreamer: Fast generation from text to 3d gaussian splatting with point cloud priors.arXiv preprint arXiv:2310.08529,

    arXiv

  47. [55]

    Flashocc: Fast and memory-efficient occupancy prediction via channel-to-height plugin.arXiv preprint arXiv:2311.12058, 2023

    Zichen Yu, Changyong Shu, Jiajun Deng, Kangjie Lu, Zong- dai Liu, Jiangyong Yu, Dawei Yang, Hui Li, and Yan Chen. Flashocc: Fast and memory-efficient occupancy prediction via channel-to-height plugin.arXiv preprint arXiv:2311.12058, 2023. 2

    2023 arXiv

  48. [56]

    Occnerf: Self- supervised multi-camera occupancy prediction with neural radiance fields.arXiv e-prints, pages arXiv–2312, 2023

    Chubin Zhang, Juncheng Yan, Yi Wei, Jiaxin Li, Li Liu, Yansong Tang, Yueqi Duan, and Jiwen Lu. Occnerf: Self- supervised multi-camera occupancy prediction with neural radiance fields.arXiv e-prints, pages arXiv–2312, 2023. 2

    2023

  49. [57]

    Egogaussian: Dynamic scene understanding from egocentric video with 3d gaussian splatting.arXiv preprint arXiv:2406.19811, 2024

    Daiwei Zhang, Gengyan Li, Jiajie Li, Micka ¨el Bressieux, Otmar Hilliges, Marc Pollefeys, Luc Van Gool, and Xi Wang. Egogaussian: Dynamic scene understanding from egocentric video with 3d gaussian splatting.arXiv preprint arXiv:2406.19811, 2024. 2

    2024 arXiv

  50. [58]

    Occformer: Dual-path transformer for vision-based 3d semantic occu- pancy prediction

    Yunpeng Zhang, Zheng Zhu, and Dalong Du. Occformer: Dual-path transformer for vision-based 3d semantic occu- pancy prediction. InICCV, pages 9433–9443, 2023. 1, 2, 6

    2023

  51. [59]

    Tclc-gs: Tightly coupled lidar-camera gaussian splatting for surrounding autonomous driving scenes.arXiv preprint arXiv:2404.02410, 2024

    Cheng Zhao, Su Sun, Ruoyu Wang, Yuliang Guo, Jun-Jun Wan, Zhou Huang, Xinyu Huang, Yingjie Victor Chen, and Liu Ren. Tclc-gs: Tightly coupled lidar-camera gaussian splatting for surrounding autonomous driving scenes.arXiv preprint arXiv:2404.02410, 2024. 2

    2024 arXiv

  52. [60]

    Lowrankocc: Tensor decomposition and low-rank recovery for vision-based 3d semantic occupancy prediction

    Linqing Zhao, Xiuwei Xu, Ziwei Wang, Yunpeng Zhang, Borui Zhang, Wenzhao Zheng, Dalong Du, Jie Zhou, and Jiwen Lu. Lowrankocc: Tensor decomposition and low-rank recovery for vision-based 3d semantic occupancy prediction. InProceedings of the IEEE/CVF Conference on Computer Vis...

    2024

  53. [61]

    Occworld: Learning a 3d occupancy world model for autonomous driving

    Wenzhao Zheng, Weiliang Chen, Yuanhui Huang, Borui Zhang, Yueqi Duan, and Jiwen Lu. Occworld: Learning a 3d occupancy world model for autonomous driving. In European Conference on Computer Vision, pages 55–72. Springer, 2025. 2

    2025

  54. [62]

    Motiongs: Exploring explicit motion guidance for deformable 3d gaussian splatting.arXiv preprint arXiv:2410.07707, 2024

    Ruijie Zhu, Yanzhe Liang, Hanzhi Chang, Jiacheng Deng, Jiahao Lu, Wenfei Yang, Tianzhu Zhang, and Yongdong Zhang. Motiongs: Exploring explicit motion guidance for deformable 3d gaussian splatting.arXiv preprint arXiv:2410.07707, 2024. 2

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.