Pith. sign in

REVIEW 4 major objections 4 minor 5 cited by

Occupancy World Model for Robots

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A pose-conditioned occupancy world model, RoboOccWorld, predicts future indoor 3D scene occupancy far more accurately than the prior outdoor-focused OccWorld model, raising next-state IoU/mIoU by 22.34/12.21 on a restructured ScanNet…

desk verdict Useful first indoor occupancy world model, but the benchmark lets a pose-conditioned warp shortcut inflate the reported gains, so the central claim needs a missing baseline before it can be trusted. read the letter →

arxiv 2505.05512 v1 pith:GTN4A3OH submitted 2025-05-07 cs.CV cs.RO

Zhang Zhang , Qiang Zhang , Wei Cui , Shuai Shi , Yijie Guo , Gang Han , Wen Zhao , Jingkai Sun
show 7 more authors
Jiahang Cao Jiaxu Wang Hao Cheng Xiaozhu Ju Zhengping Che Renjing Xu Jian Tang
This is my paper · ORCID
classification cs.CVcs.RO
keywords occupancyworldmodel3Dpredictionindoorsceneevolutionautoregressivetransformercameraposeconditioningspatio-temporalaggregationScanNetbenchmarkrobotperception
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that occupancy world models, generative models that predict fine-grained 3D semantic occupancy of a scene, can work well for indoor robots rather than only for cars on structured roads. Its central claim is that RoboOccWorld outperforms the existing OccWorld model on the new OccWorld-ScanNet benchmark by a wide margin: +22.34/+12.21 IoU/mIoU for next-state prediction and +19.48/+11.61 for six-step autoregressive prediction. The key ideas are to condition the autoregressive transformer on the next camera pose via CCSA and to combine long- and short-term spatio-temporal receptive fields via HSTA. A sympathetic reader would care because accurate future occupancy predictions could directly support robot exploration, collision avoidance, and navigation decisions in unseen indoor spaces.

What carries the argument

The two load-bearing mechanisms are the Conditional Causal State Attention (CCSA) and the Hybrid Spatio-Temporal Aggregation (HSTA). CCSA converts the next camera pose into scale and shift parameters that modulate residual and layer-normalization branches inside causal cross-attention, so the model's forecast explicitly depends on where the camera will move next. HSTA applies causal 3D convolutions with a long temporal window (kernel size 7 in time) and a short window (kernel size 3 in time, 7 or 3 in space) to capture both global temporal consistency and fine-grained local spatio-temporal cues from history. Together they let the transformer combine pose guidance with multi-scale spatio-temporal receptive fields, which the paper argues is necessary because indoor camera motion is far less regular than outdoor driving.

What would settle it

Take RoboOccWorld and replace the next camera pose in CCSA with a random vector while keeping everything else fixed; if next-state IoU stays near 49.55, pose information is not the cause of the gain. Alternatively, re-run OccWorld and RoboOccWorld on a re-split of OccWorld-ScanNet where validation scenes come from unvisited rooms and frames are sampled at random intervals; if the 22.34 IoU gap shrinks to a few points, the benchmark construction, not the model, carries the result.

Watch

Extended reading notes

Core claim

The discovery, on the paper's own terms, is that indoor occupancy evolution can be forecast from a sequence of occupancy observations if the forecast is guided by the camera pose of the next state and by multi-scale spatio-temporal aggregation of history. RoboOccWorld consists of a VQ-VAE scene tokenizer followed by an autoregressive transformer whose causal cross-attention is modulated by pose-derived scale and shift parameters, with HSTA layers built from causal 3D convolutions. On OccWorld-ScanNet (537 training and 137 validation scenes, 100 posed frames per scene), the method reports IoU/mIoU of 49.55/38.25 for next-state prediction and 31.70/23.51 averaged over six autoregressive steps, compared with 27.21/26.04 and 12.22/11.90 for OccWorld, at equal reconstruction quality of 73.39/57.82. End-to-end runs that start from monocular RGB and use either EmbodiedOcc or RoboOcc for upstream occupancy prediction also show consistent gains.

Load-bearing premise

The load-bearing premise is that the restructured OccWorld-ScanNet benchmark is a fair and representative test of indoor occupancy evolution: the scene-level 537/137 split, the 100 posed frames per scene, the 17-frame windows, and the local-annotation occupancy construction do not systematically make RoboOccWorld's task easier than the baseline's.

Editorial extensions

If this is right

  • A robot that knows its next camera pose can forecast the occupancy it will see before moving, making the world model usable as a component of a planning loop.
  • The method works both with ground-truth occupancy and end-to-end from monocular RGB through an upstream occupancy predictor, so the reported gain is not an artifact of perfect input occupancy.
  • Autoregressive rollouts remain substantially ahead of OccWorld at every step from 1 to 6, suggesting the model can sustain multi-step imagination rather than only one-step correction.
  • The two-stage recipe of a VQ-VAE tokenizer followed by an autoregressive transformer is unchanged from OccWorld, so the gains come from added conditioning and aggregation rather than a wholesale architecture change.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own ablation shows HSTA alone adds only 0.40 IoU and CCSA alone adds 6.57, but together they add 22.34, so the interaction between pose conditioning and multi-scale aggregation is the real load-bearing result; an experiment that varies the pose signal (random, noisy, or oracle) would isolate how much of the gain is genuinely pose information.
  • The benchmark samples 100 posed frames per scene and uses 17-frame windows; if many sampled frames are near-duplicates or the pose sequence is unusually smooth, the next-state task may be easier than real robot exploration, and a random-interval frame sampler or a held-out room split would test this without changing the model.
  • The same pose-conditioned causal attention could be dropped into other generative world models that already consume trajectories, such as image-based or 4D occupancy generators, by using the planned pose as the condition token in place of occupancy history.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes RoboOccWorld, a 3D occupancy world model for indoor robot scene-evolution forecasting. It introduces Conditional Causal State Attention (CCSA), which injects the next camera pose as a condition into an autoregressive transformer, and Hybrid Spatio-Temporal Aggregation (HSTA), which combines long- and short-term causal 3D convolutional windows. The authors also restructure an OccWorld-ScanNet benchmark from ScanNet-derived indoor occupancy data with 537/137 train/val scenes, 100 posed frames per scene, and a camera-centric 4.8m x 4.8m x 2.88m voxel grid. On this benchmark they report large improvements over OccWorld for both next-state and autoregressive occupancy prediction, plus end-to-end results from monocular images and an ablation study.

Significance. If the reported results are robust, the paper would make a useful contribution by extending occupancy world models from structured outdoor driving to flexible indoor robot scenarios and by proposing a benchmark for that task. The two proposed components, pose-conditioned attention and multi-scale spatio-temporal aggregation, are clearly motivated and the experimental package includes a next-state task, an autoregressive task, and an end-to-end image-based variant. Credit is due for tackling a genuinely underexplored setting and for planning to release the benchmark. However, the central claim rests on a self-constructed benchmark with a single baseline, no variance estimates, and no control for the pose information that is explicitly fed to the model, so the significance is contingent on the missing analyses being supplied.

major comments (4)
  1. [Section 4.1 and Section 3.3] The evaluation protocol does not include the most natural control baseline for a camera-centric occupancy forecasting task with given poses. Because the occupancy grid is defined in a 4.8m x 4.8m x 2.88m frustum in front of the camera and CCSA is explicitly conditioned on the next camera pose (Eqs. 7-11), a baseline that rigidly warps the ground-truth occupancy of the previous frame into the next camera coordinate frame, filling unseen voxels as free, would use exactly the same information as the model. Such a baseline could account for a substantial portion of the reported 49.55/38.25 IoU/mIoU. Table 4 shows CCSA alone reaches 33.78 and the full model jumps to 49.55, a pattern consistent with pose-conditioned copying rather than learned scene evolution. Reporting this geometric baseline is essential for the central claim that the proposed spatio-temporal modeling, rather than the pose-conditioned benchmark construction, produces the improvement.
  2. [Tables 1-2 and Section 4.4] The claim that RoboOccWorld "significantly outperforms" OccWorld is supported only by single-run point estimates against a single baseline. No error bars, multiple seeds, or statistical tests are reported, so the exact margins (22.34/12.21 IoU/mIoU and 19.48/11.61) cannot be distinguished from run-to-run variation, especially in autoregressive prediction where errors compound over six steps. At minimum, the authors should report means and standard deviations over several training runs and add at least one additional indoor-capable baseline, such as an adapted OccSora-style model or a learned flow/warping method, to support the state-of-the-art claim.
  3. [Section 4.1 and Section 4.3] The OccWorld-ScanNet benchmark construction is not described precisely enough for the evaluation to be independently assessed or reproduced. The paper does not specify how the 'local annotations' are derived from Occ-ScanNet/EmbodiedOcc-ScanNet, how the 100 frames per scene are selected, how the 12 semantic classes are defined or merged, or how the 17-frame training windows are sampled. The sentence in Section 4.3 that 'the first 16 frames are used as history frames and the last 16 frames are used as ground truth for prediction frames' is internally ambiguous for a 17-frame window. Without a precise data-generation protocol and released split definitions, the risk that the benchmark contains easy views, redundant frames, or accidental train/val overlap cannot be ruled out, and the headline comparison is difficult to interpret.
  4. [Table 4 and Section 4.5] The ablation study leaves the core interaction effect unexplained. HSTA alone improves over the no-component baseline by only 0.40 IoU (27.21 to 27.61), CCSA alone improves by 6.57 (27.21 to 33.78), yet the full model improves over CCSA alone by 15.77 IoU (33.78 to 49.55). This huge nonlinear interaction is the main evidence for the value of combining the two modules, but no additional ablations (e.g., CCSA with only the long-term window, CCSA with only the short-term window, different window sizes) or analysis is provided to show how the interaction arises. The reported jump is large enough that an implementation issue or a benchmark-specific shortcut cannot be ruled out without further investigation.
minor comments (4)
  1. [Section 4.3] The description of the 17-frame training windows should be clarified: with 17 consecutive frames, 'first 16 frames are used as history' and 'last 16 frames are used as ground truth' cannot both refer to non-overlapping sets; presumably the model predicts the 17th frame from the first 16, but the text should say so explicitly.
  2. [Equation (12)] The IoU formula uses the notation TP̸=C0 without defining whether the denominator is summed over all non-empty classes; please spell out the definition so the metric is unambiguous.
  3. [Figures 1 and 4-6] Several figure captions and inline references are unclear, e.g., 'Trajectories' in Figure 1 and the description of the HSTA window sizes in Figure 5; the figures would benefit from a legend and a precise statement of kernel sizes and tensor shapes.
  4. [Tables 1-3] The tables contain layout artifacts such as 'RoboOccWorld (ours)Occ' and repeated 'Recon' columns; please clean up the table formatting so that the reader can see which entries are reconstruction metrics and which are prediction metrics.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the derivation is empirical, the evaluation is held-out, and the pose-conditioned inputs are legitimate conditions rather than fitted targets; the geometric-reprojection concern is a benchmark-validity risk, not circular reasoning.

full rationale

The paper's chain is empirical rather than definitional: a VQ-VAE tokenizer is trained on occupancy and frozen, and the world model maps history occupancy and the next camera pose to the next occupancy, trained and evaluated on disjoint ScanNet scenes (537 training / 137 validation scenes). Equation (2) treats the next pose P as an input condition coming from the robot's decision, not as a parameter fitted to the target occupancy. The main comparisons in Tables 1-2 use ground-truth history occupancy, and Table 4 empirically separates the CCSA and HSTA contributions. Self-citations, notably RoboOcc [33], are used as an optional monocular occupancy backbone in Table 3 and are not invoked as evidence for the central world-model claim. The possible geometric shortcut of rigidly reprojecting static occupancy with the provided pose is a real benchmark-validity concern, and the paper does not report such a zero-learning baseline, but this is an evaluation-protocol risk rather than a case where a prediction reduces to its input by construction. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from the authors, and no ansatz is smuggled in via self-citation. Accordingly, no circular step is established.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim is empirical and rests on the validity of the dataset, the benchmark construction, and standard deep learning assumptions. No free parameters are derived from a theory; the listed parameters are hand-chosen architecture and training choices. The paper does not introduce new physical entities.

free parameters (5)
  • Number of history frames T = 16
    Hand-chosen; training uses 17 consecutive frames with first 16 as history (Section 4.3). No sensitivity analysis is provided, and autoregressive evaluation uses only 6 history frames.
  • HSTA kernel sizes = Long-term temporal 7, short-term temporal 3, spatial 7/3
    Chosen by hand for multi-scale windows (Eqs 3-6); no ablation over kernel sizes is reported.
  • Codebook size = 512 codes, 64 channels
    VQ-VAE tokenizer design choice (Section 4.3); the reconstruction ceiling (73.39 IoU) bounds prediction quality.
  • Number of layers = 6 encoder + 6 decoder layers, 2 ResNet middle blocks
    Architecture capacity choice (Section 4.3); no depth ablation.
  • Training schedule = 200 epochs, LR 1e-3, AdamW, 8 A100 GPUs
    Standard training choices; not justified by analysis.
assumptions (5)
  • domain assumption The dataset's occupancy annotations and camera poses are treated as ground truth.
    Section 3.1 uses Irgb, E, K and occupancy labels from ScanNet-derived benchmarks; errors in pose or labels would propagate to all results.
  • domain assumption The scene-level train/val split is free of contamination.
    Section 4.1 states scenes in training differ from validation, but the restructure procedure is not detailed enough to verify.
  • domain assumption The next camera pose is available at inference and is accurate.
    Eq. (2) conditions on P; robustness to pose noise is not tested.
  • domain assumption VQ-VAE tokenization preserves enough information for the prediction task.
    The world model operates on discrete tokens; reconstruction IoU of 73.39 bounds the achievable prediction IoU.
  • standard math Standard deep learning building blocks (transformer, 3D convolution, VQ-VAE) work as expected.
    The method relies on established architectures without formal guarantees.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Occupancy World Model for Robots." pith.science (2026). https://pith.science/paper/GTN4A3OH

@misc{pith2026250505512,
  author       = {Pith},
  title        = {Pith review of: Occupancy World Model for Robots},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GTN4A3OH}},
  note         = {Machine review of arXiv:2505.05512}
}
read the original abstract

Understanding and forecasting the scene evolutions deeply affect the exploration and decision of embodied agents. While traditional methods simulate scene evolutions through trajectory prediction of potential instances, current works use the occupancy world model as a generative framework for describing fine-grained overall scene dynamics. However, existing methods cluster on the outdoor structured road scenes, while ignoring the exploration of forecasting 3D occupancy scene evolutions for robots in indoor scenes. In this work, we explore a new framework for learning the scene evolutions of observed fine-grained occupancy and propose an occupancy world model based on the combined spatio-temporal receptive field and guided autoregressive transformer to forecast the scene evolutions, called RoboOccWorld. We propose the Conditional Causal State Attention (CCSA), which utilizes camera poses of next state as conditions to guide the autoregressive transformer to adapt and understand the indoor robotics scenarios. In order to effectively exploit the spatio-temporal cues from historical observations, Hybrid Spatio-Temporal Aggregation (HSTA) is proposed to obtain the combined spatio-temporal receptive field based on multi-scale spatio-temporal windows. In addition, we restructure the OccWorld-ScanNet benchmark based on local annotations to facilitate the evaluation of the indoor 3D occupancy scene evolution prediction task. Experimental results demonstrate that our RoboOccWorld outperforms state-of-the-art methods in indoor 3D occupancy scene evolution prediction task. The code will be released soon.

Figures

Figures reproduced from arXiv: 2505.05512 by the authors.

Figure 1
Figure 1. Given previous 3D occupancy observations and next-step trajectories, our occupancy World [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The indoor monocular images of dif￾ferent sequences. The red arrows corresponding to the images represent the direction of camera movement. Further, existing method [14] utilizes spatial ag￾gregation and temporal causal attention to ob￾tain the spatio-temporal interaction. As shown in [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The comparison between existing methods and our method on spatio-temporal interaction. The different colored blocks represent different windows and T is the time dimension. Our experimental results show that RoboOccWorld outperforms state-of-the-art methods in indoor 3D occupancy scene evolution prediction task by a substantial margin in both next state occupancy prediction and autoregressive occupancy prediction ta… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The Framework of RoboOccWorld. The RoboOccWorld is divided into 2 stages. In the first stage, given the current occupancy Ocurrent, we employ the VQ-VAE as the tokenizer for occupancy scene representation. In the second stage, we partition the scene tokens into multi-s…
Figure 5
Figure 5. Figure 5: The pipeline of the proposed HSTA. Hybrid Spatio-temporal Aggregation. Relying only on spatial ag￾gregation within frames and grid-to-grid spatio-temporal interaction between frames is not enough. Therefore, we propose multi-scale spatio-temporal interactions based on …
Figure 6
Figure 6. Figure 6: The pipeline of the proposed CCSA. Where the Causal3DConvS1 and Causal3DConvS2 denote the causal 3D convolution layers with kernel size 3 in temporal dimen￾sion and 7 or 3 in the spatial dimension. The short-term windows allow us to obtain the larger spatial receptive …
Figure 7
Figure 7. Figure 7: Qualitative Analysis on the OccWorld-ScanNet dataset in next state occupancy prediction task. The red arrows represent the next pose of the camera. It can be seen that model can successfully understand the pose changes of the camera, which utilizes the pose changes and…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. O3N: Omnidirectional Open-Vocabulary Occupancy Prediction for Urban Autonomous Agents

    cs.CV 2026-03 conditional novelty 6.5 of 10

    O3N is the first open-vocabulary occupancy prediction method that takes a single omnidirectional RGB image and labels 3D voxels with both seen and unseen semantic classes.

  2. A Comprehensive Survey on World Models for Embodied AI

    cs.CV 2025-10 conditional novelty 6.0 of 10

    A unified three-axis taxonomy — functionality, temporal modeling, spatial representation — organizes the world-model literature for embodied AI.

  3. Quo Vadis, World Modeling?

    cs.CV 2026-08 conditional novelty 5.0 of 10

    An agent-centric reframing of world modeling, replacing physical state prediction with 'information transitions' organized into six proxy functions and three empowerment levels.

  4. A Definition and Roadmap for World Models

    cs.AI 2026-07 conditional novelty 5.0 of 10

    A perspective article defining world models as finite-resource compression of physical state transitions and outlining a roadmap toward physical AGI via unified representations and interactive simulators.

  5. Humanoid Occupancy: Enabling A Generalized Multimodal Occupancy Perception System on Humanoid Robots

    cs.RO 2025-07 conditional novelty 5.0 of 10

    A humanoid-specific multimodal occupancy perception system with a new dataset, sensor layout, and a fusion network that claims state-of-the-art results on its own benchmark.

Reference graph

Works this paper leans on

37 extracted references · 23 canonical work pages · cited by 5 Pith papers

  1. [1]

    Deep hough voting for 3d object detection in point clouds

    Charles R Qi, Or Litany, Kaiming He, and Leonidas J Guibas. Deep hough voting for 3d object detection in point clouds. In proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9277–9286, 2019. 2

    work page 2019

  2. [2]

    Aligning cyber space with physical world: A comprehensive survey on embodied ai

    Yang Liu, Weixing Chen, Yongjie Bai, Xiaodan Liang, Guanbin Li, Wen Gao, and Liang Lin. Aligning cyber space with physical world: A comprehensive survey on embodied ai. arXiv preprint arXiv:2407.06886, 2024. 2

    arXiv 2024

  3. [3]

    Imvotenet: Boosting 3d object detection in point clouds with image votes

    Charles R Qi, Xinlei Chen, Or Litany, and Leonidas J Guibas. Imvotenet: Boosting 3d object detection in point clouds with image votes. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4404–4413, 2020. 2

    work page 2020

  4. [4]

    Indoor semantic segmentation using depth information

    Camille Couprie, Clément Farabet, Laurent Najman, and Yann LeCun. Indoor semantic segmentation using depth information. arXiv preprint arXiv:1301.3572, 2013. 2

    arXiv 2013

  5. [5]

    Humanoidpano: Hybrid spherical panoramic-lidar cross-modal perception for humanoid robots

    Qiang Zhang, Zhang Zhang, Wei Cui, Jingkai Sun, Jiahang Cao, Yijie Guo, Gang Han, Wen Zhao, Jiaxu Wang, Chenghao Sun, et al. Humanoidpano: Hybrid spherical panoramic-lidar cross-modal perception for humanoid robots. arXiv preprint arXiv:2503.09010, 2025. 2

    arXiv 2025

  6. [6]

    V olumetric environment representation for vision- language navigation

    Rui Liu, Wenguan Wang, and Yi Yang. V olumetric environment representation for vision- language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16317–16328, 2024. 2

    work page 2024

  7. [7]

    Embodiedscan: A holistic multi-modal 3d perception suite towards embodied ai

    Tai Wang, Xiaohan Mao, Chenming Zhu, Runsen Xu, Ruiyuan Lyu, Peisen Li, Xiao Chen, Wenwei Zhang, Kai Chen, Tianfan Xue, et al. Embodiedscan: A holistic multi-modal 3d perception suite towards embodied ai. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19757–19767, 2024. 2

    work page 2024

  8. [8]

    Embod- iedsam: Online segment any 3d thing in real time

    Xiuwei Xu, Huangxing Chen, Linqing Zhao, Ziwei Wang, Jie Zhou, and Jiwen Lu. Embod- iedsam: Online segment any 3d thing in real time. arXiv preprint arXiv:2408.11811, 2024. 2

    arXiv 2024

Show all 37 references
  1. [9]

    Trajectron++: Dynamically-feasible trajectory forecasting with heterogeneous data

    Tim Salzmann, Boris Ivanovic, Punarjay Chakravarty, and Marco Pavone. Trajectron++: Dynamically-feasible trajectory forecasting with heterogeneous data. In Computer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVIII 16, pages ...

    2020

  2. [10]

    Transformer networks for trajectory forecasting

    Francesco Giuliari, Irtiza Hasan, Marco Cristani, and Fabio Galasso. Transformer networks for trajectory forecasting. In 2020 25th international conference on pattern recognition (ICPR), pages 10335–10342. IEEE, 2021. 2, 3

    2020

  3. [11]

    Scene transformer: A unified architecture for predicting multiple agent trajectories

    Jiquan Ngiam, Benjamin Caine, Vijay Vasudevan, Zhengdong Zhang, Hao-Tien Lewis Chiang, Jeffrey Ling, Rebecca Roelofs, Alex Bewley, Chenxi Liu, Ashish Venugopal, et al. Scene transformer: A unified architecture for predicting multiple agent trajectories. arXiv preprint arXiv:21...

    2021 arXiv

  4. [12]

    Hivt: Hierarchical vector transformer for multi-agent motion prediction

    Zikang Zhou, Luyao Ye, Jianping Wang, Kui Wu, and Kejie Lu. Hivt: Hierarchical vector transformer for multi-agent motion prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8823–8833, 2022. 2, 3

    2022

  5. [13]

    Hpnet: Dynamic trajectory forecasting with historical prediction attention

    Xiaolong Tang, Meina Kan, Shiguang Shan, Zhilong Ji, Jinfeng Bai, and Xilin Chen. Hpnet: Dynamic trajectory forecasting with historical prediction attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15261–15270, 2024. 2, 3

    2024

  6. [14]

    Occworld: Learning a 3d occupancy world model for autonomous driving

    Wenzhao Zheng, Weiliang Chen, Yuanhui Huang, Borui Zhang, Yueqi Duan, and Jiwen Lu. Occworld: Learning a 3d occupancy world model for autonomous driving. In European conference on computer vision, pages 55–72. Springer, 2024. 2, 3, 6, 7

    2024

  7. [15]

    Occsora: 4d occupancy generation models as world simulators for autonomous driving

    Lening Wang, Wenzhao Zheng, Yilong Ren, Han Jiang, Zhiyong Cui, Haiyang Yu, and Jiwen Lu. Occsora: 4d occupancy generation models as world simulators for autonomous driving. arXiv preprint arXiv:2405.20337, 2024. 2, 3 10

    2024 arXiv

  8. [16]

    An efficient occupancy world model via decoupled dynamic flow and image-assisted training

    Haiming Zhang, Ying Xue, Xu Yan, Jiacheng Zhang, Weichao Qiu, Dongfeng Bai, Bingbing Liu, Shuguang Cui, and Zhen Li. An efficient occupancy world model via decoupled dynamic flow and image-assisted training. arXiv preprint arXiv:2412.13772, 2024. 2, 3

    2024 arXiv

  9. [17]

    Occllama: An occupancy-language-action generative world model for autonomous driving

    Julong Wei, Shanshuai Yuan, Pengfei Li, Qingda Hu, Zhongxue Gan, and Wenchao Ding. Occllama: An occupancy-language-action generative world model for autonomous driving. arXiv preprint arXiv:2409.03272, 2024. 2, 3

    2024 arXiv

  10. [18]

    Driving in the occupancy world: Vision-centric 4d occupancy forecasting and planning via world models for autonomous driving

    Yu Yang, Jianbiao Mei, Yukai Ma, Siliang Du, Wenqing Chen, Yijie Qian, Yuxiang Feng, and Yong Liu. Driving in the occupancy world: Vision-centric 4d occupancy forecasting and planning via world models for autonomous driving. In Proceedings of the AAAI Conference on Artificial ...

    2025

  11. [19]

    Renderworld: World model with self-supervised 3d label

    Ziyang Yan, Wenzhen Dong, Yihua Shao, Yuhang Lu, Liu Haiyang, Jingwen Liu, Haozhe Wang, Zhe Wang, Yan Wang, Fabio Remondino, et al. Renderworld: World model with self-supervised 3d label. arXiv preprint arXiv:2409.11356, 2024. 2, 3

    2024 arXiv

  12. [20]

    Occ-llm: Enhancing autonomous driving with occupancy-based large language models

    Tianshuo Xu, Hao Lu, Xu Yan, Yingjie Cai, Bingbing Liu, and Yingcong Chen. Occ-llm: Enhancing autonomous driving with occupancy-based large language models. arXiv preprint arXiv:2502.06419, 2025. 2, 3

    2025 arXiv

  13. [21]

    Dome: Taming diffusion model into high-fidelity controllable occupancy world model

    Songen Gu, Wei Yin, Bu Jin, Xiaoyang Guo, Junming Wang, Haodong Li, Qian Zhang, and Xiaoxiao Long. Dome: Taming diffusion model into high-fidelity controllable occupancy world model. arXiv preprint arXiv:2410.10429, 2024. 2, 3

    2024 arXiv

  14. [22]

    Monoscene: Monocular 3d semantic scene completion

    Anh-Quan Cao and Raoul De Charette. Monoscene: Monocular 3d semantic scene completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3991–4001, 2022. 3, 6

    2022

  15. [23]

    Tri-perspective view for vision-based 3d semantic occupancy prediction

    Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. Tri-perspective view for vision-based 3d semantic occupancy prediction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9223–9232, 2023. 3

    2023

  16. [24]

    Surroundocc: Multi-camera 3d occupancy prediction for autonomous driving

    Yi Wei, Linqing Zhao, Wenzhao Zheng, Zheng Zhu, Jie Zhou, and Jiwen Lu. Surroundocc: Multi-camera 3d occupancy prediction for autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 21729–21740, 2023. 3

    2023

  17. [25]

    Planning-oriented autonomous driving

    Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al. Planning-oriented autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17853–17862, 2023. 3

    2023

  18. [26]

    Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving

    Xiaoyu Tian, Tao Jiang, Longfei Yun, Yucheng Mao, Huitong Yang, Yue Wang, Yilun Wang, and Hang Zhao. Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving. Advances in Neural Information Processing Systems, 36:64318–64330, 2023. 3

    2023

  19. [27]

    Openoccupancy: A large scale benchmark for surrounding semantic occupancy perception

    Xiaofeng Wang, Zheng Zhu, Wenbo Xu, Yunpeng Zhang, Yi Wei, Xu Chi, Yun Ye, Dalong Du, Jiwen Lu, and Xingang Wang. Openoccupancy: A large scale benchmark for surrounding semantic occupancy perception. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p...

    2023

  20. [28]

    Indoor segmentation and support inference from rgbd images

    Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In Computer Vision–ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part V 12, pages 746–760. Spr...

    2012

  21. [29]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017. 3

    2017

  22. [30]

    Scancomplete: Large-scale scene completion and semantic segmentation for 3d scans

    Angela Dai, Daniel Ritchie, Martin Bokeloh, Scott Reed, Jürgen Sturm, and Matthias Nießner. Scancomplete: Large-scale scene completion and semantic segmentation for 3d scans. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 4578–4587, 2018. 3 11

    2018

  23. [31]

    Monocular occupancy predic- tion for scalable indoor scenes

    Hongxiao Yu, Yuqi Wang, Yuntao Chen, and Zhaoxiang Zhang. Monocular occupancy predic- tion for scalable indoor scenes. In European Conference on Computer Vision, pages 38–54. Springer, 2024. 3, 6

    2024

  24. [32]

    Embodiedocc: Embodied 3d occupancy prediction for vision-based online scene understanding

    Yuqi Wu, Wenzhao Zheng, Sicheng Zuo, Yuanhui Huang, Jie Zhou, and Jiwen Lu. Embodiedocc: Embodied 3d occupancy prediction for vision-based online scene understanding. arXiv preprint arXiv:2412.04380, 2024. 3, 4, 6, 9

    2024 arXiv

  25. [33]

    Roboocc: Enhancing the geometric and semantic scene understanding for robots

    Zhang Zhang, Qiang Zhang, Wei Cui, Shuai Shi, Yijie Guo, Gang Han, Wen Zhao, Hengle Ren, Renjing Xu, and Jian Tang. Roboocc: Enhancing the geometric and semantic scene understanding for robots. arXiv preprint arXiv:2504.14604, 2025. 3, 4, 9

    2025 arXiv

  26. [34]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017. 3, 4, 7, 9

    2017

  27. [35]

    Gaussianformer: Scene as gaussians for vision-based 3d semantic occupancy prediction

    Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. Gaussianformer: Scene as gaussians for vision-based 3d semantic occupancy prediction. In European Conference on Computer Vision, pages 376–393. Springer, 2024. 6

    2024

  28. [36]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 7

    2017 arXiv

  29. [37]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 7 12

    2016

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.