Pith. sign in

REVIEW 5 major objections 5 minor 45 references

SDGOCC: Semantic and Depth-Guided Bird's-Eye View Transformation for 3D Multimodal Occupancy Prediction

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A semantic-and-depth-guided bird's-eye view transformation lets a multimodal occupancy network reach 51.66 mIoU on Occ3D-nuScenes, while a distilled camera-only variant runs at 83 ms per frame.

desk verdict Solid incremental SOTA on Occ3D with a plausible new depth-guided view transformation, but the key depth-diffusion assumption is unvalidated and the missing code/error bars keep the result conditional. read the letter →

arxiv 2507.17083 v1 pith:XTNRKIZQ submitted 2025-07-22 cs.CV cs.AI

Zaipeng Duan , Chenxu Dang , Xuzhong Hu , Pei An , Junfeng Ding , Jie Zhan , Yunbiao Xu , Jie Ma This is my paper
classification cs.CVcs.AI
keywords 3Dsemanticoccupancypredictionmultimodalfusionbird's-eyeviewtransformationLiDAR-cameraknowledgedistillationdepthestimationautonomousdriving
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the main bottleneck in camera-to-BEV lifting is unreliable per-pixel depth, and that LiDAR's sparse depth can fix this if it is diffused only across pixels that share a semantic label. The proposed SDG view transformation builds dense, accurately located virtual points, and a fusion-plus-distillation module then transfers multimodal knowledge into image features. On Occ3D-nuScenes, the fusion-only model SDG-Fusion reaches 51.66 mIoU, beating the previous best listed 49.38 mIoU, and the distilled SDG-KL reaches 50.16 mIoU at 83 ms per frame, which the paper calls real-time. If true, this would make high-accuracy 3D semantic occupancy prediction practical for autonomous driving with a compact ResNet-50 backbone.

What carries the argument

The load-bearing mechanism is the semantic-constrained depth diffusion of Eq. (1): for each pixel, LiDAR depth from nearby points is averaged only over neighbors whose semantic mask matches the center pixel, producing a semi-dense extended depth map, which is then turned into discrete virtual point seeds by bidirectional linear incremental discretization. These seeds reduce the number of wasted virtual points compared to LSS while improving depth accuracy. The second mechanism is the fusion-to-occupancy-driven active distillation, which fuses LiDAR and camera BEV features through gated neighborhood attention and then distills the fused features into the camera branch using active-region and inactive-region weighting.

What would settle it

On a validation batch from Occ3D-nuScenes, compare the virtual-point depths produced by Eq. (1) with the actual LiDAR returns at pixels near semantic boundaries or at stacked objects; if the depth error there is large, or if removing the semantic-mask constraint from the diffusion changes mIoU by less than the 10.67-point gain the paper attributes to SDG, the claimed mechanism is not doing the work.

Watch

Extended reading notes

Core claim

The central claim is that replacing the Lift-Splat-Shoot depth distribution with a semantic- and depth-guided view transformation—where LiDAR co-point depth is averaged over neighboring pixels of the same semantic class and then discretized bilinearly—produces image BEV features that are denser and better localized, and that this is what allows a multimodal network to outperform prior methods. With this transformation plus gated BEV fusion, SDG-Fusion reports 51.66 mIoU on Occ3D-nuScenes; with additional occupancy-driven distillation, SDG-KL reports 50.16 mIoU at 83 ms per frame, using only a camera branch at inference. The paper also reports comparable or better results on the larger-range SurroundOcc-nuScenes validation set.

Load-bearing premise

The method assumes that, within a semantic class, nearby pixels really do have the same depth; when a car occludes another car or a semantic mask is wrong at a boundary, the averaged depth is wrong, and the virtual points push image features into the wrong BEV cells.

Editorial extensions

If this is right

  • A compact ResNet-50 multimodal model can exceed the previously reported best multimodal occupancy mIoU on Occ3D-nuScenes by about 2.3 points, without a transformer backbone.
  • The distilled SDG-KL variant shows that most of the accuracy can be retained at 83 ms per frame using only the camera branch, which is fast enough for real-time driving systems.
  • The ablation attributes a 10.67-point mIoU gain to the SDG view transformation alone, implying that depth-quality is the dominant factor in the improvement, with fusion contributing a further 3.15 points.
  • The method transfers to the more challenging SurroundOcc-nuScenes benchmark, where the visible-mask versions report 52.2 mIoU, suggesting the view transformation generalizes across voxel sizes and prediction ranges.
  • Because the image branch is trained to absorb fused features through active distillation, the camera-only student may inherit robustness to LiDAR sparsity at long range, where the paper reports superior IoU over the baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The size of the SDG gain suggests most of the benefit comes from sharpening depth at BEV pooling; a class-wise breakdown would likely show the largest gains on large ground-surface classes and smaller gains on thin or occluded objects, which this paper does not analyze.
  • Since distillation transfers fused features only on LiDAR-identified regions, the camera-only SDG-KL may be blind to objects the LiDAR teacher never sees; testing it with LiDAR sweeps artificially removed would reveal how much safety margin remains.
  • The same semantic-constrained diffusion idea could be applied to radar point clouds or to temporal accumulation of sweeps, with the hard semantic-equality condition replaced by a depth-uncertainty-weighted average.
  • The gated neighborhood fusion module is a generic way to align misaligned BEV features across modalities and could be reused for camera-radar or camera-map fusion without retraining the view transformation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes SDG-OCC, a multimodal (camera+LiDAR) 3D occupancy prediction framework. The two core contributions are (i) a semantic and depth-guided (SDG) view transformation that replaces standard LSS-style depth sampling with semantic-conditional LiDAR-depth diffusion plus bilinear discretization to generate virtual points, and (ii) a fusion-to-occupancy-driven active distillation (FOAD) module that fuses camera and LiDAR BEV features and then distills the fused features into image features for faster inference. Two variants are reported: SDG-Fusion, which uses fusion at inference and reaches 51.66 mIoU at 133 ms on Occ3D-nuScenes, and SDG-KL, the distilled variant, which reaches 50.16 mIoU at 83 ms. The paper also reports results on SurroundOcc-nuScenes. Ablations attribute most of the gain to the SDG view transformation, with the baseline at 37.84 mIoU and the baseline plus SDG at 48.51 mIoU.

Significance. If the reported numbers are reproducible, the result is practically significant: a ResNet-50-based fusion model outperforms the previously listed best method (RadOcc-LC, Swin-B) by about 2.3 mIoU on Occ3D-nuScenes, and the distilled variant offers a real-time option at 83 ms per frame. The SDG view transformation is a simple, potentially transferable idea for injecting LiDAR depth priors into LSS-style lifting. The paper is clearly written and the architecture is easy to follow, and the main ablation is internally consistent. However, the significance is currently limited by the absence of multi-seed variance or released code, an undefined ablation baseline, inconsistent SurroundOcc evaluation protocols, and a lack of validation for the depth-diffusion assumption that underlies the largest performance gain.

major comments (5)
  1. [§4.4, Table 3] The ablation baseline is not defined. The text says 'The baseline achieves 90.27% of IoU and 37.84% of mIoU' but does not specify which view transformation is used, whether the LiDAR branch is present, or how the baseline differs from SDG-Fusion. Since Table 3 is the only evidence that SDG contributes +10.67 mIoU, the baseline must be described precisely (e.g., LSS-based lifting with the same image backbone and occupancy head, no LiDAR fusion) for the attribution to be interpretable.
  2. [§3.4, Eq. (1)] The semantic-conditional depth diffusion is load-bearing, but the paper provides no evidence about its failure modes. In Eq. (1), depths are averaged over neighbors with the same semantic label; when a depth discontinuity lies inside one semantic class (e.g., a car occluding another car) or when the predicted mask is wrong, the averaged depth creates a phantom virtual point, and the subsequent outer-product/BEV pooling projects image features into the wrong BEV cell. Because Table 3 shows that adding SDG alone raises mIoU from 37.84 to 48.51, please add (a) an ablation replacing the diffused depth with raw projected LiDAR depth (no diffusion), (b) a quantitative depth-error analysis at object boundaries and occlusion regions, and (c) representative failure cases, to establish that the gain is not an artifact of misplaced features.
  3. [§4.3, Table 2] The SurroundOcc SOTA claim mixes evaluation protocols. SDG-KL* and SDG-Fusion* (50.4 and 52.2 mIoU) use a visible mask, while the other multimodal methods listed (Co-Occ, OccFusion, DAOcc, and the unmasked SDG-Fusion at 31.7 mIoU) do not. The asterisk note is insufficient because visible-mask evaluation can substantially inflate mIoU. Please either evaluate all methods under the same protocol or explicitly restrict the SurroundOcc SOTA claim to the masked protocol and avoid head-to-head comparison with unmasked numbers.
  4. [§3.5 and Table 1] The SDG-KL inference configuration is ambiguous. The text says 'The image features F^{C}_{bev} are processed by the occupancy prediction head to obtain the SDG-KL model', which implies camera-only inference, but Table 1 lists the input as 'C+L' for SDG-KL. If SDG-KL is camera-only at inference, the Input column should say C and the 83 ms timing should be stated as camera-only; if LiDAR is still used, the real-time claim needs to include the LiDAR branch. Please clarify and make the table consistent.
  5. [§4.2 and Table 1] All key numbers are single runs with no reported variance and the code is not released. The claimed SOTA margin over RadOcc-LC is 2.28 mIoU; without either multi-seed standard deviations or a public code release, the reader cannot assess whether this margin is significant. Please report at least three seeds for the main models or provide the code and configuration files with the revision.
minor comments (5)
  1. [Eq. (4)] Equation (4) contains an unbalanced parenthesis: the expression for F^{fuse}_{bev} has an extra opening parenthesis and should be checked carefully.
  2. [Eq. (3)] In Eq. (3), the attention formula should define n(i) and k more explicitly, and the term 'B(i, n(i))√v' is unclear; please clarify the dimension and scaling notation.
  3. [§4.3 and Table 2] The footnote for Table 2 states that '*' means a visible mask 'similar to [30]', but the text in §4.3 says the masks are generated from [32]; please reconcile these references.
  4. [Figures 1 and 4] Figures 1 and 4 contain unrendered '/uni...' placeholder glyphs in the submitted PDF; these should be regenerated with a proper font so the figures are readable.
  5. [References [22] and [23]] The Related Work discusses OccFusion and cites both [22] and [23], which appear to be two versions of the same work; please cite the journal version consistently.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: SDG-OCC is evaluated against external ground-truth occupancy, Eq. (1) uses external LiDAR depth and predicted semantic masks, and no fitted parameter is renamed as a prediction.

full rationale

The paper makes no closed-form derivation that reduces one claimed result to its own inputs. The central performance claim, 51.66 mIoU on Occ3D-nuScenes, is measured against an external benchmark with ground-truth voxel labels; it is not derived from an equation that contains the target metric. Equation (1) constructs a semi-dense depth map by averaging LiDAR depth values over same-semantic neighbors, which is an empirical depth prior based on external sensor data and predicted semantic masks, not on the occupancy labels being predicted. Ablations in Tables 3-5 quantify measured contributions of components and hyperparameters rather than re-stating a fitted quantity. The fusion and distillation losses (Eqs. 3-8) are standard feature-level operations; distillation transfers teacher features to a student, and the student is then independently evaluated against ground truth, so the evaluation is not forced by construction. No load-bearing argument rests on a self-citation or on a uniqueness theorem imported from the authors' prior work. The weakness of Eq. (1) at same-class depth discontinuities or semantic-boundary errors is a correctness and robustness concern, not a circularity, because the depth values remain externally sourced and the occupancy evaluation is independent of the diffusion's internal validity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests mainly on the validity of LiDAR depth as a training prior and on the fairness of the benchmark protocol; no free-standing mathematical derivation is involved. The listed free parameters are hyperparameters tuned on the validation set used for the final SOTA table.

free parameters (4)
  • diffusion radius r = 1 m
    Radius of the circular neighborhood in Eq. (1); chosen by hyperparameter search in Table 4, best at r=1.
  • diffusion feature layers l = 8
    Number of diffusion feature layers in the SDG view transformation; chosen in Table 4, best at l=8.
  • neighborhood size k for fusion = 7
    Size of the local neighborhood in Eq. (3); chosen in Table 5, best at k=7.
  • distillation balance weights alpha and beta = not reported
    Balancing parameters in Eq. (7); the paper does not report their values or how they were set.
assumptions (4)
  • domain assumption LiDAR points projected onto the image provide accurate depth priors at those pixels.
    Used in Section 3.4 to build the sparse co-point depth map that is diffused into Dtemp in Eq. (1).
  • domain assumption Pixels sharing a semantic label in a local neighborhood have similar true depth.
    Eq. (1) averages depth only over same-mask neighbors; if depth discontinuities occur inside one semantic class, the diffused depth is wrong.
  • domain assumption Visible-mask evaluation on SurroundOcc is an acceptable protocol for comparing with methods evaluated without the mask.
    Table 2 uses star rows with visible masks for SDG and FlashOcc; the paper does not show all baselines under the same protocol.
  • domain assumption Pre-trained backbones such as ResNet-50 and SPVCNN generalize to occupancy prediction after fine-tuning.
    Section 4.2 adopts these backbones without justification; this is standard practice but still an unverified assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SDGOCC: Semantic and Depth-Guided Bird's-Eye View Transformation for 3D Multimodal Occupancy Prediction." pith.science (2026). https://pith.science/paper/XTNRKIZQ

@misc{pith2026250717083,
  author       = {Pith},
  title        = {Pith review of: SDGOCC: Semantic and Depth-Guided Bird's-Eye View Transformation for 3D Multimodal Occupancy Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XTNRKIZQ}},
  note         = {Machine review of arXiv:2507.17083}
}
read the original abstract

Multimodal 3D occupancy prediction has garnered significant attention for its potential in autonomous driving. However, most existing approaches are single-modality: camera-based methods lack depth information, while LiDAR-based methods struggle with occlusions. Current lightweight methods primarily rely on the Lift-Splat-Shoot (LSS) pipeline, which suffers from inaccurate depth estimation and fails to fully exploit the geometric and semantic information of 3D LiDAR points. Therefore, we propose a novel multimodal occupancy prediction network called SDG-OCC, which incorporates a joint semantic and depth-guided view transformation coupled with a fusion-to-occupancy-driven active distillation. The enhanced view transformation constructs accurate depth distributions by integrating pixel semantics and co-point depth through diffusion and bilinear discretization. The fusion-to-occupancy-driven active distillation extracts rich semantic information from multimodal data and selectively transfers knowledge to image features based on LiDAR-identified regions. Finally, for optimal performance, we introduce SDG-Fusion, which uses fusion alone, and SDG-KL, which integrates both fusion and distillation for faster inference. Our method achieves state-of-the-art (SOTA) performance with real-time processing on the Occ3D-nuScenes dataset and shows comparable performance on the more challenging SurroundOcc-nuScenes dataset, demonstrating its effectiveness and robustness. The code will be released at https://github.com/DzpLab/SDGOCC.

Figures

Figures reproduced from arXiv: 2507.17083 by the authors.

Figure 1
Figure 1. Comparisons of the mIoU and inference speed (FPS) of [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (a) BEV feature map of LSS with a shape of 200×200. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The overall architecture of SDG-OCC. T-frame multi-view images and corresponding point clouds are fed into the image and LiDAR backbones to extract features. Image features are processed by a multi-task head to generate semantic masks and depth distributions, combined with LiDAR depth maps to create virtual points for image BEV features. These features are fused with point cloud BEV features and selectively transfer… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The overall architecture of the SDG view transforma [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: (a) The pipeline of SDG-Fusion. (b) The pipeline [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Qualitative results of SDG on the validation set of Occ3D-nuScenes. Each pair of rows displays results from day and low-light [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Distance-based evaluation on Occ3D-nuScenes. As the [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 29 canonical work pages

  1. [1]

    Radardistill: Boosting radar-based ob- ject detection performance via knowledge distillation from lidar features

    Geonho Bang, Kwangjin Choi, Jisong Kim, Dongsuk Kum, and Jun Won Choi. Radardistill: Boosting radar-based ob- ject detection performance via knowledge distillation from lidar features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15491– 15500, 2024. 5

    work page 2024

  2. [2]

    Monoscene: Monoc- ular 3d semantic scene completion

    Anh-Quan Cao and Raoul De Charette. Monoscene: Monoc- ular 3d semantic scene completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3991–4001, 2022. 2

    2022

  3. [3]

    Eliminating cross-modal conflicts in bev space for lidar-camera 3d object detection

    Jiahui Fu, Chen Gao, Zitian Wang, Lirong Yang, Xiaofei Wang, Beipeng Mu, and Si Liu. Eliminating cross-modal conflicts in bev space for lidar-camera 3d object detection. In 2024 IEEE International Conference on Robotics and Au- tomation (ICRA), pages 16381–16387, 2024. 5

    work page 2024

  4. [4]

    Neighborhood attention transformer

    Ali Hassani, Steven Walton, Jiachen Li, Shen Li, and Humphrey Shi. Neighborhood attention transformer. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6185–6194, 2023. 5

    work page 2023

  5. [5]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 3

    2016

  6. [6]

    Fas- tocc: Accelerating 3d occupancy prediction by fusing the 2d bird’s-eye view and perspective view

    Jiawei Hou, Xiaoyan Li, Wenhao Guan, Gang Zhang, Di Feng, Yuheng Du, Xiangyang Xue, and Jian Pu. Fas- tocc: Accelerating 3d occupancy prediction by fusing the 2d bird’s-eye view and perspective view. arXiv preprint arXiv:2403.02710, 2024. 3, 7

    arXiv 2024

  7. [7]

    Planning-oriented autonomous driving

    Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al. Planning-oriented autonomous driving. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17853–17862, 2023. 1

    work page 2023

  8. [8]

    Bevdet4d: Exploit tempo- ral cues in multi-camera 3d object detection

    Junjie Huang and Guan Huang. Bevdet4d: Exploit tempo- ral cues in multi-camera 3d object detection. arXiv preprint arXiv:2203.17054, 2022. 7

    arXiv 2022

Show all 45 references
  1. [9]

    Tri-perspective view for vision- based 3d semantic occupancy prediction

    Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. Tri-perspective view for vision- based 3d semantic occupancy prediction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9223–9232, 2023. 2, 7

    2023

  2. [10]

    Driveadapter: Breaking the coupling barrier of perception and planning in end-to-end autonomous driving

    Xiaosong Jia, Yulu Gao, Li Chen, Junchi Yan, Patrick Langechuan Liu, and Hongyang Li. Driveadapter: Breaking the coupling barrier of perception and planning in end-to-end autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 795...

    2023

  3. [11]

    Spherical transformer for lidar-based 3d recognition

    Xin Lai, Yukang Chen, Fanbin Lu, Jianhui Liu, and Jiaya Jia. Spherical transformer for lidar-based 3d recognition. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17545–17555, 2023. 1

    2023

  4. [12]

    Mseg3d: Multi-modal 3d semantic segmentation for autonomous driv- ing

    Jiale Li, Hang Dai, Hao Han, and Yong Ding. Mseg3d: Multi-modal 3d semantic segmentation for autonomous driv- ing. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 21694–21704,

  5. [13]

    V oxformer: Sparse voxel transformer for camera- based 3d semantic scene completion

    Yiming Li, Zhiding Yu, Christopher Choy, Chaowei Xiao, Jose M Alvarez, Sanja Fidler, Chen Feng, and Anima Anand- kumar. V oxformer: Sparse voxel transformer for camera- based 3d semantic scene completion. In Proceedings of the IEEE/CVF conference on computer vision and pattern...

    2023

  6. [14]

    Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers

    Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chong- hao Sima, Tong Lu, Yu Qiao, and Jifeng Dai. Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers. In European con- ference on computer vision, pages 1–18. Springer, 2022. 1

    2022

  7. [15]

    Fb-occ: 3d occupancy prediction based on forward-backward view transformation

    Zhiqi Li, Zhiding Yu, David Austin, Mingsheng Fang, Shiyi Lan, Jan Kautz, and Jose M Alvarez. Fb-occ: 3d occupancy prediction based on forward-backward view transformation. arXiv preprint arXiv:2307.01492, 2023. 3

    2023 arXiv

  8. [16]

    Fb-bev: Bev representation from forward-backward view transformations

    Zhiqi Li, Zhiding Yu, Wenhai Wang, Anima Anandkumar, Tong Lu, and Jose M Alvarez. Fb-bev: Bev representation from forward-backward view transformations. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 6919–6928, 2023. 2, 7

    2023

  9. [17]

    Bevfusion: A simple and robust lidar-camera fusion framework

    Tingting Liang, Hongwei Xie, Kaicheng Yu, Zhongyu Xia, Zhiwei Lin, Yongtao Wang, Tao Tang, Bing Wang, and Zhi Tang. Bevfusion: A simple and robust lidar-camera fusion framework. Advances in Neural Information Processing Sys- tems, 35:10421–10434, 2022. 1

    2022

  10. [18]

    Feature pyra- mid networks for object detection

    Tsung-Yi Lin, Piotr Doll ´ar, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyra- mid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 2117–2125, 2017. 3

    2017

  11. [19]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 3

    2021

  12. [20]

    Bevfusion: Multi- task multi-sensor fusion with unified bird’s-eye view repre- sentation

    Zhijian Liu, Haotian Tang, Alexander Amini, Xinyu Yang, Huizi Mao, Daniela L Rus, and Song Han. Bevfusion: Multi- task multi-sensor fusion with unified bird’s-eye view repre- sentation. In 2023 IEEE international conference on robotics and automation (ICRA), pages 2774–2781. I...

    2023

  13. [21]

    Cotr: Compact occupancy transformer for vision-based 3d occupancy prediction

    Qihang Ma, Xin Tan, Yanyun Qu, Lizhuang Ma, Zhizhong Zhang, and Yuan Xie. Cotr: Compact occupancy transformer for vision-based 3d occupancy prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19936–19945, 2024. 3, 7

    2024

  14. [22]

    Occfusion: A straightforward and effective multi-sensor fusion framework for 3d occupancy prediction

    Zhenxing Ming, Julie Stephany Berrio, Mao Shan, and Stew- art Worrall. Occfusion: A straightforward and effective multi-sensor fusion framework for 3d occupancy prediction. arXiv preprint arXiv:2403.01644, 2024. 3

    2024 arXiv

  15. [23]

    Occfusion: Multi-sensor fusion framework for 3d semantic occupancy prediction

    Zhenxing Ming, Julie Stephany Berrio, Mao Shan, and Stew- art Worrall. Occfusion: Multi-sensor fusion framework for 3d semantic occupancy prediction. IEEE Transactions on Intelligent Vehicles, 2024. 7

    2024

  16. [24]

    Co-occ: Coupling explicit feature fusion with volume rendering regularization for multi-modal 3d semantic occupancy prediction

    Jingyi Pan, Zipeng Wang, and Lin Wang. Co-occ: Coupling explicit feature fusion with volume rendering regularization for multi-modal 3d semantic occupancy prediction. IEEE Robotics and Automation Letters, 2024. 3, 7

    2024

  17. [25]

    Renderocc: Vision-centric 3d occupancy prediction with 2d rendering supervision

    Mingjie Pan, Jiaming Liu, Renrui Zhang, Peixiang Huang, Xiaoqi Li, Li Liu, and Shanghang Zhang. Renderocc: Vision-centric 3d occupancy prediction with 2d rendering supervision. arXiv preprint arXiv:2309.09502, 2023. 3

    2023 arXiv

  18. [26]

    Uniocc: Unifying vision-centric 3d oc- cupancy prediction with geometric and semantic rendering

    Mingjie Pan, Li Liu, Jiaming Liu, Peixiang Huang, Lon- glong Wang, Shanghang Zhang, Shaoqing Xu, Zhiyi Lai, and Kuiyuan Yang. Uniocc: Unifying vision-centric 3d oc- cupancy prediction with geometric and semantic rendering. arXiv preprint arXiv:2306.09117, 2023. 3

    2023 arXiv

  19. [27]

    Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unproject- ing to 3d

    Jonah Philion and Sanja Fidler. Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unproject- ing to 3d. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceed- ings, Part XIV 16, pages 194–210. Springer, 2020. 1

    2020

  20. [28]

    Categorical depth distribution network for monocular 3d object detection

    Cody Reading, Ali Harakeh, Julia Chae, and Steven L Waslander. Categorical depth distribution network for monocular 3d object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8555–8564, 2021. 3

    2021

  21. [29]

    Graphbev: To- wards robust bev feature alignment for multi-modal 3d ob- ject detection

    Ziying Song, Lei Yang, Shaoqing Xu, Lin Liu, Dongyang Xu, Caiyan Jia, Feiyang Jia, and Li Wang. Graphbev: To- wards robust bev feature alignment for multi-modal 3d ob- ject detection. arXiv preprint arXiv:2403.11848, 2024. 5

    2024 arXiv

  22. [30]

    Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving

    Xiaoyu Tian, Tao Jiang, Longfei Yun, Yucheng Mao, Huitong Yang, Yue Wang, Yilun Wang, and Hang Zhao. Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving. Advances in Neural Information Processing Systems, 36, 2024. 1, 2, 6, 7

    2024

  23. [31]

    Occgen: Gener- ative multi-modal 3d occupancy prediction for autonomous driving

    Guoqing Wang, Zhongdao Wang, Pin Tang, Jilai Zheng, Xi- angxuan Ren, Bailan Feng, and Chao Ma. Occgen: Gener- ative multi-modal 3d occupancy prediction for autonomous driving. arXiv preprint arXiv:2404.15014, 2024. 3

    2024 arXiv

  24. [32]

    Openoccupancy: A large scale benchmark for surrounding semantic occupancy perception

    Xiaofeng Wang, Zheng Zhu, Wenbo Xu, Yunpeng Zhang, Yi Wei, Xu Chi, Yun Ye, Dalong Du, Jiwen Lu, and Xin- gang Wang. Openoccupancy: A large scale benchmark for surrounding semantic occupancy perception. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sio...

    2023

  25. [33]

    Panoocc: Unified occupancy representation for camera-based 3d panoptic segmentation

    Yuqi Wang, Yuntao Chen, Xingyu Liao, Lue Fan, and Zhaox- iang Zhang. Panoocc: Unified occupancy representation for camera-based 3d panoptic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17158–17168, 2024. 7

    2024

  26. [34]

    Nerf–: Neural radiance fields without known camera parameters

    Zirui Wang, Shangzhe Wu, Weidi Xie, Min Chen, and Victor Adrian Prisacariu. Nerf–: Neural radiance fields without known camera parameters. arXiv preprint arXiv:2102.07064, 2021. 3

    2021 arXiv

  27. [35]

    Surroundocc: Multi-camera 3d occu- pancy prediction for autonomous driving

    Yi Wei, Linqing Zhao, Wenzhao Zheng, Zheng Zhu, Jie Zhou, and Jiwen Lu. Surroundocc: Multi-camera 3d occu- pancy prediction for autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 21729–21740, 2023. 2, 6, 7

    2023

  28. [36]

    Unleashing hydra: Hybrid fusion, depth consis- tency and radar for unified 3d perception

    Philipp Wolters, Johannes Gilg, Torben Teepe, Fabian Herzog, Anouar Laouichi, Martin Hofmann, and Gerhard Rigoll. Unleashing hydra: Hybrid fusion, depth consis- tency and radar for unified 3d perception. arXiv preprint arXiv:2403.07746, 2024. 3, 7

    2024 arXiv

  29. [37]

    Point transformer v3: Simpler faster stronger

    Xiaoyang Wu, Li Jiang, Peng-Shuai Wang, Zhijian Liu, Xi- hui Liu, Yu Qiao, Wanli Ouyang, Tong He, and Hengshuang Zhao. Point transformer v3: Simpler faster stronger. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4840–4851, 2024. 1

    2024

  30. [38]

    Sparsefusion: Fusing multi-modal sparse rep- resentations for multi-sensor 3d object detection

    Yichen Xie, Chenfeng Xu, Marie-Julie Rakotosaona, Patrick Rim, Federico Tombari, Kurt Keutzer, Masayoshi Tomizuka, and Wei Zhan. Sparsefusion: Fusing multi-modal sparse rep- resentations for multi-sensor 3d object detection. InProceed- ings of the IEEE/CVF International Confer...

    2023

  31. [39]

    2dpass: 2d priors assisted semantic segmentation on lidar point clouds

    Xu Yan, Jiantao Gao, Chaoda Zheng, Chao Zheng, Ruimao Zhang, Shuguang Cui, and Zhen Li. 2dpass: 2d priors assisted semantic segmentation on lidar point clouds. In European Conference on Computer Vision, pages 677–695. Springer, 2022. 1

    2022

  32. [40]

    Daocc: 3d ob- ject detection assisted multi-sensor fusion for 3d occupancy prediction

    Zhen Yang, Yanpeng Dong, and Heng Wang. Daocc: 3d ob- ject detection assisted multi-sensor fusion for 3d occupancy prediction. arXiv preprint arXiv:2409.19972, 2024. 7

    2024

  33. [41]

    Is-fusion: Instance-scene collaborative fusion for multimodal 3d ob- ject detection

    Junbo Yin, Jianbing Shen, Runnan Chen, Wei Li, Ruigang Yang, Pascal Frossard, and Wenguan Wang. Is-fusion: Instance-scene collaborative fusion for multimodal 3d ob- ject detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1490...

    2024

  34. [42]

    Flashocc: Fast and memory-efficient occupancy prediction via channel-to-height plugin

    Zichen Yu, Changyong Shu, Jiajun Deng, Kangjie Lu, Zong- dai Liu, Jiangyong Yu, Dawei Yang, Hui Li, and Yan Chen. Flashocc: Fast and memory-efficient occupancy prediction via channel-to-height plugin. arXiv preprint arXiv:2311.12058, 2023. 2, 7

    2023 arXiv

  35. [43]

    Radocc: Learning cross-modality occupancy knowledge through ren- dering assisted distillation

    Haiming Zhang, Xu Yan, Dongfeng Bai, Jiantao Gao, Pan Wang, Bingbing Liu, Shuguang Cui, and Zhen Li. Radocc: Learning cross-modality occupancy knowledge through ren- dering assisted distillation. In Proceedings of the AAAI Con- ference on Artificial Intelligence, pages 7060–70...

    2024

  36. [44]

    Occformer: Dual-path transformer for vision-based 3d semantic occu- pancy prediction

    Yunpeng Zhang, Zheng Zhu, and Dalong Du. Occformer: Dual-path transformer for vision-based 3d semantic occu- pancy prediction. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 9433–9443,

  37. [45]

    Cylindrical and asymmetrical 3d convolution networks for lidar seg- mentation

    Xinge Zhu, Hui Zhou, Tai Wang, Fangzhou Hong, Yuexin Ma, Wei Li, Hongsheng Li, and Dahua Lin. Cylindrical and asymmetrical 3d convolution networks for lidar seg- mentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9939–9948,

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.