Pith. sign in

REVIEW 4 major objections 6 minor 54 references

A causal loss that supervises which 2D pixels drive each 3D class makes the entire 2D-to-3D lift end-to-end trainable, including camera parameters, and sharply reduces sensitivity to camera perturbation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 20:40 UTC pith:IS5PUV3S

load-bearing objection Robustness claim confounded by noise augmentation, but the causal-loss idea is novel and clean-table gains are consistent. the 4 major comments →

arxiv 2509.08388 v1 pith:IS5PUV3S submitted 2025-09-10 cs.CV cs.AI

Semantic Causality-Aware Vision-Based 3D Occupancy Prediction

Dubing Chen , Huan Zheng , Yucheng Zhou , Xianfei Li , Wenlong Liao , Tao He , Pai Peng , Jianbing Shen This is my paper
classification cs.CV cs.AI
keywords 3D semantic occupancy predictionLift-Splat-Shootcausal losssemantic causal locality2D-to-3D view transformationcamera perturbation robustnessOcc3D benchmark
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Vision-based 3D semantic occupancy prediction — producing a dense, class-labeled 3D voxel grid from surround cameras — has usually been built as a modular pipeline: a depth distribution lifts 2D features to 3D using fixed, pre-calibrated camera parameters. The paper argues this modularity is the core defect, since a feature of one class can be projected onto a 3D location of another class, and the fixed mapping cannot be optimized for the final semantic task. Its central claim is that a "causal loss" can supervise the information flow of the lift itself: for a sampled semantic class, the gradient of that class's aggregated 3D features back to the 2D feature map is computed and pushed, through binary cross-entropy, toward the 2D semantic mask. Combined with an architecture that keeps those gradient maps inside a stable [0,1] range, this makes the whole 2D-to-3D stage — camera parameters included — trainable end to end; if correct, the reported gains (higher mIoU on the Occ3D benchmark and a relative mIoU drop under camera noise cut from about -32% to about -7%) translate directly into more reliable autonomous perception under real-world pose error.

Core claim

The paper's central claim is that semantic causal locality — a 3D voxel predicted as class s should be influenced mainly by 2D pixels of that same class — can be enforced as a training objective. The Causal Loss (Eq. 14) samples a class s, backpropagates the class-aggregated 3D features to the 2D feature map, and applies a BCE loss between the resulting channel-averaged attention map A_s and the 2D semantic mask. The paper argues this "regularizes the information flow" of the lift and makes previously fixed components, such as camera parameters, learnable. It instantiates the principle in the Semantic Causality-Aware Transformation (SCAT): channel-grouped lifting weights, learnable camera of

What carries the argument

The load-bearing object is the Causal Loss of Eq. (14), a gradient-as-saliency proxy: for a uniformly sampled class s, it aggregates the 3D voxels whose label is s, backpropagates them to the 2D feature map, averages over channels to get an attention map A_s (Eqs. 10-11), and supervises it with BCE against the 2D semantic mask (Eq. 12). The loss leans on Eq. (9), which states the gradient of a lifted 3D feature with respect to the 2D feature is just the depth probability p_d times the identity. To keep that signal well-behaved, SCAT constrains every lifting operation so the gradient maps stay in [0,1]: channel-grouped lifting weights replace LSS's uniform p_d, trilinear "soft filling" makes

Load-bearing premise

The scheme rests on the assumption that the backpropagated class-attention map is a faithful measure of which 2D pixels cause a given 3D class — but in the implemented pipeline the geometry network also reads those pixels, adding a gradient term that the derivation in Eq. (9) omits, and the proof that the supervised maps stay in the stable [0,1] range is deferred to a supplement that is not present.

What would settle it

Compare the surrogate gradient map of Eq. (9) with the full autograd map at a fixed checkpoint (the full map includes the geometry network's dp_d/df_i term). If the two diverge strongly where the BCE loss supervises, or if the full map leaves the claimed [0,1] range, the loss is not enforcing semantic causal locality as stated. A controlled version: stop gradients through the geometry network so Eq. (9) holds literally — if accuracy and noise robustness are unchanged, the omitted term is irrelevant; if they drop, the paper's stated mechanism is not what produces the gains.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Camera parameters become a runtime-corrected quantity rather than a trusted input: the learnable offsets absorb calibration error and pose jitter during training, which is the direct explanation for the sharp improvement in camera-noise robustness.
  • Proxy depth supervision is no longer the only handle on the lift's geometry: the 2D-to-3D mapping is now optimized for the semantic consistency the occupancy task actually needs, so depth and geometry modules can be retrained or re-targeted.
  • The method transfers across architectures: adding it to two different LSS-style baselines (BEVDetOcc and ALOcc) improves both, indicating it addresses a shared failure of the lifting paradigm rather than an artifact of one network.
  • The unbiased class-sampling reformulation keeps the extra per-iteration cost near one backward pass, so the end-to-end supervision is affordable at autonomous-driving training scales.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same "class attention should match the class mask" test could be applied to attention-based view transformations, but the paper's gradient-stability argument is specific to its normalization choices; a comparable bound would need to be derived for cross-attention lifts.
  • Because the loss supervises which pixels drive each class rather than the pixels themselves, the trained class-attention maps double as interpretability output — a free explanation of where in the image each occupied voxel class came from.
  • Uniform class sampling gives rare classes (pedestrians, traffic cones) little supervision per epoch; sampling proportional to class frequency or to current per-class error would concentrate the causal signal where the gain is largest.
  • Applying the learned camera offsets at test time to a model trained without the causal loss would separate two explanations of the robustness gain — better training signal versus genuine runtime geometric correction.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper targets the 2D-to-3D lifting stage in vision-based 3D semantic occupancy prediction. It proposes a 'Causal Loss' that backpropagates class-aggregated 3D features to the 2D feature map and supervises the resulting gradient attention map with 2D semantic masks (Eqs. 10-14). It also introduces a Semantic Causality-Aware Transformation (SCAT) with three components: Channel-Grouped Lifting (Eq. 15), Learnable Camera Offsets (Eqs. 18-19), and Normalized Convolution (Eqs. 20-21). Experiments on Occ3D with BEVDetOcc and ALOcc report clean-condition gains (mIoU 37.1->38.3 and 40.1->40.9, Table 3) and a much smaller relative mIoU drop under Gaussian camera-parameter noise (-7.3% vs -32.3%, Table 2).

Significance. The central idea of using semantic gradient maps as a supervisory signal for the lifting stage is interesting and clearly different from standard depth supervision. The clean-condition gains are consistent across two baselines, the ablation is monotonic, and the oracle upper-bound experiment in Table 1 gives supporting evidence for the semantic-causality premise. If the robustness comparison were controlled and the gradient-map derivation corrected, the contribution would be useful to the occupancy-prediction community. At present, however, two load-bearing parts of the paper need substantial work.

major comments (4)
  1. [Section 4.2 / Table 2] The robustness comparison is confounded. The text says Gaussian noise is added to camera parameters 'in training and testing,' but the vanilla rows reproduce the standard clean numbers of Table 3 (37.1 and 40.1) and then drop to 25.1 and 31.3 under noise. This indicates the vanilla models were trained without noise augmentation, whereas the +Ours models must have been trained with it (otherwise the Learnable Camera Offsets have no noisy training signal). Thus the Drop columns compare a clean-trained baseline evaluated under noise against a noise-augmented-trained model evaluated under noise. Training-time camera-parameter noise is itself a standard robustness augmentation, so the entire improvement in relative drop cannot be attributed to the causal loss or offsets. Please retrain all baselines with the same noise augmentation and report single-model clean/noisy degradation, or clearly r
  2. [Section 3.3, Eq. (9)] The gradient derivation drops a nonzero term. In Eq. (8), f_L = p_d * f_i, and p_d is produced by a geometry network that consumes f_i (p_d = F_g(f_i)). Therefore ∂(p_d f_i)/∂f_i = p_d I + f_i (∂p_d/∂f_i)^T, not p_d I as stated. If Eqs. (10)-(12) are implemented with autograd, the omitted term is automatically included and the loss does not supervise the simple 'semantic influence' map claimed. If the authors instead intend to stop gradients through p_d, that must be stated and the consequences analyzed. Either way, the central claim that the loss 'directly regularizes the information flow' through the geometry is not established as written.
  3. [Section 3.1, Theorem 1] Theorem 1 is not a theorem in its current form. 'ϵ-optimal solution' is never defined, and no convergence bound or optimization trajectory is analyzed. The proof shows that the gradient of the LSS loss differs from the gradient of an ideal loss; but any surrogate or approximate loss has different gradients, and that by itself does not imply failure to converge to a near-optimal solution. The conclusion 'prevents convergence to an ϵ-optimal solution' is therefore unsupported. Please either formalize the optimization setting and prove a concrete convergence statement, or re-frame this as empirical motivation rather than a theorem.
  4. [Section 3.4.3] The paper states 'We prove in the supplement that the derived gradient mask remains within [0, 1]' but no supplement is included in the submitted manuscript. This bound is load-bearing because the BCE loss in Eq. (12) requires A_s to lie in [0,1], and the normalized convolution is introduced specifically for gradient stability. Please include the proof, or state the assumptions under which the bound holds and verify it numerically for the actual autograd computation.
minor comments (6)
  1. [Eq. (16)] The coordinate mapping [h,w,z]^T = P·[u·d, v·d, d, 1]^T is not the standard camera projection equation unless P is an inverse-projection matrix. Please define the notation precisely and align it with the implementation.
  2. [Section 3.3, Eq. (12)] The source of the 2D ground-truth labels Y_s(u,v) is not described. Are they obtained by projecting 3D occupancy labels into each camera view? How are occlusions and out-of-view pixels handled? This is needed to reproduce the method.
  3. [Table 1] The row label 'Depth-Based LSS' is confusing because it reports 44.5 mIoU while the actual BEVDetOcc baseline is 37.1 in Table 3. Clarify that Table 1 uses ground-truth LiDAR depth, as described in the text.
  4. [Section 4.2] 'Gaussian noise with (0.1 variances)' is ambiguous. Specify whether 0.1 is the variance or standard deviation, and on which camera parameters the noise is applied.
  5. [Figure 5 caption] The caption contains stray text 'Figure 11: figure name' and 'Figure 12: figure name'. Please remove this formatting artifact.
  6. [Abstract / Section 3.4.2] The phrase 'making previously fixed components, such as camera parameters, fully learnable' overstates the contribution: the method learns a per-instance offset to the camera parameters, not the full calibration matrix as free parameters. Rephrase to avoid this overclaim.

Circularity Check

0 steps flagged

No significant circularity: the causal-loss targets are external ground-truth masks, the oracle experiment is an upper bound, and the self-citations are building blocks rather than load-bearing circular evidence.

full rationale

I walked the paper's derivation chain and found no step where a claimed prediction or first-principles result reduces by construction to its own inputs. The Causal Loss (Eq. 14) supervises the autograd-derived class-attention map A_s (Eqs. 10-12) against external 2D semantic masks and 3D occupancy ground truth; these targets are not model outputs, so the loss is a standard auxiliary supervision, not a self-fit. The oracle experiment in Table 1 (SCL-Aware LSS) is an explicit upper bound using ground-truth depth and semantics, and is not used to set constants or to generate the reported predictions. The Learnable Camera Offsets and Channel-Grouped Lifting are trained end-to-end and tested on a held-out benchmark, so the central claim does not reduce to a fitted parameter being renamed as a prediction. The paper does cite the authors' own prior work: ALOcc [7] for soft filling (Sec. 3.4.2) and [54] for the gradient-as-proxy inspiration (Sec. 1). These are transparent, non-uniqueness-building-block citations; the ablation in Table 4 (Exp. 6) isolates the soft-filling contribution, so the self-citation is not load-bearing as circular evidence. Two non-circular concerns should be flagged explicitly: (1) Sec. 3.4.3 defers the proof that the normalized-convolution gradient map remains in [0,1] to a 'supplement' that is not present in the manuscript; this is an omitted proof, not a circularity, but it leaves the stability claim unsupported in the main text. (2) Table 2's robustness comparison may be confounded: the clean baseline rows exactly match the clean-trained numbers of Table 3, so it is unclear whether the vanilla baselines were retrained with the same Gaussian camera-parameter noise as the +Ours models; if not, the Drop reduction reflects training-time augmentation rather than the proposed offsets/causal loss. This is an experimental-control issue, not a circularity. Overall, the derivation is self-contained against external benchmarks, and the minor self-citations do not make the central claim circular.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 1 invented entities

The causal loss uses external GT labels, so the ledger is dominated by modeling assumptions, not fitted parameters. The main tuned quantity is the loss weight (0.02); N_g and offset-network details are unreported. The key unproven postulates are the gradient-as-causality proxy (Sec. 3.3) and the asserted Theorem 1 (Sec. 3.1), plus the deferred [0,1] gradient-stability proof (Sec. 3.4.3).

free parameters (3)
  • Causal loss weight lambda = 0.02
    Weight of Lcausal (Eq. 14) in the total loss; set for main experiments (Sec. 4.1) and drives the +0.8 mIoU ablation gain in Table 4, with no sensitivity study.
  • Number of channel groups N_g = not reported
    Group count for Channel-Grouped Lifting (Eq. 15, Sec. 3.4.1); required to reproduce but never stated in the text.
  • Offset-network and normalization hyperparameters = unspecified
    F_offset1 and F_offset2 are 'multiple convolutional layers' (Eqs. 18-19); softmax temperature and conv kernel sizes for Normalized Convolution are not reported.
axioms (4)
  • domain assumption The 2D-to-3D pipeline is a causal chain I -> G -> L -> O, and input-gradient saliency maps are valid proxies for causal information flow.
    Sec. 3.2 and Fig. 2 posit the causal structure; Sec. 3.3 adopts gradients as the proxy, citing LayerCAM and in-context-learning work. This is asserted, not derived, and no causal-inference machinery is used.
  • domain assumption The gradient of class-aggregated 3D features w.r.t. 2D features equals p_d*I (Eq. 9), i.e., p_d is treated as independent of f_i.
    Eq. (9) in Sec. 3.3; in the actual pipeline p_d = F_g(f_i), so autograd includes dp_d/df_i, which the analysis omits.
  • domain assumption BCE between A_s and the projected 2D GT mask Y_s is a sound training objective (Eq. 12).
    Assumes 2D masks projected from 3D occupancy GT align with the 3D-class aggregation in Omega_s, so the loss is well-posed for enforcing same-class lifting.
  • standard math Theorem 1 assumptions: bounded mapping error ||delta_M||_F <= Delta_M, L_lift-Lipschitz lifting, and gradient deviation from a fixed mapping prevents epsilon-optimal convergence.
    Sec. 3.1; boundedness and Lipschitz conditions are conventional, but the epsilon-optimality conclusion is asserted without a convergence analysis (e.g., no L-smoothness or bias/variance argument).
invented entities (1)
  • Semantic causal locality (SCL) principle no independent evidence
    purpose: The ideal that a 3D class prediction should be fed predominantly by 2D regions of the same class; motivates the causal loss and the SCL-Aware oracle geometry in Table 1.
    A design postulate, not a falsifiable physical entity; its only support is the method's own benchmark gains. It is a concept rather than a particle/force/dimension-type entity.

pith-pipeline@v1.3.0-alltime-deepseek · 15157 in / 27113 out tokens · 264525 ms · 2026-08-04T20:40:32.866610+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Semantic Causality-Aware Vision-Based 3D Occupancy Prediction." pith.science (2026). https://pith.science/paper/IS5PUV3S

@misc{pith2026250908388,
  author       = {Pith},
  title        = {Pith review of: Semantic Causality-Aware Vision-Based 3D Occupancy Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IS5PUV3S}},
  note         = {Machine review of arXiv:2509.08388}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Vision-based 3D semantic occupancy prediction is a critical task in 3D vision that integrates volumetric 3D reconstruction with semantic understanding. Existing methods, however, often rely on modular pipelines. These modules are typically optimized independently or use pre-configured inputs, leading to cascading errors. In this paper, we address this limitation by designing a novel causal loss that enables holistic, end-to-end supervision of the modular 2D-to-3D transformation pipeline. Grounded in the principle of 2D-to-3D semantic causality, this loss regulates the gradient flow from 3D voxel representations back to the 2D features. Consequently, it renders the entire pipeline differentiable, unifying the learning process and making previously non-trainable components fully learnable. Building on this principle, we propose the Semantic Causality-Aware 2D-to-3D Transformation, which comprises three components guided by our causal loss: Channel-Grouped Lifting for adaptive semantic mapping, Learnable Camera Offsets for enhanced robustness against camera perturbations, and Normalized Convolution for effective feature propagation. Extensive experiments demonstrate that our method achieves state-of-the-art performance on the Occ3D benchmark, demonstrating significant robustness to camera perturbations and improved 2D-to-3D semantic consistency.

Figures

Figures reproduced from arXiv: 2509.08388 by Dubing Chen, Huan Zheng, Jianbing Shen, Pai Peng, Tao He, Wenlong Liao, Xianfei Li, Yucheng Zhou.

Figure 1
Figure 1. Figure 1: (a) Illustrates a Visual Analysis of Semantic Ambigu￾ity in VisionOcc. Inaccurate 2D-to-3D transformations may lead to positional shifts, misaligning supervision signals and resulting in semantic ambiguity. (b) Depicts the Conventional Modular 2D-to-3D Transformation Paradigm [7, 13, 27, 36], which em￾ploys depth supervision for geometry estimation, pre-calibrated camera parameters, and fixed mapping for l… view at source ↗
Figure 2
Figure 2. Figure 2: The Causal Structure of VisionOcc. It illustrates the dependency chain from the image input I to the semantic occu￾pancy output O. G: geometry for 2D-to-3D transformation. P: camera intrinsic and extrinsic. L: intermediate 3D feature. eP : errors in camera parameters. ture L, and final occupancy output O. In LSS-based meth￾ods [7, 13, 36], the pipeline starts with the image back￾bone extracting features fi… view at source ↗
Figure 3
Figure 3. Figure 3: The Overall Framework of the Proposed Semantic Causality-Aware VisionOcc. The proposed framework consists of three primary components: a backbone network for extracting 2D features, an SCAT module for transforming these features into 3D space, and an Encoder-Decoder network for learning 3D semantics. The SCAT module is supervised by our causal loss. Method mIoU mIoUD IoU Depth-Based LSS 44.5 40.4 78.9 SCL-… view at source ↗
Figure 4
Figure 4. Figure 4: The Detailed Structure of the Proposed Modules in Semantic Causality-Aware 2D-to-3D Transformation. There are three key components, including (a): Channel-Grouped Lift￾ing, (b): Learnable Camera Offsets, and (c): Normalized Convo￾lution, enabling accurate and robust 2D-to-3D transformation. g, replacing pd which uniformly lifts all channels. Ng is the number of groups. This preserves semantic distinction, … view at source ↗
Figure 11
Figure 11. Figure 11: figure name Occupancy Loss Curves with/without Our Method. [PITH_FULL_IMAGE:figures/full_fig_p007_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: figure name ALOcc+Ours) achieving much smaller drops ( [PITH_FULL_IMAGE:figures/full_fig_p007_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

54 extracted references · 32 canonical work pages · 7 internal anchors

  1. [1]

    Se- mantickitti: A dataset for semantic scene understanding of lidar sequences

    Jens Behley, Martin Garbade, Andres Milioto, Jan Quen- zel, Sven Behnke, Cyrill Stachniss, and Jurgen Gall. Se- mantickitti: A dataset for semantic scene understanding of lidar sequences. InProceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 9297–9307,

    work page

  2. [2]

    Lan- gocc: Self-supervised open vocabulary occupancy estima- tion via volume rendering.arXiv preprint arXiv:2407.17310,

    Simon Boeder, Fabian Gigengack, and Benjamin Risse. Lan- gocc: Self-supervised open vocabulary occupancy estima- tion via volume rendering.arXiv preprint arXiv:2407.17310,

    Pith/arXiv arXiv

  3. [3]

    Oc- cflownet: Towards self-supervised occupancy estimation via differentiable rendering and occupancy flow.arXiv preprint arXiv:2402.12792, 2024

    Simon Boeder, Fabian Gigengack, and Benjamin Risse. Oc- cflownet: Towards self-supervised occupancy estimation via differentiable rendering and occupancy flow.arXiv preprint arXiv:2402.12792, 2024. 3

    Pith/arXiv arXiv 2024

  4. [4]

    Gaussianflowocc: Sparse and weakly supervised occupancy estimation using gaussian splatting and temporal flow.arXiv preprint arXiv:2502.17288, 2025

    Simon Boeder, Fabian Gigengack, and Benjamin Risse. Gaussianflowocc: Sparse and weakly supervised occupancy estimation using gaussian splatting and temporal flow.arXiv preprint arXiv:2502.17288, 2025. 3

    Pith/arXiv arXiv 2025

  5. [5]

    nuscenes: A multi- modal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Gi- ancarlo Baldan, and Oscar Beijbom. nuscenes: A multi- modal dataset for autonomous driving. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11621–11631, 2020. 7

    work page 2020

  6. [6]

    Monoscene: Monoc- ular 3d semantic scene completion

    Anh-Quan Cao and Raoul de Charette. Monoscene: Monoc- ular 3d semantic scene completion. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3991–4001, 2022. 3, 8

    work page 2022

  7. [7]

    ALOcc: Adaptive Lifting-Based 3D Semantic Occupancy and Cost Volume-Based Flow Predictions

    Dubing Chen, Jin Fang, Wencheng Han, Xinjing Cheng, Junbo Yin, Chengzhong Xu, Fahad Shahbaz Khan, and Jian- bing Shen. Alocc: adaptive lifting-based 3d semantic occu- pancy and cost volume-based flow prediction.arXiv preprint arXiv:2411.07725, 2024. 1, 2, 3, 4, 5, 6, 7, 8

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  8. [8]

    AdaOcc: Adaptive Forward View Transformation and Flow Modeling for 3D Occupancy and Flow Prediction

    Dubing Chen, Wencheng Han, Jin Fang, and Jianbing Shen. Adaocc: Adaptive forward view transformation and flow modeling for 3d occupancy and flow prediction.arXiv preprint arXiv:2407.01436, 2024. 3

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  9. [9]

    Rethinking temporal fusion with a unified gradient descent view for 3d semantic occupancy prediction

    Dubing Chen, Huan Zheng, Jin Fang, Xingping Dong, Xian- fei Li, Wenlong Liao, Tao He, Pai Peng, and Jianbing Shen. Rethinking temporal fusion with a unified gradient descent view for 3d semantic occupancy prediction. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 1505–1515, 2025. 1, 3

    work page 2025

  10. [10]

    3d sketch-aware semantic scene comple- tion via semi-supervised structure prior

    Xiaokang Chen, Kwan-Yee Lin, Chen Qian, Gang Zeng, and Hongsheng Li. 3d sketch-aware semantic scene comple- tion via semi-supervised structure prior. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4193–4202, 2020. 2

    work page 2020

  11. [11]

    Ro- bust learning with jacobian regularization.arXiv preprint arXiv:1908.02729, 2019

    Judy Hoffman, Daniel A Roberts, and Sho Yaida. Ro- bust learning with jacobian regularization.arXiv preprint arXiv:1908.02729, 2019. 5

    Pith/arXiv arXiv 1908

  12. [12]

    Bevdet4d: Exploit tempo- ral cues in multi-camera 3d object detection.arXiv preprint arXiv:2203.17054, 2022

    Junjie Huang and Guan Huang. Bevdet4d: Exploit tempo- ral cues in multi-camera 3d object detection.arXiv preprint arXiv:2203.17054, 2022. 7, 8

    Pith/arXiv arXiv 2022

  13. [13]

    Bevdet: High-performance multi-camera 3d object de- tection in bird-eye-view.arXiv preprint arXiv:2112.11790,

    Junjie Huang, Guan Huang, Zheng Zhu, Yun Ye, and Dalong Du. Bevdet: High-performance multi-camera 3d object de- tection in bird-eye-view.arXiv preprint arXiv:2112.11790,

    Pith/arXiv arXiv

  14. [14]

    Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction

    Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. Tri-perspective view for vision- based 3d semantic occupancy prediction.arXiv preprint arXiv:2302.07817, 2023. 1, 2, 3, 8

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  15. [15]

    Selfocc: Self-supervised vision-based 3d oc- cupancy prediction

    Yuanhui Huang, Wenzhao Zheng, Borui Zhang, Jie Zhou, and Jiwen Lu. Selfocc: Self-supervised vision-based 3d oc- cupancy prediction. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 19946–19956, 2024

    work page 2024

  16. [16]

    Gaussianformer: Scene as gaussians for vision-based 3d semantic occupancy prediction

    Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. Gaussianformer: Scene as gaussians for vision-based 3d semantic occupancy prediction. InEuropean Conference on Computer Vision, pages 376–393. Springer,

  17. [17]

    Gausstr: Foundation model-aligned gaussian transformer for self-supervised 3d spatial understanding.arXiv preprint arXiv:2412.13193, 2024

    Haoyi Jiang, Liu Liu, Tianheng Cheng, Xinjie Wang, Tian- wei Lin, Zhizhong Su, Wenyu Liu, and Xinggang Wang. Gausstr: Foundation model-aligned gaussian transformer for self-supervised 3d spatial understanding.arXiv preprint arXiv:2412.13193, 2024. 3

    Pith/arXiv arXiv 2024

  18. [18]

    Layercam: Exploring hierarchical class activation maps for localization.IEEE Transactions on Image Processing, 30:5875–5888, 2021

    Peng-Tao Jiang, Chang-Bin Zhang, Qibin Hou, Ming-Ming Cheng, and Yunchao Wei. Layercam: Exploring hierarchical class activation maps for localization.IEEE Transactions on Image Processing, 30:5875–5888, 2021. 2, 7

    work page 2021

  19. [19]

    ProtoOcc: Accurate, Efficient 3D Occupancy Prediction Using Dual Branch Encoder-Prototype Query Decoder

    Jungho Kim, Changwon Kang, Dongyoung Lee, Sehwan Choi, and Jun Won Choi. Protoocc: Accurate, efficient 3d occupancy prediction using dual branch encoder-prototype query decoder.arXiv preprint arXiv:2412.08774, 2024. 8

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  20. [20]

    Rgbd based dimensional decomposition residual network for 3d semantic scene completion

    Jie Li, Yu Liu, Dong Gong, Qinfeng Shi, Xia Yuan, Chunxia Zhao, and Ian Reid. Rgbd based dimensional decomposition residual network for 3d semantic scene completion. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7693–7702, 2019. 2

    work page 2019

  21. [21]

    Viewformer: Exploring spatiotem- poral modeling for multi-view 3d occupancy perception via view-guided transformers

    Jinke Li, Xiao He, Chonghua Zhou, Xiaoqiang Cheng, Yang Wen, and Dan Zhang. Viewformer: Exploring spatiotem- poral modeling for multi-view 3d occupancy perception via view-guided transformers. InComputer Vision–ECCV 2024: 18th European Conference, 2024. 3

    work page 2024

  22. [22]

    Bevdepth: Acquisition of reliable depth for multi-view 3d object detec- tion.arXiv preprint arXiv:2206.10092, 2022

    Yinhao Li, Zheng Ge, Guanyi Yu, Jinrong Yang, Zengran Wang, Yukang Shi, Jianjian Sun, and Zeming Li. Bevdepth: Acquisition of reliable depth for multi-view 3d object detec- tion.arXiv preprint arXiv:2206.10092, 2022. 3, 5

    Pith/arXiv arXiv 2022

  23. [23]

    Bevstereo: Enhancing depth estimation in multi-view 3d object detection with temporal stereo

    Yinhao Li, Han Bao, Zheng Ge, Jinrong Yang, Jianjian Sun, and Zeming Li. Bevstereo: Enhancing depth estimation in multi-view 3d object detection with temporal stereo. InPro- ceedings of the AAAI Conference on Artificial Intelligence, pages 1486–1494, 2023. 3, 5

    work page 2023

  24. [24]

    V oxformer: Sparse voxel transformer for camera- based 3d semantic scene completion

    Yiming Li, Zhiding Yu, Christopher Choy, Chaowei Xiao, Jose M Alvarez, Sanja Fidler, Chen Feng, and Anima Anand- kumar. V oxformer: Sparse voxel transformer for camera- based 3d semantic scene completion. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9087–9098, 2023. 2, 3

    work page 2023

  25. [25]

    Fast-bev: A fast and strong bird’s- eye view perception baseline.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    Yangguang Li, Bin Huang, Zeren Chen, Yufeng Cui, Feng Liang, Mingzhu Shen, Fenggang Liu, Enze Xie, Lu Sheng, Wanli Ouyang, et al. Fast-bev: A fast and strong bird’s- eye view perception baseline.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 3

    work page 2024

  26. [26]

    Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers.arXiv preprint arXiv:2203.17270, 2022

    Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chong- hao Sima, Tong Lu, Qiao Yu, and Jifeng Dai. Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers.arXiv preprint arXiv:2203.17270, 2022. 3

    Pith/arXiv arXiv 2022

  27. [27]

    Fb-occ: 3d occupancy prediction based on forward-backward view transformation

    Zhiqi Li, Zhiding Yu, David Austin, Mingsheng Fang, Shiyi Lan, Jan Kautz, and Jose M Alvarez. Fb-occ: 3d occupancy prediction based on forward-backward view transformation. arXiv preprint arXiv:2307.01492, 2023. 1, 2, 3, 5, 7, 8

    Pith/arXiv arXiv 2023

  28. [28]

    Fb-bev: Bev representation from forward-backward view transformations

    Zhiqi Li, Zhiding Yu, Wenhai Wang, Anima Anandkumar, Tong Lu, and Jose M Alvarez. Fb-bev: Bev representation from forward-backward view transformations. InProceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 6919–6928, 2023. 2, 6

    work page 2023

  29. [29]

    Cascadeflow: 3d occupancy and flow prediction with cascaded sparsity sampling refine- ment framework.CVPR 2024 Autonomous Grand Challenge Track On Occupancy and Flow, 2024

    Zhimin Liao and Ping Wei. Cascadeflow: 3d occupancy and flow prediction with cascaded sparsity sampling refine- ment framework.CVPR 2024 Autonomous Grand Challenge Track On Occupancy and Flow, 2024. 3

    work page 2024

  30. [30]

    Fully sparse 3d occupancy prediction

    Haisong Liu, Yang Chen, Haiguang Wang, Zetong Yang, Tianyu Li, Jia Zeng, Li Chen, Hongyang Li, and Limin Wang. Fully sparse 3d occupancy prediction. InEuropean Conference on Computer Vision, pages 54–71. Springer,

    work page

  31. [31]

    Surroundsdf: Implicit 3d scene understanding based on signed distance field

    Lizhe Liu, Bohua Wang, Hongwei Xie, Daqi Liu, Li Liu, Zhiqiang Tian, Kuiyuan Yang, and Bing Wang. Surroundsdf: Implicit 3d scene understanding based on signed distance field. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, 2024. 1

    work page 2024

  32. [32]

    See and think: Disentangling semantic scene completion.Advances in Neural Information Processing Systems, 31, 2018

    Shice Liu, Yu Hu, Yiming Zeng, Qiankun Tang, Beibei Jin, Yinhe Han, and Xiaowei Li. See and think: Disentangling semantic scene completion.Advances in Neural Information Processing Systems, 31, 2018. 2

    work page 2018

  33. [33]

    Let Occ Flow: Self-Supervised 3D Occupancy Flow Prediction

    Yili Liu, Linzhan Mou, Xuan Yu, Chenrui Han, Sitong Mao, Rong Xiong, and Yue Wang. Let occ flow: Self- supervised 3d occupancy flow prediction.arXiv preprint arXiv:2407.07587, 2024. 3

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  34. [34]

    Fixing weight decay regularization in adam.arXiv preprint arXiv:1711.05101,

    Ilya Loshchilov, Frank Hutter, et al. Fixing weight decay regularization in adam.arXiv preprint arXiv:1711.05101,

    Pith/arXiv arXiv

  35. [35]

    Oc- treeocc: Efficient and multi-granularity occupancy predic- tion using octree queries.arXiv preprint arXiv:2312.03774,

    Yuhang Lu, Xinge Zhu, Tai Wang, and Yuexin Ma. Oc- treeocc: Efficient and multi-granularity occupancy predic- tion using octree queries.arXiv preprint arXiv:2312.03774,

    Pith/arXiv arXiv

  36. [36]

    COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Prediction

    Qihang Ma, Xin Tan, Yanyun Qu, Lizhuang Ma, Zhizhong Zhang, and Yuan Xie. Cotr: Compact occupancy transformer for vision-based 3d occupancy prediction.arXiv preprint arXiv:2312.01919, 2023. 1, 2, 3, 7, 8

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  37. [37]

    Renderocc: Vision-centric 3d occupancy pre- diction with 2d rendering supervision

    Mingjie Pan, Jiaming Liu, Renrui Zhang, Peixiang Huang, Xiaoqi Li, Hongwei Xie, Bing Wang, Li Liu, and Shang- hang Zhang. Renderocc: Vision-centric 3d occupancy pre- diction with 2d rendering supervision. In2024 IEEE Inter- national Conference on Robotics and Automation (ICRA), pages 12404–12411. IEEE, 2024. 3

    work page 2024

  38. [38]

    Lift, splat, shoot: Encod- ing images from arbitrary camera rigs by implicitly unpro- jecting to 3d

    Jonah Philion and Sanja Fidler. Lift, splat, shoot: Encod- ing images from arbitrary camera rigs by implicitly unpro- jecting to 3d. InComputer Vision–ECCV 2020: 16th Euro- pean Conference, Glasgow, UK, August 23–28, 2020, Pro- ceedings, Part XIV 16, pages 194–210. Springer, 2020. 2, 3

    work page 2020

  39. [39]

    Occupancy as set of points

    Yiang Shi, Tianheng Cheng, Qian Zhang, Wenyu Liu, and Xinggang Wang. Occupancy as set of points. InComputer Vision–ECCV 2024: 18th European Conference, 2024. 3

    work page 2024

  40. [40]

    Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving.Advances in Neural Information Processing Systems, 36, 2024

    Xiaoyu Tian, Tao Jiang, Longfei Yun, Yucheng Mao, Huitong Yang, Yue Wang, Yilun Wang, and Hang Zhao. Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving.Advances in Neural Information Processing Systems, 36, 2024. 1, 3, 7, 8

    work page 2024

  41. [41]

    Scene as occupancy

    Wenwen Tong, Chonghao Sima, Tai Wang, Li Chen, Silei Wu, Hanming Deng, Yi Gu, Lewei Lu, Ping Luo, Dahua Lin, et al. Scene as occupancy. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 8406– 8415, 2023. 1, 3

    work page 2023

  42. [42]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017. 3

    2017

  43. [43]

    Label words are an- chors: An information flow perspective for understanding in-context learning.arXiv preprint arXiv:2305.14160, 2023

    Lean Wang, Lei Li, Damai Dai, Deli Chen, Hao Zhou, Fan- dong Meng, Jie Zhou, and Xu Sun. Label words are an- chors: An information flow perspective for understanding in-context learning.arXiv preprint arXiv:2305.14160, 2023. 2

    Pith/arXiv arXiv 2023

  44. [44]

    Openoccupancy: A large scale benchmark for surrounding semantic occupancy perception.arXiv preprint arXiv:2303.03991, 2023

    Xiaofeng Wang, Zheng Zhu, Wenbo Xu, Yunpeng Zhang, Yi Wei, Xu Chi, Yun Ye, Dalong Du, Jiwen Lu, and Xin- gang Wang. Openoccupancy: A large scale benchmark for surrounding semantic occupancy perception.arXiv preprint arXiv:2303.03991, 2023. 1, 3

    Pith/arXiv arXiv 2023

  45. [45]

    Surroundocc: Multi-camera 3d oc- cupancy prediction for autonomous driving.arXiv preprint arXiv:2303.09551, 2023

    Yi Wei, Linqing Zhao, Wenzhao Zheng, Zheng Zhu, Jie Zhou, and Jiwen Lu. Surroundocc: Multi-camera 3d oc- cupancy prediction for autonomous driving.arXiv preprint arXiv:2303.09551, 2023. 1, 3

    Pith/arXiv arXiv 2023

  46. [46]

    Deep height decoupling for pre- cise vision-based 3d occupancy prediction.arXiv preprint arXiv:2409.07972, 2024

    Yuan Wu, Zhiqiang Yan, Zhengxue Wang, Xiang Li, Le Hui, and Jian Yang. Deep height decoupling for pre- cise vision-based 3d occupancy prediction.arXiv preprint arXiv:2409.07972, 2024. 8

    Pith/arXiv arXiv 2024

  47. [47]

    Renderworld: World model with self-supervised 3d label.arXiv preprint arXiv:2409.11356,

    Ziyang Yan, Wenzhen Dong, Yihua Shao, Yuhang Lu, Liu Haiyang, Jingwen Liu, Haozhe Wang, Zhe Wang, Yan Wang, Fabio Remondino, et al. Renderworld: World model with self-supervised 3d label.arXiv preprint arXiv:2409.11356,

    Pith/arXiv arXiv

  48. [48]

    Seman- tic 3d occupancy mapping through efficient high order crfs

    Shichao Yang, Yulan Huang, and Sebastian Scherer. Seman- tic 3d occupancy mapping through efficient high order crfs. In2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 590–597. IEEE, 2017. 1

    work page 2017

  49. [49]

    Flashocc: Fast and memory-efficient occupancy prediction via channel-to-height plugin.arXiv preprint arXiv:2311.12058, 2023

    Zichen Yu, Changyong Shu, Jiajun Deng, Kangjie Lu, Zong- dai Liu, Jiangyong Yu, Dawei Yang, Hui Li, and Yan Chen. Flashocc: Fast and memory-efficient occupancy prediction via channel-to-height plugin.arXiv preprint arXiv:2311.12058, 2023. 8

    Pith/arXiv arXiv 2023

  50. [50]

    Occnerf: Self- supervised multi-camera occupancy prediction with neural radiance fields.arXiv preprint arXiv:2312.09243, 2023

    Chubin Zhang, Juncheng Yan, Yi Wei, Jiaxin Li, Li Liu, Yansong Tang, Yueqi Duan, and Jiwen Lu. Occnerf: Self- supervised multi-camera occupancy prediction with neural radiance fields.arXiv preprint arXiv:2312.09243, 2023. 3

    Pith/arXiv arXiv 2023

  51. [51]

    LightOcc: Lightweight Spatial Embedding for Efficient Vision-based 3D Occupancy Prediction

    Jinqing Zhang, Yanan Zhang, Qingjie Liu, and Yunhong Wang. Lightweight spatial embedding for vision-based 3d occupancy prediction.arXiv preprint arXiv:2412.05976,

    work page internal anchor Pith review Pith/arXiv arXiv

  52. [52]

    Occformer: Dual-path transformer for vision-based 3d semantic occu- pancy prediction.arXiv preprint arXiv:2304.05316, 2023

    Yunpeng Zhang, Zheng Zhu, and Dalong Du. Occformer: Dual-path transformer for vision-based 3d semantic occu- pancy prediction.arXiv preprint arXiv:2304.05316, 2023. 2, 3

    Pith/arXiv arXiv 2023

  53. [53]

    Veon: V ocabulary- enhanced occupancy prediction

    Jilai Zheng, Pin Tang, Zhongdao Wang, Guoqing Wang, Xi- angxuan Ren, Bailan Feng, and Chao Ma. Veon: V ocabulary- enhanced occupancy prediction. InEuropean Conference on Computer Vision, pages 92–108. Springer, 2024. 3

    work page 2024

  54. [54]

    Visual in-context learning for large vision-language models

    Yucheng Zhou, Xiang Li, Qianning Wang, and Jianbing Shen. Visual in-context learning for large vision-language models. InFindings of the Association for Computational Linguistics ACL 2024, pages 15890–15902, 2024. 2

    work page 2024