TACOcc:Target-Adaptive Cross-Modal Fusion with Volume Rendering for 3D Semantic Occupancy

Luyao Lei; Shuo Xu; Xing Wei; Yifan Bai

REVIEW 4 major objections 6 minor 18 references

TACOcc:Target-Adaptive Cross-Modal Fusion with Volume Rendering for 3D Semantic Occupancy

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read TACOcc claims that per-query adaptive fusion neighborhoods plus 3D Gaussian volume-rendering supervision raise 3D occupancy mIoU to 28.4%, 1.8% above the prior multi-modal state of the art.

desk verdict The per-query k selection is a real novelty, but the missing fixed k=4 baseline blurs the central adaptive-fusion claim; still worth a serious referee's time. read the letter →

arxiv 2505.12693 v1 pith:KTL2KC5H submitted 2025-05-19 cs.CV

Luyao Lei , Shuo Xu , Yifan Bai , Xing Wei This is my paper · ORCID

classification cs.CV

keywords 3Dsemanticoccupancypredictionmulti-modalfusionadaptiveneighborhoodselectionGaussianSplattingvolumerenderingsupervisionlidar-cameraautonomousdrivingnuScenes

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to show that two weaknesses in multi-modal 3D occupancy prediction, fixed cross-modal fusion neighborhoods and sparse supervision on object surfaces, can be fixed together. The proposed framework, TACOcc, lets each query voxel choose its own fusion neighborhood size: a learned module picks one of four candidate k values, so large objects aggregate more context and small objects avoid noise. It then feeds the fused features into a 3D Gaussian Splatting volume-rendering pipeline that renders multi-view images, applies photometric and Gaussian-parameter consistency losses, and pushes 2D detail back into the 3D features. On the nuScenes validation benchmark the authors report 28.4% mIoU, surpassing the previous best multi-modal method by 1.8%, with the largest per-class gains on small, sparse objects like traffic cones and motorcycles. If true, this would make adaptive neighborhood selection and rendering-based self-supervision a scalable recipe for occupancy perception in autonomous driving.

What carries the argument

The load-bearing mechanism is a per-query dynamic neighborhood selector fused into a bidirectional symmetric retrieval unit. An MLP maps each non-zero query voxel's feature to logits over candidate k values; Gumbel-Softmax makes the discrete choice differentiable and a straight-through estimator keeps gradients flowing through the rounding or argmax, so the model can learn to widen neighborhoods for large objects and narrow them for small ones. The second mechanism is a multi-modal 3D Gaussian Splatting volume-rendering pipeline: fused features initialize Gaussian primitives, differentiable splatting renders six viewpoints, and photometric loss Lrgb plus Gaussian parameter consistency loss Lpc create a 2D-to-3D supervision path intended to recover surface detail lost to sparse annotations.

What would settle it

Compute, on the nuScenes validation set, the distribution of selected k for each semantic instance or object-size bin while controlling for point density, distance, and neighboring-object count; if small objects do not systematically receive small k and large objects large k under those controls, the target-scale adaptation claim fails. A simpler check is to shuffle the predicted k values among queries with matched density and distance and compare mIoU with the learned assignment: if a random or density-based assignment matches it, the learned mechanism is not doing the claimed work.

Watch

Extended reading notes

Core claim

TACOcc's central claim is that target-scale-adaptive bidirectional retrieval plus volume-rendering supervision is a better way to fuse lidar and image features for dense 3D semantic occupancy prediction. Instead of using a fixed number of neighboring voxels when matching geometric lidar features with semantic image features, the model learns a per-query k in {1,2,3,4} via a Gumbel-Softmax relaxation with a straight-through estimator, then performs symmetric K-nearest-neighbor retrieval in both directions: image queries retrieve lidar keys and lidar queries retrieve image keys. The volume-rendering module initializes 3D Gaussian primitives from the fused features, using both point-cloud coordinates and non-zero voxel centers, renders six camera views, and supervises with an L1 plus D-SSIM photometric loss and a Gaussian parameter consistency loss. With this dual module, TACOcc reports 41.8% IoU and 28.4% mIoU on nuScenes, 1.8 percentage points above Co-Occ, and improves traffic cone and motorcycle IoU by about 3 points; on SemanticKITTI it reports 24.7% mIoU, better than leading lidar-only methods that use extra segmentation supervision.

Load-bearing premise

The load-bearing assumption is that the network's learned per-query neighborhood size responds to the scale of the object being fused, not to scene density, distance, occlusion, or class frequency, because the paper's evidence for target-scale adaptation is scene-level k statistics rather than per-object measurements.

Editorial extensions

If this is right

If the central claim holds, dynamic k gives a better accuracy-latency trade-off than fixed neighborhoods: fixed k=3 reaches 26.7% mIoU at 0.65 s, while dynamic k in {1,2,3,4} reaches 28.4% mIoU at 0.64 s.
The reported k distributions imply the fusion module automatically specializes: in small-target-dominant scenes 60.1% of queries choose k=1 or 2, while in large-target-dominant scenes 70.3% choose k=3 or 4.
Volume-rendering supervision is not just cosmetic: adding Lrgb then Lpc to the adaptive fusion baseline raises mIoU from 26.9% to 27.1% and then to 28.4% in the ablation table.
If correct, TACOcc provides a template for combining adaptive cross-modal matching with rendering-based self-supervision on any occupancy backbone, not only lidar-camera inputs.
The range-dependent evaluation shows the method improves over Co-Occ at 25 m, 50 m, and 100 m, suggesting the gains are not confined to near-field objects.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the paper's k-distribution evidence is scene-level aggregates, so the claim that k tracks object scale would be strengthened by per-object measurements; if k instead tracks scene density or distance, a simpler density- or distance-based schedule might reproduce the gains.
Editorial inference: the 3D Gaussian Splatting volume-rendering module may be a plug-and-play supervision add-on for other occupancy predictors, since it only needs fused features and images; the paper does not test this separation.
Editorial inference: larger gains on small, sparse classes suggest rendering supervision could reduce dependence on dense 3D labels for rare classes, an annotation-cost saving the paper does not quantify.
Editorial inference: the same adaptive-neighborhood idea could transfer to other point-image fusion tasks such as 3D detection or BEV map segmentation, where fixed-radius neighbors cause the same mismatch.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The dynamic per-query k selection is the genuinely new piece here. Prior fusion work uses fixed neighborhoods or global channel attention; using Gumbel-Softmax with a STE to pick a neighborhood size per query is a real departure, and the 3DGS renderer with multi-modal initialization is a sensible upgrade over Co-Occ's NeRF-style regularizer. Credit where due: the ablations are internally consistent, each module adds mIoU, and the largest gains are on traffic cones and motorcycles, which fits the small-object story. The paper also reports SemanticKITTI results, range-based breakdowns, and latency, which is more than many occupancy papers bother with.

Now the soft spots. The load-bearing issue, as the stress-test note says, is that Table 4 never evaluates fixed k=4. The dynamic candidate set includes k=4, and in large-target scenes the model selects k=4 nearly 40% of the time. So the dynamic 1-4 improvement over fixed k=3 could be nothing more than allowing a larger maximum neighborhood. Without a fixed k=4 control, or better, per-query k correlated with object size rather than scene-level aggregates, the 'scale-adaptive' claim is not established at the granularity the abstract states. This is fixable, but it is the core of the paper, so it matters. The STE equations (Eq. 1-2) are also sloppy; the forward/backward definitions bleed into each other. The photometric loss reuses the same input images that produced the image features, so calling it 2D-3D consistency is generous; it is self-supervision. And the evaluation is a single run on the validation set with no error bars and no code release.

Who is this for? Anyone working on multi-modal occupancy or cross-modal fusion. It is a plausible SOTA result, not a discredited one. The central mechanism deserves scrutiny, not dismissal. I would send it to review, but request the missing fixed-k baseline, per-object analysis, and a cleaner presentation of the STE trick before it comes back.

Referee Report

4 major / 6 minor

Summary. The paper proposes TACOcc, a multi-modal 3D semantic occupancy prediction framework combining an adaptive cross-modal fusion module with a 3D Gaussian Splatting volume-rendering supervision module. The fusion module uses an MLP with Gumbel-Softmax and a straight-through estimator to predict a per-query neighborhood size k, then performs bidirectional symmetric retrieval between lidar and image features. The rendering module initializes Gaussians from fused features and supervises them with photometric consistency against the input images, plus a Gaussian parameter consistency loss. Experiments on nuScenes and SemanticKITTI report a 28.4% mIoU on nuScenes, surpassing Co-Occ by 1.8%, with ablations of the two modules and an efficiency analysis.

Significance. If the reported numbers are stable, the paper demonstrates a new state of the art for multi-modal 3D occupancy prediction and provides a plausible mechanism for scale-adaptive fusion. The strengths include the two-benchmark evaluation, the component ablations, and the explicit efficiency comparison. However, the central adaptive-fusion claim is currently under-supported: the missing fixed-k=4 control and the scene-level aggregation in Tables 4-6 leave open the possibility that the gains are due to a larger maximum neighborhood rather than target-adaptive behavior. The method description also contains notational inconsistencies that impede reproducibility.

major comments (4)

[Sec. 4.4, Table 4] The adaptive-fusion claim is not isolated from the maximum neighborhood size. Table 4 compares fixed k=1, 2, 3 with dynamic ranges 1-2, 1-3, and 1-4, but never evaluates fixed k=4. Since the dynamic ranges include k=4, the improvement of dynamic 1-4 over fixed k=3 (mIoU 28.4 vs 26.7, IoU 41.8 vs 38.2) could reflect simply a larger maximum neighborhood, not target-adaptive selection. Please add a fixed k=4 baseline and report per-query k statistics to separate the two effects.
[Sec. 4.4, Tables 5-6] The scale-perception mechanism is supported only by scene-level aggregate k distributions. Tables 5 and 6 report k frequencies across scenes labeled by dominant target size, which confounds target scale with scene density, object count, and average distance; no per-object or per-voxel correlation between k and object size is provided. The abstract-level claim that neighborhoods expand for large targets and shrink for small ones is therefore not established at the claimed granularity. Please provide per-instance statistics, such as average k as a function of object size or distance.
[Sec. 3.1, Eqs. (1)-(4)] Equations (1)-(4) are not stated rigorously enough to reproduce the fusion module. Equation (2) mixes a forward definition y_ste = e_k with a backward definition z_c = GumbelSoftmax(z) inside a single expression for ∂L/∂z, and the notation ∇_z L versus ∂L/∂z is inconsistent. Equation (3) defines ω_i, but Eq. (4) uses ω_I and ω_L without specifying how these weights are derived from the key matrix or how padding and the linear mapping are applied. Please rewrite the equations with consistent notation and explicit tensor dimensions.
[Sec. 4.3, Table 1 and Sec. 4.4] The empirical claims, including the headline 28.4% mIoU and all ablation deltas, are reported as single runs without code or error bars. Given the small margins over Co-Occ (1.8% mIoU) and the absence of code, the reader cannot assess whether the observed gains are stable. Please report means and standard deviations over at least three seeds for the main results and the key ablations, and release code and models to support the SOTA claim.

minor comments (6)

[Throughout] The dataset name is misspelled as 'SematicKITTI' in Table 2 and the accompanying text; it should be 'SemanticKITTI'.
[Sec. 4.4, Table 3] The 'Fix k' row in Table 3 does not specify which fixed k value is used; please state this explicitly so the comparison with the dynamic strategy can be audited.
[Sec. 3.3, Eq. (8)] The total loss in Eq. (8) switches between Lrgb and a weighted combination of Locc and Lpc, but the training schedule for this switch is not specified. Please clarify how many iterations are spent on rendering optimization before the occupancy and parameter-consistency losses become active.
[Sec. 2 and References] The citation 'BEVFormer Li et al.' lacks a year and venue; the reference entry is also incomplete. Please correct this and the typo 'V oxformer' in the reference list.
[Sec. 4.4, Figures 4-5] Figures 4 and 5 are not referenced or described in the text beyond their captions; please add a sentence explaining what they show and how they support the k-distribution analysis.
[Sec. 5] The limitation that extreme-sized objects require adjusting the candidate k range is directly relevant to the adaptive-fusion claim; please discuss whether this indicates a sensitivity to the maximum k and how it interacts with the missing fixed-k=4 control.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the SOTA claim is an externally benchmarked empirical result, and the adaptive/rendering components are trained end-to-end rather than fitted then renamed as predictions.

full rationale

The paper's central claim is an empirical benchmark result (28.4% mIoU on nuScenes validation, Table 1), not a derivation from prior theorems. The adaptive fusion module predicts per-query k via MLP/Gumbel-Softmax/STE and is trained end-to-end with the occupancy loss; the volume-rendering module uses the input images only as photometric supervision, which is a standard self-supervised auxiliary loss rather than an input definitionally producing the output. No parameter is fitted to a subset and then reported as a prediction of a closely related quantity: k is a learned latent decision, and the headline number is a direct validation measurement. There are no load-bearing self-citations: references to 'Lei et al. [2023]' and 'Wei et al. [2023]' in the text correspond to different authors (Yinjie Lei; Yi Wei) and are not invoked as uniqueness theorems. The absence of a fixed-k=4 control in Table 4 is a legitimate experimental gap (the dynamic range includes k=4, so the comparison may conflate neighborhood-size capacity with adaptivity), and the validation-set selection of k-range and lambda is a generalization risk, but these are correctness concerns, not circular reductions by construction. The paper's own limitation statement that extreme-sized objects may require manually adjusting the candidate-k range further confirms that the k-range is a design choice rather than a derived prediction. Accordingly the circularity score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on standard deep-learning building blocks plus three notable assumptions: the STE/Gumbel-Softmax gradient is valid for learning k, image photometric loss improves 3D occupancy, and SurroundOcc pseudo-labels are trustworthy. The free parameters (k range, lambda, D-SSIM window) are hand-set and not ablated.

free parameters (3)

candidate k set = 1, 2, 3, 4
The range of neighborhood sizes is a hyperparameter selected by validation performance; Table 4 shows k=1-4 gives the best IoU/mIoU and is used in all experiments.
lambda (loss weight) = 0.2
Shared weight in the rendering loss (Eq. 7) and the final loss (Eq. 8); set to 0.2 without an ablation study.
D-SSIM window size = 11x11
Used in the photometric loss; reported in implementation details without sensitivity analysis.

assumptions (3)

domain assumption Gumbel-Softmax with a Straight-Through Estimator provides a usable gradient for the discrete neighborhood-size selection.
Section 3.1 relies on this to train the dynamic k predictor; no proof or empirical validation of the gradient approximation is provided.
domain assumption Photometric consistency between rendered and real images transfers to better 3D semantic occupancy, not just better 2D appearance.
Section 3.2 assumes the Lrgb and Lpc losses improve occupancy features; only the ablation in Table 3 is offered as evidence.
domain assumption SurroundOcc automatically generated occupancy labels are a reliable ground truth for training and evaluation.
Section 4.1 describes SurroundOcc as built by automated annotation; label noise is acknowledged in the literature but not quantified here.

how reviews work

0 comments

Cite this review

Pith. "Pith review of TACOcc:Target-Adaptive Cross-Modal Fusion with Volume Rendering for 3D Semantic Occupancy." pith.science (2026). https://pith.science/paper/KTL2KC5H

@misc{pith2026250512693,
  author       = {Pith},
  title        = {Pith review of: TACOcc:Target-Adaptive Cross-Modal Fusion with Volume Rendering for 3D Semantic Occupancy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KTL2KC5H}},
  note         = {Machine review of arXiv:2505.12693}
}

read the original abstract

The performance of multi-modal 3D occupancy prediction is limited by ineffective fusion, mainly due to geometry-semantics mismatch from fixed fusion strategies and surface detail loss caused by sparse, noisy annotations. The mismatch stems from the heterogeneous scale and distribution of point cloud and image features, leading to biased matching under fixed neighborhood fusion. To address this, we propose a target-scale adaptive, bidirectional symmetric retrieval mechanism. It expands the neighborhood for large targets to enhance context awareness and shrinks it for small ones to improve efficiency and suppress noise, enabling accurate cross-modal feature alignment. This mechanism explicitly establishes spatial correspondences and improves fusion accuracy. For surface detail loss, sparse labels provide limited supervision, resulting in poor predictions for small objects. We introduce an improved volume rendering pipeline based on 3D Gaussian Splatting, which takes fused features as input to render images, applies photometric consistency supervision, and jointly optimizes 2D-3D consistency. This enhances surface detail reconstruction while suppressing noise propagation. In summary, we propose TACOcc, an adaptive multi-modal fusion framework for 3D semantic occupancy prediction, enhanced by volume rendering supervision. Experiments on the nuScenes and SemanticKITTI benchmarks validate its effectiveness.

Figures

Figures reproduced from arXiv: 2505.12693 by the authors.

**Figure 2.** Adaptive Fuser. To achieve target-aligned fusion of sparse voxel features from two modalities, a [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Qualitative comparison results on the nuScenes validation set. The top left displays the input [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Scenarios with more small targets [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]

**Figure 6.** Figure 6: Rendered images display and comparison with real images. For each viewpoint, the image above is the [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

**Figure 7.** Figure 7: Additional qualitative comparison results on the nuScenes validation set. The top left displays the [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 9 canonical work pages

[3]

Recent advances in multi- modal 3d scene understanding: A comprehensive survey and evaluation

Yinjie Lei, Zixuan Wang, Feng Chen, Guoqing Wang, Peng Wang, and Yang Yang. Recent advances in multi- modal 3d scene understanding: A comprehensive survey and evaluation. arXiv preprint arXiv:2310.15676,

arXiv
[4]

Zhiqi Li, Zhiding Yu, David Austin, Mingsheng Fang, Shiyi Lan, Jan Kautz, and Jose M Alvarez

arXiv preprint arXiv:2203.17270. Zhiqi Li, Zhiding Yu, David Austin, Mingsheng Fang, Shiyi Lan, Jan Kautz, and Jose M Alvarez. Fb-occ: 3d occupancy prediction based on forward-backward view transformation. arXiv preprint arXiv:2307.01492, 2023b. Shice Liu, Yu Hu, Yiming Zeng, Qiankun Tang, Beibei Jin, Yinhe Han, and Xiaowei Li. See and think: Disentanglin...

arXiv
[5]

Decoupled weight decay regularization

Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101,

arXiv
[7]

The comparisons with two representative multi-modal methods highlight the effectiveness of our proposed TACOcc. 14 FrontLeft Front FrontRight M-CONet Co-Occ TACOcc(ours) GTDriver’s ViewBackLeft Back BackRight driveable surface car bus truck terrain vegetation sidewalk other flat pedestrian bicycle manmade motorcycle barrier construction vehicle trailer tr...

work page 2023
[9]

Co-occ: Coupling explicit feature fusion with volume rendering regularization for multi-modal 3d semantic occupancy prediction

Jingyi Pan, Zipeng Wang, and Lin Wang. Co-occ: Coupling explicit feature fusion with volume rendering regularization for multi-modal 3d semantic occupancy prediction. IEEE Robotics and Automation Letters, 2024a. Mingjie Pan, Li Liu, Jiaming Liu, Peixiang Huang, Longlong Wang, Shanghang Zhang, Shaoqing Xu, Zhiyi Lai, and Kuiyuan Yang. Uniocc: Unifying visi...

arXiv
[10]

Renderocc: Vision-centric 3d occupancy prediction with 2d rendering supervision

Mingjie Pan, Jiaming Liu, Renrui Zhang, Peixiang Huang, Xiaoqi Li, Hongwei Xie, Bing Wang, Li Liu, and Shanghang Zhang. Renderocc: Vision-centric 3d occupancy prediction with 2d rendering supervision. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 12404–12411. IEEE, 2024b. Jonah Philion and Sanja Fidler. Lift, splat, shoot:...

work page 2024
[11]

Occupancy anticipation for efficient explo- ration and navigation

Santhosh K Ramakrishnan, Ziad Al-Halah, and Kristen Grauman. Occupancy anticipation for efficient explo- ration and navigation. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part V 16, pages 400–418. Springer,

work page 2020
[13]

Dynamic fusion for multimodal data

Gaurav Sahu and Olga Vechtomova. Dynamic fusion for multimodal data. arXiv preprint arXiv:1911.03821,

arXiv 1911

Show all 18 references

[14]

Semantic segmentation-assisted scene completion for lidar point clouds

Xuemeng Yang, Hao Zou, Xin Kong, Tianxin Huang, Yong Liu, Wanlong Li, Feng Wen, and Hongbo Zhang. Semantic segmentation-assisted scene completion for lidar point clouds. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3555–3562. IEEE,

2021
[15]

Understanding straight-through estimator in training activation quantized neural nets

Penghang Yin, Jiancheng Lyu, Shuai Zhang, Stanley Osher, Yingyong Qi, and Jack Xin. Understanding straight-through estimator in training activation quantized neural nets. arXiv preprint arXiv:1903.05662,

1903 arXiv
[17]

Provable dynamic fusion for low-quality multimodal data

Qingyang Zhang, Haitao Wu, Changqing Zhang, Qinghua Hu, Huazhu Fu, Joey Tianyi Zhou, and Xi Peng. Provable dynamic fusion for low-quality multimodal data. In International conference on machine learning, pages 41753–41769. PMLR, 2023a. Shuo Zhang, Yupeng Zhai, Jilin Mei, and Y...

arXiv
[2016]

Ssc-rs: Elevate lidar semantic scene completion with representation separation and bev fusion

Jianbiao Mei, Yu Yang, Mengmeng Wang, Tianxin Huang, Xuemeng Yang, and Yong Liu. Ssc-rs: Elevate lidar semantic scene completion with representation separation and bev fusion. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1–8. IEEE,

2023
[2018]

Differentiable point-based radiance fields for efficient view synthesis

Qiang Zhang, Seung-Hwan Baek, Szymon Rusinkiewicz, and Felix Heide. Differentiable point-based radiance fields for efficient view synthesis. In SIGGRAPH Asia 2022 Conference Papers, pages 1–12,

2022
[2020]

Lmscnet: Lightweight multiscale 3d semantic completion

Luis Roldao, Raoul De Charette, and Anne Verroust-Blondet. Lmscnet: Lightweight multiscale 3d semantic completion. In 2020 International Conference on 3D Vision (3DV), pages 111–119. IEEE,

2020
[2021]

Inverse++: Vision-centric 3d semantic occupancy prediction assisted with 3d object detection

11 Zhenxing Ming, Julie Stephany Berrio, Mao Shan, and Stewart Worrall. Inverse++: Vision-centric 3d semantic occupancy prediction assisted with 3d object detection. arXiv preprint arXiv:2504.04732,

arXiv
[2022]

Tgp: Two-modal occupancy prediction with 3d gaussian and sparse points for 3d environment awareness.arXiv preprint arXiv:2503.09941,

Mu Chen, Wenyu Chen, Mingchuan Yang, Yuan Zhang, Tao Han, Xinchi Li, Yunlong Li, and Huaici Zhao. Tgp: Two-modal occupancy prediction with 3d gaussian and sparse points for 3d environment awareness.arXiv preprint arXiv:2503.09941,

arXiv
[2023]

Categorical reparameterization with gumbel-softmax

Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144,

arXiv
[2024]

Choosing smartly: Adaptive multimodal fusion for object detection in changing environments

Oier Mees, Andreas Eitel, and Wolfram Burgard. Choosing smartly: Adaptive multimodal fusion for object detection in changing environments. In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 151–156. IEEE,

2016

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [3]

Recent advances in multi- modal 3d scene understanding: A comprehensive survey and evaluation

Yinjie Lei, Zixuan Wang, Feng Chen, Guoqing Wang, Peng Wang, and Yang Yang. Recent advances in multi- modal 3d scene understanding: A comprehensive survey and evaluation. arXiv preprint arXiv:2310.15676,

arXiv

[2] [4]

Zhiqi Li, Zhiding Yu, David Austin, Mingsheng Fang, Shiyi Lan, Jan Kautz, and Jose M Alvarez

arXiv preprint arXiv:2203.17270. Zhiqi Li, Zhiding Yu, David Austin, Mingsheng Fang, Shiyi Lan, Jan Kautz, and Jose M Alvarez. Fb-occ: 3d occupancy prediction based on forward-backward view transformation. arXiv preprint arXiv:2307.01492, 2023b. Shice Liu, Yu Hu, Yiming Zeng, Qiankun Tang, Beibei Jin, Yinhe Han, and Xiaowei Li. See and think: Disentanglin...

arXiv

[3] [5]

Decoupled weight decay regularization

Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101,

arXiv

[4] [7]

The comparisons with two representative multi-modal methods highlight the effectiveness of our proposed TACOcc. 14 FrontLeft Front FrontRight M-CONet Co-Occ TACOcc(ours) GTDriver’s ViewBackLeft Back BackRight driveable surface car bus truck terrain vegetation sidewalk other flat pedestrian bicycle manmade motorcycle barrier construction vehicle trailer tr...

work page 2023

[5] [9]

Co-occ: Coupling explicit feature fusion with volume rendering regularization for multi-modal 3d semantic occupancy prediction

Jingyi Pan, Zipeng Wang, and Lin Wang. Co-occ: Coupling explicit feature fusion with volume rendering regularization for multi-modal 3d semantic occupancy prediction. IEEE Robotics and Automation Letters, 2024a. Mingjie Pan, Li Liu, Jiaming Liu, Peixiang Huang, Longlong Wang, Shanghang Zhang, Shaoqing Xu, Zhiyi Lai, and Kuiyuan Yang. Uniocc: Unifying visi...

arXiv

[6] [10]

Renderocc: Vision-centric 3d occupancy prediction with 2d rendering supervision

Mingjie Pan, Jiaming Liu, Renrui Zhang, Peixiang Huang, Xiaoqi Li, Hongwei Xie, Bing Wang, Li Liu, and Shanghang Zhang. Renderocc: Vision-centric 3d occupancy prediction with 2d rendering supervision. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 12404–12411. IEEE, 2024b. Jonah Philion and Sanja Fidler. Lift, splat, shoot:...

work page 2024

[7] [11]

Occupancy anticipation for efficient explo- ration and navigation

Santhosh K Ramakrishnan, Ziad Al-Halah, and Kristen Grauman. Occupancy anticipation for efficient explo- ration and navigation. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part V 16, pages 400–418. Springer,

work page 2020

[8] [13]

Dynamic fusion for multimodal data

Gaurav Sahu and Olga Vechtomova. Dynamic fusion for multimodal data. arXiv preprint arXiv:1911.03821,

arXiv 1911