REVIEW 22 cited by
FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformation
read the original abstract
This technical report summarizes the winning solution for the 3D Occupancy Prediction Challenge, which is held in conjunction with the CVPR 2023 Workshop on End-to-End Autonomous Driving and CVPR 23 Workshop on Vision-Centric Autonomous Driving Workshop. Our proposed solution FB-OCC builds upon FB-BEV, a cutting-edge camera-based bird's-eye view perception design using forward-backward projection. On top of FB-BEV, we further study novel designs and optimization tailored to the 3D occupancy prediction task, including joint depth-semantic pre-training, joint voxel-BEV representation, model scaling up, and effective post-processing strategies. These designs and optimization result in a state-of-the-art mIoU score of 54.19% on the nuScenes dataset, ranking the 1st place in the challenge track. Code and models will be released at: https://github.com/NVlabs/FB-BEV.
Forward citations
Cited by 22 Pith papers
-
FDR-Occ: Factorized Dense Routing for Full-Spectrum 3D Occupancy Prediction
Factorized Dense Routing approximates unconstrained 2D-to-3D feature mixing by hierarchical tensor contractions, yielding global-context occupancy prediction that remains robust without camera extrinsics.
-
Humanoid-OmniOcc: Stereo-Based Full-View Occupancy Dataset for Embodied AI
Humanoid-OmniOcc delivers a large-scale panoramic stereo occupancy dataset for humanoid robots via Real2Sim2Real, with a model that outperforms monocular baselines in both unseen sim scenes and real settings.
-
VISA: VLM-Guided Instance Semantic Auditing for 3D Occupancy World Models
VISA improves closed-set 3D occupancy mIoU on nuScenes by using VLM instance audits as reliability-weighted semantic supervisors during training of existing world models.
-
SparseOcc++: Geometry-Aware Sparse Latent Representation for Semantic Occupancy Prediction
SparseOcc++ decouples geometry completion (via orthogonal SCF regression on sparse anchors) from semantics, improving IoU 2.3 points and running 3.9 imes faster than SparseOcc on nuScenes while 5.9 imes faster than Oc...
-
RayOcc: Occlusion-Aware Ray Occupancy Estimation via Gaussian Mixture Intensity
RayOcc models each camera ray as a non-normalized Gaussian mixture with Poisson-based occupancy probabilities, allowing multiple depth hypotheses per ray and improving Gaussian-initialized 3D occupancy prediction on nuScenes.
-
VISA: VLM-Guided Instance Semantic Auditing for 3D Occupancy World Models
Offline VLM instance audits, grounded to matched object voxels and distilled via taxonomy, attribute, and graph losses, raise OccWorld and GaussianWorld closed-set occupancy mIoU without inference-time VLM cost.
-
UnsOcc: 3D Semantic Occupancy Prediction in Unstructured Scene via Rendering Fusion
UnsOcc proposes RenderFusion and GSRefinement to improve 3D semantic occupancy prediction in unstructured scenes by enhancing cross-modal fusion and long-tail supervision, outperforming SOTA on a new mine dataset and ...
-
Height-Guided Projection Reparameterization for Camera-LiDAR Occupancy
HiPR improves 3D occupancy prediction by reparameterizing image-to-voxel projections using LiDAR-derived height priors to adapt sampling ranges to scene sparsity and height variations.
-
Height-Guided Projection Reparameterization for Camera-LiDAR Occupancy
HiPR improves 3D occupancy prediction by adaptively reparameterizing projection sampling ranges using LiDAR height priors instead of fixed uniform pillars.
-
FreeOcc: Training-Free Embodied Open-Vocabulary Occupancy Prediction
FreeOcc enables training-free open-vocabulary 3D occupancy prediction from RGB-D sequences by combining SLAM, dense Gaussian maps, off-the-shelf vision-language models, and probabilistic projection, achieving over 2x ...
-
OccTrack360: 4D Panoptic Occupancy Tracking from Surround-View Fisheye Cameras
A new benchmark and baseline method for 4D panoptic occupancy tracking with surround-view fisheye cameras, built from KITTI-360.
-
Monocular Open Vocabulary Occupancy Prediction for Indoor Scenes
A 3D Language-Embedded Gaussians framework with opacity-aware Poisson volumetric aggregation and progressive temperature decay achieves 59.50 IoU and 21.05 mIoU on Occ-ScanNet for open-vocabulary indoor occupancy.
-
TFusionOcc: T-Primitive Based Object-Centric Multi-Sensor Fusion Framework for 3D Occupancy Prediction
TFusionOcc uses a family of Student's t-distribution T-primitives and a T-mixture model for multi-sensor 3D occupancy prediction, reporting state-of-the-art results on nuScenes.
-
DVGT: Driving Visual Geometry Transformer
DVGT predicts metric-scaled global 3D point maps and ego poses from unposed multi-view driving video, beating prior geometry models on several driving benchmarks.
-
Lotus-2: Advancing Geometric Dense Prediction with Powerful Image Generative Model
Lotus-2 is a two-stage deterministic adaptation of diffusion priors that achieves state-of-the-art monocular depth estimation with only 59K training samples.
-
Semantic Causality-Aware Vision-Based 3D Occupancy Prediction
A class-conditional gradient loss (Causal Loss) plus channel-grouped lifting, learnable camera offsets, and normalized convolution raises Occ3D mIoU by 1.2/0.8 points and cuts the camera-noise mIoU drop from 32% to 7%.
-
GPOcc++: Unified Sparse Gaussian Occupancy Prediction with Visual Geometry Priors
A unified framework converts surface geometry priors into sparse Gaussian occupancy predictions and extends it to multi-view and temporal inputs.
-
Sparsity-Aware Voxel Attention and Foreground Modulation for 3D Semantic Scene Completion
VoxSAMNet introduces sparsity-aware deformable attention via a dummy node and foreground modulation with dropout plus text-guided filtering to reach new state-of-the-art mIoU of 18.2% on SemanticKITTI and 20.2% on SSC...
-
BEVPredFormer: Spatio-temporal Attention for BEV Instance Prediction in Autonomous Driving
BEVPredFormer uses attention-based temporal processing and 3D camera projection to match or exceed prior methods on nuScenes for BEV instance prediction.
-
SparseWorld-TC: Trajectory-Conditioned Sparse Occupancy World Model
A sparse transformer predicts multi-frame 3D occupancy from images without BEV or VAE tokenization and reports SOTA results on nuScenes for 1-3s forecasting under arbitrary trajectories.
-
Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Method
UniScenev2 scales occupancy-centric driving-scene generation to NuPlan scale, releasing a 3.6M-frame semantic-occupancy dataset and jointly generating occupancy, video, and LiDAR that beats published baselines on its ...
-
CaR1: A Multi-Modal Baseline for BEV Vehicle Segmentation via Camera-Radar Fusion
CaR1 achieves 57.6 IoU on nuScenes BEV vehicle segmentation by fusing camera features with grid-scattered radar features via adaptive weighting.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.