REVIEW 4 major objections 3 minor 50 references
OccTrack360 establishes a benchmark and baseline for 4D panoptic occupancy tracking from surround-view fisheye cameras, with all-direction visibility masks.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
A new benchmark and baseline method for 4D panoptic occupancy tracking with surround-view fisheye cameras, built from KITTI-360.
T0 review reviewed 2026-08-02 challenge →
load-bearing objection OccTrack360 is a plausibly valuable fisheye benchmark, but the planar-XY backward completion and thin evaluation need fixes before I'd trust its numbers. the 4 major comments →
OccTrack360: 4D Panoptic Occupancy Tracking from Surround-View Fisheye Cameras
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
The paper claims that a valid 4D fisheye occupancy tracking benchmark needs both long temporal sequences and principled visibility supervision, and that fisheye-aware lifting plus center-focused supervision yields a strong baseline. The key supervision insight is that an occlusion mask must cast rays from every boundary voxel of the voxel space, not only from occupied voxels, so that upward and other sparse directions are not silently filtered out during training. The key geometric insight is that fisheye lifting should follow the Unified Projection Model, using the mirror parameter to convert image radius to a spherical angle, with a rectification bound that prevents invalid complex project
What carries the argument
The central objects are (1) the all-direction occlusion mask, which covers every direction in the voxel domain and avoids the incomplete ray coverage of prior masks, and (2) the MEI-based fisheye FoV mask, which computes the maximum radial angle from the Unified Projection Model and extends it across the voxel space to indicate valid fisheye supervision. On the method side, the Center Focusing Module replaces boundary-sensitive voxel offsets with a product-based focus feature that peaks at instance centers, and the Fisheye-based Enhanced Lifting module unprojects image features along fisheye rays using the mirror parameter and distortion coefficients.
Load-bearing premise
The ground-truth labels for occluded rear voxels are created by aligning earlier forward-facing voxels under planar XY motion and normalizing with SVD, which silently discards vertical motion; if real vertical variation is significant, the labels carry systematic vertical error.
What would settle it
Run the benchmark's label-generation pipeline on sequences with known uphill or downhill slopes and compare the completed rear voxels against dense LiDAR or manual annotation; if vertical error grows with the slope grade, the Z-truncating alignment is falsified.
If this is right
- If OccTrack360 is adopted, researchers gain a common ground for comparing fisheye 4D occupancy trackers on long sequences, not just short clips.
- The all-direction occlusion mask can prevent training-time filtering that hides voxel ambiguity, making predictions visually and geometrically interpretable.
- The Fisheye-based Enhanced Lifting module provides a reusable recipe for adapting perspective lifting pipelines to wide-field-of-view cameras.
- The reported tracking-metric trade-off suggests that instance association, not geometry alone, is the current bottleneck for fisheye 4D tracking.
Where Pith is reading between the lines
- The backward-completion step could be transferred to pinhole benchmarks to add rear-object ground truth, but its planar motion assumption would need explicit handling for sloped roads.
- The vertical error introduced by SVD normalization could be measured by comparing completed voxels against dense LiDAR on uphill and downhill segments; if it grows with grade, the labels carry systematic bias.
- The FoV mask's manual truncation of the projected circle could be automated with a closed-form boundary polygon, making similar masks easier to build for other fisheye datasets.
- The center-focus supervision idea could generalize to any dense 3D prediction task where boundary supervision is fragile under distortion, such as fisheye BEV segmentation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces OccTrack360, a benchmark for 4D panoptic occupancy tracking from surround-view fisheye cameras, built on KITTI-360, with long temporal sequences (174–2234 frames), instance-level voxel IDs, an all-direction occlusion mask, and an MEI-based fisheye field-of-view mask. The authors also propose FoSOcc, a fisheye-oriented occupancy tracking framework with a Center Focusing Module (CFM) for instance-center supervision and a Fisheye-based Enhanced Lifting (FEL) module that extends LSS lifting to the unified projection model. Experiments are reported on Occ3D-Waymo and OccTrack360, showing CFM gains on regular categories and establishing a baseline on the new benchmark.
Significance. If the benchmark construction is sound, OccTrack360 fills a real gap: no existing benchmark provides surround-view fisheye imagery, long sequences, and instance-level voxel tracking simultaneously. The proposed all-direction occlusion mask and MEI-based FoV mask are reasonable and more principled than prior pinhole-only visibility masks. The code and benchmark are promised to be public, which strengthens reproducibility. However, the paper's central claims rest on several load-bearing issues—a mathematical inconsistency in the FEL rectification bound, evaluation on a single sequence, a missing FEL ablation, and a known but unquantified vertical bias in the backward completion step—so the current evidence does not yet fully support the stated conclusions.
major comments (4)
- The rectification bound in Eq. (10) is inconsistent with the derivation in Eqs. (3)–(5). Eq. (5) correctly derives a_max = sqrt(1/(ξ²−1)) from the discriminant condition ∆ ≥ 0, but Eq. (10) uses arect = min(a, sqrt((1+ξ)/(ξ−1))). For ξ > 1, sqrt((1+ξ)/(ξ−1)) is always larger than sqrt(1/(ξ²−1)) by a factor of (ξ+1); e.g., for ξ=1.5 the former is ≈2.24 while the correct bound is ≈0.89. Thus the min() in Eq. (10) will not clamp a to the admissible range, and the subsequent inversion via Eq. (4) can produce complex cosθ values. This undermines the geometric validity of FEL, which is a core contribution. The manuscript must correct Eq. (10), specify which root of Eq. (4) is used, and retest the method with the corrected bound.
- The OccTrack360 evaluation is performed only on the Seq06 sequence, despite the benchmark claiming diverse sequences with 174–2234 frames. A single sequence cannot support claims about long-term or diverse evaluation, and it makes the reported numbers highly sensitive to one environmental layout, one set of traffic participants, and one slope profile. The authors should either evaluate on a larger subset of their benchmark (and report per-sequence variance) or substantially temper the diversity claim. This is load-bearing because the benchmark's central selling point is its long and varied sequences.
- FEL, one of the two primary contributions, is never ablated. Table III ablates only the CFM components (instance-level normalization and supervised focus feature) on Occ3D-Waymo, which uses pinhole cameras and does not involve FEL. No experiment isolates the contribution of FEL on OccTrack360. Without such an ablation, the improvements in Table II cannot be attributed to the fisheye lifting module. Additionally, Table II shows that FoSOcc decreases OccAQ (from 17.07 to 13.11 all-FoV; 22.32 to 17.02 Fisheyes) and OccSTQ (from 14.84 to 13.32 all-FoV; 17.20 to 15.71 Fisheyes), which contradicts the abstract's claim that the method 'improves occupancy tracking quality.' The authors should clarify whether the claim is about segmentation quality only, and add the missing FEL ablation.
- The backward completion of occluded voxels assumes planar XY motion and uses SVD normalization that, as the text admits, 'inevitably disregards genuine variations along the z-axis.' On sloped terrain (Fig. 4), this introduces a systematic vertical bias in the completed occupancy. Because the all-direction occlusion mask is constructed from this occupancy by ray-casting from boundary voxels, the mask inherits the same vertical error, and every benchmark metric (OccSQ/OccAQ/OccSTQ) is evaluated against a ground truth that can be vertically misaligned by more than the 0.2 m voxel size on inclines. The paper acknowledges the issue but does not quantify its magnitude or downstream impact. This is load-bearing for the benchmark's validity; the authors must either provide a quantitative analysis of the vertical error (e.g., on segments with known slopes) or adjust the completion procedure to ac
minor comments (3)
- The symbol R is introduced as 'the filling range' but then used as if it were a binary mask via the Hadamard product R◦V. This is confusing; please define R clearly (e.g., a 3D binary mask over the fill region).
- The lifting module is called 'Spherical Lift Module (SLM)' in the Introduction but 'Fisheye-based Enhanced Lifting (FEL)' in the abstract and Section IV. Please use one name consistently.
- The choice between the two roots for cosθ is not specified. The text says 'the chosen root' but never defines which root is used. Since this is central to FEL, please state the explicit closed form, including the sign before the square root.
Circularity Check
No significant circularity: benchmark and method are derived from external data and standard camera models; the admitted z-axis bias is a validity risk, not a circular step.
full rationale
I walked the claimed derivation chain. The OccTrack360 benchmark labels are constructed from KITTI-360 bounding-box and semantic annotations by deterministic voxelization, forward projection, backward completion, ray-cast occlusion masking, and an MEI-based fisheye FoV mask. These are data-construction procedures, not predictions derived from the method being evaluated. FoSOcc is trained with standard supervised losses on the resulting labels and evaluated both on the external Occ3D-Waymo benchmark and on OccTrack360; no fitted parameter is renamed as a prediction, and no equation is defined in terms of the metric it is claimed to explain. The paper's own admission that the planar-XY SVD normalization in backward completion 'inevitably disregards genuine variations along the z-axis' (Section III, Object Completion; Fig. 4) is a legitimate ground-truth validity concern, but it is a bias shared by the supervision and the evaluation, not a circular reduction: it does not force any method's score. Self-citations to prior work such as the voxel-offset formulation or TrackOcc's metrics are used as baselines or building blocks, not as load-bearing proofs. The central contributions—all-direction occlusion masks, MEI-based FoV masks, and fisheye-aware lifting—stand independently of these citations. Therefore the appropriate finding is no significant circularity.
Axiom & Free-Parameter Ledger
axioms (4)
- domain assumption Backward completion of occluded voxels can be approximated by aligning current and previous forward-facing voxels under planar motion (rotation about Z, translation in XY), suppressing z-axis variation.
- domain assumption The fisheye cameras are exactly described by the MEI (Unified Projection Model) with a single mirror parameter ξ; if non-central or different distortion models apply, the FoV mask is invalid.
- domain assumption KITTI-360 3D bounding box tracks provide accurate instance-level dynamic poses for transforming voxels across frames.
- domain assumption The occlusion mask computed from voxel boundaries is sufficient for visibility supervision; the boundary-voxel ray construction covers all directions without missing valid rays.
Cite this review
Pith. "Pith review of OccTrack360: 4D Panoptic Occupancy Tracking from Surround-View Fisheye Cameras." pith.science (2026). https://pith.science/paper/WVD5IQJ2
@misc{pith2026260308521,
author = {Pith},
title = {Pith review of: OccTrack360: 4D Panoptic Occupancy Tracking from Surround-View Fisheye Cameras},
year = {2026},
howpublished = {\url{https://pith.science/paper/WVD5IQJ2}},
note = {Machine review of arXiv:2603.08521}
}
read the original abstract
Understanding dynamic 3D environments in a spatially continuous and temporally consistent manner is fundamental for robotics and autonomous driving. While recent advances in occupancy prediction provide a unified representation of scene geometry and semantics, progress in 4D panoptic occupancy tracking remains limited by the lack of benchmarks that support surround-view fisheye sensing, long temporal sequences, and instance-level voxel tracking. To address this gap, we present OccTrack360, a new benchmark for 4D panoptic occupancy tracking from surround-view fisheye cameras. OccTrack360 provides substantially longer and more diverse sequences (174~2234 frames) than prior benchmarks, together with principled voxel visibility annotations, including an all-direction occlusion mask and an MEI-based fisheye field-of-view mask. To establish a strong fisheye-oriented baseline, we further propose Focus on Sphere Occ (FoSOcc), a framework that addresses two core challenges in fisheye occupancy tracking: distorted spherical projection and inaccurate voxel-space localization. FoSOcc includes a Center Focusing Module (CFM) to enhance instance-aware spatial localization through supervised focus guidance, and a Fisheye-based Enhanced Lifting (FEL) that extends perspective lifting to fisheye imaging under the Unified Projection Model. Extensive experiments on Occ3D-Waymo and OccTrack360 show that our method improves occupancy tracking quality with notable gains on geometrically regular categories, and establishes a strong baseline for future research on surround-view fisheye 4D occupancy tracking. The benchmark and source code will be made publicly available at https://github.com/YouthZest-Lin/OccTrack360.
Figures
Reference graph
Works this paper leans on
-
[1]
A survey on occupancy perception for autonomous driving: The information fusion perspective,
H. Xuet al., “A survey on occupancy perception for autonomous driving: The information fusion perspective,”Information Fusion, 2025
2025
-
[2]
A survey on end-to-end perception and prediction for autonomous driving,
Y . Huet al., “A survey on end-to-end perception and prediction for autonomous driving,”Machine Intelligence Research, 2025
2025
-
[3]
ALOcc: Adaptive lifting-based 3D semantic occu- pancy and cost volume-based flow predictions,
D. Chenet al., “ALOcc: Adaptive lifting-based 3D semantic occu- pancy and cost volume-based flow predictions,” inICCV, 2025
2025
-
[4]
V oxDet: Rethinking 3D semantic occupancy prediction as dense object detection,
W. Liet al., “V oxDet: Rethinking 3D semantic occupancy prediction as dense object detection,” inNeurIPS, 2025
2025
-
[5]
Occ3D: A large-scale 3D occupancy prediction benchmark for autonomous driving,
X. Tianet al., “Occ3D: A large-scale 3D occupancy prediction benchmark for autonomous driving,” inNeurIPS, 2023
2023
-
[6]
SSCBench: A large-scale 3D semantic scene completion benchmark for autonomous driving,
Y . Liet al., “SSCBench: A large-scale 3D semantic scene completion benchmark for autonomous driving,” inIROS, 2024
2024
-
[7]
Single view point omnidirectional camera calibration from planar grids,
C. Mei and P. Rives, “Single view point omnidirectional camera calibration from planar grids,” inICRA, 2007
2007
-
[8]
SemanticKITTI: A dataset for semantic scene understanding of LiDAR sequences,
J. Behleyet al., “SemanticKITTI: A dataset for semantic scene understanding of LiDAR sequences,” inICCV, 2019
2019
-
[9]
OpenOccupancy: A large scale benchmark for surrounding semantic occupancy perception,
X. Wanget al., “OpenOccupancy: A large scale benchmark for surrounding semantic occupancy perception,” inICCV, 2023
2023
-
[10]
Scene as occupancy,
W. Tonget al., “Scene as occupancy,” inICCV, 2023
2023
-
[11]
WoodScape: A multi-task, multi-camera fisheye dataset for autonomous driving,
S. Yogamaniet al., “WoodScape: A multi-task, multi-camera fisheye dataset for autonomous driving,” inICCV, 2019
2019
-
[12]
SynWoodScape: Synthetic surround-view fisheye camera dataset for autonomous driving,
A. R. Sekkatet al., “SynWoodScape: Synthetic surround-view fisheye camera dataset for autonomous driving,”IEEE Robotics and Automa- tion Letters, 2022
2022
-
[13]
TrackOcc: Camera-based 4D panoptic occupancy tracking,
Z. Chenet al., “TrackOcc: Camera-based 4D panoptic occupancy tracking,” inICRA, 2025
2025
-
[14]
Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3D,
J. Philion and S. Fidler, “Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3D,” inECCV, 2020
2020
-
[15]
Are we ready for autonomous driving? The KITTI vision benchmark suite,
A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? The KITTI vision benchmark suite,” inCVPR, 2012
2012
-
[16]
Symphonize 3D semantic scene completion with contextual instance queries,
H. Jianget al., “Symphonize 3D semantic scene completion with contextual instance queries,” inCVPR, 2024
2024
-
[17]
GaussianOcc: Fully self-supervised and efficient 3D occupancy estimation with gaussian splatting,
W. Ganet al., “GaussianOcc: Fully self-supervised and efficient 3D occupancy estimation with gaussian splatting,” inICCV, 2025
2025
-
[18]
SA-Occ: Satellite-assisted 3D occupancy prediction in real world,
C. Chenet al., “SA-Occ: Satellite-assisted 3D occupancy prediction in real world,” inICCV, 2025
2025
-
[19]
MinkOcc: Towards real-time label-efficient semantic occupancy prediction,
S. Szeet al., “MinkOcc: Towards real-time label-efficient semantic occupancy prediction,” inIROS, 2025
2025
-
[20]
FB-OCC: 3D occupancy prediction based on forward- backward view transformation,
Z. Liet al., “FB-OCC: 3D occupancy prediction based on forward- backward view transformation,”arXiv preprint arXiv:2307.01492, 2023
Pith/arXiv arXiv 2023
-
[21]
Occupancy learning with spatiotemporal memory,
Z. Lenget al., “Occupancy learning with spatiotemporal memory,” in ICCV, 2025
2025
-
[22]
UniOcc: A unified benchmark for occupancy fore- casting and prediction in autonomous driving,
Y . Wanget al., “UniOcc: A unified benchmark for occupancy fore- casting and prediction in autonomous driving,” inICCV, 2025
2025
-
[23]
Cam4DOcc: Benchmark for camera-only 4D occupancy forecasting in autonomous driving applications,
J. Maet al., “Cam4DOcc: Benchmark for camera-only 4D occupancy forecasting in autonomous driving applications,” inCVPR, 2024
2024
-
[24]
Stream and query-guided feature aggregation for efficient and effective 3D occupancy prediction,
S. Moonet al., “Stream and query-guided feature aggregation for efficient and effective 3D occupancy prediction,”arXiv preprint arXiv:2503.22087, 2025
Pith/arXiv arXiv 2025
-
[25]
Latent gaussian splatting for 4D panoptic occupancy tracking,
M. Luzet al., “Latent gaussian splatting for 4D panoptic occupancy tracking,”arXiv preprint arXiv:2602.23172, 2026
Pith/arXiv arXiv 2026
-
[26]
PaSCo: Urban 3D panoptic scene completion with uncertainty awareness,
A.-Q. Caoet al., “PaSCo: Urban 3D panoptic scene completion with uncertainty awareness,” inCVPR, 2024
2024
-
[27]
PanoSSC: Exploring monocular panoptic 3D scene reconstruction for autonomous driving,
Y . Shiet al., “PanoSSC: Exploring monocular panoptic 3D scene reconstruction for autonomous driving,” in3DV, 2024
2024
-
[28]
Survey on fish-eye cameras and their applications in intelligent vehicles,
Y . Qianet al., “Survey on fish-eye cameras and their applications in intelligent vehicles,”IEEE Transactions on Intelligent Transportation Systems, 2022
2022
-
[29]
FisheyeBEVSeg: Surround view fisheye cam- eras based bird’s-eye view segmentation for autonomous driving,
S. Yogamaniet al., “FisheyeBEVSeg: Surround view fisheye cam- eras based bird’s-eye view segmentation for autonomous driving,” in CVPRW, 2024
2024
-
[30]
ArticuBEVSeg: Road semantic understanding and its application in bird’s eye view from panoramic vision system of long combination vehicles,
W. Liu and W. Wang, “ArticuBEVSeg: Road semantic understanding and its application in bird’s eye view from panoramic vision system of long combination vehicles,”IEEE Robotics and Automation Letters, 2025
2025
-
[31]
Dur360BEV: A real-world 360-degree single camera dataset and benchmark for bird-eye view mapping in autonomous driving,
W. Eet al., “Dur360BEV: A real-world 360-degree single camera dataset and benchmark for bird-eye view mapping in autonomous driving,” inICRA, 2025
2025
-
[32]
F2BEV: Bird’s eye view generation from surround-view fisheye camera images for automated driving,
E. U. Samaniet al., “F2BEV: Bird’s eye view generation from surround-view fisheye camera images for automated driving,” inIROS, 2023
2023
-
[33]
Exploring surround-view fisheye camera 3D object detection,
C. Liet al., “Exploring surround-view fisheye camera 3D object detection,” inAAAI, 2026
2026
-
[34]
OmniDet: Surround view cameras based multi-task visual perception network for autonomous driving,
V . R. Kumaret al., “OmniDet: Surround view cameras based multi-task visual perception network for autonomous driving,”IEEE Robotics and Automation Letters, 2021
2021
-
[35]
Surround-view fisheye BEV-perception for valet park- ing: Dataset, baseline and distortion-insensitive multi-task framework,
Z. Wuet al., “Surround-view fisheye BEV-perception for valet park- ing: Dataset, baseline and distortion-insensitive multi-task framework,” IEEE Transactions on Intelligent V ehicles, 2023
2023
-
[36]
Surround-view fisheye camera viewpoint augmentation for image semantic segmentation,
J. Choet al., “Surround-view fisheye camera viewpoint augmentation for image semantic segmentation,”IEEE Access, 2023
2023
-
[37]
DSBEV: Docking space segmentation for autonomous surface vehicle in bird’s eye view,
C. Panget al., “DSBEV: Docking space segmentation for autonomous surface vehicle in bird’s eye view,”IEEE Transactions on Industrial Informatics, 2025
2025
-
[38]
OmniOcc: Cylindrical voxel-based semantic occupancy prediction for omnidirectional vision systems,
C. Wuet al., “OmniOcc: Cylindrical voxel-based semantic occupancy prediction for omnidirectional vision systems,”IEEE Access, 2025
2025
-
[39]
EquivFisheye: A spherical fusion framework for panoramic 3D perception with surround-view fisheye cameras,
Z. Yanget al., “EquivFisheye: A spherical fusion framework for panoramic 3D perception with surround-view fisheye cameras,”In- formation Fusion, 2025
2025
-
[40]
FishBEV: Distortion-resilient bird’s eye view seg- mentation with surround-view fisheye cameras,
H. Liet al., “FishBEV: Distortion-resilient bird’s eye view seg- mentation with surround-view fisheye cameras,”arXiv preprint arXiv:2509.13681, 2025
arXiv 2025
-
[41]
Y . Sunet al., “KD360-V oxelBEV: LiDAR and 360-degree camera cross modality knowledge distillation for bird’s-eye-view segmenta- tion,”arXiv preprint arXiv:2512.15311, 2025
arXiv 2025
-
[42]
KITTI-360: A novel dataset and benchmarks for urban scene understanding in 2D and 3D,
Y . Liaoet al., “KITTI-360: A novel dataset and benchmarks for urban scene understanding in 2D and 3D,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
2023
-
[43]
COTR: Compact occupancy transformer for vision- based 3D occupancy prediction,
Q. Maet al., “COTR: Compact occupancy transformer for vision- based 3D occupancy prediction,” inCVPR, 2024
2024
-
[44]
Close-range camera calibration,
C. B. Duane, “Close-range camera calibration,”Photogramm. Eng, 1971
1971
-
[45]
MinVIS: A minimal video instance segmentation framework without video-based training,
D.-A. Huang, Z. Yu, and A. Anandkumar, “MinVIS: A minimal video instance segmentation framework without video-based training,” in NeurIPS, 2022
2022
-
[46]
CTVIS: Consistent training for online video instance segmentation,
K. Yinget al., “CTVIS: Consistent training for online video instance segmentation,” inICCV, 2023
2023
-
[47]
3D multi-object tracking: A baseline and new evaluation metrics,
X. Wenget al., “3D multi-object tracking: A baseline and new evaluation metrics,” inIROS, 2020
2020
-
[48]
4D panoptic LiDAR segmentation,
M. Aygunet al., “4D panoptic LiDAR segmentation,” inCVPR, 2021
2021
-
[49]
Deep residual learning for image recognition,
K. Heet al., “Deep residual learning for image recognition,” inCVPR, 2016
2016
-
[50]
Decoupled weight decay regularization,
I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” inICLR, 2019
2019
This paper was first reviewed by deepseek-v4-flash on August 2, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.