{"id":"5681d97b-79cf-4bc3-88a3-c4322e040bd4","arxiv_id":"2607.18078","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"VGOcc fuses frozen VGGT/DINOv2 features with sparse 3D Gaussians to reach 34.07 SC IoU and 21.75 SSC mIoU on nuScenes, besting prior vision-only occupancy methods.","lead":"VGOcc predicts 3D driving-scene occupancy from six surround-view cameras by representing the scene as sparse 3D Gaussians seeded with geometric and semantic features extracted from pretrained foundation models. It reports state-of-the-art accuracy on the nuScenes occupancy benchmark, suggesting that frozen foundation-model features improve sparse-Gaussian 3D scene understanding for autonomous driving.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'state-of-the-art' claim is supported only by a table that omits the two closest Gaussian occupancy methods, VG3S and VG3T, so the central empirical claim is currently unsubstantiated.","rationale":"The reader's weakest_assumption focuses on the reliability of frozen VGGT/DINOv2 features and the DPT depth prior for Gaussian center seeding. That is a plausible technical risk, but I see the more load-bearing issue as the completeness of the empirical comparison. The paper's abstract and conclusion explicitly claim state-of-the-art performance, yet the two most closely related published methods — VG3S and VG3T — are cited in the related work but absent from the quantitative comparison. Because the claim is empirical, omitting the strongest near-neighbor baselines means the central conclusion is not yet supported, regardless of how well the depth prior works. The reader did mention VG3S/VG3T comparisons in the rationale, so this is a partial awareness, but the designated weakest_assumption is a different concern; hence I mark disagreement. I do not call for rejection: the architecture is clearly described, the ablations are extensive and show meaningful contributions from each component, and if the omitted comparisons are added and VGOcc still wins, the paper would be accept-shaped. The current verdict of CONDITIONAL already captures this, so I recommend no change. My concrete test would settle the matter by running the missing baselines under the identical protocol rather than relying on author-reported numbers, which may use different evaluation splits or label variants.","tokens_in":12184,"tokens_out":3313,"duration_ms":38866,"concrete_test":"Obtain the official VG3S (and VG3T, if its evaluation protocol is compatible) model and evaluate it on the same nuScenes SurroundOcc split and metrics used in Table 1, computing SC IoU and SSC mIoU per class. If VG3S or VG3T reaches or exceeds VGOcc's 34.07 SC IoU or 21.75 SSC mIoU, the 'state-of-the-art' claim fails as stated. In the same evaluation, run VGOcc with at least three seeds to report mean and standard deviation, confirming the single-run numbers are stable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is empirical: VGOcc achieves state-of-the-art vision-only 3D occupancy prediction on nuScenes, with 34.07 SC IoU and 21.75 SSC mIoU. The evidence for this is Table 1, which compares against structured baselines and GaussianFormer/GaussianFormer-2. However, the paper itself cites VG3S and VG3T as closely related Gaussian occupancy methods, one of which (VG3S) explicitly adapts frozen foundation features for Gaussian occupancy learning. These methods are not included in Table 1, so the strongest related competitors are absent from the comparison. If either method reports higher SC IoU or SSC mIoU on the same nuScenes/SurroundOcc protocol, the stated 'state-of-the-art' conclusion is false. This is not an internal inconsistency; the architecture and ablations may be sound. But the headline empirical claim is only as strong as the baseline set, and omitting the two most relevant methods makes it premature. Supporting issues include single-run reporting without error bars and no code release, which further weaken the empirical claim.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes VGOcc, a sparse-Gaussian 3D semantic occupancy model for surround-view driving. It uses frozen VGGT and DINOv2 features to construct 'Visual-Geometric Gaussians': ray-depth hypotheses from a DPT head are lifted to 3D, balanced by voxel-thinned sampling, and paired with DINO attributes to initialize Gaussian queries; a pose-aware feature learning module conditions multi-level foundation tokens on camera/ray geometry and aggregates cross-view features at each refinement stage; the standard Gaussian decoder then refines and renders the occupancy field. On nuScenes with SurroundOcc labels, the authors report 34.07 SC IoU and 21.75 SSC mIoU, exceeding GaussianFormer-2 by 3.51 and 1.73, respectively. Ablations isolate the birth and feature-learning components under a fixed Gaussian budget and decoder.","tokens_in":12521,"tokens_out":10756,"duration_ms":106762,"significance":"If the reported results hold, VGOcc makes a solid contribution to vision-centric occupancy prediction by showing that frozen foundation-model cues can be converted into useful initialization and refinement signals for sparse Gaussian primitives. The design is well motivated, and the ablations are informative; the comparison with GaussianFormer-2 under the same decoder is a good control. The paper is not circular: the headline metrics come from external ground-truth labels and published baselines. However, the empirical contribution is weakened by the absence of the closest Gaussian occupancy baselines (VG3S, VG3T), by single-run results without error bars, and by the lack of code/checkpoint release. These issues need to be addressed before the 'state-of-the-art' claim can be accepted.","major_comments":[{"comment":"The headline SOTA claim is not supported by the baseline set. The Related Work explicitly names VG3S (Yan, Pei, and Shen 2026) and VG3T (Kim and Lee 2025) as sparse Gaussian occupancy methods, and VG3S is described as adapting frozen foundation features for Gaussian occupancy learning — the closest comparator to VGOcc. Neither appears in Table 1. Since the paper's central assertion is that VGOcc achieves state-of-the-art performance, the comparison must include these methods, or provide a specific justification for their exclusion and soften the claim. Please add their published nuScenes/SurroundOcc numbers and update the text accordingly.","section":"Quantitative Results / Table 1"},{"comment":"The text states that VoxFormer, COTR, and SparseOcc are compared, but these rows are absent from Table 1. The reported baseline table is therefore incomplete relative to the paper's own description. Please either add the missing rows or revise the text. Also clarify which rows use the SurroundOcc dense labels versus the original LiDAR segmentation labels, since the table mixes TPVFormer and TPVFormer†.","section":"Compared methods / Table 1"},{"comment":"All quantitative results are reported as single runs with no error bars or significance tests. Given that the central claim is a measured improvement over GaussianFormer-2, the absence of run-to-run variability makes it difficult to assess whether the 1.73 mIoU margin is stable. Please report results over multiple seeds (mean±std) for at least VGOcc and the key baselines, and make code/checkpoints available to reproduce the numbers. If multiple-seed evaluation is not standard for this venue, the paper should state this explicitly.","section":"Tables 1 and 2"}],"minor_comments":[{"comment":"Specify whether the reported results are on the nuScenes validation split, and define the ray-depth posterior supervision: number of depth bins, depth range, and construction of the ray-depth target. This is needed to reproduce Eq. (4).","section":"Experimental Setup"},{"comment":"The notation d_b in Eq. (4), and y^j_{c'ik} and δ_k in Eq. (10), are not fully defined. Please add explicit definitions so that the cross-view aggregation is reproducible.","section":"Method / Eqs. (4) and (10)"},{"comment":"Figure 1 contains informal emojis and unclear arrows that are inappropriate for a formal paper; replace with a clean schematic. Also fix typos: 'V enue' in the Table 1 header and 'vison' in the 'Compared methods' paragraph.","section":"Figure 1 and typos"},{"comment":"The caption says 'SurroundOcc dataset,' but the experiments are on nuScenes with SurroundOcc labels. Make the wording consistent.","section":"Figure 3 caption"},{"comment":"The paper motivates sparse Gaussians by efficiency but reports no runtime or memory comparison with GaussianFormer-2. A short efficiency table would strengthen the contribution and help quantify the computational overhead of the frozen backbone.","section":"Efficiency motivation"},{"comment":"The description of 'independent adapters at all levels' versus the default shared adapter could be more precise. Specify what parameter sharing the default uses and why shared adaptation improves feature consistency across the four decoder stages.","section":"Ablation B3"}],"recommendation":"major_revision","confidential_remarks":"The main risk to acceptance is the omitted VG3S/VG3T comparison. If either method already achieves ≥34.07 SC IoU or ≥21.75 SSC mIoU under the same protocol, the abstract's SOTA claim is false. I recommend asking the authors to supply that comparison before acceptance. The ablation methodology is otherwise sound, and the decoder-controlled comparison with GaussianFormer-2 is a strength."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: VGOcc is a competently built system that combines frozen visual-geometric foundation features with sparse Gaussian occupancy, and the controlled comparison against GaussianFormer-2 is genuinely useful. But the 'state-of-the-art' claim is only as strong as the baseline list, and that list omits the two methods the paper itself names as closest: VG3T and VG3S. The headline claim is therefore premature, not necessarily wrong.\n\nWhat's new: rather than feeding image features straight into a Gaussian decoder, VGOcc uses frozen VGGT/DINOv2 features twice — to seed Gaussian centers from ray-depth hypotheses (with voxel-thinned sampling to keep spatial balance) and to condition refinement features on calibrated pose/ray identity before cross-view aggregation. Keeping the decoder identical to GaussianFormer-2 is the right kind of control; it isolates the contribution of the new modules. The ablation study is thorough, with every module removed or swapped and consistent drops. The qualitative results over different weather conditions also help.\n\nSoft spots:\n\n- Missing VG3S/VG3T from Table 1 is the real problem. VG3S does essentially what the paper claims to be its core idea — adapting frozen visual-geometric features to Gaussian occupancy. The paper's related work explicitly says so, then doesn't compare. If their numbers on nuScenes/SurroundOcc are higher, the 'SOTA' sentence is false. The revision has to include those rows or explain a protocol mismatch.\n\n- Single run per configuration, no error bars. The mIoU gap over GaussianFormer-2 is 1.73; that is within the seed-to-seed variance this benchmark can produce. One run is too thin.\n\n- The depth posterior is load-bearing for center placement. Fallback queries reduce the risk the reader worried about, but the paper never shows what happens when the posterior is wrong in occluded or distant regions. A short failure-case analysis would strengthen it.\n\n- Efficiency is claimed but never measured. A frozen VGGT backbone is not cheap; no FLOPs, latency, or param count appears. For driving occupancy, that matters.\n\n- Code is only 'will be available'. That is acceptable for a preprint, but it should ship with the revision.\n\nThe mathematics and citation pattern are otherwise clean. The only self-citation is to VGD in related work, and it is not load-bearing. No circularity.\n\nWho this is for: researchers working on sparse Gaussian occupancy or on lifting foundation features to 3D driving perception. It deserves a real round of peer review — I would not desk reject — but the revision must add the nearest-baseline comparisons and enough reproducibility details before the SOTA claim is credible.","headline":"VGOcc is a competent combination of foundation features and sparse Gaussian occupancy with clean ablations, but its SOTA claim is unproven because the closest baselines are missing from the comparison.","tokens_in":12974,"tokens_out":4538,"would_cite":true,"duration_ms":45790,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Depth-grounded Gaussians improve vision-only 3D occupancy prediction, reaching 34.07 SC IoU on nuScenes.","keywords":["3D semantic occupancy prediction","vision-centric perception","sparse Gaussians","occupancy prediction","autonomous driving","visual-geometric foundation models","ray-depth","cross-view feature learning"],"falsifier":"Run a controlled experiment where the ray-depth prior is corrupted — for example, add a fixed offset to all predicted depths or replace the depth head with a constant-depth predictor — while keeping everything else fixed, and measure SC IoU. A small drop would show that the geometric birth is not the source of the gain; a large drop would confirm it. Alternatively, compare the recall of predicted non-empty depth bins against LiDAR ground truth in the farthest 20 meters of the evaluation region.","tokens_in":12129,"feed_emoji":"🚗","tokens_out":4686,"duration_ms":46370,"temperature":0.7,"pith_summary":"This paper tries to establish that sparse Gaussian primitives for camera-only 3D semantic occupancy prediction become substantially more accurate when their initialization and refinement are grounded in complementary geometric and visual-semantic cues from pretrained foundation models, rather than relying on image features alone. It introduces Visual-Geometric Gaussians, which place Gaussian centers from predicted per-ray depth hypotheses with balanced spatial sampling, initialize attributes from same-ray visual features, and refine using pose-conditioned, cross-view awareness. On the nuScenes benchmark, VGOcc reports 34.07 SC IoU and 21.75 SSC mIoU, surpassing the previous best sparse-Gaussian method by 3.51 and 1.73 points, and outperforming dense voxel and BEV methods as well. The gain matters because occupancy prediction for autonomous driving needs efficient yet geometrically faithful 3D reconstruction from ordinary cameras.","feed_headline":"Depth-grounded Gaussians achieve 34 SC IoU for 3D occupancy","feed_subtitle":"Camera-only occupancy prediction tops prior sparse-Gaussian baseline by 3.5 IoU points using ray-depth seeding and pose-aware refinement.","key_machinery":"Visual-Geometric Gaussians — the central object. They form a set of sparse Gaussian primitives whose placement comes from geometry (ray-depth hypotheses plus voxel-thinned sampling), whose non-spatial attributes come from same-ray visual features, and whose refinement features are pose-conditioned fusion of semantic, geometric, and global tokens with cross-view ray compatibility. This machinery carries the argument because the ablations show the largest drops when the geometric birth is replaced by previous query initialization (1.65/1.51 IoU/mIoU) and when pose conditioning is removed (1.32/0.61).","core_discovery":"VGOcc argues that sparse Gaussian primitives—small ellipsoids with semantic attributes—are a good representation for 3D occupancy only when their birth and refinement are steered by explicit geometry. The paper therefore builds Visual-Geometric Gaussians: ray-depth hypotheses from a frozen geometric backbone are lifted to 3D, selected by fast voxel-thinned sampling to avoid clustering on visible surfaces, and paired with same-ray visual-semantic features that initialize each primitive's opacity, class, and latent code. During refinement, a pose-aware feature module conditions multi-level tokens on each patch's calibrated ray and aggregates neighboring-view features at projected 3D points, in","pith_inferences":["Since all Gaussian centers are seeded from per-frame ray-depth hypotheses, a natural extension is to aggregate ray hypotheses across multiple frames before sampling; temporal depth priors could further stabilize occupancy in occluded regions, a direction the paper lists as future work.","The frozen foundation features set an upper bound: retraining or adapting the geometric backbone on driving-specific data rather than generic imagery is a testable way to push the method further.","The ray-compatibility weighting used in cross-view aggregation could be reused in other camera-only 3D tasks, such as monocular depth estimation or 3D detection, as a geometric attention bias; the paper does not claim this.","A stress test with systematically biased camera calibration would clarify robustness boundaries, given the large sensitivity to patch-local pose conditioning observed in the ablation study."],"forward_implications":["Sparse Gaussian occupancy models can match or beat dense voxel and BEV methods without adding Gaussian budget, by investing compute in better initialization and refinement cues.","Frozen pretrained geometric and visual features transfer to driving scenes and provide a reusable source of 3D cues for occupancy prediction.","Ray-depth-derived centers with spatially balanced sampling improve coverage of occluded and distant regions compared with confidence ranking or farthest-point sampling.","Pose-conditioned cross-view aggregation helps resolve per-ray depth ambiguity, as reflected in consistent gains across most semantic classes.","Because the decoder is unchanged, the improvement is attributable to how the Gaussians are born and how their refinement features are built, not to a new rendering operator."],"fun_headline_variants":["Ray-depth seeding plus pose-aware tuning sharpens 3D occupancy","Visual-geometric Gaussians win on nuScenes occupancy","Geometry-aware Gaussians beat sparse baseline by 3.5 IoU","Depth hypotheses seed Gaussians for sharper occupancy","Pose-aware Gaussian refinement sets new state of the art"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the lightweight depth head on frozen geometric features produces dependable per-ray depth posteriors for seeding Gaussian centers; if those posteriors are biased in occluded, distant, or low-light regions, the occupancy output degrades no matter how well the later refinement works.","fun_headline_variants_meta":{"raw":{"variants":["Ray-depth seeding plus pose-aware tuning sharpens 3D occupancy","Visual-geometric Gaussians win on nuScenes occupancy","Geometry-aware Gaussians beat sparse baseline by 3.5 IoU","Depth hypotheses seed Gaussians for sharper occupancy","Pose-aware Gaussian refinement sets new state of the art"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001296,"raw_usage":{"total_tokens":5149,"prompt_tokens":791,"completion_tokens":4358,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":535,"completion_tokens_details":{"reasoning_tokens":4275}},"tokens_in":535,"tokens_out":4358,"duration_ms":27778,"temperature":1.0,"reasoning_tokens":4275,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T16:08:57.893847+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a controlled experiment where the ray-depth prior is corrupted — for example, add a fixed offset to all predicted depths or replace the depth head with a constant-depth predictor — while keeping everything else fixed, and measure SC IoU. A small drop would show that the geometric birth is not the source of the gain; a large drop would confirm it. Alternatively, compare the recall of predicted non-empty depth bins against LiDAR ground truth in the farthest 20 meters of the evaluation region.","supporting_citations":[],"review_version":1}