{"id":"26957fb7-7fb7-4ca3-a1a2-b29ee38df4f3","arxiv_id":"2608.11077","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"LGS learns where to prune and add Gaussian primitives in feed-forward driving reconstruction using gradient-response interventions, and aggregates cross-time features, improving novel-view synthesis on Waymo and PandaSet.","lead":"This paper presents LGS, a feed-forward model that restructures LiDAR-initialized Gaussian primitives during driving-scene reconstruction by learning prune and add decisions from controlled gradient-response interventions. It reports consistent quality gains over prior art on Waymo and PandaSet, driven by a densification policy and cross-time feature aggregation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The intervention target in Eqs. 3–5 is confounded: Eq. 3 normalizes gradient magnitudes by a maximum taken within each branch separately, so the prune/add deltas reflect changes in the normalization scale across branches rather than purely local gradient responses.","rationale":"I read the paper as an empirical systems claim: intervention-induced gradient responses provide a transferable training signal for feed-forward Gaussian densification. The load-bearing link is the mapping from Delta to y_i in Section 3.2. The reader identified the proxy's transferability as the weakest assumption; my analysis sharpens this to a specific internal confound. Eq. 3's denominator is computed over each branch's primitive set, so the three branches have different normalization scales. This is not a matter of consensus or tuning; it is a property of the equations as written. If the normalization is the source of the signal, the policy could be learning to predict branch-level rescaling artifacts, and the qualitative score maps in Figures 6–7 would not establish the mechanism. I do not claim the paper is fraudulent; the empirical gains may survive a corrected normalization, but the current manuscript does not rule out the confound. The proposed test is inexpensive: it reuses the same training pipeline and only changes the normalization reference, then compares ablation numbers. I also credit the paper's three-stage training, the random-action control in Table 4, and the threshold analysis as genuine evidence that density adjustment helps; the concern is specifically about the origin of the supervision signal. Because the central mechanism is unverified under a corrected target, I keep the reader's CONDITIONAL verdict, now with the additional condition that the authors rerun the ablation under shared normalization or report the Delta-vs-loss correlation.","tokens_in":11511,"tokens_out":5982,"duration_ms":51770,"concrete_test":"Recompute the intervention targets with a shared normalization constant: in Eq. 3, replace the branch-specific max_l |(grad_{g_l} L_B)_c| with the max over the union of primitives in B1, B2, and B3 (or with the original branch B1 maximum) for each attribute. Retrain only the Gaussian Densify Policy (stages 2 and 3) using the corrected Deltas and re-run the Table 3 ablation. If the PSNR gain over the no-policy baseline drops substantially or reverses, the reported improvements are an artifact of branch-specific normalization. Also report the rank correlation between the original Delta and the corrected Delta (or between Delta and the actual change in L_render) on held-out clips; low correlation would directly falsify the proxy.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that Delta_prune and Delta_add from Eqs. 4–5 reliably indicate whether pruning or adding a primitive improves reconstruction. Eq. 3 defines r_j^B = (1/D) sum_c |(grad_{g_j} L_B)_c| / (max_l |(grad_{g_l} L_B)_c| + eps), where l indexes all Gaussian primitives in branch B. Because the original, prune, and add branches have different primitive sets, the per-branch global maximum in the denominator differs across B1, B2, and B3. Pruning a large-gradient primitive can lower the denominator and inflate the normalized responses of neighbors; adding a perturbed copy with a large gradient can raise the denominator and suppress neighbors' normalized responses. Consequently, Delta_prune = sum_j (r_j^B2 - r_j^B1) and Delta_add = sum_j (r_j^B3 - r_j^B1) mix the intended local response change with a global rescaling artifact. The sign of Delta, which is mapped to the training target y_i, may therefore encode the branch-level maximum shift rather than whether the intervention benefits reconstruction. The Densify Policy is trained on these targets, and the threshold analysis in Table 4 is built on the resulting scores; if the targets are systematically distorted, the reported 28.04 dB result does not establish the claimed intervention-guided mechanism.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LGS, a feed-forward framework for driving-scene Gaussian reconstruction. The method has two main components: a Gaussian Densify Policy that learns to prune or add Gaussian primitives using supervision derived from controlled interventions, and a Cross-Time Point Query that aggregates features from other timestamps. The interventions are applied to sampled primitives, and the resulting changes in neighbor gradient responses (Eqs. 3-5) are converted into training targets for the policy. At inference, the policy predicts prune/add scores and adjusts the Gaussian set without per-scene optimization. Experiments on Waymo and PandaSet report consistent improvements over prior feed-forward methods (e.g., full-image PSNR 28.04 vs 26.28 for UniSplat on Waymo).","tokens_in":11807,"tokens_out":5360,"duration_ms":51093,"significance":"If the intervention-based supervision is valid, the work is a meaningful step toward enabling feed-forward Gaussian reconstruction to adjust primitive composition without per-scene optimization. The two-dataset evaluation, the component ablations in Table 3, and the threshold analysis in Table 4 are useful empirical contributions. The central idea of using controlled prune/add interventions to generate training targets is novel and potentially transferable. However, the significance is conditional on resolving the normalization confound in Eq. (3) and on validating the target signal against actual reconstruction quality, not just downstream rendering metrics.","major_comments":[{"comment":"The per-branch global-max normalization confounds the intervention deltas in Eqs. (4)-(5). Because the original, prune, and addition branches contain different primitive sets, the denominator max_l |(grad_{g_l} L_B)_c| is computed over different sets in each branch. Pruning a large-gradient primitive can lower the denominator and inflate the normalized responses of its neighbors; adding a perturbed copy with a large gradient can raise the denominator and suppress neighbor responses. Thus Delta_prune and Delta_add mix the intended local response change with a branch-level rescaling artifact. Since the sign of the delta is mapped to the training target y_i, the policy may be learning to predict this scale shift rather than the local benefit of the intervention. Please either use a shared normalization reference across all branches (e.g., the original branch's maxima) or provide an analysis showing that the rescaling does not dominate the sign of the deltas.","section":"§3.2, Eq. (3)"},{"comment":"The thresholds theta_p and theta_a are set to 0.7 based on the same Table 4 that supports the final performance claim, and no validation split is described. The table shows PSNR varying between 26.94 and 28.09 dB across threshold choices, so selecting thresholds on the test set can inflate the reported 28.04 dB result. Please select thresholds on a held-out validation set and report final numbers on the test split only after freezing the thresholds, or provide a sensitivity analysis with confidence intervals.","section":"§4.4, Table 4"},{"comment":"The intervention targets are not directly validated against reconstruction quality. The text states that a negative Delta_i indicates that the operation reduces the local gradient response and a positive value indicates the opposite, but it never checks whether these signs correlate with actual changes in rendered PSNR/SSIM when the intervention is applied. Please add an oracle-style analysis on a held-out validation set: compare the learned prune/add decisions with decisions that maximize real render quality, and report the agreement rate. Without this, the causal interpretation of the improved rendering metrics remains indirect.","section":"§3.2"},{"comment":"The three-stage training creates a potential distribution mismatch for the policy. In Stage 2 the policy is trained on targets generated by the Stage-1 backbone, but in Stage 3 the backbone is fine-tuned while the policy is frozen. At inference, the policy consumes features produced by the fine-tuned backbone, whose distribution differs from the Stage-1 features seen in training. Please add an ablation that compares (i) the current pipeline with (ii) a variant that re-trains the policy after Stage 3, to show that the mismatch does not degrade the policy's decisions.","section":"§3.4"}],"minor_comments":[{"comment":"The notation Delta_i is introduced without a subscript for prune/add; please use Delta_prune_i and Delta_add_i consistently throughout the text and equations.","section":"§3.2"},{"comment":"LGSbase is defined in the Table 2 caption as 'the representation before iterative refinement,' but the term is also used in Table 1 without a definition. Please define it once in Section 4.1 or in the main text.","section":"Table 2"},{"comment":"The score maps in Figures 6 and 7 would be much more interpretable if they included a shared color scale and a quantitative measure of the claimed spatial complementarity between Addition Scores and Prune Scores.","section":"Figures 6 and 7"},{"comment":"The inference time of 1.90 s per frame is reported, but it is unclear whether this includes the three-stage training procedure or only the feed-forward inference; please clarify.","section":"§4.1"},{"comment":"No variance or repeated-seed results are reported in Tables 1-4. Given that some comparisons are close (e.g., Table 3 baseline vs CTPQ-only), please report standard deviations over at least three runs or a significance test.","section":"Various tables"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the manuscript compares against a wide set of prior methods, several of which are arXiv preprints without official implementations; I did not independently verify the baseline numbers. The lack of a validation split for the thresholds and the normalization issue in Eq. (3) are methodological risks that the authors need to address. The core idea is interesting and the experiments are extensive, so a major revision with a thorough response to the normalization concern and an oracle validation of the intervention targets would be appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is a solid feed-forward driving reconstruction paper with one genuinely new component. The Gaussian Densify Policy supervises prune/add decisions by rendering gradients of neighboring primitives before and after a controlled intervention. That specific supervision scheme (Eqs. 3–7) is not in the cited prior work, and the ablations support its effectiveness: GDP adds ~1.3 dB PSNR on top of the backbone on Waymo, and the joint model with Cross-Time Point Query reaches 28.04 dB vs 26.28 for UniSplat. The paper is clearly written and the experiments are run on two standard datasets with proper novel-view evaluation.\n\nThe main weakness is a real confound in the intervention target. Eq. 3 normalizes each attribute gradient by the maximum absolute gradient across *that branch's* primitives. Because the original, prune, and add branches have different primitive sets, the normalization scale differs across branches. Pruning a large-gradient primitive can shrink the denominator and inflate neighbors' normalized responses; adding a high-gradient copy can raise the denominator and suppress them. So Delta_prune and Delta_add mix the intended local response change with a global rescaling artifact. The sign of the target may be driven by max shifts rather than by true benefit to reconstruction. The paper doesn't test for this, and it's load-bearing for the claim that interventions reveal structural benefit. I'd want to see either a variant with a fixed normalization (say, original-branch max) or an analysis showing the per-branch max doesn't flip the signs.\n\nSecondary issues: no code or data release, no variance across runs, thresholds chosen on the target benchmark without a held-out validation split, and the relationship between LGSbase and the Table 3 baseline is under-explained. All are fixable.\n\nIf the normalization confound is addressed, this becomes a strong contribution. Even as is, it's worth referee time. I'd condition acceptance on code, variance reporting, and a direct analysis of the normalization effect. The idea is novel enough that a serious reviewer should engage with it.","headline":"A genuinely new densification-supervision idea, but the intervention target is confounded by per-branch normalization, and the empirical claims need code and variance to fully check.","tokens_in":12365,"tokens_out":3023,"would_cite":true,"duration_ms":28622,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that feed-forward Gaussian reconstruction of driving scenes can learn when to prune and add Gaussian primitives from intervention-induced changes in local gradient responses, and that the resulting policy plus cross-time…","keywords":["3D Gaussian Splatting","feed-forward reconstruction","driving scene reconstruction","densification","pruning","gradient response","cross-time feature aggregation","novel view synthesis"],"falsifier":"Take a trained LGS model and apply its prune/add policy to an out-of-distribution driving clip, such as one with substantially sparser LiDAR or a different camera rig, while holding the Gaussian count fixed by matching the action rate with random decisions. If the learned policy does not outperform the random baseline under these conditions, the intervention-response proxy has failed to transfer and the central claim that local gradient changes reveal beneficial structural adjustments is called into question.","tokens_in":11318,"feed_emoji":"🚗","tokens_out":8524,"duration_ms":61964,"temperature":0.7,"pith_summary":"Feed-forward Gaussian reconstruction of driving scenes typically keeps the initial LiDAR point-to-primitive mapping fixed, so a model can update attributes but never decide whether a primitive should be removed or duplicated. This paper introduces Learning Gaussian Structure (LGS), whose Gaussian Densify Policy predicts per-primitive prune and addition scores in a single forward pass. The training signal comes from controlled interventions: temporarily pruning or duplicating a primitive and measuring how the rendering gradients of its spatial neighbors change, arguing that a reduction in local gradient response reveals a beneficial structural adjustment. A second component, Cross-Time Point Query, retrieves features from Gaussian primitives at other timestamps so attribute prediction uses explicit cross-time evidence. If correct, feed-forward reconstruction gains the density-control ability previously limited to per-scene optimization.","feed_headline":"Prune-and-add policy lifts driving-scene reconstruction quality","feed_subtitle":"Learned density control matches per-scene optimization gains without test-time fitting.","key_machinery":"The load-bearing machinery is the Gaussian Densify Policy supervised by intervention-induced gradient responses. Concretely, after normalizing each attribute gradient by its maximum across primitives, the scalar response $r_j^B = \\frac{1}{D}\\sum_c \\left| \\frac{(\\nabla_{g_j} \\mathcal{L}^B)_c}{\\max_l |(\\nabla_{g_l} \\mathcal{L}^B)_c| + \\epsilon} \\right|$ measures how strongly a nearby primitive $j$ responds to the rendering loss under branch $B$. The intervention targets $\\Delta^{\\text{prune}}_i$ and $\\Delta^{\\text{add}}_i$ sum the response changes over the $K$ nearest neighbors, and a signed-log, z-score, sigmoid transform turns them into per-primitive supervision targets $y^a_i \\in [0,1]$. A confidence-weighted regression loss with weights $w_i = (2|y_i - 0.5|)^\\gamma$ trains a Point Transformer V3 encoder to predict the Densify Map; during inference the intervention branches are discarded and thresholds $\\theta_p = \\theta_a = 0.7$ are applied to prune and add. The second component, Cross-Time Point Query, mean-pools features from the $K$ nearest primitives at other timestamps and fuses them with the decoder feature via a residual projection, feeding both the attribute head and the densify policy.","core_discovery":"The paper's central claim is that structural decisions in feed-forward Gaussian reconstruction can be learned without per-scene optimization by treating prune and add operations as interventions and reading their effects on local rendering gradients. For a sampled primitive, the authors compute a scalar neighbor-gradient response $r_j^B$ for each branch (original, prune, add), then aggregate the differences $\\Delta^{\\text{prune}}_i$ and $\\Delta^{\\text{add}}_i$ over the primitive's $K$ spatial neighbors. These differences become training targets for the Densify Policy: a lower local gradient magnitude after an intervention means the operation helps reconstruction. At inference, the policy outputs a Densify Map of Prune and Addition Scores, and the Gaussian set is updated by pruning primitives above a threshold and adding perturbed copies of primitives whose addition score is high. On Waymo and PandaSet the full model reports consistent gains over feed-forward baselines, including full-image PSNR of 28.04 dB versus 26.28 dB for UniSplat on Waymo.","pith_inferences":["The intervention-response signal could serve as a general critic for other feed-forward 3D representations: any representation in which adding or removing an element changes neighbor gradients could be supervised the same way, including pixel-aligned Gaussians or voxel grids.","The paper's ablations show that random additions already improve PSNR over the fixed initialization, suggesting that part of the gain is restoring under-represented capacity; a natural test is whether the addition score correlates with local reconstruction error or feature uncertainty on out-of-distribution scenes.","A stronger transferability check would apply the trained policy to a dataset with substantially different LiDAR density or camera configuration without fine-tuning; the paper's premise predicts the prune/add decisions should remain sensible because they are grounded in gradient responses rather than dataset-specific statistics.","The signed-log transform and fixed thresholds introduce hyperparameters; an implicit extension, which the paper itself flags as a limitation, is to make the density adjustment scene-dependent or fully learnable."],"forward_implications":["Feed-forward Gaussian reconstruction gains a densification mechanism previously available only to per-scene optimization, without requiring test-time optimization or intervention rendering.","The Densify Map adapts primitive count and placement to scene content: on Waymo the full model uses 584.2K Gaussians versus 519.8K for the baseline, with additions concentrated on vehicle surfaces and lane markings.","Because the policy is applied in a single forward pass, it adds modest inference cost (about 0.05 s per frame for the policy) while improving rendering quality.","Cross-Time Point Query adds explicit cross-time evidence for attribute prediction and is reported to be complementary to density adjustment, improving PSNR further when combined."],"supporting_citations":[{"why":"Defines the gradient-based densification and pruning procedure that motivates the need for density control in Gaussian splatting; LGS replaces its per-scene optimization with a learned policy.","marker":"Kerbl et al. 2023"},{"why":"Supplies the feed-forward Gaussian reconstruction formulation that LGS builds on as its base architecture.","marker":"Miao et al. 2025"},{"why":"Provides the LiDAR-anchored Gaussian formulation and the PandaSet scene split and evaluation protocol used in the experiments.","marker":"Wang et al. 2026a"},{"why":"Point Transformer V3 serves as the shared spatial encoder of the Gaussian Densify Policy.","marker":"Wu et al. 2023a"},{"why":"Defines the Waymo scene split and novel-view synthesis protocol the paper follows.","marker":"Lu et al. 2024"},{"why":"Waymo Open Dataset is the primary driving-scene benchmark for the main experiments.","marker":"Sun et al. 2020"},{"why":"PandaSet provides the second benchmark and protocol for evaluation.","marker":"Xiao et al. 2021"},{"why":"UniSplat is the strongest feed-forward comparison baseline on Waymo that LGS reports improving over.","marker":"Shi et al. 2025"},{"why":"STORM is the feed-forward baseline on PandaSet that the method is compared against.","marker":"Yang et al. 2025"}],"fun_headline_variants":["Intervention gradients guide Gaussian pruning and addition","Learned prune-add policy boosts driving reconstruction","No optimization at test: intervention-based Gaussian updates","Intervention signals decide which Gaussians to prune and add"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the scalar neighbor-gradient response $r_j^B$, defined as the average normalized gradient magnitude over the primitive's $K$ spatial neighbors, is a valid and transferable proxy for whether pruning or adding that primitive helps reconstruction, so the sign of $\\Delta^{\\text{prune}}_i$ and $\\Delta^{\\text{add}}_i$ correctly supervises the policy.","fun_headline_variants_meta":{"raw":{"variants":["Intervention gradients guide Gaussian pruning and addition","Learned prune-add policy boosts driving reconstruction","No optimization at test: intervention-based Gaussian updates","Intervention signals decide which Gaussians to prune and add"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001412,"raw_usage":{"total_tokens":5710,"prompt_tokens":956,"completion_tokens":4754,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":572,"completion_tokens_details":{"reasoning_tokens":4695}},"tokens_in":572,"tokens_out":4754,"duration_ms":31182,"temperature":1.0,"reasoning_tokens":4695,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:47:43.969007+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a trained LGS model and apply its prune/add policy to an out-of-distribution driving clip, such as one with substantially sparser LiDAR or a different camera rig, while holding the Gaussian count fixed by matching the action rate with random decisions. If the learned policy does not outperform the random baseline under these conditions, the intervention-response proxy has failed to transfer and the central claim that local gradient changes reveal beneficial structural adjustments is called into question.","supporting_citations":[],"review_version":1}