Incoherent Deformation, Not Capacity: Diagnosing and Mitigating Overfitting in Dynamic Gaussian Splatting
Pith reviewed 2026-05-10 08:01 UTC · model grok-4.3
The pith
Overfitting in dynamic 3D Gaussian Splatting stems from incoherent per-Gaussian deformations rather than excess capacity.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The overfitting in dynamic 3DGS is driven by incoherent deformation, not parameter count. Splitting explains over 80 percent of the gap and produces a near-perfect log-linear relation between Gaussian count and gap size, yet a local-smoothness penalty on the per-Gaussian deformation field reduces the gap by 40.8 percent while growing the cloud by 85 percent; measured strain drops by 99.72 percent on average, and the approach generalizes across architectures and real data.
What carries the argument
Elastic Energy Regularization (EER), a penalty that measures and reduces local strain in the per-Gaussian deformation field to enforce coherence.
If this is right
- Disabling splitting collapses both the gap and the Gaussian count, confirming the capacity correlation.
- EER plus GAD and PTDrop together close 57 percent of the gap.
- The coherence benefit appears in a second deformation architecture and on real monocular video.
- Per-Gaussian strain can be read directly from checkpoints to diagnose the source of overfitting.
Where Pith is reading between the lines
- Coherence penalties may be useful in other per-point or per-primitive dynamic reconstruction methods.
- The finding suggests that deformation incoherence, not raw parameter count, could be the dominant overfitting driver in broader classes of dynamic scene models.
- Enforcing coherence while allowing growth might let future dynamic splatting systems scale to more complex motion without retraining on denser views.
Load-bearing premise
That the measured PSNR gaps on D-NeRF and HyperNeRF directly reflect overfitting from incoherent deformation rather than differences in view sampling or lighting.
What would settle it
Train a high-capacity dynamic 3DGS model whose deformation field is forced to remain locally coherent and check whether the train-test PSNR gap stays small despite the large primitive count.
Figures
read the original abstract
Dynamic 3D Gaussian Splatting methods achieve strong training-view PSNR on monocular video but generalize poorly: on the D-NeRF benchmark we measure an average train-test PSNR gap of 6.18 dB, rising to 11 dB on individual scenes. We report two findings that together account for most of that gap. Finding 1 (the role of splitting). A systematic ablation of the Adaptive Density Control pipeline (split, clone, prune, frequency, threshold, schedule) shows that splitting is responsible for over 80% of the gap: disabling split collapses the cloud from 44K to 3K Gaussians and the gap from 6.18 dB to 1.15 dB. Across all threshold-varying ablations, gap is log-linear in count (r = 0.995, bootstrap 95% CI [0.99, 1.00]), which suggests a capacity-based explanation. Finding 2 (the role of deformation coherence). We show that the capacity explanation is incomplete. A local-smoothness penalty on the per-Gaussian deformation field -- Elastic Energy Regularization (EER) -- reduces the gap by 40.8% while growing the cloud by 85%. Measuring per-Gaussian strain directly on trained checkpoints, EER reduces mean strain by 99.72% (median 99.80%) across all 8 scenes; on 8/8 scenes the median Gaussian under EER is less strained than the 1st-percentile (best-behaved) Gaussian under baseline. Alongside EER, we evaluate two further regularizers: GAD, a loss-rate-aware densification threshold, and PTDrop, a jitter-weighted Gaussian dropout. GAD+EER reduces the gap by 48%; adding PTDrop and a soft growth cap reaches 57%. We confirm that coherence generalizes to (a) a different deformation architecture (Deformable-3DGS, +40.6% gap reduction at re-tuned lambda), and (b) real monocular video (4 HyperNeRF scenes, reducing the mean PSNR gap by 14.9% at the same lambda as D-NeRF, with near-zero quality cost). The overfitting in dynamic 3DGS is driven by incoherent deformation, not parameter count.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript claims that overfitting in dynamic 3D Gaussian Splatting on monocular video (average 6.18 dB train-test PSNR gap on D-NeRF, up to 11 dB per scene) is driven by incoherent per-Gaussian deformations rather than model capacity. Systematic ablations of Adaptive Density Control show splitting accounts for >80% of the gap, with log-linear correlation (r=0.995) between Gaussian count and gap across thresholds. Elastic Energy Regularization (EER) reduces the gap 40.8% while increasing count 85% and mean strain 99.72%; combined with GAD (loss-rate-aware densification) and PTDrop (jitter-weighted dropout) the reduction reaches 57%. Results generalize to Deformable-3DGS (+40.6%) and 4 HyperNeRF scenes (14.9% gap reduction at fixed lambda).
Significance. If the empirical patterns hold, the work offers a useful diagnosis and practical mitigation for generalization failures in dynamic 3DGS by shifting attention from parameter count to deformation coherence. Strengths include consistent ablation results across 8 scenes and two architectures, direct per-Gaussian strain quantification, bootstrap confidence intervals on the correlation, and zero-cost generalization to real video. The regularizers (EER, GAD, PTDrop) provide immediately usable tools that increase capacity while improving test performance, which could influence future dynamic reconstruction pipelines.
major comments (1)
- [Finding 2] Finding 2: the claim that EER enforces coherence and thereby reduces the gap while growing the cloud by 85% is load-bearing. The manuscript must supply the exact mathematical definition of the local-smoothness penalty (including neighborhood selection and strain tensor computation) and state whether the regularizer is active only at training time or also at inference; without this the 99.72% strain reduction cannot be reproduced or verified independently.
minor comments (2)
- [Finding 1] The abstract and Finding 1 report the 80% attribution to splitting from a single ablation (disabling split). A supplementary table showing the incremental effect of disabling each ADC component individually (split, clone, prune, frequency, threshold) would make the dominance claim easier to assess.
- Implementation details for the hyper-parameter search of lambda (EER weight) and the GAD loss-rate threshold are referenced but not fully specified (grid ranges, whether tuning was global or per-scene, number of trials). Adding these in the supplementary material would address reproducibility concerns.
Simulated Author's Rebuttal
We thank the referee for the positive evaluation and the recommendation of minor revision. We address the single major comment below and will update the manuscript accordingly to improve clarity and reproducibility.
read point-by-point responses
-
Referee: [Finding 2] Finding 2: the claim that EER enforces coherence and thereby reduces the gap while growing the cloud by 85% is load-bearing. The manuscript must supply the exact mathematical definition of the local-smoothness penalty (including neighborhood selection and strain tensor computation) and state whether the regularizer is active only at training time or also at inference; without this the 99.72% strain reduction cannot be reproduced or verified independently.
Authors: We agree that the precise formulation of Elastic Energy Regularization (EER) must be stated explicitly for reproducibility. The current manuscript describes EER at a high level but does not provide the full mathematical definition, neighborhood selection details, or strain tensor computation in a self-contained way. In the revised version we will add a dedicated paragraph (or short subsection) giving the exact expression for the local-smoothness penalty, the criterion used to select neighboring Gaussians, and the formula for the per-Gaussian strain tensor. We will also state explicitly that the regularizer is active only during training and is disabled at inference. These additions will allow independent verification of the reported 99.72% mean strain reduction and the associated generalization improvements. revision: yes
Circularity Check
No significant circularity
full rationale
The paper presents empirical findings from controlled ablations on D-NeRF and HyperNeRF benchmarks, measuring PSNR gaps, Gaussian counts, and strain values directly from trained models. No mathematical derivation chain, self-definitional equations, or fitted parameters renamed as predictions exist. The central claim (incoherent deformation drives overfitting) is supported by observed correlations (e.g., r=0.995) and intervention effects (EER reducing gap while increasing count), all externally verifiable on public benchmarks without looping back to inputs by construction. Hyperparameter choices for regularizers are standard tuning and do not define the result.
Axiom & Free-Parameter Ledger
free parameters (2)
- EER regularization weight (lambda)
- GAD loss-rate threshold
axioms (2)
- domain assumption PSNR difference between train and test views measures overfitting in novel-view synthesis
- domain assumption Gaussian deformation fields can be regularized independently per point without breaking scene consistency
invented entities (3)
-
Elastic Energy Regularization (EER)
no independent evidence
-
GAD (loss-rate-aware densification)
no independent evidence
-
PTDrop (jitter-weighted Gaussian dropout)
no independent evidence
Reference graph
Works this paper leans on
-
[1]
Per-Gaussian embedding-based deformation for deformable 3D Gaussian splatting
Jeongmin Bae, Seoha Kim, Youngsik Yun, Hahyun Lee, Gun Bang, and Youngjung Uh. Per-Gaussian embedding-based deformation for deformable 3D Gaussian splatting. InEuropean Conference on Com- puter Vision (ECCV), 2024
work page 2024
-
[2]
Efficient density control for 3d gaussian splat- ting.arXiv preprint arXiv:2411.10133, 2024
Xiaobin Deng, Changyu Diao, Min Li, Ruohan Yu, and Duanqing Xu. Efficient density control for 3D Gaussian splatting.arXiv preprint arXiv:2411.10133, 2024
-
[3]
Mini-Splatting: Representing scenes with a constrained number of 11 Gaussians
Guangchi Fang and Bing Wang. Mini-Splatting: Representing scenes with a constrained number of 11 Gaussians. InEuropean Conference on Computer Vision (ECCV), 2024
work page 2024
-
[4]
Improving adaptive density control for 3D Gaussian splatting
Glenn Grubert, Florian Barthel, Anna Hilsmann, and Peter Eisert. Improving adaptive density control for 3D Gaussian splatting. InInternational Joint Conference on Computer Vision, Imaging and Com- puter Graphics Theory and Applications (VISAPP), pages 610–621. SCITEPRESS, 2025
work page 2025
-
[5]
PUP 3D-GS: Principled uncertainty pruning for 3D Gaussian splatting
Alex Hanson, Allen Tu, Vasu Singla, Mayuka Jayawardhana, Matthias Zwicker, and Tom Gold- stein. PUP 3D-GS: Principled uncertainty pruning for 3D Gaussian splatting. InIEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), 2025
work page 2025
-
[6]
SC-GS: Sparse-controlled Gaussian splatting for editable dy- namic scenes
Yi-Hua Huang, Yang-Tian Sun, Zilong Yang, Xi- aoyang Lyu, Yan-Pei Cao, and Xiaojuan Qi. SC-GS: Sparse-controlled Gaussian splatting for editable dy- namic scenes. InIEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), 2024
work page 2024
-
[7]
3D Gaussian splatting for real-time radiance field rendering
Bernhard Kerbl, Georgios Kopanas, Thomas Leimk¨ uhler, and George Drettakis. 3D Gaussian splatting for real-time radiance field rendering. In ACM Transactions on Graphics (SIGGRAPH), 2023
work page 2023
-
[8]
Yiqing Liang, Mikhail Okunev, Mikaela Angelina Uy, Runfeng Li, Leonidas Guibas, James Tompkin, and Adam W. Harley. Monocular dynamic Gaus- sian splatting: Fast, brittle, and scene complexity rules.Transactions on Machine Learning Research (TMLR), 2025
work page 2025
-
[9]
Taming 3DGS: High-quality radiance fields with limited resources
Saswat Subhajyoti Mallick, Rahul Goel, Bernhard Kerbl, Markus Steinberger, Francisco Vicente Car- rasco, and Fernando De La Torre. Taming 3DGS: High-quality radiance fields with limited resources. InSIGGRAPH Asia Conference Papers, 2024
work page 2024
-
[10]
Drop- Gaussian: Structural regularization for sparse-view Gaussian splatting
Hyunwoo Park, Gun Ryu, and Wonjun Kim. Drop- Gaussian: Structural regularization for sparse-view Gaussian splatting. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025
work page 2025
-
[11]
Keunhong Park, Utkarsh Sinha, Jonathan T. Barron, Sofien Bouaziz, Dan B. Goldman, Steven M. Seitz, and Ricardo Martin-Brualla. Nerfies: Deformable neural radiance fields. InIEEE/CVF International Conference on Computer Vision (ICCV), 2021
work page 2021
-
[12]
Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T. Barron, Sofien Bouaziz, Dan B. Gold- man, Ricardo Martin-Brualla, and Steven M. Seitz. HyperNeRF: A higher-dimensional representation for topologically varying neural radiance fields.ACM Transactions on Graphics, 40(6), 2021
work page 2021
-
[13]
D-NeRF: Neural ra- diance fields for dynamic scenes
Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. D-NeRF: Neural ra- diance fields for dynamic scenes. InIEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR), 2021
work page 2021
-
[14]
Revising densification in Gaussian splatting
Samuel Rota Bul` o, Lorenzo Porzi, and Peter Kontschieder. Revising densification in Gaussian splatting. InEuropean Conference on Computer Vision (ECCV), 2024
work page 2024
-
[15]
Steepest descent density control for compact 3D Gaussian splatting
Peihao Wang, Yuehao Wang, Dilin Wang, Sreyas Mohan, Zhiwen Fan, Lemeng Wu, Ruisi Cai, Yu- Ying Yeh, Zhangyang Wang, Qiang Liu, and Rakesh Ranjan. Steepest descent density control for compact 3D Gaussian splatting. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025
work page 2025
-
[16]
4D Gaussian splatting for real-time dynamic scene rendering
Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4D Gaussian splatting for real-time dynamic scene rendering. InIEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), 2024
work page 2024
-
[17]
DropoutGS: Dropping out gaussians for better sparse-view rendering
Yexing Xu, Longguang Wang, Minglin Chen, Sheng Ao, Li Li, and Yulan Guo. DropoutGS: Dropping out gaussians for better sparse-view rendering. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025
work page 2025
-
[18]
Deformable 3D Gaussians for high-fidelity monocular dynamic scene reconstruction
Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3D Gaussians for high-fidelity monocular dynamic scene reconstruction. InIEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), 2024
work page 2024
-
[19]
AbsGS: Recovering fine details in 3D Gaussian splatting
Zongxin Ye, Wenyu Li, Sidun Liu, Peng Qiao, and Yong Dou. AbsGS: Recovering fine details in 3D Gaussian splatting. InProceedings of the 32nd ACM International Conference on Multimedia (MM), 2024
work page 2024
-
[20]
Pixel-GS: Density control with pixel-aware gradient for 3D Gaussian splatting
Zheng Zhang, Wenbo Hu, Yixing Lao, Tong He, and Hengshuang Zhao. Pixel-GS: Density control with pixel-aware gradient for 3D Gaussian splatting. In European Conference on Computer Vision (ECCV), 2024
work page 2024
-
[21]
Gradient- direction-aware density control for 3D Gaussian splat- ting
Zheng Zhou, Yu-Jie Xiong, Jia-Chen Zhang, Chun- Ming Xia, Xihe Qiu, and Hongjian Zhan. Gradient- direction-aware density control for 3D Gaussian splat- ting. InInternational Conference on Learning Rep- resentations (ICLR), 2026. 12
work page 2026
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.