REVIEW 2 major objections 5 minor 46 references
Drop-In Perceptual Optimization for 3D Gaussian Splatting
T0 review · 2 major / 5 minor · reviewed 2026-07-13 · grok-4.5
Pith's one-line read A drop-in regularized Wasserstein Distortion loss makes 3D Gaussian Splatting reconstructions preferred by humans more than twice as often as the standard loss, without needing more Gaussians.
desk verdict Solid drop-in perceptual loss for 3DGS with a real human study and clean transfer; free parameters are ordinary, not fatal. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
WD-R: Wasserstein Distortion (local mean and standard-deviation matching in VGG feature space with fixed pooling size σ=4) lightly regularized by a scaled copy of the original L1+SSIM term, used as a pure drop-in replacement for the 2-D distortion loss that drives both parameter updates and adaptive densification.
What would settle it
A head-to-head human preference study on held-out large-scale outdoor scenes in which WD-R, under identical splat budgets and without further hyper-parameter retuning, is no longer preferred over the original L1+SSIM loss or over Perceptual-GS.
Extended reading notes
Core claim
Replacing the usual L1+SSIM training objective of 3D Gaussian Splatting with a regularized Wasserstein Distortion loss (WD-R) produces novel-view reconstructions that human raters prefer more than 2.3 times as often as the original loss and 1.5 times as often as the prior best perceptual method, while matching or reducing the number of Gaussians and improving LPIPS, DISTS and FID on multiple datasets.
Load-bearing premise
A single fixed pooling size, a hand-tuned global loss scale per dataset, a fixed regularization weight, and a short pixel-loss warm-up are enough to keep the Wasserstein term dominant and to suppress web-like artifacts on the scenes that were tested.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper argues that 3D Gaussian Splatting can be made substantially more perceptually faithful by replacing the usual L1+SSIM training objective with a drop-in 2D distortion loss, without changing densification, pruning, or architecture. After comparing the original loss, a composite of L1/L2/MS-SSIM/LPIPS, Wasserstein Distortion (WD) in VGG feature space, and a regularized variant WD-R (WD plus a modest original-loss term), the authors identify WD-R as the strongest performer. Under matched or lower splat counts (or model sizes), WD-R improves LPIPS, DISTS, FID and CMMD on 21 scenes from four datasets, is preferred by human raters more than 2.3 imes over the original loss and 1.5 imes over Perceptual-GS (39,320 pairwise ratings, Bayesian Elo), transfers to Mip-Splatting and Scaffold-GS (1.8× and 3.6× preference), and yields roughly 50% bitrate savings at matched perceptual quality under Comp-GS rate–distortion optimization.
Significance. If the results hold, the work cleanly separates perceptual modeling from 3DGS algorithmic design and shows that a single, architecture-agnostic loss change can recover fine texture at equal or lower capacity. The large-scale, pre-screened human preference study is a first for 3DGS and substantially strengthens the claim beyond automatic metrics. Successful transfer to anti-aliased, structured, and compressed pipelines, together with public code and project page, makes the contribution immediately usable and falsifiable. The free parameters (per-dataset γ, fixed β and σ, warm-up) are ordinary for adaptive densification and are documented; they do not reduce the central claim to a tautology.
major comments (2)
- §3.1–3.2, Eq. (1) and Appendix A.2 / Table 4: the global scale γ is tuned per dataset so that adaptive densification produces comparable #G (or model size). While this is the right control and is made explicit, the manuscript should state more clearly how sensitive the human-preference and metric rankings are to modest mis-calibration of γ (e.g., a small grid around the chosen values on one indoor and one outdoor scene). Without that, readers cannot judge how much of the 2.3×/1.5× preference is locked to the exact operating point.
- §4.1 and Fig. 6: pure WD can produce web-like artifacts under tight splat budgets; WD-R suppresses them via a fixed β=1/0.09. Fig. 12 shows that the WD gradient still dominates on average (~1.6×), but the paper would be stronger if it quantified residual artifact frequency (or a simple automatic detector) across all 21 scenes rather than a single Bicycle example, so that the residual failure mode of the regularizer is bounded.
minor comments (5)
- Table 1 / Table 8: PSNR/SSIM drop under WD/WD-R is expected (perception–distortion trade-off) but should be flagged more explicitly in the main text so readers do not misread the tables as a pure win on every axis.
- Appendix A.1: the ~4.5× training-time overhead of WD is acknowledged; a short note on whether feature caching or lower-resolution VGG maps can reduce it would help practitioners.
- §3.1: the choice of constant σ=4 is justified by ablation (Table 6) and human ratings, but a one-sentence pointer to the adaptive-σ experiments (Appendix C.2) in the main text would avoid the impression that only a single σ was tried.
- Fig. 1 caption and Elo figures: the “2.4× vs. original” style ratios are helpful; stating the exact Elo-to-odds conversion used (Bradley–Terry) once would make the numbers fully self-contained.
- Minor notation: L_orig is defined with coefficients 0.8 L1 + 0.2 SSIM in Appendix A.2; the same coefficients should appear when the original loss is first introduced in §2.2 / §3.1 for consistency.
Circularity Check
No significant circularity: WD-R is an empirical drop-in loss whose gains are measured by independent human preference and external metrics under explicitly matched budgets.
-
fitted input called prediction
[Sec. 2.2 Eq. (1); Appendix A.2 Table 4]
"To ensure a fair comparison under similar representation budgets, we allow γ to vary across datasets while keeping all other hyperparameters fixed, in order to align the resulting Gaussian counts across methods … We tune γ to ensure that the resulting Gaussian counts allow for a fair comparison to the baselines."
γ is chosen after the fact so that #G (or model size) of WD/WD-R matches or undercuts the baselines. The matched-budget claim therefore partly rests on this calibration. The step is mild and fully disclosed; it does not force the human-preference or metric gains themselves, which remain independent measurements.
full rationale
The paper does not claim a first-principles derivation of a unique optimal loss. It searches over three families of distortion objectives (original L1+SSIM, a composite of L1/L2/MS-SSIM/LPIPS, and WD/WD-R), selects WD-R by large-scale human Elo ratings (39 320 pairwise trials) and by LPIPS-VGG, DISTS, FID and CMMD, then shows the same objective improves Mip-Splatting, Scaffold-GS and Comp-GS under comparable resource budgets. Evaluation LPIPS uses VGG while training WD uses VGG features only for local statistics; FID/CMMD use Inception/CLIP, so the reported metrics are not the training objective. The sole mild circularity risk is the per-dataset scalar γ that is tuned so that adaptive densification yields comparable #G (or model size); this is an ordinary experimental control made explicit in Eq. (1) and Table 4, not a fitted parameter that is later re-presented as a prediction. Gradient-ratio diagnostics (Fig. 12) confirm the WD term remains dominant. No self-definitional loop, no uniqueness theorem imported from the authors, and no renaming of a known result as a new derivation. Score 1 reflects only the transparent budget-matching step.
Assumptions & free parameters
free parameters (5)
- global scale γ (per dataset / loss)
- regularization weight β = 1/0.09
- WD pooling size σ = 4
- composite-loss weights ω1–ω4
- warm-up iterations (3k–5k)
assumptions (4)
- domain assumption Adaptive densification and pruning in 3DGS are driven by gradient magnitude of the 2D distortion loss, so changing only the loss (and its global scale) is sufficient to alter both parameters and representation complexity.
- domain assumption Wasserstein Distortion on VGG features with local mean/std (Eq. 4) is a valid proxy for human texture perception.
- domain assumption Matching average Gaussian count (or model size) across methods yields a fair comparison of perceptual quality under a fixed resource budget.
- standard math Standard mathematical properties of RMSE, SSIM, LPIPS, FID, CMMD and Bradley–Terry/Elo aggregation.
invented entities (1)
-
WD-R (Wasserstein Distortion Regularized)
Cite this review
Pith. "Pith review of Drop-In Perceptual Optimization for 3D Gaussian Splatting." pith.science (2026). https://pith.science/paper/IAFZ2R4E
@misc{pith2026260323297,
author = {Pith},
title = {Pith review of: Drop-In Perceptual Optimization for 3D Gaussian Splatting},
year = {2026},
howpublished = {\url{https://pith.science/paper/IAFZ2R4E}},
note = {Machine review of arXiv:2603.23297}
}
abstract
Despite their output being ultimately consumed by human viewers, 3D Gaussian Splatting (3DGS) methods often rely on ad-hoc combinations of pixel-level losses, resulting in blurry renderings. To address this, we systematically explore perceptual optimization strategies for 3DGS by searching over a diverse set of distortion losses. We conduct the first-of-its-kind large-scale human subjective study on 3DGS, involving 39,320 pairwise ratings across several datasets and 3DGS frameworks. A regularized version of Wasserstein Distortion, which we call WD-R, emerges as the clear winner, excelling at recovering fine textures without incurring a higher splat count. WD-R is preferred by raters more than $2.3\times$ over the original 3DGS loss, and $1.5\times$ over the current best method Perceptual-GS. WD-R also consistently achieves state-of-the-art LPIPS, DISTS, and FID scores across various datasets, and generalizes across recent frameworks, such as Mip-Splatting and Scaffold-GS, where replacing the original loss with WD-R consistently enhances perceptual quality within a similar resource budget (number of splats for Mip-Splatting, model size for Scaffold-GS), and leads to reconstructions being preferred by human raters $1.8\times$ and $3.6\times$, respectively. We also find that this carries over to the task of 3DGS scene compression, with $\approx 50\%$ bitrate savings for comparable perceptual metric performance.
Reference graph
Works this paper leans on
-
[1]
In: International Conference on Learning Representations (2017),https://openreview.net/forum?id=rJxdQ3jeg
Ballé, J., Laparra, V., Simoncelli, E.P.: End-to-end optimized image compression. In: International Conference on Learning Representations (2017),https://openreview.net/forum?id=rJxdQ3jeg
2017
-
[2]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Ballé, J., Versari, L., Dupont, E., Kim, H., Bauer, M.: Good, cheap, and fast: Overfitted image compression with wasserstein distortion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 23259–23268 (June 2025)
2025
-
[3]
In: International Conference on Learning Representations (2018),https://openreview.net/forum?id= rkcQFMZRb
Ballé, J., Minnen, D., Singh, S., Hwang, S.J., Johnston, N.: Variational image compression with a scale hyper- prior. In: International Conference on Learning Representations (2018),https://openreview.net/forum?id= rkcQFMZRb
2018
-
[4]
CVPR (2022)
Barron, J.T., Mildenhall, B., Verbin, D., Srinivasan, P.P., Hedman, P.: Mip-nerf 360: Unbounded anti-aliased neural radiance fields. CVPR (2022)
2022
-
[5]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Barthel, F., Beckmann, A., Morgenstern, W., Hilsmann, A., Eisert, P.: Gaussian splatting decoder for 3d-aware generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7963–7972 (2024)
2024
-
[6]
In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025),https://openreview.net/forum?id=JoBCAKbCpC
Barthel, F., Morgenstern, W., Hinzer, P., Hilsmann, A., Eisert, P.: CGS-GAN: 3d consistent Gaussian splatting GANs for high resolution human head synthesis. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025),https://openreview.net/forum?id=JoBCAKbCpC
2025
-
[7]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
Blau, Y., Michaeli, T.: The perception-distortion tradeoff. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
2018
-
[8]
In: International Conference on Learning Representations (2025), https://arxiv.org/abs/2410.02073
Bochkovskii, A., Delaunoy, A., Germain, H., Santos, M., Zhou, Y., Richter, S.R., Koltun, V.: Depth pro: Sharp monocular metric depth in less than a second. In: International Conference on Learning Representations (2025), https://arxiv.org/abs/2410.02073
arXiv 2025
Show all 46 references
-
[9]
Caron,F.,Doucet,A.:Efficientbayesianinferenceforgeneralizedbradley–terrymodels.JournalofComputational and Graphical Statistics21(1), 174–196 (2012).https://doi.org/10.1080/10618600.2012.638220
2012 doi
-
[10]
CLIC Challenge: CLIC 2025 image task (2025),https://clic2025.compression.cc/tasks/#image
2025
-
[11]
IEEE Transactions on Pattern Analysis and Machine Intelligence p
Ding, K., Ma, K., Wang, S., Simoncelli, E.P.: Image quality assessment: Unifying structure and texture similarity. IEEE Transactions on Pattern Analysis and Machine Intelligence p. 1–1 (2020).https://doi.org/10.1109/ tpami.2020.3045810
2020
-
[12]
Nature Neuroscience14(9), 1195–1201 (2011)
Freeman, J., Simoncelli, E.P.: Metamers of the ventral stream. Nature Neuroscience14(9), 1195–1201 (2011)
2011
-
[13]
In: Proceedings of the 29th International ACM Conference on 3D Web Technology
Gong, Y.: EGGS: Edge guided gaussian splatting for radiance fields. In: Proceedings of the 29th International ACM Conference on 3D Web Technology. pp. 1–5 (2024)
2024
-
[14]
Hedman, P., Philip, J., Price, T., Frahm, J.M., Drettakis, G., Brostow, G.: Deep blending for free-viewpoint image-based rendering37(6), 257:1–257:15 (2018)
2018
-
[15]
In: Proceedings of the 31st International Conference on Neural Information Processing Systems
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: GANs trained by a two time-scale up- date rule converge to a local nash equilibrium. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. p. 6629–6640. NIPS’17, Cu...
2017
-
[16]
In: ACM SIGGRAPH 2024 Conference Papers (2024)
Huang, B., Yu, Z., Chen, A., Geiger, A., Gao, S.: 2d gaussian splatting for geometrically accurate radiance fields. In: ACM SIGGRAPH 2024 Conference Papers (2024)
2024
-
[17]
Advances in Neural Information Processing Systems37, 110412–110435 (2024)
Hyung, J., Hong, S., Hwang, S., Lee, J., Choo, J., Kim, J.H.: Effective rank analysis and regularization for enhanced 3d Gaussian splatting. Advances in Neural Information Processing Systems37, 110412–110435 (2024)
2024
-
[18]
Handaya, Tokyo, Hongo Harukicho (1917)
Ishihara, S.: Tests for color-blindness. Handaya, Tokyo, Hongo Harukicho (1917)
1917
-
[19]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1125–1134 (2017)
2017
-
[20]
Jayasumana, S., Ramalingam, S., Veit, A., Glasner, D., Chakrabarti, A., Kumar, S.: Rethinking FID: Towards a better evaluation metric for image generation (2024),https://arxiv.org/abs/2401.09603
2024 arXiv
-
[21]
Image and Vision Com- puting95, 103887 (2020).https://doi.org/10.1016/j.imavis.2020.103887,http://www.sciencedirect
Jia, S., Bruce, N.D.: Eml-net: An expandable multi-layer network for saliency prediction. Image and Vision Com- puting95, 103887 (2020).https://doi.org/10.1016/j.imavis.2020.103887,http://www.sciencedirect. com/science/article/pii/S0262885620300196
2020 doi
-
[22]
ACM Transactions on Graphics42(4) (July 2023),https://repo-sam.inria.fr/fungraph/3d-gaussian- splatting/
Kerbl, B., Kopanas, G., Leimkühler, T., Drettakis, G.: 3d gaussian splatting for real-time radiance field render- ing. ACM Transactions on Graphics42(4) (July 2023),https://repo-sam.inria.fr/fungraph/3d-gaussian- splatting/
2023
-
[23]
In: SIGGRAPH Asia 2024 Conference Papers
Kirschstein, T., Giebenhain, S., Tang, J., Georgopoulos, M., Nießner, M.: Gghead: Fast and generalizable 3d Gaussian heads. In: SIGGRAPH Asia 2024 Conference Papers. pp. 1–11 (2024)
2024
-
[24]
ACM Transactions on Graphics36(4) (2017)
Knapitsch, A., Park, J., Zhou, Q.Y., Koltun, V.: Tanks and temples: Benchmarking large-scale scene reconstruc- tion. ACM Transactions on Graphics36(4) (2017)
2017
-
[25]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and ...
2017
-
[26]
In: European conference on computer vision
Liang, Z., Zhang, Q., Hu, W., Zhu, L., Feng, Y., Jia, K.: Analytic-splatting: Anti-aliased 3d Gaussian splatting via analytic integration. In: European conference on computer vision. pp. 281–297. Springer (2024)
2024
-
[27]
In: Proceedings of the 32nd ACM International Conference on Multimedia
Liu, X., Wu, X., Zhang, P., Wang, S., Li, Z., Kwong, S.: Compgs: Efficient 3d scene representation via compressed Gaussian splatting. In: Proceedings of the 32nd ACM International Conference on Multimedia. pp. 2936–2944 (2024)
2024
-
[28]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Lu, T., Yu, M., Xu, L., Xiangli, Y., Wang, L., Lin, D., Dai, B.: Scaffold-gs: Structured 3d Gaussians for view- adaptive rendering. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 20654–20664 (2024)
2024
-
[29]
Mabyduck: Experiment strategies.https://docs.mabyduck.com/experiments/strategies(2025)
2025
-
[30]
Mabyduck: Mabyduck.https://www.mabyduck.com(2025)
2025
-
[31]
Elo metric,https://docs.mabyduck.com/experiments/metrics/elo
-
[32]
Mentzer, F., Toderici, G., Tschannen, M., Agustsson, E.: High-fidelity generative image compression (2020), https://arxiv.org/abs/2006.09965
2020 arXiv
-
[33]
In: Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., Scarlett, J
Muckley, M.J., El-Nouby, A., Ullrich, K., Jegou, H., Verbeek, J.: Improving statistical fidelity for neural image compression with implicit local likelihood models. In: Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., Scarlett, J. (eds.) Proceedings of the 40th ...
2023
-
[34]
In: 2024 58th Annual Conference on Information Sciences and Systems (CISS)
Qiu, Y., Wagner, A.B., Ballé, J., Theis, L.: Wasserstein distortion: Unifying fidelity and realism. In: 2024 58th Annual Conference on Information Sciences and Systems (CISS). pp. 1–6 (2024).https://doi.org/10.1109/ CISS59072.2024.10480168
2024
-
[35]
In: Precup, D., Teh, Y.W
Rippel, O., Bourdev, L.: Real-time adaptive image compression. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 2922–2930. PMLR, International Convention Centre, Sydney...
2017
-
[36]
In: Human Vision and Electronic Imaging XVI
Rosenholtz, R.: What your visual system sees where you are not looking. In: Human Vision and Electronic Imaging XVI. vol. 7865, pp. 343–356. SPIE (2011)
2011
-
[37]
In: The Thirty-eighth Annual Conference on Neural Information Processing Systems (2024), https://openreview.net/forum?id=W2qGSMl2Uu
Wang, Y., Li, Z., Guo, L., Yang, W., Kot, A., Wen, B.: ContextGS : Compact 3d gaussian splatting with anchor level context model. In: The Thirty-eighth Annual Conference on Neural Information Processing Systems (2024), https://openreview.net/forum?id=W2qGSMl2Uu
2024
-
[38]
In: The Thrity- Seventh Asilomar Conference on Signals, Systems & Computers, 2003
Wang, Z., Simoncelli, E., Bovik, A.: Multiscale structural similarity for image quality assessment. In: The Thrity- Seventh Asilomar Conference on Signals, Systems & Computers, 2003. vol. 2, pp. 1398–1402 Vol.2 (2003).https: //doi.org/10.1109/ACSSC.2003.1292216
-
[39]
IEEE Transactions on Image Processing13(4), 600–612 (2004).https://doi.org/10.1109/TIP
Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing13(4), 600–612 (2004).https://doi.org/10.1109/TIP. 2003.819861
2004 doi
-
[40]
In: The European Conference on Computer Vision (ECCV) (2022)
Xiangli, Y., Xu, L., Pan, X., Zhao, N., Rao, A., Theobalt, C., Dai, B., Lin, D.: Bungeenerf: Progressive neural radiance field for extreme multi-scale scene rendering. In: The European Conference on Computer Vision (ECCV) (2022)
2022
-
[41]
Nature Neuroscience (2016)
Yamins, D.L.K., DiCarlo, J.J.: Using goal-driven deep learning models to understand sensory cortex. Nature Neuroscience (2016)
2016
-
[42]
In: Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition
Yu, Z., Chen, A., Huang, B., Sattler, T., Geiger, A.: Mip-splatting: Alias-free 3d Gaussian splatting. In: Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 19447–19456 (2024)
2024
-
[43]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Zhang, J., Zhan, F., Xu, M., Lu, S., Xing, E.: Fregs: 3d Gaussian splatting with progressive frequency reg- ularization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 21424–21433 (2024)
2024
-
[44]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 586–595 (2018)
2018
-
[45]
In: European Conference on Computer Vision
Zhang, Z., Hu, W., Lao, Y., He, T., Zhao, H.: Pixel-gs: Density control with pixel-aware gradient for 3d Gaussian splatting. In: European Conference on Computer Vision. pp. 326–342. Springer (2024)
2024
-
[46]
In: Forty-second International Conference on Machine Learning (2025),https://openreview.net/forum?id=ij0vj0BC72 14 Özyılkan and Chen et al
Zhou, H., Ni, Z.: Perceptual-GS: Scene-adaptive perceptual densification for gaussian splatting. In: Forty-second International Conference on Machine Learning (2025),https://openreview.net/forum?id=ij0vj0BC72 14 Özyılkan and Chen et al. Appendix A Implementation details We fol...
2025
Reviewed July 13, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.