pith. sign in

arxiv: 2605.26880 · v1 · pith:DRCRZOZWnew · submitted 2026-05-26 · 📡 eess.IV · cs.MM

GScomp-QA: A Subjective Dataset for Quality Assessment of Compressed Gaussian Splatting

Pedro Martin , Ant\'onio Rodrigues , Jo\~ao Ascenso , Maria Paula Queluz This is my paper

Pith reviewed 2026-07-01 16:04 UTC · model grok-4.3

classification 📡 eess.IV cs.MM
keywords Gaussian Splattingcompressionsubjective quality assessmentdatasetnovel view synthesisperceptual qualityrate-distortion analysis3D reconstruction
0
0 comments X

The pith

GScomp-QA isolates compression distortions in Gaussian Splatting by using uncompressed model syntheses as reference for subjective scoring.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces GScomp-QA to evaluate how compression affects the perceptual quality of videos synthesized from Gaussian Splatting models. It generates 331 video stimuli across 13 scenes using nine existing compression methods and obtains reference videos directly from the original uncompressed models. This setup separates compression artifacts from other synthesis errors. A study collects perceptual scores from twenty participants and uses them to perform rate-distortion analysis of the compression approaches. The same scores also demonstrate that eighteen common objective quality metrics fail to account for the particular distortions that appear in compressed Gaussian Splatting output.

Core claim

GScomp-QA is a subjective quality assessment dataset that contains 331 video stimuli drawn from 13 real-world scenes and nine state-of-the-art Gaussian Splatting compression solutions. Reference videos are synthesized from the corresponding uncompressed models so that only compression-induced distortions are measured. Subjective scores gathered from twenty participants support perceptual rate-distortion evaluation of the compression methods and show that eighteen standard objective metrics do not fully capture GS-specific distortions.

What carries the argument

Reference videos synthesized from uncompressed Gaussian Splatting models, which isolate compression-induced distortions from other synthesis artifacts in the subjective evaluation.

If this is right

  • The nine compression solutions can be compared directly on perceptual rate-distortion curves derived from the collected scores.
  • Future Gaussian Splatting compression algorithms can be evaluated against the same set of stimuli and scores.
  • Development of new objective metrics that better match human judgments of GS compression distortions is enabled by the dataset.
  • The public release of the stimuli and scores establishes a common reference point for research on compressed 3D representations.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The reference-video isolation technique could be applied to other neural rendering formats such as NeRF to separate compression effects in those pipelines as well.
  • Persistent gaps between objective metrics and the subjective scores may motivate training of learned quality predictors that operate directly on splat parameters or rendered views.
  • Standardizing evaluation around this dataset could reduce reliance on proxy metrics that were developed for natural video rather than point-based 3D content.

Load-bearing premise

The perceptual scores collected from the twenty participants are reliable and representative enough to serve as a benchmark for Gaussian Splatting compression.

What would settle it

A repeat subjective test on the same stimuli with a new group of observers that produces substantially different quality rankings for the nine compression methods would show the current scores do not provide a stable benchmark.

Figures

Figures reproduced from arXiv: 2605.26880 by Ant\'onio Rodrigues, Jo\~ao Ascenso, Maria Paula Queluz, Pedro Martin.

Figure 1
Figure 1. Figure 1: GScomp-QA framework [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 4
Figure 4. Figure 4: RD curves for benchmarking GS compression. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
read the original abstract

Gaussian Splatting (GS) has emerged as an efficient representation for high-quality 3D reconstruction and novel view synthesis. However, its large model size poses challenges for storage and transmission. While several GS compression solutions have been proposed, their perceptual impact remains poorly understood due to the lack of dedicated evaluation datasets. To address this gap, this paper introduces GScomp-QA, a subjective quality assessment dataset for evaluating synthesis quality from compressed GS models. The dataset comprises 331 video stimuli from 13 real-world scenes, covering 9 state-of-the-art GS compression solutions. By using videos synthesized from uncompressed models as reference, GScomp-QA isolates compression-induced distortions from synthesis artifacts. A subjective study with 20 participants was conducted, providing reliable perceptual scores. Based on these data, GS compression solutions are evaluated through perceptual rate-distortion analysis. In addition, 18 objective quality metrics are evaluated, showing that they do not fully capture GS-specific distortions. GScomp-QA will be publicly available and provide a benchmark for evaluating GS compression solutions and supporting the development of quality metrics tailored to GS compression.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper introduces GScomp-QA, a subjective quality assessment dataset for compressed Gaussian Splatting (GS) consisting of 331 video stimuli derived from 13 real-world scenes and 9 state-of-the-art compression methods. By synthesizing reference videos from uncompressed GS models, the dataset isolates compression-induced distortions. A subjective study with 20 participants yields perceptual scores used for rate-distortion analysis of the compression methods; additionally, 18 objective quality metrics are evaluated and shown to inadequately capture GS-specific artifacts. The dataset is intended for public release as a benchmark.

Significance. If the perceptual scores prove reliable, GScomp-QA would provide the first dedicated public benchmark for perceptual evaluation of GS compression, enabling standardized comparison of methods and guiding development of GS-tailored objective metrics. The use of uncompressed-model references to separate compression from synthesis artifacts is a clear methodological strength, and the public release commitment adds reproducibility value.

major comments (2)
  1. [Subjective Study] Subjective Study section: The manuscript states that 'a subjective study with 20 participants was conducted, providing reliable perceptual scores' but reports neither inter-rater agreement statistics (ICC, Cronbach's alpha, or Kendall's W) nor per-stimulus confidence intervals or screening criteria for participants and viewing conditions. This directly undermines the central claim that the MOS values constitute a usable benchmark, as low consistency would render the rate-distortion curves and metric correlations unreliable.
  2. [Objective Metric Evaluation] Evaluation of objective metrics section: The claim that '18 objective quality metrics ... do not fully capture GS-specific distortions' requires explicit listing of the 18 metrics, the exact correlation measure employed (e.g., PLCC, SROCC), and per-method or per-scene breakdowns; without these, it is impossible to assess whether the conclusion is supported by the data or driven by a subset of metrics/scenes.
minor comments (2)
  1. [Abstract] Abstract and introduction: The number of scenes (13) and total stimuli (331) are stated, but the distribution across compression methods and bit-rate points is not summarized in a table, making it difficult to gauge coverage.
  2. [Dataset Construction] Dataset description: Clarify whether the 331 stimuli include multiple bit-rate points per method or multiple views per scene, and provide the exact protocol (e.g., ACR, DSCQS) used for the subjective ratings.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments, which highlight areas where additional details will strengthen the manuscript. We address each major comment below and will incorporate revisions to enhance transparency and reproducibility.

read point-by-point responses

  1. Referee: [Subjective Study] Subjective Study section: The manuscript states that 'a subjective study with 20 participants was conducted, providing reliable perceptual scores' but reports neither inter-rater agreement statistics (ICC, Cronbach's alpha, or Kendall's W) nor per-stimulus confidence intervals or screening criteria for participants and viewing conditions. This directly undermines the central claim that the MOS values constitute a usable benchmark, as low consistency would render the rate-distortion curves and metric correlations unreliable.

    Authors: We agree that explicit reporting of inter-rater reliability metrics is essential to substantiate the reliability of the MOS values. The original manuscript omitted these details for brevity, but the underlying data support computation of ICC(2,1), Cronbach's alpha, and Kendall's W, all of which exceed standard thresholds for acceptable agreement. We will add these statistics, along with 95% confidence intervals per stimulus and a description of participant screening (following ITU-R BT.500 guidelines) and viewing conditions (controlled lab setup with calibrated displays), in a revised Subjective Study section. This will directly address the concern and reinforce the benchmark value of the dataset. revision: yes

  2. Referee: [Objective Metric Evaluation] Evaluation of objective metrics section: The claim that '18 objective quality metrics ... do not fully capture GS-specific distortions' requires explicit listing of the 18 metrics, the exact correlation measure employed (e.g., PLCC, SROCC), and per-method or per-scene breakdowns; without these, it is impossible to assess whether the conclusion is supported by the data or driven by a subset of metrics/scenes.

    Authors: We acknowledge that the manuscript should have provided an explicit enumeration of the 18 metrics and the precise correlation coefficients. The evaluation used both PLCC and SROCC (after logistic fitting) computed on the full set of 331 stimuli, with the conclusion driven by consistently low correlations across the set rather than outliers. In the revision we will insert a table listing all 18 metrics (including PSNR, SSIM, LPIPS, VMAF, and others), report the aggregate and per-scene/per-method PLCC/SROCC values, and include a brief discussion of the strongest and weakest performers. This will allow readers to verify that the claim holds across the data. revision: yes

Circularity Check

0 steps flagged

Empirical dataset paper with no derivations or circular elements

full rationale

The paper introduces a subjective quality assessment dataset (GScomp-QA) built from video stimuli synthesized from uncompressed Gaussian Splatting models, a study with 20 participants, and evaluation of 18 existing objective metrics. No mathematical derivations, predictions, fitted parameters, or uniqueness theorems are present. The contribution is direct empirical data collection and benchmarking; all steps (stimulus generation, subjective scoring, metric correlation) are self-contained experimental procedures without reduction to self-citations, ansatzes, or renamings of prior results by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The paper rests on the domain assumption that standard subjective testing protocols yield reliable scores; no free parameters or invented entities are introduced.

axioms (1)
  • domain assumption Standard protocols for subjective quality assessment with 20 participants produce reliable perceptual scores suitable for benchmarking.
    Invoked to support the claim that the collected scores enable perceptual rate-distortion analysis and metric evaluation.

pith-pipeline@v0.9.1-grok · 5733 in / 1121 out tokens · 41976 ms · 2026-07-01T16:04:04.008144+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

32 extracted references · 32 canonical work pages

  1. [1]

    Despite its advantages, GS models may reach gigabyte-scale sizes due to the large number of Gaussians and their high-dimensional parameterization

    has recently emerged as a promising paradigm for novel view synthesis, representing scenes as collections of 3D Gaussians parameterized by spatial attributes and view-dependent appearance. Despite its advantages, GS models may reach gigabyte-scale sizes due to the large number of Gaussians and their high-dimensional parameterization. This poses major chal...

    work page doi:10.1145/1234567890 2018

  2. [2]

    The GScomp-QA dataset proposed in this work addresses these limitations along several dimensions

    includes compressed GS content, but its scope is limited to the artifacts produced by their proposed compression pipeline, without considering alternative GS compression solutions. The GScomp-QA dataset proposed in this work addresses these limitations along several dimensions. First, subjects evaluate synthesized videos rather than isolated images, enabl...

    work page 2018

  3. [3]

    introduce a fine-tuning phase to recover quality lost during the compression process (made of pruning and vector quantization operations). 3.3 Selected Visual Scenes The GScomp-QA dataset is built upon visual scenes drawn from publicly available datasets widely used in GS research [1], selected to ensure diversity in scene structure, scale, and capture co...

    work page 2018

  4. [4]

    Each video loops three times and participants can submit their scores only after the first playback

    The spatial position of the reference video and the presentation order of trials are randomized to minimize positional and contextual bias. Each video loops three times and participants can submit their scores only after the first playback. To mitigate participant fatigue, the study is organized into three balanced test sessions, each covering a represent...

    work page 1920

  5. [5]

    Table 1: Selected visual scenes specifications

    may occur when the test sequence was judged perceptually superior in quality to the baseline reference. Table 1: Selected visual scenes specifications. Visual scene #Train. images Resolution [px.] Visual scene #Train. images Resolution [px.] africa 56 1269×709 garden 161 1297×840 basket 74 1293×721 kitchen 244 1558×1039 bicycle 169 1237×822 playroom 196 1...

    work page 2018

  6. [6]

    5 Experimental Results and Analysis This section analyzes the subjective scores collected in the experiment and benchmarks the considered GS compression solutions

    This procedure enables a unified RD comparison of all compression solutions regardless of their GS baseline model. 5 Experimental Results and Analysis This section analyzes the subjective scores collected in the experiment and benchmarks the considered GS compression solutions. Section 5.1 presents and analyzes the DMOS scores, including a non-inferiority...

    work page 2018

  7. [7]

    RMSE and PWRC results are presented in the dataset webpage [3], alongside a statistical significance analysis of the metrics’ performances. Overall, NLPD achieves the highest correlation across all criteria (with PLCC of 0.88 and SROCC of 0.87), followed by ST-LPIPS, FSIM, GMSD, and VMAF, reflecting the benefits of incorporating structural sensitivity and...

    work page 2018

  8. [8]

    IEEE Transactions on Visualization and Computer Graphics 31, 8 (2025), 4429–4449

    3D Gaussian Splatting as a New Era: A Survey. IEEE Transactions on Visualization and Computer Graphics 31, 8 (2025), 4429–4449

    work page 2025

  9. [9]

    ACM Transactions on Graphics 42, 4 (July 2023)

    3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Transactions on Graphics 42, 4 (July 2023)

    work page 2023

  10. [10]

    In 2025 IEEE International Conference on Visual Communications and Image Processing (VCIP), 1–5

    3DGS-VBench: A Comprehensive Video Quality Evaluation Benchmark for 3DGS Compression. In 2025 IEEE International Conference on Visual Communications and Image Processing (VCIP), 1–5

    work page 2025

  11. [11]

    ArXiv:2511.06830 [cs.CV] https://arxiv.org/abs/2511.06830

    MUGSQA: Novel Multi-Uncertainty-Based Gaussian Splatting Quality Assessment Method, Dataset, and Benchmarks. ArXiv:2511.06830 [cs.CV] https://arxiv.org/abs/2511.06830

    work page arXiv

  12. [12]

    arXiv preprint arXiv:2311.18159 , volume=

    Compact3D: Smaller and Faster Gaussian Splatting with Vector Quantization. ArXiv:2311.18159 [cs.CV] https://arxiv.org/abs/2311.18159

    work page arXiv

  13. [13]

    ArXiv:2501.12255 [cs.CV] https://arxiv.org/abs/2501.12255

    HAC++: Towards 100X Compression of 3D Gaussian Splatting. ArXiv:2501.12255 [cs.CV] https://arxiv.org/abs/2501.12255

    work page arXiv

  14. [14]

    ArXiv:2405.20721 [cs.CV] https://arxiv.org/abs/2405.20721

    ContextGS: Compact 3D Gaussian Splatting with Anchor Level Context Model. ArXiv:2405.20721 [cs.CV] https://arxiv.org/abs/2405.20721

    work page arXiv

  15. [15]

    ArXiv:2601.14510 [cs.MM] https://arxiv.org/abs/2601.14510

    Structured Image-based Coding for Efficient Gaussian Splatting Compression. ArXiv:2601.14510 [cs.MM] https://arxiv.org/abs/2601.14510

    work page arXiv

  16. [16]

    ArXiv:2410.08017 [cs.CV] https://arxiv.org/abs/2410.08017

    Fast Feedforward 3D Gaussian Splatting Compression. ArXiv:2410.08017 [cs.CV] https://arxiv.org/abs/2410.08017

    work page arXiv

  17. [17]

    ACM Transactions on Graphics 36, 4 (2017), 1–13

    Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Transactions on Graphics 36, 4 (2017), 1–13

    work page 2017

  18. [18]

    ACM Transactions on Graphics 37, 6 (2018), 1–15

    Deep blending for free-viewpoint image-based rendering. ACM Transactions on Graphics 37, 6 (2018), 1–15

    work page 2018

  19. [19]

    ACM Transactions on Graphics 35, 3 (2016)

    Efficient 3D object segmentation from densely sampled light fields with applications to 3D reconstruction. ACM Transactions on Graphics 35, 3 (2016)

    work page 2016

  20. [20]

    Geneva: International Telecommunication Union (2023)

    500-15, Methodologies for the subjective assessment of the quality of television images. Geneva: International Telecommunication Union (2023)

    work page 2023

  21. [21]

    Computer Graphics Forum 44, 2 (2025), e70078

    3DGS.zip: A Survey on 3D Gaussian Splatting Compression Methods. Computer Graphics Forum 44, 2 (2025), e70078

    work page 2025

  22. [22]

    IEEE Transactions on Image Processing 20, 8 (Aug

    FSIM: A Feature Similarity Index for Image Quality Assessment. IEEE Transactions on Image Processing 20, 8 (Aug. 2011), 2378–2386

    work page 2011

  23. [23]

    ACM Transactions on Graphics 40, 4 (Jul

    FovVideoVDP: A Visible Difference Predictor for Wide Field-of-View Video. ACM Transactions on Graphics 40, 4 (Jul. 2021)

    work page 2021

  24. [24]

    IEEE Transactions on Image Processing 20, 5 (May 2011), 1185–1198

    Information Content Weighting for Perceptual Image Quality Assessment. IEEE Transactions on Image Processing 20, 5 (May 2011), 1185–1198

    work page 2011

  25. [25]

    Journal of Electronic Imaging 19, 1 (Jan

    Most Apparent Distortion: Full-Reference Image Quality Assessment and the Role of Strategy. Journal of Electronic Imaging 19, 1 (Jan. 2010), 011006

    work page 2010

  26. [26]

    Journal of the Optical Society of America A 34, 9 (Sep

    Perceptually Optimized Image Rendering. Journal of the Optical Society of America A 34, 9 (Sep. 2017), 1511–1525

    work page 2017

  27. [27]

    IEEE Transactions on Image Processing 13, 4 (Apr

    Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing 13, 4 (Apr. 2004), 600–612

    work page 2004

  28. [28]

    IEEE Transactions on Image Processing 15, 2 (Feb

    Image Information and Visual Quality. IEEE Transactions on Image Processing 15, 2 (Feb. 2006), 430–444

    work page 2006

  29. [29]

    IEEE Transactions on Image Processing 23, 10 (Oct

    VSI: A Visual Saliency-Induced Index for Perceptual Image Quality Assessment. IEEE Transactions on Image Processing 23, 10 (Oct. 2014), 4270–4281

    work page 2014

  30. [30]

    IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 5 (May 2022), 2567–2581

    Image Quality Assessment: Unifying Structure and Texture Similarity. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 5 (May 2022), 2567–2581

    work page 2022

  31. [31]

    Medium (Apr

    Toward a Practical Perceptual Video Quality Metric. Medium (Apr. 2017)

    work page 2017

  32. [32]

    IEEE Transactions on Image Processing 27, 5 (2018), 2499–2513

    A Perceptually Weighted Rank Correlation Indicator for Objective Image Quality Assessment. IEEE Transactions on Image Processing 27, 5 (2018), 2499–2513

    work page 2018