REVIEW 5 major objections 5 minor 12 references
SpatialQ aims to show that 3D Gaussian Splatting scene quality is best judged by feeding a vision-language model the rendered views along with depth maps, point-cloud renderings, and camera parameters, then adding a bounded correction to a
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 12:37 UTC pith:2OVRO77H
load-bearing objection Promising 3DGS quality assessment pipeline, but the SOTA claim rests on an unspecified split unit—if it's image-level, the numbers are inflated. the 5 major comments →
SpatialQ: Understanding 3D Gaussian Splatting Scene Quality via Visual-based MLLM
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
SpatialQ's central claim is that 3DGS scene quality decomposes cleanly into a base estimate from data-driven 3D-aware representations and a bounded adjustment from grounded multimodal reasoning. The base learner jointly encodes two rendered views, aggregates view-specific features to expose cross-view inconsistency, and predicts view-wise scores that are pooled; this alone exceeds all compared IQA and MLLM baselines. The reasoning module then reinforces or repairs that base score: given depth maps, point-cloud renderings, and camera parameters alongside the images, the MLLM must choose a degradation label from a closed set and a severity, which is converted into a delta-Q within a bounded ra
What carries the argument
The load-bearing object is the paired-view spatial quality encoder built on VGGT with a dedicated quality head. Two rendered views share weights, produce view features, are fused by concatenation plus attention to form a joint representation, and each view gets a quality score that is aggregated into the base score. This encoder also produces the depth maps, point-cloud visualizations, and camera parameters that the MLLM consumes. The second mechanism is the bounded correction: the final score equals the base score plus a delta-Q, where delta-Q is mapped from the MLLM's degradation type and severity and constrained to a small interval, so the reasoning module can only refine, not override, t
Load-bearing premise
The spatial evidence—depth maps and point-cloud renderings—is computed from the very rendered views whose quality is being judged, so artifacts in those images can contaminate the geometric ground truth the reasoning module relies on.
What would settle it
Take the trained SpatialQ and replace the MLLM's spatial inputs with the original images duplicated (no depth, no point cloud, no camera parameters), keeping everything else fixed; if SRCC and PLCC on 3DGS-IEval-15K remain within a small margin of the reported 0.8937 and 0.8840, the spatial grounding is not doing the claimed work. Alternatively, shuffle the degradation labels supplied to the MLLM for a fixed base score and show the final scores barely move, which would demonstrate the reasoning module is not load-bearing.
If this is right
- If the central claim holds, deploying SpatialQ for 3DGS model selection or compression tuning gives scores that track human ranks better than PSNR/SSIM or single-image deep IQA, with a plausible explanation attached.
- The decomposition implies that quality regression and degradation diagnosis can be trained or improved semi-independently; a better diagnosis module directly transfers to better score corrections.
- The closed-set degradation labels tied to compression methods mean the model can act as a compressor-aware diagnostic, identifying which compression scheme likely generated the artifacts.
- Because the model uses only rendered views plus derived geometry, it applies to any 3DGS scene without accessing the underlying Gaussian parameters, which suits real applications.
Where Pith is reading between the lines
- The paper does not ablate the MLLM's spatial inputs (images-only versus images plus geometry); if performance holds without the geometry inputs, the claimed spatial grounding would be decorative rather than load-bearing. A targeted ablation would settle this.
- The 8:2 random split presumably separates rendered images rather than scenes or compression models; if that is the case, test views of the same scenes seen in training could inflate correlation numbers relative to scene-level generalization.
- Because the adjustment is bounded and small (e.g., plus or minus 0.08 on the MOS scale), the headline gains over the base encoder mostly come from the base encoder; a reader should look for whether the MLLM's delta-Q provides statistically significant bias correction beyond the base, or is a minor fine-tune.
- The same architecture should transfer to other neural rendering representations such as NeRF or explicit mesh renderings; the paper leaves that untested.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SpatialQ, a multimodal framework for 3D Gaussian Splatting (3DGS) quality assessment. It combines a VGGT-based multi-view encoder with a dedicated quality head to produce a base quality score, and a Qwen2.5-VL MLLM that receives rendered views, depth maps, point-cloud renderings, and camera parameters to diagnose degradation type and severity, producing a bounded score adjustment. The final score is the base score plus the adjustment. The authors report state-of-the-art performance on the 3DGS-IEval-15K benchmark (SRCC 0.8937, PLCC 0.8840), with additional experiments on scene subsets, distortion-type scatter analyses, and ablations. They claim to be the first to explicitly model 3DGS scene quality through spatially-aware representations with interpretable degradation reasoning.
Significance. If the reported results are valid, SpatialQ would represent a meaningful step beyond single-image IQA for 3DGS evaluation, with a practical decomposition into a stable regressor and an interpretable reasoning module. The combination of geometric cues (depth, point clouds) with an MLLM for quality refinement is a plausible design direction, and the authors provide an extensive benchmark comparison and qualitative reasoning examples. However, the significance is conditional: the central SOTA claim rests on a train/test split whose unit is not specified, the comparison omits the 3DGS-specific QA baselines cited in the related work, and the reported KRCC direction and the MLLM ablation gain are internally inconsistent. These issues must be resolved before the contributions can be accepted at face value.
major comments (5)
- [§4.2 Table 3] The random 8:2 split is described as having 'no overlap between training and testing samples,' but the split unit is never stated. The dataset contains 760 compressed 3DGS models, each rendered from 20 viewpoints. If the split is applied to individual rendered images rather than to model/scene identities, then near-duplicate views from the same compressed model can appear in both training and test sets, allowing the encoder to memorize per-model degradation patterns and inflating SRCC/PLCC. Please specify the split unit and, ideally, report results with a model-level or scene-level split to rule out intra-model leakage.
- [§4.2 Table 3] The benchmark comparison omits the 3DGS-specific quality assessment methods cited in the paper, including GS-QA [8], NeRF-QA [21], GSC-QA [16], and the method proposed in the 3DGS-IEval-15K paper [11]. Comparing only against handcrafted IQA, deep IQA, and zero-shot MLLMs does not support the claim of 'state-of-the-art' 3DGS quality assessment. The baselines should include these 3DGS-aware approaches, especially since the related work argues they are insufficient.
- [Table 3 and Table 5] KRCC is reported with a downward arrow (↓), implying lower is better. Under that convention, SpatialQ's All-set KRCC of 0.6161 is worse than several baselines (e.g., BRISQUE 0.1495, LIQE-like models, DBCNN 0.6699 is lower? Actually 0.6699 is higher, so not worse; but BRISQUE 0.1495 is much lower and better). The text states SpatialQ achieves 'the best overall performance across all evaluation settings,' which is contradicted by the KRCC column if ↓ is the intended direction. If the arrow is a typo and higher KRCC is better, then Table 5's KRCC drop from 0.7135 (row C) to 0.6161 (row D) after adding the MLLM is a severe degradation. Please correct the arrow and reconcile the tables.
- [§4.6 Table 5] The contribution of the MLLM reasoning module is not supported by the reported numbers. Adding the MLLM changes SRCC from 0.8935 to 0.8937 (+0.0002) and PLCC from 0.8801 to 0.8840 (+0.0039), while KRCC changes from 0.7135 to 0.6161 — a 0.0974 drop if higher is better, or an improvement if lower is better, but in either case the direction is inconsistent with the table header and the qualitative claims. The authors call the PLCC improvement 'consistent' but do not provide statistical significance or per-distortion analysis. This does not convincingly demonstrate that reasoning-guided refinement improves accuracy, which is a central contribution.
- [Table 1 and §3.2] The degradation diagnosis is restricted to a closed set of labels that exactly matches the six compression methods used to generate the benchmark (LightGS, Scaffold-GS, HAC, C3DGS, CompGS, Compact 3DGS, plus 'Unknown'). This turns the MLLM's 'degradation reasoning' into a known-label classification of the generating compression method, and the score adjustment h(t,s) can directly exploit the dataset construction. The claimed interpretability and generalization are therefore limited; the paper should either evaluate on unseen degradation types or discuss this circularity explicitly.
minor comments (5)
- [§3.1-§3.3] Several components are left unspecified: the aggregation function φ in Eq. (5), the mapping h in Eqs. (9)/(11), and the bound δ in Eq. (12). These are essential for reproducibility; please provide concrete definitions or hyperparameters.
- [§2.1 heading] Typo: '3D guassian splatting' should be 'Gaussian.' Also, author affiliation line has 'Shenglin W ANG' with an extra space.
- [Figure 4] The scatter-plot description refers to 'CompE,' but the rest of the paper uses 'CompGS' and 'Compact 3DGS.' Please unify the naming.
- [References] References [40] and [27] appear to be the same paper (Zhang et al., CVPR 2023) but are numbered separately. Please deduplicate.
- [§4.4 and Table 5] The KRCC values in Table 4 (e.g., 0.7642 for MipNeRF 360) are substantially higher than those in Table 3 (0.6161 for All), but both use the ↓ notation. If ↓ means lower-better, the Table 4 numbers would be worse than Table 3, which is inconsistent; if ↑ means higher-better, the header should be changed throughout.
Circularity Check
No circular derivation: SpatialQ's scores are supervised outputs evaluated on a held-out split; the closed-set degradation labels and self-citations raise validity concerns but do not reduce to circularity.
full rationale
The paper's derivation chain is a standard supervised-learning pipeline. The base score Qbase is a learned function of multi-view images (Eqs. 1-5), the MLLM produces a structured diagnosis R={t,s} (Eqs. 6-8), and the final score is Qbase + h(R) (Eqs. 10-12). Both Qbase and h are determined by training on the 80% split of 3DGS-IEval-15K, and the reported SRCC/PLCC are evaluated on the held-out 20%. This is ordinary empirical prediction, not an equation-level identity: the test MOS is not defined as the model output. The closed-set degradation labels (Table 1: LightGS, Scaffold-GS, HAC, C3DGS, CompGS, Compact 3DGS) coincide with the dataset's six compression methods, which is a legitimate design concern about label leakage and about the interpretation of 'degradation reasoning' as compression-method classification, but it does not make the final quality prediction equivalent to an input by construction. The split-unit ambiguity (per-model vs. per-image) is a possible data-leakage risk, but the paper does not specify enough to establish leakage, and leakage would be a validity flaw rather than a circular reduction. Self-citations [17-20] appear only in related work on point-cloud quality assessment and are not load-bearing for the proposed method. No circular step can be exhibited from the paper's equations or citations.
Axiom & Free-Parameter Ledger
free parameters (4)
- Adjustment bound δ =
not reported
- Adjustment mapping h from (degradation type, severity) to ΔQ =
not specified
- View-pair aggregation function φ =
unspecified
- Train/test split ratio 8:2 =
8:2
axioms (4)
- domain assumption MOS annotations in 3DGS-IEval-15K are reliable ground truth
- domain assumption VGGT-estimated depth maps and point-cloud renderings are accurate enough to serve as geometric grounding
- ad hoc to paper Degradation categories are adequately summarized by the closed set {LightGS, Scaffold-GS, HAC, C3DGS, CompGS, Compact 3DGS, Unknown}
- domain assumption The 8:2 random split avoids data leakage
read the original abstract
3D Gaussian Splatting (3DGS) has emerged as an effective representation for novel view synthesis and 3D scene reconstruction, creating an increasing demand for reliable quality assessment. Unlike conventional image quality assessment (IQA), the quality of a 3DGS scene depends not only on the perceptual fidelity of rendered views, but also on scene-level factors such as spatial structure and cross-view consistency. Existing IQA methods are limited by their reliance on 2D perceptual cues, whereas general multimodal large language models (MLLMs) are not designed for stable quality regression and may produce unreliable judgments. To address these limitations, a multimodal quality assessment framework is developed for 3DGS scene understanding. First, a 3D-aware quality representation learning framework is introduced by augmenting a VGGT-based encoder with a dedicated quality head. Multi-view images are encoded into view-specific features and aggregated to capture cross-view consistency, while geometric cues are incorporated through joint modeling of depth and point-cloud-related structural information, enabling the learning of structure-aware quality representations beyond appearance-driven features. Second, a grounded multimodal reasoning mechanism is constructed by jointly feeding original images, depth maps, point cloud renderings, and camera parameters into a Qwen-based MLLM.
Reference graph
Works this paper leans on
-
[1]
3d gaussian splatting: Survey, technologies, challenges, and opportunities
1 Bao Y, Ding T, Huo J, et al. 3d gaussian splatting: Survey, technologies, challenges, and opportunities. IEEE Transactions on Circuits and Systems for Video Technology, 2025, 35: 6832–6852 2 Fei B, Xu J, Zhang R, et al. 3d gaussian splatting as new era: A survey. IEEE Transactions on Visualization and Computer Graphics, 2024 3 Yang D, Qin Z. Generalizab...
2025
-
[3]
Quality assessment in the era of large models: A survey
12682–12689 12 Zhang Z, Zhou Y, Li C, et al. Quality assessment in the era of large models: A survey. ACM Transactions on Multimedia Computing, Communications and Applications, 2025, 21: 1–31 13 Zhang Z, Jia Z, Li C, et al. Towards versatile multimedia quality assessment for visual communications. Science China Information Sciences, 2026, 69: 122306 14 Zh...
2025
-
[5]
Large multimodal models evaluation: a survey
1–5 23 Zhang Z, Wang J, Wen F, et al. Large multimodal models evaluation: a survey. Science China Information Sciences, 2025, 68: 221301 24 Li W, Zhang X, Zhao S, et al. Q-insight: Understanding image quality via visual reinforcement learning. In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2025 25 Zhao S, Zhang X, Li W...
2025
-
[6]
Compact 3d gaussian splatting for static and dynamic radiance fields
10349–10358 31 Lee J C, Rho D, Sun X, et al. Compact 3d gaussian splatting for static and dynamic radiance fields. arXiv preprint arXiv:2408.03822, 2024 32 Chen Y, Wu Q, Lin W, et al. Hac: Hash-grid assisted context for 3d gaussian splatting compression. In: Proceedings of the European Conference on Computer Vision. Springer,
Pith/arXiv arXiv 2024
-
[7]
Image quality assessment: from error visibility to structural similarity
20654–20664 34 Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13: 600–612 35 Sheikh H R, Bovik A C. Image information and visual quality. IEEE Transactions on Image Processing, 2006, 15: 430–444 36 Mittal A, Moorthy A K, Bovik A C. No-reference im...
2004
-
[10]
1759–1767 44 Xiong T, Li B, Guo D, et al. Llava-onevision-chat: Improving chat with preference learning, September 2024 45 Lu H, Liu W, Zhang B, et al. Deepseek-vl: towards real-world vision-language understanding. arXiv preprint arXiv:2403.05525, 2024 46 Bai S, Chen K, Liu X, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025 47 Y...
Pith/arXiv arXiv 2024
-
[12]
Image quality assessment: Unifying structure and texture similarity
1220–1230 54 Ding K, Ma K, Wang S, et al. Image quality assessment: Unifying structure and texture similarity. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 44: 2567–2581
2020
-
[2018]
Blind image quality assessment using a deep bilinear convolutional neural network
586–595 38 Zhang W, Ma K, Yan J, et al. Blind image quality assessment using a deep bilinear convolutional neural network. IEEE Transactions on Circuits and Systems for Video Technology, 2018, 30: 36–47 39 Ke J, Wang Q, Wang Y, et al. Musiq: Multi-scale image quality transformer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision,
2018
-
[2022]
Tanks and temples: Benchmarking large-scale scene reconstruction
5470–5479 50 Knapitsch A, Park J, Zhou Q Y, et al. Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Trans. Graph., 2017, 36: 1–13 51 Hedman P, Philip J, Price T, et al. Deep blending for free-viewpoint image-based rendering. ACM Transactions on Graphics (ToG), 2018, 37: 1–15 52 Wang J, Chen M, Karaev N, et al. Vggt: Visual geometry gr...
2017
-
[2023]
Exploring clip for assessing the look and feel of images
14071–14081 41 Wang J, Chan K C, Loy C C. Exploring clip for assessing the look and feel of images. In: Proceedings of AAAI, 2023 42 Wu H, Zhang Z, Zhang W, et al. Q-align: teaching lmms for visual scoring via discrete text-defined levels. In: Proceedings of the 41st International Conference on Machine Learning,
2023
-
[2024]
Com-pcqa: No-reference point cloud quality assessment via complex-valued feature learning
1–8 17 Su J, Li G, Wang S, et al. Com-pcqa: No-reference point cloud quality assessment via complex-valued feature learning. IEEE Transactions on Image Processing, 2026 18 Wang J, Gao W, Li G. Applying collaborative adversarial learning to blind point cloud quality measurement. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 1–15 19 Fan S,...
2026
-
[2025]
Artificial intelligence for virtual reality: a review
36–42 4 Wang L, Xu W, Liu Y, et al. Artificial intelligence for virtual reality: a review. Science China Information Sciences, 2026, 69: 111101 5 Zhu S, Wang G, Kong X, et al. 3d gaussian splatting in robotics: A survey. arXiv preprint arXiv:2410.12262, 2024 6 Lee M C, Chen C C, Pan C T. A multimodal sensor-integrated lower limb exoskeleton with digital t...
Pith/arXiv arXiv 2026
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.