REVIEW 4 major objections 5 minor 37 references
TSGaussian: Semantic and Depth-Guided Target-Specific Gaussian Splatting from Sparse Views
T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read In TSGaussian, semantic masks and multi-scale depth priors guide 3D Gaussian Splatting to reconstruct a specified target from sparse 360° views with higher PSNR/SSIM and lower LPIPS than prior methods.
desk verdict Practical sparse-view target-specific 3DGS recipe, but Table 1 contradicts its own 'all metrics' claim, so the headline result needs a corrected evaluation before it can be believed. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is the joint action of three modules. First, an identity encoding turns each Gaussian ellipsoid into a carrier of object identity, with a differentiable alpha-compositing renderer producing 2D semantic maps that are supervised by tracked masks and a K-nearest feature-distance loss. Second, a semantic-driven control and pruning step restricts densification to the 3D region of interest and removes Gaussians whose identity drifts, preventing background overfitting. Third, multi-scale depth regularization uses a monocular depth prior through a soft-hard depth loss that alternately fixes opacity and center, and a global-local depth loss that normalizes patches to correct small errors. Together these keep the Gaussian field focused on the target while preserving geometric plausibility in sparse-view settings.
What would settle it
Run TSGaussian on a sparse-view scene dominated by thin or translucent elements, such as a wire sculpture or a glass bottle, and compare it against plain 3DGS with identical masks; if the depth-regularized version does not exceed 3DGS in PSNR, or its rendered geometry is visibly distorted, the claim that depth priors prevent geometric degradation is not supported.
Extended reading notes
Core claim
The central claim is that a target-specific 3D Gaussian field, driven by identity encodings learned from 2D masks and regularized by multi-scale depth losses, outperforms existing splatting methods when only one-third of the views are available. Concretely, TSGaussian assigns each Gaussian a compact identity code, renders semantics via alpha-compositing, and trains with a cross-entropy identity loss plus a K-nearest-neighbor consistency loss, so Gaussians cluster by object. It then densifies and prunes based on semantic attributes rather than purely positional gradients, focusing capacity on the target. A soft-hard depth loss and a global-local depth normalization loss, both anchored to a pretrained monocular depth estimator, keep the Gaussian centers and opacities geometrically consistent across unseen views. On the bear, bouquet, garden, and self-collected citrus scenes, the method reports PSNR/SSIM/LPIPS that beat 3DGS, Gaussian Grouping, DNGaussian, and SparseGS; for the citrus scene the reported PSNR is 27.40 versus 18.90 for the best baseline.
Load-bearing premise
The approach assumes the monocular depth estimates are accurate enough, after normalization, to serve as a reliable geometric prior for the target region; if they contain systematic errors on thin, transparent, or reflective structures, the depth losses will pull Gaussian positions and opacities to wrong places.
Editorial extensions
If this is right
- With only one-third of the original views, the method reports PSNR/SSIM/LPIPS that beat 3DGS, Gaussian Grouping, DNGaussian, and SparseGS on the bear, bouquet, and garden scenes.
- By pruning Gaussians outside the target's semantic region, the method reduces background allocation and redundant computations during optimization.
- The identity encoding plus pruning lets the method output semantically clean masks of the target in novel views, not just render the object.
- The citrus dataset demonstrates the method on slender plant structures where the paper reports a PSNR of 27.40 versus 18.90 for Gaussian Grouping.
Reading between the lines
- The 8.50 dB PSNR lead on the citrus scene may reflect the semantic pruning as much as the depth prior, since removing background Gaussians should reduce overfitting to sparse training views; a testable extension is to add the same pruning to DNGaussian or Gaussian Grouping.
- Because removing depth regularization costs only 0.496 PSNR in the ablation, the depth prior may be acting as a mild stabilizer rather than the main driver of improvement; a fair reader could test whether a simpler smoothness regularizer yields a similar effect.
- The pipeline depends on the detector and the tracker to propagate masks; if either fails on an unseen object class, the identity loss receives wrong supervision, so robustness to detection and tracking errors is an implicit boundary of the method.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. TSGaussian proposes a target-specific 3D Gaussian Splatting framework for sparse-view novel view synthesis. The method uses YOLOv9 bounding boxes as prompts for SAM to obtain 2D target masks, a zero-shot tracker to maintain identity consistency, a compact 16-dimensional identity encoding per Gaussian with 2D and 3D grouping losses, a semantic-driven densification and pruning strategy, and multi-scale depth regularization borrowed from DNGaussian. The paper claims state-of-the-art performance on bear, bouquet, garden, and a newly collected citrus dataset, with the largest reported gain being a PSNR of 27.40 versus 18.90 for the best baseline on the citrus scene. The central contribution is empirical: the proposed combination of semantic constraints and depth priors is claimed to outperform existing methods for target-specific sparse-view reconstruction.
Significance. If the empirical claims were fully supported, this would be a useful contribution to sparse-view 3D reconstruction of specific objects, an application-relevant setting. The modular pipeline is sensible, the equations are coherent and mostly grounded in prior work (DNGaussian depth losses, Gaussian Grouping identity encoding), and the problem of target-specific rather than panoptic reconstruction is worth addressing. However, the significance is currently undermined by the evaluation: the main comparison table contains internal contradictions with the paper's own text, several baseline entries are implausible, the ablation is performed on a single scene, and no error bars or reproducible code verification are provided. The idea has promise, but the evidence as presented does not establish the headline claim.
major comments (4)
- [§5.2, Table 1] The text in §5.2 states that on public datasets TSGaussian outperforms all baselines in PSNR, SSIM, and LPIPS and consistently achieves lower LPIPS, but Table 1 contradicts this claim. On garden, 3DGS has higher SSIM (0.945 vs 0.942) and lower LPIPS (0.042 vs 0.049) than TSGaussian; on bouquet, Gaussian Grouping has lower LPIPS (0.127 vs 0.128). Since the abstract and introduction rest on the claim of superior results over state-of-the-art methods, this table is load-bearing. Please correct the table or qualify the claims; also verify the entries, as several baseline numbers are implausible, e.g., SparseGS garden SSIM of 0.502 at PSNR 22.16 and DNGaussian garden LPIPS of 0.795 at PSNR 20.97, which suggest a systematic evaluation-protocol or metric-computation issue rather than ordinary overfitting.
- [§5.3, Table 2] The ablation study is reported on a single scene, with no error bars or repeated runs. The 'ALL' row (27.395 PSNR, 0.942 SSIM, 0.062 LPIPS) closely matches the TSGaussian row for the citrus dataset in Table 1 (27.40, 0.942, 0.063), suggesting the ablation is conducted on one scene only. The claimed conclusions about the importance of depth regularization (PSNR drop of 0.496) and semantic constraints (PSNR drop of 9.349) should be supported by ablations across all datasets, ideally with standard deviations, before the paper can claim consistent robustness.
- [§3.4, Eqs. (5)-(11)] The depth prior is a load-bearing component: the method relies on pretrained monocular depth estimates (Ranftl et al. 2020) after soft-hard and global-local normalization. The paper provides no validation of the depth estimator's accuracy on the evaluated scenes, despite the target objects being thin, translucent, or non-Lambertian (e.g., citrus seedlings, bouquet flowers). The ablation shows that removing depth regularization costs only 0.496 PSNR, so the benefit is modest; a failure analysis or at least a per-scene breakdown of the depth loss would be needed to establish that the depth prior is not introducing systematic biases.
- [§5.1, baseline protocol] The evaluation protocol for the baselines is under-specified in a way that affects the validity of Table 1. The text says 'all compared algorithms use the same semantic masks of specific objects,' but it is not stated whether evaluation metrics are computed on the full image, on a cropped target region, or with any masking. Baselines such as 3DGS and SparseGS do not use identity masks during training, so it is unclear what is being compared. Please specify exactly how PSNR/SSIM/LPIPS are computed, whether the same training views and iterations are used for all methods, and how the target-specific evaluation is performed for methods without semantic masks.
minor comments (5)
- [§5.3, Table 2 heading] The heading 'Ablation of Deep Regularization' should read 'Ablation of Depth Regularization' for consistency with Eq. (9) and the rest of the paper.
- [Eq. (3)] The 3D regularization loss in Eq. (3) uses the notation F(e_j) log(F(e_j)/F(e'_i)) but does not define e'_i or clarify whether F outputs a probability distribution. If this is intended as a KL divergence, the normalization and summation order should be stated explicitly.
- [Limitations and Future Work] The section titled 'Limitations and Future Work' does not actually describe any limitations; it only outlines future directions. The paper should state what the method cannot handle, such as failure modes of the depth estimator or cases where YOLOv9/SAM mask predictions are incorrect.
- [§5.1 experimental settings] The sparse-view sampling is described as 'one-third of the original views' split evenly into training and test sets, but no details are given on how the one-third views are selected, whether the split is random, or how many test views are used per scene. This makes the results difficult to reproduce.
- [General] No error bars or standard deviations are reported for any quantitative result. Given the large variability commonly observed in sparse-view 3DGS training, a single run per scene is insufficient to support claims of consistent superiority.
Circularity Check
No significant circularity: all load-bearing inputs (SAM/YOLOv9 masks, monocular depth, held-out views) are external to the method, and no equation reduces to a fitted parameter.
full rationale
The derivation chain is self-contained. Semantic constraints are supervised by 2D masks produced by YOLOv9-prompted SAM, which are inputs, not outputs; Eq. (2) is cross-entropy against those masks. Depth regularization (Eqs. 5-9) penalizes rendered depth against MiDaS output (Ranftl et al. 2020), an external pretrained estimator; optimizing Gaussian centers and opacities to match it is supervised fitting, not self-prediction. Identity encoding and pruning are driven by the same mask-derived semantic attributes, so the 'target-specific' behavior is an input constraint, not a manufactured result. The central claim is empirical; although the text overstates Table 1 (e.g., garden SSIM/LPIPS and bouquet LPIPS contradict 'outperform all baselines'), that is an evidence/correctness discrepancy, not circularity. There are no load-bearing author self-citations, no imported uniqueness theorem, and no ansatz smuggled via citation: DNGaussian's soft-hard and global-local depth normalization is cited as prior work and reused as a component. Ablations quantify component contributions but do not rename the fit as prediction. Therefore the paper exhibits no significant circularity.
Assumptions & free parameters
free parameters (5)
- Identity encoding dimension =
16
- Loss weights lambda_2d, lambda_3d, lambda_id, lambda_D, lambda_SH, lambda_GL, gamma =
Not reported
- Number of nearest neighbors K in 3D regularization loss =
Not reported
- Pruning threshold and floating-mask criterion =
Not reported
- Number of training views per scene =
10 (bear), 30 (bouquet, garden), unspecified for citrus
assumptions (4)
- standard math Adaptive density control and differentiable rendering from 3DGS are assumed correct, originally from Kerbl et al. 2023.
- domain assumption The pretrained monocular depth estimator (Ranftl et al. 2020) provides depth maps that are reliable geometric priors for the target after normalization.
- domain assumption YOLOv9, SAM, and the DEVA tracker produce accurate, cross-view-consistent target masks.
- ad hoc to paper The soft-hard and global-local depth loss definitions are taken as-is from DNGaussian (Li et al. 2024b).
Cite this review
Pith. "Pith review of TSGaussian: Semantic and Depth-Guided Target-Specific Gaussian Splatting from Sparse Views." pith.science (2026). https://pith.science/paper/C2HB4BBI
@misc{pith2026241210051,
author = {Pith},
title = {Pith review of: TSGaussian: Semantic and Depth-Guided Target-Specific Gaussian Splatting from Sparse Views},
year = {2026},
howpublished = {\url{https://pith.science/paper/C2HB4BBI}},
note = {Machine review of arXiv:2412.10051}
}
read the original abstract
Recent advances in Gaussian Splatting have significantly advanced the field, achieving both panoptic and interactive segmentation of 3D scenes. However, existing methodologies often overlook the critical need for reconstructing specified targets with complex structures from sparse views. To address this issue, we introduce TSGaussian, a novel framework that combines semantic constraints with depth priors to avoid geometry degradation in challenging novel view synthesis tasks. Our approach prioritizes computational resources on designated targets while minimizing background allocation. Bounding boxes from YOLOv9 serve as prompts for Segment Anything Model to generate 2D mask predictions, ensuring semantic accuracy and cost efficiency. TSGaussian effectively clusters 3D gaussians by introducing a compact identity encoding for each Gaussian ellipsoid and incorporating 3D spatial consistency regularization. Leveraging these modules, we propose a pruning strategy to effectively reduce redundancy in 3D gaussians. Extensive experiments demonstrate that TSGaussian outperforms state-of-the-art methods on three standard datasets and a new challenging dataset we collected, achieving superior results in novel view synthesis of specific objects. Code is available at: https://github.com/leon2000-ai/TSGaussian.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
T.; Mildenhall, B.; Verbin, D.; Srinivasan, P
Barron, J. T.; Mildenhall, B.; Verbin, D.; Srinivasan, P. P.; and Hedman, P. 2022. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 5470--5479
2022
-
[4]
Cen, J.; Zhou, Z.; Fang, J.; Shen, W.; Xie, L.; Jiang, D.; Zhang, X.; Tian, Q.; et al. 2023. Segment anything in 3d with nerfs. Advances in Neural Information Processing Systems, 36: 25971--25990
work page 2023
-
[5]
Chen, Y.; Chen, Z.; Zhang, C.; Wang, F.; Yang, X.; Wang, Y.; Cai, Z.; Yang, L.; Liu, H.; and Lin, G. 2024. Gaussianeditor: Swift and controllable 3d editing with gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 21476--21485
work page 2024
-
[6]
K.; Oh, S
Cheng, H. K.; Oh, S. W.; Price, B.; Schwing, A.; and Lee, J.-Y. 2023. Tracking anything with decoupled video segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 1316--1326
2023
-
[7]
Dai, J.; Zhang, Z.; Mao, S.; and Liu, D. 2020. A View Synthesis-Based 360° VR Caching System Over MEC-Enabled C-RAN. IEEE Transactions on Circuits and Systems for Video Technology, 3843–3855
work page 2020
-
[8]
Fan, Z.; Wang, K.; Wen, K.; Zhu, Z.; Xu, D.; and Wang, Z. 2023. Lightgaussian: Unbounded 3d gaussian compression with 15x reduction and 200+ fps. arXiv preprint arXiv:2311.17245
arXiv 2023
Show all 37 references
-
[9]
Feng, Q.; Xing, Z.; Wu, Z.; and Jiang, Y.-G. 2024. Fdgaussian: Fast gaussian splatting from single image via geometric-aware diffusion model. arXiv preprint arXiv:2403.10242
2024 arXiv
-
[10]
Jain, A.; Tancik, M.; and Abbeel, P. 2021. Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
2021
-
[11]
Jin, R.; Gao, Y.; Lu, H.; and Gao, F. 2024. GS-Planner: A Gaussian-Splatting-based Planning Framework for Active High-Fidelity Reconstruction. arXiv preprint arXiv:2405.10142
2024 arXiv
-
[12]
Kerbl, B.; Kopanas, G.; Leimk \"u hler, T.; and Drettakis, G. 2023. 3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Transactions on Graphics, 42(4)
2023
-
[13]
M.; Goldberg, K.; Kanazawa, A.; and Tancik, M
Kerr, J.; Kim, C. M.; Goldberg, K.; Kanazawa, A.; and Tancik, M. 2023. Lerf: Language embedded radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 19729--19739
2023
-
[14]
C.; Lo, W.-Y.; et al
Kirillov, A.; Mintun, E.; Ravi, N.; Mao, H.; Rolland, C.; Gustafson, L.; Xiao, T.; Whitehead, S.; Berg, A. C.; Lo, W.-Y.; et al. 2023. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 4015--4026
2023
-
[15]
Kopanas, G.; Leimkühler, T.; Rainer, G.; Jambon, C.; and Drettakis, G. 2022. Neural Point Catacaustics for Novel-View Synthesis of Reflections. ACM Transactions on Graphics, 1–15
2022
-
[16]
Kopanas, G.; Philip, J.; Leimkühler, T.; and Drettakis, G. 2021. Point‐Based Neural Rendering with Per‐View Optimization. Computer Graphics Forum, 29–43
2021
-
[17]
Li, H.; Zhang, D.; Dai, Y.; Liu, N.; Cheng, L.; Li, J.; Wang, J.; and Han, J. 2024 a . GP-NeRF: Generalized Perception NeRF for Context-Aware 3D Scene Understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 21708--21718
2024
-
[18]
Li, J.; Zhang, J.; Bai, X.; Zheng, J.; Ning, X.; Zhou, J.; and Gu, L. 2024 b . Dngaussian: Optimizing sparse-view 3d gaussian radiance fields with global-local depth normalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 20775--20785
2024
-
[19]
Li, X.; Wang, H.; and Tseng, K.-K. 2023. Gaussiandiffusion: 3d gaussian splatting for denoising diffusion probabilistic models with structured noise. arXiv preprint arXiv:2311.11221
2023 arXiv
-
[20]
Lu, G.; Zhang, S.; Wang, Z.; Liu, C.; Lu, J.; and Tang, Y. 2024 a . Manigaussian: Dynamic gaussian splatting for multi-task robotic manipulation. arXiv preprint arXiv:2403.08321
2024 arXiv
-
[21]
Lu, T.; Yu, M.; Xu, L.; Xiangli, Y.; Wang, L.; Lin, D.; and Dai, B. 2024 b . Scaffold-gs: Structured 3d gaussians for view-adaptive rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 20654--20664
2024
-
[22]
Malarz, D.; Smolak, W.; Tabor, J.; Tadeja, S.; and Spurek, P. 2023. Gaussian splatting with nerf-based color and opacity. arXiv preprint arXiv:2312.13729
2023 arXiv
-
[23]
Morgenstern, W.; Barthel, F.; Hilsmann, A.; and Eisert, P. 2025. Compact 3d scene representation via self-organizing gaussian grids. In European Conference on Computer Vision, 18--34. Springer
2025
-
[24]
T.; Mildenhall, B.; Sajjadi, M
Niemeyer, M.; Barron, J. T.; Mildenhall, B.; Sajjadi, M. S.; Geiger, A.; and Radwan, N. 2022. Regnerf: Regularizing neural radiance fields for view synthesis from sparse inputs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5480--5490
2022
-
[25]
Ranftl, R.; Lasinger, K.; Hafner, D.; Schindler, K.; and Koltun, V. 2020. Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
2020
-
[26]
L.; and Frahm, J.-M
Schonberger, J. L.; and Frahm, J.-M. 2016. Structure-from-motion revisited. In Proceedings of the IEEE conference on computer vision and pattern recognition, 4104--4113
2016
-
[27]
W.; Pollefeys, M.; Tombari, F.; and Engelmann, F
Takmaz, A.; Fedele, E.; Sumner, R. W.; Pollefeys, M.; Tombari, F.; and Engelmann, F. 2023. Openmask3d: Open-vocabulary 3d instance segmentation. arXiv preprint arXiv:2306.13631
2023 arXiv
-
[28]
Tang, J.; Ren, J.; Zhou, H.; Liu, Z.; and Zeng, G. 2023. Dreamgaussian: Generative gaussian splatting for efficient 3d content creation. arXiv preprint arXiv:2309.16653
2023 arXiv
-
[29]
Wang, C.-Y.; Yeh, I.-H.; and Liao, H.-Y. M. 2024. Yolov9: Learning what you want to learn using programmable gradient information. arXiv preprint arXiv:2402.13616
2024 arXiv
-
[30]
C.; and Liu, Z
Wang, G.; Chen, Z.; Loy, C. C.; and Liu, Z. 2023. Sparsenerf: Distilling depth ranking for few-shot novel view synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 9065--9076
2023
-
[31]
Wang, Y.; Zhao, Y.; and Petzold, L. 2024. An empirical study on the robustness of the segment anything model (sam). Pattern Recognition, 110685
2024
-
[32]
Xiong, H.; Muttukuru, S.; Upadhyay, R.; Chari, P.; and Kadambi, A. 2023. Sparsegs: Real-time 360 \ deg \ sparse view synthesis using gaussian splatting. arXiv preprint arXiv:2312.00206
2023 arXiv
-
[33]
Ye, M.; Danelljan, M.; Yu, F.; and Ke, L. 2025. Gaussian grouping: Segment and edit anything in 3d scenes. In European Conference on Computer Vision, 162--179. Springer
2025
-
[34]
Yu, A.; Ye, V.; Tancik, M.; and Kanazawa, A. 2021 a . pixelNeRF: Neural Radiance Fields from One or Few Images. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
2021
-
[35]
Yu, A.; Ye, V.; Tancik, M.; and Kanazawa, A. 2021 b . pixelnerf: Neural radiance fields from one or few images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 4578--4587
2021
-
[36]
Yu, Z.; Chen, A.; Huang, B.; Sattler, T.; and Geiger, A. 2024. Mip-splatting: Alias-free 3d gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19447--19456
2024
-
[37]
U.; Bae, S.-H.; Lee, S.; and Hong, C
Zhang, C.; Han, D.; Qiao, Y.; Kim, J. U.; Bae, S.-H.; Lee, S.; and Hong, C. S. 2023. Faster segment anything: Towards lightweight sam for mobile applications. arXiv preprint arXiv:2306.14289
2023 arXiv
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.