Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

ObjectGS: Object-aware Scene Reconstruction and Scene Understanding via Gaussian Splatting

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Objects modeled as anchors with fixed one-hot IDs improve 3D Gaussian segmentation.

desk verdict A clever, well-engineered pipeline with a genuine spec gap: the paper never says how background (ID 0) is handled, so the panoptic numbers are not yet trustworthy. read the letter →

arxiv 2507.15454 v1 pith:IUPIUUPG submitted 2025-07-21 cs.GR cs.AIcs.CVcs.HC

Ruijie Zhu , Mulin Yu , Linning Xu , Lihan Jiang , Yixuan Li , Tianzhu Zhang , Jiangmiao Pang , Bo Dai This is my paper
classification cs.GRcs.AIcs.CVcs.HC
keywords 3DGaussiansplattingobject-awarereconstructioninstancesegmentationopen-vocabularypanopticone-hotIDencodingsceneeditingmeshextraction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a way to make 3D Gaussian Splatting understand objects while it reconstructs a scene, rather than treating geometry and semantics as separate steps. The method assigns each object a fixed one-hot ID, binds that ID to anchor points that spawn neural Gaussians, and supervises rendering with a classification loss. The authors claim this avoids the semantic blur that continuous learnable features suffer during alpha blending, and that it beats current open-vocabulary and panoptic segmentation methods on standard benchmarks. A reader should care because the same object-aware representation directly supports downstream uses such as pulling out an object's mesh and editing a scene by removing or recoloring a selected object.

What carries the argument

The load-bearing mechanism is the pairing of object-aware anchors with a fixed one-hot ID encoding. Anchors are initialized from the object-voted point cloud, each carrying an object ID; they grow and prune in the voxel grid while replicating or removing their ID, and each anchor spawns several neural Gaussians that inherit the ID. The Gaussian's semantic attribute is a one-hot vector whose length equals the number of objects, so alpha blending of these vectors yields a classification probability vector at each pixel; taking the argmax predicts the pixel's object ID, and a cross-entropy loss enforces the label. A variable-length feature rasterizer extends the Gaussian color attribute to carry the ID channels, making the discrete semantic rendering efficient and scene-level, so all objects are rendered in one pass and occluded objects are handled.

What would settle it

Take a scene with known object IDs, progressively corrupt a growing fraction of views' 2D masks by swapping two adjacent objects' IDs, and measure the final 3D segmentation IoU. If the cross-entropy loss simply locks in the swapped supervision, the IoU curve will fall with corruption; if the claimed geometry-guided correction works, the curve will stay high until some corruption threshold.

Watch

Extended reading notes

Core claim

The paper's central claim is that object-level 3D understanding can be built directly into Gaussian Splatting reconstruction by making each object a set of anchors, each anchor carrying a fixed object ID, and by representing Gaussian semantics as one-hot ID vectors rather than learnable continuous features. During alpha blending these vectors accumulate independently, so the rendered pixel vector is a true classification distribution over object IDs; a cross-entropy loss against ID-consistent 2D masks then supervises both geometry and semantics at once. ObjectGS claims this design resolves the semantic ambiguity that arises when continuous learned features are blended, and the experiments report gains over prior methods on open-vocabulary segmentation and panoptic segmentation, plus cleaner 3D instance point clouds and direct mesh extraction and scene editing.

Load-bearing premise

The whole pipeline assumes the 2D segmentation model produces accurate, ID-consistent masks across views; if the masks are wrong or inconsistent, the voted object IDs on the point cloud are wrong, and the cross-entropy loss will strengthen those wrong IDs rather than fix them.

Editorial extensions

If this is right

  • Objects can be selected by their ID directly after training: removing an object is deleting its anchors, and recoloring is editing the associated Gaussian color attributes.
  • Because semantic rendering runs once at scene level, occluded and partially invisible objects remain queryable, which per-object rendering cannot do.
  • The 2DGS variant of the framework exports individual object meshes via TSDF fusion using the anchors of the selected ID.
  • Segmentation and reconstruction improve together: adding the semantic loss also improves PSNR, SSIM, and LPIPS in the reported ablations.
  • The one-hot formulation needs no post-processing or classifier call at query time; text or click prompts are resolved at mask-extraction time and map directly to IDs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The discrete-semantics principle likely transfers to other neural rendering backbones: any continuous feature field used for classification could be replaced by a one-hot head with cross-entropy, at the cost of needing ID-consistent training masks.
  • The method's ceiling is set by the 2D mask source; a natural extension is to measure robustness under controlled mask corruption, and to let the geometry branch actively re-label uncertain points instead of merely inheriting votes.
  • Because the ID encoding length equals the object count, scenes with long-tailed or hierarchical categories could use coarse-to-fine one-hot groups or per-category subset encoding to keep memory flat, something the paper hints at in its overhead discussion.
  • The current open-vocabulary ability is realized at label-extraction time; a further step would attach text embeddings to each ID so that arbitrary queries can be resolved at render time without running the segmentation model again.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. ObjectGS proposes a 3D Gaussian Splatting framework that couples object-level reconstruction with instance segmentation. Starting from DEVA/Grounded-SAM 2D masks, it lifts object IDs to the COLMAP point cloud by majority, probability-based, or correspondence-based voting, initializes Scaffold-GS anchors with object IDs, and grows or prunes anchors while inheriting IDs. Each generated Gaussian carries a fixed one-hot ID encoding; alpha-blended rendering of these encodings gives per-pixel classification probabilities, which are supervised by a cross-entropy loss. The paper reports open-vocabulary segmentation on LERF-Mask and 3DOVS, panoptic segmentation on Replica and ScanNet++, and applications in mesh extraction and scene editing.

Significance. The central design hypothesis—that discrete one-hot ID semantics avoid the alpha-blending ambiguity of learnable continuous semantic features—is well motivated and supported by the ablation in Table 5, where one-hot encoding (88.19 mIoU) clearly outperforms learnable semantics (69.57 mIoU) and object-independent constraints (37.48 mIoU) on the Figurines scene. The object-anchor coupling also yields clean 3D instance labels without per-Gaussian feature distillation, and the variable-length feature rasterizer is a practical engineering contribution. If the open questions about background handling and evaluation protocol are resolved, this is a useful step for object-aware 3DGS.

major comments (4)
  1. [§3.3(c), Eq. (4)–(7); Table 2] The background class is not representable in the model as specified. Section 3.1 assigns unclassified pixels ID 0, but Eq. (4) defines one-hot vectors only for IDs 1..n, Eq. (7) has no loss term for ID'(x)=0, and Eq. (6) takes argmax over the n object-only probabilities. A background pixel therefore receives the ID of the most probable object (or an undefined argmax where no Gaussian covers the pixel). Since Table 2 reports panoptic IoU/Dice/Acc on Replica and ScanNet++, which contain large background regions, the high scores imply either that ID 0 is represented somewhere outside the described equations or that background pixels are excluded from the metrics. Please specify how background is predicted (a background anchor, a confidence threshold, or exclusion), and state clearly whether the reported metrics include ID 0. If ID 0 pixels are excluded, the task should be described as instance-level rather than panoptic segmentation.
  2. [§4.2, Table 1] The claim of 'significantly outperforming' state-of-the-art on LERF-Mask is not uniform: on the Figurines scene, Gaga achieves mIoU 90.7/mBIoU 89.0 while ObjectGS achieves 88.2/85.2. The paper reports no average or variance across scenes, so the statement in Section 4.2 overstates the evidence. Please report per-scene means with standard deviations, or rephrase the claim to identify the scenes where the improvement holds.
  3. [§4.3, Table 6; §5 Limitation] The robustness argument that the grow-and-prune mechanism 'can naturally correct some mislabeled points' is supported only qualitatively. Table 6 compares three voting strategies on the same clean DEVA masks, but it does not inject label noise into the initialization to measure correction. Given that the whole pipeline inherits object IDs from the initial point-cloud voting, a quantitative robustness test (e.g., randomly flipping a controlled fraction of initial point IDs and measuring final mIoU) would substantiate the claim; otherwise the limitation statement in Section 5 should be strengthened.
  4. [§4.1–4.2, Tables 2 and 3] The ScanNet++ evaluation protocol is underspecified: the paper says 'randomly select 7 scenes' without naming them or reporting a seed, and Table 3 does not define how precision, recall, and F1 are aggregated over instances or whether the Chamfer distance is computed per predicted instance or per ground-truth instance. This information is needed to reproduce the reported numbers and to interpret the large recall gap (66.5% vs. 86.1%) in Table 3.
minor comments (6)
  1. [Fig. 4 caption] The caption mentions 'Grounded SAM 2' but the text and references use Grounded-SAM [37]; please align the terminology.
  2. [§3.1 and Supplementary §8] The headings 'Object ID V oting' and 'V oting Algorithm' contain a typo ('V oting' should be 'Voting').
  3. [Supplementary, Algorithm 3] Algorithm 3 is structurally identical to Algorithm 1 except for renaming 'Camera poses' to 'Correspondences'; the pseudocode should show how correspondences actually replace projection.
  4. [§4.2, Table 2 and Table 8] The scene names '1ada', '3e8b', and '0d2e' in Table 8 are not defined, and the table heading alternates between 'Scannet++' and 'ScanNet++'; please provide the full list of the seven ScanNet++ scenes used and unify the naming.
  5. [§4.1 Implementation Details] The text says Gaussian color attributes are extended from 3 to N+3, but Eq. (5) suggests the ID encoding is rendered as a separate channel; please clarify how the N-dimensional semantic vector is split from the color attributes during rasterization.
  6. [Eq. (7)] The indicator notation 1[ID'(x)=i] is used without definition; please define the indicator function explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular dependency: the semantic supervision is an input from external 2D segmentation models, and the claimed outputs are benchmarked against external data and compared with the labeler itself.

full rationale

ObjectGS is an empirical architecture rather than a formal derivation, and its central claims are validated on external benchmarks. The object IDs in Eq. 4 and the cross-entropy loss in Eq. 7 are supervised by pseudo-labels from DEVA or Grounded-SAM (Sec. 3.1), which are training inputs, not outputs of the method. The predicted ID in Eq. 6 is an alpha-blended composition of fixed one-hot encodings, so inference is not defined in terms of the benchmark metric. The method also does more than replay its labels: Tables 1 and 4 show ObjectGS outperforms the DEVA labeler itself, and the Limitation section explicitly states that the approach is not merely a direct fitting of the 2D segmentation results, citing fewer holes and more 3D-consistent results than the ground truth. Self-citations to Scaffold-GS [27] provide the anchor backbone but are external, benchmarked prior work and are not used to forbid alternatives or to define the semantic contribution. One specification gap exists: Eqs. 4-7 ignore background ID 0 while Eq. 6 forces an argmax over object IDs, which could affect the reported panoptic metrics; however, this is an evaluation-consistency concern rather than circularity, because the reported numbers do not reduce to the training labels by construction. No circular step is present, so the score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The method is empirical and builds on existing frameworks. The free parameters are standard loss weights and architecture constants, tuned per dataset. The main axioms are the reliability of the 2D segmentation pseudo-labels and the validity of the Scaffold-GS anchor mechanism. No new physical entities are introduced; the object-ID-bound anchor is an architectural construct.

free parameters (4)
  • lambda_semantic = 0.01 on LERF-Mask, 0.1 on other datasets
    Per-dataset loss weight for the semantic cross-entropy term; chosen by the authors and affects the balance between reconstruction and segmentation.
  • lambda_vol = 0.0002 (3DOVS), 0.00005 (LERF-Mask), 0.00002 (Replica, ScanNet)
    Per-dataset volume regularization weight inherited from Scaffold-GS; tuned per dataset.
  • k (Gaussians per anchor) = 10
    Number of Gaussian primitives generated per anchor, following Scaffold-GS configuration.
  • N (number of objects per scene) = scene-dependent
    Determined by the 2D segmentation pipeline; sets the length of the one-hot ID encoding and the semantic channel dimension.
assumptions (4)
  • domain assumption DEVA/Grounded-SAM provide reliable, ID-consistent 2D object masks across views
    Invoked in Sec. 3.1 for object ID labeling and voting. The Limitation section acknowledges that if 2D segmentation fails, the method may fail.
  • domain assumption COLMAP point cloud and camera poses are sufficiently accurate for point-to-pixel voting
    Sec. 3.1 uses projections of the point cloud through camera poses to assign object IDs; inaccurate geometry or poses would mislabel points.
  • domain assumption Scaffold-GS anchor grow/prune mechanism is a valid base representation
    Sec. 3.2 builds object-aware anchors on top of Scaffold-GS; the method inherits the assumptions of that framework.
  • standard math Cross-entropy on one-hot rendered probabilities is a suitable segmentation training objective
    Sec. 3.3 defines the semantic loss; this is a standard supervised classification objective.
invented entities (1)
  • Object-ID-bound anchor independent evidence
    purpose: Serves as the minimal modeling unit carrying an object identity, generating Gaussians that share that identity, enabling object-level reconstruction and discrete semantic rendering.
    The anchor's object ID is directly observable through the rendered segmentation, and the paper's benchmark results provide a falsifiable test of the design. This is an architectural construct, not a physical entity, but it functions as a new representational element.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ObjectGS: Object-aware Scene Reconstruction and Scene Understanding via Gaussian Splatting." pith.science (2026). https://pith.science/paper/IUPIUUPG

@misc{pith2026250715454,
  author       = {Pith},
  title        = {Pith review of: ObjectGS: Object-aware Scene Reconstruction and Scene Understanding via Gaussian Splatting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IUPIUUPG}},
  note         = {Machine review of arXiv:2507.15454}
}
read the original abstract

3D Gaussian Splatting is renowned for its high-fidelity reconstructions and real-time novel view synthesis, yet its lack of semantic understanding limits object-level perception. In this work, we propose ObjectGS, an object-aware framework that unifies 3D scene reconstruction with semantic understanding. Instead of treating the scene as a unified whole, ObjectGS models individual objects as local anchors that generate neural Gaussians and share object IDs, enabling precise object-level reconstruction. During training, we dynamically grow or prune these anchors and optimize their features, while a one-hot ID encoding with a classification loss enforces clear semantic constraints. We show through extensive experiments that ObjectGS not only outperforms state-of-the-art methods on open-vocabulary and panoptic segmentation tasks, but also integrates seamlessly with applications like mesh extraction and scene editing. Project page: https://ruijiezhu94.github.io/ObjectGS_page

Figures

Figures reproduced from arXiv: 2507.15454 by the authors.

Figure 1
Figure 1. In the open-world setting, ObjectGS enables 3D ob [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (a) Considering semantic information during reconstruc [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The overall architecture of ObjectGS. We first use a 2D segmentation pipeline to assign object ID and lift it to 3D. Then we [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Since 2D segmentation method [37] don’t account for occluded object, it cannot be used to supervise the independent rendering of objects. In contrast, our ObjectGS render semantics in the scene level, which is occlusion-aware. 3.3. Discrete Gaussian Semantic Modeling I…
Figure 5
Figure 5. Figure 5: Qualitative comparison of open-vocabulary segmentation and 3D object queries. The red box highlights that our method can [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison of panoptic segmentation. We visualize the segmentation of anchors (ours) and Gaussians (Gaussian [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Qualitative comparison of different semantic modeling methods on 3D object query. Learnable Gaussian semantics leads to [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Ablation on different point cloud label initializations. [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Qualitative comparison of open vocabulary segmentation and 3D object query on the 3DOVS dataset. [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Qualitative comparison of open vocabulary segmentation and 3D object query on the LERF-Mask dataset. [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Qualitative comparison of 3D panoptic segmentation on the Replica dataset. [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Qualitative comparison of 3D panoptic segmentation on the Scannet++ dataset. [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 13
Figure 13. Figure 13: Qualitative comparison of 2D panoptic segmentation on the Replica dataset. [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]
Figure 14
Figure 14. Figure 14: Qualitative comparison of 2D panoptic segmentation on the Scannet++ dataset. [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Embodied Operators and Benchmarking: Toward Reusable and Deployable Embodied Intelligence Systems

    cs.AI 2026-07 conditional novelty 5.0 of 10

    Embodied operators—deployable modules with task semantics and I/O contracts—should be the unit of optimization and multi-dimensional benchmarking for reusable robot intelligence systems.

Reference graph

Works this paper leans on

60 extracted references · 39 canonical work pages · cited by 1 Pith paper

  1. [1]

    Mip-nerf: A multiscale representation for anti-aliasing neu- ral radiance fields

    Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neu- ral radiance fields. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 5855–5864,

  2. [2]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021. 2

    2021

  3. [3]

    Seg- ment anything in 3d with nerfs

    Jiazhong Cen, Zanwei Zhou, Jiemin Fang, Wei Shen, Lingxi Xie, Dongsheng Jiang, Xiaopeng Zhang, Qi Tian, et al. Seg- ment anything in 3d with nerfs. Proceedings of the Inter- national Conference on Neural Information Processing Sys- tems, 36:25971–25990, 2023. 2, 5

    work page 2023

  4. [4]

    Segment any 3d gaussians

    Jiazhong Cen, Jiemin Fang, Chen Yang, Lingxi Xie, Xi- aopeng Zhang, Wei Shen, and Qi Tian. Segment any 3d gaussians. In Proceedings of the AAAI Conference on Ar- tificial Intelligence, pages 1971–1979, 2025. 1, 2, 4, 6, 7

    work page 1971

  5. [5]

    Lifting by gaussians: A simple, fast and flexible method for 3d instance segmentation

    Rohan Chacko, Nicolai Haeni, Eldar Khaliullin, Lin Sun, and Douglas Lee. Lifting by gaussians: A simple, fast and flexible method for 3d instance segmentation. arXiv preprint arXiv:2502.00173, 2025. 4, 6

    arXiv 2025

  6. [6]

    Masked-attention mask transformer for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexan- der Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1290–1299, 2022. 2

    work page 2022

  7. [7]

    Tracking anything with decoupled video segmentation

    Ho Kei Cheng, Seoung Wug Oh, Brian Price, Alexan- der Schwing, and Joon-Young Lee. Tracking anything with decoupled video segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 1316–1326, 2023. 2, 3, 5, 8

    work page 2023

  8. [8]

    Click-gaussian: Interactive segmenta- tion to any 3d gaussians

    Seokhun Choi, Hyeonseop Song, Jaechul Kim, Taehyeong Kim, and Hoseok Do. Click-gaussian: Interactive segmenta- tion to any 3d gaussians. In European Conference on Com- puter Vision, pages 289–305. Springer, 2024. 1, 2, 7

    work page 2024

Show all 60 references
  1. [9]

    Interactive segmentation of radiance fields

    Rahul Goel, Dhawal Sirikonda, Saurabh Saini, and PJ Narayanan. Interactive segmentation of radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 4201–4211, 2023. 2

    2023

  2. [10]

    Neural 3d scene reconstruction with the manhattan-world assumption

    Haoyu Guo, Sida Peng, Haotong Lin, Qianqian Wang, Guofeng Zhang, Hujun Bao, and Xiaowei Zhou. Neural 3d scene reconstruction with the manhattan-world assumption. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 5511–5520, 2022. 5

    2022

  3. [11]

    Sam-guided graph cut for 3d instance segmentation

    Haoyu Guo, He Zhu, Sida Peng, Yuang Wang, Yujun Shen, Ruizhen Hu, and Xiaowei Zhou. Sam-guided graph cut for 3d instance segmentation. In European Conference on Com- puter Vision, pages 234–251. Springer, 2024. 3

    2024

  4. [12]

    2d gaussian splatting for geometrically accu- rate radiance fields

    Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically accu- rate radiance fields. In ACM SIGGRAPH Conference, pages 1–11, 2024. 2, 8

    2024

  5. [13]

    Vschh 2023: A benchmark for the view synthesis challenge of hu- man heads

    Youngkyoon Jang, Jiali Zheng, Jifei Song, Helisa Dhamo, Eduardo P ´erez-Pellitero, Thomas Tanay, Matteo Maggioni, Richard Shaw, Sibi Catley-Chandar, Yiren Zhou, et al. Vschh 2023: A benchmark for the view synthesis challenge of hu- man heads. In Proceedings of the IEEE/CVF Int...

    2023

  6. [14]

    Anysplat: Feed-forward 3d gaussian splatting from unconstrained views.arXiv preprint arXiv:2505.23716,

    Lihan Jiang, Yucheng Mao, Linning Xu, Tao Lu, Kerui Ren, Yichen Jin, Xudong Xu, Mulin Yu, Jiangmiao Pang, Feng Zhao, et al. Anysplat: Feed-forward 3d gaussian splatting from unconstrained views.arXiv preprint arXiv:2505.23716,

  7. [15]

    Horizon- gs: Unified 3d gaussian splatting for large-scale aerial-to- ground scenes

    Lihan Jiang, Kerui Ren, Mulin Yu, Linning Xu, Junting Dong, Tao Lu, Feng Zhao, Dahua Lin, and Bo Dai. Horizon- gs: Unified 3d gaussian splatting for large-scale aerial-to- ground scenes. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 26789–2679...

    2025

  8. [16]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42 (4):139–1, 2023. 1, 2

    2023

  9. [17]

    Lerf: Language embedded radiance fields

    Justin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa, and Matthew Tancik. Lerf: Language embedded radiance fields. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 19729–19739,

  10. [18]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 1, 2

    2023

  11. [19]

    Decomposing nerf for editing via feature field distilla- tion

    Sosuke Kobayashi, Eiichi Matsumoto, and Vincent Sitz- mann. Decomposing nerf for editing via feature field distilla- tion. Proceedings of the International Conference on Neural Information Processing Systems, 35:23311–23330, 2022. 2

    2022

  12. [20]

    The robodepth challenge: Methods and advancements towards robust depth estimation

    Lingdong Kong, Yaru Niu, Shaoyuan Xie, Hanjiang Hu, Lai Xing Ng, Benoit R Cottereau, Liangjun Zhang, Hesh- eng Wang, Wei Tsang Ooi, Ruijie Zhu, et al. The robodepth challenge: Methods and advancements towards robust depth estimation. arXiv preprint arXiv:2307.15061, 2023. 2

    2023 arXiv

  13. [21]

    Language-driven semantic seg- mentation

    Boyi Li, Kilian Q Weinberger, Serge Belongie, Vladlen Koltun, and Ren ´e Ranftl. Language-driven semantic seg- mentation. arXiv preprint arXiv:2201.03546, 2022. 6

    2022 arXiv

  14. [22]

    Open-vocabulary semantic segmentation with mask-adapted clip

    Feng Liang, Bichen Wu, Xiaoliang Dai, Kunpeng Li, Yinan Zhao, Hang Zhang, Peizhao Zhang, Peter Vajda, and Diana Marculescu. Open-vocabulary semantic segmentation with mask-adapted clip. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pa...

    2023

  15. [23]

    Su- pergseg: Open-vocabulary 3d segmentation with structured super-gaussians

    Siyun Liang, Sen Wang, Kunyi Li, Michael Niemeyer, Ste- fano Gasperini, Nassir Navab, and Federico Tombari. Su- pergseg: Open-vocabulary 3d segmentation with structured super-gaussians. arXiv preprint arXiv:2412.10231, 2024. 1, 2, 4

    2024

  16. [24]

    Weakly supervised 3d open- vocabulary segmentation

    Kunhao Liu, Fangneng Zhan, Jiahui Zhang, Muyu Xu, Yingchen Yu, Abdulmotaleb El Saddik, Christian Theobalt, Eric Xing, and Shijian Lu. Weakly supervised 3d open- vocabulary segmentation. Advances in Neural Information Processing Systems, 36:53433–53456, 2023. 5, 6

    2023

  17. [25]

    Plane2depth: Hierarchical adap- tive plane guidance for monocular depth estimation

    Li Liu, Ruijie Zhu, Jiacheng Deng, Ziyang Song, Wenfei Yang, and Tianzhu Zhang. Plane2depth: Hierarchical adap- tive plane guidance for monocular depth estimation. IEEE Transactions on Circuits and Systems for Video Technology,

  18. [26]

    Dn-4dgs: Denoised de- formable network with temporal-spatial aggregation for dy- namic scene rendering

    Jiahao Lu, Jiacheng Deng, Ruijie Zhu, Yanzhe Liang, Wenfei Yang, Xu Zhou, and Tianzhu Zhang. Dn-4dgs: Denoised de- formable network with temporal-spatial aggregation for dy- namic scene rendering. Advances in Neural Information Pro- cessing Systems, 37:84114–84138, 2024. 2

    2024

  19. [27]

    Scaffold-gs: Structured 3d gaussians for view-adaptive rendering

    Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua Lin, and Bo Dai. Scaffold-gs: Structured 3d gaussians for view-adaptive rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20654–20664, 2024. 1, 2, 3, 5

    2024

  20. [28]

    Gaga: Group any gaussians via 3d-aware memory bank

    Weijie Lyu, Xueting Li, Abhijit Kundu, Yi-Hsuan Tsai, and Ming-Hsuan Yang. Gaga: Group any gaussians via 3d-aware memory bank. arXiv preprint arXiv:2404.07977, 2024. 2, 5

    2024 arXiv

  21. [29]

    NeRF: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing scenes as neural radiance fields for view syn- thesis. In Proceedings of the European Conference on Com- puter Vision, pages 405–421, 2020. 1, 2

    2020

  22. [30]

    Spin-nerf: Multiview segmentation and perceptual inpainting with neural radiance fields

    Ashkan Mirzaei, Tristan Aumentado-Armstrong, Konstanti- nos G Derpanis, Jonathan Kelly, Marcus A Brubaker, Igor Gilitschenski, and Alex Levinshtein. Spin-nerf: Multiview segmentation and perceptual inpainting with neural radiance fields. In Proceedings of the IEEE/CVF Conferen...

  23. [31]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 2

    2023 arXiv

  24. [32]

    Gags: Granularity-aware feature distillation for language gaussian splatting

    Yuning Peng, Haiping Wang, Yuan Liu, Chenglu Wen, Zhen Dong, and Bisheng Yang. Gags: Granularity-aware feature distillation for language gaussian splatting. arXiv preprint arXiv:2412.13654, 2024. 2, 4

    2024 arXiv

  25. [33]

    Langsplat: 3d language gaussian splatting

    Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister. Langsplat: 3d language gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20051–20060, 2024. 1, 2, 4, 5, 6

    2024

  26. [34]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In Proceedings of the International Conference ...

    2021

  27. [35]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714, 2024. 2

    2024 arXiv

  28. [36]

    Octree-gs: Towards consistent real-time rendering with lod-structured 3d gaussians

    Kerui Ren, Lihan Jiang, Tao Lu, Mulin Yu, Linning Xu, Zhangkai Ni, and Bo Dai. Octree-gs: Towards consistent real-time rendering with lod-structured 3d gaussians. arXiv preprint arXiv:2403.17898, 2024. 2

    2024 arXiv

  29. [37]

    Grounded sam: Assembling open-world models for diverse visual tasks

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. Grounded sam: Assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14159 ,

    arXiv

  30. [38]

    Depthmaster: Taming diffusion models for monocular depth estimation

    Ziyang Song, Zerong Wang, Bo Li, Hao Zhang, Ruijie Zhu, Li Liu, Peng-Tao Jiang, and Tianzhu Zhang. Depthmaster: Taming diffusion models for monocular depth estimation. arXiv preprint arXiv:2501.02576, 2025. 2

    2025 arXiv

  31. [39]

    The replica dataset: A digital replica of indoor spaces

    Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, et al. The replica dataset: A digital replica of indoor spaces. arXiv preprint arXiv:1906.05797,

    1906 arXiv

  32. [40]

    Open- mask3d: open-vocabulary 3d instance segmentation

    Ayc ¸a Takmaz, Elisabetta Fedele, Robert W Sumner, Marc Pollefeys, Federico Tombari, and Francis Engelmann. Open- mask3d: open-vocabulary 3d instance segmentation. In Pro- ceedings of the International Conference on Neural Informa- tion Processing Systems, pages 68367–68390, 2023. 2

    2023

  33. [41]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 2

    2017

  34. [42]

    Ref-nerf: Struc- tured view-dependent appearance for neural radiance fields

    Dor Verbin, Peter Hedman, Ben Mildenhall, Todd Zickler, Jonathan T Barron, and Pratul P Srinivasan. Ref-nerf: Struc- tured view-dependent appearance for neural radiance fields. In 2022 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 5481–5490. IE...

    2022

  35. [43]

    Object- compositional neural implicit surfaces

    Qianyi Wu, Xian Liu, Yuedong Chen, Kejie Li, Chuanxia Zheng, Jianfei Cai, and Jianmin Zheng. Object- compositional neural implicit surfaces. In European Con- ference on Computer Vision, pages 197–213. Springer, 2022. 5

    2022

  36. [44]

    Objectsdf++: Improved object-compositional neural implicit surfaces

    Qianyi Wu, Kaisiyuan Wang, Kejie Li, Jianmin Zheng, and Jianfei Cai. Objectsdf++: Improved object-compositional neural implicit surfaces. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 21764– 21774, 2023. 4

    2023

  37. [45]

    Opengaussian: Towards point-level 3d gaussian-based open vocabulary understanding

    Yanmin Wu, Jiarui Meng, Haijie Li, Chenming Wu, Yahao Shi, Xinhua Cheng, Chen Zhao, Haocheng Feng, Errui Ding, Jingdong Wang, et al. Opengaussian: Towards point-level 3d gaussian-based open vocabulary understanding. arXiv preprint arXiv:2406.02058, 2024. 1

    2024 arXiv

  38. [46]

    Sam3d: Segment anything in 3d scenes.arXiv preprint arXiv:2306.03908, 2023

    Yunhan Yang, Xiaoyang Wu, Tong He, Hengshuang Zhao, and Xihui Liu. Sam3d: Segment anything in 3d scenes.arXiv preprint arXiv:2306.03908, 2023. 2 10

    2023 arXiv

  39. [47]

    Gaussian grouping: Segment and edit anything in 3d scenes

    Mingqiao Ye, Martin Danelljan, Fisher Yu, and Lei Ke. Gaussian grouping: Segment and edit anything in 3d scenes. In Proceedings of the European Conference on Computer Vi- sion, pages 162–179, 2024. 1, 2, 3, 5, 6, 7, 8

    2024

  40. [48]

    gsplat: An open-source library for gaussian splatting

    Vickie Ye, Ruilong Li, Justin Kerr, Matias Turkulainen, Brent Yi, Zhuoyang Pan, Otto Seiskari, Jianbo Ye, Jeffrey Hu, Matthew Tancik, et al. gsplat: An open-source library for gaussian splatting. Journal of Machine Learning Research , 26(34):1–17, 2025. 1, 5

    2025

  41. [49]

    Scannet++: A high-fidelity dataset of 3d in- door scenes

    Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai. Scannet++: A high-fidelity dataset of 3d in- door scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12–22, 2023. 5

    2023

  42. [50]

    Omniseg3d: Omniversal 3d segmentation via hierarchical contrastive learning

    Haiyang Ying, Yixuan Yin, Jinzhi Zhang, Fan Wang, Tao Yu, Ruqi Huang, and Lu Fang. Omniseg3d: Omniversal 3d segmentation via hierarchical contrastive learning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20612–20622, 2024. 2

    2024

  43. [51]

    Gsdf: 3dgs meets sdf for improved neural ren- dering and reconstruction

    Mulin Yu, Tao Lu, Linning Xu, Lihan Jiang, Yuanbo Xiangli, and Bo Dai. Gsdf: 3dgs meets sdf for improved neural ren- dering and reconstruction. Advances in Neural Information Processing Systems, 37:129507–129530, 2024. 2

    2024

  44. [52]

    Mip-splatting: Alias-free 3d gaussian splat- ting

    Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. Mip-splatting: Alias-free 3d gaussian splat- ting. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 19447–19456,

  45. [53]

    Tricky 2024 challenge on monocular depth from images of specular and transparent surfaces

    Pierluigi Zama Ramirez, Alex Costanzino, Fabio Tosi, Mat- teo Poggi, Luigi Di Stefano, Jean-Baptiste Weibel, Dominik Bauer, Doris Antensteiner, Markus Vincze, Jiaqi Li, et al. Tricky 2024 challenge on monocular depth from images of specular and transparent surfaces. In Europea...

    2024

  46. [54]

    Tiface: Improving facial reconstruction through tensorial radiance fields and implicit surfaces

    Ruijie Zhu, Jiahao Chang, Ziyang Song, Jiahuan Yu, and Tianzhu Zhang. Tiface: Improving facial reconstruction through tensorial radiance fields and implicit surfaces. arXiv preprint arXiv:2312.09527, 2023

    2023 arXiv

  47. [55]

    Ha-bins: Hierarchical adap- tive bins for robust monocular depth estimation across multi- ple datasets

    Ruijie Zhu, Ziyang Song, Li Liu, Jianfeng He, Tianzhu Zhang, and Yongdong Zhang. Ha-bins: Hierarchical adap- tive bins for robust monocular depth estimation across multi- ple datasets. IEEE Transactions on Circuits and Systems for Video Technology, 34(6):4354–4366, 2023. 2

    2023

  48. [56]

    Motiongs: Exploring explicit motion guidance for deformable 3d gaussian splatting

    Ruijie Zhu, Yanzhe Liang, Hanzhi Chang, Jiacheng Deng, Jiahao Lu, Wenfei Yang, Tianzhu Zhang, and Yongdong Zhang. Motiongs: Exploring explicit motion guidance for deformable 3d gaussian splatting. Advances in Neural Infor- mation Processing Systems, 37:101790–101817, 2024. 2

    2024

  49. [57]

    Scaledepth: Decomposing metric depth estimation into scale prediction and relative depth estimation

    Ruijie Zhu, Chuxin Wang, Ziyang Song, Li Liu, Tianzhu Zhang, and Yongdong Zhang. Scaledepth: Decomposing metric depth estimation into scale prediction and relative depth estimation. arXiv preprint arXiv:2407.08187 , 2024. 2 11 ObjectGS: Object-aware Scene Reconstruction and Sc...

    2024

  50. [58]

    Even with about 100 in- stances, overhead remains minimal with efficient parallel rasterizer

    Training Overhead Table 8 compares training time, FPS, and GPU memory across different instance counts. Even with about 100 in- stances, overhead remains minimal with efficient parallel rasterizer. Notably, since our one-hot ID encoding is not learnable parameters, it will not...

  51. [59]

    V oting Algorithm We provide the pseudo code of Algorithms 1 to 3 to clearly demonstrate the proposed voting strategies

  52. [60]

    9 to 14, which includes visualization of OVS segmentation results, panoptic segmentation results, and 3D instance seg- mentation with point clouds

    More Visualization We provide more visualization results as shown in Figs. 9 to 14, which includes visualization of OVS segmentation results, panoptic segmentation results, and 3D instance seg- mentation with point clouds. Algorithm 1 Object ID Majority V oting 1: Input: 2: Po...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.