Pith. sign in

REVIEW 5 major objections 5 minor 2 cited by

Lifting by Gaussians: A Simple, Fast and Flexible Method for 3D Instance Segmentation

T0 review · 5 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Lifting by Gaussians segments any pretrained 3D Gaussian Splatting scene into objects, parts, and subparts in minutes, with no per-scene training.

desk verdict LBG is a credible training-free 3DGS segmentation method with real speed and clean assets; the abstract overstates the 2D NVS results and the lifting step needs an explicit conflict-resolution rule. read the letter →

arxiv 2502.00173 v1 pith:F6PO3ZES submitted 2025-01-31 cs.CV

Rohan Chacko , Nicolai Haeni , Eldar Khaliullin , Lin Sun , Douglas Lee This is my paper · ORCID
classification cs.CV
keywords 3DinstancesegmentationGaussianSplattingmaskliftingSAMCLIPDINOv2open-worldassetextraction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces Lifting by Gaussians (LBG), a method that takes a pretrained 3D Gaussian Splatting (3DGS) field—an explicit scene representation made of colored Gaussian primitives—and decomposes it into objects, parts, and subparts without any per-scene training. It runs a 2D foundation model (SAM) on posed RGB frames to get class-agnostic masks, extracts CLIP and DINOv2 features for each masked region, and assigns the mask to the single Gaussian with the largest alpha-blending weight at each pixel. Those per-frame 3D fragments are then merged incrementally across frames using geometric overlap and semantic feature similarity. The claimed payoff is that any existing 3DGS (or 2DGS) reconstruction can be segmented in minutes, roughly an order of magnitude faster than contrastive-learning methods, while yielding cleaner extracted 3D assets.

What carries the argument

The load-bearing object is the per-pixel maximum-contributor Gaussian, $i^* = \arg\max_i\bigl(\alpha'_i \prod_{j=1}^{i-1}(1-\alpha'_j)\bigr)$, the single splatted Gaussian whose $\alpha$ weight dominates a pixel's rendered color. LBG assigns the pixel's SAM mask ID and CLIP/DINOv2 feature vector to that Gaussian, producing per-frame 3D fragments; fragments are merged across frames using the geometric overlap ratio $\phi_{\mathrm{geom}}(i,j) = |G_i^t \cap G_j^{t-1}| / |G_i^t|$ together with normalized cosine feature similarity. This single-primitive assignment is what suppresses semantic bleeding. The pipeline also applies SAM's whole/part/subpart hierarchy and prunes the underlying field with a view-consistency-weighted version of Mini-Splatting, a compressed 3DGS variant with far fewer Gaussians, to remove single-view floaters.

What would settle it

Render a synthetic scene with known ground truth in which one object is covered by several overlapping Gaussians of similar opacity, and orbit the camera around it. For each object pixel and each view, record the maximum-contributor Gaussian $i^* = \arg\max_i(\alpha'_i \prod_{j<i}(1-\alpha'_j))$; if the same ground-truth surface point is anchored by different Gaussians in different views, LBG's per-frame fragments for that object will contain disjoint Gaussian sets, and the merge will split the object unless the geometric overlap ratio stays high. The quantitative prediction is that LBG's 3D instance overlap score degrades as the cross-view argmax-switch rate increases.

Watch

Extended reading notes

Core claim

The central claim is that a high-quality open-world 3D instance segmentation can be produced purely by lifting 2D foundation-model outputs onto a pretrained Gaussian field, with no gradient-based optimization of the 3D semantics. For each pixel, LBG finds the Gaussian $i^*$ with the maximum $\alpha$-blending weight and assigns that pixel's 2D object ID and feature vector to the Gaussian; this single-primitive assignment keeps boundaries sharp and avoids the semantic bleeding of thresholded multi-Gaussian assignment. Per-frame fragments are merged greedily by Gaussian-set overlap and normalized cosine similarity of CLIP/DINOv2 features, and the process is repeated at SAM's part and subpart levels to yield a hierarchy. The paper reports that this training-free approach matches or beats learning-based baselines on 2D mask rendering and outperforms them on photorealistic 3D asset extraction, at a fraction of the runtime.

Load-bearing premise

The load-bearing premise is that the single Gaussian that contributes most to a pixel's rendered color is a stable, semantically meaningful 3D anchor: the same physical object must be tied to the same Gaussian or overlapping Gaussian set from every viewpoint, so identical 2D masks lift onto the same 3D fragment each frame; floaters, transparency, or view-dependent switches in that winning Gaussian would break the merge, and the paper itself modifies Mini-Splatting with a view-consistency score to suppress single-view floaters.

Editorial extensions

If this is right

  • Any existing 3DGS or 2DGS reconstruction can be segmented into objects, parts, and subparts without retraining or changing the field's parameterization.
  • Scene decomposition runs in minutes: roughly 450 seconds total on one RTX 3090, versus 3,900–5,200 seconds for the learning-based baselines, with most of the time spent in 2D model loading and feature extraction rather than 3D optimization.
  • Extracted 3D assets are cleaner: on the re-annotated LERF scenes, LBG's segmented objects render with higher PSNR and SSIM and lower LPIPS than Gaussian Grouping and SAGA.
  • The pipeline is modular: swapping SAM for a faster variant trades quality for speed, and the same lifting mechanism can deposit arbitrary 2D features, such as DINOv2 PCA maps, onto Gaussians.
  • Applied to 2DGS, LBG yields per-object meshes, so instance segmentation extends to mesh extraction with no code changes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, because LBG never optimizes the field, the same lifted-segmentation pipeline could run online while a 3DGS map is being built; the bottleneck would be the incremental merge's need to revisit accumulated fragments, not gradient training.
  • Beyond the paper, the method implies a measurable upper bound on segmentation quality: the per-view stability of the argmax-contributor Gaussian. A cheap pre-pass counting how many views each Gaussian anchors could predict where LBG will merge or fragment incorrectly and could serve as a confidence map for downstream editing.
  • Beyond the paper, LBG inherits SAM's scale inconsistencies (the paper itself notes the figurines container being merged with nearby objects because of inconsistent SAM masks). A direct next experiment, not reported in the paper, is to run LBG with two different mask sources and use fragment agreement across sources as a consistency signal.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces Lifting by Gaussians (LBG), a training-free 3D instance segmentation method for 3D Gaussian Splatting scenes. Given a pretrained Gaussian radiance field and posed RGB frames, LBG extracts 2D masks from SAM, lifts each pixel to the Gaussian with maximum alpha-blending weight, forms per-frame 3D object fragments, and greedily merges fragments across frames using geometric overlap and CLIP/DINOv2 feature similarity. The process is applied hierarchically to yield objects, parts, and subparts. The authors report an order-of-magnitude speedup over Gaussian Grouping and SAGA, strong results on their newly proposed 3D asset-fidelity benchmark, competitive 2D mask novel-view-synthesis results on LERF and 3D-OVS, and a qualitative demonstration on 2DGS.

Significance. If the central claims hold, LBG is a useful practical contribution: it performs open-vocabulary 3D instance segmentation without per-scene semantic optimization, is modular with respect to the Gaussian representation, and is substantially faster than optimization-based baselines. The proposed 3D asset-fidelity evaluation protocol is also a valuable idea, although it needs additional validation and detail. The reported ablations support the importance of CLIP/DINO features, post-processing, and the Mini-Splatting representation. However, the same-frame fragment ambiguity in the lifting step is a genuine gap in the current formulation, and the abstract overclaims superiority on 2D novel-view mask synthesis relative to the paper's own Table 3. These issues are fixable, but they are load-bearing for the paper's main claims.

major comments (5)
  1. [§3.2, 2D-to-3D Lifting] The pixel-wise rule i* = argmax_i(...) does not guarantee that a single Gaussian is assigned to at most one mask in a frame. A splat whose 2D footprint straddles a mask boundary can be the maximum alpha contributor for pixels in two different SAM masks, so the resulting fragment sets G_t_i and G_t_j need not be disjoint. When this happens, the geometric overlap ratio phi_geom(i,j) = |G_t_i ∩ G^{t-1}_j| / |G_t_i| is not a well-defined object-overlap measure, and the final object assignment depends on the order in which masks are processed within a frame. Because the incremental merge treats these fragments as atomic units, the paper should state and justify a conflict-resolution rule (for example, first-assignment-wins with a confidence tie-break, or a per-Gaussian voting step) and quantify how often conflicts occur in the reported scenes.
  2. [Abstract and Table 3] The abstract claims that LBG 'achieves superior semantic segmentation for 2D semantic novel view synthesis,' but the mIoU numbers in Table 3 do not support this. On the LERF scenes, SAGA-3DGS is higher on figurines (0.860 vs 0.822), ramen (0.803 vs 0.732), and teatime (0.874 vs 0.866); on 3D-OVS, SAGA has a higher average (96.0 vs 94.9) and a much higher sofa score (93.5 vs 87.4). The body text itself says the method is 'competitive,' which is the appropriate characterization. Please revise the abstract and any similar statements in Section 1 and the conclusion to match the reported numbers, or provide additional experiments that substantiate superiority.
  3. [§4.1 Implementation Details and Appendix F] The central claim that LBG 'operates seamlessly on any existing 3DGS reconstruction' is stronger than what the experiments establish. The quantitative results are obtained on an enhanced Mini-Splatting field: the authors add a view-consistency score and a log multiplier that prunes single-view Gaussians before applying LBG, and Table 4 shows that removing Mini-Splatting lowers mIoU from 0.822 to 0.781. There is no quantitative evaluation on an unmodified vanilla 3DGS field, on fields trained by other codebases, or on 2DGS (only a qualitative demonstration). Please either soften the claim to 'works with any Gaussian representation given a clean reconstruction' or add experiments on unmodified fields and report the view-consistency modification as part of the method parameters.
  4. [§3.2 Incremental Merging] The semantic similarity is stated as 'normalized cosine similarity' but the formula is phi_sem(f_t_i, f^{t-1}_j) = (f_t_i · f^{t-1}_j)/2. If the features are L2-normalized, cosine similarity is the dot product, not the dot product divided by 2; if the features are not normalized, dividing by 2 is not a normalization and changes the merge threshold in an undocumented way. Please specify the exact normalization of the CLIP and DINOv2 vectors, the final similarity combination rule, and the merge thresholds (including the log multiplier from Appendix F) so the merge procedure is reproducible.
  5. [§4.2 Photorealistic 3D Asset Extraction] The new 3D asset-fidelity protocol is a valuable contribution, but as described it may favor the proposed method. The ground-truth assets are created by manually selecting/removing Gaussians from the same Mini-Splatting representation that LBG segments, the paper does not specify which objects or how many are used in the subset, and the predicted-to-ground-truth matching is by minimum average MSE of rendered 2D masks over training views, a criterion that rewards clean mask renders rather than correct 3D structure. Please specify the full list of objects, the annotation procedure, and test whether the ranking in Table 2 is robust to the matching criterion and to the choice of background for the rendered object views. In addition, it is unclear whether Tables 2 and 3 report the SAGA-3DGS or SAGA-MS variant, since Section 4.1 describes two settings that train on different radiance fields.
minor comments (5)
  1. [Table 4] The header contains a typo: 'semgnetation' should be 'segmentation'.
  2. [Table 1 caption] The caption says '10x less time,' but the reported ratio is 3922.67/449.96 ≈ 8.7x for Gaussian Grouping and 5206.74/449.96 ≈ 11.6x for SAGA; please use a more precise phrasing such as 'roughly an order of magnitude.'
  3. [§3.2, notation] In the incremental merging paragraph, the existing object map is denoted O^{t-1}_j, which suggests a per-object index; it would be clearer to write O^{t-1} and denote individual objects by o^{t-1}_j.
  4. [§4.1, Dataset description] The LERF re-annotation procedure is described only briefly; please report the number of annotated salient objects per scene, the annotation tool, and the consistency of the re-annotations, since the 2D mask evaluation depends on these labels.
  5. [§5.1, Limitations] The limitations section mentions small-object failures and model loading times but does not mention the order-dependence of the greedy merge or the same-frame fragment ambiguity; adding these would make the limitations statement more complete.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: LBG's segmentation is a deterministic lift of external 2D foundation-model masks onto pretrained Gaussians, with no fitted parameter or self-citation chain that reduces the claim to its inputs.

full rationale

LBG's derivation chain is: SAM (or FastSAM) provides 2D masks; CLIP and DINOv2 provide semantic features; each pixel's mask is assigned to the Gaussian with maximum alpha-blending weight (Sec. 3.2); per-frame fragments are merged using geometric overlap and feature cosine similarity; the process is applied hierarchically. Every input to this chain is external: the 2D masks and features come from published foundation models, and the 3DGS field comes from a pretrained Mini-Splatting reconstruction. The 3D instance segmentation is therefore a deterministic function of external 2D inputs, not an output that was used to define or fit those inputs. No parameter is fitted to the benchmark before the reported segmentation is produced, so the 'prediction' is not forced by construction. The new 3D asset evaluation protocol does involve manually cleaned ground-truth fields and matching predicted objects to ground truth by minimum rendered-mask MSE over training views, but that is a benchmark design choice; it does not enter the segmentation algorithm and cannot make the derivation circular. The paper's own limitation notes and the reviewer's technical concern about the max-contributor Gaussian being shared by multiple masks describe a possible correctness or robustness issue (fragments may be non-disjoint and merging may be order-dependent), but not circularity, because the anchor is not defined in terms of the final segments and no self-citation is load-bearing. The citations to Mini-Splatting and to the foundation models are independent external support, not an unverified self-referential chain. Accordingly, the honest finding is no significant circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The method introduces no new theoretical entities. It rests on the standard alpha-compositing model, on the reliability of external foundation models, on unspecified merge thresholds, on an order-dependent greedy merge, and on a self-built, manually cleaned ground-truth protocol for 3D asset evaluation.

free parameters (2)
  • Merge similarity thresholds = not reported
    Section 3.2 requires deciding when no suitable merge match exists, based on geometric overlap ratio and normalized cosine similarity, but no numeric thresholds are given. Results are sensitive to these thresholds.
  • View-consistency log multiplier = not reported
    Appendix F adds a log multiplier on the number of views that see a Gaussian to prune single-view floaters, but the multiplier and pruning schedule are unspecified, and they materially affect the field quality used by LBG.
assumptions (4)
  • standard math The 3DGS alpha-compositing equation (Eq. 1) correctly models pixel color and per-pixel Gaussian contribution.
    The lifting step assigns each pixel to the argmax contributor in Eq. 1; if the splatting approximation is unreliable, the assignment is meaningless. Invoked in Section 3.1.
  • domain assumption SAM masks are sufficiently class-agnostic and multi-view consistent, and CLIP and DINOv2 features are similar enough across views to match object fragments.
    The entire pipeline depends on off-the-shelf foundation model outputs. The authors' own ablation shows replacing SAM with FastSAM drops figurines mIoU from 0.822 to 0.608. Sections 3.2 and 4.4.
  • ad hoc to paper Greedy incremental merging over sequential frames converges to a usable scene-level segmentation.
    Merging is order-dependent and 'greedily merges new object fragments' without a global objective, and no analysis of ordering effects is provided. Section 3.2.
  • domain assumption The manually cleaned 3DGS fields used as ground truth for 3D asset evaluation are faithful object models.
    Ground truth is created by the authors manually selecting and removing Gaussians in each LERF scene, so the evaluation depends on the correctness of that manual cleaning. Section 4.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Lifting by Gaussians: A Simple, Fast and Flexible Method for 3D Instance Segmentation." pith.science (2026). https://pith.science/paper/F6PO3ZES

@misc{pith2026250200173,
  author       = {Pith},
  title        = {Pith review of: Lifting by Gaussians: A Simple, Fast and Flexible Method for 3D Instance Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/F6PO3ZES}},
  note         = {Machine review of arXiv:2502.00173}
}
read the original abstract

We introduce Lifting By Gaussians (LBG), a novel approach for open-world instance segmentation of 3D Gaussian Splatted Radiance Fields (3DGS). Recently, 3DGS Fields have emerged as a highly efficient and explicit alternative to Neural Field-based methods for high-quality Novel View Synthesis. Our 3D instance segmentation method directly lifts 2D segmentation masks from SAM (alternately FastSAM, etc.), together with features from CLIP and DINOv2, directly fusing them onto 3DGS (or similar Gaussian radiance fields such as 2DGS). Unlike previous approaches, LBG requires no per-scene training, allowing it to operate seamlessly on any existing 3DGS reconstruction. Our approach is not only an order of magnitude faster and simpler than existing approaches; it is also highly modular, enabling 3D semantic segmentation of existing 3DGS fields without requiring a specific parametrization of the 3D Gaussians. Furthermore, our technique achieves superior semantic segmentation for 2D semantic novel view synthesis and 3D asset extraction results while maintaining flexibility and efficiency. We further introduce a novel approach to evaluate individually segmented 3D assets from 3D radiance field segmentation methods.

Figures

Figures reproduced from arXiv: 2502.00173 by the authors.

Figure 1
Figure 1. Lifting by Gaussians (LBG). LBG utilizes 2D foundation model masks to segment any pretrained 3DGS field into objects, parts, and subparts without gradient-based learning. For each frame, 2D segmentations are lifted onto the per-pixel max-contributor Gaussian, producing object fragments. These fragments are then merged into coherent, scene-level objects based on both geometric and semantic overlap. Through a hierarch… view at source ↗
Figure 2
Figure 2. LBG constructs an open-vocabulary 3D instance segmentation from a sequence of posed RGB images. A generic 2D instance segmentation model is used to segment objects, parts, and subparts in each RGB image. Semantic feature vectors are extracted for each region, and the masks are lifted to the per-pixel max-contributing Gaussian, generating per-frame 3D object fragments. These fragments are incrementally merged into co… view at source ↗
Figure 3
Figure 3. Qualitative comparison on the LERF dataset for 3D Asset extraction. We show three extracted objects per scene, with two [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison on novel view synthesis for 2D instance masks. Black regions are unassigned. We see that our 2D masks are on par with other methods. LBG picks out in￾stances across segmentation scales better than Gaussian Grouping. Compared to SAGA, our method p…
Figure 5
Figure 5. Figure 5: Ablation on using CLIP features for merging. Using only spatial proximity leads to nearby objects being grouped together (red dashed boxes). When using DINO features together with CLIP this error is fixed. 5. Conclusion In this work, we presented LBG, a novel framework…
Figure 6
Figure 6. Figure 6: Additional 3D segmentation results on LERF dataset [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 8
Figure 8. Figure 8: Qualitative comparison on the 3DOVS dataset. Black regions are unassigned. In the bed scene, Gaussian Grouping merges hand and banana objects together, resulting in segmenta￾tion failure. Similarly, LangSplat fails to segment the white sheet due to low contrast in the …
Figure 7
Figure 7. Figure 7: Additional results on novel view synthesis for 2D instance masks. For SAGA, we show images rendered at three levels: 0.1 (left), 0.5 (middle), and 1.0 (right). For our method, we show object level (left), part level (middle), and subpart level (right). Even though our …
Figure 9
Figure 9. Figure 9: LBG Segmentation on 2DGS. 2DGS with colored Gaussians according to instance IDs (left) and individually ex￾tracted meshes (right). Model for mask extraction. While the Fast-SAM model provides results in near real-time, which is desirable for most applications in roboti…
Figure 10
Figure 10. Figure 10: Additional ablation results. We show performance of our method using the standard SAM model (left) and Fast-SAM (right). F. Improvements to Mini-Splatting We adopted a technique similar to Mini-splatting to re￾move floaters from Gaussian Splatting reconstructions. 3D …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ObjectGS: Object-aware Scene Reconstruction and Scene Understanding via Gaussian Splatting

    cs.GR 2025-07 conditional novelty 6.0 of 10

    ObjectGS unifies 3D Gaussian scene reconstruction with object-level segmentation by binding each object to local anchors with fixed one-hot ID encodings, improving open-vocabulary and panoptic segmentation.

  2. The ALMA-QUARKS Survey: III. Clump-to-core fragmentation and search for high-mass starless cores

    astro-ph.GA 2025-08 unverdicted novelty 4.0 of 10

    In 139 infrared-bright massive protoclusters, ALMA resolves 1562 cores whose separations are much smaller than the Jeans length, and finds only two candidate high-mass starless cores.

Reference graph

Works this paper leans on

52 extracted references · 33 canonical work pages · cited by 2 Pith papers

  1. [1]

    Emerg- ing Properties in Self-Supervised Vision Transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing Properties in Self-Supervised Vision Transformers. In Proceedings of the International Conference on Computer Vision (ICCV), 2021. 2, 3

    work page 2021

  2. [2]

    Segment Any 3D Gaussians

    Jiazhong Cen, Jiemin Fang, Chen Yang, Lingxi Xie, Xi- aopeng Zhang, Wei Shen, and Qi Tian. Segment Any 3D Gaussians. arXiv preprint arXiv:2312.00860, 2023. 2, 3, 4, 5, 7, 12

    arXiv 2023

  3. [3]

    Scanrefer: 3d object localization in rgb-d scans using natural language

    Dave Zhenyu Chen, Angel X Chang, and Matthias Nießner. Scanrefer: 3d object localization in rgb-d scans using natural language. In European conference on computer vision, pages 202–221. Springer, 2020. 2

    work page 2020

  4. [4]

    Chang, Manolis Savva, Maciej Hal- ber, Thomas A

    Angela Dai, Angel X. Chang, Manolis Savva, Maciej Hal- ber, Thomas A. Funkhouser, and Matthias Nießner. ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2432–2443, 2017. 2

    work page 2017

  5. [5]

    Mini-Splatting: Represent- ing Scenes with a Constrained Number of Gaussians, 2024

    Guangchi Fang and Bing Wang. Mini-Splatting: Represent- ing Scenes with a Constrained Number of Gaussians, 2024. eprint: 2403.14166. 4, 5

    arXiv 2024

  6. [6]

    Brandt, Axel Feld- mann, Zhoutong Zhang, and William T

    Stephanie Fu, Mark Hamilton, Laura E. Brandt, Axel Feld- mann, Zhoutong Zhang, and William T. Freeman. FeatUp: A Model-Agnostic Framework for Features at Any Resolu- tion. In The Twelfth International Conference on Learning Representations, 2024. 5

    work page 2024

  7. [7]

    Learning 3d semantic segmentation with only 2d image supervision

    Kyle Genova, Xiaoqi Yin, Abhijit Kundu, Caroline Panto- faru, Forrester Cole, Avneesh Sud, Brian Brewington, Brian Shucker, and Thomas Funkhouser. Learning 3d semantic segmentation with only 2d image supervision. In 2021 In- ternational Conference on 3D Vision (3DV), pages 361–372. IEEE, 2021. 2

    work page 2021

  8. [8]

    Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning

    Qiao Gu, Ali Kuwajerwala, Sacha Morin, Krishna Murthy Jatavallabhula, Bipasha Sen, Aditya Agarwal, Corban Rivera, William Paul, Kirsty Ellis, Rama Chellappa, and oth- ers. Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning. In 2024 IEEE International Con- ference on Robotics and Automation (ICRA) , pages 5021–

    work page 2024

Show all 52 references
  1. [9]

    EgoLifter: Open-world 3D Segmentation for Egocentric Perception

    Qiao Gu, Zhaoyang Lv, Duncan Frost, Simon Green, Ju- lian Straub, and Chris Sweeney. EgoLifter: Open-world 3D Segmentation for Egocentric Perception. arXiv preprint arXiv:2403.18118, 2024. 2, 3

    2024 arXiv

  2. [10]

    Sugar: Surface- aligned gaussian splatting for efficient 3d mesh reconstruc- tion and high-quality mesh rendering

    Antoine Gu ´edon and Vincent Lepetit. Sugar: Surface- aligned gaussian splatting for efficient 3d mesh reconstruc- tion and high-quality mesh rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5354–5363, 2024. 2

    2024

  3. [11]

    2d gaussian splatting for geometrically ac- curate radiance fields

    Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically ac- curate radiance fields. In ACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024. 2

    2024

  4. [12]

    2d gaussian splatting for geometrically accu- rate radiance fields

    Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically accu- rate radiance fields. In SIGGRAPH 2024 Conference Papers,

    2024

  5. [13]

    Photo-SLAM: Real-time Simultaneous Localization and Photorealistic Mapping for Monocular Stereo and RGB- D Cameras

    Huajian Huang, Longwei Li, Hui Cheng, and Sai-Kit Ye- ung. Photo-SLAM: Real-time Simultaneous Localization and Photorealistic Mapping for Monocular Stereo and RGB- D Cameras. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21584– 215...

    2024

  6. [14]

    Tenenbaum, Celso Miguel de Melo, Madhava Krishna, Liam Paull, Florian Shkurti, and Antonio Torralba

    Krishna Murthy Jatavallabhula, Alihusein Kuwajerwala, Qiao Gu, Mohd Omama, Tao Chen, Shuang Li, Ganesh Iyer, Soroush Saryazdi, Nikhil Keetha, Ayush Tewari, Joshua B. Tenenbaum, Celso Miguel de Melo, Madhava Krishna, Liam Paull, Florian Shkurti, and Antonio Torralba. ConceptFu-...

    2023

  7. [15]

    Seg- ment Any 4D Gaussians, 2024

    Shengxiang Ji, Guanjun Wu, Jiemin Fang, Jiazhong Cen, Taoran Yi, Wenyu Liu, Qi Tian, and Xinggang Wang. Seg- ment Any 4D Gaussians, 2024. eprint: 2407.04504. 4

    2024 arXiv

  8. [16]

    SplaTAM: Splat Track & Map 3D Gaussians for Dense RGB-D SLAM

    Nikhil Keetha, Jay Karhade, Krishna Murthy Jatavallab- hula, Gengshan Yang, Sebastian Scherer, Deva Ramanan, and Jonathon Luiten. SplaTAM: Splat Track & Map 3D Gaussians for Dense RGB-D SLAM. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,...

    2024

  9. [17]

    3D Gaussian Splatting for Real-Time Radiance Field Rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkuehler, and George Drettakis. 3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Transactions on Graphics (TOG), 42(4):1–14, 2023. Publisher: ACM New York, NY , USA. 1, 2, 4, 5

    2023

  10. [18]

    Lerf: Language embedded radiance fields

    Justin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa, and Matthew Tancik. Lerf: Language embedded radiance fields. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 19729–19739,

  11. [19]

    Garfield: Group anything with radiance fields

    Chung Min Kim, Mingxuan Wu, Justin Kerr, Ken Gold- berg, Matthew Tancik, and Angjoo Kanazawa. Garfield: Group anything with radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21530–21539, 2024. 2, 3

    2024

  12. [20]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, and others. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4015–4026, 20...

    2023

  13. [21]

    Rethinking Open-V ocabulary Segmen- tation of Radiance Fields in 3D Space

    Hyunjee Lee, Youngsik Yun, Jeongmin Bae, Seoha Kim, and Youngjung Uh. Rethinking Open-V ocabulary Segmen- tation of Radiance Fields in 3D Space. arXiv preprint arXiv:2408.07416, 2024. 2, 5

    2024 arXiv

  14. [22]

    Weinberger, Serge Belongie, Vladlen Koltun, and Rene Ranftl

    Boyi Li, Kilian Q. Weinberger, Serge Belongie, Vladlen Koltun, and Rene Ranftl. Language-driven Semantic Seg- mentation. In International Conference on Learning Repre- sentations, 2022. 2, 7, 14

    2022

  15. [23]

    Open-vocabulary semantic segmentation with mask-adapted clip

    Feng Liang, Bichen Wu, Xiaoliang Dai, Kunpeng Li, Yinan Zhao, Hang Zhang, Peizhao Zhang, Peter Vajda, and Diana Marculescu. Open-vocabulary semantic segmentation with mask-adapted clip. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pag...

    2023

  16. [24]

    Luciddreamer: Towards high- fidelity text-to-3d generation via interval score matching

    Yixun Liang, Xin Yang, Jiantao Lin, Haodong Li, Xiao- gang Xu, and Yingcong Chen. Luciddreamer: Towards high- fidelity text-to-3d generation via interval score matching. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 6517–6526, 2024. 2

    2024

  17. [25]

    Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d

    Yiyi Liao, Jun Xie, and Andreas Geiger. Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):3292–3310, 2022. Publisher: IEEE. 2

    2022

  18. [26]

    Weakly supervised 3d open- vocabulary segmentation

    Kunhao Liu, Fangneng Zhan, Jiahui Zhang, Muyu Xu, Yingchen Yu, Abdulmotaleb El Saddik, Christian Theobalt, Eric Xing, and Shijian Lu. Weakly supervised 3d open- vocabulary segmentation. Advances in Neural Information Processing Systems, 36:53433–53456, 2023. 5, 7, 12

    2023

  19. [27]

    Humangaus- sian: Text-driven 3d human generation with gaussian splat- ting

    Xian Liu, Xiaohang Zhan, Jiaxiang Tang, Ying Shan, Gang Zeng, Dahua Lin, Xihui Liu, and Ziwei Liu. Humangaus- sian: Text-driven 3d human generation with gaussian splat- ting. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 6646–6657,

  20. [28]

    Dynamic 3d gaussians: Tracking by per- sistent dynamic view synthesis

    Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by per- sistent dynamic view synthesis. In 2024 International Con- ference on 3D Vision (3DV), pages 800–809. IEEE, 2024. 2

    2024

  21. [29]

    Gaga: Group Any Gaussians via 3D- aware Memory Bank, 2024

    Weijie Lyu, Xueting Li, Abhijit Kundu, Yi-Hsuan Tsai, and Ming-Hsuan Yang. Gaga: Group Any Gaussians via 3D- aware Memory Bank, 2024. eprint: 2404.07977. 4, 5

    2024 arXiv

  22. [30]

    Gaussian splatting slam

    Hidenobu Matsuki, Riku Murai, Paul HJ Kelly, and An- drew J Davison. Gaussian splatting slam. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 18039–18048, 2024. 2

    2024

  23. [31]

    hdb- scan: Hierarchical density based clustering

    Leland McInnes, John Healy, Steve Astels, and others. hdb- scan: Hierarchical density based clustering. J. Open Source Softw., 2(11):205, 2017. 7

    2017

  24. [32]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM , 65(1):99–106, 2021. Publisher: ACM New York, NY , USA. 2, 4

    2021

  25. [33]

    Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Review

    Thang-Anh-Quan Nguyen, Amine Bourki, M’aty’as Macud- zinski, Anthony Brunel, and Mohammed Bennamoun. Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Review. ArXiv, abs/2402.11141, 2024. 2

    2024 arXiv

  26. [34]

    Maxime Oquab, Timoth ´ee Darcet, Theo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Rus- sell Howes, Po-Yao Huang, Hu Xu, Vasu Sharma, Shang- Wen Li, Wojciech Galuba, Mike Rabbat, Mido Assran, Ni...

    2023 arXiv

  27. [35]

    OpenScene: 3D Scene Understanding with Open V ocabular- ies

    Songyou Peng, Kyle Genova, Chiyu ”Max” Jiang, An- drea Tagliasacchi, Marc Pollefeys, and Thomas Funkhouser. OpenScene: 3D Scene Understanding with Open V ocabular- ies. In CVPR, 2023. 3

    2023

  28. [36]

    Langsplat: 3d language gaussian splatting

    Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister. Langsplat: 3d language gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20051–20060, 2024. 2, 3, 5, 7, 14

    2024

  29. [37]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, and others. Learning transferable visual models from natural language supervision. In International conference on machine learn- ...

    2021

  30. [38]

    Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation

    William Shen, Ge Yang, Alan Yu, Jansen Wong, Leslie Pack Kaelbling, and Phillip Isola. Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation. In 7th Annual Conference on Robot Learning, 2023. 2, 3

    2023

  31. [39]

    Contrastive Gaussian Clustering: Weakly Supervised 3D Scene Segmentation

    Myrna C Silva, Mahtab Dahaghin, Matteo Toso, and Alessio Del Bue. Contrastive Gaussian Clustering: Weakly Supervised 3D Scene Segmentation. arXiv preprint arXiv:2404.12784, 2024. 2, 3

    2024 arXiv

  32. [40]

    Sumner, Marc Pollefeys, Federico Tombari, and Francis Engelmann

    Ayc ¸a Takmaz, Elisabetta Fedele, Robert W. Sumner, Marc Pollefeys, Federico Tombari, and Francis Engelmann. OpenMask3D: Open-V ocabulary 3D Instance Segmentation. In Advances in Neural Information Processing Systems (NeurIPS), 2023. 3

    2023

  33. [41]

    DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation

    Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation. The Twelfth International Conference on Learning Representations, 2024. 2

    2024

  34. [42]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Si- moncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004. 5

    2004

  35. [43]

    4d gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20310–20320, 2024. 2

    2024

  36. [44]

    Gs-slam: Dense visual slam with 3d gaussian splatting

    Chi Yan, Delin Qu, Dan Xu, Bin Zhao, Zhigang Wang, Dong Wang, and Xuelong Li. Gs-slam: Dense visual slam with 3d gaussian splatting. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 19595–19604, 2024. 2

    2024

  37. [45]

    Deformable 3d gaussians for high- fidelity monocular dynamic scene reconstruction

    Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high- fidelity monocular dynamic scene reconstruction. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20331–20341, 2024. 2

    2024

  38. [46]

    Gaussian grouping: Segment and edit anything in 3d scenes

    Mingqiao Ye, Martin Danelljan, Fisher Yu, and Lei Ke. Gaussian grouping: Segment and edit anything in 3d scenes. European Conference on Computer Vision (ECCV), 2023. 2, 3, 4, 5, 7

    2023

  39. [47]

    Omniseg3d: Omniversal 3d segmentation via hierarchical contrastive learning

    Haiyang Ying, Yixuan Yin, Jinzhi Zhang, Fan Wang, Tao Yu, Ruqi Huang, and Lu Fang. Omniseg3d: Omniversal 3d segmentation via hierarchical contrastive learning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20612–20622, 2024. 2, 3

    2024

  40. [48]

    RaDe-GS: Rasterizing Depth in Gaussian Splatting

    Baowen Zhang, Chuan Fang, Rakesh Shrestha, Yixun Liang, Xiaoxiao Long, and Ping Tan. RaDe-GS: Rasterizing Depth in Gaussian Splatting. arXiv preprint arXiv:2406.01467 ,

    arXiv

  41. [49]

    The Unreasonable Effectiveness of Deep Features as a Perceptual Metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In CVPR, 2018. 5

    2018

  42. [50]

    Fast Segment Any- thing, 2023

    Xu Zhao, Wenchao Ding, Yongqi An, Yinglong Du, Tao Yu, Min Li, Ming Tang, and Jinqiao Wang. Fast Segment Any- thing, 2023. eprint: 2306.12156. 8

    2023 arXiv

  43. [51]

    Feature 3dgs: Supercharging 3d gaussian splatting to enable distilled feature fields

    Shijie Zhou, Haoran Chang, Sicheng Jiang, Zhiwen Fan, Ze- hao Zhu, Dejia Xu, Pradyumna Chari, Suya You, Zhangyang Wang, and Achuta Kadambi. Feature 3dgs: Supercharging 3d gaussian splatting to enable distilled feature fields. InPro- ceedings of the IEEE/CVF Conference on Compu...

    2024

  44. [52]

    EW A volume splatting

    Matthias Zwicker, Hanspeter Pfister, Jeroen Van Baar, and Markus Gross. EW A volume splatting. InProceedings Visu- alization, 2001. VIS’01., pages 29–538. IEEE, 2001. 4 A. Appendix In this appendix, we provide further experimental re- sults, including additional 3D segmentatio...

    2001

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.