REVIEW 13 cited by
LangSplat: 3D Language Gaussian Splatting
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
abstract
Humans live in a 3D world and commonly use natural language to interact with a 3D scene. Modeling a 3D language field to support open-ended language queries in 3D has gained increasing attention recently. This paper introduces LangSplat, which constructs a 3D language field that enables precise and efficient open-vocabulary querying within 3D spaces. Unlike existing methods that ground CLIP language embeddings in a NeRF model, LangSplat advances the field by utilizing a collection of 3D Gaussians, each encoding language features distilled from CLIP, to represent the language field. By employing a tile-based splatting technique for rendering language features, we circumvent the costly rendering process inherent in NeRF. Instead of directly learning CLIP embeddings, LangSplat first trains a scene-wise language autoencoder and then learns language features on the scene-specific latent space, thereby alleviating substantial memory demands imposed by explicit modeling. Existing methods struggle with imprecise and vague 3D language fields, which fail to discern clear boundaries between objects. We delve into this issue and propose to learn hierarchical semantics using SAM, thereby eliminating the need for extensively querying the language field across various scales and the regularization of DINO features. Extensive experimental results show that LangSplat significantly outperforms the previous state-of-the-art method LERF by a large margin. Notably, LangSplat is extremely efficient, achieving a 199 $\times$ speedup compared to LERF at the resolution of 1440 $\times$ 1080. We strongly recommend readers to check out our video results at https://langsplat.github.io/
Forward citations
Cited by 13 Pith papers
-
Splat and Replace: 3D Reconstruction with Repetitive Elements
Repetitive objects in 3D scenes are registered into a shared Gaussian representation that propagates well-observed geometry and appearance to poorly observed instances, improving rendered novel views.
-
LabelGS: Label-Aware 3D Gaussian Splatting for 3D Scene Segmentation
LabelGS assigns 2D video-tracking labels to the most-contributing 3D Gaussians, with depth-based occlusion masking and a projection filter, reporting better mIoU/PSNR than Feature-3DGS with roughly 22x faster training.
-
Predict-Optimize-Distill: A Self-Improving Cycle for 4D Object Understanding
POD interleaves prediction, inverse-rendering optimization, and self-distillation to reconstruct articulated object part poses from monocular video, outperforming an optimization-only baseline and improving with video...
-
Latent Radiance Fields with 3D-aware 2D Representations
A three-stage pipeline makes VAE latent codes 3D-consistent and builds a latent radiance field, improving photorealistic novel-view synthesis in latent space.
-
SparseLGS: Sparse View Language Embedded Gaussian Splatting
SparseLGS builds open-vocabulary 3D semantic fields from 3-4 pose-free images, using learning-based pose/point-cloud estimation and a three-step multi-view semantic alignment.
-
SparseGrasp: Robotic Grasping via 3D Semantic Gaussian Splatting from Sparse Multi-View RGB Images
A language-guided robotic grasping system that reconstructs a 3D semantic scene from three RGB views and updates moved objects in about 200 ms, reporting higher grasp success than F3RM and LERF-TOGO.
-
Bootstraping Clustering of Gaussians for View-consistent 3D Scene Understanding
FreeGS bootstraps view-consistent semantics and instance indices in 3D Gaussian Splatting without needing 2D masks.
-
Gradient-Weighted Feature Back-Projection: A Fast Alternative to Feature Distillation in 3D Gaussian Splatting
A gradient-weighted back-projection of 2D features onto 3D Gaussians yields fast 2D/3D segmentation and identity encoding at quality comparable to trained feature-field methods.
-
Leveraging 2D Priors and SDF Guidance for Dynamic Urban Scene Rendering
UGSDF achieves state-of-the-art novel-view rendering of dynamic urban objects without LiDAR or 3D motion annotations by jointly optimizing SDFs and 3D Gaussians under 2D depth and point-tracking priors.
-
A Neural Representation Framework with LLM-Driven Spatial Reasoning for Open-Vocabulary 3D Visual Grounding
SpatialReasoner adds LLM-based query decomposition and a visual-properties-enhanced hierarchical feature field to 3D language fields, improving instance localization for spatial-relation queries on self-constructed be...
-
GaussianProperty: Integrating Physical Properties to 3D Gaussians with LMMs
A training-free pipeline uses SAM segmentation and GPT-4V material recognition, then votes across views to attach density, elasticity, and friction values to 3D Gaussians for simulation and grasping.
-
From Coarse to Fine: Learnable Discrete Wavelet Transforms for Efficient 3D Gaussian Splatting
AutoOpti3DGS uses learnable discrete wavelet transforms on input images to train 3DGS from coarse to fine, reducing peak Gaussian counts by about 18 to 23 percent with modest quality trade-offs.
-
OpenMaskDINO3D : Reasoning 3D Segmentation via Large Language Model
OpenMaskDINO3D reports state-of-the-art 3D reasoning segmentation with a LISA-style SEG token and object identifiers, but uses Mask3D pseudo-labels as ground truth and lacks released code.
Discussion (0). Continue with ORCID to comment.