REVIEW 7 cited by
3D Vision-Language Gaussian Splatting
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent advancements in 3D reconstruction methods and vision-language models have propelled the development of multi-modal 3D scene understanding, which has vital applications in robotics, autonomous driving, and virtual/augmented reality. However, current multi-modal scene understanding approaches have naively embedded semantic representations into 3D reconstruction methods without striking a balance between visual and language modalities, which leads to unsatisfying semantic rasterization of translucent or reflective objects, as well as over-fitting on color modality. To alleviate these limitations, we propose a solution that adequately handles the distinct visual and semantic modalities, i.e., a 3D vision-language Gaussian splatting model for scene understanding, to put emphasis on the representation learning of language modality. We propose a novel cross-modal rasterizer, using modality fusion along with a smoothed semantic indicator for enhancing semantic rasterization. We also employ a camera-view blending technique to improve semantic consistency between existing and synthesized views, thereby effectively mitigating over-fitting. Extensive experiments demonstrate that our method achieves state-of-the-art performance in open-vocabulary semantic segmentation, surpassing existing methods by a significant margin.
Forward citations
Cited by 7 Pith papers
-
Relation-Centric Open-Vocabulary 3D Gaussian Segmentation
PairGS builds a relation graph from sparse pairwise affinities on 3D Gaussians to achieve SOTA open-vocabulary segmentation with a 50x faster variant than optimization-based methods.
-
3AM: 3egment Anything with Geometric Consistency in Videos
3AM integrates MUSt3R 3D features into SAM2 via a Feature Merger and FOV-aware sampling to deliver geometry-consistent video object segmentation from RGB alone, with large gains on wide-baseline datasets.
-
GaussianSelector: Lightweight Human-Guided Object Selection in 3D Gaussian Splatting with Graph Optimization
A training-free graph-cut method selects 3D objects from Gaussian splatting scenes using sparse user scribbles, reaching 92.2 mIoU on NVOS with three interaction views.
-
E3DGS: Unified Geometric-Photometric Equivariance for 3D Gaussian Splatting via Color-as-Geometry Embedding
3D Gaussian view-dependent colors are repacked as 3×3 matrices so geometry and color rotate together, giving exact rotation-equivariant recognition and world modeling in 3DGS.
-
DualSplat: Robust 3D Gaussian Splatting via Pseudo-Mask Bootstrapping from Reconstruction Failures
DualSplat bootstraps object-level pseudo-masks from initial 3DGS reconstruction failures using residuals and SAM2 to enable robust second-pass optimization in transient-heavy scenes.
-
NG-GS: NeRF-Guided 3D Gaussian Splatting Segmentation
NG-GS uses NeRF guidance and RBF interpolation on 3DGS to produce smoother, higher-quality object segmentation boundaries.
-
ZeroSplat: Generalized Referring Segmentation in 3D Gaussian Splatting
ZeroSplat performs generalized referring segmentation in 3D Gaussian Splatting with no training or extra features, outperforming single-target baselines on two newly introduced benchmarks.
Discussion (0). Sign in to comment.