Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

SemanticSplat: Feed-Forward 3D Scene Understanding with Language-Aware Gaussian Fields

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SemanticSplat reconstructs a holistic semantic Gaussian field from sparse views that jointly encodes geometry, appearance, and multi-modal semantics, achieving target-view mIoU of 0.386 for language queries and 0.433 for point prompts on…

desk verdict A capable engineering combination of MVSplat and SAM/LSeg distillation with small but real gains over 2D teachers; the evaluation needs error bars, code, and a closer look at whether the frozen SAM decoder stays compatible at novel views. read the letter →

arxiv 2506.09565 v2 pith:Q2APSR7C submitted 2025-06-11 cs.CV

Qijing Li , Jingxiang Sun , Liang An , Zhaoqi Su , Hongwen Zhang , Yebin Liu This is my paper
classification cs.CV
keywords 3DsceneunderstandingGaussiansplattingopen-vocabularysegmentationpromptablefeed-forwardreconstructionfeaturedistillationsparse-viewsemanticfields
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SemanticSplat attempts to show that a single feed-forward network can turn a few posed images of a scene into a 3D field of small Gaussians that carries not only color and geometry but also two kinds of semantic meaning: language-aligned features for open-vocabulary queries and SAM-style segmentation features for point prompts. If true, this matters because today's 3D scene understanding either optimizes per scene (slow, needing many views) or is limited to text-only semantics with noisy geometry. The paper reports that on held-out ScanNet scenes the reconstructed field reaches target-view mIoU of 0.386 for language queries, above 2D LSeg's 0.364, and 0.433 for point prompts, above SAM's 0.426, while simultaneously producing usable novel views. The practical promise is one forward pass, no per-scene fitting, and one representation that answers 'what is this?' and 'segment this point' in 3D.

What carries the argument

The load-bearing object is the semantic anisotropic Gaussian, a 3D Gaussian whose attributes include a latent semantic vector f alongside position, opacity, covariance, and color, with the same rasterizer rendering color and semantics via the same alpha-compositing formula. Around it, the pipeline has three supporting mechanisms: a plane-sweep cost volume that matches features across views and predicts depth; multi-conditioned feature fusion, which concatenates monocular SAM and LSeg feature maps with the cost volume so geometric and semantic cues influence one another; and two-stage distillation, where a segmentation feature field is trained first with a consistency loss against SAM's frozen mask decoder, and a language field is trained second with the segmentation branch frozen, using hierarchical-mask pooling over SAM masks to enforce intra-mask semantic consistency.

What would settle it

Render the semantic field at target views, feed the rendered SAM features into the frozen mask decoder, and compare mask quality against the decoder's outputs on the true target image across viewpoints with increasing baseline; if mIoU drops steeply as viewpoint moves away from the input views, enough to erase the small reported margin over 2D SAM, the coherence claim fails.

Watch

Extended reading notes

Core claim

The central discovery is that 2D foundation-model features, segmentation embeddings from SAM and language embeddings from CLIP-LSeg, can be distilled into per-Gaussian latent attributes and rendered back to any view, producing a holistic semantic field that stays consistent across views. The paper extends 3D Gaussian Splatting by giving each Gaussian a latent semantic vector f in addition to position, opacity, covariance, and color, and renders both an RGB image and a semantic feature map with the same splatting operation. A cost volume built from cross-view feature similarities supplies geometric grounding, and multi-conditioned fusion of monocular SAM and LSeg features with that cost volume lets a U-Net predict depth and the semantic attributes together. A two-stage distillation then lifts the two modalities separately, first segmentation features with a mask-consistency loss that keeps the frozen SAM mask decoder usable, then language features with hierarchical-context-aware pooling over SAM masks, so the final field supports open-vocabulary and promptable segmentation in one feed-forward model.

Load-bearing premise

The approach assumes the frozen SAM mask decoder stays compatible with the segmentation features rendered at novel views, so masks it produces from those rendered features remain as good as masks it produces from its own image features.

Editorial extensions

If this is right

  • A single feed-forward model can serve multiple 3D tasks, including novel view synthesis, depth prediction, open-vocabulary segmentation, and promptable segmentation, from a handful of images, removing the per-scene optimization step.
  • Because semantic features are stored in 3D and rendered, the model should keep semantics consistent across views, avoiding the cross-view flip-flopping of per-frame 2D segmenters.
  • Training needs only posed RGB images and runs in about two GPU-days, so the approach is far cheaper to pretrain than large reconstruction models and needs no depth supervision.
  • The two-stage frozen-branch design lets each semantic modality be improved or swapped independently, so better segmentation or language encoders could replace SAM or LSeg without retraining the whole model.
  • The same representation can be deployed directly to unseen scenes without per-scene fitting, which is the main practical advantage over optimization-based semantic Gaussian fields.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same representation could be extended to dynamic scenes by adding a time axis to the Gaussian attributes, turning the holistic field into a video-level semantic representation.
  • Because the field is feed-forward and pose-conditioned, dropping or relaxing the pose requirement would let the method run on casual phone video; a testable intermediate step is estimating poses with a learned module and comparing target-view mIoU.
  • The hierarchical-context-aware pooling suggests a general recipe: any per-pixel 2D features can be stabilized in 3D by pooling within masks from a frozen segmenter, and one could test this with DINO or other self-supervised features to get part-level or object-level fields.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. SemanticSplat proposes a feed-forward 3D Gaussian representation that jointly reconstructs geometry, appearance, and multi-modal semantic features from sparse posed images. The method fuses multi-view cost volumes with monocular SAM and CLIP-LSeg features, predicts per-pixel anisotropic Gaussians augmented with semantic attributes, and uses a two-stage distillation procedure (first SAM segmentation features, then CLIP-LSeg language features) to build a holistic semantic field. The paper evaluates the approach on ScanNet for novel view synthesis, open-vocabulary language segmentation, and promptable segmentation, reporting target-view mIoU of 0.386 for language queries (above LSeg's 0.364) and 0.433 for point prompts (above SAM's 0.426), along with ablations on feature conditioning and mask loss.

Significance. If the reported results are robust, the paper makes a useful advance: it is among the first feed-forward frameworks to lift multiple 2D foundation-model features into a single 3D Gaussian field, enabling concurrent geometry, appearance, and semantics reconstruction without per-scene optimization. The two-stage distillation with a frozen segmentation branch is a sensible design for preventing catastrophic forgetting, and the multi-conditioned feature fusion with cost volumes is a plausible mechanism for improving cross-view consistency. The paper also provides comparisons against both optimization-based (Feature-3DGS) and generalizable (LSM) baselines, plus ablations supporting the importance of each feature condition and the mask consistency loss. However, the central quantitative claims rest on small margins (0.007 mIoU over SAM, 0.022 over LSeg) with no error bars or significance tests, and several presentation issues in the tables obscure the comparisons. The idea is promising and worth pursuing, but the evidence as presented is not yet fully convincing.

major comments (4)
  1. [Table 2] Table 2 (Promptable Segmentation) reports Acc values that exceed mIoU values for every row (e.g., SAM source: mIoU 0.684, Acc 0.427), which is inconsistent with Table 1 where mIoU is always lower than Acc. The columns appear to be swapped (the second and fourth numeric columns are likely Acc and mIoU, respectively). Please correct the column ordering and report the values consistently. If the corrected target-view margin over SAM remains 0.007 mIoU, provide confidence intervals or multiple-seed results to establish that the improvement is statistically meaningful.
  2. [Section 3.4 (Prompt-Aware Mask Refinement)] The promptable-segmentation claim depends on invoking the frozen SAM mask decoder on rendered semantic feature maps from held-out novel views. Compatibility is enforced only through a cosine-similarity loss (Eq. 2) and a mask consistency loss (Eq. 5) computed at training views. Cosine similarity is invariant to per-channel scaling and offset, while SAM's decoder is sensitive to the absolute embedding distribution; the paper does not analyze whether rendered features at novel views lie on the native SAM embedding manifold. Because the reported target-view margin over the 2D SAM baseline is only 0.007 mIoU, the manuscript should provide evidence of robustness, for example by reporting feature-distribution statistics (e.g., mean/standard deviation of rendered vs. SAM features), showing performance under small feature perturbations, or giving error bars across the 50 validation scenes.
  3. [Table 3] Table 3 (Feature-Condition Ablation) has unclear and duplicated column headers: "Compared with GT Masks" appears twice, and the row labels (full, SAM, LSeg, w/o cond.) do not specify which feature branch is active in each of the two distillation stages. Since this table is the main support for the multi-conditioned feature fusion claim, please restructure it so that each column identifies exactly which mask source (LSeg vs. GT vs. SAM) and which condition (full, SAM-only, LSeg-only, w/o condition) is used, and clarify the stage to which each block refers.
  4. [Table 1 (NVS comparison)] The novel view synthesis comparison uses MVSplat pretrained on RealEstate10K, whereas the other methods are trained on ScanNet. This is a training-data confound: the comparison is not apples-to-apples. The statement in Section 4.2 that "our method achieves comparable pixel-level quality to MVSplat despite training on lower-quality data" may be true, but the current setup does not isolate the effect of the semantic conditioning. Please retrain MVSplat on ScanNet (or on the same training set) or clearly discuss this limitation and its impact on the reported PSNR/SSIM/LPIPS values.
minor comments (6)
  1. [Throughout] There are several typos and formatting inconsistencies: "Promtable" for "Promptable" in Table 2's caption, "effictively" for "effectively" in Figure 4's caption, and inconsistent use of "show" vs. "shown" in Section 3.4. A careful proofread is recommended.
  2. [Section 3.4] The symbols LSeg_dist and LSeg_mask are used for losses in the SAM distillation branch, which is confusing because LSeg is also the name of the language model used in Section 3.5. Please rename these losses (e.g., L_dist^SAM and L_mask^SAM) to avoid ambiguity.
  3. [Equation (1)] The summation in Eq. (1) uses index i for both the Gaussian index and the view index, and the project 2D Gaussian kernel G(X) is evaluated at pixel X but the notation for per-view accumulation is not fully defined. Please clarify the indexing to make the rendering equation unambiguous.
  4. [Table 4 (Appendix)] In the supplementary ablation on the mask loss, the row "full" is not defined; please specify that it refers to the complete first-stage model from Table 3.
  5. [Section 4.1 (Implementation details)] The implementation details state that each stage is trained for 5000 iterations on 4 A100 GPUs, but do not report the number of input views, batch size, or learning-rate schedule parameters. Providing these would improve reproducibility.
  6. [References] Reference [52] (Feature 3DGS) is duplicated as [53]; please merge or renumber.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: all headline semantic metrics are evaluated against ScanNet ground-truth labels, independent of the SAM/LSeg distillation signal.

full rationale

The paper's derivation chain is a feed-forward distillation pipeline: SAM and CLIP-LSeg features are lifted into 3D Gaussians through cosine-similarity losses (Eqs. 2 and 6) and a mask-consistency loss (Eq. 5), then rendered at novel views. The final semantic evaluation in Tables 1 and 2 compares the rendered segmentation against ScanNet ground-truth masks, not against the teacher features or teacher masks. This makes the reported mIoU/Acc numbers an independent, externally grounded measurement rather than a restatement of the training objective. The baselines include the teachers themselves (LSeg in Table 1, SAM in Table 2), but a distilled model outperforming its teacher on a held-out ground-truth benchmark is not forced by construction; it is a genuine empirical outcome enabled by multi-view aggregation. The reliance on a frozen SAM mask decoder for promptable segmentation is a legitimate generalization risk, but it is not a circular step: the decoder is a fixed external component, and the mask-consistency loss only supervises training views. No fitted parameter is renamed as a prediction, and no load-bearing self-citation appears in the reference list. The inconsistent column ordering in Table 2 is a presentation issue, not evidence of circularity. Overall, the derivation is self-contained against external benchmarks, so the appropriate finding is no significant circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical entities; the semantic anisotropic Gaussian is a representation choice. The load-bearing assumptions are the reliability of the 2D teachers, the cost-volume depth, and the shared latent capacity. The model's learned weights are fitted to ScanNet data, which is standard, but the hyperparameters in the ledger are hand-picked.

free parameters (3)
  • LPIPS weight lambda_1 = 0.05
    Hand-chosen weight in the photometric loss (Eq. 8) that trades pixel fidelity against perceptual similarity; not validated with a sweep.
  • Mask loss weight lambda_mask = 0.2
    Hand-chosen weight for the SAM mask consistency loss in Eq. 9; paper reports only a with/without ablation, no sweep.
  • Depth candidates D and downsampling factor s = not stated
    Hyperparameters of the cost volume depth estimation (Sec. 3.1) are not reported, so the depth quality budget is undefined.
assumptions (4)
  • domain assumption Pre-trained SAM and CLIP-LSeg features provide a sufficiently complete and view-consistent semantic signal to supervise a 3D field.
    The entire two-stage distillation (Secs. 3.4, 3.5) assumes these 2D teachers are learnable targets for 3D lifting; the paper's segmentation performance is therefore bounded by teacher quality.
  • domain assumption Plane-sweep cost volumes built from a small number of views predict depth accurately enough for rendering consistent semantic features at novel views.
    Depth maps from cost volumes (Sec. 3.1) are the sole geometry source; if they are inaccurate the semantic field is rendered at wrong positions, degrading target-view metrics.
  • domain assumption A single shared latent semantic attribute per Gaussian can support both SAM-aligned and LSeg-aligned heads without mutual interference.
    The method uses one latent f_j for both segmentation and language heads (Sec. 3.3) and freezes stage 1 during stage 2; if the latent cannot represent both, one modality degrades.
  • domain assumption ScanNet training scenes generalize to held-out ScanNet scenes.
    The model is trained and evaluated only on ScanNet (Sec. 4.1); no cross-dataset or open-world tests are reported, so generalization is unmeasured.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SemanticSplat: Feed-Forward 3D Scene Understanding with Language-Aware Gaussian Fields." pith.science (2026). https://pith.science/paper/Q2APSR7C

@misc{pith2026250609565,
  author       = {Pith},
  title        = {Pith review of: SemanticSplat: Feed-Forward 3D Scene Understanding with Language-Aware Gaussian Fields},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q2APSR7C}},
  note         = {Machine review of arXiv:2506.09565}
}
read the original abstract

Holistic 3D scene understanding, which jointly models geometry, appearance, and semantics, is crucial for applications like augmented reality and robotic interaction. Existing feed-forward 3D scene understanding methods (e.g., LSM) are limited to extracting language-based semantics from scenes, failing to achieve holistic scene comprehension. Additionally, they suffer from low-quality geometry reconstruction and noisy artifacts. In contrast, per-scene optimization methods rely on dense input views, which reduces practicality and increases complexity during deployment. In this paper, we propose SemanticSplat, a feed-forward semantic-aware 3D reconstruction method, which unifies 3D Gaussians with latent semantic attributes for joint geometry-appearance-semantics modeling. To predict the semantic anisotropic Gaussians, SemanticSplat fuses diverse feature fields (e.g., LSeg, SAM) with a cost volume representation that stores cross-view feature similarities, enhancing coherent and accurate scene comprehension. Leveraging a two-stage distillation framework, SemanticSplat reconstructs a holistic multi-modal semantic feature field from sparse-view images. Experiments demonstrate the effectiveness of our method for 3D scene understanding tasks like promptable and open-vocabulary segmentation. Video results are available at https://semanticsplat.github.io.

Figures

Figures reproduced from arXiv: 2506.09565 by the authors.

Figure 1
Figure 1. Our approach utilizes sparse view images as input to reconstruct a holistic semantic Gaussian field, which includes both the [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. We employ multiview transformers with cross-attention to extract features from multi-view images and use cost volumes for [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Novel View Synthesis Comparisons. Our method outperforms LSM and Feature-3DGS in challenging regions and is compatible with baseline MVSplat, which shows we reconstruct the appearance successfully [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Language-based Segmentation Comparison. We visualize the segmentation from a set of categories for unseen view, our method outperforms with the other 3D method and comparably to the 2D VFMs, which indicates we effictively lift 2d foundation language-image model to 3D. …
Figure 5
Figure 5. Figure 5: Visualization of the Semantic Feature Field. We visual￾ize the language features and segmentation characteristics of the novel views, demonstrating how we elevate the 2D features into 3D while maintaining consistency across views. The visualizations are generated using…
Figure 7
Figure 7. Figure 7: Ablation study on different Conditions, HCAM and Mask loss. We visualize the segmentation results under different conditions, illustrating that all these are complementary. Multi-Conditioned Semantic Features In [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. E3DGS: Unified Geometric-Photometric Equivariance for 3D Gaussian Splatting via Color-as-Geometry Embedding

    cs.CV 2026-07 conditional novelty 6.0 of 10

    3D Gaussian view-dependent colors are repacked as 3×3 matrices so geometry and color rotate together, giving exact rotation-equivariant recognition and world modeling in 3DGS.

  2. QuASH: Using Natural-Language Heuristics to Query Visual-Language Robotic Maps

    cs.RO 2025-10 conditional novelty 6.0 of 10

    Querying VLM robot maps with an SVM trained on LLM-generated synonym/antonym embeddings outperforms cosine-threshold and single-antonym baselines on images and OpenSeg maps, but not consistently on LSeg maps.

Reference graph

Works this paper leans on

55 extracted references · 33 canonical work pages · cited by 2 Pith papers

  1. [1]

    On the opportunities and risks of foundation models.arXiv preprint arXiv:2108.07258, 2021

    Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Alt- man, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models.arXiv preprint arXiv:2108.07258, 2021

    arXiv 2021

  2. [2]

    Segment any 3d gaussians

    Jiazhong Cen, Jiemin Fang, Chen Yang, Lingxi Xie, Xiaopeng Zhang, Wei Shen, and Qi Tian. Segment any 3d gaussians. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 1971–1979, 2025

    work page 1971

  3. [3]

    pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction

    David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann. pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19457–19467, 2024

    work page 2024

  4. [4]

    Lara: Efficient large-baseline radiance fields

    Anpei Chen, Haofei Xu, Stefano Esposito, Siyu Tang, and Andreas Geiger. Lara: Efficient large-baseline radiance fields. InEuropean Conference on Computer Vision, pages 338–355. Springer, 2024

    work page 2024

  5. [5]

    Feat2gs: Probing visual foundation models with gaussian splatting.arXiv preprint arXiv:2412.09606, 2024

    Yue Chen, Xingyu Chen, Anpei Chen, Gerard Pons-Moll, and Yuliang Xiu. Feat2gs: Probing visual foundation models with gaussian splatting.arXiv preprint arXiv:2412.09606, 2024

    arXiv 2024

  6. [6]

    Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images.ECCV, 2024

    Yuedong Chen, Haofei Xu, Chuanxia Zheng, Bohan Zhuang, Marc Pollefeys, Andreas Geiger, Tat-Jen Cham, and Jianfei Cai. Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images.ECCV, 2024

    2024

  7. [7]

    Dico-nerf: Difference of cosine similarity for neural rendering of fisheye driving scenes

    Jiho Choi, Gyutae Hwang, and Sang Jun Lee. Dico-nerf: Difference of cosine similarity for neural rendering of fisheye driving scenes. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7850– 7858, 2024

    work page 2024

  8. [8]

    A space-sweep approach to true multi- image matching

    Robert T Collins. A space-sweep approach to true multi- image matching. InProceedings CVPR IEEE computer soci- ety conference on computer vision and pattern recognition, pages 358–363. Ieee, 1996

    work page 1996

Show all 55 references
  1. [9]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017

    2017

  2. [10]

    An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv preprint ar...

    2010 arXiv

  3. [11]

    Nerf-sos: Any-view self- supervised object segmentation on complex scenes.arXiv preprint arXiv:2209.08776, 2022

    Zhiwen Fan, Peihao Wang, Yifan Jiang, Xinyu Gong, De- jia Xu, and Zhangyang Wang. Nerf-sos: Any-view self- supervised object segmentation on complex scenes.arXiv preprint arXiv:2209.08776, 2022

    2022 arXiv

  4. [12]

    Large spatial model: End-to-end unposed images to semantic 3d.NeurIPS, 2024

    Zhiwen Fan, Jian Zhang, Wenyan Cong, Peihao Wang, Renjie Li, Kairun Wen, Shijie Zhou, Achuta Kadambi, Zhangyang Wang, Danfei Xu, et al. Large spatial model: End-to-end unposed images to semantic 3d.NeurIPS, 2024

    2024

  5. [13]

    Fast and efficient: Mask neural fields for 3d scene segmentation.arXiv preprint arXiv:2407.01220, 2024

    Zihan Gao, Lingling Li, Licheng Jiao, Fang Liu, Xu Liu, Wen- ping Ma, Yuwei Guo, and Shuyuan Yang. Fast and efficient: Mask neural fields for 3d scene segmentation.arXiv preprint arXiv:2407.01220, 2024

    2024 arXiv

  6. [14]

    Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions.SIAM review, 53(2):217–288, 2011

    Nathan Halko, Per-Gunnar Martinsson, and Joel A Tropp. Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions.SIAM review, 53(2):217–288, 2011

    2011

  7. [15]

    3d gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics, 42(4), 2023

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics, 42(4), 2023

    2023

  8. [16]

    Lerf: Language embedded radiance fields

    Justin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa, and Matthew Tancik. Lerf: Language embedded radiance fields. InProceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 19729–19739, 2023

    2023

  9. [17]

    Lerf: Language embed- ded radiance fields

    Justin* Kerr, Chung Min* Kim, Ken Goldberg, Angjoo Kanazawa, and Matthew Tancik. Lerf: Language embed- ded radiance fields. InInternational Conference on Computer Vision (ICCV), 2023

    2023

  10. [18]

    Garfield: Group any- thing with radiance fields

    Chung Min Kim, Mingxuan Wu, Justin Kerr, Ken Goldberg, Matthew Tancik, and Angjoo Kanazawa. Garfield: Group any- thing with radiance fields. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21530–21539, 2024

    2024

  11. [19]

    Adam: A method for stochastic optimiza- tion.arXiv preprint arXiv:1412.6980, 2014

    Diederik P Kingma. Adam: A method for stochastic optimiza- tion.arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  12. [20]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. InProceedings of the IEEE/CVF International Confer- ence on Computer Vision, pages 4015–4026, 2023. 9

    2023

  13. [21]

    Rethinking open-vocabulary segmentation of radiance fields in 3d space

    Hyunjee Lee, Youngsik Yun, Jeongmin Bae, Seoha Kim, and Youngjung Uh. Rethinking open-vocabulary segmentation of radiance fields in 3d space. InProceedings of the AAAI Conference on Artificial Intelligence, pages 4491–4498, 2025

    2025

  14. [22]

    Language-driven semantic seg- mentation

    Boyi Li, Kilian Q Weinberger, Serge Belongie, Vladlen Koltun, and Rene Ranftl. Language-driven semantic seg- mentation. InInternational Conference on Learning Repre- sentations, 2022

    2022

  15. [23]

    Langsurf: Language- embedded surface gaussians for 3d scene understanding

    Hao Li, Roy Qin, Zhengyu Zou, Diqi He, Bohan Li, Bingquan Dai, Dingewn Zhang, and Junwei Han. Langsurf: Language- embedded surface gaussians for 3d scene understanding. arXiv preprint arXiv:2412.17635, 2024

    2024

  16. [24]

    4d langsplat: 4d language gaussian splatting via multimodal large language models.arXiv preprint arXiv:2503.10437, 2025

    Wanhua Li, Renping Zhou, Jiawei Zhou, Yingwei Song, Jo- hannes Herter, Minghan Qin, Gao Huang, and Hanspeter Pfister. 4d langsplat: 4d language gaussian splatting via multimodal large language models.arXiv preprint arXiv:2503.10437, 2025

    2025 arXiv

  17. [25]

    Guibiao Liao, Kaichen Zhou, Zhenyu Bao, Kanglin Liu, and Qing Li. Ov-nerf: Open-vocabulary neural radiance fields with vision and language foundation models for 3d semantic understanding.IEEE Transactions on Circuits and Systems for Video Technology, 2024

    2024

  18. [26]

    Focal loss for dense object detection

    Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Doll´ar. Focal loss for dense object detection. InPro- ceedings of the IEEE international conference on computer vision, pages 2980–2988, 2017

    2017

  19. [27]

    Splatraj: Camera trajectory generation with semantic gaussian splatting.arXiv preprint arXiv:2410.06014, 2024

    Xinyi Liu, Tianyi Zhang, Matthew Johnson-Roberson, and Weiming Zhi. Splatraj: Camera trajectory generation with semantic gaussian splatting.arXiv preprint arXiv:2410.06014, 2024

    2024 arXiv

  20. [28]

    Swin transformer v2: Scaling up capacity and resolution

    Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, et al. Swin transformer v2: Scaling up capacity and resolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12009–12019, 2022

    2022

  21. [29]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthe- sis. InECCV, 2020

    2020

  22. [30]

    V-net: Fully convolutional neural networks for volumetric medical image segmentation

    Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In2016 fourth international conference on 3D vision (3DV), pages 565–571. Ieee, 2016

    2016

  23. [31]

    Scikit-learn: Machine learning in python.the Journal of machine Learning research, 12:2825–2830, 2011

    Fabian Pedregosa, Ga¨el Varoquaux, Alexandre Gramfort, Vin- cent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blon- del, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in python.the Journal of machine Learning research, 12:2825–2830, 2011

    2011

  24. [32]

    Langsplat: 3d language gaussian splatting

    Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister. Langsplat: 3d language gaussian splatting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20051–20060, 2024

    2024

  25. [33]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, p...

    2021

  26. [34]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  27. [35]

    U- net: Convolutional networks for biomedical image segmenta- tion

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmenta- tion. InMedical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Par...

    2015

  28. [36]

    A pixel is worth more than one 3d gaussians in single-view 3d reconstruction

    Jianghao Shen, Nan Xue, and Tianfu Wu. A pixel is worth more than one 3d gaussians in single-view 3d reconstruction. arXiv preprint arXiv:2405.20310, 2024

    2024 arXiv

  29. [37]

    Gamba: Marry gaussian splatting with mamba for single view 3d reconstruc- tion.arXiv preprint arXiv:2403.18795, 2024

    Qiuhong Shen, Zike Wu, Xuanyu Yi, Pan Zhou, Hanwang Zhang, Shuicheng Yan, and Xinchao Wang. Gamba: Marry gaussian splatting with mamba for single view 3d reconstruc- tion.arXiv preprint arXiv:2403.18795, 2024

    2024 arXiv

  30. [38]

    Distilled feature fields enable few-shot language-guided manipulation

    William Shen, Ge Yang, Alan Yu, Jansen Wong, Leslie Pack Kaelbling, and Phillip Isola. Distilled feature fields enable few-shot language-guided manipulation. In7th Annual Con- ference on Robot Learning, 2023

    2023

  31. [39]

    Language embedded 3d gaussians for open-vocabulary scene understanding

    Jin-Chuan Shi, Miao Wang, Hao-Bin Duan, and Shao-Hua Guan. Language embedded 3d gaussians for open-vocabulary scene understanding. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 5333–5343, 2024

    2024

  32. [40]

    Panoptic lifting for 3d scene understanding with neural fields

    Yawar Siddiqui, Lorenzo Porzi, Samuel Rota Bul´o, Norman M¨uller, Matthias Nießner, Angela Dai, and Peter Kontschieder. Panoptic lifting for 3d scene understanding with neural fields. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 90...

    2023

  33. [41]

    Splatter image: Ultra-fast single-view 3d recon- struction

    Stanislaw Szymanowicz, Chrisitian Rupprecht, and Andrea Vedaldi. Splatter image: Ultra-fast single-view 3d recon- struction. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10208–10217, 2024

    2024

  34. [42]

    Lgm: Large multi-view gaussian model for high-resolution 3d content creation

    Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation. InEuropean Conference on Computer Vision, pages 1–18. Springer, 2024

    2024

  35. [43]

    Bovik, H.R

    Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity.IEEE Transactions on Image Processing, 13(4): 600–612, 2004

    2004

  36. [44]

    Gmflow: Learning optical flow via global matching

    Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, and Dacheng Tao. Gmflow: Learning optical flow via global matching. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8121–8130, 2022

    2022

  37. [45]

    Unifying flow, stereo and depth estimation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(11):13941–13958, 2023

    Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, Fisher Yu, Dacheng Tao, and Andreas Geiger. Unifying flow, stereo and depth estimation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(11):13941–13958, 2023

    2023

  38. [46]

    Grm: Large gaussian reconstruction model for efficient 3d recon- 10 struction and generation

    Yinghao Xu, Zifan Shi, Wang Yifan, Hansheng Chen, Ceyuan Yang, Sida Peng, Yujun Shen, and Gordon Wetzstein. Grm: Large gaussian reconstruction model for efficient 3d recon- 10 struction and generation. InEuropean Conference on Com- puter Vision, pages 1–20. Springer, 2024

    2024

  39. [47]

    Featurenerf: Learning generalizable nerfs by distilling foundation models

    Jianglong Ye, Naiyan Wang, and Xiaolong Wang. Featurenerf: Learning generalizable nerfs by distilling foundation models. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 8962–8973, 2023

    2023

  40. [48]

    Gaus- sian grouping: Segment and edit anything in 3d scenes

    Mingqiao Ye, Martin Danelljan, Fisher Yu, and Lei Ke. Gaus- sian grouping: Segment and edit anything in 3d scenes. In European Conference on Computer Vision, pages 162–179. Springer, 2025

    2025

  41. [49]

    Gs-lrm: Large recon- struction model for 3d gaussian splatting

    Kai Zhang, Sai Bi, Hao Tan, Yuanbo Xiangli, Nanxuan Zhao, Kalyan Sunkavalli, and Zexiang Xu. Gs-lrm: Large recon- struction model for 3d gaussian splatting. InEuropean Con- ference on Computer Vision, pages 1–19. Springer, 2024

    2024

  42. [50]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. InCVPR, 2018

    2018

  43. [51]

    Shuaifeng Zhi, Tristan Laidlow, Stefan Leutenegger, and An- drew J. Davison. In-place scene labelling and understanding with implicit scene representation. InICCV, 2021

    2021

  44. [53]

    Feature 3dgs: Supercharging 3d gaussian splatting to enable distilled feature fields

    Shijie Zhou, Haoran Chang, Sicheng Jiang, Zhiwen Fan, Ze- hao Zhu, Dejia Xu, Pradyumna Chari, Suya You, Zhangyang Wang, and Achuta Kadambi. Feature 3dgs: Supercharging 3d gaussian splatting to enable distilled feature fields. InPro- ceedings of the IEEE/CVF Conference on Compu...

    2024

  45. [54]

    Feature4x: Bridging any monocular video to 4d agentic ai with versatile gaussian feature fields

    Shijie Zhou, Hui Ren, Yijia Weng, Shuwang Zhang, Zhen Wang, Dejia Xu, Zhiwen Fan, Suya You, Zhangyang Wang, Leonidas Guibas, et al. Feature4x: Bridging any monocular video to 4d agentic ai with versatile gaussian feature fields. arXiv preprint arXiv:2503.20776, 2025

    2025 arXiv

  46. [55]

    Stereo magnification: Learning view synthesis using multiplane images.arXiv preprint arXiv:1805.09817, 2018

    Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: Learning view synthesis using multiplane images.arXiv preprint arXiv:1805.09817, 2018

    2018 arXiv

  47. [56]

    Fmgs: Foundation model embedded 3d gaussian splatting for holistic 3d scene understanding.arXiv preprint arXiv:2401.01970, 2024

    Xingxing Zuo, Pouya Samangouei, Yunwen Zhou, Yan Di, and Mingyang Li. Fmgs: Foundation model embedded 3d gaussian splatting for holistic 3d scene understanding.arXiv preprint arXiv:2401.01970, 2024. 11 Table 4.Ablation Study. Impact of mask loss on segmentation. Variant mIoU↑A...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.