Pith. sign in

REVIEW 4 major objections 5 minor 52 references

OGGSplat: Open Gaussian Growing for Generalizable Reconstruction with Expanded Field-of-View

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Two photos can grow into a wider semantic 3D scene by outpaining beyond the view cone.

desk verdict A novel and well-engineered method for FOV expansion in generalizable open Gaussian reconstruction, but its benchmark's mIoU is partially self-referential and needs a fix before the headline numbers are trustworthy. read the letter →

arxiv 2506.05204 v1 pith:XI3CY5QE submitted 2025-06-05 cs.CV

Yanbo Wang , Ziyi Wang , Wenzhao Zheng , Jie Zhou , Jiwen Lu This is my paper
classification cs.CV
keywords 3DGaussianSplattinggeneralizablereconstructionopen-vocabularysemanticsimageoutpaintingsemanticinpaintingdiffusionmodelsfield-of-viewexpansionsceneunderstanding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a generalizable 3D Gaussian reconstruction system can grow new, semantically meaningful geometry beyond the region covered by its input photos. Its claim is that the open-vocabulary semantic features already stored in the Gaussians act as a strong prior for image outpainting, so a diffusion-based inpaintor that keeps RGB and semantics aligned can produce content that is both visually plausible and semantically consistent. Those inpainted views are then lifted back into 3D and optimized, expanding the field-of-view. The paper also introduces the Gaussian Outpainting (GO) benchmark to measure both generative quality and semantic plausibility of such extrapolated scenes.

What carries the argument

The load-bearing mechanism is the RGB-semantic consistent inpaintor paired with progressive Gaussian growing. An Edge Translator reads semantic features at the boundary of hollow rendered regions, maps them to class names, and builds a text prompt such as 'a room with wall, table, and chair' for the RGB diffusion inpaintor; a ControlNet-style RGB-to-semantic module then conditions the semantic diffusion inpaintor on the completed image, enforcing spatial alignment between appearance and semantics. Inpainted RGB and semantic maps are lifted to 3D using a predicted monocular depth map, aligned to the existing scene by a scale factor computed from overlapping points, and merged as new Gaussians that are refined with L1, SSIM, and cosine semantic losses. The last two stages repeat iteratively to expand the representation outward.

What would settle it

Render outpainted regions and compare their geometry against ground-truth depth or point clouds from the same viewpoints; if the lifted depth from inpainted images is systematically wrong, the grown Gaussians will be displaced and the expanded scene will show geometric distortion even when RGB and semantics look plausible.

Watch

Extended reading notes

Core claim

The central claim is that OGGSplat consistently outperforms per-scene optimization and feed-forward baselines on expanded-field-of-view reconstruction, improving FID from 50.4 and 46.4 to 37.5 and mIoU from 6.9 and 6.0 to 17.6 on the proposed GO benchmark. The mechanism is a bidirectional RGB-semantic consistent inpaintor: semantic cues near the inpainting boundary are translated into text prompts that guide RGB diffusion, and the inpainted RGB image conditions a semantic diffusion model through a ControlNet-style module, keeping the two modalities pixel-aligned. The inpainted views are lifted by predicted monocular depth and scale-aligned to the original point cloud, then merged into the Gaussian set and optimized with photometric and semantic losses. The paper argues this lets open-vocabulary queries locate objects in regions that were never visible in the input images.

Load-bearing premise

The method assumes that a monocular depth estimate from a single inpainted image, after a scale alignment, is accurate enough to place newly grown Gaussians in the correct global position, even though the paper acknowledges that scaling coordinates alone does not guarantee perfect alignment.

Editorial extensions

If this is right

  • With two overlapping uncalibrated images as input, the reconstruction can be expanded over a horizontal range of ±60 degrees and a vertical range of ±20 degrees, allowing open-vocabulary queries in previously unseen regions.
  • Keeping RGB and semantic inpainting mutually conditioned improves semantic plausibility (mIoU) in extrapolated areas compared with using either control direction alone.
  • The GO benchmark provides a reusable protocol for evaluating both generative fidelity and semantic plausibility of outpainted 3D scenes, using majority-voted labels from five open-vocabulary segmentation models.
  • Because the pipeline builds on a feed-forward reconstructor and only refines grown Gaussians with a few hundred optimization iterations, the approach remains efficient relative to per-scene optimization baselines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the authors leave implicit is that the same bidirectional inpainting loop could be applied to sequential multi-view growth, where each newly grown Gaussian set becomes context for the next outpainted view, rather than sampling a fixed schedule of angles.
  • If monocular depth in open-space or outdoor scenes improves, the method's stated indoor limitation could recede; the paper itself ties its scope to depth estimation quality, so better depth predictors would likely translate directly into broader applicability.
  • The semantic-diffusion branch could be replaced or augmented by stronger open-vocabulary segmentation models, which would probably raise mIoU most on categories where the current APE supervision is weak, such as ceiling.
  • One risk not quantified in the paper is geometric hallucination: visually and semantically plausible outpainted regions may still be geometrically wrong, since no geometric metric is reported for extrapolated regions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. OGGSplat proposes a pipeline that takes two uncalibrated RGB images, reconstructs a generalizable open-vocabulary 3D Gaussian representation using Splatt3R plus a semantic head, renders novel views, fills out-of-view regions with a pair of diffusion models that bidirectionally control RGB and semantic inpainting, and progressively grows new Gaussians initialized by lifting monocular depth from the inpainted images. The paper introduces the Gaussian Outpainting (GO) benchmark on ScanNet++ and reports that OGGSplat outperforms LangSplat and Splatt3R on FID (37.5 vs 50.4/46.4) and mIoU (17.6 vs 6.9/6.0), with qualitative demonstrations on S3DIS and smartphone captures.

Significance. The proposed system is a novel combination of generalizable Gaussian reconstruction, semantic diffusion inpainting, and progressive 3D growing; if the reported results are robust, it is a meaningful step toward semantic 3D outpainting from sparse views. The authors provide code, detailed training settings, ablations of the bidirectional controls, and a new benchmark, all of which are useful contributions. However, the quantitative evidence for the main claims is currently weakened by evaluation entanglement with APE features, absence of error bars and statistical tests, inconsistent context-pair counts between FID and mIoU, and lack of any geometric validation. These issues are fixable and should be addressed before the claims of consistent significant improvement can be accepted.

major comments (4)
  1. [§3.1, Eq. (1); §4.1; §B.5, Eq. (10)] The semantic evaluation is not independent of the method's training signal. The semantic head is supervised with APE features in Eq. (1); the GO benchmark's majority-voted ground truth includes APE [32] among the five segmenters; and the relevance filter in Eq. (10) uses APE query and canonical embeddings. Consequently, a model whose features are aligned with APE is rewarded both during supervision and during mask extraction, whereas LangSplat's CLIP-based features are judged against an APE-aligned oracle. This entanglement can inflate the reported mIoU margin (17.6 vs 6.9/6.0) and undermines the claim of independent semantic plausibility. Please remove or quantify the effect: for example, exclude APE from the voting set and from the relevance computation, or report mIoU with and without APE in the protocol.
  2. [§4.1; Table 1; B.4] The manuscript states that the inpainting/growing/evaluation is repeated five times and averages are reported, but Tables 1 and 2 show no standard deviations, confidence intervals, or significance tests. Additionally, FID is computed with 10 context pairs per scene while the IoU evaluation uses 1 context pair per scene (the IoU in B.4 is averaged over 50×16 images, not 50×16×10). The claim of consistent significant improvement on both metrics therefore rests on metrics evaluated under different input conditions without any measure of variance. Please report per-run statistics and clarify why two different context-pair settings are used.
  3. [§3.3, Eq. (7)] The geometric correctness of the grown Gaussians is not quantitatively assessed. The initialization of newly grown points relies on monocular depth from a single inpainted image aligned only by a global scale factor beta, and the paper acknowledges that "scaling point coordinates alone does not ensure perfect alignment." Since the paper's contribution is to expand the field-of-view of a 3D reconstruction, the absence of any geometry-aware metric—novel-view PSNR/SSIM/LPIPS against held-out ground truth, depth error, or point-cloud consistency—leaves the central claim of geometric coherence unverified. Please add such an evaluation, at least in regions overlapping existing ground-truth geometry, or restrict the claims accordingly.
  4. [§4.1; §B.5] The semantic results depend on several hand-set thresholds and filters: the inpainting mask threshold tau in Eq. (2), the low-confidence region definition (opacity below 0.3), the valid-prediction threshold (opacity above 0.01), and the relevance threshold of 0.5. These thresholds are not ablated and no sensitivity analysis is reported. Since mIoU is a headline result, please report mIoU across a range of these thresholds or provide a principled justification for the chosen values.
minor comments (5)
  1. [§3.2] There is a duplicated article in "Once the the initial Gaussian"; please correct the typo.
  2. [§B.2] The sentence "For RGB image inpainting model Diff sem" appears to refer to Diff rgb; the model names are used inconsistently throughout the appendix (Diff rgb for RGB and Diff sem for semantic diffusion), which creates confusion in the training description.
  3. [Table 1; §4.2] The ceiling category for OGGSplat is 0.1 mIoU whereas LangSplat is 13.4; the discussion in §4.2 mentions this limitation only qualitatively. Please state the per-category exception explicitly so readers can weigh the overall average.
  4. [§4.1; §4.2] The low-confidence region criterion (opacity below 0.3) and the valid-prediction criterion (opacity above 0.01) are introduced in different sections; it is unclear how they interact. Please clarify whether the semantic evaluation is restricted to pixels with 0.01 < opacity < 0.3 or whether the two thresholds apply to different evaluation stages.
  5. [Table 4] The Gaussian Growing column lists "hybrid (Table 3)" as its learning rate, but Table 3 lists parameter-specific learning rates; the entry is confusing and should be reworded.

Circularity Check

1 steps flagged · score 6.0 of 10

GO benchmark mIoU is partially self-referential: APE supervises OGGSplat's semantic head, votes in ground-truth generation, and gates predicted masks via APE relevance, so the semantic margin in Table 1 partly measures self-alignment with the training target.

  1. self definitional [Sec. 3.1 Eq. (1); Sec. 4.1; Sec. B.5 Eq. (10)]
    "we adopt the well-optimized vision-language APE [32] model to efficiently obtain pixel-dense open-vocabulary semantic supervision signals F_gt ... we generate ground truth labels using five state-of-the-art open-vocabulary 2D semantic segmentation models [42, 32, 50, 47, 11] ... we select regions with a relevance score greater than 0.5 as the final predicted category mask ... gqry is the query APE embedding, and gicanon represents the APE embedding of a predefined canonical phrase"

    The semantic head is optimized, via Eq. (1), to match APE features F_gt from [32]. The GO benchmark then obtains its mIoU ground truth by majority vote among five segmenters, including the same APE model ([32] in [42,32,50,47,11]), and retains only predicted masks with an APE-embedding relevance score above 0.5 (Eq. (10)). Thus the Table 1 semantic metric rewards agreement with APE: APE is simultaneously the supervision signal, a voter in the label oracle, and the embedding used to threshold masks. The claimed semantic-plausibility margin therefore reduces, in part, to self-alignment with the model used to train the semantic head, rather than independent external semantic plausibility.

full rationale

The paper's geometry and FID claims are not circular: Splatt3R initialization, depth-based lifting with scale alignment (Eq. 7), and the FID against ScanNet++ validation images are external to the semantic representation. No load-bearing self-citation was found; the cited prior works (Splatt3R, GSemSplat, LangSplat, APE) are not authored by the current authors. The identified circularity is confined to the semantic plausibility evaluation: the semantic head is optimized toward APE features (Eq. 1), APE participates in the majority-vote ground truth (Sec. 4.1), and APE embeddings are used to threshold predicted masks (Eq. 10). Consequently the mIoU numbers in Table 1 are partially self-referential and should not be read as an independent demonstration of semantic consistency. Because the headline claim rests on both FID and mIoU and the qualitative results are not affected, a score of 6 (partial circularity) is appropriate rather than 8 or higher.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method does not introduce new physical entities or forces. It relies on several hand-chosen thresholds and loss weights, and on strong assumptions that APE semantics and monocular depth are reliable in the outpainted regions. The novel algorithmic component is the bidirectional control between RGB and semantic diffusion, which is a new technique but not a newly invented entity.

free parameters (4)
  • inpainting mask threshold tau = not specified
    Eq. (2) defines the mask threshold for which pixels are inpainted; the value is not reported anywhere, despite being a central control of how much content is regenerated and where.
  • relevancy threshold for semantic masks = 0.5
    B.5, Eq. (10): pixels with APE relevance score above 0.5 are considered predicted masks; this threshold is chosen by hand and directly controls the reported mIoU.
  • evaluation opacity thresholds = 0.3 (target region), 0.01 (valid prediction)
    Section 4.1 and 4.2: regions with accumulated opacity below 0.3 define outpainted areas for semantic evaluation, and predictions with opacity above 0.01 are counted as valid; both choices influence the metric outcomes and are not justified by any sensitivity analysis.
  • loss weights lambda1, lambda2, lambda_rgb, lambda_feat = not specified
    Eq. (8-9) balances L1, SSIM, and semantic cosine losses; values are not reported, so exact reproduction requires guessing or contacting the authors.
assumptions (4)
  • domain assumption APE features are a reliable open-vocabulary semantic representation for both supervision and evaluation.
    Section 3.1 uses APE for semantic supervision, and B.5 uses APE in the relevance score; if APE is biased or noisy, the semantic quality claims and the benchmark scores are weakened.
  • domain assumption Splatt3R provides a sufficiently accurate initial Gaussian reconstruction from two uncalibrated images.
    Section 3.1 freezes the pretrained Splatt3R backbone; the method inherits all of Splatt3R's geometric errors and its limited indoor-scene applicability, as acknowledged in the conclusion.
  • ad hoc to paper Monocular depth from a single inpainted image, after global scale alignment, is a valid proxy for 3D structure in grown regions.
    Section 3.3 lifts inpainted pixels into 3D using estimated depth, and explicitly states that scaling does not ensure perfect alignment; there is no quantitative evaluation of geometric accuracy in the outpainted regions.
  • domain assumption Semantic categories near outpainting boundaries can be summarized as text prompts that guide diffusion inpainting appropriately.
    Section 3.2 selects the top 100 semantic categories and builds a prompt like 'a room with ...'; this assumes the boundary semantics are sufficient to produce plausible generation, which is a heuristic that may fail for unusual layouts or objects.

how reviews work

0 comments
Cite this review

Pith. "Pith review of OGGSplat: Open Gaussian Growing for Generalizable Reconstruction with Expanded Field-of-View." pith.science (2026). https://pith.science/paper/XI3CY5QE

@misc{pith2026250605204,
  author       = {Pith},
  title        = {Pith review of: OGGSplat: Open Gaussian Growing for Generalizable Reconstruction with Expanded Field-of-View},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XI3CY5QE}},
  note         = {Machine review of arXiv:2506.05204}
}
read the original abstract

Reconstructing semantic-aware 3D scenes from sparse views is a challenging yet essential research direction, driven by the demands of emerging applications such as virtual reality and embodied AI. Existing per-scene optimization methods require dense input views and incur high computational costs, while generalizable approaches often struggle to reconstruct regions outside the input view cone. In this paper, we propose OGGSplat, an open Gaussian growing method that expands the field-of-view in generalizable 3D reconstruction. Our key insight is that the semantic attributes of open Gaussians provide strong priors for image extrapolation, enabling both semantic consistency and visual plausibility. Specifically, once open Gaussians are initialized from sparse views, we introduce an RGB-semantic consistent inpainting module applied to selected rendered views. This module enforces bidirectional control between an image diffusion model and a semantic diffusion model. The inpainted regions are then lifted back into 3D space for efficient and progressive Gaussian parameter optimization. To evaluate our method, we establish a Gaussian Outpainting (GO) benchmark that assesses both semantic and generative quality of reconstructed open-vocabulary scenes. OGGSplat also demonstrates promising semantic-aware scene reconstruction capabilities when provided with two view images captured directly from a smartphone camera.

Figures

Figures reproduced from arXiv: 2506.05204 by the authors.

Figure 1
Figure 1. We propose OGGSplat, an open Gaussian growing method that expands the field-of-view of generalizable Gaussian reconstruction. The last three rows visualize the rendered images, their semantic maps, and category-specific heatmaps obtained by querying open-vocabulary concepts. produce distorted geometry and semantically implausible content. This highlights an urgent need for a generalizable 3D reconstruction framework… view at source ↗
Figure 2
Figure 2. OGGSplat Architecture. We first initialize an open Gaussian reconstruction, injecting semantic features via an additional semantic head. Then, the RGB-semantic consistent inpaintor applies bidirectional controls between images and semantic maps to ensure semantic plausibility and spatial alignment. Finally, the inpainted regions are lifted back to 3D and optimized to expand the Gaussians. The last two stages are per… view at source ↗
Figure 3
Figure 3. Qualitative comparisons between LangSplat, Splatt3R, and OGGSplat on the GO benchmark. The first row presents RGB images rendered from novel, out-of-scope viewpoints. The second row visualizes the heatmap when querying different text concepts. 4.2 Main Results Baseline Methods for Comparison. We select two representative baselines for comparison: LangSplat [24], a per-scene optimization model, and Splatt3R [34], a g… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Model generalization ability evaluation. Column (a) shows results where the context views are taken from the S3DIS [1]. We query bookshelf and table for each sample, respectively. In column (b), the context views are captured directly using a phone camera, and we query…
Figure 5
Figure 5. Figure 5: Ablations on the effect of semantic diffusion model. We compare open-vocabulary predictions between the SAM+CLIP offline method and our semantic diffusion inpainting module. Model Generalization Ability. Apart from ScanNet++ used for training, we also test OGGSplat’s g…
Figure 6
Figure 6. Figure 6: Qualitative comparison of bidirectional control. Row 1 shows the context images and the incomplete renderings from novel views. Rows 2 to 4 correspond to the ablation settings in [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Qualitative comparison between hybrid (shared-weight) and separate diffusion UNet [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: The architecture of the ControlNet [51]. To ensure spatial alignment between the in￾painted RGB image and its corresponding se￾mantic map, we adopt a control mechanism in￾spired by ControlNet [51], where the RGB im￾age serves as guidance for the generation of the seman…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

52 extracted references · 28 canonical work pages

  1. [32]

    Aligning and prompting everything all at once for universal visual perception

    Yunhang Shen, Chaoyou Fu, Peixian Chen, Mengdan Zhang, Ke Li, Xing Sun, Yunsheng Wu, Shaohui Lin, and Rongrong Ji. Aligning and prompting everything all at once for universal visual perception. In CVPR, 2024

    2024

  2. [1]

    3d semantic parsing of large-scale indoor spaces

    Iro Armeni, Ozan Sener, Amir R Zamir, Helen Jiang, Ioannis Brilakis, Martin Fischer, and Silvio Savarese. 3d semantic parsing of large-scale indoor spaces. In CVPR, 2016

    work page 2016

  3. [2]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In ICCV, 2021

    work page 2021

  4. [3]

    pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction

    David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann. pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction. In CVPR, 2024

    work page 2024

  5. [4]

    Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images

    Yuedong Chen, Haofei Xu, Chuanxia Zheng, Bohan Zhuang, Marc Pollefeys, Andreas Geiger, Tat-Jen Cham, and Jianfei Cai. Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images. In ECCV, 2024

    work page 2024

  6. [5]

    Depth-regularized optimization for 3d gaussian splatting in few-shot images

    Jaeyoung Chung, Jeongtaek Oh, and Kyoung Mu Lee. Depth-regularized optimization for 3d gaussian splatting in few-shot images. In CVPR, 2024

    work page 2024

  7. [6]

    Light- gaussian: Unbounded 3d gaussian compression with 15x reduction and 200+ fps

    Zhiwen Fan, Kevin Wang, Kairun Wen, Zehao Zhu, Dejia Xu, Zhangyang Wang, et al. Light- gaussian: Unbounded 3d gaussian compression with 15x reduction and 200+ fps. NeurIPS, 2024

    work page 2024

  8. [7]

    Flowr: Flowing from sparse to dense 3d reconstructions

    Tobias Fischer, Samuel Rota Bulò, Yung-Hsu Yang, Nikhil Varma Keetha, Lorenzo Porzi, Norman Müller, Katja Schwarz, Jonathon Luiten, Marc Pollefeys, and Peter Kontschieder. Flowr: Flowing from sparse to dense 3d reconstructions. arXiv preprint arXiv:2504.01647, 2025

    arXiv 2025

Show all 52 references
  1. [8]

    Colmap- free 3d gaussian splatting

    Yang Fu, Sifei Liu, Amey Kulkarni, Jan Kautz, Alexei A Efros, and Xiaolong Wang. Colmap- free 3d gaussian splatting. In CVPR, 2024

    2024

  2. [9]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. NeurIPS, 2017

    2017

  3. [10]

    Sparselgs: Sparse view language embedded gaussian splatting

    Jun Hu, Zhang Chen, Zhong Li, Yi Xu, and Juyong Zhang. Sparselgs: Sparse view language embedded gaussian splatting. arXiv preprint arXiv:2412.02245, 2024

    2024 arXiv

  4. [11]

    Collaborative vision-text representation optimizing for open-vocabulary segmentation

    Siyu Jiao, Hongguang Zhu, Jiannan Huang, Yao Zhao, Yunchao Wei, and Humphrey Shi. Collaborative vision-text representation optimizing for open-vocabulary segmentation. In ECCV, 2024

    2024

  5. [12]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 2023

    2023

  6. [13]

    Lerf: Language embedded radiance fields

    Justin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa, and Matthew Tancik. Lerf: Language embedded radiance fields. In ICCV, 2023

    2023

  7. [14]

    Auto-encoding variational bayes, 2013

    Diederik P Kingma, Max Welling, et al. Auto-encoding variational bayes, 2013

    2013

  8. [15]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In ICCV, 2023

    2023

  9. [16]

    Instancegaussian: Appearance-semantic joint gaussian representation for 3d instance- level perception

    Haijie Li, Yanmin Wu, Jiarui Meng, Qiankun Gao, Zhiyao Zhang, Ronggang Wang, and Jian Zhang. Instancegaussian: Appearance-semantic joint gaussian representation for 3d instance- level perception. arXiv preprint arXiv:2411.19235, 2024

    2024 arXiv

  10. [17]

    Spc-gs: Gaussian splatting with semantic-prompt consistency for indoor open-world free-view synthesis from sparse inputs

    Guibiao Liao, Qing Li, Zhenyu Bao, Guoping Qiu, and Kanglin Liu. Spc-gs: Gaussian splatting with semantic-prompt consistency for indoor open-world free-view synthesis from sparse inputs. arXiv preprint arXiv:2503.12535, 2025. 13

    2025 arXiv

  11. [18]

    Scaffold- gs: Structured 3d gaussians for view-adaptive rendering

    Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua Lin, and Bo Dai. Scaffold- gs: Structured 3d gaussians for view-adaptive rendering. In CVPR, 2024

    2024

  12. [19]

    Segment then splat: A unified approach for 3d open-vocabulary segmentation based on gaussian splatting

    Yiren Lu, Yunlai Zhou, Yiran Qiao, Chaoda Song, Tuo Liang, Jing Ma, and Yu Yin. Segment then splat: A unified approach for 3d open-vocabulary segmentation based on gaussian splatting. arXiv preprint arXiv:2503.22204, 2025

    2025

  13. [20]

    Repaint: Inpainting using denoising diffusion probabilistic models

    Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In CVPR, 2022

    2022

  14. [21]

    Nerf: Representing scenes as neural radiance fields for view synthesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoor- thi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 2021

    2021

  15. [22]

    Coherentgs: Sparse novel view synthesis with coherent 3d gaussians

    Avinash Paliwal, Wei Ye, Jinhui Xiong, Dmytro Kotovenko, Rakesh Ranjan, Vikas Chandra, and Nima Khademi Kalantari. Coherentgs: Sparse novel view synthesis with coherent 3d gaussians. In ECCV, 2024

    2024

  16. [23]

    Unidepth: Universal monocular metric depth estimation

    Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. Unidepth: Universal monocular metric depth estimation. In CVPR, 2024

    2024

  17. [24]

    Langsplat: 3d language gaussian splatting

    Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister. Langsplat: 3d language gaussian splatting. In CVPR, 2024

    2024

  18. [25]

    Gls: Geometry-aware 3d language gaussian splatting

    Jiaxiong Qiu, Liu Liu, Zhizhong Su, and Tianwei Lin. Gls: Geometry-aware 3d language gaussian splatting. arXiv preprint arXiv:2411.18066, 2024

    2024 arXiv

  19. [26]

    Feature splatting: Language-driven physics-based scene synthesis and editing

    Ri-Zhao Qiu, Ge Yang, Weijia Zeng, and Xiaolong Wang. Feature splatting: Language-driven physics-based scene synthesis and editing. arXiv preprint arXiv:2404.01223, 2024

    2024 arXiv

  20. [27]

    Goi: Find 3d gaussians of interest with an optimizable open-vocabulary semantic-space hyperplane

    Yansong Qu, Shaohui Dai, Xinyang Li, Jianghang Lin, Liujuan Cao, Shengchuan Zhang, and Rongrong Ji. Goi: Find 3d gaussians of interest with an optimizable open-vocabulary semantic-space hyperplane. In ACM MM, 2024

    2024

  21. [28]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, 2021

    2021

  22. [29]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714, 2024

    2024 arXiv

  23. [30]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, 2022

    2022

  24. [31]

    Structure-from-motion revisited

    Johannes L Schonberger and Jan-Michael Frahm. Structure-from-motion revisited. In CVPR, 2016

    2016

  25. [33]

    Language embedded 3d gaussians for open-vocabulary scene understanding

    Jin-Chuan Shi, Miao Wang, Hao-Bin Duan, and Shao-Hua Guan. Language embedded 3d gaussians for open-vocabulary scene understanding. In CVPR, 2024

    2024

  26. [34]

    Splatt3r: Zero-shot gaussian splatting from uncalibrated image pairs

    Brandon Smart, Chuanxia Zheng, Iro Laina, and Victor Adrian Prisacariu. Splatt3r: Zero-shot gaussian splatting from uncalibrated image pairs. arXiv preprint arXiv:2408.13912, 2024

    2024 arXiv

  27. [35]

    Cags: Open-vocabulary 3d scene under- standing with context-aware gaussian splatting

    Wei Sun, Yanzhao Zhou, Jianbin Jiao, and Yuan Li. Cags: Open-vocabulary 3d scene under- standing with context-aware gaussian splatting. arXiv preprint arXiv:2504.11893, 2025

    2025 arXiv

  28. [36]

    Dust3r: Geometric 3d vision made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. In CVPR, 2024. 14

    2024

  29. [37]

    Gsemsplat: Generalizable semantic 3d gaussian splatting from uncalibrated image pairs

    Xingrui Wang, Cuiling Lan, Hanxin Zhu, Zhibo Chen, and Yan Lu. Gsemsplat: Generalizable semantic 3d gaussian splatting from uncalibrated image pairs. arXiv preprint arXiv:2412.16932, 2024

    2024 arXiv

  30. [38]

    Motionctrl: A unified and flexible motion controller for video generation

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. In ACM SIGGRAPH 2024 Conference Papers, 2024

    2024

  31. [39]

    Opengaussian: Towards point-level 3d gaussian-based open vocabulary understanding

    Yanmin Wu, Jiarui Meng, Haijie Li, Chenming Wu, Yahao Shi, Xinhua Cheng, Chen Zhao, Haocheng Feng, Errui Ding, Jingdong Wang, et al. Opengaussian: Towards point-level 3d gaussian-based open vocabulary understanding. arXiv preprint arXiv:2406.02058, 2024

    2024 arXiv

  32. [40]

    Smartbrush: Text and shape guided object inpainting with diffusion model

    Shaoan Xie, Zhifei Zhang, Zhe Lin, Tobias Hinz, and Kun Zhang. Smartbrush: Text and shape guided object inpainting with diffusion model. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22428–22437, 2023

    2023

  33. [41]

    Sparsegs: Real-time 360 {\deg} sparse view synthesis using gaussian splatting

    Haolin Xiong, Sairisheek Muttukuru, Rishi Upadhyay, Pradyumna Chari, and Achuta Kadambi. Sparsegs: Real-time 360 {\deg} sparse view synthesis using gaussian splatting. arXiv preprint arXiv:2312.00206, 2023

    2023 arXiv

  34. [42]

    Open-vocabulary panoptic segmentation with text-to-image diffusion models

    Jiarui Xu, Sifei Liu, Arash Vahdat, Wonmin Byeon, Xiaolong Wang, and Shalini De Mello. Open-vocabulary panoptic segmentation with text-to-image diffusion models. In CVPR, 2023

    2023

  35. [43]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In CVPR, 2024

    2024

  36. [44]

    Depth anything v2

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. NeurIPS, 2024

    2024

  37. [45]

    Gaussian grouping: Segment and edit anything in 3d scenes

    Mingqiao Ye, Martin Danelljan, Fisher Yu, and Lei Ke. Gaussian grouping: Segment and edit anything in 3d scenes. In ECCV, 2024

    2024

  38. [46]

    Scannet++: A high-fidelity dataset of 3d indoor scenes

    Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai. Scannet++: A high-fidelity dataset of 3d indoor scenes. In ICCV, 2023

    2023

  39. [47]

    Convolutions die hard: Open-vocabulary segmentation with single frozen convolutional clip

    Qihang Yu, Ju He, Xueqing Deng, Xiaohui Shen, and Liang-Chieh Chen. Convolutions die hard: Open-vocabulary segmentation with single frozen convolutional clip. NeurIPS, 2023

    2023

  40. [48]

    Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis

    Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis. arXiv preprint arXiv:2409.02048, 2024

    2024 arXiv

  41. [49]

    Mip-splatting: Alias-free 3d gaussian splatting

    Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. Mip-splatting: Alias-free 3d gaussian splatting. In CVPR, 2024

    2024

  42. [50]

    Maskclip++: A mask-based clip fine-tuning framework for open-vocabulary image segmentation

    Quan-Sheng Zeng, Yunheng Li, Daquan Zhou, Guanbin Li, Qibin Hou, and Ming-Ming Cheng. Maskclip++: A mask-based clip fine-tuning framework for open-vocabulary image segmentation. arXiv preprint arXiv:2412.11464, 2024

    2024 arXiv

  43. [51]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, 2023

    2023

  44. [52]

    Fsgs: Real-time few-shot view synthesis using gaussian splatting

    Zehao Zhu, Zhiwen Fan, Yifan Jiang, and Zhangyang Wang. Fsgs: Real-time few-shot view synthesis using gaussian splatting. In ECCV, 2024. 15

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.