Pith. sign in

REVIEW 4 major objections 6 minor 82 references

A three-stage pipeline turns text into coherent ego-centric 3D Gaussian scenes by fixing multi-view consistency and depth geometry.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 23:43 UTC pith:22C7ISMD

load-bearing objection Solid systems paper for ego-centric text-to-3DGS with two usable regularizers; the random-VGG story is the softest link but ablations still show gains. the 4 major comments →

arxiv 2607.03819 v2 pith:22C7ISMD submitted 2026-07-04 cs.GR cs.AI

CGGS: Consistency-Augmented Geometric Gaussian Splatting for Ego-centric 3D Scene Generation

Zhenyu Sun , Xiaohan Zhang , Qi Liu , Huan Wang This is my paper
classification cs.GR cs.AI
keywords 3D Gaussian splattingego-centric generationmulti-view diffusionconsistency-augmented lossmutual information depth losstext-to-3Doptical flow depth
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Ego-centric 3D scene generation is hard because outward-facing camera views barely overlap and each view pulls the reconstruction in its own direction, producing inconsistent textures and warped geometry. CGGS claims that three linked stages solve this: a multi-view diffusion model regularized by a consistency-augmented loss that projects conflicting per-view gradients into a shared feature space; a flow-and-point-track depth estimator that turns those 2D images into a dense point-cloud layout without relying on classical structure-from-motion; and a 3D Gaussian optimizer that uses mutual-information depth loss plus hierarchical extra cameras to refine both appearance and structure. On 24 indoor and outdoor scenes the method reports higher text-image alignment and higher rendering fidelity than prior text-to-3D baselines. A sympathetic reader cares because the same pipeline works across domains and produces usable novel views from purely textual prompts, something panoramic or progressive-inpainting approaches have struggled to deliver without distortion.

Core claim

CGGS shows that consistency-augmented multi-view diffusion priors, followed by optical-flow-guided dense layout initialization and mutual-information depth supervision under hierarchical camera expansion, produce text-aligned ego-centric 3D Gaussian scenes that outperform earlier methods on both semantic metrics and reconstruction metrics.

What carries the argument

The consistency-augmented loss Laug, which routes per-view score-matching residuals through a frozen random VGG-16 projector so that gradient updates from different ego-centric views are forced into a common multi-scale subspace, reducing inter-view conflict while preserving text alignment.

Load-bearing premise

The claim rests on the idea that a frozen, randomly initialized multi-layer CNN is enough to turn conflicting per-view gradients into a single coherent update that simultaneously improves consistency and text fidelity.

What would settle it

Train the identical multi-view generator with and without the consistency-augmented loss on the same ego-centric trajectory set and measure whether multi-view CLIP score and cross-view feature similarity both drop when the loss is removed; if they do not, the central regularization claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Text-driven ego-centric 3D assets become usable for AR/VR and robotics without manual multi-view capture.
  • Optical-flow-plus-point-track depth estimation can replace classical SfM for sparse outward-facing trajectories.
  • Mutual-information depth loss can be dropped into other 3D Gaussian pipelines that suffer from over-smoothed geometry.
  • Hierarchical supplementary cameras give a practical way to reduce viewpoint bias during per-scene optimization.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same gradient-projection idea may transfer to any multi-view diffusion setting where cameras share little overlap, not only ego-centric 360° layouts.
  • Because the layout stage already enforces long-range point tracks, the method could be extended to dynamic scenes by replacing static tracks with temporal ones.
  • If the random VGG projector is replaced by a learned but still frozen multi-scale feature extractor, further gains in consistency might appear without re-introducing ImageNet bias.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. CGGS is a three-stage text-to-3D pipeline for ego-centric scenes. An Ego-centric Generator fine-tunes multi-view latent diffusion (CAA blocks) with a consistency-augmented loss that routes per-view noise residuals through a frozen, He-initialized VGG-16 projector (Eqs. 5–7). A Layout Decorator builds dense point-cloud layouts from optical flow and long-term point tracks rather than SfM. A Geometric Refiner then optimizes 3D Gaussians with a Mutual Information Depth (MID) loss (Eqs. 12–14) plus hierarchical supplementary cameras. On 24 indoor/outdoor scenes the method reports higher CLIP Score (26.253), Q-Align (0.839), PSNR (37.345), SSIM (0.977) and lower LPIPS (0.0193) than Text2Room, LucidDreamer, Director3D and DreamScene360, with ablations isolating each module.

Significance. Ego-centric (outward-facing, limited-overlap) text-to-3D remains under-served relative to object-centric or panoramic pipelines; a practical system that improves multi-view coherence and geometric initialization would be useful for AR/VR, robotics and synthetic data. Strengths include a clean modular design, component-wise ablations (Tables II–IV, Figs. 7–8), domain-free qualitative results despite indoor-only CAA fine-tuning, and an explicit project page. The empirical gains on both generation and self-reconstruction metrics, if robust under fairer protocols, constitute a solid systems contribution for TIP-style venues.

major comments (4)
  1. [Sec. IV-C, Eqs. (12)–(13)] Sec. IV-C, Eqs. (12)–(13): Mutual information is written for continuous depth maps, yet the manuscript never states how I(D_render; D_gt) is estimated (histogram binning, KDE, neural estimator, discretization, etc.). Continuous MI estimation is non-trivial and directly determines the behavior of L_MID versus Pearson depth loss. Without this specification the Geometric Refiner is not reproducible and the claim that MID better preserves high-frequency structure (vs. PD in Table IV / Fig. 8) cannot be verified.
  2. [Sec. V-D, Table I] Sec. V-D / Table I: LucidDreamer is seeded with the first frame produced by CGGS. This couples the baseline to the proposed generator’s style and content and inflates the apparent gap on both CLIP-based and reconstruction metrics. A fair comparison requires an independent text-only (or independently generated) seed, or at minimum an ablation that reports LucidDreamer with a neutral seed. The current protocol weakens the central outperformance claim.
  3. [Sec. IV-A, Eqs. (5)–(7)] Sec. IV-A, Eqs. (5)–(7): The paper asserts that a frozen random VGG-16 Jacobian projects conflicting per-view gradients onto a shared singular subspace that systematically improves geometric coherence. No measurement of gradient cosine similarity, singular-vector alignment, or frequency content is provided; Table II shows only modest multi-view CLIP gains (25.869 o25.949) while the larger panorama gain could arise from multi-scale smoothing or the extra loss weight alone. Either supply a direct diagnostic of the claimed mechanism or soften the theoretical language so that the contribution rests on the empirical regularizer rather than an untested inductive-bias story.
  4. [Table I, Sec. V-D] Table I reconstruction columns (PSNR/SSIM/LPIPS): These metrics measure how well 3DGS fits the method’s own generated multi-view images, not absolute geometric accuracy against real multi-view ground truth. Extremely high numbers (PSNR 37.3, SSIM 0.977) are therefore expected once the Layout Decorator supplies consistent depths, and do not by themselves establish superior scene geometry. Clarify this interpretation and, if possible, add a cross-view depth-consistency or novel-view geometric metric that is independent of the training views.
minor comments (6)
  1. [Sec. V-A] Implementation details (Sec. V-A) refer to “Geometric Decorator” while the rest of the paper uses “Geometric Refiner”; unify the name.
  2. [Tables I–V] No error bars or statistical tests are reported on the 24-scene averages in Tables I–V; even simple standard deviations would strengthen the quantitative claims.
  3. [Fig. 1] Fig. 1 caption is very long and mixes three distinct failure modes; consider splitting into sub-captions or a short bullet list for readability.
  4. [Eq. (17)] Eq. (17) re-introduces λ_SSIM without stating its relation to the earlier (1−λ_MID) weighting in Eq. (14); a single consistent loss formula would help.
  5. [Table I] Director3D reconstruction metrics are left blank with only a brief note; a short qualitative or proxy metric would make Table I more complete.
  6. [Sec. VI] Typo: “3D Gussians” in the Conclusion; also “Geometric Decorator” vs Refiner already noted.

Circularity Check

0 steps flagged

No circularity: empirical pipeline with external training data and independent ablations; no equation or claim reduces reported metrics to fitted inputs by construction.

full rationale

CGGS is a standard empirical text-to-3D pipeline. The Ego-centric Generator fine-tunes CAA blocks of a multi-view LDM on Matterport3D with the proposed Laug (Eqs. 5–7) that routes residuals through a frozen He-initialized VGG-16; Layout Decorator trains a flow+point-track depth estimator on RealEstate-10k/CO3Dv2; Geometric Refiner optimizes 3DGS with MID (Eqs. 12–14) plus hierarchical cameras. All components are trained on external real-world datasets and evaluated on held-out GPT-4 prompts across 24 scenes with standard metrics (CLIP, Q-Align, PSNR/SSIM/LPIPS). Ablations (Tabs. II–IV, Figs. 7–8) isolate each term without redefining the target quantities. No uniqueness theorem, self-citation chain, or fitted parameter is re-labeled as a first-principles prediction; the project-page link is non-load-bearing. The derivation chain is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 3 invented entities

The central empirical claim rests on a handful of hand-chosen scalars, standard diffusion and 3DGS assumptions, and three newly introduced algorithmic entities whose only evidence is the paper’s own ablations.

free parameters (5)
  • λ_aug = 0.5
    Weight of the consistency-augmented loss; set to 0.5 by hand (Sec. V-A).
  • λ_MID = 0.05
    Weight of the mutual-information depth loss; set to 0.05 (Sec. V-A).
  • N (number of base views) = 8
    Number of ego-centric views generated by the diffusion model; fixed at 8 with 45° rotation (Sec. V-A).
  • hierarchical stages n = 3
    Number of stages that progressively add virtual cameras; set to 3 (Sec. V-A).
  • λ_SSIM = 0.2
    SSIM weight inside the reconstruction loss; set to 0.2 (Sec. V-A).
axioms (4)
  • domain assumption Camera trajectories C are known a priori and can be used both for multi-view generation and for back-projection.
    Stated in problem formulation (Sec. IV) and used throughout Layout Decorator and Geometric Refiner.
  • ad hoc to paper A frozen, randomly He-initialized VGG-16 defines a stationary multi-scale metric that aligns conflicting per-view gradients without introducing semantic bias.
    Core justification of Laug (Eqs. 5–6, Sec. IV-A); no external proof is supplied.
  • domain assumption Optical flow (RAFT) plus long-term point tracks (CoTracker) supply sufficiently accurate correspondences to supervise a depth network under narrow-baseline ego-centric views.
    Assumed in Layout Decorator (Sec. IV-B); relies on off-the-shelf estimators trained on different data.
  • ad hoc to paper Mutual information between rendered and estimated depth maps is a better geometric regularizer than Pearson correlation for 3DGS under ego-centric sparsity.
    Claimed in Sec. IV-C and supported only by the paper’s own ablation (Table IV).
invented entities (3)
  • Consistency-Augmented Loss (Laug) with frozen random VGG-16 projector no independent evidence
    purpose: Harmonize multi-view score-matching gradients to improve both consistency and text alignment.
    Introduced in Sec. IV-A; independent evidence is limited to the paper’s ablation (Table II, Fig. 7).
  • Mutual Information Depth Loss (MID) no independent evidence
    purpose: Provide non-linear, entropy-based depth supervision that preserves high-frequency geometry better than linear correlation.
    Defined in Eqs. 12–13 (Sec. IV-C); only evidence is internal ablation against Pearson depth loss.
  • Layout Decorator (Flow-Depth Estimator) no independent evidence
    purpose: Produce a dense, consistent point-cloud initialization from sparse ego-centric views where classical SfM fails.
    Described in Sec. IV-B; comparison to COLMAP is internal (Table III).

pith-pipeline@v1.1.0-grok45 · 24845 in / 3388 out tokens · 27001 ms · 2026-07-11T23:43:14.123846+00:00 · methodology

0 comments
read the original abstract

Challenges remain in ego-centric 3D scene generation due to limited view overlap and the dominant influence of individual perspectives on scene interpretation. These factors hinder the creation of viewpoint-consistent and semantically aligned visual content, as well as the construction of accurate geometric structures. In this paper, we propose CGGS, a text-to-3D framework aiming to enhance 3D-content-awareness and address geometric distortions in ego-centric scene generation. Firstly, the Ego-centric Generator is proposed by fine-tuning a Multi-View Latent Diffusion Model with consistency-augmented loss to generate consistent, high-fidelity 2D content aligned with textual descriptions. Then, Layout Decorator leverages optical flow and point-track correspondence to estimate depth, therefore producing dense point clouds as coarse layouts from the ego-centric 2D priors. Building on this initialization, Geometric Refiner is proposed to enhance 3D Gaussian reconstruction via an entropy-based Mutual Information Depth Loss (MID) combined with a hierarchical optimization scheme for improving visual quality and geometric structure. Comprehensive experiments demonstrate that CGGS outperforms previous methods in generating coherent and accurate text-driven 3D scenes. Project page: [https://cggs-26.github.io/cggs26/](https://cggs-26.github.io/cggs26/).

Figures

Figures reproduced from arXiv: 2607.03819 by Huan Wang, Qi Liu, Xiaohan Zhang, Zhenyu Sun.

Figure 1
Figure 1. Figure 1: Visualization of geometric distortions in panoramic generation (using DreamScene360 [1] as an example). 1) Insufficient Text-Content Alignment: Significant textual details are omitted in the generation, such as the absence of ”mismatched frames” on the gallery wall despite being explicitly specified in the prompt. 2) Polar Geometric Distortions: Due to the inherent nature of equirectangular projection, sev… view at source ↗
Figure 2
Figure 2. Figure 2: With text prompts as input, CGGS employs three core components: the Ego-centric Generator creates ego-centric 2D priors, the Layout Decorator proposes additional scene details, and the Geometric Refiner further enhances the geometric structure and visual quality. Despite improvements for egocentric scenarios, these methods still lead to geometric and textural artifacts due to inherent inpainting limitation… view at source ↗
Figure 3
Figure 3. Figure 3: Pipeline of CGGS. It primarily comprises the Consistency-Augmented MV-LDM as Ego-centric Generator, the Flow-Depth Estimator as Layout Decorator, and the 3D Gaussian Optimization combined with MID Loss and the hierarchical optimization strategy, serving as Geometric Refiner. on the 2D displacement between s l ∗ and s, which provides relative location within local neighborhoods. CAA blocks are integrated in… view at source ↗
Figure 4
Figure 4. Figure 4: Generation results of CGGS for ego-centric multi-view priors, gaussian point clouds, novel view synthesis, and depth maps. Our method generates harmonious, domain-free 3D scenes from ego-centric views, highly aligned with complex textual descriptions. C. Geometric Refiner Based on the point clouds and structural information gen￾erated by the Layout Decorator, Geometric Refiner further incorporates depth-aw… view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparison between CGGS with other baselines. Our CGGS produces multi-view images with rich detail and superior semantic coherence, showcasing domain-agnosticity. Our results outperform other works with an accurately detailed description and unified 3D consistency. Specifically, DreamScene360 generates visual results with less major content in the horizon field; While Director3D is capable of d… view at source ↗
Figure 6
Figure 6. Figure 6: Poor quality of the rendered views from Text2Room [13] on outdoor scenes. There are large black artifacts and missing geometry. from the multi-view sequence generated by CGGS as its initial input. As for Director3D, since it does not utilize any intermediate reference images, we do not evaluate its image reconstruction quality. According to Tab. I, our CGGS method demonstrates a com￾prehensive advantage ov… view at source ↗
Figure 7
Figure 7. Figure 7: Ablation study on consistency-augmented loss Laug. Without Laug, cross-view texture discrepancies become pronounced, with abrupt background artifacts (e.g., exposed ceilings in bedroom scenes) and physically implausible anomalies (e.g., floating, distorted trees on beaches) emerging. TABLE II ABLATION STUDIES OF EGO-CENTRIC GENERATOR ON CONSISTENCY-AUGMENTED LOSS Laug . WE REPORT THE TRAINING TIME OF THE E… view at source ↗
Figure 8
Figure 8. Figure 8: The qualitative comparison demonstrates that incorpo [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗
Figure 8
Figure 8. Figure 8: Ablation studies of Geometric-Refiner on MID loss and hierarchical optimization. Here we demonstrate the qualitative comparison between the ground [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Additional generation results from CGGS. Our work can generate richly detailed, high-fidelity scenes with considerable diversity while ensuring cohesive semantic content and a harmonized visual style that faithfully reflects even the most intricate textual descriptions [PITH_FULL_IMAGE:figures/full_fig_p012_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Additional generation results from CGGS. Our work can generate richly detailed, high-fidelity scenes with considerable diversity while ensuring cohesive semantic content and a harmonized visual style that faithfully reflects even the most intricate textual descriptions. TABLE V QUANTITATIVE COMPARISON ON OUT-OF-DOMAIN SCENES. OUR CGGS ACHIEVES COMPETITIVE IMAGE QUALITY AND STRONG SEMANTIC ALIGNMENT, AND I… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

82 extracted references · 20 linked inside Pith

  1. [1]

    Dreamscene360: Unconstrained text-to-3d scene generation with panoramic gaussian splatting,

    S. Zhou, Z. Fan, D. Xu, H. Chang, P. Chari, T. Bharadwaj, S. You, Z. Wang, and A. Kadambi, “Dreamscene360: Unconstrained text-to-3d scene generation with panoramic gaussian splatting,” inECCV, 2024

    2024

  2. [2]

    Glide: Towards photorealistic image gen- eration and editing with text-guided diffusion models,

    A. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. McGrew, I. Sutskever, and M. Chen, “Glide: Towards photorealistic image gen- eration and editing with text-guided diffusion models,”arXiv preprint arXiv:2112.10741, 2021

    Pith/arXiv arXiv 2021

  3. [3]

    Hierarchical text-conditional image generation with clip latents,

    A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical text-conditional image generation with clip latents,”arXiv preprint arXiv:2204.06125, 2022

    Pith/arXiv arXiv 2022

  4. [4]

    Photoreal- istic text-to-image diffusion models with deep language understanding,

    C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. G. Lopes, B. K. Ayan, T. Salimanset al., “Photoreal- istic text-to-image diffusion models with deep language understanding,” inNeurIPS, 2022

    2022

  5. [5]

    High resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High resolution image synthesis with latent diffusion models,” inCVPR, 2022

    2022

  6. [6]

    Nerf: Representing scenes as neural radiance fields for view synthesis,

    B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,” inECCV, 2020

    2020

  7. [7]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, and et al., “Learning transferable visual models from natural language supervision,” inICML, 2021

    2021

  8. [8]

    Zero-shot text-guided object generation with dream fields,

    A. Jain, B. Mildenhall, J. T. Barron, P. Abbeel, and B. Poole, “Zero-shot text-guided object generation with dream fields,” inCVPR, 2022

    2022

  9. [9]

    Dreamfusion: Text- to-3d using 2d diffusion,

    B. Poole, A. Jain, J. T. Barron, and B. Mildenhall, “Dreamfusion: Text- to-3d using 2d diffusion,” inICLR, 2023

    2023

  10. [10]

    Magic3d: High-resolution text-to-3d content creation,

    C.-H. Lin, J. Gao, L. Tang, T. Takikawa, X. Zeng, X. Huang, K. Kreis, S. Fidler, M.-Y . Liu, and T.-Y . Lin, “Magic3d: High-resolution text-to-3d content creation,” inCVPR, 2023

    2023

  11. [11]

    Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation,

    H. Wang, X. Du, J. Li, R. A. Yeh, and G. Shakhnarovich, “Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation,” in CVPR, 2023

    2023

  12. [12]

    Mvdream: Multi- view diffusion for 3d generation,

    Y . Shi, P. Wang, J. Ye, L. Mai, K. Li, and X. Yang, “Mvdream: Multi- view diffusion for 3d generation,” inICLR, 2024

    2024

  13. [13]

    Text2room: Extracting textured 3d meshes from 2d text-to-image models,

    L. H ¨ollein, A. Cao, A. Owens, J. Johnson, and M. Nießner, “Text2room: Extracting textured 3d meshes from 2d text-to-image models,” inICCV, 2023

    2023

  14. [14]

    Luciddreamer: Domain-free generation of 3d gaussian splatting scenes,

    J. Chung, S. Lee, H. Nam, J. Lee, and K. M. Lee, “Luciddreamer: Domain-free generation of 3d gaussian splatting scenes,”arXiv preprint arXiv:2311.13384, 2023

    Pith/arXiv arXiv 2023

  15. [15]

    Text2nerf: Text-driven 3d scene generation with neural radiance fields,

    J. Zhang, X. Li, Z. Wan, C. Wang, and J. Liao, “Text2nerf: Text-driven 3d scene generation with neural radiance fields,”IEEE Transactions on Visualization and Computer Graphics, 2024

    2024

  16. [16]

    Wonderjourney: Going from anywhere to everywhere,

    H.-X. Yu, H. Duan, J. Hur, K. Sargent, M. Rubinstein, W. T. Freeman, F. Cole, D. Sun, N. Snavely, J. Wuet al., “Wonderjourney: Going from anywhere to everywhere,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  17. [17]

    Realmdreamer: Text-driven 3d scene generation with inpainting and depth diffusion,

    J. Shriram, A. Trevithick, L. Liu, and R. Ramamoorthi, “Realmdreamer: Text-driven 3d scene generation with inpainting and depth diffusion,” in 3DV, 2025

    2025

  18. [18]

    Recurrent diffusion for 3d point cloud generation from a single image,

    Y . Zhou, D. Ye, H. Zhang, X. Xu, H. Sun, Y . Xu, X. Liu, and Y . Zhou, “Recurrent diffusion for 3d point cloud generation from a single image,” IEEE Transactions on Image Processing, vol. 34, pp. 1753–1765, 2025

    2025

  19. [19]

    3d gaussian splatting for real-time radiance field rendering,

    B. Kerbl, G. Kopanas, T. Leimk ¨uhler, and G. Drettakis, “3d gaussian splatting for real-time radiance field rendering,”ACM Transactions on Graphics, 2023

    2023

  20. [20]

    Scene- dreamer360: Text-driven 3d-consistent scene generation with panoramic gaussian splatting,

    W. Li, F. Cai, Y . Mi, Z. Yang, W. Zuo, X. Wang, and X. Fan, “Scene- dreamer360: Text-driven 3d-consistent scene generation with panoramic gaussian splatting,”arXiv preprint arXiv:2408.13711, 2024

    Pith/arXiv arXiv 2024

  21. [21]

    Holodreamer: Holistic 3d panoramic world generation from text descriptions,

    H. Zhou, X. Cheng, W. Yu, Y . Tian, and L. Yuan, “Holodreamer: Holistic 3d panoramic world generation from text descriptions,”arXiv preprint arXiv:2407.15187, 2024

    Pith/arXiv arXiv 2024

  22. [22]

    Director3d: Real-world camera trajectory and 3d scene generation from text,

    X. Li, Z. Lai, L. Xu, Y . Qu, L. Cao, S. Zhang, B. Dai, and R. Ji, “Director3d: Real-world camera trajectory and 3d scene generation from text,” inNeurIPS, 2024

    2024

  23. [23]

    Prometheus: 3d-aware latent diffusion models for feed-forward text-to-3d scene generation,

    Y . Yang, J. Shao, X. Li, Y . Shen, A. Geiger, and Y . Liao, “Prometheus: 3d-aware latent diffusion models for feed-forward text-to-3d scene generation,”arXiv preprint arXiv:2412.21117, 2024

    Pith/arXiv arXiv 2024

  24. [24]

    Matterport3D: Learning from RGB-D data in indoor environments,

    A. Chang, A. Dai, T. Funkhouser, M. Halber, M. Niessner, M. Savva, S. Song, A. Zeng, and Y . Zhang, “Matterport3D: Learning from RGB-D data in indoor environments,” in3DV, 2017

    2017

  25. [25]

    Stereo magnification: Learning view synthesis using multiplane images,

    T. Zhou, R. Tucker, J. Flynn, G. Fyffe, and N. Snavely, “Stereo magnification: Learning view synthesis using multiplane images,”ACM Transactions on Graphics, vol. 37, no. 4, pp. 65:1–65:12, 2018. 15

    2018

  26. [26]

    Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction,

    J. Reizenstein, R. Shapovalov, P. Henzler, L. Sbordone, P. Labatut, and D. Novotny, “Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction,” inICCV, 2021

    2021

  27. [27]

    Mvdiffusion: Enabling holistic multi-view image generation with correspondence- aware diffusion,

    S. Tang, F. Zhang, J. Chen, P. Wang, and F. Yasutaka, “Mvdiffusion: Enabling holistic multi-view image generation with correspondence- aware diffusion,”arXiv preprint arXiv:2307.01097, 2023

    Pith/arXiv arXiv 2023

  28. [28]

    Stable diffusion 2,

    StabilityAI, “Stable diffusion 2,” https://huggingface.co/stabilityai/ stable-diffusion-2, 2023

    2023

  29. [29]

    Structure-from-motion revisited,

    J. L. Schonberger and J.-M. Frahm, “Structure-from-motion revisited,” inCVPR, 2016

    2016

  30. [30]

    Generative adversarial nets,

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adversarial nets,” in NeurIPS, 2014

    2014

  31. [31]

    Generative modeling by estimating gradients of the data distribution,

    Y . Song and S. Ermon, “Generative modeling by estimating gradients of the data distribution,” inNeurIPS, 2019

    2019

  32. [32]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” inNeurIPS, 2020

    2020

  33. [33]

    Denoising diffusion implicit models,

    J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” inICLR, 2021

    2021

  34. [34]

    Score-based generative modeling through stochastic differ- ential equations,

    Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differ- ential equations,” inICLR, 2021

    2021

  35. [35]

    Classifier-free diffusion guidance,

    J. Ho and T. Salimans, “Classifier-free diffusion guidance,”arXiv preprint arXiv:2207.12598, 2022

    Pith/arXiv arXiv 2022

  36. [36]

    Boosting latent diffusion with perceptual objectives,

    T. Berrada, P. Astolfi, M. Hall, M. Havasi, Y . Benchetrit, A. Romero- Soriano, K. Alahari, M. Drozdzal, and J. Verbeek, “Boosting latent diffusion with perceptual objectives,”arXiv preprint arXiv:2411.04873, 2025

    Pith/arXiv arXiv 2025

  37. [37]

    Diffusion model with perceptual loss,

    S. Lin and X. Yang, “Diffusion model with perceptual loss,”arXiv preprint arXiv:2401.00110, 2025

    Pith/arXiv arXiv 2025

  38. [38]

    Multidiffusion: Fus- ing diffusion paths for controlled image generation,

    O. Bar-Tal, L. Yariv, Y . Lipman, and T. Dekel, “Multidiffusion: Fus- ing diffusion paths for controlled image generation,”arXiv preprint arXiv:2302.08113, 2023

    Pith/arXiv arXiv 2023

  39. [39]

    Panogen: Text-conditioned panoramic environ- ment generation for vision-and-language navigation,

    J. Li and M. Bansal, “Panogen: Text-conditioned panoramic environ- ment generation for vision-and-language navigation,”arXiv preprint arXiv:2305.19195, 2023

    Pith/arXiv arXiv 2023

  40. [40]

    Diffcollage: Parallel generation of large content with diffusion models,

    Q. Zhang, J. Song, X. Huang, Y . Chen, and M. yu Liu, “Diffcollage: Parallel generation of large content with diffusion models,” inCVPR, 2023

    2023

  41. [41]

    Customizing 360-degree panoramas through text-to-image diffusion models,

    H. Wang, X. Xiang, Y . Fan, and J.-H. Xue, “Customizing 360-degree panoramas through text-to-image diffusion models,” inWACV, 2024

    2024

  42. [42]

    Taming stable diffusion for text to 360◦ panorama image generation,

    C. Zhang, Q. Wu, C. Cruz Gambardella, X. Huang, D. Phung, W. Ouyang, and J. Cai, “Taming stable diffusion for text to 360◦ panorama image generation,” inCVPR, 2024

    2024

  43. [43]

    Text2mesh: Text-driven 3d mesh generation,

    X. Z. Wei, K. H. Lee, Y . H. Lee, J. Lee, and A. Y . K. Y . Cheung, “Text2mesh: Text-driven 3d mesh generation,” inCVPR, 2022

    2022

  44. [44]

    Fantasia3d: Disentangling geometry and appearance for high-quality text-to-3d content creation,

    R. Chen, Y . Chen, N. Jiao, and K. Jia, “Fantasia3d: Disentangling geometry and appearance for high-quality text-to-3d content creation,” inICCV, 2023

    2023

  45. [45]

    Prolific- dreamer: High-fidelity and diverse text-to-3d generation with variational score distillation,

    Z. Wang, C. Lu, Y . Wang, F. Bao, C. Li, H. Su, and J. Zhu, “Prolific- dreamer: High-fidelity and diverse text-to-3d generation with variational score distillation,” inNeurIPS, 2024

    2024

  46. [46]

    3dtopia: Large text-to-3d generation model with hybrid diffusion priors,

    F. Hong, J. Tang, Z. Cao, M. Shi, T. Wu, Z. Chen, T. Wang, L. Pan, D. Lin, and Z. Liu, “3dtopia: Large text-to-3d generation model with hybrid diffusion priors,”arXiv preprint arXiv:2403.02234, 2024

    Pith/arXiv arXiv 2024

  47. [47]

    Rgbd2: Generative scene synthesis via incremental view inpainting using rgbd diffusion models,

    J. Zhang, Y . Xu, W. Wang, J. Yang, Y . Shen, X. Li, L. Xie, and F. Yu, “Rgbd2: Generative scene synthesis via incremental view inpainting using rgbd diffusion models,” inCVPR, 2023

    2023

  48. [48]

    Text2immersion: Generative immersive scene with 3d gaussians,

    H. Ouyang, K. Heal, S. Lombardi, and T. Sun, “Text2immersion: Generative immersive scene with 3d gaussians,”arXiv preprint arXiv:2312.09242, 2023

    Pith/arXiv arXiv 2023

  49. [49]

    Scenescape: Text- driven consistent scene generation,

    R. Fridman, A. Abecasis, Y . Kasten, and T. Dekel, “Scenescape: Text- driven consistent scene generation,”arXiv preprint arXiv:2302.01133, 2023

    Pith/arXiv arXiv 2023

  50. [50]

    Vistadream: Sampling multiview consistent images for single-view scene reconstruc- tion,

    H. Wang, Y . Liu, Z. Liu, W. Wang, Z. Dong, and B. Yang, “Vistadream: Sampling multiview consistent images for single-view scene reconstruc- tion,”arXiv preprint arXiv:2410.16892, 2024

    Pith/arXiv arXiv 2024

  51. [51]

    Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer,

    R. Ranftl, K. Lasinger, and V . Koltun, “Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 3, pp. 1624–1637, 2022

    2022

  52. [52]

    Zoedepth: Zero-shot transfer by combining relative and metric depth,

    S. F. Bhat, R. Birkl, D. Wofk, P. Wonka, and M. M ¨uller, “Zoedepth: Zero-shot transfer by combining relative and metric depth,”arXiv preprint arXiv:2302.12288, 2023

    Pith/arXiv arXiv 2023

  53. [53]

    Depth anything: Unleashing the power of large-scale unlabeled data,

    L. Yang, B. Kang, Z. Huang, X. Xu, J. Feng, and H. Zhao, “Depth anything: Unleashing the power of large-scale unlabeled data,” inCVPR, 2024

    2024

  54. [54]

    Learning a locally unified 3d point cloud for view synthesis,

    M. You, M. Guo, X. Lyu, H. Liu, and J. Hou, “Learning a locally unified 3d point cloud for view synthesis,”IEEE Transactions on Image Processing, vol. 32, pp. 5610–5622, 2023

    2023

  55. [55]

    pixelnerf: Neural radiance fields from one or few images,

    A. Yu, V . Ye, M. Tancik, and A. Kanazawa, “pixelnerf: Neural radiance fields from one or few images,” inCVPR, 2021

    2021

  56. [56]

    Nope-nerf: Optimising neural radiance field with no pose prior,

    W. Bian, Z. Wang, K. Li, J. Bian, and V . A. Prisacariu, “Nope-nerf: Optimising neural radiance field with no pose prior,” inCVPR, 2023

    2023

  57. [57]

    Latent-nerf for shape-guided generation of 3d shapes and textures,

    G. Metzer, E. Richardson, O. Patashnik, R. Giryes, and D. Cohen-Or, “Latent-nerf for shape-guided generation of 3d shapes and textures,” in CVPR, 2023

    2023

  58. [58]

    Diffusionerf: Regularizing neural radiance fields with denoising diffusion models,

    J. Wynn and D. Turmukhambetov, “Diffusionerf: Regularizing neural radiance fields with denoising diffusion models,” inCVPR, 2023

    2023

  59. [59]

    Global structure-from-motion revisited,

    L. Pan, D. Bar ´ath, M. Pollefeys, and J. L. Sch ¨onberger, “Global structure-from-motion revisited,” inECCV 2024, 2024

    2024

  60. [60]

    Flowmap: High- quality camera poses, intrinsics, and depth via gradient descent,

    C. Smith, D. Charatan, A. Tewari, and V . Sitzmann, “Flowmap: High- quality camera poses, intrinsics, and depth via gradient descent,”arXiv preprint arXiv:2404.15259, 2024

    Pith/arXiv arXiv 2024

  61. [61]

    Lgm: Large multi-view gaussian model for high-resolution 3d content creation,

    J. Tang, Z. Chen, X. Chen, T. Wang, G. Zeng, and Z. Liu, “Lgm: Large multi-view gaussian model for high-resolution 3d content creation,” in ECCV, 2024

    2024

  62. [62]

    Gs-lrm: Large reconstruction model for 3d gaussian splatting,

    K. Zhang, S. Bi, H. Tan, Y . Xiangli, N. Zhao, K. Sunkavalli, and Z. Xu, “Gs-lrm: Large reconstruction model for 3d gaussian splatting,” inECCV, 2024

    2024

  63. [63]

    Grm: Large gaussian reconstruction model for efficient 3d reconstruction and generation,

    Y . Xu, Z. Shi, W. Yifan, H. Chen, C. Yang, S. Peng, Y . Shen, and G. Wetzstein, “Grm: Large gaussian reconstruction model for efficient 3d reconstruction and generation,” inECCV, 2024

    2024

  64. [64]

    Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images,

    Y . Chen, H. Xu, C. Zheng, B. Zhuang, M. Pollefeys, A. Geiger, T.-J. Cham, and J. Cai, “Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images,” inECCV, 2024

    2024

  65. [65]

    Pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction,

    D. Charatan, S. Li, A. Tagliasacchi, and V . Sitzmann, “Pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction,” inCVPR, 2024

    2024

  66. [66]

    Splatter image: Ultra- fast single-view 3d reconstruction,

    S. Szymanowicz, C. Rupprecht, and A. Vedaldi, “Splatter image: Ultra- fast single-view 3d reconstruction,” inCVPR, 2024

    2024

  67. [67]

    Dngaussian: Optimizing sparse-view 3d gaussian radiance fields with global-local depth normalization,

    J. Li, J. Zhang, X. Bai, J. Zheng, X. Ning, J. Zhou, and L. Gu, “Dngaussian: Optimizing sparse-view 3d gaussian radiance fields with global-local depth normalization,” inCVPR, 2024

    2024

  68. [68]

    Fsgs: Real-time few-shot view synthesis using gaussian splatting,

    Z. Zhu, Z. Fan, Y . Jiang, and Z. Wang, “Fsgs: Real-time few-shot view synthesis using gaussian splatting,” inECCV, 2024

    2024

  69. [69]

    Wonderworld: Interactive 3d scene generation from a single image,

    H.-X. Yu, H. Duan, C. Herrmann, W. T. Freeman, and J. Wu, “Wonderworld: Interactive 3d scene generation from a single image,” arXiv:2406.09394, 2024

    Pith/arXiv arXiv 2024

  70. [70]

    Auto-encoding variational bayes,

    D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” in ICLR, 2014

    2014

  71. [71]

    Very deep convolutional networks for large-scale image recognition,

    K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” inICLR, 2015

    2015

  72. [72]

    Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,

    K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in ICCV, 2015

    2015

  73. [73]

    Imagenet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and F.-F. Li, “Imagenet: A large-scale hierarchical image database,” inCVPR, 2009

    2009

  74. [74]

    Raft: Recurrent all-pairs field transforms for optical flow,

    Z. Teed and J. Deng, “Raft: Recurrent all-pairs field transforms for optical flow,” inECCV, 2020

    2020

  75. [75]

    Cotracker: It is better to track together,

    N. Karaev, I. Rocco, B. Graham, N. Neverova, A. Vedaldi, and C. Rup- precht, “Cotracker: It is better to track together,” inECCV, 2024

    2024

  76. [76]

    Nexusgs: Sparse view synthesis with epipolar depth priors in 3d gaussian splatting,

    Y . Zheng, Z. Jiang, S. He, Y . Sun, J. Dong, H. Zhang, and Y . Du, “Nexusgs: Sparse view synthesis with epipolar depth priors in 3d gaussian splatting,”arXiv preprint arXiv:2503.18794, 2025

    Pith/arXiv arXiv 2025

  77. [77]

    Gpt-4 technical report,

    OpenAI, “Gpt-4 technical report,” OpenAI, Tech. Rep., 2024. [Online]. Available: https://platform.openai.com/docs/gpt-4

    2024

  78. [78]

    Tanks and temples: Benchmarking large-scale scene reconstruction,

    A. Knapitsch, J. Park, Q.-Y . Zhou, and V . Koltun, “Tanks and temples: Benchmarking large-scale scene reconstruction,” inACM Transactions on Graphics (TOG), 2017

    2017

  79. [79]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes,

    A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nießner, “Scannet: Richly-annotated 3d reconstructions of indoor scenes,” inCVPR, 2017

    2017

  80. [80]

    CLIPScore: a reference-free evaluation metric for image captioning,

    J. Hessel, A. Holtzman, M. Forbes, R. L. Bras, and Y . Choi, “CLIPScore: a reference-free evaluation metric for image captioning,” inEMNLP, 2021

    2021

Showing first 80 references.