Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

Generating 3D scenes from a single image improves when the video model is conditioned on full-frame geometry features extracted from a draft video by a geometry model, rather than on single-frame depth maps or camera embeddings.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 19:35 UTC pith:F53KNXGL

load-bearing objection Solid two-stage conditioning recipe with real fidelity gains, but the geometry-specific mechanism isn't proven and the paper oversells its metrics. the 5 major comments →

arxiv 2511.23191 v3 pith:F53KNXGL submitted 2025-11-28 cs.CV

GeoWorld: Providing Full-frame Geometry Features to Facilitate 3D Scene Generation

Yuhao Wan , Lijuan Liu , Jingzhi Zhou , Zihan Zhou , Xuying Zhang , Dongbo Zhang , Shaohui Jiao , Qibin Hou
show 1 more author
Ming-Ming Cheng
This is my paper
classification cs.CV
keywords image-to-3D3D scene generationvideo diffusionfull-frame geometry featuresgeometry alignment lossgeometry adaptation modulenovel view synthesisVGGT
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that the persistent geometric distortion and blurriness in video-based image-to-3D generation come from a too-weak conditioning signal: a single-frame depth map or a camera embedding carries too little information to keep the generated video geometrically consistent. GeoWorld replaces that signal with full-frame geometry features. The pipeline first uses a fine-tuned video model to turn the input image and camera trajectory into a coarse draft video; a multi-view geometry model extracts dense geometry features from the entire draft; a second video model is conditioned on those features through a geometry adaptation module, and a geometry alignment loss pushes the predicted views toward the ground-truth geometry in feature space. If the claim holds, a much smaller two-model pipeline can generate 3D scenes that are quantitatively and qualitatively better than larger single-model methods, while running several times faster.

Core claim

The paper's central claim is that the geometric conditioning signal, not the size of the video model, is the limiting factor in video-based image-to-3D generation. Rather than distilling geometry into a single-frame depth map or a camera embedding, GeoWorld first renders the input image along the target camera trajectory, lets a fine-tuned video model complete those partial views into a full draft video, and then feeds that draft video to a multi-view geometry model (VGGT) whose aggregator output—with the decoding stage removed—becomes full-frame geometry features. These features condition a second, geometry-constrained video model via a single-frame cross-attention layer, after a geometry a

What carries the argument

The load-bearing mechanism is the full-frame geometry feature: the output of the aggregator stage of a multi-view geometry transformer (VGGT), extracted from the draft video and used as conditioning. Before being injected into the second video model, these features go through a geometry adaptation module that pools them along the frame dimension, interpolates them in height and width to match the video latent, runs them through a trained MLP adapter, and applies a learned global weight to each token, discarding the least informative half. The same VGGT features define the geometry alignment loss L_geo = ||G(I) − G(I_pred)||², which compares the geometry features of the predicted video to tho

Load-bearing premise

The method works only if the geometry features extracted from the coarse draft video are an accurate and stable proxy for true scene geometry, and the geometry alignment loss actually enforces geometric consistency rather than matching appearance.

What would settle it

Replace the VGGT aggregator features with features from a non-geometric image encoder while keeping the alignment loss and adaptation module; if the gains over single-frame depth conditioning survive, the geometry content is not the load-bearing factor.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Full-frame geometry conditioning improves novel-view-synthesis fidelity (PSNR, SSIM) and perceptual sharpness (LPIPS) on RealEstate10K and Tanks and Temples.
  • The geometry alignment loss alone reduces geometric distortions and artifacts; the geometry adaptation module further sharpens structures (ablation in Tab. 3).
  • The same conditioned video translates into better 3D Gaussian Splatting reconstructions, with GeoWorld leading on all reported scene-generation metrics.
  • The two-stage design with two 1.3B video models is substantially faster than a larger single pipeline (7.5x in the reported comparison).
  • Discarding the bottom 50% of low-weight geometry tokens outperforms discarding 30% or 70%, indicating a clear sweet spot for conditioning noise removal.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the full-frame geometry hypothesis is the real driver, then the quality of the draft video is an upper bound on final quality; the paper's own sweet-spot discussion suggests a testable extension in which improving draft consistency (rather than just training longer) should yield further fidelity gains.
  • The same geometry alignment loss could be applied in other feature spaces (depth maps, point clouds, or another geometry backbone); a controlled comparison of feature spaces would isolate whether VGGT's aggregator space is uniquely informative or merely sufficient.
  • The global-weighting token pruning scheme is a general recipe for noisy conditioning and could transfer to other conditional generation tasks where only part of the condition is trustworthy.
  • Because the pipeline inherits VGGT's biases, swapping the geometry model for a different multi-view geometry estimator would test how much of the gain is due to geometry information per se versus the specific feature statistics.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes GeoWorld, a two-stage pipeline for generating a 3D scene from a single image and a specified camera trajectory. In the first stage, a fine-tuned video model generates a coarse draft video from rendered condition views. This draft is fed to a geometry model (VGGT), whose aggregator features are extracted as full-frame geometry tokens. In the second stage, a geometry-constrained diffusion model is trained to refine the video, using these tokens as cross-attention conditions, a geometry alignment loss (matching VGGT features of ground-truth and predicted videos), and a geometry adaptation module with token weighting and discarding. Experiments on RealEstate10K and Tanks and Temples report improved PSNR/SSIM/LPIPS over DimensionX, See3D, ViewCrafter, and FlexWorld for both novel-view synthesis and 3DGS reconstruction, with ablations showing each component contributes.

Significance. If the results are robust, the paper makes a useful empirical contribution: it demonstrates that full-frame geometry features from a pre-trained geometry model can serve as a strong condition for video-based 3D scene generation, and it introduces a geometry alignment loss that may be of independent interest. The paper is generally clear, includes visual comparisons, and reports ablations for the main design choices. However, the significance is tempered by several unmet evidential requirements: the claimed blanket superiority over baselines is contradicted by the paper's own FID/FVD numbers, the geometry alignment loss is not shown to be geometry-specific, and several claims (e.g., speed) are made without supporting measurements.

major comments (5)
  1. [Abstract / Sec. 4.2, Table 1] The abstract and Sec. 1 claim that GeoWorld 'outperforms prior methods both qualitatively and quantitatively.' This is contradicted by Table 1: on RealEstate10K, FlexWorld has better FID (30.56 vs 31.00) and better FVD (270.4 vs 311.7); on Tanks, FlexWorld also has better FID (38.69 vs 39.39) and FVD (422.6 vs 507.9). The text in Sec. 4.2 acknowledges this ('slightly higher than FlexWorld'), but the framing still claims overall superiority. Please revise the claims to be consistent with the full set of metrics, or provide statistical significance tests / confidence intervals showing the differences are not meaningful.
  2. [Sec. 4.3, Table 4] The hyperparameters, including the geometry loss weight λ (Sec. 4.1) and the token discard ratio (Table 4), appear to be selected using the RE10K test set. Table 4 is reported 'on the RE10K [60] test set,' which invites overfitting and makes the reported improvements less trustworthy. The paper should use a validation split for all model selection and report results with error bars across multiple seeds or dataset samples. Without this, the quantitative gains (e.g., Table 1 PSNR gains) may not generalize.
  3. [Sec. 3.2, Eq. (2)] The geometry alignment loss Lgeo = ||G(I) - G(I_pred)||^2 uses VGGT's aggregator features G. Since VGGT is a large transformer trained to predict geometry from RGB, its latent features are not shown to be a geometry-specific metric; they may respond to appearance, texture, and lighting. The ablation in Table 3 shows that adding this loss improves PSNR by 0.19 and LPIPS by 0.0097 on the test set, which is consistent with a generic perceptual loss. The central claim that this loss imposes 'real-world geometric constraints' would be substantially strengthened by an ablation replacing G with a non-geometric feature extractor (e.g., DINO, LPIPS, VGG) and by evaluating geometric consistency directly (e.g., depth/point-cloud error) rather than only image metrics.
  4. [Sec. 1 / Abstract, speed claim] The abstract claims GeoWorld is '7.5× faster than Hunyuan-Voyager.' No runtime measurements, hardware details, or comparisons with Hunyuan-Voyager are provided anywhere in the paper. Please add a quantitative runtime comparison table and describe the experimental setup (GPU model, batch size, resolution, inference script, etc.). Without this, the speed claim is unsupported.
  5. [Sec. 4.3, Fig. 8/9] The paper itself concedes that if the fine-tuned video model in the geometrical condition generation procedure is undertrained, the draft video 'may fail to provide sufficient geometric information and could even introduce incorrect guidance.' This is a load-bearing dependence: the geometry features are extracted from this draft, so the second stage can only be as good as the draft. The paper does not quantify how the performance of the geometry-constrained model varies with the quality of the first stage (e.g., a sweep over fine-tuning steps and the resulting final PSNR/LPIPS). Such an analysis is necessary to establish robustness of the pipeline.
minor comments (5)
  1. [Table 2] ViewCrafter is included in Table 1 and in the qualitative comparisons but is omitted from Table 2 (3D scene generation). Please either include it or explain why it is excluded (e.g., reconstruction failed).
  2. [Sec. 3.2 / Notation] The symbol I is used for both the ground-truth video and the single input image. Please distinguish these, for example using I_0 for the input image and I_gt for the ground-truth video.
  3. [Sec. 3.3] The description of the geometry adaptation module says the output g 'differs from the input x along the frame, height, and width dimensions.' Please specify the actual tensor shapes and the interpolation method (e.g., bilinear). Also clarify what a 'token' denotes after pooling/interpolation.
  4. [References] Hunyuan-Voyager is mentioned in the abstract but has no reference in the bibliography. If it is a published work, it should be cited. Also, the spelling 'MAST3R' (Sec. 3.1) should be consistent with 'MASt3R' elsewhere.
  5. [Fig. 3] The figure labels 'Geo Constrained Diffusion' and 'Geo Model' are not fully explained in the caption. Clarify that the former is the geometry-constrained diffusion model and the latter is VGGT.

Circularity Check

0 steps flagged

No significant circularity: the conditioning and supervision share VGGT features, but the central claim is evaluated with independent image-quality metrics and no parameter is fit to those metrics.

full rationale

GeoWorld's derivation chain is not circular. The full-frame geometry tokens (Sec. 3.3) are produced by feeding draft views from the first-stage video model into the frozen VGGT aggregator; the geometry alignment loss (Eq. 2, Sec. 3.2) compares VGGT features of the ground-truth video I and the predicted video I_pred. Although the same model G provides both the conditioning signal and the supervisory feature space, this is a fixed pretrained feature extractor, not a parameter being fit to the reported metrics. Matching G(I_pred) to G(I) is a form of feature/distillation loss; it does not make the claimed improvements true by construction because the evaluation uses independent PSNR/SSIM/LPIPS/FID/FVD against held-out RE10K and Tanks ground truths, and the ablations (Tab. 3) show incremental gains from GAL/GAM rather than a fitted identity. The paper's Sec. 4.3 limitation about undertrained first-stage models acknowledges fragility but is a correctness/robustness caveat, not a circular step. No load-bearing self-citation chain is present; VGGT is external prior work. The skeptical concern that Eq. 2 may behave like an appearance/perceptual loss rather than a purely geometric one is a validity risk, not a circularity, and therefore does not raise the circularity score.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The central empirical result rests on VGGT's features being a reliable geometric proxy, MAST3R providing accurate render geometry at inference, and several hand-set hyperparameters (lambda, discard ratio, training schedules) that are tuned on the test split.

free parameters (6)
  • Geometry loss weight lambda = 0.2
    Set by hand in Sec. 4.1 to balance L_diff and L_geo; no reported sensitivity analysis.
  • Token discard ratio = 50%
    Chosen via ablation on RE10K (Tab. 4); 30% and 70% reduce PSNR/SSIM, so 50% is tuned on the test set.
  • Fine-tuned video model iterations = 7000
    Selected from Fig. 8 FID curve; 11000 steps did not improve FID and still showed distortions, so 7000 is used for draft generation.
  • Geometry-constrained model iterations = 2000
    Training schedule for the final model (Sec. 4.1); no ablation shown for this value.
  • Training data subsampling = top 25% lowest camera motion (~5000 pairs)
    DL3DV pairs filtered to the smallest average translation/rotation (Sec. 4.1); shapes training distribution and is not independently validated.
  • Training hyperparameters = batch 16, lr 5e-5, 192x336, 17 frames
    Standard choices reported in Sec. 4.1; affect stability and quality but not load-bearing individually.
axioms (5)
  • domain assumption VGGT aggregator features, with decoding omitted, retain complete geometric representations useful for conditioning and supervision.
    Sec. 3.2 states the geometry model is the VGGT aggregator with the decoding stage omitted; if these features discard geometric detail, both the condition and the geometry loss degrade.
  • domain assumption Geometry alignment loss in VGGT feature space is a valid proxy for real-world 3D geometric consistency.
    Eq. (2) uses MSE between G(I) and G(I_pred); this assumes VGGT features are sensitive to geometric distortions and that matching them improves geometry rather than just appearance.
  • domain assumption At inference, MAST3R point-cloud estimation and back-projection produce condition views accurate enough for geometry extraction.
    Sec. 3.1 Rendering: during inference the point cloud of the single-frame input is estimated with MAST3R; errors here propagate to the condition views and geometry tokens.
  • domain assumption The fine-tuned first-stage video model generates condition views that are content-complete enough for VGGT to extract meaningful geometry features.
    Sec. 4.3 admits an undertrained model could 'introduce incorrect guidance'; the whole pipeline assumes the 7000-step model is in the useful regime.
  • domain assumption Diffusion model training with an added feature-matching loss does not destabilize the pretrained video prior.
    Eq. (3) combines L_diff and lambda*L_geo; the work assumes this weighted objective preserves generation quality while adding geometric constraint.

pith-pipeline@v1.3.0-alltime-deepseek · 12888 in / 15634 out tokens · 145801 ms · 2026-08-03T19:35:01.520569+00:00 · methodology

0 comments
read the original abstract

Previous works that leverage video models for image-to-3D scene generation often suffer from geometric distortions and blurry content. Using video generation models to implicitly maintain geometric consistency according to a single-frame input is ineffective. In this paper, we present a two-stage method, named $\textbf{GeoWorld}$, that renovates the image-to-3D scene generation pipeline by providing full-frame geometry features. The first-stage video generation model, followed by a multi-view geometry model, produces $\textbf{full-frame}$ geometry features, which are then used as a mental draft of geometric conditions to aid the second-stage video-generation model. A geometric loss is proposed to impose real-world geometric constraints, and a geometry adaptation module is introduced to ensure the effective utilization of geometry features. Thanks to full-frame geometric modeling, the two smaller video models in our two-stage method can generate higher-fidelity 3D scenes than SOTA methods, while being even faster, e.g. 7.5$\times$ faster than Hunyuan-Voyager. Project page: https://peaes.github.io/GeoWorld.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Pose-Aware Diffusion for 3D Generation

    cs.CV 2026-05 unverdicted novelty 5.0

    PAD synthesizes 3D geometry in observation space via depth unprojection as anchor to eliminate pose ambiguity in image-to-3D generation.

Reference graph

Works this paper leans on

60 extracted references · 22 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Gen3dsr: Generalizable 3d scene reconstruction via divide and conquer from a single view

    Andreea Ardelean, Mert ¨Ozer, and Bernhard Egger. Gen3dsr: Generalizable 3d scene reconstruction via divide and conquer from a single view. In2025 Interna- tional Conference on 3D Vision (3DV), pages 616–626. IEEE, 2025. 3

    2025

  2. [2]

    Baking gaussian splatting into diffusion denoiser for fast and scalable single-stage image-to-3d generation and reconstruction

    Yuanhao Cai, He Zhang, Kai Zhang, Yixun Liang, Mengwei Ren, Fujun Luan, Qing Liu, Soo Ye Kim, Jianming Zhang, Zhifei Zhang, et al. Baking gaussian splatting into diffusion denoiser for fast and scalable single-stage image-to-3d generation and reconstruction. InProceedings of the IEEE/CVF International Confer- ence on Computer Vision, pages 25062–25072, 2025. 2

    2025

  3. [3]

    Flexworld: Progressively expanding 8 3d scenes for flexiable-view synthesis.arXiv preprint arXiv:2503.13265, 2025

    Luxi Chen, Zihan Zhou, Min Zhao, Yikai Wang, Ge Zhang, Wenhao Huang, Hao Sun, Ji-Rong Wen, and Chongxuan Li. Flexworld: Progressively expanding 8 3d scenes for flexiable-view synthesis.arXiv preprint arXiv:2503.13265, 2025. 2, 3, 4, 5, 6, 7, 12, 15

    Pith/arXiv arXiv 2025

  4. [4]

    Luciddreamer: Domain- free generation of 3d gaussian splatting scenes.arXiv preprint arXiv:2311.13384, 2023

    Jaeyoung Chung, Suyoung Lee, Hyeongjin Nam, Jaerin Lee, and Kyoung Mu Lee. Luciddreamer: Domain- free generation of 3d gaussian splatting scenes.arXiv preprint arXiv:2311.13384, 2023. 2

    Pith/arXiv arXiv 2023

  5. [5]

    Set-the-scene: Global- local training for generating controllable nerf scenes

    Dana Cohen-Bar, Elad Richardson, Gal Metzer, Raja Giryes, and Daniel Cohen-Or. Set-the-scene: Global- local training for generating controllable nerf scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2920–2929, 2023. 2

    2023

  6. [6]

    Hiscene: creating hierarchical 3d scenes with isometric view generation

    Wenqi Dong, Bangbang Yang, Zesong Yang, Yuan Li, Tao Hu, Hujun Bao, Yuewen Ma, and Zhaopeng Cui. Hiscene: creating hierarchical 3d scenes with isometric view generation. InProceedings of the 33rd ACM Inter- national Conference on Multimedia, pages 9783–9792,

  7. [7]

    An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020. 5

    Pith/arXiv arXiv 2010

  8. [8]

    Magic- drive: Street view generation with diverse 3d geometry control.arXiv preprint arXiv:2310.02601, 2023

    Ruiyuan Gao, Kai Chen, Enze Xie, Lanqing Hong, Zhenguo Li, Dit-Yan Yeung, and Qiang Xu. Magic- drive: Street view generation with diverse 3d geometry control.arXiv preprint arXiv:2310.02601, 2023. 2

    Pith/arXiv arXiv 2023

  9. [9]

    Cat3d: Create anything in 3d with multi-view diffusion models.arXiv preprint arXiv:2405.10314, 2024

    Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul Srini- vasan, Jonathan T Barron, and Ben Poole. Cat3d: Create anything in 3d with multi-view diffusion models.arXiv preprint arXiv:2405.10314, 2024. 2

    Pith/arXiv arXiv 2024

  10. [10]

    Dist-4d: Disentangled spatiotemporal diffusion with metric depth for 4d driv- ing scene generation.arXiv preprint arXiv:2503.15208,

    Jiazhe Guo, Yikang Ding, Xiwu Chen, Shuo Chen, Bo- han Li, Yingshuang Zou, Xiaoyang Lyu, Feiyang Tan, Xiaojuan Qi, Zhiheng Li, et al. Dist-4d: Disentangled spatiotemporal diffusion with metric depth for 4d driv- ing scene generation.arXiv preprint arXiv:2503.15208,

    Pith/arXiv arXiv

  11. [11]

    Cameractrl: Enabling camera control for text-to-video generation

    Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. Cameractrl: Enabling camera control for text-to-video generation. arXiv preprint arXiv:2404.02101, 2024. 3

    Pith/arXiv arXiv 2024

  12. [12]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017. 5

    2017

  13. [13]

    Denois- ing diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denois- ing diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020. 2

    2020

  14. [14]

    Text2room: Extracting tex- tured 3d meshes from 2d text-to-image models

    Lukas H¨ollein, Ang Cao, Andrew Owens, Justin John- son, and Matthias Nießner. Text2room: Extracting tex- tured 3d meshes from 2d text-to-image models. InPro- ceedings of the IEEE/CVF International Conference on Computer Vision, pages 7909–7920, 2023. 2

    2023

  15. [15]

    Cogvideo: Large-scale pretraining for text- to-video generation via transformers.arXiv preprint arXiv:2205.15868, 2022

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text- to-video generation via transformers.arXiv preprint arXiv:2205.15868, 2022. 2

    Pith/arXiv arXiv 2022

  16. [16]

    Squeeze-and-excitation networks

    Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitation networks. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 7132– 7141, 2018. 5

    2018

  17. [17]

    V oy- ager: Long-range and world-consistent video diffusion for explorable 3d scene generation.arXiv preprint arXiv:2506.04225, 2025

    Tianyu Huang, Wangguandong Zheng, Tengfei Wang, Yuhao Liu, Zhenwei Wang, Junta Wu, Jie Jiang, Hui Li, Rynson WH Lau, Wangmeng Zuo, et al. V oy- ager: Long-range and world-consistent video diffusion for explorable 3d scene generation.arXiv preprint arXiv:2506.04225, 2025. 2

    Pith/arXiv arXiv 2025

  18. [18]

    Midi: Multi-instance diffusion for single image to 3d scene generation

    Zehuan Huang, Yuan-Chen Guo, Xingqiao An, Yunhan Yang, Yangguang Li, Zi-Xin Zou, Ding Liang, Xihui Liu, Yan-Pei Cao, and Lu Sheng. Midi: Multi-instance diffusion for single image to 3d scene generation. In Proceedings of the Computer Vision and Pattern Recog- nition Conference, pages 23646–23657, 2025. 3

    2025

  19. [19]

    3d gaussian splatting for real-time radiance field rendering.ACM Trans

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Trans. Graph., 42(4):139–1, 2023. 2

    2023

  20. [20]

    Tanks and temples: Benchmarking large-scale scene reconstruction.ACM Transactions on Graphics (ToG), 36(4):1–13, 2017

    Arno Knapitsch, Jaesik Park, Qian-Yi Zhou, and Vladlen Koltun. Tanks and temples: Benchmarking large-scale scene reconstruction.ACM Transactions on Graphics (ToG), 36(4):1–13, 2017. 5

    2017

  21. [21]

    Rgbd2: Gener- ative scene synthesis via incremental view inpainting using rgbd diffusion models

    Jiabao Lei, Jiapeng Tang, and Kui Jia. Rgbd2: Gener- ative scene synthesis via incremental view inpainting using rgbd diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8422–8434, 2023. 2

    2023

  22. [22]

    Grounding image matching in 3d with mast3r

    Vincent Leroy, Yohann Cabon, and J ´erˆome Revaud. Grounding image matching in 3d with mast3r. InEu- ropean Conference on Computer Vision, pages 71–91. Springer, 2024. 3, 4

    2024

  23. [23]

    Uniscene: Unified occupancy-centric driving scene generation

    Bohan Li, Jiazhe Guo, Hongsi Liu, Yingshuang Zou, Yikang Ding, Xiwu Chen, Hu Zhu, Feiyang Tan, Chi Zhang, Tiancai Wang, et al. Uniscene: Unified occupancy-centric driving scene generation. InProceed- ings of the Computer Vision and Pattern Recognition Conference, pages 11971–11981, 2025. 2

    2025

  24. [24]

    9 Dreamscene: 3d gaussian-based text-to-3d scene gener- ation via formation pattern sampling

    Haoran Li, Haolin Shi, Wenli Zhang, Wenjun Wu, Yong Liao, Lin Wang, Lik-hang Lee, and Peng Yuan Zhou. 9 Dreamscene: 3d gaussian-based text-to-3d scene gener- ation via formation pattern sampling. InEuropean Con- ference on Computer Vision, pages 214–230. Springer,

  25. [25]

    Genrc: Generative 3d room completion from sparse image collections

    Ming-Feng Li, Yueh-Feng Ku, Hong-Xuan Yen, Chi Liu, Yu-Lun Liu, Albert YC Chen, Cheng-Hao Kuo, and Min Sun. Genrc: Generative 3d room completion from sparse image collections. InEuropean Conference on Computer Vision, pages 146–163. Springer, 2024. 2

    2024

  26. [26]

    Director3d: Real-world camera trajectory and 3d scene generation from text.Advances in neural infor- mation processing systems, 37:75125–75151, 2024

    Xinyang Li, Zhangyu Lai, Linning Xu, Yansong Qu, Li- ujuan Cao, Shengchuan Zhang, Bo Dai, and Rongrong Ji. Director3d: Real-world camera trajectory and 3d scene generation from text.Advances in neural infor- mation processing systems, 37:75125–75151, 2024. 2

    2024

  27. [27]

    Wonderland: Navigating 3d scenes from a single image

    Hanwen Liang, Junli Cao, Vidit Goel, Guocheng Qian, Sergei Korolev, Demetri Terzopoulos, Konstantinos N Plataniotis, Sergey Tulyakov, and Jian Ren. Wonderland: Navigating 3d scenes from a single image. InProceed- ings of the Computer Vision and Pattern Recognition Conference, pages 798–810, 2025. 2

    2025

  28. [28]

    Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision

    Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, et al. Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22160–22169, 2024. 5

    2024

  29. [29]

    Idcnet: Guided video diffusion for metric- consistent rgbd scene generation with precise camera control.arXiv preprint arXiv:2508.04147, 2025

    Lijuan Liu, Wenfa Li, Dongbo Zhang, Shuo Wang, and Shaohui Jiao. Idcnet: Guided video diffusion for metric- consistent rgbd scene generation with precise camera control.arXiv preprint arXiv:2508.04147, 2025. 2, 3

    Pith/arXiv arXiv 2025

  30. [30]

    You see it, you got it: Learning 3d creation on pose-free videos at scale

    Baorui Ma, Huachen Gao, Haoge Deng, Zhengxiong Luo, Tiejun Huang, Lulu Tang, and Xinlong Wang. You see it, you got it: Learning 3d creation on pose-free videos at scale. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 2016–2029,

    2016

  31. [31]

    Fastscene: Text- driven fast 3d indoor scene generation via panoramic gaussian splatting.arXiv preprint arXiv:2405.05768,

    Yikun Ma, Dandan Zhan, and Zhi Jin. Fastscene: Text- driven fast 3d indoor scene generation via panoramic gaussian splatting.arXiv preprint arXiv:2405.05768,

    Pith/arXiv arXiv

  32. [32]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022. 2

    Pith/arXiv arXiv 2022

  33. [33]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2

    2022

  34. [34]

    Dimen- sionx: Create any 3d and 4d scenes from a single im- age with controllable video diffusion.arXiv preprint arXiv:2411.04928, 2024

    Wenqiang Sun, Shuo Chen, Fangfu Liu, Zilong Chen, Yueqi Duan, Jun Zhang, and Yikai Wang. Dimen- sionx: Create any 3d and 4d scenes from a single im- age with controllable video diffusion.arXiv preprint arXiv:2411.04928, 2024. 2, 3, 5, 6, 7, 12, 15

    Pith/arXiv arXiv 2024

  35. [35]

    Flash3d: Feed-forward generalisable 3d scene reconstruction from a single im- age.arXiv preprint arXiv:2406.04343, 2024

    Stanislaw Szymanowicz, Eldar Insafutdinov, Chuanxia Zheng, Dylan Campbell, Joao F Henriques, Christian Rupprecht, and Andrea Vedaldi. Flash3d: Feed-forward generalisable 3d scene reconstruction from a single im- age.arXiv preprint arXiv:2406.04343, 2024. 2

    Pith/arXiv arXiv 2024

  36. [36]

    Fvd: A new metric for video generation

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Ku- rach, Rapha¨el Marinier, Marcin Michalski, and Sylvain Gelly. Fvd: A new metric for video generation. 2019. 5

    2019

  37. [37]

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025. 2, 5

    Pith/arXiv arXiv 2025

  38. [38]

    Vggt: Visual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. InProceedings of the Computer Vision and Pattern Recognition Confer- ence, pages 5294–5306, 2025. 2, 3, 4

    2025

  39. [39]

    Continuous 3d perception model with persistent state

    Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A Efros, and Angjoo Kanazawa. Continuous 3d perception model with persistent state. InProceed- ings of the Computer Vision and Pattern Recognition Conference, pages 10510–10522, 2025. 3

    2025

  40. [40]

    Dust3r: Geometric 3d vision made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 20697–20709, 2024. 3

    2024

  41. [41]

    Architect: Generating vivid and interactive 3d scenes with hierarchical 2d in- painting.Advances in Neural Information Processing Systems, 37:67575–67603, 2024

    Yian Wang, Xiaowen Qiu, Jiageng Liu, Zhehuan Chen, Jiting Cai, Yufei Wang, Tsun-Hsuan Johnson Wang, Zhou Xian, and Chuang Gan. Architect: Generating vivid and interactive 3d scenes with hierarchical 2d in- painting.Advances in Neural Information Processing Systems, 37:67575–67603, 2024. 3

    2024

  42. [42]

    π3: Permutation- equivariant visual geometry learning.arXiv preprint arXiv:2507.13347, 2025

    Yifan Wang, Jianjun Zhou, Haoyi Zhu, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Jiangmiao Pang, Chunhua Shen, and Tong He. π3: Permutation- equivariant visual geometry learning.arXiv preprint arXiv:2507.13347, 2025. 3

    Pith/arXiv arXiv 2025

  43. [43]

    Image quality assessment: from error vis- ibility to structural similarity.IEEE transactions on image processing, 13(4):600–612, 2004

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error vis- ibility to structural similarity.IEEE transactions on image processing, 13(4):600–612, 2004. 5

    2004

  44. [44]

    Fast3r: Towards 3d reconstruc- tion of 1000+ images in one forward pass

    Jianing Yang, Alexander Sax, Kevin J Liang, Mikael Henaff, Hao Tang, Ang Cao, Joyce Chai, Franziska Meier, and Matt Feiszli. Fast3r: Towards 3d reconstruc- tion of 1000+ images in one forward pass. InProceed- ings of the Computer Vision and Pattern Recognition Conference, pages 21924–21935, 2025. 3 10

    2025

  45. [45]

    Layerpano3d: Layered 3d panorama for hyper-immersive scene gener- ation

    Shuai Yang, Jing Tan, Mengchen Zhang, Tong Wu, Gor- don Wetzstein, Ziwei Liu, and Dahua Lin. Layerpano3d: Layered 3d panorama for hyper-immersive scene gener- ation. InProceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Confer- ence Conference Papers, pages 1–10, 2025. 2

    2025

  46. [46]

    Cogvideox: Text-to-video diffusion models with an expert trans- former.arXiv preprint arXiv:2408.06072, 2024

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert trans- former.arXiv preprint arXiv:2408.06072, 2024. 2

    Pith/arXiv arXiv 2024

  47. [47]

    Cast: Component-aligned 3d scene re- construction from an rgb image.arXiv preprint arXiv:2502.12894, 2025

    Kaixin Yao, Longwen Zhang, Xinhao Yan, Yan Zeng, Qixuan Zhang, Wei Yang, Lan Xu, Jiayuan Gu, and Jingyi Yu. Cast: Component-aligned 3d scene re- construction from an rgb image.arXiv preprint arXiv:2502.12894, 2025. 2, 3

    Pith/arXiv arXiv 2025

  48. [48]

    Wonderworld: In- teractive 3d scene generation from a single image

    Hong-Xing Yu, Haoyi Duan, Charles Herrmann, William T Freeman, and Jiajun Wu. Wonderworld: In- teractive 3d scene generation from a single image. In Proceedings of the Computer Vision and Pattern Recog- nition Conference, pages 5916–5926, 2025. 2

    2025

  49. [49]

    Won- derjourney: Going from anywhere to everywhere

    Hong-Xing Yu, Haoyi Duan, Junhwa Hur, Kyle Sar- gent, Michael Rubinstein, William T Freeman, Forrester Cole, Deqing Sun, Noah Snavely, Jiajun Wu, et al. Won- derjourney: Going from anywhere to everywhere. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6658–6667, 2024. 2

    2024

  50. [50]

    Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis

    Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis. arXiv preprint arXiv:2409.02048, 2024. 2, 3, 5, 6, 12, 15

    Pith/arXiv arXiv 2024

  51. [51]

    Stargen: A spatiotemporal au- toregression framework with video diffusion model for scalable and controllable scene generation

    Shangjin Zhai, Zhichao Ye, Jialin Liu, Weijian Xie, Jiaqi Hu, Zhen Peng, Hua Xue, Danpeng Chen, Xiaomeng Wang, Lei Yang, et al. Stargen: A spatiotemporal au- toregression framework with video diffusion model for scalable and controllable scene generation. InProceed- ings of the Computer Vision and Pattern Recognition Conference, pages 26822–26833, 2025. 2

    2025

  52. [52]

    Gs-lrm: Large reconstruction model for 3d gaussian splatting

    Kai Zhang, Sai Bi, Hao Tan, Yuanbo Xiangli, Nanxuan Zhao, Kalyan Sunkavalli, and Zexiang Xu. Gs-lrm: Large reconstruction model for 3d gaussian splatting. In European Conference on Computer Vision, pages 1–19. Springer, 2024. 2

    2024

  53. [53]

    To- wards text-guided 3d scene composition

    Qihang Zhang, Chaoyang Wang, Aliaksandr Siarohin, Peiye Zhuang, Yinghao Xu, Ceyuan Yang, Dahua Lin, Bolei Zhou, Sergey Tulyakov, and Hsin-Ying Lee. To- wards text-guided 3d scene composition. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6829–6838, 2024. 2

    2024

  54. [54]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018. 5

    2018

  55. [55]

    Flare: Feed-forward geom- etry, appearance and camera estimation from uncali- brated sparse views

    Shangzhan Zhang, Jianyuan Wang, Yinghao Xu, Nan Xue, Christian Rupprecht, Xiaowei Zhou, Yujun Shen, and Gordon Wetzstein. Flare: Feed-forward geom- etry, appearance and camera estimation from uncali- brated sparse views. InProceedings of the Computer Vi- sion and Pattern Recognition Conference, pages 21936– 21947, 2025. 3

    2025

  56. [56]

    Ar-1-to-3: Single image to consis- tent 3d object via next-view prediction

    Xuying Zhang, Yupeng Zhou, Kai Wang, Yikai Wang, Zhen Li, Shaohui Jiao, Daquan Zhou, Qibin Hou, and Ming-Ming Cheng. Ar-1-to-3: Single image to consis- tent 3d object via next-view prediction. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 26273–26283, 2025. 2

    2025

  57. [57]

    Genxd: Generating any 3d and 4d scenes.arXiv preprint arXiv:2411.02319,

    Yuyang Zhao, Chung-Ching Lin, Kevin Lin, Zhi- wen Yan, Linjie Li, Zhengyuan Yang, Jianfeng Wang, Gim Hee Lee, and Lijuan Wang. Genxd: Generating any 3d and 4d scenes.arXiv preprint arXiv:2411.02319,

    Pith/arXiv arXiv

  58. [58]

    Stable virtual camera: Generative view synthesis with diffusion models.arXiv preprint arXiv:2503.14489, 2025

    Jensen Zhou, Hang Gao, Vikram V oleti, Aaryaman Va- sishta, Chun-Han Yao, Mark Boss, Philip Torr, Christian Rupprecht, and Varun Jampani. Stable virtual camera: Generative view synthesis with diffusion models.arXiv preprint arXiv:2503.14489, 2025. 2

    Pith/arXiv arXiv 2025

  59. [59]

    Dream- scene360: Unconstrained text-to-3d scene generation with panoramic gaussian splatting

    Shijie Zhou, Zhiwen Fan, Dejia Xu, Haoran Chang, Pradyumna Chari, Tejas Bharadwaj, Suya You, Zhangyang Wang, and Achuta Kadambi. Dream- scene360: Unconstrained text-to-3d scene generation with panoramic gaussian splatting. InEuropean Con- ference on Computer Vision, pages 324–342. Springer,

  60. [60]

    result1.mp4

    Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: Learn- ing view synthesis using multiplane images.arXiv preprint arXiv:1805.09817, 2018. 5, 7 11 A More Ablation Study Ablation on the discard ratio in the global weighting pro- cess. Table 4Ablation on the discard ratio in the global weighting pro- cess. Disca...

    Pith/arXiv arXiv 2018