Pith. sign in

REVIEW 3 major objections 5 minor 76 references

3D scenes are generated more consistently by diffusing inside a unified 3D foundation representation than inside 2D video latents.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 23:57 UTC pith:4K7ADCNJ

load-bearing objection Solid systems paper: diffusion in a unified 3D foundation-model token space with CVC and MDF beats recent 2D-latent 3DGS baselines; empirical claim holds, formal MDF story is secondary. the 3 major comments →

arxiv 2603.16100 v2 pith:4K7ADCNJ submitted 2026-03-17 cs.CV

Reevaluating the Intra-Modal Misalignment Hypothesis in CLIP

Jonas Herzog , Yue Wang This is my paper
classification cs.CV
keywords 3D scene generationdiffusion models3D Gaussian Splattingrepresentation autoencodercross-view consistencymanifold drift3D foundation models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Current diffusion methods for 3D scene generation mostly work in 2D image or video latent spaces. That makes it hard to keep appearance and geometry consistent across views. This paper argues that the right place to run the diffusion process is inside a single coherent 3D representation that already encodes geometry, appearance, and semantics together. The authors build that representation by taking a pretrained feed-forward 3D reconstructor, injecting appearance tokens, and distilling semantic structure so the resulting tokens can be decoded into renderable 3D Gaussians. They then train a conditional diffusion model in that space, add an explicit cross-view correspondence loss so denoising preserves structural matches between views, and train the decoder on mixtures of clean and drifted latents so it stays robust when sampling drifts off the manifold. The result is higher-fidelity, more multi-view-consistent 3D scenes than recent 2D-latent baselines on standard indoor and outdoor benchmarks.

Core claim

Diffusing directly in a unified 3D representation space produced by a 3D Unified Representation Autoencoder (geometry tokens augmented with appearance injection and semantic distillation), regularized by token-level cross-view correspondence and manifold-drift forcing, yields 3D Gaussian scenes whose cross-view consistency and novel-view quality exceed those of methods that diffuse in compressed 2D video latents and decode afterward.

What carries the argument

3D Unified Representation Autoencoder (3D-URAE): a frozen-or-finetuned 3D foundation encoder whose tokens are enriched by an appearance-injection branch and a semantic-distillation branch so that a single latent simultaneously carries geometry, appearance, and semantics and can be decoded into 3D Gaussians; the diffusion model then operates on these tokens, with Cross-View Correspondence loss preserving nearest-neighbor structure across views and Manifold-Drift Forcing training the decoder on linear mixtures of clean and intermediate-sampled latents.

Load-bearing premise

Training the 3D Gaussian decoder on simple linear mixtures of clean tokens and intermediate diffusion predictions is enough to cover the off-manifold drift that appears when many coupled views are sampled together at inference.

What would settle it

Run the full multi-view sampling trajectory without Manifold-Drift Forcing, measure the distance of the final latents from the 3D-URAE manifold and the resulting drop in multi-view PSNR/SSIM/LPIPS; if the decoder still produces consistent geometry and appearance without the mixture training, the drift-robustness claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Scene generators can discard frozen 2D video VAEs and the separate geometry-versus-appearance pipelines that accompany them.
  • Cross-view structural consistency becomes an explicit training signal rather than an emergent side-effect of reconstruction losses.
  • Decoder robustness to sampling drift can be improved by cheap latent interpolation instead of full end-to-end fine-tuning of the diffusion model.
  • Pretrained 3D foundation models become usable generative backbones once appearance and semantics are injected into their tokens.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same unified-token + correspondence + drift-forcing pattern could be applied to other multi-view or multi-modal foundation models beyond the particular reconstructor used here.
  • If the residual off-manifold error still grows with the number of coupled views, stronger projection operators or learned correctors may be needed at sampling time.
  • Higher-resolution training of the same pipeline would test whether the consistency gains survive when fine texture and thin structures become the limiting factor.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. OneWorld proposes diffusion-based 3D scene generation directly in a unified 3D token space rather than 2D image/video latents. The core component is 3D-URAE, which fine-tunes a feed-forward 3D foundation model (π3) by injecting appearance tokens and distilling DINOv2 semantics into geometry tokens so that a single latent can be decoded to renderable 3DGS. Conditional DiT training is regularized by a token-level Cross-View Correspondence (CVC) loss that preserves nearest-neighbor matches between target and conditioning views. Manifold-Drift Forcing (MDF) then trains the 3DGS decoder on linear mixtures of ground-truth and intermediate diffusion latents to mitigate train–inference exposure bias. Experiments on RealEstate10K, DL3DV, and WorldScore-style protocols report gains over recent 2D-latent and geometry-aware baselines (FlashWorld, Gen3R, etc.) on novel-view fidelity and multi-view consistency, with ablations attributing improvements to appearance injection, semantic distillation, CVC, and MDF.

Significance. If the empirical gains hold under broader evaluation, the work is a meaningful step toward treating pretrained 3D foundation representations as a native generative space, analogous to recent RAE-style image generation. Strengths include a clear modular design, component-wise ablations (Tables 1, 4–6), head-to-head comparisons against strong recent systems, and an explicit (if informal) analysis of multi-view sampling drift in Appendix D. The promised code release would further support reproducibility. The contribution is primarily systems/empirical rather than theoretical; the MDF Lipschitz sketch is motivational, not a quantitative guarantee.

major comments (3)
  1. Sec. 4.2 / Table 3 outdoor protocol: the outdoor WorldScore evaluation is not the official outdoor split but an author-constructed set of 500 DL3DV scenes with WorldScore camera trajectories. This is a non-standard protocol whose difficulty relative to the official outdoor distribution is uncharacterized. Because outdoor 3D Consistency / Photometric Consistency are used as headline evidence of superior multi-view coherence, the paper should either evaluate on the official outdoor split, or provide a controlled comparison showing that the custom split does not systematically favor methods trained on DL3DV.
  2. Tables 2–4: all reported metrics are single-run point estimates with no error bars, multi-seed averages, or statistical tests. Several gains over FlashWorld/Gen3R are modest (e.g., Re10K PSNR 21.57 vs 20.18/20.09; DL3DV PSNR 17.19 vs 16.02/15.94). Without variance estimates it is hard to judge whether the central claim of consistent superiority is robust, especially under the acknowledged limited data scale and low resolution (224×448).
  3. Sec. 3.3 Eq. (13) and Appendix D: MDF trains the decoder on linear interpolations α·V̂₀^(t)+(1−α)V for t∈[T1,T2]. The appendix gives a Lipschitz rollout bound on distance to the 3D-URAE manifold but no quantitative residual-drift or multi-view rendering-error guarantee under coupled N-view attention. The ablation (Table 4: w/o MDF still 20.59 PSNR) shows a real but moderate contribution; the paper should either strengthen the coverage argument (e.g., measure actual off-manifold distance of full multi-step samples vs. the linear mixtures) or clearly frame MDF as an empirical robustness heuristic rather than a theoretically justified fix for amplified multi-view drift.
minor comments (5)
  1. Fig. 1 and architecture comparison: the schematic is useful, but the claim that Gen3R “can only generate geometry and appearance separately” should be stated more carefully with a precise citation to Gen3R’s pipeline so readers can verify the contrast.
  2. Sec. 3.1 Eqs. (5)–(6): the marginal cosine and distance-matrix losses follow VA-VAE; a short note on why ReLU margins m1=m2=0.05 are preferred over a plain cosine alignment would help (Appendix A partially addresses this but is easy to miss).
  3. Sec. 4.1: training uses a 1:1 Re10K/DL3DV mixture and classifier-free text drop 0.5 “to focus on non-text-conditioned generation,” yet WorldScore evaluation uses text prompts. Clarify how text conditioning is handled at inference for the WorldScore numbers.
  4. Appendix F.1 limitations correctly note limited scale/diversity and low resolution; consider adding a short quantitative note on failure modes (e.g., thin structures, large baseline jumps) in the main text near Figs. 4–6.
  5. Typos / polish: “University of Syndey” on the title page; occasional awkward phrasing (“noise views”); ensure all arXiv concurrent works (Gen3R, FlashWorld) have stable citations once versions settle.

Circularity Check

0 steps flagged

No significant circularity: empirical 3D generation systems paper whose claims rest on external metrics and ablations, not on quantities forced by construction.

full rationale

OneWorld is a standard empirical systems paper. The central claim—that diffusion in a unified 3D-URAE token space with CVC and MDF improves cross-view NVS and WorldScore consistency over 2D-latent baselines—is supported by head-to-head numbers on external protocols (PSNR/SSIM/LPIPS, VBench I2V axes, WorldScore axes; Tabs. 2–3) and by component ablations (Tab. 4: w/o CVC and w/o MDF both degrade). Hyperparameters (λ_sem, m1/m2, τ, λ_cvc, T1/T2, α) are tuned, but the reported gains are not algebraic restatements of those fits. Appendix D’s Lipschitz rollout sketch motivates MDF as a robustness regularizer; it does not define the evaluation metrics or force the ranking versus FlashWorld/Gen3R. No self-definitional loop, no fitted-input-called-prediction, no load-bearing uniqueness theorem imported from overlapping authors, and no renaming of a known result as a derivation. Outdoor WorldScore-style evaluation reuses the DL3DV training domain, which is a distribution-match caveat, not circularity in the derivation chain. Score 0 is appropriate.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 3 invented entities

The central claim rests on standard diffusion and 3DGS machinery plus several paper-specific modeling choices and many hand-set loss weights. No new physical entities are postulated; the invented pieces are architectural modules and training objectives whose value is argued empirically on the same generation benchmarks used for the headline claim.

free parameters (5)
  • semantic distillation weight λ_sem
    Set to 0.1 after ablation; controls how strongly geometry tokens are pulled toward VFM semantics and affects both reconstruction and generation.
  • semantic margins m1=m2
    Chosen as 0.05; Tab. 5 shows reconstruction/semantic-similarity trade-offs when varied.
  • CVC threshold τ and weight λ_cvc
    Defaults τ=0.9, λ_cvc=0.2 selected via short ablations (Tab. 6); directly affect the structural regularizer that supports the consistency claim.
  • MDF timestep window [T1,T2] and mix ratio α
    T1=10, T2=20 and α~U[0,1] chosen by hand; define the drifted latents used to train decoder robustness.
  • rendering LPIPS weight λ_lpips
    Fixed at 0.05 in 3D-URAE and MDF rendering losses; shapes the perceptual reconstruction objective.
axioms (5)
  • domain assumption Clean 3D-URAE multi-view tokens concentrate on a low-dimensional manifold suitable for x0-prediction diffusion (Sec. 3.2; Appendix B citing JiT).
    Used to justify predicting clean tokens rather than velocity/noise in high-dimensional feature space.
  • domain assumption A pretrained feed-forward 3D reconstructor (π3) plus light fine-tuning provides a geometry-aware token basis that can be made appearance- and semantics-complete.
    Load-bearing premise of 3D-URAE; without it the unified space is not available.
  • domain assumption Differentiable 3DGS rendering losses are a faithful training signal for both autoencoding and decoder robustness.
    Standard in 3DGS literature; invoked throughout Sec. 3.1 and 3.3.
  • ad hoc to paper Train–inference exposure bias causes off-manifold drift that is amplified by cross-view coupling (Appendix D Claims 1–2).
    Motivates MDF; supported by a Lipschitz sketch rather than measured drift statistics.
  • standard math Standard diffusion process and v/x0 conversion identities hold in the unified token space.
    Background generative-model math used in Sec. 3.2.
invented entities (3)
  • 3D Unified Representation Autoencoder (3D-URAE) no independent evidence
    purpose: Produce a single 3D latent that jointly encodes geometry, appearance, and semantics for diffusion and 3DGS decoding.
    Core architectural construct; validated only via this paper’s reconstruction/generation ablations, not independent external theory.
  • token-level Cross-View Correspondence (CVC) loss no independent evidence
    purpose: Force predicted target tokens to preserve nearest-neighbor matching patterns to the conditioning view.
    Paper-specific regularizer; benefit shown by ablation, not by external prior validation of this exact objective.
  • Manifold-Drift Forcing (MDF) no independent evidence
    purpose: Train the 3D decoder on mixtures of sampled and ground-truth latents to tolerate inference drift.
    Named training strategy introduced here; appendix sketch does not independently establish the entity outside the reported experiments.

pith-pipeline@v1.1.0-grok45 · 25848 in / 3753 out tokens · 53673 ms · 2026-07-13T23:57:46.882342+00:00 · methodology

0 comments
read the original abstract

Recent research suggested that the embeddings produced by CLIP-like contrastive language-image training are suboptimal for image-only tasks. The main theory is that the inter-modal (language-image) alignment loss ignores intra-modal (image-image) alignment, leading to poorly calibrated distances between images. In this study, we question this intra-modal misalignment hypothesis. We reexamine its foundational theoretical argument, the indicators used to support it, and the performance metrics affected. For the theoretical argument, we demonstrate that there are no such supposed degrees of freedom for image embedding distances. For the empirical measures, our findings reveal they yield similar results for language-image trained models (CLIP, SigLIP) and image-image trained models (DINO, SigLIP2). This indicates the observed phenomena do not stem from a misalignment specific to the former. Experiments on the commonly studied intra-modal tasks retrieval and few-shot classification confirm that addressing task ambiguity, not supposed misalignment, is key for best results.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

76 extracted references · 24 linked inside Pith

  1. [1]

    Vision foundation models can be good tokenizers for latent diffusion models.arXiv preprint arXiv:2510.18457, 2025

    Tianci Bi, Xiaoyi Zhang, Yan Lu, and Nanning Zheng. Vision foundation models can be good tokenizers for latent diffusion models.arXiv preprint arXiv:2510.18457, 2025

    Pith/arXiv arXiv 2025

  2. [2]

    Must3r: Multi-view network for stereo 3d reconstruction

    Yohann Cabon, Lucas Stoffl, Leonid Antsfeld, Gabriela Csurka, Boris Chidlovskii, Jerome Revaud, and Vincent Leroy. Must3r: Multi-view network for stereo 3d reconstruction. InIEEE Conf. Comput. Vis. Pattern Recog., pages 1050–1060, 2025

    2025

  3. [3]

    pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction

    David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann. pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19457–19467, 2024

    2024

  4. [4]

    Aligning visual foundation encoders to tokenizers for diffusion models

    Bowei Chen, Sai Bi, Hao Tan, He Zhang, Tianyuan Zhang, Zhengqi Li, Yuanjun Xiong, Jianming Zhang, and Kai Zhang. Aligning visual foundation encoders to tokenizers for diffusion models. arXiv preprint arXiv:2509.25162, 2025

    arXiv 2025

  5. [5]

    Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images

    Yuedong Chen, Haofei Xu, Chuanxia Zheng, Bohan Zhuang, Marc Pollefeys, Andreas Geiger, Tat-Jen Cham, and Jianfei Cai. Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images. InEuropean conference on computer vision, pages 370–386. Springer, 2024

    2024

  6. [6]

    Mvsplat360: Feed-forward 360 scene synthesis from sparse views.Adv

    Yuedong Chen, Chuanxia Zheng, Haofei Xu, Bohan Zhuang, Andrea Vedaldi, Tat-Jen Cham, and Jianfei Cai. Mvsplat360: Feed-forward 360 scene synthesis from sparse views.Adv. Neural Inform. Process. Syst., 37:107064–107086, 2024

    2024

  7. [7]

    Lu- ciddreamer: Domain-free generation of 3d gaussian splatting scenes.arXiv preprint arXiv:2311.13384, 2023

    Jaeyoung Chung, Suyoung Lee, Hyeongjin Nam, Jaerin Lee, and Kyoung Mu Lee. Lu- ciddreamer: Domain-free generation of 3d gaussian splatting scenes.arXiv preprint arXiv:2311.13384, 2023

    Pith/arXiv arXiv 2023

  8. [8]

    Fantasyworld: Geometry- consistent world modeling via unified video and 3d prediction.arXiv preprint arXiv:2509.21657, 2025

    Yixiang Dai, Fan Jiang, Chiyu Wang, Mu Xu, and Yonggang Qi. Fantasyworld: Geometry- consistent world modeling via unified video and 3d prediction.arXiv preprint arXiv:2509.21657, 2025

    arXiv 2025

  9. [9]

    Understanding world or predicting future? a comprehensive survey of world models.ACM Computing Surveys, 58(3):1–38, 2025

    Jingtao Ding, Yunke Zhang, Yu Shang, Yuheng Zhang, Zefang Zong, Jie Feng, Yuan Yuan, Hongyuan Su, Nian Li, Nicholas Sukiennik, et al. Understanding world or predicting future? a comprehensive survey of world models.ACM Computing Surveys, 58(3):1–38, 2025

    2025

  10. [10]

    Worldscore: A unified evaluation benchmark for world generation.arXiv preprint arXiv:2504.00983, 2025

    Haoyi Duan, Hong-Xing Yu, Sirui Chen, Li Fei-Fei, and Jiajun Wu. Worldscore: A unified evaluation benchmark for world generation.arXiv preprint arXiv:2504.00983, 2025

    arXiv 2025

  11. [11]

    Scaling rectified flow transform- ers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transform- ers for high-resolution image synthesis. InInt. Conf. Mach. Learn., 2024

    2024

  12. [12]

    Scenescape: Text-driven consistent scene generation.Adv

    Rafail Fridman, Amit Abecasis, Yoni Kasten, and Tali Dekel. Scenescape: Text-driven consistent scene generation.Adv. Neural Inform. Process. Syst., 36:39897–39914, 2023

    2023

  13. [13]

    Cat3d: Create anything in 3d with multi-view diffusion models.arXiv preprint arXiv:2405.10314, 2024

    Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul Srinivasan, Jonathan T Barron, and Ben Poole. Cat3d: Create anything in 3d with multi-view diffusion models.arXiv preprint arXiv:2405.10314, 2024

    Pith/arXiv arXiv 2024

  14. [14]

    Vist3a: Text-to-3d by stitching a multi-view reconstruction network to a video generator.arXiv preprint arXiv:2510.13454, 2025

    Hyojun Go, Dominik Narnhofer, Goutam Bhat, Prune Truong, Federico Tombari, and Konrad Schindler. Vist3a: Text-to-3d by stitching a multi-view reconstruction network to a video generator.arXiv preprint arXiv:2510.13454, 2025

    arXiv 2025

  15. [15]

    Splatflow: Multi-view rectified flow model for 3d gaussian splatting synthesis

    Hyojun Go, Byeongjun Park, Jiho Jang, Jin-Young Kim, Soonwoo Kwon, and Changick Kim. Splatflow: Multi-view rectified flow model for 3d gaussian splatting synthesis. InIEEE Conf. Comput. Vis. Pattern Recog., pages 21524–21536, 2025

    2025

  16. [16]

    Videorfsplat: Direct scene-level text-to-3d gaussian splatting generation with flexible pose and multi-view joint modeling.arXiv preprint arXiv:2503.15855, 2025

    Hyojun Go, Byeongjun Park, Hyelin Nam, Byung-Hoon Kim, Hyungjin Chung, and Changick Kim. Videorfsplat: Direct scene-level text-to-3d gaussian splatting generation with flexible pose and multi-view joint modeling.arXiv preprint arXiv:2503.15855, 2025. 20

    Pith/arXiv arXiv 2025

  17. [17]

    Dong Guo, Faming Wu, Feida Zhu, Fuxing Leng, Guang Shi, Haobin Chen, Haoqi Fan, Jian Wang, Jianyu Jiang, Jiawei Wang, et al. Seed1. 5-vl technical report.arXiv preprint arXiv:2505.07062, 2025

    Pith/arXiv arXiv 2025

  18. [18]

    Gaussvideo- dreamer: 3d scene generation with video diffusion and inconsistency-aware gaussian splatting

    Junlin Hao, Peiheng Wang, Haoyang Wang, Xinggong Zhang, and Zongming Guo. Gaussvideo- dreamer: 3d scene generation with video diffusion and inconsistency-aware gaussian splatting. arXiv preprint arXiv:2504.10001, 2025

    Pith/arXiv arXiv 2025

  19. [19]

    Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  20. [20]

    Gen3r: 3d scene generation meets feed-forward reconstruction.arXiv preprint arXiv:2601.04090, 2026

    Jiaxin Huang, Yuanbo Yang, Bangbang Yang, Lin Ma, Yuewen Ma, and Yiyi Liao. Gen3r: 3d scene generation meets feed-forward reconstruction.arXiv preprint arXiv:2601.04090, 2026

    arXiv 2026

  21. [21]

    Vbench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. InIEEE Conf. Comput. Vis. Pattern Recog., pages 21807– 21818, 2024

    2024

  22. [22]

    Vbench++: Comprehensive and versatile benchmark suite for video generative models.IEEE Trans

    Ziqi Huang, Fan Zhang, Xiaojie Xu, Yinan He, Jiashuo Yu, Ziyue Dong, Qianli Ma, Nattapol Chanpaisit, Chenyang Si, Yuming Jiang, et al. Vbench++: Comprehensive and versatile benchmark suite for video generative models.IEEE Trans. Pattern Anal. Mach. Intell., 2025

    2025

  23. [23]

    Anysplat: Feed-forward 3d gaussian splatting from unconstrained views.ACM Transactions on Graphics (TOG), 44(6):1–16, 2025

    Lihan Jiang, Yucheng Mao, Linning Xu, Tao Lu, Kerui Ren, Yichen Jin, Xudong Xu, Mulin Yu, Jiangmiao Pang, Feng Zhao, et al. Anysplat: Feed-forward 3d gaussian splatting from unconstrained views.ACM Transactions on Graphics (TOG), 44(6):1–16, 2025

    2025

  24. [24]

    Lvsm: A large view synthesis model with minimal 3d inductive bias

    Haian Jin, Hanwen Jiang, Hao Tan, Kai Zhang, Sai Bi, Tianyuan Zhang, Fujun Luan, Noah Snavely, and Zexiang Xu. Lvsm: A large view synthesis model with minimal 3d inductive bias. InInt. Conf. Learn. Represent

  25. [25]

    Elucidating the design space of diffusion-based generative models.Advances in neural information processing systems, 35: 26565–26577, 2022

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models.Advances in neural information processing systems, 35: 26565–26577, 2022

    2022

  26. [26]

    3d gaussian splatting for real-time radiance field rendering.ACM Trans

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George Drettakis, et al. 3d gaussian splatting for real-time radiance field rendering.ACM Trans. Graph., 42(4):139–1, 2023

    2023

  27. [27]

    Auto-encoding variational bayes.arXiv preprint arXiv:1312.6114, 2013

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes.arXiv preprint arXiv:1312.6114, 2013

    Pith/arXiv arXiv 2013

  28. [28]

    3d and 4d world modeling: A survey.arXiv preprint arXiv:2509.07996, 2025

    Lingdong Kong, Wesley Yang, Jianbiao Mei, Youquan Liu, Ao Liang, Dekai Zhu, Dongyue Lu, Wei Yin, Xiaotao Hu, Mingkai Jia, et al. 3d and 4d world modeling: A survey.arXiv preprint arXiv:2509.07996, 2025

    arXiv 2025

  29. [29]

    Grounding image matching in 3d with mast3r

    Vincent Leroy, Yohann Cabon, and Jérôme Revaud. Grounding image matching in 3d with mast3r. InEur . Conf. Comput. Vis., pages 71–91. Springer, 2024

    2024

  30. [30]

    Back to basics: Let denoising generative models denoise.arXiv preprint arXiv:2511.13720, 2025

    Tianhong Li and Kaiming He. Back to basics: Let denoising generative models denoise.arXiv preprint arXiv:2511.13720, 2025

    Pith/arXiv arXiv 2025

  31. [31]

    Director3d: Real-world camera trajectory and 3d scene generation from text

    Xinyang Li, Zhangyu Lai, Linning Xu, Yansong Qu, Liujuan Cao, Shengchuan Zhang, Bo Dai, and Rongrong Ji. Director3d: Real-world camera trajectory and 3d scene generation from text. Adv. Neural Inform. Process. Syst., 37:75125–75151, 2024

    2024

  32. [32]

    Flashworld: High-quality 3d scene generation within seconds.arXiv preprint arXiv:2510.13678, 2025

    Xinyang Li, Tengfei Wang, Zixiao Gu, Shengchuan Zhang, Chunchao Guo, and Liujuan Cao. Flashworld: High-quality 3d scene generation within seconds.arXiv preprint arXiv:2510.13678, 2025

    arXiv 2025

  33. [33]

    Magic3d: High-resolution text-to-3d content creation

    Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3d: High-resolution text-to-3d content creation. InIEEE Conf. Comput. Vis. Pattern Recog., pages 300–309, 2023. 21

    2023

  34. [34]

    Depth anything 3: Recovering the visual space from any views.arXiv preprint arXiv:2511.10647, 2025

    Haotong Lin, Sili Chen, Junhao Liew, Donny Y Chen, Zhenyu Li, Guang Shi, Jiashi Feng, and Bingyi Kang. Depth anything 3: Recovering the visual space from any views.arXiv preprint arXiv:2511.10647, 2025

    Pith/arXiv arXiv 2025

  35. [35]

    Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision

    Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, et al. Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision. InIEEE Conf. Comput. Vis. Pattern Recog., pages 22160–22169, 2024

    2024

  36. [36]

    Reconx: Reconstruct any scene from sparse views with video diffusion model

    Fangfu Liu, Wenqiang Sun, Hanyang Wang, Yikai Wang, Haowen Sun, Junliang Ye, Jun Zhang, and Yueqi Duan. Reconx: Reconstruct any scene from sparse views with video diffusion model. arXiv preprint arXiv:2408.16767, 2024

    Pith/arXiv arXiv 2024

  37. [37]

    Zero-1-to-3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl V ondrick. Zero-1-to-3: Zero-shot one image to 3d object. InProceedings of the IEEE/CVF international conference on computer vision, pages 9298–9309, 2023

    2023

  38. [38]

    Syncdreamer: Generating multiview-consistent images from a single-view image.arXiv preprint arXiv:2309.03453, 2023

    Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Generating multiview-consistent images from a single-view image.arXiv preprint arXiv:2309.03453, 2023

    Pith/arXiv arXiv 2023

  39. [39]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Pith/arXiv arXiv 2017

  40. [40]

    Nerf: Representing scenes as neural radiance fields for view synthesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoor- thi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99–106, 2021

    2021

  41. [41]

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jégou, Julien Mairal, Patrick L...

    2024

  42. [42]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InInt. Conf. Comput. Vis., pages 4195–4205, 2023

    2023

  43. [43]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    Pith/arXiv arXiv 2023

  44. [44]

    Dreamfusion: Text-to-3d using 2d diffusion.arXiv preprint arXiv:2209.14988, 2022

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion.arXiv preprint arXiv:2209.14988, 2022

    Pith/arXiv arXiv 2022

  45. [45]

    Gen3c: 3d-informed world- consistent video generation with precise camera control

    Xuanchi Ren, Tianchang Shen, Jiahui Huang, Huan Ling, Yifan Lu, Merlin Nimier-David, Thomas Müller, Alexander Keller, Sanja Fidler, and Jun Gao. Gen3c: 3d-informed world- consistent video generation with precise camera control. InIEEE Conf. Comput. Vis. Pattern Recog., pages 6121–6132, 2025

    2025

  46. [46]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. InIEEE Conf. Comput. Vis. Pattern Recog., pages 10684–10695, 2022

    2022

  47. [47]

    Zeronvs: Zero-shot 360-degree view synthesis from a single real image.arXiv preprint arXiv:2310.17994, 2023

    Kyle Sargent, Zizhang Li, Tanmay Shah, Charles Herrmann, Hong-Xing Yu, Yunzhi Zhang, Eric Ryan Chan, Dmitry Lagun, Li Fei-Fei, Deqing Sun, et al. Zeronvs: Zero-shot 360-degree view synthesis from a single real image.arXiv preprint arXiv:2310.17994, 2023

    Pith/arXiv arXiv 2023

  48. [48]

    A recipe for generating 3d worlds from a single image

    Katja Schwarz, Denis Rozumny, Samuel Rota Bulò, Lorenzo Porzi, and Peter Kontschieder. A recipe for generating 3d worlds from a single image. InInt. Conf. Comput. Vis., pages 3520–3530, 2025. 22

    2025

  49. [49]

    Latent diffusion model without variational autoencoder.arXiv preprint arXiv:2510.15301, 2025

    Minglei Shi, Haolin Wang, Wenzhao Zheng, Ziyang Yuan, Xiaoshi Wu, Xintao Wang, Pengfei Wan, Jie Zhou, and Jiwen Lu. Latent diffusion model without variational autoencoder.arXiv preprint arXiv:2510.15301, 2025

    arXiv 2025

  50. [50]

    Mvdream: Multi- view diffusion for 3d generation.arXiv preprint arXiv:2308.16512, 2023

    Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi- view diffusion for 3d generation.arXiv preprint arXiv:2308.16512, 2023

    Pith/arXiv arXiv 2023

  51. [51]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations,

  52. [52]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations,

  53. [53]

    Dimensionx: Create any 3d and 4d scenes from a single image with controllable video diffusion

    Wenqiang Sun, Shuo Chen, Fangfu Liu, Zilong Chen, Yueqi Duan, Jun Zhang, and Yikai Wang. Dimensionx: Create any 3d and 4d scenes from a single image with controllable video diffusion. arXiv preprint arXiv:2411.04928, 2024

    Pith/arXiv arXiv 2024

  54. [54]

    Dreamgaussian: Generative gaussian splatting for efficient 3d content creation.arXiv preprint arXiv:2309.16653, 2023

    Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for efficient 3d content creation.arXiv preprint arXiv:2309.16653, 2023

    Pith/arXiv arXiv 2023

  55. [55]

    Neural discrete representation learning.Adv

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning.Adv. Neural Inform. Process. Syst., 30, 2017

    2017

  56. [56]

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Pith/arXiv arXiv 2025

  57. [57]

    Vggt: Visual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. InIEEE Conf. Comput. Vis. Pattern Recog., pages 5294–5306, 2025

    2025

  58. [58]

    Dust3r: Geometric 3d vision made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. InIEEE Conf. Comput. Vis. Pattern Recog., pages 20697–20709, 2024

    2024

  59. [59]

    π3: Permutation-equivariant visual geometry learning.arXiv preprint arXiv:2507.13347, 2025

    Yifan Wang, Jianjun Zhou, Haoyi Zhu, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Jiangmiao Pang, Chunhua Shen, and Tong He. π3: Permutation-equivariant visual geometry learning.arXiv preprint arXiv:2507.13347, 2025

    Pith/arXiv arXiv 2025

  60. [60]

    Pro- lificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation

    Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Pro- lificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. Adv. Neural Inform. Process. Syst., 36:8406–8441, 2023

    2023

  61. [61]

    Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing, 13(4): 600–612, 2004

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing, 13(4): 600–612, 2004

    2004

  62. [62]

    Ge- ometry forcing: Marrying video diffusion and 3d representation for consistent world modeling

    Haoyu Wu, Diankun Wu, Tianyu He, Junliang Guo, Yang Ye, Yueqi Duan, and Jiang Bian. Ge- ometry forcing: Marrying video diffusion and 3d representation for consistent world modeling. arXiv preprint arXiv:2507.07982, 2025

    Pith/arXiv arXiv 2025

  63. [63]

    Reconfusion: 3d reconstruction with diffusion priors

    Rundi Wu, Ben Mildenhall, Philipp Henzler, Keunhong Park, Ruiqi Gao, Daniel Watson, Pratul P Srinivasan, Dor Verbin, Jonathan T Barron, Ben Poole, et al. Reconfusion: 3d reconstruction with diffusion priors. InIEEE Conf. Comput. Vis. Pattern Recog., pages 21551– 21561, 2024

    2024

  64. [64]

    Depthsplat: Connecting gaussian splatting and depth

    Haofei Xu, Songyou Peng, Fangjinhua Wang, Hermann Blum, Daniel Barath, Andreas Geiger, and Marc Pollefeys. Depthsplat: Connecting gaussian splatting and depth. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 16453–16463, 2025

    2025

  65. [65]

    Prometheus: 3d-aware latent diffusion models for feed-forward text-to-3d scene generation

    Yuanbo Yang, Jiahao Shao, Xinyang Li, Yujun Shen, Andreas Geiger, and Yiyi Liao. Prometheus: 3d-aware latent diffusion models for feed-forward text-to-3d scene generation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 2857–2869, 2025. 23

    2025

  66. [66]

    Reconstruction vs

    Jingfeng Yao, Bin Yang, and Xinggang Wang. Reconstruction vs. generation: Taming op- timization dilemma in latent diffusion models. InIEEE Conf. Comput. Vis. Pattern Recog., 2025

    2025

  67. [67]

    World action models are zero-shot policies.arXiv preprint arXiv:2602.15922, 2026

    Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng, Shenyuan Gao, Sihyun Yu, George Kurian, Suneel Indupuru, You Liang Tan, Chuning Zhu, Jiannan Xiang, et al. World action models are zero-shot policies.arXiv preprint arXiv:2602.15922, 2026

    Pith/arXiv arXiv 2026

  68. [68]

    Wonderjourney: Going from anywhere to everywhere

    Hong-Xing Yu, Haoyi Duan, Junhwa Hur, Kyle Sargent, Michael Rubinstein, William T Freeman, Forrester Cole, Deqing Sun, Noah Snavely, Jiajun Wu, et al. Wonderjourney: Going from anywhere to everywhere. InIEEE Conf. Comput. Vis. Pattern Recog., pages 6658–6667, 2024

    2024

  69. [69]

    Wonder- world: Interactive 3d scene generation from a single image

    Hong-Xing Yu, Haoyi Duan, Charles Herrmann, William T Freeman, and Jiajun Wu. Wonder- world: Interactive 3d scene generation from a single image. InIEEE Conf. Comput. Vis. Pattern Recog., pages 5916–5926, 2025

    2025

  70. [70]

    Advances in feed-forward 3d reconstruction and view synthesis: A survey.arXiv preprint arXiv:2507.14501, 2025

    Jiahui Zhang, Yuelei Li, Anpei Chen, Muyu Xu, Kunhao Liu, Jianyuan Wang, Xiao-Xiao Long, Hanxue Liang, Zexiang Xu, Hao Su, et al. Advances in feed-forward 3d reconstruction and view synthesis: A survey.arXiv preprint arXiv:2507.14501, 2025

    arXiv 2025

  71. [71]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. InInt. Conf. Comput. Vis., pages 3836–3847, 2023

    2023

  72. [72]

    The unreason- able effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreason- able effectiveness of deep features as a perceptual metric. InIEEE Conf. Comput. Vis. Pattern Recog., pages 586–595, 2018

    2018

  73. [73]

    Genxd: Generating any 3d and 4d scenes.arXiv preprint arXiv:2411.02319, 2024

    Yuyang Zhao, Chung-Ching Lin, Kevin Lin, Zhiwen Yan, Linjie Li, Zhengyuan Yang, Jianfeng Wang, Gim Hee Lee, and Lijuan Wang. Genxd: Generating any 3d and 4d scenes.arXiv preprint arXiv:2411.02319, 2024

    Pith/arXiv arXiv 2024

  74. [74]

    Diffusion transformers with representation autoencoders.arXiv preprint arXiv:2510.11690, 2025

    Boyang Zheng, Nanye Ma, Shengbang Tong, and Saining Xie. Diffusion transformers with representation autoencoders.arXiv preprint arXiv:2510.11690, 2025

    Pith/arXiv arXiv 2025

  75. [75]

    Stereo magni- fication: learning view synthesis using multiplane images.ACM Trans

    Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magni- fication: learning view synthesis using multiplane images.ACM Trans. Graph., 37(4):1–12, 2018

    2018

  76. [76]

    Aether: Geometric-aware unified world modeling

    Haoyi Zhu, Yifan Wang, Jianjun Zhou, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Chunhua Shen, Jiangmiao Pang, and Tong He. Aether: Geometric-aware unified world modeling. InInt. Conf. Comput. Vis., pages 8535–8546, 2025. 24

    2025