Pith. sign in

REVIEW 3 major objections 5 minor 58 references

CausNVS: Autoregressive Multi-view Diffusion for Flexible 3D Novel View Synthesis

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A causal multi-view diffusion model, CausNVS, generates novel views sequentially from arbitrary input and output counts, using per-frame noise and relative camera pose encodings for stable long rollouts.

desk verdict Solid and genuinely novel autoregressive multi-view diffusion formulation, with the core flexibility claim backed by a clean causal-vs-noncausal ablation; but the long-rollout stability claim rests on an unablated drift-mitigation mechanism and the benchmark numbers are more mixed than the framing suggests. read the letter →

arxiv 2509.06579 v1 pith:GH4HQDB5 submitted 2025-09-08 cs.CV

Xin Kong , Daniel Watson , Yannick Strümpler , Michael Niemeyer , Federico Tombari This is my paper · ORCID
classification cs.CV
keywords autoregressivenovelviewsynthesismulti-viewdiffusioncausalmaskingper-framenoiseconditioningrelativecameraposeencodingKVcaching3Dconsistencyworldmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Novel view synthesis is usually done by jointly denoising a fixed set of target views, which forces the model to know all future camera poses ahead of time and limits it to one input-output configuration. CausNVS turns this into an autoregressive problem: a single model trained on eight-frame sequences learns to emit views one at a time, conditioned only on the input views and whatever it has already generated. The paper claims this works for arbitrary numbers of input views and output views across diverse camera trajectories, and that rollouts remain visually coherent for sequence lengths several times the training horizon. If correct, the same checkpoint becomes a streaming-capable 3D prior that needs no retraining, no padding, and no recomputation of past attention when the camera moves.

What carries the argument

The load-bearing mechanism is the combination of frame-wise attention with causal masking inside a pretrained latent-diffusion backbone, per-frame independent noise levels during training so conditioning on partially noisy or previously generated views is in-distribution at inference, and CaPE, a parameter-free relative camera pose encoding that rotates query and key features from pairwise pose matrices. CaPE makes attention scores invariant to global coordinate shifts, which is what allows key-value caching and pose-aware sliding windows to remain valid as the reference frame changes. The causal mask also makes every prefix length a valid training configuration, so one model covers many different input-to-output settings.

What would settle it

Use the same training data and architecture but replace per-frame independent noise levels with a single shared noise level, then evaluate rollouts of length 32 and 64; if image quality and 3D consistency degrade no faster than in the full model, the claimed drift-mitigation mechanism is not doing the work.

Watch

Extended reading notes

Core claim

The central discovery is that multi-view diffusion can be made causal without giving up quality or flexibility. Training with causal masking across frame-wise attention tokens, independently sampled noise levels per frame, and pairwise-relative camera pose encodings yields a model that generates target views autoregressively, with each new view attending only to earlier frames. At inference, previously generated views are re-fed with small assigned noise levels, and attention is restricted to a sliding window of the nearest views in pose space while cached key-value states carry longer-range context. The authors report that this combination keeps generation stable for rollouts well beyond the training length, improves as more input views are supplied, and supports customized trajectories including revisiting viewpoints.

Load-bearing premise

Long-rollout stability rests on the assumption that per-frame noise conditioning makes the model robust enough to condition on its own imperfect outputs; the paper uses this design but never isolates its contribution.

Editorial extensions

If this is right

  • A single CausNVS checkpoint covers a wide range of input and output view counts without retraining or padding, because causal teacher forcing exposes the model to every shorter prefix during training.
  • Autoregressive rollout stays stable for sequences roughly ten times the training horizon, with only mild degradation compared with shorter-sequence generation.
  • Adding more input views consistently improves image quality and 3D consistency, resolving single-view scale ambiguity from multi-view cues rather than from ground-truth scale sweeping.
  • Key-value caching in the frame-wise attention layers means inference cost grows with the window and the number of generated frames, not with the full history, making streaming generation feasible.
  • Non-causal models trained identically perform well only near their training sequence length and drift quickly when evaluated autoregressively, while the causal variant keeps consistent performance across configurations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the causal formulation makes every prefix length a training configuration, applying the same recipe to video or action-conditioned world models, with pose plus control tokens in the frame-wise attention, would be a direct test of whether the benefits extend beyond camera-only synthesis.
  • The paper reports single-view numbers without the ground-truth scale sweeping used by some baselines; a matched evaluation would show whether the single-view gap is a protocol artifact or a genuine method limitation.
  • A direct ablation that varies the noise assigned to already-generated views while keeping the sliding window fixed would separate how much of the long-rollout stability comes from noise conditioning augmentation rather than from the windowing and caching mechanism.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper introduces CausNVS, a multi-view latent diffusion model for novel view synthesis in which target views are generated autoregressively rather than jointly. The model applies causal masking across frame tokens in newly added frame-wise attention layers, samples per-frame noise levels during training, uses pairwise-relative camera pose encodings (CaPE) for coordinate-invariant conditioning, and employs a pose-aware sliding window with KV caching at inference. The central claims are that a single model supports arbitrary numbers of input and output views without retraining, that it can generate views sequentially for streaming settings, and that it remains stable for rollouts up to 10x the training length. The paper evaluates on RealEstate10K, LLFF, and DL3DV, comparing with fixed-view baselines, and includes an ablation contrasting causal and non-causal training.

Significance. If the central claims are supported, the paper makes a notable contribution: it identifies and addresses a genuine limitation of current multi-view diffusion models (fixed input/output view counts and forced joint denoising) and provides a causal formulation suited to streaming or interactive 3D applications. The combination of per-frame noise conditioning, CaPE, and KV caching is a clean design. The strongest evidence is the causal-vs-non-causal ablation (Table 2), which shows that the non-causal variant fails at short sequences and in autoregressive mode, demonstrating a concrete benefit of the causal formulation. The paper is honest about its training scale and does not overclaim SOTA on all benchmarks; the main uncertainty surrounds the long-rollout stability claim, which currently rests on mixed quantitative evidence and an unablated mechanism.

major comments (3)
  1. [§4.1, Table 1] The long-rollout stability claim is not quantitatively supported. The DL3DV short-vs-long comparison, which is the only evaluation explicitly targeting autoregressive drift, shows a drop of about 3.5-4 dB in PSNR at N=3 (14.42 vs 18.32) and N=6 (15.22 vs 18.75), and the long-sequence results fall below SEVA by 1.5-2.6 dB. The paper reports no per-frame PSNR as a function of rollout position and no non-causal baseline on the same long-sequence protocol, so it is not possible to tell whether the degradation reflects systematic drift, harder target poses in the long split, or the sliding-window forgetting earlier views. Without these controls, the statement in the Table 1 caption that the model maintains 'stable quality over long generation rollouts' is stronger than the evidence.
  2. [§3, Noise-Level Conditioning] The paper's stated mechanism for mitigating autoregressive drift—per-frame noise conditioning during training plus noise conditioning augmentation at inference—is never isolated. No ablation removes or varies the augmentation, and there is no experiment quantifying its effect on long rollouts. Since the paper explicitly attributes drift mitigation to this component, writing that it 'stabilizes subsequent predictions and improves robustness to accumulated errors,' the absence of a targeted ablation leaves the explanation for the long-rollout behavior unsupported. The stability could equally be due to the causal masking or to the pose-aware sliding window, which are also introduced in this work.
  3. [§4.2, Table 2] The interpretation of the causal vs non-causal ablation is contradicted at one evaluation point. The text states that the non-causal model 'performs well only under conditions close to its training setup and degrades significantly when evaluated with different sequence lengths,' but at F=32, N=1 the non-causal parallel model achieves 19.27 dB, which is higher than the causal AR model's 17.44 dB and also higher than the non-causal model's own F=8 result (17.52 dB). The non-causal model clearly fails at F=2 and F=4 and in autoregressive mode, so the central point about flexibility stands; however, the unqualified claim about degradation is not consistent with the full table. Please qualify the statement or explain why the F=32 comparison should not be taken at face value.
minor comments (5)
  1. [§3, Training Objective] There is a typo in the sentence describing the training objective: 'the model ˆεθ is trained o predict' should read 'trained to predict.'
  2. [Appendix A] The sentence 'These layers are zero-initialized (weights to identity, biases to zero)' is ambiguous, because zero-initialized weights are normally the opposite of identity-initialized weights; please clarify the intended initialization scheme.
  3. [References] Reference [19] is a Medium blog post on KV caching; the transformer literature contains more authoritative descriptions of key-value caching, and citing a primary source would be more appropriate.
  4. [Throughout] The paper uses both 'SEVA' and 'SEV A' with inconsistent spacing; please unify the spelling.
  5. [Figure 3] The analysis of CaPE attention scores is described only in the caption; please explain in the main text how the scores are computed and why the cyclic/linear variation with rotation/translation constitutes an SE(3)-aware inductive bias.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central claims are empirically evaluated, and the only self-citation (CaPE from EscherNet) is explicit, published, and not load-bearing in a circular sense.

full rationale

CausNVS is a training-and-evaluation paper whose claims are supported by experiments on public datasets (RealEstate10K, DL3DV, LLFF) against external baselines, not by definition or by fitted constants renamed as predictions. The core mechanism, causal masking with per-frame noise, is stated in the training objective (Eq. 1) and is directly validated through the N-to-M ablation in Table 2, where the causal model is compared with an identically trained non-causal model. The ability to handle arbitrary N-to-M configurations follows from the causal conditioning structure, but the paper then demonstrates this generalization on held-out data, so it is not circular in the pejorative sense. The use of Camera Pose Encoding (CaPE) is taken from the authors' prior EscherNet work; however, CaPE is a published, parameter-free method with a concrete formula (Eq. 2), and the paper uses it as a building block rather than as a proof of its own conclusions. No uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via citation. The paper does contain an unablated component, the noise conditioning augmentation, which is asserted to mitigate autoregressive drift; this is a missing ablation and a possible overclaim, but it is not circularity because the long-rollout behavior is measured, not derived from the augmentation's definition. Overall, the derivation chain is self-contained and the findings are falsifiable against external data, so the circularity score is low.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper is an empirical systems contribution with no derived formulas; free parameters are inference hyperparameters whose values are not fully disclosed. The axioms are domain assumptions about transfer from pretrained priors and generalization from fixed-length training to arbitrary configurations.

free parameters (3)
  • Noise conditioning augmentation strength at inference
    Small noise levels assigned to previously generated views; the exact schedule is not specified, yet the drift mitigation claim depends on it.
  • Spatial attention window size (top-K nearest views)
    K is configurable (Fig 6c shows 1, 4, full), but the default value used for main results is not stated; affects efficiency and consistency.
  • Per-frame noise level sampling distribution during training
    Independent noise levels t_i are sampled per frame, but the distribution (e.g., uniform vs. log-normal) is not specified; affects the learned denoising behavior.
assumptions (4)
  • domain assumption Finetuning a pretrained latent diffusion UNet with inserted frame-wise attention layers preserves the 2D prior while learning multi-view consistency.
    Relies on prior evidence from Cat3D and EscherNet; not proven in this paper.
  • domain assumption Random 8-frame sequences from RealEstate10K and DL3DV provide enough exposure to arbitrary input-output configurations and camera trajectories.
    The claim of arbitrary N-to-M generalization is inferred from training on fixed-length sequences with causal masking; empirical support is indirect.
  • domain assumption Teacher forcing during training teaches all prefixes, so the model can at inference handle any sequence length without retraining.
    Standard for autoregressive models, but the specific guarantee for view synthesis is not formalized.
  • ad hoc to paper CaPE attention scores varying cyclically with rotation and linearly with translation (Fig 3) reflect a useful SE(3)-aware inductive bias for spatial reasoning.
    Illustrative toy analysis with random queries/keys; no formal link to final NVS performance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CausNVS: Autoregressive Multi-view Diffusion for Flexible 3D Novel View Synthesis." pith.science (2026). https://pith.science/paper/GH4HQDB5

@misc{pith2026250906579,
  author       = {Pith},
  title        = {Pith review of: CausNVS: Autoregressive Multi-view Diffusion for Flexible 3D Novel View Synthesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GH4HQDB5}},
  note         = {Machine review of arXiv:2509.06579}
}
read the original abstract

Multi-view diffusion models have shown promise in 3D novel view synthesis, but most existing methods adopt a non-autoregressive formulation. This limits their applicability in world modeling, as they only support a fixed number of views and suffer from slow inference due to denoising all frames simultaneously. To address these limitations, we propose CausNVS, a multi-view diffusion model in an autoregressive setting, which supports arbitrary input-output view configurations and generates views sequentially. We train CausNVS with causal masking and per-frame noise, using pairwise-relative camera pose encodings (CaPE) for precise camera control. At inference time, we combine a spatially-aware sliding-window with key-value caching and noise conditioning augmentation to mitigate drift. Our experiments demonstrate that CausNVS supports a broad range of camera trajectories, enables flexible autoregressive novel view synthesis, and achieves consistently strong visual quality across diverse settings. Project page: https://kxhit.github.io/CausNVS.html.

Figures

Figures reproduced from arXiv: 2509.06579 by the authors.

Figure 1
Figure 1. Autoregressive Novel View Generation. CausNVS generates target views autoregressively from an arbitrary number of input views. Albeit trained with sequences of 8 frames, it is capable of longer rollouts maintaining 3D consistency, and results continue to improve with more input views. Abstract Multi-view diffusion models have shown promise in 3D novel view synthesis, but most existing methods adopt a non-autoregress… view at source ↗
Figure 2
Figure 2. Causal Multi-view Diffusion Pipeline. Each view is tokenized with its camera pose and noise level, and processed with frame-wise self-attention, causal masking, and CaPE. At inference time, given a variable number of conditioning views, the model performs autoregressive denoising using KV caching with spatial attention window. target camera poses {pN+1, . . . , pN+M}, where xi is an RGB image and pi is the correspon… view at source ↗
Figure 3
Figure 3. CaPE Attention Score Analysis. We analyze how attention responds to relative pose changes by initializing queries and keys randomly, and varying rotation or translation separately. With CaPE, attention scores change periodically with rotation and linearly with translation, indicating an SE(3)-aware inductive bias. Without CaPE, attention remains invariant to pose changes. Consequently, it generalizes not only to arb… view at source ↗
Figures from the paper (15 more)
Figure 4
Figure 4. Figure 4: Novel View Synthesis Results. Visual comparison of our method and baseline models across diverse scenes and camera trajectories with input images from the DL3DV, RealEstate10K, and LLFF datasets. Each example shows the input views (N), the ground-truth target, our pred…
Figure 5
Figure 5. Figure 5: Novel View Synthesis on Diverse Customized Trajectories. CausNVS generalizes to diverse camera motions, including trajectories that return back, showcasing the spatial consistency. Feed-forward methods MVSplat [6] and DepthSplat [51] perform well when the input views a…
Figure 6
Figure 6. Figure 6: Examples of key properties of CausNVS. Through qualitative examples, we demonstrate key properties of CausNVS compared to non-causal baselines. (a) Although trained with a fixed sequence length F, causal masking enables supervision for all sequence lengths up to F. Thi…
Figure 7
Figure 7. Figure 7: Novel View Synthesis on Re10K with Customized Trajectories. Results under diverse camera motion types, including trajectories that return back, showcasing our spatial consistency. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Novel View Synthesis on DL3DV with Increasing Input Views. [Part 1] Ground Truth N1F8 N2F8 N3F8 N 4F8 [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Novel View Synthesis on DL3DV with Increasing Input Views. [Part 2] 16 [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Novel View Synthesis on DL3DV with Increasing Input Views. [Part 3] Ground Truth N1F8 N2F8 N3F8 N 4F8 [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Novel View Synthesis on DL3DV with Increasing Input Views. [Part 4] 17 [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Novel View Synthesis on Re10K Compared with Baselines. [Part 1] Input Ground Truth Ours SEVA 4DiM PNVS Ours (2 views) [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Novel View Synthesis on Re10K Compared with Baselines. [Part 2] 18 [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]
Figure 14
Figure 14. Figure 14: Novel View Synthesis on Re10K Compared with Baselines. [Part 3] Input Ground Truth Ours SEVA 4DiM PNVS Ours (2 views) [PITH_FULL_IMAGE:figures/full_fig_p019_14.png]
Figure 15
Figure 15. Figure 15: Novel View Synthesis on Re10K Compared with Baselines. [Part 4] 19 [PITH_FULL_IMAGE:figures/full_fig_p019_15.png]
Figure 16
Figure 16. Figure 16: Novel View Synthesis on Re10K Compared with Baselines. [Part 5] Input Ground Truth Ours SEVA 4DiM PNVS Ours (2 views) [PITH_FULL_IMAGE:figures/full_fig_p020_16.png]
Figure 17
Figure 17. Figure 17: Novel View Synthesis on Re10K Compared with Baselines. [Part 6] 20 [PITH_FULL_IMAGE:figures/full_fig_p020_17.png]
Figure 18
Figure 18. Figure 18: Novel View Synthesis on Re10K Compared with Baselines. [Part 7] 21 [PITH_FULL_IMAGE:figures/full_fig_p021_18.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 39 canonical work pages

  1. [1]

    Cosmos world foundation model platform for physical ai.arXiv preprint arXiv:2501.03575, 2025

    Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foundation model platform for physical ai.arXiv preprint arXiv:2501.03575, 2025

    arXiv 2025

  2. [2]

    Diffusion for world modeling: Visual details matter in atari

    Eloi Alonso, Adam Jelley, Vincent Micheli, Anssi Kanervisto, Amos Storkey, Tim Pearce, and François Fleuret. Diffusion for world modeling: Visual details matter in atari. InAdvances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  3. [3]

    Align your latents: High-resolution video synthesis with latent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    work page 2023

  4. [4]

    Generative novel view synthesis with 3d-aware diffusion models

    Eric R Chan, Koki Nagano, Matthew A Chan, Alexander W Bergman, Jeong Joon Park, Axel Levy, Miika Aittala, Shalini De Mello, Tero Karras, and Gordon Wetzstein. Generative novel view synthesis with 3d-aware diffusion models. InProceedings of the International Conference on Computer Vision (ICCV), 2023

    work page 2023

  5. [5]

    Diffusion forcing: Next-token prediction meets full-sequence diffusion.Advances in Neural Information Processing Systems (NeurIPS), 2024

    Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitzmann. Diffusion forcing: Next-token prediction meets full-sequence diffusion.Advances in Neural Information Processing Systems (NeurIPS), 2024

    work page 2024

  6. [6]

    Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images

    Yuedong Chen, Haofei Xu, Chuanxia Zheng, Bohan Zhuang, Marc Pollefeys, Andreas Geiger, Tat-Jen Cham, and Jianfei Cai. Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images. In Proceedings of the European Conference on Computer Vision (ECCV), 2024

    work page 2024

  7. [7]

    Mvsplat360: Feed-forward 360 scene synthesis from sparse views.Advances in Neural Information Processing Systems (NeurIPS), 2024

    Yuedong Chen, Chuanxia Zheng, Haofei Xu, Bohan Zhuang, Andrea Vedaldi, Tat-Jen Cham, and Jianfei Cai. Mvsplat360: Feed-forward 360 scene synthesis from sparse views.Advances in Neural Information Processing Systems (NeurIPS), 2024

    work page 2024

  8. [8]

    Futuremapping: The computational structure of spatial ai systems.arXiv preprint arXiv:1803.11288, 2018

    Andrew J Davison. Futuremapping: The computational structure of spatial ai systems.arXiv preprint arXiv:1803.11288, 2018

    arXiv 2018

Show all 58 references
  1. [9]

    Srinivasan, Jonathan T

    Ruiqi Gao*, Aleksander Holynski*, Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul P. Srinivasan, Jonathan T. Barron, and Ben Poole*. Cat3d: Create anything in 3d with multi-view diffusion models.Advances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  2. [10]

    Cameractrl: Enabling camera control for text-to-video generation.Proceedings of the International Conference on Learning Representations (ICLR), 2025

    Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. Cameractrl: Enabling camera control for text-to-video generation.Proceedings of the International Conference on Learning Representations (ICLR), 2025

    2025

  3. [11]

    Denoising diffusion probabilistic models.Advances in Neural Information Processing Systems (NeurIPS), 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in Neural Information Processing Systems (NeurIPS), 2020. 10

    2020

  4. [12]

    Cascaded diffusion models for high fidelity image generation.Journal of Machine Learning Research (JMLR), 2022

    Jonathan Ho, Chitwan Saharia, William Chan, David J Fleet, Mohammad Norouzi, and Tim Salimans. Cascaded diffusion models for high fidelity image generation.Journal of Machine Learning Research (JMLR), 2022

    2022

  5. [13]

    Long short-term memory.Neural computation, 1997

    Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory.Neural computation, 1997

    1997

  6. [14]

    Lvsm: A large view synthesis model with minimal 3d inductive bias

    Haian Jin, Hanwen Jiang, Hao Tan, Kai Zhang, Sai Bi, Tianyuan Zhang, Fujun Luan, Noah Snavely, and Zexiang Xu. Lvsm: A large view synthesis model with minimal 3d inductive bias. InProceedings of the International Conference on Learning Representations (ICLR), 2025

    2025

  7. [15]

    3d gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics (TOG), 2023

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics (TOG), 2023

    2023

  8. [16]

    Openvla: An open-source vision-language-action model.arXiv preprint arXiv:2406.09246, 2024

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language-action model.arXiv preprint arXiv:2406.09246, 2024

    2024 arXiv

  9. [17]

    Eschernet: A generative model for scalable view synthesis

    Xin Kong, Shikun Liu, Xiaoyang Lyu, Marwan Taher, Xiaojuan Qi, and Andrew J Davison. Eschernet: A generative model for scalable view synthesis. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  10. [18]

    Generating worlds.https://www.worldlabs.ai/blog, 2024

    World Labs. Generating worlds.https://www.worldlabs.ai/blog, 2024

    2024

  11. [19]

    Kv caching explained

    João Lages. Kv caching explained. https://medium.com/@joaolages/ kv-caching-explained-276520203249, 2023

    2023

  12. [20]

    Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision

    Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, et al. Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  13. [21]

    Zero- 1-to-3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl V ondrick. Zero- 1-to-3: Zero-shot one image to 3d object. InProceedings of the International Conference on Computer Vision (ICCV), 2023

    2023

  14. [22]

    3dgs-enhancer: Enhancing unbounded 3d gaussian splatting with view-consistent 2d diffusion priors.Advances in Neural Information Processing Systems (NeurIPS), 2024

    Xi Liu, Chaoyi Zhou, and Siyu Huang. 3dgs-enhancer: Enhancing unbounded 3d gaussian splatting with view-consistent 2d diffusion priors.Advances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  15. [23]

    Local light field fusion: Practical view synthesis with prescriptive sampling guidelines.ACM Transactions on Graphics (TOG), 2019

    Ben Mildenhall, Pratul P Srinivasan, Rodrigo Ortiz-Cayon, Nima Khademi Kalantari, Ravi Ramamoorthi, Ren Ng, and Abhishek Kar. Local light field fusion: Practical view synthesis with prescriptive sampling guidelines.ACM Transactions on Graphics (TOG), 2019

    2019

  16. [24]

    Nerf: Representing scenes as neural radiance fields for view synthesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. InProceedings of the European Conference on Computer Vision (ECCV), 2020

    2020

  17. [25]

    Genie 2: A large-scale foundation world model, 2024

    Jack Parker-Holder, Philip Ball, Jake Bruce, Vibhavari Dasagi, Kristian Holsheimer, Christos Kaplanis, Alexandre Moufarek, Guy Scully, Jeremy Shar, Jimmy Shi, Stephen Spencer, Jessica Yung, Michael Dennis, Sultan Kenjeyev, Shangbang Long, Vlad Mnih, Harris Chan, Maxime Gazeau,...

    2024

  18. [26]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022

    2022 arXiv

  19. [27]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022

    2022

  20. [28]

    Gaia-2: A controllable multi-view generative world model for autonomous driving.arXiv preprint arXiv:2503.20523, 2025

    Lloyd Russell, Anthony Hu, Lorenzo Bertoni, George Fedoseev, Jamie Shotton, Elahe Arani, and Gianluca Corrado. Gaia-2: A controllable multi-view generative world model for autonomous driving.arXiv preprint arXiv:2503.20523, 2025

    2025 arXiv

  21. [29]

    Magi-1: Autoregressive video generation at scale, 2025

    Sand-AI. Magi-1: Autoregressive video generation at scale, 2025

    2025

  22. [30]

    Zeronvs: Zero-shot 360-degree view synthesis from a single image

    Kyle Sargent, Zizhang Li, Tanmay Shah, Charles Herrmann, Hong-Xing Yu, Yunzhi Zhang, Eric Ryan Chan, Dmitry Lagun, Li Fei-Fei, Deqing Sun, et al. Zeronvs: Zero-shot 360-degree view synthesis from a single image. InProceedings of the IEEE Conference on Computer Vision and Patte...

    2024

  23. [31]

    Adversarial diffusion distillation

    Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. InProceedings of the European Conference on Computer Vision (ECCV), 2024

    2024

  24. [32]

    Scene representation networks: Continuous 3d-structure-aware neural scene representations.Advances in Neural Information Processing Systems (NeurIPS), 2019

    Vincent Sitzmann, Michael Zollhöfer, and Gordon Wetzstein. Scene representation networks: Continuous 3d-structure-aware neural scene representations.Advances in Neural Information Processing Systems (NeurIPS), 2019

    2019

  25. [33]

    Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020

    2010 arXiv

  26. [34]

    Consistency models.Proceedings of the International Conference on Machine Learning (ICML), 2023

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models.Proceedings of the International Conference on Machine Learning (ICML), 2023

    2023

  27. [35]

    Roformer: Enhanced transformer with rotary position embedding.arXiv preprint arXiv:2104.09864, 2021

    Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding.arXiv preprint arXiv:2104.09864, 2021

    2021 arXiv

  28. [36]

    Dimen- sionx: Create any 3d and 4d scenes from a single image with controllable video diffusion.arXiv preprint arXiv:2411.04928, 2024

    Wenqiang Sun, Shuo Chen, Fangfu Liu, Zilong Chen, Yueqi Duan, Jun Zhang, and Yikai Wang. Dimen- sionx: Create any 3d and 4d scenes from a single image with controllable video diffusion.arXiv preprint arXiv:2411.04928, 2024

    2024 arXiv

  29. [37]

    Dreamgaussian: Generative gaussian splatting for efficient 3d content creation.arXiv preprint arXiv:2309.16653, 2023

    Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for efficient 3d content creation.arXiv preprint arXiv:2309.16653, 2023

    2023 arXiv

  30. [38]

    Chameleon: Mixed-modal early-fusion foundation models.arXiv preprint arXiv:2405.09818, 2024

    Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models.arXiv preprint arXiv:2405.09818, 2024

    2024 arXiv

  31. [39]

    Consistent view synthesis with pose-guided diffusion models

    Hung-Yu Tseng, Qinbo Li, Changil Kim, Suhib Alsisan, Jia-Bin Huang, and Johannes Kopf. Consistent view synthesis with pose-guided diffusion models. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  32. [40]

    Diffusion models are real-time game engines.Proceedings of the International Conference on Learning Representations (ICLR), 2025

    Dani Valevski, Yaniv Leviathan, Moab Arar, and Shlomi Fruchter. Diffusion models are real-time game engines.Proceedings of the International Conference on Learning Representations (ICLR), 2025

    2025

  33. [41]

    3d reconstruction with spatial memory.Proceedings of the Interna- tional Conference on 3D Vision (3DV), 2024

    Hengyi Wang and Lourdes Agapito. 3d reconstruction with spatial memory.Proceedings of the Interna- tional Conference on 3D Vision (3DV), 2024

    2024

  34. [42]

    Efros, and Angjoo Kanazawa

    Qianqian Wang*, Yifei Zhang*, Aleksander Holynski, Alexei A. Efros, and Angjoo Kanazawa. Continuous 3d perception model with persistent state. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  35. [43]

    Dust3r: Geometric 3d vision made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  36. [44]

    Motionctrl: A unified and flexible motion controller for video generation

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. InProceedings of SIGGRAPH, 2024

    2024

  37. [45]

    Controlling space and time with diffusion models

    Daniel Watson, Saurabh Saxena, Lala Li, Andrea Tagliasacchi, and David J Fleet. Controlling space and time with diffusion models. InProceedings of the International Conference on Learning Representations (ICLR), 2024

    2024

  38. [46]

    A learning algorithm for continually running fully recurrent neural networks.Neural computation, 1989

    Ronald J Williams and David Zipser. A learning algorithm for continually running fully recurrent neural networks.Neural computation, 1989

    1989

  39. [47]

    Barron, and Aleksander Holynski

    Rundi Wu, Ruiqi Gao, Ben Poole, Alex Trevithick, Changxi Zheng, Jonathan T. Barron, and Aleksander Holynski. CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  40. [48]

    Worldmem: Long-term consistent world simulation with memory.arXiv preprint arXiv:2504.12369, 2025

    Zeqi Xiao, Yushi Lan, Yifan Zhou, Wenqi Ouyang, Shuai Yang, Yanhong Zeng, and Xingang Pan. Worldmem: Long-term consistent world simulation with memory.arXiv preprint arXiv:2504.12369, 2025

    2025

  41. [49]

    Show-o: One single transformer to unify multimodal understanding and generation.Proceedings of the International Conference on Learning Representations (ICLR), 2025

    Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation.Proceedings of the International Conference on ...

    2025

  42. [50]

    Em distillation for one-step diffusion models.Advances in Neural Information Processing Systems (NeurIPS), 2024

    Sirui Xie, Zhisheng Xiao, Diederik Kingma, Tingbo Hou, Ying Nian Wu, Kevin P Murphy, Tim Salimans, Ben Poole, and Ruiqi Gao. Em distillation for one-step diffusion models.Advances in Neural Information Processing Systems (NeurIPS), 2024. 12

    2024

  43. [51]

    Depthsplat: Connecting gaussian splatting and depth

    Haofei Xu, Songyou Peng, Fangjinhua Wang, Hermann Blum, Daniel Barath, Andreas Geiger, and Marc Pollefeys. Depthsplat: Connecting gaussian splatting and depth. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  44. [52]

    From slow bidirectional to fast autoregressive video diffusion models

    Tianwei Yin, Qiang Zhang, Richard Zhang, William T Freeman, Fredo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast autoregressive video diffusion models. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  45. [53]

    Yu, Tristan Aumentado-Armstrong, Fereshteh Forghani, Konstantinos G

    Jason J. Yu, Tristan Aumentado-Armstrong, Fereshteh Forghani, Konstantinos G. Derpanis, and Marcus A. Brubaker. Polyoculus: Simultaneous multi-view image-based novel view synthesis. InProceedings of the European Conference on Computer Vision (ECCV), 2024

    2024

  46. [54]

    Yu, Fereshteh Forghani, Konstantinos G

    Jason J. Yu, Fereshteh Forghani, Konstantinos G. Derpanis, and Marcus A. Brubaker. Long-term photomet- ric consistent novel view synthesis with diffusion models. InProceedings of the International Conference on Computer Vision (ICCV), 2023

    2023

  47. [55]

    Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.arXiv preprint arXiv:2409.02048, 2024

    Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.arXiv preprint arXiv:2409.02048, 2024

    2024 arXiv

  48. [56]

    Transfusion: Predict the next token and diffuse images with one multi-modal model.Proceedings of the International Conference on Learning Representations (ICLR), 2025

    Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model.Proceedings of the International Conference on Learn...

    2025

  49. [57]

    Stable virtual camera: Generative view synthesis with diffusion models.arXiv preprint arXiv:2503.14489, 2025

    Jensen (Jinghao) Zhou, Hang Gao, Vikram V oleti, Aaryaman Vasishta, Chun-Han Yao, Mark Boss, Philip Torr, Christian Rupprecht, and Varun Jampani. Stable virtual camera: Generative view synthesis with diffusion models.arXiv preprint arXiv:2503.14489, 2025

    2025 arXiv

  50. [58]

    Stereo magnification: Learning view synthesis using multiplane images.Proceedings of SIGGRAPH, 2018

    Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: Learning view synthesis using multiplane images.Proceedings of SIGGRAPH, 2018. 13 CausNVS: Autoregressive Multi-view Diffusion for Flexible 3D Novel View Synthesis (Appendix) This a...

    2018

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.