Pith. sign in

REVIEW 4 major objections 6 minor 13 cited by

Video World Models with Long-term Spatial Memory

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A persistent 3D point cloud, not more context frames, lets video world models remember what they generated.

desk verdict A genuinely new memory architecture for video world models, with large reported gains on revisit-consistency but a load-bearing TSDF-filtering premise the authors never measure. read the letter →

arxiv 2506.05284 v1 pith:BXELDHNB submitted 2025-06-05 cs.CV

Tong Wu , Shuai Yang , Ryan Po , Yinghao Xu , Ziwei Liu , Dahua Lin , Gordon Wetzstein This is my paper · ORCID
classification cs.CV
keywords videoworldmodelslong-termconsistencyspatialmemorypointcloudfusionTSDFepisodicautoregressivegenerationcamera-controlled
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that video world models forget previously generated environments because their memory is purely image-based and confined to a short context window. It proposes giving the model a persistent, geometry-grounded spatial memory: a static point cloud, fused from generated frames with TSDF-Fusion, that is rendered along the requested camera path and injected into a video diffusion transformer as conditioning. A small set of historical keyframes supplies episodic memory for visual detail, while recent frames continue to provide short-term working memory. On view-recall metrics and a user study, the scheme reports substantially higher revisit consistency than point-map-conditioned baselines.

What carries the argument

The central object is the global static point cloud, a persistently updated 3D map of the static portion of the generated world, built by truncated signed distance function (TSDF) fusion. The TSDF fusion rule is the mechanism that separates static from dynamic content: voxels with inconsistent depth observations accumulate low-confidence values and are suppressed, leaving clean static geometry to guide later generations. The point cloud is retrieved by rendering it from the target camera trajectory, and the render is injected through a zero-initialized conditioning branch of the diffusion transformer. Historical keyframes, selected when newly revealed regions exceed a threshold, are attended to by the generated frames to restore appearance detail that the sparse cloud lacks.

What would settle it

Run the autoregressive pipeline on a trajectory with a large, abrupt camera rotation and compare view-recall PSNR against a smooth trajectory over the same scene; a drop toward the baseline range would indicate that the spatial-memory filter, not the generator, is the bottleneck. A second check is to place a large moving object in front of static structure and inspect the fused point cloud for ghost geometry from the moving object.

Watch

Extended reading notes

Core claim

The central claim is that long-term spatial consistency in autoregressive video generation can be externalized into a persistent 3D representation instead of being re-derived from a sliding window of frames. At each autoregressive step the paper reconstructs newly generated frames into a metric point map, suppresses dynamic content with TSDF-Fusion, and fuses the result into a global static point cloud held in one world coordinate frame. The next batch of frames is then generated with this cloud rendered from the target camera poses as an additional conditioning signal, plus a sparse set of historical reference frames attended to through cross-attention. The paper reports a view-recall PSNR of 19.10 against 11.71 to 12.16 for the compared baselines, and top average human rankings on camera accuracy, static consistency, and dynamic plausibility.

Load-bearing premise

The load-bearing premise is that TSDF-Fusion with a uniform confidence weight reliably separates static from dynamic content, so the persistent point cloud contains mostly clean static geometry; if moving objects contaminate it or large camera motions erase it, the rendered guidance misleads the generator.

Editorial extensions

If this is right

  • Revisit consistency should persist over arbitrarily long generations, since static geometry is stored in a camera-independent form rather than in a fixed-size frame window.
  • Static and dynamic content are controlled through separate channels: the point-cloud branch pins down structure while recent frames and prompts drive motion, so camera edits need not disturb dynamics.
  • Memory cost stays bounded: a compact point cloud plus a few keyframes replaces the long frame histories that current context-window methods require for consistency.
  • The 90K-clip dataset with explicit 3D memory makes it possible to train the model to use memory, which is a prerequisite for scaling the approach to longer horizons.
  • The improved consistency is specifically spatial; drift from error accumulation over time remains an open problem the paper does not address.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct stress test would measure the completeness of the fused point cloud over rolling windows and correlate it with revisit PSNR; the paper does not report such a curve, and it would reveal whether the memory or the generator is the bottleneck.
  • The same static-map-plus-keyframes design maps naturally onto embodied-agent memory, suggesting that generative world models could be coupled with online mapping so an agent's past observations condition its simulated future.
  • Combining this spatial memory with frame-packing context compression, which the authors name as future work, would target both spatial forgetting and appearance drift at once.
  • A testable extension is to vary the episodic-memory selection threshold and measure whether detail retention trades off against context load, which would indicate an optimal recall schedule.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a video world model that augments autoregressive frame generation with three memory systems: working memory (recent context frames), spatial memory (a persistent static point cloud fused via truncated signed distance function), and episodic memory (sparse historical keyframes). The spatial memory is rendered along the camera trajectory and injected through a ControlNet-style condition branch; the model is trained on a custom 90K-sample dataset derived from MiraData with Mega-SAM reconstruction. The paper reports large improvements over TrajectoryCrafter, DiffusionAsShader (DaS), and Wan2.1-Inpainting on view-recall consistency (PSNR 19.10 vs 12.16), VBench metrics, and a user study.

Significance. If the results hold, the paper presents a practical and well-motivated architecture for improving spatial consistency in long-horizon world generation. The view-recall evaluation protocol is sensible for measuring revisit consistency, and the reported gains are large. The honest discussion of failure cases and the inclusion of a custom dataset are valuable. However, the central novelty - the spatial memory - is not isolated in the ablations, the TSDF-Fusion filtering assumption is not validated, the online coordinate-frame stability is asserted without evidence, and the context-length claim is not directly evaluated. These gaps require additional experiments before the significance claim can be fully accepted.

major comments (4)
  1. [Sec. 4.4, Table 3] The ablation study in Table 3 never removes the spatial-memory condition; all three rows include the point-cloud render branch, so the claim in Sec. 4.4 that 'each component consistently contributes' is not supported for the geometry-grounded spatial memory, which is the paper's central novel component. Please add a 'w/o spatial memory' condition (e.g., removing the point-cloud render branch or replacing the fused point cloud with a blank render) and report both VBench and view-recall metrics for that condition, so the contribution of the spatial memory can be isolated.
  2. [Sec. 3.2, Eq. (2) and Fig. 6/Limitations] The statement that TSDF-Fusion with a uniform confidence weight (w_i=1) 'inherently filters out dynamic elements' is not justified by the standard weighted-averaging update in Eq. (2), which can blend inconsistent observations into ghost surfaces; conversely, the failure case in Fig. 6 shows that TSDF-Fusion can erase valid static points under large camera motion. No point-cloud quality metric (e.g., dynamic contamination rate, static recall, or geometric accuracy) is provided for the generated videos, so the reader cannot verify that the stored spatial memory is clean enough to serve as reliable conditioning. Please add such metrics and, if possible, compare equal-weight TSDF-Fusion with an outlier-rejecting fusion scheme (e.g., confidence-weighted or median-based) to substantiate the filtering claim.
  3. [Appendix A, last paragraph] The online autoregressive fusion relies on CUT3R's saved state to keep a single world coordinate frame across steps, and the text asserts that the difference between Mega-SAM (training) and CUT3R (inference) 'does not lead to a significant performance gap' without reporting any numbers. Coordinate-frame drift would directly corrupt the fused spatial memory and undermine the long-term consistency results. Please quantify the drift (e.g., average point-map alignment error across consecutive fusion steps) and report an online-vs-offline comparison on the same generated sequences, such as view-recall PSNR with Mega-SAM fusion vs CUT3R fusion.
  4. [Sec. 4.1, Tables 1-2, and abstract] The abstract claims improved 'context length,' but the experiments measure view-recall consistency on a single revisit trajectory and VBench metrics, neither of which directly evaluates how generation quality degrades with increasing distance from the recent context window. No experiment varies the number of autoregressive steps or measures performance as a function of temporal distance, so the context-length claim is unsupported. Please add an evaluation that plots a consistency metric against the number of generated frames (or the distance of the revisited frame from the last context) for the proposed method and the baselines.
minor comments (6)
  1. [Related Work, Sec. 2] Typo: 'inifinite-length videos' should be 'infinite-length videos'.
  2. [Sec. 3.4] The phrase 'to to serve as static geometry guidance' contains a duplicated preposition; please correct.
  3. [Table 1] The baseline name is misspelled as 'TrajctoryCrafter'; it should be 'TrajectoryCrafter'.
  4. [Appendix B / References] The reference to MonST3R appears as '[86? , 69]' in Appendix B; please resolve the dangling question mark and provide the correct citation.
  5. [Sec. 4.3] The user study reports average rankings but no confidence intervals or inter-rater agreement; given only 20 subjects, please report these statistics.
  6. [Sec. 4.1] Please describe how the paired frames for the view-recall metric are aligned (e.g., whether dynamic regions are masked) and whether the same noise seed is used for the forward and reversed generation; this is important for reproducibility of the main quantitative result.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the memory system is validated by ablations and external benchmarks; no central claim reduces to a fitted input or self-citation chain.

full rationale

This is a system paper without a formal derivation chain, so the enumerated circularity patterns do not apply in a load-bearing way. The claim that spatial memory improves long-term consistency is evaluated (i) by ablations that remove episodic or working memory from the same model (Table 3), (ii) against external baselines that do not use the memory mechanism (Table 1), and (iii) by VBench metrics and a user study. The view-recall metric compares forward and reversed generations of the same model; it is a self-consistency measure rather than ground-truth accuracy, but it is not circular because the baselines are evaluated under the same protocol and the ablations isolate the memory components. The dataset and inference pipelines use external reconstructions (Mega-SaM, CUT3R) and standard TSDF-Fusion; the claim that TSDF-Fusion suppresses dynamic content is an empirical assumption, explicitly acknowledged as imperfect in Figure 6 and the Limitations paragraph, not an equation that reduces the conclusion to its input. No parameter is fitted to the test set and renamed as a prediction, and no uniqueness or ansatz is imported through a self-citation. The only author-overlapping reference, DaS [25], is used as a pretrained initialization and baseline, and it is code-reproduced external work, so it does not make the argument circular.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the reliability of external geometric perception tools (Mega-SaM, CUT3R) and on the TSDF-fusion assumption that dynamic content is filtered out. These are domain assumptions with known failure modes, as the paper itself admits in the limitation section. No invented entities or fitted physical constants are introduced.

free parameters (3)
  • context frame count k = 5
    Number of recent context frames concatenated as working memory; hand-chosen, and the paper does not sweep it.
  • TSDF confidence weight w_i = 1
    Frame-dependent confidence weight in the TSDF fusion update (Sec. 3.2, Eq. 2); set to 1, not learned or tuned.
  • episodic memory insertion threshold = not specified
    Threshold on newly revealed region area that triggers adding a historical keyframe (Sec. 3.2); the value is unmentioned in the main text, only described qualitatively.
assumptions (4)
  • domain assumption Mega-SaM provides accurate camera poses and per-frame depth maps for training data construction.
    Used in Sec. 3.4 to build the geometry-grounded dataset; if reconstruction is inaccurate, the static point clouds and view-recall evaluation are corrupted.
  • domain assumption CUT3R's online recurrent reconstruction maintains a consistent world coordinate system across autoregressive steps via saved state.
    Stated in Appendix A; if the coordinate alignment drifts, fused point clouds would be misaligned and the spatial memory would be incoherent.
  • domain assumption TSDF-Fusion with uniform weights suppresses dynamic objects and preserves static structure.
    Core to the spatial memory design (Sec. 3.2, Eq. 2); the paper's own failure case (Fig. 6) shows this assumption breaks for large camera motions, causing sparse memory.
  • domain assumption The diffusion model can learn to use black-background point-cloud renderings as geometric guidance via ControlNet-style conditioning.
    This is a learned capability, not proven; the success depends on training data and architecture (Sec. 3.3), and the paper does not analyze failure modes of the conditioning.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Video World Models with Long-term Spatial Memory." pith.science (2026). https://pith.science/paper/BXELDHNB

@misc{pith2026250605284,
  author       = {Pith},
  title        = {Pith review of: Video World Models with Long-term Spatial Memory},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BXELDHNB}},
  note         = {Machine review of arXiv:2506.05284}
}
read the original abstract

Emerging world models autoregressively generate video frames in response to actions, such as camera movements and text prompts, among other control signals. Due to limited temporal context window sizes, these models often struggle to maintain scene consistency during revisits, leading to severe forgetting of previously generated environments. Inspired by the mechanisms of human memory, we introduce a novel framework to enhancing long-term consistency of video world models through a geometry-grounded long-term spatial memory. Our framework includes mechanisms to store and retrieve information from the long-term spatial memory and we curate custom datasets to train and evaluate world models with explicitly stored 3D memory mechanisms. Our evaluations show improved quality, consistency, and context length compared to relevant baselines, paving the way towards long-term consistent world generation.

Figures

Figures reproduced from arXiv: 2506.05284 by the authors.

Figure 1
Figure 1. We augment video world models with memory. In this context, we consider the conven [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our system. A latent video generation model, implemented by a diffusion transformer (DiT), is conditioned on three different memory mechanisms when autoregressively generating new frames. First, recent context frames model a short-term working memory. Second, a point cloud representation (left) is autoregressively generated along with the video frames. This long-term spatial memory contains the static pa… view at source ↗
Figure 3
Figure 3. Dataset construction pipeline. We use Mega-SaM [44] to extract camera poses and dynamic point maps from the full video clip. For the source part, dynamic regions are erased via TSDF-Fusion, and the point cloud is rendered along the target trajectory to to serve as static geometry guidance for the target part. Qwen [77] generates annotations for actions in future target frames [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Qualitative evaluation. We compare our approach to relevant baselines in several conditions. Baselines cannot accurately generate significant camera pose changes while maintaining a consistent scene (top). When revisiting a previously seen camera pose, baselines fail t…
Figure 5
Figure 5. Figure 5: Ablation of different memory mechanisms. We evaluate several variants of our model: w/o short-term working memory: the full model without recent context frames; w/o long-term episodic memory: the full model without sparse historical keyframes; full model including shor…
Figure 6
Figure 6. Figure 6: Failure case. When the distance between consecutive camera poses is too large and the trajectory exhibits overly abrupt angles, the 4D reconstruction may fail, resulting in significant ghosting artifacts between frames. Consequently, TSDF-Fusion will filter out a large…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 13 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Addressable Memory for Video World Models

    cs.CV 2026-08 conditional novelty 6.0 of 10

    Video world models can recall revisited scenes far beyond their training horizon by storing compressed memory at fixed in-distribution positions and averaging keys in a rotation-free space.

  2. HERA: Historical Evidence Routing Adapter for Physical Prediction in Latent World Models

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A memory adapter with separate retrieval and workspace registers routes historical visual evidence into a frozen video predictor and improves physical prediction accuracy on IntPhys2.

  3. WorldCycle: Self-Verifiable Reinforcement Learning for Long-Horizon Video World Models

    cs.AI 2026-08 conditional novelty 6.0 of 10

    WorldCycle post-trains interactive video world models with reinforcement learning rewards for spatial closure and temporal consistency on reversible action cycles, reducing long-horizon drift and improving composite-a...

  4. Pano2World: End-to-End 3D Generation via Unified Multi-View Sequences

    cs.CV 2026-07 unverdicted novelty 6.0 of 10

    Pano2World generates an explorable 3D Gaussian scene directly from a single indoor panorama via coarse proxy rendering, view-aware joint denoising, and a latent feature adapter.

  5. From Synchrony to Sequence: Exo-to-Ego Generation via Interpolation

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    Interpolating only the video frames between synchronized exo and ego clips already turns discontinuous cross-view generation into continuous sequence modeling and measurably improves diffusion-based Exo2Ego synthesis.

  6. ShareVerse: Multi-Agent Consistent Video Generation for Shared World Modeling

    cs.CV 2026-03 conditional novelty 6.0 of 10

    Distributed video-generation agents can create a spatiotemporally consistent shared world by tiling four views, exchanging cross-agent attention, and querying a spatial memory cache.

  7. UCM: Unified Modeling of Camera Control and Memory with Time-aware Positional Encoding Warping for World Models

    cs.CV 2026-02 conditional novelty 6.0 of 10

    A video-generation world model that warps positional encodings of memory frames to target viewpoints achieves state-of-the-art long-term consistency and camera control.

  8. Flow Equivariant World Models: Memory for Partially Observed Dynamic Environments

    cs.LG 2026-01 conditional novelty 6.0 of 10

    Flow equivariant world models use a latent memory that shifts with the agent and with inferred object motion, giving stable long-horizon prediction under partial observability.

  9. CustomX: Unified Character, Action, and Scene Customization in Video World Models

    cs.CV 2025-12 conditional novelty 6.0 of 10

    AniX generates controllable videos of a user-supplied character performing typed actions inside a user-supplied 3D scene by fine-tuning a pre-trained video generator on small locomotion datasets.

  10. A Comprehensive Survey on World Models for Embodied AI

    cs.CV 2025-10 conditional novelty 6.0 of 10

    A unified three-axis taxonomy — functionality, temporal modeling, spatial representation — organizes the world-model literature for embodied AI.

  11. PE-Field 4D: Video Generation Models as Canvas

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Warping reference tokens' positional encodings into the target view, with depth offsets and frame-level compression fixes, improves geometry-aware camera control in video diffusion transformers.

  12. Vision-Language Memory for Spatial Reasoning

    cs.CV 2025-11 conditional novelty 5.0 of 10

    A video-based vision-language model with 3D-aligned visual features and bounded dual memory achieves state-of-the-art scores on four spatial reasoning benchmarks.

  13. HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixels

    cs.CV 2025-07 conditional novelty 5.0 of 10

    A staged pipeline generates layered, mesh-based 3D worlds from text or images by combining panoramic diffusion, semantic layer decomposition, and video-based expansion.

Reference graph

Works this paper leans on

91 extracted references · 26 canonical work pages · cited by 13 Pith papers

  1. [1]

    Diffusion for world modeling: Visual details matter in atari

    Eloi Alonso, Adam Jelley, Vincent Micheli, Anssi Kanervisto, Amos J Storkey, Tim Pearce, and François Fleuret. Diffusion for world modeling: Visual details matter in atari. Advances in Neural Information Processing Systems, 37:58757–58791, 2024

    2024

  2. [2]

    Genesis: A universal and generative physics engine for robotics and beyond

    Genesis Authors. Genesis: A universal and generative physics engine for robotics and beyond. URL https://github. com/Genesis-Embodied-AI/Genesis, 2024

    2024

  3. [3]

    Essentials of human memory

    Alan Baddeley. Essentials of human memory . Psychology Press, 2013

    2013

  4. [4]

    Lindell, and Sergey Tulyakov

    Sherwin Bahmani, Ivan Skorokhodov, Guocheng Qian, Aliaksandr Siarohin, Willi Menapace, Andrea Tagliasacchi, David B. Lindell, and Sergey Tulyakov. Ac3d: Analyzing and improving 3d camera control in video diffusion transformers. Proc. CVPR, 2025

    2025

  5. [5]

    Lindell, and Sergey Tulyakov

    Sherwin Bahmani, Ivan Skorokhodov, Aliaksandr Siarohin, Willi Menapace, Guocheng Qian, Michael Vasilkovsky, Hsin-Ying Lee, Chaoyang Wang, Jiaxu Zou, Andrea Tagliasacchi, David B. Lindell, and Sergey Tulyakov. VD3d: Taming large video diffusion transformers for 3d camera control. In The Thirteenth International Conference on Learning Representations , 2025...

    2025

  6. [6]

    Recammaster: Camera-controlled generative rendering from a single video, 2025

    Jianhong Bai, Menghan Xia, Xiao Fu, Xintao Wang, Lianrui Mu, Jinwen Cao, Zuozhu Liu, Haoji Hu, Xiang Bai, Pengfei Wan, and Di Zhang. Recammaster: Camera-controlled generative rendering from a single video, 2025. URL https://arxiv.org/abs/2503.11647

    arXiv 2025

  7. [7]

    Syncammaster: Synchronizing multi-camera video generation from diverse viewpoints

    Jianhong Bai, Menghan Xia, Xintao Wang, Ziyang Yuan, Zuozhu Liu, Haoji Hu, Pengfei Wan, and Di ZHANG. Syncammaster: Synchronizing multi-camera video generation from diverse viewpoints. In The Thirteenth International Conference on Learning Representations , 2025. URL https://openreview.net/forum?id=m8Rk3HLGFx

    2025

  8. [9]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Do- minik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023

    arXiv 2023

Show all 91 references
  1. [10]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators. 2024. URL https://openai.com/research/ video-generation-models...

    2024

  2. [11]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, et al. Video generation models as world simulators. OpenAI Blog, 1:8, 2024

    2024

  3. [12]

    Gamegen-x: Interactive open-world game video generation

    Haoxuan Che, Xuanhua He, Quande Liu, Cheng Jin, and Hao Chen. Gamegen-x: Interactive open-world game video generation. arXiv preprint arXiv:2411.00769, 2024

    2024 arXiv

  4. [13]

    Diffusion forcing: Next-token prediction meets full-sequence diffusion

    Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitzmann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. In NeurIPS, 2024

    2024

  5. [14]

    Skyreels-v2: Infinite-length film generative model

    Guibin Chen, Dixuan Lin, Jiangping Yang, Chunze Lin, Juncheng Zhu, Mingyuan Fan, Hao Zhang, Sheng Chen, Zheng Chen, Chengchen Ma, Weiming Xiong, Wei Wang, Nuo Pang, Kang Kang, Zhi-Xin Xu, Yuzhe Jin, Yupeng Liang, Yu-Ning Song, Peng Zhao, Bo Xu, Di Qiu, Debang Li, Zhengcong Fei...

    2025

  6. [15]

    Flexworld: Progressively expanding 3d scenes for flexiable-view synthesis

    Luxi Chen, Zihan Zhou, Min Zhao, Yikai Wang, Ge Zhang, Wenhao Huang, Hao Sun, Ji-Rong Wen, and Chongxuan Li. Flexworld: Progressively expanding 3d scenes for flexiable-view synthesis. arXiv preprint arXiv:2503.13265, 2025

    2025 arXiv

  7. [16]

    Seine: Short-to-long video diffusion model for generative transition and prediction

    Xinyuan Chen, Yaohui Wang, Lingjun Zhang, Shaobin Zhuang, Xin Ma, Jiashuo Yu, Yali Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. Seine: Short-to-long video diffusion model for generative transition and prediction. In ICLR, 2023

    2023

  8. [17]

    Oasis: A universe in a transformer

    Etched Decart, Q McIntyre, S Campbell, Xinlei Chen, and R Wachen. Oasis: A universe in a transformer. URL: https://oasis-model. github. io, 2024

    2024

  9. [18]

    Oasis: A universe in a transformer

    Julian Decart, Quinn Quevedo, Spruce McIntyre, Xinlei Campbell, Robert Chen, and Wachen. Oasis: A universe in a transformer. 2024. URL https://oasis-model.github.io/

    2024

  10. [19]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans on image synthesis. ArXiv, abs/2105.05233, 2021. URL https://api.semanticscholar.org/CorpusID: 234357997

    2021 arXiv

  11. [20]

    Skyreels-a2: Compose anything in video diffusion transformers

    Zhengcong Fei, Debang Li, Di Qiu, Jiahua Wang, Yikun Dou, Rui Wang, Jingtao Xu, Mingyuan Fan, Guibin Chen, Yang Li, and Yahui Zhou. Skyreels-a2: Compose anything in video diffusion transformers. 2025. URL https://api.semanticscholar.org/CorpusID:277509893

    2025

  12. [21]

    The matrix: Infinite-horizon world generation with real-time moving control

    Ruili Feng, Han Zhang, Zhantao Yang, Jie Xiao, Zhilei Shu, Zhiheng Liu, Andy Zheng, Yukun Huang, Yu Liu, and Hongyang Zhang. The matrix: Infinite-horizon world generation with real-time moving control. arXiv preprint arXiv:2412.03568, 2024

    2024 arXiv

  13. [22]

    Vid-gpt: Introducing gpt-style autoregressive generation in video diffusion models

    Kaifeng Gao, Jiaxin Shi, Hanwang Zhang, Chunping Wang, and Jun Xiao. Vid-gpt: Introducing gpt-style autoregressive generation in video diffusion models. arXiv preprint arXiv:2406.10981, 2024

    2024 arXiv

  14. [23]

    Ca2- vdm: Efficient autoregressive video diffusion model with causal generation and cache sharing

    Kaifeng Gao, Jiaxin Shi, Hanwang Zhang, Chunping Wang, Jun Xiao, and Long Chen. Ca2- vdm: Efficient autoregressive video diffusion model with causal generation and cache sharing. arXiv preprint arXiv:2411.16375, 2024

    2024 arXiv

  15. [24]

    Long-context autoregressive video mod- eling with next-frame prediction

    Yuchao Gu, Weijia Mao, and Mike Zheng Shou. Long-context autoregressive video mod- eling with next-frame prediction. ArXiv, abs/2503.19325, 2025. URL https://api. semanticscholar.org/CorpusID:277313237. 12

    2025 arXiv

  16. [25]

    Diffusion as shader: 3d-aware video diffusion for versatile video generation control

    Zekai Gu, Rui Yan, Jiahao Lu, Peng Li, Zhiyang Dou, Chenyang Si, Zhen Dong, Qifeng Liu, Cheng Lin, Ziwei Liu, Wenping Wang, and Yuan Liu. Diffusion as shader: 3d-aware video diffusion for versatile video generation control. arXiv preprint arXiv:2501.03847, 2025

    2025 arXiv

  17. [26]

    Photorealistic video generation with diffusion models

    Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Fei-Fei Li, Irfan Essa, Lu Jiang, and José Lezama. Photorealistic video generation with diffusion models. In ECCV, 2024

    2024

  18. [27]

    Recurrent world models facilitate policy evolution

    David Ha and Jürgen Schmidhuber. Recurrent world models facilitate policy evolution. In Advances in Neural Information Processing Systems 31 , pages 2451–2463. Curran Associates, Inc., 2018. URL https://papers.nips.cc/ paper/7512-recurrent-world-models-facilitate-policy-evolut...

    2018

  19. [28]

    Ltx-video: Realtime video latent diffusion

    Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richard- son, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, et al. Ltx-video: Realtime video latent diffusion. arXiv preprint arXiv:2501.00103, 2024

    2024 arXiv

  20. [29]

    Dream to control: Learning behaviors by latent imagination

    Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learning behaviors by latent imagination. arXiv preprint arXiv:1912.01603, 2019

    1912 arXiv

  21. [30]

    Cameractrl: Enabling camera control for video diffusion models

    Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. Cameractrl: Enabling camera control for video diffusion models. In The Thirteenth International Conference on Learning Representations , 2025. URL https://openreview. net/forum?id=Z4evOUYrk7

    2025

  22. [31]

    Cameractrl ii: Dynamic scene exploration via camera- controlled video diffusion models

    Hao He, Ceyuan Yang, Shanchuan Lin, Yinghao Xu, Meng Wei, Liangke Gui, Qi Zhao, Gordon Wetzstein, Lu Jiang, and Hongsheng Li. Cameractrl ii: Dynamic scene exploration via camera- controlled video diffusion models. arXiv preprint arXiv:2503.10592, 2025

    2025 arXiv

  23. [32]

    Latent video diffusion models for high-fidelity long video generation

    Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity long video generation. arXiv preprint arXiv:2211.13221, 2022

    2022 arXiv

  24. [33]

    Streamingt2v: Consistent, dynamic, and extendable long video generation from text

    Roberto Henschel, Levon Khachatryan, Daniil Hayrapetyan, Hayk Poghosyan, Vahram Tade- vosyan, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Streamingt2v: Consistent, dynamic, and extendable long video generation from text. arXiv preprint arXiv:2403.14773, 2024

    2024 arXiv

  25. [34]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS, 2020

    2020

  26. [35]

    Video diffusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. In NeurIPS, 2022

    2022

  27. [36]

    Vbench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat...

    2024

  28. [37]

    Pyramidal flow matching for efficient video generative modeling

    Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Hao Jiang, Zhuang Nan, Quzhe Huang, Yang Song, Yadong Mu, and Zhouchen Lin. Pyramidal flow matching for efficient video generative modeling. ArXiv, abs/2410.05954, 2024. URL https://api.semanticscholar. org/CorpusID:273228937

    2024

  29. [38]

    Pyramidal flow matching for efficient video generative modeling

    Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Hao Jiang, Nan Zhuang, Quzhe Huang, Yang Song, Yadong Mu, and Zhouchen Lin. Pyramidal flow matching for efficient video generative modeling. In ICLR, 2025

    2025

  30. [39]

    Miradata: A large-scale video dataset with long durations and structured captions, 2024

    Xuan Ju, Yiming Gao, Zhaoyang Zhang, Ziyang Yuan, Xintao Wang, Ailing Zeng, Yu Xiong, Qiang Xu, and Ying Shan. Miradata: A large-scale video dataset with long durations and structured captions, 2024. URL https://arxiv.org/abs/2407.06358

    2024 arXiv

  31. [40]

    Videopoet: A large language model for zero-shot video generation

    Dan Kondratyuk, Lijun Yu, Xiuye Gu, Jose Lezama, Jonathan Huang, Grant Schindler, Rachel Hornung, Vighnesh Birodkar, Jimmy Yan, Ming-Chang Chiu, et al. Videopoet: A large language model for zero-shot video generation. In ICML, 2024. 13

    2024

  32. [41]

    Hunyuanvideo: A systematic framework for large video generative models

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024

    2024 arXiv

  33. [42]

    Lee, Richard Zhang, Frederik Ebert, P

    Alex X. Lee, Richard Zhang, Frederik Ebert, P. Abbeel, Chelsea Finn, and Sergey Levine. Stochastic adversarial video prediction. ArXiv, abs/1804.01523, 2018. URL https://api. semanticscholar.org/CorpusID:4591836

    2018 arXiv

  34. [43]

    Efficient spatially sparse inference for conditional gans and diffusion models

    Muyang Li, Ji Lin, Chenlin Meng, Stefano Ermon, Song Han, and Jun-Yan Zhu. Efficient spatially sparse inference for conditional gans and diffusion models. Advances in neural information processing systems, 35:28858–28873, 2022

    2022

  35. [44]

    Megasam: Accurate, fast and robust structure and motion from casual dynamic videos

    Zhengqi Li, Richard Tucker, Forrester Cole, Qianqian Wang, Linyi Jin, Vickie Ye, Angjoo Kanazawa, Aleksander Holynski, and Noah Snavely. Megasam: Accurate, fast and robust structure and motion from casual dynamic videos. In Proceedings of the IEEE/CVF Conference on Computer Vi...

    2025

  36. [45]

    Open-sora plan: Open-source large video generation model

    Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, Tanghui Jia, Junwu Zhang, Zhenyu Tang, Yatian Pang, Bin She, Cen Yan, Zhiheng Hu, Xiao wen Dong, Lin Chen, Zhang Pan, Xing Zhou, Shaoling Dong, Yonghong Tian...

    2024 arXiv

  37. [46]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. ArXiv, abs/2210.02747, 2022. URL https://api. semanticscholar.org/CorpusID:252734897

    2022 arXiv

  38. [47]

    Linfusion: 1 gpu, 1 minute, 16k image

    Songhua Liu, Weihao Yu, Zhenxiong Tan, and Xinchao Wang. Linfusion: 1 gpu, 1 minute, 16k image. ArXiv, abs/2409.02097, 2024. URL https://api.semanticscholar.org/ CorpusID:272366893

    2024 arXiv

  39. [48]

    Deep multi-scale video prediction beyond mean square error

    Michaël Mathieu, Camille Couprie, and Yann LeCun. Deep multi-scale video prediction beyond mean square error. CoRR, abs/1511.05440, 2015. URL https://api.semanticscholar. org/CorpusID:205514

    2015 arXiv

  40. [49]

    Ssm meets video diffusion models: Efficient long-term video generation with structured state spaces

    Yuta Oshima, Shohei Taniguchi, Masahiro Suzuki, and Yutaka Matsuo. Ssm meets video diffusion models: Efficient long-term video generation with structured state spaces. arXiv preprint arXiv:2403.07711, 2024

    2024 arXiv

  41. [50]

    Genie 2: A large-scale foundation world model

    J Parker-Holder, P Ball, J Bruce, V Dasagi, K Holsheimer, C Kaplanis, A Moufarek, G Scully, J Shar, J Shi, et al. Genie 2: A large-scale foundation world model. URL: https://deepmind. google/discover/blog/genie-2-a-large-scale-foundation-world-model, 2024

    2024

  42. [51]

    Genie 2: A large-scale foundation world model

    Jack Parker-Holder, Philip Ball, Jake Bruce, Vibhavari Dasagi, Kristian Holsheimer, Christos Kaplanis, Alexandre Moufarek, Guy Scully, Jeremy Shar, Jimmy Shi, Stephen Spencer, Jessica Yung, Michael Dennis, Sultan Kenjeyev, Shangbang Long, Vlad Mnih, Harris Chan, Maxime Gazeau,...

    2024

  43. [52]

    State of the art on diffusion models for visual computing

    Ryan Po, Wang Yifan, Vladislav Golyanik, Kfir Aberman, Jonathan T Barron, Amit Bermano, Eric Chan, Tali Dekel, Aleksander Holynski, Angjoo Kanazawa, et al. State of the art on diffusion models for visual computing. Computer Graphics F orum, 43(2):e15063, 2024

    2024

  44. [53]

    Skyreels-a1: Expressive portrait animation in video diffusion trans- formers

    Di Qiu, Zhengcong Fei, Rui Wang, Jialin Bai, Changqian Yu, Mingyuan Fan, Guibin Chen, and Xiang Wen. Skyreels-a1: Expressive portrait animation in video diffusion trans- formers. abs/2502.10841, 2025. URL https://api.semanticscholar.org/CorpusID: 276409428. 14

    2025 arXiv

  45. [54]

    Gen3c: 3d-informed world- consistent video generation with precise camera control

    Xuanchi Ren, Tianchang Shen, Jiahui Huang, Huan Ling, Yifan Lu, Merlin Nimier-David, Thomas Müller, Alexander Keller, Sanja Fidler, and Jun Gao. Gen3c: 3d-informed world- consistent video generation with precise camera control. In Proceedings of the IEEE/CVF Conference on Comp...

    2025

  46. [55]

    Structure-from-motion revisited

    Johannes Lutz Schönberger and Jan-Michael Frahm. Structure-from-motion revisited. In Conference on Computer Vision and Pattern Recognition (CVPR) , 2016

    2016

  47. [56]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. In ICLR, 2023

    2023

  48. [57]

    Deep unsuper- vised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. 2015

    2015

  49. [58]

    History-guided video diffusion

    Kiwhan Song, Boyuan Chen, Max Simchowitz, Yilun Du, Russ Tedrake, and Vincent Sitzmann. History-guided video diffusion. arXiv preprint arXiv:2502.06764, 2025

    2025 arXiv

  50. [59]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In ICLR, 2021

    2021

  51. [60]

    Autoregressive model beats diffusion: Llama for scalable image generation

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525, 2024

    2024 arXiv

  52. [61]

    Vidtok: A versatile and open-source video tokenizer

    Anni Tang, Tianyu He, Junliang Guo, Xinle Cheng, Li Song, and Jiang Bian. Vidtok: A versatile and open-source video tokenizer. ArXiv, abs/2412.13061, 2024. URL https://api. semanticscholar.org/CorpusID:274788955

    2024 arXiv

  53. [62]

    Tulyakov, Ming-Yu Liu, Xiaodong Yang, and Jan Kautz

    S. Tulyakov, Ming-Yu Liu, Xiaodong Yang, and Jan Kautz. Mocogan: Decomposing motion and content for video generation. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1526–1535, 2017. URL https://api.semanticscholar.org/ CorpusID:4475365

    2018

  54. [63]

    Diffusion models are real-time game engines

    Dani Valevski, Yaniv Leviathan, Moab Arar, and Shlomi Fruchter. Diffusion models are real-time game engines. arXiv preprint arXiv:2408.14837, 2024

    2024 arXiv

  55. [64]

    Phenaki: Variable length video generation from open domain textual descriptions

    R Villegas, H Moraldo, S Castro, M Babaeizadeh, H Zhang, J Kunze, PJ Kindermans, MT Saffar, and D Erhan. Phenaki: Variable length video generation from open domain textual descriptions. In ICLR, 2023

    2023

  56. [65]

    Generating the future with adversarial transformers.2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 2992–3000,

    Carl V ondrick and Antonio Torralba. Generating the future with adversarial transformers.2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 2992–3000,

    2017

  57. [66]

    Wan: Open and advanced large-scale video generative models

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025

    2025 arXiv

  58. [67]

    Lingen: Towards high-resolution minute-length text-to-video generation with linear compu- tational complexity

    Hongjie Wang, Chih-Yao Ma, Yen-Cheng Liu, Ji Hou, Tao Xu, Jialiang Wang, Felix Juefei-Xu, Yaqiao Luo, Peizhao Zhang, Tingbo Hou, Peter Vajda, Niraj Kumar Jha, and Xiaoliang Dai. Lingen: Towards high-resolution minute-length text-to-video generation with linear compu- tational ...

    2024 arXiv

  59. [68]

    Vggt: Visual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025

    2025

  60. [69]

    Efros, and Angjoo Kanazawa

    Qianqian Wang*, Yifei Zhang*, Aleksander Holynski, Alexei A. Efros, and Angjoo Kanazawa. Continuous 3d perception model with persistent state. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2025. 15

    2025

  61. [70]

    Dust3r: Geometric 3d vision made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20697–20709, June 2024

    2024

  62. [71]

    Loong: Generating minute-level long videos with autoregressive language models

    Yuqing Wang, Tianwei Xiong, Daquan Zhou, Zhijie Lin, Yang Zhao, Bingyi Kang, Jiashi Feng, and Xihui Liu. Loong: Generating minute-level long videos with autoregressive language models. arXiv preprint arXiv:2410.02757, 2024

    2024 arXiv

  63. [72]

    Motionctrl: A unified and flexible motion controller for video generation

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. In ACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024

    2024

  64. [73]

    Art-v: Auto-regressive text-to-video generation with diffusion models

    Wenming Weng, Ruoyu Feng, Yanhui Wang, Qi Dai, Chunyu Wang, Dacheng Yin, Zhiyuan Zhao, Kai Qiu, Jianmin Bao, Yuhui Yuan, et al. Art-v: Auto-regressive text-to-video generation with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog...

    2024

  65. [74]

    Day- dreamer: World models for physical robot learning

    Philipp Wu, Alejandro Escontrela, Danijar Hafner, Pieter Abbeel, and Ken Goldberg. Day- dreamer: World models for physical robot learning. In Conference on robot learning, pages 2226–2240. PMLR, 2023

    2023

  66. [75]

    Sana: Efficient high-resolution image synthesis with linear diffusion transformers

    Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, and Song Han. Sana: Efficient high-resolution image synthesis with linear diffusion transformers. ArXiv, abs/2410.10629, 2024. URL https: //api.semanticscholar.or...

    arXiv

  67. [76]

    Videogpt: Video generation using vq-vae and transformers

    Wilson Yan, Yunzhi Zhang, Pieter Abbeel, and Aravind Srinivas. Videogpt: Video generation using vq-vae and transformers. arXiv preprint arXiv:2104.10157, 2021

    2021 arXiv

  68. [77]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  69. [79]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024

    2024 arXiv

  70. [80]

    From slow bidirectional to fast autoregressive video diffusion models

    Tianwei Yin, Qiang Zhang, Richard Zhang, William T Freeman, Fredo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast autoregressive video diffusion models. In CVPR, 2025

    2025

  71. [81]

    Gamefactory: Creating new games with generative interactive videos

    Jiwen Yu, Yiran Qin, Xintao Wang, Pengfei Wan, Di Zhang, and Xihui Liu. Gamefactory: Creating new games with generative interactive videos. arXiv preprint arXiv:2501.08325, 2025

    2025

  72. [82]

    Trajectorycrafter: Redirecting camera trajectory for monocular videos via diffusion models

    Mark YU, Wenbo Hu, Jinbo Xing, and Ying Shan. Trajectorycrafter: Redirecting camera trajectory for monocular videos via diffusion models. arXiv preprint arXiv:2503.05638, 2025

    2025 arXiv

  73. [83]

    Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis

    Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis. arXiv preprint arXiv:2409.02048, 2024

    2024 arXiv

  74. [84]

    3dmatch: Learning local geometric descriptors from rgb-d reconstructions

    Andy Zeng, Shuran Song, Matthias Nießner, Matthew Fisher, Jianxiong Xiao, and Thomas Funkhouser. 3dmatch: Learning local geometric descriptors from rgb-d reconstructions. In CVPR, 2017

    2017

  75. [85]

    Recapture: Generative video camera controls for user-provided videos using masked video fine-tuning

    David Junhao Zhang, Roni Paiss, Shiran Zada, Nikhil Karnad, David E Jacobs, Yael Pritch, Inbar Mosseri, Mike Zheng Shou, Neal Wadhwa, and Nataniel Ruiz. Recapture: Generative video camera controls for user-provided videos using masked video fine-tuning. arXiv preprint arXiv:24...

    2024 arXiv

  76. [86]

    MonST3r: A simple approach for estimating geometry in the presence of motion

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jampani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming-Hsuan Yang. MonST3r: A simple approach for estimating geometry in the presence of motion. In The Thirteenth International Conference on Learning Representa- tions, 20...

    2025

  77. [87]

    Packing input frame context in next-frame prediction models for video generation

    Lvmin Zhang and Maneesh Agrawala. Packing input frame context in next-frame prediction models for video generation. arXiv preprint arXiv:2504.12626, 2025

    2025

  78. [88]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision , pages 3836–3847, 2023

    2023

  79. [89]

    Flare: Feed-forward geometry, appearance and camera estimation from uncalibrated sparse views

    Shangzhan Zhang, Jianyuan Wang, Yinghao Xu, Nan Xue, Christian Rupprecht, Xiaowei Zhou, Yujun Shen, and Gordon Wetzstein. Flare: Feed-forward geometry, appearance and camera estimation from uncalibrated sparse views. In Proceedings of the IEEE/CVF Conference on Computer Vision...

    2025

  80. [90]

    Extdm: Distribu- tion extrapolation diffusion model for video prediction

    Zhicheng Zhang, Junyao Hu, Wentao Cheng, Danda Paudel, and Jufeng Yang. Extdm: Distribu- tion extrapolation diffusion model for video prediction. In CVPR, 2024

    2024

  81. [91]

    Open-sora: Democratizing efficient video production for all

    Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. ArXiv, abs/2412.20404, 2024. URL https://api.semanticscholar.org/CorpusID: 275133398

    2024 arXiv

  82. [92]

    Is sora a world simulator? a comprehensive survey on general world models and beyond

    Zheng Zhu, Xiaofeng Wang, Wangbo Zhao, Chen Min, Nianchen Deng, Min Dou, Yuqi Wang, Botian Shi, Kai Wang, Chi Zhang, et al. Is sora a world simulator? a comprehensive survey on general world models and beyond. arXiv preprint arXiv:2405.03520, 2024. 17

    2024

  83. [2017]

    URL https://api.semanticscholar.org/CorpusID:8234308

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.