Pith. sign in

REVIEW 4 major objections 4 minor 37 references

VideoMAR: Autoregressive Video Generatio with Continuous Tokens

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a 1.4B-parameter decoder-only autoregressive video model with continuous tokens outperforms a 13B discrete-token baseline on VBench-I2V while using 0.5% of the training data and 0.2% of the GPU budget.

desk verdict A plausible continuous-token AR video model with honest internal ablations, but the claimed SOTA over Cosmos is likely an artifact of the progressive temperature schedule and the comparison is not controlled. read the letter →

arxiv 2506.14168 v2 pith:JWDVE4LZ submitted 2025-06-17 cs.CV cs.AI

Hu Yu , Biao Gong , Hangjie Yuan , DanDan Zheng , Weilong Chai , Jingdong Chen , Kecheng Zheng , Feng Zhao This is my paper · ORCID
classification cs.CVcs.AI
keywords autoregressivevideogenerationcontinuoustokensmaskedmodelsimage-to-videonext-framediffusionloss3DrotarypositionembeddingtemporalextrapolationVBench-I2V
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

VideoMAR is a decoder-only autoregressive image-to-video model that generates continuous tokens frame by frame while using masked prediction inside each frame. The paper's central claim is that temporal causality (each frame attends only to earlier frames and to visible tokens in the current frame) combined with spatial bi-directionality is the right first principle for video autoregressive models, and that this design can outperform the previous state-of-the-art autoregressive image-to-video baseline on the VBench-I2V benchmark with far fewer parameters, far less training data, and far less GPU compute. The authors attribute the result to a next-frame diffusion loss that trains masked tokens without breaking causality, short-to-long temporal curriculum learning, progressive-resolution training, and a progressive temperature schedule that suppresses error accumulation in late frames. They also report training-free spatial and temporal extrapolation through 3D rotary position embeddings, yielding longer and higher-resolution videos than the training grid.

What carries the argument

The central object is the frame-wise causal attention mask paired with the next-frame diffusion loss. The mask lets a token in frame t attend to all tokens in earlier frames and to the visible tokens in frame t, while leaving attention bidirectional within the frame, so temporal generation stays autoregressive and spatial generation stays parallel. The next-frame diffusion loss randomly masks part of a chosen frame, masks all later frames, leaves earlier frames complete, and applies the diffusion loss only to the chosen frame's masked tokens, which removes the training-inference gap that earlier mask-based video autoregressive methods face. Supporting mechanisms include a pretrained continuous video tokenizer, 3D rotary position embeddings for extrapolation, temporal short-to-long curriculum learning, a two-stage progressive-resolution schedule, and a progressive temperature schedule that lowers temperature for later frames.

What would settle it

Retrain VideoMAR from scratch with a video tokenizer trained only on the same 0.5M video-text pairs, with no external pretraining, and measure the VBench-I2V score; if the score falls far below 84.82 or the data and compute requirements jump, then the pretrained tokenizer rather than the autoregressive paradigm is carrying much of the reported result.

Watch

Extended reading notes

Core claim

VideoMAR establishes, on the paper's own terms, that masked autoregressive generation in continuous space is a viable and resource-efficient paradigm for image-to-video generation when applied temporally frame-by-frame and spatially within each frame. Concretely, a 1.4B-parameter decoder-only transformer trained on 0.5M video-text pairs with 64 GPUs attains a VBench-I2V total score of 84.82 in the second training stage, surpassing the 84.22 of the 13B discrete-token baseline and the 84.16 of the 5B baseline, both trained on 100M pairs. The design keeps complete previous frames as context, applies a frame-wise causal attention mask, and optimizes a token-wise diffusion loss only on the masked tokens of a randomly selected frame, with all later frames fully masked during that training step. The paper further claims that the same checkpoint generates arbitrary combinations of longer duration and higher resolution through 3D-RoPE, and that spatial masked parallel generation combined with temporal KV cache reduces inference time by more than 10 times relative to raster-scan next-token prediction.

Load-bearing premise

The efficiency claim presupposes that the pretrained continuous video tokenizer, which was trained on large-scale external video data, does not count as part of VideoMAR's training cost; if it is counted, the comparison with the baseline is no longer apples-to-apples.

Editorial extensions

If this is right

  • If the central claim is correct, discrete tokenization is not required for high-quality autoregressive video; continuous tokens with a masked diffusion loss can match or beat much larger discrete-token autoregressive models.
  • If the reported efficiency holds, video autoregressive training becomes feasible with small data and compute budgets, lowering the entry cost for this paradigm.
  • If the 3D-RoPE extrapolation result holds, a single fixed-resolution checkpoint can generate videos at arbitrary aspect ratios and durations without chunk-wise splitting.
  • If the progressive temperature strategy works as reported, late-frame collapse from exposure bias can be suppressed by a simple inference-time schedule that lowers temperature for later frames.
  • If the frame-wise causal mask closes the training-inference gap, mask-based autoregressive video models can retain both frame-by-frame generation and spatial parallel decoding with KV cache acceleration.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The headline efficiency comparison implicitly counts the pretrained continuous video tokenizer as free; if that tokenizer's external pretraining data and compute were included, VideoMAR's resource advantage over the baseline would shrink, and the comparison would be less apples-to-apples.
  • The same architecture is a natural candidate for unifying text-to-image, text-to-video, image-to-video, video-to-video, and video editing in one decoder-only network, which the paper lists as future work.
  • A direct testable extension is to train VideoMAR with a tokenizer trained only on the same 0.5M video-text pairs; if performance collapses, the tokenizer's pretrained visual prior is the hidden large input rather than the autoregressive design.
  • Because constant low temperature improves quality but lowers the dynamic-degree score, an adaptive per-frame temperature chosen by a lightweight quality or motion predictor could outperform the fixed exponential schedule the paper adopts.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes VideoMAR, a decoder-only masked autoregressive image-to-video model operating on continuous tokens from a pretrained video VAE. The method combines frame-wise causal attention with intra-frame bidirectional masked generation, a next-frame diffusion loss, a short-to-long temporal curriculum, two-stage progressive-resolution training, and a progressive temperature schedule at inference. The central empirical claim is that VideoMAR surpasses the previous autoregressive state of the art, Cosmos I2V, on the VBench-I2V benchmark while using 1.4B parameters, 0.5M training pairs, and 64 H20 GPUs, compared to Cosmos's 5B/13B parameters, 100M pairs, and much larger compute. Additional claims include substantial inference speedup through KV cache and spatial parallel generation, and training-free spatial and temporal extrapolation via 3D-RoPE.

Significance. If the headline comparison were properly controlled, the result would be significant: it would show that a compact continuous-token masked autoregressive model can match or beat a much larger discrete-token autoregressive model on a standard benchmark, with far fewer resources, and can extrapolate to unseen resolutions and durations. The framework is clean, the ablations in Table 4 are internally consistent, and the reported inference-time speedups (e.g., 1941s to 134s in Table 2) are concrete and credible. The qualitative extrapolation examples in Figure 4 and Appendix D are useful demonstrations of a desirable capability. However, the paper's central quantitative claim is not yet robustly supported because the comparison with Cosmos is confounded by an inference-time temperature schedule, the resource accounting excludes the pretrained tokenizer, and the reported VBench scores lack error bars. These issues make the headline margin of 0.60 points fragile and potentially an artifact of the evaluation procedure.

major comments (4)
  1. [Section 5.2 / Table 3 / Appendix B (Table 5)] The headline comparison with Cosmos is confounded by the progressive temperature schedule. Table 5 shows that lowering the temperature from 1.00 to 0.90 increases the Total Score from 82.19 to 84.82 while reducing Dynamic Degree from 24.80 to 9.87. VideoMAR-stage2's reported 84.82 in Table 3 is obtained under the progressive schedule that yields Dynamic Degree 10.98, less than half of Cosmos-13B's 18.70. The appendix's claim that the dynamic-degree drop is 'mainly attributed to failure cases' is contradicted by the monotonic decline in that dimension across the sweep. Because Cosmos is evaluated with its own recommended parameters rather than under the same temperature schedule, the reported 0.60-point advantage does not isolate the model's temporal modeling ability. The authors should report VideoMAR at temperature 1.0, or re-run Cosmos under the progressive schedule, or otherwise provide a controlled comparison that separates the model from the inference-time hyperparameter.
  2. [Section 5.1 / Abstract and Section 5.2] The resource-efficiency claim is incomplete because the visual tokenizer, Cosmos-Tokenizer, is a pretrained component developed by the Cosmos team. The statement 'All the weights are trained from scratch' in Section 5.1 can only refer to the VideoMAR backbone; the tokenizer's pretraining data and compute are external to the stated 0.5M video-text pairs and 64 H20 GPUs. Consequently, the comparison '1.4B parameters, 0.5M data, 64 GPUs vs. 5B/13B parameters, 100M data, 10000 GPUs' is not apples-to-apples unless the tokenizer is explicitly excluded or its cost is also counted for both sides. The authors should clarify this boundary or restate the efficiency claims accordingly.
  3. [Abstract / Section 5.2 / Table 3] There is an internal inconsistency in the reported headline number for VideoMAR-stage2: the abstract and Section 5.2 state 84.51, while Table 3 lists 84.82. Since the entire paper's central claim hinges on this number, the discrepancy must be resolved and the correct value used consistently throughout.
  4. [Section 5.1 / Section 5.2] The evaluation lacks any measure of variance. The paper states that 5 samples per prompt are generated for VBench-I2V, but no error bars, confidence intervals, or per-seed deviations are reported. Given that the claimed advantage over Cosmos-13B is 0.60 points (84.82 vs. 84.22), the result may be within sampling noise. The authors should provide standard deviations, multiple evaluation seeds, or a significance test, or at least discuss the expected variability of VBench scores.
minor comments (4)
  1. [Title] The title contains a typo: 'Generatio' should be 'Generation'.
  2. [Table 3] Several column headers ('Moti.', 'I2V Subj.', 'Back. Cons.', etc.) are abbreviated without being expanded in the caption; please define all abbreviations for readability.
  3. [Section 5.4] The text says the baseline 'Total mask' uses 'w/ Temperature strategy', but the first row of Table 4 shows Temperature as ✗. This is confusing; please correct either the text or the table so the baseline configuration matches the table.
  4. [Section 4.3 and Appendix B] The progressive temperature schedule is described in Section 4.3 as varying 'from 1 to 0.9 across frames', while Appendix B specifies the exponential form 0.9 + 10^-(t+1). These formulations are not obviously equivalent; please state the exact schedule used in the main experiments.

Circularity Check

1 steps flagged · score 6.0 of 10

The headline VBench SOTA margin is produced by a benchmark-tuned temperature schedule, not by an independent evaluation protocol.

  1. fitted input called prediction [Appendix B / Table 5; Section 5.2 / Table 3; Section 4.3]
    "0.90 84.82 97.92 98.39 97.17 97.27 99.58 9.87 55.95 62.48 / Ours 84.82 97.85 98.38 97.13 97.20 99.57 10.98 55.81 62.34 ... The data of Cosmos is tested with its official code and recommended parameters."

    The progressive temperature schedule is selected by sweeping VBench-I2V Total Score: Table 5 shows a monotonic increase as temperature falls from 1.00 to 0.90, and the reported stage-2 Total Score (84.82) equals the sweep's maximum (also achieved by constant 0.90). Cosmos is evaluated with its official parameters rather than with the same low-temperature/progressive protocol, so the headline 0.60-point superiority over Cosmos-13B (84.82 vs. 84.22) is not an independent prediction but the selected operating point of a benchmark-tuned inference hyperparameter. At temperature 1.00 the same model scores only 82.19, below Cosmos-13B, and Dynamic Degree falls from 24.80 to 9.87, meaning much of the aggregate gain comes from making videos more static.

full rationale

The generative method itself is largely self-contained: Equation 4.1 defines the frame-wise causal/masked factorization, the next-frame diffusion loss is a training objective derived from that factorization, and the 3D-RoPE extrapolation claim is supported by an external positional-encoding result plus the paper's own Appendix A comparison. The only self-citation, FAR [33] for the text-encoder choice, is non-load-bearing and does not carry the central claim. The material circularity is confined to the benchmark claim: the progressive temperature strategy is a VBench-swept inference hyperparameter (Table 5), the final Table-3 number equals the sweep optimum, and the Cosmos baseline is not run under the same protocol. This makes the headline 'prediction' of SOTA partially constructed by the evaluation sweep rather than independently tested. The hidden pretraining cost of Cosmos-Tokenizer is a fairness caveat about the reported resource efficiency, but it is not a circular derivation and is not scored as one.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

No new physical entities are introduced. The theoretical load is small: the method relies on a pretrained visual tokenizer, a masked autoregressive factorization, and an assumption that the training objective aligns with the frame-by-frame inference procedure. The free parameters are mostly standard training and inference hyperparameters, with the progressive temperature schedule being the one most directly tuned to the benchmark.

free parameters (5)
  • Progressive temperature schedule = 0.9 + 10^{-(t+1)} (t is frame index)
    Chosen after ablations on VBench (Supp. Table 5); lower temperature for later frames reduces accumulation error but also lowers Dynamic Degree.
  • Training masking ratio = 0.7 to 1.0
    Inherited from MAR; no video-specific search is reported.
  • Inference AR steps and diffusion steps = 64 AR steps, 100 diffusion steps
    Set by design as a quality-speed trade-off; no sweep is reported.
  • Classifier-free guidance = 3.0
    Used for all evaluations; no guidance sweep is reported.
  • Curriculum frame lengths and resolutions = Frames (5, 13, 25); 256x256 then 480x768
    Chosen by the authors to reduce long-sequence training difficulty and cost.
assumptions (5)
  • domain assumption The visual information lost by continuous-token compression with Cosmos-Tokenizer is acceptable for high-quality video generation.
    Relied on throughout the paper; the tokenizer is pretrained outside this work and is not evaluated for information loss in the video setting.
  • standard math The factorization p(S_t^m | C, S_1..S_{t-1}, S_t^v) in Equation 4.1 is a valid model of masked video generation and matches the training objective.
    This is a conditional probability factorization combined with a masked-modeling assumption, standard for autoregressive and masked generative models.
  • ad hoc to paper Masking all frames after frame t during training does not harm learning of long-range dependencies.
    Central to the next-frame diffusion loss; the paper compares against a total-mask baseline but does not prove that future-frame masking is optimal.
  • domain assumption VBench-I2V scores computed from 5 samples per prompt are a reliable basis for model comparison.
    No error bars or significance tests are reported, yet the headline comparison uses a small score margin.
  • domain assumption 3D-RoPE extrapolates to resolutions and frame counts not seen in training.
    Supported only by qualitative still frames; no quantitative evaluation of extrapolation quality is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VideoMAR: Autoregressive Video Generatio with Continuous Tokens." pith.science (2026). https://pith.science/paper/JWDVE4LZ

@misc{pith2026250614168,
  author       = {Pith},
  title        = {Pith review of: VideoMAR: Autoregressive Video Generatio with Continuous Tokens},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JWDVE4LZ}},
  note         = {Machine review of arXiv:2506.14168}
}
abstract

Masked-based autoregressive models have demonstrated promising image generation capability in continuous space. However, their potential for video generation remains under-explored. In this paper, we propose \textbf{VideoMAR}, a concise and efficient decoder-only autoregressive image-to-video model with continuous tokens, composing temporal frame-by-frame and spatial masked generation. We first identify temporal causality and spatial bi-directionality as the first principle of video AR models, and propose the next-frame diffusion loss for the integration of mask and video generation. Besides, the huge cost and difficulty of long sequence autoregressive modeling is a basic but crucial issue. To this end, we propose the temporal short-to-long curriculum learning and spatial progressive resolution training, and employ progressive temperature strategy at inference time to mitigate the accumulation error. Furthermore, VideoMAR replicates several unique capacities of language models to video generation. It inherently bears high efficiency due to simultaneous temporal-wise KV cache and spatial-wise parallel generation, and presents the capacity of spatial and temporal extrapolation via 3D rotary embeddings. On the VBench-I2V benchmark, VideoMAR surpasses the previous state-of-the-art (Cosmos I2V) while requiring significantly fewer parameters ($9.3\%$), training data ($0.5\%$), and GPU resources ($0.2\%$).

Figures

Figures reproduced from arXiv: 2506.14168 by the authors.

Figure 1
Figure 1. Autoregressive image-to-video samples of VideoMAR with continuous tokens. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Framework of VideoMAR. (a): Training flowchart of VideoMAR. We employ frame-wise causal attention mask for temporal causality. Besides, we introduce the next-frame diffusion loss to the spatial partial-masked frame, which has complete previous frames. (b): Efficient training of VideoMAR. We apply temporal short-to-long curriculum learning and spatial progressive-resolution for reducing the training difficulty and co… view at source ↗
Figure 3
Figure 3. Visual comparison between Cosmos 5B and VideoMAR on image-to-video generation. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (19 more)
Figure 4
Figure 4. Figure 4: This is achieved in a training-free manner without chunk-wise split. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 4
Figure 4. Figure 4: Spatial and temporal extrapolation capacity of VideoMAR. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Visual results of VideoMAR on video-to-video generation, with two frames condition. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Visual comparison between the total mask baseline and our temporal autoregressive [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: The effects of the temperature strategy on visual results. The late frames are prone to suffer [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Visual comparison between absolute and relative position encoding methods. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Arbitrary resolution scaling video samples with [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: Arbitrary resolution scaling video samples with [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: Arbitrary resolution scaling video samples with [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Arbitrary resolution scaling video samples with [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Additional visual comparison between Cosmos and our method on image-to-video [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: Additional visual comparison between Cosmos and our method on image-to-video [PITH_FULL_IMAGE:figures/full_fig_p019_14.png]
Figure 15
Figure 15. Figure 15: Additional visual results of our method on image-to-video generation. [PITH_FULL_IMAGE:figures/full_fig_p020_15.png]
Figure 16
Figure 16. Figure 16: Additional visual results of our method on image-to-video generation. [PITH_FULL_IMAGE:figures/full_fig_p020_16.png]
Figure 17
Figure 17. Figure 17: Additional visual results of our method on image-to-video generation. [PITH_FULL_IMAGE:figures/full_fig_p021_17.png]
Figure 18
Figure 18. Figure 18: Additional visual results of our method on video-to-video generation. [PITH_FULL_IMAGE:figures/full_fig_p021_18.png]
Figure 19
Figure 19. Figure 19: Additional visual results of our method on video-to-video generation. [PITH_FULL_IMAGE:figures/full_fig_p022_19.png]
Figure 20
Figure 20. Figure 20: Additional visual results of our method on video-to-video generation. [PITH_FULL_IMAGE:figures/full_fig_p022_20.png]
Figure 21
Figure 21. Figure 21: Finetuning results on the two additional collected motion types (blink and blossom) of our [PITH_FULL_IMAGE:figures/full_fig_p023_21.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 4 canonical work pages

  1. [1]

    Cosmos world foundation model platform for physical ai.arXiv preprint arXiv:2501.03575, 2025

    Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foundation model platform for physical ai.arXiv preprint arXiv:2501.03575, 2025

    arXiv 2025

  2. [2]

    Magi-1: Autoregressive video generation at scale

    Sand AI. Magi-1: Autoregressive video generation at scale. 2025

    work page 2025

  3. [3]

    Genie: Generative interactive environments

    Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, et al. Genie: Generative interactive environments. InForty-first International Conference on Machine Learning, 2024

    2024

  4. [4]

    Maskgit: Masked generative image transformer

    Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. Maskgit: Masked generative image transformer. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11315–11325, 2022

    work page 2022

  5. [5]

    Diffusion forcing: Next-token prediction meets full-sequence diffusion.Advances in Neural Information Processing Systems, 37:24081–24125, 2024

    Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitzmann. Diffusion forcing: Next-token prediction meets full-sequence diffusion.Advances in Neural Information Processing Systems, 37:24081–24125, 2024

    2024

  6. [6]

    Videocrafter2: Overcoming data limitations for high-quality video diffusion models

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7310–7320, 2024

    2024

  7. [7]

    Seine: Short-to-long video diffusion model for generative transition and prediction

    Xinyuan Chen, Yaohui Wang, Lingjun Zhang, Shaobin Zhuang, Xin Ma, Jiashuo Yu, Yali Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. Seine: Short-to-long video diffusion model for generative transition and prediction. InThe Twelfth International Conference on Learning Representations, 2023

    2023

  8. [8]

    Autoregressive video generation without vector quantization.arXiv preprint arXiv:2412.14169, 2024

    Haoge Deng, Ting Pan, Haiwen Diao, Zhengxiong Luo, Yufeng Cui, Huchuan Lu, Shiguang Shan, Yonggang Qi, and Xinlong Wang. Autoregressive video generation without vector quantization.arXiv preprint arXiv:2412.14169, 2024

    arXiv 2024

Show all 37 references
  1. [9]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021

    2021

  2. [10]

    Fluid: Scaling autoregressive text-to-image generative models with continuous tokens.arXiv preprint arXiv:2410.13863, 2024

    Lijie Fan, Tianhong Li, Siyang Qin, Yuanzhen Li, Chen Sun, Michael Rubinstein, Deqing Sun, Kaiming He, and Yonglong Tian. Fluid: Scaling autoregressive text-to-image generative models with continuous tokens.arXiv preprint arXiv:2410.13863, 2024

    arXiv

  3. [11]

    Maskflow: Discrete flows for flexible and efficient long video generation.arXiv preprint arXiv:2502.11234, 2025

    Michael Fuest, Vincent Tao Hu, and Björn Ommer. Maskflow: Discrete flows for flexible and efficient long video generation.arXiv preprint arXiv:2502.11234, 2025

    2025 arXiv

  4. [12]

    Streamingt2v: Consistent, dynamic, and extendable long video generation from text.arXiv preprint arXiv:2403.14773, 2024

    Roberto Henschel, Levon Khachatryan, Daniil Hayrapetyan, Hayk Poghosyan, Vahram Tade- vosyan, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Streamingt2v: Consistent, dynamic, and extendable long video generation from text.arXiv preprint arXiv:2403.14773, 2024

    2024 arXiv

  5. [13]

    Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020. 10

    2020

  6. [14]

    Step-video-ti2v technical report: A state-of-the-art text-driven image-to-video generation model.arXiv preprint arXiv:2503.11251, 2025

    Haoyang Huang, Guoqing Ma, Nan Duan, Xing Chen, Changyi Wan, Ranchen Ming, Tianyu Wang, Bo Wang, Zhiying Lu, Aojie Li, et al. Step-video-ti2v technical report: A state-of-the-art text-driven image-to-video generation model.arXiv preprint arXiv:2503.11251, 2025

    2025 arXiv

  7. [15]

    Vbench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. InProceedings of the IEEE/CVF Conference on Computer Vision and Patt...

    2024

  8. [16]

    Videopoet: A large language model for zero-shot video generation.arXiv preprint arXiv:2312.14125, 2023

    Dan Kondratyuk, Lijun Yu, Xiuye Gu, José Lezama, Jonathan Huang, Grant Schindler, Rachel Hornung, Vighnesh Birodkar, Jimmy Yan, Ming-Chang Chiu, et al. Videopoet: A large language model for zero-shot video generation.arXiv preprint arXiv:2312.14125, 2023

    2023 arXiv

  9. [17]

    Dart: Noise injection for robust imitation learning

    Michael Laskey, Jonathan Lee, Roy Fox, Anca Dragan, and Ken Goldberg. Dart: Noise injection for robust imitation learning. InConference on robot learning, pages 143–156. PMLR, 2017

    2017

  10. [18]

    Autoregressive image generation without vector quantization.Advances in Neural Information Processing Systems, 37:56424–56445, 2024

    Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization.Advances in Neural Information Processing Systems, 37:56424–56445, 2024

    2024

  11. [19]

    Fixing weight decay regularization in adam.arXiv preprint arXiv:1711.05101, 5:5, 2017

    Ilya Loshchilov, Frank Hutter, et al. Fixing weight decay regularization in adam.arXiv preprint arXiv:1711.05101, 5:5, 2017

    2017 arXiv

  12. [20]

    Chatgpt, 2022.https://openai.com/blog/chatgpt

    OpenAI. Chatgpt, 2022.https://openai.com/blog/chatgpt

    2022

  13. [21]

    Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of machine learning research, 21(140):1–67, 2020

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of machine learning research, 21(140):1–67, 2020

    2020

  14. [22]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. InInternational conference on machine learning, pages 8821–8831. Pmlr, 2021

    2021

  15. [23]

    Next block prediction: Video generation via semi-auto-regressive modeling.arXiv preprint arXiv:2502.07737, 2025

    Shuhuai Ren, Shuming Ma, Xu Sun, and Furu Wei. Next block prediction: Video generation via semi-auto-regressive modeling.arXiv preprint arXiv:2502.07737, 2025

    2025 arXiv

  16. [24]

    Consisti2v: Enhancing visual consistency for image-to-video generation.arXiv preprint arXiv:2402.04324, 2024

    Weiming Ren, Huan Yang, Ge Zhang, Cong Wei, Xinrun Du, Wenhao Huang, and Wenhu Chen. Consisti2v: Enhancing visual consistency for image-to-video generation.arXiv preprint arXiv:2402.04324, 2024

    2024 arXiv

  17. [25]

    Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063, 2024

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063, 2024

    2024

  18. [26]

    Autoregressive model beats diffusion: Llama for scalable image generation.arXiv preprint arXiv:2406.06525, 2024

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation.arXiv preprint arXiv:2406.06525, 2024

    2024 arXiv

  19. [27]

    Emu3: Next-token prediction is all you need

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869, 2024

    2024 arXiv

  20. [28]

    Loong: Generating minute-level long videos with autoregressive language models.arXiv preprint arXiv:2410.02757, 2024

    Yuqing Wang, Tianwei Xiong, Daquan Zhou, Zhijie Lin, Yang Zhao, Bingyi Kang, Jiashi Feng, and Xihui Liu. Loong: Generating minute-level long videos with autoregressive language models.arXiv preprint arXiv:2410.02757, 2024

    2024 arXiv

  21. [29]

    Art-v: Auto-regressive text-to-video generation with diffusion models

    Wenming Weng, Ruoyu Feng, Yanhui Wang, Qi Dai, Chunyu Wang, Dacheng Yin, Zhiyuan Zhao, Kai Qiu, Jianmin Bao, Yuhui Yuan, et al. Art-v: Auto-regressive text-to-video generation with diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogn...

    2024

  22. [30]

    Qwen2 technical report.arXiv preprint arXiv:2407.10671, 2024

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 technical report.arXiv preprint arXiv:2407.10671, 2024. 11

    2024 arXiv

  23. [31]

    Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024

    2024 arXiv

  24. [32]

    From slow bidirectional to fast autoregressive video diffusion models.arXiv preprint arXiv:2412.07772, 2, 2024

    Tianwei Yin, Qiang Zhang, Richard Zhang, William T Freeman, Fredo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast autoregressive video diffusion models.arXiv preprint arXiv:2412.07772, 2, 2024

    2024

  25. [33]

    Frequency autoregressive image generation with continuous tokens.arXiv preprint arXiv:2503.05305, 2025

    Hu Yu, Hao Luo, Hangjie Yuan, Yu Rong, and Feng Zhao. Frequency autoregressive image generation with continuous tokens.arXiv preprint arXiv:2503.05305, 2025

    2025

  26. [34]

    Magvit: Masked generative video transformer

    Lijun Yu, Yong Cheng, Kihyuk Sohn, José Lezama, Han Zhang, Huiwen Chang, Alexander G Hauptmann, Ming-Hsuan Yang, Yuan Hao, Irfan Essa, et al. Magvit: Masked generative video transformer. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages...

    2023

  27. [35]

    I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models.arXiv preprint arXiv:2311.04145, 2023

    Shiwei Zhang, Jiayu Wang, Yingya Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qin, Xiang Wang, Deli Zhao, and Jingren Zhou. I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models.arXiv preprint arXiv:2311.04145, 2023

    2023 arXiv

  28. [36]

    Bridging the gap between training and inference for neural machine translation.arXiv preprint arXiv:1906.02448, 2019

    Wen Zhang, Yang Feng, Fandong Meng, Di You, and Qun Liu. Bridging the gap between training and inference for neural machine translation.arXiv preprint arXiv:1906.02448, 2019

    1906 arXiv

  29. [37]

    Taming teacher forcing for masked autoregressive video generation.arXiv preprint arXiv:2501.12389, 2025

    Deyu Zhou, Quan Sun, Yuang Peng, Kun Yan, Runpei Dong, Duomin Wang, Zheng Ge, Nan Duan, Xiangyu Zhang, Lionel M Ni, et al. Taming teacher forcing for masked autoregressive video generation.arXiv preprint arXiv:2501.12389, 2025. 12 A Position Encoding In this paper, we mainly v...

    2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.