Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

FastCar: Cache Attentive Replay for Fast Auto-Regressive Video Generation on the Edge

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read FastCar speeds auto-regressive video decoding up to 2.1x by replaying cached outputs for temporally similar tokens, with quality close to the unmodified model.

desk verdict A genuinely new, practically useful trick—replaying MLP outputs guided by an attention score—with a solid empirical core, a fixable theory slip, and a real open question about long-horizon drift. read the letter →

arxiv 2505.14709 v1 pith:2BDJUJWB submitted 2025-05-17 cs.CV cs.AI

Xuan Shen , Weize Ma , Yufa Zhou , Enhao Tang , Yanyue Xie , Zhengang Li , Yifan Gong , Quanyi Wang
show 6 more authors
Henghui Ding Yiwei Wang Yanzhi Wang Pu Zhao Jun Lin Jiuxiang Gu
This is my paper · ORCID
classification cs.CVcs.AI
keywords auto-regressivevideogenerationtemporalredundancyMLPcachereplayattentionscoreedgeinferenceFPGAaccelerationdynamicresourceschedulingefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FastCar claims that the decode phase of auto-regressive video generation can be made substantially faster, without retraining, by exploiting temporal redundancy in the transformer's feed-forward (MLP) layers. Profiling shows that MLP modules, not attention, dominate decode latency, and that MLP outputs for the same spatial position in neighboring frames are highly similar. The framework introduces a Temporal Attention Score (TAS) read for free from the already-computed attention, and replays the previous frame's cached MLP output whenever the mean score clears a threshold. This gives edge deployments a training-free way to attack the true latency bottleneck rather than the attention modules targeted by sparse-attention methods. In the FPGA hardware implementation, the paper reports up to 1.77x speedup with replay alone at an 80% replay ratio and more than 2.1x when combined with sparse attention, with the VBench video-quality score dropping from 74.1% to 71.5%.

What carries the argument

The load-bearing object is the Temporal Attention Score (TAS), defined for a token $(t,i)$ as the mean over heads of the scaled dot product between the current token's query and the aligned previous-frame token's key, $\bar{s}_{t,i} = \frac{1}{h}\sum_{m=1}^{h} \langle q^{(m)}_{(t,i)}, k^{(m)}_{(t-1,i)}\rangle / \sqrt{d}$. It is the gate in the replay rule $Y_{(t,i)} = Y_{(t-1,i)}$ if $\bar{s}_{t,i} \ge \tau$, and it doubles as the theoretical quantity: a chain of Lipschitz bounds (Theorems 4.4, 4.6, and 4.7) shows that high TAS, combined with similar inputs, bounds the MLP output difference. The same per-token replay pattern is then fed to a Dynamic Resource Scheduling unit on the FPGA, which redistributes skipped batches across cores to keep utilization balanced.

What would settle it

Run the same model on 32- or 64-frame videos at a fixed replay threshold and check whether per-frame video quality relative to the unmodified model declines with frame index, or whether the replay ratio climbs toward 100 percent within a few frames; either outcome would show that the one-step stability bound does not cover the regime where replays chain together.

Watch

Extended reading notes

Core claim

The central discovery is that temporal redundancy in auto-regressive video transformers is directly readable from attention, and reading it enables safe cache replay. For each token $(t,i)$, the paper defines the Temporal Attention Score as the mean over attention heads of the scaled dot product between the token's query and the key of its aligned predecessor $(t-1,i)$. Because that score comes out of the attention module that already runs before the MLP, the gate costs nothing extra. The paper proves, under bounded-hidden-state and Lipschitz assumptions, that a high score bounds the difference between the current and previous MLP outputs (Theorem 4.7), and on that basis sets $Y_{(t,i)} = Y_{(t-1,i)}$ when the mean score is at least a threshold $\tau$, otherwise computing normally. The result is a conditional computation that skips the dominant cost of decoding for tokens whose representation is changing little.

Load-bearing premise

The load-bearing premise is that reusing the previous frame's stored output for a token stays accurate frame after frame, even when replays chain together and feed stale values into later layers; the paper's theory bounds only one step and its experiments cover only eight-frame videos.

Editorial extensions

If this is right

  • At an 80% replay ratio the framework removes 45% of decoder computations and cuts latency by 1.77x while keeping the VBench total score at 71.5%, within 2.6 points of the unmodified model's 74.1%.
  • The replay decision adds no extra computation, because the temporal attention score is read from the attention module that must run anyway, so the gain scales with the share of tokens whose MLP computation can be skipped.
  • FastCar is complementary to sparse attention: combining the two reaches more than 2.1x speedup and keeps quality stable where sparse attention alone drops sharply, so the method can be stacked with other efficiency techniques.
  • Replay is distributed unevenly across layers, with shallow and deep layers replaying most and intermediate layers replaying least, which locates temporal dynamics mainly in the middle of the network.
  • The FPGA accelerator with Dynamic Resource Scheduling converts the workload imbalance caused by selective replay into balanced core utilization, which is what turns the computation savings into the reported latency and energy-efficiency gains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension would apply the same gate to other heavy per-token modules, such as selected attention projections or normalization, whenever a temporal attention score indicates that a token's representation is barely moving; nothing in the mechanism restricts it to MLPs.
  • Because replay changes the hidden states that feed the next layer, the gate creates a feedback path: a replayed output influences the next frame's temporal attention score, which decides whether the next frame replays too. The paper's experiments stop at eight frames, so measuring replay ratio and per-frame quality on 32- or 64-frame videos would show whether that feedback stays benign.
  • The TAS signal could be reused outside generation, for example as a cheap per-token staleness measure for scheduling cache refreshes in other autoregressive modalities such as audio or multi-view sequences, where consecutive tokens are also highly correlated.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. FastCar proposes to accelerate auto-regressive video generation by exploiting temporal redundancy in MLP modules. The core idea is to compute a Temporal Attention Score (TAS) for each token and, when the score exceeds a threshold, skip the MLP computation and reuse the cached MLP output from the aligned token in the previous frame. The paper provides a theoretical analysis intended to show that high TAS implies small MLP output differences, describes an FPGA accelerator with dynamic resource scheduling, and reports experiments on VILA-U showing up to 1.77x speedup with FastCar alone and up to 2.13x when combined with sparse attention, with modest quality degradation.

Significance. The empirical direction is potentially valuable: MLP latency is indeed the bottleneck in AR video decoding, and exploiting temporal redundancy is orthogonal to sparse attention. The paper ships code and a reproducible experimental setup on a recent open model. However, the theoretical justification has a scaling error that breaks the formal link between TAS and output similarity, the FPGA measurement is not fully described, and the long-duration claims outrun the 8-frame evaluation. If the authors correct the theory and clarify the hardware methodology, the method could be a useful contribution to efficient AR video generation on edge devices.

major comments (3)
  1. [Section 4.4 / Appendix 10.1, Eq. (5)] The proof of Theorem 4.4 contains a scaling error that invalidates the stated bound. Eq. (5) defines TAS as s_{t,i} = <q_j, k_{j-}> / sqrt(d). The proof Step 1 claims that under the normalization assumption ||q_j||=||k_{j-}||=1, s_{t,i} equals the cosine similarity and then applies the Law of Cosines as ||q_j - k_{j-}||^2 = 2(1 - s_{t,i}). However, with the scaling in Eq. (5), the cosine similarity is <q_j, k_{j-}> = sqrt(d) * s_{t,i}, so the correct identity is ||q_j - k_{j-}||^2 = 2(1 - sqrt(d) * s_{t,i}). The bound in Eq. (9) and Theorem 4.7 therefore do not follow. Moreover, for the LLaMA-2-7B model used in the experiments (d=4096), sqrt(d)=64, so the normalized scores range in [-0.0156, 0.0156]. The threshold values reported in Section 6.3 (tau from 0 to -16) are all far outside this range, which would force every token to replay under Eq. (5), contradicting the reported replay ratios (3.96% at tau=0, 87.49% at tau=-8). This indicates an inconsistency between the formal definition of TAS and the score used in the implementation. The theoretical justification of the replay decision is thus not valid as written. Please correct the scaling (either redefine TAS as the unscaled cosine similarity or revise the proof) and state explicitly which score is used in the experiments.
  2. [Section 6.1, Section 6.2, Section 7 (Table 2)] The abstract and conclusion claim 'more than 2.1x decoding speedup' and advantages for 'high-resolution and long-duration video generation,' but these claims are not supported by the reported evidence. In Table 1, FastCar alone reaches at most 1.77x speedup (80% replay); the 2.1x+ figures appear only in the combined Ours+Sparse Attn rows of Table 2 (e.g., 2.13x at 87% replay with local size 16). In addition, all quality experiments use 8 frames at 256x256 resolution (Section 6.1), with no experiments on longer videos. Theorem 4.7 is a one-step Lipschitz bound: it relates the MLP output difference at adjacent frames to the input difference and TAS, but it does not analyze how that input difference evolves when earlier layers or earlier frames have already been replayed. Because a replayed output feeds into the next layer and affects the TAS used to decide further replays, the bound does not rule out error accumulation over many frames. The claimed scalability to long-duration video generation is therefore an extrapolation rather than a demonstrated result. Either add experiments on longer videos (e.g., 16 or 32 frames) and ideally a sequence-level error bound, or temper the claims.
  3. [Section 5, Section 6.1, Section 6.2 (Tables 1-2)] The FPGA evaluation is underspecified relative to the reported whole-video latencies. The text says 'Latency and power are tested using a prefill sequence length of 256' (Section 6.1) and 'we implement multiple accelerator cores on the FPGA' (Section 5), but it does not state the hardware configuration (number of cores, clock frequency, on-chip SRAM capacity, off-chip bandwidth), how the dense baseline is mapped onto the accelerator, whether the reported latency corresponds to the full generation of an 8-frame video or only a portion, or how power is measured. The power efficiency values in Tables 1 and 2 are given as GFLOPs/W, but the measured power numbers are not reported. Without this information, the claimed 2.1x speedup and energy efficiency advantages cannot be replicated or verified. Please provide a complete hardware methodology, including the mapping of the target model (or a clear statement of which components run on the FPGA) and the measurement setup.
minor comments (5)
  1. [Table 1] The units of latency appear inconsistent: the header says '(s)' but dense latency of 689.7 is more plausibly milliseconds for 8-frame, 256x256 generation on an A100. Please verify and correct the units.
  2. [Figure 4 left] The left panel of Figure 4 is difficult to read: the axes are not clearly labeled (the meaning of 'Ratio' and its relation to the LPIPS and VBench score curves is unclear). Please redraw with explicit axis labels.
  3. [References] References [4] and [12] are the same paper (Sun et al., 'Autoregressive model beats diffusion...'); please remove the duplicate.
  4. [Appendix 10.1, Step 1] The phrase 'up to sqrt(d) scaling' in Step 1 is self-contradictory: if s is scaled by sqrt(d), it is not the cosine similarity. This is related to the major scaling issue and should be clarified.
  5. [Section 4.4, Remark 4.5] Remark 4.5 states that TAS 'does not accumulate information across layers,' but because a replay decision in a shallow layer changes the hidden state that becomes the input to deeper layers, the TAS in deeper layers is affected by earlier replay decisions. The statement should be qualified to avoid overclaiming the stability of the signal.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the TAS replay gate is an inference-time heuristic with a user-set threshold, and the theoretical justification is a direct Lipschitz/cosine-similarity bound derived from the model definitions rather than a fitted quantity renamed as a prediction.

full rationale

The paper's replay decision (Eq. 8) is governed by the mean temporal attention score (Eq. 7), which is computed from the model's own query/key vectors, and the threshold tau is manually swept to obtain replay ratios rather than fitted to quality targets. The central bounds (Theorems 4.4, 4.6, 4.7) are derived from Definitions 4.1-4.3 by the law of cosines and Lipschitz continuity; Eq. (11) is a consequence of the definitions and not an assumption of the conclusion. The cited prior work of the same group ([36], [41]) supplies only standard Lipschitz-continuity and normalization assumptions in the proof, which are elementary and peripheral; the central claim does not reduce to a self-citation chain. The empirical validation uses externally measured VBench/PSNR/SSIM/LPIPS after replay and hardware latency measurements, not quantities determined by the method's own fitted parameters. Potential concerns about long-horizon drift and the mismatch between the proof's shared-K assumption and causal decoding are correctness risks, not circularity. Therefore no circular step is identified.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central mechanism relies on bounded and unit-norm q/k assumptions, Lipschitz continuity of attention and MLP, and an unstated stability of replay over frames. The threshold tau is the only free hyperparameter. No new physical entities are introduced; TAS is a defined scalar and DRS is a hardware module.

free parameters (1)
  • Threshold tau = Varied from 0 to -16 in ablations; -4 or -8 used for high replay ratio
    Manually chosen threshold on the average Temporal Attention Score that decides when to replay the previous frame's MLP output. It controls the trade-off between compute savings and quality; the theory does not prescribe a value.
assumptions (4)
  • domain assumption Hidden states are bounded in L2 norm (||x_j|| <= M)
    Used in Theorem 4.4 to bound the key matrix norm and projection differences. Not empirically verified for VILA-U.
  • domain assumption Query and key vectors are unit-normed
    Assumption (3) in Theorem 4.4 and used in the Law of Cosines step. Transformers with RMSNorm on q/k do not generally produce unit-norm vectors.
  • standard math Attention softmax and MLP are Lipschitz continuous
    Invoked in Theorems 4.4 and 4.6 with constants L and L_attn; existence is standard, but values are not estimated.
  • ad hoc to paper Replaying MLP outputs does not cause temporally accumulating error or drift
    Implicit in Eq. (8) and the 8-frame experiments; no stability analysis is provided, so the claim that FastCar handles long-duration video is not supported by the proof.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FastCar: Cache Attentive Replay for Fast Auto-Regressive Video Generation on the Edge." pith.science (2026). https://pith.science/paper/2BDJUJWB

@misc{pith2026250514709,
  author       = {Pith},
  title        = {Pith review of: FastCar: Cache Attentive Replay for Fast Auto-Regressive Video Generation on the Edge},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2BDJUJWB}},
  note         = {Machine review of arXiv:2505.14709}
}
read the original abstract

Auto-regressive (AR) models, initially successful in language generation, have recently shown promise in visual generation tasks due to their superior sampling efficiency. Unlike image generation, video generation requires a substantially larger number of tokens to produce coherent temporal frames, resulting in significant overhead during the decoding phase. Our key observations are: (i) MLP modules in the decode phase dominate the inference latency, and (ii) there exists high temporal redundancy in MLP outputs of adjacent frames. In this paper, we propose the \textbf{FastCar} framework to accelerate the decode phase for the AR video generation by exploring the temporal redundancy. The Temporal Attention Score (TAS) is proposed to determine whether to apply the replay strategy (\textit{i.e.}, reusing cached MLP outputs from the previous frame to reduce redundant computations) with detailed theoretical analysis and justification. Also, we develop a hardware accelerator on FPGA with Dynamic Resource Scheduling (DRS) based on TAS to enable better resource utilization and faster inference. Experimental results demonstrate the effectiveness of our method, which outperforms traditional sparse attention approaches with more than 2.1x decoding speedup and higher energy efficiency on the edge. Furthermore, by combining FastCar and sparse attention, FastCar can boost the performance of sparse attention with alleviated drifting, demonstrating our unique advantages for high-resolution and long-duration video generation. Code: https://github.com/shawnricecake/fast-car

Figures

Figures reproduced from arXiv: 2505.14709 by the authors.

Figure 1
Figure 1. Left: FastCar framework. We replay the cache from the previous frame to skip the computations for MLP in decoding. Replay is triggered when the average TAS exceeds a predefined threshold τ . Right Top: Latency cost of both prefill and decode phases for different sequence lengths. Right Bottom: Detailed latency cost of the decode phase for different sequence lengths. deployments for resource-constrained environments … view at source ↗
Figure 2
Figure 2. Cosine similarity for MLP outputs between neighboring frames for all 32 MLP modules. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Left: The top-level block diagram of our hardware accelerator. Right: The DRS diagram. To address this, we propose the DRS to balance the computational workloads, as shown in the right of [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Replay ratio distribution across layers for thresholds [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Visualization for the prompt "A dog wearing sunglasses on the beach.". The second and third rows are generated with threshold τ = −4 (i.e., 82% replay ratio). The third and fourth rows are generated with a sink size that extends the prompt length by one frame and 64. R…
Figure 7
Figure 7. Figure 7: Additional visualization with threshold τ = −1, −2, −4. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Reasoning Like an Economist: Post-Training on Economic Problems Induces Strategic Generalization in LLMs

    cs.AI 2025-05 conditional novelty 6.0 of 10

    Post-training an LLM on economic reasoning problems improves accuracy on economic benchmarks and, without game-specific training, raises its Nash equilibrium frequency and win rates in strategic games.

  2. 7B Fully Open Source Moxin-LLM/VLM -- From Pretraining to GRPO-based Reinforcement Learning Enhancement

    cs.CL 2024-12 conditional novelty 4.0 of 10

    The authors trained and openly released a 7B LLM, an instruction-tuned variant, a GRPO-based reasoning variant, and a VLM, claiming competitive or superior performance on zero-shot, few-shot, CoT, and VLM benchmarks.

Reference graph

Works this paper leans on

58 extracted references · 11 canonical work pages · cited by 2 Pith papers

  1. [1]

    Language models are unsupervised multitask learners

    Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019

    2019

  2. [2]

    Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    arXiv 2023

  3. [3]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    arXiv 2024

  4. [5]

    Emu3: Next-token prediction is all you need

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869, 2024

    arXiv 2024

  5. [6]

    Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis, 2024

    Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaobing Liu. Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis, 2024

    2024

  6. [7]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. 2024

    2024

  7. [8]

    Art•v: Auto-regressive text-to-video generation with diffusion models.arXiv preprint arXiv:2311.18834, 2023

    Wenming Weng, Ruoyu Feng, Yanhui Wang, Qi Dai, Chunyu Wang, Dacheng Yin, Zhiyuan Zhao, Kai Qiu, Jianmin Bao, Yuhui Yuan, Chong Luo, Yueyi Zhang, and Zhiwei Xiong. Art•v: Auto-regressive text-to-video generation with diffusion models.arXiv preprint arXiv:2311.18834, 2023

    arXiv 2023

  8. [9]

    Autoregressive video generation without vector quantization.arXiv preprint arXiv:2412.14169, 2024

    Haoge Deng, Ting Pan, Haiwen Diao, Zhengxiong Luo, Yufeng Cui, Huchuan Lu, Shiguang Shan, Yonggang Qi, and Xinlong Wang. Autoregressive video generation without vector quantization.arXiv preprint arXiv:2412.14169, 2024

    arXiv 2024

Show all 58 references
  1. [10]

    Flexvar: Flexible visual autoregressive modeling without residual prediction.arXiv preprint arXiv:2502.20313, 2025

    Siyu Jiao, Gengwei Zhang, Yinlong Qian, Jiancheng Huang, Yao Zhao, Humphrey Shi, Lin Ma, Yunchao Wei, and Zequn Jie. Flexvar: Flexible visual autoregressive modeling without residual prediction.arXiv preprint arXiv:2502.20313, 2025. 10

    2025

  2. [11]

    Progressive autoregressive video diffusion models.arXiv preprint arXiv:2410.08151, 2024

    Desai Xie, Zhan Xu, Yicong Hong, Hao Tan, Difan Liu, Feng Liu, Arie Kaufman, and Yang Zhou. Progressive autoregressive video diffusion models.arXiv preprint arXiv:2410.08151, 2024

    2024 arXiv

  3. [12]

    Autoregressive model beats diffusion: Llama for scalable image generation.arXiv preprint arXiv:2406.06525, 2024

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation.arXiv preprint arXiv:2406.06525, 2024

    2024 arXiv

  4. [13]

    Open- magvit2: An open-source project toward democratizing auto-regressive visual generation.arXiv preprint arXiv:2409.04410, 2024

    Zhuoyan Luo, Fengyuan Shi, Yixiao Ge, Yujiu Yang, Limin Wang, and Ying Shan. Open- magvit2: An open-source project toward democratizing auto-regressive visual generation.arXiv preprint arXiv:2409.04410, 2024

    2024 arXiv

  5. [14]

    Videopoet: A large language model for zero-shot video generation.arXiv preprint arXiv:2312.14125, 2023

    Dan Kondratyuk, Lijun Yu, Xiuye Gu, José Lezama, Jonathan Huang, Grant Schindler, Rachel Hornung, Vighnesh Birodkar, Jimmy Yan, Ming-Chang Chiu, et al. Videopoet: A large language model for zero-shot video generation.arXiv preprint arXiv:2312.14125, 2023

    2023 arXiv

  6. [15]

    Parallelized autoregressive visual generation.arXiv preprint arXiv:2412.15119, 2024

    Yuqing Wang, Shuhuai Ren, Zhijie Lin, Yujin Han, Haoyuan Guo, Zhenheng Yang, Difan Zou, Jiashi Feng, and Xihui Liu. Parallelized autoregressive visual generation.arXiv preprint arXiv:2412.15119, 2024

    2024 arXiv

  7. [16]

    Autoregressive models in vision: A survey.arXiv preprint arXiv:2411.05902, 2024

    Jing Xiong, Gongye Liu, Lun Huang, Chengyue Wu, Taiqiang Wu, Yao Mu, Yuan Yao, Hui Shen, Zhongwei Wan, Jinfa Huang, et al. Autoregressive models in vision: A survey.arXiv preprint arXiv:2411.05902, 2024

    2024 arXiv

  8. [17]

    A survey on video diffusion models.ACM Computing Surveys, 57(2):1–42, 2024

    Zhen Xing, Qijun Feng, Haoran Chen, Qi Dai, Han Hu, Hang Xu, Zuxuan Wu, and Yu-Gang Jiang. A survey on video diffusion models.ACM Computing Surveys, 57(2):1–42, 2024

    2024

  9. [18]

    A survey on long video generation: Challenges, methods, and prospects.arXiv preprint arXiv:2403.16407, 2024

    Chengxuan Li, Di Huang, Zeyu Lu, Yang Xiao, Qingqi Pei, and Lei Bai. A survey on long video generation: Challenges, methods, and prospects.arXiv preprint arXiv:2403.16407, 2024

    2024 arXiv

  10. [19]

    Video diffusion models: A survey.Transactions on Machine Learning Research, 2024

    Andrew Melnik, Michal Ljubljanac, Cong Lu, Qi Yan, Weiming Ren, and Helge Ritter. Video diffusion models: A survey.Transactions on Machine Learning Research, 2024. Survey Certification

    2024

  11. [20]

    Open-sora plan: Open-source large video generation model.arXiv preprint arXiv:2412.00131, 2024

    Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, et al. Open-sora plan: Open-source large video generation model.arXiv preprint arXiv:2412.00131, 2024

    2024 arXiv

  12. [21]

    Open-sora: Democratizing efficient video production for all

    Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv preprint arXiv:2412.20404, 2024

    2024 arXiv

  13. [22]

    Open-sora 2.0: Training a commercial- level video generation model in $200k.arXiv preprint arXiv:2503.09642, 2025

    Xiangyu Peng, Zangwei Zheng, Chenhui Shen, et al. Open-sora 2.0: Training a commercial- level video generation model in $200k.arXiv preprint arXiv:2503.09642, 2025

    2025 arXiv

  14. [23]

    Cogvideo: Large-scale pretraining for text-to-video generation via transformers.arXiv preprint arXiv:2205.15868, 2022

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers.arXiv preprint arXiv:2205.15868, 2022

    2022 arXiv

  15. [24]

    Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, , et al. Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024

    2024 arXiv

  16. [25]

    Hunyuanvideo: A systematic framework for large video generative models, 2024

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, et al. Hunyuanvideo: A systematic framework for large video generative models, 2024

    2024

  17. [26]

    Scalable diffusion models with transformers.arXiv preprint arXiv:2212.09748, 2022

    William Peebles and Saining Xie. Scalable diffusion models with transformers.arXiv preprint arXiv:2212.09748, 2022

    2022 arXiv

  18. [27]

    Neighboring autoregressive modeling for efficient visual generation.arXiv preprint arXiv:2503.10696, 2025

    Yefei He, Yuanyu He, Shaoxuan He, Feng Chen, Hong Zhou, Kaipeng Zhang, and Bohan Zhuang. Neighboring autoregressive modeling for efficient visual generation.arXiv preprint arXiv:2503.10696, 2025

    2025 arXiv

  19. [28]

    Pyramidal flow matching for efficient video generative modeling.arXiv preprint arXiv:2410.05954, 2024

    Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Hao Jiang, Nan Zhuang, Quzhe Huang, Yang Song, Yadong Mu, and Zhouchen Lin. Pyramidal flow matching for efficient video generative modeling.arXiv preprint arXiv:2410.05954, 2024. 11

    2024

  20. [29]

    Xattention: Block sparse attention with antidiagonal scoring.arXiv preprint arXiv:2503.16428, 2025

    Ruyi Xu, Guangxuan Xiao, Haofeng Huang, Junxian Guo, and Song Han. Xattention: Block sparse attention with antidiagonal scoring.arXiv preprint arXiv:2503.16428, 2025

    2025 arXiv

  21. [30]

    Sparse videogen: Accelerating video diffusion transformers with spatial-temporal sparsity.arXiv preprint arXiv:2502.01776, 2025

    Haocheng Xi, Shuo Yang, Yilong Zhao, Chenfeng Xu, Muyang Li, Xiuyu Li, Yujun Lin, Han Cai, Jintao Zhang, Dacheng Li, et al. Sparse videogen: Accelerating video diffusion transformers with spatial-temporal sparsity.arXiv preprint arXiv:2502.01776, 2025

    2025 arXiv

  22. [31]

    Zipar: Accelerating auto-regressive image generation through spatial locality.arXiv preprint arXiv:2412.04062, 2024

    Yefei He, Feng Chen, Yuanyu He, Shaoxuan He, Hong Zhou, Kaipeng Zhang, and Bohan Zhuang. Zipar: Accelerating auto-regressive image generation through spatial locality.arXiv preprint arXiv:2412.04062, 2024

    2024 arXiv

  23. [32]

    Flightvgm: Efficient video generation model inference with online sparsification and hybrid precision on fpgas

    Jun Liu, Shulin Zeng, Li Ding, Widyadewi Soedarmadji, Hao Zhou, Zehao Wang, Jinhao Li, Jintao Li, Yadong Dai, Kairui Wen, Shan He, Yaqi Sun, Yu Wang, and Guohao Dai. Flightvgm: Efficient video generation model inference with online sparsification and hybrid precision on fpgas....

    2025

  24. [33]

    Mobilevidfactory: Automatic diffusion- based social media video generation for mobile devices from text

    Junchen Zhu, Huan Yang, Wenjing Wang, Huiguo He, Zixi Tuo, Yongsheng Yu, Wen-Huang Cheng, Lianli Gao, Jingkuan Song, Jianlong Fu, et al. Mobilevidfactory: Automatic diffusion- based social media video generation for mobile devices from text. InProceedings of the 31st ACM Inter...

    2023

  25. [34]

    On-device sora: Enabling diffusion-based text-to-video generation for mobile devices.arXiv preprint arXiv:2502.04363, 2025

    Bosung Kim, Kyuhwan Lee, Isu Jeong, Jungmin Cheon, Yeojin Lee, and Seulki Lee. On-device sora: Enabling diffusion-based text-to-video generation for mobile devices.arXiv preprint arXiv:2502.04363, 2025

    2025 arXiv

  26. [35]

    Terdit: Ternary diffusion models with transformers, 2024

    Xudong Lu, Aojun Zhou, Ziyi Lin, Qi Liu, Yuhui Xu, Renrui Zhang, Yafei Wen, Shuai Ren, Peng Gao, Junchi Yan, and Hongsheng Li. Terdit: Ternary diffusion models with transformers, 2024

    2024

  27. [36]

    Lazydit: Lazy learning for the acceleration of diffusion transformers

    Xuan Shen, Zhao Song, Yufa Zhou, Bo Chen, Yanyu Li, Yifan Gong, Kai Zhang, Hao Tan, Jason Kuen, Henghui Ding, et al. Lazydit: Lazy learning for the acceleration of diffusion transformers. InAAAI, 2025

    2025

  28. [37]

    Vila-u: a unified foundation model integrating visual understanding and generation.arXiv preprint arXiv:2409.04429, 2024

    Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, et al. Vila-u: a unified foundation model integrating visual understanding and generation.arXiv preprint arXiv:2409.04429, 2024

    2024 arXiv

  29. [38]

    Awq: Activation-aware weight quantization for llm compression and acceleration

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-aware weight quantization for llm compression and acceleration. InMLSys, 2024

    2024

  30. [39]

    Llm-pruner: On the structural pruning of large language models

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. Llm-pruner: On the structural pruning of large language models. InAdvances in Neural Information Processing Systems, 2023

    2023

  31. [40]

    Agile-quant: Activation-guided quantization for faster inference of llms on the edge

    Xuan Shen, Peiyan Dong, Lei Lu, Zhenglun Kong, Zhengang Li, Ming Lin, Chao Wu, and Yanzhi Wang. Agile-quant: Activation-guided quantization for faster inference of llms on the edge. InAAAI, 2024

    2024

  32. [41]

    Rossi, Hao Tan, Tong Yu, Xiang Chen, Yufan Zhou, Tong Sun, Pu Zhao, Yanzhi Wang, and Jiuxiang Gu

    Xuan Shen, Zhao Song, Yufa Zhou, Bo Chen, Jing Liu, Ruiyi Zhang, Ryan A. Rossi, Hao Tan, Tong Yu, Xiang Chen, Yufan Zhou, Tong Sun, Pu Zhao, Yanzhi Wang, and Jiuxiang Gu. Numerical pruning for efficient autoregressive models.Proceedings of the AAAI Conference on Artificial Int...

    2025

  33. [42]

    SmoothQuant: Accurate and efficient post-training quantization for large language models

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. SmoothQuant: Accurate and efficient post-training quantization for large language models. In Proceedings of the 40th International Conference on Machine Learning, 2023

    2023

  34. [43]

    Efficient streaming language models with attention sinks.arXiv, 2023

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks.arXiv, 2023

    2023

  35. [44]

    Kv-compress: Paged kv-cache compression with variable compression rates per attention head, 2024

    Isaac Rehg. Kv-compress: Paged kv-cache compression with variable compression rates per attention head, 2024. 12

    2024

  36. [45]

    Kvquant: Towards 10 million context length llm inference with kv cache quantization.arXiv preprint arXiv:2401.18079, 2024

    Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Michael W Mahoney, Yakun Sophia Shao, Kurt Keutzer, and Amir Gholami. Kvquant: Towards 10 million context length llm inference with kv cache quantization.arXiv preprint arXiv:2401.18079, 2024

    2024 arXiv

  37. [46]

    Mini- cache: Kv cache compression in depth dimension for large language models.arXiv preprint arXiv:2405.14366, 2024

    Akide Liu, Jing Liu, Zizheng Pan, Yefei He, Gholamreza Haffari, and Bohan Zhuang. Mini- cache: Kv cache compression in depth dimension for large language models.arXiv preprint arXiv:2405.14366, 2024

    2024 arXiv

  38. [47]

    Model tells you what to discard: Adaptive KV cache compression for LLMs

    Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, and Jianfeng Gao. Model tells you what to discard: Adaptive KV cache compression for LLMs. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  39. [48]

    SnapKV: LLM knows what you are looking for before generation

    Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen. SnapKV: LLM knows what you are looking for before generation. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  40. [49]

    Accelerating llm inference with staged speculative decoding

    Benjamin Spector and Chris Re. Accelerating llm inference with staged speculative decoding. arXiv preprint arXiv:2308.04623, 2023

    2023 arXiv

  41. [50]

    Inference with reference: Lossless acceleration of large language models.arXiv preprint arXiv:2304.04487, 2023

    Nan Yang, Tao Ge, Liang Wang, Binxing Jiao, Daxin Jiang, Linjun Yang, Rangan Majumder, and Furu Wei. Inference with reference: Lossless acceleration of large language models.arXiv preprint arXiv:2304.04487, 2023

    2023 arXiv

  42. [51]

    Specinfer: Accelerating generative large language model serving with tree-based speculative inference and verification

    Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Zeyu Wang, Zhengxin Zhang, Rae Ying Yee Wong, Alan Zhu, Lijie Yang, Xiaoxiang Shi, et al. Specinfer: Accelerating generative large language model serving with tree-based speculative inference and verification. arXiv pre...

    2023 arXiv

  43. [52]

    Skeleton- of-thought: Prompting LLMs for efficient parallel generation

    Xuefei Ning, Zinan Lin, Zixuan Zhou, Zifu Wang, Huazhong Yang, and Yu Wang. Skeleton- of-thought: Prompting LLMs for efficient parallel generation. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  44. [53]

    Speed: Speculative pipelined execution for efficient decoding

    Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Hasan Genc, Kurt Keutzer, Amir Gholami, and Sophia Shao. Speed: Speculative pipelined execution for efficient decoding. arXiv preprint arXiv:2310.12072, 2023

    2023 arXiv

  45. [54]

    Accelerating auto-regressive text-to-image generation with training-free speculative jacobi decoding.arXiv preprint arXiv:2410.01699, 2024

    Yao Teng, Han Shi, Xian Liu, Xuefei Ning, Guohao Dai, Yu Wang, Zhenguo Li, and Xihui Liu. Accelerating auto-regressive text-to-image generation with training-free speculative jacobi decoding.arXiv preprint arXiv:2410.01699, 2024

    arXiv

  46. [55]

    Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  47. [56]

    Autoregressive image generation using residual quantization

    Doyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho, and Wook-Shin Han. Autoregressive image generation using residual quantization. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11523–11532, 2022

    2022

  48. [57]

    VBench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Comprehensive benchmark suite for video generative models. I...

    2024

  49. [58]

    The unreason- able effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreason- able effectiveness of deep features as a perceptual metric. InCVPR, 2018

    2018

  50. [59]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022. 13 Appendix 8 Additional Results 8.1 Detailed Results for VBench We provide the detailed scores of VBench in Table 3 and Table 4. Our method better maintains the generation ...

    2022 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.