Pith. sign in

REVIEW 3 major objections 5 minor 12 cited by

Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharing

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By making temporal attention causal, video diffusion models can cache conditioning frames once and reuse them at every denoising step, cutting autoregressive generation cost from quadratic to roughly linear.

desk verdict Solid efficiency contribution with measured speedups and an honest appendix, but the 'comparable quality' half of the central claim is under-supported by the current experiments. read the letter →

arxiv 2411.16375 v2 pith:LWBJGY3U submitted 2024-11-25 cs.CV

Kaifeng Gao , Jiaxin Shi , Hanwang Zhang , Chunping Wang , Jun Xiao , Long Chen This is my paper · ORCID
classification cs.CV
keywords videodiffusionmodelsautoregressivegenerationKV-cachecausaltemporalattentionlong-terminferenceaccelerationcachesharinglatent
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Autoregressive video diffusion models waste compute by re-encoding the same conditioning frames at every generation step, and that waste grows quadratically when the condition is extended to carry long-term context. This paper claims that replacing the bidirectional temporal attention in a video diffusion transformer with causal temporal attention makes those conditioning features cacheable: keys and values of the clean prefix frames are computed once and reused in every subsequent step. A second mechanism, cache sharing, gives the clean prefix its own timestep embedding $t=0$, so the same cache is valid at every denoising timestep and GPU memory no longer grows with the number of denoising steps. If the paper is right, arbitrary-length video generation can run at roughly linear cost with quality comparable to bidirectional models.

What carries the argument

The load-bearing object is a causal temporal attention mask: a lower-triangular mask $M$ in $\mathrm{Softmax}(QK^\top/\sqrt{C'} + M)$ forces every frame to attend only to its predecessors. That unidirectional computation makes it possible to maintain a temporal KV-cache queue: clean conditional frames' keys and values are written once at timestep $t=0$ and reused by every later denoising stage. Cache sharing across denoising steps is justified by giving the clean prefix its own timestep embedding $t=0$ in both training and inference, so the cached features never depend on the noise level of the target frames. Two supporting mechanisms make the pipeline practical: prefix-enhanced spatial attention concatenates a short clean sub-prefix into the spatial attention keys and values to strengthen conditioning, and cyclic temporal positional embeddings keep positional information aligned with cached keys and values once generation exceeds the training length.

What would settle it

Generate identical prompts and starting frames with a causal-fine-tuned model and with the same model restored to bidirectional attention, and compare FVD chunk by chunk; if the causal variant is materially worse on early chunks where the bidirectional model is strong, the central quality claim fails. A sharper probe: test whether recomputing the clean prefix features at every denoising timestep instead of using the shared $t=0$ cache changes the output; if the videos differ noticeably, the cache-sharing premise is violated.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that unidirectionality plus a timestep-agnostic condition feature is enough to remove almost all redundant autoregressive computation from video diffusion. With causal temporal attention, a frame's features depend only on earlier frames, so the keys and values of the already-generated clean frames can be precomputed in a 'cache writing' pass at $t=0$ and then read, not recomputed, during each denoising step of each later autoregression step. The same cache is shared across all denoising timesteps because the clean prefix is always embedded with $t=0$ rather than by the current noise-level timestep. The paper reports that this design reaches FVD scores similar to bidirectional baselines while generating 80 frames in about half the time of a fixed-condition baseline and at constant KV-cache memory independent of denoising steps.

Load-bearing premise

The assumption the argument rests on is that converting a pretrained bidirectional video diffusion transformer's temporal attention into causal attention, with only fine-tuning, does not meaningfully degrade generation quality.

Editorial extensions

If this is right

  • Autoregressive video generation with extendable conditioning becomes roughly linear in compute per generated chunk rather than quadratic, because conditional frames are never re-encoded.
  • The KV-cache queue bounds memory independently of the number of denoising steps, so longer sampling schedules or more denoising steps no longer multiply cache cost.
  • Increasing the maximum condition length $P_{\max}$ improves temporal consistency for a small additional cost, since extended context enters only as cached keys and values.
  • The causal conversion with cache sharing applies to both text-to-video and video prediction and reaches FVD scores comparable to bidirectional baselines in the reported settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the authors' caveat is right that no causal pretraining was performed, training a video diffusion transformer with causally masked temporal attention from scratch could close the remaining quality gap to bidirectional models while keeping the same cache benefits.
  • The same cache-sharing design should transfer to other iterative generative models that denoise in many steps while conditioned on fixed context, such as image-to-video, audio, or long-horizon world models, and to samplers with variable step counts at no extra cache memory.
  • The cyclic positional-embedding mechanism implicitly assumes scene statistics repeat with a period equal to the training window; very long generations beyond that window are a natural stress test of whether periodic position assignment introduces visible looping artifacts.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Ca2-VDM, an autoregressive video diffusion transformer that replaces bidirectional temporal attention with causal temporal attention and caches keys/values of clean conditional frames, sharing them across all denoising steps and storing them in a bounded queue with cyclic temporal positional embeddings. The claimed effect is that autoregressive generation cost grows roughly linearly rather than quadratically in the number of autoregression steps, that GPU memory is independent of the number of denoising steps, and that generation quality remains comparable to bidirectional baselines. The efficiency claims are supported by timing, FLOPs, and memory measurements that match the algorithmic design; the quality claim is only partially supported, with the main controlled comparison against a bidirectional extendable-condition baseline limited to the SkyTimelapse video-prediction task.

Significance. If the efficiency results hold, this is a useful contribution: the KV-cache queue for video diffusion is not a trivial extension of language-model KV-caching because the diffusion model is called repeatedly with different timesteps, and the paper correctly identifies the two obstacles of cache computation and cache storage and proposes concrete solutions. The paper provides measured time, FLOPs, and memory numbers, the complexity analysis matches the algorithm, and the code is released. The quality evidence is currently the weak part of the central claim: Appendix F explicitly concedes that fine-tuning bidirectionally pretrained Open-Sora weights into causal attention might be sub-optimal, and the only bidirectional extendable-condition baseline trained by the authors, OS-Ext, is evaluated only on SkyTimelapse. With an additional controlled text-to-video comparison against a bidirectional extendable-condition baseline, or with a claim scoped to efficiency only, the paper would be substantially stronger.

major comments (3)
  1. [§4.2, Table 7, and Appendix F] The quality half of the central claim, that quality remains comparable to bidirectional models, is not established by the current experiments. Table 7 shows Ca2-VDM below OS-Ext on motion smoothness (97.59 vs 98.93) and temporal flickering (97.14 vs 98.57), with no error bars, and this is the only direct bidirectional extendable-condition baseline trained by the authors. Appendix F additionally concedes that fine-tuning bidirectional Open-Sora weights into causal attention 'might be sub-optimal' because no causal pretraining was performed. Because the speedup is real even if quality degrades, this point is load-bearing for the full claim. Please add a controlled text-to-video comparison against an OS-Ext-style bidirectional baseline on the InternVid/MSR-VTT task, and report per-seed variance for the quality tables.
  2. [§4.2, Table 1] The zero-shot FVD comparisons in Table 1 are not apples-to-apples. Ca2-VDM is listed with condition T+I, meaning it receives the ground-truth first frame as an image condition, while several baselines (ModelScope, VideoComposer, Make-A-Video) are text-only; on MSR-VTT, Ca2-VDM matches SEINE, which is also a T+I method. Reporting a T+I model alongside text-only models under the heading 'zero-shot text-to-video' conflates the advantage of the extra condition with generation quality. The text should either restrict the state-of-the-art/comparable claim to T+I models or provide a text-only Ca2-VDM variant.
  3. [§3.3, Table 3, and §D.2] The temporal-consistency evaluation in Table 3 is described inconsistently: the caption says 'FVD between AR step 1 and i', while §D.2 says the FVD of three 16-frame chunks is computed 'w.r.t. the 16-frame ground-truth videos'. These are different quantities, and the surrounding text claims lower FVD indicates better consistency without stating which reference distribution is used. Please clarify the metric and the reference distribution; if the numbers are chunk-wise FVD to ground truth, the caption should say so, and the claim that Ca2-VDM has lower FVD than OS-Fix and StreamT2V should be tied to the correct comparison.
minor comments (5)
  1. [Abstract and §5] The abstract claims 'state-of-the-art quantitative and qualitative video generation results', while the conclusion says the model 'shows comparable generation quality with existing SOTA VDMs'; these two claims should be aligned.
  2. [§4.2 and §D.2, Table 4] The ablation text says 'generated 48 frames by 6 AR steps' for Table 4, but §D.2 says 'each model generated 96 frames with 6 AR steps and l=16'; with l=16, six AR steps produce 96 frames. Please correct the inconsistent number.
  3. [Tables 3, 4, and 7] Quality tables do not report the number of generated videos used for each FVD/VBench estimate or standard deviations; adding these would materially help assess the small quality differences reported.
  4. [§1 and footnote 1] The text contains 'All most all Existing VDMs', which should read 'Almost all existing VDMs', and the footnote marker after 'Cyclic-TPEs 1.' is orphaned.
  5. [Eq. (4)] The definition of the sub-prefix indices in Eq. (4) is terse; please clarify that h^{P-P'}_0 through h^{P-1}_0 are the clean-prefix hidden states at the current layer, and that the broadcasting for i < P is over the spatial dimension.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the efficiency gain is measured empirically and the cache mechanism is a construction with explicit training/inference alignment, not a fitted parameter renamed as a prediction.

full rationale

The paper's central efficiency claim is supported by measured wall-clock time (Table 5, Figure 6), counted FLOPs (Figure 8), and measured GPU memory (Table 6). No fitted constant is subsequently relabeled as a prediction; the KV-cache reuse follows from the architecturally imposed causal mask (Eq. 3) and the tEmb(0) assignment for clean prefixes (Eq. 2), which are design choices aligned between training and inference and then validated empirically. The self-built OS-Fix and OS-Ext baselines are controlled comparisons for the same base model, not circular evidence, and the quality comparisons are external FVD/VBench evaluations. No load-bearing self-citation or imported uniqueness theorem is present; references such as Open-Sora, Latte, and PixArt-alpha are external architecture and pretraining sources. Appendix F explicitly concedes that converting bidirectionally pretrained Open-Sora temporal attention to causal attention 'might be sub-optimal' and that no causal pretraining was performed, and Table 7 shows Ca2-VDM slightly below OS-Ext on motion smoothness (97.59 vs 98.93) and temporal flickering (97.14 vs 98.57). These are genuine correctness and robustness caveats for the 'comparable quality' half of the claim, but they are empirical risks, not circular derivations. No equation or claim reduces to its own input by definition, so the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The method introduces no new physical entities. The free parameters are dataset and architecture choices that affect the efficiency-quality trade-off. The axioms are mostly domain assumptions about whether causal attention, t=0 caching, queue eviction, and cyclic positional embeddings preserve the behavior learned during training.

free parameters (3)
  • Maximum conditional prefix length Pmax = 49 for text-to-video, 25 for video prediction
    Chosen by hand as Pmax = 1 + 3l. Controls the trade-off between long-term context and cache storage/compute; ablations in Table 4 show it affects quality.
  • Autoregressive chunk length l = 16 for text-to-video, 8 for video prediction
    Determines the number of frames generated per autoregressive step and the size of each cache write. Selected per dataset.
  • Prefix-enhanced spatial attention sub-prefix length P' = 3
    Equation (4) uses a small sub-prefix of recent clean frames for spatial attention; the paper states the cost scales with P' and that adjacent frames look similar.
assumptions (5)
  • standard math The diffusion denoising objective remains valid when clips are partially noised and clean prefix frames receive timestep embedding t=0.
    Equation (2) modifies the standard L_simple objective with a mask; the paper assumes this trains a correct conditional denoiser for the causal setup.
  • domain assumption Fine-tuning Open-Sora's bidirectional temporal attention into causal attention preserves generation quality.
    Section 4.1 initializes from Open-Sora v1.0, and Appendix F concedes that no causal pretraining was performed and that this may be sub-optimal.
  • domain assumption A clean prefix frame's features at timestep t=0 are valid conditioning for every denoising timestep t.
    Cache sharing in Section 3.3 depends on this; it holds only if no layer mixes a global timestep embedding into the prefix path.
  • domain assumption Old KV entries can be safely dequeued when the queue reaches Pmax because recent frames dominate.
    The Discussion in Section 3.3 asserts this; no quantitative evidence is given for why Pmax is a safe cutoff.
  • domain assumption Cyclic-TPE training with random offsets matches inference-time queue rebinding after the TPE budget is exhausted.
    Section 3.3 introduces Cyclic-TPEs; the model must generalize to arbitrary cyclic shifts and queue states.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharing." pith.science (2026). https://pith.science/paper/LWBJGY3U

@misc{pith2026241116375,
  author       = {Pith},
  title        = {Pith review of: Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LWBJGY3U}},
  note         = {Machine review of arXiv:2411.16375}
}
read the original abstract

With the advance of diffusion models, today's video generation has achieved impressive quality. To extend the generation length and facilitate real-world applications, a majority of video diffusion models (VDMs) generate videos in an autoregressive manner, i.e., generating subsequent clips conditioned on the last frame(s) of the previous clip. However, existing autoregressive VDMs are highly inefficient and redundant: The model must re-compute all the conditional frames that are overlapped between adjacent clips. This issue is exacerbated when the conditional frames are extended autoregressively to provide the model with long-term context. In such cases, the computational demands increase significantly (i.e., with a quadratic complexity w.r.t. the autoregression step). In this paper, we propose Ca2-VDM, an efficient autoregressive VDM with Causal generation and Cache sharing. For causal generation, it introduces unidirectional feature computation, which ensures that the cache of conditional frames can be precomputed in previous autoregression steps and reused in every subsequent step, eliminating redundant computations. For cache sharing, it shares the cache across all denoising steps to avoid the huge cache storage cost. Extensive experiments demonstrated that our Ca2-VDM achieves state-of-the-art quantitative and qualitative video generation results and significantly improves the generation speed. Code is available: https://github.com/Dawn-LX/CausalCache-VDM

Figures

Figures reproduced from arXiv: 2411.16375 by the authors.

Figure 1
Figure 1. (a): Existing autoregressive VDMs with bidirec￾tional generation. The conditional frames can be fixed￾length (Henschel et al., 2025; Zheng et al., 2024) or extend￾able. (b): Our Ca2-VDM, which uses causal generation to enable KV-cache and introduce cache sharing across all de￾noising timesteps. Cache writing stands for a partial model forward on the denoised frames (i.e., at timestep t = 0) until the KV-caches of ev… view at source ↗
Figure 2
Figure 2. Comparison of bidirectional attention (a) and [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overview of the Ca2-VDM pipeline. (a): During training, we randomly set P frames clean prefix, and set distinctive timestep embeddings, i.e., tEmb(0) for the clean prefix and tEmb(t) for the denoising target. (b): During inference, in each autoregression (AR) step, the model denoises an l-frame chunk conditioned on the spatial/temporal KV-caches shared across all timesteps (denoising stage), and then computes the ke… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Illustration of causal temporal attention (a) & (b) [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 6
Figure 6. Figure 6: Accumulated time cost w.r.t. frame ids. We show OS-Ext and Ca2-VDM with Pmax = 25 and 41, and OS-Fix with a fixed P = 8. baseline-25 w/ PE cyclic-TPE ours-25 w/ PE MAIN-v3: Ca 2-VDM OS-Ext [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Results from OS-Ext and Ca2-VDM. They have comparable quality, while [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Number of floating-point operations (FLOPs) for [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Illustration of prefix-enhanced spatial attention. [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Qualitative examples generated by GenLV ( [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Qualitative examples from GenLV (Wang et al., 2023a), StreamT2V (Henschel et al., 2025), OS-Fix, and our Ca2-VDM. Yellow arrows highlight the consecutive frames having mutations [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Comparison between OS-Ext and Ca2-VDM in terms of long-term content drift ( [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MV-Forcing: Long Multi-View Video Generation via 4D-Grounded Spatio-Temporal Self-Forcing

    cs.CV 2026-07 conditional novelty 7.0 of 10

    MV-Forcing composes temporal and view-sequential autoregression in a single diffusion model, using a recurrent 3D reconstruction model as a geometric bridge to generate arbitrarily long, multi-view consistent videos.

  2. RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation

    cs.RO 2026-07 unverdicted novelty 6.5 of 10

    Step Forcing trains a few-step autoregressive video world model so RoboWorld closed-loop rollouts plus a task-progress VLM judge recover real-world policy rankings at r=0.989 and ρ=0.970.

  3. Context-Matched Distillation: Teacher Causality for Autoregressive Video Distillation

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A causal-distillation method that scores each generated video frame only under its causal history improves few-step autoregressive video quality and camera-control adherence.

  4. Stream Forcing: Constructing Unified Training Trajectory for Robust Streaming Video Generation

    cs.CV 2026-08 conditional novelty 6.0 of 10

    Stream Forcing constructs a curriculum training trajectory in a Logit-normal parameterized sampling space to reconcile training coverage with inference consistency for streaming video diffusion, reporting FVD improvements.

  5. Wonder: Video World Model Done Better

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Wonder generates minute-scale, real-time camera-controllable video worlds from a single image or video at 16 FPS, using a rendered coordinate-field control signal, sparse full-fidelity memory, and stage-specialized di...

  6. Light Forcing: Accelerating Autoregressive Video Diffusion via Sparse Attention

    cs.CV 2026-02 conditional novelty 6.0 of 10

    Sparse attention with chunk-aware sparsity growth and hierarchical frame/block selection accelerates autoregressive video diffusion at ~1.3x with VBench quality on par with dense attention.

  7. End-to-End Training for Autoregressive Video Diffusion via Self-Resampling

    cs.CV 2025-12 conditional novelty 6.0 of 10

    Resampling Forcing trains autoregressive video diffusion models on self-resampled degraded histories with a causal mask, achieving stable long-horizon generation without a teacher or discriminator.

  8. LoViC: Efficient Long Video Generation with Context Compression

    cs.CV 2025-07 conditional novelty 6.0 of 10

    LoViC uses FlexFormer, a single-query-token Q-Former with interpolated rotary positional encoding, to compress long video-text context for efficient long-video generation.

  9. CDP: Towards Robust Autoregressive Visuomotor Policy Learning via Causal Diffusion

    cs.CV 2025-06 conditional novelty 6.0 of 10

    Causal Diffusion Policy adds historical action conditioning and attention cache sharing to diffusion-based robot policies, improving success rates on most tested manipulation tasks under degraded observations.

  10. Video World Models with Long-term Spatial Memory

    cs.CV 2025-06 conditional novelty 6.0 of 10

    An autoregressive video world model with a persistent static point-cloud spatial memory and sparse episodic keyframes improves revisit consistency over point-cloud-conditioned baselines.

  11. Long-Context State-Space Video World Models

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A hybrid state-space and local-attention architecture gives autoregressive video diffusion models long-term spatial memory with constant per-frame inference cost, demonstrated on Maze and Minecraft.

  12. MiniWorld: Democratizing the Training of Video World Models from Scratch

    cs.CV 2026-08 conditional novelty 5.0 of 10

    A compact open-source recipe trains action-conditioned streaming video world models from scratch on one 8-GPU server, using a block-causal DiT with Diffusion Forcing style noise schedules and a rolling KV cache.

Reference graph

Works this paper leans on

20 extracted references · 12 canonical work pages · cited by 12 Pith papers

  1. [4]

    Flowzero: Zero-shot text-to-video synthesis with llm-driven dynamic scene syntax

    Lu, Y ., Zhu, L., Fan, H., and Yang, Y . Flowzero: Zero-shot text-to-video synthesis with llm-driven dynamic scene syntax. arXiv preprint arXiv:2311.15813,

    arXiv

  2. [5]

    Videofusion: Decomposed diffusion models for high-quality video generation

    Luo, Z., Chen, D., Zhang, Y ., Huang, Y ., Wang, L., Shen, Y ., Zhao, D., Zhou, J., and Tan, T. Videofusion: Decomposed diffusion models for high-quality video generation. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10209–10218. IEEE Computer Society,

    work page 2023

  3. [6]

    Stylegan- v: A continuous video generator with the price, image quality and perks of stylegan2

    Skorokhodov, I., Tulyakov, S., and Elhoseiny, M. Stylegan- v: A continuous video generator with the price, image quality and perks of stylegan2. 2022 ieee. In CVPR, pp. 3616–3626,

    work page 2022

  4. [9]

    Gen-l-video: Multi-text to long video generation via temporal co-denoising

    Wang, F.-Y ., Chen, W., Song, G., Ye, H.-J., Liu, Y ., and Li, H. Gen-l-video: Multi-text to long video generation via temporal co-denoising. arXiv preprint arXiv:2305.18264, 2023a. Wang, J., Yuan, H., Chen, D., Zhang, Y ., Wang, X., and Zhang, S. Modelscope text-to-video technical report. arXiv preprint arXiv:2308.06571, 2023b. Wang, X., Yuan, H., Zhang,...

    arXiv

  5. [10]

    Live2diff: Live stream translation via uni-directional attention in video diffusion models

    Xing, Z., Fox, G., Zeng, Y ., Pan, X., Elgharib, M., Theobalt, C., and Chen, K. Live2diff: Live stream translation via uni-directional attention in video diffusion models. arXiv preprint arxiv:2407.08701,

    arXiv

  6. [12]

    Detailed Training Objectives Recall that (cf

    B. Detailed Training Objectives Recall that (cf. Sec. 3.2 in the main text) existing diffusion models (Ho et al., 2020; Nichol & Dhariwal, 2021; Peebles & Xie,

    work page 2020

  7. [13]

    (6) Since q and pθ are both Gaussian, DKL is determined by the mean µθ and covariance Σθ

    are trained with the variational lower bound of z0’s log-likelihood, formulated as Lvlb(θ) = − log pθ(z0|z1) + X t DKL (q(zt−1|zt, z0)∥pθ(zt−1|zt)). (6) Since q and pθ are both Gaussian, DKL is determined by the mean µθ and covariance Σθ. By re-parameterizing µθ as a noise prediction network ϵθ and fixing Σθ as a constant variance schedule (Ho et al., 202...

    work page 2020

  8. [15]

    UCF101 (Soomro et al., 2012)

    and for fair comparisons, we ran- domly selected a caption for each video and generated 2990 videos for evaluation. UCF101 (Soomro et al., 2012). As it only contains label names, we employed the descriptive text prompts from PY- oCo (Ge et al., 2023), and generated 2048 samples with uniform distribution for each category following (He et al., 2022; Ge et ...

    work page 2012

Show all 20 references
  1. [16]

    the first AR step

    We evaluated the FVD scores of the generated 16-frame chunk from each AR step w.r.t. the first AR step. Each model generated 512 videos for FVD calculation. E. More Experiment Results In Figure 10 and Figure 11, we show more qualitative exam- ples from GenLV (Wang et al., 2023...

    2025

  2. [18]

    VBench is pri- marily designed for text-to-video evaluation

    benchmark. VBench is pri- marily designed for text-to-video evaluation. For our assess- ment, we selected four metrics: aesthetic quality, imaging quality, motion smoothness, and temporal flickering. The first two measure spatial (appearance) quality, and the last two assess t...

    2025

  3. [20]

    Further research on approaches addressing quality degradation is warranted and may hold potential significance for long-term video generation. 15 Title Suppressed Due to Excessive Size Ca2-VDM OS-Ext Ca2-VDM OS-Ext Ca2-VDM OS-ExtCa2-VDM OS-Ext Figure 12: Comparison between OS-...

    2020

  4. [256]

    Method Aesthetic Imaging Motion Temporal Quality Quality Smoothness Flickering OS-Ext 44.39 50.74 98.93 98.57 Ca2-VDM 44.30 50.55 97.59 97.14 OS-Ext baseline

    Both models were evaluated with Pmax = 25 and 6 autoregression steps. Method Aesthetic Imaging Motion Temporal Quality Quality Smoothness Flickering OS-Ext 44.39 50.74 98.93 98.57 Ca2-VDM 44.30 50.55 97.59 97.14 OS-Ext baseline. As a result, they show comparable visual quality...

    2024

  5. [2012]

    Fvd: A new metric for video generation

    Unterthiner, T., van Steenkiste, S., Kurach, K., Marinier, R., Michalski, M., and Gelly, S. Fvd: A new metric for video generation. In ICLR 2019 Workshop DeepGenStruct ,

    2019

  6. [2015]

    Denoising diffusion implicit models

    Song, J., Meng, C., and Ermon, S. Denoising diffusion implicit models. In ICLR, 2021a. Song, Y ., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Er- mon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. In ICLR, 2021b. Soomro, K., Zam...

    arXiv

  7. [2019]

    We followed prior works (Blattmann et al., 2023b; Ge et al., 2022; Ren et al.,

    measures the similarity between generated and real videos based on the distributions on the feature space. We followed prior works (Blattmann et al., 2023b; Ge et al., 2022; Ren et al.,

    2022

  8. [2020]

    Adding conditional control to text-to-image diffusion models

    11 Title Suppressed Due to Excessive Size Zhang, L., Rao, A., and Agrawala, M. Adding conditional control to text-to-image diffusion models. In CVPR, pp. 3836–3847, 2023a. Zhang, S., Wang, J., Zhang, Y ., Zhao, K., Yuan, H., Qin, Z., Wang, X., Zhao, D., and Zhou, J. I2vgen-xl:...

    arXiv

  9. [2021]

    A white tiger in a zoo swimming in a lake…

    with 100 steps. For text-to-video, we set the classifier-free guid- ance scale as 7.5. D. Evaluation Details D.1. Datasets MSR-VTT (Xu et al., 2016). we used its official test split which contains 2990 videos, with 20 manually annotated 3OS-Fix converges faster than Ca2-VDM si...

    2016

  10. [2022]

    La- tent video diffusion models for high-fidelity long video generation

    He, Y ., Yang, T., Zhang, Y ., Shan, Y ., and Chen, Q. La- tent video diffusion models for high-fidelity long video generation. arXiv preprint arXiv:2211.13221,

    arXiv

  11. [2023]

    Streamdiffusion: A pipeline-level solu- tion for real-time interactive generation

    Kodaira, A., Xu, C., Hazama, T., Yoshimoto, T., Ohno, K., Mitsuhori, S., Sugano, S., Cho, H., Liu, Z., and Keutzer, K. Streamdiffusion: A pipeline-level solu- tion for real-time interactive generation. arXiv preprint arXiv:2312.12491,

    arXiv

  12. [2024]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Blattmann, A., Dockhorn, T., Kulal, S., Mendelevitch, D., Kilian, M., Lorenz, D., Levi, Y ., English, Z., V oleti, V ., Letts, A., et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023a. Blattmann, A., Rom...

    arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.