Towards Chunk-Wise Generation for Long Videos

Ser-Nam Lim; Siyang Zhang

REVIEW 3 major objections 5 minor 30 references

Towards Chunk-Wise Generation for Long Videos

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that selecting each chunk's initial noise with a cheap k-step denoising probe mitigates the compounding quality loss in chunk-by-chunk long video generation.

desk verdict A practical, honestly-scoped k-step noise search for chunk-by-chunk long video generation, held back by under-reported experiment details and an unvalidated ranking assumption. read the letter →

arxiv 2411.18668 v1 pith:XQQ7IGJF submitted 2024-11-27 cs.CV

Siyang Zhang , Ser-Nam Lim This is my paper · ORCID

classification cs.CV

keywords longvideogenerationdiffusionmodelsimage-to-videoautoregressivechunk-by-chunkinitialnoiseselectionk-stepsearchtraining-freeinference

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large diffusion video models are typically trained on fixed, short clips, so generating a long video in one pass exceeds GPU memory and drifts out of distribution. This paper studies a memory-feasible alternative: generate the video chunk by chunk, with an image-to-video model that each time animates the last frame of the previous chunk. The paper's central claim is that in this autoregressive setup the initial noise sampled for each chunk strongly determines whether that chunk stays clean, and that bad noises cause errors to compound across chunks. To exploit this, it proposes a k-step search that denoises several candidate noises for only k steps, scores the resulting rough videos by their minimum CLIP cosine similarity to the guide frame, and fully denoises only the best candidate. The experiments show the search meaningfully reduces degradation for smaller UNet-based I2V models, while the largest DiT models are stable enough that naive chunk-by-chunk generation already works.

What carries the argument

The central mechanism is the k-step search (Algorithm 1) combined with the train-inference noise discrepancy that motivates it. During training a diffusion model sees noisy latents of the form $z_t = \sqrt{\bar\alpha_t} z_0 + \sqrt{1-\bar\alpha_t}\epsilon$, while at inference it is handed a pure Gaussian $z_T \sim \mathcal{N}(0,I)$; these distributions are close but not identical, so some initial noises lie outside the region the model learned to denoise well. The k-step search compensates by trying $m$ noises, denoising each for $k$ steps, and scoring the rough result with $\min_j \mathrm{CosSim}(\mathrm{CLIP}(I_{\text{guide}}, V[j]))$ over frames $j$. The low-step probe is cheap, and the chosen noise then runs the full denoising schedule.

What would settle it

Run the k-step search with, say, $m=10$ candidate noises on a set of 100 guide images and compare, per image, the k-step ranking against the ranking of the same noises after full denoising measured by VBench subject consistency. If the k-step argmax matches the full-step argmax at no better than chance, or if the k-step winner's VBench score is not reliably above a randomly chosen noise, the central claim collapses.

Watch

Extended reading notes

Core claim

The paper establishes that, for a given image-to-video diffusion model and conditioning frame, different Gaussian initial noises lead to markedly different chunk quality (Table 1), and that the worst of these noises is what drives the cumulative worsening effect in chunk-by-chunk generation. Its proposed cure is the k-step search (Algorithm 1): sample m noises, run the denoiser for k steps on each, score the suboptimal videos with the minimum CLIP cosine similarity between their frames and the guide image, then fully denoise the noise that scores highest. The discovery is that such a short probe, with k as small as 8, is enough to rank noises well enough to select a good one, so the per-chunk error never gets a foothold. This is demonstrated on four I2V models, with the largest gains on the smaller UNet-based models and only marginal gains on large DiT models.

Load-bearing premise

That a video denoised for only k steps ranks candidate initial noises in the same order of quality as a fully denoised video, and that the minimum CLIP similarity of those k-step frames to the guide image is a trustworthy proxy for that quality.

Editorial extensions

If this is right

Chunk-by-chunk generation with an off-the-shelf I2V model can produce long videos without out-of-memory failures, and the k-step search keeps quality degradation in check for smaller models.
Small UNet-based I2V models, which are fast but noise-sensitive, benefit most from k-step selection; large DiT models need little intervention.
The overhead of k-step search is modest: with $m=10$ candidates and $k=8$ steps, the extra cost is 80 steps per chunk compared with 500 for brute force over 10 candidates when full sampling is 50 steps.
k-step search is training-free and works with any I2V model whose scheduler allows a reduced step count, as long as the initial noise can be fixed by a random seed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The ranking assumption between k-step and full-step quality is directly testable: on any I2V model, one can compute the fraction of times the k-step argmax matches the full-step argmax across many guide images; if that fraction is near chance for small k, the probe would need more steps.
The minimum-CLIP-similarity score may implicitly penalize motion and favor static or semantically close frames, so a temporal-aware selection metric could change which noise wins and further improve motion quality.
Because the paper notes error never fully disappears, a closed-loop monitor that re-runs the k-step probe every few chunks and resamples when the score drops could push generation beyond the demonstrated 20 chunks.
k-step search could be combined with noise-refinement methods that alter low-frequency components, turning pure random search into a two-stage candidate generation that might outperform either alone.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The genuinely new piece here is the k-step noise search: sample m noises per chunk, denoise each only k steps, score the half-baked videos, and fully denoise the winner. That is a sensible, training-free extension of best-of-N sampling to autoregressive I2V generation, and the paper is honest about what it does not solve. The authors clearly state they cannot claim infinite-length generation and that their method still relies on random sampling rather than explicit noise refinement. For a short empirical paper, that is the right scope.

What the paper does well is motivate the problem. Table 1 concretely shows that initial noise causes large quality variance in small I2V models, and the degradation pattern in chunk-by-chunk generation is real and visible. The VBench improvements for StableVideoDiffusion and ConsistI2V are consistent across four metrics and across chunk counts up to 20. That is credible evidence that the search helps those models. The negative result for large DiT models is also useful and matches the intuition that they are more noise-robust.

Now the soft spots, in proportion. The biggest one is that the core ranking assumption is not directly tested. Section 3.4 asserts that k-step evaluation indicates full-denoise quality, but Figure 6 only shows that k-step outputs become visually closer to full-step outputs as k grows. It does not show that Eq. 4 at k=8 ranks candidate noises the same way full denoising would. Without a rank-correlation or selection-agreement check, the improvement in Tables 2 and 3 could come partly from the extra compute spent on m candidates rather than from the proxy actually identifying good noises. This is a fixable gap: the authors could run full denoising on the same m candidates for a few prompts and report how often the k-step pick matches the full-step pick.

Second, the selection metric itself deserves scrutiny. Eq. 4 rewards frames that stay close to the guide image in CLIP space, which overlaps with VBench's Background Consistency and could bias toward static or near-static chunks. The paper reports no motion-dynamics metric and no human evaluation, so a static-bias story is not ruled out. I would not call this a fatal flaw, because the authors include Motion Smoothness via AMT, but static chunks also score well there. A simple motion-magnitude statistic or a user study would address it.

Third, the experimental reporting is thin: m is never specified, the number of test prompts is not stated, and Tables 2 and 3 have no variance or significance. These are mundane but necessary details. FreeInit and FrameInit are discussed but not run as baselines, which would have made the cost-quality comparison much stronger.

Who is this for? Anyone working on training-free long-video generation with small I2V models. It deserves a serious referee: the idea is simple, useful, and honestly described, but the evidence needs the missing controls and a direct validation of the ranking assumption before I would rely on the claimed mechanism.

Referee Report

3 major / 5 minor

Summary. The paper addresses long video generation by autoregressive chunk-by-chunk inference with pretrained image-to-video (I2V) diffusion models. The authors observe that the initial noise strongly influences per-chunk quality and that bad noises cause cumulative degradation. They propose a k-step search: for each chunk, m candidate noises are partially denoised for k steps, scored by the minimum CLIP cosine similarity between the frames and the guide image (Eq. 4), and the winner is fully denoised. Experiments on StableVideoDiffusion, ConsistI2V, OpenSoraPlanV1.3.0, and CogVideoX report VBench improvements for the smaller models. The paper also provides an analysis of noise sensitivity and a comparison of naive versus k-step chunk-wise generation.

Significance. If the central claim is validated, the method is practically useful: it is training-free, compatible with existing I2V models, and could reduce cumulative error accumulation in chunk-wise long-video generation, especially for lightweight models. The paper also contributes a useful empirical analysis of initial-noise sensitivity across four models. However, the validation is currently incomplete: the key proxy assumption (k-step scores rank noises like full denoising) is not directly tested, and the selection metric may induce a static-frame bias that inflates the reported VBench gains. The evidence base is also thin (m and prompt counts unspecified, no variance or significance in the main tables), so the favorable results should be treated as preliminary.

major comments (3)

[Section 3.4, Figure 6] The load-bearing assumption that 'given the same initial noise, the proposed k-step evaluation is able to indicate the overall quality of an otherwise fully denoised output' is not directly tested. Figure 6 only plots cosine similarity between k-step and 50-step outputs as k increases; it does not show that the ordering of candidate noises by Eq. (4) at k=8 matches the ordering by full-denoised quality. Without a rank-correlation or selection-accuracy analysis, the improvements in Tables 2 and 3 could be attributed to the extra computation (m times k additional denoising steps per chunk) rather than to the proposed proxy. Please report the agreement between k-step selected noises and best full-denoised noises, e.g., by comparing Eq. (4) scores at k=8 with full-denoised VBench scores or human ratings over a set of candidates.
[Section 3.5, Eq. (4)] The selection metric, minimum CLIP cosine similarity to the guide image, may favor static or near-static chunks, because a chunk that stays close to the guide trivially maximizes this score. The reported VBench metrics (Subject Consistency, Background Consistency, Temporal Flickering, Motion Smoothness) also reward static content: AMT reconstructs static frames perfectly, and pixel-level flickering is minimized. The paper reports no motion-dynamics metric or human evaluation, so the observed improvements in Table 2 may reflect a static bias rather than mitigation of cumulative degradation. Please add a quantitative motion metric (e.g., average optical flow magnitude, frame-to-frame difference, or a video motion score) and/or a human preference study, and report qualitative examples showing that motion is preserved.
[Section 4.1, Tables 2 and 3] Several experimental details needed for reproducibility and significance are missing. The number of candidate noises m used in Algorithm 1 is never specified, and the number of prompts per VBench category is not stated. Tables 2 and 3 report no variance or significance information, unlike Table 1 which reports min/max/std; therefore it is unclear whether the reported improvements are statistically reliable. Please specify m, the prompt count, and report standard deviations or confidence intervals across prompt instances, especially for the small-model improvements that support the central claim.

minor comments (5)

[Section 4.1] There is a typo: 'EulerDiscreate' should be 'EulerDiscrete'.
[Section 4.1] The sentence 'we set manual seed before every k-step and full-step sampling' should read 'we set a manual seed'.
[Algorithm 1] The variable names 'vs' and 'vcand' are not defined in the pseudocode; consider renaming them to 'output_videos' and 'candidate_videos' for clarity.
[Section 3.5, Eq. (4)] The equation numbering is inconsistent: the text says 'Eq. 4' but the equation is not numbered in the text; please add equation numbers consistently.
[Section 4.4.2] The choice k=8 is justified only by a qualitative observation from Figure 6; consider reporting quantitative results, such as selection accuracy at different k values, to support this hyperparameter choice.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the k-step selection heuristic is tested against external VBench metrics and does not reduce to its own inputs.

full rationale

The paper's central claim is that k-step denoising can rank initial noises so that selecting the best candidate mitigates chunk-by-chunk degradation. This is an empirical heuristic, not a derivation. The selection score (Eq. 4) is the minimum CLIP cosine similarity between k-step frames and the guide image, while the reported improvements are measured on VBench's Subject Consistency, Background Consistency, Temporal Flickering, Motion Smoothness, and Aesthetic Quality. None of these evaluation metrics equals the selection score by construction: Background Consistency uses CLIP cosine similarity between consecutive frames, not between frames and the guide image, and the other metrics use DINO features, pixel differences, AMT reconstruction error, or a LAION aesthetic predictor. The overlap in CLIP features is a potential proxy alignment concern, but it is not a circular reduction because the selected quantity and the reported quantity are different functions. The k=8 hyperparameter is chosen empirically from Figure 6, but it is an ablation choice, not a fitted parameter relabeled as a prediction, and the improvements in Tables 2 and 3 are reported on external VBench metrics. The paper contains no load-bearing self-citations: the cited train-inference discrepancy (FreeInit) and schedulers are external prior work. The limitation section also explicitly states that the method cannot guarantee infinite-length videos, which is inconsistent with a claim that the result is forced by definition. Unvalidated premises (e.g., that k-step rankings match full-step quality rankings, or that CLIP-to-guide similarity does not bias selection toward static chunks) are correctness risks, not circularity. No step in the derivation chain reduces to its own inputs, so the appropriate finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method rests on standard diffusion math, a domain assumption about repeated last-frame conditioning, and two ad hoc assumptions that are central to the search: that k-step outputs predict full-step quality, and that CLIP similarity to the guide frame is a good quality proxy. There are no invented physical entities. The main free parameters are k, m, the full step count, and the selection rule.

free parameters (4)

k (number of denoising steps for noise evaluation) = 8
Chosen empirically in Sec 4.1 and 4.4.2 as a trade-off between proxy quality and cost; no theoretical derivation.
m (number of candidate noises per chunk) = not stated for main experiments; example m=10 in Sec 3.4
Controls both search quality and compute; the missing value makes the reported results hard to replicate exactly.
full denoising step count s = 50 for all main experiments
Set in Sec 4.1; this choice affects absolute metric values but is consistent across baseline and k-step runs.
CLIP minimum similarity selection rule = min over frames, Eq 4
The selection rule is chosen heuristically; the paper does not validate that this metric selects the best full-quality videos.

assumptions (5)

standard math The diffusion forward process and DDIM-style sampling equations (Eqs 1-2) correctly describe the pretrained video diffusion models.
Standard background from [7] and [23], used throughout the method section.
domain assumption An I2V model conditioned on the previous chunk's last frame can be applied repeatedly to create longer videos.
The chunk-by-chunk pipeline (Sec 3.2) assumes the last frame is a sufficient conditioning signal for the next chunk.
domain assumption The train-inference discrepancy of initial pure Gaussian noise causes per-chunk quality variation.
The paper argues in Sec 3.3, following FreeInit [27], that pure Gaussian noise differs from training-time noisy latents, which explains why noise quality varies.
ad hoc to paper A k-step denoised video preserves enough structure to evaluate the quality of an initial noise.
The central unproven assumption of the method, stated in Sec 3.4 and only indirectly supported by Figure 6.
ad hoc to paper Minimum CLIP cosine similarity to the guide image is a valid proxy for chunk quality.
Chosen empirically in Sec 3.5; no evidence that it correlates with human preference or with all five VBench dimensions used.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Towards Chunk-Wise Generation for Long Videos." pith.science (2026). https://pith.science/paper/XQQ7IGJF

@misc{pith2026241118668,
  author       = {Pith},
  title        = {Pith review of: Towards Chunk-Wise Generation for Long Videos},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XQQ7IGJF}},
  note         = {Machine review of arXiv:2411.18668}
}

abstract

Generating long-duration videos has always been a significant challenge due to the inherent complexity of spatio-temporal domain and the substantial GPU memory demands required to calculate huge size tensors. While diffusion based generative models achieve state-of-the-art performance in video generation task, they are typically trained with predefined video resolutions and lengths. During inference, a noise tensor with specific resolution and length should be specified at first, and the model will perform denoising on the entire video tensor simultaneously, all the frames together. Such approach will easily raise an out-of-memory (OOM) problem when the specified resolution and/or length exceed a certain limit. One of the solutions to this problem is to generate many short video chunks autoregressively with strong inter-chunk spatio-temporal relation and then concatenate them together to form a long video. In this approach, a long video generation task is divided into multiple short video generation subtasks, and the cost of each subtask is reduced to a feasible level. In this paper, we conduct a detailed survey on long video generation with the autoregressive chunk-by-chunk strategy. We address common problems caused by applying short image-to-video models to long video tasks and design an efficient $k$-step search solution to mitigate these problems.

Figures

Figures reproduced from arXiv: 2411.18668 by the authors.

**Figure 1.** An overview of the pipeline of autoregressive chunk-by-chunk long video generation. Each time an Image-to-Video(I2V) model takes in a guide image as condition and generate a short video chunk. Then, the model will take the last frame of collected videos, and predict the next video chunk. Abstract Generating long-duration videos has always been a significant challenge due to the inherent complexity of spatiotempora… view at source ↗

**Figure 2.** Examples of degradation effect on video quality in chunk-bychunk video generation by StableVideoDiffusion [1] and ConsistI2V [20]. For each guide image, we perform naive chunk-by-chunk generation (top row) and k-step search generation (bottom row). The model created some artifacts in each chunk, and the cumulated effect will at last destroy the long video as the number of chunks increases. Our k-step search helps t… view at source ↗

**Figure 3.** Examples of long videos generated by OpenSoraPlanV1.3.0 and CogVideoX. For each guide image, we perform naive chunk-by-chunk generation (top row) and k-step search generation (bottom row). These models are more robust to initial noise. There are works such as FreeInit [27] and FrameInit [20] addressing this problem through an expensive and timeconsuming procedure that includes refining initial noises by Fast-Fourie… view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: k-step search: we first prepare m initial noises and then for each of them, call the base I2V model to only denoise for k steps, resulting in k suboptimal short video candidates. After that, we explicitly evaluate the k video candidates and find the one with the best q…

**Figure 5.** Figure 5: Examples of sampling results with the same conditioning [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

**Figure 6.** Figure 6: For each model, we apply its recommended scheduler and calculate the cosine similarity between a video generated after k steps and a video generated with 50 steps with the same noise and conditioning inputs. the even-number-frames, after which a Mean Absolute Error i…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 9 canonical work pages

[1]

Stable video diffusion: Scaling latent video diffusion models to large datasets

Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 2, 3, 6

arXiv 2023
[2]

Emerg- ing properties in self-supervised vision transformers

Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 7

2021
[3]

Videocrafter2: Overcoming data limitations for high-quality video diffu- sion models

Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffu- sion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7310– 7320, 2024. 3

2024
[4]

Control-a- video: Controllable text-to-video diffusion models with mo- tion prior and reward feedback learning, 2024

Weifeng Chen, Yatai Ji, Jie Wu, Hefeng Wu, Pan Xie, Ji- ashi Li, Xin Xia, Xuefeng Xiao, and Liang Lin. Control-a- video: Controllable text-to-video diffusion models with mo- tion prior and reward feedback learning, 2024. 2

work page 2024
[5]

Generative adversarial nets

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014. 1, 3

work page 2014
[6]

Animatediff: Animate your personalized text- to-image diffusion models without specific tuning

Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023. 3

arXiv 2023
[7]

Denoising dif- fusion probabilistic models

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 1, 3, 4, 5, 8

work page 2020
[8]

Vbench: Comprehensive bench- mark suite for video generative models

Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive bench- mark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21807–21818, 2024. 6, 7

2024

Show all 30 references

[9]

Elucidating the design space of diffusion-based generative models

Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35:26565–26577, 2022. 5, 6, 8

2022
[10]

Fifo-diffusion: Generating infinite videos from text without training

Jihwan Kim, Junoh Kang, Jinyoung Choi, and Bohyung Han. Fifo-diffusion: Generating infinite videos from text without training. arXiv preprint arXiv:2405.11473, 2024. 3

2024 arXiv
[11]

Videopoet: A large language model for zero-shot video gen- eration

Dan Kondratyuk, Lijun Yu, Xiuye Gu, Jos ´e Lezama, Jonathan Huang, Grant Schindler, Rachel Hornung, Vigh- nesh Birodkar, Jimmy Yan, Ming-Chang Chiu, et al. Videopoet: A large language model for zero-shot video gen- eration. arXiv preprint arXiv:2312.14125, 2023. 1, 2

2023 arXiv
[12]

Open-sora-plan, 2024

PKU-Yuan Lab and Tuzhan AI etc. Open-sora-plan, 2024. 2, 3, 6

2024
[13]

aesthetic-predictor

LAION-AI. aesthetic-predictor. https://github. com/LAION-AI/aesthetic-predictor, 2022. 7

2022
[14]

Amt: All-pairs multi-field transforms for efficient frame interpolation

Zhen Li, Zuo-Liang Zhu, Ling-Hao Han, Qibin Hou, Chun- Le Guo, and Ming-Ming Cheng. Amt: All-pairs multi-field transforms for efficient frame interpolation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 9801–9810, 2023. 7

2023
[15]

Pseudo numerical methods for diffusion models on manifolds.arXiv preprint arXiv:2202.09778, 2022

Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds.arXiv preprint arXiv:2202.09778, 2022. 5, 8

2022 arXiv
[16]

Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps

Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems , 35:5775–5787,
[17]

Scalable diffusion models with transformers

William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 4195–4205,
[18]

Learning transferable visual models from natural language supervi- sion

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

2021
[19]

Exploring the limits of transfer learning with a unified text-to-text transformer

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020. 3

2020
[20]

Consisti2v: Enhanc- ing visual consistency for image-to-video generation

Weiming Ren, Huan Yang, Ge Zhang, Cong Wei, Xinrun Du, Wenhao Huang, and Wenhu Chen. Consisti2v: Enhanc- ing visual consistency for image-to-video generation. arXiv preprint arXiv:2402.04324, 2024. 2, 4, 6

2024 arXiv
[21]

High-resolution image synthesis with latent diffusion models

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 6

2022
[22]

U- net: Convolutional networks for biomedical image segmen- tation

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, pa...

2015
[23]

Denoising diffusion implicit models

Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 5, 6

2010 arXiv
[24]

Autoregressive model beats diffusion: Llama for scalable image generation

Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525, 2024. 1

2024 arXiv
[25]

Modelscope text-to-video technical report

Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Modelscope text-to-video technical report. arXiv preprint arXiv:2308.06571, 2023. 3

2023 arXiv
[26]

Nuwa-infinity: Autoregressive over autoregressive generation for infinite visual synthesis

Chenfei Wu, Jian Liang, Xiaowei Hu, Zhe Gan, Jianfeng Wang, Lijuan Wang, Zicheng Liu, Yuejian Fang, and Nan Duan. Nuwa-infinity: Autoregressive over autoregressive generation for infinite visual synthesis. arXiv preprint arXiv:2207.09814, 2022. 1, 2

2022 arXiv
[27]

Freeinit: Bridging initialization gap in video dif- fusion models

Tianxing Wu, Chenyang Si, Yuming Jiang, Ziqi Huang, and Ziwei Liu. Freeinit: Bridging initialization gap in video dif- fusion models. In European Conference on Computer Vision, pages 378–394. Springer, 2025. 2, 4, 5

2025
[28]

Cogvideox: Text-to-video diffusion models with an expert transformer

Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024. 2, 3, 6

2024 arXiv
[29]

Magvit: Masked generative video transformer

Lijun Yu, Yong Cheng, Kihyuk Sohn, Jos ´e Lezama, Han Zhang, Huiwen Chang, Alexander G Hauptmann, Ming- Hsuan Yang, Yuan Hao, Irfan Essa, et al. Magvit: Masked generative video transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p...

2023
[30]

Language model beats diffusion–tokenizer is key to visual generation

Lijun Yu, Jos ´e Lezama, Nitesh B Gundavarapu, Luca Ver- sari, Kihyuk Sohn, David Minnen, Yong Cheng, Vighnesh Birodkar, Agrim Gupta, Xiuye Gu, et al. Language model beats diffusion–tokenizer is key to visual generation. arXiv preprint arXiv:2310.05737, 2023. 1, 2, 3

arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Stable video diffusion: Scaling latent video diffusion models to large datasets

Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 2, 3, 6

arXiv 2023

[2] [2]

Emerg- ing properties in self-supervised vision transformers

Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 7

2021

[3] [3]

Videocrafter2: Overcoming data limitations for high-quality video diffu- sion models

Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffu- sion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7310– 7320, 2024. 3

2024

[4] [4]

Control-a- video: Controllable text-to-video diffusion models with mo- tion prior and reward feedback learning, 2024

Weifeng Chen, Yatai Ji, Jie Wu, Hefeng Wu, Pan Xie, Ji- ashi Li, Xin Xia, Xuefeng Xiao, and Liang Lin. Control-a- video: Controllable text-to-video diffusion models with mo- tion prior and reward feedback learning, 2024. 2

work page 2024

[5] [5]

Generative adversarial nets

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014. 1, 3

work page 2014

[6] [6]

Animatediff: Animate your personalized text- to-image diffusion models without specific tuning

Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023. 3

arXiv 2023

[7] [7]

Denoising dif- fusion probabilistic models

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 1, 3, 4, 5, 8

work page 2020

[8] [8]

Vbench: Comprehensive bench- mark suite for video generative models

Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive bench- mark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21807–21818, 2024. 6, 7

2024