Pith. sign in

REVIEW 4 major objections 5 minor 30 references

Minute-Long Videos with Dual Parallelisms

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read One inference strategy generates minute-long videos 6.5 times faster across 8 GPUs.

desk verdict Solid efficiency contribution; quality claims need to catch up with the efficiency evidence before this can be taken at face value. read the letter →

arxiv 2505.21070 v2 pith:5KXJGBBL submitted 2025-05-27 cs.CV

Zeqing Wang , Bowen Zheng , Xingyi Yang , Zhenxiong Tan , Yuecong Xu , Xinchao Wang This is my paper
classification cs.CV
keywords videodiffusiontransformerdistributedinferencepipelineparallelismsequencelonggenerationblock-wisedenoisingfeaturecache
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that long video generation with diffusion transformers need not be serialized across devices. It proposes DualParal, a distributed inference strategy that splits both the frame sequence and the model layers across GPUs, using a block-wise denoising schedule in which different temporal blocks are denoised at different noise levels. A FIFO queue feeds blocks through a device pipeline in reverse order, so each GPU can compute and communicate asynchronously rather than waiting for global synchronization. The result, if correct, is that a standard pretrained video diffusion model can generate 1,025-frame videos with up to 6.54x lower latency and 1.48x lower memory than existing distributed methods on 8xRTX 4090 GPUs, with video quality comparable to existing baselines.

What carries the argument

The central object is the FIFO block queue combined with a device pipeline. Blocks of frame latents are ordered with monotonically decreasing noise levels from tail to head; each diffusion step appends a new noisy block at the tail, pops a clean block at the head, and denoises the queue in reverse order across GPUs that each hold a consecutive subset of DiT layers. This block-wise denoising is what dissolves the conflict between sequence parallelism and pipeline parallelism, and the feature cache and coordinated noise initialization patch the two failure modes the schedule creates, namely redundant communication and repetitive-noise artifacts.

What would settle it

Generate a 1,025-frame video with DualParal on the Wan2.1-1.3B model and inspect frames at block boundaries (every eight latent frames) for abrupt jumps in subject identity, background, or motion. If visible seams or duplicated content persist even with the coordinated noise initialization and NumC=8, the central assumption is false. A quantitative check would compare VBench subject-consistency and temporal-flickering scores of DualParal at 257 frames against the full-sequence baseline at 129 frames; a collapse on those two metrics would indicate the desynchronized schedule degrades temporal coherence.

Watch

Extended reading notes

Core claim

DualParal establishes that the synchronization requirement of diffusion models, where all frames must share the same noise level at each step, can be replaced by a block-wise denoising schedule without retraining. In this schedule, blocks are queued from almost clean to fully noisy, and each denoising step processes them from tail to head through a model that is itself divided across GPUs. Because adjacent blocks are denoised at neighboring noise levels, outputs can flow asynchronously between devices, eliminating the gather-and-reserialize bottleneck that arises when sequence parallelism and pipeline parallelism are naively combined. Two additions make the scheme practical: a per-GPU key-value cache that reuses self-attention features of the already-processed neighboring block, and a coordinated noise initialization that draws each new block from the full noise space while shuffling to avoid reusing identical noise patterns in concatenated blocks.

Load-bearing premise

A pretrained diffusion transformer can denoise different temporal blocks at different, unsynchronized noise levels and still produce a single temporally coherent video, without any retraining or fine-tuning.

Editorial extensions

If this is right

  • Latency and memory stop growing with total video length: the per-step workload depends on block size, not on how many blocks have been generated, so minute-long or longer videos become feasible on commodity 8-GPU machines.
  • Infinite-length generation is supported directly: new blocks can be appended indefinitely while the first clean block is popped and decoded continuously.
  • The method works on a large DiT-based video generator at 480p and 720p resolution without retraining or fine-tuning, transferring to the model's existing inference settings.
  • The bubble ratio approaches zero as the number of blocks grows, so GPU utilization improves for longer videos rather than degrading.
  • Compared with FIFO, the pipelined model split also reduces peak memory because model weights are distributed across devices rather than replicated on every device.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If desynchronized block-wise denoising is as benign for other DiT video generators as it is for the model tested here, the same scheduling trick may transfer to other large transformer video models; a direct comparison would separate a general principle from model-specific tolerance.
  • The coordinated noise initialization suggests a retraining-free recipe, full noise pool, shuffled assignment, and exclusion of the last few latents, that could also improve other sliding-window long-video methods without their fusion steps.
  • The reported efficiency gain is a steady-state result: warm-up and cool-down bubbles dominate when the number of blocks is small, so the advantage is strongest for genuinely long generations and weaker for short clips.
  • Because the feature cache is applied only to self-attention, models with additional cross-frame interaction mechanisms would need re-measurement of both the cache benefit and the noise-initialization effect.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes DualParal, a distributed inference strategy for DiT-based video diffusion models that combines temporal sequence parallelism with pipeline parallelism over model layers. The method uses a block-wise denoising FIFO queue so that different temporal blocks can be denoised at different, asynchronous noise levels, and augments this with a feature cache that reuses KV features from previously processed blocks and a coordinated noise initialization strategy. Experiments on Wan2.1 report up to 6.54x lower latency and 1.48x lower peak memory at 1,025 frames on 8xRTX 4090 GPUs relative to Ring Attention, with VBench-based quality comparisons reported at 129 and 257 frames. The abstract and conclusion additionally claim fast, artifact-free, and infinitely long video generation.

Significance. If fully validated, DualParal would be a practically useful contribution to long-video generation on commodity multi-GPU setups, and the paper has several real strengths: it directly measures latency and memory against four existing methods, includes ablations of the queue, device pipeline, and feature cache, and provides a project page and code repository. The core idea of resolving the synchronization conflict between sequence and pipeline parallelism through block-wise denoising is reasonable and extends prior work such as FIFO-Diffusion and Video-Infinity. However, the headline quality claim for 1,025-frame videos is not supported by the reported experiments, and there are mathematical and experimental-consistency issues that need to be addressed before the central claims can be accepted.

major comments (4)
  1. [§4.2 / Table 3] The efficiency claim is made for 513 and 1,025 frames in Table 2, but the quality evaluation in Table 3 only covers 129 and 257 frames. No VBench scores, human evaluation, or error bars are reported for 513 or 1,025 frames, which is precisely the regime where the desynchronized block-wise denoising and the stale feature cache are most likely to cause temporal drift or artifacts. Since the abstract promises 'artifact-free and infinitely long' generation, the missing quality evidence at the headline lengths is load-bearing. Please add quantitative quality evaluation at 513 and 1,025 frames, ideally with multiple seeds and variance estimates, or substantially soften the abstract and conclusion claims.
  2. [§3.2] The feature cache reuses Self-Attention KV features from a block at an earlier, noisier denoising state. When denoising block B'_i, the context block B_{i+1} was processed in the previous iteration as part of B'_{i+1}, so the cached K and V correspond to the previous noise level, not the current one. Over long generations this staleness can accumulate and affect temporal coherence. The paper does not analyze this effect or provide an ablation comparing cached KV against freshly recomputed KV at long video lengths. Such an experiment is needed to support the claim that the cache preserves quality.
  3. [§3.4 / Appendix A.3] Equation (3) is based on an algebraic error. The text states that the bubble size equals 1 + 2 + ... + (N-1) + 1 + 2 + ... + N, which evaluates to N^2, not N^2 - N - 1. Consequently, for N=4 the formula gives 11 instead of 16, and the stated 5.2% bubble ratio in Figure 3 does not follow from the written derivation. The asymptotic conclusion that the bubble ratio tends to zero as the number of blocks grows is unaffected, but the quantitative analysis in Section 3.4 and the proof in Appendix A.3 should be corrected.
  4. [§4.2 / Appendix A.5] The quality comparison in Table 3 includes DeepSpeed-Ulysses, but the text and Appendix A.5 state that DeepSpeed-Ulysses cannot run on 8xRTX 4090 GPUs because the number of attention heads is not divisible by 8. The manuscript does not state on how many GPUs DeepSpeed-Ulysses was run for the quality experiments. If it used a different device count than DualParal and the other baselines, the comparison in Table 3 is not properly controlled. Please specify the hardware configuration for each method in the quality evaluation, or match the device counts.
minor comments (5)
  1. [§1] There is a typo in 'Baslines'; it should be 'Baselines'.
  2. [§3.1 / Figure 1] The ordering of noise levels in the queue is described inconsistently: the text says blocks are arranged with 'progressively decreasing noise levels, ranging from 1 to T', while the Figure 1 caption says 'noise levels increasing from tail to head'. Please clarify the naming of indices and the direction of increasing noise.
  3. [§3.3 / Figure 2] The two key observations about noise initialization are supported only by qualitative examples. A quantitative comparison of the four initialization schemes in Figure 2 would make the design choice more convincing.
  4. [§4.2 / Table 3] At 257 frames, DualParal's overall VBench score is 78.17% versus 78.08% for Video-Infinity, a difference of 0.09 percentage points with no error bars or significance testing. The claim of achieving the 'highest overall score' should be tempered, and standard deviations or multiple-seed results should be reported.
  5. [§3.4] The text 'N onBubble Size' appears to be a spacing/formatting error in the denominator of Equation (3).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: efficiency claims are direct measurements against external baselines, and no fitted parameter is renamed as a prediction.

full rationale

The paper's central efficiency claim (6.54x lower latency, 1.48x lower memory at 1025 frames) is an empirical measurement against externally implemented baselines (Ring Attention, DeepSpeed-Ulysses, Video-Infinity, FIFO) on 8xRTX 4090 GPUs; none of those numbers is produced by a parameter fitted inside DualParal, so the comparison is not circular by construction. The block-wise denoising design and the feature cache are approximations justified by the pretrained model's attention structure, not by a self-citation chain, and the coordinated noise initialization is validated qualitatively (Figure 6) rather than asserted as a derived prediction. Author overlap with Video-Infinity exists, but Video-Infinity is used as a benchmark whose results are independently checkable, and its settings are reported (NumC=8 in Table 2, NumC=24 in Table 3), so the overlap is not load-bearing. The most serious weakness is evidentiary, not circular: VBench quality is reported only at 129 and 257 frames, leaving the 513/1025-frame 'artifact-free' claim unmeasured; that is an unsupported extrapolation, not a reduction of the output to the input. The bubble-ratio derivation in Appendix A.3 contains an apparent arithmetic slip (1+...+N-1+1+...+N equals N^2, not N^2-N-1), which is a correctness issue, not a circularity issue. Overall, the derivation chain is self-contained and no 'prediction' is equivalent to its inputs by definition.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central efficiency claim additionally depends on an unchanged pretrained Wan2.1 model and on the empirical observations in Figure 2 about noise spaces. The listed domain assumptions are not proven; they are the cost of applying a pretrained model outside its synchronized-noise training scheme.

free parameters (3)
  • NumB (frames per block) = 8 in main experiments
    Chosen by hand; directly controls pipeline granularity, KV memory, and generation quality.
  • NumC (context frames) = 8 for efficiency runs, 24 for Video-Infinity quality comparison
    Chosen per method and experiment; affects both communication overhead and VBench score.
  • warm-up diffusion steps = 1
    Chosen by hand to establish inter-device connections; not swept or justified by ablation.
assumptions (4)
  • domain assumption Denoising video blocks asynchronously at different noise levels, without synchronizing noise across frames, preserves the pretrained DiT model's generation quality.
    Entered in Section 3.1 when DualParal replaces the standard synchronized denoising loop with a block-wise FIFO schedule. This is the key inductive step; it is validated only empirically.
  • domain assumption Context from the immediately adjacent block, supplied via concatenation or KV cache, is sufficient for temporal coherence in long videos.
    Assumed in Sections 3.2 and 3.3; the paper's global consistency repair works only through noise initialization, not additional context.
  • domain assumption In Wan2.1, only Self-Attention needs inter-frame information; Cross-Attention and FFN can be computed without previous block context.
    Stated in Section 3.2 to justify skipping KV caching for those modules. Architecture-specific and not verified with per-module ablations.
  • domain assumption The pipeline has at least as many blocks in the queue as there are devices (N <= Block_num).
    Assumed in Equation 3 and Appendix A.3 to make the bubble ratio small; the paper acknowledges it is not true for short videos, and their own 301-frame scaling experiments have small Block_num around 9.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Minute-Long Videos with Dual Parallelisms." pith.science (2026). https://pith.science/paper/5KXJGBBL

@misc{pith2026250521070,
  author       = {Pith},
  title        = {Pith review of: Minute-Long Videos with Dual Parallelisms},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5KXJGBBL}},
  note         = {Machine review of arXiv:2505.21070}
}
abstract

Diffusion Transformer (DiT)-based video diffusion models generate high-quality videos at scale but incur prohibitive processing latency and memory costs for long videos. To address this, we propose a novel distributed inference strategy, termed DualParal. The core idea is that, instead of generating an entire video on a single GPU, we parallelize both temporal frames and model layers across GPUs. However, a naive implementation of this division faces a key limitation: since diffusion models require synchronized noise levels across frames, this implementation leads to the serialization of original parallelisms. We leverage a block-wise denoising scheme to handle this. Namely, we process a sequence of frame blocks through the pipeline with progressively decreasing noise levels. Each GPU handles a specific block and layer subset while passing previous results to the next GPU, enabling asynchronous computation and communication. To further optimize performance, we incorporate two key enhancements. Firstly, a feature cache is implemented on each GPU to store and reuse features from the prior block as context, minimizing inter-GPU communication and redundant computation. Secondly, we employ a coordinated noise initialization strategy, ensuring globally consistent temporal dynamics by sharing initial noise patterns across GPUs without extra resource costs. Together, these enable fast, artifact-free, and infinitely long video generation. Applied to the latest diffusion transformer video generator, our method efficiently produces 1,025-frame videos with up to 6.54$\times$ lower latency and 1.48$\times$ lower memory cost on 8$\times$RTX 4090 GPUs.

Figures

Figures reproduced from arXiv: 2505.21070 by the authors.

Figure 1
Figure 1. Overview of DualParal: DualParal partitions video frames into sequential blocks organized in a queue with noise levels increasing from tail to head, and distributes model layers across devices via a device pipeline. By feeding blocks into the pipeline in a reverse order (from tail to head), this block-wise denoising scheme significantly improves efficiency. To further improve performance, DualParal reuses Key-Value … view at source ↗
Figure 2
Figure 2. Examples of four different noise initializations for Wan2.1 model [ [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. presents an example of the pipeline scheduling in DualParal, exhibiting an approximate bubble ratio of 5.2%. Moreover, as Blocknum increases, the bubble ratio approaches 0%, indicating minimal device idle time in the pipeline during long video generation. 1 1 1 1 2 2 2 2 1 1 1 1 3 3 3 3 2 2 2 2 1 1 1 1 4 4 4 2 3 3 3 4 2 2 3 1 1 1 1 4 4 4 4 3 3 3 3 2 2 2 2 1 1 1 1 2 2 2 2 2 2 1 1 1 1 Device 1 Device 2 Device 3 Device… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Scalability analysis in terms of latency and memory cost: (a) and (b) show the scalability of [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Comparison of 257-frame videos. human arm. DualParal, by comparison, consistently delivers superior temporal coherence across both content and motion. Further video examples are shown in Appendix A.1. 4.3 Ablation Parallel ablation [PITH_FULL_IMAGE:figures/full_fig_p0…
Figure 6
Figure 6. Figure 6: Video frames under different condi￾tions: (a) NumC = 0 without noise initialization; (b) NumC = 8 without noise initialization; (c) NumC = 8 with coordinated noise initialization. Through the two key observations for DiT-based video models discussed in Section 3.3, we …
Figure 7
Figure 7. Figure 7: Pipeline schedule of DualParal with N = 4, T = 50, and Blocknum = 3. Blocks are denoised in reverse order, from tail to head in the queue. After diffusion step T, the first clean block is popped from the head, and all remaining blocks shift forward by one position, inc…
Figure 8
Figure 8. Figure 8: will give more details about the pipeline schedule of DualParal with sequential order. By 1 1 1 1 1 1 1 1 2 2 2 2 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 3 3 3 3 Device 1 Device 2 Device 3 Device 4 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 1 1 1 1 2 2 2 2 3 3 3 3…
Figure 9
Figure 9. Figure 9: The influence of different number of blocks on the scaling ability of DualParal. Ex￾periments are conducted on Wan2.1-1.3B (480p) using RTX4090s. To better illustrate the scaling behavior, we normalize each line. The black line represents the ideal scaling trend-propor…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

30 extracted references · 15 canonical work pages

  1. [1]

    Block diffusion: Interpolating between autoregressive and diffusion language models

    Marianne Arriola, Subham Sekhar Sahoo, Aaron Gokaslan, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Justin T Chiu, and V olodymyr Kuleshov. Block diffusion: Interpolating between autoregressive and diffusion language models. InThe Thirteenth International Conference on Learning Representations (ICLR), 2025

    work page 2025

  2. [2]

    All are worth words: A vit backbone for diffusion models

    Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All are worth words: A vit backbone for diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    work page 2023

  3. [3]

    Videocrafter1: Open diffusion models for high-quality video generation, 2023

    Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter1: Open diffusion models for high-quality video generation, 2023

    2023

  4. [4]

    Videocrafter2: Overcoming data limitations for high-quality video diffusion models

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    work page 2024

  5. [5]

    xdit: an inference engine for diffusion transformers (dits) with massive parallelism.arXiv preprint arXiv:2411.01738, 2024

    Jiarui Fang, Jinzhe Pan, Xibo Sun, Aoyu Li, and Jiannan Wang. xdit: an inference engine for diffusion transformers (dits) with massive parallelism.arXiv preprint arXiv:2411.01738, 2024

    arXiv 2024

  6. [6]

    Pipefusion: Patch-level pipeline parallelism for diffusion transformers inference.arXiv preprint arXiv:2405.14430, 2024

    Jiarui Fang, Jinzhe Pan, Jiannan Wang, Aoyu Li, and Xibo Sun. Pipefusion: Patch-level pipeline parallelism for diffusion transformers inference.arXiv preprint arXiv:2405.14430, 2024

    arXiv 2024

  7. [7]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. InAdvances in Neural Information Processing Systems (NeurIPS), 2020

    2020

  8. [8]

    Cogvideo: Large-scale pretraining for text-to-video generation via transformers

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. InThe Eleventh International Conference on Learning Representations (ICLR), 2023

    work page 2023

Show all 30 references
  1. [9]

    Gpipe: Efficient training of giant neural networks using pipeline parallelism

    Yanping Huang, Youlong Cheng, Ankur Bapna, Orhan Firat, Dehao Chen, Mia Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V Le, Yonghui Wu, and zhifeng Chen. Gpipe: Efficient training of giant neural networks using pipeline parallelism. InAdvances in Neural Information Processing Syste...

    2019

  2. [10]

    VBench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Comprehensive benchmark suite for video generative models. I...

    2024

  3. [11]

    Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models.arXiv preprint arXiv:2309.14509, 2023

    Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Shuaiwen Leon Song, Samyam Rajbhandari, and Yuxiong He. Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models.arXiv preprint arXiv:2309.14509, 2023

    2023 arXiv

  4. [12]

    Fifo-diffusion: Generating infinite videos from text without training

    Jihwan Kim, Junoh Kang, Jinyoung Choi, and Bohyung Han. Fifo-diffusion: Generating infinite videos from text without training. InAdvances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  5. [13]

    Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024

    2024 arXiv

  6. [14]

    T2v-turbo: Breaking the quality bottleneck of video consistency model with mixed reward feedback

    Jiachen Li, Weixi Feng, Tsu-Jui Fu, Xinyi Wang, Sugato Basu, Wenhu Chen, and William Yang Wang. T2v-turbo: Breaking the quality bottleneck of video consistency model with mixed reward feedback. In Advances in Neural Information Processing Systems (NeurIPS), 2024. 10

    2024

  7. [15]

    T2v-turbo-v2: Enhancing video generation model post-training through data, reward, and conditional guidance design

    Jiachen Li, Qian Long, Jian Zheng, Xiaofeng Gao, Robinson Piramuthu, Wenhu Chen, and William Yang Wang. T2v-turbo-v2: Enhancing video generation model post-training through data, reward, and conditional guidance design. InThe Thirteenth International Conference on Learning Rep...

    2025

  8. [16]

    Terapipe: Token-level pipeline parallelism for training large-scale language models

    Zhuohan Li, Siyuan Zhuang, Shiyuan Guo, Danyang Zhuo, Hao Zhang, Dawn Song, and Ion Stoica. Terapipe: Token-level pipeline parallelism for training large-scale language models. InProceedings of the 38th International Conference on Machine Learning (ICML), 2021

    2021

  9. [17]

    Ring attention with blockwise transformers for near-infinite context.arXiv preprint arXiv:2310.01889, 2023

    Hao Liu, Matei Zaharia, and Pieter Abbeel. Ring attention with blockwise transformers for near-infinite context.arXiv preprint arXiv:2310.01889, 2023

    2023 arXiv

  10. [18]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023

    2023

  11. [19]

    Open-sora 2.0: Training a commercial-level video generation model in 200k.arXiv preprint arXiv:2503.09642, 2025

    Xiangyu Peng, Zangwei Zheng, Chenhui Shen, Tom Young, Xinying Guo, Binluo Wang, Hang Xu, Hongxin Liu, Mingyan Jiang, Wenjun Li, Yuhui Wang, Anbang Ye, Gang Ren, Qianran Ma, Wanying Liang, Xiang Lian, Xiwen Wu, Yuting Zhong, Zhuangyan Li, Chaoyu Gong, Guojun Lei, Leijun Cheng, ...

    2025 arXiv

  12. [20]

    Zero bubble (almost) pipeline parallelism

    Penghui Qi, Xinyi Wan, Guangxing Huang, and Min Lin. Zero bubble (almost) pipeline parallelism. In The Twelfth International Conference on Learning Representations (ICLR), 2024

    2024

  13. [21]

    Freenoise: Tuning-free longer video diffusion via noise rescheduling

    Haonan Qiu, Menghan Xia, Yong Zhang, Yingqing He, Xintao Wang, Ying Shan, and Ziwei Liu. Freenoise: Tuning-free longer video diffusion via noise rescheduling. InThe Twelfth International Conference on Learning Representations (ICLR), 2024

    2024

  14. [22]

    Rolling diffusion models

    David Ruhe, Jonathan Heek, Tim Salimans, and Emiel Hoogeboom. Rolling diffusion models. In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024

    2024

  15. [23]

    Magi-1: Autoregressive video generation at scale, 2025

    Sand-AI. Magi-1: Autoregressive video generation at scale, 2025

    2025

  16. [24]

    Megatron-lm: Training multi-billion parameter language models using model parallelism, 2020

    Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-lm: Training multi-billion parameter language models using model parallelism, 2020

    2020

  17. [25]

    Video-infinity: Distributed long video generation.arXiv preprint arXiv:2406.16260, 2024

    Zhenxiong Tan, Xingyi Yang, Songhua Liu, and Xinchao Wang. Video-infinity: Distributed long video generation.arXiv preprint arXiv:2406.16260, 2024

    2024 arXiv

  18. [26]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. InAdvances in Neural Information Processing Systems (NeurIPS), 2017

    2017

  19. [27]

    Wan: Open and advanced large-scale video generative models, 2025

    WanTeam. Wan: Open and advanced large-scale video generative models, 2025

    2025

  20. [28]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, Da Yin, Yuxuan.Zhang, Weihan Wang, Yean Cheng, Bin Xu, Xiaotao Gu, Yuxiao Dong, and Jie Tang. Cogvideox: Text-to-video diffusion models with an ex...

    2025

  21. [29]

    Videomerge: Towards training-free long video generation

    Siyang Zhang, Harry Yang, and Ser-Nam Lim. Videomerge: Towards training-free long video generation. arXiv preprint arXiv:2503.09926, 2025

    2025 arXiv

  22. [30]

    Open-sora: Democratizing efficient video production for all.arXiv preprint arXiv:2412.20404, 2024

    Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all.arXiv preprint arXiv:2412.20404, 2024. 11 A Technical Appendices and Supplementary Material A.1 Re...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.