Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

Long Video Diffusion Generation with Segmented Cross-Attention and Content-Rich Video Data Curation

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A video diffusion model called Presto generates 15-second videos with rich, coherent multi-scene content, claiming to outperform open-source and commercial state-of-the-art systems on semantic fidelity and motion.

desk verdict A useful method-plus-data paper on long video generation whose headline numbers don't yet survive a common-protocol comparison. read the letter →

arxiv 2412.01316 v2 pith:DCPDFQ2A submitted 2024-12-02 cs.CV cs.AIcs.MM

Xin Yan , Yuxuan Cai , Qiuyue Wang , Yuan Zhou , Wenhao Huang , Huan Yang This is my paper · ORCID
classification cs.CVcs.AIcs.MM
keywords longvideogenerationSegmentedCross-Attentiontext-to-videodiffusionprogressivesub-captionsdatasetcurationVBenchDynamicDegree
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Presto is a video diffusion model that generates 15-second videos from a single text prompt while keeping the content rich and the scenes continuous. The paper claims it outperforms existing open-source and commercial text-to-video models on semantic fidelity and motion, reaching 78.5% on the VBench Semantic Score and a perfect 100% on Dynamic Degree. The core design is Segmented Cross-Attention, which splits the video latent along time so each segment attends to its own progressive sub-caption, with overlap at segment boundaries smoothing transitions. To train it, the authors build LongTake-HD, a curated set of 261k single-scene videos annotated with an overall caption and five progressive sub-captions, and show that this data contributes as much as the attention mechanism to the gains.

What carries the argument

Segmented Cross-Attention (SCA), specifically the Overlap variant (OSCA): the temporal dimension of the diffusion transformer's hidden states is evenly split into N non-overlapping segments, each segment cross-attends to its own sub-caption embeddings, and a small number of frames near each segment boundary additionally attend to the neighboring sub-caption, with the overlapping attention outputs averaged. This mechanism adds no parameters and relies on the model's self-attention to propagate global coherence across segments while each segment's dedicated sub-caption drives local content richness.

What would settle it

A controlled replication of the VBench evaluation in which all compared models are assessed at the same native frame count and frame rate, or with all outputs normalized to the same duration before scoring, would settle the claim. If Presto's 78.5% Semantic Score and 100% Dynamic Degree shrink to within noise of Allegro and Gen-3 under such normalized settings, the outperforming state-of-the-art conclusion would not survive; a reader can check this because the paper does not cite the VBench benchmark, leaving the exact scoring protocol unverifiable.

Watch

Extended reading notes

Core claim

The paper's central claim is that long video generation does not have to trade content diversity against long-range coherence. By decomposing the text condition into five progressive sub-captions and letting temporal segments of the latent attend to their respective sub-captions, a DiT-based diffusion model can keep each scene visually distinct while the storyline remains continuous. The authors argue that the Overlap Segmented Cross-Attention (OSCA) variant is the key: boundary frames attend to two adjacent sub-captions and the outputs are averaged, so scenario transitions are smooth instead of abrupt. They further claim that the curated LongTake-HD data is co-responsible for the gains, since removing its meticulous filtering drops the overall VBench score by 2.7 points and replacing SCA with a single concatenated long text drops it by 2.9.

Load-bearing premise

The load-bearing assumption is that VBench scores computed on Presto's 88-frame, pre-interpolation videos are directly comparable to leaderboard scores for models whose native frame counts, frame rates, and prompt processing differ; if VBench's dimensions are sensitive to video length or frame rate, the reported outperforming result may be an artifact of the evaluation protocol rather than of the method.

Editorial extensions

If this is right

  • A single text prompt can now produce a 15-second video with several distinct but connected scenes, including specified camera motion, without additional user input.
  • Because SCA adds no parameters, any DiT-based text-to-video model can adopt it with minimal fine-tuning, making long, multi-scene generation an architectural plug-in rather than a new model.
  • The LongTake-HD dataset, with its narrative, camera-aware progressive sub-captions, provides a training resource that other long-video models could reuse, and its curation pipeline is a template for building similar datasets from public video corpora.
  • The reported 100% Dynamic Degree suggests the method handles strong camera movement and scene transitions, though the authors note a trade-off: visual quality drops on very dynamic backgrounds and still frames are poorly supported.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The evaluation discrepancy the authors leave unaddressed: comparing 88-frame pre-interpolation outputs with leaderboard scores from models with different native durations means the quantitative claim may be sensitive to normalization; a fair test would score all models at matched length and frame rate.
  • The progressive sub-caption recipe is likely transferable beyond video: any generative model that must follow a long, time-varying narrative could decompose its conditioning into segments, but the paper only demonstrates this for video diffusion.
  • The dataset curation pipeline—scene segmentation, aesthetic and optical-flow thresholds, and LLM-based progressive caption refinement—is a reusable template, and a testable extension would be to apply it to public web video at larger scale and measure whether gains persist.
  • The authors' acknowledgment that Presto is 'not suitable for generating still frames' implies a motion bias that could be corrected by conditioning on target motion level; the paper does not explore this.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Presto, a text-to-video diffusion model that targets 15-second generation with long-range coherence and rich content. The method builds on Allegro, a DiT-based model, and introduces Segmented Cross-Attention (SCA), which splits temporal hidden states into five segments and lets each segment cross-attend to a corresponding progressive sub-caption generated by GPT-4o. The authors also introduce LongTake-HD, a curated dataset of 261k clips (47k for fine-tuning) with an overall caption and five progressive sub-captions per video. Quantitative results on VBench report a Semantic Score of 78.5% and Dynamic Degree of 100%, which the paper claims outperforms existing state-of-the-art models, and a user study reports favorable win rates. The central technical ideas are simple and parameter-free, but the main empirical comparison suffers from protocol mismatches and the ablations are not conclusive.

Significance. If the central claims hold, the paper would make a useful contribution: SCA is a parameter-free modification that could be applied broadly to DiT-based video generators, and the LongTake-HD curation pipeline with progressive sub-captions is a valuable template for long-video data construction. The paper documents the data-filtering thresholds in detail and provides a clear description of the three SCA variants. The user study is a positive aspect, as it directly compares generated videos on dimensions that matter for long-form content. However, the headline quantitative claim is weakened by the fact that Presto is evaluated on 88 pre-interpolation frames at a native 6 FPS sampling rate with GPT-4o-expanded prompts, while baselines are copied from the VBench leaderboard; the ablations show small differences without significance testing and are run at a different resolution and frame count. The significance of the work is therefore conditional on a more carefully controlled empirical evaluation.

major comments (3)
  1. [Sec. 5.2, Table 2; Abstract] The central quantitative claim is not supported by a controlled comparison. The paper states that for Presto 'we assess the original videos with 88 frames before interpolation,' while scores for Allegro, Gen-3, and TALC are taken directly from the VBench leaderboard. Because Presto samples frames at 6 FPS (Sec. 4.3), its 88 evaluation frames correspond to roughly 14.7 seconds of content, whereas the leaderboard videos are generated under different native frame rates and durations. VBench dimensions such as Dynamic Degree are computed from temporal differences, so a lower sampling rate or a longer clip can alter the score independently of generation quality; this may explain the 100% Dynamic Degree for Presto versus 55.0% for Allegro without implying genuinely superior motion. The Semantic Score comparison is also confounded: at inference, Presto expands each user prompt into five GPT-4o-generated sub-captions (Secs. 3.2 and 4.3), while the leaderboard baselines did not receive this prompt processing. Consequently, Table 2 does not establish the Abstract's claim that Presto 'outperforms existing state-of-the-art video generation methods.' The authors should re-run at least the key baselines under an identical evaluation protocol (same frame count, frame rate, interpolation status, and prompt processing) or substantially qualify the claim.
  2. [Sec. 5.4, Table 4] The ablation study does not provide strong quantitative evidence for the design choices. The differences between OSCA, SSCA, ISCA, 'w/o Meticulous Filtering,' and 'Single Long Condition' are small (1.0-2.9 points on VBench Overall Score) and are reported without error bars, confidence intervals, or significance tests. In addition, these ablations are run at 360p resolution with 40 frames, whereas the main experiments are at 720p with 88 frames, so it is unclear whether the observed ordering of strategies transfers to the actual operating point. The 'Single Long Condition' baseline concatenates the same GPT-4o-expanded sub-captions into a single long text, which controls for text length but not for the effect of the inference-time prompt expansion itself; adding a baseline that uses the original single user prompt with standard global cross-attention would help attribute the gains specifically to Segmented Cross-Attention rather than to the richer prompt.
  3. [Sec. 5.2 and Appendix C] The explanation for Presto's lower VBench Quality Score (80.6 versus 84.1 for Gen-3 and 83.1 for Allegro) is speculative. Appendix C hypothesizes that 'dynamism loss' due to motion blur and H.264 compression causes the degradation, but no controlled experiment, correlation analysis, or quantitative measurement is provided to link the observed quality drop to motion complexity. The authors should either support this explanation with evidence or present the quality degradation as an observation without assigning a causal mechanism.
minor comments (6)
  1. [Sec. 5.1] The text refers to 'Runaway Gen-3,' but the reference [31] and common usage spell it 'Runway Gen-3'; please correct the typo.
  2. [Sec. 4.2] The overlap condition is written as 'δ < [T/N]' without defining the square brackets; please clarify whether this denotes the integer part and state the units of δ.
  3. [Sec. 3.2] The example in the progressive sub-captions paragraph shows three sub-captions, while the dataset construction and model use five; please add a note that this is a simplified illustration.
  4. [Table 1] The footnote states that 'Aesthetics, Diversity, Coherence, and Quality' are evaluated on 100 random samples, but the main caption says 'Unless specifically noted otherwise, data is calculated over the entire dataset using automated metrics'; please make explicit that the Aesthetics column is also based on the 100-sample subset.
  5. [Appendix B] The sentence 'Progressive sub-captions have been demonstrated to improve semantic scores in diffusion model training [38]' is not clearly supported by reference [38], which describes a model rather than a controlled comparison; please provide a more direct citation or rephrase as a hypothesis.
  6. [General] The paper does not state whether the LongTake-HD dataset, the filtering code, or the trained model will be publicly released; given that a large part of the contribution is the dataset and curation pipeline, an availability statement would improve reproducibility.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central claims are tested against external benchmarks and the method is an architectural modification of an open-source backbone, not a renamed fit.

full rationale

The paper's derivation chain is self-contained in the relevant sense. Presto's headline numbers (78.5% VBench Semantic Score, 100% Dynamic Degree) are obtained by running the trained model on the external VBench benchmark, not by plugging fitted parameters back into the evaluation. The Segmented Cross-Attention mechanism is an architectural change requiring no additional parameters, and the reported comparisons among ISCA, SSCA, and OSCA are real ablations under the same evaluation protocol. The LongTake-HD contribution is supported by the 'w/o Meticulous Filtering' ablation, which compares curated data against a randomly sampled subset, and by the 'Single Long Condition' ablation, which is designed to separate segmented cross-attention from text-length effects. The paper builds on Allegro, which is a self-citation with overlapping authors, but Allegro is used as an open base model and comparison baseline, not as evidence for Presto's own claims. The Appendix B validation of progressive sub-captions uses text similarity between training and inference captions generated with the same LLM refinement style; however, the paper explicitly acknowledges that a direct diffusion-training comparison is left to future work, so that section is honestly labeled as indirect and does not function as a circular derivation of the main results. Remaining issues such as scoring Presto on 88 pre-interpolation frames versus leaderboard baselines and applying GPT-4o prompt expansion only to Presto are comparability or correctness concerns, not cases where a claimed result reduces to its own inputs by construction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central results rest on a chain of engineering choices: manually set data thresholds, five-way caption segmentation, an unspecified overlap delta, and the assumption that external metrics capture long-range coherence. The paper provides no independent, reproducible evidence beyond its own reported runs.

free parameters (3)
  • Data filtering thresholds (duration, FPS, resolution, aesthetics, flow, PSNR, SSIM, LPIPS, text similarity) = Tab. 5 ranges: aesthetics >=4.8 (pretrain) / >=5.0 (finetune); Unimatch flow >=40/50; PSNR [4,20]; SSIM [0,0.7]; LPIPS…
    All thresholds are manually set (Tab. 5: 'All thresholds are manually determined by the specific characteristics of the dataset'). They determine which 261k/47k videos enter training, so the claimed benefit of LongTake-HD depends on these choices.
  • Number of temporal segments N = 5
    The hidden state is split into five segments to match the five sub-captions. N is a design choice that controls the granularity of caption-video alignment and is not derived from data.
  • Overlap frames delta = not specified
    OSCA uses delta overlapping frames between adjacent segments; delta is never given numerically, making the exact method underspecified and unreproducible.
assumptions (4)
  • domain assumption The five equal temporal segments of the 88-frame latent align with the temporal progression of the five progressive sub-captions.
    SCA assumes the storyline is evenly distributed across time. If content changes unevenly, each segment attends to the wrong caption. This alignment is not validated directly in the paper.
  • domain assumption The external captioning and refinement models (Aria, GPT-4o, T5) produce captions and embeddings that preserve semantic distinctions needed by segmented cross-attention.
    The pipeline treats the outputs of these models as ground truth; no analysis of captioning error or its effect on generation is provided.
  • domain assumption VBench Semantic Score, Dynamic Degree, and the user study dimensions are valid proxies for content richness and long-range coherence.
    The paper's headline numbers rest on VBench and a user study; no calibration of these metrics against an independent notion of 'long-range coherence' is provided.
  • ad hoc to paper Self-attention across the entire sequence provides sufficient global communication to compensate for the segmented cross-attention.
    The design rationale for OSCA (Sec. 4.2) relies on self-attention to maintain global coherence, but no ablation isolates this assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Long Video Diffusion Generation with Segmented Cross-Attention and Content-Rich Video Data Curation." pith.science (2026). https://pith.science/paper/DCPDFQ2A

@misc{pith2026241201316,
  author       = {Pith},
  title        = {Pith review of: Long Video Diffusion Generation with Segmented Cross-Attention and Content-Rich Video Data Curation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DCPDFQ2A}},
  note         = {Machine review of arXiv:2412.01316}
}
read the original abstract

We introduce Presto, a novel video diffusion model designed to generate 15-second videos with long-range coherence and rich content. Extending video generation methods to maintain scenario diversity over long durations presents significant challenges. To address this, we propose a Segmented Cross-Attention (SCA) strategy, which splits hidden states into segments along the temporal dimension, allowing each segment to cross-attend to a corresponding sub-caption. SCA requires no additional parameters, enabling seamless incorporation into current DiT-based architectures. To facilitate high-quality long video generation, we build the LongTake-HD dataset, consisting of 261k content-rich videos with scenario coherence, annotated with an overall video caption and five progressive sub-captions. Experiments show that our Presto achieves 78.5% on the VBench Semantic Score and 100% on the Dynamic Degree, outperforming existing state-of-the-art video generation methods. This demonstrates that our proposed Presto significantly enhances content richness, maintains long-range coherence, and captures intricate textual details. More details are displayed on our project page: https://presto-video.github.io/.

Figures

Figures reproduced from arXiv: 2412.01316 by the authors.

Figure 1
Figure 1. Presto can generate long videos with rich content and long-range coherence. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (a) The overall architecture of our Presto, which integrates multiple text inputs concurrently. (b) The Segmented Cross-Attention strategy has three variants: 1) Isolated Segmented Cross-Attention (ISCA) directly splits the hidden states along the temporal dimension. The output is concatenated by multiple segments’ output. 2) Sequential Segmented Cross-Attention (SSCA) where each segment will see all the previous te… view at source ↗
Figure 3
Figure 3. Qualitative comparison with the baselines in our user study. Our Presto can capture intricate text details and generate long videos [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: The discarded and selected data samples of different filtering steps in LongTake-HD. We discard cases with similar keyframes and [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: The progressive sub-captions and coherent video frames of our LongTake-HD dataset. Our captions are more detailed in camera [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Our Presto can generate long videos with high scenario motion, and prioritize scenario smoothness. However, in the case of [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Qualitative comparison with the baselines in our user study. [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Qualitative comparison with the baselines in our user study. [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: More results of VBench’s prompts centering around the same theme. Presto can generate videos with accurate camera control [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LoViC: Efficient Long Video Generation with Context Compression

    cs.CV 2025-07 conditional novelty 6.0 of 10

    LoViC uses FlexFormer, a single-query-token Q-Former with interpolated rotary positional encoding, to compress long video-text context for efficient long-video generation.

  2. Frame-Level Captions for Long Video Generation with Complex Multi Scenes

    cs.CV 2025-05 conditional novelty 5.0 of 10

    Frame-level captions with per-frame cross-attention and parallel multi-window denoising reduce semantic confusion in long multi-scene video generation in the authors' internal evaluation.

Reference graph

Works this paper leans on

56 extracted references · 29 canonical work pages · cited by 2 Pith papers

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  2. [2]

    Character region awareness for text de- tection

    Youngmin Baek, Bado Lee, Dongyoon Han, Sangdoo Yun, and Hwalsuk Lee. Character region awareness for text de- tection. In CVPR, pages 9365–9374, 2019. 3

    work page 2019

  3. [3]

    Frozen in time: A joint video and image encoder for end-to-end retrieval

    Max Bain, Arsha Nagrani, G ¨ul Varol, and Andrew Zisser- man. Frozen in time: A joint video and image encoder for end-to-end retrieval. In ICCV, pages 1728–1738, 2021. 3

    2021

  4. [4]

    TALC: Time-aligned captions for multi-scene text-to-video generation

    Hritik Bansal, Yonatan Bitton, Michal Yarom, Idan Szpektor, Aditya Grover, and Kai-Wei Chang. TALC: Time-aligned captions for multi-scene text-to-video generation. arXiv preprint arXiv:2405.04682, 2024. 2, 6

    arXiv 2024

  5. [5]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luh- man, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators

  6. [6]

    Panda-70M: Captioning 70M videos with multiple cross-modality teachers

    Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao, Byung Eun Jeon, Yuwei Fang, Hsin-Ying Lee, Jian Ren, Ming-Hsuan Yang, et al. Panda-70M: Captioning 70M videos with multiple cross-modality teachers. In CVPR, pages 13320–13331,

    work page

  7. [7]

    RFC1951: Deflate compressed data format specification version 1.3, 1996

    Peter Deutsch. RFC1951: Deflate compressed data format specification version 1.3, 1996. 4

    work page 1996

  8. [8]

    The Cambridge dictionary of statistics

    Brian S Everitt. The Cambridge dictionary of statistics . Cambridge University Press 1998, 2002, 2006, 2006. 3

    work page 1998

Show all 56 references
  1. [9]

    Streamingt2v: Con- sistent, dynamic, and extendable long video generation from text

    Roberto Henschel, Levon Khachatryan, Daniil Hayrapetyan, Hayk Poghosyan, Vahram Tadevosyan, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Streamingt2v: Con- sistent, dynamic, and extendable long video generation from text. arXiv preprint arXiv:2403.14773, 2024. 1, 2

    2024 arXiv

  2. [10]

    Video dif- fusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video dif- fusion models. NeurIPS, 35:8633–8646, 2022. 1

    2022

  3. [11]

    A method for the construction of minimum-redundancy codes

    David A Huffman. A method for the construction of minimum-redundancy codes. Proceedings of the IRE, 40(9): 1098–1101, 1952. 4

    1952

  4. [12]

    Scope of va- lidity of psnr in image/video quality assessment

    Quan Huynh-Thu and Mohammed Ghanbari. Scope of va- lidity of psnr in image/video quality assessment. Electronics letters, 44(13):800–801, 2008. 3

    2008

  5. [13]

    Fifo-diffusion: Generating infinite videos from text without training

    Jihwan Kim, Junoh Kang, Jinyoung Choi, and Bohyung Han. Fifo-diffusion: Generating infinite videos from text without training. arXiv preprint arXiv:2405.11473, 2024. 1, 2

    2024 arXiv

  6. [14]

    Videopoet: A large language model for zero-shot video gen- eration

    Dan Kondratyuk, Lijun Yu, Xiuye Gu, Jos ´e Lezama, Jonathan Huang, Grant Schindler, Rachel Hornung, Vigh- nesh Birodkar, Jimmy Yan, Ming-Chang Chiu, et al. Videopoet: A large language model for zero-shot video gen- eration. arXiv preprint arXiv:2312.14125, 2023. 3

    2023 arXiv

  7. [15]

    Aria: An open multimodal native mixture-of- experts model, 2024

    Dongxu Li, Yudong Liu, Haoning Wu, Yue Wang, Zhiqi Shen, Bowen Qu, Xinyao Niu, Guoyin Wang, Bei Chen, and Junnan Li. Aria: An open multimodal native mixture-of- experts model, 2024. 4, 12

    2024

  8. [16]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out , pages 74–81, 2004. 12

    2004

  9. [17]

    Automatic evaluation of machine translation quality using longest common sub- sequence and skip-bigram statistics

    Chin-Yew Lin and Franz Josef Och. Automatic evaluation of machine translation quality using longest common sub- sequence and skip-bigram statistics. In Proceedings of the 42nd annual meeting of the association for computational linguistics (ACL-04), pages 605–612, 2004. 12

    2004

  10. [18]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 6

    2021

  11. [19]

    Freelong: Training-free long video generation with spectralblend tem- poral attention

    Yu Lu, Yuanzhi Liang, Linchao Zhu, and Yi Yang. Freelong: Training-free long video generation with spectralblend tem- poral attention. arXiv preprint arXiv:2407.19918, 2024. 1, 2

    2024 arXiv

  12. [20]

    OpenVid-1M: A large-scale high-quality dataset for text-to- video generation

    Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhen- heng Yang, Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai. OpenVid-1M: A large-scale high-quality dataset for text-to- video generation. arXiv preprint arXiv:2407.02371 , 2024. 3

    2024 arXiv

  13. [21]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021. 1

    2021 arXiv

  14. [22]

    MTVG: Multi-text video generation with text-to- video models

    Gyeongrok Oh, Jaehwan Jeong, Sieun Kim, Wonmin Byeon, Jinkyu Kim, Sungwoong Kim, Hyeokmin Kwon, and Sang- pil Kim. MTVG: Multi-text video generation with text-to- video models. arXiv preprint arXiv:2312.04086, 2023. 2

    2023 arXiv

  15. [23]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages 311–318,

  16. [24]

    PySceneDetect

    PySceneDetect Contributors. PySceneDetect. https:// www.scenedetect.com, 2024. 3

    2024

  17. [25]

    Freenoise: Tuning-free longer video diffusion via noise rescheduling.arXiv preprint arXiv:2310.15169, 2023

    Haonan Qiu, Menghan Xia, Yong Zhang, Yingqing He, Xin- tao Wang, Ying Shan, and Ziwei Liu. Freenoise: Tuning-free longer video diffusion via noise rescheduling.arXiv preprint arXiv:2310.15169, 2023. 1, 2

    2023 arXiv

  18. [26]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. JMLR, 21(140):1–67, 2020. 6

    2020

  19. [27]

    Contrastive sequential-diffusion learn- ing: An approach to multi-scene instructional video synthe- sis

    Vasco Ramos, Yonatan Bitton, Michal Yarom, Idan Szpektor, and Joao Magalhaes. Contrastive sequential-diffusion learn- ing: An approach to multi-scene instructional video synthe- sis. arXiv preprint arXiv:2407.11814, 2024. 2

    2024 arXiv

  20. [28]

    Sentence-bert: Sentence embeddings using siamese bert-networks

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Lan- guage Processing. Association for Computational Linguis- tics, 2019. 4, 12

    2019

  21. [29]

    Making monolingual sen- tence embeddings multilingual using knowledge distillation

    Nils Reimers and Iryna Gurevych. Making monolingual sen- tence embeddings multilingual using knowledge distillation. 10 In Proceedings of the 2020 Conference on Empirical Meth- ods in Natural Language Processing. Association for Com- putational Linguistics, 2020. 4, 12

    2020

  22. [30]

    MM-Diffusion: Learning multi-modal diffusion models for joint audio and video generation

    Ludan Ruan, Yiyang Ma, Huan Yang, Huiguo He, Bei Liu, Jianlong Fu, Nicholas Jing Yuan, Qin Jin, and Baining Guo. MM-Diffusion: Learning multi-modal diffusion models for joint audio and video generation. In CVPR, pages 10219– 10228, 2023. 1

    2023

  23. [31]

    Gen-3 Alpha

    RunwayML. Gen-3 Alpha. https://runwayml.com/ research/introducing-gen-3-alpha , 2024. 6

    2024

  24. [32]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. NeurIPS, 35:36479–36494, 2022. 1

    2022

  25. [33]

    LAION-5B: An open large-scale dataset for train- ing next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, et al. LAION-5B: An open large-scale dataset for train- ing next generation image-text models. NeurIPS, 35:25278– 252...

    2022

  26. [34]

    Make-A-Video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-A-Video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792 ,

    arXiv

  27. [35]

    Modern information retrieval: A brief overview

    Amit Singhal et al. Modern information retrieval: A brief overview. IEEE Data Eng. Bull., 24(4):35–43, 2001. 4, 12

    2001

  28. [36]

    MPNet: Masked and permuted pre-training for language un- derstanding

    Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. MPNet: Masked and permuted pre-training for language un- derstanding. arXiv preprint arXiv:2004.09297, 2020. 4, 12

    2004 arXiv

  29. [37]

    Video-infinity: distributed long video generation

    Zhenxiong Tan, Xingyi Yang, Songhua Liu, and Xinchao Wang. Video-infinity: distributed long video generation. arXiv preprint arXiv:2406.16260, 2024. 1, 2

    2024 arXiv

  30. [38]

    Phenaki: Variable length video generation from open domain textual description

    Ruben Villegas, Mohammad Babaeizadeh, Pieter-Jan Kin- dermans, Hernan Moraldo, Han Zhang, Mohammad Taghi Saffar, Santiago Castro, Julius Kunze, and Dumitru Erhan. Phenaki: Variable length video generation from open domain textual description. arXiv preprint arXiv:2210.02399, 2...

    arXiv

  31. [39]

    MCVD: Masked conditional video diffusion for prediction, generation, and interpolation

    Vikram V oleti, Alexia Jolicoeur-Martineau, and Chris Pal. MCVD: Masked conditional video diffusion for prediction, generation, and interpolation. NeurIPS, 35:23371–23385,

  32. [40]

    Gen-l-video: Multi-text to long video generation via temporal co-denoising

    Fu-Yun Wang, Wenshuo Chen, Guanglu Song, Han-Jia Ye, Yu Liu, and Hongsheng Li. Gen-l-video: Multi-text to long video generation via temporal co-denoising. arXiv preprint arXiv:2305.18264, 2023. 1, 2

    2023 arXiv

  33. [41]

    VideoFactory: Swap at- tention in spatiotemporal diffusions for text-to-video gener- ation

    Wenjing Wang, Huan Yang, Zixi Tuo, Huiguo He, Junchen Zhu, Jianlong Fu, and Jiaying Liu. VideoFactory: Swap at- tention in spatiotemporal diffusions for text-to-video gener- ation. arXiv preprint arXiv:2305.10874, 2023. 3

    2023 arXiv

  34. [42]

    VideoComposer: Compositional video synthesis with motion controllability

    Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Ji- uniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jin- gren Zhou. VideoComposer: Compositional video synthesis with motion controllability. NeurIPS, 36, 2024. 1

    2024

  35. [43]

    Loong: Generating minute-level long videos with autoregressive lan- guage models

    Yuqing Wang, Tianwei Xiong, Daquan Zhou, Zhijie Lin, Yang Zhao, Bingyi Kang, Jiashi Feng, and Xihui Liu. Loong: Generating minute-level long videos with autoregressive lan- guage models. arXiv preprint arXiv:2410.02757, 2024. 1, 2

    2024 arXiv

  36. [44]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. TIP, 13(4):600–612, 2004. 3

    2004

  37. [45]

    Struc- tural approaches to image quality assessment

    Zhou Wang, Alan C Bovik, and Eero P Simoncelli. Struc- tural approaches to image quality assessment. Handbook of image and video processing, 7(18), 2005. 12

    2005

  38. [46]

    Watermark-Detection

    Watermark-Detection Contributors. Watermark-Detection. https : / / github . com / boomb0om / watermark - detection, 2022. 3

    2022

  39. [47]

    Unifying flow, stereo and depth estimation

    Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, Fisher Yu, Dacheng Tao, and Andreas Geiger. Unifying flow, stereo and depth estimation. TPAMI, 2023. 3

    2023

  40. [48]

    Ad- vancing high-resolution video-language representation with large-scale video transcriptions

    Hongwei Xue, Tiankai Hang, Yanhong Zeng, Yuchong Sun, Bei Liu, Huan Yang, Jianlong Fu, and Baining Guo. Ad- vancing high-resolution video-language representation with large-scale video transcriptions. InCVPR, pages 5036–5045,

  41. [49]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024. 7

    2024 arXiv

  42. [50]

    Nuwa-xl: Diffusion over diffusion for extremely long video generation.arXiv preprint arXiv:2303.12346, 2023

    Shengming Yin, Chenfei Wu, Huan Yang, Jianfeng Wang, Xiaodong Wang, Minheng Ni, Zhengyuan Yang, Linjie Li, Shuguang Liu, Fan Yang, et al. Nuwa-xl: Diffusion over diffusion for extremely long video generation.arXiv preprint arXiv:2303.12346, 2023. 1, 2, 3

    2023 arXiv

  43. [51]

    Extracting motion and ap- pearance via inter-frame attention for efficient video frame interpolation

    Guozhen Zhang, Yuhan Zhu, Haonan Wang, Youxin Chen, Gangshan Wu, and Limin Wang. Extracting motion and ap- pearance via inter-frame attention for efficient video frame interpolation. In CVPR, pages 5682–5692, 2023. 6

    2023

  44. [52]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, pages 586–595,

  45. [53]

    Contphy: Continuum physical concept learning and reason- ing from videos

    Zhicheng Zheng, Xin Yan, Zhenfang Chen, Jingzhou Wang, Qin Zhi Eddie Lim, Joshua B Tenenbaum, and Chuang Gan. Contphy: Continuum physical concept learning and reason- ing from videos. arXiv preprint arXiv:2402.06119, 2024. 7

    2024 arXiv

  46. [54]

    Al- legro: Open the black box of commercial-level video gen- eration model

    Yuan Zhou, Qiuyue Wang, Yuxuan Cai, and Huan Yang. Al- legro: Open the black box of commercial-level video gen- eration model. arXiv preprint arXiv:2410.15458, 2024. 6, 7

    2024 arXiv

  47. [55]

    A universal algorithm for sequential data compression

    Jacob Ziv and Abraham Lempel. A universal algorithm for sequential data compression. IEEE Transactions on infor- mation theory, 23(3):337–343, 1977. 4

    1977

  48. [56]

    Compression of individual sequences via variable-rate coding

    Jacob Ziv and Abraham Lempel. Compression of individual sequences via variable-rate coding. IEEE transactions on Information Theory, 24(5):530–536, 1978. 4 11 Long Video Diffusion Generation with Segmented Cross-Attention and Content-Rich Video Data Curation Appendix A. Detail...

    1978

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.