{"id":"da57feeb-1bd1-434a-8eab-c18fa564efdc","arxiv_id":"2412.01316","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Presto lets each temporal segment of a video attend to a matching sub-caption, enabling long, content-rich videos that beat prior open-source and commercial models on VBench semantic and dynamic scores.","lead":"This paper introduces Presto, a video diffusion model that generates roughly 15-second videos by splitting the video into temporal segments and cross-attending each segment to its own sub-caption. It also contributes a curated dataset of 261k long videos with progressive captions, and reports state-of-the-art scores on the VBench benchmark for content richness and motion.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"VBench comparison mixes evaluation protocols: Presto is scored on 88 pre-interpolation frames at a reduced frame rate while baselines use leaderboard scores, and Presto receives GPT-4o-expanded sub-captions that baselines do not.","rationale":"I read the paper's central claim as: Presto outperforms existing SOTA video generation methods on long-range coherence and content richness, as evidenced by VBench Semantic Score 78.5% and Dynamic Degree 100%. For that claim to be supported, the quantitative comparison must measure Presto and baselines under equivalent conditions. The paper's own protocol undermines this equivalence: Presto is evaluated on 88 pre-interpolation frames while baselines come from the leaderboard, and Presto's inference process additionally expands the user prompt into five detailed sub-captions. Either difference alone could move the reported numbers; together they make the headline comparison unreliable. The reader's weakest_assumption identifies essentially this issue, and I agree. I checked for internal inconsistencies and found none fatal: the SCA mechanism is described coherently, the dataset curation pipeline is plausible, and the limitations section candidly acknowledges quality degradation and artifacts. No code, data, or model is released, which compounds the difficulty of verification, but that is a reproducibility problem rather than the central correctness risk. The concern is load-bearing because the paper's strongest quantitative evidence for the headline claim is exactly this VBench comparison. A single controlled re-evaluation could settle whether the result survives standard preprocessing and whether prompt expansion, rather than the proposed architecture and dataset, drives the semantic gains. Since the reader already marked the paper CONDITIONAL on this class of issue, I do not recommend changing the verdict; the condition should be made explicit as requiring this evaluation.","tokens_in":17905,"tokens_out":5978,"duration_ms":58987,"concrete_test":"Run the official VBench pipeline on the same Presto output set under three conditions: (a) the pre-interpolation 88-frame outputs as used in the paper, (b) the EMA-VFI-interpolated final videos, and (c) all methods resampled to a common temporal grid (matched fps and duration) before metrics are computed. If Dynamic Degree drops below 100% in condition (c), or if Semantic Score changes materially, the leaderboard comparison is not apples-to-apples. In the same test, also score Allegro both with and without GPT-4o prompt expansion; if Allegro plus expansion closes the Semantic Score gap, the SCA and dataset contributions are not isolated from the prompt-enrichment effect.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that Presto 'outperforms existing state-of-the-art video generation methods' rests on VBench scores that are not computed under a common evaluation protocol. In Sec. 5.2 the authors state that for Presto they 'assess the original videos with 88 frames before interpolation,' while Allegro, Gen-3, and TALC scores are taken directly from the VBench leaderboard. The paper trains at 6 FPS (Sec. 4.3), so 88 frames correspond to roughly 14.7 seconds of content at that rate; EMA-VFI is then applied in post-processing to normalize speed and extend length. If VBench's Dynamic Degree is computed from temporal frame differences or from a fixed number of uniform samples over the clip, a lower source frame rate and longer duration can inflate the per-frame motion measurement. This could explain the 100% Dynamic Degree for Presto versus 55.0 for Allegro without implying genuinely better dynamics. The Semantic Score comparison is also confounded: at inference, Presto's short prompt is expanded by GPT-4o into five progressive sub-captions (Sec. 3.2, Sec. 4.3), whereas leaderboard baselines receive only the original VBench prompt. The 'Single Long Condition' ablation in Tab. 4 concatenates those expanded sub-captions, so it does not isolate the effect of prompt expansion from the SCA mechanism. Therefore the headline comparison does not currently establish that the gains come from Segmented Cross-Attention and LongTake-HD rather than from the differing evaluation and prompting setup.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Presto, a text-to-video diffusion model that targets 15-second generation with long-range coherence and rich content. The method builds on Allegro, a DiT-based model, and introduces Segmented Cross-Attention (SCA), which splits temporal hidden states into five segments and lets each segment cross-attend to a corresponding progressive sub-caption generated by GPT-4o. The authors also introduce LongTake-HD, a curated dataset of 261k clips (47k for fine-tuning) with an overall caption and five progressive sub-captions per video. Quantitative results on VBench report a Semantic Score of 78.5% and Dynamic Degree of 100%, which the paper claims outperforms existing state-of-the-art models, and a user study reports favorable win rates. The central technical ideas are simple and parameter-free, but the main empirical comparison suffers from protocol mismatches and the ablations are not conclusive.","tokens_in":18251,"tokens_out":7866,"duration_ms":66601,"significance":"If the central claims hold, the paper would make a useful contribution: SCA is a parameter-free modification that could be applied broadly to DiT-based video generators, and the LongTake-HD curation pipeline with progressive sub-captions is a valuable template for long-video data construction. The paper documents the data-filtering thresholds in detail and provides a clear description of the three SCA variants. The user study is a positive aspect, as it directly compares generated videos on dimensions that matter for long-form content. However, the headline quantitative claim is weakened by the fact that Presto is evaluated on 88 pre-interpolation frames at a native 6 FPS sampling rate with GPT-4o-expanded prompts, while baselines are copied from the VBench leaderboard; the ablations show small differences without significance testing and are run at a different resolution and frame count. The significance of the work is therefore conditional on a more carefully controlled empirical evaluation.","major_comments":[{"comment":"The central quantitative claim is not supported by a controlled comparison. The paper states that for Presto 'we assess the original videos with 88 frames before interpolation,' while scores for Allegro, Gen-3, and TALC are taken directly from the VBench leaderboard. Because Presto samples frames at 6 FPS (Sec. 4.3), its 88 evaluation frames correspond to roughly 14.7 seconds of content, whereas the leaderboard videos are generated under different native frame rates and durations. VBench dimensions such as Dynamic Degree are computed from temporal differences, so a lower sampling rate or a longer clip can alter the score independently of generation quality; this may explain the 100% Dynamic Degree for Presto versus 55.0% for Allegro without implying genuinely superior motion. The Semantic Score comparison is also confounded: at inference, Presto expands each user prompt into five GPT-4o-generated sub-captions (Secs. 3.2 and 4.3), while the leaderboard baselines did not receive this prompt processing. Consequently, Table 2 does not establish the Abstract's claim that Presto 'outperforms existing state-of-the-art video generation methods.' The authors should re-run at least the key baselines under an identical evaluation protocol (same frame count, frame rate, interpolation status, and prompt processing) or substantially qualify the claim.","section":"Sec. 5.2, Table 2; Abstract"},{"comment":"The ablation study does not provide strong quantitative evidence for the design choices. The differences between OSCA, SSCA, ISCA, 'w/o Meticulous Filtering,' and 'Single Long Condition' are small (1.0-2.9 points on VBench Overall Score) and are reported without error bars, confidence intervals, or significance tests. In addition, these ablations are run at 360p resolution with 40 frames, whereas the main experiments are at 720p with 88 frames, so it is unclear whether the observed ordering of strategies transfers to the actual operating point. The 'Single Long Condition' baseline concatenates the same GPT-4o-expanded sub-captions into a single long text, which controls for text length but not for the effect of the inference-time prompt expansion itself; adding a baseline that uses the original single user prompt with standard global cross-attention would help attribute the gains specifically to Segmented Cross-Attention rather than to the richer prompt.","section":"Sec. 5.4, Table 4"},{"comment":"The explanation for Presto's lower VBench Quality Score (80.6 versus 84.1 for Gen-3 and 83.1 for Allegro) is speculative. Appendix C hypothesizes that 'dynamism loss' due to motion blur and H.264 compression causes the degradation, but no controlled experiment, correlation analysis, or quantitative measurement is provided to link the observed quality drop to motion complexity. The authors should either support this explanation with evidence or present the quality degradation as an observation without assigning a causal mechanism.","section":"Sec. 5.2 and Appendix C"}],"minor_comments":[{"comment":"The text refers to 'Runaway Gen-3,' but the reference [31] and common usage spell it 'Runway Gen-3'; please correct the typo.","section":"Sec. 5.1"},{"comment":"The overlap condition is written as 'δ < [T/N]' without defining the square brackets; please clarify whether this denotes the integer part and state the units of δ.","section":"Sec. 4.2"},{"comment":"The example in the progressive sub-captions paragraph shows three sub-captions, while the dataset construction and model use five; please add a note that this is a simplified illustration.","section":"Sec. 3.2"},{"comment":"The footnote states that 'Aesthetics, Diversity, Coherence, and Quality' are evaluated on 100 random samples, but the main caption says 'Unless specifically noted otherwise, data is calculated over the entire dataset using automated metrics'; please make explicit that the Aesthetics column is also based on the 100-sample subset.","section":"Table 1"},{"comment":"The sentence 'Progressive sub-captions have been demonstrated to improve semantic scores in diffusion model training [38]' is not clearly supported by reference [38], which describes a model rather than a controlled comparison; please provide a more direct citation or rephrase as a hypothesis.","section":"Appendix B"},{"comment":"The paper does not state whether the LongTake-HD dataset, the filtering code, or the trained model will be publicly released; given that a large part of the contribution is the dataset and curation pipeline, an availability statement would improve reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The core evaluation issue is the protocol mismatch in Table 2: Presto's scores are computed on 88 pre-interpolation frames at 6 FPS with GPT-4o-expanded prompts, while the baselines are taken from the leaderboard. This is fixable but requires additional experiments. I would encourage the editors to ask for a direct re-evaluation of at least Allegro under the same frame and prompt protocol, and ideally also for a baseline that ablates the prompt expansion. The ablations in Table 4 are also too weak to support the strong claims about OSCA. The dataset and code are not linked, which is a reproducibility concern for a data-centric paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look if you work on long video generation. The OSCA variant—overlapping temporal segments with averaged cross-attention at boundaries—is a clean, parameter-free idea, and the progressive sub-caption dataset is a genuinely useful contribution. The framing against MT2V is fair, and the paper is internally consistent and candid about its limitations, including the admitted drop in visual fidelity and high-motion artifacts. The ablations, though small in absolute terms, do show OSCA beating the naive single-long-condition baseline, which supports the core mechanism. The dataset comparisons in Table 1 are also a plus. The main soft spot is the VBench evaluation. Presto is scored on 88 pre-interpolation frames sampled at its native 6 FPS, while Allegro, Gen-3, and TALC scores come straight from the leaderboard—those models may have different native frame counts, frame rates, and prompt processing. On top of that, Presto's inference prompt is expanded by GPT-4o into five sub-captions, while leaderboard baselines get only the original short prompt. The 'Single Long Condition' ablation concatenates the expanded sub-captions, so it does not isolate the effect of prompt expansion from the segmented attention. That means the headline result—78.5% semantic score, 100% dynamic degree—does not yet establish that the gains come from SCA and LongTake-HD rather than from the evaluation and prompting setup. This is a real flaw, not a nitpick, but it is also fixable: rerun baselines under the same protocol, or at minimum ablate with and without GPT-4o expansion while holding the attention mechanism fixed. The paper also ships no code, data, or model, which limits near-term reproducibility, though the project page suggests some release may be coming. Bottom line: the method and dataset deserve a serious referee and are likely to be built on, but the quantitative claims need strengthening before publication. I would engage with the work, but I would not cite the headline numbers as they stand.","headline":"A useful method-plus-data paper on long video generation whose headline numbers don't yet survive a common-protocol comparison.","tokens_in":18814,"tokens_out":999,"would_cite":true,"duration_ms":9484,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A video diffusion model called Presto generates 15-second videos with rich, coherent multi-scene content, claiming to outperform open-source and commercial state-of-the-art systems on semantic fidelity and motion.","keywords":["long video generation","Segmented Cross-Attention","text-to-video diffusion","progressive sub-captions","video dataset curation","VBench","Dynamic Degree"],"falsifier":"A controlled replication of the VBench evaluation in which all compared models are assessed at the same native frame count and frame rate, or with all outputs normalized to the same duration before scoring, would settle the claim. If Presto's 78.5% Semantic Score and 100% Dynamic Degree shrink to within noise of Allegro and Gen-3 under such normalized settings, the outperforming state-of-the-art conclusion would not survive; a reader can check this because the paper does not cite the VBench benchmark, leaving the exact scoring protocol unverifiable.","tokens_in":17684,"feed_emoji":"🎬","tokens_out":5434,"duration_ms":41320,"temperature":0.7,"pith_summary":"Presto is a video diffusion model that generates 15-second videos from a single text prompt while keeping the content rich and the scenes continuous. The paper claims it outperforms existing open-source and commercial text-to-video models on semantic fidelity and motion, reaching 78.5% on the VBench Semantic Score and a perfect 100% on Dynamic Degree. The core design is Segmented Cross-Attention, which splits the video latent along time so each segment attends to its own progressive sub-caption, with overlap at segment boundaries smoothing transitions. To train it, the authors build LongTake-HD, a curated set of 261k single-scene videos annotated with an overall caption and five progressive sub-captions, and show that this data contributes as much as the attention mechanism to the gains.","feed_headline":"78.5% semantic score, 100% dynamic: long-video model tops leaders","feed_subtitle":"Segmented cross-attention lets each temporal segment read its own sub-caption, keeping 15-second videos rich yet continuous.","key_machinery":"Segmented Cross-Attention (SCA), specifically the Overlap variant (OSCA): the temporal dimension of the diffusion transformer's hidden states is evenly split into N non-overlapping segments, each segment cross-attends to its own sub-caption embeddings, and a small number of frames near each segment boundary additionally attend to the neighboring sub-caption, with the overlapping attention outputs averaged. This mechanism adds no parameters and relies on the model's self-attention to propagate global coherence across segments while each segment's dedicated sub-caption drives local content richness.","core_discovery":"The paper's central claim is that long video generation does not have to trade content diversity against long-range coherence. By decomposing the text condition into five progressive sub-captions and letting temporal segments of the latent attend to their respective sub-captions, a DiT-based diffusion model can keep each scene visually distinct while the storyline remains continuous. The authors argue that the Overlap Segmented Cross-Attention (OSCA) variant is the key: boundary frames attend to two adjacent sub-captions and the outputs are averaged, so scenario transitions are smooth instead of abrupt. They further claim that the curated LongTake-HD data is co-responsible for the gains, since removing its meticulous filtering drops the overall VBench score by 2.7 points and replacing SCA with a single concatenated long text drops it by 2.9.","pith_inferences":["The evaluation discrepancy the authors leave unaddressed: comparing 88-frame pre-interpolation outputs with leaderboard scores from models with different native durations means the quantitative claim may be sensitive to normalization; a fair test would score all models at matched length and frame rate.","The progressive sub-caption recipe is likely transferable beyond video: any generative model that must follow a long, time-varying narrative could decompose its conditioning into segments, but the paper only demonstrates this for video diffusion.","The dataset curation pipeline—scene segmentation, aesthetic and optical-flow thresholds, and LLM-based progressive caption refinement—is a reusable template, and a testable extension would be to apply it to public web video at larger scale and measure whether gains persist.","The authors' acknowledgment that Presto is 'not suitable for generating still frames' implies a motion bias that could be corrected by conditioning on target motion level; the paper does not explore this."],"forward_implications":["A single text prompt can now produce a 15-second video with several distinct but connected scenes, including specified camera motion, without additional user input.","Because SCA adds no parameters, any DiT-based text-to-video model can adopt it with minimal fine-tuning, making long, multi-scene generation an architectural plug-in rather than a new model.","The LongTake-HD dataset, with its narrative, camera-aware progressive sub-captions, provides a training resource that other long-video models could reuse, and its curation pipeline is a template for building similar datasets from public video corpora.","The reported 100% Dynamic Degree suggests the method handles strong camera movement and scene transitions, though the authors note a trade-off: visual quality drops on very dynamic backgrounds and still frames are poorly supported."],"supporting_citations":[{"why":"Allegro, the open-source 2.8B parameter base model that Presto builds on and fine-tunes.","marker":"[54]"},{"why":"TALC, the multi-scene text-to-video baseline compared in the experiments.","marker":"[4]"},{"why":"Gen-3 Alpha, the commercial baseline compared in the experiments.","marker":"[31]"},{"why":"T5 text encoder used to encode the progressive sub-captions.","marker":"[26]"},{"why":"Aria, the vision-language captioner used to generate captions for videos and frames.","marker":"[15]"},{"why":"GPT-4, the LLM used to refine progressive sub-captions and generate inference scripts.","marker":"[1]"},{"why":"EMA-VFI, the frame interpolation model used to extend videos during post-processing.","marker":"[51]"},{"why":"Panda-70M, a public source dataset used in curation and comparison.","marker":"[6]"}],"fun_headline_variants":["Segmented cross-attention keeps 15-second videos coherent and rich","No extra params: SCA splits video attention for long-range coherence","78.5% semantic, 100% dynamic: Presto's SCA and LongTake-HD data","Presto video model: per-segment sub-captions for 15-second coherence","Five sub-captions, one video: SCA maintains scenario diversity"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that VBench scores computed on Presto's 88-frame, pre-interpolation videos are directly comparable to leaderboard scores for models whose native frame counts, frame rates, and prompt processing differ; if VBench's dimensions are sensitive to video length or frame rate, the reported outperforming result may be an artifact of the evaluation protocol rather than of the method.","fun_headline_variants_meta":{"raw":{"variants":["Segmented cross-attention keeps 15-second videos coherent and rich","No extra params: SCA splits video attention for long-range coherence","78.5% semantic, 100% dynamic: Presto's SCA and LongTake-HD data","Presto video model: per-segment sub-captions for 15-second coherence","Five sub-captions, one video: SCA maintains scenario diversity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000287,"raw_usage":{"total_tokens":1674,"prompt_tokens":921,"completion_tokens":753,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":537,"completion_tokens_details":{"reasoning_tokens":647}},"tokens_in":537,"tokens_out":753,"duration_ms":6266,"temperature":1.0,"reasoning_tokens":647,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:29:19.711714+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled replication of the VBench evaluation in which all compared models are assessed at the same native frame count and frame rate, or with all outputs normalized to the same duration before scoring, would settle the claim. If Presto's 78.5% Semantic Score and 100% Dynamic Degree shrink to within noise of Allegro and Gen-3 under such normalized settings, the outperforming state-of-the-art conclusion would not survive; a reader can check this because the paper does not cite the VBench benchmark, leaving the exact scoring protocol unverifiable.","supporting_citations":[{"cited_title":"Gen-3 Alpha","cited_arxiv_id":null,"evidence_quote":"Gen-3 Alpha, the commercial baseline compared in the experiments."},{"cited_title":"Exploring the limits of transfer learning with a unified text-to-text transformer","cited_arxiv_id":null,"evidence_quote":"T5 text encoder used to encode the progressive sub-captions."},{"cited_title":"Aria: An open multimodal native mixture-of- experts model, 2024","cited_arxiv_id":null,"evidence_quote":"Aria, the vision-language captioner used to generate captions for videos and frames."},{"cited_title":"Extracting motion and ap- pearance via inter-frame attention for efficient video frame interpolation","cited_arxiv_id":null,"evidence_quote":"EMA-VFI, the frame interpolation model used to extend videos during post-processing."},{"cited_title":"Panda-70M: Captioning 70M videos with multiple cross-modality teachers","cited_arxiv_id":null,"evidence_quote":"Panda-70M, a public source dataset used in curation and comparison."}],"review_version":1}