{"id":"ee509f37-ea63-4440-a2be-345a03bb0e68","arxiv_id":"2508.03694","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"LongVie combines unified noise initialization, global control normalization, and multi-modal depth-plus-keypoint guidance to generate temporally consistent controllable videos of up to one minute.","lead":"This paper presents LongVie, a system that generates minute-long videos with fine-grained control over motion and structure, using an autoregressive diffusion model guided by depth maps and tracked keypoints. The authors also introduce a new 100-video benchmark for evaluating long controllable video generation.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOTA controllability claim rests on a reconstruction benchmark where controls are extracted from the target video itself, so the evaluation never tests whether LongVie follows arbitrary user-specified or out-of-distribution controls; a cross-video control-following test is needed.","rationale":"The reader's weakest-assumption identification matches the most load-bearing weakness: the benchmark protocol is a reconstruction test rather than a true control-following test. My read sharpens this by noting two concrete favorable details: controls are extracted from the same target video used for SSIM/LPIPS, and global normalization uses the full target video's depth statistics, both of which leak information that would be unavailable for genuinely novel user-specified controls. The small margins over Depth-LV and the absence of error bars compound the problem, but the central issue is protocol validity, not statistical noise alone. I do not reject the paper because the engineering contributions may still be useful and the proposed test could in principle be passed; the comparison should simply be run before the SOTA claim is accepted. Since the reader's verdict was already CONDITIONAL and my concern supports that conditionality rather than overturning it, the verdict remains unchanged.","tokens_in":14686,"tokens_out":5589,"duration_ms":66785,"concrete_test":"Construct a cross-video control-following test from LongVGenBench: for each of 100 source videos, extract depth and point controls, then pair those controls with a first frame and caption taken from a different video, so the controls are not derived from the target. Generate 10 clips per pair with LongVie and Depth-LV under the paper's autoregressive protocol, and measure (a) alignment of generated depth maps to the input depth controls, (b) keypoint-tracking accuracy, and (c) temporal consistency. Recompute SSIM/LPIPS against the video that actually matches the controls, not the original target. If LongVie's advantage over Depth-LV vanishes or reverses, the SOTA claim is benchmark-specific rather than a general controllability result.","verdict_should_be":"UNCHANGED","load_bearing_attack":"LongVie's central claim of 'state-of-the-art performance in long-range controllability' (Abstract, Sec. 3.1) is supported almost entirely by metrics on LongVGenBench. In Sec. 3.1, 'Control signals are extracted from the split clips,' and SSIM/LPIPS are computed against the same ground-truth videos. This is an in-distribution inversion test: the model receives perfect depth/point controls derived from the very target it is asked to reproduce, plus the original first frame, and the global normalization in Sec. 2.2 uses the full target video's 5th/95th percentile depth statistics. Such a setup cannot distinguish genuine control following from memorized reconstruction of the training pipeline. The quantitative margins are also thin: SSIM 0.557 vs. 0.508 and LPIPS 0.290 vs. 0.347 for the strongest baseline, with most VBench metrics differing by under 1%, and no error bars are reported. If LongVie is intended for practical controllability, it must be shown to follow controls that are not self-extracted from the desired output, and to retain its advantage under that harder, more realistic protocol.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LongVie, an autoregressive framework for controllable ultra-long video generation (up to one minute) built on CogVideoX. It identifies three causes of failure in long video generation: separate noise initialization across clips, per-clip control signal normalization, and single-modality guidance. To address these, LongVie introduces unified noise initialization, global control normalization over the full video, multi-modal control combining dense depth maps and sparse point maps, and a degradation-aware training strategy. The authors also introduce LongVGenBench, a benchmark of 100 long videos, and report quantitative results (VBench metrics, SSIM/LPIPS) and a user study showing improvements over adapted baselines.","tokens_in":14951,"tokens_out":5610,"duration_ms":60486,"significance":"If the claims hold, LongVie is a practical contribution to long video generation, addressing two well-recognized problems (temporal consistency and progressive quality degradation) in an autoregressive setting. The proposed benchmark fills a gap, since existing controllable-generation benchmarks focus on short clips. The paper provides a clear analysis of failure modes and includes useful ablations (e.g., robustness to blurred depth and masked keypoints). However, the core 'controllability' claim is currently supported only by a reconstruction-style evaluation where controls are extracted from the target video itself; this limits the significance of the state-of-the-art assertion. The quantitative margins over the strongest baseline are also thin, and no statistical significance is reported. The technical components are reasonable and the qualitative results suggest promise, but the evidence presented does not yet substantiate the headline claim of state-of-the-art controllability.","major_comments":[{"comment":"The evaluation protocol measures reconstruction, not control following. Control signals are extracted from the target videos ('Control signals are extracted from the split clips'), and SSIM/LPIPS are computed against the same ground-truth videos. This tests how well LongVie inverts the specific depth/point representations produced by Video Depth Anything and SpatialTracker on the training distribution; it does not test whether the model follows arbitrary user-specified or out-of-distribution controls. The user study in Table 2 uses the same protocol, so it is subject to the same limitation. To support the 'state-of-the-art controllability' claim, the authors should include a cross-video control-following experiment, e.g., generating a video from the first frame of one video and the depth/point controls of another, and evaluating whether the output preserves the content of the first frame while following the motion/geometry of the control sequence.","section":"Section 3.1 (LongVGenBench protocol)"},{"comment":"The reported gains over the strongest baseline, Depth-LV, are very small on the headline VBench metrics (Subject Consistency 87.12 vs. 87.09; Imaging Quality 64.91 vs. 64.84), and no error bars or significance tests are provided. The ablation drops are similarly tiny (e.g., Table 3, 'w/o Data Degradation' drops Subject Consistency by 0.01 and Background Consistency by 0.07). Without variance estimates or repeated runs, these differences are within the range of random noise and do not support the claim that the proposed components contribute meaningfully. The authors should report standard deviations across multiple seeds or clips, or perform a paired significance test.","section":"Table 1 and Table 3"},{"comment":"Global normalization uses the 5th/95th percentiles of 'the entire video sequence.' In the LongVGenBench evaluation, this means the depth statistics of the ground-truth target video are used to normalize the control signals, leaking information about the target and making the reconstruction task easier. In a realistic control-following scenario, the user provides the control signals, and normalization would be computed over those signals, not over the desired output. The authors should clarify how global normalization is applied when controls are not derived from the target, and ideally evaluate the effect of using global statistics estimated from user-provided controls rather than from the target video.","section":"Section 2.2 (Global Normalization) and Section 3.1"},{"comment":"The benchmark is described as '100 one-shot videos' but immediately afterwards as including 'challenging cases such as rapid scene transitions.' A one-shot video, by definition, contains no cuts, whereas rapid scene transitions imply cuts or abrupt changes. This inconsistency affects the interpretation of temporal-consistency metrics: if the videos contain cuts, it is unclear how consistency is evaluated across those transitions, especially since the training data explicitly removes scene cuts (Supplementary Section C). The authors should clarify whether the benchmark contains cuts, and if so, how the metrics treat them.","section":"Section 3.1 (LongVGenBench description)"}],"minor_comments":[{"comment":"The text says 'Equation 3 can be reformulated as' before Equation (3) is introduced; the intended reference is likely to Equation (2). Please renumber or rephrase.","section":"Section 2.2 (Feature-level degradation)"},{"comment":"The 'Random Noise' row reports Subject Consistency 86.39%, which matches the 'w/o Global Normalization' value in Table 3 (86.39%), not the 'w/o Unified Initial Noise' value (86.63%). This numerical inconsistency should be fixed or explained.","section":"Table 7"},{"comment":"The text says 'we first extract the depth map for the full 1-minute sequence' during inference, but in a real controllable generation setting the full sequence depth maps would be user inputs, not extracted from a ground-truth video. The inference procedure should be described in terms of user-provided controls.","section":"Supplementary Section C (Test Time Adaptation)"},{"comment":"The user study has 60 participants and 80 samples, but no inter-rater agreement or confidence intervals are reported. Given the small quantitative margins elsewhere, reporting variance would strengthen the subjective evaluation.","section":"Section 3.1 (User Study)"}],"recommendation":"major_revision","confidential_remarks":"The paper comes from a well-known group and addresses a timely problem, but the evaluation protocol for controllability is the central weakness. The reconstruction-style benchmark does not match the practical claim of following user-specified controls, and the reported margins are small. A cross-video control-following experiment and statistical error bars would substantially strengthen the paper. The benchmark itself could be a useful resource if properly described and released."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper right away. First, it is a genuinely useful engineering contribution: it packages several known ideas—autoregressive clip generation, unified noise initialization, global control normalization, ControlNet-style multi-modal conditioning, and a degradation-aware training strategy—into a system that demonstrably produces one-minute controllable videos. Second, the main evaluation protocol is a reconstruction test, and that weakens the headline controllability claim.\n\nWhat is actually new here is the combination and the specific degradation-aware training scheme, plus the LongVGenBench dataset. The ablations support the component choices: removing global normalization or unified noise drops consistency metrics by a few tenths of a point, and removing the degradation strategy drops imaging quality by about a full point. The SSIM/LPIPS gains over Depth-LV (0.557 vs 0.508, 0.290 vs 0.347) are nontrivial, and the user study shows a clear preference across all five dimensions. That is real evidence, and it should not be discounted.\n\nThe soft spots are real but not fatal. The benchmark extracts control signals from the target video and then asks the model to reconstruct that same video. This is an in-distribution inversion test; it does not tell you how the system follows arbitrary user-specified or out-of-distribution controls, which is what 'controllability' means in practice. The VBench metric margins over Depth-LV are mostly under one percentage point, and there are no error bars or significance tests, so the state-of-the-art claim rests on thin margins plus a favorable protocol. The 'first autoregressive framework' claim is also overstated, given StreamingT2V and other autoregressive long-video works. And there is no code or data release yet, which matters for a benchmark-heavy paper.\n\nWho gets value from this? Anyone working on controllable video generation or long-horizon consistency. The paper deserves a serious referee, but the authors should add a cross-video control-following experiment—use controls from one video to constrain generation of a different video with the same structure—and report variance across seeds. That would close the gap between the claim and the evidence. I would send it to review with that request rather than desk-reject.","headline":"A useful engineering contribution with a real evaluation gap: LongVie's benchmark measures reconstruction from self-extracted controls, so the SOTA controllability claim outruns the evidence, but the components are sensible and the paper deserves a careful peer review.","tokens_in":15490,"tokens_out":1853,"would_cite":true,"duration_ms":21688,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LongVie claims state-of-the-art one-minute controllable video generation by sharing one noise seed and normalizing control signals across the whole sequence.","keywords":["controllable video generation","long video generation","autoregressive generation","diffusion models","multi-modal control","temporal consistency","noise initialization","LongVGenBench"],"falsifier":"A crossover test would settle it: take the depth map and point trajectories from video A, feed them as the control input while generating video B, and check whether the output follows A's geometry and motion while matching B's appearance. If consistency and reconstruction scores collapse when the controls do not come from the target video itself, the reconstruction-based benchmark numbers do not establish general controllability; if they survive, the claim of arbitrary control-following at minute scale is confirmed.","tokens_in":14465,"feed_emoji":"🎬","tokens_out":11015,"duration_ms":119168,"temperature":0.7,"pith_summary":"LongVie is an autoregressive framework that generates controllable videos up to one minute long, a regime where existing control-conditioned video diffusion models drift: adjacent clips disagree in appearance and later clips look progressively worse. The paper attributes the drift to three causes — per-clip normalization of the control signal, independently sampled noise for each clip, and reliance on a single control modality — and proposes a remedy for each. Its central assertion is that sharing one noise instance across all clips, normalizing the depth control once over the entire minute, combining dense depth with sparse keypoint control, and degrading the dense input during training let a frozen text-to-video backbone stay temporally consistent and visually stable at length. If the assertion is right, fine-grained structural control extends from short clips to minute-scale footage, which supports long video editing, motion and scene transfer, and mesh-to-video rendering.","feed_headline":"One noise seed keeps AI video consistent for a full minute","feed_subtitle":"LongVie pairs depth and keypoint control with video-wide normalization to keep quality from fading across clips.","key_machinery":"The carrying mechanism is a frozen text-to-video diffusion transformer (CogVideoX) combined with two alignment tricks and two balancing tricks. Unified noise initialization draws a single latent noise tensor and reuses it as the starting point of every clip, so the generative dynamics are identical across segments; global control normalization computes the 5th and 95th percentiles of depth values over the whole one-minute sequence and scales every clip into that fixed range, so the control space has one consistent reference scale. Multi-modal control injection runs two lightweight trainable branches, one reading depth maps and one reading colorized keypoint trajectories (point maps), whose outputs are added into the frozen transformer stream through a unified zero-initialized linear layer, so conditioning enters without disturbing the pretrained backbone at initialization. Degradation-aware training perturbs the dense branch with probabilities $\\alpha$ and $\\beta$, randomly scaling its latent features and applying multi-scale fusion and blur to its input, which stops the dense modality from overwhelming the sparse one as video length grows.","core_discovery":"The paper's central claim is that the two failure modes of autoregressive long-video control — temporal inconsistency between clips and progressive visual degradation — have identifiable, fixable causes. Inconsistency comes from clip-wise control normalization, which rescales the same scene differently in successive clips, and from per-clip random noise, which changes the generative structure prior at every boundary. Degradation comes from single-modality guidance: dense depth dominates and accumulates errors, while sparse keypoints lack the spatial detail to anchor quality. LongVie counters the first pair with global percentile normalization of the depth signal over the full minute and one shared noise instance reused as the latent start of every clip, and the second pair with parallel trainable branches for depth and point maps fused into the frozen backbone through a unified zero-initialized linear layer, trained with feature- and data-level degradation of the dense input so the model learns to use both modalities. On its own LongVGenBench of 100 one-minute 1080p videos, LongVie reports the best consistency, quality, and SSIM/LPIPS reconstruction scores among the compared baselines, and a user study in which it ranks first on all five rated dimensions.","pith_inferences":["The benchmark extracts control signals from the very videos the model is scored against, so the reported numbers measure reconstruction of an in-distribution target; following genuinely arbitrary user controls — depth and keypoints drawn by hand, or borrowed from a different video — is the untested case, and would be the sharper test of the controllability claim.","Because both consistency fixes are inference-side and backbone-agnostic, the framework suggests a cheap recipe for upgrading short-clip controllable models to long-form: keep the base model frozen, share the noise plan, and normalize controls over the whole sequence.","The degradation-aware balance is trained on only one modality pair (depth and keypoints) in this paper; a testable extension is whether the same schedule generalizes to other dense/sparse pairs, such as canny edges with segmentations, or semantic maps with bounding boxes.","The paper's own limitation section puts inference at roughly 45 minutes per minute of video at 480x720, so the practical ceiling on controllable long video is sampling cost rather than control stability; combining the framework with faster samplers or distillation is the evident next step."],"forward_implications":["The two consistency fixes operate on inference-time inputs only, so any autoregressive controllable video generator built on a frozen backbone could adopt shared noise and global normalization without retraining the base model.","The balancing schedule is what protects long-run quality: in the ablations, dropping either the feature- or data-level degradation lowers the imaging-quality score by about a point, and dropping both lowers it by 1.24, while consistency metrics barely move.","Because depth and point-map conditioning drive all three downstream tasks, the same framework extends to long-range video editing, motion and scene transfer, and photorealistic rendering from animated 3D meshes, each starting from a single edited or synthesized first frame.","LongVGenBench gives the field a common measurement ground: 100 one-minute, 1080p, one-shot videos spanning real and game scenes, split into ten 49-frame clips, so that consistency, quality, and control fidelity can be compared at minute scale rather than on short clips."],"supporting_citations":[{"why":"Base text-to-video diffusion transformer whose DiT blocks LongVie freezes and extends with trainable control branches.","marker":"[37]"},{"why":"ControlNet-style architecture pattern of duplicated conditioning blocks and zero-initialized fusion layers that LongVie adapts for multi-modal injection.","marker":"[39]"},{"why":"Supplies the point-map construction procedure — keypoints tracked across frames and colorized by depth — that LongVie uses as its sparse control signal.","marker":"[11]"},{"why":"Video Depth Anything extracts the per-frame depth maps that serve as LongVie's dense control signal.","marker":"[6]"},{"why":"SpatialTracker computes the 3D point trajectories from depth and RGB that become LongVie's sparse control input.","marker":"[36]"},{"why":"VBench provides the seven evaluation metrics, including subject, background, and overall consistency, that anchor the quantitative comparison.","marker":"[18]"},{"why":"StreamingT2V is the image-driven autoregressive long-video baseline that LongVie must outperform on consistency and quality metrics.","marker":"[14]"},{"why":"FLUX supplies the fill and depth-to-image tools used to edit first frames and synthesize initial frames in transfer tasks.","marker":"[19]"}],"fun_headline_variants":["LongVie: one noise seed and global depth scale keep minute-long video consistent","Fixing video drift: shared noise and global normalization keep 1-min clips crisp","LongVie: multi-modal control stops visual decay in autoregressive video","One seed, global depth scale, and dual-modality control make 1-min video coherent","LongVie: end-to-end autoregressive control for minute-long video without fade"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that regenerating a video from control signals extracted from that same video fairly measures controllability; if following arbitrary, out-of-distribution user controls is what matters in practice, the reported state-of-the-art results do not yet establish it.","fun_headline_variants_meta":{"raw":{"variants":["LongVie: one noise seed and global depth scale keep minute-long video consistent","Fixing video drift: shared noise and global normalization keep 1-min clips crisp","LongVie: multi-modal control stops visual decay in autoregressive video","One seed, global depth scale, and dual-modality control make 1-min video coherent","LongVie: end-to-end autoregressive control for minute-long video without fade"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001086,"raw_usage":{"total_tokens":4574,"prompt_tokens":1015,"completion_tokens":3559,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":631,"completion_tokens_details":{"reasoning_tokens":3463}},"tokens_in":631,"tokens_out":3559,"duration_ms":27852,"temperature":1.0,"reasoning_tokens":3463,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T04:12:13.851934+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A crossover test would settle it: take the depth map and point trajectories from video A, feed them as the control input while generating video B, and check whether the output follows A's geometry and motion while matching B's appearance. If consistency and reconstruction scores collapse when the controls do not come from the target video itself, the reconstruction-based benchmark numbers do not establish general controllability; if they survive, the claim of arbitrary control-following at minute scale is confirmed.","supporting_citations":[{"cited_title":"Spatialtracker: Tracking any 2d pixels in 3d space","cited_arxiv_id":null,"evidence_quote":"SpatialTracker computes the 3D point trajectories from depth and RGB that become LongVie's sparse control input."}],"review_version":1}