Pith. sign in

REVIEW 4 major objections 4 minor 30 cited by

Waver: Wave Your Way to Lifelike Video Generation

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Waver is a unified text-to-video, image-to-video, and text-to-image model that ranks in the top three on public T2V and I2V leaderboards and beats open-source rivals, with its largest edge in complex motion.

desk verdict A dense industrial recipe with a genuinely useful hybrid-stream design, but the headline quality claims rest on unreleased internal evals, so treat it as a training-report contribution rather than a verified SoTA claim. read the letter →

arxiv 2508.15761 v2 pith:GLSPWZHV submitted 2025-08-21 cs.CV

Yifu Zhang , Hao Yang , Yuqi Zhang , Yifei Hu , Fengda Zhu , Chuang Lin , Xiaofeng Mei , Yi Jiang
show 2 more authors
Bingyue Peng Zehuan Yuan
This is my paper
classification cs.CV
keywords text-to-videogenerationimage-to-videorectifiedflowtransformerHybridStreamDiTdatacurationpipelinehumanevaluationvideosuper-resolutionmotionquality
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Waver is a single foundation model that generates 5–10 second videos at native 720p (upscaled to 1080p) for both text-to-video and image-to-video, plus text-to-image, without separate task-specific models. The paper's central claim is that this unified design, trained on a carefully filtered pool of more than 200 million clips with detailed captions, reaches top-three quality on the public Artificial Analysis T2V and I2V leaderboards and matches or beats commercial systems such as Veo3 and Kling2.0, especially on complex motion like sports. The report's added value is a complete, stage-by-stage recipe: architecture, data funnel, loss weights, noise schedules, and infrastructure choices. If the claims hold, Waver provides one of the most reproducible paths to industry-grade video generation and demonstrates that motion quality can be engineered through data balance and timestep sampling as much as through model scale.

What carries the argument

Task-Unified DiT: a rectified-flow transformer with a dual-encoder text conditioner (T5-XXL plus Qwen2.5-32B) and a three-part input tensor—noisy latent V, conditioning frames I, and binary mask—so T2I, T2V, and I2V share one network. Hybrid Stream: the first M layers use Dual Stream blocks (separate text/video parameters, merged only in self-attention) and the next N layers use Single Stream blocks (shared parameters), which the paper shows converges faster than either pure design. Cascade Refiner: a window-attention flow-matching model that takes degraded 720p video and learns the transition to 1080p, mixing pixel and latent degradation to fix artifacts. Supporting knobs: 3D RoPE plus fact

What would settle it

Run a preregistered blind side-by-side evaluation on an independently sampled set of complex-motion prompts, with Waver versus Veo3 and Wan2.1, reporting confidence intervals and inter-rater agreement; if Waver does not win on motion quality and prompt following, the central claim is refuted.

Watch

Extended reading notes

Core claim

On its own terms, Waver discovers that one rectified-flow DiT can serve T2I, T2V, and I2V by concatenating a noisy latent, conditional frames, and a binary mask; that a Hybrid Stream of dual-stream blocks followed by single-stream blocks converges faster than either pure design; and that motion amplitude grows when the model is pretrained at 192p, when timesteps are drawn from a sharply peaked mode distribution, and when I2V is jointly trained with T2V. A Cascade Refiner then upscales 720p output to 1080p with 40% less compute than direct 1080p generation. On Waver's internal benchmarks, side-by-side human evaluation gives Waver wins over Wan2.1 and Kling2.0 on motion, visual quality, and pr

Load-bearing premise

The headline comparisons rest on an internal human evaluation with no reported confidence intervals, rater counts, or inter-rater agreement, on two self-built benchmarks; if those judgments are biased or the benchmarks are unrepresentative, the claimed superiority over Veo3, Kling2.0, and Wan2.1 would not generalize.

Editorial extensions

If this is right

  • Other video labs can adopt the full recipe—stage-by-stage resolutions, data volumes, learning rates, and sigma shifts—to train comparable unified models without separate T2V/I2V towers.
  • A single Waver checkpoint can produce T2I, T2V, and I2V outputs, so task-specific models and duplicated training overhead become unnecessary in this design.
  • The 720p-to-1080p Cascade Refiner yields a 40% speedup over direct 1080p generation while also correcting artifacts, making high-resolution generation cheaper.
  • Mode-based timestep sampling, 192p motion pretraining, and T2V/I2V joint training each increase motion amplitude without hurting visual quality, per the paper's ablations.
  • The MLLM-based quality filter and hierarchical data funnel let the model train on 200M+ clips while keeping high-quality, balanced data, which the paper credits for complex-motion performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 192p low-resolution motion-pretraining stage is the most transferable trick: it decouples motion learning from aesthetics, and other teams with smaller compute could test it first on their own backbones.
  • A direct extension is to apply Waver's mode timestep sampling and joint T2V/I2V training to existing open backbones to test whether the motion gains transfer without the full data pipeline.
  • The hybrid-stream split (dual for the first M layers, single for the rest) suggests a scaling rule of thumb for other modalities: parameter-heavy alignment at the bottom, shared computation at the top.
  • Because the Cascade Refiner can also edit objects at high degradation weights, it may be reusable as a general video editing and latent-denoising module rather than only a super-resolver.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This technical report describes Waver, a 12B-parameter unified image/video generation model that supports T2I, T2V, and I2V with a single Task-Unified DiT and a Cascade Refiner for 720p-to-1080p upscaling. The authors present the Hybrid Stream DiT architecture, a hierarchical data curation pipeline with an MLLM-based quality model and detailed captioning, and an extensive multi-stage training and inference recipe. The central claim is that Waver achieves superior motion amplitude, temporal consistency, and prompt following, ranking in the Top 3 on both T2V and I2V leaderboards at Artificial Analysis and outperforming open-source baselines while matching commercial systems. The paper also includes infrastructure details, ablation studies, and a discussion of DiT sparsity, VAE design, and captioning quality.

Significance. If the performance claims are reliable, Waver is a significant engineering contribution: it provides a complete, detailed recipe for training a state-of-the-art unified video generation model, including data curation, multi-stage training, inference-time optimizations, and infrastructure choices. The report is unusually transparent about data volumes, learning rates, sigma shifts, and ablation settings, which is valuable for reproducibility. However, the evidence for the headline superiority claims is not yet commensurate with their strength. The central evaluation rests on unreleased self-built benchmarks and an internal human evaluation with no error bars, rater details, or third-party verification. Several margins are within plausible noise, and the admitted high-motion distortion in Sec. 8 conflicts with the motion-quality dominance claim. The useful recipe and architecture contributions deserve publication, but the benchmark-based claims need substantial strengthening before they can be accepted as stated.

major comments (4)
  1. [§6.2, Fig. 28] The headline 'consistently outperforming' claim rests entirely on the internal side-by-side human evaluation. No rater counts, per-item vote totals, confidence intervals, inter-rater agreement, or details on blinding/order randomization are reported, and the baselines' inference settings are unspecified. Some reported differences are within expected noise: on Waver-bench 1.0, Waver vs. Kling 2.0 motion quality is 31% Waver, 39% tie, 30% Kling (Fig. 28a). A 1-point raw difference with 39% ties cannot support 'significantly superior motion quality' without a statistical test. Please report variance estimates, per-item tallies, and release or otherwise make accessible the Waver-bench 1.0 and Hermes Motion Testset prompts and evaluation protocol.
  2. [§8 vs. §6.2] The Limitation section concedes that 'in high-motion scenarios, human body details such as hands and legs are prone to distortion,' while §6.2 defines Motion Quality to include Distortion (artifacts, blur, and subject consistency). Claiming 'dominant performance' on a motion-stress testset while acknowledging systematic distortion in high-motion body detail is internally tensioned. The authors should clarify which motion sub-dimensions improve and which degrade, and report Distortion separately from motion amplitude and action accuracy.
  3. [§6.1, Fig. 25, Abstract] The leaderboard date is inconsistent: the abstract says 2025-07-30, the §6.1 text says 2025-07-22, and the Fig. 25 caption says 2025-08-05. Since a leaderboard ranking is time-varying and configuration-dependent, this is not a cosmetic issue. Please provide a single canonical snapshot with an archived URL, the exact model version evaluated, and the Elo scores or vote counts behind the 'Top 3' claim.
  4. [§4.2–§4.5] Several load-bearing recipe choices are supported only by qualitative figures or by the same internal head-to-head protocol used for the final claim. Representation alignment is shown only in Fig. 11; mode sampling in Fig. 13; APG in Fig. 21; model averaging in Fig. 22 with no confidence intervals. The central superiority claim would be far more convincing if these ablations were quantified on a held-out set with error bars, or if the resulting checkpoints and self-built benchmarks were released for independent verification.
minor comments (4)
  1. [§4.3, Eq. (2), Fig. 14] Figure 14's caption refers to the mode distribution as '(Eq. 1)', but the mode distribution is Eq. (2); Eq. (1) is the logit-normal. Also define the parameters in Lognorm(0.5, 1) and Mode(1.29) explicitly.
  2. [§4.4, Fig. 18b] The text says 'a 7% improvement' after aesthetic finetuning, but the reported numbers are 39.5% vs. 32.9% (a 6.6 percentage-point difference). Please clarify whether the claim is percentage points or relative improvement.
  3. [§6.2, Fig. 28] The legend label 'Others better' is ambiguous; 'Competitor better' would be clearer. Also, the figure captions do not report the number of human votes, which should accompany stacked-bar proportions.
  4. [§2.2] The text says the refiner is 'based on Waver1.0', but Waver is introduced in this paper as the current model. Specify which checkpoint/stage 'Waver1.0' refers to, or rename it to avoid confusion.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; two minor non-load-bearing self-citations in design sections.

full rationale

This paper is a technical report of an empirical training run, not a mathematical derivation, so there is no equation chain in which an output is shown to equal an input. The central claim (state-of-the-art motion quality) is supported by (i) an external leaderboard (Artificial Analysis) and (ii) human side-by-side evaluations against Veo3, Kling2.0, and Wan2.1. Neither of those evaluations is a fitted parameter renamed as a prediction, and no reported win rate is forced by construction. The MLLM quality model is trained on 1M manually annotated clips and validated at 78% accuracy, so it is not self-validating. The self-citations that exist -- Goku for video segmentation (Sec. 3.1) and FlashVideo for the cascade refiner (Sec. 2.2) -- are design borrowings from the same team; they are not load-bearing for the performance claim, do not forbid alternative designs, and do not define the reported outcomes, so they do not constitute circularity. The reader's concern about tuning on the same 304-prompt benchmark is a selection-on-evaluation-benchmark risk, but the reported superiority is a head-to-head comparison against external models and is not a re-reporting of the selection metric; the lack of confidence intervals and rater counts is a statistical reporting deficiency, not a definitional circularity. Section 8's admission of limb distortion in high-motion scenes conflicts rhetorically with the motion-quality claim but is a consistency issue, not a circular one. Overall, the central derivation is self-contained and empirically evaluated; only minor non-load-bearing self-citations prevent a score of 0.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The paper's central claims depend on many tuned hyperparameters and assumptions about the validity of internal evaluations; no code or data is released to check them.

free parameters (7)
  • Timestep sampling parameters = Lognorm(0.5, 1) for T2I; Mode(1.29) for T2V/I2V
    Chosen based on internal ablations to increase motion amplitude (Sec. 4.3); affects all generation quality claims.
  • Representation alignment weight lambda = 0.5
    Set in Sec. 4.2; the ablation is qualitative, no sensitivity analysis.
  • I2V mixing probability = 20%
    Chosen to balance motion and condition fidelity in Sec. 4.3; no sweep reported.
  • Latent degradation weight range wd = [0.85, 0.95]
    Found empirically for refiner training in Sec. 2.2.
  • APG normalization threshold and guidance scale = 27 and 8
    Tuned in Sec. 4.5 to balance realism and artifacts.
  • Sigma shift schedule per stage = 1.0, 2.0, 3.0, 4.5
    Follows SD3 but values are adjusted per resolution (Table 2); affects training dynamics.
  • Architecture block counts = M=16 Dual, N=40 Single, 12B
    Chosen; no ablation shows optimality (Table 1).
assumptions (5)
  • domain assumption Flow matching (rectified flow) is an effective training objective for video generation.
    Adopted from SD3/Esser et al.; the paper's whole training is built on it without questioning.
  • domain assumption The Wan2.1 VAE provides a suitable latent space for video generation.
    Used as-is; the paper's VAE ablations only compare two other VAEs, not whether a better latent space exists.
  • domain assumption Human preference judgments on the authors' internal benchmarks are a valid proxy for generation quality.
    All headline claims rest on this; no external validation of the benchmarks.
  • domain assumption The MLLM quality model's 78% precision on 'high-quality' predictions is sufficient to improve training data.
    Sec. 3.2; no analysis of how false filtering affects downstream performance.
  • domain assumption Representation alignment with Qwen2.5-VL features improves semantic quality.
    Sec. 4.2; ablation is qualitative, no quantitative metric.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Waver: Wave Your Way to Lifelike Video Generation." pith.science (2026). https://pith.science/paper/GLSPWZHV

@misc{pith2026250815761,
  author       = {Pith},
  title        = {Pith review of: Waver: Wave Your Way to Lifelike Video Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GLSPWZHV}},
  note         = {Machine review of arXiv:2508.15761}
}
read the original abstract

We present Waver, a high-performance foundation model for unified image and video generation. Waver can directly generate videos with durations ranging from 5 to 10 seconds at a native resolution of 720p, which are subsequently upscaled to 1080p. The model simultaneously supports text-to-video (T2V), image-to-video (I2V), and text-to-image (T2I) generation within a single, integrated framework. We introduce a Hybrid Stream DiT architecture to enhance modality alignment and accelerate training convergence. To ensure training data quality, we establish a comprehensive data curation pipeline and manually annotate and train an MLLM-based video quality model to filter for the highest-quality samples. Furthermore, we provide detailed training and inference recipes to facilitate the generation of high-quality videos. Building on these contributions, Waver excels at capturing complex motion, achieving superior motion amplitude and temporal consistency in video synthesis. Notably, it ranks among the Top 3 on both the T2V and I2V leaderboards at Artificial Analysis (data as of 2025-07-30 10:00 GMT+8), consistently outperforming existing open-source models and matching or surpassing state-of-the-art commercial solutions. We hope this technical report will help the community more efficiently train high-quality video generation models and accelerate progress in video generation technologies. Official page: https://github.com/FoundationVision/Waver.

Figures

Figures reproduced from arXiv: 2508.15761 by the authors.

Figure 1
Figure 1. Left: Human evaluation win rates (GSB) of Waver compared to Veo3, Kling2.0, and Wan2.1 on the Waver-bench 1.0 Text-to-Video (T2V) dataset across three dimensions: motion quality, visual quality, and prompt following. Waver-bench 1.0 covers a wide range of scenarios, including sports, daily activities, landscapes, animals, animations, and more. Right: Human evaluation win rates (GSB) on the Hermes Motion Testset acro… view at source ↗
Figure 2
Figure 2. T2V samples generated by Waver. Waver is capable of generating 1080p videos at arbitrary aspect ratios, delivering high levels of aesthetic quality, realism, and motion fidelity, while simultaneously supporting both T2V and I2V tasks. 3 [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Architecture of Task-Unified DiT. 2.1 TASK-UNIFIED DIT Unified Input Formulation. To unify diverse generative tasks (T2I, T2V, I2V) within a single framework, we employ a flexible input conditioning mechanism based on a three-part input tensor. This input is formed by concatenating a primary noisy latent (V ), a conditional frames tensor (I), and a binary condition mask (M ask) along the channel dimension. The tenso… view at source ↗
Figures from the paper (30 more)
Figure 4
Figure 4. Figure 4: Comparison of 512p T2I results using different text encoders. For each case, image on the [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Loss comparison between Hybrid Stream, Dual Stream, and Single Stream structures. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Pipeline of Cascade Refiner. Window Attention We improve the efficiency of 1080p video generation by leveraging window attention to reduce computational costs. This mechanism partitions video tokens into local Wh × Ww × Wt windows and restricts attention calculations t…
Figure 7
Figure 7. Figure 7: Fig.(a) illustrates the Refiner’s output, where it has upscaled a 480p video from the first stage [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: An overview of our proposed data processing pipeline. The process consists of five main [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Overview of the data filtering process and dataset characteristics. [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: The hierarchical data filtering funnel, which progressively refines the dataset through [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 11
Figure 11. Figure 11: Qualitative comparison of generated videos with and without our representation alignment [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 12
Figure 12. Figure 12: Top: 480p T2V results without 192p pretrain. Bottom: 480p T2V results with 192p pretrain. Results for prompt “Two people are paddling two kayaks vigorously in the river.” against a baseline trained without the constraint. Both models were initialized from an identical…
Figure 13
Figure 13. Figure 13: Top: 720p T2V results with logit-normal sampling. Bottom: 720p T2V results with mode sampling. Results for prompt “In the center of the boxing ring, two male boxers are fighting each other.” [PITH_FULL_IMAGE:figures/full_fig_p015_13.png]
Figure 14
Figure 14. Figure 14: Probability density functions for timestep sampling. While Text-to-Image uses a broad logit-normal distribution (Eq. 1), Text-to-Video and Image-to-Video tasks employ a sharply peaked mode distribution (Eq. 1) to generate larger motion amplitudes. πln(t; m, s) = 1 s √…
Figure 15
Figure 15. Figure 15: Top: 720p I2V results without T2V joint training. Bottom: 720p I2V results with T2V joint training. Results for prompt “A man is skiing.” [PITH_FULL_IMAGE:figures/full_fig_p016_15.png]
Figure 16
Figure 16. Figure 16: Raw training video clips and the foreground optical flow. [PITH_FULL_IMAGE:figures/full_fig_p016_16.png]
Figure 17
Figure 17. Figure 17: Aesthetic comparison for a running scene. [PITH_FULL_IMAGE:figures/full_fig_p017_17.png]
Figure 18
Figure 18. Figure 18: Statistics and impact of our synthetic data enhancement stage. [PITH_FULL_IMAGE:figures/full_fig_p018_18.png]
Figure 19
Figure 19. Figure 19: Comparison of four videos generated by models before and after high-aesthetic finetuning. [PITH_FULL_IMAGE:figures/full_fig_p018_19.png]
Figure 20
Figure 20. Figure 20: Six different styles of videos generated by [PITH_FULL_IMAGE:figures/full_fig_p019_20.png]
Figure 21
Figure 21. Figure 21: Top: 720p T2V results with CFG=5. Bottom: 720p T2V results with the optimal APG hyperparameters (CFG=8, norm threshold=27). Results for prompt “An off-road motorcyclist speeds along a dusty motocross track, with the tires kicking up dirt and gravel.” 4.6 PROMPT REWRIT…
Figure 22
Figure 22. Figure 22: Human evaluation (SBS) results before and after model merging. Motion quality improves [PITH_FULL_IMAGE:figures/full_fig_p020_22.png]
Figure 23
Figure 23. Figure 23: Top: 720p T2V results generated with the original prompt. Bottom: 720p T2V results generated with the rewritten prompt. 5 INFRASTRUCTURE Hybrid sharded mode of FSDP. Since Fully Sharded Data Parallel (FSDP) is recognized for its simplicity and flexibility in large-sca…
Figure 24
Figure 24. Figure 24: Illustrative diagrams of Ulysses sequence parallelism, Selective Activation Checkpointing, [PITH_FULL_IMAGE:figures/full_fig_p021_24.png]
Figure 25
Figure 25. Figure 25: On the left and right are the official T2V and I2V ranking results from Artificial Analysis [PITH_FULL_IMAGE:figures/full_fig_p022_25.png]
Figure 26
Figure 26. Figure 26: Some video examples of the Waver-Bench 1.0 generated by [PITH_FULL_IMAGE:figures/full_fig_p023_26.png]
Figure 27
Figure 27. Figure 27: Some video examples of the Hermes Motion Testset generated by [PITH_FULL_IMAGE:figures/full_fig_p023_27.png]
Figure 28
Figure 28. Figure 28: Human evaluation results comparing our model (Waver) with leading competitors. Users were presented with side-by-side video comparisons and asked to choose which was better or if they were tied. The stacked bar charts show the proportion of user preference votes acros…
Figure 29
Figure 29. Figure 29: An illustration of attention maps is presented, with the query on the vertical axis and the [PITH_FULL_IMAGE:figures/full_fig_p025_29.png]
Figure 30
Figure 30. Figure 30: Comparison of generated video quality with different VAEs on the I2V task, where the [PITH_FULL_IMAGE:figures/full_fig_p026_30.png]
Figure 31
Figure 31. Figure 31: Comparison of the latent distributions and reconstructed images between CogVideoX [PITH_FULL_IMAGE:figures/full_fig_p026_31.png]
Figure 32
Figure 32. Figure 32: Ablation study of video generation results using VAEs trained with different LPIPS losses. [PITH_FULL_IMAGE:figures/full_fig_p027_32.png]
Figure 33
Figure 33. Figure 33: Top: The results of the T2V model trained using the Tarsier2 caption model. Bottom: The outcomes of the T2V model trained with our proposed caption model. In case (a), there is a noticeable distortion in the legs in the top row, which is absent in the bottom row. In c…

Discussion (0). Sign in to comment.

Forward citations

Cited by 30 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. TransVLM: A Vision-Language Framework and Benchmark for Detecting Any Shot Transitions

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    TransVLM formalizes Shot Transition Detection as identifying full temporal transition segments rather than single cut points and introduces a VLM that injects optical flow as a motion prior via simple feature fusion, ...

  2. Generative Refinement Networks for Visual Synthesis

    cs.CV 2026-04 accept novelty 7.0 of 10

    Hierarchical Binary Quantization plus global refinement AR yields 0.56 rFID reconstruction and 1.81 gFID class-conditional generation on ImageNet, with competitive T2I/T2V at 2B scale.

  3. FlashDecoder: Real-Time Latent-to-Pixel Streaming Decoder with Transformers

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A pure-Transformer streaming decoder with a fixed-size rolling KV cache decodes video latents 3.6–4.7x faster than 3D convolutional decoders with comparable PSNR.

  4. SpecLoR: Spectral Lookahead Rectification for Motion-Coherent Text-to-Video Generation

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    SpecLoR rectifies the amplitude spectrum of lookahead-estimated clean latents to natural-video priors during early ODE sampling steps, cutting physical artifacts with only four extra NFEs.

  5. Echo-Memory: A Controlled Study of Memory in Action World Models

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    A controlled study finds that block-wise state-space recurrence outperforms other memory designs for open-domain scene return in action-conditioned video models, and that standard replay metrics do not adequately meas...

  6. VidSplat: Gaussian Splatting Reconstruction with Geometry-Guided Video Diffusion Priors

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    VidSplat iteratively synthesizes novel views with geometry-guided video diffusion to enable robust Gaussian splatting reconstruction from sparse or single-image inputs.

  7. Generative Refinement Networks for Visual Synthesis

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    GRN uses hierarchical binary quantization and entropy-guided refinement to set new ImageNet records of 0.56 rFID for reconstruction and 1.81 gFID for class-conditional generation while releasing code and models.

  8. OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generation

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    OmniShow unifies text, image, audio, and pose conditions into an end-to-end model for high-quality human-object interaction video generation and introduces the HOIVG-Bench benchmark, claiming state-of-the-art results.

  9. Long-Horizon Streaming Video Generation via Hybrid Attention with Decoupled Distillation

    cs.CV 2026-04 conditional novelty 6.0 of 10

    Hybrid Forcing combines linear temporal attention for long-range retention, block-sparse attention for efficiency, and decoupled distillation to achieve real-time unbounded 832x480 streaming video generation at 29.5 FPS.

  10. Lighting-grounded Video Generation with Renderer-based Agent Reasoning

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    LiVER conditions video diffusion models on renderer-derived 3D control signals for disentangled, editable control over object layout, lighting, and camera trajectory.

  11. Rethinking Position Embedding as a Context Controller for Multi-Reference and Multi-Shot Video Generation

    cs.CV 2026-04 conditional novelty 6.0 of 10

    SideInfo-RoPE encodes reference-identity agreement as an extra rotary axis, disambiguating similar characters in multi-reference multi-shot video generation while keeping full semantic attention.

  12. Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithms

    eess.IV 2026-03 unverdicted novelty 6.0 of 10

    Video generation models can function as world simulators if efficiency gaps in spatiotemporal modeling are bridged via organized paradigms, architectures, and algorithms.

  13. AR-CoPO: Align Autoregressive Video Generation with Contrastive Policy Optimization

    cs.CV 2026-03 conditional novelty 6.0 of 10

    Chunk-level forking plus semi-on-policy contrastive GRPO aligns few-step AR video models, raising VideoAlign and holding VBench versus Self-Forcing baselines.

  14. Cross-Resolution Distribution Matching for Diffusion Distillation

    cs.CV 2026-03 conditional novelty 6.0 of 10

    Cross-resolution distribution matching with logSNR timestep alignment and predicted-noise re-injection enables high-fidelity few-step multi-resolution cascaded diffusion distillation.

  15. PhyGDPO: Physics-Aware Groupwise Direct Preference Optimization for Physically Consistent Text-to-Video Generation

    cs.CV 2025-12 conditional novelty 6.0 of 10

    PhyGDPO uses groupwise direct preference optimization with real videos as winners to make text-to-video models generate more physically plausible videos.

  16. Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformer

    cs.CV 2025-11 conditional novelty 6.0 of 10

    A 6B single-stream diffusion transformer trained with heavily curated data reaches top open-source image-generation quality in 314K H800 GPU hours, releasing Turbo and Edit variants.

  17. Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence

    cs.CV 2026-07 conditional novelty 5.0 of 10

    LingBot-Video is an open-source MoE video foundation model for embodied intelligence that scales to 120B parameters, integrates robot data, and uses multi-dimensional RL to improve physical plausibility.

  18. From SRA to Self-Flow: Data Augmentation or Self-Supervision?

    cs.CV 2026-07 unverdicted novelty 5.0 of 10

    Attention Separation ablations show that gains from SRA to Self-Flow in diffusion transformers arise mainly from noise-dimension data augmentation rather than token-level self-supervision.

  19. Ultra Flash: Scaling Real-Time Streaming Video Generation to High Resolutions

    cs.CV 2026-06 unverdicted novelty 5.0 of 10

    Ultra Flash introduces a cascaded streaming super-resolution framework with specialized training, upsampling, and optimization to enable real-time high-resolution video generation from low-res diffusion models.

  20. PixelWizard: Towards Efficient High-Fidelity Video Generation at Ultra-Large Spatial Resolution

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    PixelWizard decouples global structure from fine details via a spatiotemporal anchor and introduces Noise-Span Aligned Shortcut Training with biased sampling to achieve over 10x faster sampling for high-fidelity 2K/4K...

  21. Bernini: Latent Semantic Planning for Video Diffusion

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    Bernini is a framework that uses an MLLM planner to output semantic representations for a DiT renderer to generate or edit videos, reporting SOTA benchmark performance.

  22. Motif-Video 2B: Technical Report

    cs.CV 2026-04 unverdicted novelty 5.0 of 10

    Motif-Video 2B achieves 83.76% VBench score, beating a 14B-parameter baseline with 7x fewer parameters and substantially less training data through shared cross-attention and a three-part backbone.

  23. Tora3: Trajectory-Guided Audio-Video Generation with Physical Coherence

    cs.CV 2026-04 unverdicted novelty 5.0 of 10

    Tora3 uses shared object trajectories as kinematic priors to jointly guide visual motion and acoustic events in audio-video generation, improving realism and synchronization.

  24. Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithms

    eess.IV 2026-03 conditional novelty 5.0 of 10

    In twisted bilayer nodal d-wave superconductors, interlayer hopping creates nodes on the C2 axis and Bogoliubov flat bands when the single-layer Berry connection is parallel to that axis.

  25. JoVA: Unified Multimodal Learning for Joint Video-Audio Generation and Editing

    cs.CV 2025-12 conditional novelty 5.0 of 10

    A dual-branch diffusion transformer with joint video-audio self-attention and a keypoint-based mouth-area loss reports top lip-sync and speech metrics on two benchmarks.

  26. Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformer

    cs.CV 2025-11 unverdicted novelty 5.0 of 10

    Z-Image is an efficient 6B-parameter foundation model for image generation that rivals larger commercial systems in photorealism and bilingual text rendering through a new single-stream diffusion transformer and strea...

  27. Motif-Video 2B: Technical Report

    cs.CV 2026-04 unverdicted novelty 4.0 of 10

    Motif-Video 2B reaches 83.76% on VBench, outperforming a 14B-parameter model with 7x fewer parameters and far less training data through shared cross-attention and a three-part backbone.

  28. Advancing Open-source World Models

    cs.CV 2026-01 unverdicted novelty 4.0 of 10

    LingBot-World is presented as an open-source world model that delivers high-fidelity simulation, minute-level contextual consistency, and real-time interactivity under one second latency.

  29. World Simulation with Video Foundation Models for Physical AI

    cs.CV 2025-10 unverdicted novelty 4.0 of 10

    Cosmos-Predict2.5 unifies text-to-world, image-to-world, and video-to-world generation in one model trained on 200M clips with RL post-training, delivering improved quality and control for physical AI.

  30. Image-to-Video Diffusion: From Foundations to Open Frontiers

    cs.CV 2026-05 unverdicted novelty 3.0 of 10

    A survey that organizes diffusion image-to-video methods into a taxonomy, distills core designs in condition encoding, temporal modeling, noise prior, and upsampling, and discusses applications plus challenges.

Reference graph

Works this paper leans on

24 extracted references · 1 canonical work pages · cited by 26 Pith papers

  1. [1]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923 ,

    arXiv

  2. [3]

    PySceneDetect Developers

    https://deepmind.google/technologies/veo/veo-3/, 2025.05. PySceneDetect Developers. Pyscenedetect. URL https://www.scenedetect.com. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers ...

    arXiv 2025

  3. [5]

    Emu video: Factorizing text-to-video generation by explicit image conditioning

    Rohit Girdhar, Mannat Singh, Andrew Brown, Quentin Duval, Samaneh Azadi, Sai Saketh Rambhatla, Akbar Shah, Xi Yin, Devi Parikh, and Ishan Misra. Emu video: Factorizing text-to-video generation by explicit image conditioning. arXiv preprint arXiv:2311.10709,

    arXiv

  4. [7]

    Hunyuanvideo: A systematic framework for large video generative models

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603,

    arXiv

  5. [8]

    Efficient sequence packing without cross-contamination: Accelerating large language models without impacting performance

    Mario Michael Krell, Matej Kosec, Sergio P Perez, and Andrew Fitzgibbon. Efficient sequence packing without cross-contamination: Accelerating large language models without impacting performance. arXiv preprint arXiv:2107.02027,

    arXiv

  6. [10]

    Dynamic-i2v: Exploring image-to-video generaion models via multimodal llm

    Peng Liu, Xiaoming Ren, Fengkai Liu, Qingsong Xie, Quanlong Zheng, Yanhao Zhang, Haonan Lu, and Yujiu Yang. Dynamic-i2v: Exploring image-to-video generaion models via multimodal llm. arXiv preprint arXiv:2505.19901,

    arXiv

  7. [11]

    Step-video-t2v technical report: The practice, challenges, and future of video foundation model

    Guoqing Ma, Haoyang Huang, Kun Yan, Liangyu Chen, Nan Duan, Shengming Yin, Changyi Wan, Ranchen Ming, Xiaoniu Song, Xing Chen, et al. Step-video-t2v technical report: The practice, challenges, and future of video foundation model. arXiv preprint arXiv:2502.10248,

    arXiv

  8. [12]

    Hailuo ai

    MiniMax. Hailuo ai. https://hailuoai.com/video, 2024.09. OpenAI. Video generation models as world simulators,

    work page 2024

Show all 24 references
  1. [13]

    30 Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al

    URL https://openai.com/ index/video-generation-models-as-world-simulators/ . 30 Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust vi...

    arXiv

  2. [14]

    Smaug: Fixing failure modes of preference optimisation with dpo-positive

    Arka Pal, Deep Karkhanis, Samuel Dooley, Manley Roberts, Siddartha Naidu, and Colin White. Smaug: Fixing failure modes of preference optimisation with dpo-positive. arXiv preprint arXiv:2402.13228,

    arXiv

  3. [15]

    Sampson, Shikai Li, Simone Parmeggiani, Steve Fine, Tara Fowler, Vladan Petrovic, and Yuming Du

    Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yao Ma, Ching-Yao Chuang, David Yan, Dhruv Choudhary, Dingkang Wang, Geet Sethi, Guan Pang, Haoyu Ma, Ishan Misra, Ji Hou, Jialiang Wang, Kiran Jagadeesh, Kunpeng Li, Lu...

    arXiv

  4. [16]

    github.io/blog/qwen2.5/

    URL https://qwenlm. github.io/blog/qwen2.5/. Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. arXiv preprint arXiv:2305.18290,

    arXiv

  5. [17]

    Seaweed-7b: Cost-effective training of video generation foundation model

    Team Seawead, Ceyuan Yang, Zhijie Lin, Yang Zhao, Shanchuan Lin, Zhibei Ma, Haoyuan Guo, Hao Chen, Lu Qi, Sen Wang, et al. Seaweed-7b: Cost-effective training of video generation foundation model. arXiv preprint arXiv:2504.08685,

    arXiv

  6. [18]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792,

    arXiv

  7. [19]

    Wan: Open and advanced large-scale video generative models

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314,

    arXiv

  8. [20]

    Sparse videogen: Accelerating video diffusion transformers with spatial-temporal sparsity

    31 Haocheng Xi, Shuo Yang, Yilong Zhao, Chenfeng Xu, Muyang Li, Xiuyu Li, Yujun Lin, Han Cai, Jintao Zhang, Dacheng Li, et al. Sparse videogen: Accelerating video diffusion transformers with spatial-temporal sparsity. arXiv preprint arXiv:2502.01776,

    arXiv

  9. [21]

    Representation alignment for generation: Training diffusion transformers is easier than you think

    Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. arXiv preprint arXiv:2410.06940,

    arXiv

  10. [22]

    Native sparse attention: Hardware-aligned and natively trainable sparse attention

    Jingyang Yuan, Huazuo Gao, Damai Dai, Junyu Luo, Liang Zhao, Zhengyan Zhang, Zhenda Xie, YX Wei, Lean Wang, Zhiping Xiao, et al. Native sparse attention: Hardware-aligned and natively trainable sparse attention. arXiv preprint arXiv:2502.11089, 2025a. Liping Yuan, Jiawei Wang,...

    arXiv

  11. [2018]

    Flashvideo: Flowing fidelity to detail for efficient high-resolution video generation

    Shilong Zhang, Wenbo Li, Shoufa Chen, Chongjian Ge, Peize Sun, Yida Zhang, Yi Jiang, Zehuan Yuan, Binyue Peng, and Ping Luo. Flashvideo: Flowing fidelity to detail for efficient high-resolution video generation. arXiv preprint arXiv:2502.05179, 2025c. Zhongwei Zhang, Fuchen Lo...

  12. [2021]

    Kling ai

    Kuaishou. Kling ai. https://klingai.kuaishou.com/, 2024.06. Yunshui Li, Yiyuan Ma, Shen Yan, Chaoyi Zhang, Jing Liu, Jianqiao Lu, Ziwen Xu, Mengzhao Chen, Minrui Wang, Shiyi Zhan, et al. Model merging in pre-training of large language models. arXiv preprint arXiv:2505.12082,

    2024 arXiv

  13. [2022]

    Videollama 3: Frontier multimodal foundation models for image and video understanding

    Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, et al. Videollama 3: Frontier multimodal foundation models for image and video understanding. arXiv preprint arXiv:2501.13106, 2025a. Peiyuan Zhang, Y...

    arXiv

  14. [2023]

    Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models

    Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Shuaiwen Leon Song, Samyam Rajbhandari, and Yuxiong He. Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models. arXiv preprint arXiv:2309.14509,

    arXiv

  15. [2024]

    Seedance 1.0: Exploring the boundaries of video generation models

    Yu Gao, Haoyuan Guo, Tuyen Hoang, Weilin Huang, Lu Jiang, Fangyuan Kong, Huixia Li, Jiashi Li, Liang Li, Xiaojie Li, et al. Seedance 1.0: Exploring the boundaries of video generation models. arXiv preprint arXiv:2506.09113,

    arXiv

  16. [2025]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127,

    arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.