Pith. sign in

REVIEW 3 major objections 6 minor 43 references

ReGenVC is the first generative video codec that packs talking-head clips into roughly one-tenth the usual bitrate and still decodes them live at 24 fps on eight GPUs.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 16:29 UTC pith:4IRRVAZP

load-bearing objection Solid real-time systems result for pose-driven diffusion decoding; the joint bitrate–quality claim outruns two short visual demos. the 3 major comments →

arxiv 2607.28144 v1 pith:4IRRVAZP submitted 2026-07-30 eess.IV cs.CV

ReGenVC: End-to-End Real-Time Generative Video Coding at Ultra-Low Bitrate

Zheyuan Zhang , Johnson Wu This is my paper
classification eess.IV cs.CV
keywords generative video codingultra-low bitratereal-time diffusiontalking-head videodiffusion distillationsequence parallelismhybrid CPU-GPU
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Traditional video codecs break into blocky artifacts when forced down to a few tens of kilobytes. ReGenVC instead sends only a compressed first frame, pose keypoints, and a little metadata—about 26 kB for a short talking-head clip—and lets a generative model rebuild the video at the receiver. The hard part has always been decoder speed: multi-step diffusion is far too slow for live use. The paper shows that a four-step distilled diffusion transformer, sharded across eight GPUs with a split VAE and an overlapped pipeline, can hit a one-second-per-window budget and stream at 24 fps without underruns. If that combination holds, generative coding becomes practical for bandwidth-starved talking-head uses such as messaging and telepresence.

Core claim

ReGenVC claims to be the first end-to-end generative video codec that is both ultra-low-bitrate and real-time-decodable: it encodes a 77-frame talking-head sequence in about 26 kB—roughly one-tenth what x264/x265 need for artifact-free output—and reconstructs it at 24 fps on an 8-GPU node (972 ms per 25-frame window inside a 1000 ms budget) by combining four-step distillation with model-preserving multi-GPU parallelism rather than further shrinking the model.

What carries the argument

The real-time decoder stack: a four-step DMD2+x0-regression student of a control video diffusion transformer, run under eight-GPU unified sequence parallelism (Ulysses×Ring), a spatially-split VAE, and a three-stage overlapped pipeline whose wall-clock time is Twin ≈ (wd+d) + max(v+a, we+e).

Load-bearing premise

That sharp-looking reconstructions on one short and one longer fixed-camera single-subject clip are enough to back the quality side of the ultra-low-bitrate claim, without a talking-head benchmark or standard perceptual metrics.

What would settle it

Run the same 26 kB bitstream path on a held-out talking-head set and check whether identity consistency, temporal coherence, and perceptual scores stay clearly better than matched-rate x264/x265; or measure whether Twin stays ≤1000 ms with no underruns over long multi-window streams under realistic jitter.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Talking-head live streams and video messages can drop to roughly one-tenth today’s artifact-free bitrates while staying sharp.
  • Generative codecs no longer have to stay offline: a single 8-GPU consumer node can serve 24 fps decoding.
  • Holding one-shot conditioners on CPU frees most GPU memory for the diffusion path without quantizing the model.
  • The analytical timing model gives a concrete feasibility test for other window sizes, step counts, and GPU counts before building them.
  • Domain-specialized pose-driven priors become a practical route to real-time generative compression when general-purpose priors are too heavy.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the same model-preserving parallel schedule scales cleanly, higher-resolution or multi-person pose-driven streams may become real-time by adding devices rather than rewriting the backbone.
  • The bitrate win is mostly from sending pose plus one reference frame; any domain with an equally compact, reliable control signal could inherit a similar encoder–decoder split.
  • Hybrid CPU–GPU offload of one-shot conditioners is a reusable pattern for other diffusion services that currently park large text/image encoders on the GPU.
  • Narrow headroom (~28 ms) implies that production deployments will still need jitter buffering or further launch-overhead cuts before the pipeline is robust under load.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. ReGenVC is an end-to-end generative codec for talking-head video that transmits a neurally compressed reference frame, per-frame pose keypoints, and metadata (~26 kB for 77 frames) and reconstructs with a 4-step distilled Wan2.1-Fun-Control diffusion transformer. The authors attack decoder latency via DMD2+x0-regression distillation, 8-GPU USP (Ulysses imes Ring 4 imes2), a spatially-split VAE, a three-stage overlapped pipeline, and CPU offload of one-shot T5/CLIP conditioners, guided by the timing model Twin ≈ (wd+d)+max(v+a, we+e). On 8 imes RTX 5090D they report Twin ≈ 972 ms ≤ Trt = 1000 ms (24 fps) and ~10 imes lower bitstream than high-rate x264/x265, with visual sharpness at matched ultra-low bitrate where traditional codecs block.

Significance. If the joint claim holds, the paper fills a documented gap: surveyed diffusion generative codecs are offline, while real-time diffusion systems are not codecs. The systems package is concrete and reusable—analytical timing model (Eqs. 3–4), measured kernel costs (Table 5), steady-state decomposition matching the model (Table 4), compilation ablation (1005 o972 ms), hybrid CPU–GPU memory cut (21.1 o~7.7 GB), and model-preserving multi-GPU design rather than further quantization. That combination is a genuine step toward deployable generative compression in bandwidth-starved talking-head settings, even if domain-limited.

major comments (3)
  1. [Sec. 5.2, 5.5; Figs. 3, 5; Table 3] Sec. 5.2 and Figs. 3/5 support the quality half of the joint claim with visual comparison on a single 77-frame clip and temporal snapshots on one 240-frame clip, both fixed-camera single-subject. No LPIPS/FVD/identity-consistency scores and no generative baselines (FOMM/Face-vid2vid, T-GVC, DiSCo, etc.) are reported; Sec. 5.5 explicitly defers benchmark-scale and perceptual study. Bitstream size is solid, but “stays sharp at ~1/10 the rate” and the codec-level bitrate–quality advantage are not yet evidenced beyond these two clips. A minimal talking-head set with perceptual metrics and at least the GFVC warping line is load-bearing for the central claim as stated in the abstract and contributions.
  2. [Sec. 3.4, 5.5] Sec. 3.4 deploys the student at step 300 of 500 with λ=10 and fixed timesteps {1000,750,500,250}, and notes that vanilla DMD2 caused color drift. There is no quantitative or side-by-side comparison of the 4-step student to the 20-step teacher (or to an ablated DMD2-only student) on held-out talking-head content. Because real-time feasibility is built on this student, and Sec. 5.5 disclaims perceptual equivalence, a compact teacher-vs-student check (even on the existing clips) is needed so the latency win is not purchased by an unmeasured quality drop.
  3. [Sec. 5.3; Table 4; Eq. (4)] Table 4 reports Twin ≈ 972 ms with ~28 ms (2.8%) headroom from a single tested run; Sec. 5.3 notes sensitivity to jitter and defers long-horizon stress testing. The live-demo claim (“no observed frame underruns”) and the feasibility inequality (Eq. 4) would be much stronger with multi-run variance, max Twin, and a longer stream (minutes, not ~10 s). Without that, sustained 24 fps is demonstrated only narrowly.
minor comments (6)
  1. [Table 1] Table 1 footnote (c) asserts real-time for FOMM/Face-vid2vid without measured fps; either cite a measured rate or soften the checkmark so the “first … real-time generative codec” framing stays precise relative to the GFVC line.
  2. [Table 3] PSNR is correctly flagged as inappropriate for generative output (Table 3 caption), yet still tabulated for ReGenVC as “—”. Consider dropping the PSNR column for ReGenVC entirely or adding a perceptual proxy even on the demo clips.
  3. [Sec. 4.1–4.2] In Sec. 4.1, wd is defined as a residual required by Eq. (3); state explicitly how it is estimated from traces so the decomposition is reproducible.
  4. [Fig. 2; Table 4] Fig. 2 segment widths are said to reflect Table 4; ensure the figure is drawn to scale or labeled with the ms values to avoid mismatch with the ~78/435/399/60/301/157 breakdown.
  5. [Sec. 5.1, Table 5] Clarify resolution and latent spatial size used for the VAE split and USP token counts; single-GPU kernel times in Table 5 are otherwise hard to compare externally.
  6. Minor typos/style: “totalling” vs “totaling,” inconsistent spacing in Ulysses×Ring, and arXiv IDs in related work that post-date typical cutoffs—verify citation metadata before camera-ready.

Circularity Check

0 steps flagged

No significant circularity: engineering systems paper with measured bitrates, measured latencies, and a descriptive timing decomposition.

full rationale

ReGenVC’s load-bearing claims are empirical systems results, not first-principles derivations that collapse into their inputs. Bitrate (~26 kB) is the sum of measured stream sizes (cheng2020-attn reference + quantized DWPose keypoints + metadata; Table 3). Real-time feasibility is Twin ≤ Trt = 1000 ms, with Twin ≈ (wd+d) + max(v+a, we+e) (Eq. 3–4) decomposing measured GPU/host terms from the pipeline (Table 4: ~972 ms); the model predicts and validates wall-clock structure, it does not define success by fitting a free parameter to equal the budget. Four-step distillation (DMD2 + x0-regression, Eq. 2) is a standard training objective whose quality is not claimed as a circular proof. Related-work “first” positioning cites external systems’ reported latencies, not a self-authored uniqueness theorem. No equation reduces the joint bitrate–latency claim to a normalization choice, fitted input renamed as prediction, or self-citation chain. Weak evaluation scope (two short clips, no LPIPS/FVD) is an evidence gap, not circularity.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 2 invented entities

The result rests on pretrained generative priors and standard parallel ML systems assumptions, plus several engineering choices (4-step schedule, λ, USP tiling, 1000 ms budget) and the domain premise that pose+one reference suffice for talking heads. No new physical entities; free parameters are training/serving knobs rather than fitted scientific constants.

free parameters (6)
  • distillation timesteps {1000,750,500,250} and NFE=4 = {1000,750,500,250}, 4 steps
    Hand-selected subset of the teacher’s 20-step DDIM schedule; load-bearing for fitting the latency budget.
  • x0-regression weight λ = 10.0
    Set to 10.0 to fix color drift under vanilla DMD2; chosen during distillation, not derived.
  • checkpoint step 300 of 500 = step 300
    Selected on held-out reconstructions; affects student quality deployed in all claims.
  • USP degrees U=4, R=2 = 4×2
    Chosen for 8-GPU NVLink topology and head-count caps; determines measured d.
  • cheng2020-attn quality=6 reference size = quality 6 (~10 kB)
    Sets ~10 kB identity stream and thus total bitrate.
  • window F=25, S=24, fps=24 → Trt=1000 ms = 1000 ms
    Defines the real-time success criterion Twin ≤ Trt used throughout Sec. 4–5.
axioms (6)
  • domain assumption A strong pose-conditioned video diffusion prior (Wan2.1-Fun-Control) can regenerate acceptable talking-head appearance from one reference frame plus per-frame keypoints.
    Design principle in Sec. 3.1–3.2; enables dropping pixel residuals. Not proved here; inherited from control-video literature.
  • domain assumption USP (Ulysses×Ring) yields numerically equivalent attention to single-device execution up to reduction order, so quality is unchanged by sharding.
    Sec. 3.5; standard exact-attention parallelism assumption.
  • domain assumption Spatial VAE tiling with halo sized to the receptive field and blend-on-gather does not introduce unacceptable seams.
    Sec. 3.6; required for claiming model-preserving parallel decode/encode.
  • ad hoc to paper Steady-state per-window wall-clock obeys Twin ≈ (wd+d) + max(v+a, we+e) with p and cpu off the critical path.
    Eq. (3), Sec. 4; organizing model validated against one measured decomposition, not a universal law.
  • domain assumption Fixed-camera single-subject talking-head is the intended operating domain; multi-person and general video are out of scope.
    Sec. 1, 5.5; scopes all quality and bitrate claims.
  • standard math Standard diffusion, DMD2, CompressAI, DWPose, and transformer attention mathematics hold as in the cited works.
    Background methods Sec. 2–3; not re-derived.
invented entities (2)
  • ReGenVC end-to-end codec schedule (4-step student + USP + split VAE + 3-stage overlap + CPU conditioner offload) no independent evidence
    purpose: Name the integrated serving system that meets the 1000 ms window budget at ultra-low bitrate.
    Composite engineering artifact, not a new physical entity; independent evidence would be third-party reimplementation or public demo artifacts (not shipped here).
  • DMD2 + x0-latent regression distillation recipe for control-video no independent evidence
    purpose: Stabilize color and shrink 20-step teacher to 4-step student without changing backbone architecture.
    Augmentation of published DMD2; color-drift failure mode is empirical to this setting. No external falsifiable handle beyond authors’ held-out picks.

pith-pipeline@v1.2.0-daily-grok45 · 21218 in / 4452 out tokens · 84121 ms · 2026-07-31T16:29:11.223335+00:00 · methodology

0 comments
read the original abstract

We present ReGenVC, an end-to-end generative video codec that compresses talking-head video to an ultra-low bitrate and decodes it in real time. The encoder reduces a source clip to a compact bitstream -- a neurally compressed first frame, per-frame pose keypoints, and metadata -- totaling about 26 kB for a 77-frame sequence. The decoder is a four-step distilled diffusion transformer that reconstructs the video conditioned on the transmitted pose and reference frame. Compared with x264/x265, ReGenVC reduces the bitrate to roughly one tenth of that required by traditional codecs (about 26 kB vs. 250--280 kB for essentially artifact-free reconstruction); at a matched ultra-low bitrate, conventional codecs collapse into blocking artifacts while ReGenVC stays sharp by exploiting a strong generative prior. The central obstacle to deploying such a codec is decoder latency: multi-step sampling with transformer and VAE components is too slow for interactive use. We make the decoder real-time through four-step distillation and three model-preserving system techniques: (i) eight-GPU unified sequence parallelism (Ulysses & Ring), (ii) a spatially-split VAE, and (iii) a three-stage overlapped pipeline; an analytical timing model characterizes the real-time feasibility region. On an 8-GPU node, the system sustains 24 fps output (972 ms per 25-frame window, within the 1000 ms budget), enabling a live browser stream without observed frame underruns. A hybrid CPU-GPU deployment further runs the encoder on the CPU at 24 fps and offloads the decoder-side one-shot conditioning encoders to the CPU, reducing the per-GPU memory peak from 21.1 GB to about 7.7 GB. To our knowledge, ReGenVC is the first end-to-end generative video codec to combine ultra-low-bitrate encoding with real-time decoding on an 8-GPU system.

Figures

Figures reproduced from arXiv: 2607.28144 by Johnson Wu, Zheyuan Zhang.

Figure 1
Figure 1. Figure 1: ReGenVC end-to-end codec. The encoder transmits only identity (a neurally compressed reference frame), motion (pose [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Steady-state three-stage pipeline (Sec. 3.7). In every window three different windows are in flight on independent resources: stage (i) prepares the next window’s pose on the CPU (the next-window VAE encode e appears inside stage (ii)’s max path); stage (ii) runs diffusion followed by the overlapped VAE decode/encode for the current window on the main and dec stream; stage (iii) copies￾out and post-process… view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative comparison on a talking-head frame. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Live browser demo: ReGenVC decoding and streaming [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Reconstruction fidelity across the ∼10 s (240-frame) demo clip (Sec. 3.3). Top row: original input frames; bottom row: ReGenVC reconstructions decoded from the ∼60 kB bitstream, at six time instants spanning the sequence (timestamps on top). ReGenVC visibly preserves recognizable identity and facial detail across the tested clip. decoding and reconstruction fidelity (Figs. 4, 5) are demon￾strated on a sing… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

43 extracted references · 10 linked inside Pith

  1. [1]

    Compressai: a pytorch library and evalua- tion platform for end-to-end compression research

    Jean B ´egaint, Fabien Racap ´e, Simon Feltman, and Akshay Pushparaja. Compressai: a pytorch library and evalua- tion platform for end-to-end compression research. arXiv preprint arXiv:2011.03029, 2020. 3

    Pith/arXiv arXiv 2011

  2. [2]

    Stable video diffusion: Scaling la- tent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, Varun Jampani, and Robin Rombach. Stable video diffusion: Scaling la- tent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 5

    Pith/arXiv arXiv 2023

  3. [3]

    Generative face video coding techniques and standardization efforts: A review

    Bolin Chen, Jie Chen, Shiqi Wang, and Yan Ye. Generative face video coding techniques and standardization efforts: A review. In Data Compression Conference (DCC), 2024. 2

    2024

  4. [4]

    Learned image compression with discretized gaussian mixture likelihoods and attention modules

    Zhengxue Cheng, Heming Sun, Masaru Takeuchi, and Jiro Katto. Learned image compression with discretized gaussian mixture likelihoods and attention modules. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. 3

    2020

  5. [5]

    Unsupervised cross-lingual representation learn- ing at scale

    Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm´an, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. Unsupervised cross-lingual representation learn- ing at scale. In Annual Meeting of the Association for Com- putational Linguistics (ACL), 2020. 6

    2020

  6. [6]

    Flashattention-2: Faster attention with better par- allelism and work partitioning

    Tri Dao. Flashattention-2: Faster attention with better par- allelism and work partitioning. In International Conference on Learning Representations (ICLR), 2024. 3

    2024

  7. [7]

    Fu, Stefano Ermon, Atri Rudra, and Christopher R ´e

    Tri Dao, Daniel Y . Fu, Stefano Ermon, Atri Rudra, and Christopher R ´e. Flashattention: Fast and memory-efficient exact attention with io-awareness. In Advances in Neural Information Processing Systems (NeurIPS), 2022. 3

    2022

  8. [8]

    xdit: An inference engine for diffusion trans- formers (dits) with massive parallelism

    Jiarui Fang, Jinzhe Pan, Xibo Sun, Aoyu Li, and Jian- nan Wang. xdit: An inference engine for diffusion trans- formers (dits) with massive parallelism. arXiv preprint arXiv:2411.01738, 2024. 3

    Pith/arXiv arXiv 2024

  9. [9]

    Pipefusion: Patch-level pipeline parallelism for dif- fusion transformers inference

    Jiarui Fang, Jinzhe Pan, Aoyu Li, Xibo Sun, and Jiannan Wang. Pipefusion: Patch-level pipeline parallelism for dif- fusion transformers inference. In Advances in Neural Infor- mation Processing Systems (NeurIPS), 2025. 3

    2025

  10. [10]

    ZeroGVC: Zero-shot generative video compres- sion with autoregressive diffusion priors

    Yixin Gao, Xiaohan Pan, Lin Liu, Xin Li, Zhibo Chen, and Qi Tian. ZeroGVC: Zero-shot generative video compres- sion with autoregressive diffusion priors. arXiv preprint arXiv:2606.22371, 2026. 2, 4

    Pith/arXiv arXiv 2026

  11. [11]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. In Advances in Neural Informa- tion Processing Systems (NeurIPS), 2020. 3

    2020

  12. [12]

    Self forcing: Bridging the train-test gap in autoregressive video diffusion

    Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion. In Advances in Neural Infor- mation Processing Systems (NeurIPS), 2025. 3

    2025

  13. [13]

    Open- clip

    Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Han- naneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Open- clip. https : / / doi . org / 10 . 5281 / zenodo . 5143773, 2021. 6

    2021

  14. [14]

    Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models

    Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Min- jia Zhang, Shuaiwen Leon Song, Samyam Rajbhandari, and Yuxiong He. Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models. arXiv preprint arXiv:2309.14509, 2023. 2, 3, 5

    Pith/arXiv arXiv 2023

  15. [15]

    Streamdit: Real-time streaming text-to-video generation

    Akio Kodaira, Tingbo Hou, Ji Hou, Markos Georgopou- los, Felix Juefei-Xu, Masayoshi Tomizuka, and Yue Zhao. Streamdit: Real-time streaming text-to-video generation. arXiv preprint arXiv:2507.03745, 2025. 2, 3, 4, 6

    arXiv 2025

  16. [16]

    Streamdiffusion: A pipeline-level solution for real- time interactive generation

    Akio Kodaira, Chenfeng Xu, Toshiki Hazama, Takanori Yoshimoto, Kohei Ohno, Shogo Mitsuhori, Soichi Sugano, Hanying Cho, Zhijian Liu, Masayoshi Tomizuka, and Kurt Keutzer. Streamdiffusion: A pipeline-level solution for real- time interactive generation. InIEEE/CVF International Con- ference on Computer Vision (ICCV), 2025. 3

    2025

  17. [17]

    Neural video compression with feature modulation

    Jiahao Li, Bin Li, and Yan Lu. Neural video compression with feature modulation. In IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), 2024. 3

    2024

  18. [18]

    Free-GVC: To- wards training-free extreme generative video compression with temporal coherence

    Xiaoyue Ling, Chuqin Zhou, Chunyi Li, Yunuo Chen, Yuan Tian, Guo Lu, and Wenjun Zhang. Free-GVC: To- wards training-free extreme generative video compression with temporal coherence. arXiv preprint arXiv:2602.09868,

    arXiv

  19. [19]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maxi- milian Nickel, and Matt Le. Flow matching for generative modeling. In International Conference on Learning Repre- sentations (ICLR), 2023. 3

    2023

  20. [20]

    Ring attention with blockwise transformers for near-infinite context

    Hao Liu, Matei Zaharia, and Pieter Abbeel. Ring attention with blockwise transformers for near-infinite context. arXiv preprint arXiv:2310.01889, 2023. 2, 3, 5

    Pith/arXiv arXiv 2023

  21. [21]

    Instaflow: One step is enough for high-quality diffusion-based text-to-image generation

    Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, and Qiang Liu. Instaflow: One step is enough for high-quality diffusion-based text-to-image generation. In International Conference on Learning Representations (ICLR), 2024. 3

    2024

  22. [22]

    Latent consistency models: Synthesizing high- resolution images with few-step inference

    Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high- resolution images with few-step inference. arXiv preprint arXiv:2310.04378, 2023. 3

    Pith/arXiv arXiv 2023

  23. [23]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning (ICML), 2021. 4, 6

    2021

  24. [24]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learn- ing Research (JMLR), 2020. 4, 6

    2020

  25. [25]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 3

    2022

  26. [26]

    Progressive distillation for fast sampling of diffusion models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Confer- ence on Learning Representations (ICLR), 2022. 3

    2022

  27. [27]

    First order motion model for im- age animation

    Aliaksandr Siarohin, St ´ephane Lathuili`ere, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. First order motion model for im- age animation. In Advances in Neural Information Process- ing Systems (NeurIPS), 2019. 2, 4

    2019

  28. [28]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In International Conference on Machine Learning (ICML), 2023. 3

    2023

  29. [29]

    M3-CVC: Controllable video compression with multimodal generative models

    Rui Wan, Qi Zheng, and Yibo Fan. M3-CVC: Controllable video compression with multimodal generative models. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025. 2, 4

    2025

  30. [30]

    Wan: Open and advanced large-scale video generative models

    Wan Team et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025. 2, 4, 5, 7

    Pith/arXiv arXiv 2025

  31. [31]

    Sitaraman

    Lingdong Wang, Guan-Ming Su, Divya Kothandara- man, Tsung-Wei Huang, Mohammad Hajiesmaili, and Ramesh K. Sitaraman. Low-bitrate video compression through semantic-conditioned diffusion. arXiv preprint arXiv:2512.00408, 2025. 2, 4

    Pith/arXiv arXiv 2025

  32. [32]

    One-shot free-view neural talking-head synthesis for video conferenc- ing

    Ting-Chun Wang, Arun Mallya, and Ming-Yu Liu. One-shot free-view neural talking-head synthesis for video conferenc- ing. In IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), 2021. 2, 4

    2021

  33. [33]

    T-GVC: Trajectory-guided generative video coding at ultra-low bitrates

    Zhitao Wang, Hengyu Man, Wenrui Li, Xingtao Wang, Xi- aopeng Fan, and Debin Zhao. T-GVC: Trajectory-guided generative video coding at ultra-low bitrates. In AAAI Con- ference on Artificial Intelligence (AAAI), 2026. 1, 2, 4

    2026

  34. [34]

    Joint source-channel- generation coding: From distortion-oriented reconstruc- tion to semantic-consistent generation

    Tong Wu, Zhiyong Chen, Guo Lu, Li Song, Feng Yang, Meixia Tao, and Wenjun Zhang. Joint source-channel- generation coding: From distortion-oriented reconstruc- tion to semantic-consistent generation. arXiv preprint arXiv:2601.12808, 2026. 1, 2, 4

    arXiv 2026

  35. [35]

    Effec- tive whole-body pose estimation with two-stages distillation

    Zhendong Yang, Ailing Zeng, Chun Yuan, and Yu Li. Effec- tive whole-body pose estimation with two-stages distillation. In IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2023. 4

    2023

  36. [36]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, Da Yin, Yuxuan Zhang, Weihan Wang, Yean Cheng, Bin Xu, Xiaotao Gu, Yuxiao Dong, and Jie Tang. Cogvideox: Text-to-video diffusion models with an expert transformer. In International Conference on Learning Representations (ICL...

    2025

  37. [37]

    Conditional video generation for high-efficiency video compression

    Fangqiu Yi, Jingyu Xu, Jiawei Shao, Chi Zhang, and Xue- long Li. Conditional video generation for high-efficiency video compression. arXiv preprint arXiv:2507.15269, 2025. 1, 2, 4

    arXiv 2025

  38. [38]

    Tianwei Yin, Micha ¨el Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fr´edo Durand, and William T. Freeman. Im- proved distribution matching distillation for fast image syn- thesis. Advances in Neural Information Processing Systems (NeurIPS), 2024. 3, 5

    2024

  39. [39]

    Freeman, and Taesung Park

    Tianwei Yin, Micha ¨el Gharbi, Richard Zhang, Eli Shecht- man, Fr´edo Durand, William T. Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 3

    2024

  40. [40]

    Free- man, Fr´edo Durand, Eli Shechtman, and Xun Huang

    Tianwei Yin, Qiang Zhang, Richard Zhang, William T. Free- man, Fr´edo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast autoregressive video diffusion mod- els. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. 3

    2025

  41. [41]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In IEEE/CVF International Conference on Computer Vision (ICCV), 2023. 4

    2023

  42. [42]

    Sana-streaming: Real-time streaming video editing with hy- brid diffusion transformer.arXiv preprint arXiv:2605.30409,

    Yuyang Zhao, Yicheng Pan, Qiyuan He, Jincheng Yu, Jun- song Chen, Tian Ye, Haozhe Liu, Enze Xie, and Song Han. Sana-streaming: Real-time streaming video editing with hy- brid diffusion transformer.arXiv preprint arXiv:2605.30409,

    Pith/arXiv arXiv

  43. [43]

    Flashvsr: Towards real- time diffusion-based streaming video super-resolution.arXiv preprint arXiv:2510.12747, 2025

    Junhao Zhuang, Shi Guo, Xin Cai, Xiaohui Li, Yihao Liu, Chun Yuan, and Tianfan Xue. Flashvsr: Towards real- time diffusion-based streaming video super-resolution.arXiv preprint arXiv:2510.12747, 2025. 2, 3, 4, 6

    arXiv 2025