Pith. sign in

REVIEW 4 major objections 6 minor 57 references

Audio energy guides which video latents to recompute, enabling over 2x faster audio-driven video generation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 06:40 UTC pith:QCP5EQSP

load-bearing objection A genuinely new cross-modal caching idea with solid but under-specified experiments; the dynamic schedule has a potential feedback-loop ambiguity that needs clarification before the quality claims are fully trustworthy. the 4 major comments →

arxiv 2608.02474 v1 pith:QCP5EQSP submitted 2026-08-03 cs.CV

EchoCache: Energy-Guided Cross-Modal Caching for Efficient Audio-Driven Video Generation

Jiayu Chen , Xiaoyu Wu , Rongshan Gao , Maoliang Li , Zihao Zheng , Xinhao Sun , Hailong Zou , Guojie Luo
show 1 more author
Xiang Chen
This is my paper
classification cs.CV
keywords audio-driven video generationdiffusion transformerfeature cachingcross-modal saliencyaudio energylatent-level cache schedulingvideo generation accelerationcache quantization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that in audio-driven video generation, audio time-frequency energy reveals which video latents are critical and need recomputation during denoising. EchoCache converts that energy into a saliency mask that schedules cache reuse at both token and timestep level, with quantized cache storage. Results on two models show a 2.46x latency speedup on Wan2.2-S2V over EMTD with best quality metrics among caching baselines, while preserving lip-sync and identity consistency.

Core claim

The central claim: audio energy itself acts as the saliency map for caching. High-energy audio segments correspond to video latents that need frequent updates; low-energy segments can reuse cached features. EchoCache builds a per-segment STFT L2-norm energy vector, selects top-K segments, and masks the video latent sequence so attention and feed-forward run only on active latents. A dynamic step-latent schedule scales the active-latent ratio by visual difference, and a cache manager stores similar latents as a reference plus int8-quantized residual. The combination resolves what the paper calls temporal-semantic and computation-storage misalignments.

What carries the argument

The Cross-modal Saliency Anchor: an audio energy vector from splitting input audio into T segments, applying STFT per segment, and taking L2 norms. Top-K selection over this vector yields indices of high-energy segments, expanded into a binary mask over video latents. That mask drives latent-level caching only on active tokens, energy-guided step-latent scheduling varying active-latent ratio by visual difference, and memory-efficient cache management storing similar latents as quantized residuals.

Load-bearing premise

The load-bearing premise is that audio time-frequency energy tracks which video latents truly need recomputation, with high-energy segments marked as critical and low-energy segments safely cacheable; the paper's evidence is qualitative and the authors note this prior may miss richer cross-modal cues.

What would settle it

Run EchoCache with the saliency mask inverted: update low-energy latents and cache high-energy ones. If FVD and Sync-C stay similar to the normal configuration, the energy anchor is not doing the work. Also compute per audio segment the correlation between segment energy and the FID delta from caching that segment; near-zero or negative correlation would contradict the central claim.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • On Wan2.2-S2V over EMTD, EchoCache reports 2.46x latency speedup and 1.98x FLOP reduction with best FID, FVD, and Sync-D among caching baselines.
  • Ablations show that replacing the audio-energy anchor with uniform or random selection raises FVD and lowers Sync-C, confirming the anchor's importance.
  • The step-latent schedule is necessary: step-only caching is slower and latent-only caching raises peak memory, while the combined approach keeps memory near the original model's level.
  • The quantized-residual cache holds peak memory near baseline while avoiding the FVD degradation of naive int8 quantization.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial: because the anchor is computed once from input audio, it could be refreshed mid-generation or fused with visual-difference signals to test whether energy alone or energy-plus-state is the better scheduler.
  • Editorial: the same energy-as-saliency idea should transfer to other audio-conditioned generation tasks like speech-driven gesture or sound-to-scene synthesis, where audio dynamics concentrate in bursts.
  • Editorial: the fixed top-K mask may ignore that low-energy segments early in denoising still need updates; a step-dependent anchor could further improve quality.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes EchoCache, a training-free caching framework for audio-driven video generation. The method computes a per-segment audio time-frequency energy vector via STFT, selects Top-K high-energy segments as a cross-modal saliency anchor, and then performs latent-level caching: high-saliency latents are recomputed while low-saliency latents reuse cached features. A dynamic step-level update rate r_t = clip(r_min + α·Δ̂_t, r_min, r_max) adjusts how many latents are updated, and a quantized residual cache manager reduces memory overhead. Experiments on Wan2.2-S2V and LongCat-Avatar over HDTF and EMTD report a 2.46× speedup on Wan2.2-S2V/EMTD with the best FID, FVD, and Sync-D among caching baselines, together with ablations showing the contribution of each component.

Significance. The cross-modal idea of using audio energy to allocate cache updates is novel and relevant for the growing area of efficient audio-driven video generation. If the results hold, the paper would provide a principled alternative to purely visual-redundancy-based caching and a useful plug-and-play acceleration tool. The manuscript includes code availability, comparisons on two public models and two benchmarks, and ablations of the main components. However, confidence is tempered by a possible feedback loop in the dynamic scheduling signal, the lack of variance reporting, and the absence of a clear validation-tune split for the hyperparameters.

major comments (4)
  1. [Sec. 3.3, Eq. (1)] The update rate r_t = clip(r_min + α·Δ̂_t, r_min, r_max) is defined using the visual difference indicator Δ_t from Eq. (1), which is computed from h_t and h_{t+1}. Under EchoCache's latent-level caching, h_t for inactive positions is the reused cache, not the current network output. The manuscript does not state that Δ_t is measured before the cache mask is applied, on uncached features, or with a reference full computation. If Δ_t is computed on already-cached representations, aggressive caching drives Δ_t toward zero, which lowers r_t and encourages even more caching — a positive feedback loop that is unrelated to the true drift of the denoising trajectory. This directly affects the claimed benefit of the dynamic step-latent schedule. The authors should specify exactly which features are used to compute Δ_t and, if necessary, reformulate the schedule to avoid the feedback.
  2. [Sec. 4.1/4.2, Table 1, Fig. 7] Tables 1 and 2 report no error bars, confidence intervals, or repetition counts for any quality or latency metric. Since diffusion sampling is stochastic and the models/datasets involve randomness, the reported differences (e.g., the 2.46× vs. 1.92× speedup and the FID/FVD gaps) need variance estimates. In addition, α and τ are tuned as shown in Fig. 7, but the manuscript does not describe a validation split. If these hyperparameters are selected on the same benchmarks used to report the final numbers, the results are optimistically biased. Please add a clear tuning-validation protocol or report results across a range of hyperparameter values.
  3. [Sec. 3.2, Fig. 3(c)] The core assumption of the paper — that audio time-frequency energy is a reliable proxy for which video latents require more frequent updates — is supported only by the qualitative Fig. 3(c) and the Table 2 anchor ablation. No quantitative correlation is reported between audio segment energy and measured quality sensitivity (e.g., per-segment FVD increase under different cache ratios). Given that this assumption is the foundation of the cross-modal design, a quantitative validation on a held-out set is needed. Otherwise, the claim that the method generalizes across diverse audio-visual content is not well supported.
  4. [Sec. 4.3, 'Ablation Study on AVSync15'] The AVSync15 ablation is qualitative only (Fig. 8). No quantitative metrics such as FVD, Sync-C, Sync-D, or CSIM are reported for this in-the-wild benchmark. Since the paper claims cross-scenario generalization, the absence of any quantitative evaluation on AVSync15 weakens that claim. Please add the missing metrics or explicitly state that this is a qualitative illustration and limit the generalization claim accordingly.
minor comments (6)
  1. [Abstract] The abstract says generation quality is 'preserved,' but Table 1 shows notable degradation versus the full model (e.g., FID 60.76→66.88 and FVD 108.18→117.67 on Wan2.2-S2V/HDTF). Consider rephrasing to 'maintains competitive quality relative to the full model' or 'improves quality over other caching baselines.'
  2. [Sec. 3.2] The text refers to 'Fig. 4(a)' and 'Fig. 4(c)' when describing the analysis, but the figure is numbered Fig. 3. Please correct the cross-references.
  3. [Sec. 2.1/2.2; Sec. 3.2] There are duplicated headings: 'Audio-driven Video Generation' appears twice (Sec. 2.1 and 2.2), and the two paragraphs 'Cross-modal Saliency Analysis' and 'Quality Sensitivity Analysis' in Sec. 3.2 are nearly identical. These should be consolidated.
  4. [Sec. 3.3] The notation k_t = ⌈N·r_t⌉ is unclear. In Sec. 3.1 the total latent set is T×N and T is the number of temporal segments, but here N is described as the number of spatial latents per temporal position. It is not clear how the global update rate r_t is converted to a per-segment or per-timestep count of active latents, nor how k_t interacts with the Top-K high-energy segments. Please clarify.
  5. [Eq. (1)] The summation upper limit in Eq. (1) is written as T×N, but T is also introduced as the number of latent temporal segments. Using T in two different roles may confuse readers. Consider renaming one of them.
  6. [Table 1] In the 'Speed' columns, it would be helpful to explicitly state that speedup is computed relative to the full model (1.00×) for each dataset/model, and to clarify whether the same hardware and implementation settings are used for all baselines. The text states this, but the table caption could be clearer.

Circularity Check

0 steps flagged

No significant circularity: the central caching mechanism is evaluated on external benchmarks, and self-citations are not load-bearing.

full rationale

The paper's core claim—that audio time-frequency energy can guide latent-level cache updates—is an empirical design choice, not a by-construction consequence of its inputs. The saliency anchor is computed from STFT energy before generation and is not fitted to the quality metrics it is later compared against. The update-rate formula r_t = clip(r_min + αΔ̂_t, r_min, r_max) depends on a visual-difference signal, but α and τ are presented as tunable hyperparameters with reported sensitivity, not as parameters recovered from the benchmark outputs. The main results are external comparisons against TeaCache, MagCache, and TaylorSeer on HDTF, EMTD, and AVSync15, so the performance claim is falsifiable outside the paper's own fitted values. The only self-citations (EcoVideo and sparse-attention works by the same first author) appear in the related-work survey and in the early-caching list; they do not justify the mechanism or supply a uniqueness/necessity argument. The paper's own limitation statement concedes that the scheduling is 'partly heuristic,' which is a generality caveat rather than circularity. The skeptic's concern that Δ_t might be computed on cached rather than uncached latents is an implementation/ambiguity issue that could affect the validity of the adaptive schedule, but the paper does not define Δ_t in a way that makes the prediction equal to its input by construction. Thus no circular step can be exhibited from the text; score 2 reflects only minor non-load-bearing self-citation.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The method depends on manually chosen hyperparameters and domain assumptions about audio-energy-to-saliency correspondence. No first-principles derivation is given; the design is justified by ablations on the same benchmarks where results are reported.

free parameters (4)
  • Top-K k = not specified
    Number of high-energy audio segments selected as active latents; chosen by hand, not derived, and directly controls how many latents are updated.
  • alpha (update ratio coefficient) = default from Fig. 7 sweep (0.025-0.1); exact default not stated
    Scales normalized timestep difference to set the per-step update rate r_t; tuned on the evaluated benchmarks.
  • r_min / r_max = not stated
    Clamp bounds for the per-step update ratio; set manually and affect the quality-latency trade-off.
  • tau (cache quantization threshold) = default from Fig. 7 sweep (0.90-0.98); exact default not stated
    Cosine-similarity threshold deciding whether a latent is stored as a quantized residual or as a full-precision reference; tuned per benchmark.
axioms (4)
  • domain assumption Audio time-frequency energy correlates with the importance of corresponding video latents for generation quality.
    Stated as an observation in Sections 3.2 and 3.3; supported only by qualitative Fig. 3(c), with no quantitative correlation coefficient.
  • domain assumption Latent representations evolve smoothly across adjacent denoising steps, so cached features remain valid over intervals.
    Standard caching premise used in Section 3.1, Eqs. (1)-(2).
  • domain assumption Fusing updated active latents with cached inactive latents via the mask operation does not introduce harmful discontinuities.
    Core operation in Section 3.3; not formally analyzed.
  • domain assumption INT8 quantization of residuals below the similarity threshold is accurate enough for the reported quality.
    Validated only through the naive INT8 ablation showing quality drop; no error analysis is provided.

pith-pipeline@v1.3.0-daily-deepseek · 14510 in / 10747 out tokens · 91901 ms · 2026-08-04T06:40:34.666899+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of EchoCache: Energy-Guided Cross-Modal Caching for Efficient Audio-Driven Video Generation." pith.science (2026). https://pith.science/paper/QCP5EQSP

@misc{pith2026260802474,
  author       = {Pith},
  title        = {Pith review of: EchoCache: Energy-Guided Cross-Modal Caching for Efficient Audio-Driven Video Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QCP5EQSP}},
  note         = {Machine review of arXiv:2608.02474}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Audio-driven video generation (A2V) has achieved promising progress in synthesizing temporally coherent and audio-visually aligned videos, yet its inference remains expensive due to the iterative denoising process of diffusion models. Existing caching methods mainly exploit temporal redundancy in visual features while overlooking the cross-modal alignment of A2V, where audio drives visual generation with highly non-uniform temporal importance. In this paper, we identify two levels of misalignment in existing A2V caching methods: temporal-semantic and computation-storage misalignment. To address them, we propose EchoCache, an energy-guided cross-modal caching framework for efficient A2V generation. EchoCache leverages audio time-frequency energy as a saliency anchor to guide latent-level cache updates and further introduces a dynamic timestep-latent caching mechanism with quantized cache management for joint efficiency and memory optimization. Extensive experiments on mainstream A2V models show that EchoCache consistently improves the latency-quality trade-off while preserving generation quality and audio-visual consistency. In particular, on Wan2.2-S2V over the EMTD benchmark, EchoCache achieves a 2.46x speedup with the best overall performance. Code is available at https://github.com/IF-LAB-PKU/EchoCache.

Figures

Figures reproduced from arXiv: 2608.02474 by Guojie Luo, Hailong Zou, Jiayu Chen, Maoliang Li, Rongshan Gao, Xiang Chen, Xiaoyu Wu, Xinhao Sun, Zihao Zheng.

Figure 1
Figure 1. Figure 1: Overview of Audio-driven Video Generation. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparison of previous caching methods and the proposed EchoCache for audio-driven video generation. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: (a) A2V is inherently a cross-modal alignment process, where audio dynamics drive corresponding local visual changes. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Overview of EchoCache. (a) Cross-modal saliency anchors from audio energy. (b) Fine-grained latent-level caching. (c) [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparison of EchoCache and baseline methods on Wan2.2-S2V across the HDTF and EMTD datasets. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative comparison of EchoCache and baselines on LongCat-Avatar across the HDTF and EMTD datasets. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative ablation results on AVSync15. Compared [PITH_FULL_IMAGE:figures/full_fig_p008_8.png] view at source ↗
Figure 7
Figure 7. Figure 7: Hyperparameter sensitivity analysis of EchoCache, [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

57 extracted references · 22 linked inside Pith

  1. [1]

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Leo Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, et al. 2024. Video generation models as world simulators.OpenAI Blog1, 8 (2024), 1

    2024

  2. [2]

    Jiayu Chen, Hengyi Zhang, Maoliang Li, Minyu Li, Zihao Zheng, Xuanzhe Liu, Guojie Luo, and Xiang Chen. 2026. EcoVideo: Entropy-Orchestrated Video Generation Paradigm in Cloud-Edge Dynamics.Proceedings of the European Conference on Computer Vision (ECCV)(2026)

    2026

  3. [3]

    Ming Chen, Liyuan Cui, Wenyuan Zhang, Haoxian Zhang, Yan Zhou, Xiaohan Li, Songlin Tang, Jiwen Liu, Borui Liao, Hejia Chen, et al. 2025. Midas: Multimodal interactive digital-human synthesis via real-time autoregressive video generation. arXiv preprint arXiv:2508.19320(2025)

    Pith/arXiv arXiv 2025

  4. [4]

    Siyan Chen, Yanfei Chen, Ying Chen, Zhuo Chen, Feng Cheng, Xuyan Chi, Jian Cong, Qinpeng Cui, Qide Dong, Junliang Fan, et al . 2025. Seedance 1.5 pro: A Native Audio-Visual Joint Generation Foundation Model.arXiv preprint arXiv:2512.13507(2025)

    Pith/arXiv arXiv 2025

  5. [5]

    Yi Chen, Sen Liang, Zixiang Zhou, Ziyao Huang, Yifeng Ma, Junshu Tang, Qin Lin, Yuan Zhou, and Qinglin Lu. 2025. Hunyuanvideo-avatar: High-fidelity audio- driven human animation for multiple characters.arXiv preprint arXiv:2505.20156 (2025)

    Pith/arXiv arXiv 2025

  6. [6]

    Kun Cheng, Xiaodong Cun, Yong Zhang, Menghan Xia, Fei Yin, Mingrui Zhu, Xuan Wang, Jue Wang, and Nannan Wang. 2022. Videoretalking: Audio-based lip synchronization for talking head video editing in the wild. InSIGGRAPH Asia 2022 Conference Papers. 1–9

    2022

  7. [7]

    Joon Son Chung and Andrew Zisserman. 2016. Out of time: automated lip sync in the wild. InAsian conference on computer vision. Springer, 251–263

    2016

  8. [8]

    Weilun Feng, Haotong Qin, Chuanguang Yang, Xiangqi Li, Han Yang, Yuqi Li, Zhulin An, Libo Huang, Michele Magno, and Yongjun Xu. 2025. S 2Q-VDiT: Accurate Quantized Video Diffusion Transformer with Salient Data and Sparse Token Distillation.arXiv preprint arXiv:2508.04016(2025)

    arXiv 2025

  9. [9]

    Xin Gao, Li Hu, Siqi Hu, Mingyang Huang, Chaonan Ji, Dechao Meng, Jinwei Qi, Penchong Qiao, Zhen Shen, Yafei Song, et al. 2025. Wan-s2v: Audio-driven cinematic video generation.arXiv preprint arXiv:2508.18621(2025)

    Pith/arXiv arXiv 2025

  10. [10]

    Yu Gao, Haoyuan Guo, Tuyen Hoang, Weilin Huang, Lu Jiang, Fangyuan Kong, Huixia Li, Jiashi Li, Liang Li, Xiaojie Li, et al. 2025. Seedance 1.0: Exploring the boundaries of video generation models.arXiv preprint arXiv:2506.09113(2025)

    Pith/arXiv arXiv 2025

  11. [11]

    Yu Gao, Jiancheng Huang, Xiaopeng Sun, Zequn Jie, Yujie Zhong, and Lin Ma. 2024. Matten: Video Generation with Mamba-Attention.arXiv preprint arXiv:2405.03025(2024)

    Pith/arXiv arXiv 2024

  12. [12]

    Jiazhi Guan, Kaisiyuan Wang, Zhiliang Xu, Quanwei Yang, Yasheng Sun, Shengyi He, Borong Liang, Yukang Cao, Yingying Li, Haocheng Feng, et al. 2025. Audcast: Audio-driven human video generation by cascaded diffusion transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10678–10689

    2025

  13. [13]

    Jiazhi Guan, Zhanwang Zhang, Hang Zhou, Tianshu Hu, Kaisiyuan Wang, Dongliang He, Haocheng Feng, Jingtuo Liu, Errui Ding, Ziwei Liu, et al . 2023. Stylesync: High-fidelity generalized and personalized lip sync in style-based generator. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1505–1515

    2023

  14. [14]

    Yefei He, Luping Liu, Jing Liu, Weijia Wu, Hong Zhou, and Bohan Zhuang. 2023. Ptqd: Accurate post-training quantization for diffusion models.Advances in Neural Information Processing Systems36 (2023), 13237–13249

    2023

  15. [15]

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems30 (2017)

    2017

  16. [16]

    Jiancheng Huang, Mingfu Yan, Songyan Chen, Yi Huang, and Shifeng Chen. 2024. MagicFight: Personalized Martial Arts Combat Video Generation. InProceedings of the 32nd ACM International Conference on Multimedia. 10833–10842. doi:10. 1145/3664647.3680849

    arXiv 2024

  17. [17]

    Jiancheng Huang, Gengwei Zhang, Zequn Jie, Siyu Jiao, Yinlong Qian, Ling Chen, Yunchao Wei, and Lin Ma. 2026. M4V: Multi-Modal Mamba for Text-to-Video Generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    2026

  18. [18]

    Yubo Huang, Weiqiang Wang, Sirui Zhao, Tong Xu, Lin Liu, and Enhong Chen

  19. [19]

    Kumara Kahatapitiya, Haozhe Liu, Sen He, Ding Liu, Menglin Jia, Chenyang Zhang, Michael S Ryoo, and Tian Xie. 2025. Adaptive caching for faster video gen- eration with diffusion transformers. InProceedings of the IEEE/CVF International Conference on Computer Vision. 15240–15252

    2025

  20. [20]

    Senmao Li, Taihang Hu, Fahad Shahbaz Khan, Linxuan Li, Shiqi Yang, Yaxing Wang, Ming-Ming Cheng, and Jian Yang. 2023. Faster diffusion: Rethinking the role of unet encoder in diffusion models.arXiv preprint arXiv:2312.096083 (2023)

    Pith/arXiv arXiv 2023

  21. [21]

    Gaojie Lin, Jianwen Jiang, Chao Liang, Tianyun Zhong, Jiaqi Yang, Zerong Zheng, and Yanbo Zheng. 2025. Cyberhost: A one-stage diffusion framework for audio- driven talking body generation. InThe Thirteenth International Conference on Learning Representations

    2025

  22. [22]

    Feng Liu, Shiwei Zhang, Xiaofeng Wang, Yujie Wei, Haonan Qiu, Yuzhong Zhao, Yingya Zhang, Qixiang Ye, and Fang Wan. 2025. Timestep Embedding Tells: It’s Time to Cache for Video Diffusion Model. InProceedings of the Computer Vision and Pattern Recognition Conference. 7353–7363

    2025

  23. [23]

    Jiacheng Liu, Chang Zou, Yuanhuiyi Lyu, Junjie Chen, and Linfeng Zhang. 2025. From reusing to forecasting: Accelerating diffusion models with taylorseers. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 15853– 15863

    2025

  24. [24]

    Jiacheng Liu, Chang Zou, Yuanhuiyi Lyu, Fei Ren, Shaobo Wang, Kaixin Li, and Linfeng Zhang. 2025. Speca: Accelerating diffusion transformers with speculative feature caching. InProceedings of the 33rd ACM International Conference on Multimedia. 10024–10033

    2025

  25. [25]

    Jiayi Luo, Jiayu Chen, Jiankun Wang, Cong Wang, Hanxin Zhu, Qingyun Sun, Chen Gao, Zhibo Chen, and Jianxin Li. 2026. Attention Sparsity is Input-Stable: Training-Free Sparse Attention for Video Generation via Offline Sparsity Profiling and Online QK Co-Clustering.Proceedings of the International Conference on Machine Learning(ICML)(2026)

    2026

  26. [26]

    Jiayi Luo, Jiayu Chen, Jiankun Wang, Cong Wang, Hanxin Zhu, Qingyun Sun, Chen Gao, Zhibo Chen, and Jianxin Li. 2026. Training-Free Sparse Attention for Fast Video Generation via Offline Layer-Wise Sparsity Profiling and Online Bidirectional Co-Clustering.arXiv preprint arXiv:2603.18636(2026)

    Pith/arXiv arXiv 2026

  27. [27]

    Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. 2023. Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378(2023)

    Pith/arXiv arXiv 2023

  28. [28]

    Zhengyao Lv, Chenyang Si, Junhao Song, Zhenyu Yang, Yu Qiao, Ziwei Liu, and Kwan-Yee K Wong. 2024. Fastercache: Training-free video diffusion model acceleration with high quality.arXiv preprint arXiv:2410.19355(2024)

    Pith/arXiv arXiv 2024

  29. [29]

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. 2024. Deepcache: Accelerating diffusion models for free. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 15762–15772

    2024

  30. [30]

    Zehong Ma, Longhui Wei, Feng Wang, Shiliang Zhang, and Qi Tian. 2025. Magcache: Fast video generation with magnitude-aware cache.arXiv preprint arXiv:2506.09045(2025)

    arXiv 2025

  31. [31]

    Rang Meng, Xingyu Zhang, Yuming Li, and Chenguang Ma. 2024. EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation.arXiv preprint arXiv:2411.10061(2024)

    arXiv 2024

  32. [32]

    William Peebles and Saining Xie. 2023. Scalable diffusion models with transform- ers. InProceedings of the IEEE/CVF international conference on computer vision. 4195–4205

    2023

  33. [33]

    Tim Salimans and Jonathan Ho. 2022. Progressive distillation for fast sampling of diffusion models.arXiv preprint arXiv:2202.00512(2022)

    Pith/arXiv arXiv 2022

  34. [34]

    Seedance Team, De Chen, Liyang Chen, Xin Chen, Ying Chen, Zhuo Chen, et al

  35. [35]

    Yuzhang Shang, Zhihang Yuan, Bin Xie, Bingzhe Wu, and Yan Yan. 2023. Post- training quantization on diffusion models. InProceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition. 1972–1981

    2023

  36. [36]

    Kaixin Shen, Ruijie Quan, Linchao Zhu, Jun Xiao, and Yi Yang. 2024. Audioscenic: Audio-driven video scene editing.arXiv preprint arXiv:2404.16581(2024)

    Pith/arXiv arXiv 2024

  37. [37]

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. 2023. Consistency models. (2023)

    2023

  38. [38]

    Meituan LongCat Team. 2025. LongCat-Video-Avatar Technical Report

    2025

  39. [39]

    Meituan LongCat Team, Xunliang Cai, Qilong Huang, Zhuoliang Kang, Hongyu Li, Shijun Liang, Liya Ma, Siyu Ren, Xiaoming Wei, Rixu Xie, et al. 2025. Longcat- video technical report.arXiv preprint arXiv:2510.22200(2025)

    arXiv 2025

  40. [40]

    Linrui Tian, Siqi Hu, Qi Wang, Bang Zhang, and Liefeng Bo. 2025. Emo2: End-effector guided audio-driven avatar video generation.arXiv preprint arXiv:2501.10687(2025)

    Pith/arXiv arXiv 2025

  41. [41]

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphaël Marinier, Marcin Michalski, and Sylvain Gelly. 2019. FVD: A new metric for video genera- tion. (2019)

    2019

  42. [42]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need.Advances in neural information processing systems30 (2017)

    2017

  43. [43]

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. 2025. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314(2025)

    Pith/arXiv arXiv 2025

  44. [44]

    Cong Wei, Bo Sun, Haoyu Ma, Ji Hou, Felix Juefei-Xu, Zecheng He, Xiaoliang Dai, Luxin Zhang, Kunpeng Li, Tingbo Hou, et al. 2025. Mocha: Towards movie-grade talking character synthesis.arXiv preprint arXiv:2503.23307(2025)

    Pith/arXiv arXiv 2025

  45. [45]

    Felix Wimbauer, Bichen Wu, Edgar Schoenfeld, Xiaoliang Dai, Ji Hou, Zijian He, Artsiom Sanakoyeu, Peizhao Zhang, Sam Tsai, Jonas Kohler, et al. 2024. Cache me if you can: Accelerating diffusion models through block caching. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6211–6220

    2024

  46. [46]

    Haocheng Xi, Shuo Yang, Yilong Zhao, Chenfeng Xu, Muyang Li, Xiuyu Li, Yujun Lin, Han Cai, Jintao Zhang, Dacheng Li, et al . 2025. Sparse videogen: Accelerating video diffusion transformers with spatial-temporal sparsity.arXiv preprint arXiv:2502.01776(2025). MM ’26, November 10–14, 2026, Rio de Janeiro, Brazil. Jiayu Chen et al

    Pith/arXiv arXiv 2025

  47. [47]

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al . 2024. Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072(2024)

    Pith/arXiv arXiv 2024

  48. [48]

    Haojie Yu, Zhaonian Wang, Yihan Pan, Meng Cheng, Hao Yang, Chao Wang, Tao Xie, Xiaoming Xu, Xiaoming Wei, and Xunliang Cai. 2025. LLIA–Enabling Low- Latency Interactive Avatars: Real-Time Audio-Driven Portrait Video Generation with Diffusion Models.arXiv preprint arXiv:2506.05806(2025)

    Pith/arXiv arXiv 2025

  49. [49]

    Zhihang Yuan, Hanling Zhang, Pu Lu, Xuefei Ning, Linfeng Zhang, Tianchen Zhao, Shengen Yan, Guohao Dai, and Yu Wang. 2024. Ditfastattn: Attention compression for diffusion transformer models.Advances in Neural Information Processing Systems37 (2024), 1196–1219

    2024

  50. [50]

    Lin Zhang, Shentong Mo, Yijing Zhang, and Pedro Morgado. 2024. Audio- Synchronized Visual Animation. InProceedings of the European Conference on Computer Vision (ECCV)

    2024

  51. [51]

    Wenxuan Zhang, Xiaodong Cun, Xuan Wang, Yong Zhang, Xi Shen, Yu Guo, Ying Shan, and Fei Wang. 2023. Sadtalker: Learning realistic 3d motion coefficients for stylized audio-driven single image talking face animation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 8652–8661

    2023

  52. [52]

    Zhimeng Zhang, Lincheng Li, Yu Ding, and Changjie Fan. 2021. Flow-Guided One- Shot Talking Face Generation With a High-Resolution Audio-Visual Dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3661–3670

    2021

  53. [53]

    Xuanlei Zhao, Xiaolong Jin, Kai Wang, and Yang You. 2024. Real-time video generation with pyramid attention broadcast.arXiv preprint arXiv:2408.12588 (2024)

    Pith/arXiv arXiv 2024

  54. [54]

    Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. 2024. Open-sora: Democratizing efficient video production for all.arXiv preprint arXiv:2412.20404(2024)

    Pith/arXiv arXiv 2024

  55. [55]

    Yingjie Zhou, Xilei Zhu, Siyu Ren, Ziyi Zhao, Ziwen Wang, Farong Wen, Yu Zhou, Jiezhang Cao, Xiongkuo Min, Fengjiao Chen, et al. 2025. EvalTalker: Learning to Evaluate Real-Portrait-Driven Multi-Subject Talking Humans.arXiv preprint arXiv:2512.01340(2025)

    arXiv 2025

  56. [2025]

    Bind-your-avatar: Multi-talking-character video generation with dynamic 3d-mask-based embedding router.arXiv preprint arXiv:2506.19833(2025)

    Pith/arXiv arXiv 2025

  57. [2026]

    Seedance 2.0: Advancing Video Generation for World Complexity.arXiv preprint arXiv:2604.14148(2026)

    Pith/arXiv arXiv 2026