Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

Skipped attention blocks in video diffusion can be recovered without training by compensating with cluster centroids and spending exact compute only where that approximation fails.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-15 12:20 UTC pith:VVQDY4FN

load-bearing objection Solid systems paper: parameter-free centroid compensation + error-to-cost routing beats score-based sparse attention on two large video DiTs, with a clean bound and real Pareto gains. the 2 major comments →

arxiv 2603.08982 v2 pith:VVQDY4FN submitted 2026-03-09 cs.CV

SVG-EAR: Parameter-Free Linear Compensation for Sparse Video Generation via Error-aware Routing

Xuanyi Zhou , Qiuyang Mang , Shuo Yang , Haocheng Xi , Jintao Zhang , Huanzhi Mao , Joseph E. Gonzalez , Kurt Keutzer
show 2 more authors
Ion Stoica Alvin Cheung
This is my paper
classification cs.CV
keywords sparse attentionvideo diffusion transformersparameter-free compensationerror-aware routingsemantic clusteringcentroid approximationquality-efficiency trade-off
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Video diffusion transformers spend most of their time on quadratic attention. Sparse methods cut that cost by computing only some blocks of the attention map, but usually either discard the rest (losing context) or train extra predictors to fill them in. This paper argues that, after semantic clustering, keys and values inside each block are similar enough that their contribution can be replaced by a single centroid interaction—no new parameters and no fine-tuning. Because that cheap substitute is accurate for most blocks but fails on a few, the method does not pick blocks by attention score; it probes the expected compensation error and spends the fixed budget on the blocks with the highest error-to-cost ratio. The result is higher fidelity at the same density, and higher throughput at the same quality, on large open video models.

Core claim

Once keys and values are grouped by semantic clustering, the contribution of any uncomputed attention block can be recovered by a parameter-free linear branch that uses the cluster centroids; under a fixed density budget the right blocks to compute exactly are those with the largest compensation error relative to their size, not those with the largest attention scores.

What carries the argument

SVG-EAR: a dual mechanism that (1) approximates skipped query–key blocks by replacing keys and values with their cluster means, and (2) routes exact computation by a lightweight probe of per-block compensation error, greedily selecting highest error-to-cost blocks under a density budget.

Load-bearing premise

The method assumes that swapping a block’s true logits for their centroid versions does not meaningfully change each query’s softmax normalizer; if that normalizer shifts a lot, the error ranking and the theoretical bound both break.

What would settle it

On the same video models and prompts, measure true attention-map MSE and generation PSNR when blocks are chosen by error-to-cost versus by attention score at identical density; if error-aware routing does not reduce MSE or raise PSNR, or if measured normalizer shifts are large, the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • At matched density, attention reconstruction error falls and generation PSNR/SSIM/LPIPS improve relative to score-based sparse baselines.
  • At matched quality targets, the method can run at lower density and deliver higher end-to-end throughput (reported up to about 1.8–1.9× on the evaluated models).
  • Training-free sparse video attention becomes plug-and-play: no learned compensators and no distribution shift from fine-tuning.
  • Routing overhead can be kept negligible with streaming fused kernels, so the quality gain does not cancel the wall-clock speedup.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same centroid-plus-error-routing pattern may transfer to other long-context DiT or multimodal settings where tokens already form tight semantic clusters.
  • If clustering quality is weak (high average query–centroid distance), the residual term in the bound grows and the method should fall back toward denser exact attention or better clustering.
  • Value-aware error probes (including value means) could further tighten routing when attention mass is diffuse but value diversity is high.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes SVG-EAR, a training-free sparse attention method for video Diffusion Transformers. After semantic clustering of queries and keys, it computes a subset of blocks exactly and recovers the rest via a parameter-free linear compensation that replaces keys/values by cluster centroids. Block selection is error-aware: a lightweight O(Cq Nk d) probe estimates per-block compensation error (optionally value-aware), and blocks are greedily chosen by error-to-cost ratio under a density budget. A relaxed knapsack objective and an MSE bound (Proposition 1 / Appendix A) relate reconstruction error to clustering quality under a normalizer-stability assumption. On Wan2.2 and HunyuanVideo (720p), SVG-EAR reports a better quality–efficiency Pareto frontier than SVG, SVG2, and SpargeAttn, with up to 1.77×–1.93× speedups at PSNRs of 29.759 and 31.043.

Significance. If the results hold, the work is a useful systems-and-algorithms contribution for video DiT inference: it cleanly separates (i) parameter-free centroid compensation from (ii) error-to-cost routing, shows that score-based selection is misaligned once compensation exists, and backs the design with a transparent error bound, fused Triton kernels, and consistent gains on two strong open models. Strengths include the training-free plug-and-play design, the explicit theory–clustering link (Fig. 6b), the MSE-vs-density ablation (Fig. 6a), and end-to-end latency breakdowns that keep routing overhead small. The main practical impact is a clearer Pareto frontier for sparse video attention without fine-tuning or extra parameters.

major comments (2)
  1. [§4.2, Proposition 1, Appendix A] §4.1–4.2, Eq. (3) and Proposition 1 / Appendix A: the relaxed objective and the MSE bound both rest on the normalizer-stability assumption (masked logits replaced by centroid counterparts do not significantly change Zi). The paper motivates this via clustering and high-score dominance of unmasked tokens, and Fig. 6a shows error-aware routing still reduces attention MSE relative to top-p + compensation. That is supportive but indirect. A load-bearing check is still missing: report, on the same sampled heads/timesteps used for Fig. 6, the relative change in Zi (or in the full-map MSE ranking) when masked entries are replaced by centroids, and/or the Spearman correlation between estimated error-to-cost ratios and true per-block contribution to attention MSE. Without this, the claim that the probe ranks the blocks that matter most remains only partially validated.
  2. [§5, Table 1] §5, Table 1 / Table 4: quality is measured on 50 randomly selected prompts (T2V and I2V) with no reported variance, confidence intervals, or multi-seed statistics. The headline PSNR gaps (e.g., SVG-EAR 29.759 vs SVG2 27.668 on Wan I2V; 31.043 vs 29.445 on Hunyuan) are large enough to be plausible, but for a Pareto-frontier claim they should be accompanied by at least seed/prompt-split variability or a larger fixed public subset. This is fixable within the current experimental protocol and does not invalidate the method, but it is currently under-supported for the strength of the abstract claim.
minor comments (5)
  1. [§4.3, Eq. (8)] §4.3 introduces a value-aware error ˜ϵ² (Eq. 8) that is said to be ‘practically more accurate,’ but Table 1 and Fig. 6 do not state whether routing used ˆϵ or ˜ϵ. Please specify the default used in all main results and, if space allows, a one-line ablation.
  2. [Fig. 2, Fig. 4] Fig. 2 and Fig. 4 captions refer to ‘top-p selection (SVG2)’ and ‘mean compensation’ without stating the exact density/top-p used in those visualizations; aligning them with the Table 3 configs would help reproducibility.
  3. [Appendix C, Table 2] Appendix C Table 2 header says ‘Wan 2.1’ while the main text uses Wan2.2; please correct the model name for consistency.
  4. [§6] §6 limitation paragraph is very brief. A short note on applicability to non-DiT attention (or to image-only DiTs) and on sensitivity to QC/KC would better match the scope of the claims.
  5. [§4.1] Minor notation: M is used both as the binary routing mask and (in places) as a density-related quantity; a single consistent definition early in §4.1 would reduce ambiguity.

Circularity Check

0 steps flagged

No circularity: theory is a first-principles bound under stated assumptions; empirics compare to independent full-attention and prior sparse baselines.

full rationale

SVG-EAR’s derivation chain is self-contained and non-circular. The optimization target (Eq. 2: minimize attention-map MSE under a density budget) is relaxed to an unnormalized proxy (Eq. 3) for tractability; error-aware routing then greedily selects blocks by estimated error-to-cost ratio (Eqs. 5/8). Proposition 1 / Appendix A derives an upper bound relating true MSE to that estimated error plus a residual controlled by clustering quality δ²_q and sequence length, under an explicit normalizer-stability assumption—not by fitting parameters to the reported PSNR numbers. The residual term vanishes as clustering improves, which is an independent, falsifiable claim checked empirically in Fig. 6(b). Parameter-free centroid compensation (Eqs. 1, 7) is a direct algebraic use of cluster means after semantic clustering; it is not defined in terms of the quality metrics it is later evaluated against. Self-citations to SVG/SVG2 supply clustering, permutation, and kernel infrastructure and budget allocation, which is ordinary engineering reuse, not a load-bearing uniqueness theorem or ansatz that forces the central claim. The Pareto claim is purely empirical: PSNR/SSIM/LPIPS and speedups are measured against full attention and prior sparse methods (SpargeAttn, SVG, SVG2) on public models (Wan2.2, HunyuanVideo). No step reduces a “prediction” to a fitted input or to a definitional identity with its own inputs.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 0 invented entities

The central claim rests on standard attention algebra, the empirical observation that semantic clustering produces high within-block similarity, a handful of hand-chosen density and cluster-count hyper-parameters inherited from SVG2, and one non-trivial modeling assumption (normalizer stability) needed for both the routing proxy and the theoretical bound. No new physical entities are postulated.

free parameters (4)
  • number of query clusters QC
    Chosen by hand (default 300/400, turbo 200); controls both clustering quality and probe cost; not derived from data but fixed for all reported runs.
  • number of key clusters KC
    Chosen by hand (default 1000, turbo 500); same role as QC.
  • density / top-p budget ρ
    User- or baseline-inherited compute budget (0.85 default, 0.8 turbo); directly determines which blocks are exact vs compensated.
  • time-warm and layer-warm steps
    Inherited from SVG2 (typically first 20 % of diffusion steps and first layer kept dense); affect the measured speedup and quality.
axioms (3)
  • domain assumption After semantic clustering, keys and values inside a block are sufficiently similar that their contribution can be replaced by the cluster centroid with small error for most blocks.
    Stated as the core empirical observation in §3 and used throughout the compensation formula (Eq. 1, 7).
  • ad hoc to paper Normalizer-stability: replacing any masked logit by its centroid counterpart does not significantly change the softmax normalizer Zi for any query.
    Explicitly assumed for the relaxed objective (Eq. 3) and for Proposition 1 / Appendix A; without it the error-to-cost ranking and the bound are not justified.
  • standard math Standard scaled-dot-product attention and softmax algebra.
    Used for all attention-map definitions and the MSE objective.

pith-pipeline@v1.1.0-grok45 · 25428 in / 2624 out tokens · 26473 ms · 2026-07-15T12:20:02.657992+00:00 · methodology

0 comments
read the original abstract

Diffusion Transformers (DiTs) have become a leading backbone for video generation, yet their quadratic attention cost remains a major bottleneck. Sparse attention reduces this cost by computing only a subset of attention blocks. However, prior methods often either drop the remaining blocks, which incurs information loss, or rely on learned predictors to approximate them, introducing training overhead and potential output distribution shifting. In this paper, we show that the missing contributions can be recovered without training: after semantic clustering, keys and values within each block exhibit strong similarity and can be well summarized by a small set of cluster centroids. Based on this observation, we introduce SVG-EAR, a parameter-free linear compensation branch that uses the centroid to approximate skipped blocks and recover their contributions. While centroid compensation is accurate for most blocks, it can fail on a small subset. Standard sparsification typically selects blocks by attention scores, which indicate where the model places its attention mass, but not where the approximation error would be largest. SVG-EAR therefore performs error-aware routing: a lightweight probe estimates the compensation error for each block, and we compute exactly the blocks with the highest error-to-cost ratio while compensating for skipped blocks. We provide theoretical guarantees that relate attention reconstruction error to clustering quality, and empirically show that SVG-EAR improves the quality-efficiency trade-off and increases throughput at the same generation fidelity on video diffusion tasks. Overall, SVG-EAR establishes a clear Pareto frontier over prior approaches, achieving up to 1.77$\times$ and 1.93$\times$ speedups while maintaining PSNRs of up to 29.759 and 31.043 on Wan2.2 and HunyuanVideo, respectively.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. HyperVAttention: Efficient Sparse Attention with Spatio-Temporal Clustering for Video Diffusion

    cs.CV 2026-07 conditional novelty 5.5

    Training-free sparse attention for video DiTs cuts latency up to ~2.1× via 3D local-window clustering, hybrid step updates, and hardware-aware cluster merging while improving fidelity over prior sparse methods.

Reference graph

Works this paper leans on

26 extracted references · 14 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Monarchrt: Efficient attention for real-time video generation.arXiv preprint arXiv:2602.12271,

    Krish Agarwal, Zhuoming Chen, Cheng Luo, Yongqi Chen, Haizhong Zheng, Xun Huang, Atri Rudra, and Beidi Chen. Monarchrt: Efficient attention for real-time video generation.arXiv preprint arXiv:2602.12271,

    arXiv

  2. [2]

    Dicache: Let diffusion model determine its own cache.arXiv preprint arXiv:2508.17356,

    Jiazi Bu, Pengyang Ling, Yujie Zhou, Yibin Wang, Yuhang Zang, Dahua Lin, and Jiaqi Wang. Dicache: Let diffusion model determine its own cache.arXiv preprint arXiv:2508.17356,

    arXiv

  3. [3]

    Mixture of contexts for long video generation.arXiv preprint arXiv:2508.21058,

    Shengqu Cai, Ceyuan Yang, Lvmin Zhang, Yuwei Guo, Junfei Xiao, Ziyan Yang, Yinghao Xu, Zhenheng Yang, Alan Yuille, Leonidas Guibas, et al. Mixture of contexts for long video generation.arXiv preprint arXiv:2508.21058,

    arXiv

  4. [4]

    Rainfusion2

    Aiyue Chen, Yaofu Liu, Junjian Huang, Guang Lian, Yiwu Yao, Wangli Lan, Jing Lin, Zhixin Ma, Tingting Zhou, and Harry Yang. Rainfusion2. 0: Temporal-spatial awareness and hardware-efficient block-wise sparse attention.arXiv preprint arXiv:2512.24086, 2025a. Junsong Chen, Yuyang Zhao, Jincheng Yu, Ruihang Chu, Junyu Chen, Shuai Yang, Xianbang Wang, Yicheng...

    Pith/arXiv arXiv

  5. [5]

    Omnicache: A trajectory-oriented global perspective on training-free cache reuse for diffusion transformer models

    11 Huanpeng Chu, Wei Wu, Guanyu Feng, and Yutao Zhang. Omnicache: A trajectory-oriented global perspective on training-free cache reuse for diffusion transformer models. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 16302–16312, 2025a. Xiangxiang Chu, Renda Li, and Yong Wang. Usp: Unified self-supervised pretraining for ...

    Pith/arXiv arXiv

  6. [6]

    Streamdiffusionv2: A streaming system for dynamic and interactive video generation.arXiv preprint arXiv:2511.07399,

    Tianrui Feng, Zhi Li, Shuo Yang, Haocheng Xi, Muyang Li, Xiuyu Li, Lvmin Zhang, Keting Yang, Kelly Peng, Song Han, et al. Streamdiffusionv2: A streaming system for dynamic and interactive video generation.arXiv preprint arXiv:2511.07399,

    arXiv

  7. [7]

    Longvie 2: Multimodal controllable ultra-long video world model.arXiv preprint arXiv:2512.13604,

    Jianxiong Gao, Zhaoxi Chen, Xian Liu, Junhao Zhuang, Chengming Xu, Jianfeng Feng, Yu Qiao, Yanwei Fu, Chenyang Si, and Ziwei Liu. Longvie 2: Multimodal controllable ultra-long video world model.arXiv preprint arXiv:2512.13604,

    arXiv

  8. [8]

    Matten: Video generation with mamba-attention.arXiv preprint arXiv:2405.03025,

    Yu Gao, Jiancheng Huang, Xiaopeng Sun, Zequn Jie, Yujie Zhong, and Lin Ma. Matten: Video generation with mamba-attention.arXiv preprint arXiv:2405.03025,

    Pith/arXiv arXiv

  9. [9]

    M4v: Multi-modal mamba for text-to-video generation.arXiv preprint arXiv:2506.10915, 2025a

    Jiancheng Huang, Gengwei Zhang, Zequn Jie, Siyu Jiao, Yinlong Qian, Ling Chen, Yunchao Wei, and Lin Ma. M4v: Multi-modal mamba for text-to-video generation.arXiv preprint arXiv:2506.10915, 2025a. Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion.arXiv preprint ar...

    Pith/arXiv arXiv

  10. [10]

    Hunyuanworld 1.0: Generating immersive, explorable, and interactive 3d worlds from words or pixels.arXiv preprint, 2025a

    Team HunyuanWorld. Hunyuanworld 1.0: Generating immersive, explorable, and interactive 3d worlds from words or pixels.arXiv preprint, 2025a. Team HunyuanWorld. Hy-world 1.5: A systematic framework for interactive world modeling with real-time latency and geometric consistency.arXiv preprint, 2025b. Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia ...

    Pith/arXiv arXiv

  11. [11]

    Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603,

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603,

    Pith/arXiv arXiv

  12. [12]

    Pisa: Piecewise sparse attention is wiser for efficient diffusion transformers.arXiv preprint arXiv:2602.01077,

    Haopeng Li, Shitong Shao, Wenliang Zhong, Zikai Zhou, Lichen Bai, Hui Xiong, and Zeke Xie. Pisa: Piecewise sparse attention is wiser for efficient diffusion transformers.arXiv preprint arXiv:2602.01077,

    arXiv

  13. [13]

    Distrifusion: Distributed parallel inference for high-resolution diffusion models

    Muyang Li, Tianle Cai, Jiaxin Cao, Qinsheng Zhang, Han Cai, Junjie Bai, Yangqing Jia, Kai Li, and Song Han. Distrifusion: Distributed parallel inference for high-resolution diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7183–7193, 2024a. Muyang Li, Yujun Lin, Zhekai Zhang, Tianle Cai, Xiuyu Li,...

    arXiv

  14. [14]

    Timestep embedding tells: It’s time to cache for video diffusion model

    Feng Liu, Shiwei Zhang, Xiaofeng Wang, Yujie Wei, Haonan Qiu, Yuzhong Zhao, Yingya Zhang, Qixiang Ye, and Fang Wan. Timestep embedding tells: It’s time to cache for video diffusion model. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 7353–7363, 2025a. Hao Liu, Matei Zaharia, and Pieter Abbeel. Ring attention with blockwise...

    Pith/arXiv arXiv

  15. [15]

    Rolling forcing: Autoregressive long video diffusion in real time.arXiv preprint arXiv:2509.25161, 2025b

    Kunhao Liu, Wenbo Hu, Jiale Xu, Ying Shan, and Shijian Lu. Rolling forcing: Autoregressive long video diffusion in real time.arXiv preprint arXiv:2509.25161, 2025b. Yuxi Liu, Yipeng Hu, Zekun Zhang, Kunze Jiang, and Kun Yuan. Mixture of distributions matters: Dynamic sparse attention for efficient video diffusion transformers.arXiv preprint arXiv:2601.11641,

    Pith/arXiv arXiv

  16. [16]

    Simplifying, stabilizing and scaling continuous-time consistency models, 2025.https: //arxiv.org/abs/2410.11081

    Cheng Lu and Yang Song. Simplifying, stabilizing and scaling continuous-time consistency models, 2025.https: //arxiv.org/abs/2410.11081. Chengtao Lv, Yumeng Shi, Yushi Huang, Ruihao Gong, Shen Ren, and Wenya Wang. Light forcing: Accelerating autoregressive video diffusion via sparse attention.arXiv preprint arXiv:2602.04789,

    Pith/arXiv arXiv 2025

  17. [17]

    Fastercache: Training-free video diffusion model acceleration with high quality.arXiv preprint arXiv:2410.19355,

    Zhengyao Lv, Chenyang Si, Junhao Song, Zhenyu Yang, Yu Qiao, Ziwei Liu, and Kwan-Yee K Wong. Fastercache: Training-free video diffusion model acceleration with high quality.arXiv preprint arXiv:2410.19355,

    Pith/arXiv arXiv

  18. [18]

    Learning-to-cache: Accelerating diffusion transformer via layer caching.Advances in Neural Information Processing Systems, 37:133282–133304, 2024a

    Xinyin Ma, Gongfan Fang, Michael Bi Mi, and Xinchao Wang. Learning-to-cache: Accelerating diffusion transformer via layer caching.Advances in Neural Information Processing Systems, 37:133282–133304, 2024a. Xinyin Ma, Gongfan Fang, and Xinchao Wang. Deepcache: Accelerating diffusion models for free. InProceedings of the IEEE/CVF conference on computer visi...

    arXiv

  19. [19]

    Vorta: Efficient video diffusion via routing sparse attention.arXiv preprint arXiv:2505.18809, 2025a

    Wenhao Sun, Rong-Cheng Tu, Yifu Ding, Zhao Jin, Jingyi Liao, Shunyu Liu, and Dacheng Tao. Vorta: Efficient video diffusion via routing sparse attention.arXiv preprint arXiv:2505.18809, 2025a. Wenqiang Sun, Haiyu Zhang, Haoyuan Wang, Junta Wu, Zehan Wang, Zhenwei Wang, Yunhong Wang, Jun Zhang, Tengfei Wang, and Chunchao Guo. Worldplay: Towards long-term ge...

    arXiv

  20. [20]

    Vmoba: Mixture-of-block attention for video diffusion models.arXiv preprint arXiv:2506.23858, 2025a

    Jianzong Wu, Liang Hou, Haotian Yang, Xin Tao, Ye Tian, Pengfei Wan, Di Zhang, and Yunhai Tong. Vmoba: Mixture-of-block attention for video diffusion models.arXiv preprint arXiv:2506.23858, 2025a. Junyi Wu, Haoxuan Wang, Yuzhang Shang, Mubarak Shah, and Yan Yan. Ptq4dit: Post-training quantization for diffusion transformers.Advances in neural information ...

    Pith/arXiv arXiv

  21. [21]

    Usv: Unified sparsification for accelerating video diffusion models.arXiv preprint arXiv:2512.05754, 2025b

    Xinjian Wu, Hongmei Wang, Yuan Zhou, and Qinglin Lu. Usv: Unified sparsification for accelerating video diffusion models.arXiv preprint arXiv:2512.05754, 2025b. Haocheng Xi, Shuo Yang, Yilong Zhao, Chenfeng Xu, Muyang Li, Xiuyu Li, Yujun Lin, Han Cai, Jintao Zhang, Dacheng Li, Jianfei Chen, Ion Stoica, Kurt Keutzer, and Song Han. Sparse videogen: Accelera...

    arXiv 2025

  22. [22]

    Training-free and adaptive sparse attention for efficient long video generation, 2025.https://arxiv.org/abs/2502.21079

    Yifei Xia, Suhan Ling, Fangcheng Fu, Yujie Wang, Huixia Li, Xuefeng Xiao, and Bin Cui. Training-free and adaptive sparse attention for efficient long video generation, 2025.https://arxiv.org/abs/2502.21079. Shuai Yang, Wei Huang, Ruihang Chu, Yicheng Xiao, Yuyang Zhao, Xianbang Wang, Muyang Li, Enze Xie, Yingcong Chen, Yao Lu, et al. Longlive: Real-time i...

    Pith/arXiv arXiv 2025

  23. [23]

    Sageattention2: Efficient attention with thorough outlier smoothing and per-thread int4 quantization

    Jintao Zhang, Haofeng Huang, Pengle Zhang, Jia Wei, Jun Zhu, and Jianfei Chen. Sageattention2: Efficient attention with thorough outlier smoothing and per-thread int4 quantization. InInternational Conference on Machine Learning (ICML), 2025a. Jintao Zhang, Rundong Su, Chunyu Liu, Jia Wei, Ziteng Wang, Haoxu Wang, Pengle Zhang, Huiqiang Jiang, Haofeng Huan...

    arXiv

  24. [24]

    Large scale diffusion distillation via score-regularized continuous-time consistency.arXiv preprint arXiv:2510.08431,

    14 Kaiwen Zheng, Yuji Wang, Qianli Ma, Huayu Chen, Jintao Zhang, Yogesh Balaji, Jianfei Chen, Ming-Yu Liu, Jun Zhu, and Qinsheng Zhang. Large scale diffusion distillation via score-regularized continuous-time consistency.arXiv preprint arXiv:2510.08431,

    Pith/arXiv arXiv

  25. [25]

    Causal forcing: Autoregressive diffusion distillation done right for high-quality real-time interactive video generation.arXiv preprint arXiv:2602.02214,

    Hongzhou Zhu, Min Zhao, Guande He, Hang Su, Chongxuan Li, and Jun Zhu. Causal forcing: Autoregressive diffusion distillation done right for high-quality real-time interactive video generation.arXiv preprint arXiv:2602.02214,

    Pith/arXiv arXiv

  26. [26]

    Table 2VBench result ofSVG-EAR. ConfigSubConsis BackConsis MotionSmooth AesQual ImagQual Wan 2.2 14B, 720P, I2V0.960 0.960 0.987 0.628 0.704 SVG 0.958 0.959 0.9890.627 0.703 Sparge 0.958 0.957 0.987 0.6270.703 SVG2 0.958 0.957 0.986 0.624 0.701 SVG-EAR 0.959 0.959 0.986 0.627 0.703 SVG-EAR-Turbo 0.958 0.958 0.987 0.625 0.702 Wan 2.1 14B, 720P, Text-to-Vid...

    arXiv