pith. sign in

arxiv: 2606.23019 · v1 · pith:BLBKFHNDnew · submitted 2026-06-22 · 💻 cs.CV · cs.AI

ScalingAttention: Discovering Intrinsic Sparse Attention Topology for Video Diffusion Transformers

Ruiliang Zhou , Xuecheng Wu , Kang He , Guangyun Han , Bin Liu , Qinqin Chen , Wende Xu , Qingjie Zhao
show 1 more author
Chengru Song
This is my paper

Pith reviewed 2026-06-26 09:20 UTC · model grok-4.3

classification 💻 cs.CV cs.AI
keywords sparse attentionvideo diffusion transformersintrinsic sparse topologytraining-free accelerationblock-sparse attentionDiTWESTFAST
0
0 comments X

The pith

Video diffusion transformers converge to a stable, prompt-agnostic sparse attention topology encoded in their weights.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that although individual activations in video Diffusion Transformers depend on the input, the high-mass attention regions per head rapidly settle into one fixed pattern that does not change with the prompt. This pattern is stored in the model weights, stays the same across scales, and can be pulled out once without any training or per-input search. ScalingAttention turns this observation into a practical method by separating the discovery of the sparse pattern from the choice of how sparse each head should be, then pairing the result with a hardware-friendly kernel. The result is faster inference that keeps or improves generation quality on existing video DiT models.

Core claim

While individual activations are input-dependent, the high-mass attention regions for each head rapidly converge to a stable, prompt-agnostic Intrinsic Sparse Topology. This topology is weight-encoded, scale-invariant, and efficient to extract. ScalingAttention decouples topology discovery from sparsity control via WEST, which extracts a robust block-sparse prior mask offline, and FAST, which adaptively tunes head-wise sparsity based on diffusion fidelity requirements.

What carries the argument

The Intrinsic Sparse Topology: a weight-encoded, prompt-agnostic block-sparse attention pattern per head that remains stable across inputs and can be extracted offline without runtime search.

If this is right

  • WEST extracts a robust block-sparse prior mask offline to remove any need for runtime topology search.
  • FAST adaptively sets head-wise sparsity levels according to each head's contribution to diffusion fidelity.
  • A co-designed bit-wise block-sparse kernel delivers practical wall-clock acceleration on existing hardware.
  • The method reaches up to 1.90X end-to-end speedup on Wan2.1 while matching or exceeding full-attention fidelity.
  • The resulting sparse models sit on a new Pareto frontier relative to prior dynamic and static sparse baselines.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the topology is truly weight-encoded and stable, similar fixed patterns may exist in other transformer families and could be pre-computed once for an entire model family.
  • The separation of topology discovery from sparsity tuning suggests a general recipe for turning dense attention into sparse attention in any large transformer without retraining.
  • Offline extraction could reduce memory fragmentation during inference, making large video models easier to deploy on devices with limited on-chip memory.
  • Because the topology is claimed to be scale-invariant, the same mask might transfer across model sizes trained on the same data distribution.

Load-bearing premise

The high-mass attention regions for each head converge rapidly to a single stable, prompt-agnostic topology that can be extracted offline from weights without runtime search or fidelity loss.

What would settle it

Extract the candidate topology from weights once, then measure whether high-mass regions stay identical when the same head processes many different video prompts; large variation across prompts would falsify the prompt-agnostic claim.

Figures

Figures reproduced from arXiv: 2606.23019 by Bin Liu, Chengru Song, Guangyun Han, Kang He, Qingjie Zhao, Qinqin Chen, Ruiliang Zhou, Wende Xu, Xuecheng Wu.

Figure 1
Figure 1. Figure 1: Performance Comparison. We compare ScalingAtten￾tion with SVG and SVG2 on Video DiTs. At comparable PSNR, ScalingAttention uses up to 2× fewer attention FLOPs (i.e., lower density) than SVG2, demonstrating a superior efficiency–fidelity trade-off. Here, density is defined as 1 − Sparsity and represents the fraction of active attention blocks; this convention is used con￾sistently throughout the paper. We t… view at source ↗
Figure 2
Figure 2. Figure 2: Empirical Discovery of Intrinsic Sparse Topology. We reveal that attention sparsity in Video DiTs is governed by a static, weight-encoded structure rather than transient input variations. (a) Sparse Topology (Top-Left): While individual prompts (A vs. B) activate distinct sub-regions, their union converges to a stable, prompt-agnostic boundary, revealing a latent topology. (b) Scale Invariance (Bottom-Left… view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the ScalingAttention Framework. Our method decouples sparse attention into three phases: (a) Once per Model (WEST): We aggregate intrinsic attention patterns from a calibration set to construct a static Threshold Map, which encodes the complete sparsity hierarchy offline. (b) Once per Setting (FAST): Given a global sparsity target, we modulate a spatio-temporal fidelity surface to determine hea… view at source ↗
Figure 4
Figure 4. Figure 4: Metric Comparison. Cosine similarity (orange) satu￾rates prematurely as sparsity increases, failing to capture structural loss. The proposed Fidelity Score (blue) maintains a smooth, monotonic response, enabling precise calibration. Compared to total variation, its square-root geometry pro￾vides smoother sensitivity under progressive sparsification. For dense attention P and its sparse approximation P˜, we… view at source ↗
Figure 5
Figure 5. Figure 5: Kernel-Level Efficiency Benchmarking. We compare the normalized latency of our crm kernel kernel against FlashAttention-3 (FA3) across sequence lengths from 1K to 262K. The orange bar represents the FA3 baseline (1.0). Even at 0% sparsity (Blue), our kernel incurs minimal overhead (< 10%) due to efficient bit-mask loading. As sparsity increases (Green to Grey), latency drops significantly, demonstrating li… view at source ↗
Figure 6
Figure 6. Figure 6: Dense Kernel Overhead on Short Sequences. We compare the absolute latency (ms) of crm kernel vs. FA3 at 0% sparsity. Even on short sequences where kernel launch overheads are typically more pronounced, crm kernel maintains a comparable latency profile to FA3, with a maximum overhead of only 9.0% at N = 16384. D. Detailed Sensitivity Analysis As discussed in Section 5.4 of the main paper, we evaluate the co… view at source ↗
Figure 7
Figure 7. Figure 7: Threshold Map Gallery for WEST. We visualize representative attention structures across layers/heads and diffusion timesteps (e.g., t=0, 20, 40). Despite diverse per-prompt activations, the stable support envelope captured by WEST remains consistent, supporting the existence of an intrinsic, weight-encoded sparse topology. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Appendix visualization of aspect-ratio robustness. For the same prompt under different aspect-ratio settings, the generated videos exhibit substantial semantic-layout differences, yet the recovered sparse attention support remains nearly unchanged. This provides an additional view that the Intrinsic Sparse Topology is primarily weight-encoded rather than input-specific. 17 [PITH_FULL_IMAGE:figures/full_fi… view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative comparison on HunyuanVideo. Side-by-side frames from dense attention and ScalingAttention at global density of 55%. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_9.png] view at source ↗
read the original abstract

While Diffusion Transformers (DiTs) have revolutionized high-fidelity video generation, their reliance on 3D full attention creates a quadratic computational bottleneck. Existing sparse methods face a dilemma: dynamic pruning suffers from prohibitive runtime overhead and memory fragmentation, while static heuristics fail to capture fine-grained dependencies. In this work, we propose ScalingAttention, a training-free framework grounded in a key inductive bias: while individual activations are input-dependent, the high-mass attention regions for each head rapidly converge to a stable, prompt-agnostic Intrinsic Sparse Topology. This topology is weight-encoded, scale-invariant, and efficient to extract. ScalingAttention decouples topology discovery from sparsity control via: (1) WEST (Weight-Encoded Sparse Topology), which extracts a robust block-sparse prior mask offline to eliminate runtime search; (2) FAST (Fidelity-Aware Sensitivity Tuning), which adaptively tunes head-wise sparsity based on diffusion fidelity requirements. To ensure practical acceleration, we co-design a hardware-aligned bit-wise block-sparse kernel. Experiments on Wan2.1 show up to 1.90X end-to-end speedup with superior fidelity, establishing a new Pareto frontier over state-of-the-art baselines.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 0 minor

Summary. The manuscript introduces ScalingAttention, a training-free sparse attention framework for Video Diffusion Transformers. It rests on the inductive bias that, despite input-dependent activations, high-mass attention regions per head converge rapidly to a stable, prompt-agnostic Intrinsic Sparse Topology that is weight-encoded and scale-invariant. The method decouples discovery from control via WEST (offline extraction of a block-sparse prior mask) and FAST (head-wise fidelity-aware sparsity tuning), paired with a hardware-aligned bit-wise block-sparse kernel. Experiments on Wan2.1 are reported to yield up to 1.90X end-to-end speedup with superior fidelity relative to baselines.

Significance. If the central inductive bias is substantiated, the work would offer a practical route to static, training-free sparsification of 3D attention in video DiTs, sidestepping both the overhead of dynamic pruning and the limitations of hand-crafted heuristics. The explicit hardware co-design for the block-sparse kernel is a concrete strength that could translate to real deployment gains. The result would meaningfully advance the efficiency frontier for large-scale video generation models.

major comments (3)
  1. [Abstract, inductive bias paragraph] Abstract / inductive-bias paragraph: the claim that high-mass attention regions 'rapidly converge to a stable, prompt-agnostic Intrinsic Sparse Topology' is load-bearing for the training-free, offline-extraction guarantee, yet the manuscript supplies neither a derivation of the convergence, statistical validation across prompts/timesteps/content, nor ablation showing invariance; without this evidence the 'prompt-agnostic' and 'no fidelity loss' assertions remain untested.
  2. [Abstract] Abstract: the reported 1.90X speedup and 'superior fidelity' are presented without error bars, comparison methodology, or description of the evaluation protocol (e.g., which prompts, timesteps, or metrics), rendering the Pareto-frontier claim impossible to assess from the given text.
  3. [Inductive bias paragraph] Inductive-bias paragraph: the definition of 'high-mass' regions used to extract the topology may implicitly rely on the same attention statistics that the subsequent sparsification removes, creating a circularity risk that is not addressed by any sensitivity analysis or alternative extraction procedure.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. We address each major comment point-by-point below, indicating where revisions will be made to strengthen the manuscript.

read point-by-point responses

  1. Referee: [Abstract, inductive bias paragraph] Abstract / inductive-bias paragraph: the claim that high-mass attention regions 'rapidly converge to a stable, prompt-agnostic Intrinsic Sparse Topology' is load-bearing for the training-free, offline-extraction guarantee, yet the manuscript supplies neither a derivation of the convergence, statistical validation across prompts/timesteps/content, nor ablation showing invariance; without this evidence the 'prompt-agnostic' and 'no fidelity loss' assertions remain untested.

    Authors: The central inductive bias is grounded in empirical observations from our experiments on Wan2.1, where attention mass patterns showed rapid stabilization across diverse prompts, timesteps, and content. While a theoretical derivation of convergence is not provided (as the work is primarily empirical), we agree that additional statistical validation and invariance ablations would strengthen the claims. In the revised manuscript we will add a new subsection with quantitative metrics (e.g., average Jaccard overlap of high-mass blocks across 200 prompts) and ablation tables demonstrating prompt- and timestep-invariance. revision: yes

  2. Referee: [Abstract] Abstract: the reported 1.90X speedup and 'superior fidelity' are presented without error bars, comparison methodology, or description of the evaluation protocol (e.g., which prompts, timesteps, or metrics), rendering the Pareto-frontier claim impossible to assess from the given text.

    Authors: We agree the abstract is too terse on evaluation details. The main paper (Section 4) specifies the protocol: 100 diverse video prompts, metrics including FID, CLIP similarity, and user studies, with error bars reported in Tables 2-4 and comparisons against dynamic and static baselines under identical settings. We will revise the abstract to include a brief statement of the evaluation protocol and note that detailed results with variance appear in the experiments section. revision: yes

  3. Referee: [Inductive bias paragraph] Inductive-bias paragraph: the definition of 'high-mass' regions used to extract the topology may implicitly rely on the same attention statistics that the subsequent sparsification removes, creating a circularity risk that is not addressed by any sensitivity analysis or alternative extraction procedure.

    Authors: High-mass regions are defined from full (dense) attention maps computed on a fixed calibration set of prompts before any sparsification occurs; the resulting block-sparse mask is then applied at inference. This extraction is offline and independent of the runtime sparse computation, avoiding circularity. We will clarify this distinction in the revised text and add a sensitivity analysis comparing alternative aggregation procedures (e.g., mean vs. max pooling of attention weights) to the WEST extraction. revision: yes

Circularity Check

0 steps flagged

No circularity: inductive bias presented as empirical premise, extraction claimed weight-only

full rationale

The abstract states the core premise as an inductive bias (high-mass regions converge to a stable, prompt-agnostic, weight-encoded topology) and describes WEST as an offline extraction from weights. No equations, self-citations, or derivation steps are supplied that define the topology in terms of the attention statistics it sparsifies, fit a parameter on a subset and rename the output as prediction, or reduce the claimed result to its own inputs by construction. The method is therefore self-contained against the supplied text; the reader's moderate risk note remains speculative without explicit reduction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

Central claim rests on one domain assumption about attention convergence and introduces the Intrinsic Sparse Topology as a new entity without external falsifiable evidence.

axioms (1)
  • domain assumption High-mass attention regions for each head converge rapidly to a stable, prompt-agnostic topology
    Presented as the key inductive bias that grounds the entire framework.
invented entities (1)
  • Intrinsic Sparse Topology no independent evidence
    purpose: Provides a robust block-sparse prior mask that is weight-encoded and prompt-agnostic
    New postulated structure extracted offline; no independent evidence supplied beyond the claim.

pith-pipeline@v0.9.1-grok · 5761 in / 1324 out tokens · 34150 ms · 2026-06-26T09:20:54.344932+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

42 extracted references · 1 canonical work pages

  1. [1]

    2503.20314 , archivePrefix=

    Team Wan and Ang Wang and Baole Ai and Bin Wen and Chaojie Mao and Chen-Wei Xie and Di Chen and Feiwu Yu and Haiming Zhao and Jianxiao Yang and Jianyuan Zeng and Jiayu Wang and Jingfeng Zhang and Jingren Zhou and Jinkai Wang and Jixuan Chen and Kai Zhu and Kang Zhao and Keyu Yan and Lianghua Huang and Mengyang Feng and Ningyi Zhang and Pandeng Li and Ping...

    Pith/arXiv arXiv

  2. [2]

    2412.03603 , archivePrefix=

    Weijie Kong and Qi Tian and Zijian Zhang and Rox Min and Zuozhuo Dai and Jin Zhou and Jiangfeng Xiong and Xin Li and Bo Wu and Jianwei Zhang and Kathrina Wu and Qin Lin and Junkun Yuan and Yanxin Long and Aladdin Wang and Andong Wang and Changlin Li and Duojun Huang and Fang Yang and Hao Tan and Hongmei Wang and Jacob Song and Jiawang Bai and Jianbing Wu ...

    Pith/arXiv arXiv

  3. [3]

    2025 , eprint=

    VSA: Faster Video Diffusion with Trainable Sparse Attention , author=. 2025 , eprint=

    2025

  4. [4]

    Scalable diffusion models with

    Peebles, William and Xie, Saining , booktitle=. Scalable diffusion models with

  5. [5]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =

    Rombach, Robin and Blattmann, Andreas and Lorenz, Dominik and Esser, Patrick and Ommer, Bj\"orn , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =. 2022 , pages =

    2022

  6. [6]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =

    Liu, Ze and Ning, Jia and Cao, Yue and Wei, Yixuan and Zhang, Zheng and Lin, Stephen and Hu, Han , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =. 2022 , pages =

    2022

  7. [7]

    Proceedings of the International Conference on Machine Learning (ICML) , volume=

    Is space-time attention all you need for video understanding? , author=. Proceedings of the International Conference on Machine Learning (ICML) , volume=

  8. [8]

    Advances in neural information processing systems , volume=

    Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and Kaiser,. Advances in neural information processing systems , volume=

  9. [9]

    2502.01776 , archivePrefix=

    Haocheng Xi and Shuo Yang and Yilong Zhao and Chenfeng Xu and Muyang Li and Xiuyu Li and Yujun Lin and Han Cai and Jintao Zhang and Dacheng Li and Jianfei Chen and Ion Stoica and Kurt Keutzer and Song Han , year=. 2502.01776 , archivePrefix=

    arXiv

  10. [10]

    Advances in Neural Information Processing Systems , editor =

    Zhang, Zhenyu and Sheng, Ying and Zhou, Tianyi and Chen, Tianlong and Zheng, Lianmin and Cai, Ruisi and Song, Zhao and Tian, Yuandong and R\'. Advances in Neural Information Processing Systems , editor =

  11. [11]

    URL https://aclanthology.org/2025.acl-long.1126/

    Yuan, Jingyang and Gao, Huazuo and Dai, Damai and Luo, Junyu and Zhao, Liang and Zhang, Zhengyan and Xie, Zhenda and Wei, Yuxing and Wang, Lean and Xiao, Zhiping and Wang, Yuqing and Ruan, Chong and Zhang, Ming and Liang, Wenfeng and Zeng, Wangding. Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention. Proceedings of the 63rd ...

    work page doi:10.18653/v1/2025.acl-long.1126 2025

  12. [12]

    Zhang and Zhilin Yang and Xinyu Zhou and Mingxing Zhang and Jiezhong Qiu , year=

    Enzhe Lu and Zhejun Jiang and Jingyuan Liu and Yulun Du and Tao Jiang and Chao Hong and Shaowei Liu and Weiran He and Enming Yuan and Yuzhi Wang and Zhiqi Huang and Huan Yuan and Suting Xu and Xinran Xu and Guokun Lai and Yanru Chen and Huabin Zheng and Junjie Yan and Jianlin Su and Yuxin Wu and Neo Y. Zhang and Zhilin Yang and Xinyu Zhou and Mingxing Zha...

    Pith/arXiv arXiv

  13. [13]

    Advances in Neural Information Processing Systems , editor =

    Dao, Tri and Fu, Dan and Ermon, Stefano and Rudra, Atri and R\'. Advances in Neural Information Processing Systems , editor =

  14. [14]

    The Twelfth International Conference on Learning Representations , year=

    Efficient Streaming Language Models with Attention Sinks , author=. The Twelfth International Conference on Learning Representations , year=

  15. [15]

    Shuo Yang and Haocheng Xi and Yilong Zhao and Muyang Li and Jintao Zhang and Han Cai and Yujun Lin and Xiuyu Li and Chenfeng Xu and Kelly Peng and Jianfei Chen and Song Han and Kurt Keutzer and Ion Stoica , booktitle=

  16. [16]

    Jintao Zhang and Chendong Xiang and Haofeng Huang and Jia wei and Haocheng Xi and Jun Zhu and Jianfei Chen , booktitle=

  17. [17]

    2506.03065 , archivePrefix=

    Pengtao Chen and Xianfang Zeng and Maosen Zhao and Peng Ye and Mingzhu Shen and Wei Cheng and Gang Yu and Tao Chen , year=. 2506.03065 , archivePrefix=

    arXiv

  18. [18]

    2025 , eprint=

    Training-free and Adaptive Sparse Attention for Efficient Long Video Generation , author=. 2025 , eprint=

    2025

  19. [19]

    2412.05496 , archivePrefix=

    Juechu Dong and Boyuan Feng and Driss Guessous and Yanbo Liang and Horace He , year=. 2412.05496 , archivePrefix=

    Pith/arXiv arXiv

  20. [20]

    2025 , eprint=

    Fast Video Generation with Sliding Tile Attention , author=. 2025 , eprint=

    2025

  21. [21]

    2508.02324 , archivePrefix=

    Chenfei Wu and Jiahao Li and Jingren Zhou and Junyang Lin and Kaiyuan Gao and Kun Yan and Sheng-ming Yin and Shuai Bai and Xiao Xu and Yilei Chen and Yuxiang Chen and Zecheng Tang and Zekai Zhang and Zhengyi Wang and An Yang and Bowen Yu and Chen Cheng and Dayiheng Liu and Deqing Li and Hang Zhang and Hao Meng and Hu Wei and Jingyuan Ni and Kai Chen and K...

    Pith/arXiv arXiv

  22. [22]

    2025 , note=

    Xin Ma and Yaohui Wang and Xinyuan Chen and Gengyun Jia and Ziwei Liu and Yuan-Fang Li and Cunjian Chen and Yu Qiao , journal=. 2025 , note=

    2025

  23. [23]

    2412.20404 , archivePrefix=

    Zangwei Zheng and Xiangyu Peng and Tianji Yang and Chenhui Shen and Shenggui Li and Hongxin Liu and Yukun Zhou and Tianyi Li and Yang You , year=. 2412.20404 , archivePrefix=

    Pith/arXiv arXiv

  24. [24]

    2408.06072 , archivePrefix=

    Zhuoyi Yang and Jiayan Teng and Wendi Zheng and Ming Ding and Shiyu Huang and Jiazheng Xu and Yuanming Yang and Wenyi Hong and Xiaohan Zhang and Guanyu Feng and Da Yin and Yuxuan Zhang and Weihan Wang and Yean Cheng and Bin Xu and Xiaotao Gu and Yuxiao Dong and Jie Tang , year=. 2408.06072 , archivePrefix=

    Pith/arXiv arXiv

  25. [25]

    2402.17177 , archivePrefix=

    Yixin Liu and Kai Zhang and Yuan Li and Zhiling Yan and Chujie Gao and Ruoxi Chen and Zhengqing Yuan and Yue Huang and Hanchi Sun and Jianfeng Gao and Lifang He and Lichao Sun , year=. 2402.17177 , archivePrefix=

    Pith/arXiv arXiv

  26. [26]

    Peters and Arman Cohan , year=

    Iz Beltagy and Matthew E. Peters and Arman Cohan , year=. 2004.05150 , archivePrefix=

    Pith/arXiv arXiv 2004

  27. [27]

    Big Bird: Transformers for Longer Sequences , volume =

    Zaheer, Manzil and Guruganesh, Guru and Dubey, Kumar Avinava and Ainslie, Joshua and Alberti, Chris and Ontanon, Santiago and Pham, Philip and Ravula, Anirudh and Wang, Qifan and Yang, Li and Ahmed, Amr , booktitle =. Big Bird: Transformers for Longer Sequences , volume =

  28. [28]

    Ma, Xinyin and Fang, Gongfan and Wang, Xinchao , booktitle=

  29. [29]

    2506.16054 , archivePrefix=

    Tianchen Zhao and Ke Hong and Xinhao Yang and Xuefeng Xiao and Huixia Li and Feng Ling and Ruiqi Xie and Siqi Chen and Hongyu Zhu and Yichong Zhang and Yu Wang , year=. 2506.16054 , archivePrefix=

    arXiv

  30. [30]

    Xingyang Li and Muyang Li and Tianle Cai and Haocheng Xi and Shuo Yang and Yujun Lin and Lvmin Zhang and Songlin Yang and Jinbo Hu and Kelly Peng and Maneesh Agrawala and Ion Stoica and Kurt Keutzer and Song Han , booktitle=

  31. [31]

    2307.08691 , archivePrefix=

    Tri Dao , year=. 2307.08691 , archivePrefix=

    Pith/arXiv arXiv

  32. [32]

    Shah, Jay and Bikshandi, Ganesh and Zhang, Ying and Thakkar, Vijay and Ramani, Pradeep and Dao, Tri , journal=

  33. [33]

    2502.02770 , archivePrefix=

    Chaofan Lin and Jiaming Tang and Shuo Yang and Hanshuo Wang and Tian Tang and Boyu Tian and Ion Stoica and Song Han and Mingyu Gao , year=. 2502.02770 , archivePrefix=

    arXiv

  34. [34]

    Advances in Neural Information Processing Systems , volume=

    A variational perspective on diffusion-based generative models and score matching , author=. Advances in Neural Information Processing Systems , volume=

  35. [35]

    Companion Proceedings of the ACM Web Conference 2024 , pages=

    Is cosine-similarity of embeddings really about similarity? , author=. Companion Proceedings of the ACM Web Conference 2024 , pages=

    2024

  36. [36]

    Baldi, Pierre and Sadowski, Peter J , booktitle =

  37. [37]

    Elements of information theory , author=

  38. [38]

    2022 , editor =

    Rajbhandari, Samyam and Li, Conglong and Yao, Zhewei and Zhang, Minjia and Aminabadi, Reza Yazdani and Awan, Ammar Ahmad and Rasley, Jeff and He, Yuxiong , booktitle =. 2022 , editor =

    2022

  39. [39]

    SC20: International Conference for High Performance Computing, Networking, Storage and Analysis , pages=

    Sparse gpu kernels for deep learning , author=. SC20: International Conference for High Performance Computing, Networking, Storage and Analysis , pages=. 2020 , organization=

    2020

  40. [40]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =

    Huang, Ziqi and He, Yinan and Yu, Jiashuo and Zhang, Fan and Si, Chenyang and Jiang, Yuming and Zhang, Yuanhan and Wu, Tianxing and Jin, Qingyang and Chanpaisit, Nattapol and Wang, Yaohui and Chen, Xinyuan and Wang, Limin and Lin, Dahua and Qiao, Yu and Liu, Ziwei , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...

    2024

  41. [41]

    2025 , eprint=

    VABench: A Comprehensive Benchmark for Audio-Video Generation , author=. 2025 , eprint=

    2025

  42. [42]

    2210.09461 , archivePrefix=

    Daniel Bolya and Cheng-Yang Fu and Xiaoliang Dai and Peizhao Zhang and Christoph Feichtenhofer and Judy Hoffman , year=. 2210.09461 , archivePrefix=

    Pith/arXiv arXiv