Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

AnchorAttention: Difference-Aware Sparse Attention with Stripe Granularity

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AnchorAttention prunes attention keys by comparing scores to a cheaply computed anchor, reaching 4.6x faster prefill than FlashAttention at 128k context while keeping recall high.

desk verdict Clever anchor-based sparse attention with real speedups on spiky benchmarks, but the hard logit threshold doesn't guarantee recall on diffuse attention; fixable gaps keep it from being a clean accept. read the letter →

arxiv 2505.23520 v1 pith:XNOIKLEB submitted 2025-05-29 cs.LG

Yu Zhang , Dong Guo , Fang Wu , Guoliang Zhu , Dian Ding , Yiming Zhang This is my paper
classification cs.LG
keywords sparseattentionprefillaccelerationstripegranularitydifference-awarethresholdsinklargelanguagemodelsanchorTritonkernel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AnchorAttention is a sparse attention mechanism for the prefill phase of large language models that identifies important keys and values without sorting or computing full attention scores. The paper claims that a near-maximum attention score computed from only the initial tokens and a local window reliably anchors the global score distribution, so any key scoring within a threshold of this anchor can be kept and everything else pruned. Operating at stripe granularity, which retains the column dimension of attention blocks but not the full block, it achieves higher sparsity at the same recall as block-based methods. At 128k context, it reports a 1.44x speedup over FlexPrefill and a 4.6x speedup over FlashAttention while maintaining accuracy close to full attention.

What carries the argument

The load-bearing mechanism is the difference-aware threshold against the anchor. The anchor $x_a$ is computed as the maximum attention score over the initial key block and the local key block (Equation 1). Sparsity is then decided by the mask $\mathrm{mask} = \mathbb{I}(\mathrm{avgpool}(x_a) - \mathrm{avgpool}(Q)K^\top/\sqrt{d} \le \theta)$ (Equation 2), which avoids the sorting needed by top-k and top-cdf strategies. This anchor-guided comparison is the component that makes the method fast; the stripe granularity and discrete KV loading are the components that make it sparse at fine scale. The kernel implementation caches anchor intermediates and processes multiple query blocks per step to preserve parallelism.

What would settle it

Run AnchorAttention on a model known to lack a strong attention sink or on inputs where critical information lies in the middle of the context, and measure recall versus full attention at a fixed sparsity level. If any attention head or substantial fraction of inputs shows a large recall drop, such as recall falling below 80% while block-based methods with the same sparsity stay above 90%, the anchor is not a valid global reference for those cases.

Watch

Extended reading notes

Core claim

The paper claims that attention maps in LLMs have a stable structural commonality: the highest attention scores almost always fall in the initial-token region and the local window, and this stable maximum can be used as an anchor for global sparsity decisions. From this anchor, the method computes a difference-aware threshold: pooled queries are dotted with all keys, and any key whose score is within $\theta$ of the anchor's pooled value is selected, generating a stripe-like sparse mask without sorting operations. This discovery in turn motivates a finer granularity than block sparsity, namely stripe sparsity, where the row dimension of blocks is collapsed to one, and a discrete KV loading scheme that loads only the selected key and value rows. The paper's central claim is that combining these three pieces yields higher sparsity at equal recall and faster prefill than previous sparse attention methods, as demonstrated by a 1.44x speedup over FlexPrefill and a 4.6x speedup over FlashAttention at 128k context.

Load-bearing premise

The assumption that the highest attention score among the first tokens and the local window is a reliable stand-in for the global maximum, so that any position scoring within $\theta$ of this anchor is important and anything farther below can be safely discarded.

Editorial extensions

If this is right

  • At 128k context, AnchorAttention claims a 1.44x speedup over FlexPrefill and a 4.6x speedup over FlashAttention while achieving a higher recall rate than FlexPrefill.
  • Stripe granularity attains higher sparsity than block granularity at the same recall level, with 76.6% sparsity versus 56.3% for block selection on LLaMA-3.1-8B at 128k RULER.
  • The difference-aware strategy achieves recall comparable to top-cdf selection while eliminating the sorting overhead, with similar sparsity heatmaps.
  • The method preserves accuracy close to full attention on LongBench, RULER, and Needle-in-a-Haystack across context lengths up to 128k, without fine-tuning.
  • Ablation shows that the anchor is essential: without it, reaching comparable recall requires far higher sparsity and computation time.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The anchor-based threshold could plausibly extend to decode-phase KV cache eviction, since it provides a parameter-free criterion for which historical keys are likely important, though the paper only evaluates prefill.
  • The reliability of the anchor depends on the attention-sink phenomenon being strong; models trained without explicit sink tokens or with different positional encodings might produce flatter or differently peaked score distributions, which would weaken the threshold's pruning decisions, but this is untested beyond two models.
  • The reported 4.6x speedup is measured with a Triton kernel on a single A100; the same algorithm could behave differently on other hardware or with different block sizes, so kernel-level gains are likely implementation-sensitive even if the algorithmic sparsity gains transfer.
  • Stripe sparsity could be combined with content-based query-aware selection, such as retrieval-style key matching, to catch positions that the anchor misses in inputs where critical information sits mid-context without local peaks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes AnchorAttention, a sparse attention mechanism for LLM prefill that identifies important key-value positions by comparing attention logits against an ``anchor'' score computed from initial tokens and a local window, using a stripe-level (column-wise) granularity instead of block-level sparsity. The method avoids sorting by applying a fixed threshold θ to the difference between the anchor and each pooled query-key score, then loads discrete KV positions in a custom kernel. The authors report higher sparsity rates at the same recall level compared to block-sparse baselines, a 1.44× speedup over FlexPrefill and 4.6× over FlashAttention at 128k context, and accuracy close to full attention on RULER, LongBench, and Needle-in-a-Haystack for LLaMA-3.1-8B and Qwen2.5-7B. The paper includes kernel-level algorithms, an ablation study, and a code release.

Significance. If the central claims hold, the work is a meaningful step for prefill acceleration: it replaces expensive sorting-based selection with a cheap threshold comparison, moves from block to finer stripe granularity, and demonstrates a real kernel implementation with released code. The idea of exploiting the attention-sink phenomenon to derive a global reference score is interesting and differs from purely local heuristics. However, the significance is currently tempered by unresolved technical inconsistencies in the algorithms and evaluation, and by a mathematical property of the threshold criterion that may undermine the claimed sparsity-recall tradeoff for diffuse attention distributions. The paper's own limitations section appropriately notes the narrow model scope and prefill-only focus.

major comments (4)
  1. [Appendix C, Algorithm 2] Algorithm 2 declares the input as ``anchor score Acc'' and computes x_a = avgpool(Acc, bq). In Algorithm 1, however, Acc_i is the accumulated p·V value accumulator (a d-dimensional vector), not the scalar logit maximum. The anchor defined in Eq. (1) is derived from the maximum score, which Algorithm 1 stores in the M buffer. Using avgpool(Acc) as the anchor produces a vector, and subtracting the scalar qk in line 11 is dimensionally inconsistent. The pseudocode should use M (or the max logits) rather than Acc, and the text should state this correspondence explicitly.
  2. [Section 2.1.2, Table 1] The reported sparsity rates do not match the stated Top-K values and block sizes. At a 128k context with N=131072 keys, Block with Top-K=256 would select 32768 keys if Top-K counts 128-element blocks (75% sparsity), or 256 keys if Top-K counts individual keys (99.8% sparsity); neither equals the reported 56.3%. Stripe with Top-K=16384 would give 87.5% sparsity, not 76.6%. Please define exactly what ``Top-K'' counts in each row and recompute the table.
  3. [Section 3.2, Eq. (2)] The difference-aware mask is a hard per-key logit cutoff, not a cumulative-mass criterion. The skeptic's arithmetic is correct: with θ=12 at 128k context, 100k keys at logit x_a−13 are all pruned, yet their combined softmax contribution is about 100000·exp(−13) ≈ 0.226, which is roughly 18% of the attention mass relative to a unit anchor term. The claim of higher sparsity at the same recall is therefore not a consequence of Eq. (2) for attention distributions with broad tails. RULER and Needle-in-a-Haystack are retrieval tasks with spiky attention, so the current benchmarks do not exercise this regime. The authors should either provide evidence on tasks with diffuse attention, or modify the selection criterion to bound cumulative pruned mass.
  4. [Section 4.2, Figure 4 caption] The recall metric is not precisely defined: ``percentage of attention values that are numerically equal between the current sparse attention and the full attention'' is ambiguous. It is unclear whether recall measures matched logits, matched softmax entries, or captured probability mass, and how ties or near-equal values are counted. Since recall is the primary evaluation metric and the central claim is a sparsity-recall tradeoff, a formal definition is necessary. In addition, the experiments appear to be single runs; headline numbers and recall-sparsity curves should include variance estimates across seeds or heads.
minor comments (5)
  1. [Section 2.2.2, Figure 5] The claim that 99% and 90% of maximum attention scores are concentrated in the anchor regions would benefit from a precise definition of ``concentrated'' and from error bars across inputs.
  2. [Section 3.4 and Appendix C] The meaning of the step parameter in Algorithm 2 is unclear: the loop ``for j=2 to j_end'' appears to skip the initial key block, possibly because it is covered by the anchor, but this should be stated explicitly.
  3. [Notation throughout] Equation (3) defines the index set I without showing its dependence on the query block i, although the load_discrete operation is query-block dependent; please clarify the notation.
  4. [Figure 4 and Table 4] The threshold values are inconsistent across figures: Figure 4 uses Difference-Aware(11) while Table 4 ranges over θ=10 to 15; the relationship between these numbers should be explained.
  5. [Abstract and typing] There are several typographical and formatting issues, including ``AnchorAttention'' appearing as a run-together token in the abstract, inconsistent capitalization of ``MInference'' versus ``Minference'', and the affiliation ``Chipltech''.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the anchor is a computed input statistic, theta is an operating point, and the recall/sparsity claims are empirical measurements rather than derived predictions.

full rationale

The paper's claimed derivation chain is self-contained. The anchor in Eq. (1) is computed directly from the query and the initial/local-window keys, and Eq. (2) selects positions by a threshold theta; theta is a user-set hyperparameter, not a parameter fitted to the quantity being predicted. The recall and sparsity results are measured post hoc (Fig. 6a, Table 4), and the 'Without Anchor' ablation is an empirical control rather than a circular validation. The paper invokes no load-bearing uniqueness theorem from its own authors and contains no significant self-citation: the attention-sink observation is supported by the in-paper measurement in Fig. 5, not by an imported same-author claim. The headline 'higher sparsity at the same recall' is a benchmark comparison, not a derivation from the equations, so it cannot reduce to the method's inputs by construction. Concerns about the absolute-logit cutoff's behavior under diffuse attention (a tail of keys just below theta can carry substantial softmax mass) or about Algorithm 2 reusing Acc rather than M are correctness/implementation risks, not circularity.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The method is an empirical heuristic built on known attention patterns. It has two explicit hyperparameters (theta and step) and three domain assumptions about attention structure. No new physical or architectural entities are introduced.

free parameters (2)
  • theta (threshold) = 12
    Controls how close a key's score must be to the anchor to be kept; default 12 in all experiments after ablation sweep in Table 4.
  • step (query-block group size) = 16
    Groups step query blocks for parallel identification; if any block in the group contains a selected key, all blocks in the group are marked active. Set to 16 in experiments.
assumptions (3)
  • domain assumption Attention scores have consistent peaks at initial token and local window positions (attention sink), so the maximum over these regions approximates the global maximum.
    Section 2.2.2 and Equation 1; this empirical pattern is cited to prior work, but the method depends on it for the anchor.
  • domain assumption Column-wise correlation: a small subset of keys receives high attention across consecutive queries, so pooling queries via block-average preserves the important positions for each query in the block.
    Section 3.2, Equation 2; enables the cheap global scan using avgpool(Q).
  • domain assumption Thresholding scores relative to the anchor identifies exactly the positions needed for good recall; positions with score below anchor minus theta can be discarded safely.
    Equation 2; the core selection heuristic, validated only empirically on two models.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AnchorAttention: Difference-Aware Sparse Attention with Stripe Granularity." pith.science (2026). https://pith.science/paper/XNOIKLEB

@misc{pith2026250523520,
  author       = {Pith},
  title        = {Pith review of: AnchorAttention: Difference-Aware Sparse Attention with Stripe Granularity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XNOIKLEB}},
  note         = {Machine review of arXiv:2505.23520}
}
abstract

Large Language Models (LLMs) with extended context lengths face significant computational challenges during the pre-filling phase, primarily due to the quadratic complexity of self-attention. Existing methods typically employ dynamic pattern matching and block-sparse low-level implementations. However, their reliance on local information for pattern identification fails to capture global contexts, and the coarse granularity of blocks leads to persistent internal sparsity, resulting in suboptimal accuracy and efficiency. To address these limitations, we propose \textbf{AnchorAttention}, a difference-aware, dynamic sparse attention mechanism that efficiently identifies critical attention regions at a finer stripe granularity while adapting to global contextual information, achieving superior speed and accuracy. AnchorAttention comprises three key components: (1) \textbf{Pattern-based Anchor Computation}, leveraging the commonalities present across all inputs to rapidly compute a set of near-maximum scores as the anchor; (2) \textbf{Difference-aware Stripe Sparsity Identification}, performing difference-aware comparisons with the anchor to quickly obtain discrete coordinates of significant regions in a stripe-like sparsity pattern; (3) \textbf{Fine-grained Sparse Computation}, replacing the traditional contiguous KV block loading approach with simultaneous discrete KV position loading to maximize sparsity rates while preserving full hardware computational potential. With its finer-grained sparsity strategy, \textbf{AnchorAttention} achieves higher sparsity rates at the same recall level, significantly reducing computation time. Compared to previous state-of-the-art methods, at a text length of 128k, it achieves a speedup of 1.44$\times$ while maintaining higher recall rates.

Figures

Figures reproduced from arXiv: 2505.23520 by the authors.

Figure 1
Figure 1. (a) Block-sparse pattern, with yellow regions [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Acceleration of attention computation com [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. (a) Heatmaps vary significantly across differ [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Recall heatmaps of Sparsity Strategies using LLaMA-3.1-8B on the 128k Ruler( [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: The distribution of maximum attention scores [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: Performance metrics for recall, sparsity, and efficiency across different methods. Figures (a), (b), and (c) [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Comparison of attention patterns on Needle [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Sparsity heatmaps under different sparsity strategies. The recall heatmap corresponds to Figure [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Recall heatmaps under different sparsity identification strategies. [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Sparsity heatmaps for different sparsity strategies. [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Lag-Relative Sparse Attention In Long Context Training

    cs.CL 2025-06 conditional novelty 5.0 of 10

    Training a language model with LagKV's static sparse attention mask improves its robustness to KV-cache compression at 2x and 4x rates, while claims of overall average gains are not consistently supported.

Reference graph

Works this paper leans on

29 extracted references · 3 canonical work pages · cited by 1 Pith paper

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2024. http://arxiv.org/abs/2308.14508 Longbench: A bilingual, multitask benchmark for long context understanding

    arXiv 2024

  4. [4]

    Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever. 2019. http://arxiv.org/abs/1904.10509 Generating long sequences with sparse transformers

    arXiv 2019

  5. [5]

    Fu, Stefano Ermon, Atri Rudra, and Christopher Ré

    Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022. http://arxiv.org/abs/2205.14135 Flashattention: Fast and memory-efficient exact attention with io-awareness

    arXiv 2022

  6. [6]

    Yu Fu, Zefan Cai, Abedelkadir Asi, Wayne Xiong, Yue Dong, and Wen Xiao. 2024. http://arxiv.org/abs/2410.19258 Not all heads matter: A head-level kv cache compression method with integrated retrieval and reasoning

    arXiv 2024

  7. [7]

    Connor Holmes, Masahiro Tanaka, Michael Wyatt, Ammar Ahmad Awan, Jeff Rasley, Samyam Rajbhandari, Reza Yazdani Aminabadi, Heyang Qin, Arash Bakhtiari, Lev Kurilenko, and Yuxiong He. 2024. http://arxiv.org/abs/2401.08671 Deepspeed-fastgen: High-throughput text generation for llms via mii and deepspeed-inference

    arXiv 2024

  8. [8]

    Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. 2024. http://arxiv.org/abs/2404.06654 Ruler: What's the real context size of your long-context language models?

    arXiv 2024

Show all 29 references
  1. [9]

    Abdi, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu

    Huiqiang Jiang, Yucheng Li, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Zhenhua Han, Amir H. Abdi, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2024. http://arxiv.org/abs/2407.02490 Minference 1.0: Accelerating pre-filling for long-context llms via dynamic...

    2024 arXiv

  2. [10]

    Jean Kaddour, Joshua Harris, Maximilian Mozes, Herbie Bradley, Roberta Raileanu, and Robert McHardy. 2023. http://arxiv.org/abs/2307.10169 Challenges and applications of large language models

    2023 arXiv

  3. [11]

    Greg Kamradt. 2023. Llmtest needle in a haystack - pressure testing llms. https://github.com/gkamradt/LLMTest_NeedleInAHaystack. Accessed: [Insert Date]

    2023

  4. [12]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. http://arxiv.org/abs/2309.06180 Efficient memory management for large language model serving with pagedattention

    2023 arXiv

  5. [13]

    Xunhao Lai, Jianqiao Lu, Yao Luo, Yiyuan Ma, and Xun Zhou. 2025. http://arxiv.org/abs/2502.20766 Flexprefill: A context-aware sparse attention mechanism for efficient long-sequence inference

    2025 arXiv

  6. [14]

    Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen. 2024. https://openreview.net/forum?id=poE54GOq2l Snap KV : LLM knows what you are looking for before generation . In The Thirty-eighth Annual Confer...

    2024

  7. [15]

    Di Liu, Meng Chen, Baotong Lu, Huiqiang Jiang, Zhenhua Han, Qianxi Zhang, Qi Chen, Chengruidong Zhang, Bailu Ding, Kai Zhang, Chen Chen, Fan Yang, Yuqing Yang, and Lili Qiu. 2024. http://arxiv.org/abs/2409.10516 Retrievalattention: Accelerating long-context llm inference via v...

    2024 arXiv

  8. [16]

    Hao Liu, Matei Zaharia, and Pieter Abbeel. 2023. http://arxiv.org/abs/2310.01889 Ring attention with blockwise transformers for near-infinite context

    2023 arXiv

  9. [17]

    Maxim Milakov and Natalia Gimelshein. 2018. http://arxiv.org/abs/1805.02867 Online normalizer calculation for softmax

    2018 arXiv

  10. [18]

    Libo Qin, Qiguang Chen, Xiachong Feng, Yang Wu, Yongheng Zhang, Yinghui Li, Min Li, Wanxiang Che, and Philip S. Yu. 2024. http://arxiv.org/abs/2405.12819 Large language models meet nlp: A survey

    2024 arXiv

  11. [19]

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, ...

    2025 arXiv

  12. [20]

    Jiaming Tang, Yilong Zhao, Kan Zhu, Guangxuan Xiao, Baris Kasikci, and Song Han. 2024. http://arxiv.org/abs/2406.10774 Quest: Query-aware sparsity for efficient long-context llm inference

    2024 arXiv

  13. [21]

    Philippe Tillet, H. T. Kung, and David Cox. 2019. https://doi.org/10.1145/3315508.3329973 Triton: an intermediate language and compiler for tiled neural network computations . In Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Lang...

    2019

  14. [22]

    Hugo Touvron and et al. 2023. http://arxiv.org/abs/2302.13971 Llama: Open and efficient foundation language models

    2023 arXiv

  15. [23]

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2024. http://arxiv.org/abs/2309.17453 Efficient streaming language models with attention sinks

    2024 arXiv

  16. [24]

    Ruyi Xu, Guangxuan Xiao, Haofeng Huang, Junxian Guo, and Song Han. 2025. http://arxiv.org/abs/2503.16428 Xattention: Block sparse attention with antidiagonal scoring

    2025 arXiv

  17. [25]

    Dongjie Yang, XiaoDong Han, Yan Gao, Yao Hu, Shilin Zhang, and Hai Zhao. 2024. http://arxiv.org/abs/2405.12532 Pyramidinfer: Pyramid kv cache compression for high-throughput llm inference

    2024 arXiv

  18. [26]

    Shang Yang, Junxian Guo, Haotian Tang, Qinghao Hu, Guangxuan Xiao, Jiaming Tang, Yujun Lin, Zhijian Liu, Yao Lu, and Song Han. 2025. http://arxiv.org/abs/2502.14866 Lserve: Efficient long-sequence llm serving with unified sparse attention

    2025 arXiv

  19. [27]

    Jintao Zhang, Chendong Xiang, Haofeng Huang, Jia Wei, Haocheng Xi, Jun Zhu, and Jianfei Chen. 2025. http://arxiv.org/abs/2502.18137 Spargeattn: Accurate sparse attention accelerating any model inference

    2025

  20. [28]

    Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Re, Clark Barrett, Zhangyang Wang, and Beidi Chen. 2023. https://openreview.net/forum?id=RkRrPp7GKO H2o: Heavy-hitter oracle for efficient generative inference...

    2023

  21. [29]

    Melton, Mingquan Lin, and Rui Zhang

    Shuang Zhou, Zidu Xu, Mian Zhang, Chunpu Xu, Yawen Guo, Zaifu Zhan, Sirui Ding, Jiashuo Wang, Kaishuai Xu, Yi Fang, Liqiao Xia, Jeremy Yeung, Daochen Zha, Genevieve B. Melton, Mingquan Lin, and Rui Zhang. 2024. http://arxiv.org/abs/2409.00097 Large language models for disease ...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.