Pith. sign in

REVIEW 2 major objections 6 minor 35 references

Nearby queries pick almost the same top-k tokens, so one shared prefix scan can replace many and keep sparse-attention accuracy.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 11:00 UTC pith:2O4FMLYZ

load-bearing objection Solid production systems paper: amortize the DSA indexer along the query axis with a proxy scan; Refine’s parity claim is credible but not fully instrumented. the 2 major comments →

arxiv 2607.24593 v1 pith:2O4FMLYZ submitted 2026-07-27 cs.CL

PIVOT: Efficient Query-Group Indexing for Token-Level Sparse Attention

Hong Liu , Yuan Cheng , Lin Niu , Yi Su , Yufei Xue , Anmin Liu , Guanghua Yu , Jianchen Zhu This is my paper
classification cs.CL
keywords sparse attentiontoken-level indexingquery-group sharinglong-context inferenceDSAmulti-token predictionproxy query
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Token-level sparse attention makes the main attention cheap, but the indexer that chooses each query’s top-k tokens still scores the full prefix and becomes the bottleneck at long context. This paper argues that those per-query scans are largely wasted: neighboring queries choose highly overlapping tokens, a group’s combined top-k stays near k rather than growing like g·k, and indexer scores are long-tailed. PIVOT turns that redundancy into a training-free drop-in indexer: it averages a group into one proxy query, runs one full-prefix scan to recall a small candidate set, then either reuses the proxy top-k or re-scores the candidates per query. The same recipe covers prefill (fixed consecutive groups) and decode (one multi-token-prediction step as a group). On production DSA models it matches dense-indexer accuracy while cutting indexer cost by up to about 4× and end-to-end latency by up to about 1.6× at long context.

Core claim

The DSA indexer’s O(L²) per-query full-prefix cost is largely redundant along the query axis. Sharing one proxy scan over a small group of nearby queries, then optionally re-ranking a candidate set of size about 2k, recovers dense-indexer top-k selections closely enough that accuracy on LongBench and RULER stays on par while the indexer is accelerated several-fold.

What carries the argument

PIVOT (Proxy Indexing Via One full-prefix Traversal): mean-pool a query group into one proxy, run a single full-prefix indexer scan, form a shared top-c candidate set (default c=2k), then either assign the proxy top-k to everyone (Reuse) or re-score the candidates with each query’s own indexer and take an individual top-k (Refine).

Load-bearing premise

Nearby queries must keep selecting mostly the same tokens, so a fixed small candidate budget still covers almost everything each member would have picked on its own.

What would settle it

On long RULER contexts, grow the group size well past 4 while holding the candidate budget at 2k: if accuracy falls far below the dense DSA indexer, the shared-candidate premise has failed.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Indexer cost can be cut along a fourth axis—queries—orthogonal to token, head, and layer accelerations, so the methods can stack.
  • Prefill and decode can share one algorithm: fixed consecutive groups in prefill and MTP draft-plus-current tokens in decode.
  • PIVOT-Refine is the accuracy-preserving default; PIVOT-Reuse trades a bit of long-context recall for lower cost.
  • At short context the method can fall back to dense scoring so it never runs slower than the baseline indexer.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Any sparse selector with strong local query agreement—not only DSA—could amortize scoring the same way.
  • If multi-token prediction widths grow, decode group size grows for free and the same proxy scan may buy more amortization until the union outruns the candidate budget.
  • The long-tailed score shape suggests future work could shrink c adaptively per layer or group instead of fixing c=2k.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper addresses the observation that DeepSeek Sparse Attention (DSA) shifts, rather than eliminates, the long-context bottleneck: the lightning indexer must score every prefix token per query, an O(L²) per-layer cost. The authors observe (Fig. 2, observations O1–O3) that nearby queries select highly overlapping top-k sets, that a group's union stays near k rather than g·k, and that indexer scores are long-tailed. They propose PIVOT, a training-free drop-in replacement that aggregates a group of queries (fixed-size contiguous groups in prefill; one MTP step's tokens in decode) into a single mean-pooled proxy query, runs one shared full-prefix scan, and then either reuses the proxy top-k (PIVOT-Reuse) or re-scores a top-c candidate set per query with the exact indexer (PIVOT-Refine). On DeepSeek-V3.2 and GLM-5.1 over LongBench and RULER, Refine tracks the dense DSA indexer closely while the indexer kernel speeds up by up to ~4.8× (Reuse) and ~3.9× (Refine) at 256K, with up to ~1.6× end-to-end at 128K; Refine is slower than dense below ~16K and falls back to dense scanning there via a guardrail.

Significance. If the results hold, this is a useful and timely contribution. The DSA indexer is documented as the dominant cost at long context (81% of prefill e2e latency at 200K, per the cited IndexCache measurement), and PIVOT opens a genuinely new efficiency axis (query-axis amortization) that is orthogonal to, and composable with, token/head/layer-axis accelerators — demonstrated by the lossless +IC composition. The method is training-free, drop-in (T_t is the sole preserved interface), rides on MTP for free in decode, and is validated on two production-scale models (DeepSeek-V3.2, GLM-5.1) with kernel-level and end-to-end measurements and a latency breakdown (App. D). The hyperparameters (g=4, c=2k, mean pooling) are chosen from observations and ablated rather than smuggled in as fitted constants, and the g/c/aggregation/phase ablations behave as the mechanism predicts, which lends internal consistency. There is no circularity: accuracy claims are evaluated against external benchmarks and a dense baseline.

major comments (2)
  1. [§Method, Eqs. (4)–(7); §Observations O1–O2] Eqs. (4)–(7): the load-bearing premise of Refine — that the proxy's top-c candidate set C covers each member's true top-k (up to the local window W_t) — is never measured directly. O1/O2 (Fig. 2a–b) quantify the overlap and union of the *dense per-query* selections, not whether the *proxy* scan ranks those tokens within its top-c. A union of ~1.3–1.5k at g=4 does not by itself imply proxy top-4096 coverage, because the proxy score in Eq. (5) is computed from queries/gates averaged before the ReLU: w̄_j·ReLU(q̄_j·k_s) ≠ (1/g)Σ_t w_t,j·ReLU(q_t,j·k_s). If a key aligns positively with one member and negatively with another, the averaged dot product can fall below zero and zero out that key's proxy score — and the keys most exposed to this cancellation are exactly the member-specific margin tokens that C exists to retain. The downstream results partially mitigate this worry (Refine tracks de
  2. [§Experiments, Table 1 (RULER)] The headline accuracy claim ('matches the dense DSA indexer') rests on RULER cells computed from 20 samples per task–length pair, so the per-length standard error is on the order of 1–2 points and the key comparisons (e.g., Refine 90.40 vs DSA 90.45 at 128K on DeepSeek-V3.2; 91.79 vs 92.14 on GLM-5.1) are well within noise. The paper is candid that LongBench 'does not stress the indexer,' which leaves RULER at 64K–128K as essentially the only discriminating evidence — and there the claim is statistically underpowered. Please report per-cell confidence intervals or increase the sample count at 64K/128K, and temper 'matches' language accordingly. This matters more here than usual because the accuracy claim is also the sole empirical check on the candidate-coverage premise raised above.
minor comments (6)
  1. [Abstract; Fig. 4] Abstract/Conclusion vs. Fig. 4: the abstract and conclusion state indexer speedup 'up to 4×', but Fig. 4(a) shows up to 4.77× (Reuse, prefill) and 4.28× at 128K. The numbers should be reconciled.
  2. [Fig. 3] Fig. 3 uses emoji glyphs (🥹/😊) to annotate the DSA and PIVOT pipelines; this is out of place in a technical figure and will not render in some PDF toolchains.
  3. [§Method; App. B] Eq. (5) causal cutoff: the proxy scans only s ≤ t where t is the first group position; later group members' intervening tokens are only recovered via the local window W (App. B). This coupling between g and W deserves one sentence in the main text, since it constrains the choice of W ≥ g in decode.
  4. [Table 4] Table 4 (budget ablation): at 128K, c=8192 (90.55) barely improves on c=4096 (90.40), but c=6144 (89.85) is *worse* than both — a non-monotonicity within noise that should be acknowledged as such, especially given the 20-sample cells.
  5. [Alg. 1; §Efficiency Results] The guardrail (dense fallback for t0 < c, Alg. 1 line 7–9) is mentioned only in passing in §Efficiency Results; since it is what makes Refine never-slower-than-DSA in deployment, it deserves to be stated as an explicit design element in §Method.
  6. [Throughout] Typesetting: the arXiv text layer has pervasive missing inter-word spaces ('indexerthatfeedsit', 'thekmost relevant'), presumably a PDF extraction artifact, but worth checking the compiled source.

Circularity Check

0 steps flagged

No significant circularity: empirical systems method evaluated against external benchmarks and a dense baseline.

full rationale

PIVOT is a training-free systems redesign of the DSA indexer. Its load-bearing claims are measured end-task accuracy (LongBench, RULER) and wall-clock speedup versus the dense DSA indexer on DeepSeek-V3.2 and GLM-5.1—not first-principles predictions derived from fitted constants. Observations O1–O3 motivate design choices (group sharing, c≈2k, mean proxy) and are supported by ablations (g, c, aggregation, phase); those choices are not smuggled back as predicted outcomes. Hyperparameters are stated defaults with sensitivity tables, not identities that force the accuracy claim. Citations to DSA, MTP, HISA, IndexCache, and MISA supply baselines and orthogonal axes; none is a self-authored uniqueness theorem that forbids alternatives or closes the accuracy argument by definition. Downstream Sparse MLA and the Tt interface are left unchanged by construction of the drop-in design, which is an engineering invariant, not a circular derivation of the empirical match. Correctness risks (unmeasured proxy top-c recall of members’ true top-k; mean-before-ReLU cancellation) are real evaluation gaps but are not circular reductions of outputs to inputs.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 2 invented entities

Load-bearing content is empirical systems design plus three observational regularities about DSA indexer behavior, not a formal derivation. Free parameters are the operating knobs (g, c, W, pooling). Axioms are standard attention/complexity facts plus domain assumptions that DSA/MTP interfaces and locality properties hold on the evaluated models. Invented entities are named method components, not physical postulates.

free parameters (4)
  • group size g = 4 (default)
    Controls amortization vs. top-k union growth; default g=4 in prefill and g=d+1=4 from MTP in decode, chosen from locality observations and ablated.
  • candidate budget c = 4096 with k=2048
    Size of shared shortlist for Refine; set to 2k from union statistics (O2) and ablated around 3072–8192.
  • local window W = W=4
    Per-query recent tokens force-included with the candidate set; appendix sets W=4 (with wg=W+g-1 reservation in prefill).
  • proxy aggregation = per-head mean pooling
    How group members form the proxy query/gates; mean pooling selected over first/last after ablation.
axioms (4)
  • domain assumption DSA indexer scores are I_t,s = sum_j w_t,j ReLU(q_t,j · k_s), and Sparse MLA consumes only the resulting top-k index set T_t.
    Preliminary section; PIVOT preserves this interface and does not modify Sparse MLA or the KV cache.
  • domain assumption Nearby queries select highly overlapping top-k sets (O1), group unions stay near k (O2), and indexer scores are long-tailed (O3).
    Motivating observations on DeepSeek-V3.2 RULER-QA 128K; accuracy of small c and proxy sharing rests on these holding at deployment lengths.
  • domain assumption Prefill can partition simultaneous query positions into fixed contiguous groups; decode can treat one MTP step’s target+draft tokens as a group.
    Grouping section; decode path assumes production MTP (d=3) rather than single-token decode.
  • standard math Standard asymptotic cost model: dense per-query indexing costs O(gL) per group vs O(L) or O(L+gc) for PIVOT variants.
    Used throughout Method/Efficiency to explain why speedup grows with context length.
invented entities (2)
  • proxy query (mean-pooled indexing queries and gates over a group) no independent evidence
    purpose: Represent a query group with one full-prefix indexer scan.
    Core algorithmic construct; not an external physical entity, but a new computational object introduced by the paper.
  • PIVOT-Reuse and PIVOT-Refine selection variants no independent evidence
    purpose: Trade shared top-k speed against per-query re-score fidelity on a candidate set C.
    Named method variants defining the paper’s operating range; validated only via the paper’s benchmarks.

pith-pipeline@v1.2.0-grok45-kimik3 · 25935 in / 3509 out tokens · 62233 ms · 2026-07-31T11:00:00.877313+00:00 · methodology

0 comments
read the original abstract

Token-level sparse attention, as implemented by DeepSeek Sparse Attention (DSA) in production systems, makes the downstream attention efficient but shifts the bottleneck to the indexer that feeds it. To select the top-k tokens for each query, the indexer must still score every preceding token, incurring a cost of O(L^2) per layer for a sequence of length L. We observe that this per-query scan is largely redundant: nearby queries select highly overlapping top-k tokens, and the indexer scores are long-tailed along the key axis. We exploit these properties in PIVOT, Proxy Indexing Via One full-prefix Traversal, a training-free, drop-in replacement for the DSA indexer that shares one prefix scan across a group of nearby queries. PIVOT aggregates a group into a single proxy query, performs one shared full-prefix scan to obtain a candidate set, and then selects a top-k for each query from that set. Two variants trade speed for fidelity: PIVOT-Reuse shares the proxy top-k across the group for maximum speed, whereas PIVOT-Refine re-scores the candidate set with the indexer of each query and then selects an individual top-k, matching the dense indexer at a small additional cost. A single algorithm covers both inference phases, differing only in how groups are formed: fixed-size groups of consecutive queries in prefill, and the queries decoded together in one multi-token prediction (MTP) step in decode. On DeepSeek-V3.2 and GLM-5.1 across LongBench and RULER, PIVOT matches the accuracy of the dense DSA indexer while accelerating it by up to 4x and reducing end-to-end latency by up to 1.6x at long context.

Figures

Figures reproduced from arXiv: 2607.24593 by Anmin Liu, Guanghua Yu, Hong Liu, Jianchen Zhu, Lin Niu, Yi Su, Yuan Cheng, Yufei Xue.

Figure 1
Figure 1. Figure 1: PIVOT- faster long-context indexing passed to a downstream Sparse Multi-head Latent Attention (Sparse MLA) operator. This design supports production￾scale models such as DeepSeek-V3.2 (DeepSeek-AI 2025) and GLM-5.1 (Zeng et al. 2026). This efficiency, however, shifts the cost rather than elim￾inating it. Although the downstream Sparse MLA operator becomes cheap, the indexer that feeds it remains dense: to … view at source ↗
Figure 2
Figure 2. Figure 2: Observational study on the redundancy and sparsity of indexer top-k selection. DeepSeek-V3.2, RULER-QA 128K, deep queries (ke >20K), budget k= 2048. (a) Local. The shared-key fraction |Si∩Sj |/k forms a bright diagonal: adjacent queries share ≈ 0.8–0.9 of their top-k from shallow to deep layers(L3/L30/L58), and 0.6–0.8 even across a full g = 4 group. (b) Group-shareable. A group’s top-k union stays near th… view at source ↗
Figure 3
Figure 3. Figure 3: PIVOT amortizes the DSA indexer’s per-query prefix scans across a query group. Left (DSA): To select its top-k, every query q1, . . . , qg independently scores all prefix keys, so a group of g queries incurs g full-prefix scans. Right (PIVOT): The queries of a group (fixed-size in prefill; the current token together with its MTP draft tokens in decode) are aggregated into a single proxy query qproxy which … view at source ↗
Figure 4
Figure 4. Figure 4: Indexer speedup over the DSA baseline on DeepSeek-V3.2, from 4K to 256K. (a) Prefill and (b) decode report the indexer-kernel speedup, while (c) reports the end-to-end speedup, with the DSA baseline normalized to 1×. PIVOT-Refine falls back to DSA at 4K (dashed, 1×). where the causal cutoff at the first position of the group, t, keeps the scan valid for every member of G. This single scan, at a cost of O(L… view at source ↗
Figure 5
Figure 5. Figure 5: Per-layer view of query locality in indexer top-k selection (appendix to [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Indexer kernel latency breakdown and speedup over the DSA baseline on DeepSeek-V3.2, from 16K to 256K. (a) Prefill and (b) decode. Stacked bars show the latency of individual kernel stages normalized to DSA (= 1), with absolute latency annotated above each bar. Lines report the corresponding speedup over DSA on the right axis. PIVOT-Reuse consistently achieves around 4× speedup, while PIVOT-Refine delivers… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 17 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2512.02556 , year =

    DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models , author =. arXiv preprint arXiv:2512.02556 , year =

    Pith/arXiv arXiv

  2. [2]

    arXiv preprint arXiv:2603.28458 , year =

    HISA: Efficient Hierarchical Indexing for Fine-Grained Sparse Attention , author =. arXiv preprint arXiv:2603.28458 , year =

    Pith/arXiv arXiv

  3. [3]

    arXiv preprint , year =

    MISA: Mixture of Indexer Sparse Attention for Long-Context LLM Inference , author =. arXiv preprint , year =

  4. [4]

    arXiv preprint arXiv:2603.12201 , year =

    IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse , author =. arXiv preprint arXiv:2603.12201 , year =

    arXiv

  5. [5]

    Advances in Neural Information Processing Systems , volume=

    Minference 1.0: Accelerating pre-filling for long-context llms via dynamic sparse attention , author=. Advances in Neural Information Processing Systems , volume=

  6. [6]

    arXiv preprint arXiv:2502.20766 , year=

    Flexprefill: A context-aware sparse attention mechanism for efficient long-sequence inference , author=. arXiv preprint arXiv:2502.20766 , year=

    Pith/arXiv arXiv

  7. [7]

    arXiv preprint arXiv:2503.16428 , year=

    Xattention: Block sparse attention with antidiagonal scoring , author=. arXiv preprint arXiv:2503.16428 , year=

    Pith/arXiv arXiv

  8. [8]

    URL https://arxiv

    Moba: Mixture of block attention for long-context llms, 2025 , author=. URL https://arxiv. org/abs/2502.13189 , year=

    Pith/arXiv arXiv 2025

  9. [9]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Native sparse attention: Hardware-aligned and natively trainable sparse attention , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  10. [10]

    arXiv preprint arXiv:2603.06274 , year=

    Stem: Rethinking Causal Information Flow in Sparse Attention , author=. arXiv preprint arXiv:2603.06274 , year=

    arXiv

  11. [11]

    arXiv preprint arXiv:2407.14057 , year =

    LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference , author =. arXiv preprint arXiv:2407.14057 , year =

    Pith/arXiv arXiv

  12. [12]

    Advances in Neural Information Processing Systems , volume=

    H2o: Heavy-hitter oracle for efficient generative inference of large language models , author=. Advances in Neural Information Processing Systems , volume=

  13. [13]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    Transformers are multi-state rnns , author=. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    2024

  14. [14]

    2024 , eprint=

    SnapKV: LLM Knows What You are Looking for Before Generation , author=. 2024 , eprint=

    2024

  15. [15]

    arXiv preprint arXiv:2602.15763 , year=

    Glm-5: from vibe coding to agentic engineering , author=. arXiv preprint arXiv:2602.15763 , year=

    Pith/arXiv arXiv

  16. [16]

    Proceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: Long papers) , pages=

    Longbench: A bilingual, multitask benchmark for long context understanding , author=. Proceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: Long papers) , pages=

  17. [17]

    Annual Meeting of the Association for Computational Linguistics (ACL) , year =

    LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-Context Multitasks , author =. Annual Meeting of the Association for Computational Linguistics (ACL) , year =

  18. [18]

    arXiv preprint arXiv:2404.06654 , year =

    RULER: What's the Real Context Size of Your Long-Context Language Models? , author =. arXiv preprint arXiv:2404.06654 , year =

    Pith/arXiv arXiv

  19. [19]

    Proceedings of the 29th Symposium on Operating Systems Principles (SOSP) , year =

    Efficient Memory Management for Large Language Model Serving with PagedAttention , author =. Proceedings of the 29th Symposium on Operating Systems Principles (SOSP) , year =

  20. [20]

    A Framework for Few-shot Language Model Evaluation , author =

  21. [21]

    int8 (): 8-bit matrix multiplication for transformers at scale , author=

    Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale , author=. Advances in Neural Information Processing Systems , volume=

  22. [22]

    arXiv preprint arXiv:2307.09288 , year=

    Llama 2: Open foundation and fine-tuned chat models , author=. arXiv preprint arXiv:2307.09288 , year=

    Pith/arXiv arXiv

  23. [23]

    arXiv preprint arXiv:2505.09388 , year=

    Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=

    Pith/arXiv arXiv

  24. [24]

    International conference on algorithmic learning theory , pages=

    On the computational complexity of self-attention , author=. International conference on algorithmic learning theory , pages=. 2023 , organization=

    2023

  25. [25]

    arXiv preprint arXiv:2507.20534 , year=

    Kimi k2: Open agentic intelligence , author=. arXiv preprint arXiv:2507.20534 , year=

    Pith/arXiv arXiv

  26. [26]

    arXiv preprint arXiv:2509.24663 , year=

    Infllm-v2: Dense-sparse switchable attention for seamless short-to-long adaptation , author=. arXiv preprint arXiv:2509.24663 , year=

    arXiv

  27. [27]

    arXiv preprint arXiv:2602.03560 , year=

    HySparse: A Hybrid Sparse Attention Architecture with Oracle Token Selection and KV Cache Sharing , author=. arXiv preprint arXiv:2602.03560 , year=

    arXiv

  28. [28]

    International Conference on Learning Representations , volume=

    Duoattention: Efficient long-context llm inference with retrieval and streaming heads , author=. International Conference on Learning Representations , volume=

  29. [29]

    arXiv preprint arXiv:2509.24745 , year=

    ProxyAttn: Guided Sparse Attention via Representative Heads , author=. arXiv preprint arXiv:2509.24745 , year=

    arXiv

  30. [30]

    arXiv preprint arXiv:2410.13276 , year=

    Seerattention: Learning intrinsic sparse attention in your llms , author=. arXiv preprint arXiv:2410.13276 , year=

    Pith/arXiv arXiv

  31. [31]

    arXiv preprint arXiv:2606.04511 , year=

    SparDA: Sparse Decoupled Attention for Efficient Long-Context LLM Inference , author=. arXiv preprint arXiv:2606.04511 , year=

    Pith/arXiv arXiv

  32. [32]

    arXiv preprint arXiv:2506.08889 , year=

    Seerattention-r: Sparse attention adaptation for long reasoning , author=. arXiv preprint arXiv:2506.08889 , year=

    Pith/arXiv arXiv

  33. [33]

    arXiv preprint arXiv:2605.16928 , year=

    Full Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Steps , author=. arXiv preprint arXiv:2605.16928 , year=

    Pith/arXiv arXiv

  34. [34]

    arXiv preprint arXiv:2505.07608 , year=

    MiMo: Unlocking the Reasoning Potential of Language Model--From Pretraining to Posttraining , author=. arXiv preprint arXiv:2505.07608 , year=

    Pith/arXiv arXiv

  35. [35]

    arXiv preprint arXiv:2406.10774 , year=

    Quest: Query-aware sparsity for efficient long-context llm inference , author=. arXiv preprint arXiv:2406.10774 , year=

    Pith/arXiv arXiv