REVIEW 3 major objections 3 minor 29 references
Spend Bits Where Queries Look: KV Cache Vector Quantization with Attention-Preserving Transforms
T0 review · 3 major / 3 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read A non-orthogonal key transform lets two-bit KV caches preserve attention, and volume-equalizing vector quantization makes the fixed-width layout rate-optimal.
desk verdict A serious KV cache quantization paper whose non-orthogonal transform is genuinely new, but whose 'optimality' claims are asymptotic and the deployed rotation is a heuristic; the empirical results are strong and honestly reported. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the attention-preserving key transform $R_K = M_q^{1/2}E$ (with $M_q = Q^\top Q$ the query second moment and $E$ the eigenbasis of $M_q^{1/2}\widehat S_k M_q^{1/2}$), together with the generalized Parseval relation it satisfies. Because $R_K R_K^\top = M_q$, quantization error in the transform domain equals the query-weighted key error that controls attention logits; this is what lets a generic MSE-optimal vector quantizer serve as the attention-aware quantizer. The second mechanism is the volume-equalizing partition: coefficients are sorted by log-variance and dealt cyclically into groups so each group's volume is approximately equal, making equal-size codebooks reach the variable-rate bound of Theorem 2 while keeping a fixed-width cache layout.
What would settle it
On a fixed real cache, quantize keys with $R_K = M_q^{1/2}E$ and with the best orthogonal calibrated rotation at the same 2-bit vector quantization, then compute the exact attention-product error $\|QK^\top - Q\hat K^\top\|_F$; if an orthogonal transform achieves a smaller error, the claimed high-resolution optimality of the non-orthogonal transform fails at the deployed rate.
Extended reading notes
Core claim
The central discovery is that the optimal attention-preserving transform for keys is not orthogonal. Minimizing the key-query inner-product error under a high-resolution model yields $R_K = M_q^{1/2}E$, where $M_q$ is the query second moment and $E$ is the eigenbasis of $M_q^{1/2}\widehat S_k M_q^{1/2}$; the transform satisfies $R_K R_K^\top = M_q$, and the generalized Parseval relation $\|r-\hat r\|_2^2 = \|k-\hat k\|_{M_q}^2$ turns the attention-weighted distortion into plain MSE in the transform domain. Consequently any MSE-optimal vector quantizer can act on the transformed coefficients. For values the optimal transform is orthogonal and equals the output-covariance eigenbasis. A second result, Theorem 2, shows that under an independent-Gaussian high-resolution model, grouping coefficients into equal-volume partitions lets equal-size fixed-width codebooks attain the variable-rate rate-distortion optimum; the deployed method, NOVA-KV, combines these two pieces with a lookup-only read path.
Load-bearing premise
The entire optimality argument assumes the high-resolution quantization formula (Zador's distortion with an asymptotically vanishing error term) holds at the deployed 2 bits per element, yet the paper's own Appendix H measures a rate exponent of about 1.82 instead of the ideal 2.
Editorial extensions
If this is right
- At 2 bits per element, NOVA-KV recovers most of the long-context retrieval accuracy lost by scalar methods, tracking BF16 more closely as context grows to 128K.
- On hybrid-attention MoE models such as GPT-OSS-20B, orthogonal-transform 2-bit baselines collapse to near-zero retrieval while NOVA-KV remains effective, because energy compaction rather than variance flattening protects the dense global-attention layers.
- The read path stays a fixed-width codebook lookup plus a per-token scale; the inverse key transform is folded into the query, so decoding is within a few percent of the scalar-transform baseline at long contexts.
- Volume-equalizing grouping makes fixed-width vector quantization attain the variable-rate distortion bound, so serving engines get a fixed token bit budget without an asymptotic rate-distortion penalty.
- The transform is fitted offline from query and key statistics, so the same calibrated codebooks can be deployed across requests without per-request quantization metadata on the read path.
Reading between the lines
- If the high-resolution model degrades further at even lower rates, the exact form of the optimal transform may shift; the paper's own measurement of a rate exponent around 1.82 at 2 bits suggests a finite-rate correction term could sharpen or alter the optimality claim.
- A natural testable extension is to learn the grouping jointly with the transform and codebooks rather than by the deterministic log-variance deal, since the partition is chosen greedily and balancing volumes is NP-hard.
- The dependence on calibration queries implies the transform may need periodic recalibration under distribution shift; the paper tests cross-domain calibration but not temporal drift within a deployment.
- Because the method preserves attention logits rather than token-level outputs, it may transfer to other score-based cache consumers such as speculative decoding or retrieval heads, though the paper does not test those.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes NOVA-KV, a 2-bits-per-element KV cache quantization method for long-context LLM decoding. It formulates cache compression as transform coding where the distortion is the error in attention products, and derives two theoretical results: (i) under a high-resolution quantization model, the attention-aware key transform is non-orthogonal, takes the form R_K = M_q^{1/2}E, and converts the query-weighted key distortion into plain MSE in the transform domain (Theorem 1, Proposition 1); and (ii) under an independent-Gaussian high-resolution model, volume-equalizing partitions of transform coefficients allow equal-size VQ codebooks to attain the variable-rate distortion optimum with a fixed-width layout (Theorem 2). The method is implemented in a SGLang-based serving stack with a fused decode kernel, and evaluated on RULER NIAH, LongBench, LooGLE, and five reasoning/coding benchmarks across Llama-3.1-8B, Qwen3-4B/8B, and GPT-OSS-20B. At 2.22 BPE, NOVA-KV substantially outperforms orthogonal-transform baselines on long-context retrieval, especially on GPT-OSS-20B, while remaining competitive in decode throughput.
Significance. If the results hold, the paper makes a significant contribution to efficient LLM serving: it is the first KV-cache compressor in this line that derives both the transform and the quantization grouping from an attention-aware distortion criterion rather than from heuristic rotations, and it demonstrates the practical value of non-orthogonal transforms plus vector quantization at 2 bits. The empirical work is unusually careful: transforms and codebooks are calibrated on a disjoint domain without degrading accuracy; the high-resolution model is explicitly tested in Appendix H with a measured rate exponent near 1.82; the chunked-prefill dequantization leak is identified and closed; the throughput protocol is well documented; and code is released. The transform-quantizer and partition ablations isolate the contributions of each design element and support the central mechanism.
major comments (3)
- [§3.1, Theorem 1; Appendix F.2] For the deployed full-rank case p=d, the objective in Eq. (6) is identically zero for every invertible R, because R R† = I, so Theorem 1 does not select the orthogonal factor E that appears in the deployed transform R_K = M_q^{1/2}E. The proof's Step 5 solves a PCA problem for p<d, and the full-rank E is effectively chosen as the limit of the low-rank solutions rather than as a proven minimizer of Eq. (6). This is load-bearing because the abstract and Section 3.1 call the key transform 'optimal.' The proven optimality covers the stretch M_q^{1/2} via the Linder companding condition and the low-rank (p<d) cases, but not the full-rank rotation. Please restate Theorem 1 for p<d and either prove that the deployed full-rank E is optimal under a stated nested criterion (e.g., optimal for every p) or explicitly identify E as a heuristic chosen by energy compaction.
- [§4 Eq. (7); §5; Appendix H.3] The theoretical derivations of both the transform and the grouping rest on Zador's high-resolution formula Eq. (7) with the 2^{-2b} term. Appendix H.3 measures a rate exponent α≈1.82–1.88 at the operating point b=2, and the manuscript itself states that the results are 'asymptotic rather than finite-rate guarantees.' The empirical partition ordering in Appendix H.4 still matches Theorem 2, and equal-volume grouping would remain optimal under D_ℓ(b)=C 2^{-αb} v_ℓ^{1/g}, but Eq. (8)'s bit-allocation formula and the phrase 'attain the variable-rate optimum' in Section 6 are tied to α=2. Please make the high-resolution qualification consistent in the abstract, Section 6, and the 'optimal' summaries, and state explicitly which conclusions are robust to the measured finite-rate exponent and which are not.
- [Algorithm 3; §4; Appendix B.3] The deployed key codebooks are trained on per-token RMS-normalized coefficients with a fitted per-token scale ρ_j, as described in Algorithm 3, but Theorem 2 models unnormalized independent zero-mean Gaussians with fixed variances σ_i^2. The manuscript acknowledges that ρ_j 'is not part of the model of Theorem 2,' yet the theorem is cited in Section 4 as justifying the deployed grouping and in Section 5's ablation. Please either extend the analysis to the normalized/scaled setting (e.g., show that the volume-equalizing grouping remains optimal under per-token scaling within the model's assumptions) or state the scope of Theorem 2's applicability to the deployed quantizer more explicitly.
minor comments (3)
- [§5, first paragraph] The sentence 'The high-resolution model might not hold at2bits, we use the theory to guide our design rather than as a guarantee' has missing spacing and should be split into two sentences or joined with a semicolon.
- [Appendix B.1] The phrase 'Divided by the 2nlayernkvdL ctxB quantized cache scalars' is difficult to parse; please define n_layer, n_kv, d_L, and ctx_B in the text or introduce a clearer formula.
- [Tables 2 and 3] The captions report 'mean±95% CI' but do not state the number of rollouts or samples in the caption itself; the information is in Appendix B, but adding a short parenthetical in each caption would improve readability.
Circularity Check
No significant circularity: the transform and grouping are derived from stated high-resolution distortion models, and the predictions are evaluated on held-out coefficients rather than being equivalent to fitted inputs.
full rationale
The paper's derivation chain is self-contained and does not reduce any reported prediction to its own inputs. The key transform RK = Mq^{1/2}E is obtained as a closed-form minimizer of the stated Mq-weighted low-rank reconstruction objective (Eq. 6) via the Ky Fan/PCA argument in Appendix F.2, and the generalized Parseval identity (Prop. 1) follows algebraically from RR^T = Mq. Neither step fits a parameter to the benchmark being predicted. Theorem 2 is a direct consequence of Zador's high-resolution formula (Eq. 7): the optimal variable-rate allocation and the fixed-rate equality condition are derived in Appendix G.1, and the equal-volume grouping is chosen to satisfy that condition. The fitted quantities (Cg in Appendix H, codebooks, per-token scales) are standard estimator components, and the empirical accuracy results are out-of-sample; the calibration ablation using MMLU further shows that calibrating on the evaluation domain confers no advantage. The only self-citation (Girault et al. 2018, with co-author Ortega) is a pointer for a one-line algebraic identity and is not load-bearing. The paper explicitly flags that the high-resolution model may not hold at 2 bits and that its results are asymptotic rather than finite-rate guarantees; Appendix H even measures a rate exponent near 1.82 rather than 2. These are correctness risks in the strength of the 'optimal' label, as is the degeneracy of Theorem 1's objective at p=d, where every invertible R achieves zero reconstruction error. However, these are gaps between the claimed optimality and what is proven, not circular reductions: no equation in the paper makes a reported prediction equal, by construction, to a fitted value or to a self-citation chain.
Assumptions & free parameters
free parameters (5)
- Key transform RK =
M_q^{1/2} E, with E eigenvectors of M_q^{1/2} S_k M_q^{1/2}
- Per-group key codebooks =
256 centroids per group (8-bit index for g=4)
- Per-token key scale rho_j =
computed per token
- Sink/recent band sizes (nsink, nrec) =
64, 256
- Value transform and quantizer =
OSCAR rotation and INT2 affine quantizer
assumptions (5)
- domain assumption Zador high-resolution distortion model, Eq. (7): D_l(b) = C_g 2^{-2b} v_l^{1/g} (1+o(1))
- domain assumption Transform coefficients are independent zero-mean Gaussians
- domain assumption Calibration prompts are representative of serving distribution
- standard math Softmax row function is 1/2-Lipschitz (Gao and Pavel, 2017)
- domain assumption M_q is positive definite on calibration queries
Cite this review
Pith. "Pith review of Spend Bits Where Queries Look: KV Cache Vector Quantization with Attention-Preserving Transforms." pith.science (2026). https://pith.science/paper/MKR7HBTM
@misc{pith2026260804074,
author = {Pith},
title = {Pith review of: Spend Bits Where Queries Look: KV Cache Vector Quantization with Attention-Preserving Transforms},
year = {2026},
howpublished = {\url{https://pith.science/paper/MKR7HBTM}},
note = {Machine review of arXiv:2608.04074}
}
read the original abstract
Long-context LLM decoding reads the key-value (KV) cache at every step. Loading it takes longer than computing attention over it, so throughput is bandwidth-bound. Hence, reducing the cache size can raise both decoding speed and serving capacity. The challenge is to reduce cache size while preserving the attention products, keeping reconstruction cheap, and using a fixed per-token bit count. At two bits per element, the most competitive methods rely on orthogonal transforms. However, existing techniques are either data-oblivious or use the query statistics without deriving the transform from a distortion criterion. Moreover, they rely on transforms built on top of random or Hadamard rotations, which equalize variances across entries rather than compacting energy, and fixed-width scalar quantizers, which are suboptimal at low rates. In this paper, we formulate KV cache quantization as a transform coding problem in which distortion is the error in the attention products. We derive closed-form optimal transforms for keys and values from calibration statistics, under a high-resolution model. We show that the optimal key transform is not orthogonal and satisfies a generalized Parseval relation: the attention-aware distortion becomes mean-squared error (MSE) in the transform domain. Thus, we can use MSE-optimal vector quantizers applied directly to the transformed key coefficients. To meet the fixed-width layout requirement, we show that grouping coefficients into equal-volume partitions makes equal-size codebooks attain the variable-rate optimum under the same high-resolution model. At two bits per element, our method, termed NOVA-KV, recovers most of the long-context retrieval accuracy lost by scalar quantization methods at comparable throughput.
Reference graph
Works this paper leans on
-
[1]
gpt-oss-120b & gpt-oss-20b model card.arXiv preprint arXiv:2508.10925,
Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. gpt-oss-120b & gpt-oss-20b model card.arXiv preprint arXiv:2508.10925,
-
[11]
Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. Ruler: What’s the real context size of your long-context language models?arXiv preprint arXiv:2404.06654,
-
[12]
Livecodebench: Holistic and contamination free evaluation of large language models for code
Naman Jain, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. In International Conference on Learning Representations, volume 2025, pages 58791–58831,
work page 2025
-
[13]
Loogle: Can long-context language models understand long contexts?arXiv preprint arXiv:2311.04939,
Jiaqi Li, Mengmeng Wang, Zilong Zheng, and Muhan Zhang. Loogle: Can long-context language models understand long contexts?arXiv preprint arXiv:2311.04939,
-
[14]
Commvq: Commutative vector quantization for kv cache compression
Junyan Li, Yang Zhang, Muhammad Yusuf Hassan, Talha Chafekar, Tianle Cai, Zhile Ren, Pengsheng Guo, Foroozan Karimzadeh, Chong Wang, and Chuang Gan. Commvq: Commutative vector quantization for kv cache compression. arXiv preprint arXiv:2506.18879,
-
[15]
Jamba: A hybrid transformer-mamba language model.arXiv preprint arXiv:2403.19887,
Opher Lieber, Barak Lenz, Hofit Bata, Gal Cohen, Jhonathan Osin, Itay Dalmedigos, Erez Safahi, Shaked Meirom, Yonatan Belinkov, Shai Shalev-Shwartz, et al. Jamba: A hybrid transformer-mamba language model.arXiv preprint arXiv:2403.19887,
-
[16]
Let’s verify step by step
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. InInternational Conference on Learning Representations, volume 2024, pages 39578–39601,
2024
-
[18]
Kivi: A tuning-free asymmetric 2bit quantization for kv cache.arXiv preprint arXiv:2402.02750, 2024b
Zirui Liu, Jiayi Yuan, Hongye Jin, Shaochen Zhong, Zhaozhuo Xu, Vladimir Braverman, Beidi Chen, and Xia Hu. Kivi: A tuning-free asymmetric 2bit quantization for kv cache.arXiv preprint arXiv:2402.02750, 2024b. Stuart Lloyd. Least squares quantization in pcm.IEEE transactions on information theory, 28(2):129–137,
Show all 29 references
-
[20]
Gpqa: A graduate-level google-proof q&a benchmark.arXiv preprint arXiv:2311.12022,
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark.arXiv preprint arXiv:2311.12022,
-
[21]
Magicdec: Breaking the latency-throughput tradeoff for long context generation with speculative decoding
Ranajoy Sadhukhan, Jian Chen, Zhuoming Chen, Vashisth Tiwari, Ruihang Lai, Jinyuan Shi, Ian Yen, Avner May, Tianqi Chen, and Beidi Chen. Magicdec: Breaking the latency-throughput tradeoff for long context generation with speculative decoding. InInternational Conference on Lear...
2025
-
[22]
Fast transformer decoding: One write-head is all you need.arXiv preprint arXiv:1911.02150,
Noam Shazeer. Fast transformer decoding: One write-head is all you need.arXiv preprint arXiv:1911.02150,
1911 arXiv
-
[23]
Rotatekv: Accurate and robust 2-bit kv cache quantization for llms via outlier-aware adaptive rotations.arXiv preprint arXiv:2501.16383,
Zunhai Su, Zhe Chen, Wang Shen, Hanyu Wei, Linge Li, Huangqi Yu, and Kehong Yuan. Rotatekv: Accurate and robust 2-bit kv cache quantization for llms via outlier-aware adaptive rotations.arXiv preprint arXiv:2501.16383,
-
[26]
Efficient streaming language models with attention sinks
Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. InInternational Conference on Learning Representations, volume 2024, pages 21875–21895,
2024
-
[27]
Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025a
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025a. Songlin Yang, Jan Kautz, and Ali Hatamizadeh. Gated delta networks: Improving mamba2 wit...
2025 arXiv
-
[29]
Oscar: Offline spectral covariance-aware rotation for 2-bit kv cache quantization.arXiv preprint arXiv:2605.17757,
Zhongzhu Zhou, Donglin Zhuang, Jisen Li, Ziyan Chen, Shuaiwen Leon Song, Ben Athiwaratkun, and Xiaoxia Wu. Oscar: Offline spectral covariance-aware rotation for 2-bit kv cache quantization.arXiv preprint arXiv:2605.17757,
-
[1949]
On the properties of the softmax function with application in game theory and reinforcement learning.arXiv preprint arXiv:1704.00805,
Bolin Gao and Lacra Pavel. On the properties of the softmax function with application in game theory and reinforcement learning.arXiv preprint arXiv:1704.00805,
-
[1971]
Nqkv: A kv cache quantization scheme based on normal distribution characteristics.arXiv preprint arXiv:2505.16210,
Zhihang Cai, Xingjun Zhang, Zhendong Tan, and Zheng Wei. Nqkv: A kv cache quantization scheme based on normal distribution characteristics.arXiv preprint arXiv:2505.16210,
-
[1982]
Turboquant: Online vector quantization with near-optimal distortion rate.arXiv preprint arXiv:2504.19874,
Amir Zandieh, Majid Daliri, Majid Hadian, and Vahab Mirrokni. Turboquant: Online vector quantization with near-optimal distortion rate.arXiv preprint arXiv:2504.19874,
-
[1989]
AIME 2025: American invitational mathematics examination.https://maa
Mathematical Association of America. AIME 2025: American invitational mathematics examination.https://maa. org/math-competitions/aime,
2025
-
[1992]
doi: 10.1007/978-1-4615-3626-0
ISBN 978-0-7923-9181-4. doi: 10.1007/978-1-4615-3626-0. Benjamin Girault, Antonio Ortega, and Shrikanth S Narayanan. Irregularity-aware graph fourier transforms.IEEE Transactions on Signal Processing, 66(21):5746–5761,
-
[1998]
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, ...
2025 arXiv
-
[1999]
Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model.arXiv preprint arXiv:2405.04434, 2024a
Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, et al. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model.arXiv preprint arXiv:2405.04434, 2024a. Zechun Liu, Changsheng Zhao, ...
2025 arXiv
-
[2001]
The llama 3 herd of models.arXiv preprint arXiv:2407.21783,
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783,
-
[2002]
Measuring mathematical problem solving with the math dataset.arXiv preprint arXiv:2103.03874,
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset.arXiv preprint arXiv:2103.03874,
-
[2009]
Kitty: Accurate and efficient 2-bit kv cache quantization with dynamic channel-wise precision boost.arXiv preprint arXiv:2511.18643,
Haojun Xia, Xiaoxia Wu, Jisen Li, Robert Wu, Junxiong Wang, Jue Wang, Chenxi Li, Aman Singhal, Alay Dilipbhai Shah, Alpay Ariyak, et al. Kitty: Accurate and efficient 2-bit kv cache quantization with dynamic channel-wise precision boost.arXiv preprint arXiv:2511.18643,
-
[2022]
Expected attention: Kv cache compression by estimating attention from future queries distribution.arXiv preprint arXiv:2510.00636,
Alessio Devoto, Maximilian Jeblick, and Simon Jégou. Expected attention: Kv cache compression by estimating attention from future queries distribution.arXiv preprint arXiv:2510.00636,
-
[2023]
Evaluating large language models trained on code
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374,
-
[2024]
Longbench: A bilingual, multitask benchmark for long context understanding.arXiv preprint arXiv:2308.14508,
Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, et al. Longbench: A bilingual, multitask benchmark for long context understanding.arXiv preprint arXiv:2308.14508,
-
[2025]
Gqa: Training generalized multi-query transformer models from multi-head checkpoints
Joshua Ainslie, James Lee-Thorp, Michiel De Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai. Gqa: Training generalized multi-query transformer models from multi-head checkpoints. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processin...
2023
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.