Pith. sign in

REVIEW 4 major objections 5 minor 23 references

A JoLT for the KV Cache: Near-Lossless KV Cache Compression via Joint Tucker and JL-Residual Allocation for LLMs

T0 review · 4 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Treating the transformer KV cache as a third-order tensor, JoLT compresses it 2–3× with near-lossless quality by jointly allocating Tucker ranks and a rotated residual under a single byte budget.

desk verdict JoLT is a plausible new KV-cache compression method with a solid 512/1024 free-zone result, but the 2048-context claim rests on an unshown matched-chunk run that needs to be produced. read the letter →

arxiv 2607.12550 v2 pith:NFUK7KJ5 submitted 2026-07-14 cs.LG cs.CLmath.OC

Rahul Krishnan , Volker Schulz This is my paper · ORCID
classification cs.LGcs.CLmath.OC
keywords KVcachecompressiontensordecompositionTuckerquantizationLagrangianallocationLLMinferencenear-losslessgrouped-queryattention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the KV cache—the memory that stores previous-token keys and values during transformer inference—can be shrunk by two to three times with essentially no loss in model quality. It does this by viewing the cache at each layer as a three-dimensional tensor and compressing only the two axes that carry redundancy, then using a rotated low-bit residual to restore the energy that truncation removes. A single optimization allocates the tensor ranks and residual bit-widths together under one byte budget. If right, this would let LLM serving cut the dominant memory cost of long contexts without retraining, on both grouped-query and multi-head attention models. The paper supports the claim with perplexity, math reasoning, and long-context retrieval benchmarks, plus reconstruction-error measurements.

What carries the argument

The central machinery is threefold: (1) a partial Tucker decomposition that truncates only the token and feature modes (leaving head and layer modes intact), implemented via sequentially truncated HOSVD; (2) a rotated residual that applies a random orthogonal rotation to the truncation error before low-bit uniform quantization, spreading outlier energy so quantizer distortion drops; and (3) a Lagrangian dual solve that, for a fixed byte budget, prices Tucker ranks against residual bits on the same error scale and allocates them jointly per (layer group, K/V). This joint allocation is what lets the method move budget between keys and values and between ranks and residual, which the paper argu

What would settle it

Run a controlled perplexity experiment at 2048-token context on LLaMA-2-13B using exactly the same evaluation chunks for the uncompressed baseline and for the 2× and 3× compressed caches. If the matched-chunk deltas exceed a small tolerance (say +0.5% relative), the paper's free zone at 2048 would fail for the MHA architecture; if they land within +0.06% as claimed, the artifact explanation is confirmed.

Watch

Extended reading notes

Core claim

The paper's central claim is that the KV cache at a layer is best treated as a third-order tensor (heads × tokens × features), and that compressing only the token and feature axes with a partial Tucker decomposition, then recovering the lost energy with a rotated low-bit residual, yields a near-lossless 2–3× compression. What makes this work is a single Lagrangian dual that allocates Tucker ranks and residual bit-widths per layer group and separately for keys and values under one byte budget. The result holds for both a GQA model (Mistral-7B) and an MHA model (LLaMA-2-13B) across perplexity, GSM8K math reasoning, and RULER needle-in-a-haystack retrieval, with reconstruction error roughly ten

Load-bearing premise

The near-lossless claim at 2048-token context rests on the assertion that the measured 2–3× perplexity offsets (up to +2.5% on LLaMA) are artifacts of evaluating compressed perplexity on 100 chunks versus a 300-chunk baseline; the matched-chunk runs that would verify this are not shown in the paper.

Editorial extensions

If this is right

  • LLM serving can cut KV cache memory by 2–3× without retraining or fine-tuning, on both grouped-query and multi-head attention architectures, directly raising the context-length or batch-size ceiling.
  • The compression applies at inference time to already-trained models, so it is a drop-in memory optimization for existing deployments.
  • The method fills the 2–3× compression band that fixed-rate 4-bit quantizers cannot express, making intermediate budgets accessible.
  • The architecture split past the free zone (GQA degrades gracefully, MHA degrades sharply above 4×) tells designers to stay inside 2–3× unless they build an MHA-aware backbone.
  • Compression-time speedups (5–13× via randomized SVD) make the method practical at long contexts where exact SVD would be too slow.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's spectral finding that key spectra decay sharply while value spectra are flat (with values 2–3× harder to compress) suggests a general principle: value projections spread energy across feature directions, so future compressors should always reserve more budget for V than K; this may extend to new attention variants beyond GQA/MHA.
  • The success of the rotated residual points to a broader recipe: any low-rank backbone can be pushed to near-lossless by quantizing the residual in a rotated basis; this could be applied to other tensor-compressed states (e.g., activations, optimizer states) in LLM inference and training.
  • If the chunk-set artifact explanation at 2048 context holds, the free zone likely extends to even longer contexts, since effective token-mode rank grows sublinearly with context; this is testable by running FlashJoLT at 32K and 64K contexts.
  • The Lagrangian dual formulation is a clean way to unify rank and bit allocation; the same cost model could be reused for joint quantization-and-pruning of weights, not just the cache.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces JoLT, a KV-cache compression method that treats the cache at each layer as a third-order tensor, applies a partial Tucker decomposition truncating only the token and feature axes, and restores the discarded energy with a rotated low-bit residual. A single Lagrangian dual allocates Tucker ranks and residual bit-widths jointly, per layer group and separately for keys and values, under a byte budget. A randomized-SVD variant, FlashJoLT, is proposed for faster compression. On Mistral-7B-v0.3 (GQA) and LLaMA-2-13B (MHA), the authors report a near-lossless 2–3× "free zone" in perplexity, GSM8K accuracy, and RULER retrieval, with reconstruction error roughly an order of magnitude below cross-layer SVD and 4-bit quantization at 2×. The paper is transparent about several limitations: calibration robustness is deferred, the long-context cap policy is validated only to 8192 tokens, and decode-time reconstruction is not yet kernel-fused.

Significance. If the empirical claims hold, the paper makes a practically valuable contribution: 2–3× KV-cache compression with near-lossless quality on both GQA and MHA architectures, supported by a clean tensor formulation and an explicit joint-allocation mechanism. The strengths include evaluation on real perplexity, GSM8K, and RULER rather than only reconstruction error; ablations that isolate the residual, the joint allocator, and per-group ranks; a reproducibility reference (exact backbone) separate from the fast variant; and an epsilon-calibration procedure that is independent of the evaluation data, so I see no circularity in the main quality measurements. The central caveat is that the 2048-context leg of the free-zone claim currently rests on asserted but unreported matched-chunk runs, and some headlined comparisons are not rate-matched. These issues are fixable but load-bearing for the exact scope of the central claim.

major comments (4)
  1. [Appendix D, Tables 6–7; Section 9] The 2048-context free-zone evidence is missing. Table 6 reports Mistral 2× at −0.42% and 3× at −0.34%; Table 7 reports LLaMA 2× at +2.54% and 3× at +2.58%. The footnotes attribute these to a chunk-set artifact (100 compressed chunks vs. a 300-chunk baseline) and state that matched-chunk runs show deltas within +0.14% and +0.06%, but those matched runs are not shown anywhere. Because Section 9 asserts the free zone holds at all three context lengths, and Table 1/Figure 1 cover only T=1024, this is the most load-bearing unresolved point: if the artifact explanation is wrong, LLaMA at 2048 is not near-lossless at 2–3×. Please report the matched-chunk data in full, or narrow the claim to 512/1024. Also define what threshold is meant by "near-lossless."
  2. [Section 9 vs. Tables 3–4] The conclusion that the free zone holds "across GSM8K and RULER on both a GQA and an MHA model" is stronger than the data support. Table 3 has no LLaMA 3× GSM8K row (em dash), and Table 4 reports LLaMA RULER only at ctx-4096, while Mistral RULER is reported up to 16K. The supported statement is: perplexity at 2–3× on both models; GSM8K at 2× on LLaMA and 2–3× on Mistral; RULER at 2× on LLaMA (ctx-4096) and 2–3× on Mistral up to 16K. Please either add the missing cell and LLaMA long-context RULER, or align Section 9 with the actual table coverage.
  3. [Section 4, Eq. (2)–(4)] The separable error model e ≈ ε²(b)·τ is the objective of the Lagrangian allocation, but ε²(b) is calibrated only on a Gaussian round-trip. No evidence is provided that this model accurately describes real KV-cache residual spectra, and Section 9 says calibration robustness across text domains is "in progress." If Eq. (2) is inaccurate, the claimed optimality of the joint allocation is not established, although the measured perplexity/task results are not circular. To support the allocation claim, please validate Eq. (2) on real caches or show that final rank/bit allocations are insensitive to ε²(b) (e.g., perturb ε² and rerun the allocator).
  4. [Table 2 and Abstract] The abstract's "order-of-magnitude reduction ... over strong baselines" overstates the comparison for the int4 per-channel baseline. JoLT is reported at 2×, while int4 per-channel has a native floor achieving ≈3.97×; the table footnote discloses this, but the abstract does not. At the same rate, the fair comparison would be at ~4× or via a rate–distortion curve. The xKV comparison is close to rate-matched (both ≈2×) and does support the order-of-magnitude claim. Please restrict the abstract's 4-bit claim to a rate-matched setting or add the missing int4 point.
minor comments (5)
  1. [Sections 4 and 7] There is an inconsistency in the residual bit-width set. Section 4 defines b ∈ {0,2,4,8}, while Section 7 says that an 8-bit residual "cannot fit the byte budget above 2×" and that JoLT "fixes the residual at four bits and lets the allocator decide only whether to spend them." Clarify the actual feasible set and the default used in the experiments.
  2. [Table 5 and Appendix F, Table 8] At T=1024, R=4, the greedy allocator slightly outperforms the joint allocator (6.717 vs. 6.928 PPL). The text says greedy is "more competitive at low ratio, where the two are within noise," but the table shows a non-negligible advantage at 4×. Please clarify that the joint method's benefit appears in the high-compression regime and not at all low ratios.
  3. [Section 4, Section 3, and Appendix A] Several load-bearing comparisons are deferred to an unspecified "companion paper": the claim that a full four-mode allocator returns head/layer ranks to full size, and the full multilinear format comparison. Since these motivate the partial Tucker design, the manuscript should at least summarize the supporting numbers or provide a citation; currently the reader cannot verify these assertions.
  4. [Appendix C and Table 4] The cap policy is stated to be validated only to 8192 tokens, yet Table 4 reports Mistral RULER at 16384 with the fast method. Clarify whether the 16K results use an extrapolated qcap policy or a different configuration, and state this explicitly where the 16K numbers are presented.
  5. [General] The paper would benefit from a concise definition of "near-lossless" (e.g., PPL delta threshold and confidence-interval criterion for task accuracy). Some readers may otherwise interpret the +2.54% 2048-context LLaMA cell as contradicting the label.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: quality is measured on real perplexity/GSM8K/RULER baselines, and the only calibrated error factor is fit on a Gaussian round-trip independent of the evaluation data; the main caveats are unshown companion-paper self-citations and an asserted-but-unverified 2048-context chunk-set correction, neither of which is a circular reduction.

full rationale

The claimed derivation chain does not reduce to its own inputs. The central empirical result (Section 9) is a measured 2–3x near-lossless band on perplexity, GSM8K, and RULER for Mistral-7B and LLaMA-2-13B; these metrics are evaluated against uncompressed baselines (Tables 1, 3, 4; Appendix D), not recovered from the method's own error model. The only calibrated quantity, ε^2(b) in Eq. (2), is fit once on a Gaussian round-trip and is independent of the evaluation data; the Lagrangian dual (Eqs. (3)–(4)) enforces only the byte budget, so no parameter is fitted to the target quality numbers. The 2–3x 'free zone' is a post-hoc descriptive label and does not enter any fitting step. Two evidence gaps are flagged, but they are not circularity. First, several design-optimality claims are delegated to an unspecified 'companion paper' (Section 3: 'values are typically 2–3× harder to compress than keys ... (quantified in a companion paper)'; Section 4: 'when we instead give a full four-mode allocator freedom ... identical rank and bit allocation (detailed in a companion paper)'; Section 2: 'a full multilinear comparison in a companion paper shows Tucker is the best of these formats at every ratio'). These are unverifiable self-citations, but the paper supplies direct spectral evidence for the head/token/feature asymmetry (Section 3, Appendix A), and the headline quality result does not hinge on these companion claims. Second, Appendix D footnotes to Tables 6 and 7 assert that the 2048-context 2–3x deviations (Mistral −0.42%, LLaMA +2.54%) are 'chunk-set artifact[s]' and that matched-chunk deltas are within +0.14%/+0.06%, but those matched-chunk runs are not shown; if the correction is wrong, the 2048-context leg of the free-zone claim weakens. This is an omitted verification, not a fitted-input circularity. Related limitations (LLaMA GSM8K at 3x absent, Table 3; calibration robustness 'in progress' and cap policy validated only to 8192, Section 9) affect evidential strength, not circularity. Score 1 reflects minor self-citation/citation-gap concern only.

Assumptions & free parameters 3 free parameters · 5 assumptions · 1 invented entities

The method's central contribution is algorithmic; it introduces no new physical entities. The main free parameters are the Gaussian-calibrated residual error epsilon^2(b), the unspecified layer-group count, and the cap-policy constants. The most consequential axioms are the domain assumption that the measured spectral structure generalizes and the ad-hoc error-model factorization that guides the Lagrangian allocation.

free parameters (3)
  • epsilon^2(b) — residual recovery fraction = Not reported numerically; epsilon^2(0)=1, decreases with b
    Calibrated once on a Gaussian round-trip; used in Eq (2) to price residual bits vs Tucker ranks in the allocation. This is a fitted parameter for the error model, though not fitted to the target tasks.
  • Layer group count G and layer grouping = Not specified in text
    The method partitions L layers into G contiguous groups, but G and the grouping scheme are not reported; a free design choice affecting allocation granularity.
  • Cap policy parameters qmin and cap = qmin=32 (R<=4), 64 (R>=5); cap=min(max(qmin, ceil(T/32)), 512)
    Chosen from a real-KV calibration sweep on Mistral; the cap policy parameters are hand-selected hyperparameters.
assumptions (5)
  • standard math Tucker/ST-HOSVD and randomized SVD produce valid low-rank approximations for the cache tensor.
    Used throughout Sections 4 and 5; no formal verification is provided but these are established numerical linear algebra methods.
  • standard math Lagrangian relaxation with bisection finds a feasible near-optimal allocation over the finite (rT, rd, b) grid.
    The objective in Eq (3) is optimized via the dual in Eq (4); integrality/duality gap is not discussed, but the grid is finite and the paper reports meeting ratio targets.
  • domain assumption The spectral structure measured on Mistral layer 15 generalizes: head/layer axes incompressible, token/feature axes redundant, values 2-3x harder than keys.
    This motivates pinning head/layer modes and splitting K/V budgets; measured on a single layer/model and asserted to extend to all layers and LLaMA-13B, with companion-paper support not included.
  • ad hoc to paper The separable error model e ≈ epsilon^2(b) * tau (Eq 2) with Gaussian-calibrated epsilon^2 is accurate for real KV caches.
    The allocation's optimality relies on this factorization; Section 9 says 'Validating calibration robustness across text domains is in progress,' so the assumption is not yet established.
  • domain assumption Near-losslessness on perplexity, GSM8K, and RULER for two models defines the free zone.
    The central claim generalizes from two models; the paper itself notes the GQA/MHA split beyond the free zone, and broader sweeps are left to future work.
invented entities (1)
  • Synthetic residual singular value for tail-mass accounting
    purpose: Corrects the allocator's energy estimate when FlashJoLT truncates the token-mode SVD tail beyond qcap.
    An algorithmic bookkeeping device, not a physical or model entity; it has no falsifiable handle outside the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A JoLT for the KV Cache: Near-Lossless KV Cache Compression via Joint Tucker and JL-Residual Allocation for LLMs." pith.science (2026). https://pith.science/paper/NFUK7KJ5

@misc{pith2026260712550,
  author       = {Pith},
  title        = {Pith review of: A JoLT for the KV Cache: Near-Lossless KV Cache Compression via Joint Tucker and JL-Residual Allocation for LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NFUK7KJ5}},
  note         = {Machine review of arXiv:2607.12550}
}
read the original abstract

The key-value (KV) cache has become the dominant memory cost of transformer inference: it grows with batch size, context length, and depth, and at long context it, rather than the model weights, sets the throughput ceiling. Existing reductions fall into two families. Low-rank methods factor two-dimensional slices of the cache, either per-head matrices or cross-layer feature blocks, and quantization methods lower the bit-width of every entry. Neither exploits the fact that the cache at a layer is naturally a third-order tensor whose three axes, the heads, the tokens, and the features, carry very different amounts of redundancy. We take this tensor view directly. Our method, JoLT (Joint Lagrangian Tucker), applies a partial Tucker decomposition that compresses only the token and feature axes while leaving the head and layer axes intact, then restores the energy that truncation discards with a rotated low-bit residual: a random orthogonal rotation followed by low-bit quantization. A single Lagrangian dual allocates the Tucker ranks and the residual bit-widths together, per layer group and separately for keys and values, under one byte budget. The result is a near-lossless 2-3x compression. Perplexity stays near-lossless on both a grouped-query-attention model (Mistral-7B-v0.3) and a multi-head-attention model (LLaMA-2-13B), and GSM8K accuracy and needle-in-a-haystack retrieval hold at the uncompressed baseline at 2x on both architectures and through 3x on the GQA model. At 2x, JoLT reconstructs the cache to relative Frobenius error 0.009 (K) and 0.006 (V) on both architectures. A randomized-SVD variant, FlashJoLT, delivers a 5-13x compression-time speedup at 1024-token context and matched quality.

Figures

Figures reproduced from arXiv: 2607.12550 by the authors.

Figure 1
Figure 1. Perplexity change versus compression ratio at [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 7 linked inside Pith

  1. [1]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. InAdvances in Neural Information Processing Systems (NeurIPS), 2017

    2017

  2. [2]

    Abdelfattah, and Kai-Chiang Wu

    Chi-Chih Chang, Wei-Cheng Lin, Chien-Yu Lin, Chong-Yan Chen, Yu-Fang Hu, Pei-Shuo Wang, Ning-Chi Huang, Luis Ceze, Mohamed S. Abdelfattah, and Kai-Chiang Wu. Palu: Compressing KV-cache with low-rank projection.arXiv preprint arXiv:2407.21118, 2024

    arXiv 2024

  3. [3]

    Abdelfattah

    Chi-Chih Chang, Wei-Cheng Lin, Chien-Yu Lin, Hung-Yueh Chiang, Yash Akhauri, Xilai Dai, Huiqiang Jiang, Yucheng Li, Luis Ceze, Kai-Chiang Wu, and Mohamed S. Abdelfattah. xKV: Cross-layer KV-cache compression via aligned singular vector extraction.arXiv preprint arXiv:2503.18893, 2025

    arXiv 2025

  4. [4]

    Kivi: A tuning-free asymmetric 2bit quantization for kv cache.arXiv preprint arXiv:2402.02750, 2024

    Zirui Liu et al. Kivi: A tuning-free asymmetric 2bit quantization for kv cache.arXiv preprint arXiv:2402.02750, 2024

    arXiv 2024

  5. [5]

    TurboQuant: Online vector quantization with near-optimal distortion rate.arXiv preprint arXiv:2504.19874, 2025

    Amir Zandieh, Majid Daliri, Majid Hadian, and Vahab Mirrokni. TurboQuant: Online vector quantization with near-optimal distortion rate.arXiv preprint arXiv:2504.19874, 2025

    arXiv 2025

  6. [6]

    H2o: Heavy-hitter oracle for efficient generative inference of large language models

    Zhenyu Zhang et al. H2o: Heavy-hitter oracle for efficient generative inference of large language models. In Advances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  7. [7]

    Efficient streaming language models with attention sinks

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. InInternational Conference on Learning Representations (ICLR), 2024

    2024

  8. [8]

    SnapKV: LLM knows what you are looking for before generation

    Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen. SnapKV: LLM knows what you are looking for before generation. InAdvances in Neural Information Processing Systems (NeurIPS), 2024

    2024

Show all 23 references
  1. [9]

    Ledyard R. Tucker. Some mathematical notes on three-mode factor analysis.Psychometrika, 31(3):279–311, 1966

    1966

  2. [10]

    A multilinear singular value decomposition.SIAM Journal on Matrix Analysis and Applications, 21(4):1253–1278, 2000

    Lieven De Lathauwer, Bart De Moor, and Joos Vandewalle. A multilinear singular value decomposition.SIAM Journal on Matrix Analysis and Applications, 21(4):1253–1278, 2000

    2000

  3. [11]

    Kolda and Brett W

    Tamara G. Kolda and Brett W. Bader. Tensor decompositions and applications.SIAM Review, 51(3):455–500, 2009

    2009

  4. [12]

    Oseledets

    Ivan V . Oseledets. Tensor-train decomposition.SIAM Journal on Scientific Computing, 33(5):2295–2317, 2011

    2011

  5. [13]

    Kilmer and Carla D

    Misha E. Kilmer and Carla D. Martin. Factorization strategies for third-order tensors.Linear Algebra and its Applications, 435(3):641–658, 2011

    2011

  6. [14]

    Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman

    Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. QuaRot: Outlier-free 4-bit inference in rotated LLMs. InAdvances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  7. [15]

    SpinQuant: LLM quantization with learned rotations

    Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. SpinQuant: LLM quantization with learned rotations. InInternational Conference on Learning Representations (ICLR), 2025

    2025

  8. [16]

    Nathan Halko, Per-Gunnar Martinsson, and Joel A. Tropp. Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions.SIAM Review, 53(2):217–288, 2011

    2011

  9. [17]

    GQA: Training generalized multi-query transformer models from multi-head checkpoints

    Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai. GQA: Training generalized multi-query transformer models from multi-head checkpoints. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processin...

    2023

  10. [18]

    A new truncation strategy for the higher-order singular value decomposition.SIAM Journal on Scientific Computing, 34(2):A1027–A1052, 2012

    Nick Vannieuwenhoven, Raf Vandebril, and Karl Meerbergen. A new truncation strategy for the higher-order singular value decomposition.SIAM Journal on Scientific Computing, 34(2):A1027–A1052, 2012

    2012

  11. [19]

    Pointer sentinel mixture models.arXiv preprint arXiv:1609.07843, 2016

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models.arXiv preprint arXiv:1609.07843, 2016

    2016 arXiv

  12. [20]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of Machine Learning Research, 21(140):1–67, 2020

    2020

  13. [21]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Karl Cobbe et al. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    2021 arXiv

  14. [22]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V . Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. InAdvances in Neural Information Processing Systems (NeurIPS), 2022. 8 A JoLT for the K...

    2022

  15. [23]

    RULER: What’s the real context size of your long-context language models?arXiv preprint arXiv:2404.06654, 2024

    Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. RULER: What’s the real context size of your long-context language models?arXiv preprint arXiv:2404.06654, 2024. 9 A JoLT for the KV Cache: Near-Lossless KV Cach...

    2024 arXiv

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.