Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

RAP: KV-Cache Compression via RoPE-Aligned Pruning

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RAP compresses the KV cache by pruning whole RoPE-aligned column pairs, preserving rotation semantics so the reconstruction matrix can be absorbed — achieving 30% cuts in cache, attention parameters, and FLOPs with perplexity near baseline.

desk verdict RAP has a genuinely new construction—pruning whole RoPE pairs so the expansion matrix absorbs into Wq/Wo—but the paper's accuracy advantage over PaLU evaporates once both get the same KD/LoRA recovery. read the letter →

arxiv 2602.02599 v4 pith:CGVD3TPO submitted 2026-02-01 cs.LG cs.AI

Jihao Xin , Tian Lyu , David Keyes , Hatem Ltaief , Marco Canini This is my paper
classification cs.LGcs.AI MSC 68T0768T50
keywords KVcachecompressionRoPEstructuredpruningLLMinferencelow-rankfactorizationknowledgedistillationattentionlong-context
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Long-context LLM inference is bottlenecked by the memory and compute of the key-value cache. The paper argues that the standard way to shrink the cache — low-rank factorization of the key/value projections — breaks down in modern models because Rotary Position Embedding (RoPE) rotates feature dimensions in pairs, so the reconstruction matrix cannot be absorbed and must be recomputed at every step. RAP instead prunes whole RoPE-aligned column pairs, keeping the 2x2 rotations intact. That single granularity choice lets the expansion matrix be folded into the query and output projections, eliminating reconstruction entirely. As a result, at 30% cache compression on LLaMA-3-8B, attention parameters drop to 70%, attention FLOPs to 70.3%, and prefill/decode latency to 83%/77% of baseline, while WikiText-2 perplexity stays at 8.82 versus 8.28 for the uncompressed model.

What carries the argument

RoPE-aligned pair pruning: columns of the key and value projection matrices are grouped into the 2x2 rotation blocks that RoPE defines, scored by Fisher information, and removed or retained as whole pairs. The retained columns form A; a binary expansion matrix B maps them back to their original indices and, because pairs are preserved, commutes with RoPE. That commutativity lets B_k^T be absorbed into W_q and B_v into W_o, eliminating the full-dimension reconstruction that low-rank factorization requires. Supporting pieces: adaptive budget allocation across layers and between K and V, and a custom non-contiguous Triton RoPE kernel that avoids per-head indexing overhead. Theorem 5.1 bounds th

What would settle it

Take any RoPE-based LLM not in the evaluated set (e.g., a Qwen-3 model), run RAP at ρ=30% with the same WikiText-2 KD recipe, and evaluate on LongBench and six zero-shot tasks: if accuracy falls far below the uncompressed model while PPL stays low, the finetune-transfer assumption fails. Alternatively, numerically check the identity RoPE(XA)B = RoPE(XAB) on random weights for the binary expansion construction; any mismatch would invalidate the absorption step that produces the speedup.

Watch

Extended reading notes

Core claim

The paper's central claim is that RoPE-commutativity — the condition that RoPE(XA)B equals RoPE(XAB) — is the missing constraint for any weight-decomposition approach to KV-cache compression, and that structured pruning at the RoPE-pair granularity satisfies it by construction. Because whole column pairs are removed, the resulting binary expansion matrix B commutes with the rotation, so B_k can be absorbed into W_q and B_v into W_o. The cached states are then the low-dimensional latent projections themselves, with no reconstruction step in the inference graph. The authors prove an error bound showing pruning loss is bounded by the summed Fisher scores of removed pairs, and they identify Fish

Load-bearing premise

Accuracy preservation relies on a short LoRA knowledge-distillation finetune on WikiText-2 transferring to commonsense and long-context tasks; without KD, pruning alone raises LLaMA WikiText-2 PPL from 8.28 to 28.86 at 30% compression.

Editorial extensions

If this is right

  • At ρ=30%, RAP cuts attention parameters to 70%, attention FLOPs to 70.3%, and prefill/decode latency to 83%/77% of baseline on LLaMA-3-8B, with WikiText-2 PPL 8.82 vs 8.28.
  • The same linear reductions hold on Mistral-7B, and the abstract reports evaluation across Llama, Mistral, and Qwen models from 3B to 14B.
  • RAP composes with orthogonal compression: applying 4-bit KV-cache quantization on top keeps perplexity close to baseline across compression ratios.
  • Because the inference graph is unchanged apart from dimension reduction, RAP needs no inference-stack modifications and introduces no reconstruction overhead; a Triton kernel removes the only source of indexing overhead.
  • In worst-case architectures like single-head attention, RAP is the only evaluated method that guarantees linear reductions in KV-cache, parameters, and FLOPs simultaneously, where SVD requires ρ>50% and PaLU ρ>33%.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The rotation-block granularity principle likely generalizes beyond RoPE to any rotation-based positional embedding: any weight decomposition that prunes or factorizes at the block that the positional transform rotates should inherit the same absorption property.
  • Because the paper's accuracy numbers at 40-50% compression degrade (PPL 9.64-13.74), a reader might infer that RAP's practical operating range is 20-30% compression — enough to roughly halve the memory overhead of long-context serving at negligible quality cost.
  • The dependency on KD/LoRA recovery is the main contingency: the pruning step alone is destructive (PPL 28.86 at ρ=30%), so deployments that cannot afford even a short finetune would need to treat RAP as a compression-plus-recovery package rather than a one-shot pruner.
  • The evaluation focuses on PPL, six commonsense zero-shot tasks, and LongBench; testing on instruction-following, code, or multilingual tasks would be a natural next step to see whether the WikiText-2 LoRA finetune transfers broadly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes RAP (RoPE-Aligned Pruning), a structured-pruning method that removes whole RoPE-aligned column pairs from KV projections, rather than individual channels, so that rotary-position-embedding rotation blocks are preserved and the resulting expansion can be absorbed into downstream weights. The pipeline uses Fisher-information pair scoring, an adaptive per-layer/K-V budget allocation, a binary expansion matrix that absorbs B into Wq/Wo, and a short LoRA-based knowledge-distillation recovery. The authors claim that at ρ=30% KV-cache compression, RAP simultaneously reduces attention parameters and FLOPs by about 30%, lowers prefill/decode attention latency to 83%/77% of baseline, and preserves accuracy; they also provide a theoretical error bound and experiments on LLaMA-3-8B and Mistral-7B comparing with SVD-LLM and PaLU.

Significance. The core construction is timely and useful: because modern LLMs apply RoPE after Q/K projections, pruning whole rotation pairs is a natural and, to my knowledge, novel granularity for KV-cache compression. The resource-reduction result (Table 2, Eq. (21)) is clean: RAP gives linear reductions in KV-cache size, attention parameters, and attention FLOPs without reconstruction overhead, and the measured latency improvements (Section 6.2.3) support this. The authors also provide a useful Triton kernel for non-contiguous RoPE, detailed hyperparameters, and ablations of the scoring/budget choices. The main weakness is that the accuracy-preservation claim is supported only when the RAP pipeline is augmented with LoRA/KD, while the baselines are reported without the same recovery; Appendix D.3 shows PaLU+KD matches RAP, which undermines the stated accuracy advantage. The resource-side contribution is still defensible, but the paper overclaims the accuracy comparison as currently written.

major comments (4)
  1. [Abstract / §6.1] The abstract states 'Our evaluation across Llama, Mistral, and Qwen models from 3B to 14B shows...' but the full-text abstract and §6.1 describe experiments only on LLaMA-3-8B and Mistral-7B. No Qwen or 3B/14B results appear anywhere in the paper. This is a direct mismatch between the advertised scope and the actual evidence. Either add the promised experiments or correct the abstract to match the evaluated models.
  2. [§6.1 'Comparison Fairness of KD'; Table 3; Appendix D.3, Table 7] The main comparison is asymmetric. RAP receives a LoRA/KD recovery step (rank 8, one-epoch-ish, 2.9h on LLaMA) while SVD and PaLU are reported without KD. Appendix Table 7 shows that applying the identical KD/LoRA recipe to PaLU yields PaLU+KD PPL 8.80 vs RAP 8.82 on LLaMA and 5.70 vs 5.68 on Mistral at ρ=30%, essentially erasing the accuracy gap that Table 3 and Table 4 attribute to RAP. The statement in §6.2 that 'RAP preserves strong accuracy... surpassing PaLU' is therefore not established by the reported evidence. A matched-budget comparison, with PaLU+KD included in the main table and in the zero-shot/LongBench evaluations, is required before claiming an accuracy advantage; otherwise the accuracy claims should be explicitly separated from the resource-reduction claims.
  3. [§4.4, §6.2.4, Table 5] The accuracy-preservation claim is carried entirely by the KD/LoRA recovery. Table 5 shows that without KD, LLaMA WikiText-2 PPL degrades from 8.28 to 28.86 at ρ=30%, and only after KD does it recover to 8.82. Moreover, the recovery is trained on WikiText-2 with cutoff-len 256 (Table 15), while the paper uses zero-shot commonsense and LongBench results to claim general accuracy and long-context stability. The main text reports LongBench only at ρ=20% (Figure 9), and no matched-budget LongBench results for PaLU+KD are given. The transferability of the WikiText-2 KD recovery to long-context tasks is an empirical assumption that is not demonstrated; this should be flagged as a limitation or directly tested.
  4. [Appendix B, proof of Theorem 5.1] The proof of Theorem 5.1 relies on the statement that 'cross-pair interactions are negligible, and the Fisher matrix can be approximated as block-diagonal over RoPE pairs.' This is an unproven assumption about the Hessian structure of the loss with respect to Wk, and the additive decomposition in Eq. (16) depends on it. If the theorem is presented as a theoretical justification for Fisher pair selection, the assumption needs either a proof or an empirical validation (e.g., measuring off-diagonal Fisher blocks for the models evaluated). Without this, the theorem is conditional on an unverified approximation.
minor comments (5)
  1. [§3, Table 1] The sentence describing RoPE pair indexing is garbled: 'j= 2x−1, j′ = 2x or j=x, j ′ = x+D/2 for x∈[1, D/2]' appears to be a fragment. Please clarify the two pairing conventions and their notation.
  2. [§4.4] Typo: 'Low-Rank Adopter' should be 'Low-Rank Adapter'.
  3. [Table 16/17] The tables are labeled 'latency speedup (avg%(max%))' but main text Table 3 reports 'latency' as a percentage of baseline (e.g., 83%). Please make the direction consistent (speedup vs. relative time) in all tables to avoid confusion.
  4. [§6.2.3, Figure 16] The description of Figure 16 says 'each cell's top-right triangle is Triton, bottom-left is PyTorch'; this is not visible in the text version. Consider replacing with a clearer bar/line plot or table.
  5. [General] The references are mostly standard, but some URLs (e.g., HuggingFace model pages for unreleased models) are not archival. Consider citing stable versions or technical reports where available.

Circularity Check

1 steps flagged · score 3.0 of 10

One in-sample evaluation step is partially circular: WikiText-2 PPL is both the KD training objective and the primary reported accuracy metric; the core RoPE-commutativity derivation is independent.

  1. fitted input called prediction [Section 4.4 'Accuracy Recovery' and Section 6.2.4 'Accuracy Preservation'; Table 5 / Table 4]
    "For calibration, we use the WikiText-2 dataset by default ... We combine cross-entropy (CE) loss on ground-truth labels with Kullback-Leibler (KL) divergence between teacher and student logits: L=L_CE+L_KD ... We use PPL as the primary quality metric ... Table 5 ... RAP (w/o KD) 28.86 ... RAP 8.82"

    The central accuracy-preservation result (WikiText-2 PPL 8.82 at rho=30%) is reported on the same dataset used for KD/LoRA fine-tuning, and the CE term of the KD loss is exactly the perplexity objective minimized during training. The improvement from 28.86 (pruned) to 8.82 is therefore a training fit on the reported metric rather than an out-of-sample prediction. Held-out zero-shot and LongBench results provide some independent support, so the step is not fully load-bearing, but the paper frames PPL as 'the primary quality metric' and presents the in-sample number as main evidence of accuracy preservation.

full rationale

The paper's core mechanism is an algebraic construction: pruning whole RoPE pairs preserves the 2x2 rotation blocks, so RoPE commutes with the absorbed B matrix (Eq. 5-10). This is a direct mathematical property, not a circular derivation. The Fisher error bound (Theorem 5.1) is a standard second-order approximation with an explicit block-diagonal assumption; Corollary 5.2 is a valid minimization of the stated upper bound, not a fitted prediction. Resource-reduction claims follow by the algebra in Table 2 and Appendix C. There is no self-citation chain, no imported uniqueness theorem, and no ansatz smuggled via citation. The only circularity-adjacent step is the KD/LoRA recovery: the model is fine-tuned on WikiText-2 and then WikiText-2 PPL is used as the primary accuracy metric, so that specific number is partly forced by optimization. The held-out zero-shot results (avg. 0.65 vs 0.68 baseline at rho=30%) and LongBench (rho=20%) give independent content, and the PaLU+KD fairness issue in Appendix Table 7 is a comparison-design concern rather than circularity. Score 3 reflects one partial circular evaluation step while acknowledging the central derivation is self-contained.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method itself rests on the standard block structure of RoPE, plus a set of empirical choices: Fisher scoring, the adaptive budget heuristic, LoRA-KD recovery, and the hybrid SVD-for-Wv strategy. No hidden fitted constants are introduced into the theoretical error bound, but the bound's block-diagonal Fisher assumption is asserted rather than established.

free parameters (5)
  • LoRA rank r_LoRA = 8
    Used in KD recovery for all pruned models; affects accuracy recovery and is a tuned hyperparameter.
  • LoRA alpha = 16
    Scaling factor for LoRA adapters, chosen for recovery quality.
  • KD loss weights = alpha_CE=0.4, alpha_KD=0.6
    Combination weights in Eq. (12); tuned on validation performance.
  • KD temperature = 2.0
    Softening temperature for teacher logits in Eq. (13).
  • Fisher calibration set size = 32 windows of length 2048 from WikiText-2
    Number of calibration samples used for Fisher scoring; authors claim sensitivity is low.
assumptions (5)
  • standard math RoPE applies independent 2x2 rotations to feature pairs, so selecting or pruning whole pairs commutes with RoPE.
    This is the basis of Eq. (5) and the entire absorption construction; it follows from the block-diagonal structure of RoPE.
  • domain assumption A second-order Taylor expansion of the loss around Wk is valid, and the Fisher information approximates the Hessian.
    Standard in second-order pruning analysis (cited Frantar & Alistarh, 2023); used in Appendix B without new justification.
  • ad hoc to paper The Fisher information matrix is block-diagonal over RoPE pairs; cross-pair interactions are negligible.
    Appendix B states this directly after Eq. (15): 'cross-pair interactions are negligible.' It is asserted, not proven, and is the linchpin of the additive loss bound in Theorem 5.1.
  • domain assumption KD with LoRA on WikiText-2 transfers to commonsense and LongBench accuracy.
    Section 4.4 and Table 5 show KD is essential for accuracy recovery, but cross-task transfer is treated as an empirical given rather than a demonstrated invariant.
  • domain assumption RoPE is applied only to Q and K projections in evaluated models, so Wv can be compressed with SVD rather than RAP.
    Section 4.5's hybrid strategy relies on this architectural property of LLaMA and Mistral; it would need re-derivation for models that apply RoPE to V.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RAP: KV-Cache Compression via RoPE-Aligned Pruning." pith.science (2026). https://pith.science/paper/CGVD3TPO

@misc{pith2026260202599,
  author       = {Pith},
  title        = {Pith review of: RAP: KV-Cache Compression via RoPE-Aligned Pruning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CGVD3TPO}},
  note         = {Machine review of arXiv:2602.02599}
}
read the original abstract

Long-context inference in large language models (LLMs) is bottlenecked by the memory and compute of the key-value (KV) cache. Structured pruning is a direct way to shrink it: dropping the least useful channels of the W_k, W_v projection weights to reduce the output KV dimensions. However, modern LLMs apply Rotary Position Embedding (RoPE) after the QK projections, which rotates feature dimensions in pairs. Therefore, removing individual channels breaks these pairs, corrupting RoPE's positional semantics and rendering the pruned model unusable. We propose RoPE-Aligned Pruning (RAP), which constrains the pruning granularity to RoPE-aligned pairs rather than individual channels: removing whole pairs to keep the rotation intact. Our evaluation across Llama, Mistral, and Qwen models from 3B to 14B shows that RAP preserves accuracy at 30% KV compression (retain ratio \r{ho} = 0.7), far outperforms RoPE-blind channel pruning, stays near the strongest low-rank method at lower attention cost, and composes with orthogonal methods such as quantization.

Figures

Figures reproduced from arXiv: 2602.02599 by the authors.

Figure 1
Figure 1. SVD-based KV-Cache compression requires full-dimensional reconstruction at runtime. Parameters GPTQ AWQ RAP PaLU QServe LLMPruner KVCache FLOPs Longformer BigBird KVQuant KIVI H2O [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Optimizations of KV-Cache/FLOPs/Parameters. autoregressive objective (Vaswani et al., 2017). During the decoding phase, the model processes one token at a time. Given the input embedding Xt ∈ R 1×Dˆ at step t, each attention head projects it into queries, keys, and values: Qt = XtWq, Kt = XtWk, Vt = XtWv, (2) where Wq,Wk,Wv ∈ R Dˆ×D, and D = D/H ˆ is the per￾head dimension, H is the number of attention heads. The ne… view at source ↗
Figure 3
Figure 3. RAP Example: Weight matrix W with 12 columns is grouped into 6 RoPE pairs, then pruned 50% based on the RoPE scores [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: LLaMA PPL of pruning one layer at a time. To quantify the importance of each RoPE pair, we adopt Fisher information, which approximates the Hessian (Fran￾tar & Alistarh, 2023) by accumulating gradient statistics, offering a computationally efficient proxy for parameter…
Figure 6
Figure 6. Figure 6: FLOPs Relative to Baseline 108% 10% 20% 30% 40% 50% Compression Ratio ( ) 70% 80% 90% 100% SVD PaLU RAP Attn Model 108% 95% 100% [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 8
Figure 8. Figure 8: Zero-shot Accuracy (ρ = 30%) TQ QS TR SS LC RP QM MN 0.2 0.4 0.6 0.8 1.0 Accuracy Baseline SVD PaLU RAP [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 11
Figure 11. Figure 11: Decode Latency Relative to Baseline Acc(Baseline) PPL(Baseline) Acc(RAP) PPL(RAP) 10% 20% 30% 40% 50% Compression Ratio ( ) 0.4 0.5 0.6 Accuracy (avg) 0 10 20 30 WikiText PPL 70 [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]
Figure 16
Figure 16. Figure 16: RoPE Latency Relative to Baseline tains the best average accuracy at ρ ∈ [10%, 40%] [PITH_FULL_IMAGE:figures/full_fig_p008_16.png]
Figure 17
Figure 17. Figure 17: Parameters Relative to Baseline (Mistral) 10% 20% 30% 40% 50% Compression Ratio ( ) 0.6 0.8 1.0 SVD PaLU RAP [PITH_FULL_IMAGE:figures/full_fig_p016_17.png]
Figure 20
Figure 20. Figure 20: Zero-shot Accuracy (ρ = 30%) (Mistral) 10 2 10 3 1 2 3 4 5 6 7 8 9 10 Epoch 4 6 10 1 WikiText PPL =10% =20% =30% =40% Baseline [PITH_FULL_IMAGE:figures/full_fig_p016_20.png]
Figure 23
Figure 23. Figure 23: Measured per-head per-token FLOPs vs. compression ratio for LLaMA and Mistral. 10% 20% 30% 40% 50% Compression (%) 60 80 100 Attention size (%) LLaMA SVD PaLU RAP 10% 20% 30% 40% 50% Compression (%) Mistral [PITH_FULL_IMAGE:figures/full_fig_p016_23.png]
Figure 25
Figure 25. Figure 25: Latency speedup vs. compression ratio for LLaMA and Mistral. We plot attention-layer speedup (prefill/decode) across compression ratios; full-model values are listed in [PITH_FULL_IMAGE:figures/full_fig_p016_25.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. RoPE-Aware Bit Allocation for KV-Cache Quantization

    cs.LG 2026-06 unverdicted novelty 7.0 of 10

    Block-GTQ performs RoPE-aware greedy bit allocation on KV caches using per-block energy scores, cutting logit MAE 32-80% versus uniform TQ-MSE and lifting long-context task scores substantially at 2-3 bits per dimension.

Reference graph

Works this paper leans on

17 extracted references · 9 linked inside Pith · cited by 1 Pith paper

  1. [1]

    L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    arXiv

  2. [2]

    Complete Experimental Results This appendix provides complete tables and supplementary plots for all experiments

    D. Complete Experimental Results This appendix provides complete tables and supplementary plots for all experiments. Compression ratios are reported as 10%–50% (retain 0.9–0.5). Unless stated otherwise, results are for the same settings as the main text. D.1. Mistral Figures This section presents Mistral-specific experimental results. Figures 17, 18, 19, ...

    2025

  3. [3]

    E., and Cohan, A

    Beltagy, I., Peters, M. E., and Cohan, A. Long- former: The long-document transformer.arXiv preprint arXiv:2004.05150,

    arXiv 2004

  4. [4]

    Gptq: Accurate post-training quantization for generative pre- trained transformers.arXiv preprint arXiv:2210.17323,

    Frantar, E., Ashkboos, S., Hoefler, T., and Alistarh, D. Gptq: Accurate post-training quantization for generative pre- trained transformers.arXiv preprint arXiv:2210.17323,

    arXiv

  5. [5]

    Proof of Theorem 5.1 Proof

    B.1. Proof of Theorem 5.1 Proof. Let ∆Wk =A k −W k denote the pruning-induced perturbation to the key projection matrix. Since RAP re- moves entire RoPE pairs, ∆Wk is block-sparse with respect to the RoPE pair decomposition, with nonzero entries only on the pruned pairsp /∈ S. Assuming L is twice continuously differentiable with re- spect to Wk, a second-...

    2023

  6. [7]

    Ma, X., Fang, G., and Wang, X

    URL https://doi.org/10.1109/ ICASSP49660.2025.10889000. Ma, X., Fang, G., and Wang, X. LLM-pruner: On the structural pruning of large language models. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.),Advances in Neural Information Pro- cessing Systems, volume 36, pp. 21702–21720. Curran Associates, Inc.,

    arXiv 2025

  7. [8]

    Does quantization affect models’ perfor- mance on long-context tasks? In Christodoulopou- los, C., Chakraborty, T., Rose, C., and Peng, V

    9 RAP: RoPE-Aligned Pruning Mekala, A., Atmakuru, A., Song, Y ., Karpinska, M., and Iyyer, M. Does quantization affect models’ perfor- mance on long-context tasks? In Christodoulopou- los, C., Chakraborty, T., Rose, C., and Peng, V . (eds.),Proceedings of the 2025 Conference on Em- pirical Methods in Natural Language Processing, pp. 9422–9470, Suzhou, Chi...

    2025

  8. [9]

    ISBN 979- 8-89176-332-6

    Asso- ciation for Computational Linguistics. ISBN 979- 8-89176-332-6. doi: 10.18653/v1/2025.emnlp-main

    doi:10.18653/v1/2025.emnlp-main 2025

Show all 17 references
  1. [11]

    Peng, B., Quesnelle, J., Fan, H., and Shippole, E

    Model repository. Peng, B., Quesnelle, J., Fan, H., and Shippole, E. Yarn: Efficient context window extension of large language models.URL https://arxiv.org/abs/2309.00071,

    arXiv

  2. [12]

    Llama: Open and efficient foundation lan- guage models.arXiv preprint arXiv:2302.13971,

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozi`ere, B., Goyal, N., Hambro, E., Azhar, F., et al. Llama: Open and efficient foundation lan- guage models.arXiv preprint arXiv:2302.13971,

    arXiv

  3. [14]

    Table 5.KD ablation across compression ratios (WikiText-2 PPL, LLaMA). Compression Baseline RAP (w/o KD) RAP 10%8.288.85 8.31 20%8.2814.19 8.57 30%8.2828.86 8.82 40%8.28105.74 9.64 50%8.284189.06 13.74 Table 6.KV-projection-only per-head per-token FLOPs implied by Table 2 (one...

  4. [42]

    Accuracy (PPL + lm-eval) PPL on WikiText-2 with seqlen=2048 and batch-size=8; zero-shot tasks: OpenBookQA, HellaSwag, PIQA, ARC-Easy, ARC-Challenge, WinoGrande. Seed:

  5. [479]

    emnlp-main.479/

    URL https://aclanthology.org/2025. emnlp-main.479/. Meta AI. LLaMA-4: Open and efficient foundation language models. https: //huggingface.co/meta-llama/ Llama-4-Scout-17B-16E-Instruct ,

    2025

  6. [2017]

    Circle-rope: Cone-like decoupled rotary posi- tional embedding for large vision-language models.arXiv preprint arXiv:2505.16416, 2025a

    Wang, C., Guo, J., Li, H., Tian, Y ., Nie, Y ., Xu, C., and Han, K. Circle-rope: Cone-like decoupled rotary posi- tional embedding for large vision-language models.arXiv preprint arXiv:2505.16416, 2025a. Wang, X., Zheng, Y ., Wan, Z., and Zhang, M. SVD-LLM: Truncation-aware si...

    arXiv

  7. [2023]

    Qwen technical report.arXiv preprint arXiv:2309.16609,

    Bai, J., Bai, S., Chu, Y ., Cui, Z., Dang, K., Deng, X., Fan, Y ., Ge, W., Han, Y ., Huang, F., et al. Qwen technical report.arXiv preprint arXiv:2309.16609,

    arXiv

  8. [2024]

    Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531,

    Hinton, G., Vinyals, O., and Dean, J. Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531,

    arXiv

  9. [2025]

    Deepseek- v3 technical report.arXiv preprint arXiv:2412.19437, 2024a

    Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., et al. Deepseek- v3 technical report.arXiv preprint arXiv:2412.19437, 2024a. Liu, Z., Yuan, J., Jin, H., Zhong, S., Xu, Z., Braverman, V ., Chen, B., and Hu, X. Kivi: a tuning-free a...

    arXiv

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.