Pith. sign in

REVIEW 5 major objections 4 minor 196 references

Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMs

T0 review · 5 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Training-free low-rank compression of LLMs silently drifts from its own calibration estimates as layers are compressed, and the paper shows that two no-training corrections—refreshing calibration activations layer by layer and re-running…

desk verdict Useful empirical study of two real misalignment effects in training-free low-rank compression, but the headline C+R gains rest on per-cell hyperparameter selection on the test benchmarks; the C-only average-gap result is more solid but more modest. read the letter →

arxiv 2608.08506 v1 pith:G2CDL4YC submitted 2026-08-09 cs.AI cs.CLcs.PF

Mohanad Odema , Gabrielle De Micheli , Dayin Gou , Nilesh Malpeddi , Prathamesh Vaste , Jacob Song This is my paper · ORCID
classification cs.AIcs.CLcs.PF
keywords low-rankcompressiontraining-freeLLMcalibrationcorrectionrankallocationtruncationerrorBlockInfluencescorezero-shotaccuracy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that training-free low-rank compression of large language models quietly invalidates its own assumptions as it runs: residual errors in the calibration activations that guide each layer's decomposition accumulate and grow deeper into the model, and the layer-importance scores used to allocate ranks shift once weights are compressed. It proposes two corrections that require no training: recompute calibration activations layer by layer through the freshly compressed layers, and rerun the rank allocation for a few rounds, nudging each layer's rank ratio by the drift in its Block Influence score. Implemented on top of an existing joint-decomposition compressor and tested on eight Llama and Qwen3 models at 15, 30, and 40 percent compression, the corrections raise zero-shot accuracy by about 1 to 2.5 points over the per-weight and joint baselines and cut the average gap to the best method from 1.22 and 1.71 points down to 0.50 points. If right, this means existing training-free compressors can be made more accurate for free at inference time, with little added compression cost.

What carries the argument

The load-bearing mechanism is a revised compression loop. In the standard pipeline, calibration activations and per-layer rank ratios are collected once on the uncompressed model; in Algorithm 1 of the paper, the two are refreshed inside the compression pass. The calibration correction replaces the original activation $X$ in the reconstruction objective $O=\min\|WX-\hat{W}\hat{X}\|^2$ with the activation $\hat{X}$ produced by the already-compressed earlier layers, eliminating layer-by-layer error accumulation by construction. The rank-allocation correction recomputes Block Influence scores (the complement of the cosine similarity between a layer's input and output activations) on the compressed model and adjusts each layer's rank ratio with the delta update $\hat{\phi}_i^n = \phi_i + \mathrm{Pr}(\alpha\Delta_i^n, r_{\text{target}})$, where $\Delta_i^n$ is the normalized BI drift for layer $i$ in round $n$ and $\mathrm{Pr}$ projects the scores back onto the global retention budget. The two mechanisms are decoupled: calibration correction works at the level of the forward activations, rank correction at the level of the layer-wise budget.

What would settle it

Compute two rank allocations for one model at 40% compression: one from the paper's BI-drift rule and one from a sensitivity oracle such as leave-one-layer-out accuracy loss or per-layer Fisher information. If the BI-drift allocation is not consistently closer in accuracy to the oracle allocation than the original static allocation, the central claim fails. A cheaper signal: on Qwen3-1.7B, where the paper's own table shows the combined corrections below the baseline at 40%, check whether the reallocated ranks move budget into layers whose BI drift is positive but whose oracle sensitivity is flat.

Watch

Extended reading notes

Core claim

The central discovery is that the two misalignment channels share one root cause: training-free low-rank compression treats the original model as a static reference, while in fact every compression step changes the signals on which later steps depend. Calibration residuals grow with depth, reaching normalized mean squared errors up to 0.27 at the final layer, because each layer's decomposition is fit to activations from the uncompressed model rather than to the activations the compressed prefix will actually produce. Layer importance also drifts, with BI scores shifting by up to 0.17 after compression, so the precomputed rank-ratio vector no longer matches the compressed model. The paper's fixes update the calibration activations on the fly by passing data through each just-compressed layer, and refine the rank ratios through a damped delta rule over a few compression rounds. On the evaluated models, the combined corrections achieve 18 wins, 3 ties, and 3 losses against the joint-decomposition baseline, and 18 wins and 6 losses against the modular baseline.

Load-bearing premise

Everything rests on the assumption that the drift of Block Influence scores, measured on 128 calibration samples, faithfully reveals which layers lost importance after compression; if a layer's BI drift points the wrong way, the rank reallocation can hurt accuracy.

Editorial extensions

If this is right

  • Applying corrective calibration to an existing joint-decomposition compressor adds no inference cost and roughly 1.5–1.8x the compression time of the base joint method, yet remains 2.5–5x faster than the modular pseudo-inverse baseline.
  • Across all configurations, the average gap from the best-performing method drops to 0.50 percentage points from 1.22 for the joint baseline and 1.71 for the modular baseline, and the combined correction wins or ties 21 of 24 comparisons against the joint baseline.
  • The corrections transfer across model families and sizes: Llama models benefit most, and a 32B model improves through rank correction alone even when calibration correction alone does not help.
  • The corrections are complementary to post-hoc error compensation: adding a block-level residual path improves all tested configurations by 0.41 points on average and does not change which compression configuration is best.
  • At aggressive 40% compression, the corrected method achieves the best zero-shot scores across all tested Llama models, extending the accuracy–compression trade-off frontier beyond the baselines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural untested extension is to swap the Block Influence score for a different sensitivity oracle, such as Fisher information or leave-one-layer-out accuracy loss; if the accuracy gains persist, the correction comes from the iteration scheme itself, not from the specific importance metric.
  • Because the delta rule's dampening coefficient α behaves like a learning rate, annealing α across rounds or selecting it with a quick holdout estimate is a plausible improvement over the fixed α ∈ {0.01, 0.05} explored here.
  • The 0.50 pp average-gap number is relative to the three baselines included in this paper; other training-free compressors not in the comparison could set a different best-oracle, so the absolute claim is benchmark-relative.
  • For small Qwen3 models at aggressive 40% compression, the combined correction occasionally trails the base method, suggesting the rank-delta update can move budget in the wrong direction when BI drift is noisy; a signed or heavier-tailed update rule would be worth testing.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper studies two sources of misalignment in training-free low-rank LLM compression: residual calibration-data activation errors that accumulate across layers, and drift in layer importance scores after compression. It proposes two training-free corrections—layer-by-layer calibration correction (C) and iterative rank-allocation correction (R)—and implements them on top of the UniQL joint-decomposition framework. Evaluation across eight Llama and Qwen3 models at 15%, 30%, and 40% compression rates on five zero-shot LM-Eval tasks is reported, with claims of up to 1–2.5 percentage-point accuracy improvements and an average gap from the best method of 0.50pp versus 1.22pp for UniQL and 1.71pp for MoDeGPT. The paper also includes ablations of the two corrections, repeated-seed variance, a Qwen2.5-32B scaling experiment, and a compatibility study with the post-hoc correction method EoRA.

Significance. If the two corrections are as effective as claimed, the paper would make a useful, simple, and training-free contribution to LLM compression, since the corrections are compatible with existing joint-decomposition pipelines and add no inference cost. The paper's strengths include a broad evaluation across model families and compression rates, per-benchmark breakdowns, wall-clock compression-time comparisons, an explicit compatibility study with EoRA, and a candid limitations section. The aggregate average-gap metric is favorable for the calibration correction alone. However, the headline claims for the combined C+R method are weakened by per-cell hyperparameter selection on the evaluation benchmarks, a mismatch between the reported average-gap figure and the claims in the introduction, and a 32B experiment that contradicts the generality of the calibration-correction mechanism. These issues bear directly on the central claim and require substantive revision.

major comments (5)
  1. [Section 4.4, Table 8 caption, Appendix C.5 (Tables 10–11)] The C+R results in Table 1 appear to be selected per cell as the best configuration over α and N on the same LM-Eval benchmarks used for evaluation. Table 8's caption states that the (C+R) scores are for the best LM-Eval configuration from Table 1, and Section 4.4 describes the choice of α and N as 'quintessential.' Appendix C.5 quantifies the fragility: at N=1, α=0.05 gives mean changes of −0.20pp for Llama-3.2-1B/3B and −0.07pp for the Qwen3-4B pair, while α=0.01 gives only +0.01pp on the small Llama group. A fixed or validation-selected hyperparameter can therefore erase or reverse the R gains, and the 18/3/3 W/T/L record versus UniQL plus the 'up to 1–2.5pp' improvements are not evidence for a single robust training-free correction. The paper needs held-out hyperparameter selection, or a fixed configuration report, before the combined C+R claim is supported.
  2. [Figure 3 caption, Section 4.3, and introduction contribution bullet] The abstract and the contribution bullet attribute the 0.50pp average gap from the best method to 'our approach' with both corrections, but Figure 3 explicitly states that 'Ours here indicates after calibration correction (C)' and Section 4.3 confirms that the Average Gap analysis focuses on the calibration correction (C) alone. The combined C+R average gap is not reported. This is a mismatch between the headline summary statistic and the actual computation, and it overstates what the evidence supports. The introduction and Section 6 should attribute the 0.50pp figure to C only, or the combined C+R average gap should be computed and reported.
  3. [Section 4.5, Table 4] The Qwen2.5-32B scaling experiment contradicts the calibration-correction claim. Relative to UniQL at 40% compression, Ours (C) is worse on WikiText-2 PPL (24.16 vs. 20.50), 0-shot LM-Eval average (57.42 vs. 59.84), and MMLU (27.20 vs. 32.40). The text states that 'pure corrective calibration (C) does not lead to performance improvements' for larger models. This undermines the mechanism claim in Sections 1 and 4.3 that residual calibration error accumulation is a general source of misalignment that the C correction mitigates. The authors should present the 32B result as an explicit boundary condition and discuss what it implies for the claimed universality of the calibration-correction mechanism.
  4. [Table 3] The repeated-seed experiment does not provide statistical support for the claim that 'the findings hold.' For Llama-3.2-1B at 40% compression, UniQL is 41.78±0.48 and Ours (C+R) is 41.93±0.55, fully overlapping within one standard deviation; for Llama-3.2-3B, the two methods differ by 0.77pp with standard deviations of 0.63 and 0.45 at n=3, which is not significant without a proper test. The paper reports single-run results throughout the main tables, and the repeated-seed subsection is the only place where variance is quantified; as it stands, it does not demonstrate that the reported improvements are robust.
  5. [Section 3.1, Table 9, and Algorithm 1] The diagnostic evidence for the calibration correction is partly tautological. The NMSE reduction in Table 9 and the BI-score convergence in Algorithm 1 are computed on the same 128-sample calibration set whose activations the correction updates at each layer; feeding compressed-model activations back into the compression loop will, by construction, reduce calibration-set NMSE relative to a one-pass method. This does not by itself establish that the correction reduces true inference-time error, which is an assumption (stated in the paper's framing). The held-out LM-Eval accuracy numbers are not circular, but the paper's mechanistic interpretation should be validated on held-out activations or explicitly tied to the held-out task improvements rather than presented as direct evidence of error propagation being fixed.
minor comments (4)
  1. [Equation (9)] Equation (9) has unbalanced parentheses in the second term: 'f(|| ˆX; ˆW1,· · ·ˆWk)||)2' appears to be missing a closing parenthesis for the norm or for f(·). Please fix the notation.
  2. [Figure 3] The caption of Figure 3 says 'Ours here indicates after calibration correction (C),' but the legend and text sometimes refer to 'Ours' without the '(C)' qualifier. Consider relabeling the legend to 'Ours (C)' for consistency with Table 1.
  3. [Section 4.2] The bullet list in Section 4.2 has formatting issues: 'Model Family.Llama-3.2 models...' should have a space and consistent punctuation after the bolded item labels. This is a minor presentation issue.
  4. [Section 4.5 and Appendix E] The paper does not state whether code or checkpoints will be released. Given the empirical nature of the work and the claim of compatibility with existing frameworks, a statement about code release would improve reproducibility.

Circularity Check

1 steps flagged · score 6.0 of 10

Table 1's C+R gains are per-cell best configurations selected on the same LM-Eval benchmarks, making the headline 'improvements' fitted maxima rather than fixed-method predictions.

  1. fitted input called prediction [Section 4.4, Table 8 caption, Appendix C.5 (Tables 10–11)]
    "We observe the choice of α and N is quintessential to maximizing performance improvement gains over the baselines. ... The (C+R) scores are for the best LM-Eval configuration from Table 1."

    The reported Ours (C+R) entries in Table 1 are not outputs of a single fixed correction. Per the Table 8 caption, each cell is the best α/N configuration selected on the same five LM-Eval benchmarks that define the reported improvements. Section 4.4 admits that α and N must be chosen per model, size, and compression rate to maximize gains. Appendix C.5 quantifies the fragility: at N=1, mean LM-Avg changes are −0.10pp for small Llama models, −0.07pp for small Qwen3 models, and +0.05pp for larger models across α, so the +0.29pp, +0.39pp, and +0.72pp examples and the 18/3/3 W/T/L record are selected maxima over a test-set grid, not predictions of a fixed training-free method. The central empirical claim is therefore fitted to the evaluation set and statistically forced.

full rationale

The derivation of the two corrections is otherwise self-contained: Algorithm 1 and Eqs. (8)–(11) define the calibration and rank-allocation updates from calibration activations and BI scores, not from evaluation labels, and there is no load-bearing self-citation chain or imported uniqueness theorem. The calibration correction alone also has independent support (Figure 3's 0.50pp average gap is explicitly labeled 'Ours (C)'). The main circularity is in the evaluation protocol for the combined C+R claim: hyperparameters α and N are selected per cell on the same LM-Eval suite, then the best configuration is reported as the method's accuracy, converting a grid-search maximum into a headline improvement. The NMSE and BI-drift diagnostics are similarly in-sample (computed on the calibration set used to drive the corrections), so they are supporting analyses rather than held-out evidence. Overall, the core mechanism is not definitionally circular, but the central C+R accuracy claim reduces to a per-cell fit over evaluation outcomes, warranting a partial-circularity score of 6.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The main method introduces two tunable hyperparameters (alpha, N) and inherits epsilon as an unspecified convergence threshold; the EoRA synergy experiment adds eta and a block rank. No new physical or conceptual entities are postulated. The correctness of the R correction rests on the BI-score assumption, and the correctness of the C correction rests on the representativeness of 128 calibration samples. The central accuracy claims are measured on external benchmarks, so the ledger is moderate rather than empty.

free parameters (5)
  • alpha (dampening coefficient) = 0.01 or 0.05, selected per configuration
    Controls how much drifted BI scores change rank ratios in Eq. 10; the paper reports that best value depends on model family, size, and compression rate (Section 4.4), and Appendix C.5 recommends grid search, meaning it is tuned to the evaluation tasks.
  • N (number of compression rounds) = 1 to 3; best N chosen per configuration
    Algorithm 1 iterates up to N rounds; Figures 4-5 and Appendix C.5 show gains depend on N and the best N varies, so it is selected from evaluation results.
  • eta (EoRA dampening) = 0.1
    Introduced in Appendix E to mitigate overcorrection when stacking EoRA; not part of main method but a hand-set coefficient.
  • EoRA block rank = 256
    Chosen for the synergy experiment in Appendix E; equals memory comparable to rank-64 EoRA in the original per-weight formulation.
  • epsilon convergence threshold = not reported
    Algorithm 1 line 8 exits when DIV(phi, phi-hat) < epsilon, but epsilon is never quantified, leaving the stopping rule underspecified.
assumptions (4)
  • domain assumption BI score (1 - cosine similarity between layer input and output) is a valid measure of layer importance for rank allocation.
    Eq. 6 defines the score; Eq. 7 turns it into rank ratios. The R correction assumes drifted BI scores capture which layers need more rank after compression. Appendix G admits this is conditioned on the scoring mechanism.
  • domain assumption 128 Wikitext-2 and 128 alpaca samples provide calibration signals representative of the deployment distribution.
    Section 4.1 follows UniQL defaults; if calibration samples are unrepresentative, covariance estimates and NMSE diagnostics are misleading. Repeated-seed PPL variance (223.01 plus or minus 12.25 in Table 3) suggests sensitivity.
  • ad hoc to paper Layer-by-layer compression with compressed-model activations reduces true inference-time error, not just calibration-set error.
    Algorithm 1 line 6 passes compressed activations to the next layer; this ensures low calibration NMSE by construction, but the paper does not show the same reduction on held-out activations, and Qwen2.5-32B degrades (Table 4).
  • standard math Truncated SVD reconstruction minimizes output reconstruction error in the activation norm sense.
    Eq. 3 and all baselines rely on standard low-rank decomposition properties; this is accepted background math.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMs." pith.science (2026). https://pith.science/paper/G2CDL4YC

@misc{pith2026260808506,
  author       = {Pith},
  title        = {Pith review of: Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G2CDL4YC}},
  note         = {Machine review of arXiv:2608.08506}
}
read the original abstract

Training-free low-rank compression frameworks have been gaining prominence for LLM compression given their effectiveness in reducing model parameter count while maintaining task-level accuracy. However, existing SOTA frameworks share two key limitations: (1) residual errors in calibration data activations accumulate across layers during compression, causing misalignment between representations simulated at compression time and those experienced at inference; (2) the assumption that layer importance distribution is preserved post-compression does not hold. Together, these two effects introduce misalignment in the compression process in relation to the deployed model. We study these effects and propose a simple, training-free methodology compatible with existing frameworks to mitigate them, comprising: (1) Layer-by-Layer Compression with Calibration Correction; (2) Iterative Compression with Rank Allocation Correction. Implemented atop an existing SOTA decomposition framework, and evaluated on Llama and Qwen3 models across various benchmarks and compression rates, our approach demonstrates up to ~1-2.5 accuracy point improvements over per-weight and joint decomposition baselines on zero-shot tasks.

Figures

Figures reproduced from arXiv: 2608.08506 by the authors.

Figure 1
Figure 1. Calibration Error Residual Effects evaluated through NMSE ( [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. BI Score Divergence pre and post compression for Llama-3.1-8B and Llama-3.2-1B. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Average Gap from the Best for the different compression methods (Lower is better). [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Avg. 0-shot accuracy for Llama-3.1-8B post BI scores correction (Higher is better). [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Avg. 0-shot accuracy for Llama-3.2-1B post BI-scores correction (Higher is better). [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Residual Calibration Errors in Llama models measured through per-layer NMSE [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Residual Calibration Errors in Qwen3 models measured through per-layer NMSE [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: BI Score Divergence pre and post compression for Qwen3-{1.7B, 4B, 4B-Inst-2507} [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: BI Score Divergence pre and post compression for Llama-{3.2-3b, 2-7B}, Qwen3-8B [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: Average Gap from Best Analysis aggregated for every model (Lower is Better) [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Avg. 0-shot scores for Llama-3.2-3B post BI-scores correction (Higher is better). [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 12
Figure 12. Figure 12: Avg. 0-shot scores for Llama-2-7B post BI-scores correction (Higher is better). [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]
Figure 13
Figure 13. Figure 13: Avg. 0-shot scores for Qwen3-1.7B post BI-scores correction (Higher is better). [PITH_FULL_IMAGE:figures/full_fig_p020_13.png]
Figure 14
Figure 14. Figure 14: Avg. 0-shot scores for Qwen3-4B post BI-scores correction (Higher is better). [PITH_FULL_IMAGE:figures/full_fig_p020_14.png]
Figure 15
Figure 15. Figure 15: Avg. 0-shot scores for Qwen3-4B-Instruct-2507 post BI-scores correction (Higher [PITH_FULL_IMAGE:figures/full_fig_p020_15.png]
Figure 16
Figure 16. Figure 16: Avg. 0-shot Scores for Qwen3-8B post BI-scores correction (Higher is better). [PITH_FULL_IMAGE:figures/full_fig_p021_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

196 extracted references · 25 canonical work pages

  1. [1]

    doi:10.5281/zenodo.10256836 , url =

    Gao, Leo and Tow, Jonathan and Abbasi, Baber and Biderman, Stella and Black, Sid and DiPofi, Anthony and Foster, Charles and Golding, Laurence and Hsu, Jeffrey and Le Noac'h, Alain and Li, Haonan and McDonell, Kyle and Muennighoff, Niklas and Ociepa, Chris and Phang, Jason and Reynolds, Laria and Schoelkopf, Hailey and Skowron, Aviya and Sutawika, Lintang...

    doi:10.5281/zenodo.10256836

  2. [2]

    arXiv preprint arXiv:2407.21783 , year=

    The llama 3 herd of models , author=. arXiv preprint arXiv:2407.21783 , year=

    arXiv

  3. [3]

    arXiv preprint arXiv:2307.09288 , year=

    Llama 2: Open foundation and fine-tuned chat models , author=. arXiv preprint arXiv:2307.09288 , year=

    arXiv

  4. [4]

    arXiv preprint arXiv:2303.08774 , year=

    Gpt-4 technical report , author=. arXiv preprint arXiv:2303.08774 , year=

    arXiv

  5. [5]

    2023 , note =

    LLaMA: Open and Efficient Foundation Language Models , author =. 2023 , note =

    2023

  6. [6]

    arXiv preprint arXiv:2312.11805 , year =

    Gemini: A Family of Highly Capable Multimodal Models , author =. arXiv preprint arXiv:2312.11805 , year =

    arXiv

  7. [7]

    arXiv preprint arXiv:2403.05530 , year=

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context , author=. arXiv preprint arXiv:2403.05530 , year=

    arXiv

  8. [9]

    arXiv preprint arXiv:2505.09388 , year=

    Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=

    arXiv

Show all 196 references
  1. [10]

    2025 , month = aug, day =

    Introducing gpt-oss , howpublished =. 2025 , month = aug, day =

    2025

  2. [11]

    Advances in neural information processing systems , volume=

    Attention is all you need , author=. Advances in neural information processing systems , volume=

  3. [12]

    arXiv preprint arXiv:1810.04805 , year =

    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , author =. arXiv preprint arXiv:1810.04805 , year =

    arXiv

  4. [13]

    Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL) , year =

    BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension , author =. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL) , year =

  5. [14]

    Neurocomputing , volume=

    Roformer: Enhanced transformer with rotary position embedding , author=. Neurocomputing , volume=. 2024 , publisher=

    2024

  6. [15]

    Neurocomputing: foundations of research , pages=

    Adaptive switching circuits , author=. Neurocomputing: foundations of research , pages=

  7. [16]

    International conference on machine learning , pages=

    Linear transformers are secretly fast weight programmers , author=. International conference on machine learning , pages=. 2021 , organization=

    2021

  8. [17]

    The Thirteenth International Conference on Learning Representations , year=

    Gated Delta Networks: Improving Mamba2 with Delta Rule , author=. The Thirteenth International Conference on Learning Representations , year=

  9. [18]

    OpenAI , year =

    Language Models are Unsupervised Multitask Learners , author =. OpenAI , year =

  10. [19]

    arXiv preprint arXiv:2005.14165 , year =

    Language Models are Few-Shot Learners , author =. arXiv preprint arXiv:2005.14165 , year =

    2005 arXiv

  11. [20]

    arXiv preprint arXiv:2403.08295 , year =

    Gemma: Open Models Based on Gemini Research and Technology , author =. arXiv preprint arXiv:2403.08295 , year =

    arXiv

  12. [21]

    arXiv preprint arXiv:2503.19786 , year =

    Gemma 3 Technical Report , author =. arXiv preprint arXiv:2503.19786 , year =

    arXiv

  13. [22]

    arXiv preprint arXiv:2501.12948 , year =

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs , author =. arXiv preprint arXiv:2501.12948 , year =

    arXiv

  14. [23]

    2024 , howpublished =

    LLaMA 3.2 Model Card , author =. 2024 , howpublished =

    2024

  15. [24]

    2025 , howpublished =

    Qwen2.5-72B , author =. 2025 , howpublished =

    2025

  16. [25]

    2024 , howpublished =

    GPT-OSS: OpenAI Open-Weight Reasoning Models (20B/120B) , author =. 2024 , howpublished =

    2024

  17. [26]

    2025 , howpublished =

    Gemini 2.5: Our most intelligent AI model , author =. 2025 , howpublished =

    2025

  18. [27]

    Proceedings of the International Conference on Machine Learning (ICML) , year =

    BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models , author =. Proceedings of the International Conference on Machine Learning (ICML) , year =

  19. [28]

    2022 , eprint=

    Flamingo: a Visual Language Model for Few-Shot Learning , author=. 2022 , eprint=

    2022

  20. [29]

    Proceedings of the Conference of the European Chapter of the Association for Computational Linguistics (EACL) , year =

    Recipes for Building an Open-Domain Chatbot , author =. Proceedings of the Conference of the European Chapter of the Association for Computational Linguistics (EACL) , year =

  21. [30]

    arXiv preprint , year =

    LaMDA: Language Models for Dialog Applications , author =. arXiv preprint , year =

  22. [31]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    Unified Scaling Laws for Compressed Representations , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  23. [32]

    IEEE Transactions on Signal Processing , volume=

    The geometry of weighted low-rank approximations , author=. IEEE Transactions on Signal Processing , volume=. 2003 , publisher=

    2003

  24. [33]

    Automatica , volume=

    Structured low-rank approximation and its applications , author=. Automatica , volume=. 2008 , publisher=

    2008

  25. [34]

    Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Structured pruning learns compact and accurate models , author=. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  26. [35]

    2023 , url=

    Xinyin Ma and Gongfan Fang and Xinchao Wang , booktitle=. 2023 , url=

    2023

  27. [36]

    Tycho F. A. van der Ouderaa and Markus Nagel and Mart Van Baalen and Tijmen Blankevoort , booktitle=. The. 2024 , url=

    2024

  28. [37]

    Croci and Marcelo Gennari do Nascimento and Torsten Hoefler and James Hensman , booktitle=

    Saleh Ashkboos and Maximilian L. Croci and Marcelo Gennari do Nascimento and Torsten Hoefler and James Hensman , booktitle=. Slice. 2024 , url=

    2024

  29. [38]

    IEEE international conference on neural networks , pages=

    Optimal brain surgeon and general network pruning , author=. IEEE international conference on neural networks , pages=. 1993 , organization=

    1993

  30. [39]

    Advances in neural information processing systems , volume=

    Optimal brain damage , author=. Advances in neural information processing systems , volume=

  31. [40]

    Advances in Neural Information Processing Systems , volume=

    Woodfisher: Efficient second-order approximation for neural network compression , author=. Advances in Neural Information Processing Systems , volume=

  32. [41]

    International conference on machine learning , pages=

    Sparsegpt: Massive language models can be accurately pruned in one-shot , author=. International conference on machine learning , pages=. 2023 , organization=

    2023

  33. [42]

    arXiv preprint arXiv:2509.11177 , year=

    Optimal brain restoration for joint quantization and sparsification of llms , author=. arXiv preprint arXiv:2509.11177 , year=

  34. [43]

    arXiv preprint arXiv:2410.21271 , year=

    EoRA: Fine-tuning-free Compensation for Compressed LLM with Eigenspace Low-Rank Approximation , author=. arXiv preprint arXiv:2410.21271 , year=

  35. [44]

    The Thirteenth International Conference on Learning Representations , year=

    You Only Prune Once: Designing Calibration-Free Model Compression With Policy Learning , author=. The Thirteenth International Conference on Learning Representations , year=

  36. [45]

    T\'yr-the-Pruner: Structural Pruning

    Guanchen Li and Yixing Xu and Zeping Li and Ji Liu and Xuanwu Yin and Dong Li and Emad Barsoum , booktitle=. T\'yr-the-Pruner: Structural Pruning. 2025 , url=

    2025

  37. [46]

    Computation and Memory-Efficient Model Compression with Gradient Reweighting , author=

  38. [47]

    Forty-second International Conference on Machine Learning , year=

    Let llm tell what to prune and how much to prune , author=. Forty-second International Conference on Machine Learning , year=

  39. [48]

    arXiv preprint arXiv:2505.22689 , year=

    SlimLLM: Accurate structured pruning for large language models , author=. arXiv preprint arXiv:2505.22689 , year=

    arXiv

  40. [49]

    arXiv preprint arXiv:2505.23807 , year=

    Dlp: Dynamic layerwise pruning in large language models , author=. arXiv preprint arXiv:2505.23807 , year=

    arXiv

  41. [50]

    Ruokai Yin and Yuhang Li and Donghyun Lee and Priyadarshini Panda , booktitle=. Duo. 2025 , url=

    2025

  42. [51]

    Forty-second International Conference on Machine Learning , year=

    Targeted Low-rank Refinement: Enhancing Sparse Language Models with Precision , author=. Forty-second International Conference on Machine Learning , year=

  43. [52]

    arXiv preprint arXiv:2507.01299 , year=

    La RoSA: Enhancing LLM Efficiency via Layerwise Rotated Sparse Activation , author=. arXiv preprint arXiv:2507.01299 , year=

  44. [53]

    arXiv preprint arXiv:2410.14649 , year=

    Evopress: Accurate dynamic model compression via evolutionary search , author=. arXiv preprint arXiv:2410.14649 , year=

    arXiv

  45. [54]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Shortgpt: Layers in large language models are more redundant than you expect , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  46. [55]

    Adaptive Layer-skipping in Pre-trained

    Xuan Luo and Weizhi Wang and Xifeng Yan , booktitle=. Adaptive Layer-skipping in Pre-trained. 2025 , url=

    2025

  47. [56]

    Filipe Laitenberger and Dawid Jan Kopiczko and Cees G. M. Snoek and Yuki M Asano , booktitle=. What Layers When: Learning to Skip Compute in. 2026 , url=

    2026

  48. [57]

    The Fourteenth International Conference on Learning Representations , year=

    Rethinking Layer Relevance in Large Language Models Beyond Cosine Similarity , author=. The Fourteenth International Conference on Learning Representations , year=

  49. [58]

    2025 , url=

    Ruisi Cai and Saurav Muralidharan and Hongxu Yin and Zhangyang Wang and Jan Kautz and Pavlo Molchanov , booktitle=. 2025 , url=

    2025

  50. [59]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    Layer as Puzzle Pieces: Compressing Large Language Models through Layer Concatenation , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  51. [60]

    DenoiseRotator: Enhance Pruning Robustness for

    Tianteng Gu and Bei Liu and Bo Xiao and Ke Zeng and Jiacheng Liu and Yanmin Qian , booktitle=. DenoiseRotator: Enhance Pruning Robustness for. 2025 , url=

    2025

  52. [61]

    Forty-second International Conference on Machine Learning , year=

    Determining Layer-wise Sparsity for Large Language Models Through a Theoretical Perspective , author=. Forty-second International Conference on Machine Learning , year=

  53. [62]

    2025 , url=

    Mohammad Mozaffari and Amir Yazdanbakhsh and Zhao Zhang and Maryam Mehri Dehnavi , booktitle=. 2025 , url=

    2025

  54. [63]

    The Thirteenth International Conference on Learning Representations , year=

    Forget the Data and Fine-Tuning! Just Fold the Network to Compress , author=. The Thirteenth International Conference on Learning Representations , year=

  55. [64]

    The Fourteenth International Conference on Learning Representations , year=

    Cut Less, Fold More: Model Compression through the Lens of Projection Geometry , author=. The Fourteenth International Conference on Learning Representations , year=

  56. [65]

    Forty-second International Conference on Machine Learning , year=

    Sparsing Law: Towards Large Language Models with Greater Activation Sparsity , author=. Forty-second International Conference on Machine Learning , year=

  57. [66]

    The Thirteenth International Conference on Learning Representations , year=

    Training-Free Activation Sparsity in Large Language Models , author=. The Thirteenth International Conference on Learning Representations , year=

  58. [67]

    2025 , url=

    Donghyeon Joo and Helya Hosseini and Ramyad Hadidi and Bahar Asgari , booktitle=. 2025 , url=

    2025

  59. [68]

    Improving Model Representation and Reducing

    Zhoutong Wu and Yuan Zhang and Yiming Dong and Chenheng Zhang and Cong Fang and Kun Yuan and Zhouchen Lin , booktitle=. Improving Model Representation and Reducing. 2025 , url=

    2025

  60. [69]

    Probe Pruning: Accelerating

    Qi Le and Enmao Diao and Ziyan Wang and Xinran Wang and Jie Ding and Li Yang and Ali Anwar , booktitle=. Probe Pruning: Accelerating. 2025 , url=

    2025

  61. [70]

    2025 , url=

    Stephen Zhang and Vardan Papyan , booktitle=. 2025 , url=

    2025

  62. [71]

    The Unseen Frontier: Pushing the Limits of

    Kwanhee Lee and Hyeondo Jang and Dongyeop Lee and Dan Alistarh and Namhoon Lee , booktitle=. The Unseen Frontier: Pushing the Limits of. 2026 , url=

    2026

  63. [72]

    Yang , booktitle=

    Lawrence Liu and Alexander Liu and Mengdi Wang and Tuo Zhao and Lin F. Yang , booktitle=. 2026 , url=

    2026

  64. [73]

    arXiv preprint arXiv:2312.05821 , year=

    Asvd: Activation-aware singular value decomposition for compressing large language models , author=. arXiv preprint arXiv:2312.05821 , year=

    arXiv

  65. [74]

    2025 , url=

    Xin Wang and Yu Zheng and Zhongwei Wan and Mi Zhang , booktitle=. 2025 , url=

    2025

  66. [75]

    Chi-Heng Lin and Shangqian Gao and James Seale Smith and Abhishek Patel and Shikhar Tuli and Yilin Shen and Hongxia Jin and Yen-Chang Hsu , booktitle=. MoDe. 2025 , url=

    2025

  67. [76]

    arXiv preprint arXiv:2505.23966 , year=

    Flat-llm: Fine-grained low-rank activation space transformation for large language model compression , author=. arXiv preprint arXiv:2505.23966 , year=

  68. [77]

    Abdelfattah and Diana Marculescu , booktitle=

    Hung-Yueh Chiang and Chi-Chih Chang and Yu-Chen Lu and Chien-Yu Lin and Kai-Chiang Wu and Mohamed S. Abdelfattah and Diana Marculescu , booktitle=. Uni. 2026 , url=

    2026

  69. [78]

    2026 , url=

    Xing Hu and Zukang Xu and Zhixuan Chen and Yuan Cheng and Dawei Yang , booktitle=. 2026 , url=

    2026

  70. [79]

    arXiv preprint arXiv:2505.20112 , year=

    Ressvd: Residual compensated svd for large language model compression , author=. arXiv preprint arXiv:2505.20112 , year=

  71. [80]

    2025 , url=

    Mohammad Mozaffari and Amir Yazdanbakhsh and Maryam Mehri Dehnavi , booktitle=. 2025 , url=

    2025

  72. [81]

    arXiv preprint arXiv:2506.08436 , year=

    Olica: Efficient structured pruning of large language models without retraining , author=. arXiv preprint arXiv:2506.08436 , year=

    arXiv

  73. [82]

    arXiv preprint arXiv:2505.18413 , year=

    Latentllm: Attention-aware joint tensor compression , author=. arXiv preprint arXiv:2505.18413 , year=

    arXiv

  74. [83]

    arXiv preprint arXiv:2502.01403 , year=

    Adasvd: Adaptive singular value decomposition for large language models , author=. arXiv preprint arXiv:2502.01403 , year=

  75. [84]

    The Thirteenth International Conference on Learning Representations , year=

    Basis Sharing: Cross-Layer Parameter Sharing for Large Language Model Compression , author=. The Thirteenth International Conference on Learning Representations , year=

  76. [85]

    arXiv preprint arXiv:2511.04494 , year=

    Distribution-Aware Tensor Decomposition for Compression of Convolutional Neural Networks , author=. arXiv preprint arXiv:2511.04494 , year=

  77. [86]

    arXiv preprint arXiv:2501.19090 , year=

    Pivoting factorization: A compact meta low-rank representation of sparsity for efficient inference in large language models , author=. arXiv preprint arXiv:2501.19090 , year=

    arXiv

  78. [87]

    arXiv preprint arXiv:2505.05086 , year=

    Beyond low-rank decomposition: A shortcut approach for efficient on-device learning , author=. arXiv preprint arXiv:2505.05086 , year=

    arXiv

  79. [88]

    Lee and Shengjie Sun and Wei Xue and Yike Guo , booktitle=

    Wei Li and Lujun Li and Hao Gu and You-Liang Huang and Mark G. Lee and Shengjie Sun and Wei Xue and Yike Guo , booktitle=. MoE-. 2025 , url=

    2025

  80. [89]

    Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    Moe-i2: Compressing mixture of experts models through inter-expert pruning and intra-expert low-rank decomposition , author=. Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    2024

  81. [90]

    Hanshi Sun and Li-Wen Chang and Wenlei Bao and Size Zheng and Ningxin Zheng and Xin Liu and Harry Dong and Yuejie Chi and Beidi Chen , booktitle=. Shadow. 2025 , url=

    2025

  82. [91]

    Xiaodong Chen and Mingming Ha and Zhenzhong Lan and Jing Zhang and Jianguo Li , booktitle=. Mo. 2026 , url=

    2026

  83. [92]

    2026 , url=

    Zukang Xu and Zhixiong Zhao and Zhixuan Chen and Xing Hu and Dawei Yang , booktitle=. 2026 , url=

    2026

  84. [93]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Sola: Leveraging soft activation sparsity and low-rank decomposition for large language model compression , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  85. [94]

    2025 , url=

    Yutong Wang and Haiyu Wang and Sai Qian Zhang , booktitle=. 2025 , url=

    2025

  86. [95]

    Fan Wang and Juyong Jiang and Chansung Park and Sunghun Kim and Jing Tang , booktitle=. Ka. 2025 , url=

    2025

  87. [96]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    Data Efficient Adaptation in Large Language Models via Continuous Low-Rank Fine-Tuning , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  88. [97]

    Distribution-Aligned Decoding for Efficient

    Senkang Hu and Xudong Han and Jinqi Jiang and Yihang Tao and Zihan Fang and Yong Dai and Sam Kwong and Yuguang Fang , booktitle=. Distribution-Aligned Decoding for Efficient. 2025 , url=

    2025

  89. [98]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    Compress Large Language Models via Collaboration Between Learning and Matrix Approximation , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  90. [99]

    2025 , url=

    Uliana Parkina and Maxim Rakhuba , booktitle=. 2025 , url=

    2025

  91. [100]

    Abdelfattah and Kai-Chiang Wu , booktitle=

    Chi-Chih Chang and Wei-Cheng Lin and Chien-Yu Lin and Chong-Yan Chen and Yu-Fang Hu and Pei-Shuo Wang and Ning-Chi Huang and Luis Ceze and Mohamed S. Abdelfattah and Kai-Chiang Wu , booktitle=. Palu:. 2025 , url=

    2025

  92. [101]

    Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , month =

    Li, Lujun and Lin, Cheng and Li, Dezhi and Huang, You-Liang and Li, Wei and Wu, Tianyu and Zou, Jie and Xue, Wei and Han, Sirui and Guo, Yike , title =. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , month =. 2025 , pages =

    2025

  93. [102]

    The Thirteenth International Conference on Learning Representations , year=

    BitStack: Any-Size Compression of Large Language Models in Variable Memory Environments , author=. The Thirteenth International Conference on Learning Representations , year=

  94. [103]

    2025 , url=

    Muyang Li and Yujun Lin and Zhekai Zhang and Tianle Cai and Junxian Guo and Xiuyu Li and Enze Xie and Chenlin Meng and Jun-Yan Zhu and Song Han , booktitle=. 2025 , url=

    2025

  95. [104]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops , month =

    Thoma, Moritz and Villasante, Jorge and Aghajanzadeh, Emad and Sampath, Shambhavi Balamuthu and Mori, Pierpaolo and Groetzinger, Maximilian and Dylkin, Daniil and Vemparala, Manoj-Rohit and Fasfous, Nael and Frickenstein, Alexander and Mueller-Gritschneder, Daniel and Schlicht...

    2025

  96. [105]

    in Post-Training Quantization , author=

    Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantization , author=. The Fourteenth International Conference on Learning Representations , year=

  97. [106]

    arXiv preprint arXiv:2406.13474 , year=

    Boa: Attention-aware post-training quantization without backpropagation , author=. arXiv preprint arXiv:2406.13474 , year=

    arXiv

  98. [107]

    The Fourteenth International Conference on Learning Representations , year=

    TurboBoA: Faster and Exact Attention-aware Quantization without Backpropagation , author=. The Fourteenth International Conference on Learning Representations , year=

  99. [108]

    arXiv preprint arXiv:2410.09426 , year=

    Flatquant: Flatness matters for llm quantization , author=. arXiv preprint arXiv:2410.09426 , year=

    arXiv

  100. [109]

    arXiv preprint arXiv:2511.04063 , year=

    DartQuant: Efficient Rotational Distribution Calibration for LLM Quantization , author=. arXiv preprint arXiv:2511.04063 , year=

  101. [110]

    arXiv preprint arXiv:2510.20673 , year=

    Efficient Multi-bit Quantization Network Training via Weight Bias Correction and Bit-wise Coreset Sampling , author=. arXiv preprint arXiv:2510.20673 , year=

  102. [111]

    arXiv preprint arXiv:2505.04877 , year=

    Learning from loss landscape: Generalizable mixed-precision quantization via adaptive sharpness-aware gradient aligning , author=. arXiv preprint arXiv:2505.04877 , year=

    arXiv

  103. [112]

    Forty-second International Conference on Machine Learning , year=

    Matryoshka Quantization , author=. Forty-second International Conference on Machine Learning , year=

  104. [113]

    any4: Learned 4-bit Numeric Representation for

    Mostafa Elhoushi and Jeff Johnson , booktitle=. any4: Learned 4-bit Numeric Representation for. 2025 , url=

    2025

  105. [114]

    arXiv preprint arXiv:2502.09720 , year=

    NestQuant: Nested lattice quantization for matrix products and LLMs , author=. arXiv preprint arXiv:2502.09720 , year=

    arXiv

  106. [115]

    Advances in Neural Information Processing Systems , volume=

    Duquant: Distributing outliers via dual transformation makes stronger quantized llms , author=. Advances in Neural Information Processing Systems , volume=

  107. [116]

    arXiv preprint arXiv:2502.04420 , year=

    Kvtuner: Sensitivity-aware layer-wise mixed-precision kv cache quantization for efficient and nearly lossless llm inference , author=. arXiv preprint arXiv:2502.04420 , year=

  108. [117]

    Mahoney and Kurt Keutzer and Amir Gholami , booktitle=

    Rishabh Tiwari and Haocheng Xi and Aditya Tomar and Coleman Richard Charles Hooper and Sehoon Kim and Maxwell Horton and Mahyar Najibi and Michael W. Mahoney and Kurt Keutzer and Amir Gholami , booktitle=. QuantSpec: Self-Speculative Decoding with Hierarchical Quantized. 2025 , url=

    2025

  109. [118]

    BlockDialect: Block-wise Fine-grained Mixed Format Quantization for Energy-Efficient

    Wonsuk Jang and Thierry Tambe , booktitle=. BlockDialect: Block-wise Fine-grained Mixed Format Quantization for Energy-Efficient. 2025 , url=

    2025

  110. [119]

    Forty-second International Conference on Machine Learning , year=

    ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residuals , author=. Forty-second International Conference on Machine Learning , year=

  111. [120]

    Wei Huang and Haotong Qin and Yangdong Liu and Yawei Li and Qinshuo Liu and Xianglong Liu and Luca Benini and Michele Magno and Shiming Zhang and XIAOJUAN QI , booktitle=. SliM-. 2025 , url=

    2025

  112. [121]

    2023 , url=

    Elias Frantar and Saleh Ashkboos and Torsten Hoefler and Dan Alistarh , booktitle=. 2023 , url=

    2023

  113. [122]

    2025 , url=

    Yuhang Li and Ruokai Yin and Donghyun Lee and Shiting Xiao and Priyadarshini Panda , booktitle=. 2025 , url=

    2025

  114. [123]

    Advances in neural information processing systems , volume=

    Quip: 2-bit quantization of large language models with guarantees , author=. Advances in neural information processing systems , volume=

  115. [124]

    SpinQuant:

    Zechun Liu and Changsheng Zhao and Igor Fedorov and Bilge Soran and Dhruv Choudhary and Raghuraman Krishnamoorthi and Vikas Chandra and Yuandong Tian and Tijmen Blankevoort , booktitle=. SpinQuant:. 2025 , url=

    2025

  116. [125]

    ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning

    Yesheng Liang and Haisheng Chen and Song Han and Zhijian Liu , booktitle=. ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning. 2026 , url=

    2026

  117. [126]

    The Thirteenth International Conference on Learning Representations , year=

    LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Grid , author=. The Thirteenth International Conference on Learning Representations , year=

  118. [127]

    Rasoul Shafipour and David Harrison and Maxwell Horton and JEFFREY MARKER and Houman Bedayat and Sachin Mehta and Mohammad Rastegari and Mahyar Najibi and Saman Naderiparizi , booktitle=. Seed. 2025 , url=

    2025

  119. [128]

    2025 , url=

    Xing Hu and Yuan Cheng and Dawei Yang and Zhixuan Chen and Zukang Xu and JiangyongYu and XUCHEN and Zhihang Yuan and Zhe jiang and Sifan Zhou , booktitle=. 2025 , url=

    2025

  120. [129]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  121. [130]

    arXiv preprint arXiv:2512.24124 , year=

    OptRot: Mitigating Weight Outliers via Data-Free Rotations for Post-Training Quantization , author=. arXiv preprint arXiv:2512.24124 , year=

  122. [131]

    The Thirteenth International Conference on Learning Representations , year=

    Effective Interplay between Sparsity and Quantization: From Theory to Practice , author=. The Thirteenth International Conference on Learning Representations , year=

  123. [132]

    2025 , url=

    Xin Ding and Xiaoyu Liu and Zhijun Tu and Yun Zhang and Wei Li and Jie Hu and Hanting Chen and Yehui Tang and Zhiwei Xiong and Baoqun Yin and Yunhe Wang , booktitle=. 2025 , url=

    2025

  124. [133]

    Rongguang Ye and Ming Tang and Edith C. H. Ngai , booktitle=. On-the-Fly Adaptation to Quantization: Configuration-Aware Lo. 2026 , url=

    2026

  125. [134]

    2026 , url=

    Yeonsik Park and Hyeonseong Kim and Seungkyu Choi , booktitle=. 2026 , url=

    2026

  126. [135]

    ParetoQ: Improving Scaling Laws in Extremely Low-bit

    Zechun Liu and Changsheng Zhao and Hanxian Huang and Sijia Chen and Jing Zhang and Jiawei Zhao and Scott Roy and Lisa Jin and Yunyang Xiong and Yangyang Shi and Lin Xiao and Yuandong Tian and Bilge Soran and Raghuraman Krishnamoorthi and Tijmen Blankevoort and Vikas Chandra , ...

    2025

  127. [136]

    Castro and Torsten Hoefler and Dan Alistarh , booktitle=

    Saleh Ashkboos and Mahdi Nikdan and Soroush Tabesh and Roberto L. Castro and Torsten Hoefler and Dan Alistarh , booktitle=. 2025 , url=

    2025

  128. [137]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    LittleBit: Ultra Low-Bit Quantization via Latent Factorization , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  129. [138]

    Forty-second International Conference on Machine Learning , year=

    An architecture search framework for inference-time techniques , author=. Forty-second International Conference on Machine Learning , year=

  130. [139]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    Per-Architecture Training-Free Metric Optimization for Neural Architecture Search , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  131. [140]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    Jet-Nemotron: Efficient Language Model with Post Neural Architecture Search , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  132. [141]

    Revolutionizing Training-Free

    Haidong Kang and Lihong Lin and Hanling Wang , booktitle=. Revolutionizing Training-Free. 2025 , url=

    2025

  133. [142]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    Self-Adapting Language Models , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  134. [143]

    2022 , note =

    NVIDIA Corporation , title =. 2022 , note =

    2022

  135. [144]

    , title =

    Qualcomm Technologies, Inc. , title =. 2023 , note =

    2023

  136. [145]

    2023 , note =

    Thundercomm Corporation , title =. 2023 , note =

    2023

  137. [146]

    arXiv preprint arXiv:2506.02153 , year=

    Small Language Models are the Future of Agentic AI , author=. arXiv preprint arXiv:2506.02153 , year=

    arXiv

  138. [147]

    2024 , howpublished =

    ChatRTX , author =. 2024 , howpublished =

    2024

  139. [148]

    , author=

    Multimodal deep learning. , author=. ICML , volume=

  140. [149]

    2025 , month =

    Context Rot: How Increasing Input Tokens Impacts LLM Performance , author =. 2025 , month =

    2025

  141. [150]

    2025 , month = aug, day =

    Introducing. 2025 , month = aug, day =

    2025

  142. [151]

    arXiv preprint arXiv:2004.05150 , year=

    Longformer: The long-document transformer , author=. arXiv preprint arXiv:2004.05150 , year=

    2004 arXiv

  143. [152]

    Big Bird: Transformers for Longer Sequences , url =

    Zaheer, Manzil and Guruganesh, Guru and Dubey, Kumar Avinava and Ainslie, Joshua and Alberti, Chris and Ontanon, Santiago and Pham, Philip and Ravula, Anirudh and Wang, Qifan and Yang, Li and Ahmed, Amr , booktitle =. Big Bird: Transformers for Longer Sequences , url =

  144. [153]

    International conference on machine learning , pages=

    Transformers are rnns: Fast autoregressive transformers with linear attention , author=. International conference on machine learning , pages=. 2020 , organization=

    2020

  145. [154]

    Advances in neural information processing systems , volume=

    Flashattention: Fast and memory-efficient exact attention with io-awareness , author=. Advances in neural information processing systems , volume=

  146. [155]

    arXiv preprint arXiv:2308.13191 , year=

    Chunk, align, select: A simple long-sequence processing method for transformers , author=. arXiv preprint arXiv:2308.13191 , year=

    arXiv

  147. [156]

    arXiv preprint arXiv:2402.17463 , year=

    Training-free long-context scaling of large language models , author=. arXiv preprint arXiv:2402.17463 , year=

    arXiv

  148. [157]

    Sensors , volume=

    Efficient memory-enhanced transformer for long-document summarization in low-resource regimes , author=. Sensors , volume=. 2023 , publisher=

    2023

  149. [158]

    2019 IEEE automatic speech recognition and understanding workshop (ASRU) , pages=

    Hierarchical transformers for long document classification , author=. 2019 IEEE automatic speech recognition and understanding workshop (ASRU) , pages=. 2019 , organization=

    2019

  150. [159]

    arXiv preprint arXiv:2406.10774 , year=

    Quest: Query-aware sparsity for efficient long-context llm inference , author=. arXiv preprint arXiv:2406.10774 , year=

    arXiv

  151. [160]

    Xiang Liu and Zhenheng Tang and Peijie Dong and Zeyu Li and Liuyue and Bo Li and Xuming Hu and Xiaowen Chu , booktitle=. Chunk. 2025 , url=

    2025

  152. [161]

    arXiv preprint arXiv:2410.10819 , year=

    Duoattention: Efficient long-context llm inference with retrieval and streaming heads , author=. arXiv preprint arXiv:2410.10819 , year=

    arXiv

  153. [162]

    arXiv preprint arXiv:2502.14866 , year=

    Lserve: Efficient long-sequence llm serving with unified sparse attention , author=. arXiv preprint arXiv:2502.14866 , year=

    arXiv

  154. [163]

    arXiv preprint arXiv:2503.08879 , year=

    LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inference , author=. arXiv preprint arXiv:2503.08879 , year=

    arXiv

  155. [164]

    arXiv preprint arXiv:2311.10122 , year=

    Video-llava: Learning united visual representation by alignment before projection , author=. arXiv preprint arXiv:2311.10122 , year=

    arXiv

  156. [165]

    arXiv preprint arXiv:2409.12191 , year=

    Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution , author=. arXiv preprint arXiv:2409.12191 , year=

    arXiv

  157. [166]

    arXiv preprint arXiv:2402.08268 , year=

    World model on million-length video and language with blockwise ringattention , author=. arXiv preprint arXiv:2402.08268 , year=

    arXiv

  158. [167]

    arXiv preprint arXiv:2309.00071 , year=

    Yarn: Efficient context window extension of large language models , author=. arXiv preprint arXiv:2309.00071 , year=

    arXiv

  159. [168]

    arXiv preprint arXiv:2402.13753 , year=

    Longrope: Extending llm context window beyond 2 million tokens , author=. arXiv preprint arXiv:2402.13753 , year=

    arXiv

  160. [169]

    2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) , pages=

    Spatten: Efficient sparse attention architecture with cascade token and head pruning , author=. 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) , pages=. 2021 , organization=

    2021

  161. [170]

    arXiv preprint arXiv:2503.16428 , year=

    Xattention: Block sparse attention with antidiagonal scoring , author=. arXiv preprint arXiv:2503.16428 , year=

    arXiv

  162. [171]

    arXiv preprint arXiv:2502.20766 , year=

    Flexprefill: A context-aware sparse attention mechanism for efficient long-sequence inference , author=. arXiv preprint arXiv:2502.20766 , year=

    arXiv

  163. [172]

    and Li, Dongsheng and Lin, Chin-Yew and Yang, Yuqing and Qiu, Lili , title =

    Jiang, Huiqiang and Li, Yucheng and Zhang, Chengruidong and Wu, Qianhui and Luo, Xufang and Ahn, Surin and Han, Zhenhua and Abdi, Amir H. and Li, Dongsheng and Lin, Chin-Yew and Yang, Yuqing and Qiu, Lili , title =. 2025 , booktitle =

    2025

  164. [173]

    arXiv preprint arXiv:2204.07447 , year=

    Stretching sentence-pair NLI models to reason over long documents and clusters , author=. arXiv preprint arXiv:2204.07447 , year=

    arXiv

  165. [174]

    Transactions of the Association for Computational Linguistics , volume=

    SummaC: Re-visiting NLI-based models for inconsistency detection in summarization , author=. Transactions of the Association for Computational Linguistics , volume=. 2022 , publisher=

    2022

  166. [175]

    arXiv preprint arXiv:2309.17453 , year =

    Efficient Streaming Language Models with Attention Sinks , author =. arXiv preprint arXiv:2309.17453 , year =

    arXiv

  167. [176]

    Advances in Neural Information Processing Systems , volume=

    H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models , author=. Advances in Neural Information Processing Systems , volume=

  168. [177]

    SqueezeAttention: 2D Management of

    Zihao Wang and Bin CUI and Shaoduo Gan , booktitle=. SqueezeAttention: 2D Management of. 2025 , url=

    2025

  169. [178]

    arXiv preprint arXiv:2502.06766 , year=

    Exploiting Sparsity for Long Context Inference: Million Token Contexts on Commodity GPUs , author=. arXiv preprint arXiv:2502.06766 , year=

    arXiv

  170. [179]

    arXiv preprint arXiv:2401.06104 , year =

    Transformers are Multi-State RNNs , author =. arXiv preprint arXiv:2401.06104 , year =

    arXiv

  171. [180]

    arXiv preprint arXiv:2308.14508 , year=

    LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding , author=. arXiv preprint arXiv:2308.14508 , year=

    arXiv

  172. [181]

    2023--2024 , license =

    2023

  173. [182]

    arXiv preprint arXiv:2105.08209 , year=

    Booksum: A collection of datasets for long-form narrative summarization , author=. arXiv preprint arXiv:2105.08209 , year=

    arXiv

  174. [183]

    2023 , howpublished=

    LLMTest Needle In A Haystack - Pressure Testing LLMs , author=. 2023 , howpublished=

    2023

  175. [184]

    Multi Needle in a Haystack , year =

  176. [185]

    2024 , month = feb, day =

    Dhinakaran, Aparna and Jolley, Evan , title =. 2024 , month = feb, day =

    2024

  177. [186]

    Needle In A Haystack Evaluation , year =

  178. [187]

    arXiv preprint arXiv:2406.07230 , year =

    Needle In A Multimodal Haystack , author =. arXiv preprint arXiv:2406.07230 , year =

    arXiv

  179. [188]

    arXiv preprint arXiv:2411.19360 , year =

    DENIAHL: In-Context Features Influence LLM Needle-In-A-Haystack Abilities , author =. arXiv preprint arXiv:2411.19360 , year =

    arXiv

  180. [189]

    Lin, Chin-Yew , booktitle=

  181. [190]

    and Artzi, Yoav , booktitle=

    Zhang, Tianyi and Kishore, Varsha and Wu, Felix and Weinberger, Kilian Q. and Artzi, Yoav , booktitle=

  182. [191]

    The Thirteenth International Conference on Learning Representations , year=

    Beware of Calibration Data for Pruning Large Language Models , author=. The Thirteenth International Conference on Learning Representations , year=

  183. [192]

    Preserving

    Bowei He and Lihao Yin and Huiling Zhen and Shuqi LIU and Han Wu and Xiaokun Zhang and Mingxuan Yuan and Chen Ma , booktitle=. Preserving. 2025 , url=

    2025

  184. [193]

    Mengting Ai and Sirui Chen and Maggie Zarekarizi and Kommy Weldemariam and Jingrui He , year=

  185. [194]

    On the Impact of Calibration Data in Post-training Quantization and Pruning

    Williams, Miles and Aletras, Nikolaos. On the Impact of Calibration Data in Post-training Quantization and Pruning. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024. doi:10.18653/v1/2024.acl-long.544

    2024 doi

  186. [195]

    ACM Transactions on Knowledge Discovery from Data (TKDD) , volume=

    Compression of deep learning models for text: A survey , author=. ACM Transactions on Knowledge Discovery from Data (TKDD) , volume=. 2022 , publisher=

    2022

  187. [196]

    Transactions of the Association for Computational Linguistics , volume=

    A survey on model compression for large language models , author=. Transactions of the Association for Computational Linguistics , volume=. 2024 , publisher=

    2024

  188. [197]

    Low-power computer vision , pages=

    A survey of quantization methods for efficient neural network inference , author=. Low-power computer vision , pages=. 2022 , publisher=

    2022

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.