Pith. sign in

REVIEW 3 major objections 8 minor 43 references

DipSVD: Dual-importance Protected SVD for Efficient LLM Compression

T0 review · 3 major / 8 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper's central claim is that SVD-based compression of large language models can preserve far more performance by protecting both important channels within each weight matrix and important layers across the network than existing SVD…

desk verdict Empirical results look solid but the central whitening proof is wrong; needs major revision before acceptance. read the letter →

arxiv 2506.20353 v1 pith:MYR7UHH3 submitted 2025-06-25 cs.LG cs.AI

Xuan Ding , Rui Sun , Yunjian Zhang , Xiu Yan , Yueqi Zhou , Kaihao Huang , Suzhong Fu , Chuanlong Xie
show 1 more author
Yao Zhu
This is my paper
classification cs.LGcs.AI
keywords LLMcompressionsingularvaluedecompositionlow-rankfactorizationchannel-weightedwhiteninglayer-wiseratioFishersensitivityeffectiverankpost-training
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that SVD-based compression of large language models can be made substantially less damaging by explicitly protecting important parts of the model at two levels. Locally, it amplifies the channels that most strongly shape the input's second-order statistics before whitening the weight matrix, so that truncation discards the singular vectors that matter least. Globally, it assigns each Transformer layer its own compression ratio, pushing more of the burden onto layers with low Fisher sensitivity and low effective rank, either by a heuristic or by Bayesian optimization. The authors claim these two additions let compressed LLaMA, Vicuna, and DeepSeek models keep lower perplexity and higher zero-shot accuracy than FWSVD, ASVD, and SVD-LLM at the same compression ratios, with the gap growing as compression gets more aggressive. This matters because SVD compression is retraining-free and hardware-friendly, so improving its accuracy at high ratios directly lowers the cost of deploying large models.

What carries the argument

The machinery is channel-weighted data whitening coupled to truncated SVD of the whitened weight. Formally, with calibration activations $X$, channel importance is $\alpha_j = \sqrt{x_j^T (XX^T)x_j}$; a diagonal $D$ amplifies the top $p\%$ channels by factor $a>1$; the reweighted covariance $\tilde{X}^T\tilde{X} = D^T X^T X D$ is decomposed as $U_{\tilde{X}}\Sigma_{\tilde{X}}U_{\tilde{X}}^T$, and the whitening matrix is $S = \Sigma_{\tilde{X}}^{-1/2}U_{\tilde{X}}^T$. The paper then decomposes $WS$, truncates the smallest singular values, and reconstructs as $U\,\mathrm{Trunc}^*(\Sigma)V^T S^{-1}$. The claimed effect is that truncation loss in output space equals the root-sum-square of the removed singular values, so singular values become direct proxies for performance; that is the local-protection claim. The global-protection half is the per-layer allocation law $p_l = \frac{Q_l}{\sum_j Q_j}\cdot L\cdot(1-k)$, with $Q_l = S_l^\beta R_l^{1-\beta}$ from Fisher sensitivity $S_l$ and effective rank $R_l$, or the Bayesian optimization objective that maximizes cosine similarity between original and compressed outputs under the global budget.

What would settle it

Compute, on real calibration data, the matrix $S^{-1}\tilde{X}\tilde{X}^T(S^{-1})^T$ for any compressed layer and compare it with the identity $I$; if it is not $I$, measure the actual output perturbation caused by deleting the $i$-th singular vector and compare it with $\sigma_i$. If the perturbation is materially different from $\sigma_i$, the stated theoretical guarantee does not hold even if the method still works in practice.

Watch

Extended reading notes

Core claim

On its own terms, DipSVD's discovery is that importance-weighted whitening plus layer-wise budget allocation turns truncated SVD into a usable high-ratio LLM compressor. For a weight matrix $W$ and activation data $X$, the method reweights channels by importance $\alpha_j = \sqrt{x_j^T(XX^T)x_j}$, amplifies the top channels with a diagonal matrix $D$, forms a whitening matrix $S$ from the SVD of $\tilde{X}^T\tilde{X}$ with $\tilde{X}=XD$, then decomposes $WS$, keeps the largest singular values, and reconstructs with $S^{-1}$. The paper argues this whitening makes the output-space loss from truncating the $i$-th singular value equal to $\sigma_i$, so the total loss is the root-sum-square of the discarded singular values; this is the claimed guarantee that small singular values are the safe ones to drop. At the network level, the method computes a per-layer importance score $Q_l$ combining Fisher sensitivity and effective rank, or searches directly with Bayesian optimization, and sets preservation ratios proportional to $Q_l$, so sensitive layers are compressed less. The empirical claim is that this dual protection beats all three prior SVD compressors on every tested model and compression ratio, and that the cheap heuristic tracks the Bayesian version well.

Load-bearing premise

The proof that truncating singular value $\sigma_i$ costs exactly $\sigma_i$ in output space assumes the whitening operation makes calibration data satisfy a specific algebraic identity, but the construction as written guarantees a different, related identity, so the exact equality is a premise the paper does not actually deliver.

Editorial extensions

If this is right

  • At tested compression ratios from 20% to 50%, compressed LLaMA-7B/13B, Vicuna-7B/13B, and DeepSeek-7B show lower perplexity and higher zero-shot accuracy than FWSVD, ASVD, and SVD-LLM under identical conditions.
  • The gap over baselines widens as the compression ratio rises, so the method's advantage is largest exactly where SVD compression is otherwise least usable.
  • Both protection components are independently useful: the ablation shows each one alone beats SVD-LLM, while the full combination gives the best results.
  • The heuristic allocation reproduces the Bayesian optimization allocation with Pearson correlation above 0.64 across ratios, so the cheap method can substitute for expensive search.
  • The low-rank form $\hat{X} = \Sigma_k V_k^T X$ followed by $U_k \hat{X}$ yields a parameter-budget-derived FLOP reduction, about 40% at 40% compression on Vicuna-7B, which translates into throughput and latency gains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The importance weighting is defined purely from calibration activations, so the same channel-amplification scheme could be dropped into other low-rank or pruning pipelines that currently rely on plain SVD; the paper does not test this transfer.
  • The heuristic needs only gradients and hidden states on a small calibration set, so it is likely to scale to much larger or instruction-tuned models where Bayesian optimization over the full layer space would be too costly; this is an extrapolation, not a result in the paper.
  • A natural extension is replacing the fixed amplification factor $a$ with a per-layer or per-channel adaptive value, which the paper's limitation section itself flags as unexplored.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. The paper proposes DipSVD, an SVD-based LLM compression method with two protected components: (1) a local importance mechanism that reweights input channels and applies a data-whitening transformation before truncating singular values, and (2) a global importance mechanism that allocates layer-specific compression ratios via Bayesian optimization or a heuristic combining Fisher sensitivity and effective rank. The authors claim that channel-weighted whitening provides a direct, provable mapping between singular values and output loss, and they report consistent perplexity and zero-shot accuracy improvements over ASVD, FWSVD, and SVD-LLM across five models and several compression ratios.

Significance. If the theoretical guarantee in Sec. 3.1.2 were valid, the paper would make a useful contribution to SVD-based LLM compression by jointly addressing local and global importance. The experimental scope is broad: five models, three model families, multiple compression ratios, and both perplexity and zero-shot benchmarks, and the authors include ablations and pseudocode. However, the central proof is invalid as written, and the empirical validation does not currently separate genuine gains from calibration and tuning effects. The idea of channel-weighted whitening and layer-wise ratio allocation is plausible and may be worth pursuing, but the present manuscript does not establish its theoretical or empirical claims at the standard required for acceptance.

major comments (3)
  1. [Sec. 3.1.2, Eq. (7)] The derivation of L_i = sigma_i hinges on the whitening property S^{-1} \tilde{X} \tilde{X}^T (S^{-1})^T = I. With S = \Sigma^{-1/2} U^T obtained from \tilde{X}^T \tilde{X} = U \Sigma U^T, this identity is generally false; the standard column-whitening property is (\tilde{X} U \Sigma^{-1/2})^T (\tilde{X} U \Sigma^{-1/2}) = I, which is a different object and does not imply the row Gram identity used here. Moreover, Eq. (6) is dimensionally problematic: X in R^{m x n} and S in R^{n x n} make S^{-1} \tilde{X} undefined when m != n. Therefore the equality L_i = sigma_i and the root-sum-square expression in Eq. (8) are unsupported, and the central claim that truncating the smallest singular values of the whitened weight matrix provably minimizes output loss is not established. The empirical method may remain useful as a heuristic, but the 'direct mapping' theoretical foundation stated in Sec. 3.1 is invalid as written.
  2. [Sec. 4.2-4.4 and Tables 2-4] The empirical comparisons are reported without error bars, confidence intervals, or significance tests, and the hyperparameters a, bar, and beta appear to be selected using the same benchmark suites on which performance is reported (Figs. 4 and Sec. 4.3). In addition, calibration data are drawn from WikiText-2 while WikiText-2 perplexity is a headline result, and Fig. 5b shows that performance is best when calibration and test distributions match. As a result, the consistent gains over baselines cannot be distinguished from calibration or tuning artifacts. The paper should report variance over calibration seeds, use a disjoint validation split for hyperparameter selection, and include significance tests or effect sizes for the main tables.
  3. [Sec. 4.3, Eq. (12), Algorithm 3] The validation of the heuristic is circular as reported: beta is chosen because it maximizes the Pearson correlation in Table 5, and then that same table is presented as evidence that the heuristic agrees with Bayesian optimization. Moreover, the paper states beta = 0.25 in Sec. 4.3 but Algorithm 3 lists the default as 0.3, so the exact configuration used in the main experiments is ambiguous. The authors should specify a fixed beta chosen on a validation set and report the sensitivity of the correlation and downstream performance to beta.
minor comments (8)
  1. [Sec. 3.2.2, Eq. (11)] Equation (11) is ambiguous for H_l in R^{B x T x D}; please specify the reshaping and which matrix is decomposed (as done in Algorithm 5).
  2. [Table 3] The caption claims average accuracy of seven datasets, but the table shows only three perplexity columns and an 'Average' column; the zero-shot accuracy columns appear to be missing.
  3. [Sec. 4.1, Benchmarks] The citation for C4 is given as (Mihaylov et al., 2018b), which is the OpenbookQA paper; a proper C4 reference is needed.
  4. [Throughout] Several typos should be corrected: 'Beyesian' (Sec. 4.3), 'Sensity-based' (Table 1), 'languge' (Sec. 5), 'theoetical' and 'detemined' (Appendix B), 'varing' (Table 12), and 'th' (Appendix C.2).
  5. [Algorithm 1, line 7] The notation k_l is used both as the target compression ratio and as the result of the arg min; please use distinct symbols for the target and the selected rank.
  6. [Algorithm 2, line 8] The line 'SetS <- S S SetS' appears garbled and does not match Eq. (4) for constructing S; please align the pseudocode with the main text.
  7. [Sec. 3.2.2 and Sec. 3.3] Equation (12) is numbered twice, once in Sec. 3.2.2 and once in Sec. 3.3 Step 3, with different meanings; please renumber.
  8. [Appendix B] With m = n = d, the optimal rank from k = 0.6mn/(m+n) is 0.3d, not 0.6d as stated; the final 40% FLOP reduction is correct, but the intermediate statement should be corrected.

Circularity Check

1 steps flagged · score 3.0 of 10

β for the heuristic layer-scoring rule is selected to maximize the heuristic-vs-Bayesian correlation, and that same correlation is then reported as evidence; all other central claims are benchmarked independently.

  1. fitted input called prediction [Sec. 4.3, paragraph 'Connection with Bayesian optimization']
    "In the experiments, the heuristic method showed the strongest correlation with the optimization-based method when β = 0.25. Therefore, we uniformly set β to 0.25. Tab.5 shows the Pearson correlation coefficient between the layer-wise compression rates obtained by heuristic and Bayesian optimization across different target compression ratios. The correlation coefficients consistently exceed 0.64, indicating strong agreement between the two methods at all compression levels."

    The hyperparameter β weights Fisher sensitivity against effective rank in the heuristic importance score (Eq. 12 in Sec. 3.2.2). The paper states β was chosen because it gave the strongest correlation with Bayesian-optimization ratios, and then Tab.5 reports the resulting correlation as evidence of agreement. This is an in-sample selection loop: the reported 'strong agreement' is the same objective used to fit β, not an independent confirmation. It does not force the main benchmark results, which are measured against held-out perplexity and zero-shot tasks, so the circularity is partial and localized.

full rationale

The paper's central empirical claims—DipSVD beating FWSVD, ASVD, and SVD-LLM on perplexity and zero-shot accuracy—are evaluated on external datasets and are not constructed from the fitted parameters. Calibration on 256 WikiText-2 sentences and evaluation on WikiText-2 perplexity follows the convention of the baselines and is itself probed by the cross-calibration study in Fig. 5b. The Sec. 3.1.2 derivation of L_i = sigma_i is not circular but mathematically unsupported: the identity S^{-1} tilde-X tilde-X^T (S^{-1})^T = I is not implied by the construction S = Sigma^{-1/2} U^T from tilde-X^T tilde-X = U Sigma U^T; that construction guarantees a different whitening identity for tilde-X S. I treat this as a validity concern rather than a circularity because the conclusion does not reduce to the premise by definition. The only genuine circular step is the beta selection in Sec. 4.3, where the hyperparameter is tuned to maximize the heuristic-vs-Bayesian correlation and then that same correlation is exhibited as validation. No load-bearing self-citation or imported uniqueness theorem appears. Accordingly the circularity score is modest.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central derivation rests on a false whitening identity; the method also introduces tuned constants a, bar, beta, and tau. No new physical entities are postulated, so invented_entities is empty.

free parameters (4)
  • importance amplification factor a (weight) = 30
    Scaling applied to top channels in D; chosen from ablation in Fig. 4.
  • top channel selection ratio p (bar) = 0.03
    Fraction of channels amplified; ablation-chosen in Fig. 4.
  • beta trade-off between Fisher sensitivity and effective rank = 0.25
    Set in Sec. 4.3 because it gave the strongest correlation with Bayesian optimization ratios.
  • energy threshold tau = 0.95
    Used in effective rank computation; default stated in Algorithm 3.
assumptions (5)
  • ad hoc to paper The whitening identity S^{-1} X-tilde X-tilde^T (S^{-1})^T = I
    Invoked in Eq. (7) to conclude L_i = sigma_i; it does not follow from S constructed from X-tilde^T X-tilde and is generally false.
  • domain assumption Truncating the smallest singular values of the whitened weight matrix directly minimizes output loss
    Intended to be proved in Sec. 3.1.2, but the proof depends on the false whitening identity.
  • domain assumption Fisher sensitivity and effective rank capture layer importance
    Used to set layer-wise ratios in Eqs. (10)-(13); no independent validation except correlation with Bayesian optimization.
  • domain assumption Calibration data are representative of test data
    All ratios and whitening are computed from 256 WikiText-2 samples; Fig. 5 shows performance degrades on mismatched test sets.
  • domain assumption Bayesian optimization with cosine similarity is a good proxy for model quality
    The optimization objective in Eq. (9) is used to select layer ratios; its correlation with task performance is not established.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DipSVD: Dual-importance Protected SVD for Efficient LLM Compression." pith.science (2026). https://pith.science/paper/MYR7UHH3

@misc{pith2026250620353,
  author       = {Pith},
  title        = {Pith review of: DipSVD: Dual-importance Protected SVD for Efficient LLM Compression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MYR7UHH3}},
  note         = {Machine review of arXiv:2506.20353}
}
read the original abstract

The ever-increasing computational demands and deployment costs of large language models (LLMs) have spurred numerous compressing methods. Compared to quantization and unstructured pruning, SVD compression offers superior hardware compatibility and theoretical guarantees. However, existing SVD-based methods focus on the overall discrepancy between the original and compressed matrices while overlooking the protection of critical components within the matrix, which leads to inferior performance in the compressed models. This paper proposes a dual-level importance protection mechanism to enhance SVD-based compression methods: (1) local importance protection: preserving the most critical singular vectors within each weight matrix through channel-weighted data whitening; and (2) global importance protection: enabling less important layers to bear a greater portion of the compression burden through either a heuristic or optimization-based approach, thereby minimizing the impact of compression on critical layers. Extensive experiments demonstrate that DipSVD outperforms existing SVD-based compression approaches across multiple benchmarks, achieving superior model performance especially at high model compression ratios.

Figures

Figures reproduced from arXiv: 2506.20353 by the authors.

Figure 1
Figure 1. Overview of DipSVD. significantly improving the balance between com￾pression efficacy and inference speed. However, existing SVD-based methods typically neglect both global and local importance consider￾ations in model architecture. The failure to account for global importance, as evidenced by varying layer-wise compression sensitivity, leads to subop￾timal rank selection and significant performance degradation unde… view at source ↗
Figure 2
Figure 2. The Ql , Sl , and Rl values for each layer in Vicuna-7B at 20% compression ratio. 3.3 Integrated Compression Process DipSVD incorporates local and global significance protection mechanisms into compression: Step 1: Layer Whitening. For each layer, per￾form data whitening using the matrix X˜ T X˜ and SVD, as described in Sec.3.1.1. Step 2: Layer-Specific Compression Ratios. Use Bayesian optimization (or the heuristic… view at source ↗
Figure 3
Figure 3. Perplexity on WikiText-2 and average zero [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Ablation study of calibration datasets: (a) [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 4
Figure 4. Figure 4: Parameter ablation studies: (a) Step1 weight [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 6
Figure 6. Figure 6: Inference efficiency on Vicuna-7B under dif [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 8 canonical work pages

  1. [1]

    Rishabh Agarwal, Nino Vieillard, Piotr Stanczyk, Sabela Ramos, Matthieu Geist, and Olivier Bachem. 2023. Gkd: Generalized knowledge distillation for auto-regressive sequence models. CoRR

    work page 2023

  2. [2]

    Aida Amini, Saadia Gabriel, Peter Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. 2019. https://arxiv.org/abs/1905.13319 Mathqa: Towards interpretable math word problem solving with operation-based formalisms . Preprint, arXiv:1905.13319

    arXiv 2019

  3. [3]

    Yongqi An, Xu Zhao, Tao Yu, Ming Tang, and Jinqiao Wang. 2024. Fluctuation-based adaptive structured pruning for large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 10865--10873

    work page 2024

  4. [4]

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, and 1 others. 2020. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432--7439

    2020

  5. [5]

    Jianfei Chen, Lianmin Zheng, Zhewei Yao, Dequan Wang, Ion Stoica, Michael Mahoney, and Joseph Gonzalez. 2021. Actnn: Reducing training memory footprint via 2-bit activation compressed training. In International Conference on Machine Learning, pages 1803--1813. PMLR

    work page 2021

  6. [6]

    Zhipeng Chen, Kun Zhou, Beichen Zhang, Zheng Gong, Wayne Xin Zhao, and Ji-Rong Wen. 2023. Chatcot: Tool-augmented chain-of-thought reasoning on chat-based large language models. arXiv preprint arXiv:2305.14323

    arXiv 2023

  7. [7]

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, and 1 others. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90\ See https://vicuna. lmsys. org (accessed 14 April 2023), 2(3):6

    work page 2023

  8. [8]

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457

    arXiv 2018

Show all 43 references
  1. [9]

    Antonia Creswell, Murray Shanahan, and Irina Higgins. 2022. Selection-inference: Exploiting large language models for interpretable logical reasoning. arXiv preprint arXiv:2205.09712

    2022 arXiv

  2. [10]

    Rocktim Jyoti Das, Mingjie Sun, Liqun Ma, and Zhiqiang Shen. 2023. Beyond size: How gradients shape pruning decisions in large language models. arXiv preprint arXiv:2311.04902

    2023 arXiv

  3. [11]

    DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, and 181 others. 2025. https://arxiv.org/abs/24...

    2025 arXiv

  4. [12]

    Xuan Ding, Yao Zhu, Yunjian Zhang, and Chuanlong Xie. 2025. A sliding layer merging method for efficient depth-wise pruning in llms. arXiv preprint arXiv:2502.19159

    2025

  5. [13]

    Jinhao Duan, Renming Zhang, James Diffenderfer, Bhavya Kailkhura, Lichao Sun, Elias Stengel-Eskin, Mohit Bansal, Tianlong Chen, and Kaidi Xu. 2024. Gtbench: Uncovering the strategic reasoning limitations of llms via game-theoretic evaluations. arXiv preprint arXiv:2402.12348

    2024 arXiv

  6. [14]

    Elias Frantar and Dan Alistarh. 2023. Sparsegpt: Massive language models can be accurately pruned in one-shot. In International Conference on Machine Learning, pages 10323--10337. PMLR

    2023

  7. [15]

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2022. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323

    2022 arXiv

  8. [16]

    Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang S...

    2024 doi

  9. [17]

    Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. 2023. Minillm: Knowledge distillation of large language models. arXiv preprint arXiv:2306.08543

    2023 arXiv

  10. [18]

    Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alexander Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. 2023. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes. arXiv preprint a...

    2023 arXiv

  11. [19]

    Yen-Chang Hsu, Ting Hua, Sungen Chang, Qian Lou, Yilin Shen, and Hongxia Jin. 2022. Language model compression with weighted low-rank factorization. arXiv preprint arXiv:2207.00112

    2022 arXiv

  12. [20]

    Bo-Kyeong Kim, Geonmin Kim, Tae-Ho Kim, Thibault Castells, Shinkook Choi, Junho Shin, and Hyoung-Kyu Song. 2024. Shortened llama: A simple depth pruning for large language models. arXiv preprint arXiv:2402.02834, 11

    2024 arXiv

  13. [21]

    Yong-Deok Kim, Eunhyeok Park, Sungjoo Yoo, Taelim Choi, Lu Yang, and Dongjun Shin. 2015. Compression of deep convolutional neural networks for fast and low power mobile applications. arXiv preprint arXiv:1511.06530

    2015 arXiv

  14. [22]

    Jongwoo Ko, Sungnyun Kim, Tianyi Chen, and Se-Young Yun. 2024. Distillm: Towards streamlined distillation for large language models. arXiv preprint arXiv:2402.03898

    2024 arXiv

  15. [23]

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. Proceedings of Machine Learning and Systems, 6:87--100

    2024

  16. [24]

    Jieyi Long. 2023. Large language model guided tree-of-thought. arXiv preprint arXiv:2305.08291

    2023 arXiv

  17. [25]

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. 2023. Llm-pruner: On the structural pruning of large language models. Advances in neural information processing systems, 36:21702--21720

    2023

  18. [26]

    Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz

    Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. https://aclanthology.org/J93-2004/ Building a large annotated corpus of E nglish: The P enn T reebank . Computational Linguistics, 19(2):313--330

    1993

  19. [27]

    Xin Men, Mingyu Xu, Qingyu Zhang, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, and Weipeng Chen. 2024. Shortgpt: Layers in large language models are more redundant than you expect. arXiv preprint arXiv:2403.03853

    2024 arXiv

  20. [28]

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. https://arxiv.org/abs/1609.07843 Pointer sentinel mixture models . Preprint, arXiv:1609.07843

    2016 arXiv

  21. [30]

    Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018 b . https://arxiv.org/abs/1809.02789 Can a suit of armor conduct electricity? a new dataset for open book question answering . Preprint, arXiv:1809.02789

    2018 arXiv

  22. [31]

    Liangming Pan, Alon Albalak, Xinyi Wang, and William Yang Wang. 2023. Logic-lm: Empowering large language models with symbolic solvers for faithful logical reasoning. arXiv preprint arXiv:2305.12295

    2023 arXiv

  23. [32]

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99--106

    2021

  24. [33]

    Jiwon Song, Kyungseok Oh, Taesu Kim, Hyungjun Kim, Yulhwa Kim, and Jae-Joon Kim. 2024. Sleb: Streamlining llms through redundancy verification and elimination of transformer blocks. arXiv preprint arXiv:2402.09025

    2024 arXiv

  25. [34]

    Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. 2023. A simple and effective pruning approach for large language models. arXiv preprint arXiv:2306.11695

    2023 arXiv

  26. [35]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, pa...

    2023 arXiv

  27. [36]

    Xin Wang, Zhongwei Wan, Arvin Hekmati, Mingyu Zong, Samiul Alam, Mi Zhang, and Bhaskar Krishnamachari. 2024 a . Iot in the era of generative ai: Vision and challenges. IEEE Internet Computing

    2024

  28. [37]

    Xin Wang, Yu Zheng, Zhongwei Wan, and Mi Zhang. 2024 b . Svd-llm: Truncation-aware singular value decomposition for large language model compression. arXiv preprint arXiv:2403.07378

    2024 arXiv

  29. [38]

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, pages 38087--38099. PMLR

    2023

  30. [39]

    Zhihang Yuan, Yuzhang Shang, Yue Song, Qiang Wu, Yan Yan, and Guangyu Sun. 2023. Asvd: Activation-aware singular value decomposition for compressing large language models. arXiv preprint arXiv:2312.05821

    2023 arXiv

  31. [40]

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830

    2019 arXiv

  32. [41]

    Yang Zhang, Yanfei Dong, and Kenji Kawaguchi. 2024. Investigating layer importance in large language models. arXiv preprint arXiv:2409.14381

    2024 arXiv

  33. [42]

    Zixuan Zhou, Xuefei Ning, Ke Hong, Tianyu Fu, Jiaming Xu, Shiyao Li, Yuming Lou, Luning Wang, Zhihang Yuan, Xiuhong Li, and 1 others. 2024. A survey on efficient inference for large language models. arXiv preprint arXiv:2404.14294

    2024 arXiv

  34. [43]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  35. [44]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.