Pith. sign in

REVIEW 3 major objections 4 minor 45 references

When Fewer Layers Break More Chains: Layer Pruning Harms Test-Time Scaling in LLMs

T0 review · 3 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Layer pruning that looks harmless on knowledge benchmarks can destroy a language model's ability to reason better with more thinking time, and standard fine-tuning cannot bring that ability back.

desk verdict Useful empirical probe showing layer pruning collapses sequential test-time scaling, but the evaluation stack needs validation before the numbers are taken at face value. read the letter →

arxiv 2510.22228 v2 pith:XHEFZWLE submitted 2025-10-25 cs.LG cs.AI

Keyu Wang , Tian Lyu , Guinan Su , Lu Yin , Marco Canini , Jonas Geiping , Shiwei Liu This is my paper · ORCID
classification cs.LGcs.AI
keywords layerpruningtest-timescalinglong-chainreasoningsequentialparallelsupervisedfine-tuningloopsSelf-BLEU
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that test-time scaling — the ability of a reasoning model to improve as it spends more tokens thinking — is the capability most damaged by layer pruning, and that this damage is invisible to knowledge benchmarks. Removing just one or two layers from a 7B or 8B reasoning model flattens or collapses the accuracy-versus-thinking-token curve on math and science reasoning tasks, even when MMLU-style performance barely moves. The paper further shows that neither LoRA nor full-parameter fine-tuning recovers this scaling, and traces the failure to structural damage: pruned models fall into repetitive loops, produce less diverse reasoning trajectories, and exhibit fewer self-correction behaviors. The upshot is a trade-off: efficiency gains from depth pruning come at the cost of the mechanism that makes long reasoning work.

What carries the argument

The accuracy-versus-thinking-token curve (sequential test-time scaling) is the probe that exposes the fragility; pruning methods — ShortGPT (Block Influence based), Reverse-order (removing deep layers), and LaCo (layer collapsing) — are the interventions. Self-BLEU and counts of verification, backtracking, and subgoal behaviors provide diagnostics that link the scaling collapse to repetitive loops and lost diversity rather than to simple knowledge erasure.

What would settle it

Run AIME24 after one-layer pruning and score it two ways: exact match of the final integer answer (AIME answers are integers) and the LLM-as-a-Judge method. If exact-match accuracy holds steady while judge scores collapse, the reported collapse is partly a scoring artifact; for MATH500 and GPQA Diamond, human grading of the same outputs would provide the same check.

Watch

Extended reading notes

Core claim

The central claim is that sequential test-time scaling is extremely fragile to layer pruning. Across three pruning methods and two reasoning models, pruning one layer substantially impairs the accuracy gains normally obtained by increasing the thinking-token budget, and pruning two layers nearly eliminates them, with AIME24 accuracy approaching zero. Parallel scaling (pass@k) is more mixed: direct removal hurts it, but the merging-based method LaCo sustains or even improves it. Supervised fine-tuning delivers only marginal or negative gains for one-layer pruned models and, while it lifts two-layer pruned models somewhat, it does not restore the original scaling. The paper attributes the brea

Load-bearing premise

The entire collapse story rests on scores assigned by an automated judge model that the paper never validates against ground-truth answers or human labels on the reasoning benchmarks.

Editorial extensions

If this is right

  • Pruned models should not be assumed safe for reasoning workloads: their accuracy will stop improving — or worsen — as inference-time compute increases, even at pruning depths that leave knowledge benchmarks stable.
  • Reported retention on knowledge benchmarks after pruning is not evidence of reasoning preservation; evaluation of pruning methods should include test-time scaling curves.
  • Neither LoRA nor full-parameter fine-tuning is a reliable remedy once test-time scaling has been degraded; pruning pipelines for reasoning models need a different recovery mechanism.
  • Merging-based pruning behaves differently from direct removal, retaining parallel test-time scaling in the tested settings, which singles it out as a more promising direction for efficiency without sacrificing reasoning.
  • Because most individual layers contribute non-trivially to test-time scaling, the common practice of pruning a few layers by importance scores is not a harmless shortcut for reasoning models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the collapse is structural (looping) rather than knowledge loss, other depth-compressing operations — quantization, layer merging at larger scale, or even architectural changes that shorten the residual chain — may show the same flattening of scaling curves; a cheap test is to run the same benchmarks on quantized versions of the same models.
  • The automated judge used for MATH500, GPQA Diamond, and AIME24 is never validated against ground truth or human labels. Pruned models produce long, repetitive outputs, so a judge trained on fluent text could systematically under-grade them; scoring AIME24 by exact match of its integer answers would settle this.
  • The qualitative loops suggest pruning may disrupt the internal 'budget forcing' that makes longer thinking productive, not just the knowledge layers. If so, scaling curves could serve as a general diagnostic for whether a compression method preserves reasoning machinery, not only in pruning.
  • LaCo's preserved parallel scaling hints that merging-based compression might be combined with sampling-based inference search to recover some of what sequential scaling loses, a direction the paper does not explore.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies whether layer pruning, which is known to preserve performance on knowledge-heavy benchmarks, preserves test-time scaling in long-chain reasoning. The authors prune s1.1-7B and Qwen3-8B with three methods (ShortGPT, Reverse-order, LaCo), removing one or two layers, and measure accuracy as a function of thinking-token budget (512–8192) on MATH500, GPQA Diamond, and AIME24, together with pass@k on AIME24. They report that even one-layer pruning flattens or collapses sequential scaling curves, that two-layer pruning is worse, that LoRA and full fine-tuning do not restore the curves, and that pruned models fall into repetitive reasoning loops. They conclude that layer pruning damages the structural preconditions of test-time scaling and call for pruning methods that preserve reasoning robustness. Code is open-sourced.

Significance. If the reported effect is real, the paper identifies an important failure mode missed by MMLU-style evaluations of pruning: test-time scaling, a central mechanism for modern reasoning models, can break under minimal depth reduction. The experimental matrix (two models, three pruning strategies, three benchmarks, both sequential and parallel scaling) is a genuine strength, as is the open-sourced codebase and the qualitative failure analysis. However, the headline accuracy curves depend on an unvalidated LLM-as-a-Judge and are reported without variance estimates. These two issues affect the empirical core of the paper; the qualitative conclusion is plausible but not yet established at the quantitative level claimed.

major comments (3)
  1. [Section 3, Evaluation datasets and metrics] All sequential scaling results for MATH500 and AIME24 (Figures 2, 4, 7, and related figures) are obtained with LLM-as-a-Judge using GLM-4.5-Flash, but the paper reports no validation of this judge: no human agreement, no exact-match comparison, and no error analysis. The statement that this is 'a more robust evaluation than exact string matching' is asserted, not demonstrated. This is a load-bearing issue because the paper's own outputs (Section 5.1, Figure 5, Appendix G) show pruned models producing repetitive, self-contradictory text; a judge can systematically under-grade such outputs even when a correct final answer is present. AIME24 has deterministic integer answers and exact-match grading is standard. Please report exact-match AIME24 curves (and ideally a judge/human agreement study on a sample) to show that the collapse is not partly an evaluation artifact.
  2. [Section 3, Sequential scaling] The paper states that experiments use three seeds and report average results, but no error bars, standard deviations, or per-seed values are given in Figures 2, 7, or elsewhere. This is important for AIME24, which has only 30 problems: a difference of 0.033 corresponds to one question, and the reported AIME24 collapse (e.g., from ~0.14 to ~0.04 after one layer in Figure 2) is only a few questions wide. With three seeds, the standard error around these points is non-negligible. The qualitative pattern across many curves is reassuring, but the quantitative magnitude of the collapse is not supported without variance reporting or per-seed data.
  3. [Section 4 and Appendix D] The LoRA fine-tuning procedure selects the learning rate by evaluating on MATH500, GPQA Diamond, and AIME24 and then reports results on 'the corresponding datasets.' As written, this reads like model selection on the test set, which makes the reported LoRA curves optimistically biased. I do not think this threatens the main conclusion, because the conclusion is that SFT does not recover scaling even in the best case, but the methodology needs clarification: was a separate validation split used, or were the test sets used for learning-rate selection? If the latter, the SFT results should be recomputed with a held-out selection criterion or explicitly described as an oracle selection.
minor comments (4)
  1. [Abstract and Section 3, Figure 3] The abstract's unqualified statement that pruning 'even one or two layers can severely impair test-time scaling' is too broad: Figure 3 shows that LaCo with two layers pruned improves pass@k on AIME24 relative to the unpruned model. The body text does qualify this, but the abstract should be adjusted to reflect that the impairment is method- and scaling-mode-dependent.
  2. [Table 1] The text says 'ShortGPT and LaCo consistently exhibit markedly higher Self-BLEU scores,' but Table 1 shows Reverse-order pruned models have Self-BLEU scores below the original (e.g., s1.1-7B: 0.685 -> 0.521 and 0.442). Since this table is used to support the reduced-diversity mechanism, the claim should be qualified or the analysis should be aligned with the data.
  3. [Section 5.2, Table 2] The self-reflection heuristic counts are produced by gpt-4o-mini as an automated judge, but no validation or agreement measure is reported for this rubric. This is an analysis-level result rather than the central claim, but a brief note on reliability would strengthen it.
  4. [Appendix D, Table 6] Some table entries appear implausible or contain apparent artifacts (e.g., 'Qwen3-8B LaCo 1 layer' with a value of 0.081 at lr=4e-5 while neighboring rows are ~0.9). Please check the table formatting and ensure the numbers are accurately transcribed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's central claim rests on direct empirical measurements, not on fitted parameters or self-citation chains.

full rationale

This is an empirical measurement study. The central claim—that layer pruning impairs test-time scaling—is supported by accuracy curves (Figures 2, 3, 4, 7) obtained by directly evaluating pruned and unpruned models under sequential and parallel scaling budgets. No derived equation is used to produce the headline result, and no parameter is fitted and then renamed as a prediction. The pruning methods (ShortGPT, Reverse-order, LaCo) are taken from prior external work and are applied as interventions, not as fitted inputs to the conclusion. The mechanistic analyses (Self-BLEU, self-reflection heuristics, qualitative loops) are post-hoc interpretations of the same generations and are not used as inputs to the accuracy calculations, so they cannot make the accuracy result circular. The only self-citations (e.g., Su et al., 2025b in the test-time scaling related work) are contextual references and are not load-bearing for the paper's main finding. The unvalidated use of GLM-4.5-Flash as an LLM judge is a legitimate correctness/validation concern, but it is not circularity: the judge is an external scoring model, not a quantity defined in terms of the paper's outcome, and the paper does not claim the judge's scoring rule is derived from the pruned models' outputs. Consistently with the reader's assessment, the appropriate circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim is empirical and does not rest on a fitted model. Its weakest upstream assumptions are the reliability of the automated judge, the operational definition of test-time scaling, and the representativeness of the two models and three pruning methods. None of these assumptions is a derived equation; they are experimental choices that should be validated or stated more explicitly.

assumptions (4)
  • domain assumption LLM-as-a-Judge with GLM-4.5-Flash reliably scores correctness on MATH500, GPQA Diamond, and AIME24 generations.
    All accuracy numbers and scaling curves are built on this judge. Section 3 asserts it is 'more robust than exact string matching' but gives no validation or agreement data.
  • domain assumption Test-time scaling is faithfully operationalized by accuracy as a function of thinking-token budget and by pass@k.
    This follows established works (s1, T1), but it assumes that longer token budgets and more samples map monotonically to reasoning ability, which is what the paper uses to define 'scaling.'
  • domain assumption The two open models s1.1-7B and Qwen3-8B are representative of reasoning-tuned LLMs generally.
    The paper extends conclusions to 'reasoning-intensive LLMs' (Section 6) from two models, one of which is relatively small; other model families or scales may be more or less fragile.
  • domain assumption ShortGPT, Reverse-order, and LaCo constitute a representative sample of layer-pruning techniques.
    The paper generalizes to 'layer pruning' from three methods, two direct-removal and one merging-based; other structured pruning methods may behave differently.

how reviews work

0 comments
Cite this review

Pith. "Pith review of When Fewer Layers Break More Chains: Layer Pruning Harms Test-Time Scaling in LLMs." pith.science (2026). https://pith.science/paper/XHEFZWLE

@misc{pith2026251022228,
  author       = {Pith},
  title        = {Pith review of: When Fewer Layers Break More Chains: Layer Pruning Harms Test-Time Scaling in LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XHEFZWLE}},
  note         = {Machine review of arXiv:2510.22228}
}
read the original abstract

Layer pruning has emerged as a widely adopted technique for improving the efficiency of large language models (LLMs). Although existing methods demonstrate strong performance retention on general knowledge tasks, their effect on long-chain reasoning, a more brittle yet crucial capability, remains largely unexplored. In this work, we study the impact of layer pruning on long-chain reasoning through the lens of test-time scaling, a key mechanism in modern LLMs that enables strong reasoning capacity by allocating more computation at inference time. With extensive experiments, we demonstrate that pruning even one or two layers can severely impair test-time scaling, with performance collapsing drastically on long reasoning benchmarks even when performance on knowledge-intensive and shallow reasoning tasks remains stable. Furthermore, we find that standard supervised fine-tuning remedies fail to recover test-time scaling once it has deteriorated. Through in-depth analyses, we identify the mechanisms underlying this fragility of test-time scaling and highlight the fundamental risks of applying layer pruning to reasoning-intensive LLMs. These findings call for a rethinking of layer pruning strategies and provide insights for developing methods that preserve the robustness of reasoning. We open-source the codebase in \href{https://github.com/keyu-wang-2002/Layer-Pruning-Harms-Inference-Scaling}{https://github.com/keyu-wang-2002/Layer-Pruning-Harms-Inference-Scaling}.

Figures

Figures reproduced from arXiv: 2510.22228 by the authors.

Figure 1
Figure 1. Accuracy of the pruned Qwen3-8B by ShortGPT across various pruning depths on MMLU, GSM8K, MATH500 and AIME24. LLM-as-a-Judge is applied to MATH500 and AIME24 as described in Section 3. To study this fragility more systematically and mechanistically, we turn to test-time scaling, a re￾cently emerging technique that provides a finer-grained understanding of how LLMs acquire and exercise long reasoning abilities (Muenn… view at source ↗
Figure 2
Figure 2. Sequential test-time scaling of s1.1-7B under different pruning depths. Evaluation dimensions. We evaluate the effect of layer pruning on test-time scaling along two orthogonal dimensions: (1) Sequential scaling. We measure performance under reasoning with increasing thinking token budgets of [512, 1024, 2048, 4096, 8192]. We set the temperature to 1.0 and run experiments with three randomly selected seeds, reportin… view at source ↗
Figure 3
Figure 3. Parallel test-time scaling of s1.1-7B on AIME24 under different pruning depths. and on GPQA Diamond it even approaches or slightly surpasses the original model. Nevertheless, pruning two layers completely collapses sequential scaling, leading to a sharp drop in performance. Taken together, these findings indicate that sequential test-time scaling is highly fragile under layer pruning, breaking down once pruning dept… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Sequential test-time scaling of s1.1-7B after Full FT under different pruning depths. We denote “Full” as Full FT and “Reverse” as Reverse-order in the figure for simplicity. original model and sequential scaling is not recovered. These results indicate that SFT fails …
Figure 5
Figure 5. Figure 5: Examples of repetitive thinking from 1-layer pruned s1.1-7B model. Unlike the original model, the pruned version gets stuck in logic loops. The teal text denotes the correct reasoning trace while light blue text is the repetitive reasoning trace leading to failure, and…
Figure 6
Figure 6. Figure 6: Brute-Force of s1.1-7B Layer Ablation for Sequential Test-time scaling (last 9 layers) 5.3 BRUTE-FORCE EXPLORATION OF LAYER ABLATION To systematically assess the contribution of individual layers in both s1.1-7B and Qwen3-8B, we conduct a brute-force ablation study: pr…
Figure 7
Figure 7. Figure 7: Sequential test-time scaling of Qwen3-8B under different pruning depths. pruning consistently undermines test-time scaling once the pruning depth surpasses a very shallow threshold. D LEARNING RATE SEARCH IN LORA FINE-TUNING [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Sequential test-time scaling of Qwen3-8B after Full FT under different pruning depths. “Full” means Full FT and “Reverse” means Reverse-order in the figure for simplicity. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Sequential test-time scaling of s1.1-7B after LoRA FT under different pruning depths. “LoRA” means LoRA FT and “Reverse” means Reverse-order in the figure for simplicity. 512 1024 2048 4096 8192 0.6 0.8 1 Acc (ShortGPT LoRA) MATH500 512 1024 2048 4096 8192 0.3 0.4 0.5 …
Figure 10
Figure 10. Figure 10: Sequential test-time scaling of Qwen3-8B after LoRA FT under different pruning depths. “LoRA” means LoRA FT and “Reverse” means Reverse-order in the figure for simplicity. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Brute-force of s1.1-7B layer ablation for sequential test-time scaling (Part 1) G ADDITIONAL QUALITATIVE EXAMPLES OF REPETITIVE REASONING We provide additional qualitative examples of the repetitive reasoning phenomenon in pruned mod￾els, as discussed in Section 5.1. …
Figure 12
Figure 12. Figure 12: Brute-force of s1.1-7B layer ablation for sequential test-time scaling (Part 2) 18 [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Brute-force of Qwen3-8B layer ablation for sequential test-time scaling (Part 1) 19 [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: Brute-force of Qwen3-8B layer ablation for sequential test-time scaling (Part 2) 20 [PITH_FULL_IMAGE:figures/full_fig_p020_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 19 linked inside Pith

  1. [1]

    Slicegpt: Compress large language models by deleting rows and columns

    Saleh Ashkboos, Maximilian L Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman. Slicegpt: Compress large language models by deleting rows and columns. arXiv preprint arXiv:2401.15024, 2024

    arXiv 2024

  2. [2]

    Universal self-consistency for large language model generation

    Xinyun Chen, Renat Aksitov, Uri Alon, Jie Ren, Kefan Xiao, Pengcheng Yin, Sushant Prakash, Charles Sutton, Xuezhi Wang, and Denny Zhou. Universal self-consistency for large language model generation. arXiv preprint arXiv:2311.17311, 2023

    arXiv 2023

  3. [3]

    Simple and provable scaling laws for the test-time compute of large language models

    Yanxi Chen, Xuchen Pan, Yaliang Li, Bolin Ding, and Jingren Zhou. Simple and provable scaling laws for the test-time compute of large language models. arXiv preprint arXiv:2411.19477, 2024 a

    arXiv 2024

  4. [4]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024 b

    arXiv 2024

  5. [5]

    Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars

    Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars. arXiv preprint arXiv:2503.01307, 2025

    arXiv 2025

  6. [6]

    The language model evaluation harness, 07 2024

    Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. The languag...

    arXiv 2024

  7. [7]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  8. [8]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. International Conference on Learning Representations, 2021 a

    2021

Show all 45 references
  1. [9]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. Advances in Neural Information Processing Systems, 2021 b

    2021

  2. [10]

    Advancing language model reasoning through reinforcement learning and inference scaling

    Zhenyu Hou, Xin Lv, Rui Lu, Jiajie Zhang, Yujiang Li, Zijun Yao, Juanzi Li, Jie Tang, and Yuxiao Dong. Advancing language model reasoning through reinforcement learning and inference scaling. arXiv preprint arXiv:2501.11651, 2025

    2025 arXiv

  3. [11]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. International Conference on Learning Representations, 2022

    2022

  4. [12]

    Shortened llama: A simple depth pruning for large language models

    Bo-Kyeong Kim, Geonmin Kim, Tae-Ho Kim, Thibault Castells, Shinkook Choi, Junho Shin, and Hyoung-Kyu Song. Shortened llama: A simple depth pruning for large language models. arXiv preprint arXiv:2402.02834, 2024

    2024 arXiv

  5. [13]

    Let's verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In International Conference on Learning Representations, 2023

    2023

  6. [14]

    Pruning via merging: Compressing llms via manifold alignment based layer merging

    Deyuan Liu, Zhanyue Qin, Hairu Wang, Zhao Yang, Zecheng Wang, Fangying Rong, Qingbin Liu, Yanchao Hao, Xi Chen, Cunhang Fan, et al. Pruning via merging: Compressing llms via manifold alignment based layer merging. EMNLP, 2024

    2024

  7. [15]

    Reassessing layer pruning in llms: New insights and methods

    Yao Lu, Hao Cheng, Yujie Fang, Zeyu Wang, Jiaheng Wei, Dongwei Xu, Qi Xuan, Xiaoniu Yang, and Zhaowei Zhu. Reassessing layer pruning in llms: New insights and methods. arXiv preprint arXiv:2411.15558, 2024

    2024 arXiv

  8. [16]

    Llm-pruner: On the structural pruning of large language models

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. Llm-pruner: On the structural pruning of large language models. Advances in Neural Information Processing Systems, 2023

    2023

  9. [17]

    American invitational mathematics examination 2024 aime

    Mathematical Association of America . American invitational mathematics examination 2024 aime. https://artofproblemsolving.com/wiki/index.php/AIME_Problems_and_Solutions, 2024

    2024

  10. [18]

    Shortgpt: Layers in large language models are more redundant than you expect

    Xin Men, Mingyu Xu, Qingyu Zhang, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, and Weipeng Chen. Shortgpt: Layers in large language models are more redundant than you expect. ACL Findings, 2025

    2025

  11. [19]

    s1: Simple test-time scaling

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Cand \`e s, and Tatsunori Hashimoto. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393, 2025

    2025 arXiv

  12. [20]

    Compact language models via pruning and knowledge distillation

    Saurav Muralidharan, Sharath Turuvekere Sreenivas, Raviraj Joshi, Marcin Chochowski, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jan Kautz, and Pavlo Molchanov. Compact language models via pruning and knowledge distillation. Advances in Neural Information Processing Sy...

    2024

  13. [21]

    Gpqa: A graduate-level google-proof q&a benchmark

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. In COLM, 2024

    2024

  14. [22]

    A deeper look at depth pruning of llms

    Shoaib Ahmed Siddiqui, Xin Dong, Greg Heinrich, Thomas Breuel, Jan Kautz, David Krueger, and Pavlo Molchanov. A deeper look at depth pruning of llms. arXiv preprint arXiv:2407.16286, 2024

    2024 arXiv

  15. [23]

    Sleb: Streamlining llms through redundancy verification and elimination of transformer blocks

    Jiwon Song, Kyungseok Oh, Taesu Kim, Hyungjun Kim, Yulhwa Kim, and Jae-Joon Kim. Sleb: Streamlining llms through redundancy verification and elimination of transformer blocks. International Conference on Machine Learning, 2024

    2024

  16. [24]

    Gptailor: Large language model pruning through layer cutting and stitching

    Guinan Su, Li Shen, Lu Yin, Shiwei Liu, Yanwu Yang, and Jonas Geiping. Gptailor: Large language model pruning through layer cutting and stitching. arXiv preprint arXiv:2506.20480, 2025 a

    2025 arXiv

  17. [25]

    Rewiring experts on the fly: Continuous rerouting for better online adaptation in mixture-of-expert models

    Guinan Su, Yanwu Yang, Li Shen, Lu Yin, Shiwei Liu, and Jonas Geiping. Rewiring experts on the fly: Continuous rerouting for better online adaptation in mixture-of-expert models. arXiv preprint arXiv:2510.14853, 2025 b

    2025

  18. [26]

    The curse of depth in large language models

    Wenfang Sun, Xinyuan Song, Pengxiang Li, Lu Yin, Yefeng Zheng, and Shiwei Liu. The curse of depth in large language models. arXiv preprint arXiv:2502.05795, 2025

    2025

  19. [27]

    Challenging big-bench tasks and whether chain-of-thought can solve them

    Mirac Suzgun, Nathan Scales, Nathanael Sch \"a rli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, , and Jason Wei. Challenging big-bench tasks and whether chain-of-thought can solve them. ACL Findings, 2023

    2023

  20. [28]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599, 2025

    2025 arXiv

  21. [29]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  22. [30]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 2022

    2022

  23. [31]

    Monte carlo tree search boosts reasoning via iterative preference learning

    Yuxi Xie, Anirudh Goyal, Wenyue Zheng, Min-Yen Kan, Timothy P Lillicrap, Kenji Kawaguchi, and Michael Shieh. Monte carlo tree search boosts reasoning via iterative preference learning. arXiv preprint arXiv:2405.00451, 2024

    2024 arXiv

  24. [32]

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122, 2024 a

    2024 arXiv

  25. [33]

    Qwen3 technical report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025

    2025 arXiv

  26. [34]

    Laco: Large language model pruning via layer collapse

    Yifei Yang, Zouying Cao, and Hai Zhao. Laco: Large language model pruning via layer collapse. EMNLP Findings, 2024 b

    2024

  27. [35]

    Demystifying long chain-of-thought reasoning in llms

    Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain-of-thought reasoning in llms. International Conference on Learning Representations, 2025

    2025

  28. [36]

    Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019

    1905 arXiv

  29. [37]

    Glm-4.5: Agentic, reasoning, and coding (arc) foundation models

    Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, et al. Glm-4.5: Agentic, reasoning, and coding (arc) foundation models. arXiv preprint arXiv:2508.06471, 2025

    2025 arXiv

  30. [38]

    Openprm: Building open-domain process-based reward models with preference trees

    Kaiyan Zhang, Jiayuan Zhang, Haoxin Li, Xuekai Zhu, Ermo Hua, Xingtai Lv, Ning Ding, Biqing Qi, and Bowen Zhou. Openprm: Building open-domain process-based reward models with preference trees. In International Conference on Learning Representations, 2025

    2025

  31. [39]

    Finercut: Finer-grained interpretable layer pruning for large language models

    Yang Zhang, Yawei Li, Xinpeng Wang, Qianli Shen, Barbara Plank, Bernd Bischl, Mina Rezaei, and Kenji Kawaguchi. Finercut: Finer-grained interpretable layer pruning for large language models. arXiv preprint arXiv:2405.18218, 2024

    2024 arXiv

  32. [40]

    Texygen: A benchmarking platform for text generation models

    Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. Texygen: A benchmarking platform for text generation models. In The 41st international ACM SIGIR conference on research & development in information retrieval, pp.\ 1097--1100, 2018

    2018

  33. [41]

    Ttrl: Test-time reinforcement learning

    Yuxin Zuo, Kaiyan Zhang, Li Sheng, Shang Qu, Ganqu Cui, Xuekai Zhu, Haozhan Li, Yuchen Zhang, Xinwei Long, Ermo Hua, et al. Ttrl: Test-time reinforcement learning. arXiv preprint arXiv:2504.16084, 2025

    2025 arXiv

  34. [42]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  35. [43]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  36. [44]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  37. [45]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.