Pith. sign in

REVIEW 6 major objections 5 minor 69 references

ALPS: Attention Localization and Pruning Strategy for Efficient Alignment of Large Language Models

T0 review · 6 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ALPS restricts fine-tuning to the top 10% of attention heads, chosen by Wasserstein distance between base and task weight distributions, and reports gains over full fine-tuning and LoRA.

desk verdict Useful transfer result, but the '10% of attention parameters' claim undercounts GQA sharing and the method's efficiency story still needs a net-cost reality check. read the letter →

arxiv 2505.18799 v4 pith:C6SCKY3Y submitted 2025-05-24 cs.CL cs.AI

Hao Chen , Haoze Li , Zhiqing Xiao , Lirong Gao , Qi Zhang , Xiaomeng Hu , Ningtao Wang , Xing Fu
show 1 more author
Junbo Zhao
This is my paper
classification cs.CLcs.AI
keywords parameter-efficientfine-tuningattentionheadpruningWasserstein-1distanceLLMalignmenttask-sensitiveheadsknowledgeforgettingtransferabilitygrouped-query
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ALPS claims that aligning an LLM to a downstream task can be reduced to updating a small fraction of its attention heads, and that the right heads can be found from weight changes alone, without using activations from task data. The method compares a base model with a task-fine-tuned copy, computes for each head the Wasserstein-1 distance between the softmax-normalized distributions of the composed projection matrix $W_o = W_q W_k^\top W_v$, and keeps the top 10% of heads by this score while freezing the rest. On Llama-3.2-1B, Llama-3.2-3B, and Llama-3.1-8B, the resulting model beats full supervised fine-tuning, random head selection, layer-consistent selection, and LoRA on averaged general, math, and code benchmarks while updating far fewer parameters. The selected heads transfer to new datasets in the same task and cause less forgetting of general knowledge on MMLU and ARC-C. The central claim is that parameter-shift magnitude, measured this way, is a reliable and reusable proxy for task sensitivity.

What carries the argument

The carrying object is the Parameter Alignment Distribution (PAD) score. For each attention head, the query, key, and value projections are composed into $W_o^h = W_q^h (W_k^{g(h)})^\top W_v^{g(h)}$, normalized with a tempered softmax at temperature $\tau=1$, and compared between the base and task models with the Wasserstein-1 distance. This turns a high-dimensional weight shift into one scalar per head; Top-K selection on those scalars defines which heads receive gradient updates during fine-tuning. The appendix argues for W1 over KL divergence by showing that under a small translation $W_{o,T} = W_{o,B} + \Delta$, the W1 distance grows linearly with $\|\Delta\|$ while KL grows quadratically, so W1 does not underweight small task-specific shifts.

What would settle it

On a held-out dataset, rank heads by PAD and also measure the actual performance drop caused by ablating each head on the task (the paper's own $\Delta p(\theta_h)$ definition); if the PAD-selected top 10% shows no larger ablation drop than a random top 10%, the localization claim collapses.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that task alignment in a transformer LLM can be localized to a sparse set of attention heads, and that this set is identifiable from the weight matrices alone. Given a base model $\mathcal{M}_B$ and a task-fine-tuned model $\mathcal{M}_T$, ALPS forms per-head projection matrices $W_o^h = W_q^h (W_k^{g(h)})^\top W_v^{g(h)}$ (with key and value projections shared inside grouped-query attention groups), converts each to a probability distribution with a softmax, and scores the head by $s^{PAD}_h = W_1(P^h_B, P^h_T)$, the Wasserstein-1 distance between the base and task distributions. The top 10% of heads by this score are updated during fine-tuning; all other heads are frozen by masking their gradients to zero. The paper reports an average improvement of about 2 points over the full fine-tuning baseline across three tasks and three model scales, transfer of the selected heads to new datasets within the same task, and reduced knowledge forgetting. The load-bearing claim is that these PAD scores isolate heads whose parameters genuinely carry task-specific information.

Load-bearing premise

The method assumes that the size of an attention head's weight shift between base and task models marks how task-sensitive it is, and that this ranking stays valid on new datasets.

Editorial extensions

If this is right

  • Updating only the top 10% of attention heads yields average gains of +2.75, +1.73, and +1.25 points over full fine-tuning on the 1B, 3B, and 8B Llama-3 models.
  • Identified heads transfer across datasets within a task, so one localization run can be reused on later fine-tuning jobs instead of recomputing selections per dataset.
  • Training time drops by roughly a factor of five in the paper's settings: from 2.02 to 0.40 hours at 1B, 4.40 to 1.05 at 3B, and 7.67 to 2.18 at 8B.
  • Freezing non-selected heads preserves pre-trained knowledge better than full fine-tuning, random head selection, and LoRA on MMLU and ARC-C.
  • The retention-ratio sweep shows 10% and 30% heads outperform 50-100% retention, implying that pruning itself acts as a regularizer during alignment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The total cost of ALPS in practice includes obtaining a task-fine-tuned reference model; the paper argues this can be zero-cost when a public task model of the same architecture exists, but otherwise the full-SFT reference step belongs in the budget.
  • If PAD-selected heads transfer across datasets, a natural next test is whether they also transfer across model scales within a family; the paper's own heatmaps show different head patterns at 8B than at 1B/3B, so cross-scale transfer is not guaranteed.
  • The linear-versus-quadratic argument predicts that W1 will separate task-sensitive heads from insensitive ones earlier than KL as fine-tuning proceeds; a direct check would be to rank heads after progressively fewer training steps and see when the rankings stabilize.
  • One can validate the PAD surrogate directly on a small model by comparing its Top-10% heads with the heads whose ablation causes the largest measured performance drop; high agreement would strengthen the heuristic beyond the reported benchmark results.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 5 minor

Summary. The paper proposes ALPS, a two-stage method for parameter-efficient fine-tuning of Llama-3 LLMs. Stage 1 computes a 'parameter alignment distribution' (PAD) score for each attention head: the Wasserstein-1 distance between softmax-normalized W_o = W_q W_k^T W_v matrices of a base model and a task fine-tuned model. Stage 2 selects the top-K heads by this score and fine-tunes only those heads, freezing all others. Experiments on general, math, and code tasks with Llama-3.2-1B/3B and Llama-3.1-8B claim that ALPS updates only 10% of attention parameters while achieving roughly a 2% average improvement over full SFT, and that the selected heads transfer across datasets and reduce knowledge forgetting on MMLU/ARC-C.

Significance. If the empirical claims hold, ALPS is a useful contribution: it demonstrates that a weight-shift-based head selection can outperform full fine-tuning at a fraction of the trainable parameters, and the transfer and forgetting results are interesting. The paper includes ablations of the score metric and retention ratio, low-cost auxiliary-model variants in Appendix C, and a public code link. However, the headline parameter counting overstates sparsity under GQA, the evaluation lacks variance estimates, and the default ratio is chosen post hoc, so the quantitative claims need substantial verification before publication.

major comments (6)
  1. [Section 3.3, Eq. (8), Table 1] The claimed '10% of attention parameters' is a head-count claim, not a parameter-count claim. In the GQA models used here, K/V projections are shared within groups, so freezing a query head does not freeze that group's K/V when any other head in the group is selected. For Llama-3.1-8B (32 query heads, 8 KV groups, group size 4), a random 10% query-head selection touches about 34% of KV groups, giving roughly (0.1 Q + 0.34*0.5 Q) / 1.5 Q ≈ 18% of attention parameters, not 10%. The PAD-based selection is non-uniform, so the actual fraction differs and is never reported. Since the LoRA baseline's '~10%' is a trainable-parameter percentage, Table 1 compares unlike quantities. Please report actual trainable parameter counts and revise the abstract and headline accordingly.
  2. [Section 4.2, Table 1] All numerical results are single runs with no standard errors or significance tests. The reported gains over w/ full are 2.75, 1.73, and 1.25 points for the three model sizes, and the '2% improvement' in the abstract is an average over heterogeneous benchmarks (e.g., IFEval, GPQA, GSM8K, MATH, HumanEval, MBPP). These differences are small relative to the known variance of 0-shot GPQA and pass@1 code benchmarks. To support the 'consistently outperforms' claim, the authors should provide multiple seeds, variance estimates, and a statistical comparison.
  3. [Section 4.3, Table 3] The default retention ratio r=10% is selected after inspecting the same benchmarks used in the main results. The text says 'we adopt 10% as our default setting' based on Table 3, which evaluates on the same test benchmarks as Table 1. This makes the headline comparison post hoc relative to the test data. Please either select r on a held-out validation split or present all ratios in the main results with a clear disclosure that 10% was chosen after seeing the full benchmark table.
  4. [Appendix C and Table 11] The efficiency claim in Table 11 reports only the ALPS fine-tuning time (e.g., 0.40 hours for the 1B model) and omits the cost of obtaining the task fine-tuned model M_T. In the main experiments, M_T is a full SFT model trained on the same task data, so the true total compute is larger than reported. Appendix C addresses this with 'readily available' and low-cost M_T variants, but those results are only shown for 1B and 3B models and do not enter the paper's abstract-level efficiency claim. Please report the total compute budget, including M_T acquisition, for the main setup.
  5. [Section 3.3, Eq. (7); Section 4.5] The PAD score uses M_T, a model fully fine-tuned on the same task data, and ALPS is then evaluated on that same task. The selected heads are therefore partly an oracle for the target task rather than a purely weight-based predictor independent of task data. The transfer experiments in Table 4 mitigate this concern for cross-dataset reuse, but the paper's claim that ALPS 'avoids data dependency' (Abstract, Section 5) overstates what the main experiments show. Please clarify that the localization stage requires access to a task-related model and that the data-independence claim applies only to the head-selection mechanism, not to the overall pipeline.
  6. [Appendix E, Eqs. (10)-(12)] The theoretical justification for PAD is not rigorous. Equation (12) asserts that W1 ≈ ||Δ|| under a pure translation W_o,T = W_o,B + Δ, but W_o is a product W_q W_k^T W_v and the tempered softmax normalization in Eq. (6) is nonlinear, so the relation does not follow. The paper should present PAD as a heuristic motivated by the translation intuition, not as a derived approximation with a linear-scaling guarantee.
minor comments (5)
  1. [Abstract and Table 1] The abstract's '2% performance improvement' is not a single consistent statistic from Table 1, where the improvements are 2.75%, 1.73%, and 1.25% for the three model sizes. Please state the exact comparison and the number of benchmarks averaged.
  2. [Section 3.3, Eq. (8) and Eq. (9)] The notation is inconsistent: Eq. (8) writes θ_h = {W_q^(h), W_k^(h), W_v^(h)}, but Eq. (1) uses shared K/V indices ⌈hg/n⌉; Eq. (9) refers to H_S while Algorithm 1 defines H_T. Please unify the notation and define θ_h exactly as the set of parameters whose gradients are masked.
  3. [Section 4.5 and Table 4] The transfer experiment reports only 'General', 'Math', and 'Code' columns without specifying which benchmarks or metrics produced those numbers. Appendix G, Table 12 repeats the same ambiguity. Please name the evaluation benchmarks (e.g., Alpaca, Camel-math, CodeAlpaca) and report per-benchmark scores.
  4. [Algorithm 1 and Eq. (6)] Algorithm 1 calls Softmax(W_o,B) and Softmax(W_o,T) without the temperature τ, while Eq. (6) defines the tempered softmax and the text sets τ=1. Please make the pseudocode consistent with the equation.
  5. [Figure 4] The x-axis label 'Efficiency Ratio' is not defined in the caption or the text. It should be labeled with the retention ratio r or the actual trainable parameter fraction, and the shaded 'efficiency ratio' region needs an explicit definition.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PAD is an independent weight-shift heuristic, validated by transfer and low-cost task-model experiments.

full rationale

The derivation chain is not circular. ALPS computes the PAD score as a Wasserstein-1 distance between softmax-normalized composite attention matrices of the base model and a task fine-tuned model (Eqs. 5-7), selects the top-K heads by that score, and then fine-tunes only those heads on the task data. The head-selection heuristic is not equivalent to the reported result: no parameter of the PAD score is fitted to the evaluation benchmarks, and the selected heads are not defined as the heads that make ALPS win; they are defined by an independent weight-shift statistic. The use of a task fine-tuned model M_T is a potential source of information leakage, but the paper directly addresses this in Appendix C with low-cost or readily-available M_T variants (10% data, 10% attention parameters) that still match performance, and Table 4 shows the selected heads transfer to new datasets within the same task. These experiments break any loop in which the final results would be forced by the localization step. The main results are empirical comparisons against full SFT, random/LC selections, and LoRA on external benchmarks, so the central claim is self-contained. The only self-citation (Chen et al. 2023a in Related Work) is not load-bearing. No equation reduces to its own input, and no fitted parameter is renamed as a prediction.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The method rests on a small number of domain heuristics rather than mathematical axioms. The most important is that the magnitude of weight shift between a base model and a task-fine-tuned model marks the heads whose updates matter for that task; this is assumed in Eq. 7 and only empirically supported. The efficiency accounting also assumes a task-related model is available or cheaply obtainable, which is true for some domains but not for the main Llama-3.2 experiments. No new entities are introduced.

free parameters (2)
  • retention ratio r = 0.10 (default)
    Chosen after the ablation in Table 3 on the same benchmarks; not predicted a priori.
  • softmax temperature tau = 1
    Set to 1 for simplicity in Eq. 6; arbitrary choice that affects the PAD values but not the central ranking method.
assumptions (5)
  • domain assumption Task sensitivity equals parameter-shift magnitude.
    Sec. 3.3 and Fig. 1 assert that task parameters lead to task-sensitive heads; the PAD ranking in Eq. 7 assumes larger W1 distance implies more task-relevant head.
  • domain assumption The product W_o = W_q W_k^T W_v summarizes a head's transformation.
    Eq. 5 defines W_o following Kobayashi et al. (2020); the paper uses it as the basis for all distributions without validating that this summary preserves task-relevant signal.
  • domain assumption Softmax conversion and W1 distance are meaningful for weight matrices.
    Eq. 6-7; probability interpretation of raw weight matrices is a modeling choice, and the Appendix E translation argument is not rigorous for softmax outputs.
  • domain assumption Freezing non-selected heads prevents interference while preserving pretrained knowledge.
    Eq. 8-9; this is the pruning assumption, tested empirically in Tables 3 and 5 but not proven.
  • domain assumption Selecting 10% of query heads corresponds to updating 10% of attention parameters under GQA.
    Used in Table 1 'Attn%' and the abstract; with grouped-query attention, K/V matrices are shared by groups of query heads, so updating any head in a group updates the group KV, and the actual updated parameter fraction depends on the distribution of selected heads across groups.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ALPS: Attention Localization and Pruning Strategy for Efficient Alignment of Large Language Models." pith.science (2026). https://pith.science/paper/C6SCKY3Y

@misc{pith2026250518799,
  author       = {Pith},
  title        = {Pith review of: ALPS: Attention Localization and Pruning Strategy for Efficient Alignment of Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C6SCKY3Y}},
  note         = {Machine review of arXiv:2505.18799}
}
read the original abstract

Aligning general-purpose large language models (LLMs) to downstream tasks often incurs significant training adjustment costs. Prior research has explored various avenues to enhance alignment efficiency, primarily through minimal-data training or data-driven activations to identify key attention heads. However, these approaches inherently introduce data dependency, which hinders generalization and reusability. To address this issue and enhance model alignment efficiency, we propose the Attention Localization and Pruning Strategy (ALPS), an efficient algorithm that localizes the most task-sensitive attention heads and prunes by restricting attention training updates to these heads, thereby reducing alignment costs. Experimental results demonstrate that our method activates only 10% of attention parameters during fine-tuning while achieving a 2% performance improvement over baselines on three tasks. Moreover, the identified task-specific heads are transferable across datasets and mitigate knowledge forgetting. Our work and findings provide a novel perspective on efficient LLM alignment. The code is available at https://github.com/VoiceBeer/ALPS.

Figures

Figures reproduced from arXiv: 2505.18799 by the authors.

Figure 1
Figure 1. Process of localizing task-sensitive attention [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Preliminary results comparing ALPS against baselines across different LLM scales (Left), and downstream tasks (Right). ALPS consistently outperforms other methods, demonstrating its efficiency in diverse settings. look the intrinsic functionality of the model weight parameters, which inherently encode task-relevant information (Zhong et al., 2022; Tam et al., 2024), and can be leveraged to identify task-sensitive at… view at source ↗
Figure 3
Figure 3. Overview of ALPS framework. a) Given a base model MB and a task fine-tuned model MT , we extract their attention heads, compute weight distributions using softmax, and calculate their PAD score s P AD h . The Top-K attention heads are then selected based on S P AD, while b) pruning the remaining heads by freezing their gradient updates during fine-tuning. The resulting model MALP S outperforms MT in both performance… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Ablation study on attention head selection [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 6
Figure 6. Figure 6: Evaluation on transferability of ALPS with a new set of general, math, and code datasets. Results with † correspond to results from [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: 3D visualization of data distribution for Ultra [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Heatmaps of task-sensitive attention heads (10%) selected by our method on Llama-3.2-1B, Llama-3.2-3B, [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

69 extracted references · 4 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebr \'o n, and Sumit Sanghai. 2023. Gqa: Training generalized multi-query transformer models from multi-head checkpoints. arXiv preprint arXiv:2305.13245

    arXiv 2023

  3. [3]

    Martin Arjovsky, Soumith Chintala, and Léon Bottou. 2017. https://arxiv.org/abs/1701.07875 Wasserstein gan . Preprint, arXiv:1701.07875

    arXiv 2017

  4. [4]

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021. Program synthesis with large language models. arXiv preprint arXiv:2108.07732

    arXiv 2021

  5. [5]

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. 2024. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15(3):1--45

    2024

  6. [6]

    Sahil Chaudhary. 2023. Code alpaca: An instruction-following llama model for code generation. https://github.com/sahil280114/codealpaca

    2023

  7. [7]

    Hao Chen, Yiming Zhang, Qi Zhang, Hantao Yang, Xiaomeng Hu, Xuetao Ma, Yifan Yanggong, and Junbo Zhao. 2023 a . Maybe only 0.5\ arXiv preprint arXiv:2305.09246

    arXiv 2023

  8. [8]

    Jianhui Chen, Xiaozhi Wang, Zijun Yao, Yushi Bai, Lei Hou, and Juanzi Li. 2024. Finding safety neurons in large language models. arXiv preprint arXiv:2406.14144

    arXiv 2024

Show all 69 references
  1. [9]

    Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, et al. 2023 b . Alpagasus: Training a better alpaca with fewer data. arXiv preprint arXiv:2307.08701

    2023 arXiv

  2. [10]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374

    2021 arXiv

  3. [11]

    Kevin Clark. 2019. What does bert look at? an analysis of bert’s attention. arXiv preprint arXiv:1906.04341

    2019 arXiv

  4. [12]

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457

    2018 arXiv

  5. [13]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    2021 arXiv

  6. [14]

    Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. 2023. https://arxiv.org/abs/2305.14233 Enhancing chat language models by scaling high-quality instructional conversations . Preprint, arXiv:2305.14233

    2023 arXiv

  7. [15]

    Yu Fu, Zefan Cai, Abedelkadir Asi, Wayne Xiong, Yue Dong, and Wen Xiao. 2024. Not all heads matter: A head-level kv cache compression method with integrated retrieval and reasoning. arXiv preprint arXiv:2410.19258

    2024

  8. [16]

    Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Muennighoff, et al. 2024. https://doi.org/10.5281/zenodo.12608602 A framework for few-shot language model ...

    2024 doi

  9. [17]

    Atticus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts. 2021. Causal abstractions of neural networks. Advances in Neural Information Processing Systems, 34:9574--9586

    2021

  10. [18]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  11. [19]

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. 2024. Deepseek-coder: When the large language model meets programming--the rise of code intelligence. arXiv preprint arXiv:2401.14196

    2024 arXiv

  12. [20]

    Wes Gurnee, Neel Nanda, Matthew Pauly, Katherine Harvey, Dmitrii Troitskii, and Dimitris Bertsimas. 2023. Finding neurons in a haystack: Case studies with sparse probing. arXiv preprint arXiv:2305.01610

    2023 arXiv

  13. [21]

    Shwai He, Guoheng Sun, Zheyu Shen, and Ang Li. 2024. What matters in transformers? not all attention is needed. arXiv preprint arXiv:2406.15786

    2024 arXiv

  14. [22]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300

    2020 arXiv

  15. [23]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874

    2021 arXiv

  16. [24]

    Geoffrey Hinton. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531

    2015 arXiv

  17. [25]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685

    2021 arXiv

  18. [26]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. 2024. Qwen2.5-coder technical report. arXiv preprint arXiv:2409.12186

    2024 arXiv

  19. [27]

    Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, et al. 2018. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2704--2713

    2018

  20. [28]

    Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, and Kentaro Inui. 2020. Attention is not only a weight: Analyzing transformers with vector norms. arXiv preprint arXiv:2004.10102

    2020 arXiv

  21. [29]

    Solomon Kullback and Richard A Leibler. 1951. On information and sufficiency. The annals of mathematical statistics, 22(1):79--86

    1951

  22. [30]

    Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023 a . https://arxiv.org/abs/2303.17760 Camel: Communicative agents for "mind" exploration of large scale language model society . Preprint, arXiv:2303.17760

    2023 arXiv

  23. [31]

    Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Omer Levy, Luke Zettlemoyer, Jason Weston, and Mike Lewis. 2023 b . Self-alignment with instruction backtranslation. arXiv preprint arXiv:2308.06259

    2023 arXiv

  24. [32]

    David Lindner, János Kramár, Sebastian Farquhar, Matthew Rahtz, Thomas McGrath, and Vladimir Mikulik. 2023. https://arxiv.org/abs/2301.05062 Tracr: Compiled transformers as a laboratory for interpretability . Preprint, arXiv:2301.05062

    2023 arXiv

  25. [33]

    Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, et al. 2024. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model. arXiv preprint arXiv:2405.04434

    2024 arXiv

  26. [34]

    Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2023. https://openreview.net/forum?id=1qvx610Cu7 Is your code generated by chat GPT really correct? rigorous evaluation of large language models for code generation . In Thirty-seventh Conference on Neural Informa...

    2023

  27. [35]

    I Loshchilov. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101

    2017 arXiv

  28. [36]

    meta llama. 2023. Llama recipes: Examples to get started using the llama models from meta. https://github.com/meta-llama/llama-cookbook/tree/archive-main

    2023

  29. [37]

    Paul Michel, Omer Levy, and Graham Neubig. 2019. Are sixteen heads really better than one? Advances in neural information processing systems, 32

    2019

  30. [38]

    OpenAI. 2025. https://openai.com/index/openai-o3-mini/ Openai o3-mini

    2025

  31. [39]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2023. Gpqa: A graduate-level google-proof q&a benchmark. arXiv preprint arXiv:2311.12022

    2023 arXiv

  32. [40]

    Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. 2023. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950

    2023 arXiv

  33. [41]

    Robert Schmirler, Michael Heinzinger, and Burkhard Rost. 2024. Fine-tuning protein language models boosts predictions across diverse tasks. Nature Communications, 15(1):7407

    2024

  34. [42]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300

    2024 arXiv

  35. [43]

    Guangyuan Shi, Zexin Lu, Xiaoyu Dong, Wenlong Zhang, Xuanyu Zhang, Yujie Feng, and Xiao-Ming Wu. 2024. Understanding layer significance in llm alignment. arXiv preprint arXiv:2410.17875

    2024 arXiv

  36. [44]

    Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. 2023. Large language models encode clinical knowledge. Nature, 620(7972):172--180

    2023

  37. [45]

    Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen R Pfohl, Heather Cole-Lewis, et al. 2025. Toward expert-level medical question answering with large language models. Nature Medicine, pages 1--8

    2025

  38. [46]

    Derek Tam, Mohit Bansal, and Colin Raffel. 2024. Merging by matching models in task parameter subspaces. Transactions on Machine Learning Research

    2024

  39. [47]

    Hanlin Tang, Yang Lin, Jing Lin, Qingsen Han, Shikuan Hong, Yiwu Yao, and Gongyi Wang. 2024. Razorattention: Efficient kv cache compression through retrieval heads. arXiv preprint arXiv:2407.15891

    2024 arXiv

  40. [48]

    Hashimoto

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca

    2023

  41. [49]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  42. [50]

    Leonid Nisonovich Vaserstein. 1969. Markov processes over denumerable products of spaces, describing large systems of automata. Problemy Peredachi Informatsii, 5(3):64--72

    1969

  43. [51]

    Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. 2019. Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. arXiv preprint arXiv:1905.09418

    2019 arXiv

  44. [52]

    Zhongwei Wan, Xin Wang, Che Liu, Samiul Alam, Yu Zheng, Jiachen Liu, Zhongnan Qu, Shen Yan, Yi Zhu, Quanlu Zhang, et al. 2023. Efficient large language models: A survey. arXiv preprint arXiv:2312.03863

    2023 arXiv

  45. [53]

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022. Self-instruct: Aligning language models with self-generated instructions. arXiv preprint arXiv:2212.10560

    2022 arXiv

  46. [54]

    Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang. 2023. Magicoder: Source code is all you need. arXiv preprint arXiv:2312.02120

    2023 arXiv

  47. [55]

    Wenhao Wu, Yizhong Wang, Guangxuan Xiao, Hao Peng, and Yao Fu. 2024. Retrieval head mechanistically explains long-context factuality. arXiv preprint arXiv:2404.15574

    2024 arXiv

  48. [56]

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304.12244

    2023 arXiv

  49. [57]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, et al. 2025. https://arxiv.org/abs/2412.15115 Qwen2.5 technical report . Preprint, arXiv:2412.15115

    2025 arXiv

  50. [58]

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al. 2024. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122

    2024 arXiv

  51. [59]

    Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. 2023. Mammoth: Building math generalist models through hybrid instruction tuning. arXiv preprint arXiv:2309.05653

    2023 arXiv

  52. [60]

    Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2024. Explainability for large language models: A survey. ACM Transactions on Intelligent Systems and Technology, 15(2):1--38

    2024

  53. [61]

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223

    2023 arXiv

  54. [62]

    Zifan Zheng, Yezhaohui Wang, Yuxin Huang, Shichao Song, Mingchuan Yang, Bo Tang, Feiyu Xiong, and Zhiyu Li. 2024. Attention heads of large language models: A survey. arXiv preprint arXiv:2409.03752

    2024 arXiv

  55. [63]

    Weishun Zhong, Ben Sorscher, Daniel Lee, and Haim Sompolinsky. 2022. A theory of weight distribution-constrained learning. Advances in Neural Information Processing Systems, 35:14113--14127

    2022

  56. [64]

    Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. 2024 a . Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36

    2024

  57. [65]

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023. Instruction-following evaluation for large language models. arXiv preprint arXiv:2311.07911

    2023 arXiv

  58. [66]

    Zhenhong Zhou, Haiyang Yu, Xinghua Zhang, Rongwu Xu, Fei Huang, Kun Wang, Yang Liu, Junfeng Fang, and Yongbin Li. 2024 b . On the role of attention heads in large language model safety. arXiv preprint arXiv:2410.13708

    2024 arXiv

  59. [67]

    Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, et al. 2023. Representation engineering: A top-down approach to ai transparency. arXiv preprint arXiv:2310.01405

    2023 arXiv

  60. [68]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  61. [69]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.