Amplify Adjacent Token Differences: Enhancing Long Chain-of-Thought Reasoning with Shift-FFN

Bingning Wang; Fangyu Lei; Guang Liu; Jun Zhao; Kang Liu; Mingyu Xu; Shizhu He; Wangtao Sun; Xiangrong Zeng; Yao Xu

REVIEW 3 major objections 9 minor 50 references

Amplify Adjacent Token Differences: Enhancing Long Chain-of-Thought Reasoning with Shift-FFN

T0 review · 3 major / 9 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A token-shift edit before each feedforward layer reduces cyclical reasoning in long-CoT-distilled models and lifts accuracy beyond full fine-tuning.

desk verdict A real, incremental empirical result on reducing repetition loops in long-CoT distillation, wrapped in a mechanistic story that does not survive contact with the math. read the letter →

arxiv 2505.17153 v1 pith:6MUT3AH2 submitted 2025-05-22 cs.CL cs.AI

Yao Xu , Mingyu Xu , Fangyu Lei , Wangtao Sun , Xiangrong Zeng , Bingning Wang , Guang Liu , Shizhu He

show 2 more authors

Jun Zhao Kang Liu

This is my paper

classification cs.CLcs.AI

keywords LongChain-of-ThoughtCyclicalReasoningShift-FFNadjacenttokenrepresentationsLoRAfine-tuningparameter-efficientmathematicalrepresentationediting

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper targets a failure mode of distilled long chain-of-thought reasoning: after fine-tuning on long reasoning traces, student models often start repeating earlier inference steps until they hit the output-length cap, a behavior the authors call Cyclical Reasoning. They report that such looping correlates with small relative differences between adjacent token hidden states, measured by a quantity $M(X)$. To break the loop, they insert an 'Editor' module before each feedforward layer that mixes the previous token's representation into the current one, an architecture called Shift-FFN, which increases $M(X)$. On four math benchmarks, LoRA plus Shift-FFN outperforms both standard LoRA and full fine-tuning, e.g., Qwen2.5-7B average accuracy 51.2% with a 12.7% length-exceeded rate versus 50.4%/15.0% for LoRA and 49.3%/24.7% for full fine-tuning. A sympathetic reader should care because this points to an architecture-level, complementary fix for a data-driven distillation problem.

What carries the argument

The load-bearing piece is the Editor module placed before each FFN. For the hidden state of token $i$ after attention, it computes $f_s(x_{i-1}, x_i) = W_c\big[\mathrm{ReLU}(W_b[x_{i-1}; x_i]) \odot (W_a x_{i-1})\big]$, then feeds $\mathrm{FFN}(x_i + f_s(\cdot))$; $W_c$ is zero-initialized so training starts from the original FFN. The diagnostic object is $M(X)$, the mean over layers and positions of $\|x_i^l - x_{i-1}^l\|_2 / \|x_{i-1}^l\|_2$, which the paper uses to tie adjacent-token divergence to looping. The paper interprets Shift-FFN analytically as turning the FFN into a multi-key memory and as adding a correction term to the next layer's attention scores that depends on the previous tokens. The editor's gating, which combines both the previous and current token, is shown by ablation to be essential.

What would settle it

Generate two variants of a fine-tuned model with matching repetition rates but different $M(X)$ values (e.g., by adjusting the editor's gain), and check whether the higher-$M(X)$ variant still loops; alternatively, compute $M(X)$ on length-exceeded outputs with repeated text segments removed—if the $M(X)$ deficit disappears once repetition is excised, the correlation is an artifact of repetition rather than a cause.

Watch

Extended reading notes

Core claim

The paper's central claim is that cyclical reasoning in long-CoT-distilled models is tied to insufficient divergence between adjacent token representations, and that deliberately amplifying that divergence at the architecture level reduces the loop and improves accuracy. Concretely, the authors define the mean relative change $M(X)$ between neighboring hidden states, observe that length-exceeded outputs have lower $M(X)$, then introduce Shift-FFN, which computes $\mathrm{FFN}(x_i + f_s(x_{i-1}, x_i))$ with a gated editor, and show that this addition raises $M(X)$, lowers the percentage of outputs that exceed the 32k limit, and lifts pass@K. With Qwen2.5-7B, LoRA+Shift-FFN at rank 256 reaches 51.2% average accuracy across AIME24, AMC23, MATH500 and OlympiadBench while full fine-tuning reaches 49.3%, and the length-exceeded rate drops from 24.7% to 12.7%. The same pattern holds across model sizes (3B and 8B), data sizes from 10k to 80k training examples, and sampling temperatures.

Load-bearing premise

The argument assumes that widening the average relative change between adjacent hidden states is what prevents cyclical reasoning, rather than merely being a side effect of producing less repetitive text; the paper's evidence for this link is correlational, as it acknowledges.

Editorial extensions

If this is right

LoRA fine-tuning on long CoT needs higher rank than on short tasks: at rank 32 LoRA matches full fine-tuning on short CoT but lags on long CoT, and raising the rank from 128 to 256 lowers the length-exceeded rate from 23.7% to 15.0% on Qwen2.5-7B.
Shift-FFN improves accuracy and reduces length-exceeded outputs across all tested configurations: with 20k training samples Qwen2.5-7B LoRA+Shift-FFN (rank 256) reaches 51.2% average accuracy and a 12.7% length-exceeded rate, and the improvement persists at 10k, 40k and 80k training samples.
At equal parameter counts, Shift-FFN beats simply increasing LoRA rank: raising rank from 256 to 296 adds only 0.2% average accuracy while raising the length-exceeded rate, while Shift-FFN adds 0.8% and cuts that rate.
Shift-FFN improves exploration efficiency: on AIME24 pass@256 rises from 70.0% to 76.7%, consistent with fewer wasted loops.
Lower sampling temperatures aggravate cyclical reasoning, and Shift-FFN gives the largest gains wherever the baseline length-exceeded percentage is highest.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the $M(X)$-versus-looping link is causal, $M(X)$ could serve as a cheap early-stop or monitoring signal during long-CoT generation, letting a system halt a rollout before it burns the length budget.
The same architecture-level fix may transfer to other long-sequence failure modes beyond math, such as code generation or agent rollouts that stall in repetitive loops, since the loop mechanism is described in terms of adjacent hidden states rather than domain content.
Shift-FFN is complementary to data-side distillation improvements; combining it with redundancy-removal or structured-reasoning data curation could yield additive gains rather than overlapping ones.
A clean test would be to scale Shift-FFN to larger models and full training runs; the paper leaves this open, so whether the effect survives at 32B scale and 1M+ examples remains unknown.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know two things up front. The empirical claim is probably real: adding this gated shift module to LoRA consistently reduces repetition-driven length exceedances and nudges accuracy up on four math benchmarks across three base models, with a parameter-matched control. The mechanistic story attached to it is not supported: the theoretical derivation in Section 3.3 uses an invalid ReLU additivity step, and the paper's own diagnostic M(X) is partly circular.

What is actually new: the specific editor module (concatenating xi-1 and xi, gating with Wa xi-1, zero-init Wc) is not in the cited prior work, and the empirical finding that long-CoT distillation needs higher LoRA rank is useful. The paper also deserves credit for the parameter-matched comparison in Table 2, the ablation study, the pass@K and temperature sweeps, and for releasing data and code. The main tables are consistent, and the failure mode they target (models looping until the 32k limit) is real and practically important.

The soft spots are real but not all equal. No confidence intervals or significance tests appear anywhere; some headline gains are 0.3 to 0.8 percentage points, so the robustness of the improvement is less clear than the prose suggests. The bigger problem is the theory. Equations 7 and 8 treat ReLU as additive, writing sigma(a+b) as sigma(a)+sigma(b), which is simply false. That means the 'multi-key retrieval' and 'augmented attention score' interpretations do not follow, and the section reads as post-hoc storytelling rather than derivation. The stress-test note is right about this. Also, M(X) is the average adjacent-token difference, so repeated text has low M(X) almost by definition; Figure 6 showing high M(X) with low PE is partly a restatement of the phenomenon, not independent evidence. The authors themselves admit in Appendix A that they did not investigate deeper causes. This does not sink the empirical result—accuracy is measured independently of M(X)—but it does mean the title's causal claim ('amplify adjacent token differences' to suppress cyclical reasoning) is not established.

The reader's conditional verdict is about right. I would not desk-reject this. It deserves a serious referee, with the expectation that the mechanism section be rewritten or removed, and that the empirical claims be tested for significance. Who is this for? People working on long-CoT distillation, PEFT, and repetition failure modes. I'd cite it if I worked in that area.

Referee Report

3 major / 9 minor

Summary. This manuscript studies cyclical reasoning, i.e., length-exceeded and repetitive generations, that arises when distilling long chain-of-thought (CoT) data into smaller student models with full fine-tuning or LoRA. The authors define M(X), the mean relative change between adjacent token representations across layers, and report that length-exceeded outputs have lower M(X) than normal outputs and that higher LoRA ranks raise M(X) while lowering the length-exceeded percentage PE. Based on this correlational observation, they propose Shift-FFN, which adds a gated function of the previous token's representation to the current token's representation before each FFN. Experiments on AIME24, AMC23, MATH500, and OlympiadBench with Qwen2.5-3B, Qwen2.5-7B, and Llama3.1-8B show that LoRA+Shift-FFN improves average accuracy over LoRA, including a parameter-matched LoRA r=296 control, and over full fine-tuning, while reducing PE; ablations, pass@K curves, temperature sweeps, and data-size scaling are also reported. Section 3.3 provides a theoretical interpretation of Shift-FFN as multi-key retrieval with augmented attention scores.

Significance. The empirical contribution is practically valuable and credible if the numbers hold: the paper documents a real failure mode of low-rank LoRA in long-CoT distillation, proposes a cheap architectural fix, and validates it with a parameter-matched control (Table 2), component ablations (Table 3), pass@K comparisons (Figure 5), and data-size scaling (Figure 7). The public code and data release supports reproducibility. The correlational finding that low adjacent-token differences accompany cyclical reasoning is interesting, but the stronger causal claim (amplifying M(X) suppresses cyclical reasoning) is not established, and the Section 3.3 derivation rests on an invalid ReLU linearization; both are correctable by reframing or additional experiments rather than invalidating the measured improvements.

major comments (3)

[Section 3.3, Eqs. (7)-(8)] The derivation assumes that the ReLU is additive. Eq. (7) writes w_{i,j} = sigma(x_i^T k_j) + sigma(hat{x}_{i-1}^T k_j), and Eq. (8) writes W_down[sigma(W_up(x_i + hat{x}_{i-1}))] = h_i + hat{h}_i; both equalities require sigma(a+b) = sigma(a) + sigma(b), which is not stated and is false in general, holding only under per-component sign conditions. The same assumption underlies the attention correction terms in Eq. (11). Additionally, the simplification from Eq. (5) to f_s = W_s x_{i-1} drops the gate and the ReLU inside the Editor without validation. Consequently, the 'multi-key retrieval' and 'augmented attention score' interpretations do not follow from the mathematics as written. I recommend stating the linearization explicitly with its conditions and an empirical check of the residual, or recasting Section 3.3 as informal intuition and removing the asserted equalities.
[Sections 3.1, 4.4, Appendix A] The causal claim that amplifying M(X) suppresses cyclical reasoning is supported only by correlational evidence. The Figure 6 comparison varies LoRA rank and architecture simultaneously, so parameter count and optimization trajectory are confounded with M(X); and because repeated text has, by construction, similar adjacent hidden states, low M(X) may be a signature rather than a cause of reduced repetition. The truncation and de-duplication preprocessing in Section 3.1 partially mitigates the tautology reading for Figure 2, and the Table 2 parameter-matched control helps with the parameter-count confound, but neither establishes causal direction; Appendix A explicitly concedes that no deeper analysis was performed. Since the title and design principle rest on this mechanism, the authors should either run an intervention that manipulates adjacent-token differences independently of the architecture (e.g., scaling the shift at inference or adding controlled perturbations) and measure PE, or reframe the mechanism throughout the abstract, Section 3.1, and Section 4.4 as a hypothesis.
[Table 1, Section 4.2] The claim that Shift-FFN 'consistently yields performance improvements across all settings' is contradicted by the Llama3.1-8B AIME24 rows of Table 1, where accuracy drops from 4.6 to 3.6 at r=128 and from 5.4 to 5.1 at r=256. Several average gains are small (0.3 to 0.8 percentage points), and no confidence intervals or significance tests are reported anywhere; with only 4 runs for MATH500 and OlympiadBench and 32 for AIME24 and AMC23, a 0.8-point gain corresponds to between roughly 8 and 16 correct samples per task, which is within plausible sampling noise. Please report per-configuration variance or bootstrap intervals for both accuracy and PE, and soften the 'consistent' wording to describe the average and majority-of-configurations direction of the effect.

minor comments (9)

[Section 4.1, Training Setup] The training setup names Llama3.2-8B-Instruct, while the abstract, Table 1, and the rest of the paper refer to Llama3.1-8B; please correct the model name and confirm which model the reported results correspond to.
[Table 1] The Qwen2.5-7B Full row average is inconsistent with its components: (20.0 + 58.1 + 78.7 + 42.1)/4 = 49.7, not 49.3. The Llama3.1-8B r=256 rows also round inconsistently (33.75 should be 33.8 and 33.925 should be 33.9). Please verify all averages.
[Figure 6 vs Table 1] For AIME24, the PE values plotted for LoRA r=256 and LoRA+Shift-FFN (30.4% and 25.1%) differ from Table 1 (28.6% and 23.5%) for the same configurations; please state the evaluation setting used in Figure 6 or reconcile the numbers.
[Section 4.2] The statement that LoRA (r=256) 'reduces the Cyclical Reasoning ratio by 12% compared to full fine-tuning' is numerically unclear: full fine-tuning PE is 24.7% and LoRA r=256 is 15.0%, a drop of 9.7 percentage points (about 39% relative). Please restate the figure in percentage points.
[Section 3.2, Eq. (4)] The notation f_x is introduced in the sentence following Eq. (4), but the shift function is defined and used everywhere else as f_s; please unify the notation.
[Sections 3.3 and Appendix B] Because Qwen2.5 and Llama-3 use GLU-FFN, please state explicitly whether the shifted input x_i + f_s(x_{i-1}, x_i) is fed to both the gate and up-projection branches, and explain how that interacts with the memory-view derivation in Section 3.3, which assumes the standard two-layer FFN.
[Section 4.2] The assertion that the 32k generation limit is 'ample' for generating correct answers is used to justify PE as a proxy for cyclical reasoning; please support it with the length distribution of correctly answered outputs or qualify the claim.
[Section 3.3; Table 2 caption] Minor typos: 'From simplicity' should be 'For simplicity'; the Table 2 caption 'w.t./w.o.' should be 'with/without'; and '8 x 80G L20 GPUs' in Section 4.1 should be checked because the L20 typically has 48GB of memory.
[Section 4.1] Please report the random seeds used for the 4-run and 32-run averages so that the Table 1 numbers are reproducible.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the accuracy and length-exceeded claims are measured on external math benchmarks and on raw generation outcomes, not on the paper's own fitted diagnostic.

full rationale

The paper's central empirical claim is self-contained: LoRA+Shift-FFN accuracy and Length Exceeded Percentage in Table 1 are evaluated on AIME24, AMC23, MATH500, and OlympiadBench, with no parameter fitted to those target numbers, and the Shift-FFN ablation and rank-matched comparison in Table 2 provide independent controls. The mechanistic story around the M(X) diagnostic is not a circular reduction: M(X) is a post-hoc statistic, not a training objective, and no equation defines PE as a function of M(X) or defines M(X) from PE. Figures 2 and 6 establish only correlations, and Appendix A explicitly concedes that 'we do not conduct further analysis into the deeper reasons behind this phenomenon'; that is a causal-identification limitation, not a circularity. The Section 3.3 derivation appears to assume ReLU additivity in Eqs. 7-8, which is a correctness risk rather than a circular step. The only self-citation, KV shift [41] in the Related Work, is descriptive and not load-bearing. No load-bearing argument reduces to its own inputs, so circularity is minimal.

Assumptions & free parameters 1 free parameters · 4 assumptions · 1 invented entities

The central claim rests on standard ML training assumptions plus a few paper-specific choices: the 32k length threshold as a proxy for looping, a linearized model of the editor in the analysis, and a correlational rather than causal link between M(X) and cyclical reasoning. The only hand-chosen hyperparameter with direct impact on the headline comparison is the LoRA rank. No novel physical entities are introduced.

free parameters (1)

LoRA rank r = 128, 256 (and 296 for param-matched control)
The main comparisons use r=256; the paper observes that higher rank reduces cyclical reasoning, and Shift-FFN's Editor rank is set equal to r. This is a hand-chosen hyperparameter that affects the central comparison, but it is standard practice rather than a fitted constant.

assumptions (4)

domain assumption Exceeding the 32k generation limit indicates cyclical reasoning (looping).
Stated in Section 4.2; the paper justifies it by showing over 80% of length-exceeded outputs contain exact repetition, but 20% are classified by observed patterns, and the assumption is load-bearing for all claims about cyclical reasoning.
ad hoc to paper The linearized shift fs(x_{i-1}, x_i) approximately equals W_s x_{i-1} and the ReLU nonlinearity is treated as additive: sigma(a+b) = sigma(a) + sigma(b).
Introduced in Section 3.3 'From simplicity'; these are not valid for the actual gated ReLU editor and are not flagged as approximations in the main text.
domain assumption Lower M(X) is causally linked to cyclical reasoning and can be counteracted by architecture changes.
The paper's motivation and design depend on this correlation being actionable; Appendix A admits the deeper cause is not analyzed.
standard math Standard transformer, LoRA, and FFN-as-key-value-memory results from cited references are assumed.
Background for the architecture and the Section 3.3 analysis; these are standard results in the field.

invented entities (1)

Shift-FFN Editor module independent evidence
purpose: A trainable per-layer module inserted before each FFN that mixes the previous token's representation into the current token's representation to amplify adjacent-token differences.
The module is not physically hypothesized; its effect is measured directly by ablations and external benchmarks, so it has independent falsifiable handles (accuracy, PE, pass@K) outside the paper's own definitions.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Amplify Adjacent Token Differences: Enhancing Long Chain-of-Thought Reasoning with Shift-FFN." pith.science (2026). https://pith.science/paper/6MUT3AH2

@misc{pith2026250517153,
  author       = {Pith},
  title        = {Pith review of: Amplify Adjacent Token Differences: Enhancing Long Chain-of-Thought Reasoning with Shift-FFN},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6MUT3AH2}},
  note         = {Machine review of arXiv:2505.17153}
}

read the original abstract

Recently, models such as OpenAI-o1 and DeepSeek-R1 have demonstrated remarkable performance on complex reasoning tasks through Long Chain-of-Thought (Long-CoT) reasoning. Although distilling this capability into student models significantly enhances their performance, this paper finds that fine-tuning LLMs with full parameters or LoRA with a low rank on long CoT data often leads to Cyclical Reasoning, where models repeatedly reiterate previous inference steps until the maximum length limit. Further analysis reveals that smaller differences in representations between adjacent tokens correlates with a higher tendency toward Cyclical Reasoning. To mitigate this issue, this paper proposes Shift Feedforward Networks (Shift-FFN), a novel approach that edits the current token's representation with the previous one before inputting it to FFN. This architecture dynamically amplifies the representation differences between adjacent tokens. Extensive experiments on multiple mathematical reasoning tasks demonstrate that LoRA combined with Shift-FFN achieves higher accuracy and a lower rate of Cyclical Reasoning across various data sizes compared to full fine-tuning and standard LoRA. Our data and code are available at https://anonymous.4open.science/r/Shift-FFN

Figures

Figures reproduced from arXiv: 2505.17153 by the authors.

**Figure 1.** (Left), performance comparison of LoRA and Full Fine-Tuning (Full FT) on Accuracy (%) and Length Exceed (%) metrics for short CoT and long CoT datasets. "Accuracy" represents the average accuracy across four mathematical tasks. "Length Exceed" indicates the percentage of model outputs that exceed the maximum length limit. (Right), an example of Cyclical Reasoning. Our proposed Shift-FFN can be viewed as a representa… view at source ↗

**Figure 2.** Distribution of the M(X) for Exceed and Normal samples, comparing LoRA and Full fine-tuned models. The dashed red line represents the mean value. Analysis Setup and Findings. We train two models on a 20k long CoT using LoRA and full fine-tuning, respectively. We evaluate these models on a randomly selected set of 100 questions from the MATH500 [10], with a sampling of eight times, and exclude questions where all ei… view at source ↗

**Figure 3.** The architecture of Shift-FFN, the left side describes the process of shifting token, the right side demonstrates the detail of the Editor module. σ is the ReLU function. ⊙ and ⊕ are element-wise multiplication and addition, respectively. 3.3 Analysis From simplicity, we consider fs(xi−1, xi) = Wc Wb xi−1 = Ws xi−1 = xˆi−1 and use standard FFN(xi) = Wdown[σ(Wup xi)] in this section. From the Perspective of Neural Me… view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Proportion of length-exceeded and repetition samples in different models. 1 2 4 8 16 32 64 128256 Number of Samples K 20 30 40 50 60 70 Pass@K AMIE24 1 2 4 8 16 32 64 128256 Number of Samples K 20 30 40 50 60 Olympiad lora (r=256) lora (r=296) lora+Shift-FFN(r=256) […

**Figure 6.** Figure 6: Comparison of Mean Relative Change M(X) and Length Exceeded Percentage PE for non-length-exceeded samples across models trained with different settings on four datasets. 4.3 Compared to LoRA With the Same Number of Parameters As Shift-FFN introduces extra parameters, t…

**Figure 7.** Figure 7: The Accuracy (left) and the Length Exceeded Percentage PE (right) of different fine-tuned models under varying training sample sizes. Accuracy and Length Exceeded Percentage are the average values obtained on four datasets. 4.5 Performance of Shift-FFN with Varying Tra…

**Figure 8.** Figure 8: The Accuracy (left) and the Length Exceeded Percentage (right) of different fine-tuned models for under varying sampling temperatures on AIME24. A Limitation Our study has two main limitations: (1) Due to resource constraints, we do not conduct experiments with larger …

**Figure 9.** Figure 9: The Pass@K of different fine-tuned models for under varying sampling temperatures on [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

50 extracted references · 7 canonical work pages

[1]

Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models, April 2025

Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models, April 2025. arXiv:2503.09567 [cs]

arXiv 2025
[2]

Boolq: Exploring the surprising difficulty of natural yes/no questions

Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint arXiv:1905.10044, 2019

arXiv 1905
[3]

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning, January 2025

DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning, January 2025. arXiv:2501.12948 [cs]

arXiv 2025
[4]

Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space

Mor Geva, Avi Caciularu, Kevin Ro Wang, and Yoav Goldberg. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. arXiv preprint arXiv:2203.14680, 2022

arXiv 2022
[5]

Transformer Feed-Forward Layers Are Key-Value Memories

Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer Feed-Forward Layers Are Key-Value Memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484–5495, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics

work page 2021
[6]

The llama 3 herd of models

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

arXiv 2024
[7]

Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey, September 2024

Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey, September 2024. arXiv:2403.14608 [cs]

arXiv 2024
[8]

Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems

Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. arXiv preprint arXiv:2402.14008, 2024

arXiv 2024

Show all 50 references

[9]

Sparseadapter: An easy approach for improving the parameter-efficiency of adapters

Shwai He, Liang Ding, Daize Dong, Miao Zhang, and Dacheng Tao. Sparseadapter: An easy approach for improving the parameter-efficiency of adapters. arXiv preprint arXiv:2210.04284, 2022

2022 arXiv
[10]

Measuring mathematical problem solving with the math dataset

Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021

2021 arXiv
[11]

Parameter-efficient transfer learning for nlp

Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In International conference on machine learning , pages 2790–2799. PMLR, 2019

2019
[12]

Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-Rank Adaptation of Large Language Models, October
[13]

Openai o1 system card

Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024. 10

2024 arXiv
[14]

The power of scale for parameter-efficient prompt tuning

Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691, 2021

2021 arXiv
[15]

Patil, Matei Zaharia, Joseph E

Dacheng Li, Shiyi Cao, Tyler Griggs, Shu Liu, Xiangxi Mo, Shishir G. Patil, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!, February 2025. arXiv:2502.07374 [cs]

2025 arXiv
[16]

Numinamath

Jia LI, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Costa Huang, Kashif Rasul, Longhui Yu, Albert Jiang, Ziju Shen, Zihan Qin, Bin Dong, Li Zhou, Yann Fleureau, Guillaume Lample, and Stanislas Polu. Numinamath. [https://huggingface.co/AI-MO/NuminaMath...

2024
[17]

Prefix-Tuning: Optimizing Continuous Prompts for Generation, January 2021

Xiang Lisa Li and Percy Liang. Prefix-Tuning: Optimizing Continuous Prompts for Generation, January 2021. arXiv:2101.00190 [cs]

2021 arXiv
[18]

Alpacaeval: An automatic evaluator of instruction-following models, 2023

Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Alpacaeval: An automatic evaluator of instruction-following models, 2023

2023
[19]

Forgetting Transformer: Softmax Attention with a Forget Gate, March 2025

Zhixuan Lin, Evgenii Nikishin, Xu Owen He, and Aaron Courville. Forgetting Transformer: Softmax Attention with a Forget Gate, March 2025. arXiv:2503.02130 [cs]

2025 arXiv
[20]

In-context vectors: Making in context learning more effective and controllable through latent space steering

Sheng Liu, Haotian Ye, Lei Xing, and James Zou. In-context vectors: Making in context learning more effective and controllable through latent space steering. arXiv preprint arXiv:2311.06668, 2023

2023 arXiv
[21]

Parameter-efficient orthogonal finetuning via butterfly factorization

Weiyang Liu, Zeju Qiu, Yao Feng, Yuliang Xiu, Yuxuan Xue, Longhui Yu, Haiwen Feng, Zhen Liu, Juyeon Heo, Songyou Peng, et al. Parameter-efficient orthogonal finetuning via butterfly factorization. arXiv preprint arXiv:2311.06243, 2023

2023 arXiv
[22]

Aligning large language models with human preferences through representation engineering

Wenhao Liu, Xiaohua Wang, Muling Wu, Tianlong Li, Changze Lv, Zixuan Ling, Jianhao Zhu, Cenyuan Zhang, Xiaoqing Zheng, and Xuanjing Huang. Aligning large language models with human preferences through representation engineering. arXiv preprint arXiv:2312.15997, 2023

2023 arXiv
[23]

Deconstructing Long Chain-of-Thought: A Structured Reasoning Optimization Framework for Long CoT Distillation, March 2025

Yijia Luo, Yulin Song, Xingyao Zhang, Jiaheng Liu, Weixun Wang, GengRu Chen, Wenbo Su, and Bo Zheng. Deconstructing Long Chain-of-Thought: A Structured Reasoning Optimization Framework for Long CoT Distillation, March 2025. arXiv:2503.16385 [cs]

2025 arXiv
[24]

Rwkv: Reinventing rnns for the transformer era

Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, et al. Rwkv: Reinventing rnns for the transformer era. arXiv preprint arXiv:2305.13048, 2023

2023 arXiv
[25]

Fast transformer decoding: One write-head is all you need

Noam Shazeer. Fast transformer decoding: One write-head is all you need. arXiv preprint arXiv:1911.02150, 2019

1911 arXiv
[26]

Glu variants improve transformer

Noam Shazeer. Glu variants improve transformer. arXiv preprint arXiv:2002.05202, 2020

2002 arXiv
[27]

LLM Braces: Straightening Out LLM Predictions with Relevant Sub-Updates, March 2025

Ying Shen and Lifu Huang. LLM Braces: Straightening Out LLM Predictions with Relevant Sub-Updates, March 2025. arXiv:2503.16334 [cs]

2025 arXiv
[28]

Extracting latent steering vectors from pretrained language models

Nishant Subramani, Nivedita Suresh, and Matthew E Peters. Extracting latent steering vectors from pretrained language models. arXiv preprint arXiv:2205.05124, 2022

2022 arXiv
[29]

Augmenting self-attention with persistent memory

Sainbayar Sukhbaatar, Edouard Grave, Guillaume Lample, Herve Jegou, and Armand Joulin. Augmenting self-attention with persistent memory. arXiv preprint arXiv:1907.01470, 2019

1907 arXiv
[30]

End-to-end memory networks.Advances in neural information processing systems , 28, 2015

Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, et al. End-to-end memory networks.Advances in neural information processing systems , 28, 2015

2015
[31]

Unlocking General Long Chain-of-Thought Reasoning Capabilities of Large Language Models via Representation Engineering, March 2025

Xinyu Tang, Xiaolei Wang, Zhihao Lv, Yingqian Min, Wayne Xin Zhao, Binbin Hu, Ziqi Liu, and Zhiqiang Zhang. Unlocking General Long Chain-of-Thought Reasoning Capabilities of Large Language Models via Representation Engineering, March 2025. arXiv:2503.11314 [cs]. 11

2025 arXiv
[32]

Open Thoughts

OpenThoughts Team. Open Thoughts. https://open-thoughts.ai, January 2025

2025
[33]

Gomez, Lukasz Kaiser, and Illia Polosukhin

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention Is All You Need, December 2017. arXiv:1706.03762 [cs]

2017 arXiv
[34]

Adamix: Mixture-of-adaptations for parameter-efficient model tuning

Yaqing Wang, Sahaj Agarwal, Subhabrata Mukherjee, Xiaodong Liu, Jing Gao, Ahmed Hassan Awadallah, and Jianfeng Gao. Adamix: Mixture-of-adaptations for parameter-efficient model tuning. arXiv preprint arXiv:2205.12410, 2022

2022 arXiv
[35]

Wong, and Rui Wang

Yiming Wang, Pei Zhang, Baosong Yang, Derek F. Wong, and Rui Wang. Latent Space Chain- of-Embedding Enables Output-free LLM Self-Evaluation, March 2025. arXiv:2410.13640 [cs]

2025 arXiv
[36]

Wong, Zhuosheng Zhang, and Rui Wang

Yiming Wang, Pei Zhang, Baosong Yang, Derek F. Wong, Zhuosheng Zhang, and Rui Wang. Embedding Trajectory for Out-of-Distribution Detection in Mathematical Reasoning, October
[37]

Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, January 2023

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, January 2023. arXiv:2201.11903 [cs]

2023 arXiv
[38]

Light-r1: Curriculum sft, dpo and rl for long cot from scratch and beyond

Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Lifu Tang, Xiaowei Lv, Haosheng Zou, Yongchao Deng, Shousheng Jia, and Xiangzheng Zhang. Light-r1: Curriculum sft, dpo and rl for long cot from scratch and beyond. arXiv preprint arXiv:2503.1...

2025 arXiv
[39]

Advancing Parameter Efficiency in Fine-tuning via Representation Editing, June 2024

Muling Wu, Wenhao Liu, Xiaohua Wang, Tianlong Li, Changze Lv, Zixuan Ling, Jianhao Zhu, Cenyuan Zhang, Xiaoqing Zheng, and Xuanjing Huang. Advancing Parameter Efficiency in Fine-tuning via Representation Editing, June 2024. arXiv:2402.15179 [cs]

2024 arXiv
[40]

Manning, and Christopher Potts

Zhengxuan Wu, Aryaman Arora, Zheng Wang, Atticus Geiger, Dan Jurafsky, Christopher D. Manning, and Christopher Potts. ReFT: Representation Finetuning for Language Models, May
[41]

KV Shifting Attention Enhances Language Modeling, December 2024

Mingyu Xu, Wei Cheng, Bingning Wang, and Weipeng Chen. KV Shifting Attention Enhances Language Modeling, December 2024. arXiv:2411.19574 [cs]

2024 arXiv
[42]

A survey on knowledge distillation of large language models

Xiaohan Xu, Ming Li, Chongyang Tao, Tao Shen, Reynold Cheng, Jinyang Li, Can Xu, Dacheng Tao, and Tianyi Zhou. A survey on knowledge distillation of large language models. arXiv preprint arXiv:2402.13116, 2024

2024 arXiv
[43]

arXiv:2404.03592 [cs]

arXiv
[44]

LIMO: Less is More for Reasoning, February 2025

Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. LIMO: Less is More for Reasoning, February 2025. arXiv:2502.03387 [cs]

2025 arXiv
[45]

Root mean square layer normalization

Biao Zhang and Rico Sennrich. Root mean square layer normalization. Advances in Neural Information Processing Systems, 32, 2019

2019
[46]

Qwen2.5 technical report.arXiv preprint arXiv:2412.15115, 2024

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

2024 arXiv
[47]

Llamafactory: Unified efficient fine-tuning of 100+ language models

Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. arXiv preprint arXiv:2403.13372, 2024. 12 0.2 0.4 0.6 0.8 1.0 T emperature 13 14 15 16 17 18 19Accuracy (%) M...

2024 arXiv
[49]

AdaLoRA: Adaptive Budget Allocation for Parameter- Efficient Fine-Tuning, December 2023

Qingru Zhang, Minshuo Chen, Alexander Bukharin, Nikos Karampatziakis, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. AdaLoRA: Adaptive Budget Allocation for Parameter- Efficient Fine-Tuning, December 2023. arXiv:2303.10512 [cs]

2023 arXiv
[2021]

arXiv:2106.09685 [cs]

arXiv
[2024]

arXiv:2405.14039 [cs]

arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models, April 2025

Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models, April 2025. arXiv:2503.09567 [cs]

arXiv 2025

[2] [2]

Boolq: Exploring the surprising difficulty of natural yes/no questions

Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint arXiv:1905.10044, 2019

arXiv 1905

[3] [3]

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning, January 2025

DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning, January 2025. arXiv:2501.12948 [cs]

arXiv 2025

[4] [4]

Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space

Mor Geva, Avi Caciularu, Kevin Ro Wang, and Yoav Goldberg. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. arXiv preprint arXiv:2203.14680, 2022

arXiv 2022

[5] [5]

Transformer Feed-Forward Layers Are Key-Value Memories

Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer Feed-Forward Layers Are Key-Value Memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484–5495, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics

work page 2021

[6] [6]

The llama 3 herd of models

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

arXiv 2024

[7] [7]

Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey, September 2024

Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey, September 2024. arXiv:2403.14608 [cs]

arXiv 2024

[8] [8]

Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems

Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. arXiv preprint arXiv:2402.14008, 2024

arXiv 2024