Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

Longer Context, Deeper Thinking: Uncovering the Role of Long-Context Ability in Reasoning

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Enhancing a language model's long-context capacity before supervised fine-tuning improves reasoning accuracy, even on tasks with short inputs.

desk verdict A cheap recipe with suggestive results, but the causal claim about long-context capacity is not supported by the confounded interventions. read the letter →

arxiv 2505.17315 v2 pith:ZUBKDREF submitted 2025-05-22 cs.AI cs.CLcs.LG

Wang Yang , Zirui Liu , Hongye Jin , Qingyu Yin , Vipin Chaudhary , Xiaotian Han This is my paper · ORCID
classification cs.AIcs.CLcs.LG
keywords long-contextmodelingreasoningsupervisedfine-tuningRoPEthetascalingmodelmergingNeedle-in-a-Haystackmathematicalchain-of-thought
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Long-context ability—the capacity to process and track extended sequences—is often treated as a convenience for long inputs, but this paper argues it is a foundation for reasoning itself. The authors test this by building model variants that share the same architecture and the same reasoning fine-tuning data but differ only in long-context capacity, produced by scaling the RoPE positional-encoding parameter or by merging weights with a longer-context model. Across MATH500, AIME, and GSM8K, the variants with stronger long-context capacity score higher after fine-tuning, and the gains show up even on problems with short inputs. They also propose a training recipe: extend the context window (for example to 128K tokens) before running reasoning-specific SFT, which raised a 4K-context math model's AIME accuracy from 15.00 to 28.00.

What carries the argument

The machinery is a pair of controlled interventions on the same base model. RoPE theta scaling multiplies the base frequency of Rotary Position Embeddings, which extends the range over which relative positions are encoded without retraining; scaling factors of 1, 4, 8, 16, 32, and 64 produce a ladder of long-context capacities. Model merging linearly interpolates the weights of a base model with a 1M-context model, with merge ratios of 0.1, 0.7, and 1.0. Long-context capacity is measured by the Needle-in-a-Haystack benchmark, and reasoning is evaluated after supervised fine-tuning on short (within 8K tokens) and long (8K-16K) subsets of a reasoning dataset. These tools let the paper vary one factor at a time and link the measured long-context score to post-SFT reasoning accuracy.

What would settle it

Take a model whose RoPE theta scaling raises its Needle-in-a-Haystack score from 0 to 77 but whose base MATH500 accuracy is restored to the unscaled level by supplemental pretraining; if the post-SFT reasoning accuracy does not exceed the unscaled model's, the long-context gain is not the cause.

Watch

Extended reading notes

Core claim

The central claim is that models with stronger long-context capabilities consistently outperform their counterparts on reasoning tasks after supervised fine-tuning. The paper establishes this through controlled comparisons in which only the long-context intervention changes: scaling RoPE theta from 1x to 64x, or merging a base model with a 1M-context model at different ratios. Reasoning accuracy tracks the measured long-context ability (Needle-in-a-Haystack score) rather than the advertised context length; the best reasoning results occur at the strongest effective long-context setting, and degrade at extreme settings. A further claim is the recipe: first enhance long-context capacity, then apply reasoning SFT, which on Qwen2.5-Math-7B-Instruct improved MATH500 accuracy from 85.04 to 88.70 and AIME accuracy from 15.00 to 28.00.

Load-bearing premise

The interventions that raise long-context capacity are assumed to change only that capacity, leaving every other capability relevant to reasoning fine-tuning untouched.

Editorial extensions

If this is right

  • Reasoning-focused SFT should be preceded by a long-context extension step; the paper's recipe demonstrates this on a 4K-context math model, raising AIME accuracy from 15.00 to 28.00 after scaling theta and merging with a 1M-context model.
  • The benefits of long-context capacity transfer to short-input reasoning tasks, indicating long-context training is not only about input length.
  • The relationship between long-context capacity and reasoning is non-monotonic: reasoning accuracy peaks at the strongest effective long-context setting and degrades at extreme settings, so more context is not always better.
  • Models fine-tuned on long reasoning datasets (8K-16K responses) benefit more from strong long-context capacity than models fine-tuned on short datasets, and those long datasets require the capacity to be realized.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An untested implication is that the same recipe—extend context, then reason-tune—would help non-mathematical reasoning such as long-horizon planning or agentic tasks, where coherence over long computations matters.
  • The non-monotonic theta result warns practitioners that advertised context length is a poor proxy for effective long-context ability; a testable corollary is that picking extension strength by Needle-in-a-Haystack scores, not raw window size, maximizes reasoning gains.
  • Because the paper only varies context extension before SFT, it does not isolate which component of long-context training (positional extrapolation, attention stability, or output-length tolerance) drives the reasoning gains; a future ablation could separate these.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This paper tests the hypothesis that a language model's long-context capacity causally supports its reasoning ability. The authors present three kinds of evidence: (i) a comparison of public 32k vs 128k reasoning models on MATH500 and AIME (Table 1); (ii) failure-case analyses linking repetitive or incoherent generations to insufficient context handling (Figures 2-3); and (iii) two intervention studies in which they vary long-context ability via RoPE theta scaling (Section 3.2) or model merging (Section 3.3), then fine-tune all variants on short and long subsets of OpenR1-Math-220K. Post-SFT accuracy on MATH500, AIME, and GSM8K is plotted against Needle-in-a-Haystack (NIAH) scores, and a recipe is proposed: scale RoPE theta by 16 and merge with a 1M-context model before reasoning SFT. Applying this recipe to Qwen2.5-Math-7B-Instruct is reported to raise MATH500 from 85.04 to 88.70 and AIME from 15.00 to 28.00. The paper concludes that long-context ability should be treated as a first-class training objective.

Significance. If the causal claim were established, the paper would provide a practical recipe for improving reasoning models and a compelling argument for investing in long-context pretraining. The authors provide open-source code and make quantitative, falsifiable predictions, and the NIAH-based evaluations are reproducible in principle. However, the current evidence is not yet convincing: the interventions alter base reasoning accuracy and output length, the uncontrolled public-model comparison is not causal, and the recipe hyperparameters are selected on the same benchmarks used to report gains. The paper is best read as a motivating correlational study that needs substantially stronger controls to support its central claim.

major comments (5)
  1. [Section 3.2, Table 3] The main claim that 'models with stronger long-context capabilities consistently outperform their counterparts on reasoning tasks after SFT' (Abstract) is not supported by the RoPE theta experiment because the intervention changes base reasoning performance. In Table 3, base MATH500 accuracy falls monotonically from 24.40 at theta x1 to 14.20 at theta x64, and base average output length falls from 746 to 453 tokens; Table 5 shows similar base shifts on AIME and GSM8K. The post-SFT peak at theta x16 (Avg 59.36) therefore starts from a weaker base than the theta x1 condition (Avg 54.80 from base 24.40). Please report gains relative to the same-theta base model, or match base accuracy across conditions, before attributing the post-SFT differences to long-context capacity.
  2. [Section 3.1 and Section B, Table 6] The merge-ratio experiments were introduced as 'carefully controll[ing] the merge ratio to ensure that the base performance remains nearly unchanged,' but Table 6 contradicts this: across ratios 0, 0.1, 0.7, and 1.0, base MATH500 accuracy varies from 75.00 to 72.16 and base AIME from 8.22 to 7.33. The post-SFT differences in AIME (Avg 20.56 to 21.56) are small relative to this base shift and do not increase monotonically with the 128k NIAH score. The evidence does not isolate long-context ability as the causal variable.
  3. [Section 3.4, Table 4] The recipe validation is confounded. The first step (RoPE theta x16) drops Qwen2.5-Math-7B-Instruct's base MATH500 accuracy from 81.88 to 65.16; the merge with the 1M model restores it only to 74.12. The reported post-SFT improvement (Avg MATH500 88.70 vs 85.04) is measured against this shifted base. A proper test would include a control that restores base accuracy without changing long-context ability, or would report the post-SFT gain over the base of the same model variant. As reported, the recipe's benefit cannot be separated from the effect of destabilizing and then partially repairing the base model.
  4. [Section 3.3, Figure 9, Table 6] The explanation for the ratio-1.0 result is inconsistent with the provided data. The text says that at merge ratio 1.0 the model's 'effective long-context ability is weaker than others,' yet Table 6 reports the 32k NIAH score at ratio 1.0 as 100, the highest among the four ratios. The authors refer to 'more challenging benchmarks such as Value Tracking and Question Answering' for the long-context evaluation, but these results are not reported in the paper or in Appendix A, which instead contains the LLaMA3 RoPE scaling tables. The non-monotonic relationship between merge ratio and post-SFT reasoning therefore lacks supporting evidence in the manuscript.
  5. [Section 2.1, Table 1] Table 1 compares different public models (e.g., OpenR1-Qwen-7B, DeepSeek-R1-Distill-Qwen-7B, OpenMath-Nemotron-7B) that differ in base architecture, training data, and fine-tuning procedure, not only in context length. The abstract and Section 2.1 present this comparison as evidence that longer context leads to stronger reasoning, but it is an uncontrolled observational comparison. It can serve as motivation, but it should not be counted among the controlled experiments supporting the causal claim.
minor comments (6)
  1. [Section 3.2 heading] The heading contains a typo: 'Fine-tuning' is written as 'Finetuing'.
  2. [Section 3.1] The text says 'to access long context ability'; this should be 'to assess long-context ability,' and the hyphenation of 'Needle-in-a-Haystack' should be made consistent throughout.
  3. [Section 3.2, Figures 7-8] The text refers to 'LLaMA3-8B-Instruct' while the figure captions say 'LLaMA-3.1-8B-Instruct'; please unify the model name.
  4. [Introduction, paragraph 2] The Introduction states that the controlled study compares models with 'varying degrees of long-context pretraining,' but the RoPE scaling and model merging interventions are not pretraining methods; they are post-hoc modifications of the model weights. Please rephrase to avoid this mischaracterization.
  5. [Section 3.3, last paragraph] The sentence 'The corresponding results are provided in Section A' and the later reference 'as is shown in Section A' point to an appendix section that does not contain the Value Tracking and Question Answering evaluations; please add the missing results or correct the cross-reference.
  6. [Section 3.1, evaluation details] The metric 'pass@1(5)' is undefined; please state that it means generating five responses per question and computing pass@1 over all generated responses.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: reasoning results are measured on external benchmarks and are not derived from the long-context proxy by construction.

full rationale

Walking the derivation chain from long-context interventions (RoPE theta scaling in Sec. 3.2 and model merging in Sec. 3.3) to post-SFT reasoning accuracy (MATH500, AIME, GSM8K), the paper nowhere defines long-context ability in terms of the reasoning outcomes, nor does it fit a reasoning predictor to long-context data. Long-context capacity is operationalized by Needle-in-a-Haystack and auxiliary retrieval benchmarks, while reasoning is measured with external math benchmarks; the two measurement instruments are distinct. The recipe in Sec. 3.4 is applied to a held-out model (Qwen2.5-Math-7B-Instruct), and the reported gains are compared against that model's own base, not against a fitted curve. The only author self-citation ([44], SelfExtend) appears in Related Work and is not load-bearing for the central claim. The observation in Tables 3 and 6 that RoPE scaling and merging also move base reasoning accuracy is a threat to the internal validity of the causal claim, but it is a confound, not a definitional reduction: the paper's protocol and equations do not make post-SFT reasoning equal to the NIAH score or to a fitted function of it. Hence no circular step can be exhibited, and the honest finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on two strong assumptions: NIAH is a valid proxy for the long-context ability that matters for reasoning, and the interventions (RoPE scaling, merging) isolate long-context ability from other changes. Both are only partially supported by the paper's data.

free parameters (4)
  • RoPE theta scaling factor = 16 (chosen as best after NIAH sweep)
    Grid of factors {1,4,8,16,32,64}; x16 yields peak NIAH score and is used in the recipe. The choice is empirically selected, not derived.
  • Merge ratio for recipe = 0.3 (0.7 x RoPE16 + 0.3 x 1M model)
    Chosen to balance base performance and long-context ability; Table 4. The paper does not explain how this ratio was selected.
  • Short/long data split threshold = 8K tokens
    Samples with responses within 8K tokens are 'short'; 8K-16K are 'long'. This threshold is arbitrary and affects which data each model sees.
  • Number of samples per SFT subset = 20K per category
    Subsampled from OpenR1-Math-220K after correctness filtering. Affects training and results.
assumptions (4)
  • domain assumption Needle-in-a-Haystack accuracy measures the long-context ability relevant to reasoning.
    Used as the primary proxy for long-context capacity throughout (Section 3.1). NIAH tests single-needle retrieval, not complex multi-step reasoning over long contexts, so this is a strong assumption.
  • domain assumption Scaling RoPE theta and model merging alter long-context ability without confounding other capabilities.
    The causal attribution depends on this. But Table 3 shows base MATH500 accuracy drops from 24.40 to 14.20 as theta increases, so the interventions clearly affect other capabilities.
  • standard math pass@1(5) with five generated responses is a stable estimate of reasoning performance.
    Follows DeepSeek-R1 methodology; no significance testing is reported, so stability is assumed.
  • domain assumption Correctness filtering of SFT data ensures response quality.
    Filtering method and quality of the filter are not described in detail (Section 3.1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Longer Context, Deeper Thinking: Uncovering the Role of Long-Context Ability in Reasoning." pith.science (2026). https://pith.science/paper/ZUBKDREF

@misc{pith2026250517315,
  author       = {Pith},
  title        = {Pith review of: Longer Context, Deeper Thinking: Uncovering the Role of Long-Context Ability in Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZUBKDREF}},
  note         = {Machine review of arXiv:2505.17315}
}
read the original abstract

Recent language models exhibit strong reasoning capabilities, yet the influence of long-context capacity on reasoning remains underexplored. In this work, we hypothesize that current limitations in reasoning stem, in part, from insufficient long-context capacity, motivated by empirical observations such as (1) higher context window length often leads to stronger reasoning performance, and (2) failed reasoning cases resemble failed long-context cases. To test this hypothesis, we examine whether enhancing a model's long-context ability before Supervised Fine-Tuning (SFT) leads to improved reasoning performance. Specifically, we compared models with identical architectures and fine-tuning data but varying levels of long-context capacity. Our results reveal a consistent trend: models with stronger long-context capacity achieve significantly higher accuracy on reasoning benchmarks after SFT. Notably, these gains persist even on tasks with short input lengths, indicating that long-context training offers generalizable benefits for reasoning performance. These findings suggest that long-context modeling is not just essential for processing lengthy inputs, but also serves as a critical foundation for reasoning. We advocate for treating long-context capacity as a first-class objective in the design of future language models.

Figures

Figures reproduced from arXiv: 2505.17315 by the authors.

Figure 1
Figure 1. Impact of long-context capacity on mathematical reasoning. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Case Study: Repetition Failure. Two failure cases where the model produces clearly repetitive sentences in its answers. Such repetition is a common symptom of insufficient long-context capability, leading to strange responses and degraded reasoning quality in extended sequences. Questions: The expression 2 ∙ 3 ∙ 4 ∙ 5 + 1 is equal to 121, since multiplication is carried out before addition. However, we can obtain va… view at source ↗
Figure 3
Figure 3. Case Study: Contextual Reference Failures. Two failure cases where the model makes incorrect references to expressions introduced earlier in the problem. These errors occur in the later stages of the response and reflect a typical symptom of insufficient long-context capability. 2.2 Case study: failed reasoning cases resemble failed long-context cases. We analyze failed cases of reasoning models and long-context inp… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Top: Length distribution of three reasoning datasets. NuminaMath-CoT represents early chain-of-thought (CoT) data with short sequences, while OpenR1-Math-220K and DeepMath-103K, generated by DeepSeek-R1, exhibit significantly longer outputs. Bottom-left: Performance of…
Figure 5
Figure 5. Figure 5: Needle-in-a-Haystack Results for LLaMA-3-8B-Instruct. Performance of LLaMA-3-8B [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Pipeline for verifying the connection between reasoning and long-context ability. We first [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Visualization of the relationship between 32k long-context ability and reasoning perfor [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Visualization of the relationship between 32k long-context ability and average output length [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 9
Figure 9. Figure 9: Visualization of the relationship between 128k long-context ability and reasoning per [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: The relationship between 128k long-context ability and average output length across [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]
Figure 11
Figure 11. Figure 11: 8K Needle-in-a-Haystack Results at Each Step of the Proposed Fine-Tuning Recipe. [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 12
Figure 12. Figure 12: Visualization of the relationship between 128k long-context ability and reasoning per [PITH_FULL_IMAGE:figures/full_fig_p015_12.png]
Figure 13
Figure 13. Figure 13: 32K Needle-in-a-Haystack Results at Each Step of the Proposed Fine-Tuning Recipe. [PITH_FULL_IMAGE:figures/full_fig_p015_13.png]
Figure 14
Figure 14. Figure 14: The relationship between 128k long-context ability and average output length across [PITH_FULL_IMAGE:figures/full_fig_p015_14.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Memory-Augmented Transformers: A Systematic Review from Neuroscience Principles to Enhanced Model Architectures

    cs.LG 2025-08 unverdicted novelty 3.0 of 10

    Memory-augmented Transformer research is organized into a three-axis taxonomy bridging neuroscience memory concepts to network designs, but no new result is produced.

Reference graph

Works this paper leans on

52 extracted references · 16 canonical work pages · cited by 1 Pith paper

  1. [1]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  2. [2]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms.arXiv preprint arXiv:2501.12599, 2025

    arXiv 2025

  3. [3]

    Phi-4-reasoning technical report, 2025

    Marah Abdin, Sahaj Agarwal, Ahmed Awadallah, Vidhisha Balachandran, Harkirat Behl, Lingjiao Chen, Gustavo de Rosa, Suriya Gunasekar, Mojan Javaheripi, Neel Joshi, Piero Kauffmann, Yash Lara, Caio César Teodoro Mendes, Arindam Mitra, Besmira Nushi, Dimitris Papailiopoulos, Olli Saarikivi, Shital Shah, Vaishnavi Shrivastava, Vibhav Vineet, Yue Wu, Safoora Y...

    work page 2025

  4. [4]

    Deepmath-103k: A large-scale, challenging, decontaminated, and verifiable mathematical dataset for advancing reasoning, 2025

    Zhiwei He, Tian Liang, Jiahao Xu, Qiuzhi Liu, Xingyu Chen, Yue Wang, Linfeng Song, Dian Yu, Zhenwen Liang, Wenxuan Wang, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Deepmath-103k: A large-scale, challenging, decontaminated, and verifiable mathematical dataset for advancing reasoning, 2025

    work page 2025

  5. [5]

    Open Thoughts

    OpenThoughts Team. Open Thoughts. https://open-thoughts.ai, January 2025

    2025

  6. [6]

    Aimo-2 winning solution: Building state-of-the-art math- ematical reasoning models with openmathreasoning dataset.arXiv preprint arXiv:2504.16891, 2025

    Ivan Moshkov, Darragh Hanley, Ivan Sorokin, Shubham Toshniwal, Christof Henkel, Benedikt Schifferer, Wei Du, and Igor Gitman. Aimo-2 winning solution: Building state-of-the-art math- ematical reasoning models with openmathreasoning dataset.arXiv preprint arXiv:2504.16891, 2025

    arXiv 2025

  7. [7]

    Ntk-aware scaled rope allows llama models to have extended (8k+) context size without any fine-tuning and minimal perplexity degradation, 2023

    Bowen Peng and Jeffrey Quesnelle. Ntk-aware scaled rope allows llama models to have extended (8k+) context size without any fine-tuning and minimal perplexity degradation, 2023

    2023

  8. [8]

    Open r1: A fully open reproduction of deepseek-r1, January 2025

    Hugging Face. Open r1: A fully open reproduction of deepseek-r1, January 2025

    2025

Show all 52 references
  1. [9]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, and Ruoyu Zhang... Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    2025

  2. [10]

    Llm as a mastermind: A survey of strategic reasoning with large language models.arXiv preprint arXiv:2404.01230, 2024

    Yadong Zhang, Shaoguang Mao, Tao Ge, Xun Wang, Adrian de Wynter, Yan Xia, Wenshan Wu, Ting Song, Man Lan, and Furu Wei. Llm as a mastermind: A survey of strategic reasoning with large language models.arXiv preprint arXiv:2404.01230, 2024

    2024 arXiv

  3. [11]

    Reasoning with large language models, a survey.arXiv preprint arXiv:2407.11511, 2024

    Aske Plaat, Annie Wong, Suzan Verberne, Joost Broekens, Niki van Stein, and Thomas Back. Reasoning with large language models, a survey.arXiv preprint arXiv:2407.11511, 2024

    2024

  4. [12]

    A survey of frontiers in llm reasoning: Inference scaling, learning to reason, and agentic systems.arXiv preprint arXiv:2504.09037, 2025

    Zixuan Ke, Fangkai Jiao, Yifei Ming, Xuan-Phi Nguyen, Austin Xu, Do Xuan Long, Minzhi Li, Chengwei Qin, Peifeng Wang, Silvio Savarese, et al. A survey of frontiers in llm reasoning: Inference scaling, learning to reason, and agentic systems.arXiv preprint arXiv:2504.09037, 2025

    2025

  5. [13]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024

    2024

  6. [14]

    Dpo meets ppo: Reinforced token optimization for rlhf.arXiv preprint arXiv:2404.18922, 2024

    Han Zhong, Zikang Shan, Guhao Feng, Wei Xiong, Xinle Cheng, Li Zhao, Di He, Jiang Bian, and Liwei Wang. Dpo meets ppo: Reinforced token optimization for rlhf.arXiv preprint arXiv:2404.18922, 2024

    2024 arXiv

  7. [15]

    Proximal policy optimization algorithms, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017

    2017

  8. [16]

    Dapo: An open-source llm reinforcement learning system at scale, 2025

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wa...

    2025

  9. [17]

    Sky-t1: Train your own o1 preview model within $450

    NovaSky Team. Sky-t1: Train your own o1 preview model within $450. https://novasky- ai.github.io/posts/sky-t1, 2025. Accessed: 2025-01-09

    2025

  10. [18]

    Bespoke-stratos: The unreasonable effectiveness of reasoning distilla- tion

    Bespoke Labs. Bespoke-stratos: The unreasonable effectiveness of reasoning distilla- tion. www.bespokelabs.ai/blog/bespoke-stratos-the-unreasonable-effectiveness-of-reasoning- distillation, 2025. Accessed: 2025-01-22

    2025

  11. [19]

    Qwq-32b: Embracing the power of reinforcement learning, March 2025

    Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025

    2025

  12. [20]

    s1: Simple test-time scaling, 2025

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling, 2025

    2025

  13. [21]

    Limo: Less is more for reasoning, 2025

    Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning, 2025

    2025

  14. [22]

    Millions scale dataset distilled from r1-32b

    Sathwik Tejaswi Madhusudhan, Shruthan Radhakrishna, Jash Mehta, and Toby Liang. Millions scale dataset distilled from r1-32b. https://huggingface.co/datasets/ServiceNow-AI/R1-Distill- SFT, 2025

    2025

  15. [23]

    Llama-nemotron: Efficient reasoning models, 2025

    Akhiad Bercovich, Itay Levy, Izik Golan, Mohammad Dabbah, Ran El-Yaniv, Omri Puny, Ido Galil, Zach Moshe, Tomer Ronen, Najeeb Nabwani, Ido Shahaf, Oren Tropp, and Ehud Karpas.. Llama-nemotron: Efficient reasoning models, 2025

    2025

  16. [24]

    Patil, Matei Zaharia, Joseph E

    Dacheng Li, Shiyi Cao, Tyler Griggs, Shu Liu, Xiangxi Mo, Eric Tang, Sumanth Hegde, Kourosh Hakhamaneshi, Shishir G. Patil, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. Llms can easily learn to reason from demonstrations structure, not content, is what matters!, 2025

    2025

  17. [25]

    The first few tokens are all you need: An efficient and effective unsupervised prefix fine-tuning method for reasoning models.arXiv preprint arXiv:2503.02875, 2025

    Ke Ji, Jiahao Xu, Tian Liang, Qiuzhi Liu, Zhiwei He, Xingyu Chen, Xiaoyuan Liu, Zhijie Wang, Junying Chen, Benyou Wang, et al. The first few tokens are all you need: An efficient and effective unsupervised prefix fine-tuning method for reasoning models.arXiv preprint arXiv:250...

    2025 arXiv

  18. [26]

    Optimizing chain-of-thought reasoners via gradient variance minimization in rejection sampling and rl, 2025

    Jiarui Yao, Yifan Hao, Hanning Zhang, Hanze Dong, Wei Xiong, Nan Jiang, and Tong Zhang. Optimizing chain-of-thought reasoners via gradient variance minimization in rejection sampling and rl, 2025

    2025

  19. [27]

    Teaching large language models to reason through learning and forgetting, 2025

    Tianwei Ni, Allen Nie, Sapana Chaudhary, Yao Liu, Huzefa Rangwala, and Rasool Fakoor. Teaching large language models to reason through learning and forgetting, 2025

    2025

  20. [28]

    Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild, 2025

    Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild, 2025

    2025

  21. [29]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.arXiv preprint arXiv:2403.05530, 2024

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.arXiv preprint arXiv:2403.05530, 2024

    2024 arXiv

  22. [30]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  23. [31]

    Loredana Caruccio, Stefano Cirillo, Giuseppe Polese, Giandomenico Solimando, Shanmugam Sundaramurthy, and Genoveffa Tortora. Claude 2.0 large language model: Tackling a real-world classification problem with a new iterative prompt engineering approach.Intelligent Systems with ...

    2024

  24. [32]

    Flashattention: Fast and memory-efficient exact attention with io-awareness.Advances in Neural Information Processing Systems, 35:16344–16359, 2022

    Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory-efficient exact attention with io-awareness.Advances in Neural Information Processing Systems, 35:16344–16359, 2022

    2022

  25. [33]

    Effective long-context scaling of foundation models.arXiv preprint arXiv:2309.16039, 2023

    Wenhan Xiong, Jingyu Liu, Igor Molybog, Hejia Zhang, Prajjwal Bhargava, Rui Hou, Louis Martin, Rashi Rungta, Karthik Abinav Sankararaman, Barlas Oguz, et al. Effective long-context scaling of foundation models.arXiv preprint arXiv:2309.16039, 2023. 11

    2023 arXiv

  26. [34]

    Liger kernel: Efficient triton kernels for llm training.arXiv preprint arXiv:2410.10989, 2024

    Pin-Lun Hsu, Yun Dai, Vignesh Kothapalli, Qingquan Song, Shao Tang, Siyu Zhu, Steven Shimizu, Shivam Sahni, Haowen Ning, and Yanning Chen. Liger kernel: Efficient triton kernels for llm training.arXiv preprint arXiv:2410.10989, 2024

    arXiv

  27. [35]

    Sequence paral- lelism: Long sequence training from system perspective.arXiv preprint arXiv:2105.13120, 2021

    Shenggui Li, Fuzhao Xue, Chaitanya Baranwal, Yongbin Li, and Yang You. Sequence paral- lelism: Long sequence training from system perspective.arXiv preprint arXiv:2105.13120, 2021

    2021 arXiv

  28. [36]

    The llama 3 herd of models, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, and Angela Fan... The llama 3 herd of models, 2024

    2024

  29. [37]

    Qwen2.5 technical report

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  30. [38]

    Qwen3 technical report, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, and Dayiheng Liu ... Qwen3 technical report, 2025

    2025

  31. [39]

    Hewett, Mojan Javaheripi, Piero Kauffmann, James R

    Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Ol...

    2024

  32. [40]

    Qwen2.5-1m technical report, 2025

    An Yang, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoyan Huang, Jiandong Jiang, Jianhong Tu, Jianwei Zhang, Jingren Zhou, Junyang Lin, Kai Dang, Kexin Yang, Le Yu, Mei Li, Minmin Sun, Qin Zhu, Rui Men, Tao He, Weijia Xu, Wenbiao Yin, Wenyuan Yu, Xiafei Qiu, Xingzhang R...

    2025

  33. [41]

    From 128k to 4m: Efficient training of ultra-long context large language models

    Chejian Xu, Wei Ping, Peng Xu, Zihan Liu, Boxin Wang, Mohammad Shoeybi, and Bryan Catanzaro. From 128k to 4m: Efficient training of ultra-long context large language models. arXiv preprint, 2025

    2025

  34. [42]

    Extending context window of large language models via positional interpolation.arXiv preprint arXiv:2306.15595, 2023

    Shouyuan Chen, Sherman Wong, Liangjian Chen, and Yuandong Tian. Extending context window of large language models via positional interpolation.arXiv preprint arXiv:2306.15595, 2023

    2023 arXiv

  35. [43]

    Yarn: Efficient context window extension of large language models.arXiv preprint arXiv:2309.00071, 2023

    Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. Yarn: Efficient context window extension of large language models.arXiv preprint arXiv:2309.00071, 2023

    2023 arXiv

  36. [44]

    Llm maybe longlm: Self-extend llm context window without tuning, 2024

    Hongye Jin, Xiaotian Han, Jingfeng Yang, Zhimeng Jiang, Zirui Liu, Chia-Yuan Chang, Huiyuan Chen, and Xia Hu. Llm maybe longlm: Self-extend llm context window without tuning, 2024

    2024

  37. [45]

    Efficient streaming language models with attention sinks.arXiv preprint arXiv:2309.17453, 2023

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks.arXiv preprint arXiv:2309.17453, 2023

    2023 arXiv

  38. [46]

    Lm- infinite: Zero-shot extreme length generalization for large language models

    Chi Han, Qifan Wang, Hao Peng, Wenhan Xiong, Yu Chen, Heng Ji, and Sinong Wang. Lm- infinite: Zero-shot extreme length generalization for large language models. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics:...

    2024

  39. [47]

    Infllm: Training-free long-context extrapolation for llms with an efficient context memory

    Chaojun Xiao, Pengle Zhang, Xu Han, Guangxuan Xiao, Yankai Lin, Zhengyan Zhang, Zhiyuan Liu, and Maosong Sun. Infllm: Training-free long-context extrapolation for llms with an efficient context memory. InThe Thirty-eighth Annual Conference on Neural Information Processing Syst...

    2024

  40. [48]

    Landmark attention: Random-access infinite context length for transformers.arXiv preprint arXiv:2305.16300, 2023

    Amirkeivan Mohtashami and Martin Jaggi. Landmark attention: Random-access infinite context length for transformers.arXiv preprint arXiv:2305.16300, 2023

    2023 arXiv

  41. [49]

    Longllmlingua: Accelerating and enhancing llms in long context scenarios via prompt compression.arXiv preprint arXiv:2310.06839, 2023

    Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. Longllmlingua: Accelerating and enhancing llms in long context scenarios via prompt compression.arXiv preprint arXiv:2310.06839, 2023

    arXiv

  42. [50]

    Compressing context to enhance inference efficiency of large language models.arXiv preprint arXiv:2310.06201, 2023

    Yucheng Li, Bo Dong, Chenghua Lin, and Frank Guerin. Compressing context to enhance inference efficiency of large language models.arXiv preprint arXiv:2310.06201, 2023

    2023 arXiv

  43. [51]

    Jingyang Yuan, Huazuo Gao, Damai Dai, Junyu Luo, Liang Zhao, Zhengyan Zhang, Zhenda Xie, Y . X. Wei, Lean Wang, Zhiping Xiao, Yuqing Wang, Chong Ruan, Ming Zhang, Wenfeng Liang, and Wangding Zeng. Native sparse attention: Hardware-aligned and natively trainable sparse attention, 2025

    2025

  44. [52]

    Moba: Mixture of block attention for long-context llms.arXiv preprint arXiv:2502.13189, 2025

    Enzhe Lu, Zhejun Jiang, Jingyuan Liu, Yulun Du, Tao Jiang, Chao Hong, Shaowei Liu, Weiran He, Enming Yuan, Yuzhi Wang, Zhiqi Huang, Huan Yuan, Suting Xu, Xinran Xu, Guokun Lai, Yanru Chen, Huabin Zheng, Junjie Yan, Jianlin Su, Yuxin Wu, Yutao Zhang, Zhilin Yang, Xinyu Zhou, Mi...

    2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.