Pith. sign in

REVIEW 3 major objections 5 minor 49 references

The reasoning-effective part of reinforcement-learning updates sits mostly in the base model’s spectral geometry, so projecting those updates onto that subspace preserves gains while improving exploration and multi-domain transfer.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 05:07 UTC pith:QYPJWJIT

load-bearing objection Useful post-hoc spectral edit of full RL deltas with multi-setting evidence; the >99%/0.58% headline is post-selected, but the operational method and ablations still hold up. the 3 major comments →

arxiv 2607.03065 v1 pith:QYPJWJIT submitted 2026-07-03 cs.LG cs.AI

Spectral Rewiring for Exploration, Purification, and Model Merging

Zhilong Zhang , Hongli Yu , Huan-ang Gao , Hanlin Wu , Yuxuan Song , Wei-Ying Ma , Ya-Qin Zhang , Hao Zhou This is my paper
classification cs.LG cs.AI
keywords reinforcement learninglarge language modelsspectral subspacemodel mergingtest-time scalingreasoning elicitationpost-hoc model editingparameter geometry
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Full-parameter reinforcement learning improves language models on math, code, and related tasks, but the dense updates often narrow exploration and clash when several skills are trained or merged together. This paper argues that the useful part of those updates is largely a rewiring of skills already encoded in the base model’s singular-vector coordinates, not an arbitrary rewrite of the whole weight space. Subspace-Aligned Rewiring extracts that spectral core, discards residual directions, and rebuilds a compact update. Across model sizes, the method keeps nearly all measured post-training accuracy with a tiny fraction of parameters, raises performance under large sampling budgets, cleans mixed-domain training so suppressed skills reappear, and merges domain experts into models that beat prior merge methods and even the best single experts. The practical claim is that a training-free geometric edit can recover and recombine reasoning signal that dense RL had buried.

Core claim

Outcome-reward RL updates that improve reasoning are largely recoverable inside the pretrained spectral subspace of the base weights. Projecting the full update onto that subspace yields a compact rewiring among existing singular directions that preserves nearly all Pass@1 gains, improves high-k exploration, and reduces cross-domain interference in mixed training and expert merging.

What carries the argument

Subspace-Aligned Rewiring (SAR): isolate ΔW = W_RL − W_0, take a top-k low-rank piece, project it as U(U⊤ΔW_k V)V⊤ onto the base SVD factors, and add the resulting rewiring matrix M back to the base model. This keeps on-manifold skill recombination and drops orthogonal residual mass.

Load-bearing premise

When the base model already holds the relevant skills, the update mass that sits outside its spectral geometry can be thrown away without losing essential capability, and is often what suppresses exploration or causes domain clash.

What would settle it

Take a strong outcome-reward RL math model and a matched base; apply SAR at the paper’s reported rank and check whether Pass@1 stays within a few points of full RL while Pass@k at large k and multi-domain merge scores fall below the full RL or best single-expert baselines—especially on cases the paper flags as less projection-compatible, such as long-trained reasoning RL or direct code RL from a raw base.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A dense RL checkpoint can be edited after training into a compact spectral core that retains essentially full measured reasoning accuracy with well under 1% of parameters in the rewiring matrix for several strong recipes.
  • Removing residual off-manifold directions can reverse early Pass@k saturation and expand the set of hard problems solvable by repeated sampling.
  • Mixed-domain RL updates can be purified so suppressed coding performance rises while math and instruction-following hold steady.
  • Math and code experts can be merged through spectral filtering so the merged model exceeds both prior merge baselines and the best single-domain expert on primary metrics.
  • Post-training recipes can treat the rewiring matrix as the editable object for compression, filtering, and composition instead of the full dense delta.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the useful signal is mostly on-manifold rewiring, future RL optimizers could constrain or regularize updates to stay near the base SVD coordinates during training rather than only cleaning them afterward.
  • Projection compatibility itself becomes a diagnostic: large reconstruction gaps after SAR may flag when a run is installing new interface knowledge rather than eliciting latent skills.
  • The same spectral filter might transfer to other dense adaptation steps beyond outcome-reward RL whenever two models share a common pretrained basis.
  • Storing and shipping only the compact rewiring matrix could become a practical way to distribute multi-skill improvements without full fine-tuned weights.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Subspace-Aligned Rewiring (SAR), a post-hoc edit of full-parameter outcome-reward RL updates. It decomposes each linear weight update as ΔW = ΔW* + ΔW⊥ relative to the base model’s SVD basis, extracts a compact rewiring matrix M = U⊤ΔWk V (Eq. 5, Alg. 1), and reconstructs ΔW* = U M V⊤. The claim is that the reasoning-effective part of RL is largely concentrated in the pretrained spectral manifold Sr(W0), so SAR can preserve nearly all measured post-training gains with a tiny M (as low as ~0.58% of parameters), improve high-k Pass@k exploration, purify Mix-RL updates, and improve math–code expert merging beyond TA/TIES/DARE and even single-domain experts. Evidence spans 1.5B–32B models (Tables 1–2, Fig. 2), Mix-RL purification (Fig. 3), merging (Table 4), in-house agentic coding (Table 3), and ablations against no-projection, random, diagonal-only, and off-diagonal controls (App. E–F), with scope limits discussed in §5 and App. D.

Significance. If the geometric hypothesis holds in the stated elicitation regime, SAR is a practically useful, training-free editing tool for RL post-training: it compresses reasoning updates, can improve test-time scaling without new rollouts, and offers a spectral filter for multi-domain consolidation and merging. Strengths include a clear operational procedure, multi-scale public-model results, explicit no-projection and structural ablations, and an honest boundary analysis (JustRL long training, direct code RL, dense-critic PPO). The work is significant as an empirical geometry-of-updates paper rather than as a fully settled theory of reasoning; its main contribution is a falsifiable editing recipe with multi-setting support.

major comments (3)
  1. Table 1 and §3.1: the headline compactness claim (preserve full-RL AIME with 1%/10%/30% spectral rank; M as low as ~0.58% of parameters) is based on a per-model rank sweep that selects the smallest rank already matching full-RL performance within evaluation variance. That is post-selection on the same benchmarks used to claim preservation. Without a pre-specified rank rule, multi-seed variance, or confidence intervals, the “>99% of peak Pass@1 with ~0.58% parameters” statement is weaker than presented. Please report full rank-sweep curves (or fixed a priori budgets such as 1% for all models) and uncertainty so the geometric recovery claim is not confounded with choosing the first matching rank.
  2. §2.2–2.3 and App. F: the mechanistic claim that off-diagonal rewiring (Mij) is the geometric substrate of multi-conditional reasoning is only weakly supported. The off-diagonal-only ablation recovers most Pass@1 but loses Pass@32, while diagonal-only fails; this shows both parts matter under the chosen budget, not that off-diagonal entries specifically implement relational composition. The triangle-area story in App. B is illustrative only. Either demote the relational-reasoning interpretation to a hypothesis, or add a more direct test (e.g., controlled interventions on selected Mij blocks with predicted behavioral effects).
  3. §4.1–4.2 / Fig. 3 / Table 4: the causal claim that residual ΔW⊥ “amplifies cross-domain interference” is inferred from SAR improving coding or merged metrics after projection. That is consistent with filtering harmful residuals, but also with other explanations (regularization, effective low-rank denoising, or lucky rank choice). The no-projection controls help, yet they are still incomplete for the interference story. Please quantify residual mass / domain-specific directions more directly, or state the interference interpretation as a plausible reading rather than established mechanism.
minor comments (5)
  1. Abstract and Table 2: clarify that ~0.58% refers to the rewiring matrix M relative to total parameters, not the reconstructed ΔW*, which is larger.
  2. Algorithm 1 vs. text: the order “top-k of ΔW then project onto base SVD” should be stated once with exact rank definitions for U,V and ΔWk to avoid ambiguity about whether k is shared.
  3. Table 3: in-house agentic coding results are useful but under-specified (model identity, training delta, variance). Even a short appendix note would help external readers assess transfer.
  4. Figure 2 / evaluation: report sampling seeds or bootstrap intervals for Pass@k curves; small absolute coverage gains (25/30 vs 26/30 in Table 5) need uncertainty context.
  5. Related work: the distinction from LoRA/PiSSA/DoRA and from magnitude-based merging is present but could more sharply separate “training parameterization” from “post-hoc spectral filtering of a completed full-RL update.”

Circularity Check

1 steps flagged

Mild rank post-selection on the same AIME metrics used to claim preservation; SAR itself is not definitionally circular and multi-domain/exploration gains are independent tests.

specific steps
  1. fitted input called prediction [§3.1 Experimental Setup / Table 1; Abstract compression claim]
    "We perform a rank sweep and report the smallest retained spectral rank that preserves full-RL performance within evaluation variance. ... SAR reports the smallest retained spectral rank found by rank sweep that preserves full-RL-level performance within evaluation variance. ... extracts compact reasoning cores using as little as ∼0.58% of total parameters: it preserves over 99% of post-training performance"

    The headline preservation and parameter-efficiency numbers are obtained by selecting, per model, the smallest spectral rank that already matches full-RL AIME within evaluation variance, then reporting that those ranks preserve performance. At the chosen k, “preserves full-RL-level performance” is therefore partly forced by the selection rule rather than an independent a priori prediction. This is mild: existence of small working ranks and transfer to Pass@k / multi-domain / merging remain empirical, not definitional.

full rationale

The paper’s load-bearing construction is geometric, not tautological: ΔW* = U(U⊤ΔW V)V⊤ (Eq. 5 / Alg. 1) is defined from the base SVD and the empirical RL delta without reference to AIME, LiveCodeBench, or IFEval. Preservation, high-k scaling, Mix-RL purification, and expert merging are then measured on external benchmarks, with ablations (no-projection, random basis, diagonal/off-diagonal-only; Appendices E–F) that can and sometimes do fail. That is ordinary hypothesis-and-test structure, not self-definitional circularity. The only mild circularity-adjacent step is the per-model rank sweep that chooses the smallest k already matching full-RL AIME within variance, then headlines “preserves over 99% … with ~0.58% parameters.” Preservation at those selected ranks is partly by construction of the selection rule; the non-circular content is that such small ranks exist at all and that the same fixed projections improve Pass@k, coding, and merging without being re-fit for those outcomes. No uniqueness theorem, self-citation chain, or renamed known result forces the central claim. Boundary cases in §5/Appendix D further show the method is incomplete outside the elicitation regime, which is inconsistent with a by-construction derivation. Overall circularity is low (score 2).

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 3 invented entities

The load-bearing content is an empirical geometric hypothesis plus a free rank budget, not a parameter-free theorem. Standard linear algebra (SVD, orthogonal projection) is free; the claim that reasoning-effective RL mass lives in the base spectral manifold, and that off-manifold residual is mostly interference, is a domain assumption validated only within the reported regimes. Invented operational objects (rewiring matrix, spectral manifold as editing coordinate) organize the method but lack independent evidence outside these experiments.

free parameters (3)
  • target spectral rank fraction k = model-dependent: ~1% (DeepScaleR, OLMo-32B), 10% (POLARIS), 30% (OLMo-7B base→RL)
    Chosen by rank sweep as the smallest retained rank that preserves full-RL performance within evaluation variance (1%, 10%, 30% depending on model; top-1% used for Mix-RL and coding). Central compression and transfer claims depend on this selection.
  • which parameter groups are projected = Wq,Wk,Wv,Wo,Wgate,Wup,Wdown only
    Attention and MLP linear weights only; embeddings, norms, biases, lm_head frozen from base. Empirically necessary per Appendix C, but a design choice affecting the extracted core.
  • low-rank extraction before spectral projection = top-k component of ΔW then U⊤ΔWk V
    Algorithm 1 first extracts top-k of ΔW then projects; the dual compression step is a free algorithmic choice that interacts with reported parameter ratios.
axioms (5)
  • domain assumption SVD singular vectors of base weights form a functional basis of latent skills for read-in/read-out in linear layers.
    §2.1 motivates this from the SVD expansion and compression literature; not proved for reasoning circuits, only used as coordinate system.
  • ad hoc to paper Reasoning-effective part of outcome-reward RL updates is recoverable inside the pretrained spectral subspace of the reference model.
    Core geometric hypothesis in §2.2; the method and strongest claims rest on it. Boundary cases in §5/App. D show it is regime-dependent.
  • ad hoc to paper Orthogonal residual ΔW⊥ largely encodes residual directions that suppress exploration or amplify cross-domain interference rather than necessary capability.
    Used to interpret Pass@k gains and Mix-RL/merging improvements (§3.2, §4); supported by controls but not independently established.
  • standard math Standard SVD and orthogonal projectors PU=UU⊤, PV=VV⊤ correctly extract the subspace-aligned component.
    Linear algebra used in Eq. (5) and Algorithm 1.
  • domain assumption Outcome-reward RL primarily elicits/reorganizes knowledge already present in the base (or SFT reference) rather than installing wholly new bases when projection works.
    Stated in introduction and Discussion; aligns with cited “does RL incentivize beyond base” literature but is assumed for SAR’s scope.
invented entities (3)
  • Rewiring matrix M = U⊤ΔW V independent evidence
    purpose: Compact object encoding diagonal rescaling and off-diagonal cross-skill routing attributed to RL reasoning gains.
    Defined in §2.2; operationally useful, but “rewiring = reasoning” is interpretive. Ablations give partial independent structure (off-diagonal matters).
  • Pretrained spectral manifold Sr(W0)=span(U⊗V) no independent evidence
    purpose: Coordinate system claimed to host reasoning-effective updates and to filter interference.
    Named geometric object for projection; evidence is success of SAR vs controls, not an external measurement of a manifold.
  • Subspace-Aligned Rewiring (SAR) procedure independent evidence
    purpose: Post-hoc edit: low-rank extract ΔW, project into base SVD basis, reconstruct W0+UMV⊤.
    The paper’s method; falsifiable via reimplementation on public RL deltas.

pith-pipeline@v1.1.0-grok45 · 22217 in / 4075 out tokens · 35920 ms · 2026-07-12T05:07:44.088059+00:00 · methodology

0 comments
read the original abstract

Reinforcement learning has become a standard post-training recipe for large language models, but dense full-parameter updates create two deployment-relevant bottlenecks: suppressed reasoning performance, often reflected by premature saturation of test-time scaling, and interference when consolidating multiple capabilities through multi-domain training or model merging. We show that the reasoning-effective component of these updates is largely concentrated in the base model's spectral space, motivating Subspace-Aligned Rewiring (SAR), a post-hoc editing method that retains this spectral core while removing orthogonal components. SAR therefore preserves reasoning gains and filters residual update directions that suppress performance or amplify cross-domain interference. Across several model families and scales, SAR extracts compact reasoning cores using as little as approximately 0.58% of total parameters: it preserves over 99% of post-training performance and improves high-k exploration in mathematical reasoning, and generalizes to agentic coding by improving six of seven open benchmarks on an in-house model. SAR also purifies mixed-domain training updates by releasing suppressed coding capability while maintaining math reasoning and instruction following. It further enables model merging across experts, yielding cross-domain generalization that surpasses previous merging baselines and even the best single-domain experts. Overall, SAR shows that extracting reasoning-effective updates from parameter geometry can serve as a training-free mechanism to improve reasoning and multi-domain performance.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 36 linked inside Pith

  1. [1]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    Pith/arXiv arXiv 2024

  2. [2]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Pith/arXiv arXiv 2021

  4. [4]

    Let’s verify step by step.arXiv preprint arXiv:2305.20050, 2023

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step.arXiv preprint arXiv:2305.20050, 2023

    Pith/arXiv arXiv 2023

  5. [5]

    Hannah Rose Kirk, Bertie Vidgen, Paul Rottger, and Scott A. Hale. Understanding the effects of rlhf on llm generalisation and diversity.arXiv preprint arXiv:2310.06452, 2023

    Pith/arXiv arXiv 2023

  6. [6]

    Confronting reward model overoptimization with constrained rlhf.arXiv preprint arXiv:2310.04373, 2023

    Ted Moskovitz, Archit Singh, DJ Strouse, Tuomas Sandholm, Ruslan Salakhutdinov, Anca Dragan, and Stephen McAleer. Confronting reward model overoptimization with constrained rlhf.arXiv preprint arXiv:2310.04373, 2023

    Pith/arXiv arXiv 2023

  7. [7]

    Bradley C. A. Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V. Le, Christopher Re, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint arXiv:2407.21787, 2024

    Pith/arXiv arXiv 2024

  8. [8]

    Scaling llm test-time compute optimally can be more effective than scaling model parameters.arXiv preprint arXiv:2408.03314, 2024

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters.arXiv preprint arXiv:2408.03314, 2024

    Pith/arXiv arXiv 2024

  9. [9]

    Editing models with task arithmetic.arXiv preprint arXiv:2212.04089, 2022

    Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic.arXiv preprint arXiv:2212.04089, 2022

    Pith/arXiv arXiv 2022

  10. [10]

    Ties-merging: Resolving interference when merging models.arXiv preprint arXiv:2306.01708, 2023

    Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. Ties-merging: Resolving interference when merging models.arXiv preprint arXiv:2306.01708, 2023

    Pith/arXiv arXiv 2023

  11. [11]

    Language models are super mario: Absorbing abilities from homologous models as a free lunch.arXiv preprint arXiv:2311.03099, 2023

    Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. Language models are super mario: Absorbing abilities from homologous models as a free lunch.arXiv preprint arXiv:2311.03099, 2023

    Pith/arXiv arXiv 2023

  12. [12]

    Song Han, Huizi Mao, and William J. Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding.arXiv preprint arXiv:1510.00149, 2015

    Pith/arXiv arXiv 2015

  13. [13]

    Asvd: Activation-aware singular value decomposition for compressing large language models.arXiv preprint arXiv:2312.05821, 2023

    Zhihang Yuan, Yuzhang Shang, Yue Song, Qiang Wu, Yan Yan, and Guangyu Sun. Asvd: Activation-aware singular value decomposition for compressing large language models.arXiv preprint arXiv:2312.05821, 2023

    Pith/arXiv arXiv 2023

  14. [14]

    Svd-llm: Truncation-aware singular value decomposition for large language model compression.arXiv preprint arXiv:2403.07378, 2024

    Xin Wang, Yu Zheng, Zhongwei Wan, and Mi Zhang. Svd-llm: Truncation-aware singular value decomposition for large language model compression.arXiv preprint arXiv:2403.07378, 2024

    Pith/arXiv arXiv 2024

  15. [15]

    Halford, William H

    Graeme S. Halford, William H. Wilson, and Steven Phillips. Processing capacity defined by relational complexity: Implications for comparative, developmental, and cognitive psychology.Behavioral and Brain Sciences, 21(6): 803–831, 1998

    1998

  16. [16]

    Adam Santoro, David Raposo, David G. T. Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, and Timothy Lillicrap. A simple neural network module for relational reasoning.arXiv preprint arXiv:1706.01427, 2017

    Pith/arXiv arXiv 2017

  17. [17]

    Battaglia, Jessica B

    Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. Relational inductive biases, deep learning, and graph networks.arXiv preprint arXiv:1806.01261, 2018

    Pith/arXiv arXiv 2018

  18. [18]

    Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Li, Li Erran Yang, et al

    Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Li, Li Erran Yang, et al. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl.arXiv preprint arXiv:2502.01682, 2025. 13

    Pith/arXiv arXiv 2025

  19. [19]

    POLARIS: A post-training recipe for scaling reinforcement learning on advanced reasoning models.https://hkunlp.github.io/blog/2025/Polaris/, 2025

    POLARIS Project. POLARIS: A post-training recipe for scaling reinforcement learning on advanced reasoning models.https://hkunlp.github.io/blog/2025/Polaris/, 2025. Accessed 2026-05-15

    2025

  20. [20]

    OLMo 3: A family of open language models.arXiv preprint arXiv:2512.13961, 2025

    AI2. OLMo 3: A family of open language models.arXiv preprint arXiv:2512.13961, 2025

    Pith/arXiv arXiv 2025

  21. [21]

    Open-reasoner-zero: An open source approach to scaling reinforcement learning on the base model

    Jian Hu, Xibin Wu, Weixun Fu, Xinyu Chen, Chen Xu, Weizhi Zhu, Jiaxin Pei, Zixuan Zhong, Jiawei Zheng, Zheng Chen, et al. Open-reasoner-zero: An open source approach to scaling reinforcement learning on the base model. arXiv preprint arXiv:2503.24290, 2025

    Pith/arXiv arXiv 2025

  22. [22]

    Evaluating large language models trained on code

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021

    Pith/arXiv arXiv 2021

  23. [23]

    Self-consistency improves chain of thought reasoning in language models.arXiv preprint arXiv:2203.11171, 2022

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models.arXiv preprint arXiv:2203.11171, 2022

    Pith/arXiv arXiv 2022

  24. [24]

    Hendryx, Zifan Wang, Chen Bo Calvin Zhang, Noah Jacobson, Bing Liu, and Brad Kenstler

    Xiang Deng, Jeff Da, Edwin Pan, Yan He, Charles Ide, Kanak Garg, Niklas Lauffer, Andrew Park, Nitin Pasari, Chetan Rane, Karmini Sampath, Maya Krishnan, Srivatsa Kundurthy, Sean M. Hendryx, Zifan Wang, Chen Bo Calvin Zhang, Noah Jacobson, Bing Liu, and Brad Kenstler. Swe-bench pro: Can ai agents solve long-horizon software engineering tasks? 2025. URLhttp...

    2025

  25. [25]

    Openhands: An open platform for ai software developers as generalist agents

    Xingyao Wang, Boxuan Li, Yufan Song, Frank F Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Daniel Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng, Heng Ji, and Graham Neubig. Openhands: An open platform for ai software...

    2025

  26. [26]

    Swe-bench: Can language models resolve real-world github issues? InThe TwelfthInternational Conference on Learning Representations, 2024

    Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. Swe-bench: Can language models resolve real-world github issues? InThe TwelfthInternational Conference on Learning Representations, 2024. URLhttps://openreview.net/forum?id=VTF8yNQM66

    2024

  27. [27]

    Multi-swe-bench: A multilingual benchmark for issue resolving, 2025

    Daoguang Zan, Zhirong Huang, Wei Liu, Hanwu Chen, Linhao Zhang, Shulin Xin, Lu Chen, Qi Liu, Xiaojian Zhong, Aoyan Li, Siyao Liu, Yongsheng Xiao, Liangqiang Chen, Yuyu Zhang, Jing Su, Tianyu Liu, Rui Long, Kai Shen, and Liang Xiang. Multi-swe-bench: A multilingual benchmark for issue resolving, 2025. URL https://arxiv.org/abs/2504.02605

    Pith/arXiv arXiv 2025

  28. [28]

    Merrill, Alexander G

    Mike A. Merrill, Alexander G. Shaw, Nicholas Carlini, Boxuan Li, Harsh Raj, Ivan Bercovich, Lin Shi, Jeong Yeon Shin, Thomas Walshe, E. Kelly Buchanan, Junhong Shen, Guanghao Ye, Haowei Lin, Jason Poulos, Maoyu Wang, Marianna Nezhurina, Jenia Jitsev, Di Lu, Orfeas Menis Mastromichalakis, Zhiwei Xu, Zizhao Chen, Yue Liu, Robert Zhang, Leon Liangyu Chen, An...

    Pith/arXiv arXiv 2026

  29. [29]

    Livecodebench: Holistic and contamination free evaluation of large language models for code

    Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974, 2024

    Pith/arXiv arXiv 2024

  30. [30]

    Instruction-following evaluation for large language models.arXiv preprint arXiv:2311.07911, 2023

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models.arXiv preprint arXiv:2311.07911, 2023

    Pith/arXiv arXiv 2023

  31. [31]

    Qwen2.5 technical report.arXiv preprint arXiv:2412.15115, 2024

    Qwen Team. Qwen2.5 technical report.arXiv preprint arXiv:2412.15115, 2024. 14

    Pith/arXiv arXiv 2024

  32. [32]

    Qwen2.5-math technical report: Toward mathematical expert model via self-improvement

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowei Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2.5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122, 2024

    Pith/arXiv arXiv 2024

  33. [33]

    Stabilizing knowledge, promoting reasoning: Dual-token constraints for rlvr, 2025

    Jiakang Wang, Runze Liu, Fuzheng Zhang, Xiu Li, and Guorui Zhou. Stabilizing knowledge, promoting reasoning: Dual-token constraints for rlvr, 2025. URLhttps://arxiv.org/abs/2507.15778

    Pith/arXiv arXiv 2025

  34. [34]

    Deepcoder: A fully open-source 14b coder at o3-mini level

    Together AI. Deepcoder: A fully open-source 14b coder at o3-mini level. https://www.together.ai/blog/ deepcoder, 2025. Accessed 2026-05-15

    2025

  35. [35]

    Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?arXiv preprint arXiv:2504.13837, 2025

    Xiang Yue, Zhuo Chen, and Wenhu Chen. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?arXiv preprint arXiv:2504.13837, 2025

    Pith/arXiv arXiv 2025

  36. [36]

    Pass@k policy optimization: Solving harder reinforcement learning problems

    Christian Walder and Deep Karkhanis. Pass@k policy optimization: Solving harder reinforcement learning problems. arXiv preprint arXiv:2505.15201, 2025

    Pith/arXiv arXiv 2025

  37. [37]

    Pass@k training for adaptively balancing exploration and exploitation of large reasoning models.arXiv preprint arXiv:2508.10751, 2025

    Zhipeng Chen, Xiaobo Qin, Youbin Wu, Yue Ling, Qinghao Ye, Wayne Xin Zhao, and Guang Shi. Pass@k training for adaptively balancing exploration and exploitation of large reasoning models.arXiv preprint arXiv:2508.10751, 2025

    Pith/arXiv arXiv 2025

  38. [38]

    Maximum likelihood reinforcement learning.arXiv preprint arXiv:2602.02710, 2026

    Fahim Tajwar, Guanning Zeng, Yueer Zhou, Yuda Song, Daman Arora, Yiding Jiang, Jeff Schneider, Ruslan Salakhutdinov, Haiwen Feng, and Andrea Zanette. Maximum likelihood reinforcement learning.arXiv preprint arXiv:2602.02710, 2026

    arXiv 2026

  39. [39]

    Simko: Simple pass@k policy optimization

    Ruotian Peng, Yi Ren, Zhouliang Yu, Weiyang Liu, and Yandong Wen. Simko: Simple pass@k policy optimization. arXiv preprint arXiv:2510.14807, 2025

    Pith/arXiv arXiv 2025

  40. [40]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021

    Pith/arXiv arXiv 2021

  41. [41]

    Dora: Weight-decomposed low-rank adaptation.arXiv preprint arXiv:2402.09353, 2024

    Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. Dora: Weight-decomposed low-rank adaptation.arXiv preprint arXiv:2402.09353, 2024

    Pith/arXiv arXiv 2024

  42. [42]

    Pissa: Principal singular values and singular vectors adaptation of large language models.arXiv preprint arXiv:2404.02948, 2024

    Fanxu Meng, Zhaohui Wang, and Muhan Zhang. Pissa: Principal singular values and singular vectors adaptation of large language models.arXiv preprint arXiv:2404.02948, 2024

    Pith/arXiv arXiv 2024

  43. [43]

    Lora-xs: Low-rank adaptation with extremely small number of parameters.arXiv preprint arXiv:2405.17604, 2024

    Klaudia Balazy, Mohammadreza Banaei, Karl Aberer, and Jacek Tabor. Lora-xs: Low-rank adaptation with extremely small number of parameters.arXiv preprint arXiv:2405.17604, 2024

    Pith/arXiv arXiv 2024

  44. [44]

    On predictability of reinforcement learning dynamics for large language models.arXiv preprint arXiv:2510.00553, 2025

    Yuchen Cai, Ding Cao, Xin Xu, Zijun Yao, Yuqing Huang, Zhenyu Tan, Benyi Zhang, Guiquan Liu, and Junfeng Fang. On predictability of reinforcement learning dynamics for large language models.arXiv preprint arXiv:2510.00553, 2025

    arXiv 2025

  45. [45]

    Morris, Niloofar Mireshghallah, Mark Ibrahim, and Saeed Mahloujifar

    John X. Morris, Niloofar Mireshghallah, Mark Ibrahim, and Saeed Mahloujifar. Learning to reason in 13 parameters. arXiv preprint arXiv:2602.04118, 2026

    arXiv 2026

  46. [46]

    Enough is as good as a feast: A comprehensive analysis of how reinforcement learning mitigates task conflicts in llms

    Zixuan Ren, Jinliang Lu, Junhong Wu, Yang Zhao, Dai Dai, Hua Wu, Haifeng Wang, and Chengqing Zong. Enough is as good as a feast: A comprehensive analysis of how reinforcement learning mitigates task conflicts in llms. InInternational Conference on Learning Representations, 2026. URL https://openreview.net/forum? id=N4l4Jp50R4

    2026

  47. [47]

    JustRL: Scaling a 1.5b llm with a simple rl recipe.https://iclr-blogposts.github.io/2026/ blog/2026/justrl/, 2026

    JustRL Team. JustRL: Scaling a 1.5b llm with a simple rl recipe.https://iclr-blogposts.github.io/2026/ blog/2026/justrl/, 2026. ICLR Blogposts 2026

    2026

  48. [48]

    X-Coder: Advancing competitive programming with fully synthetic tasks, solutions, and tests.arXiv preprint arXiv:2601.06953, 2026

    Jie Wu, Haoling Li, Xin Zhang, Jiani Guo, Jane Luo, Steven Liu, Yangyu Huang, Ruihang Chu, Scarlett Li, and Yujiu Yang. X-Coder: Advancing competitive programming with fully synthetic tasks, solutions, and tests.arXiv preprint arXiv:2601.06953, 2026. 15 Appendix A Related Work Exploration and exploitation in outcome-reward RL.Recent work studies reasoning...

    arXiv 2026

  49. [49]

    The no-projection control keeps the same top-1% low-rank component but removes the projection onto the pretrained SVD subspace

    Table 8 compares SAR with three controls: replacing the pretrained spectral basis with a random 19 Table 6Head-to-head comparison on AIME 2025 and LiveCodeBench v5. The no-projection control keeps the same top-1% low-rank component but removes the projection onto the pretrained SVD subspace. SAR achieves the strongest large-kreasoning coverage and coding ...

    2025