Pith. sign in

REVIEW 4 major objections 4 minor 44 references

The paper claims that RLVR can reuse a pretrained model's singular-value spectra and acquire new behavior by changing only the associated singular frames, then builds an optimization stack that fixes the spectrum and optimizes the frames.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 12:42 UTC pith:TIFJ32FD

load-bearing objection The paper gets the spectral story right but overreaches on the coordinate-level algorithms; still worth refereeing. the 4 major comments →

arxiv 2607.19331 v1 pith:TIFJ32FD submitted 2026-07-21 cs.LG cs.AI

ISO: An RLVR-Native Optimization Stack

Hanqing Zhu , Wenyan Cong , Zhizhou Sha , Sagnik Mukherjee , Xinyuan Song , David Gonz\'alez-Mart\'inez , Xiaoxia Wu , Yuandong Tian
show 3 more authors
Shiwei Liu David Z. Pan Zhangyang "Atlas" Wang
This is my paper
classification cs.LG cs.AI
keywords reinforcement learning with verifiable rewardsspectral inheritancesingular value decompositionfixed-spectrum optimizationsingular framesmodel mergingRLVR optimizationisospectral optimization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper's central claim is that reinforcement learning with verifiable rewards (RLVR) does not need to rewrite a pretrained model's singular-value spectra. RLVR can inherit the base spectra and acquire new capabilities by moving the input and output singular frames, a pattern the authors name spectral inheritance. The paper turns this into Isospectral Optimization (ISO), which fixes the base spectrum and optimizes frame variables, both offline for merging shared-base RL experts and online as a wrapper around standard optimizers. If the claim is right, RLVR gets a purpose-built optimization layer: constrained to a fixed-spectrum family, training reaches matched accuracy in substantially fewer steps—for instance, 2.7x fewer on an 8-billion-parameter model—and checkpoint-only expert merging works without rollouts or distillation.

Core claim

At the level of each weight matrix W = UΣV^T, the paper's central discovery is that unconstrained RLVR leaves the singular values Σ close to the base values while the bulk of the checkpoint change lives in the left and right singular frames (U, V). Restoring the base spectrum after training preserves most acquired performance, and keeping it fixed throughout training still supports strong learning; freezing either singular frame instead leaves a large fraction of the update unexplained. The authors call this spectral inheritance and make it a design: parameterize post-training weights as W(U,V) = UΣ0 V^T, with Σ0 inherited from the base, and let both frames adapt. The functional claim is tha

What carries the argument

The carrier of the argument is the fixed-spectrum family F(W0): all matrices with the same singular values as the base weight W0 but arbitrary left and right orthonormal frames. ISO parameterizes weights as W(U,V) = UΣ0V^T, so every iterate stays in F(W0). Proposition 2.1 shows the distance from any checkpoint to this family is exactly the singular-value drift; Proposition 4.1 shows feasible frame motion preserves the spectrum to first order. ISO-Merger works in the Stiefel tangent space at the base factors, masks trailing modes, solves a unit-retention linear system, and retracts via the polar factor; ISO-Optimizer runs the chosen base optimizer on (U,V) and retracts with the same polar ope

Load-bearing premise

The load-bearing premise is that each weight matrix has distinct-enough singular values that stay in the same rank order during training, so the matching between base and learned singular directions is a meaningful coordinate system rather than an arbitrary choice of gauge.

What would settle it

Take a checkpoint pair from the paper's own diagnosis and recompute the unexplained-update ratio after applying random orthogonal rotations inside every cluster of nearly equal singular values. If u_iso rises from roughly 2% to tens of percent under such gauge changes, the 'both frames must adapt' conclusion is an artifact of a particular SVD sign and ordering convention rather than a property of the learned geometry.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • RLVR post-training can be redesigned as constrained optimization over singular frames with the base spectrum fixed, without losing the behavior that unconstrained RLVR acquires.
  • Checkpoint-only merging of shared-base RL specialists becomes viable: ISO-Merger recovers complementary capabilities without post-merge data, rollouts, or distillation, and outperforms the compared data-free merging baselines.
  • Fixed-spectrum training converges faster: ISO-AdamW reaches matched accuracy with 2.7x fewer steps at the 8-billion-parameter scale, with similar early gains at smaller scales and across coding and math.
  • The 'both frames must adapt' structure constrains mechanistic explanations of RLVR: remixing within one subspace or freezing one singular frame cannot explain the endpoint changes.
  • The pattern survives objective shifts between sequential RL stages, so the principle transfers beyond a single RL run.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If spectral inheritance is generic, RLVR checkpoints trained from the same base should be unusually easy to merge; ISO-Merger's success suggests frame coordinates form a low-interference basis, a testable claim about the geometry of RLVR task vectors.
  • The fixed-spectrum parameterization offers a direct ablation tool: swapping spectra between checkpoints while keeping frames fixed can localize which capabilities are carried by spectra and which by frames.
  • A cautious reading suggests the two-frame conclusion is only as strong as the simple-spectrum assumption; a gauge-robust version would need to handle near-degenerate singular blocks explicitly.
  • The online speedups likely depend on the retraction being amortized against rollout time; in asynchronous RL systems this overhead should be overlapped with generation rather than placed on the critical path.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper studies RLVR weight-space updates through the singular-value decomposition of weight matrices. It argues that RLVR checkpoints remain close to the fixed-spectrum family of their base weights (near-isospectrality), that the small spectral changes are not functionally necessary (spectral inheritance), and that a low-residual description of the checkpoint change requires both left and right singular frames to adapt while the spectrum stays fixed. These findings are operationalized as Isospectral Optimization (ISO): offline, ISO-Merger combines shared-base RL specialists by aggregating masked Stiefel tangent displacements under the base spectrum; online, ISO-Optimizer runs a base optimizer such as AdamW or Muon on the frame variables (U,V) with the base spectrum fixed. Experiments cover math and coding RLVR from 1.5B to 8B parameters and data-free merging of 2–3 RL experts; the headline result is that ISO-AdamW reaches AdamW's final accuracy on Qwen3-8B-Base in about 2.7x fewer training steps, and ISO-Merger attains the best aggregate score among the compared merging baselines.

Significance. The conceptual separation of 'inherited spectrum' from 'adapted frames' is potentially valuable for RLVR optimization and for checkpoint-only model merging. The paper is careful to calibrate raw near-isospectrality against a dimensional baseline and to use gauge-invariant projector diagnostics in Section 3; the same-base SFT/RL comparison and the sequential RL objective-shift test are well-designed controls. The theoretical propositions (Prop. 2.1, 3.1, 4.1) are standard and correctly derived, and the appendices provide explicit reconstruction classes and rank-sensitivity sweeps. If the empirical results hold, this is a substantive contribution to the RLVR optimization layer. However, the central algorithmic reading—that individual singular-vector displacements are meaningful optimization/composition coordinates—is not adequately validated against the known SVD gauge problem, and the online results are single runs with checkpoint selection. The significance therefore rests on evidence that is partly not yet established.

major comments (4)
  1. [§4.2, Eq. (20); Appendix A; Appendix E.2; Remark B.3] The frame-displacement construction is the load-bearing coordinate system of the paper, but its coordinate invariance is not established. Eqs. (20)–(23) and Algorithm 1 sum individual left/right singular-vector displacements ΔU_i, ΔV_i after only sign canonicalization. These are meaningful coordinates only if the base and post-training spectra are simple with adequate separation. Appendix A is explicitly a qualitative sanity check, and Appendix E.2 merely asserts that 'generic weight matrices have simple spectra' without testing the trained checkpoints used in Tables 1–4 or Figures 6–9. Remark B.3 shows that near-degenerate singular blocks admit in-block rotations that change ΔU_i and ΔV_i arbitrarily. Please report singular-value gap statistics (e.g., (σ_k−σ_{k+1})/σ_k) for the relevant matrices, and test the invariance of ISO-Merger and ISO-Optimizer outputs under random in-block ortho
  2. [§5.2, Table 3, Figure 9] The online experiments are single runs, and the abstract's headline convergence comparison is not accompanied by uncertainty estimates. Table 3 states that the checkpoint with the highest aggregate among the final three evaluations is selected; Figures 6–9 show individual trajectories with no seed variance. On Qwen3-8B-Base, the 2.7x claim compares one ISO-AdamW trajectory at step 100 with a continued AdamW run at step 270, but no confidence interval is given. The AdamW baselines are tuned over learning-rate sweeps while ISO-AdamW uses a single rate, so part of the apparent gain could be due to hyperparameter asymmetry or evaluation noise. Please provide multiple seeds, or at least bootstrap confidence intervals over evaluation samples, for the main convergence-speed claims; otherwise the 'substantially fewer training steps' statement should be qualified.
  3. [§3, Eq. (11), Take-away 2] The conclusion that 'both frames must remain adaptable' is inferred from reconstruction residuals, not from training-time ablations. A low unexplained-update ratio for b̂W_iso shows that a fixed-spectrum, two-frame-adaptable class provides a good representation of the endpoint; it does not demonstrate that freezing one frame during actual RLVR training prevents learning. The wording in Take-away 2 and in Section 4.1 is stronger than the evidence. Please add a controlled training run in which U is optimized while V is held at V_0 (or vice versa) under fixed Σ_0, or soften the claim to 'both frames are needed for a low-residual endpoint description.' This distinction is load-bearing because the design of ISO-Optimizer is motivated by the necessity claim.
  4. [§5.1, Tables 1 and 2] The reported aggregate gains of ISO-Merger over the strongest baselines are small relative to the per-column standard deviations. In Table 1, ISO-Merger averages 63.80 versus 62.88 for the strongest baseline; in Table 2, 44.38 versus 43.52. Individual columns have standard errors up to 3 points, and no aggregate uncertainty, paired-difference statistics, or multiple-merge repetitions are reported. The claim of 'strongest aggregate performance' should either be supported by appropriate statistics or explicitly qualified as not statistically distinguished from the top baselines.
minor comments (4)
  1. [Abstract] 'identifyspectral' in the abstract is missing a space; please proofread the final text.
  2. [§2.1, Figure 2(c)] The text says δΣ is 'approximately 10^{-2}%', but the figure axis is on a log scale from 10^{-4} to 10^{-1}; this appears to be a units/notation inconsistency. Please clarify whether the reported number is 10^{-2} (dimensionless) or 10^{-2}%.
  3. [Appendix E.5 / §4.2] The ridge parameter λ_ridge and clip range [0,1.5] are free choices; only default values are given. A brief sensitivity analysis over these two hyperparameters would help the reader understand the stability of ISO-Merger's retention coefficients.
  4. [Repository links] The code and data links appear as placeholder artifacts. Please provide the actual repository URL and dataset identifiers in the final version; the paper's central diagnostics cannot be inspected otherwise.

Circularity Check

0 steps flagged

No significant circularity; the few self-citations are motivational, not load-bearing, and the central claims are tested on held-out benchmarks.

full rationale

The derivation chain is self-contained. Near-isospectrality is measured directly on public checkpoints (DS-1.5B and Qwen3-8B, Equations 3-6 and Figures 2-3), with a dimension-aware calibration that explicitly weakens the 'preference' interpretation. The functional claims of spectral inheritance are established by interventions: restoring base spectra after training, fixing spectra during training, and computing gauge-invariant reconstruction residuals (Propositions 2.1 and 3.1; Figures 4-5). These are not fitted constants renamed as predictions. ISO-Merger and ISO-Optimizer are evaluated on held-out benchmarks (Tables 1-4 and Figures 6-9), and their algorithms do not define the target metric in terms of fitted parameters. The paper does cite its own prior work [1] for the initial observation of limited spectral drift and for broader evidence, but the present manuscript independently reproduces the key measurements and adds functional tests, so the self-citation is supplementary rather than load-bearing. Appendix A honestly states that SVD bases are gauge-dependent and offers only a qualitative index-alignment check; Appendix E.2 asserts simple spectra without proof for the trained checkpoints. These are validity caveats about coordinate meaning, not circularity, because the central functional claims are backed by gauge-invariant projectors and external benchmark evaluations.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The central scientific content is empirical; the formal backbone is standard matrix analysis. The main nonstandard ingredients are the empirical rank-truncation choices and the assumption that singular-frame coordinates remain well-defined across checkpoints. No invented physical entities are introduced.

free parameters (5)
  • rho_keep = 0.9
    ISO-Merger trailing-mode masking ratio; chosen because trailing singular modes were found empirically unstable across experts (Eq. 22, App. E.4); affects merged performance.
  • lambda_ridge = 1e-12
    Ridge stabilizer for the unit-retention coefficient solve (Eq. 28, App. E.5); small, but a free choice in the merger.
  • clip range for retention coefficients = [0, 1.5]
    Coefficient clipping bounds in Eq. 28/App. E.5; prevents sign reversal/amplification but is a tuned hyperparameter.
  • diagnostic truncation rank r = floor(0.9q)
    Rank truncation for frame-adaptability/explained-update analysis (Sec. 3, App. B.4); the conclusion is rank-stable, but 0.9 is a choice.
  • ISO optimizer learning rates = 7.5e-7 math; 1e-6 coding; 5e-5 ISO-Muon
    Online optimizer learning rates; ISO-AdamW math LR selected by comparing two values and keeping the better one (App. H); baselines were swept.
axioms (5)
  • standard math SVD, von Neumann trace inequality, Stiefel-manifold tangent geometry
    Used in Propositions 2.1, 3.1, 4.1 and Appendix B/C; accepted background.
  • domain assumption RLVR weight updates are sparse/off-principal and shared across runs
    From prior work [1,15]; motivates spectral view but not rederived.
  • domain assumption Learned RLVR checkpoints remain near the fixed-spectrum family of the base
    Empirical near-isospectrality across analyzed runs; foundational to ISO.
  • domain assumption Simple spectrum and rank-index-stable singular frames
    Frame coordinates and sign canonicalization are meaningful only if near-degenerate singular-value blocks do not mix frames; Appendix A only sanity-checks this.
  • ad hoc to paper Trailing singular modes can be discarded without losing specialist capability
    Empirical statement in App E.4; no formal criterion.

pith-pipeline@v1.3.0-alltime-deepseek · 39931 in / 14415 out tokens · 147912 ms · 2026-08-01T12:42:51.756432+00:00 · methodology

0 comments
read the original abstract

Reinforcement learning with verifiable rewards (RLVR) is rapidly advancing the reasoning capabilities of language models, yet the optimization layer that converts reward feedback into weight-space updates remains poorly understood. Building on our prior analysis (Zhu et al., 2025), we study this missing layer through the singular structure of model weights and identify spectral inheritance: RLVR can reuse the base model's weight spectra while acquiring new behavior through changes in the associated input and output singular frames. We operationalize spectral inheritance as Isospectral Optimization (ISO), an RLVR-native, fixed-spectrum optimization framework with complementary offline and online instantiations. Offline, ISO-Merger combines the frame changes of shared-base specialists into a single fixed-spectrum model, requiring no post-merge data, rollouts, gradient updates, or on-policy distillation (OPD). It recovers complementary specialist capabilities and achieves the strongest aggregate performance among the compared data-free merging methods. Online, ISO-Optimizer applies a chosen base optimizer, including AdamW and Muon, to the frame variables while keeping the base spectra fixed. Across reasoning and coding tasks ranging from 1.5B to 8B parameters, ISO-Optimizer improves accuracy in the reported runs and reaches matched scores with substantially fewer training steps. On Qwen3-8B-Base, AdamW reaches an aggregate accuracy of 0.495 after 270 training steps. ISO-AdamW reaches the same accuracy after only 100 training steps and improves further to 0.509 after 210 training steps. Together, ISO offers a concrete answer to RLVR's missing optimization layer: rather than inheriting pre-training optimization wholesale, design post-training around the structure of reward-driven adaptation: inherit the spectrum, optimize the frames.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

44 extracted references · 18 linked inside Pith

  1. [1]

    The path not taken: Rlvr provably learns off the principals.arXiv preprint arXiv:2511.08567, 2025

    Hanqing Zhu, Zhenyu Zhang, Hanxian Huang, DiJia Su, Zechun Liu, Jiawei Zhao, Igor Fedorov, Hamed Pirsiavash, Zhizhou Sha, Jinwon Lee, et al. The path not taken: Rlvr provably learns off the principals.arXiv preprint arXiv:2511.08567, 2025

    arXiv 2025

  2. [2]

    Grok: Ai assistant, 2025

    xAI. Grok: Ai assistant, 2025. Accessed: 2025-09-24, continuously updated

    2025

  3. [3]

    The next frontier of data training: Rl environments, February 2026

    Scale AI. The next frontier of data training: Rl environments, February 2026. 21 ISO: An RLVR-Native Optimization Stack

    2026

  4. [4]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Pith/arXiv arXiv 2025

  5. [5]

    Dapo: An open-source llm reinforcement learning system at scale, 2025.URL https://arxiv

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source llm reinforcement learning system at scale, 2025.URL https://arxiv. org/abs/2503.14476, 1:2, 2025

    Pith/arXiv arXiv 2025

  6. [6]

    Reinforce++: Stabilizing critic-free policy optimization with global advantage normalization.arXiv preprint arXiv:2501.03262, 2025

    Jian Hu, Jason Klein Liu, Haotian Xu, and Wei Shen. Reinforce++: Stabilizing critic-free policy optimization with global advantage normalization.arXiv preprint arXiv:2501.03262, 2025

    Pith/arXiv arXiv 2025

  7. [7]

    Maximum likelihood reinforce- ment learning.arXiv preprint arXiv:2602.02710, 2026

    Fahim Tajwar, Guanning Zeng, Yueer Zhou, Yuda Song, Daman Arora, Yiding Jiang, Jeff Schnei- der, Ruslan Salakhutdinov, Haiwen Feng, and Andrea Zanette. Maximum likelihood reinforce- ment learning.arXiv preprint arXiv:2602.02710, 2026

    arXiv 2026

  8. [8]

    slime: An llm post-training framework for rl scaling.https://github.com/THUDM/slime, 2025

    Zilin Zhu, Chengxing Xie, Xin Lv, and slime Contributors. slime: An llm post-training framework for rl scaling.https://github.com/THUDM/slime, 2025. GitHub repository. Corresponding author: Xin Lv

    2025

  9. [9]

    Hybridflow: A flexible and efficient rlhf framework

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. InProceedings of the Twentieth European Conference on Computer Systems, pages 1279–1297, 2025

    2025

  10. [10]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations, 2019

    2019

  11. [11]

    Muon is scalable for llm training.arXiv preprint arXiv:2502.16982, 2025

    Jingyuan Liu, Jianlin Su, Xingcheng Yao, Zhejun Jiang, Guokun Lai, Yulun Du, Yidao Qin, Weixin Xu, Enzhe Lu, Junjie Yan, et al. Muon is scalable for llm training.arXiv preprint arXiv:2502.16982, 2025

    Pith/arXiv arXiv 2025

  12. [12]

    Apollo: Sgd-like memory, adamw-level performance

    Hanqing Zhu, Zhenyu Zhang, Wenyan Cong, Xi Liu, Sem Park, Vikas Chandra, Bo Long, David Z Pan, Zhangyang Wang, and Jinwon Lee. Apollo: Sgd-like memory, adamw-level performance. Proceedings of Machine Learning and Systems, 7, 2025

    2025

  13. [13]

    Fantastic pretraining optimizers and where to find them.arXiv preprint arXiv:2509.02046, 2025

    Kaiyue Wen, David Hall, Tengyu Ma, and Percy Liang. Fantastic pretraining optimizers and where to find them.arXiv preprint arXiv:2509.02046, 2025

    Pith/arXiv arXiv 2025

  14. [14]

    Rl’s razor: Why online reinforcement learning forgets less.arXiv preprint arXiv:2509.04259, 2025

    Idan Shenfeld, Jyothish Pari, and Pulkit Agrawal. Rl’s razor: Why online reinforcement learning forgets less.arXiv preprint arXiv:2509.04259, 2025

    Pith/arXiv arXiv 2025

  15. [15]

    Reinforcement learning finetunes small subnetworks in large language models.arXiv preprint arXiv:2505.11711, 2025

    Sagnik Mukherjee, Lifan Yuan, Dilek Hakkani-Tur, and Hao Peng. Reinforcement learning finetunes small subnetworks in large language models.arXiv preprint arXiv:2505.11711, 2025

    arXiv 2025

  16. [16]

    On-policy distillation.Thinking Machines Lab: Connec- tionism, 2025

    Kevin Lu and Thinking Machines Lab. On-policy distillation.Thinking Machines Lab: Connec- tionism, 2025. https://thinkingmachines.ai/blog/on-policy-distillation

    2025

  17. [17]

    Glm-5: from vibe coding to agentic engineering.arXiv preprint arXiv:2602.15763, 2026

    Aohan Zeng, Xin Lv, Zhenyu Hou, Zhengxiao Du, Qinkai Zheng, Bin Chen, Da Yin, Chendi Ge, Chenghua Huang, Chengxing Xie, et al. Glm-5: from vibe coding to agentic engineering.arXiv preprint arXiv:2602.15763, 2026

    Pith/arXiv arXiv 2026

  18. [18]

    The invisible leash: Why rlvr may or may not escape its origin.arXiv preprint arXiv:2507.14843, 2025

    Fang Wu, Weihao Xuan, Ximing Lu, Mingjie Liu, Yi Dong, Zaid Harchaoui, and Yejin Choi. The invisible leash: Why rlvr may or may not escape its origin.arXiv preprint arXiv:2507.14843, 2025. 22 ISO: An RLVR-Native Optimization Stack

    arXiv 2025

  19. [19]

    Prorl:Prolongedreinforcementlearningexpandsreasoningboundariesinlargelanguagemodels

    Mingjie Liu, Shizhe Diao, Ximing Lu, Jian Hu, Xin Dong, Yejin Choi, Jan Kautz, and Yi Dong. Prorl:Prolongedreinforcementlearningexpandsreasoningboundariesinlargelanguagemodels. arXiv preprint arXiv:2505.24864, 2025

    Pith/arXiv arXiv 2025

  20. [20]

    Brorl: Scaling reinforcement learning via broadened exploration.arXiv preprint arXiv:2510.01180, 2025

    Jian Hu, Mingjie Liu, Ximing Lu, Fang Wu, Zaid Harchaoui, Shizhe Diao, Yejin Choi, Pavlo Molchanov, Jun Yang, Jan Kautz, et al. Brorl: Scaling reinforcement learning via broadened exploration.arXiv preprint arXiv:2510.01180, 2025

    arXiv 2025

  21. [21]

    Embodied-r1: Reinforced embodied reasoning for general robotic manipulation

    Yifu Yuan, Haiqin Cui, Yaoting Huang, Yibin Chen, Fei Ni, Zibin Dong, Pengyi Li, Yan Zheng, and Jianye Hao. Embodied-r1: Reinforced embodied reasoning for general robotic manipulation. arXiv preprint arXiv:2508.13998, 2025

    Pith/arXiv arXiv 2025

  22. [22]

    Dion: Distributed orthonormalized updates.arXiv preprint arXiv:2504.05295, 2025

    Kwangjun Ahn, Byron Xu, Natalie Abreu, Ying Fan, Gagik Magakyan, Pratyusha Sharma, Zheng Zhan, and John Langford. Dion: Distributed orthonormalized updates.arXiv preprint arXiv:2504.05295, 2025

    arXiv 2025

  23. [23]

    Qwen2.5 technical report, 2025

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li,...

    2025

  24. [24]

    Co-evolving llm coder and unit tester via reinforcement learning.arXiv preprint arXiv:2506.03136, 2025

    Yinjie Wang, Ling Yang, Ye Tian, Ke Shen, and Mengdi Wang. Co-evolving llm coder and unit tester via reinforcement learning.arXiv preprint arXiv:2506.03136, 2025

    arXiv 2025

  25. [25]

    Toolrl: Reward is all tool learning needs.arXiv preprint arXiv:2504.13958, 2025

    Cheng Qian, Emre Can Acikgoz, Qi He, Hongru Wang, Xiusi Chen, Dilek Hakkani-Tür, Gokhan Tur, and Heng Ji. Toolrl: Reward is all tool learning needs.arXiv preprint arXiv:2504.13958, 2025

    Pith/arXiv arXiv 2025

  26. [26]

    Memagent: Reshaping long-context llm with multi-conv rl-based memory agent.arXiv preprint arXiv:2507.02259, 2025

    Hongli Yu, Tinghong Chen, Jiangtao Feng, Jiangjie Chen, Weinan Dai, Qiying Yu, Ya-Qin Zhang, Wei-Ying Ma, Jingjing Liu, Mingxuan Wang, et al. Memagent: Reshaping long-context llm with multi-conv rl-based memory agent.arXiv preprint arXiv:2507.02259, 2025

    Pith/arXiv arXiv 2025

  27. [27]

    Editing models with task arithmetic.arXiv preprint arXiv:2212.04089, 2022

    Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic.arXiv preprint arXiv:2212.04089, 2022

    Pith/arXiv arXiv 2022

  28. [28]

    Ties-merging: Resolving interference when merging models.Advances in neural information processing systems, 36:7093–7115, 2023

    Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. Ties-merging: Resolving interference when merging models.Advances in neural information processing systems, 36:7093–7115, 2023

    2023

  29. [29]

    Task singular vectors: Reducing task interference in model merging

    AntonioAndreaGargiulo,DonatoCrisostomi,MariaSofiaBucarelli,SimoneScardapane,Fabrizio Silvestri, and Emanuele Rodola. Task singular vectors: Reducing task interference in model merging. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 18695–18705, 2025

    2025

  30. [30]

    Behavior knowledge merge in reinforced agentic models, 2026

    Xiangchi Yuan, Dachuan Shi, Chunhui Zhang, Zheyuan Liu, Shenglong Yao, Soroush Vosoughi, and Wenke Lee. Behavior knowledge merge in reinforced agentic models, 2026

    2026

  31. [31]

    Orthogonal model merging.arXiv preprint arXiv:2602.05943, 2026

    Sihan Yang, Kexuan Shi, and Weiyang Liu. Orthogonal model merging.arXiv preprint arXiv:2602.05943, 2026. 23 ISO: An RLVR-Native Optimization Stack

    arXiv 2026

  32. [32]

    Aspo: Asymmetric importance sampling policy optimization.arXiv preprint arXiv:2510.06062, 2025

    Jiakang Wang, Runze Liu, Lei Lin, Wenping Hu, Xiu Li, Fuzheng Zhang, Guorui Zhou, and Kun Gai. Aspo: Asymmetric importance sampling policy optimization.arXiv preprint arXiv:2510.06062, 2025

    Pith/arXiv arXiv 2025

  33. [33]

    Justrl: Scaling a 1.5 b llm with a simple rl recipe

    Bingxiang He, Zekai Qu, Zeyuan Liu, Yinghao Chen, Yuxin Zuo, Cheng Qian, Kaiyan Zhang, Weize Chen, Chaojun Xiao, Ganqu Cui, et al. Justrl: Scaling a 1.5 b llm with a simple rl recipe. arXiv preprint arXiv:2512.16649, 2025

    arXiv 2025

  34. [34]

    Qwen3 technical report, 2025

    Qwen Team. Qwen3 technical report, 2025

    2025

  35. [35]

    Deepmath-103k: A large-scale, challenging, decontaminated, and verifiable mathematical dataset for advancing reasoning

    Zhiwei He, Tian Liang, Jiahao Xu, Qiuzhi Liu, Xingyu Chen, Yue Wang, Linfeng Song, Dian Yu, Zhenwen Liang, Wenxuan Wang, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Deepmath-103k: A large-scale, challenging, decontaminated, and verifiable mathematical dataset for advancing reasoning. 2025

    2025

  36. [36]

    Modular manifolds.Thinking Machines Lab: Connectionism, 2025

    Jeremy Bernstein. Modular manifolds.Thinking Machines Lab: Connectionism, 2025. https://thinkingmachines.ai/blog/modular-manifolds/

    2025

  37. [37]

    Reparameterized llm training via orthogonal equivalence transformation.Advances in Neural Information Processing Systems, 38:140775–140821, 2026

    Zeju Qiu, Simon Buchholz, Tim Xiao, Maximilian Dax, Bernhard Schölkopf, and Weiyang Liu. Reparameterized llm training via orthogonal equivalence transformation.Advances in Neural Information Processing Systems, 38:140775–140821, 2026

    2026

  38. [38]

    Poet-x: Memory-efficient llm training by scaling orthogonal transformation.arXiv preprint arXiv:2603.05500, 2026

    Zeju Qiu, Lixin Liu, Adrian Weller, Han Shi, and Weiyang Liu. Poet-x: Memory-efficient llm training by scaling orthogonal transformation.arXiv preprint arXiv:2603.05500, 2026

    Pith/arXiv arXiv 2026

  39. [39]

    Spectral adapter: Fine-tuning in spectral space.arXiv preprint arXiv:2405.13952, 2024

    Fangzhao Zhang and Mert Pilanci. Spectral adapter: Fine-tuning in spectral space.arXiv preprint arXiv:2405.13952, 2024

    Pith/arXiv arXiv 2024

  40. [40]

    Stella: Subspace learning in low-rank adaptation using stiefel manifold.Advances in Neural Information Processing Systems, 38:75066–75092, 2026

    Zhizhong Li, Sina Sajadmanesh, Jingtao Li, and Lingjuan Lyu. Stella: Subspace learning in low-rank adaptation using stiefel manifold.Advances in Neural Information Processing Systems, 38:75066–75092, 2026

    2026

  41. [41]

    Pion: A spectrum-preserving optimizer via orthogonal equivalence transformation.arXiv preprint arXiv:2605.12492, 2026

    Kexuan Shi, Hanxuan Li, Zeju Qiu, Yandong Wen, Simon Buchholz, and Weiyang Liu. Pion: A spectrum-preserving optimizer via orthogonal equivalence transformation.arXiv preprint arXiv:2605.12492, 2026

    Pith/arXiv arXiv 2026

  42. [42]

    Lora without regret.Thinking Machines Lab: Connectionism, 2025

    John Schulman and Thinking Machines Lab. Lora without regret.Thinking Machines Lab: Connectionism, 2025. https://thinkingmachines.ai/blog/lora/

    2025

  43. [43]

    Stabilizing knowledge, promoting reasoning: Dual-token constraints for rlvr.arXiv preprint arXiv:2507.15778, 2025

    Jiakang Wang, Runze Liu, Fuzheng Zhang, Xiu Li, and Guorui Zhou. Stabilizing knowledge, promoting reasoning: Dual-token constraints for rlvr.arXiv preprint arXiv:2507.15778, 2025

    Pith/arXiv arXiv 2025

  44. [44]

    Livecodebench: Holistic and contamination free evaluation of large language models for code.arXiv preprint arXiv:2403.07974, 2024

    Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code.arXiv preprint arXiv:2403.07974, 2024. 24 ISO: An RLVR-Native Optimization Stack Appendix Outline The appendices are organized as fo...

    Pith/arXiv arXiv 2024