Pith. sign in

REVIEW 7 minor 22 references

Under a shared KL and clipping recipe, fan-in Muon lets reinforcement-learning agents take a larger stable optimization step than AdamW, improving late success when optimization headroom remains.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 04:13 UTC pith:Q2IDQ7FX

load-bearing objection A careful, well-hedged empirical mapping of Muon's stable operating regime in agentic RL; the shared-recipe confound is real but explicitly bounded, and the paper deserves a serious referee.

arxiv 2607.16169 v4 pith:Q2IDQ7FX submitted 2026-07-17 cs.LG cs.AI

When Does Muon Help Agentic Reinforcement Learning?

Kai Ruan , Jinghao Lin , Zihe Huang , Ziqi Zhou , Qianshan Wei , Xuan Wang , Hao Sun This is my paper
classification cs.LG cs.AI
keywords MuonAdamWagentic reinforcement learningALFWorldGRPOGiGPOGraphGPOoptimizer scale convention
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper asks when the Muon optimizer helps in reinforcement-learning post-training of language-model agents, a regime where its behavior was previously unclear. Training Qwen2.5 agents from 0.5B to 3B on ALFWorld under three credit-assignment objectives (GRPO, GiGPO, GraphGPO), it compares a fixed fan-in Muon recipe with AdamW under one shared KL and clipping configuration. The central finding is that Muon at a more aggressive learning rate (3e-5) remains stable and improves late success over the AdamW 1e-6 baseline after correction for multiple tests, whereas AdamW at comparable rates becomes unreliable. The advantage is largest when optimization headroom remains and shrinks near saturation, after tuning AdamW, or when Muon's update magnitude is matched to AdamW's. A scale-matched control shows that the effect depends on Muon's fan-in scaling convention rather than on spectral flattening alone.

Core claim

The paper's claim: under its shared KL and clipping recipe, fan-in Muon sustains a larger stable effective step than AdamW in long-horizon agentic RL. In paired 0.5B comparisons across three credit-assignment objectives, Muon at 3e-5 improved late-window success over AdamW at 1e-6 (Holm-corrected p=0.004); AUC gains were directionally positive but less uniform. At 1.5B both Muon rates raised AUC; at 3B GraphGPO, high-rate Muon learned earlier while tuned AdamW nearly closed the AUC gap. High-rate Muon applies 3.53x AdamW's hidden-matrix RMS, yet a full-budget RMS-matched control falls below both standard Muon and AdamW. The authors conclude the gain is recipe-level, tied to Muon's scaling co

What carries the argument

The central object is the Keller–Jordan fan-in Muon update. For each hidden weight matrix, Muon takes the momentum matrix, approximates its polar factor with five Newton–Schulz iterations, and applies the scaled update ΔW = −η / sqrt(max(1, d_out/d_in)) Q, with non-matrix parameters updated by AdamW. This fan-in scaling is what lets Muon apply a larger stable effective step; the paper isolates its role through applied-update RMS measurements (3.53x AdamW for the high-rate recipe) and a full-budget RMS-matched control that eliminates the gain. Spectral diagnostics show the applied update's normalized stable rank is roughly 0.6 for Muon versus 0.015 for AdamW, confirming that the transformatio

Load-bearing premise

The conclusion that AdamW lacks a stable aggressive step depends on the shared KL and clipping recipe never being tuned for AdamW at high learning rates; a jointly tuned AdamW schedule might recover stability and erase the observed margin.

What would settle it

Sweep AdamW learning rates from 1e-6 to 3e-5 on ALFWorld while jointly lowering the KL coefficient and adjusting clipping; if any AdamW configuration at a comparable applied-update RMS matches or exceeds high-rate Muon's late success and remains stable over 200 updates, the central claim that fan-in Muon uniquely supports a larger stable step under this recipe fails. Conversely, run the full-budget RMS-matched Muon control at several global magnitudes: if a magnitude above 1e-6 recovers the late-success gain, the effect is global scale rather than the fan-in convention.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • High-rate fan-in Muon is a practical candidate when validation improves without instability; the paper recommends recipe-level rate screening over a universal Muon-to-AdamW conversion rule.
  • The benefit shows up as improved late success when headroom remains and as earlier progress when the task is near saturation.
  • Tuned AdamW can recover most of the difference at 3B GraphGPO, so the result does not imply Muon is universally better.
  • RMS-matching Muon to AdamW's update magnitude removes the late-success gain, making the scaling convention a material part of the effective optimizer recipe.
  • The regime persists across all three evaluated credit-assignment objectives and transfers from ALFWorld to WebShop.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the fan-in scaling convention is the load-bearing ingredient, other optimizers that adopt a similar per-matrix magnitude rule might reproduce part of the gain; the paper does not test this directly.
  • The headroom interpretation gives a testable prediction: on harder or longer-horizon tasks with ample room for improvement, the Muon margin should widen, and on near-solved tasks it should vanish.
  • The spectral–credit compatibility idea suggests a practical monitor: track the reliability of weak singular directions during training; the Muon advantage should correlate with cases where those directions carry signal.
  • Because the shared recipe was never tuned for AdamW at high rates, an AdamW-specific KL/clipping schedule could shift the boundary; the paper acknowledges this open possibility.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

0 major / 7 minor

Summary. The paper empirically maps the operating regime of the Muon optimizer in long-horizon agentic reinforcement learning. On ALFWorld, with Qwen2.5 models at 0.5B, 1.5B, and 3B, and under GRPO, GiGPO, and GraphGPO, the authors compare a fixed AdamW 1e-6 baseline against fan-in Muon at 1e-5 and 3e-5, sweep AdamW learning rates, run scale extensions, perform applied-update spectral diagnostics, and evaluate a full-budget RMS-matched Muon control. The central claim is that, under the shared KL/clipping recipe, aggressive fan-in Muon (3e-5) is stable where larger AdamW rates are not, and improves paired late success (Holm-corrected padj=0.004), while the lower Muon rate is inconsistent, tuned AdamW nearly closes the gap at 3B GraphGPO, and the RMS-matched control does not preserve the gain. The paper frames the result as a recipe-level operating regime rather than a universal optimizer ranking.

Significance. If the result holds, it gives practitioners a concrete, testable recipe-level guideline: fan-in Muon can sustain a more aggressive effective step under this regularization configuration, and the benefit is largest when optimization headroom remains. The study is methodologically careful in several important ways: five-seed paired comparisons, Holm correction across the rate–metric family, exact estimator-stratified permutation sensitivity, explicit update-RMS measurements, a full-budget magnitude-matched control, and released code. The authors also hedge consistently: the main claim is explicitly conditional on the shared KL/clipping recipe, and the RMS-matched control is described as identifying the scaling convention rather than spectral shape in isolation. These strengths make the paper a useful empirical contribution even though it does not establish a general optimizer ranking.

minor comments (7)
  1. [Paired Effects Across Objectives; Table 8] The Discussion states that the operating regime 'persists across all three evaluated estimators,' but Table 8 shows that the GRPO high-rate late-success 95% CI is [-0.034, 0.254], i.e., it includes zero. The pooled sign test is significant, but estimator-stratified evidence for GRPO is directionally positive yet not individually significant. Please qualify the persistence claim and report which estimator(s) drive the pooled result.
  2. [Abstract; Appendix B, Table 6] The full-budget RMS-matched control appears to be a single matched run with no seed-level replication. The abstract states that this control 'removes the late-success gain,' which is stronger than what a single run can support. Please either add replication or rephrase to 'is not preserved in the single matched full-budget control' and note this explicitly where the control is introduced.
  3. [Learning-Rate Controls; Discussion] The AdamW high-rate collapse is measured under a fixed KL coefficient (0.01) and fixed clipping, with no joint rate–regularization tuning for AdamW. The Discussion appropriately concedes that 'joint rate–regularization tuning may shift this boundary.' Because the title asks a fairly general question, please make this limitation more prominent in the abstract or conclusion so that the conditional nature of the headline claim is not missed.
  4. [Background, Eq. (2)] The fan-in scaling factor is written ambiguously as 's max(1,dout/din)'. Please use the explicit notation sqrt(max(1,dout/din)) consistently with Appendix B, where the factor is described as multiplying after the Newton–Schulz step.
  5. [Appendix C.2, Table 9] The main text says the lower-rate Muon recipe 'is not consistently signed,' while Table 9 shows a significant permutation-based mean difference for Muon 1e-5. Please clarify that the sign test evaluates paired directions, whereas the permutation test pools magnitudes after discarding pairing; otherwise the two statements appear contradictory.
  6. [Figure 2 caption] The caption says 'Each point is Muon minus its matched AdamW baseline' but does not state that the points are per-seed paired differences. Please specify the seed-level pairing and the number of points per estimator.
  7. [Appendix B, Table 5] For the RMS-matched convention, please clarify what 'base rate' means (i.e., the scalar rate before applying the 0.2*sqrt(max) multiplier) so that the reported ratios can be reproduced.

Circularity Check

0 steps flagged

No material circularity: the central claim rests on direct matched experiments; the only self-citation (HGPO) is non-load-bearing.

full rationale

The paper's central claim—that fan-in Muon supports a larger stable effective step under the shared KL/clipping recipe—is supported by matched empirical comparisons, not by a derivation that reduces to its inputs. The Muon update (Eq. 2) and learning-rate conversion (Eq. 3) are cited prior formulas used only to select rates, and the paper explicitly states that the 3e-5 setting 'probes a more aggressive fan-in recipe and is not presented as rate-equivalent to AdamW 10^-6,' so the high-rate result is not forced by the conversion. The RMS-matched control is an independent check against the scale-matching hypothesis, and the measured 3.53x update-RMS ratio is a diagnostic, not a fitted parameter. The Appendix D spectral-credit hypothesis is explicitly a compact hypothesis, not a premise used to derive the empirical effect. The only self-citation is the HGPO reference (He et al. 2026), which likely shares author Q. Wei; it appears in Related Work as a non-evaluated related method and is not load-bearing for the main results. The paper also candidly flags the main limitation—'joint rate–regularization tuning may shift this boundary'—which makes the claim conditional, but the condition is an untested fairness boundary rather than circular reasoning.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 1 invented entities

The main empirical claim rests on seven hand-set learning-rate/regularization constants and on domain assumptions about environment representativeness and fairness of the shared recipe. No new physical entities are load-bearing; the Appendix D quantity is explanatory only.

free parameters (7)
  • Muon fan-in learning rate (aggressive) = 3e-5
    Chosen by hand as an aggressive probe relative to the 10x heuristic; central to the claimed regime.
  • Muon fan-in learning rate (heuristic) = 1e-5
    Chosen via the practical 10x conversion from AdamW 1e-6; effect inconsistent.
  • AdamW baseline learning rate = 1e-6
    Reference baseline; all paired claims are relative to this rate. Other rates are explored in sweeps.
  • AdamW tuned learning rate (3B GraphGPO) = 3e-6
    Post hoc tuned control; closes the AUC gap, showing baseline choice matters.
  • RMS-matched Muon learning rate = 1e-6
    Chosen so hidden-matrix update RMS matches AdamW; this control removes the late-success gain.
  • Shared KL coefficient = 0.01
    Held fixed across all comparisons; the regime conclusion is conditional on this recipe.
  • RMS conversion constant c = approx 0.2
    Taken from Liu et al. (2025) to connect fan-in and RMS-matched Muon rates; not re-derived here.
axioms (5)
  • domain assumption Five bfloat16 Newton-Schulz iterations faithfully approximate the polar factor of the momentum matrix.
    The spectral claims depend on the NS approximation behaving like ideal orthogonalization; no convergence check is reported.
  • domain assumption ALFWorld/WebShop with Qwen2.5-Instruct models are representative of long-horizon agentic RL post-training.
    Generalization beyond these environments/models is asserted through a single transfer case with no multi-environment replication.
  • domain assumption The shared KL/clipping/weight-decay recipe is a fair common bed for both optimizers.
    If AdamW required different regularization at higher rates, the stable-aggressive-step asymmetry could be an artifact; the paper acknowledges this limitation.
  • domain assumption Validation checkpoints over the fixed 200-update budget capture final quality and progress.
    Late success and normalized AUC are defined on this schedule; longer horizons are outside the evaluation.
  • domain assumption Paired sign-test and permutation inference on matched comparisons is valid despite heterogeneous estimator strata.
    Stratified p-values treat estimator cells as exchangeable; no formal mixed-effects model is used.
invented entities (1)
  • Tail-reliability ratio R_tail and fixed-basis Gaussian noise model (Appendix D) no independent evidence
    purpose: Explanatory hypothesis for when spectral flattening helps; not used to derive the main empirical claim.
    Introduced as a compact diagnostic hypothesis with no falsifiable prediction beyond re-explaining the observed regime.

pith-pipeline@v1.3.0-alltime-deepseek · 16819 in / 13956 out tokens · 149213 ms · 2026-08-04T04:13:00.846936+00:00 · methodology

0 comments
read the original abstract

Muon is competitive with AdamW in large-scale pre-training, but its operating regime in reinforcement-learning post-training remains unclear. We map this regime on ALFWorld, a sparse-reward agentic benchmark, using three group-based objectives and Qwen2.5 models from 0.5B to 3B. Under a shared KL and clipping recipe, matched optimizer comparisons and AdamW rate controls trace the usable step-size range. AdamW responds non-monotonically to rate, whereas fan-in Muon remains stable at a more aggressive effective step: at $3 \times 10^{-5}$ it improves late success over an AdamW $10^{-6}$ baseline after correction across rate-metric tests. Its normalized-AUC effect is directionally positive but less uniform; the heuristic-matched lower-rate effect is less consistent, and tuned AdamW nearly matches high-rate Muon at 3B GraphGPO. High-rate Muon applies $3.53 \times$ AdamW's hidden-matrix update RMS; a full-budget RMS-matched control removes the late-success gain. Together, these results identify a recipe-level operating regime in which fan-in Muon supports a more aggressive stable effective step under shared KL and clipping: the margin is largest when optimization headroom remains and contracts near saturation, after AdamW tuning, or under magnitude matching. The scale-matched control ties this spectral effect to Muon's scale convention rather than establishing a universal optimizer ranking. Code is available at https://github.com/x66ccff/verl-muon.

Figures

Figures reproduced from arXiv: 2607.16169 by Hao Sun, Jinghao Lin, Kai Ruan, Qianshan Wei, Xuan Wang, Zihe Huang, Ziqi Zhou.

Figure 1
Figure 1. Figure 1: Optimizer and rate evidence. Rows report late success and normalized AUC. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Paired optimizer effects at 0.5B. Each point is Muon minus its matched AdamW [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Matched GiGPO optimizer-by-credit ablation [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Scale extensions under configuration-matched optimizer comparisons. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Spectral diagnostic for a matched 1.5B GRPO com [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Training dynamics for the main GRPO AdamW and Muon ( [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Per-task late-window validation success in the three main comparisons. Gray circles denote AdamW; blue squares and [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Optimizer diagnostics under GiGPO (top) and GraphGPO (bottom). Gradient norms remain on the same scale across [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Validation success for the matched 1.5B GraphGPO comparison. Both Muon rates learn faster than AdamW; the higher [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Per-task validation trajectories for the 1.5B GiGPO comparison. All configurations approach high success, while [PITH_FULL_IMAGE:figures/full_fig_p015_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Per-task validation trajectories for the 3B GiGPO comparison. Muon at [PITH_FULL_IMAGE:figures/full_fig_p015_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Training diagnostics for the 1.5B GiGPO comparison. The faster Muon validation trajectories occur without a distinct [PITH_FULL_IMAGE:figures/full_fig_p016_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: WebShop partial-task-score trajectories for the matched 0.5B GiGPO comparison. Curves are unsmoothed evaluations [PITH_FULL_IMAGE:figures/full_fig_p016_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: GiGPO high-rate diagnostics. Validation curves show raw five-step checkpoints; the other panels show five-step [PITH_FULL_IMAGE:figures/full_fig_p018_14.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

22 extracted references · 17 linked inside Pith

  1. [1]

    InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics

    Ahmadian, A.; Cremer, C.; Gallé, M.; Fadaee, M.; Kreutzer, J.; Pietquin,O.;Üstün,A.;andHooker,S.2024.BacktoBasics:Revis- iting REINFORCE-Style Optimization for Learning from Human Feedback in LLMs. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. Chen,L.;Li,J.;andLiu,Q.2025. MuonOptimizesUnderSpectral Norm Constra...

    arXiv 2024

  2. [3]

    ArXiv:2602.22817

    Hierarchy-of-GroupsPolicyOptimizationforLong-HorizonAgen- ticTasks.InInternationalConferenceonLearningRepresentations. ArXiv:2602.22817. Huang, F.; Luo, Y.; and Chen, S

    arXiv

  3. [4]

    Jordan, K.; Jin, Y.; Boza, V.; You, J.; Cesista, F.; Newhouse, L.; and Bernstein, J

    MiMuon: Mixed Muon Optimizer with Improved Generalization for Large Models.arXiv preprint arXiv:2605.19619. Jordan, K.; Jin, Y.; Boza, V.; You, J.; Cesista, F.; Newhouse, L.; and Bernstein, J

    Pith/arXiv arXiv

  4. [5]

    Kimi Team

    SOAP, Muon, and Beyond: Pushing LLM Pretraining Scales.arXiv preprint arXiv:2607.20548. Kimi Team

    Pith/arXiv arXiv

  5. [7]

    Lion, K.; Hübler, F.; Li, B.; Orvieto, A.; and He, N

    Nor- Muon: Making Muon More Efficient and Scalable.arXiv preprint arXiv:2510.05491. Lion, K.; Hübler, F.; Li, B.; Orvieto, A.; and He, N

    arXiv

  6. [8]

    Liu,J.;Su,J.;Yao,X.;Jiang,Z.;Lai,G.;Du,Y.;etal.2025

    Muown:Row-NormControlforMuonOptimization.arXivpreprint arXiv:2605.10797. Liu,J.;Su,J.;Yao,X.;Jiang,Z.;Lai,G.;Du,Y.;etal.2025. Muon is Scalable for LLM Training.arXiv preprint arXiv:2502.16982. Liu, Y.; Wang, J.; and Zhang, T

    Pith/arXiv arXiv 2025

  7. [9]

    Liu,Z.;Zhang,R.;Wang,Z.;Zhao,Y.;Su,Y.;Yang,Z.;andZhang, Z

    Optimizer-Model Con- sistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less.arXiv preprint arXiv:2605.06654. Liu,Z.;Zhang,R.;Wang,Z.;Zhao,Y.;Su,Y.;Yang,Z.;andZhang, Z

    Pith/arXiv arXiv

  8. [10]

    Lu, Z.; Yao, Z.; Han, Z.; Wang, Z.-H.; Wu, J.; Gu, Q.; Cai, X.; Lu, W.;Xiao,J.;Zhuang,Y.;andShen,Y.2026

    Muon2: Boosting Muon via Adaptive Second-Moment Preconditioning.arXiv preprint arXiv:2604.09967. Lu, Z.; Yao, Z.; Han, Z.; Wang, Z.-H.; Wu, J.; Gu, Q.; Cai, X.; Lu, W.;Xiao,J.;Zhuang,Y.;andShen,Y.2026. Self-DistilledAgentic Reinforcement Learning.arXiv preprint arXiv:2605.15155. Ma, J.; Huang, Y.; Chi, Y.; and Chen, Y

    Pith/arXiv arXiv 2026

  9. [11]

    Meng, Z.; and Chen, K

    Preconditioning Benefits of Spectral Orthogonalization in Muon.arXiv preprint arXiv:2601.13474. Meng, Z.; and Chen, K

    arXiv

  10. [12]

    CRAFT: Counterfactual Credit Assignment from Free Sibling Rollouts for Self-Distilled Agentic Reinforcement Learning.arXiv preprint arXiv:2606.29476. NVIDIA

    Pith/arXiv arXiv

  11. [13]

    Peng, J.; Liu, Y.; Zhou, R.; Fleming, C.; Wang, Z.; Garcia, A.; andHong,M.2026

    HTMuon: Improving Muon via Heavy-Tailed Spectral Correction.arXiv preprint arXiv:2603.10067. Peng, J.; Liu, Y.; Zhou, R.; Fleming, C.; Wang, Z.; Garcia, A.; andHong,M.2026. HiPER:HierarchicalReinforcementLearning withExplicitCreditAssignmentforLargeLanguageModelAgents. arXiv preprint arXiv:2602.16165. Prime Intellect Team; Senghaas, M.; Obeid, F.; et al

    Pith/arXiv arXiv 2026

  12. [14]

    Qu, X.; Huang, P.; and Horvath, S

    INTELLECT-3: Technical Report.arXiv preprint arXiv:2512.16144. Qu, X.; Huang, P.; and Horvath, S

    arXiv

  13. [15]

    Shao, Z.; Wang, P.; Zhu, Q.; Xu, R.; Song, J.; Bi, X.; Zhang, H.; Zhang, M.; Li, Y

    Can Muon Fine-tune Adam-Pretrained Models?arXiv preprint arXiv:2605.10468. Shao, Z.; Wang, P.; Zhu, Q.; Xu, R.; Song, J.; Bi, X.; Zhang, H.; Zhang, M.; Li, Y. K.; Wu, Y.; and Guo, D

    Pith/arXiv arXiv

  14. [17]

    Hollows Out

    EnvRL: Learn from Environment Dynamics in Agentic Reinforcement Learning.arXiv preprint arXiv:2606.17680. Wei, J. 2026a. Field Notes: Why Muon “Hollows Out” in RL (and What We Plan To DO Next). https://huggingface.co/blog/bird-of- paradise/training-rl-with-muon-2. Wei, J. 2026b. Hopper: The Optimizer That Learns Parallelism 2x Faster Than Adam. https://hu...

    Pith/arXiv arXiv

  15. [18]

    arXiv preprint arXiv:2606.25852

    Semantic Consistency Policy Optimization for Reinforcement Learning of LLM Agents. arXiv preprint arXiv:2606.25852. Xue, X.; Zhou, Y.; Wang, Z.; Tang, S.; Torr, P.; Ouyang, W.; Bai, L.; and Yin, Z

    Pith/arXiv arXiv

  16. [19]

    Yang, A.; Yang, B.; Zhang, B.; Hui, B.; Zheng, B.; Yu, B.; et al

    StraTA: Incentivizing Agentic Reinforce- mentLearningwithStrategicTrajectoryAbstraction.arXivpreprint arXiv:2605.06642. Yang, A.; Yang, B.; Zhang, B.; Hui, B.; Zheng, B.; Yu, B.; et al

    Pith/arXiv arXiv

  17. [20]

    Yao, S.; Chen, H.; Yang, J.; and Narasimhan, K

    Qwen2.5 Technical Report.arXiv preprint arXiv:2412.15115. Yao, S.; Chen, H.; Yang, J.; and Narasimhan, K

    Pith/arXiv arXiv

  18. [21]

    Zhang,M.;Liu,Y.;andSchaeffer,H.2026

    DAPO: An Open-Source LLM Reinforce- ment Learning System at Scale.arXiv preprint arXiv:2503.14476. Zhang,M.;Liu,Y.;andSchaeffer,H.2026. AdamImprovesMuon: Adaptive Moment Estimation with Orthogonalized Momentum. arXiv preprint arXiv:2602.17080. Zhuang, X.; Ouyang, P.; Li, Y.; Shi, J.; Chen, Y.; Liu, S.; Qian, Y.; Liu, W.; Zhang, H.; and Razzak, I

    Pith/arXiv arXiv 2026

  19. [22]

    AMO: Adaptive Muon Orthogonalization.arXiv preprint arXiv:2605.17806. A. Experimental Hyperparameters Table 3 summarizes the shared setup and the estimator-specific settings used in the reported comparisons. Parameter Value Shared setup Models Qwen2.5-Instruct (0.5B, 1.5B, 3B) Environments ALFWorld; WebShop Training updates / evaluation interval 200 / 5 M...

    Pith/arXiv arXiv 2048

  20. [2024]

    Sheng,G.;Zhang,C.;Ye,Z.;Wu,X.;Zhang,W.;Zhang,R.;Peng, Y.;Lin,H.;andWu,C.2024

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models.arXiv preprint arXiv:2402.03300. Sheng,G.;Zhang,C.;Ye,Z.;Wu,X.;Zhang,W.;Zhang,R.;Peng, Y.;Lin,H.;andWu,C.2024. HybridFlow:AFlexibleandEfficient RLHF Framework.arXiv preprint arXiv:2409.19256. Shridhar, M.; Yuan, X.; Côté, M.-A.; Bisk, Y.; Trischler, A.; and Hausknecht, M

    Pith/arXiv arXiv 2024

  21. [2025]

    Kool,W.;vanHoof,H.;andWelling,M.2019

    Kimi K2: Open Agentic Intelligence.arXiv preprint arXiv:2507.20534. Kool,W.;vanHoof,H.;andWelling,M.2019. Buy4REINFORCE Samples, Get a Baseline for Free! InICLR Workshop on Deep RL Meets Structured Prediction. Kwon, W.; Li, Z.; Zhuang, S.; Sheng, Y.; Zheng, L.; Yu, C. H.; Gonzalez, J. E.; Zhang, H.; and Stoica, I

    Pith/arXiv arXiv 2019

  22. [2026]

    Re- thinkingMuonBeyondPretraining:SpectralFailuresandHigh-Pass Remedies for VLA and RLVR.arXiv preprint arXiv:2605.19282

    Fan,C.;Liu,G.;Hong,M.;Kompella,R.R.;andLiu,S.2026. Re- thinkingMuonBeyondPretraining:SpectralFailuresandHigh-Pass Remedies for VLA and RLVR.arXiv preprint arXiv:2605.19282. Feng,L.;Xue,Z.;Liu,T.;andAn,B.2025. Group-in-GroupPolicy Optimization for LLM Agent Training. InAdvances in Neural Information Processing Systems. ArXiv:2505.10978. Gupta, A.; Celente,...

    Pith/arXiv arXiv 2026