Pith. sign in

REVIEW 3 major objections 6 minor 22 references

Under a shared KL and clipping recipe, high-rate fan-in Muon supports a stable effective step that AdamW rate increases do not, and at 0.5B it improves late success over the AdamW baseline after multiple-comparison correction.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 21:07 UTC pith:Q2IDQ7FX

load-bearing objection A genuinely careful empirical paper: the 0.5B high-rate Muon effect is real and well-controlled; the mechanistic claim about update magnitude is weaker than the recipe-level claim, and the scale extensions need seed reporting. the 3 major comments →

arxiv 2607.16169 v3 pith:Q2IDQ7FX submitted 2026-07-17 cs.LG cs.AI

When Does Muon Help Agentic Reinforcement Learning?

Kai Ruan , Jinghao Lin , Zihe Huang , Ziqi Zhou , Qianshan Wei , Xuan Wang , Hao Sun This is my paper
classification cs.LG cs.AI
keywords Muon optimizerAdamWagentic reinforcement learningsparse-reward RLspectral orthogonalizationlearning-rate stabilityupdate-scale matchingALFWorld
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish when the Muon optimizer helps reinforcement-learning post-training for long-horizon, sparse-reward agentic tasks — not whether it universally beats AdamW. Holding the KL and clipping recipe fixed across three credit-assignment objectives and models from 0.5B to 3B, it finds that the aggressive fan-in Muon setting (learning rate 3e-5) sustains a larger stable update than AdamW can tolerate: at 0.5B it improves late-window success over the AdamW 1e-6 baseline after multiple-comparison correction, while AdamW rate increases collapse. The advantage is clearest when optimization headroom remains; near saturation, after AdamW tuning, or under update-magnitude matching, the margin shrinks to earlier learning or disappears. Diagnostics show Muon applies 3.53 times AdamW's hidden-matrix update RMS, and a full-budget RMS-matched control removes the gain, so the paper identifies the scale convention as part of the effective recipe rather than isolating spectral shape. If right, practitioners should screen Muon learning rates against headroom instead of relying on a universal conversion factor.

Core claim

The paper's central discovery is a recipe-level operating regime: with shared KL and clipping, fan-in Muon's high-rate recipe remains stable where AdamW rates above 1e-6 become unreliable, and in paired 0.5B comparisons it improves late success over the AdamW 1e-6 baseline across GRPO, GiGPO, and GraphGPO, with the direction surviving correction. The lower Muon rate is not consistently beneficial; tuned AdamW nearly closes the AUC gap at 3B GraphGPO. Applied-update measurements show Muon's hidden-matrix update RMS is 3.53 times AdamW's, and a full-budget RMS-matched control loses the standard high-rate recipe's advantage. The authors read this as evidence that the larger applied step is subs

What carries the argument

Muon is the central object: for each hidden weight matrix, it replaces the singular values of the momentum matrix with equal magnitudes through a truncated orthogonalization iteration, then scales the update by a fan-in convention (an inverse-square-root-of-input-width factor). This flattens the spectrum of the applied update while increasing its RMS relative to AdamW, measured at 3.53x in the matched probe. The machinery does two jobs: it makes a larger effective step stable under the shared KL and clipping recipe, and it changes layerwise scale allocation. The RMS-matched control — which brings Muon's global update magnitude down to AdamW-like levels — removes the late-success gain, so the

Load-bearing premise

The paper's conclusion that Muon's benefit comes from a larger applied update rather than its spectral flattening depends on the assumption that the RMS-matched control affected only the overall update size — but that control also changed how scale was distributed across layers, and that layerwise difference could be the real cause.

What would settle it

On the same sparse-reward task and recipe, train with a version of Muon whose total hidden-matrix update RMS is reduced to AdamW's by a single global rescaling of the fan-in recipe, preserving each layer's relative scale. If late success stays at the level of the aggressive fan-in recipe (or even above AdamW), the magnitude interpretation fails; if it drops to the RMS-matched control's level, the larger-step claim is supported.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • At 0.5B, replacing the fixed AdamW 1e-6 recipe with fan-in Muon at 3e-5 improves late-window success across all three evaluated objectives, with pooled direction tests surviving multiple-comparison correction.
  • Muon extends the usable learning-rate range: AdamW at 5e-6 and above loses nearly all post-update success under the shared recipe, while Muon at 3e-5 remains stable.
  • When final success saturates, the benefit shows up as earlier learning rather than higher final success; at 3B GraphGPO, tuned AdamW at 3e-6 nearly matches high-rate Muon's AUC.
  • The scale convention is part of the effect: an RMS-matched Muon that reduces hidden-matrix update to 0.80x AdamW's loses the late-success gain in the full-budget GiGPO control, so spectral flattening alone is not sufficient at matched magnitude.
  • The same 0.5B GiGPO recipe transfers to WebShop, where high-rate Muon raises exact late success from 0.008 to 0.689 and partial-score AUC from 0.078 to 0.548.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the magnitude interpretation: rescale the fan-in Muon update by a single global factor so total hidden-matrix RMS matches AdamW while preserving the fan-in layerwise ratios; if the late-success gain survives, the paper's magnitude conclusion would need revision in favor of layerwise scaling.
  • If the larger stable step is the active ingredient, then joint tuning of KL and clipping should shift the stable rate boundary; sweeping the KL coefficient and clip threshold at high-rate Muon on one estimator would map where the advantage disappears.
  • The paper's spectral-credit compatibility view predicts Muon helps most when flattened weak directions carry reliable credit information; injecting controlled noise into advantages under GRPO, GiGPO, or GraphGPO and measuring the Muon-minus-AdamW gap at increasing noise would test this prediction.
  • A connected full-pipeline test: after AdamW supervised fine-tuning, run high-rate Muon RL and measure both RL success and forgetting of pretrained knowledge, to see whether the aggressive step trades away prior capabilities in longer-horizon settings.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper investigates whether Muon improves agentic RL post-training for Qwen2.5 models on ALFWorld across three group-based objectives (GRPO, GiGPO, GraphGPO), with a WebShop transfer case. Under a shared KL/clipping recipe and fixed optimizer rates chosen a priori, it compares fan-in Muon at 1e-5 and 3e-5 against an AdamW 1e-6 baseline, and separately sweeps AdamW rates. The main 0.5B result is a Holm-corrected paired late-success advantage for high-rate Muon (p_adj=0.004), with a directionally positive but less uniform normalized-AUC effect. Scale runs at 1.5B/3B are used to argue that Muon's advantage appears as earlier progress when late success saturates. A full-budget RMS-matched Muon control removes the late-success gain, and spectral diagnostics show a flatter applied-update spectrum for Muon. The paper concludes with a recipe-level operating regime rather than a universal optimizer ranking.

Significance. The empirical design has real strengths: matched five-seed comparisons, a priori rate choices, AdamW rate sweeps, Holm correction, exhaustive permutation sensitivity, a credit-control ablation, and an appendix hypothesis explicitly marked as not evidence. If the 0.5B late-success effect is taken as the central claim, it is credible and carefully supported. The paper also usefully separates recipe-level findings from mechanistic attribution and repeatedly acknowledges the layerwise confound in the RMS-matched control. The main additional value is the map of when Muon helps (headroom, early progress, saturation), which is a practically relevant contribution even though it does not establish a universal optimizer ranking.

major comments (3)
  1. [Appendix B, Table 6; Update-Spectrum Diagnostics] The contribution bullet states that the RMS-matched control 'identif[ies] the larger applied step as a material component of the observed gain.' This is not supported by the reported control because the RMS-matched convention (0.2*sqrt(max(dout,din))) simultaneously changes global update RMS and layerwise scale allocation. The paper acknowledges this confound for spectral shape, but the magnitude attribution still requires a control that changes global RMS while preserving fan-in layerwise allocation (or vice versa). Otherwise the correct conclusion is narrower: the scaling rule as a whole is part of the effective recipe, and the larger step alone is not isolated. I recommend rephrasing or adding the missing control.
  2. [Scale Extensions (Table 2, Table 10, Appendix C.6)] The 1.5B, 3B, and WebShop results appear to be single-seed or have unspecified replication. These are used in the abstract and main text to claim persistence across scale and environment ('map this regime ... from 0.5B to 3B'; WebShop 'provides a second-environment transfer case'). The Discussion correctly labels them extension cases, but the headline claims go beyond what a single trajectory can support. Please report seed counts for every table; if these are single runs, present them explicitly as exploratory and soften the persistence language, or add repeated seeds for at least the 3B boundary case and WebShop.
  3. [Discussion / Table 8] The statement that the operating regime 'persists across all three evaluated estimators' is stronger than the estimator-specific evidence. In Table 8, the GRPO high-rate late-success 95% CI is [-0.034, 0.254] and the low-rate GiGPO CI is [-0.186, 0.239], so only GiGPO and GraphGPO high-rate are individually significant. The pooled sign test supports a directional effect, but I recommend saying 'directionally positive across estimators, with stronger evidence in GiGPO and GraphGPO' unless a formal interaction or pooled test is presented as the basis for the across-estimator claim.
minor comments (6)
  1. [Eq. (3)] The RMS conversion η_RMS ≈ η_KJ/(c√din) with c≈0.2 is stated without derivation or error analysis. Please note whether c is fitted, theoretical, or from Liu et al., and state the range of din over which the approximation holds.
  2. [Figure 5] The top panel label 'Raw gradient / momentum rank' is ambiguous. The text describes the pre-transformation first moment; please use 'pre-transform first-moment' in the axis label for consistency.
  3. [Appendix D, Eq. (6)] The expected-alignment formula for an ideal polar update would benefit from stating the normalization of G^ and P(G^), and from a one-line derivation of the 2Φ(s_i/σ_i)-1 term. As written it is a plausible diagnostic but not self-contained.
  4. [Experimental Setup / Metrics] The choice of late success as the 'primary endpoint' is made after seeing the AUC results are less uniform. A pre-registered primary endpoint or a clear statement of the decision rule would strengthen the multiple-testing story; otherwise the Holm correction over the four rate-metric tests may understate the selection done on endpoints.
  5. [WebShop, Table 12] The WebShop success numbers are dramatically better for Muon (final 0.689 vs 0.023). Since this is a single seed and a large effect, please include task-level or trajectory-level evidence that this is not an initialization or rollout artifact.
  6. [General] The paper would benefit from a reproducibility statement with code and checkpoint release plans, and from stating exactly how many seeds each table uses (including the rate sweeps and the RMS-matched control).

Circularity Check

0 steps flagged

No significant circularity: the central comparison is a fixed, a priori optimizer recipe evaluation whose conclusions summarize matched experiments rather than reducing to their inputs.

full rationale

The paper's derivation chain is empirical, not definitional. The main 0.5B result compares fixed optimizer recipes with Muon rates 1e-5/3e-5 and AdamW 1e-6 chosen before outcomes; the paired sign tests, Holm correction, and estimator-stratified permutation test are applied to matched five-seed comparisons, with no parameter fitted to the outcome and then reported as a prediction. The conversion in Eq. 3 is a design heuristic for choosing rates, not a fitted claim about results. The only potentially mechanism-asserting step is the full-budget RMS-matched control in Appendix B, which the paper itself qualifies: it 'does not isolate spectral shape' and notes that 'the fan-in and RMS-matched conventions also induce different layerwise scales.' The statement that the larger step is 'substantive rather than a nominal-rate artifact' is therefore underdetermined by a confound, but underdetermination is not circularity. The spectral diagnostic (C.7) is an independent measurement of applied-update rank, and Appendix D is explicitly 'not used as evidence for the optimizer ranking.' The only self-citation overlap is the HGPO reference (He et al. 2026, containing co-author Q. Wei), which is cited as related work and is not load-bearing for any claimed result. No step in the paper equates a fitted quantity with a prediction or imports a uniqueness result from the authors' own prior work. The conclusion summarizes the empirical comparisons and does not reduce to an input definition.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

No new entities are introduced. The paper's diagnostic quantities (normalized stable rank, update RMS) are measurements of existing optimizer behavior. Free parameters are the fixed recipe hyperparameters, which are chosen a priori and not fitted to the outcomes; the mechanism claim additionally depends on the RMS-conversion constant c≈0.2 from prior literature.

free parameters (4)
  • Muon learning rates = 1e-5 and 3e-5
    Fixed a priori to probe a heuristic-matched and an aggressive fan-in setting; the claim is recipe-specific, so these settings bound the regime.
  • AdamW baseline learning rate = 1e-6
    Chosen as the fixed reference; the rate sweep shows 3e-6 is competitive but variable, so the baseline is a conservative default rather than a tuned best.
  • RMS-match conversion constant c = 0.2
    Taken from Liu et al. (2025) and used in Eq. (3) and in the RMS-matched control; the mechanism claim depends on this conversion.
  • KL coefficient and clipping settings = 0.01 (low-var KL)
    Held fixed across all comparisons; the conclusion explicitly applies only under this shared regularization, so the recipe's settings define the claimed regime.
axioms (5)
  • domain assumption Newton-Schulz with 5 iterations in bfloat16 gives an adequate polar-factor approximation Q≈UV^T for the fan-in Muon update (Eq. 2).
    The entire Muon update mechanism relies on this numerical approximation being accurate enough; the paper uses the Keller-Jordan implementation without evaluating approximation error.
  • domain assumption GRPO, GiGPO, and GraphGPO provide distinct and representative credit-assignment conditions for long-horizon agentic RL.
    The paper generalizes its findings from three estimators; if these estimators do not span the space of credit-assignment methods, the 'persists across estimators' claim is weaker.
  • domain assumption The learning-rate conversion η_RMS ≈ η_KJ/(c√d_in) with c≈0.2 (Eq. 3) and the practical '10x heuristic' are accurate enough for the tested matrix shapes.
    Used to relate Muon conventions and to choose the lower Muon rate; if the conversion is off, the heuristic-matched comparison is misaligned.
  • domain assumption The shared KL and clipping recipe used for all comparisons is a reasonable default under which to evaluate optimizer regimes.
    The paper's conclusion is conditional on this recipe; a different regularization setting could shift the boundary.
  • ad hoc to paper Scale and WebShop results are representative despite unreported or single-seed replication.
    The paper reports single aggregate values for 1.5B/3B and WebShop without stating the number of seeds; the claim that the regime persists across scale and environment rests on this unstated assumption.

pith-pipeline@v1.3.0-alltime-deepseek · 16880 in / 12204 out tokens · 99935 ms · 2026-08-01T21:07:47.550906+00:00 · methodology

0 comments
read the original abstract

Muon is competitive with AdamW in large-scale pre-training, but its operating regime in reinforcement-learning post-training remains unclear. We map this regime on ALFWorld, a sparse-reward agentic benchmark, using three group-based objectives and Qwen2.5 models from 0.5B to 3B. Under a shared KL and clipping recipe, matched optimizer comparisons and AdamW learning-rate controls trace the usable step-size range. AdamW responds non-monotonically to its learning rate, whereas fan-in Muon remains stable at a more aggressive effective step. At a learning rate of 3 x 10^-5, it improves late success over an AdamW 10^-6 baseline after correction across rate-metric tests. Its normalized-AUC effect is directionally positive but less uniform; the heuristic-matched lower-rate effect is less consistent, and tuned AdamW nearly matches high-rate Muon on 3B GraphGPO. High-rate Muon applies 3.53 times AdamW's hidden-matrix update RMS, while a full-budget RMS-matched control removes the late-success gain. Together, these results identify a recipe-level operating regime in which fan-in Muon supports a more aggressive stable effective step under shared KL and clipping. The margin is largest when optimization headroom remains and contracts near saturation, after AdamW tuning, or under magnitude matching. The scale-matched control ties this spectral effect to Muon's scaling convention rather than establishing a universal optimizer ranking.

Figures

Figures reproduced from arXiv: 2607.16169 by Hao Sun, Jinghao Lin, Kai Ruan, Qianshan Wei, Xuan Wang, Zihe Huang, Ziqi Zhou.

Figure 1
Figure 1. Figure 1: Optimizer and rate evidence. Rows report late success and normalized AUC. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Paired optimizer effects at 0.5B. Each point is Muon minus its matched AdamW [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Matched GiGPO optimizer-by-credit ablation [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Scale extensions under configuration-matched optimizer comparisons. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Spectral diagnostic for a matched 1.5B GRPO com [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Training dynamics for the main GRPO AdamW and Muon ( [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Per-task late-window validation success in the three main comparisons. Gray circles denote AdamW; blue squares and [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Optimizer diagnostics under GiGPO (top) and GraphGPO (bottom). Gradient norms remain on the same scale across [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Validation success for the matched 1.5B GraphGPO comparison. Both Muon rates learn faster than AdamW; the higher [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Per-task validation trajectories for the 1.5B GiGPO comparison. All configurations approach high success, while [PITH_FULL_IMAGE:figures/full_fig_p015_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Per-task validation trajectories for the 3B GiGPO comparison. Muon at [PITH_FULL_IMAGE:figures/full_fig_p015_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Training diagnostics for the 1.5B GiGPO comparison. The faster Muon validation trajectories occur without a distinct [PITH_FULL_IMAGE:figures/full_fig_p016_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: WebShop partial-task-score trajectories for the matched 0.5B GiGPO comparison. Curves are unsmoothed evaluations [PITH_FULL_IMAGE:figures/full_fig_p016_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: GiGPO high-rate diagnostics. Validation curves show raw five-step checkpoints; the other panels show five-step [PITH_FULL_IMAGE:figures/full_fig_p018_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

22 extracted references · 17 linked inside Pith

  1. [1]

    InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics

    Ahmadian, A.; Cremer, C.; Gallé, M.; Fadaee, M.; Kreutzer, J.; Pietquin,O.;Üstün,A.;andHooker,S.2024.BacktoBasics:Revis- iting REINFORCE-Style Optimization for Learning from Human Feedback in LLMs. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. Chen,L.;Li,J.;andLiu,Q.2025. MuonOptimizesUnderSpectral Norm Constra...

    arXiv 2024

  2. [3]

    ArXiv:2602.22817

    Hierarchy-of-GroupsPolicyOptimizationforLong-HorizonAgen- ticTasks.InInternationalConferenceonLearningRepresentations. ArXiv:2602.22817. Huang, F.; Luo, Y.; and Chen, S

    arXiv

  3. [4]

    Jordan, K.; Jin, Y.; Boza, V.; You, J.; Cesista, F.; Newhouse, L.; and Bernstein, J

    MiMuon: Mixed Muon Optimizer with Improved Generalization for Large Models.arXiv preprint arXiv:2605.19619. Jordan, K.; Jin, Y.; Boza, V.; You, J.; Cesista, F.; Newhouse, L.; and Bernstein, J

    Pith/arXiv arXiv

  4. [5]

    Kimi Team

    SOAP, Muon, and Beyond: Pushing LLM Pretraining Scales.arXiv preprint arXiv:2607.20548. Kimi Team

    Pith/arXiv arXiv

  5. [7]

    Lion, K.; Hübler, F.; Li, B.; Orvieto, A.; and He, N

    Nor- Muon: Making Muon More Efficient and Scalable.arXiv preprint arXiv:2510.05491. Lion, K.; Hübler, F.; Li, B.; Orvieto, A.; and He, N

    arXiv

  6. [8]

    Liu,J.;Su,J.;Yao,X.;Jiang,Z.;Lai,G.;Du,Y.;etal.2025

    Muown:Row-NormControlforMuonOptimization.arXivpreprint arXiv:2605.10797. Liu,J.;Su,J.;Yao,X.;Jiang,Z.;Lai,G.;Du,Y.;etal.2025. Muon is Scalable for LLM Training.arXiv preprint arXiv:2502.16982. Liu, Y.; Wang, J.; and Zhang, T

    Pith/arXiv arXiv 2025

  7. [9]

    Liu,Z.;Zhang,R.;Wang,Z.;Zhao,Y.;Su,Y.;Yang,Z.;andZhang, Z

    Optimizer-Model Con- sistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less.arXiv preprint arXiv:2605.06654. Liu,Z.;Zhang,R.;Wang,Z.;Zhao,Y.;Su,Y.;Yang,Z.;andZhang, Z

    Pith/arXiv arXiv

  8. [10]

    Lu, Z.; Yao, Z.; Han, Z.; Wang, Z.-H.; Wu, J.; Gu, Q.; Cai, X.; Lu, W.;Xiao,J.;Zhuang,Y.;andShen,Y.2026

    Muon2: Boosting Muon via Adaptive Second-Moment Preconditioning.arXiv preprint arXiv:2604.09967. Lu, Z.; Yao, Z.; Han, Z.; Wang, Z.-H.; Wu, J.; Gu, Q.; Cai, X.; Lu, W.;Xiao,J.;Zhuang,Y.;andShen,Y.2026. Self-DistilledAgentic Reinforcement Learning.arXiv preprint arXiv:2605.15155. Ma, J.; Huang, Y.; Chi, Y.; and Chen, Y

    Pith/arXiv arXiv 2026

  9. [11]

    Meng, Z.; and Chen, K

    Preconditioning Benefits of Spectral Orthogonalization in Muon.arXiv preprint arXiv:2601.13474. Meng, Z.; and Chen, K

    arXiv

  10. [12]

    CRAFT: Counterfactual Credit Assignment from Free Sibling Rollouts for Self-Distilled Agentic Reinforcement Learning.arXiv preprint arXiv:2606.29476. NVIDIA

    Pith/arXiv arXiv

  11. [13]

    Peng, J.; Liu, Y.; Zhou, R.; Fleming, C.; Wang, Z.; Garcia, A.; andHong,M.2026

    HTMuon: Improving Muon via Heavy-Tailed Spectral Correction.arXiv preprint arXiv:2603.10067. Peng, J.; Liu, Y.; Zhou, R.; Fleming, C.; Wang, Z.; Garcia, A.; andHong,M.2026. HiPER:HierarchicalReinforcementLearning withExplicitCreditAssignmentforLargeLanguageModelAgents. arXiv preprint arXiv:2602.16165. Prime Intellect Team; Senghaas, M.; Obeid, F.; et al

    Pith/arXiv arXiv 2026

  12. [14]

    Qu, X.; Huang, P.; and Horvath, S

    INTELLECT-3: Technical Report.arXiv preprint arXiv:2512.16144. Qu, X.; Huang, P.; and Horvath, S

    arXiv

  13. [15]

    Shao, Z.; Wang, P.; Zhu, Q.; Xu, R.; Song, J.; Bi, X.; Zhang, H.; Zhang, M.; Li, Y

    Can Muon Fine-tune Adam-Pretrained Models?arXiv preprint arXiv:2605.10468. Shao, Z.; Wang, P.; Zhu, Q.; Xu, R.; Song, J.; Bi, X.; Zhang, H.; Zhang, M.; Li, Y. K.; Wu, Y.; and Guo, D

    Pith/arXiv arXiv

  14. [17]

    Hollows Out

    EnvRL: Learn from Environment Dynamics in Agentic Reinforcement Learning.arXiv preprint arXiv:2606.17680. Wei, J. 2026a. Field Notes: Why Muon “Hollows Out” in RL (and What We Plan To DO Next). https://huggingface.co/blog/bird-of- paradise/training-rl-with-muon-2. Wei, J. 2026b. Hopper: The Optimizer That Learns Parallelism 2x Faster Than Adam. https://hu...

    Pith/arXiv arXiv

  15. [18]

    arXiv preprint arXiv:2606.25852

    Semantic Consistency Policy Optimization for Reinforcement Learning of LLM Agents. arXiv preprint arXiv:2606.25852. Xue, X.; Zhou, Y.; Wang, Z.; Tang, S.; Torr, P.; Ouyang, W.; Bai, L.; and Yin, Z

    Pith/arXiv arXiv

  16. [19]

    Yang, A.; Yang, B.; Zhang, B.; Hui, B.; Zheng, B.; Yu, B.; et al

    StraTA: Incentivizing Agentic Reinforce- mentLearningwithStrategicTrajectoryAbstraction.arXivpreprint arXiv:2605.06642. Yang, A.; Yang, B.; Zhang, B.; Hui, B.; Zheng, B.; Yu, B.; et al

    Pith/arXiv arXiv

  17. [20]

    Yao, S.; Chen, H.; Yang, J.; and Narasimhan, K

    Qwen2.5 Technical Report.arXiv preprint arXiv:2412.15115. Yao, S.; Chen, H.; Yang, J.; and Narasimhan, K

    Pith/arXiv arXiv

  18. [21]

    Zhang,M.;Liu,Y.;andSchaeffer,H.2026

    DAPO: An Open-Source LLM Reinforce- ment Learning System at Scale.arXiv preprint arXiv:2503.14476. Zhang,M.;Liu,Y.;andSchaeffer,H.2026. AdamImprovesMuon: Adaptive Moment Estimation with Orthogonalized Momentum. arXiv preprint arXiv:2602.17080. Zhuang, X.; Ouyang, P.; Li, Y.; Shi, J.; Chen, Y.; Liu, S.; Qian, Y.; Liu, W.; Zhang, H.; and Razzak, I

    Pith/arXiv arXiv 2026

  19. [22]

    AMO: Adaptive Muon Orthogonalization.arXiv preprint arXiv:2605.17806. A. Experimental Hyperparameters Table 3 summarizes the shared setup and the estimator-specific settings used in the reported comparisons. Parameter Value Shared setup Model / environment Qwen2.5-0.5B-Instruct (main); Qwen2.5-1.5B-Instruct and Qwen2.5-3B-Instruct (scale) / ALFWorld (prim...

    Pith/arXiv arXiv 2048

  20. [2024]

    Sheng,G.;Zhang,C.;Ye,Z.;Wu,X.;Zhang,W.;Zhang,R.;Peng, Y.;Lin,H.;andWu,C.2024

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models.arXiv preprint arXiv:2402.03300. Sheng,G.;Zhang,C.;Ye,Z.;Wu,X.;Zhang,W.;Zhang,R.;Peng, Y.;Lin,H.;andWu,C.2024. HybridFlow:AFlexibleandEfficient RLHF Framework.arXiv preprint arXiv:2409.19256. Shridhar, M.; Yuan, X.; Côté, M.-A.; Bisk, Y.; Trischler, A.; and Hausknecht, M

    Pith/arXiv arXiv 2024

  21. [2025]

    Kool,W.;vanHoof,H.;andWelling,M.2019

    Kimi K2: Open Agentic Intelligence.arXiv preprint arXiv:2507.20534. Kool,W.;vanHoof,H.;andWelling,M.2019. Buy4REINFORCE Samples, Get a Baseline for Free! InICLR Workshop on Deep RL Meets Structured Prediction. Kwon, W.; Li, Z.; Zhuang, S.; Sheng, Y.; Zheng, L.; Yu, C. H.; Gonzalez, J. E.; Zhang, H.; and Stoica, I

    Pith/arXiv arXiv 2019

  22. [2026]

    Re- thinkingMuonBeyondPretraining:SpectralFailuresandHigh-Pass Remedies for VLA and RLVR.arXiv preprint arXiv:2605.19282

    Fan,C.;Liu,G.;Hong,M.;Kompella,R.R.;andLiu,S.2026. Re- thinkingMuonBeyondPretraining:SpectralFailuresandHigh-Pass Remedies for VLA and RLVR.arXiv preprint arXiv:2605.19282. Feng,L.;Xue,Z.;Liu,T.;andAn,B.2025. Group-in-GroupPolicy Optimization for LLM Agent Training. InAdvances in Neural Information Processing Systems. ArXiv:2505.10978. Gupta, A.; Celente,...

    Pith/arXiv arXiv 2026