Pith. sign in

REVIEW 5 major objections 6 minor 15 references

ACRL stabilizes LLM reinforcement learning under FP8 inference by adaptively scaling each token's gradient with a ratio tied to the current training–inference gap, matching the full-precision baseline's accuracy.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-07-31 23:06 UTC pith:YPOGPC3P

load-bearing objection The adaptive exponent is a real twist on token-level IS and the experiments are broad, but the Y<X branch is misdescribed and the evidence is single-run; worth a referee, with major revision. the 5 major comments →

arxiv 2607.24062 v1 pith:YPOGPC3P submitted 2026-07-27 cs.LG cs.AIcs.CL

ACRL: Adaptive Control of Training-Inference Discrepancy for Stable Reinforcement Learning

Wenwu Fan , Qihong Lin , Zhijie Xia , Zhuo Zheng , Sihao Wang , Qiang Chen , Liangsheng Zhu This is my paper
classification cs.LG cs.AIcs.CL
keywords training-inference discrepancyreinforcement learninglarge language modelsFP8 quantizationadaptive controlimportance samplingpolicy entropyoff-policy stabilization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Reinforcement learning for LLMs usually assumes the policy that generates rollouts and the policy being trained are the same. In practice, low-precision inference (FP8) makes the inference policy differ from the training policy, and this mismatch turns the update into off-policy learning that can collapse. ACRL's central idea is to control that mismatch instead of trying to correct it away: measure the per-token distance between the two policies, average it over each generated sequence, compare it to a before-training reference, and scale each token's gradient contribution by a ratio that depends on that comparison. When the gap is too large the rule pulls the training policy toward the inference policy; when too small it pushes them apart, keeping the gap in a stable band. The reported payoff is that FP8 rollouts become stable, policy entropy rises, and accuracy matches—sometimes exceeds—the high-precision BF16 baseline.

Core claim

The core claim is that the training–inference discrepancy is not a nuisance to be eliminated but a controlled variable. The ACRL update is the standard GRPO (Group Relative Policy Optimization) objective with a token-level importance ratio ρ=(π/µ)^α inserted before the clipped policy-gradient term, where α=γ·sign(A)(1−Y/X). Here Y is the average absolute distance between training and inference probabilities over the current generated sequence, X is the same average measured at step 0, and the sign of the advantage A decides whether the token update is amplified or damped. When Y exceeds X the rule shrinks the discrepancy; when Y is below X it deliberately enlarges it, on the ground that an o

What carries the argument

The carrying mechanism is the adaptive exponent α=γ·sign(A)(1−Y/X) inside the token-level ratio ρ=(π/µ)^α, with the per-token distance d=|π−µ| supplying the measurement and Y its sequence-level average. The reference X, fixed at the initial step, defines the target band. The sign of (1−Y/X) selects between two four-quadrant rules: in the Y>X regime the update moves the training policy toward the inference policy to reduce the gap; in the Y<X regime it moves the training policy away to restore it. The magnitude of (1−Y/X) scales the correction strength, so the controller is both directional and proportional. This converts a scalar, sequence-level measurement into per-token gradient scaling, a

Load-bearing premise

The load-bearing premise is that the measured per-token distance between the training and inference policies is an actionable control signal—that scaling gradient contributions by (π/µ)^α, with α derived from the current sequence gap relative to a fixed reference, actually steers the gap toward that reference in the direction claimed. The paper states this as a principle and explicitly leaves the derivation of the Y<X branch to the reader.

What would settle it

Run ACRL with the control loop broken—random sign for α, or α forced to zero—while keeping the same ratio magnitudes, and check whether the discrepancy still stays bounded. A sharper test is per-step: record Y alongside X; the control claim predicts that when Y>X the next-step discrepancy decreases on average, and when Y<X it increases. If Y does not track X in these directions, the feedback mechanism, not the ratio values, is what keeps training stable.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Inference can stay in FP8 (or MXFP8) during RL training without paying an accuracy tax: ACRL matches or beats the BF16 baseline across 3B, 7B, 32B dense, and mixture-of-experts models.
  • The same adaptive-ratio principle transfers to other RL update rules (PPO and DAPO) and to a general-knowledge benchmark, so the control idea does not depend on one algorithm or task family.
  • Stability becomes a tunable trade-off controlled by γ: medium control strength trains stably, while too-large γ over-constrains the policy and collapses training.
  • The bidirectional Y<X branch is load-bearing: ablations that only reduce the discrepancy, or fall back to plain updates when Y<X, lose accuracy and stability compared with full ACRL.
  • Because the controller also raises policy entropy, stabilization and exploration come from the same mechanism.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the paper defines the discrepancy abstractly (precision, backend kernels, nondeterminism), the same control law could in principle regulate mismatch from asynchronous rollout engines or kernel drift, not just quantization—a testable extension the authors do not run.
  • The static reference X is the most fragile part: the paper tracks it staying between 0.011 and 0.017 over 900+ steps, but on much longer runs a moving-average or data-dependent X would be a natural modification, and its effect could be measured directly.
  • The exploration claim has a gradient-level prediction: tokens with positive advantage and low probability, and negative advantage and high probability, should receive systematically larger effective weights under ACRL; inspecting the signed gradient contributions per token would test the mechanism.
  • The biased-versus-unbiased trade-off is explicit in the paper; an unbiased estimator that never vanishes (e.g., normalized weights) would isolate whether the gains come from bias or from the control signal itself.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes Adaptive Control Reinforcement Learning (ACRL), a method for stabilizing LLM RL training when inference uses low-precision (FP8/MXFP8) quantization. ACRL scales the GRPO gradient with token-level ratios ρ=(π/μ)^α, where α=γ·sign(A)(1−Y/X), with Y the current sequence-level discrepancy and X a fixed reference computed at step 0. The claimed effect is to keep the training-inference discrepancy in a bounded range, prevent collapse under aggressive quantization, increase entropy/exploration, and match or exceed the accuracy of a BF16 baseline while outperforming token-level and sequence-level importance sampling. Experiments cover Qwen2.5-3B/7B, Qwen3-32B and Qwen3-30B-A3B under GRPO, PPO, and DAPO, on math benchmarks and MMLU-Pro.

Significance. If the mechanism and results hold, the paper makes a practically useful contribution: it suggests that training-inference mismatch under low-precision inference can be treated as a controllable quantity rather than an unavoidable collapse risk, potentially removing the 'quantization tax' in LLM RL training. The empirical scope is broad—multiple model scales, dense and MoE architectures, three RL algorithms, and both math and general-knowledge benchmarks—and the paper is unusually explicit about its limitations, including the lack of independent-seed variance and the incomplete derivation of the Y<X branch. Those strengths make the central idea worth pursuing. However, the core control-theoretic mechanism is not actually derived, and the evaluation of the discrepancy is partially circular because Y is both the control input and the primary metric for the stabilization claim. The contribution is therefore promising but not yet established at the level claimed.

major comments (5)
  1. [§3.2, Eq. (11)–(12), Figure 5] The Y<X branch is misdescribed. For Quadrant I (A>0, π<μ), Y<X gives α>0 and π/μ<1, hence ρ=(π/μ)^α<1. The text calls this 'reward reduction'; in the uncorrected update, the positive advantage would push π upward toward μ, which decreases |π−μ|. ACRL therefore only dampens that decrease; it does not 'actively restore' or increase the discrepancy. The same holds in Quadrant III. The paper states 'We leave its detailed derivation to the reader,' but the omitted derivation is exactly where the claimed increase of discrepancy would need to be shown. As written, the mechanism for the Y<X branch is not supported by the equations, and the ablation in Table 10 that attributes gains to this branch lacks a stated mechanism.
  2. [§4.1–§4.2, Eq. (8)–(12)] The control law operates on d(a_{i,t}, θ_old)=|π(θ_old)−μ(θ_old)|, while the quantity to be controlled at the next step is d(a_{i,t}, θ_new)=|π(θ_new)−μ(θ_new)|, where μ(θ_new) is the newly quantized inference policy. The four-quadrant analysis in §4.2 treats μ as fixed while π is updated; it does not provide a plant model connecting the measured discrepancy to the next-step discrepancy. Without such a model, the figures showing Y staying near X are empirical observations, not a demonstration of the claimed feedback mechanism. Please add a derivation or an explicit statement of the assumed plant dynamics, and ideally a diagnostic plot of d(θ_new, μ(θ_new)) against the control signal.
  3. [§4.1, Eq. (10)–(12); Figures 9–10] The core stabilization metric Y is also the control input: α is defined through Y/X, and the same Y is plotted as evidence that ACRL 'maintains the discrepancy within a reasonable range.' This is partly circular—the controller is fed Y, so Y staying near X is not an independent validation. The external results (accuracy, entropy, reward) partially mitigate this, but the discrepancy-control claim itself needs a less confounded evaluation, such as reporting Y on held-out prompts or comparing with an open-loop variant that uses a fixed α schedule.
  4. [§5.6, Tables 1–10] All headline accuracy numbers come from a single run. The paper acknowledges this ('temporal averaging does not replace independent-seed variance'), but the central empirical claim depends on small differences: GSM8K +0.38 pp over BF16, average 7B math +1.28 pp at γ=0.65, and several values within 0.1–0.5 pp. Without multiple seeds and error bars, these differences are within plausible run-to-run noise, so the conclusion that ACRL 'matches or exceeds BF16' is not yet established. At minimum, report the number of seeds, standard deviations, or a claim phrased as single-run evidence.
  5. [§3.1, Eqs. (5)–(7)] The argument that an excessively small discrepancy harms accuracy is built on a hard constraint E≤E0 that restricts the feasible parameter set and can exclude θ*. ACRL does not impose such a hard constraint; it softly scales gradients via ρ. Therefore the mathematical motivation for the Y<X branch does not transfer directly to the algorithm. The paper should either show that the soft control is equivalent to a constraint in some limit, or provide direct evidence (beyond the Table 10 ablation) that driving Y below X degrades accuracy.
minor comments (6)
  1. [Table 3 caption] Typo: 'Mathmatical' should be 'Mathematical'.
  2. [Eq. (9)] The reference X is defined over a 'baseline dataset or initial training step'; later it is consistently computed at step 0. Please clarify whether the same dataset and batch size are used as in training, and report the standard deviation of X across different initial batches.
  3. [Figures 3 and 5] The quadrant figures are hard to parse from the captions alone. Please add axis labels, mark the direction of π movement, and explicitly show which quadrant corresponds to which row in §4.2.
  4. [Table 9] The 'Completion' column uses 'Succ'/'Fail' without definition. State whether failure means training did not finish, collapse occurred, or another stopping rule was applied.
  5. [§5.4, Computational Overhead] The 0.1% overhead figure measures only the isolated function call from cached log-probabilities. It would be useful to also report end-to-end wall-time per training step with and without ACRL, since the metric computation may involve additional I/O or synchronization.
  6. [Remark 4.2] The biasedness of the ACRL estimator is acknowledged. For fairness, please state explicitly in the experiments whether the reported 'TIS' baseline uses the same clipping and whether its ratio is truncated before or after multiplication, since this affects the comparison.

Circularity Check

0 steps flagged

No significant circularity: ACRL's update rule is an explicit feedback law and the stabilization claim is backed by external accuracy/entropy benchmarks; the omitted Y<X derivation is a proof gap, not a circular step.

full rationale

ACRL's derivation chain is not circular. The control law in Eqs. (10)-(12) is an explicit feedback rule: α = γ·sign(A)(1−Y/X) takes the measured sequence-level discrepancy Y as input, and the claim that ACRL keeps Y near X is an empirical closed-loop result (Figures 9-11), not a consequence of the definition, because no plant model is assumed that guarantees the gradient scaling ρ=(π/μ)^α will steer the next-step discrepancy. Accuracy (Tables 1, 3-6), entropy (Figures 8, 10b), and the one-sided ablations (Table 10) are external criteria, so the central stabilization claim is not evaluated solely by its own control signal. The paper contains no load-bearing self-citations: the cited prior work (Yao et al., Liu et al., LMSYS, Qi et al.) is external and is not invoked as an authority to rule out alternatives. Remark 4.2 explicitly calls the adaptive exponent 'an empirical engineering solution,' i.e., an acknowledged ansatz, not a result derived from the definition. The main flagged gap is §3.2, where the Y<X 'actively increasing discrepancy' branch is asserted with 'We leave its detailed derivation to the reader'; the skeptic's objection that the ρ<1 updates in Quadrants I/III only attenuate, rather than reverse, the natural movement of π toward μ is a proof/mechanism concern, not circularity, because the paper's success claims are not equivalent to the equations by construction. The scope limitations in §5.6 and §6 further restrict the claims rather than assume them. Hence score 0.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

No new physical entities are introduced. The central unproven premises are the actionability of the |π−μ| control signal, the validity of a static reference X, and the stochastic-dominance regularity used to claim exploration benefits. The main tuned free parameters are γ, X, and the IS clip C.

free parameters (3)
  • gamma (control strength) = 0.35 and 0.65 in main runs; ablated 0.1-2.5
    Main control gain; hand-chosen per experiment. Sensitivity analysis shows accuracy and stability depend strongly on it (Table 9), and on 7B only γ=0.65 beats BF16.
  • reference discrepancy X = 0.013 (3B), 0.010 (7B), 0.016 (32B), 0.014 (MoE), 0.031 (stress test)
    Measured from step-0 data rather than optimized, but it is a setpoint that must be supplied. Table 8 shows different X values change accuracy and entropy, so it is a de facto tunable parameter.
  • clipping constant C = 3 (5 in stress test)
    Inherited from truncated importance sampling; not justified by theory and changed between experiments.
axioms (5)
  • domain assumption π/µ ratios are well-defined; the inference policy µ assigns nonzero probability wherever training policy π does.
    Eqs. 10-11 divide by µ; under aggressive quantization or truncated sampling some tokens may have µ=0 in practice, and no support correction is given.
  • ad hoc to paper A small discrepancy regime is harmful: constraining E≤E0 excludes optimal θ* (Eqs. 6-7) and overfitting to µ hurts generalization.
    Used to justify the Y<X branch that actively increases discrepancy; no proof that ACRL's soft α scaling implements this constraint or that the excluded-optimum scenario actually occurs.
  • domain assumption Stochastic dominance inequalities (13)-(14) hold throughout training and imply π>µ tokens are relative high-probability.
    Derived from one empirical density plot (Fig. 6) of a single FP8 run, then treated as a mathematical confirmation for the exploration argument in §4.3.
  • domain assumption The step-0 reference X remains a valid control target for the whole run.
    Supported only by Table 7 over ~900 steps of one BF16 run; long-horizon drift is acknowledged as a limitation in §6.
  • domain assumption The four-quadrant update-magnitude logic (Figs. 3/5 and §4.2) actually moves the sequence-level discrepancy Y toward X.
    This is the core control premise; it is asserted rather than proven, and the Y<X branch is explicitly left underived.

pith-pipeline@v1.3.0-alltime-deepseek · 15772 in / 14070 out tokens · 121949 ms · 2026-07-31T23:06:41.257984+00:00 · methodology

0 comments
read the original abstract

Reinforcement Learning (RL) training for Large Language Models (LLMs) often suffers from instability due to the discrepancy between training and inference. This training-inference discrepancy stems from two primary factors: an architectural separation between training and inference engines, and the use of low-precision quantization in inference versus higher-precision computation in training. To address training instability issues caused by high training-inference discrepancy, we present the principles and methods for its adaptive control. We propose Adaptive Control Reinforcement Learning (ACRL), which adaptively maintains the training-inference discrepancy within a reasonable range to ensure stable RL training. Beyond stabilization, ACRL inherently increases policy entropy, thereby enhancing exploration and improving accuracy. The experimental results show that when the inference engine utilizes FP8 quantization, ACRL consistently maintains the training-inference discrepancy within a reasonable range and stabilizes RL training. Furthermore, ACRL not only matches the accuracy of the BF16 baseline but also outperforms importance sampling (IS) fixes.

Figures

Figures reproduced from arXiv: 2607.24062 by Liangsheng Zhu, Qiang Chen, Qihong Lin, Sihao Wang, Wenwu Fan, Zhijie Xia, Zhuo Zheng.

Figure 1
Figure 1. Figure 1: Overview of the ACRL framework. (A) The standard RL training loop suffers from instability and potential collapse due to an excessive discrepancy between the high-precision training policy (π) and the low-precision, quantized inference policy (µ). (B) The proposed Adaptive Control Reinforcement Learning (ACRL) framework introduces a controller that adaptively maintains this training-inference discrepancy w… view at source ↗
Figure 2
Figure 2. Figure 2: Training Probability Adjustment for Decreasing Training￾Inference Discrepancy [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: AGPA for Decreasing Training-Inference Discrepancy probability exceeds the inference probability, the training probability should be reduced toward the inference probabil￾ity. Conversely, the training probability should be increased toward the inference probability. In both cases, the training probability moves toward the inference probability to bridge the training-inference discrepancy. In the training p… view at source ↗
Figure 4
Figure 4. Figure 4: Training Probability Adjustment for Increasing Training￾Inference Discrepancy [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: AGPA for Increasing Training-Inference Discrepancy from becoming excessively large while preserving the up￾date direction(reward/punishment) with respect to the ad￾vantage sign. To prevent the training-inference discrepancy from be￾ing excessively small, we apply the complementary, re￾versed principle illustrated in [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The Probability Density of Training Policy(π) [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: AGPA for Steering Exploration 1. Quadrant I: α > 0 and π(ai,t|q,ai,<t,θold) µ(ai,t|q,ai,<t,θold) < 1. Thus, ρi,t < 1 means reward reduction. 2. Quadrant II: α > 0 and π(ai,t|q,ai,<t,θold) µ(ai,t|q,ai,<t,θold) > 1. Thus, ρi,t > 1 means reward enhancement. 3. Quadrant III: α < 0 and π(ai,t|q,ai,<t,θold) µ(ai,t|q,ai,<t,θold) > 1. Thus, ρi,t < 1 means punishment reduction. 4. Quadrant IV: α < 0 and π(ai,t|q,ai… view at source ↗
Figure 8
Figure 8. Figure 8: Training entropy for Qwen2.5-3B on GSM8K [PITH_FULL_IMAGE:figures/full_fig_p007_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: The Training-Inference Discrepancy for Qwen2.5-3B on GSM8K 0 50 100 150 Training Step 0.00 0.01 0.02 0.03 0.04 0.05 0.06 Training-Inference Discrepancy BF16 TIS MIS ACRL(γ=0.35) ACRL(γ=0.65) (a) Discrepancy 0 50 100 150 Training Step 0 2 4 6 8 Entropy BF16 TIS MIS ACRL(γ=0.35) ACRL(γ=0.65) (b) Entropy 0 50 100 150 Training Step 0.0 0.5 1.0 1.5 2.0 Grad Norm BF16 TIS MIS ACRL(γ=0.35) ACRL(γ=0.65) (c) Gradie… view at source ↗
Figure 10
Figure 10. Figure 10: Training-inference discrepancy, entropy, gradient norm, and reward of Qwen2.5-7B on Difficult Mathmatical Datasets [PITH_FULL_IMAGE:figures/full_fig_p009_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Training-inference discrepancy, entropy, and accuracy of Qwen2.5-3B on GSM8K (Truncated FP8) [PITH_FULL_IMAGE:figures/full_fig_p011_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

15 extracted references · 10 linked inside Pith

  1. [4]

    20250910

    doi: 10.64434/tml. 20250910. https://thinkingmachines.ai/blog/defeating- nondeterminism-in-llm-inference/. Kamath, A. et al. Gemma 3 technical report,

    doi:10.64434/tml

  2. [5]

    Kuzmin, A., Van Baalen, M., Ren, Y ., Nagel, M., Peters, J., and Blankevoort, T

    URL https://arxiv.org/abs/2503.19786. Kuzmin, A., Van Baalen, M., Ren, Y ., Nagel, M., Peters, J., and Blankevoort, T. FP8 quantization: The power of the exponent.Advances in Neural Information Processing Systems, 35:14651–14662,

    Pith/arXiv arXiv

  3. [6]

    Accessed: 2025-12-

    URL https://lmsys.org/ blog/2025-11-25-fp8-rl/ . Accessed: 2025-12-

    2025

  4. [8]

    S., and Lin, M

    Qi, P., Liu, Z., Zhou, X., Pang, T., Du, C., Lee, W. S., and Lin, M. Defeating the training-inference mismatch via FP16.arXiv preprint arXiv:2510.26788,

    arXiv

  5. [9]

    Proximal policy optimization algorithms

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347,

    Pith/arXiv arXiv

  6. [11]

    Klear-reasoner: Advanc- ing reasoning capability via gradient-preserving clipping policy optimization.arXiv preprint arXiv:2508.07629, 2025a

    Su, Z., Pan, L., Bai, X., Liu, D., Dong, G., Huang, J., Hu, W., Zhang, F., Gai, K., and Zhou, G. Klear-reasoner: Advanc- ing reasoning capability via gradient-preserving clipping policy optimization.arXiv preprint arXiv:2508.07629, 2025a. Su, Z., Pan, L., Lv, M., Li, Y ., Hu, W., Zhang, F., Gai, K., and Zhou, G. CE-GPPO: Coordinating entropy via gradient-...

    arXiv

  7. [12]

    Yao, F., Liu, L., Zhang, D., Dong, C., Shang, J., and Gao, J

    URL https: //arxiv.org/abs/2505.09388. Yao, F., Liu, L., Zhang, D., Dong, C., Shang, J., and Gao, J. Your efficient RL framework secretly brings you off-policy RL training, August

    Pith/arXiv arXiv

  8. [13]

    Yu, Q., Zhang, Z., Zhu, R., Yuan, Y ., Zuo, X., Yue, Y ., Dai, W., Fan, T., Liu, G., Liu, L., et al

    URL https: //fengyao.notion.site/off-policy-rl. Yu, Q., Zhang, Z., Zhu, R., Yuan, Y ., Zuo, X., Yue, Y ., Dai, W., Fan, T., Liu, G., Liu, L., et al. Dapo: An open-source LLM reinforcement learning system at scale. arXiv preprint arXiv:2503.14476,

    Pith/arXiv arXiv

  9. [14]

    Group sequence policy optimization.arXiv preprint arXiv:2507.18071,

    Zheng, C., Liu, S., Li, M., Chen, X.-H., Yu, B., Gao, C., Dang, K., Liu, Y ., Men, R., Yang, A., et al. Group sequence policy optimization.arXiv preprint arXiv:2507.18071,

    Pith/arXiv arXiv

  10. [15]

    Hyper-parameters used for experiments training

    13 ACRL: Adaptive Control of Training-Inference Discrepancy for Stable Reinforcement Learning A. Hyper-parameters used for experiments training. Table 11.Hyper-parameters used for Qwen2.5-3B Instruct on GSM8K data.train batch size 64 data.max response length 2048 actor rollout ref.actor.optim.lr 5e-7 actor rollout ref.actor.entropy coeff 0.0 actor rollout...

    arXiv 2048

  11. [26]

    Recipes for pre-training LLMs with MXFP8.arXiv preprint arXiv:2506.08027,

    12 ACRL: Adaptive Control of Training-Inference Discrepancy for Stable Reinforcement Learning Mishra, A., Stosic, D., Layton, S., and Micikevicius, P. Recipes for pre-training LLMs with MXFP8.arXiv preprint arXiv:2506.08027,

    Pith/arXiv arXiv

  12. [2017]

    Deepseekmath: Push- ing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300,

    Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y ., Wu, Y ., et al. Deepseekmath: Push- ing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300,

    Pith/arXiv arXiv

  13. [2019]

    MiniMax- M1: Scaling test-time compute efficiently with lightning attention.arXiv preprint arXiv:2506.13585,

    Chen, A., Li, A., Gong, B., Jiang, B., Fei, B., Yang, B., Shan, B., Yu, C., Wang, C., Zhu, C., et al. MiniMax- M1: Scaling test-time compute efficiently with lightning attention.arXiv preprint arXiv:2506.13585,

    Pith/arXiv arXiv

  14. [2022]

    Sparse attention post-training for mechanistic interpretability

    Draye, F., Lei, A., Posner, I., and Sch ¨olkopf, B. Sparse attention post-training for mechanistic interpretability. arXiv preprint arXiv:2512.05865,

    Pith/arXiv arXiv

  15. [2025]

    Training verifiers to solve math word problems

    Cobbe, K., Kosaraju, V ., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168,

    Pith/arXiv arXiv