Pith. sign in

REVIEW 3 major objections 5 minor 8 cited by

AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergy

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 7B model leads Qwen2.5-7B peers on math and code reasoning, the paper reports.

desk verdict Solid empirical post-training recipe from a credible group; model and data are out, but the headline SOTA numbers hinge on a decontamination step that is only described for SFT data, not for the inherited RL data. read the letter →

arxiv 2506.13284 v1 pith:B5MLVYKF submitted 2025-06-16 cs.CL cs.AIcs.LG

Zihan Liu , Zhuolin Yang , Yang Chen , Chankyu Lee , Mohammad Shoeybi , Bryan Catanzaro , Wei Ping This is my paper
classification cs.CLcs.AIcs.LG
keywords supervisedfine-tuningreinforcementlearningchain-of-thoughtreasoningmathcodegenerationGRPOSFTscalingmodelpost-training
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that supervised fine-tuning and reinforcement learning work as a pair, not as independent stages, and that a 7B model can reach the top of its size class if SFT data is scaled correctly before RL begins. It claims two SFT scaling axes help: adding unique prompts improves accuracy more than adding extra responses per prompt (fitted coefficients $a=4.831$ versus $b=2.635$ on a log scale), and training through five to six epochs, past the point of conventional overfitting, continues to improve long-chain reasoning. For RL, the paper claims a stronger SFT initialization yields a stronger final model, though the gap between initializations narrows substantially during training. The central operating rule is to set the sampling temperature so that the temperature-adjusted entropy stays near 0.3, which balances exploration and exploitation, and to apply stage-wise RL with growing response-length budgets. The resulting model, AceReason-Nemotron-1.1-7B, reaches scores the paper reports as the highest among Qwen2.5-7B-based reasoning models on AIME25 and LiveCodeBench v6.

What carries the argument

The load-bearing mechanism is a two-part recipe rather than a single formula. First, SFT data scaling: 247K math prompts plus 136K code prompts, with response-length filtering to rebalance difficulty, and multi-epoch training to the point of mild overfitting. Second, a stage-wise RL curriculum using GRPO with token-level policy-gradient advantages, starting with a short 8K warm-up that compresses reasoning paths (average response length drops from roughly 5K to 4K tokens), then progressively longer budgets of 16K, 24K, and 32K on harder prompts, with overlong filtering used in early stages and dropped in the final 32K stage. The named control rule is to keep the temperature-adjusted entropy around 0.3 during RL training: at a sampling temperature of 0.85 the entropy starts near 0.26 and climbs to about 0.38, which the paper associates with the best exploration-exploitation balance and the highest benchmark scores.

What would settle it

Run an independent contamination audit of the released SFT and RL data against AIME25 and LiveCodeBench v6 using paraphrased or lightly edited test problems that share no 9-gram with the originals; if the training data still contains recognizable versions of held-out problems, the reported RL gains on those benchmarks would shrink or disappear when retrained on a clean corpus.

Watch

Extended reading notes

Core claim

Starting from Qwen2.5-Math-7B, the authors build an SFT corpus of 383K prompts, filter it for contamination with a 9-gram overlap test, and generate responses with DeepSeek-R1. Scaling the dataset from 36K to 2.2M samples, with prompt count scaled more aggressively than responses per prompt, raises AIME24 from 41.0 to 63.0 across dataset versions v1 through v7. They then apply the stage-wise RL recipe from their earlier AceReason work: GRPO with token-level advantages, strictly on-policy rollouts, no KL term, and response-length budgets growing from 8K to 32K, interleaving math-only and code-only stages. The finding is that RL adds 10.6 points on AIME24 and 8.3 points on LiveCodeBench v6 over the already-strong SFT checkpoint, while math-only RL alone lifts code scores by several points. The final model reaches 72.6% on AIME24 (avg@64), 64.8% on AIME25, and 52.1% on LiveCodeBench v6, which the paper reports as the highest among Qwen2.5-7B-based reasoning models.

Load-bearing premise

The comparison is only as clean as the 9-gram decontamination filter: the paper assumes that removing samples with any 9-gram overlap with test benchmarks fully removes train/test leakage, and that lower-contamination benchmarks like AIME25 and LiveCodeBench v6 are not leaking through other channels.

Editorial extensions

If this is right

  • Scaling unique prompts is the higher-leverage SFT move: the fitted regression gives a prompt-count coefficient of 4.831 versus 2.635 for responses per prompt, so collecting more diverse questions should be preferred when feasible.
  • A stronger SFT start pays off after RL: the final model from SFT v7 beats the one from SFT v5, but the initial 6.6-point AIME24 gap closes to 1.6 points, implying RL can compensate for weaker SFT diversity.
  • The 8K warm-up RL stage is worth keeping even though it temporarily lowers accuracy: skipping it costs the Stage-2 result on AIME25 (51.8 versus 56.7).
  • Overlong filtering is length-budget dependent: it helps at 8K and 16K, is nearly neutral at 24K, and hurts at 32K, so the decision should be made per stage rather than globally.
  • Math-only RL transfers to code: math stages alone raise LiveCodeBench performance, and the final model solves a long tail of hard problems the SFT model misses within 128 attempts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The entropy-around-0.3 rule is stated as a rule of thumb for one model family; if it generalizes, it gives other teams a cheap diagnostic for setting RL temperature without full sweeps.
  • The narrowing of SFT-to-SFT gaps under RL hints that, at fixed compute, RL can substitute for some SFT data diversity, though the paper's AIME25 results also suggest the substitution has a ceiling when starting models differ greatly.
  • The math-to-code transfer is evidence that RL on verifiable math may train a shared reasoning skill rather than only memorizing problem templates; a testable implication is that RL on other verifiable domains should similarly improve code scores.
  • If later audits find train/test leakage beyond the 9-gram filter, the numerical rankings on AIME25 and LiveCodeBench v6 would need to be re-read as upper bounds; this is the main threat to the state-of-the-art claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper investigates the synergy between supervised fine-tuning (SFT) and reinforcement learning (RL) for building a 7B math/code reasoning model. It curates SFT data with two scaling axes (number of prompts, number of responses per prompt), fits a regression to claim prompt-count scaling matters more, and then applies a stage-wise math-then-code RL recipe inherited from AceReason-Nemotron-1.0. Through ablations, the paper reports that RL from stronger SFT models yields better final performance, that a sampling temperature keeping 'temperature-adjusted entropy' near 0.3 is beneficial, that overlong filtering helps only at short length budgets, and that math-only RL transfers to code. The final model, AceReason-Nemotron-1.1-7B, is claimed to be state-of-the-art among Qwen2.5-7B-based reasoning models on AIME25 and LiveCodeBench v6. The model and data are released.

Significance. If the central claims hold, the paper provides a practical and largely reproducible post-training recipe, with useful ablations on SFT scaling, RL temperature, overlong filtering, and cross-domain transfer. The release of the model and data is a concrete contribution, and the systematic comparison across multiple SFT initializations is valuable. However, the current evidence for the two headline insights—prompt-count scaling dominance and the temperature-adjusted-entropy rule—is statistically thin, and the SOTA claim relies on decontamination assurances that are not documented for the RL data. These issues limit the confidence with which the conclusions can be accepted as stated.

major comments (3)
  1. [§3.2.2, §4.3] The RL data decontamination status is not established. Section 3.2.2 states only that the RL data are inherited from AceReason-Nemotron-1.0, while the 9-gram overlap filter described in §3.1.1 is explicitly for SFT prompt collection. The SOTA claim in §4.3 and the RL-gain analyses in §4.5.1 and §4.5.6 are made specifically on AIME25 (released February 2025) and LiveCodeBench v6 (covering 2025/02/01–2025/05/01), which the paper itself calls lower-contamination benchmarks. The paper must state whether the inherited RL data were decontaminated against these benchmarks, and if so, how. Without this, the reported gains over the SFT model could be inflated by memorization rather than by SFT-RL synergy.
  2. [§4.4.2] The regression analysis uses seven data points (v1–v7) and three fitted parameters (a, b, c) and reports R²=0.989, but provides no standard errors, confidence intervals, cross-validation, or residual diagnostics. The conclusion that scaling the number of prompts has a larger impact than scaling responses per prompt is a central contribution in the abstract and §1, yet it rests on a comparison of two point estimates (a=4.831 vs. b=2.635) with no measure of uncertainty. The authors should add leave-one-out or bootstrap intervals and check sensitivity to the endpoints; otherwise the claim is underdetermined.
  3. [§4.5.2, Figure 8] The 'temperature-adjusted entropy' is never defined. The text and figure refer to it as the key quantity behind the rule that the training temperature should keep it around 0.3, but no equation is given, so the rule is not reproducible. In addition, the evidence comes from three temperature settings (0.6, 0.85, 1.0) on what appears to be a single SFT model. The paper should provide the definition, state which SFT initialization was used, and either show evidence that the rule transfers to other SFT models or explicitly restrict the claim to a rule of thumb for the tested setting.
minor comments (5)
  1. [§5 (Conclusion)] The conclusion states the final model scores 63.2% on AIME25 and 52.8% on LiveCodeBench V5, but Table 1 reports 64.8% and 57.2% for the same model on those benchmarks. Please correct the conclusion or clarify which model variant these numbers refer to.
  2. [§3.1.2] 'we modify the rope_theta parameter from 10,000 to 1,000,000 enable support for a context length of 128K' is missing the word 'to' before 'enable'.
  3. [§1 (Contribution 1)] Typo: 'effecive RL training' should be 'effective RL training'.
  4. [Figure 2] The training pipeline figure appears to be a screenshot from an external editor with a page header; it should be redrawn for legibility and to match the journal style.
  5. [§4.5.2] The temperature study is described as applying to 'a given model', but the manuscript does not specify which SFT checkpoint (v5, v7, or DeepSeek-R1-Distill-Qwen-7B) was used for Figure 8. Please state the model and, ideally, show that the trend holds for at least one other initialization.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the headline SOTA and SFT-RL synergy results are empirical and benchmarked against external baselines, and the cited prior AceReason work is reproduced rather than assumed.

full rationale

The paper's central claims are empirical measurements rather than derivations from their own definitions. The SFT scaling analysis in Section 4.4 is an in-sample regression summary of seven experimental data points, not a hidden prediction, and Table 1 compares against external models such as MiMo-7B-RL, Skywork-OR1-7B, o3-mini, and Magistral Small, so the benchmark numbers are not computed from the fitted parameters. The temperature-entropy rule in Section 4.5.2 is a post-hoc heuristic based on the authors' own trials; although it is not a verified law and could be criticized as descriptive rather than predictive, it does not define the reported AIME25 or LiveCodeBench accuracies. The RL recipe is inherited from the authors' own AceReason-Nemotron paper (Chen et al., 2025) and cited for design choices, but the paper re-runs and ablates the recipe and validates the final model against external benchmarks, so the self-citations are supportive rather than load-bearing. A separate contamination concern exists: Section 3.2.2 says RL data is inherited from AceReason-Nemotron-1.0 without restating a decontamination step against AIME25 or LiveCodeBench v6, and the 9-gram filter described in Section 3.1.1 is stated for SFT prompt collection only. That is a potential data-leakage or correctness risk, not a circularity of the derivation chain, because the benchmark numbers are not defined by the filter. No circular step could be exhibited with a quote-and-reduction, so the circularity score is low.

Assumptions & free parameters 3 free parameters · 5 assumptions · 1 invented entities

The central claims rest on several unverified domain assumptions (base model quality, data quality, verifier correctness, contamination removal) and one self-defined heuristic (temperature-adjusted entropy). The regression coefficients and entropy target are fitted or hand-chosen; no formal verification is provided.

free parameters (3)
  • SFT scaling regression coefficients a and b = a=4.831, b=2.635
    Fit in Section 4.4.2 to 7 dataset points to compare prompt vs response scaling.
  • Target temperature-adjusted entropy = 0.3
    Rule of thumb in Section 4.5.2, hand-chosen after multiple RL trials.
  • RL sampling temperature = 0.85
    Selected in Section 4.5.2 by comparing 0.6, 0.85, and 1.0 on a single SFT model.
assumptions (5)
  • domain assumption Qwen2.5-Math-7B is a suitable base model for the SFT+RL recipe
    All training starts from this base; no comparison with other base models at the same scale.
  • domain assumption DeepSeek-R1 generated responses are high-quality supervision for SFT
    Section 3.1.1 uses DeepSeek-R1 for all SFT response generation; quality is assumed, not validated.
  • domain assumption 9-gram overlap filtering removes test contamination
    Section 3.1.1 relies on this as the only decontamination measure; may miss semantic or paraphrased leakage.
  • domain assumption Rule-based verifiers provide correct reward signals
    Section 3.2.1 uses rule-based verification; the paper acknowledges code test cases can yield false positives/negatives.
  • ad hoc to paper Temperature-adjusted entropy is a valid proxy for exploration-exploitation balance
    Section 4.5.2 introduces this quantity and the 0.3 target; no external benchmark validates this proxy.
invented entities (1)
  • Temperature-adjusted entropy
    purpose: A scalar target for setting RL sampling temperature to balance exploration and exploitation
    Introduced in Section 4.5.2 as a rule-of-thumb; the 0.3 target is derived from the authors' own trials and has no independent validation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergy." pith.science (2026). https://pith.science/paper/B5MLVYKF

@misc{pith2026250613284,
  author       = {Pith},
  title        = {Pith review of: AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B5MLVYKF}},
  note         = {Machine review of arXiv:2506.13284}
}
read the original abstract

In this work, we investigate the synergy between supervised fine-tuning (SFT) and reinforcement learning (RL) in developing strong reasoning models. We begin by curating the SFT training data through two scaling strategies: increasing the number of collected prompts and the number of generated responses per prompt. Both approaches yield notable improvements in reasoning performance, with scaling the number of prompts resulting in more substantial gains. We then explore the following questions regarding the synergy between SFT and RL: (i) Does a stronger SFT model consistently lead to better final performance after large-scale RL training? (ii) How can we determine an appropriate sampling temperature during RL training to effectively balance exploration and exploitation for a given SFT initialization? Our findings suggest that (i) holds true, provided effective RL training is conducted, particularly when the sampling temperature is carefully chosen to maintain the temperature-adjusted entropy around 0.3, a setting that strikes a good balance between exploration and exploitation. Notably, the performance gap between initial SFT models narrows significantly throughout the RL process. Leveraging a strong SFT foundation and insights into the synergistic interplay between SFT and RL, our AceReason-Nemotron-1.1 7B model significantly outperforms AceReason-Nemotron-1.0 and achieves new state-of-the-art performance among Qwen2.5-7B-based reasoning models on challenging math and code benchmarks, thereby demonstrating the effectiveness of our post-training recipe. We release the model and data at: https://huggingface.co/nvidia/AceReason-Nemotron-1.1-7B

Figures

Figures reproduced from arXiv: 2506.13284 by the authors.

Figure 1
Figure 1. Benchmark accuracy of AceReason-Nemotron-1.1-7B on AIME 2024/2025 (avg@64), HMMT 2025 (avg@64), LiveCodeBench v5 (2024/08/01-2025/02/01, avg@8), and v6 (2025/02/01-2025/05/01, avg@8) using 32,768 output length. †Leads the effort. ‡Correspondence to: Zihan Liu <zihanl@nvidia.com>, Wei Ping<wping@nvidia.com>. © 2025 NVIDIA. All rights reserved. arXiv:2506.13284v1 [cs.CL] 16 Jun 2025 [PITH_FULL_IMAGE:figures/full_fig_… view at source ↗
Figure 2
Figure 2. Training Pipeline of AceReason-Nemotron 1.1. We start by performing math and code SFT on a base pretrained model. Next, we conduct three stages of math-only RL training with progressively growing response length, i.e., Stage-1 (8K), Stage-2 (16K), and Stage-3 (24K), to develop a math-specialized RL model. We then apply code-only RL training to enhance model’s coding capability. Lastly, we carry out a final stage of … view at source ↗
Figure 3
Figure 3. Response token length distributions for the math SFT dataset (left) and the code SFT dataset (right). 3. Method In this section, we present the details of the supervised fine-tuning and reinforcement learning processes used to train AceReason-Nemotron 1.1. The overall training pipeline is illustrated in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: Log-scaled data statistics for the number of math and code prompts and the average number of responses per prompt. Each SFT dataset consist of both math and code SFT samples. v1 v2 v3 v4 v5 v6 v7 SFT Dataset Version 40 45 50 55 60 Accuracy (%) DeepSeek-R1-Distill-Qwen-…
Figure 5
Figure 5. Figure 5: Accuracies on AIME24, AIME25, and LiveCodeBench V5 and V6 for different SFT datasets. For each SFT blend, the model is trained until the accuracy plateaus [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Accuracies over different epochs of training for SFT dataset v6 and v7 [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Math-only RL training starting from different SFT (distillation) models. The AIME24 accuracy at step-0 reflects the performance of the initial SFT checkpoints. The subsequent numbers in the figure show the final accuracy achieved at the end of each training stage: Math…
Figure 8
Figure 8. Figure 8: Left: Trajectories of temperature-adjusted entropy during RL training with different policy LLM temperature settings. Right: Impact of varying temperatures for inference and RL training. We observe that using a temperature of 0.6 for inference consistently yields bette…
Figure 9
Figure 9. Figure 9: Ablation Studies on Math-Only RL training to assess the impact of overlong filtering. In both settings, Stage-1 starts with the same SFT model, and each subsequent stage begins with the same RL model from the previous stage trained under the best-performing setting (i.…
Figure 10
Figure 10. Figure 10: Left: Ablation study comparing models trained with and without Math-Only Stage-1. For “w/o Stage-1”, the step-0 accuracy reflects the performance of the our SFT model on AIME25. In contrast, for "w/ Stage-1", the step-0 accuracy represents the final performance of Sta…
Figure 11
Figure 11. Figure 11: LiveCodeBench V5 accuracy over different Math-Only RL stages. In AceReason-Nemotron-1.0 (Chen et al., 2025), we found that math-only RL significantly improves performance on code reasoning benchmarks. We reaffirm this finding using different SFT model as initializatio…
Figure 12
Figure 12. Figure 12: Comparison of pass@K scores between AceReason-Nemotron-1.1-7B and the SFT-7B v7 model it is trained from. To compute pass@K, we generate 256 outputs per sample for AIME24 and AIME25, and 128 outputs for LiveCodeBench V5 and V6. We then randomly select K outputs, and e…
Figure 13
Figure 13. Figure 13: Comparison of problem-level solving rates between AceReason-Nemotron1.1-7B and the SFT-7B v7 model it is trained from. For each problem, accuracy is averaged over 256 outputs for AIME24 and AIME25, and over 128 outputs for LiveCodeBench V5 and V6. 6. Acknowledgement W…
Figure 14
Figure 14. Figure 14: Math-only RL training starting from different SFT (distillation) models. The AIME25 accuracy at step-0 reflects the performance of the initial SFT checkpoints. The subsequent numbers in the figure show the final accuracy achieved at the end of each training stage: Mat…
Figure 15
Figure 15. Figure 15: Pass@k results on AIME24, AIME25, LiveCodeBench V5, and V6, showcasing two SFT models and their subsequent Math-Only RL-trained versions. To compute pass@k, we generate 256 outputs per sample for AIME24 and AIME25, and 128 outputs for LiveCodeBench V5 and V6. We then …
Figure 16
Figure 16. Figure 16: Comparison of problem-level solving rates between the SFT model (v7) and the model after Math-Only RL training. For each problem, accuracy is averaged over 256 outputs for AIME24 and AIME25, and over 128 outputs for LiveCodeBench V5 and V6. 1 8 9 10 13 25 27 12 16 7 2…
Figure 17
Figure 17. Figure 17: Problem-level solving rates comparison between SFT model (v5) and after Math-Only RL training. For each problem, accuracy is averaged over 256 outputs for AIME24 and AIME25, and over 128 outputs for LiveCodeBench V5 and V6. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_17.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Scaling Latent Reasoning via Looped Language Models

    cs.CL 2025-10 unverdicted novelty 7.0 of 10

    Looped language models with latent iterative computation and entropy-regularized depth allocation achieve performance matching up to 12B standard LLMs through superior knowledge manipulation.

  2. MapTab: A Diagnostic Benchmark for Long-Horizon Multi-Criteria Multimodal Reasoning on Heterogeneous Topological Graphs

    cs.LG 2026-02 unverdicted novelty 6.0 of 10

    MapTab is a new multimodal benchmark with 328 images and nearly 200k queries that shows current MLLMs have substantial difficulty with multi-criteria route planning when visual and tabular information must be combined.

  3. Entropy-Preserving Supervised Fine-Tuning via Adaptive Self-Distillation for Large Reasoning Models

    cs.LG 2026-02 conditional novelty 6.0 of 10

    CurioSFT preserves exploration during supervised fine-tuning by distilling toward the model's own temperature-scaled distribution and adaptively increasing entropy at high-entropy tokens, improving SFT accuracy by ~2....

  4. Video Reasoning without Training

    cs.CV 2025-10 conditional novelty 6.0 of 10

    An entropy-guided, inference-time value-cache controller improves video reasoning accuracy and cuts output tokens versus RL-trained baselines.

  5. The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMs

    cs.CL 2025-07 conditional novelty 6.0 of 10

    Long-CoT SFT and RL trade off strengths in reasoning VLMs, and five combination strategies fail to produce additive gains.

  6. DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation

    cs.CL 2025-06 conditional novelty 6.0 of 10

    A 7B masked-diffusion code model plus complementary-mask GRPO (coupled-GRPO) improves benchmark scores and shifts decoding away from strict left-to-right order.

  7. A Sovereign, Open-Source Foundation Model for German and English

    cs.CL 2026-07 conditional novelty 5.5 of 10

    Soofi S 30B-A3B, a hybrid Mamba-MoE model pretrained on ~27T tokens with deliberately up-weighted German, reports the highest English and German aggregate scores among fully open base models in its comparison while ma...

  8. Attention Sink Forges Native MoE in Attention Layers: Sink-Aware Training to Address Head Collapse

    cs.CL 2026-02 unverdicted novelty 5.0 of 10

    Attention-sink weight is recast as an implicit MoE router per head, motivating a sink-aware head-balancing loss that yields small, consistent benchmark gains across three attention variants but rests on a definitional...

Reference graph

Works this paper leans on

43 extracted references · 14 canonical work pages · cited by 8 Pith papers

  1. [1]

    Opencodereasoning: Advancing data distillation for competitive coding

    Wasi Uddin Ahmad, Sean Narenthiran, Somshubra Majumdar, Aleksander Ficek, Siddhartha Jain, Jo- celyn Huang, Vahid Noroozi, and Boris Ginsburg. Opencodereasoning: Advancing data distillation for competitive coding. arXiv preprint arXiv:2504.01943, 2025. 3, 4, 5, 9

    arXiv 2025

  2. [2]

    Matharena: Evaluating llms on uncontaminated math competitions, february 2025.URL https://matharena

    Mislav Balunovic, Jasper Dekoninck, Ivo Petrov, Nikola Jovanovic, and Martin Vechev. Matharena: Evaluating llms on uncontaminated math competitions, february 2025.URL https://matharena. ai, 2025. 8

    work page 2025

  3. [3]

    Llama-Nemotron: Efficient Reasoning Models.arXiv preprint arXiv:2505.00949, 2025

    Akhiad Bercovich, Itay Levy, Izik Golan, Mohammad Dabbah, Ran El-Yaniv, Omri Puny, Ido Galil, Zach Moshe, Tomer Ronen, Najeeb Nabwani, et al. Llama-Nemotron: Efficient Reasoning Models.arXiv preprint arXiv:2505.00949, 2025. 3, 4, 8

    arXiv 2025

  4. [4]

    Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021. 3

    arXiv 2021

  5. [5]

    Acereason-nemotron: Advancing math and code reasoning through reinforcement learning

    Yang Chen, Zhuolin Yang, Zihan Liu, Chankyu Lee, Peng Xu, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. Acereason-nemotron: Advancing math and code reasoning through reinforcement learning. arXiv preprint arXiv:2505.16400, 2025. 3, 4, 6, 7, 8, 9, 15, 16

    arXiv 2025

  6. [6]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. 3, 4

    arXiv

  7. [7]

    NVLM: Open frontier-class multimodal LLMs

    Wenliang Dai, Nayeon Lee, Boxin Wang, Zhuolin Yang, Zihan Liu, Jon Barker, Tuomas Rintamaki, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. NVLM: Open frontier-class multimodal LLMs. arXiv preprint arXiv:2409.11402, 2024. 4

    arXiv 2024

  8. [8]

    Audio flamingo 2: An audio-language model with long-audio under- standing and expert reasoning abilities.arXiv preprint arXiv:2503.03983, 2025

    Sreyan Ghosh, Zhifeng Kong, Sonal Kumar, S Sakshi, Jaehyeon Kim, Wei Ping, Rafael Valle, Dinesh Manocha, and Bryan Catanzaro. Audio flamingo 2: An audio-language model with long-audio under- standing and expert reasoning abilities.arXiv preprint arXiv:2503.03983, 2025. 4

    arXiv 2025

Show all 43 references
  1. [9]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024. 8

    2024 arXiv

  2. [10]

    Deepseek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. 3, 4, 5, 6, 8

    2025 arXiv

  3. [11]

    Skywork open reasoner series, 2025

    Jujie He, Jiacai Liu, Chris Yuhao Liu, Rui Yan, Chaojie Wang, Peng Cheng, Xiaoyu Zhang, Fuxiang Zhang, Jiacheng Xu, Wei Shen, Siyuan Li, Liang Zeng, Tianwen Wei, Cheng Cheng, Bo An, Yang Liu, and Yahui Zhou. Skywork open reasoner series, 2025. Notion Blog. 3, 9

    2025

  4. [12]

    Skywork open reasoner 1 technical report.arXiv preprint arXiv:2505.22312,

    Jujie He, Jiacai Liu, Chris Yuhao Liu, Rui Yan, Chaojie Wang, Peng Cheng, Xiaoyu Zhang, Fuxiang Zhang, Jiacheng Xu, Wei Shen, et al. Skywork open reasoner 1 technical report.arXiv preprint arXiv:2505.22312,

    arXiv

  5. [13]

    Measuring coding challenge competence with apps

    Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt. Measuring coding challenge competence with apps. NeurIPS, 2021. 5 18 AceReason-Nemotron 1.1: Advancing Math and Code R...

    2021

  6. [14]

    Measuring mathematical problem solving with the math dataset.Sort, 2(4):0–6, 2021

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset.Sort, 2(4):0–6, 2021. 8

    2021

  7. [15]

    Opencoder: The open cookbook for top-tier code large language models

    Siming Huang, Tianhao Cheng, Jason Klein Liu, Jiaran Hao, Liuyihan Song, Yang Xu, J Yang, JH Liu, Chenchen Zhang, Linzheng Chai, et al. Opencoder: The open cookbook for top-tier code large language models. arXiv preprint arXiv:2411.04905, 2024. 5

    2024 arXiv

  8. [16]

    Open r1: A fully open reproduction of deepseek-r1, January 2025

    HuggingFace. Open r1: A fully open reproduction of deepseek-r1, January 2025. URLhttps://github. com/huggingface/open-r1. 9

    2025

  9. [17]

    Qwen2.5-coder technical report.arXiv preprint arXiv:2409.12186, 2024

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. Qwen2.5-coder technical report.arXiv preprint arXiv:2409.12186, 2024. 3, 4

    2024 arXiv

  10. [18]

    Livecodebench: Holistic and contamination free evaluation of large language models for code.arXiv preprint arXiv:2403.07974, 2024

    Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar- Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code.arXiv preprint arXiv:2403.07974, 2024. 8

    2024 arXiv

  11. [19]

    Numinamath

    Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Costa Huang, Kashif Rasul, Longhui Yu, Albert Jiang, Ziju Shen, Zihan Qin, Bin Dong, Li Zhou, Yann Fleureau, Guillaume Lample, and Stanislas Polu. Numinamath. [https://huggingface.co/AI-MO/NuminaMath...

    2024

  12. [20]

    Taco: Topics in algorithmic code generation dataset.arXiv preprint arXiv:2312.14852, 2023

    Rongao Li, Jie Fu, Bo-Wen Zhang, Tao Huang, Zhihong Sun, Chen Lyu, Guang Liu, Zhi Jin, and Ge Li. Taco: Topics in algorithmic code generation dataset.arXiv preprint arXiv:2312.14852, 2023. 5

    2023 arXiv

  13. [21]

    Deepseek-V3 technical report.arXiv preprint arXiv:2412.19437,

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-V3 technical report.arXiv preprint arXiv:2412.19437,

    arXiv

  14. [22]

    Is your code generated by chatGPT really correct? rigorous evaluation of large language models for code generation

    Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by chatGPT really correct? rigorous evaluation of large language models for code generation. InThirty-seventh Conference on Neural Information Processing Systems, 2023. URLhttps://openreview...

    2023

  15. [23]

    Evaluating language models for efficient code generation

    Jiawei Liu, Songrun Xie, Junhao Wang, Yuxiang Wei, Yifeng Ding, and Lingming Zhang. Evaluating language models for efficient code generation. InFirst Conference on Language Modeling , 2024. URL https://openreview.net/forum?id=IBCBMeAhmC. 8

    2024

  16. [24]

    Prorl: Prolonged reinforcement learning expands reasoning boundaries in large language models.arXiv preprint arXiv:2505.24864, 2025

    Mingjie Liu, Shizhe Diao, Ximing Lu, Jian Hu, Xin Dong, Yejin Choi, Jan Kautz, and Yi Dong. Prorl: Prolonged reinforcement learning expands reasoning boundaries in large language models.arXiv preprint arXiv:2505.24864, 2025. 3

    2025 arXiv

  17. [25]

    AceMath: Advancing frontier math reasoning with post-training and reward modeling.arXiv preprint arXiv:2412.15084, 2024

    Zihan Liu, Yang Chen, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. AceMath: Advancing frontier math reasoning with post-training and reward modeling.arXiv preprint arXiv:2412.15084, 2024. 3, 4, 5

    2024 arXiv

  18. [26]

    Deepcoder: A fully open-source 14b coder at o3-mini level, 2025

    Michael Luo, Sijun Tan, Roy Huang, Xiaoxiang Shi, Rachel Xin, Colin Cai, Ameen Patel, Alpay Ariyak, Qingyang Wu, Ce Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepcoder: A fully open-source 14b coder at o3-mini level, 2025. Notion Blog. 3, 4, 13

    2025

  19. [27]

    Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica

    Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica. DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL, 2025. Notion Blog. 4 19 AceReason-Nemotron 1.1: Advancin...

    2025

  20. [28]

    Aimo-2 winning solution: Building state-of-the-art mathematical reasoning models with openmathreasoning dataset.arXiv preprint arXiv:2504.16891, 2025

    Ivan Moshkov, Darragh Hanley, Ivan Sorokin, Shubham Toshniwal, Christof Henkel, Benedikt Schifferer, Wei Du, and Igor Gitman. Aimo-2 winning solution: Building state-of-the-art mathematical reasoning models with openmathreasoning dataset.arXiv preprint arXiv:2504.16891, 2025. ...

    2025 arXiv

  21. [29]

    s1: Simple test-time scaling.arXiv preprint arXiv:2501.19393, 2025

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettle- moyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling.arXiv preprint arXiv:2501.19393, 2025. 5

    2025 arXiv

  22. [30]

    Learning to reason with LLMs, 2024

    OpenAI. Learning to reason with LLMs, 2024. 3

    2024

  23. [31]

    QwQ-32B: Embracing the Power of Reinforcement Learning, 2025

    Qwen-Team. QwQ-32B: Embracing the Power of Reinforcement Learning, 2025. URLhttps://qwenlm. github.io/blog/qwq-32b/. 3, 4

    2025

  24. [32]

    Areal: Ant reasoning rl.https://github.com/inclusionAI/AReaL, 2025

    Ant Research RL Lab. Areal: Ant reasoning rl.https://github.com/inclusionAI/AReaL, 2025. 9

    2025

  25. [33]

    Seed-thinking-v1

    ByteDance Seed, Yufeng Yuan, Yu Yue, Mingxuan Wang, Xiaochen Zuo, Jiaze Chen, Lin Yan, Wenyuan Xu, Chi Zhang, Xin Liu, et al. Seed-thinking-v1. 5: Advancing superb reasoning models with reinforcement learning. arXiv preprint arXiv:2504.13914, 2025. 4

    2025

  26. [34]

    DeepseekMath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. DeepseekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. 3, 4, 6

    2024 arXiv

  27. [35]

    Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256,

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256,

    arXiv

  28. [36]

    Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837, 2022

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837, 2022. 3, 4

    2022

  29. [37]

    Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond.arXiv preprint arXiv:2503.10460, 2025

    Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Lifu Tang, Xiaowei Lv, et al. Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond.arXiv preprint arXiv:2503.10460, 2025. 3, 4, 8

    2025 arXiv

  30. [38]

    Mimo: Unlocking the reasoning potential of language model–from pretraining to posttraining.arXiv preprint arXiv:2505.07608, 2025

    Bingquan Xia, Bowen Shen, Dawei Zhu, Di Zhang, Gang Wang, Hailin Zhang, Huaqiu Liu, Jiebao Xiao, Jinhao Dong, Liang Zhao, et al. Mimo: Unlocking the reasoning potential of language model–from pretraining to posttraining.arXiv preprint arXiv:2505.07608, 2025. 4, 8

    2025 arXiv

  31. [39]

    Qwen2.5 technical report.arXiv preprint arXiv:2412.15115, 2024

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2.5 technical report.arXiv preprint arXiv:2412.15115, 2024. 8

    2024 arXiv

  32. [40]

    Qwen2.5-Math technical report: Toward mathematical expert model via self-improvement

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2.5-Math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122, 2024. 3, 4, 6

    2024 arXiv

  33. [41]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025. 3, 4

    2025 arXiv

  34. [42]

    DAPO: An open-source LLM reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. DAPO: An open-source LLM reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025. 4, 12

    2025 arXiv

  35. [43]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479, 2025. 4 20 AceReason-Nem...

    2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.