Pith. sign in

REVIEW 4 major objections 5 minor 18 cited by

Parallel-R1 is the first RL framework to train parallel-thinking behaviors in LLMs on general math, using a progressive curriculum that beats the GRPO baseline by 8.4% on average.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-04 21:23 UTC pith:VUMJ4IFS

load-bearing objection Parallel-R1 is a genuinely new RL recipe for instilling parallel thinking, but the headline gains are not yet separable from an uncontrolled inference-compute difference; still worth refereeing. the 4 major comments →

arxiv 2509.07980 v2 pith:VUMJ4IFS submitted 2025-09-09 cs.CL

Parallel-R1: Towards Parallel Thinking via Reinforcement Learning

Tong Zheng , Hongming Zhang , Wenhao Yu , Xiaoyang Wang , Runpeng Dai , Rui Liu , Huiwen Bao , Chengsong Huang
show 2 more authors
Heng Huang Dong Yu
This is my paper
classification cs.CL
keywords parallel thinkingreinforcement learningGRPOcurriculum learningmathematical reasoningreward designcold-start trainingexploration scaffold
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Parallel-R1 is a training framework that tries to make an LLM genuinely think in parallel—pausing its main reasoning chain, spawning several independent solution attempts, and summarizing them—rather than merely imitating such traces from synthetic data. The core proposal is a progressive curriculum that solves the cold-start problem: teach the model the parallel-thinking format on easy GSM8K problems with a small supervised fine-tuning set, then let reinforcement learning (GRPO on DAPO problems) explore and generalize the behavior on hard competition tasks. The paper reports average accuracy gains of 8.4% over the standard GRPO baseline across AIME25, AIME24, AMC23, and MATH, and a peak AIME25 score of 25.6% obtained by a two-stage RL schedule where parallel thinking is first forced for exploration and then dropped. It also documents a strategic shift during training: the model starts by using parallel paths to search for solutions and later uses them to verify an already-derived answer. If correct, this shows that parallel thinking is a learnable train-time skill, not just an architectural or prompting trick.

Core claim

The central claim is that parallel thinking—generating multiple independent reasoning paths at critical steps and then summarizing them—can be instilled in an LLM by a progressive curriculum that separates format learning from behavioral exploration. The authors show that a powerful model (DeepSeek-R1-0528-Qwen-3-8B) can produce well-formed parallel traces for 83.6% of simple GSM8K problems but for 0.0% of hard DAPO problems, so they use the easy data only to teach the <Parallel>/<Path>/<Summary> format via SFT, then switch to GRPO with an accuracy reward on DAPO so the model can explore and generalize the behavior on hard problems. The resulting Parallel-R1-Seen model averages 48.9 across A

What carries the argument

The load-bearing mechanism is the progressive curriculum: an SFT cold-start on Parallel-GSM8K teaches the control tags <Parallel>, <Path>, and <Summary>; a small RL stage on the same GSM8K set stabilizes the format with the reward R_parallel × R_acc; and a final GRPO stage on the DAPO set with accuracy-only reward lets the model learn when and how to branch on hard problems. The paper also tests a structured variant with path-window attention masks and multiverse position encodings to isolate parallel paths, but finds the causal variant works better; the full framework therefore rests on the interaction between the cold-start data and the reward schedule, not on architectural constraints.

Load-bearing premise

The progressive curriculum assumes that the parallel-thinking format learned on easy GSM8K problems transfers to hard DAPO problems once RL begins; if the format does not generalize beyond easy math, the cold-start stage fails to bootstrap RL and the benchmark gains disappear.

What would settle it

Train the same Qwen-3-4B base with GRPO on DAPO using the identical parallel-thinking prompt and the same alternating reward schedule, but without any SFT cold-start; measure the fraction of rollouts that contain a valid <Parallel> block and the final AIME25 accuracy. If valid parallel traces appear and accuracy matches Parallel-R1, cold-start SFT is unnecessary, and the paper's explanation of the cold-start bottleneck is wrong.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Models trained with this curriculum can be deployed with the parallel-thinking tags at inference, and they will branch and merge on their own, without an external orchestrator or a fixed branching schedule.
  • The same recipe can be applied to any base model: SFT on easy parallel traces, a short RL stage on the same easy set to stabilize the format, then RL on the hard target set.
  • Because the late-stage model uses parallelism for verification, downstream users get an internal consistency check that catches some wrong answers before final output.
  • The two-stage scaffold result implies that even if parallel thinking is too expensive to deploy at scale, spending 200 training steps forcing it can improve the final accuracy of the sequential model, so it should be considered a training-time intervention.
  • Reward design is decisive: purely structural rewards can destroy accuracy, purely outcome rewards fail to elicit parallel structure; the alternating schedule is a practical middle ground for production use.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: the same two-stage curriculum should work for other structured scaffolds (e.g., self-critique, tool-use blocks) if valid traces can be cheaply generated on an easy task; the key is that the format, not the content, is being bootstrapped.
  • The exploration-to-verification shift predicts that a stronger base model will place the <Parallel> block even later in generation and use shorter paths, since it can solve more problems on the first pass and only needs confirmation.
  • A focused study could replace the alternating reward schedule with a decaying parallel-reward coefficient during the RL stage, checking whether the model keeps the verification benefit without the early high-variance exploration being over-reinforced.
  • The near-0% format validity on DAPO suggests the bottleneck is not reward design but the model's inability to even emit a parallel structure on hard problems; any RL algorithm that cannot start from a valid sample will not explore that part of the policy space.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes Parallel-R1, a multi-stage reinforcement learning (RL) framework for teaching a Qwen3-4B base model to produce explicit parallel-thinking traces, using control tags <Parallel>, <Path>, and <Summary>. The training pipeline consists of a cold-start SFT stage on an easy GSM8K parallel-traces dataset, an optional RL stage on the same easy data, and a final RL stage on the harder DAPO dataset. A structured variant (Parallel-R1-Unseen) adds path-window attention masking and multiverse position encodings. The main empirical claim is an 8.4% average accuracy improvement over standard GRPO (48.9 vs 45.1 on AIME25, AIME24, AMC23, and MATH, Table 2). The paper also reports a behavioral shift from early exploration to late verification and proposes parallel thinking as a mid-training scaffold, with a peak AIME25 accuracy of 25.6%.

Significance. If the central claim holds, this is a useful and fairly original empirical contribution: it is one of the first attempts to use RL rather than SFT to instill parallel reasoning in general mathematical tasks, and it provides transparent reward analyses and a behavioral characterization of how the strategy evolves during training. The promised release of model, data, and code would strengthen reproducibility. The paper's key empirical claim is currently undermined, however, by the absence of any inference-compute control and by an undefined baseline in the scaffold experiment. These issues are fixable and do not invalidate the overall approach, but they must be addressed before the reported gains can be attributed to a learned parallel-thinking skill.

major comments (4)
  1. [Section 4.1; Table 2] The headline comparison is not inference-compute matched. Evaluation samples 16 responses per problem, and Parallel-R1's responses can contain multiple <Path> blocks, each effectively an additional candidate reasoning chain. The paper does not report average response length, token count, or FLOPs for any configuration. Thus the 8.4% average gain (48.9 vs 45.1) may reflect a larger effective test-time sample (self-consistency/best-of-k behavior) rather than a learned, transferable parallel-thinking skill. This is load-bearing for the central claim. Please report token-normalized or compute-matched accuracy (e.g., accuracy as a function of total generated tokens, or an equal-token-budget baseline), and include variance across seeds.
  2. [Figure 4; Section 4.5] The '42.9% improvement' claim is not tied to a defined baseline. Figure 4's legend says 'Baseline GRPO', but the only GRPO baseline in Table 2 has AIME25=14.8, which would make 25.6 a 73% improvement (25.6/14.8 − 1), not 42.9%. If the intended baseline is 17.9 (the 'Parallel-R1-Seen w/o RL on GSM8K' row in Table 3), that is an ablation variant, not a standard GRPO baseline, and the figure must say so explicitly. In addition, no error bars or multiple seeds are reported, and the '+200-step shift' used to align the Stage-2 curve makes the comparison difficult to verify.
  3. [Table 3; Section 4.3.2] The 'w/o Parallel Thinking Prompt' ablation does not control for the compute confound. Removing the prompt may reduce the rate of explicit <Parallel> tags, but the model can still generate long, multi-path outputs; without reporting the parallel ratio or token counts for this condition, the observed 1.8% average drop cannot be attributed to the prompt as opposed to changes in response length or other factors. This is directly relevant to the claim that the prompt helps the model 'better understand the reasoning process.'
  4. [Section 4.1; Section 4.2] No statistical reliability information is provided. All reported numbers come from single runs without standard deviations or seeds, despite mean@16 being a stochastic estimate. Given the relatively small benchmark sizes and the absence of any compute control, the reported differences (e.g., 48.9 vs 45.1) may be within run-to-run variance. At minimum, the authors should report multiple seeds or bootstrap confidence intervals.
minor comments (5)
  1. [Throughout] Typographical and naming issues: 'Parallel-SFT-Uneen' (Section 4.1) should be 'Unseen'; 'introduces' (Section 2.2) should be 'introduce'; '¡Path¿' (Section 3.4.1) is an encoding artifact; 'Baseline GRPO' and 'GRPO (DAPO)' are used inconsistently. The terms 'Seen' and 'Unseen' are never defined.
  2. [Figure 3] Please define 'relative position' explicitly in the caption or axis label, and clarify whether the plotted quantity is the mean across all <Parallel> blocks or only the first one. Error bars or a shaded confidence band would help, since the figure is central to the exploration-to-verification claim.
  3. [Section 1; Related Work] The phrase 'first reinforcement learning framework' is too strong given Pan et al. (2025) already applies RL to parallel reasoning on CountDown. The authors acknowledge this in Section 2.1, but the abstract and introduction should qualify the claim as 'first for general mathematical tasks' to avoid overclaiming.
  4. [Appendix A] The Parallel Thinking Prompt instructs the model not to explicitly mention that it is triggering parallel thinking, while Section 3.2 describes the model 'predicting' a <Parallel> token. This is not a contradiction, but the relationship between the prompt instruction and the trained behavior should be clarified.
  5. [Section 3.5.2] The reward notation is introduced separately for the causal and structured variants. Unifying the notation (e.g., defining R_parallel, R_acc, and the alternating schedule once) would improve readability and reduce the chance of misinterpreting the ablation results in Table 4.

Circularity Check

0 steps flagged

No significant circularity: the central claims are empirical benchmark comparisons with transparent reward shaping, not derivations that reduce to their own inputs.

full rationale

The paper's central claims are empirical: Parallel-R1 is trained with SFT cold-start on GSM8K plus GRPO on DAPO, and evaluated on held-out math benchmarks (AIME25, AIME24, AMC23, MATH). The reward functions are training objectives that explicitly incentivize the use of <Parallel>/<Path>/<Summary> structures, and the paper then measures how often those structures appear and how accuracy changes. This is transparent reward shaping plus behavioral tracking, not a fitted parameter renamed as a prediction. Key Finding 1 (83.6% format validity on GSM8K vs 0.0% on DAPO) is an empirical observation about data-generation difficulty, not an input-output tautology. The ablations in Table 3 compare training-stage configurations and are reported as measurements. The 'mid-training exploration scaffold' result in Figure 4 is an intervention study, not a derivation from an assumed uniqueness theorem. Self-citations (e.g., R-Zero, R1-RE, Mixture-of-Thought) appear only in related-work context and are not load-bearing for the main claims. The paper does contain limitations that a reviewer should weigh separately: Figure 4's 'Baseline GRPO' is not precisely defined, and the paper does not report token budgets or compute-matched evaluation, leaving open the possibility that accuracy gains partly reflect extra inference tokens rather than a learned transferable skill. However, that is an external-validity or confound concern, not circularity under any of the enumerated patterns: no equation defines a predicted quantity in terms of the fitted reward values, no central premise depends on a self-citation, and no known result is merely renamed. Therefore the appropriate circularity score is 0.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 1 invented entities

The method introduces several hand-chosen hyperparameters (reward schedule, step counts, reward magnitudes) and relies on assumptions about data quality and transfer that are load-bearing for the curriculum. The control tags are novel format entities, but they are internal to the method.

free parameters (6)
  • Alternating reward window W = 10 steps
    Chosen by hand in Section 3.5.2; the schedule alternates ACC/PAR rewards every W steps. Central to the reward design comparison.
  • Alternating reward split (80/20) = 80% ACC, 20% PAR
    Hand-chosen schedule in Section 3.5.2 to balance accuracy and parallel structure during stage-two RL.
  • Tiered reward values = +1.2 / +1.0 / -1.0
    Hand-chosen reward magnitudes in Section 3.5.2 for the alternating scheme; these values calibrate the incentive for parallel usage.
  • Stage 1 RL gradient steps = 35
    GRPO on GSM8K for five epochs, resulting in 35 gradient updates (Section 4.1).
  • Stage 2 RL gradient steps = 300
    GRPO on DAPO for 300 gradient updates (Section 4.1).
  • Number of parallel paths N
    Not specified in the paper. The inference workflow states N independent trajectories are generated, but the value of N is never given, affecting reproducibility.
axioms (5)
  • standard math Standard RLVR/GRPO framework provides a reliable training signal for reasoning quality
    The paper relies on GRPO (Shao et al., 2024) and verifiable rewards for training; Section 3.4.1.
  • domain assumption DeepSeek-R1-0528-Qwen-3-8B can produce valid parallel-thinking traces on easy GSM8K problems via zero-shot prompting
    Key Finding 1 (Section 3.3, Table 1): 83.7% format validity on GSM8K vs 0.0% on DAPO. The cold-start dataset depends on this.
  • ad hoc to paper Parallel thinking format learned on easy math transfers to hard math via RL
    The progressive curriculum assumes the format instilled on GSM8K generalizes to DAPO and benchmarks through RL (Section 3.4, ablation Table 3).
  • domain assumption The <Parallel> tag can be learned and reliably triggered at inference for parallel decoding
    The inference workflow (Section 3.2) depends on the model emitting the special tag; the token was absent from pretraining (Section 3.4.2).
  • domain assumption Benchmark rewards (AIME, AMC, MATH) are reliable proxies for reasoning ability
    Evaluation uses these benchmarks as the measure of success (Section 4.1).
invented entities (1)
  • <Parallel>, <Path>, <Summary> control tags no independent evidence
    purpose: Structured markers to trigger and delimit parallel thinking blocks during training and inference.
    Introduced by the authors (Section 3.2); no external falsifiable handle exists for these format tokens.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Parallel-R1: Towards Parallel Thinking via Reinforcement Learning." pith.science (2026). https://pith.science/paper/VUMJ4IFS

@misc{pith2026250907980,
  author       = {Pith},
  title        = {Pith review of: Parallel-R1: Towards Parallel Thinking via Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VUMJ4IFS}},
  note         = {Machine review of arXiv:2509.07980}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Parallel thinking has emerged as a novel approach for enhancing the reasoning capabilities of large language models (LLMs) by exploring multiple reasoning paths concurrently. However, activating such capabilities through training remains challenging, as existing methods predominantly rely on supervised fine-tuning (SFT) over synthetic data, which encourages teacher-forced imitation rather than exploration and generalization. Different from them, we propose \textbf{Parallel-R1}, the first reinforcement learning (RL) framework that enables parallel thinking behaviors for complex real-world reasoning tasks. Our framework employs a progressive curriculum that explicitly addresses the cold-start problem in training parallel thinking with RL. We first use SFT on prompt-generated trajectories from easier tasks to instill the parallel thinking ability, then transition to RL to explore and generalize this skill on harder problems. Experiments on various math benchmarks, including MATH, AMC23, and AIME, show that Parallel-R1 successfully instills parallel thinking, leading to 8.4% accuracy improvements over the sequential thinking model trained directly on challenging tasks with RL. Further analysis reveals a clear shift in the model's thinking behavior: at an early stage, it uses parallel thinking as an exploration strategy, while in a later stage, it uses the same capability for multi-perspective verification. Most significantly, we validate parallel thinking as a \textbf{mid-training exploration scaffold}, where this temporary exploratory phase unlocks a higher performance ceiling after RL, yielding a 42.9% improvement over the baseline on AIME25. Our model, data, and code will be open-source at https://github.com/zhengkid/Parallel-R1.

Figures

Figures reproduced from arXiv: 2509.07980 by Chengsong Huang, Dong Yu, Heng Huang, Hongming Zhang, Huiwen Bao, Rui Liu, Runpeng Dai, Tong Zheng, Wenhao Yu, Xiaoyang Wang.

Figure 1
Figure 1. Figure 1: An overview of the proposed framework. (Top) During inference, the model generates in a [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of the structured atten￾tion mask and position IDs, where different paths and the summary block have distinct visibility regions. Blank regions indicate to￾kens that cannot attend to each other, while colored regions indicate tokens that can. To explore an alternative solution, we introduce a struc￾tured variant of our framework, Parallel-Unseen. This model incorporates explicit inductive bias… view at source ↗
Figure 3
Figure 3. Figure 3: Dynamics of the relative position of the [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Two-stage training with parallel reasoning as a mid-training exploration scaffold. Left axis [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: A qualitative case study showcases that our model uses parallel exploration of two distinct [PITH_FULL_IMAGE:figures/full_fig_p019_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: A qualitative case study demonstrates late-stage verification through parallel thinking. [PITH_FULL_IMAGE:figures/full_fig_p020_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 18 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Regulating Branch Parallelism in LLM Serving

    cs.DC 2026-05 unverdicted novelty 7.0

    TAPER regulates LLM branch parallelism by admitting extra branches opportunistically when predicted externality fits slack, delivering 1.48-1.77x higher goodput than eager or fixed-cap baselines on Qwen3-32B while kee...

  2. On the Overscaling Curse of Parallel Thinking: System Efficacy Contradicts Sample Efficiency

    cs.LG 2026-01 unverdicted novelty 7.0

    Parallel thinking in LLMs suffers from overscaling where fixed global budgets waste samples; LanBo predicts per-sample budgets from latent states to raise utilization without hurting accuracy.

  3. LangDriveCTRL: Natural Language Controllable Driving Scene Editing with Multi-modal Agents

    cs.CV 2025-12 unverdicted novelty 7.0

    LangDriveCTRL decomposes driving videos into 3D scene graphs and uses an agentic pipeline with specialized multi-modal agents to perform language-controlled object and behavior edits, achieving nearly 2x higher instru...

  4. Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning

    cs.CL 2026-07 unverdicted novelty 6.0

    Scaling zero-RL training to a 1T-parameter model yields competitive math performance and spontaneous cognitive behaviors such as self-verification and structured formatting.

  5. Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning

    cs.CL 2026-07 conditional novelty 6.0

    A trillion-parameter model trained purely on reward feedback reaches competitive math scores and produces structured, self-verifying reasoning traces, but the claim that these behaviors emerge without hand-crafted rul...

  6. Small RL Controller, Large Language Model: RL-Guided Adaptive Sampling for Test-Time Scaling

    cs.CL 2026-06 unverdicted novelty 6.0

    RL-trained lightweight controller using answer statistics improves trade-offs among correctness, latency, and total samples in adaptive sampling for LLM test-time scaling.

  7. DeltaRubric: Generative Multimodal Reward Modeling via Joint Planning and Verification

    cs.CL 2026-05 unverdicted novelty 6.0

    DeltaRubric decomposes multimodal preference evaluation into self-generated planning and verification steps within a single model, producing large accuracy improvements on VL-RewardBench via multi-role reinforcement learning.

  8. Reinforcing Multimodal Reasoning Against Visual Degradation

    cs.CV 2026-05 unverdicted novelty 6.0

    ROMA improves MLLM robustness to seen and unseen visual corruptions by +2.3-2.4% over GRPO on seven reasoning benchmarks while matching clean accuracy.

  9. The Scaling Properties of Implicit Deductive Reasoning in Transformers

    cs.AI 2026-05 conditional novelty 6.0

    With shortcut-decorrelated training, bidirectional masking, and a corrective objective, depth scaling makes direct transformer reasoning over Horn clauses match CoT accuracy within the training horizon, but not beyond it.

  10. Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data

    cs.LG 2026-04 unverdicted novelty 6.0

    A parameter-free sampling strategy called CUTS combined with Mixed-CUTS training prevents mode collapse in RL for saturated LLM reasoning tasks and raises AIME25 Pass@1 accuracy by up to 15.1% over standard GRPO.

  11. LACE: Lattice Attention for Cross-thread Exploration

    cs.AI 2026-04 unverdicted novelty 6.0

    LACE enables parallel reasoning paths in LLMs to communicate via lattice attention and error-correct using synthetic training data, improving accuracy by over 7 points over standard parallel search.

  12. Native Parallel Reasoner: Reasoning in Parallelism via Self-Distilled Reinforcement Learning

    cs.CL 2025-12 unverdicted novelty 6.0

    NPR trains LLMs to reason in parallel via self-distilled RL, delivering up to 24.5% performance gains and 4.6x speedups with 100% genuine parallel execution on reasoning benchmarks.

  13. StatEval: A Comprehensive Benchmark for Large Language Models in Statistics

    cs.CL 2025-10 conditional novelty 6.0

    StatEval is a new 16,000-question statistics benchmark showing that even strong LLMs score below 60% on research-level statistical proof tasks.

  14. Efficient Reasoning on the Edge

    cs.LG 2026-03 accept novelty 5.5

    LoRA adapters, budget-forced GRPO, dynamic switching, parallel verification and FPTQuant enable practical chain-of-thought reasoning on quantized Qwen2.5-7B for edge devices.

  15. The Scaling Properties of Implicit Deductive Reasoning in Transformers

    cs.AI 2026-05 unverdicted novelty 5.0

    In deep Transformers using bidirectional prefix masks, implicit reasoning on Horn clauses matches explicit CoT performance across topologies and widths, but CoT is still required for depth extrapolation.

  16. LACE: Lattice Attention for Cross-thread Exploration

    cs.AI 2026-04 unverdicted novelty 5.0

    LACE enables concurrent reasoning paths in LLMs to interact via lattice attention and a synthetic training pipeline, raising accuracy more than 7 points over independent parallel search.

  17. LACE: Lattice Attention for Cross-thread Exploration

    cs.AI 2026-04 unverdicted novelty 5.0

    LACE adds lattice attention to let parallel LLM reasoning threads interact and correct errors, raising accuracy over 7 points versus standard independent sampling.

  18. Understanding Performance Gap Between Parallel and Sequential Sampling in Large Reasoning Models

    cs.CL 2026-04 unverdicted novelty 5.0

    Lack of exploration from conditioning on prior answers is the primary reason parallel sampling outperforms sequential sampling in large reasoning models.

Reference graph

Works this paper leans on

27 extracted references · 2 canonical work pages · cited by 14 Pith papers · 1 internal anchor

  1. [1]

    Large language monkeys: Scaling inference compute with repeated sampling.arXiv preprint arXiv:2407.21787,

    Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher R´e, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling.arXiv preprint arXiv:2407.21787,

    Pith/arXiv arXiv

  2. [5]

    Measuring mathematical problem solving with the math dataset.arXiv preprint arXiv:2103.03874,

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset.arXiv preprint arXiv:2103.03874,

    Pith/arXiv arXiv

  3. [7]

    Divide, Reweight, and Conquer: A Logit Arithmetic Approach for In-Context Learning

    Chengsong Huang, Langlin Huang, and Jiaxin Huang. Divide, reweight, and conquer: A logit arithmetic approach for in-context learning.arXiv preprint arXiv:2410.10074,

    work page internal anchor Pith review Pith/arXiv arXiv

  4. [8]

    Efficient test-time scaling via self-calibration.arXiv preprint arXiv:2503.00031, 2025a

    Chengsong Huang, Langlin Huang, Jixuan Leng, Jiacheng Liu, and Jiaxin Huang. Efficient test-time scaling via self-calibration.arXiv preprint arXiv:2503.00031, 2025a. Chengsong Huang, Wenhao Yu, Xiaoyang Wang, Hongming Zhang, Zongxia Li, Ruosen Li, Jiaxin Huang, Haitao Mi, and Dong Yu. R-zero: Self-evolving reasoning llm from zero data.arXiv preprint arXiv...

    Pith/arXiv arXiv

  5. [10]

    Self-rewarding vision-language model via reasoning decomposition.arXiv preprint arXiv:2508.19652,

    Zongxia Li, Wenhao Yu, Chengsong Huang, Rui Liu, Zhenwen Liang, Fuxiao Liu, Jingxi Che, Dian Yu, Jordan Boyd-Graber, Haitao Mi, et al. Self-rewarding vision-language model via reasoning decomposition.arXiv preprint arXiv:2508.19652,

    Pith/arXiv arXiv

  6. [11]

    Spiral: Self-play on zero-sum games incentivizes reasoning via multi-agent multi-turn reinforcement learning.arXiv preprint arXiv:2506.24119,

    Bo Liu, Leon Guertler, Simon Yu, Zichen Liu, Penghui Qi, Daniel Balcells, Mickel Liu, Cheston Tan, Weiyan Shi, Min Lin, et al. Spiral: Self-play on zero-sum games incentivizes reasoning via multi-agent multi-turn reinforcement learning.arXiv preprint arXiv:2506.24119,

    arXiv

  7. [12]

    Matthew Macfarlane, Minseon Kim, Nebojsa Jojic, Weijia Xu, Lucas Caccia, Xingdi Yuan, Wanru Zhao, Zhengyan Shi, and Alessandro Sordoni

    Accessed: 2025-07-30. Matthew Macfarlane, Minseon Kim, Nebojsa Jojic, Weijia Xu, Lucas Caccia, Xingdi Yuan, Wanru Zhao, Zhengyan Shi, and Alessandro Sordoni. Instilling parallel reasoning into language models. In2nd AI for Math Workshop @ ICML 2025,

    work page 2025

  8. [13]

    Jiayi Pan, Xiuyu Li, Long Lian, Charlie Snell, Yifei Zhou, Adam Yala, Trevor Darrell, Kurt Keutzer, and Alane Suhr

    URL https://openreview.net/forum?id=a3o4 b3hkwp. Jiayi Pan, Xiuyu Li, Long Lian, Charlie Snell, Yifei Zhou, Adam Yala, Trevor Darrell, Kurt Keutzer, and Alane Suhr. Learning adaptive parallel reasoning with language models.arXiv preprint arXiv:2504.15466,

    Pith/arXiv arXiv

  9. [14]

    Hogwild! inference: Parallel llm generation via concurrent attention.arXiv preprint arXiv:2504.06261,

    Gleb Rodionov, Roman Garipov, Alina Shutova, George Yakushev, Erik Schultheis, Vage Egiazarian, Anton Sinitsin, Denis Kuznedelev, and Dan Alistarh. Hogwild! inference: Parallel llm generation via concurrent attention.arXiv preprint arXiv:2504.06261,

    arXiv

  10. [15]

    Adversarial reasoning at jailbreaking time.arXiv preprint arXiv:2502.01633,

    Mahdi Sabbaghi, Paul Kassianik, George Pappas, Yaron Singer, Amin Karbasi, and Hamed Hassani. Adversarial reasoning at jailbreaking time.arXiv preprint arXiv:2502.01633,

    Pith/arXiv arXiv

  11. [16]

    Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models.arXiv preprint arXiv:2402.03300,

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models.arXiv preprint arXiv:2402.03300,

    Pith/arXiv arXiv

  12. [17]

    Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256,

    14 Technical Report Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256,

    Pith/arXiv arXiv

  13. [18]

    Mobilegui-rl: Advancing mobile gui agent through reinforcement learning in online environment.arXiv preprint arXiv:2507.05720,

    Yucheng Shi, Wenhao Yu, Zaitang Li, Yonglin Wang, Hongming Zhang, Ninghao Liu, Haitao Mi, and Dong Yu. Mobilegui-rl: Advancing mobile gui agent through reinforcement learning in online environment.arXiv preprint arXiv:2507.05720,

    Pith/arXiv arXiv

  14. [19]

    On the hardness of faithful chain-of-thought reasoning in large language models.arXiv preprint arXiv:2406.10625,

    Sree Harsha Tanneru, Dan Ley, Chirag Agarwal, and Himabindu Lakkaraju. On the hardness of faithful chain-of-thought reasoning in large language models.arXiv preprint arXiv:2406.10625,

    Pith/arXiv arXiv

  15. [20]

    To code or not to code? adaptive tool integration for math language models via expectation-maximization

    Haozhe Wang, Long Li, Chao Qu, Fengming Zhu, Weidi Xu, Wei Chu, and Fangzhen Lin. To code or not to code? adaptive tool integration for math language models via expectation-maximization. arXiv preprint arXiv:2502.00691, 2025a. Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. Vl-rethinker: Incentivizing self-reflection of vision-l...

    Pith/arXiv arXiv

  16. [23]

    Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks.arXiv preprint arXiv:2504.05118,

    Yu Yue, Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, TianTian Fan, Zhengyin Du, et al. Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks.arXiv preprint arXiv:2504.05118,

    Pith/arXiv arXiv

  17. [24]

    Accessing gpt-4 level mathematical olympiad solutions via monte carlo tree self-refine with llama-3 8b.arXiv preprint arXiv:2406.07394,

    Di Zhang, Xiaoshui Huang, Dongzhan Zhou, Yuqiang Li, and Wanli Ouyang. Accessing gpt-4 level mathematical olympiad solutions via monte carlo tree self-refine with llama-3 8b.arXiv preprint arXiv:2406.07394,

    Pith/arXiv arXiv

  18. [25]

    Learning to reason via mixture-of-thought for logical reasoning.arXiv preprint arXiv:2505.15817,

    Tong Zheng, Lichang Chen, Simeng Han, R Thomas McCoy, and Heng Huang. Learning to reason via mixture-of-thought for logical reasoning.arXiv preprint arXiv:2505.15817,

    Pith/arXiv arXiv

  19. [26]

    Dissecting logical reasoning in llms: A fine-grained evaluation and supervision study.arXiv preprint arXiv:2506.04810,

    Yujun Zhou, Jiayi Ye, Zipeng Ling, Yufei Han, Yue Huang, Haomin Zhuang, Zhenwen Liang, Kehan Guo, Taicheng Guo, Xiangqi Wang, et al. Dissecting logical reasoning in llms: A fine-grained evaluation and supervision study.arXiv preprint arXiv:2506.04810,

    arXiv

  20. [27]

    Ttrl: Test-time reinforcement learning.arXiv preprint arXiv:2504.16084,

    15 Technical Report Yuxin Zuo, Kaiyan Zhang, Li Sheng, Shang Qu, Ganqu Cui, Xuekai Zhu, Haozhan Li, Yuchen Zhang, Xinwei Long, Ermo Hua, et al. Ttrl: Test-time reinforcement learning.arXiv preprint arXiv:2504.16084,

    Pith/arXiv arXiv

  21. [1989]

    Breach in the shield: Unveiling the vulnera- bilities of large language models.arXiv preprint arXiv:2504.03714, 2025a

    Runpeng Dai, Run Yang, Fan Zhou, and Hongtu Zhu. Breach in the shield: Unveiling the vulnera- bilities of large language models.arXiv preprint arXiv:2504.03714, 2025a. Runpeng Dai, Tong Zheng, Run Yang, Kaixian Yu, and Hongtu Zhu. R1-re: Cross-domain relation extraction with rlvr.arXiv preprint arXiv:2507.04642, 2025b. Yichao Fu, Xuewei Wang, Yuandong Tia...

    Pith/arXiv arXiv

  22. [2011]

    Learning to keep a promise: Scal- ing language model decoding parallelism with learned asynchronous decoding.arXiv preprint arXiv:2502.11517,

    Tian Jin, Ellie Y Cheng, Zack Ankner, Nikunj Saunshi, Blake M Elias, Amir Yazdanbakhsh, Jonathan Ragan-Kelley, Suvinay Subramanian, and Michael Carbin. Learning to keep a promise: Scal- ing language model decoding parallelism with learned asynchronous decoding.arXiv preprint arXiv:2502.11517,

    Pith/arXiv arXiv

  23. [2021]

    Group think: Multiple concurrent reasoning agents collaborating at token level granularity.arXiv preprint arXiv:2505.11107,

    13 Technical Report Chan-Jan Hsu, Davide Buffelli, Jamie McGowan, Feng-Ting Liao, Yi-Chang Chen, Sattar Vakili, and Da-shan Shiu. Group think: Multiple concurrent reasoning agents collaborating at token level granularity.arXiv preprint arXiv:2505.11107,

    Pith/arXiv arXiv

  24. [2022]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025a

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025a. Xinyu Yang, Yuwei An, Hongyi Liu, Tianqi Chen, and Beidi Chen. Multiverse: Your language models secretly decide how to parallelize and merge generation.arXiv preprint a...

    Pith/arXiv arXiv

  25. [2023]

    Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476,

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476,

    Pith/arXiv arXiv

  26. [2024]

    Aspd: Unlocking adaptive serial-parallel decoding by exploring intrinsic parallelism in llms

    Keyu Chen, Zhifeng Shen, Daohai Yu, Haoqian Wu, Wei Wen, Jianfeng He, Ruizhi Qiao, and Xing Sun. Aspd: Unlocking adaptive serial-parallel decoding by exploring intrinsic parallelism in llms. arXiv preprint arXiv:2508.08895, 2025a. Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha W...

    Pith/arXiv arXiv

  27. [2025]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

    Pith/arXiv arXiv

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.