DialogueReason: Rule-Based RL Sparks Dialogue Reasoning in LLMs

Chen Hu; Daxin Jiang; Shuchang Zhou; Xin Wu; Yubo Shu; Zhewei Huang

REVIEW 5 major objections 5 minor 1 cited by

DialogueReason: Rule-Based RL Sparks Dialogue Reasoning in LLMs

T0 review · 5 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that dialogue-based reasoning, trained with rule-based reinforcement learning, outperforms monologue-style reasoning once a prompt contains multiple questions.

desk verdict RL-trained dialogue reasoning is a genuinely new idea, but the headline gains are confounded by unmatched inference compute and an absent answer-all control. read the letter →

arxiv 2505.07049 v1 pith:IM5PJETN submitted 2025-05-11 cs.AI

Yubo Shu , Zhewei Huang , Xin Wu , Chen Hu , Shuchang Zhou , Daxin Jiang This is my paper · ORCID

classification cs.AI

keywords dialoguereasoningreinforcementlearningchain-of-thoughtdiversitycoherencycompoundQAPPOrule-basedreward

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the standard monologue-style chain-of-thought used by large reasoning models has a specific weakness: when several problems must be solved in one prompt, the model either recycles one strategy, jumps between problems without depth, or abandons all but the first. To expose this, the authors introduce Compound-QA, which concatenates multiple independently solvable questions into a single prompt; accuracy falls steadily as the number of sub-questions grows. The paper then claims that a dialogue-based reasoning pattern, trained with a reinforcement-learning algorithm and rule-based rewards, defends against this decline. On compound versions of MATH, AIME24, and GPQA, the dialogue-trained model falls slightly below the monologue model on single questions but overtakes it once the compound factor exceeds about three, with the largest gaps on AIME24. If the claim holds, dialogue structure itself—not just more tokens—is what lets a reasoning model keep exploring diverse strategies while staying coherent.

What carries the argument

The load-bearing object is the DialogueReason training recipe: a system prompt that instructs the model to write an ultra-detailed play in which experts matched to the topic solve the problem through dialogue, wrapped in <play> and <answer> tags, combined with PPO using rule-based answer-matching rewards. The Compound-QA task is the probe that makes the effect visible, concatenating k sub-questions and measuring the fraction of correct answers. Dialogue structure supplies the mechanism: each new sub-question triggers a fresh scene and fresh characters, forcing a strategy switch, while turn-taking and explicit character boundaries keep the model from drifting between problems.

What would settle it

Run a controlled comparison on Compound-QA with two variants: homogeneous compounds whose sub-questions all need the same strategy, and heterogeneous compounds that require switching. If DialogueReason's advantage disappears on the heterogeneous set once the monologue baseline is given the same per-question token budget, or if it persists on homogeneous sets where no strategy switching is needed, the claimed diversity mechanism is not what explains the numbers.

Watch

Extended reading notes

Core claim

The central discovery is that a large language model can be trained by reinforcement learning to reason as a multi-character dialogue and that this changes its robustness on multi-question inputs. Starting from QWQ-32B or the base Qwen2.5-32B model, the authors run PPO with a reward that only checks exact answer matches, while the system prompt asks the model to write a playscript in which named expert characters discuss each sub-question, reach consensus, and then output all answers. The resulting DialogueReason model is compared with the monologue QWQ model on Compound-QA versions of MATH-500, AIME24, and GPQA-Diamond. At compound factor cbK=1 the monologue model is slightly better, but from about cbK=3 upward the dialogue model retains accuracy better; for example, on AIME24 at cbK=8 it scores 50.0% versus 35.09% for the monologue model. The authors interpret this as evidence that explicit role boundaries and scene changes give the model both diversity across sub-problems and coherency within each reasoning thread.

Load-bearing premise

The argument stands on the assumption that accuracy on Compound-QA, where independent questions are glued into one prompt, is a valid proxy for the diversity and coherency that matter in real complex reasoning, and that the single accuracy score can still distinguish those two qualities.

Editorial extensions

If this is right

At compound factors above three, DialogueReason-trained models retain more accuracy than their monologue counterparts on MATH-500, AIME24, and GPQA-Diamond; on AIME24 at cbK=8 the gap is about 15 percentage points.
The reasoning traces become interpretable as structured scenes with named experts, making it easier for a human reader to see where and why the model changes strategy.
A base model, not just a reasoning-specialized model, can acquire dialogue reasoning through rule-based RL, suggesting the pattern does not require supervised dialogue data.
On out-of-distribution open-ended prompts, the dialogue-trained model spontaneously assigns different viewpoints to different characters and carries the comparison further than the monologue model.
The same end-to-end RL training of role configuration and interaction could transfer to multi-agent systems, removing hand-coded agent workflows.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because Compound-QA conflates diversity and coherency in a single accuracy number, the observed advantage may be mostly about coherency—preventing attention drift—rather than about generating genuinely diverse strategies; a benchmark that separates the two would settle this.
A simpler alternative explanation is that the dialogue format acts as structured section delimiters, and any formatting that isolates sub-problems, such as explicit headings per question, might reproduce part of the gain; this is directly testable.
The Compound-QA construction itself could be turned into a training objective, so models learn to decompose compound or multi-hop inputs into separately verified threads, which is a natural extension the paper flags for future work.
If the dialogue benefit comes from scene resetting, then compound questions whose sub-problems share the same domain or require the same strategy should shrink the advantage, predicting an interaction between domain homogeneity and dialogue formatting.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Candid take: this paper has a genuinely new idea — training a model via rule-based RL to adopt a dialogue/play-like reasoning pattern — and a clean probe (Compound-QA) that shows monologue models degrade when multiple questions are concatenated. The released Qwen-32B-DialogueReason model is a real artifact, and the consistent accuracy advantage at higher compound factors is worth explaining. The authors also deserve credit for acknowledging in Section 5 that the Compound-QA metric conflates diversity and coherency.

The soft spot is serious. The headline comparison QWQ vs DialogueReason is not controlled for inference compute. Table 1 reports token counts only for QWQ; Table 2 never reports DialogueReason token counts. The system prompt in Figure 4 asks for a full multi-character play with scene transitions and per-question summaries, so DialogueReason almost certainly uses significantly more tokens per subquestion. If so, the 50% vs 35% gap on AIME cbK8 could be mostly test-time compute rather than a structural advantage of dialogue. Second, no monologue baseline is asked to explicitly answer every subquestion. Figure 3b shows QWQ often ignores later questions; a simple 'answer all' instruction might close much of the gap. The absence of error bars and an undefined 'Overall' metric in Table 2 are minor by comparison, and the MATH/MATH-500 overlap is a mild concern for absolute numbers but not for the relative comparison.

None of this kills the paper. The training recipe, the Compound-QA probe, and the qualitative traces are a useful package for the RL-reasoning community. But the central mechanism claim — dialogue structure improves diversity/coherency — is not yet supported by the controlled evidence. A revision that adds token-matched baselines, an answer-all monologue control, and per-seed variance would make the case much stronger.

My recommendation: yes, send this to peer review, but with a clear request for compute-matched baselines and variance reporting. The idea deserves referee time; the current comparison just is not tight enough to accept as-is.

Referee Report

5 major / 5 minor

Summary. The paper proposes DialogueReason, a dialogue-based reasoning paradigm in which a language model simulates multiple expert characters interacting to solve each subquestion of a compound prompt. The authors introduce the Compound-QA task, which concatenates k independently solvable questions into one input, and argue that accuracy on this task reveals reasoning diversity and coherency. Using PPO with a rule-based answer-matching reward, they train Qwen-QWQ-32B and Qwen2.5-Base-32B to adopt this dialogue pattern. Experiments on MATH-500, AIME24, and GPQA-Diamond show that DialogueReason degrades less than the monologue QWQ baseline as the compound factor k grows, with notable gains at cbK=8 on AIME24 (50.0% vs. 35.09%). The paper also presents qualitative case studies and discusses interpretability, controllability, and multi-agent implications. Section 5 acknowledges that the Compound-QA evaluation conflates diversity and coherency and that the current scope is limited to math and science tasks.

Significance. If the central claim holds, the paper makes a useful contribution by showing that RL can induce a structured dialogue-based reasoning pattern that is more robust to multi-question interference than standard monologue reasoning. The strengths are the formal formulation of Compound-QA as a controllable difficulty probe, the open release of the trained model, the use of rule-based rewards without fitted free parameters, and the concrete quantitative comparisons in Table 2 and Figure 5. However, the attribution of the performance gap to dialogue-based reasoning rather than to uncontrolled factors such as extra test-time compute and output-format compliance is not yet established, and the paper's own limitation statement undercuts the diversity/coherency mechanism claim. The contribution is potentially significant but currently under-supported.

major comments (5)

[Section 3.2, Table 2] The headline comparison does not control for test-time compute. Table 1 reports average CoT lengths for QWQ only, and Table 2 does not report token counts for DialogueReason. The Figure 4 system prompt instructs the model to write a full multi-character play with scene transitions and a summary for each subquestion, which is likely to consume substantially more tokens per subquestion than QWQ's monologue. For example, at cbK8 on AIME24, QWQ uses about 54,375 tokens for eight questions, roughly 6.8k per question. If DialogueReason uses more tokens per subquestion, the reported 50.0% versus 35.09% gap may reflect additional inference compute rather than improved diversity or coherency. Please report generation lengths for both models and include a compute-matched control, such as a monologue baseline with a comparable token budget.
[Section 2.4 and Section 3.2, Figure 3b] Output-format compliance is a confound. Figure 3b shows that QWQ frequently ignores later subquestions entirely and answers only the first question, which mechanically lowers its aggregate accuracy. DialogueReason's prompt and answer block naturally force a per-question response ('Here are the answers to each of the questions'), so the aggregate accuracy gain may reflect instruction following rather than improved reasoning diversity or coherency. No monologue baseline with an explicit 'answer every subquestion' instruction is reported, so the current comparison cannot separate the effect of output-format encouragement from the effect of the dialogue structure.
[Section 5, Limitations] The paper itself states that the Compound-QA evaluation is 'entangled' and that it conflates diversity and coherency, but the abstract and Section 1 claim that DialogueReason demonstrates 'superior diversity and coherency.' The quantitative experiments measure only aggregate answer accuracy and do not provide a direct metric for either diversity or coherency. Consequently, the mechanism claim (that dialogue improves diversity and coherency specifically) is inferred rather than established, and the load-bearing attribution from the accuracy gain to these two constructs is not supported by the reported measurements.
[Table 2 and Section 2.3] The 'Overall' row in Table 2 is never defined. It is not clear whether it is an unweighted average over the three datasets, an average weighted by sample size, or some other aggregate, and without this definition the headline claim of 'overall' improvement is not interpretable. In addition, the paper does not report confidence intervals or significance tests for the accuracies in Table 2; the text mentions 16 inference runs for Table 1, but no variance information is given for Table 2, so the observed differences at several cbK values may not be statistically reliable.
[Section 3.1, training data] There is a potential train-evaluation overlap. The training data include Open-Reasoner-Zero with AIME (up to 2023), OpenR1-Math-220k, and Tulu3 MATH, while the evaluation sets include MATH-500 and AIME24. MATH-500 is a subset of MATH, and MATH problems appear in the OpenR1-Math-220k and Tulu3 MATH sources. Because both QWQ and DialogueReason are evaluated on the same sets, this overlap does not invalidate the relative comparison, but it could inflate the absolute MATH-500 numbers for both models, and it should be explicitly disclosed or quantified.

minor comments (5)

[General formatting] Section cross-references are inconsistent, with 'Sec 3.2' appearing in Section 3.1 and 'Sec 3.2' used elsewhere; use a uniform citation style.
[Figure 3c] The quoted reasoning trace contains non-English fragments such as '其' and '答案为' mixed into the English text; this should be cleaned or explicitly marked as a translated excerpt.
[Section 3.1, PPO objective] The displayed PPO objective is a standard clipped surrogate, but the notation is not fully aligned with Schulman et al. and the advantage estimator definition omits the discount factor gamma; please align notation or add a clarifying sentence.
[Table 2 caption] The caption does not define 'cbK' or describe how compound questions are constructed, selected, and ordered for each dataset, and whether the same compound inputs were used for both models; this information is needed for replication.
[Figure 5] The caption labels subplots (a) through (d), but the text refers to them as 'Overall,' 'MATH-500,' 'AIME24,' and 'GPQA-Diamond'; make the correspondence explicit in the figure caption.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DialogueReason's Compound-QA gains are an empirical comparison, not a construction from the evaluation metric.

full rationale

The claimed derivation is an RL training intervention followed by benchmark evaluation; it does not fit parameters to the Compound-QA result. The PPO reward is rule-based exact-answer matching on individual training questions (Section 3.1), while the Compound-QA accuracy is an aggregate per-subquestion match on distinct evaluation sets. No equation in the paper defines the dialogue prompt or the trained policy in terms of the Compound-QA outcome, and no fitted constant is renamed as a prediction. The only self-citation (Open-Reasoner-Zero, Hu et al. 2025, with a coauthor overlap) is used as an open-source training recipe and dataset source, not as a uniqueness theorem or as justification for the central claim. The paper's own Section 5 flags that Compound-QA conflates diversity and coherency, which limits mechanistic interpretation, and the missing DialogueReason token counts and absence of an answer-all monologue control are validity threats, but these are empirical confounds rather than circular reductions. The MATH-500/OpenR1-Math overlap is a possible data-contamination concern that does not make the headline comparison circular, since the same trend is reported on AIME24 and GPQA-Diamond.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters were fit. The central claim relies on assumptions about reward validity, Compound-QA as a proxy, and dataset cleanliness.

assumptions (3)

domain assumption Rule-based reward via exact answer matching provides a sufficient learning signal for reasoning patterns.
The PPO training uses result matching on ORZ datasets; no learned reward model, so correctness of this reward assumption affects all trained models.
domain assumption Compound-QA accuracy is a valid proxy for reasoning diversity and coherency.
The paper defines Compound-QA to jointly stress diversity and coherency, but Section 5 admits these dimensions are entangled and not measured separately.
domain assumption The evaluation datasets (MATH-500, AIME24, GPQA-Diamond) are not contaminated by ORZ training data.
Training uses Open-R1-Math-220k, Tulu3 MATH, and AIME up to 2023; MATH-500 is a subset of MATH and could overlap with training data, which could inflate absolute accuracy.

how reviews work

0 comments

Cite this review

Pith. "Pith review of DialogueReason: Rule-Based RL Sparks Dialogue Reasoning in LLMs." pith.science (2026). https://pith.science/paper/IM5PJETN

@misc{pith2026250507049,
  author       = {Pith},
  title        = {Pith review of: DialogueReason: Rule-Based RL Sparks Dialogue Reasoning in LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IM5PJETN}},
  note         = {Machine review of arXiv:2505.07049}
}

read the original abstract

We propose DialogueReason, a reasoning paradigm that uncovers the lost roles in monologue-style reasoning models, aiming to boost diversity and coherency of the reasoning process. Recent advances in RL-based large reasoning models have led to impressive long CoT capabilities and high performance on math and science benchmarks. However, these reasoning models rely mainly on monologue-style reasoning, which often limits reasoning diversity and coherency, frequently recycling fixed strategies or exhibiting unnecessary shifts in attention. Our work consists of an analysis of monologue reasoning patterns and the development of a dialogue-based reasoning approach. We first introduce the Compound-QA task, which concatenates multiple problems into a single prompt to assess both diversity and coherency of reasoning. Our analysis shows that Compound-QA exposes weaknesses in monologue reasoning, evidenced by both quantitative metrics and qualitative reasoning traces. Building on the analysis, we propose a dialogue-based reasoning, named DialogueReason, structured around agents, environment, and interactions. Using PPO with rule-based rewards, we train open-source LLMs (Qwen-QWQ and Qwen-Base) to adopt dialogue reasoning. We evaluate trained models on MATH, AIME, and GPQA datasets, showing that the dialogue reasoning model outperforms monologue models under more complex compound questions. Additionally, we discuss how dialogue-based reasoning helps enhance interpretability, facilitate more intuitive human interaction, and inspire advances in multi-agent system design.

Figures

Figures reproduced from arXiv: 2505.07049 by the authors.

**Figure 2.** Illustration of the Compound-QA task, which involves reasoning over multiple sub-questions concatenated [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Three representative failure made by monologue reasoning models when tackling compound questions. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: The illustration of the dialogue-based reasoning pattern. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: The four subplots correspond to Overall (a), MATH [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

**Figure 6.** Figure 6: A case study of the dialogue reasoning on a compound question from the GPQA dataset. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

**Figure 7.** Figure 7: The case of monologue vs. dialogue reasoning on the out-of-distribution question. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Context-Aware Multi-Turn Visual-Textual Reasoning in LVLMs via Dynamic Memory and Adaptive Visual Guidance
cs.CV 2025-09 reject novelty 3.0 of 10

The proposed CAMVR framework is not supported by verifiable evidence, and the manuscript itself labels its experimental results as fabricated.

Reference graph

Works this paper leans on

14 extracted references · 3 canonical work pages · cited by 1 Pith paper

[4]

InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8055–8068

Dialcot meets ppo: Decomposing and exploring reasoning paths in smaller language models. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8055–8068. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Stein- hardt

work page 2023
[6]

Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others

Open- reasoner-zero: An open source approach to scaling up reinforcement learning on the base model.arXiv preprint arXiv:2503.24290. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others

arXiv
[7]

Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen

Openai o1 system card.arXiv preprint arXiv:2412.16720. Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen

arXiv
[8]

Charles M Macal and Michael J North

Understanding r1-zero-like training: A critical perspective.arXiv preprint arXiv:2503.20783. Charles M Macal and Michael J North

arXiv
[11]

arXiv preprint arXiv:2501.18585

Thoughts are all over the place: On the underthinking of o1-like llms. arXiv preprint arXiv:2501.18585. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others

arXiv
[12]

Thinkpatterns-21k: A systematic study on the impact of thinking patterns in llms.arXiv preprint arXiv:2503.12918. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, and 23 others

arXiv
[13]

Ling Yang, Zhaochen Yu, Bin Cui, and Mengdi Wang

Qwen2.5 technical report.arXiv preprint arXiv:2412.15115. Ling Yang, Zhaochen Yu, Bin Cui, and Mengdi Wang

arXiv
[14]

Xinyu Zhu, Junjie Wang, Lin Zhang, Yuxiang Zhang, Yongfeng Huang, Ruyi Gan, Jiaxing Zhang, and Yujiu Yang

Reasonflux: Hierarchical llm reasoning via scaling thought templates.arXiv preprint arXiv:2502.06772. Xinyu Zhu, Junjie Wang, Lin Zhang, Yuxiang Zhang, Yongfeng Huang, Ruyi Gan, Jiaxing Zhang, and Yujiu Yang

arXiv

Show all 14 references

[2005]

InProceedings of the Winter Simulation Conference, 2005., pages 14–pp

Tutorial on agent-based modeling and simulation. InProceedings of the Winter Simulation Conference, 2005., pages 14–pp. IEEE. Marvin Minsky. 1986.Society of mind. Simon and Schuster. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu

2005
[2017]

Qwen Team

Proximal policy optimiza- tion algorithms.arXiv preprint arXiv:1707.06347. Qwen Team

arXiv
[2021]

Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum

Measuring mathematical problem solving with the math dataset.arXiv preprint arXiv:2103.03874. Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum

arXiv
[2023]

InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5606–5632

Dialogue chain-of-thought distillation for commonsense-aware conversational agents. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5606–5632. Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qi...

2023
[2024]

arXiv preprint arXiv:2412.21187

Do not think that much for 2+ 3=? on the overthinking of o1-like llms. arXiv preprint arXiv:2412.21187. Guhao Feng, Bohang Zhang, Yuntian Gu, Haotian Ye, Di He, and Liwei Wang

arXiv
[2025]

Chengcheng Han, Xiaowei Du, Che Zhang, Yixin Lian, Xiang Li, Ming Gao, and Baoyuan Wang

Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948. Chengcheng Han, Xiaowei Du, Che Zhang, Yixin Lian, Xiang Li, Ming Gao, and Baoyuan Wang

arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [4]

InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8055–8068

Dialcot meets ppo: Decomposing and exploring reasoning paths in smaller language models. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8055–8068. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Stein- hardt

work page 2023

[2] [6]

Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others

Open- reasoner-zero: An open source approach to scaling up reinforcement learning on the base model.arXiv preprint arXiv:2503.24290. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others

arXiv

[3] [7]

Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen

Openai o1 system card.arXiv preprint arXiv:2412.16720. Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen

arXiv

[4] [8]

Charles M Macal and Michael J North

Understanding r1-zero-like training: A critical perspective.arXiv preprint arXiv:2503.20783. Charles M Macal and Michael J North

arXiv

[5] [11]

arXiv preprint arXiv:2501.18585

Thoughts are all over the place: On the underthinking of o1-like llms. arXiv preprint arXiv:2501.18585. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others

arXiv

[6] [12]

Thinkpatterns-21k: A systematic study on the impact of thinking patterns in llms.arXiv preprint arXiv:2503.12918. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, and 23 others

arXiv

[7] [13]

Ling Yang, Zhaochen Yu, Bin Cui, and Mengdi Wang

Qwen2.5 technical report.arXiv preprint arXiv:2412.15115. Ling Yang, Zhaochen Yu, Bin Cui, and Mengdi Wang

arXiv

[8] [14]

Xinyu Zhu, Junjie Wang, Lin Zhang, Yuxiang Zhang, Yongfeng Huang, Ruyi Gan, Jiaxing Zhang, and Yujiu Yang

Reasonflux: Hierarchical llm reasoning via scaling thought templates.arXiv preprint arXiv:2502.06772. Xinyu Zhu, Junjie Wang, Lin Zhang, Yuxiang Zhang, Yongfeng Huang, Ruyi Gan, Jiaxing Zhang, and Yujiu Yang

arXiv