Pith. sign in

REVIEW 3 major objections 5 minor 35 references

A counterfactual reward that trains critic agents to correct generator errors lifts a 1.5B reasoning model by 2.0 points with 16x fewer trainable parameters.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 16:21 UTC pith:GYD5P3Q5

load-bearing objection Real +2.0-point gain with careful controls, but the central counterfactual-advantage contribution is not statistically secured. the 3 major comments →

arxiv 2607.18006 v1 pith:GYD5P3Q5 submitted 2026-07-20 cs.LG cs.AIcs.CLcs.MA

MADA-RL: Multi-Agent Debate-Aware Reinforcement Learning for Parameter-Efficient Reasoning in Compact Models

Martino M. L. Pulici , Cuong Xuan Chu , Evgeny Kharlamov , Zifeng Ding , Volker Tresp , Yunpu Ma This is my paper
classification cs.LG cs.AIcs.CLcs.MA
keywords multi-agent debatereinforcement learningcounterfactual advantagelow-rank adaptationcompact language modelsmathematical reasoningcritic trainingtest-time scaling
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes MADA-RL, a post-training method that splits a compact language model into generator and critic roles and trains the critics with a counterfactual advantage: the critic's reward minus the generator ensemble's per-instance accuracy. The aim is to make critics specialize in fixing wrong answers rather than imitating correct ones, using only low-rank adapters so the base model stays frozen. On five mathematical reasoning benchmarks this raises a 1.5-billion-parameter model from 39.9% to 41.9% accuracy (p<0.001) with sixteen times fewer trainable parameters than full fine-tuning. The authors argue the gain comes from trained corrective behaviour—the highest critic improvement rate (19.6%) among evaluated models—rather than from the extra deliberation tokens alone. The point matters because it suggests compact models can get targeted reasoning gains under tight training budgets, with the caveat that inference-time token cost is high.

Core claim

The central claim, stated in the paper's own terms, is that a critic's advantage should measure improvement over generator consensus, not standalone correctness. MADA-RL defines A_C = R_C − 2·acc_G, where R_C is the critic's composite reward (accuracy plus brevity) and acc_G is the average accuracy of the generator ensemble on that instance. Because the baseline is per-instance and dynamic, a critic gets positive advantage only when it is correct where the generators are not; a static group mean would blur that signal. Trained this way and deployed as three generators plus three critics in a two-round debate, the system improves the base 1.5B model by 2.0 points across five math benchmarks,

What carries the argument

The load-bearing object is the counterfactual critic advantage, the identity A_C(R_C, acc_G) = R_C − 2·acc_G. It redefines the critic's baseline as the generator ensemble's per-instance symbolic accuracy instead of the usual group-mean reward, so credit is assigned to corrections of consensus errors. It rides on two supporting pieces: role specialization—generators trained on disjoint data, critics trained on prompts that concatenate the question with generator outputs—and the two-stage group-relative policy-optimization / low-rank-adapter training loop that keeps the bulk of parameters frozen. The per-instance baseline is what makes the signal 'counterfactual': the critic is rewarded for wh

Load-bearing premise

Critics are trained only on inputs of the form [question; generator responses], but at inference they are later asked to critique other critics' responses; the whole method assumes they generalize to that unseen input distribution.

What would settle it

Take the trained critics and run them on held-out questions where round-two input is a concatenation of critic outputs (the actual round-three deployment). If their accuracy on those inputs is no better than a model whose critics were trained with the standard group-relative advantage, or if the 19.6% critic improvement rate does not survive, then the counterfactual advantage is not what drives the gain; alternatively, re-run the no-counterfactual ablation with more seeds to see if the 0.8-point gap and critic-improvement difference vanish.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A 1.5B model can gain 2.0 accuracy points on math benchmarks using only about 110M trainable parameters, placing the method on the accuracy-vs-trainable-parameters Pareto front.
  • Critics trained with the counterfactual advantage correct wrong generator consensus 19.6% of the time, higher than any baseline in the critic role; the authors conclude the gain is learned corrective behaviour, not token volume.
  • The debate protocol itself is the dominant, statistically robust driver: reducing to one round or one generator-critic pair produces large significant drops (5.2 and 2.6 points).
  • At an equal trainable-parameter budget the method matches or exceeds low-rank re-trainings of the strongest baselines, implying those baselines' edge rests chiefly on data scale rather than on a mechanism low-rank adaptation preserves.
  • Training-time savings come with a real inference cost: about 33,800 tokens per question under the full protocol, so deployment must trade latency against accuracy.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves untested the train/deploy gap: critics see only generator outputs during training but see their peers' critiques at round three at inference. If this distribution shift degrades later rounds, part of the three-round gain may not come from the trained signal; a natural fix is to include critic outputs in the critic training set.
  • The counterfactual baseline could generalize beyond two roles—for example, to ensembles where the 'consensus' is a weighted mixture of several specialists, or where correctness is a continuous score rather than a binary symbolic match.
  • If the mechanism is genuinely corrective behaviour, then the debate transcript itself is a distillation target: a single model fine-tuned on the critic corrections might capture most of the gain at a fraction of the inference tokens, directly addressing the paper's main practical limitation.
  • The method's per-parameter efficiency suggests a cheap transfer recipe for 3-8B models or other reasoning domains, but the paper only tests one 1.5B mathematical-reasoning setting.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes MADA-RL, a post-training framework that specializes a compact LLM into generator and critic roles, trained with GRPO and LoRA adapters. The central contribution is a counterfactual critic advantage, A_C = R_C − 2·acc_G (§3.3), which subtracts a per-instance generator-accuracy baseline from the critic reward. On five math benchmarks, MADA-RL improves DeepSeek-R1-Distill-Qwen-1.5B from 39.9% to 41.9% (+2.0, p<0.001) with 110M trainable parameters (16× fewer than full fine-tuning). Ablations show that debate structure (rounds, number of agents) is the main driver, while the counterfactual advantage alone yields a non-significant 0.8-point gain (p=0.095). The paper attributes part of the gain to a higher critic improvement rate (19.6%) and claims the advantage teaches critics to correct generator errors rather than imitate them. The conclusion explicitly lists several open comparisons, including matched-budget training-free debate, gradient-variance analysis, and broader model/benchmark coverage.

Significance. If fully substantiated, the paper would provide a useful parameter-efficient recipe for compact-model reasoning: a clean combination of role specialization, LoRA, GRPO, and a debate-aware baseline that avoids value models and external verifiers. The paper's strengths include careful statistical reporting (standard errors, 95% CIs, Welch's t-tests, ten seeds per model), transparent cost accounting, and an unusually honest limitation section. The debate-structure ablations are robust and statistically significant. However, the central attribution to the counterfactual advantage is not yet empirically secured: the accuracy ablation is non-significant, the supporting mechanism metric lacks uncertainty quantification and a no-counterfactual control, and the paper itself acknowledges the absence of matched-inference training-free baselines. The contribution as it stands is an empirical study of a multi-agent debate training pipeline whose key novel component remains underdetermined; additional experiments are needed to justify the title's central claim.

major comments (3)
  1. [§3.3, Table 4, Table 3, Conclusion] The central claim that the counterfactual advantage drives the gain is not supported by the reported evidence. The 'No counterfactual' ablation (Table 4) drops only 0.8 points with p=0.095 and CI [−1.8, +0.2]; the paper explicitly says it does not rest on this ablation. The fallback evidence, the critic improvement rate of 19.6% (Table 3), is a single point estimate without standard error, significance test, or comparison to the no-counterfactual critic set. Without these, one cannot distinguish the counterfactual advantage from noise or from the standard GRPO advantage. The Conclusion's own limitation statement concedes that the counterfactual component is characterized only empirically. Please report the critic improvement rate for the no-counterfactual critics with bootstrap CIs, or soften the central claim to a framework whose gain is not attributed specifically to the advantage.
  2. [Conclusion, §4.3, Table 3] The paper argues that MADA-RL's gains stem from trained corrective behavior rather than deliberation volume, but it does not compare against training-free debate, voting, or LLM-as-judge at matched inference budget — a limitation the Conclusion explicitly acknowledges. Since MADA-RL uses three generators, three critics, two rounds, and 33,818 tokens/question, a training-free multi-agent debate with the same token budget is a natural control. Without it, the observed +2.0 points over the base model could in principle be due to increased test-time compute and response diversity, not to the counterfactual-trained critics. This comparison is necessary to support the mechanism-level attribution and is currently the paper's largest missing control.
  3. [Algorithm 2 (lines 8–12) vs Algorithm 1 (lines 7–8), Table 4] Critics are trained only on inputs of the form x' = [x; generator responses] (Algorithm 2, lines 8–12), but in round 3 of Algorithm 1 the critic input is [x; critic responses from round 2] (Algorithm 1, lines 7–8). The 3-round ablation (Table 4: 43.3±0.4, +1.4, p=0.016) therefore evaluates the trained critics under a distribution shift they never saw during training. The paper does not discuss or test this shift, so the 3-round improvement may not reflect the trained counterfactual signal. Please either add a round-3 training scheme (e.g., training examples with critic-conditioned inputs) or remove/qualify the 3-round result as an untrained generalization test.
minor comments (5)
  1. [§3.2, Eq. (1)–(3)] The length reward R_len is described as ranging from −0.5 to +0.5, but the piecewise definition caps incorrect outputs at 0; please clarify the exact range and its effect on the composite generator reward.
  2. [A.4] The full pairwise comparisons are exploratory and uncorrected for multiple comparisons; the paper acknowledges this, but the main text should state more prominently that the pre-specified comparisons in Table 2 are the formal claims, not the full matrix.
  3. [Table 4] The heading 'Counterfactual' for the ablation table is misleading, as the rows cover rounds, number of agents, diversity, and advantage computation; consider renaming to 'Ablations'.
  4. [Throughout] The paper alternates between 'Mada' and 'MADA-RL'; pick one naming convention and use it consistently.
  5. [Table 3] The 'Gain/parameter' column is scaled by 10^12, which is stated only in the note; please explain this scaling in the main text or in the table note more visibly.

Circularity Check

1 steps flagged

Counterfactual advantage's supporting metric is its own training objective; headline accuracy gain remains externally grounded.

specific steps
  1. self definitional [Section 3.3 (counterfactual advantage definition); Section 4.3 and Table 3 (critic improvement rate); Section 4.4 (Advantage computation)]
    "AC(ˆy,y,accG) = RC(ˆy,y) − 2 accG = 2 (Racc(ˆy,y) − accG) + Rlen(ˆy,y) ... Yet the token count alone misses a structural benefit: Mada achieves the highest critic improvement rate (19.6 %), correcting wrong generator answers more often than any other model—a direct consequence of the counterfactual advantage, which rewards critics for being correct when the generators are not."

    The critic improvement rate is P(critic correct | generator consensus wrong), i.e., the event acc_G=0 and R_acc=1. In the advantage, when acc_G=0, A_C = 2·R_acc + R_len, so the GRPO objective directly maximizes exactly this conditional probability (up to the length term). Reporting the 19.6% rate as the 'more direct signature' of the advantage is therefore reporting the optimized objective, not an independent confirmation. The no-counterfactual ablation (p=0.095) is non-significant, and the paper explicitly substitutes the improvement rate for it, so the central attribution of gains to the counterfactual component rests on a definitional metric.

full rationale

The main accuracy result (+2.0 over DeepSeek-R1-Distill-Qwen-1.5B, p<0.001) is measured on five public benchmarks and the reported accuracy numbers are not fit to those targets; the trainable-parameter accounting and token-cost comparisons are likewise externally grounded. The circularity is confined to the mechanism attribution. The paper defines the critic advantage as A_C = 2(R_acc − acc_G) + R_len, so the GRPO update directly maximizes the probability that a critic is correct when the generator consensus is wrong (acc_G=0). The 'critic improvement rate' in Table 3 measures precisely that conditional event. Section 4.4 then uses this rate as the 'more direct signature' of the counterfactual advantage after acknowledging the direct accuracy ablation is non-significant (p=0.095). This is not an independent confirmation: it is a report of the optimized objective. The comparison against other models in the critic role shows the trained critics correct more often, but without the no-counterfactual critics' improvement rate or uncertainty bounds, the causal claim that the advantage rather than debate structure drives gains is not externally secured. The separate train/serving input mismatch (critics trained on [x; generator responses] but fed [x; critic responses] in round 3) is a robustness concern, not a circularity. There is no load-bearing self-citation and no imported uniqueness theorem.

Axiom & Free-Parameter Ledger

8 free parameters · 6 axioms · 0 invented entities

The central claim rests on unverified transfer of critic training to out-of-distribution debate inputs, on exact symbolic answer matching as the reward signal, and on GRPO stability with a modified advantage. The listed hyperparameters are chosen by hand and affect all results; none is fitted to the benchmark targets in a parameter-estimation sense.

free parameters (8)
  • Generator reward accuracy:length weight ratio = 2:1 (RG = 2·Racc + Rlen)
    Hand-chosen 2:1 weighting; no sensitivity analysis shown; all agent rewards depend on it.
  • Counterfactual baseline scaling factor for acc_G = 2
    Chosen so the baseline 'is scaled consistently' with the 2·Racc term; the paper gives no derivation or ablation of this multiplier.
  • Number of generator agents N = 3
    Main configuration; ablations show 3 > 2 > 1, and larger ensembles are untested due to retraining cost.
  • Number of critic agents M = 3
    Main configuration; tied to generator count.
  • Number of debate rounds R = 2
    Main configuration; 1 round loses 5.2 points, 3 rounds add 1.4; chosen as a trade-off.
  • LoRA rank / alpha / dropout = rank=16, alpha=128, dropout=0.05
    Determines the 18.5M trainable parameters per agent; 'chosen based on empirical validation'.
  • GRPO group size / beta / epsilon = group_size=4, beta=0.04, epsilon=0.2
    RL hyperparameters selected by hand; affect stability and credit assignment.
  • Training subset size per agent = 7,500 (plus 750 validation)
    Disjoint subsets of the 29,925-example STILL-3 dataset; data scale directly determines gains.
axioms (6)
  • domain assumption GRPO remains stable and effective when the group-mean baseline is replaced by the per-instance counterfactual baseline 2·acc_G
    Section 3.3; no convergence or variance analysis is given, and the paper lists this as future work.
  • domain assumption Symbolic equivalence via latex2sympy2 and math-verify exactly captures answer correctness
    Sections 3.2 and 4.1; all rewards, acc_G, and benchmark accuracies depend on it; parser errors would corrupt the signal.
  • domain assumption Critics trained on generator-conditioned inputs transfer to round-3 inputs containing critics' own previous outputs
    Algorithm 2 line 12 vs Algorithm 1 line 7; the training distribution never includes peer critic outputs.
  • domain assumption Disjoint per-agent data subsets induce beneficial role specialization
    Section 3.1, adopted from Subramaniam et al. (2025); homogeneous-agent ablation is not significant (p=0.166).
  • domain assumption Multi-round debate improves reasoning beyond what independent sampling/self-consistency would achieve
    Section 4.4 ablation shows round count matters, but a training-free debate at matched inference budget was not evaluated.
  • standard math Welch's t-test on per-seed averages is a valid inference procedure for these comparisons
    Section 4.1; standard, but with 10 seeds per model and no multiple-comparison correction for the exploratory matrix.

pith-pipeline@v1.3.0-alltime-deepseek · 18279 in / 13893 out tokens · 138039 ms · 2026-08-01T16:21:53.774368+00:00 · methodology

0 comments
read the original abstract

Large language models achieve strong reasoning performance, but often at prohibitive training cost - a challenge that is especially acute for compact models ($\leq 4 \, \mathrm{B}$ parameters) trained under limited budgets. We introduce MADA-RL, a post-training framework that specializes compact models into generator and critic roles and trains them with a debate-aware learning signal, fine-tuning only a small subset of parameters via LoRA adapters. Our central contribution is a counterfactual critic advantage: a dynamic, role-conditioned baseline that redefines the critic's advantage as its reward minus the generator ensemble's per-instance accuracy. This explicitly optimizes critics to improve over generator consensus rather than to merely reproduce a correct answer, yielding more targeted credit assignment than static mean-reward normalization. At deployment, the specialized agents are composed in a lightweight multi-round protocol. Across five mathematical reasoning benchmarks, MADA-RL raises the accuracy of the DeepSeek-R1-Distill-Qwen-1.5B model from $39.9 \, \%$ to $41.9 \, \%$ ($+2.0$ points, $p < 0.001$) using $16$ times fewer trainable parameters than fully fine-tuned baselines, placing it on the accuracy-trainable-parameter Pareto front. It approaches, but does not surpass, the strongest baselines (DeepScaleR, STILL-3), which are trained on substantially larger datasets; we analyse this gap and the associated inference-time cost directly. A controlled study isolates the source of MADA-RL's gains: the counterfactual advantage produces the highest critic improvement rate of any model evaluated, indicating that trained critics learn to correct generator errors rather than to imitate them.

Figures

Figures reproduced from arXiv: 2607.18006 by Cuong Xuan Chu, Evgeny Kharlamov, Martino M. L. Pulici, Volker Tresp, Yunpu Ma, Zifeng Ding.

Figure 1
Figure 1. Figure 1: Multi-agent debate, adapted from Subramaniam et al. (2025). [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: presents the complete mada-rl training pipeline, and algorithm 2 goes into further details: the first block (lines 1–5) trains the generator agents; the second block (lines 6–12) constructs the debate-aware dataset D′ used to train the critic agents; the final block (lines 13–7) mirrors the generator training loop, but applies it to the critic agents. Dataset D õ Subset Di i Subset D1 i Subset DN i . . . .… view at source ↗
Figure 3
Figure 3. Figure 3: Trainable parameters and accuracy. Darker points connected with a solid line represent the Pareto [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references

  1. [1]

    Training language models to reason efficiently , url =

    Daman Arora and Andrea Zanette , journal =. Training language models to reason efficiently , url =

  2. [2]

    Tom Brown and Benjamin Mann and Nick Ryder and Melanie Subbiah and Jared D. Kaplan and Prafulla Dhariwal and Arvind Neelakantan and Pranav Shyam and Girish Sastry and Amanda Askell and Sandhini Agarwal and Ariel Herbert-Voss and Gretchen Krueger and Tom Henighan and Rewon Child and Aditya Ramesh and Daniel Ziegler and Jeffrey Wu and Clemens Winter and Chr...

  3. [3]

    An empirical study on eliciting and improving

    Zhipeng Chen and Yingqian Min and Beichen Zhang and Jie Chen and Jinhao Jiang and Daixuan Cheng and Wayne Xin Zhao and Zheng Liu and Xu Miao and Yang Lu and Lei Fang and Zhongyuan Wang and Ji-Rong Wen , journal =. An empirical study on eliciting and improving

  4. [4]

    Christiano and Jan Leike and Tom B

    Paul F. Christiano and Jan Leike and Tom B. Brown and Miljan Martic and Shane Legg and Dario Amodei , journal =. Deep reinforcement learning from human preferences , url =

  5. [5]

    Process reinforcement through implicit rewards , url =

    Ganqu Cui and Lifan Yuan and Zefan Wang and Hanbin Wang and Wendi Li and Bingxiang He and Yuchen Fan and Tianyu Yu and Qixin Xu and Weize Chen and Jiarui Yuan and Huayu Chen and Kaiyan Zhang and Xingtai Lv and Shuo Wang and Yuan Yao and Xu Han and Hao Peng and Yu Cheng and Zhiyuan Liu and Maosong Sun and Bowen Zhou and Ning Ding , journal =. Process reinf...

  6. [6]

    Reinforcement learning for reasoning in small

    Quy-Anh Dang and Chris Ngo , journal =. Reinforcement learning for reasoning in small

  7. [7]

    Computing Research Repository , title =

  8. [8]

    Tenenbaum and Igor Mordatch , journal =

    Yilun Du and Shuang Li and Antonio Torralba and Joshua B. Tenenbaum and Igor Mordatch , journal =. Improving factuality and reasoning in language models through multiagent debate , url =

  9. [9]

    Rae and Laurent Sifre , journal =

    Jordan Hoffmann and Sebastian Borgeaud and Arthur Mensch and Elena Buchatskaya and Trevor Cai and Eliza Rutherford and Diego de Las Casas and Lisa Anne Hendricks and Johannes Welbl and Aidan Clark and Tom Hennigan and Eric Noland and Katie Millican and George van den Driessche and Bogdan Damoc and Aurelia Guy and Simon Osindero and Karen Simonyan and Eric...

  10. [10]

    Hu and Yelong Shen and Phillip Wallis and Zeyuan Allen-Zhu and Yuanzhi Li and Shean Wang and Lu Wang and Weizhu Chen , journal =

    Edward J. Hu and Yelong Shen and Phillip Wallis and Zeyuan Allen-Zhu and Yuanzhi Li and Shean Wang and Lu Wang and Weizhu Chen , journal =

  11. [11]

    Aligning language models with offline learning from human feedback , url =

    Jian Hu and Li Tao and June Yang and Chandler Zhou , journal =. Aligning language models with offline learning from human feedback , url =

  12. [12]

    Geoffrey Irving and Paul Christiano and Dario Amodei , journal =

  13. [13]

    Hynek Kydlíček , note =

  14. [14]

    Encouraging divergent thinking in large language models through multi-agent debate , url =

    Tian Liang and Zhiwei He and Wenxiang Jiao and Xing Wang and Yan Wang and Rui Wang and Yujiu Yang and Shuming Shi and Zhaopeng Tu , journal =. Encouraging divergent thinking in large language models through multi-agent debate , url =

  15. [15]

    Junwei Liao and Muning Wen and Jun Wang and Weinan Zhang , journal =

  16. [16]

    Let's verify step by step , url =

    Hunter Lightman and Vineet Kosaraju and Yuri Burda and Harrison Edwards and Bowen Baker and Teddy Lee and Jan Leike and John Schulman and Ilya Sutskever and Karl Cobbe , journal =. Let's verify step by step , url =

  17. [17]

    Yen-Ting Lin , note =

  18. [18]

    Decoupled weight decay regularization , url =

    Ilya Loshchilov and Frank Hutter , journal =. Decoupled weight decay regularization , url =

  19. [19]

    Tang and Manan Roongta and Colin Cai and Jeffrey Luo and Li Erran Li and Raluca Ada Popa and Ion Stoica , note =

    Michael Luo and Sijun Tan and Justin Wong and Xiaoxiang Shi and William Y. Tang and Manan Roongta and Colin Cai and Jeffrey Luo and Li Erran Li and Raluca Ada Popa and Ion Stoica , note =

  20. [20]

    Aman Madaan and Niket Tandon and Prakhar Gupta and Skyler Hallinan and Luyu Gao and Sarah Wiegreffe and Uri Alon and Nouha Dziri and Shrimai Prabhumoye and Yiming Yang and Shashank Gupta and Bodhisattwa Prasad Majumder and Katherine Hermann and Sean Welleck and Amir Yazdanbakhsh and Peter Clark , journal =

  21. [21]

    Zhiyu Mei and Wei Fu and Kaiwei Li and Guangju Wang and Huanchen Zhang and Yi Wu , journal =

  22. [22]

    Learning to reason with

  23. [23]

    Long Ouyang and Jeff Wu and Xu Jiang and Diogo Almeida and Carroll L. Wainwright and Pamela Mishkin and Chong Zhang and Sandhini Agarwal and Katarina Slama and Alex Ray and John Schulman and Jacob Hilton and Fraser Kelton and Luke Miller and Maddie Simens and Amanda Askell and Peter Welinder and Paul Christiano and Jan Leike and Ryan Lowe , journal =. Tra...

  24. [24]

    Chanwoo Park and Seungju Han and Xingzhi Guo and Asuman Ozdaglar and Kaiqing Zhang and Joo-Kyung Kim , journal =

  25. [25]

    Zhihong Shao and Peiyi Wang and Qihao Zhu and Runxin Xu and Junxiao Song and Xiao Bi and Haowei Zhang and Mingchuan Zhang and Y. K. Li and Y. Wu and Daya Guo , journal =

  26. [26]

    Tenenbaum and Antonio Torralba and Shuang Li and Igor Mordatch , journal =

    Vighnesh Subramaniam and Yilun Du and Joshua B. Tenenbaum and Antonio Torralba and Shuang Li and Igor Mordatch , journal =. Multiagent finetuning: Self improvement with diverse reasoning chains , url =

  27. [27]

    Offline Reinforcement Learning for

    Huaijie Wang and Shibo Hao and Hanze Dong and Shenao Zhang and Yilin Bao and Ziran Yang and Yi Wu , journal =. Offline Reinforcement Learning for

  28. [28]

    Tina: Tiny reasoning models via

    Shangshang Wang and Julian Asilis and Ömer Faruk Akgül and Enes Burak Bilgin and Ollie Liu and Willie Neiswanger , journal =. Tina: Tiny reasoning models via

  29. [29]

    Le and Ed H

    Xuezhi Wang and Jason Wei andF Dale Schuurmans and Quoc V. Le and Ed H. Chi and Sharan Narang and Aakanksha Chowdhery and Denny Zhou , journal =. Self-consistency improves chain of thought reasoning in language models , url =

  30. [30]

    Chain-of-thought prompting elicits reasoning in large language models , url =

    Jason Wei and Xuezhi Wang and Dale Schuurmans and Maarten Bosma and Brian Ichter and Fei Xia and Ed Chi and Quoc Le and Denny Zhou , journal =. Chain-of-thought prompting elicits reasoning in large language models , url =

  31. [31]

    Towards large reasoning models: A survey of reinforced reasoning with large language models , url =

    Fengli Xu and Qianyue Hao and Zefang Zong and Jingwei Wang and Yunke Zhang and Jingyi Wang and Xiaochong Lan and Jiahui Gong and Tianjian Ouyang and Fanjin Meng and Chenyang Shao and Yuwei Yan and Qinglong Yang and Yiwen Song and Sijian Ren and Xinyuan Hu and Yu Li and Jie Feng and Chen Gao and Yong Li , journal =. Towards large reasoning models: A survey...

  32. [32]

    Griffiths and Yuan Cao and Karthik Narasimhan , journal =

    Shunyu Yao and Dian Yu and Jeffrey Zhao and Izhak Shafran and Thomas L. Griffiths and Yuan Cao and Karthik Narasimhan , journal =. Tree of thoughts: Deliberate problem solving with large language models , url =

  33. [33]

    Qiying Yu and Zheng Zhang and Ruofei Zhu and Yufeng Yuan and Xiaochen Zuo and Yu Yue and Weinan Dai and Tiantian Fan and Gaohong Liu and Lingjun Liu and Xin Liu and Haibin Lin and Zhiqi Lin and Bole Ma and Guangming Sheng and Yuxuan Tong and Chi Zhang and Mofan Zhang and Wang Zhang and Hang Zhu and Jinhua Zhu and Jiaze Chen and Jiangjie Chen and Chengyi W...

  34. [34]

    What's behind

    Yufeng Yuan and Yu Yue and Ruofei Zhu and Tiantian Fan and Lin Yan , journal =. What's behind

  35. [35]

    Yu Yue and Yufeng Yuan and Qiying Yu and Xiaochen Zuo and Ruofei Zhu and Wenyuan Xu and Jiaze Chen and Chengyi Wang and TianTian Fan and Zhengyin Du and Xiangpeng Wei and Xiangyu Yu and Gaohong Liu and Juncai Liu and Lingjun Liu and Haibin Lin and Zhiqi Lin and Bole Ma and Chi Zhang and Mofan Zhang and Wang Zhang and Hang Zhu and Ru Zhang and Xin Liu and ...