Pith. sign in

REVIEW 4 major objections 6 minor 59 references

The Explore-Execute Chain claims that splitting LLM reasoning into a short stochastic plan and a deterministic execution lets a model match or beat search-based test-time scaling while spending roughly a tenth of the decoding tokens.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 14:37 UTC pith:QRLCN2EB

load-bearing objection Clean idea, genuinely useful training recipe, but the abstract and Table 3 tell two different stories; fix the numbers and add error bars before the efficiency claim can be believed. the 4 major comments →

arxiv 2509.23946 v3 pith:QRLCN2EB submitted 2025-09-28 cs.LG cs.AIcs.CLstat.ML

Explore-Execute Chain: Towards an Efficient Structured Reasoning Paradigm

Kaisen Yang , Tinghe Zhang , Rushi Shah , Kaicheng Yang , Qinwei Ma , Dianbo Liu , Alex Lamb This is my paper
classification cs.LG cs.AIcs.CLstat.ML
keywords Explore-Execute Chaintest-time scalingchain-of-thoughtreinforcement learningsupervised fine-tuningreasoning efficiencyLLM reasoningdomain adaptation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that LLM reasoning should not be one undifferentiated chain: first a model should brainstorm several short high-level plans, then pick one and execute it deterministically. The authors argue that planning and execution need different computation—diversity for planning, precision for execution—and that current test-time scaling methods waste tokens by decoding many full solutions when only the expensive execution should be repeated sparingly. If the central claim holds, E2C-Select reaches 58.1% on AIME 2024 with 12.1k tokens per question, beating Self-Consistency (54.2% at 81.6k) and Forest-of-Thought (58.0% at 128.8k). The same split also supports domain adaptation by fine-tuning only the planning phase with 3.5% of the tokens of standard SFT, improving medical accuracy by up to 14.5 percentage points.

Core claim

The central claim is that the coupled reasoning distribution p(e|c) can be replaced by p'(π,e|c) = p'(π|c)·p'(e|π,c), where p'(π|c) is a short, highly informative plan and p'(e|π,c) is a highly deterministic execution that faithfully follows the plan. To realize this, the paper trains with causal SFT data (a full solution is first distilled into a plan, then a new execution is generated conditioned on that plan, so the model cannot ignore the plan), followed by RL that up-weights exploration tokens and includes an instruction reward measuring how well execution step titles match exploration step titles. After training, test-time scaling samples K cheap plans, selects the most promising one (

What carries the argument

The Explore-Execute Chain (E2C) decomposition is the central object: p'(π,e|c) = p'(π|c)·p'(e|π,c), separating a highly informative short plan from a highly deterministic execution that follows it. The two-stage training pipeline is the workhorse: causal SFT data generation (a plan is distilled from a solution, then a new execution is generated conditioned on the plan) plus RL with a token-level advantage weight λ_exp on exploration tokens and an instruction reward that scores plan-execution alignment. At inference, E2C-Select (Self LM-Judge) and E2C-Select (Semantic Cluster) convert the short-plan property into a test-time scaling strategy by sampling many plans, selecting the most promisin

Load-bearing premise

The load-bearing premise is that a two-to-four-sentence plan captures enough of the solution's critical steps that picking the best among K sampled plans and executing only it matches the accuracy of majority-voting over K full solutions; if plans routinely omit load-bearing steps, the token savings evaporate.

What would settle it

Replace the self-selected plan with a plan drawn from the lowest-scoring cluster while keeping the same execution model; if accuracy stays near the full method, the plan is not carrying the signal. Alternatively, compute an oracle best-of-K plan selector: if oracle-selected plans cannot approach the E2C-SC upper bound, the exploration phase is not informative enough to justify selecting only one plan.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • At K=32, E2C-Select (Self LM-Judge) achieves 58.1% accuracy on AIME 2024 with 12.1k tokens, surpassing Self-Consistency (54.2% at 81.6k) and matching Forest-of-Thought (58.0% at 128.8k) at about one-tenth the token cost.
  • Causal plan-then-execute data generation is load-bearing: it yields 0.998 plan adherence versus 0.499 for reverse-causal summaries, suggesting the model genuinely conditions execution on the plan.
  • Exploration-Focused SFT (EF-SFT) improves medical benchmark accuracy by up to 14.5 percentage points over standard SFT while using only 3.5% of the training tokens, and math-trained E2C models transfer zero-shot to medical reasoning at levels comparable to specialized medical models.
  • Prompt-only E2C underperforms the trained version, indicating that the structured SFT+RL training, not prompting alone, is what makes the short-plan property reliable.
  • The random-plan ablation (E2C-RP) is far worse than E2C-Select, and executing all sampled plans (E2C-SC) is the high-cost upper bound, supporting plan selection as the source of the efficiency gain.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A consequence the paper leaves implicit is that compute-optimal test-time scaling may shift from scaling the number of full solutions to scaling the number of cheap plans and investing in better plan selection, implying a scaling law dominated by selection quality rather than execution length.
  • The same plan-sample-select-execute loop could transfer to other domains where planning is cheap and execution is expensive, such as code generation, tool use, or theorem proving, with the selection mechanism (self-judge or embedding clusterer) being the main transferable component.
  • The paper states that its single-round split is untested on long-chain reasoning models; extending E2C to multi-round exploration-execution cycles, as the authors list for future work, is a natural test of whether the decomposition remains beneficial when iterative refinement matters.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces Explore-Execute Chain (E2C), a reasoning framework that structurally separates LLM inference into a stochastic, concise Exploration phase (a high-level plan) and a deterministic Execution phase. The authors propose a two-stage training pipeline: E2C-SFT, built on a synthetic dataset construction algorithm that conditions execution on the exploration plan, followed by E2C-RL, a GRPO variant with an exploration-token advantage coefficient and an instruction reward enforcing plan adherence. At inference, E2C test-time scaling samples K exploration plans and selects a smaller set via self-LM-judging or semantic clustering before executing. The paper also proposes Exploration-Focused SFT (EF-SFT) for domain adaptation, fine-tuning only the exploration component. The headline empirical claims are: on AIME'2024 at K=32, E2C-Select (Self LM-Judge) reaches 58.1% with 12.1k generated tokens, surpassing Self-Consistency (54.2%, 81.6k) and nearly matching Forest-of-Thought (58.0%, 128.8k); on medical benchmarks, EF-SFT uses 3.5% of standard SFT tokens and improves accuracy by up to 14.5%.

Significance. If the 58.1%/12.1k result is stable, the explore-execute decomposition would be a meaningful efficiency contribution to test-time scaling: it would deliver ensemble-like gains at roughly one-tenth of the decoding cost of full-solution sampling, while also improving interpretability. The training recipe, the explicit plan-adherence metric, and the public code release are concrete strengths. However, the manuscript as submitted does not yet pin down this central number: the abstract and body report conflicting accuracies for different, undefined variants; the AIME'2024 benchmark has only 30 questions, so a 4-point gap is roughly 1.2 questions; and Table 3 reports no variance or repetition. The EF-SFT improvement claimed in the text and abstract is also not supported by the numbers in Table 2. These issues are load-bearing for the paper's two main claims, so the result cannot yet be evaluated reliably; with corrections and additional experimentation the contribution could be solid.

major comments (4)
  1. [Abstract vs. §4.3/Table 3] The abstract reports 'E2C-ReAct Loop' at 53.3% accuracy with 12.4k tokens on AIME'2024 at K=32, and Tree-of-Thoughts at 50.0% with 71.3k tokens. Section 4.3 and Table 3 report the primary method 'E2C-Select (Self LM-Judge)' at 58.1% with 12.1k tokens, with ToT at 56.3% and 116.0k tokens. The variant 'E2C-ReAct Loop' is never defined in the methods or listed in Table 3, while the contribution bullet uses the 58.1% figure. Since AIME'2024 has only 30 questions, these two numbers differ by about 1.4 questions, and the abstract's ToT baseline differs from Table 3 by about 1.9 questions. The exact headline number must be resolved and the variant defined; otherwise the central efficiency-parity claim cannot be assessed.
  2. [Table 3, §4.3] No error bars, seeds, or repetition counts are reported for the test-time scaling comparison. At K=32, E2C-Select (Self LM-Judge) is 58.1%, SC is 54.2%, and FoT is 58.0%; the differences are 1.2 and 0.03 questions on a 30-question benchmark, which is within sampling noise unless variance is very low. The claim that E2C 'surpasses' SC and achieves near-parity with FoT at a tenth of the token cost requires either multiple seeds, per-question breakdowns, or bootstrap confidence intervals. Without these, the load-bearing quantitative claim is not yet demonstrated.
  3. [§4.3, Table 2, Abstract] The text states EF-SFT achieves 'significant improvements of 3.9% (Qwen3-8B) and 14.5% (Llama3.1-8B) over standard SFT', and the abstract repeats 'up to 14.5%'. Table 2, however, shows average accuracy improvements of 4.0 points (Qwen3-8B: 73.1 to 77.1) and 20.0 points (Llama3.1-8B: 47.5 to 67.5) over standard SFT. No individual benchmark column in Table 2 shows a 14.5-point gap. The claimed numbers and the table are inconsistent, and the abstract's central domain-adaptation result is therefore unsupported as reported.
  4. [Section 3.3, Algorithm 1, Table 3] The E2C-Select (Self LM-Judge) strategy is the key to the efficiency claim, but the ablations do not isolate the contribution of plan selection quality. The comparison E2C-Select vs. E2C-RP (Random Plan) shows a 12.3-point gap at K=32, but E2C-RP tests whether a randomly chosen plan from the K samples is sufficient; it does not control for selection quality versus plan informativeness or for the self-judge's bias toward plans that are consistent with its own execution training. Because the same model both generates and judges the plans, a concrete control is needed: e.g., compare self-LM-judge selection with an independent judge, with oracle selection, or against the average accuracy of randomly selected plans. This is needed to attribute the token savings to the selection mechanism rather than to the structured training alone.
minor comments (6)
  1. [Abstract] The abstract's ToT baseline (50.0%, 71.3k) differs from Table 3's ToT row (56.3%, 116.0k). The numbers should be harmonized in revision.
  2. [§3.2.1, §4.3, Algorithm numbering] Section 4.3 references 'our causal data generation strategy (Algorithm 1)', but Algorithm 1 is the test-time scaling method; the data generation algorithm is Algorithm 2. This makes the ablation description confusing for reproducibility.
  3. [§3.3] The LLM-based aggregation description says 'a powerful external LLM can be employed', but the experiments use 'Self LM-Judge' where the model itself is the judge. Clarify whether the judge is the same model, an external model, or both; the distinction matters for the interpretation of the results.
  4. [§3.2.2, Eqs. (5)–(8)] The 'quantified properties' are stated with approximation arrows and monotonicity symbols but no formal proof or precise statement of conditions. If these are heuristics, say so; if they are propositions, they need proofs or references. The appendix provides an entropy visualization but not a derivation.
  5. [Table 1] The table formatting is difficult to parse: for example, the GRPO baseline row shows '88.2 88.2' and '60.0 60.0' without clear column boundaries. Please reformat and add column headers explicitly.
  6. [Appendix A.8] The pure prompt-based E2C ablation is reported as pass@5 for K=2..5 on a base Qwen3-8B model using small K values, while the main test-time scaling comparison uses the E2C-trained model and K up to 32. This is not a matched ablation of the trained E2C framework and should be labeled as an exploratory comparison rather than evidence against prompting.

Circularity Check

1 steps flagged

Minor circularity in the plan-adherence validation; core test-time scaling and domain-adaptation claims are not circular.

specific steps
  1. fitted input called prediction [Section 3.2.2 and Appendix A.2.2 (r_instr definition); Section 4.3 / Table 4, Part A]
    "Instruction Reward: The instruction reward is specific to the E2C model and is added to the reward function when it comes to E2C model. This reward measures the alignment between the instructions generated during the exploration phase and the execution phase. It is computed by extracting the step titles from both the exploration and execution phases using regular expressions. Denote these sets of instructions as S1 (exploration) and S2 (execution). The instruction reward is defined as: rinstr = 0.1∗( |S1 ∩S 2| / max(|S1|,|S 2|) −1)"

    The RL training reward r_instr directly maximizes the overlap between exploration and execution step titles. The paper later reports near-perfect plan adherence (0.998) in Table 4 Part A as evidence that its causal data-generation algorithm is essential. Since the same overlap quantity was explicitly optimized by the instruction reward during RL training, high adherence is partly guaranteed by the training objective rather than being an independent validation of the data-generation strategy. The comparison against the flawed reverse-causal strategy retains some independent content, so the circularity is partial and localized, not central to the main test-time scaling results.

full rationale

The paper's central derivation is self-contained against external benchmarks: E2C is trained with SFT and RL on Openr1-math and DAPO-17K, and its efficiency claims are evaluated on AIME'2024 against independent baselines (SC, ToT, FoT). The test-time scaling comparison in Table 3 is an empirical evaluation, and the self LM-judge plan selection is not fitted to test answers. The EF-SFT medical adaptation is also evaluated on held-out medical benchmarks. The only circular element I identified is the plan-adherence validation in Section 4.3/Table 4, where the metric being reported as evidence is the same quantity that the RL instruction reward was designed to maximize. This affects a supporting ablation, not the central efficiency-parity claim. I also flag a non-circular reproducibility issue: the abstract reports E2C-ReAct Loop at 53.3% with 12.4k tokens and ToT at 50.0% with 71.3k tokens, while Table 3 reports E2C-Select at 58.1% with 12.1k tokens and ToT at 56.3% with 116.0k tokens; no seeds or error bars are reported. These inconsistencies are correctness risks, not circularity.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 0 invented entities

The central efficiency claims rest on the informativeness of short plans, the fidelity of the RL proxy for plan adherence, and the effectiveness of plan selection. These are domain assumptions with empirical but limited support. No new physical or formal entities are postulated.

free parameters (3)
  • lambda_exp (exploration token advantage coefficient) = not reported (stated only as >1)
    Controls the relative weight of exploration tokens in the GRPO update (Eq. 2). The value is chosen by hand, is not reported, and the training dynamics in Fig. 3 depend on it.
  • alpha (EF-SFT mixing ratio) = 10%
    Mixing ratio of base E2C dataset in Exploration-Focused SFT. Ablation in Table 4 shows 10% is best among tested values; the choice shifts reported medical accuracy.
  • M (number of semantic clusters) = 3
    Number of clusters in the semantic clustering test-time scaling variant; set by hand and affects the token budget and vote weights.
axioms (3)
  • domain assumption The regex-based step-title extraction and the Jaccard-like instruction reward faithfully measure plan-execution alignment.
    Used to compute r_instr in Appendix A.2.2; if the proxy is weak, the RL reward may reward superficial formatting rather than true plan adherence.
  • domain assumption The base model's solution can be summarized into a short high-level plan that preserves the load-bearing reasoning steps.
    Algorithm 2 generates exploration by summarizing Model_base's solution; if summarization drops key steps, SFT data teaches uninformative plans.
  • domain assumption An LLM judge and sentence-embedding clustering select plans that correlate with correctness.
    E2C-Select (Self LM-Judge) and Semantic Cluster rely on this; the E2C-RP ablation gives a lower bound but does not establish the selection signal's reliability.

pith-pipeline@v1.3.0-alltime-deepseek · 20972 in / 14515 out tokens · 368981 ms · 2026-08-04T14:37:30.129794+00:00 · methodology

0 comments
read the original abstract

Many LLMs plan before they act, yet planning and execution are often still entangled in one long generation trace, enforced only through prompts, or split across separate components. We argue that these two stages call for different computation: planning benefits from diversity and breadth, whereas execution demands precision and faithful adherence to a chosen strategy. Treating them as a single undifferentiated chain wastes tokens on routine derivation and makes it costly to explore alternative strategies at test time. We present the \textbf{Explore-Execute Chain (E\textsuperscript{2}C)}, which keeps both stages in one model but separates them structurally: a stochastic \textit{Exploration} phase drafts a concise high-level plan, and a deterministic \textit{Execution} phase carries it out. Causal SFT and RL train this split so that exploration stays informative and execution remains plan-faithful. Once plans are short yet decisive, extra inference compute can be directed to exploration rather than to repeatedly decoding full solutions. On AIME'2024 at $K{=}32$, \textbf{E\textsuperscript{2}C-ReAct Loop} reaches 53.3\% accuracy with only 12.4k tokens, outperforming Tree-of-Thoughts ($N{=}32$: 50.0\%, 71.3k). The same structure also supports lightweight domain adaptation: \textbf{Exploration-Focused SFT (EF-SFT)} updates only the planning phase, uses 3.5\% of the tokens required by standard SFT, and improves medical benchmark accuracy by up to 14.5\%.

Figures

Figures reproduced from arXiv: 2509.23946 by Alex Lamb, Dianbo Liu, Kaicheng Yang, Kaisen Yang, Qinwei Ma, Rushi Shah, Tinghe Zhang.

Figure 1
Figure 1. Figure 1: Our proposed Explore-Execute Chain (E2C) method decomposes reasoning chains into a short, high-level exploratory plan followed by a long, detailed execution (left). After optimizing these special reasoning chains using RL, it is possible to synthesize a large number of plans, use the model to pick the best plan, and then execute this plan (middle). This unlocks dramatically improved overall token efficienc… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of E2C method. The approach begins with E2C-SFT to achieve a paradigm shift, followed by a two-stage E2C-RL process that leverages the decomposition advantage of the new paradigm to boost performance. The resulting E2C-LLM can be efficiently adapted to new do￾mains via EF-SFT. The exploration stage’s high informativeness enables effective test-time scaling, implementable through semantic clusterin… view at source ↗
Figure 3
Figure 3. Figure 3: A comparison of training dynamics on the AIME’24 benchmark. The application of our [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

59 extracted references · 14 linked inside Pith

  1. [1]

    Rubicon model of action phases

    Anja Achtziger and Peter M Gollwitzer. Rubicon model of action phases. 2007

    2007

  2. [2]

    Räbsamen, Michael Wand, Hermann sundry, and Torsten Hoefler

    Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Lukas Gianinazzi, Kedar Tatwawadi, Joana Einsiedler, Daria Costanzo, Gregor J. Räbsamen, Michael Wand, Hermann sundry, and Torsten Hoefler. Graph of thoughts: Solving elaborate problems with large language models, 2023

    2023

  3. [3]

    Forest-of-thought: Scaling test-time compute for enhancing llm reasoning, 2025

    Zhenni Bi, Kai Han, Chuanjian Liu, Yehui Tang, and Yunhe Wang. Forest-of-thought: Scaling test-time compute for enhancing llm reasoning, 2025

    2025

  4. [4]

    Towards reasoning era: A survey of long chain-of-thought for reasoning in large language models, 2025

    Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards reasoning era: A survey of long chain-of-thought for reasoning in large language models, 2025

    2025

  5. [5]

    Open r1: A fully open reproduction of deepseek-r1

    deepseek . Open r1: A fully open reproduction of deepseek-r1. https://github.com/huggingface/open-r1, January 2025

    2025

  6. [6]

    The llama 3 herd of models, 2024

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and et al. The llama 3 herd of models, 2024

    2024

  7. [7]

    CoT-Space : A theoretical framework for internal slow-thinking via reinforcement learning, 2025

    Zeyu Gan, Hao Yi, and Yong Liu. CoT-Space : A theoretical framework for internal slow-thinking via reinforcement learning, 2025

    2025

  8. [8]

    Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, and Tom Goldstein

    Jonas Geiping, Sean McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R. Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, and Tom Goldstein. Scaling up test-time compute with latent reasoning: A recurrent depth approach, 2025

    2025

  9. [9]

    Hypertree planning: Enhancing llm reasoning via hierarchical thinking, 2025

    Runquan Gui, Zhihai Wang, Jie Wang, Defu Lian, Chi Ma, Huiling Zhen, Mingxuan Yuan, Jianye Hao, Enhong Chen, and Feng Wu. Hypertree planning: Enhancing llm reasoning via hierarchical thinking, 2025

    2025

  10. [10]

    DeepSeek-R1 : Incentivizing reasoning capability in llms via reinforcement learning, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and et al. DeepSeek-R1 : Incentivizing reasoning capability in llms via reinforcement learning, 2025

    2025

  11. [11]

    Reasoning with language model is planning with world model

    Shibo Hao, Yi Gu, Haodi Ma, Joshua Hong, Zhen Wang, Daisy Wang, and Zhiting Hu. Reasoning with language model is planning with world model. arXiv preprint arXiv:2305.14992, 2023

    Pith/arXiv arXiv 2023

  12. [12]

    Training large language models to reason in a continuous latent space, 2024

    Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space, 2024

    2024

  13. [13]

    Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems

    Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. arXiv preprint arXiv:2402.14008, 2024

    Pith/arXiv arXiv 2024

  14. [14]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020

    Pith/arXiv arXiv 2009

  15. [15]

    Measuring mathematical problem solving with the math dataset, 2021

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset, 2021. URL https://arxiv. org/abs/2103.03874, 2, 2024

    Pith/arXiv arXiv 2021

  16. [16]

    Hu, Kwangjun Ahn, Qinghua Liu, Haoran Xu, Manan Tomar, Ada Langford, Dinesh Jayaraman, Alex Lamb, and John Langford

    Edward S. Hu, Kwangjun Ahn, Qinghua Liu, Haoran Xu, Manan Tomar, Ada Langford, Dinesh Jayaraman, Alex Lamb, and John Langford. The belief state transformer. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=ThRMTCgpvo

    2025

  17. [17]

    What disease does this patient have? a large-scale open domain question answering dataset from medical exams

    Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 11 0 (14): 0 6421, 2021

    2021

  18. [18]

    Adam: A method for stochastic optimization

    Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    Pith/arXiv arXiv 2014

  19. [19]

    Compressing chain-of-thought in llms via step entropy, 2025

    Zeju Li, Jianyuan Zhong, Ziyang Zheng, Xiangyu Wen, Zhijian Xu, Yingying Cheng, Fan Zhang, and Qiang Xu. Compressing chain-of-thought in llms via step entropy, 2025

    2025

  20. [20]

    Fractured chain-of-thought reasoning, 2025

    Baohao Liao, Hanze Dong, Yuhui Xu, Doyen Sahoo, Christof Monz, Junnan Li, and Caiming Xiong. Fractured chain-of-thought reasoning, 2025

    2025

  21. [21]

    Wu, and Yu-Feng Liu

    Boi-Faltings Liu, Zhang-Wei Liu, Ruibo Jiang, Yisong Lyu, Yizhou Du, F. Wu, and Yu-Feng Liu. Llm+ p: Empowering large language models with optimal planning proficiency. arXiv preprint arXiv:2304.11477, 2023

    Pith/arXiv arXiv 2023

  22. [22]

    Openai o1 system card

    OpenAI . Openai o1 system card. https://openai.com/index/openai-o1-system-card/, December 2024. Updated: December 5, 2024. Accessed: 2025-09-24

    2024

  23. [23]

    Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering

    Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. In Conference on health, inference, and learning, pp.\ 248--260. PMLR, 2022

    2022

  24. [24]

    Learning adaptive parallel reasoning with language models, 2025

    Jiayi Pan, Xiuyu Li, Long Lian, Charlie Snell, Yifei Zhou, Adam Yala, Trevor Darrell, Kurt Keutzer, and Alane Suhr. Learning adaptive parallel reasoning with language models, 2025

    2025

  25. [25]

    Gorilla: Large language model connected with massive apis

    Shishir G Patil, Tianjun Zhang, Xin Wang, and Joseph E Gonzalez. Gorilla: Large language model connected with massive apis. arXiv preprint arXiv:2305.15334, 2023

    Pith/arXiv arXiv 2023

  26. [26]

    Measuring and narrowing the compositionality gap in language models

    Ofir Press, Or Yoran, Timo Schick, Idan Schmid, Ayal Fisch, Yoav Goldberg, and Kanishka Misra. Measuring and narrowing the compositionality gap in language models. arXiv preprint arXiv:2210.03350, 2022

    Pith/arXiv arXiv 2022

  27. [27]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. arXiv preprint arXiv:2305.18290, 2023

    Pith/arXiv arXiv 2023

  28. [28]

    Toolformer: Language models can teach themselves to use tools

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Tsvigun, Gautier Cances, and Najma Smaili. Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.04761, 2023

    Pith/arXiv arXiv 2023

  29. [29]

    Spurious rewards: Rethinking training signals in rlvr, 2025

    Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng, Yiping Wang, Sewoong Oh, Simon Shaolei Du, Nathan Lambert, Sewon Min, Ranjay Krishna, Yulia Tsvetkov, Hannaneh Hajishirzi, Pang Wei Koh, and Luke Zettlemoyer. Spurious rewards: Rethinking training signals in rlvr, 2025

    2025

  30. [30]

    Hybridflow: A flexible and efficient rlhf framework

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256, 2024

    Pith/arXiv arXiv 2024

  31. [31]

    Reflexion: Language agents with verbal reinforcement learning, 2023

    Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning, 2023

    2023

  32. [32]

    Scaling llm test-time compute optimally can be more effective than scaling model parameters, 2024

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters, 2024

    2024

  33. [33]

    Reasonmed: A 370k multi-agent generated dataset for advancing medical reasoning

    Yu Sun, Xingyu Qian, Weiwen Xu, Hao Zhang, Chenghao Xiao, Long Li, Yu Rong, Wenbing Huang, Qifeng Bai, and Tingyang Xu. Reasonmed: A 370k multi-agent generated dataset for advancing medical reasoning. arXiv preprint arXiv:2506.09513, 2025

    arXiv 2025

  34. [34]

    Reasoning aware self-consistency: Leveraging reasoning paths for efficient LLM sampling

    Guangya Wan, Yuqi Wu, Jie Chen, and Sheng Li. Reasoning aware self-consistency: Leveraging reasoning paths for efficient LLM sampling. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Pap...

    doi:10.18653/v1/ 2025

  35. [35]

    Baichuan-m1: Pushing the medical capability of large language models

    Bingning Wang, Haizhou Zhao, Huozhi Zhou, Liang Song, Mingyu Xu, Wei Cheng, Xiangrong Zeng, Yupeng Zhang, Yuqi Huo, Zecheng Wang, Zhengyun Zhao, et al. Baichuan-m1: Pushing the medical capability of large language models. arXiv preprint arXiv:2502.12671, 2025 a

    Pith/arXiv arXiv 2025

  36. [36]

    Emergent hierarchical reasoning in llms through reinforcement learning

    Haozhe Wang, Qixin Xu, Che Liu, Junhong Wu, Fangzhen Lin, and Wenhu Chen. Emergent hierarchical reasoning in llms through reinforcement learning. arXiv preprint arXiv:2509.03646, 2025 b

    arXiv 2025

  37. [37]

    HuatuoGPT , a general-purpose chinese medical large language model, 2024

    Junying Wang, Zhaonan Li, Renfeng Pu, Saijiang Shi, Yitong Meng, Zhaokun Wang, Yixin Liu, Jianing Zhou, Wenjia Zhang, Jialiang Chen, Yefeng Zheng, and Hong-Yin Mey. HuatuoGPT , a general-purpose chinese medical large language model, 2024

    2024

  38. [38]

    Self-consistency improves chain of thought reasoning in language models, 2022

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models, 2022

    2022

  39. [39]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pp.\ 24824--24837, 2022

    2022

  40. [40]

    Inference scaling laws: An empirical analysis of compute-optimal inference for LLM problem-solving

    Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Inference scaling laws: An empirical analysis of compute-optimal inference for LLM problem-solving. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=VNckp7JEHn

    2025

  41. [41]

    Monte carlo tree search boosts reasoning via iterative preference learning

    Yuxi Xie, Anirudh Goyal, Wenyue Zheng, Min-Yen Kan, Timothy P Lillicrap, Kenji Kawaguchi, and Michael Shieh. Monte carlo tree search boosts reasoning via iterative preference learning. arXiv preprint arXiv:2405.00451, 2024

    Pith/arXiv arXiv 2024

  42. [42]

    Softcot++: Test-time scaling with soft chain-of-thought reasoning, 2025 a

    Yige Xu, Xu Guo, Zhiwei Zeng, and Chunyan Miao. Softcot++: Test-time scaling with soft chain-of-thought reasoning, 2025 a

    2025

  43. [43]

    Adaptive termination for multi-round parallel reasoning: An universal semantic entropy-guided framework, 2025 b

    Zenan Xu, Zexuan Qiu, Guanhua Huang, Kun Li, Siheng Li, Chenchen Zhang, Kejiao Li, Qi Yi, Yuhao Jiang, Bo Zhou, Fengzong Lian, and Zhanhui Kang. Adaptive termination for multi-round parallel reasoning: An universal semantic entropy-guided framework, 2025 b

    2025

  44. [44]

    Qwen3 technical report, 2025 a

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and et al. Qwen3 technical report, 2025 a

    2025

  45. [45]

    Multiverse: Your language models secretly decide how to parallelize and merge generation, 2025 b

    Xinyu Yang, Yuwei An, Hongyi Liu, Tianqi Chen, and Beidi Chen. Multiverse: Your language models secretly decide how to parallelize and merge generation, 2025 b

    2025

  46. [46]

    React: Synergizing reasoning and acting in language models, 2022

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models, 2022

    2022

  47. [47]

    Tree of thoughts: Deliberate problem solving with large language models

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Sha, Silvio Savarese, and Tao an. Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Information Processing Systems, 2023

    2023

  48. [48]

    G o T : Effective graph-of-thought reasoning in language models

    Yao Yao, Zuchao Li, and Hai Zhao. G o T : Effective graph-of-thought reasoning in language models. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Findings of the Association for Computational Linguistics: NAACL 2024, pp.\ 2901--2921, Mexico City, Mexico, June 2024. Association for Computational Linguistics. doi:10.18653/v1/2024.findings-naacl.183....

    doi:10.18653/v1/2024.findings-naacl.183 2024

  49. [49]

    Dapo: An open-source llm reinforcement learning system at scale

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476, 2025

    Pith/arXiv arXiv 2025

  50. [50]

    Evoagent: Towards automatic multi-agent generation via evolutionary algorithms, 2024

    Siyuan Yuan, Kairui Song, Jia-Hao Chen, Xiao-Hui Tan, Dian-Hui Li, and Dong-Sheng Yang. Evoagent: Towards automatic multi-agent generation via evolutionary algorithms, 2024

    2024

  51. [51]

    VAPO : Efficient and reliable reinforcement learning for advanced reasoning tasks, 2025

    Yu Yue, Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, Tian Tian Fan, Zhengyin Du, and et al. VAPO : Efficient and reliable reinforcement learning for advanced reasoning tasks, 2025

    2025

  52. [52]

    Accessing gpt-4 level mathematical olympiad solutions via monte carlo tree self-refine with llama-3 8b, 2024

    Di Zhang, Xiaoshui Huang, Dongzhan Zhou, Yuqiang Li, and Wanli Ouyang. Accessing gpt-4 level mathematical olympiad solutions via monte carlo tree self-refine with llama-3 8b, 2024

    2024

  53. [53]

    First return, entropy-eliciting explore, 2025 a

    Tianyu Zheng, Tianshun Xing, Qingshui Gu, Taoran Liang, Xingwei Qu, Xin Zhou, Yizhi Li, Zhoufutu Wen, Chenghua Lin, Wenhao Huang, Qian Liu, Ge Zhang, and Zejun Ma. First return, entropy-eliciting explore, 2025 a

    2025

  54. [54]

    Parallel-r1: Towards parallel thinking via reinforcement learning, 2025 b

    Tong Zheng, Hongming Zhang, Wenhao Yu, Xiaoyang Wang, Xinyu Yang, Runpeng Dai, Rui Liu, Huiwen Bao, Chengsong Huang, Heng Huang, and Dong Yu. Parallel-r1: Towards parallel thinking via reinforcement learning, 2025 b

    2025

  55. [55]

    Least-to-most prompting enables complex reasoning in large language models

    Denny Zhou, Nathanael Sch \"a rli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, and Ed Chi. Least-to-most prompting enables complex reasoning in large language models. In International Conference on Learning Representations, 2023

    2023

  56. [56]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  57. [57]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  58. [58]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  59. [59]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...