{"id":"34e79bd1-91bc-4c16-95df-3aa6abd6b1a2","arxiv_id":"2411.11053","paper_version":5,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A self-trained Monte Carlo tree search over natural-language reasoning steps improves LLM code generation on complex MBPP tasks by up to 11 points.","lead":"This paper proposes SRA-MCTS, a method that has a large language model generate step-by-step reasoning with Monte Carlo tree search, then use that reasoning to write code, and fine-tune itself on the result. The method improved average code-generation accuracy on MBPP across three model sizes without external supervision, though gains on HumanEval were inconsistent.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline 11-point MBPP-Complex gain rests on a small, unquantified subset with no variance or significance tests; a handful of solved problems can drive the reported improvement, so the central claim of consistent gains across model scales is not yet statistically supported.","rationale":"I do not dispute the reader's mechanism-level concern about self-evaluation, but the most load-bearing condition for the central claim is whether the observed pass@1 deltas are statistically reliable. The paper's own Section 5.1 concession about 'overemphasizing unnecessary overthinking steps' actually refers to GPT-4o quality scoring rather than to the self-evaluator in Eq. (4), so the reader's direct citation is slightly off. More importantly, even if self-scores were biased, the end-to-end comparison to CoT/ToT could still be valid because the bias might affect all methods equally or the diversity from MCTS might dominate. By contrast, if the Complex-split deltas are within noise, then the claim of improving complex questions across scales falls. The paper reports no seeds, no intervals, and no effect sizes; the Complex splits are small and overlapping with Full. The reader's verdict of CONDITIONAL is appropriate; my concern adds a specific statistical condition that should be met before accepting the headline number. The proposed McNemar/bootstrap test would settle it.","tokens_in":13141,"tokens_out":16294,"duration_ms":149900,"concrete_test":"Run the full SRA-MCTS data-generation and LoRA fine-tuning pipeline for gemma-2-2b, Llama-3.1-8B, and Qwen2.5-14B with at least three independent training seeds, then evaluate pass@1 on the MBPP Complex split with the same GPT-4o difficulty labeler. For each model, report the mean and 95% bootstrap confidence interval of the delta over Instruct and apply McNemar's test on the paired per-question outcomes. If for any model the confidence interval includes zero or McNemar p > 0.05, the claimed consistent improvement across scales is not statistically supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that SRA-MCTS improves pass@1 across model scales, especially on complex questions. The evidence for the complex-question component comes from Table 1's Complex splits, whose sizes are never reported. For Meta-Llama-3.1-8B-Instruct, the MBPP Complex pass@1 increases by 11.32 points (33.96 to 45.28); if the Complex split contains 53 questions, this is six additional solved problems, and for gemma-2-2b it is only three. No confidence intervals, standard deviations, or paired significance tests are given, and each model is fine-tuned once, so the deltas could be within run-to-run noise. This is particularly acute for the 2B model, where several Full-split metrics decline and the positive 'Average Increment' depends on overlapping Full/Complex columns. The paper's Section 4.3 defines the splits but omits their sizes, and Section 4.4 does not specify the number of seeds. Without this statistical grounding, the '11-point improvement' headline is not established as a real effect.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SRA-MCTS, a self-driven reasoning augmentation method that uses Monte Carlo Tree Search to generate natural-language 'thinking' paths, which are then concatenated with questions and used to prompt the model to generate code; the resulting (question, thinking, code) triples are used for supervised fine-tuning. The method is evaluated on MBPP, Human-Eval, MBPP+, and Human-Eval+ across three model scales (gemma-2-2b, Meta-Llama-3.1-8B, Qwen2.5-14B), reporting pass@1 and pass@10. The central claim is that SRA-MCTS improves average pass@1 across model scales without additional supervision, with a headline 11-point improvement on the MBPP-Complex split for Meta-Llama-3.1-8B, and that the generated thinking is higher quality and more robust than CoT or ToT thinking. The paper also includes ablations showing that thinking matters, that SRA-MCTS thinking is more stable, and that self-generated data can outperform data distilled from a larger model.","tokens_in":13443,"tokens_out":4046,"duration_ms":44054,"significance":"If the results are reliable, SRA-MCTS is a simple and potentially useful contribution: it offers a fully self-supervised pipeline for generating reasoning-augmented fine-tuning data for code generation, with public code and data, and it directly addresses the question of whether small models can improve themselves without external supervision. The ablations on the existence, variants, and quality of thinking are well designed and provide valuable evidence for the mechanism, and the comparison with distillation (Table 3) is a welcome attempt to isolate the self-generation component. The central limitation is that the headline improvements rest on single runs with no statistical uncertainty, and the self-evaluation loop is a plausible source of bias that is not validated.","major_comments":[{"comment":"The central claim of consistent performance improvements across model scales is not statistically supported. The paper does not report the number of random seeds, confidence intervals, or significance tests for any of the pass@1 or pass@10 numbers in Table 1. Since the Complex split sizes are never given, the headline 11.32-point improvement for Meta-Llama-3.1-8B on MBPP-Complex could correspond to a handful of additional solved problems; on a split of 53 questions this is six examples, and for gemma-2-2b (5.66 points) it is three. Without variance estimates, the single-run deltas for small models (e.g., the 2B model's -0.50 on full MBPP) are within plausible run-to-run noise. The paper should report split sizes, the number of seeds, and at minimum paired bootstrap or sign-test results, or the claims should be rephrased as exploratory.","section":"Section 4.4 and Table 1"},{"comment":"The method's load-bearing premise is that the same LLM used to generate reasoning steps can also score those steps reliably enough for MCTS selection, but this self-evaluation loop is neither validated nor analyzed. Eq. (4) uses the model as its own evaluator with a progressive rubric, and the paper itself concedes in Section 5.1 that 'the evaluator may overemphasize unnecessary overthinking steps.' Since the entire data-generation pipeline depends on these scores, the authors should provide evidence that the reward model is at least roughly calibrated, for example by comparing self-scores against code correctness on a held-out set or against an external judge (e.g., GPT-4o), or by conducting a sensitivity analysis in which the evaluator is replaced or the scoring rubric is varied. Without this, the MCTS selectivity may be driven by confidently wrong branches, which would undermine the proposed mechanism.","section":"Section 3.1, Eq. (4), and Section 5.1"},{"comment":"The abstract's claim of 'performance improvements across different model scales' is contradicted by the HumanEval and Human-Eval+ results, where the method declines on several metrics. For example, Meta-Llama-3.1-8B's Human-Eval pass@1 drops from 47.46 to 44.07 under SRA-MCTS, and the 2B model loses 1.83 points on Human-Eval+ pass@1. The paper attributes this to task-style mismatch, but that explanation is not tested. The claim should either be scoped to MBPP-style benchmarks or supported by additional experiments isolating the task-style effect. In addition, the 'Average Increment' column in Table 1 averages over overlapping Full and Complex splits of the same benchmarks, which double-counts the easier questions and can inflate the apparent improvement; this should be clarified or the average should be restricted to non-overlapping splits.","section":"Abstract, Section 4.5, and Table 1"},{"comment":"The evaluation protocol for pass@10 is underspecified, which is particularly relevant because the abstract highlights improvements in diversity metrics. The paper does not report the number of samples per problem, the generation temperature for the test set, the exact computation of the unbiased pass@k estimator, or the number of independent runs. Without these details, the pass@10 results in Table 1 cannot be reproduced or compared with the literature, and the diversity claim is not verifiable. Please provide the full sampling configuration and, ideally, the raw sample counts.","section":"Section 4.4 and Section 4.5"}],"minor_comments":[{"comment":"The UCB1 formula is typeset ambiguously as 'r 2 · ln N n'; it should be written with clear parentheses, e.g., as r + c * sqrt(2 * ln(N) / n), to avoid confusion about whether the 2 is inside or outside the square root.","section":"Equation (1)"},{"comment":"The decontamination procedure is described as '10-gram level duplicate detection' with a threshold of 0.3, but the similarity measure is not specified; please state whether this is Jaccard similarity, overlap, or another metric, and describe how n-grams are extracted (e.g., token-based or character-based).","section":"Section 4.1"},{"comment":"The 'Average Increment' column should state the exact set of metrics and splits being averaged, and the fact that Full and Complex splits overlap should be explicitly acknowledged in the table caption.","section":"Table 1"},{"comment":"The ToT baseline is described as depth-first search with max depth 4 and branching factor 3, but the paper does not explain how the LLM proposes and evaluates intermediate steps in the ToT setup; additional implementation details are needed for reproducibility.","section":"Section 4.2"},{"comment":"The GPT-4o-based difficulty classification is not described in sufficient detail; please report the prompt used, the number of easy/medium/hard questions per benchmark, and any manual verification of the split quality.","section":"Section 4.3"},{"comment":"There are several minor wording and typographical issues, including a missing period after 'maximum of 5 retries' and a misspelling of 'reflection' as 'ref lection' in the Reflection paragraph.","section":"Section 3.1 and Section 5.1"}],"recommendation":"major_revision","confidential_remarks":"The paper has a plausible core idea and the ablations are a strength, but the statistical support for the headline claims is too thin for acceptance, and the self-evaluation circularity needs more than a passing acknowledgment. The authors should also consider narrowing the scope of the claim to MBPP-style benchmarks, as the HumanEval results undercut the 'consistent improvements' wording. I would be supportive of a revised version that adds seed-level variability, split sizes, a validation of the reward model, and full sampling details for pass@k."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this paper deserves a serious referee. The method is a reasonable extension of MCTS-based self-training to the 'thinking' portion of code generation, and the ablations are better designed than most papers in this area.\n\nWhat's new: SRA-MCTS applies MCTS to generate intermediate natural-language reasoning steps, using the same LLM as generator, evaluator, and reflector, then converts thinking to code and fine-tunes on the triples. That's distinct from ReST-MCTS*, which uses MCTS to generate complete answers, and it's a plausible way to get self-improvement without a teacher. The paper reports gains across three model scales on MBPP, with the largest effect on complex questions.\n\nWhat it does well: The ablations actually isolate the contribution. Comparing with/without thinking shows the reasoning component matters, especially on complex splits. Replacing only the thinking content (CoT vs ToT vs SRA-MCTS) controls for code quality. And the comparison against self-distillation from a 70B model is informative: self-generation is competitive or better on most benchmarks, which is the real claim of interest for small-model self-improvement. Code and data are public.\n\nThe soft spots are real but not fatal. The 11-point MBPP-Complex headline for Llama-8B lacks context: split sizes are never given, and with a typical complex split of 50–100 problems that could be 5–10 solved problems, well within run-to-run noise. There are no error bars, no multiple seeds, and the claim of 'consistent improvements' overstates things because HumanEval and HumanEval+ results are often negative, sometimes sharply (e.g., CoT drops 18.65 points). The 2B model actually declines on several full-split metrics. So the central claim should be softened from 'consistent improvements' to 'improvements on MBPP-style tasks, especially complex ones.' The paper also never compares against ReST-MCTS* or RethinkMCTS, its two closest relatives; only CoT and ToT are baselines. That's a gap for the novelty claim.\n\nOn the self-evaluation concern: the same model scores its own steps, which risks rewarding confidently wrong branches. The paper acknowledges the overthinking issue in Section 5.1, but it does not quantify how often the self-evaluator is wrong. That said, the overall results and the thinking-quality correlation suggest the scoring is doing something useful. I wouldn't call it circular.\n\nBottom line: the method is plausible, the experimental design is good, and the main weaknesses are reporting and framing. A referee should ask for split sizes, seeds, variance, and a direct run against ReST-MCTS*. This is exactly the kind of paper that should be in the review process rather than desk-rejected.","headline":"A solid self-improvement pipeline with a well-designed ablation, but the headline gain needs split sizes and error bars, and the closest MCTS baselines are missing.","tokens_in":13920,"tokens_out":2781,"would_cite":true,"duration_ms":97837,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A code model can be its own teacher: MCTS-written thinking, self-scored and fine-tuned, lifts complex-benchmark pass@1 without extra supervision, with an 11-point gain on MBPP-Complex for the 8B model.","keywords":["code generation","Monte Carlo Tree Search","self-improvement","reasoning augmentation","chain-of-thought","tree-of-thought","self-evaluation","supervised fine-tuning"],"falsifier":"Take the same SRA-MCTS-generated thinking and re-rank the tree's branch selection with an external oracle, such as a strong closed-source model or execution of the final code on held-out test cases, then fine-tune on the oracle-selected paths. If oracle-ranked thinking yields no higher pass@1 than self-ranked thinking, the self-evaluation mechanism is not the active ingredient; if it yields higher pass@1, the paper's claim that self-evaluation alone drives the improvement is falsified.","tokens_in":12986,"feed_emoji":"💻","tokens_out":10823,"duration_ms":98981,"temperature":0.7,"pith_summary":"The paper argues that a single instruction-tuned LLM can close its own improvement loop in code generation: Monte Carlo Tree Search generates multi-step natural-language 'thinking' for a question, the same model scores each step, the final thinking guides code generation, and the question-thinking-code triple is used for supervised fine-tuning. It claims that this self-driven pipeline improves pass@1 on MBPP across 2B, 8B, and 14B models without any additional supervisory signal, with the largest gains on the Complex (medium and hard) split, and that it beats CoT and ToT as data generators when the test task style matches the training set. The headline number is an 11-point improvement on MBPP-Complex for Meta-Llama-3.1-8B-Instruct. The paper also reports that adding its thinking to ground-truth code is robust, while CoT and ToT thinking degrade when the code is swapped, and that self-generated data often beats data distilled from a 70B teacher.","feed_headline":"An 11-point jump in complex code generation, self-taught","feed_subtitle":"One 8B model reaches it by fine-tuning on its own MCTS-built reasoning steps.","key_machinery":"The search tree of reasoning steps is the load-bearing object. Each node stores a state (the question plus all steps so far), an action (the next step, produced by sample decoding), a reward (the model's own score), and a reflection that guides the next expansion. Selection uses UCB1 to balance exploitation of high-scoring nodes against exploration of unvisited ones; expansion samples up to three distinct next steps; evaluation scores each step with a progressive rubric that checks single-step correctness, solution coherence, solution completeness, and solution correctness in that order; reflection either emits an <end> tag when the question is judged solved or a short hint for the next step; and backpropagation reweights parent rewards from the children's visit-weighted scores. Repeating this loop for up to five iterations assembles a 'thinking' that the same model then converts to code.","core_discovery":"On its own terms, the paper claims that high-quality intermediate reasoning is a separable and trainable ingredient of code generation: SRA-MCTS produces thinking that is more diverse and more reliably correct than the model's chain-of-thought or tree-of-thought output, and fine-tuning on that thinking improves the model's code-generation pass rate, especially on complex problems. The experimental claim is that these gains appear at every tested scale, grow with model size, and are largest on tasks whose style matches the training set (MBPP-style generate-from-scratch, not HumanEval-style function completion). The authors interpret this as evidence that small models can improve themselves through self-generated reasoning data without distillation from larger models.","pith_inferences":["The same self-search-then-finetune loop could transfer to other generation tasks where intermediate reasoning can be scored by the model itself, such as math word problems or tool-use planning; the paper does not test that transfer.","The self-evaluation bias the authors acknowledge suggests that a better scoring signal, such as compiler feedback on intermediate code or a learned reward model, could enlarge the gains; this is speculation beyond the paper's experiments.","The observed scale-dependence hints at a self-improvement plateau: if evaluation quality limits data quality, very small models may gain little or nothing, and the paper's 2B results already show a smaller average increment than the 14B results.","A direct test of the thinking's generality would be to take SRA-MCTS thinking generated by one model and use it to prompt a different model for code generation; the paper's variant experiment swaps only the code, not the thinking."],"forward_implications":["If the experimental results hold, a code LLM can bootstrap its own reasoning-augmented training data, removing the need for a larger teacher model to provide demonstrations.","The method's largest gains on medium and hard questions suggest that reasoning-augmented data matters most precisely where models currently fail.","The robustness result with ground-truth code means the thinking itself carries transferable value, not just the code it was paired with.","Pass@10 also improves on MBPP, indicating that SRA-MCTS thinking increases the diversity of viable solutions, not just the best attempt.","The consistent improvements across 2B, 8B, and 14B models indicate the pipeline is scale-robust within the tested range."],"supporting_citations":[{"why":"Supplies Monte Carlo Tree Search, the core search procedure the method adapts for reasoning-step generation.","marker":"[Coulom, 2006]"},{"why":"Supplies UCB1, the node-selection rule that balances exploitation and exploration in the tree.","marker":"[Auer et al., 2002]"},{"why":"Provides Tree-of-Thought, the nearest reasoning-search baseline that SRA-MCTS extends and compares against.","marker":"[Yao et al., 2023]"},{"why":"Presents ReST-MCTS*, the prior MCTS-based self-training approach from which SRA-MCTS differentiates by searching over thinking rather than complete answers.","marker":"[Zhang et al., 2024a]"},{"why":"Defines Chain-of-Thought, the standard reasoning data-generation baseline used in the training comparisons.","marker":"[Wei et al., 2022]"},{"why":"Releases the HumanEval benchmark, one of the two code-generation test beds.","marker":"[Chen et al., 2021]"},{"why":"Releases the MBPP benchmark, the test bed where the largest gains are reported.","marker":"[Austin et al., 2021]"},{"why":"Provides the EvalPlus framework with expanded test cases (HumanEval+ and MBPP+) used for rigorous evaluation.","marker":"[Liu et al., 2023]"}],"fun_headline_variants":["Self-improving AI code: MCTS reasoning boosts complex tasks","No teacher needed: AI learns to reason, then codes better","MCTS reasoning loop lifts code generation without extra labels","Small models get smarter at code via self-made reasoning steps","SRA-MCTS self-driven reason improves code, even for 8B models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the model's self-scores of its own reasoning steps are accurate enough for UCB1 to steer the search toward good thinking, so if the evaluator is biased, the tree selects confidently wrong branches and the training data inherits the error.","fun_headline_variants_meta":{"raw":{"variants":["Self-improving AI code: MCTS reasoning boosts complex tasks","No teacher needed: AI learns to reason, then codes better","MCTS reasoning loop lifts code generation without extra labels","Small models get smarter at code via self-made reasoning steps","SRA-MCTS self-driven reason improves code, even for 8B models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000893,"raw_usage":{"total_tokens":3821,"prompt_tokens":887,"completion_tokens":2934,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":503,"completion_tokens_details":{"reasoning_tokens":2847}},"tokens_in":503,"tokens_out":2934,"duration_ms":22294,"temperature":1.0,"reasoning_tokens":2847,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:58:18.283253+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the same SRA-MCTS-generated thinking and re-rank the tree's branch selection with an external oracle, such as a strong closed-source model or execution of the final code on held-out test cases, then fine-tune on the oracle-selected paths. If oracle-ranked thinking yields no higher pass@1 than self-ranked thinking, the self-evaluation mechanism is not the active ingredient; if it yields higher pass@1, the paper's claim that self-evaluation alone drives the improvement is falsified.","supporting_citations":[{"cited_title":"Efficient selectivity and backup operators in monte-carlo tree search","cited_arxiv_id":null,"evidence_quote":"Supplies Monte Carlo Tree Search, the core search procedure the method adapts for reasoning-step generation."},{"cited_title":"Finite-time analysis of the multiarmed bandit problem","cited_arxiv_id":null,"evidence_quote":"Supplies UCB1, the node-selection rule that balances exploitation and exploration in the tree."},{"cited_title":"Chi, Quoc V","cited_arxiv_id":null,"evidence_quote":"Defines Chain-of-Thought, the standard reasoning data-generation baseline used in the training comparisons."}],"review_version":1}