{"id":"9b982327-36b9-45a5-b4f9-d2db07d9b546","arxiv_id":"2505.00031","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A self-training method that makes an LLM write an abstract plan before each answer improves average test accuracy by about 3 points over three self-training baselines on four reasoning benchmarks.","lead":"This paper introduces LEPA, a self-training method that teaches a language model to write a general, abstract plan before solving each reasoning problem, then fine-tunes it on the plan and the correct solution. The method beat standard self-training baselines on several reasoning benchmarks, though the gains are modest and the paper provides no code or error bars.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim depends on refined plans being answer-free, but self-reflection is fed the correct answer and compliance is never checked; this is testable and load-bearing.","rationale":"The reader's weakest assumption identifies the same soft spot: LEPA's self-reflection is given the correct answer, and only a prompt instruction forbids the plan from containing it. I see no evidence in the paper that this instruction is verified, and the distinction from STaR depends on it, so the concern is genuine and testable. I also looked for more fundamental problems, such as internal inconsistencies or a broken derivation, and found none: the method is clearly described, the ablations are internally consistent, and the compute-utilization comparison is a reasonable attempt to control for extra inference tokens. The missing variance/significance reporting and the uneven STaR convergence comparison are real but secondary; the answer-leakage audit is the most decisive missing piece. Because the concern is about an unverified but checkable assumption rather than a demonstrated flaw, the appropriate verdict remains conditional rather than reject or accept. The OOD result partially supports generality, but it does not settle whether the in-distribution gains come from abstract plans or from answer-conditioned training data, so the condition should be explicit: release code/data and run the answer-stripped reflection ablation.","tokens_in":15041,"tokens_out":5107,"duration_ms":58520,"concrete_test":"Run the complete LEPA data-generation pipeline on Hendrycks MATH and one multiple-choice benchmark (e.g., BoolQ) with a single modification: replace the '[Correct Answer]' field in the self-reflection prompt with a neutral placeholder or omit it, keeping all other prompts, sampling budgets, trials, and SFT hyperparameters identical. Compare the resulting test accuracy against Table 1. If accuracy drops by more than roughly 1 point (about half the reported margin over ReSTEM), answer leakage is load-bearing; if it is unchanged, the no-leakage assumption survives. As a secondary audit, scan the original refined plans for exact or near-exact occurrences of the correct answer or option text and report the detection rate.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.1 says the reflection prompt receives the correct answer and only 'stresses' that the reflected plan should not contain problem-specific information, including detailed calculation and the correct answer. Appendix A's new-plan prompt repeats this as an instruction ('You must not contain the correct final answer in the plan'), and Section 2.3 then asserts that because plans exclude problem-specific information, LEPA 'isolates the correct answer from solution generation' and avoids STaR's false-positive information-bypassing. This assertion is load-bearing: the claimed advantage over STaR and over solution-only self-training rests on the refined plans being abstract meta-knowledge rather than answer-carrying artifacts. The paper reports no audit of generated plans, no automatic detection of numeric answers or option text inside them, and no control condition in which the correct answer is withheld during reflection. If even a meaningful fraction of refined plans leak the answer, LEPA's training set contains exactly the answer-conditioned pattern the paper criticizes in STaR, and the reported margins (e.g., +2.0 on MATH, +3.1 average) could reflect leakage rather than transferable plans. The OOD MMLU-Pro result mitigates the concern for generalization but does not verify the stated mechanism or the contrast with STaR's false-positive pathology.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LEPA (LEarning to Plan before Answering), a self-training algorithm in which an LLM first generates an abstract anticipatory plan, then generates a solution guided by that plan, and refines the plan through self-reflection when the solution is incorrect. The refined plan and correct solution are then used as supervised fine-tuning data. The authors claim that LEPA captures transferable high-level meta-knowledge and avoids the \"information bypassing\" pathology of STaR, where models may copy the correct answer without learning valid reasoning. Experiments on Hendrycks MATH, HellaSwag, BoolQ, and PIQA with Llama 3 8B Instruct report consistent improvements over ReST, ReST EM, and STaR, with an average gain of 3.1 points over the best baseline. Additional ablations address the contribution of the plan, self-reflection, and inference-compute allocation, and appendix results include an OOD evaluation on MMLU-Pro, a second base model, additional benchmarks, and a re-evaluation with Simple-Eval.","tokens_in":15241,"tokens_out":3498,"duration_ms":36652,"significance":"If the reported results hold, LEPA is a useful contribution to LLM self-training: it introduces a simple, clearly described mechanism for learning abstract plans, and the empirical picture is unusually complete for a methods paper, including ablations, OOD generalization, an alternative base model, and an independent evaluation harness. The claimed advantages are concrete and falsifiable. However, the central mechanism depends on the assumption that refined plans are genuinely answer-free and problem-agnostic, and the manuscript provides only prompt-level assurances, not empirical verification. Because the improvement over answer-augmented self-training (STaR) and over solution-only self-training rests on this assumption, the missing audit is load-bearing. The absence of variance estimates also makes the reported margins difficult to interpret. These are fixable within the manuscript's scope, so the paper merits major revision rather than rejection.","major_comments":[{"comment":"The claim that refined plans are free of problem-specific information is load-bearing but never verified. Section 2.1 states that the reflection prompt receives the correct answer and simply 'stresses' that the reflected plan should not contain problem-specific information, including detailed calculations and the correct answer. Appendix A's new-plan prompt repeats this as an instruction ('You must not contain the correct final answer in the plan'). The paper reports no audit of generated plans, no automatic detection of leaked numeric answers or option text, and no control condition in which the correct answer is withheld during reflection. If a meaningful fraction of refined plans leak the answer, the training set contains exactly the answer-conditioned pattern the paper criticizes in STaR, and the reported margins (e.g., +2.0 on MATH, +3.1 average) could reflect leakage rather than transferable plans. The OOD MMLU-Pro result mitigates generalization concerns but does not verify the stated mechanism. Please add (i) a quantitative compliance analysis over generated plans and (ii) an ablation or control that removes the correct answer from the reflection prompt or otherwise detects leakage.","section":"Section 2.1 and Appendix A"},{"comment":"All accuracy numbers are single-run point estimates with no error bars, confidence intervals, or significance tests. The improvements over the best baseline are 1.7 to 4.8 points per benchmark, and without variance estimates it is unclear whether these differences are stable across seeds and data-generation randomness. Please report results over multiple seeds (or bootstrap intervals) and state explicitly whether the same test split is used across all methods.","section":"Section 3.1, Table 1"},{"comment":"The evaluation protocol is asymmetric: STaR is reported at its highest test accuracy because its accuracy drops on MATH, while all other methods are reported at convergence. This makes the magnitude of LEPA's advantage over STaR difficult to interpret. Please report all methods under the same checkpoint-selection rule, or present both convergence and best-checkpoint numbers for every method.","section":"Section 3.1, footnote 1"}],"minor_comments":[{"comment":"The heading 'Learning generalizable high-level meta-knowledge' appears twice; the second occurrence should be titled something like 'Avoiding information bypassing in self-reflection' to match its content.","section":"Section 2.3"},{"comment":"There are typos: 'LPEA' instead of 'LEPA' in Section 2.2, and 'hiders' instead of 'hinders' in Section 3.2. Also, the method name is inconsistently written as 'ReSTEM' and 'ReST EM'; please choose one form.","section":"Section 2.2 and Section 3.2"},{"comment":"The self-reflection prompt asks 'What contents should the new plan obtain?' which appears to be a typo for 'contain'; please correct it and check the prompt text for similar errors.","section":"Appendix A"},{"comment":"The learning curves are presented without error bars or a description of how many seeds they average over; please add this information or explicitly state that the curves are single runs.","section":"Figure 3"},{"comment":"The table is split across two rows because of page width; please reformat it so that the method names and their token counts/accuracies are aligned in a single row, or use a more compact layout.","section":"Table 3"},{"comment":"The additional-base-model result on Llama 3.1 8B is reported only on Hendrycks MATH; please clarify whether the full main benchmark suite was run on that model, or state explicitly that only MATH was used.","section":"Appendix C"}],"recommendation":"major_revision","confidential_remarks":"The paper is already published at ICLR 2025; this report assesses the submitted manuscript on its merits. The main risk is the leakage assumption in Section 2.1: a quick empirical audit of generated plans would either validate or undermine the central comparison with STaR. If the audit shows widespread leakage, the central claim would be compromised; if it shows near-zero leakage, the paper would be substantially strengthened."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: LEPA is a plausible and clearly written self-training variant that adds abstract plan generation plus self-reflection, and the main results support its central claim. The one thing I'd check before trusting the mechanism: the paper never verifies that the refined plans actually exclude the correct answer. That is a testable, load-bearing assumption, and the absence of an audit bugs me more than the missing error bars.\n\nWhat's new: unlike PS Prompting (plan only at inference), HSP (pre-collected hints), or Quiet-STaR (next-token meta-tokens), LEPA closes the loop by self-generating plans, reflecting with answer feedback, and SFT on plan-solution pairs. The two-round conversation format and the didactic examples make the method easy to reproduce. Ablations are consistent: removing plans hurts, removing self-reflection hurts, and the compute-control variants show silence tokens and long solutions don't help. The OOD test (MATH -> MMLU-Pro Math) and the second model (Llama 3.1) add credibility.\n\nWhere it's soft: the stress-test concern lands. Section 2.1 says the reflection prompt receives the correct answer and 'stresses' the plan should not contain it; Appendix A's new-plan prompt says 'You must not contain the correct final answer.' But the paper never checks compliance. If even a fraction of refined plans leak the numeric answer or option text, the training set contains exactly the answer-conditioned pattern the paper criticizes in STaR, and the reported margins could be leakage rather than transferable plans. The OOD result mitigates but does not rule out leakage, since plans might leak problem-specific structure without explicit answers. I'd want a simple audit: regex for answer strings, some human inspection, and ideally a control condition withholding the answer during reflection. That's an addressable revision, not a fatal flaw.\n\nAlso minor: single run, no error bars, one 8B base model for main results; STaR is reported at its highest rather than at convergence, which makes the comparison slightly uneven. The gains (1.7-4.8 points) are useful but not transformative.\n\nFor researchers working on LLM self-training or test-time planning, this is worth reading. Bottom line: this paper deserves a serious referee. It's a solid within-subfield contribution, the method is clearly specified, and the central claim is probably true but the mechanism is under-verified. I'd send it to review and require the leakage audit, not desk-reject it.\n\nRecommendation: engage with it.","headline":"Useful self-training paper with a load-bearing but unverified no-leakage assumption; worth serious review with a required audit.","tokens_in":15793,"tokens_out":3149,"would_cite":true,"duration_ms":27736,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Training an LLM to write a high-level abstract plan before producing a step-by-step answer improves self-training accuracy on four reasoning benchmarks.","keywords":["self-training","anticipatory plans","abstract meta-knowledge","self-reflection","large language models","reasoning benchmarks","supervised fine-tuning","plan-before-answer"],"falsifier":"Inspect the stored plans from LEPA's data-generation phase and count how many contain tokens that also appear in the problem's key numbers or in the correct answer (for instance, the '45' or '9' in the meerkat example). If a non-trivial share of the plans carry such problem-specific content, the claim that the plans are abstract meta-knowledge fails, because the accuracy gain could come from answer leakage rather than transferable planning.","tokens_in":14834,"feed_emoji":"🧠","tokens_out":7925,"duration_ms":67572,"temperature":0.7,"pith_summary":"This paper argues that what a self-training LLM generates should be more than step-by-step solutions: it should also generate a high-level abstract plan before answering. The proposed algorithm, LEPA, first asks the model to produce such a plan, then a solution guided by it, and keeps the pair only when the solution is correct; wrong attempts trigger self-reflection that revises the plan. Across HellaSwag, MATH, BoolQ, and PIQA, LEPA raises test accuracy above three answer-only self-training baselines, with an average gain of 3.1 points over the best one. If the result holds, it suggests that abstract meta-knowledge, not just more rationales, is the useful content to generate in LLM self-training.","feed_headline":"LLMs gain 3.1 points by planning before answering","feed_subtitle":"Self-generated high-level plans, refined by self-reflection, beat answer-only self-training on HellaSwag, MATH, BoolQ, and PIQA.","key_machinery":"The central object is the anticipatory plan: a short, self-generated, abstract blueprint that states the high-level steps for solving a class of problems without containing problem-specific numbers, calculations, or the answer. It carries the argument because it is the only channel through which meta-knowledge enters the training data. LEPA optimizes plans by self-reflection: when a plan leads to an incorrect solution, the model is shown the problem, the plan, the wrong solution, and the correct answer, and is instructed to produce a new plan that avoids the failure while remaining general; the revised plan is then tested by solving the problem again. During supervised fine-tuning, the model is trained to produce the refined plan in one turn and the solution in the next, so planning and answering are learned as one skill.","core_discovery":"The paper's central claim is that training an LLM to emit an abstract anticipatory plan before a detailed solution makes self-training more effective than training it on solutions alone. The plan is meant to be general problem-solving meta-knowledge, applicable to similar problems, so the solution must be produced from the plan plus the problem without seeing the answer. LEPA's data-generation loop generates a plan, solves under it, and if the solution is wrong, reflects on the failure and revises the plan; only plan-solution pairs that yield correct answers enter the supervised fine-tuning set. The paper reports that on four reasoning benchmarks LEPA outperforms prior self-training algorithms consistently, including a 30.2% accuracy on MATH versus 28.2% for the best baseline.","pith_inferences":["My inference: if plans are truly abstract, they should be reusable as few-shot exemplars or as a transferable prompt library across benchmarks, a use LEPA does not itself demonstrate.","My inference: the claimed advantage over answer-augmented self-training would be sharpest to test on a benchmark where answers are easy to memorize, since leakage would show up as large gains on shallow surface similarity.","My inference: an automatic abstraction score computed on plans could be used to filter low-quality plans during data generation, which LEPA currently leaves to correctness of the final answer."],"forward_implications":["On all four benchmarks LEPA's converged accuracy exceeds the best baseline, so a model trained this way would be expected to answer more test items correctly: 91.2% on HellaSwag, 30.2% on MATH, 88.4% on BoolQ, and 85.9% on PIQA.","The MATH-to-MMLU-Pro transfer result (38.9% versus 35.8%) implies the learned plans carry over to a different math benchmark, not just to held-out items from the same dataset.","The ablations imply that the plan, not just the extra inference tokens, drives the gain: removing the plan drops MATH accuracy by 5.9 points, while replacing self-reflection with rejection sampling drops it by 1.4 points.","LEPA's compatibility with REINFORCE (30.6% versus 30.2% on MATH) implies the planning objective can be plugged into policy-gradient training rather than only supervised fine-tuning."],"supporting_citations":[{"why":"Defines STaR, the answer-augmented self-training baseline that LEPA compares against and whose false-positive solutions LEPA claims to avoid.","marker":"Zelikman et al., 2022"},{"why":"Defines ReST, the rejection-sampling self-training baseline whose MATH and other benchmark numbers LEPA must beat.","marker":"Gulcehre et al., 2023"},{"why":"Defines ReST EM, the strongest average baseline, and supplies the finding that STaR-style rationalization produces false-positive solutions.","marker":"Singh et al., 2023"},{"why":"Provides the MATH benchmark used for the main math comparison and for the out-of-distribution transfer test.","marker":"Hendrycks et al., 2021"},{"why":"Provides the HellaSwag benchmark on which LEPA shows its largest gain.","marker":"Zellers et al., 2019"},{"why":"Provides the BoolQ benchmark for yes/no reading comprehension reasoning.","marker":"Clark et al., 2019"},{"why":"Provides the PIQA benchmark for physical commonsense reasoning.","marker":"Bisk et al., 2020"},{"why":"Provides the Llama 3 8B Instruct base model used for all training runs and ablations.","marker":"Dubey et al., 2024"}],"fun_headline_variants":["Planning first boosts LLM self-training on reasoning","Teach LLMs to plan before solving for smarter self-training","Abstract plans before answers improve LLM reasoning self-training","Self-taught LLMs that plan first beat answer-only training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything depends on the refined plans being genuinely free of problem-specific information: the reflection prompt tells the model not to include the correct answer or detailed calculations, but the paper never checks whether the model obeys.","fun_headline_variants_meta":{"raw":{"variants":["Planning first boosts LLM self-training on reasoning","Teach LLMs to plan before solving for smarter self-training","Abstract plans before answers improve LLM reasoning self-training","Self-taught LLMs that plan first beat answer-only training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000649,"raw_usage":{"total_tokens":2981,"prompt_tokens":953,"completion_tokens":2028,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":569,"completion_tokens_details":{"reasoning_tokens":1963}},"tokens_in":569,"tokens_out":2028,"duration_ms":14185,"temperature":1.0,"reasoning_tokens":1963,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:50:08.675880+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect the stored plans from LEPA's data-generation phase and count how many contain tokens that also appear in the problem's key numbers or in the correct answer (for instance, the '45' or '9' in the meerkat example). If a non-trivial share of the plans carry such problem-specific content, the claim that the plans are abstract meta-knowledge fails, because the accuracy gain could come from answer leakage rather than transferable planning.","supporting_citations":[],"review_version":1}