{"id":"26fbf3a5-c1ef-47af-8e37-ac36b04b1bea","arxiv_id":"2411.14698","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A feedback-driven distillation pipeline iteratively generates harder variants of problems small models solve and similar problems for ones they miss, improving their math reasoning scores.","lead":"This paper describes a training recipe that uses a large language model to generate extra math practice problems for small models, customized by whether the small model gets each earlier problem right or wrong. The authors report that this feedback-driven data expansion boosts small-model accuracy on math benchmarks beyond previously published distillation methods.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No control for dataset size or random augmentation, so the gains attributed to the feedback mechanism may be due to adding more generated questions.","rationale":"I considered the reader's concern about ChatGPT label noise; it is real, but the most load-bearing gap is causal attribution. Even if every generated label is correct, the paper lacks a control that isolates the feedback split from data scale. The title and abstract promise a Feedback-Driven Distillation framework, and Table 2 is the only direct evidence for that mechanism; because each experimental arm adds a different number of questions, the gains are confounded with dataset size. Prior work on LLM-based question augmentation provides a plausible alternative explanation, so without the random-split and random-augmentation controls the central claim is underdetermined. The SOTA claim is further weakened by the uncontrolled comparison to prior distillation methods, but that is secondary to the mechanism itself. The proposed ablation is computationally feasible, uses the authors' own pipeline, and would decisively distinguish feedback-driven generation from generic augmentation. Since the paper can be strengthened by this experiment, the existing CONDITIONAL verdict remains appropriate rather than moving to ACCEPT or REJECT.","tokens_in":15594,"tokens_out":7263,"duration_ms":126632,"concrete_test":"Run FlanT5-base under the identical pipeline but replace the easy/hard assignment with a random assignment of seed questions to the 'complex' and 'diverse' generation instructions, adding exactly 3,594 generated questions (the same total as Table 2's All Questions arm). If GSM8K and the three transfer sets match the reported 28.05 / 48.47 / 42.8 / 56.66 within normal seed variance, the feedback split is not the cause. Also include an arm that adds 3,594 randomly sampled original GSM8K questions rewritten by the same LLM, to control for augmentation without feedback. Report test-set accuracy with standard errors over at least three seeds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that feedback-driven generation—easy questions becoming more complex, hard questions becoming diverse variants—is what lifts SLM performance. Table 2 reports gains of +4.0 GSM8K points from 1,666 complex questions, +4.3 from 1,928 diverse questions, and +8.0 from all 3,594, but every added block also increases training set size. There is no arm that adds the same number of generated questions without the easy/hard split, such as uniform generation from all seeds or random assignment of seeds to the two instruction types. Since prior work (MetaMath, WizardLM, MuggleMath) already shows that LLM question augmentation alone improves math reasoning, the observed gains could be entirely a data-quantity and data-diversity effect. The SOTA comparison in Table 1 is likewise uncontrolled for data budget, base model, and decoding settings. Thus the causal role of the feedback mechanism—the paper's main contribution—is not established by the current experiments.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Feedback-Driven Distillation (FDD), a framework to improve mathematical reasoning of small language models (SLMs, ≤1B parameters) by iteratively expanding a distillation dataset. An initial dataset is built by prompting ChatGPT (gpt-3.5-turbo) to produce Program-of-Thought rationales for GSM8K training questions, which is used to fine-tune FlanT5 models. The fine-tuned SLM then classifies questions as easy or hard; for easy questions the LLM generates more complex variants, and for hard questions it generates similar-difficulty variants. These generated questions (with majority-vote answers and matching PoTs) are added to the distillation dataset and the SLM is fine-tuned from scratch; the process is repeated over multiple rounds. Experiments on GSM8K, ASDiv, SVAMP, and MultiArith report substantial gains over the initialization-only baseline, with FlanT5-Large reaching 49.43% on GSM8K and 67.55% average accuracy, which the paper describes as state-of-the-art for SLMs.","tokens_in":15756,"tokens_out":4245,"duration_ms":45310,"significance":"If the causal claim is correct, FDD would be a useful and practical contribution: using the student's own errors to guide data generation could boost sub-billion-parameter models to performance levels normally associated with much larger models, with broad implications for low-resource deployment. The paper has genuine strengths: it evaluates on four external benchmarks, provides internal ablations for generation strategy, number of rounds, and number of reasoning paths, and it attempts a data-leakage analysis. The ablation results in Table 2 and Figures 2 and 3 are internally consistent and suggest that iterative data expansion helps. However, the central contribution—that the easy/hard feedback split is what drives the gains—is not cleanly identified because the ablation arms differ in dataset size and generation strategy simultaneously. The quality of the self-generated training data is also not validated. These gaps currently leave the main claim underdetermined, though they are addressable within the scope of the manuscript.","major_comments":[{"comment":"The ablation for question generation strategies compares adding 1,666 complex questions, 1,928 diverse questions, and all 3,594 questions against the initialization-only baseline, but these arms vary both the strategy and the number of added questions at the same time. There is no control that adds the same number of generated questions while using a uniform or reversed assignment of seeds to the complex/diverse instruction types. Since prior work (MetaMath, MuggleMath, WizardLM) shows that unguided question augmentation alone can improve math reasoning, the observed +8.0 GSM8K improvement from 'All Questions' could be a simple data-quantity or data-diversity effect rather than evidence that the easy/hard feedback mechanism is causally responsible. Please add a matched-size control (e.g., random assignment of seeds to the two instructions, or a uniform generation strategy with the same total data budget) and report the comparison.","section":"4.4, Table 2"},{"comment":"New questions are accepted whenever at least one Program-of-Thought matches the majority vote among the generated PoTs, and the majority-voted answer is used as the gold answer. The paper provides no human validation, no estimate of label noise, and no analysis of how many generated questions are flawed, ambiguous, or unanswerable. Because the method's premise is that feedback-driven generation produces a high-quality training signal, a sample-based human evaluation or an independent agreement filter is needed to rule out the possibility that the gains come from memorizing incorrect patterns rather than from learning better reasoning.","section":"3.2"},{"comment":"The state-of-the-art claim is based on cross-paper comparisons with different base models, training budgets, and decoding settings; for example, Zhu et al. [5] and [6] use different SLM backbones and dataset sizes. No controlled comparison is reported under matched base model and data budget. The internal FDD gains are convincing as an ablation, but the headline 'SOTA' claim should either be rephrased as 'strong performance relative to published numbers' or supported by a controlled run against the closest prior distillation method on the same FlanT5 backbone and a matched training budget.","section":"4.3, Table 1"},{"comment":"The ROUGE-L similarity of roughly 0.11–0.12 is not a sufficient test for data leakage. Average lexical overlap between generated questions and test questions would not detect semantically equivalent rephrasings, swapped numbers, or multi-hop compositions, which are exactly the kind of variations the method generates. The conclusion that the results 'effectively eliminat[e] the impact of data leakage' is therefore too strong. I recommend a stronger test (e.g., n-gram overlap, model-based paraphrase detection, or measuring whether test-question variants appear among generated questions) or, at minimum, a caveat that this test only addresses surface-level lexical leakage.","section":"4.7, Eq. (4), Figure 4"}],"minor_comments":[{"comment":"Section 3.4 contains the typo 'Initiation stage' where 'Initialization stage' is meant.","section":"3.4"},{"comment":"The legend labels 'Complex' and 'Diverse' are ambiguous with respect to the multi-round setup; please clarify which bars correspond to generations from easy seeds versus hard seeds in each round.","section":"Figure 4"},{"comment":"The column heading appears as 'A VG' in the table body but 'AVG' in the text; please normalize the formatting.","section":"Table 1"},{"comment":"The text writes 'SV AMP' with a space; the correct benchmark name is 'SVAMP'.","section":"4.1"},{"comment":"The instruction templates are labeled Table A.3 and A.4 but are not referenced by those numbers in the main text; please align the cross-references.","section":"Appendix A"},{"comment":"The 'AVG' column in Table 1 averages over different sets of benchmarks for different models (e.g., MultiArith is missing for several proprietary and open-source LLMs); please state this explicitly so the averages are not read as directly comparable.","section":"4.3"}],"recommendation":"major_revision","confidential_remarks":"The paper makes a useful empirical observation—that iterative feedback-guided data expansion can substantially improve SLM math reasoning—but the attribution of the gains to the easy/hard feedback mechanism is currently confounded with data quantity and diversity. I would encourage the authors to add a data-matched control and a label-quality analysis before publication; without these, the contribution, while positive, remains incremental over existing augmentation-based distillation methods."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read of the FDD paper. The idea is simple and worth taking seriously: use the student's own mistakes to split seeds, then make easy questions harder and hard questions more numerous. That dual strategy is genuinely a small twist on prior work like LLM2LLM and WizardLM, and the authors clearly know the surrounding literature. The internal ablations (Table 2, Figures 2-3) do show that adding generated data and more rounds improves accuracy over the init-only baseline. So the weak version of the claim — feedback-driven generation helps SLMs on these benchmarks — is plausibly true.\n\nThe problems start when they call it SOTA. Table 1 compares numbers taken from other papers that used different base models, training budgets, and decoding setups. That is not a controlled comparison. More importantly, Table 2 has no arm that adds the same number of generated questions without the easy/hard split, so the gains attributed to the feedback mechanism could just be more data. The stress-test note is right: prior work already shows that plain question augmentation improves math reasoning. Without a size-matched random-augmentation control, the causal role of the dual strategy is not established.\n\nThe leakage check is also weaker than the paper claims. Average ROUGE-L of 0.11–0.12 against test sets is meaningless without a baseline comparison (e.g., similarity between the original GSM8K train and test, or between random natural language sentences). The authors should fix that. And they release no code or data, which makes it hard to build on. Also, the training signal relies on ChatGPT's own majority-vote answers with no human validation; if those are systematically wrong, the gains could be memorization of flawed patterns.\n\nThe soft spots are real but fixable. The central mechanism is plausible, and the paper is honest about some limitations (cost, retraining from scratch). I'd like to see them run a control with matched data size, re-run at least one prior method under the same budget, and release the generated datasets. Then the SOTA claim might hold for FlanT5 at least.\n\nFor a reading group, it could spark a useful discussion about what counts as evidence in data-generation papers, but it's not a must-read. I probably won't cite it in the next year. Still, a serious editor should send it to review: the method is relevant, the issues are addressable, and the paper deserves a chance to be strengthened.","headline":"A plausible feedback-driven data-generation idea undermined by missing controls—worth sending to review but only after the authors address the data-size confound.","tokens_in":16312,"tokens_out":2567,"would_cite":false,"duration_ms":26123,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A feedback loop in which the student's right and wrong answers decide what new questions the teacher writes lifts sub-billion-parameter models to state-of-the-art math reasoning, with FlanT5-Large reaching 49.43% on GSM8K and 67.55%…","keywords":["knowledge distillation","small language models","mathematical reasoning","program-of-thought","feedback-driven distillation","reasoning distillation","data augmentation","GSM8K"],"falsifier":"Take the newly generated questions from a finished FDD run, have independent annotators or a strong verifier solve them without seeing the recorded answers, and measure how often the majority-vote answer is wrong; a substantial error rate would show the gains could come from training on flawed patterns rather than from the feedback mechanism. A second decisive check is to replace the easy/hard targeting with random selection of the same number of new questions: if the accuracy gap disappears, the feedback split is not what carries the claim.","tokens_in":15349,"feed_emoji":"🧮","tokens_out":12027,"duration_ms":102123,"temperature":0.7,"pith_summary":"This paper tries to establish that a small language model (under a billion parameters) can reach state-of-the-art mathematical reasoning if its distillation data is grown by a feedback loop: the student model's own right and wrong answers decide what new training questions the teacher LLM writes. Questions the student solves correctly are rewritten into harder variants, while questions it fails are used as seeds for new questions of similar difficulty, and the enlarged dataset is used to fine-tune the student from scratch each round. The reported result is that FlanT5-Large (0.76B parameters) reaches 49.43% on GSM8K and 67.55% average accuracy on four benchmarks, surpassing earlier small-model distillation methods and some 7B open-source models. If the claim holds, practical math reasoning becomes deployable on low-resource hardware using only LLM-generated training data, at the cost of teacher-LLM compute and repeated from-scratch fine-tuning, a trade-off the paper acknowledges.","feed_headline":"Teacher feedback lifts a 0.76B model to 49.43% math accuracy","feed_subtitle":"A student-driven loop lets a 0.76B model beat some 7B LLMs on math reasoning with no new human data.","key_machinery":"The load-bearing mechanism is the easy/hard feedback split that turns the student's own performance into curriculum signal. After each fine-tuning round, the student SLM is asked to solve every question in the distillation pool; a question that the student answers correctly is classified easy and one it answers incorrectly is classified hard. The teacher LLM is then prompted with two different templates — one that asks for a more complex version of the seed question, one that asks for a new question of similar difficulty — so that the newly generated data is targeted at exactly the skills the student does and does not have. The accepted questions, filtered by majority voting over several PoTs, are added to the pool and the student is retrained from scratch, and the loop repeats with a shrinking seed pool that drops the already-mastered easy questions. This mechanism is what converts the student's mistakes into larger, more complex, and more diverse training data.","core_discovery":"The central discovery is a three-stage distillation loop the paper calls Feedback-Driven Distillation (FDD). In the initialization stage, a large teacher model writes Program-of-Thought (PoT) solutions — short Python programs that an interpreter executes — for the GSM8K training problems, and the solutions whose executed answers match the gold answers form the first distillation dataset. After fine-tuning the small model on this dataset, the student's performance splits the questions into easy (solved) and hard (failed) buckets; the teacher then generates more complex variants of easy questions and similar-difficulty variants of hard ones, accepts a new question when at least one of several PoTs agrees with the majority-vote answer, and merges everything into an enlarged dataset used to fine-tune the student from scratch. Over three rounds this yields 49.43% GSM8K accuracy and a 67.55% average across GSM8K, ASDiv, SVAMP, and MultiArith for FlanT5-Large, and the ablation studies attribute the gains to combining both generation strategies, to more rounds, and to more reasoning paths per question.","pith_inferences":["I would expect the easy/hard split to give the largest advantage early in the loop, when the student's errors are most informative; at very large data budgets the gap versus uniform question generation should shrink, a prediction the paper's ablations do not test.","The majority-vote filter screens rationales, not questions: a plausible but mathematically wrong new question whose generated programs consistently agree would still enter the training set, so teacher reliability may be the real ceiling on the method.","The same loop should transfer to other domains where correctness is cheaply checkable, such as code generation and formal or executable reasoning; in domains with no executable verifier, the voting filter would be the bottleneck."],"forward_implications":["Models far below a billion parameters can reach accuracy levels previously seen only in much larger models, so mathematically capable assistants become deployable on low-resource devices.","The gains transfer outside the training distribution: models fine-tuned only on GSM8K-derived questions improve on ASDiv, SVAMP, and MultiArith, evidence that the loop teaches generalizable procedures rather than memorized answers.","Distillation data can be manufactured rather than collected: the loop keeps producing novel questions from existing ones, so the bottleneck becomes teacher-LLM compute instead of human annotation.","Each extra round and each extra reasoning path per question adds measurable accuracy, implying that data scale and diversity are active ingredients of the distillation gain, not incidental side effects."],"supporting_citations":[{"why":"GSM8K training and test sets — the seed questions the distillation dataset is built from and the in-domain evaluation benchmark.","marker":"[14]"},{"why":"Program-aided distillation baseline that supplies the Program-of-Thought format FDD fine-tunes on and the prior small-model result it must surpass.","marker":"[5]"},{"why":"The authors' earlier diverse-reasoning-format distillation, the strongest small-model baseline (42.45% on GSM8K) that FDD claims to exceed.","marker":"[6]"},{"why":"Self-Instruct's seed-instruction generation scheme, which the hard-question branch of the question generation stage adapts to create similar-difficulty new questions.","marker":"[7]"},{"why":"WizardLM's evol-instruct rewriting of instructions into harder ones, which motivates the complex-question branch applied to easy questions.","marker":"[10]"},{"why":"MetaMath, prior evidence that rewriting math questions into new forms improves mathematical reasoning in fine-tuned models.","marker":"[8]"},{"why":"MuggleMath, prior evidence that question and response augmentation shape the gains of math fine-tuning.","marker":"[9]"},{"why":"Baseline showing small models gain reasoning ability when fine-tuned on LLM-generated rationales, anchoring the comparison for FDD's gains.","marker":"[2]"}],"fun_headline_variants":["Feedback loop distills math skill into 0.76B model","Small model hits 49.43% with teacher feedback loop","Distillation loop lifts tiny model to 67.55% math average","Teacher feedback enhances small model math reasoning","Incremental distillation yields 49.43% from small model"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the teacher LLM's generated questions and the majority-vote answers chosen as their gold labels are correct, because Section 3.2 accepts a new question whenever at least one Program-of-Thought solution matches the voting result, with no human validation, no external ground truth, and no measurement of how often flawed questions slip through.","fun_headline_variants_meta":{"raw":{"variants":["Feedback loop distills math skill into 0.76B model","Small model hits 49.43% with teacher feedback loop","Distillation loop lifts tiny model to 67.55% math average","Teacher feedback enhances small model math reasoning","Incremental distillation yields 49.43% from small model"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000489,"raw_usage":{"total_tokens":2435,"prompt_tokens":999,"completion_tokens":1436,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":615,"completion_tokens_details":{"reasoning_tokens":1351}},"tokens_in":615,"tokens_out":1436,"duration_ms":9732,"temperature":1.0,"reasoning_tokens":1351,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:00:07.908075+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the newly generated questions from a finished FDD run, have independent annotators or a strong verifier solve them without seeing the recorded answers, and measure how often the majority-vote answer is wrong; a substantial error rate would show the gains could come from training on flawed patterns rather than from the feedback mechanism. A second decisive check is to replace the easy/hard targeting with random selection of the same number of new questions: if the accuracy gap disappears, the feedback split is not what carries the claim.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"WizardLM's evol-instruct rewriting of instructions into harder ones, which motivates the complex-question branch applied to easy questions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MetaMath, prior evidence that rewriting math questions into new forms improves mathematical reasoning in fine-tuned models."}],"review_version":1}