{"id":"e490bf47-e160-4582-a04c-5cf3e7b0bf10","arxiv_id":"2507.05707","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A distillation framework that composes solutions from a tool-augmented agent and a text-reasoning teacher trains a 7B model to dynamically choose between code execution and verbal reasoning, improving math benchmark accuracy.","lead":"This paper presents DualDistill, a training method that teaches a 7-billion-parameter language model to alternate between writing code for arithmetic and reasoning in plain text, by distilling examples from a tool-using agent and a text-reasoning model. The trained model, Agentic-R1, improves math problem accuracy on several benchmarks while using fewer tokens than pure text-reasoning models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's method as implemented contradicts its own composition description: Appendix A.4.2 samples y2 independently while Section 3.1 and Algorithm 1 condition on y1. The (g1=0,g2=1) 'correction' trajectories are therefore not what the paper claims, and no experiment tests the claimed mechanism.","rationale":"The reader's CONDITIONAL verdict identifies the independence assumption as the weakest link, and I agree that is the sharpest problem. The benchmark-heuristic overlap and missing error bars are real but secondary: the heuristic overlap affects how broadly the gains generalize but does not create an internal inconsistency, and the lack of error bars can be remedied by reporting variance without changing the method. The independence issue is different: the paper's formal description and its implementation disagree, and the flagship 'correction' training category depends on that disagreement. Because the paper releases code and data, this is checkable. The empirical results may survive—even an independent second teacher gives the student a negative-then-positive demonstration—but the central mechanism as claimed is unverified. Hence the verdict stays CONDITIONAL/UNCHANGED: the authors should either fix the text to describe independent sampling and argue it is sufficient, or run the conditioned version and show it behaves similarly. I credit the paper for releasing code/data and for ablating composition, which makes the required check straightforward and the path to resolution clear.","tokens_in":16181,"tokens_out":9421,"duration_ms":114409,"concrete_test":"Re-run the teacher-distillation stage with y2 actually sampled from pi(·|x,y1) (by feeding y1 into the second teacher's prompt) on the same filtered DeepMath subset, keeping budgets, balancing, and hyperparameters fixed; compare Table 1 and Table 2. Also report the new (g1,g2) category counts. If the conditioned variant yields similar or better numbers, the independence approximation is harmless and only a wording fix is needed; if the (0,1) category or overall accuracy changes materially, the reported results are for a different method than the one described.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.1 and Algorithm 1 specify that the second teacher samples y2 ~ pi(·|x,y1), i.e., conditioned on the first solution y1. Appendix A.4.2 explicitly overrides this: 'we assume conditional independence and explicitly define the teacher model inference policy as pi(· |x, y1) = pi(· |x).' The implementation therefore draws y2 without seeing y1. This is not a minor detail: 1,393 of the 2,678 training trajectories belong to the (g1=0,g2=1) 'correction' category, and the paper's central contribution is that the second teacher corrects the first. In reality, those trajectories are a random pairing of an independent failure and an independent success, stitched with a hand-written transition. The student may still learn useful 'try another strategy after a failure' behavior from such demonstrations, so the headline accuracy gains are not automatically invalid. But the mechanism claimed in the paper—conditional composition enabling targeted correction—has not been tested. A reader cannot reproduce the method as written, and the composition ablation in Table 2 does not separate the independent-sampling variant from true conditioning.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DualDistill, a framework that distills two complementary teacher policies—an agentic tool-use teacher (OpenHands) and a text-reasoning teacher (DeepSeek-R1)—into a 7B student (DeepSeek-R1-Distill-7B). Trajectories from both teachers are composed based on the correctness of their solutions, with hand-written transition segments, and the student is further refined by self-distillation. The resulting model, Agentic-R1, is evaluated on two computation-intensive benchmarks (DeepMath-L, Combinatorics300) and three standard math benchmarks (MATH500, AIME, AMC), reporting improvements over same-size single-strategy baselines and showing adaptive tool-use rates across tasks.","tokens_in":16455,"tokens_out":7720,"duration_ms":81120,"significance":"If the empirical results hold, the paper offers a simple, SFT-based alternative to reinforcement-learning approaches for unifying tool use and long-CoT reasoning, with public code and data that support reproducibility. The qualitative examples and tool-usage statistics suggest the student learns a nontrivial per-query strategy switch from a relatively small (2.6k) curated training set. However, the core mechanism of 'correction' via trajectory composition is undercut by the implementation described in the appendix, and the benchmark evidence, especially on DeepMath-L, has no statistical support. The ideas are worth pursuing after the implementation/description gap is resolved and the evidence is strengthened.","major_comments":[{"comment":"Section 3.1 and Algorithm 1 specify that the second teacher samples y2 ~ pi(·|x, y1), conditioned on the first solution y1, and the paper describes the (g1=0, g2=1) case as 'the second teacher successfully corrects it.' Appendix A.4.2, however, explicitly overrides this: 'we assume conditional independence and explicitly define the teacher model inference policy as pi(· |x, y1) = pi(· |x).' As a result, 1,393 of the 2,678 training trajectories (Table 3) in the (g1=0, g2=1) category are not demonstrations of the second teacher correcting the first; they are independent successes paired with independent failures. The central claim that composed trajectories teach the student to correct errors is therefore untested. The authors should either implement the conditioned sampling described in the main text or revise the method description and the interpretation of the ablation in Table 2, ideally adding an experiment that directly compares independent sampling with conditioned sampling.","section":"Section 3.1 / Algorithm 1 / Appendix A.4.2"},{"comment":"Appendix A.2 states that trajectory segments occurring before a transition from incorrect to correct reasoning (t−+) are omitted from the loss calculation. This means that in (0,1) trajectories, the student never sees the first teacher's erroneous solution, because the loss covers only the transition and the second solution. Even if the second teacher had been conditioned on y1, the loss would not expose the model to the error it is supposed to learn to recover from. This undercuts the claimed 'correction' supervision and should be reported and analyzed explicitly.","section":"Appendix A.2"},{"comment":"Table 1 reports results 'averaged over 5 seeds' but gives no error bars, standard deviations, or significance tests. The headline gain on DeepMath-L at budget L is 59.3% vs 56.3% (a difference of about 2.6 problems on 87 problems) and on Combinatorics300 is 49.4% vs 44.5% (a difference of about 15 problems on 300). Without variance estimates, the claim of 'substantial performance improvements' is not supported. At minimum, the authors should report per-seed results and confidence intervals or a significance test for the main comparisons.","section":"Table 1 / Section 4.4"},{"comment":"Section 4.1 constructs the DeepMath-L evaluation set by selecting problems with answers whose absolute value exceeds 1e5, while Appendix A.3.1 uses a similar numerical-scale heuristic (answers exceeding 1,000) to build the agentic-favored training subset. The reported gain on DeepMath-L may therefore reflect distributional alignment with the training selection rather than a general capability for computation-intensive problems. The authors should demonstrate robustness on an independently constructed large-answer benchmark or explicitly discuss this selection overlap and its implications.","section":"Section 4.1 / Appendix A.3.1"}],"minor_comments":[{"comment":"The abstract claims the method 'improves accuracy across a range of tasks,' but Agentic-R1 without self-distillation is 3.1 points below DeepSeek-R1-Distill on MATH500 at the standard budget (80.0 vs 83.1 in Table 1); the wording should be softened to 'improves accuracy on computation-intensive tasks while remaining competitive on standard benchmarks,' as the limitations section acknowledges.","section":"Abstract"},{"comment":"The 5-seed averages are reported without error bars or standard deviations in Table 1 and in the budget curves of Fig. 4; adding variance information would greatly strengthen the presentation.","section":"Table 1 / Fig. 4"},{"comment":"The text says the student improves 'from 44.7% to 50.9%' on Combinatorics300, but Table 1 lists the DeepSeek-R1-Distill-7B baseline as 44.5% at the large budget; the numbers should be unified.","section":"Section 4.4"},{"comment":"The inference prompt in Fig. 2 uses 'brown' to highlight tool-specific instructions, which will not be visible in grayscale; please use a different visual marker (e.g., bold or a box).","section":"Fig. 2"},{"comment":"The claim of being 'the first framework to employ distillation with trajectory composition from two heterogeneous teacher models' is stronger than necessary; consider 'we are not aware of' or restrict the claim to the specific setting of tool-augmented versus text-reasoning teachers.","section":"Related Work (Section B)"},{"comment":"The hyperparameters beta1, beta2, K, and the random first-teacher inference budget L0 are not subjected to a sensitivity analysis; a brief robustness check would increase confidence that the reported behaviors are not artifacts of a particular threshold choice.","section":"Section 3.4 / Appendix A.2"}],"recommendation":"major_revision","confidential_remarks":"The central issue is the direct contradiction between the method description in Section 3.1/Algorithm 1 and the implementation in Appendix A.4.2. I do not see evidence of intentional misreporting, but the main text must be brought into line with the actual procedure, or the experiment must be run as described. The answer-magnitude overlap between the training filter and the DeepMath-L evaluation set is also a concern that should be addressed in revision. The paper fits the journal's scope, and the authors provide code and data that would allow others to verify the corrected version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the core idea is genuinely new: distill a 7B student from two heterogeneous teachers (a tool-augmented agent and a pure text reasoner) by composing their trajectories based on correctness, then self-distill. They release code and data, and the results on arithmetic-heavy benchmarks are consistently positive. Second, the mechanism described in the main text is not what was actually run. Section 3.1 and Algorithm 1 write y2 ~ pi(·|x,y1), but Appendix A.4.2 explicitly sets pi(·|x,y1)=pi(·|x), so the second teacher samples independently and never sees y1. That makes the 1,393 (g1=0,g2=1) 'correction' trajectories not corrections at all—they are independent failures and successes stitched together with a hand-written transition. The student may still learn something useful from these demonstrations, but the specific claim that the second teacher corrects the first is untested, and the composition ablation in Table 2 does not separate conditioned from independent sampling.\n\nWhat the paper does well: the training-data curation (filtering for problems where one strategy has an advantage) is sensible; the self-distillation stage is a nice addition and gives consistent gains; and the qualitative examples in Figures 6 and 7 do show the student switching strategies, which is evidence that the distillation signal, even if not the claimed correction, teaches the intended behavior. The related work is thorough and correctly identifies the gap: prior tool-augmented reasoning uses RL or homogeneous teachers.\n\nSoft spots beyond the conditioning issue: the evaluation is thinner than the claims. DeepMath-L is 87 problems, Combinatorics300 is 300, and there are no error bars or significance tests. The large-answer filter (>1e5) overlaps with the training-set heuristic (>1000), so the headline gains are partly built into the benchmark choice. The MATH500 drop at the standard budget is acknowledged but not analyzed. These are fixable.\n\nThe conditioning discrepancy is the load-bearing weakness. If the authors had actually conditioned y2 on y1, the paper would be stronger. As it stands, the empirical finding that a 7B student can learn to invoke tools vs. reason from SFT on 2.6k composed trajectories is still interesting and worth publishing—with the mechanism caveat made explicit and a real experiment comparing conditioned vs. independent sampling.\n\nFor a reading group, I'd bring it to discuss the gap between what we write in papers and what we run. I would cite it as a distillation method for heterogeneous teachers, not for the correction mechanism. Send to peer review: yes, with a request for statistical rigor and the conditioned-sampling test.","headline":"Real distillation idea, but the implemented trajectory composition does not match the claimed correction mechanism; still deserves a referee for the data and the core method.","tokens_in":17006,"tokens_out":2954,"would_cite":true,"duration_ms":31547,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A distilled 7B model learns to choose between writing code and reasoning in text for each math problem, and beats same-size single-strategy baselines on computation-heavy benchmarks.","keywords":["multi-teacher distillation","tool-augmented reasoning","long chain-of-thought","strategy selection","mathematical reasoning","self-distillation","trajectory composition"],"falsifier":"Run the same DualDistill pipeline with the second teacher actually conditioned on the first solution, $\\pi(\\cdot \\mid x, y_1)$, and compare DeepMath-L and Combinatorics300 accuracy against the paper's independent-sampling version; if the conditioned pipeline does not improve on the independent one, the correction signal in the composed trajectories is not doing the claimed work.","tokens_in":15941,"feed_emoji":"🧮","tokens_out":8386,"duration_ms":87339,"temperature":0.7,"pith_summary":"DualDistill is a fine-tuning recipe for teaching one language model two complementary ways to solve math problems: writing text-based reasoning, and writing code that is executed by an interpreter. The paper claims that a 7B student trained on composed trajectories from a tool-using teacher and a text-reasoning teacher learns to choose which strategy fits each query, and that this choice improves accuracy on computation-heavy benchmarks while staying competitive on standard ones. The trained model, Agentic-R1, invokes code tools on most combinatorial or large-number problems and reasons in text on abstract ones, and a self-distillation stage pushes accuracy higher still. The point of the work is that strategy selection can be distilled, rather than learned by reinforcement or hand-coded.","feed_headline":"A distilled 7B model picks tools or text per math problem","feed_subtitle":"Two teachers' trajectories teach one student when to write code and when to reason, beating single-strategy baselines.","key_machinery":"The carrying mechanism is trajectory composition with a rule-based correctness gate. Two teacher policies—one agentic/tool-using and one text-reasoning—generate solutions y1 and y2; a grader assigns binary scores g1 and g2, and the pair falls into one of four composition cases, with hand-written transition segments t−+ or t++ inserted between solutions when both appear. Loss masking removes failed segments so the student learns only from the successful portion. A second mechanism, self-distillation, samples K student trajectories per problem and, using thresholds $\\beta_1$ and $\\beta_2$, adds either a correct student trajectory plus a teacher verification or an incorrect trajectory plus a corrected teacher solution to a replay buffer for another fine-tuning round.","core_discovery":"On the paper's own terms, the central discovery is that trajectory composition lets a unified student inherit heterogeneous reasoning strategies from two teachers without an explicit router. For each training problem, the two teachers produce solutions y1 and y2; a rule-based grader marks each correct or wrong, and the trajectories are concatenated in a fixed pattern—wrong-then-right becomes a correction example, right-then-right becomes a verification pair, right-then-wrong keeps only the correct trajectory, and wrong-then-wrong is discarded. Fine-tuned on about 2.6k such composed traces, the 7B student learns to switch mid-problem when a strategy is failing, and on the paper's benchmarks it beats same-size single-strategy baselines on the computation-heavy subsets, with further gains from self-distillation. The paper interprets the learned switching behavior as evidence that the student has internalized when tools and when text are the better strategy.","pith_inferences":["Editorial inference: because the implementation samples y2 independently of y1, the wrong-then-right composed trajectories demonstrate “abandon a failing strategy and start fresh” rather than “repair the specific error”; a version where the second teacher actually reads y1 might teach true correction and could perform differently.","Editorial inference: the strategy-selection signal is likely tied to the student's pretraining, which already mixes code and text; on a model that has not seen both modalities, 2.6k trajectories may teach imitation rather than selection.","Editorial inference: the reported efficiency gain could be partly an artifact of the token-budget truncation used on the first teacher, since “Accuracy at Budget” rewards any correct answer that appears before the budget.","Editorial inference: the same composition idea could be applied to more than two teachers—say search, proof assistants, or vision tools—with the correctness gate deciding which trajectories to concatenate."],"forward_implications":["A single 7B model can internalize strategy selection from supervised fine-tuning alone, with no explicit router or reinforcement-learning stage.","Computation-heavy benchmarks need not be ceded to tool-only or text-only systems; a student of the same size as either specialist can beat both by choosing per query.","The learned choice is selective in practice: tool use appears in 79.2% of Combinatorics300 problems but only 52.0% of AMC problems.","Self-distillation strengthens the student further even when the teacher's correction is restricted to text-only solutions.","Composition itself, not just the presence of two teachers, is what drives the gain: the composition ablation beats the no-composition ablation on every reported benchmark."],"supporting_citations":[{"why":"Supplies the agentic tool-use teacher trajectories that become the tool side of the composed training data.","marker":"(Wang et al., 2024)"},{"why":"Supplies the text-reasoning teacher trajectories and the student base model used for fine-tuning.","marker":"(Guo et al., 2025)"},{"why":"Source of the DeepMath problems used to curate the training set and the DeepMath-L evaluation subset.","marker":"(He et al., 2025)"},{"why":"Provides the Qwen2.5-7B baselines with and without tool use that the student must beat.","marker":"(Yang et al., 2024)"},{"why":"Defines the MATH500 standard benchmark used to check generalizability.","marker":"(Lightman et al., 2023)"},{"why":"Supplies the MathVerify fuzzy grader used in the two-step evaluation.","marker":"(HuggingFace, 2025)"}],"fun_headline_variants":["7B model learns when to code or reason for math","Dual-strategy distillation teaches a 7B when to use tools","Agentic-R1: 7B auto-switches between code and text","Training by two teachers makes a 7B pick tools wisely","From two teachers, one 7B learns strategy switching"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise, stated in the appendix, is that the second teacher's solution can be sampled without conditioning on the first solution, so a “correction” trajectory is really two independent attempts concatenated; if conditioning on the failed attempt is what would make a correction example useful, the composition gains may not survive.","fun_headline_variants_meta":{"raw":{"variants":["7B model learns when to code or reason for math","Dual-strategy distillation teaches a 7B when to use tools","Agentic-R1: 7B auto-switches between code and text","Training by two teachers makes a 7B pick tools wisely","From two teachers, one 7B learns strategy switching"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000214,"raw_usage":{"total_tokens":1386,"prompt_tokens":864,"completion_tokens":522,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":480,"completion_tokens_details":{"reasoning_tokens":433}},"tokens_in":480,"tokens_out":522,"duration_ms":5277,"temperature":1.0,"reasoning_tokens":433,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:19:32.699288+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same DualDistill pipeline with the second teacher actually conditioned on the first solution, $\\pi(\\cdot \\mid x, y_1)$, and compare DeepMath-L and Combinatorics300 accuracy against the paper's independent-sampling version; if the conditioned pipeline does not improve on the independent one, the correction signal in the composed trajectories is not doing the claimed work.","supporting_citations":[{"cited_title":"Similarly, S1 (Muen- nighoff et al., 2025) and L1 (Aggarwal and Welleck,","cited_arxiv_id":null,"evidence_quote":"Supplies the text-reasoning teacher trajectories and the student base model used for fine-tuning."}],"review_version":1}