{"id":"d8b579e8-17f6-44a0-bbcf-b941419f5ca0","arxiv_id":"2607.14895","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Merging an instruction-tuned model back with the original reasoning model restores lost reasoning and keeps most of the task gain, using only input-output data and a cheap calibration search.","lead":"This paper shows that fine-tuning a reasoning AI on plain question–answer data, then mixing its weights back with the original model, improves the AI on new tasks without sacrificing its ability to reason. The recipe costs about three dollars per adaptation on average and works for coding and summarization, where reliable automated graders are not available.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Calibration uses trace presence, not trace usefulness; paper's own §6 shows they can decouple, so α* may restore 'talking' without restoring reasoning (MATH500) even though Table 1 looks good.","rationale":"The reader's weakest-assumption analysis correctly identifies the calibration signal as the most load-bearing part of the paper: non-empty trace rate is a shallow proxy, and the paper's own interpretability experiment shows it can be satisfied without recovering performance. I agree with the reader on this point.\n\nI do not think the concern warrants rejecting the paper. Table 1 provides direct empirical evidence that the final merged models retain target-task gains and restore MATH500 in these settings; the judge validation for summarization and the cost/runtime breakdown give independent support. The issue is about whether the calibration criterion is measuring what the method claims to measure, and therefore whether the approach transfers to new RLMs and tasks. That is exactly the kind of uncertainty that should keep the verdict at CONDITIONAL rather than ACCEPT.\n\nI also note the paper is honest about one boundary: §6 reports failure on mathematical proofs and hypothesizes that reasoning supervision is needed there. This limits the abstract's broad phrasing, but the authors disclose it. The concrete test I propose would settle whether the concern actually materializes in the evaluated configurations: if ρ ≥ 0.9 always co-occurs with MATH500 restoration on the full α grid, the selection rule is validated for these settings; if not, the method's central claim needs to be weakened or the calibration signal changed.\n\nNo ad hominem is intended; the critique is on the argument's evidence structure, not on the authors. The final recommendation is to keep the CONDITIONAL verdict and request the α-grid-versus-MATH500 analysis as a condition for stronger acceptance.","tokens_in":24923,"tokens_out":8272,"duration_ms":87116,"concrete_test":"Re-run the α sweep (0, 0.25, 0.5, 0.75, 1) for all eight model/task settings and, at each α, measure ρ(Mα, Dcal), MATH500, and target-task score. Plot MATH500 loss against ρ and check whether ρ ≥ 0.9 is sufficient to imply MATH500 within one point of base. If any α satisfies ρ ≥ 0.9 but shows MATH500 loss > 1 point, the selection rule can pick a 'talking' model that has not actually recovered reasoning; the paper should then replace or augment the calibration signal (e.g., minimum trace length, intermediate-step correctness) or weaken the central claim. If no such α exists, the concern is resolved for these settings, but the analysis should still be reported because it is the direct evidence for the proxy.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The selection rule (Eqs. 2–3) chooses the largest α whose non-empty reasoning-trace rate on Dcal is ≥ ρ_min. This counts any token after the start-of-reasoning marker as reasoning. The paper's own §6 'Interpretability' demonstrates the problem: prefilling the response with 'Okay' yields a non-empty trace without recovering untuned-model performance. Thus trace presence is not, by itself, evidence of useful reasoning.\n\nThe central claim that merging 'recovers most or all of the lost reasoning capability' is therefore supported only by the final MATH500/task-score columns, not by the calibration signal that actually determines α*. The paper never shows, across the α grid, that the set {α : ρ(Mα, Dcal) ≥ 0.9} coincides with the set where MATH500 is restored. It is possible for the rule to select a large α that keeps the target-task gain and the trace rate high while general reasoning has already eroded; the good Table 1 outcomes could be specific to these eight settings rather than a consequence of the calibration criterion.\n\nThe concern is not an internal contradiction: the final test-set results are positive, and MATH500 is a reasonable held-out proxy. The weakness is that the method's advertised mechanism—calibrating on reasoning-trace rate—is not shown to be the reason the mechanism works. The paper's own §6 also concedes that on mathematical proofs the method failed, suggesting the claim that it 'recovers reasoning behavior' is too strong outside domains where IFT already provides most of the task signal.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a verifier-free method for adapting reasoning language models (RLMs) to new tasks using only input-output training data. The pipeline first performs standard instruction fine-tuning (IFT) on the target task, then linearly merges the IFT checkpoint with the original RLM. The merge coefficient α is selected on a held-out target-task calibration set as the largest α such that the non-empty reasoning-trace rate ρ(Mα,Dcal) is at least ρ_min=0.9. The method is evaluated on four open RLMs (OpenThinker 7B, Apriel Nemotron 15B, Olmo3 7B, DeepSeek R1 Qwen 7B) on Rust coding and text summarization, with MATH500 as a held-out proxy for general reasoning preservation. The main empirical finding is that standard IFT often collapses both target-task reasoning and MATH500, while the merged model restores MATH500 to within about one point of the base model in seven of eight settings while retaining a large fraction of the IFT target-task gain, at lower cost than on-policy distillation and KL-regularized IFT baselines.","tokens_in":25247,"tokens_out":6370,"duration_ms":61636,"significance":"If the reported results hold, the paper offers a practical, low-cost recipe for adapting RLMs to domains without verifiers, potentially opening up the large corpus of existing SFT data for reasoning-model adaptation. The manuscript has notable strengths: it releases code and datasets, reports a reproducible protocol (two seeds, ten evaluation runs), validates the LLM-as-a-judge setup with human correlations, gives a detailed runtime/cost breakdown, and is unusually candid about limitations, including a failed application to mathematical proofs. However, the central mechanistic claim — that calibration on non-empty trace rate is what selects a model that recovers lost reasoning — is not directly established. The positive results are empirical across the eight settings; the paper would be substantially strengthened by an analysis linking the calibration signal to actual reasoning quality and by scoping the claims accordingly.","major_comments":[{"comment":"The merge-ratio selection rule (Eq. 3) chooses α⋆ = max{α : ρ(Mα,Dcal) ≥ 0.9}, where ρ is the non-empty reasoning-trace rate (Eq. 2). The paper’s own §6 demonstrates that trace presence can be decoupled from useful reasoning: pre-filling a start-of-reasoning token with “Okay” produces a non-empty trace yet “performance of the untuned model was not recovered.” The paper never shows, across the α grid, that the set {α : ρ(Mα,Dcal) ≥ 0.9} coincides with the region where MATH500 is restored. It is therefore possible that the selected α* preserves “talking” without preserving reasoning; the good Table 1 outcomes could be particular to these eight settings rather than a consequence of the calibration criterion. Please add an α-sweep for at least one or two representative settings (e.g., OpenThinker 7B on Rust and text summarization) showing MATH500 and target-task accuracy alongside ρ(Mα,Dcal)","section":"§3, Eq. (2)–(3); §6 “Interpretability”"},{"comment":"The central claim that the method “recovers most or all of the lost reasoning capability” is operationalized only as (i) a non-empty trace rate on the target task and (ii) MATH500, a held-out mathematics benchmark. No evidence is provided that the target-task reasoning traces themselves are useful — e.g., that trace correctness correlates with final-answer correctness. Moreover, §6 states that the method “failed to improve” on mathematical proofs, so the claim should be scoped to tasks where IFT already supplies the output-side knowledge. Please either add a trace-usefulness analysis (for example, comparing per-example reasoning content of M vs. Mα* on a sample of target-task examples) or explicitly qualify the generalizability claim in the Abstract and Conclusion.","section":"§4.2, Table 1; §6 “Lightweight adaptation on reasoning domains”"},{"comment":"Algorithm 1 prescribes grid search over A_K = {j/K}, while §4.1 states the final results use “up to eight search steps and binary search.” The binary search relies on the asserted monotonicity of ρ(Mα,Dcal) in α, which is observed for some settings but not demonstrated for all (Figure 2 suggests near-monotonicity, but Olmo3 maintains 100% reasoning for all α, and the text itself notes a sharp drop around a critical ratio). Please clarify which search procedure actually produced Table 1, provide evidence for monotonicity or a caveat, and report the selected α⋆ per setting (currently absent from Table 1), as this is needed to interpret the merging results.","section":"§4.1 vs. Algorithm 1"}],"minor_comments":[{"comment":"The text says results are averaged over two seeds and ten evaluation runs, but Table 1 reports only point changes with no variance measures. Please include standard deviations or confidence intervals, at least for the main Merge rows.","section":"Table 1"},{"comment":"For Olmo3 7B, IFT does not degrade reasoning, so all methods reduce to the same IFT checkpoint. These rows cannot discriminate between methods and should be explicitly flagged as degenerate, so the reader does not count them as evidence for the merging step.","section":"Table 1 (Olmo3 rows)"},{"comment":"The claim “less than USD $3” is an average over eight settings; Table 5 shows Apriel Rust (112.7 min) and Qwen Rust (89.1 min) individually exceed $3. Please state “on average” or report the full distribution.","section":"§4.2, Runtime and cost"},{"comment":"The empty trace ε is used in Eq. (2) but is not defined until later in the same section. Define it immediately before or in the equation.","section":"§3, Eq. (2)"},{"comment":"ρ_min = 0.9 and the number of binary-search steps K are hand-set and not ablated. A brief sensitivity analysis over ρ_min would strengthen the method’s practical guidance.","section":"§3, hyperparameters"},{"comment":"Typo: “linear merginging” should be “linear merging.”","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical study with a useful practical outcome, and the authors are commendably transparent about limitations. The main blocker is that the advertised selection mechanism (non-empty trace rate) is not shown to be the reason the method works; the paper’s own §6 contains a counterexample to that mechanism. I would ask the authors to add the α-sweep analysis connecting the calibration signal to MATH500 and target-task performance, clarify the search algorithm, and scope the claims. If those revisions are made, the paper could be suitable for publication in this venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: this is a practical IFT-then-merge recipe for adapting open RLMs to domains without verifiers. The authors fine-tune on plain input-output pairs with empty reasoning traces, then linearly merge back toward the original RLM, choosing the merge ratio as the largest alpha that keeps the target-task reasoning rate at 0.9 or above on a small held-out calibration set. Across four RLMs, Rust coding, and summarization, the merged model improves task score over the unadapted model in all settings and keeps MATH500 within about a point of baseline in seven of eight. The specific combination is new, though the ingredients are not; the paper itself cites Lan et al. and prior merging work.\n\nThe empirical work is solid: held-out calibration, disjoint test sets, two seeds, ten evaluation runs, and ablations over merge methods, LoRA vs full fine-tuning, and hyperparameters. The comparison with OPD and KL is fair in spirit, and they even note those methods got a hyperparameter advantage. The case study on Apriel's type cast is a nice illustration.\n\nThe soft spot is exactly what the paper's own section 6 exposes. The calibration signal is non-empty trace rate, not reasoning quality. Prefilling the response with 'Okay' produces traces without recovering performance, so the selection rule can pick an alpha that restores talking without restoring useful reasoning. The authors never show that the set of alpha passing the rho>=0.9 filter coincides with the set restoring MATH500. Table 1's good outcomes might be a property of these models and tasks rather than a consequence of the calibration rule. That is a real limitation of the mechanism claim, though not of the final results.\n\nTwo smaller things: the abstract says 'less than USD $3' but that is an average; Apriel 15B on Rust costs about $6. And the method failed on mathematical proofs, so the domain scope is narrower than the abstract's broad phrasing suggests, though the paper is honest about this in section 6.\n\nWho this is for: anyone adapting RLMs to summarization or coding without a verifier; it is a cheap, reasonable baseline. I would send it to a serious referee: the method is useful, the evaluation is above average for this subfield, and the limitations are reportable rather than fatal. The referee should push for a reasoning-quality metric and a corrected abstract.","headline":"A practical IFT-then-merge recipe for adapting RLMs without verifiers — worth engaging, but the calibration signal is trace presence, not reasoning quality, and the abstract overstates cost.","tokens_in":25775,"tokens_out":2661,"would_cite":true,"duration_ms":27349,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Reasoning models can be adapted with only input-output pairs—no verifier, no reasoning traces—by instruction tuning then linearly merging back toward the original weights, choosing the ratio from non-empty reasoning rates on a calibration s","keywords":["reasoning language models","instruction fine-tuning","model merging","linear interpolation","reasoning-trace collapse","verifier-free adaptation","calibration-based alpha selection","code and summarization tasks"],"falsifier":"Run the pipeline on a verifiable reasoning task such as MATH500 itself, select α by the paper's rule (largest α with reasoning rate ≥ 0.9 on a calibration split), then measure answer accuracy on a held-out split. If the selected α keeps MATH500 accuracy at the IFT model's depressed level despite non-empty traces, or if the reasoning rate does not decrease monotonically with α so the binary search returns a wrong point, the calibration signal is shown to be insufficient. The paper's report that pre-filling 'Okay' yields traces without recovering performance is already a partial negative instanc","tokens_in":24778,"feed_emoji":"🧠","tokens_out":5682,"duration_ms":46436,"temperature":0.7,"pith_summary":"This paper is trying to establish a cheap, verifier-free recipe for adapting reasoning language models to tasks where only input-output pairs are available. It argues that the familiar failure mode—instruction tuning makes a reasoning model stop thinking—can be repaired by linearly interpolating the fine-tuned weights with the original reasoning model's weights. The only signal it uses to pick the interpolation coefficient is the fraction of calibration responses that contain a non-empty reasoning trace. The central finding is that this signal suffices: across four open reasoning models, the merged model keeps most of the task gain from instruction tuning while restoring held-out math reasoning to within about a point of the original model, all in under an hour and under $3. If true, large piles of existing supervised data become usable for reasoning models without any of the machinery usually needed.","feed_headline":"Merging restores reasoning lost by cheap fine-tuning for under $3","feed_subtitle":"No verifier, reward model, or stronger teacher needed; coding and summarization improve while math reasoning holds.","key_machinery":"The load-bearing mechanism is linear weight interpolation between two checkpoints of the same model: θ_α = (1−α)θ_original + α θ_IFT. The central object is the reasoning rate ρ(M′,D), the fraction of responses containing a non-empty reasoning trace, measured on a small target-task calibration set. The search picks α* = max{α : ρ(M_α, D_cal) ≥ 0.9}, exploiting the empirical monotonic decrease of reasoning rate with α. This lets the pipeline select a point close to the IFT model—where task knowledge is strongest—without ever checking answer correctness. The cheap calibration trick is that once a model stops reasoning, it emits the end-of-reasoning marker immediately after the start marker, so","core_discovery":"The paper's central claim is that a two-step procedure—standard instruction fine-tuning (supervised fine-tuning with empty reasoning traces) followed by linear model merging back toward the original reasoning model—recovers most or all of the reasoning capability lost by fine-tuning, while retaining a significant share of the target-task improvement. The merge ratio is chosen as the largest coefficient at which at least 90% of a small target-task calibration set still elicits a non-empty reasoning trace. The claim is supported on four open reasoning models and two tasks (Rust coding and text summarization), with MATH500 as a held-out check of general reasoning: the merged model keeps MATH500","pith_inferences":["Editorial inference: a testable extension is to apply the same α-selection rule on a domain where reasoning quality itself is verifiable (e.g., MATH500) and check whether the chosen α tracks answer accuracy, not just trace non-emptiness. The paper's own 'Okay'-prefill experiment shows traces can be produced without recovered performance, so trace rate is at best a proxy.","Editorial inference: if this holds broadly, the practical meaning is that almost any existing supervised fine-tuning dataset—human-written solutions, style data, knowledge patches—can be applied to a reasoning model with a one-hour training run and a simple merge step, demoting 'reasoning trace availability' from a blocker to a non-issue for many applications.","Editorial inference: the paper's exploratory result that sequential application of the method loses reasoning faster suggests a scaling ceiling: a single train-and-merge step works, but stacking many such steps is not yet a solution. A natural next question is whether repeatedly merging against the original model, or using a fused anchor, behaves differently.","Editorial inference: the method's boundary is visible in its own discussion—on domains where reasoning gains define performance (like mathematics), input-output-only adaptation fails. This points to a division of labour: trace supervision remains necessary where the thinking itself is the product; this recipe is for inserting knowledge or aligning style."],"forward_implications":["Standard instruction tuning on reasoning models is not just suboptimal but actively harmful to reasoning behavior; the paper documents drops of up to 77 points on MATH500 and 100% to 0% reasoning rates.","The recovered-reasoning benefit shows up concretely in task performance: on Rust coding, the merged model improves by an average of 7.0 points over the unadapted model, versus 3.8 from IFT alone, because the model both knows the task and still thinks about it.","The method costs less than $3 on a single H200 GPU, 22.8% and 33.0% less than the on-policy distillation and KL-regularization baselines, with hyperparameter search for the merge ratio included.","The pipeline works in both verifiable (coding) and hard-to-verify (summarization) domains, since it never needs a reward or correctness signal.","Merging techniques beyond linear (SLERP, TIES) preserve more MATH500 ability but give less target-task gain, so plain linear merging is the recommended default in this setting."],"fun_headline_variants":["Merge after SFT restores reasoning for under $3","Two-step trick: SFT then merge to keep RLM skills","No verifier, no teacher: cheap method boosts RLMs","Cost-effective reasoning recovery via SFT plus merge","For $3, merge fixes fine-tuned reasoning models"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The recipe assumes the fraction of non-empty reasoning traces on a small calibration set is a trustworthy proxy for restored reasoning quality, and that this rate drops monotonically as the merge ratio grows; if a model can produce traces without useful reasoning—as the paper's own prefill experiment shows—the chosen α may restore talking, not thinking.","fun_headline_variants_meta":{"raw":{"variants":["Merge after SFT restores reasoning for under $3","Two-step trick: SFT then merge to keep RLM skills","No verifier, no teacher: cheap method boosts RLMs","Cost-effective reasoning recovery via SFT plus merge","For $3, merge fixes fine-tuned reasoning models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000243,"raw_usage":{"total_tokens":1344,"prompt_tokens":703,"completion_tokens":641,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":447,"completion_tokens_details":{"reasoning_tokens":559}},"tokens_in":447,"tokens_out":641,"duration_ms":5748,"temperature":1.0,"reasoning_tokens":559,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T00:46:11.350191+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the pipeline on a verifiable reasoning task such as MATH500 itself, select α by the paper's rule (largest α with reasoning rate ≥ 0.9 on a calibration split), then measure answer accuracy on a held-out split. If the selected α keeps MATH500 accuracy at the IFT model's depressed level despite non-empty traces, or if the reasoning rate does not decrease monotonically with α so the binary search returns a wrong point, the calibration signal is shown to be insufficient. The paper's report that pre-filling 'Okay' yields traces without recovering performance is already a partial negative instanc","supporting_citations":[],"review_version":1}