{"id":"9e33eb79-0eec-4081-a13a-ca3788695466","arxiv_id":"2607.11997","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"The optimal training duration for expert models before merging depends on the merging method; sparsification-based merging benefits from overtrained experts.","lead":"This paper tests whether fine-tuned expert models should be stopped early or overtrained before merging them into one multi-task model. It finds that the best training length depends on the merging method, with sparsification-based methods improving when experts are overtrained.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No error bars or multiple seeds: reported gains for sparsification methods over T* could be within run-to-run noise, undermining the central empirical claim.","rationale":"The reader's weakest assumption—single-seed runs and no error bars—is the most load-bearing concern because it threatens the entire empirical basis. If the observed differences are within noise, the central claim of method-dependent optimal training duration loses support. This is more fundamental than secondary issues like overgeneralization in the conclusion, which would persist even with perfect reproducibility. The paper's own Section 6 admits the limitation but does not quantify its impact. A conditional accept is appropriate: the work is plausible and internally consistent, but the central empirical claim needs reproducibility evidence before it can be fully accepted.","tokens_in":11040,"tokens_out":12488,"duration_ms":121314,"concrete_test":"Run 5 independent fine-tuning runs (different seeds) for the five experts at each model size, using the same QLoRA recipe. Merge with TIES at T*, 1.5×, 2×, 3×, and 5×, and with DARE+TIES at T* and 5×. For each seed, compute merged accuracy on the four benchmarks. Test whether the mean difference (best overfitted duration minus T*) has a 95% bootstrap CI excluding zero for each method/size. If any CI includes zero, the claimed benefit is not established. Also vary the DARE drop seed and Greedy Soup selection seed across 10 runs to capture stochastic-method variance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central finding—that TIES and DARE+TIES peak at 1.5–5×T* with gains of 2.5–8.7 points over T*—rests on single training runs per configuration. No error bars, no multiple seeds. The fine-tuning process (data order, initialization, nondeterminism) introduces variance, and DARE and Greedy Soup add stochastic components with a single seed. Because the 'best duration' is selected as the maximum over 8 noisy checkpoints, the apparent peak is inflated by noise; even a flat underlying curve would produce a best duration by chance. Section 6 acknowledges the single-seed limitation but does not quantify it. If the observed differences (e.g., TIES at 4B: 59.8 vs 51.1 at T*) are within run-to-run noise, the method-dependent pattern and the practical recommendation collapse. The reader's weakest assumption correctly identifies this as the load-bearing risk.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies how the training duration of domain experts (relative to their individual validation optimum T*) affects the quality of merged multi-task LLMs. The authors fine-tune Qwen 3.5 models at three scales (0.8B, 2B, 4B) on five domains using QLoRA, save checkpoints from 0.25× to 5×T*, and evaluate five merging methods (Simple Averaging, Task Arithmetic, TIES, DARE+TIES, Greedy Soup). They report a method-dependent pattern: Simple Averaging peaks with undertrained experts, Task Arithmetic/Greedy Soup are intermediate, and sparsification-based methods (TIES, DARE+TIES) peak with overtrained experts. They explain this via a bias-variance-covariance decomposition and linear mode connectivity, drawing an analogy to random forests, and recommend training experts 2–5×T* when using sparsification-based merging.","tokens_in":11270,"tokens_out":4773,"duration_ms":46327,"significance":"If the empirical pattern holds, the paper challenges a widespread convention — merging experts at their individual validation optima — and offers concrete practical guidance for choosing expert training duration together with the merging method. The study is systematic in covering multiple model sizes, domains, and merging methods, and it transparently acknowledges several limitations (single model family, QLoRA/rank r=16, constant LR, single seed for stochastic methods). The random-forest analogy is attractive and the bias-variance account is a plausible explanation. However, the central claim is entirely empirical and currently rests on single training runs without error bars; moreover, the specific quantitative recommendation (2–5×T*) is not consistently supported by the Appendix tables. The paper's significance will depend on whether the key differences are statistically robust and whether the practical guidance is stated accurately.","major_comments":[{"comment":"The central empirical claim — TIES/DARE+TIES peak after T* with gains of 2.5–8.7 points, while Simple Averaging degrades by 14–23 points — is based on a single training run per configuration; Greedy Soup and DARE's stochastic pruning use a single seed. Since the 'best duration' is selected as the maximum over 8 noisy checkpoints, the apparent peak is inflated by selection noise even if the true curve is flat. Section 6 acknowledges the single-seed limitation but does not quantify its impact. Please report mean±std over at least 3 seeds for a representative subset (e.g., 4B, all methods at 1.0/1.5/2.0/3.0/5.0×T*) and/or paired significance tests. Without this, the reported method-dependent pattern could be within run-to-run noise.","section":"§6, Tables 7–9"},{"comment":"The conclusion that 'sparsification-based methods (TIES, DARE+TIES) achieve peak performance with deliberately overfitted experts (2–5 × T*)' is contradicted by the Appendix's per-domain optimal-duration tables. For example, DARE+TIES achieves its best Instruction score at 0.25×T* for Qwen3.5-2B (Table 8), at 1.5×T* for Safety on 0.8B (Table 7), and TIES is best at 1.0×T* for 2B Safety (Table 8). The overfitted-expert pattern holds on average and for many accuracy domains, but the stated 2–5×T* range is too broad. Please qualify the practical guidance to the domains/scales where it is supported, or explain why these exceptions are consistent with the claimed mechanism.","section":"§5.1, Figure 2"},{"comment":"The Abstract and Contributions state that the results are 'formalized through bias-variance decomposition,' but Section 5.1 gives no equations, no explicit estimators for bias/variance/covariance of a merged weight-space model, and no formal link between the decomposition and the merging methods. Section 6 later concedes this is 'an explanatory hypothesis rather than a proven mechanism.' Either provide the actual decomposition (definitions, assumptions, how it is computed from the fine-tuned models) or soften the 'formalize' claim. The random-forest analogy is useful as intuition, but as written the formalization is not delivered.","section":"Appendix A.1"}],"minor_comments":[{"comment":"The sentence 'For every method and size, the best training duration outperforms T*' is tautological, since T* is one of the evaluated checkpoints. Replace with a meaningful comparison, e.g., the gain of the best checkpoint over T* with a significance test or confidence interval.","section":"Table 5"},{"comment":"The 'Base (no FT)' row for Qwen3.5-4B Code is reported as '–'. If the base model was not evaluated on HumanEval at that scale, state this explicitly; otherwise fill the missing value.","section":"Figure 1"},{"comment":"The caption says 'Optimal T steps is shown as method-specific,' which is unclear. T* is defined per domain and model size in Section 3; clarify what 'method-specific' means here.","section":"§4.1"},{"comment":"The description of Greedy Soup as showing 'a more balanced selection at 4B' is vague. Quantify the selected subsets (e.g., average number of experts retained) if this is meant to support the interpretation.","section":"§5.1"},{"comment":"Figure 2 would be more informative with axis labels/units and a brief description of how each component is computed; currently it is hard to verify the claimed decrease in bias and increase in variance/covariance.","section":"References"},{"comment":"Several references are to works dated 2026 (Tran et al., Rofin et al.). Please verify these are publicly available or indicate preprint status correctly.","section":null}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is timely and the systematic sweep is a strength, but the empirical foundation is currently too fragile: single runs, no error bars, and checkpoint-selection bias. In addition, the headline practical recommendation (2–5×T* for sparsification methods) is not supported by the paper's own per-domain tables. With additional seeds and a more nuanced conclusion, the contribution could become solid; without them, the central claim is not yet established."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's central claim—sparsification-based merging methods peak when experts are trained 1.5–5× past their own validation optimum—is plausible and practically useful, but the single-run, no-error-bar experiments leave the actual gains unquantified. I'd hold off on changing my merging recipe until seeds confirm it.\n\nWhat's new: this is the first systematic study of expert training duration for decoder-only LLM merging. The contrast with Horoi et al. (2025) is clear: they found overtraining universally harmful for CLIP/T5; here the picture is method-dependent. Simple averaging degrades sharply, TIES and DARE+TIES improve past T*, with the optimum shifting later as model size grows. The practical guidance—train past T* when using sparsification—is actionable, and the safety results add a nice extra dimension.\n\nThe study itself is solid in scope: three model sizes, five domains, five merging methods, checkpoints from 0.25× to 5× T*. The per-domain tables are thorough, and Section 6 is candid about the limitations.\n\nThe soft spots, in order of severity. First is noise. Every configuration is a single training run; DARE and Greedy Soup each use one seed. The claimed 2.5–8.7 point gains for TIES are the maximum over eight checkpoints, so even a flat curve would produce a best duration by chance. The authors acknowledge this, but don't quantify it. Second, the bias-variance-covariance analysis is more illustrative than rigorous: they apply DiWA's decomposition, but don't show how the components were estimated, and the random-forest analogy is a heuristic, not a proof. Third, the results come from one model family and one QLoRA recipe, which the authors note. These are addressable, not fatal.\n\nWho it's for: model merging researchers and practitioners who need to decide where to stop training experts. With error bars, this would be a confident find; without, it's a suggestive workshop contribution. My recommendation: send to peer review, but require multiple seeds and error bars for the headline claims.","headline":"Sparsification-based merging may benefit from overtrained experts, but the single-seed results don't yet quantify the gains; worth a serious referee.","tokens_in":11722,"tokens_out":3388,"would_cite":false,"duration_ms":32576,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The optimal training duration for LLM experts before merging is method-dependent: sparsification-based methods peak with overfitted experts while simple averaging degrades sharply.","keywords":["model merging","expert training duration","early stopping","sparsification","TIES","DARE","bias-variance decomposition","LoRA"],"falsifier":"Run the same experiment with at least five seeds per configuration and compare TIES' merged accuracy at T* against its accuracy at 2–3× T*; if the mean difference is within one standard deviation of the seed spread, the method-dependent pattern would collapse.","tokens_in":10962,"feed_emoji":"🧩","tokens_out":3887,"duration_ms":35474,"temperature":0.7,"pith_summary":"This paper asks whether the common practice of fine-tuning each expert model to its individual validation loss minimum before merging is actually the right stopping point. Across three model sizes, five domains, and five merging methods, it finds the answer is no: the optimal training duration depends on the merging method. Simple weight averaging peaks with undertrained experts and degrades sharply if experts overfit, while sparsification-based methods such as TIES and DARE+TIES keep improving when experts are trained to 2–5 times their validation optimum. The paper explains the pattern through a bias-variance decomposition of merged-model error, arguing that sparsification removes the noise that overfitting adds while preserving the lower-bias task knowledge. If correct, the result changes a standard design choice in multi-task model merging: training duration and merging method should be selected together.","feed_headline":"Stop early for averaging, overfit for sparsified merging","feed_subtitle":"A study across three LLM sizes shows the best training duration depends on the merging method, not just the expert.","key_machinery":"The bias-variance-covariance decomposition of merged-model error, adapted from weight-averaging theory, is the central lens: as training passes T*, bias falls while variance and covariance rise. Sparsification-based merging methods (TIES, DARE+TIES) act as variance-reduction operators that remove small-magnitude or sign-conflicting parameter updates before averaging, analogous to how averaging in random forests tolerates deeper, higher-variance trees. This decomposition, together with linear mode connectivity measurements, explains why the optimal training duration shifts with the merging method.","core_discovery":"The paper's central claim is that expert training duration and merging method are coupled, not independent choices. Empirically, overfitted experts (2–5× the per-expert validation-optimal step T*) yield the best merged models under sparsification-based methods, while simple averaging performs best in the undertrained regime (0.25–1× T*) and loses 14–23 points by 5× T*. The mechanism proposed is that overtraining lowers bias (experts capture more task-specific signal) but raises variance and inter-expert covariance; methods like TIES and DARE+TIES filter out exactly this added noise through trimming, sign resolution, and stochastic pruning, thereby converting high-variance experts into a bett","pith_inferences":["If the single-seed limitation is addressed and the pattern holds, this would reconcile conflicting reports in the merging literature about whether overtraining helps: the answer may be method-dependent rather than universal.","The random-forest analogy suggests a testable prediction: merging methods with stronger variance reduction (e.g., lower density k) should shift the optimal training duration even further past T*.","The results imply that benchmark comparisons of merging methods may be unfair if all experts are at their individual T*; each method should be evaluated at its own optimal duration.","The reliance on a constant learning rate may amplify the overtraining regime; under learning-rate decay, the optimal overtraining multiple could shrink, a question the paper itself leaves open."],"forward_implications":["If a practitioner uses TIES or DARE+TIES, they should deliberately train each expert past its validation optimum (2–5× T* for the tested family) instead of early stopping.","If using simple averaging, experts should be undertrained (0.25–1× T*) to avoid degrading the merged model.","The choice of training duration and merging method should be optimized jointly, not sequentially.","Sparsification's variance-reduction role gives a practical rule of thumb: the stronger the merge's interference resolution, the more overfitting the experts can tolerate.","The observed rightward shift of the optimum with model size suggests larger backbones reward even more aggressive overtraining for sparsification methods."],"fun_headline_variants":["Overfit for merges? Only with sparsified methods","Merge method picks your training sweet spot","Don't stop at validation loss for merging","Averaging wants undertrained, sparsifying wants overfit","Training duration and merge method are a package deal"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The central pattern is empirical, and every result comes from a single training run per configuration with no error bars, so the claim assumes the reported gaps are larger than run-to-run noise.","fun_headline_variants_meta":{"raw":{"variants":["Overfit for merges? Only with sparsified methods","Merge method picks your training sweet spot","Don't stop at validation loss for merging","Averaging wants undertrained, sparsifying wants overfit","Training duration and merge method are a package deal"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000175,"raw_usage":{"total_tokens":1111,"prompt_tokens":719,"completion_tokens":392,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":463,"completion_tokens_details":{"reasoning_tokens":331}},"tokens_in":463,"tokens_out":392,"duration_ms":4100,"temperature":1.0,"reasoning_tokens":331,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T06:45:50.095323+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same experiment with at least five seeds per configuration and compare TIES' merged accuracy at T* against its accuracy at 2–3× T*; if the mean difference is within one standard deviation of the seed spread, the method-dependent pattern would collapse.","supporting_citations":[],"review_version":2}