{"id":"45994f11-a5d3-4d2d-8054-c3b700a64272","arxiv_id":"2607.18026","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"After truncating or expanding checkpoints to a shared shape, small-ratio weight averaging slightly improves average benchmark scores over strong Qwen sources, but headline gains are inflated by per-task best-ratio selection.","lead":"The paper tests whether two differently sized Qwen LLMs can be combined by simply resizing one to match the other and averaging the weights, with no training or alignment. It reports modest average-score gains, but the largest reported gains are per-benchmark best-ratio values, not the scores of a single merged model.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported 'best merged model' averages are per-task maxima over the ratio grid; no single checkpoint hits 0.7525 or 0.5716, so the headline quantitative claims are not supported by any actual model.","rationale":"The Reader's verdict (CONDITIONAL) already identified selection-on-eval as a secondary concern and noted in the rationale that the 'Merged' columns are per-task maxima. However, the Reader's formally stated weakest_assumption is architecture compatibility. I find the per-task-max aggregation to be the single most load-bearing issue because it directly falsifies the paper's strongest quantitative claim as written (that a single merged model reaches 0.7525 and 0.5716) and is an internal inconsistency between the protocol in Eqs. (5)-(6) and the reported tables. The corrected fixed-ratio results still show modest improvements in several settings, so the paper's qualitative conclusion is not fully refuted; the correct response is to require revision and disclosure of the corrected numbers, not outright rejection. This aligns with the Reader's CONDITIONAL verdict, so my analysis does not change the verdict. I credit the paper for its perturbation controls (Appendix C) and explicit collapse diagnostics, which limit the damage from this aggregation flaw.","tokens_in":22516,"tokens_out":6982,"duration_ms":68328,"concrete_test":"Recompute the averages in Tables 10 and 13-16 column-wise: for each fixed λ (0.98, 0.96, 0.04, 0.02) and each fixed μ (0.01-0.05), average the 16 task scores. Verify whether any fixed-ratio column reaches the reported 'Merged' averages (0.7525 and 0.5716). Then take the best fixed-ratio checkpoint for each setting and compute a bootstrap or McNemar-style significance test against the stronger source over the 16 tasks to see if the corrected margins (e.g., 0.0057 for P1, 0.0095 for P2) are statistically reliable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central quantitative claims are undermined by an aggregation inconsistency. Section 3.4 defines the merged model as the single checkpoint selected by minimizing validation loss over the ratio grid (Eqs. 5-6). However, the 'Merged' columns in Tables 3-5 and 6-9 are not scores of any single checkpoint: they are per-task maxima across the grid. For P1 (Qwen2.5-14B/32B union), the claimed average 0.7525 is the mean of the 16 task-wise best scores taken from λ ∈ {0.02,0.04,0.96,0.98}; the best fixed-λ average is 0.7481 (λ=0.98), only 0.0057 above the stronger source (0.7424) and likely within evaluation noise. For P2 intersection (Qwen2.5-3B/truncated-32B), the claimed 0.5716 is the mean of per-task best over μ; the best single-μ average is 0.5554 (μ=0.01), a gain of 0.0095 over the 0.5459 baseline. The WORST columns (e.g., 0.7036 for P1) show the same grid can average below the smaller source, confirming that no fixed-ratio model simultaneously achieves the per-task bests. Thus the abstract and Section 5 overstate the result: the existence of a single merged checkpoint beating both sources is not established, and the true margins are much smaller. The qualitative phenomenon may survive, but the paper's most striking numbers do not.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes training-free heterogeneous LLM merging by deterministic dimensional adaptation (expansion for union-style, truncation for intersection-style) followed by ratio-controlled convex interpolation between the adapted checkpoints. It reports experiments on Qwen2.5/Qwen3 model pairs (3B–32B) across math, code, NLU, commonsense, knowledge, and instruction-following benchmarks, claiming that small-ratio interpolation can improve over both source checkpoints without training, adapters, routing, or semantic alignment. The paper also identifies ratio-sensitive regimes (preservation, transfer, collapse) and a task-level seesaw effect.","tokens_in":22910,"tokens_out":7820,"duration_ms":77111,"significance":"If the central claim were fully supported, the paper would establish a remarkably simple baseline: deterministic tensor-slot expansion/truncation plus small-ratio weighted averaging can merge heterogeneous LLM checkpoints with zero training and zero alignment. The protocol is clearly specified in Eqs. (7)–(8), and the appendix gives unusually detailed per-ratio task-level scores (Tables 10–16), which is a genuine strength. The perturbation controls in Table 19 and the comparison with TIES, SLERP, and DARE in Table 20 are also useful diagnostics. However, the headline quantitative claims are not supported as stated: the 'Merged' columns in Tables 3–9 are per-task maxima over the ratio grid, not the scores of any single checkpoint. When a single fixed-ratio model is considered, the reported average improvements shrink to at most about 0.005–0.01, with no error bars, and the ratio is selected on the same benchmarks later reported as the outcome. The qualitative phenomena may survive, but the paper’s main evidence needs substantial revision.","major_comments":[{"comment":"The 'Merged' columns are not scores of a single checkpoint; they are per-task bests over the ratio grid. For P1, Table 3's gsm8k=0.9045, humanevalplus=0.6494, ifeval=0.5072, and BBH=0.8241 come from different λ values (0.98, 0.96, 0.96, 0.96 respectively), and the average 0.7525 equals the mean of the BEST column in Table 10. The best single-λ average is 0.7481 at λ=0.98 versus source B at 0.7424. Likewise, Table 6's 0.5716 is the mean of per-task bests from Table 13; the best single-μ average is 0.5554 at μ=0.01 versus 0.5459. The abstract and §5 make claims about a 'best merged model' that no actual model realizes. Please re-report all headline results as single fixed-ratio checkpoints, or explicitly relabel these as per-task oracle upper bounds and adjust every conclusion accordingly.","section":"§5 and Tables 3–9 vs. Appendix A (Tables 10–16)"},{"comment":"The ratio λ (or μ) is selected by minimizing L_eval on the same benchmark suite that is later reported as the outcome, and the 'best' result is then the maximum over the grid. This makes the reported gains in-sample selection artifacts rather than independent predictions. Even when a single ratio is used, choosing it on the test benchmarks overstates expected transfer. The paper needs a proper validation split, or at minimum it must present all grid averages and frame the results as a descriptive upper envelope. The perturbation controls in Table 19 are helpful but do not address this selection issue.","section":"§3.4, Eqs. (5)–(6)"},{"comment":"The claimed phase diagram (λ∈(0,0.1)∪(0.9,1) stable, λ∈(0.1,0.9) collapse; μ∈(0,0.1) stable, μ∈(0.1,1) collapse) is based on one GSM8K pair (Qwen2.5-14B/32B) in the appendix, with no task-level or pair-level corroboration in the main text. The terms 'flexible' and 'strict' in Tables 21–22 are never defined. If ratio sensitivity is a central contribution, it needs definitions, error bars, and evidence across more than one task/model pair before it can be presented as a general regime.","section":"§6.2 and Tables 21–22"},{"comment":"Once the per-task-best aggregation is corrected, the claimed intersection-style gains mostly vanish. For example, Table 14 shows the best single-μ average for Qwen2.5-14B is 0.7086 (μ=0.04) versus baseline 0.7045, a gain of 0.0041; Table 16 shows Qwen3-4B improving from 0.6277 to 0.6292, a gain of 0.0015. These margins are within evaluation noise, and no error bars or repeated runs are reported for any main table. The statement that truncation injection 'provides a useful performance gain' needs statistical support or must be substantially softened.","section":"§5.2 and Tables 6–9"}],"minor_comments":[{"comment":"The tables contain unexplained parenthetical annotations such as '(+0.03)', '(+0.01)', and '(+0.02)' next to Base B or Base A entries. These appear to be editing remnants and should be removed or explained. Also, 'A vg.' should be 'Avg.'.","section":"Tables 3–9"},{"comment":"Please define 'flexible' and 'strict' and describe how the reported variance is computed. Without definitions, these diagnostic tables are hard to interpret.","section":"Appendix C, Tables 21–22"},{"comment":"The figure claims to clarify 'knowledge-structure compatibility', but no method is given for computing the competence profiles or their overlap. As presented, it is an illustration rather than evidence; please either add a concrete computation or label it as a schematic.","section":"Figure 3 and §5.1"},{"comment":"The assumption that tensor names, module roles, and head structure are 'sufficiently comparable' is stated only in the appendix. Since this substantially narrows the meaning of 'heterogeneous', it should be stated prominently in the abstract or introduction as a scope condition.","section":"Appendix B.2 (last paragraph)"},{"comment":"No code, checkpoint, or artifact link is provided, and no computational details (e.g., evaluation harness versions, number of runs) are given. Please include these for reproducibility.","section":"Reproducibility"}],"recommendation":"major_revision","confidential_remarks":"The paper needs a complete re-analysis of the headline tables under a fixed-ratio protocol. If the authors cannot produce a single-checkpoint improvement beyond noise with a proper validation split, the contribution becomes a cautionary/negative result rather than the strong positive baseline claimed. I recommend major revision rather than rejection because the flaw is fixable in principle and the appendix contains enough information to re-evaluate the claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's central empirical claim doesn't survive contact with its own appendix. The 'Merged' columns in Tables 3–5 and 6–9 are per-task bests over the ratio grid, not the scores of any single checkpoint. The abstract's 0.7525 and 0.5716 are means of task-wise maxima. The best fixed-ratio averages are 0.7481 (λ=0.98) and 0.5554 (μ=0.01)—gains of 0.0057 and 0.0095 over the stronger sources, likely inside evaluation noise. So the paper's most striking numbers don't support the existence of a single merged model that beats both sources.\n\nWhat's genuinely new is the systematic framing: deterministic zero-init expansion and prefix-slice truncation as compatibility operators, then small-ratio interpolation, tested across Qwen2.5/3 pairs from 3B to 32B. The ratio-regime diagnostic—preservation, transfer, collapse—is useful, and the perturbation controls in Appendix C are a good-faith attempt to rule out generic-noise explanations. The paper also states its assumptions plainly, including the architecture-compatibility requirement in Appendix B.\n\nThe soft spots are proportional. The selection-on-eval issue is load-bearing: Eqs. (5)–(6) pick λ/μ by minimizing L_eval on the same benchmarks later reported, so the per-task best results are fitted values, not predictions. The collapse phase rests on one GSM8K pair; there are no error bars and no released code or checkpoints. The qualitative 'small-ratio can help' phenomenon may survive a corrected analysis, but the headline margins likely don't.\n\nThis paper is for practitioners who want a cheap zero-training baseline and for method developers who need a sanity check on heterogeneous merging. It deserves a serious referee, but only after the authors fix the aggregation inconsistency and re-report single-checkpoint results with a proper train/eval split. My recommendation: send to peer review with major revisions expected.","headline":"Worth a look for the ratio-regime idea, but the headline numbers are per-task maxima over the ratio grid—no single merged model achieves them.","tokens_in":23406,"tokens_out":2815,"would_cite":false,"duration_ms":28661,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Heterogeneous LLM merging can succeed with deterministic dimensional adaptation plus small-ratio weighted averaging, without training or semantic alignment.","keywords":["heterogeneous model merging","weighted averaging","model interpolation","dimensional adaptation","expansion and truncation","LLM merging","ratio sensitivity","Qwen family"],"falsifier":"A decisive check: choose lambda and mu on a held-out validation split and only then evaluate on the benchmarks used for reporting; if the selected merged checkpoint no longer beats both source averages, the headline gains are selection artifacts rather than transferred capability.","tokens_in":22392,"feed_emoji":"🧩","tokens_out":4924,"duration_ms":54911,"temperature":0.7,"pith_summary":"The paper tries to establish that large language models of different sizes can be merged by direct weighted averaging, provided their weights are first projected into a common parameter space and the mixing ratio is kept small. It proposes two deterministic projections: expanding the smaller model up to the larger model's dimensions, and truncating the larger model down to the smaller model's dimensions. Across Qwen-family pairs from 3B to 32B parameters, small-ratio interpolation often beats both source checkpoints on average by transferring complementary capabilities, while near-balanced averaging collapses. If correct, this provides a zero-training, zero-alignment baseline for heterogeneous LLM merging and suggests that the limits of simple weight fusion may bound what more complex merging methods can achieve at scale.","feed_headline":"Weight averaging merges different-sized LLMs without retraining","feed_subtitle":"Expand the smaller model or truncate the larger, then mix at small ratios to beat both checkpoints.","key_machinery":"The central machinery is a pair of deterministic tensor maps followed by convex interpolation. EXPAND places the smaller checkpoint into the larger architecture by copying compatible tensors, zero-filling new attention and MLP coordinates, and initializing added layers as residual identities; TRUNCATE projects the larger checkpoint into the smaller architecture by copying shape-compatible tensors, slicing oversized tensors along head and MLP axes, and dropping extra layers. Interpolation is performed as theta_union(lambda) = (1-lambda) E(theta_small) + lambda theta_large or theta_intersection(mu) = (1-mu) theta_small + mu T(theta_large) with small ratios. This works by keeping the merged poi","core_discovery":"The paper claims that two differently sized LLMs from the same family can be merged by first projecting one into the other's parameter space—expanding the smaller or truncating the larger—and then interpolating with a small mixing ratio. On Qwen-family pairs from 3B to 32B, deterministic expansion largely preserves the smaller model's behavior, and small-ratio interpolation transfers complementary capabilities: for example, union merging a 14B and a 32B checkpoint reaches an average of 0.7525 versus 0.7044 and 0.7424 for the two sources, and intersection merging improves a 3B model from 0.5459 to 0.5716 by injecting a truncated 32B branch with a small coefficient. The paper also reports that","pith_inferences":["If the ratio-sensitivity phase pattern holds across more model families, the collapse boundary could be predicted from representation-similarity or mode-connectivity measures, letting practitioners locate safe mixing ratios without a full benchmark sweep.","The same expand-and-truncate operators could be applied to checkpoints whose hidden dimensions are permuted rather than aligned; if gains vanish under a fixed permutation, the transfer depends on matching coordinate roles, not merely on adding structured perturbation.","A testable extension is to use the truncated large model as a parameter-space prior and then briefly fine-tune the merged checkpoint; if small-ratio averaging already captures complementary skills, it may also provide a better starting point for further training."],"forward_implications":["A zero-training, zero-alignment merging recipe exists for architecture-compatible LLM families: deterministic expansion or truncation plus small-ratio interpolation can yield a single checkpoint that beats both source models on average.","Union-style merging has two usable endpoint neighborhoods (lambda below about 0.1 and above about 0.9), while intersection-style merging is usable only at very small mu; near-balanced interpolation should be avoided.","Intersection merging transfers most when the capability gap is large, as when a truncated 32B branch improves a 3B model from 0.5459 to 0.5716, while smaller gaps produce milder gains.","Aggregate improvements hide task-level regressions, so a merged checkpoint must be validated per task before deployment even when its average score exceeds both sources."],"fun_headline_variants":["Expand and truncate: merging LLMs of different sizes","Weighted averaging merges heterogeneous LLM checkpoints","Training-free dimensional adaptation for direct LLM fusion","Small-ratio interpolation improves Qwen model merging","A simple baseline for heterogeneous LLM merging"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that after expansion or truncation, copied and sliced weights still carry out the same computation in both checkpoints; if tensor names, module roles, and head structure do not align, small-ratio averaging loses its semantic basis and the reported gains disappear.","fun_headline_variants_meta":{"raw":{"variants":["Expand and truncate: merging LLMs of different sizes","Weighted averaging merges heterogeneous LLM checkpoints","Training-free dimensional adaptation for direct LLM fusion","Small-ratio interpolation improves Qwen model merging","A simple baseline for heterogeneous LLM merging"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000982,"raw_usage":{"total_tokens":4016,"prompt_tokens":766,"completion_tokens":3250,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":510,"completion_tokens_details":{"reasoning_tokens":3177}},"tokens_in":510,"tokens_out":3250,"duration_ms":29393,"temperature":1.0,"reasoning_tokens":3177,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T16:18:22.281230+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive check: choose lambda and mu on a held-out validation split and only then evaluate on the benchmarks used for reporting; if the selected merged checkpoint no longer beats both source averages, the headline gains are selection artifacts rather than transferred capability.","supporting_citations":[],"review_version":1}