{"id":"b76d119f-3f28-405c-bc22-9d1150465750","arxiv_id":"2509.11167","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Optimization Trajectory Aware merging uses Adam second moments as a curvature proxy, first pruning task-vector edits with Fast Fisher Grafting, then reweighting survivors with a compressed curvature preconditioner.","lead":"This paper merges five specialized instruction-tuned Llama-3.1-8B checkpoints into one model by using optimizer second-moment statistics as a proxy for loss curvature. The method prunes low-saliency fine-tuning edits before curvature-weighted averaging, and it reports improved merged benchmark scores over several baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"OTA's curvature-aware aggregation uses sqrt(v), not the v that Eq. 5 and the Hessian/Fisher equivalence justify; the 'curvature-informed' weighting in Eq. 7 is internally inconsistent.","rationale":"The empirical core of the paper is a solid ablation story: FFG's saliency masking is the main driver, and the promised open-source artifacts will help reproducibility. My concern is with the 'curvature-informed' framing. In Eq. 5, v is used as the diagonal Hessian proxy; in Eq. 7, the same v enters as sqrt(v). These two uses are not interchangeable: if v ≈ F, the PSD weight C_τ in Eq. 4 should be v, and Adam's actual preconditioner is 1/(sqrt(v)+ε), not sqrt(v)+ε. No argument in §4.3 derives sqrt(v) as the curvature metric. The ablation OTA(wo FFG)=0.536 < Linear=0.551 suggests the sqrt-weighted aggregation is not reliably acting as a beneficial curvature correction on unpruned task vectors, so the improvement of full OTA over FFG-TA relies on an interaction whose mechanism is not established. This is not a rejection of the empirical method; it is a request for a direct test of the claimed mechanism. The reader's conditional verdict remains appropriate, but the specific internal inconsistency in Eq. 7 is a sharper version of the 'curvature-weighting is not Fisher-derived' concern.","tokens_in":23282,"tokens_out":11650,"duration_ms":152088,"concrete_test":"Reproduce Table 3's OTA and OTA-rank1 rows with P_τ = Diag(v_τ + ε) in Eq. 7 (and in the compressed variant), keeping FFG masks and sparsity tuning identical. Also compute the diagonal observed empirical Fisher on a held-out subset of each SFT dataset at the final checkpoint and rank-correlate it with v_τ and with √v_τ. If the v-weighted OTA row is no better than linear, or if √v_τ correlates much worse with the empirical Fisher than v_τ does, then the claimed curvature-aware aggregation in Eq. 7 is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.1 concludes that Adam's second moment v is a scaled EMA of the diagonal FIM, so H_ii ≈ v_i. FFG's saliency (Eq. 5) correctly uses v_i. However, the OTA aggregation in Eq. 7 uses P_{τ,i} = √v_{τ,i} + ε as the curvature metric in Eq. 6. If v is the diagonal Fisher, the PSD weights in the generalized merging rule (Eq. 4) should be C_τ = Diag(v_τ), as in Fisher merging; using √v is neither the Fisher weight nor Adam's actual preconditioner (which is 1/(√v+ε)). No argument in §4.3 derives √v as the curvature metric. The ablation evidence is consistent with this suspicion: OTA without FFG (curvature-weighted full task vectors) scores 0.536, below plain linear averaging (0.551), so the sqrt-weighted aggregation is not acting as a beneficial curvature correction on unpruned vectors. Combined with the fact that exp_avg_sq is an EMA over the whole trajectory (not a final-point Fisher) and Assumption 2 (perfect calibration) is unlikely after one epoch, the paper's central 'curvature-informed' interpretation is not established; the gains are currently attributable to a saliency heuristic (FFG) plus an ad hoc weighting. The empirical method may still be useful, but the theoretical claim goes beyond what the equations support.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes OTA-Merging, a two-stage recipe for merging multiple SFT checkpoints of an LLM. The method uses Adam's exp_avg_sq second-moment statistics as a 'free' diagonal Fisher/curvature proxy. Stage one, Fast Fisher Grafting (FFG), scores each task-vector coordinate by (Δw_i)^2 · v_i and grafts low-saliency coordinates back to the base model. Stage two aggregates the remaining task vectors with a preconditioner P* = Diag(√v + ε) (Eqs. 6–7). Stage three compresses v with an AdaFactor-style rank-1 approximation. Experiments merge five Llama-3.1-8B SFT experts (math, code, general instruction following, knowledge recall, precise instruction following). OTA+FFG reaches the highest average score (0.582), ahead of TIES (0.565), linear averaging (0.551), and Fisher merging (0.541). Ablations show that FFG is the main driver, and the rank-1 compressed variant scores 0.571. The paper also presents visual analyses of FFG masks and of shared curvature across experts, arguing that SFT models share a common curvature geometry, which explains why linear merging works.","tokens_in":23559,"tokens_out":5031,"duration_ms":63391,"significance":"If the central claims hold, the paper contributes a practical, nearly storage-free use of optimizer statistics for model merging, plus an interpretable sparsification technique (FFG) that appears to improve merging over strong baselines on an external benchmark. The release of code, training/evaluation scripts, and checkpoints is a concrete asset for reproducibility. The structural analyses of FFG masks are interesting and potentially useful for understanding where SFT knowledge is stored. However, the theoretical bridge connecting Adam's second moment to loss curvature is not established for the actual one-epoch SFT checkpoints, and the aggregation formula uses √v rather than the v that the theory justifies. The ablation evidence also suggests that the aggregation component, taken alone, is not beneficial. For these reasons the paper's 'curvature-informed' framing currently overreaches its support; the empirical recipe may still be valuable, but the narrative needs to be adjusted and the theoretical claims need to be either substantiated or explicitly downgraded to heuristics.","major_comments":[{"comment":"The aggregation step is internally inconsistent with the theory in §4.1. Section 4.1 argues that Adam's second moment v is a scaled EMA of the diagonal FIM, so the curvature weighting should use v (as in Fisher merging). Equation (6) instead defines P* = Diag(√v + ε), which is neither the Fisher weight nor Adam's actual preconditioner (Adam divides by √v + ε). No argument in §4.3 derives √v as a curvature metric. Please either derive this weighting from a stated approximation, replace it with v, or present it explicitly as an additional heuristic. As written, the claim that Eq. (6)–(7) is 'curvature-aware' is not supported by the paper's own derivation.","section":"§4.3, Eqs. (6)–(7)"},{"comment":"The proof that the Hessian equals the observed empirical FIM and that the mini-batch outer product is a scaled FIM requires (A) late-NTK locality and (B) perfect calibration at the fine-tuned checkpoint. For one-epoch SFT checkpoints trained from a base model, neither assumption is plausible: the model is unlikely to be perfectly calibrated, and the final exp_avg_sq is an EMA over the whole trajectory, not a final-point Fisher estimate. Thus Lemma 1 and Lemma 2 do not establish that v is the diagonal Hessian at w*. This is load-bearing because the paper's central interpretive claim rests on it. A concrete empirical check would be to compute the diagonal empirical Fisher (or a Hessian-vector-product estimate) on these checkpoints and compare it with v, or to test sensitivity to calibration. Without such evidence, the 'curvature proxy' language should be treated as a heuristic.","section":"Appendix A, Assumptions 1–2"},{"comment":"The ablation shows that curvature-only aggregation on unpruned task vectors scores 0.536, below plain linear averaging (0.551). This directly undercuts the claim that the curvature-aware aggregation in Eq. (7) is, on its own, a beneficial component of the method. The full OTA gain is driven by FFG; the aggregation component is either neutral or harmful when FFG is removed. The paper's abstract and introduction attribute interference mitigation to the curvature-aware aggregation, but the data support attributing it to FFG. To support the current claim, please show settings where the √v weighting improves over linear averaging with the same mask, or revise the narrative to present the aggregation as a secondary heuristic.","section":"Table 3, OTA (wo FFG) row"},{"comment":"The rank-1 AdaFactor-style compression is justified by the low stable rank of v (<1.3). Low stable rank is a spectral property and does not guarantee elementwise fidelity, which is what matters for per-coordinate saliency in Eq. (5) and for per-coordinate weighting in Eq. (7). The paper reports that compressed OTA retains performance (0.571 vs 0.582), but it does not report how much the FFG masks change when v is replaced by its rank-1 reconstruction. Please report mask overlap or elementwise error between full and compressed v, so readers can see whether the compression works for the selection task, not just for the final merged score.","section":"§4.4 and §5.5"}],"minor_comments":[{"comment":"The caption says 'FGG mask regions'; should be 'FFG mask regions'.","section":"Figure 1"},{"comment":"FFG is named 'Fast Fisher Grafting' and connected to OBS, but the saliency in Eq. (5) is the diagonal OBD score (H_ii Δw_i^2), not OBS's full inverse-Hessian correction. Please align the terminology and references.","section":"§3.3 / §4.2"},{"comment":"The row label 'OTA (w Linear)' is referred to in the text as 'FFG-TA (Selection Only)'. Please use one name consistently to avoid confusion.","section":"Table 3"},{"comment":"No variance or number of seeds is reported for the merging results. Since merging is deterministic but evaluation (e.g., pass@10, IFEval) may have stochastic components, please state evaluation seeds or report standard errors.","section":"§5.2 / Table 3"},{"comment":"The heatmaps downsample weight matrices by uniform row/column subsampling. Please state the exact subsampling procedure for each displayed matrix and note any distortion this introduces for the visual claims about column/row structure.","section":"§5.4 / Figure 4"},{"comment":"The notions of 'input feature curvature' and 'output feature curvature' are informal. Please define them in terms of sums of v entries over rows/columns, or state that they are visual heuristics.","section":"§5.5"},{"comment":"The stable rank plots show values below 1.3, but stable rank is bounded below by 1 by construction; the displayed range makes differences look larger than they are. Consider plotting from 1.0 with appropriate ticks.","section":"Appendix D / Stable rank"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and the empirical recipe may be useful, but the theoretical framing needs substantial revision. The main issue is that the aggregation formula is not derived from the paper's own curvature proxy, and the one ablation that isolates the aggregation component shows it below linear averaging. I would not reject: the FFG component is well-supported empirically and the open-source artifacts are valuable. However, the authors should either provide the missing derivation/empirical validation or explicitly reframe the √v weighting as a heuristic, and should temper the 'curvature-aware' claims accordingly. I also suggest the editor ask for the mask-overlap analysis between full and compressed v, since the compression claim depends on it."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: OTA+FFG is a real contribution to SFT model merging, and the paper is honest about sharing its core \"second moment as Fisher\" idea with Li et al. That said, the theoretical framing goes beyond what the equations support—and the headline numbers show that the gains come mostly from FFG, not from the curvature-aware aggregation.\n\nWhat's actually new: applying the second-moment saliency to task vectors rather than final weights, the explicit two-stage denoise-then-merge pipeline, the storage-light rank-1 compression, and the localization analysis showing structured, low-rank masks in early attention projections. These are genuinely useful pieces, and the FFG-vs-magnitude pruning comparison is compelling—especially the +0.10-0.16 gains on IFEval at high sparsity. The authors also credit concurrent work clearly, which I appreciate.\n\nThe soft spots are in the theory and in the evaluation. The bridge from Adam's exp_avg_sq to Hessian relies on perfect calibration and late-NTK locality, both questionable for one-epoch SFT checkpoints. More concretely, the stress-test note is right: the theory equates v with the diagonal Fisher, but the aggregation in Eq. 7 uses sqrt(v)+epsilon. That is Adam's preconditioner, not the Fisher weight, and no argument in Section 4.3 derives it. The fact that OTA without FFG (0.536) is below plain linear averaging (0.551) suggests the sqrt-weighted aggregation is not acting as a beneficial curvature correction on its own. So I'd read the \"curvature-informed\" claims as heuristic, not as established theory.\n\nAlso worth flagging: no error bars or repeated runs, and the per-expert sparsity tuning is not fully reported. The main gain over TIES is about 0.017 average—not huge, though consistent across several capabilities. The shared-curvature evidence is mostly visual; a quantitative overlap test would help.\n\nWho is this for? People working on model merging or post-hoc composition of SFT checkpoints. The method is simple, reproducible, and likely useful in practice. The theoretical overreach is fixable: tone down the Fisher equivalence, report variance, and report the tuned sparsity values. I'd send it to review and would want to see those revisions.","headline":"Useful, honest merging paper whose empirical gains come mostly from FFG; the curvature-weighting story is overstated, but the method and analysis deserve a serious referee.","tokens_in":24112,"tokens_out":1704,"would_cite":true,"duration_ms":22387,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Using Adam's second-moment statistics as a diagonal curvature proxy, OTA merging with FFG denoising surpasses TIES, linear averaging, and Fisher merging when consolidating five capability-specific SFT experts into one Llama-3.1-8B model.","keywords":["model merging","curvature-aware merging","Fisher information approximation","Adam second moment","task interference","sparsification","LLM fine-tuning","optimizer state"],"falsifier":"Compute the true diagonal Fisher information for the same checkpoints (by sampling from the model's predictive distribution and backpropagating log-likelihood), compare the resulting saliency masks and merge scores against OTA's exp_avg_sq-based results; if performance diverges substantially, the proxy's claimed equivalence is not the operative mechanism. Also, measuring calibration error on each expert and showing that merge quality collapses when calibration is poor would directly stress Assumption 2.","tokens_in":23129,"feed_emoji":"🧩","tokens_out":2785,"duration_ms":32284,"temperature":0.7,"pith_summary":"The paper claims that the main obstacle to merging fine-tuned models is not geometric misalignment but interference from low-saliency parameter updates, and that Adam's second-moment accumulator (exp_avg_sq) provides a free diagonal-curvature proxy for identifying and reverting those noisy updates. It introduces OTA merging: FFG sparsifies each expert's task vector by saliency score (Δw)^2·v, then curvature-weighted aggregation averages the surviving deltas. On five SFT experts (math, code, instruction following, general, knowledge) merged into Llama-3.1-8B, OTA reaches 0.582 average, vs TIES 0.565, linear 0.551, Fisher 0.541. Ablations show selection/denoising (FFG+task arithmetic = 0.560) matters more than curvature aggregation alone (0.536). Also evidence that separately fine-tuned models share nearly identical diagonal curvature, explaining why simple averaging works.","feed_headline":"Curvature from saved Adam stats beats TIES and Fisher in LLM merging","feed_subtitle":"A denoise-first, curvature-weighted merge lifts average score to 0.582 on five SFT experts consolidated into one Llama-3.1-8B.","key_machinery":"Optimization Trajectory Aware (OTA) merging (Eq. 7) combines Fast Fisher Grafting (FFG), a saliency score s_i = (Δw_i)^2 · v_i built from Adam's second-moment accumulator, with a curvature-preconditioned average using P = Diag(√v + ε). The same second-moment tensor can be compressed via a rank-one, AdaFactor-style factorization, yielding nearly storage-free curvature access while preserving most of the benefit.","core_discovery":"In the paper's own terms, the central discovery is that the diagonal curvature of the loss landscape, as proxied by Adam's second-moment estimates, is highly conserved across SFT checkpoints fine-tuned on different capabilities, and that a two-stage pipeline—curvature-guided grafting (FFG) followed by curvature-preconditioned aggregation (OTA)—exploits this shared geometry to consolidate capabilities better than existing weight-space methods. The denoising step is the engine: FFG masks most of the task-vector, keeping only parameters whose squared update times second moment is high, reverting the rest to the base weights, and thus removing conflicting low-value edits.","pith_inferences":["The perfect-calibration assumption may be weaker in practice; if it fails, the saliency score is still empirically useful but not literally a Fisher-derived quantity.","The method could likely extend to other optimizers (e.g., LAMB) or to non-SFT adaptation such as LoRA deltas, since the second-moment accumulator is optimizer-generic.","The low stable rank of the second moments suggests the loss surface is extremely flat in most weight directions; combining OTA with permutation alignment might push merged-model quality even higher.","A testable extension is to replace exp_avg_sq with a sampled empirical Fisher or other curvature approximations and check whether the merge gains persist, which would isolate whether the proxy itself or the denoising mechanism drives the improvement."],"forward_implications":["If the central claim is right, model merging can be performed without computing Fisher information separately; the optimizer state already contains the needed curvature signal.","Simple linear averaging's strong empirical performance is explained: checkpoints share similar curvature, so linear aggregation is implicitly preconditioned.","Memory-light rank-1 compression is sufficient: second moments have stable ranks around 1.3, enabling practical merging at a tiny storage overhead.","FFG's structured sparsity (early/late query/key, token embeddings) suggests task localization may guide where to interpolate or adapt in future merges."],"fun_headline_variants":["Shared curvature from Adam's second moments improves LLM merging","Curvature-aware merging using Adam's stats outperforms TIES and Fisher","Conserved curvature from Adam stats powers better LLM merging","Two-stage curvature merge: sparsify then aggregate using Adam moments"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"That each SFT checkpoint is perfectly calibrated: the model's predicted label distribution matches the true data distribution, so the Hessian equals the observed empirical Fisher information.","fun_headline_variants_meta":{"raw":{"variants":["Shared curvature from Adam's second moments improves LLM merging","Curvature-aware merging using Adam's stats outperforms TIES and Fisher","Conserved curvature from Adam stats powers better LLM merging","Two-stage curvature merge: sparsify then aggregate using Adam moments"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000632,"raw_usage":{"total_tokens":2792,"prompt_tokens":819,"completion_tokens":1973,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":563,"completion_tokens_details":{"reasoning_tokens":1900}},"tokens_in":563,"tokens_out":1973,"duration_ms":19885,"temperature":1.0,"reasoning_tokens":1900,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T16:57:27.867931+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the true diagonal Fisher information for the same checkpoints (by sampling from the model's predictive distribution and backpropagating log-likelihood), compare the resulting saliency masks and merge scores against OTA's exp_avg_sq-based results; if performance diverges substantially, the proxy's claimed equivalence is not the operative mechanism. Also, measuring calibration error on each expert and showing that merge quality collapses when calibration is poor would directly stress Assumption 2.","supporting_citations":[],"review_version":1}