REVIEW 3 major objections 7 minor 2 cited by
Soup to go: mitigating forgetting during continual learning with model averaging
T0 review · 3 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Averaging with an old checkpoint during fine-tuning, not just at the end, curbs catastrophic forgetting.
desk verdict A simple, plausibly useful continual-learning method whose headline no-buffer claim currently rests on per-dataset oracle selection of the averaging frequency; worth refereeing, but the authors need a selection protocol and variance estimates. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the averaging reset $\theta_{t+1} = \beta\theta_o + (1-\beta)\theta^*_{t+1}$, applied every $pT$ fine-tuning steps, with $\theta_o$ the previous-task checkpoint and $\theta^*_{t+1}$ the ordinary task-loss update. Two scalar hyperparameters carry the method: $\beta$ decides how much weight the old checkpoint gets, and $p$ decides how often the merge happens, so the method interpolates between no intervention and per-step shrinkage toward $\theta_o$. The theoretical bridge is the identity between this reset composed with a gradient step and one step of gradient descent on $\mathcal{L}_{\mathrm{task}}(\theta) + \frac{\lambda}{2}\|\theta - \theta_o\|^2$, which is how the paper links model averaging to classical penalty methods and why frequent averaging preserves past-task accuracy.
What would settle it
Run SFA on a pair of tasks whose fine-tuned solutions are known to lie in separate regions of weight space (for example, two tasks with disjoint label sets and no shared input structure), sweep $p$ from $1$ down to very small values, and measure old-task accuracy; if even the most frequent averaging cannot hold old-task accuracy at the level of a 5% rehearsal buffer while the new task is still learned, the single-checkpoint proxy is refuted.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that continual fine-tuning can be protected by periodically moving the current iterate back toward a fixed previous-task checkpoint, and that this is strictly better than performing the same merge once at the end. Concretely, SFA keeps $\theta_o$, the model trained on all previous tasks, and after each new-task gradient step $\theta^*_{t+1}$ it computes $\theta_{t+1} = \beta\theta_o + (1-\beta)\theta^*_{t+1}$ every $pT$ iterations and also at the end of training; $\theta_o$ is then updated to the merged result. The reported results show final average task accuracy comparable to rehearsal buffers on both Food-101 and CIFAR-100, with SFA at $p$ near $0.98$ outperforming SFA at $p=1$ (which is WiSE-FT), and SFA at $p$ near $0.25$ matching a 5-10% buffer on Math-to-Law transfer across several model families. Against other merge-only methods, SFA dominates Task Arithmetic and TIES on the language-domain pairs, and it does so while needing only two checkpoints, no past data, and no per-step penalty. The paper further claims that the averaging update is equivalent, at one extreme, to an L2-penalized gradient step, giving a mechanistic reason merging works: it constrains the distance from the previous solution.
Load-bearing premise
The method assumes that a single checkpoint from the previous tasks represents everything the past data taught, and that repeatedly averaging toward it keeps the model where both old and new tasks work well; if those two goals live in separate regions of weight space, no averaging frequency can preserve the old skills.
Editorial extensions
If this is right
- A practitioner can control the forgetting-plasticity trade-off with the single frequency parameter $p$, needing only the current weights and one stored checkpoint, with no data buffer.
- Because $p<1$ consistently beats $p=1$, the benefit comes from letting the averaged model continue training; end-only merging is a special case, not a substitute.
- On diverse-domain language fine-tuning, SFA with small $p$ matches rehearsal at 5-10% buffer sizes, so a previous checkpoint can serve as a proxy for past data even under large domain shifts.
- The equivalence with L2 regression implies that the gap between merge-based and penalty-based continual learning is largely computational: SFA reaches a penalty-like effect with infrequent, cheap merges.
- Updating $\theta_o$ to each merged model lets SFA run beyond two tasks, and the three-domain results show retention of both earlier domains while learning the third.
Reading between the lines
- If the checkpoint-as-data view is right, averaging several historical checkpoints, or Fisher-weighting them, should beat trusting only the most recent $\theta_o$ when earlier tasks resemble each other more than they resemble the latest task.
- The L2 equivalence suggests a testable extension: measuring the effective penalty strength of each SFA run and comparing it with explicit L2 curves should predict exactly which $p$ values match which penalty coefficients.
- Because parameter averaging is a variance-reduction operation, SFA may also stabilize training across random seeds and task orders; the paper reports final-checkpoint comparisons, not variance across runs.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes Sequential Fine-tuning Averaging (SFA), a continual-learning method in which, while fine-tuning on a new task, the current model is periodically averaged with a checkpoint optimized on previous tasks. The averaging frequency p and averaging weight beta control the tradeoff between retaining past-task performance and learning the new task. The authors evaluate SFA on 20-task Food-101 and CIFAR-100 image-classification streams with a ViT, on two-task and three-task language-domain sequences (Math, Law, Code) with Pythia, Llama 2, and Qwen models, and on a small MNIST two-task setup. They compare against data-buffer rehearsal, Task Arithmetic, TIES, WiSE-FT, L2 penalty, and EWC, and report that SFA matches rehearsal without storing past data and outperforms end-only merging, while also giving a derivation connecting SFA to L2-regression and a brief Bayesian motivation.
Significance. If the empirical claims hold, SFA is a strikingly simple and computationally cheap continual-learning baseline that could reduce the need for data buffers in some settings. The paper's strengths include evaluation across several model families and scales, detailed numeric tables, a clean per-step identity connecting SFA to L2-regression (Section 6), and a sanity-check experiment on MNIST (Figure 9). However, the headline claim of matching rehearsal without a buffer is currently conditional on how p is selected and on single-seed point estimates, so the central result needs strengthening before the conclusion is fully supported.
major comments (3)
- [§5.1, Algorithm 1, Figure 1] The averaging frequency p is a free hyperparameter that directly sets the forgetting-plasticity tradeoff, and the highlighted values (p=0.98/0.96 in Figure 1; p=0.25, p=0.10 in Figures 3-5 and Tables 1-7) appear to be chosen after inspecting the full two-task accuracy tradeoff curves. The abstract's contribution is a no-buffer method, but the manuscript gives no rule for selecting p when past-task data are not stored and past-task test performance is not available during training. Please provide a concrete protocol—for example, a fixed schedule, selection on current-task validation only, or a separate validation split for p—and report the resulting performance. Without this, the comparison to the 5-10% data buffer and to WiSE-FT is an oracle-selection result rather than a demonstrated no-buffer method.
- [§5.1-§5.4, Figures 1-9, Tables 1-7] No error bars, confidence intervals, or multiple-seed results are reported; the headline numbers appear to be single runs. Since the claims of being "comparable" to rehearsal and "outperforming" end-only merging often rest on small margins (e.g., Figure 3, SFA p=0.25 versus the 10% data buffer; Figure 5, left panel), the reported differences could be within run-to-run variation. Please report at least 3-5 seeds with means and standard deviations, or otherwise show that the observed gaps exceed seed noise.
- [§5.2, Figure 5 (right), Table 5] In the Math-to-Code setting with Pythia, the data-buffer baseline is explicitly acknowledged to be poorly configured ("we hypothesize that this is because of suboptimal hyperparameters"), and Table 5 shows that the buffer achieves 0 HumanEval and 0.32 GSM8K. The figure caption then states that SFA with varying p outperforms the data buffer. This is not a fair test of the no-buffer advantage. Please re-tune the buffer baseline in this setting, or remove the claim that SFA outperforms it here, and state the result for a properly configured buffer.
minor comments (7)
- [Section 6, Eqs. (4)-(7)] The L2-regression equivalence is derived only in the limit of averaging after every gradient step, whereas all experiments average every pT steps. The statement that SFA "roughly approximates" L2-regression is therefore heuristic; please state clearly in which regime the approximation is expected to hold, and consider adding a quantitative check such as comparing SFA with per-step averaging to L2-regression.
- [Section 9, Appendix A.5] The reproducibility section states that specific configurations will be released, but the appendix does not actually list learning rates, batch sizes, optimizer settings, training epochs, or evaluation hyperparameters for the image and language experiments, and the repository is not yet available. Please include these details or point to a released artifact.
- [Figure 1] The tick labels in the two panels are overlapping and hard to read (e.g., the sequence "0.75 0.50 0.25" on the right panel), and the figure would benefit from a clearer layout with separate axes or larger spacing.
- [Tables 3 and 4] The rows labeled "METAMATHQA, LAW" contain entries like "0.64, 0.86" and "0.78, 0.92" that look like two values squeezed into one cell; clarify whether these are separate metrics or formatting errors.
- [Section 5.2, Figure 5] The sentence "We first fine-tune our model Llama 2 (7B) in Fig. 3, Qwen2.5 (1.5B) in Fig. 4, and Pythia (2.8B) in Fig. 5" is a run-on; please rephrase for clarity.
- [Algorithm 1] The final conditional branch uses theta_{T+1} while the loop body defines theta*_{t+1}; align the notation so that the final merge operation is unambiguous.
- [Section 3] The assumption that a single checkpoint theta_o is a sufficient proxy for the past-task data is explicit but not stress-tested. A small experiment with a deliberately under-trained or incomplete theta_o would clarify when SFA can and cannot replace a data buffer.
Circularity Check
No construction-based or self-citation circularity; the central claims are external empirical comparisons, so the analysis is essentially self-contained, with only a minor non-load-bearing self-citation and a hyperparameter-selection caveat.
full rationale
Walking the derivation chain: Algorithm 1 defines SFA, and the paper's results are empirical comparisons on external datasets (Food-101, CIFAR-100, GSM8K, Law holdouts, Code HumanEval) against rehearsal, Task Arithmetic, TIES, WiSE-FT, L2, and EWC. No headline claim is obtained by renaming a fitted quantity as a prediction. The Section 6 L2 connection is an algebraic equivalence between the SFA update (Eqs. 4-7) and one gradient step of an L2-regularized loss (Eqs. 1-3), with the paper explicitly noting that exact equivalence requires beta = eta*lambda and alpha = eta/(1 - eta*lambda) and that infrequent averaging means SFA 'typically is not equivalent to L2-regression.' This is an identity derived from the algorithm's own update rule, not a fitted input, and it is not used to force the empirical comparisons. The statement that WiSE-FT is equivalent to SFA with p = 1 is a definitional identity used only to frame the comparison; the claim that averaging during training outperforms end-only averaging is an empirical observation on external benchmarks. The only self-citation is Dhawan et al. 2023 in a related-work sentence about Fisher merging; it is not load-bearing. The per-dataset choice of p is a hyperparameter sweep, and selecting p after inspecting the trade-off curve is a robustness or oracle-selection concern, but the paper does not define p in terms of the target result and does not relabel a fitted value as a prediction. Thus no circularity by construction or by self-citation chain; the score reflects the minor non-load-bearing self-citation and the p-selection caveat rather than any circular derivation.
Assumptions & free parameters
free parameters (2)
- p (averaging frequency) =
0.98 (Food-101), 0.96 (CIFAR-100), 0.25 (LLM cross-domain)
- beta (averaging weight) =
0.5 (default)
assumptions (3)
- domain assumption Weight-space averaging of models trained on different tasks yields a model that retains capabilities of both (linear mode connectivity or low-loss basin overlap).
- ad hoc to paper A single checkpoint fine-tuned on previous tasks is a sufficient proxy for the previous task data.
- ad hoc to paper The equivalence with L2-regression assumes an extreme per-step averaging regime and a rescaled learning rate.
Cite this review
Pith. "Pith review of Soup to go: mitigating forgetting during continual learning with model averaging." pith.science (2026). https://pith.science/paper/YED6Q7TA
@misc{pith2026250105559,
author = {Pith},
title = {Pith review of: Soup to go: mitigating forgetting during continual learning with model averaging},
year = {2026},
howpublished = {\url{https://pith.science/paper/YED6Q7TA}},
note = {Machine review of arXiv:2501.05559}
}
read the original abstract
In continual learning, where task data arrives in a sequence, fine-tuning on later tasks will often lead to performance degradation on earlier tasks. This is especially pronounced when these tasks come from diverse domains. In this setting, how can we mitigate catastrophic forgetting of earlier tasks and retain what the model has learned with minimal computational expenses? Inspired by other merging methods, and L2-regression, we propose Sequential Fine-tuning with Averaging (SFA), a method that merges currently training models with earlier checkpoints during the course of training. SOTA approaches typically maintain a data buffer of past tasks or impose a penalty at each gradient step. In contrast, our method achieves comparable results without the need to store past data, or multiple copies of parameters for each gradient step. Furthermore, our method outperforms common merging techniques such as Task Arithmetic, TIES Merging, and WiSE-FT, as well as other penalty methods like L2 and Elastic Weight Consolidation. In turn, our method offers insight into the benefits of merging partially-trained models during training across both image and language domains.
Figures
Figures from the paper (9 more)
Forward citations
Cited by 2 Pith papers
-
Upweighting Easy Samples in Fine-Tuning Mitigates Forgetting
Upweighting easy samples (low pre-trained loss) during fine-tuning reduces catastrophic forgetting without access to pre-training data, at a small cost in target performance.
-
Continual Learning in Vision-Language Models via Aligned Model Merging
PAM merges a task-specific LoRA into a global LoRA and re-initializes sign-conflicting weights during training, reducing catastrophic forgetting in continual VLM learning.
Reference graph
Works this paper leans on
-
[4]
doi: 10.1109/tpami.2021.3057446
ISSN 1939-3539. doi: 10.1109/tpami.2021.3057446. URL http://dx.doi.org/10.1109/ TPAMI.2021.3057446. Nikita Dhawan, Nicole Mitchell, Zachary Charles, Zachary Garrett, and Gintare Karolina Dziu- gaite. Leveraging function space aggregation for federated learning at scale. arXiv preprint arXiv:2311.10291,
arXiv 1939
-
[6]
Measuring forgetting of memorized training examples
Matthew Jagielski, Om Thakkar, Florian Tramer, Daphne Ippolito, Katherine Lee, Nicholas Carlini, Eric Wallace, Shuang Song, Abhradeep Thakurta, Nicolas Papernot, et al. Measuring forgetting of memorized training examples. arXiv preprint arXiv:2207.00099,
-
[8]
ISSN 1091-6490. doi: 10.1073/pnas.1611835114. URL http: //dx.doi.org/10.1073/pnas.1611835114. Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report,
-
[11]
model scales, indicating that merely scaling up parameter size does not help mitigate forgetting despite the increased capacity. But how severe is this forgetting? We quantify this by comparing a model that was trained on and has then forgotten Simpl to a model that has never seen Simpl. In Fig. 12, the pink line shows validation loss on Simpl for a model...
work page 2000
-
[13]
URL https://aclanthology.org/2020.nlpcovid19-acl.18
Association for Computa- tional Linguistics. URL https://aclanthology.org/2020.nlpcovid19-acl.18. Sean Moriarity. Continual learning with elastic weight consolidation in tensorflow
work page 2020
-
[16]
Fine-tuned language models are continual learners
Thomas Scialom, Tuhin Chakrabarty, and Smaranda Muresan. Fine-tuned language models are continual learners. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 6107–6122,
2022
-
[17]
Magicoder: Source code is all you need
Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang. Magicoder: Source code is all you need. arXiv preprint arXiv:2312.02120,
-
[18]
Metamath: Bootstrap your own mathematical questions for large language models
Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhen- guo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models. arXiv preprint arXiv:2309.12284,
Show all 22 references
-
[19]
η is a hyperpa- rameter, and Fo is a diagonal matrix with the initial model’s Fisher information
where λ = 1 , j= 1, ...,|θ| L(θt) = Ltask(θt) + X j 1 2 F (j) o (θ(j) t − θ(j) o )2 (12) where θo and θt are the weights of the initial and fine-tuning model respectively. η is a hyperpa- rameter, and Fo is a diagonal matrix with the initial model’s Fisher information. Assume ...
2022
-
[21]
Finally, we create some model merging baselines using mergekit (Goddard et al., 2024)
metrics, we also use Language Model Evaluation Harness (Gao et al., 2023). Finally, we create some model merging baselines using mergekit (Goddard et al., 2024). A.6 I NSTRUCTION DATASETS We use language generation tasks described in (Scialom et al.,
2023
-
[22]
to measure forgetting. These tasks are based on pre-existing datasets that we also reference here: Text Simplification (Simpl) (Wiki-Auto (Jiang et al., 2020)), Inquisitive Question Generation (InqQG) (Eli5 (Fan et al., 2019)), Headline Generation with Constraint (HGen) (Gigaw...
2020
-
[1995]
URL https://doi.org/10.1080/ 09540099550039318
doi: 10.1080/09540099550039318. URL https://doi.org/10.1080/ 09540099550039318. Alexander M. Rush, Sumit Chopra, and Jason Weston. A neural attention model for abstractive sentence summarization. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing,
2015 doi
-
[2015]
URL http://dx.doi.org/10
doi: 10.18653/v1/d15-1044. URL http://dx.doi.org/10. 18653/v1/D15-1044. Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, ...
-
[2016]
COVID-QA: A question an- swering dataset for COVID-19
Timo M¨oller, Anthony Reina, Raghavan Jayakumar, and Malte Pietsch. COVID-QA: A question an- swering dataset for COVID-19. In Karin Verspoor, Kevin Bretonnel Cohen, Mark Dredze, Emilio Ferrara, Jonathan May, Robert Munro, Cecile Paris, and Byron Wallace (eds.),Proceedings of t...
2020
-
[2017]
Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool
URL https://doi.org/10.7910/DVN/JBXKFD. Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101 – mining discriminative com- ponents with random forests. In European Conference on Computer Vision,
-
[2018]
Lawinformedai/claudette tos,
URL https: //statisticaloddsandends.wordpress.com/2018/12/29/ bayesian-interpretation-of-ridge-regression/ . Lawinformedai/claudette tos,
2018
-
[2019]
doi: 10.1007/s10506-019-09243-2
ISSN 1572-8382. doi: 10.1007/s10506-019-09243-2. URL http://dx.doi.org/10.1007/ s10506-019-09243-2 . David Lopez-Paz and Marc’Aurelio Ranzato. Gradient episodic memory for continual learning,
-
[2020]
doi: 10.18653/v1/2020.acl-main.709
Association for Computational Linguis- tics. doi: 10.18653/v1/2020.acl-main.709. URL https://aclanthology.org/2020. acl-main.709. James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, An- drei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Ag...
2020 doi
-
[2021]
doi: https://doi.org/10.1016/j.neunet.2020.12.003
ISSN 0893-6080. doi: https://doi.org/10.1016/j.neunet.2020.12.003. URL https://www.sciencedirect. com/science/article/pii/S0893608020304202. Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O’Brien, Eric Hal- lahan, Mohammad Aflah Khan, Shivanshu Puroh...
2020 doi
-
[2022]
An empirical study of catastrophic forgetting in large language models during continual fine-tuning
Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint arXiv:2308.08747,
-
[2023]
Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vlad Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz
URL https://zenodo.org/records/ 10256836. Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vlad Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. Arcee’s mergekit: A toolkit for merging large language models. arXiv preprint arXiv:2403.13257,
-
[2024]
Branch-train-merge: Embarrassingly parallel training of expert language models
Margaret Li, Suchin Gururangan, Tim Dettmers, Mike Lewis, Tim Althoff, Noah A Smith, and Luke Zettlemoyer. Branch-train-merge: Embarrassingly parallel training of expert language models. arXiv preprint arXiv:2208.03306,
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.