{"id":"8324c6cc-5020-46a8-b1b4-e26e89221ff8","arxiv_id":"2501.05559","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"SFA reduces forgetting in continual learning by averaging the currently training model with a checkpoint from previous tasks during fine-tuning, matching data-buffer rehearsal without storing past data.","lead":"Researchers propose Sequential Fine-tuning Averaging (SFA), which periodically blends a model's weights with a saved checkpoint from earlier tasks while fine-tuning on new ones. It aims to curb catastrophic forgetting without storing past data, and matches or beats buffer-based replay and end-only merging in vision and language tests.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The no-buffer advantage is not yet established as an operational property: SFA's averaging frequency p is selected per dataset after seeing both-task test performance, and no variance information is reported, so the match to rehearsal and the gain over WiSE-FT could be an artifact of oracle…","rationale":"The reader's CONDITIONAL verdict is appropriate. My concern differs slightly from the reader's stated weakest assumption: instead of disputing that theta_o can encode past tasks, I focus on the missing operational rule for p, which is necessary for the abstract's no-buffer claim. This is one of the weaknesses the reader lists in the rationale, so agreement is partial. The proposed rerun would settle whether the reported results are robust to a realistic selection protocol; until then, the central claim should be read as conditional on oracle p. I do not see an internal inconsistency in Algorithm 1, and the L2 connection is clearly labeled approximate, so I would not reject the paper.","tokens_in":112,"tokens_out":13106,"duration_ms":208694,"concrete_test":"Re-run the central comparisons (Food-101, CIFAR-100, Pythia Math-Law and Math-Code) with a pre-registered, past-data-free p protocol: for example, p=0.25 for all cross-domain pairs and p=0.98 for both image streams, with 3 seeds and mean +/- std, and no test-set p selection. If the buffer-comparable points and WiSE-FT advantage persist under this protocol, the concern is resolved; if not, the paper must report p selection as an oracle hyperparameter and qualify the no-buffer claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Algorithm 1 exposes p as a free hyperparameter that directly sets the forgetting-plasticity tradeoff (Figs. 3-9). The paper does not give a rule for choosing p when past-task data are unavailable, which is exactly the regime the abstract claims to handle ('without the need to store past data'). Instead, p values such as 0.98/0.96 on Food-101/CIFAR-100 and 0.25/0.10 in the language experiments appear to be chosen after inspecting final performance on both tasks (Section 5.1; Tables 1-7). No error bars or seed variance are reported. If p were fixed a priori or selected from current-task validation only, the method could forget (p too high) or underlearn (p too low); the paper supplies no protocol that would recover the reported tradeoff curve. This makes the headline comparison to a 5-10% data buffer an oracle-selection result rather than a demonstrated no-buffer continual-learning method. The underlying checkpoint-proxy assumption (theta_o sufficiently encodes past tasks) is related but secondary: even a perfect theta_o leaves p unspecified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes Sequential Fine-tuning Averaging (SFA), a continual-learning method in which, while fine-tuning on a new task, the current model is periodically averaged with a checkpoint optimized on previous tasks. The averaging frequency p and averaging weight beta control the tradeoff between retaining past-task performance and learning the new task. The authors evaluate SFA on 20-task Food-101 and CIFAR-100 image-classification streams with a ViT, on two-task and three-task language-domain sequences (Math, Law, Code) with Pythia, Llama 2, and Qwen models, and on a small MNIST two-task setup. They compare against data-buffer rehearsal, Task Arithmetic, TIES, WiSE-FT, L2 penalty, and EWC, and report that SFA matches rehearsal without storing past data and outperforms end-only merging, while also giving a derivation connecting SFA to L2-regression and a brief Bayesian motivation.","tokens_in":22736,"tokens_out":7791,"duration_ms":75463,"significance":"If the empirical claims hold, SFA is a strikingly simple and computationally cheap continual-learning baseline that could reduce the need for data buffers in some settings. The paper's strengths include evaluation across several model families and scales, detailed numeric tables, a clean per-step identity connecting SFA to L2-regression (Section 6), and a sanity-check experiment on MNIST (Figure 9). However, the headline claim of matching rehearsal without a buffer is currently conditional on how p is selected and on single-seed point estimates, so the central result needs strengthening before the conclusion is fully supported.","major_comments":[{"comment":"The averaging frequency p is a free hyperparameter that directly sets the forgetting-plasticity tradeoff, and the highlighted values (p=0.98/0.96 in Figure 1; p=0.25, p=0.10 in Figures 3-5 and Tables 1-7) appear to be chosen after inspecting the full two-task accuracy tradeoff curves. The abstract's contribution is a no-buffer method, but the manuscript gives no rule for selecting p when past-task data are not stored and past-task test performance is not available during training. Please provide a concrete protocol—for example, a fixed schedule, selection on current-task validation only, or a separate validation split for p—and report the resulting performance. Without this, the comparison to the 5-10% data buffer and to WiSE-FT is an oracle-selection result rather than a demonstrated no-buffer method.","section":"§5.1, Algorithm 1, Figure 1"},{"comment":"No error bars, confidence intervals, or multiple-seed results are reported; the headline numbers appear to be single runs. Since the claims of being \"comparable\" to rehearsal and \"outperforming\" end-only merging often rest on small margins (e.g., Figure 3, SFA p=0.25 versus the 10% data buffer; Figure 5, left panel), the reported differences could be within run-to-run variation. Please report at least 3-5 seeds with means and standard deviations, or otherwise show that the observed gaps exceed seed noise.","section":"§5.1-§5.4, Figures 1-9, Tables 1-7"},{"comment":"In the Math-to-Code setting with Pythia, the data-buffer baseline is explicitly acknowledged to be poorly configured (\"we hypothesize that this is because of suboptimal hyperparameters\"), and Table 5 shows that the buffer achieves 0 HumanEval and 0.32 GSM8K. The figure caption then states that SFA with varying p outperforms the data buffer. This is not a fair test of the no-buffer advantage. Please re-tune the buffer baseline in this setting, or remove the claim that SFA outperforms it here, and state the result for a properly configured buffer.","section":"§5.2, Figure 5 (right), Table 5"}],"minor_comments":[{"comment":"The L2-regression equivalence is derived only in the limit of averaging after every gradient step, whereas all experiments average every pT steps. The statement that SFA \"roughly approximates\" L2-regression is therefore heuristic; please state clearly in which regime the approximation is expected to hold, and consider adding a quantitative check such as comparing SFA with per-step averaging to L2-regression.","section":"Section 6, Eqs. (4)-(7)"},{"comment":"The reproducibility section states that specific configurations will be released, but the appendix does not actually list learning rates, batch sizes, optimizer settings, training epochs, or evaluation hyperparameters for the image and language experiments, and the repository is not yet available. Please include these details or point to a released artifact.","section":"Section 9, Appendix A.5"},{"comment":"The tick labels in the two panels are overlapping and hard to read (e.g., the sequence \"0.75 0.50 0.25\" on the right panel), and the figure would benefit from a clearer layout with separate axes or larger spacing.","section":"Figure 1"},{"comment":"The rows labeled \"METAMATHQA, LAW\" contain entries like \"0.64, 0.86\" and \"0.78, 0.92\" that look like two values squeezed into one cell; clarify whether these are separate metrics or formatting errors.","section":"Tables 3 and 4"},{"comment":"The sentence \"We first fine-tune our model Llama 2 (7B) in Fig. 3, Qwen2.5 (1.5B) in Fig. 4, and Pythia (2.8B) in Fig. 5\" is a run-on; please rephrase for clarity.","section":"Section 5.2, Figure 5"},{"comment":"The final conditional branch uses theta_{T+1} while the loop body defines theta*_{t+1}; align the notation so that the final merge operation is unambiguous.","section":"Algorithm 1"},{"comment":"The assumption that a single checkpoint theta_o is a sufficient proxy for the past-task data is explicit but not stress-tested. A small experiment with a deliberately under-trained or incomplete theta_o would clarify when SFA can and cannot replace a data buffer.","section":"Section 3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a promising empirical contribution that fits the journal's scope. The main obstacles are the lack of a validation-based selection protocol for p and the absence of run-to-run variance reporting; both should be addressed before the paper is accepted. The poorly configured data-buffer baseline in Math-to-Code should also be corrected."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"SFA is a simple idea with a real chance of being useful: periodically average the model fine-tuning on a new task with a checkpoint from previous tasks, with a frequency knob p that trades off stability and plasticity. The paper shows across image and language tasks that this beats end-only merging (WiSE-FT, Task Arithmetic, TIES) and comes close to rehearsal with a small buffer. The empirical sweep is broader than most: ViT on 20-task streams, three LLM families, pairs and triples of domains, plus sweeps over both p and beta. The L2 connection is informal but correctly framed as intuition, not a theorem; the authors explicitly say SFA is not equivalent to L2 except in the per-step limit. That is honest and fine.\n\nThe serious soft spot is the operational claim. The abstract says \"without the need to store past data,\" but p is chosen per dataset after inspecting final performance on both tasks. No error bars or seed counts are reported, and the best p varies a lot across settings (0.98 on Food-101, 0.25 on Math-to-Law). The paper gives no rule for picking p when you genuinely cannot look at past-task performance. So the headline comparison to a 5-10% buffer is, as presented, an oracle-selection result. The checkpoint-proxy assumption (theta_o is a good stand-in for past data) is secondary; even a perfect theta_o leaves p unspecified. A validation-based procedure, even a crude one, would go a long way.\n\nAlso: code is not yet released, some baselines are explicitly under-tuned (the data buffer in Pythia Math-to-Code), and the tables are consistent with the main claims but noisy enough that single seeds make me cautious. These are fixable, not fatal.\n\nOverall: the core idea is novel enough and cheap enough that it deserves a serious referee. I would send it to review conditional on the authors adding seed variance and a practical protocol for selecting p without access to past-task test performance, or at least clearly demoting the no-buffer claim from \"matches rehearsal\" to \"avoids storing data at the cost of needing a tuning rule.\" The math is not the concern; the evaluation protocol is.","headline":"A simple, plausibly useful continual-learning method whose headline no-buffer claim currently rests on per-dataset oracle selection of the averaging frequency; worth refereeing, but the authors need a selection protocol and variance estimates.","tokens_in":23278,"tokens_out":1745,"would_cite":true,"duration_ms":18610,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Averaging with an old checkpoint during fine-tuning, not just at the end, curbs catastrophic forgetting.","keywords":["continual learning","catastrophic forgetting","model averaging","model merging","checkpoint averaging","rehearsal-free","L2 regression","large language models"],"falsifier":"Run SFA on a pair of tasks whose fine-tuned solutions are known to lie in separate regions of weight space (for example, two tasks with disjoint label sets and no shared input structure), sweep $p$ from $1$ down to very small values, and measure old-task accuracy; if even the most frequent averaging cannot hold old-task accuracy at the level of a 5% rehearsal buffer while the new task is still learned, the single-checkpoint proxy is refuted.","tokens_in":4,"feed_emoji":"🧠","tokens_out":8802,"duration_ms":136519,"temperature":0.7,"pith_summary":"Sequential Fine-tuning Averaging (SFA) is proposed as a buffer-free way to reduce catastrophic forgetting: while the model fine-tunes on a new task, every $pT$ steps its weights are reset to a weighted average of the current weights and a checkpoint from the previous tasks. The paper's central claim is that averaging during training, not just once at the end, is what preserves old-task performance, and that one old checkpoint can stand in for stored past data. On 20-task streams from Food-101 and CIFAR-100 and on Math-to-Law and Math-to-Code fine-tuning of several language models, SFA matches rehearsal with a 5-10% data buffer and beats end-only merging methods such as WiSE-FT, Task Arithmetic, and TIES. The averaging frequency $p$ is the control knob: smaller $p$ means more averaging, stronger retention of earlier tasks, and a trade-off against how much the new task is learned. The paper also shows the reset operation algebraically approximates L2 regression, connecting merging methods to penalty-based continual learning.","feed_headline":"Periodic checkpoint averaging stops forgetting without past data","feed_subtitle":"SFA matches 5-10% rehearsal buffers on image streams and on Math, Law, and Code fine-tuning.","key_machinery":"The load-bearing object is the averaging reset $\\theta_{t+1} = \\beta\\theta_o + (1-\\beta)\\theta^*_{t+1}$, applied every $pT$ fine-tuning steps, with $\\theta_o$ the previous-task checkpoint and $\\theta^*_{t+1}$ the ordinary task-loss update. Two scalar hyperparameters carry the method: $\\beta$ decides how much weight the old checkpoint gets, and $p$ decides how often the merge happens, so the method interpolates between no intervention and per-step shrinkage toward $\\theta_o$. The theoretical bridge is the identity between this reset composed with a gradient step and one step of gradient descent on $\\mathcal{L}_{\\mathrm{task}}(\\theta) + \\frac{\\lambda}{2}\\|\\theta - \\theta_o\\|^2$, which is how the paper links model averaging to classical penalty methods and why frequent averaging preserves past-task accuracy.","core_discovery":"On the paper's own terms, the discovery is that continual fine-tuning can be protected by periodically moving the current iterate back toward a fixed previous-task checkpoint, and that this is strictly better than performing the same merge once at the end. Concretely, SFA keeps $\\theta_o$, the model trained on all previous tasks, and after each new-task gradient step $\\theta^*_{t+1}$ it computes $\\theta_{t+1} = \\beta\\theta_o + (1-\\beta)\\theta^*_{t+1}$ every $pT$ iterations and also at the end of training; $\\theta_o$ is then updated to the merged result. The reported results show final average task accuracy comparable to rehearsal buffers on both Food-101 and CIFAR-100, with SFA at $p$ near $0.98$ outperforming SFA at $p=1$ (which is WiSE-FT), and SFA at $p$ near $0.25$ matching a 5-10% buffer on Math-to-Law transfer across several model families. Against other merge-only methods, SFA dominates Task Arithmetic and TIES on the language-domain pairs, and it does so while needing only two checkpoints, no past data, and no per-step penalty. The paper further claims that the averaging update is equivalent, at one extreme, to an L2-penalized gradient step, giving a mechanistic reason merging works: it constrains the distance from the previous solution.","pith_inferences":["If the checkpoint-as-data view is right, averaging several historical checkpoints, or Fisher-weighting them, should beat trusting only the most recent $\\theta_o$ when earlier tasks resemble each other more than they resemble the latest task.","The L2 equivalence suggests a testable extension: measuring the effective penalty strength of each SFA run and comparing it with explicit L2 curves should predict exactly which $p$ values match which penalty coefficients.","Because parameter averaging is a variance-reduction operation, SFA may also stabilize training across random seeds and task orders; the paper reports final-checkpoint comparisons, not variance across runs."],"forward_implications":["A practitioner can control the forgetting-plasticity trade-off with the single frequency parameter $p$, needing only the current weights and one stored checkpoint, with no data buffer.","Because $p<1$ consistently beats $p=1$, the benefit comes from letting the averaged model continue training; end-only merging is a special case, not a substitute.","On diverse-domain language fine-tuning, SFA with small $p$ matches rehearsal at 5-10% buffer sizes, so a previous checkpoint can serve as a proxy for past data even under large domain shifts.","The equivalence with L2 regression implies that the gap between merge-based and penalty-based continual learning is largely computational: SFA reaches a penalty-like effect with infrequent, cheap merges.","Updating $\\theta_o$ to each merged model lets SFA run beyond two tasks, and the three-domain results show retention of both earlier domains while learning the third."],"supporting_citations":[{"why":"WiSE-FT, the end-only checkpoint-merge baseline that SFA with p=1 reproduces and that SFA with p<1 outperforms.","marker":"Wortsman et al., 2022b"},{"why":"Task Arithmetic, the task-vector merging baseline SFA outperforms on cross-domain language pairs.","marker":"Ilharco et al., 2023"},{"why":"Model soups, the source of the weight-averaging idea that SFA adapts to continual learning.","marker":"Wortsman et al., 2022a"},{"why":"EWC, the per-step penalty baseline that SFA approximates and outperforms on the MNIST two-task experiment.","marker":"Kirkpatrick et al., 2017"},{"why":"Rehearsal, the data-buffer method whose stored-data requirement SFA removes.","marker":"Robins, 1995"},{"why":"TIES merging, another end-only merge baseline compared on the Math-to-Law and Math-to-Code settings.","marker":"Yadav et al., 2023"},{"why":"Fisher-weighted averaging, used in the appendix to interpret the EWC penalty as a form of model merging.","marker":"Matena & Raffel, 2022"},{"why":"The two-task MNIST split used for the small-model comparison of SFA against L2 and EWC.","marker":"Moriarity, 2020"}],"fun_headline_variants":["Forgetfulness fix: average weights mid-training, no buffers","Checkpoint averaging during training beats replay buffers","Weight averaging tames catastrophic forgetting without data","Train-time averaging prevents forgetting in CL","Averaging checkpoints mid-training curbs forgetting"],"cache_read_input_tokens":25472,"weakest_assumption_plain":"The method assumes that a single checkpoint from the previous tasks represents everything the past data taught, and that repeatedly averaging toward it keeps the model where both old and new tasks work well; if those two goals live in separate regions of weight space, no averaging frequency can preserve the old skills.","fun_headline_variants_meta":{"raw":{"variants":["Forgetfulness fix: average weights mid-training, no buffers","Checkpoint averaging during training beats replay buffers","Weight averaging tames catastrophic forgetting without data","Train-time averaging prevents forgetting in CL","Averaging checkpoints mid-training curbs forgetting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000246,"raw_usage":{"total_tokens":1580,"prompt_tokens":1028,"completion_tokens":552,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":644,"completion_tokens_details":{"reasoning_tokens":481}},"tokens_in":644,"tokens_out":552,"duration_ms":5658,"temperature":1.0,"reasoning_tokens":481,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:13:35.838740+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SFA on a pair of tasks whose fine-tuned solutions are known to lie in separate regions of weight space (for example, two tasks with disjoint label sets and no shared input structure), sweep $p$ from $1$ down to very small values, and measure old-task accuracy; if even the most frequent averaging cannot hold old-task accuracy at the level of a 5% rehearsal buffer while the new task is still learned, the single-checkpoint proxy is refuted.","supporting_citations":[],"review_version":1}