{"id":"ae956a15-9c45-4706-afee-9f943e010519","arxiv_id":"2506.00477","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A two-phase 'Flashback' mechanism that adds plasticity regularization from a temporary new-task model improves stability-plasticity balance across four continual learning method families.","lead":"This paper introduces Flashback Learning, a two-phase add-on for continual learning that trains a temporary 'plastic' model on new task data, then uses both the old and the temporary models to guide final training. The result is a general plug-in that improves average accuracy of replay, distillation, regularization, and architecture-based continual learners by up to about five points.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported gains may hinge on undisclosed per-method FL hyperparameters; the paper's own ablations show sensitivity of several accuracy points, which is larger than many of the headline improvements.","rationale":"The reader's weakest assumption identifies the same load-bearing concern. Tables 1 and 2 report many gains between 0.05 and 4.91 percentage points, while the ablation studies in Tables 6 and 7 show that FL hyperparameters alone move accuracy by roughly 3 to 4 points. Without a stated protocol for choosing E1, E2, and alpha_p, and without per-dataset values, a reviewer cannot distinguish a genuine mechanism benefit from tuning. The theoretical gradient decomposition in Eq. (25) is mathematically correct but only an algebraic identity; it does not predict which alpha_p will improve the stability-plasticity trade-off, so it cannot rescue the empirical ambiguity. I am not claiming the improvements are fabricated; rather, the reporting gap is substantial enough to make the headline claim conditionally reproducible. If the authors disclose exact hyperparameters and show that a unified protocol preserves the main gains, the conditional acceptance is justified; otherwise the empirical claim remains unverified. This does not change the reader's CONDITIONAL verdict, so I keep the verdict unchanged.","tokens_in":37246,"tokens_out":5336,"duration_ms":55333,"concrete_test":"Re-run Table 1 (CI setting) with a fixed protocol, e.g., E1=10 and alpha_p=0.01 for all five methods, and report mean accuracy over at least 3 seeds. If LwF.MC on Split CIFAR-100 no longer improves by 4.91%, or if iCaRL on Split CIFAR-10 drops below its baseline, then the headline gains depend on per-method hyperparameter selection rather than on FL alone.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that FL improves average accuracy by up to 4.91% (CI) and 3.51% (TI) across CL categories. That claim rests on a protocol that is not fully disclosed. Section 6.7 says that CL hyperparameters are kept fixed but 'we adjust the FL Hyperparameters empirically,' and the paper never lists E1, E2, alpha_p for the rows of Tables 1, 2, 4, or 5. The ablation in Table 7 shows iCaRL on Split CIFAR-10 ranges from 73.52 with alpha_p=0.001 to 69.19 with alpha_p=1, against a baseline of 70.21, while LUCIR is best at alpha_p=0.01. Table 6 shows E1=10,E2=50 gives 72.39 for iCaRL, but E1=10,E2=10 gives 69.88 and E1=50,E2=10 gives 68.41. Thus the reported +2.18 gain for iCaRL can be reversed by a different alpha_p choice, and several other reported gains are below 1%. Because the same sensitivity is not quantified for every method and benchmark in the headline tables, the positive averages could reflect per-method, per-dataset selection of FL hyperparameters rather than a robust property of the bidirectional regularizer. The theoretical section does not remove this concern: Eq. (25) is an algebraic identity rewriting the two L2 losses as one interpolation target, and it does not predict which alpha_p values will improve the stability-plasticity balance. The load-bearing condition is therefore that a single, disclosed FL hyperparameter protocol reproduces the Table 1/2 improvements.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Flashback Learning (FL), a two-phase plugin for continual learning. In Phase 1, FL starts from the model trained on previous tasks, trains briefly on the new task data to obtain a \"primary model,\" and stores this as plastic knowledge. In Phase 2, FL reinitializes to the old model and trains again on the new task with a bidirectional regularization: the host method's stability loss plus a new plasticity loss that pulls the model toward the primary model. The paper derives gradient decompositions for four CL families (distillation, replay, parameter regularization, dynamic architecture) and claims that the resulting interpolation term enhances the stability-plasticity trade-off. Empirically, it reports average accuracy improvements for representative baselines in each family, for SOTA methods BEEF and FOSTER, and compares with two other two-phase plugin methods, with additional ablations on epochs, plasticity loss scale, memory size, and backbone architecture.","tokens_in":37655,"tokens_out":6268,"duration_ms":62354,"significance":"FL is a simple and modular idea with potentially broad applicability across CL method families. The paper has clear strengths: it evaluates at least one baseline from each of four CL categories, uses publicly available baseline codebases, transparently explains the X-DER Split-CIFAR-100 discrepancy, reports statistical significance tests, and includes extensive ablations. The gradient decompositions in Theorems 1-4 are algebraically correct as identities (up to typographical slips in the appendix). However, the headline empirical claim is weakened by the fact that FL-specific hyperparameters are adjusted empirically and not disclosed for the main tables, and the ablations show sensitivity of several accuracy points. The theoretical 'enhancement' claim is also only an interpretation of the interpolation form rather than a proven consequence. If a fixed, fully disclosed FL hyperparameter protocol reproduces the Table 1-2 improvements, this would be a useful and publishable plug-in method; the current manuscript does not yet establish that robustness.","major_comments":[{"comment":"The FL-specific hyperparameters E1, E2, and alpha_p are said to be 'adjusted empirically' but are never listed for the rows of Tables 1, 2, 4, or 5. This is load-bearing because the ablations show large sensitivity: Table 7 reports iCaRL on Split CIFAR-10 ranging from 73.52 (alpha_p=0.001) to 69.19 (alpha_p=1) against a 70.21 baseline, and Table 6 reports LUCIR dropping from 80.91 to 69.15 when E1/E2 are changed. Several headline gains are comparable to or smaller than this sensitivity, e.g., LUCIR+FL +1.02 on Split CIFAR-10 and X-DER+FL +0.45 on Split CIFAR-100. Without a full hyperparameter table and a fixed selection rule, one cannot exclude per-method, per-dataset selection of FL hyperparameters as the source of the reported improvements.","section":"§6.7, Tables 6-7"},{"comment":"Theorems 1-4 are algebraic rewritings of the gradient of L_FL, and the statement that the interpolation term 'will enhance the stability-plasticity trade-off' does not follow from the displayed identity. The identity holds for any alpha_s, alpha_p, and any primary model, including one that has barely trained or one that has overfit the new task, so it cannot by itself predict changes in accuracy or forgetting. To make the theoretical contribution load-bearing, the authors should either present this as motivation (the FL gradient targets a convex combination of stable and plastic responses) or add explicit conditions under which the interpolation provably reduces forgetting or improves new-task accuracy.","section":"§4, after Eq. (25)"}],"minor_comments":[{"comment":"The headings 'Ablatin study' contain a typo and should read 'Ablation study'.","section":"§6.7.1-6.7.4"},{"comment":"Theorem 4 refers to the stable feature extractor as Eq. (6), but the stable feature extractor for dynamic architecture methods is defined in Eq. (8); Eq. (6) is the parameter-regularization stable knowledge.","section":"Theorem 4"},{"comment":"Eq. (C.25) drops the parameter multipliers in the last two terms: 'eta alpha_s F_s + eta alpha_p F_p' should be 'eta (alpha_s F_s theta_s + alpha_p F_p theta_p)'. The subsequent derivation in Eq. (C.26) and Eq. (C.27) is correct, so this is a typographical error rather than a substantive flaw.","section":"Appendix C.5, Eq. (C.25)"},{"comment":"Tables 1, 2, and 4 do not report standard deviations or the number of seeds, while Table 5 does. Reporting error bars for the small gains (e.g., +0.10, +0.45) would help the reader assess whether the differences are within run-to-run variability.","section":"Tables 1, 2, 4"},{"comment":"Table 4 mixes values directly reported from [24] with the authors' own runs; the text notes a replication baseline of 68.96 for FOSTER on ImageNet-100-B0-10, but the table itself does not mark which cells are own runs. Please mark own runs explicitly and provide the FL hyperparameters used for each SOTA integration.","section":"Table 4 and §6.5.2"},{"comment":"The Stability-Plasticity Ratio is defined as Forgetting on Old Classes divided by Accuracy on New Classes, but the numerator uses Eq. (36), which averages forgetting over all previous tasks. Please clarify the wording so that the metric name matches the quantity computed.","section":"Eq. (37), §6.8.2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to have been accepted at a journal according to the footnote; if this submission is under review at another venue, the hyperparameter disclosure issue should nonetheless be resolved before publication. I do not see evidence of inappropriate citation practices; the reliance on the authors' earlier FL conference paper [20] is openly acknowledged. The fit with a continual-learning-focused journal is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read the Flashback Learning paper. The core is a two-phase plug-in that first trains a \"primary\" model on new data for a few epochs, then reinitializes and trains with an extra plasticity regularizer pulling toward that primary model alongside the host method's stability regularizer. It is a clean extension of the same group's ICCV 2023 flashback work, now generalized to distillation, replay, regularization, and architecture methods. The gradient decompositions in Theorems 1-4 are correct algebra (one typo in Eq. C.25, missing θs/θp), and the experiments cover a solid range of baselines and benchmarks with the same total epoch budget.\n\nThe empirical claim is modest but consistent: most methods improve by 0.5 to 4.9 points, with a few near-zero or negative results (LUCIR TI -0.05, X-DER CI +0.45 after the reproduction discrepancy). That is honest reporting. The added analyses (forgetting, BWT/FWT, CKA, stability-plasticity ratio) support the mechanism's direction.\n\nThe soft spots are twofold. First, the FL hyperparameters (E1, E2, αp) are adjusted \"empirically\" per method, but the paper never gives the values used for the headline tables. The ablations show these matter: iCaRL on Split CIFAR-10 ranges from 73.52 to 69.19 depending on αp, with baseline 70.21. That is exactly the kind of sensitivity that can hide overfitting to the validation set. The paper does not show this sensitivity for LwF.MC, oEWC, X-DER, FOSTER, or BEEF, so we cannot tell whether a single fixed protocol would reproduce the gains. This is the main reason for a conditional verdict. Note that the reported iCaRL number (72.39) corresponds to αp=0.01, not the best 0.001, which slightly mitigates cherry-picking, but the missing table is still a real reproducibility gap. Second, the theory section overclaims: Eq. (25) is an algebraic identity showing gradient interpolation, not a proof that stability-plasticity balance improves. The paper's own SPR metric is closer to evidence, but it's a bespoke measurement with no error bars.\n\nThe stress-test note is mostly on target. I'd push back only on the implication that the gains are likely fabricated by tuning; the broad consistency across four method families and the honest reporting of negative results argues against that. But the lack of a disclosed hyperparameter protocol is a legitimate burden on the reader.\n\nThis deserves a serious referee. It is a useful plug-in with broad coverage, and the issues are addressable: report the full FL hyperparameter table, ship the code, and tone down the theory's causal language. I'd conditionally accept.\n\nTake care.","headline":"A useful two-phase plug-in for continual learning with consistent if modest gains, though undisclosed per-method FL hyperparameters and an overclaimed theoretical section keep it from being a clean accept.","tokens_in":38141,"tokens_out":3646,"would_cite":true,"duration_ms":34157,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Flashback Learning claims that bidirectional regularization, pulling model updates toward both old and new task knowledge, improves the stability-plasticity balance in continual learning.","keywords":["continual learning","catastrophic forgetting","stability-plasticity trade-off","bidirectional regularization","knowledge distillation","memory replay","parameter regularization","dynamic architecture"],"falsifier":"Run FL on Split CIFAR-100 class-incremental with E1=10, E2=50 and compare against an ablation where the PKB is replaced by the old model's own responses, so the plasticity loss is identical to the stability loss; if the accuracy gap vanishes, the Phase-1 learner is not adding anything beyond retraining from the old checkpoint.","tokens_in":37078,"feed_emoji":"🧠","tokens_out":4645,"duration_ms":41925,"temperature":0.7,"pith_summary":"Flashback Learning is a two-phase plugin for continual learning. In Phase 1 it quickly trains the old model on new data, saving that fast learner as 'plastic knowledge'. In Phase 2 it reinitializes to the old model and trains again while regularizing toward both the old model ('stable knowledge') and the Phase-1 learner. The paper argues this bidirectional pull makes the gradient target an interpolation between old and new responses, improving the stability-plasticity trade-off. Across replay, distillation, regularization, and architecture-expansion baselines, adding FL raises average accuracy by up to 4.91% in class-incremental and 3.51% in task-incremental settings under the same training budget.","feed_headline":"Flashback training lifts continual-learning accuracy by up to 4.9%","feed_subtitle":"A two-phase plugin gives every continual-learning method a new-task target, then pulls models toward both old and new knowledge.","key_machinery":"The mechanism is a pair of knowledge bases: SKB stores whatever stability information the host method already keeps (old model, memory logits, Fisher matrix, or frozen feature extractor), and PKB stores the corresponding information extracted from the Phase-1 primary model. The load-bearing identity is the gradient interpolation shown in Theorems 1-4, where the stability and plasticity losses combine into one term, e.g. for distillation: $\\nabla_\\theta L_{\\mathrm{FL}}(\\theta)=\\nabla_\\theta L_c(\\theta)+(\\alpha_s+\\alpha_p)\\nabla_\\theta f(x;\\theta)^\\top\\bigl(f(x;\\theta)-\\tfrac{\\alpha_s f(x;\\theta_s)+\\alpha_p f(x;\\theta_p)}{\\alpha_s+\\alpha_p}\\bigr)$. This single term replaces the pure stability gradient with a pull toward an interpolation of old and new knowledge, which is what the paper identifies as the source of the improved trade-off.","core_discovery":"The paper's central claim is that the forgetting problem can be reduced by giving the model an explicit learning target from the new task as well as the old one. FL does this in two phases: Phase 1 trains a 'primary' model on new data for few epochs; Phase 2 discards that model's weights but keeps its outputs or parameters as a plasticity target, reinitializes to the old model, and trains with a loss that pulls the network toward both targets. Theorems 1-4 show for each CL category that the FL gradient is the task gradient plus a single interpolation term, e.g. Eq. (25) for distillation, where the model is pulled toward a weighted average of stable and primary responses. The paper claims this gradient interpolation yields a better stability-plasticity balance than unidirectional regularization, and demonstrates lower stability-plasticity ratio, reduced forgetting, and accuracy gains on CIFAR and ImageNet benchmarks.","pith_inferences":["A practical reading is that FL's balance knob is the ratio $\\alpha_p/\\alpha_s$, so the method could be exposed as a single tunable dial; the paper does not propose an automatic schedule for it.","Part of the gain could be a retraining effect, since Phase 2 restarts from the old weights and sees the same task data twice; isolating the pure bidirectional-loss contribution would require an ablation with the same two-pass schedule but no plasticity term.","Because PKB mirrors SKB, FL should extend to prompt-based or low-rank continual learners by treating the prompt or LoRA state as the knowledge base, a direction the paper lists as future work but does not test."],"forward_implications":["If FL works as claimed, any continual learner that already keeps stable knowledge can get a plasticity counterpart for roughly the same training cost.","Accuracy gains of up to about 4.9 points in class-incremental and 3.5 points in task-incremental settings should transfer to other benchmarks with similar task structure.","FL should lower forgetting and improve backward transfer for replay and distillation methods, not just final accuracy.","FL should remain beneficial under smaller replay buffers and across backbone changes from ResNet to vision transformers.","Plugging FL into architecture-expansion methods such as FOSTER and BEEF should improve their ImageNet-100 accuracy without requiring extra training epochs."],"supporting_citations":[{"why":"Supplies the motivating observation that many CL methods sacrifice feature plasticity while prioritizing stability.","marker":"[15]"},{"why":"Dark Experience Replay is the replay baseline whose memory-logit stable knowledge FL augments.","marker":"[26]"},{"why":"X-DER is the extended replay baseline and source of the memory-replay experimental protocol.","marker":"[22]"},{"why":"iCaRL is the replay-and-distillation baseline and provides the herding memory selection used in the experiments.","marker":"[11]"},{"why":"LUCIR is the distillation baseline whose cosine-embedding stability loss FL carries into Phase 2.","marker":"[21]"},{"why":"Online EWC is the parameter-regularization baseline; its recursive Fisher estimate forms the stable knowledge for that category.","marker":"[5]"},{"why":"LwF is the logit-distillation baseline adapted as LwF.MC for comparison.","marker":"[6]"},{"why":"FOSTER is the architecture-expansion baseline whose boosting stage hosts Phase 1 of FL.","marker":"[23]"},{"why":"BEEF is the architecture-expansion baseline with energy-based expansion and fusion, used for SOTA comparisons including BEEF-Compress.","marker":"[24]"}],"fun_headline_variants":["Flashback Learning: two-way memory boosts CL accuracy by 4.9%","Flashback Learning harmonizes stability and plasticity in CL","New CL method uses dual knowledge bases to balance learning","Flashback Learning: swift new knowledge, retained old knowledge","Flashback Learning plugin lifts CL accuracy up to 4.9%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole benefit rests on Phase 1 producing a genuinely better plasticity target than the old model within the small number of epochs E1; if Phase 1 is too short to learn anything useful or too long it starts to forget, the bidirectional pull degrades, and Table 6 shows E1=50 can hurt relative to E1=10.","fun_headline_variants_meta":{"raw":{"variants":["Flashback Learning: two-way memory boosts CL accuracy by 4.9%","Flashback Learning harmonizes stability and plasticity in CL","New CL method uses dual knowledge bases to balance learning","Flashback Learning: swift new knowledge, retained old knowledge","Flashback Learning plugin lifts CL accuracy up to 4.9%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000605,"raw_usage":{"total_tokens":2838,"prompt_tokens":977,"completion_tokens":1861,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":593,"completion_tokens_details":{"reasoning_tokens":1774}},"tokens_in":593,"tokens_out":1861,"duration_ms":13625,"temperature":1.0,"reasoning_tokens":1774,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:03:40.731591+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run FL on Split CIFAR-100 class-incremental with E1=10, E2=50 and compare against an ablation where the PKB is replaced by the old model's own responses, so the plasticity loss is identical to the stability loss; if the accuracy gap vanishes, the Phase-1 learner is not adding anything beyond retraining from the old checkpoint.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the motivating observation that many CL methods sacrifice feature plasticity while prioritizing stability."},{"cited_title":"Buzzega, M","cited_arxiv_id":null,"evidence_quote":"Dark Experience Replay is the replay baseline whose memory-logit stable knowledge FL augments."},{"cited_title":"Boschini, L","cited_arxiv_id":null,"evidence_quote":"X-DER is the extended replay baseline and source of the memory-replay experimental protocol."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"LUCIR is the distillation baseline whose cosine-embedding stability loss FL carries into Phase 2."},{"cited_title":"Wang, D.-W","cited_arxiv_id":null,"evidence_quote":"FOSTER is the architecture-expansion baseline whose boosting stage hosts Phase 1 of FL."},{"cited_title":"Wang, D.-W","cited_arxiv_id":null,"evidence_quote":"BEEF is the architecture-expansion baseline with energy-based expansion and fusion, used for SOTA comparisons including BEEF-Compress."}],"review_version":1}