{"id":"9ab85b34-d9c2-4beb-9e96-3cf43e9feb12","arxiv_id":"2502.01866","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"OCAR combines experience replay with K-FAC Fisher preconditioning and scheduled damping to improve stability and plasticity in online continual learning.","lead":"A new continual learning method, OCAR, uses the Fisher Information Matrix to precondition gradient updates during experience replay, keeping models more stable at task boundaries. It reports better anytime accuracy than prior state-of-the-art on three online continual learning benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported gains may stem from the increasing Tikhonov schedule rather than the K-FAC preconditioner, since no ablation isolates the second-order contribution.","rationale":"After reading the paper in good faith, the strongest claim is empirical: OCAR achieves higher anytime accuracy and worst-case accuracy than strong replay baselines. For this claim to be attributed to the stated mechanism, one must show that the preconditioning by FIM is the cause. The paper does include a convex experiment comparing OCAR with NGD and EWC, which suggests the Fisher helps, but it does not compare against a first-order optimizer with a similar effective step-size schedule. The τ schedule is reminiscent of a 1/t learning-rate decay, which is known to improve online learning. The missing control is therefore a serious gap in the causal story. The reader's identified weakest assumption (EMA K-FAC nonstationarity) is also relevant, but it concerns implementation robustness rather than attribution of the improvement; even a flawed EMA could still empirically beat baselines while the reported gains stem from the schedule. Our proposed test, replacing the K-FAC preconditioner with the identity while preserving the τ and λ schedules, is a direct and reproducible check. If the identity-preconditioned version matches OCAR, the paper should be revised to reframe the contribution as mostly a scheduling/regularization effect; if not, the curvature-aware mechanism is supported and the conditional acceptance can be finalized.","tokens_in":22666,"tokens_out":9037,"duration_ms":89014,"concrete_test":"Run a controlled ablation on Split-CIFAR100 and Split-TinyImageNet, using the same code, replay buffer, and hyperparameter search protocol as the paper, but with the K-FAC preconditioner replaced by the identity matrix while keeping the identical τ and λ schedules (i.e., update δ = -α(τI)^{-1}(∇N+∇B), which is equivalent to a learning-rate schedule α_t = α/(τ_0 + tΔτ)). If this 'damped ER' baseline reaches AAA/WC-Acc within a small margin of OCAR, then the curvature-aware preconditioner is not the source of the reported gains. If OCAR still outperforms by the reported margin, the second-order information is essential and the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"OCAR's central claim is that second-order curvature information (K-FAC Fisher) drives its stability-plasticity improvement. But the algorithm confounds the preconditioner with a strong first-order effect: τ grows linearly from an initial value equal to α (Algorithm 1, line 10; Section 4.3), and the effective step size in low-curvature directions is α/(σ_i + τ) ≈ α/τ (Section 5). As τ increases, the update approaches a time-decaying SGD step. The paper reports no ablation that keeps the τ and λ schedules while removing the curvature term (e.g., F=0). Thus the large AAA/WC-Acc gains on Split-CIFAR100 and Split-TinyImageNet may be explained simply by the annealing of the effective learning rate, which is a first-order mechanism. The theoretical derivation (Section 4.1) also does not enforce the KL constraint: λ is incremented heuristically with class count or time rather than chosen to meet the constraint ρ, so the 'explicit constraints' in the title do not actually hold in the implemented algorithm. This matters because the paper's contribution is specifically attributed to leverage of second-order information; without an isolation experiment, the causal claim is unsubstantiated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper formalizes replay-based online continual learning (OCL) as a sequence of second-order optimization problems with a KL-divergence stability constraint on buffer data, and proposes OCAR, which uses a K-FAC approximation of the Fisher Information Matrix to precondition the replay gradient. A Tikhonov regularization term with a time-increasing schedule is introduced to control the stability-plasticity tradeoff. The method is evaluated on Split-CIFAR100, Split-TinyImageNet, and Online CLEAR, reporting improvements on continual metrics (AAA, WC-Acc) over strong baselines, as well as combinations with ER-ACE and DER++. The authors provide code and follow the evaluation protocol of prior OCL surveys.","tokens_in":22900,"tokens_out":5491,"duration_ms":58864,"significance":"If the claimed effects are real, OCAR is a valuable contribution: it brings second-order optimization, previously used mainly in stationary or task-boundary settings, to the online continual setting and reports large gains on the metrics that measure the stability gap. The paper is also careful to use established benchmarks, a shared evaluation protocol, and public code, and it does not fit constants to test data. The main weakness is causal attribution: the algorithm confounds the K-FAC preconditioner with a first-order learning-rate annealing schedule, and no ablation isolates the second-order contribution. The theoretical framing also overstates the role of the explicit KL constraint, since the implemented λ schedule is heuristic and no ρ is enforced. These issues are fixable with additional experiments and a revised derivation, and therefore a major revision is appropriate.","major_comments":[{"comment":"The central claim that second-order curvature information drives the stability-plasticity improvement is not supported by an isolation experiment. In Algorithm 1, τ is initialized to α and incremented by Δτ at every gradient step, while the update is δ = -α(F + τI)^{-1}∇L; in the low-curvature directions the effective step size is α/(σ_i + τ) ≈ α/τ, which decays over time. This is a first-order learning-rate annealing mechanism that exists independently of the K-FAC preconditioner. Since no ablation keeps the τ and λ schedules fixed while removing the curvature term (e.g., F=0) and no ER baseline is run with the same effective step decay, the empirical AAA/WC-Acc gains on Split-CIFAR100 and Split-TinyImageNet do not yet establish that the second-order information is the causal mechanism. Please add such an ablation and report the continual metrics.","section":"Algorithm 1, line 10; Section 5"},{"comment":"The optimization problem in Eq. (2) is posed with a hard KL constraint δ^T F_B δ ≤ 2ρ, but the implemented algorithm never chooses λ to satisfy that constraint: Algorithm 1 increases λ heuristically with the number of observed classes or with time, and no value of ρ is set or measured. As a result, the 'explicit KL-divergence constraints' described in the abstract and introduction are not actually enforced by the method used in the experiments. The authors should either derive the λ schedule from the KKT conditions of Eq. (2) and verify that the constraint is active, or reframe the method as a Tikhonov-weighted replay optimizer and provide an ablation showing what the constraint term contributes.","section":"Section 4.1, Eq. (2); Algorithm 1, lines 11-16"},{"comment":"The second-order Taylor expansion of the KL divergence is written without the standard factor 1/2 in the quadratic term: the text gives ∇^T δ + δ^T H δ instead of ∇^T δ + (1/2)δ^T H δ. The derivation then jumps from Eq. (2) to the closed-form update δ* = -(H_N + H_B + λ F_B + τI)^{-1}(∇_N + ∇_B) without writing the Lagrangian or the stationary condition. The missing 1/2 factor changes the stationary condition under a strict reading of Eq. (2), and the factor is not tracked when the Hessians are later replaced by FIMs. Please write out the expansion and the KKT system explicitly, and state which factors are exact, approximated, or absorbed into hyperparameters.","section":"Section 4.1, Eq. (1)-(2)"},{"comment":"The replacement of both Hessians by Fisher Information Matrices is justified by the FIM/GGN equivalence only under model-and-data conditions that are not checked in the nonstationary OCL setting; the text itself notes the model is far from the optimum in OCL, which is exactly where the equivalence is weakest. In addition, the boundary reset policy in Section 4.3 reinitializes only the classifier factor G while retaining the activation factor A, relying on an unstated assumption that representations are consistent across tasks. Since the optimizer's behavior is driven by this approximated curvature, I ask for a targeted check, for example comparing the K-FAC preconditioner against an exact FIM or Hessian-vector-product preconditioner on a small-scale variant, and for an ablation of the reset policy on a setting where representations change substantially.","section":"Section 4.2, Section 4.3"}],"minor_comments":[{"comment":"There is a typo in the phrase 'Stability constraint:s approach' that should be corrected to 'Stability constraint:' or 'Stability constraint approach:'. Minor: please proofread the paragraph.","section":"Section 4.1, paragraph beginning 'Stability constraint:'"},{"comment":"The hyperparameter Δτ is described as the 'increase of τ' but the units are ambiguous: Algorithm 1 increments τ inside the inner loop over gradient steps, so it should be stated explicitly whether Δτ is per gradient step, per batch, or per task, and how the value was selected in the grid search.","section":"Algorithm 1 and Section 4.3"},{"comment":"Baseline numbers in Table 1 are taken from prior papers with different numbers of seeds (LPR used 10 seeds, the other methods 5) and possibly different hyperparameter-selection procedures. A sentence in the main text acknowledging this asymmetry and its likely direction of bias, rather than only in Appendix B, would improve the fairness assessment.","section":"Tables 1 and 2; Appendix B"},{"comment":"The grid-search plots over α and α/τ appear to report a single run without confidence intervals; since Section 5 uses them to support claims (1)-(4) about the stability-plasticity tradeoff, the number of seeds and any variance should be reported.","section":"Figure 2 and Appendix Figure 11"},{"comment":"For one-hot categorical targets, the derivation uses y_i^2 = y_i without stating this identity; adding a short note makes the algebra in the diagonal FIM computation transparent.","section":"Appendix C, 'Last Layer FIM'"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely to become publishable if the authors add the isolation ablation and correct the formal derivation. In my view the missing ablation is the decisive point: without it, the stated contribution ('leveraging 2nd order information') is not yet established, even though the empirical results are promising. The reused-baseline comparison is standard in this line of work, but the seed-count asymmetry with LPR should be acknowledged. No concerns about the novelty disclosure or the citation pattern beyond normal practice."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"OCAR is a serious and well-engineered entry in online continual learning, with a fresh combination of replay and K-FAC second-order preconditioning and strong anytime-accuracy numbers. The main thing to push on is that the paper's central causal claim—second-order curvature drives the gains—is not actually isolated from the increasing Tikhonov (tau) schedule.\n\nWhat is genuinely new: the formalization of replay-based OCL as a second-order constrained problem with a KL-divergence stability constraint, and the practical adaptation of K-FAC to the nonstationary stream (EMA factors, classifier reset, lambda/tau scheduling). The empirical work is solid: three benchmarks, established codebase from Soutif-Cormerais et al. and LPR, all the continual metrics reported, code available. The results on AAA and WC-Acc are large and consistent, and the integration with ER-ACE is a nice plus. I also credit the authors for an unusually frank appendix about hyperparameter selection and computational cost.\n\nThe soft spots are real but proportionate. The stress-test concern holds up: tau starts at alpha and grows every step, so the effective step in low-curvature directions decays like alpha/tau. Without an ablation that keeps the schedule and sets the Fisher to identity (or removes the K-FAC preconditioner), a large part of the AAA/WC-Acc gain could be just a time-decaying learning rate. The convex comparison against NGD in Section 6.1 does not settle this, because NGD does not get the tau/lambda schedule. Second, the derivation in 4.1 omits the 1/2 factor in the Taylor expansion and jumps to the closed-form update without writing the KKT system; the substitution of FIM for Hessian rests on citations. Third, the title/abstract claim of 'explicit KL-divergence constraints' is too strong: lambda is increased heuristically with class count or time, so the constraint is not actually enforced at a specified rho. These are fixable, but they matter because the contribution is specifically about second-order information.\n\nThe paper is not circular or dishonest; the baselines are external and the comparisons are on held-out metrics. The weak spots are about missing evidence, not about fabricated results.\n\nMy bottom line: this should go to a serious referee. The method is strong enough and the empirical payload is large enough to warrant review, but the revision should require a proper ablation isolating the curvature term and a cleaned-up derivation. I'd bring it to our reading group, and I'd cite it if I worked on OCL.","headline":"OCAR is a serious, well-engineered OCL method with strong anytime-accuracy results, but the paper's central claim that second-order curvature drives the gains needs an ablation that isolates K-FAC from the Tikhonov schedule.","tokens_in":23429,"tokens_out":2565,"would_cite":true,"duration_ms":26900,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","68T05"],"pacs":[],"model":"deepseek-v4-flash","headline":"Replay-based online continual learning can be reframed as second-order optimization with a KL-divergence stability constraint, and the resulting OCAR optimizer beats prior state-of-the-art on continual metrics in three benchmarks.","keywords":["online continual learning","replay","second-order optimization","Fisher information matrix","K-FAC","stability gap","stability-plasticity tradeoff","natural gradient"],"falsifier":"Run OCAR on Split-CIFAR100 twice: once with the default exponential-moving-average Fisher, and once with all K-FAC factors reset at every task boundary, and record the accuracy on previously seen tasks in the steps immediately after each boundary. If the EMA-carryover version does not show a consistently smaller drop than the reset version, and than plain experience replay with matched hyperparameters, then the assumption that averaged curvature stays valid across tasks is what carries the result, not the second-order update itself.","tokens_in":22416,"feed_emoji":"🧠","tokens_out":14425,"duration_ms":115023,"temperature":0.7,"pith_summary":"Online continual learning models must stay accurate at every moment of a nonstationary stream, yet replay-based methods typically suffer a sharp accuracy drop right after each task shift, the stability gap. This paper argues that the gap is partly an optimization failure: first-order replay treats all parameter directions alike, so when new-task gradients dominate at a task boundary, old knowledge is overwritten. The authors formalize replay as a joint second-order optimization problem with an explicit constraint that the model's predictions on buffered old data barely change, measured by the KL divergence, and enforce it by preconditioning gradients with the Fisher Information Matrix, approximated block-wise by K-FAC. The resulting method, OCAR, achieves the best average-anytime and worst-case accuracy among all compared methods on Split-CIFAR100, Split-TinyImageNet, and Online CLEAR, and its 2D trajectory analysis shows much smaller deviations at task boundaries than experience replay. If the paper is right, much of the instability in online continual learning can be fixed at the optimizer level rather than by inventing new losses or architectures.","feed_headline":"Fisher-matrix preconditioner shrinks the stability gap","feed_subtitle":"OCAR, a second-order replay optimizer, beats prior methods on three benchmarks.","key_machinery":"The central object is the preconditioned update rule $\\delta^*_t = -\\alpha(F_{N_t} + (1+\\lambda)F_{B_t} + \\tau I)^{-1}(\\nabla_{N_t} + \\nabla_{B_t})$, which unifies the paper's three ingredients: the replay gradients, the Fisher Information Matrix (FIM) as a stability constraint, and Tikhonov damping as a plasticity regulator. The FIM is the Hessian of the KL divergence between pre- and post-update predictive distributions on buffer data, so the constraint 'do not change predictions on old data' becomes the quadratic form $\\delta^T F \\delta \\le \\rho$; by the Fisher/Generalized-Gauss-Newton equivalence the same FIM replaces the Hessians of the two loss terms, so one curvature quantity plays three roles at once: metric of the parameter manifold, stabilizer for old tasks, and accelerator for new ones. Tractability comes from K-FAC: the FIM is approximated block-diagonally as per-layer Kronecker products of activation and gradient factors, updated by exponential moving average, and inverted cheaply through $(A \\otimes B)^{-1} = A^{-1} \\otimes B^{-1}$. Three practical mechanisms carry the method in non-iid streams: the EMA factors, with a reset of only the classifier factor when its shape changes at a class-incremental boundary; the $\\lambda$ schedule, which strengthens the stability constraint as the buffer accumulates more classes; and the $\\tau$ schedule, which grows damping each step because a fixed small $\\tau$ dangerously amplifies low-curvature directions when new classes appear.","core_discovery":"The paper's central claim is that the instability of replay-based online continual learning is a first-order optimization artifact, and that a second-order formulation removes most of it. OCAR solves, at every step, a constrained joint optimization over new and replayed data: minimize the KL divergence of the model on both batches while requiring the KL divergence between the pre-update and post-update predictive distributions on buffer data to stay below a threshold $\\rho$. A Taylor expansion turns this into a preconditioned update $\\delta^* = -\\alpha(F_{N_t} + (1+\\lambda)F_{B_t} + \\tau I)^{-1}(\\nabla_{N_t} + \\nabla_{B_t})$, where $F$ is the Fisher Information Matrix on current and buffer data, $\\lambda$ is the Lagrange multiplier of the stability constraint, $\\tau$ is Tikhonov damping, and $\\alpha$ is the learning rate. The Fisher is the 'true' Fisher, an expectation over the model's predictive distribution rather than over observed labels, and it is made tractable with a block-diagonal K-FAC approximation whose Kronecker factors are updated by exponential moving average; at class-incremental boundaries the classifier factor is reset when its shape grows, and $\\tau$ grows each step. On this basis the paper reports state-of-the-art continual metrics, average anytime accuracy and worst-case accuracy, on Split-CIFAR100 (20 tasks), Split-TinyImageNet (20 tasks), and Online CLEAR (10 tasks), with the OCAR-ACE combination best overall on the class-incremental benchmarks. The paper also identifies the ratio $\\alpha/\\tau$, rather than $\\alpha$ or $\\tau$ alone, as the control knob of the stability-plasticity tradeoff: $\\tau$ caps the maximum acceleration in low-curvature directions, while $\\alpha$ scales all directions uniformly.","pith_inferences":["If the optimizer-level story is right, a corollary the paper leaves implicit is that the stability gap should be treatable in other nonstationary settings, such as reinforcement learning or fine-tuning on drifting data, with the same damped-Fisher preconditioning even without a replay buffer.","The reported OCAR-DER++ failure suggests a testable boundary on the method: any replay loss whose gradients are not approximately the gradient of a KL-type objective will not inherit the stability guarantee, and one could quantify this by measuring how far the DER loss's Hessian deviates from the Fisher on buffer data.","The $\\tau$-scheduling result hints that the FIM's conditioning degrades as the stream lengthens, so a natural extension is adapting $\\tau$ online from the eigenvalue spectrum of the K-FAC factors instead of a fixed additive schedule.","The reset policy assumes most boundary instability lives in the classifier, yet the strong Online CLEAR results, a domain-incremental setting with a shared classifier, suggest the reset is not the main source of gains; ablating the reset there would isolate its actual contribution."],"forward_implications":["OCAR achieves the best average anytime accuracy and worst-case accuracy among all compared methods on Split-CIFAR100 and Split-TinyImageNet, and the largest gains on Online CLEAR (75.3 vs LPR's 65.2 final accuracy), indicating that the stability gap is not an inevitable property of replay methods.","Because OCAR is an optimizer rather than a new loss or architecture, it can be stacked with other replay methods: OCAR-ACE sets the best Acc, AAA, and WC-Acc overall, while OCAR-DER++ fails on TinyImageNet, which the authors attribute to DER's entropy regularization breaking the KL assumption behind the Fisher.","The $\\alpha/\\tau$ ratio, rather than the learning rate alone, controls the effective step size in low-curvature directions, so tuning the growth of $\\tau$ on a short validation stream generalizes better to longer streams than tuning a fixed damping value.","OCAR improves linear probing accuracy on both main benchmarks, suggesting that curvature-aware updates produce better feature representations over the stream, not merely a better final classifier.","The method's runtime is about three times that of plain experience replay but below several prior sophisticated methods (MIR, SCR, LPR), so the stability gain does not come at extreme computational cost."],"supporting_citations":[{"why":"Supplies the K-FAC approximation and the Kronecker-product inversion trick that make the Fisher preconditioner computationally feasible.","marker":"(Martens & Grosse, 2015)"},{"why":"Provides the Fisher/Generalized-Gauss-Newton equivalence and the natural-gradient theory that justify replacing the loss Hessians with the Fisher Information Matrix.","marker":"(Martens, 2020)"},{"why":"The critique of the empirical Fisher that grounds OCAR's choice of the true Fisher over the empirical one used by other continual learning methods.","marker":"(Kunstner et al., 2019)"},{"why":"Experience Replay, the base first-order optimization problem OCAR extends and the main baseline it improves upon.","marker":"(Chaudhry et al., 2019b)"},{"why":"Documents the stability gap in online continual learning and defines the Average Anytime Accuracy metric used to measure it.","marker":"(Caccia et al., 2022)"},{"why":"Identifies the stability gap and contributes the worst-case accuracy metric that OCAR is evaluated on.","marker":"(De Lange et al., 2023)"},{"why":"Supplies the benchmark code, experimental setup, and baseline numbers OCAR compares against, plus evidence that prior online continual learning methods often fail to beat replay baselines.","marker":"(Soutif-Cormerais et al., 2023)"},{"why":"LPR, the state-of-the-art replay-compatible optimizer OCAR mainly competes with, including the Online CLEAR setup and results.","marker":"(Yoo et al., 2024)"}],"fun_headline_variants":["Second-order replay optimizer tightens the stability gap","Curvature-aware replay beats prior continual learning methods","Fisher matrix stabilizes online replay and boosts accuracy","OCAR: second-order replay that tames forgetting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the exponential-moving-average estimate of the Fisher matrix stays trustworthy when the data distribution shifts: if the averaged curvature mixes old and new tasks in the wrong proportions, the preconditioner can amplify exactly the wrong directions and OCAR loses its stability advantage over plain replay.","fun_headline_variants_meta":{"raw":{"variants":["Second-order replay optimizer tightens the stability gap","Curvature-aware replay beats prior continual learning methods","Fisher matrix stabilizes online replay and boosts accuracy","OCAR: second-order replay that tames forgetting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000308,"raw_usage":{"total_tokens":1850,"prompt_tokens":1122,"completion_tokens":728,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":738,"completion_tokens_details":{"reasoning_tokens":667}},"tokens_in":738,"tokens_out":728,"duration_ms":6762,"temperature":1.0,"reasoning_tokens":667,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T14:11:22.891825+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run OCAR on Split-CIFAR100 twice: once with the default exponential-moving-average Fisher, and once with all K-FAC factors reset at every task boundary, and record the accuracy on previously seen tasks in the steps immediately after each boundary. If the EMA-carryover version does not show a consistently smaller drop than the reset version, and than plain experience replay with matched hyperparameters, then the assumption that averaged curvature stays valid across tasks is what carries the result, not the second-order update itself.","supporting_citations":[{"cited_title":"Limitations of the empirical fisher approximation for natural gradient descent","cited_arxiv_id":null,"evidence_quote":"The critique of the empirical Fisher that grounds OCAR's choice of the true Fisher over the empirical one used by other continual learning methods."},{"cited_title":"Layerwise Proximal Replay: A Proximal Point Method for Online Continual Learning","cited_arxiv_id":"2402.09542","evidence_quote":"LPR, the state-of-the-art replay-compatible optimizer OCAR mainly competes with, including the Online CLEAR setup and results."}],"review_version":1}