Pith. sign in

REVIEW 3 major objections 6 minor 130 references

Retaining every past task is not always best: when tasks last long enough, starting over beats joint training.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-07-11 05:00 UTC pith:HMFNVUB4

load-bearing objection Solid reframe of CL around ALE: TE = −I + transient gap, a Critical Task Duration, and Predictive CL as the design space; theory and sign patterns hold up. the 3 major comments →

arxiv 2607.05609 v1 pith:HMFNVUB4 submitted 2026-07-06 stat.ML cs.AIcs.LG

To Retain or to Adapt? Generalizing Continual Learning

Giulia Lanzillotta , Mandana Samiei , Doina Precup , Razvan Pascanu , Claire Vernade This is my paper
classification stat.ML cs.AIcs.LG MSC 68T0590C2562L20
keywords continual learningcatastrophic forgettingAverage Lifelong ErrorTransfer EfficiencyCritical Task DurationPredictive Continual Learningonline optimizationnon-stationary environments
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Continual learning has mostly been framed as a fight against forgetting: keep performance on every past task, and you have done well. This paper argues that in changing environments that goal can hurt real-time performance. The right quantity is Average Lifelong Error—the online error the learner pays as the world moves. Comparing a joint-task learner (which mixes all past data) with an independent-task learner (which restarts on each new task) produces Transfer Efficiency, which splits cleanly into Instability (the stationary bias of clinging to outdated tasks) and Transient Error (the cost of learning from a cold start). When Instability is positive and learning converges, there is a Critical Task Duration past which memory becomes a liability. The same trade-off appears in quadratic models, deep linear nets, image-classification streams, and robotic control. The authors then treat joint-task learning as only one choice of future model and propose Predictive Continual Learning: optimize expected future loss under an explicit, updatable model of what comes next, with a Window algorithm as a simple case that can beat both extremes under controlled drift.

Core claim

Under mild task-wise convergence, Transfer Efficiency decomposes exactly as TE_k = −I_k + (δ_k(ITL) − δ_k(JTL)). Whenever average Instability is positive and cumulative Transient Error grows sub-linearly in task length N, there is a finite Critical Task Duration N_∞^max beyond which retaining the full history becomes detrimental to real-time error. Joint-Task Learning is only the special case of an ergodic future model; the right objective is Predictive Continual Learning under an explicit model of future tasks.

What carries the argument

Transfer Efficiency TE_k and its exact split into Instability I_k (stationary gap between joint and independent solutions) and Transient Error δ_k (average excess risk while approaching stationarity), which together yield the closed-form Critical Task Duration N_∞^max = (Δ̄_ITL − Δ̄_JTL)/Ī.

Load-bearing premise

The argument needs the learner's parameters to settle to a stable distribution on each task, with the leftover learning cost growing slower than the task length itself—stated generally but checked carefully only for quadratic losses with aligned curvatures and small fixed step sizes.

What would settle it

On a controlled drift sequence with measured positive Instability, sweep task duration N and check whether Transfer Efficiency falls as 1/N and crosses from positive to negative near the predicted Critical Task Duration; if TE stays positive for arbitrarily large N, the threshold claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper challenges the retention-centered premise of continual learning, arguing that Joint-Task Learning (JTL) is not a universally optimal target under non-stationarity. Centering on Average Lifelong Error (ALE), it formalizes CL as the interaction of environment and learning dynamics and defines Transfer Efficiency (TE) as the gap between Independent-Task Learning (ITL) and JTL. TE decomposes exactly into Instability (stationary bias of joint training) and a Transient Error gap (learning-speed advantage of warm starts). Under task-wise convergence and sub-linear cumulative transient cost, this yields a Critical Task Duration beyond which retention becomes detrimental whenever average Instability is positive. Closed forms are derived for quadratic losses via Lyapunov analysis of SGD moments, governed by an environment Growth Rate; the same qualitative mechanisms are tested on CLEAR, MD5, controlled CIFAR variants, and MT10. Connecting CL to predictable sequences, the authors propose Predictive Continual Learning and analyze a Window algorithm that interpolates between ITL and JTL and outperforms both under controlled drift.

Significance. If the central claims hold, the paper supplies a useful conceptual and analytical reorientation of continual learning: from catastrophic-forgetting mitigation toward real-time adaptation under explicit future models. The TE decomposition and Critical Task Duration give falsifiable, environment-dependent predictions rather than a fixed retention objective. Strengths include an exact algebraic TE identity under stationarity, closed-form quadratic expressions with Lyapunov arguments that match simulations, controlled empirical checks of the predicted 1/N decay and Instability shifts, and a clean link from JTL/ITL to the predictable-sequences framework. The Window algorithm is a concrete, if simple, proof of concept that the design space between full retention and full reset is actionable. These contributions are of clear interest to the CL, online learning, and multi-task communities.

major comments (3)
  1. [Abstract / §3.1 / Lemma 6] Abstract and §3.1 / Lemma 6 (and Appendix Lemma 16): the claim that the Critical Task Duration holds under mild conditions "across linear and neural network models" rests on task-wise convergence of Q_n to a stationary Q_\infty plus \Delta_k(A)\in o(N). The appendix rigorously supports this for quadratic losses with aligned Hessians and small constant step sizes (Assumptions 17, 22, 23); for deep nets the text only invokes PL/NTK-style rates for sub-linearity. Please either (i) state more carefully that existence of N_\infty^max is conditional on task-wise stationarity (which is assumed, not proved, for ResNet/SAC), or (ii) add a short empirical diagnostic that the per-task loss trajectories on CLEAR/MD5/MT10 have effectively plateaued so that the estimated I_k is not still contaminated by unfinished transients. The algebraic TE identity (Lemma 15) is fine; the scope of the existence cla
  2. [Table 1 / §3.2 / MT10] Table 1 and MT10 setup (§3.2, Appendix B.4): ALS_ITL = 0.47\pm0.47 and ALS_JTL = 0.28\pm0.45 indicate extremely high seed-to-seed variance relative to the reported TE of -0.19\pm0.02. Given that the qualitative claim is that high Instability makes JTL worse, please clarify how TE standard errors are computed across seeds/tasks, whether some tasks dominate the average (cf. the asymmetric pairwise matrix in Fig. 21), and whether the negative-TE conclusion is robust under the B=5000 ablation mentioned in the appendix. Without that, the RL evidence is directionally consistent but statistically weaker than the supervised results.
  3. [§5.2 / Eq. (121)] §5.2 and Eq. (121): the Window algorithm is the main constructive contribution of Predictive CL, yet W\bigstar is derived from environment and optimizer parameters (\sigma_\Delta^{2}, Tr(\Sigma_x \Sigma_\Delta), \eta, B, etc.) that a lifelong agent would not know a priori. The simulations set W to the theoretical optimum. Please discuss how W could be selected or adapted online (e.g., from recent loss trends or a small validation stream), or explicitly frame the current results as an oracle upper bound on the Predictive-CL design space. Without this, the practical advantage over JTL/ITL remains partly hypothetical outside the controlled quadratic drift model.
minor comments (6)
  1. [Figure 4 / Table 2] Figure 4 / Table 2: CLEAR never crosses TE=0 within the tested N range; the breakeven is only extrapolated. A short note in the caption or text that N_\infty^max lies outside the computational budget (consistent with small I) would avoid over-reading the table as a full crossing of the critical horizon.
  2. [§2–§3] Notation: ALEk, TE, I_k, \delta_k, and Q_n are introduced carefully, but the main text sometimes uses AJTL/AITL and sometimes JTL/ITL for the same objects. A single convention after Definition 2 would help.
  3. [Figure 3] Figure 3 concept map is helpful but dense; consider moving the full literature-equivalence map to the appendix and keeping a lighter version in the main text.
  4. [§2.4 / A.2.2] Appendix A.2.2 on warm-start ITL is important for interpreting the baseline; a one-sentence pointer in the main text near Definition 2 would prevent readers from missing that W=1 recovers warm-start ITL.
  5. [Throughout] Typos / polish: "the environment Growth Rate determines" (abstract-level phrasing is fine); check "computations" vs "compute" consistency; ensure all theorem numbers in the main text match the appendix (Lemma 5 vs 15, Lemma 6 vs 16).
  6. [§6] Related work: the connection to dynamic regret and path variation V_T (Besbes et al., 2015) is well drawn; a brief explicit comparison of Growth Rate \psi(k) to V_T in the main text (not only the appendix) would strengthen the online-learning bridge.

Circularity Check

0 steps flagged

No significant circularity: TE, Instability, and Transient Error are operational definitions whose algebraic rearrangement yields the Critical Task Duration under stated convergence hypotheses.

full rationale

The load-bearing chain is: (i) ALE as the online objective (imported from Kumar et al. 2025, whose author list has no overlap with the present paper); (ii) operational definitions TE_k := ALE_k(ITL)−ALE_k(JTL), I_k := L_k(Q^JTL_∞)−L_k(Q^ITL_∞), δ_k as average excess over stationary risk; (iii) the exact identity TE_k = −I_k + (δ_ITL−δ_JTL) obtained by writing ALE_k = L_k(Q_∞)+δ_k; (iv) Critical Task Duration N_∞^max = (Δ̄_ITL−Δ̄_JTL)/Ī by solving average TE=0 under Ī>0 and Δ̄∈o(N). None of these steps fits a fitted parameter to data and renames it a prediction, nor defines X in terms of Y while claiming an independent derivation. The quadratic closed forms (Theorems 24–25, 8) and Window W⋆ (Eq. 121) are derived from SGD moment dynamics under explicit assumptions (aligned Hessians, small η, Growth Rate/MSD), then checked on simulations and external benchmarks (CLEAR, MD5, MT10, Permuted/Shuffled CIFAR). Self-citations are absent from the premise chain; Kumar et al. is an external starting point. The paper is self-contained against external benchmarks and does not reduce its central claim by construction to its inputs.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 6 invented entities

The central Critical-Task-Duration claim rests on standard SGD moment analysis plus three paper-specific modeling choices (task-wise stationarity of Q_∞, sub-linear cumulative transient, and, for closed forms, quadratic aligned Hessians). Free parameters appear only in experiments and in the Window-size formula; the qualitative existence of a finite horizon does not require fitting them. Invented entities are operational definitions that are measured on both synthetic and real streams.

free parameters (4)
  • learning rate η
    Appears in closed-form Instability/Transient expressions and in the asymptotic N_∞^max; chosen by grid search per agent/benchmark in experiments.
  • batch size B
    Scales the variance term of Instability (η/4B V_div); fixed or ablated but not derived from first principles.
  • window size W⋆
    Derived theoretically for the Continuous Drift Model (Eq. 121) yet still depends on environment and optimizer constants that must be known or estimated.
  • task duration N / drift magnitude σ_Δ²
    Swept experimentally to trace phase boundaries; the qualitative existence of N_∞^max does not require specific values, but quantitative plots do.
axioms (5)
  • domain assumption Task-wise convergence: for each task the Markov operator T_k admits a unique stationary measure Q_∞ to which Q_n converges in W_2 (Definition 12).
    Invoked for the exact TE decomposition (Lemma 15) and Critical Task Duration (Lemma 16); verified rigorously only for quadratic SGD.
  • domain assumption Cumulative Transient Error is sub-linear: Δ_k(A) ∈ o(N) (Lemma 16).
    Needed so that the 1/N-averaged transient vanishes and a finite breakeven horizon exists when Ī > 0; holds for strongly convex and PL regimes but is an assumption for general nets.
  • ad hoc to paper Quadratic losses with simultaneously diagonalizable (aligned) Hessians and bounded spectrum (Assumptions 17, 22, 23).
    Used for all closed-form Instability, Transient Potential Law and asymptotic Growth-Rate results; not required for the qualitative Critical Task Duration statement.
  • domain assumption Known task boundaries and locally stationary task-based data process (Definition 1).
    Standard CL abstraction; the theory is stated under this partition.
  • domain assumption ALE (Average Lifelong Error) is the appropriate lifelong objective under computational constraints (Kumar et al. 2025).
    The entire analysis optimizes or compares agents w.r.t. ALE; alternative objectives (worst-case, fairness, coverage) may reverse rankings.
invented entities (6)
  • Transfer Efficiency (TE_k) independent evidence
    purpose: Scalar that quantifies when JTL outperforms ITL on task k.
    Defined as ALE_k(ITL) − ALE_k(JTL); measured empirically and decomposed analytically.
  • Instability (I_k) independent evidence
    purpose: Stationary performance gap between JTL and ITL solutions.
    Defined as L_k(Q_JTL_∞) − L_k(Q_ITL_∞); estimated from converged ensembles.
  • Transient Error (δ_k) independent evidence
    purpose: Average excess risk while converging to stationarity.
    Standard in stochastic optimization; here lifted to the CL comparison.
  • Critical Task Duration (N_∞^max) independent evidence
    purpose: Closed-form horizon beyond which retention becomes detrimental when Ī > 0.
    Derived from TE = 0 under sub-linearity; the central quantitative prediction.
  • Predictive Continual Learning / Window algorithm independent evidence
    purpose: General class of algorithms that optimize under an explicit future model Φ; Window is the finite-memory instance.
    Defined in Definition 9; Window recovers JTL/ITL as limits and is shown to dominate both under controlled drift.
  • Growth Rate ψ(k) / Mean Squared Displacement m(k,W) independent evidence
    purpose: Environment descriptors that govern asymptotic TE sign and optimal window size.
    Defined to collapse the large-k analysis; measured on synthetic random-walk environments.

reviewed 2026-07-11 · how reviews work

0 comments
Cite this review

Pith. "Pith review of To Retain or to Adapt? Generalizing Continual Learning." pith.science (2026). https://pith.science/paper/HMFNVUB4

@misc{pith2026260705609,
  author       = {Pith},
  title        = {Pith review of: To Retain or to Adapt? Generalizing Continual Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HMFNVUB4}},
  note         = {Machine review of arXiv:2607.05609}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

The Continual Learning (CL) literature has long been driven by the goal of mitigating catastrophic forgetting. This objective rests on a pervasive, often unstated assumption: that a lifelong learner should approximate the Joint-Task Learning (JTL) solution and retain all previously acquired knowledge. We challenge this retention-centered premise, arguing that in non-stationary environments prioritizing retention can impede real-time adaptation. Shifting the focus to the Average Lifelong Error (ALE), we formalize CL as an online optimization problem governed by the interaction between environmental and learning dynamics. We introduce Transfer Efficiency as a quantitative measure of the tension between Instability, the bias inherited from conflicting past experience, and Transient Error, the optimization cost of learning new tasks from scratch. Under mild convergence conditions, holding across linear and neural network models, this decomposition yields a Critical Task Duration: a closed-form threshold beyond which historical knowledge transitions from a warm-start advantage to an optimization liability whenever retention induces a positive stationary bias. We validate these theoretical predictions on continual image classification and reinforcement learning benchmarks. Finally, by connecting continual learning to the online learning framework of predictable sequences, we show that JTL is only one instance of a broader family of objectives, and we propose a new general class of continual learning algorithms, which we call Predictive Continual Learning. Predictive CL algorithms optimize expected future performance under an explicit, dynamically updated model of future tasks. As a proof of concept, we analyze a Window algorithm that interpolates between JTL and Independent-Task Learning (ITL), outperforming both under controlled distributional drift.

Figures

Figures reproduced from arXiv: 2607.05609 by Claire Vernade, Doina Precup, Giulia Lanzillotta, Mandana Samiei, Razvan Pascanu.

Figure 1
Figure 1. Figure 1: General Continual-learning objectives. ITL and JTL are two lim￾iting cases of a broader continuum. ITL makes the weakest structural assumption, optimizing only for the current task. JTL makes a strong predictable-structure as￾sumption, using the accumulated past as evidence for future performance. Many ex￾isting continual-learning methods can be viewed as approximations to this JTL-like objective, because … view at source ↗
Figure 2
Figure 2. Figure 2: The environment and learning (or agent) stochastic processes, whose interaction gives rise to the history of actions and observations Hn = {(a0, o1), . . . ,(an−1, on)}. Dashed lines indicate conditioning, coloured solid lines indicate sampling. Samples of random variables are enclosed in circles, while stochastic functions in squares. The supervised￾learning inset marks the action-to-environment condition… view at source ↗
Figure 3
Figure 3. Figure 3: Concept map relating our framework to adjacent learning paradigms. The central blue spine shows our decomposition: the Average Task Error (ALEk ) is decomposed into the Transfer Efficiency (TEk ), which in turn splits into the Instability (I k ) and the Transient Error (δ k ). Each labelled arrow names the formal equivalent of the originating concept in one of four adjacent literatures (Online Learning, Co… view at source ↗
Figure 4
Figure 4. Figure 4: Representative continual-learning trajectories for supervised benchmarks and MT10. Top two rows: CLEAR and MD5 task trajectories, showing smooth shifts with positive JTL transfer versus sharp semantic shifts with negative transfer. Bottom row: selected MT10 per-task success-rate trajectories on Meta-World in an easy-to-hard ordering with batch size 5000 (drawer-close, window-close, reach). Both supervised … view at source ↗
Figure 5
Figure 5. Figure 5: confirms each of the two halves of P1. First, increasing the shift magnitude (PC-16→32, SC￾10→30→50) translates the TE curves vertically downward, exactly as the additive −Ik term in Equation (2) predicts. Second, the empirical TE is well fit by a 1/N curve across all settings, indicating that the per-step transient gap L k (Qn) − Lk (Q∞) vanishes super-linearly — the regime in which Theorem 6 applies. The… view at source ↗
Figure 6
Figure 6. Figure 6: Task Environment Growth Rates. Top Row: 2D spatial trajectories of the task optimal parameters θ k ⋆ for a sin￾gle random seed over K = 50 tasks. Each panel uses its own spatial scale (note the differing axis ranges, roughly ±1, ±4, and ±10), so that the trajectory remains visible despite the very different expansion rates. The dashed gray circles represent the the￾oretical expected distance from the origi… view at source ↗
Figure 7
Figure 7. Figure 7: Transfer Efficiency in the Continuous Drift Model. Simulated vs. theoretical TE across task durations (N, x-axis) and drift magnitudes (σ 2 ∆, y-axis), at k ∈ {10, 50, 100, 500}. Violet: positive transfer; orange: negative transfer. As k grows, the positive-transfer region shrinks, tracking the theoretical boundary from Equation (94). Simulations of this model ( [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Optimization bias shifts the Transfer Efficiency front, at fixed function class. TE in the Continuous Drift Model versus N and σ 2 ∆. Left: convex Linear Model. Middle: Deep Linear Network, Lazy Regime (NTP). Right: Deep Linear Network, Rich Regime (µP). The function class is identical across the three panels; the parametrization is not [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Real-Time Performance of the Window Algorithm. Test loss trajectories for the ITL (orange), JTL (violet), and Window (green) agents across varying task durations N. The curves are smoothed by averaging over N steps. By setting the window size to the theoretical optimum W⋆ derived in Equation (121), the Window agent consistently matches or outperforms both baselines. Notably, the optimal memory horizon is i… view at source ↗
Figure 10
Figure 10. Figure 10: Asymptotic Transfer Efficiency (k → ∞). The theoretical landscape of Transfer Efficiency in the limit of infinite tasks. The white dashed line marks the exact Critical Task Duration N∞ max derived in Equation (5). In this asymptotic regime, the boundary between positive and negative transfer becomes a vertical line, showing that the critical task horizon depends on spatial trace properties and learning ra… view at source ↗
Figure 11
Figure 11. Figure 11: Learning rate sweeps across widths for ITL training in NTP and [PITH_FULL_IMAGE:figures/full_fig_p055_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Learning rate sweeps for JTL training in NTP and [PITH_FULL_IMAGE:figures/full_fig_p055_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Loss curve for ITL training (first task) in NTP and [PITH_FULL_IMAGE:figures/full_fig_p055_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Loss curve for JTL training (three tasks) in NTP and [PITH_FULL_IMAGE:figures/full_fig_p056_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Transfer Efficiency Landscape for the Window Agent. The grids display the Transfer Efficiency of the Window agent as a function of Window Size W (x-axis) and Drift Magnitude σ 2 ∆ (y-axis). From left to right, the task duration increases (N = 100, 500, 1000). The area of positive transfer (white to orange) shrinks significantly as both task duration and drift magnitude increase. This visually confirms the… view at source ↗
Figure 16
Figure 16. Figure 16: Samples from the CLEAR benchmark. Each column corresponds to a different temporally evolving [PITH_FULL_IMAGE:figures/full_fig_p058_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Samples from the MD5 benchmark. Each row corresponds to a completely distinct classification [PITH_FULL_IMAGE:figures/full_fig_p058_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Samples from a Permuted CIFAR-10 task with a 16 [PITH_FULL_IMAGE:figures/full_fig_p060_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Samples from a Permuted CIFAR-10 task with a 32 [PITH_FULL_IMAGE:figures/full_fig_p060_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Decomposition of Transfer Efficiency and the Critical Task Duration. (a) Decomposition of Learning Curves: Expected loss L k (θn) over time n for the JTL (violet) and ITL (orange) agents. The shaded areas represent the Transient Errors (δ k ) accumulated as the agents converge toward their respective stationary risks. The gap between these asymptotic limits rep￾resents the task Instability (I k ). While J… view at source ↗
Figure 21
Figure 21. Figure 21: Pairwise ITL-vs-JTL transfer is asymmetric. Each cell is the normalized gap ∆(ti | tj ) = (ITL(ti) − JTL(ti | tj ))/ITL(ti) for the row task ti when co-trained with the column task tj . Red (∆ > 0) marks pairs where joint training (JTL) makes the row task worse than its ITL (single-task) specialist (ITL better); blue (∆ < 0) marks pairs where JTL makes the row task better (JTL better); the diagonal is 0 b… view at source ↗
Figure 22
Figure 22. Figure 22: Lazy regime simulations. From left to right the model width is increased from 32 to 1024 (2 5 folds). 66 [PITH_FULL_IMAGE:figures/full_fig_p066_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: Rich regime simulations. From left to right the model width is increased from 32 to 1024 (2 5 folds). 67 [PITH_FULL_IMAGE:figures/full_fig_p067_23.png] view at source ↗
Figure 24
Figure 24. Figure 24: Scaling of Instability and Transient Er￾ror with Network Width. Empirical measurements taken in the deep linear network setup at a fixed task duration N = 1000 and drift magnitude σ 2 ∆ = 0.05. Left: Instability (approximated as the average loss at convergence) decreases as network width scales up, but remains several orders of magnitude higher under the NTP parametrization (black circles) compared to µP … view at source ↗
Figure 25
Figure 25. Figure 25: Continuous-drift shallow-linear trajectories. ITL and JTL losses are shown across task boundaries for several drift magnitudes, together with the instantaneous JTL advantage. Small drift produces positive transfer because JTL reduces repeated transient learning costs; larger drift erodes this advantage as the joint solution becomes increasingly biased toward obsolete tasks [PITH_FULL_IMAGE:figures/full_f… view at source ↗
Figure 26
Figure 26. Figure 26: Cross-benchmark comparison at task duration N = 3000. Real-time performance trajectories of JTL and ITL agents across the supervised image-classification benchmarks at a fixed task duration. See Section B.2 for hyperparameter details. The supervised trajectories in [PITH_FULL_IMAGE:figures/full_fig_p069_26.png] view at source ↗
Figure 27
Figure 27. Figure 27: Real-time training dynamics on CLEAR (left) and MD5 (right) at task duration N = 3000. Per-task evaluation accuracy of JTL and ITL agents across the full task sequence. The two benchmarks exhibit qualitatively different transfer regimes: smooth temporal drift in CLEAR (where JTL transfer is positive) versus sharp semantic shifts in MD5 (where JTL transfer becomes detrimental). 69 [PITH_FULL_IMAGE:figures… view at source ↗
Figure 28
Figure 28. Figure 28: Window-agent trajectory zooms. The first and last ten tasks are shown for several task durations. The window agent keeps only a finite recent history, interpolating between ITL, which uses only the current task, and JTL, which retains the entire stream. The window-agent zooms in [PITH_FULL_IMAGE:figures/full_fig_p070_28.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

130 extracted references · 5 canonical work pages · 4 internal anchors

  1. [1]

    Journal of Artificial Intelligence Research , volume=

    Towards continual reinforcement learning: A review and perspectives , author=. Journal of Artificial Intelligence Research , volume=

  2. [2]

    International Conference on Machine Learning , pages=

    Minimax regret bounds for reinforcement learning , author=. International Conference on Machine Learning , pages=. 2017 , organization=

    2017

  3. [3]

    Machine learning , volume=

    Multitask learning , author=. Machine learning , volume=. 1997 , publisher=

    1997

  4. [4]

    International Conference on Machine Learning , pages=

    Characterizing implicit bias in terms of optimization geometry , author=. International Conference on Machine Learning , pages=. 2018 , organization=

    2018

  5. [5]

    Advances in Neural Information Processing Systems , volume=

    Gradient surgery for multi-task learning , author=. Advances in Neural Information Processing Systems , volume=

  6. [6]

    International conference on machine learning , pages=

    Online multi-task learning for policy gradient methods , author=. International conference on machine learning , pages=. 2014 , organization=

    2014

  7. [7]

    Advances in neural information processing systems , volume=

    Non-stationary Markov decision processes, a worst-case approach using model-based reinforcement learning , author=. Advances in neural information processing systems , volume=

  8. [8]

    2020 , school=

    Reinforcement learning in non-stationary environments , author=. 2020 , school=

    2020

  9. [9]

    International Conference on Learning Representations (ICLR) , year=

    The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting--An Analytical Model , author=. International Conference on Learning Representations (ICLR) , year=

  10. [10]

    arXiv preprint arXiv:2310.00137 , year=

    On the Disconnect Between Theory and Practice of Overparametrized Neural Networks , author=. arXiv preprint arXiv:2310.00137 , year=

    Pith/arXiv arXiv

  11. [11]

    Advances in Neural Information Processing Systems , volume=

    Continual world: A robotic benchmark for continual reinforcement learning , author=. Advances in Neural Information Processing Systems , volume=

  12. [12]

    Advances in Neural Information Processing Systems , volume=

    Continual learning in environments with polynomial mixing times , author=. Advances in Neural Information Processing Systems , volume=

  13. [13]

    1994 , school =

    Ring, Mark Bishop , title =. 1994 , school =

    1994

  14. [14]

    The biology and technology of intelligent autonomous agents , pages=

    Lifelong robot learning , author=. The biology and technology of intelligent autonomous agents , pages=. 1995 , publisher=

    1995

  15. [15]

    Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges

    Timothée Lesort and Vincenzo Lomonaco and Andrei Stoian and Davide Maltoni and David Filliat and Natalia Díaz-Rodríguez. Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges. Information Fusion. 2020. doi:https://doi.org/10.1016/j.inffus.2019.12.004

    doi:10.1016/j.inffus.2019.12.004 2020

  16. [16]

    Neural Tangent Kernel: Convergence and Generalization in Neural Networks , url =

    Jacot, Arthur and Gabriel, Franck and Hongler, Cl. Neural Tangent Kernel: Convergence and Generalization in Neural Networks , url =. Advances in Neural Information Processing Systems , editor =

  17. [17]

    International Conference on Machine Learning , pages =

    Tensor Programs IV: Feature Learning in Infinite-Width Neural Networks , author =. International Conference on Machine Learning , pages =. 2021 , organization =

    2021

  18. [18]

    Advances in Neural Information Processing Systems , volume =

    Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer , author =. Advances in Neural Information Processing Systems , volume =. 2021 , url =

    2021

  19. [19]

    ArXiv , year=

    Task Agnostic Continual Learning via Meta Learning , author=. ArXiv , year=

  20. [20]

    ArXiv , year=

    Improving and Understanding Variational Continual Learning , author=. ArXiv , year=

  21. [21]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , year=

    Learning without forgetting , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , year=

  22. [22]

    ArXiv , year=

    Adapting Auxiliary Losses Using Gradient Similarity , author=. ArXiv , year=

  23. [23]

    Neural networks , volume=

    Continual lifelong learning with neural networks: A review , author=. Neural networks , volume=. 2019 , publisher=

    2019

  24. [24]

    International Conference on Learning Representations , year=

    Functional Regularisation for Continual Learning with Gaussian Processes , author=. International Conference on Learning Representations , year=

  25. [25]

    Machine learning , volume=

    Near-optimal reinforcement learning in polynomial time , author=. Machine learning , volume=. 2002 , publisher=

    2002

  26. [26]

    arXiv preprint arXiv:2210.07931 , year=

    Sequential Learning Of Neural Networks for Prequential MDL , author=. arXiv preprint arXiv:2210.07931 , year=

    Pith/arXiv arXiv

  27. [27]

    Foundations and Trends

    Introduction to online convex optimization , author=. Foundations and Trends. 2016 , publisher=

    2016

  28. [28]

    Foundations and Trends

    Introduction to multi-armed bandits , author=. Foundations and Trends. 2019 , publisher=

    2019

  29. [29]

    Advances in minimum description length: Theory and applications , volume=

    Minimum description length tutorial , author=. Advances in minimum description length: Theory and applications , volume=. 2005 , publisher=

    2005

  30. [30]

    Measures of Complexity: Festschrift for Alexey Chervonenkis , pages=

    On Martingale Extensions of Vapnik--Chervonenkis Theory with Applications to Online Learning , author=. Measures of Complexity: Festschrift for Alexey Chervonenkis , pages=. 2015 , publisher=

    2015

  31. [31]

    Advances in Neural Information Processing Systems , volume=

    Lifelong learning with non-iid tasks , author=. Advances in Neural Information Processing Systems , volume=

  32. [32]

    International Conference on Machine Learning , pages=

    Which tasks should be learned together in multi-task learning? , author=. International Conference on Machine Learning , pages=. 2020 , organization=

    2020

  33. [33]

    Advances in neural information processing systems , volume=

    Multi-task learning as multi-objective optimization , author=. Advances in neural information processing systems , volume=

  34. [34]

    Foundations and Trends in Machine Learning , volume=

    Continual Learning as Computationally Constrained Reinforcement Learning , author=. Foundations and Trends in Machine Learning , volume=. 2025 , publisher=

    2025

  35. [35]

    Proceedings of the 20th international conference on machine learning (icml-03) , pages=

    Online convex programming and generalized infinitesimal gradient ascent , author=. Proceedings of the 20th international conference on machine learning (icml-03) , pages=

  36. [36]

    arXiv preprint arXiv:1912.13213 , year=

    A modern introduction to online learning , author=. arXiv preprint arXiv:1912.13213 , year=

    Pith/arXiv arXiv 1912

  37. [37]

    2004 , publisher=

    Convex optimization , author=. 2004 , publisher=

    2004

  38. [38]

    Advances in neural information processing systems , volume=

    Principles of risk minimization for learning theory , author=. Advances in neural information processing systems , volume=

  39. [39]

    Neural Networks , volume=

    A wholistic view of continual learning with deep neural networks: Forgotten lessons and the bridge to active and open world learning , author=. Neural Networks , volume=. 2023 , publisher=

    2023

  40. [40]

    Proceedings of the 35th International Conference on Machine Learning , pages =

    Progress & Compress: A scalable framework for continual learning , author =. Proceedings of the 35th International Conference on Machine Learning , pages =. 2018 , editor =

    2018

  41. [41]

    International Conference on Artificial Intelligence and Statistics , pages=

    Nonstationary bandit learning via predictive sampling , author=. International Conference on Artificial Intelligence and Statistics , pages=. 2023 , organization=

    2023

  42. [42]

    International Conference on Learning Representations , year=

    Fortuitous forgetting in connectionist networks , author=. International Conference on Learning Representations , year=

  43. [43]

    Operations research , volume=

    Non-stationary stochastic optimization , author=. Operations research , volume=. 2015 , publisher=

    2015

  44. [44]

    arXiv preprint arXiv:2108.06325 , year=

    Continual backprop: Stochastic gradient descent with persistent randomness , author=. arXiv preprint arXiv:2108.06325 , year=

    Pith/arXiv arXiv

  45. [45]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Lipschitz lifelong reinforcement learning , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  46. [46]

    International Conference on Machine Learning , pages=

    Theory on forgetting and generalization of continual learning , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  47. [47]

    Conference on Learning Theory , pages=

    How catastrophic can catastrophic forgetting be in linear regression? , author=. Conference on Learning Theory , pages=. 2022 , organization=

    2022

  48. [48]

    On Angles between Subspaces of a Finite Dimensional Inner Product Space Per

    Space, Finite Dimensional Inner Product , booktitle=. On Angles between Subspaces of a Finite Dimensional Inner Product Space Per. 1983 , organization=

    1983

  49. [49]

    Proceedings of the IEEE/CVF international conference on computer vision , pages=

    Online continual learning with natural distribution shifts: An empirical study with visual data , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=

  50. [50]

    Advances in neural information processing systems , volume=

    Gradient episodic memory for continual learning , author=. Advances in neural information processing systems , volume=

  51. [51]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Task-free continual learning , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  52. [52]

    Communications of the ACM , volume=

    Understanding deep learning (still) requires rethinking generalization , author=. Communications of the ACM , volume=. 2021 , publisher=

    2021

  53. [53]

    Advances in Neural Information Processing Systems , volume=

    Online structured laplace approximations for overcoming catastrophic forgetting , author=. Advances in Neural Information Processing Systems , volume=

  54. [54]

    Advances in neural information processing systems , volume=

    Dark experience for general continual learning: a strong, simple baseline , author=. Advances in neural information processing systems , volume=

  55. [55]

    Proceedings of the IEEE conference on Computer Vision and Pattern Recognition , pages=

    Packnet: Adding multiple tasks to a single network by iterative pruning , author=. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition , pages=

  56. [56]

    Conference on Lifelong Learning Agents , pages=

    Cora: Benchmarks, baselines, and metrics as a platform for continual reinforcement learning agents , author=. Conference on Lifelong Learning Agents , pages=. 2022 , organization=

    2022

  57. [57]

    Local vs Global continual learning

    Local vs Global continual learning , author=. arXiv preprint arXiv:2407.16611 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  58. [58]

    Optimization and Generalization of Regularization-Based Continual Learning: a Loss Approximation Viewpoint

    Optimization and generalization of regularization-based continual learning: a loss approximation viewpoint , author=. arXiv preprint arXiv:2006.10974 , year=

    work page internal anchor Pith review Pith/arXiv arXiv 2006

  59. [59]

    Journal of Numerical Mathematics , volume=

    Angles between subspaces and their tangents , author=. Journal of Numerical Mathematics , volume=. 2013 , publisher=

    2013

  60. [60]

    French, Robert M. , doi=. Catastrophic forgetting in connectionist networks , volume=. Trends in Cognitive Sciences , number=. 1999 , keywords=

    1999

  61. [61]

    1989 , issn =

    Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem , editor =. 1989 , issn =. doi:https://doi.org/10.1016/S0079-7421(08)60536-8 , url =

    doi:10.1016/s0079-7421(08)60536-8 1989

  62. [62]

    Proceedings of the National Academy of Sciences , volume=

    Overcoming catastrophic forgetting in neural networks , author=. Proceedings of the National Academy of Sciences , volume=. 2017 , publisher=

    2017

  63. [63]

    Proceedings of the 34th International Conference on Machine Learning (ICML) , volume=

    Continual learning through synaptic intelligence , author=. Proceedings of the 34th International Conference on Machine Learning (ICML) , volume=. 2017 , organization=

    2017

  64. [64]

    Proceedings of the European Conference on Computer Vision (ECCV) , pages=

    Memory Aware Synapses: Learning What (not) to Forget , author=. Proceedings of the European Conference on Computer Vision (ECCV) , pages=. 2018 , publisher=

    2018

  65. [65]

    Advances in Neural Information Processing Systems , volume=

    Experience replay for continual learning , author=. Advances in Neural Information Processing Systems , volume=

  66. [66]

    Nature , volume=

    Human-level control through deep reinforcement learning , author=. Nature , volume=. 2015 , publisher=

    2015

  67. [67]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=

    A continual learning survey: Defying forgetting in classification tasks , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=. 2022 , publisher=

    2022

  68. [68]

    Advances in Neural Information Processing Systems (NeurIPS) , year=

    Continual Evaluation for Lifelong Learning: Identifying the Stability Gap , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=

  69. [69]

    2006 , publisher=

    Prediction, learning, and games , author=. 2006 , publisher=

    2006

  70. [70]

    Proceedings of the eleventh annual conference on Computational learning theory , pages=

    Tracking the best regressor , author=. Proceedings of the eleventh annual conference on Computational learning theory , pages=

  71. [71]

    Nature Machine Intelligence , volume=

    Three types of incremental learning , author=. Nature Machine Intelligence , volume=. 2022 , publisher=. doi:10.1038/s42256-022-00568-3 , url=

    doi:10.1038/s42256-022-00568-3 2022

  72. [72]

    Is Multi-Task Learning an Upper Bound for Continual Learning? , year=

    Wu, Zihao and Tran, Huy and Pirsiavash, Hamed and Kolouri, Soheil , booktitle=. Is Multi-Task Learning an Upper Bound for Continual Learning? , year=

  73. [73]

    Proceedings of the 36th International Conference on Machine Learning , pages=

    Is Multi-Task Learning an Upper Bound for Continual Learning? , author=. Proceedings of the 36th International Conference on Machine Learning , pages=. 2019 , organization=

    2019

  74. [74]

    Trends in Cognitive Sciences , volume=

    Embracing Change: Continual Learning in Deep Neural Networks , author=. Trends in Cognitive Sciences , volume=. 2020 , publisher=. doi:10.1016/j.tics.2020.09.004 , url=

    doi:10.1016/j.tics.2020.09.004 2020

  75. [75]

    arXiv preprint arXiv:1707.06347 , year=

    Proximal Policy Optimization Algorithms , author=. arXiv preprint arXiv:1707.06347 , year=

    Pith/arXiv arXiv

  76. [76]

    Proceedings of the Conference on Robot Learning (CoRL) , pages=

    Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning , author=. Proceedings of the Conference on Robot Learning (CoRL) , pages=. 2020 , volume=

    2020

  77. [77]

    2018 , eprint=

    Online Learning: A Comprehensive Survey , author=. 2018 , eprint=

    2018

  78. [78]

    2013 AAAI spring symposium series , year=

    Lifelong machine learning systems: Beyond learning algorithms , author=. 2013 AAAI spring symposium series , year=

    2013

  79. [79]

    The Psychology of Learning and Motivation , volume=

    Catastrophic interference in connectionist networks: the sequential learning problem , author=. The Psychology of Learning and Motivation , volume=. 1989 , publisher=

    1989

  80. [80]

    Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1 (NeurIPS Datasets and Benchmarks 2021) , year=

    The CLEAR Benchmark: Continual LEArning on Real-World Imagery , author=. Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1 (NeurIPS Datasets and Benchmarks 2021) , year=

    2021

Showing first 80 references.

This paper was first reviewed by grok-4.5 on July 11, 2026.