Pith. sign in

REVIEW 5 major objections 5 minor 16 references

Unlocking the Power of Rehearsal in Continual Learning: A Theoretical Perspective

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that reviewing old tasks one by one after learning a new task causes less forgetting and lower generalization error than mixing old and new data together, whenever the tasks are sufficiently different.

desk verdict Clean T=2 theory for sequential vs concurrent rehearsal; the general-T proof quietly assumes fresh memory draws, which real replay and the paper's own DNN runs violate. read the letter →

arxiv 2506.00205 v1 pith:DKA7ZKSP submitted 2025-05-30 cs.LG

Junze Deng , Qinhang Wu , Peizhong Ju , Sen Lin , Yingbin Liang , Ness Shroff This is my paper · ORCID
classification cs.LG MSC 68T0562J05
keywords continuallearningcatastrophicforgettingrehearsalsequentialconcurrenthybridoverparameterizedlinearregressiongeneralizationerror
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether the standard rehearsal recipe for continual learning—train on new data and stored old data at the same time—is always the best way to fight catastrophic forgetting. The authors propose a sequential alternative: learn the new task first, then revisit each stored old task one at a time, the way a student reviews past topics before an exam. Working in an overparameterized linear regression model, they derive closed-form expressions for expected forgetting and generalization error under both strategies and prove that sequential rehearsal has smaller expected error than concurrent rehearsal whenever the tasks are sufficiently dissimilar, with the advantage growing as tasks diverge. They use this insight to design a hybrid rehearsal framework that mixes similar tasks concurrently and revisits dissimilar tasks sequentially, and they report that the hybrid method improves accuracy and reduces forgetting on image-classification benchmarks with deep networks. The paper claims to provide the first theoretical analysis of rehearsal-based continual learning.

What carries the argument

The engine of the argument is a closed-form recurrence for the expected squared distance from the trained weights to each task's ground truth, $E[\lVert w_t - w_i^*\rVert^2]$, obtained by treating SGD on squared loss as an $\ell_2$ projection onto the affine subspace of data-consistent weights. The recurrence is rolled out using projection lemmas: a random Gaussian feature matrix kills a fraction $m/p$ of a weight vector's norm, and the noise term follows an inverse-Wishart expectation. Task interference is encoded by coefficients multiplying squared distances between ground-truth vectors $\lVert w_j^* - w_k^*\rVert^2$, so the comparison of concurrent versus sequential rehearsal becomes a comparison of these coefficient families. The hybrid framework generalizes the same structure by splitting the memory buffer into similar and dissimilar task sets and applying concurrent training to one and sequential revisits to the other.

What would settle it

Run the two-task linear experiment with a persistent reservoir buffer whose stored samples are reused every time a task is revisited, and measure forgetting as the task gap grows; if the crossover predicted for the fresh-memory setting disappears or reverses, then the independence structure is driving the result rather than a general property of sequential replay.

Watch

Extended reading notes

Core claim

The paper's central claim is that how memory data are mixed with new data changes the interference structure of continual learning, and the usual choice—concurrent mixing—is not universally optimal. For an overparameterized linear model with i.i.d. Gaussian features and equal memory allocation, expected forgetting and generalization error are shown to decompose into three parts: a coefficient times each ground-truth norm, a sum over task-pair distance terms $\lVert w^*_j - w^*_k\rVert^2$ capturing interference from task dissimilarity, and a noise term. Comparing coefficients yields a threshold condition for $T=2$: concurrent rehearsal is better for similar tasks, but once $\lVert w^*_1-w^*_2\rVert^2$ crosses a threshold (with noise included in the criterion), sequential rehearsal wins on both metrics. For general $T$, Theorem 5.5 proves simultaneous gains in both forgetting and generalization when the ground-truth vectors are orthonormal, memory size $M\ge 2$, and dimension $p$ is large. The same coefficient comparison motivates the hybrid framework, whose linear-model error obeys the same structural theorem, and the authors argue this explains the hybrid advantage observed on deep networks.

Load-bearing premise

The entire theoretical comparison assumes the memory samples used at each task are fresh—never seen in earlier training—whereas real rehearsal buffers replay the same stored samples many times.

Editorial extensions

If this is right

  • When tasks are sufficiently different, sequential rehearsal has strictly lower expected forgetting and generalization error than concurrent rehearsal in the linear setting.
  • The advantage of sequential rehearsal grows as the gap between ground-truth parameters increases, so the choice between the two strategies should depend on task similarity.
  • For very similar tasks concurrent rehearsal remains preferable, so an adaptive hybrid that mixes similar tasks and revisits dissimilar ones inherits the better regime.
  • The hybrid rehearsal framework translates the theory into a practical algorithm and, on Split-CIFAR-10/100 and Split-TinyImagenet200, improves final accuracy and reduces final forgetting relative to concurrent rehearsal; the gap widens under corruption that increases task dissimilarity.
  • In the linear setting the derived closed forms reproduce simulation results exactly, confirming that the coefficient recurrences capture the expected performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the fresh-memory assumption is the true driver, then persistent replay buffers that resample the same stored examples should weaken or reverse the sequential advantage; this is testable by rerunning the linear-model simulation with fixed stored samples.
  • The oldest-to-newest rehearsal order is assumed, and the authors note it may be suboptimal; an ordering that schedules the most interfering tasks last could amplify the gains.
  • The gradient-cosine similarity division in Algorithm 1 is heuristic; a theoretically grounded similarity measure based on the distance terms $\lVert w_j^* - w_k^*\rVert^2$ could make the hybrid's threshold consistent with Theorem 5.3.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper studies rehearsal-based continual learning in an overparameterized linear regression model, comparing concurrent rehearsal (training current and memory data jointly) with a proposed sequential rehearsal (training the current task first, then revisiting old tasks one by one). The main contributions are closed-form expressions for expected forgetting and generalization error for both strategies (Theorem 5.1), a proof for T=2 that sequential rehearsal is better when tasks are dissimilar (Theorem 5.3), a general-T result under orthonormal ground-truth vectors and large p (Theorem 5.5), and a hybrid rehearsal algorithm that trains similar tasks concurrently and dissimilar tasks sequentially, validated on DNN benchmarks (Section 6). The T=2 analysis is clean and the linear-model simulations match the closed-form predictions exactly.

Significance. If the central claims hold, this would be a valuable theoretical contribution to rehearsal-based continual learning, providing the first explicit comparison of concurrent and sequential rehearsal strategies and a principled justification for adaptive rehearsal. The paper's strength is its explicit, parameter-free closed-form analysis of the T=2 case, which is rigorously derived and verified by simulation. The general-T theorem and the hybrid framework are ambitious, but several load-bearing issues in their statements and proofs currently limit their validity. The fresh-memory assumption separates the theory from the experimental protocol, and the noise omission in Theorem 5.5 is a concrete gap. With a focused revision that corrects the theorem conditions and honestly scopes the claims, this work could be a solid contribution.

major comments (5)
  1. [§5.2 / Appendix G, Theorem 5.5] Theorem 5.5 is stated under the general problem setup, which allows noise σ ≥ 0, but the proof in Appendix G drops all noise terms. In particular, Equations (66) and (67) compare [Li(wT) - Li(wi)] without the noise_T(σ) - noise_i(σ) contributions, and the subsequent inequalities do not bound these omitted terms. As written, the theorem is not proven for σ > 0; either the theorem must assume σ = 0, or the proof must incorporate a uniform bound on the noise differences for both rehearsal methods.
  2. [§5.2, Theorem 5.5 vs. its proof] The theorem states the condition "p = O(T^4 n^2 M^2)", which is an upper bound on p and is inconsistent with both the overparameterization requirement p > n+M and the proof, which uses lower bounds such as p > 2T^2(n+M)nM for the forgetting part and p > 2T^4(n+M+1)^2M for the generalization part. The condition should be a lower bound, e.g., p ≥ C T^4 n^2 M^2 for a universal constant C, or "p sufficiently large".
  3. [§3 / §6.2 / Appendix A.2] The theory assumes that "the memory data are all fresh and have not been used in previous training," with independent memory draws at every task. This independence is used in the distributional lemmas (B.2–B.10) and in the recurrences of Sections C.1 and C.2. The DNN experiments, by contrast, use reservoir sampling with a persistent buffer (Appendix A.2), so the same samples are replayed across tasks. Consequently, Theorem 5.5 and the hybrid characterization in Section H do not apply to the experimental protocol that is used to validate the framework. The T=2 comparison is unaffected because the memory is used only once, but the general-T and hybrid claims, including the abstract's sweeping statement that sequential rehearsal is better for dissimilar tasks, need either a persistent-buffer analysis or a clear scoping of the theory to fresh-memory (or generative replay) settings.
  4. [§5.2, Lemma 5.4 and Appendix F] Lemma 5.4 states the coefficient inequalities without any regularity assumptions beyond the general setup, but its proof in Section F requires large-p conditions, e.g., "if p > 2T^4(n+M)nM" for d_{i1iT} (Section F.1) and "p > (T^4+1)(n+M)M" for the c-coefficients (Section F.2). Since these thresholds are not stated in the lemma, the lemma is not established for moderate p. The lemma and Theorem 5.5 should explicitly include the required lower bounds on p.
  5. [§5.2, paragraph after Theorem 5.5] The paper extrapolates from the orthonormal extreme to all "highly different" tasks by arguing continuity and stating "we expect" sequential rehearsal to remain advantageous. This is explicitly a heuristic rather than a proof, yet the abstract claims "we show that sequential rehearsal performs better when tasks are less similar" without this caveat. The rigorous contribution is limited to T=2 (Theorem 5.3) and the orthonormal case for T≥2 (Theorem 5.5); the intermediate dissimilarity regime is not proven. The abstract and Section 5.2 should be reworded to match the actual scope of the theorems.
minor comments (5)
  1. [Appendix F, heading] The section heading "Proof of Theorem 5.4" should read "Proof of Lemma 5.4" to be consistent with the main-text numbering.
  2. [§6.2, Tables 2.1 and 2.2] Tables 2.1 and 2.2 appear to be sub-tables of Table 2, but their captions are malformed ("Table 2.1Acc(↑)" and "Table 2.2Fgt(↓)"); they should be merged into Table 2 or given proper separate captions.
  3. [§3, footnote on M/(t-1) integrality] The footnote says non-integer allocations cause only a minor error, but the magnitude of this error is not quantified; a brief bound or reference would be helpful.
  4. [§5.2, text after Lemma 5.2] The phrase "Then Theorem 5.2 suggests" should refer to Theorem 5.3, which is the theorem that formalizes the T=2 comparison.
  5. [Appendix B] The supporting lemmas are labeled B.1–B.18, but several cross-references in the proof call them "Theorem B.12" or "Theorem B.13"; the labels should be made uniform as "Lemma".

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the rehearsal comparisons are derived from explicit closed forms under stated Gaussian/overparameterized assumptions, and the self-cited lemmas are elementary facts that do not presuppose the target theorem.

full rationale

The paper's central claim is a comparison of two training recursions. Each method's expected forgetting and generalization error is obtained by iterating the min-norm SGD update (Equation (26), Sections C.1 and C.2) and by taking Gaussian expectations via the projection lemmas B.2, B.3, B.5-B.10. The coefficients in Theorem 5.1 and Proposition C.2 are explicit functions of p, n, M and the Gram-matrix statistics; no constant is fitted to the quantities being predicted. Theorem 5.3 and Theorem 5.5 then compare these closed forms. Lemma 5.4 and the proof of Theorem 5.5 use only the coefficient-wise inequalities derived in Section F, together with the orthonormal-task case ||w_i*||=1 and ||w_i*-w_j*||=2; this is a concrete instance, not an assumption equivalent to the conclusion. The citations to Lin et al. (2023) and Ju et al. (2023) are used for standard facts (the min-norm solution of an underdetermined linear system and E||P_X v||^2 = (m/p)||v||^2); these facts are stated with assumptions that do not include the rehearsal comparison, so they are independent support rather than load-bearing self-citation. The fresh-memory assumption in Section 3 is a modeling gap with respect to persistent replay buffers (and the DNN experiments), but it is an explicit stated assumption, not a circular one: the theory is derived for fresh memory draws, and the experiments are a separate empirical validation. No step was found where a prediction reduces to a fitted parameter or to a self-citation chain by construction.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The theoretical core is parameter-free in that no constants are fitted to data, but it rests on restrictive modeling assumptions: linear ground truth, Gaussian features, min-norm SGD solutions, fresh memory samples, and equal memory allocation. The algorithmic contribution introduces a per-dataset similarity threshold that is hand-set. No new physical or model entities are postulated.

free parameters (1)
  • similarity threshold tau = 0 for Split-CIFAR-10, Split-TinyImageNet200, and corrupted variants; -0.1 for Split-CIFAR-100 and Corrupted…
    The threshold controls which memory tasks are treated as dissimilar in the hybrid algorithm. It is chosen per dataset by hand, with no sensitivity analysis, and the reported gains depend on this choice.
assumptions (5)
  • domain assumption Data are generated by a linear ground truth y = x^T w* + z with i.i.d. Gaussian features and Gaussian noise (Equation 1).
    All closed-form error expressions are computed under this data model; the DNN experiments are a separate empirical extension.
  • standard math SGD on MSE in the overparameterized regime converges to the minimum l2-norm solution satisfying the interpolation constraints (Section 4).
    This is a known convergence property for linear least squares, used as the basis for the projection recurrences in the proofs.
  • ad hoc to paper Memory data are fresh and have not been used in previous training, with equal allocation M/(t-1) per old task (Section 3).
    This independence assumption makes the recurrences tractable but does not match persistent rehearsal buffers, which replay the same stored samples repeatedly.
  • ad hoc to paper Theorem 5.5 assumes orthonormal ground-truth vectors, and its proof in Appendix G drops all noise terms even though the theorem statement does not state sigma = 0.
    The general-T superiority result is proven only in this extreme dissimilarity case, with an implicit zero-noise condition, so the broad 'dissimilar tasks' claim is an extrapolation.
  • domain assumption Task similarity is measured by the squared distance between ground-truth parameter vectors, while the practical algorithm uses gradient cosine similarity.
    The theory characterizes similarity through ||w_i* - w_j*||^2; the connection to gradient-based similarity in the hybrid algorithm is heuristic rather than proven.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unlocking the Power of Rehearsal in Continual Learning: A Theoretical Perspective." pith.science (2026). https://pith.science/paper/DKA7ZKSP

@misc{pith2026250600205,
  author       = {Pith},
  title        = {Pith review of: Unlocking the Power of Rehearsal in Continual Learning: A Theoretical Perspective},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DKA7ZKSP}},
  note         = {Machine review of arXiv:2506.00205}
}
read the original abstract

Rehearsal-based methods have shown superior performance in addressing catastrophic forgetting in continual learning (CL) by storing and training on a subset of past data alongside new data in current task. While such a concurrent rehearsal strategy is widely used, it remains unclear if this approach is always optimal. Inspired by human learning, where sequentially revisiting tasks helps mitigate forgetting, we explore whether sequential rehearsal can offer greater benefits for CL compared to standard concurrent rehearsal. To address this question, we conduct a theoretical analysis of rehearsal-based CL in overparameterized linear models, comparing two strategies: 1) Concurrent Rehearsal, where past and new data are trained together, and 2) Sequential Rehearsal, where new data is trained first, followed by revisiting past data sequentially. By explicitly characterizing forgetting and generalization error, we show that sequential rehearsal performs better when tasks are less similar. These insights further motivate a novel Hybrid Rehearsal method, which trains similar tasks concurrently and revisits dissimilar tasks sequentially. We characterize its forgetting and generalization performance, and our experiments with deep neural networks further confirm that the hybrid approach outperforms standard concurrent rehearsal. This work provides the first comprehensive theoretical analysis of rehearsal-based CL.

Figures

Figures reproduced from arXiv: 2506.00205 by the authors.

Figure 1
Figure 1. An illustration of concurrent and sequential rehearsal. 5. Main Results The main theoretical results consist of two parts. First, we derive closed forms of forgetting and generalization error for both concurrent and sequential rehearsal methods. Second, we compare the performance of these two rehearsal-based schemes, concluding that sequential rehearsal outperforms concurrent rehearsal when tasks are dissimilar. 5.1… view at source ↗
Figure 2
Figure 2. Forgetting and Generalization Error vs Task Gap Several important insights can be immediately obtained from [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Evolution of Accuracy Acc for Concurrent rehearsal and Hybrid rehearsal across three datasets and their corrupted variants, corresponding to the results reported in [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Sample images for demonstrating the employed corruption schemes. 50 [PITH_FULL_IMAGE:figures/full_fig_p050_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 16 canonical work pages

  1. [1]

    ˆd(concurrent) 1 < ˆd(sequential) 1 : ˆd(sequential) 1 = 1− n p 3 1− M p 1− M 2p 2 36 Unlocking the Power of Rehearsal in Continual Learning: A Theoretical Perspective > 1− n p 3 1− M p 2 > 1− n+M p 2 1− M p = ˆd(concurrent) 1

    work page

  2. [2]

    ˆd(concurrent) 3 > ˆd(sequential) 3 ; 4

    ˆd(concurrent) 2 > ˆd(sequential) 2 when p > 4n+3M 2 ; 3. ˆd(concurrent) 3 > ˆd(sequential) 3 ; 4. ˆd(concurrent) 4 > ˆd(sequential) 4 . The proof of these relationships is provided as follows

    work page

  3. [3]

    We first lower bound ˆd(concurrent) 3 as follows

    ˆd(concurrent) 3 > ˆd(sequential) 3 . We first lower bound ˆd(concurrent) 3 as follows. ˆd(concurrent) 3 = n p 2− 2(n+M) p + (n+M) 2 p2 + M p 1− n+M p + M 2p + 3nM 2p(p−n−M−1) 37 Unlocking the Power of Rehearsal in Continual Learning: A Theoretical Perspective > n p 2− 2(n+M) p + (n+M) 2 p2 + M p 1− n+M p + M 2p + 3nM 2p2 . On the other hand, we upper bou...

    work page

  4. [4]

    We first lower bound ˆd(concurrent) 2 as follows

    ˆd(concurrent) 2 > ˆd(sequential) 2 whenp > 4n+3M 2 . We first lower bound ˆd(concurrent) 2 as follows. ˆd(concurrent) 2 = 3− 3(n+M) p nM p(p−n−M−1) + 3M 2 4p(p−n−M−1) + n+M p 2− 3n p − M p + n(n+M) p2 > 3− 3(n+M) p nM p2 + 3M 2 4p2 + n+M p 2− 3n p − M p + n(n+M) p2 >3 1− n+M p nM p2 + 2(n+M) p + n+M p − 3n p − n p + n(n+M) p2 = 2(n+M) p − 3n2 +nM+M 2 p2 ...

    work page

  5. [5]

    The proof ofˆc(concurrent) 5 >ˆc(sequential) 5 is the same asˆc(concurrent) 4 >ˆc(sequential) 4 . E.2. Comparison of Generalization Error WhenT= 3 By observing Equation (41) and Equation (43), the expressions of generalization error for both rehearsal methods share the same structure: G3 = 1 3 ˆd1(∥w∗ 1∥2 +∥w ∗ 2∥2 +∥w ∗ 3∥2) + 1 3 ˆd2 ∥w∗ 1 −w ∗ 2∥2 + 1 ...

    work page

  6. [6]

    ˆd(concurrent) 4 > ˆd(sequential) 4 . Consider: ˆd(sequential) 4 = 1− M 2p 2 n p 1− M p 1− n p + 1 + M 2p < n p 1− M p 1− n p + 1 + M 2p = n p 2− n+M p + M 2p + n2M p3 < n p 2− n+M p + M 2p + 3nM 2p(p−n−M−1) < ˆd(concurrent) 4 . F. Proof of Theorem 5.4 In this section, we prove Theorem 5.4, which helps to further compare the performance between concurrent...

    work page

  7. [7]

    We first proved (concurrent) 0T < d(sequential) 0T . According to Theorem B.11, we have: d(concurrent) 0T = 1− n p 1− n+M p T−1 < 1− n p T−2Y l=0 " 1− M (T−l−1)p T−l−1 1− n p # =d (sequential) 0T 38 Unlocking the Power of Rehearsal in Continual Learning: A Theoretical Perspective

    work page

  8. [8]

    1− M (T−k−1)p T−k−1 1− n p # 1− M (T−l−1)p T−l−2 M (T−l−1)p = T−3X l=0 l−1Y k=0

    Now, we proved (concurrent) i1iT > d(sequential) i1iT ifp >2T 4(n+M)nM. We first consider: n p T−2Y l=0 " 1− M (T−l−1)p T−l−1 1− n p # (i) < n p 1− n+M p + (n+M)M p2 T−1 (ii) < n p 1− n+M p T−1 + T 2(n+M)nM p3 ,(44) where(i)follows from Theorem B.12 and(ii)follows from Theorem B.13. We also notice that: T−2X l=0 l−1Y k=0 " 1− M (T−k−1)p T−k−1 1− n p # 1− ...

    work page

Show all 16 references
  1. [9]

    1− M (T−k−1)p T−k−1 1− n p # 1− M (T−l−1)p T−j−l−1 M (T−l−1)p = T−j−2X l=0 l−1Y k=0

    Next, we proved (concurrent) ijiT > d(sequential) ijiT ifp > T4(n+M)M, forj= 2,3, ..., T−1. We first have: T−j−1X l=0 l−1Y k=0 " 1− M (T−k−1)p T−k−1 1− n p # 1− M (T−l−1)p T−j−l−1 M (T−l−1)p = T−j−2X l=0 l−1Y k=0 " 1− M (T−k−1)p T−k−1 1− n p # 1− M (T−l−1)p T−j−l−1 M (T−l−1)p ...

  2. [10]

    The proof is straightforward: d(sequential) iT iT = 1− M (T−1)p T−1 n p < n p ≤d (concurrent) iT iT

    Last, we proved (concurrent) iT iT > d(sequential) iT iT . The proof is straightforward: d(sequential) iT iT = 1− M (T−1)p T−1 n p < n p ≤d (concurrent) iT iT

  3. [11]

    Moreover, for the other choices ofj, kwe haved (concurrent) ijkT ≥0andd (sequential) ijkT = 0. F.2. Proof of Coefficientsc i, cijk in Theorem 5.4 In this subsection, we will compare the coefficients ci, cijk under different rehearsal methods. Before we start, we first provide ...

  4. [12]

    1− M (T−l−1)p T−l−1 1− n p # − i−2Y l=0

    We provec (concurrent) i1i > c(sequential) i1i ifp >3T 4(n+M)nM. We first upper bound part of the coefficientc (sequential) i1i : n p (T−2Y l=0 " 1− M (T−l−1)p T−l−1 1− n p # − i−2Y l=0 " 1− M (i−l−1)p i−l−1 1− n p #) (i) < n p " 1− n+M p T−1 − 1− n+M p i−1# + T 2(n+M)nM p3 (5...

  5. [13]

    1− M (T−k−1)p T−k−1 1− n p # 1− M (T−l−1)p T−j−l−1 M (T−l−1)p − i−j−1X l=0 l−1Y k=0

    Next, we prove c(concurrent) iji > c(sequential) iji if p >3T4(n+M)nM , for j= 2,3, ..., i−1. We observe that c(sequential) iji consists of two parts, where the first part can be upper bounded by T−j−1X l=0 l−1Y k=0 " 1− M (T−k−1)p T−k−1 1− n p # 1− M (T−l−1)p T−j−l−1 M (T−l−1...

  6. [14]

    We prove c(concurrent) iji > c(sequential) iji for j=i, i+ 1, ..., T−1 if p > T4(n+M)M . According to the same derivation as Equations (47) and (49), we have c(sequential) iji < T−j−1X l=0 1− n+M p l M (T−l−1)p 1− n+M p T−j n p − M T 2p2 + T 2(n+M)M 2 p3 < T−j−1X l=0 1− n+M p ...

  7. [15]

    Last, we provec (concurrent) iT i > c(sequential) iT i ifp > T2(n+M)M. Consider: c(sequential) iT i = 1− M (T−1)p T−1 n p < 1− M (T−1)p n p < n p − nM p2 (i) < n p 43 Unlocking the Power of Rehearsal in Continual Learning: A Theoretical Perspective (ii) ≤c (concurrent) iT i ,(...

  8. [16]

    1− n+M p T−1 − 1− n+M p i−1# + 2 TX j=1 c(sequential) j + 2β+ 1 + 2β+ 2 ≥ 1− n p

    As discussed in Equations (51) to (55), we have G. Proof of Theorem 5.5 In this section, we prove Theorem 5.5 where we provide a particular example in which sequential rehearsal has lower forgetting and generalization than concurrent rehearsal. We first prove the forgetting pa...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.