Pith. sign in

REVIEW 3 major objections 6 minor 57 references

Enhancing Online Continual Learning with Plug-and-Play State Space Model and Class-Conditional Mixture of Discretization

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that online continual learning suffers from limited adaptability, not just forgetting, and proposes a plug-and-play selective-state-space branch with class-conditional mixture of discretization that it claims improves…

desk verdict The plug-and-play SSM branch helps, but the paper's own ablation attributes the gain to the distillation branch, not the novel routing and contrastive loss. read the letter →

arxiv 2412.18177 v1 pith:IJB236YL submitted 2024-12-24 cs.LG cs.AIcs.CV

Sihao Liu , Yibo Yang , Xiaojie Li , David A. Clifton , Bernard Ghanem This is my paper · ORCID
classification cs.LGcs.AIcs.CV
keywords onlinecontinuallearningselectivestatespacemodelmixtureofdiscretizationclass-conditionalroutingcatastrophicforgettingreplaybuffercontrastivelossMamba
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Online continual learning systems see every training example once and must keep old knowledge while absorbing new tasks. Most methods rely on replay and focus on remembering, rather than on making the model adapt quickly to the stream. This paper argues that adaptability is the missing ingredient and proposes S6MOD, a plug-and-play branch that can be attached after the backbone of an existing method. The branch runs features through a selective state space model whose discretization step is a mixture of candidate patterns, with the number of patterns chosen per class by a routing rule based on prototype margins, trained with a contrastive loss. The paper reports that S6MOD improves average accuracy in every tested configuration and lowers average forgetting in most, across seven replay baselines on CIFAR-10, CIFAR-100, and Tiny-ImageNet.

What carries the argument

The load-bearing object is S6MOD, an extra branch appended after the backbone and supervised by a fixed equiangular tight frame (ETF) classifier. Inside the branch, a selective state space model with four-direction scans produces $\hat{X} = \mathrm{SiLU}(\mathrm{Conv}(X))$, and a sparse mixture of discretization forms $\Delta = \sum_{i \in \Omega(\hat{X})} w_i \Delta_i$ from the top-$N_k$ of $N$ linear projection experts. The class-conditional router sets $N_k$ from prototype-margin uncertainty (Eqs. 9 and 10), the contrastive discretization loss (Eq. 11) enforces within-class consistency and between-class diversity of $\Delta$, and a KL term transfers the branch predictions back to the base method head. This mechanism carries the claimed adaptability because $\Delta$ controls the decay rate of the state update, so the selected patterns determine how much each class's features are allowed to shift.

What would settle it

On CIFAR-100, compare the reported routing against an oracle that sets $N_k$ from true per-class test error, and against prototypes frozen after the first task. If oracle routing does not beat prototype routing, or if frozen prototypes match flexible ones, the prototype-margin uncertainty is not carrying the reported gains. A simpler check: record $\sigma_k$ during the last task and compare it with actual per-class accuracy, since the claim predicts that high-$\sigma_k$ classes are the misclassified ones.

Watch

Extended reading notes

Core claim

The central claim is that the bottleneck in online continual learning is model adaptability rather than memory retention alone, and that a selectively discretized state space branch restores it. S6MOD computes a per-input discretization $\Delta$ as a weighted sum of the top-$N_k$ of $N$ learned discretization projections, with $N_k = \lceil N \sigma_k \rceil$ set by the class-conditional routing rule $\sigma_k = \mathrm{Avg}_c\{\exp(-\lambda_0 \|M_k - M_c\|_2)\}$ over moving-average class prototypes. Classes whose prototypes sit close to others receive more discretization patterns, allocating extra capacity to uncertain classes, while confident classes keep fewer patterns to protect old knowledge. A contrastive discretization loss pulls the $\Delta$ values of same-class samples together and pushes different-class samples apart, and an equiangular tight frame classifier with a KL-divergence term supervises the branch and the base head. Across seven replay baselines, the paper reports higher average accuracy and, in most settings, lower average forgetting, with the largest gains on Tiny-ImageNet and on the OnPro and MOE-MOSE combinations.

Load-bearing premise

The routing's uncertainty estimate assumes that pairwise distances between moving-average class prototypes are trustworthy even though every image is seen once and old classes reappear only through a small replay buffer; if those prototypes are noisy, the number of discretization patterns $N_k$ is misallocated and the claimed stability-plasticity balance does not hold.

Editorial extensions

If this is right

  • Adding S6MOD to a replay-based baseline improves average accuracy in every dataset and memory-buffer configuration reported in Table 1, with gains around one percentage point on CIFAR-10 and CIFAR-100 and larger gains on several Tiny-ImageNet settings.
  • The routing gives new-task learning comparable to activating all $N$ patterns while keeping forgetting close to the single-pattern setting, because uncertain classes receive more discretization capacity and confident classes fewer.
  • S6MOD remains effective on distillation-based baselines and with different classifiers: the branch features transfer to NCM classification as well as logit classifiers, and gains persist on MOSE and MOE-MOSE.
  • Feature embeddings of buffered data become more compact within classes and more separated across classes after adding the branch, as shown by t-SNE visualizations.
  • The total pattern count $N$ controls a plasticity-forgetting trade-off, and class-conditional routing shifts that trade-off toward lower forgetting for a given $N$.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • As an extension the paper does not test, the same branch could be attached to non-replay online continual learning methods or to task-agnostic streams, since it modifies feature adaptation rather than buffer management.
  • The prototype-margin uncertainty is one possible routing signal; replacing $\sigma_k$ with prediction entropy or gradient-based uncertainty and checking whether results move would show whether the specific margin statistic is essential.
  • Because each discretization expert is a linear projection, scaling $N$ is cheap, so a natural extension is per-layer or per-scan routing instead of a single class-level $N_k$.
  • An ablation that isolates the dynamic $\Delta$ from the auxiliary contrastive signal would clarify which component produces the reported gain.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper proposes S6MOD, a plug-and-play branch for online continual learning (OCL) methods. The branch consists of a selective state space model (S6) with a mixture of discretization, a class-conditional routing mechanism that selects the number of discretization patterns per class based on prototype-margin uncertainty, and a contrastive discretization loss. The branch is supervised by a fixed ETF classifier and combined with the base method through a KL-divergence distillation loss. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet with seven baselines (ER, OCM, OnPro, OCM-CCLDC, OnPro-CCLDC, MOSE, MOE-MOSE) report improved average accuracy and lower average forgetting in many settings, with particular gains for OnPro and OCM. The paper claims that the module 'can be integrated into most existing methods and directly improve adaptability' and that the routing and contrastive loss are responsible for the gains.

Significance. The manuscript addresses a relevant problem—improving the adaptability of online continual learners—and proposes a modular solution that is easy to graft onto existing methods. The mathematical formulation of the mixture of discretization is standard selective SSM machinery, and the pseudo-code in Algorithm 1 is a useful practical aid. The paper also reports a broad experimental comparison (3 datasets, 7 baselines, 3 buffer sizes). If the mechanistic claims were supported by solid statistical evidence, the class-conditional capacity allocation would be a noteworthy contribution. However, the current evidence base is not sufficient: most accuracy gains are within one standard deviation, the ablation shows the headline gains come from the auxiliary branch plus distillation rather than from the novel routing and contrastive loss, and key hyperparameters and update rules are unspecified. The paper's contribution is therefore promising but not yet established.

major comments (3)
  1. [Table 3 / Sec. 5.3] The ablation in Table 3 does not support the attribution of gains to the proposed mixture-of-discretization routing. The 'branch' (plain SS2D plus ETF-KL distillation) accounts for 3.28 of the 4.24-point improvement over OnPro (33.33→36.61 vs 33.33→37.57). Adding class-conditional routing alone gives +0.31, adding LCont alone gives +0.32, and adding both gives +0.96; the reported standard deviations are 0.72–1.04, so these marginal improvements are within noise. No paired significance tests are provided. The abstract credits 'the mixture of discretization' and 'class-conditional routing' for the performance gains; this claim is currently unsupported by the reported experiments.
  2. [Sec. 4.3, Eqs. (9)–(10)] The class-conditional routing rests on the uncertainty signal sigma_k, but the paper leaves its computation underspecified. The feature prototypes M_c are described as 'moving average' without giving the momentum or the update schedule. The temperature lambda_0 in Eq. (9) is a free hyperparameter and its value is not reported in Table 5 or anywhere else, although it directly controls N_k = ceil(N * sigma_k) and therefore the capacity allocated per class. Without an analysis of sensitivity to lambda_0 and a precise prototype update rule, the claim of 'dynamic, uncertainty-based adjustment' cannot be independently reproduced or tested.
  3. [Sec. 1 / Sec. 4.2] The motivating premise that 'a single discretization mechanism may fail' for OCL is asserted without direct evidence, and the paper's own ablation runs against it. Replacing the proposed S6MOD core with a plain SS2D branch yields most of the improvement (36.61 vs 37.57 final accuracy), so the benefit of the mixture of discretization over a single discretization pattern is not demonstrated. A direct comparison (single discretization vs. mixture, with identical routing and losses otherwise) is needed to establish the central mechanism.
minor comments (6)
  1. [Sec. 4.3] The text 'The institution is that classes' should read 'The intuition is that classes'.
  2. [Table 5 / Supp. A.1] Table 5 lists hyperparameters only for ER with S6MOD; the statement that 'same hyperparameter settings' were applied to all baselines needs a complete listing for every baseline in the supplementary material.
  3. [Sec. 5.2 / Table 2] The claim that 'our module is also highly effective in reducing model forgetting' is contradicted by several rows in Table 2 (e.g., MOSE on CIFAR-100 M=2000 and M=5000; MOE-MOSE on CIFAR-100 M=2000 and M=5000). The post-hoc explanation that higher forgetting is 'often due to the stronger learning capabilities' is not substantiated; please provide a quantitative analysis or soften the claim.
  4. [Fig. 2 / Fig. 5] The t-SNE visualizations are qualitative; adding quantitative metrics of cluster quality (e.g., silhouette score or nearest-class accuracy) would strengthen the claim of more discriminative features.
  5. [References / Sec. 5.1] The text says dataset settings are 'the same as those in CCLDC[45]'; reference [45] is OnPro, while CCLDC is reference [43]. Please fix the citation.
  6. [Eq. (11)] The contrastive discretization loss uses cosine similarity without a temperature or margin parameter; please clarify the design choice and its influence on the results, especially given the small batch size.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the S6MOD claim is supported by external benchmark comparisons, and the self-citation to Mamba-FSCIL is motivational rather than load-bearing.

full rationale

The paper's central claim is that S6MOD can be plugged into existing OCL methods and improve adaptability. That claim is tested directly against external baselines in Tables 1 and 2 on CIFAR-10, CIFAR-100, and Tiny-ImageNet, so it does not reduce to the paper's own definitions or fitted parameters. The class-conditional routing in Eqs. (9)-(10) does compute uncertainty from the model's own feature prototypes, but this is an explicitly stated feedback heuristic, not a case where a predicted quantity is defined in terms of the claimed output: no equation is shown to equal another by construction, and no fitted parameter is renamed as a prediction. The self-citation to Mamba-FSCIL [23] is used only to motivate the adaptability of selective SSMs; the actual evidence for S6MOD's effectiveness comes from the paper's own controlled experiments against seven baselines, and Mamba-FSCIL is an independently testable prior study. The ablation in Table 3 indicates that the branch plus LDiff accounts for most of the gain, with smaller marginal contributions from routing and LCont; this is an attribution and significance concern, not a circularity. Reproducibility gaps such as the unreported lambda_0 and unspecified prototype update rule are correctness risks, not circular steps. Overall, the derivation chain is self-contained and no circular reduction is exhibited.

Assumptions & free parameters 6 free parameters · 7 assumptions · 0 invented entities

The central claim rests on a small set of tuned hyperparameters (N, alpha, beta, unreported lambda_0, unspecified prototype update), a protocol adopted from prior benchmarks, and several domain assumptions that are asserted rather than demonstrated: that single discretization is inadequate for OCL, that prototype margins measure uncertainty, and that auxiliary losses do not interfere with the base method. No invented entities are postulated.

free parameters (6)
  • N (number of discretization patterns) = 8 (CIFAR-100) or 10 (CIFAR-10, Tiny-ImageNet), varying by buffer size
    Chosen per dataset and memory size in Supp. Table 5; the sensitivity analysis in Fig. 4 shows N trades learning new tasks against forgetting, so its value affects the reported trade-off.
  • alpha (weight of L_Diff) = 1.0 for most settings, 0.5 for Tiny-ImageNet M=10k
    Reported in Supp. Table 5; for Tiny-ImageNet M=10k the paper reduces it to diminish the module's impact.
  • beta (weight of L_Cont) = 5.0 for most settings, 2.5 for Tiny-ImageNet M=10k
    Set in ratio 1:5 with alpha (Supp. A.1); sensitivity in Table 4 shows accuracy varies by about 1 point depending on the value.
  • lambda_0 (uncertainty temperature, Eq. 9)
    Never reported anywhere in the paper; N_k = ceil(N * sigma_k) depends directly on it, and no sensitivity analysis is given.
  • feature prototype update rule and momentum
    Sec. 4.3 says prototypes are maintained by moving average but gives no update rule or momentum value, so the routing signal is not fully specified.
  • branch architecture widths (MLPs f_x, f_z, f_B, f_C, expert layers)
    Dimensions of the branch MLPs and discretization expert layers are not reported, so the added parameter count and compute cost are unknown.
assumptions (7)
  • domain assumption OCL protocol: class-disjoint tasks, each sample seen once, replay buffers allowed
    Adopted from CCLDC/OnPro protocols in Sec. 5.1; the whole evaluation and the claim about adaptability depend on this specific protocol rather than general OCL.
  • ad hoc to paper A single discretization mechanism is inadequate for OCL dynamics
    Sec. 1 and 4.2 assert this with citations [1, 15] to multiscale dynamics literature, not to OCL evidence; the ablation in Table 3 attributes most of the gain to the branch plus distillation, not to the mixture of discretization.
  • domain assumption Prototype-margin uncertainty sigma_k tracks misclassification probability
    Sec. 4.3 states this as the motivation for routing; the only support is the indirect ablation in Table 3 and Fig. 3, and the signal is computed from the same feature space being trained.
  • standard math ZOH discretization of the SSM ODE (Eq. 1 to Eq. 2)
    Standard result from Gu et al. [14], used unmodified; the paper's contribution is the mixture over Delta, not the discretization formula.
  • standard math SS2D four-direction scanning serializes images adequately for the branch
    Borrowed from VMamba [26]; the branch inherits this assumption without validation in the OCL setting.
  • domain assumption KL distillation from branch to base head improves base features without destabilizing the base optimizer
    The combination L_base + L_DR + alpha L_Diff + beta L_Cont + L_z in Eqs. 12 and 13 assumes the auxiliary losses do not harm the base method's own objectives; no analysis of interference is provided.
  • ad hoc to paper Load-balancing loss L_z is needed and behaves as in ST-MoE
    L_z is cited to [57] and included in Eq. 13, but no analysis or ablation of its effect is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Online Continual Learning with Plug-and-Play State Space Model and Class-Conditional Mixture of Discretization." pith.science (2026). https://pith.science/paper/IJB236YL

@misc{pith2026241218177,
  author       = {Pith},
  title        = {Pith review of: Enhancing Online Continual Learning with Plug-and-Play State Space Model and Class-Conditional Mixture of Discretization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IJB236YL}},
  note         = {Machine review of arXiv:2412.18177}
}
read the original abstract

Online continual learning (OCL) seeks to learn new tasks from data streams that appear only once, while retaining knowledge of previously learned tasks. Most existing methods rely on replay, focusing on enhancing memory retention through regularization or distillation. However, they often overlook the adaptability of the model, limiting the ability to learn generalizable and discriminative features incrementally from online training data. To address this, we introduce a plug-and-play module, S6MOD, which can be integrated into most existing methods and directly improve adaptability. Specifically, S6MOD introduces an extra branch after the backbone, where a mixture of discretization selectively adjusts parameters in a selective state space model, enriching selective scan patterns such that the model can adaptively select the most sensitive discretization method for current dynamics. We further design a class-conditional routing algorithm for dynamic, uncertainty-based adjustment and implement a contrastive discretization loss to optimize it. Extensive experiments combining our module with various models demonstrate that S6MOD significantly enhances model adaptability, leading to substantial performance gains and achieving the state-of-the-art results.

Figures

Figures reproduced from arXiv: 2412.18177 by the authors.

Figure 1
Figure 1. Framework of S6MOD. Our method (a) introduces a plug-and-play branch after the backbone, where features are learned through [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. t-SNE visualization of memory data at the end of training [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Impact of dynamically selecting different [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Impact of pattern number N. The dataset used is CIFAR￾100 (M = 2k). Sensitivity Analysis of total discretization patterns num￾ber N. We conduct experiments on the total number of discretization patterns in the module. The results are shown in [PITH_FULL_IMAGE:figures/…
Figure 5
Figure 5. Figure 5: T-SNE visualization of features before classification of memory data at the end of training on CIFAR-100 ( [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 34 canonical work pages

  1. [1]

    The heterogeneous multiscale method

    Assyr Abdulle, E Weinan, Bj ¨orn Engquist, and Eric Vanden- Eijnden. The heterogeneous multiscale method. Acta Nu- merica, 21:1–87, 2012. 2

    work page 2012

  2. [2]

    Memory aware synapses: Learning what (not) to forget

    Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In ECCV, pages 139–154, 2018. 2

    work page 2018

  3. [3]

    Online continual learning with maximal interfered retrieval

    Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Lau- rent Charlin, Massimo Caccia, Min Lin, and Lucas Page- Caccia. Online continual learning with maximal interfered retrieval. NeurIPS, 32, 2019. 1

    work page 2019

  4. [4]

    Blackmamba: Mixture of experts for state- space models

    Quentin Anthony, Yury Tokpanov, Paolo Glorioso, and Beren Millidge. Blackmamba: Mixture of experts for state- space models. arXiv preprint arXiv:2402.01771, 2024. 3

    arXiv 2024

  5. [5]

    Dark experience for general continual learning: a strong, simple baseline

    Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. NeurIPS, 33: 15920–15930, 2020. 2

    work page 2020

  6. [6]

    Riemannian walk for incremen- tal learning: Understanding forgetting and intransigence

    Arslan Chaudhry, Puneet K Dokania, Thalaiyasingam Ajan- than, and Philip HS Torr. Riemannian walk for incremen- tal learning: Understanding forgetting and intransigence. In ECCV, pages 532–547, 2018. 1

    work page 2018

  7. [7]

    Efficient lifelong learning with a- gem

    Arslan Chaudhry, Marc’Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a- gem. arXiv preprint arXiv:1812.00420, 2018. 2

    arXiv 2018

  8. [8]

    A decoder-only foundation model for time-series forecast- ing

    Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. A decoder-only foundation model for time-series forecast- ing. arXiv preprint arXiv:2310.10688, 2023. 1

    arXiv 2023

Show all 57 references
  1. [9]

    A continual learning survey: Defying forgetting in classification tasks

    Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ale ˇs Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. PAMI, 44(7):3366–3385,

  2. [10]

    Learning factored representations in a deep mixture of ex- perts

    David Eigen, Marc’Aurelio Ranzato, and Ilya Sutskever. Learning factored representations in a deep mixture of ex- perts. arXiv preprint arXiv:1312.4314, 2013. 4

    2013 arXiv

  3. [11]

    Switch transformers: Scaling to trillion parameter models with sim- ple and efficient sparsity

    William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with sim- ple and efficient sparsity. Journal of Machine Learning Re- search, 23(120):1–39, 2022. 3

    2022

  4. [12]

    Enhancing knowledge transfer for task incremental learning with data-free subnetwork

    Qiang Gao, Xiaojun Shan, Yuchen Zhang, and Fan Zhou. Enhancing knowledge transfer for task incremental learning with data-free subnetwork. In Advances in Neural Informa- tion Processing Systems, pages 68471–68484. Curran Asso- ciates, Inc., 2023. 2

    2023

  5. [13]

    Mamba: Linear-time sequence modeling with selective state spaces

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023. 1, 2, 3, 5

    2023 arXiv

  6. [14]

    Efficiently modeling long sequences with structured state spaces

    Albert Gu, Karan Goel, and Christopher R ´e. Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.00396, 2021. 2, 3

    2021 arXiv

  7. [15]

    Nonlinear oscilla- tions, dynamical systems, and bifurcations of vector fields

    John Guckenheimer and Philip Holmes. Nonlinear oscilla- tions, dynamical systems, and bifurcations of vector fields . Springer Science & Business Media, 2013. 2

    2013

  8. [16]

    Survey on online streaming con- tinual learning

    Nuwan Gunasekara, Bernhard Pfahringer, Heitor Murilo Gomes, and Albert Bifet. Survey on online streaming con- tinual learning. In IJCAI, pages 6628–6637, 2023. 1, 2

    2023

  9. [17]

    Online contin- ual learning through mutual information maximization

    Yiduo Guo, Bing Liu, and Dongyan Zhao. Online contin- ual learning through mutual information maximization. In ICML, 2022. 1, 2, 6

    2022

  10. [18]

    Dealing with cross-task class discrimination in online continual learning

    Yiduo Guo, Bing Liu, and Dongyan Zhao. Dealing with cross-task class discrimination in online continual learning. In CVPR, 2023. 1, 2

    2023

  11. [19]

    Jacobs, Michael I

    Robert A. Jacobs, Michael I. Jordan, Steven J. Nowlan, and Geoffrey E. Hinton. Adaptive mixtures of local experts.Neu- ral Computation, 3(1):79–87, 1991. 4

    1991

  12. [20]

    Overcoming catastrophic forgetting in neu- ral networks

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barwinska, et al. Overcoming catastrophic forgetting in neu- ral networks. Proceedings of the national academy of sc...

    2017

  13. [21]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009. 5

    2009

  14. [22]

    Ya Le and Xuan S. Yang. Tiny imagenet visual recognition challenge. 2015. 5

    2015

  15. [23]

    Mamba-fscil: Dynamic adap- tation with selective state space model for few-shot class- incremental learning

    Xiaojie Li, Yibo Yang, Jianlong Wu, Bernard Ghanem, Liqiang Nie, and Min Zhang. Mamba-fscil: Dynamic adap- tation with selective state space model for few-shot class- incremental learning. arXiv preprint arXiv:2407.06136 ,

    arXiv

  16. [24]

    Learning without forgetting

    Zhizhong Li and Derek Hoiem. Learning without forgetting. PAMI, 40(12):2935–2947, 2017. 2

    2017

  17. [25]

    Jamba: A hybrid transformer-mamba language model

    Opher Lieber, Barak Lenz, Hofit Bata, Gal Cohen, Jhonathan Osin, Itay Dalmedigos, Erez Safahi, Shaked Meirom, Yonatan Belinkov, Shai Shalev-Shwartz, et al. Jamba: A hybrid transformer-mamba language model. arXiv preprint arXiv:2403.19887, 2024. 1, 3

    2024 arXiv

  18. [26]

    Vmamba: Visual state space model

    Yue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu, Lingxi Xie, Yaowei Wang, Qixiang Ye, and Yunfan Liu. Vmamba: Visual state space model. ArXiv, abs/2401.10166, 2024. 1, 2, 3, 7

    2024 arXiv

  19. [27]

    Gradient episodic memory for continual learning

    David Lopez-Paz and Marc’Aurelio Ranzato. Gradient episodic memory for continual learning. NeurIPS, 30, 2017. 2

    2017

  20. [28]

    Online continual learning in image classification: An empirical survey

    Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyun- woo Kim, and Scott Sanner. Online continual learning in image classification: An empirical survey. Neurocomputing, 469:28–51, 2022. 1, 2

    2022

  21. [29]

    Long range language modeling via gated state spaces

    Harsh Mehta, Ankit Gupta, Ashok Cutkosky, and Behnam Neyshabur. Long range language modeling via gated state spaces. arXiv preprint arXiv:2206.13947, 2022. 3 9

    2022 arXiv

  22. [30]

    Continual lifelong learning with neural networks: A review

    German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural networks, 113:54–71,

  23. [31]

    Moe- mamba: Efficient selective state space models with mixture of experts

    Maciej Pi ´oro, Kamil Ciebiera, Krystian Kr´ol, Jan Ludziejew- ski, Michał Krutul, Jakub Krajewski, Szymon Antoniak, Pi- otr Miło ´s, Marek Cygan, and Sebastian Jaszczur. Moe- mamba: Efficient selective state space models with mixture of experts. arXiv preprint arXiv:2401.0408...

    2024 arXiv

  24. [32]

    icarl: Incremental classi- fier and representation learning

    Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classi- fier and representation learning. InCVPR, pages 2001–2010,

    2001

  25. [33]

    Experience replay for continual learning

    David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lil- licrap, and Gregory Wayne. Experience replay for continual learning. NeurIPS, 32, 2019. 1, 2, 6

    2019

  26. [34]

    Vm-unet: Vision mamba unet for medical image segmentation

    Jiacheng Ruan and Suncheng Xiang. Vm-unet: Vision mamba unet for medical image segmentation. arXiv preprint arXiv:2402.02491, 2024. 1

    2024 arXiv

  27. [35]

    Progressive neural networks

    Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Raz- van Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.04671, 2016. 2

    2016 arXiv

  28. [36]

    Learning equi-angular representa- tions for online continual learning

    Minhyuk Seo, Hyunseo Koh, Wonje Jeung, Minjae Lee, San Kim, Hankook Lee, Sungjun Cho, Sungik Choi, Hyunwoo Kim, and Jonghyun Choi. Learning equi-angular representa- tions for online continual learning. In CVPR, pages 23933– 23942, 2024. 2

    2024

  29. [37]

    Overcoming catastrophic forgetting with hard attention to the task

    Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In ICML, pages 4548–4557. PMLR,

  30. [38]

    Outra- geously large neural networks: The sparsely-gated mixture- of-experts layer

    Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outra- geously large neural networks: The sparsely-gated mixture- of-experts layer. arXiv preprint arXiv:1701.06538, 2017. 4

    2017 arXiv

  31. [39]

    Visualizing data using t-sne

    Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9 (86):2579–2605, 2008. 7

    2008

  32. [40]

    Attention is all you need

    A Vaswani. Attention is all you need. NeurIPS, 2017. 2

    2017

  33. [41]

    Afec: Active forgetting of negative transfer in continual learning

    Liyuan Wang, Mingtian Zhang, Zhongfan Jia, Qian Li, Chenglong Bao, Kaisheng Ma, Jun Zhu, and Yi Zhong. Afec: Active forgetting of negative transfer in continual learning. NeurIPS, 34:22379–22391, 2021. 2

    2021

  34. [42]

    A comprehensive survey of continual learning: theory, method and application

    Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: theory, method and application. PAMI, 2024. 1, 2

    2024

  35. [43]

    Improving plasticity in online continual learning via collaborative learning

    Maorong Wang, Nicolas Michel, Ling Xiao, and Toshihiko Yamasaki. Improving plasticity in online continual learning via collaborative learning. In CVPR, pages 23460–23469,

  36. [44]

    Online prototype learning for online con- tinual learning

    Yujie Wei, Jiaxin Ye, Zhizhong Huang, Junping Zhang, and Hongming Shan. Online prototype learning for online con- tinual learning. In ICCV, pages 18764–18774, 2023. 1

    2023

  37. [45]

    Online prototype learning for online con- tinual learning

    Yujie Wei, Jiaxin Ye, Zhizhong Huang, Junping Zhang, and Hongming Shan. Online prototype learning for online con- tinual learning. In ICCV, 2023. 1, 2, 5, 6

    2023

  38. [46]

    Supermasks in superposition

    Mitchell Wortsman, Vivek Ramanujan, Rosanne Liu, Aniruddha Kembhavi, Mohammad Rastegari, Jason Yosin- ski, and Ali Farhadi. Supermasks in superposition. NeurIPS, 33:15173–15184, 2020. 2

    2020

  39. [47]

    Neural collapse inspired attraction–repulsion-balanced loss for im- balanced learning

    Liang Xie, Yibo Yang, Deng Cai, and Xiaofei He. Neural collapse inspired attraction–repulsion-balanced loss for im- balanced learning. Neurocomputing, 527:60–70, 2023. 2

    2023

  40. [48]

    Segmamba: Long-range sequential modeling mamba for 3d medical image segmentation

    Zhaohu Xing, Tian Ye, Yijun Yang, Guang Liu, and Lei Zhu. Segmamba: Long-range sequential modeling mamba for 3d medical image segmentation. InInternational Conference on Medical Image Computing and Computer-Assisted Interven- tion, pages 578–588. Springer, 2024. 1

    2024

  41. [49]

    Orchestrate latent expertise: Advancing online contin- ual learning with multi-level supervision and reverse self- distillation

    Hongwei Yan, Liyuan Wang, Kaisheng Ma, and Yi Zhong. Orchestrate latent expertise: Advancing online contin- ual learning with multi-level supervision and reverse self- distillation. In CVPR, pages 23670–23680, 2024. 1, 2, 6

    2024

  42. [50]

    Inducing neural collapse in imbalanced learning: Do we really need a learnable clas- sifier at the end of deep neural network? In NeurIPS, pages 37991–38002

    Yibo Yang, Shixiang Chen, Xiangtai Li, Liang Xie, Zhouchen Lin, and Dacheng Tao. Inducing neural collapse in imbalanced learning: Do we really need a learnable clas- sifier at the end of deep neural network? In NeurIPS, pages 37991–38002. Curran Associates, Inc., 2022. 2, 3, 5

    2022

  43. [51]

    Neural collapse inspired feature- classifier alignment for few-shot class incremental learning

    Yibo Yang, Haobo Yuan, Xiangtai Li, Zhouchen Lin, Philip Torr, and Dacheng Tao. Neural collapse inspired feature- classifier alignment for few-shot class incremental learning. arXiv preprint arXiv:2302.03004, 2023. 2, 3, 5

    2023 arXiv

  44. [52]

    Lifelong learning with dynamically expandable net- works

    Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable net- works. arXiv preprint arXiv:1708.01547, 2017. 2

    2017 arXiv

  45. [53]

    Wilson, and Paul D

    Seniha Esen Yuksel, Joseph N. Wilson, and Paul D. Gader. Twenty years of mixture of experts. IEEE Transactions on Neural Networks and Learning Systems , 23(8):1177–1193,

  46. [54]

    Contin- ual learning through synaptic intelligence

    Friedemann Zenke, Ben Poole, and Surya Ganguli. Contin- ual learning through synaptic intelligence. In ICML, pages 3987–3995. PMLR, 2017. 2

    2017

  47. [55]

    Understanding im- balanced semantic segmentation through neural collapse

    Zhisheng Zhong, Jiequan Cui, Yibo Yang, Xiaoyang Wu, Xi- aojuan Qi, Xiangyu Zhang, and Jiaya Jia. Understanding im- balanced semantic segmentation through neural collapse. In CVPR, pages 19550–19560, 2023. 2

    2023

  48. [56]

    Vision mamba: Efficient visual representation learning with bidirectional state space model

    Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model. arXiv preprint arXiv:2401.09417, 2024. 1, 2

    2024 arXiv

  49. [57]

    Analysis of Feature Embedding,

    Barret Zoph, Irwan Bello, Sameer Kumar, Nan Du, Yanping Huang, Jeff Dean, Noam Shazeer, and William Fedus. St- moe: Designing stable and transferable sparse expert mod- els. arXiv preprint arXiv:2202.08906, 2022. 5 10 Enhancing Online Continual Learning with Plug-and-Play Stat...

    2022 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.