Pith. sign in

REVIEW 3 major objections 4 minor 2 cited by

Mamba-CL: Optimizing Selective State Space Model in Null Space for Continual Learning

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Mamba-CL claims that four null-space orthogonality constraints on the SSM parameters let a Mamba model learn new tasks without changing its outputs on old tasks, and reports state-of-the-art accuracy on four class-incremental benchmarks.

desk verdict Strong empirical results on Mamba continual learning, but the theoretical guarantee rests on a false convolution assumption for selective SSMs. read the letter →

arxiv 2411.15469 v3 pith:KAG6T6HX submitted 2024-11-23 cs.CV

De Cheng , Yue Lu , Lingfeng He , Shizhou Zhang , Xi Yang , Nannan Wang , Xinbo Gao This is my paper · ORCID
classification cs.CV
keywords continuallearningcatastrophicforgettingstatespacemodelsMambanull-spaceprojectionorthogonalgradientclass-incrementalselectivespaces
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Mamba-CL, a continual-learning method for the Mamba selective state-space model. The central claim is that catastrophic forgetting can be prevented by updating only the SSM core under four sufficient orthogonality conditions: $\delta_t \Delta A = 0$, $X_t \Delta W_\delta = 0$, $\delta_t X_t \Delta W_B = 0$, and $X_t \Delta W_C = 0$. If these hold, the SSM output on any previous-task input $X_t$ is the same before and after learning a new task, so old knowledge stays intact. The method enforces the conditions with null-space projectors built from previous-task feature covariances, and reports accuracy improvements over prior continual-learning baselines on four class-incremental benchmarks. This matters because it extends the null-space anti-forgetting recipe from convolutional and transformer layers to the recurrent, non-linear discretization inside Mamba.

What carries the argument

The load-bearing machinery is the SSM's convolutional-kernel representation. Since $Y = X * K$, the consistency objective Eq. (7) is turned into a kernel equality $K_t = K_{t+1}$. The kernel factors are the discretized transition matrix $\bar{A} = \exp(\delta A)$, the input projection $\bar{B}$ derived through zero-order hold, and the output projection $C$; the paper argues that enforcing $\bar{A}_t^k = \bar{A}_{t+1}^k$, $\bar{B}_t = \bar{B}_{t+1}$, and $C_t = C_{t+1}$ element-wise is sufficient for kernel consistency. The reduction to Eq. (25) uses monotonicity of the element-wise exponential and of the softplus discretization step to convert these equalities into orthogonality conditions on $\Delta A$, $\Delta W_B$, $\Delta W_C$, and $\Delta W_\delta$. These conditions are implemented by SVD-based projectors $H_1, H_2, H_3$ onto approximate null spaces of the previous-task feature covariance matrices.

What would settle it

Choose random $\delta_1, A_1, \delta_2, A_2$ and test whether $[\exp(\delta_1 A_1)]^k = [\exp(\delta_2 A_2)]^k$ for $k=1,2,3$ implies $\delta_1 A_1 = \delta_2 A_2$; any counterexample invalidates the derivation step in Eqs. (12)-(13). A direct check is to take a trained Mamba-CL model, verify that the four conditions in Eq. (25) hold exactly for a stored $X_t$, and measure whether $\mathrm{SSM}(X_t \mid \theta_t) - \mathrm{SSM}(X_t \mid \theta_{t+1})$ is zero to numerical precision.

Watch

Extended reading notes

Core claim

Mamba-CL's central claim is that output consistency for an SSM module reduces to four independent parameter constraints. Because the forward pass is a global convolution $Y = X * K$ with kernel $K = [CB, CAB, CA^2B, \ldots]$, keeping old outputs unchanged is equivalent to keeping the kernel unchanged. The paper decomposes kernel consistency into consistency of $\bar{A}^k$, $\bar{B}$, and $\bar{C}$, and, using the zero-order hold discretization $\bar{A} = \exp(\delta A)$ and $\bar{B} = (\delta A)^{-1}(\exp(\delta A - I))\delta B$, derives the four sufficient conditions in Eq. (25). It then constructs three null-space projectors from uncentered covariance matrices of $X_t$, $\delta_t$, and $\delta_t X_t$, so that the projected gradients satisfy the conditions. On 10- and 20-split ImageNet-R, 10-split CIFAR-100, and 10-split DomainNet, plus 50- and 100-task versions, Mamba-CL reports higher final accuracy and lower forgetting than sequential fine-tuning and than prior prompt- and adapter-based methods.

Load-bearing premise

The whole guarantee rests on assuming that equal powers of the discretized transition matrices force the products $\delta_t A_t$ and $\delta_{t+1} A_{t+1}$ to be equal, even though $\delta$ is a vector and $A$ a tensor and the exponential is element-wise; if that step fails, the four projection conditions do not actually guarantee unchanged outputs.

Editorial extensions

If this is right

  • If the four conditions hold, a previous-task input produces identical SSM outputs before and after each new task, so old representations are not drifted and no replay buffer is needed.
  • The reported gains on the four class-incremental benchmarks and on 50/100-task long sequences suggest the recipe keeps working when the number of tasks grows.
  • The ablation shows the $\delta$ projection contributes most, because $\delta$ controls discretization of both $A$ and $B$, while the $A$ projection contributes least.
  • The extra training cost is modest (about 7% more runtime on average) and the additional memory is constant in task count, so the method does not inherit the storage growth of rehearsal or expansion methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural follow-up the authors left implicit is to weaken the sufficiency assumptions: because the conditions are sufficient, not necessary, there may be larger allowable update directions that preserve old outputs and improve plasticity.
  • The paper's derivation assumes an element-wise-exponential reduction of $[\exp(\delta A)]^k$; a corrected treatment of $\delta$ as a vector and $A$ as a tensor would also transfer to other selective SSMs that share Mamba's discretization.
  • The guarantee is approximate in practice because the projectors use an approximate null space selected by a corner-point criterion; measuring output drift as a function of the number of discarded singular values would quantify how much of the reported stability comes from the approximation.
  • One could extend the method to streaming-task boundaries by updating the uncentered covariance matrices incrementally instead of recomputing them once per task; the derivation itself does not require clean task boundaries.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Mamba-CL, a rehearsal-free class-incremental learning method that fine-tunes the SSM parameters of a pre-trained Mamba vision backbone under null-space orthogonality constraints. The claimed contribution is a theoretical guarantee that the SSM output on previous-task inputs is unchanged after learning a new task, obtained by deriving four sufficient conditions in Eq. (25) on updates to A, W_B, W_C, and W_delta, and implementing those conditions through SVD-based projectors. The experimental part compares Mamba-CL with prompt- and adapter-based CL methods on ImageNet-R, CIFAR-100, and DomainNet splits, including 50- and 100-task settings, and reports consistent accuracy improvements over the Mamba-Seq baseline and prior ViT-based methods.

Significance. If the consistency guarantee were valid, the paper would be a valuable extension of null-space orthogonal-projection methods to SSM architectures, and the extensive experiments (Tables 1-5, ablation Table 3, runtime analysis in Section 5.5) provide credible evidence of empirical benefit. The paper also reports code availability and ablates each projector individually. However, the theoretical derivation that underpins the central claim is not sound, and the claimed guarantee does not follow from the stated equations. Consequently, the significance of the method as a theoretically grounded continual learning approach is not established, although the empirical observations may remain useful.

major comments (3)
  1. [Section 3.1, Eq. (4)] For the selective SSM described in Eqs. (1)-(2), the forward map is not a fixed-kernel convolution. Since B_l, C_l, and delta_l are functions of the input token x_l, the recurrence h_l = A_bar_l h_{l-1} + B_bar_l x_l, y_l = C_l h_l expands to y_l = sum_{j<=l} C_l (product_{m=j+1}^l A_bar_m) B_bar_j x_j, which is a time-varying lower-triangular operator rather than X*K with a single kernel K = [CB, CAB, CA^2 B, ...]. Therefore Eq. (9), K_t = K_{t+1}, is neither necessary nor sufficient for Eq. (7), and the derivation of Eq. (25) does not establish output consistency for the model actually used. This is the load-bearing step of the paper's theoretical claim.
  2. [Section 4.2.1, Eqs. (12)-(13)] The inference from [exp(delta_t A_t)]^k = [exp(delta_{t+1} A_{t+1})]^k to delta_t A_t = delta_{t+1} A_{t+1} is unjustified. In Eq. (2), A is defined as exp(delta A) element-wise; delta is a vector and A is a D by N tensor, so the exponent is not a scalar, and the monotonicity argument used for [exp(x)]^k does not apply to this tensor product structure. Moreover, even for a scalar argument, equality of all powers only gives equality of the positive exponentials, not directly the equality asserted in Eq. (13). This gap is load-bearing because the subsequent decomposition into delta_t = delta_{t+1} and delta_t Delta A = 0 in Eq. (14) relies on Eq. (13).
  3. [Section 4.3, Eq. (27) and the eta relaxation] Even if Eq. (25) were sufficient for output consistency, the implementation does not enforce Eq. (25) exactly. The projection matrices are built from an approximate null space selected by a corner-point threshold, and the balance factor eta in Section 4.3 replaces H by eta H + (1-eta) I with eta < 1 in the reported experiments. Consequently, the parameter updates actually applied in the experiments do not satisfy the four conditions of Eq. (25), so the abstract's claim that the method 'theoretically guarantees' output consistency is not supported by the implementation. The experiments can still be read as evaluating an approximate version of the idea, but not the exact theoretical guarantee stated in the paper.
minor comments (4)
  1. [Section 3.1, Eq. (2)] The symbol A is used both for the continuous state-transition parameter and for the discretized parameter exp(delta A) in Eq. (2), and the same overloaded notation reappears in Eqs. (10)-(14). Using distinct symbols (for example, A and A_bar) would remove substantial ambiguity from the consistency derivation.
  2. [Section 4.2.2] There is a typo: 'Cosidering' should be 'Considering'.
  3. [Related Work and Experiments] The name 'VMambda' in Section 2.1 and the inconsistent spelling 'CIF AR-100' throughout should be corrected to 'VMamba' and 'CIFAR-100'.
  4. [Section 5.1] The text says that the datasets are 'also split into 50 or 100 tasks' after having described only four benchmarks; the long-sequence results in Table 2 are clear, but the preceding sentence should specify which benchmarks receive the 50- and 100-task splits.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the SSM consistency conditions are derived from the forward equations and validated on held-out benchmarks.

full rationale

The claimed derivation chain is self-contained and does not reduce to a fit or to a self-citation. The paper starts from the SSM forward pass in Eqs. (1)-(4) and the desired output-preservation objective in Eq. (7), then derives sufficient algebraic conditions (Eqs. (10)-(25)) under which the kernel representation K is unchanged. The four conditions in Eq. (25) are not assumed to be the output-preservation property; they are concrete linear-algebra constraints obtained from the forward equations, so satisfying them is a constructive design choice rather than a renamed restatement of the conclusion. The null-space implementation (Eqs. (26)-(27)) borrows the projection technique from Wang et al. (2021) and, with overlapping authorship, Lu et al. (2024)'s VPT-NSP2, but that citation is used as an implementation tool and for the approximate-null-space selection rule, not as the justification for the Mamba-specific consistency theorem, which is derived in full in Sections 4.2-4.3. Empirical evaluation is against standard class-incremental benchmarks with held-out test sets, so the reported accuracy is external evidence. The reviewer concern that selective SSMs have input-dependent B, C, and delta and therefore are not exactly a fixed convolution in Eqs. (3)-(4) is a mathematical-fidelity or correctness risk, not a circularity pattern: the derivation does not define the conclusion into the premises, and the equations are not equivalent to the claim by construction. No circular step can be exhibited from the manuscript text.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical or architectural entities are introduced. The method relies on standard SSM parameters, null-space projectors, and a balance hyperparameter. The load-bearing assumptions are in the theoretical derivation of Section 4.2, especially the reduction of kernel consistency to factor-wise consistency and the treatment of the exponentiated product delta*A.

free parameters (3)
  • eta balance factor = 0.90 for 10-split ImageNet-R, 0.95 for others
    Hyperparameter introduced to relax the orthogonality constraints and balance stability/plasticity. Used in H1 = eta*H1 + (1-eta)*I. Chosen per benchmark by validation.
  • corner-point threshold for approximated null space = selected by maximum second derivative of singular values
    The approximate null space is selected by finding the 'corner' of the singular value spectrum via the maximum second derivative, an adaptive but ad-hoc heuristic. This determines which singular values are treated as zero and directly affects the projection matrices.
  • learning rate, epochs, scheduling = lr 0.0002, 50 epochs, decay at epoch 25 and 40, batch 200
    Standard training hyperparameters chosen by validation; not part of the core method but needed for reproduction.
assumptions (4)
  • domain assumption Consistency of the convolutional kernel K is equivalent to consistency of each factor A^k, B, and C individually (Eq. 11).
    The paper assumes that K_t = K_{t+1} is achieved by the sufficient conditions on A^k, B, and C separately. This is sufficient but not necessary, and the paper explicitly makes this reduction in Eq. (10)-(11).
  • domain assumption The exponential operation in A = exp(delta A) is element-wise and monotonic in a way that allows reducing [exp(delta_t A_t)]^k = [exp(delta_{t+1} A_{t+1})]^k to delta_t A_t = delta_{t+1} A_{t+1} in Eq. (13).
    This is a critical simplification. In Mamba, A is a D x N matrix and delta is a per-token vector, so the product delta*A is not a simple scalar multiplication. The element-wise monotonicity argument does not directly justify the tensor equality. The paper treats the exponential as element-wise and uses the monotonicity of exp(x) to cancel powers, which is a strong assumption.
  • standard math The null space of the uncentered covariance of features determines exactly the set of weight updates that leave old outputs unchanged.
    This is the standard principle from NSCL (Wang et al. 2021) applied to the Mamba block, but for the Mamba conditions it is applied to different feature spaces (X_t, delta_t, delta_t X_t). The projection onto the approximate null space is only a minimum-norm approximation, not an exact guarantee.
  • domain assumption The feature matrices X_t, delta_t, and delta_t X_t collected after each task are representative of the regions where old outputs must be preserved.
    The null-space bases are computed from the training data of each task. If the training distribution does not cover the test distribution or if the features drift across layers, the projection may not prevent forgetting on held-out old-task samples. This is standard in null-space projection methods.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mamba-CL: Optimizing Selective State Space Model in Null Space for Continual Learning." pith.science (2026). https://pith.science/paper/KAG6T6HX

@misc{pith2026241115469,
  author       = {Pith},
  title        = {Pith review of: Mamba-CL: Optimizing Selective State Space Model in Null Space for Continual Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KAG6T6HX}},
  note         = {Machine review of arXiv:2411.15469}
}
read the original abstract

Continual Learning (CL) aims to equip AI models with the ability to learn a sequence of tasks over time, without forgetting previously learned knowledge. Recently, State Space Models (SSMs), particularly the Mamba model, have achieved notable success in computer vision. Building on the strengths of SSMs, this study explores leveraging the Mamba model for CL. Therefore, we introduce Mamba-CL, a framework that continuously fine-tunes the core SSMs of the large-scale Mamba foundation model by updating parameters orthogonal to the feature subspace of previous tasks. This approach theoretically guarantees the consistency objective aiming to preserves consistent output for each SSM module across both previous and current tasks, so as to overcome catastrophic forgetting issue. Specifically, we achieve this goal by deducing the overall consistency constraints on four key time-invariant parameters in the Mamba model, streamlining its recurrent state-space structure and non-linear discretization process in SSM. In practice, we apply the null-space projection to efficiently implement the orthogonality within Mamba model. Extensive experiments on four class-incremental benchmarks demonstrate the effectiveness of Mamba-CL for anti-forgetting, achieving superior performances to state-of-the-art methods. Code is available in the supplementary materials.

Figures

Figures reproduced from arXiv: 2411.15469 by the authors.

Figure 1
Figure 1. Illustration of the proposed Mamba-CL. The backbone contains [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Effects of the orthogonal projection weight [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Results of using the pre-training parameters fur [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Performance with regard to the number of fine-tuned Mamba blocks ( [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Task-by-task accuracy changing curves of the [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Learning to Remember, Learn, and Forget in Attention-Based Models

    cs.LG 2026-02 conditional novelty 6.0 of 10

    Palimpsa adds a per-slot importance/precision state to gated linear attention, letting a fixed-size memory forget stale information and protect important information, and recovers Mamba2 as a high-forgetting limit.

  2. CKAA: Cross-subspace Knowledge Alignment and Aggregation for Robust Continual Learning

    cs.CV 2025-07 conditional novelty 5.0 of 10

    CKAA aligns features and decision boundaries across task-specific subspaces and mixes adapters by task confidence, improving class-incremental learning accuracy under misidentified task labels.

Reference graph

Works this paper leans on

46 extracted references · 26 canonical work pages · cited by 2 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Gradient based sample selection for online continual learning

    Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. Advances in neural information processing systems, 32, 2019

    work page 2019

  3. [3]

    Autoaugment: Learning augmentation strategies from data

    Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. Autoaugment: Learning augmentation strategies from data. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 113--123, 2019

    work page 2019

  4. [4]

    On the effectiveness of layernorm tuning for continual learning in vision transformers

    Thomas De Min, Massimiliano Mancini, Karteek Alahari, Xavier Alameda-Pineda, and Elisa Ricci. On the effectiveness of layernorm tuning for continual learning in vision transformers. In 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pages 3577--3586. IEEE, 2023

    work page 2023

  5. [5]

    Flattening sharpness for dynamic gradient projection memory benefits continual learning

    Danruo Deng, Guangyong Chen, Jianye Hao, Qiong Wang, and Pheng-Ann Heng. Flattening sharpness for dynamic gradient projection memory benefits continual learning. Advances in Neural Information Processing Systems, 34: 0 18710--18721, 2021

    work page 2021

  6. [6]

    Efficient architecture search for continual learning

    Qiang Gao, Zhipeng Luo, Diego Klabjan, and Fengli Zhang. Efficient architecture search for continual learning. IEEE Transactions on Neural Networks and Learning Systems, 34 0 (11): 0 8555--8565, 2022

    work page 2022

  7. [7]

    A unified continual learning framework with general parameter-efficient tuning

    Qiankun Gao, Chen Zhao, Yifan Sun, Teng Xi, Gang Zhang, Bernard Ghanem, and Jian Zhang. A unified continual learning framework with general parameter-efficient tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11483--11493, 2023

    work page 2023

  8. [8]

    Consistent prompting for rehearsal-free continual learning

    Zhanxin Gao, Jun Cen, and Xiaobin Chang. Consistent prompting for rehearsal-free continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 28463--28473, 2024

    work page 2024

Show all 46 references
  1. [9]

    Mamba: Linear-time sequence modeling with selective state spaces

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023

    2023 arXiv

  2. [10]

    Efficiently modeling long sequences with structured state spaces

    Albert Gu, Karan Goel, and Christopher R \'e . Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.00396, 2021

    2021 arXiv

  3. [11]

    Mambavision: A hybrid mamba-transformer vision backbone

    Ali Hatamizadeh and Jan Kautz. Mambavision: A hybrid mamba-transformer vision backbone. arXiv preprint arXiv:2407.08083, 2024

    2024 arXiv

  4. [12]

    The many faces of robustness: A critical analysis of out-of-distribution generalization

    Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF international c...

    2021

  5. [13]

    Curiosity-driven class-incremental learning via adaptive sample selection

    Qinghua Hu, Yucong Gao, and Bing Cao. Curiosity-driven class-incremental learning via adaptive sample selection. IEEE Transactions on Circuits and Systems for Video Technology, 32 0 (12): 0 8660--8673, 2022

    2022

  6. [14]

    OVOR : Oneprompt with virtual outlier regularization for rehearsal-free class-incremental learning

    Wei-Cheng Huang, Chun-Fu Chen, and Hsiang Hsu. OVOR : Oneprompt with virtual outlier regularization for rehearsal-free class-incremental learning. In The Twelfth International Conference on Learning Representations, 2024

    2024

  7. [15]

    Compacting, picking and growing for unforgetting continual learning

    Ching-Yi Hung, Cheng-Hao Tu, Cheng-En Wu, Chien-Hung Chen, Yi-Ming Chan, and Chu-Song Chen. Compacting, picking and growing for unforgetting continual learning. Advances in neural information processing systems, 32, 2019

    2019

  8. [16]

    Efficient movie scene detection using state-space transformers

    Md Mohaiminul Islam, Mahmudul Hasan, Kishan Shamsundar Athrey, Tony Braskich, and Gedas Bertasius. Efficient movie scene detection using state-space transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18749--18758, 2023

    2023

  9. [17]

    Introducing language guidance in prompt-based continual learning

    Muhammad Gul Zain Ali Khan, Muhammad Ferjad Naeem, Luc Van Gool, Didier Stricker, Federico Tombari, and Muhammad Zeshan Afzal. Introducing language guidance in prompt-based continual learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11...

    2023

  10. [18]

    Adam: A method for stochastic optimization

    Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  11. [19]

    Overcoming catastrophic forgetting in neural networks

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of scien...

    2017

  12. [20]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009

    2009

  13. [21]

    Evolving parameterized prompt memory for continual learning

    Muhammad Rifki Kurniawan, Xiang Song, Zhiheng Ma, Yuhang He, Yihong Gong, Yang Qi, and Xing Wei. Evolving parameterized prompt memory for continual learning. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 13301--13309, 2024

    2024

  14. [22]

    Adaptive plasticity improvement for continual learning

    Yan-Shuo Liang and Wu-Jun Li. Adaptive plasticity improvement for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7816--7825, 2023

    2023

  15. [23]

    Inflora: Interference-free low-rank adaptation for continual learning

    Yan-Shuo Liang and Wu-Jun Li. Inflora: Interference-free low-rank adaptation for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23638--23647, 2024

    2024

  16. [24]

    Vmamba: Visual state space model

    Yue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu, Lingxi Xie, Yaowei Wang, Qixiang Ye, and Yunfan Liu. Vmamba: Visual state space model. arXiv preprint arXiv:2401.10166, 2024

    2024 arXiv

  17. [25]

    Visual prompt tuning in null space for continual learning

    Yue Lu, Shizhou Zhang, De Cheng, Yinghui Xing, Nannan Wang, Peng Wang, and Yanning Zhang. Visual prompt tuning in null space for continual learning. Advances in neural information processing systems, 2024

    2024

  18. [26]

    U-mamba: Enhancing long-range dependency for biomedical image segmentation

    Jun Ma, Feifei Li, and Bo Wang. U-mamba: Enhancing long-range dependency for biomedical image segmentation. arXiv preprint arXiv:2401.04722, 2024

    2024 arXiv

  19. [27]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019

    2019

  20. [28]

    Moment matching for multi-source domain adaptation

    Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1406--1415, 2019

    2019

  21. [29]

    Prompt gradient projection for continual learning

    Jingyang Qiao, Xin Tan, Chengwei Chen, Yanyun Qu, Yong Peng, Yuan Xie, et al. Prompt gradient projection for continual learning. In The Twelfth International Conference on Learning Representations, 2023

    2023

  22. [30]

    Convolutional prompting meets language models for continual learning

    Anurag Roy, Riddhiman Moulick, Vinay K Verma, Saptarshi Ghosh, and Abir Das. Convolutional prompting meets language models for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23616--23626, 2024

    2024

  23. [31]

    Imagenet large scale visual recognition challenge

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115: 0 211--252, 2015

    2015

  24. [32]

    Gradient projection memory for continual learning

    Gobinda Saha, Isha Garg, and Kaushik Roy. Gradient projection memory for continual learning. arXiv preprint arXiv:2103.09762, 2021

    2021 arXiv

  25. [33]

    Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning

    James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascante-Bonilla, Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, and Zsolt Kira. Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning. In Proceedings of the IEE...

    2023

  26. [34]

    Simplified state space layers for sequence modeling

    Jimmy TH Smith, Andrew Warrington, and Scott W Linderman. Simplified state space layers for sequence modeling. arXiv preprint arXiv:2208.04933, 2022

    2022 arXiv

  27. [35]

    Learning 1d causal visual representation with de-focus attention networks

    Chenxin Tao, Xizhou Zhu, Shiqian Su, Lewei Lu, Changyao Tian, Xuan Luo, Gao Huang, Hongsheng Li, Yu Qiao, Jie Zhou, et al. Learning 1d causal visual representation with de-focus attention networks. arXiv preprint arXiv:2406.04342, 2024

    2024 arXiv

  28. [36]

    Selective structured state-spaces for long-form video understanding

    Jue Wang, Wentao Zhu, Pichao Wang, Xiang Yu, Linda Liu, Mohamed Omar, and Raffay Hamid. Selective structured state-spaces for long-form video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6387--6397, 2023 a

    2023

  29. [37]

    Training networks in null space of feature covariance for continual learning

    Shipeng Wang, Xiaorong Li, Jian Sun, and Zongben Xu. Training networks in null space of feature covariance for continual learning. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pages 184--193, 2021

    2021

  30. [38]

    Isolation and impartial aggregation: A paradigm of incremental learning without interference

    Yabin Wang, Zhiheng Ma, Zhiwu Huang, Yaowei Wang, Zhou Su, and Xiaopeng Hong. Isolation and impartial aggregation: A paradigm of incremental learning without interference. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 10209--10217, 2023 b

    2023

  31. [39]

    Dualprompt: Complementary prompting for rehearsal-free continual learning

    Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, et al. Dualprompt: Complementary prompting for rehearsal-free continual learning. In European Conference on Computer Vision, pages 631--648. Spring...

    2022

  32. [40]

    Learning to prompt for continual learning

    Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 139--149, 2022 b

    2022

  33. [41]

    Pytorch image models

    Ross Wightman. Pytorch image models. https://github.com/rwightman/pytorch-image-models, 2019

    2019

  34. [42]

    Scalable and order-robust continual learning with additive parameter decomposition

    Jaehong Yoon, Saehoon Kim, Eunho Yang, and Sung Ju Hwang. Scalable and order-robust continual learning with additive parameter decomposition. International Conference on Learning Representations, 2020

    2020

  35. [43]

    Continual learning of context-dependent processing in neural networks

    Guanxiong Zeng, Yang Chen, Bo Cui, and Shan Yu. Continual learning of context-dependent processing in neural networks. Nature Machine Intelligence, 1 0 (8): 0 364--372, 2019

    2019

  36. [44]

    Memory-efficient class-incremental learning for image classification

    Hanbin Zhao, Hui Wang, Yongjian Fu, Fei Wu, and Xi Li. Memory-efficient class-incremental learning for image classification. IEEE Transactions on Neural Networks and Learning Systems, 33 0 (10): 0 5966--5977, 2021

    2021

  37. [45]

    Expandable subspace ensemble for pre-trained model-based class-incremental learning

    Da-Wei Zhou, Hai-Long Sun, Han-Jia Ye, and De-Chuan Zhan. Expandable subspace ensemble for pre-trained model-based class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23554--23564, 2024

    2024

  38. [46]

    Vision mamba: Efficient visual representation learning with bidirectional state space model

    Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model. arXiv preprint arXiv:2401.09417, 2024

    2024 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.