Pith. sign in

REVIEW 3 major objections 5 minor 39 references

The paper argues that analytic continual learning fails in long-tailed streams because tail-class eigenvalues collapse to near zero, and that a geodesic-mixup perturbation called GSR restores the Gram matrix's stable rank, yielding large ac

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 04:06 UTC pith:IXTUQC4H

load-bearing objection The empirical method is promising enough to referee, but the central theorem does not apply to the algorithm as implemented. the 3 major comments →

arxiv 2607.22931 v1 pith:IXTUQC4H submitted 2026-07-24 cs.LG cs.CV

Spectral-Aware Analytic Class-Incremental Learning for Long-Tailed Distributions

Quyen Tran , Hai Nguyen , Quan Dao , Zhuowei Li , Nam Le , Trung Le , Dimitris Metaxas This is my paper
classification cs.LG cs.CV
keywords continual learningclass-incremental learninglong-tailed distributionsspectral regularizationstable rankGram matrixspherical mixupanalytic learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Analytic (recursive least squares) continual learners are fast because they invert a Gram matrix in closed form, but the authors argue that in long-tailed class streams this inversion becomes numerically meaningless: tail-class directions have near-zero eigenvalues, so their subspaces are indistinguishable from noise. A single ridge penalty cannot fix the problem because it damps all eigenvalues equally—enough to stabilize the tail over-shrinks the head. GSR treats the imbalance as a spectral-regularization problem: it synthesizes virtual tail samples by spherical (geodesic) mixup and adds their second-moment contribution as a structured perturbation that selectively inflates collapsed tail eigenvalues. The paper proves that this strictly raises the stable rank of the regularized Gram matrix under stated assumptions, and reports that it lifts tail-class accuracy by 3–4× and achieves the best known results for analytic continual learning on several benchmarks.

Core claim

The paper's central claim is that the failure of RLS-based ACL under long-tailed data is a spectral pathology, not merely data scarcity: the Gram matrix develops a skewed eigenvalue distribution where head classes dominate the principal directions and tail classes sit in near-zero directions, so inverting it amplifies noise in tail directions and destroys classification. Standard ridge regularization is isotropic—it adds the same τ to every eigenvalue—and therefore cannot stabilize the tail without over-shrinking the head. GSR instead builds an anisotropic, data-dependent perturbation Δ by spherical mixup of same-class pairs, so that the perturbed Gram matrix G+Δ selectively gains mass along

What carries the argument

The central object is the structured spectral perturbation matrix Δ induced by spherical mixup. For each tail class, augmented samples are created as z̃ = (γz1+(1−γ)z2)/‖γz1+(1−γ)z2‖ with γ~Beta(αc,αc), and the mixing intensity αc = α_base + (1−α_base)e^{−ξN_c} decreases as the class sample count N_c grows, so tail classes are mixed aggressively and head classes are barely perturbed. Δ is the difference in the second moment (Gram matrix) caused by adding these augmented samples with weight β. The paper's Theorem 1 shows that if Δ's tail block has minimum eigenvalue at least m>0 and its cross-blocks are bounded, then the stable rank of the regularized Gram matrix increases; the adaptive mixin

Load-bearing premise

The stable-rank theorem requires that the synthetic mixup perturbation adds positive variance in every tail-class direction (λ_min(U_t^T Δ U_t) ≥ m > 0), but with only a few real tail samples the perturbation is low-rank and likely has zero variance in most tail directions, so the paper's main theoretical guarantee probably does not apply to the algorithm as implemented.

What would settle it

For a typical tail class with the few samples used in the experiments, compute the smallest eigenvalue of the tail-block projected perturbation U_t^T Δ U_t; if it is zero to machine precision, Theorem 1's key hypothesis fails and the claimed stable-rank improvement is not proven for the deployed method.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Any RLS-based analytic continual learner can be made long-tailed–robust by inserting GSR's mixup-based Gram update before inversion, with no backbone retraining and O(D) cost per class.
  • Tail classes that were effectively ignored (accuracy near 10–20%) can be recovered to 3–4× their baseline accuracy while head-class accuracy drops by only about one point, so overall accuracy improves sharply.
  • The stable-rank guarantee implies the Gram matrix inversion remains numerically well-conditioned across many tasks, which should reduce the noise-amplification source of catastrophic forgetting in the analytic head.
  • Because the regularizer is anisotropic and data-driven, it avoids the manifold intrusion of isotropic Gaussian noise and the semantic bias of transferring head-class covariance to tail classes.
  • The paper's diagnosis implies that balanced-benchmark results overstate the reliability of existing analytic methods; long-tailed evaluations are needed to expose the spectral failure mode.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's own appendix concedes that a higher stable rank does not automatically imply better generalization; connecting the spectral bound to per-class error rates (e.g., via spectral entropy or tail-energy ratios) would turn the guarantee into a learning-theoretic one.
  • With only a handful of tail samples, the empirical Δ is a sum of rank-1 terms in the span of those samples, so the assumption λ_min(U_t^T Δ U_t) ≥ m > 0 is likely unmet in high-dimensional feature spaces; whether the theorem covers the implemented algorithm is an open question the paper does not address.
  • The same recipe—class-adaptive, geometry-respecting augmentation as spectral preconditioning—could be dropped into other ill-conditioned analytic estimators, such as few-shot classifiers or streaming linear discriminant analysis, where covariance matrices are built from very few examples.
  • A direct test of the mechanism is to ablate the geodesic projection: if linear mixup with the same number of samples also raises stable rank when followed by renormalization, the effect may be due to norm restoration rather than geodesic directionality.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper addresses class-incremental learning with analytic/RLS classifiers under long-tailed class distributions. It argues that the Gram-matrix inversion in RLS suffers from spectral collapse in tail classes, that isotropic ridge regularization cannot fix this, and proposes Geometry-Spectral Rectification (GSR): an adaptive, class-wise spherical mixup of tail-class features whose Gram contribution is added with weight β to the recursive least-squares update. The central theoretical claim is Theorem 1, which states that under blockwise bounds on a perturbation Δ (in particular λ_min(U_t^T Δ U_t) ≥ m > 0), the stable rank of G + Δ + τI is lower-bounded above that of G + τI. Experiments on four long-tailed CIL benchmarks with DINO-v2 and MoCo-v3 show consistent gains over the included RLS baselines, including the reported state-of-the-art results for AnaCP+GSR.

Significance. The empirical work is substantial: multiple backbones, datasets, ablations, imbalance-ratio sensitivity, class-frequency breakdowns, and complexity comparisons. If the theoretical guarantee were connected to the algorithm, the paper would make a useful conceptual contribution by framing long-tailed ACL as a spectral conditioning problem rather than a re-weighting problem. However, the proof of Theorem 1 is conditional on assumptions that are neither derived nor satisfied by the implemented perturbation; the claimed 'guarantee' therefore does not hold for GSR as specified. The experimental gains are credible, but they cannot currently be attributed to the theorem's mechanism. The paper would need either a corrected theoretical statement matched to the actual perturbation or a substantial softening of the claimed guarantee.

major comments (3)
  1. [Sec. 3.3b, Theorem 1; Eq. (9)-(10); Appendix B.4] The assumption m > 0 is load-bearing and is never established for the actual Δ. In GSR, Δ = β Σ z̃ z̃^T over augmented samples, and each z̃ is a normalized convex combination of two observed tail samples (Eq. 9). Hence range(Δ) is contained in the span of the observed tail samples. With N_c = 5 tail samples per class and D = 768–5000, U_t^T Δ U_t has rank at most N_c per class, so λ_min(U_t^T Δ U_t) = 0 for any tail space of dimension larger than N_c. The proof's lower bound (d−r)m^2 then vanishes and sufficient condition (7) cannot be satisfied. The text's claim that the rectification term 'injects variance into the null-space of the tail class manifold' (Sec. 3.3c) is geometrically impossible for this construction: interpolation between observed samples cannot create variation orthogonal to their span. This is not a small gap; the abstract's guarantee of improved stable rank depends on
  2. [Sec. 3.3b / Algorithm 1] Theorem 1 is not instantiated. It is stated for an arbitrary split r of the eigen-spectrum of G, with blockwise bounds on Δ in that basis, but Algorithm 1 never computes U_h/U_t, never selects r, and never checks the assumed bounds. Moreover, the theorem is only a sufficient condition for strict improvement: the paper's later assertion that 'GSR strictly increases sr(\tilde G + τI)' over-states what Theorem 1 proves, since condition (7) is never verified. Finally, the algorithm applies a sequence of class-wise perturbations across incremental tasks, while the theorem treats a single global Δ; no argument shows that the accumulated per-task perturbations satisfy the theorem's hypotheses.
  3. [Sec. 4.3, Fig. 2 / Appendix B.4] The empirical numerical-stability analysis does not validate the theorem's mechanism. Fig. 2 reports the global stable rank of (G + τI), which can increase simply because a PSD perturbation raises the Frobenius norm; it does not measure the tail block U_t^T Δ U_t, nor does it verify m > 0 or condition (7). The appendix itself concedes that higher stable rank does not necessarily lead to better generalization, further weakening the link from Fig. 2 to the reported accuracy gains. The accuracy improvements are real, but the paper has not shown that they are caused by the claimed spectral rectification of tail directions.
minor comments (5)
  1. [Sec. 3.1] Typo: 'RSL-based' should be 'RLS-based'.
  2. [Sec. 3.3a / Appendix B.2] The statement that spherical mixup 'prevents the synthetic sample from lying in the linear span of the supports' is inaccurate: normalization keeps z̃ in the linear span of z1 and z2, although outside the chord. This is related to the major concern above but should be corrected independently.
  3. [Appendix A.3c] The method is called 'Gaussian Spherical Regression (GSR)' here, inconsistent with 'Geometry-Spectral Rectification' in the main text.
  4. [Algorithm 1 / Appendix A.1] The claimed O(D) complexity ignores the all-pairs loop over sample pairs; the per-class complexity is O(N_c^2 D), which is O(D) only for the small N_c used in these experiments. This should be stated explicitly.
  5. [Appendix C.1] The notation for the tail shot count switches between N_c and N_min; please use a single symbol consistently.

Circularity Check

0 steps flagged

No significant circularity; the stable-rank theorem is a conditional statement and the empirical gains are not fitted predictions.

full rationale

The paper's derivation chain is not circular. GSR's augmented Gram update (Eq. 10) is an explicit construction, and Theorem 1 (Sec. 3.3b) is a conditional linear-algebra statement with stated blockwise assumptions (epsilon, eta, m, m-bar); the proof in Appendix B.4 derives the stable-rank lower bound from those assumptions rather than from the definition of GSR alone. The empirical stable-rank plot (Fig. 2) is a measurement of the actual updated matrix, not a fitted 'prediction' from the theorem. No parameter is fitted to the reported benchmark accuracies: alpha_base, xi, beta, and tau are fixed hyperparameters (Appendix C.3). The paper's self-citations (refs 23-27) concern other continual-learning works and are not load-bearing for the spectral claims. The main substantive weakness is a correctness gap, not circularity: Theorem 1 requires lambda_min(U_t^T Delta U_t) >= m > 0, but Delta is a sum of rank-1 terms z-tilde z-tilde^T with z-tilde in the span of two observed samples (Eq. 9), so the tail block is rank-deficient and the m > 0 premise is unverified and likely false; the claim that the rectification term 'injects variance into the null-space' (Sec. 3.3c) is not supported by the defining equation. The paper itself concedes in Appendix B.4 that a higher stable rank does not necessarily imply better generalization. These are concerns about proof validity and assumption satisfaction, not about circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 0 invented entities

GSR relies on standard RLS algebra, the unit-norm hypersphere assumption, the VRM equivalence between mixup and data-dependent regularization, and—critically—unproven blockwise spectral bounds on Δ. The most fragile load-bearing assumption is m>0 in Theorem 1: for the actual spherical-mixup Δ from Eq. (9), mixup samples remain in the span of the few real tail samples, so the perturbation cannot fill the full tail eigenspace.

free parameters (4)
  • α_base = 0.6
    Base mixing intensity in Eq. (8); chosen by hand, controls how much synthetic variance is injected into tail classes.
  • ξ = 5×10^-3
    Decay rate in Eq. (8); chosen by hand, controls how quickly mixing intensity falls with class sample count.
  • τ = 0.01 (DINO-v2), 0.001 (MoCo-v3)
    Ridge regularization in Eq. (2); set per backbone, a standard but hand-tuned hyperparameter.
  • β = 1
    Balancing coefficient for the rectification term in Eq. (10); set to 1 'for simplicity'.
axioms (6)
  • standard math The closed-form RLS solution W* = (G+τI)^-1 Z^T Y and its recursive updates are valid.
    Used throughout Section 3.1, Eq. (2), as the foundation of ACL.
  • domain assumption Pretrained embeddings are unit-norm on the hypersphere S^{d-1}.
    Required for spherical mixup Eq. (9) and the geometric arguments in Sec. 2.3/Appendix A; not explicitly enforced in Algorithm 1.
  • domain assumption Vicinal Risk Minimization: training on mixed samples is asymptotically equivalent to a data-dependent regularizer, so E[x̃x̃^T] ≈ G + Δ.
    Invoked in Sec. 3.3a, Eq. (5), citing [1,3,29]; this is the bridge from mixup to spectral regularization.
  • ad hoc to paper Theorem 1's blockwise bounds hold for Δ: ∥U_h^T Δ U_h∥≤ε, ∥U_h^T Δ U_t∥≤η, and λ_min(U_t^T Δ U_t) ≥ m > 0.
    These bounds are assumed in Theorem 1, Sec. 3.3b, and never derived for the spherical-mixup Δ. The m>0 condition is especially questionable because mixup samples stay in the span of the source samples.
  • ad hoc to paper Tail-class near-zero eigenvalues stem from sample scarcity rather than absence of semantic information.
    Used in Sec. 3.3b to justify that injecting variance into collapsed tail directions is safe; asserted, not proven.
  • domain assumption Higher stable rank is a meaningful proxy for numerical stability and generalization.
    The whole theoretical framing relies on stable rank; Appendix B.4 explicitly concedes that higher stable rank does not necessarily imply better generalization.

pith-pipeline@v1.3.0-alltime-deepseek · 20806 in / 14898 out tokens · 138514 ms · 2026-08-01T04:06:42.795307+00:00 · methodology

0 comments
read the original abstract

Analytic Continual Learning (ACL) offers a computationally efficient alternative to gradient-based approaches. Recent ACL methods are based on Recursive Least Squares (RLS) and have achieved the state-of-the-art results compared to other alternatives. However, they falter significantly in Class-Incremental Learning scenarios characterized by Long-Tailed distributions. While the ill-conditioning of the autocorrelation (Gram) matrix is a known limitation of RLS, we demonstrate that class imbalance exacerbates this issue into a distinct spectral pathology: "tail" classes suffer from severe spectral collapse, rendering their subspaces numerically indistinguishable from noise. Standard Ridge Regression ($L_2$) fails to address this effectively as it applies isotropic regularization - a uniform penalty that is insufficient to stabilize the tail without over-shrinking the head. To address this, we propose Geometry-Spectral Rectification (GSR), a theoretically grounded framework that treats long-tailed learning as a spectral regularization problem. Unlike standard isotropic regularization (Ridge) which uniformly penalizes all eigenvalues, GSR acts as an anisotropic spectral filter, selectively inflating the collapsed eigenvalues of tail classes. We construct a structured, data-dependent spectral perturbation matrix $\Delta$ that selectively inflates collapsed tail eigen-directions of the Gram matrix. Theoretical analysis proves that GSR guarantees an improved stable rank for the Gram matrix, ensuring numerical stability. Extensive experiments show that GSR establishes a new state-of-the-art for analytic CIL, offering a superior trade-off between computational efficiency and robust generalization in long-tailed settings.

Figures

Figures reproduced from arXiv: 2607.22931 by Dimitris Metaxas, Hai Nguyen, Nam Le, Quan Dao, Quyen Tran, Trung Le, Zhuowei Li.

Figure 1
Figure 1. Figure 1: Performance in standard and long-tailed settings of different ACL meth￾ods, on Split-Imagenet-R. See Fig.3 in Appendix D for full results on different datasets. In RLS-based ACL, the optimal weight solution W ∗ is governed by the in￾verse of the autocorrelation (Gram) matrix G. We demonstrate that in imbal￾anced streams, G suffers from severe spectral skewness. "Head" classes dominate the principal compone… view at source ↗
Figure 2
Figure 2. Figure 2: Numerical Stability Analysis. We report the Stable Rank of the matrix (G+τ I) at different incremental stages. While GACL (dashed lines) suffers from severe spectral collapse with a stable rank near 1.0, our proposed GSR (solid lines) consis￾tently helps maintain a higher stable rank across all tasks, indicating a well-conditioned feature space and effective mitigation of collapse. "img-r" stands for Split… view at source ↗
Figure 3
Figure 3. Figure 3: Performance of baselines in standard vs long-tailed settings - Alast (%) on different datasets [PITH_FULL_IMAGE:figures/full_fig_p029_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 5 linked inside Pith

  1. [1]

    Neural Comput.7(1), 108–116 (Jan 1995).https://doi.org/10.1162/neco.1995.7.1

    Bishop, C.M.: Training with noise is equivalent to tikhonov regularization. Neural Comput.7(1), 108–116 (Jan 1995).https://doi.org/10.1162/neco.1995.7.1. 108,https://doi.org/10.1162/neco.1995.7.1.108

    doi:10.1162/neco.1995.7.1 1995

  2. [2]

    In: Advances in Neural Information Pro- cessing Systems (2019)

    Cao, K., Wei, C., Gaidon, A., Arechiga, N., Ma, T.: Learning imbalanced datasets with label-distribution-aware margin loss. In: Advances in Neural Information Pro- cessing Systems (2019)

    2019

  3. [3]

    In: Leen, T., Dietterich, T., Tresp, V

    Chapelle, O., Weston, J., Bottou, L., Vapnik, V.: Vicinal risk minimization. In: Leen, T., Dietterich, T., Tresp, V. (eds.) Advances in Neural Information Pro- cessing Systems. vol. 13. MIT Press (2000),https://proceedings.neurips.cc/ paper _ files / paper / 2000 / file / ba9a56ce0a9bfa26e8ed9e10b2cc8f46 - Paper . pdf

    2000

  4. [4]

    Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res.16, 321–357 (2002).https: //doi.org/10.1613/JAIR.953,https://doi.org/10.1613/jair.953

    doi:10.1613/jair.953 2002

  5. [5]

    In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019

    Cui, Y., Jia, M., Lin, T., Song, Y., Belongie, S.J.: Class-balanced loss based on effective number of samples. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019. pp. 9268–9277. Computer Vision Foundation / IEEE (2019).https://doi.org/10.1109/CVPR. 2019.00949,http://openaccess.thecvf.com/conten...

    arXiv 2019

  6. [6]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2019)

    Deng, J., Guo, J., Xue, N., Zafeiriou, S.: Arcface: Additive angular margin loss for deep face recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2019)

    2019

  7. [7]

    10349,https://arxiv.org/abs/2408.10349

    Fang, D., Zhu, Y., Lin, Z., Chen, C., Zeng, Z., Zhuang, H.: AIR: Analytic imbalance rectifier for continual learning (2024).https://doi.org/10.48550/arXiv.2408. 10349,https://arxiv.org/abs/2408.10349

    Pith/arXiv arXiv doi:10.48550/arxiv.2408.10349 2024

  8. [8]

    In: Thirty- seventh Conference on Neural Information Processing Systems (2023),https:// openreview.net/forum?id=Asx5eDqFZl

    Goswami, D., Liu, Y., Twardowski, B., van de Weijer, J.: FeCAM: Exploiting the heterogeneity of class distributions in exemplar-free continual learning. In: Thirty- seventh Conference on Neural Information Processing Systems (2023),https:// openreview.net/forum?id=Asx5eDqFZl

    2023

  9. [9]

    In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (June 2020)

    Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (June 2020)

    2020

  10. [10]

    In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2019)

    Hou, S., Pan, X., Loy, C.C., Wang, Z., Lin, D.: Learning a unified classifier in- crementally via rebalancing. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2019)

    2019

  11. [11]

    CoRRabs/2210.04428(2022)

    Janson, P., Zhang, W., Aljundi, R., Elhoseiny, M.: A simple baseline that questions the use of pretrained-models in continual learning. CoRRabs/2210.04428(2022). https://doi.org/10.48550/ARXIV.2210.04428,https://doi.org/10.48550/ arXiv.2210.04428

    Pith/arXiv arXiv doi:10.48550/arxiv.2210.04428 2022

  12. [12]

    arXiv preprint arXiv:2110.09348 (2021)

    Jing, L., Vincent, P., LeCun, Y., Tian, Y.: Understanding dimensional collapse in contrastive self-supervised learning. arXiv preprint arXiv:2110.09348 (2021)

    Pith/arXiv arXiv 2021

  13. [13]

    In: 8th In- ternational Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020

    Kang, B., Xie, S., Rohrbach, M., Yan, Z., Gordo, A., Feng, J., Kalantidis, Y.: Decoupling representation and classifier for long-tailed recognition. In: 8th In- ternational Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net (2020),https://openreview.net/ forum?id=r1gRTCVFvB 16 Tran et al

    2020

  14. [14]

    In: International Conference on Learning Representations (2018),https://openreview.net/forum?id=Hk99zCeAb

    Karras, T., Aila, T., Laine, S., Lehtinen, J.: Progressive growing of GANs for improved quality, stability, and variation. In: International Conference on Learning Representations (2018),https://openreview.net/forum?id=Hk99zCeAb

    2018

  15. [15]

    In: European Conference on Computer Vision

    Kim, C.D., Jeong, J., Kim, G.: Imbalanced continual learning with partitioning reservoir sampling. In: European Conference on Computer Vision. pp. 411–428. Springer (2020)

    2020

  16. [16]

    In: European Conference on Computer Vision

    Liu, X., Hu, Y.S., Cao, X.S., Bagdanov, A.D., Li, K., Cheng, M.M.: Long-tailed class incremental learning. In: European Conference on Computer Vision. pp. 495–

  17. [17]

    In: Thirty-seventh Conference on Neural Information Processing Systems (2023), https://openreview.net/forum?id=aec58UfBzA

    McDonnell, M., Gong, D., Parvaneh, A., Abbasnejad, E., van den Hengel, A.: RanPAC: Random projections and pre-trained models for continual learning. In: Thirty-seventh Conference on Neural Information Processing Systems (2023), https://openreview.net/forum?id=aec58UfBzA

    2023

  18. [18]

    In: Proceedings of the AAAI Conference on Artificial Intelligence (2025)

    Momeni,S.,Mazumder,S.,Liu,B.:Continuallearningusingakernel-basedmethod over foundation models. In: Proceedings of the AAAI Conference on Artificial Intelligence (2025)

    2025

  19. [19]

    CoRRabs/2511.13880(2025).https: //doi.org/10.48550/ARXIV.2511.13880,https://doi.org/10.48550/arXiv

    Momeni, S., Xiao, C., Liu, B.: Anacp: Toward upper-bound continual learn- ing via analytic contrastive projection. CoRRabs/2511.13880(2025).https: //doi.org/10.48550/ARXIV.2511.13880,https://doi.org/10.48550/arXiv. 2511.13880

    doi:10.48550/arxiv.2511.13880 2025

  20. [20]

    In: 2024 IEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP)

    Nguyen, T., Jiang, R., Aeron, S., Ishwar, P., Brown, D.R.: On neural collapse in contrastive learning with imbalanced datasets. In: 2024 IEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP). pp. 1–6. IEEE (2024)

    2024

  21. [21]

    In: 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021

    Saha, G., Garg, I., Roy, K.: Gradient projection memory for continual learning. In: 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net (2021),https://openreview. net/forum?id=3AOj0RCNC2

    2021

  22. [22]

    In: Proceedings of the 12th Annual Conference on Computer Graphics and Interactive Techniques

    Shoemake, K.: Animating rotation with quaternion curves. In: Proceedings of the 12th Annual Conference on Computer Graphics and Interactive Techniques. p. 245–254. SIGGRAPH ’85, Association for Computing Machinery, New York, NY, USA (1985).https://doi.org/10.1145/325334.325242,https://doi.org/10. 1145/325334.325242

    arXiv 1985

  23. [23]

    In: Walsh, T., Shah, J., Kolter, Z

    Thanh, N.X., Le, A.D., Tran, Q., Le, T., Van, L.N., Nguyen, T.H.: Few-shot, no problem: Descriptive continual relation extraction. In: Walsh, T., Shah, J., Kolter, Z. (eds.) Thirty-Ninth AAAI Conference on Artificial Intelligence, Thirty-Seventh Conference on Innovative Applications of Artificial Intelligence, Fifteenth Sympo- sium on Educational Advances...

    2025

  24. [24]

    Tran, Q., Nguyen, H., Phan, H., Dao, Q., Ngo, L., Than, K., Phung, D., Metaxas, D., Le, T.: An optimal transport-driven approach for cultivating latent space in online incremental learning (2026),https://arxiv.org/abs/2211.16780

    Pith/arXiv arXiv 2026

  25. [25]

    Tran, Q., Phan, H., Le, M., Truong, T., Phung, D., Ngo, L., Nguyen, T., Ho, N., Le, T.: Leveraging hierarchical taxonomies in prompt-based continual learning (2025), https://arxiv.org/abs/2410.04327

    Pith/arXiv arXiv 2025

  26. [26]

    In: Al-Onaizan, Y., Bansal, M., Chen, Y.N

    Tran, Q., Thanh, N.X., Anh, N.H., Hai, N.L., Le, T., Ngo, L.V., Nguyen, T.H.: Preserving generalization of language models in few-shot continual relation ex- traction. In: Al-Onaizan, Y., Bansal, M., Chen, Y.N. (eds.) Proceedings of the Geometry-Spectral Rectification (GSR) 17 2024 Conference on Empirical Methods in Natural Language Processing. pp. 13771–...

    doi:10.18653/v1/2024.emnlp- 2024

  27. [27]

    In: The Thirteenth Inter- national Conference on Learning Representations (2025),https://openreview

    Tran, Q., Tran, T.L., Doan, K., Tran, T., Phung, D., Than, K., Le, T.: Boosting multiple views for pretrained-based continual learning. In: The Thirteenth Inter- national Conference on Learning Representations (2025),https://openreview. net/forum?id=AZR4R3lw7y

    2025

  28. [28]

    In: Chaudhuri, K., Salakhutdinov, R

    Verma, V., Lamb, A., Beckham, C., Najafi, A., Mitliagkas, I., Lopez-Paz, D., Ben- gio, Y.: Manifold mixup: Better representations by interpolating hidden states. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, Cal- ifornia, USA. Proceedings of Machine ...

    2019

  29. [29]

    In: Advances in Neural Information Processing Systems (NeurIPS)

    Wang, Y., Pan, X., Song, S., Zhang, H., Huang, G., Wu, C.: Implicit semantic data augmentation for deep networks. In: Advances in Neural Information Processing Systems (NeurIPS). pp. 12635–12644 (2019)

    2019

  30. [30]

    In: IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022

    Wang, Z., Zhang, Z., Lee, C., Zhang, H., Sun, R., Ren, X., Su, G., Perot, V., Dy, J.G., Pfister, T.: Learning to prompt for continual learning. In: IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022. pp. 139–149. IEEE (2022).https://doi.org/10.1109/ CVPR52688.2022.00024,https://doi.org/10.11...

    arXiv 2022

  31. [31]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    Wu, Y., Chen, Y., Wang, L., Ye, Y., Liu, Z., Guo, Y., Fu, Y.: Large scale incre- mental learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 374–382 (2019)

    2019

  32. [32]

    International Conference on Learning Representations (2018), https://openreview.net/forum?id=r1Ddp1-Rb

    Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: Beyond empirical risk minimization. International Conference on Learning Representations (2018), https://openreview.net/forum?id=r1Ddp1-Rb

    2018

  33. [33]

    In: Advances in Neu- ral Information Processing Systems

    Zhuang, H., Chen, Y., Fang, D., He, R., Tong, K., Wei, H., Zeng, Z., Chen, C.: GACL: Exemplar-free generalized analytic continual learning. In: Advances in Neu- ral Information Processing Systems. Curran Associates, Inc. (Dec 2024)

    2024

  34. [34]

    In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Zhuang, H., Weng, Z., He, R., Lin, Z., Zeng, Z.: GKEAL: Gaussian kernel em- bedded analytic learning for few-shot class incremental task. In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 7746–7755 (Jun 2023).https://doi.org/10.1109/CVPR52729.2023.00748

    arXiv 2023

  35. [35]

    Spectral-Aware Analytic Class-Incremental Learning for Long-Tailed Distributions

    Zhuang, H., Weng, Z., Wei, H., Xie, R., Toh, K.A., Lin, Z.: ACIL: Analytic class-incremental learning with absolute memorization and privacy protection. In: Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., Oh, A. (eds.) Ad- vances in Neural Information Processing Systems. vol. 35, pp. 11602–11614. Curran Associates, Inc. (2022),https://proceed...

    2022

  36. [36]

    Constructing the covariance matrixˆΣ− → C1 ≈O(D 2)

  37. [37]

    This decomposition scales cubically:C2 ≈O(D 3)

    Performing Cholesky decomposition or Eigen-decomposition to findLsuch that ˆΣ=LL T. This decomposition scales cubically:C2 ≈O(D 3)

  38. [38]

    Blind Noise

    Sampling:˜z=µ+Lϵ, ϵ∼N(0, I)− → C3 ≈O(D 2) Therefore, the overall time complexity of Gaussian sampling methods is: CGaussian ≈O(D 3) WithD≥768 1, which is computationally expensive to perform repeatedly for every tail class in every batch. 1 D= 768is the original dimension of the latent space of ViT/B-16, which we used in our experiments. In GACL and AnaCP...

  39. [39]

    8 Tran et al

    Since eG+τ Iis symmetric PSD, ∥ eG+τ I∥2 =λ max( eG+τ I) =λ max( eG) +τ. 8 Tran et al. Thus it suffices to boundλmax( eG) =λ max(G+∆). Letx∈R d with∥x∥ 2 = 1, and decompose it as x=U ha+U tb, a∈R r, b∈R d−r,∥a∥ 2 2 +∥b∥ 2 2 = 1. BecauseU h spans eigenvectors with eigenvalues in[λr, λ1]andU t spans eigen- vectors with eigenvalues in[λD, λr+1], we have the ...