Pith. sign in

REVIEW 4 major objections 5 minor 33 references

TinySubNets: An efficient and low capacity continual learning strategy

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read TinySubNets claims a forget-free continual learner can hold accuracy while using roughly a quarter of the memory that current pruning-based strategies need.

desk verdict Plausible engineering combination with credible accuracy results, but the capacity numbers that carry the central claim don't reproduce from the paper's own equations. read the letter →

arxiv 2412.10869 v3 pith:ZV55RI7W submitted 2024-12-14 cs.LG cs.AI

Marcin Pietroń , Kamil Faber , Dominik Żurek , Roberto Corizzo This is my paper · ORCID
classification cs.LGcs.AI
keywords continuallearningforget-freemethodsnetworkpruningquantizationweightsharingmodelcapacitytask-incremental
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a continual learning system can learn a long sequence of tasks in one network while using roughly a quarter to a third of the memory that leading pruning-based strategies need, without sacrificing accuracy. The proposed method, TinySubNets, combines three mechanisms: per-layer pruning with different sparsity levels, adaptive quantization that stores weights as low-bit codebook indices, and trainable masks that let similar tasks share weight values. Whether tasks share is decided by KL divergence between their replay samples; when divergence is high, a task claims its own low-bit memory bank. On Permuted MNIST, split-CIFAR100, 5 datasets, and TinyImagenet, the no-replay variant reports capacities of 22.65%, 17.62%, 24.68%, and 32.15% of the original model, against 33.7% to 188.7% for comparison methods, with accuracy within 0.5 to 2.3 points of the best baseline and highest accuracy on three scenarios. If these numbers hold, architectural continual learning becomes practical for memory-limited devices and for much longer task streams.

What carries the argument

The load-bearing mechanism is the mask tensor $M$, whose entry $M^i_{k,l}$ records the set of tasks that use weight $w^i_{k,l}$, together with the sparsity level $\Upsilon_i$ for layer $i$, which measures the fraction of weights not yet assigned to any task. Adaptive quantization maps each layer's surviving weights onto $2^\psi$ K-means centroids and stores only centroid indices plus a layer codebook, increasing bit-width $\psi$ only when accuracy drops beyond a threshold. Weight sharing is controlled by the KL divergence $D_{\mathrm{KL}}(D_t, D_p)$ between replay samples: below a threshold, the new task trains a mask over weights already used by the closest previous task; above it, the task reserves a separate memory bank with reduced bit-width. Masked gradient updates and a validation-based greedy fine-tuning pass consolidate the subnetwork and add sparsity without retraining, and masks are Huffman-coded to reduce their footprint. The reported capacity is counted as $\mathrm{CAP}_t = \sum_i (1-\Upsilon_i)|\theta_i| b + |L|2^b(32+b) + \sum_i (1-\Upsilon_i)|M_i|$, where the three terms cover pruned weights, codebook size, and compressed masks.

What would settle it

Recompute, for every method and benchmark, the real byte footprint of all saved artifacts—unpruned weights at native precision, masks, codebooks, Huffman dictionaries, and replay buffers—and compare those footprints with the paper's $\mathrm{CAP}_t$ numbers. If TSN-wr's true footprint is not roughly a third to a quarter of the baselines', the capacity claim fails even though the accuracy numbers stand.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes TinySubNets as a forget-free architectural continual learning strategy: each task is assigned a masked subnetwork, its surviving weights are quantized to codebook indices, and tasks with similar distributions are allowed to reuse previously learned weight values while dissimilar tasks receive separate low-bit memory banks. The central empirical claim is that this combination preserves or improves accuracy while using a far smaller share of the original model's capacity than its pruning-based predecessors. The no-replay variant reaches 96.63% on Permuted MNIST, 75.21% on split-CIFAR100, 91.80% on the 5 datasets scenario, and 79.81% on TinyImagenet, with reported capacities of 22.65%, 17.62%, 24.68%, and 32.15% of the original model; the replay variant records the highest accuracy on three of the four scenarios. The paper also reports zero backward transfer on every scenario, consistent with its forget-free design, and shows that a greedy post-pruning pass raises sparsity by 1 to 6 percentage points with less than 1% accuracy loss.

Load-bearing premise

The entire capacity advantage rests on the assumption that one capacity formula, developed for TinySubNets' bank-and-codebook layout, is a fair way to count memory for every compared method; the paper does not validate that formula against the actual stored artifacts of PackNet, WSN, or Ada-QPacknet, some of which are scored above 100% of the original model.

Editorial extensions

If this is right

  • A forget-free model could learn roughly three to four times more tasks within the same memory budget, postponing capacity saturation in long task sequences.
  • The no-replay variant makes architectural continual learning viable on low-memory devices, since it drops the replay buffer and still stays close to the best accuracy.
  • On homogeneous task streams, weight sharing gives most of the memory gain; on heterogeneous streams, separate banks matter more, so the divergence threshold is a practical control knob.
  • The reported zero backward transfer means each task's accuracy stays fixed after its subnetwork is consolidated, which is the property that makes the capacity savings composable across tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: the comparative capacity numbers stand or fall with the shared $\mathrm{CAP}_t$ metric; a direct artifact-footprint audit of the baselines would either confirm the three-to-fourfold claim or reduce it, independently of the accuracy results.
  • My inference: the KL threshold was set empirically to allow at most two memory banks, and the paper does not report how accuracy and capacity change between the two-bank regime and full sharing, so the method's sensitivity to this knob remains untested.
  • My inference: the TinyImagenet accuracy advantage may partly reflect the 40-task, 5-class split, which makes each per-task problem small; applying TSN to longer or class-imbalanced streams would show whether the reported gains persist.
  • My inference: if low-bit inference hardware becomes common, the FLOPs reductions the authors report for 8-bit and 4-bit weights and activations could make the capacity savings translate into energy savings in deployment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper introduces TinySubNets (TSN), a forget-free architectural continual learning method that combines per-layer adaptive pruning at different sparsity levels, adaptive non-linear quantization with codebooks, and mask-based weight sharing among tasks. Two variants are evaluated: TSN, which uses a replay buffer and KL-divergence-based sharing decisions, and TSN-wr, which drops the replay buffer and relies on pruning-based sharing. The authors claim that TSN achieves both better accuracy than existing state-of-the-art continual learning strategies and a substantially reduced model capacity (their Table 1 reports capacities around 17-32% of original for TSN-wr versus 34-99% for baselines). Experiments cover p-MNIST, split CIFAR-100, the 5-datasets scenario, TinyImageNet, and ImageNet100, with additional ablations on mask/weight/codebook memory, bit widths, and FLOPs. The central evidence for the capacity advantage is the CAP_t metric defined in the Experimental Setup.

Significance. The paper addresses a real and practically important problem: capacity saturation in forget-free architectural continual learning. The proposed combination of adaptive per-layer pruning, adaptive quantization, and weight sharing is sensible, and the empirical study is broad: five benchmarks, multiple baselines, two variants, ablations, and a public code repository. If the reported capacity numbers were reproducible, the method would offer a useful memory-accuracy trade-off. However, the central capacity metric is inconsistent with the paper's own mask-storage reporting and is not specified for non-quantized baselines; in addition, the abstract's accuracy claim is contradicted by Table 2 on the 5-datasets scenario. These issues must be resolved before the paper's main claims can be accepted.

major comments (4)
  1. [Experimental Setup (CAP_t equation); Tables 4 and 5] The capacity metric CAP_t charges the mask term Σ_i (1-Υ_i)|M_i| as exactly one bit per weight used by at least one task. This is inconsistent with Equation (4), where each mask entry M_i_{k,l} is a set of tasks, and it cannot reproduce Table 4. For p-MNIST, Table 4 reports masks at 12.5% of original capacity; with original capacity 32N bits for N weights, that is 4N bits. Using Table 5's 41.2% sparsity, the formula gives (1-0.412)N = 0.588N bits, about 1.8% of original capacity, roughly 6.8x less than the reported 12.5%. A weight shared by K tasks must be stored in K task masks (or in a set-coded representation with more than one bit), so the one-bit-per-used-weight term undercounts TSN's storage. While Huffman compression is mentioned, the CAP_t equation does not model it. Since Table 1 and the 'significantly improved capacity exploitation' claim rest entirely on this metric, the central quantitative claim is not reproducible from the equations in the paper.
  2. [Abstract/Introduction; Comparative Studies, Table 2] The claim that TSN 'achieves better results in terms of accuracy than existing state-of-the-art CL strategies' is contradicted by the paper's own Table 2. On the 5-datasets scenario, Ada-QPacknet reaches 94.1% while TSN-wr reaches 91.8% and TSN reaches 93.76%; on s-CIFAR100, WSN reaches 76.38% while TSN-wr reaches 75.21%. The text in the 'Impact of Weight Sharing' paragraph even states that 'strategies without weight sharing achieve the best accuracy (94.1%)'. The accuracy claim should be restricted to specific scenarios or rephrased as competitive accuracy with lower capacity.
  3. [Table 1; Metrics paragraph] Table 1 reports capacities above 100% for baselines (PackNet 188.67% on TinyImageNet, Ada-QPacknet 112.5%). If CAP_t is a percentage of the original model's capacity, this needs explanation. More importantly, the CAP_t formula includes a codebook term |L|·2^b·(32+b) and a bit-width b, but the paper does not state how b and the codebook term are set for methods that do not use quantization (PackNet, WSN) or that use a different quantization scheme (Ada-QPacknet). Without this specification the cross-method comparison in Table 1 is not reproducible, and the claimed 3-4x capacity reduction over WSN and Ada-QPacknet is not established.
  4. [Appendix, Formal details (KL divergence equation); Algorithm 1] The displayed formula for D_KL in the Formal details appendix, D_KL(D_t||D_{t-1}) = P(D_t)·P(D_t)/Q(D_{t-1}), is not the Kullback-Leibler divergence: it is missing the logarithm and the expectation over the reference distribution. Since Algorithm 1 uses this quantity to decide whether to share weights with a previous task, the method as written is not implementable. Please correct the formula or state explicitly which divergence is actually computed.
minor comments (5)
  1. [Hyperparameters] The bullet list states 'Fine tuning parameters - 50 iterations for each scenario, α - 0.95, β - 0.95', while the Algorithm 2 description says 'α is set to 0.95 and β to 0.05'; please reconcile this contradiction.
  2. [Table 1 footnote] The footnote '* - in case of p-MNIST two memory banks without replay memory' is unclear, since Table 1's TSN row is presented elsewhere as the replay variant; please rewrite the footnote to state exactly what is being reported.
  3. [Metrics paragraph] The CAP_t equation is not numbered; please number it so that the capacity definition can be referenced precisely.
  4. [Energy Efficiency section] The text says 'FLOPS calculations for GPU is carried out via arithmetic multiplication based on the size...'; the quantity computed is FLOPs (operation count), not FLOPS (operations per second), and the sentence should be corrected.
  5. [Appendix, Huffman encoding] The sentence 'the most frequent sub-sequences with their probabilities are are extracted' contains a duplicated 'are'; please fix the typo.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: TSN's accuracy and capacity results are experimental measurements with external benchmarks; the central claims do not reduce to the paper's own definitions or fits.

full rationale

This paper is an experimental systems paper rather than a derivation. Accuracy is measured directly on standard continual-learning benchmarks (p-MNIST, s-CIFAR100, 5 datasets, TinyImagenet, ImageNet100) against external baselines, and the accuracy numbers cannot be recovered by construction from the method's definitions. The capacity claim is computed from the explicitly defined metric CAP_t, but CAP_t is a measurement convention: sparsity, bit-width, and masks are outputs of the training process and are then counted. Hyperparameters such as initial capacity per task and the KL-divergence threshold influence the reported capacity, but they are tuning choices and do not make the capacity outcome equal to the inputs of the algorithm. Self-citations to the authors' prior Ada-QPacknet work and to Pietron et al. 2019 appear as a baseline and as inspiration for the K-means quantization step, but the load-bearing evidence is the external benchmark comparison; no uniqueness theorem or author-imported constraint is used to force the method. The paper itself states foreseeable limitations (task divergence leading to memory saturation) in the Open Challenges section. A separate reproducibility concern exists: the mask term in CAP_t does not obviously match the mask percentages in Table 4, and Table 1 reports capacities above 100% for some baselines. That is a correctness or accounting issue, not circularity, because the claims do not reduce to their own definitions by construction. Therefore no significant circularity is found.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central claim rests on experimental validation rather than derivation. The load-bearing free parameters are the KL threshold, initial capacity, adaptive bit-width tolerance, post-pruning coefficients, and replay size. Three domain assumptions are needed: task identity at inference, KL-on-replay as a sharing signal, and safety of greedy validation-based pruning. No physical or quasi-physical entities are invented.

free parameters (5)
  • KL divergence threshold omega = set empirically to have max two memory banks
    Controls whether a new task shares weights with an old task or opens a separate memory bank; directly determines the capacity-accuracy tradeoff reported in Tables 1 and 2.
  • Initial capacity per task c = 0.55 for CIFAR100, 0.5 for other scenarios
    Sets the size of the subnetwork mask allocated to each task and therefore affects both accuracy and reported capacity.
  • Quantization bit-width psi and accuracy drop threshold delta = resulting bit-width 4 or 5 bits depending on scenario
    Adaptive quantization in Appendix Algorithm 3 increases bit-width until accuracy loss falls below delta; the final bit-widths are scenario-specific and materially change capacity.
  • Fine-tuning iterations I and fitness coefficients alpha, beta = 50 iterations; alpha=0.95, beta=0.05 in Algorithm 2 text, but 0.95/0.95 in hyperparameter list
    Greedy post-pruning uses these values to trade accuracy against sparsity; the inconsistency itself undermines reproducibility.
  • Task replay memory size s_r = 50 samples per task, 15 for TinyImagenet
    Replay samples are used for KL divergence and for the TSN variant with replay; the size affects both task-similarity estimates and memory overhead.
assumptions (3)
  • domain assumption Task identity is available at inference time for mask and codebook selection
    The method requires knowing which task is being evaluated; the Appendix admits 'our method requires knowledge of the task identifier to create masks and the specific sub-network for each task'.
  • domain assumption KL divergence between replay samples from two tasks reliably predicts whether weight sharing will preserve accuracy
    Algorithm 1 lines 6-9 make the entire sharing decision from D_KL computed on small replay sets; the paper provides no evidence that this proxy predicts sharing benefit.
  • domain assumption Greedy validation-based sparsity increase in Algorithm 2 does not overfit the validation set
    The authors assert this, but there is no analysis of the number of validation samples or the multiplicity of comparisons made across 50 fine-tuning iterations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TinySubNets: An efficient and low capacity continual learning strategy." pith.science (2026). https://pith.science/paper/ZV55RI7W

@misc{pith2026241210869,
  author       = {Pith},
  title        = {Pith review of: TinySubNets: An efficient and low capacity continual learning strategy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZV55RI7W}},
  note         = {Machine review of arXiv:2412.10869}
}
read the original abstract

Continual Learning (CL) is a highly relevant setting gaining traction in recent machine learning research. Among CL works, architectural and hybrid strategies are particularly effective due to their potential to adapt the model architecture as new tasks are presented. However, many existing solutions do not efficiently exploit model sparsity, and are prone to capacity saturation due to their inefficient use of available weights, which limits the number of learnable tasks. In this paper, we propose TinySubNets (TSN), a novel architectural CL strategy that addresses the issues through the unique combination of pruning with different sparsity levels, adaptive quantization, and weight sharing. Pruning identifies a subset of weights that preserve model performance, making less relevant weights available for future tasks. Adaptive quantization allows a single weight to be separated into multiple parts which can be assigned to different tasks. Weight sharing between tasks boosts the exploitation of capacity and task similarity, allowing for the identification of a better trade-off between model accuracy and capacity. These features allow TSN to efficiently leverage the available capacity, enhance knowledge transfer, and reduce computational resource consumption. Experimental results involving common benchmark CL datasets and scenarios show that our proposed strategy achieves better results in terms of accuracy than existing state-of-the-art CL strategies. Moreover, our strategy is shown to provide a significantly improved model capacity exploitation. Code released at: https://github.com/lifelonglab/tinysubnets.

Figures

Figures reproduced from arXiv: 2412.10869 by the authors.

Figure 1
Figure 1. Model architecture evolution with TinySubNetworks (TSN). To incorporate a new task, the model-agnostic continual [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. TinySubNetworks (TSN) - scenario with physical connection sharing with disjoint values using a reduced bit-width [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. TinySubNetworks - mask with connection sharing [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: FLOPS with different quantization levels in differ [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Accuracy matrix for p-MNIST (TSN-wr) 0 1 2 3 4 5 6 7 8 9 Evaluating on Task 0 1 2 3 4 5 6 7 8 9 After learning Task 72.1 72.1 67.1 72.1 67.1 74.2 72.1 67.1 74.2 71.8 72.1 67.1 74.2 71.8 74.2 72.1 67.1 74.2 71.8 74.2 73.4 72.1 67.1 74.2 71.8 74.2 73.4 74.1 72.1 67.1 74.…
Figure 6
Figure 6. Figure 6: Accuracy matrix for CIFAR100 (TSN-wr) [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Accuracy matrix for Imagenet-100 (TSN-wr) [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Accuracy matrix for 5 datasets (TSN-wr) [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Accuracy matrix for TinyImagenet (TSN-wr) [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 18 canonical work pages

  1. [1]

    Ameya Prabhu, P. H. S. T.; and Dokania, P. K. 2020. GDumb: A Simple Approach that Questions Our Progress in Continual Learning. Lecture Notes in Computer Science (LNIP), 12347

    work page 2020

  2. [2]

    M.; New, A.; Aguilar-Simon, M.; Al-Halah, Z.; Arnold, S

    Baker, M. M.; New, A.; Aguilar-Simon, M.; Al-Halah, Z.; Arnold, S. M.; Ben-Iwhiwhu, E.; Brna, A. P.; Brooks, E.; Brown, R. C.; Daniels, Z.; Daram, A.; Delattre, F.; Dellana, R.; Eaton, E.; Fu, H.; Grauman, K.; Hostetler, J.; Iqbal, S.; Kent, C.; Ketz, N.; and et al., S. K. 2023. A domain-agnostic approach for characterization of lifelong learning systems....

    work page 2023

  3. [3]

    Buzzega, P.; Boschini, M.; Porrello, A.; Abati, D.; and Calderara, S. 2020. Dark Experience for General Continual Learning: a Strong, Simple Baseline. In Proceedings of 34th Conference on Neural Information Processing Systems (NeurIPS 2020)

    work page 2020

  4. [4]

    Chaudhry, A.; Ranzato, M.; Rohrbach, M.; and Elhoseiny, M. 2019. Efficient Lifelong Learning with A-GEM. Salk Institute for Biological Studies, arXiv:1812.00420

    arXiv 2019

  5. [5]

    Cun, Y. L. 1998. The MNIST database of handwritten digits. http://yann. lecun. com/exdb/mnist/

    work page 1998

  6. [6]

    David Lopez-Paz, M. R. 2017. Gradient Episodic Memory for Continual Learning. arXiv, https://arxiv.org/abs/1706.08840

    arXiv 2017

  7. [7]

    D \' az-Rodr \' guez, N.; Lomonaco, V.; Filliat, D.; and Maltoni, D. 2018. Don't forget, there is more than forgetting: new metrics for Continual Learning. arXiv preprint arXiv:1810.13166

    arXiv 2018

  8. [8]

    Douillard, A.; Ram \'e , A.; Couairon, G.; and Cord, M. 2022. Dytox: Transformers for continual learning with dynamic token expansion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9285--9295

    2022

Show all 33 references
  1. [9]

    Ebrahimi, S.; Meier, F.; Calandra, R.; Darrell, T.; and Rohrbach, M. 2020. Adversarial continual learning. In European Conference on Computer Vision, 386--402. Springer

    2020

  2. [10]

    Faber, K.; Corizzo, R.; Sniezynski, B.; and Japkowicz, N. 2023. VLAD: Task-agnostic VAE-based lifelong anomaly detection. Neural Networks, 165: 248--273

    2023

  3. [11]

    Jha, S.; Gong, D.; Zhao, H.; and Yao, L. 2024. NPCL: Neural Processes for Uncertainty-Aware Continual Learning. Advances in Neural Information Processing Systems, 36

    2024

  4. [12]

    Kang, H.; Mina, R. J. L.; Rizky, S.; Madjid, H.; Yoon, J.; Hasegawa-Johnson, M.; Ju-Hwang, S.; and Yoo, C. D. 2022. Forget-free Continual Learning with Winning Subnetworks. ICML, x

    2022

  5. [13]

    Kirkpatrick, J.; Pascanu, R.; Rabinowitz, N.; Veness, J.; Desjardins, G.; Rusu, A.; Milan, K.; Quan, J.; Ramalho, T.; Grabska-Barwinska, A.; Hassabis, D.; Clopath, C.; Kumaran, D.; and Hadsell, R. 2016. Overcoming catastrophic forgetting in neural networks. arXiv, https://arxi...

    2016 arXiv

  6. [14]

    Krizhevsky, A. 2009. Learning Multiple Layers of Features from Tiny Images

    2009

  7. [15]

    Le, Y.; and Yang, X. 2015. Tiny imagenet visual recognition challenge. Stanford Computer Vision Lab

    2015

  8. [16]

    Li, Y.; Yang, X.; Wang, H.; Wang, X.; and Li, T. 2024. Learning to Prompt Knowledge Transfer for Open-World Continual Learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 13700--13708

    2024

  9. [17]

    Li, Z.; and Hoiem, D. 2017. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12): 2935--2947

    2017

  10. [18]

    Liang, Y.-S.; and Li, W.-J. 2024. Loss decoupling for task-agnostic continual learning. Advances in Neural Information Processing Systems, 36

    2024

  11. [19]

    Lomonaco, V.; Maltoni, D.; and Pellegrini, L. 2019. Rehearsal-Free Continual Learning over Small Non-I.I.D. Batches. 1st Workshop on Continual Learning in Computer Vision at CVPR2020

    2019

  12. [20]

    Madaan, D.; Yin, H.; Byeon, W.; Kautz, J.; and Molchanov, P. 2023. Heterogeneous continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15985--15995

    2023

  13. [21]

    Mallya, A.; and Lazebnik, S. 2017. PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning. arXiv, https://arxiv.org/abs/1711.05769

    2017 arXiv

  14. [22]

    D.; Gong, D.; Parvaneh, A.; Abbasnejad, E.; and van den Hengel, A

    McDonnell, M. D.; Gong, D.; Parvaneh, A.; Abbasnejad, E.; and van den Hengel, A. 2024. Ranpac: Random projections and pre-trained models for continual learning. Advances in Neural Information Processing Systems, 36

    2024

  15. [23]

    I.; Kemker, R.; Part, J

    Parisi, G. I.; Kemker, R.; Part, J. L.; Kanan, C.; and Wermter, S. 2019. Continual lifelong learning with neural networks: A review. Neural networks, 113: 54--71

    2019

  16. [24]

    Pietron, M.; Karwatowski, M.; Wielgosz, M.; and Duda, J. 2019. Fast Compression and Optimization of Deep Learning Models for Natural Language Processing. 162--168

    2019

  17. [25]

    Pietron, M.; Zurek, D.; Faber, K.; and Corizzo, R. 2023. Ada-QPacknet -- Multi-Task Forget-Free Continual Learning with Quantization Driven Adaptive Pruning. In 26th European Conference on Artificial Intelligence, ECAI 2023, 1882--1889

    2023

  18. [26]

    Russakovsky, O.; Deng, J.; Su, H.; Krause, J.; Satheesh, S.; Ma, S.; Huang, Z.; Karpathy, A.; Khosla, A.; Bernstein, M.; et al. 2015. Imagenet large scale visual recognition challenge. International journal of computer vision, 115: 211--252

    2015

  19. [27]

    Saha, G.; Garg, I.; and Roy, K. 2020. Gradient Projection Memory for Continual Learning. In International Conference on Learning Representations

    2020

  20. [28]

    Saha, G.; and Roy, K. 2023. Continual learning with scaled gradient projection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 9677--9685

    2023

  21. [29]

    A.; and Etzioni, O

    Schwartz, R.; Dodge, J.; Smith, N. A.; and Etzioni, O. 2020. Green ai. Communications of the ACM, 63(12): 54--63

    2020

  22. [30]

    Wortsman, M.; Ramanujan, V.; Liu, R.; Kembhavi, A.; Rastegari, M.; Yosinski, J.; and Farhadi, A. 2020. Supermasks in Superposition. arXiv, https://arxiv.org/abs/2006.14769

    2020 arXiv

  23. [31]

    Zhang, J

    Xu, K.; D. Zhang, J. A.; Liu, L.; Liu, L.; and Wang, D. 2021. GenExp: Multi-objective pruning for deep neural network based on genetic algorithm. Neurocomputing, April

    2021

  24. [32]

    Zenke, F.; Poole, B.; and Ganguli, S. 2017. Continual Learning Through Synaptic Intelligence. arXiv, https://arxiv.org/abs/1703.04200

    2017 arXiv

  25. [33]

    Zhou, D.-W.; Wang, Q.-W.; Qi, Z.-H.; Ye, H.-J.; Zhan, D.-C.; and Liu, Z. 2023. Deep class-incremental learning: A survey. arXiv preprint arXiv:2302.03648

    2023 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.