Pith. sign in

REVIEW 4 major objections 5 minor 47 references

SEAL: Searching Expandable Architectures for Incremental Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SEAL searches for a starting network and a growth policy in a single NAS pass, expanding the model only when a capacity estimate says it must, and reports competitive accuracy with lower forgetting than EWC, SI, and LwF on three benchmarks.

desk verdict A neat constrained-OFA expansion idea undermined by an exogenous, hand-tuned expansion trigger that the paper never validates. read the letter →

arxiv 2505.10457 v3 pith:YQZON5GX submitted 2025-05-15 cs.LG cs.AIcs.CV

Matteo Gambella , Manuel Roveri This is my paper · ORCID
classification cs.LGcs.AIcs.CV
keywords incrementallearningneuralarchitecturesearchdynamicnetworkexpansiondata-incrementalcatastrophicforgettingcross-distillationonce-for-allsupernetmulti-objectiveoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SEAL claims that an incremental learner can be built by searching, in a single neural architecture search phase, both the initial DNN architecture and a policy for expanding it, so the model grows only when a capacity estimator says it cannot learn the new data. This matters because prior NAS-based continual learning either re-runs search at every task or expands unconditionally, which is wasteful in resource-constrained settings. SEAL targets data-incremental learning, where disjoint data samples arrive sequentially and are not stored for future access, and reports accuracy competitive with or better than EWC, SI, and LwF while lowering average forgetting and using fewer parameters. The central claim is that one search phase suffices to produce an architecture and expansion policy that generalize across future data splits.

What carries the argument

The central object is the pair consisting of a candidate architecture $\bar{x}$ and a 3-bit expansion vector $\vec{d}$, together with the synergy between an OFA-constrained search space and the relative capacity trigger of Eq. 10. The OFA supernet guarantees that every expansion is a valid subnetwork, so candidate evaluation becomes cheap fine-tuning from pretrained weights; the trigger, $1 - L(W_{t-1}, D_t)/L_{t-1} < \tau$, decides when the model grows; cross-distillation loss of Eq. 11 stabilizes the model after expansion; and the flatness metric of Eq. 14 guides the multi-objective NSGA-II search. This machinery carries the claim that the model expands only when necessary, in a single search phase.

What would settle it

Run SEAL on a data-incremental stream where a new split is nearly identical to the previous one but the learning rate is lowered so the loss drops slowly; if the trigger fires an expansion despite ample capacity, the capacity estimator is confounded by optimization dynamics. Conversely, a sweep of $\tau$ on a single dataset would reveal whether the threshold, rather than the architecture search, is responsible for the reported accuracy and forgetting values.

Watch

Extended reading notes

Core claim

On its own terms, SEAL is the first NAS framework to jointly optimize the starting architecture and the expansion policy, casting the problem as a bi-objective optimization of incremental accuracy (traded against parameter count) and a flatness-based robustness metric. The model expands only when a relative loss comparison indicates capacity saturation, the expansion operator is constrained to produce valid OFA subnetworks so newly added components initialize from pretrained weights, and cross-distillation training follows each expansion. Across CIFAR-10, CIFAR-100, and ImageNet16-120 with five data splits, SEAL reports accuracies of 95.35, 83.94, and 52.45 with average forgetting of 1.76, 4.81, and 3.23, outperforming the EWC, SI, and LwF baselines on most of these metrics while using smaller models.

Load-bearing premise

The whole 'expand only when necessary' behavior rests on the claim that the ratio $1 - L(W_{t-1}, D_t)/L_{t-1}$ falling below a threshold $\tau$ really means the model's capacity is exhausted; if the loss stagnates because of optimization noise, learning-rate effects, or task similarity instead, SEAL expands too early or too late, and the threshold must be hand-tuned per dataset (0.2 for CIFAR-10, 0.13 for the others).

Editorial extensions

If this is right

  • SEAL eliminates the cost of re-running NAS at every task, because one search phase yields both the architecture and a fixed expansion policy.
  • In data-incremental settings the model grows only when the capacity trigger fires, bounding the final parameter count relative to approaches that expand at every task.
  • Multi-objective search that includes a flatness objective yields smaller models and lower forgetting than single-objective accuracy-only search.
  • Cross-distillation after expansion reduces forgetting and improves knowledge transfer compared to training expanded models without distillation.
  • Expanding the last layers rather than early layers better preserves fine-grained features and gives higher accuracy in this data-incremental setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The relative loss trigger may conflate capacity saturation with optimization plateaus; a control experiment that varies learning rate or task similarity could separate these causes and reveal when the expansion policy is confounded.
  • The threshold $\tau$ is hand-tuned per dataset and no sensitivity analysis is reported, so learning $\tau$ inside the NAS objective, as the paper itself hints with a threshold-bit, is a natural testable extension.
  • The observed correlation between flatter minima and smaller models suggests that flatness could serve as a proxy for efficient capacity allocation in continual learning beyond this specific benchmark suite.
  • Because expansions are constrained to MobileNetV3-style OFA blocks, the 'single search phase' benefit may transfer to other weight-sharing supernets if the same joint architecture-and-policy search is applied.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. SEAL proposes a NAS framework for data-incremental learning. It uses an OFA supernet to define a constrained search space, and applies NSGA-II with two surrogate objectives to jointly select a base architecture and a 3-bit expansion vector. During incremental deployment, expansion is triggered by a capacity criterion (Eq. 10), and stability is maintained by cross-distillation training after each expansion. The method is evaluated on CIFAR-10, CIFAR-100, and ImageNet16-120 with five splits, comparing accuracy, forgetting, and forward transfer against EWC, SI, LwF, naive, and joint-training baselines.

Significance. If the central claims held, SEAL would be a useful contribution: it combines NAS with selective network expansion for incremental learning, introduces a flatness-aware objective, and the authors release their code, which aids reproducibility. The paper is clearly written and the objectives in Eqs. (12)–(14) are straightforward. However, the main advertised contribution—jointly searching an architecture and an expansion policy—is only partially realized, because the expansion trigger is a hand-tuned formula rather than a searched component. The empirical support is also mixed: SEAL performs well on CIFAR-100 and ImageNet16-120, but on CIFAR-10 its forgetting and forward transfer are worse than several baselines. These are fixable issues, so I view the paper as a candidate for major revision rather than rejection.

major comments (4)
  1. [§4.3, Eq. (10); §5.2] The expansion trigger in Eq. (10) is not learned or searched: τ is a hand-tuned, dataset-specific hyperparameter (τ=0.2 for CIFAR-10, τ=0.13 for CIFAR-100 and ImageNet16-120 in §5.2), while the NAS optimizes only the 3-bit expansion vector d (§4.4). The paper nevertheless claims in the abstract and §1 that the NAS 'jointly searches for both the architecture and the optimal expansion policy.' This is a partial claim: the most consequential decision—when to expand—is exogenous. The trigger is also not a clean capacity estimator: 1−L(W_{t−1},D_t)/L_{t−1} depends on the scale of L_{t−1}, the optimizer state, the fixed 3-epoch budget, and task similarity, so it can fire spuriously or remain silent regardless of true capacity saturation. No expansion counts, no τ-sensitivity analysis, and no ablation against always-expand or never-expand policies are provided, so the 'expand only when necessary' claim and the memory advantages in Tables 1–2 cannot be attributed to selective expansion. The Conclusion itself lists a 'capacity-aware control signal (e.g., a threshold-bit)' as future work, which corroborates this gap.
  2. [Table 2, CIFAR-10@5] The cross-dataset claim that SEAL 'effectively reduces forgetting' is contradicted by the CIFAR-10 row: SEAL's Forgetting is 1.76, worse than Naive (1.41), EWC (1.13), and SI (0.71), and its Forward Transfer is 0.18, far worse than Naive (0.68), EWC (1.33), SI (1.44), and LwF (1.22). Only on CIFAR-100 and ImageNet16-120 does SEAL achieve the best forgetting. The text acknowledges this in part, but the abstract and Section 5.2 still make unqualified statements about reducing forgetting. Please either restrict the claim to the datasets where it holds or provide a principled aggregation (e.g., average ranks with significance tests) that supports a cross-dataset conclusion.
  3. [§4.6, Eqs. (13)–(14); Table 1] The flatness objective H requires two unspecified hyperparameters, the noise intensity σ and the number of perturbations N, and neither is reported in Section 5 nor subject to sensitivity analysis. Table 1 does not support the claim of 'consistent improvements' from flatness: relative to SingleObj, MultiObj-Flat has lower ACC on CIFAR-10 (94.53 vs 94.82) and on ImageNet-16 (50.40 vs 52.31), and on CIFAR-100 the best ACC is MultiObj-Acc, not MultiObj-Flat. Please specify σ and N, add a sensitivity study, and soften the claim to what the data show.
  4. [§4.6, Eq. (12)] The accuracy objective uses a fixed size exponent w=0.07, but no justification or ablation is given for this value. Since the paper's memory-efficiency claim depends on the trade-off between accuracy and parameter count encoded by this exponent, its influence on the Pareto front and on the final selected models should be assessed.
minor comments (5)
  1. [§5.2] There is a typo: '0.13 fo the other datasets' should be '0.13 for the other datasets.'
  2. [Eq. (13)] The notation ACC_{DK}(W_K) is used without defining the subscript D_K; please clarify the indexing over datasets and tasks.
  3. [§2 and Table 2] The baseline name is spelled both LwF and LWF; please use one consistent spelling throughout.
  4. [Fig. 5 and §5.2] The discussion of expanding early versus late layers would benefit from quantitative support, such as the exact forward-transfer values and a significance test, since the text calls the early-layer result 'intriguing' but provides no numbers.
  5. [Tables 1 and 2] No pairwise significance tests or confidence intervals are reported, so several apparent differences may be within noise; adding such tests would strengthen the comparison.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SEAL's objectives are fitness functions, the expansion trigger is an exogenous hyperparameter, and the empirical comparisons are self-contained.

full rationale

The paper derives no prediction from its inputs by construction. The NAS objectives F (Eq. 12) and H (Eqs. 13-14) are multi-objective fitness functions used to rank candidate architectures; they are not claims that reduce to the baselines being compared. The 'expand only when necessary' decision is governed by Eq. 10, a user-set threshold tau (0.2 for CIFAR-10, 0.13 for the other datasets), which is chosen per dataset rather than derived; this is an external control parameter, not a circular step, though it does limit the strength of the 'jointly searched expansion policy' claim, as the authors themselves note in the conclusion by suggesting a capacity-aware threshold-bit as future work. The flatness metric is imported from prior work by the same group (FlatNAS, A2M) and from external flatness literature (Jiang et al.; Pittorino et al.), but it is used as an objective, not as a theorem that forces the experimental outcome. The evaluation against EWC, SI, and LwF is self-contained and does not depend on those cited works' conclusions. No equation in the paper is equivalent to the reported accuracy or forgetting numbers by construction, and no fitted parameter is renamed as a prediction. The main weaknesses, such as dataset-specific tau, absent sensitivity analysis, and unreported expansion counts, are evidential or scope limitations rather than circularity.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on a hand-tuned expansion threshold tau, a hand-set size penalty exponent w, and unstated flatness parameters sigma and N. These are chosen by hand, not derived from first principles, so the contribution is conditioned on their values. No new physical or conceptual entities are introduced.

free parameters (4)
  • tau (capacity threshold) = 0.20 (CIFAR-10), 0.13 (CIFAR-100, ImageNet16-120)
    Controls when the model expands (Eq 10). Tuned per dataset, no sensitivity analysis provided.
  • w (size exponent in objective F) = 0.07
    Weights the model-size penalty in Eq 12; chosen by hand without justification.
  • sigma (noise intensity in flatness metric) = not reported
    Required for the perturbation-based flatness metric in Eqs 13-14; value not given in the text.
  • N (number of perturbations in flatness metric) = not reported
    Number of sampled perturbations in Eq 14; not stated in the paper.
assumptions (3)
  • domain assumption Weights pretrained in the OFA supernet can be inherited by expanded subnetworks and fine-tuned to give reliable performance estimates.
    Used in Section 4.4 to justify fast evaluation and weight initialization of expansions; standard assumption from Cai et al. 2020.
  • ad hoc to paper The relative loss ratio 1 - L(W_{t-1}, D_t)/L_{t-1} is a reliable signal of capacity saturation, so that expanding below threshold tau improves plasticity without hurting stability.
    Introduced in Eq 10, Section 4.3; tau is dataset-specific and no theoretical derivation is provided.
  • ad hoc to paper Expanding the last layers preserves fine-grained features better than expanding early layers in data-incremental learning.
    Supported only by a single ablation (Fig. 5) on the same benchmark datasets; may not generalize outside these datasets.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SEAL: Searching Expandable Architectures for Incremental Learning." pith.science (2026). https://pith.science/paper/YQZON5GX

@misc{pith2026250510457,
  author       = {Pith},
  title        = {Pith review of: SEAL: Searching Expandable Architectures for Incremental Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YQZON5GX}},
  note         = {Machine review of arXiv:2505.10457}
}
read the original abstract

Incremental learning is a machine learning paradigm where a model learns from a sequential stream of tasks. This setting poses a key challenge: balancing plasticity (learning new tasks) and stability (preserving past knowledge). Neural Architecture Search (NAS), a branch of AutoML, automates the design of the architecture of Deep Neural Networks and has shown success in static settings. However, existing NAS-based approaches to incremental learning often rely on expanding the model at every task, making them impractical in resource-constrained environments. In this work, we introduce SEAL, a NAS-based framework tailored for data-incremental learning, a scenario where disjoint data samples arrive sequentially and are not stored for future access. SEAL adapts the model structure dynamically by expanding it only when necessary, based on a capacity estimation metric. Stability is preserved through cross-distillation training after each expansion step. The NAS component jointly searches for both the architecture and the optimal expansion policy. Experiments across multiple benchmarks demonstrate that SEAL effectively reduces forgetting and enhances accuracy while allocating additional capacity only when required. These results highlight the promise of combining NAS and selective expansion for efficient, adaptive learning in incremental scenarios.

Figures

Figures reproduced from arXiv: 2505.10457 by the authors.

Figure 1
Figure 1. SEAL diagram of the main components. The output of the NAS is a candidate set of tuples, each containing a NN [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. The model expansions performed by SEAL. Each dataset contains 60000 images split into 50000 train￾ing images and 10000 testing images. ImageNet16-120 is a downsized version of the ImageNet dataset (Deng et al. 2009) tailored for NAS evaluation, comprising 16×16 color images across 120 object classes. 5.2 Analysis of the results For all the experiments, the MobileNetV3-based (Howard 2017) supernet is used and the hyp… view at source ↗
Figure 5
Figure 5. Differently, early expansion yields a lower accuracy [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figures from the paper (3 more)
Figure 3
Figure 3. Figure 3: Performance distribution for our IL problem. Expansion starts from random weights vs OFA-initialized weights on [PITH_FULL_IMAGE:figures/full_fig_p007_3.png]
Figure 4
Figure 4. Figure 4: Performance distribution when models are trained with the distillation loss on CIFAR-10 and CIFAR-100 datasets. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Performance change when expanding the early layers vs the last layers of the model. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 28 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    An, S.; Lee, H.; Jo, J.; Lee, S.; and Hwang, S. J. 2024. DiffusionNAG: Predictor-guided Neural Architecture Generation with Diffusion Models. arXiv:2305.16943

    arXiv 2024

  4. [4]

    Cai, H.; Gan, C.; Wang, T.; Zhang, Z.; and Han, S. 2020. Once-for- All : Train One Network and Specialize it for Efficient Deployment . ArXiv:1908.09791 [cs, stat]

    arXiv 2020

  5. [5]

    K.; Ajanthan, T.; and Torr, P

    Chaudhry, A.; Dokania, P. K.; Ajanthan, T.; and Torr, P. H. S. 2018. Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence. In Proceedings of the European Conference on Computer Vision (ECCV)

    work page 2018

  6. [6]

    Chrabaszcz, P.; Loshchilov, I.; and Hutter, F. 2017. A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets. CoRR, abs/1707.08819

    arXiv 2017

  7. [7]

    Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, 248--255

    2009

  8. [8]

    Dohare, S.; Hernandez-Garcia, J.; Lan, Q.; Rahman, P.; Mahmood, A.; and Sutton, R. 2024. Loss of plasticity in deep continual learning. Nature, 632: 768--774

    work page 2024

Show all 47 references
  1. [9]

    Du, X.; Li, Z.; Sun, J.; Liu, F.; and Cao, Y. 2021. Evolutionary NAS in Light of Model Stability for Accurate Continual Learning. In 2021 International Joint Conference on Neural Networks (IJCNN), 1--8

    2021

  2. [10]

    Foret, P.; Kleiner, A.; Mobahi, H.; and Neyshabur, B. 2021. Sharpness-Aware Minimization for Efficiently Improving Generalization. arXiv:2010.01412

    2021 arXiv

  3. [11]

    Gambella, M.; Falcetta, A.; and Roveri, M. 2022. CNAS: Constrained Neural Architecture Search. In 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2918--2923

    2022

  4. [12]

    Gambella, M.; Pittorino, F.; and Roveri, M. 2024. FlatNAS: optimizing Flatness in Neural Architecture Search for Out-of-Distribution Robustness. In 2024 International Joint Conference on Neural Networks (IJCNN), 1--8

    2024

  5. [13]

    Gambella, M.; Pittorino, F.; and Roveri, M. 2025. Architecture-Aware Minimization (A ^2 M): How to Find Flat Minima in Neural Architecture Search. Machine Learning: Science and Technology

    2025

  6. [14]

    Gambella, M.; Pomponi, J.; Scardapane, S.; and Roveri, M. 2025. NACHOS: Neural Architecture Search for Hardware-Constrained Early-Exit Neural Networks. IEEE Transactions on Neural Networks and Learning Systems, 1--14

    2025

  7. [15]

    Gambella, M.; and Roveri, M. 2023. EDANAS : Adaptive Neural Architecture Search for Early Exit Neural Networks. In 2023 International Joint Conference on Neural Networks (IJCNN), 1--8. IEEE

    2023

  8. [16]

    Gao, Q.; Luo, Z.; Klabjan, D.; and Zhang, F. 2023. Efficient Architecture Search for Continual Learning. IEEE Transactions on Neural Networks and Learning Systems, 34(11): 8555--8565

    2023

  9. [17]

    Howard, A. G. 2017. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861

    2017 arXiv

  10. [18]

    Jeong, J.; Yu, J.; Park, G.; Han, D.; and Yoo, Y. 2023. GeNAS: Neural Architecture Search with Better Generalization. In Elkind, E., ed., Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23 , 911--919. International Joint Confer...

    2023

  11. [19]

    Jiang, Y.; Neyshabur, B.; Mobahi, H.; Krishnan, D.; and Bengio, S. 2019. Fantastic Generalization Measures and Where to Find Them

    2019

  12. [20]

    P.; Littman, M

    Kaelbling, L. P.; Littman, M. L.; and Moore, A. W. 1996. Reinforcement Learning: A Survey. arXiv:cs/9605103

    1996 arXiv

  13. [21]

    Kirkpatrick, J.; Pascanu, R.; Rabinowitz, N.; Veness, J.; Desjardins, G.; Rusu, A.; Milan, K.; Quan, J.; Ramalho, T.; Grabska-Barwinska, A.; Hassabis, D.; Clopath, C.; Kumaran, D.; and Hadsell, R. 2016. Overcoming catastrophic forgetting in neural networks. Proceedings of the ...

    2016

  14. [22]

    Krizhevsky, A. 2009. Learning Multiple Layers of Features from Tiny Images. Technical report

    2009

  15. [23]

    Li, X.; Zhou, Y.; Wu, T.; Socher, R.; and Xiong, C. 2019. Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting. In Chaudhuri, K.; and Salakhutdinov, R., eds., Proceedings of the 36th International Conference on Machine Learning, volume...

    2019

  16. [24]

    Li, Z.; and Hoiem, D. 2017. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12): 2935--2947

    2017

  17. [25]

    Liu, H.; Simonyan, K.; and Yang, Y. 2019. DARTS : Differentiable Architecture Search . ArXiv:1806.09055 [cs, stat]

    2019 arXiv

  18. [26]

    Lopez-Paz, D.; and Ranzato, M. 2017. Gradient episodic memory for continual learning. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, 6470–6479. Red Hook, NY, USA: Curran Associates Inc. ISBN 9781510860964

    2017

  19. [27]

    Lu, A.; Feng, T.; Yuan, H.; Song, X.; and Sun, Y. 2024. Revisiting Neural Networks for Continual Learning: An Architectural Perspective. In Proceedings of the Thirty-ThirdInternational Joint Conference on Artificial Intelligence, IJCAI-2024, 4651–4659. International Joint Conf...

    2024

  20. [28]

    Lu, A.; Ke, J.; Ding, C.; Fan, J.; and Sun, Y. 2025. Position: Continual Learning Benefits from An Evolving Population over An Unified Model. arXiv:2502.06210

    2025 arXiv

  21. [29]

    Lu, Z.; Deb, K.; Goodman, E.; Banzhaf, W.; and Boddeti, V. N. 2020. NSGANetV2 : Evolutionary Multi-Objective Surrogate-Assisted Neural Architecture Search. In European Conference on Computer Vision (ECCV)

    2020

  22. [30]

    A.; Pascanu, R.; and Dabney, W

    Lyle, C.; Zheng, Z.; Nikishin, E.; Pires, B. A.; Pascanu, R.; and Dabney, W. 2023. Understanding plasticity in neural networks. In International Conference on Machine Learning, 23190--23211. PMLR

    2023

  23. [31]

    Mok, J.; Na, B.; Choe, H.; and Yoon, S. 2021. AdvRush : Searching for Adversarially Robust Neural Architectures . ArXiv:2108.01289 [cs]

    2021 arXiv

  24. [32]

    Niu, S.; Wu, J.; Xu, G.; Zhang, Y.; Guo, Y.; Zhao, P.; Wang, P.; and Tan, M. 2021. AdaXpert: Adapting Neural Architecture for Growing Data. In Meila, M.; and Zhang, T., eds., Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Mac...

    2021

  25. [33]

    Nogales, M.; Gambella, M.; and Roveri, M. 2025. DYNAMAX: Dynamic computing for Transformers and Mamba based architectures. arXiv:2504.20922

    2025 arXiv

  26. [34]

    Pavone, M.; Lacava, A.; and Brunelli, D. 2024. TyBox: An Automatic Design and Code Generation Toolbox for Incremental On-Device TinyML Models. In Proceedings of the 2024 ACM SIGPLAN International Conference on Generative Programming: Concepts & Experiences (GPCE '24). ACM

    2024

  27. [35]

    Pittorino, F.; Lucibello, C.; Feinauer, C.; Perugini, G.; Baldassi, C.; Demyanenko, E.; and Zecchina, R. 2021. Entropic gradient descent algorithms and wide flat minima. In International Conference on Learning Representations

    2021

  28. [36]

    M.; and Lampinen, J

    Price, K.; Storn, R. M.; and Lampinen, J. A. 2005. Differential Evolution: A Practical Approach to Global Optimization. Natural Computing Series. Berlin, Heidelberg: Springer-Verlag. ISBN 3540209506

    2005

  29. [37]

    Shahawy, M.; Benkhelifa, E.; and White, D. 2024. Exploring the Intersection Between Neural Architecture Search and Continual Learning. IEEE transactions on neural networks and learning systems, PP

    2024

  30. [38]

    Shahawy, M.; Benkhelifa, E.; and White, D. 2025. Exploring the Intersection Between Neural Architecture Search and Continual Learning. IEEE Transactions on Neural Networks and Learning Systems, 36(7): 11776--11792

    2025

  31. [39]

    van de Ven, G.; Tuytelaars, T.; and Tolias, A. 2022. Three types of incremental learning. Nature Machine Intelligence, 4: 1--13

    2022

  32. [40]

    Wang, L.; Zhang, X.; Su, H.; and Zhu, J. 2024. A Comprehensive Survey of Continual Learning: Theory, Method and Application. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(8): 5362--5383

    2024

  33. [41]

    Wang, X.; Cao, S.; Li, M.; and Kitani, K. M. 2021. Neighborhood- Aware Neural Architecture Search . ArXiv:2105.06369 [cs]

    2021 arXiv

  34. [42]

    Wistuba, M.; Rawat, A.; and Pedapati, T. 2019. A Survey on Neural Architecture Search . ArXiv:1905.01392 [cs, stat]

    2019 arXiv

  35. [43]

    Xie, L.; Lomurno, E.; Gambella, M.; Ardagna, D.; Roveri, M.; Matteucci, M.; and Shi, Q. 2024. An Efficient Neural Architecture Search Model for Medical Image Classification. In Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and ...

    2024

  36. [44]

    Xie, L.; Lomurno, E.; Gambella, M.; Ardagna, D.; Roveri, M.; Matteucci, M.; and Shi, Q. 2025. ZO-DARTS++: An Efficient and Size-Variable Zeroth-Order Neural Architecture Search Algorithm. arXiv:2503.06092

    2025 arXiv

  37. [45]

    Yoone, J.; Yang, E.; Lee, J.; and Hwang, S. J. 2017. Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547

    2017 arXiv

  38. [46]

    E.; Hamidouche, W.; Herrou, G.; Menard, D.; and Debbah, M

    Zeinaty, C. E.; Hamidouche, W.; Herrou, G.; Menard, D.; and Debbah, M. 2025. Can LLMs Revolutionize the Design of Explainable and Efficient TinyML Models? arXiv:2504.09685

    2025 arXiv

  39. [47]

    Zenke, F.; Poole, B.; and Ganguli, S. 2017. Continual learning through synaptic intelligence. In International conference on machine learning, 3987--3995. PMLR

    2017

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.