Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Personalized Artificial General Intelligence (AGI) via Neuroscience-Inspired Continuous Learning Systems

T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that a tri-memory, neuroscience-inspired architecture incorporating fast Hebbian adaptation, slow consolidated memory, and protected permanent memory can make lifelong personalized AGI feasible on edge devices.

desk verdict A competent, honest survey and design sketch that repackages known continual-learning ideas into a Tri-Memory architecture, but the central capability claim is asserted rather than demonstrated and the microsleep decay mechanism has a real analytical hole. read the letter →

arxiv 2504.20109 v1 pith:NWZYT5L3 submitted 2025-04-27 cs.AI cs.LG

Rajeev Gupta , Suhani Gupta , Ronak Parikh , Divya Gupta , Amir Javaheri , Jairaj Singh Shaktawat This is my paper · ORCID
classification cs.AIcs.LG
keywords PersonalizedAGIEnergy-EfficientAISystemsSynapticPruningHebbianPlasticitySparseCodingDualMemoryContinualLearningonEdge
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

True AGI, the paper argues, will not come from scaling static deep models: it needs systems that learn continuously, personalize to a user, and run within the tight compute and memory budget of edge devices. To get there, the paper proposes a biologically inspired architecture built around three memory tiers (short-term, long-term, and permanent), Hebbian-style local updates for fast adaptation, sparse coding and expert gating for efficiency, and periodic 'microsleep' decay plus nightly pruning-and-replay sessions for consolidation. The central claim is that this combination can manage catastrophic forgetting and keep model size bounded, making lifelong personalized learning on devices like phones and humanoid robots feasible. The paper is a theoretical roadmap: it reports no experiments and positions itself as a synthesis of existing continual learning and neuroscience results to be validated in future work.

What carries the argument

The load-bearing object is the Tri-Memory Continual Learning system: three parameter tiers (STM, LTM, PM) operating on different timescales, governed by a consolidation module. Its distinctive operations are the microsleep global offset (a uniform negative shift applied to all weights to simulate decay), usage counters that drive adaptive pruning during nightly offline sessions, replay-based rehearsal for consolidation, and hybrid learning that combines Hebbian increments with error-driven gradient updates. The architecture also relies on sparse expert gating so that each input activates only a subset of parameters, which is what keeps the growing knowledge base within edge-device memory. The mechanism as a whole is meant to resolve the stability-plasticity dilemma by routing fast changes through STM and promoting only vetted knowledge into LTM and PM.

What would settle it

Implement the microsleep global offset and usage-counter pruning in a small network (for example, on a Split-MNIST or permuted-task benchmark) and measure, weight by weight, whether rarely used connections decay toward zero faster than frequently used ones. If the offset decays all weights at the same rate and nothing else differentiates them, the claimed selective preservation fails; a second check is whether disabling the offset changes final accuracy or memory footprint.

Watch

Extended reading notes

Core claim

The paper's discovery is a design, not an empirical result: the Tri-Memory Continual Learning architecture. It partitions model parameters into a short-term memory that adapts rapidly through Hebbian-like increments, a long-term memory that stabilizes repeatedly useful patterns through consolidation and replay, and a permanent memory that shields mission-critical weights from pruning and decay. Lightweight 'microsleep' intervals apply a uniform negative offset to all weights to simulate synaptic decay, while longer offline sessions perform usage-counter-driven pruning and replay-based rehearsal. Sparse distributed representation, implemented via expert gating, keeps only relevant subnetworks active for any input. The paper claims these mechanisms together offer a practical path to continuous, personalized learning on resource-constrained edge devices, with catastrophic forgetting managed rather than eliminated.

Load-bearing premise

The architecture's distinctive decay mechanism assumes that a single uniform negative offset applied to all weights, in combination with Hebbian increments and usage-based thresholds, selectively lets rarely used weights fade while frequently used ones stay strong; since the offset itself carries no usage information, the whole selective-decay story depends on interactions that are asserted but never quantified.

Editorial extensions

If this is right

  • If the architecture works as claimed, a phone or robot can keep learning new tasks and user habits after deployment, without a cloud-retraining loop.
  • Catastrophic forgetting would not be eliminated but managed: consolidation, replay, and pruning trade a controlled loss of rarely used knowledge for stable retention of core skills.
  • Model size stays bounded because nightly pruning and sparse expert activation reclaim capacity, avoiding the unbounded growth typical of progressive-network approaches.
  • On-device training keeps personal data local, making privacy-by-design a built-in property rather than an add-on.
  • The framework gives a concrete research program: each component already has a prototype in prior work, so the integration can be built and benchmarked incrementally.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The uniform global offset is the point to scrutinize first: because a constant negative shift carries no usage information, the selectivity of decay must come entirely from the separate Hebbian increments and from thresholds that the paper leaves unspecified; a small simulation could test whether rare weights actually fade faster than frequent ones.
  • The sentiment-modulated consolidation idea, if developed, implies a broader principle the paper only gestures at: forgetting policy should be driven by the value of an interaction, not just by frequency or recency.
  • The on-demand downloading of specialized models suggests an edge-cloud division of labor where devices fetch knowledge without uploading raw user data; a testable extension would measure whether such acquisitions integrate into LTM without displacing existing skills.
  • The tri-memory split plus microsleeps could be evaluated before full AGI ambitions: implement STM-LTM with replay and pruning on standard continual learning benchmarks and compare forgetting and memory footprint against EWC, replay, and PackNet baselines.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper is a conceptual proposal for a 'Tri-Memory Continual Learning' architecture that combines short-term, long-term, and permanent memory modules with Hebbian updates, microsleep-based global offset decay, nightly pruning, and replay-based consolidation, aimed at enabling lifelong, personalized learning on edge devices. The manuscript consists of a broad literature review of continual learning and neuroscience-inspired AI, a qualitative description of the proposed architecture (Sections 4.1–4.7), a conceptual evaluation (Section 5) that explicitly contains no empirical results, and a discussion of applications and open questions. The central claim, stated in the abstract and Section 1, is that this architecture enables lifelong, personalized learning on resource-constrained edge devices.

Significance. If the proposed architecture worked as described, it would provide a useful integrative roadmap for on-device continual learning, combining several established techniques (replay, pruning, complementary learning systems, sparse coding) into a single hardware-aware design. The literature survey is broad and generally accurate, and the paper candidly identifies the lack of empirical validation. However, as it stands, the manuscript does not demonstrate that the architecture achieves its central claim: no experiments, simulations, or formal analysis are provided, and the most distinctive mechanism—selective synaptic decay via a uniform global offset—is not shown to have the claimed selectivity. The contribution is therefore a speculative design proposal rather than a validated system, and the strength of the claims in the abstract and Section 1 exceeds what the body supports.

major comments (4)
  1. [Section 4.3] The global offset mechanism is not demonstrated to be selective. A uniform negative shift applied to every weight carries no usage information: by itself it decays all weights equally. The intended selectivity must come from Hebbian increments (Section 4.5) being large enough to offset decay for frequently used weights and too small to rescue rarely used ones, but the paper never specifies the increment size, the offset magnitude, the microsleep frequency, or the relation between these quantities and usage statistics. Without such a specification, the claim that globally offset weights 'nudge the network toward a stable equilibrium in which frequently used weights remain sizable, and rarely used weights decay toward negligible values' is unsupported and, as written, not internally justified.
  2. [Section 4.7] The assertion that 'all weights remain non-negative in this architecture (thanks in part to ReLU-like activation functions)' is not established. ReLU constrains activations, not weights, and the paper provides no mechanism that would enforce weight non-negativity under Hebbian updates (Δw = η·x·y in Section 3.2) or gradient-based updates. This matters because Section 4.1 says weights dropping below zero become inactive: if weights can be negative, that rule is ill-defined, and if weights are constrained to be non-negative, the necessary projection or clipping is never described. The pruning and decay mechanisms therefore rest on an unstated and questionable premise.
  3. [Section 5] The paper's central claim is asserted, not demonstrated. Section 5 opens with 'While no empirical results are presented,' and the entire evaluation is qualitative: Table 1 compares methods on subjective categories, and Sections 5.2–5.3 present application scenarios as 'design validation' without quantitative evidence. There is no formal analysis of forgetting bounds, memory growth, or computational cost, and there is no simulation showing that the tri-memory system, microsleep decay, and nightly pruning actually mitigate catastrophic forgetting on edge-class hardware. As a result, the abstract's claim that the architecture 'enables lifelong, personalized learning on edge devices' is not supported by the manuscript's content.
  4. [Section 4.6 vs. Section 4.1] The operational timeline is internally inconsistent with the earlier description of microsleeps. Section 4.1 explicitly states that microsleeps 'do not include replay-based rehearsals or actual large-scale pruning,' yet Section 4.6, step 4 ('Micro-Rehearsal'), says that during this phase the system 'momentarily revisits a few examples in its replay buffer' and performs 'a short backpropagation-based update.' This contradiction affects the architecture's definition: the lightweight-decay versus consolidation distinction is central to the claimed efficiency, and the paper must specify which operations occur during microsleeps and which occur only in the nightly offline window.
minor comments (4)
  1. [Table 1] The conceptual comparison table uses subjective ratings ('High', 'Moderate', 'Low') without any accompanying definitions or quantitative basis, which limits its usefulness as an evaluation of the proposed architecture.
  2. [Section 5.3] The application scenarios (personal humanoid, healthcare IoT, smartphone assistant) are plausible but are described as 'design validation' without any implemented prototype or reproducible procedure, so they do not provide evidence for the architecture's viability.
  3. [General presentation] Several references are incomplete or informal (e.g., reference [27] is a general health website and reference [26] is not fully specified), and the figures are conceptual illustrations without captions that explain the mechanisms they are intended to convey.
  4. [Section 4.6] The timeline lists 'selectively deactivates weights falling below the pruning threshold' during microsleeps, but the text in Sections 4.1 and 4.3 says formal pruning is deferred to offline sessions; the relationship between deactivation during microsleeps and offline pruning should be clarified.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is a conceptual architecture proposal with no fitted parameters, no empirical predictions, and no load-bearing self-citation chain.

full rationale

This paper does not contain a derivation chain of the kind that can be circular: it fits no parameters, reports no measurements, and makes no quantitative prediction. It explicitly states in Section 5 that 'no empirical results are presented,' and the architecture is offered as a theoretical framework and roadmap. The claimed benefits, such as mitigating catastrophic forgetting through the Tri-Memory System, are qualitative design rationales rather than results derived from equations or fitted data. The microsleep global-offset mechanism is asserted to produce selective decay, but the paper does not derive that behavior from the offset alone; it depends on unspecified Hebbian increments and thresholds. That is an absence of formal support, not a circular reduction: the paper never defines the equilibrium in terms of the offset itself, never fits the offset to data, and never renames a fitted quantity as a prediction. Likewise, the statement that the framework 'addresses the stability–plasticity dilemma by design' (Section 5.2) is a design rationale, not a theorem forced by a prior self-citation. The paper's cited support for continual learning, replay, pruning, dual-memory, and mixture-of-experts components comes from external literature with independent content. The Section 4.7 claim that ReLU-like activations imply non-negative weights is a correctness concern, but it does not make any claimed result equivalent to its own inputs. Under the required evidentiary standard, no specific circular step can be exhibited.

Assumptions & free parameters 5 free parameters · 5 assumptions · 2 invented entities

The central claim rests on a large set of unquantified mechanisms and assumptions. The paper introduces two novel entities (PM, microsleep offset) and depends on several domain assumptions about transferring biology to AI, plus unspecified parameters that would need to be tuned. None are validated.

free parameters (5)
  • Global offset magnitude = not specified
    The size of the uniform negative weight shift in microsleeps (Section 4.3) determines decay rate; no value or schedule is given.
  • Pruning threshold = adaptive, unspecified
    Section 4.4 says the threshold is 'adaptively determined' from usage statistics but gives no algorithm.
  • Consolidation threshold = not specified
    Section 4.2 says STM to LTM consolidation occurs 'when STM's accumulated changes reach a threshold or at scheduled intervals'; no criterion is defined.
  • Replay buffer size = not specified
    Section 4.4 mentions a 'small replay buffer' without size or sampling strategy.
  • Hebbian learning rate eta = not specified
    Equation in Section 3.2 defines delta w = eta*x*y but eta is unstated.
assumptions (5)
  • domain assumption Neuroscience principles (synaptic pruning, Hebbian plasticity, sparse coding, dual memory) transfer beneficially to artificial neural networks.
    Section 3 assumes these biological mechanisms yield the same efficiency and anti-interference benefits in AI as in brains.
  • ad hoc to paper A uniform global offset acts as synaptic decay that preserves frequently used weights and degrades rarely used ones.
    Section 4.3 claims this without modeling how importance is encoded in the offset operation.
  • ad hoc to paper All weights remain non-negative, making pruning thresholds trivial.
    Section 4.7 asserts non-negativity 'thanks in part to ReLU-like activation functions' without explaining how negative weights, common in trained networks, are avoided.
  • domain assumption Replay, regularization, and pruning are sufficient, when combined, to prevent catastrophic forgetting.
    Sections 4.4 and 6.1 assert this combination creates a 'robust memory stability net' with no supporting measurements.
  • domain assumption Edge devices provide sparse matrix acceleration and low-power compute.
    Section 4.7 assumes hardware such as NVIDIA Jetson supports efficient sparse operations for the pruned network.
invented entities (2)
  • Permanent Memory (PM)
    purpose: A protected parameter tier that stores core competencies and resists pruning and decay.
    Defined in Section 4.2 as an extension of the dual-memory system; no empirical or theoretical evidence establishes it as necessary or effective.
  • Microsleep global offset
    purpose: A uniform weight decay operation run in short idle periods to simulate synaptic decay.
    Proposed in Section 4.3; its claimed selective effect is not demonstrated in any experiment.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Personalized Artificial General Intelligence (AGI) via Neuroscience-Inspired Continuous Learning Systems." pith.science (2026). https://pith.science/paper/NWZYT5L3

@misc{pith2026250420109,
  author       = {Pith},
  title        = {Pith review of: Personalized Artificial General Intelligence (AGI) via Neuroscience-Inspired Continuous Learning Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NWZYT5L3}},
  note         = {Machine review of arXiv:2504.20109}
}
read the original abstract

Artificial Intelligence has made remarkable advancements in recent years, primarily driven by increasingly large deep learning models. However, achieving true Artificial General Intelligence (AGI) demands fundamentally new architectures rather than merely scaling up existing models. Current approaches largely depend on expanding model parameters, which improves task-specific performance but falls short in enabling continuous, adaptable, and generalized learning. Achieving AGI capable of continuous learning and personalization on resource-constrained edge devices is an even bigger challenge. This paper reviews the state of continual learning and neuroscience-inspired AI, and proposes a novel architecture for Personalized AGI that integrates brain-like learning mechanisms for edge deployment. We review literature on continuous lifelong learning, catastrophic forgetting, and edge AI, and discuss key neuroscience principles of human learning, including Synaptic Pruning, Hebbian plasticity, Sparse Coding, and Dual Memory Systems, as inspirations for AI systems. Building on these insights, we outline an AI architecture that features complementary fast-and-slow learning modules, synaptic self-optimization, and memory-efficient model updates to support on-device lifelong adaptation. Conceptual diagrams of the proposed architecture and learning processes are provided. We address challenges such as catastrophic forgetting, memory efficiency, and system scalability, and present application scenarios for mobile AI assistants and embodied AI systems like humanoid robots. We conclude with key takeaways and future research directions toward truly continual, personalized AGI on the edge. While the architecture is theoretical, it synthesizes diverse findings and offers a roadmap for future implementation.

Figures

Figures reproduced from arXiv: 2504.20109 by the authors.

Figure 1
Figure 1. Illustration of synaptic pruning in the biological brain. [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Synaptic pruning applied to AI. As noted earlier, Golkar et al. [23] demonstrated that allowing a small amount of controlled forgetting of old tasks (through pruning) can prevent much larger uncontrolled forgetting during future training. Essentially, by pruning weights that encoded some old task details, the model accepts a minor performance hit on that task but gains flexibility to master new tasks without overlap… view at source ↗
Figure 3
Figure 3. Donald Hebb’s Principle. In biological neurons, repeated simultaneous activation can lead to in￾creases in synaptic weight (long-term potentiation), whereas lack of coor￾dinated activity can lead to weakening (long-term depression). Variants of 11 [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Spike-Timing-Dependent Plasticity (STDP). [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Hebbian Plasticity and Rapid Personalization. [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Sparse Coding improves efficiency, reduces overlap [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Mixture-of-Experts Model [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Fast and Slow Learning - hippocampus vs neocortex [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Short term and long-term memory in AI The Titans architecture [19] offers a more structural approach, in which the attention mechanism acts as short-term contextual memory, while the long-term memory module serves as durable storage of knowledge that can be queried. An…
Figure 10
Figure 10. Figure 10: Tri-Memory System: STM, LTM, and PM memory modules with dynamic [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]
Figure 11
Figure 11. Figure 11: Sparse distributed representation This echoes the MoE idea from GLaM [24], but adapted for resource￾constrained edge devices. The gating could use context features to choose which expert to use for a given input, ensuring modularity and preserving previous capabilitie…
Figure 12
Figure 12. Figure 12: Microsleeps for Lightweight Decay: Global Offset and Minor Gradient Steps [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: Nightly (Offline) Pruning To handle more intensive tasks such as synaptic pruning and catastrophic forgetting avoidance, we reserve offline sessions, often referred to in neu￾roscience as analogs of “sleep-based consolidation.” During these extended sessions, the robo…
Figure 14
Figure 14. Figure 14: Adaptive Pruning and Replay-Based Rehearsal [PITH_FULL_IMAGE:figures/full_fig_p024_14.png]
Figure 15
Figure 15. Figure 15: Hybrid learning with Hebbian and gradient-based updates occurring at different [PITH_FULL_IMAGE:figures/full_fig_p026_15.png]
Figure 16
Figure 16. Figure 16: Timeline of operations in the Tri-Memory learning cycle, showing inference, [PITH_FULL_IMAGE:figures/full_fig_p027_16.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Exploring Kolmogorov-Arnold Network Expansions in Vision Transformers for Mitigating Catastrophic Forgetting in Continual Learning

    cs.CV 2025-07 reject novelty 3.0 of 10

    KAN-based ViTs show slight average incremental accuracy gains over MLP-ViTs in continual learning, but the paper's own data show worse forgetting on CIFAR-100 and worse last-task accuracy on MNIST.

Reference graph

Works this paper leans on

35 extracted references · 23 canonical work pages · cited by 1 Pith paper

  1. [1]

    R. C. Johnson, Edge ai devices eye lifetime learning, communications of the ACM News, March 12, 2024 (2024). URL https://cacm.acm.org/news/lifetime-learning-for-ai

    work page 2024

  2. [2]

    S. B. Laughlin, T. J. Sejnowski, Communication in neuronal networks, Science 301 (5641) (2003) 1870–1874. doi:10.1126/science.1089662

    doi:10.1126/science.1089662 2003

  3. [3]

    Goodfellow, M

    I. Goodfellow, M. Mirza, D. Xiao, A. Courville, Y. Bengio, An em- pirical investigation of catastrophic forgetting in gradient-based neural networks, arXiv preprint arXiv:1312.6211 (2015). arXiv:1312.6211. URL https://arxiv.org/abs/1312.6211

    arXiv 2015

  4. [4]

    Kirkpatrick, R

    J. Kirkpatrick, R. Pascanu, N. Rabinowitz, et al., Overcoming catas- trophic forgetting in neural networks, Proceedings of the National Academy of Sciences 114 (13) (2017) 3521–3526

    work page 2017

  5. [5]

    Zenke, B

    F. Zenke, B. Poole, S. Ganguli, Continual learning through synaptic intelligence, in: Proceedings of the 34th International Conference on Machine Learning, Vol. 70, 2017, pp. 3987–3995

    work page 2017

  6. [6]

    Robins, Catastrophic forgetting, rehearsal and pseudorehearsal, Con- nection Science 7 (2) (1995) 123–146

    A. Robins, Catastrophic forgetting, rehearsal and pseudorehearsal, Con- nection Science 7 (2) (1995) 123–146

    1995

  7. [7]

    Rebuffi, A

    S.-A. Rebuffi, A. Kolesnikov, G. Sperl, C. H. Lampert, icarl: Incremen- tal classifier and representation learning, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 2001–2010

    work page 2017

  8. [8]

    H. Shin, J. K. Lee, J. Kim, J. Kim, Continual learning with deep gen- erative replay, in: Advances in Neural Information Processing Systems, Vol. 30, 2017, pp. 2990–2999

    work page 2017

Show all 35 references
  1. [9]

    G. M. van de Ven, H. T. Siegelmann, A. S. Tolias, Brain-inspired replay for continual learning with artificial neural networks, Nature Communi- cations 11 (2020) 4069

    2020

  2. [10]

    Z. Mai, X. Ma, I. S. Kweon, Online continual learning in image classifi- cation: An empirical survey, arXiv preprint arXiv:2101.10423 (2021). 40

    2021 arXiv

  3. [11]

    A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, R. Hadsell, Progressive neural networks, arXiv preprint (2016). arXiv:1606.04671. URL https://arxiv.org/abs/1606.04671

    2016 arXiv

  4. [12]

    J. Yoon, E. Yang, J. Lee, S. J. Hwang, Lifelong learning with dy- namically expandable networks, in: International Conference on Learn- ing Representations (ICLR) , 2018, pp. 1–14, available at https:// openreview.net/forum?id=Sk7KsfW0-

    2018

  5. [13]

    Mallya, S

    A. Mallya, S. Lazebnik, Packnet: Adding multiple tasks to a single network by iterative pruning, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2018, pp. 7765–

    2018

  6. [14]

    J. L. McClelland, B. L. McNaughton, R. C. O’Reilly, Why there are complementary learning systems in the hippocampus and neocortex, Psychological Review 102 (3) (1995) 419–457

    1995

  7. [15]

    L. Wang, X. Zhang, Q. Li, M. Zhang, H. Su, J. Zhu, Y. Zhong, Incor- porating neuro-inspired adaptability for continual learning in artificial intelligence, Nature Machine Intelligence 5 (10) (2023) 992–1003

    2023

  8. [16]

    Schmidhuber, Learning complex, extended sequences using the prin- ciple of history compression, Neural Computation 4 (2) (1992) 234–242

    J. Schmidhuber, Learning complex, extended sequences using the prin- ciple of history compression, Neural Computation 4 (2) (1992) 234–242

    1992

  9. [17]

    J. Ba, G. Hinton, Using fast weights to attend to the recent past, in: Advances in Neural Information Processing Systems, 2016, pp. 4331– 4339

    2016

  10. [18]

    Munkhdalai, X

    T. Munkhdalai, X. Yuan, A. Trischler, K. Cho, Meta-learning with heb- bian fast weights, in: Advances in Neural Information Processing Sys- tems, Vol. 32, 2019, pp. 9473–9483

    2019

  11. [19]

    Behrouz, P

    A. Behrouz, P. Zhong, V. Mirrokni, Titans: Learning to memorize at test time, arXiv preprint arXiv:2501.00663 (2025). arXiv:2501.00663. URL https://arxiv.org/abs/2501.00663

    2025 arXiv

  12. [20]

    Davies, N

    M. Davies, N. Srinivasa, T.-H. Lin, M. Chinya, Y. Cao, S. H. Choday, G. Dimou, P. Joshi, N. Imam, S. Jain, et al., Loihi: A neuromorphic manycore processor with on-chip learning, IEEE Micro 38 (2018) 82–99. 41

    2018

  13. [21]

    Hinton, O

    G. Hinton, O. Vinyals, J. Dean, Distilling the knowledge in a neural network, arXiv preprint arXiv:1503.02531 (2015). arXiv:1503.02531. URL https://arxiv.org/abs/1503.02531

    2015 arXiv

  14. [22]

    S. Han, J. Pool, J. Tran, W. Dally, Learning both weights and connec- tions for efficient neural networks, in: Advances in Neural Information Processing Systems, Vol. 28, 2015, pp. 1135–1143

    2015

  15. [23]

    Golkar, M

    S. Golkar, M. Kagan, K. Cho, Continual learning via neural pruning, arXiv preprint (2019). arXiv:1903.04476

    2019 arXiv

  16. [24]

    N. Du, Y. Huang, A. M. Dai, S. Tong, D. Lepikhin, Y. Xu, C. Cui, Glam: Efficient scaling of language models with mixture-of-experts, arXiv preprint arXiv:2112.06905 (2022). arXiv:2112.06905. URL https://arxiv.org/abs/2112.06905

    2022 arXiv

  17. [25]

    Y. Kaya, I. Hacihaliloglu, O. Sener, X. Li, Efficient early-exit architec- tures for transformer-based text encoders, in: Proceedings of the ACM Web Conference 2023, 2023, pp. 1577–1587. doi:10.1145/3698767

    2023 doi

  18. [26]

    Xie, et al., A shared neural basis underlying psychiatric comorbidity, Nature Medicine (2023)

    C. Xie, et al., A shared neural basis underlying psychiatric comorbidity, Nature Medicine (2023). doi:10.1038/s41591-023-02317-4

    2023 doi

  19. [27]

    Cafasso, Synaptic pruning: Definition, early childhood, and more, https://www.healthline.com/health/synaptic-pruning, ac- cessed April 20, 2025 (n.d.)

    J. Cafasso, Synaptic pruning: Definition, early childhood, and more, https://www.healthline.com/health/synaptic-pruning, ac- cessed April 20, 2025 (n.d.)

    2025

  20. [28]

    Oja, A simplified neuron model as a principal component analyzer, Journal of Mathematical Biology 15 (1982) 267–273

    E. Oja, A simplified neuron model as a principal component analyzer, Journal of Mathematical Biology 15 (1982) 267–273

    1982

  21. [29]

    Miconi, K

    T. Miconi, K. O. Stanley, J. Clune, Differentiable plasticity: Training plastic neural networks with gradient descent, in: Proceedings of the 35th International Conference on Machine Learning (ICML) , 2018, pp. 3559–3568. doi:10.48550/arXiv.1804.02464

    2018 doi arXiv

  22. [30]

    Z. Wang, Z. Zhan, Y. Gong, G. Yuan, W. Niu, T. Jian, J. Dy, Sparcl: Sparse continual learning on the edge, in: Advances in Neural Informa- tion Processing Systems, Vol. 35, 2022, pp. 20366–20380. URL https://arxiv.org/abs/2209.09476 42

    2022 arXiv

  23. [31]

    J. L. McClelland, B. L. McNaughton, A. K. Lampinen, Integration of new information in memory: New insights from a complementary learn- ing systems perspective, Philosophical Transactions of the Royal Society B 375 (2020) 20190637. doi:10.1098/rstb.2019.0637

    2020

  24. [32]

    Santoro, S

    A. Santoro, S. Bartunov, M. Botvinick, D. Wierstra, T. Lillicrap, One- shot learning with memory-augmented neural networks, in: Proceed- ings of the 33rd International Conference on Machine Learning (ICML), PMLR, 2016, pp. 1842–1850. doi:10.48550/arXiv.1605.06065

    2016 doi arXiv

  25. [33]

    Javed, M

    K. Javed, M. White, Meta-learning representations for continual learn- ing, in: Advances in Neural Information Processing Systems, Vol. 32, PMLR, 2019, pp. 18108–18118

    2019

  26. [34]

    X. Li, X. Jin, J. Yan, J. Feng, Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting, in: Proceed- ings of the 36th International Conference on Machine Learning (ICML), PMLR, 2019, pp. 3925–3934. 43 Author biographies Rajeev Gupta is a...

    2019

  27. [7773]

    doi:10.1109/CVPR.2018.00810

    2018

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.