REVIEW 5 major objections 7 minor 31 references
Asynchronous Hebbian/anti-Hebbian networks
T0 review · 5 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Burst-gated Hebbian learning, updated neuron-by-neuron only when a cell's activity crosses a threshold and followed by a refractory pause, reproduces classic factorised representations in continuous time and prevents catastrophic…
desk verdict A neat asynchronous plasticity mechanism with a credible match to classic Hebbian/anti-Hebbian learning, but the forgetting-resistance claim needs capacity stress tests before you trust it. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is a postsynaptic gate $\beta_i$ that couples Hebbian plasticity to bursting activity. Whenever neuron $i$'s activity $y_i$ exceeds the bursting threshold $r_b$ and at least $r_r$ Euler steps have elapsed since its last update, the gate is 1 and both feed-forward and recurrent weights are updated; otherwise the gate is 0. The refractory counter $c_i$ resets after each gated update. What this does is to mimic the discrete model's 'settle then update' step: the threshold ensures that only selective neurons, whose activity reflects a stable, context-resolved state, change their weights, and the refractory period prevents the same neuron from repeatedly tracking a stimulus that stays present. The paper's comparisons show that the refractory period is essential: with no or very short refractoriness, the network collapses to the non-factorised, winner-take-all behavior of the naive continuous model.
What would settle it
Run the asynchronous network on the three stimulus sets while shrinking the neuron count until the number of available neurons is below the total number of independent factors; if the reconstruction error for the first stimulus set rises after later sets are introduced at a finite neuron count, the claim that forgetting is naturally prevented fails. A complementary check is to fix the network size and drive the refractory period toward zero, which should re-introduce forgetting and the winner-take-all receptive fields the paper associates with the continuous model.
Extended reading notes
Core claim
The paper's central claim is that a Hebbian/anti-Hebbian network in which plasticity is gated by postsynaptic bursting and followed by a refractory period can learn the same factorised, sparse representations as the classic discrete-time model, while operating entirely in continuous time. The learning rule is $\Delta w_{i,j} = \beta_i (y_i x_j - w_{i,j})$ with a matching recurrent update, where the gate $\beta_i$ is 1 only when the neuron's activity $y_i$ exceeds a threshold $r_b$ and its counter since the last update exceeds a refractory period $r_r$. This gating makes updates sparse and selective: only neurons whose receptive fields are already well activated change, and the refractory period stops a bursting neuron from being dragged toward coincident patterns. The authors report that after training, most neurons' incoming weight vectors have cosine similarity 0.95–1.0 with the discrete model's neurons, in contrast to a continuous-time model that updates every Euler step, which diverges and learns winner-take-all crosses rather than factorised stripes and bars. They also claim the same mechanism prevents catastrophic forgetting: when bars, dots, and diagonals are shown in sequence, the asynchronous network keeps its old receptive fields and low reconstruction error for all three stimulus sets, while the discrete model forgets earlier sets.
Load-bearing premise
The continual-learning claim stands on the Section 4 assumption that the bursting threshold prevents already-tuned neurons from being updated and that the network has enough capacity to represent all stimulus factors; neither condition is quantified, and the paper tests only a single stimulus order with a fixed network size.
Editorial extensions
If this is right
- Factorised representation learning no longer needs a two-timescale separation: the network learns from a continuously changing input stream with no settling phase.
- The representations learned asynchronously are nearly identical to the classic discrete model's, with most neurons reaching cosine similarity 0.95–1.0 to the discrete model's weights and matching its sparse activity and reconstruction error.
- When three stimulus families are shown one after another, old receptive fields stay intact and new inputs recruit previously unstructured neurons, so reconstruction error for all three families remains low; the discrete model's error for earlier families rises again.
- The refractory period is indispensable: shortening it to 1–10 Euler steps or setting the bursting threshold to zero makes the asynchronous model behave like the failing continuous model and develop non-factorised, winner-take-all receptive fields.
- Because plasticity events are fewer than in the continuous model, the rule is also a candidate mechanism for limiting the metabolic cost of synaptic change.
Reading between the lines
- A direct capacity experiment, not run in the paper, would shrink the number of neurons while keeping the factor count fixed; the continual-learning benefit should vanish at a finite capacity threshold, and locating that threshold would separate the refractory mechanism from simple overcapacity.
- The same burst-gating and refractory schedule could be transferred to other settle-then-learn rules, such as energy-based or equilibrium-propagation training, turning their plasticity into event-driven updates and potentially giving them the same protection against forgetting.
- Because sparse spiking may act as a natural analogue of the refractory gate, the model predicts that a spiking lateral-inhibition network with homeostasis should show the same factorised representations and continual-learning profile; a systematic rate-versus-spiking comparison would test that.
- If the refractory period exists specifically to stabilise tuned neurons, then biological manipulations that remove spine-level refractory plasticity should make receptive fields drift or old memories degrade when input statistics change, a prediction that goes beyond the paper's simulations.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes an asynchronous Hebbian/anti-Hebbian learning rule for lateral-inhibition networks. Instead of waiting for recurrent dynamics to settle before applying a plasticity update, each neuron updates its weights only when its activity exceeds a threshold, followed by a refractory period during which no further updates occur. The authors compare this rule with a discrete settle-then-update baseline and with a continuous-time baseline on Földiák-style bar stimuli, reporting that the asynchronous model learns similarly factorised receptive fields, with comparable sparsity and reconstruction error to the discrete model. They further present a sequential-learning experiment with three stimulus sets and claim that the asynchronous rule naturally prevents catastrophic forgetting, whereas the discrete model forgets previous sets.
Significance. If the central claims hold, the paper contributes a biologically motivated mechanism that removes the settle-then-update requirement of classic Hebbian/anti-Hebbian networks while retaining factorised representations, and it demonstrates a potential continual-learning benefit with fewer plasticity events. The manuscript has several strengths: the proposed rule is simple and clearly motivated by experimental observations on burst-dependent plasticity and refractoriness; the authors provide open-source code; and they include parameter sweeps over threshold, refractory period, and hold period in Fig. 2 and the appendices. At the same time, the forgetting-resistance claim is broader than the evidence, and the comparison with the discrete baseline contains ambiguities that need resolution. The match between asynchronous and discrete learning is partly by construction, since the threshold-plus-refractory design is intentionally chosen to mimic one-update-per-settled-state behavior; the paper should therefore be evaluated primarily on whether the mechanism reproduces the discrete behavior robustly and whether the forgetting result holds beyond the single high-capacity configuration shown.
major comments (5)
- [Section 4, Fig. 3] The central claim that the asynchronous rule "naturally prevents catastrophic forgetting" is only supported in a high-capacity regime and is not tested at or near the capacity limit. Section 4 states "as long as the network capacity is sufficient to represent all relevant factors," but this condition is never quantified or varied. The experiment uses n=500 neurons, m=196 inputs, and only three small stimulus sets, with a single simulation ordering. If capacity is insufficient, the proposed threshold and refractory mechanism cannot guarantee protection: a neuron tuned to an old stimulus can still be strongly activated by a new overlapping stimulus, in which case the threshold will not prevent an update. Please quantify capacity, vary the number of neurons relative to the number of factors, vary the number and complexity of sequential distributions, and report old-set reconstruction errors over multiple random orderings and initializations. Without these experiments, the abstract's unqualified forgetting claim should be softened.
- [Appendix C, Eq. (11)] The asynchronous update rule in Eq. (11), Δw_i,j = β_i(y_i x_j − w_i,j), omits a learning rate. The discrete and continuous models are specified with η=0.01 and η=0.001 respectively, but no learning rate or time-step scaling is stated for the asynchronous model, and the simulation details in Appendix D do not provide one. This matters because the update amplitude directly affects stability, convergence speed, and the number of effective plasticity events, and it is needed for a controlled comparison with the continuous model. Please specify whether η is implicitly 1, whether the update is scaled by the Euler step size, or what value was used in the reported simulations.
- [Figs. 1D, 2A-C, 3] The paper does not report repeated-seed statistics, which is important for the quantitative similarity and forgetting claims. The error bars in Fig. 1D are standard deviations over samples within each time bin, not over random initializations or stimulus orders; Figs. 2A-C and Fig. 3 appear to show single runs. Since weights are randomly initialized and the stimulus sequence is randomly ordered, the conclusions "very similar" and "prevents catastrophic forgetting" need to be supported by mean and variability over at least several seeds. Please add repeated-seed results, especially for Fig. 3, where the forgetting claim rests on a single trajectory.
- [Sections 2-3, Appendices A.3-A.4] The discrete baseline model is specified inconsistently. Section 2 says the baseline is derived from the NMF objective and points to Appendix A.3 (the Pehlevan network), whose updates include a y_i^2 factor and a dynamic learning rate η_i. However, the equations in Section 2 and the "Hebbian" rule in Appendix A.4 use Δw = η(ŷ_i x_j − w_i,j) without y_i^2 and with a fixed learning rate, and Section 3 calls this "the same learning rule as our model - Hebbian." The asynchronous rule in Eq. (11) is of the A.4 form. Please state explicitly which discrete variant is used in each figure and reconcile the text, since the comparison in Fig. 2 is only meaningful if the baseline and its parameters are unambiguous.
- [Section 3, Eqs. (9)-(11)] The learning-rate comparison between the asynchronous and continuous models is not controlled as presented. The continuous model uses η=0.001 per Euler step, while Eq. (11) has no η; if the asynchronous update is applied at full amplitude at each gated step, the effective update magnitude is far larger. This could explain differences in stability and in the number of updates in Fig. 2I. Please either introduce an explicit learning rate for the asynchronous rule or justify why the update in Eq. (11) is not scaled; otherwise the qualitative differences attributed to the threshold and refractory mechanism may be confounded with learning-rate effects.
minor comments (7)
- [Abstract] The abstract contains a corrupted sentence: "a new Hebbian learning ruleewhich is mented withusing" should read "a new Hebbian learning rule which is implemented using."
- [Figure 1 caption] There are several typos: "inihibitory" should be "inhibitory," and "asynchronuos" should be "asynchronous." Please proofread the captions.
- [Section 2] The notation "x = |x1 · · · xn|" should use m for the input dimension, since x ∈ R^m, not n.
- [Appendix C] The definition "β_i ∈ 0, 1" should be "β_i ∈ {0, 1}" to denote the set containing zero and one.
- [Appendix D] The simulation details do not state the default values of the bursting threshold r_b and refractory period r_r used in the main experiments and in Fig. 3. These values should be given in the text or by explicit reference to the code.
- [Fig. 2F and Section 3] The terms "presentation period" and "hold period" are used interchangeably; please standardize the terminology.
- [Section 3, Fig. 2E] The x-axis of Fig. 2E is in units of firing rate, but the text refers to "bursting threshold" without defining its relation to the activity variable y_i; please clarify the threshold range used.
Circularity Check
No significant circularity: the asynchronous rule is independently simulated and compared against baselines; the capacity caveat is a robustness limitation, not a circular step.
full rationale
The paper's central claims are supported by direct simulation rather than by fitting or by self-citation. The asynchronous update (Appendix C, Eq. 11 and the accompanying counter update) is a novel gating mechanism whose parameters (burst threshold r_b and refractory period r_r) are varied systematically in Fig. 2D-F rather than tuned to force the reported match. The comparison to the discrete model is an external benchmark computed with identical initialization and stimulus sequence; the high cosine similarity in Fig. 2A-C is an emergent measurement, not a quantity built into the update rule. The factorised-representation result is additionally evaluated by receptive-field structure (Fig. 1G) and Gini sparsity (Fig. 1F), which do not reduce to the discrete-model comparison. The continual-learning claim is explicitly qualified in Section 4: 'as long as the network capacity is sufficient to represent all relevant factors.' That capacity condition is not quantified, and Fig. 3 uses a single high-capacity network (n=500 for three small stimulus sets), so the forgetting-resistance claim is not tested near capacity limits. This is a limitation of evidence, not a circular derivation: the threshold/refractory mechanism could in principle fail at capacity, and the paper does not define the outcome into existence. No self-citation chain is load-bearing; the cited biological observations of plasticity refractoriness ([9], [16]) are external experimental evidence. Thus no step reduces to its own inputs by construction.
Assumptions & free parameters
free parameters (4)
- Burst threshold rb =
not stated (swept 0.2 to 1.4)
- Refractory period rr =
not stated (swept 1 to 1000)
- Asynchronous learning rate (implicit) =
1 (Eq. 11 has no eta)
- Hold period f =
50, 150, 500 Euler steps (150 in evaluation)
assumptions (4)
- standard math The recurrent dynamics y_dot = [Wx - My]_+ converge to a unique stable state when M is positive definite.
- domain assumption Weights are kept strictly positive, with W excitatory and M inhibitory, throughout simulation.
- domain assumption Plasticity refractoriness observed in vitro transfers to this rate-based Hebbian model.
- standard math Local Hebbian/anti-Hebbian updates with these functional forms learn nonnegative matrix factorization or sparse codes.
Cite this review
Pith. "Pith review of Asynchronous Hebbian/anti-Hebbian networks." pith.science (2026). https://pith.science/paper/6FE4DILE
@misc{pith2026250102402,
author = {Pith},
title = {Pith review of: Asynchronous Hebbian/anti-Hebbian networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/6FE4DILE}},
note = {Machine review of arXiv:2501.02402}
}
read the original abstract
Lateral inhibition models coupled with Hebbian plasticity have been shown to learn factorised causal representations of input stimuli, for instance, oriented edges are learned from natural images. Currently, these models require the recurrent dynamics to settle into a stable state before weight changes can be applied, which is not only biologically implausible, but also impractical for real-time learning systems. Here, we propose a new Hebbian learning rule which is implemented using plausible biological mechanisms that have been observed experimentally. We find that this rule allows for efficient, time-continuous learning of factorised representations, very similar to the classic noncontinuous Hebbian/anti-Hebbian learning. Furthermore, we show that this rule naturally prevents catastrophic forgetting when stimuli from different distributions are shown sequentially.
Figures
Reference graph
Works this paper leans on
-
[1]
An efficient code in v1? Nature, 381(6583), 1996
Roland Baddeley. An efficient code in v1? Nature, 381(6583), 1996
work page 1996
-
[2]
Possible principles underlying the transformation of sensory messages
Horace B Barlow et al. Possible principles underlying the transformation of sensory messages. Sensory Communication, 1(01):217–233, 1961
work page 1961
-
[3]
Internal representations for associative memory
Eric B Baum, John Moody, and Frank Wilczek. Internal representations for associative memory. Biological Cybernetics, 59(4):217–228, 1988
work page 1988
-
[4]
An information-maximization approach to blind separation and blind deconvolution
Anthony J Bell and Terrence J Sejnowski. An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7(6):1129–1159, 1995
work page 1995
-
[5]
Anthony J Bell and Terrence J Sejnowski. The “independent components” of natural scenes are edge filters. Vision Research, 37(23):3327–3338, 1997
work page 1997
-
[6]
Representation learning: A review and new perspectives
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence , 35(8): 1798–1828, 2013. 6
work page 2013
-
[7]
Synapse-type-specific competitive hebbian learning forms functional recurrent networks
Samuel Eckmann, Edward James Young, and Julijana Gjorgjieva. Synapse-type-specific competitive hebbian learning forms functional recurrent networks. Proceedings of the National Academy of Sciences, 121(25):e2305326121, 2024
work page 2024
-
[8]
A simple hebbian/anti- hebbian network learns the sparse, independent components of natural images
Michael S Falconbridge, Robert L Stamps, and David R Badcock. A simple hebbian/anti- hebbian network learns the sparse, independent components of natural images. Neural Compu- tation, 18(2):415–429, 2006
work page 2006
Show all 31 references
-
[9]
A synapse-specific refractory period for plasticity at individual dendritic spines
Juan C Flores and Karen Zito. A synapse-specific refractory period for plasticity at individual dendritic spines. bioRxiv, pages 2024–05, 2024
2024
-
[10]
Forming sparse representations by local anti-hebbian learning
Peter Földiak. Forming sparse representations by local anti-hebbian learning. Biological Cybernetics, 64(2):165–170, 1990
1990
-
[11]
Catastrophic forgetting in connectionist networks
Robert M French. Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences, 3(4):128–135, 1999
1999
-
[12]
Contribution of individual spikes in burst-induced long-term synaptic modification
Robert C Froemke, Ishan A Tsay, Mohamad Raad, John D Long, and Yang Dan. Contribution of individual spikes in burst-induced long-term synaptic modification. Journal of Neurophysiology, 95(3):1620–1629, 2006
2006
-
[13]
Adaptive pattern classification and universal recoding: I
Stephen Grossberg. Adaptive pattern classification and universal recoding: I. parallel de- velopment and coding of neural feature detectors. Biological Cybernetics, 23(3):121–134, 1976
1976
-
[14]
Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex
David H Hubel and Torsten N Wiesel. Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. The Journal of Physiology, 160(1):106, 1962
1962
-
[15]
Synaptic plasticity rules with physiological calcium levels
Yanis Inglebert, Johnatan Aljadeff, Nicolas Brunel, and Dominique Debanne. Synaptic plasticity rules with physiological calcium levels. Proceedings of the National Academy of Sciences , 117 (52):33639–33648, 2020
2020
-
[16]
Synaptic evidence for the efficacy of spaced learning
Enikö A Kramár, Alex H Babayan, Cristin F Gavin, Conor D Cox, Matiar Jafari, Christine M Gall, Gavin Rumbaugh, and Gary Lynch. Synaptic evidence for the efficacy of spaced learning. Proceedings of the National Academy of Sciences , 109(13):5121–5126, 2012
2012
-
[17]
Unsupervised learning by competing hidden units
Dmitry Krotov and John J Hopfield. Unsupervised learning by competing hidden units. Pro- ceedings of the National Academy of Sciences , 116(16):7723–7731, 2019
2019
-
[18]
Improved local learning rule for information maximization and related applica- tions
Ralph Linsker. Improved local learning rule for information maximization and related applica- tions. Neural Networks, 18(3):261–265, 2005
2005
-
[19]
Biologically plausible online principal component analysis without recurrent neural dynamics
Victor Minden, Cengiz Pehlevan, and Dmitri B Chklovskii. Biologically plausible online principal component analysis without recurrent neural dynamics. In 2018 52nd Asilomar Conference on Signals, Systems, and Computers , pages 104–111. IEEE, 2018
2018
-
[20]
Simplified neuron model as a principal component analyzer.Journal of Mathematical Biology, 15:267–273, 1982
Erkki Oja. Simplified neuron model as a principal component analyzer.Journal of Mathematical Biology, 15:267–273, 1982
1982
-
[21]
Sparse coding of sensory inputs
Bruno A Olshausen and David J Field. Sparse coding of sensory inputs. Current Opinion in Neurobiology, 14(4):481–487, 2004
2004
-
[22]
Energetically efficient learning in neuronal networks
Aaron Pache and Mark CW van Rossum. Energetically efficient learning in neuronal networks. Current Opinion in Neurobiology, 83:102779, 2023
2023
-
[23]
Natural patterns of activity and long-term synaptic plasticity
Ole Paulsen and Terrence J Sejnowski. Natural patterns of activity and long-term synaptic plasticity. Current Opinion in Neurobiology, 10(2):172–180, 2000
2000
-
[24]
A hebbian/anti-hebbian network derived from online non-negative matrix factorization can cluster and discover sparse features
Cengiz Pehlevan and Dmitri B Chklovskii. A hebbian/anti-hebbian network derived from online non-negative matrix factorization can cluster and discover sparse features. In 2014 48th Asilomar Conference on Signals, Systems and Computers , pages 769–775. IEEE, 2014
2014
-
[25]
Why do similarity matching objectives lead to hebbian/anti-hebbian networks? Neural Computation, 30(1):84–124, 2017
Cengiz Pehlevan, Anirvan M Sengupta, and Dmitri B Chklovskii. Why do similarity matching objectives lead to hebbian/anti-hebbian networks? Neural Computation, 30(1):84–124, 2017
2017
-
[26]
Coordinated drift of receptive fields in hebbian/anti-hebbian network models during noisy representation learning
Shanshan Qin, Shiva Farashahi, David Lipshutz, Anirvan M Sengupta, Dmitri B Chklovskii, and Cengiz Pehlevan. Coordinated drift of receptive fields in hebbian/anti-hebbian network models during noisy representation learning. Nature Neuroscience, 26(2):339–349, 2023
2023
-
[27]
Feature discovery by competitive learning
David E Rumelhart and David Zipser. Feature discovery by competitive learning. Cognitive Science, 9(1):75–112, 1985. 7
1985
-
[28]
Independent component analysis in spiking neurons
Cristina Savin, Prashant Joshi, and Jochen Triesch. Independent component analysis in spiking neurons. PLoS Computational Biology, 6(4):e1000757, 2010
2010
-
[29]
Equilibrium propagation: Bridging the gap between energy-based models and backpropagation
Benjamin Scellier and Yoshua Bengio. Equilibrium propagation: Bridging the gap between energy-based models and backpropagation. Frontiers in Computational Neuroscience, 11:24, 2017
2017
-
[30]
Synergies between intrinsic and synaptic plasticity in individual model neurons
Jochen Triesch. Synergies between intrinsic and synaptic plasticity in individual model neurons. Advances in Neural Information Processing Systems , 17, 2004
2004
-
[31]
Non-holographic associative memory
David J Willshaw, O Peter Buneman, and Hugh Christopher Longuet-Higgins. Non-holographic associative memory. Nature, 222(5197):960–962, 1969. A Local learning rules Hebbian/anti-Hebbian networks have been extensively studied and analysed, multiple functional forms have been pr...
1969
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.