Pith. sign in

REVIEW 4 major objections 5 minor 31 references

Ubiquity of Emergent Hebbian Dynamics in Regularized Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Near any nonzero stationary point of an L2-regularized loss, every learning rule's expected update aligns with the Hebbian direction, so Hebbian signatures are generic artifacts of weight decay.

desk verdict A clean stationarity identity makes Hebbian signatures a generic artifact of weight decay, but the monotonicity and per-stimulus claims outrun the proof. read the letter →

arxiv 2505.18069 v3 pith:QEWCOFCX submitted 2025-05-23 cs.LG eess.SP

David Koplow , Tomaso Poggio , Liu Ziyin This is my paper
classification cs.LGeess.SP
keywords Hebbianlearninganti-Hebbianplasticityweightdecayregularizationstationarityemergentdynamicsidentifiabilitysynaptic
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish an identifiability result: observing Hebbian or anti-Hebbian structure in synaptic updates does not uniquely identify an underlying Hebbian computation. Near a nonzero stationary point of an L2-regularized loss, the expected learning signal of essentially any update rule aligns with the Hebbian direction, with alignment growing with weight-decay strength; this is a generic equilibrium artifact of regularization. In the presence of noise the same balance mechanism produces anti-Hebbian alignment, with a predicted phase transition at weight decay $\gamma \propto \sigma^2$. A sympathetic reader should care because neuroscience measurements of LTP/LTD and synaptic plasticity are often interpreted mechanistically, and this paper shows those signatures can arise without any Hebbian computation, complicating such interpretations.

What carries the argument

The central object is the decomposition of the total weight update into a learning signal $g(x,\theta)=-\nabla_W\ell(x;W)$ and a decay term $-\gamma W$, with the Hebbian update defined as $H(x;W)=h_b(x)h_a(x)^T$ for a layer $h_b=Wh_a$. The identity that carries the argument is the stationarity balance $\mathbb{E}_x[g]=\gamma W$, which converts the contractive force of weight decay into a positive correlation between the learning signal and the expansive Hebbian direction. For the noise result, the machinery is the same balance applied to a linear regression loss $(\ell(w)=(w^Tx-y)^2)$ with injected Gaussian noise $\epsilon\sim\mathcal{N}(0,\sigma I)$, where the $\sigma^2\|x\|^2$ term makes the alignment negative.

What would settle it

Train a ReLU network on heterogeneous inputs to a nonzero stationary point with at least two weight-decay strengths, and compute per-example alignment $\mathbb{E}_x[\langle -\nabla_W\ell(x;W), h_b(x)h_a(x)^T\rangle]$ and its average. If the per-example quantity is not positive or does not increase with $\gamma$, the stronger version of the claim fails. Separately, in a noisy regression setting, vary $\sigma^2$ and $\gamma$ independently and check whether the zero-alignment contour in the $(\sigma^2,\gamma)$ plane follows a line $\gamma\propto\sigma^2$; a different boundary shape would falsify the anti-Hebbian tradeoff prediction.

Watch

Extended reading notes

Core claim

At stationarity of the penalized loss $L(W)=\mathbb{E}_x[\ell(x;W)]+\frac{\gamma}{2}\|W\|_F^2$, stationarity forces the expected learning signal to satisfy $\mathbb{E}_x[g(x,\theta)]=\gamma W$. Taking the Frobenius inner product with the expected Hebbian update $\mathbb{E}_x[h_a h_b^T]$ gives $\mathrm{Tr}(\mathbb{E}_x[g]\,\mathbb{E}_x[h_a h_b^T])=\gamma\,\mathbb{E}[\|h_b\|^2]>0$ for any $W\neq 0$, so any rule with weight decay looks Hebbian on average near stationarity. Under the stronger norm-decoupling assumption $\|h_a(x)\|^2=c_a$ almost surely, the per-stimulus alignment is $C(W^\star)=\gamma c_a \|W^\star\|_F^2>0$ at a stationary point. Adding noise to the weights in linear regression makes the alignment $\mathbb{E}[(\Delta_{\mathrm{SGD}}w)^T(\Delta_{\mathrm{Hebb}}w)]\approx -\sigma^2 c_0+\gamma c_1$, so strong noise flips the signature anti-Hebbian and the two effects trade off across a phase boundary $\gamma\propto\sigma^2$.

Load-bearing premise

The paper's sharper per-stimulus and monotonic-in-$\gamma$ predictions rest on Assumption 2: the presynaptic activation norm $\|h_a(x)\|^2$ is the same constant $c_a$ for every input, almost surely. Real ReLU networks and heterogeneous inputs violate this, and without it the expectation $\mathbb{E}[\|h_a\|^2 \nabla_{h_b}^T\ell\, h_b]$ need not factor into the positive quantity used in the proof; the averaged alignment result (Eq. 12) does not require this assumption.

Editorial extensions

If this is right

  • Observed Hebbian alignment at convergence in a regularized system is not evidence that the system is implementing a Hebbian rule.
  • The alignment strength is predicted to grow monotonically with weight-decay strength for any learning rule whose expected update matches the stationarity balance.
  • Noise and weight decay compete: in regression settings, the sign of Hebbian alignment is governed by the ratio $\gamma/\sigma^2$, predicting anti-Hebbian signatures in high-noise regimes.
  • Because the effect appears long before convergence and for random update rules, time-course or rule-identity checks cannot by themselves distinguish emergent from mechanistic Hebbian dynamics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the universal alignment result holds, then any homeostatic mechanism that acts like decay—synaptic normalization, spine shrinkage, slow biochemical turnover—should imprint Hebbian structure onto whatever learning rule a circuit uses, so the presence of Hebbian structure carries no information about the local plasticity rule.
  • The $\gamma\propto\sigma^2$ phase boundary suggests a quantitative experimental design: in a circuit where intrinsic noise and decay-like homeostasis can be manipulated independently, anti-Hebbian plasticity should dominate in the high-noise, low-decay corner and Hebbian plasticity in the opposite corner.
  • The authors' own observation that larger models and some layers show anti-Hebbian alignment even at high $\gamma$ hints that the stationarity condition is the fragile part; a theory of transient, out-of-stationary Hebbian phases would need to replace Assumption 2 with a condition that survives ReLU and heterogeneous inputs.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper argues that Hebbian and anti-Hebbian plasticity signatures can arise as emergent epiphenomena of L2-regularized learning near stationarity, so their observation in neural data does not by itself imply an underlying mechanistic Hebbian computation. For a layer h_b = W h_a with learning signal g(x,W) and full update g - γW, stationarity gives E[g] = γW, from which the paper derives the averaged alignment Tr(E[g] E[h_a h_b^T]) = γ E[||h_b||^2] > 0 (Eq. 12). Under a stronger norm-decoupling assumption, the per-stimulus alignment is C(W*) = γ c_a ||W*||^2_F (Eq. 8 and Lemma 1 in Appendix C.5.2). The paper also analyzes additive noise in a linear regression model and claims an anti-Hebbian phase with a phase boundary γ ∝ σ^2 (Eq. 18), supported by simulations on MLPs and transformers with SGD, Adam, direct feedback alignment, and a random-network learning signal.

Significance. If the central claim survives, the paper delivers a useful identifiability caveat for neuroscience: Hebbian-looking update statistics can be equilibrium artifacts of regularization rather than evidence of Hebbian computation. The main strengths are the clear formal setup in Appendix C.5, the parameter-free nature of the universal averaged alignment result (Eq. 12), and the breadth of empirical controls, including a RandomNN that cannot learn yet still exhibits the alignment. The paper also states explicit limitations about larger networks and non-last layers. However, the headline monotonic-in-γ claim is not actually proven, the stronger per-stimulus result relies on an unverified and often violated norm-decoupling assumption, and the anti-Hebbian phase boundary is asserted rather than derived. These gaps affect the paper's main quantitative predictions and must be addressed.

major comments (4)
  1. [Section 3, Eq. (8), Appendix C.5.2 Lemma 1] The claim that Hebbian alignment increases monotonically with weight-decay strength is not established by the proof as written. Lemma 1 gives C(W*) = γ c_a ||W*||^2_F, but W* itself depends on γ through the stationarity condition E[g(x;W*)] = γW*, and nothing in the proof controls how ||W*||^2_F scales with γ. The paper's own linear regression example illustrates the problem: if w*(γ) ≈ c/(λ+γ), then γ||w*(γ)||^2 is non-monotone in γ. Table 1 also contains entries marked '–' where the weights collapsed to zero at the largest γ, exactly the regime where ||W*|| shrinks. The abstract's 'increasing monotonically with decay strength' and Section 1 point 1 therefore go beyond what the mathematics proves. Please either prove monotonicity under explicit conditions on the stationary path or rephrase these statements as empirical observations with appropriate qualifications.
  2. [Appendix C.5.2, Assumption 2; Section 3, Eqs. (7)-(8)] The per-stimulus correlation result and Eq. (8) require Assumption 2, that ||h_a(x)||^2 = c_a almost surely. This condition is not verified in any experiment, and it is violated by ReLU activations and heterogeneous inputs, both of which appear in the paper's own experiments. The main text's informal 'weak decoupling condition' is also stated differently from Assumption 2, which adds ambiguity. The averaged universal result in Eq. (12) survives without Assumption 2, but the stronger claim that the learning signal and the Hebbian rule are statistically correlated per stimulus is not supported by the reported experiments. Please either verify the norm-condition empirically (for example, by reporting the distribution of ||h_a(x)||^2 during training) or restrict the per-stimulus claims to settings where the assumption plausibly holds, such as normalized activations or neural collapse.
  3. [Section 4, Eq. (18)] The phase-boundary statement γ ∝ σ^2 is asserted through Eq. (18), where c_0 and c_1 are said to be 'positive coefficients' that can be treated as constants with respect to weight decay and noise. No derivation is given for this approximation, and the preceding exact computation in Eqs. (16)-(17) is a single-step calculation in which the stationary weight v itself generally depends on γ and σ. In nonlinear networks this is a heuristic, not a proven scaling law. Figure 4 provides qualitative support, but the boundary is not fitted or tested statistically. Please either derive the boundary from a solvable stationary distribution or clearly mark Eq. (18) as an approximation and report how the boundary was estimated from the simulations.
  4. [Section 6; Section 3, Figure 3] The paper's own limitations section states that strong average anti-Hebbian alignment was often seen in a subset or all layers even at high weight decay, and Section 4 notes difficulty reproducing the effect outside the last few layers of larger networks. These observations sit in tension with the universal monotonic-in-γ claim and with Theorem 1's neighborhood statement, which predicts positive alignment near a nondegenerate stationary point for any layer satisfying the assumptions. The paper attributes the discrepancy to stationarity not holding, but offers no diagnostic test of this explanation. Please reconcile the theoretical scope with the empirical scope, for example by stating explicitly which layers and training regimes the theory is expected to cover and by reporting stationarity-gap measurements alongside the alignment measurements.
minor comments (5)
  1. [Eqs. (4) and (7)] Equations (4) and (7) contain stray markup (“⌏⟨⟨➪ rl➫ l⟩⟩...”) that will render incorrectly; please replace these with the intended labels 'learning signal' and 'Hebbian update'.
  2. [Throughout] Several typos and spacing issues should be fixed: 'CIF AR-10' should be 'CIFAR-10', 'F rozen Paramaters' in Appendix C.1.5 should be 'Frozen Parameters', and 'Full W eight Update' in Appendix C.3.1 should be 'Full Weight Update'.
  3. [Section 3, informal decoupling statement] The main text's statement of the decoupling condition, “‖h_a‖^2 = E[‖h_a‖^2]”, is ambiguous and does not match the almost-sure constant-norm condition of Assumption 2 in Appendix C.5.2; please state the exact assumption used in the main text.
  4. [Table 1] The '–' entries in Table 1 are only explained in the table title as indicating weight collapse; please define this in the caption and discuss how these points are treated when claiming a monotone trend, since they occur at the largest γ.
  5. [Section 3, Classification paragraph] The sentence 'Although we still detect this trend in larger MLPS, we occasionally observe some layers behaving in an anti-Hebbian direction as the weight increases' appears to mean 'as the weight decay increases'; please correct the wording.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the universal Hebbian-alignment identity follows from the stationarity condition and the definition of the Hebbian update, but it is not a fitted prediction; only a minor non-load-bearing self-citation keeps the score above zero.

full rationale

The paper's claimed derivation chain is not circular. The central universal-alignment statement (Eq. 12) is obtained by substituting the stationarity condition E[g]=γW into the definition of alignment against the Hebbian update h_b h_a^T; this is a direct algebraic identity, but the positivity γE||h_b||²>0 is a theorem, not an assumed output, and no parameter is fitted to produce it. The stronger per-stimulus result (Eq. 8) relies on Assumption 2 (constant presynaptic norm), which is stated openly and not imported from prior work; the self-citation to Ziyin et al. (2024) only motivates this assumption and is not load-bearing. The monotonic-in-γ claim and the γ∝σ² phase boundary are not fully derived because the stationary point W★ and the constants c0,c1 depend on quantities the paper does not control, but that is a derivation gap or correctness risk, not circularity. The empirical tests (Table 1, Figure 4) are independent of the theoretical identities. One minor self-citation to the authors' own prior work appears in the decoupling-condition discussion; it is not used to prove the main theorem, so it does not constitute circularity beyond a score of 2.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The theory introduces no fitted constants and no new physical entities. Its load-bearing assumptions are the norm-decoupling condition (Assumption 2) and the stationarity balance E[g] = γW. The phase boundary in Section 4 uses c0 and c1 as unspecified positive coefficients treated as constants with respect to γ and σ, which is an approximation rather than a fitted parameter.

assumptions (6)
  • domain assumption Loss gradient obeys the chain rule ∇_W ℓ = ∇_{h_b} ℓ h_a^T (Assumption 1)
    Used in Lemma 1 to compute the alignment of the learning signal with the Hebbian update; standard for differentiable networks.
  • ad hoc to paper Presynaptic activation norms are constant across inputs: ∥h_a(x)∥^2 = c_a almost surely (Assumption 2)
    Needed to factor the expectation in Eq. 7 and prove C(W*) > 0 for the SGD-type result; not satisfied by ReLU or heterogeneous inputs and not verified empirically.
  • standard math Local boundedness and Lipschitz continuity of g and H near W* (Assumption 3)
    Used only for the quantitative neighborhood bound (Lemma 2 and Theorem 1); standard regularity condition.
  • domain assumption Nondegenerate stationary point, with invertible Jacobian of the drift field (Assumption 4)
    Used in Theorem 2 to convert distance to the stationary point into a bound on the stationarity gap; excludes degenerate critical points.
  • domain assumption Noise enters additively as ε ~ N(0, σI) on weights before each step, within a linear regression loss
    Basis of the anti-Hebbian derivation in Section 4; the generalization to deep nonlinear networks is not proven.
  • domain assumption At stationarity the expected learning signal exactly balances weight decay: E[g] = γW
    This is the definition of a stationary point of the penalized loss (Eq. 24); all alignment results follow from it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Ubiquity of Emergent Hebbian Dynamics in Regularized Learning." pith.science (2026). https://pith.science/paper/QEWCOFCX

@misc{pith2026250518069,
  author       = {Pith},
  title        = {Pith review of: Ubiquity of Emergent Hebbian Dynamics in Regularized Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QEWCOFCX}},
  note         = {Machine review of arXiv:2505.18069}
}
read the original abstract

Hebbian and anti-Hebbian plasticity are widely observed in the brain and are classically modeled as mechanistic, local homosynaptic rules stabilized by homeostatic constraints. This raises an identifiability question: does observing Hebbian/anti-Hebbian structure in synaptic updates uniquely imply an underlying Hebbian computation? We identify an alternative, emergent route. We show that near stationarity, L2 weight decay generically drives the \emph{learning-signal} component of many update rules to align with a Hebbian direction, with alignment increasing monotonically with decay strength. This Hebbian-like signature is not specific to SGD and can arise even for non-learning or random update rules long before learning has ceased. We further show that stochastic noise in the learning signal can induce anti-Hebbian alignment, yielding a simple tradeoff with weight decay and a phase boundary in regression settings. These mechanisms do not replace standard Hebbian theory; they can coexist with genuine Hebbian plasticity and complicate the interpretation of synaptic measurements, motivating experiments that distinguish mechanistic Hebbian computation from emergent Hebbian signatures.

Figures

Figures reproduced from arXiv: 2505.18069 by the authors.

Figure 1
Figure 1. Balance of contractive and expansive forces. For deep learning, the noise and weight decay are, respectively, expansion and contraction forces. When they do not balance, the gradient must fill in the gap – if noise outweighs weight decay, the gradient must appear contractive; otherwise, it appears expansive. Similarly, for biology, the Hebbian dynamics is always expansive, and the anti-Hebbian dynamics is always con… view at source ↗
Figure 2
Figure 2. The left shows example weight updates with a high alignment between the learning signal (−∇W ℓ) and the Hebbian update at the end of training with high weight decay, while the right image displays an example update at the end of training with no weight decay which has very low alignment. This figure shows a 20x20 subset of the direction of the Hebbian and learning signal updates for the second layer of an SCE after … view at source ↗
Figure 3
Figure 3. The diagram on the left shows that the trend of weight decay increasing Hebbian alignment of the learning signal is robust across different activation functions. The diagram on the right shows that the trend can generalize to deeper networks. The SCE MLPs were modified by varying the activation functions across Linear, ReLU, Sigmoid, and Tanh (left) and increasing the depth to 6 and layer width to dimension 512 (sho… view at source ↗
Figures from the paper (17 more)
Figure 4
Figure 4. Figure 4: As the noise increases, the Hebbian alignment decreases and higher weight decays lead to higher Hebbian alignment (right). The figure on the left displays a heatmap of the Hebbian alignment of the learning signal at convergence for a number of different additive noises…
Figure 5
Figure 5. Figure 5: Best performance of the model is achieved when it is not Hebbian or anti-Hebbian on average. The left image displays the student validation loss for the experiment in [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: For some activations at low learning rates, there is a sharp jump in Hebbian alignment of the weight update when training with SGD; the size of this jump depends on initial conditions. During this phase, the weight norm decreases monotonically, suggesting the effect is…
Figure 7
Figure 7. Figure 7: The optimal performance seems to be at a critical position between [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: The optimal performance seems to be at a critical position between [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]
Figure 9
Figure 9. Figure 9: This diagram shows the effect of model size on Hebbian alignment [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]
Figure 10
Figure 10. Figure 10: Empirically, the Hebbian alignment of the learning signal increases [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]
Figure 11
Figure 11. Figure 11: This figure shows an example SCE run with an identical training [PITH_FULL_IMAGE:figures/full_fig_p023_11.png]
Figure 12
Figure 12. Figure 12: The alignment decreases as the fraction of parameters of the standard [PITH_FULL_IMAGE:figures/full_fig_p023_12.png]
Figure 13
Figure 13. Figure 13: Again, there is a clear trend that even for the Adam optimizer, as [PITH_FULL_IMAGE:figures/full_fig_p024_13.png]
Figure 14
Figure 14. Figure 14: Additive noise to the input can also lead to anti-Hebbian learning. [PITH_FULL_IMAGE:figures/full_fig_p024_14.png]
Figure 15
Figure 15. Figure 15: Comparison of Hebbian alignment for learning signal vs. full weight [PITH_FULL_IMAGE:figures/full_fig_p025_15.png]
Figure 16
Figure 16. Figure 16: Other regularization techniques have a variety of effects on the Heb [PITH_FULL_IMAGE:figures/full_fig_p026_16.png]
Figure 17
Figure 17. Figure 17: With weight decay, even after the first epoch ( [PITH_FULL_IMAGE:figures/full_fig_p027_17.png]
Figure 18
Figure 18. Figure 18: No standard interpretation of Hebbian learning produces alignment with SGD at convergence. The plots above show a different learning setup than the stan￾dard SCE; rather than training with SGD and computing the alignment of the learning signal with a Hebbian update at…
Figure 19
Figure 19. Figure 19: The full weight update of neurons in the neural network strongly [PITH_FULL_IMAGE:figures/full_fig_p028_19.png]
Figure 20
Figure 20. Figure 20: Early into training neurons often take on roles where their updates [PITH_FULL_IMAGE:figures/full_fig_p028_20.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

31 extracted references · 27 canonical work pages

  1. [1]

    Hebbian and anti-hebbian spike-timing-dependent plasticity of human cortico-cortical connections

    Giacomo Koch, Viviana Ponzo, Francesco Di Lorenzo, Carlo Caltagirone, and Domenica Veniero. Hebbian and anti-hebbian spike-timing-dependent plasticity of human cortico-cortical connections. Journal of Neuroscience , 33(23):9725--9733, 2013

    work page 2013

  2. [2]

    Hebbian plasticity requires compensatory processes on multiple timescales

    Friedemann Zenke and Wulfram Gerstner. Hebbian plasticity requires compensatory processes on multiple timescales. Philosophical transactions of the royal society B: biological sciences , 372(1715):20160259, 2017

    work page 2017

  3. [3]

    A mechanism for the hebb and the anti-hebb processes underlying learning and memory

    John Lisman. A mechanism for the hebb and the anti-hebb processes underlying learning and memory. Proceedings of the National Academy of Sciences , 86(23):9574--9578, 1989

    work page 1989

  4. [4]

    Anti-hebbian long-term potentiation in the hippocampal feedback inhibitory circuit

    Karri P Lamsa, Joost H Heeroma, Peter Somogyi, Dmitri A Rusakov, and Dimitri M Kullmann. Anti-hebbian long-term potentiation in the hippocampal feedback inhibitory circuit. Science , 315(5816):1262--1266, 2007

    work page 2007

  5. [5]

    The organization of behavior: A neuropsychological theory

    Donald Olding Hebb. The organization of behavior: A neuropsychological theory . Psychology press, 2005

    2005

  6. [6]

    Learning representations by back-propagating errors

    David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back-propagating errors. nature , 323(6088):533--536, 1986

    work page 1986

  7. [7]

    Theories of error back-propagation in the brain

    James CR Whittington and Rafal Bogacz. Theories of error back-propagation in the brain. Trends in cognitive sciences , 23(3):235--250, 2019

    work page 2019

  8. [8]

    Backpropagation and the brain

    Timothy P Lillicrap, Adam Santoro, Luke Marris, Colin J Akerman, and Geoffrey Hinton. Backpropagation and the brain. Nature Reviews Neuroscience , 21(6):335--346, 2020

    work page 2020

Show all 31 references
  1. [9]

    Spike timing--dependent plasticity: a hebbian learning rule

    Natalia Caporale and Yang Dan. Spike timing--dependent plasticity: a hebbian learning rule. Annu. Rev. Neurosci. , 31(1):25--46, 2008

    2008

  2. [10]

    Spike-timing-dependent synaptic plasticity depends on dendritic location

    Robert C Froemke, Mu-ming Poo, and Yang Dan. Spike-timing-dependent synaptic plasticity depends on dendritic location. Nature , 434(7030):221--225, 2005

    2005

  3. [11]

    Neuromodulation of spike-timing-dependent plasticity: past, present, and future

    Zuzanna Brzosko, Susanna B Mierau, and Ole Paulsen. Neuromodulation of spike-timing-dependent plasticity: past, present, and future. Neuron , 103(4):563--581, 2019

    2019

  4. [12]

    Simplified neuron model as a principal component analyzer

    Erkki Oja. Simplified neuron model as a principal component analyzer. Journal of Mathematical Biology , 15:267–--273, 1982

    1982

  5. [13]

    Synaptic plasticity: taming the beast

    Larry F Abbott and Sacha B Nelson. Synaptic plasticity: taming the beast. Nature neuroscience , 3(11):1178--1183, 2000

    2000

  6. [14]

    Synaptic plasticity forms and functions

    Jeffrey C Magee and Christine Grienberger. Synaptic plasticity forms and functions. Annual review of neuroscience , 43(1):95--117, 2020

    2020

  7. [15]

    Backpropagation without weight transport

    John F Kolen and Jordan B Pollack. Backpropagation without weight transport. In Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94) , volume 3, pages 1375--1380. IEEE, 1994

    1994

  8. [16]

    Equivalence of backpropagation and contrastive hebbian learning in a layered network

    Xiaohui Xie and H Sebastian Seung. Equivalence of backpropagation and contrastive hebbian learning in a layered network. Neural computation , 15(2):441--454, 2003

    2003

  9. [17]

    Generalization of equilibrium propagation to vector field dynamics

    Benjamin Scellier, Anirudh Goyal, Jonathan Binas, Thomas Mesnard, and Yoshua Bengio. Generalization of equilibrium propagation to vector field dynamics. arXiv preprint arXiv:1808.04873 , 2018

    2018 arXiv

  10. [18]

    Biologically-plausible learning algorithms can scale to large datasets

    Will Xiao, Honglin Chen, Qianli Liao, and Tomaso Poggio. Biologically-plausible learning algorithms can scale to large datasets. In International Conference on Learning Representations, (ICLR 2019) , 2019

    2019

  11. [19]

    Equivalence of equilibrium propagation and recurrent backpropagation

    Benjamin Scellier and Yoshua Bengio. Equivalence of equilibrium propagation and recurrent backpropagation. Neural Computation , 31(2):312--329, 2019

    2019

  12. [20]

    Towards scaling difference target propagation by learning backprop targets

    Maxence M Ernoult, Fabrice Normandin, Abhinav Moudgil, Sean Spinney, Eugene Belilovsky, Irina Rish, Blake Richards, and Yoshua Bengio. Towards scaling difference target propagation by learning backprop targets. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari...

    2022

  13. [21]

    Heterosynaptic circuits are universal gradient machines, 2025

    Liu Ziyin, Isaac Chuang, and Tomaso Poggio. Heterosynaptic circuits are universal gradient machines, 2025

    2025

  14. [22]

    Formation of representations in neural networks

    Liu Ziyin, Isaac Chuang, Tomer Galanti, and Tomaso Poggio. Formation of representations in neural networks. arXiv preprint arXiv:2410.03006 , 2024

    2024 arXiv

  15. [23]

    Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex

    Elie L Bienenstock, Leon N Cooper, and Paul W Munro. Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex. Journal of Neuroscience , 2(1):32--48, 1982

    1982

  16. [24]

    Prevalence of neural collapse during the terminal phase of deep learning training

    Vardan Papyan, XY Han, and David L Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences , 117(40):24652--24663, 2020

    2020

  17. [25]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009

    2009

  18. [26]

    The janus effects of sgd vs gd: high noise and low rank

    Mengjia Xu, Tomer Galanti, Akshay Rangamani, Lorenzo Rosasco, and Tomaso Poggio. The janus effects of sgd vs gd: high noise and low rank. CBMM Memmos , 2023

    2023

  19. [27]

    Direct feedback alignment provides learning in deep neural networks

    Arild N kland. Direct feedback alignment provides learning in deep neural networks. Advances in neural information processing systems , 29, 2016

    2016

  20. [28]

    Sensitivity to perturbations in vivo implies high noise and suggests rate coding in cortex

    Michael London, Arnd Roth, Lisa Beeren, Michael H \"a usser, and Peter E Latham. Sensitivity to perturbations in vivo implies high noise and suggests rate coding in cortex. Nature , 466(7302):123--127, 2010

    2010

  21. [29]

    Noise and fluctuation of finite learning rate stochastic gradient descent, 2021

    Kangqiao Liu, Liu Ziyin, and Masahito Ueda. Noise and fluctuation of finite learning rate stochastic gradient descent, 2021

    2021

  22. [30]

    Bailey, Maurizio Giustetto, Yan-You Huang, and et al

    Craig. Bailey, Maurizio Giustetto, Yan-You Huang, and et al. Is heterosynaptic modulation essential for stabilizing hebbian plasticity and memory. Nature Reviews Neuroscience , 1:11--20, 2000

    2000

  23. [31]

    Altered heterosynaptic plasticity impairs visual discrimination learning in adenosine a1 receptor knock-out mice

    Renee Chasse, Alexey Malyshev, Roslyn Holly Fitch, and Maxim Volgushev. Altered heterosynaptic plasticity impairs visual discrimination learning in adenosine a1 receptor knock-out mice. Journal of Neuroscience , 41(21):4631--4640, 2021

    2021

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.