Pith. sign in

REVIEW 2 major objections 2 minor 1 cited by

Supervised Hebbian learning in Deep Counterstream Associative Networks

T0 review · 2 major / 2 minor · reviewed 2026-06-30 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Deep associative networks can be trained to high MNIST accuracy with local Hebbian rules by sending simultaneous opposing activity waves from input and output layers.

desk verdict This sketches a counterstream wave mechanism for local Hebbian training in deep nets but the MNIST performance claim has no supporting numbers or protocol. read the letter →

arxiv 2606.29528 v1 pith:JIHLPVA3 submitted 2026-06-28 cs.NE

Andreas Knoblauch This is my paper · ORCID
classification cs.NE
keywords HebbianlearningdeepneuralnetworkssupervisedassociativeMNISTcounterstreambackpropagationalternativelocalrules
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces supervised counterstream learning as a biologically inspired alternative to error backpropagation for training deep neural networks. Instead of symmetric weights or separate error channels, the method starts two activity waves at once, one from the input layer and one from the output layer, that travel toward each other. When the waves meet in a hidden layer, simple local Hebbian-type rules link the input patterns to the target patterns bidirectionally. This process repeats over time and reduces classification errors. On binarized MNIST the resulting networks reach test accuracy comparable to standard backpropagation architectures even without full hyperparameter tuning.

What carries the argument

The counterstream mechanism in which opposing activity waves meet in hidden layers to enable bidirectional Hebbian linking of input and target patterns.

What would settle it

Training runs that disable simultaneous initiation of the opposing waves or prevent their meeting in hidden layers would show whether accuracy on binarized MNIST falls to chance levels or stays comparable to backpropagation.

Watch

Extended reading notes

Core claim

The central claim is that deep associative networks can be trained in a supervised manner by initiating two activity waves simultaneously at the input and output layers that travel in opposite directions to meet in hidden layers, where local Hebbian-type learning rules then link the corresponding activity pattern sequences bidirectionally and thereby decrease error rates over training time, all without requiring symmetric connectivity or a separate processing channel for error signals.

Load-bearing premise

The method assumes that two activity waves can be started at the same time from the input and output layers, travel in opposite directions, and meet in hidden layers so that local Hebbian rules can correctly associate the patterns.

Editorial extensions

If this is right

  • Training becomes possible using only local Hebbian rules and recognition of output errors without symmetric connectivity.
  • The same forward activity channel carries both recognition signals and correcting target activity.
  • No separate mathematical operations such as subtractions or inversions are required for learning.
  • Deep hierarchies can reduce error rates through repeated bidirectional pattern linking.
  • Test accuracy on binarized MNIST reaches levels comparable to more complex architectures despite incomplete hyperparameter search.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If biological networks can generate and align such opposing waves, the mechanism offers a candidate explanation for supervised learning in cortex without explicit backpropagation circuitry.
  • The same wave-meeting process might be tested on non-image data to determine whether the accuracy result generalizes beyond binarized MNIST.
  • Removing the requirement for symmetric weights could simplify hardware implementations of deep networks that use only local updates.
  • Extending the method to recurrent or spiking networks would test whether the counterstream idea remains effective when timing of wave arrival becomes variable.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes supervised counterstream learning for deep associative networks as a biologically plausible alternative to backpropagation. Two activity waves are initiated simultaneously at the input and output layers, propagate in opposite directions through the same channel, and meet in a hidden layer; local Hebbian-type rules then link the patterns bidirectionally to reduce error. The abstract asserts that this achieves high test accuracy on binarized MNIST comparable to more demanding architectures, despite the method's simplicity and an incomplete hyperparameter optimization.

Significance. If the empirical performance is demonstrated with quantitative results and the wave-meeting mechanism is shown to operate without unstated global coordination or pre-wired structure, the approach could provide a simpler local-learning alternative that avoids weight symmetry and separate error channels. The emphasis on purely local Hebbian updates is a potential strength, but the current lack of supporting data prevents assessment of whether the result would meaningfully advance the field.

major comments (2)
  1. [Abstract] Abstract: the central empirical claim that 'a high test accuracy is achieved on the (binarized) MNIST data set that is comparable to more demanding architectures' is unsupported; no numerical accuracy values, error bars, baseline comparisons, training protocol, or hyperparameter details are supplied, leaving the claim without visible evidence.
  2. [Abstract] Abstract: the description that 'two activity waves are initiated at the same time in input and output layers and then traveling in opposite directions to meet in one of the hidden layers' supplies no mechanism for simultaneous initiation, layer selection, or synchronization per input-target pair. This coordination assumption is load-bearing for the claim that only local Hebbian rules and 'recognition of errors' suffice without additional global signals or network structure.
minor comments (2)
  1. [Abstract] Abstract contains the repeated phrase 'high high test accuracy' and the misspelling 'optimzation'.
  2. [Abstract] The abstract states that hyperparameter optimization is incomplete but provides no information on which parameters were varied or the search method used.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We agree that the abstract requires supporting numerical evidence and will revise it to include specific accuracy figures, training details, and comparisons. We will also expand the description of wave initiation to address synchronization concerns while maintaining the focus on local rules. Point-by-point responses follow.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central empirical claim that 'a high test accuracy is achieved on the (binarized) MNIST data set that is comparable to more demanding architectures' is unsupported; no numerical accuracy values, error bars, baseline comparisons, training protocol, or hyperparameter details are supplied, leaving the claim without visible evidence.

    Authors: We accept the criticism. The abstract summarizes results without quantitative support for conciseness. The full manuscript contains the experimental outcomes on binarized MNIST. In revision we will update the abstract with the reported test accuracy, error bars if available, baseline comparisons, and a brief note on the training protocol and incomplete hyperparameter search to make the claim directly supported. revision: yes

  2. Referee: [Abstract] Abstract: the description that 'two activity waves are initiated at the same time in input and output layers and then traveling in opposite directions to meet in one of the hidden layers' supplies no mechanism for simultaneous initiation, layer selection, or synchronization per input-target pair. This coordination assumption is load-bearing for the claim that only local Hebbian rules and 'recognition of errors' suffice without additional global signals or network structure.

    Authors: The referee is correct that the abstract provides no explicit mechanism. The manuscript assumes coordinated presentation of input and target during supervised training to start the opposing waves, with meeting occurring via propagation timing in the associative network. We will revise the methods section to clarify this assumption, discuss whether it can be realized with purely local signals, and note any remaining requirements for global coordination as a limitation rather than claiming it is fully avoided. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity detected; derivation is self-contained.

full rationale

The paper proposes a novel supervised counterstream Hebbian mechanism in deep associative networks, with the central claim resting on empirical MNIST results and local learning rules applied to oppositely propagating activity waves. No load-bearing steps reduce by construction to fitted parameters, self-citations, or renamed prior results; the abstract and description present the wave-meeting and bidirectional linking as a new assumption set independent of the target performance metric. This matches the default expectation of non-circularity for a mechanism paper with external benchmark evaluation.

Assumptions & free parameters 0 free parameters · 0 assumptions · 1 invented entities

Abstract-only review provides insufficient detail to enumerate specific free parameters or axioms; the method implicitly relies on the feasibility of simultaneous opposing waves and local Hebbian updates being sufficient for supervised learning.

invented entities (1)
  • counterstream activity waves
    purpose: to backpropagate target activity through the forward activity channel
    Introduced as the core mechanism for error correction without separate channels.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Supervised Hebbian learning in Deep Counterstream Associative Networks." pith.science (2026). https://pith.science/paper/JIHLPVA3

@misc{pith2026260629528,
  author       = {Pith},
  title        = {Pith review of: Supervised Hebbian learning in Deep Counterstream Associative Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JIHLPVA3}},
  note         = {Machine review of arXiv:2606.29528}
}
read the original abstract

Modern machine learning applications employ deep neural networks training with the error backpropagation algorithm. Although this algorithm is very effective, it lacks biological realism. For example, backpropagation requires symmetric connectivity, and a separate neural processing channel for error signals. Prior works have therefore proposed a number of more realistic alternatives for error backpropagation. However, most of them still suffer from demanding preassumptions that may be not fulfilled in the real brain, for example, they often still require either symmetric connectivity or two separate processing channels, and often require also special mathematical operations like subtractions or function inversions. Here I propose supervised counterstream learning in deep associative networks as a simpler approach that requires only recognition of errors during training, and then backpropagates correcting target activity through the same activity channel as used for forward propagation. For this, two activity waves are initiated at the same time in input and output layers and then traveling in opposite directions to meet in one of the hidden layers. By employing simple local Hebbian-type learning rules, the corresponding activity pattern sequences get linked bidirectionally, thereby decreasing error rates over time. Despite its simplicity and an incomplete hyperparameter optimzation, a high high test accuracy is achieved on the (binarized) MNIST data set that is comparable to more demanding architectures.

Figures

Figures reproduced from arXiv: 2606.29528 by the authors.

Figure 1
Figure 1. Upating synaptic weights by backpropagation learning (1.2,1.3) requires a “forward-pass” to [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Threshold-based binarization methods to convert gray-scale or RGB images into binary pattern [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Rank-based binarization methods to convert gray-scale or RGB images to binary pattern [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Binary codeword matrix for multi inverval block coding using either pixel values ( [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Deep Counterstream Associative Network: The network consists of an input layer [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Block structure and topography of hidden layers: Each hidden layer [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Operation modes of Deep Counterstream Associative Network. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Accuracy for MNIST test data as function of learning strength [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Accuracy for MNIST test data as function of anatomical connectivity [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Accuracy for MNIST test data as function of the receptive field (RF) size [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: Accuracy for MNIST test data as function of block number [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 12
Figure 12. Figure 12: Accuracy for MNIST test data as function of number of active neurons [PITH_FULL_IMAGE:figures/full_fig_p015_12.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Constrained Hebbian Learning Supports Efficient Representational Allocation under Structural Constraints

    cs.LG 2026-07 conditional novelty 4.0 of 10

    A constrained Hebbian rule produces audiovisual representations with lower task-information cost (retained input information per unit of task-relevant information) than sparse backpropagation and DDTP at comparable ac...

Reference graph

Works this paper leans on

113 extracted references · 8 canonical work pages · cited by 1 Pith paper

  1. [1]

    Abel and S

    R. Abel and S. Ullman. Biologically inspired learning model for instructed vision. InAdvances in Neural Information Processing Systems, volume 37, 2024

    2024

  2. [2]

    A study of retrieval algorithms of sparse mes- sages in networks of neural cliques

    Ala Aboudib, Vincent Gripon, and Xiaoran Jiang. A study of retrieval algorithms of sparse mes- sages in networks of neural cliques. InProceedings of the 6th International Conference on Ad- vanced Cognitive Technologies and Applications (Cognitive 2014), pages 140–146, May 2014

    2014

  3. [3]

    Storing sparse messages in networks of neural cliques.IEEE Transactions on Neural Networks and Learning Systems, 25:980–989, 2014

    Behrooz Kamary Aliabadi, Claude Berrou, Vincent Gripon, and Xiaoran Jiang. Storing sparse messages in networks of neural cliques.IEEE Transactions on Neural Networks and Learning Systems, 25:980–989, 2014

    2014

  4. [4]

    Alkam, E

    T. Alkam, E. Tarshizi, and A.H. Van Benschoten. Reinforcement learning at the interface of artificial intelligence and cognitive science.Neuroscience, 585:289–312, 2025

    2025

  5. [5]

    Bishop.Pattern recognition and machine learning.Springer, New York, 2006

    C.M. Bishop.Pattern recognition and machine learning.Springer, New York, 2006

    2006

  6. [6]

    Braitenberg

    V . Braitenberg. Cell assemblies in the cerebral cortex. In R. Heim and G. Palm, editors,Lec- ture notes in biomathematics (21). Theoretical approaches to complex systems., pages 171–188. Springer-Verlag, Berlin Heidelberg New York, 1978

    1978

  7. [7]

    Braitenberg and A

    V . Braitenberg and A. Sch ¨uz.Anatomy of the cortex. Statistics and geometry.Springer-Verlag, Berlin, 1991

    1991

  8. [8]

    A. E. Bryson and Y . C. Ho.Applied Optimal Control: Optimization, Estimation, and Control. Blaisdell, New York, 1969

    1969

Show all 113 references
  1. [9]

    Buckingham and D.J

    J.T. Buckingham and D.J. Willshaw. Performance characteristics of the associative net.Network: Computation in Neural Systems, 3:407–414, 1992

    1992

  2. [10]

    Cauwenberghs

    G. Cauwenberghs. A fast stochastic error-descent algorithm for supervised learning and optimiza- tion. In S. J. Hanson, J. D. Cowan, and C. L. Giles, editors,Advances in Neural Information Processing Systems 5 (NIPS 1992), pages 244–251, San Mateo, CA, 1993. Morgan Kaufmann

    1992

  3. [11]

    L. Deng. The MNIST database of handwritten digit images for machine learning research.IEEE Signal Processing Magazine, 29(6):141–142, 2012

    2012

  4. [12]

    Felleman and D.C

    D.J. Felleman and D.C. Van Essen. Distributed hierarchical processing in the primate cerebral cortex.Cerebral Cortex, 1:1–47, 1991

    1991

  5. [13]

    Nearest neighbour search using binary neural networks

    Demetrio Ferro, Vincent Gripon, and Xiaoran Jiang. Nearest neighbour search using binary neural networks. InProceedings of the International Joint Conference on Neural Networks (IJCNN), July 2016

    2016

  6. [14]

    Fiete, M.S

    I.R. Fiete, M.S. Fee, and H.S. Seung. Model of birdsong learning based on gradient estimation by dynamic perturbation of neural conductances.Journal of Neurophysiology, 98(4):2038–2057, 2007

  7. [15]

    Fremaux and W

    N. Fremaux and W. Gerstner. Neuromodulated spike-timing-dependent plasticity, and theory of three-factor learning rules.Frontiers in Neural Circuits, 9:85, 2016

    2016

  8. [16]

    Friston, C.D

    K.J. Friston, C.D. Frith, and R.S.J. Frackowiak. Principal component analysis learning algorithms: A neurobiological analysis.Proceedings of the Royal Society of London. Series B: Biological Sciences, 254(1339):47–54, 1993

    1993

  9. [17]

    Gripon and C

    V . Gripon and C. Berrou. Sparse neural networks with large learning diversity.IEEE Transactions on Neural Networks, 22(7):1087–1096, 2011

    2011

  10. [18]

    Gripon and C

    V . Gripon and C. Berrou. Nearly-optimal associative memories based on distributed constant weight codes. InProceedings of the IEEE Information Theory and Applications Workshop (ITA), pages 269–273, 2012. 21

    2012

  11. [19]

    Maximum likelihood associative memories

    Vincent Gripon and Michael Rabbat. Maximum likelihood associative memories. InProceedings of the IEEE Information Theory Workshop (ITW), pages 1–5, September 2013

    2013

  12. [20]

    Hebb.The organization of behavior

    D.O. Hebb.The organization of behavior. A neuropsychological theory.Wiley, New York, 1949

    1949

  13. [21]

    M.H. Hennig. Theoretical models of synaptic short term plasticity.Frontiers in Computational Neuroscience, 7(45):1–10, 2013

    2013

  14. [22]

    Holtmaat and K

    A. Holtmaat and K. Svoboda. Experience-dependent structural synaptic plasticity in the mam- malian brain.Nature Reviews Neuroscience, 10:647–658, 2009

    2009

  15. [23]

    Hopfield

    J.J. Hopfield. Neural networks and physical systems with emergent collective computational abil- ities.Proceedings of the National Academy of Science, USA, 79:2554–2558, 1982

    1982

  16. [24]

    Inoue, F

    P. Inoue, F. R ¨ohrbein, and A. Knoblauch. Guiding sparse neural networks with neurobiological principles to elicit biologically plausible representations.arxiv:2603.03234, 2026

    2026

  17. [25]

    Jabri and B

    M. Jabri and B. Flower. Weight perturbation: An optimal architecture and learning technique for analog vlsi feedforward and recurrent multilayer networks.IEEE Transactions on Neural Networks, 3(1):154–157, 1992

    1992

  18. [26]

    Kanerva.Sparse Distributed Memory.MIT Press, Cambridge, MA, 1988

    P. Kanerva.Sparse Distributed Memory.MIT Press, Cambridge, MA, 1988

    1988

  19. [27]

    I. Kanter. Potts-glass models of neural networks.Physical Review A, 37(7):2739—-2742, 1988

    1988

  20. [28]

    Kingma and J.L

    D.P. Kingma and J.L. Ba. Adam: A method for stochastic optimization. In Y . Bengio and Y . LeCun, editors,3rd Proceedings of the International Conference on Learning Representations (ICLR), arXiv:1412.6980v9, 2015

    2015 arXiv

  21. [29]

    Knoblauch

    A. Knoblauch. On the computational benefits of inhibitory neural associative networks. HRI-EU Report 07-05, Honda Research Institute Europe GmbH, D-63073 Offenbach/Main, Germany, May 2007

    2007

  22. [30]

    Knoblauch

    A. Knoblauch. Neural associative memory and the Willshaw-Palm probability distribution.SIAM Journal on Applied Mathematics, 69(1):169–196, 2008

    2008

  23. [31]

    Knoblauch

    A. Knoblauch. Neural associative networks with optimal bayesian learning. HRI-EU Report 09- 02, Honda Research Institute Europe GmbH, D-63073 Offenbach/Main, Germany, May 2009

    2009

  24. [32]

    Knoblauch

    A. Knoblauch. The role of structural plasticity and synaptic consolidation for memory and am- nesia in a model of cortico-hippocampal interplay. In J. Mayor, N. Ruh, and K. Plunkett, editors, Connectionist Models of Behavior and Cognition II: Proceedings of the 11th Neural Com...

    2009

  25. [33]

    Knoblauch

    A. Knoblauch. Optimal synaptic learning in non-linear associative memory. InProceedings of the International Joint Conference on Neural Networks (IJCNN), pages 3205–3211, Barcelona, Spain, 2010. IEEE World Congress on Computational Intelligence (WCCI)

    2010

  26. [34]

    Knoblauch

    A. Knoblauch. Neural associative memory with optimal bayesian learning.Neural Computation, 23(6):1393–1451, 2011

    2011

  27. [35]

    Knoblauch

    A. Knoblauch. Method and device for realizing an associative memory based on inhibitory neural networks., 2012. European Patent EP 2,012,260, filed June 22, 2007, issued April 11, 2012; U.S. Patent No. 8,335,752, issued December 18, 2012; JP Patent No. 5297451, issued June 21, 2013

    2012

  28. [36]

    Knoblauch

    A. Knoblauch. Impact of structural plasticity on memory formation and decline. In A. van Ooyen and M. Butz, editors,Rewiring the Brain: A Computational Approach to Structural Plasticity in the Adult Brain., pages 361–386, London, UK, 2017. Elsevier/Academic Press

    2017

  29. [37]

    Knoblauch

    A. Knoblauch. Neural auto-association with optimal Bayesian learning.arxiv:2412.18349, 2024

    2024

  30. [38]

    Knoblauch.Intelligente Lernende Systeme.Springer-Vieweg, Wiesbaden, 2026

    A. Knoblauch.Intelligente Lernende Systeme.Springer-Vieweg, Wiesbaden, 2026. 22

    2026

  31. [39]

    Knoblauch, F

    A. Knoblauch, F. Hauser, M.-O. Gewaltig, E. K ¨orner, and G. Palm. Does spike-timing-dependent synaptic plasticity couple or decouple neurons firing in synchrony?Frontiers in Computational Neuroscience, 6(55):1–27, 2012

    2012

  32. [40]

    Knoblauch, E

    A. Knoblauch, E. K ¨orner, U. K ¨orner, and F.T. Sommer. Structural plasticity has high memory capacity and can explain graded amnesia, catastrophic forgetting, and the spacing effect.PLoS ONE, 9(5):e96485:1–19, 2014

    2014

  33. [41]

    Knoblauch and G

    A. Knoblauch and G. Palm. Scene segmentation by spike synchronization in reciprocally con- nected visual areas. I. Local effects of cortical feedback.Biological Cybernetics, 87(3):151–167, 2002

    2002

  34. [42]

    Knoblauch and G

    A. Knoblauch and G. Palm. Scene segmentation by spike synchronization in reciprocally con- nected visual areas. II. Global assemblies and synchronization on larger space and time scales. Biological Cybernetics, 87(3):168–184, 2002

    2002

  35. [43]

    Knoblauch and G

    A. Knoblauch and G. Palm. Bidirectional associative memory with block coding: A comparison of iterative retrieval methods. In I.V . Tetko, V . Kurkova, P. Karpov, and F. Theis, editors,Artificial Neural Networks and Machine Learning – ICANN 2019: Theoretical Neural Computation...

    2019

  36. [44]

    Knoblauch and G

    A. Knoblauch and G. Palm. Iterative retrieval and block coding in autoassociative and heteroasso- ciative memory.Neural Computation, 32(1):205–260, 2020

    2020

  37. [45]

    Knoblauch, G

    A. Knoblauch, G. Palm, and F.T. Sommer. Memory capacities for synaptic and structural plasticity. Neural Computation, 22(2):289–341, 2010

    2010

  38. [46]

    Knoblauch and F.T

    A. Knoblauch and F.T. Sommer. Structural plasticity, effectual connectivity, and memory in cortex. Frontiers in Neuroanatomy, 10(63):1–20, 2016

    2016

  39. [47]

    B. Kosko. Bidirectional associative memories.IEEE Transactions on Systems, Man, and Cyber- netics, 18:49–60, 1988

    1988

  40. [48]

    Krotov and J.J

    D. Krotov and J.J. Hopfield. Unsupervised learning by competing hidden units.Proceedings of the National Academy of Science, USA, 116(16):7723–7731, 2019

    2019

  41. [49]

    Kryzhanovsky and V .M

    B.V . Kryzhanovsky and V .M. Kryzhanovsky. A binary pattern classification using Potts model. Optical Memory and Neural Networks (Information Optics), 17(4):308–316, 2008

    2008

  42. [50]

    Kryzhanovsky, L.B

    B.V . Kryzhanovsky, L.B. Litinskii, and A.L. Mikaelian. Vector-neuron models of associa- tive memory. In2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541; IJCNN-04), Volume 2, pages 909–914, 2004

    2004

  43. [51]

    Kryzhanovsky, B

    V . Kryzhanovsky, B. Kryzhanovsky, and A. Fonarev. Application of potts-model perceptron for binary patterns identification. In V . Kurkova-Pohlova, R. Neruda, and J. Koutnik, editors,Proceed- ings of the 18th International Conference on Artificial Neural Networks (ICANN), Par...

    2008

  44. [52]

    A. Lansner. Associative memory models: from the cell-assembly theory to biophysically detailed cortex simulations.Trends in Neurosciences, 32(3):178–186, 2009

    2009

  45. [53]

    Lansner and ¨O

    A. Lansner and ¨O. Ekeberg. A one-layer feedback artificial neural network with a Bayesian learn- ing rule.International Journal of Neural Systems, 1(1):77–87, 1989

    1989

  46. [54]

    Lansner, N.B

    A. Lansner, N.B. Ravichandran, A. Knoblauch, and P. Herman. Benchmarking Hebbian learning rules for associative memory.arxiv:2401.00335v2, 2025

    2025

  47. [55]

    LeCun, L

    Y . LeCun, L. Bottou, Y . Bengio, and P. Haffner. Gradient-based learning applied to document recognition.Proceedings of the IEEE, 86(11):2278–2324, 1998. 23

    1998

  48. [56]

    D.-H. Lee, S. Zhang, A. Fischer, and Y . Bengio. Difference target propagation. InMachine Learning and Knowledge Discovery in Databases (ECML PKDD), pages 498–515, 2015

    2015

  49. [57]

    Lillicrap, D

    T.P. Lillicrap, D. Cownden, D.B. Tweed, and C.J. Akerman. Random synaptic feedback weights support error backpropagation for deep learning.Nature Communications, 7(1):13276, 2016

    2016

  50. [58]

    Lillicrap, A

    T.P. Lillicrap, A. Santoro, L. Marris, C.J. Akerman, and G. Hinton. Backpropagation and the brain. Nature Reviews Neuroscience, 21:335–346, 2020

    2020

  51. [59]

    Bistable, irregular firing and population oscillations in a modular attractor memory network.PLoS Computational Biology, 6(6):e1000803, 2010

    Mikael Lundqvist, Albert Compte, and Anders Lansner. Bistable, irregular firing and population oscillations in a modular attractor memory network.PLoS Computational Biology, 6(6):e1000803, 2010

    2010

  52. [60]

    Attractor dynamics in a modular network model of neocortex.Network: Computation in Neural Systems, 17(3):253–276, 2006

    Mikael Lundqvist, Markus Rehn, Mikael Djurfeldt, and Anders Lansner. Attractor dynamics in a modular network model of neocortex.Network: Computation in Neural Systems, 17(3):253–276, 2006

    2006

  53. [61]

    McClelland, B.L

    J.L. McClelland, B.L. McNaughton, and R.C. O’Reilly. Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connec- tionist models of learning and memory.Psychological Review, 102(3):419–457, 1995

    1995

  54. [62]

    A.A. Minai. Covariance learning of correlated patterns in competitive networks.Neural Compu- tation, 9(3):667–681, 1997

    1997

  55. [63]

    Ming and H

    G.L. Ming and H. Song. Adult neurogenesis in the mammalian central nervous system.Annu. Rev. Neurosci., 28:223–250, 2005

    2005

  56. [64]

    A. Nokland. Direct feedback alignment provides learning in deep neural networks. InAdvances in Neural Information Processing Systems, volume 29, 2016

    2016

  57. [65]

    E. Oja. Simplified neuron model as a principal component analyzer.Journal of Mathematical Biology, 15(3):267–273, 1982

    1982

  58. [66]

    E. Oja. Principal components, minor components, and linear neural networks.Neural Networks, 5(6):927–935, 1992

    1992

  59. [67]

    Ororbia and A

    A.G. Ororbia and A. Mali. Biologically motivated algorithms for propagating local target rep- resentations. InProceedings of the AAAI Conference on Artificial Intelligence, volume 33(01), pages 4651–4658, 2019

    2019

  60. [68]

    G. Palm. On associative memories.Biological Cybernetics, 36:19–31, 1980

    1980

  61. [69]

    Palm.Neural Assemblies

    G. Palm.Neural Assemblies. An Alternative Approach to Artificial Intelligence.Springer, Berlin, 1982

    1982

  62. [70]

    G. Palm. Computing with neural networks.Science, 235:1227–1228, 1987

    1987

  63. [71]

    G. Palm. On associative memories. In E. Caianiello, editor,Physics of Cognitive Processes, pages 380–422. World Scientific, 1987

    1987

  64. [72]

    G. Palm. On the information storage capacity of local learning rules.Neural Computation, 4(5):703–711, 1992

    1992

  65. [73]

    G. Palm. Neural associative memories and sparse coding.Neural Networks, 37:165–171, 2013

    2013

  66. [74]

    G. Palm, A. Knoblauch, F. Hauser, and A. Sch¨uz. Cell assemblies in the cerebral cortex.Biological Cybernetics, 108(5):559–572, 2014

    2014

  67. [75]

    G. Palm, F. Schwenker, and F.T. Sommer. Associative memory networks and sparse similarity preserving codes. In V . Cherkassky, J.H. Friedman, and H. Wechsler, editors,From statistics to neural networks: Theory and pattern recognition applications, NATO ASI Series F, pages 283–

  68. [76]

    Springer Verlag, Berlin, Heidelberg, 1994. 24

    1994

  69. [77]

    Palm and F.T

    G. Palm and F.T. Sommer. Information capacity in recurrent McCulloch-Pitts networks with sparsely coded memory states.Network, 3:177–186, 1992

    1992

  70. [78]

    Palm and F.T

    G. Palm and F.T. Sommer. Associative data storage and retrieval in neural nets. In E. Domany, J.L. van Hemmen, and K. Schulten, editors,Models of Neural Networks III, pages 79–118. Springer- Verlag, New York, 1996

    1996

  71. [79]

    D.B. Parker. Learning-logic: casting the cortex of the human brain in silicon. Technical Report Tr- 47, Center for Computational Research in Economics and Management Science. MIT Cambridge, MA., 1985

    1985

  72. [80]

    Pulverm ¨uller.The neuroscience of language: on brain circuits of words and serial order.Cam- bridge University Press, Cambridge, UK, 2003

    F. Pulverm ¨uller.The neuroscience of language: on brain circuits of words and serial order.Cam- bridge University Press, Cambridge, UK, 2003

    2003

  73. [81]

    Rao and D.H

    R.P. Rao and D.H. Ballard. Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects.Nature Neuroscience, 2(1):79–87, 1999

    1999

  74. [82]

    Ravichandran, A

    N. Ravichandran, A. Lansner, and P. Herman. Spiking representation learning for associative memories.Frontiers in Neuroscience, 18:1439414, 2024

    2024

  75. [83]

    An overview of gradient descent optimization algorithms.arXiv preprint arXiv:1609.04747, 2016

    Sebastian Ruder. An overview of gradient descent optimization algorithms.arXiv preprint arXiv:1609.04747, 2016

    2016 arXiv

  76. [84]

    Rumelhart, G.E

    D.E. Rumelhart, G.E. Hinton, and R.J. Williams. Learning representations by back-propagating errors.Nature, 323(6088):533–536, 1986

    1986

  77. [85]

    Rumelhart, J.L

    D.E. Rumelhart, J.L. McClelland, and PDP Research Group, editors.Parallel distributed pro- cessing, Explorations in the Microstructure of Cognition, Volume 1: Foundations. MIT Press, Cambridge, MA, 1986

    1986

  78. [86]

    Sa-Couto and A

    L. Sa-Couto and A. Wichert. Storing object-dependent sparse codes in a willshaw associative network.Neural Computation, 32(1):136–152, 2020

    2020

  79. [87]

    Schmidgall, J

    S. Schmidgall, J. Achterberg, T. Miconi, L. Kirsch, R. Ziaei, S.P. Hajiseyedrazi, and J. Eshraghian. Brain-inspired learning in artificial neural networks: A review.APL Machine Learning, 2(2):021501, 2024

    2024

  80. [88]

    Schultz, P

    W. Schultz, P. Dayan, and P.R. Montague. A neural substrate of prediction and reward.Science, 275:1593–1599, 1997

    1997

  81. [89]

    Predictive reward signal of dopamine neurons.Journal of Neurophysiology, 80:1–27, 1998

    Wolfram Schultz. Predictive reward signal of dopamine neurons.Journal of Neurophysiology, 80:1–27, 1998

    1998

  82. [90]

    Sejnowski

    T.J. Sejnowski. Storing covariance with nonlinearly interacting neurons.Journal of Mathematical Biology, 4:303–321, 1977

    1977

  83. [91]

    Sommer and G

    F.T. Sommer and G. Palm. Improved bidirectional retrieval of sparse patterns stored by Hebbian learning.Neural Networks, 12:281–297, 1999

    1999

  84. [92]

    G. Song, R. Xu, and J. Lafferty. Convergence and alignment of gradient descent with random backpropagation weights.arXiv:2106.06044, 2021

    2021

  85. [93]

    Steinbuch

    K. Steinbuch. Die Lernmatrix.Kybernetik, 1:36–45, 1961

    1961

  86. [94]

    Stricker, F

    P. Stricker, F. R ¨ohrbein, and A. Knoblauch. Energy-efficient information representation in mnist classification using biologically inspired learning. InProceedings of the 10th bwHPC Sympo- sium: High-performance and data-intensive computing in Baden-W¨urttemberg, pages 13–28...

    2026

  87. [95]

    Stricker, F

    P. Stricker, F. R¨ohrbein, and A. Knoblauch. Weight perturbation and competitive hebbian plasticity for training sparse excitatory neural networks. In2024 International Joint Conference on Neural Networks (IJCNN), pages 1–8, Yokohama, Japan, 2024. 25

    2024

  88. [96]

    Sutton and A.G

    R.S. Sutton and A.G. Barto.Reinforcement Learning: An Introduction.Adaptive Computation and Machine Learning. The MIT Press, Cambridge, MA, 1998

    1998

  89. [97]

    Tang, M.A

    Z. Tang, M.A. Buice, F.A. Chowdhury, B. Doiron, M.R. Joglekar, G.K. Ocker, and E. Shea-Brown. Biologically plausible training mechanisms for self-supervised learning.Frontiers in Computa- tional Neuroscience, 16:789253, 2022

    2022

  90. [98]

    Tsodyks and M.V

    M.V . Tsodyks and M.V . Feigel’man. The enhanced storage capacity in neural networks with low activity level.Europhysics Letters, 6:101–105, 1988

    1988

  91. [99]

    S. Ullman. Sequence seeking and counter streams: A computational model for bidirectional in- formation flow in the visual cortex.Cerebral Cortex, 5(1):1–11, 1995

    1995

  92. [100]

    Ullman, L

    S. Ullman, L. Assif, A. Strugatski, B.-Z. Vatashsky, H. Levy, A. Netanyahu, and A. Yaari. Image interpretation by iterative bottom-up top-down processing.arxiv:2105.05592, 2021

    2021

  93. [101]

    Waydo, A

    S. Waydo, A. Kraskov, R.Q. Quiroga, I. Fried, and C. Koch. Sparse representation in the human medial temporal lobe.Journal of Neuroscience, 26(40):10232–10234, 2006

    2006

  94. [102]

    P. J. Werbos.Beyond regression: new tools for prediction and analysis in the behavioral sciences. PhD thesis, Harvard University, 1974

    1974

  95. [103]

    Whittington and R

    J.C.R. Whittington and R. Bogacz. An approximation of the error backpropagation algorithm in a predictive coding network.Neural Computation, 29(5):1229–1262, 2017

    2017

  96. [104]

    Whittington and R

    J.C.R. Whittington and R. Bogacz. Theories of error back-propagation in the brain.Trends in Cognitive Sciences, 23(3):235–250, 2019

    2019

  97. [105]

    Williams

    R.J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning.Machine Learning, 8:229–256, 1992

    1992

  98. [106]

    Willshaw, O.P

    D.J. Willshaw, O.P. Buneman, and H.C. Longuet-Higgins. Non-holographic associative memory. Nature, 222:960–962, 1969

    1969

  99. [107]

    Willshaw and P

    D.J. Willshaw and P. Dayan. Optimal plasticity in matrix memories: what goes up must come down.Neural Computation, 2:85–93, 1990

    1990

  100. [108]

    Wilson and B.L

    M.A. Wilson and B.L. McNaughton. Reactivation of hippocampal ensemble memories during sleep.Science, 265:676–679, 1994

    1994

  101. [109]

    F.Y . Wu. The potts model.Reviews of Modern Physics, 54:235–268, 1982

    1982

  102. [110]

    Z. Yao, V . Gripon, and M. Rabbat. A gpu-based associative memory using sparse neural net- works. InProceedings of the IEEE International Conference on High-Performance-Computing & Simulation (HPCS), pages 688–692, 2014

    2014

  103. [111]

    H. Zhou. Activation learning by local competitions.arxiv:2209.13400, 2022

    2022

  104. [112]

    Zucker and W.G

    R.S. Zucker and W.G. Regehr. Short-term synaptic plasticity.Annual Review of Physiology, 64:355–405, 2002

    2002

  105. [113]

    Z ¨uge, C

    P. Z ¨uge, C. Klos, and R.-M. Memmesheimer. Weight versus node perturbation learning in tempo- rally extended tasks: Weight perturbation often performs similarly or better.Physical Review X, 13:021006, 2023. 26

    2023

Pith tools

Reviewed June 30, 2026 · model on record in the stance chip above.