Neural auto-association with optimal Bayesian learning

Andreas Knoblauch

REVIEW 4 major objections 4 minor 72 references

Neural auto-association with optimal Bayesian learning

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read In auto-associative memory, the optimal Bayesian rule regains top capacity once noise is re-estimated each step.

desk verdict A credible explanation of the BCPNN-vs-Bayesian anomaly via noise-estimate mismatch and an adaptive noise mechanism, but the headline capacity claim rests on a hand-picked configuration without error bars. read the letter →

arxiv 2412.18349 v1 pith:LXZUZU6G submitted 2024-12-24 cs.NE

Andreas Knoblauch This is my paper · ORCID

classification cs.NE MSC 92B2068T05

keywords associativememoryauto-associationoptimalBayesianlearningBCPNNiterativeretrievaladaptivenoiseestimationstoragecapacity

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks why a theoretically optimal learning rule for associative neural memories can lose to its approximations, and whether the optimum can be restored. In the auto-associative setting (input and output are the same pattern), the Bayesian maximum-likelihood rule is optimal only under naive-Bayes assumptions: independent pattern components, independent query noise, and one-step retrieval. The paper shows that the reported superiority of the BCPNN rule over this "optimal" rule is not an implementation error; it follows from violations of those assumptions, especially patterns with a fixed number of active units, winner-take-all selection, and iterative retrieval in which any fixed noise estimate becomes stale. It then introduces Adaptive Noise Estimation (ANE), which re-estimates noise after each retrieval step, and a core-retrieval schedule under which the Bayesian rule reaches the highest measured capacity, M = 1603 at 90% correct retrieval, above every BCPNN variant tested. The result matters because it separates "the rule is wrong" from "the assumptions are violated," and it shows a concrete way to recover theoretical optimality in recurrent retrieval.

What carries the argument

The engine of the argument is the log-odds decision variable $x_j = \log r_j$, where $r_j = \Pr[u^\mu_j = 1 \mid \tilde u, M(j)] / \Pr[u^\mu_j = 0 \mid \tilde u, M(j)]$; the neuron fires when $x_j \ge 0$. Its weights are functions of the synaptic counters $M_{11}(ij)$, $M_{01}(ij)$, $M_{10}(ij)$, $M_{00}(ij)$ and of the assumed noise probabilities $p_{01}, p_{10}$ (equivalently $\lambda, \kappa$). This machinery does two jobs. First, it makes optimality concrete: with independent components and correct noise, thresholding the odds ratio minimizes expected Hamming distance. Second, it localizes the failure mode: when $\lambda, \kappa$ are frozen at initial values, or when the rule is numerically stabilized by replacing $M_{11}$ with $\max(M_{11}, \eta\epsilon_s^2 M)$, the network is effectively implementing a different noise assumption, and BCPNN arises as the sparse-pattern, low-add-noise limit of the same formula. ANE replaces the frozen estimates with measured per-step values, which is the mechanism that restores the Bayesian rule's advantage.

What would settle it

Run the benchmark that produced the anomaly (fixed-activity patterns, winner-take-all selection, iterative retrieval, $n = 1024$, $k = 32$, 10% query noise) but feed the Bayesian rule the actual noise measured from the network output at each step instead of frozen or stabilized estimates. If any BCPNN variant still stores more patterns at 90% correct retrieval, the paper's central explanation fails; if the Bayesian rule matches or exceeds every variant, the explanation holds.

Watch

Extended reading notes

Core claim

The paper's central claim is that the optimal Bayesian associative network extends to auto-association, and that when it appears to be beaten by BCPNN the cause is model mismatch, not faulty implementation. The decision rule computes, for each neuron, a log-odds ratio from stored usage counters and noise estimates; under independence this minimizes expected Hamming distance in one step. With iterative retrieval, however, the input noise shrinks each pass, so a learning rule that keeps the initial noise estimates fixed is optimizing for the wrong noise level; numerically stabilized BCPNN survives this by effectively assuming very low noise, which is why it can look superior for fixed-activity patterns with winner-take-all readout. The paper's remedy is ANE: estimate $\lambda_{\rm est}(t)$ and $\kappa_{\rm est}(t)$ from the previous output, and in the first step use core-retrieval with a high threshold so that the only errors are false negatives, making the noise predictable. Under that schedule the exact Bayesian rule reaches $M = 1603$ patterns at 90% correctness for $n = 1024$, $k = 32$, exceeding all BCPNN variants, while for independently generated patterns it reaches about $M = 1222$.

Load-bearing premise

The load-bearing premise is that the mathematical proof of optimality for input-output associative memories still applies when input and output are the same pattern, and that after each retrieval round the remaining noise can still be described by the same two average probabilities; if either premise fails, the comparison loses its theoretical anchor.

Editorial extensions

If this is right

The apparent superiority of stabilized BCPNN over the optimal Bayesian rule in iterative auto-association is resolved as a consequence of stale or stabilized noise estimates, so no implementation error needs to be invoked.
For any recurrent associative network that iterates to convergence, noise estimates should be updated per pass; fixed estimates derived from the initial query are only guaranteed optimal for a single step.
With predictable first-step noise (core-retrieval: activate only a high-confidence subset), the exact Bayesian rule regains the global maximum measured capacity, $M = 1603$ for $n = 1024$, $k = 32$.
If synapses cannot be adapted during retrieval (fixed weights, zero noise estimates), stabilized BCPNN remains the best practical choice, with about $M = 1439$ for fixed-activity patterns, so the best rule depends on whether weights may vary across retrieval steps.
Adaptive noise estimation also raises capacity for independent-component patterns, from about 1115 to about 1222 stored patterns for the Bayesian rule.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

I would expect ANE to generalize to other attractor networks: any iterative refinement process that re-feeds its own output should re-estimate the noise of that output rather than assume the input noise remains constant.
The core-retrieval result suggests a testable retrieval policy for cortical models: a first, conservative pass that admits only high-confidence units, followed by a relaxed pass; this matches two-phase or oscillatory retrieval accounts without requiring slow synaptic changes.
Because BCPNN's robustness comes from behaving like a low-noise estimator, comparisons between learning rules should control for the noise-estimation schedule and stabilization; otherwise the ranking may reflect hyperparameters rather than learning principles.
A natural extension would be per-unit or per-synapse noise estimation: the paper's ANE uses a global two-parameter model, so if retrieval noise becomes spatially structured, adaptive per-synapse estimates could push capacity further.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper is worth a serious read, and a serious referee. It convincingly explains the reported anomaly that (stabilized) BCPNN can beat the theoretically optimal Bayesian rule in auto-associative iterative retrieval: the Bayesian rule is only optimal when the noise estimates match the true query noise at each step, and BCPNN's numerical stabilization (or equivalently, using low fixed noise estimates) roughly compensates for the inevitable mismatch in later iterations. That explanation is the main contribution, and it is plausible. The auto-associative derivation in Sections 2–3 is largely a reworking of the author's earlier hetero-associative theory, but it is cleanly done and useful for the comparison.

The genuinely new piece is the ANE (adaptive noise estimation) mechanism, including the core-retrieval schedule that makes the second-step noise predictable and thereby gives the Bayesian rule the best reported capacity. I think this is a real idea worth building on, and the paper is honest about the caveats: it flags the untested step oscillation in footnote 17, notes that the ANE estimates are only averages, and acknowledges the theoretical transfer from hetero- to auto-association is asserted rather than proven.

The soft spots are real and match the stress-test note. The headline M=1603 is an interpolated estimate at a single hand-selected step (t=6) with alpha and beta tuned on the same data; the adjacent steps give 1591–1603, and no significance test is reported. The claim that the Bayesian rule achieves “the overall maximum” is not backed by variance or validation. That is a proportionate concern, not a fatal one: the ordering B-WTA > BCPNN-WTA is consistent across several steps and alpha values, but the exact margin is fragile. Second, the optimality argument for auto-association is inherited from the hetero-associative proof in [34] via a “should hold” statement; if that transfer fails, the theoretical foundation weakens. The paper would be stronger with a direct derivation or an explicit conjecture with supporting experiments.

Who should read it: researchers working on associative memory, Bayesian learning rules, or BCPNN implementations. They will get a clear map of when the various rules diverge and a practical mechanism that improves capacity. I would not desk-reject this. Send it to peer review, but the referee should insist on error bars, a train/validation split for the hyperparameters, and either a proof or a precise conjecture for the auto-associative optimality claim.

Referee Report

4 major / 4 minor

Summary. The paper studies auto-associative neural memories under a naive-Bayes generative model. It derives the Bayesian learning rule for one-step retrieval, shows that BCPNN and its variants are approximations of this rule in sparse/low-noise limits, and then runs an extensive numerical comparison of the Bayesian rule against several stabilized and noisy BCPNN variants. The main empirical findings are that, for Willshaw patterns, the Bayesian rule is generally superior, while for Palm patterns and k-WTA retrieval the BCPNN variants can match or exceed it; the paper attributes this to violations of the naive-Bayes assumptions and to noise-estimate mismatch over iterative retrieval. The paper then introduces Adaptive Noise Estimation (ANE), including a core-retrieval schedule, and reports that the Bayesian rule with ANE reaches the highest overall capacity, M=1603 for Palm patterns at 90% correct retrieval (Table 6).

Significance. If the quantitative claims survive scrutiny, the paper would make a useful contribution: it provides a clean derivation of the auto-associative Bayesian rule, a systematic comparison of BCPNN variants, a plausible mechanistic explanation for the previously reported BCPNN advantage, and a novel ANE mechanism with core-retrieval that is falsifiable in further experiments. The numerical study is extensive, covering multiple network sizes, pattern types, and retrieval iterations, which is a strength. However, the headline capacity claims currently rest on selected hyperparameter configurations and point estimates without error bars or a validation protocol, and the theoretical optimality claim is not sharply delimited to the one-step setting. These issues need to be resolved before the central quantitative conclusions can be accepted.

major comments (4)

[§4.5, Tables 5 and 6] The headline capacity M=1603 is an interpolated point estimate from a single configuration (Palm patterns, α=0.96875, β=0.001, iteration step 6). The paper states that each data point averages over 100 networks and 100 retrievals, but no standard deviations, confidence intervals, or significance tests are reported. Table 6 shows M0.9 for B-WTA oscillating between 1591 and 1603 across steps 4–10, and footnote 17 explicitly says the oscillation was not tested for significance. Since the central claim is that B-WTA exceeds BCPNN-WTA (1603 vs 1518) and BCPNN2/3-WTA (1559), the absence of variance or significance testing leaves open that the ordering is sampling noise or a selection artifact. Please report error bars or confidence intervals and run significance tests for the B-WTA vs BCPNN-WTA differences.
[§4.5, Table 5] The configuration that produces the maximum is selected on the same data used to report the result. Table 5 scans α ∈ {0.78125, ..., 0.96875} and β ∈ {0.1, ..., 0}; the winning α=0.96875 is the largest scanned value, and β=0.001 is a point on the scanned grid. No train/validation separation is described, so the reported 1603 is the maximum of a search over the hyperparameter grid. This is a selection-artifact risk, especially because the margin over the next-best rule is only about 5% and adjacent hyperparameter settings in Table 6 differ by comparable amounts. Please evaluate the chosen schedule on independent test data or use a nested validation procedure, and report capacities for neighboring α and β values as a sensitivity check.
[§3, Eqs. (12)–(16)] The theoretical claim that the Bayesian rule is optimal for auto-association is asserted via reference to [34] rather than proven as a formal theorem for the auto-associative case. The derivation of the odds-ratio decision rule is standard for one-step retrieval under naive-Bayes independence, but the paper does not state the precise conditions under which this rule minimizes expected Hamming distance or maximizes capacity, nor how the one-step optimality interacts with k-WTA and iterative retrieval. Section 5, item 2, later concedes that the Bayesian rule cannot be guaranteed optimal during iterative retrieval. Since the abstract and title make general optimality claims, please either provide a precise optimality statement for the auto-associative one-step case and clearly mark iterative/WTA/Palm settings as heuristic, or soften the claims accordingly.
[§4.4, Tables 3 and 4] The ANE method estimates the noise schedule from measured output noise at a reference M (M=1400 for Palm patterns, M=1200 for Willshaw patterns) and then evaluates capacity at other M values using that same schedule. This is an in-sample use of retrieval statistics: the noise estimates are derived from the same kind of data on which the capacity is reported, and footnote 11 concedes that the estimates are only averages with substantial per-network and per-pattern variation. The reported improvements (e.g., Willshaw/B from 851 to 1056) may therefore be optimistic. Please describe ANE as a parameter-estimation procedure with a clear training/validation protocol, or report the sensitivity of the capacity estimates to the choice of reference M.

minor comments (4)

[§4.5, bullet list] The bullet stating 'The other rules BCPNN2/3-WTA are in between (M → 1559/55)' appears to contain a typo; it should read M = 1559 or M → 1559.
[Table 8 caption] The entry 'see ZNA' for BCPNN with ANE on Willshaw patterns should presumably read 'see ZNE'.
[Table 8 caption] The caption says '20 percent input noise λ = 0.9, κ = 0.1'; this is confusing because λ=0.9 and κ=0.1 correspond to 10% miss noise plus 10% add noise. Please specify the noise composition explicitly.
[§4.5, Table 5] For clarity, please state explicitly in the text or table caption that α=0.96875 and β=0.001 were selected as the best values on the scanned grid, and indicate how many configurations were tested in total.

Circularity Check

1 steps flagged · score 4.0 of 10

Oracle-aided ANE feeds the correct-retrieval answer back into the noise estimates, partially compromising the capacity gain; the core Bayesian derivation is otherwise self-contained.

fitted input called prediction [Section 4.4, Table 3 caption and surrounding text (Adaptive Noise Estimation)]
"λest/κest are estimations of input noise (as before), ϵ is output noise (mean errors normalized to mean activity k, pcorr is fraction of correct retrievals, f10/f01 are mean number of false negative/false positive components, and correspondingly λout := 1 − f10, κout := f01/k code mean output noise measurements. ANE means here to use λest(t + 1) := λout(t) and κest(t + 1) := κout(t) for t > 1."

ANE sets the next step's input-noise estimates to the previous step's measured output noise. Those measurements (f10/f01, hence λout/κout) are defined relative to the original stored pattern, which is precisely the target the network is supposed to retrieve. Thus, at step t+1 the synaptic weights are computed from information derived from the correct answer, so the reported ANE improvement is an oracle-feedback effect rather than an independent retrieval prediction. The estimate and the target are the same object, so the capacity gains in Table 4 are, at least partly, forced by construction.

full rationale

The Bayes-optimal one-step rule (15,16) is derived in Section 3 from the odds ratio and the naive-Bayes factorization, so the optimality claim does not reduce entirely to the self-citation [34]; the numerical comparisons of B, BCPNN, BCPNN2 and BCPNN3 are independent simulations with explicitly specified rules. The only clear circular step is the ANE feedback in Section 4.4: λest(t+1):=λout(t) uses false-negative/positive counts against the ground-truth stored patterns, i.e. it tunes the model with the answer to the very task being evaluated. The headline maximum M=1603 in Table 6 is produced by the fixed core-retrieval schedule (38) with α=0.96875 and β=0.001, which is a designed protocol rather than a per-step oracle measurement; the selection of α, β and the step t, together with the absence of significance tests (footnote 17), is a statistical-validity concern rather than a further definitional circularity. Self-citations to [34] are frequent but not load-bearing for the quantitative comparisons, since the formulas and rules are stated in the text and the experiments are external simulations. Overall, the central comparison retains independent content, but one load-bearing 'adaptive noise estimation' step partially reduces to oracle feedback.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the naive-Bayes independence assumption, the two-parameter noise model, and the unproven transfer of optimality from hetero-association; the capacity numbers depend on several tuned hyperparameters.

free parameters (6)

λest/κest = 0.8/0.2 to 1/0, with optimal near 0.999/0.001
Used as input noise estimates for learning; performance varies strongly with them (Tables 1 and 2).
α = 0.96875 (core-retrieval optimum), 1.0312 (halo)
Controls the number of active units in the first retrieval step of ANE; tuned to maximize capacity (Tables 5 to 7).
β = 0.001
Noise estimate for late retrieval steps; tuned in Tables 5 to 7.
η = 2
Scaling factor in the stabilized BCPNN M'11; tuned in Table 2.
Reference M for ANE = 1400 (Palm), 1200 (Willshaw)
Chosen moderately larger than the max capacity without ANE for noise measurement; affects ANE results (Section 4.4).
Rounding adjustments for Palm queries = λest=0.90625, κest=0.09375
Hand-adjusted to account for rounding of λk and κk to integers; used in Table 3.

assumptions (4)

domain assumption Naive Bayes independence: query components are conditionally independent given the stored unit activity.
Used to derive the factorized likelihood in Eq. (13); known to be violated by Palm patterns, which is central to the anomaly.
domain assumption Two-parameter noise model: query noise is described by scalar p01/p10 (or λ/κ) identical across all units and synapses.
Assumed in Eqs. (20-22) and used throughout the simulations.
domain assumption Optimality transfers from hetero-association to auto-association.
The paper cites [34] rather than re-proving optimality for auto-association; Section 3 states this transfer without proof.
domain assumption Output noise after each iterative step is described by the same binomial transition model with updated λ,κ.
Underlies ANE; not independently justified for network-generated noise, which the paper acknowledges varies per synapse (footnote 11).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Neural auto-association with optimal Bayesian learning." pith.science (2026). https://pith.science/paper/LXZUZU6G

@misc{pith2026241218349,
  author       = {Pith},
  title        = {Pith review of: Neural auto-association with optimal Bayesian learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LXZUZU6G}},
  note         = {Machine review of arXiv:2412.18349}
}

read the original abstract

Neural associative memories are single layer perceptrons with fast synaptic learning typically storing discrete associations between pairs of neural activity patterns. Previous works have analyzed the optimal networks under naive Bayes assumptions of independent pattern components and heteroassociation, where the task is to learn associations from input to output patterns. Here I study the optimal Bayesian associative network for auto-association where input and output layers are identical. In particular, I compare performance to different variants of approximate Bayesian learning rules, like the BCPNN (Bayesian Confidence Propagation Neural Network), and try to explain why sometimes the suboptimal learning rules achieve higher storage capacity than the (theoretically) optimal model. It turns out that performance can depend on subtle dependencies of input components violating the ``naive Bayes'' assumptions. This includes patterns with constant number of active units, iterative retrieval where patterns are repeatedly propagated through recurrent networks, and winners-take-all activation of the most probable units. Performance of all learning rules can improve significantly if they include a novel adaptive mechanism to estimate noise in iterative retrieval steps (ANE). The overall maximum storage capacity is achieved again by the Bayesian learning rule with ANE.

Figures

Figures reproduced from arXiv: 2412.18349 by the authors.

**Figure 1.** Output noise ϵ as a function of stored memories for networks of size n = 1024, where each pattern has k = √ n = 32 one-entries using one-step-retrieval for queries with 10 percent input miss/add noise (λ = 0.9, κ = 0.1). Upper panels correspond to hetero-association, lower panels to autoassociation. Left panels correspond to Willshaw patterns (independent components and noise). Right panels correspond to Palm patte… view at source ↗

**Figure 2.** Results for auto-association with iterative retrieval (max. 100 iterations) for Willshaw (left) and Palm patterns (right) for n = 1024, k = 32, λ = 0.9, κ = 0.1. A,B: Output noise ϵ as function of stored memories M, similar to previous [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

**Figure 3.** Pattern Capacity Mϵ at output noise level ϵ = 0.01. Experimental setup is as in previous [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Pattern Capacity Mpcorr at correctness level pcorr = 0.9. Experimental setup is as in previous Figs. 2,3 for network sizes n = 196, 361, 576, 1024 and pattern activity k = √ n = 14, 19, 24, 32. Numbers of legends correspond to (interpolated) pattern capacities for eac…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

72 extracted references · 69 canonical work pages

[34]

Knoblauch

A. Knoblauch. Neural associative memory with optimal bayesian learning. Neural Computation, 23(6):1393–1451, 2011

work page 2011
[1]

J.S. Albus. A theory of cerebellar function. Mathematical Biosciences, 10:25–61, 1971

work page 1971
[2]

J. Backus. Can programming be liberated from the von Neumann style? A functional style and its algebra of programs. Communications of the ACM, 21(8):613–641, 1978

work page 1978
[3]

Bogacz, M.W

R. Bogacz, M.W. Brown, and C. Giraud-Carrier. Model of familiarity discrimination in the perirhi- nal cortex. Journal of Computational Neuroscience, 10:5–23, 2001. 21A common neurophysiological interpretation of iterative retrieval in associative networks is that they may correspond to phases of enhanced gamma-band oscillations (25-60 Hz) observed in the ...

work page 2001
[4]

Braitenberg

V . Braitenberg. Cell assemblies in the cerebral cortex. In R. Heim and G. Palm, editors, Lec- ture notes in biomathematics (21). Theoretical approaches to complex systems. , pages 171–188. Springer-Verlag, Berlin Heidelberg New York, 1978

work page 1978
[5]

Buckingham and D.J

J.T. Buckingham and D.J. Willshaw. Performance characteristics of the associative net. Network: Computation in Neural Systems, 3:407–414, 1992

work page 1992
[6]

Burks, H.H

A.W. Burks, H.H. Goldstine, and J. von Neumann. Preliminary discussion of the logical design of an electronic computing instrument. Report 1946, U.S. Army Ordonance Department, 1946

work page 1946
[7]

Cover and P.E

T.M. Cover and P.E. Hart. Nearest neighbor pattern classification. IEEE Transactions on Informa- tion Theory, 13(1):21–27, 1967

work page 1967

Show all 72 references

[8]

Diederich and M

S. Diederich and M. Opper. Learning of correlated patterns in spin-glass networks by local learning rules. Physical Review Letters, 58(9):949–952, 1987

1987
[9]

R. Fay, U. Kaufmann, A. Knoblauch, H. Markert, and G. Palm. Combining visual attention, object recognition and associative information processing in a neurobotic system. In S. Wermter, G. Palm, and M. Elshaw, editors,Biomimetic Neural Learning for Intelligent Robots, volume 35...

2005
[10]

Fix and J.L

E. Fix and J.L. Hodges. Discriminatory analysis. nonparametric discrimination: Consistency prop- erties. Report, USAF School of Aviation Medicine, Randolph Field, TX, 1951

1951
[11]

Fransen and A

E. Fransen and A. Lansner. A model of cortical associative memory based on a horizontal network of connected columns. Network: Computation in Neural Systems, 9:235–264, 1998

1998
[12]

Fries, J.H

P. Fries, J.H. Reynolds, A.E. Rorie, and R. Desimone. Modulation of oscillatory neuronal synchro- nization by selective visual attention. Science, 291:1560–1563, 2001

2001
[13]

Gardner-Medwin

A.R. Gardner-Medwin. The recall of events through the learning of associations between their parts. Proceedings of the Royal Society of London Series B, 194:375–402, 1976

1976
[14]

C.M. Gray. The temporal correlation hypothesis of visual feature integration: Still alive and well. Neuron, 24:31–47, 1999

1999
[15]

C.M. Gray, P. K ¨onig, A.K. Engel, and W. Singer. Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature, 338:334–337, 1989

1989
[16]

Gripon and C

V . Gripon and C. Berrou. Sparse neural networks with large learning diversity. IEEE Transactions on Neural Networks, 22(7):1087–1096, 2011

2011
[17]

Gripon and C

V . Gripon and C. Berrou. Nearly-optimal associative memories based on distributed constant weight codes. In Proceedings of the IEEE Information Theory and Applications Workshop (ITA) , pages 269–273, 2012

2012
[18]

Gripon, J

V . Gripon, J. Heusel, M. L¨owe, and F. Vermet. A comparative study of sparse associative memories. Journal of Statistical Physics, 164(1):105–129, 2016

2016
[19]

Gripon, M

V . Gripon, M. L¨owe, and F. Vermet. Associative memories to accelerate approximate nearest neigh- bor search. Applied Sciences, 8(9):1676, 2018

2018
[20]

D.O. Hebb. The organization of behavior. A neuropsychological theory.Wiley, New York, 1949

1949
[21]

M.H. Hennig. Theoretical models of synaptic short term plasticity. Frontiers in Computational Neuroscience, 7(45):1–10, 2013

2013
[22]

Hopfield

J.J. Hopfield. Neural networks and physical systems with emergent collective computational abili- ties. Proceedings of the National Academy of Science, USA, 79:2554–2558, 1982. 23

1982
[23]

Indyk and R

P. Indyk and R. Motwani. Approximate nearest neighbors: towards removing the curse of dimen- sionality. In Proceedings of the 30th ACM Symposium on Theory of Computing , pages 604–613, 1998

1998
[24]

Indyk, R

P. Indyk, R. Motwani, P. Raghavan, and S. Vempala. Locality-preserving hashing in multidimen- sional spaces. InProceedings of the 29th ACM Symposium on Theory of Computing, pages 618–625, 1997

1997
[25]

Johansson and A

C. Johansson and A. Lansner. Towards cortex sized artificial neural systems. Neural Networks, 20(1):48–61, 2007

2007
[26]

P. Kanerva. Sparse Distributed Memory. MIT Press, Cambridge, MA, 1988

1988
[27]

Knoblauch

A. Knoblauch. Neural associative memory for brain modeling and information retrieval. Informa- tion Processing Letters, 95:537–544, 2005

2005
[28]

Knoblauch

A. Knoblauch. On the computational benefits of inhibitory neural associative networks. HRI-EU Report 07-05, Honda Research Institute Europe GmbH, D-63073 Offenbach/Main, Germany, May 2007

2007
[29]

Knoblauch

A. Knoblauch. Best-match hashing with inhibitory associative networks for real-world object recog- nition. HRI-EU Report 08-05, Honda Research Institute Europe GmbH, D-63073 Offenbach/Main, Germany, October 2008

2008
[30]

Knoblauch

A. Knoblauch. Neural associative memory and the Willshaw-Palm probability distribution. SIAM Journal on Applied Mathematics, 69(1):169–196, 2008

2008
[31]

Knoblauch

A. Knoblauch. Neural associative networks with optimal bayesian learning. HRI-EU Report 09-02, Honda Research Institute Europe GmbH, D-63073 Offenbach/Main, Germany, May 2009

2009
[32]

Knoblauch

A. Knoblauch. Comparison of the Lansner/Ekeberg rule to optimal Bayesian learning in neural associative memory. HRI-EU Report 10-06, Honda Research Institute Europe GmbH, D-63073 Offenbach/Main, Germany, April 2010

2010
[33]

Knoblauch

A. Knoblauch. Optimal synaptic learning in non-linear associative memory. In Proceedings of the International Joint Conference on Neural Networks (IJCNN), pages 3205–3211, Barcelona, Spain,
[35]

Knoblauch

A. Knoblauch. Efficient associative computation with discrete synapses. Neural Computation, 28(1):118–186, 2016

2016
[36]

Knoblauch

A. Knoblauch. Impact of structural plasticity on memory formation and decline. In A. van Ooyen and M. Butz, editors, Rewiring the Brain: A Computational Approach to Structural Plasticity in the Adult Brain., pages 361–386, London, UK, 2017. Elsevier/Academic Press

2017
[37]

Knoblauch, H

A. Knoblauch, H. Markert, and G. Palm. An associative cortical model of language understanding and action planning. In J. Mira and J.R. Alvarez, editors, Artificial Intelligence and Knowledge Engineering Applications: A Bioinspired Approach. Proceedings of IWINAC 2005, First I...

2005
[38]

Knoblauch and G

A. Knoblauch and G. Palm. Pattern separation and synchronization in spiking associative memories and visual areas. Neural Networks, 14:763–780, 2001

2001
[39]

Knoblauch and G

A. Knoblauch and G. Palm. Scene segmentation by spike synchronization in reciprocally connected visual areas. I. Local effects of cortical feedback. Biological Cybernetics, 87(3):151–167, 2002. 24

2002
[40]

Knoblauch and G

A. Knoblauch and G. Palm. Scene segmentation by spike synchronization in reciprocally connected visual areas. II. Global assemblies and synchronization on larger space and time scales. Biological Cybernetics, 87(3):168–184, 2002

2002
[41]

Knoblauch and G

A. Knoblauch and G. Palm. Iterative retrieval and block coding in autoassociative and heteroasso- ciative memory. Neural Computation, 32(1):205–260, 2020

2020
[42]

Knoblauch, G

A. Knoblauch, G. Palm, and F.T. Sommer. Memory capacities for synaptic and structural plasticity. Neural Computation, 22(2):289–341, 2010

2010
[43]

Knoblauch, F.T

A. Knoblauch, F.T. Sommer, M.-O. Gewaltig, R. Kupper, U. K¨orner, and E. K¨orner. On the collec- tive computational abilities of inhibitory neurons. In A. Zador, A. Pouget, and Z. Mainen, editors, Proceedings of the 5th Computational and Systems Neuroscience Meeting (COSYNE) ,...

2008
[44]

Krotov and J.J

D. Krotov and J.J. Hopfield. Dense associative memory for pattern recognition. arXiv e-prints 1606.01164, 2016

2016 arXiv
[45]

A. Lansner. Associative memory models: from the cell-assembly theory to biophysically detailed cortex simulations. Trends in Neurosciences, 32(3):178–186, 2009

2009
[46]

A. Lansner. BCPNN can perform better than optimal Bayesian learning, 2024. Personal communi- cation, 07/11/2024

2024
[47]

A. Lansner. Recent implementations of BCPNN are numerically stabilized with ϵs = 1 1+M to avoid infinite synaptic weights., 2024. Personal communication, 26/11/2024

2024
[48]

Lansner and ¨O

A. Lansner and ¨O. Ekeberg. An associative network solving the ”4-bit adder problem”. In M. Caudill and C. Butler, editors,Proceedings of the IEEE First International Conference on Neural Networks, pages II–549, San Diego, CA, 1987

1987
[49]

Lansner and ¨O

A. Lansner and ¨O. Ekeberg. A one-layer feedback artificial neural network with a Bayesian learning rule. International Journal of Neural Systems, 1(1):77–87, 1989

1989
[50]

Lansner and A

A. Lansner and A. Holst. A higher order Bayesian neural network with spiking units. International Journal of Neural Systems, 7(2):115–128, 1996

1996
[51]

Lansner, N.B

A. Lansner, N.B. Ravichandran, and P. Herman. Benchmarking Hebbian learning rules for associa- tive memory. arxiv:2401.00335, 2023

2023 arXiv
[52]

G. Laurent. Olfactory network dynamics and the coding of multidimensional signals. Nature Reviews Neuroscience, 3:884–895, 2002

2002
[53]

D. Marr. A theory of cerebellar cortex. Journal of Physiology, 202(2):437–470, 1969

1969
[54]

D. Marr. Simple memory: a theory for archicortex. Philosophical Transactions of the Royal Society of London, Series B, 262:24–81, 1971

1971
[55]

Martinez Mayorquin

R.H. Martinez Mayorquin. Sequence learning in the Bayesian Confidence Propagation Neural Network. PhD thesis, KTH Royal Institute of Technology, Stockholm, Sweden, 2022

2022
[56]

Motwani, A

R. Motwani, A. Naor, and R. Panigrahy. Lower bounds on locality sensitive hashing. InProceedings of the 22nd Annual Symposium on Computational Geometry, pages 154–157, 2006

2006
[57]

G. Palm. On associative memories. Biological Cybernetics, 36:19–31, 1980

1980
[58]

G. Palm. Neural Assemblies. An Alternative Approach to Artificial Intelligence. Springer, Berlin, 1982

1982
[59]

G. Palm. Memory capacities of local rules for synaptic modification. A comparative review. Con- cepts in Neuroscience, 2:97–128, 1991. 25

1991
[60]

G. Palm. On the information storage capacity of local learning rules. Neural Computation , 4(5):703–711, 1992

1992
[61]

G. Palm, A. Knoblauch, F. Hauser, and A. Sch¨uz. Cell assemblies in the cerebral cortex. Biological Cybernetics, 108(5):559–572, 2014

2014
[62]

Palm and M

G. Palm and M. Palm. Parallel associative networks: The PAN-system and the Bacchus-chip. In U. Ramacher, U. R ¨uckert, and J.A. Nossek, editors, Proceedings of the 2nd International Confer- ence on Microelectronics for Neural Networks. Kyrill&Method Verlag, Munich, 1991

1991
[63]

Pulverm ¨uller

F. Pulverm ¨uller. The neuroscience of language: on brain circuits of words and serial order. Cam- bridge University Press, Cambridge, UK, 2003

2003
[64]

E.T. Rolls. A theory of hippocampal function in memory. Hippocampus, 6:601–620, 1996

1996
[65]

Rosenblatt

F. Rosenblatt. The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65(6):386–408, 1958

1958
[66]

Schwenker, F.T

F. Schwenker, F.T. Sommer, and G. Palm. Iterative retrieval of sparsely coded associative memory patterns. Neural Networks, 9:445–455, 1996

1996
[67]

Steinbuch

K. Steinbuch. Die Lernmatrix. Kybernetik, 1:36–45, 1961

1961
[68]

Willshaw, O.P

D.J. Willshaw, O.P. Buneman, and H.C. Longuet-Higgins. Non-holographic associative memory. Nature, 222:960–962, 1969

1969
[69]

Willshaw and P

D.J. Willshaw and P. Dayan. Optimal plasticity in matrix memories: what goes up must come down. Neural Computation, 2:85–93, 1990

1990
[70]

Z. Yao, V . Gripon, and M. Rabbat. A gpu-based associative memory using sparse neural networks. In Proceedings of the IEEE International Conference on High-Performance-Computing & Simula- tion (HPCS), pages 688–692, 2014

2014
[71]

Zucker and W.G

R.S. Zucker and W.G. Regehr. Short-term synaptic plasticity.Annual Review of Physiology, 64:355– 405, 2002. 26

2002
[2010]

IEEE World Congress on Computational Intelligence (WCCI)

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [34]

Knoblauch

A. Knoblauch. Neural associative memory with optimal bayesian learning. Neural Computation, 23(6):1393–1451, 2011

work page 2011

[2] [1]

J.S. Albus. A theory of cerebellar function. Mathematical Biosciences, 10:25–61, 1971

work page 1971

[3] [2]

J. Backus. Can programming be liberated from the von Neumann style? A functional style and its algebra of programs. Communications of the ACM, 21(8):613–641, 1978

work page 1978

[4] [3]

Bogacz, M.W

R. Bogacz, M.W. Brown, and C. Giraud-Carrier. Model of familiarity discrimination in the perirhi- nal cortex. Journal of Computational Neuroscience, 10:5–23, 2001. 21A common neurophysiological interpretation of iterative retrieval in associative networks is that they may correspond to phases of enhanced gamma-band oscillations (25-60 Hz) observed in the ...

work page 2001

[5] [4]

Braitenberg

V . Braitenberg. Cell assemblies in the cerebral cortex. In R. Heim and G. Palm, editors, Lec- ture notes in biomathematics (21). Theoretical approaches to complex systems. , pages 171–188. Springer-Verlag, Berlin Heidelberg New York, 1978

work page 1978

[6] [5]

Buckingham and D.J

J.T. Buckingham and D.J. Willshaw. Performance characteristics of the associative net. Network: Computation in Neural Systems, 3:407–414, 1992

work page 1992

[7] [6]

Burks, H.H

A.W. Burks, H.H. Goldstine, and J. von Neumann. Preliminary discussion of the logical design of an electronic computing instrument. Report 1946, U.S. Army Ordonance Department, 1946

work page 1946

[8] [7]

Cover and P.E

T.M. Cover and P.E. Hart. Nearest neighbor pattern classification. IEEE Transactions on Informa- tion Theory, 13(1):21–27, 1967

work page 1967