{"id":"bcafe9cf-865f-47cb-82d0-3fa11ae076c9","arxiv_id":"2412.18349","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Auto-association under the optimal Bayesian rule is clarified: BCPNN beats it when naive-Bayes assumptions fail, and adaptive noise estimation with core-retrieval pushes Bayesian capacity to 1603 patterns at n=1024.","lead":"The paper derives the auto-associative version of an optimal Bayesian learning rule for neural associative memories, then shows that under fixed noise assumptions a simpler rule (BCPNN) can outperform it, and introduces adaptive noise estimation that restores the Bayesian rule's advantage. The result explains a reported anomaly and offers a practical way to increase the storage capacity of brain-like memory networks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline capacity M=1603 is an interpolated estimate from a hyperparameter-selected ANE schedule; no error bars or validation procedure are reported, so the claimed margin over BCPNN may be sampling noise.","rationale":"The concern is load-bearing because the central claim is precisely a numerical capacity ordering; that ordering is supported only by point estimates from a configuration selected on the same data. The reader's verdict identified the theoretical transfer as the weak assumption, but that transfer is not needed for the empirical claim: even if auto-associative optimality were unproven, the Bayesian rule could still achieve the highest measured capacity. What is needed is that the measured capacity is real and not an artifact of hyperparameter selection or sampling noise. The paper itself flags the significance issue in footnote 17 and acknowledges the oscillation in Table 6, which supports the concern. Agreement with the reader is partial: we agree the paper deserves a conditional verdict, but the specific load-bearing issue is empirical robustness rather than the theory transfer.","tokens_in":30605,"tokens_out":4372,"duration_ms":40198,"concrete_test":"Perform a holdout evaluation at n=1024, k=32, Palm patterns, λ=0.9, κ=0.1. Use 500 networks to select α, β, and step number t by maximizing pcorr at M=1600; then evaluate B-WTA and BCPNN-WTA (plus BCPNN2/3-WTA) on 500 independent networks at the chosen hyperparameters, reporting pcorr with 95% bootstrap CIs at M=1600 and the interpolated M0.9. If the CIs overlap or the held-out M0.9 of B-WTA does not exceed that of BCPNN-WTA by more than a minimal bound (e.g., 5%), the headline capacity ordering is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's abstract claims the Bayesian rule with ANE achieves 'the overall maximum storage capacity' (M=1603), but this number is an interpolated capacity estimate from a single hand-picked configuration: Palm patterns, α=0.96875, β=0.001, at iteration step t=6 (Table 6). The paper states that each data point averages over 100 networks and 100 retrievals, yet no standard deviations, confidence intervals, or significance tests are reported anywhere. The selection of t=6 is suspect: Table 6 shows M0.9 for B-WTA oscillating between 1591 and 1603 across steps 4–10, and the author's own footnote 17 admits the oscillation was 'not tested' for significance. Additionally, α and β were optimized on the same data: Table 5 scans α ∈ {0.78125,...,0.96875} and β ∈ {0.1, ..., 0}; the winning β=0.001 is a point on that scanned grid. No train/validation separation is described. Since the central claim is an empirical victory of B-WTA over BCPNN-WTA (1603 vs 1518, ~5%) and over BCPNN2/3-WTA (1559), the absence of variance or validation leaves open that the reported ordering is a selection artifact. A related unstated assumption is that the core-retrieval schedule (38) makes the step-2 and step-3+ noise exactly what the fixed α, β prescribe; in individual networks the noise still varies, as the paper concedes in footnote 11 for the simpler ANE scheme.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies auto-associative neural memories under a naive-Bayes generative model. It derives the Bayesian learning rule for one-step retrieval, shows that BCPNN and its variants are approximations of this rule in sparse/low-noise limits, and then runs an extensive numerical comparison of the Bayesian rule against several stabilized and noisy BCPNN variants. The main empirical findings are that, for Willshaw patterns, the Bayesian rule is generally superior, while for Palm patterns and k-WTA retrieval the BCPNN variants can match or exceed it; the paper attributes this to violations of the naive-Bayes assumptions and to noise-estimate mismatch over iterative retrieval. The paper then introduces Adaptive Noise Estimation (ANE), including a core-retrieval schedule, and reports that the Bayesian rule with ANE reaches the highest overall capacity, M=1603 for Palm patterns at 90% correct retrieval (Table 6).","tokens_in":30970,"tokens_out":6854,"duration_ms":66812,"significance":"If the quantitative claims survive scrutiny, the paper would make a useful contribution: it provides a clean derivation of the auto-associative Bayesian rule, a systematic comparison of BCPNN variants, a plausible mechanistic explanation for the previously reported BCPNN advantage, and a novel ANE mechanism with core-retrieval that is falsifiable in further experiments. The numerical study is extensive, covering multiple network sizes, pattern types, and retrieval iterations, which is a strength. However, the headline capacity claims currently rest on selected hyperparameter configurations and point estimates without error bars or a validation protocol, and the theoretical optimality claim is not sharply delimited to the one-step setting. These issues need to be resolved before the central quantitative conclusions can be accepted.","major_comments":[{"comment":"The headline capacity M=1603 is an interpolated point estimate from a single configuration (Palm patterns, α=0.96875, β=0.001, iteration step 6). The paper states that each data point averages over 100 networks and 100 retrievals, but no standard deviations, confidence intervals, or significance tests are reported. Table 6 shows M0.9 for B-WTA oscillating between 1591 and 1603 across steps 4–10, and footnote 17 explicitly says the oscillation was not tested for significance. Since the central claim is that B-WTA exceeds BCPNN-WTA (1603 vs 1518) and BCPNN2/3-WTA (1559), the absence of variance or significance testing leaves open that the ordering is sampling noise or a selection artifact. Please report error bars or confidence intervals and run significance tests for the B-WTA vs BCPNN-WTA differences.","section":"§4.5, Tables 5 and 6"},{"comment":"The configuration that produces the maximum is selected on the same data used to report the result. Table 5 scans α ∈ {0.78125, ..., 0.96875} and β ∈ {0.1, ..., 0}; the winning α=0.96875 is the largest scanned value, and β=0.001 is a point on the scanned grid. No train/validation separation is described, so the reported 1603 is the maximum of a search over the hyperparameter grid. This is a selection-artifact risk, especially because the margin over the next-best rule is only about 5% and adjacent hyperparameter settings in Table 6 differ by comparable amounts. Please evaluate the chosen schedule on independent test data or use a nested validation procedure, and report capacities for neighboring α and β values as a sensitivity check.","section":"§4.5, Table 5"},{"comment":"The theoretical claim that the Bayesian rule is optimal for auto-association is asserted via reference to [34] rather than proven as a formal theorem for the auto-associative case. The derivation of the odds-ratio decision rule is standard for one-step retrieval under naive-Bayes independence, but the paper does not state the precise conditions under which this rule minimizes expected Hamming distance or maximizes capacity, nor how the one-step optimality interacts with k-WTA and iterative retrieval. Section 5, item 2, later concedes that the Bayesian rule cannot be guaranteed optimal during iterative retrieval. Since the abstract and title make general optimality claims, please either provide a precise optimality statement for the auto-associative one-step case and clearly mark iterative/WTA/Palm settings as heuristic, or soften the claims accordingly.","section":"§3, Eqs. (12)–(16)"},{"comment":"The ANE method estimates the noise schedule from measured output noise at a reference M (M=1400 for Palm patterns, M=1200 for Willshaw patterns) and then evaluates capacity at other M values using that same schedule. This is an in-sample use of retrieval statistics: the noise estimates are derived from the same kind of data on which the capacity is reported, and footnote 11 concedes that the estimates are only averages with substantial per-network and per-pattern variation. The reported improvements (e.g., Willshaw/B from 851 to 1056) may therefore be optimistic. Please describe ANE as a parameter-estimation procedure with a clear training/validation protocol, or report the sensitivity of the capacity estimates to the choice of reference M.","section":"§4.4, Tables 3 and 4"}],"minor_comments":[{"comment":"The bullet stating 'The other rules BCPNN2/3-WTA are in between (M → 1559/55)' appears to contain a typo; it should read M = 1559 or M → 1559.","section":"§4.5, bullet list"},{"comment":"The entry 'see ZNA' for BCPNN with ANE on Willshaw patterns should presumably read 'see ZNE'.","section":"Table 8 caption"},{"comment":"The caption says '20 percent input noise λ = 0.9, κ = 0.1'; this is confusing because λ=0.9 and κ=0.1 correspond to 10% miss noise plus 10% add noise. Please specify the noise composition explicitly.","section":"Table 8 caption"},{"comment":"For clarity, please state explicitly in the text or table caption that α=0.96875 and β=0.001 were selected as the best values on the scanned grid, and indicate how many configurations were tested in total.","section":"§4.5, Table 5"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a self-contained technical report that draws heavily on the author's prior work [34], including the optimality result, the implementation, and the numerical methodology. The novelty lies mainly in the auto-associative reformulation, the explanation of the BCPNN anomaly, and the ANE mechanism. Given the journal context, the quantitative headline (M=1603) needs stronger statistical support and a clear validation protocol; with those additions, the paper could be a solid contribution. I do not see a fundamental flaw in the derivations, but the current presentation overstates the strength of the empirical comparison."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth a serious read, and a serious referee. It convincingly explains the reported anomaly that (stabilized) BCPNN can beat the theoretically optimal Bayesian rule in auto-associative iterative retrieval: the Bayesian rule is only optimal when the noise estimates match the true query noise at each step, and BCPNN's numerical stabilization (or equivalently, using low fixed noise estimates) roughly compensates for the inevitable mismatch in later iterations. That explanation is the main contribution, and it is plausible. The auto-associative derivation in Sections 2–3 is largely a reworking of the author's earlier hetero-associative theory, but it is cleanly done and useful for the comparison.\n\nThe genuinely new piece is the ANE (adaptive noise estimation) mechanism, including the core-retrieval schedule that makes the second-step noise predictable and thereby gives the Bayesian rule the best reported capacity. I think this is a real idea worth building on, and the paper is honest about the caveats: it flags the untested step oscillation in footnote 17, notes that the ANE estimates are only averages, and acknowledges the theoretical transfer from hetero- to auto-association is asserted rather than proven.\n\nThe soft spots are real and match the stress-test note. The headline M=1603 is an interpolated estimate at a single hand-selected step (t=6) with alpha and beta tuned on the same data; the adjacent steps give 1591–1603, and no significance test is reported. The claim that the Bayesian rule achieves “the overall maximum” is not backed by variance or validation. That is a proportionate concern, not a fatal one: the ordering B-WTA > BCPNN-WTA is consistent across several steps and alpha values, but the exact margin is fragile. Second, the optimality argument for auto-association is inherited from the hetero-associative proof in [34] via a “should hold” statement; if that transfer fails, the theoretical foundation weakens. The paper would be stronger with a direct derivation or an explicit conjecture with supporting experiments.\n\nWho should read it: researchers working on associative memory, Bayesian learning rules, or BCPNN implementations. They will get a clear map of when the various rules diverge and a practical mechanism that improves capacity. I would not desk-reject this. Send it to peer review, but the referee should insist on error bars, a train/validation split for the hyperparameters, and either a proof or a precise conjecture for the auto-associative optimality claim.","headline":"A credible explanation of the BCPNN-vs-Bayesian anomaly via noise-estimate mismatch and an adaptive noise mechanism, but the headline capacity claim rests on a hand-picked configuration without error bars.","tokens_in":31452,"tokens_out":1300,"would_cite":true,"duration_ms":14671,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["92B20","68T05"],"pacs":[],"model":"deepseek-v4-flash","headline":"In auto-associative memory, the optimal Bayesian rule regains top capacity once noise is re-estimated each step.","keywords":["associative memory","auto-association","optimal Bayesian learning","BCPNN","iterative retrieval","adaptive noise estimation","storage capacity"],"falsifier":"Run the benchmark that produced the anomaly (fixed-activity patterns, winner-take-all selection, iterative retrieval, $n = 1024$, $k = 32$, 10% query noise) but feed the Bayesian rule the actual noise measured from the network output at each step instead of frozen or stabilized estimates. If any BCPNN variant still stores more patterns at 90% correct retrieval, the paper's central explanation fails; if the Bayesian rule matches or exceeds every variant, the explanation holds.","tokens_in":30423,"feed_emoji":"🧠","tokens_out":11073,"duration_ms":93586,"temperature":0.7,"pith_summary":"This paper asks why a theoretically optimal learning rule for associative neural memories can lose to its approximations, and whether the optimum can be restored. In the auto-associative setting (input and output are the same pattern), the Bayesian maximum-likelihood rule is optimal only under naive-Bayes assumptions: independent pattern components, independent query noise, and one-step retrieval. The paper shows that the reported superiority of the BCPNN rule over this \"optimal\" rule is not an implementation error; it follows from violations of those assumptions, especially patterns with a fixed number of active units, winner-take-all selection, and iterative retrieval in which any fixed noise estimate becomes stale. It then introduces Adaptive Noise Estimation (ANE), which re-estimates noise after each retrieval step, and a core-retrieval schedule under which the Bayesian rule reaches the highest measured capacity, M = 1603 at 90% correct retrieval, above every BCPNN variant tested. The result matters because it separates \"the rule is wrong\" from \"the assumptions are violated,\" and it shows a concrete way to recover theoretical optimality in recurrent retrieval.","feed_headline":"Bayesian rule beats BCPNN once noise is re-estimated each step","feed_subtitle":"Updating noise estimates each retrieval step restores the Bayesian rule's storage edge, reaching 1603 patterns.","key_machinery":"The engine of the argument is the log-odds decision variable $x_j = \\log r_j$, where $r_j = \\Pr[u^\\mu_j = 1 \\mid \\tilde u, M(j)] / \\Pr[u^\\mu_j = 0 \\mid \\tilde u, M(j)]$; the neuron fires when $x_j \\ge 0$. Its weights are functions of the synaptic counters $M_{11}(ij)$, $M_{01}(ij)$, $M_{10}(ij)$, $M_{00}(ij)$ and of the assumed noise probabilities $p_{01}, p_{10}$ (equivalently $\\lambda, \\kappa$). This machinery does two jobs. First, it makes optimality concrete: with independent components and correct noise, thresholding the odds ratio minimizes expected Hamming distance. Second, it localizes the failure mode: when $\\lambda, \\kappa$ are frozen at initial values, or when the rule is numerically stabilized by replacing $M_{11}$ with $\\max(M_{11}, \\eta\\epsilon_s^2 M)$, the network is effectively implementing a different noise assumption, and BCPNN arises as the sparse-pattern, low-add-noise limit of the same formula. ANE replaces the frozen estimates with measured per-step values, which is the mechanism that restores the Bayesian rule's advantage.","core_discovery":"The paper's central claim is that the optimal Bayesian associative network extends to auto-association, and that when it appears to be beaten by BCPNN the cause is model mismatch, not faulty implementation. The decision rule computes, for each neuron, a log-odds ratio from stored usage counters and noise estimates; under independence this minimizes expected Hamming distance in one step. With iterative retrieval, however, the input noise shrinks each pass, so a learning rule that keeps the initial noise estimates fixed is optimizing for the wrong noise level; numerically stabilized BCPNN survives this by effectively assuming very low noise, which is why it can look superior for fixed-activity patterns with winner-take-all readout. The paper's remedy is ANE: estimate $\\lambda_{\\rm est}(t)$ and $\\kappa_{\\rm est}(t)$ from the previous output, and in the first step use core-retrieval with a high threshold so that the only errors are false negatives, making the noise predictable. Under that schedule the exact Bayesian rule reaches $M = 1603$ patterns at 90% correctness for $n = 1024$, $k = 32$, exceeding all BCPNN variants, while for independently generated patterns it reaches about $M = 1222$.","pith_inferences":["I would expect ANE to generalize to other attractor networks: any iterative refinement process that re-feeds its own output should re-estimate the noise of that output rather than assume the input noise remains constant.","The core-retrieval result suggests a testable retrieval policy for cortical models: a first, conservative pass that admits only high-confidence units, followed by a relaxed pass; this matches two-phase or oscillatory retrieval accounts without requiring slow synaptic changes.","Because BCPNN's robustness comes from behaving like a low-noise estimator, comparisons between learning rules should control for the noise-estimation schedule and stabilization; otherwise the ranking may reflect hyperparameters rather than learning principles.","A natural extension would be per-unit or per-synapse noise estimation: the paper's ANE uses a global two-parameter model, so if retrieval noise becomes spatially structured, adaptive per-synapse estimates could push capacity further."],"forward_implications":["The apparent superiority of stabilized BCPNN over the optimal Bayesian rule in iterative auto-association is resolved as a consequence of stale or stabilized noise estimates, so no implementation error needs to be invoked.","For any recurrent associative network that iterates to convergence, noise estimates should be updated per pass; fixed estimates derived from the initial query are only guaranteed optimal for a single step.","With predictable first-step noise (core-retrieval: activate only a high-confidence subset), the exact Bayesian rule regains the global maximum measured capacity, $M = 1603$ for $n = 1024$, $k = 32$.","If synapses cannot be adapted during retrieval (fixed weights, zero noise estimates), stabilized BCPNN remains the best practical choice, with about $M = 1439$ for fixed-activity patterns, so the best rule depends on whether weights may vary across retrieval steps.","Adaptive noise estimation also raises capacity for independent-component patterns, from about 1115 to about 1222 stored patterns for the Bayesian rule."],"supporting_citations":[{"why":"Supplies the optimal Bayesian learning rule, the hetero-associative optimality theory, and the exact implementation this paper extends to auto-association.","marker":"[34]"},{"why":"Reports the observed anomaly where BCPNN outperforms the optimal Bayesian rule, which motivates the paper's investigation.","marker":"[46]"},{"why":"Documents the numerical stabilization with $M'_{11} = \\max(M_{11}, \\eta \\epsilon_s^2 M)$ used in recent BCPNN implementations.","marker":"[47]"},{"why":"Defines the original BCPNN rule that reappears as the sparse-pattern low-add-noise limit of the Bayesian weight formula.","marker":"[48, 49]"},{"why":"Establishes the stabilized BCPNN variants and their recommended scaling factor that the paper compares against.","marker":"[50, 25, 55, 51]"},{"why":"Provides the iterative retrieval framework used for recurrent passes in the experiments.","marker":"[66]"},{"why":"Defines the fixed-activity versus independent-component pattern statistics that distinguish the two benchmark conditions.","marker":"[30]"}],"fun_headline_variants":["Adaptive noise re-estimation restores Bayesian storage edge","Bayesian associative network hits top storage with noise tracking","Optimal Bayesian rule outperforms when noise is updated","Adaptive noise estimate lifts Bayesian memory to 1603 patterns","Iterative noise re-estimation gives Bayesian rule top capacity"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the mathematical proof of optimality for input-output associative memories still applies when input and output are the same pattern, and that after each retrieval round the remaining noise can still be described by the same two average probabilities; if either premise fails, the comparison loses its theoretical anchor.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive noise re-estimation restores Bayesian storage edge","Bayesian associative network hits top storage with noise tracking","Optimal Bayesian rule outperforms when noise is updated","Adaptive noise estimate lifts Bayesian memory to 1603 patterns","Iterative noise re-estimation gives Bayesian rule top capacity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001085,"raw_usage":{"total_tokens":4561,"prompt_tokens":994,"completion_tokens":3567,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":610,"completion_tokens_details":{"reasoning_tokens":3501}},"tokens_in":610,"tokens_out":3567,"duration_ms":22095,"temperature":1.0,"reasoning_tokens":3501,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:46:49.765815+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the benchmark that produced the anomaly (fixed-activity patterns, winner-take-all selection, iterative retrieval, $n = 1024$, $k = 32$, 10% query noise) but feed the Bayesian rule the actual noise measured from the network output at each step instead of frozen or stabilized estimates. If any BCPNN variant still stores more patterns at 90% correct retrieval, the paper's central explanation fails; if the Bayesian rule matches or exceeds every variant, the explanation holds.","supporting_citations":[{"cited_title":"Knoblauch","cited_arxiv_id":null,"evidence_quote":"Supplies the optimal Bayesian learning rule, the hetero-associative optimality theory, and the exact implementation this paper extends to auto-association."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Reports the observed anomaly where BCPNN outperforms the optimal Bayesian rule, which motivates the paper's investigation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents the numerical stabilization with $M'_{11} = \\max(M_{11}, \\eta \\epsilon_s^2 M)$ used in recent BCPNN implementations."},{"cited_title":"Schwenker, F.T","cited_arxiv_id":null,"evidence_quote":"Provides the iterative retrieval framework used for recurrent passes in the experiments."},{"cited_title":"Knoblauch","cited_arxiv_id":null,"evidence_quote":"Defines the fixed-activity versus independent-component pattern statistics that distinguish the two benchmark conditions."}],"review_version":1}