{"id":"dba5b986-3eed-4ae1-9279-b1e8b85dedf7","arxiv_id":"2411.17485","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Overlapping associative memories can be stored in all-inhibitory spiking networks at up to about half a memory per neuron by pinning each memory to a vertex of a hypercubic latent manifold.","lead":"The authors show how a spiking neural network can store many overlapping memories by carving out a high-dimensional cube in the network's latent space and placing each memory at a cube corner. The result gives a recipe for Hopfield-style memory in biologically plausible spiking circuits, with storage growing linearly with network size.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The fixed-point condition is verified only against the assumed equal-rate pattern eta, not against actual spiking rates; reported capacities are empirical, load-tuned results.","rationale":"The paper presents a genuinely useful geometric construction and reproducible spiking simulations, with public code, and the empirical recall results are real evidence in its favor. However, the strongest claim -- that vertices of the hypercubic manifold are stable overlapping attractors with a direct Hopfield correspondence, at capacity linear in N -- rests on the condition D eta = xi. Since eta is not measured but assumed to have equal firing rates kappa for active neurons, and since the implemented network modifies D by a separately tuned self-connection gamma, the mathematical construction certifies only an idealized rate model, not the simulated spiking network. The reported capacity values are therefore empirical thresholds rather than derived consequences, which is exactly the kind of gap the reader flagged. I see no evidence of internal contradiction or overclaiming beyond this; the concern is that the central theoretical guarantee has an unverified load-bearing assumption. The concrete test of comparing Eq. (8) against measured spike-filtered rates would settle the matter directly. Because the simulations already demonstrate the capacity phenomenologically, a conditional verdict is appropriate, and the reader's CONDITIONAL judgment should stand unchanged.","tokens_in":11546,"tokens_out":8048,"duration_ms":84060,"concrete_test":"For a network with N=100 and p=40 under the optimized rule, initialize at each stored pattern, simulate recall, and compute time-averaged filtered spike rates <r_i> over a stationary window after recall. Define eta_meas = <r> and evaluate rho_mu = ||D eta_meas - xi_mu|| / ||xi_mu|| for each pattern. If rho_mu is below about 0.05 for each pattern, the equal-rate construction (Eq. 11) is validated and the concern is resolved; if rho_mu is large while recall succeeds, Eq. (8) is not what stabilizes the vertices and the Hopfield-correspondence claim needs revision. Also report whether all active neurons have comparable average rates; if rates vary by more than about 20%, the kappa assumption fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central stability condition is Eq. (8), D eta_mu = xi_mu. Because Eq. (4) is underdetermined, the paper posits Eq. (11), eta_mu = ReLU(kappa E xi_mu), i.e., all active neurons fire at the same average rate kappa, 'to make this problem tractable' (Sec. 2.2.3). This assumption is used to construct D_pinv and D_opt, and Fig. 3e quantifies Eq. (8) with this assumed eta, not with measured spike-filtered rates. The actual spiking dynamics are oscillatory (footnote 1) and are stabilized by a self-connection gamma that must be re-optimized for every load (Fig. 4c); for the Hebbian and pseudoinverse rules this self-connection is imposed post hoc, so Eq. (8) is broken in the implemented network (Appendix C.1). Moreover, as written, Eq. (10) combined with Eq. (11) gives D eta = kappa xi, not xi, unless kappa is fixed to 1 (a choice never stated). Thus the claim that stable overlapping attractors at the vertices follow from the construction is not established: the reported p/N ~ 0.3 and 0.5 thresholds are empirical, load-tuned results without error bars or comparison to prior spiking memory models. The load-bearing question is whether actual time-averaged firing rates at a recalled vertex satisfy Eq. (8) with the designed D.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a geometric framework for storing overlapping binary memories in low-rank, all-inhibitory spiking networks. The authors map memory patterns to vertices of a hypercube in a latent space, design a rank constraint W = ED with an encoder E = [I_K; -I_K], and introduce Hebbian, pseudoinverse, and optimized decoders intended to satisfy the fixed-point condition D eta_mu = xi_mu. They report that the pseudoinverse and optimized rules store overlapping patterns, achieve an approximately linear storage capacity p/N around 0.3 and 0.5 in simulations, and show pattern completion, including an MNIST example. The paper also provides simulation code in a public repository.","tokens_in":11877,"tokens_out":7759,"duration_ms":76245,"significance":"If the central claims hold, the paper offers a scalable constructive recipe for spiking associative memory with overlapping patterns, directly connecting Hopfield-type memory to the latent-boundary framework of spiking networks. The geometric viewpoint is interpretable, the code is released, and the simulations demonstrate stable recall and pattern completion in networks up to N=1600. However, the analytical stability condition is only checked against an assumed equal-rate neural pattern rather than measured spiking rates, the capacity thresholds are empirical and come without error bars, and a normalization inconsistency with the parameter kappa undermines the stated derivation of the pseudoinverse rule. These issues bear directly on the paper's two headline claims.","major_comments":[{"comment":"The constant kappa in eta_mu = ReLU(kappa E xi_mu) is never assigned a value. Combining the pseudoinverse decoder with this definition gives D_pinv eta_mu = kappa xi_mu (using E^T ReLU(E xi) = xi), which equals xi_mu only if kappa = 1 and the vertex coordinate c is normalized. As written, Eq. (8) is therefore not satisfied by the pseudoinverse rule for a general kappa, and the latent fixed point would be kappa xi_mu rather than xi_mu. The authors should state the normalization of kappa and c, or rescale the decoder accordingly.","section":"Sec. 2.2.3, Eqs. (10)-(11)"},{"comment":"The headline claim of linear storage capacity is supported only by reading thresholds off a single overlap-versus-load curve at N=400, 800, and 1600, with no error bars, no distribution over pattern draws, and no analytical derivation of the p/N values of 0.3 and 0.5. The dimension-counting argument in Sec. 2.2.3 is only a necessary condition (p <= 2K = N) and does not establish why the spiking dynamics should remain stable up to the reported loads. The authors should either provide a fixed-point analysis of the designed decoder or add a systematic capacity measurement with confidence intervals and a defined success criterion.","section":"Sec. 3.2, Fig. 4b"},{"comment":"The verification of Eq. (8) in Fig. 3e uses the assumed pattern eta from Eq. (11), not the time-averaged firing rates of the simulated network. For the Hebbian and pseudoinverse rules, the self-connection gamma is imposed post hoc, so the implemented weight matrix is no longer W = ED and the exact constraint D eta = xi is broken in the simulated network. Since the spiking dynamics are oscillatory and gamma is tuned for each load (Fig. 4c), the claim that stable attractors sit exactly at the hypercube vertices is not directly established for the actual spiking system. The authors should measure the time-averaged firing rates at a recalled vertex and show that D<r> is approximately xi, or explain why the post-hoc self-connection does not affect the fixed-point condition.","section":"Sec. 2.2.3, Fig. 3e, Appendix C.1"}],"minor_comments":[{"comment":"The sentence 'Given that D has KN = 2K^2 elements, this suggests that the linear system can only have a solution provided p <= 2K' is a heuristic necessary-condition argument, not a proof of feasibility for the specific ReLU-defined eta matrix; the text should label it as such.","section":"Sec. 2.2.3, Eq. (12)"},{"comment":"The extra 0th latent dimension is introduced only in the appendix, while the main text describes a (K+1)-dimensional boundary before Eq. (7) defines a K-dimensional encoder; the relation between the rank-K latent dynamics and the rank-(K+1) weight matrix should be clarified in the main text.","section":"Appendix C.2"},{"comment":"The caption does not define the quantity on the y-axis or explain the 'horizontal line indicates precise stability'; please state what is being plotted and how the horizontal line is determined.","section":"Figure 3e"},{"comment":"The introduction cites prior spiking associative memory models but does not quantify the memory loads they achieve; adding a brief quantitative comparison would help support the scalability claim.","section":"Sec. 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a promising empirical demonstration, and the geometric framework is genuinely interesting, but the theoretical claims currently outrun the analysis. The kappa inconsistency is easy to repair, and the capacity/stability issues could be addressed with additional measurements of the actual spiking rates and more systematic capacity curves. I would encourage the authors to resubmit after revision rather than reject, since the core simulations appear to support the qualitative claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [Colleague],\n\nHere's my take on Podlaski & Machens. The paper has a genuinely new result: a simple recipe for storing many overlapping associative memories in a low-rank, all-inhibitory spiking network, using hypercube vertices on a latent manifold as the memory states. The construction is clean—E=[I;-I], thresholds making a hypercube, and three learning rules (Hebbian, pseudoinverse, optimized least-squares) translated onto the decoder. The simulations back the central claims: recall, pattern completion, and an MNIST demonstration at 1568 neurons. Code is available. If you work on spiking memory, this is worth a careful read.\n\nThe soft spots are real but not fatal. The capacity scaling is empirical, read off three network sizes (N=400, 800, 1600) without error bars. The stability condition Eq (8) is verified against the assumed equal-rate pattern, not against measured spiking rates. The kappa in Eq (11) is never fixed; as written, the pseudoinverse rule gives D eta = kappa xi, not xi, unless kappa=1. That looks like a presentational slip, not a deep flaw. More substantively, the self-connection gamma is tuned per load, and for Hebbian and pseudoinverse it's imposed post hoc, so the implemented network deviates from the derived condition. The all-inhibitory extra dimension is imported from Podlaski & Machens (2024), so that proof is elsewhere.\n\nThe reader's 'circularity' concern is overstated. Choosing patterns first and then fitting weights is standard Hopfield procedure; capacity measurement of a learning rule is not circular.\n\nWho is this for? Computational neuroscientists interested in spiking attractors and neuromorphic memory. The paper is a case study, and it is transparent about being one. With error bars, a baseline comparison, and a stated kappa/gamma, it would be much stronger. As it stands, it deserves a serious referee, but I'd expect revisions.\n\nRecommendation: send to peer review.","headline":"A clean construction for overlapping spiking associative memory with linear empirical capacity, but the theory is weaker than it looks and several implementation details need tightening.","tokens_in":12387,"tokens_out":9648,"would_cite":true,"duration_ms":103258,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["92B20"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper shows that low-rank spiking networks with purely inhibitory connectivity can store up to about one overlapping memory per two neurons as stable attractors, by placing memories at the vertices of a hypercubic latent manifold.","keywords":["associative memory","spiking neural networks","low-rank connectivity","Hopfield network","latent manifold","storage capacity","pattern completion","all-inhibitory networks"],"falsifier":"Run the optimized rule at load $p/N=0.55$ in a network of $N=400$ neurons and measure the final overlap for each stored pattern after recall; if any pattern's overlap falls below the recall criterion, the reported $0.5N$ capacity is not a sharp limit. A second test is to sample each active neuron's target rate independently around $\\kappa$; if the attractors lose stability at small spreads near capacity, the equal-rate assumption is doing the work.","tokens_in":11327,"feed_emoji":"🧠","tokens_out":7304,"duration_ms":66989,"temperature":0.7,"pith_summary":"Associative memory in spiking neural networks has usually been limited to a few non-overlapping patterns. This paper claims that an all-inhibitory, rank-constrained spiking network can store many overlapping binary memories by arranging them at the vertices of a hypercube in a low-dimensional latent space. With pseudoinverse or optimized decoders, the authors report that storage capacity grows linearly with neuron number—roughly $0.3N$ and $0.5N$ memories respectively—and that noisy initial states reliably complete to the stored pattern. If correct, this gives a concrete recipe for high-capacity, content-addressable memory in biologically plausible circuits, and a direct bridge between spiking dynamics and the classical Hopfield model.","feed_headline":"Spiking networks store one memory per two neurons","feed_subtitle":"A hypercube latent geometry gives all-inhibitory spiking circuits Hopfield-like recall and pattern completion.","key_machinery":"The load-bearing object is the rank-$(K+1)$ convex, piecewise-linear latent manifold: recurrent weights are decomposed as $W=ED$, the latent variable is $y=Dr$, and each neuron's threshold becomes a hyperplane face of a hypercubic cone. The decoders are what implement memory: requiring $D\\eta_\\mu=\\xi_\\mu$ makes the leak dynamics at each vertex point back toward the vertex, turning it into an attractor. The extra latent dimension adds a constant inhibitory component so the boundary dynamics are stable, and a negative self-connection $-\\gamma$ (tuned per load) prevents each neuron from stabilizing only its own face.","core_discovery":"The central result is that each memory is a stable attractor exactly when the decoder satisfies $D\\eta_\\mu=\\xi_\\mu$ for every pattern $\\mu$, with neural patterns defined as $\\eta_\\mu=\\mathrm{ReLU}(\\kappa E\\xi_\\mu)$. With the encoding $E=[I_K;-I_K]$, the $N=2K$ neurons form the faces of a $K$-dimensional hypercube (plus one extra latent dimension that guarantees all-inhibitory stability), and the $2^K$ vertices are the candidate binary patterns. The Hebbian rule cannot meet the equality constraint and produces spurious mixtures; the pseudoinverse rule $D=\\xi Q^{-1}\\xi^\\top E^\\top$ stores up to about $0.3N$ patterns, and the optimized least-squares rule extends this to about $0.5N$ in simulations of up to 1600 neurons. These networks also complete patterns from noisy initial states, correcting up to 30% flipped bits at low load, and the same construction stores correlated MNIST-like patterns.","pith_inferences":["A natural next test is whether the same hypercube recipe can host dense associative memories with exponential capacity; the paper does not explore higher-order vertices or beyond-pairwise interactions.","The equal-rate assumption predicts that firing-rate heterogeneity among active neurons shrinks capacity; injecting rate variance into $\\eta_\\mu$ is a direct stress test the paper leaves undone.","Because the optimal self-inhibition $\\gamma$ depends on load, a biological implementation would need an activity-dependent reset or adaptation strength, an implicit constraint the authors do not discuss.","The encoding $N=2K$ costs two neurons per latent dimension; an asymmetric or non-orthogonal encoder could change the capacity coefficient, but the paper does not analyze that trade-off."],"forward_implications":["The pseudoinverse and optimized decoders turn hypercube vertices into stable spiking attractors, while the Hebbian rule fails because it cannot satisfy the equality constraint $D\\eta_\\mu=\\xi_\\mu$.","Storage capacity scales linearly with the number of neurons, at about $0.3N$ patterns for the pseudoinverse rule and $0.5N$ for the optimized rule.","Overlapping memories work: every pattern activates half the neurons on average, so any pair shares about 50% of its active neurons, and the network still separates them.","Pattern completion is robust at low load (up to 30% of pattern bits flipped) and degrades gracefully to zero tolerance as the load approaches capacity.","The construction requires only inhibitory recurrent connections, so the mechanism is compatible with circuits that lack long-range excitatory recurrence."],"supporting_citations":[{"why":"Defines the original associative memory model that this paper's memory patterns and Hebbian rule directly mirror.","marker":"Hopfield, 1982"},{"why":"Introduces the pseudoinverse method for storing patterns without errors, which the paper adapts to the spiking equality constraint.","marker":"Personnaz et al., 1985"},{"why":"Extends pseudoinverse storage to spin-glass-like networks and underlies the linear-system feasibility argument.","marker":"Kanter and Sompolinsky, 1987"},{"why":"Provides the all-inhibitory boundary-stability construction that the extra latent dimension relies on.","marker":"Podlaski and Machens, 2024"},{"why":"Establishes the convex-optimization view of spiking network latent dynamics used to derive the latent boundary.","marker":"Mancoo et al., 2020"},{"why":"Gives the geometric robustness framework for spiking attractors, including the role of self-connections.","marker":"Calaim et al., 2022"},{"why":"Underlies the capacity bound $p \\le 2K$ that the paper cites for the linear system of vertex constraints.","marker":"Cover, 1965"},{"why":"Provides the statistical capacity perspective that motivates comparing the spiking construction to perceptron and Hopfield capacity.","marker":"Gardner, 1988"}],"fun_headline_variants":["Hypercube geometry gives spiking networks Hopfield-like recall","Spiking networks store overlapping memories on hypercube vertices","Low-rank spiking net stores up to half a memory per neuron","All-inhibitory spiking nets use hypercube for pattern completion","From Hebb to hypercube: spiking associative memory scales linearly"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The construction assumes that active neurons in a memory fire at one common average rate and that the latent-manifold equations capture the spiking network exactly, so that stabilizing each vertex in the reduced system is enough to stabilize the spiking attractor.","fun_headline_variants_meta":{"raw":{"variants":["Hypercube geometry gives spiking networks Hopfield-like recall","Spiking networks store overlapping memories on hypercube vertices","Low-rank spiking net stores up to half a memory per neuron","All-inhibitory spiking nets use hypercube for pattern completion","From Hebb to hypercube: spiking associative memory scales linearly"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000256,"raw_usage":{"total_tokens":1589,"prompt_tokens":971,"completion_tokens":618,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":587,"completion_tokens_details":{"reasoning_tokens":532}},"tokens_in":587,"tokens_out":618,"duration_ms":5833,"temperature":1.0,"reasoning_tokens":532,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:03:56.964718+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the optimized rule at load $p/N=0.55$ in a network of $N=400$ neurons and measure the final overlap for each stored pattern after recall; if any pattern's overlap falls below the recall criterion, the reported $0.5N$ capacity is not a sharp limit. A second test is to sample each active neuron's target rate independently around $\\kappa$; if the attractors lose stability at small spreads near capacity, the equal-rate assumption is doing the work.","supporting_citations":[{"cited_title":"Information storage and retrieval in spin-glass like neural networks","cited_arxiv_id":null,"evidence_quote":"Introduces the pseudoinverse method for storing patterns without errors, which the paper adapts to the spiking equality constraint."},{"cited_title":"Associative recall of memory without errors","cited_arxiv_id":null,"evidence_quote":"Extends pseudoinverse storage to spin-glass-like networks and underlies the linear-system feasibility argument."},{"cited_title":"Approximating nonlinear functions with latent boundaries in low-rank excitatory-inhibitory spiking networks","cited_arxiv_id":null,"evidence_quote":"Provides the all-inhibitory boundary-stability construction that the extra latent dimension relies on."},{"cited_title":"Understanding spiking networks through convex optimization","cited_arxiv_id":null,"evidence_quote":"Establishes the convex-optimization view of spiking network latent dynamics used to derive the latent boundary."},{"cited_title":"The geometry of robustness in spiking neural networks","cited_arxiv_id":null,"evidence_quote":"Gives the geometric robustness framework for spiking attractors, including the role of self-connections."},{"cited_title":"Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition","cited_arxiv_id":null,"evidence_quote":"Underlies the capacity bound $p \\le 2K$ that the paper cites for the linear system of vertex constraints."},{"cited_title":"The space of interactions in neural network models","cited_arxiv_id":null,"evidence_quote":"Provides the statistical capacity perspective that motivates comparing the spiking construction to perceptron and Hopfield capacity."}],"review_version":1}