{"id":"9ff03784-65e8-48e4-b350-c538a7cd380c","arxiv_id":"2511.02584","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Maximizing per-neuron redundancy between recurrent and target inputs in Hopfield networks yields a memory capacity of 1.59–1.70, versus 0.14 for the Hebbian rule.","lead":"The paper shows that neurons in a Hopfield network store memories when their recurrent input and the target teaching signal carry redundant information about the output, and that directly training each neuron to maximize this redundancy gives a memory capacity of about 1.59 patterns per neuron, versus 0.14 for the standard Hebbian rule. It offers an information-theoretic design principle for associative memories and a practical learning rule that beats two strong existing bas","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The infomorphic network's recurrent weights are never specified to have zero diagonal; if self-connections are learned, capacity 1.59 is trivially inflated.","rationale":"The reader's weakest_assumption about one-step training vs iterated testing is plausible, but maximizing redundancy is conceptually about making the recurrent input carry the target's information, so transfer to target-free dynamics is expected and could be checked. The self-connection omission is a more direct and more easily fatal threat to the central numerical claim: if the infomorphic network learns self-loops, the reported α≈1.59 is not a property of recurrent associative memory but of trivial self-feedback. The paper otherwise has positive features—20 seeds, bootstrapped CIs, clear pseudocode—but the missing diagonal specification and absent code leave a concrete loophole. I would keep the verdict CONDITIONAL, pending the diagonal-zero check, rather than rejecting outright, because the authors may have intended the classical zero-diagonal convention to carry over.","tokens_in":15137,"tokens_out":15460,"duration_ms":180285,"concrete_test":"Set the diagonal of w^R to zero for all infomorphic neurons during training (and verify the same masking for the Hebbian and baseline implementations), then re-run the capacity protocol of §A.3 for N=100, 20 seeds. If the median α_c stays at ≈1.59, the concern is resolved; if it drops materially, the headline capacity depends on self-feedback.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"In A.1, the infomorphic recurrent input is defined as r_i = Σ_j w^R_ij y_j (Eq. 8) with no j≠i restriction, and Table 1/Pseudocode 3 update all weights without diagonal masking. The classical zero-diagonal convention of §2.1 is not explicitly carried over. Since training initializes the network at the target pattern ξ and uses T = w_T ξ_i, a learned self-loop w^R_ii adds a direct term w^R_ii ξ_i to the recurrent input. Maximizing Π_red can then be achieved by making w^R_ii large and positive, which makes every pattern a fixed point of the test-time sign dynamics regardless of the other weights. This would inflate α_c relative to the Hebbian baseline (which has w_ii=0) and the MPF/Descent L2 comparisons, and it would mean the reported capacity is not evidence for redundancy as a recurrent associative-memory principle. The manuscript does not state that the infomorphic network enforces w_ii=0, and no code is provided to check.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies Hopfield networks through the lens of Partial Information Decomposition (PID). It first analyzes a classical Hebbian network and reports that, below the memory capacity, each neuron's output is predominantly determined redundantly by the recurrent input and a target/teaching input, while unique and synergistic contributions rise only above capacity. Based on this observation, the authors construct 'infomorphic' Hopfield networks in which each neuron directly maximizes the redundant information between its recurrent input and a target input, using a differentiable PID measure and soft binning. They report a memory capacity of α_c ≈ 1.59 for the heuristic redundancy goal, rising to α_c ≈ 1.7 after optimizing the PID goal parameters, compared with α_H ≈ 0.14 for Hebbian learning and lower capacities for MPF and Descent L2 baselines. The paper interprets redundancy maximization as a design principle for associative memory.","tokens_in":15425,"tokens_out":6178,"duration_ms":68091,"significance":"If the reported capacities are correct, the result is significant: a purely local, information-theoretic objective would nearly reach the theoretical storage bound α = 2 for binary Hopfield networks, far exceeding classical Hebbian learning and matching or beating recent learning rules. The paper's strengths include the use of multiple seeds and bootstrap confidence intervals, systematic comparison with MPF and Descent L2 baselines, a finite-size analysis, and a clear constructive link between an empirical PID observation and a trainable objective. The central claim, however, rests on several empirical details that are not fully pinned down — most importantly whether the infomorphic recurrent weights include self-connections, and how the capacity definition handles excluded finite-size failures. These issues are addressable but must be resolved before the main claim can be accepted.","major_comments":[{"comment":"The infomorphic recurrent input is defined as r_i = Σ_j w^R_ij y_j with no j≠i restriction, and no diagonal masking is mentioned in the architecture, parameter table, or training pseudocode. The zero-diagonal convention is stated only for the Hebbian rule in §2.1. If self-connections are learned, a large positive w^R_ii adds a direct term w^R_ii ξ_i to the recurrent input during both training and testing, making every stored pattern a fixed point of the test-time sign dynamics regardless of the other weights. This would trivially inflate the reported capacity relative to the zero-diagonal baselines. The manuscript must state explicitly whether w^R_ii = 0 is enforced; if not, the capacities should be rerun with diagonal masking. Since code is not yet available, this ambiguity is load-bearing.","section":"A.1/A.4, Eq. (8), Pseudocode 3"},{"comment":"The capacity is defined as the largest α such that a_cos(α) > 0.95 'for all α < α_c', but with the caveat 'excluding finite size effects.' Appendix E and Fig. 7B show that for N = 100, training fails at low memory loads because some neurons have zero entropy. The headline value α_c ≈ 1.59 therefore excludes a range of α values, and it is unclear how many data points are excluded or on what criterion. Please report the raw accuracy curves including the low-α failures, specify the exclusion rule precisely, and preferably repeat the capacity measurements at larger N so that the exclusion is not needed. Without this, the capacity is not an unambiguous property of the learning rule.","section":"A.3, Fig. 7B"},{"comment":"Training optimizes a one-step, target-aided, stochastic sigmoid objective: the network is run once from the exact pattern with T present, and gradients of the PID estimate are used. Testing removes T and iterates deterministic sign dynamics for up to 100 steps. The paper does not analyze whether gradients of the one-step redundancy objective transfer to the iterated attractor condition. Fig. 7B demonstrates that training can fail in some low-α regimes, so the transfer is not trivially guaranteed. This does not invalidate the empirical capacity result, but it weakens the interpretation that redundancy maximization per se is the operative principle; the capacity may instead be a property of the specific training pipeline. I would like to see either a fixed-point/stability analysis of the learned weights or control experiments with longer training rollouts and/or target-absent perturbations","section":"A.2 vs. A.3"},{"comment":"The paper uses the PID profile of the trained infomorphic networks (Fig. 2D) to 'explain' the capacity transition, but Π_red is precisely the objective that was maximized during training. Its high value below capacity is therefore partly by construction, not independent evidence for redundancy as a principle. The independent support is the capacity comparison in Fig. 2C and Fig. 5, which is strong. Please reframe the PID analysis as a descriptive check of the optimized solution, or compare it against networks trained with other objectives, rather than presenting it as a confirmation of the principle.","section":"§3.2, Fig. 2D"}],"minor_comments":[{"comment":"The text says both optimized goals 'maximize redundancy,' but G(ii) has γ_red = 0.25 and γ_unq,T = 0.48; redundancy is not the dominant term. Please clarify that only G(i) is redundancy-dominated, while G(ii) resembles the mutual-information objective.","section":"§3.4, Eq. (7)"},{"comment":"Stability is described in Appendix A.3.1, but the main text refers to 'subsection A.2' for details on stability. Please correct the cross-reference.","section":"§3.5, A.3.1"},{"comment":"The caption says 'The curves in A-D' but the figure contains only panels A, B, and C. Please adjust.","section":"Figure 5 caption"},{"comment":"The paper states that code 'will be made available together with publication.' Given that the diagonal self-connection ambiguity and the capacity exclusion rule cannot be checked without code, I recommend providing the code with the revision or at least an explicit pseudocode statement of whether the diagonal is masked.","section":"Code Availability"},{"comment":"The formula for the expected number of zero-entropy neurons is stated without derivation. A short derivation would help the reader understand when finite-size effects are expected to be negligible, especially for the N = 100 capacity measurements.","section":"Appendix E"}],"recommendation":"major_revision","confidential_remarks":"The most serious issue is the self-connection ambiguity. If the infomorphic network has no explicit zero-diagonal constraint, the reported capacities may be inflated by a trivial self-loop mechanism, and the main conclusion could collapse. The capacity-exclusion rule and the one-step-to-iterated transfer are also important but are more clearly fixable. I would suggest asking the authors to confirm whether w_ii = 0 is enforced, provide code, and rerun capacities with an enforced zero diagonal if necessary. If the capacities survive, the paper would be a solid contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: the paper has a genuinely interesting observation — in classical Hebbian Hopfield networks below capacity, per-neuron PID shows redundancy dominating, and a learning rule that directly maximizes that redundancy reportedly reaches capacity ~1.59, with optimized variants up to 1.70. But there is a serious, checkable gap: nothing in the manuscript states that the infomorphic network enforces zero self-connections. Eq (8) sums over all j with no j≠i restriction, Table 1 initializes all w^R entries from a normal distribution, and Pseudocode 3 updates every weight. If self-loops are learned, a large positive w^R_ii makes every stored pattern a fixed point regardless of other weights, so the reported capacity would be inflated and the comparison to Hebbian (which sets w_ii=0) and to MPF/Descent L2 would be unfair. This must be resolved before the headline claim can be trusted.\n\nWhat the paper does well: the PID analysis in Fig 1D is a clean diagnostic, and the observation that mutual information works by implicitly maximizing redundancy is a nice explanatory link. The constructive demonstration that a local PID objective can reach capacities in that range, if the diagonal issue is resolved, would be a real step beyond the infomorphic proof-of-concept [13]. The authors report 20 seeds with bootstrap CIs, and the appendices are detailed enough to reproduce most of the pipeline.\n\nSoft spots, in proportion: the capacity definition in A.3 explicitly excludes finite-size effects, and Fig 7B shows training failures at low α for N=100; those exclusions are post-hoc and could make the capacity estimate optimistic. The optimized goals (1.68–1.70) come from fitting γ on the capacity metric itself; verifying on 20 held-out seeds helps, but it is still selection on the test metric. The one-step training to iterated attractor transfer is assumed, not proven; empirically it seems to work here, so I'd call that a minor concern. And the 'principle' interpretation rests on one specific PID measure, I_sx^∩, plus the observed correlation — the constructive result doesn't depend on that interpretation, but the framing is stronger than the evidence.\n\nBottom line: this deserves serious peer review, but only after the authors confirm the diagonal is zero (or release code showing it). If they do, the central claim is likely solid and important. As written, I wouldn't cite the capacity numbers. Send it to review, but ask for code and an explicit statement about self-connections.","headline":"Interesting but unverified: redundancy-maximization capacity numbers may be inflated by undiscussed self-connections; the PID analysis of Hebbian networks is a solid contribution on its own.","tokens_in":15968,"tokens_out":4066,"would_cite":false,"duration_ms":41204,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["94A17","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"Per-neuron redundancy maximization suffices to build Hopfield memories near capacity 1.7.","keywords":["associative memory","Hopfield networks","partial information decomposition","redundancy maximization","infomorphic neurons","memory capacity","information-theoretic learning","local learning rules"],"falsifier":"Train a redundancy-maximizing infomorphic network at a load below 1.59 and check whether the trained weights satisfy the attractor condition sign(∑_j w_ij ξ_j) = ξ_i for every stored pattern. If the fixed-point condition fails while the one-step redundancy objective is high, the reported capacity comes from the training pipeline rather than from the redundancy principle itself.","tokens_in":14975,"feed_emoji":"🧠","tokens_out":5853,"duration_ms":62576,"temperature":0.7,"pith_summary":"The paper asks what local information-processing goal a neuron should pursue so that a recurrent network can store and retrieve patterns. Using partial information decomposition, it first shows that in classical Hopfield networks, successful retrieval coincides with high redundant information shared by the recurrent input and the teaching signal, while unique and synergistic information stay near zero and rise only after capacity is exceeded. It then constructs 'infomorphic' Hopfield networks in which every neuron is trained by gradient ascent to maximize that redundant information between its two inputs. With only this local goal, the network stores about 1.59 patterns per neuron, more than ten times the classical Hebbian capacity, and optimized composite goals reach about 1.7, approaching the theoretical bound of 2. A sympathetic reader would take this as evidence that redundancy maximization, rather than coincidence-based Hebbian plasticity, is the underlying principle of associative memory.","feed_headline":"Redundancy maximization pushes Hopfield memory capacity to 1.7","feed_subtitle":"A neuron-wise information goal stores over ten times more patterns than classical Hebbian learning.","key_machinery":"Partial Information Decomposition (PID), specifically the differentiable I_sx^∩ redundancy measure, applied to each neuron's two inputs—the weighted recurrent sum R and the target pattern element T—together with a soft kernel-smoothed binning that makes the discrete PID estimates differentiable with respect to the weights. This converts an information-theoretic analysis tool into a local learning objective: each infomorphic neuron updates its recurrent weights by gradient ascent on a goal function (typically G = Π_red) computed from one synchronous forward pass with the target present; at test time the target is removed and retrieval runs the deterministic sign dynamics.","core_discovery":"The central claim is that the local principle governing associative memory is the maximization of redundant information between the recurrent input R and the target teaching signal T about the neuron's output Y. In classical Hebbian networks, this PID redundancy is the dominant information atom below capacity and collapses exactly as retrieval fails. Networks whose weights are trained by gradient ascent on the per-neuron goal G = Π_red store all patterns up to a memory load of 1.59, and composite goals found by hyperparameter search reach 1.7, outperforming minimum probability flow and L2-descent baselines while staying below the α = 2 bound.","pith_inferences":["Because training and testing differ—one forward pass with the target present versus iterated sign dynamics without it—the reported capacity essentially tests a train-to-attractor transfer; a direct check of the fixed-point condition would separate principle from pipeline.","The differentiable soft-binned PID estimate introduces estimation hyperparameters (e.g., the number of recurrent bins) that likely modulate the measured capacity; a bin-count sweep would probe the robustness of the 1.59–1.7 numbers.","The same redundancy objective could be applied to networks with hidden units or layered architectures; whether the principle scales beyond single-layer visible neurons is a natural follow-up.","The paper itself notes the rule is computationally expensive and biologically implausible, which sets a concrete challenge: approximate the redundancy gradient with local statistics to see whether the principle survives under biologically plausible constraints."],"forward_implications":["If correct, binary Hopfield networks can be trained by a purely local, per-neuron objective to reach memory loads far above Hebbian learning and competitive with recent goal-based rules.","The success of classical mutual-information objectives is explained by their implicit maximization of redundancy, not by anything special about mutual information itself.","The sharp rise of unique and synergistic information at the capacity boundary provides a diagnostic signature for when a learning rule is about to fail.","Optimized goals suggest that suppressing recurrent-unique and synergistic information slightly improves capacity, pointing to concrete design targets for future learning rules.","Spatially correlated patterns, which hurt Hebbian learning, can improve the capacity of redundancy-trained networks, so the principle may extend to structured data."],"fun_headline_variants":["Redundancy rule lifts Hopfield memory tenfold","Maximizing redundancy boosts Hopfield capacity to 1.59","Hopfield networks get 11x memory via redundancy goal","Redundancy maximization: new principle for associative memory","Neuron-wise redundancy beats Hebbian in Hopfield nets"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing assumption is that maximizing a one-step, target-aided redundancy estimate during training also makes the no-target, iterated sign dynamics converge to the stored patterns.","fun_headline_variants_meta":{"raw":{"variants":["Redundancy rule lifts Hopfield memory tenfold","Maximizing redundancy boosts Hopfield capacity to 1.59","Hopfield networks get 11x memory via redundancy goal","Redundancy maximization: new principle for associative memory","Neuron-wise redundancy beats Hebbian in Hopfield nets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000145,"raw_usage":{"total_tokens":1022,"prompt_tokens":754,"completion_tokens":268,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":498,"completion_tokens_details":{"reasoning_tokens":185}},"tokens_in":498,"tokens_out":268,"duration_ms":3262,"temperature":1.0,"reasoning_tokens":185,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T00:07:07.218141+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a redundancy-maximizing infomorphic network at a load below 1.59 and check whether the trained weights satisfy the attractor condition sign(∑_j w_ij ξ_j) = ξ_i for every stored pattern. If the fixed-point condition fails while the one-step redundancy objective is high, the reported capacity comes from the training pipeline rather than from the redundancy principle itself.","supporting_citations":[],"review_version":1}