Pith. sign in

REVIEW 4 major objections 5 minor 37 references

Redundancy Maximization as a Principle of Associative Memory Learning in Hopfield Networks

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Per-neuron redundancy maximization suffices to build Hopfield memories near capacity 1.7.

desk verdict Interesting but unverified: redundancy-maximization capacity numbers may be inflated by undiscussed self-connections; the PID analysis of Hebbian networks is a solid contribution on its own. read the letter →

arxiv 2511.02584 v2 pith:JA6QPW55 submitted 2025-11-04 cs.IT cs.LGcs.NEmath.ITphysics.comp-ph

Mark Blümel , Andreas C. Schneider , Valentin Neuhaus , David A. Ehrlich , Marcel Graetz , Michael Wibral , Abdullah Makkeh , Viola Priesemann This is my paper · ORCID
classification cs.ITcs.LGcs.NEmath.ITphysics.comp-ph MSC 94A1768T07
keywords associativememoryHopfieldnetworkspartialinformationdecompositionredundancymaximizationinfomorphicneuronscapacityinformation-theoreticlearninglocalrules
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks what local information-processing goal a neuron should pursue so that a recurrent network can store and retrieve patterns. Using partial information decomposition, it first shows that in classical Hopfield networks, successful retrieval coincides with high redundant information shared by the recurrent input and the teaching signal, while unique and synergistic information stay near zero and rise only after capacity is exceeded. It then constructs 'infomorphic' Hopfield networks in which every neuron is trained by gradient ascent to maximize that redundant information between its two inputs. With only this local goal, the network stores about 1.59 patterns per neuron, more than ten times the classical Hebbian capacity, and optimized composite goals reach about 1.7, approaching the theoretical bound of 2. A sympathetic reader would take this as evidence that redundancy maximization, rather than coincidence-based Hebbian plasticity, is the underlying principle of associative memory.

What carries the argument

Partial Information Decomposition (PID), specifically the differentiable I_sx^∩ redundancy measure, applied to each neuron's two inputs—the weighted recurrent sum R and the target pattern element T—together with a soft kernel-smoothed binning that makes the discrete PID estimates differentiable with respect to the weights. This converts an information-theoretic analysis tool into a local learning objective: each infomorphic neuron updates its recurrent weights by gradient ascent on a goal function (typically G = Π_red) computed from one synchronous forward pass with the target present; at test time the target is removed and retrieval runs the deterministic sign dynamics.

What would settle it

Train a redundancy-maximizing infomorphic network at a load below 1.59 and check whether the trained weights satisfy the attractor condition sign(∑_j w_ij ξ_j) = ξ_i for every stored pattern. If the fixed-point condition fails while the one-step redundancy objective is high, the reported capacity comes from the training pipeline rather than from the redundancy principle itself.

Watch

Extended reading notes

Core claim

The central claim is that the local principle governing associative memory is the maximization of redundant information between the recurrent input R and the target teaching signal T about the neuron's output Y. In classical Hebbian networks, this PID redundancy is the dominant information atom below capacity and collapses exactly as retrieval fails. Networks whose weights are trained by gradient ascent on the per-neuron goal G = Π_red store all patterns up to a memory load of 1.59, and composite goals found by hyperparameter search reach 1.7, outperforming minimum probability flow and L2-descent baselines while staying below the α = 2 bound.

Load-bearing premise

The load-bearing assumption is that maximizing a one-step, target-aided redundancy estimate during training also makes the no-target, iterated sign dynamics converge to the stored patterns.

Editorial extensions

If this is right

  • If correct, binary Hopfield networks can be trained by a purely local, per-neuron objective to reach memory loads far above Hebbian learning and competitive with recent goal-based rules.
  • The success of classical mutual-information objectives is explained by their implicit maximization of redundancy, not by anything special about mutual information itself.
  • The sharp rise of unique and synergistic information at the capacity boundary provides a diagnostic signature for when a learning rule is about to fail.
  • Optimized goals suggest that suppressing recurrent-unique and synergistic information slightly improves capacity, pointing to concrete design targets for future learning rules.
  • Spatially correlated patterns, which hurt Hebbian learning, can improve the capacity of redundancy-trained networks, so the principle may extend to structured data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because training and testing differ—one forward pass with the target present versus iterated sign dynamics without it—the reported capacity essentially tests a train-to-attractor transfer; a direct check of the fixed-point condition would separate principle from pipeline.
  • The differentiable soft-binned PID estimate introduces estimation hyperparameters (e.g., the number of recurrent bins) that likely modulate the measured capacity; a bin-count sweep would probe the robustness of the 1.59–1.7 numbers.
  • The same redundancy objective could be applied to networks with hidden units or layered architectures; whether the principle scales beyond single-layer visible neurons is a natural follow-up.
  • The paper itself notes the rule is computationally expensive and biologically implausible, which sets a concrete challenge: approximate the redundancy gradient with local statistics to see whether the principle survives under biologically plausible constraints.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies Hopfield networks through the lens of Partial Information Decomposition (PID). It first analyzes a classical Hebbian network and reports that, below the memory capacity, each neuron's output is predominantly determined redundantly by the recurrent input and a target/teaching input, while unique and synergistic contributions rise only above capacity. Based on this observation, the authors construct 'infomorphic' Hopfield networks in which each neuron directly maximizes the redundant information between its recurrent input and a target input, using a differentiable PID measure and soft binning. They report a memory capacity of α_c ≈ 1.59 for the heuristic redundancy goal, rising to α_c ≈ 1.7 after optimizing the PID goal parameters, compared with α_H ≈ 0.14 for Hebbian learning and lower capacities for MPF and Descent L2 baselines. The paper interprets redundancy maximization as a design principle for associative memory.

Significance. If the reported capacities are correct, the result is significant: a purely local, information-theoretic objective would nearly reach the theoretical storage bound α = 2 for binary Hopfield networks, far exceeding classical Hebbian learning and matching or beating recent learning rules. The paper's strengths include the use of multiple seeds and bootstrap confidence intervals, systematic comparison with MPF and Descent L2 baselines, a finite-size analysis, and a clear constructive link between an empirical PID observation and a trainable objective. The central claim, however, rests on several empirical details that are not fully pinned down — most importantly whether the infomorphic recurrent weights include self-connections, and how the capacity definition handles excluded finite-size failures. These issues are addressable but must be resolved before the main claim can be accepted.

major comments (4)
  1. [A.1/A.4, Eq. (8), Pseudocode 3] The infomorphic recurrent input is defined as r_i = Σ_j w^R_ij y_j with no j≠i restriction, and no diagonal masking is mentioned in the architecture, parameter table, or training pseudocode. The zero-diagonal convention is stated only for the Hebbian rule in §2.1. If self-connections are learned, a large positive w^R_ii adds a direct term w^R_ii ξ_i to the recurrent input during both training and testing, making every stored pattern a fixed point of the test-time sign dynamics regardless of the other weights. This would trivially inflate the reported capacity relative to the zero-diagonal baselines. The manuscript must state explicitly whether w^R_ii = 0 is enforced; if not, the capacities should be rerun with diagonal masking. Since code is not yet available, this ambiguity is load-bearing.
  2. [A.3, Fig. 7B] The capacity is defined as the largest α such that a_cos(α) > 0.95 'for all α < α_c', but with the caveat 'excluding finite size effects.' Appendix E and Fig. 7B show that for N = 100, training fails at low memory loads because some neurons have zero entropy. The headline value α_c ≈ 1.59 therefore excludes a range of α values, and it is unclear how many data points are excluded or on what criterion. Please report the raw accuracy curves including the low-α failures, specify the exclusion rule precisely, and preferably repeat the capacity measurements at larger N so that the exclusion is not needed. Without this, the capacity is not an unambiguous property of the learning rule.
  3. [A.2 vs. A.3] Training optimizes a one-step, target-aided, stochastic sigmoid objective: the network is run once from the exact pattern with T present, and gradients of the PID estimate are used. Testing removes T and iterates deterministic sign dynamics for up to 100 steps. The paper does not analyze whether gradients of the one-step redundancy objective transfer to the iterated attractor condition. Fig. 7B demonstrates that training can fail in some low-α regimes, so the transfer is not trivially guaranteed. This does not invalidate the empirical capacity result, but it weakens the interpretation that redundancy maximization per se is the operative principle; the capacity may instead be a property of the specific training pipeline. I would like to see either a fixed-point/stability analysis of the learned weights or control experiments with longer training rollouts and/or target-absent perturbations
  4. [§3.2, Fig. 2D] The paper uses the PID profile of the trained infomorphic networks (Fig. 2D) to 'explain' the capacity transition, but Π_red is precisely the objective that was maximized during training. Its high value below capacity is therefore partly by construction, not independent evidence for redundancy as a principle. The independent support is the capacity comparison in Fig. 2C and Fig. 5, which is strong. Please reframe the PID analysis as a descriptive check of the optimized solution, or compare it against networks trained with other objectives, rather than presenting it as a confirmation of the principle.
minor comments (5)
  1. [§3.4, Eq. (7)] The text says both optimized goals 'maximize redundancy,' but G(ii) has γ_red = 0.25 and γ_unq,T = 0.48; redundancy is not the dominant term. Please clarify that only G(i) is redundancy-dominated, while G(ii) resembles the mutual-information objective.
  2. [§3.5, A.3.1] Stability is described in Appendix A.3.1, but the main text refers to 'subsection A.2' for details on stability. Please correct the cross-reference.
  3. [Figure 5 caption] The caption says 'The curves in A-D' but the figure contains only panels A, B, and C. Please adjust.
  4. [Code Availability] The paper states that code 'will be made available together with publication.' Given that the diagonal self-connection ambiguity and the capacity exclusion rule cannot be checked without code, I recommend providing the code with the revision or at least an explicit pseudocode statement of whether the diagonal is masked.
  5. [Appendix E] The formula for the expected number of zero-entropy neurons is stated without derivation. A short derivation would help the reader understand when finite-size effects are expected to be negligible, especially for the N = 100 capacity measurements.

Circularity Check

1 steps flagged · score 2.0 of 10

No significant circularity: capacity is an external benchmark; only Figure 2D's redundancy profile is a constructed-observation tautology.

  1. fitted input called prediction [Section 3.2, Figure 2D; training in Appendix A.2]
    "we first use the simple neuron goal function G= Π red, i.e., we train each neuron to maximize the redundant information between its inputs about the output. ... As shown in Figure 2D, redundancy is uniformly high below αred c apart from some finite size effects for low numbers of patterns discussed in Appendix E."

    The weights are trained by gradient ascent on G=Πred (A.2, Eq. 5), so measuring high Πred in the resulting networks is a restatement of the training objective, not an emergent confirmation that redundancy underlies memory. This is a fitted objective reported as an observed information profile. However, this observation is auxiliary: the capacity claim αred c =1.59 is obtained from target-free sign-dynamics retrieval and is not implied by the objective, so the central derivation is not circular.

full rationale

The central claim is not forced by definition. Training maximizes Πred, but capacity is measured under test-time sign dynamics without target input and compared against MPF and Descent L2; those numbers could in principle fail, so they are independent evidence for the learning rule. The paper also provides a non-circular check: networks trained on I(Y:T) (not on Πred) show a PID profile dominated by redundancy below capacity (Figure 3C), supporting the redundancy interpretation without optimizing the same atom. Self-citations (I_sx∩ from Makkeh et al. [21]; infomorphic framework from Makkeh et al. [13]; CMA-ES analogy to Schneider et al. [14]) are methodological and not used as a uniqueness theorem or to forbid alternatives, so they are not load-bearing. The only constructed element is Figure 2D, where high Πred in G=Πred-trained networks restates the training target; I score this as a minor, non-central circular observation. I do not score the possible absence of a zero-diagonal constraint (Eq. 8 has no j≠i restriction) as circularity: it is a specification/correctness confound that should be checked in code, not an identity between input and output.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The reported capacities depend on several hand-set training and PID estimation choices. The optimized goal weights are explicitly fitted to capacity via CMA-ES. No constants are derived from theory; the central number α_red=1.59 comes from simulations with these settings. No new physical entities are introduced.

free parameters (7)
  • target weight w_T = 2.3
    Set by hand; chosen as 'resulted in fastest learning and best performance' (A.2, Table 1). The training objective and resulting weights depend on this value.
  • soft-binning bin counts (n_r, n_t) = 60 recurrent bins, 2 target bins
    Chosen for PID estimation (Table 1, Appendix D); capacity results depend on the discretization of continuous recurrent inputs.
  • kernel width fractions (σ_r, σ_t, l) = σ_r=0.5, σ_t=1e-6, l proportional to bin size
    Soft-binning smoothing parameters (Appendix D) control the gradient estimate of the PID objective.
  • training hyperparameters (η, epochs, λ_r, optimizer) = η=0.05, epochs=5000, λ_r=1e-3, Adam
    Hand-set training choices (Table 1) that affect whether gradient ascent reaches high-capacity solutions.
  • capacity and stability thresholds (θ, ϵ) = θ=0.95, ϵ=0.95
    Accuracy thresholds define the reported capacity α_c and stability f_max (A.3, A.3.1); different thresholds would change the headline numbers.
  • optimized goal weights γ for G(i) = γ_unq,T=-0.27, γ_unq,R=-0.68, γ_red=0.68, γ_syn=-0.77, γ_res=-0.80
    Found by CMA-ES maximizing capacity (Eq. 6, Table 2); fitted to the evaluation metric, not predicted.
  • optimized goal weights γ for G(ii) = γ_unq,T=0.48, γ_unq,R=-0.16, γ_red=0.25, γ_syn=0.04, γ_res=-0.63
    Second best optimized goal from CMA-ES (Eq. 7, Table 2); also fitted to capacity.
assumptions (4)
  • domain assumption The I_sx∩ redundancy measure (Makkeh et al. 2021) resolves the PID underdetermination, and the atoms computed with it are the right decomposition for the principle.
    Eq. 3 underdetermined; paper chooses I_sx∩ for differentiability (A.2). The entire redundancy-based claim depends on this measure.
  • domain assumption The soft binning estimates p(R,T) with negligible bias and provides gradients that guide gradient ascent to good optima.
    Appendix D: a kernel-based histogram is substituted for the true joint distribution; no convergence or unbiasedness guarantee is given.
  • ad hoc to paper One recurrent step from the exact stored pattern, with the target input present during training, is representative of the test condition: iterated dynamics from stored/noisy patterns without target input.
    Training (A.2) uses one synchronous step with T; testing (A.3) iterates up to 100 steps without T. The transfer is assumed and only empirically validated.
  • domain assumption Patterns are i.i.d. unbiased ±1 for the main capacity results.
    Standard Hopfield setup; capacity numbers do not apply to correlated patterns except the preliminary Appendix F.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Redundancy Maximization as a Principle of Associative Memory Learning in Hopfield Networks." pith.science (2026). https://pith.science/paper/JA6QPW55

@misc{pith2026251102584,
  author       = {Pith},
  title        = {Pith review of: Redundancy Maximization as a Principle of Associative Memory Learning in Hopfield Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JA6QPW55}},
  note         = {Machine review of arXiv:2511.02584}
}
read the original abstract

Associative memory, traditionally modeled by Hopfield networks, enables the retrieval of previously stored patterns from partial or noisy cues. Yet, the local computational principles which are required to enable this function remain incompletely understood. To formally characterize the local information processing in such systems, we employ a recent extension of information theory -- Partial Information Decomposition (PID). PID decomposes the contribution of different inputs to an output into unique information from each input, redundant information across inputs, and synergistic information that emerges from combining different inputs. Applying this framework to individual neurons in classical Hopfield networks we find that below the memory capacity, the information in a neuron's activity is characterized by high redundancy between the external pattern input and the internal recurrent input, while synergy and unique information are close to zero until the memory capacity is surpassed and performance drops steeply. Inspired by this observation, we use redundancy maximization at each neuron as an information-theoretic learning goal. This dramatically increases the network's memory capacity to 1.59, a more than tenfold improvement over the 0.14 capacity of classical Hopfield networks, and also outperforming recent state-of-the-art implementations of Hopfield networks. Overall, this work establishes redundancy maximization as a new design principle for associative memories and opens pathways for new associative memory models based on information-theoretic goals.

Figures

Figures reproduced from arXiv: 2511.02584 by the authors.

Figure 1
Figure 1. For classical Hopfield networks trained with Hebbian learning, redundant information between target and recurrent input coincides with successful memory storage. A: Schematic of the analysis set-up for Hopfield networks. To measure how information is represented, each neuron is compared to a non-driving target input T that provides the ground-truth pattern, in addition to its recurrent input R. B: Each neuron in the… view at source ↗
Figure 2
Figure 2. Redundancy maximization between recurrent connections and a target is a sufficient principle for memorization in Hopfield networks, achieving a memory capacity of α red c ≈ 1.59. A: Schematic of the infomorphic Hopfield model. During training of the infomorphic Hopfield network, the recurrent connections wR are updated using gradient ascent on the goal function G = Πred. B: Each neuron in the infomorphic Hopfield ne… view at source ↗
Figure 3
Figure 3. Infomorphic Hopfield networks trained with a classical mutual information goal achieve high memory capacity by implicitly maximizing redundancy. A: The redundancy goal and two alternative learning objectives based on classical information theory: Maximizing the mutual information between output and target, G = I(Y : T), and maximizing the co-information, G = I(Y : R : T). B: A performance comparison of the two alter… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Hyperparameter optimization reveals composite information goals which outperform redundancy maximization. A: The memory capacity landscape αc as a function of the goal parameters γi reveals the performance across large parts of the parameter space. The goal parameters …
Figure 5
Figure 5. Figure 5: In terms of memory capacity, the infomorphic approach outperforms other high-performance learning methods. A: Comparison of the performance of the Hebbian learning rule, our heuristic and optimized infomorphic goals, and two high-performance methods, the descent L2 met…
Figure 6
Figure 6. Figure 6: In the soft binning procedure, the histograms are smoothed at the bin edges resulting in non-zero gradients of the histogram. A: The contribution of a single sample to a bin covering the range [−3, 3]. The contribution scales with the distance to the center of the bin.…
Figure 7
Figure 7. Figure 7: Finite size effects for very low memory loads can be counteracted by increasing the network size. A: The mean redundancy of neurons for different network sizes shows that finite size effects can be reduced for small memory loads α by increasing the number of neurons. F…
Figure 8
Figure 8. Figure 8: Infomorphic Hopfield networks increase their capacity when trained on spatially correlated patterns. A: When spatial correlations p within patterns are introduced, capacity improves for networks trained on the redundancy goal. B: Introducing the same correlations as in…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 1 canonical work pages

  1. [13]

    Schneider, David A

    Abdullah Makkeh, Marcel Graetz, Andreas C. Schneider, David A. Ehrlich, Viola Priesemann, and Michael Wibral. A general framework for interpretable neural learning based on local information-theoretic goal functions. Proceedings of the National Academy of Sciences, 122(10):e2408125122, 2025. doi:10.1073/pnas.2408125122. URLhttps://www.pnas.org/doi/abs/10....

    doi:10.1073/pnas.2408125122 2025

  2. [1]

    Neural networks and physical systems with emergent collective computational abilities.Proceed- ings of the national academy of sciences, 79(8):2554–2558, 1982

    John J Hopfield. Neural networks and physical systems with emergent collective computational abilities.Proceed- ings of the national academy of sciences, 79(8):2554–2558, 1982

    1982

  3. [2]

    The nobel prize in physics 2024 – press release

    Nobel Prize Outreach. The nobel prize in physics 2024 – press release. https://www.nobelprize.org/ prizes/physics/2024/press-release/, 2024. Accessed: 2025-10-10

    2024

  4. [3]

    Hopfield networks is all you need

    Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, Philipp Seidl, Michael Widrich, Lukas Gruber, Markus Holzleitner, Thomas Adler, David Kreil, Michael K Kopp, Günter Klambauer, Johannes Brandstetter, and Sepp Hochreiter. Hopfield networks is all you need. InInternational Conference on Learning Representations, 2021. URLhttps://openreview.net/forum?id=tL89RnzIiCd

    2021

  5. [4]

    Efficient and optimal binary hopfield associative memory storage using minimum probability flow.arXiv preprint arXiv:1204.2916, 2012

    Christopher Hillar, Jascha Sohl-Dickstein, and Kilian Koepsell. Efficient and optimal binary hopfield associative memory storage using minimum probability flow.arXiv preprint arXiv:1204.2916, 2012

    arXiv 2012

  6. [5]

    Pavel Tolmachev and Jonathan H. Manton. New insights on learning rules for hopfield networks: Memory and objective function minimisation. In2020 International Joint Conference on Neural Networks (IJCNN), page 1–8. IEEE, July 2020. doi:10.1109/ijcnn48605.2020.9207405. URL http://dx.doi.org/10.1109/IJCNN48605. 2020.9207405

    arXiv 2020

  7. [6]

    C. E. Shannon. A mathematical theory of communication.The Bell System Technical Journal, 27(3):379–423,

  8. [7]

    Cover and Joy A

    Thomas M. Cover and Joy A. Thomas.Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing). Wiley-Interscience, USA, 2006. ISBN 0471241954

    2006

Show all 37 references
  1. [8]

    Nonnegative decomposition of multivariate information.arXiv preprint arXiv:1004.2515, 2010

    Paul L Williams and Randall D Beer. Nonnegative decomposition of multivariate information.arXiv preprint arXiv:1004.2515, 2010

    2010 arXiv

  2. [9]

    Opening the black box of deep neural networks via information.arXiv preprint arXiv:1703.00810, 2017

    Ravid Shwartz-Ziv and Naftali Tishby. Opening the black box of deep neural networks via information.arXiv preprint arXiv:1703.00810, 2017

    2017 arXiv

  3. [10]

    On information plane analyses of neural network classifiers—a review.IEEE Transactions on Neural Networks and Learning Systems, 33(12):7039–7051, 2021

    Bernhard C Geiger. On information plane analyses of neural network classifiers—a review.IEEE Transactions on Neural Networks and Learning Systems, 33(12):7039–7051, 2021. 12 Redundancy Maximization as a Principle of Associative Memory Learning

    2021

  4. [11]

    A measure of the complexity of neural representations based on partial information decomposition

    David Alexander Ehrlich, Andreas Christian Schneider, Viola Priesemann, Michael Wibral, and Abdullah Makkeh. A measure of the complexity of neural representations based on partial information decomposition. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL h...

    2023

  5. [12]

    Gutknecht, Fernando E

    Aaron J. Gutknecht, Fernando E. Rosas, David A. Ehrlich, Abdullah Makkeh, Pedro A. M. Mediano, and Michael Wibral. Shannon invariants: A scalable approach to information decomposition, 2025. URL https: //arxiv.org/abs/2504.15779

    2025 arXiv

  6. [14]

    Schneider, Valentin Neuhaus, David A

    Andreas C. Schneider, Valentin Neuhaus, David A. Ehrlich, Abdullah Makkeh, Alexander S. Ecker, Viola Priesemann, and Michael Wibral. What should a neuron aim for? designing local objective functions based on information theory. InThe Thirteenth International Conference on Lear...

    2025

  7. [15]

    Neurons with graded response have collective computational properties like those of two-state neurons.Proceedings of the national academy of sciences, 81(10):3088–3092, 1984

    John J Hopfield. Neurons with graded response have collective computational properties like those of two-state neurons.Proceedings of the national academy of sciences, 81(10):3088–3092, 1984

    1984

  8. [16]

    Storing infinite numbers of patterns in a spin-glass model of neural networks.Physical Review Letters, 55(14):1530, 1985

    Daniel J Amit and Hanoch Gutfreund. Storing infinite numbers of patterns in a spin-glass model of neural networks.Physical Review Letters, 55(14):1530, 1985

    1985

  9. [17]

    Thomas M. Cover. Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition.IEEE Trans. Electron. Comput., 14:326–334, 1965. URL https://api.semanticscholar. org/CorpusID:18251470

    1965

  10. [18]

    The space of interactions in neural network models.Journal of physics A: Mathematical and general, 21(1):257, 1988

    Elizabeth Gardner. The space of interactions in neural network models.Journal of physics A: Mathematical and general, 21(1):257, 1988

    1988

  11. [19]

    Bits and pieces: Understanding information decomposition from part-whole relationships and formal logic.Proceedings of the Royal Society A, 477(2251): 20210110, 2021

    Aaron J Gutknecht, Michael Wibral, and Abdullah Makkeh. Bits and pieces: Understanding information decomposition from part-whole relationships and formal logic.Proceedings of the Royal Society A, 477(2251): 20210110, 2021

    2021

  12. [20]

    Information decomposition of target effects from multi-source interactions: Perspectives on previous, current and future work.Entropy, 20(4):307, 2018

    Joseph T Lizier, Nils Bertschinger, Jürgen Jost, and Michael Wibral. Information decomposition of target effects from multi-source interactions: Perspectives on previous, current and future work.Entropy, 20(4):307, 2018

    2018

  13. [21]

    Introducing a differentiable measure of pointwise shared information.Physical Review E, 103(3):032149, 2021

    Abdullah Makkeh, Aaron J Gutknecht, and Michael Wibral. Introducing a differentiable measure of pointwise shared information.Physical Review E, 103(3):032149, 2021

    2021

  14. [22]

    Coherent infomax as a computational goal for neural systems.Bulletin of mathematical biology, 73(2):344–372, 2011

    Jim W Kay and WA Phillips. Coherent infomax as a computational goal for neural systems.Bulletin of mathematical biology, 73(2):344–372, 2011

    2011

  15. [23]

    Completely derandomized self-adaptation in evolution strategies

    Nikolaus Hansen and Andreas Ostermeier. Completely derandomized self-adaptation in evolution strategies. Evolutionary computation, 9(2):159–195, 2001

    2001

  16. [24]

    Learning of correlated patterns in spin-glass networks by local learning rules.Phys

    Sigurd Diederich and Manfred Opper. Learning of correlated patterns in spin-glass networks by local learning rules.Phys. Rev. Lett., 58:949–952, Mar 1987. doi:10.1103/PhysRevLett.58.949. URL https://link.aps. org/doi/10.1103/PhysRevLett.58.949

    1987 doi

  17. [25]

    Amit, Hanoch Gutfreund, and H

    Daniel J. Amit, Hanoch Gutfreund, and H. Sompolinsky. Spin-glass models of neural networks.Phys. Rev. A, 32:1007–1018, Aug 1985. doi:10.1103/PhysRevA.32.1007. URL https://link.aps.org/doi/10.1103/ PhysRevA.32.1007. 13 Redundancy Maximization as a Principle of Associative Memor...

    1985 doi

  18. [26]

    New method for parameter estimation in probabilistic models: minimum probability flow.Physical review letters, 107(22):220601, 2011

    Jascha Sohl-Dickstein, Peter B Battaglino, and Michael R DeWeese. New method for parameter estimation in probabilistic models: minimum probability flow.Physical review letters, 107(22):220601, 2011

    2011

  19. [27]

    Self-control in sparsely coded networks.Physical review letters, 80(13):2961, 1998

    DRC Dominguez and Désiré Bollé. Self-control in sparsely coded networks.Physical review letters, 80(13):2961, 1998

    1998

  20. [28]

    Mutual information and topology 1: Asymmetric neural network

    David Dominguez, Kostadin Koroutchev, Eduardo Serrano, and Francisco B Rodríguez. Mutual information and topology 1: Asymmetric neural network. InAdvances in Neural Networks–ISNN 2004: International Symposium on Neural Networks, Dalian, China, August 2004, Proceedings, Part I ...

    2004

  21. [29]

    Structured information in small-world neural networks.Physical Review E—Statistical, Nonlinear , and Soft Matter Physics, 79(2):021909, 2009

    David Dominguez, Mario González, Eduardo Serrano, and Francisco B Rodríguez. Structured information in small-world neural networks.Physical Review E—Statistical, Nonlinear , and Soft Matter Physics, 79(2):021909, 2009

    2009

  22. [30]

    An application of the principle of maximum information preservation to linear systems.Advances in neural information processing systems, 1, 1988

    Ralph Linsker. An application of the principle of maximum information preservation to linear systems.Advances in neural information processing systems, 1, 1988

    1988

  23. [31]

    Hopfield

    Dmitry Krotov and John J. Hopfield. Dense associative memory for pattern recognition. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors,Advances in Neural Information Processing Systems, vol- ume 29. Curran Associates, Inc., 2016. URL https://proceedings.ne...

    2016

  24. [32]

    A new frontier for Hopfield Networks.Nature Reviews Physics, pages 1–2, 2023

    Dmitry Krotov. A new frontier for Hopfield Networks.Nature Reviews Physics, pages 1–2, 2023

    2023

  25. [33]

    On the storage capacity of hopfield models with correlated patterns.Annals of Applied Probability, 8, 11 1998

    Matthias Löwe. On the storage capacity of hopfield models with correlated patterns.Annals of Applied Probability, 8, 11 1998. doi:10.1214/aoap/1028903378

    1998

  26. [34]

    Formal theory of creativity, fun, and intrinsic motivation (1990–2010).IEEE Transactions on Autonomous Mental Development, 2(3):230–247, 2010

    Jürgen Schmidhuber. Formal theory of creativity, fun, and intrinsic motivation (1990–2010).IEEE Transactions on Autonomous Mental Development, 2(3):230–247, 2010. doi:10.1109/TAMD.2010.2056368

    1990

  27. [35]

    Partial information decomposition for continuous variables based on shared exclusions: Analytical formulation and estimation.Physical Review E, 110(1):014115, 2024

    David A Ehrlich, Kyle Schick-Poland, Abdullah Makkeh, Felix Lanfermann, Patricia Wollstadt, and Michael Wibral. Partial information decomposition for continuous variables based on shared exclusions: Analytical formulation and estimation.Physical Review E, 110(1):014115, 2024

    2024

  28. [36]

    Information-theoretic neural networks for unsupervised learning: mathematical and statistical considera- tions

    Jim Kay. Information-theoretic neural networks for unsupervised learning: mathematical and statistical considera- tions. Technical Report 1573387449478062080, Scottish Agricultural Statistics Service, 1994. 14 Redundancy Maximization as a Principle of Associative Memory Learni...

    1994

  29. [1948]

    doi:10.1002/j.1538-7305.1948.tb01338.x

    1948

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.