REVIEW 7 cited by
Large Associative Memory Problem in Neurobiology and Machine Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Dense Associative Memories or modern Hopfield networks permit storage and reliable retrieval of an exponentially large (in the dimension of feature space) number of memories. At the same time, their naive implementation is non-biological, since it seemingly requires the existence of many-body synaptic junctions between the neurons. We show that these models are effective descriptions of a more microscopic (written in terms of biological degrees of freedom) theory that has additional (hidden) neurons and only requires two-body interactions between them. For this reason our proposed microscopic theory is a valid model of large associative memory with a degree of biological plausibility. The dynamics of our network and its reduced dimensional equivalent both minimize energy (Lyapunov) functions. When certain dynamical variables (hidden neurons) are integrated out from our microscopic theory, one can recover many of the models that were previously discussed in the literature, e.g. the model presented in "Hopfield Networks is All You Need" paper. We also provide an alternative derivation of the energy function and the update rule proposed in the aforementioned paper and clarify the relationships between various models of this class.
Forward citations
Cited by 7 Pith papers
-
Verbalizable Representations Form a Global Workspace in Language Models
Language models represent their current reasoning in a small, readable set of verbalizable vectors (the J-space) that functions like a global workspace.
-
Exponential Capacity in Multilayer Hetero-Associative Neural Networks
A multilayer exponential Hopfield network stores e^{Nρ_L} hetero-associative patterns, with ρ_L∼L log2 and basins that match simulated, immune-receptor, and language data.
-
Collective gene dynamics leave signatures of decision landscapes in cell fate coordinates
A Hopfield-inspired model maps gene expression to cell fate coordinates and identifies straight, curved, or clustered trajectories as signatures of three decision landscape classes.
-
Distributed Dynamic Associative Memory via Online Convex Optimization
Tree-based distributed online gradient descent gives sublinear static regret and path-length-dependent dynamic regret for heterogeneous multi-agent associative memory with communication delays.
-
Yet another exponential Hopfield model
An exponential Hopfield model with binary neurons, defined by exponentials of quadratic losses, stores exponentially many patterns and retains exponential capacity under noise, with a quantified basin threshold.
-
Novel Complex-Valued Hopfield Neural Networks with Phase and Magnitude Quantization
Two new activation functions quantize magnitude and phase in complex-valued Hopfield networks, enlarging the state space and showing empirical convergence in small trials.
-
Hopfield Networks as Models of Emergent Function in Biology
A review article that explains Hopfield network mathematics and interprets the dynamics as signal retrieval, subspace projection, and energy landscape descent, then surveys applications to cell fate, self-assembly, an...
Discussion (0). Continue with ORCID to comment.