Pith. sign in

REVIEW 3 major objections 4 minor 21 references

Optimal packing of attractor states in neural representations

T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read For uniform and cyclic Markov chains, the optimal packing of neural attractor states is solved under a symmetry ansatz, yielding a simplex for uniform statistics and a generally non-square quadrilateral for four-state cycles.

desk verdict A fresh normative framework for neural geometry with a genuinely non-square M=4 result, but the symmetry-inheritance claim is over-sold and an appendix arithmetic slip needs fixing. read the letter →

arxiv 2504.12429 v1 pith:FQIL7U3Q submitted 2025-04-16 q-bio.NC

John J. Vastola This is my paper · ORCID
classification q-bio.NC
keywords statesoptimaltransitioninternalneuralpackingproblemstatistics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Animals need internal brain states that track the outside world, like where the head points or where the body is. Brain states that are often used together should be close together, so the brain can switch quickly, but states that are too close will be confused by neural noise. This paper turns that trade-off into a math problem: place one attractor state per world state so that the brain's own transition statistics match the world's transition statistics, while paying a small penalty for high activity.

The authors write a cost function that combines three pieces: how well the brain's transitions match the environment's, how accurately the brain can decode which state it is in, and how much activity the states require. They show the cost only depends on the pairwise distances between states, not on where the whole cluster sits. Then they use symmetry: if the environment's transition rules look the same under some rearrangements, they look for arrangements of states that also look the same under those rearrangements. For a fully uniform environment, symmetry forces all states to be equally far apart, forming a simplex, like the corners of a triangle. For a four-state ring, symmetry leaves two distances free: the ring edges and the diagonals. The optimum generally does not form a perfect square, which is a concrete, unexpected prediction.

The results are exact only within the symmetric ansatz, and the paper does not prove this ansatz is the true global optimum. One formula in the uniform case also has a small counting error. Still, the framework gives a clean way to think about why ring attractors and grid-like codes have the geometry they do, and it offers toy predictions that could be tested in simulations or data.

Extended reading notes

Core claim

For Markov chains with uniform transition statistics, the optimal arrangement of M attractor states is an (M-1)-simplex with all pairwise distances equal; for M=4 cyclic statistics, the optimal arrangement is a D4-symmetric quadrilateral whose edge length d and diagonal L generally do not satisfy L/d = sqrt(2) (Sections 4.1, 4.2). If the paper is correct, symmetries of environmental transition statistics are reflected in the geometry of optimal neural representations.

Load-bearing premise

The global minimum of the objective J (Eq. 8) is assumed to inherit the symmetries of the Markov chain, via an unproved application of the Purkiss principle. The paper states the problem is 'probably not convex', that it must 'settle for something weaker', and that a rigorous analysis is not pursued; if the true optimum is not symmetric, the reported packings are only symmetry-constrained stationary points, not the optimal packing claimed in the title and abstract. Location: Section 3, paragraph beginning 'For convex optimization problems' and the sentence 'A more rigorous analysis of Eq. 11 may be able to show that Waterhouse's precise formulation of the Purkiss principle applies, although we do not pursue such an analysis here.'

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The manuscript formulates a normative toy model for arranging M internal attractor states in a neural representation. Environment statistics are modeled by a Markov chain, internal transitions by distance-dependent rates, and encoding/decoding by a noisy symmetric channel; the objective J (Eq. 8) trades off the KL divergence between the desired and actual internal transition distributions against a firing-rate regularization. After a change of variables, J depends only on the mean location and pairwise distances (Eqs. 10–11), and the mean is set to zero. Relying on a Purkiss-style symmetry argument, the paper restricts to symmetric configurations and claims to solve two cases: uniform statistics, for which all distances are equal (an (M−1)-simplex), and M=4 cyclic statistics, for which the optimum is a D4-symmetric quadrilateral with two distances d and L. A large-b approximation for general cyclic topologies is also sketched. Code is provided.

Significance. The paper's modeling pipeline is clear and the reduction to pairwise distances is a useful simplification; the uniform-prior decoding step and the symmetry of the objective under Markov-chain permutations are correctly derived. The two-parameter model yields concrete, falsifiable geometric predictions, and the availability of code supports reproducibility. However, the central optimality claim is not established: the symmetry restriction is an assumption about the global minimizer, not a proof, and there are concrete arithmetic and geometric errors in key results. In its current form the contribution is best read as a symmetry-constrained analysis of a plausible objective rather than as a proof of optimal packing.

major comments (3)
  1. [Section 3, Sections 4.1–4.2] The paper's central claim that it has found the optimal packing is not supported. The text explicitly says Eq. (8) is 'probably not convex', that the authors 'settle for something weaker' in the spirit of the Purkiss principle, and that a rigorous analysis of Eq. (11) 'is not pursued'. Sections 4.1 and 4.2 then optimize only the symmetry-constrained ansatz (all distances equal, or two distances d and L under D4). Because a symmetric stationary point of a nonconvex function need not be a global minimum, and because the objective is defined over all configurations in R^D, the results as stated ('optimal packing', 'optimal solution') go beyond what is proven. This is load-bearing for the abstract and title. The authors should either prove global optimality for the two solved cases (e.g., by a convexity or monotonicity argument on the reduced objectives, combined with explicit feasible regions) or explicitly reframe the contribution as the optimal symmetric packing.
  2. [Appendix B, Section 4.1] The coefficient of e^{-d^2} in the uniform-case p_int and in the displayed J is wrong. For x≠y, the number of ordered pairs (a,b) with a≠x, b≠y, and a≠b is (M−1)^2 − (M−2) = M^2 − 3M + 3, not M(M−1) − 2(M−2) + 1 = M^2 − 3M + 5. Direct counting for M=2 gives one cross term, whereas the printed formula gives three. Consequently the objective plotted in Fig. 2 and the associated phase-transition statements must be recomputed; this is an arithmetic error in a central equation, not a typo.
  3. [Section 4.2] The claimed large-b degenerate solution 'd = 0 and L ≠ 0' is not realizable as a Euclidean configuration. If the four side distances d12=d23=d34=d41 are zero, then all four vertices coincide and the diagonals L=d13=d24 must also be zero. The feasible line-degenerate configuration for the four-cycle is the opposite limiting case, L=0 with d>0 (two pairs of states coalesce). The text and Fig. 3 panels that describe this phase therefore need correction, and the optimization should explicitly enforce the Euclidean embedding constraints on (d,L), at minimum the triangle inequalities for the face triangles.
minor comments (4)
  1. [Section 4.3] The section is titled 'Optimal packing for cyclic topology' but, for M>4, it only derives an approximate large-b system of equations and states that it cannot be solved analytically. The section should be retitled or clearly scoped as an approximate analysis, since the abstract's 'cyclic transition statistics' case is only exactly solved for M=4.
  2. [Section 2] The statement that setting p(x|x)=0 is 'without loss of generality' needs a sentence of justification, since deleting self-loops generally changes transition probabilities on a fixed time scale.
  3. [Appendix D, Eqs. (33)–(35)] The exponents in the sums after the derivative appear to mix d_k^2 and d_k^2/2; please recheck these equations against the bracket in Eq. (32) and the derivative of Z.
  4. [Figure 3b] The caption says the objective is plotted versus d and L, but the main text describes a two-variable landscape; please specify whether the figure shows a contour or a surface and state the fixed parameter values.
Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central results rest on two hand-specified model parameters (b, alpha) and a set of explicit modeling assumptions about transition dynamics, encoding, and decoding. No external data or benchmarks are used, so the quantitative predictions are conditional on these choices. The symmetry-to-optimality step is an unproven assumption, transparently flagged by the authors.

free parameters (2)
  • b (encoding bias)
    Controls noise in the encoding/decoding model (Eq. 3). The paper sweeps b and reports optimal distance versus b, but b is not fitted to any neural data.
  • alpha (regularization strength)
    Sets the firing-rate penalty in J (Eq. 7). No data are used to fix alpha; quantitative results depend on it.
assumptions (4)
  • domain assumption Internal transition probabilities depend only on pairwise distance via q(z_y|z_x) = (1-delta_xy) exp(-D^2/2)/Z(z_x) (Eq. 2)
    Assumes equal attractor widths and depths and a Kramers-like escape rate; if real attractors have heterogeneous depths, the objective changes.
  • domain assumption Encoding noise is isotropic in the noise-whitened metric and controlled by a single bias b (Eq. 3)
    A specific, simplified encoding model; Poisson spiking or probabilistic population codes would alter p_int.
  • domain assumption Decoding uses a uniform prior over states (Eq. 4)
    The Bayesian decoder assumes uniform prior occupancy; this is a modeling choice.
  • domain assumption The global optimum of J inherits the Markov chain's symmetries (Purkiss principle, Waterhouse 1983)
    Invoked without proof despite J being possibly non-convex; Section 3 acknowledges this and does not pursue a rigorous analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Optimal packing of attractor states in neural representations." pith.science (2026). https://pith.science/paper/FQIL7U3Q

@misc{pith2026250412429,
  author       = {Pith},
  title        = {Pith review of: Optimal packing of attractor states in neural representations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FQIL7U3Q}},
  note         = {Machine review of arXiv:2504.12429}
}
read the original abstract

Animals' internal states reflect variables like their position in space, orientation, decisions, and motor actions -- but how should these internal states be arranged? Internal states which frequently transition between one another should be close enough that transitions can happen quickly, but not so close that neural noise significantly impacts the stability of those states, and how reliably they can be encoded and decoded. In this paper, we study the problem of striking a balance between these two concerns, which we call an `optimal packing' problem since it resembles mathematical problems like sphere packing. While this problem is generally extremely difficult, we show that symmetries in environmental transition statistics imply certain symmetries of the optimal neural representations, which allows us in some cases to exactly solve for the optimal state arrangement. We focus on two toy cases: uniform transition statistics, and cyclic transition statistics. Code is available at https://github.com/john-vastola/optimal-packing-neurreps23 .

Figures

Figures reproduced from arXiv: 2504.12429 by the authors.

Figure 1
Figure 1. Schematic of optimal packing problem. a. We want environment transition statis￾tics p(y|x) to typically match the combination of encoding, internal dynamics, and decoding, or equivalently for this diagram to commute. b. Intuitively, the ge￾ometric structure of the attractor landscape should match the structure of the Markov chain; for example, states with frequent transitions ought to be closer together than states … view at source ↗
Figure 2
Figure 2. Solution of packing problem for a Markov chain with a uniform topology (see top right graphs). a. The optimal solution has the distance between all states equal, which means the geometry is that of a (M − 1)-simplex. b. The objective function versus the distance d. Note the bifurcation as the bias decreases. c. The optimal distance as a function of the bias. It is zero for very small or large biases. 4.1. Optimal pa… view at source ↗
Figure 3
Figure 3. Solution of packing problem for a Markov chain with a square topology. a. The optimal solution has two undetermined distances d and L; depending on their ratio, non-square solutions are possible. b. The objective function versus d and L for different bias values. Note phase-transition-like behavior for small and large biases. c. The optimal d and L values generally do not have L/d = √ 2 when a nontrivial solution ex… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 9 canonical work pages

  1. [1]

    Keinath, Xue-Xin Wei, and Mark P

    Zaki Ajabi, Alexandra T. Keinath, Xue-Xin Wei, and Mark P. Brandon. Population dynamics of head-direction neurons during drift and reorientation. Nature, 615 0 (7954): 0 892--899, Mar 2023. ISSN 1476-4687. doi:10.1038/s41586-023-05813-2. URL https://doi.org/10.1038/s41586-023-05813-2

    work page doi:10.1038/s41586-023-05813-2 2023

  2. [2]

    Barry and N

    C. Barry and N. Burgess. Neural mechanisms of self-location. Current Biology, 24 0 (8): 0 R330--R339, 2014. ISSN 0960-9822. doi:https://doi.org/10.1016/j.cub.2014.02.049. URL https://www.sciencedirect.com/science/article/pii/S0960982214002176

    work page doi:10.1016/j.cub.2014.02.049 2014

  3. [3]

    Yoram Burak and Ila R. Fiete. Fundamental limits on persistent activity in networks of noisy neurons. Proceedings of the National Academy of Sciences, 109 0 (43): 0 17645--17650, 2012. doi:10.1073/pnas.1117386109. URL https://www.pnas.org/doi/abs/10.1073/pnas.1117386109

    work page doi:10.1073/pnas.1117386109 2012

  4. [4]

    Zheng, and Kevin Chen-Chuan Chang

    HongYun Cai, Vincent W. Zheng, and Kevin Chen-Chuan Chang. A comprehensive survey of graph embedding: Problems, techniques, and applications. IEEE Transactions on Knowledge and Data Engineering, 30 0 (9): 0 1616--1637, 2018. doi:10.1109/TKDE.2018.2807452

    arXiv 2018

  5. [5]

    Neural representations of motor plans, desired trajectories, and controlled objects

    Paul Cisek. Neural representations of motor plans, desired trajectories, and controlled objects. Cognitive Processing, 6 0 (1): 0 15--24, Mar 2005. ISSN 1612-4790. doi:10.1007/s10339-004-0046-7. URL https://doi.org/10.1007/s10339-004-0046-7

    work page doi:10.1007/s10339-004-0046-7 2005

  6. [6]

    The sphere packing problem in dimension 24

    Henry Cohn, Abhinav Kumar, Stephen Miller, Danylo Radchenko, and Maryna Viazovska. The sphere packing problem in dimension 24 . Annals of Mathematics, 185 0 (3): 0 1017 -- 1033, 2017. doi:10.4007/annals.2017.185.3.8. URL https://doi.org/10.4007/annals.2017.185.3.8

    doi:10.4007/annals.2017.185.3.8 2017

  7. [7]

    Driscoll, Lea Duncker, and Christopher D

    Laura N. Driscoll, Lea Duncker, and Christopher D. Harvey. Representational drift: Emerging theories for continual learning and experimental future directions. Current Opinion in Neurobiology, 76: 0 102609, 2022. ISSN 0959-4388. doi:https://doi.org/10.1016/j.conb.2022.102609. URL https://www.sciencedirect.com/science/article/pii/S0959438822001039

    arXiv 2022

  8. [8]

    Aldo Faisal, Luc P

    A. Aldo Faisal, Luc P. J. Selen, and Daniel M. Wolpert. Noise in the nervous system. Nature Reviews Neuroscience, 9 0 (4): 0 292--303, Apr 2008. ISSN 1471-0048. doi:10.1038/nrn2258. URL https://doi.org/10.1038/nrn2258

    doi:10.1038/nrn2258 2008

Show all 21 references
  1. [9]

    Gold and Michael N

    Joshua I. Gold and Michael N. Shadlen. The neural basis of decision making. Annual Review of Neuroscience, 30 0 (1): 0 535--574, 2007. doi:10.1146/annurev.neuro.29.051605.113038. URL https://doi.org/10.1146/annurev.neuro.29.051605.113038. PMID: 17600525

    2007

  2. [10]

    Reaction-rate theory: fifty years after K ramers

    Peter H\"anggi, Peter Talkner, and Michal Borkovec. Reaction-rate theory: fifty years after K ramers. Rev. Mod. Phys., 62: 0 251--341, Apr 1990. doi:10.1103/RevModPhys.62.251. URL https://link.aps.org/doi/10.1103/RevModPhys.62.251

    1990 doi

  3. [11]

    Hulse and Vivek Jayaraman

    Brad K. Hulse and Vivek Jayaraman. Mechanisms underlying the neural computation of head direction. Annual Review of Neuroscience, 43 0 (1): 0 31--54, 2020. doi:10.1146/annurev-neuro-072116-031516. URL https://doi.org/10.1146/annurev-neuro-072116-031516. PMID: 31874068

    2020 doi

  4. [12]

    H.A. Kramers. Brownian motion in a field of force and the diffusion model of chemical reactions. Physica, 7 0 (4): 0 284--304, 1940. ISSN 0031-8914. doi:https://doi.org/10.1016/S0031-8914(40)90098-2. URL https://www.sciencedirect.com/science/article/pii/S0031891440900982

    1940 doi

  5. [13]

    Basnak, Rachel I

    Anna Kutschireiter, Melanie A. Basnak, Rachel I. Wilson, and Jan Drugowitsch. Bayesian inference in ring attractor networks. Proceedings of the National Academy of Sciences, 120 0 (9): 0 e2210622120, 2023. doi:10.1073/pnas.2210622120. URL https://www.pnas.org/doi/abs/10.1073/p...

    2023 doi

  6. [14]

    Beck, Peter E

    Wei Ji Ma, Jeffrey M. Beck, Peter E. Latham, and Alexandre Pouget. Bayesian inference with probabilistic population codes. Nature Neuroscience, 9 0 (11): 0 1432--1438, Nov 2006. ISSN 1546-1726. doi:10.1038/nn1790. URL https://doi.org/10.1038/nn1790

    2006 doi

  7. [15]

    Zavatone-Veth

    Paul Masset, Shanshan Qin, and Jacob A. Zavatone-Veth. Drifting neuronal representations: Bug or feature? Biological Cybernetics, 116 0 (3): 0 253--266, Jun 2022. ISSN 1432-0770. doi:10.1007/s00422-021-00916-3. URL https://doi.org/10.1007/s00422-021-00916-3

    2022 doi

  8. [16]

    Carsen Stringer, Marius Pachitariu, Nicholas Steinmetz, Matteo Carandini, and Kenneth D. Harris. High-dimensional geometry of population responses in visual cortex. Nature, 571 0 (7765): 0 361--365, Jul 2019. ISSN 1476-4687. doi:10.1038/s41586-019-1346-5. URL https://doi.org/1...

    2019 doi

  9. [17]

    van Vreeswijk and H

    C. van Vreeswijk and H. Sompolinsky. Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science, 274 0 (5293): 0 1724--1726, 1996. doi:10.1126/science.274.5293.1724. URL https://www.science.org/doi/abs/10.1126/science.274.5293.1724

    1996

  10. [18]

    Vastola, Zach Cohen, and Jan Drugowitsch

    John J. Vastola, Zach Cohen, and Jan Drugowitsch. Is the information geometry of probabilistic population codes learnable? In Sophia Sanborn, Christian Shewmake, Simone Azeglio, Arianna Di Bernardo, and Nina Miolane, editors, Proceedings of the 1st NeurIPS Workshop on Symmetry...

    2023

  11. [19]

    Viazovska

    Maryna S. Viazovska. The sphere packing problem in dimension 8. Annals of Mathematics, 185 0 (3): 0 991--1015, 2017. ISSN 0003486X. URL http://www.jstor.org/stable/26395747

    2017

  12. [20]

    Waterhouse

    William C. Waterhouse. Do symmetric problems have symmetric solutions? The American Mathematical Monthly, 90 0 (6): 0 378--387, 1983. ISSN 00029890, 19300972. URL http://www.jstor.org/stable/2975573

    1983

  13. [21]

    Sphere packings

    Chuanming Zong. Sphere packings. Springer Science & Business Media, 2008

    2008

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.