Pith. sign in

REVIEW 3 major objections 5 minor 81 references

Minimax entropy: The statistical physics of optimal models

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Feature selection reduces to minimizing the entropy of the maximum entropy model.

desk verdict A clean, honest review that derives the known minimax entropy principle from MDL; worth a serious referee, but the 'optimal' claim only holds under exact feature averages and large T. read the letter →

arxiv 2505.01607 v1 pith:Y3PWT6P2 submitted 2025-05-02 q-bio.QM

David P. Carcamo , Nicholas J. Weaver , Purushottam D. Dixit , Christopher W. Lynn This is my paper · ORCID
classification q-bio.QM
keywords minimaxentropymaximumprincipleminimumdescriptionlengthfeatureselectionstatisticalphysicsIsingmodelGaussiangraphicalneuralactivity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper derives a rule for choosing which measured features to include in a statistical model: among all allowed feature sets, the best one is the set whose maximum entropy model has the lowest entropy. The derivation starts from the minimum description length (MDL) principle, which equates good models with short encodings of data. For maximum entropy models, the description length turns out to equal the model's entropy, so minimizing description length over feature sets is the same as minimizing entropy. The paper shows that this minimax entropy principle also selects the model closest to the true distribution and the one capturing the most information, assuming the measured features are exact. If correct, it gives a principled, parameter-free way to find compressed models of complex systems, from neural populations to textures.

What carries the argument

The central object is the maximum entropy distribution $P_F(x) = \frac{1}{Z} \exp\left(\sum_{\mu} \lambda_\mu f_\mu(x)\right)$, a Boltzmann distribution whose Lagrange multipliers $\lambda_\mu$ are fit so that the model reproduces the measured feature averages $\langle f_\mu(x)\rangle_{exp}$. The load-bearing identity is $L(P_F) = S(P_F)$: for these models, the minimum description length of a dataset consistent with the features equals the model's entropy, so minimizing description length across feature sets is identical to minimizing entropy. In the case of tree-structured correlations, the information $I_G$ decomposes into a sum of pairwise mutual informations, which reduces the optimization to a minimum spanning tree problem and yields an exact, efficient solution.

What would settle it

Take a system with a known ground-truth distribution, draw finite samples, compute empirical feature averages, apply the minimax entropy rule to select features, and compare the resulting model's KL divergence to the truth against a model selected with a finite-sample complexity penalty. If the minimax-selected model is consistently worse on held-out samples, the exact-averages assumption fails in realistic settings.

Watch

Extended reading notes

Core claim

The central claim is that the optimal set of features $F^*$ is the one for which the maximum entropy model $P_{F^*}(x)$ has the minimum entropy $S(P_{F^*})$ among all allowed sets of features. The key identity is that the description length of any maximum entropy model equals its entropy, $L(P_F) = S(P_F)$ (Eqs. 10-11). Since the MDL principle says the best model is the one with minimum description length, and each feature set yields exactly one maximum entropy model, feature selection becomes the minimax problem $F^* = \arg\min_F S(P_F)$, with $P_F$ itself the entropy-maximizing distribution subject to the empirical feature averages. The same argument shows that $F^*$ minimizes the Kullback-Leibler divergence $D_{KL}(P_{true} \| P_F)$ and maximizes the information $I_F = S(P_{ind}) - S(P_F)$ contained in the selected features, provided the measured averages are exact. This unifies the maximum entropy principle (choosing the model for given features) with the minimax entropy principle (choosing the features themselves).

Load-bearing premise

The load-bearing premise is that the experimentally measured feature averages are exact, with no sampling noise; if the data are finite or noisy, empirical fluctuations can be fit by extra features and the equivalence between low model entropy and closeness to the true distribution breaks down.

Editorial extensions

If this is right

  • Feature selection for maximum entropy models reduces to a single objective—minimizing the fitted model's entropy—with no separate regularization term needed.
  • In neural recordings, a small fraction of optimally chosen correlations can capture most of the information, enabling highly compressed descriptions of brain activity.
  • For tree-structured correlation networks, the optimal model is found exactly via minimum spanning trees, giving an efficient solution for high-dimensional Ising models.
  • The same principle applies across multiple distributions at once, allowing a shared set of features to compress data sets such as handwritten digits.
  • When entropy reduction is submodular, the greedy algorithm is guaranteed to be within $1 - 1/e$ of the global optimum, providing a worst-case performance bound.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • With finite data, the minimax entropy optimum will overfit, so a practical extension is to add a finite-sample correction or complexity penalty to the description length; the paper notes this term is absent.
  • The identity $L(P_F) = S(P_F)$ suggests that model entropy alone could serve as a model-selection criterion in other maximum entropy applications, such as species distribution modeling or natural language processing.
  • The continuous, parameterized-feature version sketched in the paper could link minimax entropy to representation learning, where learned features are optimized rather than selected from a discrete set.
  • Whether entropy reduction is submodular for real datasets is an open empirical question; if it is, greedy feature selection would be near-optimal in practice.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper claims to derive the minimax entropy principle from the minimum description length (MDL) principle. The central argument is that for a maximum-entropy model P_F(x) built from feature set F, the description length equals the model entropy, L(P_F)=S(P_F) (Eqs. 10-11), so the optimal feature set is the one whose maximum-entropy model has the smallest entropy (Eqs. 12-13). Under the assumption that empirical feature averages equal the true averages, the paper also shows that minimizing S(P_F) is equivalent to minimizing the KL divergence to the true distribution and to maximizing the information gain relative to an independent model. The authors illustrate the principle with a three-spin Ising example, then survey applications to Gaussian graphical models, Ising models of neural populations, texture modeling, and multiple-distribution models of handwritten digits. They discuss greedy algorithms, exact solutions for trees and generalized series-parallel networks, and open challenges such as submodularity and parameterized features.

Significance. If the idealized claim holds, the paper provides a clean unification of feature selection with maximum-entropy modeling, and the exactly solvable tree and GSP results are valuable practical contributions. The derivation is self-contained and the main equalities are easy to verify, which is a strength. However, the central claim is stated without the finite-sample qualifications that the formal argument actually requires, and the applications to real neural and brain data inherit that gap. The paper is therefore a useful perspective with a correct core, but its headline claim overreaches as written.

major comments (3)
  1. [Section III.A, Eq. (15); Section II.B, Eqs. (9)-(11)] The equalities L(P_F)=S(P_F) and D_KL(P_true||P_F)=S(P_F)-S(P_true) rely on the assumption, stated near Eq. (15), that the measured feature averages are exact: <f_mu(x)>_exp = <f_mu(x)>_true. For finite or noisy samples this assumption fails. If the fitted model matches the empirical averages but not the true averages, the KL divergence gains an additional term sum_mu lambda_mu (<f_mu>_exp - <f_mu>_true), and minimizing S(P_F) is no longer equivalent to minimizing D_KL(P_true||P_F). Because all applications in Section IV use real, finite data without a finite-sample correction or holdout validation, the statement that F* is 'optimal for the system' is not justified for those datasets. The authors should either restrict the claim to the noiseless limit or add a model-complexity/regularization term and discuss overfitting.
  2. [Section II.A-B; Appendix A] The quantity L(P) in Eq. (4) is the expected per-symbol code length averaged over datasets consistent with the observed features, not the full MDL stochastic complexity of the observed data. Standard MDL also charges for the cost of describing the model, including the feature set F and the parameters lambda_mu, and no such term appears in Eqs. (9)-(13). The paper's claim that the minimax entropy criterion is derived 'starting only from the MDL principle' (Introduction) is therefore overstated: the derivation shows that, among models with a fixed feature budget, the maximum-entropy model minimizes the data-coding cost, and that among feature sets of equal size the cross-entropy is minimized by minimizing model entropy. The authors should either reformulate the claim as a fixed-budget cross-entropy minimization or incorporate a model-complexity term and explain how the minimax criterion would change.
  3. [Section IV.B-C; Figs. 3-4] The applications to fMRI and neuronal recordings minimize entropy using empirical covariances and correlations from finite samples, but the paper does not quantify the sample sizes relative to the number of constraints or test predictive performance on held-out data. As a result, the reported 'optimal' networks could be selecting features that fit sampling fluctuations rather than genuine structure. I ask the authors to add a brief discussion of how the exact-averages assumption is approximated in these datasets and to state clearly that the reported improvements are measured for the empirical distributions, not for the true distributions.
minor comments (5)
  1. [Section V.C] The phrase 'computing the the gradients' contains a duplicated article and should read 'computing the gradients'.
  2. [Section IV.A] The spelling 'na¨ıve' uses a nonstandard dieresis; the standard English spelling is 'naive'.
  3. [References] Several references lack complete publication data: Ref. [3] has no year, Ref. [7] has no year or page range, Ref. [62] has no year, and Ref. [63] has no year. These should be completed.
  4. [Section IV.E] The multiple-distribution formulation in Eq. (26) does not state the objective being optimized over the shared feature set; the text should specify whether the criterion is the sum, average, or another combination of the entropies S(P_alpha).
  5. [Figure 1(b)] The caption says 'the optimal bound indicates the minimum entropy' but does not describe how the bound is computed; a sentence on the search procedure would improve reproducibility.

Circularity Check

0 steps flagged · score 1.0 of 10

No circularity in the central derivation; minor self-citations in application sections only.

full rationale

The derivation chain is self-contained. Appendix A derives the maximum-entropy distribution Q* as the MDL minimizer over P for fixed features, using the large-T saddle point of the histogram weight; this is an input-to-output derivation, not an assumption of the target. Equations (9)-(11) then show L(P_F)=S(P_F) for every maximum-entropy model by substituting the Boltzmann form and using the constraint averages <f>_chi = <f>_exp = <f>_P; this is a mathematical identity, not a definition of entropy as description length. The minimax statement (12)-(13) is the immediate corollary argmin_F L = argmin_F S, with the inner max expressing that P_F is the max-entropy model. The KL-divergence and information reinterpretations in Sec. III are likewise algebraic consequences of the same exact-constraint assumption. No fitted parameter is relabeled as a prediction: the neural and texture examples evaluate models on covariances/filters not used as constraints, so the 'predictions' are genuine. The paper's stated assumption of exact experimental averages (Sec. III.A) is a limitation for finite data, but it is an explicitly stated premise, not a circular import. The self-citations [8]-[11] support applications and solvability claims (e.g., GSP maximality) rather than the central L=S identity, so they are not load-bearing. Score 1 reflects only the presence of author self-citations in supporting application sections, not circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No numbers are fitted in the theoretical derivation. The toy Ising example uses fixed couplings; the neuroscience figures come from prior data analyses and do not introduce new free parameters in this paper. No new particles, forces, dimensions, or conserved quantities are introduced.

assumptions (4)
  • domain assumption Minimum description length is the correct criterion for model selection.
    Sec. II and the abstract take MDL as the starting point; the entire derivation of minimax entropy follows from minimizing a code length.
  • domain assumption The measured feature averages are exact and equal the true distribution averages (<f_mu>_exp = <f_mu>_true).
    Sec. III.A states this explicitly before deriving DKL(Ptrue||PF)=S(PF)-S(Ptrue); finite-sample noise is not modeled.
  • domain assumption The complexity of the model (number or type of features) is fixed or constrained when comparing feature sets.
    Sec. I and II.B compare models for a given level of complexity, but Eq. (12) as written has no complexity penalty; minimizing entropy alone would favor including all features unless the allowed set is constrained.
  • standard math Maximum entropy distributions exist and are unique for the considered feature sets.
    Used throughout to write PF in Boltzmann form and to assert that the model matches the constraints; standard in refs [12,13].

how reviews work

0 comments
Cite this review

Pith. "Pith review of Minimax entropy: The statistical physics of optimal models." pith.science (2026). https://pith.science/paper/Y3PWT6P2

@misc{pith2026250501607,
  author       = {Pith},
  title        = {Pith review of: Minimax entropy: The statistical physics of optimal models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y3PWT6P2}},
  note         = {Machine review of arXiv:2505.01607}
}
read the original abstract

When constructing models of the world, we aim for optimal compressions: models that include as few details as possible while remaining as accurate as possible. But which details -- or features measured in data -- should we choose to include in a model? Here, using the minimum description length principle, we show that the optimal features are the ones that produce the maximum entropy model with minimum entropy, thus yielding a minimax entropy principle. We review applications, which range from machine learning to optimal models of biological networks. Naive implementations, however, are limited to systems with small numbers of states and features. We therefore require new theoretical insights and computational techniques to construct optimal compressions of high-dimensional datasets arising in large-scale experiments.

Figures

Figures reproduced from arXiv: 2505.01607 by the authors.

Figure 1
Figure 1. FIG. 1. Useful models as optimal compressions. (a) Illus [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. FIG. 2. Minimizing divergence and maximizing informa [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. FIG. 3. Minimax entropy models of human neural activity. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: FIG. 4. Minimax entropy models of large populations of neu [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: FIG. 5. Minimax entropy model of texture. (a) Target image [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: FIG. 6. Minimax entropy models of handwritten digits. (a) [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

81 extracted references · 77 canonical work pages

  1. [1]

    For a given set of features F ={fµ(x)}, one must first solve the traditional maximum entropy prob- lem for the parameters λµ such that the model PF(x) matches the specified features [Eq. (1)]

    work page

  2. [2]

    This algorithm, while prohibitively inefficient in most cases (see below), provides a general solution to the min- imax entropy problem

    One must then repeat step 1 for each allowed set of featuresF in order to identify the optimal set F∗ that minimizes the model entropy S(PF). This algorithm, while prohibitively inefficient in most cases (see below), provides a general solution to the min- imax entropy problem. To illustrate this solution, consider a three-spin Ising model with collective...

    work page

  3. [3]

    After M steps, we have converged on the locally optimal set of features F∗ ={f1(x),...,f M(x)}. Although this greedy algorithm is not generally guaran- teed to converge to the globally optimal features [42, 43], it provides an approximate solution with a significant in- crease in efficiency [6, 7, 10]. In turn, this opens the door for practical applicatio...

    work page

  4. [4]

    To begin, select the single allowed feature f1(x) whose empirical average⟨f1(x)⟩exp yields the max- imum entropy model with minimum entropy

    work page

  5. [5]

    After selecting m < Mfeatures f1(x),...,f m(x), search over the remaining allowed features to find the one fm+1(x) that yields the lowest entropy S(PF), whereF ={f1(x),...,f m(x),fm+1(x)}

    work page

  6. [6]

    Minimax Entropy Principle and Its Application to Texture Modeling,

    Song Chun Zhu, Ying Nian Wu, and David Mum- ford, “Minimax Entropy Principle and Its Application to Texture Modeling,” Neural Computation 9, 1627–1660 (1997)

    work page 1997

  7. [7]

    A mathematical theory of commu- nication,

    Claude E Shannon, “A mathematical theory of commu- nication,” Bell Syst. Tech. J. 27, 379–423 (1948)

    work page 1948

  8. [8]

    Cover and Joy A

    Thomas M. Cover and Joy A. Thomas,Elements of Infor- mation Theory, 2nd ed. (John Wiley & Sons, Inc, 2006)

    work page 2006

Show all 81 references
  1. [9]

    Maximum entropy as a special case of the minimum description length criterion (corresp.),

    M. Feder, “Maximum entropy as a special case of the minimum description length criterion (corresp.),” IEEE Trans. Inf. Theory 32, 847–849

  2. [10]

    Peter D Gr¨ unwald, The minimum description length principle (MIT press, 2007)

    2007

  3. [11]

    Model selection and the principle of minimum description length,

    Mark H Hansen and Bin Yu, “Model selection and the principle of minimum description length,” J. Am. Stat. Assoc. 96, 746–774 (2001)

    2001

  4. [12]

    Information theory and statistical me- chanics,

    Edwin T Jaynes, “Information theory and statistical me- chanics,” Phys. Rev. 106, 620 (1957)

    1957

  5. [13]

    immediately hints at an algorithmic solution:

  6. [14]

    FRAME: filters, random fields, and minimax entropy towards a unified theory for texture modeling,

    Song Chun Zhu, Yingnian Wu, and D. Mumford, “FRAME: filters, random fields, and minimax entropy towards a unified theory for texture modeling,” in Pro- ceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition , pp. 686–693

  7. [15]

    Exactly solvable sta- tistical physics models for large neuronal populations,

    Christopher W Lynn, Qiwei Yu, Rich Pang, William Bialek, and Stephanie E Palmer, “Exactly solvable sta- tistical physics models for large neuronal populations,” ArXiv , arXiv–2310 (2023)

    2023

  8. [16]

    Exact minimax entropy models of large-scale neuronal activity,

    Christopher W Lynn, Qiwei Yu, Rich Pang, Stephanie E Palmer, and William Bialek, “Exact minimax entropy models of large-scale neuronal activity,” arXiv preprint arXiv:2402.00007 (2023)

    2023 arXiv

  9. [17]

    Statisti- cal physics of large-scale neural activity with loops,

    David P. Carcamo and Christopher W. Lynn, “Statisti- cal physics of large-scale neural activity with loops,” , arXiv:2412.18115

    arXiv

  10. [18]

    Quantifying the com- pressibility of the human brain,

    Nicholas J. Weaver, Joshua I. Faskowitz, Richard F. Bet- zel, and Christopher W. Lynn, “Quantifying the com- pressibility of the human brain,” In preparation (2025)

    2025

  11. [19]

    Principles of maximum entropy and maximum cal- iber in statistical physics,

    Steve Press´ e, Kingshuk Ghosh, Julian Lee, and Ken A Dill, “Principles of maximum entropy and maximum cal- iber in statistical physics,” Rev. Mod. Phys. 85, 1115– 1141 (2013)

    2013

  12. [20]

    Inverse statistical problems: from the inverse ising prob- lem to data science,

    H Chau Nguyen, Riccardo Zecchina, and Johannes Berg, “Inverse statistical problems: from the inverse ising prob- lem to data science,” Adv. Phys. 66, 197–261 (2017)

    2017

  13. [21]

    Statistical physics of inference: Thresholds and algorithms,

    Lenka Zdeborov´ a and Florent Krzakala, “Statistical physics of inference: Thresholds and algorithms,” Adv. Phys. 65, 453–552 (2016)

    2016

  14. [22]

    The in- verse problem in classical statistical mechanics,

    JT Chayes, L Chayes, and Elliott H Lieb, “The in- verse problem in classical statistical mechanics,” Com- mun. Math. Phys. 93, 57–121 (1984)

    1984

  15. [23]

    History of the lenz-ising model,

    Stephen G Brush, “History of the lenz-ising model,” Rev. Mod. Phys. 39, 883 (1967)

    1967

  16. [24]

    Weak pairwise correlations imply strongly correlated network states in a neural popula- tion,

    Elad Schneidman, Michael J Berry II, Ronen Segev, and William Bialek, “Weak pairwise correlations imply strongly correlated network states in a neural popula- tion,” Nature 440, 1007 (2006)

    2006

  17. [25]

    Pairwise maximum entropy model explains the role of white matter structure in shap- ing emergent co-activation states,

    Arian Ashourvan, Preya Shah, Adam Pines, Shi Gu, Christopher W Lynn, Danielle S Bassett, Kathryn A Davis, and Brian Litt, “Pairwise maximum entropy model explains the role of white matter structure in shap- ing emergent co-activation states,” Commun. Biol.4, 210 (2021, PMC7887247)

    2021

  18. [26]

    Spontaneous brain activity emerges from pairwise interactions in the larval zebrafish brain,

    Richard E. Rosch, Dominic R. W. Burrows, Christo- pher W. Lynn, and Arian Ashourvan, “Spontaneous brain activity emerges from pairwise interactions in the larval zebrafish brain,” Phys. Rev. X 14, 031050 (2024)

    2024

  19. [27]

    Collective behav- ior of place and non-place neurons in the hippocampal network,

    Leenoy Meshulam, Jeffrey L Gauthier, Carlos D Brody, David W Tank, and William Bialek, “Collective behav- ior of place and non-place neurons in the hippocampal network,” Neuron 96, 1178–1191 (2017). 11

    2017

  20. [28]

    Searching for collective behavior in a large network of sensory neurons,

    Gaˇ sper Tkaˇ cik, Olivier Marre, Dario Amodei, Elad Schneidman, William Bialek, and Michael J Berry II, “Searching for collective behavior in a large network of sensory neurons,” PLoS Comput. Biol. 10, e1003408 (2014)

    2014

  21. [29]

    Prediction of spatiotemporal patterns of neural activity from pairwise correlations,

    Olivier Marre, Sami El Boustani, Yves Fr´ egnac, and Alain Destexhe, “Prediction of spatiotemporal patterns of neural activity from pairwise correlations,” Phys. Rev. Lett. 102, 138101 (2009)

    2009

  22. [30]

    Protein 3D structure computed from evolutionary sequence variation,

    Debora S Marks, Lucy J Colwell, Robert Sheridan, Thomas A Hopf, Andrea Pagnani, Riccardo Zecchina, and Chris Sander, “Protein 3D structure computed from evolutionary sequence variation,” PLoS One 6, e28766 (2011)

    2011

  23. [31]

    Identification of di- rect residue contacts in protein–protein interaction by message passing,

    Martin Weigt, Robert A White, Hendrik Szurmant, James A Hoch, and Terence Hwa, “Identification of di- rect residue contacts in protein–protein interaction by message passing,” Proc. Natl. Acad. Sci. U.S.A. 106, 67– 72 (2009)

    2009

  24. [32]

    An evolution-based model for design- ing chorismate mutase enzymes,

    William P Russ, Matteo Figliuzzi, Christian Stocker, Pierre Barrat-Charlaix, Michael Socolich, Peter Kast, Donald Hilvert, Remi Monasson, Simona Cocco, Mar- tin Weigt, et al., “An evolution-based model for design- ing chorismate mutase enzymes,” Science 369, 440–445 (2020)

    2020

  25. [33]

    Direct-coupling analysis of residue coevolution captures native contacts across many protein families,

    Faruck Morcos, Andrea Pagnani, Bryan Lunt, Arianna Bertolino, Debora S. Marks, Chris Sander, Riccardo Zecchina, Jos´ e N. Onuchic, Terence Hwa, and Martin Weigt, “Direct-coupling analysis of residue coevolution captures native contacts across many protein families,” Proceeding...

    2011 doi

  26. [34]

    Using the prin- ciple of entropy maximization to infer genetic interac- tion networks from gene expression patterns,

    Timothy R Lezon, Jayanth R Banavar, Marek Cieplak, Amos Maritan, and Nina V Fedoroff, “Using the prin- ciple of entropy maximization to infer genetic interac- tion networks from gene expression patterns,” Proc. Natl. Acad. Sci. U.S.A. 103, 19033–19038 (2006)

    2006

  27. [35]

    Quantifying extrinsic noise in gene expression using the maximum entropy framework,

    Purushottam D Dixit, “Quantifying extrinsic noise in gene expression using the maximum entropy framework,” Biophys. J. 104, 2743–2750 (2013)

    2013

  28. [36]

    Transferable model for chromosome architecture,

    Michele Di Pierro, Bin Zhang, Erez Lieberman Aiden, Peter G Wolynes, and Jos´ e N Onuchic, “Transferable model for chromosome architecture,” Proc. Natl. Acad. Sci. U.S.A. 113, 12168–12173 (2016)

    2016

  29. [37]

    Multiscale modeling of genome organization with maximum entropy optimization,

    Xingcheng Lin, Yifeng Qi, Andrew P Latham, and Bin Zhang, “Multiscale modeling of genome organization with maximum entropy optimization,” J. Chem. Phys. 155 (2021)

    2021

  30. [38]

    A maximum-entropy model to predict 3D structural ensembles of chromatin from pairwise distances with applications to interphase chromosomes and structural variants,

    Guang Shi and D Thirumalai, “A maximum-entropy model to predict 3D structural ensembles of chromatin from pairwise distances with applications to interphase chromosomes and structural variants,” Nat. Commun. 14, 1150 (2023)

    2023

  31. [39]

    Learning the distribution of single-cell chromosome con- formations in bacteria reveals emergent order across ge- nomic scales,

    Joris JB Messelink, Muriel CF van Teeseling, Jacqueline Janssen, Martin Thanbichler, and Chase P Broedersz, “Learning the distribution of single-cell chromosome con- formations in bacteria reveals emergent order across ge- nomic scales,” Nat. Commun. 12, 1963 (2021)

    2021

  32. [40]

    A maximum-entropy model for predicting chromatin contacts,

    Pau Farr´ e and Eldon Emberly, “A maximum-entropy model for predicting chromatin contacts,” PLoS Com- put. Biol. 14, e1005956 (2018)

    2018

  33. [41]

    Statistical mechanics for natural flocks of birds,

    William Bialek, Andrea Cavagna, Irene Giardina, Thierry Mora, Edmondo Silvestri, Massimiliano Viale, and Aleksandra M Walczak, “Statistical mechanics for natural flocks of birds,” Proc. Natl. Acad. Sci. U.S.A. 109, 4786–4791 (2012)

    2012

  34. [42]

    Dynamical maximum entropy approach to flocking,

    Andrea Cavagna, Irene Giardina, Francesco Ginelli, Thierry Mora, Duccio Piovani, Raffaele Tavarone, and Aleksandra M Walczak, “Dynamical maximum entropy approach to flocking,” Phys. Rev. E 89, 042707 (2014)

    2014

  35. [43]

    Social inter- actions dominate speed control in poising natural flocks near criticality,

    William Bialek, Andrea Cavagna, Irene Giardina, Thierry Mora, Oliver Pohl, Edmondo Silvestri, Massim- iliano Viale, and Aleksandra M Walczak, “Social inter- actions dominate speed control in poising natural flocks near criticality,” Proc. Natl. Acad. Sci. U.S.A.111, 7212– 7217 (2014)

    2014

  36. [44]

    Surges of collective human ac- tivity emerge from simple pairwise correlations,

    Christopher W Lynn, Lia Papadopoulos, Daniel D Lee, and Danielle S Bassett, “Surges of collective human ac- tivity emerge from simple pairwise correlations,” Phys. Rev. X 9, 011022 (2019)

    2019

  37. [45]

    Network information and connected correlations,

    Elad Schneidman, Susanne Still, Michael J Berry, and William Bialek, “Network information and connected correlations,” Phys. Rev. Lett. 91, 238701 (2003)

    2003

  38. [46]

    14 (Oxford University Press, USA, 2021)

    James P Sethna, Statistical Mechanics: Entropy, Order Parameters, and Complexity , Vol. 14 (Oxford University Press, USA, 2021)

    2021

  39. [47]

    Critical behavior of the three-dimensional ising model: A high-resolution monte carlo study,

    Alan M Ferrenberg and DP Landau, “Critical behavior of the three-dimensional ising model: A high-resolution monte carlo study,” Phys. Rev. B 44, 5081 (1991)

    1991

  40. [48]

    R Gary Parker and Ronald L Rardin, Discrete optimiza- tion (Elsevier, 2014)

    2014

  41. [49]

    3 (Springer, 2005)

    Dieter Jungnickel, Graphs, networks and algorithms , Vol. 3 (Springer, 2005)

    2005

  42. [50]

    David Edwards, Introduction to graphical modelling (Springer Science & Business Media, 2012)

    2012

  43. [51]

    Model selection and estimation in the Gaussian graphical model,

    Ming Yuan and Yi Lin, “Model selection and estimation in the Gaussian graphical model,” Biometrika 94, 19–35 (2007)

    2007

  44. [52]

    The Gaussian graphical model in cross-sectional and time-series data,

    Sacha Epskamp, Lourens J Waldorp, Ren´ e M˜ ottus, and Denny Borsboom, “The Gaussian graphical model in cross-sectional and time-series data,” Multivar. Behav. Res. 53, 453–480 (2018)

    2018

  45. [53]

    FastGGM: an efficient algorithm for the inference of Gaussian graphical model in biological net- works,

    Ting Wang, Zhao Ren, Ying Ding, Zhou Fang, Zhe Sun, Matthew L MacDonald, Robert A Sweet, Jieru Wang, and Wei Chen, “FastGGM: an efficient algorithm for the inference of Gaussian graphical model in biological net- works,” PLoS Comput. Biol. 12, e1004755 (2016)

    2016

  46. [54]

    Gaussian graphical models: An algebraic and geometric perspective,

    Caroline Uhler, “Gaussian graphical models: An algebraic and geometric perspective,” (2017), arXiv:1707.04345

    2017 arXiv

  47. [55]

    The WU-Minn Human Connectome Project: An overview,

    David C Van Essen, Stephen M Smith, Deanna M Barch, Timothy EJ Behrens, Essa Yacoub, Kamil Ugurbil, Wu- Minn HCP Consortium, et al. , “The WU-Minn Human Connectome Project: An overview,” Neuroimage 80, 62– 79 (2013)

    2013

  48. [56]

    Network neuro- science,

    Danielle S Bassett and Olaf Sporns, “Network neuro- science,” Nat. Neurosci. 20, 353–364 (2017)

    2017

  49. [57]

    The physics of brain network structure, function and control,

    Christopher W. Lynn and Danielle S. Bassett, “The physics of brain network structure, function and control,” Nat. Rev. Phys. 1, 318–332 (2019)

    2019

  50. [58]

    Maximum entropy models for antibody diversity,

    Thierry Mora, Aleksandra M Walczak, William Bialek, and Curtis G Callan Jr, “Maximum entropy models for antibody diversity,” Proc. Natl. Acad. Sci. U.S.A. 107, 5405–5410 (2010)

    2010

  51. [59]

    High- dimensional geometry of population responses in visual cortex,

    Carsen Stringer, Marius Pachitariu, Nicholas Steinmetz, Matteo Carandini, and Kenneth D. Harris, “High- dimensional geometry of population responses in visual cortex,” Nature 571, 361–365 (2019). 12

    2019

  52. [60]

    Rodney J Baxter, Exactly solved models in statistical me- chanics (Elsevier, 2016)

    2016

  53. [61]

    Approximating discrete probability distributions with dependence trees,

    CKCN Chow and Cong Liu, “Approximating discrete probability distributions with dependence trees,” IEEE Trans. Inf. Theory 14, 462–467 (1968)

    1968

  54. [62]

    Cristopher Moore and Stephan Mertens, The nature of computation (Oxford University Press, 2011)

    2011

  55. [63]

    Fundamentals of the exact renormal- ization group,

    Oliver J Rosten, “Fundamentals of the exact renormal- ization group,” Phys. Rep. 511, 177–272 (2012)

    2012

  56. [64]

    Spontaneous behaviors drive multidimensional, brainwide activity,

    Carsen Stringer, Marius Pachitariu, Nicholas Steinmetz, Charu Bai Reddy, Matteo Carandini, and Kenneth D. Harris, “Spontaneous behaviors drive multidimensional, brainwide activity,” Science 364, eaav7893 (2019)

    2019

  57. [65]

    Host lifestyle affects human microbiota on daily timescales,

    Lawrence A David, Arne C Materna, Jonathan Fried- man, Maria I Campos-Baptista, Matthew C Black- burn, Allison Perrotta, Susan E Erdman, and Eric J Alm, “Host lifestyle affects human microbiota on daily timescales,” Genome Biology 15 (2014), 10.1186/gb- 2014-15-7-r89

    2014 doi

  58. [66]

    Neural network-based face detection,

    Henry Rowley, Shumeet Baluja, and Takeo Kanade, “Neural network-based face detection,” IEEE Transac- tions on Pattern Analysis and Machine Intelligence 20, 23–38 (1998)

    1998

  59. [67]

    Understand- ing bag-of-words model: A statistical framework,

    Yin Zhang, Rong Jin, and Zhi-Hua Zhou, “Understand- ing bag-of-words model: A statistical framework,” Inter- national Journal of Machine Learning and Cybernetics 1, 43–52 (2010)

    2010

  60. [68]

    The mnist database of handwritten digit im- ages for machine learning research,

    Li Deng, “The mnist database of handwritten digit im- ages for machine learning research,” IEEE Signal Pro- cessing Magazine 29, 141–142

  61. [69]

    Thermodynamic inference of data manifolds,

    Purushottam D. Dixit, “Thermodynamic inference of data manifolds,” Phys. Rev. Res. 2, 023201

  62. [70]

    A maximum entropy approach to species dis- tribution modeling,

    Steven J Phillips, Miroslav Dud´ ık, and Robert E Schapire, “A maximum entropy approach to species dis- tribution modeling,” in Proceedings of the twenty-first in- ternational conference on Machine learning (2004) p. 83

    2004

  63. [71]

    Learning to parse natural lan- guage with maximum entropy models,

    Adwait Ratnaparkhi, “Learning to parse natural lan- guage with maximum entropy models,” Mach. Learn.34, 151–175 (1999)

    1999

  64. [72]

    Solution of’solvable model of a spin glass’,

    David J Thouless, Philip W Anderson, and Robert G Palmer, “Solution of’solvable model of a spin glass’,” Phi- los. Mag. 35, 593–601 (1977)

    1977

  65. [73]

    Mean-field theory of boltzmann ma- chine learning,

    Toshiyuki Tanaka, “Mean-field theory of boltzmann ma- chine learning,” Phys. Rev. E 58, 2302 (1998)

    1998

  66. [74]

    Ising model for neural data: model quality and approximate methods for extracting functional connectivity,

    Yasser Roudi, Joanna Tyrcha, and John Hertz, “Ising model for neural data: model quality and approximate methods for extracting functional connectivity,” Phys. Rev. E 79, 051915 (2009)

    2009

  67. [75]

    Neuronal avalanches in neocortical circuits,

    John M Beggs and Dietmar Plenz, “Neuronal avalanches in neocortical circuits,” J. Neurosci. 23, 11167–11177 (2003)

    2003

  68. [76]

    Thermodynamics and signatures of crit- icality in a network of neurons,

    Gaˇ sper Tkaˇ cik, Thierry Mora, Olivier Marre, Dario Amodei, Stephanie E. Palmer, Michael J. Berry, and William Bialek, “Thermodynamics and signatures of crit- icality in a network of neurons,” Proc. Natl. Acad. Sci. U.S.A. 112, 11508–11513 (2015)

    2015

  69. [77]

    Coarse graining, fixed points, and scaling in a large population of neu- rons,

    Leenoy Meshulam, Jeffrey L Gauthier, Carlos D Brody, David W Tank, and William Bialek, “Coarse graining, fixed points, and scaling in a large population of neu- rons,” Phys. Rev. Lett. 123, 178103 (2019)

    2019

  70. [78]

    Extended mean-field the- ories for networks of real neurons,

    Luca Di Carlo, Francesca Mignacco, Christopher W Lynn, and William Bialek, “Extended mean-field the- ories for networks of real neurons,” arXiv preprint arXiv:2504.15197 (2025)

    2025 arXiv

  71. [79]

    Adaptive Cluster Expansion for the Inverse Ising Problem: Convergence, Algorithm and Tests,

    S. Cocco and R. Monasson, “Adaptive Cluster Expansion for the Inverse Ising Problem: Convergence, Algorithm and Tests,” J. Stat. Phys. 147, 252–314 (2012)

    2012

  72. [80]

    58 (Elsevier, 2005)

    Satoru Fujishige, Submodular functions and optimization, Vol. 58 (Elsevier, 2005)

    2005

  73. [81]

    Submodular func- tion maximization

    Andreas Krause and Daniel Golovin, “Submodular func- tion maximization.” Tractability 3, 3 (2014)

    2014

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.