Modeling Discrete Data with High-Order Vector Potts Models
Pith reviewed 2026-06-28 08:56 UTC · model grok-4.3
The pith
q-state spin models generalize the vector Potts model to capture arbitrary high-order interactions in discrete data.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
q-state spin models form a complete family of maximum entropy models that generalize the vector Potts model to include long-range and arbitrary high-order interactions in discrete data. Their statistical properties are fully captured by the algebraic structure of their interactions, as shown via loop expansion of the partition function. Models related by gauge transformations share the same partition function and represent the same abstract statistical model despite different interaction orders.
What carries the argument
q-state spin models, which extend the vector Potts model using algebraic structures for interactions, with loop expansion revealing invariance under gauge transformations.
If this is right
- Models equivalent under gauge transformations represent the same statistics but can use interactions of different orders.
- The algebraic structure determines all statistical properties, allowing focus on interaction structure rather than specific orders.
- Minimally complex models have a closed-form expression for marginal likelihood, enabling fast model selection on discrete data.
- These models can be applied to infer higher-order correlations in systems like protein sequences or neural activity.
Where Pith is reading between the lines
- Choosing different gauge representations might simplify fitting high-order models to data by reducing effective order.
- This framework could extend maximum entropy modeling to other discrete variable systems beyond the examples given.
- The invariance property may help in developing more efficient algorithms for parameter estimation in high-dimensional discrete data.
- Applying these models to real datasets could reveal previously hidden higher-order structures in complex systems.
Load-bearing premise
The statistical properties of the spin models are fully captured by the algebraic structure of their interactions.
What would settle it
A calculation or simulation where two gauge-equivalent models exhibit different partition functions or statistical properties would falsify the claim.
Figures
read the original abstract
Modeling high-dimensional data is challenging, yet essential to understanding many complex systems. Maximum entropy models such as Ising and Potts models have been used extensively to capture pairwise interactions from correlation patterns in data, allowing to infer graphical representations of complex systems from observations (e.g., from protein sequences or neural population activity). Recently, there has been growing interest in modeling higher-order correlation patterns involving simultaneously three or more variables. While progress has been made in binary data with high-order Ising models, we extend this framework to the more general case of discrete data. We introduce q-state spin models, a complete family of maximum entropy models that generalize the vector Potts model to include long-range and arbitrary high-order interactions. In the pairwise case, our models allow for more diverse interaction types compared to the standard vector Potts model. We discuss their statistical interpretation with examples and relate them to discrete Fourier analysis. Using a loop expansion of the partition function, we show that the statistical properties of spin models are fully captured by the algebraic structure of their interactions. We define gauge transformations under which this structure, and thus the partition function, remains invariant. Models equivalent under gauge transformations can be seen as different representations of the same abstract statistical model, despite generally having interactions of different orders, extending results from the binary case. For practical application to data analysis, we focus on a subset of models known in the binary case as Minimally Complex Models, generalizing them to discrete data. We obtain a closed-form expression for the marginal likelihood of these models, enabling fast model selection. We illustrate their use with simple real-world examples.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces q-state spin models as a complete family of maximum entropy models for discrete (q-state) data that generalize the vector Potts model to arbitrary high-order and long-range interactions. It claims that a loop expansion of the partition function demonstrates that statistical properties are fully determined by the algebraic structure of the interactions, leading to gauge transformations under which the structure and partition function are invariant (allowing equivalent models of different orders). It further derives a closed-form marginal likelihood for the subset of Minimally Complex Models to enable fast model selection and illustrates the approach with real-world examples.
Significance. If the central claims on completeness, exact capture via loop expansion, and closed-form marginal likelihood hold, the work would provide a principled extension of high-order maxent models from binary to general discrete data, with direct implications for graphical modeling in fields such as protein sequence analysis and neural data. The explicit treatment of gauge equivalence and the practical model-selection formula are potential strengths for reproducibility and applicability.
major comments (2)
- [Abstract / loop-expansion section] Abstract and the section introducing the loop expansion: the claim that 'using a loop expansion of the partition function, we show that the statistical properties of spin models are fully captured by the algebraic structure of their interactions' is load-bearing for the completeness and gauge-invariance results, yet the provided text gives no explicit derivation, truncation argument, or demonstration that the series is exact (rather than perturbative) or that omitted diagrams preserve the claimed invariance. This directly affects whether models of different orders are rigorously equivalent.
- [Section on Minimally Complex Models / marginal likelihood] The derivation of the closed-form marginal likelihood for Minimally Complex Models (mentioned in the abstract) is central to the practical contribution; without the explicit steps or assumptions under which the expression is obtained, it is impossible to assess whether it generalizes the binary case without introducing hidden parameters or approximations.
minor comments (1)
- [Abstract] The abstract states that pairwise models 'allow for more diverse interaction types compared to the standard vector Potts model' but does not specify which additional interaction types are enabled or how they relate to the Fourier-analysis connection mentioned later.
Simulated Author's Rebuttal
We thank the referee for their thoughtful review and positive assessment of the potential significance of our work on q-state spin models. We address each major comment below and will revise the manuscript to provide the requested explicit derivations and clarifications.
read point-by-point responses
-
Referee: [Abstract / loop-expansion section] Abstract and the section introducing the loop expansion: the claim that 'using a loop expansion of the partition function, we show that the statistical properties of spin models are fully captured by the algebraic structure of their interactions' is load-bearing for the completeness and gauge-invariance results, yet the provided text gives no explicit derivation, truncation argument, or demonstration that the series is exact (rather than perturbative) or that omitted diagrams preserve the claimed invariance. This directly affects whether models of different orders are rigorously equivalent.
Authors: We agree that the loop expansion underpins the claims of completeness and gauge invariance. The current manuscript provides a high-level outline of the approach and its implications but does not include the full step-by-step derivation or truncation argument. In the revised version, we will expand the relevant section (and add an appendix if needed) with the explicit loop expansion, including the series terms, demonstration that it is exact for the partition function in this context, and verification that omitted diagrams preserve the algebraic invariance. This will rigorously establish the equivalence of models under gauge transformations. revision: yes
-
Referee: [Section on Minimally Complex Models / marginal likelihood] The derivation of the closed-form marginal likelihood for Minimally Complex Models (mentioned in the abstract) is central to the practical contribution; without the explicit steps or assumptions under which the expression is obtained, it is impossible to assess whether it generalizes the binary case without introducing hidden parameters or approximations.
Authors: We acknowledge that while the manuscript states the closed-form result and its utility for model selection, the explicit derivation steps and assumptions are not detailed in the provided text. In the revision, we will include the full derivation, specifying the assumptions (e.g., the structure of Minimally Complex Models) and confirming that the expression generalizes the binary case exactly without additional parameters or approximations. revision: yes
Circularity Check
No significant circularity in derivation chain
full rationale
The paper defines q-state spin models as a maxent family generalizing vector Potts models, then applies a loop expansion of the partition function to relate statistical properties to interaction algebra and introduces gauge transformations preserving the structure. These are presented as direct analytical consequences of the model definition and standard perturbative techniques rather than any reduction of a claimed prediction back to fitted inputs, self-citations, or ansatzes by construction. No load-bearing step equates an output quantity to its own inputs via redefinition or renaming; the completeness and invariance claims rest on explicit expansions and transformations whose validity is independent of the target results. The derivation is therefore self-contained.
Axiom & Free-Parameter Ledger
axioms (2)
- domain assumption Maximum entropy principle suffices to capture essential statistics from correlation patterns
- ad hoc to paper Loop expansion of the partition function fully determines statistical properties from interaction algebra
invented entities (1)
-
q-state spin models
no independent evidence
Reference graph
Works this paper leans on
-
[1]
Information theory and statistical mechanics
Edwin T Jaynes. Information theory and statistical mechanics. Physical review, 106(4):620, 1957
1957
-
[2]
Inverse statistical problems: from the inverse Ising problem to data science
H Chau Nguyen, Riccardo Zecchina, and Johannes Berg. Inverse statistical problems: from the inverse Ising problem to data science. Advances in Physics , 66(3):197–261, 2017
2017
-
[3]
Weak pairwise correlations imply strongly correlated network states in a neural population
Elad Schneidman, Michael J Berry, Ronen Segev, and William Bialek. Weak pairwise correlations imply strongly correlated network states in a neural population. Nature, 440(7087):1007–1012, 2006
2006
-
[4]
Ex” for Extraversion, “ES
Simona Cocco, Christoph Feinauer, Matteo Figliuzzi, Rémi Monasson, and Martin Weigt. Inverse statistical physics of protein sequences: a key issues review. Reports on Progress in Physics, 81(3):032601, 2018. 86 arXiv Template A Preprint q =3q =5 Scheme 2 — standardized dataScheme 1 — direct map from raw data C Ex E S A O E Ex E S A C O E Entropy Figure A2...
2018
-
[5]
Using the principle of entropy maximization to infer genetic interaction networks from gene expression patterns
Timothy R Lezon, Jayanth R Banavar, Marek Cieplak, Amos Maritan, and Nina V Fedoroff. Using the principle of entropy maximization to infer genetic interaction networks from gene expression patterns. Proceedings of the National Academy of Sciences , 103(50):19033–19038, 2006
2006
-
[6]
Identification of direct residue contacts in protein–protein interaction by message passing
Martin Weigt, Robert A White, Hendrik Szurmant, James A Hoch, and Terence Hwa. Identification of direct residue contacts in protein–protein interaction by message passing. Proceedings of the National Academy of Sciences , 106(1):67–72, 2009
2009
-
[7]
Direct-coupling analysis of residue coevolution captures native contacts across many protein families
Faruck Morcos, Andrea Pagnani, Bryan Lunt, Arianna Bertolino, Debora S Marks, Chris Sander, Ric- cardo Zecchina, José N Onuchic, Terence Hwa, and Martin Weigt. Direct-coupling analysis of residue coevolution captures native contacts across many protein families. Proceedings of the National Academy of Sciences , 108(49):E1293–E1301, 2011
2011
-
[8]
Improved contact prediction in proteins: using pseudolikelihoods to infer Potts models
Magnus Ekeberg, Cecilia Lövkvist, Yueheng Lan, Martin Weigt, and Erik Aurell. Improved contact prediction in proteins: using pseudolikelihoods to infer Potts models. Physical Review E , 87(1):012707, 2013
2013
-
[9]
ACE: adaptive cluster expansion for maximum entropy graphical model inference
John P Barton, Eleonora De Leonardis, Alice Coucke, and Simona Cocco. ACE: adaptive cluster expansion for maximum entropy graphical model inference. Bioinformatics, 32(20):3089–3097, 2016
2016
-
[10]
Benchmarking inverse statistical approaches for protein structure and design with exactly solvable models
Hugo Jacquin, Amy Gilson, Eugene Shakhnovich, Simona Cocco, and Rémi Monasson. Benchmarking inverse statistical approaches for protein structure and design with exactly solvable models. PLoS computational biology, 12(5):e1004889, 2016
2016
-
[11]
Inference of compressed Potts graphical models
Francesca Rizzato, Alice Coucke, Eleonora de Leonardis, John P Barton, Jérôme Tubiana, Remi Monasson, and Simona Cocco. Inference of compressed Potts graphical models. Physical Review E , 101(1):012309, 2020. 87 arXiv Template A Preprint
2020
-
[12]
Networks beyond pairwise interactions: Structure and dynamics
Federico Battiston, Giulia Cencetti, Iacopo Iacopini, Vito Latora, Maxime Lucas, Alice Patania, Jean- Gabriel Young, and Giovanni Petri. Networks beyond pairwise interactions: Structure and dynamics. Physics reports, 874:1–92, 2020
2020
-
[13]
The physics of higher-order interactions in complex systems
Federico Battiston, Enrico Amico, Alain Barrat, Ginestra Bianconi, Guilherme Ferraz de Arruda, Benedetta Franceschiello, Iacopo Iacopini, Sonia Kéfi, Vito Latora, Yamir Moreno, et al. The physics of higher-order interactions in complex systems. Nature Physics , 17(10):1093–1098, 2021
2021
-
[14]
On the typical properties of inverse problems in statistical mechanics
Iacopo Mastromatteo. On the typical properties of inverse problems in statistical mechanics. arXiv preprint arXiv:1311.0190 , 2013
Pith/arXiv arXiv 2013
-
[15]
The stochastic complexity of spin models: Are pairwise models really simple? Entropy, 20(10):739, 2018
Alberto Beretta, Claudia Battistin, Clélia De Mulatier, Iacopo Mastromatteo, and Matteo Marsili. The stochastic complexity of spin models: Are pairwise models really simple? Entropy, 20(10):739, 2018
2018
-
[16]
Duality in generalized ising models and phase transitions without local order parame- ters
Franz J Wegner. Duality in generalized ising models and phase transitions without local order parame- ters. Journal of Mathematical Physics , 12(10):2259–2272, 1971
1971
-
[17]
Doing Bayesian data analysis: A tutorial with R, JAGS, and Stan
John Kruschke. Doing Bayesian data analysis: A tutorial with R, JAGS, and Stan . Academic Press, 2014
2014
-
[18]
Stochastic complexity and modeling
Jorma Rissanen. Stochastic complexity and modeling. The annals of statistics , pages 1080–1100, 1986
1986
-
[19]
Fisher information and stochastic complexity
Jorma J Rissanen. Fisher information and stochastic complexity. IEEE transactions on information theory, 42(1):40–47, 1996
1996
-
[20]
The minimum description length principle
Peter D Grünwald. The minimum description length principle . MIT press, 2007
2007
-
[21]
Statistical inference, Occam’s razor, and statistical mechanics on the space of probability distributions
Vijay Balasubramanian. Statistical inference, Occam’s razor, and statistical mechanics on the space of probability distributions. Neural computation, 9(2):349–368, 1997
1997
-
[22]
Counting probability distributions: Differen- tial geometry and model selection
In Jae Myung, Vijay Balasubramanian, and Mark A Pitt. Counting probability distributions: Differen- tial geometry and model selection. Proceedings of the National Academy of Sciences, 97(21):11170–11175, 2000
2000
-
[23]
Bayesian inference of minimally complex models with interactions of arbitrary order
Clélia de Mulatier and Matteo Marsili. Bayesian inference of minimally complex models with interactions of arbitrary order. Physical Review E , 111(5):054307, 2025
2025
-
[24]
Some generalized order-disorder transformations
Renfrey Burnard Potts. Some generalized order-disorder transformations. In Mathematical proceedings of the cambridge philosophical society , volume 48, pages 106–109. Cambridge University Press, 1952
1952
-
[25]
Alternative approach to maximum-entropy inference
Y Tikochinsky, NZ Tishby, and Raphael David Levine. Alternative approach to maximum-entropy inference. Physical Review A , 30(5):2638, 1984
1984
-
[26]
Statistical theory of equations of state and phase transitions
Chen-Ning Yang and Tsung-Dao Lee. Statistical theory of equations of state and phase transitions. i. theory of condensation. Physical Review, 87(3):404, 1952
1952
-
[27]
Violation of lee-yang circle theorem for ising phase transitions on complex networks
M Krasnytska, Bertrand Berche, Yu Holovatch, and Ralph Kenna. Violation of lee-yang circle theorem for ising phase transitions on complex networks. Europhysics Letters, 111(6):60009, 2015
2015
-
[28]
Partition function zeros for the ising model on complete graphs and on annealed scale-free networks
Mariana Krasnytska, Bertrand Berche, Yu Holovatch, and Ralph Kenna. Partition function zeros for the ising model on complete graphs and on annealed scale-free networks. Journal of Physics A: Mathematical and Theoretical, 49(13):135001, 2016
2016
-
[29]
Potts models and random-cluster processes with many-body interactions
Geoffrey Grimmett. Potts models and random-cluster processes with many-body interactions. Journal of Statistical Physics , 75:67–121, 1994
1994
-
[30]
The Potts model
Fa-Yueh Wu. The Potts model. Reviews of modern physics , 54(1):235, 1982
1982
-
[31]
Blume–capel model: Estimation of a three stable state network for- 1, 0 and+ 1 data
Lourens Waldorp, Jonas Dalege, Maarten Marsman, Adam Finnemann, Irene Ferri, and Han LJ van der Maas. Blume–capel model: Estimation of a three stable state network for- 1, 0 and+ 1 data. Physica A: Statistical Mechanics and its Applications , page 131554, 2026
2026
-
[32]
Theory of the first-order magnetic phase change in u o 2
Martin Blume. Theory of the first-order magnetic phase change in u o 2. Physical Review, 141(2):517, 1966
1966
-
[33]
On the possibility of first-order phase transitions in ising systems of triplet ions with zero-field splitting
Hans Willem Capel. On the possibility of first-order phase transitions in ising systems of triplet ions with zero-field splitting. Physica, 32(5):966–988, 1966. 88 arXiv Template A Preprint
1966
-
[34]
Rikkert Hindriks, Tommy A. A. Broeders, Menno M. Schoonheim, Linda Douw, Fernando Santos, Wessel van Wieringen, and Prejaas K. B. Tewarie. Higher-order functional connectivity analysis of resting-state functional magnetic resonance imaging data using multivariate cumulants. Human Brain Mapping, 45(5):e26663, 2024
2024
-
[35]
A haplotype inference method based on sparsely connected multi-body ising model
Masashi Kato, Qian Ji Gao, Hiroshi Chigira, Hiroyuki Shindo, and Masato Inoue. A haplotype inference method based on sparsely connected multi-body ising model. In Journal of Physics: Conference Series , volume 233, page 012022. IOP Publishing, 2010
2010
-
[36]
On sparse, spec- tral and other parameterizations of binary probabilistic models
David Buchman, Mark Schmidt, Shakir Mohamed, David Poole, and Nando De Freitas. On sparse, spec- tral and other parameterizations of binary probabilistic models. In Artificial Intelligence and Statistics , pages 173–181. PMLR, 2012
2012
-
[37]
Ising-like models on arbitrary graphs: The hadamard way
Rémy Mosseri. Ising-like models on arbitrary graphs: The hadamard way. Physical Review E , 91(1):012142, 2015
2015
-
[38]
Representing multivariate patterns of binary data: {0, 1}versus {±1}-conventions in the high-order Ising-like models
Ebo Peerbooms and Clelia de Mulatier. Representing multivariate patterns of binary data: {0, 1}versus {±1}-conventions in the high-order Ising-like models. In preparation, 2026
2026
-
[39]
High-order community detection in data with minimally complex models and application to modeling the human brain
Emanuele Agrimi, Carlo Orientale Caputo, Lotte Wolfenter, Sam Kamphof, Jean-Hugues Lestang, Tom- maso Gili, Eugenio Piasini, and Clelia de Mulatier. High-order community detection in data with minimally complex models and application to modeling the human brain. In preparation, 2026
2026
-
[40]
Information geometry on hierarchy of probability distributions
S-I Amari. Information geometry on hierarchy of probability distributions. IEEE transactions on information theory , 47(5):1701–1711, 2002
2002
-
[41]
Beyond sin and cos
Martin E Muldoon and Abraham A Ungar. Beyond sin and cos. Mathematics Magazine , 69(1):3–14, 1996
1996
-
[42]
A Course in the Theory of Groups , volume 80
Derek JS Robinson. A Course in the Theory of Groups , volume 80. Springer Science & Business Media, 2012
2012
-
[43]
Information and the accuracy attainable in the estimation of statistical parameters
C Radhakrishna Rao et al. Information and the accuracy attainable in the estimation of statistical parameters. Bull. Calcutta Math. Soc , 37(3):81–91, 1945
1945
-
[44]
Methods of information geometry , volume 191
Shun-ichi Amari and Hiroshi Nagaoka. Methods of information geometry , volume 191. American Math- ematical Soc., 2000
2000
-
[45]
Criticality of mostly informative samples: a bayesian model selection approach
Ariel Haimovici and Matteo Marsili. Criticality of mostly informative samples: a bayesian model selection approach. Journal of Statistical Mechanics: Theory and Experiment , 2015(10):P10013, 2015
2015
-
[46]
On maximum entropy and inference
Luigi Gresele and Matteo Marsili. On maximum entropy and inference. Entropy, 19(12):642, 2017
2017
-
[47]
An invariant form for the prior probability in estimation problems
Harold Jeffreys. An invariant form for the prior probability in estimation problems. Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences , 186(1007):453–461, 1946
1946
-
[48]
Loopless algorithms for generating permutations, combinations, and other combinatorial configurations
Gideon Ehrlich. Loopless algorithms for generating permutations, combinations, and other combinatorial configurations. Journal of the ACM (JACM) , 20(3):500–513, 1973
1973
-
[49]
The art of computer programming, volume 4A: combinatorial algorithms, part 1
Donald E Knuth. The art of computer programming, volume 4A: combinatorial algorithms, part 1 . Pearson Education India, 2011
2011
-
[50]
Aaron De Clerq and Clélia de Mulatier. MinCompSpin(mcmpy): a package for model selection among minimally complex models for discrete data, 2026. https://doi.org/10.5281/zenodo.20490993
-
[51]
Supreme Court database, version 2011 release 3
Harold J Spaeth, Lee Epstein, Andrew D Martin, Jeffrey A Segal, Theodore J Ruger, and Sara C Benesh. Supreme Court database, version 2011 release 3. http://supremecourtdatabase.org, 2024
2011
-
[52]
Statistical mechanics of the US supreme court
Edward D Lee, Chase P Broedersz, and William Bialek. Statistical mechanics of the US supreme court. Journal of Statistical Physics , 160(2):275–301, 2015
2015
-
[53]
The development of markers for the big-five factor structure
Lewis R Goldberg. The development of markers for the big-five factor structure. Psychological assess- ment, 4(1):26, 1992
1992
-
[54]
https://openpsychometrics.org/ _rawdata/
Answers to the ipip big five factor markers, version 11/8/2018. https://openpsychometrics.org/ _rawdata/
2018
-
[55]
Strong optimality of the normalized ml models as universal codes and information in data
Jorma Rissanen. Strong optimality of the normalized ml models as universal codes and information in data. IEEE Transactions on Information Theory , 47(5):1712–1717, 2001. 89
2001
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.