Geometry of R\'enyi Entropy on the Majorization Lattice
Pith reviewed 2026-05-25 06:09 UTC · model grok-4.3
The pith
Rényi entropy is subadditive on the majorization lattice for every order α in [0, ∞].
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
We establish a fundamental relation between the comonotone coupling and the independent coupling associated with a collection of marginal distributions. Consequently, we show that, for every order α ∈ [0,∞], the Rényi entropy is subadditive on the majorization lattice. We further characterize the supermodular regime, showing that Rényi entropy is supermodular on the majorization lattice for α ∈ {0} ∪ [1,∞]. For the Tsallis entropy, we show that it also satisfies subadditivity on the majorization lattice, for every order α ∈ [0,∞). Finally, we show that, unlike the Rényi entropy, the Tsallis entropy is supermodular on the majorization lattice for every α ∈ [0,∞).
What carries the argument
The relation between the comonotone coupling and the independent coupling of marginal distributions, which directly yields the subadditivity of Rényi entropy on the lattice.
If this is right
- Rényi entropy of the lattice join is at most the sum of the individual entropies for any α.
- Supermodularity holds exactly when α is zero or at least one.
- Tsallis entropy is subadditive for every α and supermodular for every α.
- These lattice properties apply uniformly to the complete lattice of ordered probability distributions.
Where Pith is reading between the lines
- The subadditivity may supply new upper bounds when combining diversity measures from separate sources.
- The distinction between Rényi and Tsallis supermodularity regimes could affect which entropy is preferred in lattice-based optimization problems.
- The coupling relation might extend to other information measures that depend on joint distributions.
Load-bearing premise
The fundamental relation between the comonotone coupling and the independent coupling associated with a collection of marginal distributions.
What would settle it
A concrete collection of marginal distributions for which the Rényi entropy of their comonotone coupling fails to satisfy the inequality that would imply subadditivity on the lattice join.
Figures
read the original abstract
Majorization is a stochastic ordering relation that compares the relative diversity of probability distributions with numerous applications in econometrics, spectral theory, and ecology. It is well-known that the majorization partial order forms a complete lattice on the set of ordered probability distributions. In this work, we study the properties of R\'enyi entropy on the majorization lattice. We establish a fundamental relation between the comonotone coupling and the independent coupling associated with a collection of marginal distributions. Consequently, we show that, for every order $\alpha \in [0,\infty]$, the R\'enyi entropy is subadditive on the majorization lattice. We further characterize the supermodular regime, showing that R\'enyi entropy is supermodular on the majorization lattice for $\alpha \in \{0\} \,\cup \, [1,\infty]$. For the Tsallis entropy, we show that it also satisfies subadditivity on the majorization lattice, for every order $\alpha \in [0,\infty)$. Finally, we show that, unlike the R\'enyi entropy, the Tsallis entropy is supermodular on the majorization lattice for every $\alpha \in [0,\infty)$.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript claims that Rényi entropy is subadditive on the majorization lattice for every α ∈ [0,∞] by establishing a relation between the comonotone coupling and the independent coupling of marginal distributions; it further shows supermodularity on the lattice for α ∈ {0} ∪ [1,∞]. Parallel results are given for Tsallis entropy: subadditivity for all α ∈ [0,∞) and supermodularity for all α ∈ [0,∞).
Significance. If the central coupling relation holds, the work supplies a lattice-theoretic characterization of entropy functionals under majorization, distinguishing subadditive and supermodular regimes. This extends classical properties of entropy and may inform bounds in stochastic orders. The paper does not report machine-checked proofs or reproducible code.
major comments (1)
- [Section establishing the fundamental coupling relation] The derivation establishing the comonotone-independent coupling relation (invoked to prove subadditivity for the full interval α ∈ [0,∞]): for α < 1 the Rényi functional reverses monotonicity relative to the usual stochastic order, so the manuscript must explicitly verify that the inequality direction required for the join still holds; this step is load-bearing for the claim covering α < 1 and is not addressed by the abstract statement alone.
minor comments (2)
- The abstract states the supermodular regime for Rényi entropy but does not indicate whether the proof for α = 0 is handled separately from the α ≥ 1 case.
- Notation for the join operation on the majorization lattice should be introduced before its first use in the main results.
Simulated Author's Rebuttal
We thank the referee for the careful reading and for highlighting the need for explicit verification of the inequality direction in the coupling relation when α < 1. We address the major comment below.
read point-by-point responses
-
Referee: [Section establishing the fundamental coupling relation] The derivation establishing the comonotone-independent coupling relation (invoked to prove subadditivity for the full interval α ∈ [0,∞]): for α < 1 the Rényi functional reverses monotonicity relative to the usual stochastic order, so the manuscript must explicitly verify that the inequality direction required for the join still holds; this step is load-bearing for the claim covering α < 1 and is not addressed by the abstract statement alone.
Authors: We agree that the reversal of monotonicity for α < 1 requires an explicit check that the comonotone coupling still produces the correct inequality direction for subadditivity on the join. In the revised version we will insert a short dedicated paragraph immediately after the statement of the coupling relation. The paragraph will compute the relevant Rényi expressions for a pair of comonotone and independent couplings when 0 ≤ α < 1 and confirm that the inequality direction remains the one needed to bound the entropy of the join from above. revision: yes
Circularity Check
No circularity: derivations rest on standard coupling and lattice definitions
full rationale
The central claim establishes a relation between comonotone and independent couplings of marginals, then uses it to bound Rényi entropy on the join of the majorization lattice. This relation is invoked as a derived property of the couplings themselves (not defined via the entropy functional or fitted to data), and the subadditivity and supermodularity results follow from the lattice order and the functional properties of Rényi entropy. No self-citation chains, self-definitional equations, fitted inputs renamed as predictions, or ansatzes smuggled via prior work appear in the derivation chain. The argument is therefore self-contained against external benchmarks of coupling theory and majorization.
Axiom & Free-Parameter Ledger
axioms (2)
- domain assumption The majorization partial order forms a complete lattice on the set of ordered probability distributions.
- domain assumption Comonotone coupling and independent coupling are well-defined for any collection of marginal distributions.
Forward citations
Cited by 2 Pith papers
-
The Sharma-Mittal Entropy is Subadditive and Supermodular on the Majorization Lattice
Sharma-Mittal entropy is proven to be subadditive and supermodular on the majorization lattice of n-dimensional probability distributions.
-
The Sharma-Mittal Entropy is Subadditive and Supermodular on the Majorization Lattice
Sharma-Mittal entropy is proven subadditive and supermodular on the majorization lattice of n-dimensional probability distributions.
Reference graph
Works this paper leans on
-
[1]
Image thresholding using Tsallis entropy , url =
M. Image thresholding using Tsallis entropy , url =. Pattern Recognition Letters , keywords =. 2004 , bdsk-url-1 =. doi:https://doi.org/10.1016/j.patrec.2004.03.003 , issn =
-
[2]
On measures of entropy and information , author=. Proceedings of the fourth Berkeley symposium on mathematical statistics and probability, volume 1: contributions to the theory of statistics , volume=. 1961 , organization=
work page 1961
-
[3]
Nonextensive statistical mechanics and economics , url =
Constantino Tsallis and Celia Anteneodo and Lisa Borland and Roberto Osorio , doi =. Nonextensive statistical mechanics and economics , url =. Physica A: Statistical Mechanics and its Applications , keywords =. 2003 , bdsk-url-1 =
work page 2003
- [4]
-
[5]
Innovation Representation of Stochastic Processes With Application to Causal Inference , year=
Painsky, Amichai and Rosset, Saharon and Feder, Meir , journal=. Innovation Representation of Stochastic Processes With Application to Causal Inference , year=
-
[6]
Entropic Causal Inference , url =
Kocaoglu, Murat and Dimakis, Alexandros and Vishwanath, Sriram and Hassibi, Babak , doi =. Entropic Causal Inference , url =. Proceedings of the AAAI Conference on Artificial Intelligence , month =. 2017 , bdsk-url-1 =
work page 2017
-
[7]
Entropic Causal Inference: Graph Identifiability , url =
Compton, Spencer and Greenewald, Kristjan and Katz, Dmitriy A and Kocaoglu, Murat , booktitle =. Entropic Causal Inference: Graph Identifiability , url =. 2022 , bdsk-url-1 =
work page 2022
-
[8]
Acta Mathematica Academiae Paedagogicae Ny
On some properties of Tsallis entropy on majorization lattice , author=. Acta Mathematica Academiae Paedagogicae Ny
-
[9]
Yadav, Anuj Kumar and Shkel, Yanina Y. , booktitle=. Approximation Guarantees for Minimum Rényi Entropy Functional Representations , year=
-
[10]
C. Tsallis and A.R. Plastino and W.-M. Zheng , doi =. Power-law sensitivity to initial conditions---New entropic representation , url =. Chaos, Solitons & Fractals , number =. 1997 , bdsk-url-1 =
work page 1997
-
[11]
Possible generalization of Boltzmann-Gibbs statistics , url =
Tsallis, Constantino , date =. Possible generalization of Boltzmann-Gibbs statistics , url =. Journal of Statistical Physics , number =. 1988 , bdsk-url-1 =. doi:10.1007/BF01016429 , id =
-
[12]
Antolín, J. and López-Rosa, S. and Angulo, J. C. and Esquivel, R. O. , title =. The Journal of Chemical Physics , volume =. 2010 , month =. doi:10.1063/1.3298911 , url =
-
[13]
Jensen--Tsallis divergence for supervised classification under data imbalance , url =
Squicciarini, Antonio and Trigano, Tom and Luengo, David , date =. Jensen--Tsallis divergence for supervised classification under data imbalance , url =. Machine Learning , number =. 2025 , bdsk-url-1 =. doi:10.1007/s10994-025-06791-4 , id =
-
[14]
Jawad, Abdul and Bamba, Kazuharu and Younas, Muhammad and Qummer, Saba and Rani, Shamaila , doi =. Tsallis, R. Symmetry , number =. 2018 , bdsk-url-1 =
work page 2018
-
[15]
Jensen--Renyi's--Tsallis Fuzzy Divergence Information Measure with its Applications , url =
Kadian, Ratika and Kumar, Satish , date =. Jensen--Renyi's--Tsallis Fuzzy Divergence Information Measure with its Applications , url =. Communications in Mathematics and Statistics , number =. 2022 , bdsk-url-1 =. doi:10.1007/s40304-020-00228-1 , id =
-
[16]
Cicalese, F. and Vaccaro, U. , journal=. Supermodularity and subadditivity properties of the entropy on the majorization lattice , year=
-
[17]
Some simple inequalities satisfied by convex functions , author=. Messenger Math. , volume=
-
[18]
and Kumar Yadav, Anuj , booktitle=
Shkel, Yanina Y. and Kumar Yadav, Anuj , booktitle=. Information Spectrum Converse for Minimum Entropy Couplings and Functional Representations , year=
-
[19]
Information theoretic measures of distances and their econometric applications , year=
Cicalese, Ferdinando and Gargano, Luisa and Vaccaro, Ugo , booktitle=. Information theoretic measures of distances and their econometric applications , year=
-
[20]
Infinite Divisibility of Information , year=
Li, Cheuk Ting , journal=. Infinite Divisibility of Information , year=
-
[21]
Efficient Approximate Minimum Entropy Coupling of Multiple Probability Distributions , year=
Li, Cheuk Ting , journal=. Efficient Approximate Minimum Entropy Coupling of Multiple Probability Distributions , year=
-
[22]
Geometry of Rényi Entropy on the Majorization Lattice , author=. 2026 , journal=
work page 2026
-
[23]
An Information Theoretic Approach to Probability Mass Function Truncation , year=
Cicalese, Ferdinando and Gargano, Luisa and Vaccaro, Ugo , booktitle=. An Information Theoretic Approach to Probability Mass Function Truncation , year=
-
[24]
Minimum-Entropy Couplings and Their Applications , year=
Cicalese, Ferdinando and Gargano, Luisa and Vaccaro, Ugo , journal=. Minimum-Entropy Couplings and Their Applications , year=
-
[25]
A Tighter Approximation Guarantee for Greedy Minimum Entropy Coupling , year=
Compton, Spencer , booktitle=. A Tighter Approximation Guarantee for Greedy Minimum Entropy Coupling , year=
-
[26]
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics , pages =
Minimum-Entropy Coupling Approximation Guarantees Beyond the Majorization Barrier , author =. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics , pages =. 2023 , editor =
work page 2023
-
[27]
Parker, D. Stott and Ram, Prasad , doi =. The Construction of Huffman Codes is a Submodular ("Convex") Optimization Problem Over a Lattice of Binary Trees , url =. 1999 , bdsk-url-1 =. https://doi.org/10.1137/S0097539796311077 , journal =
-
[29]
Extremal elements of a sublattice of the majorization lattice and approximate majorization , url =
Massri, C and Bellomo, G and Holik, F and Bosyk, G M , doi =. Extremal elements of a sublattice of the majorization lattice and approximate majorization , url =. Journal of Physics A: Mathematical and Theoretical , month =. 2020 , bdsk-url-1 =
work page 2020
- [30]
-
[31]
R.B. Bapat , doi =. Majorization and singular values. III , url =. Linear Algebra and its Applications , pages =. 1991 , bdsk-url-1 =
work page 1991
-
[32]
Yeung, R.W. , journal=. A new outlook on Shannon's information measures , year=
-
[33]
L. Lovász , title=. Mathematical Programming The State of the Art , chapter=. 1983 , month=. doi:10.1007/978-3-642-68874-4_10 , url=
-
[34]
The Size of a Share Must Be Large , url =
Csirmaz, L. The Size of a Share Must Be Large , url =. Journal of Cryptology , number =. 1997 , bdsk-url-1 =. doi:10.1007/s001459900029 , id =
- [35]
-
[36]
Polymatroidal dependence structure of a set of random variables , url =
Satoru Fujishige , doi =. Polymatroidal dependence structure of a set of random variables , url =. Information and Control , number =. 1978 , bdsk-url-1 =
work page 1978
-
[37]
Rényi divergence and majorization , year=
van Erven, Tim and Harremoës, Peter , booktitle=. Rényi divergence and majorization , year=
-
[38]
Conditions for a Class of Entanglement Transformations , author =. Phys. Rev. Lett. , volume =. 1999 , month =. doi:10.1103/PhysRevLett.83.436 , url =
-
[39]
Fan, Yanqin and Henry, Marc and Pass, Brendan and Rivero, Jorge A. , title =. The Review of Economics and Statistics , pages =. 2024 , month =. doi:10.1162/rest_a_01539 , url =
-
[40]
Journal of the Royal Statistical Society Series A: Statistics in Society , volume =
Jorda, Vanesa and Sarabia, José María and Jäntti, Markus , title =. Journal of the Royal Statistical Society Series A: Statistics in Society , volume =. 2021 , month =. doi:10.1111/rssa.12702 , url =
-
[41]
Quantum majorization and a complete set of entropic conditions for quantum thermodynamics , url =
Gour, Gilad and Jennings, David and Buscemi, Francesco and Duan, Runyao and Marvian, Iman , date =. Quantum majorization and a complete set of entropic conditions for quantum thermodynamics , url =. Nature Communications , number =. 2018 , bdsk-url-1 =. doi:10.1038/s41467-018-06261-7 , id =
-
[42]
Butt, Saad Ihsan and Javed, Iram and Agarwal, Praveen and Nieto, Juan J. , date =. Newton--Simpson-type inequalities via majorization , url =. Journal of Inequalities and Applications , number =. 2023 , bdsk-url-1 =. doi:10.1186/s13660-023-02918-0 , id =
-
[43]
Palomar, Daniel P. and Jiang, Yi , title =. Foundations and Trends in Communications and Information Theory , volume =. 2007 , month =. doi:10.1561/0100000018 , url =
-
[44]
Matrix Majorization in Large Samples , year=
Farooq, Muhammad Usman and Fritz, Tobias and Haapasalo, Erkka and Tomamichel, Marco , journal=. Matrix Majorization in Large Samples , year=
-
[45]
and Nema, Aditya and Strelchuk, Sergii , date =
Elkouss, David and Maity, Ananda G. and Nema, Aditya and Strelchuk, Sergii , date =. Communications Physics , title =. 2026 , bdsk-url-1 =. doi:10.1038/s42005-026-02583-x , id =
- [46]
-
[47]
M. O. Lorenz , journal =. Methods of Measuring the Concentration of Wealth , urldate =
-
[48]
Sitzungsberichte der Berliner Mathematischen Gesellschaft , volume =
Schur, Issai , title =. Sitzungsberichte der Berliner Mathematischen Gesellschaft , volume =
-
[49]
Inequalities: Theory of Majorization and Its Applications , author=. 2010 , publisher=
work page 2010
- [50]
-
[51]
The Measurement of the Inequality of Incomes , urldate =
Hugh Dalton , journal =. The Measurement of the Inequality of Incomes , urldate =
-
[52]
J. Antolín, S. López-Rosa, J. C. Angulo, and R. O. Esquivel. Jensen–tsallis divergence and atomic dissimilarity for position and momentum space electron densities. The Journal of Chemical Physics , 132(4):044105, 01 2010
work page 2010
-
[53]
R.B. Bapat. Majorization and singular values. iii. Linear Algebra and its Applications , 145:59--70, 1991
work page 1991
-
[54]
Saad Ihsan Butt, Iram Javed, Praveen Agarwal, and Juan J. Nieto. Newton--simpson-type inequalities via majorization. Journal of Inequalities and Applications , 2023(1):16, 2023
work page 2023
-
[55]
On some properties of tsallis entropy on majorization lattice
PK Bhatia, Surender Singh, and Vinod Kumar. On some properties of tsallis entropy on majorization lattice. Acta Mathematica Academiae Paedagogicae Ny \' regyh \'a ziensis , 31(2):331--340, 2015
work page 2015
-
[56]
Entropic causal inference: Graph identifiability
Spencer Compton, Kristjan Greenewald, Dmitriy A Katz, and Murat Kocaoglu. Entropic causal inference: Graph identifiability. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning , volume 162 of Proceedings of Machine Learning Research , ...
work page 2022
-
[57]
Information theoretic measures of distances and their econometric applications
Ferdinando Cicalese, Luisa Gargano, and Ugo Vaccaro. Information theoretic measures of distances and their econometric applications. In 2013 IEEE International Symposium on Information Theory , pages 409--413, 2013
work page 2013
-
[58]
An information theoretic approach to probability mass function truncation
Ferdinando Cicalese, Luisa Gargano, and Ugo Vaccaro. An information theoretic approach to probability mass function truncation. In 2019 IEEE International Symposium on Information Theory (ISIT) , pages 702--706, 2019
work page 2019
-
[59]
Minimum-entropy couplings and their applications
Ferdinando Cicalese, Luisa Gargano, and Ugo Vaccaro. Minimum-entropy couplings and their applications. IEEE Transactions on Information Theory , 65(6):3436--3451, 2019
work page 2019
-
[60]
Minimum-entropy coupling approximation guarantees beyond the majorization barrier
Spencer Compton, Dmitriy Katz, Benjamin Qi, Kristjan Greenewald, and Murat Kocaoglu. Minimum-entropy coupling approximation guarantees beyond the majorization barrier. In Francisco Ruiz, Jennifer Dy, and Jan-Willem van de Meent, editors, Proceedings of The 26th International Conference on Artificial Intelligence and Statistics , volume 206 of Proceedings ...
work page 2023
-
[61]
A tighter approximation guarantee for greedy minimum entropy coupling
Spencer Compton. A tighter approximation guarantee for greedy minimum entropy coupling. In 2022 IEEE International Symposium on Information Theory (ISIT) , pages 168--173, 2022
work page 2022
-
[62]
The size of a share must be large
L \'a szl \'o Csirmaz. The size of a share must be large. Journal of Cryptology , 10(4):223--231, 1997
work page 1997
-
[63]
F. Cicalese and U. Vaccaro. Supermodularity and subadditivity properties of the entropy on the majorization lattice. IEEE Transactions on Information Theory , 48(4):933--938, 2002
work page 2002
-
[64]
The measurement of the inequality of incomes
Hugh Dalton. The measurement of the inequality of incomes. The Economic Journal , 30(119):348--361, 1920
work page 1920
-
[65]
Maity, Aditya Nema, and Sergii Strelchuk
David Elkouss, Ananda G. Maity, Aditya Nema, and Sergii Strelchuk. A finite sufficient set of conditions for catalytic majorization. Communications Physics , 2026
work page 2026
-
[66]
Matrix majorization in large samples
Muhammad Usman Farooq, Tobias Fritz, Erkka Haapasalo, and Marco Tomamichel. Matrix majorization in large samples. IEEE Transactions on Information Theory , 70(5):3118--3144, 2024
work page 2024
-
[67]
Yanqin Fan, Marc Henry, Brendan Pass, and Jorge A. Rivero. Multidimensional inequality measurement via optimal transport. The Review of Economics and Statistics , pages 1--45, 11 2024
work page 2024
-
[68]
Polymatroidal dependence structure of a set of random variables
Satoru Fujishige. Polymatroidal dependence structure of a set of random variables. Information and Control , 39(1):55--72, 1978
work page 1978
- [69]
-
[70]
The geometry of coalition power: Majorization, lattices, and displacement in multiwinner elections
Qian Guo, Yidan Hu, and Rui Zhang. The geometry of coalition power: Majorization, lattices, and displacement in multiwinner elections. arXiv preprint arXiv:2601.16723 , 2026
-
[71]
Quantum majorization and a complete set of entropic conditions for quantum thermodynamics
Gilad Gour, David Jennings, Francesco Buscemi, Runyao Duan, and Iman Marvian. Quantum majorization and a complete set of entropic conditions for quantum thermodynamics. Nature Communications , 9(1):5352, 2018
work page 2018
-
[72]
Some simple inequalities satisfied by convex functions
Godfrey H Hardy. Some simple inequalities satisfied by convex functions. Messenger Math. , 58:145--152, 1929
work page 1929
-
[73]
G.H. Hardy, J.E. Littlewood, and G. P \'o lya. Inequalities . Cambridge Mathematical Library. Cambridge University Press, 1952
work page 1952
-
[74]
Tsallis, r \'e nyi and sharma-mittal holographic dark energy models in loop quantum cosmology
Abdul Jawad, Kazuharu Bamba, Muhammad Younas, Saba Qummer, and Shamaila Rani. Tsallis, r \'e nyi and sharma-mittal holographic dark energy models in loop quantum cosmology. Symmetry , 10(11), 2018
work page 2018
-
[75]
Inequality measurement with grouped data: Parametric and non-parametric methods
Vanesa Jorda, José María Sarabia, and Markus Jäntti. Inequality measurement with grouped data: Parametric and non-parametric methods. Journal of the Royal Statistical Society Series A: Statistics in Society , 184(3):964--984, 07 2021
work page 2021
-
[76]
Extremal Combinatorics: With Applications in Computer Science
Stasys Jukna. Extremal Combinatorics: With Applications in Computer Science . Springer Publishing Company, Incorporated, 2nd edition, 2011
work page 2011
-
[77]
Murat Kocaoglu, Alexandros Dimakis, Sriram Vishwanath, and Babak Hassibi. Entropic causal inference. Proceedings of the AAAI Conference on Artificial Intelligence , 31(1), Feb. 2017
work page 2017
-
[78]
Jensen--renyi's--tsallis fuzzy divergence information measure with its applications
Ratika Kadian and Satish Kumar. Jensen--renyi's--tsallis fuzzy divergence information measure with its applications. Communications in Mathematics and Statistics , 10(3):451--482, 2022
work page 2022
-
[79]
Efficient approximate minimum entropy coupling of multiple probability distributions
Cheuk Ting Li. Efficient approximate minimum entropy coupling of multiple probability distributions. IEEE Transactions on Information Theory , 67(8):5259--5268, 2021
work page 2021
-
[80]
Infinite divisibility of information
Cheuk Ting Li. Infinite divisibility of information. IEEE Transactions on Information Theory , 68(7):4257--4271, 2022
work page 2022
-
[81]
M. O. Lorenz. Methods of measuring the concentration of wealth. Publications of the American Statistical Association , 9(70):209--219, 1905
work page 1905
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.