REVIEW 4 major objections 3 minor 1 cited by
On the entropy growth of sums of iid discrete random variables
T0 review · 4 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read The paper proves that the Shannon entropy of the sum of N iid discrete random variables is bounded below by (r(X)/2) log N plus a constant, where r(X) is a new invariant called the incommensurability rank.
desk verdict A plausible generalization of entropy growth bounds, but the abstract leaves the key invariant r(X) undefined and possibly infinite for some 'arbitrary' discrete variables, so the proof needs scrutiny before I'd trust it. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The incommensurability rank r(X): a positive integer attached to the distribution, informally the number of rationally independent step sizes in the support. It carries the argument by reducing an arbitrary discrete sum to an r-dimensional lattice-like structure, where the known entropy growth of lattice and multinomial sums can be applied. This rank, rather than the detailed probabilities, determines the logarithmic lower bound.
What would settle it
Take X with support {0, 1, α} where α is irrational (e.g., α=√2), so r(X)=2, and compute H(S_N) for growing N. If the ratio H(S_N)/log N converges to a value strictly below 1, the claimed lower bound H ≥ (r/2) log N with r=2 fails for that distribution.
Extended reading notes
Core claim
The central claim is that for any iid discrete random variable X, the entropy of the sum S_N satisfies H(S_N) ≥ (r(X)/2) log N + c, where r(X) is the incommensurability rank introduced in the paper. The rank is a positive integer that measures the number of rationally independent step sizes in the support of X; the cases r(X)=1 correspond to lattice distributions and multinomial distributions, whose asymptotic entropy is already known. The derivation does not rely on central limit theorems but builds on those known r=1 expressions. In essence, the paper asserts that the logarithmic growth rate of entropy is a purely algebraic invariant of the distribution's support, independent of the fine-g
Load-bearing premise
The argument depends on the premise that the incommensurability rank r(X) completely determines the logarithmic growth rate of the sum's entropy, so that no other feature of the distribution—such as tail probabilities or the precise probabilities on the support—can change the exponent.
Editorial extensions
If this is right
- For every fixed discrete distribution, the entropy of the N-fold sum grows at least as (r(X)/2) log N, so higher incommensurability rank forces faster information growth.
- The known r=1 results for lattice and multinomial sums are recovered as special cases, giving a unified statement across previously separate settings.
- Because the bound avoids central limit theorems, it applies to heavy-tailed and otherwise non-classical discrete distributions.
- The rank depends only on the support, so the logarithmic growth rate can be predicted without computing the distribution of the sum.
Reading between the lines
- If the same bound holds for Rényi entropies, the incommensurability rank could become a common complexity measure across entropy orders, not just Shannon entropy.
- For a concrete distribution such as X uniform on {0, 1, √2}, one expects H(S_N) to grow as log N; this is a testable numerical prediction of the paper's claim.
- The paper proves only the lower direction; the true entropy may be asymptotically (r/2) log N + O(1), but establishing the matching upper bound for arbitrary distributions would require additional arguments.
- A similar invariant could be defined for independent but not identically distributed variables, or for sums of continuous variables on tori, suggesting a broader algebraic theory of entropy growth.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims to establish an asymptotic lower bound on the Shannon entropy of sums of N iid discrete random variables: H ≥ (r(X)/2) log N + cst, where r(X) is a newly introduced positive integer called the incommensurability rank. The authors state that the proof does not rely on central limit theorems but instead builds on known entropy asymptotics for multinomial distributions and sums of iid lattice random variables, both corresponding to r(X)=1. The review is based on the abstract only; the full text, including the definition of r(X) and the derivation, is not available.
Significance. If the result holds, it would provide a clean structural invariant—the incommensurability rank—governing the logarithmic entropy growth for arbitrary discrete iid sums, unifying and extending the known lattice and multinomial cases. The claimed avoidance of CLT arguments and the connection to additive structure of the support are potentially interesting. However, because the definition of r(X) and the proof are not accessible, the significance is conditional on the missing details being correct.
major comments (4)
- [Abstract (definition of r(X))] The central bound H ≥ (r(X)/2) log N + cst is stated for arbitrary iid discrete random variables, but r(X) is never defined in the abstract. The claim that it is a positive integer is not enough to make the theorem well posed. If r(X) is intended as the rank of the additive subgroup generated by the support, then for a distribution with countably infinite rationally independent support (e.g., weights on {0} ∪ {√p : p prime} with finite entropy), r(X) is infinite, making the bound either undefined or vacuous. The paper must give the precise definition of r(X), state the class of distributions for which it is finite, and prove finiteness for that class.
- [Abstract (proof of r>1 case)] The abstract builds on the r=1 cases (multinomial and lattice sums) but claims a result for all positive integer values of r. The extension from r=1 to general r is the load-bearing step, and no proof sketch is provided. The known r=1 results are independent and give some plausibility, but they do not by themselves establish the general bound. The missing derivation for r>1 must be supplied or referenced in a verifiable way.
- [Abstract (bound statement)] The constant 'cst' is not specified; in particular it is unclear whether it depends on X, on r(X), or on both, and whether the inequality is meant for all N or only asymptotically large N. The bound as written is not falsifiable without this specification. It should be clarified, for example as 'there exists a constant C(X) such that for all N ≥ N_0, H ≥ (r(X)/2) log N + C(X)'.
- [Abstract (circularity concern)] Since r(X) is introduced in the same paper specifically to parameterize this bound, there is a risk that the result is tailored: any positive-integer-valued function of X could be defined so that the inequality holds. The paper should anchor r(X) in an independent additive-combinatorial definition (e.g., the rank of the group generated by the support) and show that the bound is genuinely informative, not merely a restatement of the definition.
minor comments (3)
- [Abstract] The phrase 'arbitrary iid discrete random variables' should be qualified: do the variables need finite entropy, finite expected log-weight, or any moment condition? The abstract currently leaves this open.
- [Abstract] Please provide references for the 'known expressions of the asymptotic entropy of the multinomial distribution and sums of iid lattice random variables' so the reader can verify the r=1 base cases.
- [Abstract] The notation 'cst' is nonstandard; consider using 'O(1)' or a named constant with explicit dependencies.
Circularity Check
No circularity identifiable from the abstract; the claim rests on known r=1 results and an unexhibited definition of r(X).
full rationale
Based on the abstract alone, the derivation chain is not shown to reduce to its own inputs. The stated inputs are the known asymptotic entropy expressions for the multinomial distribution and sums of iid lattice random variables, which are external results and correspond to r(X)=1. The bound H ≥ (r(X)/2) log N + cst is presented in terms of a newly introduced quantity r(X), but the abstract does not define r(X) or provide equations, so there is no quoted evidence that r(X) is defined in terms of the entropy growth it purports to predict, nor that the r>1 argument is a renamed version of the r=1 results. The skeptic's concern that r(X) may be undefined or infinite for some discrete distributions is a well-definedness or correctness issue, not a circularity, and without the full text it cannot be evaluated. No self-citation is load-bearing because the only cited results are described as known external expressions. Therefore no circular step can be exhibited, and the honest finding is no significant circularity.
Assumptions & free parameters
free parameters (1)
- cst =
unspecified
assumptions (3)
- standard math Known asymptotic entropy of sums of iid lattice random variables
- standard math Known asymptotic entropy of the multinomial distribution
- domain assumption Incommensurability rank r(X) is well-defined for every discrete random variable X
invented entities (1)
-
Incommensurability rank r(X)
Cite this review
Pith. "Pith review of On the entropy growth of sums of iid discrete random variables." pith.science (2026). https://pith.science/paper/ANYKMYTI
@misc{pith2026250805348,
author = {Pith},
title = {Pith review of: On the entropy growth of sums of iid discrete random variables},
year = {2026},
howpublished = {\url{https://pith.science/paper/ANYKMYTI}},
note = {Machine review of arXiv:2508.05348}
}
abstract
We derive an asymptotic lower bound on the Shannon entropy $H$ of sums of $N$ arbitrary iid discrete random variables. The derived bound $H \geq \frac{r(X)}{2}\log(N) + {\it cst}$ is given in terms of the incommensurability rank $r(X)$ of the random variable -- a positive integer quantity that we introduce. The derivation does not rely on central limit theorems, but builds upon the known expressions of the asymptotic entropy of the multinomial distribution and sums of iid lattice random variables, which correspond to the case $r(X)=1$.
Forward citations
Cited by 1 Pith paper
-
The entropic coherence is a necessary resource for non-energy preserving gates
Entropic coherence of a battery, the relative entropy between its state and its energy-dephased version, must grow at least as the logarithm of the desired gate precision, making finite-dimensional batteries fundament...
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.