Reply to "Comment on `R\'enyi entropy yields artificial biases not in the data and incorrect updating due to the finite-size data' "

G. Baris Bagci; Thomas Oikonomou

REVIEW 3 major objections 3 minor 17 references

Reply to "Comment on `R\'enyi entropy yields artificial biases not in the data and incorrect updating due to the finite-size data' "

T0 review · 3 major / 3 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Rényi entropy violates the subset-independence axiom in maximum-entropy inference.

desk verdict The reply's new continuous example is built on incorrect subset posteriors—the q_S1 in Eq. (5) doesn't have the mean U_1 it claims—so the central demonstration of Rényi subset-independence violation does not hold up. read the letter →

arxiv 1908.02003 v1 pith:PBJWNSNI submitted 2019-08-06 cond-mat.stat-mech

Thomas Oikonomou , G. Baris Bagci This is my paper · ORCID

classification cond-mat.stat-mech PACS 05.20.-y02.50.Tt89.70.Cf

keywords RényientropymaximuminferenceShore-JohnsonaxiomssubsetindependenceartificialbiasTsallisq-exponentialdistributionLagrangemultipliers

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This reply defends the claim that using Rényi entropy as a maximum-entropy objective with linear constraints produces posterior distributions that violate the subset-independence axiom of consistent inference, meaning the inferred distribution for a subset inherits information from the whole data set that it should not. The authors exhibit a continuous two-subset example in which the Rényi maximum-entropy distributions do not satisfy the required identity, while the ordinary logarithmic entropy does. They also argue that the Comment's proposed corrections—a concavity-under-majorization criterion and a modified composition rule—are not fixes: the former misidentifies the admissible normalization range, and the latter fixes the functional form before any maximization, making entropy maximization redundant and introducing artificial bias. If the reply is right, any inference pipeline that maximizes Rényi entropy under ordinary linear constraints is unreliable for subset-consistent updating, and the standard logarithmic entropy is the unique consistent choice for such constraints.

What carries the argument

The load-bearing object is the Shore–Johnson subset-independence identity $q_D(x)=m(S_1)q_{S_1}(x)+m(S_2)q_{S_2}(x)$ with $m(S_i)=\int_{S_i}q_D(x)\,dx$, together with the distinction between choosing an entropy before maximization (the axiom stage) and comparing entropies after maximization. The paper's explicit check uses the continuous MaxEnt posteriors: ordinary exponentials for the logarithmic entropy, and $q$-exponentials $e_x^q=[1+(q-1)q^{-1}x]^{1/(q-1)}$ for Rényi entropy. The identity fails because the Rényi mixing coefficients in Eq. (6) contain $U$, the global mean, and this $U$-dependence is the mechanism that creates artificial subset bias. The reply also uses the normalization conditions $q<1$ and $q>\beta U/(1+\beta U)$ (and analogously for subset means) to reject $q>1$ values admitted by the Comment's concavity-under-majorization criterion.

What would settle it

Directly solve the constrained Rényi maximization for $D=[0,\infty)$ with mean $U$ without assuming the paper's $q$-exponential form, for example by numerical optimization over a fine grid, and test whether the resulting $q_D$, $q_{S_1}$, $q_{S_2}$ satisfy $q_D=m(S_1)q_{S_1}+m(S_2)q_{S_2}$ with $m(S_i)=\int_{S_i}q_D\,dx$. If the true maximizers satisfy the identity, or if the coefficients turn out not to depend on $U$, the central claim is refuted. A simpler check: determine whether Eq. (5) satisfies the Euler–Lagrange stationarity condition for Rényi entropy with the linear mean constraint at the stated $q$ values.

Watch

Extended reading notes

Core claim

The central claim is a violation, not a mathematical contradiction: when Rényi entropy is maximized under a linear mean constraint, the optimized distribution $q_D(x)$ on a set $D$ split into disjoint subsets $S_1$ and $S_2$ cannot be written as $q_D(x)=m(S_1)q_{S_1}(x)+m(S_2)q_{S_2}(x)$, where $m(S_i)$ is the probability mass the full optimum assigns to $S_i$. The paper computes the Rényi posteriors for $D=[0,\infty)$, $S_1=[0,1)$, $S_2=[1,\infty)$ in terms of $q$-exponentials, and finds that the coefficients $m(S_1)$ and $m(S_2)$ depend on the mean $U$ of the whole set $D$, not on the means of the subsets. Substituting the expressions into the axiom equation fails numerically at representative parameters ($\beta=0.1$, $q=0.55$), whereas the analogous logarithmic-entropy posteriors satisfy the identity exactly. The reply reads the Shore–Johnson axioms as pre-maximization requirements: they constrain which functional may be maximized in the first place, so an equivalence that holds only after maximization, as in the Comment, is not sufficient.

Load-bearing premise

The argument assumes that the $q$-exponential expressions in Eq. (5) are the true Rényi maximum-entropy distributions under the linear mean constraint and the stated normalization conditions; if another distribution maximizes Rényi entropy for those constraints, the $U$-dependence of the computed coefficients would not establish a violation of subset independence.

Editorial extensions

If this is right

Under linear mean constraints, Rényi-entropy maximum-entropy inference cannot be used for subset-consistent updating: the posterior for a part of the space depends on the global mean of the whole space.
The ordinary logarithmic (trace-form) entropy satisfies the subset-independence identity in the same example, so the paper supports the conclusion that for linear constraints it is the unique consistent choice.
The Comment's modified composition rule $g(p_{ij})=g(u_i)g(v_j)$ forces $g(x)\sim x^q$, so the distribution is fixed before maximization; this makes the entropy maximization procedure redundant rather than a derivation.
The admissible $q>1$ range for Rényi maximizers should be discarded because $q$ then encodes the data interval $x_{\max}$, making the deformation parameter data-dependent and not independent of the mathematical variable $x$.
Because Rényi entropy is monotonically related to Tsallis entropy yet only Rényi violates both subset and system independence under linear constraints, monotone equivalence does not preserve consistency properties.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same two-subset test could be run as a general diagnostic on any candidate entropy: compute the true maximizers and compare $m(S_1)q_{S_1}+m(S_2)q_{S_2}$ with $q_D$. This would convert the paper's single example into a family of falsifiable checks for deformed entropies.
Because the paper finds that mixing coefficients carry the global mean $U$, a practical finite-sample signature is that Rényi-maximum-entropy fits will shift when the sampled range expands, even if the underlying distribution is unchanged; the paper argues this conceptually but does not present sample-size simulations.
The reply's pre-maximization reading of the axioms suggests a general principle: two entropies that are monotonically related can receive different consistency verdicts, so ordering entropies by information content alone does not settle which one may be maximized for inference.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Quick take: this reply is worth reading if you're following the Rényi-versus-Shannon MaxEnt debate, but its new centerpiece does not work. The continuous example meant to show that Rényi entropy violates Shore-Johnson subset independence is constructed with the wrong subset posteriors.

What's actually new: the paper isolates JK's Lagrange-multiplier and composition-rule moves and objects to them. The observation that JK modify the multipliers to get their equivalence is fair, and the point that their composition rule makes the maximization redundant is also worth noting. These are legitimate criticisms, though they are extensions of the authors' earlier PRE work rather than fresh results.

The soft spot is load-bearing. In Eq. (5), the authors write q_S1 with the same beta as the global distribution and claim U_1 is its mean. It isn't. For their own numbers (q=0.55, beta=0.1, U=10), the global posterior gives m(S1) ≈ 0.365 and U_1 ≈ 0.43. Plugging U_1 into their q_S1 formula gives a distribution whose actual mean is about 0.48. So the q_S1 they test against Eq. (1) is not the Rényi MaxEnt posterior for the subset with mean U_1; it's just a member of the same family with the wrong parameter. The same issue afflicts q_S2. That means the violation they plot is an artifact of using distributions that don't satisfy the subset constraints. The conditional q_D/m(S1) would have the right mean and would be a legitimate candidate for the subset posterior, though whether it is the true MaxEnt posterior is itself the question. As it stands, the demonstration doesn't establish the violation.

The other announced support, the Livesey-Skilling criterion that only trace-form entropies satisfy subset independence, is asserted without proof here. That may be true, but the reader has to take it on faith.

Who gets value: people tracking the SJ-axioms literature. The reply usefully catalogues JK's moves, but the numerical core is not sound. It should be reviewed because the debate matters and the flaws are correctable in principle, but a referee would need to insist on honest subset maximizers before publication.

I would not cite this version.

Regards.

Referee Report

3 major / 3 minor

Summary. This manuscript is a reply to Jizba and Korbel's Comment on the authors' earlier claim that Rényi entropy yields artificial biases in maximum-entropy inference. The reply makes three main points: first, that Schur-concavity, as used by JK, does not correctly identify the normalizable range of the Rényi parameter q; second, that the Rényi entropy violates the Shore-Johnson subset-independence axiom, with a new continuous example comparing Rényi and Shannon posteriors; and third, that JK's modified composition rule for system independence makes the entropy-maximization procedure redundant and introduces biases. The paper concludes that only trace-form entropies satisfy subset independence per the Livesey-Skilling criterion, though it states that the proof of this criterion is deferred to a separate paper.

Significance. If the central claim is correct, the reply would substantially strengthen the case that Rényi entropy is unsuitable for maximum-entropy inference under linear constraints, directly rebutting JK's defense. The paper's explicit continuous example in Eqs. (5)-(6) and Fig. 1 is a concrete attempt to test the Shore-Johnson subset-independence axiom, and the identification of the dependence of the coefficients m(Si) on the whole-set mean U is a useful diagnostic. However, the central numerical demonstration is invalid as written because the subset posteriors in Eq. (5) are not the actual Rényi maximum-entropy distributions for the stated subset means, and the key Livesey-Skilling assertion is explicitly unproved in this reply. The significance is therefore conditional on correction of the demonstration and on resolution of the interpretive dispute over the SJ axioms.

major comments (3)

[3. Subset independence axiom, Eqs. (5)-(6)] The distributions qS1(x) and qS2(x) in Eq. (5) are not the Rényi maximum-entropy posteriors for the subsets S1 and S2 with means U1 and U2. For q=0.55 and β=0.1, using the value U1 ≈ 0.43 that follows from Eq. (6) and the global mean U=10, direct integration gives ∫0^1 x qS1(x) dx ≈ 0.49, not U1. The reason is that the parameter a=(1-q)β/q in Eq. (5) is inherited from the global problem; the true subset maximizer must have a subset-specific parameter determined by the constraint ∫x p = U1, which is generically different. Therefore the comparison of the l.h.s. and r.h.s. of Eq. (1) in Fig. 1b does not test the subset-independence axiom as the authors intend; the claimed violation is an artifact of using subset posteriors that do not satisfy the mean constraints stated in the paper.
[3. Subset independence axiom, final paragraph] The assertion that the Livesey-Skilling criterion shows that only trace-form entropies satisfy subset independence is explicitly stated to be outside the scope of this reply and deferred to a separate publication. This assertion is one of the main conclusions of the paper (item (ii) in the summary), and the reply does not provide the promised proof or a reference to an existing proof. Without this support, the paper's central claim is not established, and the conclusion should be weakened or the proof included.
[4. System independence axiom] The criticism that JK 'are forced to change the Lagrange multipliers' is not a technical demonstration of an error. In the standard maximum-entropy formalism, Lagrange multipliers are determined by the constraints and are allowed to differ when the domain or the constraint set changes. Requiring the same β for the whole set and for subsets is a strong conditional-preservation property that is not a general requirement of the maximum-entropy method. The reply therefore does not substantively rebut JK's treatment of system independence; it merely restates a preference for a particular interpretation of how multipliers should behave.

minor comments (3)

[1. Uniqueness axiom] The claim that q > 1 values are 'irrelevant' because 'the deformation parameter carries information about x' conflates the domain of the random variable with finite-size data; the normalization bound 0 < q < 1 + 1/xmax depends on the domain, not on the data set, so this argument needs clarification.
[3. Subset independence axiom, Eq. (5)] The notation for the q-exponential, ex_q := [1+(q-1)q^{-1}x]^{1/(q-1)}, uses x both as the function argument and as a subscript, which is confusing; consider writing exp_q(x) for clarity.
[3. Subset independence axiom, Fig. 1] The values of U, U1, and U2 used in the figure are not given explicitly; the text only states that β=0.1 and q=0.55 were chosen, so the reader cannot reproduce the plot without additional computation.

Circularity Check

0 steps flagged · score 2.0 of 10

No construction-level circularity: the subset-independence counterexample is a direct calculation, but the reply leans on the authors' own prior papers and defers the Livesey-Skilling proof.

full rationale

The central check in Eqs. (1)-(6) is not circular. The paper inserts the stated Shannon and Rényi maximum-entropy posteriors into the Shore-Johnson subset-independence equality, computes the coefficients from Eq. (2), and verifies numerically whether Eq. (1) holds. There is no fitted parameter renamed as a prediction: the means U, U1, U2 are properties of the chosen posteriors, and the claimed violation is a computed consequence of those formulas. The Shannon case is a positive control, not an input. The principal weakness is evidentiary rather than definitional: the reply opens by presenting the violation as already shown in Ref. [2] (same authors), Eq. (5)'s q-exponential forms are asserted without derivation in this reply, and the paper concedes that the Livesey-Skilling justification "will be demonstrated by the current authors separately." Those self-referential supports reduce the independent weight of the reply, but they do not make Eq. (1) hold or fail by construction. Since the core derivation is self-contained algebra once Eq. (5) is granted, this is at most a minor self-citation concern, not a circularity in the derivation chain.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No invented entities. The reply's claims rest on the SJ axiomatic framework, on a contested pre-maximization interpretation, and on two deferred assertions. The numerical example uses arbitrary beta=0.1 and q=0.55; the violation is meant to be generic in q.

free parameters (2)

beta (inverse temperature) = 0.1
Chosen arbitrarily for the numerical example in Fig. 1; the violation argument is not fitted to this value.
q (Renyi parameter) = 0.55
Chosen within the admissible range for the numerical example; the argument is meant to hold across the allowed q interval.

assumptions (4)

domain assumption Shore-Johnson axioms should be applied at the pre-maximization stage, i.e., to the functional being maximized, before maximization.
This interpretive premise underlies the rejection of JK's post-maximization equivalence argument; the comment disputes it. Location: Section 3.
domain assumption For q>1, the Renyi parameter is not independent of the support bound xmax, so q>1 values are invalid in the MaxEnt context.
Used to dismiss Schur-concavity for q>1; stated with a normalization inequality but not rigorously derived here. Location: Section 1.
ad hoc to paper Only trace-form entropies satisfy the subset-independence axiom via the Livesey-Skilling criterion.
Asserted without proof and explicitly deferred to a future paper. Location: Section 3.
domain assumption For linearly averaged constraints, the Shannon entropy is the unique consistent choice.
Stated in the final paragraph as a conclusion, but no derivation is provided in this reply. Location: final paragraph before 'To sum up'.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Reply to "Comment on `R\'enyi entropy yields artificial biases not in the data and incorrect updating due to the finite-size data' "." pith.science (2026). https://pith.science/paper/PBJWNSNI

@misc{pith2026190802003,
  author       = {Pith},
  title        = {Pith review of: Reply to "Comment on `R\'enyi entropy yields artificial biases not in the data and incorrect updating due to the finite-size data' "},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PBJWNSNI}},
  note         = {Machine review of arXiv:1908.02003}
}

read the original abstract

We reply to the Comment by Jizba and Korbel [arXiv:1905.00729v1] by first pointing out that the Schur-concavity proposed by them falls short of identifying the correct intervals of normalization for the optimum probability distribution even though normalization is a must ingredient in the entropy maximization procedure. Secondly, their treatment of the subset independence axiom requires a modification of the Lagrange multipliers one begins with thereby rendering the optimization less trustworthy. We also explicitly demonstrate that the R\'enyi entropy violates the subset independence axiom and compare it with the Shannon entropy. Thirdly, the new composition rule offered by Jizba and Korbel are shown to yield probability distributions even without a need for the entropy maximization procedure at the expense of creating artificial bias in the data.

Figures

Figures reproduced from arXiv: 1908.02003 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 17 canonical work pages

[1]

Uniqueness axiom : In our assessment of this axiom, we have used the concavity as a criterion which led us to the q interval being (0 , 1). Inspecting the R´ enyi maxi- mum distribution one can see that for a variable interval x ∈ (xmin, x max) with a proper xmin the distribution is normalizable for 0 < q < 1 + ( xmax)− 1. Apparently, in this inequality w...

work page
[2]

Oikonomou and G.B

T. Oikonomou and G.B. Bagci, Phys. Rev. E 99 (2019), 032134

work page 2019
[3]

The crux of their argument can be traced back to the idea that the maximization of f (∑ i g(pi)) should yield the same result as ∑ i g(pi)

Subset independence axiom : JK, contrary to our opinion, argue that the R´ enyi entropy satisﬁes this ax- iom. The crux of their argument can be traced back to the idea that the maximization of f (∑ i g(pi)) should yield the same result as ∑ i g(pi). This certainly looks ∗ Electronic address: thomas.oikonomou@nu.edu.kz correct prima facie when one conside...

work page
[4]

strong system independence is added to the SJ desiderata

System independence axiom : First of all, JK in this part of their comment compare our discussion with their modiﬁed SJ axiom in Ref. [ 5] (“... strong system independence is added to the SJ desiderata”) and not with the original SJ axiom in [ 3]. In this sense their comment is irrelevant for our discussion in [ 2]. Albeit, let us closely inspect the modi...

work page 2018
[5]

( 1) we verify that the latter is not satisﬁed

and ( 6) into Eq. ( 1) we verify that the latter is not satisﬁed. For the reader’s convenience we demonstrate graphically our results. In Figs. 1a) and 1b) we plot the l.h.s and r.h.s. of Eq. ( 1) for the Shannon and R´ enyi entropies, respectively, for the randomly chosen values β = 0 . 1 and q = 0 . 55 (normalization conditions taken into account). /s32...

work page
[6]

Comment on "R\'enyi entropy yields artifficial biases not in the data and incorrect updating due to the infinite-size data"

P. Jizba and J. Korbel, arXiv:1905.00729v1, Accepted in Phys. Rev. E (2019)

work page Pith review arXiv 2019
[7]

J. E. Shore and R. W. Johnson, IEEE Trans. Inf. Theory IT-26, 26 (1980)

work page 1980
[8]

Hence, it does not impose any restrictions on the quantum information theoretical applications of this en- tropy as long as entropy optimization is not included

Our result is valid whenever one optimizes the R´ enyi en- tropy. Hence, it does not impose any restrictions on the quantum information theoretical applications of this en- tropy as long as entropy optimization is not included

work page

Show all 17 references

[9]

Jizba and J

P. Jizba and J. Korbel, Phys. Rev. Lett. 122 (2019), 120601

2019
[10]

Press´ e, K

S. Press´ e, K. Ghosh, J. Lee and K. A. Dill, Entropy 17 (2015), 5043

2015
[11]

Press´ e, K

S. Press´ e, K. Ghosh, J. Lee and K. A. Dill, Phys. Rev. Lett. 111 (2013), 180604

2013
[12]

Karabulut, Eur

H. Karabulut, Eur. J. Phys. 27 (2006), 709

2006
[13]

Oikonomou and G.B

T. Oikonomou and G.B. Bagci, Phys. Lett. A 381 (2017), 207

2017
[14]

S. Abe, S. Mart ´ ınez, F. Pennini and A. Plastino, Phys. 4 Lett. A 281 (2001) 126; S. Abe, Physica A 368 (2006) 430

2001
[15]

Plastino, M

A. Plastino, M. C. Rocca and F. Pennini, Phys. Rev. E 94 (2016) 012145

2016
[16]

Hanel and S

R. Hanel and S. Thurner, Eur. Phys. Lett. 93 (2011), 20006

2011
[17]

Oikonomou and G.B

T. Oikonomou and G.B. Bagci, Phys. Rev. E 97 (2018), 012104

2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Uniqueness axiom : In our assessment of this axiom, we have used the concavity as a criterion which led us to the q interval being (0 , 1). Inspecting the R´ enyi maxi- mum distribution one can see that for a variable interval x ∈ (xmin, x max) with a proper xmin the distribution is normalizable for 0 < q < 1 + ( xmax)− 1. Apparently, in this inequality w...

work page

[2] [2]

Oikonomou and G.B

T. Oikonomou and G.B. Bagci, Phys. Rev. E 99 (2019), 032134

work page 2019

[3] [3]

The crux of their argument can be traced back to the idea that the maximization of f (∑ i g(pi)) should yield the same result as ∑ i g(pi)

Subset independence axiom : JK, contrary to our opinion, argue that the R´ enyi entropy satisﬁes this ax- iom. The crux of their argument can be traced back to the idea that the maximization of f (∑ i g(pi)) should yield the same result as ∑ i g(pi). This certainly looks ∗ Electronic address: thomas.oikonomou@nu.edu.kz correct prima facie when one conside...

work page

[4] [4]

strong system independence is added to the SJ desiderata

System independence axiom : First of all, JK in this part of their comment compare our discussion with their modiﬁed SJ axiom in Ref. [ 5] (“... strong system independence is added to the SJ desiderata”) and not with the original SJ axiom in [ 3]. In this sense their comment is irrelevant for our discussion in [ 2]. Albeit, let us closely inspect the modi...

work page 2018

[5] [5]

( 1) we verify that the latter is not satisﬁed

and ( 6) into Eq. ( 1) we verify that the latter is not satisﬁed. For the reader’s convenience we demonstrate graphically our results. In Figs. 1a) and 1b) we plot the l.h.s and r.h.s. of Eq. ( 1) for the Shannon and R´ enyi entropies, respectively, for the randomly chosen values β = 0 . 1 and q = 0 . 55 (normalization conditions taken into account). /s32...

work page

[6] [6]

Comment on "R\'enyi entropy yields artifficial biases not in the data and incorrect updating due to the infinite-size data"

P. Jizba and J. Korbel, arXiv:1905.00729v1, Accepted in Phys. Rev. E (2019)

work page Pith review arXiv 2019

[7] [7]

J. E. Shore and R. W. Johnson, IEEE Trans. Inf. Theory IT-26, 26 (1980)

work page 1980

[8] [8]

Hence, it does not impose any restrictions on the quantum information theoretical applications of this en- tropy as long as entropy optimization is not included

Our result is valid whenever one optimizes the R´ enyi en- tropy. Hence, it does not impose any restrictions on the quantum information theoretical applications of this en- tropy as long as entropy optimization is not included

work page