Pith. sign in

REVIEW 3 major objections 3 minor 56 references

Discrete Layered Entropy, Conditional Compression and a Tighter Strong Functional Representation Lemma

T0 review · 3 major / 3 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read For any pair of random variables, a common randomness variable $S$ exists that recovers $Y$ from $X$ and $S$ while adding fewer than 1.29 bits to the mutual information.

desk verdict A genuinely new information measure with a likely-correct but under-verified improvement to the strong functional representation lemma. read the letter →

arxiv 2501.13736 v4 pith:ODYOUD2G submitted 2025-01-23 cs.IT math.IT

Cheuk Ting Li This is my paper · ORCID
classification cs.ITmath.IT MSC 94A1794A2960E15
keywords discretelayeredentropystrongfunctionalrepresentationlemmaone-shotchannelsimulationconditionalcompressionnon-prefixcodesShannonapproximationmaximuminformationtheory
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces the discrete layered entropy $\Lambda$, defined by sorting a distribution's probabilities and weighting the $i$-th largest by the difference $i\log i-(i-1)\log(i-1)$. $\Lambda$ is piecewise linear, always lies between min-entropy and Shannon entropy, and approximates Shannon entropy within a logarithmic gap, which makes it useful in linear programming and maximum-entropy problems. Its key property is conditional: $\Lambda(X\setminus Y)=\Lambda(X|Y)$, so $\Lambda$ treats the conditional random variable $X|Y$ as a genuine object. The paper uses this to prove that for every pair $X,Y$ there is a common randomness $S$ independent of $X$ with $Y$ recoverable from $(X,S)$ and $\Lambda(Y|S)\le I(X;Y)+\Lambda(\mathrm{Geom}(1/2))

What carries the argument

The load-bearing object is the discrete layered entropy $\Lambda(p)=\sum_{i=1}^{\infty}p^{\downarrow}(i)(i\log i-(i-1)\log(i-1))$, the upper concave envelope of the conditional min-entropy. It is the unique function that satisfies the conditioning property $\Lambda(X\setminus Y)=\Lambda(X|Y)$ and equals $\log k$ for uniform $X$ over $k$ outcomes, and it is also the largest such function that never exceeds Shannon entropy. The proof of the strong functional representation lemma reduces to a one-variable analytic inequality $g(t)\le \Lambda(\mathrm{Geom}(1/2))+(t-1)g'(1)$ for $t>0$, where $g(t)=\Lambda(\mathrm{Geom}(t/(t+1)))+\log t$; the paper verifies this inequality by a rigorous case analysis using exact rational arithmetic and Sturm sequences.

What would settle it

Use exact rational or high-precision interval arithmetic to evaluate $g(t)=\Lambda(\mathrm{Geom}(t/(t+1)))+\log t$ for all $t>0$ and test whether $g(t)\le\Lambda(\mathrm{Geom}(1/2))+(t-1)g'(1)$ always holds; a counterexample would refute the main theorem. Alternatively, search over small finite alphabets for $X,Y$ and every candidate $S$ independent of $X$ with $H(Y|X,S)=0$, and check whether $\Lambda(Y|S)-I(X;Y)$ is always at least $1.29$; exceeding that for every $S$ would falsify Theorem 14.

Watch

Extended reading notes

Core claim

The central discovery is a strengthened strong functional representation lemma. For any (not necessarily discrete) random variables $X,Y$, there exists a random variable $S$, independent of $X$, such that $H(Y|X,S)=0$ and $\Lambda(Y|S)\leq I(X;Y)+\Lambda(\mathrm{Geom}(1/2))$, where the constant $\Lambda(\mathrm{Geom}(1/2))<1.29$ is the discrete layered entropy of the geometric distribution with parameter $1/2$. Consequently $H(Y|S)<I(X;Y)+\log(I(X;Y)+3.4)+1$, improving the previous best bound $I+\log(I+2)+2$ for every mutual information $I$. The paper also shows the optimal constant for the non-prefix channel-simulation task lies between $0.086$ and $1.29$, so the extra cost over mutual information is a genuine non-zero constant of information theory rather than an artefact of prefix-free coding.

Load-bearing premise

The entire improvement rests on the inequality that a particular function built from the geometric distribution never rises above its tangent line at $t=1$; if even one value of $t$ violates it, the $1.29$-bit and $\log(I+3.4)+1$ bounds do not follow, and the appendix's verification of that inequality includes a step that refers back to the same appendix.

Editorial extensions

If this is right

  • For every pair $X,Y$, one-shot channel simulation with unlimited common randomness can be done with a non-prefix description length below $I(X;Y)+1.29$ bits, and with a prefix-free description length below $I(X;Y)+\log(I(X;Y)+3.4)+1$.
  • These bounds beat the previous strongest functional representation lemma for all mutual information values; the prefix bound $I+\log(I+3.4)+1$ is tighter than $I+\log(I+2)+2$ for every $I$.
  • One-shot lossy source coding with prefix codes achieves expected length at most $R(D)+\log(R(D)+3.4)+3$, improving the earlier $R(D)+\log(R(D)+2)+4.01$.
  • The discrete layered entropy can replace Shannon entropy in maximum-entropy linear programs, and the resulting optimum is within a logarithmic gap of the true entropy optimum.
  • The exact optimal additive constant $c_n^*$ for non-prefix channel simulation is bracketed between $0.086$ and $1.29$, meaning the gap between mutual information and simulation cost is an intrinsic non-zero constant of entropy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the $I+\log(I+3.4)+1$ bound is nearly tight, then one-shot channel simulation is essentially characterized by mutual information plus a universal additive term, and pinning down the exact constant would close the remaining gap.
  • The same pattern of proving a non-prefix bound first and converting it to a prefix bound through $\Lambda$ could be carried over to other one-shot problems where prefix-freeness is an artificial constraint, such as privacy or randomness extraction.
  • The three-way closeness of $\Lambda(X|Y)$, $H(X|Y)$, and $H(X\setminus Y)$ suggests that in one-shot analyses conditional compression and conditional entropy can be interchanged up to logarithmic corrections, which may simplify future coding proofs.
  • Because $\Lambda$ is the largest conditioning-compatible underestimate of $H$, an analogous construction might yield similar approximations for Renyi or other entropies, with comparable linear-programming and simulation payoffs.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper introduces the discrete layered entropy Λ(p), a piecewise-linear approximation of Shannon entropy, and develops its properties: concavity, Schur concavity, approximate closeness to H(X) within a logarithmic gap, a conditioning property Λ(X|Y)=Λ(X\Y), and operational connections to one-to-one non-prefix codes and conditional compression. The central application is a strengthened strong functional representation lemma (Theorem 14): for arbitrary X,Y there exists S independent of X with H(Y|X,S)=0 and Λ(Y|S) ≤ I(X;Y)+Λ(Geom(1/2)) < I(X;Y)+1.29, yielding H(Y|S) < I(X;Y)+log(I(X;Y)+3.4)+1. The proof in Appendix J reduces the new constant to a one-variable analytic inequality (49), verified by a case analysis using Sturm sequences and exact rational arithmetic.

Significance. If the main bound is valid, it is a genuine improvement over the previous best SFRL constants ([3], [14], [15]) and demonstrates that discrete layered entropy is a useful technical tool for one-shot channel simulation and lossy compression. The paper also contains several elegant structural results, such as the conditioning property, the axiomatic characterization in Theorems 12–13, and the operational interpretation of Λ as the length of optimal non-prefix codes. Many of the supporting propositions are proved carefully in appendices. However, the headline numerical improvement is conditional on a computer-assisted proof that is not reproducible from the manuscript: the Sturm/SymPy certificates are not shipped, and a key step in Appendix J contains a self-reference. These issues are fixable but prevent the main theorem from being audited as printed.

major comments (3)
  1. [Appendix J, proof of Theorem 14] The chain leading to (47) begins with the sentence "By the same arguments as Appendix J," which appears inside Appendix J itself. This is a self-reference that makes the derivation of Λ(Y|S) ≤ Λ(K|S) circular as printed. The authors should replace this with a direct argument or cite a specific prior result (e.g., Propositions 3 or 10, or the Poisson functional representation lemma from [14]). This step is load-bearing because it connects the construction of K to the bound on Λ(Y|S).
  2. [Appendix J, cases 1, 3, 4 of the proof of (49)] The verification of the key inequality g(t) ≤ Λ(Geom(1/2)) + (t−1)g′(1) is delegated to Sturm-sequence checks performed with SymPy, but no code, worksheet, certificate, or explicit polynomial is provided. Since (49) is the load-bearing inequality for the I+1.29 bound and for the derived bounds (26)–(27), the main theorem is not independently auditable from the manuscript. The authors should provide the verification script or a complete certificate (e.g., sign patterns of the Sturm sequence or certified rational bounds) in a supplement or appendix.
  3. [Appendix J, case 4 and Eq. (52)] The assertion that d²g/dt² ≤ −0.013 on [0.975,1.025] via a Sturm computation with k=14, m=70 is also unshipped; without this concavity bound, the conclusion (49) on that interval does not follow. Additionally, the tail sum in (52), −2^{−m−1}(m²+2m+2), is stated without derivation. It is used to lower-bound g′(1) for t>1, so it should be derived (or a reference given) for the proof to be self-contained.
minor comments (3)
  1. [Theorem 15 proof] In the proof of Theorem 15, the displayed bound "H(Y|X) ≤ I + log(I + 3.4) + 0.99" should presumably read "H(Y|S)"; Theorem 14 gives a bound on H(Y|S), not on H(Y|X).
  2. [Section V-C] The text refers to "Proposition 9" when defining Λ(X) via Shannon entropy; the intended reference appears to be Theorem 9.
  3. [Appendix J, Eq. (54)] The rational upper bound in (54) is asserted to follow from the continued fraction bound (53), but the algebra is not shown; a brief derivation would help the reader verify that the case analysis indeed covers the stated interval.

Circularity Check

0 steps flagged · score 2.0 of 10

No constructional circularity: Theorem 14 is a genuine derivation that reduces to an analytic inequality proven by Sturm sequences; the self-referential Appendix J wording and same-author citations are auditability issues, not circular reductions.

full rationale

The derivation chain is not circular. Theorem 14 reduces the bound Λ*_n ≤ I + Λ(Geom(1/2)) to the analytic inequality (49), g(t) ≤ Λ(Geom(1/2)) + (t−1)g'(1) for all t>0; this inequality is proved by a case analysis with exact rational arithmetic and Sturm sequences, and its constants are evaluated from the definition of Λ rather than fitted to the theorem's conclusion. The intermediate steps (a)–(c) are exactly the previously proved Proposition 3 properties (Schur concavity, concavity, monotone linearity), so no step is defined in terms of the target bound. The invocation of [14] for the geometric K construction is load-bearing, but it is a prior published theorem, not the present conclusion, and is therefore independent support rather than a self-citation loop. The only textual oddity is in Appendix J, where the phrase 'By the same arguments as Appendix J' appears inside Appendix J itself; however, the following inequalities (a)–(c) are explicitly stated, so this is a proof-writing/auditability defect rather than a constructional circularity. The main caveat is auditability, not circularity: the proof of (49) delegates Cases 1, 3, and 4 to exact-rational Sturm computations with parameters such as (m,k) = (18,5), (20,8), and (70,14), and no verification script or certificate is shipped. If that inequality were false, the headline bound would fail, but that is a correctness/reproducibility risk, which the instructions direct not to count as circularity. No fitted parameter is renamed as a prediction, and no known empirical pattern is merely relabeled. Hence no circular step is exhibited, and the paper receives a low non-circularity score.

Assumptions & free parameters 1 free parameters · 2 assumptions · 1 invented entities

The central claim rests on one imported Poisson functional representation lemma and a numerically verified analytic inequality. Apart from the auxiliary eta, there are no parameters fitted to data; the main risk is the unshipped computational proof and the self-referential proof step.

free parameters (1)
  • eta in Theorem 14 = 0.77 in the final bound (optimizable in general)
    Auxiliary parameter converting the Lambda-bound into the H-bound; the theorem holds for every eta>0, and the 0.77 choice only makes the displayed constant nicer, so it is not load-bearing.
assumptions (2)
  • domain assumption Poisson functional representation construction: for every X,Y there exist S,K with S independent of X, H(Y|K,S)=0, H(K|X,S)=0, and K|X,Y ~ Geom(rho(x,y)) with rho(x,y) >= (2^iota(x;y)+1)^-1.
    Imported from prior work [14] and [23, Lemma 12]; this is the starting point of the proof of Theorem 14 in Appendix J and is a published external theorem, but it is also authored by the present paper's author.
  • ad hoc to paper The inequality g(t) <= Lambda(Geom(1/2)) + (t-1)g'(1) for all t>0, where g(t)=Lambda(Geom(t/(t+1)))+log t.
    This inequality is central to the bound E[g(2^-iota)] <= Lambda(Geom(1/2)). It is proved in Appendix J by interval casework and Sturm sequence checks, but the proof contains a self-referential 'Appendix J' step and the SymPy verification code is not shipped.
invented entities (1)
  • Discrete layered entropy Lambda(X) independent evidence
    purpose: A piecewise-linear entropy-like functional that lower-bounds Shannon entropy within a logarithmic gap and satisfies the conditioning property Lambda(X|Y)=Lambda(X\Y); used for LP approximations and the improved SFRL bound.
    This is a new mathematical object rather than a physical entity. Its independent evidence is the falsifiable mathematical consequences it generates, notably the approximation bound in Proposition 5 and the improved SFRL bound in Theorem 14, which are stated independently of the definition's motivation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Discrete Layered Entropy, Conditional Compression and a Tighter Strong Functional Representation Lemma." pith.science (2026). https://pith.science/paper/ODYOUD2G

@misc{pith2026250113736,
  author       = {Pith},
  title        = {Pith review of: Discrete Layered Entropy, Conditional Compression and a Tighter Strong Functional Representation Lemma},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ODYOUD2G}},
  note         = {Machine review of arXiv:2501.13736}
}
abstract

We study a quantity called discrete layered entropy, which approximates the Shannon entropy within a logarithmic gap. Compared to the Shannon entropy, the discrete layered entropy is piecewise linear, approximates the expected length of the optimal one-to-one non-prefix code, and satisfies an elegant conditioning property. These properties make it useful for approximating the Shannon entropy in linear programming and maximum entropy problems, studying the optimal length of conditional encoding, and bounding the entropy of monotonic mixture distributions. In particular, it can give a bound $I(X;Y)+\log(I(X;Y)+3.4)+1$ for the strong functional representation lemma which is optimal within $2.8$ bits, and significantly improves upon the best known bound.

Figures

Figures reproduced from arXiv: 2501.13736 by the authors.

Figure 1
Figure 1. Left: Contour plot of the Shannon entropy [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Top: The conditional encoding setting. Bottom: The one-shot channel simulation setting. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 6
Figure 6. This improved bound shows the usefulness of [PITH_FULL_IMAGE:figures/full_fig_p004_6.png] view at source ↗
Figures from the paper (6 more)
Figure 3
Figure 3. Figure 3: Top: Diagram of the conditional variable-length encoding setting, where the encoder writes a variable-length description [PITH_FULL_IMAGE:figures/full_fig_p010_3.png]
Figure 4
Figure 4. Figure 4: Illustration of R(X) = S pY |X {(H(X|Y ), H(X\Y ))} showing the extreme points (Λ(X), H(X)) and (H(X), H(X)). • For conditional prefix codes, ℓ ∗ c ≈ H(X|Y ): H(X|Y ) ≤ ℓ ∗ c < H(X|Y ) + 1. (18) • For unconditional prefix codes, ℓ ∗ u ≈ H(X\Y ): H(X\Y ) ≤ ℓ ∗ u < H(X\Y…
Figure 5
Figure 5. Figure 5: Top: Diagram of one-shot variable-length channel simulation, where the encoder writes a variable-length description [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Comparison between Theorem 14 and previous upper bounds [14], [15] on [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Contour plot of the discrete m-layered entropy Λ[m] (p) for m ∈ {1, 2, 3, 4, 5, ∞} and p : {1, 2, 3} → [0, 1] being a ternary probability mass function. The red points are the points where Λ[m] (p) = H(p). They are also the vertices of the polytope {(p, z) ∈ R3 × R : 0…
Figure 8
Figure 8. Figure 8: Contour plot of the discrete Rényi layered entropy [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 47 canonical work pages

  1. [3]

    Strong functional representation lemma and applications to coding theorems,

    C. T. Li and A. El Gamal, “Strong functional representation lemma and applications to coding theorems,” IEEE Transactions on Information Theory , vol. 64, no. 11, pp. 6967–6978, Nov 2018

    2018

  2. [14]

    A unified framework for one-shot achievability via the Poisson matching lemma,

    C. T. Li and V . Anantharam, “A unified framework for one-shot achievability via the Poisson matching lemma,” IEEE Transactions on Information Theory, vol. 67, no. 5, pp. 2624–2651, 2021

    2021

  3. [15]

    Pointwise redundancy in one-shot lossy compression via Poisson functional representation,

    C. T. Li, “Pointwise redundancy in one-shot lossy compression via Poisson functional representation,” in International Zurich Seminar on Information and Communication (IZS 2024) , 2024

    work page 2024

  4. [1]

    Randomized quantization with exact error distribution,

    M. Hegazy and C. T. Li, “Randomized quantization with exact error distribution,” in 2022 IEEE Information Theory Workshop (ITW) . IEEE, 2022, pp. 350–355

    work page 2022

  5. [2]

    Rejection-sampled universal quantization for smaller quantization errors,

    C. W. Ling and C. T. Li, “Rejection-sampled universal quantization for smaller quantization errors,” in 2024 IEEE International Symposium on Information Theory (ISIT), 2024, pp. 1883–1888

    work page 2024

  6. [4]

    A lower bound on the expected length of one-to-one codes,

    N. Alon and A. Orlitsky, “A lower bound on the expected length of one-to-one codes,” IEEE Transactions on Information Theory , vol. 40, no. 5, pp. 1670–1672, 1994

    work page 1994

  7. [5]

    New bounds on the expected length of one-to-one codes,

    C. Blundo and R. De Prisco, “New bounds on the expected length of one-to-one codes,” IEEE Transactions on Information Theory , vol. 42, no. 1, pp. 246–250, 1996

    work page 1996

  8. [6]

    Minimum expected length of fixed-to-variable lossless compression without prefix constraints,

    W. Szpankowski and S. Verdú, “Minimum expected length of fixed-to-variable lossless compression without prefix constraints,” IEEE Transactions on Information Theory, vol. 57, no. 7, pp. 4017–4025, 2011

    work page 2011

Show all 56 references
  1. [7]

    Information theory and statistical mechanics,

    E. T. Jaynes, “Information theory and statistical mechanics,” Physical review, vol. 106, no. 4, p. 620, 1957

    1957

  2. [8]

    A minimum information principle: Theory and practice,

    F. Snickars and J. W. Weibull, “A minimum information principle: Theory and practice,” Regional science and urban economics , vol. 7, no. 1-2, pp. 137–168, 1977

    1977

  3. [9]

    Guessing and entropy,

    J. L. Massey, “Guessing and entropy,” in Proceedings of 1994 IEEE International Symposium on Information Theory . IEEE, 1994, p. 204

    1994

  4. [10]

    An inequality on guessing and its application to sequential decoding,

    E. Arikan, “An inequality on guessing and its application to sequential decoding,” IEEE Transactions on Information Theory , vol. 42, no. 1, pp. 99–105, 1996

    1996

  5. [11]

    Asymptotics and non-asymptotics for universal fixed-to-variable source coding,

    O. Kosut and L. Sankar, “Asymptotics and non-asymptotics for universal fixed-to-variable source coding,” IEEE Transactions on Information Theory , vol. 63, no. 6, pp. 3757–3772, 2017

    2017

  6. [12]

    Entanglement-assisted capacity of a quantum channel and the reverse Shannon theorem,

    C. H. Bennett, P. W. Shor, J. Smolin, and A. V . Thapliyal, “Entanglement-assisted capacity of a quantum channel and the reverse Shannon theorem,” IEEE Transactions on Information Theory , vol. 48, no. 10, pp. 2637–2655, 2002

    2002

  7. [13]

    The communication complexity of correlation,

    P. Harsha, R. Jain, D. McAllester, and J. Radhakrishnan, “The communication complexity of correlation,” IEEE Transactions on Information Theory , vol. 56, no. 1, pp. 438–449, Jan 2010

    2010

  8. [16]

    Noiseless coding of correlated information sources,

    D. Slepian and J. K. Wolf, “Noiseless coding of correlated information sources,” IEEE Trans. Inf. Theory , vol. 19, no. 4, pp. 471–480, Jul. 1973

    1973

  9. [17]

    An inequality on entropy,

    R. McEliece and Z. Yu, “An inequality on entropy,” in Proceedings of 1995 IEEE International Symposium on Information Theory , 1995, pp. 329–. 33

    1995

  10. [18]

    On measures of entropy and information,

    A. Rényi, “On measures of entropy and information,” in Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics . The Regents of the University of California, 1961

    1961

  11. [19]

    Information measures and capacity of order α for discrete memoryless channels,

    S. Arimoto, “Information measures and capacity of order α for discrete memoryless channels,” Topics in information theory , 1977

    1977

  12. [20]

    Encoding tasks and rényi entropy,

    C. Bunte and A. Lapidoth, “Encoding tasks and rényi entropy,” IEEE Transactions on Information Theory , vol. 60, no. 9, pp. 5065–5076, 2014

    2014

  13. [21]

    Distributed task encoding,

    A. Bracher, A. Lapidoth, and C. Pfister, “Distributed task encoding,” in 2017 IEEE International Symposium on Information Theory (ISIT) . IEEE, 2017, pp. 1993–1997

    2017

  14. [22]

    Public vs private coin in bounded-round information,

    M. Braverman and A. Garg, “Public vs private coin in bounded-round information,” in International Colloquium on Automata, Languages, and Programming. Springer, 2014, pp. 502–513

    2014

  15. [23]

    Channel simulation: Theory and applications to lossy compression and differential privacy,

    C. T. Li, “Channel simulation: Theory and applications to lossy compression and differential privacy,” Foundations and Trends® in Communications and Information Theory , vol. 21, no. 6, pp. 847–1106, 2024. [Online]. Available: http://dx.doi.org/10.1561/0100000141

    2024 doi

  16. [24]

    Extended Gray–Wyner system with complementary causal side information,

    C. T. Li and A. El Gamal, “Extended Gray–Wyner system with complementary causal side information,” IEEE Transactions on Information Theory , vol. 64, no. 8, pp. 5862–5878, 2017

    2017

  17. [25]

    Coding of an information source having ambiguous alphabet and the entropy of graphs

    J. Korner et al. , “Coding of an information source having ambiguous alphabet and the entropy of graphs.” in 6th Prague conference on Information Theory, etc. Academia, Prague, 1971, pp. 411–425

    1971

  18. [26]

    From the information bottleneck to the privacy funnel,

    A. Makhdoumi, S. Salamatian, N. Fawaz, and M. Médard, “From the information bottleneck to the privacy funnel,” in 2014 IEEE Information Theory Workshop (ITW 2014). IEEE, 2014, pp. 501–505

    2014

  19. [27]

    A metric between probability distributions on finite sets of different cardinalities and applications to order reduction,

    M. Vidyasagar, “A metric between probability distributions on finite sets of different cardinalities and applications to order reduction,” IEEE Transactions on Automatic Control , vol. 57, no. 10, pp. 2464–2477, 2012

    2012

  20. [28]

    Entropic causal inference,

    M. Kocaoglu, A. G. Dimakis, S. Vishwanath, and B. Hassibi, “Entropic causal inference,” in Thirty-First AAAI Conference on Artificial Intelligence , 2017

    2017

  21. [29]

    Minimum-entropy couplings and their applications,

    F. Cicalese, L. Gargano, and U. Vaccaro, “Minimum-entropy couplings and their applications,” IEEE Transactions on Information Theory , vol. 65, no. 6, pp. 3436–3451, 2019

    2019

  22. [30]

    Efficient approximate minimum entropy coupling of multiple probability distributions,

    C. T. Li, “Efficient approximate minimum entropy coupling of multiple probability distributions,” IEEE Transactions on Information Theory , vol. 67, no. 8, pp. 5259–5268, 2021

    2021

  23. [31]

    A new outlook on Shannon’s information measures,

    R. W. Yeung, “A new outlook on Shannon’s information measures,” IEEE Transactions on Information Theory , vol. 37, no. 3, pp. 466–474, 1991

    1991

  24. [32]

    A logarithmic decomposition for information,

    K. J. Down and P. A. Mediano, “A logarithmic decomposition for information,” in 2023 IEEE International Symposium on Information Theory (ISIT) . IEEE, 2023, pp. 150–155

    2023

  25. [33]

    A Poisson decomposition for information and the information-event diagram,

    C. T. Li, “A Poisson decomposition for information and the information-event diagram,” in 2024 IEEE International Symposium on Information Theory (ISIT). IEEE, 2024, pp. 3189–3194

    2024

  26. [34]

    A. W. Marshall, I. Olkin, and B. C. Arnold, Inequalities: theory of Majorization and its Applications . New York, Dordrecht, Heidelberg, London: Springer, 2011

    2011

  27. [35]

    On channel simulation with causal rejection samplers,

    D. Goc and G. Flamich, “On channel simulation with causal rejection samplers,” in 2024 IEEE International Symposium on Information Theory (ISIT) , 2024, pp. 1682–1687

    2024

  28. [36]

    The redundancy of non-singular channel simulation,

    G. Flamich, S. M. Sriramu, and A. B. Wagner, “The redundancy of non-singular channel simulation,” arXiv preprint arXiv:2501.14053 , 2025

    2025

  29. [37]

    A. G. Wilson, Entropy in Urban and Regional Modelling: Retrospect and Prospect , ser. Monographs in Spatial and Environmental Systems Analysis. London: Pion, 1970. [Online]. Available: https://doi.org/10.4324/9780203142608

    1970 doi

  30. [38]

    Derivation of nested transport models within a mathematical programming framework,

    S. Brice, “Derivation of nested transport models within a mathematical programming framework,” Transportation Research Part B: Methodological , vol. 23, no. 1, pp. 19–28, 1989

    1989

  31. [39]

    An entropy-based modal split model,

    K. O. Jörnsten and J. T. Lundgren, “An entropy-based modal split model,” Transportation Research Part B: Methodological, vol. 23, no. 5, pp. 345–359, 1989

    1989

  32. [40]

    A note on solution of large sparse maximum entropy problems with linear equality constraints,

    J. Eriksson, “A note on solution of large sparse maximum entropy problems with linear equality constraints,” Mathematical Programming, vol. 18, pp. 146–154, 1980

    1980

  33. [41]

    Linear programming with entropic perturbation,

    S.-C. Fang and H.-S. J. Tsao, “Linear programming with entropic perturbation,” Zeitschrift für Operations Research , vol. 37, no. 2, pp. 171–186, 1993

    1993

  34. [42]

    Sinkhorn distances: Lightspeed computation of optimal transport,

    M. Cuturi, “Sinkhorn distances: Lightspeed computation of optimal transport,” in Advances in Neural Information Processing Systems , 2013, pp. 2292– 2300

    2013

  35. [43]

    An automated theorem proving framework for information-theoretic results,

    C. T. Li, “An automated theorem proving framework for information-theoretic results,” IEEE Transactions on Information Theory , vol. 69, no. 11, pp. 6857–6877, 2023

    2023

  36. [44]

    The undecidability of conditional affine information inequalities and conditional independence implication with a binary constraint,

    ——, “The undecidability of conditional affine information inequalities and conditional independence implication with a binary constraint,” IEEE Transactions on Information Theory , vol. 68, no. 12, pp. 7685–7701, 2022

    2022

  37. [45]

    First-order theory of probabilistic independence and single-letter characterizations of capacity regions,

    ——, “First-order theory of probabilistic independence and single-letter characterizations of capacity regions,” IEEE Transactions on Information Theory, vol. 69, no. 12, pp. 7584–7601, 2023

    2023

  38. [46]

    A method for the construction of minimum-redundancy codes,

    D. A. Huffman, “A method for the construction of minimum-redundancy codes,” Proceedings of the IRE , vol. 40, no. 9, pp. 1098–1101, 1952

    1952

  39. [47]

    Why the Shannon and Hartley entropies are ‘natural’,

    J. Aczél, B. Forte, and C. T. Ng, “Why the Shannon and Hartley entropies are ‘natural’,” Advances in applied probability , vol. 6, no. 1, pp. 131–146, 1974

    1974

  40. [48]

    The quantum reverse Shannon theorem and resource tradeoffs for simulating quantum channels,

    C. H. Bennett, I. Devetak, A. W. Harrow, P. W. Shor, and A. Winter, “The quantum reverse Shannon theorem and resource tradeoffs for simulating quantum channels,” IEEE Transactions on Information Theory , vol. 60, no. 5, pp. 2926–2959, May 2014

    2014

  41. [49]

    Adaptive greedy rejection sampling,

    G. Flamich and L. Theis, “Adaptive greedy rejection sampling,” in 2023 IEEE International Symposium on Information Theory (ISIT) . IEEE, 2023, pp. 454–459

    2023

  42. [50]

    Compression of sources of probability distributions and density operators,

    A. Winter, “Compression of sources of probability distributions and density operators,” arXiv preprint quant-ph/0208131 , 2002

    2002 arXiv

  43. [51]

    Pollard, A user’s guide to measure theoretic probability

    D. Pollard, A user’s guide to measure theoretic probability . Cambridge University Press, 2002, no. 8

    2002

  44. [52]

    A mathematical theory of communication,

    C. E. Shannon, “A mathematical theory of communication,” Bell system technical journal , vol. 27, no. 3, pp. 379–423, 1948

    1948

  45. [53]

    The complexity of nonuniform random number generation,

    D. E. Knuth and A. C. Yao, “The complexity of nonuniform random number generation,” Algorithms and Complexity: New Directions and Recent Results, pp. 357–428, 1976

    1976

  46. [54]

    Optimal redundancy in exact channel synthesis,

    S. M. Sriramu and A. B. Wagner, “Optimal redundancy in exact channel synthesis,” in 2024 IEEE International Symposium on Information Theory (ISIT), 2024, pp. 1913–1918

    2024

  47. [55]

    A. N. Khovanskii, The application of continued fractions and their generalizations to problems in approximation theory . Noordhoff Groningen, 1963

    1963

  48. [56]

    SymPy: symbolic computing in Python,

    A. Meurer, C. P. Smith, M. Paprocki, O. ˇCertík, S. B. Kirpichev, M. Rocklin, A. Kumar, S. Ivanov, J. K. Moore, S. Singh et al. , “SymPy: symbolic computing in Python,” PeerJ Computer Science, vol. 3, p. e103, 2017

    2017

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.