REVIEW 2 major objections 4 minor 30 references
Semantic Rate-Distortion Theory with Applications
T0 review · 2 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read This paper establishes a rate limit for semantic compression and derives a closed form for binary sources.
desk verdict The central equivalence theorem rests on an invalid max/expectation interchange; the binary example makes the failure concrete. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the information semantic rate-distortion function RI(Dp,Do) = min_{p_{Y|X}: Edp(pS|X,pS|Y)<=Dp, Edo(X,Y)<=Do} I(X;Y), defined with a conditional semantic probability distortion dp and an observation distortion do. The proof machinery uses the Poisson representation lemma to construct codes for the achievability direction (Theorem 1), a standard converse with a lower-semicontinuity condition for Theorem 2, and symmetry reduction plus calculus for the binary closed form (Theorem 3). The key conceptual mechanism is that semantic fidelity is measured at the level of the posterior distribution p_{S|X} versus p_{S|Y}, not by a point estimate of meaning, which lets ambiguity a
What would settle it
Set n=2 for a binary source with Hamming observation distortion and Do=0.25. Under the paper's Definition 3, E[max_i d_H(X_i,Y_i)] <= 0.25 forces the two symbols to be reproduced exactly with probability at least 0.75, so the operational rate per symbol must be at least about 0.59 bits; the paper's symbolic-only formula gives 1-h2(0.25) ≈ 0.189 bits per symbol, which is impossible for the operational problem.
Extended reading notes
Core claim
The central claim is that the minimum rate at which a semantic source can be compressed, subject to a budget on how much the receiver's inferred semantic distribution p_{S|Y} can differ from the transmitter's p_{S|X}, is the solution of a single-letter optimization problem. Theorems 1 and 2 together state RI(Dp,Do) = RO(Dp,Do) when RI is lower semicontinuous, so the information-theoretic quantity is the true operational limit. Theorem 3 evaluates this limit for the binary doubly symmetric case: with C=|1-2q| and a(Do) as defined, R(Dp,Do) equals 1 - h2((1 - sqrt(1 - 2Dp/C))/2) for Dp <= a(Do), and equals the classical 1 - h2(min{Do,1/2}) above that threshold. The idea is that semantic ambigu
Load-bearing premise
The load-bearing assumption is that sequence-level distortion can be controlled component-wise—specifically, that E[max_i dp(p_{S_i|X_i}, p_{S_i|Y_i})] equals max_i E[dp(...)] and likewise for the observation distortion; without this interchange, the operational rate-distortion function may not equal the information-theoretic function computed in the paper.
Editorial extensions
If this is right
- If RI=RO, then the operational semantic rate-distortion region is characterized by a single-letter optimization, and any code sequence achieving RI is optimal up to vanishing rate loss.
- For the binary doubly symmetric case, the closed form shows that for small semantic distortion Dp the rate is set entirely by Dp, while for larger Dp it degenerates to the classical Hamming rate-distortion function; the threshold a(Do) governs the switch.
- The reported experiments imply that adding a conditional-semantic-distortion penalty can yield large accuracy gains at the same bit rate, e.g., from about 62% to about 94% at 12 bits in the paper's digit-recognition setup.
- When both semantic and symbolic fidelity are required, the two constraints can be tuned independently, enabling systems that preserve task meaning while limiting observable distortion.
Reading between the lines
- Editorial inference: The paper's sequence-level distortion is defined as the maximum per-component distortion, which is stronger than the usual average distortion; under an average-distortion definition the binary closed form would likely change, perhaps to a convex combination of the semantic and symbolic constraints.
- Editorial inference: The equality RI=RO is stated as conditional on lower semicontinuity, and the appendices interchange expectation with a maximum over components; if that interchange is not valid, the operational interpretation of the closed form requires an alternative proof or a modified distortion definition.
- Editorial inference: A natural testable extension is to replace total variation with KL divergence in the same binary setup and compare the resulting threshold and rate curves against end-task accuracy, which would indicate which divergence measure better predicts practical semantic fidelity.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper develops a semantic rate-distortion framework with dual constraints: a semantic probability distortion constraint (based on conditional distributions p_{S|X}, p_{S|Y}) and a traditional symbolic distortion constraint. It defines an operational semantic rate-distortion function using sequence-level max distortion (Definition 3) and an information semantic rate-distortion function using per-letter constraints (Definition 8). Theorems 1 and 2 claim that these functions coincide under a lower-semicontinuity condition, and Theorem 3 derives a closed-form expression for the doubly symmetric binary source with TV semantic distortion and Hamming symbolic distortion. The paper also presents MNIST experiments indicating that adding a KL semantic term improves recognition accuracy.
Significance. The problem addressed is relevant, and the modeling choice of conditional semantic distributions rather than point estimates is well motivated. The paper correctly leverages known rate-distortion-perception techniques, and the closed-form computation in Theorem 3 is an explicit, non-circular minimization. However, the central claim—that the operational rate-distortion function equals the information rate-distortion function—rests on an invalid interchange of maximum and expectation in the achievability proof. For the paper's own binary Hamming example, the claimed rates are incompatible with Fano's inequality, so the main theorem is false as stated. The experimental section is heuristic and illustrative, not a substitute for a correct coding theorem.
major comments (2)
- [Appendix A, Eqs. (32)–(33)] The achievability proof replaces E[max_i d_o(X_i,Y_i)] with max_i E[d_o(X_i,Y_i)] (and similarly for d_p). This identity is false: for nonnegative random variables, E[max] ≥ max E, generally strictly. Under Definition 3, sequence distortion is the maximum, so the constructed channel satisfying per-letter constraints does not meet the operational constraints (8)–(9). This is load-bearing: for binary Hamming distortion, E[max_i d_H] = P(X^n≠Y^n). Fano's inequality gives R ≥ 1 − D_o − h_2(D_o)/n for any code with E[max] ≤ D_o. Taking D_o=0.4 and D_p>a(D_o), Theorem 3 predicts R ≈ 0.029, whereas the Fano bound requires R ≥ 0.6 in the limit. Thus the stated operational rate-distortion function is not achievable; Theorem 1 and the operational reading of Theorem 3 collapse.
- [Appendix D, Proposition 5] The proof of Proposition 5 asserts without demonstration that \tilde I(t) is nonincreasing on [ (1−2A)/(1−A), 1 ]. This monotonicity is essential for the claimed minimizer (76). The text says "one can verify" but no derivative or convexity/majorization argument is given. This is a separate gap in the closed-form derivation, though secondary to the achievability error.
minor comments (4)
- [Definition 7] The name "Operation Semantic Rate Distortion Function" appears to be a typo for "Operational Semantic Rate Distortion Function"; the term "operational" is used elsewhere. Please harmonize.
- [Section 5.1] The experiments treat the pre-trained classifier output as the true semantic conditional distribution p_{S|y}. This is a strong modeling assumption; the experiments should explicitly acknowledge that they illustrate a heuristic objective rather than directly validate the rate-distortion bounds.
- [Equation (19)] The two cases of the piecewise expression use inconsistent spacing ("D p" vs "D_p") and the boundary D_p = a(D_o) is only included in the first interval. Please make the notation uniform and state the convention for the boundary.
- [Throughout] The manuscript contains several typographical errors (e.g., "efficiency" in the abstract, "Operation" for "Operational", and irregular whitespace in matrix expressions). A careful proofread is needed.
Circularity Check
No significant circularity: the coding theorem and binary closed form are obtained by explicit optimization; the only author-overlap citation is background, and the main defect is a non-circular mathematical error in Appendix A.
full rationale
The paper's central claim is a coding theorem equating an operational semantic rate-distortion function (Definition 7) with an information-theoretic one (Definition 8). These are distinct objects: one is an infimum over code sequences satisfying expected max-distortion constraints, the other is a minimum over conditional distributions satisfying per-symbol expected distortion constraints. The equivalence is not assumed; Theorem 1 is proved via a Poisson-representation code construction and Theorem 2 via a standard converse argument. The binary closed form in Theorem 3 is derived from an explicit constrained minimization of I(w,z) under the constraints Lambda(w,z) <= Dp and Gamma(w,z) <= Do. No parameter is fitted to the target rate, and no load-bearing conclusion is imported from a self-citation. The only author-overlapping reference is [5], which is background on deep-learning semantic communication and is not used to justify the theorems. The main substantive concern is mathematical rather than circular: Appendix A's equations (32)-(33) assert E[max_i d(X_i,Y_i)] = max_i E[d(X_i,Y_i)] for i.i.d. components, which is false in general and invalidates the achievability proof as written. This is an error in the derivation chain, not a definitional reduction of the conclusion to the assumptions. Accordingly, no circular step is identified; the score reflects only the minor background self-citation and the paper's close reliance on known rate-distortion-perception methods, which it explicitly acknowledges.
Assumptions & free parameters
free parameters (1)
- gamma =
0, 0.01, 0.1, 0.5, 100
assumptions (6)
- domain assumption The pair (S,X) has a known joint distribution p_{S,X}, and the sequence (S_i,X_i) is i.i.d.
- domain assumption Markov chain S - X - Y, equivalently p_{S,X,Y} = p_{Y|X} p_{S,X}.
- domain assumption The channel is perfect; semantic compression is the only source of loss.
- domain assumption The information semantic rate-distortion function R_I(D_p,D_o) is lower semicontinuous, or the finite-alphabet sufficient condition (16) holds.
- standard math Poisson representation lemma of [25].
- ad hoc to paper The pre-trained classifier output is treated as the true semantic conditional distribution p_{S|y}, and the original label is treated as p_{S|x}.
invented entities (1)
-
Latent semantic variable S
Cite this review
Pith. "Pith review of Semantic Rate-Distortion Theory with Applications." pith.science (2026). https://pith.science/paper/FAEEP4FX
@misc{pith2026250910061,
author = {Pith},
title = {Pith review of: Semantic Rate-Distortion Theory with Applications},
year = {2026},
howpublished = {\url{https://pith.science/paper/FAEEP4FX}},
note = {Machine review of arXiv:2509.10061}
}
read the original abstract
Artificial intelligence (AI) is ushering in a new era for communication. As a result, the establishment of a semantic communication framework is putting on the agenda. Based on a realistic semantic communication model, this paper develops a rate-distortion framework for semantic compression. Different from the existing works primarily focusing on decoder-side estimation of intrinsic meaning and ignoring its inherent issues, such as ambiguity and polysemy, we exploit a constraint of conditional semantic probability distortion to effectively capture the essential features of practical semantic exchanges in an AI-assisted communication system. With the help of the methods in rate-distortion-perception theory, we establish a theorem specifying the minimum achievable rate under this semantic constraint and a traditional symbolic constraint and obtain its closed-form limit for a particular semantic scenario. From the experiments in this paper, bounding conditional semantic probability distortion can effectively improve both semantic transmission accuracy and bit-rate efficiency. Our framework bridges information theory and AI, enabling potential applications in bandwidth-efficient semantic-aware networks, enhanced transceiver understanding, and optimized semantic transmission for AI-driven systems.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
A mathematical theory of communication,
C. E. Shannon, “A mathematical theory of communication,” The Bell System Technical Jour- nal, vol. 27, no. 3, pp. 379–423, 1948
1948
-
[2]
Recent contributions to the mathematical theory of communication,
W. Weaver, “Recent contributions to the mathematical theory of communication,” ETC: A Review of General Semantics , vol. 10, no. 4, pp. 261–281, 1953. [Online]. A vailable: http://www.jstor.org/stable/42581364 31
arXiv 1953
-
[3]
An outline of a theory of semantic information,
R. Carnap and Y. Bar-Hillel, “An outline of a theory of semantic information,” Journal of Symbolic Logic, vol. 19, no. 3, pp. 230–232, 1952
1952
-
[4]
Towards a theory of semantic communication,
J. Bao, P. Basu, M. Dean, C. Partridge, A. Swami, W. Leland, and J. A. Hendler, “Towards a theory of semantic communication,” in 2011 IEEE Netw. Sci. Workshop (NSW) , 2011, pp. 110–117
2011
-
[5]
Deep learning enabled semantic communication systems,
H. Xie, Z. Qin, G. Y. Li, and B.-H. Juang, “Deep learning enabled semantic communication systems,” IEEE Trans. Signal Process. , vol. 69, pp. 2663–2675, 2021
2021
-
[6]
Semantic communication: From philosophical conceptions towards a mathematical framework,
J. Gholipour, R. F. Schaefer, and G. P. Fettweis, “Semantic communication: From philosophical conceptions towards a mathematical framework,” 2025. [Online]. A vailable: https://arxiv.org/abs/2505.01342
arXiv 2025
-
[7]
A rate-distortion framework for characterizing semantic information,
J. Liu, W. Zhang, and H. V. Poor, “A rate-distortion framework for characterizing semantic information,” in 2021 IEEE Int. Symp. Inf. Theory (ISIT) , 2021, pp. 2894–2899
2021
-
[8]
An indirect rate-distortion characterization for semantic sources: General model and the case of gaussian observation,
J. Liu, S. Shao, W. Zhang, and H. V. Poor, “An indirect rate-distortion characterization for semantic sources: General model and the case of gaussian observation,” IEEE Trans. Commun., vol. 70, no. 9, pp. 5946–5959, 2022
2022
Show all 30 references
-
[9]
Semantic compression with side information: A rate-distortion perspective,
T. Guo, Y. Wang, J. Han, H. Wu, B. Bai, and W. Han, “Semantic compression with side information: A rate-distortion perspective,” ArXiv, vol. abs/2208.06094, 2022. [Online]. A vailable:https://api.semanticscholar.org/CorpusID:251554744
2022 arXiv
-
[10]
Protecting semantic information using an efficient secret key,
T. Guo, J. Han, H. Wu, Y. Wang, B. Bai, and W. Han, “Protecting semantic information using an efficient secret key,” in 2022 IEEE Int. Symp. Inf. Theory (ISIT) , 2022, pp. 2660–2665
2022
-
[11]
Rate-distortion theory for strategic semantic communication,
Y. Xiao, X. Zhang, Y. Li, G. Shi, and T. Başar, “Rate-distortion theory for strategic semantic communication,” in 2022 IEEE Inf. Theory Workshop (ITW) , 2022, pp. 279–284
2022
-
[12]
Semantic communication of learnable concepts,
F. Pase, S. Kobus, D. Gündüz, and M. Zorzi, “Semantic communication of learnable concepts,” in 2023 IEEE Int. Symp. Inf. Theory (ISIT) , 2023, pp. 731–736
2023
-
[13]
The perception-distortion tradeoff,
Y. Blau and T. Michaeli, “The perception-distortion tradeoff,” in 2018 IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2018, pp. 6228–6237. 32
2018
-
[14]
Rethinking lossy compression: The rate-distortion-perception tradeoff,
——, “Rethinking lossy compression: The rate-distortion-perception tradeoff,” in Proc. 36th Int. Conf. Mach. Learn., ICML 2019, 9-15 June 2019, Long Beach, California, USA , ser. Proceedings of Machine Learning Research, K. Chaudhuri and R. Salakhutdinov, Eds., vol. 97. PMLR, 2...
2019
-
[15]
A coding theorem for the rate-distortion-perception function,
L. Theis and A. B. Wagner, “A coding theorem for the rate-distortion-perception function,” in Neural Compression: From Information Theory to Applications – Workshop @ ICLR 2021 ,
2021
-
[16]
On the rate-distortion-perception function,
J. Chen, L. Yu, J. Wang, W. Shi, Y. Ge, and W. Tong, “On the rate-distortion-perception function,” IEEE J. Sel. Area Inf. Theory , vol. 3, no. 4, pp. 664–673, 2022
2022
-
[17]
The rate-distortion-perception trade-off with side information,
Y. Hamdi and D. Gündüz, “The rate-distortion-perception trade-off with side information,” in 2023 IEEE Int. Symp. Inf. Theory (ISIT) , 2023, pp. 1056–1061
2023
-
[18]
Conditional rate-distortion-perception trade-off,
X. Niu, D. Gündüz, B. Bai, and W. Han, “Conditional rate-distortion-perception trade-off,” in 2023 IEEE Int. Symp. Inf. Theory (ISIT) , 2023, pp. 1068–1073
2023
-
[19]
Rate-distortion-perception tradeoff based on the conditional-distribution perception measure,
S. Salehkalaibar, J. Chen, A. Khisti, and W. Yu, “Rate-distortion-perception tradeoff based on the conditional-distribution perception measure,” IEEE Trans. Inf. Theory , vol. 70, no. 12, pp. 8432–8454, 2024
2024
-
[20]
The rate-distortion-perception trade-off: the role of private randomness,
Y. Hamdi, A. B. Wagner, and D. Gündüz, “The rate-distortion-perception trade-off: the role of private randomness,” in 2024 IEEE Int. Symp. Inf. Theory (ISIT) , 2024, pp. 1083–1088
2024
-
[21]
Optimal neural compressors for the rate-distortion- perception tradeoff,
E. Lei, H. Hassani, and S. S. Bidokhti, “Optimal neural compressors for the rate-distortion- perception tradeoff,” 2025. [Online]. A vailable: https://arxiv.org/abs/2503.17558
2025 arXiv
-
[22]
Rate-distortion-perception theory for semantic com- munication,
J. Chai, Y. Xiao, G. Shi, and W. Saad, “Rate-distortion-perception theory for semantic com- munication,” in 2023 IEEE 31st Int. Conf. Netw. Protocols (ICNP) , 2023, pp. 1–6
2023
-
[23]
Semantic communication via rate distortion perception bottleneck,
Z. Zhao and C. Wang, “Semantic communication via rate distortion perception bottleneck,”
-
[24]
Rate–distortion–perception trade-off in information theory, generative models, and intelligent communications,
X. Niu, B. Bai, N. Guo, W. Zhang, and W. Han, “Rate–distortion–perception trade-off in information theory, generative models, and intelligent communications,” Entropy, vol. 27, no. 4, 2025. 33
2025
-
[25]
Strong functional representation lemma and applications to coding theorems,
C. T. Li and A. E. Gamal, “Strong functional representation lemma and applications to coding theorems,” IEEE Trans. Inf. Theory , vol. 64, no. 11, pp. 6967–6978, 2018
2018
-
[26]
T. M. Cover, Elements of Information Theory , 2nd ed. John Wiley & Sons, 2006
2006
-
[27]
Conditional probability models for deep image compression,
F. Mentzer, E. Agustsson, M. Tschannen, R. Timofte, and L. V. Gool, “Conditional probability models for deep image compression,” in 2018 IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2018, pp. 4394–4402
2018
-
[28]
Generative adversarial networks for extreme learned image compression,
E. Agustsson, M. Tschannen, F. Mentzer, R. Timofte, and L. Van Gool, “Generative adversarial networks for extreme learned image compression,” in 2019 IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2019, pp. 221–231. 34
2019
-
[2021]
A vailable: https://openreview.net/forum?id=BzUaLGtKecs
[Online]. A vailable: https://openreview.net/forum?id=BzUaLGtKecs
-
[2024]
A vailable: https://arxiv.org/abs/2405.09995
[Online]. A vailable: https://arxiv.org/abs/2405.09995
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.