REVIEW 4 major objections 4 minor 23 references
Non-Asymptotic Achievable Rate-Distortion Region for Indirect Wyner-Ziv Source Coding
T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read This paper establishes that the finite-blocklength rate for the indirect Wyner-Ziv problem—where a decoder must reconstruct both an observed source and a latent, unobserved source—is the asymptotic rate-distortion function plus a…
desk verdict A promising first finite-blocklength treatment of indirect Wyner-Ziv with two distortion constraints, but the main theorem's proof has a genuine misstep in Eq. (63) that needs repair before the result is reliable. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The argument rests on three pieces. The conditional Poisson Matching Lemma converts the probability that the encoder's and decoder's auxiliary random variables disagree into a bound involving a ratio of conditional densities, producing the one-shot excess-probability bound of Theorem 2. The multidimensional Berry-Esseen theorem is then applied to the three-dimensional information-density-distortion vector $(\iota(U;X)-\iota(U;Y),\, d(X,\hat{X}),\, d_s(S,\hat{S}))$, whose Gaussian limit defines the dispersion region $S(V,\epsilon)$ and yields the $1/\sqrt{n}$ term. The Blahut-Arimoto algorithm, obtained by differentiating the Lagrangian of the asymptotic optimization, provides the optimal test channel $P^\star_{U|X}$ used to evaluate the bound numerically for a classification example.
What would settle it
Compute the covariance matrix $V$ for a specific Gaussian-mixture example with parameters where $\theta_0$ and $\theta_1$ produce a nearly singular $V$, then run the Blahut-Arimoto algorithm and evaluate the excess probability from Theorem 2; if the actual excess probability exceeds the bound from Theorem 3 for moderate $n$, the claimed second-order rate would be falsified.
Extended reading notes
Core claim
The central claim is that the finite-blocklength indirect Wyner-Ziv rate-distortion function is bounded above by $R_{WZ}(D,D_s) + e_1 S(V,\epsilon)/\sqrt{n} + 2\log n/n$ for $n$ sufficiently large, where $R_{WZ}(D,D_s)$ is the asymptotic function from Theorem 1, $e_1$ selects the rate coordinate, and $S(V,\epsilon)$ is the dispersion region of the Gaussian limit of the information-density-distortion vector. In the asymptotic regime, the paper characterizes $R_{WZ}(D,D_s)$ as the infimum of $I(X;U)-I(U;Y)$ over test channels satisfying two distortion constraints, one on the observed source $X$ and one on the latent source $S$ through a modified distortion measure. For finite $n$, the paper shows that codes exist whose rate tracks the asymptotic rate with a second-order penalty that vanishes as $n$ grows.
Load-bearing premise
The proof of Theorem 3 applies a multidimensional Berry-Esseen theorem to the information-density-distortion vector, which requires the covariance matrix $V$ to be positive definite and the third moments to exist for every parameter choice; if $V$ is singular or the densities have heavy tails, the Gaussian approximation may fail.
Editorial extensions
If this is right
- If Theorem 3 is correct, there exist $(n,M,D,D_s,\epsilon)$-codes whose rate is within $e_1 S(V,\epsilon)/\sqrt{n} + 2\log n/n$ of the asymptotic indirect Wyner-Ziv rate.
- The achievable non-asymptotic region expands toward the asymptotic region as blocklength $n$ or excess probability $\epsilon$ increases, matching the numerical behavior reported in the paper.
- The proposed Blahut-Arimoto algorithm lets one compute the indirect rate-distortion region for source models such as Gaussian mixtures where no closed-form expression is available.
- For binary classification labels with Hamming distortion, the method quantifies the rate needed to achieve a target semantic accuracy jointly with an observation-distortion constraint.
- Because the Poisson matching bound avoids separate packing and covering arguments, the second-order term is tighter than bounds built from channel resolvability codes.
Reading between the lines
- The same second-order technique should extend to $M$-ary latent labels and to tasks beyond classification, such as clustering or parameter estimation, as long as the distortion on $S$ is per-letter and the information-density-distortion vector has third moments.
- The unstated requirement that $V$ be positive definite suggests that practical implementations should regularize the empirical covariance or use a truncated Gaussian approximation when distortions are nearly deterministic.
- The random-guessing zero-rate baseline for $D_s>0.5$ could be folded directly into the optimization as a rate floor, which would sharpen the bound at small blocklengths.
- A converse bound for the same indirect setup would let one test whether the $1/\sqrt{n}$ dispersion term is tight or merely an achievability artifact.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper considers the indirect Wyner-Ziv source coding problem in which a length-n observation X is encoded with decoder-only side information Y, and the decoder must reconstruct both X and a latent source S under two distortion constraints. The authors state an asymptotic rate-distortion function (Theorem 1), a one-shot excess-probability bound obtained from the conditional Poisson matching lemma (Theorem 2), and a second-order achievability bound (Theorem 3) of the form R_WZ(D,D_s) plus a dispersion term over sqrt(n) plus 2 log n/n. They also propose a Blahut-Arimoto algorithm for the associated optimization and evaluate the finite-blocklength region numerically for a Gaussian-mixture classification model.
Significance. If proved, Theorem 3 would be a useful finite-blocklength result for indirect Wyner-Ziv coding and would extend the Poisson-matching approach to problems with two distortion constraints. The paper is transparent about its proof ingredients, namely the conditional Poisson matching lemma and the multivariate Berry-Esseen theorem, and no fitted parameters enter the bound; the numerical example is concrete and the BA algorithm is a reasonable extension of existing techniques. The main result is not established as written because of a direction error in the key probability bound in Appendix C and missing regularity conditions for the Berry-Esseen step. These issues appear local and correctable, so the contribution is potentially significant but needs a major revision.
major comments (4)
- [Appendix C, Eqs. (62)-(63)] The inequality connecting (62) and (63) has the wrong direction. The event in (62) is a union of coordinatewise exceedances: {sum j_i,1 > log M - log n} union {sum j_i,2 > nD_s} union {sum j_i,3 > nD}. The event in (63) is the componentwise lower-orthant condition {sum j_i >= (log M, nD_s, nD) - log n}, which is an intersection-type event. For a general random vector Z=(A,B,C), P(A>a or B>b or C>c) is not bounded above by P(A>=a', B>=b', C>=c') for lowered thresholds; for example, a realization with only A large leaves the intersection event false. Therefore (63) does not upper-bound (62), and this step is the only bridge from the Poisson matching bound to the multivariate Berry-Esseen approximation. The proof should be rewritten so that Berry-Esseen is applied to the complement event {sum j_i <= threshold}, since 1 - P[excess] is exactly the probability that all three coordinates are below their thresholds; as written, Theorem 3 is unproved.
- [Appendix C, Eqs. (59), (63), and (65)] The coordinate ordering is inconsistent. The vector (59) is (i(X;U)-i(U;Y), d(X,xhat), d_s(S,shat)), so the second coordinate is the observation distortion and the third is the semantic distortion. The threshold vector in (63) is (log M, nD_s, nD) - log n, which places nD_s on the d coordinate and nD on the d_s coordinate. The vector b* in (65) appears to use the order (log M/n, D, G), which would be consistent with (59) only if G = D_s. This makes the derivation and the definition of the dispersion set in (25) unverifiable; please fix the ordering and state clearly which coordinate is which.
- [Theorem 3 and Appendix C, Theorem 4] The multidimensional Berry-Esseen theorem is invoked without its hypotheses. Theorem 4 requires the unnormalized summands U_i to have finite third moments and the covariance matrix V to be positive definite; Theorem 3 only says V is positive semi-definite in (25) and imposes no moment conditions on the distortion measures defined in (2)-(3). If V is singular, the Gaussian vector B in (25) is degenerate and the standard O(1/sqrt(n)) Berry-Esseen statement does not apply; if the source has heavy tails with respect to d or d_s, the third moments of the entries of (59) may be infinite. Please state explicit regularity conditions in Theorem 3, or restrict the theorem to models satisfying them and handle the singular case separately.
- [Appendix A, Eqs. (30)-(34)] The binning error calculation in the asymptotic proof of Theorem 1 is not correct as written. The probability that a given codeword u(i') falls into the received bin B(1) is 2^{-nR_2}, not 2^{-n(R_1-R_2)}; and the number of codewords in the whole codebook that are jointly typical with a fixed y is of order 2^{n(R_1 - I(U;Y))}. The resulting exponent should be n(R_2 - R_1 + I(U;Y) - epsilon), and the condition for P[E4] -> 0 is R_2 > R_1 - I(U;Y), which is the standard Wyner-Ziv condition when R_1 is close to I(X;U). The printed condition R_1-R_2 < I(U;Y)+epsilon has the wrong direction and does not follow from the displayed calculation. This should be corrected, since Theorem 1 is one of the two central claims.
minor comments (4)
- [Section IV-B, Eq. (24)] Eq. (24) states an additive term 2*gamma/M, but the proof of Theorem 3 uses and needs an additive term 1/n; no argument is given for the displayed constant. Please reconcile the two bounds.
- [Theorem 3 statement] The notation inf_R in (26) is unclear: the left-hand side already takes an infimum over R in Definition 2, and the right-hand side's second term e_1 S(V,epsilon)/sqrt(n) is a set rather than a scalar unless a particular b* in S(V,epsilon) is chosen. Please spell out how R enters b* and the sense in which the set-valued dispersion region is used.
- [References] Reference [11] duplicates reference [9]; the two entries should be merged or distinguished.
- [After Theorem 3] The word "infinum" should be "infimum", and the sentence "The infinum is taken on the rate R that implicitly appears in the second term" is too terse to be useful without the clarification requested above.
Circularity Check
No significant circularity: the achievability bounds rest on the external Poisson matching lemma and the multidimensional Berry-Esseen theorem; the authors' self-citation [21] is not load-bearing.
full rationale
The paper's derivation chain is self-contained against external results. Theorem 1's asymptotic rate-distortion function is obtained in Appendix A from standard achievability and converse arguments for Wyner-Ziv coding, with the modified distortion measure credited to [3]; the result is not assumed in its own proof. Lemma 1 and Lemma 2 are proved directly from the optimization in Theorem 1. Theorem 2 is derived from the conditional Poisson matching lemma of [14], applied to the actual coding scheme; no fitted parameter, calibration, or target-dependent choice is used to obtain the excess-probability bound. Theorem 3 is proved in Appendix C by applying the multidimensional Berry-Esseen theorem of [23] to the information-density-distortion vector (59); the Gaussian approximation and the dispersion region are defined from the covariance of that vector, and the proof does not rely on a prior result whose content is equivalent to the theorem. The citation of the authors' own work [21] appears only as an ancillary reference for a standard technique ('as in [13], [21]') and is not load-bearing. The BA algorithm is used solely for numerical evaluation and does not enter the proofs of Theorems 2 or 3. The paper also openly notes the looseness of the finite-blocklength Gaussian approximation at small n, which is an accuracy limitation rather than a circularity. The skeptic's concern about the direction of inequality in Eq. (63) and the unstated positive-definiteness/moment conditions in Theorem 3 are proof-correctness issues, not instances of a claim reducing to its own inputs; therefore they do not raise the circularity score.
Assumptions & free parameters
assumptions (5)
- domain assumption The triplet (S_i,X_i,Y_i) is i.i.d. from P_SXY = P_S P_{XY|S}, with P_XY positive where needed.
- domain assumption The auxiliary random variable U satisfies the Markov chain U - X - Y and the test channel P_{U|X} satisfies absolute continuity P_UX << P_U x P_X and P_UY << P_U x P_Y.
- standard math The Poisson matching lemma (Lemma 3) is correct for Polish alphabets with base measure nu.
- standard math The multidimensional Berry-Esseen theorem applies to the information-density-distortion vector with finite third moments and positive definite covariance V.
- domain assumption For the classification example, S is binary uniform and (X,Y) is a Gaussian mixture with covariance matrices Sigma_0 and Sigma_1.
Cite this review
Pith. "Pith review of Non-Asymptotic Achievable Rate-Distortion Region for Indirect Wyner-Ziv Source Coding." pith.science (2026). https://pith.science/paper/FOB2WOFD
@misc{pith2026250717432,
author = {Pith},
title = {Pith review of: Non-Asymptotic Achievable Rate-Distortion Region for Indirect Wyner-Ziv Source Coding},
year = {2026},
howpublished = {\url{https://pith.science/paper/FOB2WOFD}},
note = {Machine review of arXiv:2507.17432}
}
abstract
In the Wyner-Ziv source coding problem, a source $X$ has to be encoded while the decoder has access to side information $Y$. This paper investigates the indirect setup, in which a latent source $S$, unobserved by both the encoder and the decoder, must also be reconstructed at the decoder. This scenario is increasingly relevant in the context of goal-oriented communications, where $S$ can represent semantic information obtained from $X$. This paper derives the indirect Wyner-Ziv rate-distortion function in asymptotic regime and provides an achievable region in finite block-length. Furthermore, a Blahut-Arimoto algorithm tailored for the indirect Wyner-Ziv setup, is proposed. This algorithm is then used to give a numerical evaluation of the achievable indirect rate-distortion region when $S$ is treated as a classification label.
Figures
Reference graph
Works this paper leans on
-
[5]
A rate distortion approach to goal-oriented communication,
P. A. Stavrou and M. Kountouris, “A rate distortion approach to goal-oriented communication,” in2022 IEEE International Symposium on Information Theory (ISIT), 2022, pp. 590–595
work page 2022
-
[7]
Distributed indirect source coding with decoder side information,
J. Tang, Q. Yang, and D. G¨ und¨ uz, “Distributed indirect source coding with decoder side information,”arXiv preprint arXiv:2405.13483, 2024
arXiv 2024
-
[1]
The rate-distortion function for source coding with side information at the decoder,
A. Wyner and J. Ziv, “The rate-distortion function for source coding with side information at the decoder,”IEEE Transac- tions on information Theory, vol. 22, no. 1, pp. 1–10, 1976
work page 1976
-
[2]
Information transmission with additional noise,
R. Dobrushin and B. Tsybakov, “Information transmission with additional noise,”IRE Transactions on Information Theory, vol. 8, no. 5, pp. 293–304, 1962
work page 1962
-
[3]
Indirect rate distortion problems,
H. Witsenhausen, “Indirect rate distortion problems,”IEEE Transactions on Information Theory, vol. 26, no. 5, pp. 518– 521, 1980
work page 1980
-
[4]
A rate-distortion framework for characterizing semantic information,
J. Liu, W. Zhang, and H. V. Poor, “A rate-distortion framework for characterizing semantic information,” in2021 IEEE Inter- national Symposium on Information Theory (ISIT). IEEE, 2021, pp. 2894–2899
work page 2021
-
[6]
Semantic compression with side information: A rate-distortion perspective,
T. Guo, Y. Wang, J. Han, H. Wu, B. Bai, and W. Han, “Semantic compression with side information: A rate-distortion perspective,”arXiv:2208.06094, 2022
arXiv 2022
-
[8]
H. Yang, Y. Shi, S. Shao, and X. Yuan, “Joint data and semantics lossy compression: Nonasymptotic and second-order achievability bounds,”arXiv preprint arXiv:2401.14962, 2024
work page Pith review arXiv 2024
Show all 23 references
-
[9]
Channel coding rate in the finite blocklength regime,
Y. Polyanskiy, H. V. Poor, and S. Verd´ u, “Channel coding rate in the finite blocklength regime,”IEEE Transactions on Information Theory, vol. 56, no. 5, pp. 2307–2359, 2010
2010
-
[10]
Fixed-length lossy compression in the finite blocklength regime,
V. Kostina and S. Verdu, “Fixed-length lossy compression in the finite blocklength regime,”IEEE Transactions on Information Theory, vol. 58, no. 6, pp. 3309–3338, 2012
2012
-
[11]
Channel coding rate in the finite blocklength regime,
Y. Polyanskiy, H. V. Poor, and S. Verdu, “Channel coding rate in the finite blocklength regime,”IEEE Transactions on Information Theory, vol. 56, no. 5, pp. 2307–2359, 2010
2010
-
[12]
On the dispersions of three network information theory problems,
V. Y. F. Tan and O. Kosut, “On the dispersions of three network information theory problems,”IEEE Transactions on Information Theory, vol. 60, no. 2, pp. 881–903, 2014
2014
-
[13]
Nonasymp- totic and second-order achievability bounds for coding with side-information,
S. Watanabe, S. Kuzuoka, and V. Y. Tan, “Nonasymp- totic and second-order achievability bounds for coding with side-information,”IEEE Transactions on Information Theory, vol. 61, no. 4, pp. 1574–1605, 2015
2015
-
[14]
A unified framework for one- shot achievability via the poisson matching lemma,
C. T. Li and V. Anantharam, “A unified framework for one- shot achievability via the poisson matching lemma,”IEEE Transactions on Information Theory, vol. 67, no. 5, pp. 2624– 2651, 2021
2021
-
[15]
Thomas and A
M. Thomas and A. T. Joy,Elements of information theory. Wiley-Interscience, 2006
2006
-
[16]
The rate-distortion function for source coding with side information at the decoder-ii: General sources,
A. D. Wyner, “The rate-distortion function for source coding with side information at the decoder-ii: General sources,”In- formation and control, vol. 38, pp. 60–80, 1978
1978
-
[17]
An algorithm for computing the capacity of ar- bitrary discrete memoryless channels,
S. Arimoto, “An algorithm for computing the capacity of ar- bitrary discrete memoryless channels,”IEEE Transactions on Information Theory, vol. 18, no. 1, pp. 14–20, 1972
1972
-
[18]
F. M. Willems,Computation of the Wyner-Ziv rate-distortion function. Technische Hoge-school Eindhoven, 1983
1983
-
[19]
Strong functional representation lemma and applications to coding theorems,
C. T. Li and A. E. Gamal, “Strong functional representation lemma and applications to coding theorems,” in2017 IEEE International Symposium on Information Theory (ISIT), 2017, pp. 589–593
2017
-
[20]
Non-asymptotic achievability bounds in multiuser information theory,
S. Verd´ u, “Non-asymptotic achievability bounds in multiuser information theory,” in2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2012, pp. 1–8
2012
-
[21]
Distributed source coding for parametric and non-parametric regression,
J. Wei, E. Dupraz, and P. Mary, “Distributed source coding for parametric and non-parametric regression,”arXiv preprint arXiv:2404.18688, 2024
2024 arXiv
-
[22]
Successive refinement of information,
W. Equitz and T. Cover, “Successive refinement of information,” IEEE Transactions on Information Theory, vol. 37, no. 2, pp. 269–275, 1991
1991
-
[23]
On the Rate of Convergence in the Multivariate CLT,
F. Gotze, “On the Rate of Convergence in the Multivariate CLT,”The Annals of Probability, vol. 19, no. 2, pp. 724 – 739, 1991. Appendix A Proof of Theorem 1 The proof is a combination of the rate-distortion prob- lem with two constraints [15] and the conventional Wyner- Ziv [1...
1991
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.