REVIEW 4 major objections 6 minor 20 references
Rate-Distortion Region for Distributed Indirect Source Coding with Decoder Side Information
T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read The paper claims an exact rate-distortion region for distributed indirect source coding when the sources are conditionally independent given decoder side information.
desk verdict Plausible but unproved: the exact rate-distortion region for distributed indirect source coding rests on omitted proofs, and the numerical example never exercises the indirect setting. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing structure is the conditional independence X_1,...,X_M ⊥ given Y, which forces the mutual-information cost to decouple across encoders. The argument runs through auxiliary random variables W_i introduced by test channels q_i(w_i|x_i); the rate of encoder i is measured by I(X_i;W_i) − I(W_i;Y), the classical single-encoder side-information expression, and the decoder is a mapping q'(t|y,w) chosen as a Bayes detector to minimize expected distortion. The optimization is carried out on the Lagrangian L_λ = Σ_i [I(W_i;X_i) − I(W_i;Y)] + λ(E[d(T, ˆT)] − D), and the algorithm alternates updates of the back-channel Q_i(w_i|y), the encoder test channel q_i(w_i|x_i), and the decoder q'(t|y,w), producing a monotonically non-increasing Lagrangian. The same Lagrangian is the object used to prove the region.
What would settle it
Take a small joint distribution with X_1 and X_2 conditionally independent given Y, set T to a nonlinear function such as XOR, and compute the true minimal sum-rate by exhaustive search over all encoder and decoder maps for a distortion target; if any achievable point lies below the sum of the two individual claimed bounds, or if the rectangular claimed region contains points that exhaustive search shows are not achievable, the claimed exact region fails.
Extended reading notes
Core claim
The central claim is that when X_1,...,X_M are conditionally independent given the side information Y, the true rate-distortion region R*(D) for reconstructing an indirect source T is exactly the set of rate tuples satisfying R_i ≥ I(X_i;W_i) − I(W_i;Y) for every i, for some auxiliary variables W_i such that a decoding function g(W,Y) achieves average distortion at most D. The paper derives this by taking an achievable region built from independent per-encoder test channels and an outer bound adapted from a general multiterminal bound, and showing the two coincide under the conditional-independence assumption. A corollary reduces the region to M separate single-encoder side-information constraints, so the encoders do not need to coordinate beyond sharing the side-information statistics. The claimed region is stated for discrete memoryless sources with a single-letter distortion measure; the general case without conditional independence is left open.
Load-bearing premise
The argument assumes that the existing outer and inner bounds for reconstructing the sources directly can be transferred unchanged to reconstructing a latent variable T, even though the detailed proofs of that transfer are omitted here.
Editorial extensions
If this is right
- A rate tuple is achievable exactly when each encoder meets its individual bound $R_i \ge I(X_i;W_i) - I(W_i;Y)$, so no coordination among encoders is needed beyond knowing the joint statistics.
- The single-letter expression gives a computable benchmark for task-oriented compression: for any discrete joint distribution satisfying the conditional-independence condition, the boundary of the rate-distortion region can be traced by sweeping the Lagrange multiplier.
- In the $M=1$ limit the region reduces to the classical remote side-information rate-distortion function, and the paper's numerical algorithm reproduces the analytic curve.
- For general correlated sources (not conditionally independent given $Y$), the inner and outer regions need not coincide, so the exact region for that case remains open.
Reading between the lines
- If the proof transfer holds, a natural extension is to Gaussian sources with squared-error distortion, where the per-encoder bounds would likely become closed-form water-filling expressions; the paper does not derive this.
- A concrete test of the claim would be to compute the region for a small binary example where the latent variable $T$ is a non-separable function of the sources and compare with exhaustive search over quantizers; a mismatch would indicate that the indirect distortion needs extra terms in the outer bound.
- The alternating-minimization algorithm is only guaranteed to converge to the global minimum when the Lagrangian is convex; for non-convex cases the authors resort to random restarts, so the numerical region should be treated as an inner bound unless independently certified.
- If confirmed, the result supplies a design rule for distributed learning systems: each client's compression overhead is exactly its individual side-information cost, independent of how many other clients are compressing correlated observations.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper considers a distributed source coding problem in which M encoders observe correlated sources X_1,...,X_M, a decoder has side information Y, and the goal is to reconstruct a latent variable T under a distortion constraint. The main theoretical claim (Theorem 3) is that, when the sources are conditionally independent given Y, the rate-distortion region is given by individual Wyner-Ziv inequalities R_i ≥ I(X_i;W_i) − I(W_i;Y). The paper also proposes an iterative Blahut-Arimoto-style algorithm to compute the region and presents numerical examples for M=2 binary sources. The abstract and introduction frame the problem in the context of task-oriented semantic communication and distributed learning.
Significance. An exact single-letter characterization for this indirect multiterminal source coding problem would be a useful contribution, particularly for task-oriented communication. The numerical algorithm, if properly justified, could provide a practical tool for evaluating such regions. However, as submitted, the central theorem is asserted without proof: the proofs of Lemma 1, Lemma 2, Theorem 3, and Corollary 4 are either omitted, deferred, or dismissed as 'easily established' from prior works. The convergence analysis of the algorithm is internally contradictory, and the numerical validation only covers the M=1 case. Hence the paper currently offers a conjecture and a heuristic algorithm rather than a verifiable information-theoretic result.
major comments (4)
- [Section II, Lemma 1, Lemma 2, Theorem 3, Corollary 4] The central theorem is not proved. Lemma 1's proof is omitted with 'we omit the detailed proof here due to space limitations'; Lemma 2 and Theorem 3 are dismissed with 'can be easily established based on Wagner's proof'; Corollary 4's proof is deferred to a longer version. This is load-bearing because the cited results [15] and [18] are for direct reconstruction of the observed sources, whereas the present problem imposes a single distortion constraint on a latent variable T that no encoder observes. It is not automatic that the auxiliary variables that satisfy the rate lower bounds also satisfy the distortion constraint, nor that the outer-bound auxiliaries can be chosen conditionally independent given Y so that the subset-sum conditions in (2) collapse to the individual conditions (7). Without these steps, the claimed exact region Ra(D)=Ro(D)=R*(D) is unsupported.
- [Section III, Convergence analysis] The convergence argument is internally contradictory. The text first claims that the Lagrangian is convex and therefore the proposed iterative optimization framework achieves the global minimum, then immediately concedes that the expected distortion term includes a product of variables and 'the Lagrangian may exhibit non-convex behavior'. This contradiction means the algorithm's convergence to the true rate-distortion function is not established; monotone non-increasing Lagrangian values alone do not imply global optimality in non-convex problems, and the appeal to [21] only supports 'highly effective inner bounds', not exact computation.
- [Section IV, numerical example] The numerical validation is insufficient. The only external comparison is the M=1 case (Fig. 4), which reduces to a point-to-point problem and does not exercise the coupling among encoders. The M=2 example with T={X1,X2} and sum-Hamming distortion is not compared against any known region, outer bound, or independent computation. Therefore the numerical section cannot support the claimed exact region for M>1.
- [Section III, Eq. (8)] The Lagrangian in (8) minimizes the unweighted sum of rates ∑_{i∈M}(I(W_i;X_i)−I(W_i;Y)) plus a distortion penalty. Since the rate-distortion region is a set of M-dimensional rate vectors, minimizing only the sum rate with a single Lagrange multiplier λ traces at best one boundary point per λ; to characterize the full region, a weighted sum with distinct per-encoder weights is generally required. The paper does not justify that equal weights suffice, and in asymmetric problems they generally do not, so the proposed algorithm does not compute the entire rate-distortion region as claimed.
minor comments (6)
- [Corollary 4] The statement says 'for all l ∈ {1,...,M}', but the variable is i; this should be corrected to 'for all i ∈ {1,...,M}'.
- [Section I, last paragraph] There is a typo: 'considerd' should be 'considered'.
- [Equation (2)] The definition of A^c as 'all elements in the set A ⊆ {1,...,M} that are not in A' is circular; it should be the complement of A in {1,...,M}.
- [Lemma 6, Eq. (13)] The notation in (13) is confusing: the Lagrangian is written as Lλ(Q, q*_{\m}, q'), but the right-hand side sums over all i∈M, mixing subscripts. The expression should be clarified.
- [Section IV, M=1 claim] The paper states that when M=1 the problem reduces to the traditional point-to-point Wyner-Ziv problem. This is only true if T equals the observed source X1; in general it is the remote Wyner-Ziv problem, so the comparison in Fig. 4 needs the condition T=X1 to be stated.
- [References] Reference [20] is malformed: 'C. Q., T. M. Cover, and J. A. Thomas' should be 'T. M. Cover and J. A. Thomas'.
Circularity Check
No circular derivation: the rate region is quoted from independent prior results and the only self-citation is motivational, so the score reflects only a minor non-load-bearing self-citation.
full rationale
Walked the claimed derivation chain. Lemma 1 (achievability, region (2)-(3)) is delegated to Gastpar's independent multiple-source Wyner-Ziv result, with the paper stating: "This can be proven by extending the achievable region proof for the Wyner-Ziv problem with multiple sources, as established by Gastpar (Theorem 2, [18]). However, we omit the detailed proof here due to space limitations." Lemma 2 and Theorem 3 (outer bound and equality) are delegated to Wagner's independent multiterminal bound: "Lemma 2 and Theorem 3 can be easily established based on Wagner's proof, see [15]." Corollary 4 is deferred: "The rigorous proof will be provided in a longer version." These are real rigor gaps and potential correctness risks, because the cited theorems concern direct reconstruction of X_1,...,X_M rather than the latent T, but a missing or outsourced proof is not circular. The collapse of (2)/(4) to the per-encoder bound (7) uses the stated conditional-independence hypothesis, not the theorem being proved. The numerical section computes the very expression in (7) via (32) and validates the M=1 case against the external Wyner-Ziv result [22], so no fitted parameter is renamed as a prediction. The only self-citation, [1] (co-author Q. Yang), appears in the introduction as contextual motivation and is not load-bearing. Therefore no circular step is identified; the score of 2 reflects only this minor non-load-bearing self-citation, not any circularity in the derivation.
Assumptions & free parameters
free parameters (1)
- BSC crossover probabilities p1=p2 =
0.3
assumptions (4)
- domain assumption Finite-alphabet discrete memoryless source with fixed joint distribution p(t,x1,...,xM,y) known to the system.
- domain assumption Conditional independence of X1,...,XM given Y.
- ad hoc to paper Wagner's outer bound and Gastpar's achievable scheme transfer unchanged to the indirect latent-variable setting with distortion on T.
- ad hoc to paper Convexity of the Lagrangian for global convergence.
Cite this review
Pith. "Pith review of Rate-Distortion Region for Distributed Indirect Source Coding with Decoder Side Information." pith.science (2026). https://pith.science/paper/E2X2UIOL
@misc{pith2026250113784,
author = {Pith},
title = {Pith review of: Rate-Distortion Region for Distributed Indirect Source Coding with Decoder Side Information},
year = {2026},
howpublished = {\url{https://pith.science/paper/E2X2UIOL}},
note = {Machine review of arXiv:2501.13784}
}
abstract
This paper studies a variant of the rate-distortion problem motivated by task-oriented semantic communication and distributed learning systems, where $M$ correlated sources are independently encoded for a central decoder. The decoder has access to correlated side information in addition to the messages received from the encoders and aims to recover a latent random variable under a given distortion constraint, rather than recovering the sources themselves. We characterize the exact rate-distortion function for the case where the sources are conditionally independent given the side information. Furthermore, we develop a distributed Blahut-Arimoto (BA) algorithm to numerically compute the rate-distortion function. Numerical examples are provided to demonstrate the effectiveness of the proposed approach in calculating the rate-distortion region.
Figures
Reference graph
Works this paper leans on
-
[15]
An improved outer bound for multiterminal source coding,
A. B. Wagner and V . Anantharam, “An improved outer bound for multiterminal source coding,” IEEE Transactions on Information Theory , vol. 54, no. 5, pp. 1919–1937, 2008
work page 1919
-
[18]
The Wyner-Ziv problem with multiple sources,
M. Gastpar, “The Wyner-Ziv problem with multiple sources,” IEEE Transactions on Information Theory , vol. 50, no. 11, pp. 2762–2768, 2004
work page 2004
-
[21]
Computing the rate distortion region for the ceo problem with independent sources,
G. Ku, J. Ren, and J. M. Walsh, “Computing the rate distortion region for the ceo problem with independent sources,” IEEE Transactions on Signal Processing, vol. 63, no. 3, pp. 567–575, 2014
work page 2014
-
[1]
Semantic-preserved communication system for highly efficient speech transmission,
T. Han, Q. Yang, Z. Shi, S. He, and Z. Zhang, “Semantic-preserved communication system for highly efficient speech transmission,” IEEE Journal on Selected Areas in Communications , vol. 41, no. 1, pp. 245– 259, 2022
work page 2022
-
[2]
Two-terminal source coding with common sum reconstruction,
T. Adikari and S. Draper, “Two-terminal source coding with common sum reconstruction,” in 2022 IEEE International Symposium on Infor- mation Theory (ISIT) . IEEE, 2022, pp. 1420–1424
work page 2022
-
[3]
How to encode the modulo-two sum of binary sources (corresp.),
J. Korner and K. Marton, “How to encode the modulo-two sum of binary sources (corresp.),” IEEE Transactions on Information Theory , vol. 25, no. 2, pp. 219–221, 1979
work page 1979
-
[4]
Distributed lossy computation with structured codes: From discrete to continuous sources,
A. Pastore, S. H. Lim, C. Feng, B. Nazer, and M. Gastpar, “Distributed lossy computation with structured codes: From discrete to continuous sources,” in 2023 IEEE International Symposium on Information Theory (ISIT). IEEE, 2023, pp. 1681–1686
work page 2023
-
[5]
Federated learning with quantized global model updates,
M. M. Amiri, D. Gunduz, S. R. Kulkarni, and H. V . Poor, “Federated learning with quantized global model updates,” arXiv:2006.10672, 2020
arXiv 2006
Show all 20 references
-
[6]
Improving the worst- case bidirectional communication complexity for nonconvex distributed optimization under function similarity,
K. Gruntkowska, A. Tyurin, and P. Richt ´arik, “Improving the worst- case bidirectional communication complexity for nonconvex distributed optimization under function similarity,” arXiv:2402.06412, 2024
2024 arXiv
-
[7]
Convergence of federated learning over a noisy downlink,
M. M. Amiri, D. G ¨und¨uz, S. R. Kulkarni, and H. V . Poor, “Convergence of federated learning over a noisy downlink,” IEEE Transactions on Wireless Communications, vol. 21, no. 3, pp. 1422–1437, 2022
2022
-
[8]
The role of fidelity in goal-oriented semantic communication: A rate distortion approach,
P. A. Stavrou and M. Kountouris, “The role of fidelity in goal-oriented semantic communication: A rate distortion approach,” IEEE Transactions on Communications , 2023
2023
-
[9]
A rate-distortion framework for char- acterizing semantic information,
J. Liu, W. Zhang, and H. V . Poor, “A rate-distortion framework for char- acterizing semantic information,” in 2021 IEEE International Symposium on Information Theory (ISIT) . IEEE, 2021, pp. 2894–2899
2021
-
[10]
Information transmission with addi- tional noise,
R. Dobrushin and B. Tsybakov, “Information transmission with addi- tional noise,” IRE Transactions on Information Theory , vol. 8, no. 5, pp. 293–304, 1962
1962
-
[11]
Transmission of noisy information to a noisy receiver with minimum distortion,
J. Wolf and J. Ziv, “Transmission of noisy information to a noisy receiver with minimum distortion,” IEEE Transactions on Information Theory , vol. 16, no. 4, pp. 406–411, 1970
1970
-
[12]
Semantic compression with side information: A rate-distortion perspective,
T. Guo, Y . Wang, J. Han, H. Wu, B. Bai, and W. Han, “Semantic compression with side information: A rate-distortion perspective,” arXiv preprint arXiv:2208.06094, 2022
2022 arXiv
-
[14]
Noiseless coding of correlated information sources,
D. Slepian and J. Wolf, “Noiseless coding of correlated information sources,” IEEE Transactions on information Theory , vol. 19, no. 4, pp. 471–480, 1973
1973
-
[16]
Towards an algebraic network information theory: Distributed lossy computation of linear functions,
S. H. Lim, C. Feng, A. Pastore, B. Nazer, and M. Gastpar, “Towards an algebraic network information theory: Distributed lossy computation of linear functions,” in 2019 IEEE International Symposium on Information Theory (ISIT) . IEEE, 2019, pp. 1827–1831
2019
-
[19]
Computing the channel capacity and rate-distortion function with two-sided state information,
S. Cheng, V . Stankovic, and Z. Xiong, “Computing the channel capacity and rate-distortion function with two-sided state information,” IEEE transactions on information theory , vol. 51, no. 12, pp. 4418–4425, 2005
2005
-
[20]
C. Q., T. M. Cover, and J. A. Thomas, Elements of information theory . Elements of information theory, 2006
2006
-
[22]
The rate-distortion function for source coding with side information at the decoder,
A. Wyner and J. Ziv, “The rate-distortion function for source coding with side information at the decoder,” IEEE Transactions on information Theory, vol. 22, no. 1, pp. 1–10, 1976
1976
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.