REVIEW 2 major objections 3 minor 19 references
Multi-Terminal Remote Generation and Estimation Over a Broadcast Channel With Correlated Priors
T0 review · 2 major / 3 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read When decoder priors share a nontrivial common variable, a two-stage broadcast/unicast sampling scheme pays the block-level divergence once and then refines per decoder.
desk verdict The hierarchical sampling scheme is a genuine idea, but the paper's advertised communication-cost reduction is not actually established because Lemma 2's chi-squared factor can erase the block-level saving. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the two-stage (hierarchical) importance-sampling scheme built on the Gács-Körner common variable $C$, defined as the largest-entropy random variable that is a deterministic function of each decoder's prior, $C=g_1(Y_1)=g_2(Y_2)$. The first stage runs minimal random coding on the block variable $C$ with target $p_{QC}$ and prior $p_C$, producing one index $m_k$ that is broadcast to every decoder. The second stage runs conditional minimal random coding on each decoder's prior $p_{Y_i|C=c}$ with target $p_{Q|C=c}$, producing refinement indices $\ell_{i,k}$ sent over unicast. The machinery that carries the cost argument is the KL chain-rule decomposition $D_{KL}(p_Q\|p_{Y_i}) = D_{KL}(p_{QC}\|p_C) + \mathbb{E}_{Q_C}[D_{KL}(p_{Q|C}\|p_{Y_i|C})]$, which shows the block term is common to all decoders while only the conditional term depends on $i$. Theorem 1 bounds the bias of the two-stage estimator in terms of tail probabilities of the log importance ratios, and Lemma 2 converts this into an average communication-complexity bound.
What would settle it
Simulate the two-stage scheme on a block-structured prior of the form $Y_1=(C,A)$, $Y_2=(C,B)$ with independent non-atomic $C,A,B$ and a target $p_Q$ for which $p_{QC}\neq p_C$; compare the realized average bits per sample with $O(D_{KL}(p_{QC}\|p_C))$ plus the summed conditional divergence term. If the broadcast stage does not save the block term across decoders, or if the estimation error exceeds the Theorem 1 bound at the stated confidence, the paper's central claim is contradicted.
Extended reading notes
Core claim
The central discovery is a hierarchical sampling method for remote generation and estimation that exploits a common random variable shared by correlated priors. Given a nonzero Gács-Körner common information $C=g_1(Y_1)=g_2(Y_2)$, the encoder first draws $n_c$ samples from $p_C$, importance-samples one block index $m_k$ with respect to $p_{QC}$, and broadcasts $m_k$ to all decoders. Each decoder then draws $n_{i,c}$ samples from its conditional prior $p_{Y_i|C=c}$, importance-samples a refinement index $\ell_{i,k}$ with respect to $p_{Q|C=c}$, and receives it over unicast. The average communication cost per sample is $O(D_{KL}(p_{QC}\|p_C))$ for the broadcast stage plus $\frac{\chi^2(p_C,p_{QC})+1}{(n_c-1)/n_c}\mathbb{E}_{Q_C}\big[\sum_i O(D_{KL}(p_{Q|C}\|p_{Y_i|C}))\big]$ for the refinement stage, so the block-level divergence is paid once instead of once per decoder. Theorem 1 and Proposition 2 give finite-sample upper bounds on the bias and estimation error, and Corollary 2 shows that when the function $f$ is constant on each block the sample complexity reduces to the block level.
Load-bearing premise
The scheme's advantage requires the joint prior $p_{Y_1,Y_2}$ to have nonzero Gács-Körner common information, meaning there is a nontrivial block variable $C$ that is a deterministic function of both observations; for typical continuous or smoothly correlated priors this $C$ is trivial and the broadcast stage carries no savings.
Editorial extensions
If this is right
- If the priors have nonzero Gács-Körner common information, the two-stage scheme's average communication cost per sample is the block-level divergence plus the sum of per-decoder refinement divergences, so the shared part is paid once for all decoders.
- The block-level term can be reduced by choosing a non-maximal common partition that minimizes the KL divergences with the target, and the encoder can transmit the corresponding partitioning functions once, with the overhead vanishing as the number of samples $K$ grows.
- When the function $f$ is constant on each block of the partition, the point-to-point hierarchical scheme needs only block-level sample complexity, giving a tighter error bound than one-level minimal random coding.
- The scheme extends to more than two decoders and to multiple hierarchy levels through generalized common information, with the savings amplified as the number of receivers grows.
- The analysis provides finite-sample bias and estimation-error upper bounds, not just asymptotic rates, so the scheme comes with explicit sample counts for a target confidence level.
Reading between the lines
- Beyond the paper, one could approximate the scheme for continuous priors by coarsening the priors into blocks; the broadcast gain would then be limited by the fidelity of that coarsening, a tradeoff the paper does not quantify.
- A natural extension is to interpret the two stages as successive refinement, with the broadcast block as a coarse description and the unicast step as per-decoder innovation, which may connect this scheme to rate-distortion formulations of remote estimation.
- Because the paper deliberately omits improved index coding such as ordered random coding, replacing minimal random coding in both stages would likely preserve the once-per-block cost structure while shrinking the log-index terms in the communication cost.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper studies a multi-terminal remote estimation problem in which an encoder with access to a target distribution p_Q helps N decoders estimate (possibly different) functionals over p_Q, while each decoder has access to a correlated prior p_{Y_i} through shared randomness. The authors refine the known minimal random coding (MRC) importance-sampling bound into a high-probability estimation-error guarantee (Proposition 1), then introduce a hierarchical sampling scheme that first samples a common block variable C via MRC and then refines the sample conditionally on C for each decoder. In the multi-terminal setting, the common block is chosen as the Gács-Körner common information of the correlated priors, so that the block-level index can be broadcast and only refinement indices are sent over unicast links. The paper claims that this reduces communication cost because the block-level KL divergence is paid once instead of once per decoder, and it provides bias, error, and cost bounds (Theorem 1, Corollaries 1--2, Lemma 2).
Significance. The hierarchical MRC construction and the use of Gács-Körner common information to coordinate a broadcast sampling step are original and clearly presented. The formal bias and estimation-error bounds, together with the edge-case recovery of Lemma 1, are useful sanity checks and the unicast bounds are of independent interest. If the communication-cost reduction were rigorously established, the scheme would be a meaningful advance for multi-terminal estimation with block-correlated priors. However, the central claimed advantage is not proven: Lemma 2 contains a chi-squared factor multiplying the conditional-divergence terms, and the informal comparison in Section IV ignores this factor. Moreover, the regime of non-zero Gács-Körner common information is acknowledged in Remark 2 to be rare in practice, which limits the scope of the result regardless of the technical gap.
major comments (2)
- [Section IV, Lemma 2] The claimed communication-cost reduction is not established by the provided bounds. Lemma 2 upper-bounds the average hierarchical cost by O(D_KL(p_QC||p_C)) + ((chi^2(p_C,p_QC)+1)/((n_c-1)/n_c)) * E_{Q_C}[sum_i O(D_KL(p_{Q|C}||p_{Y_i|C}))], and Section IV then argues that the block-level divergence term is paid once rather than once per decoder. This ignores the multiplicative chi^2 factor on the conditional terms. Since chi^2(p_C,p_QC) can be exponentially large in D_KL(p_QC||p_C) (indeed chi^2+1 >= exp(D_KL) by Jensen's inequality), the second term can outweigh any saving from paying the block divergence once. No theorem states that the hierarchical cost is at most the standard unicast cost; the informal comparison in Section IV is therefore not sufficient for the abstract's claim that the scheme shows a reduction in communication cost. The authors should either provide a formal comparison with an explicit condition on chi^2 or a choice of n_c that keeps the factor bounded, or weaken the claim accordingly.
- [Section III-C and Section IV] The comparison between hierarchical and standard schemes is made only at the level of O(.) expressions, without tracking the constants and the required sample sizes needed to achieve the same estimation-error guarantees. The standard scheme's cost also depends on the n_i required by Lemma 1 for a prescribed confidence level, and the hierarchical scheme's cost depends on n_c and n_{i,c} in a coupled way. To substantiate the statement in Section V that 'significant gains can be achieved', the authors should state a concrete theorem or a family of priors for which, with identical target bias and error parameters (epsilon, hat-epsilon_i, K), the hierarchical per-sample expected communication cost is provably smaller than the standard per-decoder MRC cost. Without such a statement, the cost-reduction claim remains a heuristic comparison of upper bounds.
minor comments (3)
- [Section III-B, Corollary 2] The condition 'nc >= log(D_KL(p_QC||p_C)+t_c)' appears to be a typo: it should be 'nc >= exp(D_KL(p_QC||p_C)+t_c)', consistent with Theorem 1 and Lemma 1.
- [Section IV] The notation 'forall c,i in C' in the sentence 'When n_c >= exp(...) and forall c,i in C: n_{i,c} >= ...' is confusing: C is the alphabet of the common variable, while i indexes the decoders. It should be 'for all c in C and i in {1,2}'.
- [Section IV] The 'standard scheme' is described only informally as selecting an index from n_i samples for each decoder. For a rigorous comparison with the hierarchical scheme, the standard scheme should be stated with the same level of formality, including the exact number of samples n_i and the resulting per-transmission cost in bits.
Circularity Check
No significant circularity: the derivation is self-contained and builds on external benchmark results (Chatterjee–Diaconis and Gács–Körner), with no fitted inputs or load-bearing self-citations.
full rationale
The paper's derivation chain is not circular. The core bounds rest on Lemma 1, which is quoted from Chatterjee and Diaconis [3, Theorem 2.1], an external result with stated assumptions that do not include the target claim. The multi-terminal scheme invokes the Gács–Körner common information theorem [18], also an external mathematical fact, and uses it to guarantee the existence of the deterministic functions g1 and g2; this is an assumption about the prior structure, not a conclusion derived from the paper's own claims. The hierarchical sampling analysis (Theorem 1, Proposition 2, Lemma 2) is obtained by applying importance-sampling concentration to a two-stage MRC procedure, and the bounds are explicitly checked against degenerate partitions: when each block is a single symbol the bound reduces to the standard MRC-type bound, and when there is one block it recovers Lemma 1. These consistency checks are reductions to known results, not circular reasoning. The claimed communication-cost advantage is asserted informally in Section IV as the block-divergence term being incurred once rather than per decoder, but this is a comparison statement about the structure of the upper bound, not a fitted parameter renamed as a prediction. The only self-citation is reference [17] (BICompFL), which is a background citation in the introduction and is not load-bearing for any theorem, bound, or scheme in this paper. A possible weakness noted by a skeptical reader is that the Lemma 2 bound contains a chi-squared multiplicative factor that is not tracked in the prose comparison, so the claimed reduction in communication cost may not be established for all distributions; however, that is a correctness or tightness concern, not circularity. No step in the paper defines its output in terms of its input, fits a parameter and then predicts a closely related quantity, or imports a uniqueness conclusion solely from the authors' own prior work. The manuscript even acknowledges that Gács–Körner common information structures 'can be rare in practice' (Remark 2), which further indicates that the premise is an openly stated assumption rather than a conclusion forced by the analysis. Accordingly, the appropriate finding is no significant circularity.
Assumptions & free parameters
assumptions (4)
- standard math Lemma 1 (Chatterjee-Diaconis importance sampling sample-size bound) holds as stated.
- standard math Gács-Körner common information exists: for a joint distribution with non-zero common information there are deterministic functions g1,g2 yielding a common variable C, and conditioned on the maximal C the variables are independent.
- domain assumption The encoder can sample from p_Q and from p_Y1,Y2 and all conditionals p_Yi|C; decoders know the joint prior and can generate correlated samples via shared randomness.
- domain assumption The functions f_i are measurable and integrable enough for the L2/L4 norms to be finite.
Cite this review
Pith. "Pith review of Multi-Terminal Remote Generation and Estimation Over a Broadcast Channel With Correlated Priors." pith.science (2026). https://pith.science/paper/F3RQSTDZ
@misc{pith2026250507016,
author = {Pith},
title = {Pith review of: Multi-Terminal Remote Generation and Estimation Over a Broadcast Channel With Correlated Priors},
year = {2026},
howpublished = {\url{https://pith.science/paper/F3RQSTDZ}},
note = {Machine review of arXiv:2505.07016}
}
read the original abstract
We study the multi-terminal remote estimation problem under a rate constraint, in which the goal of the encoder is to help each decoder estimate a function over a certain distribution -- while the distribution is known only to the encoder, the function to be estimated is known only to the decoders, and can also be different for each decoder. The decoders can observe correlated samples from prior distributions, instantiated through shared randomness with the encoder. To achieve this, we employ remote generation, where the encoder helps decoders generate samples from the underlying distribution by using the samples from the prior through importance sampling. While methods such as minimal random coding can be used to efficiently transmit samples to each decoder individually using their importance scores, it is unknown if the correlation among the samples from the priors can reduce the communication cost using the availability of a broadcast link. We propose a hierarchical importance sampling strategy that facilitates, in the case of non-zero G\'acs-K\"orner common information among the priors of the decoders, a common sampling step leveraging the availability of a broadcast channel. This is followed by a refinement step for the individual decoders. We present upper bounds on the bias and the estimation error for unicast transmission, which is of independent interest. We then introduce a method that splits into two phases, dedicated to broadcast and unicast transmission, respectively, and show the reduction in communication cost.
Figures
Reference graph
Works this paper leans on
-
[1]
Generative adversarial nets,
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adversarial nets,” Advances in neural information processing systems, vol. 27, 2014
2014
-
[2]
Auto-encoding variational bayes,
D. P. Kingma, M. Wellinget al., “Auto-encoding variational bayes,” in International Conference on Learning Representations, 2014
work page 2014
-
[3]
The sample size required in importance sampling,
S. Chatterjee and P. Diaconis, “The sample size required in importance sampling,”The Annals of Applied Probability, vol. 28, no. 2, pp. 1099– 1135, 2018
work page 2018
-
[4]
Minimal random code learning: Getting bits back from compressed model parameters,
M. Havasi, R. Peharz, and J. M. Hernández-Lobato, “Minimal random code learning: Getting bits back from compressed model parameters,” inInternational Conference on Learning Representations, 2019
work page 2019
-
[5]
Algorithms for the communication of samples,
L. Theis and N. Yosri, “Algorithms for the communication of samples,” inInternational Conference on Machine Learning. PMLR, 2022, pp. 21 308–21 328
work page 2022
-
[6]
Channel simulation: Theory and applications to lossy com- pression and differential privacy,
C. T. Li, “Channel simulation: Theory and applications to lossy com- pression and differential privacy,”Foundations and Trends® in Commu- nications and Information Theory, vol. 21, no. 6, pp. 847–1106, 2024
2024
-
[7]
Some Notes on the Sample Complexity of Approximate Channel Simulation
G. Flamich and L. Wells, “Some notes on the sample complexity of approximate channel simulation,”arXiv preprint arXiv:2405.04363, 2024
work page Pith review arXiv 2024
-
[8]
Compressing images by encoding their latent representations with relative entropy coding,
G. Flamich, M. Havasi, and J. M. Hernández-Lobato, “Compressing images by encoding their latent representations with relative entropy coding,”Advances in Neural Information Processing Systems, vol. 33, pp. 16 131–16 141, 2020
work page 2020
Show all 19 references
-
[9]
Strong functional representation lemma and applications to coding theorems,
C. T. Li and A. El Gamal, “Strong functional representation lemma and applications to coding theorems,”IEEE Transactions on Information Theory, vol. 64, no. 11, pp. 6967–6978, 2018
2018
-
[10]
Fast relative entropy coding with a* coding,
G. Flamich, S. Markou, and J. M. Hernández-Lobato, “Fast relative entropy coding with a* coding,” inInternational Conference on Machine Learning. PMLR, 2022, pp. 6548–6577
2022
-
[11]
Faster relative entropy coding with greedy rejection coding,
——, “Faster relative entropy coding with greedy rejection coding,” Advances in Neural Information Processing Systems, vol. 36, 2024
2024
-
[12]
Accelerating relative entropy coding with space partitioning,
J. He, G. Flamich, and J. M. Hernández-Lobato, “Accelerating relative entropy coding with space partitioning,”arXiv preprint arXiv:2405.12203, 2024
2024 arXiv
-
[13]
Importance matching lemma for lossy compression with side information,
B. Phan, A. Khisti, and C. Louizos, “Importance matching lemma for lossy compression with side information,” inInternational Conference on Artificial Intelligence and Statistics. PMLR, 2024, pp. 1387–1395
2024
-
[14]
Information compression in the ai era: Recent advances and future challenges,
J. Chen, Y . Fang, A. Khisti, A. Ozgur, N. Shlezinger, and C. Tian, “Information compression in the ai era: Recent advances and future challenges,”arXiv preprint arXiv:2406.10036, 2024
2024 arXiv
-
[15]
Dp-rec: Private & communication-efficient federated learning,
A. Triastcyn, M. Reisser, and C. Louizos, “Dp-rec: Private & communication-efficient federated learning,”arXiv preprint arXiv:2111.05454, 2021
2021 arXiv
-
[16]
Adaptive compression in federated learning via side information,
B. Isik, F. Pase, D. Gunduz, S. Koyejo, T. Weissman, and M. Zorzi, “Adaptive compression in federated learning via side information,” in International Conference on Artificial Intelligence and Statistics, 2024, pp. 487–495
2024
-
[17]
BICompFL: Stochastic federated learning with bi-directional compres- sion,
M. Egger, R. Bitar, A. Wachter-Zeh, N. Weinberger, and D. Gündüz, “BICompFL: Stochastic federated learning with bi-directional compres- sion,”arXiv preprint arXiv:2502.00206, 2025
2025 arXiv
-
[18]
Common information is far less than mutual information,
P. Gacs and J. Körner, “Common information is far less than mutual information,”Problems of Control and Information Theory, vol. 2, 01 1973
1973
-
[19]
When is a function securely computable?
H. Tyagi, P. Narayan, and P. Gupta, “When is a function securely computable?”IEEE Transactions on Information Theory, vol. 57, no. 10, pp. 6337–6350, 2011
2011
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.