Estimating Rate-Distortion Functions Using the Energy-Based Model

Huihui Wu; Lingyi Chen; Shitong Wu; Sicheng Xu; Wenyi Zhang

REVIEW 4 major objections 5 minor 28 references

Estimating Rate-Distortion Functions Using the Energy-Based Model

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single energy network can estimate high-dimensional rate-distortion functions and the optimal conditional distribution.

desk verdict A clean and genuinely new single-network EBRD estimator, but the only theoretical guarantee is invalid for the paper's own Gaussian experiments and the empirical evidence lacks error bars and the obvious baseline. read the letter →

arxiv 2507.15700 v2 pith:NM6ZNU3N submitted 2025-07-21 cs.IT math.IT

Shitong Wu , Sicheng Xu , Lingyi Chen , Huihui Wu , Wenyi Zhang This is my paper

classification cs.ITmath.IT MSC 94A3494A1768T07

keywords rate-distortionfunctionenergy-basedmodelBlahut-ArimotoalgorithmLangevinMonteCarloneuralnetworklossysourcecodingfreeenergydualformulation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the rate-distortion function—the minimal compression rate achievable at a fixed distortion—can be estimated by training a single neural network that acts as an energy function, and that the same network also yields the optimal conditional distribution for reconstruction. Classical iterative algorithms such as the BA algorithm fail in high dimensions because they require discretizing the reproduction space, while recent neural alternatives either estimate only the marginal distribution or rely on an auxiliary mutual-information estimator. The authors show that the dual form of the rate-distortion problem is a free energy, so representing the optimal marginal as a Boltzmann distribution makes the optimal conditional a tilted version of the same energy. Their training update is a contrastive gradient between samples drawn from the conditional and the marginal by Langevin Monte Carlo, which cancels the intractable normalizing constants. If the method works as claimed, high-dimensional rate-distortion curves and the associated optimal decoders become computable with one network.

What carries the argument

The load-bearing object is the energy-based Boltzmann parametrization $q_\theta(y)=e^{-E_\theta(y)}/Z_\theta$, where $Z_\theta=\int_{\mathcal{Y}}e^{-E_\theta(y)}dy$. The tilted energy $E'_\theta(x,y)=E_\theta(y)+\beta\rho(x,y)$ turns it into the optimal conditional $p_\theta(y|x)=e^{-E'_\theta(x,y)}/\int_{\mathcal{Y}}e^{-E'_\theta(x,y)}dy$. Because the distortion weight enters additively, the gradient of the dual objective splits into two expectations over the same network gradient, one under the conditional-induced marginal $P^\theta_Y$ and one under the model marginal $Q^\theta_Y$, and the normalizing constants cancel in their difference. Langevin MCMC supplies samples from both distributions using only gradients of $E_\theta$ and $E'_\theta$, so the algorithm trains a single network by contrastive gradient descent, with no partition function evaluation and no auxiliary mutual-information estimator.

What would settle it

Take a vector Gaussian source with a known closed-form rate-distortion curve, run EBRD at several values of $\beta$ with increasing Langevin step counts and batch sizes, and check whether the estimated $(R,D)$ pairs converge to the theoretical curve; if the gap cannot be closed by more sampling and a tuned step size, the gradient estimator is not faithful and the central claim fails.

Watch

Extended reading notes

Core claim

The central discovery is an energy-based reformulation of the rate-distortion problem in which one neural network $E_\theta(y)$ encodes everything needed. Writing the optimal marginal as $q_\theta(y)=e^{-E_\theta(y)}/\int_{\mathcal{Y}} e^{-E_\theta(y)}dy$ and the optimal conditional as $p_\theta(y|x)=e^{-E_\theta(y)-\beta\rho(x,y)}/\int_{\mathcal{Y}} e^{-E_\theta(y)-\beta\rho(x,y)}dy$, the paper observes that the rate-distortion dual objective $L(\theta)$ is a free energy and derives a tractable gradient $\nabla_\theta L(\theta)=\mathbb{E}_{P^\theta_Y}[\nabla_\theta E_\theta(Y)]-\mathbb{E}_{Q^\theta_Y}[\nabla_\theta E_\theta(Y)]$ in which the partition functions cancel. The gradient is estimated by Langevin MCMC from the conditional and marginal Boltzmann distributions, and gradient descent on $\theta$ trains the energy network. The paper also proves, under a uniform-approximation and compact-parameter assumption, that the Monte Carlo estimate of the objective converges almost surely to the true rate-distortion dual optimum, and demonstrates accurate RD curves for scalar Gaussian and Laplacian sources, vector Gaussian sources in dimensions 2, 5, and 10, and visually faithful conditional reconstruction on a Gaussian mixture.

Load-bearing premise

The method stands or falls on whether Langevin Monte Carlo draws from the tilted conditional and marginal energy distributions are close enough to exact samples that the estimated gradient points the right way, and on whether a neural network in the chosen family can represent the optimal energy well enough; neither is guaranteed in the high-dimensional cases the paper targets.

Editorial extensions

If this is right

High-dimensional rate-distortion curves become estimable where BA-style discretization is exponential, because the same energy network is updated by gradient descent on samples.
The optimal conditional distribution $p(y|x)$ comes out of the same network, giving a reconstruction map for lossy compression without training a separate decoder.
The algorithm needs no normalizing constant and no auxiliary mutual-information estimator, simplifying the pipeline compared with prior energy-based rate-distortion work.
As $\beta\to\infty$, conditional samples coincide with the source, so the training reduces to ordinary energy-based model training, connecting the method to established practice.
The consistency result justifies estimating $(R,D)$ by plugging samples into the empirical dual objective.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the learned energy is a parametric density for the optimal reproduction marginal, the same network could in principle be reused as the sampling side of an operational lossy codec; the paper does not develop this.
A likely practical bottleneck is MCMC mixing: the theory assumes exact samples and a sufficiently expressive network, so in very high dimensions or at high rates accuracy depends on chain length and step size, as the paper's own vector Gaussian results begin to show.
The gradient identity holds for any distortion measure, so the method is not tied to squared error; testing it on perceptually motivated distortions would be a natural extension.
The consistency theorem relies on a compact parameter domain and uniform approximation of the optimal energy; extending it to unbounded energy networks and unbounded reproduction spaces would be needed for full rigor.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The new thing here is the specific training loss and gradient identity (4): one energy network, with the gradient expressed as a difference between expectations under the conditional and marginal energy distributions. That is a real algorithmic variant, distinct from [13]'s auxiliary mutual-information estimator, and Proposition 1's derivation is clean. The experiments on scalar Gaussian and Laplacian sources land close to the theoretical curves, and the conditional-sampling visualization on the Gaussian mixture is qualitatively consistent with what the RD theory predicts. Credit where due: this is a sensible and simple way to get both the curve and the conditional model from a single network.

The soft spots are substantial. The stress-test note is right: Theorem 1's proof assumes a neural network can uniformly approximate the optimal energy function on the whole reproduction space, and for the Gaussian source with squared distortion that optimal energy is unbounded on R. The approximation step fails, so the claimed asymptotic guarantee does not cover the paper's own headline experiments. Separately, even if that assumption were repaired, the theorem concerns the infimum of the empirical loss over a compact parameter set, not the point reached by the Langevin-SGD trajectory on a nonconvex objective. There is no mixing or bias analysis for the finite-step Langevin chains, which is the engine of the algorithm. The empirical section also omits error bars and does not compare against [13], the closest prior method, in any of the figures. These are not minor quibbles; the central claim that EBRD 'estimates' the RD function currently rests on curve matching without a valid proof or a controlled baseline.

That said, the paper is not incoherent. The objective is the true RD dual, and the estimated (R,D) pairs are self-consistent with the learned conditional model. If the training dynamics work in practice, the method is useful. But the evidence that it reliably works in high dimensions is thinner than the abstract suggests.

Who gets value: researchers working on neural approaches to rate-distortion and energy-based sampling. I would not cite it in my own work until the theory is repaired and the baseline/error-bar gap is closed, but it deserves a serious referee. The right call is to send it to review with a clear demand that Theorem 1 be fixed or properly scoped, and that the experiments include [13] and variance over seeds.

Referee Report

4 major / 5 minor

Summary. The paper proposes EBRD, an energy-based method for estimating rate-distortion (RD) functions of continuous sources. The method represents the reproduction marginal q(y) by a Boltzmann distribution e^{-E_θ(y)}/Z_θ and optimizes the dual objective L(θ) = -E_{P_X}[log ∫ e^{-E_θ(y)-βρ(X,y)}dy - log ∫ e^{-E_θ(y)}dy], with the gradient (4) estimated by Langevin MCMC samples from the conditional and marginal models. After training, (R,D) pairs are obtained from the empirical dual objective and distortion estimator in Appendix A. Experiments cover scalar Gaussian and Laplacian sources, vector Gaussian sources, and conditional-sampling visualizations for a two-dimensional Gaussian mixture. The paper also states a consistency theorem (Theorem 1) for the empirical dual objective.

Significance. Proposition 1 is a clean and useful identity: it expresses the gradient of the RD dual objective as a difference of energy-gradient expectations under the conditional and marginal energy-based models, avoiding both the normalization constant and an auxiliary mutual-information estimator. If the optimization and sampling issues are resolved, the method would be an attractive single-network alternative to prior neural RD estimators, and the reconstruction of the optimal conditional distribution is a valuable feature. The experimental setup, using closed-form RD curves and NERD as a baseline, is appropriate for validation. However, the theoretical support is currently incomplete in ways that affect the central claim, and the high-dimensional empirical claim lacks quantitative backing.

major comments (4)

[Appendix A, Theorem 1] The proof of Theorem 1 relies on the assumption that the optimal energy E*(y) can be uniformly approximated on the whole reproduction space Y by a neural network. For the paper's own Gaussian experiments (§IV-A and §IV-B), Y = R or R^d and the optimal marginal for X~N(0,1) with squared distortion is q*(y) = N(0,1-D), so E*(y) = y^2/(2(1-D)) + const is unbounded. A finite neural network (for example, a ReLU network, which is eventually linear on each unbounded ray) cannot approximate this function uniformly on all of R. Consequently, the step 'there exist Θ and θhat ∈ Θ such that |E*(y)-E_θhat(y)|<ξ for all y∈Y' cannot hold, and the derivation of |F*−F_Θ|<ε/2 is not valid for the examples reported in the paper.
[Appendix A, proof of Theorem 1] Independently of the unbounded-domain issue, the proof does not justify the passage from pointwise laws of large numbers to the uniform bound used in the inequality |bF_N−F_Θ| ≤ sup_{θ∈Θ}|bL(θ)−L(θ)|. The text invokes the LLN for each θ without proving that N_0 can be chosen uniformly over the compact set Θ; an argument based on equicontinuity, bracketing, or a uniform LLN is needed. In addition, the displayed bound on |q*(y)−q_θhat(y)| divides by Z_θhat without a lower bound, so that part of the proof is also incomplete. These gaps must be repaired for any consistency statement of the form (6).
[Proposition 1 and Algorithm 1] Equation (4) is the exact gradient only when the expectations are taken under the exact conditional distribution P^θ_{Y|X} and marginal Q^θ_Y. Algorithm 1 replaces these by K-step Langevin chains with a fixed step size ε initialized from N(0,I), and no bias, mixing, or convergence analysis is supplied; for the multimodal Gaussian-mixture target of §IV-C, the fixed initialization and finite K may not explore the target adequately. Moreover, Theorem 1 concerns inf_{θ∈Θ} bL(θ), not the parameter reached by stochastic gradient descent on the nonconvex objective L(θ), and no consistency is proved for the distortion estimator D or for R = bL(θ)−βD at the end of Appendix A. The paper therefore does not rigorously connect the trained energy function E_θ to F*.
[Section IV-B, Fig. 3] The abstract's high-dimensional claim is not quantitatively supported. For d=10, the figure shows both EBRD and NERD deviating from the theoretical curve at high rates, and no error bars, multiple-seed statistics, or sensitivity analysis with respect to β, ε, K, and N are reported. In addition, the closest prior work [13], which the paper explicitly positions against, is never compared experimentally, so the claimed advantage over [13] in accuracy or simplicity is not demonstrated.

minor comments (5)

[Section II-B and Section IV-A] There are typos: 'alphbet' should be 'alphabet' in Section II-B, and 'correspoding' should be 'corresponding' in Section IV-A.
[Appendix A, Eq. (5)] The manuscript should specify how the samples y_j in (5) are obtained: whether they are fresh Langevin MCMC draws from the trained Q^θ_Y, how many such samples are used, and whether they are independent of the x_i samples; otherwise the bias of the estimator is unclear.
[Figures 2-6] Reporting single-run results without error bars is insufficient given the stochasticity of Langevin MCMC and SGD; please add at least mean and standard deviation over several independent runs.
[Section III-B] The statement that the classical EBM and the EBRD model become 'asymptotically equivalent' as β→∞ is informal; a precise statement, for example in terms of gradient bias or distributional convergence, would be helpful.
[Proposition 1] The derivation assumes interchange of differentiation and integration; a brief regularity condition on E_θ and ρ would make the proof complete.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the derivation is a standard variational dual with an energy parametrization, the RD estimate is benchmarked against closed-form curves, and the only self-citation is non-load-bearing.

full rationale

The derivation chain is not circular. The paper starts from the standard RD dual F(q) = -E_PX[log ∫ q(y)e^{-βρ(X,y)} dy] (Eq. 2), reparametrizes q via an energy function, and derives the gradient (4) exactly using the Boltzmann conditional pθ(y|x) = e^{-Eθ(y)-βρ(x,y)}/Zθ(x). This is a legitimate variational reparametrization: the energy model is the optimization variable, not the target RD value itself. The estimated rate in Appendix A, R = bL(θ) - βD, is the standard Lagrangian identity for the model's own conditional distribution; it is not asserted to equal the true RD function by construction. The correctness of the RD estimate rests on whether minimizing L(θ) reaches the optimal dual value, which the paper attempts to justify via Theorem 1 and tests against independent closed-form Gaussian, Laplacian, and vector-Gaussian curves. Even if Theorem 1's uniform-approximation argument is flawed on unbounded reproduction spaces, that is a correctness or rigor concern, not circularity. The only self-citation, [6], is used to motivate the exponential complexity of discretized BA algorithms and carries no load in the derivation. No equation or fitted parameter is renamed as a prediction, and no load-bearing premise is justified solely by a self-citation chain. Therefore the paper does not exhibit the reduction patterns required for a circularity finding.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central derivation relies on standard RD duality, the energy-based parameterization from [13], and several unproved regularity and convergence assumptions for MCMC and neural approximation. No new physical entities are introduced, and no constants are fitted to data; the main free choices are hyperparameters of the optimizer and sampler.

free parameters (4)

Lagrangian multiplier beta = swept across values (not fitted)
Chosen by hand to trace the RD curve; each value yields one (R,D) point. Not fitted to data.
Langevin step size epsilon = 1.2e-2 for Gaussian, 3.5e-2 for Laplacian; others unstated
MCMC hyperparameter chosen per experiment; the algorithm's accuracy depends on it.
MCMC steps K = 50 for Gaussian, 80 for Laplacian; others unstated
MCMC iteration count chosen by hand; affects sample quality and gradient accuracy.
Batch size N = not stated in main text
Monte Carlo sample count for gradient and R/D estimates; the asymptotic theorem requires large N.

assumptions (6)

standard math The rate-distortion dual form F(q) = -E_P log ∫ q(y)e^{-βρ}dy (Eq. 2) equals the free energy and its minimum gives the RD function at slope β.
Classical rate-distortion duality from Rose [21] and Landau-Lifshitz [15], used as the optimization objective.
domain assumption Lemma 1 from [13]: representing the optimal marginal q(y) by energy E_θ(y) lets the conditional p(y|x) be represented by E_θ(y)+βρ(x,y).
Adopted from [13] without proof; foundational to the single-network parameterization.
domain assumption Langevin MCMC samples converge to the target Boltzmann distributions so the gradient estimates in (4) are unbiased.
Algorithm 1 relies on this; no mixing conditions or step-size schedules are given.
ad hoc to paper In Theorem 1, the optimal density satisfies q*(y)=e^{-E*(y)}/Z* with E*(y)>0 and is uniformly approximable by a neural network on the whole, possibly unbounded, reproduction space Y.
Universal approximation is invoked without compactness or growth conditions on Y; not justified for Gaussian and Laplacian tails.
domain assumption Existence of a compact parameter domain Θ and boundedness of ∫ e^{-βρ(x,y)}dy (Theorem 1).
Stated in Theorem 1; needed for the empirical-loss consistency proof.
ad hoc to paper Gradient descent on the nonconvex loss L(θ) reaches a near-global optimum.
Theorem 1 only shows the infimum of the empirical loss converges; it does not analyze the SGD or MCMC training trajectory used in Algorithm 1.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Estimating Rate-Distortion Functions Using the Energy-Based Model." pith.science (2026). https://pith.science/paper/NM6ZNU3N

@misc{pith2026250715700,
  author       = {Pith},
  title        = {Pith review of: Estimating Rate-Distortion Functions Using the Energy-Based Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NM6ZNU3N}},
  note         = {Machine review of arXiv:2507.15700}
}

read the original abstract

The rate-distortion (RD) theory is one of the key concepts in information theory, providing theoretical limits for compression performance and guiding the source coding design, with both theoretical and practical significance. The Blahut-Arimoto (BA) algorithm, as a classical algorithm to compute RD functions, encounters computational challenges when applied to high-dimensional scenarios. In recent years, many neural methods have attempted to compute high-dimensional RD problems from the perspective of implicit generative models. Nevertheless, these approaches often neglect the reconstruction of the optimal conditional distribution or rely on unreasonable prior assumptions. In face of these issues, we propose an innovative energy-based modeling framework that leverages the connection between the RD dual form and the free energy in statistical physics, achieving effective reconstruction of the optimal conditional distribution.The proposed algorithm requires training only a single neural network and circumvents the challenge of computing the normalization factor in energy-based models using the Markov chain Monte Carlo (MCMC) sampling. Experimental results demonstrate the significant effectiveness of the proposed algorithm in estimating high-dimensional RD functions and reconstructing the optimal conditional distribution.

Figures

Figures reproduced from arXiv: 2507.15700 by the authors.

**Figure 3.** Comparison of EBRD and NERD algorithms with the [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

**Figure 5.** Visualization of conditional sampling with different [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗

Figures from the paper (1 more)

**Figure 6.** Figure 6: Comparison between the original distribution and the [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

28 extracted references · 19 canonical work pages

[13]

Rate-distortion theory by and for energy-based models,

Q. Li and C. Guyot, “Rate-distortion theory by and for energy-based models,” IEEE Transactions on Communications , 2024

work page 2024
[1]

A mathematical theory of communication,

C. E. Shannon, “A mathematical theory of communication,” The Bell System Technical Journal, vol. 27, no. 3, pp. 379–423, 1948

1948
[2]

Coding theorems for a discrete source with a fidelity criterion,

C. E. Shannon et al. , “Coding theorems for a discrete source with a fidelity criterion,” IRE Nat. Conv. Rec , vol. 4, no. 142-163, p. 1, 1959

work page 1959
[3]

End-to-end optimized image compression,

J. Ball ´e, V . Laparra, and E. P. Simoncelli, “End-to-end optimized image compression,” in 5th International Conference on Learning Representa- tions, ICLR 2017 , 2017

work page 2017
[4]

Nonlinear transform coding,

J. Ball ´e, P. A. Chou, D. Minnen, S. Singh, N. Johnston, E. Agustsson, S. J. Hwang, and G. Toderici, “Nonlinear transform coding,” IEEE Journal of Selected Topics in Signal Processing , vol. 15, no. 2, pp. 339– 353, 2020

work page 2020
[5]

Computation of channel capacity and rate-distortion func- tions,

R. Blahut, “Computation of channel capacity and rate-distortion func- tions,” IEEE transactions on Information Theory , vol. 18, no. 4, pp. 460–473, 1972

work page 1972
[6]

On convergence of discrete schemes for computing the rate-distortion function of continuous source,

L. Chen, S. Wu, W. Zhang, H. Wu, and H. Wu, “On convergence of discrete schemes for computing the rate-distortion function of continuous source,” in 2024 IEEE International Symposium on Information Theory (ISIT), 2024, pp. 410–415

work page 2024
[7]

Neural estimation of the rate- distortion function with applications to operational source coding,

E. Lei, H. Hassani, and S. S. Bidokhti, “Neural estimation of the rate- distortion function with applications to operational source coding,” IEEE Journal on Selected Areas in Information Theory , vol. 3, no. 4, pp. 674– 686, 2022

work page 2022

Show all 28 references

[8]

Towards empirical sandwich bounds on the rate-distortion function,

Y . Yang and S. Mandt, “Towards empirical sandwich bounds on the rate-distortion function,” in International Conference on Learning Rep- resentations, 2022

2022
[9]

beta-vae: Learning basic visual concepts with a constrained variational framework

I. Higgins, L. Matthey, A. Pal, C. P. Burgess, X. Glorot, M. M. Botvinick, S. Mohamed, and A. Lerchner, “beta-vae: Learning basic visual concepts with a constrained variational framework.” ICLR (Poster), vol. 3, 2017

2017
[10]

Estimating the rate- distortion function by wasserstein gradient descent,

Y . Yang, S. Eckstein, M. Nutz, and S. Mandt, “Estimating the rate- distortion function by wasserstein gradient descent,” Advances in Neural Information Processing Systems , vol. 36, 2024

2024
[11]

Denoising diffusion probabilistic models,

J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems , vol. 33, pp. 6840– 6851, 2020

2020
[12]

Score-based generative modeling through stochastic differen- tial equations,

Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differen- tial equations,” in 9th International Conference on Learning Represen- tations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 , 2021

2021
[14]

Mutual information neural estimation,

M. I. Belghazi, A. Baratin, S. Rajeswar, S. Ozair, Y . Bengio, R. D. Hjelm, and A. C. Courville, “Mutual information neural estimation,” in Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsm ¨assan, Stockholm, Sweden, July 10-15, 2018 ...

2018
[15]

L. D. Landau and E. M. Lifshitz, Statistical Physics, Part I , 3rd ed. Oxford: Pergamon, 1980

1980
[16]

Correlation functions and computer simulations,

G. Parisi, “Correlation functions and computer simulations,” Nuclear Physics B, vol. 180, no. 3, pp. 378–384, 1981

1981
[17]

Representations of knowledge in complex systems,

U. Grenander and M. I. Miller, “Representations of knowledge in complex systems,” Journal of the Royal Statistical Society: Series B (Methodological), vol. 56, no. 4, pp. 549–581, 1994

1994
[18]

A tutorial on energy-based learning,

Y . LeCun, S. Chopra, R. Hadsell, M. Ranzato, F. Huang et al., “A tutorial on energy-based learning,” Predicting structured data, vol. 1, no. 0, 2006

2006
[19]

How to train your energy-based models,

Y . Song and D. P. Kingma, “How to train your energy-based models,” arXiv preprint arXiv:2101.03288 , 2021

2021 arXiv
[20]

Max- imum entropy generators for energy-based models,

R. Kumar, S. Ozair, A. Goyal, A. Courville, and Y . Bengio, “Max- imum entropy generators for energy-based models,” arXiv preprint arXiv:1901.08508, 2019

1901 arXiv
[21]

A mapping approach to rate-distortion computation and analysis,

K. Rose, “A mapping approach to rate-distortion computation and analysis,” IEEE Transactions on Information Theory , vol. 40, no. 6, pp. 1939–1952, 1994

1939
[22]

T. M. Cover, Elements of Information Theory . John Wiley & Sons, 1999

1999
[23]

Rate-distortion theory,

T. Berger, “Rate-distortion theory,” Wiley Encyclopedia of Telecommuni- cations, 2003

2003
[24]

Multilayer feedforward networks are universal approximators,

K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural Networks , vol. 2, no. 5, pp. 359–366, 1989

1989
[25]

Universal approximation with deep narrow networks,

P. Kidger and T. Lyons, “Universal approximation with deep narrow networks,” in Conference on Learning Theory (COLT) , 2020, pp. 2306– 2327

2020
[26]

R. M. Gray, Probability, Random Processes, and Ergodic Properties . Springer Science & Business Media, 2009. APPENDIX A ESTIMATION OF RATE AND DISTORTION Similar to [7], we generate samples from the distribution Y using the well-trained energy function Eθ, and subsequently es-...

2009
[27]

From this definition, one obtains F ∗ ≤ FΘ ≤ L(ˆθ), and then |F ∗ − FΘ| < ε 2

Next, define FΘ = inf θ∈Θ L(ˆθ). From this definition, one obtains F ∗ ≤ FΘ ≤ L(ˆθ), and then |F ∗ − FΘ| < ε 2 . (7) Further, one has | bFN − FΘ| = | inf θ∈Θ bL(θ) − inf θ∈Θ L(θ)| ≤sup θ∈Θ | bL(θ) − L(θ)|. It follows from the law of large numbers [26] that for any given ϵ > 0,...
[28]

(8) Combining (7) and (8), one finally has |F ∗ − bFN | < ε,∀ N > N0

Therefore | bFN − FΘ| < ε 2 . (8) Combining (7) and (8), one finally has |F ∗ − bFN | < ε,∀ N > N0. This concludes the proof of the theorem

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [13]

Rate-distortion theory by and for energy-based models,

Q. Li and C. Guyot, “Rate-distortion theory by and for energy-based models,” IEEE Transactions on Communications , 2024

work page 2024

[2] [1]

A mathematical theory of communication,

C. E. Shannon, “A mathematical theory of communication,” The Bell System Technical Journal, vol. 27, no. 3, pp. 379–423, 1948

1948

[3] [2]

Coding theorems for a discrete source with a fidelity criterion,

C. E. Shannon et al. , “Coding theorems for a discrete source with a fidelity criterion,” IRE Nat. Conv. Rec , vol. 4, no. 142-163, p. 1, 1959

work page 1959

[4] [3]

End-to-end optimized image compression,

J. Ball ´e, V . Laparra, and E. P. Simoncelli, “End-to-end optimized image compression,” in 5th International Conference on Learning Representa- tions, ICLR 2017 , 2017

work page 2017

[5] [4]

Nonlinear transform coding,

J. Ball ´e, P. A. Chou, D. Minnen, S. Singh, N. Johnston, E. Agustsson, S. J. Hwang, and G. Toderici, “Nonlinear transform coding,” IEEE Journal of Selected Topics in Signal Processing , vol. 15, no. 2, pp. 339– 353, 2020

work page 2020

[6] [5]

Computation of channel capacity and rate-distortion func- tions,

R. Blahut, “Computation of channel capacity and rate-distortion func- tions,” IEEE transactions on Information Theory , vol. 18, no. 4, pp. 460–473, 1972

work page 1972

[7] [6]

On convergence of discrete schemes for computing the rate-distortion function of continuous source,

L. Chen, S. Wu, W. Zhang, H. Wu, and H. Wu, “On convergence of discrete schemes for computing the rate-distortion function of continuous source,” in 2024 IEEE International Symposium on Information Theory (ISIT), 2024, pp. 410–415

work page 2024

[8] [7]

Neural estimation of the rate- distortion function with applications to operational source coding,

E. Lei, H. Hassani, and S. S. Bidokhti, “Neural estimation of the rate- distortion function with applications to operational source coding,” IEEE Journal on Selected Areas in Information Theory , vol. 3, no. 4, pp. 674– 686, 2022

work page 2022