Progressive Compression with Universally Quantized Diffusion Models

Justus C. Will; Stephan Mandt; Yibo Yang

REVIEW 3 major objections 6 minor 14 references

Progressive Compression with Universally Quantized Diffusion Models

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Replacing Gaussian noise with uniform noise makes a diffusion model's training loss equal its actual progressive-compression bit cost.

desk verdict A sound theoretical core — uniform-noise diffusion turns the NELBO into an actual bitrate — but the practical claims outrun the evidence, which needs per-step rate analysis and code. read the letter →

arxiv 2412.10935 v2 pith:KWRTFVGT submitted 2024-12-14 cs.LG cs.CV

Yibo Yang , Justus C. Will , Stephan Mandt This is my paper · ORCID

classification cs.LGcs.CV

keywords progressiveimagecompressiondiffusionmodelsuniversalquantizationuniformnoisechannelvariationalrate-distortionrate-realismneuralcodecs

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a diffusion-based image codec whose training loss—the negative evidence lower bound (ELBO)—is, by construction, an actual bitrate. Replacing the Gaussian noise in the forward process with uniform noise lets the sender communicate each latent through universal quantization, and the paper shows that the negative ELBO of the resulting model equals the end-to-end compression cost. The payoff is progressive coding: bits can be transmitted incrementally and decoded with progressively improving quality, all with a single model, and compute grows linearly with the number of diffusion steps rather than exponentially as in Gaussian-diffusion codecs. On CIFAR-10 and ImageNet 64×64 the method beats JPEG and JPEG2000 across all bit-rates, matches or exceeds the progressive neural codec CTC, and keeps improving at high bit-rates where other codecs plateau.

What carries the argument

The load-bearing object is the uniform noise channel and its simulator, universal quantization. Given a latent $Y$, the sender perturbs it with a shared dither $U' \sim U(-\Delta/2, \Delta/2)$, quantizes $Y+U'$ on a grid of width $\Delta$, entropy-codes the index $K$ under the discretized model $p(z_{t-1}|z_t)$, and the receiver reconstructs $K - U'$; Zamir and Feder's identity $H[K|U'] = I(Y; \tilde Y)$ makes the coding cost exactly the mutual information of the uniform channel when the density model is the true convolution. UQDM sets the reverse process to $g_\theta(z_{t-1}; z_t, t) \star U(-\Delta(t)/2, \Delta(t)/2)$, with $g_\theta$ a Gaussian or logistic density centered at the denoising prediction $b(t) z_t + c(t) \hat{x}_\theta(z_t, t)$, so the NELBO telescopes into the bits sent at each step. The mechanism converts a diffusion training loss into an operational bitrate, which is what makes progressive decoding with a single model computationally feasible.

What would settle it

Compress a fixed set of images with UQDM using increasing numbers of diffusion steps and compare the actual bits per dimension against the theoretical limiting rate of Gaussian diffusion. If UQDM's rate does not approach that limit as the step count grows, or if the real file size deviates from the NELBO by more than the reported 3%, the claimed equality between training loss and operational bitrate is not holding in practice.

Watch

Extended reading notes

Core claim

The central discovery is that a diffusion model can be rebuilt around uniform noise channels so that the variational bound it is trained on is not just a bound on likelihood but the exact expected codelength of a practical progressive codec. In UQDM the forward posterior $q(z_{t-1}|z_t,x)$ is a uniform distribution matched in mean and variance to the Gaussian posterior of VDM, and the reverse model is a learned density convolved with uniform noise. Because universal quantization simulates a uniform channel with entropy cost $h(\tilde Y) - \log \Delta$, the per-step KL terms in the NELBO become the bits actually sent, with the initial Gaussian latent shared by a seeded random generator at negligible cost. The authors show that as the number of steps grows the uniform forward process converges in distribution to the Gaussian diffusion process, yet in practice a small number of steps (around 4 or 5) is best, and they find that learning the reverse-process variance substantially improves the rate. They report that the real file size stays within 3% of the theoretical NELBO.

Load-bearing premise

The method rests on the trained reverse-process variance keeping the per-step coding cost small with only a few diffusion steps; the paper's own analysis finds a constant overhead per step when only moments are matched, and increasing the number of steps makes the NELBO worse, so the competitive results depend on an empirical property of the learned variance rather than on the asymptotic convergence theorem.

Editorial extensions

If this is right

A single UQDM model covers the full rate-distortion curve: stopping after fewer steps gives low bit-rates and coarser reconstructions, and sending more steps refines them up to lossless.
Encoding and decoding require only a few neural-network evaluations (around 4 steps on images), so the codec avoids relative-entropy-coding primitives whose runtime scales exponentially with the number of bits.
Because the training loss is an actual bitrate, any improvement in the diffusion model's likelihood translates directly into better compression without changing the codec.
Flow-based reconstructions match the distortion of denoised predictions while giving substantially better realism as measured by FID, so the same model can be aimed at either fidelity or realism.
At high bit-rates UQDM keeps improving where BPG, CTC, and other baselines plateau, suggesting that progressive diffusion coding can be practically deployed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Left implicit in the paper but a direct extension: running UQDM on a pretrained autoencoder's latent space would inherit the compute savings of latent diffusion and likely scale to larger images.
The paper's learned-variance implementation spends minutes on a CPU for a 32×32 image because it builds a separate entropy-coding table per dimension; this is an engineering bottleneck that parallel entropy coding should remove.
The demonstrated convergence of the uniform forward process to the Gaussian one as steps grow suggests UQDM could be used as an empirical probe for how much of the Gaussian-diffusion compression bound is lost to the uniform channel's per-step overhead.
The rate-realism results point toward a single-model variable-rate codec for applications that want realistic reconstructions at ultra-low bit-rates, though the paper itself does not claim perfect realism.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth your time. The core construction is sound: replace Gaussian noise with uniform noise in the forward process, define the reverse density as a learned gθ convolved with uniform noise, and the NELBO becomes a real codelength under universal quantization. That is a genuine step past Theis et al.'s Gaussian codec, which needs intractable channel simulation. The paper also proves the continuous-time limit matches VDM, a nice formal result.

What's new: UQDM's forward process and the corresponding coding scheme are not in the cited literature. The proofs in the appendix are real, the file-size check (within 3% of NELBO) is concrete, and the ImageNet64 comparison includes CTC, a legitimate progressive codec. Those are points in the paper's favor.

Soft spots: the practical claims depend on the learned reverse variance at T≈4. Appendix B.2 shows that a moment-matched variance costs a constant per-step KL overhead, and the learned scale sθ is meant to avoid that. But the paper never reports a per-step Lt decomposition, the average learned scale, or a fixed-vs-learned variance ablation on CIFAR or ImageNet64. Without those, you can't tell how much of the bitrate is progressive refinement and how much is uniform-channel mismatch. The VDM curves are hypothetical, so they don't prove competitiveness against an actual Gaussian codec. And the 5-minute CPU time per CIFAR image for the learned-variance version is acknowledged but unsolved; 'practical deployment' overstates it. Only 4–5 rate points per model, no error bars, and code is promised for acceptance.

None of this breaks the central idea. The NELBO-bitrate identity is mathematically clean, and the experimental gaps are the usual first-paper sins. The right move is to send it to review and ask for code, per-step rate analysis, and the missing ablation. If those come back, the empirical claims will be testable.

Bottom line: this deserves serious refereeing. The idea will likely have legs in learned compression.

Referee Report

3 major / 6 minor

Summary. The paper proposes a new class of diffusion models, Universally Quantized Diffusion Models (UQDM), in which the forward process uses uniform noise instead of Gaussian noise. The central claim is that the negative ELBO of this model is exactly the end-to-end lossless compression cost when the codec is implemented with universal quantization, thereby avoiding the exponential runtime of Gaussian channel simulation required by previous diffusion-based progressive codecs. The authors specify the forward and reverse processes, prove a continuous-time convergence of the forward process to the Gaussian case, and present progressive compression experiments on toy, CIFAR-10, and ImageNet64 data, reporting competitive rate-distortion and rate-realism results against JPEG, JPEG2000, BPG, CTC, and CDC.

Significance. If the central identity holds, the paper makes a valuable conceptual and practical contribution: it connects a diffusion model's training loss directly to an operational bitrate and provides a tractable progressive codec with O(T) neural network evaluations per image. The paper includes a clear theoretical derivation of the NELBO-to-bitrate correspondence and a central limit theorem for the forward process, and it reports that the actual file sizes are within 3% of the theoretical NELBO, which is a strong empirical validation of the coding scheme. The main limitations are that the practical demonstration relies on very few diffusion steps (T ≈ 4), the reported rate-distortion curves contain only a handful of points, and the appendix's analysis of the per-step overhead contains a sign error that undermines one of the paper's theoretical motivations.

major comments (3)

[Appendix B.2] There is a sign error in the derivation of the per-step rate. The function h(z) is defined as h(z) = -log(G_t(z+μ_t+Δ_t/2) - G_t(z+μ_t-Δ_t/2)), which is nonnegative because the CDF difference lies in [0,1]. The text nevertheless states that h(z) ≤ 0 and writes L_{t-1} ≈ -(1/6)[4h(0)+h(-Δ_t/2)+h(Δ_t/2)] ≥ (1/3) log 2. For nonnegative h, the left-hand expression is nonpositive, so the inequality and the claimed constant per-step overhead do not follow as stated. Please correct the sign and re-derive, or remove the quantitative claim.
[Sections 5.2, 5.3 and Appendix B.2] The practical claim of competitive bitrates rests on the learned reverse-process variance reducing the per-step overhead that the fixed-variance analysis predicts, but the paper reports no per-step L_t decomposition, no average learned variance s_θ(z), and no ablation of fixed versus learned variance on CIFAR-10 or ImageNet64. Without these numbers, the reader cannot determine whether the bitrates in Figures 3 and 4 reflect progressive refinement information or are dominated by the uniform-channel mismatch overhead. Please add a per-step rate breakdown and a fixed-variance ablation on image data.
[Algorithms 1 and 2] The entropy-coding step is specified as 'Entropy-code k_t using p(z_{t-1}|z_t)', but p(z_{t-1}|z_t) is a continuous density, whereas k_t is a discrete index whose coding distribution must be the discretization of p(z_{t-1}|z_t) on a grid of width Δ(t) offset by the dither u_t. This offset dependence is described in Section 2 but is not made explicit in the algorithms. Please state the exact coding distribution P(k_t | z_t, u_t) so that the claimed equality between the sum of L_t terms and the actual transmitted bitrate is fully operational.

minor comments (6)

[Abstract and Section 5.3] The abstract says the method gives results that 'exceed existing progressive codecs', but Section 5.3 states the results are 'comparable to, if not slightly better than, CTC'. Please align the wording with the actual comparison.
[Section 3.2] The learned variance is written as σ_θ² = σ_Q²(t) ⊙ s_θ(z_t); since the network is time-indexed, it would be clearer to write s_θ(z_t, t) to make the time dependence explicit.
[Appendix B.3] Applying the VDM probability-flow ODE update (eq. 17) to UQDM is justified only by an analogy with the continuous-time limit; this is a heuristic, and the text should state that it is an approximation rather than implying that the discrete-time backward processes are equivalent.
[Figure 6 caption] The caption contains an incomplete sentence: 'conceptually this is equivalent to When using fixed reverse-process variances'. Please rewrite.
[Section 5.1] The observation that larger T improves sample quality (Figure 7) but worsens compression NELBO (Figure 2) is interesting but potentially confusing; a brief explanation of why per-step overhead accumulates with T would help the reader.
[Appendix C.2] The learned-variance model takes about 5 minutes to compress a CIFAR image due to a naive per-coordinate CDF-table loop; reporting the fixed-variance model's speed or using a parallel entropy coder would better support the 'practical deployment' claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the NELBO-to-bitrate identity is a constructive equality built on the external Zamir-Feder theorem, not a relabeled fit.

full rationale

The paper's central claim—that the UQDM negative ELBO equals the end-to-end progressive compression cost under universal quantization—is not circular; it is a constructive identity. The reverse-process density p(z_{t-1}|z_t) = g_theta * U(-Delta/2, Delta/2) is explicitly chosen so that the entropy-coding surrogate f_theta of Eq. (2) coincides with the model distribution, and the Zamir-Feder universal quantization theorem supplies the per-step codelength as the same KL term appearing in Eq. (1). Thus the operational bitrate is derived from an external, cited information-theoretic result rather than from fitting a parameter and then predicting the same quantity. The initial Gaussian term is set to approximately zero by sharing a seed and enforcing q(z_T|x) ≈ p(z_T), and this simplification is disclosed rather than disguised. The learned reverse-process variance is a trained component, but the paper verifies actual file sizes against the theoretical NELBO (within 3%) and evaluates against external codecs (JPEG, JPEG2000, BPG, CTC, CDC), so the empirical claims are not forced by construction. The continuous-time equivalence (Theorem A.1) is proved independently via the Lindeberg-Feller CLT. Self-citations such as Yang & Mandt (2023) and Pandey, Rudolph, and Mandt (2023) appear only as baselines or related work and are not load-bearing. The finite-T performance question raised in Appendix B.2 is a legitimate empirical/engineering concern about learned variances, but it does not make the derivation circular.

Assumptions & free parameters 3 free parameters · 5 assumptions · 1 invented entities

The central mechanism relies on well-established universal quantization and arithmetic coding, plus standard CLT. The main free constants are the number of diffusion steps, the noise schedule endpoints, and the learned reverse-process variance, all chosen or fitted to make the compression performance work. The uniform-noise forward process is a new construction without independent external evidence.

free parameters (3)

Number of diffusion steps T = 4 (CIFAR10/ImageNet64), 5 (swirl data)
Chosen empirically; larger T degrades NELBO due to per-step rate overhead (Fig. 2), so the method depends on a particular small T.
Noise schedule endpoints gamma0, gammaT = learned per dataset
Parameterize sigma_t^2 = sigmoid(gamma_t) and learn the endpoints; authors note learned schedule gave no significant improvement over linear, so a fixed linear schedule is used in practice.
Reverse-process variance scale s_theta(z_t) = network output, learned
Learning per-coordinate variance substantially improves NELBO (Fig. 2); without it, moment-matched variance incurs a constant per-step overhead (Appendix B.2).

assumptions (5)

domain assumption Universal quantization (Zamir-Feder) achieves the target uniform-noise channel at cost I(Y;Y+U), and with surrogate model f_theta the cost becomes E[KL(u(·|y) || f_theta(·))].
External information-theoretic result, cited in Section 2, used to equate NELBO to coding cost.
standard math Arithmetic coding can code K at a length arbitrarily close to its entropy.
Used in the progressive coding scheme; the paper reports <3% overhead.
standard math Lindeberg-Feller CLT applies to the triangular array of scaled uniform variables.
Used in proof of Theorem A.1.
domain assumption alpha_T=0 and sigma_T=1, so q(z_T|x)=N(0,I)=p(z_T) and L_T≈0.
Standard choice in VDM to make prior independent of x; the model sets this to avoid transmitting z_T.
ad hoc to paper The flow-based ODE update of VDM (eq. 17) also approximately solves the reverse process of UQDM at small T.
Used in experiments to obtain flow reconstructions; justified only by the continuous-time equivalence and empirical similarity, no rigorous proof.

invented entities (1)

Uniform-noise forward diffusion process q(z_{t-1}|z_t,x)=U(b(t)z_t+c(t)x−Δ(t)/2, ...)
purpose: Replaces Gaussian noise so progressive coding can use universal quantization instead of intractable channel simulation.
A new mathematical construction in Section 3.1. It has no external falsifiable prediction; its validity is tied to the UQ theory and the paper's own experiments.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Progressive Compression with Universally Quantized Diffusion Models." pith.science (2026). https://pith.science/paper/KWRTFVGT

@misc{pith2026241210935,
  author       = {Pith},
  title        = {Pith review of: Progressive Compression with Universally Quantized Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KWRTFVGT}},
  note         = {Machine review of arXiv:2412.10935}
}

read the original abstract

Diffusion probabilistic models have achieved mainstream success in many generative modeling tasks, from image generation to inverse problem solving. A distinct feature of these models is that they correspond to deep hierarchical latent variable models optimizing a variational evidence lower bound (ELBO) on the data likelihood. Drawing on a basic connection between likelihood modeling and compression, we explore the potential of diffusion models for progressive coding, resulting in a sequence of bits that can be incrementally transmitted and decoded with progressively improving reconstruction quality. Unlike prior work based on Gaussian diffusion or conditional diffusion models, we propose a new form of diffusion model with uniform noise in the forward process, whose negative ELBO corresponds to the end-to-end compression cost using universal quantization. We obtain promising first results on image compression, achieving competitive rate-distortion and rate-realism results on a wide range of bit-rates with a single model, bringing neural codecs a step closer to practical deployment.

Figures

Figures reproduced from arXiv: 2412.10935 by the authors.

**Figure 2.** Results on swirl data. The VDM curves correspond to the hypothetical performance of [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** Progressive lossy compression performance of UQDM on the CIFAR10 dataset, comparing [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (7 more)

**Figure 4.** Figure 4: Progressive lossy compression performance of UQDM on the Imagenet64 dataset, compar [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

**Figure 5.** Figure 5: Example progressive reconstructions from UQDM trained with [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

**Figure 7.** Figure 7: Unconditional samples from UQDM models trained with varying [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]

**Figure 6.** Figure 6: Left: 1000 samples from the toy swirl source. Right: Additional results on swirl data. We examined the compression performance of applying universal quantization to a pre-trained VDM model; conceptually this is equivalent to When using fixed reverse-process variances, …

**Figure 8.** Figure 8: Additional results on ImageNet 64x64 data. [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]

**Figure 9.** Figure 9: Additional example reconstructions , chosen at roughly similar (high) bitrates. [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]

**Figure 10.** Figure 10: Additional example reconstructions , chosen at roughly similar (low) bitrates. [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 11 canonical work pages

[1]

posterior

− SNR(v), 14 where SNR(s) := α2 s σ2s . It can be verified that E [ωt] = 0, Var (ωt) = TX v=t+1 δ2 v|tI = σ4 t α2 t [SNR(t) − SNR(T )]I = β2 T |tI, or in other words, at any step t our forward-process “posterior” distribution q(zt|zT , x) has the same mean and variance as in the Gaussian case. A.3 C ONVERGENCE TO THE GAUSSIAN CASE We show that both forwar...

work page 2021
[3]

Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameters

Marton Havasi, Robert Peharz, and Jos´e Miguel Hern´andez-Lobato. Minimal random code learning: Getting bits back from compressed model parameters. arXiv preprint arXiv:1810.00440,

work page Pith review arXiv
[4]

High-Fidelity Image Compression with Score-Based Generative Models

Emiel Hoogeboom, Eirikur Agustsson, Fabian Mentzer, Luca Versari, George Toderici, and Lucas Theis. High-Fidelity Image Compression with Score-Based Generative Models. arXiv preprint arXiv:2305.18231,

arXiv
[5]

probability-flow

Song et al. (2021c) further note that an alternative approximate solution to the SDE can be obtained by deterministically reversing a “probability-flow” ODE (see also Theis et al. (2022)). Specifically, this involves repeatedly evaluatingzt−1 = f (zt, t), where f for VDM is defined as f (zt, t) = αt−1 αt zt + σt−1 − αt−1 αt σt ˆϵt = σt−1 σt zt + αt−1 − σt...

work page 2021
[6]

Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation

Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rombach. Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation. arXiv preprint arXiv:2403.12015,

arXiv
[7]

Lossy Compression with Gaussian Diffusion

Lucas Theis, Tim Salimans, Matthew D Hoffman, and Fabian Mentzer. Lossy Compression with Gaussian Diffusion. arXiv preprint arXiv:2206.08889,

arXiv
[10]

15 It therefore suffices to show that ωt converges in distribution to N (0, β2 T |tI) in the continuous-time limit

− SNR(v), where SNR(s) := α2 s σ2s . 15 It therefore suffices to show that ωt converges in distribution to N (0, β2 T |tI) in the continuous-time limit. Since the different coordinates of ωt are independent, we focus on a single coordinate and study the continuous-time limit of a scalar Ωt, given by a sum of scaled uniform variables, Ωt := TX v=t+1 √ 12σ2...

work page 2019
[11]

We conclude by the Lindeberg-Feller theorem that Ωt = Xn,1 +

(15) = 0 (16) since Uj ∼ U(−1/2, 1/2), and it follows that E[X 2 n,j1{|Xn,j| > ϵ}] = 0 for all j for all sufficiently large n. We conclude by the Lindeberg-Feller theorem that Ωt = Xn,1 + ... + Xn,n d − → N(0, β2 T |t) as T → ∞. Applying the above argument coordinate-wise then proves the original statement. 16 Corollary A.1.1. If we assume σT and αT to be...

work page 2021

Show all 14 references

[13]

Right: Ablation of the influence of model size on validation loss

Flow-based reconstructions achieve similar distortion (as meassured with PSNR) than denoised predictions at higher fidelity (as meassured with FID). Right: Ablation of the influence of model size on validation loss. Bars are labeled with the number of parameters for each model...

2021
[128]

When compressing a single 32x32 CIFAR image, we observe file size overhead ≤ 3% of the theoretical NELBO

We verified that our UQDM implementation based on tensorflow-compression achieves file size close the theoretical NELBO. When compressing a single 32x32 CIFAR image, we observe file size overhead ≤ 3% of the theoretical NELBO. In terms of computation speed, it takes our model ...

2021
[1992]

The forward process is defined by q(zt|x) := N (αtx, σ2 t I), where αt and σ2 t are positive scalar-valued functions of t

13 APPENDIX A F ORWARD PROCESS DETAILS A.1 G AUSSIAN (DDPM/VDM) For completeness and reference, we restate the forward process and related conditionals given in (Kingma et al., 2021). The forward process is defined by q(zt|x) := N (αtx, σ2 t I), where αt and σ2 t are positive ...

2021
[2020]

Communication requirements for generating correlated random variables

Paul Cuff. Communication requirements for generating correlated random variables. In 2008 IEEE International Symposium on Information Theory, pp. 1393–1397. IEEE,

2008
[2022]

DeepHQ: Learned Hierarchical Quantizer for Progressive Deep Image Coding

Jooyoung Lee, Se Yoon Jeong, and Munchurl Kim. DeepHQ: Learned Hierarchical Quantizer for Progressive Deep Image Coding. arXiv preprint arXiv:2408.12150,
[2024]

On Channel Simulation with Causal Rejection Samplers

Daniel Goc and Gergely Flamich. On Channel Simulation with Causal Rejection Samplers. arXiv preprint arXiv:2401.16579,

arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

posterior

− SNR(v), 14 where SNR(s) := α2 s σ2s . It can be verified that E [ωt] = 0, Var (ωt) = TX v=t+1 δ2 v|tI = σ4 t α2 t [SNR(t) − SNR(T )]I = β2 T |tI, or in other words, at any step t our forward-process “posterior” distribution q(zt|zT , x) has the same mean and variance as in the Gaussian case. A.3 C ONVERGENCE TO THE GAUSSIAN CASE We show that both forwar...

work page 2021

[2] [3]

Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameters

Marton Havasi, Robert Peharz, and Jos´e Miguel Hern´andez-Lobato. Minimal random code learning: Getting bits back from compressed model parameters. arXiv preprint arXiv:1810.00440,

work page Pith review arXiv

[3] [4]

High-Fidelity Image Compression with Score-Based Generative Models

Emiel Hoogeboom, Eirikur Agustsson, Fabian Mentzer, Luca Versari, George Toderici, and Lucas Theis. High-Fidelity Image Compression with Score-Based Generative Models. arXiv preprint arXiv:2305.18231,

arXiv

[4] [5]

probability-flow

Song et al. (2021c) further note that an alternative approximate solution to the SDE can be obtained by deterministically reversing a “probability-flow” ODE (see also Theis et al. (2022)). Specifically, this involves repeatedly evaluatingzt−1 = f (zt, t), where f for VDM is defined as f (zt, t) = αt−1 αt zt + σt−1 − αt−1 αt σt ˆϵt = σt−1 σt zt + αt−1 − σt...

work page 2021

[5] [6]

Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation

Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rombach. Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation. arXiv preprint arXiv:2403.12015,

arXiv

[6] [7]

Lossy Compression with Gaussian Diffusion

Lucas Theis, Tim Salimans, Matthew D Hoffman, and Fabian Mentzer. Lossy Compression with Gaussian Diffusion. arXiv preprint arXiv:2206.08889,

arXiv

[7] [10]

15 It therefore suffices to show that ωt converges in distribution to N (0, β2 T |tI) in the continuous-time limit

− SNR(v), where SNR(s) := α2 s σ2s . 15 It therefore suffices to show that ωt converges in distribution to N (0, β2 T |tI) in the continuous-time limit. Since the different coordinates of ωt are independent, we focus on a single coordinate and study the continuous-time limit of a scalar Ωt, given by a sum of scaled uniform variables, Ωt := TX v=t+1 √ 12σ2...

work page 2019

[8] [11]

We conclude by the Lindeberg-Feller theorem that Ωt = Xn,1 +

(15) = 0 (16) since Uj ∼ U(−1/2, 1/2), and it follows that E[X 2 n,j1{|Xn,j| > ϵ}] = 0 for all j for all sufficiently large n. We conclude by the Lindeberg-Feller theorem that Ωt = Xn,1 + ... + Xn,n d − → N(0, β2 T |t) as T → ∞. Applying the above argument coordinate-wise then proves the original statement. 16 Corollary A.1.1. If we assume σT and αT to be...

work page 2021