REVIEW 3 major objections 5 minor 17 references
Fast Training-free Perceptual Image Compression
T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Adding Gaussian noise to a decoded image and then denoising it with a pretrained diffusion model provably improves any codec's perceptual quality, and the paper implements this at 0.1 second decoding time.
desk verdict SDEdit as a universal training-free perceptual post-processor for codecs is a solid, practical idea; the theory is mostly right but Theorem 3.1 needs its in-distribution condition and the SDE monotonicity proof has a gap. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the two-step 'add noise, then denoise' decoder update, in which the decoded image $\hat{X}$ is convolved with the same Gaussian kernel used by the forward diffusion process and then mapped back by a trained reverse diffusion or PF-ODE solver. The argument reduces to two inequalities: for any two distributions, convolving both with the same Gaussian kernel strictly decreases their KL divergence, and the reverse SDE solver strictly decreases (while the ODE solver preserves) the divergence between the denoised marginal and the source marginal; chaining these gives Theorem 3.1. Tweedie's formula, $E[X_0|X_t] = X_t + \sigma_t^2 s_\theta(t,X_t)$, supplies the posterior-mean estimate needed to set the noise level at the double-MSE operating point and to define the diffusion-inversion constraint of the slow preset. A one-step consistency model $g_\theta(t,X_t) \approx \Phi_{\mathrm{ODE}}(t,X_t)$ is the component that makes the fast preset possible, since it solves the PF-ODE in a single neural network evaluation. The same transform doubles as a perception-distortion knob: sweeping $\sigma_t$ from zero up to the selected value moves the output from the codec's best-PSNR point toward its best-FID point.
What would settle it
Take a codec and an out-of-distribution test set (for example, chest X-rays with a diffusion model trained on ImageNet), apply the fast preset while sweeping $\sigma_t$ from zero upward, and plot FID against PSNR; the theorem predicts monotone improvement in FID up to roughly the double-MSE point, so a curve that rises or reverses before that point would falsify the central claim for that distribution. A cheaper direct check is to test the medium preset on a held-out domain and compare the empirical KL/FID of $\hat{X}_{0|t}$ against the base codec's output at the same bitrate.
Extended reading notes
Core claim
The paper's central claim is that perceptual quality can be improved for any codec by the fixed decoder-side transformation $\hat{X}_t = \hat{X} + \mathcal{N}(0,\sigma_t^2 I)$ followed by $\hat{X}_{0|t} = \Phi(\hat{X}_t,t)$, where $\Phi$ is a reverse-SDE or probability-flow-ODE solver. Theorem 3.1 asserts that whenever $p(\hat{X}) \neq p(X_0)$, this transformation satisfies $D_{\mathrm{KL}}(p(\hat{X}_{0|t})\|p(X_0)) < D_{\mathrm{KL}}(p(\hat{X})\|p(X_0))$, and that the divergence decreases monotonically as $\sigma_t$ grows. The proof chains the Markov-convolution contraction of KL divergence with the recently established contraction property of reverse diffusion, so the same guarantee holds whether the denoiser is an SDE solver or an ODE solver. From this, the paper derives a noise-selection rule that sets $\sigma_t$ at the point where the denoised image's MSE reaches twice the base codec's estimated optimal MSE, matching the perception-distortion trade-off bound. The paper reports that the fast preset (one-step consistency model) changes ELIC's BD-FID on LSUN Bedroom from $-1.56$ to $-17.34$ in about 0.1 seconds, the medium preset reaches $-23.46$ in a few seconds, and the slow preset reaches $-26.58$ in about 14 seconds, comparable to the previous training-free codec at roughly a third to a sixth of its decoding time.
Load-bearing premise
The load-bearing premise is that the pretrained diffusion and consistency models' learned score function equals the true score of the source image distribution, so that the Gaussian-convolution contraction and the reverse-diffusion contraction both hold for the images being compressed; when this fails, as on out-of-distribution content, the paper's own failure case shows a boy reconstructed as furniture.
Editorial extensions
If this is right
- Any bitstream-compatible codec can be upgraded to a perceptual codec by a pure decoder-side change, preserving the encoder and the bitstream format.
- Decoding latency for training-free perceptual compression falls from about one minute to roughly 0.1 seconds with the consistency-model preset, or 0.1–10 seconds with ODE/SDE solvers.
- MSE-optimized codecs such as ELIC and traditional codecs such as VTM gain large FID improvements (ELIC's BD-FID on LSUN Bedroom changes from -1.56 to -23.46 in the medium preset) with PSNR loss contained within the 3 dB perception-distortion bound.
- The method can also be applied on top of an already-perceptual codec such as MS-ILLM, further improving its FID.
- A single scalar, the injected noise level $\sigma_t$, provides an explicit perception-distortion trade-off, matching the behavior of convex interpolation between MSE-optimal and perception-optimal reconstructions.
Reading between the lines
- The contraction proof does not depend on the base codec being an image codec, so the same perturb-then-denoise step should transfer to any reconstruction task with a matching generative prior, such as speech, video, or medical imaging; the paper does not test this.
- The paper's own failure case (a boy reconstructed as furniture on an LSUN-bedroom-trained model) implies the guarantee is conditional on the source lying in the diffusion model's training distribution; a direct test would be to measure FID gains on a shifted domain and look for a noise level beyond which quality reverses.
- The double-MSE rule for choosing $\sigma_t$ depends on the base codec's MSE being a good estimate of the optimal distortion; for non-MSE codecs like MS-ILLM the paper substitutes the Hyper codec's MSE, so the selected operating point could be slightly off the true optimum, and sweeping $\sigma_t$ on a validation set would reveal whether that matters.
- The fast preset inherits whatever biases the consistency model has; since consistency models are currently available mainly for small-image diffusion models, scaling the method to high resolution or to video likely requires distillation of larger models, which the paper flags as an open limitation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a training-free perceptual image compression method: after decoding, add Gaussian noise to the decoded image and then denoise with a pre-trained unconditional diffusion or consistency model. The authors claim a theoretical guarantee (Theorem 3.1) that this operation strictly reduces the KL divergence between the distribution of the compressed image and the source distribution, and that the reduction is larger for larger noise levels. They instantiate the method in three presets (fast, medium, slow) that trade decoding time against perceptual quality, and they report large BD-FID improvements over the base codecs ELIC, VTM, and MS-ILLM on LSUN Bedroom and ImageNet, with decoding times as low as 0.1 s. The paper also demonstrates that the method can be applied to non-differentiable codecs and can achieve perception-distortion trade-off by adjusting the noise level.
Significance. If the theoretical claims held in the stated generality, the paper would make training-free perceptual image compression practical by cutting decoding time from about a minute to 0.1–10 s, while remaining applicable to non-differentiable codecs such as VTM. The empirical results are strong: for example, on LSUN Bedroom, Proposed-ELIC (medium preset) reaches a BD-FID of −23.46 compared with −1.56 for ELIC, and the fast preset achieves a BD-FID of −17.34 in 0.1 s, outperforming all conditional perceptual codecs reported in Table 3. The paper also provides a proof chain that draws on the data processing inequality and Nie et al.'s SDE/ODE contraction theorem, and it ships source code. The main weaknesses are that the theorem is stated without the conditions its proof requires (exact score and in-distribution source), and the SDE monotonicity argument in Appendix A contains an unjustified substitution; these issues affect the advertised generality and one of the two central theoretical claims.
major comments (3)
- [Section 3.1, Theorem 3.1; Appendix A, Eqs. (18)–(21); Appendix C.4, Fig. 7] Theorem 3.1 is stated as a universal guarantee for 'any codec' without qualification, but the proof invokes Nie et al.'s contraction theorem, which requires the reverse SDE/ODE to be driven by the exact score of the source distribution p(X_t). In practice, the score model is a finite-capacity network trained on a specific dataset, and for out-of-distribution inputs the inequality can reverse; the paper's own failure case in Appendix C.4 (Fig. 7, a boy reconstructed as furniture) demonstrates this. The theorem statement should explicitly condition on the score model being accurate (or at least sufficiently close) for the source distribution, and the abstract's claim of 'theoretical guarantee' for 'any existing codec' should be softened accordingly.
- [Appendix A, Eqs. (25)–(26)] The proof of the monotonicity part of Theorem 3.1 (Eq. 13) is invalid as written. In the second line of Eq. (26), the integral from 0 to s retains the conditional distribution p(X_{τ|t}) from the t-case, whereas the expression for D_KL(p(\hat X_{0|s}) || p(X0)) requires p(X_{τ|s}). These two conditional distributions differ for t ≠ s, so the final equality is unjustified. Without an additional argument showing that the Fisher-information integral with p(X_{τ|t}) over [0,s] dominates or is dominated by the corresponding integral with p(X_{τ|s}), the monotonicity claim is unproven.
- [Section 3.2, Eqs. (15) and (17)] The theoretical guarantee of Theorem 3.1 applies only to exact SDE/ODE solvers, but the fast and slow presets use approximations that are not covered. The fast preset replaces the PF-ODE solver with a consistency model (Eq. 15), which is a one-step distilled approximation; the slow preset adds a DPS gradient step (Eq. 17) that changes the reverse dynamics. No error bounds or contraction guarantees are provided for these modifications, so the advertised 'theoretical guarantee' does not extend to the two presets that are central to the speed claims (0.1 s and 0.1–10 s). The manuscript should either state the theorem for exact solvers only and present the fast/slow results as empirical, or provide an analysis of the approximation error.
minor comments (5)
- [Section 3.1, Eq. (13)] In Eq. (13), the quantities p(X_{0|t}) and p(X_{0|s}) should be p(\hat X_{0|t}) and p(\hat X_{0|s}) to match the notation in the rest of the paper, and the parentheses in 'p(X0|s||p(X0)' are unbalanced.
- [Appendix A, Eq. (18)] The proof writes the Gaussian kernel as N(0, t^2 I), while the main text uses σ_t^2; the relationship σ_t = t for the VE diffusion used in the experiments should be stated explicitly.
- [Section 3.1] The statement 'the first "Add noise" step obviously reduces the divergence' is imprecise: the data processing inequality gives ≤, not necessarily <. The strict inequality in Eq. (22) is needed for the theorem's conclusion and should be justified or stated as a condition on the Gaussian kernel.
- [Table 3] In the fast setting, IVC-ELIC has a BD-FID of 208.50, which is an order of magnitude worse than any other entry; this outlier should be commented on in the text to avoid misleading readers about the comparison.
- [Section 4.1] The evaluation uses only 1000 images per dataset; given the stochasticity of the method and the use of FID, the paper should report variance or confidence intervals, or at least acknowledge this limitation.
Circularity Check
No circular derivation: Theorem 3.1 rests on an external contraction theorem and standard DPI; the remaining issues are omitted assumptions and a proof gap, not self-reference.
full rationale
The paper's central claim is that adding Gaussian noise to a decoded image and then running a pre-trained diffusion denoiser reduces KL divergence to the source distribution. The proof chains Eq. 18 (Gaussian convolution reduces KL by Markov/data-processing), Nie et al.'s external Theorem 3.1 (Eqs. 19-21, SDE contraction / ODE preservation), and arrives at Eq. 22. None of these equations define the conclusion into the premises: Eq. 18 is a standard DPI inequality, and Eq. 19 is an imported external theorem, not a result of the present authors or their self-citations. The noise-level selection in Eq. 14 is anchored to the Blau-Michaeli distortion lower bound rather than fitted to FID values, and the preset implementations are chosen empirically by measured speed-FID trade-offs, not presented as predictions forced by construction. The self-citations that appear (e.g., IVC / Xu et al. 2024) are used as prior-work baselines and comparisons, and they are not load-bearing for Theorem 3.1. Two non-circular weaknesses do exist: Theorem 3.1 is stated unconditionally but its proof via Eq. 19 requires the score model to be the true score of p(X_t), which the authors themselves acknowledge can fail out-of-distribution in Appendix C.4 (the boy-to-furniture failure case). Additionally, the SDE monotonicity proof in Eqs. 25-26 replaces the Fisher integrand p(X_{τ|t}) with p(X_{τ|s}) in the final equality, which is not justified as written. These are correctness/rigor concerns about overbroad claims, not instances where a prediction reduces by construction to its input. Therefore the circularity score is low.
Assumptions & free parameters
free parameters (3)
- Noise level σ_t² =
per Eq. 14 (doubles MSE)
- DPS gradient step size ζ =
not reported
- Preset time boundaries and solver step counts =
0.1s / 5s / 10s boundaries
assumptions (5)
- domain assumption The pre-trained score model satisfies sθ(t, Xt) = ∇ log p_t(Xt) for the source distribution p(X0), and the ODE/SDE solvers are exact
- standard math Gaussian convolution strictly reduces KL divergence (data processing inequality)
- standard math Nie et al. 2024: SDE reverse steps strictly decrease KL; ODE reverse steps preserve KL
- domain assumption The distortion lower bound E[||E[X0|Y] - X0||²] equals the base codec's MSE (or Hyper's MSE for MS-ILLM)
- domain assumption Test images are in-distribution for the pre-trained EDM/ADM diffusion and consistency models
Cite this review
Pith. "Pith review of Fast Training-free Perceptual Image Compression." pith.science (2026). https://pith.science/paper/446WU5VO
@misc{pith2026250616102,
author = {Pith},
title = {Pith review of: Fast Training-free Perceptual Image Compression},
year = {2026},
howpublished = {\url{https://pith.science/paper/446WU5VO}},
note = {Machine review of arXiv:2506.16102}
}
abstract
Training-free perceptual image codec adopt pre-trained unconditional generative model during decoding to avoid training new conditional generative model. However, they heavily rely on diffusion inversion or sample communication, which take 1 min to intractable amount of time to decode a single image. In this paper, we propose a training-free algorithm that improves the perceptual quality of any existing codec with theoretical guarantee. We further propose different implementations for optimal perceptual quality when decoding time budget is $\approx 0.1$s, $0.1-10$s and $\ge 10$s. Our approach: 1). improves the decoding time of training-free codec from 1 min to $0.1-10$s with comparable perceptual quality. 2). can be applied to non-differentiable codec such as VTM. 3). can be used to improve previous perceptual codecs, such as MS-ILLM. 4). can easily achieve perception-distortion trade-off. Empirically, we show that our approach successfully improves the perceptual quality of ELIC, VTM and MS-ILLM with fast decoding. Our approach achieves comparable FID to previous training-free codec with significantly less decoding time. And our approach still outperforms previous conditional generative model based codecs such as HiFiC and MS-ILLM in terms of FID. The source code is provided in the supplementary material.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[4]
URL https://api.semanticscholar.org/ CorpusID:263605744. T. Deveney, J. Stanczuk, L. M. Kreusser, C. Budd, and C.-B. Schönlieb. Closing the ode-sde gap in score-based diffusion models through the fokker-planck equation.arXiv preprint arXiv:2311.15996,
-
[5]
P. Dhariwal and A. Nichol. Diffusion models beat gans on image synthesis. ArXiv, abs/2105.05233,
-
[8]
A. Lugmayr, M. Danelljan, A. Romero, F. Yu, R. Timofte, and L. R. Van Gool. Inpainting using denoising diffusion probabilistic models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11461–11471. S. Luo, Y . Tan, L. Huang, J. Li, and H. Zhao. Latent consistency models: Synthesizing high-resolution images with few...
-
[9]
Y . Ma, Y . Zhai, C. Yang, J. Yang, R. Wang, J. Zhou, K. Li, Y . Chen, and R. Wang. Variable rate roi image compression optimized for visual quality. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1936–1940,
work page 1936
-
[11]
F. Mentzer, G. Toderici, M. Tschannen, and E. Agustsson. High-fidelity generative image compression. ArXiv, abs/2006.09965,
arXiv 2006
-
[12]
J. Song, C. Meng, and S. Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020a. Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020b. Y . Song, P. Dhariwal, M. Chen, and I. Sutskever. Consis...
arXiv 2010
-
[13]
URL https://api.semanticscholar.org/CorpusID:257280191. L. Theis, T. Salimans, M. D. Hoffman, and F. Mentzer. Lossy compression with gaussian diffu- sion. ArXiv, abs/2206.08889,
-
[16]
R. Yang and S. Mandt. Lossy image compression with conditional diffusion models. arXiv preprint arXiv:2209.06950,
Show all 17 references
-
[17]
For the first part, we note that ˆXt, Xt is a convolution of ˆX, X0 by a Gaussian kernel N (0, t2I)
12 A Proof of Main Results Proof. For the first part, we note that ˆXt, Xt is a convolution of ˆX, X0 by a Gaussian kernel N (0, t2I). By property of Markov chain, we have DKL (p( ˆXt)||p(Xt)) < DKL (p( ˆX)||p(X0)). (18) Next, we can adopt the Theorem 3.1 of Nie et al. [2024],...
2024
-
[1982]
Ballé, D
J. Ballé, D. C. Minnen, S. Singh, S. J. Hwang, and N. Johnston. Variational image compression with a scale hyperprior. ArXiv, abs/1802.01436,
-
[2011]
T. Xu, Z. Zhu, D. He, Y . Li, L. Guo, Y . Wang, Z. Wang, H. Qin, Y . Wang, J. Liu, et al. Idempotence and perceptual image compression. arXiv preprint arXiv:2401.08920,
-
[2019]
Agustsson, D
E. Agustsson, D. C. Minnen, G. Toderici, and F. Mentzer. Multi-realism image compression with a conditional generator. ArXiv, abs/2212.13824,
-
[2020]
Hoogeboom, E
E. Hoogeboom, E. Agustsson, F. Mentzer, L. Versari, G. Toderici, and L. Theis. High-fidelity image compression with score-based generative models. arXiv preprint arXiv:2305.18231,
-
[2021]
C. Meng, Y . He, Y . Song, J. Song, J. Wu, J.-Y . Zhu, and S. Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.01073,
-
[2022]
N. F. Goose, J. Petersen, A. Wiggers, T. Xu, and G. Sautiere. Neural image compression with a diffusion-based decoder. arXiv preprint arXiv:2301.05489,
-
[2023]
Chung, J
H. Chung, J. Kim, M. T. McCann, M. L. Klasky, and J. C. Ye. Diffusion posterior sampling for general noisy inverse problems. ArXiv, abs/2209.14687, 2022a. URL https://api.semanticscholar. org/CorpusID:252596252. H. Chung, J. Kim, M. T. Mccann, M. L. Klasky, and J. C. Ye. Diffu...
-
[2024]
Z. Yan, F. Wen, and P. Liu. Optimally controllable perceptual lossy compression. arXiv preprint arXiv:2206.10082,
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.