Pith. sign in

REVIEW 2 major objections 5 minor 38 references

Optimizing Learned Image Compression on Scalar and Entropy-Constraint Quantization

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read Retraining on true quantized data cuts bitrate up to 2.2 percent

desk verdict Solid incremental finetuning trick, with TCQ the new piece; the missing control on approximate latents leaves the causal claim under-supported. read the letter →

arxiv 2506.08662 v1 pith:J73RY33P submitted 2025-06-10 cs.LG cs.AIcs.ITmath.IT

classification cs.LGcs.AIcs.ITmath.IT
keywords learnedimagecompressionquantization-awaretrainingtrellis-codedquantizationuniformscalarfine-tuningrate-distortionoptimizationvariationalautoencoderBD-rate
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Learned image compression models are trained with smooth approximations of quantization because true quantization has zero gradients almost everywhere. The paper proposes an extra finetuning step: after normal end-to-end training, retrain parts of the network on latents that have been quantized in exactly the way they will be at inference. This consistently improves rate-distortion performance for both uniform scalar quantization and, more strongly, for entropy-constraint trellis-coded quantization, with no added inference cost. The gains reach about 1-2% average bitrate savings on Kodak and up to 2.2% on TecNick, measured by BD-rate.

What carries the argument

The central mechanism is a post-training finetuning stage in which the decoder (and, for the uniform scalar case, also the hypercoder) is retrained on quantized latents produced by the actual inference-stage quantizer, rather than on noise-perturbed or soft-quantized latents. For trellis-coded quantization, the quantizer is a state-dependent vector quantizer with two scalar quantizers and a Viterbi trellis search that jointly accounts for rate and distortion; because its decisions depend on past samples, the entropy model, and a distortion measure, it is especially poorly approximated by per-sample smooth substitutes, which is why the retraining benefit is largest there.

What would settle it

Retrain the same trellis-coded decoder with quantization recomputed on the fly at every training step instead of loading a static offline dataset, then compare BD-rate; if the gain disappears, the reported improvement depends on the frozen-coder distribution match rather than on learning from true quantization itself.

Watch

Extended reading notes

Core claim

The paper claims that replacing smooth quantization approximations with genuinely quantized encoder-produced latents during a finetuning stage yields consistent coding gains for learned variational autoencoder codecs, without changing the inference procedure. For uniform scalar quantization, retraining only the decoder gives average BD-rate improvements of -0.87% to -1.24% on Kodak, and additionally retraining the hypercoder improves this to -1.14% to -1.73%. For trellis-coded quantization, decoder-only retraining gives larger improvements of roughly -1.9% to -2.0% on Kodak and about -2.0% to -2.3% on TecNick, supporting the idea that complex entropy-constraint quantizers are hardest to approximate smoothly and therefore benefit most from true quantized training data.

Load-bearing premise

The pre-generated set of trellis-quantized latents, produced with the auxiliary entropy-estimation network frozen, remains representative of what the retrained decoder will encounter; updating that network would change the quantization choices and break the match.

Editorial extensions

If this is right

  • Decoder-only retraining leaves the bitstream unchanged, so the improvements transfer to any pre-encoded stream without altering the decoder's inference-time complexity.
  • The pattern of gains rising for more complex quantizers suggests the same finetuning recipe can improve other hard-to-approximate quantization schemes.
  • Retaining the decoder alongside the hypercoder for uniform scalar quantization adds roughly 0.3-0.5 percentage points of BD-rate gain over decoder-only retraining, by letting the entropy model adapt to true quantization indices.
  • The authors predict that repeatedly generating true quantized data and retraining until convergence could push the gains further, extending their single-pass approach.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next experiment is to recompute trellis-coded quantization on the fly during retraining, which would let the hypercoder update and might close the distribution gap the paper identifies between offline training data and inference latents.
  • The same offline-dataset approach could be tested on autoregressive entropy models, where quantization decisions also depend on previously encoded latents and are likely harder to approximate than per-sample noise.
  • Because the benefit scales with how poorly a smooth approximation models the quantizer, the method may matter most at low bitrates, where large step sizes make uniform-noise assumptions least accurate; the paper's low-rate results are consistent with this.
  • A direct test of distribution shift would be to measure the divergence between pre-generated trellis-quantized latents and latents produced by a hypercoder updated after retraining; the paper's own outlook section implies this mismatch is the main unresolved risk.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes a post-training finetuning step for learned image compression networks: after conventional end-to-end training with smooth quantization surrogates, the decoder (and for uniform scalar quantization also the hypercoder) is retrained on latents obtained by applying the actual inference-time quantizer (USQ or TCQ) to the encoder's output. Experiments on Kodak and TecNick report consistent BD-rate reductions of about 1–2% relative to the anchor models, with TCQ showing larger gains than USQ, and no inference-complexity increase. The authors interpret this as evidence that training on correctly quantized data is beneficial, especially for entropy-constrained quantizers that are hard to approximate.

Significance. If the causal claim holds, the proposed finetuning is a simple, practically useful step that can be added to any learned codec without changing the inference pipeline. The paper's strengths include per-image BD-rate tables showing consistency across all 24 Kodak images, use of standard datasets and external baselines, and an honest discussion of limitations in Section 3.5. The main weaknesses are the absence of a control isolating the quantization-data change and the lack of statistical/reproducibility details. The work is not a theoretical contribution, but an empirical recipe with plausible and potentially transferable value.

major comments (2)
  1. [§3.1–3.3, Eq. (13), Tables 1–3] The central claim is that retraining on 'correctly quantized data' causes the observed BD-rate gains, but the experimental protocol changes two variables simultaneously: the training data distribution (true quantized latents vs. the smooth approximations of Eqs. (8)–(9) or the TCQ perturbation of Section 2.3) and the training objective (decoder retraining in Eq. (13) drops the rate term from Eq. (10), and the hypercoder retraining uses a different loss/λ as well). No control experiment finetunes the same weights on the original approximate latents with the same loss and learning-rate schedule. Without such a control, the gains in Tables 1–3 could be due solely to additional training on a distortion-only objective, which would undermine the paper's title and abstract. This is especially acute for the TCQ decoder-only result in Table 2, where the ~2% improvement is attributed exclusively to the true TCQ latents.
  2. [§3.2, §5, §3.5] The reported gains are small (1–2% BD-rate) but are presented without error bars, multiple-seed experiments, or released code/models. In addition, the construction of the offline TCQ dataset is underspecified: Section 3.2 says only that 'TCQ was applied offline to the entire dataset, pre-generating a comprehensive pool of TCQ reconstructed features z_hat,' without stating the pool size, the exact Viterbi cost weighting, or whether the same encoder/hypercoder checkpoints were used throughout. These omissions make it impossible to assess the statistical significance of the gains or to reproduce the experiments independently. Given that the central claim depends on consistent small gains, this lack of evidentiary detail is a load-bearing issue.
minor comments (5)
  1. [§3.1–3.3, Tables 1–3] Please clarify the mapping between the λ values (128 to 2048) and the 'high' and 'low' bitrate regimes used in the BD-rate tables; the term 'high' could correspond to either large or small λ depending on the convention.
  2. [§2.3] The notation for the TCQ training perturbation is hard to parse: \bar z_i^0 and \bar z_i^1 are defined with a subscript i but the selection is described over entries; please define the index and the sgn operation precisely.
  3. [Throughout] The test-set name is spelled inconsistently ('TecNick' vs. 'Tecnick'), and 'Bjøntegaard' appears in several variants; please unify the spelling.
  4. [§3] Please state whether the anchor models are the exact checkpoints from [12] and [18] or were retrained by the authors using those references' settings; this is important for assessing the fairness of the BD-rate comparisons.
  5. [Figure 2] The caption states 'Average PSNR-curves' but does not explain how the averaging is performed across images or whether the curves are aligned by bitrate; please specify the averaging procedure.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the finetuning gains are empirical measurements against anchors, not derived from the method's own definitions.

full rationale

This paper reports an empirical finetuning procedure; there is no mathematical derivation chain whose conclusion could be identical to its premises. The anchor models are taken from the authors' prior work [12,18], and the retrained models are compared against those anchors on held-out Kodak/TecNick test sets using BD-rate and PSNR measured from actual bitstreams. The claimed gains are therefore read off test data, not implied by any fitted equation. The main scientific vulnerability—that decoder retraining changes the loss to distortion-only and adds more training, so the specific contribution of 'correctly quantized data' is not isolated by a control condition (Section 3.1, Eq. 13 vs Eq. 10)—is a confound in experimental attribution, not a circular definition or a self-citation that forces the result. Section 3.5's admission that the offline TCQ latent set may not match inference distributions is likewise a stated limitation, not a circular step. Self-citations to [12,18] supply the anchor architectures and quantization approximations, but the finetuning gains are measured directly and would be independently checkable by reproduction. Hence no circularity.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical or architectural entities. It relies on standard learned-compression assumptions, hand-chosen hyperparameters, and an offline data-generation step whose validity is acknowledged as limited.

free parameters (3)
  • Lagrange multiplier lambda values = 128, 256, 512, 1024, 2048
    Chosen by hand to cover different rate-distortion operating points; not fitted to test data but determine which anchor models are used.
  • Learning rate decay factor alpha = (1.131)^-1
    A hand-picked decay schedule used whenever loss saturation is reached; affects training dynamics but is not tuned to the test set.
  • TCQ rate-distortion balance weight = Empirically chosen in [18]
    The TCQ cost criterion uses an empirically chosen tradeoff between rate and latent distortion inherited from prior work; this paper does not re-fit it.
assumptions (3)
  • domain assumption Latent variables are normally distributed.
    Section 2.1 assumes a Gaussian distribution for entropy coding, a standard approximation in learned image compression.
  • domain assumption The arithmetic coder achieves the entropy limit.
    Section 2.3 states this assumption explicitly while estimating bitrate.
  • ad hoc to paper Offline TCQ data generated with a frozen hypercoder remains representative.
    Section 3.2 pre-generates TCQ quantized features; Section 3.5 admits this may differ from inference distributions if the hypercoder changes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Optimizing Learned Image Compression on Scalar and Entropy-Constraint Quantization." pith.science (2026). https://pith.science/paper/J73RY33P

@misc{pith2026250608662,
  author       = {Pith},
  title        = {Pith review of: Optimizing Learned Image Compression on Scalar and Entropy-Constraint Quantization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J73RY33P}},
  note         = {Machine review of arXiv:2506.08662}
}
read the original abstract

The continuous improvements on image compression with variational autoencoders have lead to learned codecs competitive with conventional approaches in terms of rate-distortion efficiency. Nonetheless, taking the quantization into account during the training process remains a problem, since it produces zero derivatives almost everywhere and needs to be replaced with a differentiable approximation which allows end-to-end optimization. Though there are different methods for approximating the quantization, none of them model the quantization noise correctly and thus, result in suboptimal networks. Hence, we propose an additional finetuning training step: After conventional end-to-end training, parts of the network are retrained on quantized latents obtained at the inference stage. For entropy-constraint quantizers like Trellis-Coded Quantization, the impact of the quantizer is particularly difficult to approximate by rounding or adding noise as the quantized latents are interdependently chosen through a trellis search based on both the entropy model and a distortion measure. We show that retraining on correctly quantized data consistently yields additional coding gain for both uniform scalar and especially for entropy-constraint quantization, without increasing inference complexity. For the Kodak test set, we obtain average savings between 1% and 2%, and for the TecNick test set up to 2.2% in terms of Bj{\o}ntegaard-Delta bitrate.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

38 extracted references · 38 canonical work pages

  1. [1]

    The approach offers a compet- itive alternative to modern coding standards like VVC [3] or HEVC

    INTRODUCTION Lossy image compression using neural networks has developed into a promising coding technology [1,2]. The approach offers a compet- itive alternative to modern coding standards like VVC [3] or HEVC

  2. [2]

    NETWORK ARCHITECTURE 2.1. General autoencoder description The network used in this paper is based on the autoencoder design of [12, 18] and follows the basic architecture of V AEs, where an input image x is transformed by an encoder network to the latent representation z ∈ RN , where N describes the total number of la- tent coefficients; see Figure 1. A q...

  3. [3]

    Inference re- sults were calculated on fully sized Kodak images [28] of resolution 768 × 512, and TecNick images [29] with a resolution of 1200 ×

    TRAINING EXPERIMENTS ON QUANTIZED LA TENTS DETERMINED BY THE ENCODER The experimental V AEs were trained on a subset of the Imagenet dataset [27], with cropped 256 × 256 luma blocks. Inference re- sults were calculated on fully sized Kodak images [28] of resolution 768 × 512, and TecNick images [29] with a resolution of 1200 ×

  4. [4]

    Optimizing Learned Image Compression on Scalar and Entropy-Constraint Quantization

    while its compression efficiency still continues improve. Varia- tional autoencoders (V AEs) [5, 6] are the most prominent instances of these networks. They non-linearly transform the input image into latents which are then quantized and encoded using a jointly opti- mized entropy model and arithmetic coding [7]. The decoder per- forms another non-linear ...

  5. [5]

    Nonlinear Transform Coding,

    Johannes Ball ´e, Philip A. Chou, David Minnen, Saurabh Singh, Nick Johnston, Eirikur Agustsson, Sung Jin Hwang, and George Toderici, “Nonlinear Transform Coding,” IEEE Journal of Selected Topics in Signal Processing, vol. 15, no. 2, pp. 339–353, 2021

  6. [6]

    Testing on the higher resolution TecNick set produced BD-rates of −1.98% and −2.29% as listed in Table 4

    Similar to the previous experiment, the improvement is consistent over all rate points and test images, showing larger improvements than USQ, with an average BD-rate of −1.97% for high bitrates, and −1.90% for low bitrates over the Kodak image set. Testing on the higher resolution TecNick set produced BD-rates of −1.98% and −2.29% as listed in Table 4. As...

  7. [7]

    CONCLUSION We show that incorporating a retraining of the decoder network into the training process of learned image compression networks fur- ther increases the reconstruction quality without increasing inference runtime. Especially if the quantization function is more compli- cated and harder to approximate with a gradient-friendly function, optimizing ...

  8. [8]

    The JPEG AI Standard: Providing Efficient Human and Machine Visual Data Consumption,

    Jo ˜ao Ascenso, Elena Alshina, and Touradj Ebrahimi, “The JPEG AI Standard: Providing Efficient Human and Machine Visual Data Consumption,” IEEE MultiMedia, vol. 30, no. 1, pp. 100–111, 2023

Show all 38 references
  1. [9]

    Transformer-based Image Compression,

    Ming Lu, Peiyao Guo, Huiqing Shi, Chuntong Cao, and Zhan Ma, “Transformer-based Image Compression,” 2021

  2. [10]

    Overview of the Versatile Video Coding (VVC) Standard and its Applications,

    Benjamin Bross, Ye-Kui Wang, Yan Ye, Shan Liu, Jianle Chen, Gary J. Sullivan, and Jens-Rainer Ohm, “Overview of the Versatile Video Coding (VVC) Standard and its Applications,” IEEE Transactions on Circuits and Systems for Video Technol- ogy, vol. 31, no. 10, pp. 3736–3764, 2021

  3. [11]

    Overview of the high efficiency video coding (HEVC) standard,

    Gary J Sullivan, Jens-Rainer Ohm, Woo-Jin Han, and Thomas Wiegand, “Overview of the high efficiency video coding (HEVC) standard,” IEEE Transactions on circuits and systems for video technology, vol. 22, no. 12, pp. 1649–1668, 2012

  4. [12]

    Rate- Distortion Optimized Encoding for Deep Image Compression,

    Michael Sch ¨afer, Sophie Pientka, Jonathan Pfaff, Heiko Schwarz, Detlev Marpe, and Thomas Wiegand, “Rate- Distortion Optimized Encoding for Deep Image Compression,” IEEE Open Journal of Circuits and Systems , vol. 2, pp. 633– 647, 2021

  5. [13]

    End- to-end optimized image compression,

    Johannes Ball ´e, Valero Laparra, and Eero P. Simoncelli, “End- to-end optimized image compression,” 2017

  6. [14]

    An Introduction to Arithmetic Coding,

    G. G. Langdon, “An Introduction to Arithmetic Coding,” IBM Journal of Research and Development, vol. 28, no. 2, pp. 135– 149, 1984

  7. [15]

    Variational image compression with a scale hyperprior,

    Johannes Ball ´e, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston, “Variational image compression with a scale hyperprior,” 2018

  8. [16]

    Joint Autoregressive and Hierarchical Priors for Learned Image Compression,

    David Minnen, Johannes Ball ´e, and George D Toderici, “Joint Autoregressive and Hierarchical Priors for Learned Image Compression,” in Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds. 20...

  9. [17]

    Channel-Wise Autore- gressive Entropy Models for Learned Image Compression,

    David Minnen and Saurabh Singh, “Channel-Wise Autore- gressive Entropy Models for Learned Image Compression,” in 2020 IEEE International Conference on Image Processing (ICIP), 2020, pp. 3339–3343

  10. [18]

    Learned Image Compression With Discretized Gaus- sian Mixture Likelihoods and Attention Modules,

    Zhengxue Cheng, Heming Sun, Masaru Takeuchi, and Jiro Katto, “Learned Image Compression With Discretized Gaus- sian Mixture Likelihoods and Attention Modules,” inProceed- ings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), June 2020

  11. [19]

    Deep Learning-Based Image Compression with Trellis Coded Quantization,

    Binglin Li, Mohammad Akbari, Jie Liang, and Yang Wang, “Deep Learning-Based Image Compression with Trellis Coded Quantization,” in 2020 Data Compression Conference (DCC) , 2020, pp. 13–22

  12. [20]

    End- to-end optimization of nonlinear transform codes for percep- tual quality,

    Johannes Ball ´e, Valero Laparra, and Eero P. Simoncelli, “End- to-end optimization of nonlinear transform codes for percep- tual quality,” in 2016 Picture Coding Symposium (PCS), 2016, pp. 1–5

  13. [21]

    Lossy Image Compression with Compressive Au- toencoders,

    Lucas Theis, Wenzhe Shi, Andrew Cunningham, and Ferenc Husz´ar, “Lossy Image Compression with Compressive Au- toencoders,” 2017

  14. [22]

    Nonuniform-to-Uniform Quantiza- tion: Towards Accurate Quantization via Generalized Straight- Through Estimation,

    Zechun Liu, Kwang-Ting Cheng, Dong Huang, Eric P. Xing, and Zhiqiang Shen, “Nonuniform-to-Uniform Quantiza- tion: Towards Accurate Quantization via Generalized Straight- Through Estimation,” in Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition...

  15. [23]

    Three Gaps for Quantisation in Learned Image Compres- sion,

    Shi Pan, Chris Finlay, Chri Besenbruch, and William Knotten- belt, “Three Gaps for Quantisation in Learned Image Compres- sion,” in Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR) Workshops, June 2021, pp. 720–726

  16. [24]

    Comprehensive Com- parisons of Uniform Quantization in Deep Image Compres- sion,

    Koki Tsubota and Kiyoharu Aizawa, “Comprehensive Com- parisons of Uniform Quantization in Deep Image Compres- sion,” IEEE Access, vol. 11, pp. 4455–4465, 2023

  17. [25]

    Trellis-Coded Quantization for End-to-End Learned Image Compression,

    Karsten S ¨uhring, Michael Sch ¨afer, Jonathan Pfaff, Heiko Schwarz, Detlev Marpe, and Thomas Wiegand, “Trellis-Coded Quantization for End-to-End Learned Image Compression,” in 2022 IEEE International Conference on Image Processing (ICIP), 2022, pp. 3306–3310

  18. [26]

    The viterbi algorithm,

    G.D. Forney, “The viterbi algorithm,” Proceedings of the IEEE, vol. 61, no. 3, pp. 268–278, 1973

  19. [27]

    Soft then hard: Rethinking the quantization in neural image compression,

    Zongyu Guo, Zhizheng Zhang, Runsen Feng, and Zhibo Chen, “Soft then hard: Rethinking the quantization in neural image compression,” 2021

  20. [28]

    High-resolution quantiza- tion theory and the vector quantizer advantage,

    T.D. Lookabaugh and R.M. Gray, “High-resolution quantiza- tion theory and the vector quantizer advantage,” IEEE Trans- actions on Information Theory , vol. 35, no. 5, pp. 1020–1033, 1989

  21. [29]

    Calculation of average PSNR differences between RD-curves,

    Gisle Bjontegaard, “Calculation of average PSNR differences between RD-curves,” ITU SG16 Doc. VCEG-M33 , 2001

  22. [30]

    Den- sity Modeling of Images using a Generalized Normalization Transformation,

    Johannes Ball ´e, Valero Laparra, and Eero P. Simoncelli, “Den- sity Modeling of Images using a Generalized Normalization Transformation,” 2016

  23. [31]

    Drop an Octave: Reducing Spatial Redundancy in Convolu- tional Neural Networks With Octave Convolution,

    Yunpeng Chen, Haoqi Fan, Bing Xu, Zhicheng Yan, Yannis Kalantidis, Marcus Rohrbach, Shuicheng Yan, and Jiashi Feng, “Drop an Octave: Reducing Spatial Redundancy in Convolu- tional Neural Networks With Octave Convolution,” in Pro- ceedings of the IEEE/CVF International Conferen...

  24. [32]

    Quantization and Entropy Coding in the Versatile Video Coding (VVC) Standard,

    Heiko Schwarz, Muhammed Coban, Marta Karczewicz, Tzu- Der Chuang, Frank Bossen, Alexander Alshin, Jani Lainema, Christian R. Helmrich, and Thomas Wiegand, “Quantization and Entropy Coding in the Versatile Video Coding (VVC) Standard,” IEEE Transactions on Circuits and Systems ...

  25. [34]

    ImageNet: A large-scale hierarchical image database,

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei, “ImageNet: A large-scale hierarchical image database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248–255

  26. [35]

    Kodak image dataset,

    “Kodak image dataset,” http://r0k.us/graphics/kodak/, last ac- cessed on 2024/02/06

  27. [36]

    Testimages: A large-scale archive for testing visual devices and basic image processing algo- rithms (sampling1200rgbset).,

    N. Asuni and A. Giachetti, “Testimages: A large-scale archive for testing visual devices and basic image processing algo- rithms (sampling1200rgbset).,”

  28. [37]

    Tensorflow,

    “Tensorflow,” https://www.tensorflow.org/, last accessed on 2023/10/23

  29. [38]

    Rdonet: Rate-distortion optimized learned image compression with variable depth,

    Fabian Brand, Kristian Fischer, Alexander Kopte, Marc Wind- sheimer, and Andr´e Kaup, “Rdonet: Rate-distortion optimized learned image compression with variable depth,” in Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Jun...

  30. [1200]

    The experiments were implemented, and optimization was performed, using the Tensorflow Deep Learning package [30]. One epoch consists of250 batches with 8 (for the USQ models) and4 (for Img high low Img high low 1 -0.62% -0.95% 13 -0.52% -0.87% 2 -1.22% -1.51% 14 -0.75% -1.11%...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.