REVIEW 2 major objections 6 minor 27 references
Flexible Mixed Precision Quantization for Learned Image Compression
T0 review · 2 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Flexible mixed-precision quantization assigns each layer a bit-width from the fractional change in RD loss, beating uniform 8-bit quantization at equal model size.
desk verdict FMPQ is a plausible mixed-precision quantization scheme for LIC, but the Algorithm 1 pseudocode contradicts the text and makes the reported results irreproducible as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the per-layer RD-loss sensitivity $\zeta_n(b)$, the absolute fractional change in rate-distortion loss when a single layer is quantized to $b$ bits while all other layers stay full precision. The bit-assignment rule is: for each layer, pick the smallest $b$ such that $\zeta_n(b) < \beta$, where $\beta$ is a tolerance threshold. A variable-step adaptive search over $\beta$ targets a desired compression ratio $CR_{\mathrm{target}}$, with step sizes growing when the achieved ratio is far from target and shrinking when it overshoots. This carries the argument because it converts a combinatorial bit-width search into per-layer thresholding plus a one-dimensional search.
What would settle it
For one of the evaluated codecs at a fixed target model size, compare FMPQ's bit-width assignment against several random assignments with the same total model size, and against a brute-force joint search over assignments on a small network. If a random assignment or a joint-search assignment matches or beats the zeta-based assignment in BD-Rate, the claim that single-layer RD-loss sensitivity selects the better bit-width distribution is refuted.
Extended reading notes
Core claim
The paper claims that the right currency for allocating quantization bits in a learned codec is the fractional rate-distortion loss change, not weight entropy or raw quantization error. For layer $n$, the sensitivity is $\zeta_n(b) = \left| (RD_{\mathrm{quant},n}(b) - RD_{\mathrm{full}}) / RD_{\mathrm{full}} \right|$, where $RD_{\mathrm{quant},n}(b)$ is the RD loss of the full-precision model with only layer $n$ quantized to $b$ bits on a small calibration set. FMPQ assigns the smallest candidate bit-width satisfying $\zeta_n(b) < \beta$, with the tolerance $\beta$ tuned by an adaptive search to hit a target compression ratio. After assignment, the network is fine-tuned with quantization-aware training using only the RD loss, with learnable weight quantization step sizes and a leaky-clip gradient estimate. On three learned compression architectures, this yields BD-Rate improvements over 8-bit fixed-precision quantization at essentially identical model sizes, and the adaptive search converges in far fewer iterations than exhaustive threshold search.
Load-bearing premise
The method assumes that the sensitivity of each layer measured in isolation, with all other layers left full precision, reliably predicts the best bit-width assignment once every layer is quantized at the same time.
Editorial extensions
If this is right
- At the same model size as 8-bit fixed-precision quantization, FMPQ reduces BD-Rate by 0.96, 2.34, and 1.16 percentage points on the Kodak set for the three evaluated learned codecs, with similar gains on the Tecnick and CLIC sets.
- Lowering $CR_{\mathrm{target}}$ trades model size against rate-distortion performance: on one of the codecs, reducing $CR_{\mathrm{target}}$ from 1.0 to 0.60 shrinks the model from 19.26 MB to 12.27 MB at the cost of 2.49 percentage points of BD-Rate.
- The adaptive search converges in 6 to 12 iterations for compression ratios from 0.99 down to 0.50, whereas an exhaustive search grows from 7 to 900 iterations, saving hours at low target sizes.
- Bit-width distributions are not uniform: main encoder and decoder layers require higher precision than hyperprior layers, and the final encoder and decoder layers are the most quantization-sensitive.
Reading between the lines
- Because sensitivity is measured one layer at a time with all others full precision, an obvious extension is to re-estimate $\zeta_n$ after a coarse assignment or to measure joint sensitivity of layer pairs; if quantization errors interact, the isolated ranking could change.
- The same $\zeta_n(b)$ criterion could be applied to activation bit-widths and to entropy-model parameters, which the current experiments keep at fixed precision, potentially yielding further model-size savings.
- Since the adaptive search keeps iterations nearly constant as the target ratio drops, the method is likely to scale to larger learned codecs where exhaustive beta search would be impractical.
- The calibration set is only 16 images, so a useful stress test would be to vary calibration size and content and check whether the selected bit-width distribution and BD-Rate remain stable.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes Flexible Mixed Precision Quantization (FMPQ), a method for assigning different bit-widths to different layers of a learned image compression model. Layer sensitivity is quantified by the fractional change in rate-distortion loss when that layer is quantized to a candidate bit-width, and a threshold beta is used to select the bit-width. An adaptive search over beta is proposed to meet a target model-size constraint efficiently. Experiments on Scale Hyperprior, Mean Scale Hyperprior, and Cheng Anchor 2020 report BD-Rate improvements over 8-bit fixed-precision quantization at roughly equal model size on Kodak, Tecnick, and CLIC, along with comparisons to prior quantized LIC methods and an analysis of the learned bit-width distributions.
Significance. If the claimed behavior holds, FMPQ is a simple and practical way to obtain a modest but consistent coding gain over uniform 8-bit quantization of LIC models at the same model size, and the adaptive search substantially reduces the cost of finding a bit-width distribution for a given size constraint. The paper's strengths are the use of the actual RD loss as the assignment criterion rather than a proxy such as weight entropy, evaluation across three architectures and three datasets, and the public release of source code. However, the central claim depends on the reproducibility of the bit-assignment procedure and on the untested assumption that per-layer sensitivities measured in isolation determine the best joint mixed-precision assignment; the current manuscript does not yet establish those points.
major comments (2)
- [Section III-A, Algorithm 1] Algorithm 1 as written is internally inconsistent with the prose and with Fig. 2. The text states that FMPQ assigns the lowest bit-width satisfying zeta_n(b) < beta, and Fig. 2 indicates that the bit-width is decremented when zeta_n < beta. The pseudocode instead decrements only on the failure branch: it iterates b from bmax down to 2 and, whenever zeta_n >= beta, sets b_theta_n <- b - 1 and breaks. Thus the chosen bit-width is one below the first failing bit-width in a descending scan, which is not the lowest bit-width satisfying zeta_n < beta and can violate the beta tolerance. A layer with zeta_n < beta at every tested b finishes the loop with b_theta_n = 2, so the least sensitive layers are pushed to the lowest bit-width without ever checking whether an intermediate bit-width would also satisfy the criterion. The pseudocode also computes zeta_n using B_theta, which already contains previously assigned sub-8-bit layers, contradicting Eq. (3), which defines RD_quantized,n(b) as the loss of an identical full-precision model with only the nth layer quantized. Because Tables II-IV could have been produced by either the text version or the pseudocode version, the reported method is not reproducible as written. Please correct the pseudocode to match the described algorithm and clarify which version was used in the experiments.
- [Section III-A, Eq. (3); Table II] The causal claim that the RD-loss-sensitivity criterion is responsible for the observed improvement is not validated. Eq. (3) measures each layer's sensitivity with all other layers at full precision, but the final quantized model has all layers quantized simultaneously and is then fine-tuned; quantization errors in different layers can interact, and the paper never tests whether isolated sensitivities predict the best joint assignment. The experiments do not include a control such as random mixed-precision bit assignments at matched model size, an assignment based on a different criterion (e.g., weight entropy), or a joint sensitivity measure. Without such an ablation, the BD-Rate gains in Table II could stem from the general benefit of mixed precision or from the QAT recipe rather than from the specific zeta-based criterion. Please add at least one matched-size control to support the central claim.
minor comments (6)
- [Section III-C, Algorithm 2] The control flow of Algorithm 2 is ambiguous: the line 'beta <- beta + alpha_beta' appears under the inner else-if branch depending on indentation, but the surrounding text says the increment is applied at each search step. Please restructure the pseudocode with explicit begin/end blocks so the update rule is unambiguous.
- [Table I and Section IV-C] Table I's caption states that the exhaustive search uses an initial value of 0.01 and fixed increments of 0.01, while Section IV-C says 'With an initial value of beta set to 1'. Please reconcile these values.
- [Table III] The header 'Compression Raio' should be 'Compression Ratio'.
- [Appendix B, Eq. (4)] Eq. (4) refers to 'the quantization parameters s and w', but the quantization parameters introduced in Eq. (2) are s and z; please correct the notation and clarify whether the bias term (Cout) is quantized at the same bit-width b.
- [Algorithm 1] Algorithm 1 loops 'for n = 0 to N' while the text elsewhere numbers layers from 1 (e.g., Fig. 3 uses layers [0,3] for the encoder), and the final iteration count is unclear. Please unify the indexing convention.
- [Section IV-D, Table IV] In the comparison with the method from [13], the authors explain the worse BD-Rate by noting that [13] uses four quality levels whereas FMPQ uses six. Since BD-Rate is sensitive to the number and placement of RD points, please also report a comparison computed on the same set of lambda values to substantiate this explanation.
Circularity Check
No significant circularity: the bit-width assignment is a sensitivity heuristic validated on held-out datasets, not a fit to the reported metric.
full rationale
FMPQ's assignment rule uses the measured fractional change in RD-loss, ζn(b) = |(RD_quantized,n(b) − RD_full-precision)/RD_full-precision|, as in Eq. (3), and the text states that 'the smallest bit-width from L that satisfies ζn(b) < β, is set as the value of bn.' The bit-width search is performed on a 16-image calibration set D_calib, and the quantized model is then fine-tuned with quantization-aware training; final BD-Rate values are measured on Kodak, Tecnick, and CLIC against full-precision baselines, as reported in Table II. Nothing in Eq. (3) or Algorithm 1 forces the post-QAT test BD-Rate to be lower than 8-bit FPQ; the reported improvement is an empirical outcome rather than an identity. The adaptive search only enforces a model-size constraint through CRtarget, which is an experimental design choice and not a hidden fit to BD-Rate. Self-citations [4] and [5] are background LIC references and are not load-bearing. The only notable defect is internal: Algorithm 1's pseudocode ('if ζn ≥ β then bθn ← b − 1; break') contradicts the text's 'assigns the lowest bit-width that satisfies ζn < β', and the loop accumulates previously quantized layers rather than isolating layer n as Eq. (3) states. This is a reproducibility and correctness concern, not circularity, so it does not raise the circularity score.
Assumptions & free parameters
free parameters (6)
- beta (RD-loss tolerance threshold) =
approximately 0.01 initially, varies with CRtarget
- Adaptive search multiplier (large gap) =
5
- Adaptive search multiplier (medium gap) =
2
- Search reversal factor =
0.1
- Initial adaptive step alpha_beta =
1
- Calibration dataset size =
16 images
assumptions (4)
- ad hoc to paper Per-layer sensitivity measured in isolation predicts optimal joint mixed-precision bit assignment.
- domain assumption Sixteen COCO images adequately represent the RD-loss landscape for calibration.
- domain assumption Quantization-aware training using only the RD-loss, with leaky-clip, recovers performance lost to weight quantization.
- domain assumption Uniform, channel-wise quantization with learnable scale and zero-point is sufficient for LIC weight quantization.
Cite this review
Pith. "Pith review of Flexible Mixed Precision Quantization for Learned Image Compression." pith.science (2026). https://pith.science/paper/OUTME62D
@misc{pith2026250601221,
author = {Pith},
title = {Pith review of: Flexible Mixed Precision Quantization for Learned Image Compression},
year = {2026},
howpublished = {\url{https://pith.science/paper/OUTME62D}},
note = {Machine review of arXiv:2506.01221}
}
read the original abstract
Despite its improvements in coding performance compared to traditional codecs, Learned Image Compression (LIC) suffers from large computational costs for storage and deployment. Model quantization offers an effective solution to reduce the computational complexity of LIC models. However, most existing works perform fixed-precision quantization which suffers from sub-optimal utilization of resources due to the varying sensitivity to quantization of different layers of a neural network. In this paper, we propose a Flexible Mixed Precision Quantization (FMPQ) method that assigns different bit-widths to different layers of the quantized network using the fractional change in rate-distortion loss as the bit-assignment criterion. We also introduce an adaptive search algorithm which reduces the time-complexity of searching for the desired distribution of quantization bit-widths given a fixed model size. Evaluation of our method shows improved BD-Rate performance under similar model size constraints compared to other works on quantization of LIC models. We have made the source code available at gitlab.com/viper-purdue/fmpq.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[13]
Integer quantized learned image compression,
Geun-Woo Jeon, SeungEun Yu, and Jong-Seok Lee, “Integer quantized learned image compression,” in 2023 IEEE International Conference on Image Processing (ICIP) . IEEE, 2023, pp. 2755–2759
work page 2023
-
[1]
Variational image compression with a scale hyperprior,
Johannes Ball ´e, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston, “Variational image compression with a scale hyperprior,” arXiv preprint arXiv:1802.01436 , 2018
arXiv 2018
-
[2]
Joint au- toregressive and hierarchical priors for learned image compression,
David Minnen, Johannes Ball ´e, and George D Toderici, “Joint au- toregressive and hierarchical priors for learned image compression,” Advances in neural information processing systems , vol. 31, 2018
work page 2018
-
[3]
Learned image compression with discretized gaussian mixture like- lihoods and attention modules,
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, and Jiro Katto, “Learned image compression with discretized gaussian mixture like- lihoods and attention modules,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 7939– 7948
work page 2020
-
[4]
Lossy image compression with quantized hierarchical vaes,
Zhihao Duan, Ming Lu, Zhan Ma, and Fengqing Zhu, “Lossy image compression with quantized hierarchical vaes,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2023, pp. 198–207
work page 2023
-
[5]
Qarv: Quantization-aware resnet vae for lossy image compression,
Zhihao Duan, Ming Lu, Jack Ma, Yuning Huang, Zhan Ma, and Fengqing Zhu, “Qarv: Quantization-aware resnet vae for lossy image compression,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
2023
-
[6]
Mlic: Multi-reference entropy model for learned image compression,
Wei Jiang, Jiayu Yang, Yongqi Zhai, Peirong Ning, Feng Gao, and Ronggang Wang, “Mlic: Multi-reference entropy model for learned image compression,” in Proceedings of the 31st ACM International Conference on Multimedia , 2023, pp. 7618–7627
work page 2023
-
[7]
Integer networks for data compression with latent-variable models,
Johannes Ball ´e, Nick Johnston, and David Minnen, “Integer networks for data compression with latent-variable models,” in International Conference on Learning Representations , 2018
work page 2018
Show all 27 references
-
[8]
Efficient neural image decoding via fixed-point inference,
Weixin Hong, Tong Chen, Ming Lu, Shiliang Pu, and Zhan Ma, “Efficient neural image decoding via fixed-point inference,” IEEE Transactions on Circuits and Systems for Video Technology , vol. 31, no. 9, pp. 3618–3630, 2020
2020
-
[9]
End- to-end learned image compression with fixed point weight quantization,
Heming Sun, Zhengxue Cheng, Masaru Takeuchi, and Jiro Katto, “End- to-end learned image compression with fixed point weight quantization,” in 2020 IEEE International Conference on Image Processing (ICIP) . IEEE, 2020, pp. 3359–3363
2020
-
[10]
Learned image compression with fixed-point arithmetic,
Heming Sun, Lu Yu, and Jiro Katto, “Learned image compression with fixed-point arithmetic,” in 2021 Picture Coding Symposium (PCS) . IEEE, 2021, pp. 1–5
2021
-
[11]
Q-lic: Quantizing learned image compression with channel splitting,
Heming Sun, Lu Yu, and Jiro Katto, “Q-lic: Quantizing learned image compression with channel splitting,” IEEE Transactions on Circuits and Systems for Video Technology, 2022
2022
-
[12]
Rate-distortion optimized post- training quantization for learned image compression,
Junqi Shi, Ming Lu, and Zhan Ma, “Rate-distortion optimized post- training quantization for learned image compression,” IEEE Transac- tions on Circuits and Systems for Video Technology , 2023
2023
-
[14]
Rethinking differentiable search for mixed-precision neural networks,
Zhaowei Cai and Nuno Vasconcelos, “Rethinking differentiable search for mixed-precision neural networks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 2349–2358
2020
-
[15]
Entropy-driven mixed-precision quantization for deep network design,
Zhenhong Sun, Ce Ge, Junyan Wang, Ming Lin, Hesen Chen, Hao Li, and Xiuyu Sun, “Entropy-driven mixed-precision quantization for deep network design,” Advances in Neural Information Processing Systems , vol. 35, pp. 21508–21520, 2022
2022
-
[16]
Instance-aware dynamic neural network quantization,
Zhenhua Liu, Yunhe Wang, Kai Han, Siwei Ma, and Wen Gao, “Instance-aware dynamic neural network quantization,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, 2022, pp. 12434–12443
2022
-
[17]
Lossy and lossless (l2) post-training model size compression,
Yumeng Shi, Shihao Bai, Xiuying Wei, Ruihao Gong, and Jianlei Yang, “Lossy and lossless (l2) post-training model size compression,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 17546–17556
2023
-
[18]
Up or down? adaptive rounding for post- training quantization,
Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Christos Louizos, and Tijmen Blankevoort, “Up or down? adaptive rounding for post- training quantization,” in International Conference on Machine Learn- ing. PMLR, 2020, pp. 7197–7206
2020
-
[19]
Qdrop: Randomly dropping quantization for extremely low-bit post-training quantization,
Xiuying Wei, Ruihao Gong, Yuhang Li, Xianglong Liu, and Fengwei Yu, “Qdrop: Randomly dropping quantization for extremely low-bit post-training quantization,” arXiv preprint arXiv:2203.05740 , 2022
2022 arXiv
-
[20]
Learned step size quantiza- tion,
Steven K Esser, Jeffrey L McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmendra S Modha, “Learned step size quantiza- tion,” arXiv preprint arXiv:1902.08153 , 2019
1902 arXiv
-
[21]
Network quantization with element-wise gradient scaling,
Junghyup Lee, Dohyung Kim, and Bumsub Ham, “Network quantization with element-wise gradient scaling,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 6448– 6457
2021
-
[22]
Mr. biq: Post-training non-uniform quantization based on minimizing the reconstruction error,
Yongkweon Jeon, Chungman Lee, Eulrang Cho, and Yeonju Ro, “Mr. biq: Post-training non-uniform quantization based on minimizing the reconstruction error,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 12329–12338
2022
-
[23]
Compressai: a pytorch library and evaluation platform for end-to-end compression research,
Jean B ´egaint, Fabien Racap ´e, Simon Feltman, and Akshay Pushparaja, “Compressai: a pytorch library and evaluation platform for end-to-end compression research,” arXiv preprint arXiv:2011.03029 , 2020
2011 arXiv
-
[24]
Microsoft coco: Common objects in context,
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Per- ona, Deva Ramanan, Piotr Doll ´ar, and C Lawrence Zitnick, “Microsoft coco: Common objects in context,” in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proce...
2014
-
[25]
Kodak lossless true color image suite (photocd pcd0992),
Eastman Kodak, “Kodak lossless true color image suite (photocd pcd0992),” URL http://r0k. us/graphics/kodak , vol. 6, 1993
1993
-
[26]
Testimages: A large data archive for display and algorithm testing,
Nicola Asuni and Andrea Giachetti, “Testimages: A large data archive for display and algorithm testing,” Journal of Graphics Tools , vol. 17, no. 4, pp. 113–125, 2013
2013
-
[27]
Workshop and challenge on learned image compression (clic2020),
George Toderici, Wenzhe Shi, Radu Timofte, Johannes Balle Lu- cas Theis, Eirikur Agustsson, Nick Johnston, and Fabian Mentzer, “Workshop and challenge on learned image compression (clic2020),” 2020. APPENDIX A. Coding performance of quantized LIC models We compare the performa...
2020
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.