Pith. sign in

REVIEW 3 major objections 5 minor 59 references

Exploiting Latent Properties to Optimize Neural Codecs

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Neural codecs can exploit decoder-side entropy gradients and lattice quantizers to save 1–3% rate at equal quality.

desk verdict The lattice VQ drop-in is a solid, checkable contribution; the KKT-based Latent Shift theory is invalid, leaving an empirically plausible heuristic in need of better evidence. read the letter →

arxiv 2501.01231 v1 pith:LIFS76IB submitted 2025-01-02 cs.CV cs.LG

Muhammet Balcilar , Bharath Bhushan Damodaran , Karam Naser , Franck Galpin , Pierre Hellier This is my paper · ORCID
classification cs.CVcs.LG MSC 94A0894A3468T07
keywords neuralimagecompressionvideoentropygradientlatentshiftuniformvectorquantizationhexagonallatticetruncatedoctahedronrate-distortionoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Neural image and video codecs compress by learning latent codes that are quantized and entropy-coded; nearly all current models use fixed-width uniform scalar quantization. The paper argues that, for expressive enough networks, non-uniform scalar quantization cannot beat uniform scalar quantization, so the remaining cheap gain is vector quantization with predefined optimal grids — regular hexagons in 2D and truncated octahedra in 3D — applied to off-the-shelf codecs without retraining. Its second and more consequential claim is that the gradient of the entropy with respect to the decoded latents, which the decoder can compute, is correlated with the reconstruction-error gradient, which the decoder cannot compute; shifting the decoded latents along that entropy gradient therefore improves distortion at equal rate. Across several pretrained image and video codecs, the two mechanisms together save 1–3% of bitrate at the same quality, with larger gains at low bitrates, and the entropy-gradient trick alone transfers to a traditional video codec with a small gain.

What carries the argument

The two load-bearing mechanisms are the space-tessellation lattice quantizer and Latent Shift. The lattice quantizer is a fixed, non-learned grid — regular hexagons in 2D, truncated octahedra in 3D — chosen because these shapes tile space with minimum moment of inertia; quantization becomes nearest-lattice-point assignment, and the PMF of each vector code is obtained by numerically integrating the product of the scalar latent PDFs over the lattice cell. Latent Shift is the operation that, after decoding, moves the main latents as $\hat{y} \leftarrow \hat{y} + \rho_h \nabla_{\hat{y}}(-\log p_h(\hat{y};\hat{z},\Theta))$ and the side latents as $\hat{z} \leftarrow \hat{z} + \rho_f \nabla_{\hat{z}}(-\log p_f(\hat{z};\Psi))$, where the step sizes $\rho_h,\rho_f$ are chosen from eight candidates at the encoder and transmitted as side information. The connecting identity is the per-sample KKT condition from Corollary 2.1, which equates the direction of the entropy gradient at the decoder with the negative of the reconstruction-error gradient at the encoder, thereby licensing the former as a proxy for the latter.

What would settle it

On any pretrained neural codec, compute the per-image correlation between the entropy gradient and the reconstruction-error gradient with respect to the decoded main latents; if a substantial fraction of images show near-zero or positive correlation, or if Latent Shift's average BD-rate gain no longer exceeds the paper's Sign Shift baseline at equal signaling bits, the central correlation claim is falsified. A more direct test is to evaluate the Jacobian identity in Corollary 2.1 numerically on a real trained model and check whether the chain rule from network parameters to decoded latents is invertible in practice.

Watch

Extended reading notes

Core claim

The paper's central claim is twofold. First, Theorem 1 states that any non-uniform scalar quantizer used by a neural codec can be re-expressed as an invertible elementwise nonlinearity followed by nearest-integer rounding, so once the encoder and decoder are expressive enough, uniform scalar quantization is sufficient and non-uniform scalar grids cannot improve rate–distortion performance. Because vector quantization is not covered by that argument, the paper replaces the scalar grid with predefined uniform lattice quantizers: a regular hexagonal grid in two dimensions and a truncated octahedral grid in three, with cell volume matched to the latent distribution, and computes the needed code probabilities by numerical integration over the lattice cells. Second, applying Karush–Kuhn–Tucker conditions to the codec's training objective and assuming they hold per input image, the paper derives Corollary 2.1: the total entropy gradient with respect to the decoded main latents is proportional and opposite to the reconstruction-error gradient, even though the latter is unavailable at the decoder. Measured correlations between the two gradients range from about -0.1 to -0.5, and the paper exploits this by shifting the decoded latents by a small multiple of the entropy gradient (Latent Shift), with the step size selected from eight candidates and signaled in the bitstream. On the tested image codecs the combined method yields about 1–3% BD-rate savings, and on two neural video codecs about 2–2.7%.

Load-bearing premise

The load-bearing premise is that a codec trained to satisfy the KKT conditions on average over its training set also satisfies them for each individual image, so the entropy gradient computed at the decoder points in the same direction as the reconstruction-error gradient that the decoder cannot see; the measured -0.1 to -0.5 correlations show this premise is only approximately true.

Editorial extensions

If this is right

  • Off-the-shelf neural codecs, with no retraining and no architecture changes, can save roughly 1–3% of bitrate at equal quality by switching to a hexagonal or truncated octahedral lattice and applying Latent Shift.
  • The gains grow as bitrate shrinks: the paper reports around 3.5% BD-rate gain at the low-quality end for one image codec and over 4% for a video codec, so the mechanism matters most exactly where compression is hardest.
  • Latent Shift combines almost additively with encoder-side fine-tuning: on one codec the fine-tuning gain of -5.77% becomes -7.47% when Latent Shift is added, indicating the two mechanisms are largely orthogonal.
  • A universal-step-size variant of Latent Shift transfers to a traditional video codec, saving about 0.1% on luma and more on chroma channels while leaving encoding and decoding time essentially unchanged.
  • The decoder-side overhead of Latent Shift is below 1% because Gaussian entropy gradients have closed forms; the extra encoding cost comes almost entirely from the step-size search.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The per-sample KKT assumption is the fragile hinge of the paper: KKT holds in expectation over the training set, so the reported -0.1 to -0.5 correlations are empirical facts rather than consequences; a codec trained on a different distribution or with a different loss may show much weaker correlation and smaller gains.
  • The paper's own upper-bound experiment — true gradients would yield roughly 25% rate saving — implies the proxy currently captures only a small fraction of the available signal; a learned shift function parameterized by local latent statistics could plausibly recover more of that headroom.
  • Retraining the decoder through a continuous relaxation of the hexagonal or octahedral grid, which the paper's conclusion leaves as future work, would address the sub-optimality caused by mismatch between lattice cells and the trained scalar-grid decoder and could push vector-quantization gains beyond the reported 1–3%.
  • Any training change that strengthens per-image stationarity — shorter amortization gaps, per-image adaptation, or explicit gradient-alignment regularizers — should increase the Latent Shift gain; the paper's fine-tuning experiment already points in that direction.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes two post-hoc, training-free enhancements for neural image and video codecs. First, it argues (Theorem 1) that non-uniform scalar quantization is redundant for sufficiently expressive neural codecs, and proposes replacing uniform scalar quantization with fixed uniform vector quantization using hexagonal (2D) or truncated-octahedral (3D) lattices, with PMFs computed by numerical integration. Second, it proposes 'Latent Shift': after decoding, shift the latents along the (available) entropy gradient, which the authors claim is correlated with the (unavailable) reconstruction-error gradient via KKT conditions. Experiments on several pretrained codecs and datasets report 1–3% BD-rate savings for the combined method, and about 0.1% for a traditional codec (ECM).

Significance. The VQ proposal is theoretically well-motivated and is validated by analytical MSE calculations and simulations; the experimental comparison spans eight image/video codecs and multiple datasets, and the gains, while modest, are consistently negative BD-rate improvements. The Latent Shift proposal, if valid, is attractive because it requires no retraining, adds negligible decoding complexity, and includes sensible baselines (Random Shift, Scalar Shift, Sign Shift). However, the theoretical derivation of the proxy relationship is invalid as written, and the empirical correlations are weak; the practical gain therefore rests on the heuristic value of the step, for which the paper provides only limited statistical support. The authors should be credited for the breadth of experiments and for including complexity analysis and a standardized-codec evaluation.

major comments (3)
  1. [Appendix C-B / Corollary 2.1] The proof of Corollary 2.1 is mathematically invalid. The step 'multiply both sides with Jacobian matrix J(Φ, ẑ) where J(Φ,ẑ)_{i,j}=∂Φ_j/∂ẑ_i' is not a valid change of gradient variables. For a composition f(ẑ(Φ)), the chain rule gives ∇_ẑ f = (∂ẑ/∂Φ)^T ∇_Φ f, not multiplication by ∂Φ/∂ẑ. Moreover, Φ is an independent parameter vector, not a function of ẑ, so ∂Φ_j/∂ẑ_i is generally undefined (and in any case not the correct transposed Jacobian). Consequently equations (5) and (6) do not follow from (3) and (4), and the KKT derivation does not establish the claimed correlation of −1. In addition, (3) and (4) are expectations over the training distribution; the paper drops the expectation without justification to obtain per-sample conditions. The authors should either supply a correct derivation or explicitly restate the entropy-gradient proxy as an empirical heuristic.
  2. [Section IV (Corollary 2.1 and following)] The theoretical prediction is a correlation of −1 between the entropy gradient and the reconstruction-error gradient, but the measured correlations are only in the range −0.1 to −0.5 (and −0.15 to 0.1 for the side latents). The paper acknowledges this discrepancy but still uses the KKT result as the motivation for Latent Shift. Because the derivation in Appendix C-B is invalid, the empirical correlation is the only support for the proxy, and the reported values are weak. No confidence intervals, significance tests, or per-image breakdowns are provided for these correlations. The authors should provide bootstrap or permutation-based intervals and demonstrate that the correlation is not an artifact of the particular test images or of the step-size search over eight candidates.
  3. [Section V-A (Tables I and II)] The headline claim of 1–3% BD-rate savings is based on averages over datasets, with no error bars, per-image variance, or significance tests. Given the small magnitude of the gains, it is important to show that the improvement is consistent across images rather than driven by a few outliers. In particular, for the Latent Shift component, the per-image step-size search (3 bits per step, reported as included in the bitstream) should be compared with a universal step size to disentangle the contribution of the gradient direction from the benefit of per-image signaling. The Random Shift baseline in Table IV is only evaluated on two codecs, and the caption states that its 10-bit overhead is ignored, which favors the baseline; this should be stated more prominently and the comparison repeated with the overhead included.
minor comments (5)
  1. [Section V-B] The text 'ranging from 5 to 10 orders of magnitude higher' should read 'a factor of 5 to 10 higher' (Table III shows x10.1, not 10^10).
  2. [Section V-D] The statement 'despite traditional codecs not using gradient-based optimization ... we can still argue that the KKT condition exists' is not supported; please clarify or remove.
  3. [Table IV caption] The sentence 'we ignored these bits' should be clarified: the alternatives are evaluated without their signaling overhead, which is favorable to them; for a fair comparison the overhead should be included.
  4. [Figure 5c] The axes should be labeled precisely; the text refers to 'correlation between improvement on reconstruction quality and correlation of gradients' but the caption and text should define the exact quantities plotted.
  5. [Section III-A] In Figure 3, 'RD performances of different volume uniform SQ' should be 'different grid volumes' or 'unit-volume grids' for clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's contributions are empirical recipes evaluated on independent test data; the invalid KKT-to-Jacobian step is a correctness concern, not a circular reduction.

full rationale

The paper's two contributions are (i) replacing uniform scalar quantization with predefined uniform vector quantization grids, and (ii) shifting decoded latents along the entropy gradient as a proxy for the unavailable reconstruction-error gradient. Neither reduces to its own inputs by construction. The uniform-VQ contribution is an empirical adaptation of off-the-shelf codecs using predefined hexagonal/truncated-octahedral lattices; the PMF computation in Eq. (2) is an explicitly stated numerical integration of the baseline entropy model over the new grid, not a hidden fit to the reported gains. The latent-shift contribution uses a per-image step-size search with the chosen step explicitly signaled (3 bits each), and it is benchmarked against random, scalar, and sign-shift alternatives, so the measured gains are not forced by a fitted parameter being renamed as a prediction. The KKT derivation in Section IV and Appendix C-B is mathematically problematic: the per-sample stationarity assumption is unjustified and the Jacobian multiplication does not correctly change gradient variables from network parameters to latents. However, mathematical invalidity is not circularity; the paper transparently reports that the measured correlations are far from the predicted -1 (ranging from -0.1 to -0.5 for Eq. (6) and -0.15 to 0.1 for Eq. (5)) and explicitly attributes the discrepancy to the assumption holding only in expectation. The self-citations ([34], [35]) are not load-bearing: the present paper contains the formal proofs and the experiments supporting the central claims, and the JVET adoption is an external standardization outcome rather than a substitute for evidence. Overall, the central claims stand or fall on independent empirical evaluation, not on a circular derivation chain.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claims rest on a few free parameters (step-size candidates, ECM alpha, lattice volume), several domain assumptions inherited from the codec's entropy model, and two ad hoc assumptions introduced by the paper: the per-sample KKT condition and the invalid Jacobian variable change. The latter is the most fragile part of the theoretical story. No new physical entities are introduced.

free parameters (3)
  • Step size candidates for Latent Shift (rho_f, rho_h) = 8 candidates each, values not specified
    The encoder tests 8 step sizes per gradient and signals the best index with 3 bits; the choice of candidate set is a hand-picked design parameter that affects the achievable gain.
  • ECM universal shift coefficient alpha = not reported in the paper
    For the traditional codec extension, alpha = a*rho is fine-tuned over the training set and hard-coded into sender and receiver (Section V-D), a fitted constant.
  • VQ lattice cell volume = 1 (unitary)
    Chosen to match the 1-bin width of the original uniform SQ quantizer; not re-optimized, which limits the achievable VQ gain.
assumptions (5)
  • ad hoc to paper The KKT stationarity conditions for the training loss hold for each individual input, not just in expectation over the training set.
    Corollary 2.1 drops the expectation in equations (3)-(4), an assumption the authors acknowledge and that is false in general; the entire Latent Shift motivation depends on it.
  • ad hoc to paper The Jacobian of parameters Phi with respect to latents z is invertible enough to re-express gradients, i.e., dPhi_j/dz_i exists and can be used to change variables.
    Appendix C-B multiplies by J(Phi,z) to convert gradient w.r.t. Phi into gradient w.r.t. z; this is not a valid chain rule step because z is not a function of Phi alone.
  • domain assumption A shallow neural network can approximate the invertible monotonic function f that converts non-uniform SQ borders into uniform borders.
    Theorem 1 relies on universal approximation; the paper assumes expressivity is sufficient in modern codecs.
  • domain assumption The latent dimensions are independent given the hyperprior, so the joint PDF factorizes as a product of marginals for VQ PMF computation.
    Equation (2) integrates the product of scalar PDFs over the lattice cell, which is the entropy model's own independence assumption, not a new claim.
  • standard math Lattice quantizers (hexagonal, truncated octahedron) are optimal for uniform sources at high rates (Gersho's result), and this translates to lower MSE at the same rate for the approximately Gaussian latent sources.
    Used in Section III-A and Appendix B to justify Hex-Quant and Oct-Quant.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exploiting Latent Properties to Optimize Neural Codecs." pith.science (2026). https://pith.science/paper/LIFS76IB

@misc{pith2026250101231,
  author       = {Pith},
  title        = {Pith review of: Exploiting Latent Properties to Optimize Neural Codecs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LIFS76IB}},
  note         = {Machine review of arXiv:2501.01231}
}
read the original abstract

End-to-end image and video codecs are becoming increasingly competitive, compared to traditional compression techniques that have been developed through decades of manual engineering efforts. These trainable codecs have many advantages over traditional techniques, such as their straightforward adaptation to perceptual distortion metrics and high performance in specific fields thanks to their learning ability. However, current state-of-the-art neural codecs do not fully exploit the benefits of vector quantization and the existence of the entropy gradient in decoding devices. In this paper, we propose to leverage these two properties (vector quantization and entropy gradient) to improve the performance of off-the-shelf codecs. Firstly, we demonstrate that using non-uniform scalar quantization cannot improve performance over uniform quantization. We thus suggest using predefined optimal uniform vector quantization to improve performance. Secondly, we show that the entropy gradient, available at the decoder, is correlated with the reconstruction error gradient, which is not available at the decoder. We therefore use the former as a proxy to enhance compression performance. Our experimental results show that these approaches save between 1 to 3% of the rate for the same quality across various pretrained methods. In addition, the entropy gradient based solution improves traditional codec performance significantly as well.

Figures

Figures reproduced from arXiv: 2501.01231 by the authors.

Figure 1
Figure 1. Block diagram of the state-of-the art neural codecs. The five dark [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. a) f : y → z transforms a non-uniform quantization map (grid borders b0, . . . bn and grid centers c1, . . . cn) into a uniform map (centers are located on integer where borders are at the middle of two consecutive centers). b) Uniform SQ grids on 2D c) Optimal uniform VQ grids on 2D. (a) (b) (c) [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. RD performances of different volume uniform SQ, regular hexagon [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Correlation between the gradients of the entropy and of the recon [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: BD-Rates of our proposals from baseline codecs for different quality. a) mbt2018mean image codec on Kodak test set b) SSF video codec on UVG [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: a) Regular hexagon and b) Truncated octahedron located at origin. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Histogram of correlation between gradients wrt main latents. The data [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 47 canonical work pages

  1. [1]

    Video compression with rate-distortion autoencoders,

    A. Habibian, T. v. Rozendaal, J. M. Tomczak, and T. S. Cohen, “Video compression with rate-distortion autoencoders,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2019, pp. 7033–7042

    work page 2019

  2. [2]

    Rethinking lossy compression: The rate- distortion-perception tradeoff,

    Y . Blau and T. Michaeli, “Rethinking lossy compression: The rate- distortion-perception tradeoff,” in Proceedings of the 36th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, K. Chaudhuri and R. Salakhutdinov, Eds., vol. 97. PMLR, 09–15 Jun 2019, pp. 675–685

    work page 2019

  3. [3]

    Auto-encoding variational bayes,

    D. P. Kingma, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, ICLR, 2013

    arXiv 2013

  4. [4]

    Lossy image compres- sion with compressive autoencoders,

    L. Theis, W. Shi, A. Cunningham, and F. Husz ´ar, “Lossy image compres- sion with compressive autoencoders,” arXiv preprint arXiv:1703.00395, ICLR, 2017

    arXiv 2017

  5. [5]

    End-to-end optimized image compression,

    J. Ball ´e, V . Laparra, and E. P. Simoncelli, “End-to-end optimized image compression,” arXiv preprint arXiv:1611.01704, ICLR , 2017

    arXiv 2017

  6. [6]

    Vari- ational image compression with a scale hyperprior,

    J. Ball ´e, D. Minnen, S. Singh, S. J. Hwang, and N. Johnston, “Vari- ational image compression with a scale hyperprior,” arXiv preprint arXiv:1802.01436, ICLR, 2018

    arXiv 2018

  7. [7]

    Joint autoregressive and hier- archical priors for learned image compression,

    D. Minnen, J. Ball ´e, and G. D. Toderici, “Joint autoregressive and hier- archical priors for learned image compression,” in Advances in Neural Information Processing Systems , S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds., vol. 31, 2018. 6https://github.com/esa/torchquad

    work page 2018

  8. [8]

    Channel-wise autoregressive entropy models for learned image compression,

    D. Minnen and S. Singh, “Channel-wise autoregressive entropy models for learned image compression,” in ICIP, 2020, pp. 3339–3343

    work page 2020

Show all 59 references
  1. [9]

    Learned image com- pression with discretized gaussian mixture likelihoods and attention modules,

    Z. Cheng, H. Sun, M. Takeuchi, and J. Katto, “Learned image com- pression with discretized gaussian mixture likelihoods and attention modules,” in CVPR, 2020

    2020

  2. [10]

    Enhanced invertible encoding for learned image compression,

    Y . Xie, K. L. Cheng, and Q. Chen, “Enhanced invertible encoding for learned image compression,” in Proceedings of the ACM International Conference on Multimedia , 2021

    2021

  3. [11]

    Checkerboard context model for efficient learned image compression,

    D. He, Y . Zheng, B. Sun, Y . Wang, and H. Qin, “Checkerboard context model for efficient learned image compression,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 14 771–14 780

    2021

  4. [12]

    Learning accurate entropy model with global reference for image compression,

    Y . Qian, Z. Tan, X. Sun, M. Lin, D. Li, Z. Sun, H. Li, and R. Jin, “Learning accurate entropy model with global reference for image compression,” arXiv preprint arXiv:2010.08321, ICLR , 2021

    2010 arXiv

  5. [13]

    Joint global and local hierarchical priors for learned image compression,

    J.-H. Kim, B. Heo, and J.-S. Lee, “Joint global and local hierarchical priors for learned image compression,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2022, pp. 5992–6001

    2022

  6. [14]

    Learned image compression with mixed transformer-cnn architectures,

    J. Liu, H. Sun, and J. Katto, “Learned image compression with mixed transformer-cnn architectures,” in Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , 2023, pp. 14 388– 14 397

    2023

  7. [15]

    Dvc: An end-to-end deep video compression framework,

    G. Lu, W. Ouyang, D. Xu, X. Zhang, C. Cai, and Z. Gao, “Dvc: An end-to-end deep video compression framework,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 11 006–11 015

    2019

  8. [16]

    Scale-space flow for end-to-end optimized video compres- sion,

    E. Agustsson, D. Minnen, N. Johnston, J. Balle, S. J. Hwang, and G. Toderici, “Scale-space flow for end-to-end optimized video compres- sion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 8503–8512

    2020

  9. [17]

    Aivc: Artificial intelligence based video codec,

    T. Ladune and P. Philippe, “Aivc: Artificial intelligence based video codec,” arXiv preprint arXiv:2202.04365 , 2022

    2022 arXiv

  10. [18]

    End-to-end rate-distortion optimized learned hierarchical bi-directional video compression,

    M. A. Yılmaz and A. M. Tekalp, “End-to-end rate-distortion optimized learned hierarchical bi-directional video compression,” IEEE Transac- tions on Image Processing , 2021

    2021

  11. [19]

    Extending neural p-frame codecs for b-frame coding,

    R. Pourreza and T. Cohen, “Extending neural p-frame codecs for b-frame coding,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 6680–6689

    2021

  12. [20]

    Deep contextual video compression,

    J. Li, B. Li, and Y . Lu, “Deep contextual video compression,” in Advances in Neural Information Processing Systems , M. Ranzato, A. Beygelzimer, Y . Dauphin, P. Liang, and J. W. Vaughan, Eds., vol. 34. Curran Associates, Inc., 2021, pp. 18 114– 18 125. [Online]. Available: ht...

    2021

  13. [21]

    Hybrid spatial-temporal entropy modelling for neural video compression,

    ——, “Hybrid spatial-temporal entropy modelling for neural video compression,” in Proceedings of the 30th ACM International Conference on Multimedia , ser. MM ’22. New York, NY , USA: Association for Computing Machinery, 2022, p. 1503–1511. [Online]. Available: https://doi.org/...

    2022

  14. [22]

    Gersho and R

    A. Gersho and R. M. Gray, Vector quantization and signal compression. Springer Science & Business Media, 2012, vol. 159

    2012

  15. [23]

    Soft-to-hard vector quantization for end- to-end learning compressible representations,

    E. Agustsson, F. Mentzer, M. Tschannen, L. Cavigelli, R. Timofte, L. Benini, and L. V . Gool, “Soft-to-hard vector quantization for end- to-end learning compressible representations,” Advances in neural in- formation processing systems , vol. 30, 2017

    2017

  16. [24]

    Unified multivariate gaussian mixture for efficient neural image compression,

    X. Zhu, J. Song, L. Gao, F. Zheng, and H. T. Shen, “Unified multivariate gaussian mixture for efficient neural image compression,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 17 591–17 600

    2022

  17. [25]

    Nvtc: Nonlinear vector transform coding,

    R. Feng, Z. Guo, W. Li, and Z. Chen, “Nvtc: Nonlinear vector transform coding,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 6101–6110

    2023

  18. [26]

    LVQ-V AE:end-to- end hyperprior-based variational image compression with lattice vector quantization,

    S. Kudo, Y . Bandoh, S. Takamura, and M. Kitahara, “LVQ-V AE:end-to- end hyperprior-based variational image compression with lattice vector quantization,” 2023. [Online]. Available: https://openreview.net/forum? id=1pGmKJvneD7

    2023

  19. [27]

    Lvqac: Lattice vector quantization coupled with spatially adaptive companding for efficient learned image compression,

    X. Zhang and X. Wu, “Lvqac: Lattice vector quantization coupled with spatially adaptive companding for efficient learned image compression,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 10 239–10 248

    2023

  20. [28]

    Reducing the amortization gap of entropy bottleneck in end-to-end image compression,

    M. Balcilar, B. Damodaran, and P. Hellier, “Reducing the amortization gap of entropy bottleneck in end-to-end image compression,” in Picture Coding Symposium (PCS) , 2022

    2022

  21. [29]

    Improving inference for neural image compression,

    Y . Yang, R. Bamler, and S. Mandt, “Improving inference for neural image compression,” in Advances in Neural Information Processing IEEE TRANSACTIONS ON IMAGE PROCESSING 14 Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, Eds., vol. 33, 2020, pp. 573–584

    2020

  22. [30]

    Soft then hard: Rethinking the quantization in neural image compression,

    Z. Guo, Z. Zhang, R. Feng, and Z. Chen, “Soft then hard: Rethinking the quantization in neural image compression,” in Proceedings of the 38th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, M. Meila and T. Zhang, Eds., vol. 139. PM...

    2021

  23. [31]

    Content adaptive optimization for neural image compression,

    J. Campos, S. Meierhans, A. Djelouah, and C. Schroers, “Content adaptive optimization for neural image compression,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2019, pp. 6–10

    2019

  24. [32]

    Content adaptive and error propagation aware deep video compression,

    G. Lu, C. Cai, X. Zhang, L. Chen, W. Ouyang, D. Xu, and Z. Gao, “Content adaptive and error propagation aware deep video compression,” in ECCV, vol. 12347, 2020, pp. 456–472

    2020

  25. [33]

    Overfitting for fun and profit: Instance-adaptive data compression,

    T. van Rozendaal, I. A. Huijben, and T. S. Cohen, “Overfitting for fun and profit: Instance-adaptive data compression,” arXiv preprint arXiv:2101.08687, ICLR, 2021

    2021 arXiv

  26. [34]

    Latent-shift: Gradient of entropy helps neural codecs,

    M. Balcilar, B. B. Damodaran, K. Naser, F. Galpin, and P. Hellier, “Latent-shift: Gradient of entropy helps neural codecs,” in 2023 IEEE International Conference on Image Processing (ICIP) , 2023, pp. 920– 924

    2023

  27. [35]

    Ee2-4.1: Shifting quantization center,

    M. Balcilar, K. Naser, F. Gaplin, and F. Le L ´eannec, “Ee2-4.1: Shifting quantization center,” JVET-AE0125, 2023. [Online]. Available: https://jvet-experts.org/doc end user/current document.php?id=13079

    2023

  28. [36]

    Asymmetric numeral systems,

    J. Duda, “Asymmetric numeral systems,” arXiv preprint arXiv:0902.0271, 2009

    2009 arXiv

  29. [37]

    Universally quantized neural compression,

    E. Agustsson and L. Theis, “Universally quantized neural compression,” Advances in neural information processing systems, vol. 33, pp. 12 367– 12 376, 2020

    2020

  30. [38]

    Optimum quantizer performance for a class of non-gaussian memoryless sources,

    N. Farvardin and J. Modestino, “Optimum quantizer performance for a class of non-gaussian memoryless sources,” IEEE Transactions on Information Theory, vol. 30, no. 3, pp. 485–497, 1984

    1984

  31. [39]

    Spectra of quantized signals,

    W. R. Bennett, “Spectra of quantized signals,” The Bell System Technical Journal, vol. 27, no. 3, pp. 446–472, 1948

    1948

  32. [40]

    Video coding using learned latent gan compression,

    M. Shukor, B. B. Damodaran, X. Yao, and P. Hellier, “Video coding using learned latent gan compression,” In Proceedings of the 30th ACM International Conference on Multimedia (MM ’22) , 2022

    2022

  33. [41]

    Nonlinear transform coding,

    J. Ball ´e, P. A. Chou, D. Minnen, S. Singh, N. Johnston, E. Agustsson, S. J. Hwang, and G. Toderici, “Nonlinear transform coding,” IEEE J. Sel. Top. Signal Process. , vol. 15, no. 2, pp. 339–353, 2021. [Online]. Available: https://doi.org/10.1109/JSTSP.2020.3034501

    2021

  34. [42]

    On the structure of vector quantizers,

    A. Gersho, “On the structure of vector quantizers,” IEEE Transactions on Information Theory , vol. 28, no. 2, pp. 157–166, 1982

    1982

  35. [43]

    Asymptotically optimal block quantization,

    ——, “Asymptotically optimal block quantization,” IEEE Transactions on information theory , vol. 25, no. 4, pp. 373–380, 1979

    1979

  36. [44]

    On the computation of integrals of bivariate gaussian distribution,

    V . Savaux and L. Le Magoarou, “On the computation of integrals of bivariate gaussian distribution,” in 2020 IEEE Symposium on Computers and Communications (ISCC) . IEEE, 2020, pp. 1–6

    2020

  37. [45]

    Miettinen, Nonlinear multiobjective optimization

    K. Miettinen, Nonlinear multiobjective optimization . Springer Science & Business Media, 2012, vol. 12

    2012

  38. [46]

    Multiple-gradient descent algorithm (mgda) for multi- objective optimization,

    J.-A. D ´esid´eri, “Multiple-gradient descent algorithm (mgda) for multi- objective optimization,” Comptes Rendus Mathematique , vol. 350, no. 5-6, pp. 313–318, 2012

    2012

  39. [47]

    Compressai: a pytorch library and evaluation platform for end-to-end compression research,

    J. B ´egaint, F. Racap ´e, S. Feltman, and A. Pushparaja, “Compressai: a pytorch library and evaluation platform for end-to-end compression research,” arXiv preprint arXiv:2011.03029 , 2020

    2011 arXiv

  40. [48]

    Kodak Lossless True Color Image Suite (PhotoCD PCD0992)

    E. Kodak, “Kodak Lossless True Color Image Suite (PhotoCD PCD0992).” [Online]. Available: http://r0k.us/graphics/kodak

  41. [49]

    CLIC: Challenge on learned image compression,

    “CLIC: Challenge on learned image compression,” http://compression.cc

  42. [50]

    Uvg dataset: 50/120fps 4k sequences for video codec analysis and development,

    A. Mercat, M. Viitanen, and J. Vanne, “Uvg dataset: 50/120fps 4k sequences for video codec analysis and development,” in Proceedings of the 11th ACM Multimedia Systems Conference , 2020, pp. 297–302

    2020

  43. [51]

    Calculation of average psnr differences between rd curves,

    G. Bjontegaard, “Calculation of average psnr differences between rd curves,” in ITU-T SG16/Q6, 13ˆ¡ th¿ VCEG Meeting, Austin, Texas, USA, April 2001 , 2001

    2001

  44. [52]

    Common test conditions and evaluation procedures for enhanced compression tool testing,

    M. Karczewicz and Y . Ye, “Common test conditions and evaluation procedures for enhanced compression tool testing,” JVET-Y2017, 2022

    2022

  45. [53]

    Closest point search in lattices,

    E. Agrell, T. Eriksson, A. Vardy, and K. Zeger, “Closest point search in lattices,” IEEE Transactions on Information Theory , vol. 48, no. 8, pp. 2201–2214, 2002

    2002

  46. [54]

    Fast quantizing and decoding and algorithms for lattice quantizers and codes,

    J. Conway and N. Sloane, “Fast quantizing and decoding and algorithms for lattice quantizers and codes,” IEEE Transactions on Information Theory, vol. 28, no. 2, pp. 227–232, 1982

    1982

  47. [55]

    Algorithm description of enhanced compression model 9 (ecm 9),

    M. Coban, R.-L. Liao, K. Naser, , J. Str ¨om, and L. Zhang, “Algorithm description of enhanced compression model 9 (ecm 9),” JVET-AD2025, 2023

    2023

  48. [56]

    Multilayer feedforward networks are universal approximators,

    K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural networks, vol. 2, no. 5, pp. 359–366, 1989. Muhammet Balcilar joined InterDigital Inc. in 2021 where he is currently working as a Senior Scientist. He received his B....

    1989

  49. [2013]

    He also holds a PhD degree from the University of Nantes in France in

  50. [2015]

    In 2015-2020, he was a post-doctoral researcher with IRISA, Universite Bre- tagne Sud

    He is currently a senior staff scientist with InterDigital Inc., France. In 2015-2020, he was a post-doctoral researcher with IRISA, Universite Bre- tagne Sud. He was awarded the Prestige and Marie Curie post-doctoral fellowship. His current research interests includes machine...

    2015

  51. [2017]

    He is currently working as a principal scientist in video compression at InterDigital Inc

    His PhD studies were conducted within the European Marie Curie Initial Training Network with the PROVISION project. He is currently working as a principal scientist in video compression at InterDigital Inc. He is actively involved in the standardization process and he is chair...

    1998

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.