{"id":"a5c6efcf-b146-48e7-940d-21c7f9055436","arxiv_id":"2501.12330","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"The paper attributes the gap between ideal and practical lossy image coding to five effects, and reports an estimated rate-distortion upper bound that beats VTM by up to 35 percent on Kodak.","lead":"Learned image codecs are far from Shannon's theoretical compression limit, and this paper breaks that shortfall into five distinct effects. Experiments quantify three of them, reporting up to 35 percent rate savings over the VVC reference codec on Kodak images.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Estimated RD rate in Eq. 10 omits the hyperlatent Z cost even though Z is optimized per sample and needed by the decoder; the 35% benchmark may not be a valid upper bound.","rationale":"The reader's weakest assumption identifies a real confound in isolating the digitization effect, but the headline 35% figure has a more direct vulnerability. The reported estimated rate is the KL for Y only, while the model's entropy model is conditioned on Z. Omitting Z's rate makes the reported number smaller than the true cost of the stochastic system, so it cannot be assumed to be an upper bound on the rate-distortion function. This threatens the paper's strongest claim directly rather than only the decomposition of the gap into effects. No rate term for Z appears in Eq. 10 or anywhere else in Sec. IV, and no artifacts are released, so the omission cannot be checked post hoc. The issue is fixable: a reimplementation with the full ELBO rate would settle whether the 35% figure survives. If the full rate still gives about 35% savings, the benchmark claim is robust; if not, the contribution reduces to the taxonomy and empirical observations. Because the paper's verdict was already conditional, the appropriate disposition remains CONDITIONAL, with the stated conditions extended to include explicit accounting of the hyperlatent rate.","tokens_in":16775,"tokens_out":8708,"duration_ms":95677,"concrete_test":"Recompute the estimated RD curves in Fig. 5 for Res-context by optimizing Y and Z per sample under the full ELBO objective, with rate R = KL(P_Y|X=x || Q_Y|Z) + KL(P_Z|X=x || Q_Z) (using -log Q_Z(z) if Z is deterministic), for the same four lambda values on Kodak. Compare the resulting BD-rate against VTM-22.01 with the paper's evaluation protocol; if the savings fall materially below 35%, the headline benchmark is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim is the 35% rate saving of the estimated RD function over VTM (Sec. IV). For this to be a valid upper bound on the true RD function, every bit needed to communicate the chosen latent must be counted. In the hyperprior/context architecture, the main latent Y is encoded with a conditional Gaussian QY whose parameters are supplied by a hyperlatent Z; Sec. IV.A.3 states that both Y and Z are optimized per sample. Eq. 10, however, computes the estimated rate as KL(P_Y|X=x || Q_Y) only, with no KL or entropy term for Z appearing anywhere. Since Z is image-dependent, a decoder cannot reconstruct QY without receiving Z, so the reported rate is incomplete. Consequently, the 'estimated upper bound' may lie below the true RD function, and the 35% benchmark is not established. This is more direct than the digitization confound: even if architectures were perfectly matched, the estimated curve is missing a mandatory rate component. The paper's use of 'without regarding the asymptotic effect' (Sec. III-C) does not justify omitting Z; that clause concerns one-shot coding overhead, not side information.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes that the gap between Shannon's rate-distortion function and the performance of practical learned image codecs is caused by five effects: modeling, approximation, amortization, digitization, and asymptotic. It formalizes these effects in a latent-variable optimization framework, then reports experiments on the Kodak dataset using two learned image coding architectures with varying capacity, with and without per-sample optimization. The estimated rate-distortion bound is reported to achieve up to 35% rate savings over VTM, and the empirical deterministic system up to 24%; the paper interprets these gaps as quantifying the amortization, digitization, and asymptotic effects.","tokens_in":17041,"tokens_out":6198,"duration_ms":68019,"significance":"If the central benchmark were valid, the 35% rate-savings claim would be an important reference point for learned image coding, and the five-effect taxonomy would provide a useful language for discussing why practical codecs fall short of the Shannon bound. The paper's strengths include systematic scaling experiments over model size, the use of per-sample optimization to probe the amortization effect, and comparisons against VTM and recent learned codecs on a standard dataset. The taxonomy is reasonable as an organizational contribution. However, the quantitative headline is not currently supported because the estimated-rate calculation in Eq. (10) omits the mandatory hyperlatent cost, and the digitization-effect quantification is confounded by architectural and objective differences.","major_comments":[{"comment":"The rate in Eq. (10) is computed only for the main latent Y, as KL(P_{Y|X=x} || Q_Y) / log(2). However, Sec. IV.A.3 states that both Y and Z are per-sample optimized, and in the hyperprior architecture the parameters of Q_Y are supplied by the hyperlatent Z; a decoder cannot reconstruct Q_Y without receiving Z. The estimated rate should therefore include a side-information term for Z, for example KL(P_{Y,Z|X=x} || Q_{Y,Z}) or an equivalent cost. Omitting Z means Eq. (10) is not a valid upper bound on the rate-distortion function, and the 35% rate-savings claim in Sec. IV.B is not established. The sentence in Sec. III-C about not regarding the asymptotic effect concerns one-shot coding overhead, not the cost of transmitting the hyperlatent, so it does not justify the omission.","section":"IV.A.3 and Eq. (10)"},{"comment":"The gap between the estimated RD curve (stochastic Gaussian latents, no quantization) and the empirical RD curve (deterministic quantized latents under Eq. (6)) is used to quantify the digitization effect. These two systems differ not only in the presence of quantization but also in the form of PY|X (Gaussian noise versus deterministic rounding), in the training objective (Eq. (5) versus Eq. (6)), and in the inference procedure. The measured gap therefore also contains approximation and modeling differences, so the reported magnitudes for the digitization effect are not isolated. The paper's phrase 'provide some insights' is more cautious, but the surrounding analysis and Figs. 3-4 present the gap as the digitization effect; this requires either a controlled comparison or a clear downgrading of the quantitative claim.","section":"IV.B and Fig. 4"},{"comment":"The five effects are identified and defined by construction rather than derived as a formal decomposition of the total gap. In particular, the boundaries between the modeling effect, approximation effect, and amortization effect depend on the choice of the reference optimal PY|X and on the restricted feasible set A, which are not made operational. This is acceptable for a conceptual taxonomy, but the paper should state explicitly that the effects are not shown to be disjoint or exhaustive; otherwise the reader may infer a quantitative decomposition that the theoretical analysis does not provide.","section":"III.A"}],"minor_comments":[{"comment":"The word 'quantitively' should be 'quantitatively' in the Abstract and in Sec. I.","section":"Abstract and Sec. I"},{"comment":"The passage introducing Eq. (10) is unfinished: 'We calculated the estimated rate as For x ~ PX ...' should be rewritten as a complete derivation.","section":"IV.A.2"},{"comment":"The sentence containing Eq. (4) has a grammatical error ('Equation 4 is minimized it over all feasible'); please revise.","section":"III.A.2"},{"comment":"The claim that the asymptotic effect is negligible in current advanced deterministic schemes is asserted without quantitative support; a short calculation of the one-shot overhead for the coded image sizes would strengthen the claim.","section":"III.B"},{"comment":"Please clarify whether the reported 'up to' rate savings refer to the best image, the best model, or the dataset average, since the text uses both 'up to' and averaged BD-rate values.","section":"Figs. 2-4"}],"recommendation":"major_revision","confidential_remarks":"The strongest issue is the missing hyperlatent cost in Eq. (10); this is a direct accounting error in the central benchmark and must be fixed before publication. The digitization-effect quantification also needs a controlled comparison. If the authors can recompute the estimated bound with a correct joint rate term and adjust the claims accordingly, the paper could become publishable, since the taxonomy and the scaling experiments are useful."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe useful part of this paper is the five-effect decomposition of the gap between Shannon rate-distortion theory and learned image codecs: modeling, approximation, amortization, digitization, and asymptotic effects. That taxonomy is a clear synthesis of known ideas from the VAE literature and source coding, and it gives practitioners a shared vocabulary. The experiments that scale model width and apply per-sample optimization are also real work, and the trends they show are credible.\n\nThe problem is the headline number. The claimed 35% rate saving over VTM is computed with Eq. 10, which counts only KL(P_{Y|X}||Q_Y) bits. But the model is a hyperprior/context architecture, and Sec. IV.A.3 says both Y and Z are optimized per sample. The decoder cannot know the parameters of Q_Y without receiving Z, so any valid rate must include the cost of Z. Eq. 10 omits it entirely. The note in Sec. III-C about ignoring the asymptotic effect is about one-shot coding overhead, not about side information. Therefore the 'estimated rate-distortion function' is not a valid upper bound on the true RD function; the 35% figure is not established.\n\nThis is a more direct flaw than the digitization confound. The digitization estimate compares a stochastic Gaussian system with a deterministic quantized system that differ in architecture, noise model, and training objective, so the reported magnitudes are not causal. But that is a softer, fixable issue, and the authors only claim it 'provides some insights'.\n\nI also note that no code or artifacts are released, which makes it hard to check whether the 35% figure would survive after adding the Z term. My guess is the corrected bound would be weaker, though it might still beat VTM by a meaningful margin. The taxonomy stands regardless.\n\nBottom line: this paper deserves serious peer review. The central quantitative claim is fixable, the taxonomy is valuable, and the scaling experiments are worth publishing. I would want to see the corrected rate computation and ideally released code before citing the numbers, but the conceptual framing is a useful contribution.\n\nRecommendation: send it to review, with a strong request to address the missing hyperlatent cost and to report the corrected upper bound.","headline":"The five-effect taxonomy and scaling experiments are useful, but the 35% RD upper bound omits the hyperlatent rate and is not a valid upper bound.","tokens_in":17524,"tokens_out":4560,"would_cite":false,"duration_ms":49157,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["94A34","94A08"],"pacs":[],"model":"deepseek-v4-flash","headline":"Lossy image coding's shortfall to Shannon's rate-distortion limit splits into five named effects, and an estimated upper bound beats VTM by 35% on Kodak.","keywords":["information theory","learned image coding","lossy image coding","rate-distortion function","amortization effect","digitization effect","quantization surrogate","per-sample optimization"],"falsifier":"Hold the network architecture, entropy model, and loss fixed and train one version with continuous Gaussian latents and the Eq. (10) rate plus another with quantization and the Eq. (6) entropy rate; if the BD-rate gap between the two curves is much smaller than the residual gap reported in Fig. 4, the magnitudes attributed to digitization are not supported.","tokens_in":16563,"feed_emoji":"🖼️","tokens_out":10454,"duration_ms":98531,"temperature":0.7,"pith_summary":"Lossy image compression is bounded in principle by Shannon's rate-distortion function—the minimum bitrate needed to reconstruct a source within a given distortion—but practical codecs do not know how far they sit from that bound. This paper argues the shortfall is not one inefficiency but five distinct effects: modeling, approximation, amortization, digitization, and asymptotic, each with its own cause. Quantitatively, it reports that an estimated upper bound on the rate-distortion function, computed from continuous Gaussian latent variables without quantization, achieves 35% rate savings over the VVC reference codec VTM on Kodak, while the best empirical quantized model reaches 24%. If the bound is tight, current codecs—traditional and learned—still have large headroom, and the five-effect taxonomy says which bottleneck to attack next.","feed_headline":"Estimated image-coding bound beats VTM by 35% on Kodak","feed_subtitle":"The paper splits the shortfall into five named effects and quantifies the last three, mapping where future codecs can still gain.","key_machinery":"The argument is carried by the contrast between two optimization objectives. Eq. (5), a continuous stochastic latent system, treats the latent as a Gaussian random variable and uses a KL divergence as the rate estimate; because it avoids quantization and entropy coding, it serves as an upper-bound estimator of the true rate-distortion function. Eq. (6), the deterministic quantized system used by practical codecs, treats the latent as a discrete variable obtained by rounding and uses its entropy as the rate. Per-sample optimization (Eqs. (7)-(8)) turns the amortization effect into a measured quantity: the rate-distortion gain from optimizing latents for each image rather than using shared network parameters. The residual gap between the estimated and empirical curves is then attributed to the digitization effect, while one-shot coding bounds place the asymptotic effect.","core_discovery":"The paper's central claim is that the gap between the ideal rate-distortion function and the empirical rate-distortion function of state-of-the-art learned image coding decomposes into five effects: a modeling effect from imperfect entropy models and lossy synthesis, an approximation effect from restricting the form of the encoder mapping, an amortization effect from sharing one network across all images instead of optimizing per image, a digitization effect from quantization and its training surrogates, and an asymptotic effect from coding finite-length images in one shot. The paper quantifies the last three by comparing a continuous stochastic latent system whose rate is computed as a KL divergence (Eq. 10) with a deterministic quantized system whose rate is an entropy, and by turning on per-sample optimization. The estimated rate-distortion function achieves 35% rate savings over VTM on Kodak, extending a previously reported 30% bound, while the empirical deterministic system achieves up to 24% rate savings; under MS-SSIM the corresponding savings are 36% and 22%. This is presented as a new benchmark that reveals substantial untapped potential in lossy image coding.","pith_inferences":["The five-effect decomposition should transfer to neural video, audio, and other latent-based source codecs; repeating the estimated-versus-empirical comparison on video would show whether learned video codecs have headroom comparable to the 35% reported here.","A practical stochastic coder that communicates continuous Gaussian latents with relative entropy coding would test whether the estimated bound is reachable in one shot; if its rate approaches the estimated curve, the digitization and asymptotic effects are removable in principle rather than fundamental.","The widening estimated-empirical gap with model size hints at a digitization wall: as transforms and entropy models improve, the quantization surrogate may become the binding constraint, so better surrogates could unlock further gains without larger networks."],"forward_implications":["If the estimated upper bound is tight, the 35% rate saving over VTM on Kodak means conventional codecs operate well above Shannon's limit, so a codec that realizes the bound would need roughly one-third fewer bits at the same PSNR.","Per-sample optimization consistently improves both the estimated and empirical rate-distortion curves, with gains shrinking as model size grows; amortization is therefore a real but declining overhead that larger transforms gradually absorb.","The residual gap between the estimated and empirical curves, attributed to digitization, widens with model complexity in the Res-hyper family and shrinks when a stronger context entropy model is added, so the digitization effect is architecture-dependent rather than a fixed tax.","Because the asymptotic effect is negligible for deterministic systems that entropy-code discrete latents, practical learned codecs are not meaningfully limited by finite-blocklength effects; the room for improvement lies in modeling, approximation, amortization, and digitization."],"supporting_citations":[{"why":"Defines the rate-distortion function as the theoretical limit against which the whole gap analysis is measured.","marker":"[1]"},{"why":"Links the Lagrangian-relaxed rate-distortion objective to variational autoencoder training, grounding the continuous stochastic formulation used for estimation.","marker":"[2]"},{"why":"Supplies the upper-bound algorithm for estimating the rate-distortion function that the experiments use.","marker":"[4]"},{"why":"Reports the previous 30% rate-savings upper bound on Kodak that this paper extends to 35%.","marker":"[5]"},{"why":"State-of-the-art learned image compression model compared in Fig. 5 and shown to sit below the estimated bound.","marker":"[30]"},{"why":"State-of-the-art frequency-aware transformer codec compared in Fig. 5, serving as a baseline for the empirical gap.","marker":"[31]"},{"why":"Supplies the mixed quantization surrogate used in training deterministic systems, a key practical source of the digitization effect.","marker":"[38]"},{"why":"Provides per-sample optimization with Stochastic Gumbel Annealing, used to quantify and reduce the amortization and digitization effects.","marker":"[50]"},{"why":"Informs the quantization-surrogate training choices and the analysis of train-test mismatch behind the digitization effect.","marker":"[51]"},{"why":"Provides the one-shot achievability bound that defines the asymptotic effect.","marker":"[55]"}],"fun_headline_variants":["Five effects split the codec gap; three now quantified","New benchmark: 35% rate cut over VTM on Kodak","Why learned codecs fall short: five effects, three measured","Codec gap decomposed: five effects, three quantified"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The quantitative estimates of the digitization effect assume the difference between the smooth stochastic model and the quantized practical model is mainly the act of quantization, even though the two systems also differ in architecture, noise model, and training objective.","fun_headline_variants_meta":{"raw":{"variants":["Five effects split the codec gap; three now quantified","New benchmark: 35% rate cut over VTM on Kodak","Why learned codecs fall short: five effects, three measured","Codec gap decomposed: five effects, three quantified"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000672,"raw_usage":{"total_tokens":3056,"prompt_tokens":938,"completion_tokens":2118,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":554,"completion_tokens_details":{"reasoning_tokens":2048}},"tokens_in":554,"tokens_out":2118,"duration_ms":15340,"temperature":1.0,"reasoning_tokens":2048,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:16:10.690638+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Hold the network architecture, entropy model, and loss fixed and train one version with continuous Gaussian latents and the Eq. (10) rate plus another with quantization and the Eq. (6) entropy rate; if the BD-rate gap between the two curves is much smaller than the residual gap reported in Fig. 4, the magnitudes attributed to digitization are not supported.","supporting_citations":[{"cited_title":"Learned image compression with mixed Transformer-CNN architectures,","cited_arxiv_id":null,"evidence_quote":"State-of-the-art learned image compression model compared in Fig. 5 and shown to sit below the estimated bound."},{"cited_title":"Frequency-aware transformer for learned image compression,","cited_arxiv_id":null,"evidence_quote":"State-of-the-art frequency-aware transformer codec compared in Fig. 5, serving as a baseline for the empirical gap."},{"cited_title":"Channel-wise autoregressive entropy models for learned image compression,","cited_arxiv_id":null,"evidence_quote":"Supplies the mixed quantization surrogate used in training deterministic systems, a key practical source of the digitization effect."},{"cited_title":"Improving inference for neural image compression,","cited_arxiv_id":null,"evidence_quote":"Provides per-sample optimization with Stochastic Gumbel Annealing, used to quantify and reduce the amortization and digitization effects."},{"cited_title":"Strong functional representation lemma and applications to coding theorems,","cited_arxiv_id":null,"evidence_quote":"Provides the one-shot achievability bound that defines the asymptotic effect."}],"review_version":1}