{"id":"f8057eca-69d8-4b6d-a159-e679071161ea","arxiv_id":"2412.10935","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A uniform-noise diffusion model plus universal quantization enables a single-model progressive image codec that is computationally tractable and competitive with classic and neural baselines.","lead":"A new type of diffusion model, UQDM, replaces Gaussian noise with uniform noise so that images can be compressed progressively with efficient universal quantization instead of slow relative-entropy coding. The single model yields lossy reconstructions at multiple bitrates and lossless compression, and it beats classic codecs like JPEG and JPEG2000 in the reported tests.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The NELBO-to-bitrate identity is internally consistent, but the practical claim depends on the learned reverse variance at T≈4; Appendix B.2's own per-step overhead is never decomposed in the reported results, so the competitive bitrates are not yet explained.","rationale":"I checked the chain that makes NELBO equal to coding cost. UQ's subtractive dither theorem and the identity E_y KL(u(·|y)||fθ) = I + KL(p_Ytilde||fθ) make Eq. (2) correct; the forward and reverse models are matched so the KL terms are realizable channel-simulation costs; the CLT proof in Appendix A.3 is standard and sound. Thus I do not see a mathematical flaw in the central identity. The vulnerability is where the reader puts it: for the method to be practical at T≈4, the learned variance must make each Lt small. Appendix B.2 itself quantifies a constant per-step overhead in the moment-matched case and says learned variance helps, but the paper never reports the per-step decomposition or the learned variance values. Since Figures 3–4 compare against a hypothetical Gaussian-REC baseline, the competitive claim rests on an unobserved empirical property. This warrants the same CONDITIONAL verdict as the reader: accept the theoretical contribution, require per-step rate diagnostics and code before the practical claim is taken as established. My concern is load-bearing but not fatal; it can be settled by the proposed decomposition check. I therefore recommend no verdict change.","tokens_in":18739,"tokens_out":10753,"duration_ms":78241,"concrete_test":"Re-implement (or use released code) the T=4 UQDM for CIFAR-10 and record per-step Lt for t=1..4 plus L_{x|z0} at the operating point shown in Figure 3, and record the learned variance scaling sθ(z) averaged over coordinates and images. If Σ_t Lt is comparable to or larger than L_{x|z0}, or if sθ(z) ≥ 1 for most coordinates, then the Appendix B.2 per-step overhead is present and the reported bitrate is not explained by progressive refinement alone. Cross-check by computing VDM's NELBO with the same denoiser at T=4; if UQDM's T=4 NELBO exceeds VDM's T=4 NELBO by more than the predicted overhead, the uniform channel is the bottleneck.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central construction is mathematically coherent: with subtractive dither U′ and surrogate density fθ = gθ * U(−Δ/2, Δ/2), the codelength for step t is exactly the KL term in Eq. (1), so the NELBO is an operational bitrate and the codec costs O(T) network evaluations. The load-bearing point is therefore not the identity but the finite-T behavior of the learned reverse variance. Appendix B.2 shows that with the moment-matched choice σ_g^2 = β^2, each diffusion step carries a constant KL overhead of at least about (1/3) log 2 bits/dim; for T=4 on CIFAR-10 this alone would be roughly 4096 bits per image. The paper's learned variance σ_g^2 = sθ(z)β^2 is introduced to escape this overhead, but no per-step Lt decomposition, no average sθ(z), and no ablation of fixed versus learned variance are reported for CIFAR-10/ImageNet64. Without those numbers, one cannot distinguish a bitrate that reflects progressive refinement information from one that is dominated by the uniform-channel mismatch overhead; the hypothetical VDM(T=1000) curve cannot settle this because it is not an implementable codec.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a new class of diffusion models, Universally Quantized Diffusion Models (UQDM), in which the forward process uses uniform noise instead of Gaussian noise. The central claim is that the negative ELBO of this model is exactly the end-to-end lossless compression cost when the codec is implemented with universal quantization, thereby avoiding the exponential runtime of Gaussian channel simulation required by previous diffusion-based progressive codecs. The authors specify the forward and reverse processes, prove a continuous-time convergence of the forward process to the Gaussian case, and present progressive compression experiments on toy, CIFAR-10, and ImageNet64 data, reporting competitive rate-distortion and rate-realism results against JPEG, JPEG2000, BPG, CTC, and CDC.","tokens_in":19050,"tokens_out":16538,"duration_ms":151376,"significance":"If the central identity holds, the paper makes a valuable conceptual and practical contribution: it connects a diffusion model's training loss directly to an operational bitrate and provides a tractable progressive codec with O(T) neural network evaluations per image. The paper includes a clear theoretical derivation of the NELBO-to-bitrate correspondence and a central limit theorem for the forward process, and it reports that the actual file sizes are within 3% of the theoretical NELBO, which is a strong empirical validation of the coding scheme. The main limitations are that the practical demonstration relies on very few diffusion steps (T ≈ 4), the reported rate-distortion curves contain only a handful of points, and the appendix's analysis of the per-step overhead contains a sign error that undermines one of the paper's theoretical motivations.","major_comments":[{"comment":"There is a sign error in the derivation of the per-step rate. The function h(z) is defined as h(z) = -log(G_t(z+μ_t+Δ_t/2) - G_t(z+μ_t-Δ_t/2)), which is nonnegative because the CDF difference lies in [0,1]. The text nevertheless states that h(z) ≤ 0 and writes L_{t-1} ≈ -(1/6)[4h(0)+h(-Δ_t/2)+h(Δ_t/2)] ≥ (1/3) log 2. For nonnegative h, the left-hand expression is nonpositive, so the inequality and the claimed constant per-step overhead do not follow as stated. Please correct the sign and re-derive, or remove the quantitative claim.","section":"Appendix B.2"},{"comment":"The practical claim of competitive bitrates rests on the learned reverse-process variance reducing the per-step overhead that the fixed-variance analysis predicts, but the paper reports no per-step L_t decomposition, no average learned variance s_θ(z), and no ablation of fixed versus learned variance on CIFAR-10 or ImageNet64. Without these numbers, the reader cannot determine whether the bitrates in Figures 3 and 4 reflect progressive refinement information or are dominated by the uniform-channel mismatch overhead. Please add a per-step rate breakdown and a fixed-variance ablation on image data.","section":"Sections 5.2, 5.3 and Appendix B.2"},{"comment":"The entropy-coding step is specified as 'Entropy-code k_t using p(z_{t-1}|z_t)', but p(z_{t-1}|z_t) is a continuous density, whereas k_t is a discrete index whose coding distribution must be the discretization of p(z_{t-1}|z_t) on a grid of width Δ(t) offset by the dither u_t. This offset dependence is described in Section 2 but is not made explicit in the algorithms. Please state the exact coding distribution P(k_t | z_t, u_t) so that the claimed equality between the sum of L_t terms and the actual transmitted bitrate is fully operational.","section":"Algorithms 1 and 2"}],"minor_comments":[{"comment":"The abstract says the method gives results that 'exceed existing progressive codecs', but Section 5.3 states the results are 'comparable to, if not slightly better than, CTC'. Please align the wording with the actual comparison.","section":"Abstract and Section 5.3"},{"comment":"The learned variance is written as σ_θ² = σ_Q²(t) ⊙ s_θ(z_t); since the network is time-indexed, it would be clearer to write s_θ(z_t, t) to make the time dependence explicit.","section":"Section 3.2"},{"comment":"Applying the VDM probability-flow ODE update (eq. 17) to UQDM is justified only by an analogy with the continuous-time limit; this is a heuristic, and the text should state that it is an approximation rather than implying that the discrete-time backward processes are equivalent.","section":"Appendix B.3"},{"comment":"The caption contains an incomplete sentence: 'conceptually this is equivalent to When using fixed reverse-process variances'. Please rewrite.","section":"Figure 6 caption"},{"comment":"The observation that larger T improves sample quality (Figure 7) but worsens compression NELBO (Figure 2) is interesting but potentially confusing; a brief explanation of why per-step overhead accumulates with T would help the reader.","section":"Section 5.1"},{"comment":"The learned-variance model takes about 5 minutes to compress a CIFAR image due to a naive per-coordinate CDF-table loop; reporting the fixed-variance model's speed or using a parallel entropy coder would better support the 'practical deployment' claim.","section":"Appendix C.2"}],"recommendation":"major_revision","confidential_remarks":"The core theoretical identity appears sound and the paper is a good fit for the journal. The main concern is the sign error in Appendix B.2, which is used to motivate the learned-variance design, and the absence of per-step rate ablations on the image datasets that the headline results depend on. I recommend major revision rather than rejection because the issues are identifiable and fixable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth your time. The core construction is sound: replace Gaussian noise with uniform noise in the forward process, define the reverse density as a learned gθ convolved with uniform noise, and the NELBO becomes a real codelength under universal quantization. That is a genuine step past Theis et al.'s Gaussian codec, which needs intractable channel simulation. The paper also proves the continuous-time limit matches VDM, a nice formal result.\n\nWhat's new: UQDM's forward process and the corresponding coding scheme are not in the cited literature. The proofs in the appendix are real, the file-size check (within 3% of NELBO) is concrete, and the ImageNet64 comparison includes CTC, a legitimate progressive codec. Those are points in the paper's favor.\n\nSoft spots: the practical claims depend on the learned reverse variance at T≈4. Appendix B.2 shows that a moment-matched variance costs a constant per-step KL overhead, and the learned scale sθ is meant to avoid that. But the paper never reports a per-step Lt decomposition, the average learned scale, or a fixed-vs-learned variance ablation on CIFAR or ImageNet64. Without those, you can't tell how much of the bitrate is progressive refinement and how much is uniform-channel mismatch. The VDM curves are hypothetical, so they don't prove competitiveness against an actual Gaussian codec. And the 5-minute CPU time per CIFAR image for the learned-variance version is acknowledged but unsolved; 'practical deployment' overstates it. Only 4–5 rate points per model, no error bars, and code is promised for acceptance.\n\nNone of this breaks the central idea. The NELBO-bitrate identity is mathematically clean, and the experimental gaps are the usual first-paper sins. The right move is to send it to review and ask for code, per-step rate analysis, and the missing ablation. If those come back, the empirical claims will be testable.\n\nBottom line: this deserves serious refereeing. The idea will likely have legs in learned compression.","headline":"A sound theoretical core — uniform-noise diffusion turns the NELBO into an actual bitrate — but the practical claims outrun the evidence, which needs per-step rate analysis and code.","tokens_in":19557,"tokens_out":3859,"would_cite":true,"duration_ms":34866,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Replacing Gaussian noise with uniform noise makes a diffusion model's training loss equal its actual progressive-compression bit cost.","keywords":["progressive image compression","diffusion models","universal quantization","uniform noise channel","variational diffusion models","rate-distortion","rate-realism","neural codecs"],"falsifier":"Compress a fixed set of images with UQDM using increasing numbers of diffusion steps and compare the actual bits per dimension against the theoretical limiting rate of Gaussian diffusion. If UQDM's rate does not approach that limit as the step count grows, or if the real file size deviates from the NELBO by more than the reported 3%, the claimed equality between training loss and operational bitrate is not holding in practice.","tokens_in":18517,"feed_emoji":"🖼️","tokens_out":12448,"duration_ms":98605,"temperature":0.7,"pith_summary":"This paper proposes a diffusion-based image codec whose training loss—the negative evidence lower bound (ELBO)—is, by construction, an actual bitrate. Replacing the Gaussian noise in the forward process with uniform noise lets the sender communicate each latent through universal quantization, and the paper shows that the negative ELBO of the resulting model equals the end-to-end compression cost. The payoff is progressive coding: bits can be transmitted incrementally and decoded with progressively improving quality, all with a single model, and compute grows linearly with the number of diffusion steps rather than exponentially as in Gaussian-diffusion codecs. On CIFAR-10 and ImageNet 64×64 the method beats JPEG and JPEG2000 across all bit-rates, matches or exceeds the progressive neural codec CTC, and keeps improving at high bit-rates where other codecs plateau.","feed_headline":"Uniform-noise diffusion turns training loss into real bits","feed_subtitle":"A single model streams lossy-to-lossless image bits and beats JPEG and JPEG2000 at every bit-rate.","key_machinery":"The load-bearing object is the uniform noise channel and its simulator, universal quantization. Given a latent $Y$, the sender perturbs it with a shared dither $U' \\sim U(-\\Delta/2, \\Delta/2)$, quantizes $Y+U'$ on a grid of width $\\Delta$, entropy-codes the index $K$ under the discretized model $p(z_{t-1}|z_t)$, and the receiver reconstructs $K - U'$; Zamir and Feder's identity $H[K|U'] = I(Y; \\tilde Y)$ makes the coding cost exactly the mutual information of the uniform channel when the density model is the true convolution. UQDM sets the reverse process to $g_\\theta(z_{t-1}; z_t, t) \\star U(-\\Delta(t)/2, \\Delta(t)/2)$, with $g_\\theta$ a Gaussian or logistic density centered at the denoising prediction $b(t) z_t + c(t) \\hat{x}_\\theta(z_t, t)$, so the NELBO telescopes into the bits sent at each step. The mechanism converts a diffusion training loss into an operational bitrate, which is what makes progressive decoding with a single model computationally feasible.","core_discovery":"The central discovery is that a diffusion model can be rebuilt around uniform noise channels so that the variational bound it is trained on is not just a bound on likelihood but the exact expected codelength of a practical progressive codec. In UQDM the forward posterior $q(z_{t-1}|z_t,x)$ is a uniform distribution matched in mean and variance to the Gaussian posterior of VDM, and the reverse model is a learned density convolved with uniform noise. Because universal quantization simulates a uniform channel with entropy cost $h(\\tilde Y) - \\log \\Delta$, the per-step KL terms in the NELBO become the bits actually sent, with the initial Gaussian latent shared by a seeded random generator at negligible cost. The authors show that as the number of steps grows the uniform forward process converges in distribution to the Gaussian diffusion process, yet in practice a small number of steps (around 4 or 5) is best, and they find that learning the reverse-process variance substantially improves the rate. They report that the real file size stays within 3% of the theoretical NELBO.","pith_inferences":["Left implicit in the paper but a direct extension: running UQDM on a pretrained autoencoder's latent space would inherit the compute savings of latent diffusion and likely scale to larger images.","The paper's learned-variance implementation spends minutes on a CPU for a 32×32 image because it builds a separate entropy-coding table per dimension; this is an engineering bottleneck that parallel entropy coding should remove.","The demonstrated convergence of the uniform forward process to the Gaussian one as steps grow suggests UQDM could be used as an empirical probe for how much of the Gaussian-diffusion compression bound is lost to the uniform channel's per-step overhead.","The rate-realism results point toward a single-model variable-rate codec for applications that want realistic reconstructions at ultra-low bit-rates, though the paper itself does not claim perfect realism."],"forward_implications":["A single UQDM model covers the full rate-distortion curve: stopping after fewer steps gives low bit-rates and coarser reconstructions, and sending more steps refines them up to lossless.","Encoding and decoding require only a few neural-network evaluations (around 4 steps on images), so the codec avoids relative-entropy-coding primitives whose runtime scales exponentially with the number of bits.","Because the training loss is an actual bitrate, any improvement in the diffusion model's likelihood translates directly into better compression without changing the codec.","Flow-based reconstructions match the distortion of denoised predictions while giving substantially better realism as measured by FID, so the same model can be aimed at either fidelity or realism.","At high bit-rates UQDM keeps improving where BPG, CTC, and other baselines plateau, suggesting that progressive diffusion coding can be practically deployed."],"supporting_citations":[{"why":"Supplies universal quantization: simulating a uniform noise channel at entropy cost $h(\\tilde Y)-\\log\\Delta$, the mechanism that makes UQDM's NELBO an actual bitrate.","marker":"(Zamir & Feder, 1992)"},{"why":"Introduced the idea of progressive compression with a diffusion model's NELBO and denoising predictions.","marker":"(Ho et al., 2020)"},{"why":"The Gaussian-diffusion progressive codec whose exponential channel-simulation cost UQDM bypasses, and the source of flow-based reconstruction.","marker":"(Theis et al., 2022)"},{"why":"Provides the VDM Gaussian forward process, the ELBO framework, and the continuous-time limit that UQDM is shown to approach.","marker":"(Kingma et al., 2021)"},{"why":"Establishes the universally quantized neural-compression setup and the surrogate density model $g_\\theta \\star U(-\\Delta/2,\\Delta/2)$ that UQDM adopts.","marker":"(Agustsson & Theis, 2020)"},{"why":"Documents the exponential runtime of causal rejection samplers for Gaussian channel simulation, the bottleneck UQDM removes.","marker":"(Goc & Flamich, 2024)"},{"why":"Motivates learning the reverse-process variance, which the paper finds is the key to competitive UQDM rates.","marker":"(Nichol & Dhariwal, 2021)"},{"why":"Provides the DDIM-style non-Markovian forward-process factorization and the probability-flow update used for reconstructions.","marker":"(Song et al., 2021a)"},{"why":"CTC is the progressive neural codec baseline that UQDM matches or exceeds on ImageNet64.","marker":"(Jeon et al., 2023)"}],"fun_headline_variants":["Uniform-noise diffusion: training loss becomes file size","Single diffusion codec from lossy to near-lossless","Progressive bits: ELBO equals codelength in this model","One model, every bit-rate, exact compression from ELBO"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method rests on the trained reverse-process variance keeping the per-step coding cost small with only a few diffusion steps; the paper's own analysis finds a constant overhead per step when only moments are matched, and increasing the number of steps makes the NELBO worse, so the competitive results depend on an empirical property of the learned variance rather than on the asymptotic convergence theorem.","fun_headline_variants_meta":{"raw":{"variants":["Uniform-noise diffusion: training loss becomes file size","Single diffusion codec from lossy to near-lossless","Progressive bits: ELBO equals codelength in this model","One model, every bit-rate, exact compression from ELBO"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000198,"raw_usage":{"total_tokens":1351,"prompt_tokens":912,"completion_tokens":439,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":528,"completion_tokens_details":{"reasoning_tokens":371}},"tokens_in":528,"tokens_out":439,"duration_ms":4904,"temperature":1.0,"reasoning_tokens":371,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:28:39.023437+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compress a fixed set of images with UQDM using increasing numbers of diffusion steps and compare the actual bits per dimension against the theoretical limiting rate of Gaussian diffusion. If UQDM's rate does not approach that limit as the step count grows, or if the real file size deviates from the NELBO by more than the reported 3%, the claimed equality between training loss and operational bitrate is not holding in practice.","supporting_citations":[],"review_version":1}