{"id":"c2470b26-d7bd-4874-b751-539e70a8b4f4","arxiv_id":"2502.01189","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":8.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Using fixed codebooks of noise vectors in diffusion sampling yields images that carry their own compressed bit-streams and enables a strong perceptual image codec.","lead":"Researchers replace the random noise used inside image-generation diffusion models with fixed sets of pre-sampled noise vectors, so each generated image is fully described by a short list of indices. The same mechanism turns pretrained diffusion models into low-bit-rate image compressors, with the authors reporting state-of-the-art perceptual quality at low bit rates plus built-in compressed restoration.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Greedy per-step index selection (Eq. 7) has no finite-K guarantee; a one-step look-ahead ablation would test whether the SOTA compression claims depend on this unproven compounding assumption.","rationale":"The reader's weakest_assumption identifies exactly the gap I consider most load-bearing: the codec's success depends on the greedy per-step selection rule, and the only theoretical support (Prop. 6.1) is an asymptotic K→∞ result for a closely related loss, not the finite-K greedy rule used in practice. My proposed one-step look-ahead test would directly measure whether greedy selection is near-optimal; if it is, the concern is resolved, and if not, the compression claims would need to be qualified. The paper's own Discussion section admits the lack of theoretical explanation, which supports treating this as a genuine limitation rather than a manufactured objection. The empirical evidence is extensive and the method is well described, so I do not see grounds to reject or mark unverdictable; CONDITIONAL remains the appropriate verdict pending the proposed check.","tokens_in":31550,"tokens_out":13732,"duration_ms":145785,"concrete_test":"Run the compression scheme on a subset of Kodak24 and ImageNet256 with a one-step look-ahead selection rule: at each timestep i, for each of the K candidate noises, compute the denoiser output \\hat{x}_{0|i-1} for the resulting x_{i-1}, and select the index minimizing ||x0 - \\hat{x}_{0|i-1}|| (or minimizing LPIPS). Compare the rate-distortion curves at matched BPP to the greedy rule of Eq. (7). If look-ahead yields substantially better PSNR/LPIPS, the greedy compounding assumption is load-bearing and the current empirical claims are not robust; if the curves coincide, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central codec claim rests on the greedy selection rule in Eq. (7): at each reverse step i, the codebook vector maximizing the inner product with the residual x0 - \\hat{x}_{0|i} is transmitted. This is a locally optimal choice, and the paper offers no analysis of whether these choices keep the whole trajectory close to x0 over T=1000 steps. Proposition 6.1 justifies a related loss (Eq. 9) only in the K→∞ limit, where the codebook densely covers R^n; in the actual operating regime (e.g., K=16, latent dimension ~10^4), the selected vector's cosine similarity with the residual is of order sqrt(2 log K / n) ≈ 0.02, so the per-step guidance signal is weak and the K→∞ theory does not bound the accumulated error. The paper acknowledges in Sec. 7 that it does not explain why these selection rules work. Thus the rate-distortion-perception numbers in Fig. 5 and Tables 2–5 are supported only by empirical evidence, not by the theoretical framework. If the greedy choices compound unfavorably, the reconstructed image could drift from the target and the SOTA claims would not transfer to other models or datasets.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Denoising Diffusion Codebook Models (DDCM), which replace the continuous Gaussian noise draws in DDPM reverse sampling with entries from fixed, pre-sampled codebooks of iid Gaussian vectors (Eq. (6)). The authors show empirically that DDCM retains DDPM-level sample quality in both pixel and latent spaces even at codebook size K=64. They then turn DDCM into a lossy image codec by selecting, at each reverse step, the codebook entry with the largest inner product with the residual between the target image and the current denoised estimate (Eq. (7)). They report state-of-the-art perceptual compression at low bit rates across several datasets and compare favourably to many baselines. The framework is extended to compressed conditional generation, including image restoration and face restoration, with an accompanying asymptotic interpretation (Prop. 6.1) and several guidance variants. The paper includes code and a demo.","tokens_in":31830,"tokens_out":10100,"duration_ms":100787,"significance":"If the empirical claims hold, the paper makes a useful and surprising contribution: the continuous noise space of DDPMs is highly redundant, and a discrete codebook representation can simultaneously serve as a generative latent space and as a zero-shot lossy codec. The codec is practical, requires no training for the pre-trained diffusion model, and the index-stream representation is directly interpretable. The paper also provides a promising framework for compressed conditional generation. The main weaknesses are that the theoretical proposition is asymptotic and does not cover the finite-K greedy selection used in practice, and that the state-of-the-art claim is unqualified despite documented underperformance at high bit rates. The empirical results, if accompanied by the requested ablations and qualifications, would be a solid contribution.","major_comments":[{"comment":"The proof of Proposition 6.1 derives the update xi−1 = xi − fi(xi) + g_i^2 ∇ log p_i(x_i|y) (Eq. (28)) and calls it a discretization of the probability flow ODE (Eq. (18)). However, the probability flow ODE in Eq. (18) has score coefficient g^2/2, not g^2. The derived update is a deterministic discretization of the reverse-time SDE (Eq. (17)) with the stochastic term replaced by its conditional mean, not a discretization of the probability flow ODE. As written, the proposition is not established; please correct the statement or adjust the coefficient, or rephrase it as an approximation of the reverse-time SDE with zeroed noise.","section":"Sec. 6, Prop. 6.1; App. C.1.2"},{"comment":"The compression codec's success rests on the greedy per-step selection rule (7), but the paper provides no finite-K analysis of this rule. Proposition 6.1 covers the related loss (9) only in the K→∞ limit, and Sec. 7 explicitly disclaims a theoretical explanation for the selection strategies. In the operating regime (e.g., K=16, latent dimension n≈10^4), the expected cosine similarity between a selected codebook vector and the residual is about sqrt(2 log K / n) ≈ 0.02, so the per-step signal is weak, and errors can compound over T=1000 steps. I recommend adding an ablation that compares Eq. (7) with a one-step look-ahead or short-horizon selection on at least one dataset; this would indicate whether the reported rate-distortion-perception numbers depend on the unproven compounding assumption.","section":"Sec. 5, Eq. (7); Sec. 7"},{"comment":"The abstract's claim of 'state-of-the-art perceptual image compression results' is unqualified, but the paper's own results in Fig. 5 and Tables 2–5 show that at the highest bit rates DDCM underperforms several baselines (for example, on Kodak24, PerCo (SD) and CRDR-R achieve lower FID and higher PSNR than Ours at approximately 0.13–0.15 BPP in Table 3). Please qualify the claim to the low-to-mid bit-rate regime in the abstract and in the conclusions.","section":"Abstract; Sec. 5"}],"minor_comments":[{"comment":"The text describes the matching-pursuit refinement as a 'convex combination' of codebook entries, but after the renormalization in Eq. (15) the transmitted coefficients no longer form a convex combination; please rephrase or justify the terminology.","section":"App. B.5, Eqs. (13)–(15)"},{"comment":"The bit-stream length formula (T−1)(log2(K)M + C(M−1)) treats C as a number of bits per coefficient, while App. B.5 defines C as the number of quantization levels; the formula should involve ceil(log2 C), or the text should clearly state that C is the number of bits per coefficient.","section":"Sec. 5, bit-rate formula"},{"comment":"The paper reports all metrics as single point estimates without stating whether they come from one evaluation or are averaged over multiple runs or seeds; for FID computed over random subsets, please provide the number of subsets or confidence intervals, or explicitly note that the codebooks are fixed and the evaluations are deterministic.","section":"Tables 2–7"},{"comment":"In the derivation of the equivalence between Eq. (7) and the L_P loss, the factor σ_i√(ᾱ_i)/(1−ᾱ_i) appears in Eq. (37) but is dropped inconsistently before reaching Eq. (44); please verify the algebra or define the scaling convention explicitly.","section":"App. C.2, Eq. (37)"}],"recommendation":"major_revision","confidential_remarks":"The paper is a strong empirical contribution with code and broad experiments, but the theoretical proposition has a concrete coefficient error and the finite-K behavior of the greedy selection is unaddressed. The abstract overstates the results by omitting the high-bit-rate underperformance. These issues are fixable with a revised proposition statement, a qualification of the SOTA claim, and an ablation study, so major revision seems appropriate rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the takeaway: DDCM is the rare paper where the headline mechanism is genuinely new and the empirical work mostly supports it. The idea is to replace the continuous Gaussian noise drawn during DDPM reverse sampling with vectors sampled once from fixed codebooks. The chosen indices then are a lossless bitstream for any generated image, and the same machinery, with greedy index selection, becomes a zero-shot compression codec for real images using any pretrained DDM. That is a real advance, and the paper deserves credit for the simple framing and for the breadth of validation (Kodak, DIV2K, CLIC, ImageNet, several baselines, plus restoration and guidance applications).\n\nThe strongest evidence: DDCM with K=64 matches DDPM FID in both pixel and latent space; at low bit rates the codec reports better perceptual quality (FID) and better or comparable distortion than perceptually-oriented baselines. The matching-pursuit extension for higher rates and the adapted timestep scheme are thoughtful engineering. The paper is also honest in Sec. 7 that it does not explain why these selection rules work.\n\nThe soft spots are real but not fatal. Proposition 6.1 is an asymptotic statement: it shows that if K→∞ the conditional generation scheme approaches a probability-flow ODE. It says nothing about the finite-K greedy rule in Eq. (7), which is the rule that actually produces the SOTA compression numbers. The stress-test note is right that the per-step guidance signal is weak in high dimension when K is small, and nothing in the paper bounds the accumulation of greedy errors over 1000 steps. The paper's own Discussion concedes this. Also, there are no error bars anywhere, and the abstract's \"state-of-the-art\" needs qualification because the body shows underperformance at the highest bit rates.\n\nI would not block a decision on the missing finite-K theory; the empirical claims are extensive and credible, and the mechanism itself is worth the field's attention. But the authors should be asked to tone down the abstract, add variance estimates, and either extend the theory beyond K→∞ or explicitly separate the theoretical interpretation from the empirical codec claims. This paper should get serious peer review, not a desk rejection.","headline":"A simple, clever mechanism—replacing Gaussian noise with fixed codebook vectors—that turns pretrained diffusion models into perceptual codecs; empirically strong, theoretically thin.","tokens_in":32320,"tokens_out":2455,"would_cite":true,"duration_ms":26841,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that diffusion sampling can be driven by tiny fixed codebooks of Gaussian vectors, preserving generation quality and giving every image a lossless bit-stream that also makes a state-of-the-art low-rate perceptual codec.","keywords":["denoising diffusion probabilistic models","codebook sampling","lossless image bit-streams","perceptual image compression","compressed conditional generation","image restoration","posterior sampling","matching pursuit"],"falsifier":"A controlled ablation with the same fixed codebooks could, at a randomly chosen small fraction of steps, replace the greedy rule with the worst-matching codebook entry; if the decoded image's PSNR and FID barely move, the selection rule is not load-bearing, whereas a large drop would confirm that it is.","tokens_in":31377,"feed_emoji":"🖼️","tokens_out":10331,"duration_ms":100853,"temperature":0.7,"pith_summary":"This paper tries to establish that the continuous Gaussian noise used in denoising diffusion sampling is largely redundant: replacing each draw with one of $K$ fixed codebook vectors, chosen at random, preserves sample quality and diversity at $K=64$. The same machinery becomes a lossy image codec when the codebook vectors are chosen greedily to align with the residual between a target image and the model's current estimate. The paper claims this codec achieves state-of-the-art perceptual quality at low bit rates while also producing a lossless index representation of every generated image. It further claims a general framework for compressed conditional generation, with Proposition 6.1 connecting the selection rule to posterior sampling in the $K\\to\\infty$ limit.","feed_headline":"Fixed noise codebooks keep diffusion quality, add bit-streams","feed_subtitle":"The same codebook samples compress real images at ~0.02 bits per pixel, beating prior perceptual codecs.","key_machinery":"The load-bearing object is a chain of fixed codebooks: for each reverse step $i$ there is a set $C_i$ of $K$ vectors sampled once from $\\mathcal{N}(0,I)$ and never updated. Sampling uses $x_{i-1}=\\mu_i(x_i)+\\sigma_i C_i(k_i)$, so the only stochasticity is the index sequence; compression and conditional generation reduce to choosing $k_i$ by Eq. (7) or Eq. (8). The matching-pursuit variant refines each chosen noise as a convex combination of $M$ codebook entries to raise the bit rate, and Proposition 6.1 identifies the $K\\to\\infty$ limit of the posterior-matching loss with a probability-flow ODE, giving the selection rules a score-based interpretation.","core_discovery":"DDCM replaces the standard Gaussian noise $z_i$ in the DDPM update with $C_i(k_i)$, a vector from a fixed codebook of $K$ i.i.d. Gaussian vectors per timestep, and uses the index sequence as the compressed representation. The paper shows empirically that this discrete process matches DDPM's FID at $K=64$ on ImageNet 256 and on MS-COCO with Stable Diffusion 2.1, and that as a codec it selects $k_i=\\arg\\max_{k}\\langle C_i(k), x_0-\\hat{x}_{0|i}\\rangle$, achieving lower FID and LPIPS than prior methods at roughly $0.03$--$0.15$ bits per pixel. For conditional generation it chooses indices by minimizing a task loss; with the loss $\\|C_i(k)-\\sigma_i\\nabla_{x_i}\\log p_i(y|x_i)\\|^2$, the paper proves that as $K\\to\\infty$ the process becomes a discretization of the probability-flow ODE over the posterior $p_0(x_0|y)$. Image compression is a special case with $y=x_0$, so the same codebook sampling produces both a sample and its bit-stream.","pith_inferences":["The paper notes that DDCM indices form an image tokenization; a concrete next step it does not take is training a sequence model to predict those indices from text, which would make a pre-trained diffusion backbone text-conditional without fine-tuning it.","Because the index rule is gradient-free and can optimize any per-image criterion, the same codec could be applied to non-differentiable quality metrics beyond NIQE and TOPIQ, and to video by coding residual index differences between frames.","Entropy coding the index stream, learned dictionary codebooks, or shared codebooks across timesteps would cut bit rate without changing the core mechanism; the paper lists these as future work but does not quantify them.","The $K\\to\\infty$ equivalence suggests a scaling study: measuring how FID and codec distortion vary as $K$ increases would quantify how redundant the continuous Gaussian noise space really is; the paper does not run that analysis."],"forward_implications":["Any pre-trained DDPM can be converted into a codec or conditional sampler without retraining, because only the sampling-time noise source and the index rule change.","Every image produced by DDCM carries a lossless index bit-stream of $(T-1)\\log_2(K)$ bits, so repeated decoding with the same indices reproduces it exactly.","At low bit rates the codec scores lower FID and LPIPS than prior perceptual methods on Kodak, DIV2K, CLIC2020, and ImageNet while maintaining competitive PSNR.","For inverse problems, restored images are generated together with compressed bit-streams, and the paper reports better perceptual quality than DPS and DDNM in colorization and $4\\times$ super-resolution.","With the posterior-matching loss, the $K\\to\\infty$ limit of the selection rule is a probability-flow ODE over the posterior, placing compression and conditional generation under one theoretical umbrella."],"supporting_citations":[{"why":"It defines the DDPM forward and reverse process and the denoiser that DDCM modifies by replacing Gaussian draws with codebook entries.","marker":"Ho et al. (2020)"},{"why":"It supplies the score-based SDE and probability-flow ODE formalism used to prove Proposition 6.1.","marker":"Song et al. (2020)"},{"why":"It provides the pre-trained ImageNet diffusion model and classifier-guidance recipe used for the sampling and compression experiments.","marker":"Dhariwal & Nichol (2021)"},{"why":"It provides the Stable Diffusion latent text-to-image model used for the latent-space evaluations.","marker":"Rombach et al. (2022)"},{"why":"It defines the main zero-shot diffusion-compression baseline, PSC, evaluated against on the same pre-trained model.","marker":"Elata et al. (2024)"},{"why":"It supplies the matching-pursuit algorithm used to refine selected noises into convex combinations and raise the bit rate.","marker":"Mallat & Zhang (1993)"},{"why":"It defines the rate-distortion-perception tradeoff that frames the compression and restoration comparisons.","marker":"Blau & Michaeli (2019)"},{"why":"It supplies the DPS posterior-sampling baseline compared for colorization and super-resolution.","marker":"Chung et al. (2023)"},{"why":"It supplies prior diffusion-based compression work that motivates the zero-shot codec setup.","marker":"Theis et al. (2022)"}],"fun_headline_variants":["Diffusion codebooks: quality preserved, bit-streams added","DDCM: one model for image generation and lossy compression","Codebook diffusion: generate images and bit-streams together","Noise codebooks: diffusion quality with built-in compression"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The greedy per-step index rule is assumed to keep the whole trajectory close to the target, while the only proven statement is a $K\\to\\infty$ limit for a related loss; if locally optimal choices compound into large drift, the codec and restoration claims weaken.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion codebooks: quality preserved, bit-streams added","DDCM: one model for image generation and lossy compression","Codebook diffusion: generate images and bit-streams together","Noise codebooks: diffusion quality with built-in compression"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00036,"raw_usage":{"total_tokens":1970,"prompt_tokens":990,"completion_tokens":980,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":606,"completion_tokens_details":{"reasoning_tokens":911}},"tokens_in":606,"tokens_out":980,"duration_ms":8726,"temperature":1.0,"reasoning_tokens":911,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T16:17:44.181138+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled ablation with the same fixed codebooks could, at a randomly chosen small fraction of steps, replace the greedy rule with the worst-matching codebook entry; if the decoded image's PSNR and FID barely move, the selection rule is not load-bearing, whereas a large drop would confirm that it is.","supporting_citations":[],"review_version":1}