{"id":"d0e13b03-ad48-4eb2-8e45-53e7f952817b","arxiv_id":"2412.05926","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"BiDM achieves the first fully binarized (W1A1) diffusion model, with FID 22.74 on LSUN-Bedrooms, by adding timestep-aware binary structures and patch-based distillation.","lead":"A new training scheme fully binarizes diffusion model weights and activations to 1 bit, reporting FID 22.74 on LSUN-Bedrooms, far better than prior binarization baselines. The work is a step toward running generative models on resource-limited devices.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline '52.7× OPs savings' is unsupported by the paper's own measured runtime: Appendix B shows an 8.07× speedup for BiDM, with the extra TBS operations adding ~4 ms per convolution; the theoretical OPs metric omits these overheads.","rationale":"The paper's most valuable and best-supported result is the generation-quality improvement: FID 22.74 for a W1A1 diffusion model, with ablations showing TBS and SPD each contribute and with a large margin over the strongest baseline (59.44). The FID claim is internally consistent and survives scrutiny. However, the paper makes a second headline claim of 52.7× OPs savings, and this is directly contradicted by its own Appendix B latency data, which show only 8.07× speedup over FP32 on the tested hardware and a 23% slowdown relative to the DoReFa 1-bit baseline. The theoretical OPs metric (Appendix A) excludes the channel-averaging, tiny-convolution, and elementwise-multiply steps that are part of the XNOR-style inference path and that account for ~19% of measured per-convolution latency. Because this efficiency number appears in the abstract and is described as 'excellent', it is load-bearing for the paper's overall claim, even though the quality result stands. The reader's weakest_assumption identified precisely this gap between theoretical and measured efficiency, and the evidence in the manuscript supports that concern. A conditional acceptance that requires clarifying the efficiency reporting (and ideally supplying the omitted patch-size hyperparameter) is the appropriate outcome; I do not see grounds to reject or to fully accept without that clarification.","tokens_in":21482,"tokens_out":15251,"duration_ms":146955,"concrete_test":"Using the released code and the same deployment environment as Appendix B (e.g., Qualcomm Snapdragon 855 Plus with the Larq library), measure end-to-end latency for the full 200-step DDIM sampling of BiDM and of the FP32 LDM-4 on LSUN-Bedrooms, then compute the ratio. If the measured end-to-end speedup is approximately 8× rather than 52.7×, the abstract and §4.3 should be revised to report measured speedup alongside the theoretical OPs savings.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract and §4.3 present 'up to excellent 28.0× storage and 52.7× OPs savings' as a headline benefit of BiDM, but the paper's own runtime benchmark (Appendix B, Table 8) contradicts this. For a single convolution, BiDM/XNOR-Net takes 21,864.8 µs versus 176,371.0 µs for FP32, i.e., an 8.07× speedup, not 52.7×. The theoretical OPs in Table 4 use the Appendix A formula total OPs = 1/64 BOPs + FLOPs, which omits the full-precision operations that Appendix B explicitly lists as steps (3)–(6): channel-summing I_f into A (2,948.8 µs), the tiny convolution with k′ (1,133.3 µs), and two pointwise multiplies (83.2 + 4.3 µs). These steps consume about 19% of BiDM's measured latency. Moreover, the same Table 8 shows the simpler DoReFa 1-bit baseline runs at 17,699.5 µs, 23% faster than BiDM, yet the main text states BiDM 'achieves the same 28.0× memory efficiency and 52.7× computational savings as the XNOR baseline'. The 52.7× figure is thus an asymptotic BOPs argument, not a realistic efficiency claim for the proposed architecture. A reader relying on the abstract would overestimate BiDM's practical speedup by roughly a factor of 6.5. This does not invalidate the FID improvement (22.74 vs. 59.44), but it is a load-bearing weakness in one of the paper's two central claims. The paper should report measured end-to-end speedup alongside theoretical OPs and temper the 'excellent' characterization.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes BiDM, a fully binarized (W1A1) diffusion model method. It introduces Timestep-friendly Binary Structure (TBS), which uses learnable activation scaling and cross-timestep feature connections, and Space Patched Distillation (SPD), which applies attention-guided distillation on spatial patches. Experiments on DDIM (CIFAR-10) and LDM-4/LDM-8 (LSUN-Bedrooms, LSUN-Churches, FFHQ) report large FID improvements over binarization baselines, e.g., 22.74 vs. 59.44 on LSUN-Bedrooms, with claimed 28.0x storage and 52.7x OPs savings. The authors state that BiDM is the first fully binarized diffusion model capable of generating viewable images.","tokens_in":21903,"tokens_out":7749,"duration_ms":78173,"significance":"If the reported results replicate, this is a substantial empirical advance: it would be the first W1A1 binarization of latent diffusion models with non-degenerate generative quality, and the ablation study (Table 3) gives evidence that both TBS and SPD contribute to the improvement. The paper provides code, training settings, and an unusually transparent runtime benchmark in Appendix B. The main weakness is that the headline efficiency claim (52.7x OPs savings) is a theoretical bit-operation count that is not matched by the paper's own measured 8.07x wall-clock speedup, and the cross-timestep connections impose additional overhead not included in that measurement. The central FID claim is plausible and well supported by the ablations, but the efficiency framing needs correction.","major_comments":[{"comment":"The abstract and §4.3 present '28.0x storage and 52.7x OPs savings' as headline benefits without clearly labeling them as theoretical bit-operation counts. Appendix B, Table 8, however, reports a measured per-convolution runtime of 176,371.0 us for FP32 vs. 21,864.8 us for XNOR-Net/BiDM, an 8.07x speedup, and the simpler DoReFa baseline runs at 17,699.5 us, about 23% faster than BiDM. The 52.7x figure is obtained from the formula OPs = BOPs/64 + FLOPs, which omits the full-precision steps (3)-(6) of the XNOR scaling that Table 8 shows consume roughly 19% of BiDM's latency, and it also does not include the cross-timestep blending operations of TBS. Please report measured end-to-end or per-layer speedup alongside the theoretical OPs, and revise the abstract and §4.3 so that the reader is not led to expect a 52.7x practical speedup.","section":"Abstract, §4.3, Table 4, Appendix B"},{"comment":"The patch size p is never specified. Equation (13) divides features into p^2 patches and Eq. (15) sums the patch-wise losses, but Appendix A gives learning rates, lambda, alpha_init, and the number of TBS connections without ever stating the value of p used for SPD. Since SPD is one of the two central contributions, omitting p makes the method irreproducible from the text. Please state the exact p used for each model and dataset, and clarify whether p is tuned.","section":"§3.3, Eq. (13)-(15), Appendix A"},{"comment":"Equation (12) is ambiguous about which features are used in the cross-timestep connection. It writes Concat(D^{t-1}_m(·), (1-alpha^{t-1}_{m+1}) U^{t-1}_{m+1}(·) + alpha^{t-1}_{m+1} U^t_{m+1}(·)), i.e., the first argument is the previous timestep's encoder feature, whereas the standard U-Net skip connection in Eq. (5) uses D^t_m. This could mean that the method replaces the current encoder feature with the previous one, or it could be a typo for D^t_m. Please clarify the exact data flow, including how the previous-step features are cached and what happens at the first sampling step.","section":"§3.2, Eq. (12)"}],"minor_comments":[{"comment":"The heading 'Space Patched Distilation' contains a typo; it should be 'Distillation'.","section":"§3.3 heading"},{"comment":"The table column header 'FID↓/convolution' mixes an image-quality metric with a runtime measurement; the table reports runtime only, so the header should be reworded.","section":"Appendix B, Table 8"},{"comment":"Reference [4], 'Binarized diffusion model for image super-resolution', appears in the bibliography but is never cited in the body. If that prior work already performs full binarization of a diffusion model, the 'first fully binarized' claim in the abstract needs qualification.","section":"References"},{"comment":"The paper states in the NeurIPS checklist that random seeds are fixed, but no error bars or multiple-seed results are reported for the main FID/IS numbers in Tables 1-3. Please state explicitly whether the reported numbers are single runs and, if feasible, add variance information for the main comparisons.","section":"§4 and Appendix A"},{"comment":"Please specify the optimizer (e.g., Adam), weight decay, and any learning-rate schedule used for QAT; the current text gives learning rate and batch size but not the optimizer.","section":"Appendix A"},{"comment":"The terms LDM-4 and LDM-8 are used without definition in the main text; please state that they refer to the latent-space downsampling factor of the LDM backbone.","section":"§4, LDM-4/LDM-8"},{"comment":"The norm in Eq. (15) is written as ||.||_2 but is applied to matrices; please state explicitly that this is the Frobenius norm.","section":"Eq. (15)"},{"comment":"The table is labeled 'Tabel 9'; please correct the spelling.","section":"Appendix B, Table 9"}],"recommendation":"major_revision","confidential_remarks":"The core FID result appears credible and the ablations are well structured. The main concern is that the efficiency claim, which is half of the paper's headline, is not supported by the paper's own measured runtime; this is fixable but requires changing the abstract and §4.3. Please also ask the authors to verify the priority claim against reference [4], which is listed but never cited. The missing patch size p is a simple but important reproducibility gap."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe one thing to know: this is the first paper that actually binarizes both weights and activations of a diffusion model to 1 bit, and the FID improvement is real. On LSUN-Bedrooms they get 22.74 FID versus 59.44 for the best general binarization baseline. That is a legitimate step forward for the subfield.\n\nWhat's genuinely new: the TBS structure (learnable activation scaling plus cross-timestep connections) and SPD (patch-level attention distillation). The ablations show both components contribute, with TBS taking FID from 106.62 to 35.23 and SPD to 40.62, and the combination to 22.74. The paper also reports its own runtime measurements in Appendix B, which is more than many system papers do.\n\nThe soft spots are in the efficiency claim and experimental rigor. The headline '52.7x OPs savings' is not supported by the paper's own numbers. The measured runtime on a Snapdragon 855 Plus shows 8.07x speedup over FP32 for a single convolution, and BiDM is actually 23% slower than the simpler DoReFa baseline because of the extra full-precision steps (channel averaging, tiny convolution, pointwise multiplies). The theoretical OPs figure counts bitwise operations as 1/64 of a FLOP and ignores those overheads. A reader relying on the abstract would overestimate the real speedup by a factor of about six. The storage savings (28x) are fine, but the computational savings should be reported as measured, not asymptotic.\n\nThere is also a missing hyperparameter: the patch size p for SPD is never specified. No error bars or multiple seeds are reported; the checklist mentions fixing random seeds, but for a result this far from the full-precision reference, one run is thin. Training budgets for the baselines aren't fully standardized either, though the paper does include training-time experiments that partly address this.\n\nThe central claim—that full binarization of DMs is possible with acceptable generation quality—holds up. The efficiency framing needs a serious rewrite. This paper deserves peer review, but a reviewer should insist on measured end-to-end speedups, the patch size, and variance estimates before the numbers go into the record.\n\nNet: worth engaging with. It is the first W1A1 diffusion work and the FID result is a real data point, but the efficiency story should be read with a strong caveat.","headline":"First real W1A1 diffusion model with a genuine FID win, but the headline 52.7x OPs savings does not survive the paper's own runtime measurements.","tokens_in":22466,"tokens_out":2174,"would_cite":true,"duration_ms":21285,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Diffusion models can be pushed all the way to 1-bit weights and activations without collapsing, provided the binary structure tracks timestep-varying activation ranges and training is guided by patch-level attention distillation.","keywords":["diffusion models","binarization","1-bit quantization","weights and activations","model compression","efficient inference","knowledge distillation","latent diffusion"],"falsifier":"Run the released BiDM model end-to-end on the same GPU or edge device, measure total sampling latency against the full-precision model, and compare the ratio with the claimed 52.7x OPs savings; the paper's own appendix already reports an 8.07x per-convolution speedup on a Snapdragon 855 Plus, so an end-to-end ratio near 8x rather than 52.7x would falsify the practical-efficiency claim.","tokens_in":21292,"feed_emoji":"🖼️","tokens_out":14237,"duration_ms":116419,"temperature":0.7,"pith_summary":"Diffusion models generate high-quality images but need large memory and compute, and standard quantization usually stops at 4 bits or more because 1-bit binarization makes generation degrade or collapse. This paper tries to establish that full 1-bit binarization of diffusion models is viable if the binary structure is timestep-aware and training is guided patch-wise by a full-precision teacher. On a latent diffusion model, it reports a W1A1 model that generates recognizable LSUN-Bedrooms images with FID 22.74 (lower is better), versus 59.44 for the strongest prior binary baseline, while claiming 28.0x storage and 52.7x theoretical operation savings. If the claims hold, fully binarized diffusion models become a practical compression target for low-resource deployment rather than a degenerate extreme.","feed_headline":"Diffusion models survive full 1-bit binarization","feed_subtitle":"BiDM reaches FID 22.74 on bedrooms, beating prior binary baselines while cutting storage 28x.","key_machinery":"The load-bearing mechanism is the pair (TBS, SPD). TBS modifies the classic binary-convolution approximation $I * W \\approx (\\mathrm{sign}(I) \\otimes \\mathrm{sign}(W)) \\odot (A * k\\alpha)$: instead of the fixed averaging filter $k$, it learns a tiny depth-1 convolution $k$, so the binary activation quantizer can match timestep-dependent activation ranges; it also adds a cross-timestep feature connection $\\mathrm{Concat}(D^{t-1}_m, (1-\\alpha^{t-1}_{m+1})U^{t-1}_{m+1} + \\alpha^{t-1}_{m+1}U^{t}_{m+1})$ with learnable scalars, letting the current step borrow information from the similar previous step. SPD partitions intermediate features into $p^2$ patches, computes normalized spatial attention maps $P_{i,j}P_{i,j}^{\\mathsf{T}}$ for binary and full-precision models, and sums patch-wise L2 losses into the training objective $L = L_{\\mathrm{DM}} + \\frac{\\lambda}{2d+1}\\sum_m L^m_{\\mathrm{SPD}}$. Together, these components restore activation-range fidelity and give the discrete model a spatially local optimization target.","core_discovery":"The central claim is that a diffusion model can be fully binarized—both weights and activations reduced to 1 bit—without collapsing, provided the quantization respects two properties of diffusion models: activation ranges shift dramatically across long denoising horizons yet stay similar between adjacent timesteps, and image-generation features are spatially local. BiDM implements this with a Timestep-friendly Binary Structure (TBS), which makes the scaling factor of the binary convolution learnable through a tiny convolution so activation ranges can adapt per timestep, and connects each timestep's features to the previous timestep's through a learnable blending weight; it also adds Space Patched Distillation (SPD), which splits intermediate features into patches and enforces normalized attention-map agreement with a full-precision teacher patch by patch. On LDM-4 for LSUN-Bedrooms 256x256, the W1A1 BiDM achieves FID 22.74, down from 106.62 for the XNOR-style baseline and 59.44 for the strongest prior W1A1 baseline, while keeping the 28.0x storage and 52.7x OPs savings of the binary baseline.","pith_inferences":["One testable extension: applying TBS's learnable timestep-adaptive activation scaling to other time-varying quantized models, such as video diffusion or autoregressive models with recurrent states, would show whether the temporal adaptivity transfers beyond image diffusion.","The paper's own training-efficiency table indicates that cross-timestep connections roughly double the number of sampling steps needed during training, so part of BiDM's accuracy gain is bought with extra training compute; equal-wall-clock-time comparisons may be the fairer benchmark.","If the extra TBS operations (channel averaging, tiny convolution, pointwise scaling) were fused into the binary kernels or precomputed offline, the gap between the theoretical 52.7x OPs saving and the measured 8.07x runtime speedup could narrow; this is an engineering path the paper notes but does not demonstrate.","SPD's per-patch attention loss could plausibly be replaced by other local patch statistics, such as per-patch Gram matrices, and still help; comparing variants on the same binary U-Net would isolate what part of the patch mechanism matters."],"forward_implications":["Fully binarized latent diffusion models can generate viewable images: on LSUN-Bedrooms 256x256, BiDM reaches FID 22.74, a 2.6x improvement over the strongest prior W1A1 baseline at FID 59.44.","Pixel-space diffusion models also survive full binarization: on CIFAR-10, BiDM raises the Inception Score from 4.23 for the XNOR-style baseline to 5.18 and reduces FID by roughly 28% relative to that baseline.","The model inherits the binary baseline's 28.0x storage reduction and 52.7x theoretical operation savings, with only minor added floating-point work for the scaling and cross-timestep connections, according to the paper's efficiency analysis.","The two design ideas—timestep-adaptive activation scaling and patch-level attention distillation—are presented as general remedies for the temporal and spatial properties of diffusion models, not as fixes tied to a single architecture.","BiDM is, per the paper, the only fully binarized diffusion model whose generated samples are visually acceptable, which would open 1-bit deployment of diffusion models in low-resource settings."],"supporting_citations":[{"why":"Supplies the binary-convolution approximation that TBS modifies by making the scaling convolution learnable, and serves as the vanilla W1A1 baseline.","marker":"[49]"},{"why":"XNOR++ introduces trainable activation scale factors, the approach BiDM contrasts with its timestep-adaptive learnable tiny convolution; also a baseline.","marker":"[2]"},{"why":"DeepCache documents high-dimensional feature similarity across adjacent timesteps, the observation behind TBS's cross-timestep connections.","marker":"[38]"},{"why":"Q-DM demonstrates that diffusion activation distributions depend on the denoising timestep, motivating the timestep-aware quantizer.","marker":"[30]"},{"why":"TDQ supplies complementary evidence, cited with Q-DM, that adjacent timesteps share activation distributions while distant ones differ.","marker":"[55]"},{"why":"ReActNet provides the shortcut-connection design and binary training practices BiDM reuses, and is a W1A1 baseline in all experiments.","marker":"[33]"},{"why":"BinaryDM is the closest prior work, quantizing diffusion weights while leaving activations at higher width; BiDM's full W1A1 result is set against it.","marker":"[77]"},{"why":"EfficientDM is the quantization-aware fine-tuning baseline for diffusion models that BiDM compares with at 1 bit; its TALSQ quantizer also targets activation range variation.","marker":"[15]"},{"why":"ReSTE is the strongest general binarization baseline on LSUN-Bedrooms (FID 59.44) that BiDM outperforms at W1A1.","marker":"[62]"}],"fun_headline_variants":["1-bit diffusion models: FID 22.74, 28x storage cut","Fully binary diffusion: FID 22.74, 28x smaller","BiDM: 1-bit diffusion, 28x storage savings","Diffusion at 1-bit: first to work, FID 22.74","BiDM: 1-bit diffusion, 28x storage, 52.7x OPs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline efficiency claim assumes that the extra full-precision operations introduced by TBS—channel averaging, the tiny convolution, pointwise multiplications, and cross-timestep blending—add negligible inference cost; the paper's own appendix runtime measurement shows the binary model with these additions runs at 21.86 ms per convolution versus 17.70 ms for a simpler binary baseline and only 8.07x faster than full precision, not 52.7x.","fun_headline_variants_meta":{"raw":{"variants":["1-bit diffusion models: FID 22.74, 28x storage cut","Fully binary diffusion: FID 22.74, 28x smaller","BiDM: 1-bit diffusion, 28x storage savings","Diffusion at 1-bit: first to work, FID 22.74","BiDM: 1-bit diffusion, 28x storage, 52.7x OPs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000857,"raw_usage":{"total_tokens":3797,"prompt_tokens":1099,"completion_tokens":2698,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":715,"completion_tokens_details":{"reasoning_tokens":2603}},"tokens_in":715,"tokens_out":2698,"duration_ms":18874,"temperature":1.0,"reasoning_tokens":2603,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:11:16.646750+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the released BiDM model end-to-end on the same GPU or edge device, measure total sampling latency against the full-precision model, and compare the ratio with the claimed 52.7x OPs savings; the paper's own appendix already reports an 8.07x per-convolution speedup on a Snapdragon 855 Plus, so an end-to-end ratio near 8x rather than 52.7x would falsify the practical-efficiency claim.","supporting_citations":[{"cited_title":"Xnor-net: Imagenet classification using binary convolutional neural networks","cited_arxiv_id":null,"evidence_quote":"Supplies the binary-convolution approximation that TBS modifies by making the scaling convolution learnable, and serves as the vanilla W1A1 baseline."},{"cited_title":"Q-dm: An efficient low- bit quantized diffusion model","cited_arxiv_id":null,"evidence_quote":"Q-DM demonstrates that diffusion activation distributions depend on the denoising timestep, motivating the timestep-aware quantizer."},{"cited_title":"Reactnet: Towards precise binary neural network with generalized activation functions","cited_arxiv_id":null,"evidence_quote":"ReActNet provides the shortcut-connection design and binary training practices BiDM reuses, and is a W1A1 baseline in all experiments."},{"cited_title":"Estimator meets equilibrium perspective: A rectified straight through estimator for binary neural networks training","cited_arxiv_id":null,"evidence_quote":"ReSTE is the strongest general binarization baseline on LSUN-Bedrooms (FID 59.44) that BiDM outperforms at W1A1."}],"review_version":1}