{"id":"e0ae2541-0163-4d77-9086-2337f78fbc68","arxiv_id":"2506.15381","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"DDIS generates training-like images from a frozen classifier by steering Stable Diffusion with batch-normalization statistics and an optimized per-class token, improving data-free distillation and pruning.","lead":"This paper makes a frozen image classifier teach a text-to-image diffusion model to recreate its missing training data. The method yields sharper synthetic images that improve data-free knowledge distillation and pruning on ImageNet and PACS.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 7 and Algorithm 1 use a plus-sign DDIM update that is not a valid reverse diffusion step; with no released code, the reported DAG-guided synthesis and downstream results are not reproducible as written.","rationale":"The reader rightly flags the DAG assumption that BN running statistics sufficiently encode the training domain, and the paper's own limitation paragraphs support that caution. However, the more immediate blocker is that the algorithmic specification in Sections 3.1 and 3.3 does not match standard DDIM/DDPM sampling. The reported results could still be valid if the equations contain transcription errors and the implementation follows a standard diffusion library, but without code or a corrected derivation the central claim is not independently checkable. I therefore keep the reader's CONDITIONAL verdict: the paper needs either corrected equations or a released implementation before the empirical claims can be accepted. My concern is about the reproducibility of the argument, not about author intent; a simple typo would resolve it, and the concrete test distinguishes that case from a substantive algorithmic error.","tokens_in":19085,"tokens_out":13844,"duration_ms":147040,"concrete_test":"Reimplement Algorithm 1 literally with the plus-sign update from Eq. 7 using Stable Diffusion 2.1 and a ResNet-34 ImageNet teacher; generate 1,000 images per class and measure FID against the ImageNet training set. If the literal implementation gives FID far above the reported 30.31 and the samples lack recognizable structure, the published update is not the one that produced Table 2. Equivalently, replace the first term in Eq. 7 with the standard minus-sign DDIM update and check that the paper's qualitative results and FID are reproduced; the released code's prev-sample line settles which variant was actually used.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The sampling equations in Section 3 are internally inconsistent with standard DDIM/DDPM. Eq. 7 defines z_{t-1} = sqrt(alpha_bar_{t-1}) * ((z_t + (1 - alpha_bar_t) * eps_t^phi) / sqrt(alpha_bar_t)) + ..., and Algorithm 1 line 8 repeats this update. Standard DDIM predicts the clean latent as (z_t - sqrt(1 - alpha_bar_t) * eps) / sqrt(alpha_bar_t), so the plus sign and the coefficient (1 - alpha_bar_t) instead of sqrt(1 - alpha_bar_t) both appear wrong. Eq. 2 for the DDPM posterior mean also uses non-standard coefficients, and Algorithm 1 line 4 references epsilon_{t+1} before it is defined. Taken literally, the DAG-guided sampling described in the paper does not implement reverse diffusion; at the early sampled timesteps the plus-sign term would add a large predicted-noise component and should destroy image structure. Since no code is released, the central claim that DDIS produces the images behind Tables 1-2 cannot be checked from the paper as written. If these are only transcription typos and the implementation uses a standard DDIM update, the substantive method may survive, but the discrepancy must be resolved before the empirical claims can be accepted. The BN-statistics sufficiency concern raised by the reader is real but secondary: it applies only once the sampling update is correctly specified, and Section 5 already concedes that DAG is BN-only and weak on Sketch.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DDIS, a data-free image synthesis method that combines a frozen text-to-image diffusion model (Stable Diffusion 2.1) with two alignment mechanisms: Domain Alignment Guidance (DAG), which steers diffusion latents so that feature statistics of decoded images match the running Batch Normalization statistics of a pretrained classifier, and a Class Alignment Token (CAT), a pseudo-word embedding optimized with cross-entropy loss against the classifier. The method is evaluated on PACS art/cartoon, Style-Aligned manga/caricature, and ImageNet-1k using IS, FID, Precision, Recall, and on downstream data-free knowledge distillation and pruning. The paper claims state-of-the-art DFIS performance and that CAT resolves lexical ambiguity of class labels.","tokens_in":19432,"tokens_out":7623,"duration_ms":68815,"significance":"If the method works as described, it offers a practical recipe for synthesizing proxy training data from only a frozen pretrained classifier plus an off-the-shelf text-to-image model; the downstream DFKD and pruning accuracies on real test sets are externally grounded and not circular. The first systematic extension of DFIS to non-photo domains (art, cartoon, manga, caricature) is a useful contribution. The main empirical results are, however, currently not reproducible from the paper because the sampling update is invalid as written, and the theoretical framing of DAG conflates a heuristic loss with a likelihood. These issues are load-bearing and must be resolved before the empirical claims can be accepted.","major_comments":[{"comment":"The sampling update in Eq. (7) and Algorithm 1 is not a valid reverse-diffusion step. Standard DDIM predicts the clean latent as (z_t - sqrt(1 - alpha_bar_t) * eps) / sqrt(alpha_bar_t), whereas the paper writes z_t + (1 - alpha_bar_t) * eps in the numerator and multiplies the additional epsilon term by (1 - alpha_bar_t) instead of sqrt(1 - alpha_bar_t). Taken literally, the plus sign adds the predicted noise at early timesteps and should corrupt the sample. Eq. (2) for the DDPM posterior mean is also nonstandard. Since no code is released, the images behind Tables 1 and 2 cannot be reproduced as written. The authors should correct Eq. (2), Eq. (7), and Algorithm 1 (including the index error involving epsilon_{t+1}), or release code so the actual sampling procedure can be verified.","section":"Section 3.1, Eq. (7); Algorithm 1, line 8"},{"comment":"The score factorization p(z_t | mu, sigma^2) proportional to p(z_t) p(mu, sigma^2 | z_t) is not well-defined: mu and sigma^2 are the fixed running statistics of the pretrained BN layers, constants rather than random variables generated by z_t, so log p(mu, sigma^2 | z_t) has no probabilistic meaning. Replacing that term with the gradient of L_BN therefore makes DAG a heuristic, not a likelihood-based guidance. This would be acceptable if the paper explicitly stated the heuristic status and gave evidence that BN running statistics are a sufficient summary of the training domain, but the current derivation overclaims. Section 5's concession that DAG only applies to BN models does not repair the logical gap in the factorization.","section":"Section 3.3, Eqs. (8)-(10)"},{"comment":"The DAG hyperparameters lambda_BN and s_g are selected using the confidence score of generated images (Appendix A.2), and several appendix design-choice evaluations use classifier confidence as the success metric (Tables 5-7). Because the CAT is also optimized against the same classifier, the reported improvements may partly reflect fitting the classifier's own confidence rather than fidelity to the true training distribution. In addition, the main quantitative tables report single runs with no seeds. Please report means and standard deviations over multiple seeds, and either fix DAG hyperparameters a priori or validate the final choice on a criterion that does not use the classifier's own confidence.","section":"Appendix A.2; Tables 5-7"},{"comment":"The claim of state-of-the-art performance in data-free applications is only supported by comparisons with DI, PII, and unguided SD. Table 2 does not include established data-free knowledge distillation baselines such as DAFL, ZSKD, or DFAD, and Figure 6 similarly compares only the same DFIS sources for pruning. The state-of-the-art claim should be restricted to the compared DFIS baselines, or the comparison set should be expanded to include representative DFKD and data-free pruning methods.","section":"Table 2; Figure 6"}],"minor_comments":[{"comment":"The update on line 4 uses epsilon_{t+1} before it is introduced; line 3 defines epsilon_t. Please make the index bookkeeping consistent.","section":"Algorithm 1, line 4"},{"comment":"The return set {x_hat_0^1, ..., x_hat_0^C} is not produced by the loop as written, because x_hat_0 is overwritten each iteration; either store per-class outputs or rephrase the pseudocode.","section":"Algorithm 2, line 13"},{"comment":"The prompt template is given as 'A/An {S_c} {class label}' in the text, while Figure 1 shows 'A photo of S_c tiger cat'; please specify the exact prompt format used in the experiments.","section":"Section 3.4; Figure 1"},{"comment":"The row labels 'SD w/o DAG' and 'SD w/o CAT' appear swapped relative to the check marks, since the row labeled 'SD w/o DAG' has a check in the DAG column; please relabel or add a clarifying note.","section":"Table 3"},{"comment":"The parameter w and the procedure for re-weighting the CAT embedding via the cross-attention map are not defined; please add a formal definition.","section":"Figure 7"},{"comment":"The claim of being 'the first' Diffusion-assisted DFIS method should be qualified by an explicit literature check; if no prior work exists, state that directly with the search scope.","section":"Abstract and Introduction"}],"recommendation":"major_revision","confidential_remarks":"The technical issues in the diffusion update and the score factorization appear fixable, and the downstream evaluation on real test sets is a genuine strength. I would support a revision rather than rejection. Given the incorrect equations, requesting code release or a corrected derivation in the revision seems appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a solid contribution to data-free image synthesis, and the main idea is genuinely useful: instead of inverting a classifier in raw pixel space, run a frozen Stable Diffusion and guide it with the classifier's BN statistics (DAG) and a per-class optimized text token (CAT). The results are impressive—huge FID and DFKD gains over DI and PII on PACS and ImageNet, and the first multi-domain DFIS (art, cartoon, manga, caricature). The CAT trick convincingly fixes lexical ambiguity (e.g., tiger cat vs tiger). The downstream KD and pruning numbers are on real test sets, so the utility claim is not circular.\n\nThat said, there is a load-bearing technical flaw in the paper as written. Equations (7) and (12) and Algorithm 1 describe a DDIM update with a plus sign where the standard DDIM predicts the clean latent as (z_t - sqrt(1-ᾱ_t) ε)/√ᾱ_t. The paper instead writes (z_t + (1-ᾱ_t) ε̃)/√ᾱ_t. That is not a valid reverse step; at early timesteps the plus sign would add a large noise component and destroy the image. With no released code, the reader cannot verify the results. I suspect it's a transcription error—the rest of the method is coherent, and the reported images look like they came from a proper sampler—but a paper that cannot be reproduced from its equations is not acceptable as is.\n\nThe other weaknesses are less severe. The DDIM bug also appears in the derivation of Eq. (2) (coefficients look off). There are no error bars or multiple seeds; the baseline list is thin (only DI and PII, with NI dismissed). Some appendix evaluations use classifier confidence, which is self-referential; however, the main tables use external tasks, so that's minor. The DAG relies on BN statistics as a proxy for domain; the paper's own limitation section admits it only works on BN models and fails on Sketch, which is honest and actually bolsters credibility.\n\nWho is this for? Researchers working on data-free KD, pruning, or model inversion with diffusion priors. They will find the CAT/DAG insights worth building on. I would not cite it in its current form because the sampling equations are not reproducible. But it deserves a serious peer review—the idea is sound and the empirical upside is real. The reviewers should ask for a corrected derivation, code release, and at least one more recent baseline.\n\nBest,\n[Your name]","headline":"The method and results are strong, but the DDIM update in Eqs. 7/12 is non-standard and, without code, the paper is not reproducible as written.","tokens_in":19961,"tokens_out":2973,"would_cite":false,"duration_ms":30491,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T45","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"A frozen text-to-image diffusion model, guided by a classifier's batch-norm statistics and one optimized class token, can generate proxy images close to the true training distribution, achieving state-of-the-art data-free distillation and…","keywords":["data-free image synthesis","diffusion models","batch normalization statistics","domain alignment","knowledge distillation","model pruning","text-to-image generation","class token embedding"],"falsifier":"Train a classifier on a domain whose per-layer feature statistics are deliberately matched to a different visual domain (e.g., renormalize a sketch dataset so its layer-wise means and variances coincide with a color-photo dataset at every batch-norm layer), then run DDIS against it: if the generated images follow the statistics rather than the actual content, the BN-statistics premise is doing the work, and if they follow neither domain, the guidance signal is too weak to matter. A more direct check is to apply DDIS to a model without batch-norm layers, such as a GroupNorm-based ResNet or a vision transformer trained on the same data, and observe whether the domain-alignment benefit disappears as the paper's Section 5 limitation implies it should.","tokens_in":18876,"feed_emoji":"🖼️","tokens_out":12911,"duration_ms":107040,"temperature":0.7,"pith_summary":"The paper claims that data-free image synthesis stops being a fragile search in pixel space once a frozen text-to-image diffusion model supplies the image prior, and two alignment mechanisms keep the samples on distribution. Domain Alignment Guidance (DAG) steers the diffusion latent at every step so that the generated image's per-layer feature statistics match the batch-normalization running statistics stored inside the pretrained classifier, which the paper treats as the encoding of the training domain. A single Class Alignment Token (CAT) per class, a learnable pseudo-word added to the prompt, absorbs the class-specific visual details that the bare label name does not convey. The claim is that this combination outperforms previous data-free synthesis methods on ImageNet-1k and on art, cartoon, manga, and caricature domains, and that the resulting proxy data supports knowledge distillation and pruning at accuracy close to the original-data baseline. Why this matters: shared pretrained models could keep their utility for downstream tasks even when their training data is withheld.","feed_headline":"Diffusion prior plus classifier stats tops data-free synthesis","feed_subtitle":"Guided by batch-norm statistics, frozen Stable Diffusion nears original-data distillation scores.","key_machinery":"Two mechanisms carry the argument. Domain Alignment Guidance (DAG) is a per-step latent correction applied during diffusion sampling: the decoded image $\\hat{x}_t = D(z_t)$ is passed through the frozen classifier, the squared error between its layer-wise feature statistics and the classifier's batch-norm running statistics, $\\mathcal{L}_{BN}(\\hat{x}_t) = \\sum_{l=1}^{L} \\big( \\|\\mu_l(\\hat{x}_t) - \\mu_l\\|^2 + \\|\\sigma_l^2(\\hat{x}_t) - \\sigma_l^2\\|^2 \\big)$, is differentiated with respect to the latent, and the update $\\tilde{z}_t = z_t - \\eta \\nabla_{z_t} \\mathcal{L}_{BN}(D(z_t))$ is folded into the Classifier-Free Guidance sampling loop. The Class Alignment Token (CAT) is a single learnable pseudo-word embedding $v_c$ inserted into the prompt 'A/An $\\{S_c\\}$ {class label}'; only this $1 \\times 784$ embedding vector is optimized, via cross-entropy loss from the classifier on the final image, with gradient skipping to the last diffusion step to keep memory bounded. DAG supplies the domain direction and CAT supplies the class direction; the work claims each is necessary, since vanilla Stable Diffusion lacks the domain signal and class-only guidance drifts across domains.","core_discovery":"The central claim, stated on the paper's own terms, is that a pre-trained text-to-image diffusion model can serve as the missing natural-image prior for data-free image synthesis, and that aligning the generated images to the given classifier's internal statistics at both the domain and class level closes the distribution gap that prior DFIS methods leave open. DAG rewrites the unconditional score $\\nabla_{z_t} \\log p(z_t)$ as a conditional score $\\nabla_{z_t} \\log p(z_t \\mid \\mu, \\sigma^2)$, where the conditioning comes from the Bayes-factor gradient of a batch-norm-statistics matching loss $\\mathcal{L}_{BN}$; CAT adds a per-class learnable pseudo-word embedding to the prompt and optimizes it with cross-entropy loss backpropagated through the final denoising step. The paper reports that DDIS beats DeepInversion and PlugInInversion on every evaluated metric across ImageNet-1k and PACS/Style-Aligned domains, and that data-free knowledge distillation with DDIS synthetic data lands within roughly 1 to 4 accuracy points of training the student on the original data (e.g., 41.68 versus 43.30 top-1 for ResNet-34 to ResNet-18 on ImageNet-1k), with pruned-model fine-tuning superior to all baselines at every pruning ratio.","pith_inferences":["A testable extension of the paper's own logic: the DAG mechanism is not tied to batch normalization in principle, since any differentiable statistic of the frozen model's feature maps (GroupNorm moments, Gram matrices, attention statistics) could carry the domain signal; whether non-BN architectures can be steered the same way is open, and the paper's Section 5 confirms DAG currently cannot handle","The same $\\mathcal{L}_{BN}$ alignment could double as a domain-shift probe: if DAG genuinely pulls latents toward the training domain, the residual loss value during sampling measures how far a prompt's output domain sits from the classifier's domain, which could serve label-free test-time adaptation or out-of-distribution detection.","The paper's ablation implies the diffusion prior alone is not the source of the gains: vanilla SD distillation accuracy is 33.02 versus 41.68 with DDIS on ImageNet (ResNet-34 to ResNet-18), suggesting the alignment terms, not the generative model, are the differentiator; a cheaper generative prior matched with DAG might reach similar results.","The early-stopping rule (70 percent batch accuracy) and single-seed CAT optimization likely under-sample intra-class variance, so the reported Recall values (0.32-0.77) could improve by optimizing the token over multiple seeds or adding a diversity regularizer without changing the method's core."],"forward_implications":["A surrogate training set for a pretrained classifier can be synthesized from Stable Diffusion plus batch-norm statistics alone: on ImageNet-1k, DDIS reports FID 30.31 versus 187.63 for DeepInversion and 220.62 for PlugInInversion, with distillation accuracy within about 1.6 points of the original-data baseline for ResNet-34 to ResNet-18.","Data-free knowledge distillation becomes practical on non-photographic domains: DDIS is the first DFIS method to handle art-painting, cartoon, manga, and caricature domains, where prior methods produce near-random images (e.g., precision below 0.01 on cartoon and manga).","The CAT embedding resolves lexical ambiguity in class names: classes like 'tiger cat', 'beach wagon', and 'mail bag' are generated correctly even though the bare class label misleads Stable Diffusion, because the optimized token pins down the intended concept.","Pruned models can be fine-tuned without original data: DDIS synthetic samples beat all baselines across 50-90 percent pruning ratios on ImageNet and PACS for both ResNet-34 and VGG-16.","Generation is cheap after the per-class token is found: optimizing one $1 \\times 784$ embedding takes about 7.5 minutes per class and is reused for unlimited sampling, reducing the total iterations for 100k ImageNet images from 8,000K (DeepInversion) and 1,120K (PlugInInversion) to 30K."],"supporting_citations":[{"why":"Supplies the text-to-image latent diffusion model (Stable Diffusion) whose natural-image prior DDIS builds upon.","marker":"(Rombach et al., 2022)"},{"why":"Defines the DeepInversion baseline and the batch-norm-statistics regularizer that DAG generalizes and must outperform.","marker":"(Yin et al., 2020)"},{"why":"Provides the classifier-free guidance sampling loop into which DAG injects its gradient.","marker":"(Ho & Salimans, 2022)"},{"why":"The textual-inversion idea of encoding a concept in a pseudo-word token embedding that CAT adapts.","marker":"(Gal et al., 2022)"},{"why":"The PACS benchmark supplies the art-painting and cartoon domains used to test cross-domain synthesis.","marker":"(Li et al., 2017)"},{"why":"ImageNet-1k is the large-scale photo-domain dataset whose withheld training set is being approximated.","marker":"(Russakovsky et al., 2015)"},{"why":"The classifier-guidance score decomposition that motivates treating the BN-statistics loss as a conditional-score term.","marker":"(Dhariwal & Nichol, 2021)"},{"why":"Batch normalization defines the running statistics that DAG treats as the encoding of the training domain.","marker":"(Ioffe & Szegedy, 2015)"}],"fun_headline_variants":["Diffusion prior beats data-free synthesis baselines","Batch-norm-guided diffusion closes data-free gap","Data-free synthesis gets a diffusion boost","Aligning domain and class makes synthetic data real","Stable Diffusion as prior for data-free distillation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"DAG assumes the running mean and variance stored in every batch-normalization layer of the frozen classifier are a sufficient summary of the training domain, so that minimizing the squared gap between generated-image statistics and those running statistics at every diffusion step pulls the latent toward the true training distribution; if those statistics do not encode the domain, the guidance term supplies a wrong or noisy signal, and the paper's own limitation that DAG only works for batch-norm models exposes exactly this dependence.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion prior beats data-free synthesis baselines","Batch-norm-guided diffusion closes data-free gap","Data-free synthesis gets a diffusion boost","Aligning domain and class makes synthetic data real","Stable Diffusion as prior for data-free distillation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000219,"raw_usage":{"total_tokens":1493,"prompt_tokens":1042,"completion_tokens":451,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":658,"completion_tokens_details":{"reasoning_tokens":382}},"tokens_in":658,"tokens_out":451,"duration_ms":4983,"temperature":1.0,"reasoning_tokens":382,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:35:08.599035+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a classifier on a domain whose per-layer feature statistics are deliberately matched to a different visual domain (e.g., renormalize a sketch dataset so its layer-wise means and variances coincide with a color-photo dataset at every batch-norm layer), then run DDIS against it: if the generated images follow the statistics rather than the actual content, the BN-statistics premise is doing the work, and if they follow neither domain, the guidance signal is too weak to matter. A more direct check is to apply DDIS to a model without batch-norm layers, such as a GroupNorm-based ResNet or a vision transformer trained on the same data, and observe whether the domain-alignment benefit disappears as the paper's Section 5 limitation implies it should.","supporting_citations":[{"cited_title":"M., Li, Z., Mallya, A., Hoiem, D., Jha, N","cited_arxiv_id":null,"evidence_quote":"Defines the DeepInversion baseline and the batch-norm-statistics regularizer that DAG generalizes and must outperform."}],"review_version":2}