{"id":"d4cce6ab-5da5-4f94-917c-de28d9ee1d22","arxiv_id":"2411.15393","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A gradient-free diffusion sampler that selects a reference class from a pretrained classifier's predictions and adapts guidance strength improves class-conditional fidelity, but its Precision gains are measured with the same classifier used for guidance.","lead":"This paper introduces a gradient-free way to use a pretrained classifier during diffusion sampling: it steers the generated image away from the class the classifier currently considers most likely instead of computing gradients. The method improves classifier-reported accuracy and, combined with Autoguidance, reaches FD_DINOv2 23.09 on ImageNet 512, although part of the accuracy gain is circular.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Precision gains may be circular: GFCG steers with the same classifier that later scores Precision, so the headline Precision advantage over ATG is not independent evidence of fidelity.","rationale":"The reader's weakest assumption is the right one. The paper's headline quantitative claim is a simultaneous gain in FD_DINOv2 and Precision, but the Precision half is not measurable from the reported experiments because the guidance loop and the evaluation loop share the exact same classifier objective. Steering away from the second-most-likely class and toward the target class, as judged by C, is expected to increase C's posterior on the final image even if the image is no better for humans or other classifiers. This does not make the method useless—the FD_DINOv2 result, if computed correctly, is an independent signal, and a held-out classifier test could vindicate the Precision claim—but it means the current evidence for 'consistently improves class prediction accuracy' is weaker than stated. I therefore keep the reader's CONDITIONAL verdict rather than upgrading or rejecting: the central construction is coherent and testable, but it needs a non-circular evaluation before the Precision numbers can be accepted. The supplementary's admission of 7 extra NFEs is an additional inconsistency that should be corrected in revision, but it is less fundamental than the metric circularity.","tokens_in":18815,"tokens_out":5583,"duration_ms":51255,"concrete_test":"Recompute the Precision column for the EDM2-XXL GFCG+ATG and ATG rows of Table 1, and for the SD 1.5 rows of Table 2, using a held-out classifier that was never used during GFCG sampling — e.g., a ViT-L/16 or Swin-L fine-tuned on ImageNet for the class-conditional setting, and a separately trained ResNet-50/ViT on Bird Species for the text-to-image setting. If the Precision advantage over ATG/CFG persists, the circularity concern is resolved; if it shrinks or reverses, the headline Precision gain is an artifact of optimizing the same evaluator.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing weakness is that the Precision metric is evaluated with the same classifier that is used for guidance. In §3.2, Eq. 5 and Eq. 6 and Algorithm 3 use p(c|x0_hat) from a pretrained ResNet-101 ImageNet classifier to select cref and ω; §4 then defines Precision as the fraction of generated samples classified correctly 'using the real image classifier', which is the same ResNet-101 for ImageNet and the same Bird Species classifier for SD 1.5. GFCG therefore increases Precision by construction: each guided step moves the sample away from the top competing class as judged by that very classifier. This is reward hacking on the evaluation metric, not independent evidence that the generated image is semantically closer to the target class. The FD_DINOv2 numbers are independent and do support a real distributional-fidelity gain, but they do not measure class accuracy. A second, separate inconsistency is that the abstract's 'without additional computational overhead' is contradicted by Supp. §8.4, which reports 7 additional NFEs for the 4-step x0_hat estimation. That is fixable by rewording; the circular Precision is the load-bearing flaw.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Gradient-free Classifier Guidance (GFCG), a sampling-time method that uses a pretrained classifier in inference mode to adaptively select a reference class cref and a guidance scale omega at each denoising step. When the classifier confidence in the desired class cdes falls below a threshold, the method steers the sample away from the most competing class via the update bD = omega*Dm(x_t,t,cdes) - (omega-1)*Dg(x_t,t,cref). The method is combined with CFG and ATG in mixed or additive modes, and is evaluated on EDM2 ImageNet 512x512 and on Stable Diffusion 1.5 with a fine-grained Bird Species classifier. The main claims are consistent improvements in class prediction accuracy (Precision), complementarity with CFG/ATG, and a record FD_DINOv2 of 23.09 on ImageNet 512 with GFCG+ATG.","tokens_in":42,"tokens_out":9207,"duration_ms":203431,"significance":"If the results hold, GFCG is a practically valuable contribution: it avoids backpropagation through the diffusion network, makes adaptive use of an off-the-shelf classifier, and can be layered on top of CFG or ATG. The manuscript has real strengths: it builds on a public code base (EDM2), reports extensive ablations over alpha, beta, tau, ts, scp, guidance model choice, and classifier choice, includes a compression-based justification for preferring FD_DINOv2 over FID, and provides a random-seed variation study in the supplement. However, the headline Precision gains are measured with the same classifier that generates the guidance signal, so they are not independent evidence of class-label fidelity; the FD_DINOv2 results are independent and do suggest a real distributional improvement, but the 'record' claim is benchmarked only against a local rerun of ATG, and the 'no additional computational overhead' assertion is contradicted by the supplementary NFE accounting. These issues affect the central empirical claims and need to be addressed.","major_comments":[{"comment":"The Precision metric is circular with respect to the guidance signal. Guidance in Eq. (5) uses the classifier's confidence p(c_des|x0_hat) to set omega and select cref, and Algorithm 3 uses that same classifier during sampling. Precision is then defined in Section 4 as the percentage of generated samples classified correctly by 'the real image classifier', which for ImageNet is the same ResNet-101 described in Section 4.1 and for SD 1.5 is the same Bird Species classifier used for guidance. Consequently, any method that raises this classifier's confidence on the desired class will increase Precision by construction, even if the image is not semantically closer to the class from an independent viewpoint. The paper should re-measure Precision with an independent classifier (different architecture and/or training data) that is not used for guidance, and should report the difference; without this, the Precision gains in Tables 1 and 2 (e.g., 95.4% vs. 90.6% and 32.3% vs. 27.3%) cannot be interpreted as evidence of genuine class-label alignment.","section":"§3.2 (Eq. 5), §4, Tables 1–2"},{"comment":"The claim of 'without additional computational overhead' is contradicted by the implementation details. Supp. §8.4 states that the multi-step x0_hat estimation with T'=4 'introduces 7 additional NFEs' (63 to 70), and Table 6 confirms that the main-paper configuration uses 70 NFEs versus a 63-NFE baseline. Since the headline ImageNet results in Table 1 are all reported for scp set to its maximum and T'=4, the extra cost is part of the reported configuration, not an optional add-on. Please either remove the 'no additional computational overhead' wording, report runtime and NFE side-by-side for all compared methods, or clearly restrict the claim to configurations with a single-step x0_hat estimate.","section":"Abstract and §5 vs. Supp. §8.4 (Table 6)"},{"comment":"The 'record FD_DINOv2 of 23.09' is not supported by the reported comparison. The only ATG baseline in Table 1 for EDM2-XXL is the authors' rerun at 24.83, and the text acknowledges that this rerun is 'a little worse than the published value' in the ATG paper. A record claim should be benchmarked directly against the best published number for the same model, dataset, and evaluation protocol; otherwise 23.09 is merely an improvement over a locally rerun baseline. The random-seed study in Supp. §8.1 is a good step, but it is shown for the EDM2-S configuration and does not cover the XXL headline. Please state the published ATG FD_DINOv2 for this setting and demonstrate that 23.09 improves on it.","section":"§4.1, Table 1; Abstract"},{"comment":"The resource-requirement narrative is also misleading regarding the need for a second diffusion model. GFCG's update in Eq. (6) requires a forward pass Dg_phi(x_t,t,cref) from a separate guidance network; for the reported ImageNet experiments this is the (XS,T/16) or (M,T/3.5) model, which must be available or trained. The statement in Section 5 that GFCG avoids 'requiring the training of an extra unconditional model, as seen with classifier-free guidance' is technically about an unconditional model, but it obscures that GFCG still requires an auxiliary conditional model for the reference class. Please clarify this second-model requirement, and state explicitly whether GFCG can be run with Dg = Dm when no smaller guidance model is available.","section":"§3.2 (Eq. 6), §5"},{"comment":"The core steering assumption is that classifier probabilities computed on an intermediate denoised estimate x0_hat reliably indicate the final image's class content. The paper does not provide validation of this assumption; the failure cases in Supp. §10.1 (e.g., the TIT MOUSE and TEAL examples) show that GFCG can amplify incorrect features rather than correct them. Please report an empirical check of this assumption, for example by measuring the agreement between the classifier's argmax on x0_hat at intermediate steps and the final classified class of the generated sample, or by analyzing how the reliability of p(c|x0_hat) depends on the denoising schedule and the step budget T'.","section":"§3.2, Eq. (5); Supp. §10.1"}],"minor_comments":[{"comment":"The sentence 'Image generation using diffusion models have demonstrated...' has a subject-verb agreement error; 'have' should agree with 'models' or the sentence should be rephrased to 'Image generation using diffusion models has demonstrated...'.","section":"Abstract"},{"comment":"There is a typographical error in 'Forscp, it is used to determine...'; it should read 'For scp, it is used to determine...'.","section":"§3.3"},{"comment":"The sentence 'The best FD DINOv2 of all is achieved when the additive guidance of GFCG and CFG, setting a SOTA performance of 33.39' is grammatically incomplete and also appears to refer to GFCG+ATG+CFG (the configuration with omega_ATG=2.45 and omega_CFG=1.60 in Table 1), not merely GFCG+CFG; please rephrase and make the naming consistent.","section":"§4.1"},{"comment":"The notation 'j∼{1,2,...,N}−{des}' is unclear; it should be written as j ∈ {1,...,N} \\ {des} to denote all classes except the desired one.","section":"Equation (7)"}],"recommendation":"major_revision","confidential_remarks":"The circular Precision evaluation is the main obstacle; the authors should be pushed to add an independent-classifier evaluation and to temper the 'no overhead' and 'record' claims. The FD_DINOv2 results appear to be the strongest independent evidence, so the paper may become acceptable after these revisions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Take a look at this if you work on guidance. The new thing here is using a frozen classifier, forward pass only, to adaptively pick both the reference class and the guidance scale at each timestep. Eq. 6 is the standard CFG combination with a classifier-selected reference class in place of the unconditional one; the contribution is the selection mechanism. The algorithm is clearly specified, the ablations are reasonably thorough (they vary alpha, beta, ts, scp, classifier backbone, guidance model), and they are honest about failure cases like TIT MOUSE in the supplement. The FD_DINOv2 gains when GFCG is stacked with ATG are plausible and not explained away by the circularity issue.\n\nThe soft spots, in order. First and most important, the Precision metric is the same ResNet-101 that produces the guidance signal. GFCG explicitly moves samples away from the top competing class as judged by that classifier, so Precision as top-1 accuracy of that classifier is boosted by construction. That means the Precision gap over ATG (94.3 vs 90.2) is not independent evidence of fidelity. A held-out classifier would fix this and should be required. Second, the 'record' FD_DINOv2 of 23.09 is compared to their rerun of ATG (24.83) rather than the published ATG value, which I believe is lower. Same-seed comparison is good practice, but you need both; otherwise the record claim reads as being made against a weaker baseline. Third, 'without additional computational overhead' is contradicted by Supp. 8.4, which reports 7 extra NFEs for the 4-step x0_hat estimate. That is a minor wording issue and fixable.\n\nOverall, the central mechanism is coherent, and the FD_DINOv2 numbers do suggest a real distributional fidelity improvement when combined with ATG. But until Precision is measured with an independent classifier, I would not treat the classification-accuracy claim as established. The paper deserves a serious referee: the idea is useful, the experiments are extensive, and the flaws are addressable. I would send it to review with a request for the held-out classifier, a published-baseline comparison, and a corrected overhead statement.","headline":"Adaptive reference-class guidance is a real idea, but the Precision headline is circular until an independent classifier scores it.","tokens_in":19630,"tokens_out":1873,"would_cite":true,"duration_ms":17245,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A pretrained classifier used only for inference can guide diffusion sampling toward a target class, and does so without gradient backpropagation.","keywords":["gradient-free guidance","diffusion models","classifier guidance","class-conditional generation","text-to-image generation","adaptive guidance scale","reference class","FD_DINOv2"],"falsifier":"Compare the classifier's top competing class on intermediate denoised estimates with the actual failure mode of the final image as judged by a held-out classifier or human labels; if the intermediate top competitor frequently is not what the final image gets confused with, the steering target is wrong and GFCG should degrade. A direct control is to rerun GFCG with the classifier's softmax replaced by a fixed random reference class: if precision still rises, the classifier signal is not the cause.","tokens_in":18622,"feed_emoji":"🎯","tokens_out":7949,"duration_ms":60098,"temperature":0.7,"pith_summary":"Diffusion models generate diverse images, but steering them toward a target class usually costs either gradient backpropagation or training an unconditional model. This paper proposes a third route: use a pretrained classifier in inference mode only, and at each denoising step ask it to classify a denoised estimate of the final image. If the desired class is not confident, the sampler picks the most confusable competing class as a reference and steers away from it, with a guidance scale that adapts to the classifier's confidence. Across class-conditional and text-to-image models, the method improves class prediction accuracy and combines with existing guidance methods to improve fidelity while preserving diversity.","feed_headline":"No-gradient classifier guidance sets an ImageNet fidelity record","feed_subtitle":"Steering samples away from the most confused class lifts precision to 94.3% while preserving diversity.","key_machinery":"The load-bearing mechanism is the time-adaptive pair $(\\omega, c_{\\rm ref})$ that a pretrained classifier derives from a denoised estimate of the image. Equation (5) turns classifier confidence into a continuous guidance scale, and the reference-class rule makes the guidance target not a fixed unconditional model but the class the model is most likely to confuse with the desired one. Algorithm 3 plugs this into the contrast update $bD = \\omega D^m(x_t,t,c_{\\rm des}) - (\\omega-1)D^g(x_t,t,c_{\\rm ref})$, with optional multi-step denoising for $\\hat{x}_0$ and stochastic reference-class sampling. This lets a single off-the-shelf classifier replace backpropagation while keeping the contrast-based guidance form of classifier-free guidance.","core_discovery":"The paper claims that a pretrained classifier, used only in inference mode, can guide diffusion sampling toward a desired class by adaptively choosing a reference class and guidance strength at each time step. For a desired class $c_{\\rm des}$ and noisy sample $x_t$, the sampler estimates the clean image $\\hat{x}_0$, obtains probabilities $p(c|\\hat{x}_0)$, and when $p(c_{\\rm des}|\\hat{x}_0) < \\tau$ it sets $\\omega = 1 + \\alpha \\exp(-\\beta(p(c_{\\rm des}|\\hat{x}_0)-\\tau))$ and selects as reference class the highest-probability competing class (or the second-highest when the desired class is already most probable). The denoising update then takes the contrast form $bD = \\omega D^m(x_t,t,c_{\\rm des}) - (\\omega-1)D^g(x_t,t,c_{\\rm ref})$, the same linear extrapolation as classifier-free guidance but with a classifier-chosen, time-adaptive reference. The paper reports that this consistently raises classification precision and, combined with Autoguidance on ImageNet 512, achieves a record FD_DINOv2 of 23.09 with 94.3% precision versus Autoguidance's 24.83 and 90.2%; on a 525-species fine-grained bird task it lifts precision from 5.3% (no guidance) to 32.3%.","pith_inferences":["If intermediate classifier confidence is a reliable signal, the same adaptive reference idea should work for any attribute with a pretrained predictor, not just class labels; one could steer style, sentiment, or even regression targets by defining a 'most confusing' alternative.","The confidence threshold $\\tau$ could double as a stopping rule: guidance is active only while the model is genuinely confused, which may reduce over-correction artifacts that plain CFG exhibits at high scales.","Because the headline Precision is measured with the same classifier family used for guidance, a stronger test would use a held-out classifier or human labels; the method's true fidelity gain may be smaller or larger than reported.","A testable scaling prediction is that better classifiers produce better guidance; varying classifier architecture and accuracy should move both FD_DINOv2 and precision monotonically."],"forward_implications":["Classifier guidance no longer requires a differentiable classifier or backpropagation, so any off-the-shelf classifier can steer generation at roughly the cost of one extra forward pass per guidance prediction.","GFCG layers onto existing gradient-free methods: in mixed mode it adds no additional function evaluations, and in additive mode it improves both fidelity and diversity at the cost of doubled evaluations.","On the paper's ImageNet 512 evaluation, GFCG combined with Autoguidance reaches FD_DINOv2 23.09 and 94.3% precision, beating Autoguidance's 24.83 and 90.2% under identical random seeds.","The same mechanism transfers to text-to-image models, where a fine-grained 525-species bird classifier lifts precision from 5.3% without guidance and 27.3% with CFG to 32.3% with GFCG."],"supporting_citations":[{"why":"Establishes gradient-based classifier guidance as the baseline that GFCG replaces with no backpropagation.","marker":"[6]"},{"why":"Introduces classifier-free guidance, whose contrast update form GFCG reuses with a classifier-chosen reference.","marker":"[10]"},{"why":"Introduces Autoguidance, the state-of-the-art baseline GFCG combines with and improves on for FD_DINOv2 and precision.","marker":"[16]"},{"why":"Supplies the class-conditional diffusion model and codebase on which the main ImageNet experiments run.","marker":"[17]"},{"why":"Defines the FD_DINOv2 metric used for the headline image-quality results.","marker":"[33]"},{"why":"Provides the limited-guidance-interval idea behind the ts hyperparameter for when GFCG starts.","marker":"[19]"}],"fun_headline_variants":["Gradient-free classifier guidance hits ImageNet fidelity record","No-backprop classifier guidance sets new ImageNet fidelity bar","Inference-only classifier steers diffusion to fidelity record","Classifier without gradients: diffusion fidelity record and higher precision"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that classifier probabilities computed on an intermediate denoised estimate of the image reliably indicate the final image's class content, so steering away from the top competing class improves the final sample; if those probabilities are noisy or biased, the guidance will optimize the classifier rather than the image.","fun_headline_variants_meta":{"raw":{"variants":["Gradient-free classifier guidance hits ImageNet fidelity record","No-backprop classifier guidance sets new ImageNet fidelity bar","Inference-only classifier steers diffusion to fidelity record","Classifier without gradients: diffusion fidelity record and higher precision"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000515,"raw_usage":{"total_tokens":2586,"prompt_tokens":1115,"completion_tokens":1471,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":731,"completion_tokens_details":{"reasoning_tokens":1407}},"tokens_in":731,"tokens_out":1471,"duration_ms":11200,"temperature":1.0,"reasoning_tokens":1407,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:21:54.625806+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the classifier's top competing class on intermediate denoised estimates with the actual failure mode of the final image as judged by a held-out classifier or human labels; if the intermediate top competitor frequently is not what the final image gets confused with, the steering target is wrong and GFCG should degrade. A direct control is to rerun GFCG with the classifier's softmax replaced by a fixed random reference class: if precision still rises, the classifier signal is not the cause.","supporting_citations":[{"cited_title":"Diffusion models beat GANs on image synthesis","cited_arxiv_id":null,"evidence_quote":"Establishes gradient-based classifier guidance as the baseline that GFCG replaces with no backpropagation."},{"cited_title":"Classifier-free diffusion guidance","cited_arxiv_id":null,"evidence_quote":"Introduces classifier-free guidance, whose contrast update form GFCG reuses with a classifier-chosen reference."},{"cited_title":"Analyzing and improving the training dynamics of diffusion models","cited_arxiv_id":null,"evidence_quote":"Supplies the class-conditional diffusion model and codebase on which the main ImageNet experiments run."},{"cited_title":"Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models","cited_arxiv_id":null,"evidence_quote":"Defines the FD_DINOv2 metric used for the headline image-quality results."}],"review_version":1}