{"id":"4efd62e1-875b-4042-9038-dfea9c7064ec","arxiv_id":"2412.00205","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A diffusion-sampling method estimates pixel-wise aleatoric uncertainty as the variance of denoising scores over re-noised copies and uses it to guide generation toward lower FID.","lead":"This paper proposes a training-free way to estimate pixel-level uncertainty in diffusion model images by re-noising and re-scoring the same sample multiple times, then using that uncertainty map to filter bad images and guide sampling. The method makes small FID improvements on ImageNet and CIFAR-10, but its theoretical link to the curvature of the noise distribution is not justified as stated.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 9 is a valid Fisher-information identity only for draws from the marginal q(Xt), but Algorithm 1 estimates variance under q(Xt|X̂0), so the curvature claim is unproven.","rationale":"I read the paper as making two distinct claims: (1) the proposed U_t estimates pixel-wise aleatoric uncertainty, and (2) this estimate is theoretically grounded as the curvature of the marginal noising distribution, which in turn justifies uncertainty-guided sampling in Eq. 10. The reader's weakest assumption identifies exactly the gap in claim (2): Algorithm 1 samples perturbed inputs from q(Xt|X̂0), while Eq. 9 requires draws from the marginal q(Xt). The Appendix A proof is a correct standard derivation of the Fisher information identity, but it is not connected to Algorithm 1. There is no importance weighting, no correction term, and no argument that the plug-in conditional variance converges to the marginal score variance. This is not a matter of disagreement with the community consensus; it is an internal gap between the stated theorem and the algorithm it is claimed to justify. I also note the paper's own Sec. 3.5 says 'we don't estimate the full variance-covariance matrix, but only the diagonal elements,' which is fine for the heuristic but does not repair the distribution mismatch. The empirical results are modest and, as the reader notes, include cases where the baseline wins, so the reject verdict is not undermined. If the theoretical claim were removed and the method presented purely as a heuristic, the paper might merit a different, weaker verdict; as written, the central claim is unsupported. I therefore agree with the reader and recommend no change to the verdict.","tokens_in":15813,"tokens_out":4047,"duration_ms":40952,"concrete_test":"Run a small numerical check that isolates the theory from the learned model. Define a known marginal q_t(x) as a Gaussian mixture, e.g. q_t = 0.5 N(-1,1) + 0.5 N(1,1), and compute the true Fisher information I = E_{q_t}[(d/dx log q_t(x))^2]. Then choose a fixed X̂0 (say -1) and, exactly as in Algorithm 1, draw M = 10,000 samples from q(·|X̂0) = N(√ᾱ X̂0, 1-ᾱ) for a typical variance schedule. Estimate U = Var(s(x_i)) with s(x) = d/dx log q_t(x). If U differs from I by more than Monte-Carlo tolerance, Eq. 9's approximation fails for the actual sampling distribution used by Algorithm 1. Repeat for several X̂0 values and for a learned-score analogue; the discrepancy should persist unless an unstated averaging over X̂0 is added and derived.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing theoretical step is the chain from Eq. 8 to Eq. 9 to Eq. 10. Equation 9 is the standard Fisher-information identity for a single distribution q(Xt): E_q[s s^T] = -E_q[∂² log q], where s = ∇ log q, valid when expectations are over draws from that same q. Algorithm 1, however, computes U_t as the sample variance of scores evaluated at X̂t^i = √ᾱ_t X̂0 + √(1-ᾱ_t) ε_i (Algorithm 1, line 4, using Eq. 2). Thus each X̂t^i is drawn from the conditional Gaussian q(·|X̂0), centered at the DDIM point estimate X̂0 from Eq. 7, not from the marginal q_t(Xt) = ∫ p_data(X0) q(Xt|X0) dX0. The proof in Appendix A proves only the Fisher identity under regularity conditions; it never shows that E_{q(·|X̂0)}[s(X)s(X)^T] equals the marginal Fisher information I(q_t). For a mixture density such as q_t = 0.5 N(-1,1) + 0.5 N(1,1), the variance of the score under one conditional component can differ substantially from the marginal Fisher information, so the missing step is not a technicality. Consequently, the asserted equivalence between U_t and the second derivative of the noising distribution is unsupported, and the rationale for the guidance update in Eq. 10, framed as maximizing the curvature, collapses. The empirical FID improvements may still indicate a useful heuristic, but the paper's central theoretical claim does not follow from the provided derivation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a training-free method to estimate pixel-wise aleatoric uncertainty during diffusion-model sampling. At each denoising step the algorithm builds a DDIM estimate X̂0 from the current state Xt, draws M Gaussian-perturbed samples around X̂0, runs the score network on these samples, and defines the uncertainty map as the diagonal of the sample variance of the resulting scores. The authors claim that this uncertainty approximates the curvature of the noising distribution, specifically the expected second derivative of log q(Xt), and they use the uncertainty map to guide sampling by applying gradient ascent to high-uncertainty pixels. Experiments on ImageNet and CIFAR-10 with ADM, U-ViT, DDPM, and Stable Diffusion compare the method with BayesDiff and MC-Dropout for low-quality-sample filtering, reconstruction uncertainty metrics (AUSE/AURG), and FID under guidance.","tokens_in":16237,"tokens_out":9358,"duration_ms":91564,"significance":"The idea of a training-free, scheduler-agnostic uncertainty signal that can both filter samples and steer generation is attractive, and the paper's strengths include a public code release, a broad set of models and datasets, and a favorable NFE comparison (20 vs. roughly 130 for BayesDiff). If the curvature interpretation were valid, the method would offer a principled and cheap way to improve sampling quality. However, the central theoretical equivalence is not established: the Fisher-information identity in Eq. (9) is proven for the marginal q_t, while Algorithm 1 estimates a conditional plug-in variance. In addition, the empirical gains are modest and inconsistent: BayesDiff attains a better FID on ImageNet256 in Table 1, and the guided-sampling improvements in Table 3 are about one FID point without error bars or multiple seeds. As it stands, the contribution is an ungrounded heuristic with limited evidence of practical benefit.","major_comments":[{"comment":"The main theoretical claim is not established. Appendix A proves the standard Fisher-information identity E_q[s s^T] = -E_q[∂² log q(s)] for draws from the marginal distribution q_t(X_t), but Algorithm 1 computes the sample variance of scores at points X̂_i^t drawn from the conditional Gaussian q(·|X̂0), where X̂0 is the DDIM point estimate from Eq. (7). No importance weighting, correction term, or additional argument shows that the plug-in conditional variance equals the marginal Fisher information. This is not a technicality: for a mixture such as q_t = 0.5 N(-1,1) + 0.5 N(1,1), the variance of the score under one conditional component can differ substantially from the marginal Fisher information. The centered sample variance in Eq. (8) also coincides with the uncentered second moment in Eq. (9) only if the mean score under the sampling distribution is zero, which is not guaranteed for the conditional plug-in. Consequently, the equivalence between U_t and the curvature of the noising distribution is unsupported, and the rationale for the guidance update in Eq. (10) collapses.","section":"Sec. 3.5, Eq. (9), Appendix A, Algorithm 1"},{"comment":"The guidance mechanism is not justified. The update ε̂_t = ε_t + λ(I[U_t > p] · ∂U_t/∂ε_t) treats ∂U_t/∂ε_t as a gradient-ascent direction, but U_t depends on ε_t through the DDIM estimate X̂0 in Eq. (7) and then through the M perturbed samples in Algorithm 1. The paper does not specify how this gradient is computed, nor does it show that stepping ε_t along this direction increases the curvature of q_t as claimed. The sentence in Sec. 3.6 that maximizing the second derivative improves the convergence rate and cites [33] is not supported by that reference, which is a statistics textbook rather than an optimization or diffusion result. As written, the guidance step is a heuristic whose stated theoretical motivation is not valid.","section":"Sec. 3.6, Eq. (10), Algorithm 2"},{"comment":"The empirical evidence for the central practical claims is weak. In Table 3, the FID improvements from uncertainty guidance are about 0.9–1.1 points on 10,000 generated samples, with no standard errors, confidence intervals, or multiple seeds; FID differences of this size are within the sampling noise of the metric. In Table 1, the proposed filtering method is not uniformly better than the baseline: BayesDiff achieves a clearly lower FID on ImageNet256 (6.81 vs. 7.80), and several of the reported improvements are very small (e.g., 3.289 vs. 3.254 on ImageNet64). The abstract's claim of 'state-of-the-art performance' is therefore not supported by the reported numbers, and the guided-sampling result lacks a comparison to a similarly priced baseline with the same compute budget.","section":"Table 3 and Table 1"},{"comment":"The guidance results depend on several hand-chosen hyperparameters—M = 5, p = 95, λ = 1.0, and the uncertainty timestep window (steps 45–48)—but no sensitivity analysis is provided for the ImageNet or CIFAR-10 benchmarks. The only ablation (Fig. 10) is qualitative and on Stable Diffusion, where changes in p and λ can substantially alter image content. Since the method is proposed as a general guidance technique, the lack of ablations on the benchmark datasets makes it unclear whether the small FID gains are robust or are the result of tuned settings.","section":"Sec. 4.1 and Fig. 10"}],"minor_comments":[{"comment":"The reparametrization is written as X_t = √ᾱ_t X0 + (1-ᾱ_t) ε; it should be X_t = √ᾱ_t X0 + √(1-ᾱ_t) ε. Equation (1) also appears to define the conditional distribution of X_t given X_{t-1}, not X_{t-1} given X_t, as the notation suggests.","section":"Sec. 3.2, Eq. (2)"},{"comment":"The text says the uncertainty estimation results are presented in Table 4, but the results described (filtering FID and comparisons with BayesDiff/MC-Dropout) appear in Table 1; Table 4 in the appendix reports Precision/Recall. Please correct the cross-reference.","section":"Sec. 4.2"},{"comment":"The use of the percentile threshold is ambiguous. Algorithm 2 uses a per-image threshold 'percentile(U_t, p)', while Sec. 4.1 describes a global 95-th percentile 'of the uncertainty computed over the 10,000 samples'. Please clarify which implementation was used and whether p is the same in both places.","section":"Algorithm 2 and Sec. 4.1"},{"comment":"The proof is stated for a scalar x and assumes conditions (15)–(16), but the paper uses vector-valued image tensors and a diagonal of the full covariance matrix. The vector case and the boundary decay of ∂q/∂x at infinity are not discussed, so the proof as written does not directly cover the setting of the main text.","section":"Appendix A"},{"comment":"There are numerous presentation issues: 'threhsolds' in Algorithm 2, 'V AEs' in the Related Work, inconsistent capitalization such as 'Imagenet64' vs. 'ImageNet64' and 'U-VIT' vs. 'U-ViT', and Table 1's caption says 'with and without uncertainty guidance' when the experiment is actually low-quality filtering.","section":"Throughout"}],"recommendation":"reject","confidential_remarks":"The load-bearing problem is the distribution mismatch in the central theoretical claim: Eq. (9) requires samples from the marginal q_t, but Algorithm 1 uses the conditional plug-in q(·|X̂0). This is not a local proof gap that a revision can patch within the current scope; it invalidates the paper's main justification. A possible path for the authors would be to resubmit a version that explicitly reframes the method as an empirical heuristic, removes or substantially weakens the curvature claims, and provides rigorous ablations and error bars for the guidance results. The ImageNet256 result where BayesDiff outperforms the proposed method should also be addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Good to see this preprint. The core idea is clean: at each sampling step, take the DDIM prediction of the clean image, re-noise it M times, and compute the variance of the denoising scores over those re-noised copies. That gives a pixel-wise uncertainty map without retraining, and it costs far fewer function evaluations than BayesDiff. The guidance update—pushing the score along the gradient of the high-uncertainty pixels—is also new, and the qualitative results on Stable Diffusion are genuinely suggestive. The paper is honest about some limitations, e.g. the failure case with hands in Fig. 13.\n\nWhere it falls apart is the theoretical justification. The claim in Sec. 3.5 is that the score variance approximates the Fisher information of the marginal noising distribution q_t(X_t). That identity (Eq. 9) is standard, and the Appendix proves it correctly for a single distribution. But Algorithm 1 computes the variance of scores evaluated at X̂_t^i ~ q(· | X̂0), the conditional Gaussian centered at the DDIM point estimate, not at draws from the marginal q_t(X_t). The Appendix never bridges that gap, and the stress-test example with a mixture shows it isn't a technicality: the conditional score variance can be far from the marginal Fisher information. So the curvature interpretation and the 'maximizing the second derivative' rationale for guidance are unsupported. What remains is a plausible heuristic, and the empirical FID gains (about 1 point on guidance, consistent across datasets) are consistent with a heuristic that happens to work.\n\nThe experiments have their own soft spots: no error bars or significance tests, the ImageNet256 row in Table 1 actually favors BayesDiff, and the guidance experiments in Table 3 compare only against 'normal' sampling rather than a standard guidance baseline. The overclaim in the abstract ('state-of-the-art') doesn't match the paper's own tables.\n\nStill, this deserves a serious referee. The gap in the literature is real—no other training-free pixel-wise uncertainty estimator for diffusion models also steers sampling—and the empirical evidence, while not decisive, warrants a second look. A referee should ask for a rewritten theory section that either fixes the marginal-vs-conditional gap or explicitly reframes the method as a heuristic, and for stronger baselines with error bars. I'd lean toward rejecting the current version but inviting a resubmission.","headline":"Genuinely new training-free uncertainty estimator for diffusion sampling, but the theoretical curvature claim does not survive contact with Algorithm 1.","tokens_in":16702,"tokens_out":2280,"would_cite":false,"duration_ms":20913,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pixel-wise score variance doubles as a diffusion model's quality map and steering signal.","keywords":["diffusion models","aleatoric uncertainty","pixel-wise uncertainty estimation","uncertainty-guided sampling","score-based generative modeling","Fisher information","FID","DDIM"],"falsifier":"On a Gaussian mixture with a known closed-form score, compute the exact marginal Fisher information $-\\mathbb{E}[\\partial^2\\log q_t(X_t)/\\partial X_t^2]$ and compare it with Algorithm 1's $U_t$ at the same time steps. If $U_t$ deviates systematically from the marginal curvature, especially when the DDIM reconstruction $\\hat{X}_0$ lands far from any clean sample, the Eq. 9 interpretation is refuted.","tokens_in":15578,"feed_emoji":"🖼️","tokens_out":11700,"duration_ms":92164,"temperature":0.7,"pith_summary":"The paper proposes a training-free way to read pixel-wise aleatoric uncertainty out of a diffusion model while it samples: at each denoising step, predict the clean image, renoise it $M$ times, and take the variance of the model's denoising scores across those perturbed inputs. It then shows that, under regularity conditions, this variance equals the expected second derivative of the noising distribution's log-density, so each uncertainty map is a curvature map of the diffusion process. The same map guides sampling: pixels above a percentile threshold get their scores pushed along the gradient of the uncertainty, and the authors report that this improves FID by about one point on ImageNet and CIFAR-10 and also filters out low-quality samples. The practical value is that diffusion models gain a cheap, quantitative quality signal and a steering mechanism without retraining or extra networks.","feed_headline":"Uncertainty maps steer diffusion sampling to better FID","feed_subtitle":"Variance of denoising scores acts as curvature of the noise distribution, and guiding on it cuts FID by about one point.","key_machinery":"The load-bearing object is Algorithm 1's uncertainty map: starting from the current noisy image $X_t$, compute the DDIM predicted clean image $\\hat{X}_0$, draw $M$ noisy samples $\\hat{X}_t^i \\sim q(X_t|\\hat{X}_0)$, evaluate the denoising network on each, and set $U_t$ to the diagonal variance of the stacked scores. Algorithm 2 turns that map into a guidance update $\\hat{\\varepsilon}_t = \\varepsilon_t + \\lambda\\,\\mathbb{I}[U_t > p]\\cdot \\partial U_t/\\partial \\varepsilon_t$, where $\\mathbb{I}$ is the indicator of pixels above the $p$-th percentile threshold and $\\lambda$ is the update strength. The identity connecting the two is Eq. 9, the Fisher-information relation between score variance and curvature; it licenses reading $U_t$ as second-order information about the noising distribution and gives the guidance update its stated rationale.","core_discovery":"The central claim is that the diagonal variance of the scores $\\varepsilon_\\theta(\\hat{X}_t^i,t)$ over $M$ Monte-Carlo renoised versions of the DDIM-predicted clean image, $U_t = \\mathrm{diag}(\\mathrm{Var}_i[\\varepsilon_\\theta(\\hat{X}_t^i,t)])$, approximates the expected outer product of the score, $\\mathbb{E}[\\nabla_{X_t}\\log q_t(X_t)\\,\\nabla_{X_t}\\log q_t(X_t)^\\top]$, and by the Fisher-information identity this equals the negative expected Hessian $-\\mathbb{E}[\\partial^2 \\log q_t(X_t)/\\partial X_t\\partial X_t^\\top]$. In the paper's own terms, $U_t$ is therefore a Monte-Carlo estimate of the curvature of the marginal noising distribution $q_t(X_t)=\\int p_{\\mathrm{data}}(X_0)q(X_t|X_0)\\,dX_0$, with high values marking pixels the model finds hard to reconstruct consistently. The authors further claim that gradient ascent on $U_t$ at pixels above a percentile threshold steers the reverse process toward higher-quality images, and their experiments report consistent FID gains across resolutions and model families.","pith_inferences":["We infer that the curvature reading depends on the plug-in conditional $q(X_t|\\hat{X}_0)$ standing in for the true marginal $q_t(X_t)$; a direct comparison of $U_t$ with the exact Fisher information on a synthetic density would be the natural test.","The update rule differentiates $U_t$ with respect to the score $\\varepsilon_t$, a heuristic move not derived from Eq. 9; ablating it against a simpler high-uncertainty score scaling would isolate what the gradient term contributes.","If the curvature interpretation survives that test, the same map could support other uses not explored here, such as artifact localization, adaptive compute allocation, or quality control in safety-critical generation.","The reported FID gains are about one point, and the appendix shows that large guidance strengths or low percentiles change scene content, so the working range of $p$ and $\\lambda$ needs calibration per model."],"forward_implications":["A diffusion model can report where it is uncertain at the final denoising steps for 20 extra score evaluations in the paper's setup ($M=5$ over four steps), compared with about 130 for the existing pixel-wise Bayesian baseline.","Filtering the generated pool using the uncertainty score improves FID over random selection on ImageNet and CIFAR-10, providing a practical low-quality filter.","Guiding the sampler on high-uncertainty pixels improves FID by about one point across ADM, U-ViT, and DDPM at resolutions from 64 to 512.","The method transfers across samplers such as DDIM and second-order DPM solvers, and to text-to-image models, with qualitative results showing fewer artifacts.","Uncertainty concentrates in foreground elements and in the late denoising steps, so the maps localize the pixels the model struggles to synthesize."],"supporting_citations":[{"why":"Supplies the sensitivity-as-uncertainty proxy that motivates the perturbation scheme.","marker":"[35]"},{"why":"Provides the DDIM predicted-clean-image formula used in Algorithm 1 and the sampler used in the experiments.","marker":"[44]"},{"why":"Establishes the score-gradient relation that connects score variance to the curvature of the noising distribution.","marker":"[47]"},{"why":"Supply the regularity conditions and the Fisher-information identity used in Eq. 9.","marker":"[12, 33]"},{"why":"The existing pixel-wise diffusion uncertainty baseline whose function-evaluation overhead and FID results are compared against.","marker":"[30]"},{"why":"Provides the high-resolution backbone used for the ImageNet64 and ImageNet128 experiments.","marker":"[10]"},{"why":"Defines the noising and denoising distributions and the score-matching setup on which the method is built.","marker":"[21]"},{"why":"Defines the FID metric used for all quality and filtering comparisons.","marker":"[19]"},{"why":"Provides the text-to-image model used for qualitative generalization tests.","marker":"[41]"}],"fun_headline_variants":["Pixel-wise uncertainty guides diffusion to lower FID","Score variance as curvature steers diffusion sampling","Aleatoric maps sharpen diffusion FID gains","Uncertainty-guided diffusion improves FID scores","Variance of denoising scores betters diffusion FID"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The identity in Eq. 9 is argued for the marginal noising distribution $q_t(X_t)$, but Algorithm 1 computes its variance from samples of the conditional distribution $q(X_t|\\hat{X}_0)$ built from one DDIM-predicted clean image, and the paper gives no argument that the plug-in conditional variance equals the marginal Fisher information.","fun_headline_variants_meta":{"raw":{"variants":["Pixel-wise uncertainty guides diffusion to lower FID","Score variance as curvature steers diffusion sampling","Aleatoric maps sharpen diffusion FID gains","Uncertainty-guided diffusion improves FID scores","Variance of denoising scores betters diffusion FID"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00016,"raw_usage":{"total_tokens":1235,"prompt_tokens":948,"completion_tokens":287,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":214}},"tokens_in":564,"tokens_out":287,"duration_ms":3085,"temperature":1.0,"reasoning_tokens":214,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:37:34.808887+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a Gaussian mixture with a known closed-form score, compute the exact marginal Fisher information $-\\mathbb{E}[\\partial^2\\log q_t(X_t)/\\partial X_t^2]$ and compare it with Algorithm 1's $U_t$ at the same time steps. If $U_t$ deviates systematically from the marginal curvature, especially when the DDIM reconstruction $\\hat{X}_0$ lands far from any clean sample, the Eq. 9 interpretation is refuted.","supporting_citations":[{"cited_title":"In: Proceedings of the AAAI Conference on Artificial Intelligence","cited_arxiv_id":null,"evidence_quote":"Supplies the sensitivity-as-uncertainty proxy that motivates the perturbation scheme."},{"cited_title":"In: International Conference on Learning Representations (2020) 1, 4, 5, 7","cited_arxiv_id":null,"evidence_quote":"Provides the DDIM predicted-clean-image formula used in Algorithm 1 and the sampler used in the experiments."},{"cited_title":"In: In- ternational Conference on Learning Representations (2021), https://openreview.net/forum? id=PxTIG12RRHS 4, 5, 6","cited_arxiv_id":null,"evidence_quote":"Establishes the score-gradient relation that connects score variance to the curvature of the noising distribution."},{"cited_title":"Advances in neural information processing systems 34, 8780–8794 (2021) 1, 3, 6","cited_arxiv_id":null,"evidence_quote":"Provides the high-resolution backbone used for the ImageNet64 and ImageNet128 experiments."},{"cited_title":"Advances in neural information pro- cessing systems 33, 6840–6851 (2020) 1, 3, 6","cited_arxiv_id":null,"evidence_quote":"Defines the noising and denoising distributions and the score-matching setup on which the method is built."},{"cited_title":"Ad- vances in neural information processing systems 30 (2017) 6, 7","cited_arxiv_id":null,"evidence_quote":"Defines the FID metric used for all quality and filtering comparisons."},{"cited_title":"In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","cited_arxiv_id":null,"evidence_quote":"Provides the text-to-image model used for qualitative generalization tests."}],"review_version":1}