REVIEW 4 major objections 5 minor 23 references
For inverse problems, diffusion solvers should target the local mode of the posterior at each denoising step, not the posterior mean or a full posterior sample.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-04 11:10 UTC pith:WAQA6TXQ
load-bearing objection Useful optimization-based solver with an over-sold 'local MAP' story; the Gaussian covariance approximation collapses the mode-seeking mechanism, but the empirical breadth and JPEG/quantization gains keep it worth refereeing. the 4 major comments →
Local MAP Sampling for Diffusion Models
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that the iterative procedure behind many optimization-based diffusion solvers should be understood as solving local maximum-a-posteriori subproblems, x*_0(t, x_t, y) = arg max p(x0 | xt, y), rather than as posterior sampling or an approximation to global MAP. The paper shows this local MAP step coincides with posterior-mean sampling only when the conditional posterior is Gaussian; otherwise it introduces a mode-seeking bias that avoids between-mode and low-density reconstructions. From this perspective, Tweedie Moment Projected Diffusion and methods like DiffPIR, DCDP and Resample emerge as special cases with particular covariance choices. For practical use, the paper pr
What carries the argument
The load-bearing object is the local MAP subproblem combined with the Gaussian approximation of the diffusion-model posterior p(x0 | xt). The paper models p(x0 | xt) as N(x0; m_{0|t}, Sigma_{0|t}), then replaces the true conditional covariance with a scalar multiple of the inverse signal-to-noise ratio, Sigma ≈ (k/SNR) I, which turns the local MAP objective into a weighted sum of a prior-matching term and a data-consistency term. A reformulation of that objective into a convex combination with weights mu_t = sigma_t^2/(sigma_t^2 + k1^2) gives an automatically annealed, numerically stable cost that is minimized by a few gradient steps at each diffusion step. The gradient step for non-differen
Load-bearing premise
The framework assumes that the conditional posterior p(x0 | xt) can be approximated by a Gaussian with isotropic covariance, and that the local MAP of that Gaussian—which equals the conditional mean—captures the mode-seeking intent; if that approximation is poor, the objective is not actually the local MAP and the probabilistic interpretation breaks down.
What would settle it
Construct a two-component Gaussian-mixture prior (as in the paper's toy example) with well-separated modes and run LMAPS on a linear measurement. If the isotropic Gaussian approximation forces the per-step 'local MAP' to land on the posterior mean of the mixture rather than on the higher-peak mode, the method will fail to exhibit the mode-seeking behaviour that distinguishes local MAP from DPS; check the fraction of trajectories whose final reconstruction lies inside the high-density region of a single mixture component.
If this is right
- Optimization-based diffusion solvers that alternate denoising and data consistency now have a unified probabilistic reading: each inner loop solves a local MAP problem with a specific covariance choice.
- Because local MAP equals posterior-mean sampling only in Gaussian conditionals, LMAPS is expected to behave differently from DPS precisely on nonlinear, non-differentiable, or mixture-like inverse problems—where it reports its largest gains.
- The isotropic covariance approximation yields a two-parameter (k1, k2) objective whose weights anneal automatically from measurement-driven early steps to prior-driven late steps, removing the need for hand-tuned lambda_t schedules.
- Surrogate gradients allow the same local MAP framework to handle non-differentiable forward operators, extending diffusion-based restoration to JPEG and quantization tasks that previously required specialized methods.
- On scientific benchmarks (linear inverse scattering, CS-MRI, black-hole imaging) the reconstruction improves by more than 1.5 dB PSNR on the scattering tasks.
Where Pith is reading between the lines
- If the local-MAP view is taken literally, LMAPS is best understood not as a sampler but as a deterministic reconstruction algorithm: running it with different seeds explores different trajectories, and the spread of outputs reflects trajectory instability rather than calibrated posterior uncertainty.
- The isotropic covariance assumption is most questionable at intermediate noise levels where the true conditional covariance is anisotropic; a natural extension is to replace k/SNR with a diagonal or low-rank covariance estimated cheaply from the denoiser Jacobian, preserving the mode-seeking behavior with a better-conditioned objective.
- The surrogate-gradient trick for non-differentiable operators suggests a general recipe: any forward model whose evaluation is available but whose Jacobian is not can be plugged into LMAPS by pairing it with a differentiable approximation, which may apply to other discrete corruption models such as masking with unknown masks or learned compression codecs.
- Because global MAP remains open and LMAPS does not converge to it, the framework points toward a two-stage reconstruction: a mode-seeking local-MAP pass to land in a high-density region followed by a local refinement that maximizes the full posterior p(x0|y).
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Local MAP Sampling (LMAPS), a diffusion-based inverse problem solver that, at each reverse step, replaces the conditional-mean denoiser by a point estimate obtained from the local MAP problem p(x0|xt,y). The authors derive an objective using a Gaussian approximation of p(x0|xt) with an isotropic covariance proportional to k/SNR, reformulate it into a scale-invariant convex combination of prior and data terms (Eq. 13), and provide a surrogate gradient for non-differentiable forward models. They claim LMAPS unifies TMPD and optimization-based solvers, is mode-seeking relative to DPS, and achieves state-of-the-art PSNR/SSIM/LPIPS on 10 image restoration and 3 scientific inverse problems.
Significance. The framework is appealing and the experiments are extensive, with substantial reported gains on several benchmarks. The analytical toy example (Appendix A) clearly illustrates the conceptual difference between posterior mean and local MAP for a Gaussian mixture. If the method truly implemented mode-seeking local MAP, it would be a valuable contribution to the ongoing discussion about MAP versus posterior sampling in diffusion models. However, as presented, the practical algorithm does not implement the mode-seeking property it claims: the Gaussian prior approximation removes the multi-modality that motivates the MAP/mean distinction, and for linear H the local MAP reduces to the conditional mean. The empirical results are therefore not evidence for the mode-seeking mechanism. The authors also do not provide code or a sensitivity analysis for the new hyperparameters, which limits reproducibility and the strength of the SOTA claims.
major comments (4)
- [Sec. 3.2 vs Secs. 4.1–4.2] The central theoretical claim is that LMAPS introduces a mode-seeking bias that distinguishes it from DPS. The paper itself states (Sec. 3.2) that local MAP equals the conditional mean whenever p(x0|xt,y) is Gaussian, and that this happens 'in linear-Gaussian inverse problems with a Gaussian diffusion prior approximation.' In the practical algorithm (Sec. 4.1), p(x0|xt) is approximated by N(m0|t, (k/SNR_t)I). For any linear H, the resulting posterior is Gaussian, so the objective solved in Eq. (13) yields exactly the conditional mean. Algorithm 4 therefore solves a reweighted quadratic objective, not a mode-seeking MAP. For nonlinear H, the approximation is a single Gaussian prior, so the solved objective is not the true local MAP of the multimodal diffusion prior. The toy example in Appendix A cannot be reproduced by Algorithm 4 because the mixture is replaced by its first two moments.
- [Secs. 4.1–4.2, Eqs. (12)–(15)] The paper's unification claim is largely definitional. Eq. (13) is explicitly acknowledged to be equivalent to the existing optimization objective Eq. (14) up to the weighting schedule, and Eq. (15) is the closed-form posterior mean of a Gaussian-linear model, not a MAP estimator with mode-seeking behavior. Under the isotropic covariance assumption, the local MAP solution for linear H is exactly the conditional mean. Thus the 'local MAP' framework is an interpretation of a known objective, and the novel content reduces to the specific k1,k2 weighting, the reformulation, and the surrogate gradient. The paper should clearly separate the new algorithmic contribution from the re-description of existing methods, and avoid overclaiming the unification.
- [Table 4, Sec. 5] The method introduces two new hyperparameters (k1,k2) in addition to learning rate and step counts; Table 4 shows these are set separately for each of the 13 tasks, with no sensitivity analysis, selection protocol, or cross-validation. In the absence of such analysis, the reported 'state-of-the-art' results on these same benchmarks may reflect tuned hyperparameters rather than a general advantage of the framework. The abstract also states LMAPS 'achieves the best results in 46/60' cases, while Sec. 5.2 reports '49 out of 60'; this discrepancy needs to be resolved if the empirical headline is to be credible.
- [Sec. 4.3, Eq. (18)] For JPEG restoration and quantization, the surrogate operator is H' = I, so the gradient is approximated by 2(H(x0)-y). This is a straight-through estimator; the assumption that H(x0) ~ H'(x0) is not satisfied for quantization, where H is a step function. The paper reports >2 dB gains on JPEG and quantization tasks but does not validate the surrogate approximation. A justification or an ablation showing the contribution of this surrogate is needed before these results can be attributed to the LMAPS framework.
minor comments (5)
- [Section 7] There is a typo: 'that that iteratively' should be 'that iteratively'.
- [Table 3] The table header uses 'MAPS' instead of 'LMAPS'; please correct.
- [Table 4 and Sec. C.4] Two typos: '0..01' should be '0.01', and 'lost of hyper-paramers' should be 'list of hyperparameters'.
- [Table 2 vs Table 6] For DPS on CS-MRI 8x noiseless, Table 2 reports 20.82 dB while Table 6 reports 22.82 dB. Please reconcile the inconsistency.
- [Sec. 4.1 / Algorithm 4] The notation r in Algorithm 4 includes a 10^-6 term in the denominator, but this is not mentioned in Eq. (13). Clarify whether this is a numerical epsilon or part of the weighting.
Circularity Check
Under the paper's own Gaussian covariance ansatz, the implemented objective is equivalent to the existing optimization-based objective Eq. (14), and the local-MAP/mode-seeking distinction from DPS is defined away; the empirical SOTA is therefore not a validation of the local-MAP theory.
specific steps
-
self definitional
[Sec. 3.2 ('When are DPS and local MAP equivalent?') and Sec. 4.1 ('Isotropic approximation of Σ0|t'), Eq. (12)]
"These two coincide if and only if E[x0|xt, y] = arg max p(x0|xt, y), for example if p(x0|xt, y) is (uni-variate or multi-variate) Gaussian. ... we can proceed as in previous work [Boys et al., 2023, Song et al., 2023b] by projecting onto the first two moments using a Gaussian approximation, p(x0|xt) ≈ N(x0; m0|t, Σ0|t) ... we assume Σ0|t ≈ k/SNR I. Then x∗0 in Eq. (11) is reduced to: x∗0 = arg min SNR/k ∥x0 − m0|t∥2 + 1/σ2y ∥y − H(x0)∥2."
The paper's own equivalence condition says DPS and local MAP coincide whenever the conditional posterior is Gaussian. Sec. 4.1 then makes that condition true for the implemented method by assuming p(x0|xt) is Gaussian and Σ0|t ≈ (k/SNR)I; for any linear H, p(x0|xt,y) is Gaussian, so mode = mean. Algorithm 4 therefore solves the same conditional-mean-type objective that Sec. 3.2 identifies as non-mode-seeking. The claimed local-MAP/mode-seeking mechanism is defined into equivalence with DPS by the covariance ansatz, so the reported gains are evidence about the regularized objective Eq. (13)/(14), not about the local-MAP theory.
-
renaming known result
[Sec. 4.1 ('Relationship to optimization-based methods'), Eq. (14) vs Eq. (13)]
"Previous optimization-based approaches [Song et al., 2023c, Li et al., 2024, Zhu et al., 2023] solve for x∗0 through the following objective: x∗0 = arg min ∥x0 − m0|t∥2 + λt ∥y − H(x0)∥2, ... These methods can be viewed as special cases of our framework by setting Σ0|t = λt σ2y I in Eq. (11)."
The 'unification' of optimization-based methods under local MAP is a renaming by construction: Eq. (13) is exactly Eq. (14) with weights reparameterized via 2k2/k1^2 = k/(α_t^2 σ_y^2), and any objective of this form can be labeled 'local MAP' by choosing an isotropic covariance Σ0|t. No property of the true diffusion posterior is used to single out this objective; the framework reduces to the known optimization objective by definition of the free covariance/weight parameters.
full rationale
The paper contains real, externally benchmarked experiments, and I am not treating the empirical comparisons themselves as circular. The TMPD recovery in Eq. (15) is a genuine closed-form derivation, and there is no load-bearing self-citation chain: DAPS is by a different author group, and the authors' own [Zhang et al., 2025b] appears only in related work. Per-task k1,k2 choices in Table 4 limit the parameter-free status of the empirical claims, but that is a standard hyperparameter-tuning concern rather than derivation circularity. However, the central theoretical contribution—local MAP as a mode-seeking alternative to DPS—collapses by the paper's own equations: Sec. 3.2 states DPS and local MAP coincide for Gaussian conditionals, and Sec. 4.1 adopts a Gaussian prior approximation plus isotropic covariance, placing Algorithm 4 in exactly that Gaussian regime for linear H. Simultaneously, the claimed unification of optimization-based solvers is an explicit equivalence between Eq. (13) and Eq. (14) with a reparameterized weight schedule. These two reductions mean the 'local MAP' label is largely definitional, and the reported SOTA is not a test of the mode-seeking mechanism. This is partial circularity of the central claim, not a fully fabricated derivation, hence score 6. I also note the abstract's '46/60' vs. Sec. 5.2's '49/60' discrepancy as a reporting inconsistency, but that is not a circularity issue.
Axiom & Free-Parameter Ledger
free parameters (4)
- k1 (trust-region parameter, per task) =
e.g., 0.15 (SR), 0.5 (Box inpaint), 10 (phase retrieval), 0.05 (BH 3%)
- k2 (data-consistency scale, per task) =
e.g., 20 (SR), 100 (Gaussian deblur), 0.3 (phase retrieval), 0.05 (BH 3%)
- learning rate eta (per task) =
0.02–1 depending on task (Table 4)
- number of diffusion steps and gradient steps (N, K) =
e.g., N=200, K=100 for most tasks; N=100, K=200 for black hole
axioms (5)
- domain assumption p(x0 | xt) is approximated by a Gaussian N(x0; m_0|t, Σ_0|t)
- ad hoc to paper Isotropic covariance Σ_0|t ≈ k/SNR I
- domain assumption Tweedie denoiser m_0|t(xt) equals the true conditional mean E[x0|xt]
- ad hoc to paper For non-differentiable H, a differentiable surrogate H' with H(x0) ≈ H'(x0) exists and the error is negligible
- domain assumption Measurement noise is Gaussian: y = H(x0) + z, z ~ N(0, σ_y² I)
read the original abstract
Diffusion Posterior Sampling (DPS) provides a principled Bayesian approach to inverse problems by sampling from $p(x_0 \mid y)$. While posterior sampling is valuable for capturing uncertainty and multi-modality, many classical and practical inverse problem settings ultimately prioritize accurate point estimation -- most notably the MAP estimator, which has long served as a standard reconstruction objective in imaging and scientific applications. We introduce Local MAP Sampling (LMAPS), a new inference framework that iteratively solves local MAP subproblems along the diffusion trajectory. This perspective clarifies their connection to global MAP and DPS, offering a unified probabilistic interpretation for optimization-based methods. Building on this foundation, we develop practical algorithms with a covariance approximation motivated by a Gaussian prior assumption, and a reformulated objective for stability and interpretability. Across a broad set of image restoration and scientific tasks, LMAPS achieves state-of-the-art performance.
Figures
Reference graph
Works this paper leans on
-
[1]
Diffusion posterior sampling for general noisy inverse problems
Hyungjin Chung, Jeongsol Kim, Michael T Mccann, Marc L Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. arXiv preprint arXiv:2209.14687,
-
[5]
Solv- ing inverse problems with latent diffusion models via hard data consistency
Bowen Song, Soo Min Kwon, Zecheng Zhang, Xinyu Hu, Qing Qu, and Liyue Shen. Solv- ing inverse problems with latent diffusion models via hard data consistency. arXiv preprint arXiv:2307.08123, 2023c. Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, and Luc Van Gool. Denoising diffusion models for plug-and-play image restoration...
-
[6]
Decoupled data consistency with diffusion purification for image restoration
Xiang Li, Soo Min Kwon, Shijun Liang, Ismail R Alkhouri, Saiprasad Ravishankar, and Qing Qu. Decoupled data consistency with diffusion purification for image restoration. arXiv preprint arXiv:2403.06054,
-
[9]
A variational perspective on solving inverse problems with diffusion models
Morteza Mardani, Jiaming Song, Jan Kautz, and Arash Vahdat. A variational perspective on solving inverse problems with diffusion models. arXiv preprint arXiv:2305.04391,
-
[11]
A general framework for inference-time scaling and steering of diffusion models
Raghav Singhal, Zachary Horvitz, Ryan Teehan, Mengye Ren, Zhou Yu, Kathleen McKeown, and Rajesh Ranganath. A general framework for inference-time scaling and steering of diffusion models. arXiv preprint arXiv:2501.06848,
-
[12]
Guided flows for generative modeling and decision making
Qinqing Zheng, Matt Le, Neta Shaul, Yaron Lipman, Aditya Grover, and Ricky TQ Chen. Guided flows for generative modeling and decision making. arXiv preprint arXiv:2311.13443,
-
[13]
Measurement-aligned flow for inverse problem
Shaorong Zhang, Rob Brekelmans, Yunshu Wu, and Greg Ver Steeg. Measurement-aligned flow for inverse problem. arXiv preprint arXiv:2506.11893, 2025b. Luhuan Wu, Brian Trippe, Christian Naesseth, David Blei, and John P Cunningham. Practical and asymptotically exact conditional sampling in diffusion models. Advances in Neural Information Processing Systems, ...
-
[14]
Particle denoising diffusion sampler
Angus Phillips, Hai-Dang Dau, Michael John Hutchinson, Valentin De Bortoli, George Deligianni- dis, and Arnaud Doucet. Particle denoising diffusion sampler. arXiv preprint arXiv:2402.06320,
-
[15]
Sitcom: Step-wise triple-consistent diffusion sampling for inverse prob- lems
Ismail Alkhouri, Shijun Liang, Cheng-Han Huang, Jimmy Dai, Qing Qu, Saiprasad Ravishankar, and Rongrong Wang. Sitcom: Step-wise triple-consistent diffusion sampling for inverse prob- lems. arXiv preprint arXiv:2410.04479,
-
[16]
Manifold preserving guided diffusion
Yutong He, Naoki Murata, Chieh-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka, Dongjun Kim, Wei- Hsiang Liao, Yuki Mitsufuji, J Zico Kolter, Ruslan Salakhutdinov, et al. Manifold preserving guided diffusion. arXiv preprint arXiv:2311.16424,
-
[17]
Guidance with spherical gaussian constraint for conditional diffusion
Lingxiao Yang, Shutong Ding, Yifan Cai, Jingyi Yu, Jingya Wang, and Ye Shi. Guidance with spherical gaussian constraint for conditional diffusion. arXiv preprint arXiv:2402.03201,
-
[18]
Classifier-free diffusion guidance
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598,
-
[19]
Rethinking diffusion posterior sampling: From conditional score estimator to maximizing a posterior
Tongda Xu, Xiyan Cai, Xinjie Zhang, Xingtong Ge, Dailan He, Ming Sun, Jingjing Liu, Ya-Qin Zhang, Jian Li, and Yan Wang. Rethinking diffusion posterior sampling: From conditional score estimator to maximizing a posterior. arXiv preprint arXiv:2501.18913,
-
[20]
Inverse problems with diffusion models: A map estimation perspective
Sai Bharath Chandra Gutha, Ricardo Vinuesa, and Hossein Azizpour. Inverse problems with diffusion models: A map estimation perspective. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (W ACV), pages 4153–4162. IEEE,
2025
-
[21]
fastmri: An open dataset and benchmarks for accelerated mri
Jure Zbontar, Florian Knoll, Anuroop Sriram, Tullie Murrell, Zhengnan Huang, Matthew J Muckley, Aaron Defazio, Ruben Stern, Patricia Johnson, Mary Bruno, et al. fastmri: An open dataset and benchmarks for accelerated mri. arXiv preprint arXiv:1811.08839,
-
[23]
We model quantization by discretizing the measurement into2 nbits uniformly spaced levels
Quantization. We model quantization by discretizing the measurement into2 nbits uniformly spaced levels. Formally, the forward operator is defined as H(x) = ⌊x·(2 nbits −1) + 0.5⌋ 2nbits −1 ,(32) which rounds the inputxto the nearest quantization level. In this work, we focus on the challenging case of 2-bit quantization, where only four distinct measurem...
2024
-
[256]
NFE refers to diffusion timesteps, while optimization steps refer to inner loop optimizations in respective methods
The non-parallel single- image sampling time on the FFHQ 256 dataset using one NVIDIA A6000 GPU. NFE refers to diffusion timesteps, while optimization steps refer to inner loop optimizations in respective methods. Configuration ODE Steps Optimization Steps NFE Second/Image PSNR DAPS 5 100 200 110 29.19 SITCOM – 30 600 73 29.93 DPS – – 1000 138 25.87 MAPS ...
2025
-
[2018]
Feynman-kac correc- tors in diffusion: Annealing, guidance, and product of experts
Marta Skreta, Tara Akhound-Sadegh, Viktor Ohanesian, Roberto Bondesan, Al ´an Aspuru-Guzik, Arnaud Doucet, Rob Brekelmans, Alexander Tong, and Kirill Neklyudov. Feynman-kac correc- tors in diffusion: Annealing, guidance, and product of experts. arXiv preprint arXiv:2503.02819,
-
[2019]
Imagenet: A large-scale hi- erarchical image database
10 Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee,
2009
-
[2022]
Pseudoinverse-guided diffusion models for inverse problems
Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Pseudoinverse-guided diffusion models for inverse problems. In International Conference on Learning Representations, 2023b. Yingqing Guo, Yukang Yang, Hui Yuan, and Mengdi Wang. Training-free guidance beyond differ- entiability: Scalable path steering with tree search in diffusion and flow models...
-
[2023]
Zero-shot image restoration using denoising diffusion null-space model
Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. arXiv preprint arXiv:2212.00490,
-
[2024]
Monte carlo guided diffusion for bayesian linear inverse problems
Gabriel Cardoso, Yazid Janati El Idrissi, Sylvain Le Corff, and Eric Moulines. Monte carlo guided diffusion for bayesian linear inverse problems. arXiv preprint arXiv:2308.07983,
-
[2025]
Benjamin Boys, Mark Girolami, Jakiw Pidstrigach, Sebastian Reich, Alan Mosca, and O Deniz Akyildiz
URLhttps:// openreview.net/forum?id=U3PBITXNG6. Benjamin Boys, Mark Girolami, Jakiw Pidstrigach, Sebastian Reich, Alan Mosca, and O Deniz Akyildiz. Tweedie moment projected diffusions for inverse problems. arXiv preprint arXiv:2310.06721,
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.