{"id":"1b77c888-e091-45e8-8389-f4584c65e8f3","arxiv_id":"2505.22391","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Post-hoc distillation with a PDE-residual loss on final samples avoids the Jensen gap and yields one-step physics-constrained generation.","lead":"PIDDM distills a diffusion model into a one-step generator and adds a physics-equation penalty on the final output, improving equation satisfaction. This gives faster and more accurate generative surrogates for PDEs, useful for simulation, inverse problems, and partial-data reconstruction.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PDE-satisfaction gains are measured with the same finite-difference residual used for training and refinement; without an independent solver, the central 'improved physics satisfaction' claim may be circular.","rationale":"The reader's weakest assumption is that the finite-difference operator R faithfully represents the true PDE operator, and that is exactly the load-bearing point I find. The concern is stronger than 'coarse grid': the same R is used for the training loss (Eq. 8), the optional refinement (Algorithm 2), the downstream inference objective (Algorithm 3), and the reported PDE-error metric. This means the headline physical-satisfaction numbers could be optimized without genuinely improving physical fidelity. The MMSE and SMSE results partially support distributional fidelity, but they do not validate the physical-constraint claim independently. The authors are transparent about the limitation, and the method is sensible and clearly described, so I do not see a fatal flaw. A single independent-solver cross-check would determine whether the central claim holds; until then, CONDITIONAL is the appropriate verdict. No change to the reader's verdict is needed.","tokens_in":20294,"tokens_out":3810,"duration_ms":51758,"concrete_test":"Recompute the PDE-error rows of Tables 1 and 4 for PIDDM-1, PIDDM-ref, ECI, DiffusionPDE, D-Flow, PIDM, and Vanilla using an independent high-order or adaptive solver--for example, spectral/Chebyshev collocation for Poisson and Darcy, or a 256x256 fourth-order finite-volume discretization for all equations--instead of the finite-difference R used in training and refinement. If PIDDM's margin over baselines shrinks or reverses under this independent residual, the reported PDE gains are partly artifacts of fitting the training-time operator; if the margin persists, the discretization-fidelity concern is settled. Also report the per-sample correlation between the training-time residual R and the independent residual on the same generated fields.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 3.1 defines the physics residual R(x) via finite differences (Eq. 2); Eq. 8 adds lambda_train * ||R(x)||^2 to the distillation loss; Algorithm 2 refines the latent noise by gradient descent on ||R(d_theta'(epsilon))||^2; and Section 5 reports 'PDE Error' as |R(x)|^2. The same operator is therefore used as training objective, refinement objective, and evaluation metric. If R is a coarse or biased discretization of the true PDE operator F, the student can minimize the reported error by fitting the discretized stencil (for example, by smoothing or exploiting cancellation errors) while the true physical residual remains large. The central claim that PIDDM 'significantly improves PDE satisfaction' over baselines thus rests on the unverified equivalence between R and the actual physical law. The authors acknowledge sensitivity to the residual operator in the Limitations section, but no experiment validates R against an independent solver, a finer grid, or a different discretization family. This is not an accusation of fabrication; it is a concrete circularity in the headline metric that would be resolved by an independent cross-check.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PIDDM, a post-hoc distillation framework for physics-constrained generation with diffusion models. A teacher diffusion model is trained with the standard (unconstrained) denoising objective; then a student model is trained to map noise to the teacher's deterministic ODE outputs while also minimizing a finite-difference PDE residual on the final sample, thereby avoiding the Jensen's Gap that arises when constraints are applied to the posterior mean. The method also supports optional refinement of the latent noise and is extended to forward/inverse problems and partial-observation reconstruction. Experiments on Darcy, Poisson, Burgers, and several additional PDE benchmarks report improved PDE-residual errors, comparable or better MMSE/SMSE, and lower NFE relative to guidance-based baselines.","tokens_in":20624,"tokens_out":7947,"duration_ms":85055,"significance":"If the empirical claims are validated, the core idea is significant: it decouples physics enforcement from the diffusion trajectory, yields one-step generation, and naturally handles several downstream tasks. The conceptual argument for avoiding Jensen's Gap is clean and the experimental scope is broad, covering eight PDE systems. The paper also includes ablations on key hyperparameters and distillation variants. However, the headline metric is the same finite-difference residual used as the training and refinement objective, so the central claim of improved 'PDE satisfaction' is not yet independently established; the absence of error bars further weakens the comparative claims. The significance is therefore contingent on additional validation.","major_comments":[{"comment":"The reported 'PDE Error' is the squared norm of the same finite-difference residual R that is used as the training loss in Eq. (8) and as the refinement objective in Algorithm 2. Consequently, the student can improve this metric by fitting the discretized stencil rather than the true PDE operator, especially if R is coarse or biased. The Limitations section concedes that constructing a reliable residual operator is challenging with coarse or low-accuracy finite-difference schemes. Because 'significantly improves PDE satisfaction' is the paper's central empirical claim, the authors should add an independent cross-check: for example, evaluate all methods with a higher-order or different discretization, or compare against a numerical solver on a held-out test set, and show that the ranking persists. Without such a check, the headline metric is partly circular.","section":"§3.1 Eq. (2); §4.2 Eq. (8); Algorithm 2; §5 PDE Error; Limitations"},{"comment":"All quantitative results are single-seed point estimates without error bars or statistical significance tests. Statements such as 'PIDDM significantly improves PDE satisfaction' and 'surpasses all prior methods' are not supported statistically. The authors should repeat experiments over at least three seeds and report mean ± standard deviation, and where possible use paired comparisons on the same latent noise. This is necessary for the reader to assess whether the reported differences (e.g., Darcy MMSE 0.112 vs 0.108 for PIDDM-1 vs Vanilla) are meaningful.","section":"Tables 1–4"},{"comment":"The claim that 'PIDDM-1 already surpasses all prior methods that incorporate physical constraints during training or sampling' is contradicted by the table for several fidelity metrics: Poisson SMSE (ECI 0.291 vs PIDDM-1 0.326), Burger SMSE (DiffusionPDE 0.103 vs PIDDM-1 0.133), and Burger MMSE (DiffusionPDE 0.064 vs PIDDM-1 0.152). The claim should be qualified to reference PDE Error or to acknowledge that fidelity is competitive but not uniformly best; the exceptions should be discussed. As written, the text overstates the evidence.","section":"§5.1, Table 1"},{"comment":"The contribution 'first explicit empirical demonstration of Jensen's Gap' is not established. The MoG experiment compares DPS, with a tuned guidance weight of 0.035, to the ground-truth conditional ODE; the observed velocity errors and constraint violations may reflect DPS's particular guidance approximation rather than Jensen's Gap per se. The training-time experiment in Fig. 2d shows that adding a PIDM-style residual loss increases the diffusion loss, which is a trade-off, but it does not directly measure the discrepancy between R(E[x0|xt]) and E[R(x0)|xt]. A direct measurement of these two quantities for the same score model at multiple noise levels, or a controlled comparison where the only difference is whether the constraint is applied to the posterior mean or the final sample, would substantiate the claim.","section":"§3.3, Figure 2"}],"minor_comments":[{"comment":"In Algorithm 1, line 5, the residual is written as R(hat(x)), which is ambiguous; it should be R(d_theta'(epsilon)) (or R(x_pred)) to match Eq. (8).","section":"§4.2, Algorithm 1"},{"comment":"The description of the Helmholtz/Poisson dataset repeats the Darcy Flow setup verbatim ('We adopt the Darcy Flow setup introduced in DiffusionPDE...'); the specific PDE, boundary conditions, and generation process for the Helmholtz/Poisson dataset should be stated explicitly.","section":"Appendix B.2"},{"comment":"The reconstruction Coef MSE values for PIDDM RF-1 and RF-2 (0.913 and 0.954) are an order of magnitude larger than for PIDDM (0.128), yet the text states that RF-2 'achieves the best overall performance across forward and inverse tasks.' Please clarify whether these numbers are correct or transposed, and discuss the reconstruction trade-off.","section":"Table 3"},{"comment":"The text says the number of refinement steps N is set to 0 or 50, but Tables 1 and 2 report NFE = 80 for the refinement/optimization settings (0.080 × 10^3). Please define precisely how NFE is counted for Algorithm 2 and Algorithm 3, and reconcile the stated value of N with the reported NFE.","section":"Appendix C.3; Tables 1–2"},{"comment":"The bullet 'Theoretically sound' is not supported by a formal theorem or proposition. The conceptual argument against Jensen's Gap is clear, but to avoid overclaiming, either add a formal statement (e.g., a proposition about the training objective or the refinement convergence) or soften the wording to 'conceptually grounded'.","section":"Contributions list"},{"comment":"There are several typos and inconsistencies: 'DiuffsionPDE' in §5, 'Diffusin process' in Appendix A.1, and duplicated references [36,37,38] in the bibliography. A careful proofread is needed.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a machine-learning journal and the central idea is simple, sensible, and likely to attract interest. The main risk is the circularity of the headline metric: PDE Error is the same finite-difference residual used for training and refinement, so the reported improvements may be partly by construction. If the authors provide an independent validation of the residual operator (e.g., a different discretization or a numerical solver), the paper could become a solid contribution. The lack of error bars and the overstatements in §5.1 are also important to address. I would not reject on the current evidence, but the revision must address the circularity concern directly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Plain take: the paper does something useful and new — it distills a diffusion teacher and applies the PDE residual loss to the student's final output, sidestepping the Jensen's gap that plagues guidance-based methods. The one-step inference is a real practical win, and the experiments are broad.\n\nThe strongest evidence is the synthetic Jensen's gap demonstration: the MoG experiment cleanly shows that DPS-style guidance on the posterior mean fails a hard constraint while the distilled student satisfies it with preserved marginals. That gives the core motivation teeth.\n\nThe soft spot is the circularity the stress-test flags. The same finite-difference operator R appears in the training loss (Eq. 8), the optional refinement (Algorithm 2), and the headline PDE Error metric (Table 1). If R is a coarse or biased discretization of the true PDE, the student is rewarded for fitting the stencil rather than the physics. The authors acknowledge sensitivity to the residual operator in the limitations section, but they never validate R against an independent solver, a finer grid, or a different discretization family. That is a concrete gap in evidence, not a fabrication. The MMSE/SMSE numbers are independent and look fine, so the method is not a mirage — but the dramatic PDE-error reductions are partly by construction.\n\nOther issues are minor by comparison: no error bars, single seed per experiment, no code release, and the claim of being 'theoretically sound' is overstated — there is no formal theorem, just a sound conceptual argument. The ablation on lambda_train even shows the discretization-induced bias, which supports the concern.\n\nWho is this for? Someone working on physics-constrained generative modeling who wants a cheap way to impose constraints at inference time. The method is simple enough to be a drop-in ingredient. It deserves a serious referee; the central idea holds up, and the circularity can be addressed with an independent residual cross-check and seed variability. I'd accept for review and ask for that experiment before recommending acceptance.","headline":"Useful distillation trick for physics-constrained diffusion, but the headline PDE-error gain is partly circular; worth a rigorous referee.","tokens_in":21028,"tokens_out":2296,"would_cite":true,"duration_ms":27769,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Post-hoc distillation enforces PDE laws on final diffusion samples, avoiding the Jensen's Gap and enabling one-step generation.","keywords":["diffusion models","physics-informed distillation","PDE-constrained generation","Jensen's gap","one-step generation","forward and inverse problems","partial observation reconstruction","finite-difference residual"],"falsifier":"Train or evaluate PIDDM on a PDE where the loss residual $R$ uses a coarse grid (for example 32$\\times$32), then evaluate the same generated samples with a high-resolution or spectral discretization of the same PDE; if the samples' residual on the independent discretization is not correspondingly small, the claim that constraints are enforced on the true sample rather than the discrete proxy would be falsified. A second check: on the correlated mixture-of-Gaussians example, PIDDM should keep the constraint deviation near $2\\times10^{-3}$ while matching the target marginal; if either constraint satisfaction or distributional fidelity breaks under different random seeds, the claim of avoiding the trade-off weakens.","tokens_in":20054,"feed_emoji":"⚛️","tokens_out":4467,"duration_ms":45166,"temperature":0.7,"pith_summary":"The paper claims that the standard way of injecting PDE constraints into diffusion models—evaluating the residual on the denoising estimate $\\mathbb{E}[x_0|x_t]$—is theoretically mismatched, because a nonlinear operator applied to an average is not the average of the operator applied to samples. It proposes to sidestep this Jensen's Gap by distilling the diffusion model into a one-step student that maps noise directly to a clean solution, adding the PDE residual loss only at that final output. The claim is that this decoupling keeps the generative fidelity of the unconditional teacher while substantially improving PDE satisfaction, at a fraction of the sampling cost. A sympathetic reader would care because it offers a simple route to physically consistent generative surrogates for forward simulation, inverse recovery, and reconstruction from partial observations.","feed_headline":"One-step distillation enforces PDE laws on diffusion samples","feed_subtitle":"Evaluating physics on the final output, not the noisy mean, avoids the Jensen's Gap trade-off.","key_machinery":"The central object is the distilling student network $d_{\\theta'}(\\epsilon)$ together with the finite-difference physics residual operator $R(x) = (F[u], B[a])^\\top$. The distillation objective couples a regression loss to noise–data pairs produced by the teacher's deterministic ODE with a PDE-residual loss evaluated on the student's one-step output. This moves constraint enforcement from the noisy trajectory to the clean endpoint, which is exactly what removes the Jensen's Gap. The optional refinement step and downstream inference both update the latent noise $\\epsilon$ by gradient descent on $R$, which is cheap because the student is one-step.","core_discovery":"The central claim is that post-hoc distillation with PDE constraints on the final sample $x = d_{\\theta'}(\\epsilon)$ avoids the Jensen's Gap that plagues training-time and sampling-time guidance, and yields one-step generation that is both more physics-satisfying and distributionally faithful than PIDM, DiffusionPDE, ECI-sampling, and D-Flow baselines. The method trains a standard teacher diffusion model with a linear flow schedule, generates paired noise–data samples $(\\epsilon, x_0)$ by deterministic ODE integration, and trains a student $d_{\\theta'}$ with loss $L_{\\text{total}} = \\mathbb{E}[\\|d_{\\theta'}(\\epsilon)-x_0\\|^2] + \\lambda_{\\text{train}}\\|R(x)\\|^2$. Because $R$ is evaluated on the actual generated field rather than on a posterior mean estimate, the mismatch $R(\\mathbb{E}[x_0|x_t]) \\neq \\mathbb{E}[R(x_0)|x_t]$ never enters the objective. The paper also demonstrates the gap empirically on a mixture-of-Gaussians example and in training curves, and shows that the distilled student supports forward, inverse, and reconstruction tasks by latent-noise optimization with an observation mask.","pith_inferences":["Because the finite-difference residual $R$ is both the training objective and the evaluation metric, the reported PDE-error gains are relative to that specific discretization; an independent higher-order or spectral residual check would distinguish true physics satisfaction from fitting the discrete operator.","The same post-hoc distillation recipe could be applied to other hard constraints—conservation laws, bounds, symmetries—whenever a teacher diffusion model can produce paired noise–output samples, not just PDE residuals.","One-step generation makes latent-space optimization cheap enough for real-time control or data assimilation, a possibility the paper notes but does not develop.","If the Jensen's Gap is indeed the dominant source of the fidelity–physics trade-off, then any future method that enforces constraints on the exact final sample should also avoid the trade-off, making the specific distillation mechanism one instance of a more general design principle."],"forward_implications":["One-step generation with a single function evaluation achieves PDE residuals lower than all tested guidance-based baselines, and an optional 50-step refinement lowers them further.","Distributional fidelity (MMSE and SMSE) does not degrade when the physics loss is added, unlike training-time PIDM whose diffusion loss rises under residual supervision.","The same distilled student solves forward, inverse, and partial-observation reconstruction tasks by latent-noise optimization with an observation mask, matching or beating D-Flow at a fraction of its function evaluations.","Better noise–data coupling via Rectified Flow or distribution matching distillation further improves the trade-off between sample quality and constraint satisfaction."],"supporting_citations":[{"why":"Identifies the Jensen's Gap and provides the main training-time baseline (PIDM) that adds residual loss on the posterior mean.","marker":"[3]"},{"why":"Supplies the DiffusionPDE baseline using diffusion posterior sampling and provides several PDE datasets used in experiments.","marker":"[23]"},{"why":"Provides the ECI-sampling hard-constraint baseline and the FNO-based setup that the paper adopts for teacher and student models.","marker":"[8]"},{"why":"Establishes the vanilla knowledge-distillation-of-a-diffusion-model formulation that the paper extends with a PDE residual loss.","marker":"[43]"},{"why":"Demonstrates that straighter noise–data couplings ease one-step distillation, motivating the linear flow schedule and Rectified Flow variants.","marker":"[39]"},{"why":"Serves as the D-Flow baseline that optimizes initial noise through the full sampling trajectory to satisfy the PDE.","marker":"[4]"},{"why":"Provides the diffusion posterior sampling method used by DiffusionPDE and in the mixture-of-Gaussians demonstration of the Jensen's Gap.","marker":"[9]"},{"why":"Supplies the Fourier Neural Operator architecture used for both teacher and student networks and the original Darcy/Poisson/Burgers datasets.","marker":"[34]"},{"why":"Supplies the distribution matching distillation variant used in ablations to improve the fidelity of the distilled student.","marker":"[63]"}],"fun_headline_variants":["Post-hoc distillation skips Jensen's gap in PDE diffusion","One-step physics-guided diffusion without Jensen's gap","Distill PDE constraints into final samples, not noisy means","One-step diffusion that obeys PDEs via post-hoc distillation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the discretized finite-difference operator $R$ used in training and evaluation is a faithful proxy for the true PDE operator $F$; if the discretization is coarse or biased, the student learns to satisfy the discrete residual rather than the actual physical law, and the reported gains are measured with the same biased ruler.","fun_headline_variants_meta":{"raw":{"variants":["Post-hoc distillation skips Jensen's gap in PDE diffusion","One-step physics-guided diffusion without Jensen's gap","Distill PDE constraints into final samples, not noisy means","One-step diffusion that obeys PDEs via post-hoc distillation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000482,"raw_usage":{"total_tokens":2461,"prompt_tokens":1104,"completion_tokens":1357,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":720,"completion_tokens_details":{"reasoning_tokens":1291}},"tokens_in":720,"tokens_out":1357,"duration_ms":10377,"temperature":1.0,"reasoning_tokens":1291,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:07:42.789229+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train or evaluate PIDDM on a PDE where the loss residual $R$ uses a coarse grid (for example 32$\\times$32), then evaluate the same generated samples with a high-resolution or spectral discretization of the same PDE; if the samples' residual on the independent discretization is not correspondingly small, the claim that constraints are enforced on the true sample rather than the discrete proxy would be falsified. A second check: on the correlated mixture-of-Gaussians example, PIDDM should keep the constraint deviation near $2\\times10^{-3}$ while matching the target marginal; if either constraint satisfaction or distributional fidelity breaks under different random seeds, the claim of avoiding the trade-off weakens.","supporting_citations":[{"cited_title":"DiffusionPDE: Generative PDE-solving under partial observation","cited_arxiv_id":null,"evidence_quote":"Supplies the DiffusionPDE baseline using diffusion posterior sampling and provides several PDE datasets used in experiments."},{"cited_title":"Maddix, Abdul Fatir Ansari, Andrew Stuart, Michael W","cited_arxiv_id":null,"evidence_quote":"Provides the ECI-sampling hard-constraint baseline and the FNO-based setup that the paper adopts for teacher and student models."}],"review_version":1}