{"id":"a86e8097-dc3b-44e0-af82-fc855d718424","arxiv_id":"2411.13277","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Functional normalizing flows on Hilbert spaces yield a discretization-invariant variational approximation of the posterior for PDE inverse problems.","lead":"The authors introduce a normalizing-flow framework for Bayesian inverse problems of PDEs that works directly on infinite-dimensional function spaces, using transformations of a Gaussian prior that provably keep the transformed measure equivalent to the prior. A conditional variant, CNF-iVI, takes measurement data of varying dimensions as input, so new measurements can be processed without retraining the whole model.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 2.4's printed RN derivative has the wrong sign for ⟨u, fθ(u)−u⟩; it contradicts the appendix derivation and Cameron-Martin, so the printed loss in Algorithms 1 and 3 is incorrect.","rationale":"The reader's verdict already identifies the sign error in Theorem 2.4, and I agree this is the most load-bearing issue: the theorem's printed RN derivative is used verbatim in the KL loss in Section 2.2 and in Algorithms 1 and 3. The appendix proof (Section 7.3) derives the exponent as ½∥f(u)−u∥²_H + ⟨u, f(u)−u⟩_H, while the theorem statement and the subsequent expectation in Section 2.2 use +⟨u, u−f(u)⟩_H. The constant-shift case f(u)=u+m makes the contradiction with Cameron-Martin explicit: the printed density would assign mass exp(−⟨u,m⟩+½∥m∥²) to N(m,I), which is not the correct density exp(+⟨u,m⟩+½∥m∥²). This is an internal inconsistency in the central claim, not a mere numerical typo, because the training loss as printed is not the KL divergence to the posterior. The finite-rank limitation identified as the reader's weakest_assumption is a genuine representational constraint, but it does not invalidate the method's correctness; it limits its expressive power. Since the sign error is readily correctable (the appendix itself contains the correct sign), a conditional acceptance remains appropriate. My proposed check—evaluating the formula for a single-layer constant shift on a finite-dimensional Gaussian—would decisively confirm the sign error. The reader's weakest_assumption field points to the finite-rank restriction rather than this sign error, so my agreement is disagree in that specific sense, although the reader's overall verdict did flag the sign issue.","tokens_in":38609,"tokens_out":7360,"duration_ms":70923,"concrete_test":"Test the formula on the single-layer constant shift f(u)=u+m with m in the Cameron-Martin space, using a finite-dimensional Gaussian prior N(0,I) on R^d with m≠0. Compare the printed density at u+m (which equals exp(−⟨u,m⟩+½∥m∥²)) and the appendix-derived density (exp(+⟨u,m⟩+½∥m∥²)) against the exact density of N(m,I) evaluated at u+m; the appendix-derived expression matches, while the printed expression fails. This can be checked by direct substitution or by a one-line numerical evaluation, and it decisively confirms the sign error in Theorem 2.4.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is Theorem 2.4, whose printed Radon-Nikodym derivative is used as the KL loss in Section 2.2 and in Algorithms 1 and 3. The printed exponent is ½⟨fθ(u)−u, fθ(u)−u⟩_H + ⟨u, u−fθ(u)⟩_H. However, Appendix 7.3 derives, for a single layer with F(u)=m, the reciprocal density with exponent +½∥m∥²_H + ⟨u,m⟩_H; for a general composition the derivation gives +⟨u, fθ(u)−u⟩_H, not +⟨u, u−fθ(u)⟩_H. The two signs differ. For the constant shift f(u)=u+m with m∈H, the printed formula yields dµfθ/dµ0(u+m) = exp(½∥m∥²_H − ⟨u,m⟩_H), whereas the Cameron-Martin formula (Bogachev, Theorem 2.4.5) gives exp(½∥m∥²_H + ⟨u,m⟩_H). Thus the theorem statement is internally inconsistent with its own proof and contradicts a classical result in the simplest nontrivial case. Because the expectation identity in Section 2.2 and the loss in Algorithms 1 and 3 are written with the printed sign, the optimization objective as stated is not the KL divergence to the posterior. This is a correctness issue in the central claim, not merely a representational limitation.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper develops an infinite-dimensional variational inference method for Bayesian inverse problems of PDEs. Starting from a Gaussian prior on a separable Hilbert space, the authors define functional normalizing flows as compositions of maps f^(n)(u)=u+F^(n)(u), give conditions under which the push-forward is equivalent to the prior, and derive a Radon-Nikodym derivative formula that is then used as the KL loss in Algorithms 1 and 3. Four concrete layers are proposed (functional Householder, projected transformation, planar, and Sylvester flows), a discretization-invariance theorem is stated, and a conditional variant (CNF-iVI) is introduced to amortize training across measurement data. Numerical experiments for a linear elliptic inverse problem and a nonlinear Darcy-flow inverse problem compare the approximate posteriors with pCN samples and report mesh-independent behavior.","tokens_in":38856,"tokens_out":10265,"duration_ms":115828,"significance":"If the main formula and the supporting theory are corrected, the contribution is valuable to the PDE-constrained Bayesian inversion community: it provides one of the few infinite-dimensional normalizing-flow constructions with an explicit RN derivative, together with concrete layer families and a public code repository. The paper is also to be credited for benchmarking against pCN, for demonstrating quantitative covariance comparisons, and for attempting to verify discretization invariance numerically rather than only claiming it. However, the central RN-derivative formula contains a sign error that makes the printed theorem false, the discretization-invariance theorem is stated under hypotheses that the numerical examples do not satisfy, and the conditional training gradient in Algorithm 2 appears to miss a score term. These issues currently block acceptance.","major_comments":[{"comment":"The printed Radon-Nikodym derivative has the wrong sign in the linear term. The theorem states the exponent ½⟨fθ(u)−u, fθ(u)−u⟩_H + ⟨u, u−fθ(u)⟩_H, but the one-layer derivation in Appendix 7.3 gives Λ_F(u) = |det1(I+DF(u))| exp(−½∥F(u)∥²_H − ⟨u,F(u)⟩_H), so the reciprocal density must contain +⟨u,F(u)⟩_H = +⟨u, fθ(u)−u⟩_H. The final display of Appendix 7.3 repeats the incorrect sign instead of correcting it. For the constant shift f(u)=u+m with m∈H, the printed formula yields exp(½∥m∥²_H − ⟨u,m⟩_H), whereas the Cameron-Martin formula gives exp(½∥m∥²_H + ⟨u,m⟩_H). Since the same expression is used as the loss in Section 2.2 and in Algorithms 1 and 3, the optimization objective as written is not the KL divergence to the posterior. The sign must be flipped throughout Theorem 2.4, Theorems 2.5–2.8, and the algorithms.","section":"Theorem 2.4 and Appendix 7.3"},{"comment":"The discretization-invariance theorem assumes Hu is continuously embedded in C(D), but the numerical examples use Hu = L²(Ω) in Section 4.1 and Hu = L∞(Ω) in Section 4.2. Neither space embeds continuously into C(D) for the domains considered, so the stated theorem does not apply to the examples that are used to verify discretization invariance. The proof of Theorem 2.10 in Appendix 7.8 works under different, partially unstated regularity conditions (it invokes pointwise evaluation of a∈K⊂C(D) and Riemann-sum convergence). The paper needs to either restrict the claims to settings satisfying the theorem's hypotheses, or extend the theorem and the examples consistently.","section":"Theorem 2.10 and Sections 4.1, 4.2"},{"comment":"All four concrete flow layers are parameterized so that Im(F^(n)) ⊂ span{φ_1,…,φ_M}, the span of the first M prior eigenvectors. Consequently fθ(u)−u always lies in that M-dimensional subspace, and every component of u orthogonal to span{φ_1,…,φ_M} is left unchanged by the push-forward. The paper never states this structural limitation: any posterior deviation in the orthogonal complement is invisible to the approximation, regardless of training time. The role of M as a modeling assumption should be made explicit, and the paper should discuss whether the chosen numerical problems are ones where the data mainly inform the leading modes.","section":"Sections 2.3.2, 2.4.2, and Algorithms 1–3"},{"comment":"The gradient estimator for conditional training appears to be missing the score-function term. The objective is q(λ)=E_{ν_λ(v)}[log(dν_λ/dµ0)−log(dµ/dµ0)], and line 6 draws samples from ν_{λ_k}(v_i). If samples are drawn from the measure being optimized, the unbiased gradient of E_{u∼ν_λ}[f_λ(u)] is E[∇_λ f_λ(u) + f_λ(u)∇_λ log p_λ(u)], where p_λ=dν_λ/dµ0. The displayed formula only differentiates the log-ratio at the sampled points and omits the second term. Unless the authors intend a reparameterized gradient (which is not stated), the update is biased and the CNF-iVI training in Section 5 is not minimizing the stated objective.","section":"Algorithm 2, line 7"}],"minor_comments":[{"comment":"The sentence introducing eq. (2.10) says 'Each layer of functional Householder flow employs the transformation'; it should say functional planar flow.","section":"Section 2.4.1"},{"comment":"In the definition of functional Sylvester flow, the domains of A_n and B_n are transposed: A_n should map R^M to H_u and B_n should map H_u to R^M, consistent with the subsequent parameterization and with Theorem 2.9. As printed, the composition A_n h(B_n u + b_n) is not well-defined.","section":"Section 2.4.2"},{"comment":"The proofs of Theorems 2.7, 2.8, 2.5, and 2.6 twice state the eigenvalue condition as 'not in (−∞,1]' instead of 'not in (−∞,−1]'. This is a typo but should be fixed because the numerical condition imposed on the diagonal elements is > −1.","section":"Appendix 7.5 and 7.6"},{"comment":"The entry c(x1,x2) = −0.455003 for the 50×50 discretization level differs from all other rows by two orders of magnitude and is almost certainly a decimal-point typo. As printed, this figure contradicts the discretization-invariance claim and should be corrected.","section":"Table 9"},{"comment":"There are several typographical errors, including '0.0.02755' in Table 2 and 'discreate' in Tables 2, 3, 5, 6, 8, 9, 11, and 12. These should be corrected.","section":"Table 2 and surrounding text"},{"comment":"The text states that M=20 is 'in Subsection 2.4', but functional projected transformation flow is introduced in Subsection 2.3.2.","section":"Section 4.1.2"}],"recommendation":"major_revision","confidential_remarks":"The sign error in Theorem 2.4 is local and fixable, and the numerical experiments may still be valid if the implementation used the corrected sign, but the printed paper is internally inconsistent. The discretization-invariance mismatch and the conditional gradient issue require more substantial revision. I did not find problems of attribution: the variational inference framework is appropriately tied to the cited work of Pinski et al. and of the authors' group. The paper is within scope for a numerical-analysis / scientific-computing venue once the theoretical statements and algorithms are made consistent."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Jing, quick take on arXiv:2411.13277. The genuinely new thing: the paper gives conditions (finite rank, image in Cameron–Martin space, no point spectrum in (−∞,−1]) so that pushing a Gaussian prior through a composition of maps keeps the pushforward equivalent to the prior, and gives a closed-form Radon–Nikodym derivative. They build four concrete layers (Householder, projected, planar, Sylvester), prove discretization invariance, and test on elliptic and Darcy problems against pCN. The numerics look careful, and the code is on GitHub. That is the good part, and it is worth taking seriously.\n\nThe soft spots. First, Theorem 2.4 as printed has the wrong sign in the exponent. The statement gives +⟨u, u−fθ(u)⟩; the appendix derivation gives +⟨u, fθ(u)−u⟩. For f(u)=u+m with m in the Cameron–Martin space, the printed formula gives exp(½∥m∥² − ⟨u,m⟩), which violates Cameron–Martin. This is the loss used in Algorithms 1 and 3, so as written the objective is not the KL divergence. The appendix is correct, so I expect a typo, but it must be fixed. Second, Theorem 2.10 supposes Hu ↪ C(D), but the experiments use L2 spaces that do not embed into C(D); the proof sketches a Riemann-sum argument that might be repairable, but the statement as it stands does not cover the test cases. Third, all four concrete flows are finite-rank with image in span{φ1,…,φM}, the first M prior eigenvectors, so posterior changes in higher modes are structurally invisible to the model. That is a real modeling limitation, not a numerical detail. Minor: some tables have typos—Table 2 has “0.0.02755” and Table 6 has a c(x1,x2) entry around −0.455 for the 15×15 level.\n\nBottom line: the framework is a solid contribution to variational inference for PDE inverse problems, and the identified flaws are specific and correctable. I would give it a serious referee, and I would cite the corrected version.","headline":"A genuinely useful infinite-dimensional normalizing flow framework for Bayesian inverse problems, but the printed Radon–Nikodym formula in the central theorem has a sign error that contradicts its own appendix and the Cameron–Martin theorem.","tokens_in":39403,"tokens_out":4270,"would_cite":true,"duration_ms":46400,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F15","65N21"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper establishes conditions under which normalizing flows can be defined in infinite-dimensional function spaces for Bayesian inverse problems of PDEs, yielding a mesh-independent variational inference algorithm with an explicit…","keywords":["inverse problems","infinite-dimensional variational inference","functional normalizing flow","Bayesian analysis for functions","partial differential equations","discretization invariance","conditional normalizing flow"],"falsifier":"For the single-layer constant shift $f(u)=u+m$ with $m$ in the Cameron-Martin space, compare the printed Theorem 2.4 density with the classical Cameron-Martin density $\\exp(\\langle m,u\\rangle_H - \\tfrac12\\|m\\|_H^2)$; this fixes the sign of the inner-product term and decides whether the loss used by Algorithms 1 and 3 is the correct Radon-Nikodym derivative.","tokens_in":38360,"feed_emoji":"🌀","tokens_out":8660,"duration_ms":89632,"temperature":0.7,"pith_summary":"This paper proposes a way to run variational inference for Bayesian inverse problems of PDEs directly in infinite-dimensional function space, so the algorithm behaves consistently as the numerical mesh is refined. The authors construct normalizing flows, compositions of simple transformations, that map a Gaussian prior measure into a rich family of approximate posterior measures while remaining equivalent to the prior, which is what makes a Radon-Nikodym derivative and a KL-divergence loss exist. They state conditions on each transformation: a finite-rank perturbation with image in the Cameron-Martin space, plus a spectral condition that keeps each layer invertible. Four concrete flows are given, and a conditional variant learns to map measurement data to flow parameters so new datasets do not require full retraining. Numerical experiments on an elliptic source problem and steady-state Darcy flow match preconditioned Crank-Nicolson posteriors and confirm the claimed mesh independence.","feed_headline":"Normalizing flows give mesh-free Bayesian inversion for PDEs","feed_subtitle":"A function-space variational family preserves measure equivalence and matches MCMC posteriors on elliptic and Darcy problems.","key_machinery":"The central object is the layer $f^{(n)}_\\theta(u)=u+F^{(n)}_\\theta(u)$ on a separable Hilbert space, with the image of $F^{(n)}$ contained in the Cameron-Martin space $H$ of the Gaussian prior, the operator $F^{(n)}$ of finite rank, and the point spectrum of $DF^{(n)}$ avoiding $(-\\infty,-1]$. The key identity is the Radon-Nikodym formula of Theorem 2.4, which expresses the density of the transformed measure through a product of Fredholm-Carleman determinants, here determinants for operators of the form $I+K$ with $K$ finite rank, multiplied by a Cameron-Martin-type exponential. This identity is what converts KL-divergence minimization into a Monte-Carlo-estimable loss, and the spectral condition is what guarantees each layer is invertible and the measures stay mutually absolutely continuous.","core_discovery":"The paper claims that for a composition $f_\\theta = (I+F^{(N)})\\circ\\cdots\\circ(I+F^{(1)})$ acting on a separable Hilbert space with Gaussian prior $\\mu_0$, if each $F^{(n)}$ is finite rank with image in the Cameron-Martin space, each layer is bijective, and no point spectrum of $DF^{(n)}$ lies in $(-\\infty,-1]$, then the push-forward measure is equivalent to the prior and its Radon-Nikodym derivative is $$\\frac{d\\mu_{f_\\$\\theta$}}{d\\mu_0}(f_\\$\\theta$(u)) = \\prod_{n=1}^N \\left|\\det_1\\big($Df^{{(n)}}$(u_{n-1})\\big)\\right|^{-1} \\exp\\!\\left(\\tfrac12\\langle f_\\$\\theta$(u)-u,\\,f_\\$\\theta$(u)-u\\rangle_H + \\langle u,\\,u-f_\\$\\theta$(u)\\rangle_H\\right).$$ This explicit density turns the infinite-dimensional change of measure into a computable loss, so the flow parameters can be optimized by stochastic gradient methods to approximate the posterior. The paper then constructs four layers, functional Householder, projected transformation, planar, and Sylvester flows, that satisfy the conditions, and a neural-network-controlled conditional version that produces an approximate posterior for new measurement data after a single forward pass.","pith_inferences":["Editorial extension: because every concrete flow layer is built from the first $M$ prior eigenvectors, the method's expressive range is confined to a finite-dimensional subspace, and problems where the data chiefly inform higher modes would require a different construction.","Editorial extension: the conditional network encodes measurement information by projecting onto the same prior eigenbasis, so the amortized variant inherits the mesh dependence of that truncation rather than being fully mesh-independent.","Editorial extension: replacing the fixed prior eigenbasis with a learned or wavelet basis could extend the same conditions to non-Gaussian or Besov priors, but the paper does not demonstrate such an extension."],"forward_implications":["If the conditions hold, the KL divergence between the flow-induced measure and the posterior can be estimated from prior samples, making posterior approximation a tractable stochastic optimization problem in function space.","The discretization-invariance theorem means the same trained flow can be evaluated on any mesh, so the method belongs to the Bayesianize-then-discretize family and avoids retraining as grids are refined.","The conditional variant CNF-iVI lets a new measurement dataset produce an approximate posterior immediately, with an additional warm-start retraining step for higher accuracy.","Numerical comparisons with pCN on an elliptic source problem and steady-state Darcy flow show similar posterior means and covariance functions while yielding effectively independent samples with much higher effective sample sizes than the MCMC baseline."],"supporting_citations":[{"why":"Supplies the Gaussian measure theory, Cameron-Martin space, and Fredholm-Carleman determinant used to prove Theorem 2.4.","marker":"[7]"},{"why":"Supplies the infinite-dimensional variational inference formulation and the requirement that approximating measures be equivalent to the prior.","marker":"[39]"},{"why":"Supplies the Gaussian-parameterized KL approximation framework and optimization viewpoint that the flows extend.","marker":"[38]"},{"why":"Supplies the infinite-dimensional Bayesian inverse problem setup and the pCN sampler used as benchmark.","marker":"[17]"},{"why":"Supplies the finite-dimensional normalizing flow construction that the functional flows generalize.","marker":"[40]"},{"why":"Supplies the Sylvester flow architecture adapted into the functional Sylvester flow.","marker":"[46]"},{"why":"Supplies the definition of discretization invariance used in Theorem 2.10.","marker":"[29]"},{"why":"Supplies a prior infinite-dimensional variational inference approach that this work extends with functional flows.","marker":"[23]"}],"fun_headline_variants":["Functional flows tame PDE inverse problems","Mesh-free Bayesian inversion via functional normalizing flows","Infinite-dim flows for PDE posteriors, discretization-invariant","Normalizing flows in Hilbert space solve inverse problems","Measure-preserving flows for Bayesian PDE inversion"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the true posterior differs from the prior mainly through the first $M$ eigenmodes of the prior covariance, because every flow layer is a finite-rank perturbation built from those modes and leaves all higher modes untouched.","fun_headline_variants_meta":{"raw":{"variants":["Functional flows tame PDE inverse problems","Mesh-free Bayesian inversion via functional normalizing flows","Infinite-dim flows for PDE posteriors, discretization-invariant","Normalizing flows in Hilbert space solve inverse problems","Measure-preserving flows for Bayesian PDE inversion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000744,"raw_usage":{"total_tokens":3353,"prompt_tokens":1015,"completion_tokens":2338,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":631,"completion_tokens_details":{"reasoning_tokens":2277}},"tokens_in":631,"tokens_out":2338,"duration_ms":17385,"temperature":1.0,"reasoning_tokens":2277,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:38:04.974639+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For the single-layer constant shift $f(u)=u+m$ with $m$ in the Cameron-Martin space, compare the printed Theorem 2.4 density with the classical Cameron-Martin density $\\exp(\\langle m,u\\rangle_H - \\tfrac12\\|m\\|_H^2)$; this fixes the sign of the inner-product term and decides whether the loss used by Algorithms 1 and 3 is the correct Radon-Nikodym derivative.","supporting_citations":[{"cited_title":"Bogachev, Gaussian Measures, American Mathematical Society, Providence, RI, 1998","cited_arxiv_id":null,"evidence_quote":"Supplies the Gaussian measure theory, Cameron-Martin space, and Fredholm-Carleman determinant used to prove Theorem 2.4."},{"cited_title":"Pinski, Geoffrey Simpson, Andrew M","cited_arxiv_id":null,"evidence_quote":"Supplies the infinite-dimensional variational inference formulation and the requirement that approximating measures be equivalent to the prior."},{"cited_title":"Pinski, Geoffrey Simpson, Andrew M","cited_arxiv_id":null,"evidence_quote":"Supplies the Gaussian-parameterized KL approximation framework and optimization viewpoint that the flows extend."},{"cited_title":"Stuart, The Bayesian approach to inverse problems, Handbook of uncertainty quantification","cited_arxiv_id":null,"evidence_quote":"Supplies the infinite-dimensional Bayesian inverse problem setup and the pCN sampler used as benchmark."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the finite-dimensional normalizing flow construction that the functional flows generalize."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Sylvester flow architecture adapted into the functional Sylvester flow."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the definition of discretization invariance used in Theorem 2.10."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies a prior infinite-dimensional variational inference approach that this work extends with functional flows."}],"review_version":1}