pith. sign in

arxiv: 2605.29373 · v2 · pith:W5PTTA22new · submitted 2026-05-28 · 💻 cs.LG · cs.NA· math.NA

Deep Adaptive Dimension Reduction for Bayesian Inference in Inverse Problems

Yueyang Wang , Xili Wang , Kejun Tang , Xiaoliang Wan , Tao Zhou , Chao Yang This is my paper

Pith reviewed 2026-06-29 08:51 UTC · model grok-4.3

classification 💻 cs.LG cs.NAmath.NA
keywords bayesian inferenceinverse problemsdimension reductionvariational flownormalizing flowsfourier neural operatorpdesurrogate modeling
0
0 comments X

The pith

Variational flow integrates nonlinear dimension reduction with dual normalizing flows to approximate complex posteriors for high-dimensional inverse problems

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper proposes a deep adaptive dimension-reduction Bayesian inference framework based on the Variational Flow model. The model combines VAE-based nonlinear dimension reduction with dual normalizing flows on the latent prior and encoder to overcome the bijective limitation of standard flows while achieving a strictly higher evidence lower bound than VAE. An iterative prior updating strategy shifts the prior mean toward high-probability posterior regions without manual tuning, and the components close an adaptive loop with a fine-tuned Fourier Neural Operator surrogate that generates better samples to refine itself. Numerical tests on a 100-dimensional Rosenbrock problem and three PDE inverse problems show competitive or superior accuracy versus MCMC, UKI, and SVGD baselines, with clearest gains under high noise or high dimensions.

Core claim

We propose a deep adaptive dimension-reduction Bayesian inference framework based on the Variational Flow (VF) model. Since standard normalizing flows are restricted by bijective mappings and cannot directly reduce dimensions, VF overcomes this limitation by integrating VAE-based nonlinear dimension reduction with dual normalizing flows for the latent prior and encoder. This design provides a strictly higher evidence lower bound than VAE and allows more flexible approximation of complex posterior distributions. We further introduce an iterative prior updating strategy that gradually moves the prior mean toward high-probability posterior regions, avoiding manual prior tuning. These components

What carries the argument

Variational Flow (VF) model that integrates VAE-based nonlinear dimension reduction with dual normalizing flows for the latent prior and encoder

If this is right

  • VF achieves a strictly higher evidence lower bound than standard VAE.
  • Iterative prior updating avoids manual tuning while concentrating samples in posterior regions.
  • The closed adaptive loop lets VF samples refine the FNO surrogate and the improved surrogate sharpens posterior inference.
  • The approach yields competitive or superior accuracy to MCMC, UKI, and SVGD, most noticeably in high-noise and high-dimensional regimes.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The adaptive loop structure could be paired with surrogate models other than FNO for forward problems outside PDEs.
  • Dimension reduction via VF might allow the framework to scale beyond the tested 100 dimensions where direct MCMC becomes intractable.
  • The dual-flow construction could be combined with other variational families to further increase posterior flexibility without changing the outer loop.

Load-bearing premise

The iterative prior updating strategy moves the prior mean toward high-probability posterior regions without introducing bias or instability in the closed adaptive loop with the FNO surrogate.

What would settle it

A comparison on the 100-dimensional Rosenbrock problem with high-noise observations in which the method fails to match or exceed MCMC accuracy on posterior metrics would falsify the performance claim.

Figures

Figures reproduced from arXiv: 2605.29373 by Chao Yang, Kejun Tang, Tao Zhou, Xiaoliang Wan, Xili Wang, Yueyang Wang.

Figure 1
Figure 1. Figure 1: Comparison of the 2D marginal posterior distributions on the [PITH_FULL_IMAGE:figures/full_fig_p013_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Surrogate fitting error eS across adaptive stages of 1D Darcy Flow problem. Columns: noise levels δ ∈ {1%, 5%, 10%}. Solid/dashed lines: d = 64/32. Overall, our method converges to lower errors in fewer stages [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative comparison for the 1D Darcy Flow problem with [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Surrogate fitting error eS across adaptive stages of 2D Darcy Flow problem. Columns: noise levels δ ∈ {1%, 5%, 10%}. Solid/dashed lines: d = 64/32. Overall, our VF method converges to lower errors in fewer stages. The quantitative inversion results for the 2D Darcy problem are summarized in [PITH_FULL_IMAGE:figures/full_fig_p016_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparison for the 2D Darcy Flow problem with [PITH_FULL_IMAGE:figures/full_fig_p017_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Surrogate fitting error eS across adaptive stages of 2D Navier-Stokes problem. Columns: noise levels δ ∈ {1%, 5%, 10%}. Solid/dashed lines: d = 64/32. Overall, our VF method converges to lower errors in fewer stages. Darcy case, the VF method accurately captures the complex vortex structures in ω0 and preserves their spatial distributions, resulting in final vorticity fields ω(·, T) at T = 1 that closely m… view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative comparison for the 2D Navier-Stokes problem with [PITH_FULL_IMAGE:figures/full_fig_p020_7.png] view at source ↗
read the original abstract

Solving high-dimensional PDE-governed inverse problems is often challenging due to complex non-Gaussian posterior distributions, expensive forward model evaluations, and misspecified prior information. To address these issues, we propose a deep adaptive dimension-reduction Bayesian inference framework based on the Variational Flow (VF) model. Since standard normalizing flows are restricted by bijective mappings and cannot directly reduce dimensions, VF overcomes this limitation by integrating VAE-based nonlinear dimension reduction with dual normalizing flows for the latent prior and encoder. This design provides a strictly higher evidence lower bound than VAE and allows more flexible approximation of complex posterior distributions. We further introduce an iterative prior updating strategy that gradually moves the prior mean toward high-probability posterior regions, avoiding manual prior tuning. These components form a closed adaptive loop together with an adaptively fine-tuned Fourier Neural Operator (FNO) surrogate: VF generates posterior-concentrated samples to refine the surrogate, while the updated surrogate further improves posterior inference. Numerical experiments on a 100-dimensional Rosenbrock problem and three standard PDE-governed inverse problems show that our method delivers competitive or superior accuracy compared with MCMC, UKI, and SVGD baselines across all tested configurations, with the most pronounced advantages emerging in challenging scenarios such as high-noise observations and high-dimensional parameter spaces.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes a Variational Flow (VF) framework for dimension reduction in Bayesian inference of high-dimensional PDE-governed inverse problems. VF integrates VAE-based nonlinear reduction with dual normalizing flows; this is combined with an iterative prior-mean update and a closed adaptive loop that retrains an FNO surrogate on VF-generated posterior samples. The central claim is that the resulting method achieves competitive or superior accuracy to MCMC, UKI, and SVGD baselines on a 100-dimensional Rosenbrock test and three standard PDE inverse problems, with the largest gains in high-noise and high-dimensional regimes.

Significance. If the numerical superiority and lack of bias in the adaptive loop can be rigorously established, the approach would address a practically important bottleneck: expensive forward models with misspecified priors and non-Gaussian posteriors. The combination of learned dimension reduction and surrogate adaptation could materially lower the cost of posterior sampling in such settings.

major comments (2)
  1. [Numerical experiments] The abstract asserts superior or competitive accuracy but supplies no quantitative metrics, error bars, ablation studies, or description of experimental controls; soundness cannot be evaluated from the given information. This directly affects the central empirical claim.
  2. [Method (iterative prior updating and closed adaptive loop)] The iterative prior updating strategy is asserted to move the prior mean toward high-probability posterior regions "without introducing bias or instability" inside the closed VF-FNO loop, yet no contraction mapping, bias bound, or even an ablation that disables the update while holding other components fixed is supplied. This assumption is load-bearing for the claim that the fixed point recovers the true posterior.
minor comments (2)
  1. [Variational Flow model] Notation for the evidence lower bound comparison between VF and VAE should be made explicit (e.g., which terms differ and by how much).
  2. The manuscript should state the precise definition of "accuracy" used in the comparisons (RMSE on parameters, posterior mean error, predictive coverage, etc.).

Simulated Author's Rebuttal

2 responses · 1 unresolved

We thank the referee for the constructive feedback on the empirical presentation and the theoretical grounding of the adaptive components. We address each major comment below and indicate revisions where appropriate.

read point-by-point responses

  1. Referee: [Numerical experiments] The abstract asserts superior or competitive accuracy but supplies no quantitative metrics, error bars, ablation studies, or description of experimental controls; soundness cannot be evaluated from the given information. This directly affects the central empirical claim.

    Authors: The full manuscript (Section 4 and supplementary material) reports quantitative results on the 100-dimensional Rosenbrock problem and three PDE inverse problems, including relative L2 errors on posterior means, KL divergences to MCMC reference posteriors, and standard deviations computed over 10 independent runs with error bars. Ablation tables isolate the contributions of the VAE-based reduction, dual flows, prior updating, and FNO adaptation, with explicit experimental controls (same random seeds, identical observation noise levels, and fixed network architectures). We have revised the abstract to include a concise summary of the key metrics (e.g., average 15-30% error reduction versus UKI in high-noise regimes) so that the central claim is supported at a glance. revision: yes

  2. Referee: [Method (iterative prior updating and closed adaptive loop)] The iterative prior updating strategy is asserted to move the prior mean toward high-probability posterior regions "without introducing bias or instability" inside the closed VF-FNO loop, yet no contraction mapping, bias bound, or even an ablation that disables the update while holding other components fixed is supplied. This assumption is load-bearing for the claim that the fixed point recovers the true posterior.

    Authors: We acknowledge the value of a theoretical guarantee. The manuscript supplies empirical support: the closed loop is run until the prior mean stabilizes, and the resulting posteriors are compared directly to long-run MCMC on all test problems, showing no detectable bias beyond sampling error. We have added an ablation (new Figure 7 and Table 5) that freezes the prior mean after the first iteration while keeping VF and FNO adaptation active; the ablation exhibits higher error and slower convergence, confirming the update's contribution. A formal contraction mapping or bias bound for the joint VF-FNO iteration is not derived in the current work. revision: partial

standing simulated objections not resolved
  • A rigorous contraction mapping or bias bound establishing that the iterative prior-mean update inside the closed VF-FNO loop recovers the true posterior without systematic bias.

Circularity Check

0 steps flagged

No circularity: empirical validation of proposed adaptive loop stands independent of inputs.

full rationale

The paper introduces a VF-based dimension reduction with iterative prior updating and FNO surrogate in a closed loop, then reports numerical accuracy on Rosenbrock and PDE inverse problems against MCMC/UKI/SVGD baselines. No equations, predictions, or uniqueness claims are shown to reduce by construction to fitted parameters, self-citations, or renamed inputs. The performance claims rest on external test cases rather than any self-referential derivation. This is the normal non-circular outcome for a methodological proposal with held-out empirical checks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only the abstract is available, so no concrete free parameters, axioms, or invented entities can be extracted with certainty. The VF model itself appears to be the central new construction.

pith-pipeline@v0.9.1-grok · 5771 in / 1163 out tokens · 31935 ms · 2026-06-29T08:51:53.651618+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

9 extracted references · 3 canonical work pages · 2 internal anchors

  1. [1]

    Deep Adaptive Dimension Reduction for Bayesian Inference in Inverse Problems

    Introduction.Bayesian inverse problems governed by partial differential equations (PDEs) arise widely in science and engineering, including subsurface flow modeling, medical imaging, and climate science [30, 34]. The primary task is to recover unknown parameters from noisy, incomplete observations by characterizing the pos- terior distribution. In practic...

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  2. [2]

    Preliminaries. 2.1. Bayesian Inverse Problems.Let Ω s ⊂R n be a bounded spatial domain andξ∈Ω p ⊂R d be the unknown parameter vector. We consider physical systems governed by parametric partial differential equations: L(u(x;m ξ(x));m ξ(x)) =s(x),∀(x,ξ)∈Ω s ×Ω p,(2.1) B(u(x;m ξ(x));m ξ(x)) =g(x),∀(x,ξ)∈∂Ω s ×Ω p,(2.2) wherem ξ(x) is the coefficient field c...

  3. [3]

    dual-flow

    Variational Flow. 3.1. Background: VAEs and Normalizing Flows. 3.1.1. Variational Autoencoder (VAE).To approximate the true distribu- tionp x of datax∈R d, VAE [19] introduces a latent variablez∈R k (k < dfor dimension reduction) and optimizes the Evidence Lower Bound (ELBO): (3.1)L θ,ϕ(x) =−D KL qz|x,ϕ pz + Z qz|x,ϕ logp x|z,θ dz, where the decoder, enco...

  4. [4]

    Deep Adaptive Bayesian Inference. 4.1. Overall Framework.The deep adaptive framework addresses two coupled challenges: the out-of-distribution (OOD) inaccuracy of the surrogate and the diffi- culty of specifying an appropriate prior mean. We tackle both through an alternating optimization loop between the VF posterior approximation and the surrogate model...

  5. [5]

    Numerical Experiments. 5.1. Classes of Problem Studied.The performance of the proposed frame- work is evaluated on two types of problems. First, to specifically assess the posterior approximation capability of the VF model, we consider a challenging high-dimensional synthetic distribution. Second, to validate the complete framework, which incorpo- rates b...

  6. [6]

    To isolate the generative performance of the models, the iterative prior updating module is not employed in this example

    A 100-dimensional Rosenbrock inverse problem:In Subsection 5.2, we utilize this problem to evaluate the capability of various methods in ap- proximating complex, non-Gaussian posterior distributions. To isolate the generative performance of the models, the iterative prior updating module is not employed in this example. The numerical results demonstrate t...

  7. [7]

    The unknown fieldm ξ(x) is mod- eled as a Gaussian random field parameterized via a truncated Karhunen- Lo` eve (KL) expansion

    PDE-governed inverse problem:In Subsections 5.3, 5.4, and 5.5, we evaluate our framework on three PDE-governed inverse problems, where ob- servations are corrupted by relative Gaussian noise: (5.1)y=G(ξ ref) +δmax |G(ξref)| η,η∼ N(0,I), with noise amplitudeδ∈ {1%,5%,10%}. The unknown fieldm ξ(x) is mod- eled as a Gaussian random field parameterized via a ...

    work page arXiv 2073

  8. [8]

    The vectorξ∈R d forms the finite-dimensional parameter space for inference

    +τ 2 −l . The vectorξ∈R d forms the finite-dimensional parameter space for inference. Here, we infer the log-permeability field from 36 uniformly spaced pressure observa- tions. The forward PDE is solved numerically using a 71×71 spatial grid. Fig. 4: Surrogate fitting errore S across adaptive stages of 2D Darcy Flow problem. Columns: noise levelsδ∈ {1%,5...

  9. [9]

    Conclusions.We present a deep adaptive dimension-reduction Bayesian in- ference framework for high-dimensional PDE-governed inverse problems, addressing non-Gaussian posteriors, surrogate out-of-distribution (OOD) errors, and prior mis- specification. Our core Variational Flow (VF) model integrates VAE-based dimen- sionality reduction with dual normalizin...

    work page internal anchor Pith review Pith/arXiv arXiv 2021