Pith. sign in

REVIEW 3 major objections 5 minor 33 references

Cost-effective Reduced-Order Modeling via Bayesian Active Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read BayPOD-AL claims that actively querying the most error-relevant full-order simulations makes reduced-order surrogate learning several times cheaper, with an error-guided acquisition rule outperforming both uncertainty-guided and random…

desk verdict Plausible AL-for-ROM combination with a real early-step win; the 3-4x cost-saving headline isn't in the data shown. read the letter →

arxiv 2506.22645 v1 pith:YQV5A6WO submitted 2025-06-27 cs.LG stat.ML

Amir Hossein Rahmati , Nathan M. Urban , Byung-Jun Yoon , Xiaoning Qian This is my paper · ORCID
classification cs.LGstat.ML
keywords activelearningreduced-ordermodelsproperorthogonaldecompositionBayesianinferenceuncertaintyquantificationsampleefficiencyerror-boundedacquisitionsurrogatemodeling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that building reduced-order surrogate models of expensive physics simulations can be made much cheaper by choosing which simulation runs to collect, rather than collecting a large random dataset. It proposes BayPOD-AL, an active learning loop wrapped around Bayesian proper orthogonal decomposition, in which each step queries the full-order simulator at the input parameter judged most informative. Two query strategies are compared: one guided by predictive variance, and one guided by an upper bound on the expected approximation error. The paper reports that the error-guided strategy keeps mean squared error far below the uncertainty-guided and random baselines for the first ten query rounds, and reaches the same accuracy with roughly three to four times less training data.

What carries the argument

The load-bearing object is the acquisition function $a^{(i)}_{p_U}$, defined as the space-time average of an informativeness measure $M^{(i,t)}(q(\cdot|D_L, p_U, x))$, with the next query chosen as $p^* = \arg\max_i a^{(i)}_{p_U}$. For BayPOD-EAL the measure is an upper bound $U^{(t)}(p_U, x)$ on the expected squared approximation error, derived by treating the ground-truth full-order solution as a Gaussian process with a symmetric, positive-definite, translation-invariant, twice-differentiable kernel and bounded variance. For BayPOD-UAL the measure is the predictive posterior variance estimated by Monte Carlo sampling. Both ride on BayPOD, a variational-Bayesian proper orthogonal decomposition model whose posterior over POD coefficients and bases supplies the uncertainty, and whose physics embedding (a particular solution plus homogeneous-boundary POD basis) provides the reduced-order form.

What would settle it

Compute BayPOD-EAL's claimed error upper bound $U^{(t)}(p_U, x)$ for held-out heat-rod diffusivities and compare it with the actual squared prediction error at the points the acquisition selects; if the bound is violated there, or if the same active-learning loop applied to a system whose output is clearly not a Gaussian process with the assumed kernel shows no advantage over random sampling, the load-bearing assumption fails.

Watch

Extended reading notes

Core claim

The central claim is that, for reduced-order modeling of a parameterized heat-diffusion system, an active learning acquisition function that directly targets the ROM's approximation error rather than raw model uncertainty is markedly more sample-efficient. Concretely, after 5 active learning iterations (250 new full-order snapshots), BayPOD-EAL achieves the best empirical performance, and over the first 10 iterations it reports on average 4.7x and 9x lower MSE than BayPOD-UAL and 6x and 6.5x lower MSE than random sampling on low- and high-temporal-resolution test sets. After 15 to 20 iterations, BayPOD-EAL and random sampling converge to comparable performance, which the authors read as a factor-of-3-to-4 reduction in the computational cost of assembling the training dataset. The same advantage holds when test data have a temporal resolution four times finer than the training snapshots.

Load-bearing premise

The approach assumes the true simulator output can be modeled as a Gaussian process with a smooth, translation-invariant kernel and bounded variance; if that fails for a given physical system, the error upper bound guiding BayPOD-EAL may not be a genuine bound.

Editorial extensions

If this is right

  • BayPOD-EAL can cut the number of full-order simulator runs needed to train a POD-based ROM by a factor of 3 to 4 while eventually matching random-sampling accuracy.
  • Predictive uncertainty alone is a weak guide for ROM active learning: BayPOD-UAL beats random sampling initially but stagnates or degrades, consistent with model mismatch between the surrogate and the true system.
  • Objective-driven, error-targeting acquisition transfers to test settings with four times finer temporal resolution than the training data, showing that resolution mismatch is not fatal.
  • The active-learning gain is largest in the first 5 to 10 query rounds, i.e. exactly when data are scarcest and each full-order simulation is most expensive.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension of this result is that the same error-bounded acquisition recipe could be applied to other parameterized PDE or ODE emulation problems, with the caveat that the Gaussian-process error-bound assumptions must be checked for each new system.
  • The reported factor-of-3-to-4 saving is measured in simulator runs, not wall-clock time; the practical benefit would compound when each full-order simulation is expensive and shrink if acquisition overhead dominates.
  • A batch variant that queries several input parameters per round, or combines the error bound with a diversity constraint, might close the gap between the 10-iteration advantage and the 15-to-20-iteration convergence point with random sampling.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes BayPOD-AL, an active learning framework for reduced-order modeling based on Bayesian proper orthogonal decomposition (BayPOD). Two acquisition strategies are evaluated: BayPOD-UAL, which selects inputs by posterior predictive variance, and BayPOD-EAL, which selects inputs using an estimated error upper bound inspired by Savvides et al. (2024). Experiments on a one-dimensional heat-rod problem compare these strategies against random sampling over ten active learning iterations, reporting MSE on low- and high-temporal-resolution test sets. The authors claim that BayPOD-EAL outperforms the alternatives over the first ten iterations and that a 3-4x reduction in full-order-model simulation cost is achieved because EAL reaches comparable performance to random sampling after 15-20 iterations.

Significance. If the central claims hold, the paper makes a useful contribution by demonstrating that active learning can reduce the number of expensive full-order-model simulations needed to train Bayesian reduced-order surrogates. The comparison against both uncertainty-based and random acquisition, the use of five random initializations with reported means and standard deviations, and the evaluation under a temporal-resolution mismatch are strengths. The main significance is limited by two issues: the headline cost-reduction factor relies on data that are not shown, and the error-guided acquisition function is not fully specified. The paper does not provide code or data, which further hampers reproducibility.

major comments (3)
  1. [Section 3, final paragraph] The claim that "after 15 to 20 AL iterations, both BayPOD-EAL and random sampling provide comparable performance, meaning that BayPOD-AL reduces the computational cost related to training data by a factor of 3 to 4" is not supported by the evidence in the paper. Figure 2 plots AL steps 0 through 10 only, and no table or figure reports MSE for iterations 11-20. Moreover, the inference requires a well-defined target MSE and a comparison of the iteration at which each method first reaches that target; comparable performance at iterations 15-20 does not by itself imply that EAL at iteration 5 matches random sampling at iteration 15-20. Because the abstract and conclusion emphasize cost-effectiveness, this is a load-bearing gap. Please add results for AL steps 11-20 or recast the claim using a threshold-based comparison of the number of FOM simulations needed to reach a specified error.
  2. [Appendix C and Eq. (2)] The error-guided acquisition function is not reproducible because the measure function U(t)(pU, x) is never defined. Appendix C states "we set M = U in (2)" and refers to Savvides et al. for an upper-bound result, but no explicit formula for U is provided, nor is it explained how the BayPOD variational posterior is used to compute the bound in practice. Since BayPOD-EAL is the principal methodological contribution, please provide the full definition of U, including any kernel hyperparameters, and outline its computation from q(·|DL).
  3. [Appendix C and Appendix D] The theoretical justification of BayPOD-EAL rests on the assumptions from Savvides et al. that the ground-truth function can be modeled as a Gaussian process with a symmetric, positive definite, translation-invariant, twice-differentiable kernel and bounded variance. The heat-rod problem includes a particular solution with a linear steady-state component and a time-dependent Dirichlet boundary condition f(x=0,t)=3 sin(2t), and it is not verified that the required assumptions hold for this system. If U is not a valid upper bound for the actual error, the motivation for calling the method "error-guided" is weakened, even though the empirical performance could still be valid. Please verify the bound empirically on held-out FOM solutions (for example, by comparing U with actual squared errors) or explicitly state that U is used as a heuristic proxy rather than a certified bound.
minor comments (5)
  1. [Author affiliations] There is a typo in the affiliation: "Brookheaven National Laboratory" should be "Brookhaven National Laboratory."
  2. [Section 1, Introduction] The word "interoperability" in the final sentence of the first paragraph is likely intended to be "interpretability"; please check the wording.
  3. [Eq. (2)] The index range "i ∈ {0, . . . , np}" appears to be off by one if np denotes the number of unlabeled inputs; it should presumably run over 1, . . . , np. Also, the symbol np is used both for the number of inputs and for the number of time points (np_t), which is confusing; please disambiguate.
  4. [Figure 2] The legend includes "Empirical Minimum," but the text never defines what this curve represents; please add a definition in Section 3 or in the figure caption.
  5. [Table 1] The caption says "at the 6th AL iterations" while the text says "after 5 AL iterations (250 new snapshots)"; please clarify whether AL steps are indexed from zero or one, and make the caption consistent with the text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the acquisition functions derive from the BayPOD posterior or an external GP error bound, and the reported test MSE is measured on held-out FOM snapshots.

full rationale

BayPOD-AL's central derivation is the AL acquisition loop: BayPOD's variational posterior supplies predictive variance (UAL) or feeds a Savvides et al. error upper bound (EAL); the acquisition function (2)-(3) selects FOM parameters, and test MSE is computed on held-out FOM snapshots. No equation in the paper defines the reported low-resolution or high-resolution MSE in terms of the acquisition values, and no fitted parameter is renamed as a prediction. The error bound U(t)(pU,x) is adopted from an external AISTATS result, not derived here, so its use does not reduce the paper's claims to its own inputs. BayPOD and Rahmati et al. are self-citations, but BayPOD is a separately published method with its own published inference derivation, so reusing it is normal method composition rather than circularity. The main substantive weaknesses are correctness concerns, not circularity: the Section 3 claim that BayPOD-EAL and random sampling become comparable after 15-20 AL iterations is not evidenced by Figure 2, which stops at 10 iterations, and the Appendix C GP assumption underlying U is not verified for the heat rod system. These concerns affect the strength of the empirical cost-reduction claim but do not make any prediction equivalent by construction to its inputs.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claims depend on hand-chosen hyperparameters (K, network size, batch sizes), a strong Gaussian-process assumption for the error bound, and the quality of variational inference. There are no newly invented physical entities.

free parameters (4)
  • K (POD basis dimension) = 7
    Hand-chosen to balance accuracy and dimensionality; not derived from data.
  • NN architecture for coefficient mapping = 2 hidden layers, 50 nodes, ReLU
    Taken from Boluki et al. 2024 without a sensitivity study.
  • Initial labeled set size = 50 snapshots
    Starting DL size, hand-chosen for the active learning loop.
  • Acquisition batch size = 50 snapshots per AL step
    Number of fixed time points used for each queried diffusivity parameter, hand-chosen.
assumptions (3)
  • domain assumption The full-order model solution can be represented as a Gaussian process with a symmetric positive definite, translation-invariant, twice-differentiable kernel and bounded variance.
    Required for the Savvides et al. error bound used in BayPOD-EAL (Appendix C); not verified for the heat rod solution, which contains a linear steady state and time-dependent boundary terms.
  • domain assumption Mean-field variational inference provides a sufficiently accurate posterior for acquisition decisions.
    BayPOD's posterior is approximated via mean-field variational inference; the quality of this approximation directly affects reliability of the uncertainty and error estimates.
  • ad hoc to paper A low-rank POD basis with K=7 captures the system dynamics well enough for meaningful error comparison.
    The choice of K is made without an energy retention criterion or sensitivity analysis, yet it determines the expressiveness of the ROM and the acquisition signal.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Cost-effective Reduced-Order Modeling via Bayesian Active Learning." pith.science (2026). https://pith.science/paper/YQV5A6WO

@misc{pith2026250622645,
  author       = {Pith},
  title        = {Pith review of: Cost-effective Reduced-Order Modeling via Bayesian Active Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YQV5A6WO}},
  note         = {Machine review of arXiv:2506.22645}
}
read the original abstract

Machine Learning surrogates have been developed to accelerate solving systems dynamics of complex processes in different science and engineering applications. To faithfully capture governing systems dynamics, these methods rely on large training datasets, hence restricting their applicability in real-world problems. In this work, we propose BayPOD-AL, an active learning framework based on an uncertainty-aware Bayesian proper orthogonal decomposition (POD) approach, which aims to effectively learn reduced-order models from high-fidelity full-order models representing complex systems. Experimental results on predicting the temperature evolution over a rod demonstrate BayPOD-AL's effectiveness in suggesting the informative data and reducing computational cost related to constructing a training dataset compared to other uncertainty-guided active learning strategies. Furthermore, we demonstrate BayPOD-AL's generalizability and efficiency by evaluating its performance on a dataset of higher temporal resolution than the training dataset.

Figures

Figures reproduced from arXiv: 2506.22645 by the authors.

Figure 1
Figure 1. A schematic illustration of the proposed BayPOD-AL framework. efficiency in this work. 2.3 BayPOD-AL We now present our active learning framework, BayPOD￾AL, by exploring different uncertainty-based active learn￾ing (UAL) strategies, leveraging the inherent UQ capa￾bilities of BayPOD. Consider DL and DU the iteratively updated ‘labeled’ and ‘unlabeled’ datasets correspond￾ing to collected snapshots from high-fidelit… view at source ↗
Figure 2
Figure 2. Performance comparison of BayPOD￾EAL, BayPOD-UAL, and random sampling strate￾gies on ‘low’ (left) and ‘high’ (right) temporal resolution test datasets. We train BayPOD on the temporally low￾resolution training snapshots and report the AL performances on both temporally low-(n p t = 50) and high-resolution (n p t = 200) test datasets [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 16 canonical work pages

  1. [1]

    J. T. Ash, C. Zhang, A. Krishnamurthy, J. Langford, and A. Agarwal. Deep batch active learning by diverse, uncertain gradient lower bounds, 2020. URL https://arxiv.org/abs/1906.03671

    arXiv 2020

  2. [2]

    Benner, S

    P. Benner, S. Gugercin, and K. Willcox. A survey of projection-based model reduction methods for parametric dynamical systems. SIAM Review, 57 0 (4): 0 483--531, 2015. doi:10.1137/130932715. URL https://doi.org/10.1137/130932715

    doi:10.1137/130932715 2015

  3. [3]

    Besselink, U

    B. Besselink, U. Tabak, A. Lutowska, N. van de Wouw , H. Nijmeijer, D. Rixen, M. Hochstenbach, and W. Schilders. A comparison of model reduction techniques from structural dynamics, numerical mathematics and systems and control. Journal of Sound and Vibration, 332 0 (19): 0 4403--4422, 2013. ISSN 0022-460X. doi:https://doi.org/10.1016/j.jsv.2013.03.025. U...

    work page doi:10.1016/j.jsv.2013.03.025 2013

  4. [4]

    Boluki, X

    S. Boluki, X. Qian, and E. R. Dougherty. Experimental design via generalized mean objective cost of uncertainty. IEEE Access, 7: 0 2223--2230, 2019. doi:10.1109/ACCESS.2018.2886576

    arXiv 2019

  5. [5]

    Boluki, S

    S. Boluki, S. Z. Dadaneh, E. R. Dougherty, and X. Qian. Bayesian proper orthogonal decomposition for learnable reduced-order models with uncertainty quantification. IEEE Transactions on Artificial Intelligence, 5 0 (3): 0 1162--1173, 2024. doi:10.1109/TAI.2023.3268609

    arXiv 2024

  6. [6]

    A. M. DeGennaro, N. M. Urban, B. T. Nadiga, and T. Haut. Model structural inference using local dynamic operators. International Journal for Uncertainty Quantification, 9 0 (1): 0 59–83, 2019. ISSN 2152-5080. doi:10.1615/int.j.uncertaintyquantification.2019025828. URL http://dx.doi.org/10.1615/Int.J.UncertaintyQuantification.2019025828

    work page doi:10.1615/int.j.uncertaintyquantification.2019025828 2019

  7. [7]

    M. Guo, S. A. McQuarrie, and K. E. Willcox. Bayesian operator inference for data-driven reduced-order modeling. Computer Methods in Applied Mechanics and Engineering, 402: 0 115336, 2022. ISSN 0045-7825. doi:https://doi.org/10.1016/j.cma.2022.115336. URL https://www.sciencedirect.com/science/article/pii/S0045782522004273. A Special Issue in Honor of the L...

    work page arXiv 2022

  8. [8]

    Hacohen, A

    G. Hacohen, A. Dekel, and D. Weinshall. Active learning on a budget: Opposite strategies suit high and low budgets, 2022. URL https://arxiv.org/abs/2202.02794

    arXiv 2022

Show all 33 references
  1. [9]

    Hesthaven and S

    J. Hesthaven and S. Ubbiali. Non-intrusive reduced order modeling of nonlinear problems using neural networks. Journal of Computational Physics, 363: 0 55--78, 2018. ISSN 0021-9991. doi:https://doi.org/10.1016/j.jcp.2018.02.037. URL https://www.sciencedirect.com/science/articl...

    2018 doi

  2. [10]

    H. Hino. Active learning: Problem settings and recent developments, 2020. URL https://arxiv.org/abs/2012.04225

    2020 arXiv

  3. [11]

    S. M. Hirsh, D. A. Barajas-Solano, and J. N. Kutz. Sparsifying priors for bayesian uncertainty quantification in model discovery, 2021. URL https://arxiv.org/abs/2107.02107

    2021 arXiv

  4. [12]

    Houlsby, F

    N. Houlsby, F. Huszár, Z. Ghahramani, and M. Lengyel. Bayesian active learning for classification and preference learning, 2011. URL https://arxiv.org/abs/1112.5745

    2011 arXiv

  5. [13]

    Lagaris, A

    I. Lagaris, A. Likas, and D. Fotiadis. Artificial neural networks for solving ordinary and partial differential equations. IEEE Transactions on Neural Networks, 9 0 (5): 0 987–1000, 1998. ISSN 1045-9227. doi:10.1109/72.712178. URL http://dx.doi.org/10.1109/72.712178

    1998 doi

  6. [14]

    Munjal, N

    P. Munjal, N. Hayat, M. Hayat, J. Sourati, and S. Khan. Towards robust and reproducible active learning using neural networks, 2022. URL https://arxiv.org/abs/2002.09564

    2022 arXiv

  7. [15]

    P. Pant, R. Doshi, P. Bahl, and A. Barati Farimani. Deep learning for reduced order modelling and efficient temporal evolution of fluid simulations. Physics of Fluids, 33 0 (10), Oct. 2021. ISSN 1089-7666. doi:10.1063/5.0062546. URL http://dx.doi.org/10.1063/5.0062546

    2021 doi

  8. [16]

    T. Penzl. Algorithms for model reduction of large dynamical systems. Linear Algebra and its Applications, 415 0 (2): 0 322--343, 2006. ISSN 0024-3795. doi:https://doi.org/10.1016/j.laa.2006.01.007. URL https://www.sciencedirect.com/science/article/pii/S0024379506000371. Specia...

    2006 doi

  9. [17]

    A. H. Rahmati, M. Fan, R. Zhou, N. M. Urban, B.-J. Yoon, and X. Qian. Understanding uncertainty-based active learning under model mismatch, 2024. URL https://arxiv.org/abs/2408.13690

    2024 arXiv

  10. [18]

    M. Raissi. Deep hidden physics models: Deep learning of nonlinear partial differential equations, 2018. URL https://arxiv.org/abs/1801.06637

    2018 arXiv

  11. [19]

    Raissi, P

    M. Raissi, P. Perdikaris, and G. E. Karniadakis. Physics informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations, 2017. URL https://arxiv.org/abs/1711.10561

    2017 arXiv

  12. [20]

    Raissi, P

    M. Raissi, P. Perdikaris, and G. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378: 0 686--707, 2019. ISSN 0021-9991. doi:h...

    2019 doi

  13. [21]

    P. Ren, Y. Xiao, X. Chang, P.-Y. Huang, Z. Li, B. B. Gupta, X. Chen, and X. Wang. A survey of deep active learning, 2021

    2021

  14. [22]

    S. H. Rudy, S. L. Brunton, J. L. Proctor, and J. N. Kutz. Data-driven discovery of partial differential equations, 2016. URL https://arxiv.org/abs/1609.06401

    2016 arXiv

  15. [23]

    Saifullah, S

    S. Saifullah, S. Agne, A. Dengel, and S. Ahmed. Analyzing the potential of active learning for document image classification, 11 2022

    2022

  16. [24]

    Savvides, H

    R. Savvides, H. Luu, and K. Puolam \"a ki. Error bounds for any regression model using G aussian processes with gradient information. Proceedings of Machine Learning Research, 238: 0 397--405, 2024. ISSN 2640-3498. URL http://aistats.org/aistats2024/. International Conference ...

    2024

  17. [25]

    B. Settles. Active learning literature survey. 2009

    2009

  18. [26]

    Swischuk, L

    R. Swischuk, L. Mainini, B. Peherstorfer, and K. Willcox. Projection-based model reduction: Formulations for physics-based machine learning. Computers & Fluids, 179: 0 704--717, 2019. ISSN 0045-7930. doi:https://doi.org/10.1016/j.compfluid.2018.07.021. URL https://www.scienced...

    2019 doi

  19. [27]

    S. Verma. A survey on machine learning applied to dynamic physical systems, 2020. URL https://arxiv.org/abs/2009.09719

    2020 arXiv

  20. [28]

    Wang and Y

    D. Wang and Y. Shang. A new active labeling method for deep learning. In 2014 International Joint Conference on Neural Networks (IJCNN), pages 112--119, 2014. doi:10.1109/IJCNN.2014.6889457

    2014

  21. [29]

    J. Wu, J. Chen, and D. Huang. Entropy-based active learning for object detection with progressive diversity constraint, 2022. URL https://arxiv.org/abs/2204.07965

    2022 arXiv

  22. [30]

    B.-J. Yoon, X. Qian, and E. R. Dougherty. Quantifying the objective cost of uncertainty in complex dynamical systems. IEEE Transactions on Signal Processing, 61 0 (9): 0 2256--2266, 2013. doi:10.1109/TSP.2013.2251336

    2013

  23. [31]

    B.-J. Yoon, X. Qian, and E. R. Dougherty. Quantifying the multi-objective cost of uncertainty. IEEE Access, 9: 0 80351--80359, 2021. doi:10.1109/ACCESS.2021.3085486

    2021

  24. [32]

    G. Zhao, E. Dougherty, B.-J. Yoon, F. J. Alexander, and X. Qian. Bayesian active learning by soft mean objective cost of uncertainty. In A. Banerjee and K. Fukumizu, editors, Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, volume 130...

    2021

  25. [33]

    Zhu and N

    Y. Zhu and N. Zabaras. Bayesian deep convolutional encoder–decoder networks for surrogate modeling and uncertainty quantification. Journal of Computational Physics, 366: 0 415–447, Aug. 2018. ISSN 0021-9991. doi:10.1016/j.jcp.2018.04.018. URL http://dx.doi.org/10.1016/j.jcp.20...

    2018 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.