Pith. sign in

REVIEW 4 major objections 5 minor 65 references

Single- to multi-fidelity history-dependent learning with uncertainty quantification and disentanglement: application to data-driven constitutive modeling

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single hierarchical framework learns history-dependent material behavior from noisy or noiseless, single- or multi-fidelity data while quantifying and separating epistemic from aleatoric uncertainty.

desk verdict Useful multi-fidelity recurrent framework with a genuinely useful hidden-state transfer result, but the UQ disentanglement claim is only as strong as its untested Gaussianity/bias assumption. read the letter →

arxiv 2507.13416 v1 pith:UR5CNNDT submitted 2025-07-17 cs.LG cs.AI

Jiaxiang Yi , Bernardo P. Ferreira , Miguel A. Bessa This is my paper · ORCID
classification cs.LGcs.AI
keywords multi-fidelitylearningvarianceestimationBayesianrecurrentneuralnetworksuncertaintydisentanglementaleatoricepistemichistory-dependentconstitutivemodelingself-consistentclusteringanalysisdata-drivenplasticity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Data-driven constitutive modeling—learning a material's stress response from strain histories—usually requires large, expensive high-fidelity datasets. This paper argues that one hierarchical framework, built on variance estimation Bayesian recurrent neural networks (VeBRNNs), can instead learn from any combination of single- or multi-fidelity data, noisy or noiseless, while reporting both the model's epistemic uncertainty and the inherent data noise (aleatoric uncertainty). The method is modular: simpler deterministic RNNs and plain Bayesian RNNs emerge as special cases. Across four test scenarios—noisy single-fidelity, deterministic two-fidelity, noisy two-fidelity, and mixed noisy/noiseless two-fidelity—the framework predicts the mean stress response accurately, recovers near-zero noise when data is deterministic, and discovers the correct heteroscedastic noise distribution when data is noisy. If correct, this lets engineers substitute cheap low-fidelity simulations for expensive high-fidelity ones without losing uncertainty awareness.

What carries the argument

The central object is the variance estimation Bayesian recurrent neural network (VeBRNN), whose cooperative training (Algorithm 1) fixes the mean network, then trains a variance network on the Gamma-distributed squared residuals to capture heteroscedastic aleatoric variance, then runs pSGLD-based Bayesian inference to update the mean and produce epistemic variance. Its load-bearing identity is the variance decomposition $\mathrm{Var}(y) = s^2(x) + \mathrm{Var}_{p(\theta|D)}[f(x)]$ (Eq. 42), which splits total predictive variance into aleatoric and epistemic parts. For multi-fidelity learning, the framework combines a low-fidelity RNN whose hidden state is fed into a high-fidelity residual RNN (MF-Residual-Hidden), implementing $f^h(x) = f^l(x) + r(x)$.

What would settle it

Take a synthetic history-dependent dataset with known ground-truth noise that is deliberately non-Gaussian (for instance, Student-t residuals with 3 degrees of freedom or a skewed distribution), train a VeBRNN with the same algorithm, and compare the predicted aleatoric variance against the true generative variance: if the predicted aleatoric distribution diverges from the truth or the residual quantile-quantile plot rejects Gaussianity while epistemic intervals stay narrow, the disentanglement claim fails.

Watch

Extended reading notes

Core claim

The central claim is that VeBRNNs—recurrent networks trained cooperatively in three steps: deterministic mean fitting, deterministic variance-network fitting for aleatoric noise, and Bayesian posterior sampling via pSGLD for epistemic uncertainty—can be embedded into a multi-fidelity architecture $f^h(x) = g(f^l(x), x) + r(x)$, and that this combination learns history-dependent constitutive behavior from data of mixed fidelity and noise. The paper shows the best performing configuration transfers the hidden state of the low-fidelity recurrent network into the high-fidelity residual network rather than transferring decoded stress outputs. On four datasets generated from finite-element (DNS) and self-consistent clustering (SCA) simulations of elastoplastic microstructures, the method matches or beats single-fidelity models at equal total computational cost, especially for out-of-distribution paths, and its predicted aleatoric distributions align with ground-truth noise (or collapse to zero variance on noiseless data), while epistemic uncertainty narrows as training data grows.

Load-bearing premise

The load-bearing premise is that the data noise is Gaussian with diagonal covariance and that fixing the aleatoric variance (Step 2) while updating the mean via Bayesian inference (Step 3) is enough to keep the two uncertainties disentangled; if the noise is truly non-Gaussian or the Step-1 mean is biased, the aleatoric/epistemic split degrades, and the paper does not quantify this coupling error.

Editorial extensions

If this is right

  • Engineers can train uncertainty-aware constitutive models by combining a small number of expensive high-fidelity simulations with many cheap low-fidelity ones, with accuracy comparable to or better than high-fidelity-only training at the same total budget.
  • When data is noiseless, the method discovers near-zero aleatoric variance, so it can be used even when the analyst is unsure whether noise is present.
  • Epistemic uncertainty decreases as training data grows and widens for out-of-distribution inputs, providing a calibrated 95% prediction interval (PICP near 0.95) for in-distribution data.
  • The same framework reduces to simpler deterministic or Bayesian networks when only mean predictions or only epistemic uncertainty are needed, so the method is a single generalization rather than a separate new model for each scenario.
  • Low-fidelity data improves out-of-distribution generalization: MF models shrink the gap between in-distribution and out-of-distribution relative error compared with single-fidelity models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the Gaussian likelihood assumption in Eq. (2) fails, the variance network will absorb any non-Gaussian residual structure into the aleatoric estimate; a straightforward test is to train on synthetic residuals drawn from a Student-t or skewed distribution and check whether the aleatoric/epistemic split recovers the known generative variance.
  • The hidden-state transfer trick that beat decoded-output transfer here may carry over to other sequence-to-sequence surrogate problems (e.g., dynamics of soft robots, battery cycling), since the hidden state encodes a compact summary of the entire history.
  • The reported heuristic that 30–70% of the total budget should go to low-fidelity data suggests an immediate active-learning strategy: allocate the next sample to whichever fidelity reduces expected epistemic variance most, which the authors note but do not implement.
  • The framework's modularity implies it should extend to more than two fidelities by chaining the same $f^h = g(f^l) + r$ recursion, as the flowchart in Figure 4 indicates, though the paper only demonstrates two-fidelity cases.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a hierarchical multi-fidelity learning framework for history-dependent constitutive modeling, in which deterministic recurrent networks, variance-estimation recurrent networks, and Bayesian recurrent networks (VeBRNNs) can be combined or ablated. The framework trains a low-fidelity model first, then transfers its hidden state to a high-fidelity residual model, and optionally estimates aleatoric and epistemic uncertainties through a cooperative three-step training scheme. Four datasets generated from RVE/SVE simulations with FEA and SCA cover single-fidelity noisy data, deterministic multi-fidelity data, noisy multi-fidelity data, and mixed noisy-LF/deterministic-HF data. The reported experiments compare the proposed models against single-fidelity baselines under a total-cost budget and validate mean predictions against RVE-based DNS ground truth and noise statistics against 100-repeat SVE simulations.

Significance. If the central claims hold, the paper is significant for data-driven mechanics: it offers a modular architecture spanning deterministic RNNs, BRNNs, and multi-fidelity Bayesian models, demonstrates the approach on physically generated datasets with external ground-truth means and noise statistics, and is backed by open-source tools (CRATE). The ablation of four multi-fidelity transfer configurations is also useful and goes beyond a single architectural choice. The main caveat is that the headline capability—accurately predicting the mean while discovering the noise distribution and separating it from epistemic uncertainty—rests on a Gaussian likelihood and an unbiased Step-1 mean network, and neither condition is tested in the paper. This is a fixable gap, but it currently limits the scope of the central claim.

major comments (4)
  1. [§3.1, Algorithm 1, Eqs. (2), (36), (39), (42)] The disentanglement claim in Eq. (42) is load-bearing but its identifiability is assumed rather than demonstrated. Step 2 of Algorithm 1 fits the variance network to squared residuals under a Gamma log-likelihood (Eq. (36)) while holding the Step-1 mean fixed; if that mean is biased, which is the generic case for finite data, the fitted variance absorbs the squared bias. Step 3 then freezes s^2(x;phi) and computes the epistemic variance conditionally on it, so a biased Step-1 mean can contaminate both uncertainty components. In addition, the likelihood in Eq. (2) and the Gamma derivation in §D.2 presuppose Gaussian residuals, and the evaluation metric in Eq. (28) is the closed-form Wasserstein distance between Gaussians. The paper never reports normality checks on the 100-repeat SVE residuals. The abstract's claim that the method "discovers the noise distribution" is therefore supported only for the special case of near-unbiased means and Gaussian noise. I ask the authors to add residual-Gaussianity diagnostics (QQ plots or tests computed from the 100 SVE repeats), a nonparametric or distribution-free distance for W_A, and a numerical experiment or analysis quantifying how Step-1 bias propagates into the estimated aleatoric and epistemic variances; if such an experiment is not feasible, the claims should be narrowed accordingly.
  2. [§3.2] The sentence "To the best of our knowledge, RNNs have not yet been considered in a MF setting" is contradicted by the paper's own reference [52], Conti et al., "Multi-fidelity surrogate modeling using long short-term memory networks" (CMAME, 2023). The novelty statement should be revised to say that the proposed combination of recurrent architectures with the transfer-plus-residual framework, or the Bayesian/UQ extension, is new; otherwise the contribution is overstated. This is not merely cosmetic, because the multi-fidelity recurrent architecture is presented as a headline contribution.
  3. [§4] Every comparison in Section 4 is reported as a single run with no variability over training seeds. The claims that VeBRNN and RNN have "comparable" relative error, or that PICP "achieves" the target 0.95, are point estimates that could change substantially with another initialization; the posterior samples in Step 3 do not randomize the deterministic Step-1 and Step-2 fits. I ask for means and spreads over several independent training runs, or a justification that the reported differences are larger than seed noise. Relatedly, the budget comparisons in Figures 16 and 17 use the fixed cost ratios in Table 1; since these ratios are approximate and path-dependent, the conclusions about the optimal LF-budget fraction (30–70%) should be accompanied by a sensitivity analysis over the cost ratios.
  4. [Algorithm 1] Algorithm 1 selects the optimal model by i* = arg max_i LMglk[i], but no definition or computation details for the log marginal likelihood are given in the paper or the appendices. Since this selection directly determines the reported predictions, please provide the estimator used (for example a harmonic-mean approximation or Laplace approximation) or state explicitly that the criterion is only a heuristic. Without this, the model-selection step is not reproducible from the manuscript.
minor comments (5)
  1. [Footnote 7] Footnote 7 lists the stress components as "σ11, σ12, and σ2"; the last component should be σ22.
  2. [Table 1] The cost-ratio row appears to have one fewer entry than the six fidelity columns, and the first entry is printed as "1.01/20". This makes the cost ratios difficult to parse; please reformat the row so each column is unambiguous.
  3. [Figure 12] The caption states Tc = 10/20 + 2000/120 = 17.16, but the arithmetic gives 10/20 + 2000/120 = 17.17; please correct the rounding inconsistency.
  4. [§D.3] Step 3 says the posterior distribution is obtained from Eq. (40), but Eq. (40) is the predictive distribution. The posterior is defined in Eq. (38); please fix this cross-reference.
  5. [Eq. (42)] In Eq. (42), the expectation E[s^2(x;phi)] is written and then immediately replaced by s^2(x;phi); the text should state explicitly that this is valid because the aleatoric variance is frozen from Step 2 before Step 3 is run.

Circularity Check

0 steps flagged · score 2.0 of 10

No circularity found: the central mean and uncertainty predictions are validated against independent ground-truth simulations; self-citations provide method provenance but do not force the results by construction.

full rationale

The central empirical claims are not circular. The mean predictions and uncertainty estimates are tested against external ground truth that is not used in training: noiseless RVE/DNS simulations define the ground-truth mean, and 100 repeated SVE simulations define the ground-truth aleatoric variance (Table 1, Section 2). The VeBRNN cooperative training algorithm and the MF decomposition f^h = g(f^l,x) + r are taken from the authors' prior work [19,20], but the paper is explicit that the MF form was 'postulated' (Section 3.2), and it does not rely on a uniqueness theorem: Section E compares four MF architectural variants and selects the best empirically. The variance network in Step 2 estimates s^2 from residuals of a fixed mean, and Eq. (42) is the standard law of total variance under the Gaussian likelihood of Eq. (2); the reported aleatoric variance is validated against held-out repeat statistics (e.g., Figure 8), so it is not a fitted quantity renamed as a prediction. The Gaussian assumption in Eqs. (2) and (28) is an untested modeling and evaluation assumption, and the reliance on an unbiased Step-1 mean is an identifiability risk; these are correctness concerns, not circular reductions. Self-citations [19,20,53] provide method provenance and are not load-bearing in the sense of forcing the results by construction.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

No new physical entities are introduced. The central claim rests on distributional assumptions (Gaussian likelihood, Gamma residuals), the sequential training assumption underpinning disentanglement, the transferability of LF information, and the cost model used for comparisons. These are all explicit or standard in the cited literature, but they are not proved in this paper.

free parameters (4)
  • Ncluster = 3 and 18
    Number of SCA clusters chosen to define low-fidelity data. It sets the bias and cost of the LF simulations, and the relative advantage of MF models depends on this choice.
  • cost ratios c = 1, 1/20, 1/36, 1/60, 1/120, 1/200
    Relative simulation times per path measured on one platform (Table 1); used in Eq. (5) to normalize all MF vs SF comparisons.
  • network hyperparameters = 128 hidden units (mean), 8 (variance), learning rates 0.001/0.01, epochs 1000-10000
    Hand-set architecture and training choices from prior work [8,53]; not tuned per dataset, but affect all results.
  • confidence level alpha = 0.05
    Used to define the 95% intervals for PICP and MPIW; a standard choice, not model-dependent.
assumptions (6)
  • domain assumption Stress observations follow a Gaussian distribution with diagonal covariance at each pseudo-time step (Eq. 2).
    This likelihood assumption underpins the mean/variance decomposition and the Gamma-distributed squared residual in the variance network (Section D.2).
  • standard math If y is Gaussian, then (y - f(x))^2 follows a Gamma distribution.
    Follows from Gaussianity of the residuals, assuming the mean network is accurate; used to train the aleatoric variance network.
  • domain assumption The aleatoric variance estimated in Step 2 remains fixed when the mean is updated in Step 3 (Eq. 42).
    The disentanglement of epistemic and aleatoric uncertainty assumes the two components are independent and that sequential training converges within K=2 iterations.
  • domain assumption Low-fidelity data is informative and transferable to high-fidelity prediction via hidden states (Section 3.2, Figure 6).
    The MF framework only helps when LF data carries useful information; the paper shows cases where MF is not beneficial (e.g., low-quality LF or high budgets).
  • domain assumption Cost ratios in Table 1 represent the true time cost of generating a strain-stress path on the hardware used (Section 2, Eq. 5).
    All equal-budget comparisons of MF vs SF models use these ratios; deviations would change the conclusions about data allocation.
  • domain assumption pSGLD provides a reliable approximation to the posterior (Section A.1.2).
    Epistemic uncertainty estimates depend on the sampler's convergence and the unit Gaussian prior assumption; no verification of convergence is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Single- to multi-fidelity history-dependent learning with uncertainty quantification and disentanglement: application to data-driven constitutive modeling." pith.science (2026). https://pith.science/paper/UR5CNNDT

@misc{pith2026250713416,
  author       = {Pith},
  title        = {Pith review of: Single- to multi-fidelity history-dependent learning with uncertainty quantification and disentanglement: application to data-driven constitutive modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UR5CNNDT}},
  note         = {Machine review of arXiv:2507.13416}
}
read the original abstract

Data-driven learning is generalized to consider history-dependent multi-fidelity data, while quantifying epistemic uncertainty and disentangling it from data noise (aleatoric uncertainty). This generalization is hierarchical and adapts to different learning scenarios: from training the simplest single-fidelity deterministic neural networks up to the proposed multi-fidelity variance estimation Bayesian recurrent neural networks. The versatility and generality of the proposed methodology are demonstrated by applying it to different data-driven constitutive modeling scenarios that include multiple fidelities with and without aleatoric uncertainty (noise). The method accurately predicts the response and quantifies model error while also discovering the noise distribution (when present). This opens opportunities for future real-world applications in diverse scientific and engineering domains; especially, the most challenging cases involving design and analysis under uncertainty.

Figures

Figures reproduced from arXiv: 2507.13416 by the authors.

Figure 1
Figure 1. Schematic summarizing different data generation methods for discovering the constitutive behavior of [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. RVE and SVE strain-stress path data: (a) illustration of biphasic material RVE and three SVEs; (b) example [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Performance comparison between FEA-based DNS and SCA in the computational homogenization of biphasic [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (29 more)
Figure 4
Figure 4. Figure 4: Flowchart of the proposed practical multi-fidelity framework. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Flowchart for choosing a neural network architecture that learns from data of given fidelity. UQ is an [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Schematic of the proposed multi-fidelity recurrent neural network architectures for predicting uncertainty [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Comparison of test performance metrics on Dataset 1 (single-fidelity problem) based on [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Predictions of VeBRNN on an ID test path after training with 2000 paths ( [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Results obtained for the fully deterministic MF model (RNN+RNN) trained on Dataset 2 ( [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Results of RNN+VeBRNN model trained on Dataset 2 ( [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Prediction of RNN+VeBRNN for a training dataset ( [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Results for the VeBRNN+VeBRNN model, when compared with a single-fidelity VeBRNN trained on HF [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 13
Figure 13. Figure 13: Results for VeBRNN+RNN model trained on Dataset 4 ( [PITH_FULL_IMAGE:figures/full_fig_p016_13.png]
Figure 14
Figure 14. Figure 14: Predictions of the LF model (VeBRNN) for training data obtained by SCA of an RVE with [PITH_FULL_IMAGE:figures/full_fig_p017_14.png]
Figure 15
Figure 15. Figure 15: Prediction of VeBRNN+RNN model for Dataset 4 ( [PITH_FULL_IMAGE:figures/full_fig_p017_15.png]
Figure 16
Figure 16. Figure 16: Results for Dataset 3 (ROM]+DNS g with Ncluster = 18) when considering different budget percentages of LF data acquisition for a total budget cost of Tc = 15. The performance metrics of the VeBRNN+VeBRNN model are compared with the two single-fidelity models obtained …
Figure 17
Figure 17. Figure 17: Results for Dataset 4 (ROM]+DNS with Ncluster = 18) when considering different budget percentages of LF data acquisition for a given total budget cost: (a) Tc = 20, (b) Tc = 200, and (c) Tc = 400. Each point is labeled with the number of training paths used at the LF …
Figure 18
Figure 18. Figure 18: Schematic of Hamiltonian mechanics system [PITH_FULL_IMAGE:figures/full_fig_p023_18.png]
Figure 19
Figure 19. Figure 19: Schematic of VI [58]: The ellipse represents the search space of the proposed distribution, p(θ|D) is the true posterior distribution, the KL divergence is the distance between the two distributions defined as DKL (q(z)||pθ(z|x)), and the goal is to minimize it. Accor…
Figure 20
Figure 20. Figure 20: Generation of the random polynomial strain paths. [PITH_FULL_IMAGE:figures/full_fig_p028_20.png]
Figure 21
Figure 21. Figure 21: Convergence analysis of the size of the microstructure volume element and the origin of aleatoric uncertainty [PITH_FULL_IMAGE:figures/full_fig_p028_21.png]
Figure 22
Figure 22. Figure 22: compares the noisy results obtained from DNS g (i.e., FEA of an SVE) with the noisy results obtained from ROM (i.e., SCA of an SVE). This allows for visualizing the differences in aleatoric uncertainty when data is obtained ]from different simulation methods (FEA vs. …
Figure 23
Figure 23. Figure 23: Bias error introduced by SCA-RVE when compared to DNS-RVE and considering different numbers of clusters for the SCA method. The columns from left to right refer to the different stress components: σ11, σ12, and σ22, respectively. 30 [PITH_FULL_IMAGE:figures/full_fig_…
Figure 24
Figure 24. Figure 24: Schematic of the proposed multi-fidelity recurrent neural network architecture: [PITH_FULL_IMAGE:figures/full_fig_p033_24.png]
Figure 25
Figure 25. Figure 25: Schematic of the proposed multi-fidelity recurrent neural network architectures: [PITH_FULL_IMAGE:figures/full_fig_p033_25.png]
Figure 26
Figure 26. Figure 26: Schematic of the proposed multi-fidelity recurrent neural network architecture: [PITH_FULL_IMAGE:figures/full_fig_p034_26.png]
Figure 27
Figure 27. Figure 27: Comparison of different knowledge transfer configurations for [PITH_FULL_IMAGE:figures/full_fig_p035_27.png]
Figure 28
Figure 28. Figure 28: Comparison of different multi-fidelity model configurations proposed in this paper for [PITH_FULL_IMAGE:figures/full_fig_p036_28.png]
Figure 29
Figure 29. Figure 29: Comparison of different multi-fidelity model configurations proposed in this paper for [PITH_FULL_IMAGE:figures/full_fig_p037_29.png]
Figure 30
Figure 30. Figure 30: Predictions of the RNN with 2000 training paths ( [PITH_FULL_IMAGE:figures/full_fig_p037_30.png]
Figure 31
Figure 31. Figure 31: Results for RNN+RNN model trained on Dataset 3 ( [PITH_FULL_IMAGE:figures/full_fig_p038_31.png]
Figure 32
Figure 32. Figure 32: Results for the RNN+VeBRNN model, when compared with a single-fidelity VeBRNN trained on HF data. [PITH_FULL_IMAGE:figures/full_fig_p038_32.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

65 extracted references · 51 canonical work pages

  1. [52]

    Hesthaven

    Paolo Conti, Mengwu Guo, Andrea Manzoni, and Jan S. Hesthaven. Multi-fidelity surrogate modeling using long short-term memory networks.Computer Methods in Applied Mechanics and Engineering, 404:115811, 2023

    work page 2023

  2. [1]

    Artificial neural networks for solving ordinary and partial differential equations.IEEE transactions on neural networks, 9(5):987–1000, 1998

    Isaac E Lagaris, Aristidis Likas, and Dimitrios I Fotiadis. Artificial neural networks for solving ordinary and partial differential equations.IEEE transactions on neural networks, 9(5):987–1000, 1998

    1998

  3. [2]

    Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.Journal of Computational physics, 378:686–707, 2019

    2019

  4. [3]

    Deep learning.nature, 521(7553):436–444, 2015

    Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning.nature, 521(7553):436–444, 2015. 19 APREPRINT- SEPTEMBER16, 2025

    work page 2015

  5. [4]

    Ghaboussi, J

    J. Ghaboussi, J. H. Garrett, and X. Wu. Knowledge-based modeling of material behavior with neural networks. Journal of Engineering Mechanics, 117(1):132–153, 1991

    work page 1991

  6. [5]

    Bessa, R

    M.A. Bessa, R. Bostanabad, Z. Liu, A. Hu, Daniel W. Apley, C. Brinson, W. Chen, and Wing Kam Liu. A framework for data-driven analysis of materials under uncertainty: Countering the curse of dimensionality. Computer Methods in Applied Mechanics and Engineering, 320:633–667, 2017

    work page 2017

  7. [6]

    Computational homogenization of nonlinear elastic materials using neural networks.International Journal for Numerical Methods in Engineering, 104(12):1061–1084, 2015

    BA Le, Julien Yvonnet, and Q-C He. Computational homogenization of nonlinear elastic materials using neural networks.International Journal for Numerical Methods in Engineering, 104(12):1061–1084, 2015

    work page 2015

  8. [7]

    Bessa, and Wing Kam Liu

    Zeliang Liu, M.A. Bessa, and Wing Kam Liu. Self-consistent clustering analysis: An efficient multi-scale scheme for inelastic heterogeneous materials.Computer Methods in Applied Mechanics and Engineering, 306:319–341, 2016

    work page 2016

Show all 65 references
  1. [8]

    Mozaffar, R

    M. Mozaffar, R. Bostanabad, W. Chen, K. Ehmann, J. Cao, and M. A. Bessa. Deep learning predicts path-dependent plasticity.Proceedings of the National Academy of Sciences, 116(52):26414–26420, 2019

    2019

  2. [9]

    Ling Wu, Van Dung Nguyen, Nanda Gopala Kilingar, and Ludovic Noels. A recurrent neural network-accelerated multi-scale model for elasto-plastic heterogeneous materials subjected to random cyclic and non-proportional loading paths.Computer Methods in Applied Mechanics and Engin...

    2020

  3. [10]

    Xiaolong He and Jiun-Shyan Chen. Thermodynamically consistent machine-learned internal state variable approach for data-driven modeling of path-dependent materials.Computer Methods in Applied Mechanics and Engineering, 402:115348, 2022. A Special Issue in Honor of the Lifetime...

    2022

  4. [11]

    A physics-informed deep neural network for surrogate modeling in classical elasto-plasticity.Computers and Geotechnics, 159:105472, 2023

    Mahdad Eghbalian, Mehdi Pouragha, and Richard Wan. A physics-informed deep neural network for surrogate modeling in classical elasto-plasticity.Computers and Geotechnics, 159:105472, 2023

    2023

  5. [12]

    Kalina, Jörg Brummund, WaiChing Sun, and Markus Kästner

    Karl A. Kalina, Jörg Brummund, WaiChing Sun, and Markus Kästner. Neural networks meet anisotropic hyperelasticity: A framework based on generalized structure tensors and isotropic tensor functions.Computer Methods in Applied Mechanics and Engineering, 437:117725, 2025

    2025

  6. [13]

    Automated model discovery of finite strain elastoplasticity from uniaxial experiments.Computer Methods in Applied Mechanics and Engineering, 435:117653, 2025

    Asghar Arshad Jadoon, Knut Andreas Meyer, and Jan Niklas Fuhg. Automated model discovery of finite strain elastoplasticity from uniaxial experiments.Computer Methods in Applied Mechanics and Engineering, 435:117653, 2025

    2025

  7. [14]

    Consistent machine learning for topology optimization with microstructure-dependent neural network material models.Journal of the Mechanics and Physics of Solids, 196:106015, 2025

    Harikrishnan Vijayakumaran, Jonathan B Russ, Glaucio H Paulino, and Miguel A Bessa. Consistent machine learning for topology optimization with microstructure-dependent neural network material models.Journal of the Mechanics and Physics of Solids, 196:106015, 2025

    2025

  8. [15]

    Bernardo P Ferreira and Miguel A Bessa. Automatically differentiable model updating (adimu): conventional, hy- brid, and neural network material model discovery including history-dependency.arXiv preprint arXiv:2505.07801, 2025

    2025 arXiv

  9. [16]

    Ferreira, F.M

    B.P. Ferreira, F.M. Andrade Pires, and M.A. Bessa. Adaptivity for clustering-based reduced-order modeling of localized history-dependent phenomena.Computer Methods in Applied Mechanics and Engineering, 393:114726, 2022

    2022

  10. [17]

    Ferreira, F

    Bernardo P. Ferreira, F. M. Andrade Pires, and Miguel A. Bessa. Crate: A python package to perform fast material simulations.Journal of Open Source Software, 8(87):5594, 2023

    2023

  11. [18]

    Deep Bayesian active learning with image data

    Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep Bayesian active learning with image data. In Doina Precup and Yee Whye Teh, editors,Proceedings of the 34th International Conference on Machine Learning, volume 70 ofProceedings of Machine Learning Research, pages 1183–1192...

    2017

  12. [19]

    Jiaxiang Yi, Ji Cheng, and Miguel A. Bessa. Practical multi-fidelity machine learning: fusion of deterministic and bayesian models, 2024

    2024

  13. [20]

    Jiaxiang Yi and Miguel A. Bessa. Cooperative bayesian and variance networks disentangle aleatoric and epistemic uncertainties, 2025

    2025

  14. [21]

    Bayesian deep learning and a probabilistic perspective of generalization

    Andrew G Wilson and Pavel Izmailov. Bayesian deep learning and a probabilistic perspective of generalization. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors,Advances in Neural Information Processing Systems, volume 33, pages 4697–4708. Curran Associ...

    2020

  15. [22]

    Rajendra Acharya, Vladimir Makarenkov, and Saeid Nahavandi

    Moloud Abdar, Farhad Pourpanah, Sadiq Hussain, Dana Rezazadegan, Li Liu, Mohammad Ghavamzadeh, Paul Fieguth, Xiaochun Cao, Abbas Khosravi, U. Rajendra Acharya, Vladimir Makarenkov, and Saeid Nahavandi. A review of uncertainty quantification in deep learning: Techniques, applic...

    2021

  16. [23]

    Psaros, Xuhui Meng, Zongren Zou, Ling Guo, and George Em Karniadakis

    Apostolos F. Psaros, Xuhui Meng, Zongren Zou, Ling Guo, and George Em Karniadakis. Uncertainty quantification in scientific machine learning: Methods, metrics, and comparisons.Journal of Computational Physics, 477:111902, 2023

    2023

  17. [24]

    Bessa, Piotr Glowacki, and Michael Houlder

    Miguel A. Bessa, Piotr Glowacki, and Michael Houlder. Bayesian machine learning in metamaterial design: Fragile becomes supercompressible.Advanced Materials, 31(48):1904845, 2019

    2019

  18. [25]

    Dongil Shin, Andrea Cupertino, Matthijs H. J. de Jong, Peter G. Steeneken, Miguel A. Bessa, and Richard A. Norte. Spiderweb nanomechanical resonators via bayesian optimization: Inspired by nature and guided by machine learning.Advanced Materials, n/a(n/a):2106248

  19. [26]

    McDowell

    Yan Wang and David L. McDowell. Uncertainty quantification in materials modeling. In Yan Wang and David L. McDowell, editors,Uncertainty Quantification in Multiscale Materials Modeling, Elsevier Series in Mechanics of Advanced Materials, pages 1–40. Woodhead Publishing, 2020

    2020

  20. [27]

    MIT press Cambridge, MA, 2006

    Christopher KI Williams and Carl Edward Rasmussen.Gaussian processes for machine learning, volume 2. MIT press Cambridge, MA, 2006

    2006

  21. [28]

    Heteroscedastic gaussian process regression for material structure-property relationship modeling.Computer Methods in Applied Mechanics and Engineering, 431:117326, 2024

    Ozge Ozbayram, Audrey Olivier, and Lori Graham-Brady. Heteroscedastic gaussian process regression for material structure-property relationship modeling.Computer Methods in Applied Mechanics and Engineering, 431:117326, 2024

    2024

  22. [29]

    Mcmc using hamiltonian dynamics.Handbook of markov chain monte carlo, 2(11):2, 2011

    Radford M Neal. Mcmc using hamiltonian dynamics.Handbook of markov chain monte carlo, 2(11):2, 2011

    2011

  23. [30]

    Position: Bayesian deep learning is needed in the age of large-scale ai.arXiv preprint arXiv:2402.00809, 2024

    Theodore Papamarkou, Maria Skoularidou, Konstantina Palla, Laurence Aitchison, Julyan Arbel, David Dunson, Maurizio Filippone, Vincent Fortuin, Philipp Hennig, José Miguel Hernández-Lobato, et al. Position: Bayesian deep learning is needed in the age of large-scale ai.arXiv pr...

    2024 arXiv

  24. [31]

    Springer Science & Business Media, 2012

    Radford M Neal.Bayesian learning for neural networks, volume 118. Springer Science & Business Media, 2012

    2012

  25. [32]

    Blei, Alp Kucukelbir, and Jon D

    David M. Blei, Alp Kucukelbir, and Jon D. McAuliffe. Variational inference: A review for statisticians.Journal of the American Statistical Association, 112(518):859–877, 2017

    2017

  26. [33]

    Dropout as a bayesian approximation: Representing model uncertainty in deep learning

    Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In Maria Florina Balcan and Kilian Q. Weinberger, editors,Proceedings of The 33rd International Conference on Machine Learning, volume 48 ofProceedings of Mac...

    2016

  27. [34]

    Simple and scalable predictive uncertainty estimation using deep ensembles

    Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In I. Guyon, U. V on Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors,Advances in Neural Information...

    2017

  28. [35]

    Shields, and Lori Graham-Brady

    Audrey Olivier, Michael D. Shields, and Lori Graham-Brady. Bayesian neural networks for uncertainty quantifica- tion in data-driven materials modeling.Computer Methods in Applied Mechanics and Engineering, 386:114079, 2021

    2021

  29. [36]

    Pasparakis, Lori Graham-Brady, and Michael D

    George D. Pasparakis, Lori Graham-Brady, and Michael D. Shields. Bayesian neural networks for predicting uncertainty in full-field material response, 2024

    2024

  30. [37]

    Discovering uncertainty: Bayesian constitutive artificial neural networks.Computer Methods in Applied Mechanics and Engineering, 433:117517, 2025

    Kevin Linka, Gerhard A Holzapfel, and Ellen Kuhl. Discovering uncertainty: Bayesian constitutive artificial neural networks.Computer Methods in Applied Mechanics and Engineering, 433:117517, 2025

    2025

  31. [38]

    Sparse bayesian recurrent neural networks

    Sotirios P Chatzis. Sparse bayesian recurrent neural networks. InMachine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2015, Porto, Portugal, September 7-11, 2015, Proceedings, Part II 15, pages 359–372. Springer, 2015

    2015

  32. [39]

    Bayesian recurrent neural networks, 2017

    Meire Fortunato, Charles Blundell, and Oriol Vinyals. Bayesian recurrent neural networks, 2017

    2017

  33. [40]

    Scalable bayesian learning of recurrent neural networks for language modeling, 2017

    Zhe Gan, Chunyuan Li, Changyou Chen, Yunchen Pu, Qinliang Su, and Lawrence Carin. Scalable bayesian learning of recurrent neural networks for language modeling, 2017

    2017

  34. [41]

    Barnn: A bayesian autoregressive and recurrent neural network.arXiv preprint arXiv:2501.18665, 2025

    Dario Coscia, Max Welling, Nicola Demo, and Gianluigi Rozza. Barnn: A bayesian autoregressive and recurrent neural network.arXiv preprint arXiv:2501.18665, 2025

    2025 arXiv

  35. [42]

    Preconditioned stochastic gradient langevin dynamics for deep neural networks, 2015

    Chunyuan Li, Changyou Chen, David Carlson, and Lawrence Carin. Preconditioned stochastic gradient langevin dynamics for deep neural networks, 2015

    2015

  36. [43]

    Estimating the mean and variance of the target probability distribution

    David A Nix and Andreas S Weigend. Estimating the mean and variance of the target probability distribution. In Proceedings of 1994 ieee international conference on neural networks (ICNN’94), volume 1, pages 55–60. IEEE, 1994. 21 APREPRINT- SEPTEMBER16, 2025

    1994

  37. [44]

    A multi-fidelity machine learning approach to high throughput materials screening.npj Computational Materials, 8(1):257, 2022

    Clyde Fare, Peter Fenner, Matthew Benatan, Alessandro Varsi, and Edward O Pyzer-Knapp. A multi-fidelity machine learning approach to high throughput materials screening.npj Computational Materials, 8(1):257, 2022

    2022

  38. [45]

    Haftka, and Nam H

    Chanyoung Park, Raphael T. Haftka, and Nam H. Kim. Remarks on multi-fidelity surrogates.Structural and Multidisciplinary Optimization, 55(3):1029–1050, 2017

    2017

  39. [46]

    Remarks on multi-output gaussian process regression.Knowledge- Based Systems, 144:102–121, 2018

    Haitao Liu, Jianfei Cai, and Yew-Soon Ong. Remarks on multi-output gaussian process regression.Knowledge- Based Systems, 144:102–121, 2018

    2018

  40. [47]

    A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse pde problems.Journal of Computational Physics, 401:109020, 2020

    Xuhui Meng and George Em Karniadakis. A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse pde problems.Journal of Computational Physics, 401:109020, 2020

    2020

  41. [48]

    Multi-fidelity bayesian optimization via deep neural networks

    Shibo Li, Wei Xing, Robert Kirby, and Shandian Zhe. Multi-fidelity bayesian optimization via deep neural networks. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors,Advances in Neural Information Processing Systems, volume 33, pages 8521–8531. Curran A...

    2020

  42. [49]

    An enhanced variable-fidelity optimization approach for constrained optimization problems and its parallelization.Structural and Multidisciplinary Optimization, 65(7):188, 2022

    Ji Cheng, Qiao Lin, and Jiaxiang Yi. An enhanced variable-fidelity optimization approach for constrained optimization problems and its parallelization.Structural and Multidisciplinary Optimization, 65(7):188, 2022

    2022

  43. [50]

    Multi-fidelity optimization via surrogate modelling

    Alexander I.J Forrester, András Sóbester, and Andy J Keane. Multi-fidelity optimization via surrogate modelling. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 463(2088):3251–3269, 2007

    2007

  44. [51]

    Multi-fidelity bayesian neural networks: Algorithms and applications.Journal of Computational Physics, 438:110361, 2021

    Xuhui Meng, Hessam Babaee, and George Em Karniadakis. Multi-fidelity bayesian neural networks: Algorithms and applications.Journal of Computational Physics, 438:110361, 2021

    2021

  45. [53]

    Taylan Turan, Jiaxiang Yi, and Miguel A

    Aleksandr Dekhovich, O. Taylan Turan, Jiaxiang Yi, and Miguel A. Bessa. Cooperative data-driven modeling. Computer Methods in Applied Mechanics and Engineering, 417:116432, 2023

    2023

  46. [54]

    Deshpande, Soumya Ghosh, Tin D

    Sameer K. Deshpande, Soumya Ghosh, Tin D. Nguyen, and Tamara Broderick. Are you using test loglikelihood correctly?, 2024

    2024

  47. [55]

    L. V . Kantorovich. Mathematical methods of organizing and planning production.Management Science, 6(4):366– 422, 1960

    1960

  48. [56]

    How to evaluate uncertainty estimates in machine learning for regression?Neural Networks, 173:106203, May 2024

    Laurens Sluijterman, Eric Cator, and Tom Heskes. How to evaluate uncertainty estimates in machine learning for regression?Neural Networks, 173:106203, May 2024

    2024

  49. [57]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2014

    2014

  50. [58]

    Probabilistic machine learning: Advanced topics, 2022

    K Murphy. Probabilistic machine learning: Advanced topics, 2022

    2022

  51. [59]

    Bayesian learning via stochastic gradient langevin dynamics

    Max Welling and Yee W Teh. Bayesian learning via stochastic gradient langevin dynamics. InProceedings of the 28th international conference on machine learning (ICML-11), pages 681–688, 2011

    2011

  52. [60]

    An overview of gradient descent optimization algorithms.arXiv preprint arXiv:1609.04747, 2016

    Sebastian Ruder. An overview of gradient descent optimization algorithms.arXiv preprint arXiv:1609.04747, 2016

    2016 arXiv

  53. [61]

    Elastic properties of reinforced solids: some theoretical principles.Journal of the Mechanics and Physics of Solids, 11(5):357–372, 1963

    Rodney Hill. Elastic properties of reinforced solids: some theoretical principles.Journal of the Mechanics and Physics of Solids, 11(5):357–372, 1963

    1963

  54. [62]

    Ferreira, F

    Bernardo P. Ferreira, F. M. Andrade Pires, and Miguel A. Bessa. Adaptive clustering-based reduced-order modeling framework: Fast and accurate modeling of localized history-dependent phenomena. 2021

    2021

  55. [63]

    Hands- on bayesian neural networks—a tutorial for deep learning users.IEEE Computational Intelligence Magazine, 17(2):29–48, 2022

    Laurent Valentin Jospin, Hamid Laga, Farid Boussaid, Wray Buntine, and Mohammed Bennamoun. Hands- on bayesian neural networks—a tutorial for deep learning users.IEEE Computational Intelligence Magazine, 17(2):29–48, 2022. 22 APREPRINT- SEPTEMBER16, 2025 A Bayesian inference me...

    2022

  56. [64]

    = (θt−1,v t−1) Half step for momentum:v ′ 1 2 =v ′ t − η 2 ∇ε(θ ′ 0) forl= 1 :L−1do θ ′ l =θ ′ l−1 +ηΣ −1v ′ l− 1 2 v ′ l+ 1 2 =v ′ l− 1 2 −η∇ε(θ ′ l) end Full step for location:θ ′ L =θ ′ L− 1 2 +ηΣ −1v ′ L− 1 2 Half step for momentum:v ′ L =v ′ L− 1 2 − η 2 ∇ε(θ ′ L) Compute...

    2025

  57. [65]

    or VI [32, 58] are required; we provide details in the Appendix A. After obtaining the posterior distribution of the parameters in Equation (38), the predicted posterior distribution for any unknown pointx ′1:T can be computed as p(y ′ 1:T |x ′ 1:T ,D) = Z p(x ′ 1:T |x ′ 1:T ,...

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.