Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

BARNN: A Bayesian Autoregressive and Recurrent Neural Network

T0 review · 2 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read BARNN claims a general recipe for turning any autoregressive or recurrent network into a Bayesian one: evolve the weights alongside the states, sample per-timestep weights from a variational posterior, and use a temporal…

desk verdict A useful, empirically solid UQ method whose 'Bayesian' prior is actually a within-batch regularizer; worth reviewing, but the principled claim needs rework. read the letter →

arxiv 2501.18665 v2 pith:K24MLFHK submitted 2025-01-30 cs.LG cs.AI

Dario Coscia , Max Welling , Nicola Demo , Gianluigi Rozza This is my paper · ORCID
classification cs.LGcs.AI
keywords BayesiandeeplearningvariationaldropoutautoregressivemodelsrecurrentneuralnetworksuncertaintyquantificationPDEsolversmoleculargenerationmixtureofposteriors
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

BARNN proposes a general recipe for turning any autoregressive or recurrent neural network into a Bayesian one. The paper's central claim is that by letting the network weights evolve alongside the observable states, sampling new weights at every time step from a variational posterior, one obtains predictive distributions with calibrated uncertainty estimates. The method derives a temporal evidence lower bound, reuses variational dropout for scalability, and introduces a time-dependent mixture-of-posteriors prior (tVAMP) to keep inference efficient. On PDE forecasting (Burgers, Kuramoto-Sivashinsky, Korteweg-de Vries) and SMILES-based molecule generation, the paper reports that the resulting model is at least as accurate as deterministic baselines and better calibrated, with a particular gain in long-range dependencies such as ring closure in molecules. A sympathetic reader comes away with the promise that any existing autoregressive or recurrent architecture can be retrofitted with Bayesian uncertainty by adding a small encoder and doing Monte Carlo sampling at inference.

What carries the argument

The load-bearing mechanism is the time-indexed reparametrization of the weights, ω_t^l = α_t^l Ω^l (1+ε) with ε ∼ N(0,I), where Ω are the static neural-network weights, α_t are positive per-layer dropout coefficients output by a small encoder E_ψ that reads previous states (and, for RNNs, the hidden state), and ε provides stochasticity. This makes sampling practical for large networks via the local reparametrization trick, which samples activations rather than full weight matrices. The companion tVAMP prior p(ω_t) = ∫ p(y_{0:t−1}) q(ω_t | y_{0:t−1}) dy_{0:t−1} is approximated using batch statistics β_t and γ_t; because both posterior and prior are normal with means proportional to the same Ω, the KL divergence in the ELBO becomes independent of Ω, turning Bayesian inference into a standard log-likelihood (or MSE/cross-entropy) loss plus a cheap KL regularizer per layer. That regularizer is what the paper credits for calibrated, well-shaped predictive uncertainties, and the encoder plus Monte Carlo sampling is what transfers the recipe to arbitrary autoregressive or recurrent architectures.

What would settle it

Train BARNN on the PDE or molecule task with batch size 1, where the tVAMP prior statistics coincide with the single sample and the KL term in equation (9) vanishes; if the ECE and NLL remain as good as with full batches, the calibration gain comes from the input-dependent Monte Carlo sampling rather than from the proposed variational prior.

Watch

Extended reading notes

Core claim

The central object is a joint distribution over states and weights, p(y_0:T, ω_1:T)=∏_t p(y_t | y_<t, ω_t) p(ω_t), in which the weights are refreshed at every time step instead of being fixed after training. The paper derives a variational lower bound whose per-step form resembles a VAE ELBO, and then parametrizes the posterior over weights using variational dropout: each layer's weights are reparametrized as ω_t^l = α_t^l Ω^l (1+ε), with static base weights Ω, time-dependent scalar dropout coefficients α_t produced by a small encoder, and Gaussian noise ε. For the prior it introduces the tVAMP prior, the aggregate of variational posteriors over past states, whose batch-estimated statistics make the KL term independent of Ω and hence cheap to compute. The paper's claim is that this combination—time-varying weights, the temporal ELBO, and the tVAMP prior—yields a Bayesian autoregressive or recurrent model that matches or beats deterministic accuracy while providing calibrated and sharp uncertainties. Empirically, it reports lower NLL and ECE than Monte Carlo dropout, input perturbation, ARD dropout, and PDE-Refiner baselines on the three PDEs, and better validity, uniqueness, novelty, and molecular property statistics in SMILES generation.

Load-bearing premise

The load-bearing premise is that the KL regularizer in the BARNN objective is a genuine constraint: because the tVAMP prior is estimated from the same batch statistics as the posterior, that term can shrink toward zero, and the paper's calibrated uncertainties are only attributable to the Bayesian machinery if the KL term is actually active during training.

Editorial extensions

If this is right

  • Any existing neural PDE solver or RNN language model can be turned Bayesian with a small encoder and Monte Carlo resampling at inference, without redesigning the backbone.
  • Calibrated uncertainty is available at roughly 30 stochastic forward passes, and if uncertainty is not needed, the maximum-a-posteriori weights give deterministic-speed predictions with the same reported RMSE.
  • Long-range dependencies improve: in SMILES generation, ring-closure errors drop by about 30% over a dropout LSTM and 50% over a plain LSTM, with larger gains as the number of rings grows.
  • On Burgers, Kuramoto-Sivashinsky, and Korteweg-de Vries, BARNN achieves lower negative log-likelihood and expected calibration error than Monte Carlo dropout, input perturbation, ARD dropout, and PDE-Refiner over 320-step unrolls.
  • The method supplies a decomposition of predictive variance into epistemic and aleatoric components via the law of total variance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: if the KL regularizer is genuinely active, the same retrofit should transfer to transformer-style language models, giving token-level calibrated probabilities; the reported molecule results are a proxy for that, not a demonstration on transformers.
  • My inference: since the tVAMP prior is re-estimated from the current batch at every step, the method is best understood as amortized, input-dependent variational dropout; whether it is a fully Bayesian posterior over weights depends on the prior being held fixed after training, which the batch-estimated form does not strictly do.
  • My inference: the convergence of RMSE, NLL, and ECE at about 30 ensemble members suggests a practical testable recipe: set ensemble size by monitoring ECE during rollout rather than using a fixed 100-member ensemble.
  • My inference: if the gains persist when the encoder is applied to the hidden states of a large pretrained model, this could become a lightweight calibration layer for deployed systems; a direct comparison against temperature scaling and conformal prediction would clarify what the Bayesian component adds.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper introduces BARNN, a variational Bayesian framework for autoregressive and recurrent models. The authors define a joint distribution over states and time-indexed network weights, derive a temporal ELBO, and approximate the posterior with a variational dropout parameterization in which per-layer dropout coefficients alpha_t are produced by an encoder. A new 'temporal VAMP' prior is defined as the aggregated posterior in time and estimated from minibatch statistics. Experiments on a synthetic time series, on Burgers/KS/KdV PDE forecasting with Fourier Neural Operators, and on SMILES-based molecule generation with LSTMs report competitive or better accuracy and better calibration (NLL, ECE) than MC dropout, ARD, input perturbation, PDE Refiner, and dropout LSTM baselines.

Significance. If the framework is sound, BARNN is a broadly applicable retrofit: it adds a small encoder and MC sampling to any autoregressive/recurrent model, with public code and a complete set of derivations in the appendix. The empirical evaluation is substantial and uses standard UQ metrics over two very different domains, which gives the headline accuracy and calibration claims some support. The main reservation is that the Bayesian interpretation of the method is not currently supported by the implemented objective: the tVAMP prior is re-estimated from the same minibatch as the posterior, so the KL term in Eq. (9) is a within-batch regularizer rather than a divergence to a fixed prior. This does not automatically invalidate the empirical results, but it changes what the method is and requires a fix or a reframing before the 'principled Bayesian' claim can be accepted.

major comments (2)
  1. [§3.3, Eqs (7)-(9)] The tVAMP prior is not a fixed prior. Eq (8) defines beta_t and gamma_t as averages of the alpha_t values produced by the encoder on the current mini-batch, and Eq (9) then measures each sample's posterior against those same batch statistics. For a batch of size 1, beta_t = gamma_t = alpha_t and the KL is exactly 0; for any batch in which all alpha_t are equal, the KL is also 0. Consequently the KL term imposes no absolute constraint on the scale of alpha_t, and the objective in Eq (4) is not the ELBO of the generative model in Eq (2). Appendix A.2 maximizes the prior with q held fixed, but Algorithm 2 re-estimates the prior from the current batch at every gradient step, so the conditions of that proof are not met. The authors should replace the batch-estimated prior by a proper VAMP-style pseudo-input prior, or by a moving average or held-out estimate of beta_t and gamma_t, and should verify that the resulting objective is still a lower bound or an unbiased stochastic estimate of one.
  2. [§3.4 and Appendix D, Algorithm 2] The implemented loss is inconsistent with the stated model and the sign in the algorithm appears wrong. The text says the objective is the minimization of the one-step MSE plus the negative KL divergence, i.e., MSE + D_KL as a loss, which matches maximizing Eq (4). Algorithm 2, however, writes L = ||y_hat_t^k - y_t||^2 - D_KL(alpha_t) and then descends the gradient; minimizing this would drive the KL upward, which is not the ELBO. In addition, the Gaussian observation model in Eq (10) contains a fixed observation noise sigma_t that is said not to be learned, but its value is never reported; if the likelihood term is to be MSE/(2 sigma_t^2), the relative weight of the KL regularization depends on an unreported hyperparameter. Please correct the sign and report sigma_t, or state explicitly that Algorithm 2 is the intended objective and adjust the model equation and the derivation of the ELBO accordingly.
minor comments (5)
  1. [Appendix A.4, Eq (27)] The predictive distribution should be E_{omega_t ~ q_phi}[p(y_t | y_{0:t-1}, omega_t)]; the displayed formula contains an extra q_phi factor inside the expectation, which would make the expression dimensionally inconsistent with a probability distribution.
  2. [Table 1 and Section 4.1] The column header in Table 1 says 'MSE (↓)' while the text in Section 4.1 refers to RMSE; please align the notation.
  3. [Figures 12 and 13] The captions for Figures 12 and 13 appear to be swapped: Figure 12 is captioned as a t-SNE plot but the text refers to it as temperature versus validity, and Figure 13's caption describes density distributions while the figure shows temperature versus uniqueness.
  4. [Figure 4] The 'MAP estimate' curve is not defined in the text; please specify how the MAP weights are extracted from the variational posterior (for example, alpha_t = 1) and whether the same encoder is used at inference.
  5. [Section 4.3] The claim that BARNN excels at modelling long-range dependencies would be strengthened by a control experiment that varies sequence length or ring count while holding the architecture fixed; the current ring-closure analysis is suggestive but indirect.

Circularity Check

1 steps flagged · score 6.0 of 10

The tVAMP prior is the aggregated posterior estimated from the current batch's own αt, so Eq. (9)'s KL regularizer is a self-comparison that vanishes at batch size 1; the 'principled Bayesian' claim partially reduces by construction.

  1. self definitional [Section 3.3, Eqs. (7)-(9); Appendix A.2, Eqs. (19)-(23); Algorithm 2 in Appendix D.1]
    "we can find that the best prior for the lower bound in eq.(4) is given by (proof in Appendix A.2): p(ωt) = ∫ p(y0:t−1) q(ωt | y0:t−1) dy0:t−1 (7) ... βl t = 1/N Σ_{k=1}^N αl t(y^k_{0:t−1}), γl t = sqrt(1/N Σ_{k=1}^N (αl t(y^k_{0:t−1}))²) (8) ... DKL [qϕ(ωt | y0:t−1)∥p(ωt)] = Σ_l |Ωl|/2 [((αl t − βl t)/γl t)² + (αl t/γl t)² − 1 − 2 ln(αl t/γl t)] (9)"

    Equations (7)-(9) in sequence make the prior a function of the posterior: p(ωt) is defined as the aggregated q (Eq. 7), then Eq. (8) estimates βt, γt from the same batch αt values that define each sample's q in Eq. (6). Substituting N=1 (or a homogeneous batch) gives βt=γt=αt, and Eq. (9) collapses to |Ωl|/2[(0)²+1−1−2ln1] = 0. Hence the 'Bayesian' KL never measures distance to a fixed prior; it is a within-batch dispersion penalty with no absolute constraint on dropout scale or predictive variance. Algorithm 2 recomputes the prior from the current encoder output each step, so the optimized objective is not an ELBO of any fixed Bayesian model. The tVAMP 'best prior' claim thus reduces by construction to a self-comparison; only the MC sampling of q supplies genuine uncertainty content.

full rationale

The ELBO derivation (Appendix A.1), the variational-dropout reparameterization (Eq. 6), and the KL computation (Appendix A.3) are self-contained and standard; Eq. (6) is announced as an assumption, not imported as external fact. The 'best prior' fixed-point (Eq. 7) is the classical VAMP result, proved in Appendix A.2 rather than resting on the co-authored Tomczak-Welling citation, and the only self-citations (Coscia et al. 2023, 2024) are software/background, so no load-bearing self-citation or smuggled ansatz is present. The circular step is the practical instantiation: Eq. (8) estimates the prior's βt, γt from the current mini-batch's αt — the same parameters that define the posterior — making the KL in Eq. (9) a self-comparison that is identically zero when the batch has one member or is homogeneous. The 'principled Bayesian prior' component of the central claim therefore reduces by construction to a within-batch dispersion penalty, and the optimization objective is not the ELBO of any fixed Bayesian model. This is partial, not total, circularity: the predictive distribution is a genuine Monte Carlo average over sampled weights, and the reported NLL, ECE and RMSE are computed on held-out data against external baselines (Dropout, ARD, Perturb, Refiner, LSTM), so the empirical evaluation stands on its own.

Assumptions & free parameters 3 free parameters · 5 assumptions · 1 invented entities

The central claim rests on a data-dependent prior built from the variational posterior itself, a per-layer scalar posterior restriction, and an unstated observation-noise variance. These are the main uncharged entries the reader must pay for.

free parameters (3)
  • alpha_t (per-layer dropout coefficients) = learned via encoder E_psi
    Scalar multiplier per layer at each time step that sets the posterior variance in eq (6); learned by the encoder during training.
  • batch size N for prior aggregation = 256 for molecules; not reported for PDE
    beta_t and gamma_t in eq (8) are computed on the current batch; with N=1 the KL in eq (9) is identically zero, so the Bayesian regularization vanishes as a function of a hand-set hyperparameter.
  • observation noise variance sigma_t = not learned, value not stated
    In Section 3.4 the state distribution has standard deviation sigma_t 'not learned', but its value is never given; the aleatoric term in eq (28) is therefore constant and the reported NLL uses only epistemic variance.
assumptions (5)
  • domain assumption Joint autoregressive factorization p(y0:T, w1:T) = prod_t p(y_t | y0:t-1, w_t) p(w_t)
    Eq (2); the weights w_t are a priori independent across time, so the model is not a dynamic weight process.
  • ad hoc to paper Optimal prior is the aggregated posterior p*(w_t) = integral p(y0:t-1) q(w_t | y0:t-1) dy0:t-1
    Appendix A.2, eq (19); the 'prior' is defined as a functional of the variational posterior itself, making the KL regularizer self-referential.
  • ad hoc to paper Training minibatch approximates the data distribution for beta_t and gamma_t
    Eq (8) and (23); the aggregated posterior is estimated from the current batch, which can include the same samples used in the likelihood term, a double use of data.
  • ad hoc to paper Posterior factorizes over time and layers as q(w_t | y0:t-1) = prod_l N(alpha_t^l Omega^l, (alpha_t^l Omega^l)^2)
    Eqs (5)-(6); a per-layer scalar multiplicative noise restricts the uncertainty to a rank-1 scaling per layer.
  • standard math ELBO derivation and local reparametrization trick
    Appendices A.1 and Algorithm 1; standard results from Kingma and Welling 2014 and Kingma et al. 2015.
invented entities (1)
  • Temporal Variational Mixture of Posteriors prior (tVAMP)
    purpose: Prior distribution over network weights at each time step, constructed from the aggregated variational posterior over training data (eq 7-8).
    The prior is derived from the model's own posterior on the training set; it supplies no external falsifiable prediction beyond the calibration metrics reported on the same benchmarks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BARNN: A Bayesian Autoregressive and Recurrent Neural Network." pith.science (2026). https://pith.science/paper/K24MLFHK

@misc{pith2026250118665,
  author       = {Pith},
  title        = {Pith review of: BARNN: A Bayesian Autoregressive and Recurrent Neural Network},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K24MLFHK}},
  note         = {Machine review of arXiv:2501.18665}
}
read the original abstract

Autoregressive and recurrent networks have achieved remarkable progress across various fields, from weather forecasting to molecular generation and Large Language Models. Despite their strong predictive capabilities, these models lack a rigorous framework for addressing uncertainty, which is key in scientific applications such as PDE solving, molecular generation and Machine Learning Force Fields. To address this shortcoming we present BARNN: a variational Bayesian Autoregressive and Recurrent Neural Network. BARNNs aim to provide a principled way to turn any autoregressive or recurrent model into its Bayesian version. BARNN is based on the variational dropout method, allowing to apply it to large recurrent neural networks as well. We also introduce a temporal version of the "Variational Mixtures of Posteriors" prior (tVAMP-prior) to make Bayesian inference efficient and well-calibrated. Extensive experiments on PDE modelling and molecular generation demonstrate that BARNN not only achieves comparable or superior accuracy compared to existing methods, but also excels in uncertainty quantification and modelling long-range dependencies.

Figures

Figures reproduced from arXiv: 2501.18665 by the authors.

Figure 1
Figure 1. BARNN generative model (a) and inference model (b). Solid lines represent the generative process, whereas dotted [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Prediction and uncertainty intervals for different PDEs at last time-step. The figure depicts the CFD solution (red [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. RMSE and NLL for different Neural Solvers [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Variation of RMSE for increasing ensemble mem [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Error analysis for SMILES, mean and std shown [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Caption [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 7
Figure 7. Figure 7: Exemplary of 1-dimensional Burgers rollout. [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]
Figure 8
Figure 8. Figure 8: Exemplary of 1-dimensional Kuramoto-Sivashinsky rollout. [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]
Figure 9
Figure 9. Figure 9: Exemplary of 1-dimensional Korteweg de Vries rollout. [PITH_FULL_IMAGE:figures/full_fig_p023_9.png]
Figure 10
Figure 10. Figure 10: t-SNE representation of different molecular prop [PITH_FULL_IMAGE:figures/full_fig_p024_10.png]
Figure 12
Figure 12. Figure 12: t-SNE representation of different molecular prop [PITH_FULL_IMAGE:figures/full_fig_p025_12.png]
Figure 14
Figure 14. Figure 14: Samples of generated molecules with BARNN. [PITH_FULL_IMAGE:figures/full_fig_p025_14.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Single- to multi-fidelity history-dependent learning with uncertainty quantification and disentanglement: application to data-driven constitutive modeling

    cs.LG 2025-07 conditional novelty 5.0 of 10

    A multi-fidelity Bayesian recurrent neural network framework predicts history-dependent material responses while separately quantifying aleatoric and epistemic uncertainties.

Reference graph

Works this paper leans on

59 extracted references · 47 canonical work pages · cited by 1 Pith paper

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Rethinking uncertainty estimation in natural language generation

    Aichberger, L., Schweighofer, K., and Hochreiter, S. Rethinking uncertainty estimation in natural language generation. arXiv preprint arXiv:2412.15176, 2024

    arXiv 2024

  3. [3]

    Bar-Sinai, Y., Hoyer, S., Hickey, J., and Brenner, M. P. Learning Data-Driven Discretizations for Partial Differential Equations . National Academy of Sciences, 116 0 (31): 0 15344--15349, 2019

    work page 2019

  4. [4]

    An Essay towards solving a Problem in the Doctrine of Chances

    Bayes, T. An Essay towards solving a Problem in the Doctrine of Chances. By the late Rev. Mr. Bayes, FRS communicated by Mr. Price, in a letter to John Canton, A.M.F.R.S. Philosophical transactions of the Royal Society of London, 1763

    work page

  5. [5]

    A Neural Probabilistic Language Model

    Bengio, Y., Ducharme, R., and Vincent, P. A Neural Probabilistic Language Model . Advances in neural information processing systems, 2000

    work page 2000

  6. [6]

    B., and Stuart, A

    Bhattacharya, K., Hosseini, B., Kovachki, N. B., and Stuart, A. M. Model Reduction and Neural Networks for Parametric PDEs . The SMAI journal of computational mathematics, 7: 0 121--157, 2021

    work page 2021

  7. [7]

    P., Lucic, A., Stanley, M., Brandstetter, J., Garvan, P., Riechert, M., Weyn, J., Dong, H., Vaughan, A., et al

    Bodnar, C., Bruinsma, W. P., Lucic, A., Stanley, M., Brandstetter, J., Garvan, P., Riechert, M., Weyn, J., Dong, H., Vaughan, A., et al. Aurora: A Foundation Model of the Atmosphere . arXiv preprint arXiv:2405.13063, 2024

    arXiv 2024

  8. [8]

    Brandstetter, J., Welling, M., and Worrall, D. E. Lie Point Symmetry Data Augmentation for Neural PDE Solvers . In International Conference on Machine Learning, 2022 a

    work page 2022

Show all 59 references
  1. [9]

    E., and Welling, M

    Brandstetter, J., Worrall, D. E., and Welling, M. M essage P assing N eural PDE S olvers. In International Conference on Learning Representations, 2022 b

    2022

  2. [10]

    Physics-informed neural networks for advanced modeling

    Coscia, D., Ivagnes, A., Demo, N., and Rozza, G. Physics-informed neural networks for advanced modeling. Journal of Open Source Software, 8 0 (87): 0 5352, 2023

    2023

  3. [11]

    Generative Adversarial Reduced Order Modelling

    Coscia, D., Demo, N., and Rozza, G. Generative Adversarial Reduced Order Modelling . Scientific Reports, 2024

    2024

  4. [12]

    Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-Sensitive Learning

    Depeweg, S., Hernandez-Lobato, J.-M., Doshi-Velez, F., and Udluft, S. Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-Sensitive Learning . In International Conference on Machine Learning, 2018

    2018

  5. [13]

    Wavenet: A Generative Model for Raw Audio

    Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., Kavukcuoglu, K., et al. Wavenet: A Generative Model for Raw Audio . arXiv preprint arXiv:1609.03499, 2016

    2016 arXiv

  6. [14]

    A., Welling, M., and van de Meent, J.-W

    Eijkelboom, F., Bartosh, G., Naesseth, C. A., Welling, M., and van de Meent, J.-W. Variational Flow Matching for Graph Generation . arXiv preprint arXiv:2406.04843, 2024

    2024 arXiv

  7. [15]

    Evans, L. C. Partial Differential Equations . In American Mathematical Society, 2022

    2022

  8. [16]

    and The PyTorch Lightning team

    Falcon, W. and The PyTorch Lightning team . PyTorch Lightning . PyTorch Lightning , March 2019. doi:10.5281/zenodo.3828935. URL https://github.com/Lightning-AI/lightning

    2019 doi

  9. [17]

    and Ghahramani, Z

    Gal, Y. and Ghahramani, Z. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning . In International Conference on Machine Learning, 2016

    2016

  10. [18]

    Practical Variational Inference for Neural Networks

    Graves, A. Practical Variational Inference for Neural Networks . Advances in Neural Information Processing Systems, 24, 2011

    2011

  11. [19]

    Hinton, G. E. and Van Camp, D. Keeping the Neural Networks Simple by Minimizing the Description Length of the Weights . In Conference on Computational Learning Theory, 1993

    1993

  12. [20]

    Denoising Diffusion Probabilistic Models

    Ho, J., Jain, A., and Abbeel, P. Denoising Diffusion Probabilistic Models . Advances in Neural Information Processing Systems, 2020

    2020

  13. [21]

    How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering

    Jiang, Z., Araki, J., Ding, H., and Neubig, G. How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering . Transactions of the Association for Computational Linguistics, 2021

    2021

  14. [22]

    Variational Dropout via Empirical Bayes

    Kharitonov, V., Molchanov, D., and Vetrov, D. Variational Dropout via Empirical Bayes . In arXiv preprint arXiv:1811.00596, 2018

    2018 arXiv

  15. [23]

    Kingma, D. P. and Ba, J. L. Adam: A method for stochastic optimization. International Conference on Learning Representations, 2014

    2014

  16. [24]

    Kingma, D. P. and Welling, M. Auto-Encoding Variational Bayes . In International Conference on Learning Representations, 2014

    2014

  17. [25]

    P., Salimans, T., and Welling, M

    Kingma, D. P., Salimans, T., and Welling, M. V ariational D ropout and the L ocal R eparameterization T rick. In Advances in Neural Information Processing Systems, 2015

    2015

  18. [26]

    Benchmarking Regressive Conditional Diffusion Models for Turbulent Flow Simulation

    Kohl, G., Chen, L., and Thuerey, N. Benchmarking Regressive Conditional Diffusion Models for Turbulent Flow Simulation . In ICML 2024 AI for Science Workshop, 2024

    2024

  19. [27]

    Bayesian Dark Knowledge

    Korattikara, A., Rathod, V., Murphy, K., and Welling, M. Bayesian Dark Knowledge . In International Conference on Neural Information Processing Systems, 2015

    2015

  20. [28]

    GraphCast: Learning Skillful Medium-Range Global Weather Forecasting

    Lam, R., Sanchez-Gonzalez, A., Willson, M., Wirnsberger, P., Fortunato, M., Alet, F., Ravuri, S., Ewalds, T., Eaton-Rosen, Z., Hu, W., et al. GraphCast: Learning Skillful Medium-Range Global Weather Forecasting . arXiv preprint arXiv:2212.12794, 2022

    2022 arXiv

  21. [29]

    Learning Skillful Medium-Range Global Weather Forecasting

    Lam, R., Sanchez-Gonzalez, A., Willson, M., Wirnsberger, P., Fortunato, M., Alet, F., Ravuri, S., Ewalds, T., Eaton-Rosen, Z., Hu, W., et al. Learning Skillful Medium-Range Global Weather Forecasting . In American Association for the Advancement of Science, 2023

    2023

  22. [30]

    Learning Deep Generative Models of Graphs

    Li, Y., Vinyals, O., Dyer, C., Pascanu, R., and Battaglia, P. Learning Deep Generative Models of Graphs . arXiv preprint arXiv:1803.03324, 2018

    2018 arXiv

  23. [31]

    B., Azizzadenesheli, K., Bhattacharya, K., Stuart, A., Anandkumar, A., et al

    Li, Z., Kovachki, N. B., Azizzadenesheli, K., Bhattacharya, K., Stuart, A., Anandkumar, A., et al. Fourier Neural Operator for Parametric Partial Differential Equations . In International Conference on Learning Representations, 2020

    2020

  24. [32]

    Physics-Informed Neural Operator for Learning Partial Differential Equations

    Li, Z., Zheng, H., Kovachki, N., Jin, D., Chen, H., Liu, B., Azizzadenesheli, K., and Anandkumar, A. Physics-Informed Neural Operator for Learning Partial Differential Equations . ACM/JMS Journal of Data Science, 2024

    2024

  25. [33]

    PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers

    Lippe, P., Veeling, B., Perdikaris, P., Turner, R., and Brandstetter, J. PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers . In Advances in Neural Information Processing Systems, 2024

    2024

  26. [34]

    Constrained Graph Variational Autoencoders for Molecule Design

    Liu, Q., Allamanis, M., Brockschmidt, M., and Gaunt, A. Constrained Graph Variational Autoencoders for Molecule Design . Advances in Neural Information Processing Systems, 31, 2018

    2018

  27. [35]

    and Welling, M

    Louizos, C. and Welling, M. Multiplicative Normalizing Flows for Variational Bayesian Neural Networks . In International Conference on Machine Learning, 2017

    2017

  28. [36]

    V ariational D ropout S parsifies D eep N eural N etworks

    Molchanov, D., Ashukha, A., and Vetrov, D. V ariational D ropout S parsifies D eep N eural N etworks. In International Conference on Machine Learning, 2017

    2017

  29. [37]

    Chemical Language Modeling with Structured State Space Sequence Models

    \"O z c elik, R., de Ruiter, S., Criscuolo, E., and Grisoni, F. Chemical Language Modeling with Structured State Space Sequence Models . Nature Communications, 15 0 (1): 0 6176, 2024

    2024

  30. [38]

    M., et al

    Papamarkou, T., Skoularidou, M., Palla, K., Aitchison, L., Arbel, J., Dunson, D., Filippone, M., Fortuin, V., Hennig, P., Hern \'a ndez-Lobato, J. M., et al. Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI . In International Conference on Machine Learning, 2024

    2024

  31. [39]

    Pytorch: An imperative style, high-performance deep learning library

    Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019

    2019

  32. [40]

    Learning mesh-based simulation with graph networks

    Pfaff, T., Fortunato, M., Sanchez-Gonzalez, A., and Battaglia, P. Learning mesh-based simulation with graph networks. In International Conference on Learning Representations, 2021

    2021

  33. [41]

    Pichi, F., Moya, B., and Hesthaven, J. S. A Graph Convolutional Autoencoder Approach to Model Order Reduction for Parametrized PDEs . Journal of Computational Physics, pp.\ 112762, 2024

    2024

  34. [42]

    Language Models are Unsupervised Multitask Learners

    Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language Models are Unsupervised Multitask Learners . OpenAI blog, 2019

    2019

  35. [43]

    Rasmussen, C. E. and Williams, C. K. I. Gaussian Processes for Machine Learning . The MIT Press, 2005

    2005

  36. [44]

    Advanced Reduced Order Methods and Applications in Computational Fluid Dynamics

    Rozza, G., Stabile, G., and Ballarin, F. Advanced Reduced Order Methods and Applications in Computational Fluid Dynamics . SIAM, 2022

    2022

  37. [45]

    Learning to Simulate Complex Physics with Graph Networks

    Sanchez-Gonzalez, A., Godwin, J., Pfaff, T., Ying, R., Leskovec, J., and Battaglia, P. Learning to Simulate Complex Physics with Graph Networks . In International Conference on Machine Learning, 2020

    2020

  38. [46]

    Long Short-term Memory

    Schmidhuber, J., Hochreiter, S., et al. Long Short-term Memory . Neural Comput, 9 0 (8): 0 1735--1780, 1997

    1997

  39. [47]

    Bio-xlstm: Generative modeling, representation and in-context learning of biological and chemical sequences

    Schmidinger, N., Schneckenreiter, L., Seidl, P., Schimunek, J., Hoedt, P.-J., Brandstetter, J., Mayr, A., Luukkonen, S., Hochreiter, S., and Klambauer, G. Bio-xlstm: Generative modeling, representation and in-context learning of biological and chemical sequences. arXiv preprin...

    2024 arXiv

  40. [48]

    H., Kogej, T., Tyrchan, C., and Waller, M

    Segler, M. H., Kogej, T., Tyrchan, C., and Waller, M. P. Generating Focused Molecule Libraries for Drug Discovery with Recurrent Neural Networks . ACS Central Science, 2018

    2018

  41. [49]

    GraphAF: a Flow-based Autoregressive Model for Molecular Graph Generation

    Shi, C., Xu, M., Zhu, Z., Zhang, W., Zhang, M., and Tang, J. GraphAF: a Flow-based Autoregressive Model for Molecular Graph Generation . In International Conference on Learning Representations, 2020

    2020

  42. [50]

    Simm, G., Pinsler, R., and Hern \'a ndez-Lobato, J. M. Reinforcement Learning for Molecular Design Guided by Quantum Mechanics . In International Conference on Machine Learning, 2020

    2020

  43. [51]

    Dropout: A Simple Way to Prevent Neural Networks from Overfitting

    Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. Dropout: A Simple Way to Prevent Neural Networks from Overfitting . In The Journal of Machine Learning Research, 2014

    2014

  44. [52]

    and Welling, M

    Tomczak, J. and Welling, M. VAE with a V amp P rior. In International Conference on Artificial Intelligence and Statistics, 2018

    2018

  45. [53]

    Neural Autoregressive Distribution Estimation

    Uria, B., C \^o t \'e , M.-A., Gregor, K., Murray, I., and Larochelle, H. Neural Autoregressive Distribution Estimation . Journal of Machine Learning Research, 2016

    2016

  46. [54]

    A., Wang, Y., Blankevoort, T., and Welling, M

    Van Baalen, M., Louizos, C., Nagel, M., Amjad, R. A., Wang, Y., Blankevoort, T., and Welling, M. Bayesian Bits: Unifying Quantization and Pruning . Advances in neural information processing systems, 2020

    2020

  47. [55]

    and Hinton, G

    Van der Maaten, L. and Hinton, G. Visualizing data using t-sne. Journal of machine learning research, 9 0 (11), 2008

    2008

  48. [56]

    Attention Is All You Need

    Vaswani, A. Attention Is All You Need . Advances in Neural Information Processing Systems, 2017

    2017

  49. [57]

    P., Bhatt, U., Neiswanger, W., Salakhutdinov, R., and Morency, L.-P

    Xiao, Y., Liang, P. P., Bhatt, U., Neiswanger, W., Salakhutdinov, R., and Morency, L.-P. Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysis . In Findings of the Association for Computational Linguistics: EMNLP 2022, 2022

    2022

  50. [58]

    X., Robeyns, M., Wang, X., and Aitchison, L

    Yang, A. X., Robeyns, M., Wang, X., and Aitchison, L. Bayesian Low-rank Adaptation for Large Language Models . In International Conference on Learning Representations, 2022

    2022

  51. [59]

    J., Blackshaw, J., Corbett, S., de Veij, M., Ioannidis, H., Lopez, D

    Zdrazil, B., Felix, E., Hunter, F., Manners, E. J., Blackshaw, J., Corbett, S., de Veij, M., Ioannidis, H., Lopez, D. M., Mosquera, J. F., et al. The ChEMBL Database in 2023: a Drug Discovery Platform Spanning Multiple Bioactivity Data Types and Time Periods . Nucleic Acids Re...

    2023

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.