{"id":"25829262-7ed3-4a3e-b852-e6e5da784a72","arxiv_id":"2504.14854","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A learned-score Langevin hypernetwork approximates the posterior over neural ODE weights, giving uncertainty estimates for history-dependent physical models.","lead":"This paper proposes a machine-learning framework that estimates uncertainty in neural network models of time-dependent physical processes by training a secondary network to generate weight samples through a Langevin-style random walk. The authors demonstrate it on chemical reactions and material simulations, comparing it with standard variational inference.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Temporal independence in Eq. (18) is load-bearing: with path-correlated data, the product likelihood over time overcounts evidence, so the claimed UQ bands are not trustworthy for evolving processes.","rationale":"The reader's weakest assumption matches the most load-bearing threat to the central claim. For the claim to hold, the posterior must reflect the information in time-series data. Eq. (18) treats each time point as an independent observation, which is inconsistent with the SDE-generated and memory-dependent datasets used in the demonstrations. The paper explicitly acknowledges this approximation can be relaxed, but none of the reported experiments relax it, and the evaluation metrics are marginal in time, so they cannot reveal the resulting overconfidence. This is not a disagreement with consensus; it is a correctness risk internal to the paper's likelihood model. A concrete check via residual autocorrelation and joint coverage would settle whether the concern is quantitative or merely formal. Secondary issues remain: Eq. (14) appears to use a path KL with an omitted initial-term and a normalization that conflicts with App. A for diffusion coefficient gamma*sqrt(2), and Eq. (12) writes the score as the gradient of the density rather than of the log-density. These are real inconsistencies and should be corrected, but they are less directly tied to the empirical UQ claim than the temporal-independence assumption. Since no code or data are available and the main objective has unresolved normalization issues, the conditional verdict remains appropriate.","tokens_in":21121,"tokens_out":15104,"duration_ms":153158,"concrete_test":"Estimate temporal autocorrelation of the MLE residuals on the held-out viscoelastic SVE validation data. If the lag-1 autocorrelation is non-negligible, replace the independent-time Gaussian likelihood in Eq. (18) with a multivariate Gaussian using the estimated residual covariance (e.g., an AR(1) or Matérn kernel per trajectory) and retrain the LS sampler. Check whether the posterior predictive bands widen materially and whether the W1 distance in Eq. (27) changes by more than the run-to-run variability. Alternatively, compute joint trajectory coverage: the fraction of held-out trajectories whose full time path lies inside the 68% posterior predictive band; if this is far below 0.68 while per-time marginal coverage is near 0.68, the factorized likelihood overcounts correlated evidence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the Langevin-sampling hypernetwork produces predictive uncertainty that matches data variability for evolving processes. This requires a likelihood that correctly weights the evidence. Eq. (18) factorizes the likelihood over time points within each trajectory, treating per-time residuals as conditionally independent. In every demonstration the data are path-dependent: SDE trajectories in Sec. 4.1 and viscoelastic SVE responses under random-walk loadings in Sec. 4.2. Residuals at nearby times are therefore strongly correlated. A product of per-time marginals inflates the effective sample size by roughly N_t, which artificially sharpens the posterior and narrows predictive bands. The paper concedes this immediately after Eq. (18) ('This approximation can be relaxed...'), but all reported UQ bands, W1 comparisons, and parity plots use the factorized form. Moreover, the evaluation metrics are time-marginal (KDE densities, per-time standard-deviation bands), so they cannot detect miscalibrated joint or trajectory-level uncertainty. Thus the demonstrated agreement with data variability may reflect the independence assumption rather than the method's ability to quantify uncertainty for evolving processes.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a hypernetwork framework for Bayesian uncertainty quantification of neural-network models of history-dependent processes. The data model is a neural ODE for the hidden state evolution together with a trainable observation model. The posterior over a subset of model weights is represented by terminal samples of an SDE whose drift is a learned score network, and the training objective is an ELBO whose KL term is a pathwise KL between prior and posterior SDEs. The method is demonstrated on a synthetic Langevin-dynamics example with known analytical posterior statistics, on a Schlögl reaction network with multimodal response, and on a viscoelastic composite material model, with comparisons to black-box variational inference and Bayesian-last-layer variants.","tokens_in":21361,"tokens_out":12518,"duration_ms":111218,"significance":"If the central claims hold, the paper presents a scalable and fully differentiable alternative to MCMC and standard VI for UQ in NODE-based models. Strengths include a concrete algorithm (Alg. 1), a nontrivial external benchmark with known joint posterior statistics in Sec. 4.1, a pathwise KL derivation in App. A that is largely correct, and a Bayesian-last-layer extension that reduces sampling cost while retaining predictive bands comparable to the fully stochastic treatment in Sec. 4.2. The demonstrations show that the Langevin hypernetwork can capture multimodal response densities that diagonal-covariance BBVI misses. However, several formula-level inconsistencies in the objective and a strong temporal-independence assumption in the likelihood currently limit the strength of the claims about trajectory-level uncertainty.","major_comments":[{"comment":"The likelihood in Eq. (18) factorizes over time points within each trajectory. All demonstrations use data that are strongly autocorrelated in time: SDE trajectories in Sec. 4.1 and random-walk-driven viscoelastic responses in Sec. 4.2. For the Langevin-dynamics example, the OU autocorrelation between adjacent output times is exp(-gamma*Delta_t) ≈ 0.92, so residuals at neighboring times are far from independent. A product of per-time marginals overstates the evidence and systematically narrows the posterior and predictive bands. The paper concedes immediately after Eq. (18) that this approximation can be relaxed, but all reported evaluations (W1 distances, KDE densities, per-time standard-deviation bands) are time-marginal and cannot detect the resulting misspecification of joint or trajectory-level uncertainty. Because the central claim is that the method produces predictive uncertainty matching data variability for evolving processes, this approximation is load-bearing. Please either add an experiment or analysis with a multivariate likelihood that models temporal residual correlation, or explicitly reframe the claims as marginal-calibration claims and justify the independence likelihood for that purpose.","section":"Sec. 3, Eq. (18); Sec. 4"},{"comment":"The drift of the posterior SDE is written as f* = grad_w pi(w|D) and called 'the score of the posterior.' The score is the gradient of the log density, grad_w log pi(w|D). Eq. (8) correctly uses the log posterior, and App. C derives derivatives of log densities. As written, Eqs. (10)-(12) are inconsistent with Eq. (9) and with the KL expression in Eq. (14), which are based on drifts of SDEs. Please correct Eqs. (10)-(12) and the surrounding text to grad_w log pi(w|D), and state explicitly that the network NN_w in Eq. (12) approximates this log-density gradient.","section":"Sec. 3, Eqs. (10)-(12)"},{"comment":"For the SDE dw = f dτ + gamma*sqrt(2) dBτ, the transition covariance in App. A is ΣΔτ with Σ = 2γ²I, so the pathwise KL in Eq. (A-16) contributes (1/2) E[||f* - f0||²_Σ] = E[||f* - f0||²/(4γ²)]. Eq. (14) gives E[||f* - f0||²/(2γ²)], a factor of two larger. Because the KL term balances the likelihood term in the ELBO, this changes the objective actually optimized. Please align Eq. (14) with App. A (for example, by replacing γ with sqrt(2)γ or by redefining the diffusion normalization in Eq. (3)). Additionally, Eq. (14) omits the initial-condition term D_KL(q0||p0) that appears in Eq. (A-16); if the initial distributions are fixed this term is constant with respect to θ, but the stated equality should be amended accordingly.","section":"Sec. 3, Eq. (14); App. A, Eq. (A-16)"}],"minor_comments":[{"comment":"The text says 'Weiner process'; this should be 'Wiener process.'","section":"Sec. 3, after Eq. (9)"},{"comment":"The notation 'ΔB_k ∝ N(0, sqrt(Δτ) I)' is imprecise; a Brownian increment over Δτ is normally distributed as ΔB_k ∼ N(0, Δτ I). Please use standard increment notation to avoid confusion.","section":"App. A, around Eq. (A-1) and Eq. (A-13)"},{"comment":"The observation noise variance σ² is estimated from the same data used for training (σ_D from the data spread and σ_MLE from the MLE fit). This is an empirical-Bayes-like procedure; please discuss the potential effect of using the data twice and, if feasible, include a sensitivity analysis of the reported W1 metrics to the noise estimate.","section":"Sec. 3, Eq. (21)"},{"comment":"The word 'converage' in the caption should be 'coverage.'","section":"Sec. 4.2.1, Fig. 11 caption"},{"comment":"The text says 'analogous to Eq. (36)' when referring to the evolution equation for the internal state; the intended reference is likely Eq. (4), not Eq. (36).","section":"Sec. 4.2, Eq. (36)"},{"comment":"The concluding sentence describes the method as 'a competing alternative to ensemble methods such as the Stein variational gradient descent algorithm,' but no numerical comparison to SVGD is provided in this paper; please temper this claim or add a supporting reference/experiment.","section":"Sec. 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal and the central idea is promising, but the temporal-independence likelihood and the inconsistencies in Eqs. (10)-(14) need to be addressed before the claims about trajectory-level UQ can be taken at face value. The authors should be encouraged to fix the formula-level errors and to add at least one experiment or quantitative discussion of temporal correlation. I do not see grounds for rejection, and the analytical Langevin benchmark is a useful verification element."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's my take. The paper does a genuinely useful thing: it packages a learned-score Langevin sampler as a hypernetwork for NODE models of history-dependent processes, and shows that a Bayesian-last-layer variant with reoptimized point estimates recovers most of the uncertainty at a fraction of the cost. The chemical and material demonstrations are well chosen, and the analytical Langevin case with known parameter statistics is a real check, not just a visual. The appendix derivation of the path KL is transparent and mostly correct. That is a good core.\n\nThe soft spots are real but not disqualifying. There are small technical inconsistencies: Eqs. (10)-(12) write the score as grad_w pi instead of grad_w log pi, and Eq. (14) differs from App. A by a factor of 2 for gamma=1. Those are fixable typos, but they should be corrected.\n\nThe bigger issue is the likelihood in Eq. (18). It factorizes over time points within each trajectory. For the path-dependent data used everywhere in the paper, residuals within a trajectory are strongly correlated, so that product overcounts evidence and artificially narrows posterior bands. The paper acknowledges the approximation right after the equation, but all demonstrations and all evaluation metrics are time-marginal, so the reported UQ could be mis-calibrated in a way the metrics won't catch. This is not a reason to reject; it is a reason to demand either a correlated-residual likelihood or a sensitivity analysis showing the bands are robust. The noise variance sigma is also estimated from the same data, which is a minor circularity.\n\nNo code or data are released, which makes it hard to fully verify the numerical results. That is the main barrier to confidence. The central algorithm is clear, the theory in the appendix is mostly sound, and the BLL comparison is practically valuable. This paper is for people doing UQ on neural ODE models in mechanics and chemistry, not for a general ML audience. I would send it to peer review; it deserves serious referee time. With code, data, and a corrected likelihood discussion, it could be a solid contribution to the scientific ML / engineering UQ subfield.","headline":"A useful, mostly sound UQ method for neural ODE models, but the time-factorized likelihood is a load-bearing approximation that the paper never tests.","tokens_in":21901,"tokens_out":2506,"would_cite":true,"duration_ms":25749,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Langevin sampling hypernetwork gives neural-ODE models uncertainty that tracks data variability more closely than standard variational inference in the demonstrated cases, with a cheaper last-layer variant preserving most of the fidelity.","keywords":["uncertainty quantification","neural ordinary differential equations","Langevin sampling","hypernetwork","variational inference","Bayesian neural networks","stochastic differential equations","material modeling"],"falsifier":"Run Algorithm 1 on a synthetic time series generated from the Langevin dynamics model with known parameters and with observation noise drawn from an AR(1) process with positive autocorrelation at fixed marginal variance; compare the inferred posterior variance of $(W,b)$ to the known analytical posterior. If the variance shrinks below the true value as autocorrelation increases, the independence assumption in Eq. (18) is the limiting assumption.","tokens_in":20942,"feed_emoji":"🎲","tokens_out":7810,"duration_ms":68721,"temperature":0.7,"pith_summary":"The paper proposes a way to put principled uncertainty on neural-network models of evolving processes: it trains a sampler, a neural network that acts as the drift of a Langevin stochastic differential equation, to generate plausible weight vectors for the data model, whose terminal samples approximate the Bayesian posterior. The data model is a neural ordinary differential equation (NODE) for hidden state evolution coupled with an observation network, so the framework applies to history-dependent processes like chemical reactions and material response. The central claim, demonstrated on chemical kinetics and viscoelastic composites, is that this Langevin-sampling hypernetwork produces predictive uncertainty that tracks the variability in the data more faithfully than standard black-box variational inference, and that a cheaper variant with only the last network layers stochastic preserves most of that fidelity. A sympathetic reader would care because it offers a fully differentiable, scalable alternative to MCMC for attaching calibrated uncertainty bands to dynamical surrogate models used in engineering.","feed_headline":"Langevin sampler puts uncertainty bands on neural-ODE models","feed_subtitle":"Chemical and material tests show the learned sampler tracks data spread more closely than variational inference.","key_machinery":"The load-bearing object is the score-based Langevin sampler as a hypernetwork. A neural network $\\mathrm{NN}_w(w;\\theta)$ replaces the unknown score of the posterior $\\nabla_w \\log \\pi(w\\mid D)$, and its drift field, together with Brownian diffusion, defines an SDE whose stationary distribution is the target posterior. The prior is a unit-covariance Ornstein-Uhlenbeck process, so the KL divergence between prior and posterior random processes has the closed form in Eq. (14): an expectation over Langevin samples of $\\tfrac{1}{2}\\|\\mathrm{NN}_w - f_0\\|^2$ integrated in pseudo-time. This converts Bayesian inference into a differentiable optimization over $\\theta$ with no acceptance step, and it also makes possible a divided parameter space $w = \\{w^{(d)}, w^{(s)}\\}$ in which some weights are sampled while others are re-optimized point estimates, the Bayesian last layer variant.","core_discovery":"The paper asserts that approximate posterior inference for NODE-based data models can be carried out by a hypernetwork in which a score network $\\mathrm{NN}_w(w;\\theta)$ drives an SDE $dw = \\mathrm{NN}_w\\,d\\tau + \\sqrt{2}\\,dB_\\tau$; after integration to pseudo-time $\\tau_f$, the terminal weights are treated as samples from the posterior over the data-model parameters. The score network and the data model are trained jointly by maximizing an ELBO whose KL term is the path KL between the posterior SDE and an Ornstein-Uhlenbeck prior SDE, reducing to an expectation of the squared drift difference. On a classical Langevin model with an analytical posterior, the inferred joint density of the two data-model parameters matches the known covariance structure, whereas a full-covariance BBVI surrogate overestimates parameter variances by roughly 70% and distorts their correlation. On the bistable Schl\\\"ogl reaction, the LS ensemble captures both modes of the response density while diagonal-covariance BBVI explores only the wider mode. On material data, restricting stochasticity to the last layers and re-optimizing the remaining parameters gives uncertainty comparable to a fully stochastic treatment at lower sampling cost.","pith_inferences":["Extending the likelihood to account for serially correlated residuals, which the paper flags in Eq. (18) as an approximation, would change the inferred posterior widths; a testable version is to feed AR(1) noise with known autocorrelation and check whether uncertainty bands widen appropriately.","The success of the last-layer-plus-reoptimization scheme suggests a general design rule for deep surrogate models: if the inner layers are free to adapt, only the output layer needs to carry the posterior, a hypothesis that could be checked on larger-scale NODE architectures.","The amortized nature of the learned score network points toward online settings, where new trajectory data could update the posterior by continuing to train $\\theta$ rather than re-running MCMC.","The same score-SDE construction could incorporate physics constraints, such as weight bounds or convexity, by restricting the output of $\\mathrm{NN}_w$; the paper lists this as future work."],"forward_implications":["In the Langevin-dynamics exemplar, the posterior over the two model parameters $(W,b)$ recovered by LS matches the analytical marginal and joint statistics, while BBVI overestimates the standard deviations and inflates the correlation.","On the Schl\\\"ogl reaction, the LS ensemble reproduces the bimodal response distribution, whereas the diagonal-covariance BBVI baseline collapses onto the wider mode.","On hyperelastic material data, the Bayesian last layer with re-optimized deterministic parameters achieves a Wasserstein-1 distance 0.71 times that of the full stochastic treatment, while fixing the deterministic parameters at their MLE values makes it 1.18 times.","The framework lets the user tune the stochastic/deterministic split of the network parameters, trading sampling cost against uncertainty fidelity, and the convergence plots show the result is fairly insensitive to the number of replicas and SDE steps once past a modest threshold.","Because the sampler is fully differentiable and has no acceptance criterion, it can be embedded in a host simulation or optimal experimental design loop."],"supporting_citations":[{"why":"Supplies the neural ordinary differential equation data model (hidden-state evolution plus observation network) that the framework wraps with a sampler.","marker":"[2]"},{"why":"Supplies the evidence lower bound formulation used as the training loss for the approximate posterior.","marker":"[7]"},{"why":"Introduces stochastic gradient Langevin dynamics, the sampling paradigm that the hypernetwork sampler implements.","marker":"[10]"},{"why":"Provides the Ornstein-Uhlenbeck prior generative SDE and the process-KL setup used to regularize the sampler.","marker":"[17]"},{"why":"Contributes scalable gradients for stochastic differential equations, supporting end-to-end training of the drift network.","marker":"[19]"},{"why":"Establishes the diffusion-limit connection for neural SDEs, grounding the KL divergence between prior and posterior processes.","marker":"[20]"},{"why":"Introduces hypernetworks, the architecture in which the sampler generates the data-model weights.","marker":"[30]"},{"why":"Motivates the Bayesian last layer variant that restricts stochasticity to the final network layers.","marker":"[42, 43]"},{"why":"Defines black-box variational inference, the baseline method the demonstrations compare against.","marker":"[45]"}],"fun_headline_variants":["Langevin hypernetwork quantifies neural-ODE uncertainty","Score-driven Langevin sampler for neural-ODE uncertainty","Posterior for neural ODEs learned via Langevin SDE","Langevin sampler captures multi-modal uncertainty in neural ODEs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that observations at different time steps can be treated as independent (Eq. 18); if the residuals of the true process are correlated in time, this product likelihood overstates evidence and the posterior uncertainty bands will be too narrow.","fun_headline_variants_meta":{"raw":{"variants":["Langevin hypernetwork quantifies neural-ODE uncertainty","Score-driven Langevin sampler for neural-ODE uncertainty","Posterior for neural ODEs learned via Langevin SDE","Langevin sampler captures multi-modal uncertainty in neural ODEs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000683,"raw_usage":{"total_tokens":3080,"prompt_tokens":907,"completion_tokens":2173,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":523,"completion_tokens_details":{"reasoning_tokens":2103}},"tokens_in":523,"tokens_out":2173,"duration_ms":12467,"temperature":1.0,"reasoning_tokens":2103,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:38:59.815899+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Algorithm 1 on a synthetic time series generated from the Langevin dynamics model with known parameters and with observation noise drawn from an AR(1) process with positive autocorrelation at fixed marginal variance; compare the inferred posterior variance of $(W,b)$ to the known analytical posterior. If the variance shrinks below the true value as autocorrelation increases, the independence assumption in Eq. (18) is the limiting assumption.","supporting_citations":[{"cited_title":"Neural ordinary differential equa- tions","cited_arxiv_id":null,"evidence_quote":"Supplies the neural ordinary differential equation data model (hidden-state evolution plus observation network) that the framework wraps with a sampler."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Ornstein-Uhlenbeck prior generative SDE and the process-KL setup used to regularize the sampler."},{"cited_title":"Scalable gradients for stochastic differential equations","cited_arxiv_id":null,"evidence_quote":"Contributes scalable gradients for stochastic differential equations, supporting end-to-end training of the drift network."},{"cited_title":"Black Box Variational Inference","cited_arxiv_id":null,"evidence_quote":"Defines black-box variational inference, the baseline method the demonstrations compare against."}],"review_version":1}