pith. sign in

arxiv: 2511.18107 · v2 · submitted 2025-11-22 · 💻 cs.LG · stat.ML

Active Learning with Selective Time-Step Acquisition for PDEs

Yegon Kim , Hyunsu Kim , Gyeonghoon Ko , Juho Lee This is my paper

Pith reviewed 2026-05-17 05:49 UTC · model grok-4.3

classification 💻 cs.LG stat.ML
keywords active learningPDE surrogate modelingselective time-step acquisitionvariance reductionnumerical solverstrajectory samplingcomputational efficiency
0
0 comments X

The pith

By acquiring only the most important time steps from numerical solvers and approximating the rest with the surrogate, active learning can explore more diverse PDE trajectories within a fixed budget.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Traditional active learning for PDE surrogate models acquires entire trajectories from costly numerical solvers, limiting the number of examples that fit inside a computational budget. This paper introduces STAP, which instead solves the solver only for selected high-value time steps and lets the current surrogate fill in the others. Lowering the cost of each training trajectory frees budget to evaluate a larger and more varied collection of candidate trajectories. The method includes a new acquisition function that scores sets of time steps by how much they are expected to reduce model variance. Results on benchmark PDEs indicate the selective strategy improves the final surrogate compared with full-trajectory acquisition.

Core claim

The paper establishes that strategically generating only the most important time steps with the numerical solver while employing the surrogate model to approximate the remaining steps reduces the cost incurred by each trajectory and thus allows the active learning algorithm to try out a more diverse set of trajectories given the same budget, with the acquisition function estimating the utility of time-step sets via approximated variance reduction.

What carries the argument

The STAP acquisition function, which scores a candidate set of time steps by estimating the variance reduction that would result from acquiring those steps from the solver and using the surrogate for the rest.

If this is right

  • More candidate trajectories can be considered during each active learning iteration for the same solver budget.
  • Training data can cover a wider range of initial conditions and parameters because each example costs less.
  • The surrogate can achieve lower prediction error by incorporating information from a greater number of distinct trajectories.
  • Longer-time or higher-resolution PDE problems become tractable under fixed computational limits.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same partial-acquisition idea could be tested on other sequential simulation tasks such as molecular dynamics trajectories or fluid-flow rollouts.
  • If the surrogate proves reliable at filling gaps, future work could acquire even sparser subsets of time steps per trajectory.
  • Combining STAP with multi-fidelity solvers, where cheap models handle the approximated steps, is a natural next direction.
  • The approach may need safeguards on problems with sharp fronts or sensitive dependence on initial data where interpolation errors grow quickly.

Load-bearing premise

The surrogate model's approximations for the non-acquired time steps must remain accurate enough not to degrade the overall quality of the learned surrogate or the active learning decisions.

What would settle it

Compare surrogate test error after active learning runs that spend identical total solver time on full trajectories versus on STAP-selected partial trajectories; if the full-trajectory version yields equal or lower error, the selective approach fails to deliver its claimed advantage.

Figures

Figures reproduced from arXiv: 2511.18107 by Gyeonghoon Ko, Hyunsu Kim, Juho Lee, Yegon Kim.

Figure 1
Figure 1. Figure 1: PCA of surrogate model hidden layer’s activa￾tion patterns on states of the incompressible Navier-Stokes equation. The left figure highlights states within 10 trajec￾tories, and the right figure highlights the same number of states chosen randomly. gate models. Acquiring entire trajectories is inefficient mainly for two reasons. First, states within a trajectory are often strongly correlated, undermining t… view at source ↗
Figure 2
Figure 2. Figure 2: Illustrated overview of STAP. This illustration describes one round of AL. 2.2. Problem Setting Our goal is to obtain a surrogate model Gˆ that approximates the expensive numerical solver G with low error Eu0∼p(u0) h err  (G (i) [u 0 ])L i=1,(Gˆ(i) [u 0 ])L i=1i (3) where err(·, ·) is an error metric. Obtaining the surrogate model requires sampling training data from the numerical solver, which incurs a … view at source ↗
Figure 3
Figure 3. Figure 3: Log RMSE of AL strategies, measured across 10 rounds of acquisition. Each round incurs constant cost of data acquisition, namely the budget B. leave out Core-Set (Sener and Savarese, 2017) because it generally underperforms compared to the above methods, according to both Holzmüller et al. (2023) and Musekamp et al. (2024). 5.2. Target PDEs We evaluate our method on a range of PDEs. The first is the Burger… view at source ↗
Figure 4
Figure 4. Figure 4: Quantiles of log RMSE on Burgers measured across 10 rounds of acquisition. (a) Burgers (b) KdV (c) KS (d) INS (e) CNS [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Timesteps chosen by SBAL+STAP. Each row corresponds to an acquired trajectory, where the black cells indicate the selected time steps. We show twenty trajectories acquired in the first rounds of active learning. plots that SBAL+STAP outperforms other AL baselines in a robust manner. Most notably, it improves performance on the KS equation, where no other baseline improves signifi￾cantly over Random selecti… view at source ↗
Figure 6
Figure 6. Figure 6: From left to right, ground truth trajectory on KS, trajectory predicted by surrogate model trained with 32 trajectories, and trajectory predicted by surrogate model trained with one trajectory. T from 100 to 10, which reduces the cost by a factor of 10. We call this variant STAP 10. The wall-clock time of each baseline method and STAP is measured with a single NVIDIA GeForce RTX 2080 Ti GPU, and summarized… view at source ↗
Figure 7
Figure 7. Figure 7: Active learning with initial training dataset containing one trajectory, as opposed to 32 trajectories in the main experiment. STAP is robust to initially inaccurate surrogate models [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: PCA of FNO hidden layer activation for PDE states sampled by SBAL (left) and SBAL+STAP (right) during the first round of active learning on KS. of the second variant. 5.8. Out-of-distribution Synthetic Inputs Inaccurate surrogate models might synthesize inputs that lie far from the ground truth distribution, harming the rep￾resentativeness (Dongrui Wu, 2018). Indeed, under limited training data, the surrog… view at source ↗
Figure 10
Figure 10. Figure 10: Task settings assumed by previous works in active learning of PDEs. There are three primary tasks in active learning for PDEs, each depending on the type of surrogate model being trained. The first task, univariate Quantity of Interest (QoI) prediction, focuses on learning a model to directly predict a scalar QoI, denoted as y, from an initial condition u 0 . The second task, single-state prediction, invo… view at source ↗
Figure 11
Figure 11. Figure 11: Example trajectories of PDEs. (a), (b), (c): Horizontal and vertical axes represent the temporal and spatial domain. (d), (e): Two-dimensional states at six time points are shown. shocks arise due to the the advection term u∂xu, while the presence of the diffusion term ∂xxu prevents the formation of discontinuities in the wave. We set ν = 0.01 to ensure the formation of sharp gradients while maintaining n… view at source ↗
Figure 12
Figure 12. Figure 12: Log RMSE quantiles 19 [PITH_FULL_IMAGE:figures/full_fig_p019_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Log RMSE of AL strategies with multi-step FNO, across 10 rounds of acquisition [PITH_FULL_IMAGE:figures/full_fig_p021_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Log RMSE of AL strategies, measured across 20 rounds of acquisition. E. Comparison of Costs with Our Numerical Solvers Our paper does not claim direct computational speedups on our benchmark PDEs; instead, it relies on the benchmark PDEs as proxies reflecting realistic, expensive simulations. Our surrogate metric, the number of numerical solver-simulated timesteps, effectively represents relative computat… view at source ↗
Figure 15
Figure 15. Figure 15: PCA of FNO hidden layer’s activation pattern for both entire trajectories (black) and sparsely sampled time steps (red) 22 [PITH_FULL_IMAGE:figures/full_fig_p022_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Log RMSE of AL strategies on time-dependent INS [PITH_FULL_IMAGE:figures/full_fig_p024_16.png] view at source ↗
read the original abstract

Accurately solving partial differential equations (PDEs) is critical to understanding complex scientific and engineering phenomena, yet traditional numerical solvers are computationally expensive. Surrogate models offer a more efficient alternative, but their development is hindered by the cost of generating sufficient training data from numerical solvers. In this paper, we present a novel framework for active learning in PDE surrogate modeling that reduces this cost. Unlike the existing AL methods for PDEs that always acquire entire PDE trajectories, our approach, STAP (**S**elective **T**ime-Step **A**cquisition for **P**DEs), strategically generates only the most important time steps with the numerical solver, while employing the surrogate model to approximate the remaining steps. This reduces the cost incurred by each trajectory and thus allows the active learning algorithm to try out a more diverse set of trajectories given the same budget. To accommodate this novel framework, we develop an acquisition function that estimates the utility of a set of time steps by approximating its resulting variance reduction. We demonstrate the effectiveness of our method on several benchmark PDEs.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces STAP, a selective time-step acquisition framework for active learning in PDE surrogate modeling. Unlike standard methods that acquire full trajectories, STAP uses the numerical solver only for selected important time steps and approximates the rest with the surrogate. A new acquisition function is proposed to estimate the variance reduction utility of the selected time-step sets. The effectiveness is demonstrated on several benchmark PDEs.

Significance. If the surrogate approximations preserve both model quality and acquisition correctness, the method could meaningfully lower the per-trajectory cost of data generation for PDE surrogates, permitting greater trajectory diversity within a fixed computational budget. This directly targets a practical bottleneck in existing active-learning pipelines for time-dependent PDEs.

major comments (2)
  1. [Acquisition function and algorithm description] The acquisition function estimates utility via approximated variance reduction over trajectories that are only partially solved by the numerical integrator. Because the surrogate fill-in is used both to train the model and to compute the acquisition scores, errors in early rounds (when the surrogate is trained on very few points) directly influence which time-step sets are chosen next. The manuscript provides no analysis, error bounds, or ablation that quantifies how this feedback affects selection quality or overall convergence.
  2. [Experimental results] The central empirical claim is that selective acquisition yields more diverse trajectories and better surrogates for the same budget. The abstract asserts effectiveness on benchmark PDEs, yet the evaluation must report concrete metrics (e.g., relative L2 error vs. number of solver calls, comparison against full-trajectory AL baselines, and diversity measures) to substantiate that the promised cost reduction is realized without degrading final accuracy.
minor comments (2)
  1. [Abstract] The abstract would be strengthened by including one or two quantitative highlights (error reduction, budget savings) rather than a purely qualitative statement of effectiveness.
  2. [Method overview] Clarify the precise definition of a 'trajectory' and the criterion used to designate 'most important' time steps; inconsistent usage appears in the method overview.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and insightful comments, which have helped us improve the manuscript. We address each major comment below and have revised the paper accordingly to provide additional analysis and experimental details.

read point-by-point responses

  1. Referee: [Acquisition function and algorithm description] The acquisition function estimates utility via approximated variance reduction over trajectories that are only partially solved by the numerical integrator. Because the surrogate fill-in is used both to train the model and to compute the acquisition scores, errors in early rounds (when the surrogate is trained on very few points) directly influence which time-step sets are chosen next. The manuscript provides no analysis, error bounds, or ablation that quantifies how this feedback affects selection quality or overall convergence.

    Authors: We appreciate the referee's observation regarding the potential impact of early surrogate errors on acquisition decisions. This feedback loop is inherent to model-based active learning approaches. While the original manuscript emphasized the overall framework and empirical results, we acknowledge the absence of targeted analysis on this issue. In the revised manuscript, we have added a dedicated discussion subsection on the robustness of the acquisition function to initial model inaccuracies, along with an ablation study that examines selection behavior and convergence under varying numbers of initial training points. These additions empirically demonstrate that the method maintains effective selection quality despite early-round errors. We do not provide formal error bounds, as the approach is primarily algorithmic and empirical rather than theoretical. revision: yes

  2. Referee: [Experimental results] The central empirical claim is that selective acquisition yields more diverse trajectories and better surrogates for the same budget. The abstract asserts effectiveness on benchmark PDEs, yet the evaluation must report concrete metrics (e.g., relative L2 error vs. number of solver calls, comparison against full-trajectory AL baselines, and diversity measures) to substantiate that the promised cost reduction is realized without degrading final accuracy.

    Authors: We agree that more explicit and quantitative reporting strengthens the empirical claims. The original experiments included benchmark results and visualizations demonstrating the benefits of selective acquisition, but we have revised the experimental section to include plots of relative L2 error versus the number of solver calls, direct side-by-side comparisons with full-trajectory active learning baselines, and quantitative diversity metrics (such as average pairwise trajectory distances and coverage of the parameter space). These updated results confirm that STAP achieves comparable or superior surrogate accuracy while incurring substantially lower per-trajectory solver costs, thereby enabling greater trajectory diversity within a fixed budget. revision: yes

Circularity Check

0 steps flagged

No circularity; derivation is self-contained

full rationale

The paper introduces STAP as a new selective time-step acquisition strategy for PDE active learning. The acquisition function is defined to estimate variance reduction over partial trajectories using the current surrogate model, which is a standard construction in Bayesian active learning and does not reduce any claimed prediction to a fitted input by construction. No self-citations are invoked as load-bearing uniqueness theorems, no ansatz is smuggled, and no known result is merely renamed. The central claim (cost reduction via selective acquisition enabling more diverse trajectories) rests on the explicit new framework rather than tautological re-use of inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Review is based solely on the abstract; specific free parameters or axioms in the acquisition function are not described. The framework implicitly assumes standard active learning and surrogate modeling components.

axioms (1)
  • domain assumption Surrogate model approximations for unselected time steps are sufficiently accurate for the active learning loop to remain effective.
    Required for the cost-reduction claim to hold without introducing large errors.

pith-pipeline@v0.9.0 · 5490 in / 1294 out tokens · 59318 ms · 2026-05-17T05:49:53.297651+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

9 extracted references · 9 canonical work pages · 2 internal anchors

  1. [1]

    Deep batch active learning by diverse, uncertain gradient lower bounds

    Arthurs, Christopher J and Andrew P King (2021). “Active training of physics-informed neural networks to aggre- gate and interpolate parametric solutions to the Navier- Stokes equations”. In:Journal of Computational Physics 438, p. 110364 (cit. on p. 5). Ash, Jordan T et al. (2019). “Deep batch active learning by diverse, uncertain gradient lower bounds”....

    work page arXiv 2021

  2. [2]

    Bayesian Active Learning for Classification and Preference Learning

    Academic press (cit. on p. 1). Holzmüller, David et al. (2023). “A framework and bench- mark for deep batch active learning for regression”. In: Journal of Machine Learning Research24.164, pp. 1–81 (cit. on pp. 1, 4–7, 17). Houlsby, Neil et al. (2011). “Bayesian active learning for classification and preference learning”. In:arXiv preprint arXiv:1112.5745...

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  3. [3]

    Adam: A Method for Stochastic Optimization

    Cambridge University Press (cit. on p. 16). Kassam, Aly-Khan and Lloyd N Trefethen (2005). “Fourth- order time-stepping for stiff PDEs”. In:SIAM Journal on Scientific Computing26.4, pp. 1214–1233 (cit. on p. 16). Kingma, Diederik P (2014). “Adam: A method for stochas- tic optimization”. In:arXiv preprint arXiv:1412.6980 (cit. on p. 6). Kirsch, Andreas, Jo...

    work page internal anchor Pith review Pith/arXiv arXiv 2005

  4. [4]

    Active learning for neural PDE solvers.CoRR, abs/2408.01536,

    Springer Science & Business Media (cit. on p. 1). Musekamp, Daniel et al. (2024). “Active Learning for Neu- ral PDE Solvers”. In:arXiv preprint arXiv:2408.01536 (cit. on pp. 1–6). Pestourie, Raphaël et al. (2020). “Active learning of deep surrogates for PDEs: application to metasurface design”. In:npj Computational Materials6.1, p. 164 (cit. on pp. 1, 5)....

    work page arXiv 2024

  5. [5]

    how much total uncertainty will be reduced by sampling these time steps

    The second task,single-state prediction, involves learning a model to predict a single state transition fromu 0 tou 1 over a fixed time interval∆t. The third task,autoregressive trajectory prediction, aims to approximate the ground truth evolution operatorGusing a surrogate model to predict the entire time evolution of the states. Fig. 10 provides a visua...

    work page 2009

  6. [6]

    (a), (b), (c): Horizontal and vertical axes represent the temporal and spatial domain

    These 14 Active Learning with Selective Time-Step Acquisition for PDEs (a)Burgers (b)KdV (c)KS (d)INS (e)CNS Figure 11:Example trajectories of PDEs. (a), (b), (c): Horizontal and vertical axes represent the temporal and spatial domain. (d), (e): Two-dimensional states at six time points are shown. shocks arise due to the the advection termu∂ xu, while the...

    work page 2022

  7. [7]

    or carefully crafted synthetic data (Jarrin et al., 2006; Kusner et al., 2017). For 1D equations, the states are sampled from truncated Fourier series with random coefficients (Brandstetter et al., 2022a), and for the 2D INS equation, states are sampled from a Gaussian random field as described in Z. Li et al. (2020). Similarly, for the CNS equation, the ...

    work page 2006

  8. [8]

    (15) The metrics are averaged across all trajectories in the test set

    (13) Similarly, theNRMSEis defined as sP i,j ∥ui(xj)− ˆui(xj)∥2 2P i,j ∥ui(xj)∥2 2 (14) and theMAEas 1 LNx LX i=1 NxX j=1 |ui(xj)− ˆui(xj)|. (15) The metrics are averaged across all trajectories in the test set. We also report their logarithmic values averaged across all AL rounds, following Holzmüller et al. (2023). Note that we do not use a committee’s ...

    work page 2023

  9. [9]

    Initial Bernoulli sampling always performs the worst, possibly because they rarely see the time steps at the end. D.4. Time-dependent Incompressible Navier-Stokes We have performed an experiment on a time-dependent incompressible Navier Stokes equation, simply by using the time- dependent external force in our current INS equation. The new forcing term is...

    work page 2075