Pith. sign in

REVIEW 3 major objections 5 minor 27 references

Early Stopping Bayesian Optimization for Controller Tuning

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Pith's one-line read Stopping bad controller trials early can cut Bayesian optimization time by up to 48 percent in simulation and 35 percent on hardware, without sacrificing final controller performance.

desk verdict A clean, practical BO-for-control paper that makes a genuinely new move (early stopping inside the optimizer) and backs it with code and hardware, but its noise-blind stopping rule means the 'no quality loss' claim is only empirical, not guaranteed. read the letter →

arxiv 2501.11532 v1 pith:BKJDE2JQ submitted 2025-01-20 eess.SY cs.SY

classification eess.SYcs.SY
keywords BayesianoptimizationcontrollertuningearlystoppingpartialobservationsGaussianprocesssurrogatetime-integratedcostexperimentationtime
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that Bayesian optimization for controller tuning wastes time when every trial runs a fixed-length episode, because a trial that is already behaving badly can be declared hopeless long before the episode ends. It introduces early-stopping BO (ESBO), which aborts an episode as soon as the cumulative cost passes the current best observed cost, and three heuristics that let the optimizer learn from the resulting partial episodes. Across five simulated tuning problems and one three-tank hardware experiment, the best heuristic, ESBO-GP, reaches the same final controller performance as standard BO while using up to 48 percent less simulation time and 35 percent less hardware time. If the claim holds, automated controller tuning becomes substantially less time-consuming without changing the quality of the final controller.

What carries the argument

The central mechanism is the stopping rule $S(j_{1,k}, \ldots, j_{t,k}, J^*_k)$, which returns true as soon as the cumulative cost $\sum_{\tau=1}^{t} j_{\tau,k}$ reaches the current best observed cost $J^*_k$; when this happens the episode is aborted. Because the partial sum underestimates the true cost, ESBO builds virtual data points: ESBO-C assigns pessimistic Gaussian-process predictions capped by crash-constraint bounds, ESBO-TR reinterprets fitness as the negative time at which cumulative cost would cross $J^*_k$, and ESBO-GP trains separate Gaussian processes on the unobserved time sections and samples a probabilistic completion of the partial episode. The virtual data points guide the acquisition function away from regions whose episodes were stopped early, while preserving the higher-fidelity information of fully completed episodes.

What would settle it

A decisive test would run ESBO-GP and vanilla BO on a noisy benchmark with a known optimum and high observation noise, tracking final regret after a fixed budget; if ESBO-GP's final regret is consistently worse than vanilla BO's when the initial samples contain an unusually low $J^*_k$, the threshold rule is the cause. A second check compares the two on a problem whose cost is concentrated late in the episode, where early stopping should save little time and could remove useful information.

Watch

Extended reading notes

Core claim

For objectives that are sums of nonnegative per-time-step costs, the paper claims that the standard fixed-episode evaluation in Bayesian optimization can be replaced by an early-stopping rule without corrupting the surrogate model. The rule stops an episode at the first time $T_k$ where $\sum_{\tau=1}^{T_k} j_{\tau,k} \ge J^*_k$, the current best observed cost. Because the resulting partial sum underestimates the true cost, the paper constructs virtual observations: a pessimistic crash-style completion, a time-reformulated fitness, or a probabilistic completion from Gaussian processes trained on the missing time sections. In the experiments, the probabilistic completion (ESBO-GP) gives the largest time savings and a final solution comparable to vanilla BO.

Load-bearing premise

The load-bearing premise is that the current best observed cost $J^*_k$ is a reliable threshold for declaring a trial hopeless, even though it is a noisy finite-sample estimate; if an unlucky low observation makes the threshold too aggressive, the optimizer can be steered away from good parameters.

Editorial extensions

If this is right

  • Practitioners tuning controllers with Bayesian optimization can switch to ESBO-GP and expect the optimization campaign to finish in roughly half the simulation time, with final controller quality comparable to vanilla BO.
  • The query efficiency of ESBO stays close to vanilla BO, meaning the time savings come from shorter episodes rather than from needing fewer parameter evaluations.
  • Random search with the same early-stopping rule becomes more competitive than without stopping, but still underperforms the Bayesian optimization variants, so the speedup is not merely an artifact of the stopping rule.
  • The three heuristics provide a baseline set for future work on partial observations in Bayesian optimization, with methods that use all available partial data (ESBO-GP) tending to perform best.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The early-stopping threshold $J^*_k$ is a noisy estimate, so under high observation noise the rule could systematically avoid regions whose early samples were unlucky; a natural extension is a threshold that accounts for uncertainty in $J^*_k$ rather than using the raw best observation.
  • The same pattern should transfer to any black-box optimization whose objective is a time-integral of nonnegative costs, not just control: simulation-based design or hyperparameter tuning where trials can be cut short could benefit from the same early-stopping machinery.
  • A risk-aware variant could stop even earlier by predicting the probability that the remaining cost can still beat $J^*_k$, using the section-specific Gaussian processes; the paper notes such a probabilistic early stop as a future possibility.
  • The hardware result at 35 percent time saving suggests real-world applicability, but the noise-free simulation savings may shrink as noise grows; testing on a noisy benchmark with a known optimum would quantify the bias from the noisy threshold.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Early Stopping Bayesian Optimization (ESBO) for controller tuning with time-integrated costs. It introduces a stopping rule (Eq. 12) that aborts an episode once the cumulative cost reaches the current best observed cost, and three heuristics (ESBO-C, ESBO-TR, ESBO-GP) to incorporate the resulting partial observations into a GP surrogate. The method is evaluated on five simulation tasks and one three-tank hardware experiment, reporting time savings up to 48% in simulation and 35% in hardware relative to vanilla BO while reaching comparable final performance.

Significance. If the empirical claims hold, ESBO offers a practically useful way to reduce experimentation time in controller tuning, with a simple, parameter-free stopping rule and three heuristics for handling partial observations. The paper contributes a new problem formulation, a public implementation, and a benchmark suite spanning five simulation tasks and a hardware testbed. Strengths include the clear hyperparameter settings, the reproducibility of the code, and the comparison against standard baselines. The main limitations are the lack of any theoretical or systematic analysis of the stopping rule's behavior under observation noise and the limited statistical power of the empirical evaluation; the central claim is plausible but not yet fully established.

major comments (3)
  1. [Sec. V-A, Eq. (12)] The stopping rule uses the current best observed cost J*_k as a threshold, but J*_k is a noisy finite-sample estimate (Eq. (4)). A single unlucky low-noise observation can set J*_k below the true cost of the global optimum; thereafter the rule aborts any trial whose cumulative cost crosses that artificially low value, even if the full-episode cost would have beaten the incumbent. Because stopped trials enter the surrogate only via virtual data (Sec. V-B), the search is systematically steered away from regions that could contain the optimum. The paper does not analyze this bias or provide a noise-robust modification, and the single hardware noise level (Sec. VI-D) cannot certify the general claim in Sec. VII that ESBO works 'without sacrificing final solution quality.' Please add a synthetic benchmark that sweeps the observation-noise level or a theoretical analysis of the stopping rule's safety.
  2. [Sec. VI-C, Fig. 2] The simulation results are reported only as median curves without error bars, confidence bands, or significance tests. The headline 48% speedup is a point estimate based on the median final performance of ESRS, and with 10 seeds per task the observed differences may be within noise. Please report per-task results with interquartile ranges or bootstrap intervals, and make explicit how many seeds are used for each of the five tasks.
  3. [Sec. VI-D, Fig. 4] The hardware experiment shows a statistically significant advantage only in a transient interval (around 15-26 min); after about 30 minutes the rank-sum test is insignificant. This weakens the 'up to 35%' speedup claim, since the time-to-target comparison is based on a single noise realization and no correction for multiple comparisons is applied. Please report the distribution of times-to-target across the eight runs and apply a proper multiple-testing correction or a time-to-target summary.
minor comments (5)
  1. [Sec. VI-A] The sentence 'The simulation and hardware experiments aim at determinining the of-the-box-performance' contains typos; it should read 'determining the out-of-the-box performance.'
  2. [Sec. VI-C] In the sentence 'The simulation time needed to achieve the median final performance of ESRS is reduced by 48% from 62% to 32% percent', the word 'percent' is redundant, and the baseline 'median final performance of ESRS' should be defined explicitly as a target value.
  3. [Sec. V-B, Eq. (17)] The index p is used before it is defined; please specify that p denotes the section containing T_l.
  4. [Algorithm 1, line 6] The phrase 'identify current optimum from Dk' is ambiguous in noisy settings; it should be 'identify the current best observed cost from Dk' to distinguish it from the true optimum J*.
  5. [Sec. V-B, ESBO-C] The crash-constraint formula in Eq. (13) is imported from reference [13] with only a brief explanation; please provide a derivation or cite a peer-reviewed source so that the min/max bounds are self-contained.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the early-stopping rule is a valid sufficient condition, and the empirical claims are benchmarked against external baselines.

full rationale

The central derivation chain is self-contained and does not reduce to its inputs. Equation (12) stops an episode when the cumulative non-negative cost reaches the current best observed cost. Because the full-episode cost is the sum of non-negative per-step costs, reaching the current best before the end implies the full cost cannot be smaller than the current best. This is a mathematical sufficient condition, not a definitional or fitted relation, so it is not circular. The ESBO heuristics for handling partial observations are explicitly heuristic and are evaluated against vanilla BO, random search, and early-stopping random search on five simulation tasks and one hardware experiment; the final performance comparison is against actual full-episode outcomes, not against quantities derived from the proposed method's own virtual data. The paper does cite the authors' prior work, notably [13] for crash-constraint handling and [5] for benchmarks and metrics. These self-citations are not load-bearing for the central early-stopping claim: the crash-constraint formula is a borrowed auxiliary mechanism, and the benchmarks are external test problems and baselines. The reported speedup is an empirical observation about the proposed algorithm, not a prediction derived from a fitted parameter. The noise sensitivity of the threshold J*_k in Eq. (12) is a legitimate robustness concern, but it is a correctness risk rather than a circularity: the rule is not defined in terms of the outcomes it is claimed to predict. No uniqueness theorem is imported from the authors' prior work, and no known result is merely renamed. Overall, the derivation and evaluation are independent of the conclusions in a way that does not exhibit circular reasoning.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a sum-of-non-negative-cost objective and on standard GP assumptions. The ESBO-GP heuristic additionally assumes that separate GPs can predict unobserved time sections. No new physical entities are introduced.

free parameters (2)
  • GP length-scale bounds = correlation 0.1 at between half domain and 1% of domain
    Set by hand in Sec. VI-A to keep length scales within a similar order as the optimization domain; affects surrogate smoothness and results.
  • ESBO-C pessimism multiplier = 3
    Eq. (13) uses 3 standard deviations above the pessimistic mean, inherited from crash-constraint literature [13]; a tuning constant not derived from the problem.
assumptions (5)
  • domain assumption Objective is a sum of non-negative per-step costs (Eq. 2).
    Restricts ESBO to MSE/MAE/LQR-type costs and excludes settling time or frequency-domain metrics.
  • standard math Gaussian process surrogate with zero mean and anisotropic squared-exponential kernel.
    Assumed in Sec. IV-A; standard BO modeling choice.
  • domain assumption Observation noise is additive, iid Gaussian (Eq. 4).
    Used in GP updates; the hardware experiment estimates its variance.
  • ad hoc to paper The per-segment cost in ESBO-GP can be modeled by independent GPs across time sections.
    Introduced in Sec. V-B(c) to predict unobserved episode portions; lacks theoretical justification.
  • domain assumption Early stopping rule compares cumulative cost to the current best observation J*_k, treated as a reliable threshold.
    Eq. (12) uses noisy J*_k; in noisy settings this may stop episodes prematurely.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Early Stopping Bayesian Optimization for Controller Tuning." pith.science (2026). https://pith.science/paper/BKJDE2JQ

@misc{pith2026250111532,
  author       = {Pith},
  title        = {Pith review of: Early Stopping Bayesian Optimization for Controller Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BKJDE2JQ}},
  note         = {Machine review of arXiv:2501.11532}
}
read the original abstract

Manual tuning of performance-critical controller parameters can be tedious and sub-optimal. Bayesian Optimization (BO) is an increasingly popular practical alternative to automatically optimize controller parameters from few experiments. Standard BO practice is to evaluate the closed-loop performance of parameters proposed during optimization on an episode with a fixed length. However, fixed-length episodes can be wasteful. For example, continuing an episode where already the start shows undesirable behavior such as strong oscillations seems pointless. Therefore, we propose a BO method that stops an episode early if suboptimality becomes apparent before an episode is completed. Such early stopping results in partial observations of the controller's performance, which cannot directly be included in standard BO. We propose three heuristics to facilitate partially observed episodes in BO. Through five numerical and one hardware experiment, we demonstrate that early stopping BO can substantially reduce the time needed for optimization.

Figures

Figures reproduced from arXiv: 2501.11532 by the authors.

Figure 1
Figure 1. Illustrative example of ESBO-GP: The first and the second samples [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Results of the simulation study. For an explanation of the metrics see Sec. VI-B. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Three tank hardware test bed at the model factory of [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 25 canonical work pages

  1. [1]

    Garnett, Bayesian Optimization

    R. Garnett, Bayesian Optimization . Cambridge University Press, 2023

  2. [2]

    Automatic LQR tuning based on Gaussian process global optimization,

    A. Marco, P. Hennig, J. Bohg, S. Schaal, and S. Trimpe, “Automatic LQR tuning based on Gaussian process global optimization,” in 2016 IEEE International Conference on Robotics and Automation (ICRA), IEEE, 2016, pp. 270–277

  3. [3]

    Experimental automatic calibration of a semi- active suspension controller via bayesian optimization,

    G. Savaia, Y . Sohn, S. Formentin, G. Panzani, M. Corno, and S. M. Savaresi, “Experimental automatic calibration of a semi- active suspension controller via bayesian optimization,” Control Engineering Practice, vol. 112, p. 104 826, 2021

  4. [4]

    Contextual tuning of model predictive control for autonomous racing,

    L. P. Fr ¨ohlich, C. K ¨uttel, E. Arcari, L. Hewing, M. N. Zeilinger, and A. Carron, “Contextual tuning of model predictive control for autonomous racing,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2022, pp. 10 555–10 562

  5. [5]

    Stenger and D

    D. Stenger and D. Abel, Benchmark of bayesian optimization and metaheuristics for control engineering tuning problems with crash constraints, 2022. [Online]. Available: https://arxiv.org/ abs/2211.02571

  6. [6]

    Bayesian optimization of soft exosuits using a metabolic estimator stopping process,

    M. Kim, C. Liu, J. Kim, S. Lee, A. Meguid, C. J. Walsh, and S. Kuindersma, “Bayesian optimization of soft exosuits using a metabolic estimator stopping process,” in 2019 International Con- ference on Robotics and Automation (ICRA) , 2019, pp. 9173–9179

  7. [7]

    Extremum seeking controller tuning for heat pump optimization using failure-robust bayesian optimization,

    A. Chakrabarty, D. J. Burns, M. Guay, and C. R. Laughman, “Extremum seeking controller tuning for heat pump optimization using failure-robust bayesian optimization,” Journal of Process Control, vol. 120, pp. 86–96, 2022

  8. [8]

    Taking the Human Out of the Loop: A Review of Bayesian Opti- mization,

    B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and N. de Freitas, “Taking the Human Out of the Loop: A Review of Bayesian Opti- mization,” Proceedings of the IEEE , vol. 104, no. 1, pp. 148–175, 2016

Show all 27 references
  1. [9]

    Automated reinforcement learning (autorl): A survey and open problems,

    J. Parker-Holder, R. Rajan, X. Song, A. Biedenkapp, Y . Miao, T. Eimer, B. Zhang, V . Nguyen, R. Calandra, A. Faust, et al. , “Automated reinforcement learning (autorl): A survey and open problems,” Journal of Artificial Intelligence Research , vol. 74, pp. 517–568, 2022

  2. [10]

    A survey on policy search algorithms for learning robot controllers in a handful of trials,

    K. Chatzilygeroudis, V . Vassiliades, F. Stulp, S. Calinon, and J.-B. Mouret, “A survey on policy search algorithms for learning robot controllers in a handful of trials,” IEEE Transactions on Robotics , vol. 36, no. 2, pp. 328–347, 2020

  3. [11]

    Automated tuning of nonlinear kalman filters for optimal trajectory tracking performance of auvs,

    M. Nitsch, D. Stenger, and D. Abel, “Automated tuning of nonlinear kalman filters for optimal trajectory tracking performance of auvs,” IFAC-PapersOnLine, vol. 56, no. 2, pp. 11 608–11 614, 2023, 22nd IFAC World Congress

  4. [12]

    Min-max hyperparam- eter tuning, with application to fault detection,

    J. Marzat, H. Piet-Lahanier, and E. Walter, “Min-max hyperparam- eter tuning, with application to fault detection,” 18th IFAC World Congress, 2011

  5. [13]

    Automatic tuning of control engineering algo- rithms with Bayesian optimization,

    D. Stenger, “Automatic tuning of control engineering algo- rithms with Bayesian optimization,” Dissertation, Rheinisch- Westf¨alische Technische Hochschule Aachen, 2023. [Online]. Avail- able: https : / / publications . rwth - aachen . de / record/971795

  6. [14]

    Safe contextual Bayesian optimization for sustainable room tem- perature PID control tuning,

    M. Fiducioso, S. Curi, B. Schumacher, M. Gwerder, and A. Krause, “Safe contextual Bayesian optimization for sustainable room tem- perature PID control tuning,” in Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, (IJCAI-19), Jul. 2019,...

  7. [15]

    Performance-oriented model learning for control via multi- objective Bayesian optimization,

    G. Makrygiorgos, A. D. Bonzanini, V . Miller, and A. Mes- bah, “Performance-oriented model learning for control via multi- objective Bayesian optimization,”Computers & Chemical Engineer- ing, vol. 162, p. 107 770, 2022

  8. [16]

    Safety-aware cascade controller tuning using con- strained Bayesian optimization,

    M. Khosravi, C. K ¨onig, M. Maier, R. S. Smith, J. Lygeros, and A. Rupenyan, “Safety-aware cascade controller tuning using con- strained Bayesian optimization,” IEEE Transactions on Industrial Electronics, vol. 70, no. 2, pp. 2128–2138, 2023

  9. [17]

    On controller tuning with time-varying bayesian optimization,

    P. Brunzema, A. V on Rohr, and S. Trimpe, “On controller tuning with time-varying bayesian optimization,” in 2022 IEEE 61st Con- ference on Decision and Control (CDC) , 2022, pp. 4046–4052

  10. [18]

    Virtual vs. real: Trading off simulations and physical experiments in reinforcement learning with bayesian optimization,

    A. Marco, F. Berkenkamp, P. Hennig, A. P. Schoellig, A. Krause, S. Schaal, and S. Trimpe, “Virtual vs. real: Trading off simulations and physical experiments in reinforcement learning with bayesian optimization,” in 2017 IEEE International Conference on Robotics and Automation...

  11. [19]

    Local bayesian optimization for controller tuning with crash constraints,

    A. von Rohr, D. Stenger, D. Scheurenberg, and S. Trimpe, “Local bayesian optimization for controller tuning with crash constraints,” at–Automatisierungstechnik, vol. 72, no. 4, pp. 281–292, 2024

  12. [20]

    Bayesian optimiza- tion with safety constraints: Safe and automatic parameter tuning in robotics,

    F. Berkenkamp, A. Krause, and A. P. Schoellig, “Bayesian optimiza- tion with safety constraints: Safe and automatic parameter tuning in robotics,” Machine Learning, vol. 112, pp. 3713–3747, 2021

  13. [21]

    Robot learning with crash constraints,

    A. Marco, D. Baumann, M. Khadiv, P. Hennig, L. Righetti, and S. Trimpe, “Robot learning with crash constraints,” IEEE Robotics and Automation Letters , vol. 6, no. 2, pp. 1439–1446, 2021

  14. [22]

    Swersky, J

    K. Swersky, J. Snoek, and R. P. Adams, Freeze-thaw bayesian optimization, 2014. [Online]. Available: https://arxiv.org/ abs/1406.3896

  15. [23]

    Bayesian optimization meets Bayesian optimal stopping,

    Z. Dai, H. Yu, B. K. H. Low, and P. Jaillet, “Bayesian optimization meets Bayesian optimal stopping,” in Proceedings of the 36th International Conference on Machine Learning (ICML) , PMLR, 2019, pp. 1496–1506

  16. [24]

    Bayesian optimization with expensive integrands,

    S. Toscano-Palmerin and P. I. Frazier, “Bayesian optimization with expensive integrands,” SIAM Journal on Optimization , vol. 32, no. 2, pp. 417–444, 2022

  17. [25]

    Gaussian Processes for Machine Learning,

    C. E. Rasmussen and Williams C. K. I., “Gaussian Processes for Machine Learning,” 2006

  18. [26]

    Max-value entropy search for efficient bayesian optimization,

    Z. Wang and S. Jegelka, “Max-value entropy search for efficient bayesian optimization,” in International Conference on Machine Learning, PMLR, 2017, pp. 3627–3635

  19. [27]

    Evaluation of data enhanced model predictive control for a coupled tank system,

    D. Scheurenberg, S. Stemmler, and D. Abel, “Evaluation of data enhanced model predictive control for a coupled tank system,” in 2023 IEEE Conference on Control Technology and Applications (CCTA), 2023, pp. 79–84

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.