Pith. sign in

REVIEW 3 minor 31 references

Gray-Box Optimization using Optimism in the Face of Uncertainty

T0 review · 0 major / 3 minor · reviewed 2026-06-26 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A structure-exploiting method for gray-box optimization minimizes a lower confidence bound on the objective to achieve improved regret bounds.

desk verdict Gray-box OFU exploits known loss structure plus a recent multi-output bound to tighten regret over plain linear bandits. read the letter →

arxiv 2606.17726 v1 pith:LOE3TOPN submitted 2026-06-16 math.OC

Katrin Baumgärtner , Léo Simpson , Moritz Diehl This is my paper · ORCID
classification math.OC
keywords gray-boxoptimizationoptimisminthefaceofuncertaintyregretanalysisstochasticlinearbanditslowerconfidenceboundparameterestimationsequentialdecisionmaking
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to establish that known structure in the form of a loss function and an admissible parameter set can be used to build a lower confidence bound whose minimization yields a sequential decision rule with stronger theoretical guarantees. This setup covers problems where an unknown parametric model is observed through noise and the objective is the composition of that model with a fixed loss. A reader would care because the resulting method generalizes the linear stochastic bandit setting and supplies a regret analysis that tightens existing bounds for that special case.

What carries the argument

The structure-exploiting lower confidence bound minimization that constructs an optimistic surrogate from the known loss and admissible parameter set.

What would settle it

A linear stochastic bandit instance in which the proposed method's cumulative regret fails to improve on existing bounds or in which the new parameter confidence sets do not produce measurably tighter intervals than prior constructions.

Watch

Extended reading notes

Core claim

The paper introduces a method for sequential gray-box optimization that uses optimism in the face of uncertainty by minimizing a lower confidence bound on the true objective, constructed using the known loss function and an a priori set of admissible parameters. It provides a detailed regret analysis that improves state-of-the-art results for linear stochastic bandits through the use of a recent bound on parameter confidence sets from multi-output linear least-squares estimation, and demonstrates superior performance in numerical examples.

Load-bearing premise

The loss function and an a priori set of admissible parameters are known in advance.

Editorial extensions

If this is right

  • The method applies directly to the general gray-box setting and recovers the contextual linear bandit problem as a special case.
  • Regret bounds improve on prior work for linear stochastic bandits because of tighter confidence sets for multi-output least-squares estimates.
  • Numerical comparisons show lower regret than methods that treat the problem as a black-box without using the known loss or parameter set.
  • The approach trades exploration against exploitation by repeatedly minimizing the lower confidence bound on the composed objective.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the admissible parameter set can be tightened from data over time, the method might adaptively reduce exploration without changing the algorithm.
  • The same construction could be tested on control problems where the stage cost is known but the dynamics parameters are learned online.
  • Extending the regret analysis to nonlinear losses or non-Euclidean parameter sets would require only the corresponding confidence-set bound.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

0 major / 3 minor

Summary. The paper considers sequential gray-box optimization where the objective is the composition of a known loss function and an unknown parametric model estimated from noisy observations. This generalizes contextual stochastic linear bandits. The authors propose an OFU method that minimizes a lower confidence bound on the objective, exploiting the known loss and admissible parameter set. They provide a regret analysis improving on linear-bandit SOTA via a recent multi-output least-squares confidence-set bound, with numerical examples showing gains over structure-agnostic baselines.

Significance. If the regret analysis holds, the work advances gray-box optimization by showing how known structure can be exploited for tighter bounds and better empirical performance. The explicit attribution to an external confidence-set bound avoids circularity and is a methodological strength; the reduction to the linear case is cleanly presented.

minor comments (3)
  1. [Abstract] The abstract claims an improvement on SOTA for linear bandits but does not quantify the improvement or name the specific prior bounds being superseded; adding this would clarify the contribution.
  2. [Regret analysis section] The noise model assumptions (e.g., sub-Gaussianity, independence) should be stated explicitly when invoking the external multi-output least-squares bound to confirm applicability without additional restrictions.
  3. [Numerical examples] Numerical examples are referenced but lack reported metrics, baseline details, or statistical significance; including a table of regret values or performance gaps would strengthen the empirical claims.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their positive summary of the manuscript, recognition of its methodological contributions, and recommendation for minor revision. No specific major comments were listed in the report.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity detected

full rationale

The paper's regret analysis for the gray-box OFU method explicitly attributes its improvement on linear stochastic bandit SOTA to a recently published external bound on multi-output linear least-squares confidence sets. The problem setup states the loss function and admissible parameter set as known inputs used to construct the lower confidence bound, with the derivation reducing to standard OFU principles plus this external reference rather than any self-defined quantity, fitted parameter renamed as prediction, or self-citation chain. No load-bearing step reduces by construction to the paper's own equations or prior self-citations.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

The abstract invokes standard domain assumptions from the stochastic bandit literature but introduces no new free parameters, ad-hoc axioms, or invented entities.

assumptions (1)
  • domain assumption Parameters of the model can be estimated from noisy observations via multi-output linear least-squares, yielding usable confidence sets.
    This assumption underpins the construction of the lower confidence bound and the regret analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Gray-Box Optimization using Optimism in the Face of Uncertainty." pith.science (2026). https://pith.science/paper/LOE3TOPN

@misc{pith2026260617726,
  author       = {Pith},
  title        = {Pith review of: Gray-Box Optimization using Optimism in the Face of Uncertainty},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LOE3TOPN}},
  note         = {Machine review of arXiv:2606.17726}
}
read the original abstract

This paper considers sequential gray-box optimization where the objective function is given as the composition of a loss function and a parametric model. Crucially, the parameters of the model are unknown and need to be iteratively estimated from noisy observations of the model outputs. This problem setup generalizes the parametric black-box optimization problem known as (contextual) stochastic linear bandit. To address the sequential gray-box optimization problem, we propose a structure-exploiting method that leverages known problem structure given in terms of the loss function and an a priori set of admissible parameters. The method is based on the principle of optimism in the face of uncertainty and trades off exploration and exploitation by minimizing a lower confidence bound on the true objective function. We provide a detailed regret analysis of the novel method, improving on state-of-the-art results for the special case of linear stochastic bandits due to the use of a recently published bound for the parameter confidence sets arising in multi-output linear least-squares estimation. Numerical examples illustrate the superior performance of structure-exploiting methods compared to structure-agnostic approaches.

Figures

Figures reproduced from arXiv: 2606.17726 by the authors.

Figure 1
Figure 1. Black-box vs. gray-box modelling: We consider a scalar input-output model z = a0 + PK i=1 (ai cos(iu) + bi sin(iu)), where θ = (a0, . . . , aK, b1, . . . , bK), which corresponds to a Fourier series of order K = 10, and a loss l(u, z) = 0.01u 2 + z 2 . The a priori estimate of the model and the corresponding confidence set (which is discussed in more detail in Section 3) are shown in the top left plot. The bottom le… view at source ↗
Figure 2
Figure 2. Linear bandit with additional structural knowledge of the true parameter: Con￾fidence ellipsoid as well as the parameter values θn picked by the optimizer at iteration n. The shaded area indicates the infeasible region with respect to the constraint θ ∈ Θ. Consider f(u, θ) = u ⊤θ, U = [0, 1]2 and l(u, z) = z. The measurement noise is normally distributed with variance σ 2 v = 0.04. The estimator uses µ0 = (0, 0), Λ0… view at source ↗
Figure 3
Figure 3. Linear bandit with additional structural knowledge of the true parameter: Com￾parison of the cumulative regret obtained over 300 independent simulations where the true parameter is uniformly sampled from Θ. The whiskers indicate the min￾imum and maximum values, and the triangle indicates the mean. 7.2 Exploiting structure given by the loss With the second example, we illustrate the performance gain obtained from exp… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Comparison of the structure-agnostic and the structure-exploiting approach for Example 2. The top row shows the prior confidence set for the objective, while the middle row shows the confidence set after evaluating at u1 = −1 and observing the model output y1 = (1., 0.…
Figure 5
Figure 5. Figure 5: Cost incurred by the proposed optimistic method in comparison to the nominal and explicit dual approach for the steel recycling example. On the one hand, dim(θ) > dim(θBB), i.e., the gray-box approach needs to identify more unknown parameters. On the other hand, the gr…
Figure 6
Figure 6. Figure 6: Concentration estimates together with the confidence bounds for the steel recy￾cling example. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 5 canonical work pages

  1. [1]

    Improved algorithms for linear stochastic bandits

    Yasin Abbasi-Yadkori, D \'a vid P \'a l, and Csaba Szepesv \'a ri. Improved algorithms for linear stochastic bandits. Advances in neural information processing systems, 24, 2011

    2011

  2. [2]

    Associative reinforcement learning using linear probabilistic concepts

    Naoki Abe and Philip M Long. Associative reinforcement learning using linear probabilistic concepts. In ICML, pages 3--11. Citeseer, 1999

    1999

  3. [3]

    Bayesian optimization of composite functions

    Raul Astudillo and Peter Frazier. Bayesian optimization of composite functions. In International Conference on Machine Learning, pages 354--363. PMLR, 2019

    2019

  4. [4]

    Thinking inside the box: A tutorial on grey-box bayesian optimization

    Raul Astudillo and Peter I Frazier. Thinking inside the box: A tutorial on grey-box bayesian optimization. In 2021 Winter Simulation Conference (WSC), pages 1--15. IEEE, 2021

    2021

  5. [5]

    Using confidence bounds for exploitation-exploration trade-offs

    Peter Auer. Using confidence bounds for exploitation-exploration trade-offs. Journal of Machine Learning Research, 3 0 (Nov): 0 397--422, 2002

    2002

  6. [6]

    Botorch: A framework for efficient monte-carlo bayesian optimization

    Maximilian Balandat, Brian Karrer, Daniel Jiang, Samuel Daulton, Ben Letham, Andrew G Wilson, and Eytan Bakshy. Botorch: A framework for efficient monte-carlo bayesian optimization. Advances in neural information processing systems, 33: 0 21524--21538, 2020

    2020

  7. [7]

    R. Bellman. Dynamic programming . Princeton University Press, 1957

    1957

  8. [8]

    Safe controller optimization for quadrotors with gaussian processes

    Felix Berkenkamp, Angela P Schoellig, and Andreas Krause. Safe controller optimization for quadrotors with gaussian processes. In 2016 IEEE I nternational C onference on R obotics and A utomation (ICRA) , pages 491--496. IEEE, 2016

    2016

Show all 31 references
  1. [9]

    Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics

    Felix Berkenkamp, Andreas Krause, and Angela P Schoellig. Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics. Machine Learning, 112 0 (10): 0 3713--3747, 2023

    2023

  2. [10]

    Bristow, M

    D.A. Bristow, M. Tharayil, and A.G. Alleyne. A survey of iterative learning control. IEEE Control Systems, 26 0 (3): 0 96--114, June 2006. ISSN 1066-033X. doi:10.1109/MCS.2006.1636313

    2006 doi

  3. [11]

    A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning

    Eric Brochu, Vlad M Cora, and Nando De Freitas. A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599, 2010

    2010 arXiv

  4. [12]

    Regret analysis of stochastic and nonstochastic multi-armed bandit problems

    S \'e bastien Bubeck and Nicolo Cesa-Bianchi. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. arXiv preprint arXiv:1204.5721, 2012

    2012 arXiv

  5. [13]

    No-regret algorithms for multi-task bayesian optimization

    Sayak Ray Chowdhury and Aditya Gopalan. No-regret algorithms for multi-task bayesian optimization. In International Conference on Artificial Intelligence and Statistics, pages 1873--1881. PMLR, 2021

    2021

  6. [14]

    Stochastic linear optimization under bandit feedback

    Varsha Dani, Thomas P Hayes, and Sham M Kakade. Stochastic linear optimization under bandit feedback. In COLT, volume 2, page 3, 2008

    2008

  7. [15]

    BOHB : Robust and efficient hyperparameter optimization at scale

    Stefan Falkner, Aaron Klein, and Frank Hutter. BOHB : Robust and efficient hyperparameter optimization at scale. In International conference on machine learning, pages 1437--1446. PMLR, 2018

    2018

  8. [16]

    A. A. Feldbaum. Dual control theory i. Avtomat. i Telemekh., 21 0 (9): 0 1240--1249, 1960

    1960

  9. [17]

    Filatov and Heinz Unbehauen

    Nikolai M. Filatov and Heinz Unbehauen. Survey of adaptive dual control methods. IEE Proceedings - Control Theory and Applications, 147 0 (1), 2000

    2000

  10. [18]

    Parametric bandits: The generalized linear case

    Sarah Filippi, Olivier Cappe, Aur \'e lien Garivier, and Csaba Szepesv \'a ri. Parametric bandits: The generalized linear case. Advances in neural information processing systems, 23, 2010

    2010

  11. [19]

    An implicit and explicit dual model predictive control formulation for a steel recycling process

    Andrea Ghezzi, Florian Messerer, Jacopo Balocco, Vincenzo Manzoni, and Moritz Diehl. An implicit and explicit dual model predictive control formulation for a steel recycling process. European Journal of Control, 74: 0 100841, 2023

    2023

  12. [20]

    Efficient performance-based MPC tuning in high dimensions using bayesian optimization over sparse subspaces

    Akshay Kudva, Melanie T Huynh, Ali Mesbah, and Joel A Paulson. Efficient performance-based MPC tuning in high dimensions using bayesian optimization over sparse subspaces. IFAC-PapersOnLine, 58 0 (14): 0 458--463, 2024

    2024

  13. [21]

    Bandit algorithms

    Tor Lattimore and Csaba Szepesv \'a ri. Bandit algorithms. Cambridge University Press, 2020

    2020

  14. [22]

    A contextual-bandit approach to personalized news article recommendation

    Lihong Li, Wei Chu, John Langford, and Robert E Schapire. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web, pages 661--670, 2010

    2010

  15. [23]

    Modifier-adaptation methodology for real-time optimization

    Alejandro Marchetti, Benoit Chachuat, and Dominique Bonvin. Modifier-adaptation methodology for real-time optimization. Industrial & engineering chemistry research, 48 0 (13): 0 6022--6033, 2009

    2009

  16. [24]

    The application of B ayesian methods for seeking the extremum

    Jonas Mockus, Vytautas Tiesis, and Antanas Zilinskas. The application of B ayesian methods for seeking the extremum. Towards Global Optimization, 2 0 (117-129): 0 2, 1978

    1978

  17. [25]

    Generalized linear models

    John Ashworth Nelder and Robert WM Wedderburn. Generalized linear models. Journal of the Royal Statistical Society Series A: Statistics in Society, 135 0 (3): 0 370--384, 1972

    1972

  18. [26]

    Cobalt: Constrained bayesian optimization of computationally expensive grey-box models exploiting derivative information

    Joel A Paulson and Congwen Lu. Cobalt: Constrained bayesian optimization of computationally expensive grey-box models exploiting derivative information. Computers & Chemical Engineering, 160: 0 107700, 2022

    2022

  19. [27]

    Linearly parameterized bandits

    Paat Rusmevichientong and John N Tsitsiklis. Linearly parameterized bandits. Mathematics of Operations Research, 35 0 (2): 0 395--411, 2010

    2010

  20. [28]

    Schoellig, Fabian L

    Angela P. Schoellig, Fabian L. Mueller, and Raffaello D'Andrea. Optimization-based iterative learning for precise quadrocopter trajectory tracking. Autonomous Robots, 33 0 (1-2): 0 103--127, 2012. ISSN 0929-5593

    2012

  21. [29]

    Finite-sample bounds for multi-output system identification

    Léo Simpson, Katrin Baumgärtner, Johannes Köhler, and Moritz Diehl. Finite-sample bounds for multi-output system identification. arXiv preprint 2603.19073, 2026

    2026

  22. [30]

    Volckaert, J

    M. Volckaert, J. Swevers, and M. Diehl. A two step optimization based iterative learning control algorithm. In Dynamic Systems and Control Conference, Cambridge, Massachusetts, September 2010

    2010

  23. [31]

    Bayesian optimization with structured measurements: A vector-valued rkhs framework

    Wenbin Wang and Colin N Jones. Bayesian optimization with structured measurements: A vector-valued rkhs framework. arXiv preprint 2605.09775, 2026

    2026 arXiv

Pith tools

Reviewed June 26, 2026 · model on record in the stance chip above.