pith. sign in

arxiv: 2512.07868 · v2 · submitted 2025-11-26 · 💻 cs.LG · cs.AI· stat.ML

Bayesian Optimization for Function-Valued Responses under Min-Max Criteria

Pouya Ahadi , Reza Marzban , Ali Adibi , Kamran Paynabar This is my paper

Pith reviewed 2026-05-17 04:08 UTC · model grok-4.3

classification 💻 cs.LG cs.AIstat.ML
keywords Bayesian optimizationfunctional data analysismin-max optimizationGaussian processesfunctional principal component analysisacquisition functionsblack-box optimization
0
0 comments X

The pith

A Bayesian optimization method minimizes the largest error anywhere in a functional response instead of the average error.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes MM-FBO, a framework that treats the response as a function over an index such as time or wavelength and directly targets the point of greatest deviation from the target. Functional responses are first reduced via functional principal component analysis to a handful of scores, each modeled by its own Gaussian process surrogate. An acquisition function then integrates uncertainty to drive selection of the next input point toward lowering the current worst-case expected error. Theoretical results bound the discretized min-max objective and show that the acquisition converges to the true optimum once the surrogate is accurate. Experiments on synthetic functions and two physics applications confirm lower peak errors than methods that optimize integrated error.

Core claim

MM-FBO represents functional responses using functional principal component analysis and builds Gaussian process surrogates on the resulting scores. It introduces an integrated uncertainty acquisition function that minimizes the maximum expected error across the functional domain, with a discretization bound for the worst-case objective and a consistency result that the acquisition converges to the true min-max objective as surrogate accuracy increases and uncertainty vanishes.

What carries the argument

The integrated uncertainty acquisition function that balances exploitation of the current worst-case expected error with exploration of functional-domain uncertainty derived from the FPCA score models.

If this is right

  • In electromagnetic scattering design the method avoids large deviations at any single wavelength even when average performance looks acceptable.
  • For vapor phase infiltration processes the optimizer focuses evaluations on inputs that reduce the highest local infiltration error.
  • As the number of evaluations grows the Gaussian process models on the principal component scores tighten, driving the acquisition closer to the true min-max optimum.
  • Standard integrated-error Bayesian optimization can converge to points that leave unacceptable peak errors in some regions of the response domain.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same min-max framing could be applied to spatial field responses in fluid or heat transfer simulations where local hotspots matter more than the spatial average.
  • It may be worth testing whether replacing FPCA with other basis expansions alters the worst-case performance on responses that contain sharp localized features.
  • The consistency guarantee suggests the framework could serve as a template for adding worst-case criteria to other surrogate-based methods that currently optimize averages.

Load-bearing premise

A low-dimensional set of functional principal components can capture the full range of worst-case deviations in the response without omitting critical local peaks.

What would settle it

An experiment on a synthetic functional benchmark where the input that minimizes maximum error differs from the input that minimizes average error, with direct measurement of whether MM-FBO returns a smaller true maximum error than integrated-error baselines.

Figures

Figures reproduced from arXiv: 2512.07868 by Ali Adibi, Kamran Paynabar, Pouya Ahadi, Reza Marzban.

Figure 1
Figure 1. Figure 1: Representative trajectories drawn from each simulator. Each panel shows multiple [PITH_FULL_IMAGE:figures/full_fig_p013_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Regret versus iteration with median and interquartile bands. [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Normalized regret trajectories showing convergence speed and stability. [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Box plots of final regret distributions across replications. [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: AUOC distributions summarizing full regret trajectories. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Schematic of a metasurface generated through a basis-driven geometric param [PITH_FULL_IMAGE:figures/full_fig_p016_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Electromagnetic scattering results with budget [PITH_FULL_IMAGE:figures/full_fig_p017_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Electromagnetic scattering results with budget [PITH_FULL_IMAGE:figures/full_fig_p018_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: VPI results. Our method achieves markedly smaller final regret and AUOC, with [PITH_FULL_IMAGE:figures/full_fig_p019_9.png] view at source ↗
read the original abstract

Bayesian optimization is widely used for optimizing expensive black box functions, but most existing approaches focus on scalar responses. In many scientific and engineering settings the response is functional, varying smoothly over an index such as time or wavelength, which makes classical formulations inadequate. Existing methods often minimize integrated error, which captures average performance but neglects worst case deviations. To address this limitation we propose min-max Functional Bayesian Optimization (MM-FBO), a framework that directly minimizes the maximum error across the functional domain. Functional responses are represented using functional principal component analysis, and Gaussian process surrogates are constructed for the principal component scores. Building on this representation, MM-FBO introduces an integrated uncertainty acquisition function that balances exploitation of worst case expected error with exploration across the functional domain. We provide two theoretical guarantees: a discretization bound for the worst case objective, and a consistency result showing that as the surrogate becomes accurate and uncertainty vanishes, the acquisition converges to the true min-max objective. We validate the method through experiments on synthetic benchmarks and physics inspired case studies involving electromagnetic scattering by metaphotonic devices and vapor phase infiltration. Results show that MM-FBO consistently outperforms existing baselines and highlights the importance of explicitly modeling functional uncertainty in Bayesian optimization.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes min-max Functional Bayesian Optimization (MM-FBO) for black-box optimization with functional responses. Functional outputs are represented via functional principal component analysis (FPCA), Gaussian process surrogates are fit to the resulting scores, and an integrated uncertainty acquisition function is introduced to directly target the maximum error across the functional domain rather than integrated error. Two theoretical results are stated: a discretization bound on the worst-case objective and a consistency guarantee that the acquisition converges to the true min-max objective as surrogate accuracy increases and uncertainty vanishes. The approach is evaluated on synthetic benchmarks and physics-inspired case studies in electromagnetic scattering by metaphotonic devices and vapor phase infiltration, where it outperforms existing baselines.

Significance. If the discretization bound and consistency result hold without hidden circularity and the experimental outperformance is reproducible, the work addresses a genuine gap in functional Bayesian optimization by prioritizing worst-case rather than average performance. The combination of FPCA representation with an acquisition that explicitly integrates uncertainty across the domain is a constructive contribution, and the physics-inspired experiments provide relevant validation. The theoretical guarantees, even if standard in parts, add value when tied to the min-max criterion.

major comments (2)
  1. [Theoretical guarantees (discretization bound and consistency result)] The consistency result (stated in the abstract and presumably proved in the theoretical section) frames convergence to the true min-max objective as surrogate accuracy increases. However, this relies on the FPCA representation faithfully preserving the sup-norm deviations that define the min-max criterion. Standard FPCA controls L2 error but does not automatically bound pointwise supremum deviations; without an explicit sup-norm control or remainder term bound on the truncation error, the discretization bound and consistency claim are not fully supported for the stated objective.
  2. [Functional response representation via FPCA] The central modeling step assumes that a low-dimensional FPCA basis suffices to capture functional responses for the min-max acquisition. If retained components miss localized peaks or high-frequency structure, the surrogate and integrated uncertainty acquisition will systematically underestimate worst-case error. The manuscript should either derive a sup-norm error bound relative to the min-max objective or provide empirical diagnostics showing that truncation bias is negligible compared with the GP uncertainty term.
minor comments (2)
  1. [Abstract] The abstract introduces the 'integrated uncertainty acquisition function' without a one-sentence description of its functional form or how it trades off worst-case expected error against domain-wide exploration; adding this would improve readability.
  2. [Experiments] In the experimental section, reporting the number of independent runs, statistical significance tests, and error bars on the performance plots would make the outperformance claims easier to assess.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed comments on the theoretical foundations of MM-FBO. The observations regarding sup-norm control in the FPCA representation and consistency guarantees are well-taken and help clarify the scope of our results. We respond to each major comment below and indicate the revisions we will make.

read point-by-point responses

  1. Referee: The consistency result (stated in the abstract and presumably proved in the theoretical section) frames convergence to the true min-max objective as surrogate accuracy increases. However, this relies on the FPCA representation faithfully preserving the sup-norm deviations that define the min-max criterion. Standard FPCA controls L2 error but does not automatically bound pointwise supremum deviations; without an explicit sup-norm control or remainder term bound on the truncation error, the discretization bound and consistency claim are not fully supported for the stated objective.

    Authors: We agree that the distinction between L2 and sup-norm is important for the min-max criterion. The discretization bound and consistency result are formally stated for the finite-dimensional objective obtained after FPCA truncation; the acquisition function is defined on the scores and converges to the min-max value of the truncated response as GP uncertainty vanishes. To make the connection to the original functional objective explicit, we will revise the theoretical section to include a short discussion of uniform-norm truncation error under standard smoothness assumptions on the response functions (e.g., Hölder or Sobolev regularity), citing known FPCA convergence rates in the uniform norm. This addition clarifies the scope without changing the stated theorems. revision: partial

  2. Referee: The central modeling step assumes that a low-dimensional FPCA basis suffices to capture functional responses for the min-max acquisition. If retained components miss localized peaks or high-frequency structure, the surrogate and integrated uncertainty acquisition will systematically underestimate worst-case error. The manuscript should either derive a sup-norm error bound relative to the min-max objective or provide empirical diagnostics showing that truncation bias is negligible compared with the GP uncertainty term.

    Authors: This concern is valid for responses with sharp localized features. In the reported experiments the functional responses are smooth (electromagnetic scattering and infiltration profiles), and we retain components explaining at least 95% of variance. We will add empirical diagnostics to the revised experimental section: sup-norm reconstruction error plots on held-out trajectories and a direct comparison of truncation bias versus GP predictive standard deviation at the points where the acquisition evaluates worst-case error. These diagnostics will demonstrate that, for the problem classes considered, truncation bias remains smaller than the uncertainty term driving exploration. revision: yes

Circularity Check

0 steps flagged

No circularity: derivation relies on standard GP consistency and FPCA representation without self-referential reduction

full rationale

The paper defines MM-FBO via FPCA truncation of functional responses followed by independent GP surrogates on the scores, then constructs an integrated uncertainty acquisition for the min-max objective. The stated discretization bound and consistency result are presented as limits under vanishing surrogate error and uncertainty, drawing on generic GP convergence properties rather than equating any output to a fitted input by construction. No self-citation chain, ansatz smuggling, or renaming of known results is load-bearing for the central claims; the framework retains independent content in its choice of min-max criterion and acquisition design.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The method rests on standard assumptions of functional data analysis and Gaussian processes without introducing new physical entities or heavily fitted parameters beyond the framework itself.

axioms (1)
  • domain assumption Functional responses vary smoothly and can be well-approximated by a finite set of principal components
    Invoked to reduce the functional output to scalar scores suitable for standard GP modeling.
invented entities (1)
  • Integrated uncertainty acquisition function no independent evidence
    purpose: Balances exploitation of worst-case expected error with exploration across the functional domain
    New component of the MM-FBO framework introduced to handle min-max criteria.

pith-pipeline@v0.9.0 · 5521 in / 1264 out tokens · 60216 ms · 2026-05-17T04:08:17.998347+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

3 extracted references · 3 canonical work pages

  1. [1]

    A gentle introduction to bayesian optimization

    Antonio Candelieri. A gentle introduction to bayesian optimization. InWinter Simulation Conference, 2021. Eleonora Cara, Irdi Murataj, Gianluca Milano, Natascia De Leo, Luca Boarino, and Federico Ferrarese Lupi. Recent advances in sequential infiltration synthesis applications using block copolymers.Nanomaterials, 11(4):994, 2021. Mingkun Chen, Robert Lup...

    work page 2021

  2. [2]

    Bayesian emulation of complex multi–output and dynamic computer models.Journal of Statistical Planning and Inference, 140(3):640–651, 2010

    Ahadi, Marzban, Adibi and Paynabar Stefano Conti and Anthony O’Hagan. Bayesian emulation of complex multi–output and dynamic computer models.Journal of Statistical Planning and Inference, 140(3):640–651, 2010. Chaofan Huang, Yi Ren, Emily K McGuinness, Mark D Losego, Ryan P Lively, and V Roshan Joseph. Bayesian optimization of functional output in inverse...

    work page arXiv 2010

  3. [3]

    24 Min-Max Bayesian Optimization Appendix A

    Ahadi, Marzban, Adibi and Paynabar design of photonic nanostructures: breaking the geometric complexity.Acs Photonics, 9 (2):714–721, 2022. 24 Min-Max Bayesian Optimization Appendix A. In this appendix, we prove Proposition 1: ProofFixθ∈Θ. For anyλ∈Λ choose an indexm(λ)∈ {1, . . . , T}with|λ−λ m(λ)| ≤h T . By the modulus of continuity, e(θ, λ)≤e θ, λm(λ) ...

    work page 2022