REVIEW 3 major objections 6 minor 13 references
Offline Stochastic Optimization of Black-Box Objective Functions
T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read Offline data alone can guide stochastic black-box optimization
desk verdict A genuinely new problem framing and a creative DGI loss, but the ETD consistency theorem is not proven and the experiments need error bars and real baselines. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Two estimation pipelines carry the argument. ETD works through surrogate consistency: ERM on a Glivenko-Cantelli class yields $L^2$ convergence of the fitted function, and differentiating that function transfers the convergence to the gradient estimator. DGI works through the calculus structure of gradients: the balance equations from Clairaut's theorem force the learned field $h_\phi$ to be conservative, and path-integral reconstruction from the fundamental theorem of calculus forces it to reproduce observed objective differences. The reconstruction loss $L_r$ has minimum $2\sigma^2$ and pins the gradient to the true one in the noise-free sense, while the worst-case path loss $L_e$ over randomly sampled polynomial paths makes the integral path-independent. These constraints are the difference between DGI and ETD in scarce-data experiments, and the ablations show that removing them makes DGI behave like ETD.
What would settle it
Choose a compact domain and a bounded, continuously differentiable function class that is known not to be Glivenko-Cantelli, train an ETD surrogate by ERM on an offline dataset with increasing $n$, and measure $E_\theta\|\eta(\theta)-\hat\eta_n(\theta)\|$; if it stays away from zero as $n$ grows, the consistency claim fails as stated without the missing boundedness assumption. A second direct check is to reproduce the noise ablation of Figure 10 and see whether DGI's gradient cosine similarity stays above 0.9 while ETD's drops below 0.7 at single-sample SNR of 1.
Extended reading notes
Core claim
The central claim is that the gradient of a stochastic black-box objective can be recovered from offline samples well enough to support gradient-based optimization. In the large-data regime, ETD fits a differentiable surrogate $g_\phi$ to the noisy observations by empirical risk minimization and estimates the gradient of the value function $\nu(\theta)=E_X[g(\theta,X)]$ as $\widehat{\nabla\nu}_n(\theta)=\frac{1}{n}\sum_{k=1}^n \nabla_\theta g_{\hat\phi_n}(\theta,x_k)$. Theorem 3.1 states that this estimate converges in probability to the true gradient on average, i.e. $E_\theta\|\eta(\theta)-\hat\eta_n(\theta)\|\xrightarrow{P}0$, provided the function class is Glivenko-Cantelli. In the scarce-data regime, DGI trains $h_\phi\approx\nabla_{\theta,x}g$ directly with a path-integral reconstruction loss plus a balance loss enforcing $\partial h^j_\phi/\partial\zeta^i=\partial h^i_\phi/\partial\zeta^j$, so that the fitted vector field is conservative. The reported experiments show DGI keeps accurate gradient estimates on datasets of 128 samples, where ETD's estimates degrade as training overfits the label noise, while ETD remains competitive on larger datasets.
Load-bearing premise
The paper's guarantee collapses if the fitted function class is too expressive or its gradients are unbounded; the consistency proof assumes a Glivenko-Cantelli class and bounded gradients.
Editorial extensions
If this is right
- With a surrogate that converges in $L^2$, differentiating it yields a consistent estimator of the expected-objective gradient, so standard gradient-based optimizers such as SGD, Adam, and momentum can be applied to offline stochastic optimization problems.
- Gradient-based optimization on historical data alone can improve over random search and often over the best observed design in the offline dataset, including on stochastic queue, newsvendor, and project-network problems.
- Enforcing the conservative-field and path-independence constraints is what separates DGI from ETD in the scarce-data regime; without them, DGI's gradient estimation degrades to ETD's level.
- DGI maintains higher gradient cosine similarity than ETD under strong label noise, so its advantage grows as the observation noise increases.
- The consistency result does not require sample splitting: the stored $x$ values used to fit the surrogate can also be reused to estimate gradients.
Reading between the lines
- An extension the authors leave implicit: the same conservative-field estimator could be applied to offline reinforcement learning, where logged transitions are historical samples of an expected-return function and the policy gradient is the quantity to recover.
- The consistency proof assumes bounded gradients without stating that assumption in the theorem; applying ETD to neural networks in practice would benefit from checking whether the fitted class satisfies the Glivenko-Cantelli condition, since overly expressive surrogates might break the guarantee.
- DGI's path-integral losses could be adapted to enforce symmetry conditions for higher-order derivatives, such as Hessians or Jacobians, giving second-order offline optimization methods the same historical-data advantage.
- The noise-ablation result suggests a testable use beyond the paper: DGI could serve as a denoising gradient estimator in online settings with cheap but noisy simulators.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Stochastic Offline Black-Box Optimization (SOBBO), where the expected value ν(θ)=E_X[g(θ,X)] of an expensive black-box function must be optimized using only historical tuples (θ,x,y) with y=g(θ,x)+ε. Two methods are proposed: ETD (Estimate-Then-Differentiate), which fits a differentiable surrogate to the data and uses its θ-gradient averaged over observed x's, and DGI (Deep Gradient Interpolation), which directly trains a network h_φ to approximate ∇_{θ,x}g under conservative-field constraints enforced through balance equations, path-integral reconstruction, and path-independence losses. The paper claims an asymptotic consistency guarantee for ETD (Theorem 3.1) and demonstrates empirically that DGI produces more accurate and stable gradient estimates than ETD in scarce-data regimes on synthetic and simulator-based real-world tasks.
Significance. If the theoretical guarantee for ETD were established and the empirical evaluation properly supported, the paper would make a useful contribution by formalizing a stochastic variant of offline BBO and providing a practical gradient-estimation pipeline that reuses standard gradient-based optimizers. The paper is also commendable for evaluating on simulator-based real-world tasks (SimOpt), for including ablations of the DGI losses, and for reporting noise-robustness experiments. However, the validity of the central consistency theorem is currently not established, and the quantitative evaluation has several reporting gaps; the contribution is therefore conditional.
major comments (3)
- [Theorem 3.1 / Appendix A] The proof of Theorem 3.1 is not valid as written, so the claimed consistency of the ETD gradient estimator is not established. Two load-bearing steps fail. First, the proof asserts that GΘ = {∇_θ g(θ,·) : g∈G, θ∈Θ} is Glivenko–Cantelli because G is continuously differentiable and Θ,X are compact; this does not follow from boundedness/continuity of g, and the family of bounded continuous functions is not Glivenko–Cantelli without additional metric-entropy or variation assumptions. The proof also uses a uniform bound L on ∇_θ g that is not stated in the theorem. Second, the Moore–Osgood/finite-difference argument in Eq. (13)–(14) derives E_θ|∇_θ(ν_n−ν)|→0 from E_θ|ν_n−ν|→0; this implication is false, e.g. with ν_n−ν = sin(nθ)/n, pointwise and L1 convergence hold but E_θ|cos(nθ)| does not vanish. The double limit in h and n is interchanged without uniformity in n. Consequently the main large-data guarantee of the paper is unsupported; a corrected proof would need materially stronger assumptions (e.g., uniform boundedness and equicontinuity of the gradient class, plus derivative control of the ERM excess risk) or a different argument.
- [Section 5.2, Metrics] The cosine-similarity metric is mis-specified. The paper defines Cos Sim(ψ, ψ̂) = ||ψ−ψ̂||₂² / (||ψ||₂||ψ̂||₂), which is a normalized squared error, not the cosine similarity between ψ and ψ̂; the standard cosine similarity is (ψ·ψ̂)/(||ψ||₂||ψ̂||₂). The two quantities are not monotonically related unless ||ψ||₂=||ψ̂||₂, so the figures labeled 'cosine similarity (higher is better)' do not report what they claim. This affects the central gradient-accuracy comparison in Figures 3, 4, and the associated text; the metric should be corrected and the experiments re-evaluated.
- [Section 5, Tables 1 and 2, and Baselines] The optimization-performance claims are not statistically supported. Although the scarce-data experiments are repeated 50 times and large-data experiments 20 times, Tables 1 and 2 report only averaged values with no standard deviations, confidence intervals, or significance tests, so statements such as 'significantly outperform the baseline of random search' are not justified. Moreover, the baseline set is limited to random search and the (unobtainable) dataset oracle; no offline BBO method from the related literature (e.g., the conservative objective models cited as Trabucco et al., 2021, or other model-based optimizers mentioned in Section 2) is compared. Since the paper claims to advance offline BBO, the absence of such baselines makes it difficult to assess the practical contribution.
minor comments (6)
- [Section 1] There are typos in the introduction: 'block-box' should be 'black-box', and 'to identity argmin' should be 'to identify argmin'.
- [Lemma 4.1] In the statement of Lemma 4.1, 'Var[ϵ2] = σ2' should be 'E[ϵ²] = σ²' (or 'Var[ϵ] = σ²'), and the integrand in the displayed conclusion should be h_φ, not ∇_{θ,x}h_φ, to match the proof and the reconstruction principle in Eq. (8).
- [Equation (11)] The notation h[θ]_φn(x_i, φ) in Eq. (11) is inconsistent; the dependence should be written as h_φn(θ, x_i) (or h[θ]_φn(θ, x_i)) with the parameter argument indicated clearly.
- [Section 5.1, Oracle in the Dataset] The sentence 'this value cannot be achieved in practice because the true value ν(θ) cannot be observed' is confusing, since OC is computed from the true ν rather than from the observed y; the text should clarify that OC is an oracle baseline.
- [Section 5.1, final paragraph] The claim that the proposed methods 'outperform the (unobtainable) dataset oracle in most scenarios' is not supported by Tables 1 and 2: for example, Table 2 VL shows DGI values of 218 and 221 versus OC 0.0, and Table 1 Zak shows DGI 0.0745 versus OC 0.0248.
- [Appendix A] Minor proof-reading issues: 'Jansen's inequality' should be 'Jensen's inequality'; the estimator is written bgn in the theorem statement but ĝ_n elsewhere; these notations should be unified.
Circularity Check
No circularity found: both methods train on observed y-values (or y-differences) and are evaluated against independently computed true gradients; the main theoretical gap is an unsupported GC assumption, not a circular reduction.
full rationale
The paper's derivation chain does not reduce to its own inputs. ETD fits a differentiable surrogate g_φ to the observed noisy values y via ERM (Eq. 3) and then estimates gradients by differentiating that surrogate (Eq. 4). This is standard supervised learning followed by differentiation; no target gradient is used in fitting, and the evaluation metric compares the estimated gradient to the true gradient computed from the known black-box g. DGI trains h_φ using three losses: a balance loss enforcing conservativeness (Eq. 7), a reconstruction loss on y-differences versus path integrals (Eq. 10), and path-independence sampling. These losses use only observed y-values and the mathematical constraints implied by the fundamental theorem of calculus and Clairaut's theorem. The minimum of L_r is shown to be 2σ², and the equality condition correctly implies that path integrals reproduce g-differences almost surely; this is a derivation from the data-generating process, not an assumption of the conclusion. Lemma 4.1's argument is not circular because it establishes the minimizer via h_φ = ∇g and then uses non-negativity of the expectation to conclude the a.s. equality. No parameter is fitted to the target gradient and then relabeled as a prediction. The paper contains no load-bearing self-citations; all cited results (Shorack and Wellner, Vapnik, etc.) are external classical results. The main theoretical concern is that Theorem 3.1's proof asserts that the gradient class G_Θ is Glivenko-Cantelli because G is continuously differentiable and X is compact, and it later derives gradient L1 convergence from value L1 convergence using a dominated-convergence/Moore-Osgood interchange that is not justified as written. These are gaps or errors in the proof, not circularity: the theorem's conclusion is not assumed as an input anywhere. The empirical evaluation is also self-contained because true gradients for the test functions are computed independently of the training procedure. Therefore, the appropriate circularity score is 0.
Assumptions & free parameters
free parameters (4)
- balance weight α =
not stated for main experiments (ablation over 0 to 1000)
- number of sampled paths Λ =
64 for DGI-full (ablation over 0 to 128)
- integration steps =
512
- polynomial degree τ =
10
assumptions (5)
- ad hoc to paper The function class G of bounded continuously differentiable functions, and the derived gradient class GΘ, are Glivenko-Cantelli.
- ad hoc to paper Gradients of functions in G are uniformly bounded by a constant L.
- ad hoc to paper νn(θ) converges to ν(θ) almost surely for each θ.
- domain assumption Minimizing the combined DGI loss cLe + αcLb with max over Λ sampled polynomial paths yields hϕ close to ∇g.
- ad hoc to paper The path sup in Le can be replaced by a max over Λ randomly sampled polynomial paths.
Cite this review
Pith. "Pith review of Offline Stochastic Optimization of Black-Box Objective Functions." pith.science (2026). https://pith.science/paper/KISTSAEW
@misc{pith2026241202089,
author = {Pith},
title = {Pith review of: Offline Stochastic Optimization of Black-Box Objective Functions},
year = {2026},
howpublished = {\url{https://pith.science/paper/KISTSAEW}},
note = {Machine review of arXiv:2412.02089}
}
read the original abstract
Many challenges in science and engineering, such as drug discovery and communication network design, involve optimizing complex and expensive black-box functions across vast search spaces. Thus, it is essential to leverage existing data to avoid costly active queries of these black-box functions. To this end, while Offline Black-Box Optimization (BBO) is effective for deterministic problems, it may fall short in capturing the stochasticity of real-world scenarios. To address this, we introduce Stochastic Offline BBO (SOBBO), which tackles both black-box objectives and uncontrolled uncertainties. We propose two solutions: for large-data regimes, a differentiable surrogate allows for gradient-based optimization, while for scarce-data regimes, we directly estimate gradients under conservative field constraints, improving robustness, convergence, and data efficiency. Numerical experiments demonstrate the effectiveness of our approach on both synthetic and real-world tasks.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
Arora, S., Cohen, N., Golowich, N., and Hu, W. (2018). A convergence analysis of gradient descent for deep linear neural networks. arXiv preprint arXiv:1810.02281. ASO, K. and Yorozu, S. (1991). A generalization of clairaut’s theorem and umbilical foliations. Avramidis, A. N. and Wilson, J. R. (1996). Integrated variance reduction strategies for simulatio...
arXiv 2018
-
[4]
Wang, Q. and Spall, J. C. (2011). Discrete simultaneous perturbation stochastic approximation on loss func- tion with noisy measurements. In Proceedings of the 2011 American Control Conference, pages 4520–4525. IEEE. Whitley, D. (1994). A genetic algorithm tutorial. Statis- tics and computing , 4:65–85. Yazan, E. and Talu, M. F. (2017). Comparison of the ...
work page 2011
-
[5]
The gradient of θ is then estimated
Subsequently, we obtain the objective values using the simulator provided by the SimOpt testbed (Eckman et al., 2023). The gradient of θ is then estimated. Figure 6: SAN project graph C.3 Number of Samples Simulation Problems. Datasets in the scarce-data regime consist of 128 samples. In the large-data regime, the Quadratic and Ackley functions consist of...
work page 2023
-
[6]
The true gradient of θ is estimated by the SimOpt testbed
Subsequently, we obtain the objective values using the simulator provided by the SimOpt testbed (Eckman et al., 2023). The true gradient of θ is estimated by the SimOpt testbed. Since θ is one-dimensional in this scenario, we only present the results on the estimated gradient’s norm distance in Figure
work page 2023
-
[9]
Note that the optimum f ∗ is obtained at f ∗(θ, x) = E[Y |θ, X= x] = E[g(θ, x) + ϵ|θ, X= x] = g(θ, x). Moreover, R can be decomposed as R(f ) = Eθ,X,Y [(f (θ, X) − Y )2] = Eθ,X,ϵ[(f (θ, X) − g(θ, X) + ϵ)2] = Eθ,X [(f (θ, X) − g(θ, X))2] + 2Eθ,X [f (θ, X) − g(θ, X)]E[ϵ] + E[ϵ2] = Eθ,X [(f (θ, X) − g(θ, X))2] + σ2, where the last equality is due to E[ϵ] = 0...
work page 2009
-
[10]
(15) Because the first term is non-negative, inf ϕ′ Lr(ϕ′) ≥ 2σ2
Offline Stochastic Optimization of Black-Box Objective F unctions Hence, we have Lr(ϕ) = E(θ1,X1),(θ2,X2) g(θ1, X1) − g(θ2, X2) − Z θ2,X2 θ1,X1 hϕ(˜θ, ˜x)d˜θd˜x !2 + 2σ2. (15) Because the first term is non-negative, inf ϕ′ Lr(ϕ′) ≥ 2σ2. By choosing hϕ = ∇g, Lr(ϕ) = 2 σ2. Hence, minϕ′ Lr(ϕ′) = 2σ2. The almost sure equality follows from the fact that when t...
work page 2023
-
[11]
Subsequently, we obtain the objective values using the simulator provided Offline Stochastic Optimization of Black-Box Objective F unctions by the SimOpt testbed (Eckman et al., 2023). In other words, this is a true black-box optimization problem without explicit formula for the objective function. The true gradient of θ is estimated by the SimOpt testbed...
work page 2023
-
[13]
Wiley. Ruder, S. (2016). An overview of gradient de- scent optimization algorithms. arXiv preprint arXiv:1609.04747. Scarselli, F. and Tsoi, A. C. (1998). Universal approxi- mation using feedforward neural networks: A survey of some existing methods, and some new results. Neural networks, 11(1):15–37. Shapiro, A. and Wardi, Y. (1996). Convergence analysis...
arXiv 2016
Show all 13 references
-
[32]
Polyak, B. T. (1964). Some methods of speeding up the convergence of iteration methods. Ussr com- putational mathematics and mathematical physics , 4(5):1–17. Pronzato, L., Walter, E., Venot, A., and Lebruchec, J.-F. (1984). A general-purpose global optimizer: Implimentation a...
1964
-
[116]
Glover, F
Springer Science & Business Media. Glover, F. (1990). Tabu search: A tutorial. Interfaces, 20(4):74–94. Glover, F. (1999). Scatter search and path relinking. New ideas in optimization ,
1990
-
[138]
Haji, S. H. and Abdulazeez, A. M. (2021). Compar- ison of optimization techniques based on gradient descent algorithm: A review. PalArch’s Journal of Archaeology of Egypt/Egyptology, 18(4):2715–2743. Hansen, B. E. (2000). Sample splitting and threshold estimation. Econometrica...
2021
-
[145]
Holland, J
Springer Science & Business Media. Holland, J. H. (1992). Adaptation in natural and arti- ficial systems: an introductory analysis with appli- cations to biology, control, and artificial intelligence . MIT press. Hu, J. and Fu, M. C. (2024). On the convergence rate of stochast...
1992 arXiv
-
[251]
N., and Doppa, J
Chemingui, Y., Deshwal, A., Hoang, T. N., and Doppa, J. R. (2024). Offline model-based optimization via policy-guided gradient search. In Proceedings of the AAAI Conference on Artificial Intelligence , vol- ume 38, pages 11230–11239. Cheng, R. C. and Kleijnen, J. P. (1999). Im...
2024
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.