When can we improve on sample average approximation for stochastic optimization?
Pith reviewed 2026-05-24 19:20 UTC · model grok-4.3
The pith
Sample average approximation remains effective in stochastic optimization even when the true distribution is known, though Bayesian methods can improve on it in quadratic cases.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
In the quadratic test set the sample average approximation is remarkably effective and only consistently outperformed by a Bayesian approach; in the portfolio test set bagging, MLE and a Bayesian approach all do well.
What carries the argument
Direct performance comparison of sample average approximation to bagging, kernel smoothing, MLE, and Bayesian methods across two test problems: a quadratic objective with univariate decision variable, and five-stock portfolio optimization under different covariance structures.
If this is right
- Bayesian methods can deliver consistent gains over sample average approximation when the objective is quadratic and the decision variable is univariate.
- Bagging and maximum likelihood estimation become competitive with sample average approximation in portfolio problems with covariance uncertainty.
- Kernel smoothing does not reliably improve on sample average approximation in the tested settings.
- Sample average approximation serves as a strong default that is hard to beat across both problem classes.
Where Pith is reading between the lines
- The relative value of each method appears to hinge on the specific form of interaction between the random variables and the decision variables.
- Testing on additional problem classes, such as non-convex or high-dimensional objectives, would help map the conditions under which each alternative is preferable.
- Computational trade-offs between the methods were not quantified, so practical adoption would require balancing observed gains against runtime costs.
Load-bearing premise
The two chosen test problems are representative enough of practical stochastic optimization instances that the observed performance ordering generalizes.
What would settle it
Running the same methods on a new test problem with high-dimensional nonlinear interactions where sample average approximation is consistently outperformed by kernel smoothing would challenge the reported ordering.
read the original abstract
We explore the performance of sample average approximation in comparison with several other methods for stochastic optimization when there is information available on the underlying true probability distribution. The methods we evaluate are (a) bagging; (b) kernel smoothing; (c) maximum likelihood estimation (MLE); and (d) a Bayesian approach. We use two test sets, the first has a quadratic objective function allowing for very different types of interaction between the random component and the univariate decision variable. Here the sample average approximation is remarkably effective and only consistently outperformed by a Bayesian approach. The second test set is a portfolio optimization problem in which we use different covariance structures for a set of 5 stocks. Here bagging, MLE and a Bayesian approach all do well.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper empirically compares sample average approximation (SAA) with bagging, kernel smoothing, maximum likelihood estimation (MLE), and a Bayesian approach for stochastic optimization when information on the true distribution is available. On a univariate quadratic objective with varied random-component interactions, SAA is reported as remarkably effective with only the Bayesian method consistently outperforming it; on a 5-stock mean-variance portfolio with several covariance structures, bagging, MLE, and Bayesian methods all perform well.
Significance. If the reported performance ordering proves robust, the work would usefully illustrate the strength of SAA on simple problems and identify settings where other methods yield gains, offering practical guidance for stochastic optimization. The study supplies no machine-checked proofs, reproducible code, or parameter-free derivations.
major comments (2)
- [Test problem descriptions (abstract and experimental sections)] The headline claim about 'when we can improve on SAA' rests on results from only two narrow families (univariate quadratic objective and 5-asset portfolio). No argument is supplied that these instances capture the relevant dimensions of difficulty (e.g., decision-variable dimension, non-convexity, constraint structure, or tail behavior), so the observed ranking may not generalize.
- [Empirical evaluation and results] The abstract states clear empirical outcomes (SAA only consistently beaten by Bayesian on the quadratic set; bagging/MLE/Bayesian all succeed on the portfolio set) but supplies no information on sample sizes, number of replications, statistical significance, or variance across runs, preventing verification of the central performance claims.
minor comments (1)
- [Method descriptions] Clarify exactly how the 'information on the true distribution' is supplied to each competing method (bagging, kernel smoothing, MLE, Bayesian) so that the comparisons can be replicated.
Simulated Author's Rebuttal
We thank the referee for the constructive comments on our manuscript. We address each major comment below and indicate the revisions we plan to make.
read point-by-point responses
-
Referee: [Test problem descriptions (abstract and experimental sections)] The headline claim about 'when we can improve on SAA' rests on results from only two narrow families (univariate quadratic objective and 5-asset portfolio). No argument is supplied that these instances capture the relevant dimensions of difficulty (e.g., decision-variable dimension, non-convexity, constraint structure, or tail behavior), so the observed ranking may not generalize.
Authors: The two test problems were selected as illustrative cases to demonstrate contrasting behaviors of SAA relative to the other methods, rather than as a claim of broad generality. The quadratic example varies the interaction structure between the random component and decision variable, while the portfolio example varies covariance structures. We acknowledge that these do not exhaustively cover dimensions such as high-dimensional decisions, non-convexity, or heavy tails. In revision we will add explicit discussion in the introduction and conclusions motivating the choice of these problems and stating the limited scope of the empirical findings. revision: partial
-
Referee: [Empirical evaluation and results] The abstract states clear empirical outcomes (SAA only consistently beaten by Bayesian on the quadratic set; bagging/MLE/Bayesian all succeed on the portfolio set) but supplies no information on sample sizes, number of replications, statistical significance, or variance across runs, preventing verification of the central performance claims.
Authors: We agree that the experimental details are insufficient for verification. The full manuscript will be revised to report sample sizes, number of replications, measures of variability, and any statistical comparisons in the experimental sections. We will also ensure the abstract does not overstate outcomes without these supporting details. revision: yes
Circularity Check
No circularity: purely empirical comparison on fixed test instances
full rationale
The manuscript contains no derivations, fitted parameters, self-citations used as load-bearing premises, or ansatzes. All reported performance orderings are direct numerical outcomes of running the listed methods (SAA, bagging, kernel smoothing, MLE, Bayesian) on two explicitly defined test families whose random components and decision variables are enumerated in the text. Because there is no derivation chain at all, no step can reduce to its own inputs by construction. The work is therefore self-contained against external benchmarks.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.