REVIEW 2 major objections 3 minor 1 cited by
This paper proves a randomized resampling estimator can reach true ε-stationarity for stochastic convex problems, where the actual subdifferential contains a small element, without Lipschitz or smoothness assumptions.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-02 08:09 UTC pith:4YTDSWD3
load-bearing objection Novel and credible route to genuine ε-stationarity without Lipschitz gradients; the proof has a real but repairable gap in Lemma 4.6 that the authors must fix. the 2 major comments →
Finding a stationary point of a stochastic convex problem
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The core claim is that for a stochastic convex problem—or more generally for a stochastic maximal monotone operator A_P (a set-valued map whose graph is maximal among monotone maps), with the normal cone of the constraint X added—the output of the proposed resampling procedure satisfies dist(0, A_P(bx_n)+N_X(bx_n)) → 0 in probability. This means the actual subdifferential of the population objective contains a vector of vanishing norm, the genuine first-order stationarity condition, not a surrogate like a small Moreau-envelope gradient. The proof shows that a positive-measure set of random starting points map to ε-stationary points under the empirical proximal-point update, so sampling enoug
What carries the argument
The central object is the Minty parameterization of the graph of a maximal monotone operator, M_{λ,A}(x) = (prox_{λA}(x), Y_λA(x)), a bi-Lipschitz homeomorphism between R^d and the graph. Its work is to turn the stationarity condition 0 ∈ A(x)+N_X(x) into a measure-preservation statement: the graph piece over the ε-stationary set is d-dimensional, and a dimension-theory lemma (Lemma 4.6) partitions it into pieces of complementary dimension, of which at least one has a positive-measure base. This lets the authors show that a positive fraction of random centers u map via the empirical proximal point to ε-stationary points, making randomized resampling effective.
Load-bearing premise
The load-bearing premise is the graph-decomposition Lemma 4.6(iii): for some k, the set of points whose subdifferential fiber contains a k-simplex of positive measure must itself have positive (d−k)-dimensional Hausdorff measure; if this positive-measure base fails to exist, the proof that a positive fraction of random centers land in the stationary pre-image collapses, with the secondary premise being that A_P+N_X is maximal monotone.
What would settle it
Construct a maximal monotone operator A (e.g., the subdifferential of a convex function) satisfying Assumption A.1 for which, for every k and ε>0, the set X_{k,ε} of Lemma 4.6 has zero (d−k)-dimensional Hausdorff measure—then the claimed positive-probability sampling event has no measure-theoretic support and the proof fails. Alternatively, simulate Algorithm 2 on a finite-dimensional analogue of the Section 1.3 example and check whether the selected point's population subdifferential residual fails to converge in probability.
If this is right
- For convex losses, Algorithm 2 returns a point whose population subdifferential contains a vector of norm at most ε with probability tending to one (Corollary 3.1).
- The same conclusion holds for stochastic variational inequalities and general maximal monotone operators satisfying Assumption A.1.
- In conformal prediction, ε-stationarity of the quantile loss implies coverage at level 1−α−ε, so the result yields distribution-free prediction sets certified by stationarity rather than by optimality.
- The authors prove that no neighborhood of the optimum maps entirely into the stationary set under the empirical proximal mapping, so some randomization is essential (Proposition 3).
- Rates of convergence and distribution-free guarantees remain open (Questions 1 and 2); the guarantee is asymptotic and depends on constants not explicitly computable from the problem data.
Where Pith is reading between the lines
- Because the proof needs only a positive fraction of starting points, a practical heuristic is to run the resampling with a modest number of draws and select by empirical subdifferential; if the dimension-theoretic constants were ever made quantitative, the same algorithm would immediately inherit a finite-sample rate.
- The graph decomposition suggests a link between stationarity and the geometry of singular sets of convex functions: functions whose subdifferential fibers are k-dimensional only on zero-measure bases would defeat the resampling argument, so the class of 'bad' functions may be topologically small.
- For conformal prediction, the result implies that any method that finds an ε-stationary point of the quantile loss—not necessarily a minimizer—yields valid coverage, opening a new design axis for conditional coverage.
- The impossibility result and the open Question 2 together suggest that any finite-sample rate must degrade with dimension, and that a rate-free statement that the empirical prox maps almost all points near the optimum into the stationary set is false.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies the problem of finding points satisfying true ε-stationarity, dist(0, ∂F_P(x)+N_X(x)) ≤ ε, for stochastic convex optimization, and the analogous formulation for stochastic maximal monotone operators. Because uniform Hausdorff convergence of subdifferentials does not hold in general, the authors develop a dimension-theoretic decomposition of the graph of the population subdifferential/operator. Using the Minty parameterization, they argue that positive-measure pieces of the graph survive under sampling, so that the preimage of the ε-stationary set under the empirical proximal map has positive Lebesgue measure. This leads to randomized resampling algorithms whose output satisfies the stationary residual convergence in probability (Theorems 1–2 and Corollary 3.1). The results are asymptotic and non-quantitative, and the paper is explicit about the lack of rates and about several open questions.
Significance. If the proof is completed as indicated, the result is significant: it shows that genuine ε-stationarity — not a Moreau-envelope or proximity surrogate — is statistically attainable for nonsmooth stochastic convex problems under only local moment and maximal monotonicity assumptions, without individual differentiability or Lipschitz continuity. The paper is self-contained and does not fit parameters to data; the constants arise from geometric measure theory and are non-quantitative, a limitation the authors state openly. The examples involving medians and conformal prediction make the notion of stationarity concrete and motivate the target. The graph-decomposition technique goes beyond existing singular-set results for convex functions and appears to be a novel and useful proof strategy, though one part of it currently needs repair.
major comments (2)
- [§4.4, Lemma 4.6] The lemma opens with 'Assume G has inductive dimension indG=d (and hence Hausdorff measure 0<H^d(G)<∞).' The implication indG=d ⇒ H^d(G)<∞ is false for general compact subsets of R^{2d}; a compact set of Hausdorff dimension d can have infinite d-dimensional Hausdorff measure. The finiteness is load-bearing: Eq. (10) uses the Fubini-type bound H^{d−k}(π(K)) ≤ C_{k,d−k}H^d(K)/ε, and Section 4.5 applies Lemma 4.6 to G_P without having established H^d(G_P)<∞. The gap is repairable via the Minty parameterization (7): since G_P is contained in the bi-Lipschitz image of a bounded subset of R^d, H^d(G_P) ≤ Lip(M)^d Leb(M^{-1}(G_P)) < ∞. This repair needs to be written into the proof; as stated, Lemma 4.6 asserts a false implication.
- [§3.1, proof of Theorem 2] The proof states that applying Theorem 1 shows P(Leb(X^-_{0,n})≥c)→1 whenever B_{1/2}⊂\hat B. But Theorem 1's conclusion is stated for a set of good x restricted to ||x−x_0||≤4λε and for pre-images landing in S(ε)∩B, with B=x_0+bB. X^-_{0,n} is not a subset of that restricted set, and \hat B⊃B_{1/2} does not by itself imply that \hat B contains the radius-4λε ball around x_0. The step can be repaired by taking ε≤b/8 and applying Theorem 1 with the neighborhood size b/2 (which is valid because Assumption A.1(iii) holds at smaller radii), but this argument is currently missing.
minor comments (3)
- [§4.3, Lemma 4.5(ii)] The proof that the graph piece G has dimension d is terse. Since G contains M_{λ,A}(K), the lower bound dim G ≥ d is clear, but the upper bound requires noting that M_{λ,A}^{-1}(G) is a bounded subset of R^d (because prox_{λA}(x)∈prox_{λA}(K) and Y_{λA}(x)∈rB), so ind M^{-1}(G)≤d. Please spell this out.
- [§3.1, Algorithm 1] The loop 'for i=1,...,√n' should state that √n is rounded to an integer, or use m_n=⌊√n⌋; otherwise the pseudo-code is not well-defined for general n.
- [§4.7, Example 6] In the decomposition for f(x)=|x|, the sentence about H^0({1})=H^0({−1})=1 seems to justify the 0-dimensional base set but is not connected to the subsequent use of H^1. Clarify the choice of base sets X_0 and X_1.
Circularity Check
No significant circularity: the stationarity target is not an input, Assumption A.1 is a stated domain condition, and no fitted parameter is relabeled as a prediction.
full rationale
The paper's derivation chain is self-contained: Theorem 2 and Corollary 3.1 reduce to Theorem 1, which is proved through the Minty parameterization (Section 4.3), the dimension-theoretic decomposition Lemma 4.6, the sampling survival Lemma 4.9, and the measure-preservation argument in Section 4.6. Assumption A.1 is explicitly a domain assumption on the population operator, not the conclusion; in the convex subdifferential case it is automatic by standard external facts about convex analysis. Lemma 4.1, the pointwise empirical convergence of Hausdorff distances, is proved in Appendix B.1 via symmetrization and concentration, rather than assumed or fitted. The algorithm's selection step uses an independent sample P_n' and Lemma 3.3 only as a statistical estimator of the population subdifferential; no fitted constant is later called a prediction. The self-citations [7] and [24] appear in motivational examples and in a discussion of distribution-free optimization, and neither is load-bearing for Theorem 1, Theorem 2, or Corollary 3.1. The only notable issue is a non-circular correctness gap: Lemma 4.6 states "Assume G has inductive dimension indG = d (and hence Hausdorff measure 0 < H^d(G) < ∞)", but the implication indG = d ⇒ H^d(G) < ∞ is false for general compact sets, and H^d(G) < ∞ is used in inequality (10). This is a mathematical proof gap, repairable via the Minty bi-Lipschitz parameterization and compactness, not a reduction of the result to its own inputs. Hence no circularity step is present.
Axiom & Free-Parameter Ledger
axioms (5)
- standard math Maximal monotone operators have a bi-Lipschitz Minty parameterization (Rockafellar–Wets Thm 12.15).
- standard math Va˘inˇste˘in's first theorem, the sum theorem, and the volume lower bound indX≥k ⇒ H^k(X)>0.
- standard math Federer's Fubini-type inequality for Hausdorff measures (Thm 2.10.25).
- domain assumption For convex f_z, ∂F_P(x)=E[∂f_z(x)] and ∂f is maximal monotone.
- domain assumption Assumption A.1: A_P+N_X maximal monotone, nonempty solution set, p≥2 moments near x0.
read the original abstract
We consider the problem of finding stationary points for stochastic convex optimization problems. Rather than surrogates to stationarity, such as a proximity-to-stationarity guarantee or small gradient of the Moreau envelope, we ask for a stronger notion: that the subdifferential of the objective actually contains a small element. This criterion is non-trivial, because subdifferentials of convex functions fail to converge uniformly, even in arbitrarily small neighborhoods of the optimum. Our convergence guarantees rely on dimension theory to decompose the graph of the subdifferential of a convex function, showing how stochastic sampling preserves "pieces" of these graphs, and allowing effective application of proximal-point-like methods.
Figures
Forward citations
Cited by 1 Pith paper
-
Lipschitzian SLLNs for random functions
Empirical averages of random locally Lipschitz functions converge to their expectation in the Lipschitz pseudometric under separability or NIP-definability conditions, giving uniform convergence of subdifferentials.
Reference graph
Works this paper leans on
-
[1]
Agarwal, P
A. Agarwal, P. L. Bartlett, P. Ravikumar, and M. J. Wainwright. Information-theoretic lower bounds on the oracle complexity of convex optimization.IEEE Transactions on Information Theory, 58(5):3235–3249, 2012
2012
-
[2]
G. Alberti. On the structure of singular sets of convex functions.Calculus of Variations and Partial Differential Equations, 2(1):17–27, 1994. doi: 10.1007/BF01234313
-
[3]
Alberti and L
G. Alberti and L. Ambrosio. A geometrical approach to monotone functions inR n. Mathematische Zeitschrift, 230(2):259–316, 1999
1999
-
[4]
G. Alberti, L. Ambrosio, and P. Cannarsa. On the singularities of convex functions. Manuscripta Mathematica, 76(3–4):421–435, 1992. doi: 10.1007/BF02567770. 31
-
[5]
Ambrosio, P
L. Ambrosio, P. Cannarsa, and H. M. Soner. On the propagation of singularities of semi- convex functions.Annali della Scuola Normale Superiore di Pisa - Classe di Scienze, Serie 4, 20(4):597–616, 1993
1993
-
[6]
A. N. Angelopoulos and S. Bates. Conformal prediction: A gentle introduction.Foun- dations and Trends in Machine Learning, 16(4), 2023
2023
-
[7]
F. Areces and J. C. Duchi. Distribution free M-estimation.arXiv:2505.22807 [math.ST], 2025
Pith/arXiv arXiv 2025
-
[8]
Asi and J
H. Asi and J. C. Duchi. The importance of better models in stochastic optimization. Proceedings of the National Academy of Sciences, 116(46):22924–22930, 2019
2019
-
[9]
Aubin and H
J.-P. Aubin and H. Frankowska.Set-Valued Analysis. Birkh¨ auser, 1990
1990
-
[10]
R. F. Barber, E. J. Cand` es, A. Ramdas, and R. J. Tibshirani. The limits of distribution- free conditional predictive inference.Information and Inference, 10(2):455–482, 2021
2021
-
[11]
H. H. Bauschke and P. L. Combettes.Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, Second edition, 2017
2017
-
[12]
D. P. Bertsekas. Stochastic optimization problems with nondifferentiable cost functionals. Journal of Optimization Theory and Applications, 12(2):218–231, 1973
1973
-
[13]
D. P. Bertsekas.Convex Optimization Algorithms. Athena Scientific, 2015
2015
-
[14]
P. Bianchi. Ergodic convergence of a stochastic proximal point algorithm.SIAM Journal on Optimization, 26(4):2235–2260, 2016
2016
-
[15]
Boyd and L
S. Boyd and L. Vandenberghe.Convex Optimization. Cambridge University Press, 2004
2004
-
[16]
Brøndsted and R
A. Brøndsted and R. T. Rockafellar. On the subdifferentiability of convex functions. Proceedings of the American Mathematical Society, 16:605–611, 1965
1965
-
[17]
Cauchois, S
M. Cauchois, S. Gupta, and J. Duchi. Knowing what you know: valid and validated confidence sets in multiclass and multilabel prediction.Journal of Machine Learning Research, 22(81):1–42, 2021
2021
-
[18]
P. L. Combettes and J. I. Madariaga. Asymptotic analysis of stochastic splitting methods for multivariate monotone inclusions.arXiv:2512.03023 [math.OC], 2025
arXiv 2025
-
[19]
Davis and D
D. Davis and D. Drusvyatskiy. Stochastic model-based minimization of weakly convex functions.SIAM Journal on Optimization, 29(1):207–239, 2019
2019
-
[20]
Davis and D
D. Davis and D. Drusvyatskiy. Graphical convergence of subgradients in nonconvex optimization and learning.Mathematics of Operations Research, 47(1):209–231, 2022
2022
-
[21]
Davis and B
D. Davis and B. Grimmer. Proximally guided stochastic subgradient method for nons- mooth, nonconvex problems.SIAM Journal on Optimization, 29(3):1908–1930, 2019
1908
-
[22]
V. H. de la Pe˜ na and E. Gin´ e.Decoupling: From Dependence to Independence. Springer, 1999
1999
-
[23]
Drusvyatskiy and C
D. Drusvyatskiy and C. Paquette. Efficiency of minimizing compositions of convex func- tions and smooth maps.Mathematical Programming, Series A, 178(1–2):503–558, 2019. 32
2019
-
[24]
J. C. Duchi. Sample-conditional coverage in conformal prediction. InAdvances in Neural Information Processing Systems 38, 2025
2025
-
[25]
I. Ekeland. On the variational principle.Journal of Mathematical Analysis and Applica- tions, 47(2):324–353, 1974
1974
-
[26]
Engelking.Dimension Theory
R. Engelking.Dimension Theory. North-Holland Publishing Company, 1978
1978
-
[27]
Federer.Geometric Measure Theory
H. Federer.Geometric Measure Theory. Springer Berlin, Heidelberg, 1969
1969
-
[28]
D. J. Foster, A. Sekhari, O. Shamir, N. Srebro, K. Sridharan, and B. Woodworth. The complexity of making the gradient small in stochastic convex optimization. InProceedings of the Thirty Second Annual Conference on Computational Learning Theory, 2019
2019
-
[29]
Gibbs, J
I. Gibbs, J. Cherian, and E. Cand` es. Conformal prediction with conditional guarantees. Journal of the Royal Statistical Society, Series B, 87(4):1100–1126, 2025
2025
-
[30]
Hiriart-Urruty and C
J. Hiriart-Urruty and C. Lemar´ echal.Convex Analysis and Minimization Algorithms I. Springer, New York, 1993
1993
-
[31]
Hurewicz and H
W. Hurewicz and H. Wallman.Dimension Theory. Princeton University Press, 1941
1941
-
[32]
Juditsky, A
A. Juditsky, A. Nemirovski, and C. Tauvel. Solving variational inequalities with the stochastic mirror-prox algorithm.Stochastic Systems, 1(1):17–58, 2011
2011
-
[33]
Kenderov
P. Kenderov. Multivalued monotone mappings are almost everywhere single-valued.Stu- dia Mathematica, 56(3):199–203, 1976
1976
-
[34]
Kotsalis, G
G. Kotsalis, G. Lan, and T. Li. Simple and optimal methods for stochastic variational inequalities, I: operator extrapolation.SIAM Journal on Optimization, 32(3):2041–2073, 2022
2041
-
[35]
J. Lei. Classification with confidence.Biometrika, 101(4):755–769, 2014
2014
-
[36]
Lei and L
J. Lei and L. Wasserman. Distribution-free prediction bands for non-parametric regres- sion.Journal of the Royal Statistical Society, Series B, 76(1):71–96, 2014
2014
-
[37]
Maggi.Sets of Finite Perimeter and Geometric Variational Problems: An Introduction to Geometric Measure Theory
F. Maggi.Sets of Finite Perimeter and Geometric Variational Problems: An Introduction to Geometric Measure Theory. Cambridge University Press, 2012
2012
-
[38]
J.-J. Moreau. Proximit´ e et dualit´ e dans un espace Hilbertian.Bulletin de la Soci´ et´ e Math´ ematique de France, 93:273–299, 1965
1965
-
[39]
Nemirovski, A
A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro. Robust stochastic approximation approach to stochastic programming.SIAM Journal on Optimization, 19(4):1574–1609, 2009
2009
-
[40]
Nesterov.Introductory Lectures on Convex Optimization
Y. Nesterov.Introductory Lectures on Convex Optimization. Kluwer Academic Publish- ers, 2004
2004
-
[41]
R. T. Rockafellar. Measurable dependence of convex sets and functions on parameters. Journal of Mathematical Analysis and Applications, 28:4–25, 1969
1969
-
[42]
R. T. Rockafellar.Convex Analysis. Princeton University Press, 1970. 33
1970
-
[43]
R. T. Rockafellar. On the maximality of sums of nonlinear monotone operators.Trans- actions of the American Mathematical Society, 149(1):75–88, 1970
1970
-
[44]
R. T. Rockafellar. Monotone operators and the proximal point algorithm.SIAM Journal on Control and Optimization, 14:877–898, 1976
1976
-
[45]
R. T. Rockafellar and R. J. B. Wets.Variational Analysis. Springer, New York, 1998
1998
-
[46]
Romano, E
Y. Romano, E. Patterson, and E. J. Cand` es. Conformalized quantile regression. In Advances in Neural Information Processing Systems 32, 2019
2019
-
[47]
F. Ruan. On the uniform convergence of subdifferentials in stochastic optimization and learning.Mathematics of Operations Research, To Appear, 2025. doi: 10.1287/moor. 2024.0533
arXiv 2025
-
[48]
Shalev-Shwartz, O
S. Shalev-Shwartz, O. Shamir, N. Srebro, and K. Sridharan. Stochastic convex opti- mization. InProceedings of the Twenty Second Annual Conference on Computational Learning Theory, 2009
2009
-
[49]
Shapiro and H
A. Shapiro and H. Xu. Uniform laws of large numbers for set-valued mappings and subdifferentials of random functions.Journal of Mathematical Analysis and Applications, 325(2):1390–1399, 2007
2007
-
[50]
L. Tian and J. Royset. Failure of uniform laws of large numbers for subdifferentials and beyond.arXiv:2511.16568 [math.OC], 2025
arXiv 2025
-
[51]
A. W. van der Vaart and J. A. Wellner.Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, New York, 1996
1996
-
[52]
V. Vovk, A. Grammerman, and C. Saunders. Machine-learning applications of algorith- mic randomness. InProceedings of the Sixteenth International Conference on Machine Learning, 1999
1999
-
[53]
M. J. Wainwright.High-Dimensional Statistics: A Non-Asymptotic Viewpoint. Cam- bridge University Press, 2019. 34
2019
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.