Pith. sign in

REVIEW 3 major objections 3 minor 1 cited by

Adversarial learning for nonparametric regression: Minimax rate and adaptive estimation

T0 review · 3 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Under SODA perturbations of size r, no regression estimator can beat the rate $r^{q(1\wedge\beta)} + n^{-q\beta/(2\beta+d)}$ in adversarial $L_q$ risk; the paper proves this and attains it.

desk verdict The finite-q minimax rate is a genuine step forward, but Lemma 1's proof as written swaps c and \bar c on the decreasing intervals, so the lower bound does not yet cover asymmetric SODA attacks. read the letter →

arxiv 2506.01267 v1 pith:24QV6N32 submitted 2025-06-02 stat.ML cs.LGmath.STstat.MEstat.TH

Jingfu Peng , Yuhong Yang This is my paper
classification stat.MLcs.LGmath.STstat.MEstat.TH MSC 62G0862G20
keywords nonparametricregressionadversarialrobustnessminimaxratelocalpolynomialestimationHöldersmoothnessSODAattacksadaptivebandwidthselection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks what any estimator can guarantee when future inputs $X$ may be adversarially perturbed inside a set $A(x)$ of diameter $r$ before the estimator sees them. It proves that for regression functions with Hölder smoothness $\beta$ on $[0,1]^d$, the minimax adversarial $L_q$-risk is exactly of order $r^{q(1\wedge\beta)} + n^{-q\beta/(2\beta+d)}$ for $1\le q<\infty$, and $r^{1\wedge\beta} + (n/\log n)^{-\beta/(2\beta+d)}$ for the sup-norm risk, provided each attack set contains a line segment of length proportional to $r$ along one fixed direction (the SODA condition). The attack term is unavoidable and is forced by a lower-bound construction using a periodic base function with period $8r$ along the attack direction. A piecewise local polynomial estimator with bandwidth $h\asymp r\vee n^{-1/(2\beta+d)}$ attains the rate, and a data-driven bandwidth-selection version reaches it up to a logarithmic factor without knowing $\beta$ or $r$. If correct, this settles the sample-complexity cost of robustness in nonparametric regression and shows a phase transition: attacks below a critical magnitude cost nothing, while stronger attacks force the rate to depend on $r$.

What carries the argument

The load-bearing objects are the SODA attack class and the piecewise local polynomial estimator. SODA (Definition 4) requires that each perturbation set $A(x)$ contain a line segment of length at least of order $r$ inside the line $\{x+kv:k\in\mathbb R\}\cap\Omega$ for a fixed unit direction $v$; this geometric condition makes the attack term $r^{q(1\wedge\beta)}$ unavoidable and covers all $\ell_p$-ball attacks. The PP estimator (3.11) fits a regularized local polynomial at the nearest discretization point $u_x$ and evaluates it at $x$, with bandwidth $h\asymp r\vee n^{-1/(2\beta+d)}$; the piecewise structure turns pointwise convergence into the localized sup-norm control that adversarial risk demands. For the lower bound, the construction uses a periodic base function $f_0$ and a localized Fano argument; for the adaptive upper bound, a data-driven bandwidth-selection rule over a geometric grid of smoothness levels selects the bandwidth without knowing $\beta$ or $r$.

What would settle it

A concrete check is to keep the Hölder class fixed but take $A(x)$ to be the intersection of $\Omega$ with a sphere of radius $r$ (so the set has diameter $r$ but contains no chord of length proportional to $r$ inside $\Omega$), then compute or simulate the minimax adversarial $L_q$ risk. If the rate falls back to the standard $n^{-q\beta/(2\beta+d)}$ whenever $r^{1\wedge\beta}$ is negligible relative to the standard term, the SODA line-segment condition is essential; if the $r$-term persists, the lower bound holds under a weaker geometric assumption.

Watch

Extended reading notes

Core claim

The central discovery is a minimax theorem: the adversarial risk separates into a statistical term and an attack term. Theorem 1 states that, for $1\le q<\infty$, $\inf_{\hat f}\sup_{P_{(X,Y)}\in\mathcal P(\beta)} R_{A,q}(\hat f,f)\asymp r^{q(1\wedge\beta)}+n^{-q\beta/(2\beta+d)}$ uniformly over the SODA attack class $A\in\mathcal T(r)$, and for $q=\infty$ the rate is $r^{1\wedge\beta}+(n/\log n)^{-\beta/(2\beta+d)}$. The lower bound is proved by localizing Fano's method around a base function $f_0$ whose flat-top and flat-bottom profile repeats with period $8r$ along the attack direction, so any function close to $f_0$ in $L_q$ still has adversarial variation $G_{A,q}\ge C r^{1\wedge\beta}$; the upper bound is proved for a piecewise local polynomial estimator whose bandwidth grows to $r$ when the attack dominates. The same two-term structure holds for the sup-norm risk, with the usual $\log n$ penalty in the standard term. The adaptive estimator attains these rates up to a logarithmic factor across scales of smoothness and attack magnitude, and matches the minimax rate exactly in sup norm.

Load-bearing premise

The load-bearing premise is the SODA condition: every attack set must contain a straight segment of length proportional to the attack radius $r$ along one fixed global direction, and if an attack of the same diameter is built without such segments, the lower-bound construction no longer forces the $r^{q(1\wedge\beta)}$ term to appear.

Editorial extensions

If this is right

  • No estimator can improve on the two-term rate uniformly over Hölder classes and SODA attacks, so any defense that claims faster worst-case convergence must restrict the function class, the attack geometry, or the risk metric.
  • There is a critical attack magnitude, $r\sim n^{-1/(2\beta+d)}$ for $\beta\le1$ and $r\sim n^{-\beta/(2\beta+d)}$ for $\beta>1$: below it the adversarial risk matches the standard rate, above it the $r$-term dominates and the attack is successful.
  • The piecewise local polynomial estimator with bandwidth $h\asymp r\vee n^{-1/(2\beta+d)}$ attains the minimax rate, meaning that robust estimation under future $X$-attacks requires only the correct bandwidth choice once the attack magnitude is known.
  • The data-driven estimator attains the minimax rate up to an extra logarithmic factor without knowledge of $\beta$ or $r$, and is exactly minimax in the sup-norm case.
  • For the sup-norm risk the standard statistical term carries the usual $n/\log n$ penalty, so uniform robustness is logarithmically more expensive than $L_q$ robustness with finite $q$.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • (Editorial extension) If the SODA geometry is weakened, the phase transition likely survives only for attacks that contain such segments; for segment-free attacks of the same diameter, robust and standard rates may coincide, which would make the line-segment condition the true boundary of the phenomenon.
  • (Editorial extension) The same two-term decomposition suggests a benchmark for adversarial training algorithms: to be minimax optimal, any training-based defense must control localized sup-norm error, not merely pointwise or $L_2$ error, over the perturbation sets.
  • (Editorial extension) For anisotropic Hölder classes, the attack direction should enter the minimax rate through the smallest smoothness along the perturbed coordinate, so an attacker optimally chooses the least smooth direction; this is consistent with the paper's Remark 1 but is not proved there.
  • (Editorial extension) The equivalence with the TRADES objective shown in the appendix implies that the same rates bound the sample complexity of that regularized training loss up to constants, so the minimax result carries over to a commonly used defense objective.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper studies minimax rates for nonparametric regression under adversarial L_q risks with future X-attacks. It introduces (v, r)-SODA perturbation sets, proves two-term rates r^{q(1∧β)} + n^{-qβ/(2β+d)} for 1≤q<∞ and r^{1∧β} + (n/log n)^{-β/(2β+d)} for q=∞, constructs a piecewise local polynomial estimator attaining these rates, and proposes a Lepski-type adaptive estimator that attains the rates up to logarithmic factors over scales of β and r.

Significance. If the proofs are completed, this is a substantial contribution: it gives the first general minimax characterization for adversarial nonparametric regression under L_q risks with 1≤q<∞, identifies the phase transition in the attack radius, and provides a rate-optimal estimator that works for all attacks of bounded diameter, not just SODA. The appendix contains detailed proofs with explicit events and constants, and the adaptive estimator addresses unknown β and r. The lower-bound construction around a periodic base function is original and the localized Fano argument is clearly structured. The main caveats are the two gaps in Definition 4 and Lemma 1 described below.

major comments (3)
  1. [Section 2.4, Definition 4] Definition 4(ii) allows c = \bar c = 0. In that case, for every interior x, the intersection of A(x) with the line x + Rv is a singleton, because k'_min(x) = k_min(x) ∨ 0 = 0 and k'_max(x) = k_max(x) ∧ 0 = 0. Such an attack still satisfies the formal definition with r > 0 if the diameter is attained in directions orthogonal to v, e.g., A(x) = {x + u : u·v = 0, ||u|| ≤ r} ∩ Ω. The lower-bound construction in Section 4.1 builds f0 that varies only along v, so for this attack the adversarial term r^{q(1∧β)} cannot be forced and Theorem 1 would be false as stated. The informal claim in Section 2.4 that condition (ii) gives a line segment of length on the order of at least r is not implied by the formal definition. The authors should add a uniform lower bound such as c ∨ \bar c ≥ c_0 > 0, and should also ensure that the constants in Lemma 1 do not degenerate as c + \bar c → 0.
  2. [Section 4.1, Lemma 1, Eqs. (4.17)–(4.18)] On the decreasing interval [a_{k-1}+6r, a_{k-1}+8r], f0 is decreasing in the first coordinate, so the supremum of f0 over A(x) is attained at the left endpoint x - (c r)e_1 and the infimum at the right endpoint x + (\bar c r)e_1. The displayed definitions in (4.18) use x - (\bar c r)e_1 for δ^u_0 and those in (4.17) use x + (c r)e_1 for δ^l_0 on that interval. When c ≠ \bar c, at least one of these points is not an element of A(x), so the identities u0 = f0∘δ^u_0 and l0 = f0∘δ^l_0 fail. Consequently, the pushforward bound (4.14), the inequality (4.13), and the resulting lower bound (4.19) are not established for general SODA constants. The fix is local — swap the constants on the decreasing interval — but the proof must then re-verify the pushforward bound ≤ 2 with the corrected maps, including boundary truncation near the vertices of Ω.
  3. [Appendix C.4, proof of Theorem 3 for q = ∞] The stated proof of Theorem 3 for q = ∞ is omitted with the sentence 'we omit the repetitive details here for brevity.' Since Theorem 3 is stated for all 1 ≤ q ≤ ∞ and the logarithmic loss differs between q < ∞ and q = ∞, the authors should either provide a complete proof for q = ∞ or give a precise reduction to the arguments of Sections B.4 and C.3, including the treatment of the selected bandwidth and the high-probability events in the sup-norm setting.
minor comments (3)
  1. [Section 4.1, Eq. (4.20)] The passage to the displayed lower bound with the integral over [0, (\bar c + c)r] is not fully justified; please specify the subregion of x where the sup-minus-inf difference is at least a constant times x_1^β, and show that the resulting κ_1 is bounded below uniformly in c, \bar c under the corrected Definition 4.
  2. [Section 2.4] The informal statement that condition (ii) ensures the line segment 'has length on the order of at least r' should be made formal in the definition, for example by requiring c ∨ \bar c ≥ c_0 > 0 for a universal constant c_0, so that the lower-bound constants in Theorem 4 are uniform over A ∈ T(r).
  3. [Section 3.2, Theorem 3] The adaptive construction uses M ≍ n, which is much larger than the M ≍ 1/h used in Theorem 2; the authors should state explicitly why the approximation error 1/M^{qβ} and the cardinality bound Card(I_x) ≤ (2rM+2)^d remain harmless in the proof of Theorem 3.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the adversarial rate terms are derived from explicit Hölder estimates and explicit base-function constructions; the only same-author dependency is the delegated q=∞ lower bound, which is a non-circular self-citation.

full rationale

The central finite-q lower bound is self-contained: the r^{q(1∧β)} term comes from Lemma 4, a Hölder Taylor bound on |f(x')-f(x)|, and from Lemma 1, where the base function f0 in (4.5)/(4.7) is constructed explicitly and shown to satisfy G_{A,q}(f0) ≥ κ1 r^{1∧β}. No parameter is fitted to the target rate, and the localized Fano reduction in Section 4.2 does not presuppose the adversarial risk being derived. The PP and adaptive upper bounds are also proved directly from kernel estimation and Lepski-type arguments. The only author-overlapping dependency is the q=∞ lower bound, which is delegated to the same-author preprint Peng and Yang (2024) in Section 4 and Appendix D.2 rather than re-derived; this is a self-citation and a proof omission, but it is not definitional or constructional circularity, since the cited result is a distinct sup-norm theorem for additive perturbation sets and the paper reduces the SODA case to it by choosing an additive subset A' ⊆ A. The endpoint issue flagged in Lemma 1 for asymmetric SODA constants is a proof-correctness concern, not a circularity.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The finite-q results are derived under Definitions 1-4 and kernel Assumptions 1-2; no parameters are fitted to data. The lower bound additionally invokes standard Fano/local metric entropy machinery (Section 4.2 and Lemma 3) and the q=∞ lower bound imports Peng and Yang (2024).

assumptions (6)
  • domain assumption The regression function lies in the Hölder class F(β,Cβ) (Definition 3).
    Used throughout Theorem 1 and in Lemma 4 to bound the deterministic deviation r^{1∧β}; the phase transition in (3.1) depends on β.
  • domain assumption The design density is bounded above and below on [0,1]^d (Definition 2).
    Needed for the local design events in Lemma 6 and for converting pointwise risk bounds into Lq risk bounds; ensures every cell of the PP partition has comparable probability.
  • domain assumption Errors are sub-Gaussian with parameter σ (Definition 1).
    Used in concentration inequalities and Lemma 7 for maxima of sub-Gaussian variables in the upper bound.
  • domain assumption The attack A is a (v,r)-SODA (Definition 4).
    Needed for the lower bound; condition (ii) guarantees line segments of length proportional to r, which the base function f0 exploits. The upper bound holds for the larger class T'(r).
  • domain assumption Kernel satisfies Assumptions 1-2 (lower/upper bounds and monotonicity).
    Assumption 1 controls eigenvalue concentration in Lemma 6; Assumption 2 makes the ratio matrix K_{h'h} well-defined for the Lepski selection.
  • standard math Local metric entropy and Fano's inequality (Yang and Barron, 1999; Tsybakov, 2009).
    Invoked in Lemma 3 and Appendix D.1 to convert packing entropy into a minimax lower bound.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adversarial learning for nonparametric regression: Minimax rate and adaptive estimation." pith.science (2026). https://pith.science/paper/24QV6N32

@misc{pith2026250601267,
  author       = {Pith},
  title        = {Pith review of: Adversarial learning for nonparametric regression: Minimax rate and adaptive estimation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/24QV6N32}},
  note         = {Machine review of arXiv:2506.01267}
}
abstract

Despite tremendous advancements of machine learning models and algorithms in various application domains, they are known to be vulnerable to subtle, natural or intentionally crafted perturbations in future input data, known as adversarial attacks. While numerous adversarial learning methods have been proposed, fundamental questions about their statistical optimality in robust loss remain largely unanswered. In particular, the minimax rate of convergence and the construction of rate-optimal estimators under future $X$-attacks are yet to be worked out. In this paper, we address this issue in the context of nonparametric regression, under suitable assumptions on the smoothness of the regression function and the geometric structure of the input perturbation set. We first establish the minimax rate of convergence under adversarial $L_q$-risks with $1 \leq q \leq \infty$ and propose a piecewise local polynomial estimator that achieves the minimax optimality. The established minimax rate elucidates how the smoothness level and perturbation magnitude affect the fundamental limit of adversarial learning under future $X$-attacks. Furthermore, we construct a data-driven adaptive estimator that is shown to achieve, within a logarithmic factor, the optimal rate across a broad scale of nonparametric and adversarial classes.

Figures

Figures reproduced from arXiv: 2506.01267 by the authors.

Figure 1
Figure 1. An illustration of the (v, r)-SODA satisfying the conditions in Definition 4. The shaded region A(x) denotes the perturbation set corresponding to the input x. Next, we assume that for each x ∈ Ω, the perturbation set satisfies x ∈ A(x) ⊆ Ω. The magnitude of the adversarial attack is measured by r = rn ≜ sup x∈Ω,x′∈A(x) ∥x ′ − x∥. We introduce the following definition to characterize a required geometric nature of t… view at source ↗
Figure 2
Figure 2. Diagram of the minimax rates for a fixed [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. An example of f0 defined in (4.5) with Cβ = 2, β = 1/2, and r = 0.02. let a+ ≜ max(a, 0). 4.1 Construction of base distributions Without loss of generality, we focus on an adversarial attack A ∈ T (r) with the principle direction v = (1, 0, . . . , 0)⊤. Otherwise, we can construct new functions from the functions f0 defined below by rotations of axes. Note that the rotation of a function does not change the smoothne… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: An example of f0 defined in (4.7) with Bβ = 1 and r = 0.05. We construct the base function f0 : [0, 1]d → R as: f0(x) =    0 0 ≤ x1 < 2r, Bβψ0  x−2rn 1−4rn  2r ≤ x1 < 1 − 2r, Bβ exp(−1) 1 − 2r ≤ x1 ≤ 1, (4.7) where Bβ > 0 is a constant. An example of (4.7) is …

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Generalization and Trade-off in Adversarial Training: An RKHS Perspective via Kernel Integral Operators

    stat.ML 2026-07 accept novelty 7.0 of 10

    RKHS adversarial training loses minimax prediction rate via noise in the mixed robustness term; noise-debiased AT restores the polynomial minimax rate up to a log factor.

Reference graph

Works this paper leans on

114 extracted references · 72 canonical work pages · cited by 1 Pith paper

  1. [1]

    and Tsybakov, A

    Audibert, J.-Y. and Tsybakov, A. B. (2007). Fast learning rates for plug-in classifiers. The Annals of Statistics , 35(2):608--633

    2007

  2. [2]

    Awasthi, P., Frank, N., and Mohri, M. (2021). On the existence of the adversarial B ayes classifier. In Advances in Neural Information Processing Systems , volume 34, pages 2978--2990

    2021

  3. [3]

    Barron, A., Birgé, L., and Massart, P. (1999). Risk bounds for model selection via penalization. Probability Theory and Related Fields , 113(3):301--413

    1999

  4. [4]

    Bartlett, P. L. and Mendelson, S. (2006). Empirical minimization. Probability Theory and Related Fields , 135(3):311--334

    2006

  5. [5]

    E., and Nemirovski, A

    Ben-Tal, A., Ghaoui, L. E., and Nemirovski, A. (2009). Robust Optimization . Princeton University Press

    2009

  6. [6]

    Bertin, K. (2004). Minimax exact constant in sup-norm for nonparametric regression with random design. Journal of Statistical Planning and Inference , 123(2):225--242

    2004

  7. [7]

    N., Cullina, D., and Mittal, P

    Bhagoji, A. N., Cullina, D., and Mittal, P. (2019). Lower bounds on adversarial robustness from optimal transport. In Advances in Neural Information Processing Systems , volume 32, pages 7496--7508

    2019

  8. [8]

    Biggio, B., Corona, I., Maiorca, D., Nelson, B., S rndi \' c , N., Laskov, P., Giacinto, G., and Roli, F. (2013). Evasion attacks against machine learning at test time. In Machine Learning and Knowledge Discovery in Databases , pages 387--402. Springer Berlin Heidelberg

    2013

Show all 114 references
  1. [9]

    Biggio, B., Nelson, B., and Laskov, P. (2012). Poisoning attacks against support vector machines. In Proceedings of the 29th International Conference on Machine Learning

    2012

  2. [10]

    Birg\' e , L. (1983). Approximation dans les espaces métriques et théorie de l'estimation. Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete , 65(2):181--237

    1983

  3. [11]

    Brown, L. D. and Low, M. G. (1996). A constrained risk inequality with applications to nonparametric functional estimation . The Annals of Statistics , 24(6):2524--2535

    1996

  4. [12]

    Bungert, L., García Trillos, N., and Murray, R. (2023). The geometry of adversarial training in binary classification . Information and Inference: A Journal of the IMA , 12(2):921--968

    2023

  5. [13]

    Cai, T. T. and Low, M. G. (2005). Nonparametric estimation over shrinking neighborhoods: Superefficiency and adaptation . The Annals of Statistics , 33(1):184--213

    2005

  6. [14]

    Cai, T. T. and Wei, H. (2021). Transfer learning for nonparametric classification: Minimax rate and adaptive classifier . The Annals of Statistics , 49(1):100--128

    2021

  7. [15]

    and Wagner, D

    Carlini, N. and Wagner, D. A. (2017). Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy , pages 39--57

    2017

  8. [16]

    Catoni, O. (2004). Statistical Learning Theory and Stochastic Optimization Ecole d'Eté de Probabilités de Saint-Flour XXXI - 2001 . École d'Été de Probabilités de Saint-Flour, 1851. Springer Berlin Heidelberg, 1st ed. 2004. edition

    2004

  9. [17]

    Chhor, J., Sigalla, S., and Tsybakov, A. B. (2024). Benign overfitting and adaptive nonparametric regression. Probability Theory and Related Fields , 189(3):949--980

    2024

  10. [18]

    Cleveland, W. S. (1979). Robust locally weighted regression and smoothing scatterplots. Journal of the American Statistical Association , 74(368):829--836

    1979

  11. [19]

    Cohen, J., Rosenfeld, E., and Kolter, J. Z. (2019). Certified adversarial robustness via randomized smoothing. In Proceedings of the 36th International Conference on Machine Learning , volume 97, pages 1310--1320. PMLR

    2019

  12. [20]

    Dan, C., Wei, Y., and Ravikumar, P. (2020). Sharp statistical guaratees for adversarially robust gaussian classification. In Proceedings of the 37th International Conference on Machine Learning , volume 119, pages 2345--2355. PMLR

    2020

  13. [21]

    Devroye, L., Gy \"o rfi, L., and Lugosi, G. (1996). A Probabilistic Theory of Pattern Recognition . Springer New York, NY

    1996

  14. [22]

    Dobriban, E., Hassani, H., Hong, D., and Robey, A. (2023). Provable tradeoffs in adversarially robust classification. IEEE Transactions on Information Theory , 69(12):7793--7822

    2023

  15. [23]

    Dohmatob, E. (2024). Consistent adversarially robust linear classification: N on-parametric setting. In Proceedings of the 41st International Conference on Machine Learning , volume 235, pages 11149--11164. PMLR

    2024

  16. [24]

    Donhauser, K., Tifrea, A., Aerni, M., Heckel, R., and Yang, F. (2021). Interpolation can hurt robust generalization even when there is no noise. In Advances in Neural Information Processing Systems , volume 34, pages 23465--23477

    2021

  17. [25]

    Duchi, J., Hashimoto, T., and Namkoong, H. (2023). Distributionally robust losses for latent covariate mixtures. Operations Research , 71(2):649--664

    2023

  18. [26]

    Duchi, J. C. and Namkoong, H. (2021). Learning models with uniform performance via distributionally robust optimization . The Annals of Statistics , 49(3):1378--1406

    2021

  19. [27]

    Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., and Song, D. (2018). Robust physical-world attacks on deep learning visual classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , ...

    2018

  20. [28]

    and Gijbels, I

    Fan, J. and Gijbels, I. (1995). Data-driven bandwidth selection in local polynomial fitting: V ariable bandwidth and spatial adaptation. Journal of the Royal Statistical Society: Series B (Methodological) , 57(2):371--394

    1995

  21. [29]

    Fano, R. (1961). Transmission of Information: A Statistical Theory of Communication . The MIT Press, Cambridge

    1961

  22. [30]

    Frank, N. S. and Niles-Weed, J. (2024). Existence and minimax theorems for adversarial surrogate risks in binary classification. Journal of Machine Learning Research , 25(58):1--41

    2024

  23. [31]

    Ga\" ffas, S. (2005). Convergence rates for pointwise curve estimation with a degenerate design. Mathematical Methods of Statistics , 14(1):1--27

    2005

  24. [32]

    Ga\" ffas, S. (2007a). On pointwise adaptive curve estimation based on inhomogeneous data. ESAIM: Probability and Statistics , 11:344--364

    2007

  25. [33]

    Ga\" ffas, S. (2007b). Sharp estimation in sup norm with random design. Statistics & Probability Letters , 77(8):782--794

    2007

  26. [34]

    Ga\" ffas, S. (2009). Uniform estimation of a signal based on inhomogeneous data. Statistica Sinica , 19(2):427--447

    2009

  27. [35]

    R., and Ruggeri, F

    Gallego, V., Naveiro, R., Redondo, A., Insua, D. R., and Ruggeri, F. (2024). Protecting classifiers from attacks. Statistical Science , 39(3):449--468

    2024

  28. [36]

    Geisser, S. (1993). Predictive Inference . Chapman and Hall/CRC

    1993

  29. [37]

    and Lepski, O

    Goldenshluger, A. and Lepski, O. (2008). Universal pointwise selection rule in multivariate function estimation. Bernoulli , 14(4):1150--1190

    2008

  30. [38]

    and Nemirovski, A

    Goldenshluger, A. and Nemirovski, A. (1997). On spatially adaptive estimation of nonparametric regression. Mathematical Methods of Statistics , 6(2):135--170

    1997

  31. [39]

    and Tsybakov, A

    Goldenshluger, A. and Tsybakov, A. (2001). Adaptive prediction and estimation in linear regression with infinitely many parameters. The Annals of Statistics , 29(6):1601--1619

    2001

  32. [40]

    Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning . MIT Press. http://www.deeplearningbook.org

    2016

  33. [41]

    J., Shlens, J., and Szegedy, C

    Goodfellow, I. J., Shlens, J., and Szegedy, C. (2015). Explaining and harnessing adversarial examples. In 3rd International Conference on Learning Representations

    2015

  34. [42]

    z yzak, A., and Walk, H

    Gy \"o rfi, L., Kohler, M., Kr\. z yzak, A., and Walk, H. (2002). A Distribution-Free Theory of Nonparametric Regression . Springer New York

    2002

  35. [43]

    and Zhang, T

    Hao, Y. and Zhang, T. (2024). The surprising harmfulness of benign overfitting for adversarial robustness. arXiv preprint arXiv:2401.12236

    2024 arXiv

  36. [44]

    and Javanmard, A

    Hassani, H. and Javanmard, A. (2024). The curse of overparametrization in adversarial training: Precise analysis of robust generalization for random features regression. The Annals of Statistics , 52(2):441--465

    2024

  37. [45]

    Hastie, T., Montanari, A., Rosset, S., and Tibshirani, R. J. (2022). Surprises in high-dimensional ridgeless least squares interpolation . The Annals of Statistics , 50(2):949--986

    2022

  38. [46]

    Ibragimov, I. A. and Khas'minskii, R. Z. (1982). Bounds for the risks of non-parametric regression estimates. Theory of Probability & Its Applications , 27(1):84--99

    1982

  39. [47]

    and Soltanolkotabi, M

    Javanmard, A. and Soltanolkotabi, M. (2022). Precise statistical analysis of classification accuracies for adversarial training. The Annals of Statistics , 50(4):2127--2156

    2022

  40. [48]

    Javanmard, A., Soltanolkotabi, M., and Hassani, H. (2020). Precise tradeoffs in adversarial training for linear regression. In Proceedings of Thirty Third Conference on Learning Theory , volume 125, pages 2034--2078. PMLR

    2020

  41. [49]

    and Nemirovski, A

    Juditsky, A. and Nemirovski, A. (2000). Functional aggregation for nonparametric regression. The Annals of Statistics , 28(3):681--712

    2000

  42. [50]

    B., Lepski, O

    Juditsky, A. B., Lepski, O. V., and Tsybakov, A. B. (2009). Nonparametric estimation of composite functions . The Annals of Statistics , 37(3):1360--1404

    2009

  43. [51]

    and Loh, P.-L

    Khim, J. and Loh, P.-L. (2018). Adversarial risk bounds via function transformation. arXiv preprint arXiv:1810.09519

    2018 arXiv

  44. [52]

    Kolmogorov, A. N. and Tikhomirov, V. M. (1959). -entropy and -capacity of sets in function spaces. Uspekhi Matematicheskikh Nauk , 14(2(86)):3--86

    1959

  45. [53]

    Koltchinskii, V. (2006). Local Rademacher complexities and oracle inequalities in risk minimization . The Annals of Statistics , 34(6):2593--2656

    2006

  46. [54]

    Korostelev, A. P. and Tsybakov, A. B. (1993). Minimax Theory of Image Reconstruction . Lecture Notes in Statistics. Springer New York

    1993

  47. [55]

    Kumano, S., Kera, H., and Yamasaki, T. (2023). Adversarial training from mean field perspective. In Advances in Neural Information Processing Systems , volume 36, pages 75097--75150

    2023

  48. [56]

    and Bayraktar, E

    Lai, L. and Bayraktar, E. (2020). On the adversarial robustness of robust estimators. IEEE Transactions on Information Theory , 66(8):5097--5109

    2020

  49. [57]

    Le Cam, L. (1986). Asymptotic Methods in Statistical Decision Theory . Springer-Verlag New York

    1986

  50. [58]

    LeCam, L. (1973). Convergence of estimates under dimensionality restrictions. The Annals of Statistics , 1(1):38--53

    1973

  51. [59]

    and Mendelson, S

    Lecu\' e , G. and Mendelson, S. (2009). Aggregation via empirical risk minimization. Probability Theory and Related Fields , 145(3):591--613

    2009

  52. [60]

    and Raginsky, M

    Lee, J. and Raginsky, M. (2018). Minimax statistical learning with W asserstein distances. In Advances in Neural Information Processing Systems , volume 31, pages 2692--2701

    2018

  53. [61]

    Lepski, O. V. (1991). On a problem of adaptive estimation in G aussian white noise. Theory of Probability & Its Applications , 35(3):454--466

    1991

  54. [62]

    Lepski, O. V. (2015). Adaptive estimation over anisotropic functional classes via oracle approach. The Annals of Statistics , 43(3):1178--1242

    2015

  55. [63]

    V., Mammen, E., and Spokoiny, V

    Lepski, O. V., Mammen, E., and Spokoiny, V. G. (1997). Optimal spatial adaptation to inhomogeneous smoothness: A n approach based on kernel estimates with variable bandwidth selectors. The Annals of Statistics , 25(3):929--947

    1997

  56. [64]

    Lepski, O. V. and Serdyukova, N. (2014). Adaptive estimation under single-index constraint in a regression model. The Annals of Statistics , 42(1):1--28

    2014

  57. [65]

    Lepski, O. V. and Spokoiny, V. G. (1997). Optimal pointwise adaptive methods in nonparametric estimation. The Annals of Statistics , 25(6):2512--2546

    1997

  58. [66]

    Liu, C., Jiao, Y., Wang, J., and Huang, J. (2024). Nonasymptotic bounds for adversarial excess risk under misspecified models. SIAM Journal on Mathematics of Data Science , 6(4):847--868

    2024

  59. [67]

    Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. (2018). Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Representations

    2018

  60. [68]

    and Tsybakov, A

    Mammen, E. and Tsybakov, A. B. (1999). Smooth discrimination analysis. The Annals of Statistics , 27(6):1808--1829

    1999

  61. [69]

    A., Rao, A

    Mehrabi, M., Javanmard, A., Rossi, R. A., Rao, A. B., and Mai, T. (2021). Fundamental tradeoffs in distributionally adversarial training. In Proceedings of the 38th International Conference on Machine Learning , volume 139, pages 7544--7554. PMLR

    2021

  62. [70]

    Min, Y., Chen, L., and Karbasi, A. (2021). The curious case of adversarially robust models: M ore data can help, double descend, or hurt generalization. In Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence , volume 161, pages 129--139. PMLR

    2021

  63. [71]

    Mustafa, W., Lei, Y., and Kloft, M. (2022). On the generalization analysis of adversarial learning. In Proceedings of the 39th International Conference on Machine Learning , volume 162, pages 16174--16196. PMLR

    2022

  64. [72]

    D., Wu, X., Jha, S., and Swami, A

    Papernot, N., McDaniel, P. D., Wu, X., Jha, S., and Swami, A. (2016). Distillation as a defense to adversarial perturbations against deep neural networks. In IEEE Symposium on Security and Privacy , pages 582--597

    2016

  65. [73]

    Peng, J., Li, Y., and Yang, Y. (2024). On optimality of M allows model averaging. Journal of the American Statistical Association , pages 1--26

    2024

  66. [74]

    and Yang, Y

    Peng, J. and Yang, Y. (2024). Minimax rates of convergence for nonparametric regression under adversarial attacks. arXiv preprint arXiv:2410.09402

    2024 arXiv

  67. [75]

    Pollard, D. (1990). Empirical processes: Theory and applications. NSF-CBMS Regional Conference Series in Probability and Statistics , 2:i--86

    1990

  68. [76]

    Pydi, M. S. and Jog, V. S. (2020). Adversarial risk via optimal transport and optimal couplings. In Proceedings of the 37th International Conference on Machine Learning , volume 119, pages 7814--7823. PMLR

    2020

  69. [77]

    M., Yang, F., Duchi, J., and Liang, P

    Raghunathan, A., Xie, S. M., Yang, F., Duchi, J., and Liang, P. (2019). Adversarial training can hurt generalization. In ICML 2019 Workshop on Identifying and Understanding Deep Learning Phenomena

    2019

  70. [78]

    Roth, K., Kilcher, Y., and Hofmann, T. (2020). Adversarial training is a form of data-dependent operator norm regularization. In Advances in Neural Information Processing Systems , volume 33, pages 14973--14985

    2020

  71. [79]

    and Dohmatob, E

    Scetbon, M. and Dohmatob, E. (2023). Robust linear regression: G radient-descent, early-stopping, and beyond. In Proceedings of The 26th International Conference on Artificial Intelligence and Statistics , volume 206, pages 11583--11607. PMLR

    2023

  72. [80]

    Schmidt, L., Santurkar, S., Tsipras, D., Talwar, K., and Madry, A. (2018). Adversarially robust generalization requires more data. In Advances in Neural Information Processing Systems , volume 31, pages 5019--5031

    2018

  73. [81]

    Schmidt-Hieber, J. (2020). Nonparametric regression using deep neural networks with ReLU activation function . The Annals of Statistics , 48(4):1875--1897

    2020

  74. [82]

    and Ben-David, S

    Shalev-Shwartz, S. and Ben-David, S. (2014). Understanding Machine Learning: From Theory to Algorithms . Cambridge University Press

    2014

  75. [83]

    Shapiro, A. (2017). Distributionally robust stochastic programming. SIAM Journal on Optimization , 27(4):2258--2275

    2017

  76. [84]

    Sinha, A., Namkoong, H., and Duchi, J. C. (2018). Certifying some distributional robustness with principled adversarial training. In 6th International Conference on Learning Representations

    2018

  77. [85]

    Spokoiny, V. G. (1998). Estimation of a function with discontinuities via local polynomial fit with an adaptive window choice. The Annals of Statistics , 26(4):1356--1378

    1998

  78. [86]

    and Jegelka, S

    Staib, M. and Jegelka, S. (2017). Distributionally robust deep learning as a generalization of adversarial training. In NIPS workshop on Machine Learning and Computer Security

    2017

  79. [87]

    Stone, C. J. (1975). Nearest neighbor estimators of a nonlinear regression function. In Computer Science and Statistics: 8th Annual Symposium on the Interface , pages 413--418. Health Sciences Computing Facility, UCLA

    1975

  80. [88]

    Stone, C. J. (1977). Consistent nonparametric regression. The Annals of Statistics , 5(4):595--620

    1977

  81. [89]

    Stone, C. J. (1982). Optimal global rates of convergence for nonparametric regression. The Annals of Statistics , 10(4):1040--1053

    1982

  82. [90]

    V., and Sakurai, K

    Su, J., Vargas, D. V., and Sakurai, K. (2019). One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation , 23(5):828--841

    2019

  83. [91]

    J., and Fergus, R

    Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I. J., and Fergus, R. (2014). Intriguing properties of neural networks. In 2nd International Conference on Learning Representations

    2014

  84. [92]

    and Yang, Y

    Tang, R. and Yang, Y. (2023). Minimax rate of distribution estimation on unknown submanifolds under adversarial losses. The Annals of Statistics , 51(3):1282--1308

    2023

  85. [93]

    Tsipras, D., Santurkar, S., Engstrom, L., Turner, A., and Madry, A. (2019). Robustness may be at odds with accuracy. In 7th International Conference on Learning Representations

    2019

  86. [94]

    Tsybakov, A. B. (1986). Robust reconstruction of functions by the local-approximation method. Problems of Information Transmission , 22(2):133--146

    1986

  87. [95]

    Tsybakov, A. B. (2003). Optimal rates of aggregation. In Learning Theory and Kernel Machines , pages 303--313. Springer Berlin Heidelberg

    2003

  88. [96]

    Tsybakov, A. B. (2009). Introduction to Nonparametric Estimation . Springer New York

    2009

  89. [97]

    Tu, Z., Zhang, J., and Tao, D. (2019). Theoretical analysis of adversarial learning: A minimax approach. In Advances in Neural Information Processing Systems , volume 32, pages 12259--12269

    2019

  90. [98]

    van de Geer, S. A. (2000). Empirical Processes in M-estimation . Cambridge university press

    2000

  91. [99]

    Vapnik, V. N. (1999). The Nature of Statistical Learning Theory . Springer Science & Business Media

    1999

  92. [100]

    Wainwright, M. J. (2019). High-Dimensional Statistics: A Non-Asymptotic Viewpoint . Cambridge University Press

    2019

  93. [101]

    Wang, Z., Paterlini, S., Gao, F., and Yang, Y. (2014). Adaptive minimax regression estimation over sparse _q -hulls. Journal of Machine Learning Research , 15(50):1675--1711

    2014

  94. [102]

    Wegkamp, M. (2003). Model selection in nonparametric regression . The Annals of Statistics , 31(1):252--273

    2003

  95. [103]

    and Kolter, J

    Wong, E. and Kolter, J. Z. (2018). Provable defenses against adversarial examples via the convex outer adversarial polytope. In Proceedings of the 35th International Conference on Machine Learning , volume 80, pages 5283--5292. PMLR

    2018

  96. [104]

    B., Carroll, R

    Xiao, Z., Linton, O. B., Carroll, R. J., and Mammen, E. (2003). More efficient local polynomial estimation in nonparametric regression with autocorrelated errors. Journal of the American Statistical Association , 98(464):980--992

    2003

  97. [105]

    Xing, Y., Zhang, R., and Cheng, G. (2021). Adversarially robust estimate and risk analysis in linear regression. In Proceedings of The 24th International Conference on Artificial Intelligence and Statistics , volume 130, pages 514--522. PMLR

    2021

  98. [106]

    Yang, Y. (1999a). Minimax nonparametric classification. i. rates of convergence. IEEE Transactions on Information Theory , 45(7):2271--2284

    1999

  99. [107]

    Yang, Y. (1999b). Model selection for nonparametric regression. Statistica Sinica , 9(2):475--499

    1999

  100. [108]

    Yang, Y. (2001). Adaptive regression by mixing. Journal of the American Statistical Association , 96(454):574--588

    2001

  101. [109]

    Yang, Y. (2004). Aggregating regression procedures to improve performance. Bernoulli , 10(1):25--47

    2004

  102. [110]

    and Barron, A

    Yang, Y. and Barron, A. (1999). Information-theoretic determination of minimax rates of convergence. The Annals of Statistics , 27(5):1564--1599

    1999

  103. [111]

    Yin, D., Ramchandran, K., and Bartlett, P. L. (2019). Rademacher complexity for adversarially robust generalization. In Proceedings of the 36th International Conference on Machine Learning , volume 97, pages 7085--7094. PMLR

    2019

  104. [112]

    P., Ghaoui, L

    Zhang, H., Yu, Y., Jiao, J., Xing, E. P., Ghaoui, L. E., and Jordan, M. I. (2019). Theoretically principled trade-off between robustness and accuracy. In Proceedings of the 36th International Conference on Machine Learning , volume 97, pages 7472--7482

    2019

  105. [113]

    Zhang, Y., Li, Y., Cui, L., Cai, D., Liu, L., Fu, T., Huang, X., Zhao, E., Zhang, Y., Chen, Y., et al. (2023). Siren's song in the ai ocean: A survey on hallucination in large language models. arXiv preprint arXiv:2309.01219

    2023 arXiv

  106. [114]

    and Wan, Z

    Zhao, P. and Wan, Z. (2024). Robust nonparametric regression under poisoning attack. Proceedings of the AAAI Conference on Artificial Intelligence , 38(15):17007--17015

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.