Pith. sign in

REVIEW 3 major objections 6 minor 49 references

Sparse-penalized deep networks achieve minimax-optimal regression rates under covariate shift for a broad class of dependent processes.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 10:10 UTC pith:GNWGPT6C

load-bearing objection A genuinely new dependent-data extension for deep Huber and quantile regression under covariate shift, with sound-looking bounded-ratio rates, but the claimed optimality for unbounded density ratios rests on an unjustified δ→∞ argument and should be revised. the 3 major comments →

arxiv 2607.20309 v1 pith:GNWGPT6C submitted 2026-07-22 stat.ML cs.LG

Adaptive deep nonparametric regression from dependent data under covariate shift

William Kengne , Ehud Mossa Ockegna This is my paper
classification stat.ML cs.LG MSC 62G0862M1068T07
keywords deep neural networkscovariate shiftnonparametric regressionHuber regressionquantile regressiondependent datageneralized Bernstein inequalityminimax optimality
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper establishes that sparse-penalized deep neural networks can achieve, up to logarithmic factors, the minimax-optimal L2 error for nonparametric regression under covariate shift, even when the training observations are dependent rather than i.i.d. The setting covers both Huber and quantile regression with Hölder-smooth regression functions, and the target error is measured under the test distribution Q, whose covariate density may differ from the training distribution P. The key device is a generalized Bernstein inequality that measures dependence through an 'effective number of observations' φ(n), unifying i.i.d., φ-mixing, strong-mixing, and C-mixing processes. When the density ratio q/p is uniformly bounded, the reweighted estimator attains the rate φ(n)^{-2s/(2s+d)}; a truncated density ratio preserves optimality when the ratio is unbounded but has moments of any order. The sparse penalty makes the estimators adaptive, so architecture and tuning parameters do not need to know the smoothness s.

Core claim

The paper proposes a sparse-penalized deep network (SPDNN) estimator that reweights each sample by the density ratio r(x)=q_X0(x)/p_X0(x), and proves its L2(Q_X0) error is O((log φ(n))^ν φ(n)^{-2s/(2s+d)}) for any ν>3, up to log factors the minimax rate, whenever the process satisfies a generalized Bernstein inequality with effective sample size φ(n). This covers i.i.d. (φ(n)=n), φ-mixing (n), exponential α-mixing (n/(log n)^2), and geometrically C-mixing (n/(log n)^{2/ϱ}) processes. With r unknown, a two-step pre-training estimates r from unlabeled source and target samples and achieves the same rate for large enough m. For unbounded r with finite moments of any order, truncating the ratio

What carries the argument

The load-bearing object is Assumption (A1), a generalized Bernstein inequality holding for every bounded mean-zero function of the process; it packages all dependence structures through an effective number of observations φ(n) ≤ n. The sparse penalty J_n(h) = Σ_j π_{λ_n,τ_n}(|θ_j(h)|), with π nondecreasing and π(x)=λ_n for x>τ_n (e.g., clipped L1), provides adaptation to the unknown smoothness s. Reweighting by r(x) or its truncation T_η r turns the source risk into the target risk; when r is unknown, a least-squares SPDNN estimates it from unlabeled samples of both distributions. The proof decomposes the L2(Q) error into a statistical term controlled by (A1) and an approximation term for Hö

Load-bearing premise

The paper's dependent-data results hold only if the process satisfies Assumption (A1), a uniform Bernstein inequality with an effective number of observations φ(n) ≤ n; if dependence is too strong for such an inequality to hold, the claimed φ(n)-rates do not follow.

What would settle it

Take a stationary ergodic process with long memory (e.g., fractional Gaussian noise with Hurst parameter > 1/2) for which the generalized Bernstein inequality in (A1) is believed to fail, fit the reweighted SPDNN estimator to simulated data under a known covariate shift, and compare the empirical L2(Q_X0) error with the predicted φ(n)^{-2s/(2s+d)} decay. A decay strictly slower than the theory prescribes, or a failure of the tail bound for a simple bounded test function, would show the assumptions are tighter than stated.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • For any process satisfying (A1), the reweighted SPDNN estimator adaptively attains the minimax rate φ(n)^{-2s/(2s+d)} up to log factors, without knowing the smoothness s.
  • The unweighted estimator remains minimax optimal when the density ratio is bounded; under only a second moment it degrades to φ(n)^{-s/(2s+d)}, showing reweighting is essential.
  • Truncating the density ratio at a data-dependent threshold extends optimality to unbounded ratios with moments of any order.
  • The two-step pre-training estimator splits its error into a density-ratio estimation term (m unlabeled samples) plus a regression term (n labeled samples); for large enough m the regression rate dominates.
  • The same rates hold, up to log factors, for i.i.d., φ-mixing, exponential α-mixing, and geometrically C-mixing processes, so independence and weak dependence are unified.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The φ(n) abstraction is a plug-in principle: any dependence model that admits a Bernstein inequality of the stated form automatically inherits these rates, so the theory likely extends to non-standard settings once such inequalities are available.
  • The precise threshold prescription for the truncated ratio could inform practice in other importance-weighting problems (e.g., off-policy evaluation), where clipping is a common but ad hoc heuristic.
  • A natural experiment would simulate an AR(1) or TAR process with known α-mixing coefficients, induce a covariate shift, and check whether empirical L2(Q) errors track φ(n)^{-2s/(2s+d)} with the predicted log-powers; this would calibrate how tight the logarithmic factors really are.
  • If the smoothness class were replaced by composition-structured or piecewise-smooth functions, the architecture choices would change but the φ(n)-based error decomposition should remain the template.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper develops sparse-penalized deep neural network (SPDNN) estimators for nonparametric Huber and quantile regression under covariate shift when observations are dependent. The authors assume a generalized Bernstein inequality (A1) with an effective sample size φ(n), covering i.i.d., φ-mixing, exponential α-mixing, and geometrically C-mixing processes. They provide non-asymptotic L2(Q_X0) error bounds for unweighted, reweighted, and two-step pre-training reweighted estimators, with the density ratio known, estimated, or truncated. The stated central claim is that these estimators attain, up to logarithmic factors, the minimax optimal rate φ(n)^{-2s/(2s+d)} on Hölder classes, including the case where the density ratio is unbounded but has moments of any order.

Significance. If the claims are fully established, the paper would be a useful unification: it extends the i.i.d. covariate-shift results of [11] to dependent data, adds Huber regression, and replaces a sparsity constraint with a sparse penalty, which is computationally more tractable. The explicit dependence on φ(n) and the summary table for several dependence structures are helpful. The main proofs contain detailed calculations for the Huber loss and for the reweighted excess risk decomposition. However, the advertised minimax optimality under unbounded density ratios, and the optimality claim for dependent processes, are not actually established by the results as written; these issues affect the abstract, the introduction, Theorem 3.4, Theorem 4.2, and Table 1.

major comments (3)
  1. [§3.2, Theorem 3.4 and §7.8, Eq. (7.34)] The claim that the truncated-ratio estimator 'remains optimal' whenever r has moments of any order is not established. For fixed δ>0 the bound in (3.20) has exponent (δ/(1+δ))·2s/(2s+d), which is strictly smaller than 2s/(2s+d). The text says that as δ→∞ the bound coincides with the optimal rate, but δ is not a data parameter: U_μ in (A3) depends on μ=1+δ, and the bias term in (7.34) is U_μ/η^δ with η=φ(n)^{2s/((1+δ)(2s+d))}. The final O(·) suppresses U_μ, which can grow arbitrarily fast with μ (e.g., for a log-normal ratio, U_μ=e^{μ^2/2}). Choosing δ=δ_n→∞ injects an n-dependent constant that can destroy the rate. Thus Theorem 3.4 provides only a family of suboptimal bounds for fixed δ, not the advertised minimax rate under (A3) with all moments. This also affects Proposition 4.2, Theorem 4.2, and the corresponding rows of Table 1. The claim should be weakened, or an explicit uniform co
  2. [§2.3 (A1), §3.2, §5] The 'minimax optimal' statements for dependent data are upper-bound-only. The rates in Theorems 3.1–4.2 are of the form φ(n)^{-2s/(2s+d)} (up to logs), but the only lower bound invoked is Stone's i.i.d. result [42]. Assumption (A1) is a concentration condition that is satisfied with a very small φ(n) by processes with strong dependence; over the class of all processes satisfying (A1), no estimator can have a rate faster than the worst φ(n), so φ(n)^{-2s/(2s+d)} is not a lower bound for that class. For the specific mixing classes listed in Section 5, a matching minimax lower bound is not proved or quoted. The abstract and Section 3.2 should either provide/quote such lower bounds for these dependent classes or state more precisely that the paper establishes upper bounds matching the i.i.d. minimax rate with φ(n) playing the role of n.
  3. [§7.3, Eq. (7.16); §7.5; §7.9] Several load-bearing steps are imported from the authors' own unpublished or very recent preprints [21,22], and the present paper does not state or verify those results. In particular, (7.16) is asserted to follow from Corollary 3.4 of [22] verbatim; Theorem 3.2's proof says 'the rest ... is obtained by following the same steps as in the proof of Theorem 4.1 in [21]'; Proposition 4.1 and Proposition 4.2 similarly rely on 'the same steps as' [21]. Since the optimal-rate claims depend on these imported results, the manuscript should include the precise statements and either prove them in the appendix or cite published versions with a clear verification that their assumptions are satisfied in the present setting.
minor comments (6)
  1. [Theorem 4.2] The statement says 'Consider the estimator brS defined in (4.1)' but the theorem concerns the truncated estimator brη,S defined in (4.3).
  2. [§7.12, Eq. (7.47)] The exponent '2α/(2α+d)' appears to be a typo; it should be 2s1/(2s1+d). Similarly, (7.48) uses 's' where s2 is meant.
  3. [§7.5, after Eq. (7.41)] The sentence 'we get from (7.42)' should refer to (7.41).
  4. [§4.2, Remark 4.1] 'the convergence rate in (4.1)' should be 'the convergence rate in (4.7)'.
  5. [§5, Table 1] Typographical issues: 'Geoemtrically' should be 'Geometrically'; 'satisfaying' in Section 2.3 should be 'satisfying'.
  6. [§3.2, Theorem 3.4] The theorem chooses η and ν4 depending on the smoothness parameter s, which is in tension with the paper's 'adaptive' claim. Please clarify whether an s-free choice, such as ν4=1 as in Corollary 3.1, is sufficient for the stated rate.

Circularity Check

3 steps flagged

Dependent-data rates rest largely on the authors' own framework [21,22]; the covariate-shift/Huber/adaptive parts are new, so partial self-citation, not construction-identity circularity.

specific steps
  1. self citation load bearing [Section 7.3, proof of Theorem 3.1, after Eq. (7.16)]
    "ED[RP(bhD)-RP(hHσ,n)] ≤ ED[RP(bhD)-RP(h0)] ≲ (logφ(n))^ν / (φ(n))^{2s/(2s+d)} ,(7.16) where the last inequality above holds from Corollary 3.4 in [22]."

    The unweighted estimator's dependent-data rate is the base from which the advertised L2(Q_X0) minimax bound is obtained by multiplying by Γ. That rate is not proved here; it is imported from Corollary 3.4 of the authors' own framework preprint [22]. Thus the paper's first main dependent-data result is a self-citation rather than a self-contained derivation. The novel covariate-shift step is only the inequality ||h-h0||_{2,Q}^2 ≤ Γ||h-h0||_{2,P}^2.

  2. self citation load bearing [Section 7.5, proof of Theorem 3.2]
    "The rest of the proof is obtained by following the same steps as in the proof of Theorem 4.1 in [21] dealing with φ(n) instead of n (α). This leads to, for sufficiently large n, E[B1,n] ≤ ∫_0^∞ P(B1,n > ρ)dρ ≲ Γ/φ(n), and E[B2,n] ≤ ... Hence, the theorem follows."

    Theorem 3.2 is the oracle inequality on which all reweighted estimator results (Theorems 3.3, 3.4, 4.1, 4.2) depend. Its proof delegates the entire tail/concentration argument to the authors' own [21], with φ(n) inserted in place of n^α. The dependent-data reweighted rates are therefore the same theorem from [21] with a symbol substitution, plus the covariance-shift weighting. This is load-bearing self-citation, even though the weighting/truncation ideas are new.

  3. self citation load bearing [Section 7.9, proof of Proposition 4.1]
    "Therefore, since r∈ C^s(X,Γ) and in addition to (4.5) and (4.6), one can follow the same steps as in the proof of Theorem 4.1 and Corollary 4.3 in [21] (with φ(m) instead of n(α) and with κ= 2) to obtain the result of the proposition."

    The density-ratio estimator's L2(P_X0) rate is essential to the two-stage pre-training estimators in Theorems 4.1 and 4.2, but the proposition is obtained by 'following the same steps' as [21] rather than by an argument contained in this paper. The dependence component of the density-ratio estimation step is therefore inherited from the authors' prior framework, again making a central result rest on self-citation.

full rationale

No construction-identity circularity was found: the sparse-penalized estimators are defined by empirical risk plus a penalty, the density ratio is estimated by least squares, the truncation threshold is chosen explicitly, and no fitted parameter is renamed as a prediction. The main circularity-type issue is repeated load-bearing importation of dependent-data theory from the authors' own prior work: Theorem 3.1 uses 'Corollary 3.4 in [22]' for the unweighted rate; Theorem 3.2 says the proof is 'the same steps as in the proof of Theorem 4.1 in [21]'; and Proposition 4.1 says to follow [21] for the density-ratio rate. Since [22] is the authors' framework preprint and [21] is by the same first author, this is self-citation that carries the quantitative backbone of the dependent-data extension rather than independent external support. The covariate-shift reweighting, truncation, Huber analysis, and two-stage pretraining are nonetheless new and argued separately, so the central claim retains independent content beyond the self-citation chain. Separately, the statement after Theorem 3.4 that the bound 'coincides' with the minimax rate as δ→∞ is a correctness gap (the constants depend on U_μ and δ; see Eq. (7.34)), not a circularity. Overall score 4.

Axiom & Free-Parameter Ledger

4 free parameters · 7 axioms · 0 invented entities

The paper's results rest on five explicit assumptions (A1)-(A5) plus Hölder smoothness of the target function and the density ratio, and on the availability/independence of unlabeled target samples for the two-step estimators. The free parameters are the penalty tuning (λ_n, τ_n), the truncation level η, and the smoothness-dependent network architecture exponents. No new formal entities are posited; the 'effective number of observations' φ(n) is a borrowed concept from prior work.

free parameters (4)
  • penalty strength λ_n = λ_n ≍ (log φ(n))^{ν3}/φ(n)^{ν4} with 0<ν4≤1; e.g., ν4=1 in the bounded-ratio case and ν4=d/(2s+d) in the truncated case
    Chosen by hand to balance estimation and approximation; depends on the unknown smoothness s in the truncated case, so full adaptivity is not achieved.
  • penalty transition τ_n = τ_n ≤ 1/(16 Γ K_ℓτ (L_n+1)((N_n+1)B_n)^{L_n+1} φ(n))
    Defines the threshold beyond which the sparse penalty saturates; a tuning parameter chosen by hand in (3.14).
  • truncation level η = η ≍ φ(n)^{2s/((1+δ)(2s+d))}
    Used in the truncated density-ratio estimators (3.18)-(3.19) and (4.3); depends on the unknown smoothness s and moment exponent δ.
  • network architecture exponents = N_n ≳ φ(n)^{d/(2s+d)}, B_n ≳ φ(n)^{4(s+d)/(2s+d)}
    The stated architecture depends on s; the text argues it can be chosen independent of s by taking maximal exponents, but as stated these are smoothness-dependent and the practical choice is not fully specified.
axioms (7)
  • domain assumption (A1) Generalized Bernstein inequality with effective number of observations φ(n) for bounded mean-zero functions of the process
    Used throughout to control dependent-data fluctuations; all rates are expressed in terms of φ(n). Invoked in Theorems 3.1-3.4 and 4.1-4.2 and in every proof of the oracle inequalities.
  • domain assumption (A2) Density ratio r = q/p is uniformly bounded by Γ
    Used to convert L2(P) error to L2(Q) error and to bound reweighted risk; assumed in Theorem 3.1 part 1, Theorem 3.2, Theorem 3.3, Theorem 4.1.
  • domain assumption (A3) The μ-th moment of r under P_X0 is finite
    Relaxes boundedness; used in Theorem 3.1 part 2, Theorem 3.4, Proposition 4.2, Theorem 4.2.
  • domain assumption (A4) Conditional distribution regularity for quantile regression (and bounded conditional density)
    Provides the lower-bound-to-excess-risk connection (2.6) and the positivity of excess risk in Proposition 3.1; used in Lemmas 3.1-3.2 and all quantile-regression results.
  • domain assumption (A5) Density ratio is bounded away from zero (Υ > 0)
    Used in the proof of Theorem 4.1 to convert an L2(Q) bound on the pretraining estimator into an L2(P) bound; Equation (7.39).
  • domain assumption h0 belongs to the Hölder class C^s(X,K), and in Section 4 r belongs to C^{s1}(X,Γ)
    The approximation rates of DNNs for Hölder functions are invoked via Theorem 3.2 of [20] and used in (7.17) and (7.30).
  • domain assumption The extra unlabeled samples S_P and S_Q used for density-ratio estimation are independent of the training sample D
    Stated in Section 4.1 as a common hypothesis; needed for the clean decomposition in the proof of Theorem 4.1.

pith-pipeline@v1.3.0-alltime-deepseek · 35448 in / 14019 out tokens · 118665 ms · 2026-08-01T10:10:35.455073+00:00 · methodology

0 comments
read the original abstract

Covariate shift often occurs because, in many real applications, the source and the target observations may be generated from different distributions. In this case, the standard metric under the source distribution is not appropriate. This paper considers deep neural network estimators for nonparametric quantile and Huber regression under covariate shift and from dependent observations. We deal with a generalized Bernstein-type inequality that is satisfied by many classical models, including i.i.d. observations, $\phi$-mixing, strong mixing, and $\mathcal{C}$-mixing processes. To perform the covariate shift phenomenon, we propose a sparse-penalized deep neural network (SPDNN) estimator that takes into account the discrepancy between the source and target distributions of the data. When the density ratio (between the source and target distributions of the covariate) is unknown, a two steps pre-training procedure is carried out: the first step is devoted to the construction of a least squares SPDNN estimator of the density ratio; which is used in the second step to perform a pre-training reweighted SPDNN estimator of the regression function. For both the quantile and the Huber regression, non-asymptotic error bounds of the proposed SPDNN estimators are established in the class of H\"older smooth functions. These estimators can adaptively attain (up to a logarithmic factor) the minimax optimal convergence rate from i.i.d. data as well as from several classical time series models.

Figures

Figures reproduced from arXiv: 2607.20309 by Ehud Mossa Ockegna, William Kengne.

Figure 1
Figure 1. Figure 1: Our main contribution: settings considered with the associated SPDNN estimators. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 3 linked inside Pith

  1. [1]

    Minimax optimality of deep neural networks on dependent data via pac-bayes bounds

    Alquier, P., and Kengne, W. Minimax optimality of deep neural networks on dependent data via pac-bayes bounds. Electronic Journal of Statistics 19 , 2 (2025), 5895--5924

    2025

  2. [2]

    On deep learning as a remedy for the curse of dimensionality in nonparametric regression

    Bauer, B., and Kohler, M. On deep learning as a remedy for the curse of dimensionality in nonparametric regression

  3. [3]

    l1-penalized quantile regression in high-dimensional sparse models

    Belloni, A., and Chernozhukov, V. l1-penalized quantile regression in high-dimensional sparse models. The Annals of Statistics 39 , 1 (2011), 82--130

    2011

  4. [4]

    Geometric ergodicity of nonlinear autoregressive models with changing conditional variances

    Chen, M., and Chen, G. Geometric ergodicity of nonlinear autoregressive models with changing conditional variances. Canadian Journal of Statistics 28 , 3 (2000), 605--614

    2000

  5. [5]

    Learning bounds for importance weighting

    Cortes, C., Mansour, Y., and Mohri, M. Learning bounds for importance weighting. Advances in neural information processing systems 23\/ (2010)

    2010

  6. [6]

    Variable selection and estimation with the seamless-l 0 penalty

    Dicker, L., Huang, B., and Lin, X. Variable selection and estimation with the seamless-l 0 penalty. Statistica Sinica\/ (2013), 929--962

    2013

  7. [7]

    Doukhan, P. Mixing . Springer New York, New York, NY, 1994, pp. 15--23

    1994

  8. [8]

    Duan, C., Jiao, Y., Kang, L., Lu, X., and Yang, J. Z. On convergence rates of deep nonparametric regression under covariate shift

  9. [9]

    How do noise tails impact on deep relu networks? The Annals of Statistics 52 , 4 (2024), 1845--1871

    Fan, J., Gu, Y., and Zhou, W.-X. How do noise tails impact on deep relu networks? The Annals of Statistics 52 , 4 (2024), 1845--1871

    2024

  10. [10]

    Variable selection via nonconcave penalized likelihood and its oracle properties

    Fan, J., and Li, R. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American statistical Association 96 , 456 (2001), 1348--1360

    2001

  11. [11]

    Deep nonparametric quantile regression under covariate shift

    Feng, X., He, X., Jiao, Y., Kang, L., and Wang, C. Deep nonparametric quantile regression under covariate shift. Journal of Machine Learning Research 25 , 385 (2024), 1--50

    2024

  12. [12]

    Towards a unified analysis of kernel-based methods under covariate shift

    Feng, X., He, X., Wang, C., Wang, C., and Zhang, J. Towards a unified analysis of kernel-based methods under covariate shift. Advances in Neural Information Processing Systems 36\/ (2023), 73839--73851

    2023

  13. [13]

    Domain adaptation for medical image analysis: a survey

    Guan, H., and Liu, M. Domain adaptation for medical image analysis: a survey. IEEE Transactions on Biomedical Engineering 69 , 3 (2021), 1173--1185

    2021

  14. [14]

    Hang, H., Feng, Y., Steinwart, I., and Suykens, J. A. Learning theory estimates with observations from general stationary stochastic processes. Neural computation 28 , 12 (2016), 2853--2889

    2016

  15. [15]

    A bernstein-type inequality for some mixing processes and dynamical systems with an application to learning

    Hang, H., and Steinwart, I. A bernstein-type inequality for some mixing processes and dynamical systems with an application to learning. The Annals of Statistics\/ (2017), 708--743

    2017

  16. [16]

    Deep neural networks learn non-smooth functions effectively

    Imaizumi, M., and Fukumizu, K. Deep neural networks learn non-smooth functions effectively. In The 22nd international conference on artificial intelligence and statistics\/ (2019), PMLR, pp. 869--878

    2019

  17. [17]

    Advantage of deep neural networks for estimating functions with singularity on hypersurfaces

    Imaizumi, M., and Fukumizu, K. Advantage of deep neural networks for estimating functions with singularity on hypersurfaces. Journal of Machine Learning Research 23 , 111 (2022), 1--54

    2022

  18. [18]

    Jiao, Y., Lin, H., Luo, Y., and Yang, J. Z. Deep transfer learning: Model framework and error analysis. arXiv preprint arXiv:2410.09383\/ (2024)

    Pith/arXiv arXiv 2024

  19. [19]

    Deep nonparametric regression on approximate manifolds: Nonasymptotic error bounds with polynomial prefactors

    Jiao, Y., Shen, G., Lin, Y., and Huang, J. Deep nonparametric regression on approximate manifolds: Nonasymptotic error bounds with polynomial prefactors. The Annals of Statistics 51 , 2 (2023), 691--716

    2023

  20. [20]

    Excess risk bound for deep learning under weak dependence

    Kengne, W. Excess risk bound for deep learning under weak dependence. Mathematical Methods in the Applied Sciences 48 , 6 (2025), 6844--6850

    2025

  21. [21]

    Deep learning from strongly mixing observations: Sparse-penalized regularization and minimax optimality

    Kengne, W., and Wade, M. Deep learning from strongly mixing observations: Sparse-penalized regularization and minimax optimality. Journal of Complexity\/ (2025), 101978

    2025

  22. [22]

    A general framework for deep learning

    Kengne, W., and Wade, M. A general framework for deep learning. arXiv preprint arXiv:2512.23425\/ (2025)

    arXiv 2025

  23. [23]

    Robust deep learning from weakly dependent data

    Kengne, W., and Wade, M. Robust deep learning from weakly dependent data. Neural Networks 185\/ (2025), 107227

    2025

  24. [24]

    Fast convergence rates of deep neural networks for classification

    Kim, Y., Ohn, I., and Kim, D. Fast convergence rates of deep neural networks for classification. Neural Networks 138\/ (2021), 179--197

    2021

  25. [25]

    W., Sagawa, S., Marklund, H., Xie, S

    Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al. Wilds: A benchmark of in-the-wild distribution shifts. In International conference on machine learning\/ (2021), PMLR, pp. 5637--5664

    2021

  26. [26]

    On the rate of convergence of a deep recurrent neural network estimate in a regression problem with dependent data

    Kohler, M., and Krzy \.z ak, A. On the rate of convergence of a deep recurrent neural network estimate in a regression problem with dependent data. Bernoulli 29 , 2 (2023), 1663--1685

    2023

  27. [27]

    Adaptive deep learning for nonlinear time series models

    Kurisu, D., Fukami, R., and Koike, Y. Adaptive deep learning for nonlinear time series models. Bernoulli 31 , 1 (2025), 240--270

    2025

  28. [28]

    Ma, C., Pathak, R., and Wainwright, M. J. Optimally tackling covariate shift in rkhs-based nonparametric regression. The Annals of Statistics 51 , 2 (2023), 738--761

    2023

  29. [29]

    Theoretical analysis of deep neural networks for temporally dependent observations

    Ma, M., and Safikhani, A. Theoretical analysis of deep neural networks for temporally dependent observations. Advances in Neural Information Processing Systems 35\/ (2022), 37324--37334

    2022

  30. [30]

    H., and Chatterjee, S

    Madrid Padilla, O. H., and Chatterjee, S. Risk bounds for quantile trend filtering. Biometrika 109 , 3 (2022), 751--768

    2022

  31. [31]

    Exponential inequalities and functional estimations for weak dependent data: applications to dynamical systems

    Maume-Deschamps, V. Exponential inequalities and functional estimations for weak dependent data: applications to dynamical systems. Stochastics and Dynamics 6 , 04 (2006), 535--560

    2006

  32. [32]

    Bernstein inequality and moderate deviations under strong mixing conditions

    Merlev \`e de, F., Peligrad, M., and Rio, E. Bernstein inequality and moderate deviations under strong mixing conditions. In High dimensional probability V: the Luminy volume , vol. 5. Institute of Mathematical Statistics, 2009, pp. 273--293

    2009

  33. [33]

    Smooth function approximation by deep neural networks with general activation functions

    Ohn, I., and Kim, Y. Smooth function approximation by deep neural networks with general activation functions. Entropy 21 , 7 (2019), 627

    2019

  34. [34]

    Nonconvex sparse regularization for deep neural networks and its optimality

    Ohn, I., and Kim, Y. Nonconvex sparse regularization for deep neural networks and its optimality. Neural computation 34 , 2 (2022), 476--517

    2022

  35. [35]

    Padilla, O. H. M., Tansey, W., and Chen, Y. Quantile regression with relu networks: Estimators and minimax rates. Journal of Machine Learning Research 23 , 247 (2022), 1--42

    2022

  36. [36]

    Optimal approximation of piecewise smooth functions using deep relu neural networks

    Petersen, P., and Voigtlaender, F. Optimal approximation of piecewise smooth functions using deep relu neural networks. Neural Networks 108\/ (2018), 296--330

    2018

  37. [37]

    W., Cannings, T

    Reeve, H. W., Cannings, T. I., and Samworth, R. J. Adaptive transfer learning. The Annals of Statistics 49 , 6 (2021), 3618--3649

    2021

  38. [38]

    Concentration of measure inequalities for markov chains and -mixing processes

    Samson, P.-M. Concentration of measure inequalities for markov chains and -mixing processes. The Annals of Probability 28 , 1 (2000), 416--461

    2000

  39. [39]

    Nonparametric regression using deep neural networks with relu activation function

    Schmidt-Hieber, J. Nonparametric regression using deep neural networks with relu activation function

  40. [40]

    Local convergence rates of the nonparametric least squares estimator with applications to transfer learning

    Schmidt-Hieber, J., and Zamolodtchikov, P. Local convergence rates of the nonparametric least squares estimator with applications to transfer learning. Bernoulli 30 , 3 (2024), 1845--1877

    2024

  41. [41]

    L., and Huang, J

    Shen, G., Jiao, Y., Lin, Y., Horowitz, J. L., and Huang, J. Nonparametric estimation of non-crossing quantile regression process with deep requ neural networks. Journal of Machine Learning Research 25 , 88 (2024), 1--75

    2024

  42. [42]

    Stone, C. J. Optimal global rates of convergence for nonparametric regression. The annals of statistics\/ (1982), 1040--1053

    1982

  43. [43]

    Measuring robustness to natural distribution shifts in image classification

    Taori, R., Dave, A., Shankar, V., Carlini, N., Recht, B., and Schmidt, L. Measuring robustness to natural distribution shifts in image classification. In Advances in Neural Information Processing Systems\/ (2020), vol. 33, pp. 18583--18599

    2020

  44. [44]

    Optimal kernel quantile learning with random features

    Wang, C., and Feng, X. Optimal kernel quantile learning with random features. arXiv preprint arXiv:2408.13591\/ (2024)

    Pith/arXiv arXiv 2024

  45. [45]

    Deep regression for repeated measurements under covariate shift

    Wang, Y., Xing, X., and Xu, W. Deep regression for repeated measurements under covariate shift. arXiv preprint arXiv:2605.24854\/ (2026)

    Pith/arXiv arXiv 2026

  46. [46]

    A minimax theory of nonparametric regression under covariate shift

    Zamolodtchikov, P. A minimax theory of nonparametric regression under covariate shift. arXiv preprint arXiv:2603.05897\/ (2026)

    arXiv 2026

  47. [47]

    Nearly unbiased variable selection under minimax concave penalty

    Zhang, C.-H. Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics 38 , 2 (2010), 894--942

    2010

  48. [48]

    Analysis of multi-stage convex relaxation for sparse regularization

    Zhang, T. Analysis of multi-stage convex relaxation for sparse regularization. Journal of Machine Learning Research 11 , 3 (2010)

    2010

  49. [49]

    Effective number of observations and unbiased estimators of variance for autocorrelated data-an overview

    Zi e ba, A. Effective number of observations and unbiased estimators of variance for autocorrelated data-an overview. Metrology and Measurement Systems , 1 (2010)

    2010