Pith. sign in

REVIEW 1 major objections 1 minor 1 cited by

ReLU networks estimate target regression under covariate shift by weighting with the density ratio and achieve minimax optimal rates via polynomial-dimension approximation bounds.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-29 23:56 UTC pith:XKRGUEB4

load-bearing objection The paper's real contribution is a ReLU approximation result with polynomial dimension dependence that lets them get non-asymptotic minimax rates for density-ratio transfer learning in repeated-measures regression. the 1 major comments →

arxiv 2605.24854 v1 pith:XKRGUEB4 submitted 2026-05-24 stat.ME

Deep Regression for Repeated Measurements under Covariate Shift

Yingxuan Wang , Xiangyu Xing , Wangli Xu This is my paper
classification stat.ME
keywords nonparametric regressioncovariate shiftrepeated measurementstransfer learningReLU neural networksdensity ratio estimationminimax optimal ratesapproximation theory
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper addresses nonparametric regression when responses cannot be observed in the target domain but are available in a source domain whose covariates follow a different distribution. It transfers information by reweighting source observations with the density ratio between the two covariate distributions, then fits the target regression function with ReLU feedforward neural networks. Both the known-ratio and unknown-ratio cases are treated, along with bounded and finite-moment ratio conditions. The central technical step is a new approximation result showing that the constants governing network size and error grow only polynomially with dimension rather than exponentially, which removes the dominant source of the curse of dimensionality and produces non-asymptotic bounds that match the minimax rate for repeated-measurements designs.

Core claim

Under the repeated-measurements setting, the proposed ReLU-FNN estimators of the target regression function, after density-ratio correction, attain the minimax optimal convergence rate; the proof relies on a novel approximation theory in which the constants that appear in the network-parameter bounds depend polynomially, rather than exponentially, on the input dimension.

What carries the argument

Density-ratio reweighting to correct covariate shift, paired with ReLU feedforward neural network approximation of both the regression function and (when unknown) the density ratio, where the approximation constants depend polynomially on dimension.

Load-bearing premise

The target regression function and the density ratio can be approximated to the required accuracy by ReLU neural networks under the given smoothness conditions, and the density ratio is either uniformly bounded or has finite moments.

What would settle it

Numerical experiments in which the observed convergence rate of the estimator is slower than the stated minimax rate, or in which the network approximation error constants are observed to grow exponentially rather than polynomially with dimension.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • The same polynomial-dimension approximation theory supplies sharper stochastic-error bounds for both the known-ratio and unknown-ratio estimators.
  • The estimators remain consistent and rate-optimal when the density ratio is unbounded but satisfies only finite-moment conditions.
  • Separate procedures are given for the case in which the density ratio is known versus the case in which it must be estimated jointly with the regression function.
  • The approach directly accommodates the repeated-measurements structure in the source domain.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The polynomial dependence on dimension may make the method viable in moderately high-dimensional covariate settings where exponential constants would have rendered neural-network approximation impractical.
  • The same approximation technique could be applied to other transfer-learning or distribution-shift problems whose error analysis is currently limited by exponential dimension dependence.
  • Testing the method on data sets with known repeated-measurements structure would provide a direct check on whether the claimed rate improvement materializes in finite samples.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 1 minor

Summary. The manuscript develops a transfer learning framework for nonparametric regression with repeated measurements under covariate shift, leveraging a source domain with observable responses. It proposes ReLU feedforward neural network estimators for the target regression function (and density ratio when unknown), handling both uniformly bounded and finite-moment density ratio cases. Non-asymptotic error bounds are established that achieve minimax optimal rates under the repeated measurements setting, supported by a new approximation theory in which network parameter constants scale polynomially (rather than exponentially) with dimension.

Significance. If the central claims hold, the work would advance nonparametric regression under covariate shift by delivering sharper stochastic error bounds via improved ReLU approximation theory that mitigates the curse of dimensionality. The explicit handling of known/unknown density ratio and bounded/unbounded cases, together with the minimax optimality result, represents a substantive theoretical contribution; the numerical simulations and real-data example provide supporting empirical evidence.

major comments (1)
  1. [Theoretical analysis] Theoretical analysis section: the novel approximation result asserting polynomial (rather than exponential) dependence of network constants on dimension is load-bearing for both the sharper non-asymptotic stochastic error bounds and the claimed mitigation of the curse of dimensionality. The precise statement of the theorem (including the dependence on smoothness indices, network depth/width, and the precise function classes for the regression function and density ratio) must be given explicitly, together with the key steps of the proof, to substantiate the minimax rate claim.
minor comments (1)
  1. [Abstract] Abstract: the distinction between the four scenarios (known/unknown density ratio crossed with bounded/finite-moment) is central but is described only at a high level; a single clarifying sentence on the data requirements for each case would improve readability.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback and positive assessment of the manuscript's contributions. The request for explicit details on the approximation theorem is reasonable and will be addressed by expanding the theoretical section in the revision.

read point-by-point responses

  1. Referee: [Theoretical analysis] Theoretical analysis section: the novel approximation result asserting polynomial (rather than exponential) dependence of network constants on dimension is load-bearing for both the sharper non-asymptotic stochastic error bounds and the claimed mitigation of the curse of dimensionality. The precise statement of the theorem (including the dependence on smoothness indices, network depth/width, and the precise function classes for the regression function and density ratio) must be given explicitly, together with the key steps of the proof, to substantiate the minimax rate claim.

    Authors: We agree that the approximation theorem is central to the claims and that its current presentation would benefit from greater explicitness. In the revised manuscript, we will add a dedicated subsection that states the theorem in full, specifying: (i) the Holder smoothness indices α for the regression function and β for the density ratio; (ii) the precise dependence of network depth L and width W on these indices and dimension d (with constants scaling as O(d^C) for some C independent of d); (iii) the function classes (e.g., bounded or finite-moment density ratios, Sobolev-type balls for the regression function). We will also outline the key proof steps: first, a univariate ReLU approximation lemma with polynomial constants; second, a tensor-product construction that preserves the polynomial scaling in d; third, an error decomposition separating approximation, estimation, and density-ratio estimation errors. These additions will directly support the minimax optimality and the mitigation of the curse of dimensionality. revision: yes

Circularity Check

0 steps flagged

No significant circularity in derivation chain

full rationale

The paper derives non-asymptotic error bounds achieving minimax rates under repeated measurements and covariate shift from explicit modeling assumptions on the target regression function and density ratio (both known and unknown cases, bounded or finite-moment). The central novelty is a ReLU FNN approximation result with polynomial (vs. exponential) dimension dependence, presented as an independent technical contribution rather than a fit or self-citation. No quoted step reduces a 'prediction' to a fitted input by construction, invokes a self-citation as the sole justification for uniqueness, or renames a known result; the bounds follow from standard smoothness and moment conditions without internal definitional loops. The derivation is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claims rest on standard smoothness and moment assumptions typical for nonparametric regression and neural approximation theory; no free parameters or invented entities are introduced in the abstract.

axioms (2)
  • domain assumption The regression function and density ratio satisfy smoothness conditions permitting approximation by ReLU FNNs with the stated error rates
    Invoked to obtain the non-asymptotic bounds and minimax optimality in both known and unknown density ratio scenarios.
  • domain assumption Covariate shift holds, with source and target distributions differing only through the marginal covariate law
    Core modeling assumption enabling the density-ratio correction framework.

pith-pipeline@v0.9.1-grok · 5761 in / 1300 out tokens · 33484 ms · 2026-06-29T23:56:58.399086+00:00 · methodology

0 comments
read the original abstract

This paper studies nonparametric regression with repeated measurements when the response in the target domain is unobservable or costly to collect. We adopt a transfer learning framework that leverages a source domain with observable responses under covariate shift. The target regression function is estimated by correcting the distribution shift via the density ratio. We consider both known and unknown density ratio scenarios, which reflect different data available for nonparametric regression estimation. In both cases, we further address two settings: the uniformly bounded density ratio and the unbounded case with finite moment conditions. Under the unknown density ratio scenario, both the density ratio and the target regression function are estimated using rectified linear unit (ReLU) feedforward neural networks (FNNs), whereas under the known density ratio scenario, only the target regression function is estimated by ReLU FNNs. Theoretically, we establish non-asymptotic error bounds for the proposed estimators and prove that they achieve the minimax optimal convergence rate under the repeated measurements setting. Notably, we develop a novel approximation theory where the constants of the network parameters depend polynomially, rather than exponentially as in existing works, on the dimension, thereby mitigating the curse of dimensionality. Consequently, we derive sharper non-asymptotic bounds for the stochastic error. The finite sample performance of the proposed method is demonstrated through numerical simulations and a real data application.

Figures

Figures reproduced from arXiv: 2605.24854 by Wangli Xu, Xiangyu Xing, Yingxuan Wang.

Figure 1
Figure 1. Figure 1: Normalized histograms and smoothed empirical density curves of the eight co [PITH_FULL_IMAGE:figures/full_fig_p026_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Predictive performance of the estimators NE and URE on the target domain. (a) [PITH_FULL_IMAGE:figures/full_fig_p026_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Adaptive deep nonparametric regression from dependent data under covariate shift

    stat.ML 2026-07 conditional novelty 6.0

    Sparse-penalized deep networks for Huber and quantile regression under covariate shift are shown to attain minimax rates (up to logs) for dependent data satisfying a generalized Bernstein inequality.

Reference graph

Works this paper leans on

32 extracted references · cited by 1 Pith paper

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...

  2. [2]

    D., Sklar, M., Berk, R., Buja, A., and Zhao, L

    Azriel, D., Brown, L. D., Sklar, M., Berk, R., Buja, A., and Zhao, L. (2022). Semi-supervised linear regression. Journal of the American Statistical Association , 117(540), 2238--2251

    2022

  3. [3]

    L., Harvey, N., Liaw, C., and Mehrabian, A

    Bartlett, P. L., Harvey, N., Liaw, C., and Mehrabian, A. (2019). Nearly-tight VC -dimension and pseudodimension bounds for piecewise linear neural networks. Journal of Machine Learning Research , 20(63), 1--17

    2019

  4. [4]

    and Kohler, M

    Bauer, B. and Kohler, M. (2019). On deep learning as a remedy for the curse of dimensionality in nonparametric regression. The Annals of Statistics , 47(4), 2261--2285

    2019

  5. [5]

    Cai, T. T. and Wei, H. (2021). Transfer learning for nonparametric classification. The Annals of Statistics , 49(1), 100--128

    2021

  6. [6]

    and Cai, T

    Chakrabortty, A. and Cai, T. (2018). Efficient and adaptive linear regression in semi-supervised settings. The Annals of Statistics , 46(4), 1541--1572

    2018

  7. [7]

    Y., Su, W., and Zhao, X

    Deng, S., Liu, K. Y., Su, W., and Zhao, X. (2025). Semiparametric inference for longitudinal data with informative observation times and terminal event. Statistica Sinica , 36(1)

    2025

  8. [8]

    Feng, X., He, X., Jiao, Y., Kang, L., and Wang, C. (2024). Deep nonparametric quantile regression under covariate shift. Journal of Machine Learning Research , 25(385), 1--50

    2024

  9. [9]

    Feng, X., He, X., Wang, C., Wang, C., and Zhang, J. (2023). Towards a unified analysis of kernel-based methods under covariate shift. Advances in Neural Information Processing Systems , 36, 73839--73851

    2023

  10. [10]

    Freudenthal, H. (1942). Simplizialzerlegungen von beschr \"a nkter Flachheit . Annals of Mathematics , 43(3), 580--582

    1942

  11. [11]

    and Pocock, S

    Frison, L. and Pocock, S. J. (1992). Repeated measures in clinical trials: analysis using mean summary statistics and its implications for design. Statistics in Medicine , 11(13), 1685--1704

    1992

  12. [12]

    Gy \"o rfi, L., Kohler, M., Krzy \.z ak, A., and Walk, H. (2002). A distribution-free theory of nonparametric regression , Springer, New York

    2002

  13. [13]

    R., Rice, J

    Hoover, D. R., Rice, J. A., Wu, C. O., and Yang, L.-P. (1998). Nonparametric smoothing estimates of time-varying coefficient models with longitudinal data. Biometrika , 85(4), 809--822

    1998

  14. [14]

    and Sampson, A

    Iosif, A.-M. and Sampson, A. R. (2014). A model for repeated clustered data with informative cluster sizes. Statistics in Medicine , 33(5), 738--759

    2014

  15. [15]

    Jiao, Y., Shen, G., Lin, Y., and Huang, J. (2023). Deep nonparametric regression on approximate manifolds: Nonasymptotic error bounds with polynomial prefactors. The Annals of Statistics , 51(2), 691--716

    2023

  16. [16]

    and Langer, S

    Kohler, M. and Langer, S. (2021). On the rate of convergence of fully connected deep neural network regression estimates. The Annals of Statistics , 49(4), 2231--2249

    2021

  17. [17]

    and Carroll, R

    Lin, X. and Carroll, R. J. (2000). Nonparametric function estimation for clustered data when the predictor is measured without/with error. Journal of the American Statistical Association , 95(450), 520--534

    2000

  18. [18]

    Lipton, Z., Wang, Y.-X., and Smola, A. (2018). Detecting and correcting for label shift with black box predictors, in `International Conference on Machine Learning', PMLR, pp. 3122--3130

    2018

  19. [19]

    Liu, J., Liu, X., Lian, H., and Xu, W. (2026). Fixed effects bayesian testing in high-dimensional linear mixed models. Scandinavian Journal of Statistics , 53(1), 442--481

    2026

  20. [20]

    Lv, Y., Zhu, X., Zhu, Z., and Qu, A. (2020). Nonparametric cluster analysis on multiple outcomes of longitudinal data. Statistica Sinica , 30(4), 1829--1856

    2020

  21. [21]

    Ma, C., Pathak, R., and Wainwright, M. J. (2023). Optimally tackling covariate shift in RKHS -based nonparametric regression. The Annals of Statistics , 51(2), 738--761

    2023

  22. [22]

    A., Cubuk, E

    Oliver, A., Odena, A., Raffel, C. A., Cubuk, E. D., and Goodfellow, I. (2018). Realistic evaluation of deep semi-supervised learning algorithms, in `Advances in Neural Information Processing Systems', Vol. 31, Curran Associates, Inc

    2018

  23. [23]

    and Voigtlaender, F

    Petersen, P. and Voigtlaender, F. (2018). Optimal approximation of piecewise smooth functions using deep ReLU neural networks. Neural Networks , 108, 296--330

    2018

  24. [24]

    Schmidt-Hieber, J. (2020). Nonparametric regression using deep neural networks with ReLU activation function. The Annals of Statistics , 48(4), 1875--1897

    2020

  25. [25]

    Sugiyama, M., Nakajima, S., Kashima, H., Buenau, P., and Kawanabe, M. (2007). Direct importance estimation with model selection and its application to covariate shift adaptation, in `Advances in Neural Information Processing Systems', Vol. 20

    2007

  26. [26]

    Sugiyama, M., Suzuki, T., and Kanamori, T. (2012). Density-ratio matching under the bregman divergence: a unified framework of density-ratio estimation. Annals of the Institute of Statistical Mathematics , 64(5), 1009--1044

    2012

  27. [27]

    Wang, N. (2003). Marginal nonparametric kernel regression accounting for within-subject correlation. Biometrika , 90(1), 43--52

    2003

  28. [28]

    Xu, K., Guo, W., Xiong, M., Zhu, L., and Jin, L. (2016). An estimating equation approach to dimension reduction for longitudinal data. Biometrika , 103(1), 189--203

    2016

  29. [29]

    Yan, S., Yao, F., and Zhou, H. (2025). Deep regression for repeated measurements. Journal of the American Statistical Association , 120(552), 2461--2472

    2025

  30. [30]

    Yang, C., Cheung, Y.-M., Ding, J., and Tan, K. C. (2022). Concept drift-tolerant transfer learning in dynamic environments. IEEE Transactions on Neural Networks and Learning Systems , 33(8), 3857--3871

    2022

  31. [31]

    Yarotsky, D. (2017). Error bounds for approximations with deep ReLU networks. Neural Networks , 94, 103--114

    2017

  32. [32]

    and Chen, S

    Zheng, X. and Chen, S. X. (2024). Dynamic synthetic control method for evaluating treatment effects in auto-regressive processes. Journal of the Royal Statistical Society Series B: Statistical Methodology , 86(1), 155--176

    2024