pith. sign in

arxiv: 1907.08074 · v1 · pith:Z3KILIZMnew · submitted 2019-07-18 · 📊 stat.ME · math.ST· stat.TH

Scalar-on-function local linear regression and beyond

Fr\'ed\'eric Ferraty , Stanislav Nagy This is my paper

Pith reviewed 2026-05-24 19:50 UTC · model grok-4.3

classification 📊 stat.ME math.STstat.TH
keywords scalar-on-function regressionlocal linear regressionfunctional derivativenonparametric estimationprojection approachconsistencysingle-functional index model
0
0 comments X

The pith

Local linear regression via projection outperforms local constant regression for scalar responses on functional predictors and yields a consistent estimator of the functional derivative.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces a projection-based local linear estimator for regressing a scalar on a random function in the nonparametric setting. Asymptotic analysis shows this estimator improves on the local constant version, and the same construction produces a consistent estimator for the derivative of the regression operator with respect to the functional argument. The derivative estimator is then applied to fit single-functional index models on real data, while simulations demonstrate reliable finite-sample behavior. A reader cares because accurate operator estimation and direct access to its derivative enable both better prediction and new modeling strategies for functional inputs.

Core claim

The paper establishes that the functional local linear regression based on a projection approach outperforms its local constant counterpart asymptotically, while the local linear estimator of the functional derivative is consistent. On simulated data both estimators exhibit good finite-sample properties, and the derivative estimator supplies an original fast method for fitting single-functional index models on real data.

What carries the argument

The projection-based functional local linear regression estimator, which simultaneously targets the regression operator and its functional derivative.

If this is right

  • The local linear estimator has superior asymptotic performance relative to local constant regression.
  • The estimator of the functional derivative is consistent.
  • The derivative estimator yields a fast method for single-functional index models.
  • Both estimators display good finite-sample properties on simulated data.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The derivative estimator could be inserted into other functional regression models that rely on index structures.
  • Bandwidth rules derived for this estimator might transfer to related functional nonparametric problems.
  • Direct access to the derivative opens routes to sensitivity analysis or gradient-based optimization over functional predictors.

Load-bearing premise

The projection approach together with standard nonparametric smoothness conditions on the regression operator, kernel, and bandwidth rates must hold for the asymptotic expansions and consistency to go through.

What would settle it

A dataset or simulation satisfying the paper's smoothness and bandwidth conditions in which the local linear estimator exhibits larger mean squared error than the local constant estimator.

Figures

Figures reproduced from arXiv: 1907.08074 by Fr\'ed\'eric Ferraty, Stanislav Nagy.

Figure 1
Figure 1. Figure 1: Mean (squares or triangles) of ORMSEPreg (each time over 100 runs) according to different noise-to-signal ratios (nsr), learning sample sizes and band￾width selection methods (CV/AICC). (a) worst case (b) most favorable case ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● … view at source ↗
Figure 2
Figure 2. Figure 2: (a) n = 100 and nsr = 0.4, (b) n = 500 and nsr = 0.05. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Mean (squares or triangles) of ORMSEPderiv (each time over 100 runs) according to different noise-to-signal ratios (nsr), learning sample sizes and band￾width selection methods (CV or AICC) used for computing mb ; (a) md0 Xi is built with hderiv, (b) md0 Xi is built with hreg. Smaller values of nsr correspond to lower ORMSEPderiv. (a) worst case (b) most favorable case 0.0 0.4 0.8 −3 −1 1 2 3 0.0 0.4 0.8 −… view at source ↗
Figure 4
Figure 4. Figure 4: Functional derivatives m0 Xi (solid lines) and their predictions md0 Xi with hderiv (dashed lines). 15 [PITH_FULL_IMAGE:figures/full_fig_p015_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The oracle bandwidths h oracle deriv versus hderiv (solid circles), and versus hreg (squares). On the horizontal axis, the averages of the oracle bandwidths h oracle deriv are displayed. additional randomness into the local linear estimation of the functional derivatives. However, [PITH_FULL_IMAGE:figures/full_fig_p016_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Berkeley growth dataset: (a) growth velocity profiles; (b) functional [PITH_FULL_IMAGE:figures/full_fig_p022_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Berkeley growth dataset: (a) link function [PITH_FULL_IMAGE:figures/full_fig_p023_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Estimates based on the 1–10 growth velocity profiles (black points) and [PITH_FULL_IMAGE:figures/full_fig_p050_8.png] view at source ↗
read the original abstract

Regressing a scalar response on a random function is nowadays a common situation. In the nonparametric setting, this paper paves the way for making the local linear regression based on a projection approach a prominent method for solving this regression problem. Our asymptotic results demonstrate that the functional local linear regression outperforms its functional local constant counterpart. Beyond the estimation of the regression operator itself, the local linear regression is also a useful tool for predicting the functional derivative of the regression operator, a promising mathematical object on its own. The local linear estimator of the functional derivative is shown to be consistent. On simulated datasets we illustrate good finite sample properties of both proposed methods. On a real data example of a single-functional index model we indicate how the functional derivative of the regression operator provides an original and fast, widely applicable estimating method.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes a projection-based local linear regression estimator for scalar-on-function nonparametric regression. It derives asymptotic results showing that this estimator outperforms the corresponding local constant estimator in terms of bias order, establishes consistency of the local linear estimator for the functional derivative of the regression operator, and reports good finite-sample performance on simulated data together with an application to estimating a single-functional index model on real data.

Significance. If the asymptotic claims hold under the invoked nonparametric regularity conditions (smoothness of the regression operator, kernel properties, and bandwidth rates), the work supplies a direct functional-data extension of the well-known bias-reduction property of local linear regression and supplies a consistent estimator for the functional derivative, which is useful for index-model estimation. The simulation study and real-data illustration provide supporting evidence of practical applicability.

major comments (2)
  1. [§3] §3 (asymptotic results): the claimed superiority of the local linear estimator over local constant regression is stated to follow from standard bias-order arguments, but the manuscript does not explicitly quantify how the additional projection step interacts with the functional smoothness assumptions; a short expansion showing that the projection error does not cancel the bias improvement would strengthen the central claim.
  2. [Theorem on derivative consistency] Theorem on derivative consistency (near §4): the consistency proof for the functional-derivative estimator invokes bandwidth rates that are stricter than those needed for the regression operator itself; it is not shown whether these rates remain compatible with the data-driven bandwidth selection used in the simulations.
minor comments (2)
  1. The choice of projection dimension (or basis truncation) is described only qualitatively; an explicit data-driven rule or sensitivity analysis would improve reproducibility.
  2. Notation for the functional derivative (e.g., the symbol used for the Fréchet derivative) should be introduced once and used consistently across the theoretical and empirical sections.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the positive evaluation and the recommendation for minor revision. We address the two major comments below and will incorporate the suggested clarifications into the revised manuscript.

read point-by-point responses

  1. Referee: [§3] §3 (asymptotic results): the claimed superiority of the local linear estimator over local constant regression is stated to follow from standard bias-order arguments, but the manuscript does not explicitly quantify how the additional projection step interacts with the functional smoothness assumptions; a short expansion showing that the projection error does not cancel the bias improvement would strengthen the central claim.

    Authors: We agree that an explicit quantification would strengthen the presentation. In the revision we will add a short paragraph in §3 showing that, under the maintained smoothness assumptions on the regression operator and the kernel and projection properties already stated in the paper, the additional projection error term is of strictly smaller order than the leading bias term and therefore preserves the bias-order improvement of the local-linear estimator relative to the local-constant estimator. revision: yes

  2. Referee: [Theorem on derivative consistency] Theorem on derivative consistency (near §4): the consistency proof for the functional-derivative estimator invokes bandwidth rates that are stricter than those needed for the regression operator itself; it is not shown whether these rates remain compatible with the data-driven bandwidth selection used in the simulations.

    Authors: The referee is correct that the bandwidth conditions required for derivative consistency are stricter. In the simulations we use cross-validation, which is known to attain the optimal rates (up to logarithmic factors) under the smoothness and design conditions already assumed. In the revision we will insert a brief remark after the theorem stating that the data-driven bandwidths satisfy the stricter conditions with probability approaching one, thereby ensuring compatibility between the theoretical rates and the bandwidths employed in the numerical study. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper's central claims rest on standard nonparametric asymptotic analysis under regularity conditions (smoothness of the regression operator, kernel and bandwidth assumptions) applied to a projection-based local linear estimator. These conditions are invoked as external inputs to derive bias/variance improvements and consistency results for both the regression operator and its functional derivative; no equations or steps reduce by construction to fitted parameters, self-definitions, or load-bearing self-citations. The derivation chain is self-contained against external benchmarks and does not exhibit any of the enumerated circularity patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract supplies no information on free parameters, axioms, or invented entities; full manuscript required to audit modeling assumptions or bandwidth choices.

pith-pipeline@v0.9.0 · 5661 in / 1072 out tokens · 30862 ms · 2026-05-24T19:50:24.411709+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

57 extracted references · 57 canonical work pages

  1. [1]

    Ferraty, R

    Ait-Sa¨ ıdi, A., F. Ferraty, R. Kassa, and P. Vieu (2008). Cross-validated esti- mations in the single-functional index model. Statistics 42 (6), 475–494

    work page 2008

  2. [2]

    Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In Second International Symposium on Information Theory (Tsahkadsor, 1971), pp. 267–281. Akad´ emiai Kiad´ o, Budapest

    work page 1973

  3. [3]

    Allen, D. M. (1974). The relationship between variable selection and data augmentation and a method for prediction. Technometrics 16, 125–127. 53

    work page 1974

  4. [4]

    Antoniadis, and I

    Amato, U., A. Antoniadis, and I. De Feis (2006). Dimension reduction in functional regression with applications. Comput. Statist. Data Anal. 50 (9), 2422–2446

    work page 2006

  5. [5]

    Ba´ ıllo, A. and A. Gran´ e (2009). Local linear regression for functional predictor and scalar response. J. Multivariate Anal. 100 (1), 102–111

    work page 2009

  6. [6]

    Elamine, and A

    Berlinet, A., A. Elamine, and A. Mas (2011). Local linear regression for func- tional data. Ann. Inst. Statist. Math. 63 (5), 1047–1075

    work page 2011

  7. [7]

    Bosq, D. (2000). Linear processes in function spaces , Volume 149 of Lecture Notes in Statistics . Springer-Verlag, New York

    work page 2000

  8. [8]

    Bourbaki, N. (2004). Integration I. Chapters 1–6 . Elements of Mathematics (Berlin). Springer-Verlag, Berlin

    work page 2004

  9. [9]

    Cai, T. T. and P. Hall (2006). Prediction in functional linear regression. Ann. Statist. 34 (5), 2159–2179

    work page 2006

  10. [10]

    Ferraty, and P

    Cardot, H., F. Ferraty, and P. Sarda (1999). Functional linear model. Statist. Probab. Lett. 45 (1), 11–22

    work page 1999

  11. [11]

    Hall, and H.-G

    Chen, D., P. Hall, and H.-G. M¨ uller (2011). Single and multiple index func- tional regression models with nonparametric link. Ann. Statist. 39 (3), 1720– 1747

    work page 2011

  12. [12]

    Fan, and J

    Cheng, M.-Y., J. Fan, and J. S. Marron (1997). On automatic boundary corrections. Ann. Statist. 25 (4), 1691–1708

    work page 1997

  13. [13]

    Kneip, and P

    Crambes, C., A. Kneip, and P. Sarda (2009). Smoothing splines estimators for functional linear regression. Ann. Statist. 37 (1), 35–72

    work page 2009

  14. [14]

    Pousse, and Y

    Dauxois, J., A. Pousse, and Y. Romain (1982). Asymptotic theory for the principal component analysis of a vector random function: some applications to statistical inference. J. Multivariate Anal. 12 (1), 136–154

    work page 1982

  15. [15]

    de Boor, C. (1978). A practical guide to splines , Volume 27 of Applied Math- ematical Sciences. Springer-Verlag, New York-Berlin

    work page 1978

  16. [16]

    Fan, J. (1992). Design-adaptive nonparametric regression. J. Amer. Statist. Assoc. 87 (420), 998–1004

    work page 1992

  17. [17]

    Fan, J. (1993). Local linear regression smoothers and their minimax efficien- cies. Ann. Statist. 21 (1), 196–216. 54

    work page 1993

  18. [18]

    Fan, J. and I. Gijbels (1992). Variable bandwidth and local linear regression smoothers. Ann. Statist. 20 (4), 2008–2036

    work page 1992

  19. [19]

    Fan, J. and I. Gijbels (1996). Local polynomial modelling and its applications, Volume 66 of Monographs on Statistics and Applied Probability . Chapman & Hall, London

    work page 1996

  20. [20]

    Mas, and P

    Ferraty, F., A. Mas, and P. Vieu (2007). Nonparametric regression on func- tional data: inference and practical aspects. Aust. N. Z. J. Stat. 49 (3), 267–286

    work page 2007

  21. [21]

    Van Keilegom, and P

    Ferraty, F., I. Van Keilegom, and P. Vieu (2010). On the validity of the bootstrap in non-parametric functional regression. Scand. J. Stat. 37 (2), 286– 306

    work page 2010

  22. [22]

    Ferraty, F. and P. Vieu (2006). Nonparametric functional data analysis . Springer Series in Statistics. Springer, New York

    work page 2006

  23. [23]

    K¨ ohler, H.-G

    Gasser, T., W. K¨ ohler, H.-G. M¨ uller, A. Kneip, R. Largo, L. Molinari, and A. Prader (1984). Velocity and acceleration of height growth using kernel esti- mation. Ann. Hum. Biol. 11 (5), 397–411

    work page 1984

  24. [24]

    Gervini, D. and T. Gasser (2005). Nonparametric maximum likelihood esti- mation of the structural mean of a sample of curves.Biometrika 92 (4), 801–820

    work page 2005

  25. [25]

    Hall, P. and M. Hosseini-Nasab (2006). On properties of functional principal components analysis. J. R. Stat. Soc. Ser. B Stat. Methodol. 68 (1), 109–126

    work page 2006

  26. [26]

    Hall, P. and J. S. Marron (1997). On the role of the shrinkage parameter in local linear smoothing. Probab. Theory Related Fields 108 (4), 495–516

    work page 1997

  27. [27]

    M¨ uller, and J.-L

    Hall, P., H.-G. M¨ uller, and J.-L. Wang (2006). Properties of principal compo- nent methods for functional and longitudinal data analysis. Ann. Statist. 34 (3), 1493–1517

    work page 2006

  28. [28]

    M¨ uller, and F

    Hall, P., H.-G. M¨ uller, and F. Yao (2009). Estimation of functional derivatives. Ann. Statist. 37 (6A), 3307–3329

    work page 2009

  29. [29]

    H¨ ardle, W. and T. M. Stoker (1989). Investigating smooth multiple regression by the method of average derivatives.J. Amer. Statist. Assoc. 84(408), 986–995

    work page 1989

  30. [30]

    Tibshirani, and J

    Hastie, T., R. Tibshirani, and J. Friedman (2001). The elements of statistical learning. Springer Series in Statistics. Springer-Verlag, New York

    work page 2001

  31. [31]

    Hsing, T. and R. Eubank (2015). Theoretical foundations of functional data analysis, with an introduction to linear operators . Wiley Series in Probability and Statistics. John Wiley & Sons, Ltd., Chichester. 55

    work page 2015

  32. [32]

    Hurvich, C. M., J. S. Simonoff, and C.-L. Tsai (1998). Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion. J. R. Stat. Soc. Ser. B Stat. Methodol. 60 (2), 271–293

    work page 1998

  33. [33]

    James, G. M. (2002). Generalized linear models with functional predictors. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 (3), 411–432

    work page 2002

  34. [34]

    and J.-L

    Jiang, C.-R. and J.-L. Wang (2011). Functional single index models for lon- gitudinal data. Ann. Statist. 39 (1), 362–388

    work page 2011

  35. [35]

    Karhunen, K. (1946). Zur Spektraltheorie stochastischer Prozesse. Ann. Acad. Sci. Fennicae. Ser. A. I. Math.-Phys. 34 , 1–7

    work page 1946

  36. [36]

    Kneip, A. and T. Gasser (1992). Statistical tools to analyze data representing a sample of curves. Ann. Statist. 20 (3), 1266–1305

    work page 1992

  37. [37]

    Kokoszka, P. and M. Reimherr (2017). Introduction to functional data anal- ysis. Texts in Statistical Science Series. CRC Press, Boca Raton, FL

    work page 2017

  38. [38]

    Lo` eve, M. (1946). Fonctions al´ eatoires ` a d´ ecomposition orthogonale exponen- tielle. Revue Sci. 84, 159–162

    work page 1946

  39. [39]

    and S.-H

    Lu, T.-T. and S.-H. Shiou (2002). Inverses of 2 × 2 block matrices. Comput. Math. Appl. 43 (1-2), 119–129

    work page 2002

  40. [40]

    Mammen, E. (1993). Bootstrap and wild bootstrap for high-dimensional linear models. Ann. Statist. 21 (1), 255–285

    work page 1993

  41. [41]

    M¨ uller, H.-G. and U. Stadtm¨ uller (2005). Generalized functional linear mod- els. Ann. Statist. 33 (2), 774–805

    work page 2005

  42. [42]

    M¨ uller, H.-G. and F. Yao (2008). Functional additive models. J. Amer. Statist. Assoc. 103 (484), 1534–1544

    work page 2008

  43. [43]

    M¨ uller, H.-G. and F. Yao (2010). Additive modelling of functional gradients. Biometrika 97 (4), 791–805

    work page 2010

  44. [44]

    R: A language and environment for statistical comput- ing

    R Core Team (2018). R: A language and environment for statistical comput- ing. Vienna, Austria: R Foundation for Statistical Computing

    work page 2018

  45. [45]

    Ramsay, J. O. and X. Li (1998). Curve registration. J. R. Stat. Soc. Ser. B Stat. Methodol. 60 (2), 351–363

    work page 1998

  46. [46]

    Ramsay, J. O. and B. W. Silverman (2002). Applied functional data analysis . Springer Series in Statistics. Springer-Verlag, New York. 56

    work page 2002

  47. [47]

    Ramsay, J. O. and B. W. Silverman (2005). Functional data analysis (Second ed.). Springer Series in Statistics. Springer, New York

    work page 2005

  48. [48]

    Rao, C. R. (1958). Some statistical methods for comparison of growth curves. Biometrics 14 (1), 1–17

    work page 1958

  49. [49]

    Redd, A. (2012). A comment on the orthogonalization of B-spline basis func- tions and their derivatives. Stat. Comput. 22 (1), 251–257

    work page 2012

  50. [50]

    Reiss, P. T. and R. T. Ogden (2007). Functional principal component regres- sion and functional partial least squares. J. Amer. Statist. Assoc. 102 (479), 984–996

    work page 2007

  51. [51]

    Ruppert, D. and M. P. Wand (1994). Multivariate locally weighted least squares regression. Ann. Statist. 22 (3), 1346–1370

    work page 1994

  52. [52]

    Stone, M. (1974). Cross-validatory choice and assessment of statistical pre- dictions. J. Roy. Statist. Soc. Ser. B 36 , 111–147

    work page 1974

  53. [53]

    Stone, M. (1977). An asymptotic equivalence of choice of model by cross- validation and Akaike’s criterion. J. Roy. Statist. Soc. Ser. B 39 (1), 44–47

    work page 1977

  54. [54]

    Tuddenham, R. D. and M. M. Snyder (1954). Physical growth of California boys and girls from birth to eighteen years. Publications in child development. University of California, Berkeley 1 (2), 183–364

    work page 1954

  55. [55]

    Wu, C.-F. J. (1986). Jackknife, bootstrap and other resampling methods in regression analysis. Ann. Statist. 14 (4), 1261–1350

    work page 1986

  56. [56]

    M¨ uller, and J.-L

    Yao, F., H.-G. M¨ uller, and J.-L. Wang (2005). Functional data analysis for sparse longitudinal data. J. Amer. Statist. Assoc. 100 (470), 577–590

    work page 2005

  57. [57]

    Zygmund, A. (2002). Trigonometric series. Vol. I, II (Third ed.). Cambridge Mathematical Library. Cambridge University Press, Cambridge. 57

    work page 2002