Transfer Learning for Moderate-Dimensional Ridge-Regularized Robust Linear Regression

Lingfeng Lyu; Xiao Guo; Zongqi Liu

arxiv: 2603.29575 · v2 · submitted 2026-03-31 · 📊 stat.ME

Transfer Learning for Moderate-Dimensional Ridge-Regularized Robust Linear Regression

Lingfeng Lyu , Xiao Guo , Zongqi Liu This is my paper

Pith reviewed 2026-05-13 23:37 UTC · model grok-4.3

classification 📊 stat.ME

keywords transfer learningridge regressionrobust estimationmoderate dimensionsasymptotic risklinear modelsestimation error

0 comments

The pith

Leveraging source data substantially improves asymptotic estimation accuracy for ridge-regularized robust linear regression in moderate dimensions.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops Trans-RR, a transfer learning procedure that merges a robust ridge estimator fitted on source data with a correction term computed from the target study. In the moderate-dimensional regime where the number of predictors is comparable to sample size and coefficients need not be sparse, the method yields a lower asymptotic estimation error than the conventional single-study robust ridge estimator. A sympathetic reader would care because many applied regression tasks have scarce target observations yet can draw on related source studies, and the approach delivers measurable gains without extra assumptions on sparsity. The analysis explicitly tracks how the size of the discrepancy between source and target controls whether transfer helps or harms performance. Simulations and one real-data example illustrate both the improvement and the boundary cases.

Core claim

Trans-RR combines a robust ridge estimator from the source study with a robust ridge correction based on the target study; under mild assumptions on the data-generating processes and the source-target discrepancy, its asymptotic estimation error is strictly smaller than that of the single-study ridge-regularized robust estimator.

What carries the argument

The Trans-RR estimator, obtained by fusing the source robust ridge estimator with a target-based correction term, whose asymptotic risk is derived in closed form.

If this is right

Estimation error decreases whenever the source and target distributions are sufficiently close.
The improvement holds without any sparsity assumption on the coefficient vector.
Negative transfer occurs once the source-target discrepancy exceeds a threshold characterized in the asymptotics.
The same qualitative gains appear in both simulations and a real-data example.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same fusion idea could be applied to other M-estimators whose asymptotic expansions are available.
When multiple source studies are present, a weighted combination of their estimators might further reduce risk.
In practice, a discrepancy diagnostic could be used to decide whether to invoke transfer or fall back to the target-only estimator.

Load-bearing premise

The source and target data follow linear models whose moments are uniformly bounded and whose distributional discrepancy satisfies a mild quantitative bound.

What would settle it

A simulation or data set in which the source and target distributions differ only modestly yet the finite-sample mean squared error of Trans-RR exceeds that of the single-study estimator would refute the claimed improvement.

Figures

Figures reproduced from arXiv: 2603.29575 by Lingfeng Lyu, Xiao Guo, Zongqi Liu.

**Figure 1.** Figure 1: presents boxplots of the estimation error ∥βb −β0∥ 2 for cases I–III and κ = 1, 4. The red point in each boxplot marks the theoretical value r 2 ρ , obtained by numerically solving the system in Theorem 1 under the corresponding simulation specification. We observe that the empirical distribution of ∥βb − β0∥ 2 is centered close to this value, and its dispersion decreases as n and p become larger [PITH_FU… view at source ↗

**Figure 2.** Figure 2: Theoretical curves of rρ as a function of ∥β0 − wb ∥ for five values of τ under cases I–III, obtained by numerically solving Corollary 1. The three panels correspond to cases I, II, and III, respectively. 4.3 Comparison with existing methods To evaluate when transfer is beneficial, we compare our method with several competing procedures across the three scenarios described above. We set p = 400, n = p and … view at source ↗

**Figure 3.** Figure 3: Boxplots of relative estimation errors (log scale) across 1000 replications for [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗

read the original abstract

This paper studies transfer learning for ridge-regularized robust linear regression in the moderate-dimensional regime, where the number of predictors is of the same order as the sample size and the regression coefficients are not assumed to be sparse. We propose Trans-RR, which combines a robust ridge estimator from a source study with a robust ridge correction based on the target study. Under mild assumptions, we characterize the asymptotic estimation error of the proposed estimator and show that leveraging source data can substantially improve estimation accuracy relative to the traditional single-study ridge-regularized robust estimator. Simulation results and a real-data analysis support the theory and illustrate both positive and negative transfer as the discrepancy between the source and target studies varies.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

Trans-RR gives a workable way to blend source and target data for robust ridge regression in the moderate-dimensional case, with asymptotics that track when the transfer helps.

read the letter

The paper's main move is to define Trans-RR as a combination of a robust ridge estimator fitted on the source data and a correction term computed on the target data. This targets the setting where p is on the order of n and the coefficients are not sparse. They derive the asymptotic estimation error under assumptions on the data-generating process and the source-target discrepancy, then show that the combined estimator can beat the usual single-study robust ridge when the studies are close enough. Simulations and a real-data example are used to illustrate both positive and negative transfer as the discrepancy changes. That combination of explicit construction, asymptotic characterization, and balanced empirical checks is the useful part. The random-matrix approach to the error formula is standard for this regime and appears to be applied cleanly here. One soft spot is that the assumptions on the discrepancy and the robust loss function are labeled mild but still need to hold for the improvement to appear; if the source and target differ more than the theory allows, the gain disappears. The finite-sample behavior is only shown through simulations, so the practical margin of improvement could be smaller than the asymptotics suggest. The real-data analysis helps but is not detailed enough in the abstract to judge how representative it is. This paper is aimed at statisticians who already work with robust regression and have access to auxiliary datasets. A reader looking for concrete transfer methods in the non-sparse moderate-dimensional regime would find the estimator and the error formula worth examining. It deserves a serious referee because the contribution is focused, the theory is stated, and the empirical section checks both directions of transfer rather than only the favorable case.

Referee Report

1 major / 2 minor

Summary. The manuscript proposes Trans-RR, a transfer-learning estimator for ridge-regularized robust linear regression in the moderate-dimensional regime where p is of the same order as n and coefficients are not assumed sparse. It combines a robust ridge estimator fitted on source data with a correction term based on target data. Under mild assumptions on the data-generating process and source-target discrepancy, the paper derives an asymptotic characterization of the estimation error of Trans-RR and shows that source data can substantially reduce error relative to the single-study ridge-robust estimator. Simulations and a real-data example illustrate both positive and negative transfer as discrepancy varies.

Significance. If the asymptotic characterization holds, the work provides a concrete extension of random-matrix techniques to transfer learning for robust regression without sparsity, yielding explicit error expressions that clarify when auxiliary data helps. This is useful in moderate-dimensional settings common in statistics and applications where robust estimation matters and target samples are limited.

major comments (1)

[§3] §3 (asymptotic analysis): the characterization of the asymptotic error for Trans-RR relies on a fixed-point equation whose uniqueness and stability under the stated mild assumptions on source-target discrepancy should be verified explicitly; without this, the claimed improvement over the single-study estimator may not hold uniformly.

minor comments (2)

[Simulations] The simulation section would benefit from reporting standard errors or confidence bands on the plotted estimation errors to allow visual assessment of variability across replications.
[Model and Notation] Notation for the source and target sample sizes (n_s, n_t) and dimensions (p) should be introduced consistently in the model section before being used in the asymptotic statements.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the positive assessment of our manuscript and for the constructive comment on the asymptotic analysis. We address the major comment below and will incorporate the suggested verification in the revision.

read point-by-point responses

Referee: [§3] §3 (asymptotic analysis): the characterization of the asymptotic error for Trans-RR relies on a fixed-point equation whose uniqueness and stability under the stated mild assumptions on source-target discrepancy should be verified explicitly; without this, the claimed improvement over the single-study estimator may not hold uniformly.

Authors: We appreciate this observation. The fixed-point equation arises from the random-matrix characterization of the ridge-robust estimator and the transfer correction. Under the paper's mild assumptions (bounded moments, sub-Gaussian noise, and a discrepancy parameter bounded away from the critical threshold), the map defining the fixed point is a contraction mapping on a compact set. In the revised manuscript we will add an explicit lemma (new Lemma 3.3) proving uniqueness and local stability of the solution by verifying the Lipschitz constant is strictly less than one when the source-target discrepancy satisfies the stated bound. This directly confirms that the asymptotic improvement over the single-study estimator holds uniformly in the regime considered. The proof is a standard contraction argument and does not alter any other results. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected in derivation chain

full rationale

The paper derives an asymptotic characterization of the estimation error for Trans-RR under mild assumptions on the data-generating process and source-target discrepancy using standard random-matrix techniques in the moderate-dimensional regime. This fixed-point characterization is obtained directly from the model equations and is not reduced by construction to any fitted parameter, self-defined quantity, or self-citation chain. The claimed improvement over the single-study ridge-robust estimator follows by direct comparison of the two derived expressions. Simulations and real-data analysis serve as external illustration rather than the logical basis of the result. No load-bearing step matches any of the enumerated circularity patterns; the derivation remains self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review prevents identification of concrete free parameters or axioms; the method presumably relies on standard ridge penalty choice and moment conditions typical for robust regression asymptotics.

pith-pipeline@v0.9.0 · 5406 in / 1059 out tokens · 48600 ms · 2026-05-13T23:37:48.312912+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

13 extracted references · 13 canonical work pages

[1]

Bastani, H. (2021). Predicting with proxies: Transfer learning in high dimension.Manage- ment Science 67, 2964–2984. Bhatia, R. (1997).Matrix Analysis. New York: Springer. Cai, T. T. and H. Pu (2024). Transfer learning for nonparametric regression: Non- asymptotic minimax analysis and adaptive procedure.arXiv preprint arXiv:2401.12272. Cai, T. T. and H. W...

work page arXiv 2021
[2]

Li, S., T

American Mathematical Soc. Li, S., T. T. Cai, and H. Li (2022). Transfer learning for high-dimensional linear regression: Prediction, estimation and minimax optimality.Journal of the Royal Statistical Society Series B: Statistical Methodology 84, 149–173. 21 Liu, D., J. Luo, B. Johnson, H. Chew, J. Blais, A. Deik, F. Paul, R. L. Hanson, J. P. Cran- dall, ...

work page 2022
[3]

Portnoy, S. (1987). A central limit theorem applicable to robust regression estimators. Journal of Multivariate Analysis 22, 24–50. Sai Li, Linjun Zhang, T. T. C. and H. Li (2024). Estimation and inference for high- dimensional generalized linear models with knowledge transfer.Journal of the American Statistical Association 119, 1274–1285. Stroock, D. W. ...

work page 1987
[4]

Furthermore, L(n)∥ψ∥ ∞ = O(1)

•F2.∥ψ∥ ∞ = O(1).ψ ′ has Lipschitz constant L(n). Furthermore, L(n)∥ψ∥ ∞ = O(1). •F3.α <1/6 andα+ 1/3<2 min(1/2,e). •F4.there exists constantCsuch that E(λ 4 i )≤C. •F5.λ i’s may have different distributions. The fraction of occurrences for each pos- sible combination of distributions for (ϵ i, λi) has a limit asn→ ∞. S.2 Proof for Theorem 1 We call F(δ) ...

work page 2018
[5]

S.2.2 On∥ bδ∥and∥ bδ−δ 0∥ Lemma S.3.Defineq n(b) =n −1Pn i=1 xiψ{eϵi +x ⊤ i b},q n ∈R p. IfD ψ is then×ndiagonal matrix with(i, i)-entryψ{eϵ i +x ⊤ i δ0}, ∥bδ∥ ≤ 1 τ ∥qn(δ0)∥= 1 τ r 1 n21⊤DψXX ⊤Dψ1, and ifD ψ(ξi) is then×ndiagonal matrix with(i, i)-entryψ(eϵ i), ∥bδ−δ 0∥ ≤ ∥δ 0∥+ 1 τ ∥qn(0)∥=∥δ 0∥+ 1 τ r 1 n21⊤Dψ(ξi)XX ⊤Dψ(ξi)1, Also, ∥qn(δ0)∥2 ≤ 1⊤D2 ψ1 ...

work page 2018
[6]

Applying Lemma S.2 we have ∥bδ− eδi∥ ≤ 1 τ ∥Ri∥

Therefore,f( eδi) =R i. Applying Lemma S.2 we have ∥bδ− eδi∥ ≤ 1 τ ∥Ri∥. i. OnR i Next, we provide a bound forR i. Lemma S.4.We have ∥ηi∥ ≤ 1√nτ ∥xi∥√n |ψ(eri,(i))|, and ∥Ri∥ ≤ ∥bΣ∥2 sup j̸=i ψ′ j{γ⋆(xj,bδ(i),η i)} −ψ ′ j(erj,(i)) 1√nτ ∥xi∥√n |ψ(eri,(i))|. Proof.We have Ri = 1 n X j̸=i [ψ′ j{γ⋆(xj,bδ(i),η i)} −ψ ′ j(erj,(i))]xjx⊤ j ηi. Note thatS=n −1P j̸...

work page 2013
[7]

Now our assumptionsO6concerning sup i |λi|= O Lk(polyLog(n)) guarantee that the bounds we announced are valid

Therefore, by AssumptionO4on c n we have sup j̸=i |∥Xj∥/√n|= O Lk(1). Now our assumptionsO6concerning sup i |λi|= O Lk(polyLog(n)) guarantee that the bounds we announced are valid. 36 Consequences We have the following result. Proposition S.3.Under AssumptionsO1-O6, we have ∥Ri∥= O Lk [L(n)]∥ψ∥2 ∞ nτ polyLog(n) . Furthermore, the same bound hold forsup 1≤...

work page 2018
[8]

The same type of results are true forvar(∥ bδ−δ 0∥2)andvar(∥ bβ−β 0∥2)provided that ∥δ0∥= O(polyLog(n))

More precisely, we have var(∥bδ∥2) = O polyLog(n) n1−2α . The same type of results are true forvar(∥ bδ−δ 0∥2)andvar(∥ bβ−β 0∥2)provided that ∥δ0∥= O(polyLog(n)). Proof.We use the Efron-Stein inequality (Efron and Stein, 1981): ifWis a function of nindependent random variables, andW (i) is any function of all those random variables except thei-th, var(W)≤...

work page 1981
[9]

39 Using the results of Theorem S.1 we have E |∥bδ−δ 0∥2 − ∥eδi −δ 0∥2|2 = O polyLog(n) n2−2α = o(n−1), provided thatα <1/2

= 2(bδ− eδi)⊤(bδ−δ 0)− ∥bδ− eδi∥2, by the Cauchy–Schwarz inequality we have |∥bδ−δ 0∥2 − ∥eδi −δ 0∥2|2 = OL1(∥bδ− eδi∥4) + q OL1(polyLog(n))∥bδ− eδi∥4, since E(∥bδ−δ 0∥k) exists and is bounded bykpolyLog(n)/τ k following from assumption O7and Lemma S.3. 39 Using the results of Theorem S.1 we have E |∥bδ−δ 0∥2 − ∥eδi −δ 0∥2|2 = O polyLog(n) n2−2α = o(n−1),...

work page 2018
[10]

We conclude that [f(eδ)]p =− 1 n nX i=1 xi(p)δi,p =− 1 n nX i=1 xi(p){ψ′(γ∗ i,p)−ψ ′(ri,[p])}x⊤ i (bγext −eb). iii. Representation off( eb) Aggregating all the results we have obtained so far, we see that f(eb) =− 1 n nX i=1 di,pxix⊤ i (bγext −eb) ={bp −δ 0(p) +bw(p)−w0(p)} n 1 n nX i=1 di,pxix⊤ i o   (Sp +τI) −1up −1   , (S.15) which implies (S.12). ...

work page 2018
[11]

Recall that bδ(i) is independent ofX i and that E(X i) =0, cov(X i) = I and that, for any finitek, the firstkmoments of its entries are bounded uniformly inn

is asymptoticallyN(0,E(∥ bβ− β0∥2)). Recall that bδ(i) is independent ofX i and that E(X i) =0, cov(X i) = I and that, for any finitek, the firstkmoments of its entries are bounded uniformly inn. We have shown that in Proposition S.4 that var(∥ bβ−β 0∥2)→0. In light of Lemma S.3, we also know that E(∥ bβ−β 0∥2) is uniformly bounded. Furthermore, in the pr...

work page 2010
[12]

Independence ofX i andX j gives E n ei(w1αi+w2αj)|X(ij),ϵ (ij) o = E eiw1αi|X(ij),ϵ (ij) E eiw2αj |X(ij),ϵ (ij)

63 Now, we have E n ei(w1αi+w2αj)1En (ij) o = E h 1En (ij) E n ei(w1αi+w2αj)|X(ij),ϵ (ij) oi , since 1En (ij) is a deterministic function of (X (ij),ϵ (ij)). Independence ofX i andX j gives E n ei(w1αi+w2αj)|X(ij),ϵ (ij) o = E eiw1αi|X(ij),ϵ (ij) E eiw2αj |X(ij),ϵ (ij) . Also, the conditional Gaussian approximation established above implies that 1En (ij) ...

work page 2000
[13]

Gathering all these results, we have |ψ′(Ri)−ψ ′(prox(λ2 i cτ ρ)(eri,(i)))|= O Lk{(∥ψ∥∞ + 1)n−1/2+3α polyLog(n)}

Using Lemma S.11, we can write |prox(c iρ)(eri,(i))−prox(λ 2 i cτ ρ)(eri,(i))| ≤ ∥ψ∥ ∞|ci −λ 2 i cτ | and hence |ψ′{prox(ciρ)(eri,(i))} −ψ ′{prox(λ2 i cτ ρ)(eri,(i))}|= O Lk(∥ψ∥∞n−1/2+3α polyLog(n)). Gathering all these results, we have |ψ′(Ri)−ψ ′(prox(λ2 i cτ ρ)(eri,(i)))|= O Lk{(∥ψ∥∞ + 1)n−1/2+3α polyLog(n)}. 69 So we have shown thatδ n(cτ) = OLk(n−1/2...

work page 2018

[1] [1]

Bastani, H. (2021). Predicting with proxies: Transfer learning in high dimension.Manage- ment Science 67, 2964–2984. Bhatia, R. (1997).Matrix Analysis. New York: Springer. Cai, T. T. and H. Pu (2024). Transfer learning for nonparametric regression: Non- asymptotic minimax analysis and adaptive procedure.arXiv preprint arXiv:2401.12272. Cai, T. T. and H. W...

work page arXiv 2021

[2] [2]

Li, S., T

American Mathematical Soc. Li, S., T. T. Cai, and H. Li (2022). Transfer learning for high-dimensional linear regression: Prediction, estimation and minimax optimality.Journal of the Royal Statistical Society Series B: Statistical Methodology 84, 149–173. 21 Liu, D., J. Luo, B. Johnson, H. Chew, J. Blais, A. Deik, F. Paul, R. L. Hanson, J. P. Cran- dall, ...

work page 2022

[3] [3]

Portnoy, S. (1987). A central limit theorem applicable to robust regression estimators. Journal of Multivariate Analysis 22, 24–50. Sai Li, Linjun Zhang, T. T. C. and H. Li (2024). Estimation and inference for high- dimensional generalized linear models with knowledge transfer.Journal of the American Statistical Association 119, 1274–1285. Stroock, D. W. ...

work page 1987

[4] [4]

Furthermore, L(n)∥ψ∥ ∞ = O(1)

•F2.∥ψ∥ ∞ = O(1).ψ ′ has Lipschitz constant L(n). Furthermore, L(n)∥ψ∥ ∞ = O(1). •F3.α <1/6 andα+ 1/3<2 min(1/2,e). •F4.there exists constantCsuch that E(λ 4 i )≤C. •F5.λ i’s may have different distributions. The fraction of occurrences for each pos- sible combination of distributions for (ϵ i, λi) has a limit asn→ ∞. S.2 Proof for Theorem 1 We call F(δ) ...

work page 2018

[5] [5]

S.2.2 On∥ bδ∥and∥ bδ−δ 0∥ Lemma S.3.Defineq n(b) =n −1Pn i=1 xiψ{eϵi +x ⊤ i b},q n ∈R p. IfD ψ is then×ndiagonal matrix with(i, i)-entryψ{eϵ i +x ⊤ i δ0}, ∥bδ∥ ≤ 1 τ ∥qn(δ0)∥= 1 τ r 1 n21⊤DψXX ⊤Dψ1, and ifD ψ(ξi) is then×ndiagonal matrix with(i, i)-entryψ(eϵ i), ∥bδ−δ 0∥ ≤ ∥δ 0∥+ 1 τ ∥qn(0)∥=∥δ 0∥+ 1 τ r 1 n21⊤Dψ(ξi)XX ⊤Dψ(ξi)1, Also, ∥qn(δ0)∥2 ≤ 1⊤D2 ψ1 ...

work page 2018

[6] [6]

Applying Lemma S.2 we have ∥bδ− eδi∥ ≤ 1 τ ∥Ri∥

Therefore,f( eδi) =R i. Applying Lemma S.2 we have ∥bδ− eδi∥ ≤ 1 τ ∥Ri∥. i. OnR i Next, we provide a bound forR i. Lemma S.4.We have ∥ηi∥ ≤ 1√nτ ∥xi∥√n |ψ(eri,(i))|, and ∥Ri∥ ≤ ∥bΣ∥2 sup j̸=i ψ′ j{γ⋆(xj,bδ(i),η i)} −ψ ′ j(erj,(i)) 1√nτ ∥xi∥√n |ψ(eri,(i))|. Proof.We have Ri = 1 n X j̸=i [ψ′ j{γ⋆(xj,bδ(i),η i)} −ψ ′ j(erj,(i))]xjx⊤ j ηi. Note thatS=n −1P j̸...

work page 2013

[7] [7]

Now our assumptionsO6concerning sup i |λi|= O Lk(polyLog(n)) guarantee that the bounds we announced are valid

Therefore, by AssumptionO4on c n we have sup j̸=i |∥Xj∥/√n|= O Lk(1). Now our assumptionsO6concerning sup i |λi|= O Lk(polyLog(n)) guarantee that the bounds we announced are valid. 36 Consequences We have the following result. Proposition S.3.Under AssumptionsO1-O6, we have ∥Ri∥= O Lk [L(n)]∥ψ∥2 ∞ nτ polyLog(n) . Furthermore, the same bound hold forsup 1≤...

work page 2018

[8] [8]

The same type of results are true forvar(∥ bδ−δ 0∥2)andvar(∥ bβ−β 0∥2)provided that ∥δ0∥= O(polyLog(n))

More precisely, we have var(∥bδ∥2) = O polyLog(n) n1−2α . The same type of results are true forvar(∥ bδ−δ 0∥2)andvar(∥ bβ−β 0∥2)provided that ∥δ0∥= O(polyLog(n)). Proof.We use the Efron-Stein inequality (Efron and Stein, 1981): ifWis a function of nindependent random variables, andW (i) is any function of all those random variables except thei-th, var(W)≤...

work page 1981

[9] [9]

39 Using the results of Theorem S.1 we have E |∥bδ−δ 0∥2 − ∥eδi −δ 0∥2|2 = O polyLog(n) n2−2α = o(n−1), provided thatα <1/2

= 2(bδ− eδi)⊤(bδ−δ 0)− ∥bδ− eδi∥2, by the Cauchy–Schwarz inequality we have |∥bδ−δ 0∥2 − ∥eδi −δ 0∥2|2 = OL1(∥bδ− eδi∥4) + q OL1(polyLog(n))∥bδ− eδi∥4, since E(∥bδ−δ 0∥k) exists and is bounded bykpolyLog(n)/τ k following from assumption O7and Lemma S.3. 39 Using the results of Theorem S.1 we have E |∥bδ−δ 0∥2 − ∥eδi −δ 0∥2|2 = O polyLog(n) n2−2α = o(n−1),...

work page 2018

[10] [10]

We conclude that [f(eδ)]p =− 1 n nX i=1 xi(p)δi,p =− 1 n nX i=1 xi(p){ψ′(γ∗ i,p)−ψ ′(ri,[p])}x⊤ i (bγext −eb). iii. Representation off( eb) Aggregating all the results we have obtained so far, we see that f(eb) =− 1 n nX i=1 di,pxix⊤ i (bγext −eb) ={bp −δ 0(p) +bw(p)−w0(p)} n 1 n nX i=1 di,pxix⊤ i o   (Sp +τI) −1up −1   , (S.15) which implies (S.12). ...

work page 2018

[11] [11]

Recall that bδ(i) is independent ofX i and that E(X i) =0, cov(X i) = I and that, for any finitek, the firstkmoments of its entries are bounded uniformly inn

is asymptoticallyN(0,E(∥ bβ− β0∥2)). Recall that bδ(i) is independent ofX i and that E(X i) =0, cov(X i) = I and that, for any finitek, the firstkmoments of its entries are bounded uniformly inn. We have shown that in Proposition S.4 that var(∥ bβ−β 0∥2)→0. In light of Lemma S.3, we also know that E(∥ bβ−β 0∥2) is uniformly bounded. Furthermore, in the pr...

work page 2010

[12] [12]

Independence ofX i andX j gives E n ei(w1αi+w2αj)|X(ij),ϵ (ij) o = E eiw1αi|X(ij),ϵ (ij) E eiw2αj |X(ij),ϵ (ij)

63 Now, we have E n ei(w1αi+w2αj)1En (ij) o = E h 1En (ij) E n ei(w1αi+w2αj)|X(ij),ϵ (ij) oi , since 1En (ij) is a deterministic function of (X (ij),ϵ (ij)). Independence ofX i andX j gives E n ei(w1αi+w2αj)|X(ij),ϵ (ij) o = E eiw1αi|X(ij),ϵ (ij) E eiw2αj |X(ij),ϵ (ij) . Also, the conditional Gaussian approximation established above implies that 1En (ij) ...

work page 2000

[13] [13]

Gathering all these results, we have |ψ′(Ri)−ψ ′(prox(λ2 i cτ ρ)(eri,(i)))|= O Lk{(∥ψ∥∞ + 1)n−1/2+3α polyLog(n)}

Using Lemma S.11, we can write |prox(c iρ)(eri,(i))−prox(λ 2 i cτ ρ)(eri,(i))| ≤ ∥ψ∥ ∞|ci −λ 2 i cτ | and hence |ψ′{prox(ciρ)(eri,(i))} −ψ ′{prox(λ2 i cτ ρ)(eri,(i))}|= O Lk(∥ψ∥∞n−1/2+3α polyLog(n)). Gathering all these results, we have |ψ′(Ri)−ψ ′(prox(λ2 i cτ ρ)(eri,(i)))|= O Lk{(∥ψ∥∞ + 1)n−1/2+3α polyLog(n)}. 69 So we have shown thatδ n(cτ) = OLk(n−1/2...

work page 2018