Pith. sign in

REVIEW 2 major objections 5 minor 15 references

Single Point Transductive Prediction

T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Knowing a single test point in advance removes the regularization bias that regularized linear predictors otherwise pay.

desk verdict Solid, genuinely novel theory for single-point transductive prediction; the JM half delivers the advertised bound, the OM half is narrower than the abstract says, and Table 1 has a row that looks copied from OLS. read the letter →

arxiv 1908.02341 v4 pith:QB3PRLBH submitted 2019-08-06 stat.ML cs.LG

Nilesh Tripuraneni , Lester Mackey This is my paper · ORCID
classification stat.MLcs.LG MSC 62J0562J07
keywords transductivepredictionregularizationbiasdebiasedLassoorthogonalmomentshigh-dimensionallinearregressionridgedistributionshift
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether seeing one test point $x_*$ before fitting lets a linear prediction method beat the usual train-then-predict recipe. It argues yes: two transductive estimators, one based on the Javanmard–Montanari debiasing program and one based on orthogonal moment estimation, can turn a high-dimensional regularized fit into a prediction along $x_*$ whose excess risk is $O(\sigma_\epsilon^2 x_*^\top \Omega x_*/n)$, which is dimension-free $O(\sigma_\epsilon^2 \|x_*\|_2^2/n)$ under standard bounded-eigenvalue conditions. In the same settings, ridge with optimally tuned regularization is shown to pay $\omega(\|x_*\|_2^2 \sigma_\epsilon^2/n)$ and the Lasso to pay $\Omega(\|x_*\|_{(s)}^2 \sigma_\epsilon^2 \log p/n)$ in worst-case directions, so transduction buys an explicit rate improvement, not just a constant. The gain is largest when the test point is soft-sparse, and experiments show benefits under distribution shift and even on data from the training distribution.

What carries the argument

The JM-style estimator (3) computes a vector $w$ that approximately inverts the sample covariance along $x_*$, solving $\min_w w^\top \hat{\Sigma}_n w$ subject to $\|\hat{\Sigma}_n w - x_*\|_\infty \leq \lambda_w$, then applies a one-step correction $w^\top X^\top (y - X\hat{\beta})/n$ to the pilot prediction $\langle x_*, \hat{\beta}\rangle$. The orthogonal-moment estimator (8) rotates the data so that $\langle x_*, \beta_0\rangle$ becomes a single coefficient $\theta_0$, splits the sample, fits nuisance regressions $f$ and $g$ on one fold, and solves a Neyman-orthogonal moment equation on the other, with a threshold on the estimated treatment-noise variance to control the denominator. Neyman orthogonality makes the second estimator first-order insensitive to nuisance error, and the data splitting makes the moment equation treat the nuisance fits as fixed.

What would settle it

Generate data from a sparse linear model with sub-Gaussian covariates that violate Assumption 5—for instance, let the transformed first-coordinate noise $\eta$ have variance that depends on $z$—fit the orthogonal-moment estimator, and measure $E[(\hat{y} - \langle x_*, \beta_0\rangle)^2]$ at a sample size where the Gaussian-design bound predicts $O(\|x_*\|_2^2/n)$. If the risk exceeds the claimed dimension-free rate by a factor growing with $\log p$, the OM half of the central claim fails outside its proven regime.

Watch

Extended reading notes

Core claim

In its strongest form, the paper claims that the $x_*$-prediction risk of regularized linear estimators is dominated by a regularization bias that can be removed by knowing $x_*$. Two estimators are proved to achieve $E[(\hat{y} - \langle x_*, \beta_0\rangle)^2] \leq O(\sigma_\epsilon^2 x_*^\top \Omega x_*/n)$ after an initial pilot fit, matching the oracle variance of the OLS prediction along $x_*$ while keeping the usual regularized bias under control. Against this, the paper proves lower bounds: for Gaussian designs, ridge tuned to minimize prediction or estimation error can suffer at least $\omega(\|x_*\|_2^2 \sigma_\epsilon^2/n)$ in directions correlated with $\beta_0$, and the Lasso must suffer $\Omega(\|x_*\|_{(s)}^2 \sigma_\epsilon^2 \log p / n)$ for worst-case $s$-sparse $\beta_0$. The transductive gains are thus explicit rate improvements in the single-test-point setting.

Load-bearing premise

The load-bearing premise is Assumption 5, that after a rotation putting the test point on the first axis, the noise $\eta$ in the transformed first coordinate is independent of the remaining covariates $z$ and the regression of the first coordinate on $z$ is linear; this holds for Gaussian designs but is not guaranteed under the paper's general sub-Gaussian design assumption, so the orthogonal-moment half of the proof covers a narrower regime than the abstract suggests.

Editorial extensions

If this is right

  • Predictions for a known test point can reach the OLS-optimal dimension-free rate $O(\|x_*\|_2^2 \sigma_\epsilon^2/n)$ even when the full parameter vector cannot be estimated consistently.
  • Regularization tuned by cross-validation or training-distribution risk is systematically over-regularized for a fixed test direction; transduction removes that bias.
  • Because the gain condition $\|x_*\|_2 / \|x_*\|_{(s)} \lesssim \sqrt{\log p}$ is observable, a practitioner can decide per test point whether to use a transductive or inductive predictor.
  • Benchmarking regularized learners by average held-out risk understates their error on specific test directions; single-point transductive baselines are a sharper comparison.
  • Under distribution shift, transductive prediction can beat cross-validated ridge, Lasso, and elastic net even when the test point is a single sample from a shifted distribution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An immediate extension the paper leaves implicit is batched transduction: applying the single-point procedure to each test point separately gives per-point rates, so average risk over a test set inherits the single-point guarantee without requiring the test distribution to match the training distribution.
  • The double-robustness structure of Theorem 5 suggests a fully nonparametric version: if the $g$-regression uses a black-box learner and only its squared error $r_{g,2}$ is controlled, the OM estimator should remain consistent for $x_*$ even when the linear model for the treatment is misspecified.
  • One could construct a single-point transductive classifier by embedding classes in a linear score and applying the JM-style correction to the margin direction of the test input; the risk analysis transfers, while the label noise structure differs from the additive $\epsilon$ setting.
  • Because the lower bound against the Lasso requires $\beta_0$ to align its signs with the largest coordinates of $x_*$, a conservative user could look for that alignment in the pilot estimate and deploy transduction only when the estimated $\hat{\beta}_0$ agrees strongly with $x_*$.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper studies single-point transductive prediction in the random-design linear model. For a fixed test direction x*, it proves lower bounds showing that ridge regression with an optimally tuned penalty and the Lasso can incur prediction bias that is dimension-dependent in certain directions. It then proposes two transductive estimators: a Javanmard–Montanari-style debiasing estimator (Section 3.1) and a thresholded orthogonal-moment estimator (Section 3.2), and proves non-asymptotic upper bounds on their x*-prediction risk. Under the paper's assumptions, the JM estimator attains O(sigma_epsilon^2 x*^T Omega x*/n) risk, which is dimension-free O(sigma_epsilon^2 ||x*||_2^2/n) in natural scaling. The OM estimator attains a similar rate in expectation, and the paper gives corollaries for Lasso and ridge pilots and for the corresponding high-dimensional regimes. The experimental section compares the transductive rules with cross-validated ridge, Lasso, and elastic net on synthetic shifted designs and five real datasets.

Significance. If the results stand, the paper makes a useful conceptual and technical contribution: it shows that access to a single test point can be exploited to reduce regularization bias, and it connects debiased-Lasso and orthogonal-moment techniques to mean-squared prediction error rather than only to confidence-interval construction. The lower bounds for ridge and Lasso are explicit and provide benchmark rates for this transductive problem. The paper is also a strength in transparency: the main theorems are proved in appendices with stated assumptions, the constants are tracked at the level of polynomial-order notation, and code for the experiments is provided. The key caveat is that the OM half of the paper establishes its dimension-free guarantee in a narrower design regime than the abstract suggests, and one empirical table has a reporting inconsistency that currently prevents full evaluation of the JM method's real-data behavior.

major comments (2)
  1. [§3.2, Theorem 5 and Appendix E.1] The dimension-free O(sigma_epsilon^2 x*^T Omega x*/n) guarantee for the OM estimator is proved only under Assumption 5 (eta independent of z) together with the additional linearity assumption g0(z) = z^T g0. These conditions are not consequences of the general sub-Gaussian design Assumptions 2–3; they hold, for example, for Gaussian designs but not for all sub-Gaussian designs covered by the paper's setup. The footnote in Section 3.2 claims that Assumption 5 can be replaced by E[eta|z]=0 plus conditional sub-Gaussianity, but no proof or reference is supplied for that claim, and even that replacement would not imply the linearity of g0 used in the proof. Since Corollaries 4–5 and the abstract's phrasing that two transductive rules attain the dimension-free rate depend on Theorem 5, the paper should either add the missing proof, state prominently that the OM guarantee is restricted to the Gaussian-like regime described by Assumption 5 and linear g0, or revise the abstract and discussion accordingly.
  2. [§4.3, Table 1 and §G.3] In four of the five real datasets, the reported JM(Lasso) RMSE is exactly equal to the reported OLS RMSE: Wine 1.0118, Parkinson 12.7916, Forest Fires 82.7147, and Fertility 0.3988. The text and Appendix G.3 state that, when the convex program in Eq. (4) is infeasible, the JM estimator falls back to the base Lasso pilot, which would reproduce the Lasso entries, not the OLS entries. This discrepancy indicates an implementation or reporting issue for JM(Lasso) that must be clarified or corrected before the real-data claims about the JM estimator can be evaluated. The Triazines row is not identical to OLS but also not equal to the Lasso row, which adds to the need for an explanation of the JM implementation details in the table.
minor comments (5)
  1. [§1.2] The sentence 'The expressions beta0 indicates the number of non-zero elements in beta0' appears to contain a typo; it should presumably read 'The expression ||beta0||_0 indicates the number of non-zero elements in beta0.' Please clarify the notation.
  2. [§3.2, OM algorithm] The algorithm description splits the data into two folds of size n/2, but no care is taken for odd n. Since some real datasets in Table 2 have odd sizes (Fertility has n=100, which is even, but other datasets may have odd counts after splitting), either assume n is even or specify how to handle the last point.
  3. [§4.3, table captions] The paper's text says error bars in Section 4.1 represent +/-1 standard error over 20 runs; however Figures 2–5 do not display visible error bars in the captions. Please clarify whether error bars are omitted from those plots or are too small to see, and state this in the captions.
  4. [§G.3, JM feasibility heuristic] The text says lambda_w is chosen as the smallest value in a set for which the convex program in Eq. (4) is feasible, but it does not specify the grid or the stopping criterion in enough detail to reproduce the JM results exactly. Since the table issue in Major Comment 2 may stem from this heuristic, please provide the full grid and the feasibility tolerance.
  5. [§C.2, Proposition 10] The dual norm of the top-k norm is stated as max(||x||_1/k, ||x||_infty), which is correct only when the dual is taken with the convention used here; please add a short derivation or reference so that readers do not confuse the top-k norm with the standard k-support norm.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central risk bounds are proven from stated assumptions using external, non-self-cited results; the only self-citations are not load-bearing.

full rationale

I walked the derivation chain and found no circular step. The central claims are non-asymptotic upper bounds for two transductive estimators and lower bounds for ridge and Lasso. The JM-style estimator (3)-(4) is an explicitly acknowledged generalization of Javanmard-Montanari debiasing, and the proof of Theorem 3 decomposes the error as (1/n)w^T X^T epsilon + <x_star - Sigma_n w, beta_hat - beta_0>, which is an identity separating a variance term from a pilot-error term. The pilot error is then bounded using external Lasso oracle inequalities from Bellec et al. and Wainwright, not by assuming the conclusion. The OM estimator is likewise an acknowledged thresholded variant of the orthogonal moment estimator of Chernozhukov et al.; Theorem 5 expands the empirical moment equation into A, B1, B2, C terms and bounds each under Assumption 5 and the linear-g0 condition. No fitted parameter is renamed as a prediction, and the target risk (2) is defined with the true beta_0, independent of any estimated quantity. The lower bounds in Theorems 1-2 are proved from Gaussian/Wishart spectral facts and external Lasso support-recovery arguments, not from the upper-bound estimators. The only self-citation of an author, Mackey et al. 2017, appears in a future-work remark and in an experimental variance-selection formula; neither is load-bearing for the main theorems. The footnote relaxing Assumption 5 is an unproved scope assertion, and the stated OM guarantee does rely on Assumption 5 and linear g0; this is a limitation of the OM half, but it is not a circular reduction. Overall the paper is self-contained relative to external, well-specified results in the sense required by the circularity pass, so the appropriate score is 0.

Assumptions & free parameters 2 free parameters · 9 assumptions · 0 invented entities

All upper bounds require standard random-design assumptions (well-specified model, bounded covariance, sub-Gaussian design and noise). The lower bounds are proven for Gaussian designs. The OM estimator additionally requires Assumption 5 (eta independent of z) and linear g0, restricting its theory to Gaussian-like designs. External concentration and Lasso-consistency theorems are imported without proof. No invented entities are introduced.

free parameters (2)
  • tau = sigma_eta^2 / 4
    Threshold in the OM estimator to control the denominator mu_2; set by hand in the proof (Section 3.2, Theorem 5), not estimated from data.
  • lambda_w = 8a sqrt(C_cond) kappa^2 ||x*||_2 sqrt(log(p∨n)/n), a>0
    Regularization parameter for the JM convex program (4); chosen to ensure feasibility with high probability. The constant a is a universal proof constant, not fitted.
assumptions (9)
  • domain assumption Well-specified linear model (Assumption 1)
    Used throughout; y_i = x_i^T beta0 + epsilon_i with E[epsilon_i|X]=0.
  • domain assumption Bounded covariance (Assumption 2)
    Requires Sigma_ii <= 1/2, sigma_max(Sigma) <= C_max, sigma_min(Sigma) >= C_min; used in all upper bounds and concentration arguments.
  • domain assumption Sub-Gaussian design (Assumption 3)
    Design vectors are sub-Gaussian with parameter kappa; used for Chernoff-type deviations and RE conditions.
  • domain assumption Sub-Gaussian noise (Assumption 4)
    Noise is sub-Gaussian with variance sigma_epsilon^2; used in moment bounds and Lasso tail bounds.
  • domain assumption Independent treatment noise eta (Assumption 5)
    In the reparameterized model, eta is independent of z; needed for Lemma 7 and Theorem 5. The paper notes it is not essential and can be relaxed.
  • domain assumption Gaussian design and Gaussian noise for lower bounds (Theorems 1-2)
    Lower bounds are proven for i.i.d. N(0,I_p) covariates and Gaussian noise; sufficient to establish worst-case bias.
  • standard math External Lasso tail bounds (Bellec et al. 2016, Thm 4.2/9.1)
    Used to bound E||beta_L(lambda)-beta0||_q^k in proofs of Corollaries 3 and 5; accepted published results.
  • standard math Concentration of Gaussian random matrices (Wainwright 2019, Thm 6.1)
    Used in Theorem 1 proof to control eigenvalues of the sample covariance.
  • standard math Wishart eigendecomposition properties (Bishop et al. 2018)
    Used in Theorem 1 and Lemma 2 to reduce expectations over X to scalar eigenvalue expectations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Single Point Transductive Prediction." pith.science (2026). https://pith.science/paper/QB3PRLBH

@misc{pith2026190802341,
  author       = {Pith},
  title        = {Pith review of: Single Point Transductive Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QB3PRLBH}},
  note         = {Machine review of arXiv:1908.02341}
}
abstract

Standard methods in supervised learning separate training and prediction: the model is fit independently of any test points it may encounter. However, can knowledge of the next test point $\mathbf{x}_{\star}$ be exploited to improve prediction accuracy? We address this question in the context of linear prediction, showing how techniques from semi-parametric inference can be used transductively to combat regularization bias. We first lower bound the $\mathbf{x}_{\star}$ prediction error of ridge regression and the Lasso, showing that they must incur significant bias in certain test directions. We then provide non-asymptotic upper bounds on the $\mathbf{x}_{\star}$ prediction error of two transductive prediction rules. We conclude by showing the efficacy of our methods on both synthetic and real data, highlighting the improvements single point transductive prediction can provide in settings with distribution shift.

Figures

Figures reproduced from arXiv: 1908.02341 by the authors.

Figure 1
Figure 1. Lasso vs. OM and JM Lasso prediction without distribu￾tion shift. Hyperparameters are set according to theory (see Sec￾tion 4.1). Left: p = 200, sβ0 = 20. Right: p = 200, sβ0 = 100 [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Ridge vs. OM ridge prediction (p = 200) under train-test distribution shift. Hyperparameters are set according to theory. 0 500 1000 1500 2000 2500 3000 Number of Sample Points (n) −4 −3 −2 −1 0 1 2 3 4 Log Average Prediction RMSRE Distribution Shift in Cov. Matrix Ridge (λ=CV) OM (λ=CV) 0 500 1000 1500 2000 2500 3000 Number of Sample Points (n) −2 −1 0 1 2 3 4 Log Average Prediction RMSRE Distribution Shift in Mean… view at source ↗
Figure 3
Figure 3. Ridge vs. OM ridge prediction (p = 200) under train-test distribution shift. Hyperparameters are set according to CV. 0 500 1000 1500 2000 2500 3000 Number of Sample Points (n) 0 1 2 3 4 5 6 7 Log Average Prediction RMSRE Distribution Shift in Cov. Matrix Lasso (λ=Theory) OM (λ=Theory) JM (λ=Theory) 0 500 1000 1500 2000 2500 3000 Number of Sample Points (n) 2 4 6 8 10 12 14 Log Average Prediction RMSRE Distribution … view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Lasso vs. OM and JM Lasso prediction (p = 200) under mean (sβ0 = 100) or covariance (sβ0 = 20) train-test distribution shifts. Hyperparameters are set according to theory. mean shifting, we generated x? ∼ N (10β0, Ip) for each problem instance. The first and second plo…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 15 canonical work pages

  1. [1]

    For A = 1 n/2 ∑n/2 i=1ηiϵi, note that E[m(ti,yi,θ 0, z⊤ i f0, g0(zi))|zi] = 0 so it follows that, E[A2] =O( 1 n E[η2ϵ2]) = 1 nσ2 ϵσ2 η

    work page

  2. [2]

    Accordingly, for sufficiently largep, we ‖ ˆβL(λ)− β0‖1≤K1s √ logp n with probability at least 1−O(exp(−cn))−O(p−γ/2)

    Hence, forδ0≍p−γ/2, log(1/δ0) s log(1/δ(λ))→ 0. Accordingly, for sufficiently largep, we ‖ ˆβL(λ)− β0‖1≤K1s √ logp n with probability at least 1−O(exp(−cn))−O(p−γ/2). Define the setS1 = [x⊤ ⋆ ˆβL(λ) +‖x⋆‖∞Ks β0 √ logp n , x⊤ ⋆ ˆβL(λ)− ‖x⋆‖∞Ks β0 √ logp n ] for future reference. Single Point Transductive Prediction In the case of the dense loading regime we ...

    work page

  3. [3]

    Note E[∇g(z)m(ti,yi,θ 0, z⊤ i f0, g0(zi))|zi] = 0 using independence of ϵi and the fact E[ϵi] = 0

    ForB2 =∑n i=1 ∆g(zi)ϵi. Note E[∇g(z)m(ti,yi,θ 0, z⊤ i f0, g0(zi))|zi] = 0 using independence of ϵi and the fact E[ϵi] = 0. Once again using independence, E[B2 2] = 1 n E[ϵ2(∆g(z))2]≤O( 1 nσ2 ϵr2 g) Single Point Transductive Prediction

    work page

  4. [4]

    The sub-exponential tail bound gives, Pr   1 n n∑ 𝓁=j z2 j≥ x⋆Ωx⋆ +t  ≤ exp [ −n/2 min((t/κ′ 1)2,t/κ′ 1) ] whereκ′ 1 = 8κ2/Cmin‖x⋆‖2

    and is mean-zero. The sub-exponential tail bound gives, Pr   1 n n∑ 𝓁=j z2 j≥ x⋆Ωx⋆ +t  ≤ exp [ −n/2 min((t/κ′ 1)2,t/κ′ 1) ] whereκ′ 1 = 8κ2/Cmin‖x⋆‖2

    work page

  5. [5]

    Finally we can easily convert these tail bounds into moment bounds, Single Point Transductive Prediction Corollary 6

    Hence, since on the eventF(a), we have thatw⊤Σnw≤ (x⋆Ω)⊤ΣnΩx⋆ (recall w0 = Ωx⋆ is feasible onF(a)), Pr [ w⊤Σnw≥ x⋆Ωx⋆ +t ] ≤ Pr [ {w⊤Σnw≥ x⋆Ωx⋆ +t}∩F (a) ] + Pr [ {w⊤Σnw≥ x⋆Ωx⋆ +t}∩F (a)c] ≤ Pr   1 n n∑ 𝓁=j z2 j≥ x⋆Ωx⋆ +t   + 0≤ exp [ −n/2 min((t/κ′ 1)2,t/κ′ 1)) ] , since by definition on the eventF(a)c the convex program outputs w = 0 and x⋆Ωx⋆≥ 1/Cma...

    work page

  6. [7]

    Note E[∇z⊤fm(ti,yi,θ 0, z⊤ i f0, g0(zi))|zi] = 0 since E[ηi|zi] = 0

    ForB1 = 1 n/2 ∑n/2 i=1 ∆f(zi)ηi. Note E[∇z⊤fm(ti,yi,θ 0, z⊤ i f0, g0(zi))|zi] = 0 since E[ηi|zi] = 0. So we have using sub-gaussianity of the random vector z, sub-gaussianity ofη and independence that, E[B2 1] =O( 1 n E[(∆f(z))2η2])≤O( 1 nr2 f,2σ2 η)

    work page

  7. [9]

    For C = 1 n ∑n i=1 ∆g(zi)∆f(zi). Note that in general for the remainder term E[∇z⊤f,g(z)m(ti,yi,θ 0, z⊤ i f0, g0(zi))|zi] ⁄= 0 ; however in some cases we can exploit unless we can exploit unconditional orthogonality: E[∇z⊤f,g(z)m(ti,yi,θ 0, z⊤ i f0, g0(zi))] = 0 to obtain an improved rate although this is not mentioned in the main text. • In the absence o...

    work page 2009

  8. [10]

    Thus, E[Zk q ] = ∫ ∞ 0 ktk−1 Pr[Zq >t ]dt = ∫ T 0 ktk−1 + ∫ ∞ T ktk−1e−t 2 ≤ Tk + ∫ ∞ 0 ktk−1e−t 2 ≤Tk + k(k− 1) 2

    = s φ2 0 log(1/δ(λ)). Thus, E[Zk q ] = ∫ ∞ 0 ktk−1 Pr[Zq >t ]dt = ∫ T 0 ktk−1 + ∫ ∞ T ktk−1e−t 2 ≤ Tk + ∫ ∞ 0 ktk−1e−t 2 ≤Tk + k(k− 1) 2 . which implies the conclusion, ‖ ˆβL(λ)− β0‖k q≤ (49 8 Tλs 1/q s log(1/δ(λ)) )k + (49 8 λs1/q s log(1/δ(λ)) )k k(k− 1) 2 ≤ Single Point Transductive Prediction (49λs1/q 8φ2 0 )k + ( 49 8 (8 + 2 √ 2)σ s1−1/q√n )k k(k− 1)...

    work page 2016

Show all 15 references
  1. [11]

    G(u)>F (u). Then, ‖u‖2≤ (√ log(1/δ0) s log(1/δ(λ)) + 40 √ π/2σ λ√sγ√n ) ‖Xu‖2/√n (12) Single Point Transductive Prediction Thus, ∆∗≤ 2λ(1 +τ)√s‖u‖2 + 2G(u) 2λ√s(1 +τ +γ) (√ log(1/δ0) s log(1/δ(λ)) + 40 √ π/2σ λ√sγ√n ) ‖Xu‖2/√n≤ 2λ2s(1 +τ +γ)2 ( log(1/δ0) s log(1/δ(λ)) + 800πσ ...

  2. [12]

    In this case, ∆∗≤ 2λ  (1 +γ +τ)√s‖u‖2− (1−γ−τ) p∑ j=s+1 u♯ j   = ∆ (14) Since ∆> 0, u belongs to theSRE(s,c 0) cone and henceφ0(s,c 0)‖u‖2≤‖ Xu‖2

    G(u)≤F (u). In this case, ∆∗≤ 2λ  (1 +γ +τ)√s‖u‖2− (1−γ−τ) p∑ j=s+1 u♯ j   = ∆ (14) Since ∆> 0, u belongs to theSRE(s,c 0) cone and henceφ0(s,c 0)‖u‖2≤‖ Xu‖2. So, ∆∗≤ ∆≤ 2(1 +γ +τ)λ√s φ0(s,c 0) ‖Xu‖/√n≤ ((1 +γ +τ)λ√s φ0(s,c 0) ) )2 +‖Xu‖2/n (15) Assembling the two cases we...

  3. [13]

    G(u)>F (u), then using Equations 11 and 13 we have, ‖Xu‖2 2/n≤ 2λ2s(1 +γ)2( log(1/δ0) s log(1/δ(λ))) + 1600πσ 2(1 +γ)2 γ2n Combining the previous display with (12) we have, ‖u‖2≤ (√ 2λ2s(1 +γ)2( log(1/δ0) s log(1/δ(λ))) + √ 1600πσ 2(1 +γ)2 γ2n )(√ log(1/δ0) s log(1/δ(λ)) + 40 ...

  4. [14]

    Equations 11 and 14 implies that ∆≥ ∆∗≥ 0 a.s

    G(u)≤F (u). Equations 11 and 14 implies that ∆≥ ∆∗≥ 0 a.s. Hence u is contained inSRE(s, 1+γ 1−γ ), and ‖u‖2≤ ‖Xu‖2 nφ0(s, 1+γ 1−γ )≤ (1 +γ)λ√s φ2 0(s, 1+γ 1−γ ) using (11) and (15), and recalling we setτ = 0. Assembling these two cases we conclude, (1 +γ)‖ ˆβL(λ)− β0‖2≤ 3 ( C...

  5. [15]

    bad" eventsX /∈E n(s, 7) where we cannot guarantee a “fast

    = s φ2 0 log(1/δ(λ)) for q∈{ 1, 2}. As before, E[Zk q ]≤Tk +k(k− 1). Since E[‖ ˆβL(λ)− β0‖k q] = E[(‖ ˆβL(λ)− β0‖q−c +c)k]≤ 2k−1 ( E[(‖ ˆβL(λ)− β0‖q−c)k] +ck ) , we conclude, E[‖ ˆβL(λ)− β0‖k 1]≤ 2k−1 (( 13 Tλs s log(1/δ(λ)) )k + ( 13 λs s log(1/δ(λ)) )k k(k− 1) 2 + (250000σ2 ...

    2012

  6. [2017]

    Hoerl, A

    URL http://archive.ics.uci.edu/ml. Hoerl, A. E. and Kennard, R. W. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1):55–67, 1970. Hsu, D., Kakade, S. M., and Zhang, T. Random design analysis of ridge regression. In Conference on learning theo...

    2018 arXiv

  7. [8237]

    URL http://dx

    doi: 10.1561/2200000072. URL http://dx. doi.org/10.1561/2200000072. Cai, T. T. and Guo, Z. Confidence intervals for high- dimensional linear regression: Minimax rates and adap- tivity. The Annals of statistics, 45(2):615–646, 2017. Single Point Transductive Prediction Chao, S.-...

    doi

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.