Online Statistical Inference of Constrained Stochastic Optimization via Random Scaling

Sen Na; Wanrong Zhu; Wei Biao Wu; Xinchen Du

REVIEW 2 major objections 4 minor 13 references

Online Statistical Inference of Constrained Stochastic Optimization via Random Scaling

T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A random scaling statistic gives constrained online inference without covariance matrix estimation.

desk verdict A solid extension of random scaling to constrained second-order stochastic optimization; the pivotal limit is new and the proof is careful, though the local regularity assumptions are a genuine soft spot and the coverage claims are a bit rosy. read the letter →

arxiv 2505.18327 v1 pith:YPQ5MBYN submitted 2025-05-23 stat.ML cs.LGcs.NAmath.NAmath.OCmath.STstat.COstat.TH

Xinchen Du , Wanrong Zhu , Wei Biao Wu , Sen Na This is my paper · ORCID

classification stat.MLcs.LGcs.NAmath.NAmath.OCmath.STstat.COstat.TH MSC 62L2062F1290C5560F17

keywords randomscalingonlinestatisticalinferencestochasticsequentialquadraticprogrammingconstrainedoptimizationasymptoticnormalityfunctionalcentrallimittheoremmatrix-freesketchedNewtonmethods

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that online statistical inference for constrained stochastic optimization can be done without estimating the limiting covariance matrix. The authors study an adaptive inexact stochastic sequential quadratic programming (AI-SSQP) method, whose sketched Newton steps produce primal-dual iterates, and they show that the averaged iterates are asymptotically normal with a smaller limiting covariance than the last iterate. To turn this into confidence intervals, they build a random scaling matrix $V_t$ from the iterates alone and studentize the averaged error with it. The resulting statistic converges to a distribution free of unknown parameters, so the method is asymptotically valid, matrix-free, and matches the computational cost of first-order online inference. If correct, this would be the first such procedure for nonlinear constrained problems via second-order iterates.

What carries the argument

The central object is the random scaling matrix $V_t = \frac{1}{t^2}\sum_{i=1}^t i^2(\bar s_i - \bar s_t)(\bar s_i - \bar s_t)^\top$, where $\bar s_t = (\bar x_t, \bar\lambda_t)$ is the averaged primal-dual iterate. It is not a covariance estimator; it is a self-normalizer built recursively from the iterates. The argument pairs the functional central limit theorem for partial sums of AI-SSQP errors with the continuous mapping theorem: the statistic $\sqrt{t}\,w^\top(\bar s_t - s_\star)/\sqrt{w^\top V_t w}$ becomes a continuous functional of the limiting Brownian motion, forcing the unknown scaling $\sqrt{w^\top \bar\Xi_\star w}$ to cancel and leaving the pivotal ratio $W_1(1)/\sqrt{\int_0^1 (W_1(r)-rW_1(1))^2\,dr}$. The sketching-induced projection operators $C_\star$ and $\tilde C_\star$ enter through the covariance $\bar\Xi_\star$ but cancel in the pivotal limit.

What would settle it

Run AI-SSQP with randomized Kaczmarz sketching on the paper's nonlinearly constrained logistic regression, record $\sqrt{t}\,w^\top(\bar s_t - s_\star)/\sqrt{w^\top V_t w}$ over many independent runs at a fixed large $t$, and compare the empirical quantiles with 3.875, 5.323, 6.747, and 8.613; a mismatch would falsify Theorem 4.2. More directly, choose $w$ inside the singular normal subspace $\mathrm{Span}((G^\star)^\top)\otimes 0_m$, where the paper's nondegeneracy condition fails, and check whether the statistic degenerates or produces coverage away from nominal.

Watch

Extended reading notes

Core claim

The paper claims that for any direction $w$ not lying in the singular subspace $\mathrm{Span}((G^\star)^\top)\otimes 0_m$, the studentized averaged AI-SSQP error satisfies $\sqrt{t}\,w^\top(\bar s_t - s_\star)/\sqrt{w^\top V_t w} \Rightarrow W_1(1)/\sqrt{\int_0^1 (W_1(r)-rW_1(1))^2\,dr}$, where $W_1$ is standard Brownian motion and $s_\star=(x_\star,\lambda_\star)$. Because the right-hand side contains no unknown parameters, quantiles can be tabulated once and confidence intervals follow immediately from the corollary. The result rests on a functional central limit theorem for the primal-dual partial sum process of AI-SSQP, with limiting Brownian motion scaled by $\bar\Xi_\star$, and on a stopping-time localization that keeps the nonconvex iterates inside a regular neighborhood until the global almost-sure convergence has taken effect. The paper also claims that averaged iterates are statistically more efficient than last iterates, with $\bar\Xi_\star \preceq \tilde\Xi_\star$, and that the gap from the minimax optimal covariance $\Xi_\star$ decays exponentially in the number of sketching steps.

Load-bearing premise

All AI-SSQP iterates must stay inside a fixed bounded set where the constraint Jacobian has full row rank and the reduced Hessian is positive definite, and since the iterates are not projected and the feasible set may be nonconvex, nothing in the algorithm itself guarantees this.

Editorial extensions

If this is right

Asymptotically valid confidence intervals can be built for any fixed direction $w$ outside the singular subspace using tabulated quantiles, with no covariance estimation.
The procedure is fully online and recursive, costing $O((d+m)^2)$ per iteration rather than the $O((d+m)^3)$ of plug-in matrix inversion.
Averaged iterates are more efficient than last iterates; with the sketching solver suppressed the limiting covariance reduces to the minimax optimal $\Xi_\star$, and with sketching the efficiency loss decays like $(1-\gamma_S)^\tau$.
The inference method needs only a bounded $(2+\delta)$-moment on gradient noise, weaker than the sub-Gaussian or fourth-moment conditions used in earlier constrained inference procedures.
The framework applies to nonlinear, nonconvex equality-constrained problems, going beyond the strongly convex or unconstrained settings of prior work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same self-normalizer should extend to inequality-constrained problems through active-set SQP reductions, since those methods asymptotically become equality-constrained near the solution.
The pivotal law already appears in cointegration and robust testing, which suggests the statistic may be insensitive to some noise-distribution misspecification, although the paper does not claim robustness.
A direct testable extension is to vary the sketching steps $\tau$ for a fixed problem and check that coverage approaches nominal for every $\tau \geq \tau^\star$, with interval length shrinking like $t^{-1/2}$ once the averaged iterate enters the normal regime.
The unprojected, nonconvex iterates may leave the basin of a regular KKT point; practical safeguards such as projection, restarts, or a verified LICQ condition would be needed before deployment on general black-box constraints.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a real extension of random scaling to constrained second-order stochastic optimization. The pivotal limit in Theorem 4.2 is new, and the proof looks internally consistent. The main soft spot is not the martingale/FCLT machinery but the gap between the global convergence theorem and the local regularity assumptions used for the pivotal result.

What the paper does well: it establishes asymptotic normality of averaged AI-SSQP iterates, proves the FCLT under sketching and adaptive stepsizes, and shows the random scaling statistic is pivotal, with explicit quantiles. The appendices are detailed; the decomposition in Lemma A.3 and the stopping-time localization are carefully handled. The claim that this is the first matrix-free online inference for constrained nonconvex problems via second-order iterates is plausible. Credit the authors for sidestepping covariance estimation entirely rather than adding another inconsistent plug-in estimator.

Soft spots: Assumption 3.1 requires all iterates to stay in a compact set with LICQ and a positive-definite reduced Hessian. The iterates are unprojected; for the sphere constraint, LICQ fails at x=0. The global convergence theorem (Theorem 3.4) only ensures convergence to a stationary point, not a regular one. The paper simply assumes regular convergence ("we assume from now on"). That is standard for local asymptotics, but it is a real gap between the stated general problem and the theorem's conditions. The stopping-time localization in Lemma C.1 controls iterates only up to a stopping time; it requires the iterates to enter the regular basin. For the experiments, x_star is linearly spaced from 0 to 1, so for d≥2 the norm is positive and the KKT point is regular, but the paper never verifies that iterates avoid the singular set. This makes the result conditional on a regularity event that is plausible but unproven.

The empirical section overstates coverage performance. The text says "generally close to 95%" but the full tables show several AveRS entries below 90% (e.g., 87%, 88.5%, 89.5%). Coverage rates come from 200 runs with no standard errors, so sampling noise is nontrivial. This is a presentation issue, not a flaw in the theory.

Citation pattern: the paper builds heavily on the authors' own AI-SSQP results, but the new pivotal and FCLT results do not reduce to those, so it is not circular.

Who it's for: statisticians and optimization researchers working on online inference for constrained problems. A serious referee should engage with it. I'd send it to review; the local regularity gap can be addressed by clarifying assumptions or adding a verification step, and the empirical claims should be toned down.

Referee Report

2 major / 4 minor

Summary. The paper develops an online statistical inference procedure for equality-constrained stochastic optimization using Adaptive Inexact Stochastic Sequential Quadratic Programming (AI-SSQP) iterates. The authors establish asymptotic normality of the averaged primal-dual iterate (Theorem 3.5), a functional central limit theorem (Theorem 4.1), and a pivotal random-scaling limit for linear functionals (Theorem 4.2) whose limiting distribution is the self-normalized Brownian bridge, free of unknown parameters. The procedure is fully online and matrix-free, with a scaling matrix update that avoids matrix inversion. Numerical experiments on constrained linear and logistic regression with spherical constraints compare the proposed method (AveRS) against plug-in, batch-means, and batch-free covariance estimators.

Significance. If Theorem 4.2 is correct, this is the first asymptotically valid, matrix-free online inference procedure for nonlinear constrained stochastic optimization based on second-order iterates, and the comparison of averaged versus last-iterate efficiency (Propositions 3.6 and 3.7) is a useful contribution. The proof infrastructure is a clear strength: Lemma A.3 provides a clean martingale decomposition, Section B.4 verifies the Lindeberg condition and tightness for the FCLT, and the limiting covariance formula (17) is internally consistent with the recursion. The pivotal statistic avoids the inconsistent plug-in covariance estimator of Na and Mahoney (2025) and reduces the per-iteration inference cost to O((d+m)^2). The paper also supplies quantile tables and detailed appendices that make the derivation reproducible.

major comments (2)

[Section 3.1, Lemma A.2(b), and Section 5.1] Assumption 3.1 requires all iterates (xt, λt) to lie in a fixed closed bounded convex set X × Λ with Gt full row rank. For the experimental constraint c(x) = ||x||^2 − R^2, G(x) = 2x^T, so full row rank is equivalent to ||x|| > 0. The AI-SSQP update (10) is unprojected and the safeguard (11) does not enforce ||xt|| > 0; a transient passage near x = 0 would make Kt nearly singular and violate the uniform bound ||Kt^{-1}|| ≤ ΥK in Lemma A.2(b). The stopping-time localization (Lemma C.1, Eq. (C.5)–(C.6)) only controls the iterates up to the first exit from a regular region, and the proof of Lemma B.3 relies on the event that the stopping time is infinite. Thus Theorem 4.2 and the FCLT are conditional on a high-probability event that is not verified for the nonconvex spherical constraints used in the experiments. The authors should either prove that AI-SSQP with the safeguarded stepsize keeps iterates in a set satisfying LICQ for this constraint, or explicitly restrict the theoretical claims to settings where Assumption 3.1 is known to hold and present the spherical-constraint experiments as heuristic support only.
[Section 5.2, Table 2, and Appendix D] The paper claims that AveRS 'consistently achieves promising coverage rates, generally close to 95% across the majority of scenarios,' and the abstract states 'superior performance.' However, in several configurations the reported coverage is substantially below nominal: for example, in d = 5 with τ = 20, linear regression under Equi-correlation r = 0.2 gives 87.0% (Table 6, d=5 block) and r = 0.1 gives 88.5%; similar undercoverage appears in other d = 5, τ = 20 cells. With 200 independent runs, the standard error of a 95% coverage estimate is about 1.6%, so deviations of 7–8 percentage points are not a finite-sample artifact. The presentation should quantify the distribution of coverage across all configurations or explain why the asymptotic regime fails for small d and small τ, rather than highlighting only the cases where coverage is near nominal.

minor comments (4)

[Theorem 4.2] The condition w ∉ Span((G⋆)^T) ⊗ 0_m is notationally unclear; it would be clearer to write w ∉ { (v, 0) : v ∈ Span((G⋆)^T) }.
[Proof of Theorem 4.2, Section B.5] The continuous mapping theorem is applied to a functional f ↦ f(1)/sqrt(∫_0^1 (f(r) − r f(1))^2 dr), which is not continuous at functions whose denominator vanishes. Since this set has Wiener measure zero, the argument is valid, but the zero-set issue should be stated explicitly.
[Algorithm 1] The recursive update for Vt proceeds through P_t and Q_t, which are not individually positive semidefinite; numerical cancellation may produce small negative eigenvalues in finite precision. It would be helpful to note this and, if needed, symmetrize the output.
[Section 5.1] The phrase 'linearly spanned between 0 and 1' should specify whether the endpoints are included; this matters for the coordinatewise average functional and for the condition in Theorem 4.2 when d is small.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the pivotal random-scaling limit is derived from a martingale FCLT and does not reduce by construction to the AI-SSQP convergence inputs.

full rationale

I walked the derivation chain from the AI-SSQP recursion (Lemma A.3) through the averaged CLT (Theorem 3.5) and FCLT (Theorem 4.1) to the pivotal statistic (Theorem 4.2). The only load-bearing external inputs are standard martingale CLT/FCLT results (Hall and Heyde, Duflo), the AI-SSQP convergence and sketching-error lemmas from the authors' prior work, and the regularity conditions on the KKT point. None of these inputs contains the target random-scaling limit. In particular, the random scaling matrix V_t is not constructed from the limiting covariance matrix; it is a function of the averaged iterates only, and the proof of Theorem 4.2 shows algebraically that the covariance cancels in the ratio after applying the continuous mapping theorem. The FCLT in Theorem 4.1 is proved in the paper via martingale FCLT and a stopping-time localization, not imported as an assumption. The self-citations to Na and Mahoney (2025) and Na et al. (2024) supply technical lemmas such as global convergence, sketching error bounds, and bounded inverses with stated conditions; those lemmas do not include the pivotal distribution and are not fitted to data. Therefore, there is no self-definitional, fitted-input, or citation-imported circularity. The fragility of Assumption 3.1 for nonconvex spherical constraints is a correctness and robustness concern, not a circularity.

Assumptions & free parameters 5 free parameters · 8 assumptions · 0 invented entities

The central claim rests on standard smoothness, moment, and sketching assumptions from the AI-SSQP literature, plus algorithmic stepsize conditions. No new physical or mathematical entities are postulated. The free parameters are tuning choices (stepsizes, exponents, regularization) that do not enter the pivotal limit; they only affect whether the stated conditions hold.

free parameters (5)

c_beta = 1 (experiments)
Scaling constant in stepsize beta_t = c_beta/(t+1)^beta. The theory allows any c_beta > 0; chosen by hand in experiments, not fitted to data.
beta = 0.501 (experiments)
Stepsize exponent in beta_t. Must lie in (0.5,1); the experiment uses 0.501 for faster convergence, a hand choice, not fitted.
c_chi = 1 (experiments)
Scaling constant in adaptivity gap chi_t = c_chi/(t+1)^chi. The theory requires c_chi > 0; chosen by hand.
chi = 1.002 (experiments, with chi_t = beta_t^2)
Adaptivity gap exponent. Must satisfy chi > beta + 0.5; the experiment sets chi = 2*beta, a hand choice, not fitted.
delta_t (Hessian regularization) = unspecified in experiments
Regularization added to averaged Hessian to ensure positive definiteness on the constraint null space. The theory only requires it to be large enough; the exact value is a tuning choice.

assumptions (8)

domain assumption Assumption 3.1: iterates stay in a closed bounded set, f and c are twice continuously differentiable, Lagrangian Hessian is Lipschitz, G_t has full row rank with uniform lower bound, and B_t is positive definite on the null space of G_t.
Used to guarantee existence and uniqueness of the SQP subproblem solution and to control all error terms in the recursion; quoted in Section 3.1.
domain assumption Assumption 3.2: unbiased gradient and Hessian estimates, bounded (2+delta) moment for gradient noise, and bounded second moment of Hessian estimates.
Provides the martingale difference structure and the moment bounds needed for the Lindeberg condition and the FCLT; stated in Section 3.1.
domain assumption Assumption 3.3: sketching distribution satisfies E[||S|| ||S^dagger||] <= Upsilon_S and E[K_t S (S^T K_t^2 S)^dagger S^T K_t | x_t, lambda_t] >= gamma_S I.
Ensures linear convergence of the sketching solver and the contraction factor rho = 1 - gamma_S < 1; used throughout the proofs, especially Lemma A.1.
domain assumption The limit point (x_star, lambda_star) is a regular KKT point: G_star has full row rank and the reduced Hessian of the Lagrangian is positive definite on Ker(G_star).
Needed for the local linearization and for the limiting covariance to be finite; standard for constrained M-estimators, stated after Theorem 3.4.
domain assumption cov(gradient of Lagrangian at the solution) is positive definite in the tangential directions, i.e. cov(grad F(x_star; xi)) >> 0.
Imposed in Theorem 4.2 to ensure w^T Xibar_star w > 0 for directions not in the singular subspace; stated before Theorem 4.2.
domain assumption Stepsize sequences beta_t = c_beta/(t+1)^beta and chi_t = c_chi/(t+1)^chi with beta in (0.5,1), chi > beta + 0.5.
These algorithmic conditions make the adaptive stepsize contribution negligible in the FCLT and the averaged normality theorem; stated in Theorem 3.5.
standard math Martingale CLT and functional CLT from Hall and Heyde (2014) and Duflo (2013).
Used to establish asymptotic normality of the averaged iterates and the FCLT in Theorems 3.5 and 4.1.
standard math Continuous mapping theorem from Whitt (2002).
Used to pass from the FCLT for the partial sum process to the limiting distribution of the studentized statistic in Theorem 4.2.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Online Statistical Inference of Constrained Stochastic Optimization via Random Scaling." pith.science (2026). https://pith.science/paper/YPQ5MBYN

@misc{pith2026250518327,
  author       = {Pith},
  title        = {Pith review of: Online Statistical Inference of Constrained Stochastic Optimization via Random Scaling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YPQ5MBYN}},
  note         = {Machine review of arXiv:2505.18327}
}

abstract

Constrained stochastic nonlinear optimization problems have attracted significant attention for their ability to model complex real-world scenarios in physics, economics, and biology. As datasets continue to grow, online inference methods have become crucial for enabling real-time decision-making without the need to store historical data. In this work, we develop an online inference procedure for constrained stochastic optimization by leveraging a method called Sketched Stochastic Sequential Quadratic Programming (SSQP). As a direct generalization of sketched Newton methods, SSQP approximates the objective with a quadratic model and the constraints with a linear model at each step, then applies a sketching solver to inexactly solve the resulting subproblem. Building on this design, we propose a new online inference procedure called random scaling. In particular, we construct a test statistic based on SSQP iterates whose limiting distribution is free of any unknown parameters. Compared to existing online inference procedures, our approach offers two key advantages: (i) it enables the construction of asymptotically valid confidence intervals; and (ii) it is matrix-free, i.e. the computation involves only primal-dual SSQP iterates $(\boldsymbol{x}_t, \boldsymbol{\lambda}_t)$ without requiring any matrix inversions. We validate our theory through numerical experiments on nonlinearly constrained regression problems and demonstrate the superior performance of our random scaling method over existing inference procedures.

Figures

Figures reproduced from arXiv: 2505.18327 by the authors.

**Figure 1.** The averaged iterate error ∥x¯t−x ⋆∥ and the last iterate error ∥xt−x ⋆∥ for constrained linear and logistic regression with varying sketching steps τ . We use d = 20 and Equi-correlation design with r = 0.2 as an illustrative example. Next, we investigate the empirical performance of our random scaling inference procedure alongside four other inference methods that employ different covariance matrix estimators. A… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 7 canonical work pages

[1]

  τY j=1 (I−K ⋆Sj(S⊤ j (K⋆)2Sj)†S⊤ j K⋆)     τY j=1 (I−K ⋆Sj(S⊤ j (K⋆)2Sj)†S⊤ j K⋆)   ⊤ # =∥Ξ ⋆∥ ·E

χ−β+0.5 <∞, where the last inequality is due to χ > β+ 0.5. By Kronecker’s lemma in Lemma A.5, we obtain the result in(d). •Proof of (e).The result in(e)is trivial due to Lemma B.2. B.2 Proof of Proposition 3.6 By the definitions of ¯Ξ⋆ in (17) and Ξ ⋆ in (14), we have ¯Ξ⋆ −Ξ ⋆ = (I−C ⋆)−1E h (I− eC⋆)Ξ⋆(I− eC⋆)⊤ i (I−C ⋆)−1 −Ξ ⋆ = (I−C ⋆)−1E[( eC⋆ −C ⋆)Ξ⋆...

work page 2023
[6]

X. Li, J. Liang, and Z. Zhang. Online statistical inference for nonlinear stochastic approximation with markovian data.arXiv preprint arXiv:2302.07690,

arXiv
[7]

Y. Luo, X. Huo, and Y. Mei. Covariance estimators for the root-sgd algorithm in online learning. arXiv preprint arXiv:2212.01259,

arXiv
[10]

(c) The first result follows from the independence among {St,j}j, and the second result is from (Na and Mahoney, 2025, Corollary 5.4)

(b) The first result follows directly from (A.2) and the independence between randomness of ξt and {St,j}j; the second result is from (Na and Mahoney, 2025, (D.10)) . (c) The first result follows from the independence among {St,j}j, and the second result is from (Na and Mahoney, 2025, Corollary 5.4). ■ The next lemma provides bounds and convergence result...

work page 2025
[12]

Combining the above two displays and plugging into (C.10), we complete the proof

β = Υ(1) µ βt. Combining the above two displays and plugging into (C.10), we complete the proof. D Additional Experimental Results In this section, we present the complete experimental results in Tables 3-8 for constrainedlinear and lo- gistic regression problems. We vary the dimension d∈ {5, 20, 40} and study three different design co- variances Σa: Iden...

work page 2020
[20]

In other words, the more accurate the Newton di- rection employed, the closer the method converges to the optimal solution. For the same setup of the 36 method, the inference procedures based on the averaged iterate ( AvePlugIn, AveBM, AveRS) consis- tently yield the same mean absolute error, which is generally smaller, by half to one order of magni- tude...

work page 2021
[1951]

Roy and K

A. Roy and K. Balasubramanian. Online covariance estimation for stochastic gradient descent under markovian sampling.arXiv preprint arXiv:2308.01481,

arXiv
[1997]

K. M. Abadir and P. Paruolo. Simple robust testing of regression hypotheses: A comment. Econometrica, 70(5):2097–2099,

work page

Show all 13 references

[2008]

Z. Wei, W. Zhu, and W. B. Wu. Weighted averaged stochastic gradient descent: Asymptotic normality and optimality.arXiv preprint arXiv:2307.06915,

arXiv
[2014]

Kuang, M

W. Kuang, M. Anitescu, and S. Na. Online covariance matrix estimation in sketched newton methods.arXiv preprint arXiv:2502.07114,

arXiv
[2016]

C. Chen, F. Tung, N. Vedula, and G. Mori. Constraint-aware deep neural network compression. In Computer Vision – ECCV 2018, pages 409–424. Springer International Publishing,

2018
[2018]

Y. Fang, S. Na, M. W. Mahoney, and M. Kolar. Fully stochastic trust-region sequential quadratic programming for equality-constrained optimization problems.SIAM Journal on Optimization, 34 (2):2007–2037,

2007
[2023]

Jiang, A

L. Jiang, A. Roy, K. Balasubramanian, D. Davis, D. Drusvyatskiy, and S. Na. Online covariance estimation in nonsmooth stochastic approximation.arXiv preprint arXiv:2502.05305,

arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

  τY j=1 (I−K ⋆Sj(S⊤ j (K⋆)2Sj)†S⊤ j K⋆)     τY j=1 (I−K ⋆Sj(S⊤ j (K⋆)2Sj)†S⊤ j K⋆)   ⊤ # =∥Ξ ⋆∥ ·E

χ−β+0.5 <∞, where the last inequality is due to χ > β+ 0.5. By Kronecker’s lemma in Lemma A.5, we obtain the result in(d). •Proof of (e).The result in(e)is trivial due to Lemma B.2. B.2 Proof of Proposition 3.6 By the definitions of ¯Ξ⋆ in (17) and Ξ ⋆ in (14), we have ¯Ξ⋆ −Ξ ⋆ = (I−C ⋆)−1E h (I− eC⋆)Ξ⋆(I− eC⋆)⊤ i (I−C ⋆)−1 −Ξ ⋆ = (I−C ⋆)−1E[( eC⋆ −C ⋆)Ξ⋆...

work page 2023

[2] [6]

X. Li, J. Liang, and Z. Zhang. Online statistical inference for nonlinear stochastic approximation with markovian data.arXiv preprint arXiv:2302.07690,

arXiv

[3] [7]

Y. Luo, X. Huo, and Y. Mei. Covariance estimators for the root-sgd algorithm in online learning. arXiv preprint arXiv:2212.01259,

arXiv

[4] [10]

(c) The first result follows from the independence among {St,j}j, and the second result is from (Na and Mahoney, 2025, Corollary 5.4)

(b) The first result follows directly from (A.2) and the independence between randomness of ξt and {St,j}j; the second result is from (Na and Mahoney, 2025, (D.10)) . (c) The first result follows from the independence among {St,j}j, and the second result is from (Na and Mahoney, 2025, Corollary 5.4). ■ The next lemma provides bounds and convergence result...

work page 2025

[5] [12]

Combining the above two displays and plugging into (C.10), we complete the proof

β = Υ(1) µ βt. Combining the above two displays and plugging into (C.10), we complete the proof. D Additional Experimental Results In this section, we present the complete experimental results in Tables 3-8 for constrainedlinear and lo- gistic regression problems. We vary the dimension d∈ {5, 20, 40} and study three different design co- variances Σa: Iden...

work page 2020

[6] [20]

In other words, the more accurate the Newton di- rection employed, the closer the method converges to the optimal solution. For the same setup of the 36 method, the inference procedures based on the averaged iterate ( AvePlugIn, AveBM, AveRS) consis- tently yield the same mean absolute error, which is generally smaller, by half to one order of magni- tude...

work page 2021

[7] [1951]

Roy and K

A. Roy and K. Balasubramanian. Online covariance estimation for stochastic gradient descent under markovian sampling.arXiv preprint arXiv:2308.01481,

arXiv

[8] [1997]

K. M. Abadir and P. Paruolo. Simple robust testing of regression hypotheses: A comment. Econometrica, 70(5):2097–2099,

work page