Pith. sign in

REVIEW 3 major objections 1 minor 54 references

Toward Optimal Statistical Inference in Noisy Linear Quadratic Reinforcement Learning over a Finite Horizon

T0 review · 3 major / 1 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Noisy LQ reinforcement learning gets valid, online confidence intervals

desk verdict The paper is unverifiable from the supplied text — corrupted mojibake with the wrong arXiv header — but the abstract describes a plausible and important contribution that deserves a clean-copy review. read the letter →

arxiv 2508.08436 v1 pith:CLDORF7D submitted 2025-08-11 math.ST stat.TH

Bo Pan , Jianya Lu , Yafei Wang , Hao Li , Bei Jiang , Linglong Kong This is my paper
classification math.STstat.TH
keywords policygradientlinearquadraticregulatorreinforcementlearningstatisticalinferenceonlinebootstrapconfidenceintervalsasymptoticnormalityfinitehorizon
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper studies statistical inference for policy-gradient (PG) methods in finite-horizon linear-quadratic reinforcement learning, where the system has linear dynamics with known or unknown drift and quadratic costs. It claims that, under noisy observations, the PG estimator and its objective loss have exact asymptotic distributions, and it constructs an online bootstrap that updates the PG estimate together with randomly perturbed copies as data arrive. The bootstrap is shown to be distributionally consistent, and the resulting confidence intervals for the learned policy and expected cost are valid both asymptotically and in finite samples, with quantile approximation error of order $n^{-1/4}$. If correct, this gives practitioners a simple, principled way to quantify the uncertainty of learned LQ controllers in both offline and fully online settings. Note: the supplied full-text file is an unreadable encoding whose page header identifies a different arXiv ID, so the theorem-level statements could not be independently checked here.

What carries the argument

The central object is the online bootstrap for the policy-gradient estimate: maintain the PG update and a collection of randomly perturbed copies of it, and refresh all of them with each new observation. The load-bearing identity is the exact asymptotic distribution of the PG estimator (and of the objective loss), which lets the perturbed copies reproduce the estimator's sampling distribution. The $n^{-1/4}$ quantile-approximation rate is what allows finite-sample confidence statements.

What would settle it

Run the proposed online bootstrap on a scalar LQ system while driving the exploration-noise variance to zero at a fast rate; if the resulting confidence intervals under-cover as n grows (coverage below nominal by more than simulation noise), the excitation condition is violated and the asymptotic claim fails. Alternatively, simulate with a heavy-tailed disturbance (e.g., t-distributed with 3 degrees of freedom) and check whether the empirical coverage of the bootstrap intervals falls below the nominal level.

Watch

Extended reading notes

Core claim

The central claim is that uncertainty quantification for policy-gradient methods in noisy LQ RL is tractable. For linear dynamics with quadratic cost over a finite horizon, the paper derives exact asymptotic distributions for both the policy-gradient estimator and the corresponding objective loss. It then proposes an online bootstrap procedure: alongside the main PG estimate, maintain a set of randomly perturbed PG estimates, updating all of them as new observations arrive. The paper proves this procedure is distributionally consistent, meaning the bootstrap distribution tracks the estimator's sampling distribution, and that confidence intervals built from it have asymptotic and non-asymptot

Load-bearing premise

The load-bearing premise is that the noise and exploration are well behaved: the dynamics disturbance is zero-mean with sufficiently light tails and enough moments, and the exploration noise keeps the state covariance non-degenerate over the finite horizon, so the drift parameter and policy-gradient estimator are identifiable, asymptotically normal, and have an invertible covariance.

Editorial extensions

If this is right

  • Learned LQ policies can be reported with confidence intervals, not just point estimates, in both offline and streaming settings.
  • The $n^{-1/4}$ quantile error gives a concrete sample-size target for a desired interval precision.
  • The result covers unknown drift parameters, so it applies when the system model is partially identified from data.
  • It provides a template for uncertainty quantification in other differentiable RL objectives: plug in the estimator, add perturbed copies, and update them online.
  • In high-stakes control, operators can flag policies whose expected-cost intervals are too wide before deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The proof likely rests on a fixed-horizon martingale or CLT argument; extending to discounted infinite-horizon LQ may need a different mixing argument, since the asymptotic covariance changes.
  • The $n^{-1/4}$ rate is characteristic of bootstrap quantile accuracy under a CLT; a natural test is whether the coverage error in simulations decays as $n^{-1/4}$ rather than $n^{-1/2}$.
  • Heavy-tailed disturbance or weak exploration noise would break the normal asymptotics; a robust bootstrap (e.g., based on self-normalization) would be a natural follow-up.
  • The method should transfer to LQ regulators with stochastic cost or partial-state observations, but the exact covariance would need recomputation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 1 minor

Summary. The paper claims to establish statistical inference foundations for policy gradient (PG) methods in finite-horizon noisy Linear Quadratic Reinforcement Learning (LQ RL), for both known and unknown drift parameters. The abstract states that exact asymptotic distributions are derived for the PG estimator and the objective loss, and that an online bootstrap procedure, which updates perturbed PG estimates alongside the main estimate, is distributionally consistent and yields confidence intervals with both asymptotic and non-asymptotic validity. A specific quantile approximation rate of n^{-1/4} is claimed, with applicability to offline and fully online settings and numerical experiments. The only readable content is the abstract; the supplied full text is unreadable mojibake on every page, and the running headers identify the document as arXiv:2508.08441v3 [q-bio.QM], not arXiv:2508.08436. Consequently, no theorem statement, assumption, proof, algorithm description, or numerical result could be inspected.

Significance. If the results hold, the paper would address an important gap: uncertainty quantification for policy-gradient methods in LQ RL, an area where most existing work focuses on convergence rates rather than distributional inference. Exact asymptotics for PG estimators and an online bootstrap with a non-asymptotic n^{-1/4} quantile error would be a substantive theoretical contribution with clear practical relevance for high-stakes sequential decision-making. However, the evidentiary basis is currently absent: no mathematical claim can be verified from the provided manuscript. The paper cannot be credited for machine-checked proofs or reproducible code, because none are accessible. The potential significance is real, but the manuscript in its current form does not permit any technical assessment.

major comments (3)
  1. [Full text (all pages)] The supplied full text is unreadable mojibake on every page, and the page headers identify the document as arXiv:2508.08441v3 [q-bio.QM] (9 May 2026), not arXiv:2508.08436. No theorem statement, assumption list, proof, or experimental detail can be audited. This blocks every central claim in the abstract: exact asymptotic distributions, distributional consistency of the online bootstrap, asymptotic and non-asymptotic validity, and the n^{-1/4} quantile approximation rate. This is not a cosmetic defect; it is a load-bearing evidentiary problem that makes any substantive verification impossible.
  2. [Abstract] The abstract does not state the regularity conditions needed to support the claims: zero-mean and moment assumptions on the dynamics disturbance, sufficient excitation from the exploration noise, identifiability of the drift parameter, and invertibility of the asymptotic covariance of the PG estimator. These conditions are exactly what would determine whether the claimed n^{-1/4} rate follows rather than a slower Berry-Esseen-type bound, and whether the bootstrap is valid in the fully online setting. Because the assumptions section is unreadable, the central claims rest on unverifiable premises.
  3. [Numerical experiments] The abstract mentions numerical experiments across noisy linear dynamical systems, but no figures, tables, or readable text from the experiments can be inspected. Thus there is no way to assess the empirical support for the claimed finite-sample behavior or the practical performance of the proposed intervals.
minor comments (1)
  1. [Submission file] The uploaded PDF must be replaced with a legible version whose running headers match the arXiv identifier and subject class of this submission; the current rendering is unusable.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity identifiable in readable material; the supplied full text is unreadable mojibake from a different arXiv ID, so the body cannot be audited.

full rationale

The only legible portion is the abstract, which presents theorem-shaped claims: exact asymptotic distributions for the PG estimators and objective loss, distributional consistency of the online bootstrap, asymptotic and non-asymptotic validity, and an n^{-1/4} quantile-approximation rate. These are stated as derived outputs rather than as fitted inputs, and no fitted parameter is relabeled as a prediction in the readable text. There is also no visible self-citation invoked as load-bearing, no imported uniqueness theorem forcing a choice, and no ansatz smuggled in via citation. The supplied full text is corrupted mojibake whose page headers identify arXiv:2508.08441v3 [q-bio.QM], not arXiv:2508.08436; consequently no equation-level reduction (Eq. X = Eq. Y by construction) can be quoted or exhibited. Under the rule that circularity may be claimed only when the paper's own text permits exhibiting the specific reduction, no circular step can be scored from the available evidence. The corrupted rendering and the arXiv-ID mismatch are evidentiary and verifiability concerns, not a demonstration of circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The ledger is assembled from the abstract because the body is unreadable. The paper leans on four categories of unstated support: the LQ domain structure, regularity of noise and excitation, standard stochastic approximation machinery, and finite-horizon boundedness. No free parameters or invented entities are visible at abstract level; the unknown drift parameter is the object of inference, not an ad hoc fit. The honest summary: everything beyond the abstract's description is pulled from a body that could not be read, so the ledger's main entry is 'unverified regularity conditions.'

assumptions (4)
  • domain assumption Linear dynamics with additive zero-mean noise and quadratic cost; the optimal policy is linear and the PG objective has the standard LQ structure.
    Abstract: 'linear dynamics with both known and unknown drift parameters are controlled subject to a quadratic cost.' The entire inference framework is built for this setting and would not transfer to nonlinear systems.
  • domain assumption Regularity of noise and exploration: moments, sufficient excitation, identifiable drift, invertible asymptotic covariance.
    Required for exact asymptotics and bootstrap validity; the precise conditions are not readable in the supplied full text, so this is inferred from the abstract's claims of asymptotically normal estimators and n^{-1/4} rates.
  • standard math Standard stochastic approximation and bootstrap consistency machinery: suitable step sizes, Lipschitz gradient, and smoothness of the objective so the online bootstrap distribution converges to the same limit as the PG estimator.
    Distributional consistency of online bootstrap rests on such classical results; invoked implicitly by the abstract's 'we prove distributionally consistent' with no proof text available.
  • domain assumption Boundedness over the finite horizon: costs and state covariances do not blow up, so the objective and its gradient are well behaved.
    Finite-horizon LQ with noisy exploration requires controlled second moments; the abstract's finite-horizon framing is where this enters, with details unreadable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Toward Optimal Statistical Inference in Noisy Linear Quadratic Reinforcement Learning over a Finite Horizon." pith.science (2026). https://pith.science/paper/CLDORF7D

@misc{pith2026250808436,
  author       = {Pith},
  title        = {Pith review of: Toward Optimal Statistical Inference in Noisy Linear Quadratic Reinforcement Learning over a Finite Horizon},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CLDORF7D}},
  note         = {Machine review of arXiv:2508.08436}
}
abstract

Recent developments in Reinforcement learning have significantly enhanced sequential decision-making in uncertain environments. Despite their strong performance guarantees, most existing work has focused primarily on improving the operational accuracy of learned control policies and the convergence rates of learning algorithms, with comparatively little attention to uncertainty quantification and statistical inference. Yet, these aspects are essential for assessing the reliability and variability of control policies, especially in high-stakes applications. In this paper, we study statistical inference for the policy gradient (PG) method for noisy Linear Quadratic Reinforcement learning (LQ RL) over a finite time horizon, where linear dynamics with both known and unknown drift parameters are controlled subject to a quadratic cost. We establish the theoretical foundations for statistical inference in LQ RL, deriving exact asymptotics for both the PG estimators and the corresponding objective loss. Furthermore, we introduce a principled inference framework that leverages online bootstrapping to construct confidence intervals for both the learned optimal policy and the corresponding objective losses. The method updates the PG estimates along with a set of randomly perturbed PG estimates as new observations arrive. We prove that the proposed bootstrapping procedure is distributionally consistent and that the resulting confidence intervals achieve both asymptotic and non-asymptotic validity. Notably, our results imply that the quantiles of the exact distribution can be approximated at a rate of $n^{-1/4}$, where $n$ is the number of samples used during the procedure. The proposed procedure is easy to implement and applicable to both offline and fully online settings. Numerical experiments illustrate the effectiveness of our approach across a range of noisy linear dynamical systems.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

54 extracted references · 49 canonical work pages

  1. [1]

    Normal approximation for stochastic gradient descent via non-asymptotic rates of martingale clt

    Andreas Anastasiou, Krishnakumar Balasubramanian, and Murat A Erdogdu. Normal approximation for stochastic gradient descent via non-asymptotic rates of martingale clt. In Conference on Learning Theory , pages 115--137. PMLR, 2019

    work page 2019

  2. [2]

    Adaptive control

    Karl Johan str \"o m. Adaptive control. In Mathematical System Theory: The Influence of RE Kalman , pages 437--450. Springer, 1995

    work page 1995

  3. [3]

    Logarithmic regret for episodic continuous-time linear-quadratic reinforcement learning over a finite-time horizon

    Matteo Basei, Xin Guo, Anran Hu, and Yufei Zhang. Logarithmic regret for episodic continuous-time linear-quadratic reinforcement learning over a finite-time horizon. Journal of Machine Learning Research , 23(178):1--34, 2022

    work page 2022

  4. [4]

    Adaptive control with the stochastic approximation algorithm: Geometry and convergence

    Arthur Becker, P Kumar, and Ching-Zong Wei. Adaptive control with the stochastic approximation algorithm: Geometry and convergence. IEEE Transactions on Automatic Control , 30(4):330--338, 1985

    work page 1985

  5. [5]

    Dynamic programming and optimal control: Volume I , volume 4

    Dimitri Bertsekas. Dynamic programming and optimal control: Volume I , volume 4. Athena scientific, 2012

    work page 2012

  6. [6]

    Reinforcement learning applied to linear quadratic regulation

    Steven Bradtke. Reinforcement learning applied to linear quadratic regulation. Advances in Neural Information Processing Systems , 5, 1992

    work page 1992

  7. [7]

    Linear-quadratic mean-field reinforcement learning: convergence of policy gradient methods

    Ren \'e Carmona, Mathieu Lauri \`e re, and Zongjun Tan. Linear-quadratic mean-field reinforcement learning: convergence of policy gradient methods. arXiv preprint arXiv:1910.04295 , 2019

    arXiv 1910

  8. [8]

    Statistical inference for online decision making via stochastic gradient descent

    Haoyu Chen, Wenbin Lu, and Rui Song. Statistical inference for online decision making via stochastic gradient descent. Journal of the American Statistical Association , 116(534):708--719, 2021

    work page 2021

Show all 54 references
  1. [9]

    Statistical inference for model parameters in stochastic gradient descent

    Xi Chen, Jason D Lee, Xin T Tong, and Yichen Zhang. Statistical inference for model parameters in stochastic gradient descent. 2020

    2020

  2. [10]

    Robust inference via multiplier bootstrap

    Xi Chen and Wen-Xin Zhou. Robust inference via multiplier bootstrap. 2020

    2020

  3. [11]

    On the sample complexity of the linear quadratic regulator

    Sarah Dean, Horia Mania, Nikolai Matni, Benjamin Recht, and Stephen Tu. On the sample complexity of the linear quadratic regulator. Foundations of Computational Mathematics , 20(4):633--679, 2020

    2020

  4. [12]

    Challenges of reinforcement learning

    Zihan Ding and Hao Dong. Challenges of reinforcement learning. Deep Reinforcement Learning: Fundamentals, Research and Applications , pages 249--272, 2020

    2020

  5. [13]

    Challenges of real-world reinforcement learning: definitions, benchmarks and analysis

    Gabriel Dulac-Arnold, Nir Levine, Daniel J Mankowitz, Jerry Li, Cosmin Paduraru, Sven Gowal, and Todd Hester. Challenges of real-world reinforcement learning: definitions, benchmarks and analysis. Machine Learning , 110(9):2419--2468, 2021

    2021

  6. [14]

    On the convergence theory of debiased model-agnostic meta-reinforcement learning

    Alireza Fallah, Kristian Georgiev, Aryan Mokhtari, and Asuman Ozdaglar. On the convergence theory of debiased model-agnostic meta-reinforcement learning. Advances in Neural Information Processing Systems , 34:3096--3107, 2021

    2021

  7. [15]

    Online bootstrap confidence intervals for the stochastic gradient descent estimator

    Yixin Fang, Jinfeng Xu, and Lei Yang. Online bootstrap confidence intervals for the stochastic gradient descent estimator. Journal of Machine Learning Research , 19(78):1--21, 2018

    2018

  8. [16]

    Global convergence of policy gradient methods for the linear quadratic regulator

    Maryam Fazel, Rong Ge, Sham Kakade, and Mehran Mesbahi. Global convergence of policy gradient methods for the linear quadratic regulator. In International conference on machine learning , pages 1467--1476. PMLR, 2018

    2018

  9. [17]

    Convergence of policy gradient methods for finite-horizon exploratory linear-quadratic control problems

    Michael Giegrich, Christoph Reisinger, and Yufei Zhang. Convergence of policy gradient methods for finite-horizon exploratory linear-quadratic control problems. SIAM Journal on Control and Optimization , 62(2):1060--1092, 2024

    2024

  10. [18]

    Reinforcement learning for linear-convex models with jumps via stability analysis of feedback controls

    Xin Guo, Anran Hu, and Yufei Zhang. Reinforcement learning for linear-convex models with jumps via stability analysis of feedback controls. SIAM Journal on Control and Optimization , 61(2):755--787, 2023

    2023

  11. [19]

    Policy gradient methods for the noisy linear quadratic regulator over a finite horizon

    Ben Hambly, Renyuan Xu, and Huining Yang. Policy gradient methods for the noisy linear quadratic regulator over a finite horizon. SIAM Journal on Control and Optimization , 59(5):3359--3391, 2021

    2021

  12. [20]

    Bootstrapping upper confidence bound

    Botao Hao, Yasin Abbasi Yadkori, Zheng Wen, and Guang Cheng. Bootstrapping upper confidence bound. Advances in Neural Information Processing Systems , 32, 2019

    2019

  13. [21]

    Model-based or model-free, a review of approaches in reinforcement learning

    Qingyan Huang. Model-based or model-free, a review of approaches in reinforcement learning. In 2020 International Conference on Computing and Data Science (CDS) , pages 219--221. IEEE, 2020

    2020

  14. [22]

    Improved zeroth-order variance reduced algorithms and analysis for nonconvex optimization

    Kaiyi Ji, Zhe Wang, Yi Zhou, and Yingbin Liang. Improved zeroth-order variance reduced algorithms and analysis for nonconvex optimization. In International conference on machine learning , pages 3100--3109. PMLR, 2019

    2019

  15. [23]

    On incomplete learning and certainty-equivalence control

    N Bora Keskin and Assaf Zeevi. On incomplete learning and certainty-equivalence control. Operations Research , 66(4):1136--1167, 2018

    2018

  16. [24]

    Iterated least squares in multiperiod control

    Tze Leung Lai and Herbert Robbins. Iterated least squares in multiperiod control. Advances in Applied Mathematics , 3(1):50--73, 1982

    1982

  17. [25]

    Fast inference for quantile regression with tens of millions of observations

    Sokbae Lee, Yuan Liao, Myung Hwan Seo, and Youngki Shin. Fast inference for quantile regression with tens of millions of observations. Journal of Econometrics , page 105673, 2024

    2024

  18. [26]

    Statistical estimation and inference via local sgd in federated learning

    Xiang Li, Jiadong Liang, Xiangyu Chang, and Zhihua Zhang. Statistical estimation and inference via local sgd in federated learning. arXiv preprint arXiv:2109.01326 , 2021

    2021 arXiv

  19. [27]

    Unifying offline causal inference and online bandit learning for data driven decision

    Ye Li, Hong Xie, Yishi Lin, and John CS Lui. Unifying offline causal inference and online bandit learning for data driven decision. In Proceedings of the Web Conference 2021 , pages 2291--2303, 2021

    2021

  20. [28]

    A generalized reinforcement-learning model: Convergence and applications

    Michael L Littman and Csaba Szepesv \'a ri. A generalized reinforcement-learning model: Convergence and applications. In ICML , volume 96, pages 310--318, 1996

    1996

  21. [29]

    A review of uncertainty for deep reinforcement learning

    Owen Lockwood and Mei Si. A review of uncertainty for deep reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment , volume 18, pages 155--162, 2022

    2022

  22. [30]

    Gradient estimation in model-based reinforcement learning: a study on linear quadratic environments

    \^A ngelo Greg \'o rio Lovatto, Thiago Pereira Bueno, and Leliane Nunes de Barros. Gradient estimation in model-based reinforcement learning: a study on linear quadratic environments. In Brazilian Conference on Intelligent Systems , pages 33--47. Springer, 2021

    2021

  23. [31]

    Certainty equivalence is efficient for linear quadratic control

    Horia Mania, Stephen Tu, and Benjamin Recht. Certainty equivalence is efficient for linear quadratic control. Advances in Neural Information Processing Systems , 32, 2019

    2019

  24. [32]

    A linear quadratic regulator based speed control for remote-controlled racing cars

    Olusesi Ayobami Meadows, Arthur Rodriguez, and Ahmed Tijani Salawudeen. A linear quadratic regulator based speed control for remote-controlled racing cars. In 2022 IEEE Nigeria 4th International Conference on Disruptive Technologies for Sustainable Development (NIGERCON) , pag...

    2022

  25. [33]

    Black-box generalization: Stability of zeroth-order learning

    Konstantinos Nikolakakis, Farzin Haddadpour, Dionysis Kalogerias, and Amin Karbasi. Black-box generalization: Stability of zeroth-order learning. Advances in neural information processing systems , 35:31525--31541, 2022

    2022

  26. [34]

    Acceleration of stochastic approximation by averaging

    Boris T Polyak and Anatoli B Juditsky. Acceleration of stochastic approximation by averaging. SIAM Journal on Control and Optimization , 30(4):838--855, 1992

    1992

  27. [35]

    Temporal difference models: Model-free deep rl for model-based control

    Vitchyr Pong, Shixiang Gu, Murtaza Dalal, and Sergey Levine. Temporal difference models: Model-free deep rl for model-based control. arXiv preprint arXiv:1802.09081 , 2018

    2018 arXiv

  28. [36]

    Online bootstrap inference for policy evaluation in reinforcement learning

    Pratik Ramprasad, Yuantong Li, Zhuoran Yang, Zhaoran Wang, Will Wei Sun, and Guang Cheng. Online bootstrap inference for policy evaluation in reinforcement learning. Journal of the American Statistical Association , 118(544):2901--2914, 2023

    2023

  29. [37]

    The unintended consequences of discount regularization: Improving regularization in certainty equivalence reinforcement learning

    Sarah Rathnam, Sonali Parbhoo, Weiwei Pan, Susan Murphy, and Finale Doshi-Velez. The unintended consequences of discount regularization: Improving regularization in certainty equivalence reinforcement learning. In International Conference on Machine Learning , pages 28746--287...

    2023

  30. [38]

    Implicit bias of policy gradient in linear quadratic control: Extrapolation to unseen initial states

    Noam Razin, Yotam Alexander, Edo Cohen-Karlik, Raja Giryes, Amir Globerson, and Nadav Cohen. Implicit bias of policy gradient in linear quadratic control: Extrapolation to unseen initial states. arXiv preprint arXiv:2402.07875 , 2024

    2024 arXiv

  31. [39]

    Efficient estimations from a slowly convergent robbins-monro process

    David Ruppert. Efficient estimations from a slowly convergent robbins-monro process. Technical report, Cornell University Operations Research and Industrial Engineering, 1988

    1988

  32. [40]

    Gaussian approximation and multiplier bootstrap for polyak-ruppert averaged linear stochastic approximation with applications to td learning

    Sergey Samsonov, Eric Moulines, Qi-Man Shao, Zhuo-Song Zhang, and Alexey Naumov. Gaussian approximation and multiplier bootstrap for polyak-ruppert averaged linear stochastic approximation with applications to td learning. In The Thirty-eighth Annual Conference on Neural Infor...

    2024

  33. [41]

    Dynamic causal effects evaluation in a/b testing with a reinforcement learning framework

    Chengchun Shi, Xiaoyu Wang, Shikai Luo, Hongtu Zhu, Jieping Ye, and Rui Song. Dynamic causal effects evaluation in a/b testing with a reinforcement learning framework. Journal of the American Statistical Association , 118(543):2059--2071, 2023

  34. [42]

    Learning optimal controllers by policy gradient: Global optimality via convex parameterization

    Yue Sun and Maryam Fazel. Learning optimal controllers by policy gradient: Global optimality via convex parameterization. In 2021 60th IEEE Conference on Decision and Control (CDC) , pages 4576--4581. IEEE, 2021

    2021

  35. [43]

    Optimal scheduling of entropy regularizer for continuous-time linear-quadratic reinforcement learning

    Lukasz Szpruch, Tanut Treetanthiploet, and Yufei Zhang. Optimal scheduling of entropy regularizer for continuous-time linear-quadratic reinforcement learning. SIAM Journal on Control and Optimization , 62(1):135--166, 2024

    2024

  36. [44]

    Robust exploration in linear quadratic reinforcement learning

    Jack Umenberger, Mina Ferizbegovic, Thomas B Sch \"o n, and H kan Hjalmarsson. Robust exploration in linear quadratic reinforcement learning. Advances in Neural Information Processing Systems , 32, 2019

    2019

  37. [45]

    Distributed lqr design for identical dynamically coupled systems: Application to load frequency control of multi-area power grid

    Eleftherios E Vlahakis, Leonidas D Dritsas, and George D Halikias. Distributed lqr design for identical dynamically coupled systems: Application to load frequency control of multi-area power grid. In 2019 IEEE 58th Conference on Decision and Control (CDC) , pages 4471--4476. I...

    2019

  38. [46]

    First-order regret in reinforcement learning with linear function approximation: A robust estimation approach

    Andrew J Wagenmaker, Yifang Chen, Max Simchowitz, Simon Du, and Kevin Jamieson. First-order regret in reinforcement learning with linear function approximation: A robust estimation approach. In International Conference on Machine Learning , pages 22384--22429. PMLR, 2022

    2022

  39. [47]

    Residual bootstrap exploration for bandit algorithms

    Chi-Hua Wang, Yang Yu, Botao Hao, and Guang Cheng. Residual bootstrap exploration for bandit algorithms. arXiv preprint arXiv:2002.08436 , 2020

    2002 arXiv

  40. [48]

    Exact asymptotics for linear quadratic adaptive control

    Feicheng Wang and Lucas Janson. Exact asymptotics for linear quadratic adaptive control. Journal of Machine Learning Research , 22(265):1--112, 2021

    2021

  41. [49]

    Continuous-time mean--variance portfolio selection: A reinforcement learning framework

    Haoran Wang and Xun Yu Zhou. Continuous-time mean--variance portfolio selection: A reinforcement learning framework. Mathematical Finance , 30(4):1273--1308, 2020

    2020

  42. [50]

    Stochastic zeroth-order optimization in high dimensions

    Yining Wang, Simon Du, Sivaraman Balakrishnan, and Aarti Singh. Stochastic zeroth-order optimization in high dimensions. In International conference on artificial intelligence and statistics , pages 1356--1365. PMLR, 2018

    2018

  43. [51]

    Leveraging linear quadratic regulator cost and energy consumption for ultrareliable and low-latency iot control systems

    Haojun Yang, Kuan Zhang, Kan Zheng, and Yi Qian. Leveraging linear quadratic regulator cost and energy consumption for ultrareliable and low-latency iot control systems. IEEE Internet of Things Journal , 7(9):8356--8371, 2020

    2020

  44. [52]

    Provably global convergence of actor-critic: A case for linear quadratic regulator with ergodic cost

    Zhuoran Yang, Yongxin Chen, Mingyi Hong, and Zhaoran Wang. Provably global convergence of actor-critic: A case for linear quadratic regulator with ergodic cost. Advances in Neural Information Processing Systems , 32, 2019

    2019

  45. [53]

    Online covariance matrix estimation in stochastic gradient descent

    Wanrong Zhu, Xi Chen, and Wei Biao Wu. Online covariance matrix estimation in stochastic gradient descent. Journal of the American Statistical Association , 118(541):393--404, 2023

    2023

  46. [54]

    Uncertainty quantification and exploration for reinforcement learning

    Yi Zhu, Jing Dong, and Henry Lam. Uncertainty quantification and exploration for reinforcement learning. Operations Research , 72(4):1689--1709, 2024

    2024

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.