Online Optimization with Unknown Time-Varying Parameters from Noisy Gradient Measurements

Maziar Raissi; Shivanshu Tripathi

arxiv: 2605.22251 · v1 · pith:7VO3NMYSnew · submitted 2026-05-21 · 🧮 math.OC · cs.SY· eess.SY

Online Optimization with Unknown Time-Varying Parameters from Noisy Gradient Measurements

Shivanshu Tripathi , Maziar Raissi This is my paper

Pith reviewed 2026-05-22 04:33 UTC · model grok-4.3

classification 🧮 math.OC cs.SYeess.SY

keywords online optimizationtime-varying parametersnoisy gradientstracking errorGauss-Markov estimatorinstrumental variable estimatorparameter forecasting

0 comments

The pith

A bound on the expected tracking error is derived for online optimization with latent time-varying parameters reconstructed from noisy gradient measurements.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines online optimization where the cost depends on unmeasurable latent parameters that evolve according to unknown linear stochastic dynamics. Access is limited to finite noisy gradient measurements rather than direct parameter values. The proposed method reconstructs the parameters using a Gauss-Markov estimator, identifies the dynamics via an instrumental-variable estimator, and forecasts future parameters to locate the minimizer. This yields a bound on the expected tracking error between the algorithm output and the true time-varying minimizer. A sympathetic reader would care because the approach handles cases where parameters cannot be observed directly and their evolution law is unknown in advance.

Core claim

By reconstructing latent parameters from noisy gradient measurements with a Gauss-Markov estimator and identifying their unknown linear stochastic dynamics via an instrumental-variable estimator, the method forecasts future parameters and computes the minimizer such that the expected tracking error remains bounded.

What carries the argument

The combination of Gauss-Markov estimation for reconstructing latent parameters from noisy gradients and instrumental-variable estimation for identifying unknown linear stochastic dynamics, followed by forecasting to predict the minimizer.

If this is right

The expected tracking error between the algorithm output and the true minimizer stays bounded over time.
The approach succeeds even though the parameters remain unmeasurable and their dynamics are unknown.
The method applies to strongly convex costs whose linear term follows linear stochastic evolution.
Numerical examples confirm that the reconstruction and identification steps enable effective forecasting.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The technique could be tested against standard online optimizers that do not reconstruct or identify parameters explicitly.
Extensions might examine robustness when the linear stochastic assumption on dynamics is mildly violated.
In applied settings such as adaptive control, the bound could guide how often to collect new gradient measurements.

Load-bearing premise

The latent parameters can be sufficiently well reconstructed from finite noisy gradient measurements using the Gauss-Markov estimator and their dynamics can be identified via the instrumental-variable estimator despite the unknown linear stochastic evolution.

What would settle it

A numerical simulation or experiment in which the true expected tracking error grows without bound while the derived bound predicts otherwise would falsify the central claim.

Figures

Figures reproduced from arXiv: 2605.22251 by Maziar Raissi, Shivanshu Tripathi.

**Figure 2.** Figure 2: The figure reports the RMSE of the predicted optimizer over [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Figure 3: The figure shows the parameter estimation and dynamics-identification [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

read the original abstract

We study online optimization problems in which the cost function depends on latent, time-varying parameters that are unmeasurable and governed by unknown dynamics. Specifically, we consider a strongly convex cost function whose linear term evolves according to unknown linear stochastic dynamics, while the algorithm has access only to finite noisy gradient measurements. We propose a solution that uses control theoretic tools to reconstruct the latent parameters from gradient observations using a Gauss-Markov estimator, then identifies the parameter dynamics using an instrumental-variable estimator, and finally forecasts the parameters to compute the future minimizer. We provide a bound on the expected tracking error. We illustrate the effectiveness of our algorithm on a series of numerical examples.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper claims an expected tracking error bound for online convex optimization by reconstructing latent parameters from noisy gradients via Gauss-Markov, identifying unknown linear dynamics via instrumental variables, and forecasting the minimizer.

read the letter

The main thing to know is that this paper gives a bound on expected tracking error in online optimization when the cost depends on unmeasurable time-varying parameters that evolve according to unknown linear stochastic dynamics, and the only observations are finite noisy gradients. The approach reconstructs the parameters with a Gauss-Markov estimator, identifies the dynamics with an instrumental-variable estimator, and then forecasts ahead to track the moving minimizer. That specific sequence for this setting is the concrete new piece. It is not a routine extension of single-estimator work in the literature. The numerical examples show the method running on a few test cases and producing reasonable tracking, which is useful to see. The connection to control-theoretic tools for handling latent dynamics in adaptive systems is also drawn clearly enough. The central claim rests on the two estimators delivering small enough error so the bound closes. The instrumental-variable step typically requires instruments uncorrelated with process noise and persistent excitation of the regressors. With completely unknown dynamics and only noisy gradient measurements, those conditions are not automatic. If the paper does not supply explicit excitation requirements or a full error propagation analysis that accounts for finite samples, the bound may hold only under additional assumptions that are not stated up front. The abstract itself is thin on derivation steps, which is why the soundness looks limited from the summary alone. A reader working on online optimization or adaptive control with hidden parameter dynamics would find the pipeline and the bound worth examining. Someone already using Gauss-Markov or IV methods in related problems could pick up the forecasting step for their own tracking-error analysis. The work is coherent on its own terms and engages the relevant control and optimization literature without obvious internal contradictions. I would send it to peer review. The problem is practical and the proposed pipeline is specific enough that referees can check the conditions and tighten the analysis if needed.

Referee Report

2 major / 2 minor

Summary. The paper studies online optimization where a strongly convex cost depends on latent time-varying parameters evolving under unknown linear stochastic dynamics. Only finite noisy gradient measurements are available. The proposed three-step method reconstructs the latent parameters via a Gauss-Markov estimator, identifies the unknown dynamics via an instrumental-variable estimator, forecasts the minimizer, and derives a bound on the expected tracking error. Effectiveness is shown via numerical examples.

Significance. If the tracking-error bound is shown to hold under explicit, checkable conditions on the estimators, the work would usefully combine system-identification tools with online optimization. The explicit error bound and the use of standard but carefully combined estimators (Gauss-Markov plus instrumental variables) constitute a concrete, falsifiable contribution that could be applied to drifting-parameter problems in adaptive control and resource allocation.

major comments (2)

[§4.2, Theorem 1] §4.2, Theorem 1 (tracking-error bound): the proof assumes that the instrumental-variable estimator produces sufficiently small identification error, yet the manuscript does not state or verify the persistent-excitation condition on the regressors or the uncorrelatedness of the instruments with the process noise when the only observations are noisy gradients of the cost. Without these, the forecast error can grow and the bound does not close.
[§3.1, Eq. (8)] §3.1, Eq. (8) (Gauss-Markov estimator): the reconstruction error analysis is given for infinite data; the finite-sample bound used in the tracking-error derivation should be stated explicitly, including the dependence on the noise variance and the number of gradient measurements.

minor comments (2)

[§2] Notation for the linear stochastic evolution matrices (A, B, C) should be introduced once in §2 and used consistently thereafter.
[§5] Numerical examples: add a table reporting the observed tracking error versus the theoretical bound for different noise levels and horizons.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive report. The comments highlight important aspects that will strengthen the theoretical contributions of the paper. We respond to each major comment below.

read point-by-point responses

Referee: [§4.2, Theorem 1] §4.2, Theorem 1 (tracking-error bound): the proof assumes that the instrumental-variable estimator produces sufficiently small identification error, yet the manuscript does not state or verify the persistent-excitation condition on the regressors or the uncorrelatedness of the instruments with the process noise when the only observations are noisy gradients of the cost. Without these, the forecast error can grow and the bound does not close.

Authors: We agree that the conditions for the instrumental-variable estimator to yield small identification error need to be made explicit. In the revised version, we will add a new subsection or remark in §4.2 that states the persistent excitation condition on the regressors derived from the noisy gradient measurements and the requirement that the instruments are uncorrelated with the process noise. We will show that these conditions can be satisfied under standard assumptions on the excitation of the input and the noise properties, thereby ensuring that the identification error remains bounded and the tracking-error bound closes. revision: yes
Referee: [§3.1, Eq. (8)] §3.1, Eq. (8) (Gauss-Markov estimator): the reconstruction error analysis is given for infinite data; the finite-sample bound used in the tracking-error derivation should be stated explicitly, including the dependence on the noise variance and the number of gradient measurements.

Authors: The referee is correct that the current analysis in §3.1 focuses on the asymptotic behavior of the Gauss-Markov estimator. To address this, we will include an explicit finite-sample error bound for the parameter reconstruction in the revised manuscript. This bound will be derived using concentration inequalities and will explicitly depend on the gradient noise variance and the number of available measurements, which will then be propagated into the tracking-error analysis of Theorem 1. revision: yes

Circularity Check

0 steps flagged

No circularity: bound derived from standard estimators without reduction to inputs by construction

full rationale

The paper reconstructs latent parameters via Gauss-Markov estimator from noisy gradients, identifies unknown linear stochastic dynamics via instrumental-variable estimator, forecasts the minimizer, and derives a bound on expected tracking error. No quoted equation or step shows the tracking-error bound reducing to a fitted quantity defined by the same estimators, nor does any load-bearing premise rely on self-citation chains or ansatz smuggled from prior author work. The derivation draws on external control-theoretic tools and standard estimator properties under stated assumptions, remaining self-contained against external benchmarks as the central result is a proven bound rather than a renaming or self-referential prediction.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

Only the abstract is available, so the ledger is limited to explicitly stated modeling choices; no free parameters or invented entities are named, and the listed axioms are the core domain assumptions required for the problem formulation.

axioms (2)

domain assumption The cost function is strongly convex.
Stated as the setting for the online optimization problem.
domain assumption The linear term of the cost evolves according to unknown linear stochastic dynamics.
Defines the latent parameter process that must be reconstructed and forecasted.

pith-pipeline@v0.9.0 · 5640 in / 1314 out tokens · 60483 ms · 2026-05-22T04:33:20.904995+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

We propose a solution that uses control theoretic tools to reconstruct the latent parameters from gradient observations using a Gauss-Markov estimator, then identifies the parameter dynamics using an instrumental-variable estimator
IndisputableMonolith/Foundation/RealityFromDistinction reality_from_one_distinction unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

the unknown parameter vector θ(t) evolves under unknown linear, stochastic dynamics: θ(t+1)=Aθ(t)+w_p(t)

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

29 extracted references · 29 canonical work pages

[1]

P. M. Wensing, M. Posa, Y . Hu, A. Escande, N. Mansard, and A. Del Prete. Optimization-based control for dynamic legged robots. IEEE Transactions on Robotics, 40:43–63, 2024

work page 2024
[2]

D. P. Bertsekas.Dynamic Programming and Optimal Control, volume 2. Athena Scientific, 4th edition, 2018

work page 2018
[3]

F. Y . Jakubiec and A. Ribeiro. D-MAP: Distributed Maximum a Pos- teriori Probability Estimation of Dynamic Systems.IEEE Transactions on Signal Processing, 61(2):450–466, 2013

work page 2013
[4]

Zinkevich

M. Zinkevich. Online Convex Programming and Generalized Infinites- imal Gradient Ascent.International Conference on Machine Learning, 928–936, 2003

work page 2003
[5]

Shalev-Shwartz

S. Shalev-Shwartz. Online learning and online convex optimization. Foundations and Trends in Machine Learning, 4(2):107–194, 2012

work page 2012
[6]

E. C. Hall and R. M. Willett. Online convex optimization in dynamic environments.IEEE Journal of Selected Topics in Signal Processing, 9(4):647–662, 2015

work page 2015
[7]

S. C.H. Hoi, D. Sahoo, J. Lu, and P. Zhao. Online learning: A comprehensive survey.Neurocomputing, 459:249–289, 2021

work page 2021
[8]

Simonetto, E

A. Simonetto, E. Dall’Anese, S. Paternain, G. Leus, and G. B. Gian- nakis. Time-varying convex optimization: Time-structured algorithms and applications.Proceedings of the IEEE, 108(11):2032–2048, 2020

work page 2032
[9]

Dall’Anese, A

E. Dall’Anese, A. Simonetto, S. Becker, and L. Madden. Optimization and learning with information streams: Time-varying algorithms and applications.IEEE Signal Processing Magazine, 37(3):71–83, 2020

work page 2020
[10]

Simonetto, A

A. Simonetto, A. Mokhtari, A. Koppel, G. Leus, and A. Ribeiro. A class of prediction-correction methods for time-varying convex optimization. IEEE Transactions on Signal Processing, 64(17):4576–4591, 2016

work page 2016
[11]

Simonetto, and E

A. Simonetto, and E. Dall’Anese. Prediction-Correction Algorithms for Time-Varying Constrained Optimization.IEEE Transactions on Signal Processing, 65(20):5481–5494, 2017

work page 2017
[12]

Fazlyab, S

M. Fazlyab, S. Paternain, V . M. Preciado, and A. Ribeiro. Prediction- correction interior-point method for time-varying convex optimization. IEEE Transactions on Automatic Control, 63(7):1973–1986, 2017

work page 1973
[13]

Davydov, V

A. Davydov, V . Centorrino, A. Gokhale, G. Russo, and F. Bullo. Contracting dynamics for time-varying convex optimization.arXiv preprint arXiv:2305.15595, 2023

work page arXiv 2023
[14]

Bianchin and B

G. Bianchin and B. Van Scoy. The Internal Model Principle of Time- Varying Optimization.arXiv preprint arXiv:2407.08037, 2024

work page arXiv 2024
[15]

Bianchin and B

G. Bianchin and B. Van Scoy. The Discrete-time Internal Model Principle of Time-Varying Optimization: Limitations and Algorithm Design.IEEE Conference on Decision and Control (CDC), 2025

work page 2025
[16]

Bastianello, R

N. Bastianello, R. Carli, and S. Zampieri. Internal model-based online optimization.IEEE Trans. on Automatic Control, 69(1):689–696, 2024

work page 2024
[17]

Casti, N

U. Casti, N. Bastianello, R. Carli, and S. Zampieri. A control theoretical approach to online constrained optimization.Automatica, 176, 2025

work page 2025
[18]

Bianchin and B

G. Bianchin and B. Van Scoy. Feedback Optimization of Dynamical Systems in Time-Varying Environments: An Internal Model Principle Approach.arXiv preprint arXiv:2508.03503, 2025

work page arXiv 2025
[19]

Tripathi, A

S. Tripathi, A. A. Al Makdah, and F. Pasqualetti. Online Optimization with Unknown Time-varying Parameters.IEEE Control Systems Letters, 2024

work page 2024
[20]

Simchowitz, H

M. Simchowitz, H. Mania, S. Tu, M. I. Jordan, and B. Recht. Learning Without Mixing: Towards a Sharp Analysis of Linear System Identifi- cation.Conference on Learning Theory (COLT), 439–473, 2018

work page 2018
[21]

Tsiamis and G

A. Tsiamis and G. J. Pappas. Finite Sample Analysis of Stochastic System Identification.IEEE Conference on Decision and Control (CDC), 3648–3654, 2019

work page 2019
[22]

Sarkar and A

T. Sarkar and A. Rakhlin. Near optimal finite time identification of arbitrary linear dynamical systems.International Conference on Machine Learning (ICML), 5610–5618, 2019

work page 2019
[23]

S. Sun, W. Han, and X. Wang. Finite-sample identification of partially observed linear systems.arXiv preprint, 2025

work page 2025
[24]

S ¨oderstr¨om.Errors-in-Variables Methods in System Identification

T. S ¨oderstr¨om.Errors-in-Variables Methods in System Identification. Springer, 2018

work page 2018
[25]

Y . Zhou, X. Zhao, J. Liu, and N. Li. Instrumental-variable methods for stochastic linear system identification.IEEE Transactions on Automatic Control, 2025

work page 2025
[26]

Galrinho and H

A. Galrinho and H. Hjalmarsson. Refined instrumental-variable methods for system identification.Automatica, 2023

work page 2023
[27]

R. E. Kalman. A New Approach to Linear Filtering and Prediction Problems.Journal of Basic Engineering, 82(1):35–45, 1960

work page 1960
[28]

S. Hu, X. Lyu, P. Duan, D. Shi, and L. Shi. An Autocovariance Least- Squares-Based Data-Driven Kalman Filter for Unknown Systems.arXiv preprint arXiv:2505.19077, 2025

work page arXiv 2025
[29]

S. Liu, K. Chen, and J. Eising. Online Data-Driven Adaptive Con- trol for Unknown Linear Time-Varying Systems.arXiv preprint arXiv:2401.15278, 2024

work page arXiv 2024

[1] [1]

P. M. Wensing, M. Posa, Y . Hu, A. Escande, N. Mansard, and A. Del Prete. Optimization-based control for dynamic legged robots. IEEE Transactions on Robotics, 40:43–63, 2024

work page 2024

[2] [2]

D. P. Bertsekas.Dynamic Programming and Optimal Control, volume 2. Athena Scientific, 4th edition, 2018

work page 2018

[3] [3]

F. Y . Jakubiec and A. Ribeiro. D-MAP: Distributed Maximum a Pos- teriori Probability Estimation of Dynamic Systems.IEEE Transactions on Signal Processing, 61(2):450–466, 2013

work page 2013

[4] [4]

Zinkevich

M. Zinkevich. Online Convex Programming and Generalized Infinites- imal Gradient Ascent.International Conference on Machine Learning, 928–936, 2003

work page 2003

[5] [5]

Shalev-Shwartz

S. Shalev-Shwartz. Online learning and online convex optimization. Foundations and Trends in Machine Learning, 4(2):107–194, 2012

work page 2012

[6] [6]

E. C. Hall and R. M. Willett. Online convex optimization in dynamic environments.IEEE Journal of Selected Topics in Signal Processing, 9(4):647–662, 2015

work page 2015

[7] [7]

S. C.H. Hoi, D. Sahoo, J. Lu, and P. Zhao. Online learning: A comprehensive survey.Neurocomputing, 459:249–289, 2021

work page 2021

[8] [8]

Simonetto, E

A. Simonetto, E. Dall’Anese, S. Paternain, G. Leus, and G. B. Gian- nakis. Time-varying convex optimization: Time-structured algorithms and applications.Proceedings of the IEEE, 108(11):2032–2048, 2020

work page 2032

[9] [9]

Dall’Anese, A

E. Dall’Anese, A. Simonetto, S. Becker, and L. Madden. Optimization and learning with information streams: Time-varying algorithms and applications.IEEE Signal Processing Magazine, 37(3):71–83, 2020

work page 2020

[10] [10]

Simonetto, A

A. Simonetto, A. Mokhtari, A. Koppel, G. Leus, and A. Ribeiro. A class of prediction-correction methods for time-varying convex optimization. IEEE Transactions on Signal Processing, 64(17):4576–4591, 2016

work page 2016

[11] [11]

Simonetto, and E

A. Simonetto, and E. Dall’Anese. Prediction-Correction Algorithms for Time-Varying Constrained Optimization.IEEE Transactions on Signal Processing, 65(20):5481–5494, 2017

work page 2017

[12] [12]

Fazlyab, S

M. Fazlyab, S. Paternain, V . M. Preciado, and A. Ribeiro. Prediction- correction interior-point method for time-varying convex optimization. IEEE Transactions on Automatic Control, 63(7):1973–1986, 2017

work page 1973

[13] [13]

Davydov, V

A. Davydov, V . Centorrino, A. Gokhale, G. Russo, and F. Bullo. Contracting dynamics for time-varying convex optimization.arXiv preprint arXiv:2305.15595, 2023

work page arXiv 2023

[14] [14]

Bianchin and B

G. Bianchin and B. Van Scoy. The Internal Model Principle of Time- Varying Optimization.arXiv preprint arXiv:2407.08037, 2024

work page arXiv 2024

[15] [15]

Bianchin and B

G. Bianchin and B. Van Scoy. The Discrete-time Internal Model Principle of Time-Varying Optimization: Limitations and Algorithm Design.IEEE Conference on Decision and Control (CDC), 2025

work page 2025

[16] [16]

Bastianello, R

N. Bastianello, R. Carli, and S. Zampieri. Internal model-based online optimization.IEEE Trans. on Automatic Control, 69(1):689–696, 2024

work page 2024

[17] [17]

Casti, N

U. Casti, N. Bastianello, R. Carli, and S. Zampieri. A control theoretical approach to online constrained optimization.Automatica, 176, 2025

work page 2025

[18] [18]

Bianchin and B

G. Bianchin and B. Van Scoy. Feedback Optimization of Dynamical Systems in Time-Varying Environments: An Internal Model Principle Approach.arXiv preprint arXiv:2508.03503, 2025

work page arXiv 2025

[19] [19]

Tripathi, A

S. Tripathi, A. A. Al Makdah, and F. Pasqualetti. Online Optimization with Unknown Time-varying Parameters.IEEE Control Systems Letters, 2024

work page 2024

[20] [20]

Simchowitz, H

M. Simchowitz, H. Mania, S. Tu, M. I. Jordan, and B. Recht. Learning Without Mixing: Towards a Sharp Analysis of Linear System Identifi- cation.Conference on Learning Theory (COLT), 439–473, 2018

work page 2018

[21] [21]

Tsiamis and G

A. Tsiamis and G. J. Pappas. Finite Sample Analysis of Stochastic System Identification.IEEE Conference on Decision and Control (CDC), 3648–3654, 2019

work page 2019

[22] [22]

Sarkar and A

T. Sarkar and A. Rakhlin. Near optimal finite time identification of arbitrary linear dynamical systems.International Conference on Machine Learning (ICML), 5610–5618, 2019

work page 2019

[23] [23]

S. Sun, W. Han, and X. Wang. Finite-sample identification of partially observed linear systems.arXiv preprint, 2025

work page 2025

[24] [24]

S ¨oderstr¨om.Errors-in-Variables Methods in System Identification

T. S ¨oderstr¨om.Errors-in-Variables Methods in System Identification. Springer, 2018

work page 2018

[25] [25]

Y . Zhou, X. Zhao, J. Liu, and N. Li. Instrumental-variable methods for stochastic linear system identification.IEEE Transactions on Automatic Control, 2025

work page 2025

[26] [26]

Galrinho and H

A. Galrinho and H. Hjalmarsson. Refined instrumental-variable methods for system identification.Automatica, 2023

work page 2023

[27] [27]

R. E. Kalman. A New Approach to Linear Filtering and Prediction Problems.Journal of Basic Engineering, 82(1):35–45, 1960

work page 1960

[28] [28]

S. Hu, X. Lyu, P. Duan, D. Shi, and L. Shi. An Autocovariance Least- Squares-Based Data-Driven Kalman Filter for Unknown Systems.arXiv preprint arXiv:2505.19077, 2025

work page arXiv 2025

[29] [29]

S. Liu, K. Chen, and J. Eising. Online Data-Driven Adaptive Con- trol for Unknown Linear Time-Varying Systems.arXiv preprint arXiv:2401.15278, 2024

work page arXiv 2024