Pith. sign in

REVIEW 4 major objections 4 minor 31 references

Maximizing the Value of Predictions in Control: Accuracy Is Not Enough

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read In the linear-quadratic regulator, prediction power is exactly a conditional-covariance sum, so two predictors with the same mean-square error can have very different control value.

desk verdict Solid LQR contribution, but the general lower-bound theorem is not proven as written because of a missing third-derivative condition. read the letter →

arxiv 2506.04497 v1 pith:ON75U5D6 submitted 2025-06-04 math.OC

classification math.OC MSC 93C5593E2049N10
keywords predictionpowerstochasticpredictionslinearquadraticregulatoronlineoptimalcontrolconditionalcovariancedecision-focusedlearningmodelpredictiveaccuracy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the value of a prediction in optimal control is set by the stochastic dependence between the prediction and the disturbance, not by how small the prediction error is. It defines prediction power as the drop in expected total cost from using a predictor optimally instead of ignoring it. In the time-varying linear-quadratic regulator, prediction power has a closed form: a sum over time of a trace of a cost-weighted conditional covariance of the optimal feedforward action. The same structure gives a general lower bound beyond LQR, and examples show that equal-accuracy predictors can differ sharply in the cost improvement they enable.

What carries the argument

The load-bearing quantity is the surrogate-optimal action $\bar{u}_t^{*}(\Xi)$, the action an oracle with full knowledge of future disturbances would take at time $t$. Prediction power equals the expected drop in the conditional covariance of this action when the predictor's extra information is used. In the general setting this exact identity is replaced by two sufficient conditions: a quadratic growth condition on the expected Q-function difference (Condition 4.1) and a positive expected conditional covariance of the optimal policy's action (Condition 4.2). Infimal convolution then propagates a Gaussian variance reduction in the disturbance into a lower bound on the action's covariance.

What would settle it

Simulate the time-varying LQR system of Theorem 3.2 for Gaussian disturbances and compare the measured cost difference between the optimal predictive and no-prediction policies with the closed-form trace formula; any systematic mismatch would refute the exact expression. For the general bound, exhibit a non-Gaussian pair $(W_t,V_t(\theta))$ with the same variance reduction $\lambda_t(\theta)>0$ as in Assumption 4.7 but with zero prediction power, which would falsify Theorem 4.8.

Watch

Extended reading notes

Core claim

The central claim is that in time-varying LQR, prediction power is $P(\theta)=\sum_{t=0}^{T-1}\operatorname{Tr}\{(R_t+B_t^{\top}P_{t+1}B_t)\,\mathbb{E}[\operatorname{Cov}(\bar{u}_t^{\theta}(I_t(\theta))\mid F_t(0))]\}$, where $\bar{u}_t^{\theta}$ is the feedforward part of the optimal policy. The paper rewrites this as the reduction in expected conditional covariance of the oracle optimal action $\bar{u}_t^{*}(\Xi)$ when conditioning on the predictor's history rather than the baseline history. It then proves a general lower bound of the same shape under two structural conditions, so the LQR formula is not an artifact of quadratic costs; the same covariance mechanism yields a strict lower bound on prediction power for well-conditioned non-quadratic costs.

Load-bearing premise

The strict-gain conclusion for general costs rests on pairing each disturbance with a jointly Gaussian prediction whose conditional variance is reduced by a positive amount, and the proof also quietly needs the control dimension to be no larger than the state dimension.

Editorial extensions

If this is right

  • Prediction power in LQR can be evaluated from data by regressing oracle actions on histories, avoiding nested conditional expectations.
  • Mean-square prediction error cannot rank predictors for control; improving prediction accuracy can even lower prediction power.
  • Online policy optimization can at best approach prediction power, and only when the optimal predictive policy lies in its policy class.
  • Under well-conditioned costs and Gaussian paired predictions, even weakly dependent predictions yield strictly positive cost improvement.
  • The general lower bound reduces comparing two policies over the whole horizon to per-step properties of the optimal predictive policy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural model-selection rule emerges: among predictors with similar error, prefer the one whose prediction covariance aligns with the cost-weighted directions $PHP$, since that alignment is what the LQR formula rewards.
  • The closed form suggests training predictors to maximize the trace term directly, a criterion that may differ substantially from minimizing prediction MSE.
  • The paper's roadmap for multi-step dependence indicates that the Gaussian-pairing assumption is a proof device rather than a structural necessity; if variance reduction can be propagated through infimal convolution, the strict-gain conclusion should extend to predictors that look several steps ahead.
  • The quantity $P(\theta)/T$ can serve as a practical benchmark for how much of the theoretically available improvement a concrete online policy optimizer achieves.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces a stochastic-dependence framework for predictions in online control, defining prediction power as P(θ) = J*(0) − J*(θ), the cost improvement obtained by optimally using a predictor θ relative to a no-prediction baseline. For time-varying LQR with quadratic costs, it derives an exact closed-form expression for P(θ) (Theorem 3.2) and uses it to show that prediction accuracy (MSE) does not determine control value, with numerical examples and a connection to online policy optimization. For general dynamics and costs, the paper states a sufficient-condition lower bound (Theorem 4.3) and instantiates it for LTV dynamics with strongly convex smooth costs and Gaussian paired predictions (Theorem 4.8). The LQR derivation is self-contained and convincing; the general lower bound, however, has several load-bearing proof gaps, including an unproven third-order smoothness requirement, a factor-of-2 constant issue, and a dimension mismatch in the covariance-transfer lemma.

Significance. If the general results can be repaired, this is a valuable contribution: it provides a computable metric for the benefit of stochastic predictions in LQR, identifies stochastic dependence between predictions and disturbances as the key quantity, and offers a modular sufficient-condition framework for non-quadratic problems. The exact LTV LQR formula alone is a solid advance over prediction-error-based analyses, and the paper gives reproducible simulation code for its examples. The authors are also careful to distinguish 'power of a policy class' from 'power of predictions', which helps clarify the relationship to prior work such as [25].

major comments (4)
  1. [Lemma 4.6 and Theorem 4.8] Lemma 4.6 states Condition 4.1 with M_t = µ_u I, but the proof in Appendix C.4 supports only M_t = (µ_u/2)I under the paper's standard strong-convexity convention (the same convention used in Lemma C.1 and Theorem D.1). Indeed, the conditional expected Q is minimized at π^θ_t and is at least µ_u-strongly convex in u, so the gap between evaluating it at u and at π^θ_t is at least (µ_u/2)‖u−π^θ_t‖². Since M_t enters linearly in Theorem 4.3 and Theorem 4.8, the stated lower bound is too large by a factor of 2; please correct the constant or explicitly adopt a nonstandard convention.
  2. [Lemma C.2 and Theorem D.1] The proof of Lemma C.2 obtains Cov[∇ω(X)] ⪰ σ₀ µ_ω² I by applying Theorem D.1 with g = ∇ω. Theorem D.1 additionally requires condition (53), i.e., a uniform bound on the Hessian of every component of g; for g = ∇ω this is a uniform bound on the third derivatives of ω. Assumption 4.5 bounds only second derivatives of h^x_t and h^u_t, and the infimal-convolution recursion in Lemma 4.6 is not shown to preserve any third-order bound. Strong convexity plus smoothness does not imply such a bound globally, so the covariance-transfer step, Eq. (14), and Theorem 4.8 are not established as written. The authors should either add an explicit third-order smoothness assumption on the costs and prove it is propagated through the recursion, or provide a different proof of Lemma C.2.
  3. [Lemma C.2 and Eq. (14)] The factor n in Lemma C.2 and in Eq. (14) appears to be the state dimension, but the trace of Cov[u(f□Bω)(X)] is over the m-dimensional action space. In the last step of the proof of Lemma C.3, the inequality Tr{B^T Cov[∇₁f(·)]B} ≥ n σ₀ σ_min(B)² should be Tr{B^T Cov[∇₁f(·)]B} ≥ m σ₀ σ_min(B)² (or rank(B)·σ₀σ_min(B)²), because B^T C B is m×m. Thus Eq. (14) should contain m rather than n, and the numerical bound in Theorem 4.8 is inflated whenever m < n.
  4. [Assumption 4.5 and Lemma C.2] The use of σ_min(B)² in Lemma C.2 and in the proof of Lemma C.3 requires B to have full column rank, hence m ≤ n and rank(B_t) = m for every t. Assumption 4.5 does not state this, and Theorem 4.8 does not inherit it from Lemma C.3. If B_t is rank-deficient or m > n, the bound in (14) is vacuous or ill-defined. Please add the dimension/full-rank condition explicitly to Assumption 4.5 (or to Theorem 4.8) and verify that it is needed at every step.
minor comments (4)
  1. [Appendix C.8 (proof of Lemma C.3)] The function c is used without definition in the proof of Lemma C.3; it should presumably be f (or h^u_t), and the smoothness constants in the displayed inequalities should be aligned with that choice.
  2. [Lemma 4.6 and Eq. (13)] The symbol b² in Eq. (13) is never defined; please define it explicitly, for example as an upper bound on ‖B_t‖², or write the expression in terms of ∥B_t∥².
  3. [Example 3.3 and Appendix A.5.1] The displayed matrix θ = [[1,0],[0.99,0.141]] has θθ^T with largest eigenvalue close to 1.99, which violates the stated constraint θθ^T ⪯ (1/2)I; please adjust the example parameters or the constraint so that the construction is consistent.
  4. [Appendix D.1] In the chain of inequalities for ∥R∥, the bound Lℓ√d γ^{3/4} appears to be typographically written as Lℓ d² γ^{3/4}; this does not affect the limiting argument but should be cleaned up for correctness.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the LQR formula and general lower bound are derived from the definitions of prediction power and the control problem via explicit algebra, not from fitted parameters or self-citation chains.

full rationale

I find no circularity in the paper's derivation chain. The central LQR result (Theorem 3.2) is derived by induction: Proposition 3.1 obtains the optimal policy and Q-function from the Bellman-style recursions (3), and Theorem 3.2 then sums per-step cost differences along the baseline trajectory. The quantity P(θ)=J*(0)-J*(θ) is defined independently of the formula, and the closed-form expression is an algebraic consequence, not a restatement of the definition. No parameter is fitted to data and then renamed as a prediction. The general lower bound Theorem 4.3 is a conditional implication: Conditions 4.1 and 4.2 describe growth of the optimal Q-function and the conditional covariance of the optimal action, neither of which mentions P(θ); the proof uses a performance-difference decomposition and obtains the bound without assuming the conclusion. Section 4.1 derives the sufficient conditions from Assumptions 4.5 and 4.7 using convex analysis and Lemma C.2; even if those technical lemmas contain gaps (e.g., the third-derivative condition in Theorem D.1 or the exact strong-convexity constant in Lemma 4.6), those are correctness concerns, not circularity, because the claimed bound does not reduce to the definition of prediction power. Self-citations are present but not load-bearing: [25] motivates the notion of prediction power and [15,16] support standard well-conditioned assumptions, but the central derivations are carried out in the paper. The evaluation algorithm in Appendix A.4 estimates conditional covariances from data, but it is an estimation procedure for the already-derived closed form, not a fitted parameter presented as a prediction. Overall, the paper is self-contained against external benchmarks, so the honest finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central theorems rely on standard dynamic programming plus the two structural conditions (quadratic growth, action covariance). No free parameters are fitted. The main limitations are domain assumptions about Gaussianity, stability, and control/state dimensions.

assumptions (4)
  • domain assumption Assumption 4.5: h^x_t is µ_x-strongly convex and ℓ_x-smooth, h^u_t is µ_u-strongly convex and ℓ_u-smooth, µ_A I ⪯ A_t^T A_t ⪯ ℓ_A I, µ_B I ⪯ B_t^T B_t ⪯ ℓ_B I, and ℓ_A < 1 (open-loop stability).
    Needed for Lemma 4.6 and for the infimal convolution to preserve strong convexity; restricts to open-loop stable well-conditioned systems.
  • domain assumption Assumption 4.7: (W_t, V_t(θ)) is jointly Gaussian and independent across time, with V_t(0)=0 and Cov[W_t] - Cov[W_t | V_t(θ)] ⪰ λ_t(θ) I for λ_t(θ) ≥ 0.
    Enables Lemma C.2's covariance transfer from prediction to action variance; limits predictions to single-time dependence on the current disturbance.
  • standard math Lemma C.2's infinite-decomposability property: the random input X can be written as a sum of i.i.d. summands with fourth moments O(N^{-2}), satisfied by Gaussian and Poisson distributions.
    Used in Theorem D.1 to pass covariance through gradients; restricts the disturbance distribution class.
  • ad hoc to paper Unstated dimension condition m ≤ n (control dimension no larger than state dimension).
    Lemma C.3 uses σ_min(B) > 0; if m > n, the lower bound is vacuous, but the paper never states this condition.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Maximizing the Value of Predictions in Control: Accuracy Is Not Enough." pith.science (2026). https://pith.science/paper/ON75U5D6

@misc{pith2026250604497,
  author       = {Pith},
  title        = {Pith review of: Maximizing the Value of Predictions in Control: Accuracy Is Not Enough},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ON75U5D6}},
  note         = {Machine review of arXiv:2506.04497}
}
read the original abstract

We study the value of stochastic predictions in online optimal control with random disturbances. Prior work provides performance guarantees based on prediction error but ignores the stochastic dependence between predictions and disturbances. We introduce a general framework modeling their joint distribution and define "prediction power" as the control cost improvement from the optimal use of predictions compared to ignoring the predictions. In the time-varying Linear Quadratic Regulator (LQR) setting, we derive a closed-form expression for prediction power and discuss its mismatch with prediction accuracy and connection with online policy optimization. To extend beyond LQR, we study general dynamics and costs. We establish a lower bound of prediction power under two sufficient conditions that generalize the properties of the LQR setting, characterizing the fundamental benefit of incorporating stochastic predictions. We apply this lower bound to non-quadratic costs and show that even weakly dependent predictions yield significant performance gains.

Figures

Figures reproduced from arXiv: 2506.04497 by the authors.

Figure 1
Figure 1. Example 3.4: Prediction Vt(1)is available. Candidate policy: ut = −Kxt + Υtvt(1). 0 20000 40000 60000 80000 Time step −0.5 0.0 0.5 1.0 1.5 2.0 2.5 Average cost improvement against ¯π Prediction power (P(2)/T) M-GAPS Optimal predictive policy (π 2 ) [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 3
Figure 3. An illustration of why predictions are helpful, corresponding to Example [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 5
Figure 5. Example 3.3: Control cost - ρ curve. A.5.1 Instantiation of Example 3.3 We instantiate Example 3.3 with the following parameters: A =  1 0.1 0 1  , B =  0 0.1  , Q =  1 1  , R = (1), and θ :=  1 0.99 0 0.141 . Under different values of coefficient ρ, we train a linear regressor to predict each entry of Wt from Vt(θ) (or Vt(I)) over a train dataset with 64000 independent samples. We plot in the MSE - ρ curve … view at source ↗
Figures from the paper (1 more)
Figure 6
Figure 6. Figure 6: Example A.2: MSE - time curve. A.6 Details of Example 3.4 We instantiate Example 3.4 with the same dynamics and costs as Example 3.3, i.e., A =  1 0.1 0 1  , B =  0 0.1  , Q =  1 1  , and R = (1). To build the predictors, we sample the true disturbance Wt i.i.d. …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

31 extracted references · 30 canonical work pages

  1. [25]

    Yu, C., Shi, G., Chung, S.-J., Yue, Y., and Wierman, A. (2020). The power of predictions in online control. Advances in Neural Information Processing Systems, 33:1994–2004

  2. [1]

    Agarwal, N., Bullins, B., Hazan, E., Kakade, S., and Singh, K. (2019). Online control with adversarial disturbances. InInternational Conference on Machine Learning, pages 111–119. PMLR

  3. [2]

    Amos, B., Jimenez, I., Sacks, J., Boots, B., and Kolter, J. Z. (2018). Differentiable MPC for End-to- end Planning and Control. InAdvances in Neural Information Processing Systems, volume 31. Curran Associates, Inc

  4. [3]

    (2017).First-order methods in optimization

    Beck, A. (2017).First-order methods in optimization. SIAM

  5. [4]

    Chen, N., Agarwal, A., Wierman, A., Barman, S., and Andrew, L. L. (2015). Online convex opti- mization using predictions. InProceedings of the 2015 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, pages 191–204. 11

  6. [5]

    Chen, N., Comden, J., Liu, Z., Gandhi, A., and Wierman, A. (2016). Using predictions in online optimization: Looking forward with an eye on the past.ACM SIGMETRICS Performance Evaluation Review, 44(1):193–206

  7. [6]

    Soda: Anadaptivebitratecontrollerforconsistenthigh-qualityvideostreaming

    Chen, T., Lin, Y., Christianson, N., Akhtar, Z., Dharmaji, S., Hajiesmaili, M., Wierman, A., and Sitaraman,R.K.(2024). Soda: Anadaptivebitratecontrollerforconsistenthigh-qualityvideostreaming. In Proceedings of the ACM SIGCOMM 2024 Conference, pages 613–644

  8. [7]

    L., Amos, B., and Kolter, J

    Donti, P. L., Amos, B., and Kolter, J. Z. (2017). Task-based End-to-end Model Learning in Stochastic Optimization. InAdvancesinNeuralInformationProcessingSystems ,volume30,LongBeach,CA,USA. Curran Associates, Inc

Show all 31 references
  1. [8]

    Predict, then Optimize

    Elmachtoub, A. N. and Grigas, P. (2022). Smart “Predict, then Optimize”.Management Science, 68(1):9–26

  2. [9]

    Hazan, E. et al. (2016). Introduction to online convex optimization.Foundations and Trends® in Optimization, 2(3-4):157–325

  3. [10]

    and Langford, J

    Kakade, S. and Langford, J. (2002). Approximately optimal approximate reinforcement learning. In Proceedings of the Nineteenth International Conference on Machine Learning, pages 267–274

  4. [11]

    Robustnessandconsistency in linear quadratic control with untrusted predictions.Proceedings of the ACM on Measurement and Analysis of Computing Systems, 6(1):1–35

    Li,T.,Yang,R.,Qu,G.,Shi,G.,Yu,C.,Wierman,A.,andLow,S.(2022). Robustnessandconsistency in linear quadratic control with untrusted predictions.Proceedings of the ACM on Measurement and Analysis of Computing Systems, 6(1):1–35

  5. [12]

    Li, Y., Chen, X., and Li, N. (2019). Online optimal control with linear dynamics and predictions: Algorithms and regret analysis.Advances in Neural Information Processing Systems, 32

  6. [13]

    Li, Y., Qu, G., and Li, N. (2018). Using predictions in online optimization with switching costs: A fast algorithm and a fundamental limit. In2018 Annual American Control Conference (ACC), pages 3008–3013. IEEE

  7. [14]

    Lin, Y., Goel, G., and Wierman, A. (2020). Online optimization with predictions and non-convex losses. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 4(1):1–32

  8. [15]

    Bounded-regretmpcviaperturbationanalysis: Prediction error, constraints, and nonlinearity

    Lin,Y.,Hu,Y.,Qu,G.,Li,T.,andWierman,A.(2022). Bounded-regretmpcviaperturbationanalysis: Prediction error, constraints, and nonlinearity. Advances in Neural Information Processing Systems, 35:36174–36187

  9. [16]

    Lin, Y., Hu, Y., Shi, G., Sun, H., Qu, G., and Wierman, A. (2021). Perturbation-based regret analysis ofpredictivecontrolinlineartimevaryingsystems. AdvancesinNeuralInformationProcessingSystems , 34:5174–5185

  10. [17]

    A., Anand, E

    Lin, Y., Preiss, J. A., Anand, E. T., Li, Y., Yue, Y., and Wierman, A. (2023). Online adaptive policy selection in time-varying systems: No-regret via contractive perturbations. InThirty-seventh Conference on Neural Information Processing Systems

  11. [18]

    Onlinepolicy optimization in unknown nonlinear systems

    Lin,Y.,Preiss,J.A.,Xie,F.,Anand,E.,Chung,S.-J.,Yue,Y.,andWierman,A.(2024). Onlinepolicy optimization in unknown nonlinear systems. InThe Thirty Seventh Annual Conference on Learning Theory, pages 3475–3522. PMLR

  12. [19]

    Mandi,J.,Kotary,J.,Berden,S.,Mulamba,M.,Bucarey,V.,Guns,T.,andFioretto,F.(2024).Decision- Focused Learning: Foundations, State of the Art, Benchmark and Future Opportunities.Journal of Artificial Intelligence Research, 80:1623–1701. 12

  13. [20]

    Marsden, J. E. and Tromba, A. (2003).Vector calculus. Macmillan

  14. [21]

    Neural-fly enables rapid learning for agile flight in strong winds.Science Robotics, 7(66):eabm6597

    O’Connell,M.,Shi,G.,Shi,X.,Azizzadenesheli,K.,Anandkumar,A.,Yue,Y.,andChung,S.-J.(2022). Neural-fly enables rapid learning for agile flight in strong winds.Science Robotics, 7(66):eabm6597

  15. [22]

    Rutten, D., Christianson, N., Mukherjee, D., and Wierman, A. (2023). Smoothed online optimization withunreliablepredictions. ProceedingsoftheACMonMeasurementandAnalysisofComputingSystems , 7(1):1–36

  16. [23]

    Wang, I., Becker, C., Van Parys, B., and Stellato, B. (2024). Learning Decision-Focused Uncertainty Sets in Robust Optimization

  17. [24]

    End-to-endconformalcalibration for optimization under uncertainty.arXiv preprint arXiv:2409.20534

    Yeh,C.,Christianson,N.,Wu,A.,Wierman,A.,andYue,Y.(2024). End-to-endconformalcalibration for optimization under uncertainty.arXiv preprint arXiv:2409.20534

  18. [26]

    Yu, C., Shi, G., Chung, S.-J., Yue, Y., and Wierman, A. (2022). Competitive control with delayed imperfect information. In2022 American Control Conference (ACC), pages 2604–2610. IEEE

  19. [27]

    Zhang, R., Li, Y., and Li, N. (2021). On the regret analysis of online lqr control with predictions. In 2021 American Control Conference (ACC), pages 697–703. IEEE. 13 Appendices A Proofs and Examples for LTV Dynamics with Quadratic Costs A.1 Proof of Proposition 3.1 Recall th...

  20. [28]

    u0 + u1 = −1

    + E [h2(u0 + u1 + W1)] = min u0,u1 2u2 0 + u2 1 + 1, s.t. u0 + u1 = −1. 26 Theequationholdsbecausetheplannedtrajectorymustavoidthehugecostattimestep 2. Solvingthisgives u0 = − 1

  21. [29]

    Thus, implementing MPC incurs a total cost that is at least2u2 0 = 2

  22. [30]

    In this case, the expected cost incurred isp

    In contrast, if one just pick u0 = 0, the agent can picku1 based on the prediction revealed at time step2: u1 = ( 0 if W1 = 0, −1 otherwise. In this case, the expected cost incurred isp. Thus, we can claim that MPC is not the optimal policy when p <2

  23. [31]

    It(θ) =

    TheunderlyingreasonthatMPCissuboptimalisbecauseitdoesnotconsiderwhatinformationmay be available when we make the decision in the future. In this specific example, sinceW1 is revealed at time 1, we don’t need to verify about the small probability event that leads to a huge loss...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.