REVIEW 3 major objections 5 minor 12 references
A constrained online learner's own trajectory can certify a regret bound that is uniformly no worse, and often much tighter, than the standard worst-case guarantee.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 02:17 UTC pith:6D356UQJ
load-bearing objection The refined fixed-step bound is real and worth having; the AdaOGD-PFS O(√G_T)-with-feasibility claim is oversold — the paper's own Corollary 5 turns the leading term into O(√T) when gradients are benign. the 3 major comments →
Data-Dependent Regret and Polyak Corrections for Constrained Online Convex Optimization
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim: OGD with a Polyak feasibility step satisfies Reg_T ≤ 2R²/η + (η/2)G_T − (1/(2η))P_T + G_f ρ T/σ, where G_T is the trajectory's accumulated squared gradient norm and P_T is a nonnegative Polyak correction — the cumulative squared distance the feasibility projection moves the iterate toward the linearized constraint. The improvement over the prior bound, Δ_T = (η/2)(G_f²T − G_T) + P_T/(2η), is always nonnegative and splits into an environment-driven gradient refinement and an algorithm-driven projection-slack term. The adaptive variant replaces the fixed step size with η_t = c/√(ε₀ + past squared gradients), giving regret of order √(G_T+ε₀) under the paper's parameter choice
What carries the argument
The load-bearing object is the strong Pythagorean inequality for projection onto the halfspace H_t = {x: g(x_t)+s_tᵀ(x−x_t)+ρ≤0}: for any x in the shrunk feasible set, ‖x_{t+1}−x‖² ≤ ‖y_t−x‖² − δ_t, where δ_t = ‖y_t − Π_{H_t}(y_t)‖². This retained slack is the Polyak correction; summing it over rounds gives the negative term in the regret bound. The second device is the telescoping of the OGD distance recursion without replacing each ‖∇f_t(x_t)‖² by its worst-case value G_f², which preserves the data-dependent accumulation G_T.
Load-bearing premise
The claim that the adaptive method has both per-round feasibility and O(√G_T) regret depends on keeping the step-size regularizer small while still satisfying the constraint; the only regularizer the paper supplies for per-round feasibility grows linearly with T, which turns the leading term into O(√T) whenever the observed squared-gradient sum is much smaller than T, and the paper explicitly disclaims a schedule that avoids this.
What would settle it
Run AdaOGD-PFS on a long trajectory whose gradients are all small except rare spikes, so G_T/T→0. The only regularizer the paper gives for per-round feasibility is ε₀ = 2G_g²G_f²R²T/(ξ²α²) = Θ(T); the bound's leading term 2R√(2(G_T+ε₀)) is then at least 2R√(2ε₀) = Θ(√T), so it does not shrink with √G_T. If a schedule with ε₀=o(T) still satisfies g(x_t)≤0 for all t, the gap closes; otherwise the advertised 'O(√G_T) with per-round feasibility' claim is not delivered by the stated parameter choice.
If this is right
- Every run of the original fixed-step algorithm obtains a regret certificate that is uniformly no worse than the prior worst-case bound, and strictly tighter whenever gradients are not all maximal or the feasibility step is active at least once.
- The improvement splits into two auditable components, gradient refinement and Polyak correction; the paper's experiments show 34–37% from the former and 1–8% from the latter on standard instances, with the Polyak share rising to about 50% when gradients are adversarially held at worst case.
- AdaOGD-PFS achieves O(√G_T) regret instead of O(G_f√T), with leading constants independent of G_f, while preserving per-round feasibility when its regularizer is chosen as in the paper's Corollary 5.
- On instances where the linearized constraint is active in a constant fraction of rounds with bounded violation, P_T = Ω(T) makes the Polyak correction the dominant source of tightening; the paper's adversarial experiment reports actual regret 74 against a prior bound of 3,541 at T=10,000.
- Because G_T and P_T are computable online at zero extra oracle cost, the refined bound can be reported as an anytime, per-trajectory tightness gap during deployment.
Where Pith is reading between the lines
- The same strong-Pythagorean slack argument should transfer to any projected or proximal online method — ordinary projected OGD, alternating projections, multi-constraint Polyak steps — where a projection slack term can be subtracted from the regret bound. The paper gestures at this but does not quantify it.
- The practical payoff of AdaOGD-PFS is best framed as 'no worst-case gradient knowledge needed' rather than 'smaller bound': in every experiment the fixed-step bound with known G_f is numerically smaller than the adaptive bound. The O(√G_T) advantage only materializes when G_T=o(G_f²T) and G_f is unknown or overestimated.
- The Polyak correction can double as a step-size tuning diagnostic: P_t≈0 suggests the step is so conservative that the gradient step never crosses the linearized constraint, while a saturated P_t/(G_f²t) suggests overshoot. A doubling-trick wrapper could make the practical tuning self-certifying; the paper flags but does not formalize it.
- Because the refined bound is per-trajectory, it changes the nature of the guarantee from a pre-hoc worst-case statement to a post-hoc certificate; this is what makes it usable in safety-critical deployments where an operator wants to know how much slack remains after the fact.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies constrained online convex optimization with per-round feasibility, focusing on OGD with a Polyak feasibility step (OGD-PFS). Its main analytical contribution is Theorem 1: by retaining the observed gradient accumulation G_T = ∑‖∇f_t(x_t)‖² and the nonnegative Polyak correction P_T = ∑δ_t, the standard regret bound is tightened to Reg_T ≤ 2R²/η + (η/2)G_T − (1/(2η))P_T + G_f ρT/σ. The improvement over the prior worst-case bound is Δ_T = (η/2)(G_f²T − G_T) + P_T/(2η) ≥ 0. The paper further proposes AdaOGD-PFS (Algorithm 2), an AdaGrad-style variant, and claims O(√G_T) regret with per-round feasibility. Experiments on ball- and halfspace-constrained problems report 38–43% tighter bounds and decompose the gain into gradient-refinement and Polyak-correction parts.
Significance. If Theorem 1 is correct, it is a genuinely useful observation: the standard proof of OGD-PFS can be sharpened at zero algorithmic cost, yielding an auditable per-trajectory regret certificate involving observable quantities. The proof of Theorem 1 appears internally sound, and the experiments are transparent, including an adversarial sequence that isolates the Polyak correction. The adaptive part is less convincing: the advertised O(√G_T) rate with per-round feasibility is not obtained by the stated results when G_T is small. The paper deserves publication only after the claims are rescaled to what the theorems actually deliver.
major comments (3)
- [Abstract; §5.5, Corollary 5] The abstract and the §1 contribution bullet state that AdaOGD-PFS achieves O(√G_T) regret while preserving per-round feasibility. This combination is not supported by the paper's own results. Corollary 5 forces ε₀ = 2G_g²G_f²R²T/(ξ²α²) = Ω(T) to obtain per-round feasibility. Substituting into Eq. (13) gives a leading term 2R√2√(G_T + O(T)), which is O(√T) whenever G_T = o(T). Thus the data-dependent advantage of the adaptive method disappears exactly in the small-G_T regime where O(√G_T) would beat O(G_f√T). Remark 3 partially discloses this, but the abstract and the contribution summary omit the caveat. The headline claim should be revised to 'O(√(G_T + ε₀)) regret' or the feasibility guarantee should be separated from the data-dependent rate.
- [§1, §4.1, Corollary 5] The claim that AdaOGD-PFS requires 'no knowledge of G_f' is accurate for the regret rate in Theorem 2, but the per-round feasibility guarantee in Corollary 5 uses ε₀ = 2G_g²G_f²R²T/(ξ²α²), which depends on G_f and G_g. The main-text contribution bullet ('requiring no knowledge of G_f' together with 'preserving per-round feasibility') is therefore misleading. Remark 3 is commendable but appears only later; the abstract and contributions should carry the qualification that the regret rate is G_f-free while the feasibility certificate is not.
- [§5.5, Eq. (13); §5.4, Proposition 2(b)] The discussion around Proposition 2(b) and the 'adversarial experiment' should be made more careful. A constant linearized violation c>0 in every round is used to infer P_T = Ω(T) and hence a −Ω(T^{3/2}) correction under η = O(1/√T). Such a regime may be inconsistent with the geometric contraction of Lemma 3 and the bounded-gradient assumption for large T, since a single gradient step can move the iterate by only O(η). The proposition is conditional, but the interpretation in §5.4 as evidence of a fundamental non-tightness of the prior bound needs a concrete instance satisfying Assumptions 1–4 or an explicit caveat.
minor comments (5)
- [Corollary 5, Eq. (15)] Equation (15) appears to have a typesetting error: the leading term is printed as '2R√2GT + 2ϵ0', which is linear in G_T rather than √(G_T + ε₀). It should be 2R√2√(G_T + ε₀) (or the equivalent). The current display obscures the claimed rate.
- [Table 1] In the Theorem 2 row, the first regret term should be typeset with the square root spanning (G_T + ε₀), e.g., 2R√(2(G_T+ε₀)), for consistency with Eq. (13).
- [§7.2, Table 3] The abstract's '38–43%' improvement refers to Theorem 1; Theorem 2's improvement is 31–41% in Table 3. This should be stated explicitly in the abstract or experiments so that the reader does not attribute the larger number to AdaOGD-PFS.
- [Appendix B.2, Figure 4] The figure caption says '≈72% gap' while the text and Table 3 imply G_T/(G_f²T) ≈ 0.28, i.e., about 72% relative gap. The '76% gap' text in the figure panel is inconsistent; unify the number.
- [§5.4] The sentence 'whether Reg_T = O(1) holds rigorously on this subclass is left as a corollary-style consequence' is vague. Either state the precise conjecture or remove it.
Circularity Check
No circularity: the improvement terms are post-hoc algebraic identities from the algorithm's own iterates, and the AdaOGD-PFS caveat is a parameterization overclaim, not a derivation that reduces to its inputs.
full rationale
Walking the derivation chain: Theorem 1 (Eq. 6) is obtained by telescoping the strong Pythagorean inequality (Lemma 1), which is an external textbook inequality (Bauschke & Combettes, Cor. 4.10; Beck, Theorem 9.8), plus convexity and the data-dependent sums G_T and P_T defined in Definitions 1 and 2. The 'improvement' Delta_T = (eta/2)(G_f^2 T - G_T) + P_T/(2 eta) (Corollary 4) is an algebraic identity following from G_T <= G_f^2 T and P_T >= 0; it is a post-hoc certificate, not a fitted prediction. The paper itself states, 'Because P_T is a post-hoc quantity evaluated after execution, the resulting bounds are inherently instance-dependent rather than worst-case guarantees.' The AdaGrad-style O(sqrt(G_T)) bound (Theorem 2) is transparently inherited from Duchi et al. (2011) and Auer et al. (2002), and the paper explicitly disclaims novelty of the tuning principle ('we make no claim of a new data-dependent tuning principle'), so no ansatz is smuggled via citation. The self-citations (Zhang 2026a,b; Zhang et al. 2026) appear only in related-work positioning and do not support any load-bearing step. The only valid concern is the skeptic's: Corollary 5's feasibility choice epsilon_0 = Omega(T) makes the Theorem 2 leading term sqrt(G_T + O(T)), which is O(sqrt(T)) when G_T = o(T); but this is a scope/parameterization overstatement of the headline, expressly disclaimed in Remark 3 ('we explicitly disclaim that this paper provides a G_f-free alternative parameter choice that simultaneously achieves O(sqrt(G_T)) regret and per-round feasibility'), not a circular derivation. No step equates the conclusion to an input by construction.
Axiom & Free-Parameter Ledger
free parameters (5)
- η (step size) =
2R/(G_f√T) in Corollary 3; general η>0 in Theorem 1
- ρ (shrinkage) =
0 in experiments; α/√T in Corollaries 1 and 5
- c (AdaOGD scale) =
R√2 in Theorem 2 and Corollary 5
- ε₀ (AdaGrad regularizer) =
G_f² in experiments; 2G_g²G_f²R²T/(ξ²α²) in Corollary 5
- G_f (experimental envelope) =
max_t ‖∇f_t(x_t)‖ on the executed trajectory
axioms (6)
- domain assumption Assumptions 1–4: bounded X⊆R_B, ‖∇f_t‖≤G_f, ‖s‖≤G_g, and ‖s‖≥σ on the -ε level set
- domain assumption Known strictly feasible starting point with margin α (Corollaries 1 and 5)
- standard math Strong Pythagorean inequality ‖Π_C y − x‖² ≤ ‖y−x‖² − ‖y−Π_C y‖² (Bauschke & Combettes, Cor. 4.10)
- standard math Slater/normal-cone characterization (Rockafellar Thm 23.7) in Lemma 2
- standard math AdaGrad telescoping inequality ∑ b_t/√S_t ≤ 2(√S_{T+1}−√S_1) plus past-gradient correction
- domain assumption Oblivious adversary (cost sequence fixed before interaction)
read the original abstract
Constrained online convex optimization requires minimizing regret against adversarial convex costs while satisfying a convex constraint at every round, as needed in safety-critical applications. A computationally efficient method combines online gradient descent with a Polyak feasibility step, using one constraint evaluation and one subgradient per round. Although this method achieves O(sqrt(T)) regret with per-round feasibility, we derive a tighter, data-dependent analysis by retaining two quantities omitted by the standard worst-case argument. First, we replace the gradient envelope G_f^2 T with the observed accumulation G_T = sum_t ||grad f_t(x_t)||^2. Second, we identify a nonnegative Polyak correction P_T that measures the cumulative squared displacement caused by feasibility projections and enters the regret bound with a negative sign. The resulting improvement, Delta_T = (eta/2)(G_f^2 T - G_T) + P_T/(2 eta), is always nonnegative. We further propose AdaOGD-PFS, an adaptive-step-size method that achieves O(sqrt(G_T)) regret while preserving per-round feasibility. Experiments on ball- and halfspace-constrained problems improve the regret bound by 38 to 43 percent, with both data-dependent gradients and Polyak corrections contributing substantially.
Figures
Reference graph
Works this paper leans on
-
[2]
Projection-free bandit convex optimization
Lin Chen, Mingrui Zhang, and Amin Karbasi. Projection-free bandit convex optimization. InProceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, volume 89 of Proceedings of Machine Learning Research, pp. 2047–2056. PMLR,
2047
-
[3]
Figure 3 shows the percentage contribution of gradient refinement versus Polyak correction for each problem
B.2 Bound Component Decomposition Wefurthervisualizetheinternalstructureoftheboundimprovementfromthreecomplementaryperspectives. Figure 3 shows the percentage contribution of gradient refinement versus Polyak correction for each problem. Figure 4 traces the cumulative growth of bothGt andPt over time, revealing that the gap betweenGt andG 2 ft 22 widens s...
2000
-
[4]
Online convex optimization with time-varying constraints.arXiv preprint arXiv:1702.04783,
Michael J Neely and Hao Yu. Online convex optimization with time-varying constraints.arXiv preprint arXiv:1702.04783,
-
[7]
generalized Pythagorean projection
Rearranging:∥p t−x∥ 2≤∥y t−x∥ 2−δ t. Finally, sincex∈X ρ⊆RB, non-expansiveness ofΠ RB gives ∥xt+1−x∥ 2 =∥Π RB(pt)−x∥ 2≤∥p t−x∥ 2≤∥y t−x∥ 2−δt. To clarify the relation of this lemma to the generalized Pythagorean projection analysis, we note that the inequality∥p t−x∥ 2≤∥y t−x∥ 2−δt used above is thestrong(rather than weak) form of the Pythagorean inequali...
2017
-
[8]
Sinceg(v) =−ρ >−ϵ, we havev /∈C ϵ
Whenρ < ϵ: letCϵ ={u:g(u)≤−ϵ}. Sinceg(v) =−ρ >−ϵ, we havev /∈C ϵ. Letw= Π Cϵ(v). By the same boundary argument as Step 1,g(w) =−ϵ. By the normal cone characterization (Slater’s condition forC ϵ holds sinceming <−ϵ), there existµ >0andˆs w∈∂g(w)withv−w=µˆs w. Sinceg(w) =−ϵ, Assumption 4 gives∥ˆsw∥≥σ. By monotonicity of∂g(Rockafellar, 1970):(˜sv−ˆsw)⊤(v−w)≥...
1970
-
[9]
Therefore(B) = c 2 ∑ tbt/√St≤c (√GT +ϵ 0−√ϵ0 )
For non-negativebt :=∥∇f t(xt)∥2 andS t =ϵ 0 +∑ i<tbi: T∑ t=1 bt√St ≤2 (√ ST+1− √ S1 ) = 2 (√ GT +ϵ 0−√ϵ0 ) . Therefore(B) = c 2 ∑ tbt/√St≤c (√GT +ϵ 0−√ϵ0 ) . We note in passing that the inequality∑T t=1bt/√St≤2( √ ST+1−√S1)used above is the standard tele- scoping bound that holds when the denominator includes the currentbt, i.e.,St+1 instead ofSt (Duchi ...
2011
-
[10]
The past-gradient overheadcG 2 f/(2√ϵ0) evaluates, under the chosenϵ0 = 2G 2 gG2 fR2/(ξ2ρ2)andc=R √ 2, toξρG f/(2Gg); substitutingρ=α/ √ T givesξαG f/(2Gg √ T), which is absorbed into the leading √ Tterm in equation 15 and contributes a vanishing additive constant relative to the dominant2R√2GT + 2ϵ0 piece. B Additional Experiments B.1 AdaOGD-PFS vs Fixed...
2000
-
[12]
B.3 Sensitivity Analysis We examine how the bound improvement varies with the step sizeηand the shrinkage parameterρ
Right: cumulative Polyak correctionP t growing steadily over time, with the active constraint fraction (dashed) converging to≈0.97. B.3 Sensitivity Analysis We examine how the bound improvement varies with the step sizeηand the shrinkage parameterρ. Figure 6 shows that largerηincreases both sources of improvement (gradient refinement and Polyak correction...
2000
-
[2011]
Jacob Steinhardt and Percy Liang
doi: 10.1561/2200000018. Jacob Steinhardt and Percy Liang. Adaptivity and optimism: An improved exponentiated gradient algo- rithm. InProceedings of the 31st International Conference on Machine Learning, volume 32 ofProceedings of Machine Learning Research, pp. 1593–1601. PMLR,
-
[2016]
doi: 10.1561/2400000013. 16 Elad Hazan and Satyen Kale. Projection-free online learning. InProceedings of the 29th International Conference on Machine Learning, pp. 521–528,
-
[2017]
Amir Beck.First-order methods in optimization
doi: 10.1007/978-3-319-48311-5. Amir Beck.First-order methods in optimization. SIAM,
-
[2020]
Dynamic regret with untrusted decision predictions via heterogeneous expert aggregation
Wentao Zhang. Dynamic regret with untrusted decision predictions via heterogeneous expert aggregation. Transactions on Machine Learning Research, 2026a. ISSN 2835-8856. URLhttps://openreview.net/ forum?id=LWsEyfdnp9. Wentao Zhang. Multi-constraint online convex optimization with adversarial constraints.Transactions on Machine Learning Research, 2026b. ISS...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.