{"id":"e4e09cf3-080b-45cc-b15a-ac61d84201d9","arxiv_id":"2412.20017","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"SLIP is the first single-loop stochastic bilevel optimizer with eO(1/epsilon^4) oracle complexity under unbounded upper-level smoothness, both in expectation and with high probability.","lead":"The authors introduce SLIP, a single-loop algorithm for stochastic bilevel optimization with a nonconvex upper-level objective that has unbounded smoothness. It finds an epsilon-stationary point in eO(1/epsilon^4) stochastic gradient and Hessian-vector oracle calls, matching the best prior double-loop rate while being simpler to run.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The proof's lower-level error guarantee 1/(8L_1) can exceed Assumption 3.1's local radius 1/sqrt(2(L_{x,1}^2+L_{y,1}^2)) when L_{y,1} is large relative to L_{x,1}, so the relaxed-smoothness inequalities used in Lemma D.10 and Lemma C.4 are not justified as stated.","rationale":"The reader's weakest_assumption identifies exactly the same locality gap: the lower-level error bound 1/(8 L_1) is not guaranteed to lie inside Assumption 3.1's radius when L_{y,1} dominates L_{x,1}. I agree with that reading. The central algorithmic construction is otherwise plausible: the single-loop design is concrete, the proof appendix is substantial, the rate eO(1/epsilon^4) is internally consistent with the stated parameter choices, and the comparison against BO-REP is reasonable. The high-probability oracle assumption in Assumption 4.2 is also nonstandard, but it is secondary to the main expectation result and is not the root cause of the proof gap. The locality issue is load-bearing because Lemma D.10 and Lemma C.4 are essential to both Theorem 4.1 and Theorem 4.3, and no additional assumption or argument in the paper closes the gap. Since the reader already conditioned the verdict on resolving exactly this issue, my stress-test does not change the recommended verdict: the paper should remain CONDITIONAL pending a clarification or strengthening of Assumption 3.1, or a proof that the relevant inequalities hold beyond the stated local radius.","tokens_in":55623,"tokens_out":9023,"duration_ms":96776,"concrete_test":"Fix l_{g,1}/mu = 1, L_{x,1} = 1, L_{y,1} = 10, so Assumption 3.1's radius is R = 1/sqrt(202) ~ 0.0704 and Lemma 4.6 guarantees only ||y_t - y_t^*|| <= 1/(8 sqrt(2)) ~ 0.0884. Re-derive Lemma D.10's bias bound for this parameter instance and exhibit a pair (x, y_t), (x, y_t^*) at distance 0.0884 that satisfies Assumption 3.1 on the R-ball but violates the relaxed-smoothness inequality at that distance. If such a pair exists, the proof requires an explicit condition such as L_{y,1}^2 <= (32(1 + l_{g,1}^2/mu^2) - 1) L_{x,1}^2, or a separate slice-wise smoothness assumption; if no such pair exists, the locality gap is closed only by an argument not present in the paper.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Assumption 3.1 certifies the two gradient Lipschitz inequalities only for pairs w, w' with ||w - w'|| <= R := 1/sqrt(2(L_{x,1}^2 + L_{y,1}^2)). Lemma 4.6 / Lemma D.7 only guarantees ||y_t - y_t^*|| <= 1/(8 L_1), where L_1 = sqrt(1 + l_{g,1}^2/mu^2) L_{x,1}. When L_{y,1}^2 is large compared with L_{x,1}^2, R is much smaller than 1/(8 L_1). For example, with l_{g,1}/mu = 1 and L_{y,1} = 10 L_{x,1}, R ~ 0.0704/L_{x,1} while the guaranteed distance is ~ 0.0884/L_{x,1}, so the pair (x, y_t), (x, y_t^*) can lie outside the radius where Assumption 3.1 applies. Lemma D.10's hypergradient bias bound uses Assumption 3.1 at exactly this separation, and Lemma C.4's descent inequality is also invoked under the same local smoothness condition. No argument in the paper extends the relaxed-smoothness inequalities beyond the stated radius, and no condition is imposed relating L_{y,1} to L_{x,1} or the condition number. Thus, as written, the proof does not cover the claimed parameter regime. The same gap propagates to Theorem 4.3 through Lemma D.7 and to the z-tracking estimates that rely on the same event.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SLIP, a single-loop stochastic algorithm for bilevel optimization where the upper-level function is nonconvex with possibly unbounded smoothness (relaxed smoothness in the sense of Zhang et al.) and the lower-level function is strongly convex. The algorithm performs a short warm-start on the lower-level variable and then simultaneously updates x by normalized SGD with momentum, y by SGD, and z by a stochastic linear-system step. The main theoretical claims are (i) an Õ(1/ε^4) oracle complexity for finding an ε-stationary point of the bilevel problem, in expectation (Theorem 4.1), and (ii) the same complexity with high probability under an additional stronger noise assumption (Theorem 4.3). The paper also presents experiments on hyper-representation learning and data hyper-cleaning showing that SLIP outperforms several baselines.","tokens_in":56009,"tokens_out":7677,"duration_ms":71051,"significance":"If correct, the result is significant: it is the first single-loop algorithm for stochastic bilevel optimization under unbounded smoothness, and it matches the rate of the double-loop BO-REP algorithm (Hao et al., 2024) while being simpler and more practical. The proof strategy, which connects the lower-level tracking problem to stochastic optimization under distributional drift, is a nice conceptual contribution. The paper ships a full proof skeleton in the appendix, with detailed parameter choices and supporting lemmas, and its experiments include a fair comparison with multiple baseline methods. The high-probability guarantee (Theorem 4.3) is also presented as a first for this setting. However, the validity of the proofs is currently compromised by a locality-radius gap in the use of the relaxed-smoothness assumption, and the interpretation of the 'expectation' theorem is more subtle than the abstract suggests. These issues are technically fixable but require substantial revision.","major_comments":[{"comment":"The proof uses the relaxed-smoothness inequalities of Assumption 3.1 at separations that are not guaranteed to lie inside the radius where the assumption applies. Assumption 3.1 certifies the two gradient inequalities only for pairs w, w′ with ‖w−w′‖ ≤ R := 1/sqrt(2(L_{x,1}^2+L_{y,1}^2)). Lemma 4.6 only guarantees ‖y_t − y*_t‖ ≤ 1/(8L_1), where L_1 = sqrt(1+l_{g,1}^2/μ^2) L_{x,1}. When L_{y,1} is large compared with L_{x,1}, R is smaller than 1/(8L_1); for example, with l_{g,1}/μ = 1 and L_{y,1} = 10 L_{x,1}, R ≈ 0.0704/L_{x,1} while the guaranteed distance is ≈ 0.0884/L_{x,1}. Lemma D.10 (Eq. D.22) invokes Assumption 3.1 at exactly this separation to bound the hypergradient bias, and Lemma C.4's descent inequality is used under the same local-smoothness condition. No condition in the paper relates L_{y,1} to L_{x,1} or to the condition number, so the inequalities are not justified in the claimed parameter regime. The same gap propagates to the z-tracking estimates (Lemma D.8 and Lemma E.1) and hence to Theorem 4.3. The authors should either impose an explicit condition such as L_{y,1} = O(L_{x,1}) (up to constants) or prove that the iterates satisfy ‖y_t − y*_t‖ ≤ R under the stated assumptions.","section":"§4.3, Lemma 4.6 / Lemma D.7, Lemma D.10, Lemma C.4"},{"comment":"The statement 'converges to an ε-stationary point in expectation' is not a standard expectation guarantee. The theorem asserts that with probability at least 1−2δ over the randomness in σ(F̃^1_{T0} ∪ F^1_T), the conditional expectation satisfies (1/T) Σ E‖∇Φ(x_t)‖ ≤ 14ε, where the expectation is taken over F̃^e_T. This is a high-probability bound on a conditional expectation, not a bound on the unconditional expectation E[(1/T)Σ‖∇Φ(x_t)‖]. The abstract's 'both in expectation and with high probability' overstates the result; Theorem 4.1 is a weaker, conditional statement. The authors should restate the theorem as a conditional guarantee or provide an unconditional expectation bound by controlling the probability-2δ failure event. This distinction matters for the interpretation of the paper's headline rate.","section":"§4.2.1, Theorem 4.1"},{"comment":"The high-probability result relies on Assumption 4.2, which requires the almost-sure bound ‖(∇²_yy G − ∇²_yy g)z‖ ≤ σ_z for every z. This is a nonstandard and very strong oracle property: standard bounds on Hessian noise, such as bounded spectral norm of the Hessian error, would only give ‖(H−ĥ)z‖ ≤ σ‖z‖, which is not uniformly bounded over all z. The justification in Appendix E.1 constructs a noise matrix with entries proportional to τ/‖z‖, which is undefined at z = 0 and is not a plausible model of real stochastic Hessian-vector products. Since Lemma E.6 and Theorem 4.3 depend essentially on this assumption, the paper should either replace it with a more standard condition (e.g., bounded noise on the Hessian-vector product along the actual trajectory, which is plausible given the boundedness of z_t under the proven events) or discuss the realistic settings where the current Assumption 4.2 is satisfied.","section":"§4.2.2, Assumption 4.2 and Appendix E.1"}],"minor_comments":[{"comment":"Typo: 'output dimention' should be 'output dimension'.","section":"§5.1"},{"comment":"Typo: 'baselinses' should be 'baselines'.","section":"Figure 2 caption"},{"comment":"Grammar: 'a unbounded smooth' should be 'an unbounded smooth'.","section":"§1"},{"comment":"The phrase 'over over' appears in the guarantee; remove the duplicated 'over'.","section":"Theorem 4.1 statement"},{"comment":"The row for 'SLIP (This work, Theorem 4.3; high probability guarantee)' lists the stochastic setting as 'General expectation'; it should be labeled 'High probability' to distinguish it from the expectation result.","section":"Table 1"},{"comment":"The phrase 'the work of (Hao et al., 2024) designed' is awkward; use 'Hao et al. (2024) designed'.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper's main technical contribution is promising, but the locality-radius gap in the relaxed-smoothness argument is a genuine correctness issue that currently invalidates the proofs as stated. The fix (adding a relation between L_{y,1} and L_{x,1}, or strengthening the radius control) is within the scope of a major revision. I would also encourage the authors to be more careful in distinguishing the conditional expectation result from a true 'in expectation' result, and to reconsider the realism of Assumption 4.2. The experimental section is solid but lacks statistical significance claims; this is minor relative to the theory issues."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"SLIP is a real contribution, but the main proof has a load-bearing gap at the locality radius. The paper gives the first single-loop algorithm for stochastic bilevel optimization with unbounded upper-level smoothness, matching the double-loop BO-REP rate of eO(1/epsilon^4) and adding a high-probability guarantee. That alone justifies refereeing.\n\nWhat is good: the algorithm is genuinely simple and the analysis is novel. The lower-level error control in Lemma 4.6 — constant per-iterate error plus small weighted average — is a clever way to avoid BO-REP's requirement of an accurate y at every iteration. The connection to stochastic optimization under distributional drift is appropriate and new in bilevel settings. The appendix gives a complete proof skeleton with explicit parameter choices.\n\nThe problem: Assumption 3.1 certifies the two relaxed-smoothness inequalities only within radius R = 1/sqrt(2(L_x1^2 + L_y1^2)). The lower-level tracking bound in Lemma D.7 only gives ||y_t - y*_t|| <= 1/(8 L_1), where L_1 = sqrt(1 + l_g1^2/mu^2) L_x1. When L_y1 is large relative to L_x1, 1/(8L_1) exceeds R. For example, with l_g1/mu = 1 and L_y1 = 10 L_x1, the guaranteed distance is roughly 0.088/L_x1 while the radius is about 0.070/L_x1. Lemma D.10's bias bound and Lemma C.4's descent inequality invoke Assumption 3.1 at exactly this separation. No condition in the paper prevents this. As written, Theorem 4.1 does not cover the claimed parameter regime, and Theorem 4.3 inherits the issue through Lemma D.7.\n\nThe high-probability result relies on Assumption 4.2, an almost-sure bound on the Hessian noise in every direction. That is a spectral-norm bound, strong but not absurd, and it is not needed for the expectation result. The lack of error bars in the experiments is a minor issue.\n\nThe circularity concern is unfounded: the imported lemmas from the earlier BO-REP paper are published proofs used as tools, not the target conclusion.\n\nBottom line: this deserves a serious referee. The gap is real but likely fixable — either by adding a condition relating L_y1 to L_x1 or by redefining L_1 to include L_y1 at the cost of larger constants. I would recommend conditional accept or major revision, with the radius condition as the main point to resolve.","headline":"SLIP is a genuine first single-loop algorithm for unbounded-smoothness bilevel optimization, but the proof has a real locality-radius gap when L_y1 is large relative to L_x1, so it needs a fix before the claimed rate is fully supported.","tokens_in":56533,"tokens_out":5133,"would_cite":true,"duration_ms":48759,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C26","90C15","90C30"],"pacs":[],"model":"deepseek-v4-flash","headline":"SLIP is a single-loop stochastic bilevel optimizer that reaches an $\\epsilon$-stationary point in $\\widetilde{O}(1/\\epsilon^4)$ oracle calls under unbounded upper-level smoothness, matching the double-loop baseline and the…","keywords":["bilevel optimization","unbounded smoothness","relaxed smoothness","single-loop algorithm","stochastic optimization","distributional drift","hypergradient","meta-learning"],"falsifier":"Run SLIP on a problem where $L_{y,1}$ is much larger than $L_{x,1}$ and record, at each iterate, the distance $\\|y_t-y^*(x_t)\\|$ alongside the radius $1/\\sqrt{2(L_{x,1}^2+L_{y,1}^2)}$ within which the relaxed-smoothness bounds of Assumption 3.1 are certified; if the tracking ball radius $1/(8L_1)$ exceeds the certified radius while the algorithm still runs, then the descent inequality used in the proof is not guaranteed to hold at those iterates and the argument would need an additional condition.","tokens_in":55392,"feed_emoji":"🎯","tokens_out":9790,"duration_ms":89119,"temperature":0.7,"pith_summary":"This paper asks whether a single-loop algorithm can match the near-optimal oracle complexity of double-loop methods for stochastic bilevel optimization when the upper-level objective is nonconvex with unbounded smoothness, the regime that arises in meta-learning with recurrent neural networks. It proposes SLIP: a short logarithmic warm-start for the lower-level variable, then simultaneous updates of the lower-level variable by SGD, the linear-system estimator by SGD, and the upper-level variable by normalized stochastic gradient descent with momentum. The central claim is that SLIP finds an $\\epsilon$-stationary point in $\\widetilde{O}(1/\\epsilon^4)$ stochastic gradient or Hessian-vector product oracle calls, both in expectation and, under a stronger oracle assumption, with high probability; this matches the double-loop baseline and is nearly optimal up to logarithms. The insight that makes this possible is that the lower-level variable need not be accurate at every iteration: only a constant per-iteration tracking error plus a small momentum-weighted average is required, and the paper obtains both from a distributional-drift analysis.","feed_headline":"Single-loop bilevel optimizer hits near-optimal eO(1/epsilon^4)","feed_subtitle":"A single-loop stochastic bilevel method with expectation and high-probability guarantees under unbounded smoothness.","key_machinery":"The load-bearing object is the lower-level tracking error $\\|y_t - y^*(x_t)\\|$, analyzed through a distributional-drift lemma (Lemma 4.4) for SGD with a slowly moving minimizer. The lemma bounds the tracking error by the contraction factor $(1-\\mu\\alpha/2)^t$ plus a noise and drift term depending on the upper-level step radius $R$; applied to SLIP with $R=\\eta$, it yields Lemma 4.6: per-iteration error at most $1/(8L_1)$ and momentum-weighted average error at most $3\\epsilon/(32L_0)$. The other carrying mechanism is the moving-average hypergradient estimator $m_{t+1} = \\beta m_t + (1-\\beta)(\\nabla_x F(x_t,y_t;\\xi'_t) - \\nabla^2_{xy} G(x_t,y_t;\\zeta'_t)z_t)$ combined with the normalized update $x_{t+1}=x_t - \\eta m_{t+1}/\\|m_{t+1}\\|$, which fixes the step length $\\eta$ in the drift bound and converts the estimator bias into terms dominated by the weighted lower-level error and the gradient norm.","core_discovery":"On its own terms, the paper's discovery is that accurate estimation of the lower-level minimizer at every outer iteration—the property that forced earlier algorithms into a nested double loop—is not necessary for convergence under unbounded smoothness. SLIP instead controls $\\|y_t - y^*(x_t)\\|$ through a distributional-drift analysis: the lower-level SGD track the moving minimizer $y^*(x_t)$ because the normalized upper-level step keeps $\\|x_{t+1}-x_t\\| = \\eta$ small. Lemma 4.6 shows that with high probability the per-iteration tracking error stays below a constant $1/(8L_1)$, while the momentum-weighted average of the error is at most $O(\\epsilon)$. That weighted average is exactly what controls the bias term $L_{x,1}\\|y_t-y^*_t\\|\\|\\nabla\\Phi(x_t)\\|$ in the hypergradient estimator, so the moving-average hypergradient error can be bounded and the descent lemma closes. The resulting complexity is $\\widetilde{O}(\\epsilon^{-4})$ in expectation (Theorem 4.1) and with high probability (Theorem 4.3), which the paper argues is nearly optimal.","pith_inferences":["The analysis suggests a transferable principle: for single-loop bilevel methods, a constant per-iteration lower-level error plus a small momentum-weighted average may suffice in place of accurate inner solves, which could extend to other hypergradient estimators or to settings where the lower-level problem is not strongly convex.","Because the drift lemma only requires $\\|x_{t+1}-x_t\\|$ to be small, SLIP should tolerate a slowly time-varying lower-level objective; a natural extension would be nonstationary or continual bilevel optimization where $g$ changes between outer steps.","The proof's parameter choices are intricate, so a practical extension not tested in the paper is an adaptive or simplified step-size schedule that preserves the $\\widetilde{O}(1/\\epsilon^4)$ rate while removing the logarithmic tuning factors.","A concrete regime worth checking is $L_{y,1} \\gg L_{x,1}$: the locality radius in Assumption 3.1 shrinks with $L_{y,1}$, and whether the guaranteed tracking ball remains inside that radius determines whether the theorem covers the motivating recurrent-network examples."],"forward_implications":["If SLIP's guarantees hold as stated, it reaches an $\\epsilon$-stationary point in $\\widetilde{O}(1/\\epsilon^4)$ stochastic gradient or Hessian-vector oracle calls in expectation, with no mean-square smoothness assumption on the stochastic gradient oracle.","The same iteration complexity holds with probability at least $1-4\\delta$ under the stronger Assumption 4.2, giving, per the paper, the first high-probability convergence guarantee for stochastic bilevel optimization under unbounded smoothness.","The rate matches the double-loop baseline and is optimal up to logarithmic factors against the known $\\Omega(\\epsilon^{-4})$ lower bound for smooth nonconvex stochastic optimization, so further improvement would require additional structural assumptions.","The single-loop structure eliminates periodic lower-level refinement; the experiments report consistent wall-clock speedups over the double-loop baseline and other bilevel optimizers on hyper-representation learning and data hyper-cleaning.","The warm-start stage needs only a logarithmic number of SGD steps, so the algorithm avoids the polynomial-length initialization refinement used by earlier double-loop methods."],"supporting_citations":[{"why":"It defines the $(L_{x,0},L_{x,1},L_{y,0},L_{y,1})$-smoothness setting, provides the double-loop BO-REP baseline whose $\\widetilde{O}(1/\\epsilon^4)$ rate SLIP matches, and supplies key hypergradient and descent lemmas reused in the analysis.","marker":"(Hao et al., 2024)"},{"why":"It provides the distributional-drift SGD machinery, including the one-step improvement, distance recursion, and high-probability tracking results that Lemma 4.4 generalizes.","marker":"(Cutler et al., 2023)"},{"why":"It introduces relaxed smoothness and motivates it by the loss landscape of recurrent neural networks, which is the empirical basis for the unbounded-smoothness assumption.","marker":"(Zhang et al., 2020c)"},{"why":"It lays the foundation for stochastic bilevel optimization and derives the hypergradient formula $\\nabla\\Phi(x) = \\nabla_x f(x,y^*(x)) - \\nabla^2_{xy} g(x,y^*(x))z^*(x)$ that SLIP's estimator approximates.","marker":"(Ghadimi & Wang, 2018)"},{"why":"It supplies the $\\Omega(\\epsilon^{-4})$ lower bound for smooth nonconvex stochastic optimization, which the paper uses to claim that the $\\widetilde{O}(1/\\epsilon^4)$ rate is nearly optimal.","marker":"(Arjevani et al., 2023)"},{"why":"It provides the martingale concentration lemma used in the high-probability proof to control the cumulative hypergradient estimation error.","marker":"(Liu et al., 2023)"}],"fun_headline_variants":["SLIP: single-loop bilevel at near-optimal O(1/ε^4)","Single-loop bilevel: SLIP matches lower bounds under unbounded smoothness","Near-optimal bilevel without nested loops: SLIP's promise","SLIP: one loop for near-optimal bilevel","Unbounded smoothness? SLIP still gets O(1/ε^4)"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof's descent and bias bounds rely on the relaxed-smoothness inequalities of Assumption 3.1 holding near every iterate, but the paper certifies them only within a local radius that shrinks with $L_{y,1}$, while the tracking lemma guarantees the iterates stay within a different ball of radius $1/(8L_1)$; no stated condition ensures the guaranteed ball lies inside the certified ball.","fun_headline_variants_meta":{"raw":{"variants":["SLIP: single-loop bilevel at near-optimal O(1/ε^4)","Single-loop bilevel: SLIP matches lower bounds under unbounded smoothness","Near-optimal bilevel without nested loops: SLIP's promise","SLIP: one loop for near-optimal bilevel","Unbounded smoothness? SLIP still gets O(1/ε^4)"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000796,"raw_usage":{"total_tokens":3586,"prompt_tokens":1109,"completion_tokens":2477,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":725,"completion_tokens_details":{"reasoning_tokens":2377}},"tokens_in":725,"tokens_out":2477,"duration_ms":15644,"temperature":1.0,"reasoning_tokens":2377,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:41:01.890328+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SLIP on a problem where $L_{y,1}$ is much larger than $L_{x,1}$ and record, at each iterate, the distance $\\|y_t-y^*(x_t)\\|$ alongside the radius $1/\\sqrt{2(L_{x,1}^2+L_{y,1}^2)}$ within which the relaxed-smoothness bounds of Assumption 3.1 are certified; if the tracking ball radius $1/(8L_1)$ exceeds the certified radius while the algorithm still runs, then the descent inequality used in the proof is not guaranteed to hold at those iterates and the argument would need an additional condition.","supporting_citations":[{"cited_title":"Bilevel optimization under unbounded smoothness: A new algorithm and convergence analysis","cited_arxiv_id":null,"evidence_quote":"It defines the $(L_{x,0},L_{x,1},L_{y,0},L_{y,1})$-smoothness setting, provides the double-loop BO-REP baseline whose $\\widetilde{O}(1/\\epsilon^4)$ rate SLIP matches, and supplies key hypergradient and descent lemmas reused in the analysis."},{"cited_title":"Stochastic optimization under distributional drift","cited_arxiv_id":null,"evidence_quote":"It provides the distributional-drift SGD machinery, including the one-step improvement, distance recursion, and high-probability tracking results that Lemma 4.4 generalizes."},{"cited_title":"C., Foster, D","cited_arxiv_id":null,"evidence_quote":"It supplies the $\\Omega(\\epsilon^{-4})$ lower bound for smooth nonconvex stochastic optimization, which the paper uses to claim that the $\\widetilde{O}(1/\\epsilon^4)$ rate is nearly optimal."},{"cited_title":"Near-optimal non-convex stochastic optimization under generalized smoothness","cited_arxiv_id":null,"evidence_quote":"It provides the martingale concentration lemma used in the high-probability proof to control the cumulative hypergradient estimation error."}],"review_version":1}