Pith. sign in

REVIEW 4 major objections 5 minor 34 references

Zeroth-Order Optimization is Secretly Single-Step Policy Optimization

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proves that Gaussian-smoothed zeroth-order optimization is exactly single-step REINFORCE with a baseline, and uses that equivalence to build a faster ZOO algorithm, ZoAR, that averages recent function evaluations and reuses past…

desk verdict Gaussian ZOO=REINFORCE is a correct but mostly re-labeling insight; ZoAR shows promise, but its theoretical support has a concrete sphere-sampling error and a convergence bound that doesn't close. read the letter →

arxiv 2506.14460 v1 pith:4Z4IFJ7U submitted 2025-06-17 cs.LG

Junbin Qiu , Zhengpeng Xie , Xiangda Yan , Yongjie Yang , Yao Shu This is my paper · ORCID
classification cs.LG MSC 90C5668T05
keywords zeroth-orderoptimizationREINFORCEpolicygradientbaselinesubtractionqueryreuseGaussiansmoothingvariancereductionfinitedifferences
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proves that Gaussian-smoothed zeroth-order optimization (ZOO) is exactly single-step REINFORCE with a baseline, not merely analogous to it. On the objective side, the smoothed function that ZOO implicitly minimizes is identical to the expected reward of a single-step policy whose action is a Gaussian perturbation of the parameters. On the gradient side, the standard finite-difference estimator equals the REINFORCE estimator with the baseline set to the function value at the current parameters. If this equivalence is correct, the entire toolbox of policy-gradient variance reduction transfers directly to ZOO, and the paper exhibits one such transfer in the ZoAR algorithm, which uses an averaged baseline and query reuse to cut gradient variance and speed convergence.

What carries the argument

The load-bearing identity is the score-function form of the Gaussian perturbation: with $x=\theta+\mu u$ and policy $\pi_\theta(x)=\mathcal{N}(\theta,\mu^2 I_d)$, one has $\nabla_\theta\ln\pi_\theta(x)=(x-\theta)/\mu^2=u/\mu$. Substituting $x_k=\theta+\mu u_k$ into the REINFORCE estimator $(9)$ turns it term-by-term into the ZOO finite-difference estimator $(2)$ when the baseline is $f(\theta;\xi)$, which is the paper's Theorem 3.2. The same substitution proves the objective equivalence (Theorem 3.1), and the importance-sampling extension (Theorem 3.3) computes the ratio $\gamma=\pi_\theta(x_k)/p(x_k)$ for Gaussian, uniform-on-sphere, and coordinate-basis proposals, giving the learning-rate rescaling rule of Corollary 3.4.

What would settle it

Run ZoAR with history depth $N>1$ on a simple quadratic objective in $d=10$ with a fixed learning rate and compare the empirical mean-squared error of the gradient estimate to the bound of Theorem B.5; if the squared-bias term does not scale as predicted with $\eta^2 d^2 (N-1)/(1-\beta_2)$ and instead dominates the variance at practical settings, the bias-control claim fails. Separately, test the optimal-baseline formula $b_t^*=\frac{1}{N}\sum_{n=1}^N F_\mu(\theta_{t-n})$ under $u\sim\mathcal{N}(0,I_d)$: since the proof requires $\|u\|^2=1$, a direct variance computation with Gaussian perturbations would show whether the averaged baseline remains variance-minimizing or needs a distribution-dependent correction.

Watch

Extended reading notes

Core claim

The central claim is that the Gaussian-smoothed ZOO objective $F_\mu(\theta)=\mathbb{E}_{u}[F(\theta+\mu u)]$ equals the single-step policy optimization objective $J(\theta)=\mathbb{E}_{x\sim\pi_\theta}[F(x)]$ when the policy is the reparameterized Gaussian $x=\theta+\mu u$ with $u\sim\mathcal{N}(0,I_d)$. Building on that, the paper proves the gradient estimator $\hat\nabla F(\theta)=\frac{1}{K}\sum_k \frac{f(\theta+\mu u_k;\xi)-f(\theta;\xi)}{\mu}u_k$ is identical to the REINFORCE estimator with Gaussian policy and baseline $b(\xi)=f(\theta;\xi)$, revealing that the subtraction of $f(\theta;\xi)$ in ZOO is exactly a REINFORCE variance-reducing baseline. For non-Gaussian samplers, the paper extends the equivalence through importance sampling, obtaining a scaled identity $\hat\nabla_{\mathrm{IS}}J(\theta)=\gamma\hat\nabla F(\theta)$ with explicit $\gamma$ factors, and shows that ZOO and REINFORCE achieve identical convergence when learning rates are scaled by $\gamma$. These equivalences are the paper's first contribution; the second is ZoAR, which replaces the single-point baseline with an average of recent function values in a history buffer and reuses all buffered queries in the gradient estimate, with theorems bounding the resulting variance reduction, bias, and convergence.

Load-bearing premise

The algorithmic gain of ZoAR rests on the premise that the bias introduced by reusing historical queries stays small at practical learning rates; the paper's formal bound (Theorem C.4) leaves a constant floor $B_2$ depending on gradient variance, and the remark asserts without a quantitative guarantee that 'this bias can be small with a small learning rate $\eta$.'

Editorial extensions

If this is right

  • If ZOO estimators are REINFORCE estimators, then every variance-reduction technique developed for policy gradients, such as learned baselines, critics, or GAE-style advantage estimation, becomes a candidate for improving ZOO sample efficiency.
  • The baseline view explains why the subtraction of $f(\theta;\xi)$ helps in practice: it is not a finite-difference artifact but a provably variance-reducing REINFORCE baseline, and the averaged baseline in ZoAR is a Monte Carlo estimate of the variance-minimizing baseline $\mathbb{E}[R(x)]$.
  • Query reuse, analogous to experience replay, increases the effective batch size without additional function queries, with a quantifiable bias-variance trade-off: larger history depth $N$ reduces variance but adds bias that scales with $\eta^2 d^2 (N-1)/(1-\beta_2)$, so a small learning rate keeps the bias small.
  • For non-Gaussian perturbation distributions, the importance-sampling scaling factors $\gamma$ prescribe principled learning-rate rescaling, meaning a single ZOO implementation can switch sampling distributions while preserving the same effective convergence trajectory.
  • The convergence guarantee of ZoAR depends directly on the gradient variance $V=(\sigma_\xi^2+\sigma_\mu^2)/(NK\mu^2)$, so any further reduction of $V$ yields a directly proportional speedup in the variance-dominated regime.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next step the authors leave implicit is to replace the stored-average baseline with a parameterized or learned estimate of the expected function value, effectively importing actor-critic ideas into ZOO; this could reduce bias when the objective landscape shifts quickly.
  • The theorem's optimal-baseline result is proven for $u\sim\mathrm{Unif}(S^{d-1})$ where $\|u\|^2=1$; for $u\sim\mathcal{N}(0,I_d)$ the same formula may not be exactly variance-minimizing, so the optimal baseline likely depends on the perturbation distribution, suggesting a distribution-specific baseline design.
  • The bias from query reuse grows with history depth $N$ and learning rate $\eta$, so a linear or adaptive decay schedule for the history length, which the paper mentions as future work, would likely trade a small loss of variance reduction for a large gain in bias control.
  • If the equivalence holds for single-step REINFORCE, it should extend to multi-step settings: a ZOO estimator that reuses queries across several parameter updates is formally the same as a truncated or bootstrapped policy gradient, which could justify importing temporal-difference style smoothing into ZOO.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper claims that Gaussian-smoothed zeroth-order gradient estimation is exactly single-step REINFORCE with a baseline. Section 3 defines a single-step policy objective with policy x=theta+mu u, shows J=F_mu, and shows by substitution that estimator (9) with b(xi)=f(theta;xi) coincides with estimator (2). Section 4 proposes ZoAR, which replaces the single-point baseline with an average of recent function values and reuses historical queries; Appx B contains bias, optimal-baseline, variance, and convergence theorems under u~Unif(S^{d-1}). Experiments on synthetic functions, adversarial attacks, and LLM fine-tuning report faster convergence. The algebraic core Thm 3.2 is correct; however, the Appx B theory has a fundamental sphere-smoothing error, the importance-sampling generalization is not mathematically valid for singular proposals, and the formal convergence bound has a nonvanishing floor. The empirical comparisons may still be valid, but they cannot compensate for the theoretical gaps as they stand.

Significance. The exact equivalence in Thm 3.2 is verifiable and provides a clean vocabulary transfer between ZOO and REINFORCE; this is a useful interpretive contribution, although it is essentially a constructed identity rather than a deep structural theorem. The proposed ZoAR algorithm is simple, and the reported empirical improvements over Vanilla ZOO, ZoHS, and ReLIZO in Figs. 1, 3, Table 1, and Fig. 4 are a genuine strength. However, the second contribution's theoretical justification is load-bearing and the appendix results fail as stated. The paper does not include code, but the experimental protocol with shared update rules and hyperparameters is described in Appx D. In its current form the significance is limited by the invalid Appx B; a thorough revision is needed.

major comments (4)
  1. [Appx C.4 / Thm B.3] The claim that estimator (13) is unbiased for the average of nabla F_mu(theta_{t-n}) fails for u~Unif(S^{d-1}). For F(theta)=a*theta, E[(F(theta+mu u)-F(theta))/mu u] = a/d, while Eq. (3) with u~Unif(S^{d-1}) defines F_mu with gradient a. The proof's step (b) cites Flaxman et al. (2005b) Lemma 2.1 but redefines F_mu as E_{u~Unif(B^d)}[F(theta+mu u)], which is not the objective in Eq. (3); even for that ball-smoothed objective, the lemma gives E[F(theta+mu u)u]=(mu/d)nabla F_mu^B(theta), so the estimator is (1/d)nabla F_mu^B(theta), not nabla F_mu(theta). Because Thms B.4, B.5, and C.4 (Eq. (49)) all build on Thm B.3, the ZoAR variance and convergence theory is not established as stated.
  2. [Appx C.3 / Thm 3.3] The importance-sampling identity in Eq. (10) is not valid for singular proposals. The Gaussian policy expectation is over R^d, so p must be a density with respect to Lebesgue measure. For u~Unif(S^{d-1}), the stated p(x)=1/Area(S^{d-1}(mu)) is a density with respect to surface measure, and for u~Unif({e_i}), p(x)=1/d is a density with respect to counting measure; neither is a Lebesgue density. Thus pi_theta(x)/p(x) in Eq. (11) is not a Radon-Nikodym derivative and the estimator does not estimate nabla J(theta) in the usual Monte Carlo sense. The gamma factors in Thm 3.3 are formal density ratios, and Corollary 3.4's learning-rate rescaling is unsupported for sphere and coordinate sampling. The statement should be restricted to absolutely continuous proposals or re-derived using a common reference measure.
  3. [Appx C.7 / Thm C.4] The additive term B_2 in Eq. (49) is not shown to vanish under the stated choices eta=O(epsilon^2), 1-beta_2=O(epsilon^2). In the definition of B_2, the term sqrt(2/(beta_1(1-beta_2)))(1+beta_1)G contains G=2G_mu sqrt(zeta) sqrt(d[V+...]) with V=(sigma_xi^2+sigma_mu^2)/(NK mu^2) independent of eta, so this part of B_2 is of order sqrt(V)/epsilon as epsilon goes to 0 for fixed V. The remark after Thm B.6 says the bias can be small with a small learning rate eta, but the V-dependent part of B_2 is not controlled by eta. Consequently Eq. (49) does not establish convergence to a first-order stationary point, and the claim that query reuse enhances convergence is not supported by this theorem.
  4. [Appx C.6 / Eq. (29)] In the proof of Thm B.5, step (a) replaces E||sum_{n,k} z_{n,k}||^2 with sum_{n,k} E||z_{n,k}||^2, where z_{n,k}=(f(theta_{t-n}+mu u_{n,k};xi)-b_t)u_{n,k}/mu. This equality requires the terms to be uncorrelated with zero mean; independence alone is not enough because E[z_{n,k}] is proportional to the smoothed gradient and is nonzero. The omitted cross terms are nonnegative, so the displayed expression is not an upper bound as written. The variance term V in Thm B.5 and the constants Xi and G in Thm C.4 are therefore not derived by the given proof.
minor comments (5)
  1. [Sec. 3.1, Eq. (6)] The second equality writes f(theta;xi) inside the expectation over x; it should be f(x;xi).
  2. [Sec. 5 and Appx D] The perturbation distribution used in the experiments is not specified; please state whether u~N(0,I_d) or another distribution is used, since Appx B assumes u~Unif(S^{d-1}).
  3. [Appx E.1, Fig. 2] Because the compared ZOO and REINFORCE updates are algebraically identical by Thm 3.2 and Eq. (12), the numerical agreement in Fig. 2 is by construction; it should be presented as a sanity check of the implementation rather than as independent empirical validation of the equivalence.
  4. [Appx D.1 and Appx F] Minor typos include "gradeint", "proptotional", "schudule", and "retrive"; these should be corrected.
  5. [Figs. 1 and 3] Reporting only five-run averages without error bands makes it hard to assess the significance of the claimed speedups; adding standard deviations or confidence intervals would help.

Circularity Check

2 steps flagged · score 7.0 of 10

The headline equivalences are definitional: the single-step PO objective and the REINFORCE baseline are constructed to reproduce ZOO, so the claimed unification and its Fig. 2 validation reduce to a change of variables.

  1. self definitional [Theorem 3.1, Section 3.1; proof in Appendix C.1]
    "Theorem 3.1 (Objective Equivalence). Let the policy πθ(x) be defined via the reparameterization x=θ+µu, where u is a random vector drawn from a distribution p(u) independent of θ. Then, the single-step PO objective J(θ) defined in (6) is identical to the ZOO smoothed objective Fµ(θ) defined in (3) using the same distribution p(u), i.e., J(θ)=Fµ(θ)."

    Sec. 3.1 defines the single-step PO objective with reward R0=-F(x), and the theorem chooses πθ via x=θ+µu. Under those definitions, J(θ)=E_u[F(θ+µu)] is the same expectation as Fµ(θ) in (3); the proof in C.1 simply says the substitution 'is precisely the definition' of Fµ. Thus the 'equivalence' is a change of variables plus a chosen reward, not a derived property of either ZOO or PO. The theorem asserts an identity that is true by construction, so it cannot serve as independent evidence for a 'previously unrecognized connection'.

  2. self definitional [Theorem 3.2, Section 3.2; proof in Appendix C.2]
    "Substituting xk=θ+µuk and b(ξ)=f(θ;ξ) into the REINFORCE estimator (9): ... = 1/K Σ (f(θ+µuk;ξ)-f(θ;ξ))/µ uk. This is exactly the Gaussian-smoothed ZOO gradient estimator ∇̂F(θ) in (2)."

    Eq. (9) is a generic REINFORCE estimator with a free baseline b(ξ). By setting b(ξ)=f(θ;ξ), the score term (xk−θ)/µ^2 = uk/µ makes the 'REINFORCE' expression exactly (2) for every sample. The proof is substitution, not derivation: the estimator and the ZOO finite-difference estimator are the same algorithm in different notation. Consequently the paper's statement that ZOO baseline subtraction 'corresponds precisely to using a baseline' is true but definitional; the variance-reduction result is imported from REINFORCE only after the baseline has been chosen to match ZOO's center subtraction.

full rationale

Two headline equivalences are constructed identities. Theorem 3.1 defines the single-step PO objective with reward R=-F and policy x=θ+µu, making J(θ) literally E_u[F(θ+µu)] = Fµ(θ); the proof in C.1 says the substitution 'is precisely the definition' of Fµ. Theorem 3.2 sets the REINFORCE baseline to b(ξ)=f(θ;ξ), so (xk−θ)/µ^2 (f(xk;ξ)−f(θ;ξ)) equals (f(θ+µuk;ξ)−f(θ;ξ))/µ uk by algebra; the two estimators are the same function. The same construction drives Thm 3.3 via γ=πθ(x)/p(x). These equalities are not derived from extra assumptions, and the Fig. 2 experiment runs the same update under both labels, so it cannot be evidence for the equivalence. The ZoAR algorithm and its empirical gains are independent, non-circular contributions; self-citations to Shu et al. 2025b only set the proof framework. However, Appendix B's theory has a separate correctness problem that should be weighed: the proof of Thm B.3 says that under u∼Unif(S^{d−1}) step (b) follows from Flaxman et al. Lemma 2.1 'utilizing the definition Fµ(θ)=E_{u∼Unif(B^d)}[F(θ+µu)]', which changes the smoothing from (3) and omits the dimension factor d; for F(θ)=a·θ, E[(a·u)u]=a/d, not ∇Fµ. Thus Thm B.3, B.4, B.5, and C.4 inherit an error. This is a correctness defect, not a circularity. Overall, the central theoretical 'prediction' reduces by construction, so the circularity score is 7 rather than lower.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The algorithmic claims rest on Lipschitz/bounded-variance assumptions B.1 and B.2, on the restriction to spherical perturbations for the variance analysis, and on the implicit assertion that the query-reuse bias is negligible at practical learning rates. No new physical or mathematical entities are introduced; the free parameters are hand-chosen hyperparameters mu, N, K, and eta.

free parameters (4)
  • smoothing radius mu = 0.05 (synthetic), 0.5 (adversarial attack), 0.01 (LLM fine-tuning)
    Hand-chosen hyperparameter controlling the perturbation scale. The theory treats it as fixed, and the variance term scales as 1/mu^2.
  • history depth N = 6 (synthetic, attack), 15 or 50 (LLM)
    Hand-chosen buffer length controlling the bias-variance tradeoff in Thm B.5. Larger N reduces variance but increases squared bias.
  • queries per iteration K = 10 (synthetic), 2 (attack, LLM)
    Hand-chosen number of new queries per iteration. The variance term scales as 1/(NK).
  • learning rate eta = 0.001 (synthetic), 0.01 (attack), 0.00005 (LLM)
    Standard optimizer hyperparameter; the bias bound in Thm B.5 is O(eta^2).
assumptions (5)
  • domain assumption Assumption B.1: bounded function values and Lipschitz continuity of F and its gradient
    Used in Thm B.5 and the convergence proof to bound bias and smoothness of F_mu. Standard for ZOO theory but restrictive for adversarial and LLM losses.
  • domain assumption Assumption B.2: bounded variance of f and of the smoothed value
    Used to bound the variance term V=(sigma_xi^2+sigma_mu^2)/(NK mu^2) in Thm B.5 and the convergence bound.
  • ad hoc to paper The ZoAR variance theory is developed only for u~Unif(S^{d-1}), where ||u||^2=1
    The paper states it follows 'the common practice in (Shu et al., 2025b) to prove under u~Unif(S^{d-1})', but the algorithm and experiments are not restricted to sphere sampling, so the optimal-baseline result may not transfer.
  • domain assumption Bounded gradient of the smoothed objective, |nabla_i F_mu(theta)| <= G_mu
    Additional assumption in Thm C.4 (Appx C.7) used to bound the first-moment terms in the AdaZO convergence proof.
  • standard math Standard probability tools: Stein's lemma, Flaxman et al. Lemma 2.1, Jensen, Cauchy-Schwarz, geometric series
    Used in Thm B.3 and the convergence proofs to relate function-value expectations to gradients and to sum bias terms.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Zeroth-Order Optimization is Secretly Single-Step Policy Optimization." pith.science (2026). https://pith.science/paper/4Z4IFJ7U

@misc{pith2026250614460,
  author       = {Pith},
  title        = {Pith review of: Zeroth-Order Optimization is Secretly Single-Step Policy Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4Z4IFJ7U}},
  note         = {Machine review of arXiv:2506.14460}
}
read the original abstract

Zeroth-Order Optimization (ZOO) provides powerful tools for optimizing functions where explicit gradients are unavailable or expensive to compute. However, the underlying mechanisms of popular ZOO methods, particularly those employing randomized finite differences, and their connection to other optimization paradigms like Reinforcement Learning (RL) are not fully elucidated. This paper establishes a fundamental and previously unrecognized connection: ZOO with finite differences is equivalent to a specific instance of single-step Policy Optimization (PO). We formally unveil that the implicitly smoothed objective function optimized by common ZOO algorithms is identical to a single-step PO objective. Furthermore, we show that widely used ZOO gradient estimators, are mathematically equivalent to the REINFORCE gradient estimator with a specific baseline function, revealing the variance-reducing mechanism in ZOO from a PO perspective.Built on this unified framework, we propose ZoAR (Zeroth-Order Optimization with Averaged Baseline and Query Reuse), a novel ZOO algorithm incorporating PO-inspired variance reduction techniques: an averaged baseline from recent evaluations and query reuse analogous to experience replay. Our theoretical analysis further substantiates these techniques reduce variance and enhance convergence. Extensive empirical studies validate our theory and demonstrate that ZoAR significantly outperforms other methods in terms of convergence speed and final performance. Overall, our work provides a new theoretical lens for understanding ZOO and offers practical algorithmic improvements derived from its connection to PO.

Figures

Figures reproduced from arXiv: 2506.14460 by the authors.

Figure 1
Figure 1. Comparison of convergence among different zeroth-order optimization algorithms on four synthetic functions. All curves are averaged over 5 independent runs [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Equivalence of ZOO and REINFORCE with two different baselines. The y-axis denotes the gap between the current function value and the optimal function value. The green curves denote to the average baseline defined in (12), while the red curves denote to the single-point baseline bt = f(θt−1; ξ). All curves are averaged over 5 independent runs. 0 10000 20000 Iters (T) 4 6 8 Optimality Gap (log scale) 5.33£ Quadratic V… view at source ↗
Figure 3
Figure 3. Comparison of convergence among different zeroth-order optimization algorithms on four synthetic functions under R-AdaZO setting. The x-axis denotes the number of iterations, and the y-axis denotes the gap between the current function value and the optimal function value, i.e. F(θ) − minθ′ F(θ ′ ). All curves are averaged over 5 independent runs. E. Additional Experiments E.1. The Equivalence between ZOO and REINFOR… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Training loss comparison between Vanilla ZOO and ZoAR for the LLM fine-tuning under different model sizes on SST2 and Copa datasets. Each curve is averaged over 3 independent runs. E.3. Memory-Efficient LLM Fine-Tuning The pursuit of memory-efficient fine-tuning for la…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 14 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Zo-adamm: Zeroth-order adaptive momentum method for black-box optimization

    Chen, X., Liu, S., Xu, K., Li, X., Lin, X., Hong, M., and Cox, D. Zo-adamm: Zeroth-order adaptive momentum method for black-box optimization. In Proc. NeurIPS , 2019

    2019

  3. [3]

    On the convergence of prior-guided zeroth-order optimization algorithms

    Cheng, S., Wu, G., and Zhu, J. On the convergence of prior-guided zeroth-order optimization algorithms. In Proc. NeurIPS , 2021

    2021

  4. [4]

    T., and McMahan, H

    Flaxman, A., Kalai, A. T., and McMahan, H. B. Online convex optimization in the bandit setting: Gradient descent without a gradient. In Proc. SODA , 2005 a

    work page 2005

  5. [5]

    D., Kalai, A

    Flaxman, A. D., Kalai, A. T., and McMahan, H. B. Online convex optimization in the bandit setting: gradient descent without a gradient. In Proc. SODA , 2005 b

    work page 2005

  6. [6]

    and Lan, G

    Ghadimi, S. and Lan, G. Stochastic first- and zeroth-order methods for nonconvex stochastic programming. SIAM J. Optim. , 23 0 (4): 0 2341--2368, 2013

    2013

  7. [7]

    Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization

    Ghadimi, S., Lan, G., and Zhang, H. Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization. Math. Program., 155 0 (1-2): 0 267--305, 2016

    2016

  8. [8]

    Optimizing large-scale hyperparameters via automated learning algorithm

    Gu, B., Liu, G., Zhang, Y., Geng, X., and Huang, H. Optimizing large-scale hyperparameters via automated learning algorithm. arXiv:2102.09026 , 2021

    arXiv 2021

Show all 34 references
  1. [9]

    Hu, W., Shu, Y., Yu, Z., Wu, Z., Lin, X., Dai, Z., Ng, S.-K., and Low, B. K. H. Localized zeroth-order prompt optimization. In Proc. NeurIPS , 2024

    2024

  2. [10]

    Zo-adamu optimizer: Adapting perturbation by the momentum and uncertainty in zeroth-order optimization

    Jiang, S., Chen, Q., Pan, Y., Xiang, Y., Lin, Y., Wu, X., Liu, C., and Song, X. Zo-adamu optimizer: Adapting perturbation by the momentum and uncertainty in zeroth-order optimization. In Proc. AAAI , 2024

    2024

  3. [11]

    Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In Proc. ICLR , 2015

    2015

  4. [12]

    Gradient-based learning applied to document recognition

    Lecun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, pp.\ 2278--2324, 1998

    1998

  5. [13]

    A comprehensive linear speedup analysis for asynchronous stochastic parallel optimization from zeroth-order to first-order

    Lian, X., Zhang, H., Hsieh, C., Huang, Y., and Liu, J. A comprehensive linear speedup analysis for asynchronous stochastic parallel optimization from zeroth-order to first-order. In Proc. NIPS , 2016

    2016

  6. [14]

    Zeroth-order stochastic variance reduction for nonconvex optimization

    Liu, S., Kailkhura, B., Chen, P., Ting, P., Chang, S., and Amini, L. Zeroth-order stochastic variance reduction for nonconvex optimization. In Proc. NeurIPS , 2018 a

    2018

  7. [15]

    D., and Amini, L

    Liu, S., Li, X., Chen, P., Haupt, J. D., and Amini, L. Zeroth-order stochastic projected gradient descent for nonconvex optimization. In Proc. GlobalSIP , 2018 b

    2018

  8. [16]

    D., Chen, D., and Arora, S

    Malladi, S., Gao, T., Nichani, E., Damian, A., Lee, J. D., Chen, D., and Arora, S. Fine-tuning language models with just forward passes. In Proc. NeurIPS , 2023

    2023

  9. [17]

    A., and Michailidis, G

    Nazari, P., Tarzanagh, D. A., and Michailidis, G. Adaptive first-and zeroth-order methods for weakly convex stochastic optimization problems. arXiv:2005.09261 , 2020

    2005 arXiv

  10. [18]

    Nesterov, Y. E. and Spokoiny, V. G. Random gradient-free minimization of convex functions. Found. Comput. Math., 17 0 (2): 0 527--566, 2017

    2017

  11. [19]

    Evolution strategies as a scalable alternative to reinforcement learning

    Salimans, T., Ho, J., Chen, X., and Sutskever, I. Evolution strategies as a scalable alternative to reinforcement learning. arXiv:1703.03864 , 2017

    2017 arXiv

  12. [20]

    Proximal policy optimization algorithms

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. arXiv:1707.06347 , 2017

    2017 arXiv

  13. [21]

    Shu, Y., Dai, Z., Sng, W., Verma, A., Jaillet, P., and Low, B. K. H. Zeroth-order optimization with trajectory-informed derivative estimation. In Proc. ICLR , 2023

    2023

  14. [22]

    Shu, Y., Lin, X., Dai, Z., and Low, B. K. H. Federated zeroth-order optimization using trajectory-informed surrogate gradients. In Workshop on Differentiable Almost Everything (ICML), 2024

    2024

  15. [23]

    Shu, Y., Hu, W., Ng, S.-K., Low, B. K. H., and Yu, F. R. Ferret: Federated full-parameter tuning at scale for large language models. In Proc. ICML , 2025 a

    2025

  16. [24]

    Refining adaptive zeroth-order optimization at ease

    Shu, Y., Zhang, Q., He, K., and Dai, Z. Refining adaptive zeroth-order optimization at ease. In Proc. ICML , 2025 b

    2025

  17. [25]

    Stein, C. M. Estimation of the mean of a multivariate normal distribution. The annals of Statistics, pp.\ 1135--1151, 1981

    1981

  18. [26]

    Sutton, R. S. and Barto, A. G. Reinforcement learning - an introduction, 2nd Edition. MIT Press, 2018

    2018

  19. [27]

    S., McAllester, D

    Sutton, R. S., McAllester, D. A., Singh, S., and Mansour, Y. Policy gradient methods for reinforcement learning with function approximation. In Proc. NIPS , 1999

    1999

  20. [28]

    Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., and Bowman, S. R. GLUE : A multi-task benchmark and analysis platform for natural language understanding. In Proc. ICLR , 2019. In the Proceedings of ICLR

    2019

  21. [29]

    Relizo: Sample reusable linear interpolation-based zeroth-order optimization

    Wang, X., Qin, X., Yang, X., and Yan, J. Relizo: Sample reusable linear interpolation-based zeroth-order optimization. In Proc. NeurIPS , 2024

    2024

  22. [30]

    Williams, R. J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn., 8: 0 229--256, 1992

    1992

  23. [31]

    Unlocking black-box prompt tuning efficiency via zeroth-order optimization

    Zhan, H., Chen, C., Ding, T., Li, Z., and Sun, R. Unlocking black-box prompt tuning efficiency via zeroth-order optimization. In Proc. EMNLP (Findings) , 2024

    2024

  24. [32]

    V., Mihaylov, T., Ott, M., Shleifer, S., Shuster, K., Simig, D., Koura, P

    Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., Mihaylov, T., Ott, M., Shleifer, S., Shuster, K., Simig, D., Koura, P. S., Sridhar, A., Wang, T., and Zettlemoyer, L. Opt: Open pre-trained transformer language models, 2022

    2022

  25. [33]

    D., Yin, W., Hong, M., Wang, Z., Liu, S., and Chen, T

    Zhang, Y., Li, P., Hong, J., Li, J., Zhang, Y., Zheng, W., Chen, P., Lee, J. D., Yin, W., Hong, M., Wang, Z., Liu, S., and Chen, T. Revisiting zeroth-order optimization for memory-efficient LLM fine-tuning: A benchmark. In Proc. ICML , 2024

    2024

  26. [34]

    Quzo: Quantized zeroth-order fine-tuning for large language models, 2025

    Zhou, J., Yang, Y., Zhen, K., Liu, Z., Zhao, Y., Banijamali, E., Mouchtaris, A., Wong, N., and Zhang, Z. Quzo: Quantized zeroth-order fine-tuning for large language models, 2025

    2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.