REVIEW 3 cited by
Hessian-Aware Zeroth-Order Optimization for Black-Box Adversarial Attack
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Zeroth-order optimization is an important research topic in machine learning. In recent years, it has become a key tool in black-box adversarial attack to neural network based image classifiers. However, existing zeroth-order optimization algorithms rarely extract second-order information of the model function. In this paper, we utilize the second-order information of the objective function and propose a novel \textit{Hessian-aware zeroth-order algorithm} called \texttt{ZO-HessAware}. Our theoretical result shows that \texttt{ZO-HessAware} has an improved zeroth-order convergence rate and query complexity under structured Hessian approximation, where we propose a few approximation methods for estimating Hessian. Our empirical studies on the black-box adversarial attack problem validate that our algorithm can achieve improved success rates with a lower query complexity.
Forward citations
Cited by 3 Pith papers
-
Learning a Zeroth-Order Optimizer for Fine-Tuning LLMs
A learned per-block noise-scale generator improves zeroth-order (gradient-free) fine-tuning of LLMs and can be trained once on one task and reused elsewhere.
-
Subspace-based Approximate Hessian Method for Zeroth-Order Optimization
ZO-SAH accelerates zeroth-order optimization by estimating and using subspace Hessians via quadratic fitting with evaluation reuse, achieving faster convergence on logistic regression and neural network benchmarks.
-
KerZOO: Kernel Function Informed Zeroth-Order Optimization for Accurate and Accelerated LLM Fine-Tuning
Kernel functions with vanishing third-moment conditions reduce the leading bias term in zeroth-order gradient estimates, yielding faster LLM fine-tuning than MeZO and HiZOO on several classification and generation benchmarks.
Discussion (0). Continue with ORCID to comment.