ZOAF: Towards Efficient Zeroth-Order Optimization for Analog/RF Circuit Design
Pith reviewed 2026-06-28 11:38 UTC · model grok-4.3
The pith
Zeroth-order optimization recovers usable gradient directions from black-box circuit simulations and improves final design quality while using fewer simulator calls than existing methods.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
ZOAF recovers gradient-descent directions from a small number of black-box circuit simulations by using a hybrid ZO schedule that begins with random-direction estimates for budget-efficient exploration and later switches to coordinate-wise estimates for accurate refinement, together with one-shot quasi-random multi-start to focus evaluations and a sliding-window monitor that triggers early stops and box-projected updates; on three distinct schematics this produces the best median final value on every reported figure of merit, the most robust worst-case behavior across seeds, and 1.3 to 3.8 times fewer simulator calls to convergence than state-of-the-art baselines.
What carries the argument
The hybrid zeroth-order scheduling method that switches from random-direction estimates early to coordinate-wise estimates late, recovering descent directions from black-box simulations.
If this is right
- The method achieves the best median final value on every reported figure of merit across the three test schematics.
- It delivers up to an order-of-magnitude advantage in median peaking on the 22-parameter two-stage amplifier.
- It exhibits the most robust worst-case performance across random seeds compared with baselines.
- It reduces the number of simulator calls required to reach convergence by a factor between 1.3 and 3.8.
Where Pith is reading between the lines
- The same early random-direction to late coordinate-wise switch could shorten evaluation budgets in other black-box engineering tasks that lack adjoint access.
- One-shot quasi-random initialization may help when objective landscapes contain the multiple local optima common in circuit sizing.
- The sliding-window monitor could be adapted to maintain constraints in zeroth-order methods applied to simulation-driven problems outside electronics.
Load-bearing premise
The hybrid scheduling, one-shot multi-start, and sliding-window monitor will produce comparable gains on circuit problems beyond the three evaluated schematics and under different simulator budgets.
What would settle it
A fourth distinct circuit schematic on which ZOAF requires at least as many simulator calls as the strongest surrogate baseline to reach the same final figure of merit.
Figures
read the original abstract
Circuit optimization is an indispensable step in analog/RF IC design. Classical fast gradient-based optimization methods are typically infeasible due to lack of access to simulator source code and the technical barriers to implementing adjoint methods. Therefore, surrogate-based black-box optimization is widely used in practice; however, it can be costly to build and sensitive to hyperparameters, whereas population heuristics often suffer from slow convergence and large evaluation counts under tight simulator-call budgets. To address these limitations, we propose the Zeroth-Order Analog/RF Framework (ZOAF), which recovers gradient-descent directions from a small number of black-box circuit simulations, combining the benefits of both gradient-based optimization and black-box optimization. We also employ several surrogate-free techniques to improve the efficiency and accuracy, including (1) a hybrid ZO scheduling method that switches between random-direction ZO for budget-efficient exploration and coordinate-wise ZO for accurate late-stage refinement, (2) one-shot quasi-random multi-start to focus evaluations, and (3) a sliding-window monitor that triggers early stops and box-projected updates to maintain feasibility. Evaluated on three distinct schematics, ZOAF consistently outperforms state-of-the-art baselines, achieving the best median final value on every reported figure of merit -- with up to an order-of-magnitude advantage in median peaking on the 22-parameter two-stage amplifier -- together with the most robust worst-case behavior across seeds, while reducing simulator calls to convergence by $1.3$--$3.8\times$. Code is publicly available at https://github.com/LiyanTan111/ZOAF.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes the Zeroth-Order Analog/RF Framework (ZOAF) for optimizing analog/RF circuits using zeroth-order methods to estimate gradients from black-box simulations. It introduces a hybrid ZO scheduling (random-direction early for exploration, coordinate-wise late for refinement), one-shot quasi-random multi-start, and a sliding-window monitor for early stopping and feasibility. The central claim is that on three distinct schematics, ZOAF achieves the best median final values on all figures of merit, superior worst-case robustness, and reduces simulator calls by 1.3–3.8× compared to state-of-the-art baselines.
Significance. If the empirical results hold after additional controls, ZOAF could offer a practical surrogate-free method for circuit optimization under tight simulator budgets, combining gradient-based efficiency with black-box applicability. The public code release at https://github.com/LiyanTan111/ZOAF is a clear strength supporting reproducibility.
major comments (2)
- Abstract: the outperformance claims (best median FoM on every metric, up to order-of-magnitude advantage on the 22-parameter amplifier, 1.3–3.8× fewer simulator calls) rest on evaluations of only three schematics with no ablation studies removing the hybrid ZO schedule, one-shot multi-start, or sliding-window monitor, so it is impossible to attribute the gains to the proposed components rather than a standard ZO estimator.
- Experimental section: no details are supplied on baseline implementations, statistical significance tests, error bars on medians, or data-exclusion rules, preventing verification of the quantitative speedups and robustness claims.
minor comments (1)
- Abstract: the term 'median peaking' is ambiguous; clarify the exact figure of merit or metric to which it refers.
Simulated Author's Rebuttal
We thank the referee for the constructive comments on our manuscript. We address each major comment below and outline the revisions we will make to address the concerns raised.
read point-by-point responses
-
Referee: Abstract: the outperformance claims (best median FoM on every metric, up to order-of-magnitude advantage on the 22-parameter amplifier, 1.3–3.8× fewer simulator calls) rest on evaluations of only three schematics with no ablation studies removing the hybrid ZO schedule, one-shot multi-start, or sliding-window monitor, so it is impossible to attribute the gains to the proposed components rather than a standard ZO estimator.
Authors: We note that evaluation on three distinct schematics is standard practice in analog/RF circuit optimization papers, given the substantial engineering effort and simulation cost required for each new design. Our results compare ZOAF against state-of-the-art baselines that rely on conventional zeroth-order estimators, and the reported gains are consistent across all figures of merit and all three circuits. Nevertheless, we agree that explicit ablation studies would strengthen attribution of the observed improvements to the individual proposed components. In the revised manuscript we will add a dedicated ablation subsection that disables the hybrid ZO scheduling, the one-shot quasi-random multi-start, and the sliding-window monitor in turn, reporting the resulting changes in median FoM and simulator-call counts. This will allow readers to quantify the contribution of each element. revision: yes
-
Referee: Experimental section: no details are supplied on baseline implementations, statistical significance tests, error bars on medians, or data-exclusion rules, preventing verification of the quantitative speedups and robustness claims.
Authors: We agree that the current experimental section lacks sufficient implementation and statistical detail. The revised version will expand this section to provide: (i) complete hyperparameter settings and implementation notes for every baseline; (ii) results of statistical significance tests (Wilcoxon rank-sum) comparing median performance; (iii) interquartile ranges or box-plot error bars on all median figures; and (iv) explicit statements of any data-exclusion criteria and the precise convergence rules used. The already-released public code repository will be updated with the corresponding baseline reproduction scripts to ensure full verifiability. revision: yes
Circularity Check
No circularity: empirical combination of existing ZO techniques with no self-referential derivations or load-bearing self-citations.
full rationale
The paper presents ZOAF as a practical combination of standard zeroth-order gradient estimators with three heuristic enhancements (hybrid random-to-coordinate scheduling, quasi-random multi-start, sliding-window early stopping). No equations, uniqueness theorems, or fitted parameters are defined in terms of the target performance metrics; the central claims rest on direct empirical comparison against baselines on three fixed schematics. No self-citations are invoked to justify core components, and the method is described as implementable from public code without internal redefinition of its own outputs. This is the normal case of an applied engineering paper whose validity is external to any internal derivation chain.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
Stochastic testing simulator for integrated circuits and mems: Hierarchical and sparse techniques,
Z. Zhang, X. Yang, G. Marucci, P. Maffezzoni, I. A. M. Elfadel, G. Kar- niadakis, and L. Daniel, “Stochastic testing simulator for integrated circuits and mems: Hierarchical and sparse techniques,” inProceedings of the IEEE 2014 Custom Integrated Circuits Conference. IEEE, 2014, pp. 1–8
2014
-
[2]
Stochastic testing method for transistor-level uncertainty quantification based on generalized polynomial chaos,
Z. Zhang, T. A. El-Moselhy, I. M. Elfadel, and L. Daniel, “Stochastic testing method for transistor-level uncertainty quantification based on generalized polynomial chaos,”IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 32, no. 10, pp. 1533– 1545, 2013
2013
-
[3]
A multiparameter moment-matching model-reduction approach for gen- erating geometrically parameterized interconnect performance models,
L. Daniel, O. C. Siong, L. S. Chay, K. H. Lee, and J. White, “A multiparameter moment-matching model-reduction approach for gen- erating geometrically parameterized interconnect performance models,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 23, no. 5, pp. 678–693, 2004
2004
-
[4]
Stochastic collocation with non-gaussian cor- related process variations: Theory, algorithms, and applications,
C. Cui and Z. Zhang, “Stochastic collocation with non-gaussian cor- related process variations: Theory, algorithms, and applications,”IEEE Transactions on Components, Packaging and Manufacturing Technol- ogy, vol. 9, no. 7, pp. 1362–1375, 2018
2018
-
[5]
Compact reduced-order modeling of weakly nonlinear analog and rf circuits,
P. Li and L. T. Pileggi, “Compact reduced-order modeling of weakly nonlinear analog and rf circuits,”IEEE Transactions on computer-aided design of integrated circuits and systems, vol. 24, no. 2, pp. 184–203, 2005
2005
-
[6]
Circuit optimization via adjoint lagrangians,
C. W. Wuet al., “Circuit optimization via adjoint lagrangians,” in 1997 Proceedings of IEEE International Conference on Computer Aided Design (ICCAD). IEEE, 1997, pp. 281–288
1997
-
[7]
Feasible adjoint sensitivity technique for EM design optimization,
N. K. Georgieva, S. Glavic, M. H. Bakr, and J. W. Bandler, “Feasible adjoint sensitivity technique for EM design optimization,”IEEE trans- actions on microwave theory and techniques, vol. 50, no. 12, pp. 2751– 2758, 2002
2002
-
[8]
Adjoint techniques for sensitivity analysis in high-frequency structure CAD,
N. K. Nikolova, J. W. Bandler, and M. H. Bakr, “Adjoint techniques for sensitivity analysis in high-frequency structure CAD,”IEEE Transac- tions on Microwave Theory and Techniques, vol. 52, no. 1, pp. 403–419, 2004
2004
-
[9]
Reliable space- mapping optimization integrated with em-based adjoint sensitivities,
S. Koziel, S. Ogurtsov, J. W. Bandler, and Q. S. Cheng, “Reliable space- mapping optimization integrated with em-based adjoint sensitivities,” IEEE transactions on microwave theory and techniques, vol. 61, no. 10, pp. 3493–3502, 2013
2013
-
[10]
Analog circuit design optimization based on symbolic simulation and simulated anneal- ing,
G. G. Gielen, H. C. Walscharts, and W. M. Sansen, “Analog circuit design optimization based on symbolic simulation and simulated anneal- ing,”IEEE Journal of solid-state circuits, vol. 25, no. 3, pp. 707–713, 2002
2002
-
[11]
Analog circuit optimization system based on hybrid evolu- tionary algorithms,
B. Liu, Y . Wang, Z. Yu, L. Liu, M. Li, Z. Wang, J. Lu, and F. V . Fern´andez, “Analog circuit optimization system based on hybrid evolu- tionary algorithms,”Integration, vol. 42, no. 2, pp. 137–148, 2009
2009
-
[12]
M. F. Barros, J. M. Guilherme, and N. C. Horta,Analog circuits and systems optimization based on evolutionary computation techniques. Springer, 2010, vol. 9
2010
-
[13]
An analog circuit design and optimization system with rule-guided genetic algorithm,
R. Zhou, P. Poechmueller, and Y . Wang, “An analog circuit design and optimization system with rule-guided genetic algorithm,”IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 41, no. 12, pp. 5182–5192, 2022
2022
-
[14]
Analog circuit design optimization through the particle swarm optimization technique,
M. Fakhfakh, Y . Cooren, A. Sallem, M. Loulou, and P. Siarry, “Analog circuit design optimization through the particle swarm optimization technique,”Analog integrated circuits and signal processing, vol. 63, no. 1, pp. 71–82, 2010
2010
-
[15]
Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (cma-es),
N. Hansen, S. D. M ¨uller, and P. Koumoutsakos, “Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (cma-es),”Evolutionary Computation, vol. 11, no. 1, pp. 1–18, 2003
2003
-
[16]
An efficient bayesian optimization approach for automated optimization of analog circuits,
W. Lyu, P. Xue, F. Yang, C. Yan, Z. Hong, X. Zeng, and D. Zhou, “An efficient bayesian optimization approach for automated optimization of analog circuits,”IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 65, no. 6, pp. 1954–1967, 2017
1954
-
[17]
Batch bayesian optimization via multi-objective acquisition ensemble for automated analog circuit design,
W. Lyu, F. Yang, C. Yan, D. Zhou, and X. Zeng, “Batch bayesian optimization via multi-objective acquisition ensemble for automated analog circuit design,” inInternational conference on machine learning. PMLR, 2018, pp. 3306–3314
2018
-
[18]
An efficient multi-fidelity bayesian optimization approach for analog circuit synthesis,
S. Zhang, W. Lyu, F. Yang, C. Yan, D. Zhou, X. Zeng, and X. Hu, “An efficient multi-fidelity bayesian optimization approach for analog circuit synthesis,” inProceedings of the 56th annual design automation conference 2019, 2019, pp. 1–6
2019
-
[19]
Locomobo: A local constrained multiobjective bayesian optimization for analog circuit sizing,
K. Touloupas and P. P. Sotiriadis, “Locomobo: A local constrained multiobjective bayesian optimization for analog circuit sizing,”IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 41, no. 9, pp. 2780–2793, 2021
2021
-
[20]
Scalable global optimization via local bayesian optimization,
D. Eriksson, M. Pearce, J. Gardner, R. D. Turner, and M. Poloczek, “Scalable global optimization via local bayesian optimization,”Advances in neural information processing systems, vol. 32, 2019
2019
-
[21]
Uncertainty quantification for integrated circuits: Stochastic spectral methods,
Z. Zhang, I. A. M. Elfadel, and L. Daniel, “Uncertainty quantification for integrated circuits: Stochastic spectral methods,” in2013 IEEE/ACM International Conference on Computer-Aided Design (ICCAD). IEEE, 2013, pp. 803–810
2013
-
[22]
A primer on zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications,
S. Liu, P.-Y . Chen, B. Kailkhura, G. Zhang, A. O. Hero III, and P. K. Varshney, “A primer on zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications,” IEEE Signal Processing Magazine, vol. 37, no. 5, pp. 43–54, 2020
2020
-
[23]
Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models,
P.-Y . Chen, H. Zhang, Y . Sharma, J. Yi, and C.-J. Hsieh, “Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models,” inProceedings of the 10th ACM workshop on artificial intelligence and security, 2017, pp. 15–26
2017
-
[24]
Tensor- compressed back-propagation-free training for (physics-informed) neural networks,
Y . Zhao, X. Yu, Z. Chen, Z. Liu, S. Liu, and Z. Zhang, “Tensor- compressed back-propagation-free training for (physics-informed) neural networks,”arXiv preprint arXiv:2308.09858, 2023
arXiv 2023
-
[25]
Zo- adamm: Zeroth-order adaptive momentum method for black-box opti- mization,
X. Chen, S. Liu, K. Xu, X. Li, X. Lin, M. Hong, and D. Cox, “Zo- adamm: Zeroth-order adaptive momentum method for black-box opti- mization,”Advances in neural information processing systems, vol. 32, 2019
2019
-
[26]
Poor man’s training on mcus: A memory-efficient quantized back-propagation-free approach,
Y . Zhao, H. Li, I. Young, and Z. Zhang, “Poor man’s training on mcus: A memory-efficient quantized back-propagation-free approach,”ACM 12 Transactions on Design Automation of Electronic Systems, vol. 30, no. 5, pp. 1–33, 2025
2025
-
[27]
Revisiting zeroth-order optimization for memory-efficient llm fine-tuning: A benchmark,
Y . Zhang, P. Li, J. Hong, J. Li, Y . Zhang, W. Zheng, P.-Y . Chen, J. D. Lee, W. Yin, M. Honget al., “Revisiting zeroth-order optimization for memory-efficient llm fine-tuning: A benchmark,”arXiv preprint arXiv:2402.11592, 2024
arXiv 2024
-
[28]
Fine-tuning language models with just forward passes,
S. Malladi, T. Gao, E. Nichani, A. Damian, J. D. Lee, D. Chen, and S. Arora, “Fine-tuning language models with just forward passes,”Ad- vances in Neural Information Processing Systems, vol. 36, pp. 53 038– 53 075, 2023
2023
-
[29]
Adazeta: Adaptive zeroth-order tensor-train adaption for memory- efficient large language models fine-tuning,
Y . Yang, K. Zhen, E. Banijamali, A. Mouchtaris, and Z. Zhang, “Adazeta: Adaptive zeroth-order tensor-train adaption for memory- efficient large language models fine-tuning,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, pp. 977–995
2024
-
[30]
Gaussian processes in machine learning,
C. E. Rasmussen, “Gaussian processes in machine learning,” inSummer school on machine learning. Springer, 2003, pp. 63–71
2003
-
[31]
Ado-llm: Analog design bayesian optimization with in-context learning of large language models,
Y . Yin, Y . Wang, B. Xu, and P. Li, “Ado-llm: Analog design bayesian optimization with in-context learning of large language models,” inPro- ceedings of the 43rd IEEE/ACM International Conference on Computer- Aided Design, 2024, pp. 1–9
2024
-
[32]
Multi-objective bayesian optimization for analog/rf circuit synthesis,
W. Lyu, F. Yang, C. Yan, D. Zhou, and X. Zeng, “Multi-objective bayesian optimization for analog/rf circuit synthesis,” inProceedings of the 55th annual design automation conference, 2018, pp. 1–6
2018
-
[33]
A gaussian process surrogate model assisted evolutionary algorithm for medium scale expensive optimization problems,
B. Liu, Q. Zhang, and G. G. Gielen, “A gaussian process surrogate model assisted evolutionary algorithm for medium scale expensive optimization problems,”IEEE Transactions on Evolutionary Computation, vol. 18, no. 2, pp. 180–192, 2013
2013
-
[34]
An evolutionary algorithm- based approach to robust analog circuit design using constrained multi- objective optimization,
G. Nicosia, S. Rinaudo, and E. Sciacca, “An evolutionary algorithm- based approach to robust analog circuit design using constrained multi- objective optimization,” inInternational Conference on Innovative Tech- niques and Applications of Artificial Intelligence. Springer, 2007, pp. 7–20
2007
-
[35]
Gaspad: A general and efficient mm-wave integrated circuit synthesis method based on surrogate model assisted evolutionary algorithm,
B. Liu, D. Zhao, P. Reynaert, and G. G. Gielen, “Gaspad: A general and efficient mm-wave integrated circuit synthesis method based on surrogate model assisted evolutionary algorithm,”IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 33, no. 2, pp. 169–182, 2014
2014
-
[36]
An efficient method for antenna design optimization based on evolutionary computation and machine learning techniques,
B. Liu, H. Aliakbarian, Z. Ma, G. A. Vandenbosch, G. Gielen, and P. Excell, “An efficient method for antenna design optimization based on evolutionary computation and machine learning techniques,”IEEE transactions on antennas and propagation, vol. 62, no. 1, pp. 7–18, 2013
2013
-
[37]
A parallel surrogate model assisted evolutionary algorithm for electromagnetic design optimization,
M. O. Akinsolu, B. Liu, V . Grout, P. I. Lazaridis, M. E. Mognaschi, and P. Di Barba, “A parallel surrogate model assisted evolutionary algorithm for electromagnetic design optimization,”IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 3, no. 2, pp. 93–105, 2019
2019
-
[38]
H. Wang, J. Yang, H.-S. Lee, and S. Han, “Learning to design circuits,” arXiv preprint arXiv:1812.02734, 2018
arXiv 2018
-
[39]
Autockt: Deep reinforcement learning of analog circuit designs,
K. Settaluri, A. Haj-Ali, Q. Huang, K. Hakhamaneshi, and B. Nikolic, “Autockt: Deep reinforcement learning of analog circuit designs,”arXiv preprint arXiv:2001.01808, 2020
arXiv 2001
-
[40]
Dnn-opt: An rl inspired optimization for analog circuit sizing using deep neural networks,
A. F. Budak, P. Bhansali, B. Liu, N. Sun, D. Z. Pan, and C. V . Kashyap, “Dnn-opt: An rl inspired optimization for analog circuit sizing using deep neural networks,”arXiv preprint arXiv:2110.00211, 2021
arXiv 2021
-
[41]
Y . Uhlmann, M. Essich, L. Bramlage, J. Scheible, and C. Curio, “Deep reinforcement learning for analog circuit sizing with an electrical design space and sparse rewards,” inProceedings of the 2022 ACM/IEEE Workshop on Machine Learning for CAD, ser. MLCAD ’22. ACM, Sep. 2022, p. 21–26. [Online]. Available: http://dx.doi.org/10.1145/3551901.3556474
-
[42]
A circuit attention network-based actor-critic learning approach to robust analog transistor sizing,
Y . Li, Y . Lin, M. Madhusudan, A. Sharma, S. Sapatnekar, R. Harjani, and J. Hu, “A circuit attention network-based actor-critic learning approach to robust analog transistor sizing,” in2021 ACM/IEEE 3rd Workshop on Machine Learning for CAD (MLCAD). IEEE, 2021, pp. 1–6
2021
-
[43]
Parasitic-aware analog circuit sizing with graph neural networks and bayesian optimization,
M. Liu, W. J. Turner, G. F. Kokai, B. Khailany, D. Z. Pan, and H. Ren, “Parasitic-aware analog circuit sizing with graph neural networks and bayesian optimization,” in2021 Design, automation & test in Europe conference & exhibition (DATE). IEEE, 2021, pp. 1372–1377
2021
-
[44]
Post-layout simulation driven analog circuit sizing,
X. Gao, H. Zhang, S. Ye, M. Liu, D. Z. Pan, L. Shen, R. Wang, Y . Lin, and R. Huang, “Post-layout simulation driven analog circuit sizing,” Science China Information Sciences, vol. 67, no. 4, p. 142401, 2024
2024
-
[45]
Graph of circuits with gnn for exploring the optimal design space,
A. Shahane, S. Swapna Manjiri, A. Jain, and S. Kumar, “Graph of circuits with gnn for exploring the optimal design space,”Advances in neural information processing systems, vol. 36, pp. 6014–6025, 2023
2023
-
[46]
Online convex optimization in the bandit setting: gradient descent without a gradient,
A. D. Flaxman, A. T. Kalai, and H. B. McMahan, “Online convex optimization in the bandit setting: gradient descent without a gradient,” arXiv preprint cs/0408007, 2004
Pith/arXiv arXiv 2004
-
[47]
Random gradient-free minimization of convex functions,
Y . Nesterov and V . Spokoiny, “Random gradient-free minimization of convex functions,”Foundations of Computational Mathematics, vol. 17, no. 2, pp. 527–566, 2017
2017
-
[48]
signsgd via zeroth-order oracle,
S. Liu, P.-Y . Chen, X. Chen, and M. Hong, “signsgd via zeroth-order oracle,” inInternational conference on learning representations, 2019
2019
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.