REVIEW 3 major objections 6 minor 31 references
Simulator-guided search lifts analog topology generation success from 12% to 65% on a six-component benchmark.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-02 05:16 UTC pith:OZTEZABT
load-bearing objection A credible test-time-search recipe for analog topology generation, with a real but addressable data-provenance confound. the 3 major comments →
EXPLORE: Exploration with Guided Search for Analog Topology Generation using Language Models
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
EXPLORE claims that test-time search, guided by language-model priors and validated by a circuit simulator, is a better decoding strategy for analog topology generation than greedy, beam, or sampling-and-filter decoding. On the FM adjacency-matrix formulation, a PUCT-style tree search that auto-commits near-certain structural tokens via p-filtering (p=0.99), expands only low-confidence decision tokens, beam-completes partial topologies, and simulates each candidate reaches 65% success at tolerance 0.01 on six components versus 33% for sampling-and-filter and 12% for one-shot decoding, under 100 transformer generations, with over 20x lower MSE. It keeps an advantage at 7-9 components, while a
What carries the argument
LM-guided MCTS with p-filtering: a Monte Carlo tree search whose node selection uses PUCT scores that multiply the exploration term by the language model's token probability. Because over half the tokens in the FM adjacency-matrix formulation are structural and carry no topology choice, p-filtering auto-commits any token with model probability at least 0.99 and only spawns top-k children at genuine decision points; partial sequences are beam-completed and simulated, and the resulting voltage/efficiency reward is backpropagated. This concentrates a fixed simulator budget on edge decisions between component pairs.
Load-bearing premise
The load-bearing assumption is that the composition of the 32k fine-tuning set does not matter; if those topologies are mostly EXPLORE's own prior search outputs, the reported success-rate gap could partly reflect training on the search distribution rather than search quality itself.
What would settle it
Retrain the model on 32k six-component topologies drawn only from random enumeration, then rerun EXPLORE's test-time search on the same validation set; if the 65% success rate drops toward the 33% sampling-and-filter level, the gain is driven by data provenance rather than by test-time search. A smaller check: report the success-rate curve with the 32k set replaced by 32k independent Stage-1 random circuits while holding the search algorithm and simulator budget unchanged.
If this is right
- At fixed inference budget (100 generations), structured search outperforms all four tested decoding baselines on 6-component circuits at tolerance 0.01, and lowers MSE by over 20x versus sampling-and-filter.
- The method extends generation to 7-9 components where one-shot decoding scores near zero, with EXPLORE reaching 0.21, 0.26, and 0.16 success respectively.
- Filtering near-certain structural tokens skips 24-48% of token expansions, reducing simulator calls without hurting final search quality when p=0.99.
- The same search loop doubles as a data-collection engine, cutting the fraction of sub-2%-efficiency 6-component topologies from 66.1% to 18.2% and raising the over-90%-efficiency fraction from 8.3% to 23.3%.
- All methods fail on 10-component circuits (0.00 success), so the paper frames its 7-9 component results as a scalability trend rather than practical readiness.
Where Pith is reading between the lines
- Editorial: the 32k fine-tuning set is assembled with 'model-native' data collection, and the paper does not state how much of it consists of EXPLORE's own outputs; an ablation using only random-enumeration data at the same size would separate test-time search quality from training-on-search-output effects.
- Editorial: p-filtering's threshold sweep suggests formulations that remove structural tokens entirely could concentrate search budget even more; compressed adjacency representations are a natural next target.
- Editorial: the clean zero at 10 components implies the current simulator-budget regime will not scale indefinitely; replacing some simulations with a learned surrogate could extend the approach.
- Editorial: the reward's equal 0.5/0.5 weighting of voltage and efficiency error is untuned; per-target weighting or a Pareto treatment could raise effective success without changing the search.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes EXPLORE, a test-time search framework for analog topology generation. It combines an encoder-decoder LM (Flan-T5-base with the LaMAGIC float-input adjacency-matrix formulation) with PUCT-style MCTS, top-k expansion, p-filtering of high-confidence structural tokens, and NGSPICE simulation rewards. The authors also construct a 350k 6-component corpus via a two-stage pipeline in which Stage 2 uses the trained model plus EXPLORE itself as a data generator. On a 6-component benchmark at tolerance 0.01, EXPLORE reports success 0.65 vs 0.12 (Greedy), 0.33 (Sampling+Filter), 0.32 (Beam Search), and 0.31 (MCTS-Base) with the FM-32k model, with lower MSE; results extend to 7–9 components. The paper includes uniform-random validation sets (App. B.4) and sweeps of the exploration constant c and p-filtering threshold p.
Significance. If the reported gains are reproducible and cleanly attributable to test-time search, EXPLORE would be a meaningful advance for LM-based analog design: it demonstrates that a fixed simulator budget can be spent more effectively on guided partial expansions than on independent sampling, and it introduces a practical search-based data-collection loop. I credit the authors for pre-empting the curated-benchmark objection with the random-100/random-500 splits in App. B.4, for the MCTS-Base ablation that isolates LM-prior guidance, and for the systematic p-filtering analysis. However, the headline claim is currently confounded by the unknown composition of the fine-tuning data, and the lack of variance reporting makes it hard to assess how much of the 0.65 vs 0.33 gap is signal. The stated 10-component all-zero regime is appropriately framed as outside current capability.
major comments (3)
- [§3.2, §4.1, App. B.3] The two-stage pipeline in §3.2 explicitly uses the trained model plus EXPLORE to assemble the 350k 6-component corpus, and App. B.3 shows that this Stage-2 data is far richer than random enumeration (23.3% vs 8.3% above 90% efficiency). Section 4.1 states only that models are 'finetuned with 1k and 32k 6-component circuits' and does not disclose whether these sets are Stage-1 random, Stage-2 EXPLORE-native, or a mix. If the 32k set is enriched with EXPLORE-generated topologies, then every decoder shares a model already fine-tuned on the output distribution of the search being tested, so the headline 0.65 vs 0.33 conflates test-time search with training on search-native data. This is not full circularity because NGSPICE targets come from LaMAGIC's validation set, but an ablation separating 'more random data' from 'EXPLORE-native data' (e.g., random-only vs mixed 1k/32k, evaluated with all
- [Table 1, §4.2] The fairness claim is undermined by the footnote 'All methods use up to 100 Transformer generations' while Greedy is defined as one-shot generation in §4.1. If Greedy uses only one generation, the 12%→65% comparison is not under the same budget; if it is run 100 times, a deterministic greedy decoder would not improve, so the comparison needs clarification. More generally, the unit 'Transformer generations' is not defined for Beam Search (one run with width 20 counts as 20 generations?) or for MCTS (one rollout with k=3 and b=1 may generate several sequences, and p-filtering changes the number of tokens per generation). Because the central claim is superior performance under a fixed 100-generation budget, the paper must define the budget unit precisely and report the actual number of evaluated sequences per target for each method.
- [Tables 1–4, Fig. 4] All success rates are single point estimates on 100 or 500 targets, with no seeds, standard errors, or confidence intervals. MCTS and Sampling+Filter are stochastic, and the models themselves are trained once, so run-to-run variance is unreported. The large gaps—e.g., 0.65 vs 0.31 in Table 1—are suggestive, but the smaller differences (Table 3: 0.84 vs 0.70; Table 4: 0.73 vs 0.59) could be within noise. Please report means over at least three seeds, or at minimum exact binomial 95% confidence intervals for the 100-sample benchmarks, and show variance on the iteration-wise curves in Fig. 4.
minor comments (6)
- [Abstract] The main-text abstract says 'over 20×' lower MSE, but the front-page abstract excerpt in the submitted text says 'over 20%'. Make the two versions consistent.
- [Table 4] The Greedy row is not actual greedy decoding but 'the first per-target sample of the S+F generations.' This is an important disclosure; move it from the footnote into the main text or the table caption so readers are not misled.
- [§3.2] Please state explicitly how many of the 350k 6-component circuits come from Stage 1 vs Stage 2. Currently only quality distribution statistics are given, not counts.
- [Algorithm 1 / §3.1] The pseudocode condition 'len(node.top-p tokens)==1' is confusing; I assume it means the top-1 token has probability ≥ p. Please clarify the notation and define 'top-p tokens' here.
- [Fig. 4d / §4.1] The runtime breakdown lists total time 9.22h for n=100; specify the hardware, whether this includes cache warm-up, and whether the same budget was used for the baseline runtimes.
- [§5] The conclusion claims superiority 'over prior RL-based search methods,' but the experiments compare only to decoding baselines (Greedy, Beam, S+F, MCTS-Base). Either add an RL/BO baseline from §2.2 on 6-comp or temper the conclusion to 'decoding strategies'.
Circularity Check
Data-provenance loop: §3.2 builds the 6-component corpus with EXPLORE itself, and §4.1 fine-tunes on unspecified '6-component circuits' from that corpus, partially conflating training data with test-time search.
specific steps
-
other
[Section 3.2 (Stage 2) and Section 4.1 (Models and datasets), Table 1]
"Once a model has acquired a basic understanding of the formulation from Stage-1 data, we reuse the trained model together with the EXPLORE framework as a data generator. ... Using this pipeline we assemble, to our knowledge, the largest 6-component converter dataset to date (350k samples) ... we extend models trained with 3,4,5-components to be finetuned with 1k and 32k 6-component circuits and leverage our decoding algorithm to evaluate."
The 6-component fine-tuning sets are not specified as Stage-1 random-only or Stage-2 EXPLORE outputs; the only 6-component corpus described is the two-stage pipeline whose Stage-2 is generated by EXPLORE. If the 1k/32k sets draw from that corpus, the model in Table 1 has been fine-tuned on the search's own output distribution. Appendix B.3 shows Stage-2 data is sharply enriched (23.3% >90% efficiency vs 8.3% random). Thus the headline 65% vs 33% measures, at least in part, the model's alignment with the EXPLORE generator rather than pure test-time search; an ablation separating random-only from EXPLORE-native training data is required and is absent.
full rationale
The paper's central empirical claim—that EXPLORE is a superior decoding strategy under a fixed 100-generation budget—rests on models fine-tuned with 1k/32k 6-component circuits whose provenance is never stated. Section 3.2 presents only one 6-component collection pipeline, and its Stage 2 is explicitly 'model-native data collection' using the trained model plus EXPLORE. If the fine-tuning sets are drawn from this corpus, then the model's prior has been shaped by the very search procedure being evaluated. All baselines share the same model, so the comparison is not wholly circular, and the external LaMAGIC validation targets and NGSPICE simulator provide independent grounding. Nevertheless, the missing random-only vs EXPLORE-native ablation means the reported 65% success rate and 20× MSE improvement conflate test-time search quality with training on the search's own output distribution. This is a genuine self-referential loop in the data pipeline, though not a definitional collapse; the evaluation targets remain external, so the circularity is partial rather than total.
Axiom & Free-Parameter Ledger
free parameters (7)
- PUCT exploration constant c =
4 (swept 0.5, 1, 4)
- p-filtering threshold p =
0.99 (swept 0.5, 0.7, 0.9, 0.99)
- top-k expansion width k =
3
- beam search width b =
1
- reward weights (voltage, efficiency) =
0.5 / 0.5
- per-target generation budget =
100 rollouts
- evaluation tolerance t =
0.01 (sweep 0.01–0.10)
axioms (7)
- domain assumption NGSPICE simulation accurately returns output voltage and conversion efficiency for candidate topologies
- domain assumption LaMAGIC's FM text formulation is a faithful, decision-complete encoding of the circuit hypergraph
- domain assumption Structural tokens in FM carry no topological choice and constitute >50% of tokens
- domain assumption Beam-search completion of partial topologies yields valid circuits often enough that simulator budget is well spent
- domain assumption Fine-tuning data for the evaluated model is not biased by the framework being tested
- domain assumption The 6-comp benchmark (100 high-efficiency samples mined from the 7k LaMAGIC validation set) is representative of the target generation task
- standard math MCTS/PUCT with a noisy reward oracle explores the search space effectively at the given budget
read the original abstract
Automating analog circuit topology design is essential to reduce the extensive manual effort required to meet increasingly diverse and customized application demands. Recent advances have applied sequence-to-sequence fine-tuning on pretrained language models to directly generate circuit topologies from user specifications in a single pass. However, these one-shot generation methods failed to generate complex circuits due to their exponentially growing search spaces and limited training datasets. In this paper, we present EXPLORE, a search-enhanced framework that integrates simulator-guided Monte Carlo Tree Search (MCTS) with transformer-based decoding to enable test-time scaling for analog topology generation. By leveraging language-model priors and bypassing high-confidence structural tokens, EXPLORE allocates expensive simulator budget primarily toward topology-altering decisions during search. On a 6-component benchmark at a tight tolerance of 0.01, EXPLORE raises the success rate from 12% for one-shot generation and 33% for a sampling-and-filter baseline to 65%, and lowers MSE by over 20% relative to sampling-and-filter under the same search budget. These results establish EXPLORE as the first framework to integrate structured test-time search with LM decoding for analog topology generation, and a practical step toward scaling LLM-driven design automation.
Figures
Reference graph
Works this paper leans on
-
[1]
Chen-Chia Chang, Wan-Hsuan Lin, Yikang Shen, Yiran Chen, and Xin Zhang
-
[2]
Chen-Chia Chang, Wan-Hsuan Lin, Yikang Shen, Guanglei Zhou, Yiran Chen, and Xin Zhang. 2026. LaMAGIC: Advanced Circuit Formulations for Language-Model- based Topology Generation for Analog Integrated Circuits.ACM Transactions on Design Automation of Electronic Systems31, 5 (April 2026), 1–21. doi:10.1145/ 3799428
2026
-
[3]
Chen-Chia Chang, Yikang Shen, Shaoze Fan, Jing Li, Shun Zhang, Ningyuan Cao, Yiran Chen, and Xin Zhang. 2024. LaMAGIC: Language-Model-based Topology Generation for Analog Integrated Circuits. InProceedings of the 41st International Conference on Machine Learning(Vienna, Austria)(ICML’24). JMLR.org, Article 241, 10 pages. https://proceedings.mlr.press/v235...
2024
-
[4]
Zihao Chen, Jiangli Huang, Yiting Liu, Fan Yang, Li Shang, Dian Zhou, and Xuan Zeng. 2024. Artisan: Automated Operational Amplifier Design via Domain-specific Large Language Model. InProceedings of the 61st ACM/IEEE Design Automation Conference(San Francisco, CA, USA)(DAC ’24). Association for Computing Ma- chinery, New York, NY, USA, Article 39, 6 pages....
arXiv 2024
-
[5]
Zehao Dong, Weidong Cao, Muhan Zhang, Dacheng Tao, Yixin Chen, and Xuan Zhang. 2023. CktGNN: Circuit Graph Neural Network for Electronic Design Automation. InInternational Conference on Learning Representations (ICLR). https: //openreview.net/forum?id=NE2911Kq1sp
2023
-
[6]
Shaoze Fan, Ningyuan Cao, Shun Zhang, Jing Li, Xiaoxiao Guo, and Xin Zhang
-
[7]
Shaoze Fan, Haoshu Lu, Shun Zhang, Ningyuan Cao, Xin Zhang, and Jing Li. 2024. Graph-Transformer-based Surrogate Model for Accelerated Converter Circuit Topology Design. InProceedings of the 61st ACM/IEEE Design Automation Confer- ence(San Francisco, CA, USA)(DAC ’24). Association for Computing Machinery, New York, NY, USA, Article 172, 6 pages. doi:10.11...
arXiv 2024
-
[8]
Jian Gao, Weidong Cao, Junyi Yang, and Xuan Zhang. 2025. AnalogGenie: A Generative Engine for Automatic Discovery of Analog Circuit Topologies. In International Conference on Learning Representations (ICLR). https://openreview. net/forum?id=jCPak79Kev
2025
-
[9]
Jian Gao, Weidong Cao, and Xuan Zhang. 2025. AnalogGenie-Lite: Enhancing Scalability and Precision in Circuit Topology Discovery through Lightweight Graph Modeling. InInternational Conference on Machine Learning (ICML). https: //openreview.net/forum?id=KRk0WTII0I
2025
-
[10]
Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. 2023. Reasoning with Language Model is Planning with World Model. InConference on Empirical Methods in Natural Language Processing (EMNLP). 8154–8173. doi:10.18653/v1/2023.emnlp-main.507
-
[11]
Levente Kocsis and Csaba Szepesvári. 2006. Bandit Based Monte-Carlo Planning. InMachine Learning: ECML 2006, Johannes Fürnkranz, Tobias Scheffer, and Myra Spiliopoulou (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 282–293. doi:10. 1007/11871842_29
2006
-
[12]
Kishor Kunal, Meghna Madhusudan, Arvind K Sharma, Wenbin Xu, Steven M Burns, Ramesh Harjani, Jiang Hu, Desmond A Kirkpatrick, and Sachin S Sapatnekar
-
[13]
Yao Lai, Sungyoung Lee, Guojin Chen, Souradip Poddar, Mengkang Hu, David Z. Pan, and Ping Luo. 2025. AnalogCoder: Analog Circuit Design via Training-Free Code Generation. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 379–387. doi:10.1609/aaai.v39i1.32016
-
[14]
Yao Lai, Souradip Poddar, Sungyoung Lee, Guojin Chen, Mengkang Hu, Bei Yu, Ping Luo, and David Z. Pan. 2026. AnalogCoder-Pro: Unifying Analog Cir- cuit Generation and Optimization via Multi-modal LLMs.IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD)(2026). doi:10.1109/TCAD.2026.3673493
arXiv 2026
-
[15]
Mankowitz, Esme Sutherland Robson, Pushmeet Kohli, Nando de Freitas, Koray Kavukcuoglu, and Oriol Vinyals
Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d’Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel J. Mankowitz, Esme Sutherland Robson, Pushm...
2022
-
[16]
Jialin Lu, Liangbo Lei, Jiangli Huang, Fan Yang, Li Shang, and Xuan Zeng. 2023. Automatic Op-Amp Generation From Specification to Layout.IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems42, 12 (2023), 4378–4390. doi:10.1109/TCAD.2023.3296374
arXiv 2023
-
[17]
Asal Mehradfar, Xuzhe Zhao, Yue Niu, Sara Babakniya, Mahdi Alesheikh, Hamidreza Aghasi, and Salman Avestimehr. 2024. AICircuit: A Multi-Level Dataset and Benchmark for AI-Driven Analog Integrated Circuit Design. InMachine Learn- ing and the Physical Sciences Workshop, NeurIPS. https://arxiv.org/abs/2407.18272
Pith/arXiv arXiv 2024
-
[18]
2011.Ngspice Users Manual Version
Paolo Nenzi and Holger Vogt. 2011.Ngspice Users Manual Version
2011
-
[19]
Yichen Shi, Zhuofu Tao, Yuhao Gao, Tianjia Zhou, Cheng Chang, Yaxing Wang, Bingyu Chen, Genhao Zhang, Alvin Liu, Zhiping Yu, Ting-Jung Lin, and Lei He
-
[20]
Zhuofu Tao, Yichen Shi, Yiru Huo, Rui Ye, Zonghang Li, Li Huang, Chen Wu, Na Bai, Zhiping Yu, Ting-Jung Lin, et al . 2024. AMSNet: Netlist Dataset for AMS Circuits. In2024 IEEE LLM Aided Design Workshop (LAD). IEEE, 1–5. doi:10.1109/ LAD62341.2024.10691781
arXiv 2024
-
[21]
Prashanth Vijayaraghavan, Luyao Shi, Ehsan Degan, Vandana Mukherjee, and Xin Zhang. 2025. AutoCircuit-RL: Reinforcement Learning-Driven LLM for Automated Circuit Topology Generation. InInternational Conference on Machine Learning (ICML). https://openreview.net/forum?id=NvYwrQbzOb
2025
-
[22]
Griffiths, Yuan Cao, and Karthik Narasimhan
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of Thoughts: Deliberate Problem Solving with Large Language Models. InAdvances in Neural Information Processing Systems (NeurIPS). https://openreview.net/forum?id=5Xc1ecxO1h
2023
-
[23]
https://pkgs.fedoraproject.org/repo/extras/ngspice/ngspice23-manual.pdf/ eb0d68eb463a41a0571757a00a5b9f9d/ngspice23-manual.pdf Accessed: 2023
2023
-
[24]
AMSnet-KG: A Netlist Dataset for LLM-based AMS Circuit Auto-Design Us- ing Knowledge Graph RAG.ACM Transactions on Design Automation of Electronic Systems (TODAES)30, 6, Article 94 (2025). doi:10.1145/3736166
doi:10.1145/3736166 2025
-
[25]
Tenenbaum, and Chuang Gan
Shun Zhang, Zhenfang Chen, Yikang Shen, Mingyu Ding, Joshua B. Tenenbaum, and Chuang Gan. 2023. Planning with Large Language Models for Code Generation. InInternational Conference on Learning Representations (ICLR). https://openreview. net/forum?id=Lr8cOOtYbfL
2023
-
[26]
Zhenxin Zhao and Lihong Zhang. 2022. Analog Integrated Circuit Topology Synthesis With Deep Reinforcement Learning.IEEE Transactions on Computer- Aided Design of Integrated Circuits and Systems41, 12 (2022), 5138–5151. doi:10. 1109/TCAD.2022.3153437 , , Zhou et al. Appendix A Further Explanation of methodologies A.1 P-UCB node selection To guide explorati...
arXiv 2022
-
[28]
Yuxuan Yin, Yu Wang, Boxun Xu, and Peng Li. 2024. ADO-LLM: Analog Design Bayesian Optimization with In-Context Learning of Large Language Models. In International Conference on Computer-Aided Design (ICCAD). doi:10.1145/3676536. 3676816
doi:10.1145/3676536 2024
-
[29]
Haoyi Zhang, Shizhao Sun, Yibo Lin, Runsheng Wang, and Jiang Bian. 2025. AnalogXpert: Automating Analog Topology Synthesis by Incorporating Circuit Design Expertise into Large Language Models. In2025 International Symposium of Electronics Design Automation (ISEDA). 772–777. doi:10.1109/ISEDA65950.2025. 11100627
arXiv 2025
-
[2019]
In Proceedings of the 56th Annual Design Automation Conference 2019
ALIGN: Open-source Analog Layout Automation from the Ground Up. In Proceedings of the 56th Annual Design Automation Conference 2019. 1–4. doi:10. 1145/3316781.3323471
arXiv 2019
-
[2021]
In2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD)
From Specification to Topology: Automatic Power Converter Design via Reinforcement Learning. In2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD). 1–9. doi:10.1109/ICCAD51958.2021.9643552
arXiv 2021
-
[2025]
InProceedings of the 42nd International Conference on Machine Learning(Vancouver, Canada)(ICML’25)
LaMAGIC2: Advanced Circuit Formulations for Language Model-Based Analog Topology Generation. InProceedings of the 42nd International Conference on Machine Learning(Vancouver, Canada)(ICML’25). JMLR.org. https://openreview. net/forum?id=Y0zXGw0GUk
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.