REVIEW 4 major objections 6 minor 30 references
Inverse Design in Distributed Circuits Using Single-Step Reinforcement Learning
T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A single-step reinforcement learning framework, DCIDA, generates distributed resonator circuits that match target transfer functions more closely than template-based baselines, without requiring topology templates.
desk verdict DCIDA is a solid, novel method for single-step RL in distributed circuit design, but its headline error reductions are only shown against the same neural surrogate used for training, so the practical claim is weaker than the abstract suggests. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the injective interdependent mapping defined by Theorems 4.2 and 4.3: given the previous resonator's center, functions $p_x$ and $p_y$ map continuous actions $a_x, a_y \in [0,1]$ and a discrete direction $a_d \in \{0,1,2\}$ into the current resonator's center, using shift deviation $d_s$ and gap deviation $d_g$ to keep all resonators within the boundary $B = aN + g(N-1)$. The mapping is deterministic and one-to-one, transforming a continuous near-infinite placement space into a bounded, learnable search space while preserving interdependencies among design decisions.
What would settle it
Run the circuits generated by DCIDA and by Circuit-GNN through a full-wave electromagnetic simulator (e.g., HFSS or Sonnet) and recompute $\epsilon_{\text{db}}$; if DCIDA's advantage shrinks or reverses, the reported improvements are artifacts of the surrogate. A complementary test: choose a target transfer function that requires a layout violating the sequential placement rule (e.g., a resonator positioned to the left of a previously placed one) and check whether DCIDA can generate it.
Extended reading notes
Core claim
DCIDA formulates the inverse design problem as a Single-Step Markov Decision Process in which a policy network with parameters $\theta$ maps a constant input tensor $I$ to a product of conditional distributions over $D = 8N - 5$ design dimensions, covering slit directions, slit offsets, resonator lengths, and placement offsets. Sampling yields a compound action $A$, which is translated by deterministic interdependent mapping functions into the physical representation $\xi = (p, a, u, s)$ of $N$ square resonators inside a bounded region $\mathcal{B}$. The mapping is injective, so each sampled action corresponds to exactly one layout, and it encodes sequential placement rules (each resonator positioned relative to the previous one) that capture conditional dependencies. The policy is optimized with a PPO-style objective that uses a running reward baseline and KL and entropy regularizers, allowing the reward to come from a non-differentiable EM simulator or a neural estimator. On the Circuit-GNN dataset, Transformer-based DCIDA achieves lower dB-domain transfer-function error than Circuit-GNN and θ-Resonance on nearly all topology types for 3-, 4-, 5-, and 6-resonator circuits, including a 66.82% average error reduction on four-resonator circuits and visibly better matches on challenging targets with multiple peaks and valleys.
Load-bearing premise
The pre-trained neural network that approximates the electromagnetic transfer function is accurate and unbiased enough that the error reductions measured through it reflect real circuit behavior.
Editorial extensions
If this is right
- Inverse design of microwave filters can proceed without a priori topology templates, which is closer to how specifications arise in real RF design flows.
- Because the learned policy can be evaluated by any black-box simulator, design tools that are non-differentiable can serve directly as reward sources.
- The single-step compound-action scheme sidesteps credit assignment across partial designs, potentially carrying over to other fixed-dimensional black-box design problems.
- Transformer-based DCIDA's reported reductions (up to 66.82% on four-resonator circuits) indicate that near-continuous placement with interdependent constraints can outperform grid- and template-based search.
- The method's stronger fits on complex transfer functions with multiple peaks and valleys suggest it scales to more demanding design specifications than current baselines.
Reading between the lines
- The largest reported gains rely on the pre-trained forward model; validating a subset of generated designs with full-wave EM simulation or fabrication would test whether the error reductions persist under a more accurate evaluator.
- The sequential left-to-right placement rule assumes an optimal layout can be represented in that canonical order; circuits where the best arrangement requires a different ordering may be unreachable unless the policy can also choose the order.
- A natural extension is to predict the resonator count $N$ from the target transfer function, eliminating the last remaining template input, though this will require comparing designs across different $N$.
- If replicated in other physical design tasks with geometric placement (e.g., antenna or lens design), the interdependent injective mapping plus single-step sampling could provide a general recipe for one-shot black-box design.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces DCIDA, a single-step reinforcement-learning framework for inverse design of distributed circuits. Given a target transfer function s21 and the number of resonators N, a policy network decodes a constant input into a compound action sampled from conditional beta and categorical distributions. Deterministic mapping functions translate these raw actions into physical resonator layouts within predefined boundaries. The policy is trained with a PPO-style objective in which the reward is the negative dB-domain error between the target transfer function and the transfer function predicted by a forward model. Experiments on the Circuit-GNN dataset compare Transformer-based DCIDA with Circuit-GNN and theta-Resonance, reporting lower epsilon_db on most topology types, including reductions up to 66.82% on 4-resonator circuits. The paper also presents ablations, convergence analyses, and visualizations of challenging cases.
Significance. If validated with a true electromagnetic simulator, DCIDA would be a useful contribution to automated distributed-circuit design: it removes template-topology assumptions, handles continuous placement, and uses a single-step action representation that avoids the partial-design reward problem. The conditional-action architecture and the bounded deterministic mapping are sensible design choices for non-differentiable simulators. The paper's ablations (boundary vs. interdependent functions) and CDF comparisons provide useful insight. However, the current evidence is generated entirely through the same neural surrogate used as the training reward, and the claimed injectivity of the mapping is not actually satisfied. These gaps are fixable, but they currently limit the strength of the central claim.
major comments (4)
- [Section 5.1 and Section 4.2.3, Eq. (11)] The central empirical claim is evaluated with the same pre-trained neural forward model that provides the training reward. Section 5.1 states that 'we applied the same pre-trained forward model as an approximator of the simulator to evaluate the transfer functions of all generated circuits,' and Eq. (11) defines the reward R(ξ) = −ε_db using the output of the forward model (or EM simulator). Because DCIDA's policy is trained to maximize this surrogate-based reward, the lower ε_db values in Tables 1 and 2 may reflect exploitation of surrogate-specific errors rather than physically closer designs. Section 1 itself notes that pre-trained model predictions are less accurate than EM simulators, yet no full-wave EM validation is reported. To support the abstract's claim of 'significant reductions in design error,' the authors should evaluate at least a subset of generated circuits with an EM simulator, report ε_db under the EM solver, and check whether the surrogate's ranking of designs agrees with the EM solver's ranking.
- [Section 4.2.2, Theorems 4.2 and 4.3, Eq. (6)] The mapping is not injective as claimed. For a_d ∈ {0,1} in Theorem 4.2, h_x^(i) is an affine function of a_x^(i). When a_f = 0, Eq. (6) gives f = 0 and the deviation function d_s(l,r,u_s) = u_s(r−l)+l evaluates to 0 because l = 0 and r = a_f = 0, so h_x^(i) = x_{i−1} regardless of a_x^(i). Thus uncountably many action tuples map to the same physical center. For a_d = 0, h_y^(i) is also fixed independent of a_y^(i), making a_y redundant. The abstract and Figure 2 caption's phrase 'injective interdependent map' is therefore false. The same degeneracy appears in Theorem 4.3. The authors should either modify the mapping so that each design dimension remains sensitive to its action (for example, by ensuring d_s > 0 always), or remove the injectivity claim and discuss the consequences for learning the conditional dependencies.
- [Section 4.2.2, Theorem 4.1] The restriction of all designs to the area P defined by B = aN + g(N−1) is an assumption that the near-optimal design for each target lies within these boundaries. The paper provides no evidence for this assumption, such as a check that the Circuit-GNN dataset's template designs (which presumably realize the targets) satisfy the boundary constraints, nor does it report sensitivity of the results to the predefined maximum gap ratio g_max and the boundary size. If the optimal layout for a target requires, for example, a gap larger than g_max or a resonator centered outside the interval [0, B−a], DCIDA cannot produce it, which would invalidate the 'near-optimal' characterization. Please justify the boundary choice empirically or restrict the claims to the resulting design subspace.
- [Section 5.2, Tables 1 and 2] The reported means and standard deviations do not substantiate the word 'significant' used in the abstract and Section 5.2. The paper does not state the number of independent training runs or how many per-target errors form each average, and no significance tests are reported. Several comparisons show differences far smaller than the standard deviations (e.g., Table 2, 5-resonator topology 0: 1.08±0.35 vs 1.07±0.29; topology 1: 1.31±0.39 vs 1.30±0.41). I recommend reporting per-target paired differences, the number of seeds, and a paired significance test (e.g., Wilcoxon signed-rank), or tempering the strength of the claim accordingly.
minor comments (6)
- [Section 5.1] The phrase 'Transformed-based DCIDA' is a typo for 'Transformer-based DCIDA'.
- [Section 4.2.3 and Section 5.1] The 'anomalous rate α_a' is listed in the experiment setup but is never defined or used in the objective; either define it or remove it from the setup.
- [Eq. (10), case a_d^(n)=1] The expression is missing an operator; it should read 'max(y_{n−1}−a−d_g, (a−B)/2), a_d^(n)=1'.
- [Appendix B, Figure 11] The text refers to the 'Orange area and gray area,' but the figure appears to be in grayscale; label the regions directly in the figure for clarity.
- [Appendix D] The notation '(256,512,1024) (256) (1)' is ambiguous; a table listing batch size, mini-batch size, epoch, and number of backpropagation loops would be much clearer.
- [Eqs. (1) and (11)] The notation for the predicted transfer function varies between \hat{Y}_i and \hat{Y}_i(ξ); please align the notation across equations.
Circularity Check
No significant circularity: DCIDA optimizes against a fixed external surrogate and evaluates on the same surrogate, which is a validation limitation rather than a definitional reduction.
full rationale
The paper's derivation chain is self-contained: the target transfer function enters only through the scalar reward R(ξ) = −ε_db (Eq. 11), and the policy parameters are optimized with the PPO-style objective in Eq. 14. The mapping constants and boundary definitions in Section 4.2.2 are manually chosen, not fitted to the target or to the reported errors. The claim that DCIDA reduces design error is an empirical comparison where all methods are evaluated with the same pre-trained forward model, as stated in Section 5.1: 'we applied the same pre-trained forward model as an approximator of the simulator to evaluate the transfer functions of all generated circuits.' This is not a circular reduction: the forward model is a fixed function from an external prior work (Circuit-GNN), and using it both as reward and evaluator is a shared benchmark condition, not a parameter that is fit and then relabeled as a prediction. The main weakness is external validity—if the neural surrogate misranks physical circuit responses, the reported ε_db improvements may not transfer to EM-simulated circuits—but that is a correctness risk, not a circular derivation. The self-citation to θ-Resonance (Mortazavi et al., 2022) involves overlapping authors, but it is used only as a baseline and motivation, not as load-bearing justification. Theorems 4.1–4.3 are constructive bounds and explicit mappings rather than imported uniqueness results. Overall, no step in the claimed derivation reduces to its own inputs by construction.
Assumptions & free parameters
free parameters (4)
- shift factor constants =
0, 0.2, 0.5
- gap ratio bounds =
g_min, g_max (values not specified)
- base resonator length L =
unspecified
- slit position scaling factor =
1/8
assumptions (3)
- domain assumption The pre-trained forward model from Circuit-GNN is an accurate proxy for the physical EM simulator.
- ad hoc to paper The parametrized design space defined by the boundaries in Theorem 4.1 and the interdependent functions in Theorems 4.2 and 4.3 contains all near-optimal designs for the target transfer functions.
- domain assumption The reward signal -epsilon_db (average L1 distance in dB) is a sufficient objective for design quality.
Cite this review
Pith. "Pith review of Inverse Design in Distributed Circuits Using Single-Step Reinforcement Learning." pith.science (2026). https://pith.science/paper/XHKVM6L5
@misc{pith2026250608029,
author = {Pith},
title = {Pith review of: Inverse Design in Distributed Circuits Using Single-Step Reinforcement Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/XHKVM6L5}},
note = {Machine review of arXiv:2506.08029}
}
read the original abstract
The goal of inverse design in distributed circuits is to generate near-optimal designs that meet a desirable transfer function specification. Existing design exploration methods use some combination of strategies involving artificial grids, differentiable evaluation procedures, and specific template topologies. However, real-world design practices often require non-differentiable evaluation procedures, varying topologies, and near-continuous placement spaces. In this paper, we propose DCIDA, a design exploration framework that learns a near-optimal design sampling policy for a target transfer function. DCIDA decides all design factors in a compound single-step action by sampling from a set of jointly-trained conditional distributions generated by the policy. Utilizing an injective interdependent ``map", DCIDA transforms raw sampled design ``actions" into uniquely equivalent physical representations, enabling the framework to learn the conditional dependencies among joint ``raw'' design decisions. Our experiments demonstrate DCIDA's Transformer-based policy network achieves significant reductions in design error compared to state-of-the-art approaches, with significantly better fit in cases involving more complex transfer functions.
Figures
Figures from the paper (9 more)
Reference graph
Works this paper leans on
-
[1]
Understanding the impact of entropy on policy optimization
Ahmed, Z., Le Roux, N., Norouzi, M., and Schuurmans, D. Understanding the impact of entropy on policy optimization. In Chaudhuri, K. and Salakhutdinov, R. (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp.\ 151--160. PMLR, 09--15 Jun 2019
work page 2019
-
[2]
F., Jiang, Z., Zhu, K., Mirhoseini, A., Goldie, A., and Pan, D
Budak, A. F., Jiang, Z., Zhu, K., Mirhoseini, A., Goldie, A., and Pan, D. Z. Reinforcement learning for electronic design automation: Case studies and perspectives: (invited paper). In 2022 27th Asia and South Pacific Design Automation Conference (ASP-DAC), pp.\ 500--505, 2022. doi:10.1109/ASP-DAC52403.2022.9712578
arXiv 2022
-
[3]
Cao, W., Benosman, M., Zhang, X., and Ma, R. Domain knowledge-infused deep learning for automated analog/radio-frequency circuit parameter optimization. In Proceedings of the 59th ACM/IEEE Design Automation Conference, DAC '22, pp.\ 1015–1020, New York, NY, USA, 2022. Association for Computing Machinery
work page 2022
-
[4]
Cao, Y., Wang, G., and Zhang, Q.-J. A new training approach for parametric modeling of microwave passive components using combined neural networks and transfer functions. IEEE Transactions on Microwave Theory and Techniques, 57 0 (11): 0 2727--2742, 2009
work page 2009
-
[5]
Characterization of the dissolution of water microdroplets in oil
Colleran, D., Portmann, C., Hassibi, A., Crusius, C., Mohan, S., Boyd, S., Lee, T., and del Mar Hershenson, M. Optimization of phase-locked loop circuits via geometric programming. In Proceedings of the IEEE 2003 Custom Integrated Circuits Conference, 2003., pp.\ 377--380, 2003. doi:10.1109/CICC.2003.1249422
work page Pith review arXiv 2003
-
[6]
de Ory, M. C., Rodriguez, D., Magaz, M. T., Rollano, V., Granados, D., and Gomez, A. Low loss hybrid nb/au superconducting resonators for quantum circuit applications, 2024
work page 2024
-
[7]
Ckt GNN : Circuit graph neural network for electronic design automation
Dong, Z., Cao, W., Zhang, M., Tao, D., Chen, Y., and Zhang, X. Ckt GNN : Circuit graph neural network for electronic design automation. In The Eleventh International Conference on Learning Representations, 2023
work page 2023
-
[8]
Feng, F., Gongal-Reddy, V.-M.-R., Zhang, C., Ma, J., and Zhang, Q.-J. Parametric modeling of microwave components using adjoint neural networks and pole-residue transfer functions with em sensitivity analysis. IEEE Transactions on Microwave Theory and Techniques, 65 0 (6): 0 1955--1975, 2017
work page 1955
Show all 30 references
-
[9]
Erdse: efficient reinforcement learning based design space exploration method for cnn accelerator on resource limited platform
Feng, K., Fan, X., An, J., Wang, X., Di, K., Li, J., Lu, M., and Li, C. Erdse: efficient reinforcement learning based design space exploration method for cnn accelerator on resource limited platform. Graphics and Visual Computing, 4: 0 200024, 2021
2021
-
[10]
Automated design and optimization of distributed filtering circuits via reinforcement learning, 2024
Gao, P., Yu, T., Wang, F., and Yuan, R.-Y. Automated design and optimization of distributed filtering circuits via reinforcement learning, 2024
2024
-
[11]
Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows
Ghraieb, H., Viquerat, J., Larcher, A., Meliga, P., and Hachem, E. Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows. Phys. Rev. Fluids, 6: 0 053902, May 2021
2021
-
[12]
He, H. et al. Deep learning for distributed circuit design. PhD thesis, Massachusetts Institute of Technology, 2020
2020
-
[13]
Microstrip Filters for RF/Microwave Applications
Hong, J.-S. Microstrip Filters for RF/Microwave Applications. Wiley, 2nd edition, 2011. ISBN 9780470408773
2011
-
[14]
and Lancaster, M
Hong, J.-S. and Lancaster, M. Couplings of microstrip square open-loop resonators for cross-coupled planar microwave filters. IEEE Transactions on Microwave Theory and Techniques, 44 0 (11): 0 2099--2109, 1996
1996
-
[15]
C., Mendler - D \" u nner, C., and Hardt, M
Hsu, C. C., Mendler - D \" u nner, C., and Hardt, M. Revisiting design choices in proximal policy optimization. CoRR, abs/2009.10897, 2020
2009 arXiv
-
[16]
M., Wang, S., Goldie, A., Mirhoseini, A., Jiang, J
Jiang, Z., Songhori, E. M., Wang, S., Goldie, A., Mirhoseini, A., Jiang, J. W., Lee, Y.-J., and Pan, D. Z. Delving into macro placement with reinforcement learning. 2021 ACM/IEEE 3rd Workshop on Machine Learning for CAD (MLCAD), pp.\ 1--3, 2021
2021
-
[17]
Design of microwave filters
Levy, R., Snyder, R., and Matthaei, G. Design of microwave filters. IEEE Transactions on Microwave Theory and Techniques, 50 0 (3): 0 783--793, 2002
2002
-
[18]
and Horta, N
Louren c o, N. and Horta, N. Genom-pof: Multi-objective evolutionary synthesis of analog ics with corners validation. In Proceedings of the 14th Annual Conference on Genetic and Evolutionary Computation, GECCO '12, pp.\ 1119–1126. Association for Computing Machinery, 2012
2012
-
[19]
Lu, Y., Chan, W., Guo, D., Kundu, S., Khandelwal, V., and Lim, S. K. RL-CCD: concurrent clock and data optimization using attention-based self-supervised reinforcement learning. In 60th ACM/IEEE Design Automation Conference, DAC 2023, San Francisco, CA, USA, July 9-13, 2023 , ...
2023
-
[20]
Lu, Y.-C., Nath, S., Khandelwal, V., and Lim, S. K. Rl-sizer: Vlsi gate sizing for timing optimization using deep reinforcement learning. In 2021 58th ACM/IEEE Design Automation Conference (DAC), pp.\ 733--738, 2021. doi:10.1109/DAC18074.2021.9586138
2021
-
[21]
Batch B ayesian optimization via multi-objective acquisition ensemble for automated analog circuit design
Lyu, W., Yang, F., Yan, C., Zhou, D., and Zeng, X. Batch B ayesian optimization via multi-objective acquisition ensemble for automated analog circuit design. In Proceedings of the 35th International Conference on Machine Learning, pp.\ 3306--3314. PMLR, 2018
2018
-
[22]
McConaghy, T., Palmers, P., Steyaert, M., and Gielen, G. G. E. Trustworthy genetic programming-based synthesis of analog circuit topologies using hierarchical domain-specific building blocks. IEEE Transactions on Evolutionary Computation, 15 0 (4): 0 557--570, 2011
2011
-
[23]
V., Laudon, J., Ho, R., Carpenter, R., and Dean, J
Mirhoseini, A., Goldie, A., Yazgan, M., Jiang, J., Songhori, E., Wang, S., Lee, Y.-J., Johnson, E., Pathak, O., Bae, S., Nazi, A., Pak, J., Tong, A., Srinivasa, K., Hang, W., Tuncer, E., Babu, A., Le, Q. V., Laudon, J., Ho, R., Carpenter, R., and Dean, J. Chip placement with d...
2020
-
[24]
S., Qin, T., and Yan, N
Mortazavi, M. S., Qin, T., and Yan, N. Theta-resonance: A single-step reinforcement learning method for design space exploration, 2022
2022
-
[25]
Proximal policy optimization algorithms, 2017
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms, 2017
2017
-
[26]
Autockt: deep reinforcement learning of analog circuit designs
Settaluri, K., Haj-Ali, A., Huang, Q., Hakhamaneshi, K., and Nikolic, B. Autockt: deep reinforcement learning of analog circuit designs. In Proceedings of the 23rd Conference on Design, Automation and Test in Europe, DATE '20, pp.\ 490–495, San Jose, CA, USA, 2020. EDA Consort...
2020
-
[27]
Gcn-rl circuit designer: Transferable transistor sizing with graph neural networks and reinforcement learning
Wang, H., Wang, K., Yang, J., Shen, L., Sun, N., Lee, H.-S., and Han, S. Gcn-rl circuit designer: Transferable transistor sizing with graph neural networks and reinforcement learning. In Proceedings of the 57th ACM/EDAC/IEEE Design Automation Conference, DAC '20, 2020
2020
-
[28]
Ironman-pro: Multiobjective design space exploration in hls via reinforcement learning and graph neural network-based modeling
Wu, N., Xie, Y., and Hao, C. Ironman-pro: Multiobjective design space exploration in hls via reinforcement learning and graph neural network-based modeling. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 42 0 (3): 0 900–913, mar 2023
2023
-
[29]
Circuit- GNN : Graph neural networks for distributed circuit design
Zhang, G., He, H., and Katabi, D. Circuit- GNN : Graph neural networks for distributed circuit design. In Proceedings of the 36th International Conference on Machine Learning, volume 97, pp.\ 7364--7373. PMLR, 2019
2019
-
[30]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.