Pith. sign in

REVIEW 5 major objections 5 minor 49 references

PICopilot claims that the limiting factor in LLM-assisted PIC design is retrieval quality, not raw model power, and that its summary-based RAG pipeline plus multi-agent feedback solves all 48 tasks in its own benchmark.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

A training-free multi-agent LLM framework with summary-based hybrid retrieval generates correct PIC design tool scripts from natural language, passing all 48 self-built benchmark tasks where the best baseline passes 27.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection Strong system, unverifiable headline: the evaluation's success criterion contradicts its own evaluator design. the 5 major comments →

arxiv 2608.01791 v1 pith:MMR4ZXAR submitted 2026-08-03 cs.ET cs.AI

PICopilot: An LLM-based Agentic Framework for Assisting Photonic Integrated Circuit Design via Script Generation

classification cs.ET cs.AI
keywords photonic integrated circuitsdesign automationscript generationlarge language modelsretrieval-augmented generationmulti-agent systemsnatural language to codelayout design and DRC
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PICopilot tries to close the scripting bottleneck in photonic integrated circuit design by having LLM agents write Python scripts from natural-language instructions. Its central claim is that a multi-agent architecture plus a retrieval pipeline specifically built for PIC tool manuals makes this reliable: on a benchmark of 48 tasks spanning layout, design-rule checking, and simulation, it completes all 48, while a general RAG setup on a strong general-purpose model completes at most 27. If true, it matters because PIC design tools are fragmented across vendors with different APIs, and designers currently spend much of their time writing glue scripts rather than designing. The paper argues that the advantage comes from mimicking how human designers retrieve knowledge: generate targeted queries, match them against LLM-generated summaries of manual chunks, then hand the original chunks to a programmer agent.

Core claim

On the paper's own terms, the discovery is that retrieval quality, not model strength, is the binding constraint on LLM script generation in PIC design. The tailored retrieval-augmented generation pipeline—query generation plus summary-based hybrid dense/sparse retrieval over per-tool manual databases—outperforms a generic dense-retriever RAG, and the multi-agent decomposition with feedback adds further gains. The framework reports 48/48 tasks solved, an overall Pass@1 of 93.9%, and overhead that is small in absolute cost and latency compared with direct LLM calls or manual script writing.

What carries the argument

The load-bearing object is the summary-based hybrid retriever combined with per-tool manual databases. Each database stores original manual chunks plus LLM-generated summaries; the retriever scores query–summary matches with both dense cosine similarity and BM25, fuses the ranked lists by weighted reciprocal rank fusion, and returns the original chunks as references for the programmer agent. This carries the argument because component-removal experiments show the largest performance drop when this tailored retrieval is replaced with a general one.

Load-bearing premise

The central claim rests on the authors' own 48-task benchmark, where success is whatever their comparative procedure counts as functionally identical to a hand-written ground-truth script; a biased task set or a lenient success judgment would invalidate the 48/48 result and every comparison derived from it.

What would settle it

Reproduce the benchmark independently: execute each generated script in a fresh installation of the PIC tools and compare actual outputs against the ground truth. The paper's Section 3.6 says running PIC tools is impractical, while Section 4.1.4 says success is determined by executing scripts; reconciling that contradiction, or finding that success was judged by inspection rather than real execution, would settle whether the 48/48 claim holds.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Designers can describe a layout, DRC, or simulation step in plain English and receive executable Python, eliminating most manual API lookups across vendor tools.
  • Adding a new PIC tool only requires building a new manual database and retriever, so the system can track a fast-moving ecosystem without retraining.
  • Because the method is training-free and costs less than a cent per task on average, small design teams can deploy it without dedicated GPU training infrastructure.
  • A static checker plus custom check library catches syntax and common logic errors without running expensive simulations, making iterative refinement cheap in practice.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Testable extension not in the paper: the query–summary retrieval paradigm may transfer to other engineering scripting domains with fragmented tool manuals, such as electronic design automation or scientific simulation, where matching English queries to code-heavy documentation is equally hard.
  • Testable extension not in the paper: directly measuring retrieval hit rate against human experts' chosen manual pages would isolate retrieval quality from the leniency of the LLM-based evaluator; the paper reports end-to-end success but not per-query retrieval precision.
  • Testable extension not in the paper: releasing the 48-task benchmark and an independent execution protocol would allow deterministic reproduction of the 48/48 result; the contradiction between the paper's statement that executing PIC tools is impractical and its claim that success was determined by executing scripts makes the judging protocol the highest-value place to audit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces PICopilot, an LLM-based multi-agent framework that generates Python scripts for photonic integrated circuit (PIC) design from natural-language task descriptions. The framework comprises a task planner, function-specific script generators with a summary-based hybrid RAG pipeline (query generator, multi-database retrievers, dense+BM25 fusion), a script synthesizer, and an LLM-based script evaluator with an adaptive feedback loop. Experiments on a self-constructed 48-task benchmark report that PICopilot solves all 48 tasks, outperforming GPT-5, DeepSeek-V3.2, and Qwen3-Coder under zero-shot and general-RAG settings, with modest overhead. The paper also presents an ablation study on the advanced task set showing contributions of the tailored RAG pipeline and multi-agent architecture.

Significance. If the empirical claims hold, PICopilot would be a valuable contribution to photonic design automation: it is a training-free, domain-adapted system that combines a purpose-built retrieval pipeline with agent orchestration to generate tool-invoking scripts, a task distinct from prior netlist- or layout-oriented PIC-LLM work. The system design is thoughtful, particularly the summary-based retrieval paradigm and multi-database structure, which are clearly motivated by the fragmentation of PIC tool APIs. The overhead analysis is useful for practical deployment. However, the central claim—48/48 tasks solved—rests entirely on the evaluation protocol, and the paper contains a direct contradiction between the stated execution-based success criterion and the described non-execution-based evaluator. Because the benchmark is self-constructed and the judging procedure is unspecified, the headline result is not currently verifiable. The paper would benefit from a clear, reproducible evaluation pipeline, a released benchmark, and sensitivity analyses of its hand-tuned parameters.

major comments (5)
  1. [§4.1.4 vs §3.6] The success criterion is internally contradictory. §4.1.4 states: "The success of a trial is determined by executing the generated script. If the script's execution result is functionally identical to the test task's ground truth, the trial is considered successful." Yet §3.6 states that executing generated scripts is "impractical because running PIC design tools is extremely time-consuming" and that the Script Evaluator instead uses an LLM with a static code checker and a custom check library. The paper never explains how the 48-task experiments were scored: were scripts actually executed (if so, how were the FDTD-scale tasks run?), or was the §3.6 LLM-based evaluator used as the judge? If the latter, then the Pass@k and #Solved numbers reflect an LLM's static judgment, not functional execution, and the 48/48 claim is not substantiated. This is load-bearing for every quantitative compar
  2. [§4.1.3 and §4.1.4] The benchmark and judging protocol lack sufficient detail for reproducibility. The 48 tasks are "carefully selected" but Table 3 gives only generic placeholders (e.g., "create a layout of ... by ...") and states that some descriptions are omitted due to space limitations. The criterion "functionally identical to the test task's ground truth" is not operationalized: no judge is specified (human? LLM? exact output comparison?), no tolerance for numerical/semantic equivalence is given, and no examples of failure/success are shown. The ground-truth scripts were "verified by running" them, which makes the benchmark's own reference standard execution-based, but this does not establish that the evaluated scripts were executed. The authors should release the full task descriptions, ground-truth scripts, and the exact judging protocol (including any LLM-evaluator prompts), or the results cannot b
  3. [§3.4.3 and §4.1.1] Several free parameters are tuned on the same tasks used for evaluation, which risks optimistic bias. The RRF fusion weights are set to (w_dense, w_sparse) = (0.7, 0.3) "based on actual testing" (§3.4.3); top-k is set to k=5 and the maximum feedback iterations to N_f=3 (§4.1.1). No sensitivity analysis or train/test separation is reported for these choices. Since the 48-task benchmark is the only evaluation set, it is unclear whether the reported gains reflect genuine improvements or overfitting to this specific set. The authors should either report a parameter sweep, validate on a held-out set, or at least discuss the stability of results across reasonable parameter ranges.
  4. [§3.6] The custom check library used by the Script Evaluator is built from "common errors found in generated PIC design scripts"—possibly including errors from PICopilot's own outputs. If the evaluator's checks are aligned with the types of errors that PICopilot's feedback loop is designed to fix, the evaluation becomes partially self-referential: the framework is judged by a rubric derived from its own failure modes. The paper should describe the check library's contents, how it was constructed, and whether it was designed independently of the evaluation protocol. Ideally, the final decision should be a functional comparison (execution) or an independently specified set of acceptance tests.
  5. [Table 5 and §4.4] The ablation study is conducted only on the 'Advanced' task set (24 tasks). While this is reasonable for isolating difficult cases, the paper does not report ablations on Basic and Medium tasks, leaving open the question of whether the tailored RAG and multi-agent components help uniformly or only on composite tasks. Given that #Solved on Advanced strongly drives the total (24/48), reporting the full-task-set ablation would strengthen the claim that both components are essential across difficulty levels.
minor comments (5)
  1. [Abstract and §4.3] The abstract states PICopilot improves performance "without incurring substantial extra latency or cost," but Table 4 shows PICopilot's average cost (0.67×10^-2 $) is higher than Qwen3-Coder and DeepSeek-V3.2 baselines (both zero-shot and RAG), and its LLM time (35.88 s avg) exceeds several baselines. The claim should be qualified to indicate the increase is small in absolute terms and still below GPT-5, rather than implying no increase.
  2. [Table 3] The benchmark table would be much more informative with the actual task descriptions or a link to a hosted dataset. The current ellipses ("...") make it impossible to judge task difficulty or evaluate the appropriateness of the difficulty partitioning.
  3. [§3.4.2] The database construction workflow mentions a "human expert verification step" but does not specify how many experts, how they verified the summaries, or whether inter-annotator agreement was considered. A brief description of this verification protocol would improve confidence in the database's correctness.
  4. [§4.1.1] The choice of LLMs (Qwen3-Coder for programming, DeepSeek-V3.2 for other agents) is reasonable, but the paper should justify why DeepSeek-V3.2 is used for planning/evaluation rather than the stronger GPT-5, especially since GPT-5 is used as a baseline. A sentence explaining this choice (e.g., cost, latency, or empirical performance) would prevent confusion about the baseline comparison.
  5. [Throughout] Minor language issues: "pioneer" (contributions) is informal; "the first tool to pioneer" is redundant. The paper would benefit from a careful proofreading pass.

Circularity Check

3 steps flagged

The 48/48 headline result is partly self-referential: retrieval hyperparameters are tuned on the same benchmark, and the stated execution-based success criterion is contradicted by an LLM-based evaluator whose check library is built from the system's own generated-script errors.

specific steps
  1. fitted input called prediction [Sec 3.4.3 (RRF weights) and Sec 4.1.1 (k, N_f); results in Table 2]
    "w_r is the importance weight and we set(w_dense,w_sparse)=(0.7,0.3) based on actual testing. ... We set the top-k retrieved documents per query to k=5, and the maximum iteration count N_f=3."

    The RRF fusion weights are explicitly chosen 'based on actual testing' rather than on a held-out validation set, and the retrieval count k and feedback iteration limit N_f are fixed without any described validation split. All reported Pass@1, Pass@5, and #Solved numbers in Table 2 are computed on the same 48-task benchmark used for this tuning. The headline success rates are therefore in-sample performance of a system whose retrieval hyperparameters were fit to those very tasks, not an independent prediction.

  2. self definitional [Sec 4.1.4 (Metrics) vs Sec 3.6 (Script Evaluator)]
    "The success of a trial is determined by executing the generated script. If the script's execution result is functionally identical to the test task's ground truth, the trial is considered successful. ... Although directly executing generated scripts for evaluation is common and effective, it is impractical because running PIC design tools is extremely time-consuming... Therefore, we design this agent to perform evaluation via an LLM equipped with a static code checker and a custom check library."

    Section 4.1.4 defines success as execution-based functional identity with the ground truth, but Section 3.6 states that executing PIC tools is impractical and the actual evaluator is an LLM with a static checker and a custom check library. The operational success criterion is therefore not the claimed execution comparison but the judgment of an LLM evaluator that is itself a component of PICopilot. The reported 48/48 and all baseline comparisons rest on this self-contained evaluator, not on an independent functional check of the generated scripts.

  3. self definitional [Sec 3.6 (custom check library)]
    "We establish this library by collecting common errors found in generated PIC design scripts and rewriting them as check prompts."

    The evaluator's check library is derived from errors observed in generated PIC design scripts, i.e., from the behavior of the very kind of system being evaluated. The feedback loop then iterates until this self-derived evaluator is satisfied, and the same evaluator is used to declare success. This closes the evaluation loop: the rubric is constructed from the system's own outputs, so 'correctness' as measured is not an externally anchored functional comparison but a match to a rubric generated from the system's own error modes.

full rationale

The paper's central empirical claim is that PICopilot solves all 48 tasks and outperforms LLM baselines. This claim is not supported by an independent evaluation: Sec 4.1.4 defines success as executing the generated script and comparing its output to ground truth, but Sec 3.6 says execution is impractical and the actual evaluator is an LLM with a static checker and a custom check library. The check library is built from 'common errors found in generated PIC design scripts,' so the evaluation rubric is derived from the system's own outputs; the feedback loop then iterates until this self-derived evaluator is satisfied. Additionally, the RRF fusion weights are set 'based on actual testing' and k=5, N_f=3 are fixed without a held-out split, all on the same 48-task benchmark whose Pass@1/#Solved are reported. Thus the headline numbers are in-sample and self-judged rather than independent predictions. This is not a case of problematic self-citation; the author-built benchmark is not itself circular. The circularity lies in the evaluation and hyperparameter tuning loop, making the 48/48 result partially reduce to the system's own fitted parameters and self-assessment.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 3 invented entities

The reported performance rests on a modest set of hand-set hyperparameters (RRF fusion weights tuned "based on actual testing", top-k=5, N_f=3), several domain assumptions (Python coverage of PIC tools, fidelity of LLM-generated manual summaries, objectivity of the execution-based success judgment), and three introduced software mechanisms (summary-based hybrid retrieval, custom check library, feedback loop) whose independent evidence is the internal ablation in Table 5. No new physical entities or external falsifiable handles are introduced; the burden is the standard one for a self-benchmarked ML system.

free parameters (4)
  • RRF fusion weights (w_dense, w_sparse) = (0.7, 0.3)
    Set "based on actual testing" (Sec 3.4.3); directly controls retrieval ranking and was tuned on the same 48-task benchmark used for the headline results.
  • Top-k retrieved documents per query = 5
    Chosen in Sec 4.1.1; sets the retrieval recall ceiling for the Programmer Agent.
  • Maximum feedback iterations N_f = 3
    Chosen in Sec 4.1.1; bounds the revision loop and affects Pass@1 within 15 trials.
  • RRF smoothing constant c = 60
    Default BM25/RRF smoothing constant (Sec 3.4.3); mild influence on merge ranking.
axioms (5)
  • domain assumption Python covers all PIC design tool invocations (layout, DRC, simulation)
    Sec 2.1 asserts Python is the default because "almost all PIC design tools support Python-based calls"; if a tool family (e.g., Lumerical's own scripting language, cited in [33]) is not Python-first, the framework's coverage narrows.
  • domain assumption LLM-generated, expert-verified manual summaries are faithful retrieval indices
    Sec 3.4.2 replaces raw chunks with LLM summaries matched against queries; lossy summaries would drop API names that the sparse retriever needs to find.
  • domain assumption Execution output comparison to ground truth is an objective, unbiased success measure
    Sec 4.1.4 defines success by "functionally identical" output but never states who compares or how; if the designer-judge is not blinded, bias is possible.
  • ad hoc to paper The 48 hand-selected tasks are representative of real-world PIC scripting
    Sec 4.1.3: tasks are "carefully selected from actual PIC design flows" with ground truth written by a PIC designer; no external validation, inter-rater agreement, or holdout split is reported.
  • standard math Standard retrieval math (cosine, BM25, RRF) and the Pass@k estimator are applied correctly
    Sec 3.4.3 and Sec 4.1.4; formulas match the literature and no derivation gaps were spotted.
invented entities (3)
  • Summary-based hybrid retrieval paradigm (Query Generator + summary matching + dense/sparse fusion) independent evidence
    purpose: Bridge the cross-modal gap between natural-language queries and code-heavy manual pages, and reduce semantic interference between tools.
    Ablation (Table 5) removes the tailored RAG and Advanced Pass@1 drops from 90.6 to 60.3, giving in-paper causal evidence; the evidence is internal, not an external falsifiable handle.
  • Custom check library for the Script Evaluator Agent no independent evidence
    purpose: Focus the LLM evaluator on error-prone patterns in PIC design scripts.
    No isolated ablation; built from "common errors found in generated PIC design scripts" (Sec 3.6), which may include errors on the test tasks themselves, so its credit is entangled with the benchmark.
  • Adaptive feedback loop across agents independent evidence
    purpose: Iteratively revise scripts in response to evaluator diagnoses (up to N_f=3 iterations).
    Removing the multi-agent architecture (which includes the loop) drops Advanced Pass@1 from 90.6 to 54.4 (Table 5); in-paper evidence only.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of PICopilot: An LLM-based Agentic Framework for Assisting Photonic Integrated Circuit Design via Script Generation." pith.science (2026). https://pith.science/paper/MMR4ZXAR

@misc{pith2026260801791,
  author       = {Pith},
  title        = {Pith review of: PICopilot: An LLM-based Agentic Framework for Assisting Photonic Integrated Circuit Design via Script Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MMR4ZXAR}},
  note         = {Machine review of arXiv:2608.01791}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

The rapid development of photonic integrated circuits (PICs) is shifting the design flow from traditional graphical user interface (GUI)-based methods to script-based methods for higher flexibility, portability, and maintainability. However, script-based design introduces new challenges, requiring designers to possess additional proficiency in tool application programming interfaces (APIs) and programming. It also demands greater effort and time because it is inherently less intuitive and more complex than GUI-based methods. As PICs grow in scale and complexity, the productivity gap between design needs and manual scripting capabilities continues to widen. To address this gap, we introduce PICopilot, the first large language model (LLM)-based agentic framework that assists in PIC design via automated design script generation from natural language instructions. PICopilot leverages a multi-agent architecture with a feedback mechanism and a specifically designed retrieval-augmented generation (RAG) pipeline, achieving a high success rate and reliability. Experimental results on a benchmark of diverse PIC scripting tasks demonstrate that PICopilot successfully completes all 48 tasks and outperforms other LLM-based approaches without incurring substantial extra latency or cost, even solving 21 more tasks than the advanced GPT-5 model with a general RAG pipeline.

Figures

Figures reproduced from arXiv: 2608.01791 by Jiang Xu, Wei Zhang, Xiaohan Jiang, Zeyu Li.

Figure 1
Figure 1. Figure 1: An illustration of the GUI-based PIC design and the [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of PICopilot. retrieval-augmented generation (RAG) [35] are widely adopted and proven effective. ICL enables LLMs to infer task-specific patterns through representative examples embedded in prompts, allowing the model to mimic the desired output without additional training. RAG, on the other hand, augments the LLM’s domain knowledge by retrieving relevant references from curated external databases… view at source ↗
Figure 3
Figure 3. Figure 3: An illustration of our agent design techniques. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: An illustration of the retrieval paradigm of the PIC [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: An illustration of the retrieval process in the general RAG pipeline and our RAG pipeline. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 33 canonical work pages

  1. [1]

    Shupeng Ning, Hanqing Zhu, Chenghao Feng, Jiaqi Gu, Zhixing Jiang, Zhoufeng Ying, Jason Midkiff, Sourabh Jain, May H Hlaing, David Z Pan, et al . 2024. Photonic-electronic integrated circuits for high-performance computing and ai accelerators.Journal of Lightwave Technology(2024)

  2. [2]

    Shawn Yohanes Siew, Bo Li, Feng Gao, Hai Yang Zheng, Wenle Zhang, Pengfei Guo, Shawn Wu Xie, Apu Song, Bin Dong, Lian Wee Luo, et al. 2021. Review of silicon photonics technology and platform development.Journal of Lightwave Technology39, 13 (2021), 4374–4389

  3. [3]

    Farshid Ashtiani, Alexander J Geers, and Firooz Aflatouni. 2022. An on-chip photonic deep neural network for image classification.Nature606, 7914 (2022), 501–506

  4. [4]

    Zhihao Xu, Tiankuang Zhou, Muzhou Ma, ChenChen Deng, Qionghai Dai, and Lu Fang. 2024. Large-scale photonic chiplet Taichi empowers 160-TOPS/W artificial general intelligence.Science384, 6692 (2024), 202–209

  5. [5]

    Saumil Bandyopadhyay, Alexander Sludds, Stefan Krastanov, Ryan Hamerly, Nicholas Harris, Darius Bunandar, Matthew Streshinsky, Michael Hochberg, and Dirk Englund. 2024. Single-chip photonic deep neural network with forward-only training.Nature Photonics18, 12 (2024), 1335–1343

  6. [6]

    Sufi R Ahmed, Reza Baghdadi, Mikhail Bernadskiy, Nate Bowman, Ryan Braid, Jim Carr, Chen Chen, Pietro Ciccarella, Matthew Cole, John Cooke, et al. 2025. Universal photonic artificial intelligence acceleration.Nature640, 8058 (2025), 368–374

  7. [7]

    Gdsfactory. 2023. GDSFactory 9.20.6. https://gdsfactory.github.io/gdsfactory/

  8. [8]

    Luceda. 2025. Luceda IPKISS. https://www.lucedaphotonics.com/luceda- photonics-design-platform

  9. [9]

    Siemens. 2025. L-Edit Photonics. https://eda.sw.siemens.com/en-US/ic/ic- custom/photonic/l-edit-photonics/

  10. [10]

    Matthias Köfferlein. 2020. KLayout

  11. [11]

    Spark Photonics. 2025. Check Mate DRC. https://www.sparkphotonics.com/ checkmatedrc

  12. [12]

    ANSYS Inc. 2025. Ansys Lumerical INTERCONNECT. https://www.ansys.com/ products/optics/interconnect

  13. [13]

    ANSYS Inc. 2025. Ansys Lumerical FDTD. https://www.ansys.com/products/ optics/fdtd

  14. [14]

    ANSYS Inc. 2025. Ansys Lumerical MODE. https://www.ansys.com/products/ optics/mode

  15. [15]

    Flexcompute. 2025. FAST, MODERN PHOTONIC SIMULATIONS. https://www. flexcompute.com/tidy3d/

  16. [16]

    Synopsys Inc. 2025. Synopsys OptSim. https://www.synopsys.com/photonic- solutions/optocompiler/optsim-photonic-ic.html

  17. [17]

    Yinyi Liu, Bohan Hu, Zhenguo Liu, Peiyu Chen, Linfeng Du, Jiaqi Liu, Xianbin Li, Wei Zhang, and Jiang Xu. 2023. FIONA: Photonic-Electronic CoSimulation Frame- work and Transferable Prototyping for Photonic Accelerator. In2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD). IEEE, 1–9

  18. [18]

    Xiaohan Jiang, Yinyi Liu, Peiyu Chen, Wei Zhang, and Jiang Xu. 2025. PICELF: An Automatic Electronic Layer Layout Generation Framework for Photonic Integrated Circuits. In2025 Design, Automation & Test in Europe Conference (DATE). IEEE, 1–7

  19. [19]

    Hao Chen, Yuzhe Ma, and Yeyu Tong. 2025. Bi-Level Optimization Accelerated DRC-Aware Physical Design Automation for Photonic Devices. In2025 Design, Automation & Test in Europe Conference (DATE). IEEE, 1–7

  20. [20]

    Yuchao Wu, Xiaofei Yu, Xianyi Feng, Yeyu Tong, and Yuzhe Ma. 2025. Constraints- aware Adaptive Routing with Hybrid Waveguides for Photonic Integrated Cir- cuits. In2025 IEEE/ACM International Conference on Computer Aided Design (ICCAD). IEEE, 1–8

  21. [21]

    Jingyu Pan, Guanglei Zhou, Chen-Chia Chang, Isaac Jacobson, Jiang Hu, and Yiran Chen. 2025. A survey of research in large language models for electronic design automation.ACM Transactions on Design Automation of Electronic Systems 30, 3 (2025), 1–21

  22. [22]

    Mingjie Liu, Teodor-Dumitru Ene, Robert Kirby, Chris Cheng, Nathaniel Pinckney, Rongjian Liang, Jonah Alben, Himyanshu Anand, Sanmitra Banerjee, Ismet Bayraktaroglu, et al. 2023. Chipnemo: Domain-adapted llms for chip design. arXiv preprint arXiv:2311.00176(2023)

  23. [23]

    Haoyuan Wu, Zhuolun He, Xinyun Zhang, Xufeng Yao, Su Zheng, Haisheng Zheng, and Bei Yu. 2024. Chateda: A large language model powered autonomous agent for eda.IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems43, 10 (2024), 3184–3197

  24. [24]

    Wenzhao Sun, Yanan Han, Bijian Lan, Qing Peng, and Jing Wan. 2025. Ana- sizecoder: Code generator for analog integrated circuit sizing automation via large language model. In2025 International Symposium of Electronics Design Automation (ISEDA). IEEE, 817–822

  25. [25]

    Yiting Wang, Wanghao Ye, Yexiao He, Yiran Chen, Gang Qu, and Ang Li. 2025. MCP4EDA: LLM-Powered Model Context Protocol RTL-to-GDSII Automation with Backend Aware Synthesis Optimization.arXiv preprint arXiv:2507.19570 (2025)

  26. [26]

    Bingyang Liu, Haoyi Zhang, Xiaohan Gao, Zichen Kong, Xiyuan Tang, Yibo Lin, Runsheng Wang, and Ru Huang. 2025. Layoutcopilot: An llm-powered multi-agent collaborative framework for interactive analog layout design.IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems(2025)

  27. [27]

    Yao Lai, Sungyoung Lee, Guojin Chen, Souradip Poddar, Mengkang Hu, David Z Pan, and Ping Luo. 2025. Analogcoder: Analog circuit design via training-free code generation. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 379–387

  28. [28]

    Yao Lai, Souradip Poddar, Sungyoung Lee, Guojin Chen, Mengkang Hu, Bei Yu, Ping Luo, and David Z Pan. 2025. Analogcoder-pro: Unifying analog circuit generation and optimization via multi-modal llms.arXiv preprint arXiv:2508.02518 (2025)

  29. [29]

    Jason Liu, Ankita Sharma, Cheick Doumbia, and Joyce KS Poon. 2024. Towards large-language model assisted layout of silicon photonic integrated circuits. In European Conference on Integrated Optics. Springer, 441–447

  30. [30]

    Yuchao Wu, Xiaofei Yu, Hao Chen, Yang Luo, Yeyu Tong, and Yuzhe Ma. 2025. PICBench: Benchmarking LLMs for Photonic Integrated Circuits Design. In2025 Design, Automation & Test in Europe Conference (DATE). IEEE, 1–6

  31. [31]

    Ankita Sharma, YuQi Fu, Vahid Ansari, Rishabh Iyer, Fiona Kuang, Kashish Mistry, Raisa Islam Aishy, Sara Ahmad, Joaquin Matres, Dirk R Englund, et al

  32. [32]

    Chen-Chia Chang, Chia-Tung Ho, Yaguang Li, Yiran Chen, and Haoxing Ren

  33. [33]

    Ansys Canada Ltd. 2025. Lumerical scripting language. https://optics.ansys.com/ hc/en-us/articles/360037228834-Lumerical-scripting-language-By-category

  34. [34]

    InProceedings of the 2025 International Symposium on Physical Design

    Drc-coder: Automated drc checker code generation using llm autonomous agent. InProceedings of the 2025 International Symposium on Physical Design. 143–151

  35. [35]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems33 (2020), 9459–9474

  36. [36]

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, et al . 2024. A survey on in-context learning. InProceedings of the 2024 conference on empirical methods in natural language processing. 1107–1128

  37. [37]

    Daoguang Zan, Zhirong Huang, Wei Liu, Hanwu Chen, Linhao Zhang, Shulin Xin, Lu Chen, Qi Liu, Xiaojian Zhong, Aoyan Li, et al. 2025. Multi-swe-bench: A multilingual benchmark for issue resolving.arXiv preprint arXiv:2504.02605 (2025)

  38. [38]

    John K Ousterhout. 1993. An Introduction to TCL and TK

  39. [39]

    Python Software Foundation. 2001. ast — Abstract syntax trees. https://docs. python.org/3/library/ast.html

  40. [40]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems35 (2022), 24824–24837

  41. [41]

    LangChain. 2025. LangChain. https://www.langchain.com/langchain

  42. [42]

    Python Software Foundation. 2026. pyflakes 3.4.0. https://pypi.org/project/ pyflakes/

  43. [43]

    Aixin Liu DeepSeek-AI, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, et al. 2025. DeepSeek-V3. 2: Pushing the Frontier of Open Large Language Models.arXiv preprint arXiv:2512.02556(2025)

  44. [44]

    Qwen Team. 2025. Qwen3 Technical Report. arXiv:2505.09388 [cs.CL] https: //arxiv.org/abs/2505.09388

  45. [45]

    Xiangyang Li, Kuicai Dong, Yi Quan Lee, Wei Xia, Yichun Yin, Hao Zhang, Yong Liu, Yasheng Wang, and Ruiming Tang. 2024. Coir: A comprehensive benchmark for code information retrieval models.URL https://arxiv. org/abs/2407.02883 (2024)

  46. [46]

    Henrique Schechter Vera, Sahil Dua, Biao Zhang, Daniel Salz, Ryan Mullins, Sindhu Raghuram Panyam, Sara Smoot, Iftekhar Naim, Joe Zou, Feiyang Chen, et al. 2025. Embeddinggemma: Powerful and lightweight text representations. arXiv preprint arXiv:2509.20354(2025)

  47. [47]

    Mark Chen. 2021. Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374(2021)

  48. [48]

    OpenAI. 2025. Introducing GPT-5. https://openai.com/index/introducing-gpt-5/

  49. [2025]

    AI Agents for Photonic Integrated Circuit Design Automation.arXiv preprint arXiv:2508.14123(2025)

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.