REVIEW 4 major objections 3 minor 20 references
AgenticECO: An Agentic Framework for ECO on 3D Integrated Circuits
T0 review · 4 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read This paper claims that an evidence-gated agentic framework can repair post-route hybrid-bond defects in 3D integrated circuits more often and with far less disturbance than full reroute or stock repair, by making each edit attributable thro
desk verdict A genuinely transparent agentic-ECO framework with real audit artifacts, but the headline 7/9 vs 2/9 gap rests on single runs and the paper's own disclosed protocol breaches; worth serious refereeing, not desk rejection. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the evidence-gated control loop plus the EcoRoute executor. EcoRoute drives the unmodified pinned router on a copy of the design so that only nets incident to the moved instances are rerouted, every other net is frozen and checked net-by-net by geometry hash, and clock nets are never admitted; this makes the repair attributable to the edit. The loop's decision point—Advance, Query, or Revise—lets the agent refute a whole candidate family on evidence rather than resample a fixed procedure.
What would settle it
Re-run each of the nine sealed cases, say five times per arm, under the identical budget and measure the distribution of cleared cases and disturbance; if the 7/9 versus 2/9 gap collapses toward parity or the disturbance figures vary by more than a factor of two, the central claim fails.
Extended reading notes
Core claim
The paper claims that AgenticECO, an evidence-gated multi-agent framework paired with the EcoRoute executor, can repair post-route hybrid-bond cut-spacing defects in 3D ICs that standard full reroute and stock repair leave standing. Across nine natural defect cases under identical budgets, AgenticECO clears seven versus two for both baselines, with a mean disturbance of 0.66% over cleared cases and zero clock nets touched, and a rerun with a different LLM backbone under the same sealed contract clears all nine. The central design is that the orchestrator never sees the full design or emits free-form edits: it gathers typed, provenance-bearing evidence from read-only specialists, synthesizes
Load-bearing premise
The evaluation assumes that one official run per arm per case under the fixed 40-call/3-signoff budget gives stable, representative results; if LLM or router nondeterminism is large, the headline gap and disturbance numbers could change materially on re-run, and the paper reports no repeated seeds or confidence intervals.
Editorial extensions
If this is right
- Repairs that full reroute and stock repair cannot achieve become reachable under a fixed tool budget, at roughly one-hundredth the disturbance.
- A signoff result can be attributed to the edit that caused it, because untouched nets are certified byte-identical by a disturbance ledger.
- The method transfers across LLM backbones under the same sealed contract, indicating the residuals on one backbone are search floors, not physical limits.
- Removing the minimal-disturbance executor collapses the system into a full reroute; removing the visual occupancy instrument loses only contested landing sites; removing delegation fails the hardest cases.
- Every accepted repair passes routing, fresh extraction, timing, DRC, and structural-equivalence gates, so accepted states are externally verifiable.
Reading between the lines
- If the single-run evaluation is noisy, the 7/9 versus 2/9 gap could shrink on re-runs; running each sealed case several times under the same budget would give the first real confidence interval.
- The same evidence-gated, verified-execution pattern could apply to other post-route repair tasks—timing ECO, EM/IR fixes—where an edit's effect is only observable after physical execution.
- The disturbance ledger idea generalizes: any tool-using agent that modifies a stateful engineering database could certify that its change is isolated to the intended objects.
- The paper's own boundary section notes the I2C closure was in part a development run; a fully sealed, held-out natural-defect benchmark would be the natural scale-out test.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents AgenticECO, an evidence-gated agentic framework for post-route ECO repair of hybrid-bond cut-spacing defects in 3D ICs, together with EcoRoute, a minimal-disturbance ECO-routing executor, and an independent artifact-only verifier. On nine natural defects across GCD, I2C, and UART on a pinned TaiWei/ASAP7 flow, the authors report that AgenticECO clears 7/9 cases versus 2/9 for both full reroute and stock repair, with 0.66% mean disturbance over cleared cases, zero clock nets touched, and a GPT-5.6 cross-backbone rerun under the same sealed contract clearing 9/9. Extensive appendices document clean-room controls, a hash-chained evidence ledger, per-cell results, component ablations, visual feasibility campaigns, and explicit scope limitations.
Significance. If the reported results hold, this is a substantial systems contribution: it is the first agentic framework for 3D-IC ECO, and the design of the attribution machinery—EcoRoute freezing non-incident nets with per-net geometry hashes, an independent verifier that consumes no model text, and a replayable audit ledger—is well motivated and carefully engineered. The release of code, pinned environment, and per-episode audit artifacts, together with unusually candid appendix disclosures, is a clear strength. The central claim of superiority over full reroute and stock repair is plausible but currently rests on single-run measurements with several disclosed protocol exceptions, so the quantitative headline is not yet a stable comparative estimate.
major comments (4)
- [Section 4.1 and Appendix I] The central comparative claim (7/9 vs 2/9, 0.66% disturbance) is based on exactly one run per arm per case: Section 4.1 states 'Every method runs each case once' and Appendix I repeats 'every method arm executes exactly once against the sealed state under one budget.' No repeated seeds, confidence intervals, or variance information are reported. Given LLM and router nondeterminism, a single realization cannot establish that the gap is stable; the paper should either provide repeated-run evidence or explicitly re-scope the headline claim from comparative superiority to an existence proof.
- [Appendix O and Table 7] The sealed-protocol guarantee is not uniformly satisfied. Appendix O discloses that the rank-2 candidate's exact-placement (M0) check was omitted contrary to the sealed rule, the interactive Codex session lacks a sealed model transcript and exact sampling metadata, the UART no-edit control failed, and the GPT-5.6 single-agent GCD-2 cell overran the call budget with 16 unregistered engine invocations and moved one bond via. Table 7 reports the overrunning cell 'as-is.' Because each arm runs once, these exceptions cannot be bounded by variance. The cross-backbone 9/9 and the 7/9 headline therefore are not uniformly protocol-identical measurements; a strict re-run could change the numbers. The authors should either re-run the affected arms under a fully uniform sealed protocol or prominently qualify the abstract and Section 4.2 to state that the reported comparisons include disclosed protoc
- [Section 4.2 vs Appendix I] The main text states that the single-agent control 'gives up only delegation, orchestration, and preregistered revision,' but Appendix I discloses that AgenticECO arms may receive a predeclared eight-rule cross-case playbook, while the clean-room single-agent baseline receives no repair experience. The appendix explicitly says this makes the comparison 'an end-to-end systems comparison, not a causal estimate of delegation alone.' This is a material mismatch between the main-text attribution claim and the actual protocol. Either the single-agent arm should receive the same playbook, or the main text should state that the comparison is of the full AgenticECO system versus a minimal single-agent baseline.
- [Appendix L, Step 4] The claim that acceptance gates are frozen and 'may never be revised' (Section 3.3, Appendix C.1) is contradicted by the account in Appendix L, Step 4, where the verifier's freeze gate was amended after a fail-closed rejection—described as 'the contract, not the measurement, is what gets amended' in the 'stricter-to-looser direction.' This is a change to the verifier, not merely to a repair procedure. The authors should state clearly when this amendment occurred relative to all official runs, and if it predates them, say so explicitly; as written, the main text overstates the immutability of the verification gate.
minor comments (3)
- [Abstract / Section 4.2] The abstract and Section 4.2 report '0.66% mean disturbance over cleared cases.' The denominator should be stated more prominently: it is the mean over the seven cleared cases, not over all nine, and the two uncleared cases are excluded.
- [Table 1] For the Opus 4.8 AgenticECO row, 'DRC0 cases' is 7/9 while 'All gates pass' is 5/9. The text explains the timing trips, but the table would benefit from an explicit note so readers do not conflate DRC clearance with full gate acceptance.
- [Appendix M] The held-out UART hash-random campaign includes one development-exposed anchor, so the held-out claim rests on 4/4 versus 4/4, not 5/5. This caveat is disclosed in the appendix table note but should also appear in the main-text summary of the visual campaigns.
Circularity Check
No significant circularity: outcomes are measured against external baselines; the low-disturbance invariant is a designed constraint, not a fitted prediction.
full rationale
The paper's central derivation chain is self-contained. AgenticECO generates a deterministic repair program from typed evidence, EcoRoute executes registered edits under an explicit freeze invariant, and an independent verifier applies fixed signoff gates. The headline 7/9 vs 2/9 result is an empirical comparison against external baselines (full reroute and stock repair) on nine natural post-route defects, with a cross-backbone rerun independently re-verified from raw artifacts. Disturbance is defined in Eq. (6) via per-net geometry hashes, and Eqs. (10)-(11) impose the freeze invariant; the reported low disturbance is therefore a designed property of the executor rather than a fitted prediction. The paper itself states this framing: 'Localized execution constrains disturbance and the verifier measures it.' The Appendix L attribution replay re-checks the same executor invariant and is presented as a verification of the executor's contract, not as an independent discovery used to derive repair success. Self-citations (ChatEDA, Divergent Thoughts) appear only as contextual related work and carry no load-bearing premise; no uniqueness theorem is imported from the authors. The preregistered visual and diagnostic studies use frozen policies and held-out inputs, so they do not reduce to fitted parameters. Appendix O discloses real protocol limitations—omitted M0 check on a rank-2 candidate, absent sealed Codex transcript, failed UART no-edit control, and a GCD-2 budget overrun—and explicitly scopes the results as mechanism evidence rather than statistical superiority. Those are correctness and robustness concerns about single-run evaluation, not circularity: they do not make any claim equivalent to its own inputs by construction. Accordingly, no circular step is identified.
Assumptions & free parameters
free parameters (4)
- verification budget K =
40 tool calls
- signoff budget =
3 attempts
- escalation bound k =
k <= 3
- delegation width =
up to 3 parallel specialists
assumptions (5)
- domain assumption The ASAP7 predictive PDK and the pinned TaiWei flow with OpenROAD build are a faithful substrate for 3D-IC post-route ECO.
- domain assumption The acceptance gates (routing completion, DRC, max/min timing, structural equivalence, preservation) define a correct repair.
- domain assumption LLM backbones (Claude Opus 4.8, GPT-5.6) provide stable enough reasoning that single-run comparisons are meaningful.
- domain assumption Natural defect cases generated by HBT-density and utilization sweeps are representative of post-route 3D-IC defects.
- standard math The geometry-hash ledger correctly detects any change in untouched nets.
invented entities (3)
-
EcoRoute executor
independent evidence
-
V-Pix visual occupancy instrument
independent evidence
-
Evidence ledger
independent evidence
Cite this review
Pith. "Pith review of AgenticECO: An Agentic Framework for ECO on 3D Integrated Circuits." pith.science (2026). https://pith.science/paper/SBGTQGSN
@misc{pith2026260803738,
author = {Pith},
title = {Pith review of: AgenticECO: An Agentic Framework for ECO on 3D Integrated Circuits},
year = {2026},
howpublished = {\url{https://pith.science/paper/SBGTQGSN}},
note = {Machine review of arXiv:2608.03738}
}
read the original abstract
As Moore's law slows, the industry is turning to three-dimensional integration; yet in merged 3D-IC flows, routed designs expose bond-level defects with no 2D analogue, and post-route engineering change orders (ECO) remain manual, expertise-bound work. Worse, the standard edit-then-fully-reroute practice entangles a repair with router churn, so a signoff number cannot be attributed to the edit that motivated it. We present AgenticECO, an evidence-gated tool-using agent workflow for 3D-IC ECO on the open-source TaiWei flow, paired with EcoRoute, a minimal-disturbance ECO-routing layer that drives the unmodified pinned router so a repair is attributable to its edit. Across nine matched natural defect cases under identical budgets, AgenticECO clears seven versus two for both full reroute and stock repair, at 0.66\% mean disturbance over cleared cases and zero clock nets touched, and a cross-backbone rerun under the same sealed contract clears all nine. Controlled studies show that the repair moves are necessary under preservation, that occupancy-aware choice buys legal landings rather than repair success, and that under tightened clocks minimal disturbance flips accept versus reject. Three preregistered visual studies localize the pixel instrument's edge to contested landing sites, and a preregistered blind diagnostic exactly restores every held-out injected defect, the only arm with zero wrong edits. Every accepted result passes routing, fresh extraction, max/min timing, DRC, and structural-equivalence gates. Code, environment, and per-episode audit artifacts are released as supplementary material.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[5]
The agent is also the only method with zero wrong edits anywhere in the grid
exceed the agent’s worst (+29.1%, Seed 4), and on that same Seed 5 the agent recovers+51.1%. The agent is also the only method with zero wrong edits anywhere in the grid. O Scope and Claim Boundaries Withtherawevidencenowonthetable,thissectiondrawsthe line around what it supports. The main paper claims mech- anism evidence on a pinned engine rather than s...
work page 2022
-
[6]
Au- tomated QoR Improvement in OpenROAD with Coding Agents.arXiv preprint arXiv:2601.06268. Ghose, A.; Kahng, A. B.; Kundu, S.; and Wang, Z
-
[7]
InProceedings of the 2025 ACM/IEEE International Symposium on Machine Learning for CAD, 1–13
ORFS-Agent:Tool-UsingAgentsforChipDesignOptimiza- tion. InProceedings of the 2025 ACM/IEEE International Symposium on Machine Learning for CAD, 1–13. Gou, Z.; Shao, Z.; Gong, Y.; Shen, Y.; Yang, Y.; Duan, N.; and Chen, W
work page 2025
-
[8]
CRITIC: Large Language Models Can Self-CorrectwithTool-InteractiveCritiquing.arXivpreprint arXiv:2305.11738. Jiang, W.; Chhabria, V. A.; and Sapatnekar, S. S
-
[9]
InProceedings of the 2024 ACM/IEEE Interna- tional Symposium on Machine Learning for CAD, 1–7
IR-Aware ECO Timing Optimization Using Reinforcement Learning. InProceedings of the 2024 ACM/IEEE Interna- tional Symposium on Machine Learning for CAD, 1–7. Ku,B.W.;Chang,K.;andLim,S.K.2019. Compact-2D:A PhysicalDesignMethodologytoBuildTwo-TierGate-Level 3-D ICs.IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 39(6): 1151–11...
work page 2024
-
[10]
PDAGENT-BENCH: Characterizing, Grounding, and Architecting LLM Agents for VLSI Physical Design
PDAGENT-BENCH: Characterizing, Grounding, and Architecting LLM Agents for VLSI Physical Design. arXiv preprint arXiv:2606.17253. Liu, B.; Zhang, H.; Gao, X.; Kong, Z.; Tang, X.; Lin, Y.; Wang, R.; and Huang, R
-
[11]
LayoutCopilot: An LLM- PoweredMultiagentCollaborativeFrameworkforInteractive Analog Layout Design.IEEE Transactions on Computer- Aided Design of Integrated Circuits and Systems, 44(8): 3126–3139. Liu, M.; Ene, T.-D.; Kirby, R.; Cheng, C.; Pinckney, N.; Liang,R.;Alben,J.;Anand,H.;Banerjee,S.;Bayraktaroglu, I.;etal.2023. ChipNeMo:Domain-AdaptedLLMsforChip D...
arXiv 2023
-
[12]
Retrieve, Schedule, Reflect: LLM Agents for Chip QoR Optimization
Retrieve, Schedule, Reflect: LLM Agents for Chip QoR Optimization.arXiv preprint arXiv:2603.13767. Madaan, A.; Tandon, N.; Gupta, P.; Hallinan, S.; Gao, L.; Wiegreffe, S.; Alon, U.; Dziri, N.; Prabhumoye, S.; Yang, Y.; et al
Show all 20 references
-
[13]
Pentapati, S.; Chang, K.; and Lim, S
JARVIS: A Multi-Agent Code Assistant for High-Quality EDA Script Generation.arXiv preprint arXiv:2505.14978. Pentapati, S.; Chang, K.; and Lim, S. K
-
[14]
Shi, Z.; Gao, S.; Chen, X.; Feng, Y.; Yan, L.; Shi, H.; Yin, D.;Ren,P.;Verberne,S.;andRen,Z.2024
Open3DBench: Open- Source Benchmark for 3D-IC Backend Implementation and PPA Evaluation.arXiv preprint arXiv:2503.12946. Shi, Z.; Gao, S.; Chen, X.; Feng, Y.; Yan, L.; Shi, H.; Yin, D.;Ren,P.;Verberne,S.;andRen,Z.2024. LearningtoUse Tools via Cooperative and Interactive Agents...
2024 arXiv
-
[15]
https://news.siemens.com/en- us/siemens-fuse-eda-ai-agent/
Siemens launchesFuseEDAAIAgent. https://news.siemens.com/en- us/siemens-fuse-eda-ai-agent/. Synopsys.2026.AgenticAI:AutomatingEngineeringWork- flowswithAIAgents.https://www.synopsys.com/ai/agentic- ai.html. Thakur,S.;Ahmad,B.;Pearce,H.;Tan,B.;Dolan-Gavitt,B.; Karri, R.; and Garg, S
2026
-
[17]
Wu, B.-Y.; Sharma, U.; Rovinski, A.; and Chhabria, V
EvoDRC: A Self-Evolving Agentic Frame- work for Automated DRC Violation Repair.arXiv preprint arXiv:2607.20019. Wu, B.-Y.; Sharma, U.; Rovinski, A.; and Chhabria, V. A. 2025a. OpenROAD Agent: An Intelligent Self-Correcting Script Generator for OpenROAD. InProceedings of the 20...
2025 arXiv
-
[19]
Zhou, A.; Yan, K.; Shlapentokh-Rothman, M.; Wang, H.; and Wang, Y.-X
LLM4EDA: EmergingProgressinLargeLanguageModelsforElectronic Design Automation.arXiv preprint arXiv:2401.12224. Zhou, A.; Yan, K.; Shlapentokh-Rothman, M.; Wang, H.; and Wang, Y.-X
-
[2016]
Wang, X.; Chen, Y.; Yuan, L.; Zhang, Y.; Li, Y.; Peng, H.; andJi,H.2024
The Chips Are Down for Moore’s Law.Nature, 530(7589): 144–147. Wang, X.; Chen, Y.; Yuan, L.; Zhang, Y.; Li, Y.; Peng, H.; andJi,H.2024. ExecutableCodeActionsElicitBetterLLM Agents. InProceedingsofthe41stInternationalConference on Machine Learning. Wu, B.-Y.; Ho, C.-T.; Yang, H...
2024
-
[2019]
InProceedings of the 56th Annual Design Automation Conference, 1–4
Toward an Open-Source Digital Flow: First Learnings from the OpenROAD Project. InProceedings of the 56th Annual Design Automation Conference, 1–4. Blocklove,J.;Garg,S.;Karri,R.;andPearce,H.2023. Chip- Chat:ChallengesandOpportunitiesinConversationalHard- ware Design. InProceedi...
2023
-
[2022]
InNeurIPS 2022 Foundation Models for Decision Making Workshop
ReAct: Synergizing Reasoning and Acting in Language Models. InNeurIPS 2022 Foundation Models for Decision Making Workshop. Zang, Z.; Song, Y.; Wang, A.; Ling, B. W.-K.; Sun, Q.; Lei, Z.;Yang,F.;Zhuo,C.;andLuo,J.2025.TheDawnofAgentic EDA: A Survey of Autonomous Digital Chip Des...
2022
-
[2023]
InProceedings of the 2023 IEEE/ACM International Conference on Computer-Aided Design
GPT4AIGChip: Towards Next- Generation AI Accelerator Design Automation via Large Language Models. InProceedings of the 2023 IEEE/ACM International Conference on Computer-Aided Design. Ghose, A.; Jang, J.; Kahng, A. B.; and Lee, J
2023
-
[2024]
Cadence Design Systems
Can EDA Tool Feedback Improve Verilog Generation by LLMs?arXiv preprint arXiv:2411.11856. Cadence Design Systems
-
[2025]
InPro- ceedings of the 2025 8th Artificial Intelligence and Cloud Computing Conference, 407–415
Large Language Models for 3D IC Space Planning. InPro- ceedings of the 2025 8th Artificial Intelligence and Cloud Computing Conference, 407–415. Deng, J.; Chen, Z.; Wei, X.; Xing, T.; Wen, C.; and Zhuo, C
2025
-
[2026]
Fu, Y.; Zhang, Y.; Yu, Z.; Li, S.; Ye, Z.; Li, C.; Wan, C.; and Lin, Y
Can AI Agents Really Complete RTL-to-GDS? Lessons from Benchmarking Tool-Interactive EDA Work- flows.arXiv preprint arXiv:2607.17528. Fu, Y.; Zhang, Y.; Yu, Z.; Li, S.; Ye, Z.; Li, C.; Wan, C.; and Lin, Y. C
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.