{"id":"92923a9e-08b2-4819-bbcd-84fb2bb5b014","arxiv_id":"2505.02168","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"CircuitFusion is a pretrained multimodal circuit encoder that fuses HDL code, structural graphs, and GPT-generated function summaries, and achieves state-of-the-art early-stage PPA prediction across five tasks.","lead":"The paper introduces CircuitFusion, a machine learning model that learns a shared representation of hardware circuits from three views: the hardware description code, the circuit graph, and a natural-language summary of its function. The authors show it predicts post-synthesis timing, power, and area better than task-specific baselines, and can do so with little or no task-specific training.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Circuit-level outperformance claims rest on only 8 test designs with no significance testing; retrieval k is tuned on the same test set.","rationale":"The reader's weakest assumption correctly identifies the small test set and absence of significance testing as the central vulnerability. My stress-test agrees and sharpens the concern: for WNS, TNS, power, and area, the evaluation unit is the complete design, so the effective sample is 8, not the thousands of sub-circuits used for the slack task. The paper's own statement 'We apply an 80/20 training/test split... 8 reserved for testing' makes this explicit. I also note two aggravating factors the reader did not emphasize: the retrieval count k is tuned on test-set performance in Section 4.4, and the retrieval vectorstore is built from the same four benchmark families that supply the test designs, so zero-shot retrieval may exploit near-duplicate sub-circuits. These factors reinforce the conditional verdict rather than overturning it. Positive aspects remain: the method is well-motivated, the code is public, the baselines include hardware-specific, text, and software-code encoders, and the authors even show that applying S1 and S4 to baseline encoders improves them, which is a useful control. But the quantitative headline claims need per-design significance testing or a larger, more diverse test split before they can be accepted as consistently demonstrated. The reader's CONDITIONAL verdict is therefore appropriate, and my analysis does not move it.","tokens_in":24456,"tokens_out":4604,"duration_ms":65319,"concrete_test":"Recompute the four circuit-level rows of Table 2 as per-design MAPE for all 8 test circuits, for CircuitFusion and for the strongest baseline on each task; then run a paired permutation test or Wilcoxon signed-rank test on the 8 paired per-design errors. If any 95% confidence interval for the MAPE difference includes zero, or if more than one test design favors the baseline, the 'consistently outperforms' claim is not supported at the claimed strength. Additionally, re-select the retrieval number k by cross-validation on the training split only, and report whether the fine-tuned MAPE advantage survives when k is not chosen on the test set.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that CircuitFusion consistently outperforms task-specific SOTA is load-bearing on the circuit-level evaluations of WNS, TNS, power, and area. Section 4 applies an 80/20 split by complete designs, leaving 8 test circuits, and each of these four metrics is a single number per design. With n=8, one outlier design can shift MAPE by several percentage points, and the paper reports no per-design breakdown, no variance across seeds or splits, and no significance test. The fine-tuned comparison in Table 2 therefore does not establish 'consistent' outperformance: the reported 5-13 percentage-point MAPE improvements could be driven by a small number of favorable test designs. This is compounded by two related evaluation choices. First, the retrieval hyperparameter k is selected using the test set: Section 4.4 states 'we set the retrieval number to 1' based on Table 3, which reports test-set top-k results; tuning on the test set inflates the reported fine-tuning and zero-shot numbers. Second, the retrieval vectorstore is built from training designs in the same four open-source benchmark families as the 8 test designs, so top-1 retrieval may return near-duplicate sub-circuits carrying near-identical PPA labels, making the zero-shot results hard to interpret as generalization. None of these issues invalidates the method, but they mean the headline quantitative claims are not yet statistically secured.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CircuitFusion, a multimodal and implementation-aware circuit encoder for RTL-stage design quality prediction. Circuits are split into register-bound sub-circuits, each represented by HDL code, a structural graph, and a GPT-generated functionality summary. Three unimodal encoders feed a cross-attention fusion encoder, which is pre-trained with four self-supervised tasks, including masked graph modeling, intra- and cross-modal contrastive learning, masked summary modeling, and alignment with a netlist encoder. For downstream tasks, the authors propose retrieval-augmented inference, which retrieves similar known sub-circuits and uses their quality metrics as references, supporting both fine-tuning and zero-shot prediction. Across five tasks (slack, WNS, TNS, power, area), the paper reports consistent MAPE improvements of 5 to 13 percentage points over task-specific SOTA baselines, together with ablations of the proposed strategies and modalities.","tokens_in":24696,"tokens_out":3105,"duration_ms":42286,"significance":"If the reported results are statistically reliable, CircuitFusion is a useful step toward general RTL-stage circuit representations: it is the first such encoder to fuse three circuit modalities and to explicitly align RTL with post-synthesis netlists, and the retrieval-augmented inference idea is practical and clearly explained. The paper ships an open-source repository and includes detailed ablations of strategies and modalities, which strengthens reproducibility. However, the headline claim of consistent state-of-the-art performance is currently supported by only 8 test designs for four of the five tasks, with no significance testing; retrieval hyperparameter k is selected on the test set; and the closest prior encoder from the same group is cited but not compared against. These issues leave the central quantitative claim plausible but not yet statistically secured.","major_comments":[{"comment":"The circuit-level evaluations for WNS, TNS, power, and area rest on only 8 test designs (33/8 split by complete design), with each metric being a single number per design. The paper reports no per-design breakdown, no variance across splits or seeds, and no significance test. With n=8, the reported 5-13 percentage-point MAPE improvements could be driven by a few favorable designs. Please report per-design errors, add multiple splits or bootstrap confidence intervals, and state whether the improvements are statistically significant.","section":"Section 4, Table 2"},{"comment":"The retrieval hyperparameter k is selected on the test set: the text states 'we set the retrieval number to 1' based on Table 3, which reports test-set top-k results. Selecting hyperparameters on the test set can inflate both the fine-tuned and zero-shot numbers. Please select k on a validation split, or report results across a range of k without test-set-based selection, and discuss the sensitivity of the final numbers to this choice.","section":"Section 4.4, Table 3"},{"comment":"The zero-shot retrieval setting is a label lookup from a vectorstore containing known circuits' quality metrics, and the store is built from the same four open-source benchmark families as the 8 test designs. Top-1 retrieval may therefore return near-duplicate sub-circuits carrying near-identical labels, which makes the zero-shot results hard to interpret as generalization to unseen implementations. Please quantify retrieval similarity (e.g., distance of retrieved neighbors), and report a version in which same-design or same-benchmark-family sub-circuits are excluded from the store.","section":"Section 4.4, Figure 4"},{"comment":"The authors cite their own prior ASP-DAC 2025 encoder (Fang et al., 2025), described as a self-supervised, pre-trained, cross-stage-aligned circuit encoder, but this direct predecessor is not included as a baseline in Table 2 or the ablations. Since CircuitFusion appears to extend that framework with multimodal fusion, sub-circuit splitting, and retrieval, at least one comparison against it is needed to isolate the incremental contributions claimed in the paper.","section":"Related Works and Section 4.2"}],"minor_comments":[{"comment":"The WNS MAPE for NV-Embed-v1 is reported as 17% in Table 2 but 26% in the 'ori' row of Table 10. Please clarify which value corresponds to the same evaluation setup.","section":"Table 2 vs Table 10"},{"comment":"Equation (8) defines an InfoNCE-style loss but labels it NCE, while the text frequently refers to CL; please define CL consistently as the InfoNCE loss used throughout the paper.","section":"Section 3.3, Eq. (8)"},{"comment":"The ablation bar charts show single numbers without error bars or markers of variance; adding error bars across seeds or at least stating the number of runs would make the ablation claims more robust.","section":"Figures 9 and 10"},{"comment":"The claim of being the 'first multimodal and implementation-aware circuit encoder' should be qualified with respect to earlier multimodal timing predictors such as Wang et al. (2023b), which also fuse multiple circuit representations; the novelty claim would be stronger if framed as first to fuse HDL code, graph, and summary for a general RTL-stage encoder.","section":"Introduction and Related Works"},{"comment":"The multi-clock experiment appears to use only two combined designs; please state explicitly that this is an illustrative case study rather than a full evaluation of multi-clock support.","section":"Appendix G, Table 12"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the journal's scope and the core idea is promising, but the quantitative claims need statistical hardening before acceptance. The missing comparison to the authors' own prior encoder is an important omission that should be addressed in revision. I do not see grounds for rejection; the issues appear fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Zhiyao and coauthors have built something real here. CircuitFusion is the first circuit representation model I know of that fuses HDL code, structural graph, and GPT-generated functionality summary at the RTL stage, and the sub-circuit splitting plus retrieval-augmented inference are sensible, well-engineered ideas. The architecture is not exotic, but the combination of four self-supervised tasks and the implementation-aware alignment to netlists is a legitimate step beyond single-modality encoders like SNS v2. Code is public, and the ablation study is more thorough than most: removing any modality or strategy hurts, and applying the splitting and retrieval tricks to baseline encoders improves them, which is a good control. The scaling plot, though thin, is also a plus.\n\nThe weak link is the evaluation. Four of the five tasks are circuit-level, with one number per complete design, and the test set is only 8 designs. That means Table 2's 5-13 point MAPE improvements over task-specific baselines could easily be driven by one or two favorable test circuits. There are no error bars, no seed variance, no significance test, and no per-design breakdown. On top of that, the retrieval count k is chosen by looking at Table 3's test-set results, which inflates the reported fine-tuning and zero-shot numbers. And because the retrieval vectorstore is built from training designs within the same four benchmark families as the test designs, top-1 retrieval may be pulling near-duplicate sub-circuits with near-identical labels; the zero-shot results are therefore hard to read as genuine generalization. The paper also does not compare against the authors' own closest prior self-supervised encoder from ASP-DAC 2025, which is a notable omission given the incremental novelty question.\n\nNone of this makes me doubt the method itself. The architecture is coherent, the ablations are internally consistent, and the direction is clearly useful for early-stage design quality estimation. But the central claim of consistent SOTA outperformance is not statistically secured as written. A serious referee should ask for a larger test split, cross-validation or repeated random splits, a comparison to the prior encoder, and a retrieval baseline that rules out label leakage. If those come back clean, this becomes a solid foundation-model paper for the ML-for-EDA community. I would cite it for the multimodal fusion idea and sub-circuit splitting, and I would bring it to a reading group, but I would not yet repeat the quantitative claims in my own writing.","headline":"CircuitFusion is a genuinely new multimodal RTL encoder with a plausible design, but the headline PPA numbers rest on only 8 test circuits with no significance testing, so the quantitative claims are provisional.","tokens_in":25250,"tokens_out":1268,"would_cite":true,"duration_ms":20501,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"One self-supervised, multimodal circuit encoder beats task-specific supervised models on five design-quality prediction tasks, and retrieving similar known circuits enables zero-shot prediction.","keywords":["circuit representation learning","multimodal learning","RTL design quality prediction","self-supervised pre-training","retrieval-augmented inference","PPA estimation","functionally equivalent transformation","chip design automation"],"falsifier":"Recompute the five-task comparison on a larger held-out set — say 30 or more complete designs, from benchmark families or design styles not used in pre-training — and check whether the reported 5 to 13 percentage point MAPE advantage over the task-specific baselines persists. In the same test, replace the retrieved reference metrics with metrics from randomly chosen dissimilar circuits: if prediction error is unchanged, retrieval is not doing the work the paper attributes to it.","tokens_in":24224,"feed_emoji":"⚡","tokens_out":21302,"duration_ms":195132,"temperature":0.7,"pith_summary":"CircuitFusion seeks to establish that one general-purpose circuit encoder can do the work of many task-specific models in chip design. It represents every circuit at the granularity of a single register's logic cone, in three modalities at once: the hardware code itself, a structural graph of logic operators, and an LLM-written plain-language summary of what the logic does. The fused encoder is pre-trained with self-supervised tasks that exploit facts unique to circuits — logic can be rewritten into functionally identical but structurally different forms, and an RTL description and its synthesized netlist are the same circuit at two design stages. On five early-stage design quality prediction tasks, the paper reports that this one approach consistently beats the state-of-the-art task-specific supervised model for each task, with 5 to 13 percentage points lower mean absolute percentage error after fine-tuning, and even enables zero-shot prediction by retrieving the quality metrics of the most similar known circuits. If the claim holds, designers could get timing, power, and area feedback on freshly written code without running synthesis and without commissioning a new model per task.","feed_headline":"One circuit encoder beats single-task models on five design tasks","feed_subtitle":"A single fused code-graph-summary encoder predicts timing, power, and area before synthesis and even zero-shot.","key_machinery":"The load-bearing unit is the register-boundary sub-circuit, called a sub-circuit in the paper: backtracing all combinational logic that feeds one register yields a slice describing exactly one state transition in one clock cycle, and applying the same split to the synthesized netlist keeps the RTL and netlist views functionally aligned by construction. Each sub-circuit is expressed in three modalities — HDL code, an operator graph, and an LLM-generated functionality summary — and encoded by three unimodal encoders: a graph transformer, a BERT-initialized summary transformer, and a frozen LLM text encoder for long code. A summary-centric fusion encoder then cross-attends the summary tokens against a mixup of graph and code embeddings. Four self-supervised objectives train this stack: masked graph modeling; intra- and cross-modal InfoNCE contrastive learning with functionally equivalent transformations as positive samples; masked summary modeling plus summary–mixup matching; and a cross-stage contrastive alignment between RTL fusion embeddings and a pre-trained netlist graph encoder. The same unit carries inference: embeddings of known sub-circuits are stored in a vector store, the measured quality metrics of the most similar retrieved sub-circuits are concatenated with the target embedding, and a lightweight regressor maps the combined vector to the prediction — this retrieval mechanism is what delivers both the fine-tuning gains and zero-shot prediction.","core_discovery":"The paper's claim is that circuits are inherently multimodal and that prior circuit representation learning, which reads only the structural graph, leaves most of the information needed for design-quality prediction unused. The discovery is that fusing three aligned views — Verilog code, an operator-level graph, and a generated functionality summary — at the grain of a register-boundary sub-circuit, and aligning the resulting representation with the gate-level netlist during pre-training, produces a representation that transfers across tasks. Pre-training combines four objectives: masked graph modeling and intra-modal contrastive learning whose positive pairs come from functionally equivalent transformations; cross-modal contrastive alignment; masked summary modeling with summary–mixup matching in a summary-centric cross-attention fusion encoder; and a contrastive alignment that pulls RTL fusion embeddings toward netlist graph embeddings. At inference, embeddings of known sub-circuits are indexed, and the measured quality metrics of the most similar retrieved sub-circuits are concatenated with the target embedding before regression, which improves fine-tuning and alone constitutes zero-shot prediction. Evaluated on five tasks, the single CircuitFusion model beats the task-specific state of the art on every task (12% vs 17% MAPE on slack, 11% vs 16% on WNS, 15% vs 25% on TNS, 13% vs 26% on power, and 11% vs 16% on area), and it beats general text and software code encoders by larger margins.","pith_inferences":["My inference: the register-cone split is a reusable currency beyond quality prediction — since each sub-circuit encodes one complete state transition, the same embedding index could support functional search over IP blocks, verification triage, or retrieval-conditioned RTL generation, all testable with the released model and no further pre-training.","My inference: retrieval-augmented inference assumes the vector store covers the distribution of new designs, which the paper does not test; error should grow as retrieved circuits become functionally less similar, and that degradation could be measured by ablating the store across benchmark families.","My inference: the implementation-aware alignment principle extends to later design stages — aligning circuit embeddings to floorplan, clock-tree, or layout data, not just netlists, should further sharpen early-stage predictions, and the same contrastive recipe could be reused directly.","My inference: because contrastive positives are generated by Boolean-equivalent transformations, the embedding space is organized by functional equivalence rather than surface syntax; a testable consequence is that structurally different implementations of the same function should embed as near neighbors, which would make the encoder useful for design-space exploration."],"forward_implications":["One pre-trained encoder covers many tasks: the same frozen CircuitFusion, fine-tuned with a lightweight regressor in about five minutes per task, beats the specialized supervised model built for each of the five tasks, removing the need for task-specific feature engineering.","Zero-shot prediction is possible: for designs with no fine-tuning labels, retrieving the top-1 most similar known sub-circuits and using their measured quality metrics as the prediction yields 21% MAPE on slack, versus 50–57% for general text and code encoders doing the same retrieval.","The application-side strategies transfer: adding sub-circuit splitting (S1) and retrieval-augmented inference (S4) to generic encoders such as the LLM text encoder and software code encoders improves their MAPE noticeably, though CircuitFusion still leads.","Performance scales with model and data: increasing the pre-trained model from 270M to 500M parameters lowers average fine-tuned error from about 19% to 12%, and growing pre-training data from 25% to 100% of the corpus gives a comparable drop.","Faster design iteration: because the predictions happen at the RTL stage, designers receive timing, power, and area feedback before running synthesis, and the paper notes the same predictions can feed early timing-optimization settings in synthesis flows."],"supporting_citations":[{"why":"RTL-Timer, the task-specific SOTA baseline for slack, WNS, and TNS prediction that CircuitFusion must beat.","marker":"(Fang et al., 2024a)"},{"why":"MasterRTL, the task-specific SOTA baseline for power and area prediction.","marker":"(Fang et al., 2024b)"},{"why":"SNS v2, the self-supervised pre-trained circuit encoder baseline for synthesis prediction.","marker":"(Xu et al., 2023)"},{"why":"NV-Embed-V1, the frozen LLM text encoder that serves as the code-modality backbone with 32K token capacity.","marker":"(Lee et al., 2024)"},{"why":"Yosys, used for HDL-to-Verilog conversion and functionally equivalent transformations that create contrastive positive samples.","marker":"(Wolf et al., 2013)"},{"why":"ABC, used to produce Boolean-equivalent but structurally different circuits for data augmentation and netlist contrastive learning.","marker":"(Brayton & Mishchenko, 2010)"},{"why":"Graphormer, the graph-transformer architecture on which the RTL graph encoder is built.","marker":"(Ying et al., 2021)"},{"why":"BERT, whose first six layers initialize the summary encoder and whose last six layers initialize the fusion encoder.","marker":"(Devlin, 2018)"},{"why":"ITC'99, one of the four open-source benchmark families that supply the pre-training and evaluation circuits.","marker":"(Corno et al., 2000)"}],"fun_headline_variants":["Multimodal circuit encoder beats single-task models on five design tasks","Code-graph-summary fusion powers chip design predictions that beat SOTA","CircuitFusion: one encoder, five design tasks, all state-of-the-art results","Retrieval-augmented circuit encoder enables zero-shot chip design prediction","Aligning RTL and netlist embeddings yields general circuit representations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that eight held-out test circuits (the 20% side of an 80/20 split of 41 designs) can reliably gauge prediction quality: four of the five headline metrics are one number per entire design, and no variance or significance testing is reported.","fun_headline_variants_meta":{"raw":{"variants":["Multimodal circuit encoder beats single-task models on five design tasks","Code-graph-summary fusion powers chip design predictions that beat SOTA","CircuitFusion: one encoder, five design tasks, all state-of-the-art results","Retrieval-augmented circuit encoder enables zero-shot chip design prediction","Aligning RTL and netlist embeddings yields general circuit representations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000789,"raw_usage":{"total_tokens":3576,"prompt_tokens":1138,"completion_tokens":2438,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":754,"completion_tokens_details":{"reasoning_tokens":2342}},"tokens_in":754,"tokens_out":2438,"duration_ms":19116,"temperature":1.0,"reasoning_tokens":2342,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:59:51.624160+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the five-task comparison on a larger held-out set — say 30 or more complete designs, from benchmark families or design styles not used in pre-training — and check whether the reported 5 to 13 percentage point MAPE advantage over the task-specific baselines persists. In the same test, replace the retrieved reference metrics with metrics from randomly chosen dissimilar circuits: if prediction error is unchanged, retrieval is not doing the work the paper attributes to it.","supporting_citations":[{"cited_title":"Abc: An academic industrial-strength verification tool","cited_arxiv_id":null,"evidence_quote":"ABC, used to produce Boolean-equivalent but structurally different circuits for data augmentation and netlist contrastive learning."},{"cited_title":"Rt-level itc'99 benchmarks and first atpg results","cited_arxiv_id":null,"evidence_quote":"ITC'99, one of the four open-source benchmark families that supply the pre-training and evaluation circuits."}],"review_version":1}