{"id":"3f71b4d2-c866-4584-b955-4ddfe15693a7","arxiv_id":"2506.15907","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"Pieceformer, a self-supervised hybrid graph transformer with partitioned training, improves graph similarity ranking for VLSI designs by 24.9% MAE over a GIN baseline and enables up to 89% faster KL partitioning.","lead":"This paper proposes Pieceformer, a machine learning model that compares how similar two chip design layouts are by looking at their structure. It is meant to help engineers reuse past design solutions to speed up new chip design work.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported 24.9% MAE gain is not interpretable because the synthetic evaluation never documents a train/test split; the model may be memorizing the exact graphs it ranks.","rationale":"The reader's REJECT verdict is well supported by multiple evaluation gaps, and my stress-test does not change it. However, I identify a different load-bearing concern than the reader's stated weakest_assumption. The reader focuses on the architectural limitation that attention never crosses partition boundaries (Section IV-C), which is a real theoretical constraint but is not the first thing that must be settled: even a perfectly expressive encoder would not support the central claim if the evaluation protocol allows the model to see the test graphs during training. The paper never states a train/test split for the synthetic MAE experiment, and the reported numbers are therefore ambiguous. The reader's rationale does mention the missing split, so there is partial agreement, but the reader's formal weakest_assumption field names the partition-boundary issue instead. I keep the verdict at UNCHANGED because the paper should still be rejected: the central quantitative claim lacks a documented evaluation protocol, the real-world clustering result is only visually assessed, and the KL case study omits the node-correspondence step required to transfer a partition. My concrete test targets the most direct route to settling whether the 24.9% MAE claim is real: retraining under a strict held-out-scale split and comparing held-out MAE. If the advantage persists under that protocol, this particular objection is resolved, though the other evaluation gaps would still need addressing before acceptance.","tokens_in":9462,"tokens_out":5043,"duration_ms":56387,"concrete_test":"Run a leave-one-scale-out experiment on the synthetic dataset: train each encoder on all graph scales except one, then evaluate MAE only on the held-out scale's 16 graphs (base plus 15 derived). Report per-scale held-out MAE for GIN and Pieceformer. If the 24.9% average improvement shrinks substantially or disappears on held-out scales, the reported ranking gain is attributable to memorization rather than to a generalizable similarity representation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing weakness is that the synthetic ranking evaluation does not establish generalization. Section V-A1 defines 16-graph groups (a base graph plus 15 derived graphs), and Section V-B reports MAE after training, but the paper never states which graphs are used for training versus evaluation. If the encoder was trained on the same graphs that are later embedded and ranked, the 24.9% improvement over GIN could reflect per-graph memorization rather than a generalizable similarity function. This is especially plausible under the InfoGraph contrastive objective, where graph-level embeddings are optimized against the exact node sets of the training graphs; no held-out graph group, cross-validation, or data-split table is described anywhere in Section V. The paper's own emphasis on 'label-free' and 'few as four graphs' makes the absence of a split even more consequential, because small training sets are particularly prone to overfitting. Without a split, the numeric headline in the abstract and the strongest claim is not scientifically interpretable. The CircuitNet clustering claim also relies on visual inspection of UMAP projections (Fig. 4), but the synthetic-data train/test gap is the most direct threat to the central quantitative result.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces Pieceformer, a self-supervised graph similarity framework for VLSI design reuse, which combines a message-passing GNN with a partitioned linear transformer (PGT) and trains embeddings via the InfoGraph contrastive objective. Input graphs are split into METIS subgraphs of configurable size, and subgraph embeddings are concatenated to form the graph embedding. The authors evaluate on a synthetic dataset of base graphs with 15 derived graphs per base, reporting a 24.9% average MAE reduction over a GIN baseline in similarity ranking, and on the CircuitNet dataset, where they claim the only method to correctly separate all design groups in UMAP projections. A case study applies the similarity ranking to initialize a Kernighan-Lin partitioner, reporting up to 89% runtime reduction.","tokens_in":9801,"tokens_out":8605,"duration_ms":80517,"significance":"The problem of scalable, label-free graph similarity for VLSI is timely, and the partitioned training pipeline is a pragmatic contribution to transformer scalability. The self-supervised setting and the JumpStart formulation are potentially useful. However, the current experimental protocol does not establish that the reported gains reflect generalization: the synthetic evaluation lacks a described train/test split, the baseline is not matched in partitioning, and the CircuitNet clustering claim is qualitative. If the experiments are properly reconducted, the framework could be a solid contribution; as written, the central quantitative claims are not yet supported.","major_comments":[{"comment":"The synthetic evaluation does not specify which graphs are used for training versus ranking. Section V-A1 describes 16-graph groups (base G_B and 15 derived graphs), and Section V-B reports MAE after training, but the paper never states whether the InfoGraph encoder was trained on the same graphs that are later embedded and ranked. If training and evaluation share graph instances, the 24.9% MAE improvement over GIN may reflect per-graph memorization rather than a generalizable similarity function. Please provide an explicit train/test split (e.g., hold out entire graph groups, or reserve a subset of derived graphs), and report MAE on the held-out graphs. Without this, the headline result is not scientifically interpretable.","section":"V-B, Fig. 2"},{"comment":"The baseline GIN is not partitioned, so the reported improvement cannot be attributed to the hybrid MP+PGT architecture versus the partitioning scheme. To isolate the effect of partitioning, include a partitioned GIN baseline (GIN applied to the same METIS subgraphs with concatenated embeddings, or GIN+PGT without the transformer). Also report standard deviations across multiple random seeds or graph groups; Fig. 2 shows a single MAE value per scale with no error bars, so the 24.9% average reduction has no measure of variance.","section":"V-B, Fig. 2"},{"comment":"The partition size (500) is selected using the same synthetic evaluation data on which Fig. 2 reports MAE. Because the hyperparameter is tuned on the test set, the reported gains are optimistic. Please either use a separate validation split for partition-size selection or report MAE across a range of partition sizes with error bars, and state clearly which numbers are selected post hoc.","section":"V-B, Fig. 3"},{"comment":"The claim that Pieceformer is the only method to correctly cluster all design groups is based on visual inspection of UMAP projections. This is not a quantitative evaluation. Please report cluster-quality metrics computed on the graph embeddings (e.g., silhouette score, adjusted Rand index against the known design groups, or k-nearest-neighbor accuracy), and specify the UMAP hyperparameters (n_neighbors, min_dist) and any normalization steps. In addition, clarify whether the model used for CircuitNet embeddings was trained on the synthetic dataset or on CircuitNet; if the latter, describe the training set and ensure the clustering evaluation is not conducted on the training data.","section":"V-C, Fig. 4"},{"comment":"The KL partitioning case study lacks essential details. First, it is not described how the partition of the most similar design is transferred to the target graph, which has a different node set; a mapping between nodes must be specified (e.g., by node degree or embedding alignment). Second, runtimes are reported as single numbers (53% and 89% reduction); KL is a randomized algorithm, so the comparison should report mean and standard deviation over many random initializations for both the JumpStart and random-start baselines. Third, the target graphs in this study have 100 and 1,000 nodes, which is far smaller than the 49k-node scale claimed in Table I, so the practical scalability claim is not demonstrated by this case study.","section":"V-D"},{"comment":"Because attention is applied only within each METIS subgraph, the 'global attention' of PGT is not global across the graph; cross-partition dependencies are invisible to the encoder. The paper does not provide evidence that VLSI graph similarity is dominated by intra-partition structure. Please add an ablation or analysis (e.g., compare METIS partitioning with random partitioning, or add a cross-partition interaction layer) to justify this architectural choice, or revise the terminology to avoid overstating the model's global reach.","section":"IV-C"}],"minor_comments":[{"comment":"The reference [5] cited for Performer is 'Masked language modeling for proteins via linearly scalable long-context transformers' (Choromanski et al., 2020), which is not the Performer paper. Please cite the correct source (e.g., Choromanski et al., 'Rethinking Attention with Performers', ICLR 2021).","section":"IV-B"},{"comment":"The amount of 'limited number of nodes/edges' in the edit operations is never quantified. Please specify the distribution or range of edit counts used to generate the derived graphs.","section":"V-A1"},{"comment":"The abstract and contribution list claim the method is 'effective even with minimal training data (as few as four graphs in practice)', but no experiment with only four training graphs is reported. Either add such an experiment or remove the claim.","section":"V-A1 and Conclusion"},{"comment":"The text says CircuitNet contains over 20,000 samples, but Table I lists 50 graphs. Clarify whether the 50 graphs in Table I are a subset of the 20,000 samples or the entire dataset used in this paper.","section":"V-C and Table I"},{"comment":"The runtime analysis is presented as a small table embedded in the figure, and the text references 'ΔRuntime' without defining the baseline for the percentage change. Please move this to a separate table with a clear baseline and annotate the units (s/epoch).","section":"Fig. 4"}],"recommendation":"major_revision","confidential_remarks":"The core idea is promising and the paper addresses an important problem, but the experimental evaluation in its current form does not support the headline quantitative claims. I would encourage the editor to require a thorough revision with a proper train/test split, matched baselines, and error bars; if the authors cannot provide these, the claims should be substantially scaled back."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing: the paper is worth reading but the headline number isn't. The 24.9% MAE reduction over GIN is presented as the central result, yet Section V never documents which synthetic graphs were used for training and which for evaluation. If the encoder was trained on the same graphs it later ranks, the improvement could come from fitting those exact graphs rather than from a generalizable similarity function. The paper's own emphasis on 'label-free' and 'as few as four graphs' makes the missing split more damaging, not less. This is the load-bearing weakness, and the stress-test note correctly identifies it.\n\nWhat's new here: the combination of InfoGraph-style contrastive learning with a linear transformer over METIS partitions, applied to VLSI graph similarity, is something I haven't seen. The JumpStart framing for design reuse is a useful way to think about the problem, and the synthetic data generation with controlled edit operations is a clever approach to building ground-truth rankings without human labels. The runtime/memory analysis of the partitioned pipeline is practical and the figures are clear.\n\nThe soft spots, in order of seriousness:\n- No train/test split (above).\n- No partitioned GIN baseline. The comparison is GIN (full graph) vs PGT (partitioned transformer) vs GIN+PGT. To claim the transformer adds value, you need a GIN that also sees the same METIS partitions. Otherwise the gain could just be from partitioning.\n- The CircuitNet evaluation is purely visual (UMAP projections). No cluster metrics, no numbers. 'Only method that clusters all groups' is not backed by a quantitative measure.\n- The KL case study omits the node-correspondence step. To initialize a target graph with a partition from a similar graph you need a mapping between node sets. The paper doesn't describe this, so the 89% runtime reduction is hard to interpret.\n- Partition size is tuned on the same synthetic evaluation data (Fig. 3), which is a mild circularity.\n\nOverall: the ideas are interesting and the presentation is honest about many limitations. But the empirical claims exceed what the experiments support. The fixes are straightforward: hold out synthetic groups, add a GIN+partition baseline, report cluster separability numbers, and explain how partitions transfer. I'd like to see this after revision. For now, I'd treat the 24.9% with suspicion, but the kernel is worth a serious referee.","headline":"Worth a referee's time, but the core 24.9% claim needs a documented train/test split and a partitioned GIN baseline before it means anything.","tokens_in":10145,"tokens_out":4165,"would_cite":false,"duration_ms":43020,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pieceformer claims that unlabeled VLSI circuits can be ranked by structural similarity with a partitioned linear-attention transformer, and that the resulting similarity score can JumpStart downstream EDA tasks, cutting partitioning…","keywords":["graph similarity","VLSI design reuse","self-supervised learning","graph transformer","partitioned training","linear attention","knowledge transfer","circuit netlist"],"falsifier":"Generate pairs of synthetic VLSI graphs that are identical within every roughly 500-node partition but differ only in a few long-range edges that cross the chosen partition boundaries. If Pieceformer ranks these pairs as more similar than pairs differing by the same number of within-partition edits, then cross-partition structure is invisible to the learned similarity; if ranking MAE stays at the same level as for full-graph edits, the concatenated embedding preserves enough global structure.","tokens_in":9276,"feed_emoji":"⚡","tokens_out":8711,"duration_ms":78101,"temperature":0.7,"pith_summary":"The paper claims that a large chip-design (VLSI) circuit can be judged similar to other circuits by an embedding learned entirely from graph structure, with no human labels, and that this similarity score can transfer a known good design solution to a new circuit. Pieceformer combines local message passing with a partitioned graph transformer built on linear attention, training on roughly 500-node subgraphs of real netlists and stitching the subgraph embeddings back together. On synthetic gate-level graphs this lowers mean absolute error in similarity ranking by 24.9% on average against the message-passing baseline, and on the real-world CircuitNet set it is the only tested method whose learned clusters separate every design group. In a Kernighan-Lin partitioning case study, using the most similar design's configuration as the starting point cuts runtime by 53% on 100-node graphs and up to 89% on 1000-node graphs. The importance, if the claim holds, is that design reuse no longer depends on expensive edit-distance computations or subjective similarity judgments.","feed_headline":"Chip-design similarity model cuts EDA runtime up to 89%","feed_subtitle":"Learns graph similarity without labels and reuses the best prior design, beating baselines on real netlists.","key_machinery":"The load-bearing object is the hybrid MP+PGT encoder: a message-passing layer that injects local context, followed by a partitioned graph transformer with linear attention that attends within roughly 500-node subgraphs instead of the full graph. The partitioner splits each netlist in preprocessing, and the pipeline concatenates the per-subgraph embeddings to reconstruct a graph-level embedding; a contrastive loss maximizes mutual information between node-level and graph-level embeddings of the same graph. This division of labor removes the quadratic attention cost and memory blow-up that makes full transformers infeasible above about 1.1k nodes, and it also avoids the softmax saturation that degrades attention when too many nodes compete, which is why a partition size around 500 minimizes ranking error.","core_discovery":"On its own terms, the paper's discovery is that the bottleneck for similarity-driven design reuse is not the similarity definition or the data, but the encoder's receptive field: message-passing alone is too local, full graph transformers are too memory-hungry and suffer attention dilution on multi-thousand-node circuits, while a pipeline that partitions each graph into roughly 500-node subgraphs, runs a linear-attention transformer on each partition, prepends a message-passing layer, and concatenates the subgraph embeddings into one graph embedding under a contrastive self-supervised objective gives accurate rankings and clean design-group clusters. This encoder produces graph-level embeddings whose L2 distance orders synthetic graphs by edit-distance-like structural change and separates real 28nm design groups that other encoders mix up. The same embedding, used as a JumpStart source selector, accelerates KL partitioning by reusing the winning partition of the nearest design.","pith_inferences":["A natural extension the paper leaves open is that, because attention never crosses partition boundaries, cross-partition wiring can only be encoded through the concatenated embedding; adding a lightweight global summary over partitions might capture long-range structure the current pipeline would miss.","Not tested in the paper: the 89% KL speedup is measured against a random initialization, so extrapolating to commercial EDA flows would require comparison against expert-tuned starts, a comparison the paper itself notes is complicated by a chicken-or-egg problem.","If the roughly 500-node partition sweet spot generalizes, the partitioned linear-attention recipe could transfer to other large-graph similarity tasks outside VLSI, such as program or social-network comparison, where labeled similarity is scarce.","A direct test of the partition-blindness assumption: generate synthetic edit operations that only rewire edges across the chosen partition boundaries, then check whether ranking MAE deteriorates; if it does, part of the similarity signal is an artifact of partition choice."],"forward_implications":["Design libraries can be searched by structural similarity without any labeled similarity scores, so a new netlist can inherit partitioning, synthesis, or floorplan configurations from the nearest prior design.","Graph size stops being a hard barrier to transformer training on circuits: the partitioned pipeline handles netlists of tens of thousands of nodes on a single GPU while using more of the GPU's compute.","Ranking accuracy holds across a wide range of sizes, with a reported 40.7% MAE improvement over the message-passing baseline at 1,000 nodes and a 24.9% average reduction across all tested scales.","The KL partitioning case study indicates that the speedup grows with graph size, 53% at 100 nodes and up to 89% at 1,000 nodes, concentrating the benefit exactly where EDA iteration is most expensive.","Because the method is label-free and task-agnostic, the same pretrained similarity score can JumpStart any iterative EDA task with a parameterized initial state, including synthesis configuration reuse and power grid tuning."],"supporting_citations":[{"why":"Supplies the contrastive mutual-information objective that makes the training label-free.","marker":"[24]"},{"why":"Provides the real-world CircuitNet 28nm netlist data used to test cluster separability of design groups.","marker":"[4]"},{"why":"Companion CircuitNet dataset version also used for the real-world evaluation.","marker":"[28]"},{"why":"Defines the message-passing baseline and the WL-expressiveness ceiling the paper claims to exceed.","marker":"[27]"},{"why":"Provides the full-attention graph transformer that motivates the linear partitioned backbone through its quadratic attention and cubic preprocessing.","marker":"[29]"},{"why":"Supplies the multilevel graph partitioner used to split large netlists into roughly 500-node subgraphs.","marker":"[13]"},{"why":"Provides the linear-attention transformer formulation used as the partitioned backbone.","marker":"[5]"},{"why":"Supports the claim that full-graph attention degrades at large scale through softmax saturation, motivating partitioning.","marker":"[21]"},{"why":"Defines the Kernighan-Lin partitioning task used in the JumpStart case study.","marker":"[14]"}],"fun_headline_variants":["Chip design reuse: 89% faster via self-supervised graph transformer","Pieceformer: Graph transformer for VLSI reuse cuts time 89%","Self-supervised graph model accelerates chip partitioning 89%","Scalable graph transformer improves VLSI similarity, cuts runtime 89%","Graph transformer learns chip similarity, speeds reuse up to 89%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that chopping each circuit into roughly 500-node subgraphs and concatenating their embeddings keeps the global structure that determines similarity; because attention never crosses subgraph boundaries, any design similarity carried by cross-partition wiring is invisible to the model.","fun_headline_variants_meta":{"raw":{"variants":["Chip design reuse: 89% faster via self-supervised graph transformer","Pieceformer: Graph transformer for VLSI reuse cuts time 89%","Self-supervised graph model accelerates chip partitioning 89%","Scalable graph transformer improves VLSI similarity, cuts runtime 89%","Graph transformer learns chip similarity, speeds reuse up to 89%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000753,"raw_usage":{"total_tokens":3314,"prompt_tokens":871,"completion_tokens":2443,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":487,"completion_tokens_details":{"reasoning_tokens":2358}},"tokens_in":487,"tokens_out":2443,"duration_ms":14472,"temperature":1.0,"reasoning_tokens":2358,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:28:49.004739+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate pairs of synthetic VLSI graphs that are identical within every roughly 500-node partition but differ only in a few long-range edges that cross the chosen partition boundaries. If Pieceformer ranks these pairs as more similar than pairs differing by the same number of within-partition edits, then cross-partition structure is invisible to the learned similarity; if ranking MAE stays at the same level as for full-graph edits, the concatenated embedding preserves enough global structure.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the contrastive mutual-information objective that makes the training label-free."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the real-world CircuitNet 28nm netlist data used to test cluster separability of design groups."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Companion CircuitNet dataset version also used for the real-world evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Kernighan-Lin partitioning task used in the JumpStart case study."}],"review_version":1}