{"id":"2f77e399-9a67-4215-a7ac-5328647f3277","arxiv_id":"2602.19330","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Generic graph coarsening of gate-level netlists cuts GNN memory by about 17x and training time by about 3x, but makes clock-skew prediction R^2 negative on unseen designs.","lead":"This paper introduces CTS-Bench, a benchmark of 4,860 clock tree synthesis designs with paired raw and clustered graphs, and shows that graph coarsening trades memory and speed for accuracy in clock-skew prediction. It matters because it gives the chip-design machine-learning community a standardized way to test whether compressed netlist graphs can still support reliable clock analysis.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unequal model capacity confounds the Raw-vs-Clustered comparison; the negative R^2 attributed to coarsening may be a hyperparameter artifact.","rationale":"I read the paper in good faith. CTS-Bench is a potentially valuable resource: it provides paired raw/clustered graphs from a reproducible OpenLane flow, a large multi-architecture dataset, and a clear zero-shot split. The efficiency gains (17.2x VRAM, 3x training time) are plausible and not the main issue. The load-bearing claim, however, is that coarsening—not model size or learning rate—causes the catastrophic loss of clock-skew fidelity. The experimental protocol in Table 3 varies model capacity at the same time as graph resolution, so the causal attribution is confounded. The reader's weakest_assumption correctly identifies this, and my analysis agrees. The concern is concrete and testable: one matched-hyperparameter rerun determines whether the negative-R^2 result survives. Until that is done, the central claim should be treated as conditional, not established. I also agree that the 'generic clustering' framing overreaches: a single custom physics-aware clustering algorithm is not evidence about all generic clustering. These are standard controlled-variable and external-validity concerns, not signs of misconduct or internal inconsistency. Therefore the verdict remains CONDITIONAL, and no adjustment is needed from the reader's assessment.","tokens_in":9991,"tokens_out":4657,"duration_ms":45981,"concrete_test":"Retrain all Clustered models (GCN, GraphSAGE, GATv2) using exactly the Raw hyperparameters from Table 3 (hidden=64, lr=0.001, placement-dim=32, cts-dim=16) on the same 80/20 seen split and the same 500 Zipdiv examples; keep batch size 32 and 100 epochs. Report skew R^2 on seen and unseen. If Clustered R^2 remains negative, the coarsening interpretation is supported; if it becomes comparable to Raw (>=0.9 seen, >=0 on Zipdiv), the claim fails. As a symmetric control, train Raw with Clustered hyperparameters to show capacity alone reproduces the gap. Optionally, replace the custom clustering with METIS at a similar compression ratio to test the 'generic' wording.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central inference—that coarsening destroys CTS-relevant fidelity—rests on comparing Raw and Clustered models trained with different capacities. Table 3 specifies Raw: hidden dim 64, LR 0.001, placement-MLP 32, CTS-MLP 16; Clustered: hidden dim 16, LR 0.0005, placement-MLP 8, CTS-MLP 4. Thus the Clustered arm is a strictly weaker model on three axes. The observed collapse in skew R^2 (e.g., GCN -2.23 on Zipdiv; 'frequently' <0 on seen data) is therefore not identifiable as an effect of graph coarsening: a smaller network with a 2x-lower learning rate can exhibit mean-level or worse predictions for reasons unrelated to graph structure. Since the abstract and conclusion make a causal claim about coarsening ('generic graph clustering techniques can fundamentally compromise CTS learning objectives'), the experimental design must isolate graph resolution. It currently does not. A secondary issue: the paper calls the custom BFS/gravity-merge clustering 'generic' without comparing to standard coarseners (e.g., METIS) or a task-agnostic baseline, so even the label 'generic' is unsupported. The abstract's 'global physical metrics remain unchanged' is also not supported by the reported numbers: Power and Wirelength MAE rise from 0.06 to 0.10 (~67%) between Raw and Clustered (Figs. 5-6). All three issues point in the same direction: the headline trade-off may be an artifact of unequal training budgets.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces CTS-Bench, a dataset of 4,860 post-placement, pre-CTS physical design solutions across five open-source architectures, with paired raw gate-level graphs and clustered proxy graphs plus 15 post-CTS ground-truth metrics. Using GCG/GraphSAGE/GATv2 multi-task regression on skew, power, and wirelength, the authors report that clustering reduces peak VRAM by up to 17.2x and training time by 3x, but that clustered models frequently achieve negative R² for clock-skew prediction, especially on an unseen Zipdiv hold-out. The paper concludes that generic graph clustering can fundamentally compromise CTS learning objectives even when global physical metrics remain unchanged, and positions CTS-Bench as a testbed for task-aware coarsening and GNN acceleration.","tokens_in":10442,"tokens_out":3364,"duration_ms":32404,"significance":"If the empirical claims are established, CTS-Bench would be a useful community resource: it is one of few open ML-EDA benchmarks focused specifically on the placement-CTS interface, it provides paired multi-scale graph representations, and the containerized OpenLane pipeline supports reproducible extensions. The central headline result—that coarsening destroys local CTS fidelity while preserving global metrics—is precisely the kind of trade-off that the community needs quantified. However, the current experimental design does not support that headline: the Raw and Clustered arms differ in model capacity and learning rate, the 'global metrics unchanged' wording is contradicted by the paper's own MAE numbers, and only one custom, paper-specific clustering algorithm is tested. The dataset itself is potentially valuable, but the causal claim about 'generic graph clustering' is not yet demonstrated.","major_comments":[{"comment":"The Raw-vs-Clustered comparison is confounded by unequal model capacity. Table 3 sets Raw hidden dim 64 / LR 0.001 / placement-MLP 32 / CTS-MLP 16, while Clustered uses hidden dim 16 / LR 0.0005 / placement-MLP 8 / CTS-MLP 4. The Clustered arm is therefore weaker on every axis, so the observed R² collapse—including the headline GCN -2.23 on Zipdiv—cannot be attributed to graph coarsening without an equal-capacity control. Please retrain both representations with identical hidden dimension, learning rate, and MLP sizes, or run a matched capacity/lr sweep; also report mean/std over at least 5 seeds. This is load-bearing for the abstract's claim that 'generic graph clustering techniques can fundamentally compromise CTS learning objectives.'","section":"§3, Table 3"},{"comment":"The statement that global physical metrics 'remain unchanged' is not supported by the presented numbers. For GCN on seen data, Power MAE and Wirelength MAE are both 0.06 for Raw (Fig. 5) and 0.10 for Clustered (Fig. 6), a ~67% relative increase. Similar increases appear for the other backbones. Please either revise the wording to describe a modest degradation, with confidence intervals, or provide a statistical test showing the difference is not significant. Since the paper's conclusion is explicitly framed as 'even when global physical metrics remain unchanged,' this mismatch is central to the trade-off message.","section":"Abstract and §4.2, Figures 5–6"},{"comment":"The term 'generic graph clustering' overstates the evidence. The only coarsening evaluated is the custom three-step BFS/gravity-merge method with paper-specific thresholds (spread >0.05, Manhattan distance <0.05, cosine >0.9). There is no comparison to standard coarseners such as METIS, spectral clustering, or algebraic multigrid, nor to a simple randomized contraction control. Therefore the conclusion that 'generic graph clustering techniques' fail is not established; at most it applies to this particular clustering recipe. Please add at least one standard task-agnostic baseline and one randomized baseline with matched node counts.","section":"§2.1, §2.2, and §4"},{"comment":"The dataset accounting is unclear. The text says 'We generate 486 unique placements across all five designs' but also 'an additional 500 data points from a Zipdiv core for zero-shot generalization testing.' If 4,860 = 486 placements × 10 CTS variants, then the Zipdiv points cannot be 'additional' to 486 placements unless the arithmetic is different. Please give a per-architecture table of placements and data points, state explicitly which architectures are in the training split and which form Zipdiv, and confirm that no Zipdiv data is used in training. This is needed for the zero-shot claim to be reproducible.","section":"§2.1 and §3"}],"minor_comments":[{"comment":"The roadmap sentence says 'Section 1 reviews related benchmarking efforts' but the related-work discussion appears in the same section as the introduction; Section 2 is the methodology. Please correct the cross-reference.","section":"§1"},{"comment":"References [19] and [20] are the same paper (EDA-schema, GLSVLSI 2024) and are duplicated. Please merge or remove one.","section":"References"},{"comment":"The metric is called 'Mean Average Error' in Section 3; this should be 'Mean Absolute Error' (MAE).","section":"Abstract and §3"},{"comment":"The 'Pareto Gap' / 'Total Pareto Distance' is defined and listed as a contribution but is not used in any experiment or reported result. Either use it in evaluation or remove it to avoid over-claiming.","section":"§2.2"},{"comment":"The axis labels switch between 'Raw Nodes' / 'Clustered Nodes' and 'raw gate-level graphs/clustered proxy graphs.' Also, the compression ratio 13.3x is for node count, while the VRAM reduction is 17.2x; please clarify the relationship, since memory reduction may also reflect smaller hidden dimensions in the clustered models.","section":"Figure 3 and §2.1"},{"comment":"The table lists 10 randomization knobs but the text says 'randomize seven placement parameters.' Please reconcile the count, and note which knobs are applied at CTS stage.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The dataset and generation pipeline are likely to be a useful contribution to the ML-EDA community, and the direction of the benchmark is worthwhile. However, the headline empirical claim is not identifiable with the current experimental setup. The authors should be asked to run the full comparison with matched model capacity, add standard clustering baselines, and report seed variance. If the negative-R² result persists under those controls, the paper would be strong; if it disappears, the paper becomes a dataset-with-observations rather than a negative result about coarsening. The current version is not ready for acceptance without that load-bearing revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The dataset is a real addition to ML-EDA, but the headline claim about coarsening destroying CTS fidelity is currently confounded by unequal model capacity between the raw and clustered arms.\n\nWhat's actually new: CTS-Bench gives you 4,860 converged post-placement designs across five architectures, each with paired raw gate-level and clustered graphs, plus CTS ground truth (skew, power, wirelength) and an automated OpenLane-based pipeline. No prior benchmark isolates the placement-CTS interface like this. That alone is worth having.\n\nWhat it does well: the memory/runtime numbers (17.2x VRAM reduction, 3x training speedup) are concrete, the zero-shot holdout on Zipdiv is a sensible stress test, and the authors are honest that raw graphs also struggle on unseen designs. The Pareto Gap metric is a reasonable idea, though it's defined but never actually used as a label or target.\n\nSoft spots, in rough order of severity. First, the capacity confound: Table 3 gives the raw models hidden dim 64 and LR 0.001, while clustered models get hidden dim 16 and LR 0.0005, and smaller MLPs. A smaller network with a 2x lower learning rate can easily collapse to near-mean predictions regardless of graph structure. So the negative R2 values on clustered graphs cannot be attributed to coarsening until the arms are run at matched capacity (or with a proper ablation). That's a load-bearing gap. Second, the abstract says 'global physical metrics remain unchanged,' but the paper's own numbers show Power and Wirelength MAE go from 0.06 to 0.10—a 67% increase, not unchanged. That's an overstatement. Third, the clustering baseline is a custom BFS/gravity-merge method, which is called 'generic' without comparing to standard coarseners like METIS or other task-agnostic methods. So the scope of the conclusion is unsupported. Finally, there's no dataset URL or version, and no seeds or variance across runs are reported, which is a problem for a benchmark paper.\n\nWho should read it: anyone working on graph coarsening for physical design, or building GNN surrogates for CTS-style local metrics. The dataset is the contribution; the empirical trade-off claims need a re-run with matched capacity before being taken at face value.\n\nRecommendation: send it to peer review, yes—the resource deserves serious evaluation—but require the authors to fix the capacity confound, temper the 'unchanged global metrics' claim, add a standard coarsener baseline, and provide the dataset link. As it stands, the central negative-R2 result is plausible but not established.","headline":"Useful new dataset, but the central negative-R2 claim is confounded by unequal model capacity between the raw and clustered arms.","tokens_in":10848,"tokens_out":3689,"would_cite":true,"duration_ms":31294,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper shows that generic graph coarsening, while cutting memory by 17.2x and training time by 3x, destroys the structural information needed to predict clock skew, driving R^2 below zero on unseen designs.","keywords":["clock tree synthesis","graph coarsening","graph neural networks","electronic design automation","benchmark","clock skew prediction","zero-shot generalization","physical design"],"falsifier":"Retrain the clustered GNNs with the same hidden size and learning rate as the raw models and run the same held-out divider design. If R^2 returns to positive values or approaches the raw model's 0.0-0.2 range, the coarsening-harm claim is weakened. Alternatively, replace the benchmark's custom clustering baseline with a standard multilevel graph partitioner at the same compression ratio and compare skew R^2.","tokens_in":9942,"feed_emoji":"⏱️","tokens_out":4434,"duration_ms":38735,"temperature":0.7,"pith_summary":"The paper introduces CTS-Bench, a benchmark of 4,860 converged physical designs with paired raw gate-level and clustered graphs, to measure how graph coarsening affects GNN predictions of clock tree outcomes. Its central claim is that generic graph clustering can fundamentally compromise CTS learning objectives: clustering cuts peak GPU memory up to 17.2x and training time up to 3x, but on clock-skew prediction the clustered models frequently score negative R^2, especially zero-shot on an unseen design (e.g., -2.23 for a GCN). Global metrics like total power and wirelength remain predictable after coarsening, so the damage is specific to local, placement-sensitive metrics. If true, this means simple coarsening is not a safe scalability fix for CTS-critical learning tasks and that task-aware clustering is required.","feed_headline":"Coarsened graphs cut VRAM 17x but break clock-skew models","feed_subtitle":"Clustering keeps power and wirelength predictions accurate, but clock-skew R2 drops below zero on unseen designs.","key_machinery":"The benchmark's multi-scale representation pair: a Raw graph whose nodes are standard cells with one-hop flip-flop neighborhoods, and a Clustered graph built by a three-step physics-aware algorithm (atomic BFS clusters around flip-flops, high-spread filtering, and gravity-vector-aligned merging that respects control nets and physical proximity), achieving about 13.3x compression. The paired representations plus the normalized Pareto Gap scoring framework are what allow the paper to separate efficiency gains from fidelity loss.","core_discovery":"The paper's core discovery is an accuracy-efficiency trade-off with a sharp local-metric cliff. Using clock skew prediction as a representative CTS task, models trained on the benchmark's raw gate-level graphs reach roughly 0.90 R^2 on seen designs, while models trained on clustered graphs—despite preserving global aggregates—lose the fine-grained spatial resolution needed to tell high-skew from low-skew regions, frequently dropping below zero and to -2.23 on an unseen architecture. The authors interpret this as evidence that coarsening removes structural information essential to clock distribution even when global physical metrics such as power and wirelength remain unchanged.","pith_inferences":["If the clustered models were given the same hidden dimensions and learning rate as the raw models, part of the fidelity gap might close; the paper's Table 3 confounds coarsening with capacity.","The benchmark's Pareto Gap scores could be reused as placement-quality labels independent of any GNN, potentially enabling placement optimization before CTS.","A natural next experiment is task-aware coarsening that explicitly preserves flop-to-flop distances or skew-critical edges; the benchmark's paired graphs make this directly testable.","Because only one held-out architecture is used for zero-shot evaluation, the headline negative R^2 numbers should be read as evidence of fragility rather than a universal law; a multi-design holdout would sharpen the claim."],"forward_implications":["Coarsened graphs remain a viable proxy for global metrics (total power, wirelength) on seen designs, with MAE comparable to raw graphs, making them useful on memory-limited hardware.","For clock skew, clustered models are not merely slightly worse; their negative R^2 means predictions are worse than predicting the mean, so any CTS-critical use of coarsening must preserve local structural information.","The 17.2x VRAM reduction and 3x training speedup shift GNN training from memory-bound to compute-bound, suggesting clustered representations can support larger batches or larger designs on the same accelerator.","Zero-shot generalization is weak even on raw graphs (R^2 0.0-0.2 on the unseen design), so coarsening alone cannot explain all generalization failure; training data diversity matters too."],"fun_headline_variants":["Speed vs. signal: coarsening trades 17x memory for broken skew models","Cheaper GNN graphs miss clock skew, sink R² below zero","Coarsening saves 17x VRAM but scrubs clock-skew signal","Benchmark shows coarsening kills GNN clock-skew accuracy","17x memory savings wipe out GNN clock-skew predictions"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The central claim that coarsening itself destroys skew fidelity rests on a comparison where the clustered models use 16-unit hidden layers and 0.0005 learning rate while the raw models use 64 units and 0.001; if those are equated, the fidelity loss could shrink or disappear.","fun_headline_variants_meta":{"raw":{"variants":["Speed vs. signal: coarsening trades 17x memory for broken skew models","Cheaper GNN graphs miss clock skew, sink R² below zero","Coarsening saves 17x VRAM but scrubs clock-skew signal","Benchmark shows coarsening kills GNN clock-skew accuracy","17x memory savings wipe out GNN clock-skew predictions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000192,"raw_usage":{"total_tokens":1208,"prompt_tokens":794,"completion_tokens":414,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":538,"completion_tokens_details":{"reasoning_tokens":316}},"tokens_in":538,"tokens_out":414,"duration_ms":4617,"temperature":1.0,"reasoning_tokens":316,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T21:38:20.349169+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the clustered GNNs with the same hidden size and learning rate as the raw models and run the same held-out divider design. If R^2 returns to positive values or approaches the raw model's 0.0-0.2 range, the coarsening-harm claim is weakened. Alternatively, replace the benchmark's custom clustering baseline with a standard multilevel graph partitioner at the same compression ratio and compare skew R^2.","supporting_citations":[],"review_version":1}