{"id":"633fbb6a-6448-4519-8f7b-fac7adfe7ee1","arxiv_id":"2412.07167","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Using RL to refine existing macro placements, rather than generate them from scratch, improves wirelength, congestion, and timing on chip placement benchmarks.","lead":"This paper proposes MaskRegulate, a reinforcement learning policy that refines existing macro placements instead of placing them from scratch, reporting wirelength, congestion, and timing improvements. It also adds a regularity reward that pushes macros to chip edges, and tests the approach on standard chip benchmarks with a commercial EDA tool.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The regulator-vs-placer conclusion rests on an unisolated DREAMPlace initialization: the Vanilla-MaskRegulate ablation changes both the MDP and the starting layout.","rationale":"The reader's weakest assumption is exactly the load-bearing concern: the DREAMPlace initialization is never isolated from the regulator MDP. My reading of the paper confirms this. Section 4.2 states that MaskRegulate uses DREAMPlace for the initial macro placement, and Appendix B.1's Vanilla-MaskRegulate ablation preserves that initialization while claiming the only difference is the problem formulation. That claim is inaccurate: the initial layout, the action space, and the step structure all change together. The paper's internal ablations do provide real evidence that regularity helps and that the overall MaskRegulate pipeline outperforms several strong baselines, so the work is not without support. However, because the central conceptual contribution is the regulator MDP itself, and all comparisons supporting that contribution share the DREAMPlace warm start, the conclusion is conditional on an additional controlled experiment. This does not change the reader's CONDITIONAL verdict: the concern is substantial but addressable with the proposed initialization-controlled ablation and a non-RL refinement baseline.","tokens_in":21058,"tokens_out":3736,"duration_ms":43904,"concrete_test":"Train Vanilla-MaskRegulate (no regularity reward) from three different initial macro placements: (a) the DREAMPlace placement used in the paper, (b) a MaskPlace from-scratch placement, and (c) a random feasible macro placement. Compare global HPWL against MaskPlace trained from scratch with the same episode budget. If (b) and (c) still beat MaskPlace by a comparable margin, the regulator MDP is supported; if the advantage largely disappears without the DREAMPlace initialization, the original comparison is confounded. As an additional control, apply a simple non-RL greedy or local-search macro refinement from the same DREAMPlace initialization; if it closes most of the gap to MaskRegulate, the contribution is the refinement step rather than the RL policy specifically.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the RL regulator formulation, not a better starting point, drives the improvements over from-scratch RL placers. The evidence does not currently isolate this. In Section 4.2, MaskRegulate and its vanilla ablation start from a DREAMPlace macro placement, while MaskPlace starts from an empty canvas. Appendix B.1 (Table 6) describes Vanilla-MaskRegulate as differing from MaskPlace 'only in the problem formulation,' but the initial state is also entirely different: a complete DREAMPlace placement versus an empty canvas. The action space differs too: the regulator relocates existing macros, with the PositionMask modified so unadjusted macros can be overlapped (Section 3.1), whereas MaskPlace places macros sequentially. Thus the comparison conflates the MDP reformulation with a warm start and a different action set. The training-budget asymmetry (1000 vs 2000 episodes) does not resolve this, since a good initialization can reduce the episodes needed. The MaskPlace+RegularMask versus MaskRegulate ablation has the same confound, so it also cannot separate the regulator formulation from the DREAMPlace initialization. Without a controlled test that varies initialization independently of the MDP, the headline 'RL policy as macro regulator rather than macro placer' is not yet established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MaskRegulate, an RL-based macro placement method that acts as a \"regulator\" by refining an existing placement (obtained from DREAMPlace) rather than placing macros from scratch, and integrates a regularity metric into the state and reward. Experiments on ICCAD 2015 and ISPD 2005 compare against DREAMPlace, AutoDMP, WireMask-EA, and MaskPlace, reporting global HPWL, regularity, and PPA metrics from Cadence Innovus. The central claims are that the regulator formulation improves placement quality and PPA over from-scratch RL placers, that adding regularity further helps, and that the method generalizes better to unseen chips.","tokens_in":21314,"tokens_out":4237,"duration_ms":41052,"significance":"If the regulator formulation is validated, this is a worthwhile contribution: it reframes RL macro placement as a refinement step, which naturally provides denser rewards and allows RL to exploit the output of analytical placers. The paper provides code, reports 5-run means for proxy metrics, compares with several strong baselines, and evaluates PPA with a commercial tool. However, the key causal claim that the MDP reformulation (rather than the DREAMPlace initialization) drives the improvements is not yet established by the reported ablations, and the PPA results rest on single selected runs. These issues are load-bearing for the paper's central narrative.","major_comments":[{"comment":"The ablation 'Vanilla-MaskRegulate' is described as differing from MaskPlace 'only in the problem formulation,' but the manuscript itself indicates two additional differences. First, Section 4.2 states that MaskRegulate uses DREAMPlace to obtain an initial macro placement, while MaskPlace starts from an empty canvas; the Vanilla-MaskRegulate ablation also starts from that DREAMPlace placement. Second, Section 3.1 modifies the PositionMask so that grids occupied by unadjusted macros become available for placement, which is a different action space from MaskPlace's sequential placement. The training budget also differs (1000 vs 2000 episodes). Consequently, Table 6 cannot isolate the effect of the regulator MDP; the improvement could be largely due to the warm start. I request a controlled experiment that varies initialization independently of the MDP formulation (e.g., allowing MaskPlace to start from a DREAMPlace placement, or training the regulator from an empty canvas), or an equivalent analysis that separates these factors.","section":"Section 4.2 / Appendix B.1 (Table 6)"},{"comment":"The generalization comparison between MaskRegulate and MaskPlace has the same confound as the main ablation: MaskRegulate is initialized with DREAMPlace on the test chips, whereas MaskPlace begins from an empty canvas. The claim of superior generalization (Section 4.3) therefore conflates initialization with the regulator formulation. The controlled experiment suggested above is also needed here to support the generalization claim.","section":"Section 4.3 (Table 2)"},{"comment":"PPA metrics are measured on a single selected run per method (the run with the best global HPWL), with no variance or multiple independent runs reported. Since the headline PPA improvements (e.g., 17.08% routing wirelength and 37.89% TNS over MaskPlace) are central to the paper's impact, this selection makes the quantitative claims fragile, particularly given that proxy-metric variances in Table 1 are sometimes substantial. Please report PPA for multiple runs or otherwise demonstrate that the reported improvements are robust to run-to-run variability.","section":"Section 4.1 (Table 1)"}],"minor_comments":[{"comment":"The phrase 'which results in limits useful information' is ungrammatical; it should be 'limits useful information' or 'limited useful information.'","section":"Abstract"},{"comment":"In the sentence 'which is is currently one of the largest open-source benchmarks,' the word 'is' is duplicated.","section":"Section 1"},{"comment":"'evaluated byCadence Innovus' is missing a space; it should be 'evaluated by Cadence Innovus.'","section":"Table 1 caption"},{"comment":"The terms 'Regularity Mask' and 'RegularMask' are used interchangeably; standardize the terminology throughout.","section":"Section 3.1 / Figure 2 / Figure 3"},{"comment":"The hyperparameters 'Mask soft coefficient' and 'Grid soft coefficient' are not defined or explained; please clarify their roles.","section":"Appendix A.3 (Table 4)"},{"comment":"The sentence 'MaskRegulate consistently improves regularity on all four unseen chips and enhances global HPWL on three chips' is ambiguous because Table 11 shows different improvement patterns across the three source methods; specify exactly which comparison is being summarized.","section":"Appendix B.4"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant problem and has a plausible method, but the central claim that the regulator formulation (rather than the DREAMPlace warm start) drives the improvements is not yet supported. I recommend requiring the initialization-controlled ablation before acceptance. The PPA single-run reporting should also be addressed. These are fixable within the paper's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the regulator-vs-placer framing is a real idea, and MaskRegulate seems to work, but the evidence for the \"formulation\" part of the claim is weaker than the paper suggests.\n\nThe new MDP design — an RL policy that refines existing macro placements instead of placing from scratch — is genuinely novel in the RL-placement literature. The regularity reward is also a sensible addition that the EDA industry cares about and most RL placers ignore. On the empirical side, the paper does a lot right: eight ICCAD 2015 benchmarks, five-run means for proxy metrics, commercial-tool PPA evaluation, generalization tests across unseen chips, and fine-tuning of other methods' placements. The code is released. That is a solid package.\n\nThe soft spot is the one the stress-test note identifies, and it is real. The central claim is that the regulator formulation, not the starting point, produces the improvements. But the key ablations do not isolate that. Vanilla-MaskRegulate differs from MaskPlace in the problem formulation, the initial state (a complete DREAMPlace placement vs. an empty canvas), and the action space (unadjusted macros can be overlapped). So the ablation conflates the MDP change with a warm start and a different action set. The training-budget asymmetry does not fix this. The MaskPlace+RegularMask vs. MaskRegulate comparison has the same confound. To establish the headline, the authors need a controlled test that varies the initialization independently of the MDP — e.g., run MaskPlace from a DREAMPlace-initialized canvas, or run MaskRegulate from an empty canvas.\n\nTwo smaller issues. First, the PPA numbers — the most impressive in the paper — come from a single selected run per method, chosen by best HPWL. That is understandable given the cost of commercial tools, but it means the big percentage improvements in rWL, congestion, and timing have no error bars. The authors should at least report which chips are robust across seeds, or provide a second run for the main comparisons. Second, the closest existing refinement method, IncreMacro [26], is cited but never compared. Since the paper argues for a refinement-based formulation, that omission is conspicuous.\n\nOn balance, this is a useful paper for the RL-for-chip-design community. The idea is worth pursuing, the experiments are mostly careful, and the limitations section is honest. It deserves a serious referee, but the reviewer should push for the initialization-isolation experiment, PPA error bars, and an IncreMacro comparison before the central claim can be taken as established.","headline":"The regulator MDP is a genuinely new idea and the system appears to work, but the headline claim that the formulation itself (rather than the DREAMPlace warm start) drives the gains is not yet isolated.","tokens_in":21827,"tokens_out":1611,"would_cite":true,"duration_ms":20070,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that recasting RL for macro placement as a refinement regulator—adjusting an existing layout instead of placing from scratch—improves placement quality, PPA metrics, and generalization, and that adding a regularity…","keywords":["reinforcement learning","macro placement","placement refinement","regularity","chip design","half-perimeter wirelength","PPA","MaskRegulate"],"falsifier":"Run the regulator, including the vanilla variant, from a random or empty initial layout while keeping all other training details fixed, or let the from-scratch MaskPlace baseline start from the same DREAMPlace-initialized canvas; if the advantage largely disappears, the regulator formulation itself is not what drives the reported gains.","tokens_in":1607,"feed_emoji":"🔧","tokens_out":1736,"duration_ms":61711,"temperature":0.7,"pith_summary":"The paper tries to establish that reinforcement learning for macro placement works better when the policy acts as a regulator that refines an existing placement than as a placer that builds one from scratch. The reason offered is that a regulator sees a complete layout at every step, giving it rich state information and dense, accurate rewards, which from-scratch placers lack. The paper shows that its method, MaskRegulate, improves global half-perimeter wirelength on every tested ICCAD 2015 circuit and reports large PPA gains over the from-scratch RL placer MaskPlace, including 17.08% lower routed wirelength and 37.89% better total negative slack. It also reports better generalization to unseen chips and to the ISPD 2005 benchmark, and it shows that adding a regularity objective—encouraging macros near the chip edge—further improves results. If true, this reframes the role of RL in chip design from generating placements to improving any placement produced by another method.","feed_headline":"RL regulator refines chip macros, beats from-scratch placers","feed_subtitle":"Denser rewards and a regularity signal let this policy improve any existing layout and cut congestion and timing violations.","key_machinery":"The load-bearing mechanism is a reformulated Markov decision process: instead of placing macros one by one onto an empty canvas, the policy modifies one macro at a time on a complete existing layout, with all other macro positions visible in the state. The state is encoded as pixel-level masks—a PositionMask of feasible cells (including cells occupied by not-yet-adjusted macros), a WireMask of HPWL change normalized to $[-1,1]$, and a new RegularMask measuring change in edge proximity—fused by a convolutional encoder-decoder that outputs an action-probability matrix. The reward is a normalized blend $r = \\alpha r_{\\text{wire}} + (1-\\alpha) r_{\\text{reg}}$ with $\\alpha = 0.7$, trained with proximal policy optimization. This formulation gives the policy dense, accurate reward signals and lets it operate as a regulator that can be applied to any initial placement.","core_discovery":"MaskRegulate, an RL policy trained as a macro regulator, consistently outperforms the from-scratch RL placer MaskPlace with which it shares most of its architecture and training settings: on the eight ICCAD 2015 circuits, global HPWL improves on every chip, the average rank across proxy and PPA metrics is best, routed wirelength drops 17.08%, horizontal and vertical congestion overflow drop 73.08% and 38.81%, worst negative slack improves 18.35%, total negative slack improves 37.89%, and the number of violation points drops 46.17%. The same policy generalizes better to unseen chips than a from-scratch placer and can fine-tune placements produced by MaskPlace, AutoDMP, and WireMask-EA. Adding the regularity signal to the state and reward yields higher regularity than all baselines on all eight chips.","pith_inferences":["If the regulator formulation is genuinely the source of the gains, the same refinement framing could be applied after other weak initial placements, converting any analytical, black-box, or learning-based placer into a hybrid search-plus-RL pipeline.","The regularity measure used here, edge proximity $\\min\\{x, X_{\\max}-x\\} + \\min\\{y, Y_{\\max}-y\\}$, is only one possible operationalization; a testable extension is whether more structured regularity terms, such as alignment or symmetric placement constraints, yield further PPA gains without hurting wirelength.","The demonstrated ability to fine-tune placements from several distinct methods hints that a trained regulator could serve as a general placement-polisher in industry flows, but the paper does not yet test the policy on truly out-of-distribution netlist sizes or aspect ratios, so that claim remains open."],"forward_implications":["RL policies for macro placement can be trained as generic improvement operators that post-process any existing placement, not just as one-shot placers.","Because the regulator sees a complete layout at each step, it receives denser and more accurate reward signals, which should shorten training relative to from-scratch placement.","Adding a regularity reward pushes macros toward the chip periphery, reducing macro blockage and the congestion and timing problems that blockage causes.","A policy trained on a few chips transfers to unseen chips and to other benchmarks, suggesting regulator-style training is a path toward reusable chip-placement policies."],"supporting_citations":[{"why":"MaskPlace is the from-scratch RL placer baseline it must beat and the source of the shared policy architecture, state representation, and wirelength reward design.","marker":"[16]"},{"why":"DREAMPlace provides the initial macro placements that MaskRegulate refines and is also used to place standard cells for computing global HPWL.","marker":"[19]"},{"why":"Proximal policy optimization is the reinforcement learning algorithm used to train the regulator policy.","marker":"[27]"},{"why":"The ICCAD 2015 benchmark suite is the main test bed, the source of the eight chips on which proxy and PPA metrics are reported.","marker":"[14]"},{"why":"Incremacro grounds the regularity motivation by explaining how macro blockage in the chip center harms downstream placement and routing.","marker":"[26]"},{"why":"The ISPD 2005 benchmark suite is used for additional generalization experiments that test whether a pre-trained regulator transfers across benchmarks.","marker":"[25]"}],"fun_headline_variants":["RL regulator refines chip macros, beats from-scratch placers","RL regulator beats from-scratch placement on chip PPA","MaskRegulate fine-tunes macros, outperforms from-scratch RL","RL as macro regulator: better PPA than from-scratch placers","Regulator beats placer: RL fine-tunes chip macros effectively"],"cache_read_input_tokens":24064,"weakest_assumption_plain":"The comparison assumes the gains come from the regulator's MDP formulation and not simply from starting with a good DREAMPlace placement, because the vanilla-regulator ablation still uses DREAMPlace initialization and the from-scratch baselines start from an empty canvas, so the effect of initialization alone is never isolated.","fun_headline_variants_meta":{"raw":{"variants":["RL regulator refines chip macros, beats from-scratch placers","RL regulator beats from-scratch placement on chip PPA","MaskRegulate fine-tunes macros, outperforms from-scratch RL","RL as macro regulator: better PPA than from-scratch placers","Regulator beats placer: RL fine-tunes chip macros effectively"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000689,"raw_usage":{"total_tokens":3158,"prompt_tokens":1016,"completion_tokens":2142,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":632,"completion_tokens_details":{"reasoning_tokens":2047}},"tokens_in":632,"tokens_out":2142,"duration_ms":14477,"temperature":1.0,"reasoning_tokens":2047,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:04:20.298231+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the regulator, including the vanilla variant, from a random or empty initial layout while keeping all other training details fixed, or let the from-scratch MaskPlace baseline start from the same DREAMPlace-initialized canvas; if the advantage largely disappears, the regulator formulation itself is not what drives the reported gains.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MaskPlace is the from-scratch RL placer baseline it must beat and the source of the shared policy architecture, state representation, and wirelength reward design."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DREAMPlace provides the initial macro placements that MaskRegulate refines and is also used to place standard cells for computing global HPWL."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The ICCAD 2015 benchmark suite is the main test bed, the source of the eight chips on which proxy and PPA metrics are reported."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Incremacro grounds the regularity motivation by explaining how macro blockage in the chip center harms downstream placement and routing."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The ISPD 2005 benchmark suite is used for additional generalization experiments that test whether a pre-trained regulator transfers across benchmarks."}],"review_version":1}