{"id":"4b25d992-646e-4331-a8c5-03c53d0a6d2a","arxiv_id":"2607.14008","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"Restarting RL episodes from previously discovered high-performing circuit configurations improves sample efficiency and success rate in analog sizing benchmarks.","lead":"This paper introduces Lighthouse RL, a reinforcement-learning training strategy that restarts every episode from a set of the best circuit settings found so far ('lighthouses') instead of from scratch, to speed up analog circuit sizing. On two amplifiers it reports faster convergence and higher success rates than Bayesian optimization and prior RL baselines, but without released code or hyperparameters the gains cannot yet be independently reproduced.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Lighthouse selection rule lacks ablation; gains may stem from randomized resets rather than the strategic distance-based selection.","rationale":"The reader's weakest assumption identifies the same core issue: the lighthouse-selection rule's reliance on objective-space proximity to upper bounds without evidence of transfer to parameter-space usefulness. I agree this is the most load-bearing threat to the central claim. The paper's experiments are too narrow to establish transfer, and the missing random-reset baseline on the two main circuits means the specific choice of lighthouses is not isolated. The proposed concrete test directly ablates the selection rule, which would settle whether the mechanism is necessary. This does not change the CONDITIONAL verdict; it reinforces the conditions under which the claim would be accepted.","tokens_in":10782,"tokens_out":7938,"duration_ms":71643,"concrete_test":"Run the two-stage OpAmp experiment (Table 3) with three variants under the same budget and 5 seeds: (a) Lighthouse RL as described; (b) same algorithm but with lighthouses selected uniformly at random from P instead of top-k by d(o_p, o_max); (c) RL backbone with reset states drawn uniformly from the parameter bounds. Compare success rate, inference steps, and training sample efficiency. If (b) or (c) matches (a) within seed variance, the Eq. (9) selection rule is not the cause of the gains; if (a) significantly beats both, the strategic selection is validated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the 'lighthouse' reset strategy—specifically, selecting reset points by the deficit-distance ranking of Eq. (9)—is a general plug-and-play improvement for RL-based black-box optimization. This hinges on an untested transfer property: that configurations closest to o_max in objective space are the best starting points for reaching arbitrary targets sampled from U[o_min, o_max]. The paper provides no analysis of this transfer, and the only random-reset baseline appears in the 2D toy (Fig. 2b), not in the two circuit experiments of Table 3 where the backbone uses a fixed start. Consequently, the reported 1.55–1.72x sample-efficiency gains and high success rates could be produced by the mere randomization of initial states or by resetting to any previously discovered configuration, independent of the specific distance metric. The paper's own limitation statement (Sec. 6) admits that lighthouse diversity 'depends on training dynamics rather than explicit diversity enforcement,' underscoring that the selection rule is not well-controlled. If the selection rule is irrelevant, the method's contribution reduces to 'start from a good state,' which is not the strategic plug-and-play enhancement claimed.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Lighthouse RL, a reset strategy for episodic reinforcement-learning-based black-box optimization. The method maintains a set of high-performing ('lighthouse') states discovered during training and initializes subsequent episodes from these states, with a two-phase procedure: Phase 1 builds a priority queue of incomplete solutions and a set of successful solutions; Phase 2 resets episodes only from selected lighthouse states. Experiments on a 2D sphere benchmark and two analog circuits (two-stage and multistage op amps, simulated with an open PDK and SPICE) compare against Bayesian optimization, RoSeOpt variants, and an RL backbone. The authors claim up to 1.72x training-efficiency gains, higher within-distribution and extrapolation success rates, fewer inference steps, and better objective maximization, and state that the reset strategy is a general plug-and-play enhancement for any RL-based optimization approach.","tokens_in":11173,"tokens_out":6819,"duration_ms":70706,"significance":"If validated, the proposed reset mechanism is simple, potentially algorithm-agnostic, and relevant to expensive black-box optimization. The paper has practical strengths: it uses a realistic open-source PDK/SPICE setup, two nontrivial analog circuits, multiple seeds for the main experiments, and an explicit limitation statement in Section 6. The central idea is easy to implement and could be a useful addition to the RL-for-optimization toolbox. However, as it stands the evidence is insufficient: the key strategic-selection mechanism is not isolated by ablation, the headline improvements are not backed by statistical tests, some reported quantities have enormous variance, and an entire experiment reports only a single best seed.","major_comments":[{"comment":"The headline claims of 'significant improvements' are not supported by any significance test. Several key comparisons have overlapping or enormous standard deviations: multistage extrapolation reports 29.2±37.0% versus BO's 10.0±1.8%, and two-stage extrapolation reports 75.2±26.8% versus the RL backbone's 50.4±41.6%. With 5 seeds, paired tests or confidence intervals are needed; the multistage extrapolation mean should not be presented as reliable when the standard deviation exceeds the mean.","section":"§5.2–5.4, Table 3"},{"comment":"The central claim is that the specific deficit-distance ranking of Eq. (9) provides a strategic advantage. This is not tested. The only random-reset comparison appears in the 2D toy (Fig. 2b) and is qualitative; the circuit experiments compare only against a fixed-start backbone. No ablation compares lighthouse states selected by Eq. (9) against random resets, uniformly random initial states, or arbitrary previously visited configurations. The paper's own limitation note in Section 6 — that lighthouse diversity 'depends on training dynamics rather than explicit diversity enforcement' — weakens the mechanistic claim. Without such an ablation, the reported 1.55–1.72x gains could be due to resetting in general rather than to strategic selection.","section":"§4.1, Eq. (9); Fig. 2b; Table 3"},{"comment":"The objective-maximization experiment reports only the single best result over 3 seeds, with no variance, and uses only Phase 1 of Algorithm 1 rather than the full Lighthouse RL method. It compares only to an RL backbone and a human design point. The statement that 'Lighthouse RL outperforms all baselines in all objectives' is therefore not supported. Report all seeds (or mean±standard deviation) and include the same baselines used in the other experiments.","section":"§5.5, Table 4"},{"comment":"The method is not reproducible as written. The values of the reset-candidate count n, lighthouse count k, successful-set size m, target-resampling period N_update, episode length, reward constants R and r_min, objective weights α_i, normalization constants o_norm_i, and SAC hyperparameters are missing. Phase 2's termination condition 'if current time step > maximum then break' is also not defined. These parameters should be reported or the code released.","section":"Algorithm 1; §4.1–4.2"},{"comment":"The abstract and conclusion claim that the reset strategy can be integrated with any RL-based black-box optimization approach and is plug-and-play. The experiments only instantiate it with the authors' SAC/GAT backbone. No demonstration with a different RL algorithm (e.g., PPO or TD3) is provided, so the generality claim is not tested. Either add such an experiment or soften the claim.","section":"§1, §6, §5.2"}],"minor_comments":[{"comment":"The abbreviations T.S.E., C, and D are not defined in the caption; C and D are unclear, and T.S.E. is defined only later in the text.","section":"Table 3 caption"},{"comment":"The text says 'we select the first k solutions of P with minimal d(o_s, o_max)'; the subscript appears to be a typo, since P entries are indexed by o_p, not o_s.","section":"§4.1"},{"comment":"The modification of RoSeOpt's action discretization ('0.25×' and '0.5×') is not specified precisely; provide details or cite the original discretization.","section":"§5.2"},{"comment":"The figure reports 'total number of successful episodes over 12,000 training time steps' without error bars or a clear statement of how many seeds are shown; add variability or state it is a single seed.","section":"Fig. 2a"},{"comment":"The phrase '100% vs. 0–87% success rate' is ambiguous and potentially misleading because Table 3 shows the RL backbone also reaches 100% within distribution on the two-stage OpAmp. Specify the circuit and conditioning.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The core idea is plausible and the paper addresses a practically relevant problem with an appropriate experimental platform. However, the current evidence is under-powered and under-ablated: the strategic selection rule is not isolated, statistical tests are absent, and one key experiment reports only the best of three seeds. I would ask the authors to add the missing ablations and statistical reporting, disclose all hyperparameters, and temper the plug-and-play generality claim. This is fixable within the paper's scope, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should look at this one. The core idea—maintaining a set of near-feasible configurations and resetting episodes from them—is a genuine variation on Go-Explore, and it is distinct from the cited prior work: RoSeOpt uses BO to pick initial points, DNN-Opt resets on a per-step schedule, and HER reshapes sparse rewards. The two-phase algorithm is simple and the mechanism is clearly described. The 2D toy includes a random-reset baseline and shows the distance-ranked selection helps there. On the two-stage OpAmp, the within-distribution numbers are strong: 100% success, 3.8 inference steps, 1.55x training efficiency. The paper is also honest that lighthouse diversity is not explicitly enforced, which is good.\n\nThe soft spots are real but mostly addressable. No significance tests anywhere, and several headline numbers carry enormous error bars—75.2±26.8 extrapolation on the two-stage, 29.2±37.0 on the multistage, which is worse than a coin flip. The objective-maximization experiment reports the single best of three seeds, so we have no sense of variance. There is no code or data, which matters for a purely empirical paper. The abstract overstates things: \"100% vs 0-87%\" hides that the RL backbone also hits 100% within distribution, and the within-distribution multistage success of 87.2% comes with a standard deviation of 18 points.\n\nThe stress-test concern about the selection rule is legitimate. The paper's central claim is that the deficit-distance ranking of Eq. (9) picks good starting points. But the circuit experiments only compare Lighthouse RL against fixed-start and BO-initialized baselines—not against random resets or resets to any previously-discovered state. The 2D toy does include a random-reset control, but a two-sphere problem does not tell us much about a 29-dimensional amplifier. So we genuinely cannot tell whether the gains come from the strategic ranking or simply from starting episodes from non-trivial configurations. That is a testable ablation, and the authors' own limitation statement about uncontrolled diversity makes it more pressing. I do not see this as a fatal flaw—the mechanism is plausible, and the toy suggests ranking matters—but it is the single most important missing experiment.\n\nFor whom? Anyone working on RL-based circuit sizing or on reset strategies for episodic black-box optimization. The paper deserves a serious referee, and I would send it out. But the referee should insist on an ablation comparing distance-ranked resets against random and uniform resets on at least the multistage circuit, plus proper variance reporting and ideally released code. If those are added, this could be a solid contribution to a narrow but practically important subfield.","headline":"A plausible and clearly written reset-based exploration recipe for RL circuit sizing, with a novel angle but a missing control that should be added before the strategic-selection claim is trusted.","tokens_in":11606,"tokens_out":1890,"would_cite":false,"duration_ms":20727,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Starting RL episodes from high-performing 'lighthouse' configurations makes circuit sizing far faster and more reliable, and the trick works as a plug-in for any RL-based black-box optimizer.","keywords":["lighthouse reset","circuit sizing","reinforcement learning","black-box optimization","sample efficiency","generalization","strategic reset points","analog design"],"falsifier":"Construct a multi-modal benchmark with two widely separated basins, train Lighthouse RL with targets drawn only from one basin, then test with targets whose feasible region lies entirely in the other basin; if success rate stays near zero while a random-reset RL baseline succeeds, the lighthouse mechanism fails to transfer and the core assumption is false.","tokens_in":1463,"feed_emoji":"🎯","tokens_out":1649,"duration_ms":41073,"temperature":0.7,"pith_summary":"Lighthouse RL claims that a simple change to how reinforcement learning episodes begin can make black-box optimization, specifically analog circuit sizing, dramatically more sample-efficient and generalizable. Instead of always starting from a fixed point or a random reset, the agent begins episodes from a small set of high-performing parameter configurations discovered during training, called lighthouses. These starting points sit close to the objectives' upper bounds, so exploration is steered toward promising regions. On a 2D benchmark, an 11-parameter two-stage op amp, and a 29-parameter multistage amplifier, the method reports up to 1.72x faster training, 100% within-distribution success on the two-stage circuit, and 75.2% success on extrapolation targets, with far fewer inference steps than Bayesian optimization or RL backbones. If true, the reset strategy is a drop-in improvement for any RL algorithm tackling expensive black-box problems, where every saved simulation matters.","feed_headline":"Lighthouse resets make RL circuit sizing up to 1.72x faster","feed_subtitle":"Starting episodes from high-performing states reaches new design targets in a few steps, not dozens.","key_machinery":"The key object is the lighthouse state: a parameter configuration that comes close to meeting all minimum objectives but does not quite satisfy them, and is therefore near the boundary of the feasible region. The algorithm tracks these in a priority queue sorted by a deficit-only distance d(o_p, o_max) that measures, in normalized objective units, how far each candidate falls short of the upper performance bounds. This distance serves double duty: it ranks candidates for selection as lighthouses, and it is used to pick the n closest candidates to a newly sampled target as reset points during exploration. The two-phase procedure then alternates between discovering these states and exploiting","core_discovery":"The central discovery is that strategically resetting an RL episode from previously discovered near-optimal configurations, called lighthouses, guides exploration toward feasible regions and substantially improves both sample efficiency and generalization. The algorithm runs in two phases: first it explores, maintaining a priority queue of promising-but-incomplete solutions, ranked by deficit-only distance to the upper objective bounds, and a set of fully successful solutions. Once enough successes accumulate, it selects the top-k incomplete solutions as lighthouses and switches to an exploitation phase where every episode starts from a randomly chosen lighthouse. This reset mechanism can be","pith_inferences":["If the core transfer assumption holds (objective-space proximity to upper bounds implies parameter-space proximity to new targets), the mechanism generalizes beyond circuit sizing to any expensive black-box problem where partial successes can be reused, such as experimental design or materials discovery.","The deficit-only distance in Eq. (9) could be replaced by richer similarity measures (e.g., adding parameter-space distance or gradient proxies) to handle cases where feasible regions for different targets are disjoint; a testable variant would compare reset selection strategies on a multi-modal benchmark.","Since lighthouse diversity is not explicitly enforced and depends on training dynamics, introducing a diversity penalty or clustering when selecting the top-k lighthouses could improve robustness on multi-modal landscapes, an extension the paper itself flags as future work.","The transition from Phase 1 to Phase 2 is governed by hyperparameters m, n, k with no sensitivity analysis; ablating these would clarify how much of the gain comes from the reset mechanism versus the particular exploration budget."],"forward_implications":["Any RL-based black-box optimizer can incorporate Lighthouse RL's reset strategy as a plug-and-play improvement, since it only changes where episodes start and does not touch the underlying RL algorithm.","Circuit designers can reuse discovered lighthouse states as starting configurations for new target specs, converting training-time exploration into reusable design knowledge.","The reported gains scale with problem dimensionality (1.55x training efficiency for 11 parameters, 1.72x for 29), suggesting the method is most valuable for high-dimensional, simulation-heavy design tasks.","Extrapolation success on more demanding objectives than seen in training is markedly higher than baselines (75.2% vs. up to 50.4% on the two-stage op amp), implying the learned policy transfers beyond its training distribution.","The method also improves pure objective maximization: on the multistage amplifier it achieves gain of 118.1 dB, bandwidth of 24.5 MHz, phase margin of 89.7 degrees, and gain margin of 59.6 dB, exceeding the human-designed reference."],"fun_headline_variants":["Lighthouse resets slash RL circuit optimization steps","Strategic resets give RL circuit sizing 1.72x speedup","Start RL episodes at lighthouses for faster circuit design","Lighthouse RL: resets improve circuit sizing success","Reset points boost RL sample efficiency in circuits"],"cache_read_input_tokens":12928,"weakest_assumption_plain":"The method assumes that a parameter configuration close to the upper objective bounds in this deficit-only distance is also a good starting point for reaching a newly sampled target's feasible region; if feasible regions for different targets are disjoint in parameter space, the resets can mislead the agent.","fun_headline_variants_meta":{"raw":{"variants":["Lighthouse resets slash RL circuit optimization steps","Strategic resets give RL circuit sizing 1.72x speedup","Start RL episodes at lighthouses for faster circuit design","Lighthouse RL: resets improve circuit sizing success","Reset points boost RL sample efficiency in circuits"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000625,"raw_usage":{"total_tokens":2699,"prompt_tokens":680,"completion_tokens":2019,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":424,"completion_tokens_details":{"reasoning_tokens":1941}},"tokens_in":424,"tokens_out":2019,"duration_ms":13815,"temperature":1.0,"reasoning_tokens":1941,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T03:01:52.418788+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a multi-modal benchmark with two widely separated basins, train Lighthouse RL with targets drawn only from one basin, then test with targets whose feasible region lies entirely in the other basin; if success rate stays near zero while a random-reset RL baseline succeeds, the lighthouse mechanism fails to transfer and the core assumption is false.","supporting_citations":[],"review_version":1}