{"id":"dc73a425-3a67-42dc-aee7-da0d075a9084","arxiv_id":"2411.16019","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"M3 trains one Mamba-based RL agent to size multiple analog circuits, mixing real and synthetic simulator data on a schedule, and reports roughly 10x better sample efficiency than SAC baselines.","lead":"M3 is a reinforcement learning system that uses the Mamba sequence model to optimize several analog circuits with a single neural network, mixing real and simulated data on a schedule. If the reported sample-efficiency gains survive replication, analog circuit designers could size transistors with far fewer expensive circuit simulations.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table III's mean-based success criteria can be satisfied with a small minority of successful episodes; per-seed success rates in Table IV are as low as 50%, so the 10x sample-efficiency claim overstates reliable multi-circuit optimization.","rationale":"I read the paper as claiming that a single Mamba-based MBRL agent can optimize multiple analog circuits with distinct topologies/specs in ~10,000 simulator calls, and that M3 achieves ~10x sample efficiency over SAC-Mamba. The most load-bearing assumption is not the padding representation (which is empirically validated by the reported learning curves), but rather that the evaluation criteria used to declare 'success' are meaningful. Both criteria in Table III are aggregate means that can be satisfied with a minority of successful episodes. The paper's own Table IV shows per-seed success rates as low as 50% at 15,000 steps, so the 'all circuits' success is an artifact of averaging. Without per-seed data or confidence intervals, the quantitative claim is underdetermined. This concern is concrete and testable, unlike the architectural concern about padding, which would require a different experiment (e.g., ablation with masking) but does not directly invalidate the reported numbers. I agree with the reader's CONDITIONAL verdict but for a different reason; the paper should be revised to report per-seed success rates, confidence intervals for Table III, and a stricter definition of successful optimization.","tokens_in":10584,"tokens_out":7901,"duration_ms":65698,"concrete_test":"Request the authors to release per-seed evaluation records for M3 at 10,500 steps and SAC-Mamba at 150,000 steps. For each circuit and seed, compute the fraction of evaluation episodes with FoM ≥ -0.02 (success). Then compute (a) the median success rate across seeds, and (b) the fraction of seeds that achieve ≥80% success on all four circuits. If the median success rate is below 0.5 for any circuit at 10,500 steps, or if fewer than 5/10 seeds succeed on all circuits, the statement 'successfully obtains positive mean rewards' should be qualified as 'some seeds' and the 10x claim limited to that weak criterion.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of 10x sample efficiency rests on Table III's criteria: 'Mean Episode Reward ≥ 0' and 'Mean Episode Length ≤ 25' across all circuits. Both are averages over seeds and episodes. The reward function (Eq. 1) assigns +10 when FoM ≥ -0.02 and FoM (≤ -0.02) otherwise, so a mean reward ≥ 0 can be achieved with only ~1-5% successful episodes if typical failures have FoM around -0.1 to -0.5. The length criterion is satisfied with ~20% successful episodes (success ends episodes early, failures hit Tep=30). Table IV shows per-seed success rates at 15,000 steps are 7/10, 5/10, 10/10, 10/10; one circuit has only 50% success. At the reported 10,500 steps, the per-seed rates are presumably lower. Thus 'successfully obtains positive mean rewards' is a weak operationalization of 'optimizing all circuits.' A policy that occasionally finds good designs can satisfy Table III while failing most runs. Since no raw data or code are provided, the reader cannot verify whether the 10x advantage persists under stricter criteria such as median success rate or all-seed success. This is a load-bearing threat to the headline claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes M3, a model-based RL method that combines the Mamba sequence architecture with an effective scheduling scheme (real-synthetic data ratio, rollout count, and update iteration number) to optimize multiple analog circuits with a single agent. The authors evaluate M3 on four circuits (2SOA, R2SOA, 2STIA, Comp) against SAC/PPO variants with Mamba, Transformer, LSTM, and fully connected architectures. They report that M3 achieves positive mean episode rewards for all circuits at 10,500 environment steps and claim roughly 10x sample efficiency over SAC with Mamba, with per-seed success rates at 15,000 steps shown in Table IV.","tokens_in":10952,"tokens_out":4701,"duration_ms":41656,"significance":"If substantiated, the claimed result—a single MBRL agent handling multiple analog circuits with distinct parameters and target specifications using only O(10^4) simulator calls—would be a meaningful step for analog circuit optimization. The paper's strengths include a clear pseudocode (Algorithm 1), a transparent hyperparameter table (Table I), explicit inference-time measurements for Mamba versus Transformer, and a comparison set spanning several RL backbones and architectures. However, the evidence as presented does not yet fully support the headline claims: the success criteria are mean-based and can be satisfied with low per-seed success rates, the 10x sample-efficiency figure is contradicted in part by Table IV, no significance tests are provided, and the scheduling contribution is not isolated by ablations. The paper also does not release code or raw data, limiting independent verification.","major_comments":[{"comment":"The success criteria 'Mean Episode Reward >= 0' and 'Mean Episode Length <= 25' are averages over episodes and seeds. Since Eq. (1) gives +10 when FoM >= -0.02 and FoM (which is <= -0.02) otherwise, a mean reward of zero can be achieved with only a small fraction of successful episodes (e.g., around 5-10% if typical failures have FoM between -0.1 and -0.5). Thus the claim that M3 'successfully obtains positive mean episode rewards for all circuits before 12,000 steps' does not establish reliable multi-circuit optimization. Please report per-seed success rates at the 10,500-step milestone and justify the threshold criteria against a stricter definition such as a minimum fraction of successful episodes per seed.","section":"Section IV-B, Table III and Eq. (1)"},{"comment":"The per-seed success rates in Table IV show M3 after 15,000 steps totals 32/40 (7/10, 5/10, 10/10, 10/10), identical to SAC with Mamba after 150,000 steps (5/10, 7/10, 10/10, 10/10); M3 is actually worse on R2SOA (5/10 versus 7/10). Therefore the sentence 'M3 outperforms SAC-Mamba by about 10x in sample efficiency' overstates the comparison, because the 10x figure comes only from the mean-based criteria in Table III. Please provide a matched comparison, e.g., environment steps required to achieve a fixed per-seed success rate, and include significance tests (bootstrap or Mann-Whitney across the 10 seeds) for the milestone times.","section":"Section IV-B, Table IV"},{"comment":"The paper argues that the effective scheduling contributes to the improvement, but the only comparison is against MBRL with fixed values (real-synthetic ratio 0.05, rollouts 10, update iterations 20). There is no ablation isolating the effects of alpha(t), R(t), and Ta(t) individually. Without such ablations, the claimed benefit of the scheduling mechanism over the Mamba architecture itself is not established. Please report experiments that vary one schedule component at a time, or otherwise quantify the contribution of each schedule.","section":"Section III-C and Q2"},{"comment":"The multi-circuit representation pads variable-length observations and actions into a single token sequence and uses Mamba's final context vector for the actor, critic, and model outputs. This assumes that the padded representation, without masking or explicit topology encoding, carries enough information for a single policy to distinguish circuits and learn appropriate actions for each. No analysis of padding positions, sequence order, or masking is provided. A failure of this representation would invalidate the multi-circuit claim even if the scheduling works. Please include ablations such as randomizing the padding order, using a mask to ignore padding, or adding a one-hot circuit identifier, or provide direct evidence that the context vector encodes circuit identity.","section":"Section III-B"}],"minor_comments":[{"comment":"The phrase 'real-synthetic dat ratio' contains a typo; it should read 'real-synthetic data ratio'.","section":"Section III-C"},{"comment":"The text says 'All other parameters are same with those listed in Table III' but the hyperparameters are listed in Table I, not Table III.","section":"Section IV-B"},{"comment":"The row 'RI, RI 1, 7' appears to have a typo; based on the text, it should be 'RI, RF 1, 7'.","section":"Table I"},{"comment":"The circuit list in the text says 'C2SOA' but the rest of the paper and Table II use 'R2SOA'. Please harmonize the notation.","section":"Section IV-A"},{"comment":"The caption says 'learning curves of episode lengths and success rates' but the plots show episode lengths and episode rewards; please correct.","section":"Figure 5 caption"},{"comment":"The caption says 'The first environment steps satisfying some criteria till 200,000 environment steps' but does not state whether the reported steps are an average over seeds, the median, or from a single representative seed. Clarify this.","section":"Table III"},{"comment":"The references [8] and [9] both relate to model-based RL scheduling; the sentence 'inspired by [8, 9]' is clear, but consider citing [8] and [9] separately in Section III-C where their ideas are used.","section":"Section II"}],"recommendation":"major_revision","confidential_remarks":"The paper comes from an industry lab and a university group with prior work in this area (CRoNuS, Insight). The self-citations are relevant and not excessive. The novelty is incremental—combining Mamba with an existing scheduling idea—but that is acceptable for a systems/application venue. The main concern is that the reported evaluation criteria are too weak to support the '10x' claim, and Table IV already hints that the advantage may be smaller or even negative on some circuits. The paper would benefit from a more rigorous statistical treatment and per-seed results at the claimed milestone. I would encourage the editor to send this for revision rather than reject, since the core idea is plausible and the missing analyses are within scope to add."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The new thing here is real: Mamba plus scheduled MBRL for multi-circuit analog sizing is a combination I haven't seen before. The paper formulates the problem cleanly, handles variable-length observations and actions via padding and a final context vector, and compares against a reasonable set of baselines (SAC variants, PPO, transformer, LSTM, MBRL without scheduling). Mamba is smaller and faster than the transformer baseline, and the scheduling idea—ramping the real-synthetic ratio, rollout count, and update iterations—is well motivated by prior work. If the sample-efficiency gain holds under stricter criteria, this is a practical contribution for analog EDA.\n\nThat said, I agree with the stress-test concern. The reward function gives +10 for success and a negative FoM otherwise, so a mean episode reward >= 0 can be achieved with only a small fraction of successful episodes. The episode-length criterion is similarly loose. Table IV shows that at 15,000 steps M3 succeeds on only 5/10 seeds for R2SOA and 7/10 for 2SOA; at the claimed 10,500 steps the rates are presumably lower. So \"successfully obtains positive mean rewards\" overstates reliable multi-circuit optimization. The 10x comparison in Table III is therefore not trustworthy as stated. Worse, no code, data, or netlists are shipped, so the reader can't verify. There are also no ablations isolating the schedule's contribution (e.g., fixed ratio vs. scheduled ratio with the same total compute) and no significance tests. The \"first online learning framework for multi-circuit optimization\" claim is a bit broad given prior multi-task RL work on circuits, though the specific Mamba+MBRL-scheduling combination does appear new.\n\nWho is this for? Analog circuit sizing researchers, especially those working on MBRL and shared agents. They'll get a useful data point, but they should treat the quantitative claims with caution until the evaluation is tightened.\n\nRecommendation: Worth sending to peer review. The core idea is legitimate and the experiments are extensive enough to deserve referee time. A good referee should ask for per-seed success-rate curves over time, a stricter success criterion (e.g., all-seed success or median), ablations of the scheduling hyperparameters, and ideally code/data release. The headline claim needs re-verification before it can be taken at face value.","headline":"Real new combination (Mamba + scheduled MBRL for multi-circuit analog sizing), but the 10x sample-efficiency claim overstates reliability because it rests on mean-based criteria that a minority of successful seeds can satisfy.","tokens_in":11405,"tokens_out":3048,"would_cite":false,"duration_ms":27648,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","68T05"],"pacs":[],"model":"deepseek-v4-flash","headline":"M3 claims that a single Mamba-based RL agent with scheduled model-based training can concurrently optimize multiple analog circuits of different topologies and specifications in roughly 10,000 simulator calls.","keywords":["Mamba","model-based reinforcement learning","analog circuit optimization","transistor sizing","multi-task reinforcement learning","sample efficiency","effective scheduling","state space model"],"falsifier":"Run the identical four-circuit benchmark with the same hyperparameters but require a stricter success criterion, e.g., that each of ten seeds reaches a positive episode reward for every circuit; or replicate with a different random ordering of the padded observation dimensions. If the reported 10,500-step positive mean reward is not reproduced within, say, 50,000 environment steps under either change, the central multi-circuit claim would be disproved.","tokens_in":10407,"feed_emoji":"⚡","tokens_out":7326,"duration_ms":60654,"temperature":0.7,"pith_summary":"M3 aims to show that a single reinforcement-learning agent can optimize several analog circuits at once—circuits with different topologies, parameter counts, and target specifications—rather than requiring a bespoke agent per circuit. The paper proposes combining the Mamba sequence architecture (used for the actor, critic, and a learned neural simulator) with a model-based RL loop whose three key training parameters are scheduled: the number of rollouts, the fraction of real versus synthetic data in each batch, and the number of agent update iterations per environment step. This scheduling starts synthetic-heavy for exploration and gradually shifts to real-data-heavy fine-tuning. If correct, the result matters because analog circuit sizing is dominated by expensive simulator calls, and the paper reports positive mean episode rewards for all four test circuits at 10,500 environment steps, roughly 10 times more sample-efficient than the best non-M3 baseline.","feed_headline":"One Mamba agent masters four analog circuits in ~10,500 steps","feed_subtitle":"M3's scheduled model-based training beats per-circuit RL agents by 10x on sample efficiency.","key_machinery":"The load-bearing component is the Mamba architecture used in all three networks—actor, critic, and environment model—where variable-length observations and actions from different circuits are padded to a single token sequence and read out through the final context vector. Around it, the effective scheduling mechanism adjusts three model-based RL hyperparameters as functions of environment step $t$: the rollout number $R(t)$, the real-synthetic data ratio $\\alpha(t)$, and the actor-critic update iterations $T_a(t)$, each interpolated linearly between an initial and final value and then clipped. The reward is the figure-of-merit $\\mathrm{FoM} = \\sum_i \\min\\{d(m_{c,i}, n_{c,i}), 0\\}$ with a sparse bonus of 10 when all normalized gaps are within tolerance. The scheduling is what converts a large volume of early, possibly inaccurate synthetic experience into exploration, then shifts to real data for exploitation.","core_discovery":"On the paper's own terms, the discovery is that Mamba-assisted model-based RL with effective scheduling—called M3—is the first framework to solve multi-circuit optimization in a single online learning run. With four benchmark circuits (two-stage op-amp, its complementary variant, transimpedance amplifier, and comparator) that differ in target-specification count, parameter type, and dimensionality, M3 obtains a positive mean episode reward for all circuits before 12,000 environment steps (precisely at 10,500 steps), a criterion no non-Mamba baseline met within 200,000 steps. Against the closest competitor, SAC with Mamba, M3 reaches the criterion about 10 times faster, and in the reported success-rate table M3 trained for 15,000 steps is comparable or better than SAC with Mamba trained for 150,000 steps on three of four circuits. The authors attribute the gain to two mechanisms working together: Mamba's linear-time state-space modeling, which they argue aligns naturally with the Markov structure of circuit simulation, and the scheduled increase of rollout count, real-data ratio, and update iterations.","pith_inferences":["The paper's headline claim is about mean episode reward averaged over seeds; the per-run success rates in its Table IV are 5/10 to 10/10 per circuit at 15,000 steps, so the honest reading is that the agent typically—not always—sizes each circuit within the reported budget.","The padded-input representation is never stress-tested: no ablation permutes the ordering of padded dimensions or masks unused entries, so a natural follow-up is to test order invariance and compare against a graph-based encoder that explicitly knows each circuit's topology.","The authors hypothesize, but do not demonstrate, that Mamba's state-space model matches the MDP structure of circuit simulation; a matched-parameter-count ablation against an LSTM or a gated transformer would separate the architecture's inductive bias from the scheduling's contribution.","Because the scheduling schedule is hand-set (initial/final values and a scale of 15,000 steps), an adaptive version that schedules from model uncertainty or real-data error could further cut the simulator budget; the paper does not explore this."],"forward_implications":["If the result holds, analog circuit designers no longer need to train a separate RL agent for each topology; one Mamba-based agent can handle a set of circuits with different parameter and specification dimensions.","The reported ~10,000 simulator calls to reach positive mean reward across all circuits makes online MBRL practical for circuit sizing, where each simulation is the main computational bottleneck.","Mamba's linear-time inference (about 5.1 ms versus 44.1 ms for the transformer) means that, at equal or better sample efficiency, the wall-clock cost per environment step is an order of magnitude lower, favoring sequence models over attention for this task.","The scheduling principle—start with synthetic-heavy batches and more rollouts only later, then fine-tune with real data—is stated generally enough to apply to other multi-task MBRL problems where simulator calls are expensive.","A single shared network also removes the memory and bookkeeping overhead of managing separate per-circuit policies, which is the practical motivation the paper gives for multi-circuit optimization."],"supporting_citations":[{"why":"Supplies the Mamba architecture used for the actor, critic, and environment model.","marker":"[5]"},{"why":"Provides the state-space-duality argument the paper uses to motivate Mamba over transformers for circuit RL.","marker":"[6]"},{"why":"Establishes the principle of scheduling model-based RL training parameters that M3's rollout and data-ratio schedules are based on.","marker":"[8]"},{"why":"Defines the standard MBRL recipe (fixed real-synthetic ratio, rollout count) that M3 compares against and adjusts.","marker":"[17]"},{"why":"Soft actor-critic is the base off-policy algorithm used for the actor and critic updates.","marker":"[16]"},{"why":"Provides the observation and reward formulation for the analog-circuit MDP that M3 follows.","marker":"[1]"},{"why":"Prior model-based RL approach for analog circuit sizing that demonstrates MBRL's sample-efficiency advantage in this domain.","marker":"[23]"},{"why":"Prior autoregressive-transformer neural simulator for analog circuits, the multi-circuit approach M3 claims to extend with Mamba.","marker":"[25]"}],"fun_headline_variants":["M3: one run, four circuits, 10x sample efficiency","Mamba-backed MBRL masters multiple analog circuits at once","First RL agent to optimize four circuits in a single run","M3 achieves 10x sample efficiency on multi-circuit optimization","Scheduled Mamba RL: 10,500 steps to master four circuits"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that, after padding the different circuits' observations and actions into one token sequence, the Mamba network's final context vector captures enough of each circuit's identity and target specification that a single policy trained this way transfers across all four circuits.","fun_headline_variants_meta":{"raw":{"variants":["M3: one run, four circuits, 10x sample efficiency","Mamba-backed MBRL masters multiple analog circuits at once","First RL agent to optimize four circuits in a single run","M3 achieves 10x sample efficiency on multi-circuit optimization","Scheduled Mamba RL: 10,500 steps to master four circuits"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000412,"raw_usage":{"total_tokens":2118,"prompt_tokens":915,"completion_tokens":1203,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":531,"completion_tokens_details":{"reasoning_tokens":1114}},"tokens_in":531,"tokens_out":1203,"duration_ms":10387,"temperature":1.0,"reasoning_tokens":1114,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:37:43.778362+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the identical four-circuit benchmark with the same hyperparameters but require a stricter success criterion, e.g., that each of ten seeds reaches a positive episode reward for every circuit; or replicate with a different random ordering of the padded observation dimensions. If the reported 10,500-step positive mean reward is not reproduced within, say, 50,000 environment steps under either change, the central multi-circuit claim would be disproved.","supporting_citations":[{"cited_title":"On effective scheduling of model-based reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Establishes the principle of scheduling model-based RL training parameters that M3's rollout and data-ratio schedules are based on."},{"cited_title":"When to trust your model: Model-based policy optimization,","cited_arxiv_id":null,"evidence_quote":"Defines the standard MBRL recipe (fixed real-synthetic ratio, rollout count) that M3 compares against and adjusts."},{"cited_title":"Soft actor-critic: Off-policy maximum entropy deep reinforce- ment learning with a stochastic actor,","cited_arxiv_id":null,"evidence_quote":"Soft actor-critic is the base off-policy algorithm used for the actor and critic updates."},{"cited_title":"Autockt: Deep reinforcement learning of analog circuit designs,","cited_arxiv_id":null,"evidence_quote":"Provides the observation and reward formulation for the analog-circuit MDP that M3 follows."},{"cited_title":"Cronus: Circuit rapid optimization with neural simulator,","cited_arxiv_id":null,"evidence_quote":"Prior model-based RL approach for analog circuit sizing that demonstrates MBRL's sample-efficiency advantage in this domain."}],"review_version":1}