{"id":"cc828382-fd84-4e69-b55b-801e7a376e55","arxiv_id":"2509.11198","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A reinforcement learning framework (RL-QAS) discovers compact variational quantum circuit architectures for Iris and binary MNIST classification, outperforming a simple strongly-entangling-layer baseline.","lead":"This paper trains a reinforcement learning agent to automatically design small quantum circuits for classification tasks, and tests it on Iris and a two-digit subset of MNIST. The practical interest is that it shows automated circuit search can find compact, accurate circuits, while also showing the approach struggles to scale beyond toy datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reward is computed on the same fixed test split used for final reporting, so the headline test accuracies are selected values, not unbiased estimates of generalization.","rationale":"The reader's CONDITIONAL verdict already captures the key risk; my analysis identifies one of the two fragile premises as the single most load-bearing concern. The strongest claim—that RL-QAS discovers low-complexity circuits with high test accuracy and that RL is a viable QAS strategy—rests on the test accuracies in Table 3. The most serious problem is that those accuracies are not unbiased generalization estimates because the reward is computed on the same test split used for final selection and reporting. This is an internal-validity threat: the agent explicitly maximizes test accuracy, and the best-of-many-selected value is reported. With roughly 9,000 unique PQCAs for Iris and only 45 test samples, the reported 100% could be a multiple-testing artifact. The paper's own statement that 'the reward is calculated based solely on test accuracy' confirms the design. This differs from the reader's other concern about small datasets, which is a generalizability limitation but does not threaten the correctness of the measurement. The proposed concrete test—using a validation split for reward and a final untouched test split—would settle whether the bias is material. If the held-out accuracy is comparable, the concern is resolved; if not, the central claim is weakened. Since the reader's CONDITIONAL verdict already requires addressing this issue, I recommend no change to the verdict. I partially agree with the reader's weakest_assumption because I prioritize the selection-bias mechanism over the small-dataset concern as the decisive flaw.","tokens_in":11017,"tokens_out":7427,"duration_ms":92739,"concrete_test":"Run a three-way split experiment: partition each dataset into training, validation, and test sets (e.g., 60/10/30 for Iris and MNIST 2). Use validation accuracy as the reward in the RL outer loop, with the same hyperparameters and search budget as in Table 1. After the search, select the PQCA with the highest validation accuracy and evaluate it once on the untouched test split. Compare this held-out test accuracy with the TeA values in Table 3. If the held-out accuracy drops by more than a few percentage points (e.g., >5 on Iris, >10 on MNIST), the reported numbers are inflated by selection on the test set. Repeat across 5-10 random splits to estimate the bias.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The empirical support for the central claim (RL-QAS discovers circuits with high test accuracy, making RL viable for QML architecture search) rests on Table 3's TeA values (e.g., 100% on Iris, 91% on MNIST 2). But these accuracies are not honest out-of-sample estimates: the reward in Eq. (3) is calculated from the same 30% test split (see Experimental Setup: 'the reward is calculated based solely on test accuracy'), and the best PQCA is then reported on that same split. The agent searches over many architectures (over 9,000 unique PQCAs for Iris) and maximizes this reward, so the reported TeA is the maximum of a large set of test-set evaluations—a textbook case of selection bias. There is no separate validation set, and Table 3 reports only the best TeA without a distribution over seeds or architectures. The gap between selected and true generalization accuracy grows with the number of evaluated circuits and shrinks with test-set size; with a 45-sample Iris test set, the 100% figure could be an artifact. This is an internal-validity flaw, not merely a limitation of dataset scope, and it directly undermines the 'high test accuracy' evidence for RL viability.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes RL-QAS, a two-loop reinforcement-learning framework for quantum architecture search (QAS) applied to classification. An outer-loop PPO agent incrementally constructs parameterized quantum circuit architectures (PQCAs), while an inner loop trains and evaluates each candidate on a quantum machine learning task. The reward is a weighted combination of test accuracy and circuit complexity (Eq. 3). The framework is evaluated on Iris (binary and 3-class) and binary MNIST (0/1), and compared against random and strongly-entangling-layer baselines. The authors report that RL-QAS finds low-complexity circuits with 100% test accuracy on Iris and 91% on MNIST 2, and conclude that RL is a viable approach for QAS in QML. The paper includes code, training metadata, and architectural analyses.","tokens_in":11242,"tokens_out":4622,"duration_ms":57809,"significance":"If the reported results were unbiased, the paper would make a modest but useful contribution: it is among the first to apply RL-based QAS to classification tasks, and it provides a reusable framework, caching mechanism, and detailed architectural pattern analysis. The decoupled outer/inner loop and the complexity-aware reward are sensible design choices. However, the central empirical claim is weakened by a selection-bias problem (test accuracy used both as reward and as final report) and by the very limited benchmark scope. The paper is therefore best viewed as a promising feasibility study whose main quantitative claims need re-validation before the viability conclusion can be accepted.","major_comments":[{"comment":"The reward in Eq. (3) is computed from test accuracy on the fixed 30% test split, and Table 3 reports TeA on that same split. With over 9,000 unique PQCAs evaluated for Iris (§Macro-Analysis), the reported 100% TeA is the maximum of a large set of evaluations on the test set, not an unbiased estimate of generalization. There is no separate validation set. This selection bias inflates the headline accuracies and directly undermines the central claim of 'high test accuracy'. Please introduce a held-out test set used only for final reporting, use a validation set for reward/selection, and report mean/standard error over seeds/architectures (or at least the full distribution).","section":"Experimental Setup / Eq. (3), Table 3"},{"comment":"The claim that RL is a viable approach for QAS in QML is supported almost entirely by two small datasets. The MNIST 2 agent did not converge (Table 2), and the best MNIST 2 TeA in Table 3 comes from a non-converged run. Please either add more diverse benchmarks with statistical significance tests, or substantially weaken the conclusion to a feasibility study. At minimum, report metrics over the 3 seeds rather than only the best circuit.","section":"Table 2 / Discussion"},{"comment":"The text repeatedly states that RL-QAS outperforms a random baseline, but no quantitative comparison is provided in Table 3 or the main text. Since the random baseline is one of the two main baselines used to support the viability claim, please include its accuracy/complexity numbers or clearly reference the supplementary figure where these data appear.","section":"Results (Performance of the RL-QAS Agent)"}],"minor_comments":[{"comment":"The second line of Eq. (4) uses 'r=' but should define P_delta; this is likely a typographical copy-paste error.","section":"Eq. (4)"},{"comment":"The table uses inconsistent decimal separators (e.g., 11,87 vs 11.84) and contains a stray '1' at the bottom; please format uniformly.","section":"Table 2"},{"comment":"'This thesis addresses this gap' should be 'This paper addresses this gap'.","section":"Related Work"},{"comment":"The statement that computing the reward from test accuracy 'promotes generalization' is misleading; using test data in the reward is a form of selection on the test set. Please rephrase and address the issue directly (see major comment 1).","section":"Experimental Setup"}],"recommendation":"major_revision","confidential_remarks":"The selection-bias issue is serious enough that I would not recommend acceptance without a re-analysis or explicit re-labeling. If computational constraints prevent re-running with a proper split, the authors should at least re-run the best architectures on a new holdout split and report those numbers, and soften the conclusion accordingly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper applies RL-based quantum architecture search to two small classification tasks (Iris and binary MNIST). That is a genuine, if modest, extension: prior RL-QAS work targeted QAOA, VQE, and state diagonalization, not QML classification. The framework is assembled from published components (tensor encoding, multidiscrete actions, illegal-action masking, two-loop evaluation), and the authors cite those sources fairly. The code is public, the implementation details are concrete, and the analysis of discovered circuits—gate frequencies, depth patterns, cost landscapes—is careful and useful. The authors also openly admit that the MNIST agent did not converge and that the evaluation is limited to noise-free simulations. That candor earns respect.\n\nThe soft spot is real, and the stress-test note lands precisely: the reward in Eq. (3) is computed from the same 30% test split used to report final test accuracy in Table 3. The agent searches over thousands of architectures and maximizes that reward, so the reported TeA values (100% on Iris, 91% on MNIST 2) are maxima over many test-set evaluations, not unbiased generalization estimates. With only 45 Iris test samples, 100% is easy to over-select. This is a selection-bias problem, and it is load-bearing for the headline \"high test accuracy\" claim. The fix is straightforward: use a validation split for the reward and a held-out test set for final reporting, or at least report selection-corrected numbers with variance over seeds.\n\nTwo smaller issues. First, \"viable\" is established only against a random agent and a single SEL baseline; the paper cites evolutionary, predictor-based, and differentiable QAS methods but does not compare against them. That weakens the general claim, though not fatally. Second, the abstract leans on the MNIST result even though training never converged; a revision should either make the training work or soften that part of the claim.\n\nNone of this is a takedown. For Iris, RL-QAS does discover circuits that beat a one-layer SEL with far fewer gates; the qualitative conclusion that RL can find compact, performant architectures for simple classification problems is plausible. The paper just needs to be honest about what the test numbers actually mean.\n\nI would send this to peer review, because the framework is sound enough and the flaw is fixable. A good referee would ask for the validation split, error bars, and a revised MNIST discussion. I would not cite the current numbers in my own work, but I would keep the paper in mind as a useful data point for RL-QAS in QML.","headline":"Competent, incremental RL-QAS for classification, but the reward-uses-test-split setup makes the headline accuracies selected values, not honest generalization estimates.","tokens_in":11867,"tokens_out":2568,"would_cite":false,"duration_ms":32611,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A reinforcement-learning agent can automatically discover compact, high-accuracy parameterized quantum circuit architectures for classification, beating standard hand-designed circuits on Iris and binary MNIST.","keywords":["quantum architecture search","reinforcement learning","variational quantum circuits","quantum machine learning","circuit design","classification","Iris dataset","MNIST"],"falsifier":"Hold out a fresh, untouched split of the Iris and binary MNIST data, rerun the search using only training accuracy for reward, and compare the final circuits' accuracy on that split against the standard strongly-entangling-layer baseline; if the RL circuits no longer beat the baseline, the reported advantage came from peeking at the test set.","tokens_in":10779,"feed_emoji":"⚛️","tokens_out":8595,"duration_ms":84333,"temperature":0.7,"pith_summary":"This paper tries to show that reinforcement learning can automate the design of variational quantum circuits for machine-learning tasks, a job normally done by hand. The agent builds circuits gate by gate, and each candidate is trained and scored on a classification task. On the full Iris dataset, the best discovered circuit—four gates including one CNOT, depth three—reaches 100% test accuracy; on binary MNIST, a circuit with six rotations and eight CNOTs reaches 91% test accuracy. The authors argue this makes RL a viable strategy for quantum architecture search, and that the resulting low-depth circuits are well suited to near-term quantum hardware. If correct, automated search could lower the expertise barrier for designing quantum machine-learning models.","feed_headline":"Reinforcement learning finds quantum circuits that hit 100% accuracy","feed_subtitle":"A search agent discovers low-depth circuits that beat standard designs on two classification tasks.","key_machinery":"The engine is a two-loop search-and-evaluate procedure. In the outer loop, a policy-gradient reinforcement learning agent represents a partially built circuit as a three-dimensional binary tensor (qubits by gate-and-control-pair choices by depth layers), picks a gate and target qubits, and updates the tensor. The inner loop trains that circuit's parameters with a classical optimizer on cross-entropy loss and returns its test accuracy. The reward balances a performance term built from test accuracy against a complexity term penalizing remaining depth and gate count, with an extended-horizon boost for later actions; an illegal-action mechanism blocks duplicate gates and depth violations. A has","core_discovery":"The central discovery is that a reinforcement-learning agent exploring a tensor representation of quantum circuits can return architectures that are simultaneously smaller and more accurate than a standard benchmark circuit. For the three-class Iris problem, the best found circuit has four gates (three rotations plus one CNOT) at depth three and reaches 100% test accuracy, while a one-layer strongly-entangling-layer baseline reaches 66% test accuracy. For binary MNIST (digits 0 vs 1, PCA-reduced to 32 features), the best found circuit has six rotation gates and eight CNOTs at depth seven and reaches 91% test accuracy, compared with 77% for the one-layer baseline and 93% for a two-layer basel","pith_inferences":["Because the same 30% test split is used both to compute the reward and to select the final circuit, the reported test accuracies are likely optimistic; a fair evaluation would hold out a third split for reporting.","The recurring 'CNOT early, rotations later' pattern could be extracted as a prior for human ansatz design or to seed other search methods, though the paper does not claim it as a general design rule.","A direct next test would apply the same search to a larger or noisy classification task and compare against random search with an equal evaluation budget; the viability claim would be much stronger if that comparison holds.","Replacing the full training of every candidate with a learned performance predictor could address the scaling bottleneck the paper identifies for more complex tasks."],"forward_implications":["Automated architecture search could replace manual ansatz design in quantum machine learning, reducing the human expertise and trial-and-error currently required.","The discovered circuits' small gate counts and shallow depths make them better suited to noisy near-term quantum hardware than deeper standard architectures.","Rewarding test accuracy during search can push the agent toward circuits that generalize, not just fit training data.","Caching circuit evaluations by their structural hash makes the search efficient enough to work despite the exponential size of the architecture space.","Scaling to harder tasks will require better search stability and evaluation, since the binary-MNIST run did not converge."],"fun_headline_variants":["RL agent finds quantum circuits that beat baselines","Quantum architecture search via RL yields compact circuits","Reinforcement learning discovers smaller, more accurate quantum circuits","RL-designed quantum circuits boost accuracy on Iris and MNIST","Automated quantum circuit search hits 100% on Iris"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The central claim rests on assuming that high test accuracy on two small, balanced, noise-free datasets—Iris and binary MNIST—with the agent selected on those same test splits, is representative of how the strategy would perform across quantum machine learning tasks.","fun_headline_variants_meta":{"raw":{"variants":["RL agent finds quantum circuits that beat baselines","Quantum architecture search via RL yields compact circuits","Reinforcement learning discovers smaller, more accurate quantum circuits","RL-designed quantum circuits boost accuracy on Iris and MNIST","Automated quantum circuit search hits 100% on Iris"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000152,"raw_usage":{"total_tokens":1027,"prompt_tokens":716,"completion_tokens":311,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":460,"completion_tokens_details":{"reasoning_tokens":235}},"tokens_in":460,"tokens_out":311,"duration_ms":4357,"temperature":1.0,"reasoning_tokens":235,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T16:54:04.772160+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Hold out a fresh, untouched split of the Iris and binary MNIST data, rerun the search using only training accuracy for reward, and compare the final circuits' accuracy on that split against the standard strongly-entangling-layer baseline; if the RL circuits no longer beat the baseline, the reported advantage came from peeking at the test set.","supporting_citations":[],"review_version":1}