{"id":"76a71277-a584-42c6-a6bf-3528faec8e9d","arxiv_id":"2502.03086","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A 120x120 quantum RBM on D-Wave Pegasus generated 1.68 million synthetic attack samples for CICIDS2017, yielding F1 scores on par with or slightly above SMOTE and random oversampling.","lead":"The authors embedded a 120x120 quantum restricted Boltzmann machine on D-Wave's Pegasus processor and used it to generate 1.68 million synthetic attack samples to rebalance the CICIDS2017 intrusion detection dataset. They report small F1-score gains over SMOTE and random oversampling, plus a custom embedding method that runs much faster than D-Wave's default tool.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported 4.2M-record balanced dataset implies synthetic samples were added before the train/test split, contaminating the test set; the Table IX performance claim is therefore not evaluable.","rationale":"The reader's weakest assumption was that Algorithm 1 is a valid minor embedding of the 120x120 RBM onto Pegasus. That is a legitimate concern, but it is not the most load-bearing issue. Even a perfectly valid embedding would not make the central performance claim trustworthy if the evaluation protocol is contaminated. The 4.2-million-record figure is a direct numerical contradiction of the stated protocol in Section IV.D and Figure 1, which say synthetic samples are combined with the original training data. The only consistent reading of the reported total is that balancing happened on the full dataset before the train/test split, or equivalently, that the QRBM was trained on all attack data including the 126,161 test attack instances. Either way, the test set is no longer independent, so Tables VII-IX cannot support the abstract's claim of significant improvement. I reject the current manuscript because the core experimental result is not interpretable as reported. If the authors can demonstrate that the split was performed before balancing and that the QRBM never saw test attack samples, the manuscript could be reconsidered after rerunning the evaluation and correcting the reported dataset size.","tokens_in":10389,"tokens_out":7787,"duration_ms":73069,"concrete_test":"Re-run the pipeline with an explicit 70/30 train/test split before any balancing. Train the QRBM only on the 294,377 training attack samples and add synthetic samples only to the training partition; verify that the balanced training set size is 2,946,032 (1,473,016 benign + 1,473,016 attack), not 4,208,618. Then evaluate all classifiers on the original 757,454-record test set untouched by synthetic data. If the QRBM Decision Tree/Random Forest F1 scores in Table IX are not reproduced, or no longer beat RandomOversampler, the reported 4.2M balanced dataset and its performance gains are artifacts of test contamination.","verdict_should_be":"REJECT","load_bearing_attack":"The paper's own arithmetic exposes a train/test leakage problem. Section IV.D reports 1,683,771 synthetic attack samples, 'creating a perfectly balanced dataset of 4,208,618 samples.' The only arithmetic that yields 4,208,618 is 2,524,847 (the total deduplicated dataset in Table IV) plus 1,683,771. If synthetic data had been combined only with the original training partition (1,767,393 records), the balanced training set would be 3,451,164 records, not 4,208,618. Thus the pipeline appears to balance the entire dataset before evaluation, or else to count test records in the balanced product. Consequently, the 126,161 attack samples in the test partition (Table IV) are part of the data used to train/generate QRBM samples, and the test set contains synthetic attack samples derived from those test examples. This makes the classifier metrics in Table IX untrustworthy: the QRBM is evaluated partly on data generated from the test set, artificially inflating precision, recall, and F1 and invalidating the comparison with SMOTE and RandomOversampler. The embedding concern raised by the reviewer is real, but even a perfect embedding cannot rescue the central claim if the evaluation is contaminated.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a custom minor-embedding algorithm to place a restricted Boltzmann machine with 120 visible and 120 hidden units on D-Wave's Pegasus architecture, trains it on binarized CICIDS2017 attack samples, and generates roughly 1.68 million synthetic attack records. The resulting balanced dataset is compared against SMOTE and RandomOversampler across seven classifiers, with precision, recall, and F1 reported in Tables VI-IX. The authors claim that this constitutes one of the largest QRBM implementations to date and that QRBM-based balancing significantly improves IDS classifier performance while completing balancing in sub-second time.","tokens_in":10684,"tokens_out":5062,"duration_ms":48913,"significance":"If the evaluation were sound, this would be a notable demonstration of scaling QRBMs to nontrivial sizes on current quantum hardware and applying them to a concrete imbalanced-data problem. The paper's strengths include a concrete, described embedding algorithm (Algorithm 1), the use of real QPU sampling, and a practical dataset. However, the central quantitative claims currently rest on a contaminated evaluation and on an unvalidated embedding, so the significance is conditional on substantial correction.","major_comments":[{"comment":"The arithmetic in Section IV.D indicates that the synthetic samples were added to the full dataset, not to the training partition. The paper reports 1,683,771 synthetic attack samples 'creating a perfectly balanced dataset of 4,208,618 samples.' The only way to obtain 4,208,618 from the numbers in Table IV is 2,524,847 (total) + 1,683,771 (synthetic attack). If the synthetic samples had been combined only with the training partition (1,767,393 records), the total would be 3,451,164, and the training attack count would be 294,377 + 1,683,771 = 1,978,148, which is not balanced against 1,473,016 benign training samples. Thus the pipeline appears to balance the entire dataset before evaluation, meaning the test partition (including its 126,161 attack samples) is used to generate QRBM samples and the test set contains synthetic attack samples derived from test examples. This contamination invalidates the classifier metrics in Table IX and the comparison with SMOTE and RandomOversampler.","section":"Section IV.D, Table IV"},{"comment":"The embedding algorithm is validated only by wall-clock time and by the number of chains longer than six qubits. There is no check that the implemented Ising parameters (h,J) match the trained RBM weights, no report of chain-breakage rates during sampling, and no comparison between hardware samples and samples from the trained classical RBM. Consequently, the claim that the deployed distribution is the trained 120x120 QRBM distribution is not established. A concrete test, such as comparing the empirical visible-unit marginals or the energy statistics of hardware samples against classical Gibbs samples, is needed.","section":"Section IV.B, Algorithm 1, Table III"},{"comment":"All reported metrics in Tables VI-IX appear to come from single runs with no error bars, no repeated seeds, and no statistical significance testing. Differences of 0.1-0.5 percentage points (e.g., Random Forest F1 of 95.96 for RandomOversampler versus 96.1 for QRBM; KNN F1 of 90.5 versus 90.91) are not established as meaningful. The paper's claim of 'significantly improving' detection rates therefore is not supported by the presented evidence.","section":"Tables VI-IX"}],"minor_comments":[{"comment":"The timing figures are inconsistent: Section IV.C reports a default annealing time of 20 µs and 124 µs readout, while the Discussion reports QPU sampling time of 134.92 µs and readout time of 94 µs; Section IV.D says each annealing process lasts only 20 µs. These numbers need reconciliation before the millisecond/sub-second timing claims can be assessed.","section":"Section IV.C, Section IV.D, Discussion"},{"comment":"The header 'QRMB' in Table V is a typo; it should read 'QRBM.'","section":"Table V"},{"comment":"The sentence 'We therefore contribute by demonstrate the practical feasibility...' has a grammatical error; it should be 'by demonstrating.'","section":"Section VII"},{"comment":"In Algorithm 2, lines 9-11 use the unclear denominator 'm1' and the notation V^T H/m versus V'^T H'/m1; either the denominators should be m throughout or the mini-batch size should be defined separately. As written, the update rule is ambiguous.","section":"Algorithm 2"},{"comment":"The Related Work section claims the algorithm enables minor embedding 'up to a 172x120 RBM,' but Table III reports results only for 60x60 and 120x120. The 172x120 claim is not supported by any experimental result in the paper.","section":"Related Work"}],"recommendation":"major_revision","confidential_remarks":"The evaluation contamination in Section IV.D is the decisive issue. If the authors can rerun the pipeline with a strict train/test split that prevents synthetic samples from entering the test set, the corrected numbers would determine whether the central claim survives. The embedding validation issue is also fixable but would require additional hardware or simulation experiments. These corrections are within the scope of a revision, so I do not recommend rejection at this stage."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the honest picture. The genuinely new thing in this paper is Algorithm 1: a deterministic minor embedding that puts a 120x120 RBM on the Pegasus topology in milliseconds, where minorminer takes minutes and produces long chains. As an engineering contribution, that's real, and the paper spells out the algorithm well enough to reproduce. If I took one thing from this work, it's that.\n\nThe problem is the evaluation that supports the headline claim that QRBM balancing beats SMOTE and RandomOversampler. The arithmetic in Section IV.D shows the pipeline leaked the test set. The paper says 1,683,771 synthetic attack samples create a perfectly balanced dataset of 4,208,618 samples. The only way to get that number is to add the synthetic samples to the full deduplicated dataset (2,524,847), not to the training partition (1,767,393). If synthetic data had gone only into training, the balanced training set would be 3,451,164, which is not what's reported. So either the balancing was done after the train/test split, or test records were included in the balanced product. Either way, the test partition in Table IV contains synthetic attack records derived from training on data that includes those test attack samples. That contaminates Table IX. The QRBM metrics are not an honest comparison.\n\nSecondary issues, in order. First, all metrics in Tables VI-IX come from single runs with no variance measures. Some QRBM advantages are under half a percentage point; that is noise. Second, there is no classical RBM baseline, so we cannot tell whether the quantum hardware is buying anything over a standard RBM trained with CD. Third, the embedding is validated only by wall-clock time and long-chain counts, not by checking that the programmed Ising coupling matrix reproduces the trained RBM weights or that the samples resemble the intended distribution. The appendix's Algorithm 2 also has a typo in the denominator, but that's minor.\n\nCredit where it's due: the embedding algorithm is a solid, reproducible piece of work, and the idea of using QRBMs for class balancing in IDS is worth exploring. But the central comparative claim does not hold as written. The paper deserves a serious referee, because the embedding contribution is nontrivial, but the quantitative evaluation needs to be redone with a clean train/test separation, proper baselines, and multiple runs before the results can be trusted.\n\nI'd bring this to a reading group as a cautionary example, and I'd cite the embedding algorithm, but I would not cite the performance table.","headline":"The 120x120 Pegasus embedding is a real engineering contribution, but the balancing evaluation is invalidated by test-set leakage in the reported arithmetic.","tokens_in":11199,"tokens_out":4054,"would_cite":true,"duration_ms":35871,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a 120-by-120 quantum restricted Boltzmann machine embedded on the Pegasus annealer generated 1.68 million synthetic attack samples and improved intrusion-detection classifiers beyond SMOTE and RandomOversampler.","keywords":["Quantum Computing","Quantum Machine Learning (QML)","Quantum Restricted Boltzmann Machines (QRBM)","Restricted Boltzmann Machines (RBM)","Dataset Balancing","Unbalanced Data","Generative Models","Intrusion Detection Systems"],"falsifier":"For the claimed 120x120 model, retrieve the programmed coupling matrix from the QPU after embedding and compare bit-for-bit with the J matrix that Algorithm 1 returns; any rerouted or missing edge means the hardware is not sampling from the trained QRBM. A second check is to train a small QRBM with known weights, anneal many samples, and compare visible-unit marginals with Gibbs-sampling marginals of the same model; mismatch beyond shot noise would refute the sampling claim.","tokens_in":10205,"feed_emoji":"⚛️","tokens_out":8531,"duration_ms":78415,"temperature":0.7,"pith_summary":"This paper argues that a quantum restricted Boltzmann machine (QRBM) can be made large enough to act as a practical generative model for fixing class imbalance in intrusion detection data. The authors embedded a QRBM with 120 visible and 120 hidden units on the Pegasus quantum annealer using a custom embedding algorithm, trained it on binary-encoded attack samples, and used repeated quantum annealing to synthesize over 1.6 million attack records. Adding these to the original training data produced a balanced set of more than 4.2 million records, and classifiers trained on that set improved precision, recall, and F1 scores relative to the same classifiers trained on data balanced with SMOTE or RandomOversampler. The balancing step itself took 0.33 seconds. If this holds, it would mean current quantum annealing hardware can contribute to a routine but consequential machine-learning preprocessing task, producing synthetic minority-class data faster than classical baselines while improving detection performance.","feed_headline":"Quantum annealer balances intrusion dataset in 0.33 seconds","feed_subtitle":"A 120x120 quantum Boltzmann machine beats SMOTE and oversampling on detection metrics.","key_machinery":"The load-bearing object is the paper's custom embedding algorithm (Algorithm 1), which maps the bipartite QRBM onto the Pegasus qubit graph by segmenting visible and hidden nodes according to periodicity parameters and assigning physical qubits in repeating vertical and horizontal bands, with ferromagnetic intra-layer couplings set to -1. It is what lets the authors fit a 120-visible, 120-hidden model on hardware where the vendor's default embedding tool takes hundreds of seconds and produces long chains; the paper reports no chains longer than six qubits and runtimes near 0.014 seconds. The second mechanism is a quantum-annealing training loop (Algorithm 2), adapted from the paper's reference [9], that replaces the Gibbs-sampling model expectation of contrastive divergence with samples drawn from the annealer.","core_discovery":"The paper's central claim is that a 120-by-120 restricted Boltzmann machine, embedded on Pegasus by the authors' own mapping routine, samples well enough to serve as a generative model for intrusion-detection data. The RBM energy is converted to an Ising Hamiltonian, the weights are trained with an adapted contrastive-divergence procedure in which the model expectation comes from quantum annealing, and new samples are generated by repeated annealing calls. After preprocessing CICIDS2017 into 48 features and 120-bit vectors, the QRBM produced 1,683,771 synthetic attack samples, yielding a balanced dataset of 4,208,618 records. On this balanced data, Random Forest, Decision Tree, and KNN reached F1 scores near 90-96 percent, marginally above the same classifiers trained on SMOTE- or RandomOversampler-balanced data, and the balancing time was 0.33 seconds.","pith_inferences":["A comparison the paper does not run: a classical RBM of the same 120x120 architecture trained by Gibbs sampling on the same binary features. If it matched the QRBM's F1 gains, the advantage would be the generative model rather than the quantum sampler; if not, the annealer is doing measurable work.","The embedding routine's parametric structure suggests it transfers to the newer Zephyr topology by changing periodicity parameters, which would give a direct scaling test: how large a bipartite RBM can be embedded as qubit degree grows.","The reported 0.33 seconds covers only the balancing step; a full cost comparison with GAN-type generators would need training time, preprocessing, and readout included.","Because annealing is stochastic, an obvious extension is to measure variance of downstream F1 across repeated independent annealing runs; if variance is small, QRBM balancing is safe to use in automated pipelines."],"forward_implications":["If the central claim holds, a quantum annealer can embed and sample from a 120-by-120 restricted Boltzmann machine in a real preprocessing workflow, not just in proof-of-concept demonstrations.","Dataset balancing for intrusion detection can be completed in well under a second, with downstream classifiers matching or exceeding those trained on SMOTE- or RandomOversampler-balanced data.","The custom embedding removes the default embedding tool as the bottleneck, making QRBMs up to 172x120 reachable on current hardware.","Binarizing network-flow features to a fixed-width bit vector and generating minority-class samples by annealing appears to preserve enough statistical structure to improve precision, recall, and F1 across several classifier families."],"supporting_citations":[{"why":"Supplies the quantum-annealing training procedure (Algorithm 2) that the paper adapts to estimate the model expectation with annealer samples.","marker":"[9]"},{"why":"Defines SMOTE, the main classical baseline that QRBM-generated data is compared against.","marker":"[7]"},{"why":"Introduces the CICIDS2017 dataset used for all experiments and class-imbalance statistics.","marker":"[21]"},{"why":"Provides contrastive divergence, the training objective whose model expectation the paper replaces with quantum-annealing samples.","marker":"[13]"},{"why":"Documents the Pegasus topology whose qubit connectivity the embedding algorithm exploits.","marker":"[16]"},{"why":"Describes the Pegasus connectivity graph and supports the embedding strategy's assumptions about available couplings.","marker":"[8]"},{"why":"Earlier balancing study on CICIDS2017 that motivates the need for better synthetic minority-class generation.","marker":"[1]"},{"why":"GAN-based data balancing baseline that the paper discusses as an alternative generative approach.","marker":"[18]"},{"why":"Older Chimera topology whose limited connectivity motivated moving to Pegasus for larger QRBMs.","marker":"[22]"},{"why":"Maps RBM energies to Ising Hamiltonians, the step that lets the QRBM be programmed onto the annealer.","marker":"[2]"}],"fun_headline_variants":["Quantum Boltzmann machine balances intrusion dataset in 0.33s","120x120 QRBM trains on D-Wave Pegasus, outperforms SMOTE","QRBM generates 1.6M attack samples to correct imbalance","Quantum generative model edges out classical oversampling on IDS"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the custom mapping places the 120-by-120 model onto the physical qubits faithfully, so that the couplings and chains actually encode the trained RBM's weights and the annealer samples from that model rather than from a distorted graph.","fun_headline_variants_meta":{"raw":{"variants":["Quantum Boltzmann machine balances intrusion dataset in 0.33s","120x120 QRBM trains on D-Wave Pegasus, outperforms SMOTE","QRBM generates 1.6M attack samples to correct imbalance","Quantum generative model edges out classical oversampling on IDS"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000475,"raw_usage":{"total_tokens":2336,"prompt_tokens":904,"completion_tokens":1432,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":520,"completion_tokens_details":{"reasoning_tokens":1356}},"tokens_in":520,"tokens_out":1432,"duration_ms":11445,"temperature":1.0,"reasoning_tokens":1356,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T05:57:41.465386+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For the claimed 120x120 model, retrieve the programmed coupling matrix from the QPU after embedding and compare bit-for-bit with the J matrix that Algorithm 1 returns; any rerouted or missing edge means the hardware is not sampling from the trained QRBM. A second check is to train a small QRBM with known weights, anneal many samples, and compare visible-unit marginals with Gibbs-sampling marginals of the same model; mismatch beyond shot noise would refute the sampling claim.","supporting_citations":[{"cited_title":"Quantum-assisted training of restricted boltzmann machines","cited_arxiv_id":null,"evidence_quote":"Supplies the quantum-annealing training procedure (Algorithm 2) that the paper adapts to estimate the model expectation with annealer samples."},{"cited_title":"Smote: synthetic minority over-sampling technique","cited_arxiv_id":null,"evidence_quote":"Defines SMOTE, the main classical baseline that QRBM-generated data is compared against."},{"cited_title":"To- ward generating a new intrusion detection dataset and intrusion traffic characterization","cited_arxiv_id":null,"evidence_quote":"Introduces the CICIDS2017 dataset used for all experiments and class-imbalance statistics."},{"cited_title":"Pegasus topology, 2020","cited_arxiv_id":null,"evidence_quote":"Documents the Pegasus topology whose qubit connectivity the embedding algorithm exploits."},{"cited_title":"To- ward constructing a balanced intrusion detection dataset based on cicids2017","cited_arxiv_id":null,"evidence_quote":"Earlier balancing study on CICIDS2017 that motivates the need for better synthetic minority-class generation."},{"cited_title":"Data-balancing algorithm based on generative adversarial network for robust network intrusion detection","cited_arxiv_id":null,"evidence_quote":"GAN-based data balancing baseline that the paper discusses as an alternative generative approach."},{"cited_title":"Quan- tum optimization of fully connected spin glasses","cited_arxiv_id":null,"evidence_quote":"Older Chimera topology whose limited connectivity motivated moving to Pegasus for larger QRBMs."}],"review_version":1}