{"id":"e94b30cd-90f8-42e9-9250-b8b64fa4c72c","arxiv_id":"2411.09336","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A matrix product state simulator framework runs quantum kernel SVMs at 165 features and 6400 samples, finding a CPU-to-GPU crossover and an empirical accuracy scaling with data and features.","lead":"This paper builds a matrix product state simulator framework for quantum kernel support vector machines and demonstrates it on a 165-feature, 6400-sample Bitcoin transaction classification task. The authors report a CPU/GPU runtime crossover and claim the first evidence that quantum kernel accuracy improves with feature count and training set size.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Test-set regularization selection (Artifacts A5/A6) undermines the central 'improves with scale' and quantum-vs-Gaussian claims; a validation-split reanalysis is required.","rationale":"The simulation/engineering contribution is solid and well-supported: the MPS framework, the CPU/GPU crossover analysis (Fig. 5, Table I), the parallelization scaling (Fig. 8), and the truncation-error bound (Eq. 8) are technically credible and backed by public artifacts, so I do not urge rejection. However, the paper's central performance claim—the one highlighted in the abstract and in contribution C2.1—is not adequately supported by the reported experiments. The weakest point is exactly what the reader identified: the SVM regularization coefficient is selected using test-set AUC, per the artifact descriptions in Appendices G and H. This is not a merely cosmetic issue; selecting C to maximize test AUC inflates the reported performance and, crucially, can create a spurious monotone improvement with feature count because the selection freedom grows with the number of configurations. The claim 'first evidence of quantum model performance at scale' depends on this curve being genuine. The absence of classical baselines in Fig. 10 further weakens the claim that the improvement is quantum-specific: many classical models would also improve with more features and more data, so the trend is not by itself evidence of quantum advantage or even quantum-relevant scaling. I agree fully with the reader's weakest_assumption: the regularization selection is the load-bearing vulnerability. The concrete test I propose—a clean holdout validation protocol—is straightforward to run because the artifacts are provided and the only change is in the evaluation script, not in the MPS simulation machinery. If the authors re-analyze and the qualitative conclusions hold with proper validation, the paper can be accepted with its claims; if not, the headline should be tempered to a simulation-scale contribution. Either way, the engineering results (C1.1-C1.3) remain valuable and reproducible, which is why the appropriate verdict is CONDITIONAL rather than REJECT.","tokens_in":23945,"tokens_out":1908,"duration_ms":16846,"concrete_test":"Re-run the A5/A6 pipelines with a modified protocol: split the current test set into validation (for C selection, C in [0.01, 4]) and final test sets, then report AUC on the untouched final test for each kernel and feature-count configuration. If the monotone improvement in Fig. 10 and the quantum-over-Gaussian margins in Table II persist under this protocol with non-overlapping error bars, the central claim survives; if the gains shrink or reverse, the headline claim should be downgraded to a claim about simulation scale only.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's headline claim—'quantum kernel model performance improves as the feature dimension and training data increases' and the quantum-over-Gaussian result in Table II—rests on selecting, per experiment, the SVM regularization coefficient C that maximizes test-set AUC. Artifact A5 states plot.py 'picks the best AUC score among the different possible choices of the regularization coefficient,' and Artifact A6 reports Table II by 'pick[ing] the regularization coefficient with highest AUC.' This is test-set (or jointly train/test) selection; the same data used to report performance is used to choose C. Because C is fit to the test set, the reported AUCs are optimistically biased and the monotone improvement in Fig. 10 (0.933 to 2.44% gain from 100 to 165 features at 6400 samples) could be an artifact of selecting the best of many C values—the more features, the more freedom to overfit the test set. The same issue affects Table II: the quantum kernel may appear to outperform the Gaussian kernel simply because C has been selected on the test set for both, and the claim of 'first evidence of quantum model performance at scale' (C2.1) is not separable from this selection artifact. Error bars are absent, so the 0.877 vs 0.892 Gaussian gap and the 2.44% improvement lack uncertainty quantification. A proper evaluation would hold out a validation set for C selection, or use nested cross-validation, and report standard deviations across repeated splits.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Matrix Product State (MPS) simulation framework for constructing quantum kernels, with an ITensors-based CPU backend and a new pytket-cutensornet GPU backend, together with two parallelization strategies for Gram-matrix construction. The authors benchmark runtime and memory scaling as a function of interaction distance, qubit count, and dataset size, and report a CPU/GPU crossover near interaction distance d = 10 with bond dimension about 320. They then apply the simulator to the Elliptic Bitcoin dataset with up to 165 features and 6400 training points, reporting that quantum kernel AUC improves with feature count and training set size and that a quantum circuit ansatz with suitable hyperparameters outperforms a Gaussian kernel on test metrics. The manuscript includes a detailed artifact appendix linking each figure and table to reproducible scripts and archived data.","tokens_in":24252,"tokens_out":3183,"duration_ms":34222,"significance":"If the performance claims survive a proper evaluation protocol, the paper would make a meaningful contribution: it demonstrates a practical MPS-based route to quantum kernel experiments at a scale (165 qubits/features, 6400 training points) well beyond what state-vector simulation permits, and it ships unusually complete computational artifacts, including machine-readable raw outputs and scripts for every figure and table. The runtime scaling study, the crossover analysis, and the parallelization strategy are valuable and appear well supported by the supplied artifacts. The classification-performance claims, however, are currently the weakest link: they rest on test-set-based selection of the SVM regularization coefficient, and the reported metric differences are presented without uncertainty estimates, so the headline \"performance improves with scale\" and the quantum-versus-Gaussian comparison are not yet established at the standard required for a claim of \"first evidence of quantum model performance at scale.\"","major_comments":[{"comment":"The central performance claims in Figures 9-10 and Table II are undermined by test-set-based selection of the SVM regularization coefficient C. Artifact A5 states that plot.py \"picks the best AUC score among the different possible choices of the regularization coefficient,\" and Artifact A6 states that to_csv.py \"picks the regularization coefficient with highest AUC\" after averaging metrics over six samples. Because the same test data used to report performance is also used to choose C, the reported AUCs are optimistically biased, and the monotone improvement seen in Figure 10 (e.g., the 2.44% gain from 100 to 165 features at 6400 samples) could be inflated by selecting the best of many C values per experiment. The same issue affects the quantum-versus-Gaussian comparison in Table II, since the Gaussian kernel's regularization coefficient is selected in the same way. I request a reanalysis using a held-out validation set for C selection, or nested cross-validation, with the selected C then applied to the reported test set. Without this, the claims C2.1 and C2.2 are not supported.","section":"Appendix G (Artifact A5) and Appendix H (Artifact A6)"},{"comment":"No uncertainty quantification is provided for the averaged metrics. The text says \"Each run consists of 6 data samples and the metrics are averaged over the 6 runs,\" but Table II reports single numbers without standard deviations or confidence intervals. The reported differences are small: for example, the Gaussian AUC is 0.892 and the best quantum AUC is 0.904, and Figure 10 reports a 2.44% improvement. Without error bars or repeated data-split estimates, it is impossible to assess whether these gaps are meaningful or within sampling noise. I ask the authors to report means and standard deviations across multiple random splits and, ideally, a paired comparison between quantum and Gaussian kernels on the same splits.","section":"Section III-B and Table II"},{"comment":"The claim \"first evidence of quantum model performance at scale\" overreaches the experimental design. The experiments show that one specific MPS-simulated quantum kernel combined with an SVM improves on one financial dataset under hyperparameters selected on the test set. Even after the validation-set reanalysis requested above, this would be evidence about this quantum kernel implementation on this dataset, not a general statement about \"quantum model performance.\" The manuscript should temper the abstract and conclusion accordingly, for instance by saying \"these results provide initial evidence that this MPS-simulated quantum kernel can scale to 165 features and 6400 points\" and by acknowledging that the comparison is to a single Gaussian kernel baseline with a fixed bandwidth rule.","section":"Abstract and Conclusion; Appendix A (C2.1)"},{"comment":"The feature map itself is not prespecified: the text states that \"the Pauli operators have been chosen empirically based on model performance.\" This is another, earlier level of model selection on the same data, and it compounds the test-set C-selection problem. For the performance claims to be clean, the authors should either specify the full model-selection protocol (including how the Pauli operators and feature map hyperparameters were chosen) or present the results as an exploratory study that requires confirmation on held-out data.","section":"Section II-A"}],"minor_comments":[{"comment":"The affiliation line contains a typo: \"V entures\" should read \"Ventures.\"","section":"Title page"},{"comment":"\"state of art\" should be \"state of the art\" (or \"state-of-the-art\").","section":"Section I"},{"comment":"\"embarrasingly parallel\" should be \"embarrassingly parallel.\"","section":"Section III-A"},{"comment":"The caption reads \"sample sizes. on the Elliptic Bitcoin data set\" with an extra period; it should read \"sample sizes on the Elliptic Bitcoin data set.\"","section":"Figure 10 caption"},{"comment":"The phrase \"down selected and seeded to a specified dimension\" would be clearer as \"down-selected and seeded to a specified size,\" and the hyphen should be used consistently.","section":"Section III-B"},{"comment":"The table caption says \"Average of the largest bond dimension\" but the columns are labeled χ; please clarify in the caption or table that the reported values are the average over data points of the maximum virtual bond dimension after simulation.","section":"Table I"},{"comment":"The artifact descriptions are very thorough, which is commendable; however, the descriptions of the regularization selection should be highlighted in the main text, because they determine the interpretation of the headline performance claims.","section":"Appendix G/H"}],"recommendation":"major_revision","confidential_remarks":"The paper is a strong systems/engineering contribution with unusually complete artifacts, and the scaling results for the simulator are likely to be reproducible. The main performance claims, however, are currently supported only by a test-set-selected regularization protocol. I believe a major revision is appropriate rather than rejection, because the central claims are defensible if the authors add a validation-based reanalysis and uncertainty estimates. I would also encourage the editor to weigh whether the phrase \"first evidence of quantum model performance at scale\" should be softened even after the reanalysis."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid engineering paper wrapped in an overclaimed performance narrative. The MPS-based quantum kernel framework is real, reproducible, and genuinely scales further than prior work (165 features, 6400 training points). The resource scaling results—CPU/GPU crossover, round-robin parallelization, memory trajectories—are well supported by the artifacts. But the headline claim, 'first evidence of quantum model performance at scale,' does not survive contact with the appendices.\n\nWhat's actually new and good: they ship code and data for an MPS simulator pipeline with two backends (ITensors on CPU, pytket-cutensornet on GPU), a clean statement of the O(mχ^3) inner-product cost, and a careful treatment of SVD truncation error. The crossover at d≈10, χ≈320 is documented with medians and quartiles. The parallelization strategy is sensible, and the runtime model in Figure 8 is convincing. The comparison with Ref [20] is specific and fair. These contributions will be useful to anyone benchmarking quantum kernels on classical data.\n\nThe soft spots are concentrated in Section III-B and the claims built on it. Artifact A5 states plot.py 'picks the best AUC score among the different possible choices of the regularization coefficient,' and A6 does the same after averaging. That is test-set selection: the same data that reports performance is used to choose C. It biases the monotone improvement in Figure 10 and the quantum-over-Gaussian comparison in Table II, where the gaps are small (0.902 vs 0.892) and there are no error bars. The paper also omits the classical baseline from Figures 9 and 10, so 'improves with features/data' is not shown to be quantum-specific. The feature-map Pauli operators were 'chosen empirically based on model performance,' a lesser but related worry. The conclusion that increasing expressivity (d, depth) doesn't help is less affected—those trends are consistent with kernel concentration—but the quantitative claims need rework.\n\nBottom line: the simulation framework deserves publication and a serious referee. The performance claims need a validation split (or nested CV) with repeated splits and error bars before they can be cited. As written, I'd treat the scaling/engineering part as solid and the 'first evidence' part as unsupported.","headline":"Solid engineering and reproducible scaling, but the headline performance claims rest on test-set-selected regularization and need a validation-split redo.","tokens_in":24765,"tokens_out":2961,"would_cite":true,"duration_ms":27830,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Using a matrix-product-state simulator, this paper runs quantum kernel classification on 165 features and 6,400 training points and reports that test performance improves with both feature count and data size.","keywords":["quantum machine learning","quantum kernel methods","matrix product state simulation","support vector machine","tensor networks","GPU acceleration","Elliptic Bitcoin dataset","kernel bandwidth"],"falsifier":"Run the scale experiment with the SVM regularization coefficient selected on a held-out validation set rather than on the test set; if test AUC then fails to rise with feature count or with sample size across 300, 1500, and 6400 training points, the paper's central scaling claim would be refuted.","tokens_in":23738,"feed_emoji":"⚛️","tokens_out":7026,"duration_ms":66141,"temperature":0.7,"pith_summary":"Quantum kernel methods have been hard to benchmark because simulating quantum feature maps scales exponentially with feature count. The paper builds a Matrix Product State (MPS) simulator that represents the quantum state compactly, with controlled truncation, and exploits the fact that each Gram-matrix entry is an independent inner product that can be spread across processors. Using this framework it trains a quantum-kernel support vector machine on the Elliptic Bitcoin data set with 165 features and 6,400 training points, far beyond prior experiments. The central empirical claim is that the model's test AUC continues to improve as both feature dimension and training data grow, with a 2.44% AUC gain from 100 to 165 features at the largest sample size, and that a tuned quantum kernel outperforms a Gaussian kernel on this data. If true, this is the first evidence that quantum kernel models behave well at a scale meaningful for industry data sets.","feed_headline":"Quantum kernel models improve with scale in 165-feature simulation","feed_subtitle":"Matrix-product-state simulator pushes quantum SVM to 6,400 training samples and shows test AUC rising with features.","key_machinery":"The machinery is a Matrix Product State (MPS) tensor-network representation of the quantum feature state, updated with SVD truncation that keeps discarded squared singular values below $10^{-16}$, so simulation error sits at 64-bit floating-point level. A circuit ansatz encodes each data point by exponentiating an Ising Hamiltonian with single-qubit $Z$ fields and $XX$ couplings along a linear chain of tunable interaction distance $d$ (equations (4) and (5)), and the Gram matrix entry $K_{ij} = |\\langle\\psi(x_i)|\\psi(x_j)\\rangle|^2$ is computed by contracting two MPS in $O(m\\chi^3)$ time. Because each entry is independent, the workload tiles across parallel processes (no-messaging or round-robin), which is what makes the 6,400-sample kernel feasible.","core_discovery":"The paper's central discovery is that a quantum kernel SVM, realized through MPS simulation rather than a quantum computer, can be pushed to 165 qubits (features) and 6,400 training samples while keeping memory per state below 15 KiB for the linear ansatz, and that classification quality rises with scale. The authors report that test AUC at 6,400 samples improves steadily as features increase, that the quantum kernel outperforms the Gaussian kernel at kernel bandwidths $\\gamma = 0.5$ and $1.0$, and that increasing circuit depth beyond a few layers degrades performance through kernel concentration. They also identify a runtime crossover between CPU and GPU implementations at interaction distance $d = 10$ and bond dimension $\\chi \\approx 320$, beyond which the GPU backend is faster for both simulation and inner-product computation.","pith_inferences":["If the improvement with feature count and data size reproduces under an independent validation-set rule for choosing the SVM regularization coefficient, the result would strengthen the case that quantum kernels can be useful classical surrogates at industrial scale.","The same MPS framework could be pointed at other high-dimensional tabular benchmarks (fraud, credit scoring, drug discovery) to test whether the Elliptic result is data-specific; a natural next experiment is to fix depth and bandwidth while sweeping dataset size.","The crossover at $\\chi \\approx 320$ gives a simple rule of thumb for practitioners: monitor the MPS bond dimension, and only move to GPUs when it approaches a few hundred.","Because the simulator is essentially noiseless, comparing these kernels against hardware noise models could reveal how much of the observed advantage survives on real devices, where exponential concentration may set in earlier."],"forward_implications":["Training a 6,400-sample quantum kernel on 165 features completes in about 3 hours on 32 GPUs, and extrapolations put 64,000 samples at roughly 30 hours on 320 GPUs.","For the simple linear-chain ansatz ($d=1$), the CPU backend is the cheaper option; the GPU backend only wins once interaction distance reaches $d=10$ and bond dimension reaches roughly 320.","Adding features and training samples is predicted to keep improving test AUC for quantum kernels, at least on structured tabular data like Elliptic, provided overfitting is controlled by enough samples.","Deeper encoding circuits hurt rather than help: test performance drops as depth grows, consistent with exponential concentration of kernel values.","A quantum kernel with bandwidth $\\gamma = 0.5$ or $1.0$ can beat a Gaussian kernel on this classification task, while smaller $\\gamma$ and larger interaction distances lead to weak encodings or overfitting."],"supporting_citations":[{"why":"Defines the quantum feature map and kernel Gram-matrix construction that this framework implements.","marker":"[22]"},{"why":"Benchmarking study reporting mixed results for quantum kernels at small scale, which motivates the push to larger scale.","marker":"[7]"},{"why":"Numerical evidence against quantum fidelity kernel advantage on classical data, the prior negative result this paper's positive scaling result is set against.","marker":"[8]"},{"why":"Establishes exponential concentration and untrainability in quantum kernel methods, used to explain the depth-driven performance loss.","marker":"[15]"},{"why":"Alternative GPU tensor-network quantum SVM whose scaling and truncation approach this paper distinguishes from its own.","marker":"[20]"},{"why":"CPU MPS simulation backend used in the cross-platform runtime and scaling experiments.","marker":"[24]"},{"why":"Provides the GPU tensor contraction and decomposition primitives on which the custom GPU backend is built.","marker":"[25]"},{"why":"Motivates the kernel bandwidth parameter $\\gamma$ as necessary for scaling to larger quantum feature spaces.","marker":"[26]"},{"why":"The Elliptic Bitcoin data set used for all classification and resource-scaling experiments.","marker":"[32]"}],"fun_headline_variants":["MPS simulation scales quantum kernels to 165 features","Quantum kernel SVM processes 6,400 samples via MPS","Quantum kernel performance improves with size, MPS shows","CPU-GPU crossover found in MPS quantum kernel sims"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported performance improvements depend on choosing the SVM regularization coefficient that gives the best score on the test data, so those improvements could shrink or vanish if the coefficient were chosen on a separate validation set instead.","fun_headline_variants_meta":{"raw":{"variants":["MPS simulation scales quantum kernels to 165 features","Quantum kernel SVM processes 6,400 samples via MPS","Quantum kernel performance improves with size, MPS shows","CPU-GPU crossover found in MPS quantum kernel sims"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00026,"raw_usage":{"total_tokens":1540,"prompt_tokens":846,"completion_tokens":694,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":462,"completion_tokens_details":{"reasoning_tokens":627}},"tokens_in":462,"tokens_out":694,"duration_ms":6582,"temperature":1.0,"reasoning_tokens":627,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:45:23.380923+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the scale experiment with the SVM regularization coefficient selected on a held-out validation set rather than on the test set; if test AUC then fails to rise with feature count or with sample size across 300, 1500, and 6400 training points, the paper's central scaling claim would be refuted.","supporting_citations":[{"cited_title":"Supervised learning with quantum- enhanced feature spaces,","cited_arxiv_id":null,"evidence_quote":"Defines the quantum feature map and kernel Gram-matrix construction that this framework implements."},{"cited_title":"Better than classical? the subtle art of benchmarking quantum machine learning models,","cited_arxiv_id":null,"evidence_quote":"Benchmarking study reporting mixed results for quantum kernels at small scale, which motivates the push to larger scale."},{"cited_title":"Numerical evidence against advantage with quantum fidelity kernels on classical data,","cited_arxiv_id":null,"evidence_quote":"Numerical evidence against quantum fidelity kernel advantage on classical data, the prior negative result this paper's positive scaling result is set against."},{"cited_title":"Exponential concentration and untrainability in quantum kernel methods,","cited_arxiv_id":null,"evidence_quote":"Establishes exponential concentration and untrainability in quantum kernel methods, used to explain the depth-driven performance loss."},{"cited_title":"The ITensor Software Library for Tensor Network Calculations,","cited_arxiv_id":null,"evidence_quote":"CPU MPS simulation backend used in the cross-platform runtime and scaling experiments."},{"cited_title":"cuquantum sdk: A high-performance library for accelerating quantum science,","cited_arxiv_id":null,"evidence_quote":"Provides the GPU tensor contraction and decomposition primitives on which the custom GPU backend is built."},{"cited_title":"Importance of kernel bandwidth in quantum machine learning,","cited_arxiv_id":null,"evidence_quote":"Motivates the kernel bandwidth parameter $\\gamma$ as necessary for scaling to larger quantum feature spaces."},{"cited_title":"Elliptic dataset from kaggle","cited_arxiv_id":null,"evidence_quote":"The Elliptic Bitcoin data set used for all classification and resource-scaling experiments."}],"review_version":1}