{"id":"fd925e77-1c5f-40f3-97cc-56a10e317222","arxiv_id":"2509.04536","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Q-SafeML applies quantum distance metrics, such as trace distance and fidelity, to compare correct and incorrect predictions of quantum classifiers, offering a way to flag unsafe or unreliable QML behavior.","lead":"This paper proposes Q-SafeML, a method that uses quantum distance measures to compare the correct and incorrect predictions of quantum machine learning models, as a way to flag unreliable behavior. Early tests on two quantum classifiers show some metrics correlate with accuracy, but the results are not statistically significant.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Q-SafeML's central correlation is only as sound as the unspecified density-matrix construction in Eq. 1; if predictions are encoded as diagonal label mixtures, the quantum metrics reduce to label-frequency distances and the safety signal is an artifact of partitioning by correctness.","rationale":"The reader's weakest assumption identifies Eq. 1 as the point where correctness-relevant information could be lost. I agree and sharpen it: the paper never operationalizes the mapping from predictions to density matrices, so it is possible that the distance metrics are computed from classical label frequencies rather than from any quantum state geometry. If that is what the code does, the central claim collapses: the method would not be a quantum safety monitor but a roundabout way of re-deriving accuracy from the same label counts used to define 'correct' and 'incorrect' sets. The concrete test I propose settles this by inspecting the repository and, if necessary, recomputing Table 1 from classical label distributions. This concern is load-bearing because it attacks the mechanism through which Q-SafeML is supposed to produce a safety signal; it is not merely a complaint about small sample size or missing baselines. I also note the paper's strengths: code is promised, the limitations section is unusually honest about simulator reliance and non-significance, and the framework is clearly described at a high level. But the high-level description hides the crucial representational detail. The reader's REJECT verdict remains appropriate and I do not propose any adjustment.","tokens_in":8247,"tokens_out":3051,"duration_ms":32793,"concrete_test":"Inspect the GitHub repository's density-matrix construction for the VQC experiments. If predictions are one-hot labels or softmax vectors placed on the diagonal of a density matrix, recompute all four metrics from the empirical label distributions of the correct and incorrect sets using classical formulas (total variation, squared Bhattacharyya, Hellinger, KL divergence) and compare to Table 1. If the values match Table 1 to numerical precision, the quantum metrics carry no state-space information beyond label frequencies, and the claimed safety signal is an artifact of partitioning by correctness. If they do not match, report the actual construction; if the construction uses non-orthogonal or entangled states, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The method's central step is Eq. 1, but the paper never states how a classifier prediction is converted into a density matrix. Section 4.5 says Q-SafeML compares 'sets of correctly and incorrectly classified predictions' without defining the mapping from a prediction to |ψ_i⟩ or the probabilities p_i. The implementation choice is decisive: if each predicted class label is represented by an orthonormal basis state |i⟩ and p_i is the empirical frequency of that label in the set, then the average density matrix is diagonal with entries equal to label frequencies. For such states, trace distance between the correct and incorrect set averages is exactly the total variation distance between the two label distributions; fidelity is the squared Bhattacharyya coefficient; Bures distance and quantum relative entropy are deterministic functions of those label counts. All four 'quantum' metrics then contain no information beyond how often each class appears in the correct versus incorrect partition. The reported correlations with accuracy would then be a tautology of the partitioning procedure—accuracy determines the counts that determine the distances—rather than evidence that quantum state geometry captures classifier reliability. This is not a purely philosophical worry: Table 1 reports metric values and accuracies for only four datasets, and if the code constructs density matrices in this diagonal way, the correlations in Section 5.2 are fully determined by label frequencies. The paper itself concedes the correlations are not statistically significant, but the more fundamental problem is that the quantity being correlated may be a deterministic transform of accuracy itself, not an independent safety signal.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Q-SafeML, a post-classification safety-assessment method for quantum machine learning classifiers. It represents classifier predictions as density matrices and compares sets of correctly and incorrectly classified predictions using four quantum distance metrics: trace distance, fidelity, Bures distance, and quantum relative entropy. The authors report Pearson correlations between these metrics and classifier accuracy on a Variational Quantum Classifier (VQC) over four datasets and on a Quantum Convolutional Neural Network (QCNN) for digit classification. They also sketch a threshold-based monitoring framework. The paper claims this enables informed human oversight and concept-drift detection, although Section 4.5 explicitly states that the method is model-dependent, post-classification, and does not evaluate input drift.","tokens_in":8611,"tokens_out":6085,"duration_ms":61533,"significance":"If the method worked as claimed, it would address a real gap: safety monitoring for QML is underdeveloped. The paper is among the first to propose quantum distance metrics for this purpose, and it provides an initial set of experiments on two QML architectures with code availability. However, the central methodological details are unspecified, the experimental evidence is too thin to support the conclusions, and the abstract's concept-drift claim conflicts with the method description. As it stands, the contribution reduces to a preliminary, label-dependent observation that some quantum distance metrics correlate with accuracy on four datasets; it does not establish a runtime safety-monitoring method.","major_comments":[{"comment":"The abstract states Q-SafeML 'detects distances between operational and training data addressing the concept drifts', but §4.5 says the method does not evaluate input drift and is post-classification. §6.4 further concedes the method was 'applied post-hoc'. These descriptions are incompatible; the central claim of a runtime safety monitor and concept-drift detector is therefore unsupported. The paper must either specify how the offline, label-based analysis transfers to operational unlabeled data, or be reframed as an offline validation diagnostic.","section":"Abstract vs §4.5"},{"comment":"The mapping from a classifier prediction to the density matrix ρ is never specified. Equation (1) is a generic mixture; the text does not state how a prediction becomes |ψ_i⟩ and p_i, nor how a set of predictions is aggregated into one density matrix. If p_i are empirical label frequencies and |ψ_i⟩ are orthonormal basis states, the four metrics become deterministic functions of label counts, and the correlations in §5.2 would be an artifact of partitioning by correctness. This load-bearing encoding must be defined and ideally compared with a classical label-frequency baseline.","section":"§4, Eq. (1), and §4.4"},{"comment":"The reported Pearson correlations (quantum relative entropy r = 0.54, trace distance r = 0.48) are based on only four datasets. The paper itself states 'these results are not statistically significant', yet §6.1 interprets them as 'indicating some potential as reliability indicators'. No confidence intervals, permutation tests, or additional datasets are provided. Since the entire method is justified by these correlations, the empirical evidence is insufficient to support the conclusions.","section":"§5.2, Table 1"},{"comment":"The threshold-based monitoring framework is described only informally: metrics are normalized and 'the system flags any samples whose metric values exceed predefined thresholds'. There is no explanation of how thresholds are chosen, what object (sample? class? set?) is evaluated, or any quantitative evaluation showing that threshold breaches correspond to misclassifications. Figure 7 is an illustration, not a validation. This is central to the claimed 'informed human oversight' and requires a concrete algorithm and evaluation.","section":"§5.4"},{"comment":"The two compared sets are defined as 'one containing misclassified predictions for a specific label, and the other containing correctly classified ones'. Computing this distance therefore requires ground-truth labels. At deployment, labels are typically unavailable, so the online phase is not well-defined: what unlabeled data is compared with what reference distribution? If the method is intended only for offline validation, that should be stated explicitly and the safety-monitoring ambitions scaled back accordingly.","section":"§4.4 and Fig. 2"}],"minor_comments":[{"comment":"The table header contains a typo: 'T rue Accuracy' should be 'True Accuracy'. The header row formatting is also inconsistent.","section":"Table 1"},{"comment":"There are typographical errors: 'nd uncertainty' should be 'and uncertainty', and the phrase 'SafeML the method encountered greater complexity' is grammatically garbled.","section":"§6.1"},{"comment":"The caption of Figure 5 refers to 'QNN Classes' while the text in §5.3 says QCNN. Please standardize the terminology across the paper.","section":"Fig. 5 vs text"},{"comment":"The synthetic datasets 'Family' and 'Transport' are not described: no feature counts, sample sizes, or label distributions are given. This limits reproducibility, even with the code link.","section":"§5.2"}],"recommendation":"reject","confidential_remarks":"The manuscript reads like a preliminary undergraduate study: the experimental design has very low statistical power, and the central methodological step (density-matrix encoding) is left unspecified, making the reported correlations difficult to interpret. The self-citation pattern is noticeable but not by itself problematic; what matters is the lack of engagement with straightforward classical baselines (e.g., label-frequency distances) that could distinguish genuine quantum-state effects from simple count-based artifacts. In its current form the paper would not be suitable for a serious journal in this field."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth knowing: this paper is the first to adapt SafeML to quantum classifiers using quantum distance metrics. That is a real gap. The core idea—compare sets of correct and incorrect predictions with trace distance, fidelity, Bures distance, and quantum relative entropy—is original in combination, though each component is known. The authors are honest about using simulators and about the correlations not being statistically significant. Code is linked on GitHub.\n\nThe main problem is Eq. 1. The paper says predictions are transformed into density matrices but never defines how a prediction becomes |ψ_i⟩ and p_i. The stress-test worry is justified: if the encoding is just class labels as orthonormal basis states with empirical frequencies, the density matrix is diagonal and all four metrics reduce to deterministic functions of label counts in the two sets. That is not exactly a tautology—the distance between correct and incorrect label distributions contains confusion-matrix information beyond raw accuracy—but it would gut the quantum-specific claim and make the correlations a byproduct of the partitioning rule. The paper must specify the encoding and, ideally, show the actual code.\n\nSecond, the abstract says Q-SafeML detects distances between operational and training data, addressing concept drift. Section 4.5 explicitly says it does not evaluate input drift and is post-classification. The body is right, and the abstract needs to be brought in line.\n\nOther soft spots: four VQC datasets and one QCNN dataset, no error bars, no baselines, and post-hoc interpretation of which metric is 'best.' The discussion overplays what a non-significant correlation of 0.54 can support, though the authors do hedge appropriately.\n\nOverall, this is a proof-of-concept for a niche that deserves attention. The idea is plausible, the writing is clear, and the limitations are acknowledged. But the method is underspecified at its central step, and the evidence is too thin to support the current claims. I'd send it to peer review with expectations of major revision: precise algorithm definition, a corrected abstract, and either stronger experiments or a much more cautious framing. The code availability is a point in its favor; a referee should check whether the implementation matches the described method.\n\nIn sum: worth engaging with, not worth citing yet.","headline":"A plausible first step toward QML safety monitoring, but Eq. 1's underspecification and weak evidence make it a reject-with-revision rather than a contribution in current form.","tokens_in":9057,"tokens_out":3589,"would_cite":false,"duration_ms":34510,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Quantum distances flag unreliable quantum classifiers","keywords":["quantum machine learning","SafeML","quantum distance metrics","density matrices","safety monitoring","concept drift","variational quantum classifier","quantum convolutional neural network"],"falsifier":"Run the same variational quantum classifier on a larger set of labeled datasets and compute the Pearson correlation between the Q-SafeML distances and true accuracy; if the confidence interval for trace distance or quantum relative entropy includes zero at adequate statistical power, the core signal is absent. A sharper test: shuffle the label assignment of the correct and misclassified sets and recompute the distances—if shuffled distances correlate with accuracy as strongly as the real ones, the signal is an artifact of the encoding rather than of correctness.","tokens_in":8181,"feed_emoji":"⚛️","tokens_out":5588,"duration_ms":53369,"temperature":0.7,"pith_summary":"This paper claims that a classical technique for monitoring machine-learning reliability, SafeML, can be adapted to quantum machine learning by replacing standard statistical distances with distances defined on quantum states. The proposed method, Q-SafeML, encodes a quantum classifier's correct and incorrect predictions for each label as density matrices and measures how far apart those two sets are using trace distance, fidelity, Bures distance, and quantum relative entropy. Across a variational quantum classifier and a quantum convolutional neural network, the paper reports that the resulting distances move with true accuracy, with quantum relative entropy and trace distance showing the strongest correlations and metric divergence flagging unreliable classes. If this holds, Q-SafeML gives operators a post-classification, model-dependent safety signal for QML systems without waiting for an integrated training-time monitor.","feed_headline":"Quantum distances flag unreliable quantum classifiers","feed_subtitle":"A quantum-native twist on SafeML compares correct versus wrong outputs to surface model weaknesses.","key_machinery":"The load-bearing object is the density-matrix encoding ρ = Σ_i p_i |ψ_i⟩⟨ψ_i|, which maps each set of classifier outputs to a quantum mixed state. The argument then runs through four quantum distance metrics defined on such states: trace distance, fidelity, Bures distance, and quantum relative entropy. These metrics quantify how distinguishable the correct-output set is from the incorrect-output set, and the comparison of those distances against actual accuracy is what produces the safety signal.","core_discovery":"Q-SafeML's central claim is that the reliability of a quantum classifier can be assessed after the fact by comparing the quantum states associated with its wrong predictions against those associated with its right ones. Concretely, for each label the method forms two sets—misclassified predictions and correctly classified ones—constructs a density matrix for each set, and computes a quantum distance between them. The distance is then read against the model's measured accuracy. Across the VQC toy datasets and the QCNN digit experiments, trace distance was the most consistent and interpretable, Bures distance responded most strongly in noisy and mixed-state conditions, and quantum relative ent","pith_inferences":["The correlation evidence in the paper rests on a small number of datasets and is explicitly not statistically significant; a larger benchmark across more datasets and architectures would be needed before any single metric is treated as a dependable reliability indicator.","If the density-matrix encoding preserves error-related information, the same correct-versus-incorrect output comparison could be pushed earlier into the pipeline, for instance as a loss term or an online monitor, though the paper only applies it post-hoc.","On real quantum hardware, the paper expects Bures distance and quantum relative entropy to behave differently under noise, so simulator-based results are likely optimistic; testing on hardware noise models would reveal which metric is actually usable.","The method's logic might transfer to quantum regression or reinforcement learning if meaningful output states can be built for those settings, but the paper only claims classification."],"forward_implications":["Q-SafeML gives a post-classification safety check that works across at least two QML architectures, a variational quantum classifier and a quantum convolutional neural network.","Trace distance behaves as the most stable, interpretable indicator of classifier confusion, making it a default candidate for monitoring.","Bures distance is suited to noisy and mixed-state conditions expected on real quantum hardware, potentially offering a more hardware-realistic signal.","When fidelity and trace distance disagree, the paper suggests this divergence marks ambiguous classification boundaries, so a multi-metric view is safer than relying on one distance.","Normalized thresholds over the metrics can flag individual samples as potentially unsafe, supporting human oversight during deployment."],"supporting_citations":[{"why":"Introduces SafeML, the classical statistical-distance monitoring framework that Q-SafeML adapts to quantum settings.","marker":"[8]"},{"why":"Provides the standard quantum-information definitions, including trace distance, that ground the quantum metric calculations.","marker":"[23]"},{"why":"Defines fidelity for mixed quantum states, used in Equation (3) and in the Bures distance derivation.","marker":"[17]"},{"why":"Supplies the Bures distance as a metric over quantum state spaces, used in Equation (4).","marker":"[11]"},{"why":"Provides the transition-probability formulation underlying the Bures metric for mixed states.","marker":"[26]"},{"why":"Defines quantum relative entropy and its information-theoretic role, grounding Equation (5).","marker":"[28]"},{"why":"Describes the variational quantum classifier architecture used as one of the experimental testbeds.","marker":"[15]"},{"why":"Provides the out-of-distribution detection background that motivates SafeML's original goal and the shift to quantum settings.","marker":"[21]"}],"fun_headline_variants":["Quantum distances expose weak QML classifiers","Post-hoc quantum state gaps reveal model risk","Q-SafeML: comparing right and wrong outputs for ML safety","Trace distance best for quantum ML safety in tests","Wrong predictions vs right ones: quantum distance measures trust"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The mapping of a classifier's outputs to density matrices (Equation 1) preserves the information that separates correct from incorrect predictions; if the encoding or the aggregation smooths that signal away, the reported distance-to-accuracy correlations will not generalize.","fun_headline_variants_meta":{"raw":{"variants":["Quantum distances expose weak QML classifiers","Post-hoc quantum state gaps reveal model risk","Q-SafeML: comparing right and wrong outputs for ML safety","Trace distance best for quantum ML safety in tests","Wrong predictions vs right ones: quantum distance measures trust"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000361,"raw_usage":{"total_tokens":1773,"prompt_tokens":715,"completion_tokens":1058,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":459,"completion_tokens_details":{"reasoning_tokens":985}},"tokens_in":459,"tokens_out":1058,"duration_ms":11345,"temperature":1.0,"reasoning_tokens":985,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T10:30:22.991287+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same variational quantum classifier on a larger set of labeled datasets and compute the Pearson correlation between the Q-SafeML distances and true accuracy; if the confidence interval for trace distance or quantum relative entropy includes zero at adequate statistical power, the core signal is absent. A sharper test: shuffle the label assignment of the correct and misclassified sets and recompute the distances—if shuffled distances correlate with accuracy as strongly as the real ones, the signal is an artifact of the encoding rather than of correctness.","supporting_citations":[{"cited_title":"In: Model-Based Safety and Assessment: 7th International Sympo- sium, IMBSA 2020, Lisbon, Portugal, September 14–16, 2020, Proceedings 7","cited_arxiv_id":null,"evidence_quote":"Introduces SafeML, the classical statistical-distance monitoring framework that Q-SafeML adapts to quantum settings."},{"cited_title":"Cambridge University Press (2002)","cited_arxiv_id":null,"evidence_quote":"Provides the standard quantum-information definitions, including trace distance, that ground the quantum metric calculations."},{"cited_title":"Journal of Modern Optics41(12), 2315–2323 (1994) QSafeML: Safety Evaluation of Quantum ML Models 17","cited_arxiv_id":null,"evidence_quote":"Defines fidelity for mixed quantum states, used in Equation (3) and in the Bures distance derivation."},{"cited_title":"Transactions of the American Mathemat- ical Society 135, 199–212 (1969)","cited_arxiv_id":null,"evidence_quote":"Supplies the Bures distance as a metric over quantum state spaces, used in Equation (4)."},{"cited_title":"Reports on Mathematical Physics9(2), 273–279 (1976)","cited_arxiv_id":null,"evidence_quote":"Provides the transition-probability formulation underlying the Bures metric for mixed states."},{"cited_title":"Reviews of Modern Physics74(1), 197–234 (2002)","cited_arxiv_id":null,"evidence_quote":"Defines quantum relative entropy and its information-theoretic role, grounding Equation (5)."},{"cited_title":"Bachelor’s thesis, University of Barcelona (2018), https://diposit.ub.edu/dspace/bitstream/2445/140318/1/ GIL%20FUSTER%20Elies%20Miquel.pdf","cited_arxiv_id":null,"evidence_quote":"Describes the variational quantum classifier architecture used as one of the experimental testbeds."}],"review_version":1}