REVIEW 5 major objections 4 minor 30 references
Q-SafeML: Safety Assessment of Quantum Machine Learning via Quantum Distance Metrics
T0 review · 5 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read Quantum distances flag unreliable quantum classifiers
desk verdict A plausible first step toward QML safety monitoring, but Eq. 1's underspecification and weak evidence make it a reject-with-revision rather than a contribution in current form. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the density-matrix encoding ρ = Σ_i p_i |ψ_i⟩⟨ψ_i|, which maps each set of classifier outputs to a quantum mixed state. The argument then runs through four quantum distance metrics defined on such states: trace distance, fidelity, Bures distance, and quantum relative entropy. These metrics quantify how distinguishable the correct-output set is from the incorrect-output set, and the comparison of those distances against actual accuracy is what produces the safety signal.
What would settle it
Run the same variational quantum classifier on a larger set of labeled datasets and compute the Pearson correlation between the Q-SafeML distances and true accuracy; if the confidence interval for trace distance or quantum relative entropy includes zero at adequate statistical power, the core signal is absent. A sharper test: shuffle the label assignment of the correct and misclassified sets and recompute the distances—if shuffled distances correlate with accuracy as strongly as the real ones, the signal is an artifact of the encoding rather than of correctness.
Extended reading notes
Core claim
Q-SafeML's central claim is that the reliability of a quantum classifier can be assessed after the fact by comparing the quantum states associated with its wrong predictions against those associated with its right ones. Concretely, for each label the method forms two sets—misclassified predictions and correctly classified ones—constructs a density matrix for each set, and computes a quantum distance between them. The distance is then read against the model's measured accuracy. Across the VQC toy datasets and the QCNN digit experiments, trace distance was the most consistent and interpretable, Bures distance responded most strongly in noisy and mixed-state conditions, and quantum relative ent
Load-bearing premise
The mapping of a classifier's outputs to density matrices (Equation 1) preserves the information that separates correct from incorrect predictions; if the encoding or the aggregation smooths that signal away, the reported distance-to-accuracy correlations will not generalize.
Editorial extensions
If this is right
- Q-SafeML gives a post-classification safety check that works across at least two QML architectures, a variational quantum classifier and a quantum convolutional neural network.
- Trace distance behaves as the most stable, interpretable indicator of classifier confusion, making it a default candidate for monitoring.
- Bures distance is suited to noisy and mixed-state conditions expected on real quantum hardware, potentially offering a more hardware-realistic signal.
- When fidelity and trace distance disagree, the paper suggests this divergence marks ambiguous classification boundaries, so a multi-metric view is safer than relying on one distance.
- Normalized thresholds over the metrics can flag individual samples as potentially unsafe, supporting human oversight during deployment.
Reading between the lines
- The correlation evidence in the paper rests on a small number of datasets and is explicitly not statistically significant; a larger benchmark across more datasets and architectures would be needed before any single metric is treated as a dependable reliability indicator.
- If the density-matrix encoding preserves error-related information, the same correct-versus-incorrect output comparison could be pushed earlier into the pipeline, for instance as a loss term or an online monitor, though the paper only applies it post-hoc.
- On real quantum hardware, the paper expects Bures distance and quantum relative entropy to behave differently under noise, so simulator-based results are likely optimistic; testing on hardware noise models would reveal which metric is actually usable.
- The method's logic might transfer to quantum regression or reinforcement learning if meaningful output states can be built for those settings, but the paper only claims classification.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Q-SafeML, a post-classification safety-assessment method for quantum machine learning classifiers. It represents classifier predictions as density matrices and compares sets of correctly and incorrectly classified predictions using four quantum distance metrics: trace distance, fidelity, Bures distance, and quantum relative entropy. The authors report Pearson correlations between these metrics and classifier accuracy on a Variational Quantum Classifier (VQC) over four datasets and on a Quantum Convolutional Neural Network (QCNN) for digit classification. They also sketch a threshold-based monitoring framework. The paper claims this enables informed human oversight and concept-drift detection, although Section 4.5 explicitly states that the method is model-dependent, post-classification, and does not evaluate input drift.
Significance. If the method worked as claimed, it would address a real gap: safety monitoring for QML is underdeveloped. The paper is among the first to propose quantum distance metrics for this purpose, and it provides an initial set of experiments on two QML architectures with code availability. However, the central methodological details are unspecified, the experimental evidence is too thin to support the conclusions, and the abstract's concept-drift claim conflicts with the method description. As it stands, the contribution reduces to a preliminary, label-dependent observation that some quantum distance metrics correlate with accuracy on four datasets; it does not establish a runtime safety-monitoring method.
major comments (5)
- [Abstract vs §4.5] The abstract states Q-SafeML 'detects distances between operational and training data addressing the concept drifts', but §4.5 says the method does not evaluate input drift and is post-classification. §6.4 further concedes the method was 'applied post-hoc'. These descriptions are incompatible; the central claim of a runtime safety monitor and concept-drift detector is therefore unsupported. The paper must either specify how the offline, label-based analysis transfers to operational unlabeled data, or be reframed as an offline validation diagnostic.
- [§4, Eq. (1), and §4.4] The mapping from a classifier prediction to the density matrix ρ is never specified. Equation (1) is a generic mixture; the text does not state how a prediction becomes |ψ_i⟩ and p_i, nor how a set of predictions is aggregated into one density matrix. If p_i are empirical label frequencies and |ψ_i⟩ are orthonormal basis states, the four metrics become deterministic functions of label counts, and the correlations in §5.2 would be an artifact of partitioning by correctness. This load-bearing encoding must be defined and ideally compared with a classical label-frequency baseline.
- [§5.2, Table 1] The reported Pearson correlations (quantum relative entropy r = 0.54, trace distance r = 0.48) are based on only four datasets. The paper itself states 'these results are not statistically significant', yet §6.1 interprets them as 'indicating some potential as reliability indicators'. No confidence intervals, permutation tests, or additional datasets are provided. Since the entire method is justified by these correlations, the empirical evidence is insufficient to support the conclusions.
- [§5.4] The threshold-based monitoring framework is described only informally: metrics are normalized and 'the system flags any samples whose metric values exceed predefined thresholds'. There is no explanation of how thresholds are chosen, what object (sample? class? set?) is evaluated, or any quantitative evaluation showing that threshold breaches correspond to misclassifications. Figure 7 is an illustration, not a validation. This is central to the claimed 'informed human oversight' and requires a concrete algorithm and evaluation.
- [§4.4 and Fig. 2] The two compared sets are defined as 'one containing misclassified predictions for a specific label, and the other containing correctly classified ones'. Computing this distance therefore requires ground-truth labels. At deployment, labels are typically unavailable, so the online phase is not well-defined: what unlabeled data is compared with what reference distribution? If the method is intended only for offline validation, that should be stated explicitly and the safety-monitoring ambitions scaled back accordingly.
minor comments (4)
- [Table 1] The table header contains a typo: 'T rue Accuracy' should be 'True Accuracy'. The header row formatting is also inconsistent.
- [§6.1] There are typographical errors: 'nd uncertainty' should be 'and uncertainty', and the phrase 'SafeML the method encountered greater complexity' is grammatically garbled.
- [Fig. 5 vs text] The caption of Figure 5 refers to 'QNN Classes' while the text in §5.3 says QCNN. Please standardize the terminology across the paper.
- [§5.2] The synthetic datasets 'Family' and 'Transport' are not described: no feature counts, sample sizes, or label distributions are given. This limits reproducibility, even with the code link.
Circularity Check
No significant circularity; distance metrics are measured post-hoc and correlated with accuracy, not derived from it; self-citations are contextual, not load-bearing.
full rationale
The paper's derivation chain is: Eq. 1 defines a density-matrix encoding of predictions; Eqs. 2–5 define trace distance, fidelity, Bures distance, and quantum relative entropy; Section 4 forms two sets (misclassified vs. correctly classified predictions) from validation labels and computes these distances; Section 5.2 correlates those distances with true accuracy. No equation makes a predicted distance equal to accuracy, and no parameter is fitted to accuracy and then renamed as a safety measure. The correlations are measured post-hoc on fixed model outputs, and the paper explicitly cautions that they are not statistically significant (Section 5.2). The post-hoc use of labels is a genuine limitation, as the paper itself acknowledges ('SafeML was applied post-hoc, i.e. after model predictions were made. This may have limited its potential', Section 6.4), but it does not make the reported correlation circular: the distance values are functions of the quantum states in the two sets, not of the accuracy number alone. The skeptical reduction to label-frequency distances depends on an unstated implementation choice (encoding each predicted class as an orthonormal basis state with empirical frequencies); the paper does not specify that construction, so no Eq. X = Eq. Y reduction can be exhibited. Self-citations (e.g., SafeML [8] and extensions) are prior work cited for context; none is invoked as a uniqueness theorem or as the sole justification for Q-SafeML's validity. I therefore find no significant circularity and rate the paper 2 for the presence of minor, non-load-bearing self-citation.
Assumptions & free parameters
free parameters (3)
- safety threshold values (Section 5.4)
- number of PCA components =
6
- metric normalization constants =
sqrt(2) for Bures, 1-fidelity
assumptions (4)
- standard math Standard quantum information distance metrics (trace distance, fidelity, Bures distance, quantum relative entropy) are well-defined for density matrices.
- domain assumption Density matrix representation (Eq. 1) faithfully captures QML classifier outputs for safety assessment.
- domain assumption Distance between sets of correct and incorrect predictions is a meaningful proxy for model reliability.
- ad hoc to paper Classical SafeML is incompatible with quantum outputs.
Cite this review
Pith. "Pith review of Q-SafeML: Safety Assessment of Quantum Machine Learning via Quantum Distance Metrics." pith.science (2026). https://pith.science/paper/ZOUFCXXG
@misc{pith2026250904536,
author = {Pith},
title = {Pith review of: Q-SafeML: Safety Assessment of Quantum Machine Learning via Quantum Distance Metrics},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZOUFCXXG}},
note = {Machine review of arXiv:2509.04536}
}
read the original abstract
The rise of machine learning in safety-critical systems has paralleled advancements in quantum computing, leading to the emerging field of Quantum Machine Learning (QML). While safety monitoring has progressed in classical ML, existing methods are not directly applicable to QML due to fundamental differences in quantum computation. Given the novelty of QML, dedicated safety mechanisms remain underdeveloped. This paper introduces Q-SafeML, a safety monitoring approach for QML. The method builds on SafeML, a recent method that utilizes statistical distance measures to assess model accuracy and provide confidence in the reasoning of an algorithm. An adapted version of Q-SafeML incorporates quantum-centric distance measures, aligning with the probabilistic nature of QML outputs. This shift to a model-dependent, post-classification evaluation represents a key departure from classical SafeML, which is dataset-driven and classifier-agnostic. The distinction is motivated by the unique representational constraints of quantum systems, requiring distance metrics defined over quantum state spaces. Q-SafeML detects distances between operational and training data addressing the concept drifts in the context of QML. Experiments on QCNN and VQC Models show that this enables informed human oversight, enhancing system transparency and safety.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
DIN SPEC 92005: Machine Learning – Uncertainty Quantification. DIN SPEC 92005, Beuth Verlag GmbH (DIN - Deutsches Institut für Normung), Berlin, Ger- many(2022), https://www.din.de/en/wdc-beuth:din21:343195966,standardfor methods and measures for quantifying uncertainty in machine learning models
work page 2022
-
[2]
International Journal of Computing (2019)
Adepoju, O., et al.: Quantum computing: A paradigm shift in computational tech- nology. International Journal of Computing (2019)
work page 2019
-
[3]
arXiv preprint arXiv:2501.14412 (2025) 16 O
Ahmed, T., Kashif, M., Marchisio, A., Shafique, M.: Quantum neural net- works: A comparative analysis and noise robustness evaluation. arXiv preprint arXiv:2501.14412 (2025) 16 O. Dunn et al
arXiv 2025
-
[4]
In: Computer Safety, Reliability, and Security
Akram, M.N., Ambekar, A., Sorokos, I., Aslansefat, K., Schneider, D.: Stadre and stadro: Reliability and robustness estimation of ml-based forecasting using statisti- cal distance measures. In: Computer Safety, Reliability, and Security. SAFECOMP 2022 Workshops: DECSoS, DepDevOps, SASSUR, SENSEI, USDAI, and WAISE Munich, Germany, September 6–9, 2022, Proc...
work page 2022
-
[5]
Aslansefat, K., Hashemian, M., Walker, M., Akram, M.N., Sorokos, I., Papadopou- los, Y.: Explaining black boxes with a smile: Statistical model-agnostic inter- pretability with local explanations. IEEE Software (2023)
work page 2023
-
[6]
Aslansefat, K., Kabir, S., Abdullatif, A., Vasudevan, V., Papadopoulos, Y.: To- ward improving confidence in autonomous vehicle software: A study on traffic sign recognition systems. Computer54(8), 66–76 (2021)
work page 2021
-
[7]
Aslansefat, K., Nikolaou, P., Walker, M., Akram, M.N., Sorokos, I., Reich, J., Kolios, P., Michael, M.K., Theocharides, T., Ellinas, G., et al.: Safedrones: Real- time reliability evaluation of uavs using executable digital dependable identities. In: Model-Based Safety and Assessment: 8th International Symposium, IMBSA 2022, Munich, Germany, September 5–7...
work page 2022
-
[8]
Aslansefat, K., Sorokos, I., Whiting, D., Tavakoli Kolagari, R., Papadopoulos, Y.: Safeml: safety monitoring of machine learning classifiers through statistical differ- ence measures. In: Model-Based Safety and Assessment: 7th International Sympo- sium, IMBSA 2020, Lisbon, Portugal, September 14–16, 2020, Proceedings 7. pp. 197–211. Springer (2020)
work page 2020
Show all 30 references
-
[9]
Quantum Science and Technology4(4), 043001 (2019)
Benedetti, M., Lloyd, E., Sack, S., Fiorentini, M.: Parameterized quantum circuits as machine learning models. Quantum Science and Technology4(4), 043001 (2019). https://doi.org/10.1088/2058-9565/ab4eb5
2019 doi
-
[10]
Bergler, M., Kolagari, R.T., Lundqvist, K.: Case study on the use of the safeml ap- proachintrainingautonomousdrivingvehicles.In:ImageAnalysisandProcessing– ICIAP2022:21stInternationalConference,Lecce,Italy,May23–27,2022,Proceed- ings, Part III. pp. 87–97. Springer (2022)
2022
-
[11]
Transactions of the American Mathemat- ical Society 135, 199–212 (1969)
Bures, D.: An extension of kakutani’s theorem on infinite product measures to the tensor product of semifinitew∗-algebras. Transactions of the American Mathemat- ical Society 135, 199–212 (1969)
1969
-
[12]
IEEE Access10, 39769–39781 (2022)
Cho, H., Lee, K., Choi, N., Kim, S., Lee, J., Yang, S.: Online safety zone estimation and violation detection for nonstationary objects in workplaces. IEEE Access10, 39769–39781 (2022)
2022
-
[13]
arXiv preprint arXiv:2312.10801 (2023)
Farhad, A.H., Sorokos, I., Akram, M.N., Aslansefat, K., Schneider, D.: Scope com- pliance uncertainty estimate. arXiv preprint arXiv:2312.10801 (2023)
2023 arXiv
-
[14]
In: Model-Based Safety and Assessment: 8th International Symposium, IMBSA 2022, Munich, Germany, September 5–7, 2022, Proceedings
Farhad, A.H., Sorokos, I., Schmidt, A., Akram, M.N., Aslansefat, K., Schneider, D.: Keep your distance: Determining sampling and distance thresholds in machine learning monitoring. In: Model-Based Safety and Assessment: 8th International Symposium, IMBSA 2022, Munich, Germany,...
2022
-
[15]
Bachelor’s thesis, University of Barcelona (2018), https://diposit.ub.edu/dspace/bitstream/2445/140318/1/ GIL%20FUSTER%20Elies%20Miquel.pdf
Gil Fuster, E.M.: Variational quantum classifier. Bachelor’s thesis, University of Barcelona (2018), https://diposit.ub.edu/dspace/bitstream/2445/140318/1/ GIL%20FUSTER%20Elies%20Miquel.pdf
2018
-
[16]
IBM Quantum Computing Blog (2024), https://www.ibm.com/quantum/blog/ ibm-quantum-roadmap-2025?lnk=ushpv18r1
IBM: Ibm roadmap to quantum-centric supercomputers (updated 2024). IBM Quantum Computing Blog (2024), https://www.ibm.com/quantum/blog/ ibm-quantum-roadmap-2025?lnk=ushpv18r1
2024
-
[17]
Journal of Modern Optics41(12), 2315–2323 (1994) QSafeML: Safety Evaluation of Quantum ML Models 17
Jozsa, R.: Fidelity for mixed quantum states. Journal of Modern Optics41(12), 2315–2323 (1994) QSafeML: Safety Evaluation of Quantum ML Models 17
1994
-
[18]
In: 2022 International Conference on Computing, Electronics & Communications Engineering (iCCECE)
Kabir, S., Aslansefat, K., Gope, P., Campean, F., Papadopoulos, Y.: Combining drone-based monitoring and machine learning for online reliability evaluation of wind turbines. In: 2022 International Conference on Computing, Electronics & Communications Engineering (iCCECE). pp. ...
2022
-
[19]
Neural Networks175, 106288 (2024)
Kim, B., Kim, B., Hyun, Y.: Investigation of out-of-distribution detection across various models and training methodologies. Neural Networks175, 106288 (2024). https://doi.org/10.1016/j.neunet.2024.106288
2024
-
[20]
Nature 464(7285), 45–53 (2010)
Ladd, T., Jelezko, F., Laflamme, R., Nakamura, Y., Monroe, C., O’Brien, J.: Quan- tum computers. Nature 464(7285), 45–53 (2010). https://doi.org/10.1038/ nature08812
2010
-
[21]
arXiv preprint arXiv:1706.02690 (2020)
Liang, S., Li, Y., Srikant, R.: Enhancing the reliability of out-of-distribution image detection in neural networks. arXiv preprint arXiv:1706.02690 (2020)
2020 arXiv
-
[22]
Google Blog (2024), https://blog.google/technology/research/google-willow-quantum-chip/
Neven, H.: Meet willow, our state-of-the-art quantum chip. Google Blog (2024), https://blog.google/technology/research/google-willow-quantum-chip/
2024
-
[23]
Cambridge University Press (2002)
Nielsen, M.A., Chuang, I.L.: Quantum Computation and Quantum Information. Cambridge University Press (2002)
2002
-
[24]
ACM Computing Surveys55(6) (2022)
Paleyes, A., Urma, R.G., Lawrence, N.: Challenges in deploying machine learning: A survey of case studies. ACM Computing Surveys55(6) (2022). https://doi. org/10.1145/3533378
2022 doi
-
[25]
Physical Review A102(3), 032420 (2020)
Schuld, M., Sweke, R., Meyer, J.: The effect of data encoding on the expressive power of variational quantum machine learning models. Physical Review A102(3), 032420 (2020). https://doi.org/10.1103/PhysRevA.102.032420
2020 doi
-
[26]
Reports on Mathematical Physics9(2), 273–279 (1976)
Uhlmann, A.: The transition probability in the state space of a *-algebra. Reports on Mathematical Physics9(2), 273–279 (1976)
1976
-
[27]
Retrieved fromhttps://artificialintelligenceact.eu (2024)
Union, E.: The eu artificial intelligence act: Regulatory framework for trustworthy ai. Retrieved fromhttps://artificialintelligenceact.eu (2024)
2024
-
[28]
Reviews of Modern Physics74(1), 197–234 (2002)
Vedral, V.: The role of relative entropy in quantum information theory. Reviews of Modern Physics74(1), 197–234 (2002)
2002
-
[29]
Reliability Engineering & System Safety 211, 107530 (2021)
Xu, Z., Saleh, J.: Machine learning for reliability engineering and safety applica- tions: Review of current status and future opportunities. Reliability Engineering & System Safety 211, 107530 (2021). https://doi.org/10.1016/j.ress.2021. 107530
2021 doi
-
[30]
IEEE Transactions on Software Engineering 51(1), 82–105 (2025).https://doi.org/10.1109/tse.2024.3491496
Zolfagharian, A., Abdellatif, M., Briand, L., S, R.: Smarla: A safety monitoring approach for deep reinforcement learning agents. IEEE Transactions on Software Engineering 51(1), 82–105 (2025).https://doi.org/10.1109/tse.2024.3491496
2025
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.