pith. machine review for the scientific record. sign in

arxiv: 2604.25649 · v1 · submitted 2026-04-28 · 💻 cs.LG

Recognition: unknown

Towards interpretable AI with quantum annealing feature selection

Authors on Pith no claims yet

Pith reviewed 2026-05-07 16:29 UTC · model grok-4.3

classification 💻 cs.LG
keywords interpretable AIquantum annealingfeature map selectionconvolutional neural networksexplainable AIclass disentanglementGradCAM
0
0 comments X

The pith

Encoding feature map selection as a quantum optimization problem solved by annealing produces more transparent explanations for CNN predictions than gradient-based methods.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a technique to explain convolutional neural network decisions in image classification by picking out the most relevant feature maps for each prediction. It turns this selection into a constrained optimization task that quantum annealing can solve. Compared to GradCAM and GradCAM++, the resulting explanations show better separation between different classes, making the model's logic clearer. A reader would care about this because it helps verify whether the network is focusing on meaningful patterns instead of artifacts, which matters for reliable use in high-stakes settings. The authors also look at the annealing process itself, tracking energy gaps and solution probabilities to understand its reliability.

Core claim

By formulating the problem of selecting important feature maps as a quantum constrained optimization problem and solving it with quantum annealing, the approach achieves improved class disentanglement, where the model's decision boundaries are more distinct and its reasoning more transparent than when using GradCAM or GradCAM++.

What carries the argument

Quantum annealing used to solve the combinatorial optimization of selecting the most contributory feature maps in a CNN for a given prediction.

If this is right

  • The explanations become more transparent by highlighting distinct features the model uses for specific classes.
  • Model biases or incorrect pattern learning can be identified more easily through the clearer reasoning.
  • Improved model design is possible by focusing development on the key feature maps identified.
  • Trust in the AI system increases because users can better understand the basis for predictions.
  • Analysis of the energy gap and success probability provides insight into why the quantum method performs well in practice.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the quantum solution scales, it could enable feature selection for deeper networks or larger images where classical solvers struggle.
  • Similar optimization encodings might apply to interpreting other architectures like transformers by selecting key attention heads.
  • The lack of quantitative metrics suggests that future work could measure disentanglement with explicit scores to confirm the visual improvements.
  • Testing on datasets with known biases would show whether the method reliably detects those biases.

Load-bearing premise

That the quantum annealing solution to the feature selection problem yields explanations that are superior in transparency and class separation compared to classical methods, even though the comparison relies on qualitative observation rather than numerical benchmarks.

What would settle it

Running the method and GradCAM on the same set of images and finding that the quantum-derived explanations do not show visibly clearer class boundaries or that the selected feature maps overlap more across classes.

Figures

Figures reproduced from arXiv: 2604.25649 by Alba Cervera-Lierta, Bruno Juli\'a-D\'iaz, Emanuele Costa, Francesco Aldo Venturelli, Miguel A. Gonz\'alez Ballester, Sikha O K.

Figure 1
Figure 1. Figure 1: FIG. 1. Representation of the FS algorithm reformulated as a QUBO problem. 1 view at source ↗
Figure 2
Figure 2. Figure 2: FIG. 2. (Top) Class–class correlation map for the FS algorithm for a ResNet-18 constrained to view at source ↗
Figure 3
Figure 3. Figure 3: FIG. 3. Qualitative comparison example of GradCAM and GradCAM++ with our QA- view at source ↗
Figure 4
Figure 4. Figure 4: FIG. 4. Energy gap ∆ view at source ↗
Figure 5
Figure 5. Figure 5: (left) shows the median value F for each class of the dataset and for different values of τ . Since the number of filtered FMs d may vary from image to im￾age and that measure is what determines the number of qubits of the QA protocol, we also compute F as a func￾tion of d and show it in the right part of Fig.5 (right). In both figures, the error bars represent the 1st and 3rd quartile of the samples. For … view at source ↗
Figure 6
Figure 6. Figure 6: FIG. 6. Example of the checkpoints. The train and validation view at source ↗
Figure 8
Figure 8. Figure 8: FIG. 8. Gaussian fit of the view at source ↗
Figure 9
Figure 9. Figure 9: FIG. 9. Average of ∆ view at source ↗
Figure 10
Figure 10. Figure 10: FIG. 10. (Left) Plot of ∆ view at source ↗
Figure 11
Figure 11. Figure 11: FIG. 11. Overlap between subsets of single FMs sampled by the QA for airplane, ship and truck classes. The algorithm is view at source ↗
Figure 12
Figure 12. Figure 12: FIG. 12. Comparison between state-of-the-art methods and the FS-based on QA for some samples of each image class. view at source ↗
read the original abstract

Deep learning models are used in critical applications, in which mistakes can have serious consequences. Therefore, it is crucial to understand how and why models generate predictions. This understanding provides useful information to check whether the model is learning the right patterns, detect biases in the data, improve model design, and build systems that can be trusted. This work proposes a new method for interpreting Convolutional Neural Networks in image classification tasks. The approach works by selecting the most important feature maps that contribute to each prediction. To solve this combinatorial problem, we encode it into a quantum constrained optimization problem and propose to solve it using quantum annealing. We evaluate our method against the state-of-the-art explainable AI techniques, specifically GradCAM and GradCAM++, and observe an improved class disentanglement, i.e. the model's decision boundaries become more distinct and its reasoning more transparent. This demonstrates that our approach enhances the quality of explanations, making it easier to understand which features the model relies on for specific predictions. In addition, we study the computational behavior of the quantum annealing algorithm. Specifically, we analyze the minimum energy gap of the system during computation and the probability that the algorithm finds the correct solution. These analyses provide theoretical insight into why the method works effectively in practice.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript proposes a method for interpreting CNNs in image classification by framing the selection of important feature maps as a combinatorial optimization problem, which is encoded as a quantum constrained optimization task and solved via quantum annealing. It claims this yields improved class disentanglement relative to GradCAM and GradCAM++, resulting in more distinct decision boundaries and transparent reasoning, and supplements the approach with an analysis of the quantum annealing process including the minimum energy gap and solution-finding probability.

Significance. If the superiority in interpretability is rigorously quantified, the work could meaningfully advance quantum-enhanced explainable AI by showing how quantum solvers handle combinatorial feature selection in deep models. The theoretical analysis of annealing dynamics provides a useful methodological contribution that goes beyond purely empirical claims.

major comments (2)
  1. [Abstract] Abstract and evaluation section: The central claim of 'improved class disentanglement' (more distinct decision boundaries and transparent reasoning) versus GradCAM/GradCAM++ is presented without any defined quantitative metric (e.g., per-class feature overlap, mutual information with logits, or faithfulness score), error bars, dataset details, or statistical tests. This renders the observation unverifiable and prevents assessment of effect size.
  2. [Evaluation] Evaluation section: No ablation is reported comparing the quantum-annealing solution against a classical solver (e.g., simulated annealing or integer programming) applied to the identical combinatorial objective. Without this, it is impossible to isolate whether any observed improvement stems from the quantum hardware, the encoding itself, or the feature-map selection formulation.
minor comments (1)
  1. [Method] The encoding of the feature-map selection objective into the quantum Ising or QUBO form would benefit from an explicit equation or small worked example to clarify the penalty terms and constraints.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback, which highlights important areas for strengthening the rigor of our claims. We address each major comment below and have prepared revisions to the manuscript.

read point-by-point responses
  1. Referee: [Abstract] Abstract and evaluation section: The central claim of 'improved class disentanglement' (more distinct decision boundaries and transparent reasoning) versus GradCAM/GradCAM++ is presented without any defined quantitative metric (e.g., per-class feature overlap, mutual information with logits, or faithfulness score), error bars, dataset details, or statistical tests. This renders the observation unverifiable and prevents assessment of effect size.

    Authors: We agree that the original presentation relied on qualitative observations of more distinct decision boundaries. In the revised manuscript we will introduce an explicit quantitative metric for class disentanglement (average pairwise Jaccard overlap of selected feature maps across classes) together with a faithfulness score obtained by measuring prediction degradation after ablating the selected maps. The evaluation section will also report error bars over repeated runs, name the datasets explicitly, and include statistical significance tests to quantify effect sizes. revision: yes

  2. Referee: [Evaluation] Evaluation section: No ablation is reported comparing the quantum-annealing solution against a classical solver (e.g., simulated annealing or integer programming) applied to the identical combinatorial objective. Without this, it is impossible to isolate whether any observed improvement stems from the quantum hardware, the encoding itself, or the feature-map selection formulation.

    Authors: We concur that an ablation against classical solvers on the identical objective is necessary to isolate contributions. The revised version will add results from simulated annealing on the same QUBO formulation for all problem sizes that remain computationally feasible, along with a discussion of why exact integer programming does not scale to the instance sizes solved by quantum annealing. This will clarify the respective roles of the solver and the feature-selection encoding. revision: yes

Circularity Check

0 steps flagged

No circularity: independent encoding and external baseline comparison

full rationale

The paper proposes encoding feature-map selection as a quantum constrained optimization problem solved by annealing, then evaluates the resulting explanations against independent GradCAM/GradCAM++ baselines. No equations, fitted parameters, or self-citations are shown that reduce the claimed improvement in class disentanglement to a tautology or construction from the inputs themselves. The derivation chain remains self-contained against external methods and does not invoke uniqueness theorems or ansatzes from prior author work.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only the abstract is available; no free parameters, axioms, or invented entities can be extracted or audited.

pith-pipeline@v0.9.0 · 5539 in / 1045 out tokens · 57869 ms · 2026-05-07T16:29:58.571956+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

57 extracted references · 5 canonical work pages · 3 internal anchors

  1. [1]

    As a CNN architecture, we use a pretrained ResNet-18

    dataset that contains 5000 images in total that belong to 10 distinct classes (for more details, check the IV A section). As a CNN architecture, we use a pretrained ResNet-18. An important point in our benchmarks is that we need to constrain the maximum number of FMs in the last convolutional layer,N f, to simulate the QA protocol. If we had access QA har...

  2. [2]

    to extract the eigenvalues and eigenstates of ˆH(s) at each time steps= τ 0.01. F. Complexity of the QA protocol Following the approximate adiabatic theorem [37, 55], the time evolutionτneeded to end in the ground state at the end of the protocol, scales asτ∝∆ −2 min. Therefore, one needs to prove that ∆min does not decrease exponen- tially with the syste...

  3. [3]

    Arceci, S

    L. Arceci, S. Barbarino, R. Fazio, and G. E. Santoro. Dissipative landau-zener problem and thermally assisted quantum annealing. Physical Review B, 96(5):054301, 2017

  4. [4]

    Azad et al

    R. Azad et al. Advances in medical image analysis with vision transformers: A comprehensive review. Medical Image Analysis, 91:103000, 2024

  5. [5]

    Barahona

    F. Barahona. On the computational complexity of ising spin glass models. Journal of Physics A: Mathematical and General, 15(10):3241, oct 1982

  6. [6]

    Bhagawati and T

    R. Bhagawati and T. Subramanian. An approach of a quantum-inspired document ranking algorithm by using feature selection methodology. International Journal of Information Technology, 15(8):4041–4053, 2023

  7. [7]

    Bharti, A

    K. Bharti, A. Cervera-Lierta, T. H. Kyaw, T. Haug, S. Alperin-Lea, A. Anand, M. Degroote, H. Heimonen, J. S. Kottmann, T. Menke, et al. Noisy intermediate- scale quantum algorithms. Reviews of Modern Physics, 94(1):015004, 2022

  8. [8]

    Buhrmester, D

    V. Buhrmester, D. M¨ unch, and M. Arens. Analysis of explainers of black box deep neural networks for com- puter vision: A survey. Machine Learning and Knowledge Extraction, 3(4):966–989, 2021

  9. [9]

    Chattopadhay, A

    A. Chattopadhay, A. Sarkar, P. Howlader, and V. N. Balasubramanian. Grad-cam++: Generalized gradient- based visual explanations for deep convolutional net- works. In 2018 IEEE Winter Conference on Applications of Computer Vision, pages 839–847, 2018

  10. [10]

    J. Chen, L. Song, M. J. Wainwright, and M. I. Jordan. Learning to explain: An information-theoretic perspec- tive on model interpretation. In International Conference on Machine Learning, pages 883–892. PMLR, 2018

  11. [11]

    Coates, A

    A. Coates, A. Y. Ng, and H. Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pages 215–223. JMLR Workshop and Conference Proceedings, 2011

  12. [12]

    Croitoru, V

    F.-A. Croitoru, V. Hondru, R. T. Ionescu, and M. Shah. Diffusion models in vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(9):10850–10869, 2023

  13. [13]

    Das and B

    A. Das and B. K. Chakrabarti. Quantum annealing and related optimization methods, volume 679. Springer Sci- ence & Business Media, 2005

  14. [14]

    J. S. Dehesa. Lanczos method of tridiagonalization, ja- cobi matrices and physics. Journal of Computational and Applied Mathematics, 7(4):249–259, 1981

  15. [15]

    J. Deng, W. Dong, R. Socher, et al. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255. IEEE, 2009

  16. [16]

    Towards A Rigorous Science of Interpretable Machine Learning

    F. Doshi-Velez and B. Kim. Towards a rigorous sci- ence of interpretable machine learning. arXiv preprint arXiv:1702.08608, 2017

  17. [17]

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

  18. [18]

    Ferrari Dacrema, F

    M. Ferrari Dacrema, F. Moroni, R. Nembrini, N. Ferro, G. Faggioli, and P. Cremonesi. Towards feature selection for ranking and classification exploiting quantum anneal- ers. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2814–2824, 2022

  19. [19]

    J. B. Ghosh. Computational aspects of the maximum di- versity problem. Operations research letters, 19(4):175– 181, 1996

  20. [20]

    Goodfellow, J

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, et al. Gener- ative adversarial nets. In Advances in Neural Information Processing Systems, volume 27, 2014

  21. [21]

    X.-F. Han, H. Laga, and M. Bennamoun. Image-based 3d object reconstruction: State-of-the-art and trends in the deep learning era. IEEE transactions on pattern analysis and machine intelligence, 43(5):1578–1604, 2019

  22. [22]

    K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learn- ing for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

  23. [23]

    Hellstern, V

    G. Hellstern, V. Dehn, and M. Zaefferer. Quantum com- puter based feature selection in machine learning. IET Quantum Communication, 5(3):232–252, 2024

  24. [24]

    J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020

  25. [25]

    Jovi´ c, K

    A. Jovi´ c, K. Brki´ c, and N. Bogunovi´ c. A re- view of feature selection methods with applica- tions. In 2015 38th international convention on information and communication technology, electronics and microelectronics (MIPRO), pages 1200–1205. Ieee, 2015

  26. [26]

    A. Khan, A. Sohail, U. Zahoora, and A. S. Qureshi. A survey of the recent architectures of deep convolu- tional neural networks. Artificial Intelligence Review, 53(8):5455–5516, 2020

  27. [27]

    D. P. Kingma and J. Ba. Adam: A method for stochas- tic optimization. International Conference on Learning Representations, 2015

  28. [28]

    Krizhevsky, I

    A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, volume 25, 2012

  29. [29]

    C.-C. Kuo, F. Glover, and K. S. Dhir. Analyzing and modeling the maximum diversity problem by zero-one programming. Decision Sciences, 24(6):1171–1185, 1993

  30. [30]

    LeCun and Y

    Y. LeCun and Y. Bengio. Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks, 1998

  31. [31]

    S. M. Lundberg and S.-I. Lee. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems, volume 30, pages 4765– 4774, 2017

  32. [32]

    Mart´ ı, M

    R. Mart´ ı, M. Gallego, A. Duarte, and E. G. Pardo. Heuristics and metaheuristics for the maximum diversity problem. Journal of Heuristics, 19(4):591–615, 2013

  33. [33]

    M¨ ucke, R

    S. M¨ ucke, R. Heese, S. M¨ uller, M. Wolter, and N. Pi- atkowski. Feature selection on quantum computers. Quantum Machine Intelligence, 5(1):11, 2023

  34. [34]

    M. H. M. Noor. A survey on state-of-the-art deep learning applications and challenges. arXiv preprint arXiv:2403.17561, 2024

  35. [35]

    Panchenko

    D. Panchenko. The sherrington-kirkpatrick model. 15 Springer Science & Business Media, 2013

  36. [36]

    Paszke, S

    A. Paszke, S. Gross, F. Massa, et al. Pytorch: An im- perative style, high-performance deep learning library. In Advances in Neural Information Processing Systems, vol- ume 32, 2019

  37. [37]

    B. K. Patra, R. Launonen, V. Ollikainen, and S. Nandi. A new similarity measure using bhattacharyya co- efficient for collaborative filtering in sparse data. Knowledge-Based Systems, 82:163–177, 2015

  38. [38]

    Qamar and N

    T. Qamar and N. Z. Bawany. Understanding the black- box: towards interpretable and reliable deep learning models. PeerJ Computer Science, 9:e1629, 2023

  39. [39]

    Rajak, S

    A. Rajak, S. Suzuki, A. Dutta, and B. K. Chakrabarti. Quantum annealing: An overview. Philosophical Transactions of the Royal Society A, 381(2241):20210417, 2023

  40. [40]

    J. C. Rangel et al. A survey on convolutional neural net- works and their performance limitations in image recog- nition tasks. Journal of Sensors, 2024:2797320, 2024

  41. [41]

    M. E. Rayed, S. S. Islam, S. I. Niha, J. R. Jim, M. M. Kabir, and M. Mridha. Deep learning for medical image segmentation: State-of-the-art advancements and chal- lenges. Informatics in medicine unlocked, 47:101504, 2024

  42. [42]

    M. T. Ribeiro, S. Singh, and C. Guestrin. ” why should i trust you?” explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1135–1144, 2016

  43. [43]

    Rombach, A

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022

  44. [44]

    Romero, S

    S. Romero, S. Gupta, V. Gatlin, R. S. Chapkin, and J. J. Cai. Quantum annealing for enhanced feature selec- tion in single-cell rna sequencing data analysis. Quantum Machine Intelligence, 7(2):114, 2025

  45. [45]

    Ronneberger, P

    O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmen- tation. In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer, 2015

  46. [46]

    Russakovsky, J

    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3):211–252, 2015

  47. [47]

    A. M. Salih, Z. Raisi-Estabragh, I. B. Galazzo, P. Radeva, S. E. Petersen, K. Lekadir, and G. Menegaz. A perspec- tive on explainable artificial intelligence methods: Shap and lime. Advanced Intelligent Systems, 7(1):2400304, 2025

  48. [48]

    R. R. Selvaraju, M. Cogswell, A. Das, et al. Grad- cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision, pages 618–626, 2017

  49. [49]

    Very Deep Convolutional Networks for Large-Scale Image Recognition

    K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition.arXiv preprint arXiv:1409.1556, 2014

  50. [50]

    Sohl-Dickstein, E

    J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using nonequi- librium thermodynamics. In International conference on machine learning, pages 2256–2265. pmlr, 2015

  51. [51]

    Trigka, E

    M. Trigka, E. Dritsas, and N. D. Lagaros. A comprehen- sive survey of deep learning approaches in image process- ing. Sensors, 25(2):531, 2025

  52. [52]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30, pages 5998– 6008, 2017

  53. [53]

    Virtanen, R

    P. Virtanen, R. Gommers, T. E. Oliphant, M. Haber- land, T. Reddy, D. Cournapeau, E. Burovski, P. Pe- terson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, ˙I. Po- lat, Y. Feng, E. W. Moore, J. VanderPlas, D. Laxalde, J. Perktold, R. Cimrman, I. Henr...

  54. [54]

    Vonder Haar, T

    L. Vonder Haar, T. Elvira, and O. Ochoa. An analysis of explainability methods for convolutional neural net- works. Engineering Applications of Artificial Intelligence, 117:105606, 2023

  55. [55]

    J. Yoon, J. Jordon, and M. van der Schaar. Invase: Instance-wise variable selection using neural networks. In International Conference on Learning Representations, 2019

  56. [56]

    Zhang, Y

    Q. Zhang, Y. N. Wu, and S.-C. Zhu. Interpretable con- volutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8827–8836, 2018

  57. [57]

    ˇZnidariˇ c

    M. ˇZnidariˇ c. Scaling of the running time of the quan- tum adiabatic algorithm for propositional satisfiability. Physical Review A—Atomic, Molecular, and Optical Physics, 71(6):062305, 2005