Explainable bank failure prediction models: Counterfactual explanations to reduce the failure risk
Pith reviewed 2026-05-23 22:51 UTC · model grok-4.3
The pith
Nearest Instance Counterfactual Explanation, especially with cost-sensitive balancing, produces the highest quality suggestions for reducing bank failure risk.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The Nearest Instance Counterfactual Explanation method yields higher quality counterfactual explanations, mainly using the cost sensitive approach. The Multi Objective Counterfactual and Nearest Instance Counterfactual Explanation methods outperform others regarding validity, proximity, and sparsity metrics, with the cost sensitive approach providing the most desirable counterfactual explanations.
What carries the argument
Counterfactual generation methods (WhatIf, Multi Objective Counterfactual Explanation, Nearest Instance Counterfactual Explanation) evaluated on ML models for bank failure prediction, scored on validity, proximity, sparsity, and plausibility across balancing strategies.
If this is right
- Bank managers can receive minimal, actionable changes to input features to reverse a failure prediction from black-box models.
- Cost-sensitive resampling should be preferred when generating counterfactuals for imbalanced failure datasets.
- Performance of counterfactual methods varies by balancing strategy, so the choice of method must be validated per dataset.
- Combining these explanations with logistic regression or other interpretable models could further improve regulatory acceptance.
Where Pith is reading between the lines
- If the counterfactuals prove plausible in practice, they could be embedded in regulatory early-warning systems to trigger targeted interventions.
- Extending the evaluation to non-US bank datasets would test whether the superiority of Nearest Instance and cost-sensitive methods holds across different regulatory environments.
- Integrating these methods into live monitoring dashboards could allow banks to simulate 'what-if' scenarios before making operational changes.
Load-bearing premise
The suggested changes in the counterfactuals are feasible for real banks to implement and the models capture genuine drivers of failure rather than dataset artifacts.
What would settle it
A study that tracks actual banks following the suggested feature changes and measures whether their real failure rates drop would falsify the claim if no reduction occurs.
Figures
read the original abstract
The accuracy and understandability of bank failure prediction models are crucial. While interpretable models like logistic regression are favored for their explainability, complex models such as random forest, support vector machines, and deep learning offer higher predictive performance but lower explainability. These models, known as black boxes, make it difficult to derive actionable insights. To address this challenge, using counterfactual explanations is suggested. These explanations demonstrate how changes in input variables can alter the model output and suggest ways to mitigate bank failure risk. The key challenge lies in selecting the most effective method for generating useful counterfactuals, which should demonstrate validity, proximity, sparsity, and plausibility. The paper evaluates several counterfactual generation methods: WhatIf, Multi Objective, and Nearest Instance Counterfactual Explanation, and also explores resampling methods like undersampling, oversampling, SMOTE, and the cost sensitive approach to address data imbalance in bank failure prediction in the US. The results indicate that the Nearest Instance Counterfactual Explanation method yields higher quality counterfactual explanations, mainly using the cost sensitive approach. Overall, the Multi Objective Counterfactual and Nearest Instance Counterfactual Explanation methods outperform others regarding validity, proximity, and sparsity metrics, with the cost sensitive approach providing the most desirable counterfactual explanations. These findings highlight the variability in the performance of counterfactual generation methods across different balancing strategies and machine learning models, offering valuable strategies to enhance the utility of black box bank failure prediction models.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper evaluates several counterfactual explanation methods (WhatIf, Multi Objective Counterfactual, Nearest Instance Counterfactual Explanation) paired with imbalance-handling strategies (undersampling, oversampling, SMOTE, cost-sensitive) for post-hoc explanation of black-box bank failure prediction models (random forest, SVM, deep learning) trained on US bank data. It claims that the Nearest Instance and Multi Objective methods, especially under the cost-sensitive approach, produce superior counterfactuals according to validity, proximity, and sparsity metrics and thereby offer actionable strategies to mitigate bank failure risk.
Significance. If the comparative results hold after addressing the gaps below, the work supplies concrete, method-level guidance on generating explanations from high-accuracy but opaque models in a high-stakes financial domain where regulatory interpretability requirements are strict. The explicit inclusion of multiple balancing techniques and the focus on an applied prediction task distinguish it from purely methodological counterfactual papers.
major comments (2)
- [Abstract] Abstract: The abstract asserts that useful counterfactuals must satisfy four properties—validity, proximity, sparsity, and plausibility—yet only the first three are quantified in the reported experiments. No quantitative or qualitative evaluation of plausibility (feasibility of suggested changes to capital ratios, asset composition, etc., under regulatory or economic constraints) is provided, directly weakening the central claim that the generated explanations can reduce failure risk.
- [Results / Experimental Setup] §4 (or equivalent results section): The evaluation omits any comparison against inherently interpretable baselines such as logistic regression or decision trees, even though the introduction notes that these models are already favored for explainability. Without this baseline, it is unclear whether the added complexity of black-box models plus counterfactual generation is necessary or superior to simpler alternatives that require no post-hoc explanation.
minor comments (2)
- [Experimental Setup] The manuscript does not report dataset size, number of features, class imbalance ratio, or cross-validation scheme, making it difficult to assess the stability of the validity/proximity/sparsity numbers.
- [Results] No statistical significance tests or error bars are shown for the metric comparisons across methods and balancing strategies.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback on our manuscript. We address each major comment below, indicating planned revisions where appropriate to strengthen the work.
read point-by-point responses
-
Referee: [Abstract] Abstract: The abstract asserts that useful counterfactuals must satisfy four properties—validity, proximity, sparsity, and plausibility—yet only the first three are quantified in the reported experiments. No quantitative or qualitative evaluation of plausibility (feasibility of suggested changes to capital ratios, asset composition, etc., under regulatory or economic constraints) is provided, directly weakening the central claim that the generated explanations can reduce failure risk.
Authors: We agree that plausibility is a key property for actionable counterfactuals in the bank failure domain and that its omission from the quantitative evaluation represents a limitation. The abstract introduces four properties as desiderata for useful counterfactuals, but the experiments focus on the three that can be directly measured via standard metrics. We will revise the abstract to explicitly note the three evaluated metrics and add a new subsection discussing plausibility qualitatively, drawing on domain knowledge of US bank regulatory constraints (e.g., capital ratio thresholds and asset composition feasibility). This will better support the risk-mitigation claims without overclaiming quantitative results. revision: yes
-
Referee: [Results / Experimental Setup] §4 (or equivalent results section): The evaluation omits any comparison against inherently interpretable baselines such as logistic regression or decision trees, even though the introduction notes that these models are already favored for explainability. Without this baseline, it is unclear whether the added complexity of black-box models plus counterfactual generation is necessary or superior to simpler alternatives that require no post-hoc explanation.
Authors: The manuscript centers on post-hoc counterfactual explanations for black-box models that achieve higher predictive performance on imbalanced bank failure data, as noted in the introduction. We acknowledge that including interpretable baselines would help contextualize the value of the black-box + counterfactual approach. We will add a comparison in the results section against logistic regression and decision trees, reporting both predictive metrics (to show performance trade-offs) and inherent interpretability (via feature importance or rules). This will clarify when the added complexity is justified. revision: yes
Circularity Check
No circularity: empirical benchmark of off-the-shelf methods
full rationale
The paper conducts a standard empirical comparison of existing counterfactual generation algorithms (WhatIf, Multi Objective, Nearest Instance Counterfactual Explanation) applied to bank-failure classifiers, using public US bank data and standard balancing techniques. Reported results are direct measurements of validity, proximity, and sparsity on held-out instances; no equations, fitted parameters, or self-citations are invoked to derive the performance numbers. The central claims rest on observable differences across methods rather than any reduction of outputs to author-defined inputs or prior self-work. Plausibility is listed as a desideratum but its absence from the evaluation is a completeness issue, not a circularity in the reported metrics.
Axiom & Free-Parameter Ledger
free parameters (2)
- Hyperparameters of counterfactual generators and base ML models
- Parameters of resampling methods (SMOTE ratio, cost matrix weights)
axioms (2)
- domain assumption Counterfactual explanations can be produced that simultaneously satisfy validity, proximity, sparsity, and plausibility on bank financial data.
- domain assumption The chosen ML models (RF, SVM, DL) achieve higher predictive performance than logistic regression on the US bank failure task.
Lean theorems connected to this paper
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
The results indicate that the Nearest Instance Counterfactual Explanation method yields higher quality counterfactual explanations, mainly using the cost sensitive approach... validity, proximity, and sparsity metrics, with the cost sensitive approach providing the most desirable counterfactual explanations.
-
IndisputableMonolith/Foundation/BranchSelection.leanbranch_selection unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
Plausibility refers to the extent to which a CE is logical and realistic... Plausibility assesses the logical coherence and realism of a CE x′.
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
-
[1]
Krist´ of, T. & Vir´ ag, M. EU-27 bank failure prediction with C5. 0 decision trees and deep learning neural networks. Research In International Business And Finance . 61 pp. 101644 (2022)
work page 2022
-
[2]
Le, H. & Viviani, J. Predicting bank failure: An improvement by implementing a machine-learning approach to classical financial ratios. Research In International Business And Finance. 44 pp. 16-25 (2018)
work page 2018
-
[3]
Gogas, P., Papadimitriou, T. & Agrapetidou, A. Forecasting bank failures and stress testing: A machine learning approach. International Journal Of Forecasting. 34, 440-455 (2018)
work page 2018
-
[4]
Carmona, P., Climent, F. & Momparler, A. Predicting failure in the US bank- ing sector: An extreme gradient boosting approach. International Review Of Eco- nomics And Finance. 61 pp. 304-323 (2019)
work page 2019
-
[5]
Petropoulos, A., Siakoulis, V., Stavroulakis, E. & Vlachogiannakis, N. Predict- ing bank insolvencies using machine learning techniques. International Journal Of Forecasting. 36, 1092-1113 (2020)
work page 2020
-
[6]
Manthoulis, G., Doumpos, M., Zopounidis, C., Galariotis, E. & Baourakis, G. Bank failure prediction: a comparison of machine learning approaches. Financial Risk Management And Modeling . pp. 349-366 (2021)
work page 2021
-
[7]
Arrieta, A., Diaz-Rodriguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R. Others Explainable Artifi- cial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion. 58 pp. 82-115 (2020) 18 Gunonu et al
work page 2020
-
[8]
Samek, W. & M¨ uller, K. Towards explainable artificial intelligence. Explainable AI: Interpreting, Explaining And Visualizing Deep Learning . pp. 5-22 (2019)
work page 2019
-
[9]
Momparler, A., Carmona, P. & Climent, F. Banking failure prediction: a boosting classification tree approach. Spanish Journal Of Finance And Accounting/Revista Espa˜ nola De Financiaci´ on Y Contabilidad. 45, 63-91 (2016)
work page 2016
-
[10]
Park, M., Son, H., Hyun, C. & Hwang, H. Explainability of machine learning models for bankruptcy prediction. Ieee Access. 9 pp. 124887-124899 (2021)
work page 2021
- [11]
-
[12]
Liu, J., Li, C., Ouyang, P., Liu, J. & Wu, C. Interpreting the prediction results of the tree-based gradient boosting models for financial distress prediction with an explainable machine learning approach. Journal Of Forecasting. 42, 1112-1137 (2023)
work page 2023
-
[13]
Karimi, A., Barthe, G., Balle, B. & Valera, I. Model-agnostic counterfactual ex- planations for consequential decisions. International Conference On Artificial In- telligence And Statistics . pp. 895-905 (2020)
work page 2020
- [14]
-
[15]
Tanyel, T., Ayvaz, S. & Keserci, B. Beyond known reality: Exploiting counterfac- tual explanations for medical research. ArXiv Preprint arXiv:2307.02131 . (2023)
-
[16]
Tsiakmaki, M. & Ragos, O. A Case Study of Interpretable Counterfactual Ex- planations for the Task of Predicting Student Academic Performance. 2021 25th International Conference On Circuits, Systems, Communications And Computers (CSCC). pp. 120-125 (2021)
work page 2021
- [17]
-
[18]
Interpretable Credit Application Predictions With Counterfactual Explanations
Grath, R., Costabello, L., Van, C., Sweeney, P., Kamiab, F., Shen, Z. & Lecue, F. Interpretable credit application predictions with counterfactual explanations. ArXiv Preprint arXiv:1811.05245 . (2018)
work page internal anchor Pith review Pith/arXiv arXiv 2018
-
[19]
Dastile, X., Celik, T. & Vandierendonck, H. Model-agnostic counterfactual expla- nations in credit scoring. IEEE Access. 10 pp. 69543-69554 (2022)
work page 2022
- [20]
- [21]
-
[22]
An evaluation study on counterfactual expla- nations of bank failure prediction models
Gunonu, S., Altun, G., & Cavus, M. An evaluation study on counterfactual expla- nations of bank failure prediction models. Proceedings of the V. International Ap- plied Statistics Congress, Istanbul, Turkiye, May 21-23, 2024. pp. 383–388 (2024)
work page 2024
-
[23]
Wexler, J., Pushkarna, M., Bolukbasi, T., Wattenberg, M., Vi´ egas, F. & Wilson, J. The what-if tool: Interactive probing of machine learning models. IEEE Trans- actions On Visualization And Computer Graphics . 26, 56-65 (2019)
work page 2019
-
[24]
Brughmans, D., Leyman, P. & Martens, D. Nice: an algorithm for nearest in- stance counterfactual explanations. Data Mining And Knowledge Discovery . pp. 1-39 (2023)
work page 2023
-
[25]
Dandl, S., Molnar, C., Binder, M. & Bischl, B. Multi-objective counterfactual explanations. International Conference On Parallel Problem Solving From Nature . pp. 448-469 (2020) Explainable bank failure prediction models 19
work page 2020
-
[26]
Counterfactuals in explainable artificial intelligence (XAI): Evidence from human reasoning
Byrne, R. Counterfactuals in explainable artificial intelligence (XAI): Evidence from human reasoning.. IJCAI. pp. 6276-6282 (2019)
work page 2019
-
[27]
Artelt, A. & Hammer, B. On the computation of counterfactual explanations–A survey. ArXiv Preprint arXiv:1911.07749 . (2019)
-
[28]
Keane, M. & Smyth, B. Good counterfactuals and where to find them: A case-based technique for generating counterfactuals for explainable AI (XAI).Case-Based Rea- soning Research And Development: 28th International Conference, ICCBR 2020, Salamanca, Spain, June 8–12, 2020, Proceedings 28 . pp. 163-178 (2020)
work page 2020
-
[29]
Smyth, B. & Keane, M. A few good counterfactuals: generating interpretable, plausible and diverse counterfactual explanations. International Conference On Case-Based Reasoning. pp. 18-32 (2022)
work page 2022
-
[30]
Artelt, A., Vaquet, V., Velioglu, R., Hinder, F., Brinkrolf, J., Schilling, M. & Ham- mer, B. Evaluating the robustness of counterfactual explanations.2021 IEEE Sym- posium Series On Computational Intelligence (SSCI) . pp. 01-09 (2021)
work page 2021
-
[31]
Dandl, S., Hofheinz, A., Binder, M., Bischl, B. & Casalicchio, G. Counterfac- tuals: an R package for counterfactual explanation methods. ArXiv Preprint arXiv:2304.06569. (2023)
-
[32]
Hvilshøj, F., Iosifidis, A. & Assent, I. On quantitative evaluations of counterfactu- als. ArXiv Preprint arXiv:2111.00177 . (2021)
-
[33]
Ge, Y., Liu, S., Li, Z., Xu, S., Geng, S., Li, Y., Tan, J., Sun, F. & Zhang, Y. Counterfactual evaluation for explainable AI. ArXiv Preprint arXiv:2109.01962 . (2021)
-
[34]
Du Jardin, P. & S´ everin, E. Predicting corporate bankruptcy using a self-organizing map: An empirical study to improve the forecasting horizon of a financial failure model. Decision Support Systems. 51, 701-711 (2011)
work page 2011
-
[35]
Manthoulis, G., Doumpos, M., Zopounidis, C. & Galariotis, E. An ordinal classifi- cation framework for bank failure prediction: Methodology and empirical evidence for US banks. European Journal Of Operational Research. 282, 786-801 (2020)
work page 2020
-
[36]
Audrino, F., Kostrov, A. & Ortega, J. Predicting US bank failures with MI- DAS logit models. Journal Of Financial And Quantitative Analysis . 54, 2575-2603 (2019)
work page 2019
-
[37]
Bank failure prediction models addressing imbalanced data and out-of-time performance
Gunonu, S., Altun, G., & Cavus, M. Bank failure prediction models addressing imbalanced data and out-of-time performance. Proceedings of the 7th International Researchers, Statisticians, and Young Statisticians Congress, Istanbul, Turkiye, November 2-5, 2023. pp. 185–199 (2023)
work page 2023
-
[38]
Chawla, N., Bowyer, K., Hall, L. & Kegelmeyer, W. SMOTE: synthetic minority over-sampling technique. Journal Of Artificial Intelligence Research . 16 pp. 321- 357 (2002)
work page 2002
-
[39]
Goorbergh, R., Smeden, M., Timmerman, D. & Van Calster, B. The harm of class imbalance corrections for risk prediction models: illustration and simulation using logistic regression. Journal Of The American Medical Informatics Association . 29, 1525-1534 (2022)
work page 2022
-
[40]
Junior, J. & Pisani, P. Performance and model complexity on imbalanced datasets using resampling and cost-sensitive algorithms. Fourth International Workshop On Learning With Imbalanced Domains: Theory And Applications . pp. 83-97 (2022)
work page 2022
-
[41]
Stando, A., Cavus, M. & Biecek, P. The effect of balancing methods on model behavior in imbalanced classification problems. Fifth International Workshop On Learning With Imbalanced Domains: Theory And Applications . pp. 16-30 (2024)
work page 2024
-
[42]
An Experimental Study on the Rashomon Effect of Balancing Methods in Imbalanced Classification
Cavus, M. & Biecek, P. An Experimental Study on the Rashomon Effect of Bal- ancing Methods in Imbalanced Classification. ArXiv Preprint arXiv:2405.01557 . (2024) 20 Gunonu et al
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[43]
Carriero, A., Luijken, K., Hond, A., Moons, K., Calster, B. & Smeden, M. The harms of class imbalance corrections for machine learning based prediction models: a simulation study. ArXiv Preprint arXiv:2404.19494 . (2024)
-
[44]
Elor, Y. & Averbuch-Elor, H. To SMOTE, or not to SMOTE? ArXiv Preprint arXiv:2201.08528. (2022)
-
[45]
Grinsztajn, L., Oyallon, E. & Varoquaux, G. Why do tree-based models still out- perform deep learning on typical tabular data? Advances In Neural Information Processing Systems. 35 pp. 507-520 (2022)
work page 2022
-
[46]
Breiman, L., Friedman, J., Olshen, R. & Stone, C. Classification and Regression Trees. (Wadsworth,1984)
work page 1984
- [47]
-
[48]
Geurts, P., Ernst, D. & Wehenkel, L. Extremely randomized trees. Machine Learn- ing. 63 pp. 3-42 (2006)
work page 2006
-
[49]
Counterfactual explanations and how to find them: literature review and benchmarking
Guidotti, R. Counterfactual explanations and how to find them: literature review and benchmarking. Data Mining And Knowledge Discovery . pp. 1-55 (2022)
work page 2022
-
[50]
Verma, S., Dickerson, J. & Hines, K. Counterfactual explanations for machine learning: A review. ArXiv Preprint arXiv:2010.10596 . 2 pp. 1 (2020)
- [51]
-
[52]
Wachter, S., Mittelstadt, B. & Russell, C. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harv. JL And Tech.. 31 pp. 841 (2017)
work page 2017
-
[53]
Dar, U. & Pillmore, B. fdicdata: Accessing FDIC Bank Data. (2023), https://CRAN.R-project.org/package=fdicdata, R package version 0.1.0
work page 2023
-
[54]
Lipton, Z., Elkan, C. & Naryanaswamy, B. Optimal thresholding of classifiers to maximize F1 measure. Machine Learning And Knowledge Discovery In Databases: European Conference, ECML PKDD 2014, Nancy, France, September 15-19, 2014. Proceedings, Part II 14. pp. 225-239 (2014)
work page 2014
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.