REVIEW 3 major objections 4 minor 15 references
Explainable Artificial Intelligence Credit Risk Assessment using Machine Learning
T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read This paper claims that LightGBM, explained by SHAP and LIME, gives the best credit-risk accuracy and approval-default trade-off among three ensemble models.
desk verdict LightGBM's 'most business-optimal' claim is contradicted by the paper's own Table 3—lowest recall, 85.7% false-negative rate—so the main conclusion does not follow from the reported metrics. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the comparative evaluation pipeline rather than a single new identity: three ensemble classifiers trained on the same engineered financial features, with class imbalance corrected by SMOTE and hyperparameters selected by grid-search cross-validation, are scored by accuracy, precision, recall, and ROC-AUC and then translated into business metrics (approval rate, default rate, false-positive rate, false-negative rate). SHAP and LIME are the explainability layer, producing global feature rankings and local per-applicant reasons, while a rule-based risk engine converts model probabilities into Low, Moderate, and High risk bands mapped to Approve, Review, or Reject decisions.
What would settle it
Re-running the pipeline on a time-based or applicant-ID-disjoint split and recomputing ROC-AUC, approval rate, and default rate for all three models would settle the claim: if LightGBM no longer holds the highest ROC-AUC or the best default-versus-approval trade-off, the central conclusion fails.
Extended reading notes
Core claim
The paper's central claim is that, after identical preprocessing, SMOTE balancing, grid-search hyperparameter tuning, and feature engineering, LightGBM dominates the other two ensemble models on the measured criteria: it reaches the highest accuracy (90.07%), the highest ROC-AUC (0.7203), and the best business metrics, sanctioning more loans while keeping the default share among approved loans lower than XGBoost or Random Forest. XGBoost comes close on accuracy (88.74%) but its approved loans carry a higher default risk, and Random Forest, despite the highest recall (0.3040), has the lowest accuracy (82.03%) and the worst business outcome. The authors further assert that SHAP feature rankings show all three models depend mainly on external credit-report variables, and that LIME explanations make each applicant's risk factors inspectable by lenders and borrowers.
Load-bearing premise
The reported ranking assumes the test set was genuinely unseen, with applicant-level data merged by ID strictly before the train/test split; the paper does not specify the split procedure, and any leakage across that boundary would make the accuracy and ROC-AUC figures optimistic.
Editorial extensions
If this is right
- If LightGBM is the best model, lenders using this pipeline should expect more approved applicants than with XGBoost or Random Forest and fewer defaults among those approved loans.
- The SHAP rankings imply that external credit-source data carries most of the predictive signal, so data quality on those bureau variables is more valuable than fine-tuning demographic features.
- Because XGBoost's accuracy is close but its default rate among approved loans is higher, accuracy alone is not a safe model-selection criterion for credit decisions.
- Per-applicant LIME and SHAP reports supply a concrete artifact for regulatory review and customer dispute, making automated decisions auditable.
Reading between the lines
- The paper does not explore how the Approve, Review, and Reject thresholds affect the LightGBM ranking; the reported comparison is at one operating point, and shifting thresholds could reorder the models.
- All three models show low precision at the chosen threshold, meaning most applicants flagged as high-risk are actually non-defaulters; a lender-aware threshold tuned to the relative cost of false rejections versus false approvals could change the business comparison.
- A direct follow-up test would be to audit the train/test split for applicant-ID leakage; if the split is clean and LightGBM still wins, the claim generalizes, and if not, the reported metrics are optimistic.
- The dominance of external credit-source features suggests richer bureau data may help all three models; testing each model with and without those features would show how much of the advantage is data-driven rather than algorithm-driven.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper describes a credit-risk-assessment pipeline that preprocesses loan-application data, trains XGBoost, LightGBM, and Random Forest models with SMOTE and GridSearchCV, and uses SHAP and LIME to generate explanatory reports. The central claim, stated in the abstract and reiterated in Section 4D, is that LightGBM is the most business-optimal model, achieving the highest accuracy and the best trade-off between approval and default rates. The empirical section reports Table 3: LightGBM accuracy 90.07%, precision 0.2757, recall 0.1434, ROC AUC 0.7203; XGBoost accuracy 88.74%, precision 0.2199, recall 0.1566, ROC AUC 0.7033; Random Forest accuracy 82.03%, precision 0.1652, recall 0.3040, ROC AUC 0.6808.
Significance. A valid, reproducible comparison of explainable ensemble models for credit risk would be of practical interest, and the paper's use of a multi-source feature set with SHAP and LIME reports is a sensible design direction. However, the manuscript as written does not provide enough experimental detail to support its conclusions: the data are unnamed and the train/test split is unspecified, no uncertainty quantification is reported, and the headline conclusion is contradicted by the paper's own Table 3. The contribution is therefore currently at the level of a descriptive case study rather than a verified scientific claim.
major comments (3)
- [Section 4C/4D and Table 3] The central claim that LightGBM is the most business-optimal model is contradicted by the paper's own reported metrics. Table 3 gives LightGBM a recall of 0.1434, implying a false-negative rate of 85.7% on defaulters, the highest among the three models; Section 4C asserts that LightGBM is effective at minimizing false negatives, and Section 4D concludes it has the best default/approval trade-off. These statements cannot both hold at the same decision threshold, so the headline conclusion is not derivable from the reported results.
- [Section 3E and Section 5] The claimed approval/default trade-off is never operationalized. Section 3E states that thresholds are set 'as per user requirements', but no numerical thresholds, no cost matrix, no approval rates, and no default rates among approved loans are provided. Section 5 states that LightGBM achieves the 'highest approval rate of 95%', but no corresponding default rate is reported, making it impossible to verify the claimed business optimality even under the authors' best-case evaluation assumptions.
- [Proposed Methodology (Data and Evaluation)] The paper does not name the dataset, does not describe the train/test split, and does not report error bars or confidence intervals for any metric. The merging procedure described in Section 3A (aggregating multiple tables by applicant ID) could in principle introduce leakage if training and test rows are drawn from overlapping applicants, but the manuscript gives no information to rule this out. Consequently, the reported accuracy, precision, recall, and ROC AUC cannot be assessed for reliability or generalizability.
minor comments (4)
- [Various] There are numerous typographical errors, including 'LighGBM' in the caption of Figure 2, 'customed' in Section 3A, and 'pf' instead of 'of' in Section 4D; the paper would benefit from a careful proofreading pass.
- [Section 4D] The text says Random Forest 'comes with lots pf negative factors' and 'underperforms hence is unreliable', but Table 3 shows Random Forest has the highest recall (0.3040), which the paper itself acknowledges in Section 4B; the discussion should reconcile this rather than dismiss Random Forest without qualification.
- [Section 2] The related-work references are incomplete in places (for example, reference [4] has 'Kousarziya' as an author and an incomplete name) and the mapping between the numbered citations and the Table 1 summary is not always clear.
- [Section 5] The claim that report generation is 'the most remarkable novelty of the system' is overstated, as automated XAI report generation is already common in credit-scoring tools; this framing should be toned down.
Circularity Check
No significant circularity: the paper is an empirical model comparison whose claims are not derived from fitted parameters, self-citations, or definitional equivalences.
full rationale
This paper is an empirical machine-learning comparison, not a derivation chain, so the circularity patterns enumerated do not apply. The central claim that LightGBM is 'most business-optimal' is asserted from reported evaluation metrics and business-impact language, but no equation or fitted parameter is reused as a prediction. Hyperparameter tuning with GridSearchCV on ROC-AUC and then reporting ROC-AUC is standard model selection, not a fitted input renamed as a prediction. The paper cites existing literature and tool documentation (SHAP, DNV), but none of these citations is load-bearing in a way that reduces the paper's own result to the cited work; there is no self-citation chain, no imported uniqueness theorem, and no ansatz smuggled in via citation. The internal inconsistency between Table 3 (LightGBM recall 0.1434, i.e., FNR of 85.7%) and the Section 4C claim that LightGBM minimizes false negatives is a correctness and evidentiary problem, not a circularity problem: the conclusion is unsupported by the reported numbers, but it is not equivalent to the inputs by construction. Similarly, the absence of approval-rate and default-rate tables means the 'best trade-off' claim is unverifiable, but that is a missing-evidence issue rather than circular reasoning. Under the stated hard rules, no circular step can be exhibited with a quotation showing Eq. X = Eq. Y by construction or a fitted parameter renamed as prediction, so the honest finding is no significant circularity (score 0).
Assumptions & free parameters
free parameters (4)
- SMOTE oversampling ratio =
not specified
- Outlier clipping threshold =
3 standard deviations
- Risk category thresholds =
not specified
- Hyperparameters (XGBoost, LightGBM, RF) =
not specified
assumptions (3)
- domain assumption The train/test split is leakage-free and the test set is representative.
- domain assumption The metrics (accuracy, ROC AUC) are computed on a held-out test set.
- ad hoc to paper Clipping numerical values at 3 standard deviations retains the signal relevant to default prediction.
Cite this review
Pith. "Pith review of Explainable Artificial Intelligence Credit Risk Assessment using Machine Learning." pith.science (2026). https://pith.science/paper/GYZAXAQP
@misc{pith2026250619383,
author = {Pith},
title = {Pith review of: Explainable Artificial Intelligence Credit Risk Assessment using Machine Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/GYZAXAQP}},
note = {Machine review of arXiv:2506.19383}
}
read the original abstract
This paper presents an intelligent and transparent AI-driven system for Credit Risk Assessment using three state-of-the-art ensemble machine learning models combined with Explainable AI (XAI) techniques. The system leverages XGBoost, LightGBM, and Random Forest algorithms for predictive analysis of loan default risks, addressing the challenges of model interpretability using SHAP and LIME. Preprocessing steps include custom imputation, one-hot encoding, and standardization. Class imbalance is managed using SMOTE, and hyperparameter tuning is performed with GridSearchCV. The model is evaluated on multiple performance metrics including ROC-AUC, precision, recall, and F1-score. LightGBM emerges as the most business-optimal model with the highest accuracy and best trade off between approval and default rates. Furthermore, the system generates applicant-specific XAI visual reports and business impact summaries to ensure transparent decision-making.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
D. A. Dura and A. M. Cazacu, “The impact of artificial intelligence on credit risk assessment and busi- ness model transformation in the financial sector,”Journal of Law and Economic Development , vol. 6, no. 1, pp. 199–203, 2024, [Online]. Available: https://ideas.repec.org/a/cbu/jrnlec/y2024v6ip199-203. html
work page 2024
-
[2]
Credit risk prediction using machine learning and deep learning: A study on credit card customers,
V . Chang, S. Sivakulasingam, H. Wang, S. T. Wong, M. A. Ganatra, and J. Luo, “Credit risk prediction using machine learning and deep learning: A study on credit card customers,”Risks, vol. 12, no. 11, p. 174, Nov. 2024, [Online]. Available: https://www.mdpi.com/2227-9091/12/11/174
work page 2024
-
[3]
KACDP: A Highly Interpretable Credit Default Prediction Model
K. Liu and J. Zhao, “Kacdp: A highly interpretable credit default prediction model,” arXiv preprint arXiv:2411.17783, Nov. 2024, [Online]. Available: https://arxiv.org/abs/2411.17783
work page Pith review arXiv 2024
-
[4]
A. Hussain, M. A. Khan, A. A. K. S, and K. Kousarziya, “Enhancing credit scoring mod- els with artificial intelligence: A comparative study of traditional methods and ai-powered techniques,” in A Modern Approach to AI – Integrating Machine Learning with Agile Prac- tices, Oct. 2024, pp. 99–107, [Online]. Available: https://www.researchgate.net/publication...
work page 2024
-
[5]
Predictive analytics in credit risk management for banks: A comprehensive re- view,
W. A. Addy, C. E. Ugochukwu, A. T. Oyewole, O. C. Ofodile, O. B. Adeoye, and C. C. Okoye, “Predictive analytics in credit risk management for banks: A comprehensive re- view,” GSC Advanced Research and Reviews , vol. 18, no. 2, pp. 434–449, Nov. 2024, [On- line]. Available: https://www.researchgate.net/publication/378672514 Predictive analytics in credit ...
-
[6]
Credit risk assessment and financial decision support using explainable artificial intelligence,
M. K. Nallakaruppan, H. Chaturvedi, V . Grover, B. Balusamy, P. Jaraut, J. Bahadur, V . P. Meena, and I. A. Hameed, “Credit risk assessment and financial decision support using explainable artificial intelligence,” Risks, vol. 12, no. 10, p. 164, Oct. 2024, [Online]. Available: https://www.mdpi.com/ 2227-9091/12/10/164. 14
work page 2024
-
[7]
A. S. Al-Afeef, “Credit risk prediction with and without weights of evidence using conventional pre- dictive statistical models,” Cogent Economics & Finance , vol. 12, no. 1, pp. 1–15, 2024, [Online]. Available: https://doi.org/10.1080/23322039.2024.2338971
-
[8]
Artificial intelligence in credit risk: A literature review,
P. Widagdo, R. A. Pratiwi, H. Nurlinda, N. Nurbaeti, R. Ismiwati, F. R. Kurniawan, and S. Yusri- ani, “Artificial intelligence in credit risk: A literature review,” in Proc. 6th Int. Seminar on Busi- ness, Economics, Social Science, and Technology (ISBEST) , vol. 3, 2023, [Online]. Available: https: //conference.ut.ac.id/index.php/isbest/article/view/1472
work page 2023
Show all 15 references
-
[9]
Estimating value-at-risk in the eurusd currency cross from implied volatilities using machine learning methods and quantile regression,
H. M. Blom, P. E. de Lange, and M. Risstad, “Estimating value-at-risk in the eurusd currency cross from implied volatilities using machine learning methods and quantile regression,”Journal of Risk and Financial Management, vol. 16, no. 7, p. 312, Jul. 2023, [Online]. Available...
2023
-
[10]
The value of open banking data for application credit scoring: Case study of a norwegian bank,
L. O. Hjelkrem, P. E. de Lange, and E. Nesset, “The value of open banking data for application credit scoring: Case study of a norwegian bank,”Journal of Risk and Financial Management, vol. 15, no. 12, p. 597, 2022, [Online]. Available: https://www.mdpi.com/1911-8074/15/12/597
2022
-
[11]
Credit risk modeling using predictive analytics,
M. Yusuff, “Credit risk modeling using predictive analytics,” Apr. 2021, [Online]. Available: https: //www.researchgate.net/publication/386887850 Credit Risk Modeling Using Predictive Analytics
2021
-
[12]
Credit risk analysis using machine and deep learning mod- els,
P. M. Addo, D. Guegan, and B. Hassani, “Credit risk analysis using machine and deep learning mod- els,” Risks, vol. 6, no. 2, p. 38, 2018, [Online]. Available: https://www.mdpi.com/2227-9091/6/2/38
2018
-
[13]
Application of artificial intelligence (artificial neural network) to assess credit risk: A predictive model for credit card scoring,
M. S. Islam, L. Zhou, and F. Li, “Application of artificial intelligence (artificial neural network) to assess credit risk: A predictive model for credit card scoring,” Master’s thesis, Blekinge Insti- tute of Technology, 2009, [Online]. Available: https://www.diva-portal.org/...
2009
-
[14]
Optimizing human-ai collaboration,
DNV , “Optimizing human-ai collaboration,” [Online]. Available: https://www.dnv.com/research/ future-of-digital-assurance/optimizing-human-ai-collaboration/
-
[15]
Shap: Beeswarm plot,
S. M. Lundberg and H. Chen, “Shap: Beeswarm plot,” [Online]. Available: https://shap.readthedocs. io/en/latest/example notebooks/api examples/plots/beeswarm.html. 15
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.