REVIEW 4 major objections 7 minor 21 references
The paper claims that TabNet, a deep network built for tabular data, detects bank transaction fraud with 97.39% accuracy and a ROC-AUC of 0.9739, considerably outperforming DNN, GRU, LSTM, and CNN1D on a balanced Indian transaction dataset.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 14:48 UTC pith:RHDCO2AC
load-bearing objection The TabNet result is an artifact of evaluation leakage: SMOTE before splitting plus in-sample final prediction invalidate the 0.9739 AUC, and the paper's internal numbers don't agree. the 4 major comments →
Prediction of bank transaction fraud using TabNet an adaptive deep learning architecture
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper's central claim is that TabNet outperforms DNN, GRU, LSTM, and CNN1D for bank transaction fraud detection on a balanced Indian dataset, achieving ROC-AUC 0.9739 and 97.39% accuracy. The authors attribute this to TabNet's sequential attention mechanism and sparse feature masks, which dynamically select relevant features at each decision step, yielding better generalization on tabular data and fewer false positives and negatives than the comparison architectures, which either collapsed to single-class predictions (DNN, CNN1D) or performed only marginally better than random (LSTM, GRU).
What carries the argument
TabNet, a deep learning architecture for tabular data whose core mechanism is a sequential attention module that learns sparse feature masks. At each decision step the model selects a subset of features to attend to, which provides built-in interpretability (the masks show which features drove the decision), regularization, and adaptation to individual transactions. This is the mechanism the paper identifies as responsible for the performance gap.
Load-bearing premise
SMOTE oversampling is applied to the entire dataset before the cross-validation split, so synthetic fraud cases generated from training-fold neighborhoods can appear inside validation folds, making the validation AUC reflect information from training data rather than genuine out-of-sample performance.
What would settle it
Run the identical TabNet pipeline but apply SMOTE only within each training fold after the split, then evaluate on untouched validation folds; if the ROC-AUC falls materially below 0.9739, the paper's headline result is driven by data leakage rather than TabNet's architecture.
If this is right
- Banks could replace rule-based fraud filters with TabNet-based scoring to reduce false positives while catching most fraud.
- The interpretability via feature masks can support audit trails and regulatory compliance under AML and KYC expectations.
- TabNet's design removes the need for extensive manual feature engineering, making deployment in real-time transaction pipelines more practical.
- Sequence-based models (LSTM, GRU) are confirmed as poor fits for static tabular transaction data, guiding architecture choice in future studies.
- The EDA insight that fraud clusters on weekdays mid-morning can be used to tune operational monitoring windows.
Where Pith is reading between the lines
- The reported performance numbers should be treated with caution because SMOTE was applied before splitting into cross-validation folds, which can leak synthetic training examples into validation folds and inflate AUC; a leakage-free re-evaluation would likely lower the TabNet advantage.
- A direct test: rerun the same pipeline with SMOTE applied inside each fold, and also report the confusion matrix on a held-out real (non-synthetic) test set, to see whether the near-perfect separation holds.
- The full-model retrain and evaluation on the same full dataset (per Algorithm 1) may additionally report training performance, so the headline accuracy could partly reflect memorization rather than generalization.
- Cross-validation count inconsistency (text says 3-fold; Table 4 says StratifiedKFold(5)) needs resolution before the result can be reproduced.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes TabNet for detecting fraudulent bank transactions from a Kaggle dataset of Indian transactions. The authors apply SMOTE to balance classes, conduct EDA, and compare TabNet against DNN, GRU, LSTM, and CNN1D using 3-fold cross-validation. The headline claim is that TabNet achieves ROC-AUC 0.9739 and accuracy 97.39%, considerably outperforming the other models, with interpretability advantages. The manuscript also discusses XAI and policy implications. The central claim depends entirely on the validity of the evaluation pipeline.
Significance. If the reported performance were valid, the study would offer a practically useful benchmark for interpretable deep learning in bank fraud detection, particularly for tabular data. The EDA visualizations and the explicit comparison of five architectures are useful starting points. However, the paper provides no code or detailed hyperparameters, and the evaluation design contains data leakage and in-sample evaluation. As a result, the main empirical claim is unsupported, and the comparison cannot be interpreted. The internal arithmetic inconsistencies further reduce confidence. The paper's contribution is therefore not established.
major comments (4)
- [Section 4.3, 4.5, Algorithm 1] SMOTE is applied to the entire dataset in Section 4.3 before the cross-validation pipeline of Section 4.5 and Algorithm 1. When oversampling is performed before splitting, synthetic examples generated from training-fold neighborhoods can land inside validation folds. This leaks information into the validation process and biases all cross-validated metrics, including the TabNet AUC and accuracy in Table 5 and Fig. 15. Correct practice is to apply SMOTE inside each fold after the split. This issue alone invalidates the reported CV performance.
- [Algorithm 1 steps 13–14] The final model is retrained on the full dataset and then evaluated by predicting on the same dataset (steps 13–14). The resulting confusion matrix (Fig. 15a) and the 'full set' metrics in Table 5 are therefore in-sample. For a flexible deep model, in-sample evaluation is substantially optimistic and cannot support the claim that TabNet generalizes better than the other architectures. An independent test set or nested CV is required.
- [Section 5.2, Table 4, Table 5] Internal inconsistencies make the reported numbers unreliable. Table 4 specifies StratifiedKFold(5) while the text and Algorithm 1 describe 3-fold cross-validation. The confusion matrix in Fig. 15a sums to 121,023 correct out of 124,496, which is 97.21%, not the reported 97.39%. Moreover, reporting 97.39% precision alongside zero false positives is arithmetically impossible: with no false positives, precision would be 100%. These discrepancies need to be resolved before the results can be assessed.
- [Section 5.2, Table 5] The baseline models are reported to perform near chance (AUC ≈ 0.5), with DNN and CNN1D collapsing to predicting a single class even after SMOTE balancing. This is suspicious and suggests possible implementation issues (e.g., missing class weighting, suboptimal learning rates, or inadequate training). The manuscript does not provide hyperparameters or training details for the baselines, so the reader cannot verify that the comparison is fair. Without such details, the claim that TabNet 'considerably outperformed' the competition is not well supported.
minor comments (7)
- [Section 3.3] The research objectives table repeats objective 1 for both rows; likely a typo.
- [Section 4.2] The phrase 'IoT-enabled environment' is mentioned but no IoT data is used; clarify or remove.
- [Section 4.3] The original dataset size before SMOTE is not explicitly summed; the text gives 62,248 and 3,287 (sum 65,535) before stating 124,496 after SMOTE. State the original total explicitly.
- [Algorithm 1] SMOTE is not included in the algorithm pseudocode despite being a key preprocessing step; add it for reproducibility.
- [Section 4.4] Reference to 'Fig. 1' for TabNet architecture is incorrect; Fig. 1 is the literature distribution chart. Use a proper architecture figure.
- [Section 5.2, Table 5] Table headers 'Full Set Accuracy (1)' and 'Full Set AUC (1)' are unclear; clarify what the '(1)' denotes.
- [Section 2.2.3] The equation for P(Fraud) appears garbled; fix the typesetting.
Circularity Check
TabNet's reported 0.9739 AUC/97.39% accuracy are in-sample or SMOTE-contaminated fits, not independent predictions, so the central outperformance claim is unsupported.
specific steps
-
fitted input called prediction
[Algorithm 1 (Section 4.5, steps 13-14); reported in Section 5.2, Table 5, Figs. 15a/15b]
"13: Retrain final model M_final on full dataset D with best parameters 14: Predict on D and export: • Predictions with probabilities • Feature importance (if available) • ROC curve and confusion matrix"
The headline TabNet accuracy (97.39%), ROC-AUC (0.9739), and confusion matrix are presented as predictive performance, but Algorithm 1 constructs them by training on the full dataset D and then 'Predict on D' — i.e., evaluating on the same data used for fitting. By construction this measures training-set fit, not out-of-sample prediction. Table 5 labels the column 'Full Set Accuracy (1)' and 'Full set AUC (1)', and Section 5.2 says evaluation used '3-fold cross-validation and final training on the full dataset.' The reported claim that TabNet 'considerably outperformed the competition' therefore rests on in-sample metrics plus CV scores that inherit the same leakage.
-
other
[Section 4.3 (Dataset preparation) combined with Section 4.5 / Algorithm 1 pipeline]
"The original dataset comprises features with the class label as “Is_fraud,” but the class labels are imbalanced, with Class 0 and Class 1 counting 62248 and 3287, respectively. Henceforth, the dataset was balanced using SMOTE to balance the class labels. The balanced dataset has a class 0 and class 1 count of 62248 and 62248, respectively."
SMOTE is applied to the full 124,496-row dataset before any train/validation split, and only then is the data 'subjected to a 3-Fold Cross-Validation Pipeline' (Section 4.5). Synthetic fraud instances generated from the neighborhoods of training-fold examples can therefore fall inside validation folds, so validation predictions are partly predictions of points constructed from training data. The reported mean CV ROC-AUC of 0.9739 is thus not an independent estimate; it is contaminated by the fitted data-generation process. The model is being 'validated' on information that came from its own training side.
full rationale
The central claim — that TabNet reaches ROC-AUC 0.9739, accuracy 97.39%, and 'considerably outperformed the competition' — depends entirely on the performance estimates in Section 5.2 and Table 5. Two construction-level problems make those estimates unfit as predictions. First, Algorithm 1 explicitly retrains the final model on the full dataset D and then 'Predict on D' to export the ROC curve and confusion matrix; the reported full-set metrics are therefore in-sample training fits, not predictions. Second, SMOTE is applied before the cross-validation split (Section 4.3), so the three-fold validation folds contain synthetic minority-class samples generated from training-fold neighborhoods. This leaks training information into the validation partition and inflates the CV AUC. Even if the Fig. 15 confusion matrix is interpreted as a pooled 3-fold CV confusion matrix rather than Algorithm 1's step-14 output, it is still built from SMOTE-contaminated validation predictions. Internal inconsistencies reinforce the unreliability: Table 4 states StratifiedKFold(5) while Section 5.2 describes 3-fold CV; the stated 97.39% accuracy exceeds 121,023/124,496 = 97.21%; and a claimed precision of 97.39% is inconsistent with zero false positives in the confusion matrix. No load-bearing self-citation chain was found; the paper cites external literature, and the issue is not author self-reference. The circularity is that the paper's own evaluation protocol turns the model's training fit into the reported 'prediction' performance, so the headline result reduces by construction to the fitted model rather than to an independent out-of-sample test.
Axiom & Free-Parameter Ledger
free parameters (5)
- SMOTE oversampling ratio / k-neighbors =
1:1 class balance; k not reported
- TabNet hyperparameters =
max_epochs=100, batch_size=1024, virtual_batch_size=128; remaining architecture unreported
- Baseline deep-model configurations =
not reported
- Classification threshold =
0.5 (implied)
- Categorical feature encoding =
label encoding mentioned; details not given
axioms (5)
- domain assumption Kaggle dataset labels are correct and the features do not leak the target
- domain assumption SMOTE-generated synthetic fraud samples are representative of real fraud
- domain assumption Balanced-data metrics transfer to the operational imbalanced setting
- domain assumption TabNet's sparse attention masks provide interpretable explanations
- domain assumption The dataset consists of actual Indian bank transactions
read the original abstract
The development of online banking has brought about an increase in fraudulent operations, which is a major problem for banks. This study delves into the urgent requirement for interpretable, scalable, and top-notch fraud detection systems by using TabNet, an adaptable deep learning framework, on a Kaggle dataset consisting of actual bank transactions in India. Maximizing operational risk management by improving the accuracy of transaction anomaly detection and ensuring regulatory compliance through transparent models is the goal. We utilize a supervised learning pipeline that incorporates the Synthetic Minority Oversampling Technique (SMOTE) to ensure that classes are balanced. Subsequently, we conduct thorough exploratory data analysis (EDA) to identify patterns of fraud, both during specific times and across behaviors. On this dataset, five different deep learning architectures are tested: DNN, GRU, LSTM, CNN1D, and TabNet. Assessment of predictive performance was carried out using a 3-fold cross-validation framework. With a ROC-AUC of 0.9739 and an accuracy of 97.39 %, TabNet considerably outperformed the competition. The method of sparse feature selection used improved interpretability, generalized better on tabular data, and produced fewer false positives and negatives. Critical insights for operational fraud detection systems and a contribution to the broader literature on explainable AI (XAI) in financial decision-making are offered by the findings. Goals 8 and 16 of the Sustainable Development Agenda are supported by this study, which promotes inclusive economic growth and institutional transparency. Supporting strong, policy-compliant, and interpretable decision-support systems, it also offers practical use for real-time implementation in banking infrastructure.
Reference graph
Works this paper leans on
-
[1]
K., AL-hattali, Z
AL-kiyumi, R. K., AL-hattali, Z. N., & Ahmed, E. R. (2021). Operational risk management and customer complaints in Omani banks. Journal of Government Information, 5 (1), 200 –
2021
-
[6]
Rehman, S. A., & Hashim, F. (2020). Impact of fraud risk assessment on good corporate governance: Case of public listed companies in Oman. Business Systems Research, 11(1), 16–30. https://doi.org/10.2478/bsrj-2020-0002 Sadgali, I., Sael, N., & Benabbou, F. (2020). Adaptive model for credit card fraud detection. International Journal of Interactive Mobile ...
arXiv 2020
-
[8]
https://doi.org/10.1155/2022/2611063 ,
-
[10]
e2088-e2088 https://doi.org/10.7717/peerj-cs.2088 . Mustafa Abdul Salam, Fouad, K. M., Elbably, D. L., & Elsayed, S. M. (2024). Federated learning model for credit card fraud detection with data balancing techniques. Neural Computing and Applications. https://doi.org/10.1007/s00521-023-09410-2 . Nandi, A. K., Randhawa, K. K., Chua, H. S., Seera, M., & Lim...
arXiv 2088
-
[14]
https://doi.org/10.1007/s44163-024-00109-4 Xiuguo, W., & Shengyong, D. (2022). An analysis on financial statement fraud detection for Chinese listed companies using deep learning. IEEE Access, 10, 22516–22532. https://doi.org/10.1109/ACCESS.2022.3153478 Xu, J.. GenAI and LLM for Financial Institutions: A Corporate Strategic Survey. https://doi.org/10.2139...
arXiv 2022
-
[18]
https://doi.org/10.3390/ electronics11050756 Alkhalili, M., Qutqut, M. H., & Almasalha, F. (2021). Investigation of applying machine learning for watch-list filtering in anti-money laundering. IEEE Access, 9 , 18481 – 18496. https://doi.org/10.1109/ACCESS.2021.3052313 Almazroi, A. A., & Ayub, N. (2023). Online payment fraud detection model using machine l...
arXiv 2021
-
[29]
https://doi.org/10.3390/ risks7010029 Madhurya, M. J., Gururaj, H. L., Soundarya, B. C., Vidyashree, K. P., & Rajendra, A. B. (2022). Exploratory analysis of credit card fraud detection using machine learning techniques. Global Transitions Proceedings, 3(1), 31–37. https://doi.org/10.1016/j.gltp.2022.04.006 Mashrur, A., Luo, W., Zaidi, N. A., & Robles-Kel...
arXiv 2022
-
[38]
http://www.ijrar.com/upload_issue/ijrar_issue_1810.pdf . Khashan, O. A. (2024). Blockchain-machine learning fusion for enhanced malicious node detection in wireless sensor networks. Knowledge-Based Systems, 304 , 112557. https://doi.org/10.1016/j.knosys.2024.112557 . B.S. Prashanth et al. International Review of Economics and Finance 106 (2026) 104916 26 ...
arXiv 2024
-
[42]
https://doi.org/10.3991/ijim.v15i05.17173 Jessica, A., Raj, F. V., & Sankaran, J. (2023). Credit card fraud detection using machine learning techniques. In ViTECoN 2023 — 2nd IEEE int. conf. vis. towar. emerg. trends commun. netw. technol. proc . IEEE. https://doi.org/10.1109/ViTECoN58111.2023.10157162 . Jiang, S., Dong, R., Wang, J., & Xia, M. (2023). Cr...
arXiv 2023
-
[112]
https://doi.org/10.1016/j.procs.2020.06.014 Bajpai, Dr. A. (2024). Evaluating the impact of artificial intelligence on enhancing tax compliance and financial regulation. SSRN Electronic Journal . https://doi.org/ 10.2139/ssrn.4922459 . Baker, M. R., Mahmood, Z. N., & Shaker, E. H. (2022). Ensemble learning with supervised machine learning models to predic...
-
[210]
https://doi.org/10.15282/jgi.5.1.2021.7031 Alarfaj, F. K., Malik, I., Khan, H. U., Almusallam, N., Ramzan, M., & Ahmed, M. (2022). Credit card fraud detection using state-of-the-art machine learning and deep learning algorithms. IEEE Access, 10 , 39700 – 39715. https://doi.org/10.1109/ACCESS.2022.3166891 . Alatawi, M. N. (2025). Detection of fraud in IoT ...
arXiv 2021
-
[305]
Kalbande, D., Prabhu, P., Gharat, A., & Rajabally, T
https:// doi.org/10.3390/systems11060305 . Kalbande, D., Prabhu, P., Gharat, A., & Rajabally, T. (2021). A fraud detection system using machine learning. In 2021 12th int. conf. comput. commun. netw.technol. ICCCNT (pp. 1 – 7). IEEE. https://doi.org/10.1109/ICCCNT51525.2021.9580102 . Kannan, S., & Srinath, M. V. (2018). Autoregressive-based outlier algori...
arXiv 2021
-
[516]
Alharbi, A., et al
https://api.semanticscholar.org/CorpusID:225056799 . Alharbi, A., et al. (2022). A Novel text2IMG mechanism of credit card fraud detection: A deep learning approach. Electron, 11 (5), 1 –
2022
-
[518]
https://doi.org/10.18280/ria.360401 Can, B., Yavuz, A. G., Karsligil, E. M., & Guvensan, M. A. (2020). In A Closer Look Into the Characteristics of Fraudulent Card Transactions, 8 pp. 166095 – 166109). IEEE Access. https://doi.org/10.1109/access.2020.3022315 . CARRACEDO, P., Herv ´as, D., & Soriano-Gonzalez, R.. Class Imbalance in Insurance Fraud Detectio...
arXiv 2020
-
[662]
https://doi.org/10.3390/ electronics11040662 Alghofaili, Y., Albattah, A., & Rassam, M. A. (2020). A financial fraud detection model based on LSTM deep learning technique. Journal of Applied Security Research, 15 , 498 –
2020
-
[1130]
https://doi.org/10.1057/s41599-024-03606-0 Hussein, A. S., Khairy, R. S., Mohamed Najeeb, S. M., & Salim Alrikabi, H. T. (2021). Credit card fraud detection using fuzzy rough nearest neighbor and sequential minimal optimization with logistic regression. International Journal of Interactive Mobile Technologies, 15 (5), 24 –
-
[1925]
Dhieb, N., Ghazzai, H., Besbes, H., & Massoud, Y
https://doi.org/10.11591/ijai.v13.i2.pp1925-1934 . Dhieb, N., Ghazzai, H., Besbes, H., & Massoud, Y. (2020). A secure ai-driven architecture for automated insurance systems: Fraud detection and risk measurement. IEEE Access, 8 , 58546 – 58558 . Ding, Y., Kang, W., Feng, J., Peng, B., & Yang, A. (2023). Credit card fraud detection based on improved variati...
arXiv 1934
-
[2022]
Chen, Y., Zhao, C., Xu, Y., & Nie, C. (2025). Year-over-Year Developments in Financial Fraud Detection via Deep Learning: A Systematic Literature Review. ArXiv.org . https:// arxiv.org/abs/2502.00201 . Cherkaoui, O., Anoun, H., & Maizate, A. (2024). A benchmark of health insurance fraud detection using machine learning techniques. IAES International Journ...
Pith/arXiv arXiv 2025
-
[2023]
Fursov, I., Kovtun, E., Rivera-Castro, R., Zaytsev, A., Khasyanov, R., Spindler, M., & Burnaev, E
Technical report, federal trade commission . Fursov, I., Kovtun, E., Rivera-Castro, R., Zaytsev, A., Khasyanov, R., Spindler, M., & Burnaev, E. (2022). Sequence embeddings help detect insurance fraud. IEEE Access, 10 , 32060 – 32074. https://doi.org/10.1109/ACCESS.2022.3149480 . Hemant Palivela, Vinay Rishiwal, Bhushan, S., Alotaibi, A., Agarwal, U., Kuma...
arXiv 2022
-
[3806]
https://doi.org/10.1007/s12652-020-01677-w Ashfaq, T., Khalid, R., Yahaya, A. S., Aslam, S., Azar, A. T., Alsafari, S., & Hameed, I. A. (2022a). Detection mechanism (pp. 1 – 20). MDPI . Ashfaq, T., Khalid, R., Yahaya, A. S., Aslam, S., Azar, A. T., Alsafari, S., & Hameed, I. A. (2022b). A machine learning and blockchain-based efficient fraud detection mec...
arXiv 2022
-
[5332]
https://doi.org/10.1007/s40747-023-01016-4 . Chen, S. (2022). Cryptocurrency financial risk analysis based on deep machine learning. Complexity , 1 –
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.