{"id":"aeb2c684-6250-4512-b424-30061205b466","arxiv_id":"2412.04183","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A hybrid XGBoost-deep neural network with LDA feature reduction is reported to reach 99.45% accuracy on Lending Club credit data, though the evaluation has likely data leakage.","lead":"This paper tests whether Linear Discriminant Analysis can shrink a large credit dataset before training machine learning models without hurting accuracy. The authors report a hybrid XGBoost-deep network model reaching 99.45% accuracy on Lending Club loan data, but the evaluation has likely data leakage and other methodological flaws.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SMOTE-before-split leakage is the load-bearing flaw: synthetic samples enter the test set, inflating the 99.45% accuracy that the LDA-XG-DNN claim rests on.","rationale":"The reader's REJECT is justified. The paper's abstract and Section IV-B claim that LDA feature reduction improves or maintains the performance of the XG-DNN hybrid, but every reported performance figure depends on an evaluation protocol in which SMOTE is applied before the train/test split. That is not a stylistic choice; it directly injects synthetic test instances whose nearest neighbors are training instances, so the test accuracy is an upper bound rather than an honest generalization estimate. The leaky post-loan features identified in Section IV-C make the XAI analysis equally unreliable. The claimed reduction to 21 LDA components for a 10-class target is mathematically impossible, which further weakens confidence that the experiments were run as described. These are internal correctness problems, not disagreements with field consensus, so they are properly load-bearing. A corrected protocol could in principle still show an LDA benefit, but the manuscript currently provides no such evidence, and no code or hyperparameters are supplied to verify the pipeline. I agree with the reader's weakest assumption and see no reason to change the verdict.","tokens_in":8146,"tokens_out":4174,"duration_ms":45988,"concrete_test":"Correct the protocol and re-run: split the original data into 80% train and 20% test before any SMOTE, fit SMOTE only on the training split, train XG-DNN with and without LDA on that training set, and evaluate both on the untouched original test set, also excluding post-loan features such as out_prncp, total_rec_prncp, and recoveries. If LDA-based XG-DNN no longer reaches or exceeds the non-LDA baseline by the claimed margin, the SMOTE-before-split leakage is what produced the headline result rather than any benefit of LDA.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III-A applies SMOTE to the full dataset before the 80/20 train/test split described in Section III-B, so synthetic instances generated from the entire data are present in the test set. Because SMOTE creates points as convex combinations of nearby real instances, the model is effectively evaluated on interpolations of its training data; the reported 99.45% accuracy and 99% F1 for XG-DNN are therefore not out-of-sample estimates. The explainability analysis in Section IV-C reinforces the problem: the dominant features are out_prncp, total_rec_prncp, and recoveries, which are post-loan outcome variables unavailable at application time, so the feature-importance story is also contaminated. A separate internal inconsistency, Section III-C claiming 21 LDA components for a 10-class target when LDA can produce at most 9 discriminative directions, shows that the pipeline as reported cannot be reproduced as stated. For the central claim to be true, the evaluation would have to survive a corrected split-and-SMOTE protocol on features available at scoring time; the paper provides no such evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes using Linear Discriminant Analysis (LDA) as a dimensionality-reduction step before training several machine learning and deep learning models on the Lending Club loan dataset, with an emphasis on a hybrid XGBoost-DNN model (XG-DNN). The authors report that LDA-based XG-DNN achieves 99.45% accuracy and a 99% F1 score, outperforming all other models, and they use LIME and Morris Sensitivity Analysis to interpret the model decisions. The central empirical claim is that LDA can reduce model complexity without harming performance or explainability.","tokens_in":8335,"tokens_out":3814,"duration_ms":38605,"significance":"If the reported results were valid, the paper would offer a potentially useful demonstration that a simple linear dimensionality-reduction technique can cut computational cost while preserving accuracy and explainability in credit scoring, a domain where both are valued. The work also attempts to combine local (LIME) and global (Morris) explanations, which is a commendable breadth. However, the paper is purely empirical and supplies no code, data splits, or reproducibility artifacts, and the evaluation protocol as described contains several load-bearing flaws. These flaws—SMOTE leakage into the test set, target leakage through post-loan outcome features, and an impossible LDA component count—mean that the headline numbers cannot be interpreted as valid out-of-sample credit-scoring performance. The potential significance is therefore largely unsubstantiated by the manuscript as written.","major_comments":[{"comment":"The preprocessing section (III-A) applies SMOTE to the full dataset, and only afterward does the modeling section (III-B) describe splitting into 80% training and 20% test data. This order means synthetic instances generated from the entire dataset can appear in the test set. Because SMOTE creates interpolations of real neighbors, the model is effectively evaluated on points close to its training distribution, so the reported test accuracy and F1 are not out-of-sample estimates. The central claim of 99.45% accuracy for LDA-based XG-DNN rests on this contaminated evaluation. The paper should split first, apply SMOTE only to the training portion, and then evaluate on the untouched test set.","section":"III-A and III-B"},{"comment":"The explainability analysis identifies out_prncp (remaining outstanding principal), total_rec_prncp (principal received to date), and recoveries as the dominant features for both LIME and Morris Sensitivity Analysis. These are post-loan outcome variables that are not available at the time a credit application is scored; using them to predict current loan status is a different task from credit scoring and constitutes target leakage. The model may be largely reading off loan repayment status from variables that are themselves consequences of repayment. The paper should restrict features to those available at application time and repeat the performance and XAI analyses, otherwise both the predictive-accuracy claim and the interpretability discussion are misleading.","section":"IV-C"},{"comment":"Section III-C states that applying LDA as a feature reduction technique leaves 21 features from 107 features. The target variable loan_status has 10 classes, so LDA can produce at most 9 discriminant dimensions (c-1). A pipeline that outputs 21 LDA components cannot be reproduced as described. This is a load-bearing inconsistency for the central method. The authors should clarify whether they used a different technique (e.g., PCA, or selecting a subset of original features) or correct the description to match the actual component count.","section":"III-C"},{"comment":"LDA is applied only to XGBoost and XG-DNN after these two models were observed to be the best performers without LDA. This post-hoc selection, made after seeing the non-LDA results, raises a multiple-comparisons concern: the reported improvement of XG-DNN with LDA is selected from a larger implicit set of possible model-LDA combinations. The paper should either pre-register the selection rule or report results for all models both with and without LDA, so the comparison is not conditioned on the outcome it is claiming to demonstrate.","section":"IV-B"}],"minor_comments":[{"comment":"The text in Section IV-A says the Deep Neural Network achieved 100% sensitivity and specificity, but Table I lists DNN sensitivity as 99.20 and specificity as 99.10; these numbers should be reconciled.","section":"IV-A"},{"comment":"The phrase 'the LDA-based XG-DNN hybrid model shows the lowest specificity at 99.85' is confusing because 99.85 is a very high value; rephrasing to compare it to the XGBoost specificity of 99.98 would be clearer.","section":"IV-B"},{"comment":"Several in-text citation numbers do not match the reference list: the citation to Jasmina Nalić et al. appears as [21], which is out of range, and the citation to Shrawan Kumar Trivedi appears as [16], which corresponds in the bibliography to Nalić et al.; the reference list should be carefully renumbered and cross-checked.","section":"Literature Review and References"},{"comment":"Figure 6 is referenced in Section IV-A ('shown in Figure 6') but the figure is not included in the manuscript text, making the accuracy comparison difficult to follow.","section":"Figures"}],"recommendation":"reject","confidential_remarks":"The manuscript is a benchmark-style empirical study with no code or data-release artifacts, and the reported evaluation is invalidated by SMOTE leakage and target leakage through post-loan outcome features. The LDA component-count inconsistency further undermines reproducibility. Even if the authors corrected these issues, the contribution would be an incremental empirical comparison rather than a methodological advance; the use of both LIME and Morris analysis is a positive angle, but it does not compensate for the invalid central evaluation. I recommend rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about arXiv:2412.04183. First, the headline result—99.45% accuracy for an LDA-reduced XGBoost-DNN hybrid—is inflated by a leaky evaluation protocol. Second, the paper has an internal contradiction that makes the pipeline as written impossible to reproduce.\n\nThe paper does a few things well. It runs the standard benchmark suite on Lending Club (eight models, six metrics), and it tries to look at local and global explainability with LIME and Morris sensitivity. The question it asks—whether LDA feature reduction can shrink the input space without wrecking accuracy or interpretability—is a legitimate one for resource-constrained deployment.\n\nBut the execution does not support the claim. SMOTE is applied in Section III-A before the 80/20 split in Section III-B. Synthetic samples are convex combinations of real points, so the test set contains interpolations of training data. The reported accuracies are in-sample by construction. The explainability analysis reinforces the problem: LIME and Morris both rank out_prncp, total_rec_prncp, and recoveries as top features. Those are post-loan outcome variables—you cannot know them when a loan application comes in. So the interpretability story is also contaminated.\n\nThere is also a standalone technical error. Section III-C says LDA leaves 21 features from 107. The target loan_status has 10 classes, so LDA can produce at most 9 discriminative directions. 21 is impossible for the supervised transform as described. That is not a minor typo; it means the reported pipeline cannot be run as stated.\n\nThe citation pattern is fine and the prose is readable. I do not see anything deliberately deceptive. But the empirical core is not credible. No code or hyperparameters are shipped, so nothing can be checked.\n\nWho gets value from this paper? Maybe someone collecting examples of common credit-scoring pitfalls—SMOTE leakage and outcome leakage—as a teaching case. As a research contribution, it does not clear the bar.\n\nI would desk-reject. The flaws are load-bearing, not polish issues. If the authors rerun the whole thing with SMOTE inside the training fold only and drop outcome variables, the comparison would be informative. As written, the numbers are not out-of-sample and the method description does not hang together.","headline":"LDA feature reduction in credit scoring is a reasonable question, but this paper's SMOTE-before-split leakage and impossible LDA dimensionality make the headline accuracy untrustworthy.","tokens_in":8905,"tokens_out":2061,"would_cite":false,"duration_ms":19678,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that LDA feature reduction plus an XGBoost-DNN hybrid reaches 99.45% accuracy on Lending Club credit data while keeping explanations intact.","keywords":["credit scoring","Linear Discriminant Analysis","Lending Club dataset","XG-DNN hybrid","XGBoost","explainable AI","LIME","Morris sensitivity analysis"],"falsifier":"Re-run the pipeline with SMOTE applied only inside the training split and with out_prncp, total_rec_prncp, and recoveries removed; if accuracy and F1 fall well below 99.45%, the reported result depends on leakage rather than on LDA.","tokens_in":7922,"feed_emoji":"📊","tokens_out":6417,"duration_ms":59840,"temperature":0.7,"pith_summary":"The paper tries to show that Linear Discriminant Analysis can be used as a feature-reduction step in credit scoring without sacrificing accuracy or explainability. On the Lending Club dataset, it reduces 107 features to 21 and then trains an XGBoost-and-deep-network hybrid called XG-DNN, reporting 99.45% accuracy and a 99% F1 score, marginally above the same hybrid without LDA. The paper compares eight models with and without LDA, and applies LIME and Morris Sensitivity Analysis to show that the reduced model still explains its decisions through interpretable loan features. If correct, this matters for resource-constrained lenders who want cheaper, faster scoring models that remain transparent.","feed_headline":"LDA slashes credit features to 21, keeps 99.45% accuracy","feed_subtitle":"Hybrid XG-DNN model with LDA also keeps LIME and Morris explanations intact, easing compute in credit scoring.","key_machinery":"The machinery is LDA as a supervised projector: it finds linear combinations of the 107 original features that best separate loan-status classes and keeps 21 dimensions. Onto that lower-dimensional space the paper fits XG-DNN, a hybrid that feeds XGBoost's refined feature representation into a deep neural network to capture higher-order interactions. Explanations come from two post-hoc tools: LIME, which builds local surrogate explanations for individual predictions, and Morris Sensitivity Analysis, a one-step-at-a-time global sensitivity method that ranks features by elementary effects. Together, the LDA projection is the load-bearing step that lowers model complexity, and the two XAI tools are what the paper uses to claim transparency is preserved.","core_discovery":"The central claim is that LDA-based feature reduction is not a cost paid in performance: the XG-DNN hybrid trained on the 21-dimensional LDA projection reaches the best accuracy (99.45%) and ties the best F1 score (99%) among all tested configurations, while the same model without LDA reaches 99.44% accuracy, and the paper reports a slight sensitivity trade-off (72.79% sensitivity). The paper also claims that the reduced model remains explainable: LIME attributes individual predictions mainly to features like the outstanding principal and principal received to date, and Morris Sensitivity Analysis ranks recoveries, out_prncp, and total_rec_prncp as the most influential. Thus the paper concludes that dimensionality reduction can improve or maintain predictive performance and that post-hoc XAI can recover transparency lost in the projection.","pith_inferences":["A natural next experiment is to compare LDA with PCA or variance-based feature selection under the same split; the paper does not isolate whether the gain comes from dimensionality reduction itself or from LDA's class-separating objective.","Because LDA components are linear mixtures of the original features, the LIME and Morris explanations may not exactly describe what the reduced model sees unless the projection is mapped back to original feature space.","The very high accuracy levels suggest that some influential features, such as remaining principal and recoveries, may be known only after the loan is issued; dropping them would give a sterner test of true scoring performance.","If the approach transfers, other high-dimensional tabular credit datasets would be natural test beds, but the paper demonstrates the method only on Lending Club."],"forward_implications":["If the claim holds, lenders can drop from 107 to 21 features before training and still get top accuracy, cutting storage and computation.","The XG-DNN hybrid becomes a practical option for resource-constrained settings where training a full-size deep model is too costly.","Feature reduction via LDA does not automatically destroy local or global interpretability; LIME and Morris still identify meaningful loan features.","The measured gain from LDA is small on accuracy (99.45 versus 99.44 percent), so the main practical benefit is computational rather than predictive."],"supporting_citations":[{"why":"supplies the Lending Club loan dataset, the empirical basis for every reported accuracy and F1 score.","marker":"[14]"},{"why":"identifies LDA among the best classifiers in peer-to-peer lending, motivating its use as a feature-reduction technique.","marker":"[10]"},{"why":"proposes a hybrid of gradient boosting and neural network for online credit scoring, the design the XG-DNN model builds on.","marker":"[12]"},{"why":"provides the scaled elementary-effects version of the Morris method used for global sensitivity analysis.","marker":"[18]"},{"why":"cited as the global sensitivity-analysis technique underlying the Morris method applied to the XG-DNN model.","marker":"[17]"}],"fun_headline_variants":["LDA trims credit features to 21, hybrid hits 99.45% accuracy","No accuracy loss: LDA reduces credit features, keeps 99% F1","XG-DNN with LDA: 99.45% credit score accuracy, transparent","LDA cuts features to 21, XG-DNN keeps 99% F1 and XAI","Explainable credit scoring: LDA slims features, tops 99% F1"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results assume that SMOTE applied before the 80/20 split does not leak synthetic samples into the test set, and that features such as remaining principal, principal received, and recoveries are available when a loan is being scored.","fun_headline_variants_meta":{"raw":{"variants":["LDA trims credit features to 21, hybrid hits 99.45% accuracy","No accuracy loss: LDA reduces credit features, keeps 99% F1","XG-DNN with LDA: 99.45% credit score accuracy, transparent","LDA cuts features to 21, XG-DNN keeps 99% F1 and XAI","Explainable credit scoring: LDA slims features, tops 99% F1"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000769,"raw_usage":{"total_tokens":3399,"prompt_tokens":929,"completion_tokens":2470,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":545,"completion_tokens_details":{"reasoning_tokens":2354}},"tokens_in":545,"tokens_out":2470,"duration_ms":15713,"temperature":1.0,"reasoning_tokens":2354,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:39:12.878927+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the pipeline with SMOTE applied only inside the training split and with out_prncp, total_rec_prncp, and recoveries removed; if accuracy and F1 fall well below 99.45%, the reported result depends on leakage rather than on LDA.","supporting_citations":[{"cited_title":"15, 2020","cited_arxiv_id":null,"evidence_quote":"supplies the Lending Club loan dataset, the empirical basis for every reported accuracy and F1 score."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"identifies LDA among the best classifiers in peer-to-peer lending, motivating its use as a feature-reduction technique."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"cited as the global sensitivity-analysis technique underlying the Morris method applied to the XG-DNN model."}],"review_version":1}