{"id":"18079c0c-d665-49b6-bbd2-28a3882895da","arxiv_id":"2411.17783","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"KANs applied to GMSC credit data produce a small AUC gain over borrowed baseline numbers, with interpretability shown only through the model's own attribution scores.","lead":"KACDP is a credit-default prediction model built from Kolmogorov-Arnold Networks, tested on the GMSC dataset, and it reports slightly better ROC_AUC and F1 than three classical baselines. A generalist would read this as another case of applying a new neural architecture to a standard tabular problem, with interpretability claims that are not yet strongly validated.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central outperformance claim rests on baselines borrowed from Han et al. [31] without a shared train/test split or preprocessing protocol, so the reported AUC and F1 gains are not currently evidence of superiority.","rationale":"The reader's weakest assumption is exactly the load-bearing flaw: the baselines are taken from another paper instead of being measured under the same protocol. The authors' own sentence in Section 4.2 makes this explicit, so the concern is not a speculative attack on their integrity; it is a concrete limitation of the comparison. The reported AUC differences are small enough that protocol differences in split, missing-data handling, or hyperparameter tuning could plausibly reverse them, and the absence of uncertainty intervals means even a controlled comparison would need replication to support the claim. If a rerun under the same protocol showed KACDP_OP still winning, the claim would become credible, but the current manuscript does not provide that evidence. The F1 metric is also questionable, as the reader notes, but the borrowed-baseline problem alone is sufficient to deny the central performance claim, so I do not move away from the REJECT verdict.","tokens_in":11972,"tokens_out":12019,"duration_ms":132592,"concrete_test":"Re-run Logistic Regression, XGBoost, and SVM on the exact GMSC train/validation/test split, missing-value imputation, and feature preprocessing used to obtain KACDP_OP's 0.8670 ROC_AUC, with the same hyperparameter-search budget, and report ROC_AUC with 95% bootstrap confidence intervals. If XGBoost's AUC equals or exceeds KACDP_OP's, or if the confidence interval covers the reported difference, the Table 1 outperformance claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.2 states that Logistic Regression, XGBoost, and SVM results were 'directly cited' from Han et al. [31] because that paper used the same GMSC dataset. This is an explicit admission that the baselines were not re-run under the authors' own protocol. Section 4.1.1 says preprocessing and feature engineering are 'based on [31]', but the paper never specifies the train/validation/test split, missing-value imputation for MonthlyIncome and NumberOfDependents, feature scaling, or hyperparameter search used for KACDP_OP. The reported AUC gaps are small (0.0036 over XGBoost, 0.0167 over Logistic Regression), so split-dependent noise or preprocessing differences could easily account for them. No confidence intervals, bootstrap estimates, or significance tests are provided. Because the central claim is that KACDP_OP outperforms these baselines, and that claim is supported only by a cross-paper comparison of numbers produced under unverified and possibly different conditions, the headline result is not established by the evidence in Table 1.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes applying Kolmogorov-Arnold Networks (KANs) to personal credit default prediction, a first claimed application in this domain. It constructs two KAN-based models, KACDP_OP (width [10,4,1]) and KACDP_OI (width [10,1]), and evaluates them on the GMSC dataset using ROC_AUC and F1. The reported results in Table 1 show KACDP_OP with AUC 0.8670, against 0.8503 for Logistic Regression, 0.8634 for XGBoost, and 0.8555 for SVM, with F1 values around 0.9675. The paper also claims interpretability through feature attribution scores and model structure visualization. The central claims are that KACDP outperforms mainstream baselines and provides transparent decision-making.","tokens_in":12215,"tokens_out":8589,"duration_ms":79923,"significance":"The idea of using KANs for credit scoring is timely, given the financial industry's need for interpretable models, and the paper uses a publicly available dataset with a clear problem formulation. The model-structure visualizations are a useful illustration of KAN-based decision paths. If properly validated, this would be a useful application note. However, the empirical evidence as presented does not establish the claimed superiority: the baseline numbers are borrowed from another study, the F1 metric is not fully specified and is implausibly high for the data's default rate, and the interpretability analysis relies on a feature-attribution method originating from the same group that proposed KANs, with no independent validation. The contribution is therefore currently a promising but unsubstantiated pilot study rather than a demonstrated improvement.","major_comments":[{"comment":"The central outperformance claim is not supported because the baseline scores for Logistic Regression, XGBoost, and SVM are not obtained under the authors' own protocol. Section 4.2 states that these results were 'directly cited' from Han et al. [31] because that paper used the same GMSC dataset. The manuscript does not establish that Han et al. used the same train/validation/test split, the same preprocessing (missing-value imputation, feature scaling, feature engineering), the same class-imbalance handling, or the same hyperparameter tuning as KACDP. The reported AUC gains are small (0.0036 over XGBoost and 0.0167 over Logistic Regression) and could easily be explained by split-dependent noise or differing preprocessing. A valid comparison must rerun all baselines on the same data folds under identical conditions and report confidence intervals, bootstrap estimates, or repeated-run variability.","section":"§4.2, Table 1"},{"comment":"The F1 score is threshold-dependent, but the paper never specifies the decision threshold used to convert the KAN's probability output into binary predictions, nor does it report precision and recall. On the GMSC dataset, whose default rate is roughly 6.7%, F1 values around 0.9675 are implausibly high unless a nonstandard threshold, resampling scheme, or evaluation set is used. The near-constant F1 across models, grid sizes, and optimizers (Tables 2 and 3) further suggests that the metric as computed is not discriminating. The authors must state the threshold, describe any class-imbalance handling, report precision and recall, and provide error bars or significance tests.","section":"§4.1.2 and Table 1"},{"comment":"The interpretability claim is supported only by feature attribution scores computed with the KAN 2.0 method [34], which originates from the same research group that introduced the KAN architecture. No comparison is made with model-agnostic interpretability tools such as SHAP or LIME, and no validation (e.g., ablation studies, case-level analysis, or faithfulness metrics) shows that these scores reflect the model's actual decision process. The paper should cross-check the attribution rankings with an independent method or otherwise demonstrate that the visualizations and scores provide reliable, non-obvious insight.","section":"§4.3.1"},{"comment":"The experimental setup is not reproducible: the manuscript does not report the train/validation/test split sizes or random seeds, how missing values in MonthlyIncome and NumberOfDependents were imputed, whether features were scaled, how the class imbalance was handled, how the decision threshold was selected, or how many independent runs the reported metrics are based on. The appendix hyperparameter study also does not clarify whether the tables report single-run results; the text says depth and width are studied, but only grid, optimizer, and learning rate are actually varied. These omissions prevent the reader from verifying the reported AUC and F1 values.","section":"§4.1.1 and Appendix"}],"minor_comments":[{"comment":"The model name is repeatedly written 'Credit Dedault Predict' instead of 'Credit Default Predict'; this typo appears in the abstract and throughout the paper.","section":"Abstract and §3.2"},{"comment":"The sentence 'In addition, Reinforcement Learning (DL), Recurrent Neural Network (RNN)' appears to be a typo; the intended entry is likely 'Deep Learning (DL)'.","section":"§2.1"},{"comment":"There are broken cross-references: 'Figure ??' appears in both Sections 4.3.1 and 4.3.2, and the feature index list in Section 4.3.1 is not properly presented as a numbered list.","section":"§4.3.1 and §4.3.2"},{"comment":"The mathematical formulas are typeset incorrectly (e.g., '2n+1X' and 'Pn p=1') and should be replaced with proper display math for the Kolmogorov-Arnold representation theorem and the KAN layer definitions.","section":"§3.1"},{"comment":"The hyperparameter sensitivity section claims to study the effect of width and depth, but no experiments varying width or depth are reported; the experiments only vary grid, optimizer, and learning rate.","section":"Appendix"}],"recommendation":"major_revision","confidential_remarks":"The main issue is that the empirical comparison is not controlled: baseline numbers are borrowed from another study, and the F1 reporting is incomplete and implausibly high for this dataset. These are fixable in principle by rerunning baselines under a common protocol and fully specifying the evaluation procedure. If the authors cannot or will not provide such a controlled comparison, the paper should not be accepted. The interpretability analysis also needs to be benchmarked against independent methods before the claims can be taken seriously."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take on KACDP. The paper does one thing: it trains a KAN on the GMSC credit dataset and reports a ROC_AUC of 0.8670, against 0.8503 for logistic regression and 0.8634 for XGBoost. That is the new result. The novelty is a routine application of a published architecture to a standard benchmark. That is fine for a method note, but the evidence as presented does not support the headline claim.\n\nWhat the paper does well: the hyperparameter sensitivity appendix is useful and relatively transparent, the two KAN configurations are clearly described, and the interpretability section at least shows something beyond a black box. The writing is readable, and the claim of being the first KAN application to individual credit default on this dataset is plausible.\n\nThe soft spots are not minor. Section 4.2 says the logistic, XGBoost, and SVM numbers were 'directly cited' from Han et al. because they used the same dataset. That means no shared train/test split, no shared preprocessing, no shared feature engineering, no shared hyperparameter search. The AUC gaps are 0.0036 and 0.0167, well within the range of split-dependent noise. There are no error bars, confidence intervals, or significance tests. Table 1 does not establish superiority.\n\nSecond, the F1 values around 0.967 are suspicious for a dataset with a default rate of about 6.7%. The paper never states whether F1 is computed for the positive class, negative class, or macro-averaged. At those values it looks like a majority-class metric or a mis-specified one. This needs to be clarified before any comparison.\n\nThird, the conclusion claims the model outperforms Random Forest, MLP, and RNN, none of which appear in Table 1. That overclaim is load-bearing.\n\nFourth, the interpretability evidence rests entirely on the KAN's own feature attribution scores, developed by the same group that proposed KAN. No independent validation against SHAP or LIME.\n\nThis paper is for practitioners who want a quick look at KAN on a tabular credit benchmark, but the current version is not a reliable reference. My verdict: the central performance claim is not demonstrated. I would not cite it. The fix is straightforward: re-run the baselines under the same protocol, define F1 precisely, and add uncertainty estimates. If that were done, it could become a useful short paper. As submitted, I would send it back for a controlled comparison rather than engage it as a serious referee.","headline":"Routine KAN application on GMSC with a small AUC edge, but the comparison is borrowed from another paper and the F1 metric looks off; the headline result is not supported as submitted.","tokens_in":12722,"tokens_out":5850,"would_cite":false,"duration_ms":54308,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["91G40","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"Applying Kolmogorov-Arnold networks to individual credit default prediction yields a model whose ROC_AUC (0.8670) exceeds logistic regression (0.8503), XGBoost (0.8634), and SVM (0.8555) on the GMSC dataset, while retaining…","keywords":["Kolmogorov-Arnold networks","credit default prediction","interpretability","ROC AUC","F1 score","GMSC dataset","feature attribution","B-spline activations"],"falsifier":"Run logistic regression, XGBoost, and SVM on the exact same GMSC train/test split and preprocessing used for KACDP_OP, each with its own hyperparameter tuning, and check whether KACDP_OP's ROC_AUC of 0.8670 still beats them; if any baseline reaches or exceeds that value, the paper's central outperformance claim fails.","tokens_in":11761,"feed_emoji":"📊","tokens_out":5772,"duration_ms":46894,"temperature":0.7,"pith_summary":"The paper claims that a credit-default model built from Kolmogorov-Arnold Networks (KANs) outperforms traditional logistic regression, XGBoost, and SVM on the GMSC dataset, achieving a ROC_AUC of 0.8670 versus 0.8503, 0.8634, and 0.8555, and an F1 of 0.9675 versus 0.9665, 0.9669, and 0.9665. It also claims the model is transparent: feature attribution scores name the dominant risk factors, and the network structure diagram lets a user trace each input through learned univariate functions to the output. This matters because regulated financial institutions need models that do more than predict accurately; they must be able to explain why each borrower was scored as they were. The paper's contribution is the first use of KANs for individual credit default prediction, arguing the architecture's interpretability costs little in accuracy.","feed_headline":"Kolmogorov-Arnold nets beat XGBoost on default prediction","feed_subtitle":"A transparent credit-scoring model tops 0.867 AUC and still shows why each borrower was flagged.","key_machinery":"The central object is the Kolmogorov-Arnold Network layer, in which each connection carries a learnable univariate function $\\phi(x) = w_b \\operatorname{silu}(x) + w_s \\sum_i c_i B_i(x)$ combining a base SILU activation with a B-spline. Because the Kolmogorov-Arnold representation theorem expresses any continuous multivariate function as a composition of sums of univariate functions, stacking these layers lets the model's decision be decomposed into per-feature contributions. This decomposition is what carries the interpretability claim: the trained functions themselves give a feature's attribution score, and the network graph doubles as a decision-path diagram.","core_discovery":"On the paper's own terms, KACDP_OP, a KAN with width [10, 4, 1], achieves the best ROC_AUC among the compared models on the GMSC dataset (0.8670), while the simpler KACDP_OI (width [10, 1]) reaches 0.8640, still ahead of the cited logistic regression and SVM results. Feature attribution scores identify the borrower's debt-to-income ratio (2.2836) and credit-card utilization (1.3468) as the strongest drivers, and the model's structural visualization shows how each input is processed by a learned activation function before a final summation.","pith_inferences":["My extension: if the univariate decomposition, rather than raw capacity, drives the AUC gain, then a KAN with similar parameter count should beat an MLP on tabular finance data; this is testable and goes beyond the paper's comparison set.","My extension: the same visualization and attribution machinery could transfer to other regulated tabular domains such as insurance pricing, fraud detection, or medical risk scoring, but the paper does not demonstrate those transfers.","My extension: the 'first application' claim is a novelty statement that could be checked against earlier spline-based or functional-link networks; if such works exist, the paper's lasting value would be the benchmark result rather than the architectural first."],"forward_implications":["If the numbers hold under a common protocol, lenders could adopt KACDP-style models to obtain XGBoost-level accuracy with per-prediction explanations that satisfy audit requirements.","The simple one-layer variant (KACDP_OI) loses only 0.003 AUC to the deeper version, suggesting interpretability does not force a large accuracy sacrifice.","Feature-attribution output pointing to debt-to-income ratio and credit-card utilization gives lenders a concrete, data-driven shortlist of variables to monitor or collect more carefully.","Training a KAN with Adam on the 223,958-row GMSC dataset took roughly 10 seconds, making the architecture practical for routine credit-scoring pipelines."],"supporting_citations":[{"why":"Introduces the Kolmogorov-Arnold Network architecture with learnable activation functions that the KACDP model is built on.","marker":"[11]"},{"why":"Supplies the GMSC data preprocessing, feature selection, and the logistic regression, XGBoost, and SVM baseline scores that KACDP is compared against.","marker":"[31]"},{"why":"Provides the feature attribution scoring method used to quantify each input's importance in the interpretability analysis.","marker":"[34]"},{"why":"Motivates the regulatory demand for explainable models, framing the interpretability requirement the model is designed to satisfy.","marker":"[10]"}],"fun_headline_variants":["KAN credit model beats baselines, shows its work","KACDP: high default-prediction AUC with clear reasoning","New KAN default predictor: 0.867 AUC, fully interpretable","Interpretable KAN tops credit risk baselines on GMSC"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The outperformance claim assumes the baseline scores quoted from Han et al. were computed under the same data preprocessing, feature engineering, train/test split, and tuning as the authors' own KACDP runs; if those conditions differ, the reported AUC differences are not meaningful.","fun_headline_variants_meta":{"raw":{"variants":["KAN credit model beats baselines, shows its work","KACDP: high default-prediction AUC with clear reasoning","New KAN default predictor: 0.867 AUC, fully interpretable","Interpretable KAN tops credit risk baselines on GMSC"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000299,"raw_usage":{"total_tokens":1717,"prompt_tokens":921,"completion_tokens":796,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":537,"completion_tokens_details":{"reasoning_tokens":722}},"tokens_in":537,"tokens_out":796,"duration_ms":10319,"temperature":1.0,"reasoning_tokens":722,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:08:27.494612+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run logistic regression, XGBoost, and SVM on the exact same GMSC train/test split and preprocessing used for KACDP_OP, each with its own hyperparameter tuning, and check whether KACDP_OP's ROC_AUC of 0.8670 still beats them; if any baseline reaches or exceeds that value, the paper's central outperformance claim fails.","supporting_citations":[{"cited_title":"Personal credit default prediction fusion frame- work based on self-attention and cross-network algorithms","cited_arxiv_id":null,"evidence_quote":"Supplies the GMSC data preprocessing, feature selection, and the logistic regression, XGBoost, and SVM baseline scores that KACDP is compared against."},{"cited_title":"Kan 2.0: Kolmogorov-arnold networks meet science, 2024","cited_arxiv_id":null,"evidence_quote":"Provides the feature attribution scoring method used to quantify each input's importance in the interpretability analysis."},{"cited_title":"GLocalX - From Local to Global Explanations of Black Box AI Models","cited_arxiv_id":null,"evidence_quote":"Motivates the regulatory demand for explainable models, framing the interpretability requirement the model is designed to satisfy."}],"review_version":1}