{"id":"45cd02b8-7277-49ab-8cec-d7b3a48431a1","arxiv_id":"2509.11847","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"On Swiss employment data, explainable boosting machines predict long-term unemployment almost as accurately as XGBoost while remaining fully interpretable.","lead":"This paper compares interpretable and black-box machine learning models for predicting long-term unemployment using Swiss administrative data. It finds that an interpretable model, the explainable boosting machine, is nearly as accurate as the best black-box model while remaining transparent and fair.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central EBM-vs-XGB comparison depends on an untested assumption that data leakage affects all models equally; if leakage is model-dependent, the reported performance gap could change.","rationale":"The reader's weakest_assumption correctly identifies the temporal-leakage issue as the most load-bearing point. The paper's entire empirical contribution is a comparison of AUC values across models, and the headline claim is explicitly that the gap between EBM and XGB is small. That claim is only meaningful if the measured AUCs reflect genuine predictive signal rather than accidental temporal leakage. The paper itself flags this in Sec. 6.2, but dismisses the risk with an unverified assertion that leakage would affect all models similarly. This is not a matter of internal inconsistency; it is an unresolved empirical risk. The proposed test directly addresses the risk by reconstructing features with a clear temporal cutoff and re-running the comparison. Should the gap remain stable under that test, the concern is resolved and the central claim is strengthened. Since the paper is already CONDITIONAL and the concern reinforces the need for that condition, no verdict change is needed.","tokens_in":19513,"tokens_out":6824,"duration_ms":79466,"concrete_test":"Obtain the raw asal dataset and reconstruct the asal-derived features using only records dated before each unemployment episode's start; in particular, recompute beitragsmonate_vor_rf, taggeld_anspr_bins, and vers_verdienst with a look-back window that ends at the episode start. Retrain LR, EBM, RF, GB, and XGB on the four training folds and evaluate on the 2019 test set. If the EBM−XGB AUC gap changes by more than roughly 0.003 or the model ordering changes, the Sec. 6.2 assumption fails and the central claim needs qualification.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—EBM performs nearly as well as XGB—rests on the untested assumption in Sec. 6.2 that data leakage affects all models comparably. Several features (e.g., beitragsmonate_vor_rf, taggeld_anspr_bins, vers_verdienst) are derived from monthly insurance-payment records (asal, App. A.1) whose aggregation window relative to the unemployment-episode start is not specified. If these values are measured over the 12-month outcome window, they directly encode the outcome. The paper acknowledges 'the temporal order of data collection is not known for at least some features' but then asserts, without empirical support, that leakage would only cause 'quantitative differences' across models. Since the headline result is a quantitative AUC gap (test 2019: EBM 0.7593 vs XGB 0.7619, Table 5), even a small model-dependent leakage effect could change the gap's magnitude or sign—and thus the conclusion that interpretable models are nearly as accurate as black-box models. The absence of error bars on this gap compounds the problem, making it impossible to assess whether the 0.0026 difference is robust even under the no-leakage assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper empirically compares interpretable and black-box machine learning models for long-term unemployment (LTU) risk prediction using Swiss administrative unemployment data from 2014–2019. The models are logistic regression (LR), explainable boosting machines (EBM), random forests (RF), gradient boosting (GB), and XGBoost (XGB). Using a moving-window cross-validation design, the paper reports AUC for years 2015–2019, with 2019 held out as the test year. The main empirical claim is that EBM, an inherently interpretable model, achieves predictive performance close to XGB (test 2019 AUC: EBM 0.7593 vs. XGB 0.7619, Table 5). The paper also investigates sparsity via backward selection, smoothing of EBM feature functions, and post-hoc fairness mitigation using the Hardt et al. (2016) equalized-odds post-processing method. The paper argues that interpretable profiling is a viable, accountable alternative to black-box profiling. The manuscript acknowledges in Sec. 6.2 that the temporal order of data collection is unknown for at least some features, creating a possible data leakage risk, but asserts without empirical evidence that leakage would affect all models qualitatively similarly.","tokens_in":19859,"tokens_out":5507,"duration_ms":68477,"significance":"If the central empirical finding is robust, the paper makes a useful contribution to the interpretable-ML and public-employment-services literature by showing that an interpretable additive model can approach the accuracy of a state-of-the-art tree ensemble on a large administrative dataset. The study has several strengths: it uses full administrative records with roughly 165,000 observations per year, employs a time-series-aware evaluation scheme, and releases code for reproduction on GitHub. The fairness analysis, while exploratory, demonstrates a concrete post-processing approach compatible with interpretable models. The significance is qualified by the unresolved leakage question and the lack of uncertainty quantification, both of which are load-bearing for the headline claim. The paper does not present a new algorithm or derivation; its value is as an applied empirical comparison.","major_comments":[{"comment":"The leakage concern is acknowledged in Sec. 6.2 but the paper's response is an untested assertion: 'data leakage would affect all models, even though there may be quantitative differences.' This is load-bearing because the headline result is a quantitative AUC difference (test 2019: EBM 0.7593 vs. XGB 0.7619, Table 5). Features such as beitragsmonate_vor_rf, taggeld_anspr_bins, and vers_verdienst are derived from the monthly insurance-payment system (asal, App. A.1), and the outcome is also defined from monthly benefit receipt over the first 12 months. If the measurement windows of these features overlap the outcome window, they can directly encode the outcome. The claim that such leakage affects all models comparably is not supported by any experiment. Please provide a sensitivity analysis, e.g., rerun the comparisons on a feature set restricted to variables determined before the unempl","section":"§6.2, App. A.1; features beitragsmonate_vor_rf, taggeld_anspr_bins, vers_verdienst"},{"comment":"All AUC comparisons are reported as point estimates without confidence intervals, bootstrap errors, or significance tests. For example, the validation-fold EBM-minus-XGB gaps are 0.0069, 0.0065, 0.0076, and 0.0090, while the test gap is 0.0026. It is impossible to assess whether 'nearly as well' is a statistically and practically meaningful statement. Moreover, the test conclusion rests on a single year (2019). Please provide uncertainty quantification for the AUC differences (e.g., bootstrap or DeLong intervals) and explicitly state whether the observed test-year gap is distinguishable from zero at the available sample size. If possible, also evaluate on additional held-out years or report repeated moving-window test evaluations.","section":"§5.1, Table 5, Fig. 1"},{"comment":"The model comparison is not fully controlled. XGB hyperparameters were tuned with Optuna, while EBM's key hyperparameter, the number of interactions, was manually set to a low value (30) for interpretability reasons. RF and GB hyperparameters are taken from an unpublished thesis (Gasser 2022). The manuscript states a 'fair' comparison, but this is not established: a manually constrained EBM could understate or overstate the gap depending on how interactions affect performance. Please report a sensitivity analysis over EBM interaction counts (e.g., 0, 10, 30, 60) and state whether the near-parity conclusion holds. Hyperparameters borrowed from non-public sources should be fully documented in the appendix or code so the results are reproducible without access to the thesis.","section":"Appendix B, §5.1"},{"comment":"The abstract claims that fairness mitigation can enhance transparency and fairness 'with only minor losses in performance.' The reported mitigation reduces TPR from 80% to 66% (Fig. 6d), a 14-percentage-point drop, which the paper itself describes as making the predictor 'not as efficient.' This is not a minor loss at the original operating point unless some other performance measure, such as accuracy, is the criterion. The paper reports that accuracy increases after mitigation, but the relevant trade-off depends on the cost structure of false negatives versus false positives, which is not specified. In addition, the post-processing equalizes FPR under a balanced-accuracy constraint, while the text and figures emphasize normalized false positives; the mismatch between the targeted fairness quantity and the displayed quantity should be clarified.","section":"§5.4, Fig. 6, Abstract"}],"minor_comments":[{"comment":"The phrase 'train-test data (years 2018-2019)' is confusing; the test set is only year 2019 according to Table 2. Please rephrase to 'test data (year 2019)'.","section":"§3.1"},{"comment":"The text says 'smoothing was applied to individual, off-the-shelf numerical feature functions of one sparse EBM-30 in the first fold' but does not say whether the smoothing parameters were selected on that fold's validation data or on the training data. Please state this explicitly to avoid an optimistic-bias concern.","section":"§5.3"},{"comment":"The backward-selection figure caption says 'the rightmost model, with 57 main features and 30 interactions, is the full EBM,' but the main text refers to the full EBM with 57 features in Sec. 4.4 as well. Please ensure the labeling is consistent between the figure, caption, and text.","section":"§5.2, Fig. 8"},{"comment":"The claim that this is the first empirical investigation of interpretable LTU profiling should be softened or qualified, since the paper builds on Gasser (2022, 2023) and there is related work on EBM for social outcomes. A more precise statement would be 'the first comparison of EBM to black-box models in LTU profiling with Swiss administrative data.'","section":"§2.2"},{"comment":"The data availability statement says data can be obtained from SECO, but the manuscript relies on two non-public theses (Gasser 2022, 2023) for parts of the data construction and hyperparameters. Please ensure all necessary preprocessing decisions are described in the paper or in the released code, so that a third party with data access can reproduce the clean dataset without consulting the theses.","section":"Appendix D.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is an applied ML contribution with a strong policy framing. The central claim is plausible but currently rests on an untested leakage assumption and point estimates without uncertainty quantification. These are fixable within the manuscript's scope. I would not reject on the grounds that the result disagrees with the common 'black-box superiority' position; the issue is internal support. The reliance on non-public Gasser theses is a reproducibility concern that should be addressed by making all hyperparameters and preprocessing decisions explicit in the paper or code."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Tim,\n\nThe one thing you should know: this is a real empirical attempt to show that an interpretable model (EBM) is nearly as accurate as the best black-box model (XGB) for long-term unemployment risk in Swiss administrative data. The result is plausible, and the paper is honest about its limits, but the headline claim needs error bars and a leakage sensitivity check before I'd treat the gap as settled.\n\nWhat's actually new: it applies EBM to LTU profiling, and it adds sparsity, smoothing, and fairness post-processing experiments on the same data. The stakeholder framing is well done. It doesn't oversell the fairness mitigation—it shows the TPR drops from 80% to 66% when equalizing false-positive rates. Code is on GitHub and the data access path is described, which matters for reproducibility.\n\nWhere it's soft: the central comparison (EBM 0.7593 vs XGB 0.7619 on the 2019 test year) has no confidence intervals or significance test, and it's a single year. That 0.0026 gap is the whole ballgame. Second, the paper admits in Sec 6.2 that the temporal order of data collection is unknown for some features, so leakage is possible, and then asserts that leakage would affect all models similarly. That assertion is untested. If leakage inflates EBM more or less than XGB, the gap could change size or sign. Third, the abstract says 'without compromising performance,' which is too strong given the measured small gap and the losses from smoothing and fairness mitigation. Finally, the EBM interaction count was set by hand, and the hyperparameter tuning was asymmetric.\n\nNone of these kill the paper. The limitations are acknowledged, and the proof-of-concept value is real. But a referee should ask for error bars on the AUC table, a leakage robustness analysis (e.g., dropping the suspicious features), and checked language in the abstract.\n\nBottom line: send it to peer review. With a serious referee, it can become a solid contribution to the interpretable-profiling literature.","headline":"A credible proof-of-concept that explainable boosting machines come close to XGBoost for long-term unemployment profiling, but the headline needs error bars and a leakage check before I'd trust the gap.","tokens_in":20261,"tokens_out":2257,"would_cite":true,"duration_ms":25761,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Interpretable models can profile long-term unemployment risk almost as accurately as black-box models, with far greater transparency.","keywords":["long-term unemployment","interpretable machine learning","explainable boosting machines","fairness","profiling","Switzerland","administrative data","predictive performance"],"falsifier":"Obtain the same Swiss administrative data, conduct a strict audit of each feature's collection time, and exclude any feature whose values could have been known at the prediction time; if EBM's test-year AUC then falls substantially below XGBoost's (e.g., a gap above 0.02), the paper's 'nearly as good' claim is refuted.","tokens_in":19428,"feed_emoji":"⚖️","tokens_out":4603,"duration_ms":49482,"temperature":0.7,"pith_summary":"This paper asks whether transparent machine-learning models can replace opaque black-box models for profiling long-term unemployment (LTU) risk. Using Swiss administrative records on about a million jobseekers, it compares logistic regression and explainable boosting machines (EBM) against random forests, gradient boosting, and XGBoost. The central result is that EBM performs nearly as well as the best black-box model: on the 2019 test year, EBM achieves an AUC of 0.7593 versus XGBoost's 0.7619, a gap of only 0.003. The paper also shows that making EBM sparser and smoothing its feature functions costs little accuracy, and that post-processing can equalize false-positive rates across age groups without destroying interpretability. If true, this means employment services could deploy accountable, understandable profiling without sacrificing much predictive power.","feed_headline":"Interpretable AI nearly matches black-box unemployment-risk models","feed_subtitle":"Swiss data show explainable boosting within 0.003 AUC of XGBoost, while staying transparent about why a person gets a risk score.","key_machinery":"The central object is the Explainable Boosting Machine (EBM), a generalized additive model whose prediction function is a sum of univariate feature functions plus a limited number of pairwise interactions. Because the entire prediction function can be visually inspected, EBM is globally interpretable by design, and local explanations are exact rather than approximate. The argument is carried by demonstrating that this additive structure retains most of the predictive power of tree-based black-box ensembles; the experiments hinge on a moving-window cross-validation over years 2014–2018 with 2019 held out as test, and on a post-processing threshold method that equalizes false-positive rates ac","core_discovery":"The paper's central claim is that the accuracy-interpretability tradeoff is real but small: in a realistic time-series validation on Swiss public-employment data, the interpretable EBM comes within a hair's breadth of XGBoost's performance while remaining globally interpretable. It further claims that interpretability can be actively enhanced—by removing unimportant features via backward selection, by smoothing numerical feature functions with cubic splines to eliminate overfitting artifacts, and by applying group-fairness post-processing to equalize false positives across age groups—without incurring more than minor predictive losses. The paper presents this as a proof-of-concept that inter","pith_inferences":["If the 0.003 AUC gap generalizes, the practical choice between EBM and XGBoost on tabular social data should be driven by transparency needs, not accuracy; the common belief that black-box models are needed for high stakes may be largely unfounded.","The stated leakage risk suggests a concrete extension: re-running the comparison on a subset of features whose collection time is fully documented would test whether the near-parity holds once all outcome information is verifiably removed.","The fairness experiment is deliberately minimal (one protected attribute, one metric); a natural next step is evaluating intersectional attributes such as age and nationality together, using in-processing methods rather than only threshold post-processing.","Because the underlying data are not public, independent replication would require obtaining the same data from the authorities; a parallel benchmark on public administrative datasets with similar structure could provide a cost-effective external check on the near-parity claim."],"forward_implications":["Public employment services that currently rely on black-box models such as random forests could adopt EBM and obtain comparable or better predictive performance with complete transparency.","EBM's global feature functions allow caseworkers to justify individual risk scores to jobseekers, supporting algorithmic recourse and trust.","Sparse EBMs retaining only 30 or 45 main features keep most of the full model's AUC, indicating that profiling can be made simpler without large accuracy losses.","Smoothing numerical feature functions removes overfitting artifacts and reduces one form of individual unfairness, since similar jobseekers then receive similar feature contributions.","Fairness post-processing can equalize false-positive rates across age groups while preserving interpretability, though the true-positive rate drops from 80% to 66%—an explicit accuracy-fairness tradeoff."],"fun_headline_variants":["Interpretable AI rivals black-box for jobless risk","Swiss data: transparent models nearly match black-box","Explainable AI keeps pace with black-box profiling","Fair, interpretable model close behind black-box on Swiss data","Transparent AI nearly as accurate as opaque model in employment study"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that any temporal data leakage affects all compared models equally, so the near-parity between EBM and XGBoost would survive a clean-data re-run; if leakage is model-specific, the measured AUC gap could be misleading.","fun_headline_variants_meta":{"raw":{"variants":["Interpretable AI rivals black-box for jobless risk","Swiss data: transparent models nearly match black-box","Explainable AI keeps pace with black-box profiling","Fair, interpretable model close behind black-box on Swiss data","Transparent AI nearly as accurate as opaque model in employment study"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000184,"raw_usage":{"total_tokens":1115,"prompt_tokens":662,"completion_tokens":453,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":406,"completion_tokens_details":{"reasoning_tokens":374}},"tokens_in":406,"tokens_out":453,"duration_ms":6184,"temperature":1.0,"reasoning_tokens":374,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T16:42:07.223126+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Obtain the same Swiss administrative data, conduct a strict audit of each feature's collection time, and exclude any feature whose values could have been known at the prediction time; if EBM's test-year AUC then falls substantially below XGBoost's (e.g., a gap above 0.02), the paper's 'nearly as good' claim is refuted.","supporting_citations":[],"review_version":1}