Pith. sign in

REVIEW 4 major objections 5 minor 16 references

Transparent and Fair Profiling in Employment Services: Evidence from Switzerland

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Pith's one-line read Interpretable models can profile long-term unemployment risk almost as accurately as black-box models, with far greater transparency.

desk verdict A credible proof-of-concept that explainable boosting machines come close to XGBoost for long-term unemployment profiling, but the headline needs error bars and a leakage check before I'd trust the gap. read the letter →

arxiv 2509.11847 v1 pith:4IBED5AE submitted 2025-09-15 cs.LG cs.CY

classification cs.LGcs.CY
keywords long-termunemploymentinterpretablemachinelearningexplainableboostingmachinesfairnessprofilingSwitzerlandadministrativedatapredictiveperformance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether transparent machine-learning models can replace opaque black-box models for profiling long-term unemployment (LTU) risk. Using Swiss administrative records on about a million jobseekers, it compares logistic regression and explainable boosting machines (EBM) against random forests, gradient boosting, and XGBoost. The central result is that EBM performs nearly as well as the best black-box model: on the 2019 test year, EBM achieves an AUC of 0.7593 versus XGBoost's 0.7619, a gap of only 0.003. The paper also shows that making EBM sparser and smoothing its feature functions costs little accuracy, and that post-processing can equalize false-positive rates across age groups without destroying interpretability. If true, this means employment services could deploy accountable, understandable profiling without sacrificing much predictive power.

What carries the argument

The central object is the Explainable Boosting Machine (EBM), a generalized additive model whose prediction function is a sum of univariate feature functions plus a limited number of pairwise interactions. Because the entire prediction function can be visually inspected, EBM is globally interpretable by design, and local explanations are exact rather than approximate. The argument is carried by demonstrating that this additive structure retains most of the predictive power of tree-based black-box ensembles; the experiments hinge on a moving-window cross-validation over years 2014–2018 with 2019 held out as test, and on a post-processing threshold method that equalizes false-positive rates ac

What would settle it

Obtain the same Swiss administrative data, conduct a strict audit of each feature's collection time, and exclude any feature whose values could have been known at the prediction time; if EBM's test-year AUC then falls substantially below XGBoost's (e.g., a gap above 0.02), the paper's 'nearly as good' claim is refuted.

Watch

Extended reading notes

Core claim

The paper's central claim is that the accuracy-interpretability tradeoff is real but small: in a realistic time-series validation on Swiss public-employment data, the interpretable EBM comes within a hair's breadth of XGBoost's performance while remaining globally interpretable. It further claims that interpretability can be actively enhanced—by removing unimportant features via backward selection, by smoothing numerical feature functions with cubic splines to eliminate overfitting artifacts, and by applying group-fairness post-processing to equalize false positives across age groups—without incurring more than minor predictive losses. The paper presents this as a proof-of-concept that inter

Load-bearing premise

The load-bearing premise is that any temporal data leakage affects all compared models equally, so the near-parity between EBM and XGBoost would survive a clean-data re-run; if leakage is model-specific, the measured AUC gap could be misleading.

Editorial extensions

If this is right

  • Public employment services that currently rely on black-box models such as random forests could adopt EBM and obtain comparable or better predictive performance with complete transparency.
  • EBM's global feature functions allow caseworkers to justify individual risk scores to jobseekers, supporting algorithmic recourse and trust.
  • Sparse EBMs retaining only 30 or 45 main features keep most of the full model's AUC, indicating that profiling can be made simpler without large accuracy losses.
  • Smoothing numerical feature functions removes overfitting artifacts and reduces one form of individual unfairness, since similar jobseekers then receive similar feature contributions.
  • Fairness post-processing can equalize false-positive rates across age groups while preserving interpretability, though the true-positive rate drops from 80% to 66%—an explicit accuracy-fairness tradeoff.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the 0.003 AUC gap generalizes, the practical choice between EBM and XGBoost on tabular social data should be driven by transparency needs, not accuracy; the common belief that black-box models are needed for high stakes may be largely unfounded.
  • The stated leakage risk suggests a concrete extension: re-running the comparison on a subset of features whose collection time is fully documented would test whether the near-parity holds once all outcome information is verifiably removed.
  • The fairness experiment is deliberately minimal (one protected attribute, one metric); a natural next step is evaluating intersectional attributes such as age and nationality together, using in-processing methods rather than only threshold post-processing.
  • Because the underlying data are not public, independent replication would require obtaining the same data from the authorities; a parallel benchmark on public administrative datasets with similar structure could provide a cost-effective external check on the near-parity claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper empirically compares interpretable and black-box machine learning models for long-term unemployment (LTU) risk prediction using Swiss administrative unemployment data from 2014–2019. The models are logistic regression (LR), explainable boosting machines (EBM), random forests (RF), gradient boosting (GB), and XGBoost (XGB). Using a moving-window cross-validation design, the paper reports AUC for years 2015–2019, with 2019 held out as the test year. The main empirical claim is that EBM, an inherently interpretable model, achieves predictive performance close to XGB (test 2019 AUC: EBM 0.7593 vs. XGB 0.7619, Table 5). The paper also investigates sparsity via backward selection, smoothing of EBM feature functions, and post-hoc fairness mitigation using the Hardt et al. (2016) equalized-odds post-processing method. The paper argues that interpretable profiling is a viable, accountable alternative to black-box profiling. The manuscript acknowledges in Sec. 6.2 that the temporal order of data collection is unknown for at least some features, creating a possible data leakage risk, but asserts without empirical evidence that leakage would affect all models qualitatively similarly.

Significance. If the central empirical finding is robust, the paper makes a useful contribution to the interpretable-ML and public-employment-services literature by showing that an interpretable additive model can approach the accuracy of a state-of-the-art tree ensemble on a large administrative dataset. The study has several strengths: it uses full administrative records with roughly 165,000 observations per year, employs a time-series-aware evaluation scheme, and releases code for reproduction on GitHub. The fairness analysis, while exploratory, demonstrates a concrete post-processing approach compatible with interpretable models. The significance is qualified by the unresolved leakage question and the lack of uncertainty quantification, both of which are load-bearing for the headline claim. The paper does not present a new algorithm or derivation; its value is as an applied empirical comparison.

major comments (4)
  1. [§6.2, App. A.1; features beitragsmonate_vor_rf, taggeld_anspr_bins, vers_verdienst] The leakage concern is acknowledged in Sec. 6.2 but the paper's response is an untested assertion: 'data leakage would affect all models, even though there may be quantitative differences.' This is load-bearing because the headline result is a quantitative AUC difference (test 2019: EBM 0.7593 vs. XGB 0.7619, Table 5). Features such as beitragsmonate_vor_rf, taggeld_anspr_bins, and vers_verdienst are derived from the monthly insurance-payment system (asal, App. A.1), and the outcome is also defined from monthly benefit receipt over the first 12 months. If the measurement windows of these features overlap the outcome window, they can directly encode the outcome. The claim that such leakage affects all models comparably is not supported by any experiment. Please provide a sensitivity analysis, e.g., rerun the comparisons on a feature set restricted to variables determined before the unempl
  2. [§5.1, Table 5, Fig. 1] All AUC comparisons are reported as point estimates without confidence intervals, bootstrap errors, or significance tests. For example, the validation-fold EBM-minus-XGB gaps are 0.0069, 0.0065, 0.0076, and 0.0090, while the test gap is 0.0026. It is impossible to assess whether 'nearly as well' is a statistically and practically meaningful statement. Moreover, the test conclusion rests on a single year (2019). Please provide uncertainty quantification for the AUC differences (e.g., bootstrap or DeLong intervals) and explicitly state whether the observed test-year gap is distinguishable from zero at the available sample size. If possible, also evaluate on additional held-out years or report repeated moving-window test evaluations.
  3. [Appendix B, §5.1] The model comparison is not fully controlled. XGB hyperparameters were tuned with Optuna, while EBM's key hyperparameter, the number of interactions, was manually set to a low value (30) for interpretability reasons. RF and GB hyperparameters are taken from an unpublished thesis (Gasser 2022). The manuscript states a 'fair' comparison, but this is not established: a manually constrained EBM could understate or overstate the gap depending on how interactions affect performance. Please report a sensitivity analysis over EBM interaction counts (e.g., 0, 10, 30, 60) and state whether the near-parity conclusion holds. Hyperparameters borrowed from non-public sources should be fully documented in the appendix or code so the results are reproducible without access to the thesis.
  4. [§5.4, Fig. 6, Abstract] The abstract claims that fairness mitigation can enhance transparency and fairness 'with only minor losses in performance.' The reported mitigation reduces TPR from 80% to 66% (Fig. 6d), a 14-percentage-point drop, which the paper itself describes as making the predictor 'not as efficient.' This is not a minor loss at the original operating point unless some other performance measure, such as accuracy, is the criterion. The paper reports that accuracy increases after mitigation, but the relevant trade-off depends on the cost structure of false negatives versus false positives, which is not specified. In addition, the post-processing equalizes FPR under a balanced-accuracy constraint, while the text and figures emphasize normalized false positives; the mismatch between the targeted fairness quantity and the displayed quantity should be clarified.
minor comments (5)
  1. [§3.1] The phrase 'train-test data (years 2018-2019)' is confusing; the test set is only year 2019 according to Table 2. Please rephrase to 'test data (year 2019)'.
  2. [§5.3] The text says 'smoothing was applied to individual, off-the-shelf numerical feature functions of one sparse EBM-30 in the first fold' but does not say whether the smoothing parameters were selected on that fold's validation data or on the training data. Please state this explicitly to avoid an optimistic-bias concern.
  3. [§5.2, Fig. 8] The backward-selection figure caption says 'the rightmost model, with 57 main features and 30 interactions, is the full EBM,' but the main text refers to the full EBM with 57 features in Sec. 4.4 as well. Please ensure the labeling is consistent between the figure, caption, and text.
  4. [§2.2] The claim that this is the first empirical investigation of interpretable LTU profiling should be softened or qualified, since the paper builds on Gasser (2022, 2023) and there is related work on EBM for social outcomes. A more precise statement would be 'the first comparison of EBM to black-box models in LTU profiling with Swiss administrative data.'
  5. [Appendix D.1] The data availability statement says data can be obtained from SECO, but the manuscript relies on two non-public theses (Gasser 2022, 2023) for parts of the data construction and hyperparameters. Please ensure all necessary preprocessing decisions are described in the paper or in the released code, so that a third party with data access can reproduce the clean dataset without consulting the theses.

Circularity Check

0 steps flagged · score 2.0 of 10

No circularity: the EBM-vs-XGB result is a held-out empirical comparison, not a derived quantity.

full rationale

The paper's central claim is an empirical benchmark result: EBM reaches AUC 0.7593 vs XGB 0.7619 on the held-out 2019 test set (Table 5). This is a measured comparison on data not used for training or hyperparameter tuning, not a quantity derived from a fitted parameter, a definition, or an equation. The outcome is defined directly from benefit records, and the features come from separate administrative sources; no construction makes one model's test score equal to another's or to a training fit. The acknowledged possibility of temporal data leakage (Sec. 6.2) is a data-quality limitation that could affect the comparison, but the paper does not build the leakage assumption into the reported AUCs, so this is not circularity. The paper cites the author's own prior work (e.g., Räz 2021, 2024a) and the non-public Gasser theses for outcome definition, hyperparameters, and interpretability framing; these are inputs to the experiment, not conclusions that the experiment is claimed to derive. Consequently, no load-bearing step reduces to its own input, and the central comparison is self-contained as an empirical evaluation.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central empirical result is a model comparison. It relies on the outcome definition, on the assumption of limited data leakage, and on several user-chosen hyperparameters (EBM interactions, smoothing parameters, TPR threshold). These are reasonable but add degrees of freedom to the experiments.

free parameters (5)
  • EBM interaction count = 30
    Set by hand to a low value to preserve interpretability; the paper acknowledges this trades off some performance (Appendix B).
  • EBM learning rate = 0.0014
    Tuned via Optuna; the paper notes tuning is expensive for EBM (Appendix B).
  • LR L1 regularization = 0.1
    Chosen to balance sparsity (115/120 non-zero features) and AUC (Sec. 5.2).
  • Smoothing parameter lambda = Per feature
    Chosen individually for five numerical features based on a tradeoff between smoothing and AUC; values are not reported (Sec. 5.3).
  • TPR threshold = 0.8
    Chosen for the fairness scenario to target 80% of true positives, motivated by Gasser (2022) (Sec. 4.5).
assumptions (5)
  • domain assumption A jobseeker is LTU if they receive unemployment benefits during each of the first 12 months of unemployment.
    Defines the outcome variable (Sec. 3.1). Different definitions could change the learning problem.
  • domain assumption The administrative features are temporally compatible with the outcome despite unknown data-collection order.
    The paper flags possible leakage (Sec. 6.2) but assumes it affects all models comparably.
  • standard math ROC-AUC is the appropriate metric for model comparison.
    Used throughout; reasonable for imbalanced outcomes and threshold-independent ranking.
  • domain assumption Equalized false-positive rates across age groups is a relevant fairness criterion.
    A normative choice in the fairness mitigation scenario (Sec. 4.5); the paper notes fairness is context dependent.
  • ad hoc to paper Local fluctuations in EBM feature functions are artifacts of overfitting.
    Classified by a domain expert rather than by a formal criterion (Sec. 5.3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Transparent and Fair Profiling in Employment Services: Evidence from Switzerland." pith.science (2026). https://pith.science/paper/4IBED5AE

@misc{pith2026250911847,
  author       = {Pith},
  title        = {Pith review of: Transparent and Fair Profiling in Employment Services: Evidence from Switzerland},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4IBED5AE}},
  note         = {Machine review of arXiv:2509.11847}
}
read the original abstract

Long-term unemployment (LTU) is a challenge for both jobseekers and public employment services. Statistical profiling tools are increasingly used to predict LTU risk. Some profiling tools are opaque, black-box machine learning models, which raise issues of transparency and fairness. This paper investigates whether interpretable models could serve as an alternative, using administrative data from Switzerland. Traditional statistical, interpretable, and black-box models are compared in terms of predictive performance, interpretability, and fairness. It is shown that explainable boosting machines, a recent interpretable model, perform nearly as well as the best black-box models. It is also shown how model sparsity, feature smoothing, and fairness mitigation can enhance transparency and fairness with only minor losses in performance. These findings suggest that interpretable profiling provides an accountable and trustworthy alternative to black-box models without compromising performance.

Figures

Figures reproduced from arXiv: 2509.11847 by the authors.

Figure 1
Figure 1. Performance Score (AUC) for five folds. Years 2015-2018: validate; year 2019: test. [PITH_FULL_IMAGE:figures/full_fig_p012_1.png] view at source ↗
Figure 2
Figure 2. Performance Score (AUC) for XGB, full EBM, LR, as well as three sparse EBMs [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗
Figure 3
Figure 3. Numerical feature ‘insured income’ (vers_verdienst), in raw version (off-the-shelf [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Performance Score (AUC) for original EBM-30 (sparse EBM with 30 most impor [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]
Figure 5
Figure 5. Figure 5: Confusion matrices (normalized) for three age groups, and overall performance [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Confusion matrices (normalized) for three age groups, and overall performance [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: Effect of L1 regularization on number of non-zero features and predictive perfor￾mance of LR. Note that an AUC of 0.5 corresponds to random predictions (uninformative model). 31 [PITH_FULL_IMAGE:figures/full_fig_p031_7.png]
Figure 8
Figure 8. Figure 8: Performance Score (AUC) for EBMs obtained from backward selection, retaining [PITH_FULL_IMAGE:figures/full_fig_p032_8.png]
Figure 9
Figure 9. Figure 9: Global explanation (feature importance plot) generated by EBM interface. It shows [PITH_FULL_IMAGE:figures/full_fig_p032_9.png]
Figure 10
Figure 10. Figure 10: Numerical feature ‘age’ (alter), in raw version (off-the-shelf EBM) and after ap [PITH_FULL_IMAGE:figures/full_fig_p033_10.png]
Figure 11
Figure 11. Figure 11: Numerical feature ‘number of months of previous contributions’ [PITH_FULL_IMAGE:figures/full_fig_p033_11.png]
Figure 12
Figure 12. Figure 12: Numerical feature ‘rate of desired employment’ (vermittlungsgrad_asal), in raw [PITH_FULL_IMAGE:figures/full_fig_p034_12.png]
Figure 13
Figure 13. Figure 13: Example of a local explanation generated by EBM interface for a full EBM. Fic [PITH_FULL_IMAGE:figures/full_fig_p034_13.png]
Figure 14
Figure 14. Figure 14: Cumulative gains curve for EBM-30 model. This curve can serve as a basis for [PITH_FULL_IMAGE:figures/full_fig_p035_14.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

16 extracted references · 7 linked inside Pith

  1. [5]

    Arbeitsmarkt / Arbeitslosenversicherung

    For eachn, a fraction of0.5interactions of the total number of main effects were added; the rightmost model, with 57 main features and 30 interactions, is the full EBM. Figure 9: Global explanation (feature importance plot) generated by EBM interface. It shows the (average) feature importance of the 15 most important features for fold 1 (trained on data f...

  2. [10]

    Ravikumar, P., J

    Interpretml: A unified framework for machine learning interpretability.arXiv preprint arXiv:1909.09223. Ravikumar, P., J. Lafferty, H. Liu, and L. Wasserman

  3. [11]

    InProceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21

    Group Fairness: Independence Revisited. InProceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21. New York, NY, USA: Association for Computing Machinery, pp. 129–137. ———. 2024a. ML interpretability: Simple isn’t easy.Studies in History and Philosophy of Science103: 159–67. ArXiv:2211.13617. ———. 2024b. Reliability...

  4. [12]

    ArXiv:1703.00810

    Opening the black box of Deep Neural Networks via Information. ArXiv:1703.00810. Van den Berg, G. J., M. Kunaschk, J. Lang, G. Stephan, and A. Uhlendorff

  5. [13]

    Weerts, H., M

    Gam changer: Editing generalized additive models with interactive visualization.arXiv preprint arXiv:2112.03245. Weerts, H., M. Dudík, R. Edgar, A. Jalali, R. Lutz, and M. Madaio

  6. [14]

    InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems

    Explaining it your way-findings from a co-creative design workshop on designing XAI applications with AI end-users from the public sector. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. pp. 1–14. Zezulka, S.andK.Genin.2024. Fromthefairdistributionofpredictionstothefairdistribution of social goods: Evaluating the impact of ...

  7. [2015]

    Barocas, S., M

    Identification of latest trends and current developments in methods to profile jobseekers in European Public Employment Services . Barocas, S., M. Hardt, and A. Narayanan. 2022.Fairness and Machine Learning. fairml- book.org. Biran, O. and C. Cotton

  8. [2016]

    Advances in Neural Information Processing Systems

    Equality of Opportunity in Supervised Learning. Advances in Neural Information Processing Systems. Hastie, T. and R. Tibshirani. 1990.Generalized Additive Models. No. 43 in Monographs on Statistics and Applied Probability. Chapman & Hall/CRC. Hastie, T., R. Tibshirani, and J. Friedman. 2009.The Elements of Statistical Learning. Springer Series in Statisti...

Show all 16 references
  1. [2017]

    The Stanford Encyclopedia of Philosophy

    Justice. The Stanford Encyclopedia of Philosophy. Molnar, C. 2020.Interpretable machine learning. Lulu.com. 24 Nori, H., S. Jenkins, P. Koch, and R. Caruana

  2. [2018]

    ArXiv:1606.03490

    The Mythos of Model Interpretability.Queue16(3): 31–57. ArXiv:1606.03490. Lou, Y., R. Caruana, J. Gehrke, and G. Hooker

  3. [2019]

    InProceedings of the 25th ACM SIGKDD inter- national conference on knowledge discovery & data mining

    Optuna: A next-generation hyperparameter optimization framework. InProceedings of the 25th ACM SIGKDD inter- national conference on knowledge discovery & data mining. pp. 2623–2631. Allhutter, D., A.Mager, F.Cech, F.Fischer, andG.Grill.2020. DERAMS-ALGORITHMUS. Eine Soziotechn...

  4. [2020]

    Evaluating time series forecasting models: An empirical study on performance estimation methods.Machine Learning109(11): 1997–

  5. [2021]

    Körtner, J

    Fairness in algorithmic profiling: A German case study.arXiv preprint arXiv:2108.04134. Körtner, J. and G. Bonoli

  6. [2022]

    ” beruf int. Y N berufskl_3 # jobs in class 3 “

    Proceed with caution.Canadian Journal of Philosophy52(1): 6–25. 26 Appendix A Data A.1 Raw Datasets The following is a qualitative description of the raw datasets used to construct the clean dataset.The same raw datasets have been used previously in (Gasser, 2022, 2023). For a...

  7. [2023]

    Gasser, M

    (in) Accuracy in Algorithmic Profiling of the Unemployed– An Exploratory Review of Reporting Standards.Social Policy and Society: 1–14. Gasser, M. 2022.Interpretable rule-ensemble methods in the analysis of long-term unemploy- ment. Semester paper, ETH Zürich. 23 ———. 2023.Fai...

  8. [2024]

    Goodman, B

    Long-term fairness inquiries and pursuits in machine learning: A survey of notions, methods, and challenges.arXiv preprint arXiv:2406.06736. Goodman, B. and S. Flaxman

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.