REVIEW 5 major objections 4 minor 29 references
Non-Heuristic Selection via Hybrid Regularized and Machine Learning Models for Insurance
T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper argues that combining Lasso-style regularized feature selection with gradient-boosting classifiers yields competitive, interpretable travel-insurance purchase models, with CatBoost on Lasso-selected features reaching an AUC of…
desk verdict Headline AUC/F1 numbers are inflated by target-derived features computed before the split; the pipeline is standard, so the paper's empirical claims do not survive scrutiny. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the two-stage hybrid pipeline itself: a penalized binomial logistic regression whose coefficients, shrunk by an L1 (Lasso), L2 (Ridge), or combined (Elastic Net) penalty, define the selected feature subset, followed by an ensemble classifier trained only on that subset. Lasso in particular zeroes out coefficients, which is what turns the selection step into a sparse, auditable list of predictors. Both stages are tuned by random search with stratified k-fold cross-validation that maximizes AUC, and the authors identify the CatBoost-on-Lasso pairing as the configuration where the sparse selection and the nonlinear classifier reinforce each other.
What would settle it
Re-run the CatBoost–Lasso pipeline with these two target-derived features either removed or recomputed within each training fold on the test set; if the AUC drops appreciably from 0.861 toward the roughly 0.81 level of the regularized models alone, the headline result is an artifact of target leakage rather than a property of the hybrid strategy.
Extended reading notes
Core claim
The central claim is that variable selection through a regularized logistic regression, followed by training a nonlinear classifier on the selected variables, is an effective non-heuristic recipe for predicting travel-insurance purchase. In the empirical comparison, the combination of Lasso selection and a CatBoost classifier gave the best hybrid performance on the held-out test set: AUC 0.8611, precision 0.8024, recall 0.8141, and F1-score 0.8082, ahead of the other fourteen regularizer-classifier combinations tested. The paper presents this as a way to keep the interpretability of the estimation process—the selection stage yields readable coefficients—while recovering most of the predictive power of full black-box models, which reached AUC 0.9023 for LightGBM on all 35 engineered variables. Its simulation study, using the Friedman regression function with sample sizes n = 200, 500, 1000 and with 5, 10, or 50 predictors including pure noise, shows hybrid models matching or beating both pure black-box and purely regularized models in RMSE across settings.
Load-bearing premise
The reported scores hold only if the engineered features MovingAvgInsurance and ClusterInsuranceRate—averages of the purchase target computed by age group and by cluster before the data were split—carry no test-set information into the predictors.
Editorial extensions
If this is right
- Insurers can deploy the CatBoost–Lasso pipeline as a documented, auditable model whose selection stage yields readable coefficients alongside an AUC of 0.861 and an F1 of 0.808.
- The simulation results indicate the hybrid strategy keeps its advantage when many predictors are irrelevant noise, which is the typical high-dimensional regime for customer data.
- Because the pipeline reduces the 35 engineered variables to a smaller Lasso-selected subset, it cuts storage and inference costs while retaining most of the predictive power of full black-box models.
- Choosing between hybrid and pure black-box models becomes a stated trade: roughly 0.04 of AUC (0.861 vs. 0.902) is the price of interpretability, dimensionality reduction, and faster scoring.
Reading between the lines
- The two engineered features MovingAvgInsurance and ClusterInsuranceRate are group averages of the target variable computed before the data split; re-running the pipeline with these features recomputed inside each training fold would test whether the reported gains are leakage-free—a check the paper does not include.
- Adding a plain logistic regression on the same Lasso-selected features as a baseline would isolate how much of the hybrid's gain comes from CatBoost's nonlinearity rather than from the selection step alone.
- The hybrid recipe transfers naturally to other supervised insurance problems—claim frequency, lapse, cross-sell—where regulators require interpretable inputs; the paper's own simulation supports the pattern being dataset-independent.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes a hybrid pipeline for predicting travel insurance purchase: it engineers 35 features from 10 original variables, selects features with Ridge/Lasso/Elastic Net, trains five ensemble classifiers on the selected features, and evaluates them with stratified 5-fold cross-validation. On a 2,697-row open dataset it reports CatBoost combined with Lasso as the best hybrid model (AUC = 0.861, F1 = 0.808), and a simulation study on Friedman's regression function is used to argue generalizability. The authors claim that the hybrid approach reconciles high predictive power with interpretability.
Significance. The applied comparison of regularized feature selection with ensemble classifiers is potentially useful, and the authors provide a GitHub repository with scripts and a simulation study, which are strengths. However, the central empirical claim is invalid as presented because two engineered features are direct aggregates of the response computed before data partitioning, so every reported classification metric is contaminated by target leakage. The undisclosed 'empirically applied weights' and threshold choices also undermine the 'non-heuristic' claim in the title. If the analysis were redone with a leakage-free protocol and fully disclosed feature construction, the paper could make a modest contribution, but the current results cannot be evaluated as evidence for the stated conclusions.
major comments (5)
- [Section 2 and Supplementary Table 6] Supplementary Table 6 defines MovingAvgInsurance as 'Moving average of insurance uptake by age group' and ClusterInsuranceRate as 'Average insurance uptake rate per cluster,' while the target variable is TravelInsurance. Section 2 states that feature engineering, including these variables, was completed before the stratified data partition ('After this stage of generating the transformed database, a systematic training and validation process was implemented, using stratified data partitioning via k-fold'). These predictors are therefore functions of the response values of the full dataset, including test observations. Consequently, the test-set metrics in Tables 3-5, including the headline CatBoost+Lasso AUC = 0.861 and F1 = 0.808, are inflated and do not measure generalization. Moreover, the paper provides no deployment procedure for computing a new customer's age-group or cluster average of the target at prediction time, so these features are not usable in the claimed application. This is an internal inconsistency with the validation protocol described in Algorithm 2, and it invalidates the central empirical claim.
- [Section 2 and Algorithm 2] Algorithm 2 states in Step 1 'Split D into D_train and D_test' and then performs feature selection on D_train, which would be acceptable only if all feature engineering were performed inside the training folds. Section 2, however, explicitly describes feature engineering before partitioning. The two descriptions cannot both be true. If Algorithm 2 is the intended protocol, the text must be corrected and all engineered features must be recomputed within each training fold; if the Section 2 text is intended, the leakage described in the previous comment applies to all reported results. This ambiguity must be resolved before the reported metrics can be interpreted.
- [Section 2 and Supplementary Table 6] The 'empirically applied weights' for InsuranceScore and RiskScore, as well as the thresholds for HighIncome, HighIncome90, HighChronicDiseases, LargeFamily, LowDependence, the boundaries of AgeGroup, and the number of clusters k in k-means, are not disclosed or justified. These are free parameters chosen on the full dataset, and the paper's title claim of 'Non-Heuristic Selection' is not supported unless these choices are derived within the validation loop or demonstrated to be insensitive. A sensitivity analysis or nested cross-validation is needed before the 'non-heuristic' claim can be accepted.
- [Table 4] Table 4 reports hyperparameters that are invalid for the named algorithms: LightGBM with max_depth = 337 and eta = 52, XGBoost with max_depth = 394 and num_leaves = 3, and H2O GBM with num_leaves = 4 and learning_rate = 139. These values appear to be misaligned or mislabeled columns, which prevents independent replication and calls into question the reliability of the table's other entries, including the reported performance metrics.
- [Section 5] The simulation study in Section 5 uses the Friedman regression function (Eq. (1)) with a continuous response and RMSE, whereas the empirical study is a binary classification problem evaluated with AUC and F1. The simulation therefore does not demonstrate generalizability of the classification pipeline, nor does it address the target-leakage problem identified above. The statement in Section 5.2 that the results are generalizable is unsupported by the presented experiments.
minor comments (4)
- [Table 4] The column header 'Hiperparameters' is misspelled, and the text describing Table 4 states that H2O GBM with Elastic Net achieved the highest recall (0.8183), but the table reports recall = 0.8100 for that row.
- [References] Reference [13] and reference [21] are duplicates of the same 'Regularization paths for generalized linear models via coordinate descent' paper, and the citation numbering should be consolidated.
- [Table 3] For Random Forest, the hyperparameters are listed as 'iterations = 185, mtry = 10'; 'iterations' is not a standard Random Forest parameter in the packages cited, and 'mtry' is more commonly associated with the randomForest package, so the table should specify the exact software interface.
- [General] The paper alternates between 'Lasso' and 'LASSO'; the notation should be standardized throughout, including in Tables 4 and 5.
Circularity Check
Target-derived features MovingAvgInsurance and ClusterInsuranceRate leak the response into the predictors, so the headline CatBoost+Lasso AUC/F1 reduces to smoothed target values.
-
self definitional
[Section 2 (Feature engineering, before data partitioning); Supplementary Table 6; Table 4/5]
"Finally, aggregated information based on similar groups was added, such as cluster scores generated by grouping algorithms and moving averages by age groups, enriching the modeling with collective references. ... After this stage of generating the transformed database, a systematic training and validation process was implemented, using stratified data partitioning via k-fold (5 folds). ... MovingAvgInsurance Moving average of insurance uptake by age group; ClusterInsuranceRate Average insurance uptake rate per cluster."
MovingAvgInsurance and ClusterInsuranceRate are constructed from the response variable (insurance uptake) before the train/test split. Thus each test observation's predictor values encode its own label and the labels of other test observations. The paper's central empirical claim, CatBoost+Lasso with AUC=0.8611 and F1=0.8082, is obtained using these target-derived features, so the reported performance is not a prediction from independent covariates but a partially smoothed version of the target itself. This reduces the headline result to a self-referential construction: the outcome is used to define the predictors that then 'predict' the outcome.
full rationale
The only load-bearing circularity is the leakage of the response into two engineered covariates. The paper defines MovingAvgInsurance and ClusterInsuranceRate as averages of insurance uptake, then performs feature engineering before the stratified split, so test-fold labels enter the predictor matrix. Every metric in Tables 3 and 4, including the abstract's headline AUC and F1, is therefore contaminated and likely inflated. This is a direct, quotable reduction of the prediction to the target, not a mere methodological disagreement. The simulation study in Section 5 is independent and does not rely on leaked features, and the paper does not lean on self-citations or imported uniqueness theorems, so the circularity is partial rather than total. Nevertheless, the central empirical claim of the paper is not self-contained: its best-performing model uses target-smoothed inputs, making the reported predictive advantage partly an artifact of construction.
Assumptions & free parameters
free parameters (6)
- InsuranceScore weights =
not reported (empirically applied)
- RiskScore weights =
not reported
- k-means cluster count (k) =
not reported
- Percentile thresholds =
0.75, 0.90
- Family size threshold =
3
- Age group boundaries =
young, adult-y, H-age, senior (exact cutoffs not reported)
assumptions (3)
- domain assumption All engineered features are computed without using test-set target information.
- domain assumption The Friedman function simulation is representative of the insurance classification problem.
- ad hoc to paper The empirically applied weights for score variables are valid and not overfit.
Cite this review
Pith. "Pith review of Non-Heuristic Selection via Hybrid Regularized and Machine Learning Models for Insurance." pith.science (2026). https://pith.science/paper/KOGXWSK3
@misc{pith2026250605609,
author = {Pith},
title = {Pith review of: Non-Heuristic Selection via Hybrid Regularized and Machine Learning Models for Insurance},
year = {2026},
howpublished = {\url{https://pith.science/paper/KOGXWSK3}},
note = {Machine review of arXiv:2506.05609}
}
read the original abstract
In this study, machine learning models were tested to predict whether or not a customer of an insurance company would purchase a travel insurance product. For this purpose, secondary data provided by an open-source website that compiles databases from statistical modeling competitions were used. The dataset used presents approximately 2,700 records from an unidentified company in the tourism insurance sector. Initially, the feature engineering stage was carried out, which were selected through regularized models: Ridge, Lasso and Elastic-Net. In this phase, gains were observed not only in relation to dimensionality, but also in the maintenance of interpretative capacity, through the coefficients obtained. After this process, five classification models were evaluated (Random Forests, XGBoost, H2O GBM, LightGBM and CatBoost) separately and in a hybrid way with the previous regularized models, all these stages using the k-fold stratified cross-validation technique. The evaluations were conducted by traditional metrics, including AUC, precision, recall and F1 score. A very competitive hybrid model was obtained using CatBoost combined with Lasso feature selection, achieving an AUC of 0.861 and an F1 score of 0.808. These findings motivate us to present the effectiveness of using hybrid models as a way to obtain high predictive power and maintain the interpretability of the estimation process
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
Procura por seguro viagem dis- para no primeiro bimestre de 2024,
C. N. das Seguradoras (CNseg), “Procura por seguro viagem dis- para no primeiro bimestre de 2024, ” https://cnseg.org.br/noticias/ procura-por-seguro-viagem-dispara-no-primeiro-bimestre-de-2024, 2024, acesso em: maio 2025
work page 2024
-
[2]
Tendências na indústria de seguros 2023,
Accenture, “Tendências na indústria de seguros 2023, ” https://www. accenture.com/content/dam/accenture/final/accenture-com/document/ Accenture-Insurance-Trends-2023-Brazil-V2.pdf, 2023, acesso em: maio 2025
work page 2023
-
[3]
S. T. Lim, J. Y. Yuan, K. W. Khaw, and X. Chew, “Predicting travel insurance pur- chases in an insurance firm through machine learning methods after covid-19, ” Journal of Informatics and Web Engineering, vol. 2, no. 2, pp. 43–58, 2023
work page 2023
-
[4]
X. Li, “Exploring the potential of machine learning techniques for predicting travel insurance claims: A comparative analysis of four models, ”Academic Journal of Com- puting & Information Science, vol. 6, no. 4, pp. 118–125, 2023
work page 2023
-
[5]
Insurance risk prediction using machine learning,
R. Sahai, A. Al-Ataby, S. Assi, M. Jayabalan, P. Liatsis, C. K. Loy, A. Al-Hamid, S. Al- Sudani, M. Alamran, and H. Kolivand, “Insurance risk prediction using machine learning, ”Lecture Notes in Networks and Systems, vol. 165, pp. 419–433, 2023
work page 2023
-
[6]
Z. Sun, “Research on changes in travel insurance premiums driven by climate change: A case study of hong kong region, ”Journal of Environmental Management, vol. 325, pp. 116–123, 2024
work page 2024
-
[7]
X. Tian, J. Todorovic, and Z. Todorovic, “A machine-learning-based business ana- lytical system for insurance customer relationship management and cross-selling, ” Journal of Applied Business and Economics, vol. 25, no. 6, pp. 256–270, 2023
work page 2023
-
[8]
L. Breiman, “Random forests, ”Machine Learning, vol. 45, no. 1, pp. 5–32, 2001
work page 2001
Show all 29 references
-
[9]
Xgboost: A scalable tree boosting system,
T. Chen and C. Guestrin, “Xgboost: A scalable tree boosting system, ” inProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2016, pp. 785–794
2016
-
[10]
Lightgbm: A highly effi- cient gradient boosting decision tree,
G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, and W. Ma, “Lightgbm: A highly effi- cient gradient boosting decision tree, ” inAdvances in Neural Information Processing Systems, vol. 30, 2017. 24
2017
-
[11]
glmnet: Lasso and elastic-net regularized generalized linear models,
J. Friedman, T. Hastie, and R. Tibshirani, “glmnet: Lasso and elastic-net regularized generalized linear models, ” 2024, r package version 4.1-5. [Online]. Available: https://cran.r-project.org/web/packages/glmnet/index.html
2024
-
[12]
R: A language and environment for statistical computing,
R. C. Team, “R: A language and environment for statistical computing, ” 2024, r Foundation for Statistical Computing, Vienna, Austria. [Online]. Available: https://www.R-project.org/
2024
-
[13]
Regularization paths for generalized lin- ear models via coordinate descent,
J. Friedman, T. Hastie, and R. Tibshirani, “Regularization paths for generalized lin- ear models via coordinate descent, ”Journal of Statistical Software, vol. 33, no. 1, pp. 1–22, 2010
2010
-
[14]
Regression shrinkage and selection via the lasso,
R. Tibshirani, “Regression shrinkage and selection via the lasso, ”Journal of the Royal Statistical Society: Series B (Methodological), vol. 58, no. 1, pp. 267–288, 1996
1996
-
[15]
Regularization and variable selection via the elastic net,
H. Zou and T. Hastie, “Regularization and variable selection via the elastic net, ” Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 67, no. 2, pp. 301–320, 2005
2005
-
[16]
Random search for hyper-parameter optimization,
J. Bergstra and Y. Bengio, “Random search for hyper-parameter optimization, ”Jour- nal of Machine Learning Research, vol. 13, no. 1, pp. 281–305, 2012
2012
-
[17]
Algorithms for hyper-parameter optimization,
J. Bergstra, R. Bardenet, Y. Bengio, and B. Kégl, “Algorithms for hyper-parameter optimization, ” inAdvances in Neural Information Processing Systems, vol. 24, 2011
2011
-
[18]
Ridge regression: Biased estimation for nonorthog- onal problems,
A. E. Hoerl and R. W. Kennard, “Ridge regression: Biased estimation for nonorthog- onal problems, ”Technometrics, vol. 12, no. 1, pp. 55–67, 1970
1970
-
[19]
A study of cross-validation and bootstrap for accuracy estimation and model selection,
R. Kohavi, “A study of cross-validation and bootstrap for accuracy estimation and model selection, ” inIJCAI, vol. 14, no. 2, 1995, pp. 1137–1145
1995
-
[20]
The use of the area under the roc curve in the evaluation of machine learning algorithms,
A. P. Bradley, “The use of the area under the roc curve in the evaluation of machine learning algorithms, ”Pattern recognition, vol. 30, no. 7, pp. 1145–1159, 1997
1997
-
[21]
Regularization paths for generalized lin- ear models via coordinate descent,
J. Friedman, T. Hastie, and R. Tibshirani, “Regularization paths for generalized lin- ear models via coordinate descent, ”Journal of statistical software, vol. 33, no. 1, pp. 1–22, 2010
2010
-
[22]
Hastie, R
T. Hastie, R. Tibshirani, and J. Friedman,The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed. New York: Springer, 2009
2009
-
[23]
James, D
G. James, D. Witten, T. Hastie, and R. Tibshirani,An Introduction to Statistical Learn- ing: With Applications in R. New York: Springer, 2013. 25
2013
-
[24]
Catboost: gradient boosting with cate- gorical features support,
A. V. Dorogush, V. Ershov, and A. Gulin, “Catboost: gradient boosting with cate- gorical features support, ”arXiv preprint arXiv:1810.11363, 2018
2018 arXiv
-
[25]
Catboost: Unbiased boosting with categorical features,
L. Prokhorenkova, G. Gusev, A. Vorobev, A. V. Dorogush, and A. Gulin, “Catboost: Unbiased boosting with categorical features, ” inAdvances in neural information pro- cessing systems, vol. 31, 2018, pp. 6638–6648
2018
-
[26]
Permutation importance: a cor- rected feature importance measure,
A. Altmann, L. Toloşi, O. Sander, and T. Lengauer, “Permutation importance: a cor- rected feature importance measure, ”Bioinformatics, vol. 26, no. 10, pp. 1340–1347, 2010
2010
-
[27]
A unified approach to interpreting model predictions,
S. M. Lundberg and S.-I. Lee, “A unified approach to interpreting model predictions, ” inAdvances in neural information processing systems, vol. 30, 2017, pp. 4765–4774
2017
-
[28]
Hybrid non-heuristic variable selection models via regularization for black box models applied to the insurance sector,
L. G. . R. Moral, “Hybrid non-heuristic variable selection models via regularization for black box models applied to the insurance sector, ” https://github.com/G4LV40/ regularization-blackbox-pipeline, 2025, all source code, simulation routines, and supporting materials for this study
2025
-
[29]
Multivariate adaptive regression splines,
J. H. Friedman, “Multivariate adaptive regression splines, ”The Annals of Statistics, vol. 19, no. 1, pp. 1–67, 1991. 7 Supplementary Materials Table 6: Variable names and descriptions used in the predictive model Variable name Description Age Customer’s age (years) AnnualInco...
1991
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.