REVIEW 3 major objections 8 minor 34 references
AKSSAM claims that automatic knot selection can give generalized additive models sparse B-spline bases whose predictive accuracy matches P-splines.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 08:29 UTC pith:QRHM7JWH
load-bearing objection AKSSAM is a genuine extension of A-splines to GAMs with a clean Fellner-Schall scheme and solid theory, but the central simulation evidence for 'comparable predictive performance' is weak because it is training-set only; the real-data CV is supportive and the paper deserves a serious referee. the 3 major comments →
Automatic knot selection in smooth additive models
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
AKSSAM establishes that explicit knot selection can be automated for GAMs without sacrificing predictive accuracy. The method builds on the A-splines result that an interior knot is removable exactly when the (q+1)-th order difference of the B-spline coefficients at that knot is zero, and approximates the L0 penalty on those differences by an adaptive ridge surrogate. Extending this to multiple additive components requires solving an identifiability problem: because each B-spline basis sums to one, the design matrix is rank deficient, so the paper groups an identifiability penalty with the fitting criterion and proves the remaining matrix is positive definite, making the Fellner-Schall updat
What carries the argument
The central mechanism is the adaptive ridge quadratic surrogate of the L0 pseudonorm: a knot is retained when the surrogate term omega_l (Delta^{q+1} alpha_l)^2 is approximately 1 and dropped when it is near 0, where omega_l = ((Delta^{q+1} alpha_l)^2 + epsilon^2)^{-1}. This converts the intractable L0 problem into a sequence of quadratic problems. The second piece is a customized Fellner-Schall algorithm operating on the block-diagonal penalties, one per covariate, with the identifiability penalty folded into the fit; a positive-definiteness lemma justifies the updates. Together they allow simultaneous coefficient estimation, weight updating, and automatic penalty-parameter tuning.
Load-bearing premise
The load-bearing premise is that the adaptive ridge weights correctly identify removable knots — keeping a knot when the surrogate term is near 1 and dropping it when near 0 — because this rule is imported from univariate A-splines and is not re-proved for simultaneous multi-component estimation or the final identifiability-penalized refit.
What would settle it
Run AKSSAM on a simulated additive model with a known true function that has a deliberately sparse knot representation. After the selected knot set is obtained, refit on the trimmed basis and compare the resulting curve with the true function over the whole covariate domain. If dropping a knot that the surrogate classified as removable changes the fitted curve by more than a small numerical tolerance, the L0-surrogate identification has failed. A stronger stress test uses two highly correlated covariates: if concurvity distorts the coefficient differences, the method may retain or drop the wro
If this is right
- GAMs can be fitted with a much smaller B-spline basis: in the simulations, effective degrees of freedom drop from 259 basis functions for P-splines to roughly 40 for AKSSAM with comparable MSE.
- Penalty-parameter tuning for knot selection no longer requires grid search, since the Fellner-Schall scheme estimates each component's penalty parameter automatically.
- The method applies to non-Gaussian responses through P-IRLS, so sparse knot selection is available for Poisson and binomial GAMs, not just Gaussian additive models.
- If the data are highly noisy or the sample is small, P-splines remain the better choice, so the sparsity benefit of AKSSAM is clearest in informative settings.
- Selected knot locations concentrate in well-defined regions of the covariate domain and are stable across cross-validation folds, giving a more interpretable picture of where changes in the response occur.
Where Pith is reading between the lines
- If the adaptive ridge surrogate remains reliable under simultaneous estimation of several components, AKSSAM's sparse fits could be directly plugged into global optimization as cheap surrogate objectives or constraints, where the dimension of the basis controls the cost of solving the surrogate problem.
- A natural stress test would be to feed the selected knot sets back into an unpenalized refit and check whether the final curve is within a small tolerance of the full-basis fit; failures would localize exactly where the L0-surrogate identification breaks down.
- The identifiability-penalty-as-pseudo-observations view suggests a mixed-model interpretation that might extend to shape-constrained or variable-selection variants, where the same Fellner-Schall logic could tune additional penalties.
- The knot locations selected by AKSSAM could themselves be used as changepoint estimates or as discretization points for subsequent models, but the paper does not test this directly.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes AKSSAM, an automatic knot-selection method for smooth additive models and generalized additive models. It extends the univariate A-splines framework of Goepp et al. (2025) by using an L0-pseudonorm quadratic surrogate penalty for each additive component and a customized Fellner–Schall update to estimate the per-component penalty parameters, avoiding grid search. The authors prove that the Fellner–Schall conditions hold when the identifiability penalty is grouped with the fitting criterion (Lemma 3.1 and Theorem 3.3), and they provide an alternating optimization algorithm with a final unpenalized refit on the selected knots. The method is evaluated through two simulation studies (Gaussian and Poisson) and two real-data applications (electric_load and PimaIndiansDiabetes), comparing with P-splines, GeDS, and a PSO-based approach. The central claim is that AKSSAM achieves comparable predictive performance while producing substantially sparser models in terms of B-spline basis functions.
Significance. If the empirical claims are supported, AKSSAM would be a useful alternative to regularization-based GAM fitting in applications where model sparsity and interpretability are important, such as surrogate modeling. The paper's theoretical treatment is a real strength: the handling of the identifiability penalty is careful, and the verification of the Fellner–Schall conditions in Section 3.3 is explicit and well structured. The authors also provide public code and reproducible experiments, and the comparisons include state-of-the-art knot-selection methods. The main weakness is empirical: the simulation studies, which are the primary controlled evidence for the predictive-performance claim, report only training-data metrics. The real-data cross-validations are supportive but limited to two datasets. The hyperparameter protocol also deserves scrutiny.
major comments (3)
- [Section 4.1, Tables 3 and 5] The simulations are the main controlled evidence for the abstract's claim of 'comparable performance', but Section 4.1 explicitly states that 'all performance metrics are computed on the training data'. Training MSE and training-based BIC do not measure predictive performance. In fact, Table 3 shows AKSSAM's training MSE is higher than P-splines in every setting, so the claimed advantage rests on BIC, which rewards the sparsity that AKSSAM is designed to produce. In addition, the final refit in Algorithm 1 (line 15) is an unpenalized least-squares fit, so no roughness penalty remains to control variance on unseen data. The real-data 5-fold cross-validation is supportive, but the simulations are the natural place to report test error. I ask the authors to add out-of-sample evaluation (e.g., repeated holdout or cross-validation) in the simulation studies, or to weaken the paper's predictiv
- [Section 3.1, Eq. (13); Section 3.2.2] The knot-selection rule is imported from univariate A-splines: a knot is retained when the surrogate term ω_l(Δ^{q+1}α_l)^2 is near 1 and dropped when near 0. The theoretical justification in Goepp et al. (2025) is for univariate B-splines without the identifiability penalty P_I. In the additive/GAM setting, the coefficients are estimated simultaneously with per-component penalties and with P_I included in the objective (Section 3.2.2). The paper does not prove that the surrogate-to-L0 equivalence still identifies removable knots in this setting, nor that the final unpenalized refit on the selected knots reproduces the penalized solution. This is a correctness-risk concern for the core mechanism, not a presentation issue. I recommend adding a numerical or theoretical verification—for example, comparing the fitted curves before trimming with the final refit and reporting the discrepancy,
- [Section 4.1, 'Regarding tolerances...'] The tolerance values Tol1, Tol2, ItTol1, ItTol2, and related parameters are stated to have been 'selected after extensive preliminary experimentation'. The same applies to the initial penalty parameter λ0=10, ε=10^{-5}, and the upper bound in Remark 4.1. If these choices are tuned on the same data used to report results, the comparisons can be optimistic, especially because competitor methods are run with default settings. The paper does not report a sensitivity analysis to these hyperparameters, except for a brief remark about λ0. Since one of the paper's selling points is that AKSSAM is 'fully automatic', the tuning dependence of the results should be quantified or a principled default protocol should be established.
minor comments (8)
- [Section 1] Typo: 'The remainder of his paper is organized as follows' should be 'this paper'.
- [Section 2.2] Typo: 'the estimated cuve' should be 'curve'.
- [Section 4.1 and Appendix A.1.3] The simulation description lists SNR values {1,2,4}, but Appendix A.1.3 is headed 'n=300, SNR=3'; the figure content appears to be SNR=4. Please correct the heading.
- [Section 4.1 and Algorithm 1] The text mentions Tol3 and ItTol3, but Algorithm 1 defines only Tol1, ItTol1, Tol2, ItTol2. If Tol3/ItTol3 are used for the P-IRLS inner loop in the GAM version, they should be defined and included in the pseudocode or removed.
- [Section 3.3, Remark 3.4] The index range 'l=1,...,1,...,k_r-2q_r-2' contains an apparent typo; should be 'l=1,...,k_r-2q_r-2'.
- [Section 4.2] For the real-data tables, please clarify in the text or table captions which metrics are computed on held-out folds and which, if any, are computed on training folds. BIC is listed alongside cross-validated MSE/AUC, and its role in a cross-validation comparison should be explicit.
- [Appendix B.2] Typo: 'electric_road' should be 'electric_load' in the appendix overview.
- [Appendix A.2] Typos: 'togehter' and 'fucntions' should be 'together' and 'functions'.
Circularity Check
No significant circularity: AKSSAM is an extension of external A-spline and Fellner-Schall machinery with local proofs; self-citations are non-load-bearing.
full rationale
The derivation chain is not self-referential. The L0-surrogate knot-selection objective (10)-(12) and the knot-removal justification are taken from Goepp et al. (2025), an external published source, together with the adaptive ridge surrogate of Frommlet and Nuel (2016) and Rippe et al. (2012). The custom Fellner-Schall update is justified by a local verification (Lemma 3.1 and Theorem 3.3) of the external Wood and Fasiolo (2017) conditions, not by invoking the present authors' prior work. Algorithm 1 directly optimizes the defined objective (20)-(23)/(28), so the selected-knot count and EDF are computational outputs, not fitted constants renamed as predictions. The self-citations that appear (Cuesta et al. 2025; Navarro-García et al. 2023, 2024; Rodríguez-Álvarez et al. 2019) are motivational context or future-work directions and carry no weight in the derivation. Non-circular limitations are acknowledged in the text: Section 4.1 explicitly says "all performance metrics are computed on the training data," which weakens the predictive claim but is a validation concern, not a circular reduction; Remark 4.1 similarly admits that neither the adaptive ridge loop nor its combination with Fellner-Schall enjoys global convergence guarantees. I therefore find no circular step; the central claim rests on an externally sourced method plus locally proved conditions, and no equation reduces to its own input by construction.
Axiom & Free-Parameter Ledger
free parameters (5)
- epsilon =
1e-5
- initial penalty parameters lambda0 =
10 * 1_p
- convergence tolerances =
Tol1=Tol2=Tol3=1e-5, ItTol1=ItTol2=ItTol3=100
- lambda upper bound and safeguard =
10^4 upper bound; 10^-1 safeguard
- initial knot count per covariate =
k=47 (40 internal intervals)
axioms (5)
- domain assumption Assumption 1: each design matrix B_r has full column rank, and the only linear dependencies in B are those induced by the partition-of-unity property.
- domain assumption Knot-removal theorem of Goepp et al. (2025): if a (q+1)-th order coefficient difference vanishes, removing the corresponding internal knot yields exactly the same fitted function.
- standard math Fellner-Schall Theorem 1 of Wood and Fasiolo (2017) applies to the augmented model with fixed adaptive weights.
- standard math B-spline bases form a partition of unity: B_r 1_mr = 1_n.
- domain assumption In GAMs, P-IRLS converges and the weight matrix Ω has positive diagonal elements.
read the original abstract
B-spline regression constitutes a widely used framework for nonparametric modeling. The performance of this methodology depends on specifying the number and placement of changepoints, known as knots, prior to the estimation process. Such knot sequence determines the dimension of the B-spline basis used to represent the regression function and the number of coefficients to be estimated. Therefore, the knots' choice affects the model's flexibility, influencing its smoothness and goodness-of-fit. Traditionally, this problem has been addressed either by explicitly selecting knots, via knot-selection algorithms, or by regularization methods, such as P-splines, which automatically tune the regressor's smoothness. The latter have become the standard in generalized additive models (GAMs). In contrast, knot-selection techniques, frequently neglected because of computational or modeling limitations, provide certain advantages which can be valuable in some contexts. In this work, we introduce a novel explicit knot-selection technique for GAMs based on an extension of the adaptive splines (A-splines) knot selection methodology, combined with a customized Fellner-Schall scheme for tuning the associated parameters. Our approach is evaluated on various synthetic and real datasets and compared with P-splines and state-of-the-art knot-selection techniques. The results indicate comparable performance, while producing models built on a substantially smaller number of basis elements.
Figures
Reference graph
Works this paper leans on
-
[1]
Cuesta, M., D'Ambrosio, C., Durban, M., Guerrero, V., and Trindade, R. S. (2025). On leveraging constrained smooth additive regression models for global optimization. arXiv:2510.14122v2
Pith/arXiv arXiv 2025
-
[2]
De Boor, C. (1978). A Practical Guide to Splines , volume 27 of Applied Mathematical Sciences . Springer New York
1978
-
[3]
Denison, D., Mallick, B., and Smith, A. (1998). Automatic B ayesian curve fitting. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 60(2):333--350
1998
-
[4]
S., Kaishev, V
Dimitrova, D. S., Kaishev, V. K., Lattuada, A., Guillén, E. L. S., and Verrall, R. J. (2025). GeDS: Geometrically Designed Spline Regression . R Core Team. R package version 0.3.3
2025
-
[5]
S., Kaishev, V
Dimitrova, D. S., Kaishev, V. K., Lattuada, A., and Verrall, R. J. (2023). Geometrically designed variable knot splines in generalized (non-)linear models. Applied Mathematics and Computation , 436:127493
2023
-
[6]
Eilers, P. H. C. and Marx, B. D. (1996). Flexible smoothing with B -splines and penalties. Statistical Science , 11(2):89--121
1996
-
[7]
Friedman, J. H. (1991). Multivariate adaptive regression splines. The Annals of Statistics , 19(1):1--67
1991
-
[8]
and Nuel, G
Frommlet, F. and Nuel, G. (2016). An adaptive ridge procedure for L_0 regularization. PloS ONE , 11(2):e0148620
2016
-
[9]
Gaillard, P., Goude, Y., Plagne, L., Dubois, T., and Thieurmel, B. (2020). opera: Online Prediction by Expert Aggregation . R package version 1.2.0
2020
-
[10]
and Jüttler, B
Giust, A. and Jüttler, B. (2022). Weighted isogeometric collocation based on spline projectors. Computer Methods in Applied Mechanics and Engineering , 391:114554
2022
-
[11]
Goepp, V., Bouaziz, O., and Nuel, G. (2025). Spline regression with automatic knot selection. Computational Statistics & Data Analysis , 202:108043
2025
-
[12]
Green, P. J. (1995). Reversible jump M arkov C hain M onte C arlo computation and B ayesian model determination. Biometrika , 82(4):711--732
1995
-
[13]
and Tibshirani, R
Hastie, T. and Tibshirani, R. (1986). Generalized additive models. Statistical Science , 1(3):297--310
1986
-
[14]
Hollaway, M. J. and Killick, R. (2024). Detection of spatiotemporal changepoints: a generalised additive model approach. Statistics and Computing , 34(5):162
2024
-
[15]
Jupp, D. L. (1978). Approximation to data by splines with free knots. SIAM Journal on Numerical Analysis , 15(2):328--343
1978
-
[16]
Kim, H., Bang, S., and Kang, J. (2026). Simultaneous selection of knots and variables in additive models. Computational Statistics , 41:10
2026
-
[17]
Leisch, F., Dimitriadou, E., and Hornik, K. (2026). mlbench: Machine Learning Benchmark Problems . R Core Team. R package version 2.1-8
2026
-
[18]
D., Romano, E., and Campagna, R
Magistris, A. D., Romano, E., and Campagna, R. (2025). Adaptive G eneralized P - S plines for F unctional D ata: A S tatistical F ramework via B lockwise GSVD . Statistics and Computing , 35(6):203
2025
-
[19]
Navarro-Garc \' a, M., Guerrero, V., and Durban, M. (2024). A mathematical optimization approach to shape-constrained generalized additive models. Expert Systems with Applications , 255:124654
2024
-
[20]
Navarro-García, M., Guerrero, V., and Durban, M. (2023). On constrained smoothing and out-of-range prediction using P -splines: A conic optimization approach. Applied Mathematics and Computation , 441:127679
2023
-
[21]
Nelder, J. A. and Wedderburn, R. W. (1972). Generalized linear models. Journal of the Royal Statistical Society Series A: Statistics in Society , 135(3):370--384
1972
-
[22]
R., Presnell, B., and Turlach, B
Osborne, M. R., Presnell, B., and Turlach, B. A. (1998). Knot selection for regression splines via the lasso. Computing Science and Statistics , pages 44--49
1998
-
[23]
C., Meulman, J
Rippe, R. C., Meulman, J. J., and Eilers, P. H. C. (2012). Visualization of genomic changes by segmented smoothing using an L_0 penalty. PloS ONE , 7(6):e38230
2012
-
[24]
X., Durban, M., Lee, D.-J., and Eilers, P
Rodr \' guez- \'A lvarez, M. X., Durban, M., Lee, D.-J., and Eilers, P. H. (2019). On the estimation of variance parameters in non-standard generalised linear mixed models: Application to penalised smoothing. Statistics and Computing , 29:483--500
2019
-
[25]
Seck, N. A. and Denuit, M. (2022). Adaptive S plines for C ontinuous F eatures in R isk A ssessment. CAS E - F orum , Summer. Available at: https://forum.casact.org/article/38389-adaptive-splines-for-continuous-features-in-risk-assessment
2022
-
[26]
W., and Young, D
Spiriti, S., Eubank, R., Smith, P. W., and Young, D. (2013). Knot selection for least-squares and penalized splines. Journal of Statistical Computation and Simulation , 83(6):1020--1036
2013
-
[27]
Thielmann, A., Kneib, T., and S \"a fken, B. (2025). Enhancing adaptive spline regression: an evolutionary approach to optimal knot placement and smoothing parameter selection. Journal of Computational and Graphical Statistics , pages 1--16
2025
-
[28]
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology , 58(1):267--288
1996
-
[29]
Wang, X., Zhao, Y., Ni, Q., and Tang, S. (2024). Nonparametric density estimation with nonuniform b-spline bases. Journal of Computational and Applied Mathematics , 440:115648
2024
-
[30]
Wood, S. N. (2017). Generalized Additive Models: An Introduction with R . Chapman and Hall/CRC, Boca Raton, FL, 2 edition
2017
-
[31]
Wood, S. N. (2020). Inference and computation with generalized additive models and their extensions. Test , 29(2):307--339
2020
-
[32]
Wood, S. N. (2025). mgcv: Mixed GAM Computation Vehicle with Automatic Smoothness Estimation . R Core Team. R package version 1.9-3
2025
-
[33]
Wood, S. N. and Fasiolo, M. (2017). A generalized F ellner- S chall method for smoothing parameter optimization with application to tweedie location, scale and shape models. Biometrics , 73(4):1071--1081
2017
-
[34]
Yuan, Y., Chen, N., and Zhou, S. (2013). Adaptive B -spline knot selection using multi-resolution basis set. IIE Transactions , 45(12):1263--1277
2013
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.