Pith. sign in

REVIEW 4 major objections 4 minor 24 references

Non-null Shrinkage Regression and Subset Selection via the Fractional Ridge Regression

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fractional ridge regression uses a penalty that vanishes once at most m coefficients remain, so extreme regularization yields a best m-variable model rather than the zero model, and moderate regularization lets strong signals escape…

desk verdict New penalty family with a neat recursion and real non-null shrinkage; empirical case needs work, but this deserves a serious referee. read the letter →

arxiv 2505.23925 v2 pith:BPWMGZVB submitted 2025-05-29 stat.ME

Sihyung Park (1) , Leonard A. Stefanski (1) ((1) North Carolina State University) This is my paper
classification stat.ME MSC 62J0762J05
keywords FractionalridgeregressionPenalizedSparsemodelingFeatureselectionForwardrecursivealgorithmIterativelyreweightedestimationCoordinatedescentBestsubset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Fractional ridge (Fridge) regression is a penalized least-squares method whose penalty $P_m(\beta)=\sum_{j_1<\cdots

What carries the argument

The central object is the Fridge penalty $P_m(g)=\sum_{j_1<\cdots<j_{m+1}} g_{j_1}\cdots g_{j_{m+1}}$ for $g(\beta_j)=|\beta_j|$ or $g(\beta_j)=\beta_j^2$: a sum over all subsets of size $m+1$ of products of coefficient magnitudes. Its defining property is that $P_m(g)=0$ iff at least $p-m$ coefficients are zero, so the penalty does not constrain a model of size $\le m$ and, at extreme regularization, the estimator becomes the best $m$-variable least-squares fit. The forward recursive identities $V^k(g)=(P_{k-1}(g)-V^{k-1}(g))\circ g$ and $P_k(g)=\mathbf{1}^\top V^k(g)/(k+1)$ compute the penalty in $O(mp)$ operations, and the coordinate-descent update of Theorem 1, $\hat\beta_j^{\mathrm{new}}\leftarrow S(n^{-1}\sum_i x_{ij}(y_i-\hat y_i^{(j)}),\,\lambda P_{m-1}(\hat g_{-[j]}))/(n^{-1}\sum_i x_{ij}^2)$, carries the optimization.

What would settle it

Take a small problem, say $p=10$ and $n=50$, enumerate all subsets of size $m$ to find the true minimizer of the squared error subject to $P_m(\beta)=0$, and compare with the output of the proposed IRL and coordinate-descent algorithms; any run with a strictly larger objective value or a different support shows the algorithms are not solving the Fridge problem. Separately, check on simulated data whether the $\lambda_{m,\max}$ computed with $\hat\beta=\epsilon\mathbf{1}_p$ satisfies inequality (14) for the converged estimate; a counterexample would break the claimed guarantee of at most $m$ nonzero coefficients.

Watch

Extended reading notes

Core claim

The central discovery is that replacing the Lasso's sum of absolute coefficients with a sum of products of coefficient magnitudes—one product for every subset of size $m+1$—changes the target of shrinkage. Because $P_m(\beta)=0$ if and only if at least $p-m$ of the $\beta_j$ are zero, the solution as $\lambda\to\infty$ minimizes squared error subject to having at most $m$ nonzero coefficients: a best subset model, not the zero model. The authors show this selectively penalized objective can be computed efficiently and solved by weighted-Lasso and weighted-ridge iterations or by coordinate descent with a soft-thresholding update whose shrinkage amount is $\lambda P_{m-1}(g_{-[j]})$. In their simulations Fridge with a target model size near the true size gives smaller mean squared error and smaller selected models than Lasso, adaptive Lasso, and SCAD, and the selected variable sets are not nested as $m$ grows, which they argue helps when predictors are correlated.

Load-bearing premise

The load-bearing assumption is that the coordinate-descent and iteratively reweighted algorithms, started from a ridge estimate, find a good near-global minimizer of the non-convex Fridge objective—and that the crude plug-in $\hat\beta=\epsilon\mathbf{1}_p$ used to set $\lambda_{m,\max}$ really satisfies the sufficient condition in (14).

Editorial extensions

If this is right

  • Under extreme regularization, Fridge computes a best $m$-variable OLS fit without enumerating $\binom{p}{m}$ subsets, so subset selection of fixed size becomes feasible at large $p$.
  • The solution path is non-nested in the target size $m$: increasing $m$ can swap variables in and out, which the paper argues is an advantage when predictors within a correlated group compete.
  • Because the Fridge weight $w_j=P_{m-1}(g)-V^{m-1}(g)[j]$ can be exactly zero, some coefficients are left unpenalized while others are shrunk, which is the mechanism that avoids bias for strong signals.
  • The $\lambda$ threshold in Theorem 1 gives a principled upper bound on cross-validation grids: for $\lambda$ large enough, the Fridge model has at most $m$ nonzero coefficients.
  • In the reported simulations, Fridge with target size at or below the true model size achieves lower test MSE and smaller selected models than Lasso, adaptive Lasso, and SCAD.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to use the extreme Fridge as a deterministic best-subset approximation in high dimensions; one could test whether the $m$-variable support it returns matches exhaustive search on small-$p$ problems before trusting it at large $p$.
  • Because the penalty is a product over coefficients, Fridge might be adapted to sparse interaction or graphical-model problems where the natural sparsity is on pairs or higher-order terms rather than single coefficients.
  • The non-nested path suggests that averaging Fridge models across target sizes could behave like a model average over subset sizes; this is an implication of the paper's geometry, not something it investigates.
  • A testable prediction of the selective-shrinkage mechanism is that Fridge's advantage over Lasso grows as true nonzero coefficients become larger and more correlated; the paper's designs are consistent with that, but a systematic study varying signal strength and correlation would settle it.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces Fractional Ridge (Fridge) regression, a penalized least-squares estimator whose penalty P_m(g) is the sum of products of coefficient functions over all subsets of size m+1. By construction, P_m(g)=0 if and only if at most m coefficients are nonzero, so under extreme regularization the estimator targets the best m-variable OLS model rather than the null model. The authors provide a forward recursive algorithm for computing the penalty in O(mp) operations, an iteratively reweighted Lasso/ridge formulation, and a coordinate descent update rule with a threshold for lambda that forces at least p-m coefficients to zero. The paper reports simulation studies (p=60, 600; two collinear designs) and a prostate cancer data application, claiming improved MSE and sparsity relative to Lasso, adaptive Lasso, and SCAD, plus a non-nested selection path and a best-subset-type extreme Fridge procedure.

Significance. The Fridge penalty is a simple and computationally attractive way to encode a non-null shrinkage target, and the forward recursive algorithm is a genuine algorithmic contribution. Lemma 1 and Theorem 1 are derived cleanly, and the O(mp) operation count makes the penalty feasible for moderately large problems. If the proposed algorithms reliably minimize the non-convex objective, the method could provide a useful complement to Lasso-type procedures, especially for highly correlated predictors. However, the practical value of the central claims currently rests on unproven optimization convergence and on empirical evidence that contains internal inconsistencies; the strengths are therefore conditional on substantial additional verification.

major comments (4)
  1. [Section 3 and Section 4.3] The paper's central practical claim that extreme Fridge performs best subset selection without exhaustive search depends on the coordinate descent and iteratively reweighted algorithms finding a global minimizer of a non-convex objective. Theorem 1 only establishes a necessary condition for a coordinate-wise minimum; no convergence theorem, stationarity result, or local optimality certificate is provided. Because best subset selection is NP-hard, a polynomial-time heuristic cannot be guaranteed to solve it exactly, so the claim in Section 4.3 that the selected predictors are 'best in terms of an error metric' is not justified. The authors should either prove a meaningful convergence result (e.g., convergence to a stationary point with conditions) or provide numerical evidence from multiple random initializations and, for small p, comparisons with exhaustive best-subset enumeration.
  2. [Section 3.2, Equation (14)] The practical sparsity threshold lambda_{m,max} is computed by plugging beta-hat = epsilon 1_p with epsilon = 0.001 into Equation (14), and the paper states only that this 'typically satisfies' the sufficient condition without quantification. The denominator depends on g*, the unobserved probability limit of the estimated coefficient functions, so the plug-in is not verified. The authors should report the empirical distribution of the ratio of the left-hand side of (14) to the plug-in value across their simulations and data application, or otherwise demonstrate that the condition holds, because the guarantee that at most m coefficients are nonzero depends on this step.
  3. [Section 4.2 and Table 1] There is a direct numeric inconsistency in the main empirical claim: the text states that Fridge with TMS=3 achieved the smallest average minimum testing MSE of 7.55 and that Lasso's average MSE was 8.14, while Table 1 reports 0.99 and 1.53 for the same entries. The units or the metric must differ, but this is not explained anywhere. This inconsistency undermines confidence in the empirical record, and it must be resolved and corrected before the simulation results can be assessed.
  4. [Section 4.2, Equation (18)] The 'best potential training error' metric uses the true coefficient vector beta* to choose the tuning parameter, giving the methods oracle information that would not be available in practice. This metric cannot support comparative claims about predictive performance; for example, Figure 2's upper panels and the surrounding discussion present this oracle-based comparison as evidence of Fridge's advantage. The paper should base its headline comparisons on the cross-validated testing error, and if the 'potential' metric is retained, it should be clearly labeled as an oracle diagnostic rather than as a performance comparison with Lasso, adaptive Lasso, or SCAD.
minor comments (4)
  1. [Section 2.3] The operation count is stated as O(mp) and then as (3m+1)p - 1 operations; please make the dependence on the cost of evaluating g explicit and clarify whether O(mp) is in m and p jointly or in either argument.
  2. [Section 4.1] The convergence criterion and truncation threshold (coefficients below 10^-4 set to zero) are reasonable, but the paper does not say whether the reported model sizes are computed before or after the final truncation; please state this explicitly.
  3. [Section 4.3] The statement that 'when TMS values were greater than or equal to the true model size, Fridge successfully selected the true non-zero coefficients' is based on a single illustrative solution path plot and should be qualified as an example rather than a general empirical finding.
  4. [General] No code or data are provided, which makes the simulation studies and the prostate application difficult to verify independently; providing a public repository would materially strengthen the paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the extreme-regularization best-subset property follows transparently from the definition of the Fridge penalty, while the empirical and algorithmic content is independent.

full rationale

The paper's central mathematical observation—that P_m(β)=0 iff at least p−m coefficients are zero, so the λ→∞ limit minimizes squared error subject to at most m non-zero coefficients—is literally built into the definition of the penalty: 'By construction, P_m(β) is zero if and only if at least (p−m) of the β_j are zero.' This is a design property stated openly, not a hidden empirical claim or a fitted quantity later relabeled as a prediction. The subsequent contribution is computational (the O(mp) forward recursion, coordinate-descent update, and iteratively reweighted algorithms) and empirical (simulations and a prostate application comparing Fridge to Lasso, adaptive Lasso, and SCAD). Those comparisons are self-contained and do not rely on the conclusion being tested. There are no self-citations or imported uniqueness theorems carrying the argument. The crude plug-in β̂=ε1_p with ε=0.001 used to set λ_{m,max} is an unverified heuristic ('typically satisfies' the sufficient condition), and the non-convex optimization lacks a global-optimality guarantee; both are correctness/robustness concerns rather than circular reductions. The numerical inconsistency between Section 4.2 (testing MSE 7.55 vs 8.14) and Table 1 (0.99 vs 1.53) also affects the reliability of the empirical record but does not make any derivation equivalent to its inputs by construction. Therefore no circular step is present.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method's behavior depends on user-selected target size m and standard λ tuning; the practical sparsity threshold adds a hand-picked ε; the non-convex optimization relies on an unproven convergence assumption; and the penalty family itself is an ad hoc construction.

free parameters (4)
  • Target model size m = User-specified (0 to p-1)
    Controls the maximum number of non-zero coefficients under extreme regularization; the paper suggests selecting it via extreme Fridge validation but provides no formal criterion.
  • Regularization parameter λ = Tuned by cross-validation
    Standard tuning parameter for the penalty strength; the paper searches a log-linear grid.
  • ε for λ_{m,max} plug-in = 0.001
    Used to compute a crude maximum λ via β̂=ε1_p in equation (14); the paper states it 'typically satisfies' the condition without proof.
  • Coefficient truncation threshold = 1e-4
    Coefficients below this magnitude are set to zero during iteration and in final estimates, affecting the effective model size and sparsity.
assumptions (4)
  • domain assumption Standard i.i.d. linear model with centered predictors and response
    Section 2.1 assumes (X_i, Y_i) are i.i.d. and centered; the method and proofs are developed in this setting.
  • ad hoc to paper The penalty form P_m(g) with g(x)=|x| or g(x)=x^2 is a valid construction
    Section 2.1 introduces the product-of-coefficients penalty specifically to induce the target-size property; it is not derived from first principles.
  • domain assumption The probability limit g* of ĝ exists for the λ threshold formula
    Theorem 1 and the threshold in (14) rely on g*, the assumed probability limit of the estimate, which is not guaranteed in finite samples.
  • domain assumption Coordinate descent or IRL on the non-convex Fridge objective converges to an acceptable solution from a ridge initialization
    Section 3.2 acknowledges non-convexity but provides no convergence guarantees; the reported results depend on this assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Non-null Shrinkage Regression and Subset Selection via the Fractional Ridge Regression." pith.science (2026). https://pith.science/paper/BPWMGZVB

@misc{pith2026250523925,
  author       = {Pith},
  title        = {Pith review of: Non-null Shrinkage Regression and Subset Selection via the Fractional Ridge Regression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BPWMGZVB}},
  note         = {Machine review of arXiv:2505.23925}
}
abstract

$\ell_p$-norm penalization, notably the Lasso, has become a standard technique, extending shrinkage regression to subset selection. Despite aiming for oracle properties and consistent estimation, existing Lasso-derived methods still rely on shrinkage toward a null model, necessitating careful tuning parameter selection and yielding monotone variable selection. This research introduces Fractional Ridge Regression, a novel generalization of the Lasso penalty that penalizes only a fraction of the coefficients. Critically, Fridge shrinks the model toward a non-null model of a prespecified target size, even under extreme regularization. By selectively penalizing coefficients associated with less important variables, Fridge aims to reduce bias, improve performance relative to the Lasso, and offer more intuitive model interpretation while retaining certain advantages of best subset selection.

Figures

Figures reproduced from arXiv: 2505.23925 by the authors.

Figure 1
Figure 1. Geometry of the Fridge penalty. The left two plots illustrate the Lasso penalty [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. Best potential training MSE and testing MSE when [PITH_FULL_IMAGE:figures/full_fig_p019_2.png] view at source ↗
Figure 3
Figure 3. Best potential training MSE and testing MSE, when [PITH_FULL_IMAGE:figures/full_fig_p020_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Select solution paths from D1 with p = 60. Paths of true non-zero coefficients are indicated in thick lines. great extent by increases in specificities. Furthermore, in [PITH_FULL_IMAGE:figures/full_fig_p021_4.png]
Figure 5
Figure 5. Figure 5: Best testing MSE of the extreme Fridge. (Top) [PITH_FULL_IMAGE:figures/full_fig_p025_5.png]
Figure 6
Figure 6. Figure 6: Validation MSE in the prostate cancer data, computed by extreme Fridge with [PITH_FULL_IMAGE:figures/full_fig_p028_6.png]
Figure 7
Figure 7. Figure 7: Coefficient estimates for predictors in the prostate cancer data. Box plots represent [PITH_FULL_IMAGE:figures/full_fig_p028_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

24 extracted references · 22 canonical work pages

  1. [1]

    & Huang, J

    Breheny, P. & Huang, J. (2011), ‘Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection’, The Annals of Applied 32 Statistics 5(1), 232–253

    work page 2011

  2. [2]

    (1995), ‘Better subset regression using the nonnegative garrote’,Technometrics 37(4), 373–384

    Breiman, L. (1995), ‘Better subset regression using the nonnegative garrote’,Technometrics 37(4), 373–384

    work page 1995

  3. [3]

    & Tibshirani, R

    Efron, B., Hastie, T., Johnstone, I. & Tibshirani, R. (2004), ‘Least angle regression’,The Annals of Statistics32(2), 407–499

    work page 2004

  4. [4]

    Fan, J. & Li, R. (2001), ‘Variable selection via nonconcave penalized likelihood and its oracle properties’,Journal of the American Statistical Association96(456), 1348–1360

    work page 2001

  5. [5]

    H., Hastie, T

    Friedman, J. H., Hastie, T. & Tibshirani, R. (2010), ‘Regularization paths for generalized linear models via coordinate descent’,Journal of Statistical Software33, 1–22

    work page 2010

  6. [6]

    E., Viallon, V

    Ghaoui, L. E., Viallon, V. & Rabbani, T. (2011), ‘Safe feature elimination for the lasso and sparse supervised learning problems’. URL: https://arxiv.org/abs/1009.4219

    arXiv 2011

  7. [7]

    Hastie, T., Tibshirani, R., Friedman, J. H. & Friedman, J. H. (2009),The elements of statistical learning: data mining, inference, and prediction, Vol. 2, Springer

    work page 2009

  8. [8]

    D., Joulin, A., Bach, F

    Hocking, T. D., Joulin, A., Bach, F. & Vert, J.-P. (2011), Clusterpath an algorithm for clustering using convex fusion penalties,in ‘Proceedings of the 28th International Conference on Machine Learning’, pp. 745–752

    work page 2011

Show all 24 references
  1. [9]

    Hoerl, A. E. & Kennard, R. W. (1970), ‘Ridge regression: Biased estimation for nonorthog- onal problems’,Technometrics12(1), 55–67

    1970

  2. [10]

    & Yin, Y

    Li, Y., Lu, F. & Yin, Y. (2022), ‘Applying logistic lasso regression for the diagnosis of atypical crohn’s disease’,Scientific Reports12(1), 11340

    2022

  3. [11]

    & Zhang, C

    Lu, Y., Zhou, Y., Qu, W., Deng, M. & Zhang, C. (2011), ‘A lasso regression model for the construction of microrna-target regulatory networks’,Bioinformatics 27(17), 2406–2413. 33

    2011

  4. [12]

    O., Schulz-Streeck, T

    Ogutu, J. O., Schulz-Streeck, T. & Piepho, H.-P. (2012), Genomic selection using regularized linear regression models: ridge regression, lasso, elastic net and their extensions,in ‘BMC Proceedings’, Vol. 6, Springer, pp. 1–6

    2012

  5. [13]

    & Vovk, V

    Saunders, C., Gammerman, A. & Vovk, V. (1998), Ridge regression learning algorithm in dual variables,in ‘Proceedings of the 15th International Conference on Machine Learning’, p. 515–521

    1998

  6. [14]

    A., Kabalin, J

    Stamey, T. A., Kabalin, J. N., McNeal, J. E., Johnstone, I. M., Freiha, F., Redwine, E. A. & Yang, N. (1989), ‘Prostate specific antigen in the diagnosis and treatment of adenocarcinoma of the prostate. ii. radical prostatectomy treated patients’,The Journal of Urology141(5), ...

    1989

  7. [15]

    K., Narasimhan, B

    Tay, J. K., Narasimhan, B. & Hastie, T. (2023), ‘Elastic net regularization paths for all generalized linear models’,Journal of Statistical Software106, 1–31

    2023

  8. [16]

    (1996), ‘Regression shrinkage and selection via the lasso’,Journal of the Royal Statistical Society Series B: Statistical Methodology58(1), 267–288

    Tibshirani, R. (1996), ‘Regression shrinkage and selection via the lasso’,Journal of the Royal Statistical Society Series B: Statistical Methodology58(1), 267–288

    1996

  9. [17]

    & Tibshirani, R

    Tibshirani, R., Bien, J., Friedman, J., Hastie, T., Simon, N., Taylor, J. & Tibshirani, R. J. (2012), ‘Strong rules for discarding predictors in lasso-type problems’,Journal of the Royal Statistical Society Series B: Statistical Methodology74(2), 245–266. van de Geer, S., Bühl...

    2012

  10. [18]

    T., Chen, Y

    Wu, T. T., Chen, Y. F., Hastie, T., Sobel, E. & Lange, K. (2009), ‘Genome-wide association analysis by lasso penalized logistic regression’,Bioinformatics 25(6), 714–721

    2009

  11. [19]

    & Lin, Y

    Yuan, M. & Lin, Y. (2006), ‘Model selection and estimation in regression with grouped 34 variables’, Journal of the Royal Statistical Society Series B: Statistical Methodology 68(1), 49–67

    2006

  12. [20]

    (2010), ‘Nearly unbiased variable selection under minimax concave penalty’, The Annals of Statistics38(2), 894–942

    Zhang, C.-H. (2010), ‘Nearly unbiased variable selection under minimax concave penalty’, The Annals of Statistics38(2), 894–942

    2010

  13. [21]

    (2009), ‘Some sharp performance bounds for least squares regression withL1 regularization’, The Annals of Statistics37(5A), 2109–2144

    Zhang, T. (2009), ‘Some sharp performance bounds for least squares regression withL1 regularization’, The Annals of Statistics37(5A), 2109–2144

    2009

  14. [22]

    Zhao, P. & Yu, B. (2006), ‘On model selection consistency of lasso’,Journal of Machine learning research7(Nov), 2541–2563

    2006

  15. [23]

    (2006), ‘The adaptive lasso and its oracle properties’,Journal of the American Statistical Association101(476), 1418–1429

    Zou, H. (2006), ‘The adaptive lasso and its oracle properties’,Journal of the American Statistical Association101(476), 1418–1429

    2006

  16. [24]

    & Hastie, T

    Zou, H. & Hastie, T. (2005), ‘Regularization and variable selection via the elastic net’, Journal of the Royal Statistical Society Series B: Statistical Methodology67(2), 301–320. 35

    2005

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.