REVIEW 3 major objections 5 minor 21 references
pared: Model selection using multi-objective optimization
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Multi-objective tuning identifies sparser networks than AIC alone.
desk verdict Modest but useful application note for an R package; the Pareto-front claim is under-validated and there is a clear alpha parametrization error in the elastic net description. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a Gaussian-process surrogate model for each objective (deviance, coefficient count, roughness, edge count, or shared-edge count) as a function of the tuning parameters. A sequential optimization strategy iteratively proposes new hyperparameter settings that improve the estimated Pareto front, avoiding the need to refit the underlying penalized model at every candidate point. The Pareto front is the set of non-dominated solutions, where improving any objective would worsen at least one other.
What would settle it
Re-fit the underlying penalized models at every hyperparameter setting on the reported Pareto front and recompute the exact objectives; if any reported Pareto-optimal point is strictly dominated by another feasible point in the exact objectives, the surrogate-based Pareto set is not faithful.
Extended reading notes
Core claim
The central claim is that a Gaussian-process-based multi-objective optimizer can efficiently trace the Pareto-optimal set of hyperparameter values for four widely used penalized models, and that for the group graphical lasso this set contains models with fewer selected edges than the model chosen by AIC alone. The paper demonstrates this on protein-signaling network data for ovarian, uterine corpus endometrial, and uterine carcinosarcoma cancer types, producing a Pareto front over AIC, total edge count, and number of shared edges across the three networks in about 2.4 minutes. The authors treat this as evidence that the package offers a principled way to balance fit, sparsity, and structural coherence.
Load-bearing premise
The Gaussian process surrogate models faithfully approximate each objective over the tuning parameter domain, so that the reported Pareto set is close to the true Pareto set.
Editorial extensions
If this is right
- Users of the elastic net and fused lasso can view the complete trade-off between sparsity and fit instead of selecting one penalty value from a grid search.
- For joint graphical lasso models, the method surfaces solutions with more shared edges across groups, making cross-group network comparisons more direct.
- The reported runtime of 2.4 minutes for a 20-protein, three-network analysis indicates the approach is practical on moderate-sized biological problems.
- The same workflow extends naturally to any penalized model with several conflicting objectives, because the optimizer is separated from the model-fitting engine.
Reading between the lines
- A direct check of surrogate accuracy would compare the Gaussian-process-predicted objectives with the exact values at each reported Pareto point; the paper does not include such validation.
- The sparser-than-AIC result hints that single-criterion selection may systematically overfit in edge density; a benchmark across several datasets, not just the gynecological cancer example, could test this.
- The same surrogate-based multi-objective machinery could be applied to other settings where fit and interpretability conflict, such as clustering with stability objectives or survival models with sparsity.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces pared, an R package that applies Gaussian-process-based multi-objective optimization (via the GPareto package) to hyperparameter selection for four penalized statistical models: elastic net, fused lasso, fused graphical lasso, and group graphical lasso. The authors frame model selection as a trade-off among criteria such as fit (deviance, AIC, or residual sum of squares), sparsity, shrinkage, smoothness, or cross-group similarity, and the package returns a set of points on the Pareto front that the user can explore interactively. The paper's central demonstration is an application of the group graphical lasso to proteomic networks from three gynecological cancer types, where pared reportedly finds a Pareto-optimal model that is sparser than the model selected by AIC alone.
Significance. If the Pareto-optimality claims are validated, pared addresses a real need: a user-friendly R tool for multi-objective model selection in penalized regression and graphical models, with readily interpretable interactive graphics. The package is reproducible in principle, with code and vignettes on GitHub, and it extends multi-objective optimization to models of substantive interest in bioinformatics. However, the current manuscript provides no simulation-based validation, no comparison with grid search or other multi-objective optimizers, and no verification that the returned solutions are non-dominated under the true objectives; these omissions currently limit the paper's contribution to a software description rather than a demonstrated method.
major comments (3)
- [Section 2.2 and Section 3] The central claim that pared returns models that lie on the Pareto frontier rests on the accuracy of the Gaussian process surrogate models used by GPareto, yet the manuscript provides no validation of these surrogates and no details of the optimization settings. Specifically, the paper does not report the number of initial evaluations, the acquisition function, or the stopping criterion, nor does it re-fit the returned candidate hyperparameters to confirm that the solutions are non-dominated in the true objectives. Without such confirmation, the headline demonstration in Section 3 is vacuous: the displayed 'Pareto-optimal' model could be dominated by another point in the true objective space. I recommend adding emulator diagnostics (e.g., cross-validated prediction error for each objective) and/or a verification step in which the returned Pareto set is re-evaluated with the exact model fit, with the true objective values reported.
- [Section 3] The illustrative analysis does not provide quantitative evidence for the claim that pared identified a 'sparser' model than the AIC-selected model. The text states that the Pareto-optimal model in Figure 1(b) is sparser, but it does not report the number of edges in either network, the objective values at the selected point, or how the Pareto set was generated (e.g., the range of lambda_1 and lambda_2 searched). Without these numbers, the reader cannot assess whether the claimed improvement is real or whether the displayed solution is a reasonable trade-off. I recommend reporting the edge counts, the AIC values, and the set of Pareto-optimal tuning parameters used.
- [Section 2.2 and Section 3] There are no simulation studies or comparisons to alternative approaches, such as grid search over the hyperparameter grid or other multi-objective optimizers. This is a notable gap for a software paper that claims to 'efficiently explore trade-offs' and 'identify hyperparameter values that result in fitted models which lie on the Pareto frontier.' Since GPareto is used as a black-box optimizer, the specific contribution of pared is the integration with these statistical models; to support this contribution, the paper should at least demonstrate on a small simulated example that the Pareto front recovered by pared is close to the true Pareto front (e.g., obtained by exhaustive grid search). The current manuscript leaves the efficiency and correctness of the optimization entirely unexamined.
minor comments (5)
- [Section 2.1] The elastic net equation and the accompanying text disagree on the meaning of alpha: the displayed objective is lambda [ alpha * ||beta||_1 + (1-alpha) * ||beta||_2 ], which gives ridge at alpha=0 and lasso at alpha=1, whereas the text states the reverse ('lasso (alpha = 0) to ridge (alpha = 1)'). Please correct either the equation or the text.
- [Section 2.1] The heading 'F used lasso' contains a typo and should read 'Fused lasso'. Affiliations also contain typos ('Virgina', 'T exas'), which should be fixed.
- [Section 3] The phrase 'derived the Pareto-optimal set' overstates what the surrogate-based optimization can guarantee; since the Pareto front is approximated by a GP emulator, the set should be described as an 'approximation' or 'estimation' of the Pareto-optimal set.
- [Abstract and Section 4] The paper mentions that the package provides vignettes illustrating application to simulated data, but no such simulation example is described in the manuscript; a brief summary of the simulated vignette would help the reader understand the package's behavior in a controlled setting.
- [References] The paper cites Das et al. (2020) for the pathway analysis in the illustration, but does not provide the full pathway names or a rationale for selecting the set of 20 proteins; adding this context would improve reproducibility.
Circularity Check
No significant circularity: the Pareto-front construction is defined by user-selected objectives and an external Gaussian-process optimizer, and the only self-citation is illustrative.
full rationale
pared's central claim—that it identifies hyperparameter values on a Pareto front balancing user-specified objectives (fit, sparsity, shrinkage/roughness/cross-group similarity)—is an application of multi-objective optimization implemented in the external GPareto package (Section 2.2). Objectives are stated as user choices, not derived from the method; no parameter is fit to a subset and then renamed as a prediction; and no equation in the paper reduces to a prior result by construction. The sole self-citation (Das et al., 2020) is used only to select pathways for the illustrative gynecological-cancer data, not as evidence for the optimization method, and the comparator is AIC as recommended by Danaher et al. (2014). Whether the GP surrogate is accurate enough for the reported points to be truly non-dominated is a separate correctness/validation question, not circularity. Thus the derivation chain is self-contained.
Assumptions & free parameters
assumptions (3)
- domain assumption The Gaussian process surrogate models in GPareto accurately approximate the objective functions over the hyperparameter domain.
- domain assumption The chosen objective functions (deviance, sparsity, shrinkage, roughness, AIC, shared edges) are appropriate and sufficient for model selection.
- standard math Standard formulations of elastic net, fused lasso, and joint graphical lasso as given in the cited literature.
Cite this review
Pith. "Pith review of pared: Model selection using multi-objective optimization." pith.science (2026). https://pith.science/paper/W3FMEFMX
@misc{pith2026250521730,
author = {Pith},
title = {Pith review of: pared: Model selection using multi-objective optimization},
year = {2026},
howpublished = {\url{https://pith.science/paper/W3FMEFMX}},
note = {Machine review of arXiv:2505.21730}
}
read the original abstract
Motivation: Model selection is a ubiquitous challenge in statistics. For penalized models, model selection typically entails tuning hyperparameters to maximize a measure of fit or minimize out-of-sample prediction error. However, these criteria fail to reflect other desirable characteristics, such as model sparsity, interpretability, or smoothness. Results: We present the R package pared to enable the use of multi-objective optimization for model selection. Our approach entails the use of Gaussian process-based optimization to efficiently identify solutions that represent desirable trade-offs. Our implementation includes popular models with multiple objectives including the elastic net, fused lasso, fused graphical lasso, and group graphical lasso. Our R package generates interactive graphics that allow the user to identify hyperparameter values that result in fitted models which lie on the Pareto frontier. Availability: We provide the R package pared and vignettes illustrating its application to both simulated and real data at https://github.com/priyamdas2/pared.
Figures
Reference graph
Works this paper leans on
-
[1]
Akaike, H. (1974). A new look at the statistical model identification. IEEE T Automat Contr , 19(6):716--723
work page 1974
-
[2]
Berger, A., A., K., et al. (2018). A comprehensive pan-cancer molecular study of gynecologic and breast cancers. Cancer Cell , 33(4):690--705
work page 2018
-
[3]
Collette, Y. and Siarry, P. (2003). Multiobjective Optimization: Principles and Case Studies . Springer-Verlag, Berlin, Heidelberg
work page 2003
-
[4]
Danaher, P., Wang, P., and Witten, D. M. (2014). The joint graphical lasso for inverse covariance estimation across multiple classes. J Roy Stat Soc B , 76(2):373--397
work page 2014
-
[5]
Das, P. et al. (2020). NExUS : Bayesian simultaneous network estimation across unequal sample sizes. Bioinformatics , 36(3):798--804
work page 2020
-
[6]
Friedman, J., Hastie, T., and Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics , 9(3):432--441
work page 2008
-
[7]
Karl, F., Pielok, T., Moosbauer, J., Pfisterer, F., Coors, S., et al. (2023). Multi-objective hyperparameter optimization in machine learning—an overview. ACM Trans Evol Learn Optim , 3(4):1--50
work page 2023
-
[8]
Li, J. et al. (2013a). TCPA : a resource for cancer functional proteomics data. Nat Methods , 10:1046--1047
work page 2013
Show all 21 references
-
[9]
Li, S., Hsu, L., Peng, J., and Wang, P. (2013b). Bootstrap inference for network construction with an application to a breast cancer microarray study. Ann Appl Stat , 7(1):391
2013
-
[10]
Liu, H., Roeder, K., and Wasserman, L. (2010). Stability approach to regularization selection (StARS) for high dimensional graphical models. Adv Neur In , 23
2010
-
[11]
and B \"u hlmann, P
Meinshausen, N. and B \"u hlmann, P. (2010). Stability selection. J Roy Stat Soc B , 72(4):417--473
2010
-
[12]
and Nowak, G
Roberts, S. and Nowak, G. (2014). Stabilizing the lasso against cross-validation variability. Comput Stat Data An , 70:198--211
2014
-
[13]
Schwarz, G. (1978). Estimating the dimension of a model. Ann Stat , pages 461--464
1978
-
[14]
Snoek, J., Larochelle, H., and Adams, R. P. (2012). Practical B ayesian optimization of machine learning algorithms. Adv Neur In , 25
2012
-
[15]
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J Roy Stat Soc B , 58(1):267--288
1996
-
[16]
Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., and Knight, K. (2005). Sparsity and smoothness via the fused lasso. J Roy Stat Soc B , 67(1):91--108
2005
-
[17]
M., van de Brug, T., and van de Wiel, M
van Nee, M. M., van de Brug, T., and van de Wiel, M. A. (2023). Fast marginal likelihood estimation of penalties for group-adaptive elastic net. J Comput Graph Stat , 32(3):950--960
2023
-
[18]
and Wang, L
Wu, Y. and Wang, L. (2020). A survey of tuning parameter selection for high-dimensional regression. Annu Rev Stat Appl , 7(1):209--226
2020
-
[19]
Xue, B., Zhang, M., and Browne, W. N. (2012). Particle swarm optimization for feature selection in classification: A multi-objective approach. IEEE T Cybernetics , 43(6):1656--1671
2012
-
[20]
and Hastie, T
Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net. J Roy Stat Soc B , 67(2):301--320
2005
-
[21]
degrees of freedom
Zou, H., Hastie, T., and Tibshirani, R. (2007). On the “degrees of freedom” of the lasso. Ann Stat , 35(5):2173--2192
2007
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.