{"id":"a65bc33b-5670-48c0-a1b9-10ad06ff16c7","arxiv_id":"2509.06383","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Variational Garrote with automatic differentiation yields more robust sparse variable selection than Ridge and LASSO in very sparse regimes, and the sharp rise in selection uncertainty marks the true number of relevant variables.","lead":"This paper tests a physics-based sparse regression method, the Variational Garrote, and finds it selects variables more reliably than Ridge or LASSO when very few variables matter. Its uncertainty measure also shows a sharp jump that can hint at how many variables are truly relevant.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Heterogeneous sparsity measures make VG's comparative advantage and the uncertainty-transition signal unreliable","rationale":"The reader's weakest assumption identifies exactly the load-bearing weakness: rho_model is not shown to be comparable across Ridge, LASSO, and VG. My stress test confirms this is the crux of the paper's empirical case. The performance comparison at fixed rho_model and the use of sigma_sel to infer rho_data both depend on this measure. The real-data disagreement in inferred rho_data (3 vs 4 vs 2.5) is a concrete red flag that the measure or theory is not method-independent. The paper does provide internally consistent mean-field derivations and the synthetic experiments are suggestive, but without a common sparsity ruler the headline claims are not established. The proposed concrete test directly addresses this by using a uniform thresholding rule across methods, which would either validate the current conclusions or reveal them as artifacts. Given the current evidence, the reader's CONDITIONAL verdict remains appropriate; the authors should be required to run such a test and release code/error bars before full acceptance.","tokens_in":14129,"tokens_out":7046,"duration_ms":78696,"concrete_test":"Recompute all experiments with a single method-independent sparsity definition: for each model, sort variables by a common importance score (e.g., |w_i| for Ridge/LASSO, m_i|w_i| for VG) and threshold to exactly k = round(N rho_model) active variables. Replot Egen and Esel vs rho_model, and recompute sigma_sel and its inferred rho_data. If VG's advantage in the sparse regime diminishes, or if the sigma_sel minimum no longer identifies rho_data across all methods consistently, the central claims are artifacts of the heterogeneous mask definitions.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central comparison of VG versus Ridge and LASSO rests on rho_model being a comparable measure of sparsity. For VG, rho_model is the sum of soft mask values m_i in [0,1]; for LASSO, it is the count of weights above an elbow from a binary-mixture fit; for Ridge, it is the count of weights above a threshold derived from the smallest weights of an unregularized fit. These definitions are heterogeneous and arbitrarily calibrated, and the paper never tests whether equal rho_model corresponds to equal effective model complexity. A concrete symptom appears in the real-data experiments: using the same theoretical curve (Eq. 19) to infer rho_data from sigma_sel gives 3, 4, and 2.5 for Ridge, LASSO, and VG on the CC dataset. If the sparsity measures were method-independent and the theory correct, these estimates should coincide. Their disagreement indicates either non-comparable masks or a flawed theoretical curve, which would invalidate both the comparative performance claim and the transition signal for estimating the correct number of relevant variables.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper revisits the Variational Garrote (VG), a Bayesian variable-selection method with explicit binary selection variables, and proposes a modern automatic-differentiation implementation. The authors compare VG against Ridge and LASSO on synthetic spike-and-slab data (N=M=256) at three true sparsity levels and on two UCI regression datasets. They report that VG is especially accurate in highly sparse regimes, shows more consistent selection across sparsity levels, and exhibits a sharp transition in selection uncertainty that can be used to estimate the number of relevant variables. A mean-field theory is presented for selection error and selection uncertainty, and the latter is inverted on real data to infer data sparsity.","tokens_in":14423,"tokens_out":7233,"duration_ms":83339,"significance":"If the claims hold, the work provides a practical, scalable implementation of a relatively underused sparse-regression method and offers an interpretable, analytically motivated uncertainty curve for choosing the number of relevant variables. The mean-field derivation of E_sel and sigma_sel is transparent, and the use of 20,000 synthetic ensembles is a strength. However, the central comparative claims and the proposed transition signal rest on a sparsity measure that is defined differently across methods and on a theoretical curve whose assumptions are not tested. Without error bars, a valid teacher distribution, and a demonstrated real-data generalization improvement, the significance is limited; the conclusions are currently not fully supported by the evidence presented.","major_comments":[{"comment":"The central comparison uses rho_model as a common sparsity measure, but its definition is model-dependent. For VG it is the sum of soft masks; for LASSO it is a count above a zero-centered binary-mixture 'elbow'; for Ridge it is a count above a threshold derived from the smallest weights of an unregularized fit. No evidence is given that equal rho_model corresponds to equal model complexity. A concrete symptom: inverting Eq. (19) on the Community Crimes data yields rho_data estimates of 3 (Ridge), 4 (LASSO), and 2.5 (VG). If the sparsity measures were comparable and the theory correct, these estimates should agree. Their disagreement undermines both the comparative performance claim and the practical use of the transition curve.","section":"Sec. III.A.2 and III.B"},{"comment":"The spike-and-slab teacher distribution is not normalized for the densities used. The slab density is written as (1/2) rho_data on each side over 1<|w|2. Thus the actual fraction of nonzero teacher weights is not rho_data for the intermediate and high-density regimes, which are precisely the regimes in Figs. 2(a,b) and 3(d,f). This invalidates the controlled sparsity calibration on which the comparison relies.","section":"Eq. (14)"},{"comment":"All performance and selection curves are ensemble averages over 20,000 realizations, but no error bars or confidence intervals are shown. Claims such as 'VG achieves the lowest E_gen' in Fig. 2(c) and 'VG shows a sharp transition in sigma_sel' cannot be assessed for statistical significance. Since the practical proposal (use the transition to estimate the true number of variables) depends on the sharpness and reliability of the increase in sigma_sel, a quantification of ensemble variability is necessary.","section":"Figs. 2 and 3"},{"comment":"The conclusion states that VG 'achieves low generalization error' on real-world data, but Fig. 4 reports only masks, sigma_sel, and inferred rho_data distributions. No real-data E_gen results are shown for CC or BF. Additionally, the 'sharp transition' is not directly observed on real data; it is assumed from Eq. (19) and then used to infer rho_data. This is partly circular and does not validate the proposed signal as a general tool.","section":"Sec. IV and Fig. 4"},{"comment":"The paper acknowledges that for LASSO and VG it is difficult to tune rho_model below rho_data, and that the low-density visualizations are based on extrapolation. This is a load-bearing limitation: the claimed advantage of VG in the under-selection regime rests on this extrapolated region, and the experimental basis for the left branch of the V-shaped curves is thin.","section":"Sec. III.A.4"}],"minor_comments":[{"comment":"The definition of sigma_sel uses ⟨m_i⟨⟨1−m_i⟩, which is ambiguous. The appendix and subsequent usage suggest ⟨m_i⟩(1−⟨m_i⟩), but the displayed formula should be made explicit.","section":"Eq. (18)"},{"comment":"Calling the normalized mixture coefficients P(rho_data) is misleading: they are coefficients of a nonnegative linear regression, not a Bayesian posterior. Please rename or explain the interpretation.","section":"Appendix B"},{"comment":"Reference [40] is incomplete: 'arXiv: Statistics Theory (2010)' lacks a title and ID. The RSS discussion of LASSO [26] should also cite Tibshirani's original paper correctly.","section":"References"},{"comment":"The statement that for Ridge 'all mask values are theoretically m_i=1' is confusing because Ridge weights are nonzero but are not selection masks; please clarify that this refers to the absence of exact zeros.","section":"Sec. III.A.2"},{"comment":"The derivation of Eq. (13) eliminates beta using the stationarity condition (12), but the text does not note that this makes the loss a profile likelihood rather than the original variational bound. This is a useful clarification for readers.","section":"Sec. II.B"}],"recommendation":"major_revision","confidential_remarks":"The manuscript addresses a relevant problem and has a clear synthetic-experiment design, but the current evidence does not establish the central claims. The non-comparability of rho_model across methods is the most serious issue; the theoretical curve in Eq. (19) is derived under assumptions that appear violated by the actual masks, and the inconsistent inferred rho_data on real data is a red flag. The authors should either calibrate the sparsity measures (e.g., by simulation showing equal effective model complexity) or limit the claims to method-specific sparsity levels. I also recommend that the editors require error bars and the real-data generalization results before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read of the VG paper. The method itself is not new—it's Kappen and Gomez's VG—but the systematic comparison across sparsity regimes and the claim that the selection uncertainty curve gives a practical way to identify the number of relevant variables are genuinely worth discussing. The mean-field derivations in the appendix are transparent, and the synthetic experiments are well organized. The observation that VG is more consistent in very sparse regimes is plausible, and the beta-elimination and automatic-differentiation tweaks are sensible, if minor.\n\nThe soft spot, and it's a load-bearing one, is the definition of rho_model. For VG it's the average mask value; for LASSO it's a count of weights above a binary-mixture elbow; for Ridge it's a count above a threshold from an unregularized fit. These are ad hoc, and there's no evidence they measure the same thing across methods. The real-data experiment makes the problem concrete: using the same theoretical curve, Eq. (19), to infer rho_data from sigma_sel gives N*rho_data = 3 for Ridge, 4 for LASSO, and 2.5 for VG on the CC dataset. If the sparsity measures were comparable and the theory correct, these should coincide. They don't, which means either the masks are incompatible or the mean-field curve is not capturing the actual uncertainty. Either way, the comparative performance claims and the transition signal are on shaky ground.\n\nSmaller issues: no error bars despite 20,000 ensembles, only M/N = 1 is considered, and no code is provided. The real-data inference of rho_data also feels circular, since the same curve is used to infer and then validate the transition.\n\nI don't think the paper is incoherent—the authors know the literature and the empirical work is mostly careful—but the central comparison needs a common, principled measure of sparsity. With that fixed, and with validation on data with known sparsity, this could become a useful paper. As it stands, it's a credible draft that deserves peer review but needs substantial revision. I wouldn't cite it in its current form.","headline":"Interesting uncertainty-transition heuristic, but the heterogeneous sparsity measures across methods undermine the comparative claims.","tokens_in":14854,"tokens_out":3349,"would_cite":false,"duration_ms":38655,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62J07","62F15"],"pacs":[],"model":"deepseek-v4-flash","headline":"Variational Garrote beats LASSO in sparse variable selection","keywords":["Variational Garrote","sparse regression","variable selection","variational inference","mean-field approximation","selection uncertainty","LASSO","Ridge regression"],"falsifier":"Simulate spike-and-slab data with a known number of relevant variables (say 5 of 256); grid-search the regularization parameters so that LASSO and VG admit exactly the same number of selected variables. If LASSO's selection error and generalization error then equal VG's, the claimed VG advantage is an artifact of how sparsity is measured; if VG still wins, the soft-mask mechanism is confirmed. A second check: with true rho_data known, test whether the elbow-threshold rule applied to LASSO weights recovers rho_model = rho_data at optimal regularization; a systematic mismatch would invalidate th","tokens_in":14016,"feed_emoji":"🎯","tokens_out":7280,"duration_ms":72192,"temperature":0.7,"pith_summary":"The paper argues that Variational Garrote (VG), a Bayesian regression method with explicit binary on/off variables, becomes the most reliable sparse-selection tool in highly sparse settings when implemented with automatic differentiation. It claims that at the same permitted sparsity level, VG identifies the true relevant variables more consistently than Ridge or LASSO because its soft mask variables can keep several alternatives active instead of concentrating on a single dominant feature. The paper also reports a sharp rise in selection uncertainty once the model admits more variables than the data warrant, and argues this transition can be used to estimate the true number of relevant predictors. If right, VG plus its uncertainty curve gives practitioners a principled way to choose sparsity and identify key variables in high-dimensional data.\"","feed_headline":"Variational Garrote beats LASSO in sparse variable selection","feed_subtitle":"Its uncertainty curve pinpoints the true number of relevant predictors, a signal real-world data confirm.","key_machinery":"The engine is the variational free energy F(m,w) obtained from the VG posterior by marginalizing binary selection variables s_i with a factorized Bernoulli approximation Q(s). F combines a reconstruction energy (including a variance term from s_i^2=s_i), an entropy term that resists masks collapsing, and a -gamma sum m_i sparsity penalty; the inverse temperature beta is eliminated via beta=M/2E. Its minimizer gives masks m_i in [0,1], mean-field probabilities that variable i is selected. The paper also derives piecewise analytic mean-field estimates for selection error E_sel and selection uncertainty sigma_sel as functions of rho_model and rho_data, which let it recognize the over-selection","core_discovery":"On synthetic data whose true weights are drawn from a spike-and-slab distribution with known density rho_data, the paper compares Ridge, LASSO, and VG at matched model density rho_model. Its central finding is that in the highly sparse regime (rho_data around 2% relevant variables), VG has the lowest generalization error and the lowest selection error E_sel. The reason the paper identifies is the soft mask: VG assigns fractional mask values 0