{"id":"7f4ab06a-c268-4cc3-8b96-81b80517a1a1","arxiv_id":"1908.07220","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A sparse Bayesian learning method whose weight prior is scaled by the noise variance, yielding a noise-dependent pruning threshold and improved noise variance estimates.","lead":"This paper proposes a sparse Bayesian regression method, BLS, that combines the Bayesian Lasso's hierarchical prior with the Relevance Vector Machine's fast estimation routine, and scales the weight prior by the noise variance. A generalist reader might care because the method claims to adapt its sparsity to the noise level and to produce better noise variance estimates on noisy data.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central empirical claims hinge on Equation (19), whose inverse-Gamma hyperparameters c and d are never specified; the reported noise-variance estimates and the pruning threshold that depends on them are therefore not reproducible.","rationale":"The reader's weakest assumption correctly identifies the unspecified c and d as the key gap. My independent check of the derivations confirms that Equations (13)-(23) are algebraically sound: the coordinate-wise maximum of the log marginal likelihood is at tau_i = 0 when q_i^2 - s_i <= lambda*sigma^{-2}, and the decomposition in Equation (23) follows. However, the applied claim that BLS is noise-robust and gives better sigma^2 estimates is supported by simulations that depend on the sigma^2 update, Equation (19). Since c and d are never specified, any claimed advantage could be tuned by their choice; the d term is particularly sensitive because it is not scaled by sample size. This does not invalidate the method, but it does mean the central empirical claim is not currently verifiable. The verdict should remain conditional until the authors provide c and d values (or show the results are insensitive to them) and ideally release code. I therefore agree with the reader's CONDITIONAL verdict and recommend no change.","tokens_in":16166,"tokens_out":22835,"duration_ms":205286,"concrete_test":"Implement the BLS algorithm exactly as in Section 3.1 (linear spline kernel, N = 200, Sinc function, 100 generations) and run three settings of the inverse-Gamma prior: c = d = 0, c = d = 1, and c = d = 10. Compare the mean estimated sigma and MSE against the BLS rows of Table 1 for sigma = 0.01 and sigma = 0.1. If the c = d = 0 setting does not match the reported numbers within sampling error, or if the estimated sigma shifts by more than 20% between settings, then the paper's empirical claims are not robust to the unspecified hyperparameters and the manuscript must be revised to state and justify c and d.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that BLS is robust to noise and estimates sigma better than FRVM and FLAP—rests on two legs: the analytical pruning threshold in Equation (17) and the simulation comparisons in Section 3. The analytical derivation is internally consistent: the derivative of the log marginal likelihood with respect to tau_i is correct, and the condition q_i^2 - s_i <= lambda*sigma^{-2} for pruning is derived properly. However, the empirical leg is load-bearing and is not reproducible. The update for sigma^2 in Equation (19) is sigma^2 = (y^T * C_tilde^{-1} * y + 2d)/(N + 2c + 2), where c and d are the shape and scale of the inverse-Gamma prior on sigma^2. The manuscript never states the values used in the simulations. This matters because the pruning condition in Equation (17) depends directly on sigma^2, so any arbitrary shift in sigma^2 changes which basis functions are retained. Moreover, the d term in the numerator is not scaled by N, so for low-noise data (e.g., sigma = 0.01, N = 200, y^T * C_tilde^{-1} * y of order 0.1), setting d = 1 instead of d = 0 changes the estimate of sigma^2 by an order of magnitude. Without c and d (and without code), Tables 1 and 3 cannot be reproduced, and the claimed advantage in variance estimation could be an artifact of a favorable but unreported hyperparameter choice. This is an addressable reproducibility defect, not a mathematical contradiction, so it warrants a conditional rather than a reject verdict.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a sparse Bayesian learning method, BLS, that combines the hierarchical Laplace-prior formulation of the Bayesian Lasso (Park and Casella 2008) with the fast marginal-likelihood maximization of Tipping and collaborators. The weight prior is scaled by the noise variance, and the authors derive an analytical pruning condition (Eq. 17) under which an individual hyperparameter is set to zero. They also give updates for λ (Eq. 18) and σ² (Eq. 19), and they compare BLS with the RVM and the Fast Laplace method on one- and two-dimensional Sinc simulations, as well as with the Bayesian Lasso and Lasso on two real datasets. The central claims are that BLS is sparse, fast, robust to noise, and provides more stable estimates of the noise variance than the compared methods.","tokens_in":16513,"tokens_out":12004,"duration_ms":113984,"significance":"If the empirical claims hold, the paper offers a conceptually simple and computationally fast sparse kernel method whose pruning threshold automatically adapts to the noise level, and it gives a useful alternative for estimating σ² in sparse Bayesian regression. The analytical derivation in Section 2.1 is internally consistent; I checked the derivative of the marginal log-likelihood, the quadratic form, and the threshold condition, and they are correct apart from minor typos. The paper does not ship code, and the missing hyperparameter values are a serious reproducibility gap. The noise robustness is a direct design consequence of the σ²-scaled prior rather than an independent prediction, so circularity is not a concern; the main risk is empirical reproducibility.","major_comments":[{"comment":"The update for σ² uses inverse-Gamma hyperparameters c and d, but the manuscript never states the values used in the simulations, and the corresponding hyperparameters a and b in Eq. (18) are only described as being \"set to low values.\" Because the pruning rule in Eq. (17) depends directly on σ², the retained basis functions and therefore every row of Tables 1, 2, and 3 depend on these unspecified choices. The reported advantage of BLS in estimating σ² could be an artifact of an unreported prior setting. Please report c, d, a, and b, and provide either code or a complete algorithmic protocol; without this, the central empirical claims are not reproducible.","section":"Section 3, Eq. (19)"},{"comment":"The BLS estimates of σ at the lowest noise level are several times the true value: Table 1 reports 0.029 for true σ = 0.01 and Table 3 reports 0.053 for true σ = 0.01. This weakens the blanket statements in Section 1.3 and Section 5 that BLS gives \"more precise\" or \"closest\" estimates of the noise variance. The paper should qualify these claims and discuss the upward bias at low noise, including how it depends on the unspecified hyperparameters c and d.","section":"Tables 1 and 3"},{"comment":"The simulation MSE is reported as ||y* − f(x)||²/N, but the text does not state whether y* is evaluated on the training grid or on an independent test grid. Since the basis functions are centered at training inputs, an in-sample reconstruction error is not the same as predictive performance. Please define the evaluation points explicitly and, ideally, report out-of-sample error as well.","section":"Section 3, Tables 1–3"}],"minor_comments":[{"comment":"The expression \"L(τ)/dτ_i\" should read \"dL(τ)/dτ_i\" in the sentence describing the derivative at τ_i = 0.","section":"Section 2.1, after Eq. (16)"},{"comment":"The sentence \"Θ2 < s_i² + 2s_iλσ⁻²\" should state \"√Θ < s_i² + 2s_iλσ⁻²\".","section":"Section 2.1, threshold discussion"},{"comment":"The equality case q_i² − s_i = λσ⁻² is not handled in the if-else structure; although it has measure zero, the implementation needs a convention for this boundary.","section":"Algorithm 1"},{"comment":"The sentence \"20 genes for 120 samples\" appears inconsistent with the description of 200 predictors; this is likely a typo and should be corrected.","section":"Section 4.1.2"},{"comment":"The shaded region is labeled a \"95% confidence interval\" but it is constructed from the predictive variance in Eq. (25); it should be called a predictive interval.","section":"Figure 1"},{"comment":"The factorization leading to \"σ⁻² q̃_i² − s̃_i ≤ λ\" involves dividing both sides of the previous inequality by σ⁻²; this step should be stated explicitly for clarity.","section":"Eq. (23)"}],"recommendation":"major_revision","confidential_remarks":"The analytical part is the strongest contribution and is sound; the reason for major revision is the missing empirical reproducibility. If the authors supply the hyperparameter values and code, the empirical claims can be checked. The low-noise σ estimates in Tables 1 and 3 should also be addressed, since they qualify the paper's main variance-estimation claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: BLS is a small but real variation on Babacan's Fast Laplace method. The prior variance is scaled by sigma^2, so the pruning threshold inherits the noise level. The derivation of the threshold in Equation (17) is internally consistent; I checked the quadratic and the q_i^2 - s_i condition, and the algebra lines up. This is not a huge step forward, but it is a legitimate model variant and the paper develops it cleanly.\n\nWhat is genuinely new: the conditional Laplace prior from Park and Casella is plugged into the type-II fast marginal likelihood machinery, and the resulting sparsity criterion depends on sigma^2. The comparison against Fast Laplace in Section 2.3 is honest about the difference. The simulations are reasonably designed—same data across methods, 100 repetitions, several noise levels—and the BLS estimates of sigma in Tables 1 and 3 are consistently closer to the true values than FRVM or FLAP. The real-data comparisons with the Bayesian Lasso and ordinary Lasso are a useful addition. Credit is earned there.\n\nThe soft spots are mostly about reproducibility. The update for sigma^2 in Equation (19) uses an inverse-Gamma prior with shape c and scale d, and I cannot find the values used in the simulations anywhere in the paper. The same applies to the Gamma hyperparameters a and b for lambda. This matters because the pruning threshold depend on sigma^2, and the sigma^2 update depends directly on d. A favorable but unreported choice of d could explain part of the reported advantage in variance estimation. No code is provided, so Tables 1 and 3 cannot be independently checked. The abstract's claim about \"irregular datasets\" is also never defined, which is vague rather than fatal. Minor typos in Equation (4) and Equation (18) are not load-bearing.\n\nOne more point: the noise robustness is partly built into the prior, so the simulations demonstrate a design consequence rather than an independent prediction. That lowers the novelty but does not make the work wrong.\n\nOverall, this is a solid incremental methodology paper. The analytical core holds up, the empirical results are plausible, and the authors engage honestly with the relevant literature. The missing hyperparameter values and lack of code are addressable and should be fixed before publication, but they do not amount to a mathematical flaw.\n\nMy recommendation: send it to peer review. A serious referee should ask for exact hyperparameter settings, a reproducibility appendix, and preferably code. I would not desk reject it.","headline":"A legitimate but incremental variant of Fast Laplace—noise-scaled prior is a real difference, the threshold math checks out, but missing hyperparameter values and code make the empirical claims unverifiable as reported.","tokens_in":17023,"tokens_out":2253,"would_cite":false,"duration_ms":26504,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62J07","62F15","68T05"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes BLS, a sparse Bayesian learning model whose pruning threshold is set by the noise variance, so it prunes exactly zero weights and estimates the noise level accurately.","keywords":["Bayesian Lasso","Sparse Bayesian Learning","Relevance Vector Machine","type-II maximum likelihood","noise robustness","hierarchical models","kernel basis functions","variable selection"],"falsifier":"Run the one-dimensional Sinc experiment at $\\sigma = 0.5$ while varying $c$ and $d$ in Equation (19) over a wide range; if the estimated $\\sigma$ and the mean squared error move substantially away from the paper's reported values, or if holding $\\sigma^2$ fixed at the true value changes which basis functions are pruned, the noise-robustness claim does not hold as stated.","tokens_in":15981,"feed_emoji":"🎯","tokens_out":8181,"duration_ms":74917,"temperature":0.7,"pith_summary":"The paper proposes a sparse Bayesian learning model, BLS, that takes the hierarchical prior structure of the Bayesian Lasso and estimates its hyperparameters with the fast type-II maximum-likelihood algorithm used by the Relevance Vector Machine. The central claim is that because every weight carries its own variance hyperparameter, and because the rule for setting that hyperparameter to zero contains the noise variance $\\sigma^2$, the method produces exact zero weights while automatically adjusting its pruning to the amount of noise in the data. The authors argue this matters because the ordinary Bayesian Lasso is not sparse and because existing sparse Bayesian methods such as the Relevance Vector Machine and Fast Laplace estimate the noise variance poorly. On simulated and real data, the paper reports that BLS gives the lowest mean squared error, stable sparse models, and the most accurate estimates of $\\sigma$ among the methods compared.","feed_headline":"Noise level decides which Bayesian Lasso weights survive","feed_subtitle":"BLS sets each weight's pruning threshold from the noise variance, yielding sparser and more stable fits on noisy data.","key_machinery":"The load-bearing object is the hierarchical scale-mixture representation of the Laplace prior: each weight $w_i$ gets a Gaussian prior $w_i \\mid \\tau_i, \\sigma^2 \\sim N(0, \\tau_i \\sigma^2)$ with $\\tau_i \\sim \\mathrm{Exponential}(\\lambda/2)$, so that integrating out $\\tau_i$ recovers the Laplace density. The optimization machinery is the fast marginal likelihood maximization that updates one $\\tau_i$ at a time; the derivative of the log marginal likelihood with respect to $\\tau_i$ is a quadratic, yielding the closed-form update and the pruning threshold in Equation (17). The decisive feature is that $\\sigma^2$ enters that threshold, which is what differentiates BLS from the Fast Laplace method. The estimate of $\\sigma^2$ in Equation (19) is derived from an inverse-Gamma prior with hyperparameters $c$ and $d$.","core_discovery":"On its own terms, the paper establishes that the Bayesian Lasso's Laplace prior, written as a scale mixture of Gaussians with per-weight variance hyperparameters $\\tau_i$, can be optimized by fast marginal likelihood maximization instead of Gibbs sampling. The optimization yields a closed-form pruning rule: $\\tau_i = 0$ whenever $q_i^2 - s_i < \\lambda \\sigma^2$, which after rescaling reads $\\tilde{q}_i^2 - \\tilde{s}_i < \\lambda$; because $\\sigma^2$ appears explicitly, the threshold grows with the noise level and the model prunes more aggressively in noisy data. Simulations with one- and two-dimensional Sinc functions at several noise levels show BLS achieving the lowest average mean squared error and the most accurate estimates of the noise standard deviation, while the comparison methods give heavily biased $\\sigma$ estimates. Applied to variable selection, the same algorithm selects a small, stable set of variables on the diabetes and mammalian eye gene-expression data, with prediction error competitive with the Lasso and the Bayesian Lasso.","pith_inferences":["The authors leave implicit that BLS is a natural candidate for compressive sensing, since Fast Laplace is already used for signal reconstruction and BLS adds a noise-dependent pruning rule to that setting.","An untested implication is that the reported advantage in estimating $\\sigma^2$ may depend on the unspecified inverse-Gamma hyperparameters $c$ and $d$; varying them would show how much of the advantage is structural rather than tuned.","Because the pruning criterion reduces to $\\tilde{q}_i^2 - \\tilde{s}_i < \\lambda$ when the noise scaling is factored out, BLS can be viewed as a bridge between Lasso-type $\\lambda$ shrinkage and RVM-type marginal likelihood pruning, suggesting a unified way to relate regularization strength to noise level."],"forward_implications":["If BLS works as claimed, a user can obtain a sparse kernel model without running a Gibbs sampler, which removes the main computational bottleneck of the Bayesian Lasso.","Because the pruning threshold scales with $\\sigma^2$, the number of retained relevance vectors should adapt to noise automatically, giving denser fits for clean data and sparser fits for noisy data.","The predictive distribution in Equations (24)-(25) supplies both a point prediction and a prediction interval in closed form, so the method is usable for online or repeated prediction tasks.","The same algorithm, with the kernel matrix replaced by the design matrix, yields a variable-selection procedure that achieves exact zero coefficients, something the Gibbs-sampling Bayesian Lasso does not do."],"supporting_citations":[{"why":"Supplies the hierarchical Bayesian Lasso structure, including the Gaussian prior with variance $\\tau_i \\sigma^2$ and the exponential prior on $\\tau_i$, that BLS adopts.","marker":"Park & Casella (2008)"},{"why":"Supplies the sparse Bayesian learning framework and the type-II maximum likelihood estimation on which BLS's inference is based.","marker":"Tipping (2001)"},{"why":"Supplies the analysis of the marginal likelihood that underlies the fast single-hyperparameter updates.","marker":"Faul & Tipping (2002)"},{"why":"Supplies the fast marginal likelihood maximization algorithm used to update one $\\tau_i$ at a time.","marker":"Tipping et al. (2003)"},{"why":"Defines the Fast Laplace method, the closest competitor whose Laplace-prior sparsity BLS modifies by conditioning on $\\sigma^2$.","marker":"Babacan et al. (2010)"},{"why":"Provides the scale-mixture-of-Gaussians identity that justifies representing the Laplace prior hierarchically.","marker":"Andrews & Mallows (1974)"},{"why":"Defines the Lasso baseline used in the variable-selection comparison.","marker":"Tibshirani (1996)"},{"why":"Provides the diabetes dataset used for the empirical variable-selection comparison.","marker":"Efron et al. (2004)"},{"why":"Provides the mammalian eye gene-expression dataset used to test high-dimensional variable selection.","marker":"Scheetz et al. (2006)"}],"fun_headline_variants":["Noise-driven pruning makes Bayesian Lasso sparser","BLS: sparse Bayesian Lasso via noise-aware thresholds","Bayesian Lasso prunes more with higher noise","Noise level sets Bayesian Lasso sparsity threshold"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claimed noise robustness and accurate noise-variance estimation rest on the inverse-Gamma prior for $\\sigma^2$ in Equation (19), whose hyperparameters $c$ and $d$ the paper never specifies; if those values are chosen poorly, the pruning threshold inherits the error.","fun_headline_variants_meta":{"raw":{"variants":["Noise-driven pruning makes Bayesian Lasso sparser","BLS: sparse Bayesian Lasso via noise-aware thresholds","Bayesian Lasso prunes more with higher noise","Noise level sets Bayesian Lasso sparsity threshold"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000245,"raw_usage":{"total_tokens":1518,"prompt_tokens":909,"completion_tokens":609,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":525,"completion_tokens_details":{"reasoning_tokens":557}},"tokens_in":525,"tokens_out":609,"duration_ms":6195,"temperature":1.0,"reasoning_tokens":557,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:22:46.517334+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the one-dimensional Sinc experiment at $\\sigma = 0.5$ while varying $c$ and $d$ in Equation (19) over a wide range; if the estimated $\\sigma$ and the mean squared error move substantially away from the paper's reported values, or if holding $\\sigma^2$ fixed at the true value changes which basis functions are pruned, the noise-robustness claim does not hold as stated.","supporting_citations":[{"cited_title":"& Casella, G","cited_arxiv_id":null,"evidence_quote":"Supplies the hierarchical Bayesian Lasso structure, including the Gaussian prior with variance $\\tau_i \\sigma^2$ and the exponential prior on $\\tau_i$, that BLS adopts."}],"review_version":1}