{"id":"03d649b2-7d20-49ef-ad0a-22619a4d5e31","arxiv_id":"2412.05726","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A closed-form proximal operator for jointly updating coefficients and their adaptive Lasso penalties enables debiased variable selection with arbitrary sparsity structure in nonlinear models.","lead":"This paper develops a proximal gradient method that jointly optimizes regression coefficients and their Lasso penalty strengths, allowing adaptive, debiased variable selection in complex nonlinear models. A closed-form proximal operator for the penalty term makes the method fast enough for large-scale structured sparsity problems.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 2's sβsλ≥1 branch has the step-size factors swapped relative to its own proof; the stated closed-form prox is numerically wrong (e.g., sβ=4, sλ=1, β0=1, λ0=1.5), so the load-bearing proximal operator is false as written.","rationale":"The paper has a genuinely useful idea and extensive experiments; the empirical results and the existence of code are real evidence. However, the central claim rests on the closed-form proximal operator for λ|β|. As typeset, Theorem 2's second branch is inconsistent with its own proof: the cost comparison says choose λ0 when λ0²/sλ < β0²/sβ (equivalently λ0√sβ > |β0|√sλ), while the theorem states the opposite product. The counterexample with sβ=4, sλ=1, β0=1, λ0=1.5 makes the theorem's prediction 1.125 while the true prox value is 0.125 at λ0. This is not a matter of unverified regularity; it is a false numerical statement. Theorem 5's zero branch similarly appears to use λ0² where stationarity gives λ0, and its sign of the log coefficient is inconsistent with the algorithm's `-log λ` term. If these are typesetting errors, they must be corrected before the construction is reproducible; if not, the algorithm's update is wrong in exactly the regime the theorem claims to cover. The reader's convergence concern (Li-Pong conditions unverified, Adam step sizes not bounded by 1/l) is real and remains a second obstacle, but the proximal formula error is more load-bearing because it undermines the core operator itself. A brute-force numerical check would settle it in minutes. Verdict stays CONDITIONAL: the paper should not be accepted until the central theorems are corrected and verified, but the errors look fixable and the empirical framework is promising.","tokens_in":26003,"tokens_out":26121,"duration_ms":236110,"concrete_test":"Numerically verify Theorems 2 and 5 by brute-force global minimization of P1 and P2 (e.g., dense grid plus scipy.optimize multistart) for the counterexample above and for 100 random inputs with sβsλ≥1 and negative log coefficient. If the closed-form solutions disagree with the numerical minima on any case, the theorems as stated are false and the paper's central proximal step needs correction.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 3.1, Theorem 2 gives the closed-form prox for P1. For sβsλ ≥ 1 it states λ* = λ0 if λ0√sλ > |β0|√sβ and λ*=0 otherwise. The proof's cost comparison (λ0²/(2sλ) versus β0²/(2sβ)) instead yields the condition λ0√sβ > |β0|√sλ. A direct check: with sβ=4, sλ=1, β0=1, λ0=1.5, the stated condition is false (1.5 > 2), so the theorem predicts λ*=0 with proximal cost 1.125, while λ*=λ0 gives β*=0 with cost 0.125; the true minimizer is λ0. Theorem 5 has a parallel apparent error: the zero-branch solution and condition use λ0² where the stationarity solution gives λ0, and the sign convention for the log term is inconsistent with the algorithm's `-log λ` prox. Since the algorithm's update is exactly this prox operator, the central construction is not correctly specified as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a unified proximal-gradient framework for adaptive Lasso problems in which the per-coordinate penalty coefficients λ are treated as decision variables and optimized jointly with the regression coefficients β. The central objects are closed-form proximal operators for the variable-penalty L1 function λ|β| (Theorem 2) and for its log-penalized variant λ|β| + a log λ (Theorem 5). The authors state oracle properties for the resulting estimator, introduce priors that encode group and overlapping-group sparsity structure, and evaluate the method on synthetic data, UCI benchmarks, and two social-science case studies. The main claim is that this one-algorithm-fits-all approach is competitive with, and often faster than, specialized solvers such as glmnet and MLGL across Gaussian, Bernoulli, Negative Binomial, and Cauchy likelihoods.","tokens_in":26250,"tokens_out":9805,"duration_ms":91069,"significance":"If the central construction were correct, the paper would make a useful contribution: moving sparsity structure from the nonsmooth penalty into a smooth prior while keeping a single proximal operator is an attractive idea, and the empirical scope is broad, including non-Gaussian likelihoods, structured sparsity, real data, and large-scale case studies. The authors provide code and appear to have run a substantial battery of comparisons. However, the load-bearing proximal operator is stated incorrectly in Theorem 2, and several theoretical statements either are unverified or appear false as written. These issues affect the correctness of the algorithm actually deployed, not merely the presentation, so the current version cannot be accepted. With careful correction of the proximal operators and a honest restatement of the convergence and oracle-property results, the framework could be a valuable addition to the sparse-regression toolbox.","major_comments":[{"comment":"The sβsλ ≥ 1 branch is incorrect as stated. The theorem gives λ* = λ0 when λ0√sλ > |β0|√sβ and λ* = 0 otherwise, but the proof in Appendix A compares the costs λ0²/(2sλ) and β0²/(2sβ) and therefore yields the opposite placement of sβ and sλ: λ* = λ0 when |β0|√sλ < λ0√sβ. A concrete counterexample: with sβ = 4, sλ = 1, β0 = 1, λ0 = 1.5, the theorem's condition is false (1.5 > 2), so it predicts λ* = 0 with proximal cost 1.125, whereas (β*,λ*) = (0,1.5) has proximal cost 0.125 and is the true minimizer. Since the algorithm in Section 3.3 applies exactly this proximal operator, the central construction is not correctly specified as written. Remark 3 reproduces the same incorrect branch and should be corrected together with the theorem.","section":"Section 3.1, Theorem 2"},{"comment":"The log-term proximal operator has several apparent errors. In the zero branch, Eq. (12) and the surrounding condition use λ0² where the stationarity equation gives λ0; the correct λ* at β = 0 is (λ0 + sqrt(λ0² + 4 sλ a))/2, not (λ0² + sqrt(λ0² + 4 sλ a))/2. The condition should also involve |β0|, not β0, since the theorem is stated for arbitrary real β0. In addition, the theorem is written for the objective λ|β| + a log λ, while the algorithm in Section 3.3 applies the prox of λ|β| − (1/τ) log λ; with the implemented sign, a is negative and the square root in the formulas can fail to be real. The relation between a and τ and the sign convention must be clarified and the formulas re-derived for the objective actually used in the experiments.","section":"Section 3.2, Theorem 5"},{"comment":"The convergence discussion invokes Li and Pong (2015), Theorem 5, but none of the hypotheses are verified for the objectives in this paper. The theorem requires a twice continuously differentiable convex q such that ∇²h + ∇²q has spectrum bounded by l, and a step size below 1/l; the paper does not establish such a bound for any of the negative log-likelihoods or priors used in Section 6. Moreover, the actual algorithm uses Adam-based step sizes and stochastic minibatches, for which the deterministic proximal-gradient result does not directly apply. The text itself concedes that the theorem does not imply the existence of cluster points. This leaves the main algorithmic claim—that the proximal iteration reliably finds a useful stationary point—without theoretical support, and the gap is load-bearing for the 'one-algorithm-fits-all' statement in Section 7.1.","section":"Section 3.3.1, Convergence"},{"comment":"Part 2 of Theorem 8 is not established by the given proof and appears to be false under the paper's own assumptions. The proof states that both |β| and τλ* are decreasing in |β|, but the first term is increasing. For the half-Cauchy prior used in Section 5.1, one can compute the derivative of |β| + g'_{τn}(|β|) at |β| = 0 as 1 − τn²/(ρ'(1)² + ρ''(1)) = 1 − τn² (with the paper's normalization), which is negative for the linearly growing τn = nτ0 used in Theorem 12. Hence the minimum is not at β = 0 for large n. The proof of the oracle property in Appendix B is also a sketch: the zero-coordinate result is asserted after a sign-of-derivative argument, and the expansion mixes terms of different orders in √N without a complete justification. Since the oracle property is one of the paper's stated contributions, this needs a correct statement and a complete proof.","section":"Section 4, Theorem 8"}],"minor_comments":[{"comment":"There are numerous typos, including 'Deonte' for 'Denote' in Section 1, 'wieghts' in Section 2.1, and 'sythetic' in the abstract; the paper would benefit from a careful proofreading pass.","section":"Throughout"},{"comment":"The notation is inconsistent: the main text uses sβ and sλ, while the figure caption and the reduced form in Eq. (11) use sx and sλ (or sxsλ); unify the notation.","section":"Figure 2 and Section 3.1"},{"comment":"The notation pγ = C(0,1)+ and λi|γg(i) ∼ N(γg(i), 1/√N)+ is nonstandard; please define the half-Cauchy and truncated-normal conventions explicitly, including which side is truncated.","section":"Section 5.2"},{"comment":"Lemma 7, part 4, contains an n in the denominator that is absent from the corresponding statement in Theorem 8; the subscripted τn and τ notation should be made consistent throughout Section 4.","section":"Lemma 7 and Theorem 8"},{"comment":"The oracle property is stated as Theorem 12 in the main text but is labeled Theorem 9 in Appendix B; align the numbering and the assumptions between the two statements.","section":"Appendix B and theorem numbering"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is salvageable, but the current version has load-bearing errors in the central proximal operator and unverified convergence and oracle-property statements. I recommend major revision rather than rejection, because the corrections appear to be local in scope: the proof of Theorem 2 actually contains the correct condition, and Theorem 5 can be re-derived for the implemented sign convention. The empirical comparisons are extensive and suggest the method may work in practice, so a careful revision with corrected theorems and a more cautious statement of the guarantees could make this a useful contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The core idea—treating the Lasso penalty coefficients as extra decision variables and using one proximal operator for any smooth prior—is genuinely useful, and the closed-form prox for λ|β| is a real piece of work. But the paper as written has a load-bearing error in Theorem 2. In the sβsλ ≥ 1 branch the condition is stated as λ0√sλ > |β0|√sβ, whereas the proof's own comparison gives λ0√sβ > |β0|√sλ. Try sβ=4, sλ=1, β0=1, λ0=1.5: the stated condition sends you to λ*=0 with proximal cost 1.125, while λ*=λ0 gives cost 0.125. So the theorem is false as written. Theorem 5 has a parallel issue in the zero branch: it uses λ0² where stationarity gives λ0. These look like simple swaps or typos, not a broken research direction, but they sit in the central construction. An editor should ask for a corrected, proofread version before the claims are accepted.\n\nWhat is actually good: the variable-penalty L1 proximal operator appears to be new; the MAP interpretation of reweighted ℓ1 is a nice unification; and shifting structured sparsity into the smooth prior so the same prox handles overlapping groups is a clean idea. The experiments are broad—four likelihoods, synthetic and real data, two case studies—and the code is available. The runtime comparisons are honest, and Section 7.3 openly acknowledges limitations like noisy coefficient trajectories and poor conditioning. That transparency earns credit.\n\nSoft spots beyond the theorem typo. The convergence argument invokes Li and Pong but never verifies the required Hessian bounds or step-size conditions for these objectives, and the Adam step sizes are not guaranteed to satisfy them. The oracle property proof is a sketch following Fan–Li, with an unusual 1/N neighborhood, and it does not cover the structured priors used in the experiments. The empirical section tunes several hyperparameters on the same data and omits debiased lasso baselines, so the competitiveness claim is not as clean as it could be. None of these would be fatal if the theorem statement were right and the convergence section were tightened.\n\nWho this is for: anyone working on sparse estimation with nonconvex or non-Gaussian likelihoods; the one-algorithm-for-any-structure angle is attractive. It deserves a serious referee, but the referee should insist on a corrected theorem and a verified convergence argument before publication.","headline":"A genuinely useful framework for debiased structured sparsity with a new proximal operator, but Theorem 2 is numerically wrong as stated (swapped inequality) and needs correction before the method can be trusted.","tokens_in":26782,"tokens_out":3490,"would_cite":false,"duration_ms":31236,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62J07","65K05","62F12"],"pacs":[],"model":"deepseek-v4-flash","headline":"Treating adaptive Lasso penalty strengths as decision variables yields one proximal-gradient algorithm for debiased variable selection with any smooth likelihood and sparsity structure.","keywords":["variable selection","proximal algorithms","adaptive Lasso","structured sparsity","penalized regression","oracle property","MAP estimation","non-Gaussian regression"],"falsifier":"Take a small negative-binomial regression with correlated predictors, run the paper's proximal-gradient algorithm from many random starting points with its default step-size schedule, and check whether all runs settle on the same coefficients at a point where no small perturbation improves the joint objective; disagreement or non-stationary endpoints would falsify the claim that one algorithm reliably handles arbitrary smooth likelihoods.","tokens_in":25797,"feed_emoji":"🎯","tokens_out":14428,"duration_ms":110519,"temperature":0.7,"pith_summary":"Adaptive Lasso shrinks each coefficient with its own penalty weight, but choosing those weights is a bottleneck in complex models. This paper treats the weights themselves as decision variables, placing a prior on them and estimating them jointly with the coefficients in a maximum-a-posteriori sense. The key move is a closed-form proximal operator for the two-dimensional function $\\lambda|\\beta|$, which makes joint optimization by proximal gradient tractable and lets any sparsity structure be encoded through a smooth prior on $\\lambda$ rather than through a bespoke nonsmooth penalty. The authors prove an oracle property for the resulting estimator and show empirically that it matches or beats specialized packages on Gaussian and non-Gaussian regressions while scaling to large nonlinear applications such as vaccine-behavior and migration models.","feed_headline":"One proximal operator learns Lasso weights and coefficients together","feed_subtitle":"Same machinery handles any smooth likelihood and any sparsity structure by placing structure in a prior.","key_machinery":"The load-bearing object is the proximal operator — the map sending a point to the minimizer of the function plus a quadratic proximity term — applied to the variable-penalty norm $g(\\beta,\\lambda)=\\lambda|\\beta|$ and its log-augmented variant $\\lambda|\\beta|+a\\log\\lambda$. Because the MAP objective separates across coordinates, the proximal problem reduces to a two-dimensional scalar problem, and the paper gives its closed-form solution: for $s_\\beta s_\\lambda<1$, the optimal $\\lambda^*$ is $\\lambda_0$ when $\\lambda_0\\ge|\\beta_0|/s_\\beta$, and otherwise $(\\lambda_0-s_\\lambda|\\beta_0|)_+/(1-s_\\lambda s_\\beta)$, with $\\beta^*=(|\\beta_0|-s_\\beta\\lambda^*)_+\\operatorname{sgn}(\\beta_0)$; the log term replaces this with a square-root formula that keeps $\\lambda$ strictly positive. This operator lets a single proximal-gradient loop optimize $\\beta$ and $\\lambda$ together, and because sparsity structure lives in the smooth prior $p_\\lambda$, the same operator handles arbitrary structure.","core_discovery":"The central claim is that adaptive Lasso can be learned end to end: instead of preselecting penalty coefficients $\\lambda_p$ with a pilot estimator or iteratively reweighting them, one minimizes $$\\min_{\\$\\beta$,\\$\\lambda$} L(\\$\\beta$)+\\sum_p(\\tau\\lambda_p|\\beta_p|-\\log\\lambda_p)-\\log p_\\$\\lambda$(\\$\\lambda$)$$ over coefficients and penalty strengths jointly. The authors establish a closed-form proximal operator for the nonconvex term $\\lambda|\\beta|$, including a log-barrier variant, so proximal gradient iterations can update $\\beta$ and $\\lambda$ together. They prove that with an independent sufficiently diffuse prior the estimator has the oracle property: it recovers the zero pattern with probability tending to one and is asymptotically normal with the same covariance as if the zero coefficients were known. The same machinery, they argue, imposes group, overlapping-group, or hierarchical sparsity simply by choosing a smooth hyperprior on $\\lambda$, shifting structure from the nonsmooth to the smooth part of the objective. Numerical studies on synthetic and real data and two social-science applications are offered as evidence that one algorithm is competitive with structure-specific implementations.","pith_inferences":["If the convergence gap is closed, this framework could become a default black-box sparse-estimation routine, since the same proximal operator would serve any likelihood and any sparsity structure; a testable extension is to compare it against bespoke solvers on graph- and tree-structured sparsity benchmarks.","The same variable-coefficient prox idea may extend to other penalties such as SCAD, MCP, or bridge penalties by letting their tuning parameters vary; closed forms for those proximal operators would generalize the approach beyond Lasso-type terms.","Numerical evidence that structured priors can improve prediction even when the effect-hierarchy principle is weak suggests that group assumptions in complicated likelihoods are best chosen by validation rather than by prior belief.","With the log term removed, the bare $\\lambda|\\beta|$ proximal operator exhibits the possibility of driving $\\lambda$ to zero, hinting at a formulation that induces sparsity directly in penalty strengths as well as coefficients."],"forward_implications":["A single proximal operator replaces the bespoke prox computations for group Lasso, overlapping group Lasso, and hierarchical sparsity; users can impose new structures by writing a smooth prior.","For large-data nonlinear models the method scales to objects such as the hurdle gravity model with about two hundred thousand candidate interaction terms, by building the interactions on the fly inside minibatches.","The oracle property means that, asymptotically, the estimator selects the zero pattern correctly and has the same limiting distribution for nonzero coefficients as an oracle that knew the pattern.","On synthetic non-Gaussian regressions and eight benchmark datasets the method is competitive in accuracy and usually faster than specialized packages, and it also handles Cauchy and negative-binomial likelihoods that those packages do not.","Active coefficients are less biased than under a fixed-penalty Lasso because each $\\lambda_p$ can shrink to a small value for strong signals."],"supporting_citations":[{"why":"introduces the adaptive Lasso and its oracle property, the penalty form this paper generalizes by learning lambda.","marker":"Zou (2006)"},{"why":"supplies the nonconcave-penalty framework and oracle-property definitions the asymptotic results extend.","marker":"Fan and Li (2001)"},{"why":"proposes reweighted $\\ell_1$ minimization, which the paper reinterprets as a coordinate-descent case of its joint MAP objective.","marker":"Candes et al. (2008)"},{"why":"provides the proximal-gradient machinery and proximal-operator background the algorithm is built on.","marker":"Parikh et al. (2014)"},{"why":"gives the nonconvex convergence theorem invoked to claim descent and stationarity of cluster points.","marker":"Li and Pong (2015)"},{"why":"defines soft thresholding, the Lasso proximal operator that the variable-penalty prox generalizes.","marker":"Donoho (1995)"},{"why":"introduces group Lasso, a structured-sparsity competitor whose proximal structure the prior-based approach avoids.","marker":"Yuan and Lin (2006)"},{"why":"provides the coordinate-descent software baseline used for speed and accuracy comparisons on GLMs.","marker":"Friedman et al. (2010a)"},{"why":"supplies the stochastic proximal-gradient convergence result used for the large-scale migration application.","marker":"Rosasco et al. (2014)"}],"fun_headline_variants":["Lasso's penalty weights become trainable parameters","Jointly optimize coefficients and Lasso penalties via proximal gradient","End-to-end learning of Lasso penalties with a proximal operator","One proximal iterate tunes both coefficients and penalty weights","Adaptive Lasso with learned penalties: oracle property included"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method depends on the iterative optimization landing on a good solution rather than stalling or drifting, and the paper does not verify that its chosen step sizes satisfy the conditions its convergence guarantee requires.","fun_headline_variants_meta":{"raw":{"variants":["Lasso's penalty weights become trainable parameters","Jointly optimize coefficients and Lasso penalties via proximal gradient","End-to-end learning of Lasso penalties with a proximal operator","One proximal iterate tunes both coefficients and penalty weights","Adaptive Lasso with learned penalties: oracle property included"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000566,"raw_usage":{"total_tokens":2699,"prompt_tokens":982,"completion_tokens":1717,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":598,"completion_tokens_details":{"reasoning_tokens":1639}},"tokens_in":598,"tokens_out":1717,"duration_ms":12944,"temperature":1.0,"reasoning_tokens":1639,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:25:06.546937+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a small negative-binomial regression with correlated predictors, run the paper's proximal-gradient algorithm from many random starting points with its default step-size schedule, and check whether all runs settle on the same coefficients at a point where no small perturbation improves the joint objective; disagreement or non-stationary endpoints would falsify the claim that one algorithm reliably handles arbitrary smooth likelihoods.","supporting_citations":[{"cited_title":"Enhancing sparsity by reweighted _1 minimization","cited_arxiv_id":null,"evidence_quote":"proposes reweighted $\\ell_1$ minimization, which the paper reinterprets as a coordinate-descent case of its joint MAP objective."},{"cited_title":"Proximal algorithms","cited_arxiv_id":null,"evidence_quote":"provides the proximal-gradient machinery and proximal-operator background the algorithm is built on."},{"cited_title":"Global convergence of splitting methods for nonconvex composite optimization","cited_arxiv_id":null,"evidence_quote":"gives the nonconvex convergence theorem invoked to claim descent and stationarity of cluster points."}],"review_version":1}