{"id":"ccfbd709-58da-49a5-8bf7-c031a7f64e62","arxiv_id":"2412.16558","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"PNAIS adapts importance sampling proposals using scaled Newton proximal steps, enabling efficient estimation for targets that are not differentiable.","lead":"This paper introduces PNAIS, an adaptive importance sampling algorithm for non-smooth target distributions that combines Newton-type scaling with proximal gradient updates of the proposal parameters. It reports lower estimation error than ablated versions in constrained and sparse-prior examples, which matters for Bayesian inference with sparsity-inducing priors and constraints.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Covariance update (13) couples the damping factor to proposal width; if backtracking ever sets θ<1 in sparse-prior targets, the Gaussian proposal is lighter-tailed than the target and importance weights are unbounded, yet no θ/ESS diagnostics are reported.","rationale":"The reader's stated weakest assumption is that proximal-Newton convergence without resampling transfers to the full resampling loop. I do not think that is the load-bearing point: the paper makes no theoretical claim that the adaptation converges inside PNAIS, and importance-sampling validity only requires the conditional proposal densities to be proper, which they are. The real soft spot is the unexplored coupling between the backtracking damping θ in (11) and the covariance adaptation in (13). If θ<1, Σ=θΓ is smaller in the smooth-part metric than the nonsmooth target's effective scale, so π/q grows without bound and the estimator variance can be infinite. This would make the Table II gains in Example B a small-sample artifact rather than a general property. The paper includes no θ trace, no ESS, and no tail-ratio diagnostic, so the question is open. The proposed check settles it. I credit the paper for candidly reporting Table S-III, where the first-order variant wins under LR resampling in Example B; that honesty supports a conditional rather than dismissive verdict, and I keep the reader's CONDITIONAL as no change.","tokens_in":12289,"tokens_out":20990,"duration_ms":199273,"concrete_test":"Instrument Example B (Section IV-B) to record the accepted θ^{(t)}_n and the per-iteration maximum normalized weight and effective sample size over the 100 runs. Then: (a) if any θ<1 is accepted, estimate the weight tail ratio max_x π(x)/q(x) on a radial grid using that θ; (b) rerun with θ forced to 1 and also with Student-t proposals (ν=5), keeping all other settings identical. If the MSE and ESS are unchanged when θ is restricted to 1, the concern is resolved. If forcing θ=1 degrades performance, or if θ<1 yields unbounded weight ratios, the Table II gains are not robust to the damping-covariance coupling.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The empirical claim rests on the adaptation in (10)-(13). The covariance update (13) sets Σ=A=θΓ, so for a Gaussian proposal the precision is θ^{-1}Γ^{-1}=θ^{-1}∇²f when Γ=(∇²f)^{-1}. The damping θ is chosen by a backtracking line search reducing θ until the target density increases. For targets of the form (2) with g=α||x||₁, the target has the smooth quadratic factor plus exp(-α||x||). If θ=1, the proposal is heavier-tailed than the target and weights decay. But if the line search ever accepts θ<1, then log(π/q) ≈ (1/(2θ)-1/2)(x-μ)^T ∇²f(μ)(x-μ) - α||x|| + linear, which tends to +∞ along any direction where ∇²f(μ) is positive definite. The importance weights are then unbounded and the IS variance can be infinite. The paper reports no accepted θ values, no effective sample sizes, and no tail diagnostics for Example B, so the low relative MSE in Table II may reflect a favorable regime where θ stays near 1 rather than a robust property of the adaptation. This makes the general claim of efficiency for nonsmooth sparse priors conditional on the line search rarely shrinking θ.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PNAIS (Proximal Newton Adaptive Importance Sampler), an adaptive importance sampling method for target densities of the form exp(-f-g), where f is differentiable and g is convex but nonsmooth (e.g., an L1 penalty or an indicator of a constraint set). At each iteration, after a 'glocal' resampling step, each proposal mean is updated by one scaled proximal Newton step (Eq. 10) that uses the gradient and Hessian of f, and each proposal covariance is set to the same scaling matrix (Eq. 13) used in that step. The paper reports an ablation study on two 2D examples (a Gaussian mixture truncated to a simplex, and a Gaussian likelihood with an L1 prior) plus a high-dimensional constrained example in the supplement, showing that PNAIS often attains lower relative mean squared error than ablated variants. The central claim is that the proximal Newton adaptation improves efficiency for non-smooth targets without breaking the validity of the importance sampling estimators.","tokens_in":12526,"tokens_out":10472,"duration_ms":94721,"significance":"If the empirical claims hold, PNAIS is a useful extension of gradient-based adaptive importance sampling to nonsmooth targets, with potential impact in Bayesian signal processing and sparse regression. The use of proximal operators is principled, and the ablation design (comparing with no adaptation, no covariance adaptation, robust covariance adaptation, and gradient-only mean adaptation) is sensible. The paper's main strength is the algorithmic combination of a damped proximal Newton step with population Monte Carlo resampling, which is plausible and clearly explained. A notable weakness is the absence of any theoretical analysis of the full resampled algorithm, and the empirical evidence is reported without uncertainty quantification. The paper also stops short of addressing a potential tail-heaviness problem in the proposal adaptation when the damping parameter is below one. These issues limit the strength of the general efficiency claim, but they are addressable in a revision.","major_comments":[{"comment":"The covariance update (13) sets Σ_n^(t+1) = A(µ~_n^(t)) = θ_n^(t) Γ(µ~_n^(t)). For a target of the form (2) with a convex quadratic f and g = α||x||_1, the Gaussian proposal with covariance θΓ is lighter-tailed than the target whenever θ<1, because the proposal precision is θ^{-1}∇²f. Then log(π/q) contains the positive quadratic term (1/(2θ)-1/2)(x-µ)^T∇²f(µ)(x-µ), which tends to +∞ as ||x||→∞, so the importance weights are unbounded and the IS variance can be infinite. The paper reports no accepted θ values, no effective sample sizes, and no tail diagnostics for Example B, so the low relative MSE in Table II may reflect a favorable regime where the backtracking keeps θ near 1 rather than a robust property of the adaptation. Please provide an analysis or a safeguard (e.g., decouple the damping for the covariance so that the proposal is never lighter-tailed than the smooth part, or use a heavier-tailed proposal family), and report θ/ESS statistics in the experiments.","section":"Section III-A/B"},{"comment":"The convergence guarantees cited for the mean update (10) apply to deterministic iterates without resampling (see [42], [6]). In PNAIS, the sequence fed into (10) is the resampled mean µ~_n^(t) from the GLR step (Section III-A), which depends on the weighted particle pool and is therefore stochastic; the descent properties or the assumptions of [42] (e.g., Lipschitz continuity of the Hessian along the iterates) are not established for the resampled sequence. The paper also gives no argument that the self-normalized estimators built from the final weighted sample pool are consistent as (N,K,T) grow. Since the central claim is about estimator accuracy, the paper should either provide a rigorous justification of the full resampled algorithm under stated conditions, or explicitly frame PNAIS as a heuristic whose properties are only empirical.","section":"Section III-A/B"},{"comment":"All relative MSE results are reported as point estimates over 100 independent runs, with no standard errors, confidence intervals, or significance tests. For example, in Table II (right), the relative MSE for E[X] is 1.56e-5 for PNAIS(13) versus 1.10e-5 for PNAIS-grad(13), a difference of only about 40% that may not be statistically significant at 100 runs; the text's claim that PNAIS is 'superior' in this example is not supported without uncertainty quantification. Please provide standard errors, paired statistical comparisons, or boxplots for the main tables so that the superiority claims can be properly assessed.","section":"Section IV"}],"minor_comments":[{"comment":"The notation for the proximity operator in Eq. (10) is inconsistent with Eq. (6) and with the supplementary material (which uses prox_{A^{-1},g}). Please clarify whether the metric matrix is A or A^{-1} both in the main text and in the inner-solver description.","section":"Section III-B"},{"comment":"The ground-truth integrals for Example A are said to be 'determined by numerical integration with a rough grid.' Please specify the grid resolution and provide an accuracy check (e.g., convergence under grid refinement) so that the benchmark values can be considered reliable.","section":"Section IV"},{"comment":"The sentence 'PNAIS shows superior performance with the Hessian-based covariance adaptation and without metric acceleration in the proximal gradient mean adaptation' is ambiguous: in Table II, the PNAIS-grad variant with Eq. (13) actually achieves lower MSE for E[X] and E[X^2] than the full PNAIS, while PNAIS is better for Z. Please rephrase to avoid contradicting the table.","section":"Section IV"},{"comment":"The paper would benefit from a brief discussion of the choice of Gaussian proposals in the light of the tail-heaviness concern raised above; a Student-t proposal with finite degrees of freedom would be a safer default for nonsmooth targets.","section":"Section III"},{"comment":"No code is released. Given that the algorithm is described in full, providing reference code (even a MATLAB/Python snippet) would substantially improve reproducibility and uptake.","section":null}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThis paper does something new and reasonable: it ports a scaled proximal Newton step into the mean adaptation of an adaptive importance sampler, with two safety rules for when the smooth part is not log-concave. The covariance update (13) that ties proposal scale to the inverse Hessian is a natural local Gaussian approximation. The ablation is well structured, the high-dimensional supplement is a good-faith effort, and the reported gains are often large.\n\nThat said, the evidence is not fully pinned down. There is no convergence or unbiasedness analysis of the full algorithm with resampling; the cited optimization results are for the iteration without resampling. The paper reports point estimates of relative MSE over 100 runs but no standard errors, so \"largely superior\" is hard to judge when differences are sometimes within a factor of 2-3. No code or data is released, and the closest prior proximal AIS method ([53]) is cited but never compared against. That last omission matters because the entire pitch is better handling of nonsmooth targets.\n\nThe stress-test concern about theta<1 producing lighter-tailed Gaussian proposals than an exp(-alpha||x||) target is mathematically valid: the importance weights would be unbounded. In Example B, though, f is a convex quadratic, so the undamped Newton step should usually be accepted and theta should stay at 1; the bad regime may not arise there. Still, the paper reports no accepted theta values and no ESS, so it cannot rule the regime out, and in nonconvex examples (Example A) damping might shrink covariances in ways that hurt. This deserves a note in the paper, not a rejection.\n\nOverall, the central idea is plausible and the experiments are consistent with it. The missing error bars, code, and baseline comparison are the soft spots, not the method itself. A serious referee could fix most of this in revision. I would send it to review.\n\nBest,\n[You]","headline":"Plausible extension of adaptive importance sampling to nonsmooth targets via a proximal Newton step; the experiments support it, but missing error bars, code, and the closest baseline leave the case conditional.","tokens_in":13077,"tokens_out":2544,"would_cite":false,"duration_ms":25153,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["65C05","62F15","90C25","65K05"],"pacs":[],"model":"deepseek-v4-flash","headline":"A scaled Newton proximal step lets importance samplers adapt proposals to targets that split into a smooth part and a convex nonsmooth part, extending gradient-based adaptation to nondifferentiable Bayesian models.","keywords":["adaptive importance sampling","proximal methods","Newton method","nonsmooth target distributions","proposal adaptation","population Monte Carlo","sparse priors","preconditioning"],"falsifier":"Take a unimodal target of the form $\\exp(-f-g)$ with a unique minimizer, run PNAIS with GLR resampling, and compare the adapted means and estimator MSE against the same update with resampling disabled; if the resampled means drift away from the minimizer or the MSE fails to decrease while the no-resampling variant converges, the transfer assumption is violated.","tokens_in":12063,"feed_emoji":"🎲","tokens_out":11062,"duration_ms":89126,"temperature":0.7,"pith_summary":"Adaptive importance samplers improve their proposals over time, but most gradient-based adaptation requires a differentiable target. This paper proposes PNAIS, which adapts both the means and the covariances of a population of proposals using a scaled Newton proximal gradient step, so that the target need only split into a smooth part and a convex nonsmooth part. Experiments on a simplex-constrained Gaussian mixture and a Gaussian likelihood with a sparse $\\ell^1$ prior show that PNAIS attains substantially lower relative mean squared error than non-adaptive and first-order ablated variants. If the result holds, gradient-based proposal adaptation becomes available for a wide class of nonsmooth Bayesian models, including sparsity-promoting priors and hard constraints.","feed_headline":"Proximal Newton step adapts importance samplers to nonsmooth targets","feed_subtitle":"Sparse and constrained Bayesian targets can now be sampled with Hessian-aware proposal moves.","key_machinery":"The load-bearing device is the scaled proximal Newton update in equation (10), where the proximity operator of $g$ is taken with respect to the metric $A^{-1}$ with $A = \\theta\\,\\Gamma$; this simultaneously plays the role of a gradient step for the smooth part $f$ and a proximal step for the nonsmooth part $g$. The matrix $\\Gamma$ is the inverse Hessian of $f$ when it is positive definite, and the previous proposal covariance otherwise, and $\\theta$ is a damped factor tuned by backtracking to keep the step stable. The covariance of each proposal is then set equal to $A$, coupling the proposal spread to the local curvature. When the proximal step has no closed form, a dual forward-backward algorithm computes it, and the 'glocal' resampling scheme (GLR) interleaves local and global resampling to form the next generation of proposals.","core_discovery":"The central claim is that for a target of the form $\\tilde{\\pi}(x) \\propto \\exp(-f(x)-g(x))$ with $f$ differentiable and $g$ convex nonsmooth, the mean update $\\mu^{(t+1)}_n = \\mathrm{prox}_{A^{-1},g}\\bigl(\\tilde{\\mu}^{(t)}_n - A(\\tilde{\\mu}^{(t)}_n)\\nabla f(\\tilde{\\mu}^{(t)}_n)\\bigr)$ with $A(\\tilde{\\mu}) = \\theta\\,\\Gamma(\\tilde{\\mu})$, where $\\Gamma$ is the inverse Hessian of $f$ when positive definite and the current proposal covariance otherwise, together with the covariance update $\\Sigma^{(t+1)}_n = A(\\tilde{\\mu}^{(t)}_n)$, produces a sequence of proposals that efficiently concentrate mass on the target. The proximal operator of $g$ handles the nonsmooth part (projection for constraints, soft-thresholding for $\\ell^1$), while the Newton scaling supplies local second-order information of the smooth part. The paper demonstrates this by showing that, on the tested examples, PNAIS achieves the lowest relative MSE among the variants considered, particularly for the normalization constant estimate.","pith_inferences":["A natural ablation the paper does not run is to disable resampling entirely; comparing PNAIS with and without GLR would isolate whether the optimization step's convergence guarantees survive the resampling loop, and would directly test the weakest assumption.","If the Hessian-inverse covariance choice is read as a local Laplace approximation, PNAIS can be seen as building an adaptive Gaussian approximation of a nonsmooth posterior; this interpretation is not developed in the paper.","Because the update only requires a prox for $g$, the same scheme could in principle accommodate nonconvex regularizers with well-defined prox (e.g., MCP or SCAD), though the safe rules and convergence arguments would need revisiting.","A benchmark against other sampling schemes that handle nonsmooth targets, such as proximal MCMC with Moreau-Yosida smoothing, would clarify where PNAIS sits relative to existing tools; the current comparisons are only against ablated self-variants."],"forward_implications":["Sparsity-inducing priors such as $\\ell^1$ and hard constraints such as simplex or $\\ell^2$-ball domains can be handled by AIS without smoothing or variable transformation.","The covariance rule (13) provides a data-dependent proposal scale, removing the need to hand-tune a fixed isotropic covariance.","The algorithm's core only needs a computable proximity operator for $g$, so new nonsmooth terms can be plugged in without changing the machinery.","The reported computational time is comparable to first-order and non-adaptive variants, suggesting second-order information comes at little extra cost.","The GLR scheme's combination of diversity-preserving local resampling with global resampling every $\\Delta$ iterations appears to keep proposals from collapsing while still letting optimization move them."],"supporting_citations":[{"why":"Establishes convergence of the proximal Newton iteration for composite convex objectives, the scheme whose no-resampling convergence the paper relies on.","marker":"[42]"},{"why":"Provides the scaled proximal gradient framework and convergence results that justify the metric-proximal update (10)-(13).","marker":"[6]"},{"why":"The proximal MCMC counterpart; motivates the preconditioned update and the covariance choice (13) via Langevin diffusion.","marker":"[18]"},{"why":"Supplies the optimized PMC framework and the GLR resampling scheme used in PNAIS; also defines the banana-shaped high-dimensional test target.","marker":"[26]"},{"why":"Robust covariance adaptation used as the ablation baseline (PNAIS-rcov) to isolate the effect of the Hessian-based covariance rule.","marker":"[24]"},{"why":"Dual forward-backward algorithm used to compute the proximal step under a general metric when no closed form exists.","marker":"[15]"},{"why":"Backtracking procedure for the damped factor $\\theta$, used to keep the proximal Newton step stable for non-log-concave targets.","marker":"[7]"}],"fun_headline_variants":["Hessian-aware proposals tame nonsmooth targets in adaptive sampling","Proximal Newton boosts importance sampling for nonsmooth Bayesian targets","Newton scaling sharpens adaptive samplers on constrained, sparse targets","Adaptive sampler learns via proximal Newton updates for nonsmooth targets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the convergence and stability of the proximal Newton iteration, which the paper only cites for the no-resampling setting, carry over unchanged when the same update is embedded inside the stochastic resampling loop of the sampler; Section III-B explicitly states the convergence result 'without resampling' and offers no analysis of the resampled iteration.","fun_headline_variants_meta":{"raw":{"variants":["Hessian-aware proposals tame nonsmooth targets in adaptive sampling","Proximal Newton boosts importance sampling for nonsmooth Bayesian targets","Newton scaling sharpens adaptive samplers on constrained, sparse targets","Adaptive sampler learns via proximal Newton updates for nonsmooth targets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000522,"raw_usage":{"total_tokens":2509,"prompt_tokens":913,"completion_tokens":1596,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":529,"completion_tokens_details":{"reasoning_tokens":1524}},"tokens_in":529,"tokens_out":1596,"duration_ms":10552,"temperature":1.0,"reasoning_tokens":1524,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:27:56.163433+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a unimodal target of the form $\\exp(-f-g)$ with a unique minimizer, run PNAIS with GLR resampling, and compare the adapted means and estimator MSE against the same update with resampling disabled; if the resampled means drift away from the minimizer or the MSE fails to decrease while the no-resampling variant converges, the transfer assumption is violated.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes convergence of the proximal Newton iteration for composite convex objectives, the scheme whose no-resampling convergence the paper relies on."},{"cited_title":"Becker and J","cited_arxiv_id":null,"evidence_quote":"Provides the scaled proximal gradient framework and convergence results that justify the metric-proximal update (10)-(13)."},{"cited_title":"Corbineau, D","cited_arxiv_id":null,"evidence_quote":"The proximal MCMC counterpart; motivates the preconditioned update and the covariance choice (13) via Langevin diffusion."},{"cited_title":"Elvira and E","cited_arxiv_id":null,"evidence_quote":"Supplies the optimized PMC framework and the GLR resampling scheme used in PNAIS; also defines the banana-shaped high-dimensional test target."},{"cited_title":"El-Laham, V","cited_arxiv_id":null,"evidence_quote":"Robust covariance adaptation used as the ablation baseline (PNAIS-rcov) to isolate the effect of the Hessian-based covariance rule."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Dual forward-backward algorithm used to compute the proximal step under a general metric when no closed form exists."},{"cited_title":"Bonettini, I","cited_arxiv_id":null,"evidence_quote":"Backtracking procedure for the damped factor $\\theta$, used to keep the proximal Newton step stable for non-log-concave targets."}],"review_version":1}