{"id":"f4c0b78f-7e94-4e03-af23-4fed741c5d93","arxiv_id":"2411.17667","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A neural-network posterior can be written as a log-concave mixture when the parameter count is large, making sampling tractable, with separate risk guarantees of N^{-1/4} and N^{-1/3} for a discrete-prior variant.","lead":"This paper introduces a representation of a Bayesian posterior for single-layer neural networks that splits it into log-concave pieces, so that sampling the posterior reduces to sampling from easier distributions. It also proves statistical risk bounds for a grid-based version, but the two results are not yet merged into a single practical algorithm.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Log-concavity alone does not imply rapid sampling: the marginal score requires a nested inner MCMC whose bias and mixing time are unanalyzed, so Theorem 1 does not by itself establish the advertised polynomial-time algorithm.","rationale":"The reader's weakest_assumption identified the unanalyzed black-box log-concave sampler, and I agree that this is the main unsecured step. My stress-test sharpens this to a specific sub-concern: the score of the marginal density pi_b(z) is itself an expectation under the reverse conditional pi_b(w|z), so the advertised algorithm requires a nested MCMC whose bias and mixing time are never analyzed. Even if each individual log-concave density can be sampled efficiently by existing algorithms, the outer process with a noisy, biased score does not automatically converge to pi_b(z). The paper also does not verify the hypotheses needed for the cited samplers: the reverse conditional is only weakly log-concave, and no log-Sobolev constant, warm-start distribution, or condition-number bound is given. The risk bounds in Sec. 6 apply to a different (discrete) prior, and the paper honestly defers the connection to future work; this is a real gap but secondary to the sampling claim because the abstract's computational promise rests on Theorem 1. An experimental check on a small instance can reveal whether the nested-score scheme is numerically feasible, while a full resolution would require an analytical bias-variance bound. I therefore keep the reader's conditional verdict unchanged: the mathematical results are likely correct, but the advertised rapid computation is not yet established.","tokens_in":83937,"tokens_out":6461,"duration_ms":66823,"concrete_test":"Run the proposed two-level sampler on a small synthetic problem (e.g., K=2, d=10, N=20, beta chosen so beta*N >= 2 and K*d >= A3*(beta*N)^2, with tanh activation and the l1-uniform prior). Compute the exact posterior mean by deterministic grid integration over (B_1^d)^K. Then run outer MALA on pi_b(z), where the score is estimated by an inner MCMC (e.g., 100, 10^3, 10^4 iterations per score call), and record the total number of inner oracle calls needed to bring the posterior-mean error below 0.01. If this total exceeds a low-degree polynomial in Kd and 1/epsilon (e.g., grows like 10^6 or worse even at these tiny sizes), the unanalyzed nested sampling step is the bottleneck; if it stays modest, the concern is weakened, though a proof would still be missing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central computational claim—that the posterior can be sampled by drawing z from pi_b(z) and then w from pi_b(w|z), with both densities log-concave—is not backed by a polynomial-time sampling guarantee. Theorem 1 (Sec. 4.1) proves log-concavity of pi_b(w|z) and pi_b(z), but log-concavity alone does not yield a mixing-time bound. The reverse conditional is only weakly log-concave on (B_1^d)^K, flat in directions orthogonal to the rows of X (as noted in Sec. 7). The marginal's score is an expectation under that same reverse conditional (Remark 2, Sec. 5.3), so every outer-step gradient evaluation requires an inner MCMC run. The paper gives no bias/variance or ergodicity analysis for this nested scheme. Moreover, the conditions required by the cited samplers (e.g., [20,38])—strong log-concavity, known log-Sobolev constant, warm start, or a bound on the condition number in isotropic position—are not verified. Without such verification, the claim of 'rapid' computation (abstract, Sec. 1) is an assumption, not a theorem. This gap is acknowledged in Sec. 8 ('The further details of sampling must be worked out'), but it is load-bearing: the abstract's promise that 'neuron parameters can be sampled from the posterior by only sampling log-concave densities' is the bridge between the representation theorem and the advertised algorithm.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies posterior sampling and risk guarantees for single-hidden-layer neural networks with ℓ1-bounded internal weights and fixed outer weights. The main structural result (Theorem 1) shows that, under a continuous uniform prior on (B_1^d)^K and the condition Kd ≥ C(βN)^2, the posterior π_b(w) can be written as ∫ π_b(w|z)π_b(z) dz, where the reverse conditional π_b(w|z) and the induced marginal π_b(z) are log-concave. The main statistical results (Theorems 2–4) are risk bounds for a discrete uniform prior on a grid: arbitrary-sequence squared regret of order O((log d)/N)^{1/4}, iid mean-squared risk of the same order, and expected Kullback divergence of order O((log d)/N)^{1/3} under Gaussian errors with β=1/σ². The paper claims that these ingredients yield rapid Bayesian computation via sampling from log-concave densities, with the sampling algorithm treated as a black box.","tokens_in":95,"tokens_out":6096,"duration_ms":113226,"significance":"The log-concave coupling construction is a meaningful structural insight: it identifies an auxiliary-variable decomposition that removes the multimodality obstruction for a class of neural-network posteriors, and it is proved with a detailed, largely self-contained argument that carefully tracks constants. The discrete-prior risk bounds are also valuable and are obtained via Barron's index of resolvability together with a new grid-discretization approximation lemma. However, the advertised computational claim is not established at the theorem level. The paper proves log-concavity, not rapid mixing, and the nested score-estimation scheme is left unanalyzed. The rigorous contributions are the representation theorem and the risk bounds; the step from these to a polynomial-time Bayesian training algorithm is conditional on unverified sampling assumptions, as the paper itself acknowledges in Section 8.","major_comments":[{"comment":"The central computational claim is not proven. Theorem 1 establishes only that π_b(w|z) and π_b(z) are log-concave; it does not provide any mixing-time, convergence-rate, or bias/variance analysis for the two-level MCMC scheme that the paper proposes. As Remark 2 notes, the score of the marginal π_b(z) is itself an expectation under the reverse conditional π_b(w|z), so every gradient evaluation in an outer MCMC step requires an inner MCMC run. The paper gives no ergodicity or finite-sample error analysis for this nested procedure, and Section 8 explicitly states that 'the further details of sampling must be worked out.' Thus the abstract's statement that neuron parameters 'can be sampled from the posterior by only sampling log-concave densities' is an algorithmic premise, not a theorem, and it is load-bearing for the title and for the claimed polynomial-time training algorithm.","section":"§4.1, Theorem 1; §5.3, Remark 2; §8"},{"comment":"The reverse conditional π_b(w|z) is only weakly log-concave: Section 7 notes that it is flat in directions orthogonal to the rows of the data matrix X. The paper cites Ball Walk and Hit-and-Run results [38,33] for weakly log-concave targets, but it does not verify their hypotheses for this family of densities—e.g., any bound on the condition number, a warm-start distribution, or a log-Sobolev inequality. Moreover, the regime Kd ≥ A_3(βN)^2 with βN ≥ 2 typically forces d to be much larger than N, which makes the flat directions high-dimensional. Without an iteration-complexity bound that accounts for these flat directions, the claimed reduction to log-concave sampling does not yield a polynomial-time algorithm, even granting access to a generic log-concave sampler.","section":"§7, Theorem 1"},{"comment":"The paper's two main results apply to different priors and are not connected. The log-concave coupling of Theorem 1 is for the continuous uniform prior on (B_1^d)^K, while the risk bounds in Theorems 2–4 are for the discrete uniform prior on a grid. The paper acknowledges in Section 8 that bridging these results is future work. Consequently, the paper does not provide an end-to-end procedure that is simultaneously provably polynomial-time and provably risk-controlled. The contributions should be presented as separate representation and risk results, or the missing bridge should be supplied before the 'rapid Bayesian computation' claim is made.","section":"§4.2, §6, §8"}],"minor_comments":[{"comment":"The statement 'we adopt a sampling problem we can solve instead of an optimization problem we cannot' is premature given that the sampling problem is not yet proved to be solvable in polynomial time.","section":"§1"},{"comment":"Theorem 1 and condition (4.6): the restriction parameter δ should be explicitly defined with its allowed range in the theorem statement, since it is a threshold that also appears in the definition of C_z in (5.18).","section":"§4.1"},{"comment":"The paper would benefit from a pseudocode description of the proposed two-level sampler (outer MCMC on z, inner MCMC for the score), so that the required verifications are concrete.","section":"§5.3 and §8"},{"comment":"There are numerous typographical artifacts in the full-text version (e.g., equation (5.18) and the surrounding display), and a careful proofreading pass is needed.","section":"Full text"},{"comment":"Remark 3 handles rounding of the optimal ℓ* in the proof of Theorem 7 heuristically; a formal treatment with explicit rounding terms would strengthen the proof.","section":"§5.4, Remark 3"}],"recommendation":"major_revision","confidential_remarks":"This is a mathematically substantial paper, but the gap between the log-concavity representation and the advertised rapid sampling is serious. I would encourage the editor to consider it only after the authors either (i) supply mixing-time bounds for the two-level scheme, or (ii) substantially reframe the title and abstract to present the log-concavity representation and risk bounds as conditional/structural results. The paper also leans on the authors' previous ISIT/workshop papers; the novelty relative to those should be clarified."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe real content here is a representation theorem, not a sampling algorithm. For a single-hidden-layer net with continuous uniform prior on ℓ1 balls, McDonald and Barron prove the posterior can be written as a mixture π(w)=∫π(w|z)π(z)dz where both the reverse conditional and the induced marginal are log-concave, provided Kd ≥ C(βN)^2 and a log condition on K. That is new, and the proof machinery—the constrained auxiliary set, the Hölder-based covariance control, the near-constancy of the normalizing term—looks sound. The risk bounds for the discrete grid prior (fourth-root squared error, cube-root KL under Gaussian data) are not groundbreaking rates but are carefully derived, with explicit constants and a clean index-of-resolvability argument.\n\nThe soft spot is exactly where the abstract points: 'rapid' sampling. Theorem 1 gives log-concavity, but log-concavity does not by itself give a polynomial-time sampler. The reported algorithm samples z from π(z), whose score is itself an expectation under π(w|z) (Remark 2), so every gradient step needs an inner MCMC run. No bias, variance, or mixing-time analysis is given for this nested scheme. The reverse conditional is only weakly log-concave—flat in directions orthogonal to the data rows—and the paper does not verify that the cited samplers' conditions (strong log-concavity, known log-Sobolev constants, warm starts) hold here. The authors acknowledge this in Section 8 ('the further details of sampling must be worked out'), and the abstract's 'can be sampled by only sampling log-concave densities' is true only in the weak sense that the target densities are log-concave; it is an assumption, not a theorem, that fast sampling follows.\n\nA second, related gap: the log-concavity result uses the continuous uniform prior; the risk bounds use a discrete grid prior. The paper explicitly says bridging them is future work. So the advertised 'polynomial time Bayesian training with risk control' is not delivered by this paper.\n\nProportionately: the representation theorem is a genuine step, the proofs are detailed and likely correct, and the literature is handled honestly (self-citations are to prior results that are the basis of the construction). This deserves a serious referee. I would not desk-reject; I would send it out with the expectation of major revision—mainly a reframed abstract that does not overclaim the computational payoff, and ideally some discussion of how the nested MCMC bias might be controlled.\n\nRecommendation: accept the paper's core mathematical content as a contribution to Bayesian computation, but require the authors to separate clearly the representation theorem from the algorithmic promise.","headline":"A solid, honest representation theorem for neural-net posteriors, but the advertised 'rapid sampling' remains an unproven assumption; deserves peer review with a reframing.","tokens_in":84818,"tokens_out":2345,"would_cite":false,"duration_ms":23275,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F15","62M45","65C05"],"pacs":[],"model":"deepseek-v4-flash","headline":"A log-concave coupling rewrites a multimodal neural-net posterior as a mixture of samplable densities, and a discrete-prior version carries fourth-root risk bounds.","keywords":["Neural Networks","Bayesian Methods","Sampling","Statistical Learning","log-concave coupling","Markov Chain Monte Carlo","risk bounds"],"falsifier":"Take a small instance satisfying Theorem 1's conditions, such as $K=2$ and $d$ large enough that $Kd \\ge A_3(\\beta N)^2$, and evaluate the Hessian of $\\log \\pi_b(z)$ at several points $z$ in the constrained set using the identity $\\nabla^2 \\log \\pi_b(z) = \\kappa(-I + \\kappa \\, \\mathrm{Cov}_{\\pi_b(w|z)}[Xw])$; if any eigenvalue is positive for a valid $z$, the marginal is not log-concave and Theorem 1 fails.","tokens_in":83762,"feed_emoji":"🧠","tokens_out":8455,"duration_ms":70191,"temperature":0.7,"pith_summary":"The paper claims that the posterior distribution over the interior weights of a single-hidden-layer neural network can be rewritten as a mixture whose components are all log-concave, provided the network is wide enough in a specific sense. Because log-concave densities can be sampled in polynomial time, this turns Bayesian training into a sequence of log-concave sampling problems. The paper also proves risk bounds for a discretized uniform prior: squared-error generalization error decays like $(\\log d / N)^{1/4}$, and, under Gaussian data with matched gain, expected Kullback divergence decays like $(\\log d / N)^{1/3}$. The payoff would be a Bayesian training procedure with both polynomial-time sampling and statistical risk control.","feed_headline":"Two-level coupling makes neural-net posteriors samplable","feed_subtitle":"Bayesian training reduces to log-concave sampling, with fourth-root statistical risk bounds.","key_machinery":"The central object is the log-concave coupling: a joint distribution $\\pi_b(w,z) = \\pi_b(w)\\, \\pi_b(z|w)$ that preserves the original posterior $\\pi_b(w)$ as its marginal while making both the reverse conditional $\\pi_b(w|z)$ and the induced marginal $\\pi_b(z)$ log-concave. The forward coupling is a Gaussian $z_{i,k} \\sim \\mathrm{Normal}(w_k \\cdot x_i, 1/\\kappa)$, truncated to a set that bounds the bilinear term in the joint log-likelihood, with $\\kappa$ scaled as $\\sqrt{3/2}\\, E_2 \\, \\beta \\, C_N \\, V / K$, large enough to force the Hessian of $\\log \\pi_b(w|z)$ negative definite. Log-concavity of the marginal is shown through the Hessian identity $\\nabla^2 \\log \\pi_b(z) = \\kappa\\bigl(-I + \\kappa\\, \\mathrm{Cov}_{\\pi_b(w|z)}[Xw]\\bigr)$, so it suffices to bound the conditional covariance; the proof uses a Hölder inequality with high-order moments of the uniform prior and a cumulant generating function bound on the tilted prior.","core_discovery":"Theorem 1 establishes a log-concave coupling: for a continuous uniform prior on $(B_1^d)^K$, with $K \\ge 2$, $d \\ge 2$, $\\beta N \\ge 2$, and under $K \\log(2Kd/\\delta) \\le \\beta N$ and $Kd \\ge A_3(\\beta N)^2$, the posterior $\\pi_b(w)$ equals $\\int \\pi_b(w|z) \\pi_b(z) \\, dz$, where each reverse conditional $\\pi_b(w|z)$ is log-concave and the induced marginal $\\pi_b(z)$ is log-concave (strictly, if the inequality is strict). The auxiliary variable $z$ comes from a Gaussian forward coupling $z_{i,k} \\sim \\mathrm{Normal}(w_k \\cdot x_i, 1/\\kappa)$ restricted to a high-probability set, with $\\kappa$ chosen just large enough to cancel the potentially positive second-derivative terms in the Hessian of the log posterior. The score of the marginal is itself an expectation under the reverse conditional, so both levels are accessible to MCMC. Separately, for the discrete grid prior, the paper proves regret and risk bounds of order $(\\log d / N)^{1/4}$, and a Kullback risk bound of order $(\\log d / N)^{1/3}$ under Gaussian data with $\\beta = 1/\\sigma^2$; the risk control is for the discrete prior while the log-concavity construction uses the continuous prior, a gap the paper explicitly leaves for future work.","pith_inferences":["A testable strengthening would be to verify numerically the conjecture in Section 8 that the prior covariance dominates the conditional covariance up to a constant; if true, the condition $Kd \\ge C(\\beta N)^2$ could drop to $Kd \\ge C(\\beta N)$, improving the scaling of the method.","The dimension-inflation trick suggests an extension where the same coupling works for the original low-dimensional inputs after lifting; one could test whether the inflated sampler's mixing time degrades gracefully with the inflation factor, which the paper does not analyze.","The paper's risk bounds hold for a discrete grid prior while the coupling uses the continuous prior; the coupled discrete/continuous construction sketched in Section 8, if carried through, would yield a single polynomial-time algorithm with the stated risk guarantees.","Because the score of the marginal is itself a conditional expectation, the two-level MCMC scheme resembles a stochastic gradient sampler; one could try replacing the inner MCMC with an unbiased estimator of the conditional mean to obtain a single-loop algorithm, though the paper does not analyze this."],"forward_implications":["If the log-concave coupling holds, posterior means and predictive densities for the continuous-uniform-prior model can be computed by sampling only log-concave densities, so Bayesian training becomes a polynomial-time procedure whenever log-concave samplers are available.","The risk theorems imply that the Cesàro posterior mean estimates any function in the variational class to squared error $O((\\log d)/N)^{1/4}$ with no assumptions on the error distribution beyond finite variance.","Under independent Gaussian noise with $\\beta = 1/\\sigma^2$, the predictive density converges in expected Kullback divergence at rate $O((\\log d)/N)^{1/3}$, a faster rate than the fourth-root squared-error bound.","The parameter choices balancing the bounds require $K = O(N^{1/4})$ and $d = \\Omega(N^{5/4})$ in the natural scaling; the paper notes that repeating input coordinates can artificially inflate $d$ and satisfy the width condition without changing the function class.","Adaptive priors over a finite grid of $K$, $d$, $\\beta$, and $V$ values can be handled by running independent samplers in parallel and weighing by posterior probabilities, so the user need not know the right hyperparameters in advance."],"supporting_citations":[{"why":"Supplies the $\\ell_1$-controlled approximation bound for ReLU and squared-ReLU ridge functions that defines the variation class and underpins the risk proofs.","marker":"[30]"},{"why":"Supplies the index-of-resolvability method that converts prior mass on good approximators into regret and risk bounds.","marker":"[7]"},{"why":"Provides polynomial-mixing guarantees for weakly log-concave densities on convex sets, used to justify treating the reverse conditional as samplable.","marker":"[38]"},{"why":"Supplies the polynomial-time mixing result for strongly log-concave densities used for the marginal sampler.","marker":"[20]"},{"why":"Supplies the measure-decomposition viewpoint for posterior sampling from multimodal distributions that the paper generalizes to neural-net posteriors.","marker":"[40]"},{"why":"Supplies the Brascamp-Lieb inequality invoked to control variance terms in the marginal-Hessian calculations.","marker":"[13]"},{"why":"Gives the best-known optimization-based risk bound $O((\\log d/N)^{1/2})$ that the Bayesian risk bounds are compared against.","marker":"[10]"}],"fun_headline_variants":["Log-concave coupling makes neural-net posteriors samplable","Rapid Bayesian nets via log-concave posterior coupling","Auxiliary variables turn posterior MCMC into log-concave sampling","Bayesian neural nets: log-concave mixing with risk control"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that one can actually sample from the two log-concave densities in the coupling—the reverse conditional $\\pi_b(w|z)$ and the marginal $\\pi_b(z)$, whose score is itself an expectation under $\\pi_b(w|z)$—in a low-polynomial number of MCMC iterations; the paper treats this as a black box and provides no mixing-time or finite-sample error analysis.","fun_headline_variants_meta":{"raw":{"variants":["Log-concave coupling makes neural-net posteriors samplable","Rapid Bayesian nets via log-concave posterior coupling","Auxiliary variables turn posterior MCMC into log-concave sampling","Bayesian neural nets: log-concave mixing with risk control"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000209,"raw_usage":{"total_tokens":1564,"prompt_tokens":1261,"completion_tokens":303,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":877,"completion_tokens_details":{"reasoning_tokens":241}},"tokens_in":877,"tokens_out":303,"duration_ms":3242,"temperature":1.0,"reasoning_tokens":241,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:51:40.947895+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a small instance satisfying Theorem 1's conditions, such as $K=2$ and $d$ large enough that $Kd \\ge A_3(\\beta N)^2$, and evaluate the Hessian of $\\log \\pi_b(z)$ at several points $z$ in the constrained set using the identity $\\nabla^2 \\log \\pi_b(z) = \\kappa(-I + \\kappa \\, \\mathrm{Cov}_{\\pi_b(w|z)}[Xw])$; if any eigenvalue is positive for a valid $z$, the marginal is not log-concave and Theorem 1 fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the $\\ell_1$-controlled approximation bound for ReLU and squared-ReLU ridge functions that defines the variation class and underpins the risk proofs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the index-of-resolvability method that converts prior mass on good approximators into regret and risk bounds."},{"cited_title":"Lovász and S","cited_arxiv_id":null,"evidence_quote":"Provides polynomial-mixing guarantees for weakly log-concave densities on convex sets, used to justify treating the reverse conditional as samplable."},{"cited_title":"Dwivedi, Y","cited_arxiv_id":null,"evidence_quote":"Supplies the polynomial-time mixing result for strongly log-concave densities used for the marginal sampler."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Brascamp-Lieb inequality invoked to control variance terms in the marginal-Hessian calculations."},{"cited_title":"Complexity, Statistical Risk, and Metric Entropy of Deep Nets Using Total Path Variation","cited_arxiv_id":"1902.00800","evidence_quote":"Gives the best-known optimization-based risk bound $O((\\log d/N)^{1/2})$ that the Bayesian risk bounds are compared against."}],"review_version":1}