{"id":"6f314d44-8134-424a-b325-0d3992e42990","arxiv_id":"2607.06252","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":3,"one_line_summary":"By folding normalization into a KL-based objective over un-normalized potentials, neural likelihood approximation becomes a strictly convex problem with provable consistency.","lead":"The paper shows that learning a likelihood function for Bayesian inverse problems can be made a strictly convex optimization problem by absorbing normalization into the training objective. This removes the need for restrictive normalized model classes like normalizing flows and comes with convergence guarantees.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"The consistency proof (Theorem 3.7) requires the true potential L to lie in a finite-dimensional C^1 class F with bounded derivatives (Assumption 3), but the paper's motivation centers on infinite-dimensional Hilbert-space inverse problems where this condition is not verified.","rationale":"The reader correctly identified the two key gaps: (1) the function-space convexity vs. parameter-space non-convexity gap (Remark 2.9), and (2) the finite-dimensional restriction of the consistency result vs. the infinite-dimensional motivation. I agree with both points. On reflection, the finite-dimensional restriction (Assumption 3) is the more structurally load-bearing concern because it limits the consistency theorem's scope, whereas the parameter-space non-convexity is a standard caveat in neural network optimization that affects all such methods. However, the authors are transparent about both limitations, the convexity theorem (Theorem 2.8) is correct and independently valuable, and the consistency proof is sound within its stated assumptions. The numerical experiments, while lacking error bars and multiple seeds, demonstrate the approach on non-trivial PDE-based problems. The CONDITIONAL verdict is appropriate: the theory is sound within its stated scope, but the gap between the infinite-dimensional motivation and finite-dimensional consistency, combined with unverified assumptions in the experiments, prevents full ACCEPT. No verdict change is needed.","tokens_in":21716,"tokens_out":810,"duration_ms":526763,"concrete_test":"Verify that the neural network classes used in Section 4's experiments satisfy Assumption 3: specifically, check whether the free-form network f_theta(x,y) with 4 fully-connected layers has bounded C^1 norm on R^{n+m} (including the weight-boundedness conditions needed for the Nickl-Pötscher bracketing result). If not, either constrain the architecture (e.g., spectral normalization on weights) and re-run, or explicitly state that the experiments are illustrative rather than covered by the theory.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim has two pillars: strict convexity of Phi in function space (Theorem 2.8) and consistency of empirical minimizers (Theorem 3.7). The convexity result is clean and the proof is correct — it follows from Hölder's inequality applied to the log-partition function, with the equality condition giving identifiability. The consistency result, however, depends on Assumption 3, which restricts to H=R^n and requires F to be a bounded subset of C^1(R^{n+m}, -2). The bracketing entropy argument (Theorem 3.9, from Nickl-Pötscher) then gives finite bracketing numbers, enabling the uniform law of large numbers in Theorem 3.6. The concern is not that the proof is wrong — it checks out — but that the paper's headline framing ('motivated by Bayesian inverse problems' on 'function spaces,' with 'infinite-dimensional Hilbert space H') creates a gap between what is promised and what is delivered. The consistency guarantee, which is the main theoretical contribution beyond convexity, only holds after discretization. The authors acknowledge this in the conclusion ('consistency result is established under finite-dimensional spaces... due to the scope of existing empirical process theory'), so this is transparent rather than hidden. However, the practical implications are not assessed: the numerical experiments use neural network parametrizations that are neither verified to satisfy the C^1 boundedness condition nor shown to find the global optimum in parameter space (Remark 2.9 acknowledges the latter). So the two conditions needed for the theory to bite — (a) F contains L and has finite bracketing entropy, (b) optimization reaches the function-space minimizer — are both unverified in the experiments. This does not undermine the convexity contribution, which stands independently, but it weakens the consistency claim's practical relevance.","agreement_with_reader":"agree"},"referee_report":{"model":"glm-5.2","summary":"This paper proposes a convex approximation framework for neural likelihood estimation in Bayesian inverse problems. The key idea is to work with un-normalized negative log-likelihood (NLL) potentials and fold normalization into the training objective, yielding a functional Φ(f) = E_λ[f(x;y)] + E_π[log Z_f(y)] that is strictly convex in f (Theorem 2.8). The authors establish consistency of empirical minimizers of a data-driven approximation Φ_N to the true potential L as the sample size grows (Theorem 3.7), under assumptions that include a sub-Gaussian prior, a convex closed function class F containing L with quadratic growth bounds, and finite-dimensional H = R^n with C^1 regularity for bracketing entropy arguments. Numerical experiments on a deblurring problem and a nonlinear PDE-based semiconductor doping inverse problem demonstrate the practical viability of the approach.","tokens_in":21934,"tokens_out":1359,"duration_ms":309988,"significance":"The paper makes a genuine theoretical contribution by identifying that the KL-based objective for likelihood approximation is convex in the un-normalized potential, a non-trivial observation that distinguishes the framework from normalizing-flow-based approaches where the normalized function class is not convex. The consistency result (Theorem 3.7) is a clean application of classical M-estimation theory (van der Vaart, Theorem 3.1) combined with bracketing entropy results from Nickl-Pötscher. The decomposition in Lemma 2.7 is verified and provides the foundation for the data-driven estimator. The numerical experiments are illustrative and include a practically relevant PDE-based inverse problem with significant computational speedups. The framework is transparent about its limitations, particularly the finite-dimensional restriction of the consistency proof.","major_comments":[{"comment":"§3.1, Assumption 3 and Theorem 3.7: The consistency guarantee requires H = R^n and F to be a bounded subset of C^1(R^{n+m}, -2), while the paper's motivation and formulation (§1.2, §2) emphasize infinite-dimensional Hilbert space H. The convexity results (Lemma 2.7, Theorem 2.8) are indeed stated in the infinite-dimensional setting, but the consistency result — which is the main theoretical contribution beyond convexity — only holds after discretization. The authors acknowledge this in §5, but the gap between the infinite-dimensional framing and the finite-dimensional theorem is load-bearing for the paper's claimed contribution to 'Bayesian inverse problems on function spaces.' The authors should either (a) clarify in the introduction and theorem statements that the consistency result is finite-dimensional and discuss what is needed for the infinite-dimensional extension, or (b) provide,","section":null},{"comment":"§4, Remark 2.9: The convexity of Φ is in function space, but the numerical experiments use neural network parametrizations where optimization in parameter space remains non-convex. The paper does not assess whether gradient-based training actually finds the global optimum (or a near-optimal point) in the parameter space for the experiments in §4. Since the practical utility of the convexity result depends on this, some discussion of optimization quality — e.g., comparing solutions from multiple random initializations, or reporting training loss variability — would strengthen the connection between theory and practice. This is acknowledged in Remark 2.9 but not addressed experimentally.","section":null},{"comment":"§4.2: The calibrated residual approximation fails to generate accurate posterior variance and means (L^1 bias of 1.072 vs. 0.1839 for free-form), but no diagnosis is offered. Since the calibrated residual is a natural variant of the proposed framework, understanding why it fails — whether due to optimization difficulty, insufficient expressivity, or the joint learning of forward map and noise covariance — would be valuable for practitioners. At minimum, the authors should state whether this failure is reproducible across initialization seeds.","section":null}],"minor_comments":[{"comment":"§2.1, Eq. (7): The lower bound involves 'for every ε > 0 there exists δ > 0' — the dependence of δ on ε is somewhat unusual and could be clarified with a brief remark on why this formulation is needed rather than a fixed pair (C_2^-, C_2^+).","section":null},{"comment":"§3.2: The M(N) schedule is left unspecified ('we leave the choice of an optimal function M open'). While acceptable for a consistency result, a brief discussion of practical choices used in the experiments (what M(N) was used?) would help reproducibility.","section":null},{"comment":"§4.1: The pCN step size β = 0.2 is stated without justification. A brief remark on acceptance rates or sensitivity to β would be helpful.","section":null},{"comment":"Table 1: The GP regression column lists 'GP(x,y)' as output but the text describes a squared exponential covariance. Clarify whether the GP regresses the NLL directly or the log-likelihood.","section":null},{"comment":"§4.2: Training details mention '30 epochs' but no learning rate, optimizer, or batch size is given. These should be specified for reproducibility.","section":null},{"comment":"Reference [11] (Helin & Schneider) is listed as 'in preparation, 2026' — this should be updated to a citable form if available by the time of revision.","section":null},{"comment":"§2.2, Definition 2.3: The notation F_Φ for the equivalence class is introduced but occasionally the paper writes F when F_Φ is meant (e.g., in Theorem 3.5, 'for any f ∈ F_Φ' vs. Theorem 3.6, 'sup_{f ∈ F}'). Consistency in notation would improve readability.","section":null}],"recommendation":"major_revision","confidential_remarks":"The core mathematical content is sound: the convexity proof (Theorem 2.8) via Hölder's inequality is correct and the consistency argument (Theorem 3.7) via van der Vaart's Theorem 3.1 with bracketing entropy from Nickl-Pötscher checks out. The main issue is the gap between the infinite-dimensional motivation and the finite-dimensional consistency theorem, which the authors are transparent about but which nonetheless affects the paper's positioning. I would encourage the authors to either tone down the infinite-dimensional framing in the introduction or make a serious attempt at the infinite-dimensional extension. The neural network optimization gap (Remark 2.9) is a known and honest limitation, but some experimental evidence that the non-convex parameter optimization is well-behaved would significantly strengthen the paper's practical claims."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a careful and constructive report. The referee correctly identifies the core contributions (convexity of the KL-based objective in the un-normalized potential, and the consistency result) and raises three substantive points: (1) the gap between infinite-dimensional framing and the finite-dimensional consistency theorem, (2) the lack of experimental evidence on optimization quality in neural network parameter space, and (3) the undiagnosed failure of the calibrated residual approximation in Section 4.2. We address each below and describe revisions we will make.","responses":[{"response":"The referee is correct that there is a genuine gap between the infinite-dimensional setting in which the convexity results (Lemma 2.7, Theorem 2.8) are formulated and the finite-dimensional setting required for the consistency theorem (Theorem 3.7). We do not dispute this. The root cause is that the bracketing entropy result we rely on (Theorem 3.9, from Nickl–Pötscher) is stated for bounded subsets of Hölder spaces on R^{n+m}, and extending this to function classes on infinite-dimensional domains is non-trivial and beyond what the current manuscript establishes. We will adopt option (a): we will revise the introduction (§1.2 and §1.3) and the theorem statements in §3 to state explicitly and prominently that the consistency result (Theorem 3.7) is finite-dimensional, while the convexity results hold in the infinite-dimensional setting. We will also add a dedicated discussion of what would be needed for the infinite-dimensional extension — specifically, an appropriate bracketing entropy theory for function classes on Hilbert-space-valued domains, which is an active area of research (e.g., Nickl's recent work on Bayesian non-linear statistical inverse problems). We agree with the referee that the current framing could mislead readers into thinking the consistency result applies directly to function spaces, and this will be corrected.","revision_made":"yes","referee_comment":"§3.1, Assumption 3 and Theorem 3.7: The consistency guarantee requires H = R^n and F to be a bounded subset of C^1(R^{n+m}, -2), while the paper's motivation and formulation (§1.2, §2) emphasize infinite-dimensional Hilbert space H. The gap between the infinite-dimensional framing and the finite-dimensional theorem is load-bearing for the paper's claimed contribution to 'Bayesian inverse problems on function spaces.' The authors should either (a) clarify in the introduction and theorem statements that the consistency result is finite-dimensional and discuss what is needed for the infinite-dimensional extension, or (b) provide the infinite-dimensional extension."},{"response":"The referee raises a fair point. Remark 2.9 acknowledges that convexity in function space does not imply convexity in parameter space, but the experiments do not provide evidence about whether gradient-based optimization reliably finds near-optimal solutions. We will address this by adding experiments comparing training loss and posterior quality across multiple random initializations for the deblurring problem (§4.1). Specifically, we will report the variability in the final training objective and in the resulting posterior L^1 bias across at least 5 independent random seeds. This will give practitioners a concrete sense of optimization stability. We note that the convexity result still provides value through the identifiability guarantee (uniqueness of the population minimizer in F_Φ), which underlies the consistency analysis, even if the parameter-space optimization landscape is non-convex. We will make this point more explicit in the revised Remark 2.9.","revision_made":"yes","referee_comment":"§4, Remark 2.9: The convexity of Φ is in function space, but the numerical experiments use neural network parametrizations where optimization in parameter space remains non-convex. The paper does not assess whether gradient-based training actually finds the global optimum (or a near-optimal point) in the parameter space for the experiments in §4. Some discussion of optimization quality — e.g., comparing solutions from multiple random initializations, or reporting training loss variability — would strengthen the connection between theory and practice."},{"response":"We agree that the failure of the calibrated residual approximation in §4.2 is insufficiently diagnosed. Based on our experience with the experiments, we believe the most likely cause is the joint learning of the forward map and the noise covariance: the additional diagonal layer estimating Γ introduces a non-identifiability between the forward map error and the noise level, which can destabilize training. However, we have not yet systematically verified this hypothesis or confirmed reproducibility across seeds. We will add experiments reporting the calibrated residual results across multiple initialization seeds to determine whether the failure is consistent. We will also add a diagnostic comparison: training the calibrated residual with the noise covariance fixed to the true value (which reduces it to the residual approximation) versus learned, to isolate whether the joint estimation of Γ is the source of the problem. If time constraints prevent completing the full diagnostic, we will at minimum state clearly that the failure is reproducible across seeds and offer our best assessment of the cause, while being transparent that a complete diagnosis is left for future work.","revision_made":"partial","referee_comment":"§4.2: The calibrated residual approximation fails to generate accurate posterior variance and means (L^1 bias of 1.072 vs. 0.1839 for free-form), but no diagnosis is offered. Understanding why it fails — whether due to optimization difficulty, insufficient expressivity, or the joint learning of forward map and noise covariance — would be valuable for practitioners. At minimum, the authors should state whether this failure is reproducible across initialization seeds."}],"tokens_in":21589,"tokens_out":1173,"duration_ms":141339,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"The main thing to know: this paper proves that the KL-based objective for neural likelihood approximation is strictly convex when you optimize over un-normalized potentials rather than constrained normalized densities (Theorem 2.8). The proof is a clean Hölder inequality argument with a sharp equality condition giving identifiability. This is a genuine structural improvement over normalizing-flow parametrizations, where the feasible set is not convex. The consistency result (Theorem 3.7) — empirical minimizers converge to the true potential — is also new for this setting and the proof via bracketing entropy (Nickl-Pötscher) and van der Vaart's consistency theorem checks out line by line. The KL decomposition in Lemma 2.7, which makes the objective evaluable from joint data without knowing the true likelihood, is known in the SBI literature but is correctly attributed and serves as the right foundation for the convexity argument. The numerical experiments on deblurring and a semiconductor doping PDE problem are reasonable demonstrations: the free-form approximation matches the true posterior well, and the 200x speedup over finite-element solves is credible. Credit is due for formulating the problem in a Hilbert space setting for the convexity theory and being transparent about where the consistency result stops (finite dimensions only). The authors acknowledge both limitations — the function-space vs. parameter-space gap (Remark 2.9) and the finite-dimensional restriction — in the conclusion rather than hiding them. Now the soft spots. The gap between convexity in f and non-convexity in neural network parameters θ is the real issue. The convexity result gives identifiability of the minimizer in function space, but the experiments use neural network parametrizations that are never verified to satisfy the C^1 boundedness conditions of Assumption 3, and there is no check that gradient-based training reaches the function-space optimum. The stress-test concern about finite-dimensional vs. infinite-dimensional framing is valid but somewhat overstated: the authors are upfront about this, and the convexity result (the stronger contribution) does hold in the Hilbert space setting. The consistency result is the weaker pillar and its practical relevance is limited by both the finite-dimensional restriction and the unverified optimization convergence. Minor: no error bars or multiple seeds in the experiments, no code shipped. These are addressable. Who is this for? Researchers in simulation-based inference and Bayesian inverse problems who care about theoretical foundations. The convexity contribution stands on its own and is worth engaging with. The consistency result is a first step that others can build on. This deserves a serious referee. The main things to push on in review: (1) can the authors verify that their neural network class satisfies the bracketing entropy conditions, or at least discuss what goes wrong, and (2) can they add multiple seeds and error bars to the experiments. The finite-dimensional limitation is acceptable as a stated scope restriction.","headline":"Convexity of KL objective for un-normalized neural likelihood approximation is clean and new; consistency proof is correct but limited to finite dimensions and unverified in experiments.","tokens_in":22749,"tokens_out":666,"would_cite":true,"duration_ms":99165,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G05","65C05","65N21","65N75","60B12"],"pacs":[],"model":"glm-5.2","headline":"Folding normalization into the loss makes likelihood learning convex","keywords":["neural likelihood approximation","Bayesian inverse problems","convex optimization","KL divergence","un-normalized models","consistency","Monte Carlo estimation","posterior sampling"],"falsifier":"If gradient-based training of neural network surrogates in this framework systematically converges to different local optima in function space (not just parameter space) for different random initializations, the practical value of the convexity guarantee would be undermined.","tokens_in":21763,"feed_emoji":"","tokens_out":1091,"duration_ms":118463,"temperature":0.7,"pith_summary":"The paper addresses a core bottleneck in neural likelihood approximation for Bayesian inverse problems: standard approaches require the learned likelihood to be a properly normalized probability distribution, which forces the use of restrictive parametric families (like normalizing flows) that may not contain the true likelihood. The authors propose a different formulation: optimize directly over un-normalized negative log-likelihood functions, and absorb the normalization constant into the training objective itself. They show that the resulting objective—expected negative log-likelihood plus a log-normalizer term—is strictly convex in the un-normalized potential function. This means that, at the level of the function being learned (rather than the neural network parameters representing it), the optimization landscape has a unique global minimum. They further prove that when the objective is estimated from finite data using a nested Monte Carlo scheme, the minimizer converges in probability to the true likelihood as the sample size grows. The key identity enabling all of this is that the expected KL-divergence between the true posterior and the approximate posterior decomposes into a constant plus two terms—E[f(x,y)] and E[log Z_f(y)]—both computable from samples of the joint distribution without knowing the true likelihood.","feed_headline":"Folding normalization into the loss makes likelihood learning convex","feed_subtitle":"By optimizing over un-normalized potentials and absorbing the normalizer into the objective, the authors prove strict convexity and data驱ven","key_machinery":"Strict convexity via Hölder inequality applied to log-partition function; nested Monte Carlo estimation of normalizer; bracketing entropy bounds for Hölder-class functions","core_discovery":"The central object is the objective function Phi(f) = E_lambda[f(x;y)] + E_pi[log Z_f(y)], where f is an un-normalized negative log-likelihood and Z_f(y) is its normalizing integral. The paper proves this objective is strictly convex in f (Theorem 2.8), with the proof turning on a single application of Hölder's inequality to the log-normalizer term. This convexity holds in the function-space formulation: the set of normalized densities (exp(-f) integrating to one) is not convex, but the set of un-normalized potentials is, and the normalization is handled implicitly by the objective. The consistency result (Theorem 3.7) then follows by verifying two classical conditions—well-separated minimum","pith_inferences":["The gap between convexity in function space and non-convexity in parameter space suggests that architectural choices ensuring the parameter-to-function map is well-behaved could make the practical optimization problem more tractable, though this is not addressed in the paper.","The consistency proof is restricted to finite-dimensional H = R^n due to the reliance on classical bracketing entropy results for Hölder spaces on Euclidean domains; extending to infinite-dimensional Hilbert spaces would require new empirical process theory for function classes on abstract spaces.","The nested Monte Carlo structure of the estimator means the choice of inner sample size M(N) as a function of outer sample size N affects convergence; the paper proves convergence but does not optimize this trade-off, which could matter for practical efficiency.","The convexity result depends on the function class F being convex and closed; whether commonly used neural network architectures induce function classes that are approximately convex in the relevant sense is an empirical question left open."],"forward_implications":["Convexity in function space provides an identifiability guarantee: even if neural network training finds different parameter settings, they must represent the same optimal likelihood function, which constrains the solution space.","The free-form approximation (learning f directly without assuming a forward map or noise model) can recover posteriors accurately even when the observational noise level is unknown, as demonstrated in the deblurring experiment.","The semiconductor doping profile experiment shows that neural likelihood surrogates can be approximately 200x faster than PDE-based forward model evaluations on CPU, enabling MCMC-based posterior sampling in minutes rather than days.","The framework extends naturally to residual and calibrated residual approximations where partial knowledge of the forward model or noise structure is available, with the free-form variant performing best when noise is unknown."],"fun_headline_variants":["Un-normalized potentials make neural likelihood learning convex","Convex neural likelihood approximation via un-normalized potentials","Folding the normalizer into the loss yields convex likelihood learning","Working with un-normalized potentials makes likelihood training convex","Neural likelihood learning is convex over un-normalized potentials"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The convexity guarantee holds in function space, but when the un-normalized potential is represented by a neural network, the optimization problem in network parameters remains non-convex. The practical benefit of the convexity result depends on whether gradient-based training can actually find the global optimum in parameter space, which the convexity result alone does not ensure.","fun_headline_variants_meta":{"raw":{"variants":["Un-normalized potentials make neural likelihood learning convex","Convex neural likelihood approximation via un-normalized potentials","Folding the normalizer into the loss yields convex likelihood learning","Working with un-normalized potentials makes likelihood training convex","Neural likelihood learning is convex over un-normalized potentials","Absorbing normalization into the objective gives convex likelihood estimation","Strict convexity for neural likelihood approximation through un-normalized potentials","Un-normalized potentials give convex objectives for Bayesian inverse problems"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":837,"prompt_tokens":565,"completion_tokens":272,"prompt_tokens_details":null},"tokens_in":565,"tokens_out":272,"duration_ms":12799,"temperature":1.0,"reasoning_tokens":210,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-08T11:52:16.045368+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If gradient-based training of neural network surrogates in this framework systematically converges to different local optima in function space (not just parameter space) for different random initializations, the practical value of the convexity guarantee would be undermined.","supporting_citations":[],"review_version":1}