{"id":"4a954845-b5f6-4d7b-bbde-f3f268b6e432","arxiv_id":"2507.07833","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A neural network trained to predict latent parameter scores can approximate the Fisher score and support Fisher forecasting and gradient-based Bayesian inference from simulations.","lead":"This paper trains a neural network to learn the Fisher score, the gradient of the likelihood with respect to model parameters, from simulations alone. The authors show the learned scores reproduce analytic results in a toy model and a two-parameter weak lensing example, enabling Fisher forecasts and Hamiltonian Monte Carlo sampling.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Auxiliary-noise score is only validated on a differentiable simulator; the smoothing bias in Eq. 5 is unquantified for black-box simulators, so the headline extension to non-differentiable models is not yet supported.","rationale":"The reader's weakest-assumption analysis identifies the same load-bearing concern: the auxiliary-noise construction changes the model, and the paper validates only on a differentiable simulator. My stress-test read confirms this is the correct point of maximum risk. The core derivation of Eq. 2 is internally consistent, and the regression-loss minimizer does recover the conditional mean, so I do not see a mathematical error in the method itself. The weakness is in the scope of evidence: the abstract's promise about non-differentiable simulators is exactly the regime never exercised, and the smoothing bias from Eq. 5 is not characterized beyond a single fixed σ. The proposed fix is concrete and feasible because the benchmark is low-dimensional and ground-truth likelihoods are available even for black-box simulators. This does not require rejecting the paper; it supports the reader's CONDITIONAL verdict and motivates an added experiment or explicit limitation. I therefore recommend UNCHANGED relative to the reader's verdict.","tokens_in":12026,"tokens_out":6426,"duration_ms":72238,"concrete_test":"Run FSM on a deliberately non-differentiable 2-parameter benchmark simulator with a known analytic likelihood, e.g., a Gaussian likelihood with known Fisher matrix or a standard SBI benchmark implemented through an opaque compiled code path. Train with auxiliary noise σ scaled to the parameter range, testing σ in {1e-4, 1e-3, 1e-2, 1e-1} times that range. For a fixed observation x_obs, compare the learned score to a brute-force finite-difference score computed from simulator evaluations at θ~ ± h on a dense grid, reporting median and maximum relative error and the implied bias in the Fisher matrix. If the error grows with σ as Eq. 5 predicts, or if the σ needed for sub-percent error makes the target (θ - θ~)/σ² too large to train stably, the claim that FSM extends to non-differentiable simulators requires qualification.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.2 and Algorithm 2 introduce an auxiliary noise model θ = θ~ + w and train s(x, θ~) against ∇θ~ log P(θ | θ~). By the identity in Eq. 2, the network therefore estimates ∇θ~ log P(x | θ~), the score of the extended model, not the original score. Appendix B (Eq. 5) shows this equals ∫ ∇θ log P(x | θ) P(θ | θ~, x) dθ, i.e., a convolution of the true score with the conditional posterior P(θ | θ~, x). The approximation is accurate only when that kernel is narrow relative to the curvature scale of the likelihood. The paper fixes σ = 1e-3 in the weak-lensing experiment and validates against jax-cosmo, a differentiable simulator with autodiff ground truth. No experiment uses a genuinely non-differentiable simulator, so the central promise of the abstract remains untested. Furthermore, decreasing σ to reduce bias makes the regression target (θ - θ~)/σ² large (order 1/σ), which can amplify network approximation error exactly in the region where the score is needed. In a Gaussian location model, the extended Fisher information is (Σ + σ²I)⁻¹, so forecasts are generically biased low; the paper does not quantify this contraction for the weak-lensing example. This is an addressable but real gap: the method may still work for many simulators, but the current evidence does not establish it.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript proposes Fisher Score Matching (FSM), a simulation-based method to train a neural network to approximate the Fisher score s(x, θ) = ∇θ log p(x | θ). The central identity (Eq. 2) represents the score as a posterior expectation of a latent score ∇θ log p(θ* | θ), which can be learned by regression (Eq. 3). For models without a natural latent variable, the authors introduce an auxiliary variable θ~ with a noise model p(θ | θ~), and show that the learned estimator approximates the score of the extended model, which is a smoothed version of the true score (Eq. 5). The method is validated on a two-dimensional linear Gaussian model and on a weak lensing angular power spectrum example using jax-cosmo, with comparisons to analytical and autodiff ground truth for HMC posterior sampling and Fisher matrix estimation.","tokens_in":12312,"tokens_out":4783,"duration_ms":55144,"significance":"The paper addresses a useful problem: obtaining Fisher scores and gradient-based Bayesian inference from simulators that are not differentiable. The derivation in Eq. (2) is correct, the least-squares regression argument is standard, and the authors provide code for reproducible experiments. The visual agreement with analytical and autodiff ground truth in both the toy and weak lensing examples is encouraging. However, the practical significance rests on controlling the bias introduced by auxiliary noise in non-decomposable models; the current evidence does not yet establish the headline claim for genuinely non-differentiable simulators.","major_comments":[{"comment":"The auxiliary-noise procedure trains the network to estimate ∇θ~ log p(x | θ~), the score of the extended model, not the original score ∇θ log p(x | θ). As Appendix B shows, this is a convolution of the true score with P(θ | θ~, x), and the approximation is accurate only when that kernel is sharply peaked. The paper states this but provides no quantitative bound or diagnostic in terms of σ and the likelihood curvature. The weak lensing validation in §3.2 uses jax-cosmo, a differentiable simulator, with σ = 1e-3 fixed, and all comparisons are made against autodiff ground truth; no experiment uses a genuinely non-differentiable simulator. For a Gaussian location model, the extended Fisher information is (Σ + σ²I)⁻¹, so forecasts are generically biased low. To support the abstract's claim that the method extends to non-differentiable simulators, the authors should either demonstrate the method on a non-differentiable forward model or provide an explicit, testable bound or diagnostic for the smoothing bias.","section":"Section 2.2, Appendix B, Eq. (5)"},{"comment":"The validation is qualitative: the learned score fields, HMC posteriors, and Fisher matrix contours are compared visually, with no error bars, no quantitative score errors, and no numerical comparison of posterior or Fisher matrix elements. The paper itself reports in §3.2 that score magnitudes are underestimated by roughly 40% for implausible parameters, yet the downstream claims that the learned scores are 'suitable' for forecasting and inference rely on visual contour agreement. The authors should report metrics such as relative L2 error of the learned score in the high-likelihood region, parameter bias and coverage of the HMC posterior relative to the true-score posterior, and relative errors in Fisher matrix elements or eigenvalue ratios.","section":"Section 3.2, Figures 2, 3, and 4"},{"comment":"The claim that the Fisher score approximation is independent of the chosen training proposal is only true for the exact minimizer of the loss in Eq. (3). In finite-sample practice, the paper acknowledges that the training set contains few examples for parameters that are implausible given the data, leading to factor-of-two errors in score magnitude. Because HMC trajectories traverse regions of moderate posterior probability, the statement that this 'will not affect' inference needs quantitative support, not just the observation that the final posterior contours look similar. A quantitative comparison of the posterior moments or credible intervals would strengthen this load-bearing point.","section":"Section 4 and Section 3.2"}],"minor_comments":[{"comment":"There is a typo: 'workshop' should be 'workshop'.","section":"Footnote 1"},{"comment":"The output line writes s(x, θ~) ≈ ∇θ log p(x | θ~); the subscript on the gradient should be θ~ rather than θ.","section":"Algorithm 2"},{"comment":"The condition 'both covariance matrices are positive definite' should be stated as Σθ* and Σn − Σθ* being positive definite; the latter is an additional constraint on the decomposition.","section":"Section 2.2, Eq. (4)"},{"comment":"The axis labels in the figure are partially garbled in the manuscript rendering; please check the final PDF.","section":"Figure 1"},{"comment":"The JAX-COSMO reference entry formats the arXiv identifier as a journal field; please standardize the bibliography style.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The core idea is simple, correct, and potentially useful, but the manuscript's central promise of handling non-differentiable simulators is not yet supported by the experiments. The required changes are feasible within the scope of the paper: add a non-differentiable validation or a quantitative bias diagnostic, and report numerical agreement metrics. No concerns about novelty or citation practices beyond the minor formatting issues noted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core identity in Eq. 2 is right, and the regression loss in Eq. 3 is a sensible way to learn the Fisher score. But this specific identity and MSE regression already appear in Brehmer et al. 2020, so the novelty is narrower than the abstract suggests. What is new is the explicit framing for Fisher forecasting and HMC, plus the auxiliary-noise trick for non-decomposable simulators. That framing is useful and the paper is honest about the main limitation: Appendix B shows the auxiliary-noise estimator targets a smoothed version of the true score, Eq. 5, and the approximation depends on the noise being sharply peaked. The paper does not hide this, but it also does not quantify the bias it introduces.\n\nThe experiments are clean as proofs of concept. The toy linear Gaussian matches the analytic score, and the weak-lensing example shows the learned score, posterior, and Fisher matrix agree visually with autodiff ground truth. Code is available. That is real evidence the method works when the simulator is differentiable and the latent-noise model is well-behaved.\n\nThe soft spots are proportionate. First, no experiment uses a genuinely non-differentiable simulator, so the central promise of the abstract is untested. The weak-lensing example uses jax-cosmo, which is differentiable; it validates the regression, not the auxiliary-noise trick in the regime it was designed for. Second, the smoothing bias is unquantified. The stress-test note is correct that in a Gaussian location model the extended Fisher information is (Sigma + sigma^2 I)^-1, so forecasts are biased low, and the paper does not measure this contraction for the weak-lensing case. Third, the results are qualitative: no error bars, no quantitative score errors. The reported 40% underestimation for implausible parameters is hand-waved as irrelevant to high-likelihood regions, which is plausible but not demonstrated.\n\nThese are addressable gaps, not fatal flaws. The method may well work for many non-differentiable simulators, but the current evidence does not establish it. The paper would benefit from a test on a real black-box simulator and a characterization of the bias as a function of sigma.\n\nWho is this for? People doing SBI or Fisher forecasting with expensive simulators will find the framing useful. It deserves a serious referee, but the referee should push for a non-differentiable validation and quantitative error metrics.","headline":"Clean and correct core trick, modestly novel; the headline claim about non-differentiable simulators is not actually tested.","tokens_in":12835,"tokens_out":1939,"would_cite":true,"duration_ms":24328,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes Fisher score matching, a training objective that lets a neural network recover the gradient of the log-likelihood from simulator samples alone, and shows the learned score drives Fisher forecasts and Hamiltonian Monte…","keywords":["Fisher score","score matching","simulation-based inference","likelihood-free inference","Hamiltonian Monte Carlo","Fisher information matrix","cosmological parameter estimation","latent variable model"],"falsifier":"Take a non-differentiable simulator with a low-dimensional parameter space, estimate the true log-likelihood gradient by finite differences from a large number of simulations at fiducial points, train the Fisher score matching network with the paper's noise scale choice, and check whether the difference between the learned and finite-difference scores shrinks as the noise scale is reduced; if it does not shrink, the peakedness assumption fails.","tokens_in":11830,"feed_emoji":"🎯","tokens_out":7239,"duration_ms":71495,"temperature":0.7,"pith_summary":"Simulation-based models usually hide their likelihood, so the gradient of the log-likelihood with respect to parameters—the Fisher score—is out of reach unless the simulator is differentiable. This paper shows that the Fisher score is a conditional expectation of a latent-model score, which means an ordinary regression network trained on simulated data can learn it. If correct, this gives a purely simulation-driven route to Fisher information matrices, maximum likelihood estimation, and gradient-based Bayesian sampling for non-differentiable simulators. The paper validates the idea on a linear Gaussian model and a two-parameter weak lensing example, where learned scores closely match analytically or autodifferentiably computed ground truth for plausible data-parameter pairs.","feed_headline":"Fisher score learned from simulations powers forecasting and inference","feed_subtitle":"Even non-differentiable simulators can yield score fields for Fisher matrices, maximum likelihood, and Hamiltonian Monte Carlo sampling.","key_machinery":"The load-bearing identity is $\\nabla_\\theta \\log P(x|\\theta) = \\mathbb{E}_{P(\\theta^*|x,\\theta)}[\\nabla_\\theta \\log P(\\theta^*|\\theta)]$, derived from the Markov chain $\\theta \\to \\theta^* \\to x$; it converts the Fisher score into a Bayes least squares regression target, which is why the simple mean squared error loss of Equation (3) suffices. The second mechanism is the auxiliary-noise construction $\\theta = \\tilde\\theta + w$ for indecomposable simulators, which replaces the original model by an extended one whose score is a convolution of the true score with $P(\\theta|\\tilde\\theta,x)$; the method then relies on that kernel being sharply peaked so the smoothed score approximates the true one.","core_discovery":"Under a latent-variable model with Markov chain $\\theta \\to \\theta^* \\to x$, the Fisher score satisfies $\\nabla_\\theta \\log P(x|\\theta) = \\mathbb{E}_{P(\\theta^*|x,\\theta)}[\\nabla_\\theta \\log P(\\theta^*|\\theta)]$, so the intractable likelihood gradient becomes the posterior mean of a tractable latent score. Since the minimizer of mean squared error over simulated triplets $(x,\\theta,\\theta^*)$ is exactly that posterior mean, training $s(x,\\theta)$ to predict $\\nabla_\\theta \\log P(\\theta^*|\\theta)$ makes $s$ converge to the true Fisher score, with no derivative of the simulator required. For models without a natural latent, the paper adds auxiliary parameter noise $\\theta = \\tilde\\theta + w$ and learns the score of the extended model $\\nabla_{\\tilde\\theta} \\log P(x|\\tilde\\theta)$, which equals the true score convolved with the sharp kernel $P(\\theta|\\tilde\\theta,x)$ and is accurate when that kernel is concentrated. The experiments show the learned score field recovers the analytic score in the linear Gaussian case and the autodifferentiated score in the weak lensing case, and that the score can be used to estimate Fisher matrices and to run Hamiltonian Monte Carlo posterior sampling.","pith_inferences":["A practical certification recipe suggested by the method: for a new non-differentiable simulator, train with a few noise scales and compare the resulting Fisher matrices against finite-difference estimates on a low-dimensional parameter subspace to detect when the peakedness assumption breaks.","The same regression framework could be extended to learn the score as a function of the noise scale, letting one check convergence by seeing whether the learned score stabilizes as $\\sigma \\to 0$.","Because the method only needs forward samples, it could be combined with active learning that concentrates simulation budget around the fiducial parameters, improving score accuracy exactly in the region forecasts and inference use.","If the score can be learned at field level for high-dimensional summaries, it would connect naturally to field-level Fisher analyses and simulation-based inference pipelines."],"forward_implications":["A trained score estimator yields the Fisher information matrix at any parameter value as the covariance of the score over data samples, enabling forecasts without an analytic likelihood.","The learned score provides a likelihood-free gradient for Hamiltonian Monte Carlo, so posterior sampling can use gradient-based exploration on simulators that are not differentiable.","Maximum-likelihood-style point estimates become available by finding where the learned score vanishes, with no need to specify a prior during score training.","The training objective is prior-independent: the same score network can be reused across different analysis choices of prior, since only the simulator and latent model enter the loss.","For decomposable models, any valid latent decomposition trains the network to the same true Fisher score, so the choice of decomposition does not bias the result."],"supporting_citations":[{"why":"Supplies the latent-variable decomposition idea that the paper extends; the paper notes the same fact is exploited there.","marker":"(Brehmer et al., 2020)"},{"why":"Provides the differentiable cosmological simulation code whose autodifferentiated gradients are the ground truth for the weak lensing validation.","marker":"(Campagne et al., 2023)"},{"why":"Establishes the score-matching training paradigm of regressing a network on a conditional score.","marker":"(Song & Ermon, 2019)"},{"why":"Defines the Fisher-information-matrix forecasting usage in cosmology that the learned score is intended to support.","marker":"(Tegmark et al., 1997)"},{"why":"Shows the Fisher score at a fiducial parameter is a locally sufficient statistic, justifying its use for data compression and forecasting.","marker":"(Heavens et al., 2017)"},{"why":"Extends the locally sufficient statistic result and motivates score-based optimal compression, a downstream use of the learned score.","marker":"(Alsing & Wandelt, 2018)"}],"fun_headline_variants":["Simulation-trained Fisher scores unlock forecasting and inference","Score matching learns Fisher scores from non-differentiable simulators","Fisher scores from simulations enable gradient-free Bayesian inference","Learn true Fisher scores from simulated data, no gradients needed","Neural score matching enables Fisher forecasting for any simulator"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's accuracy for a non-decomposable simulator rests on the added parameter noise being so small that the artificially blurred model still has essentially the same score as the original simulator, and when the simulator is not differentiable there is no direct way to verify that equivalence.","fun_headline_variants_meta":{"raw":{"variants":["Simulation-trained Fisher scores unlock forecasting and inference","Score matching learns Fisher scores from non-differentiable simulators","Fisher scores from simulations enable gradient-free Bayesian inference","Learn true Fisher scores from simulated data, no gradients needed","Neural score matching enables Fisher forecasting for any simulator"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000724,"raw_usage":{"total_tokens":3228,"prompt_tokens":909,"completion_tokens":2319,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":525,"completion_tokens_details":{"reasoning_tokens":2242}},"tokens_in":525,"tokens_out":2319,"duration_ms":19042,"temperature":1.0,"reasoning_tokens":2242,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:31:03.454694+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a non-differentiable simulator with a low-dimensional parameter space, estimate the true log-likelihood gradient by finite differences from a large number of simulations at fiducial points, train the Fisher score matching network with the paper's noise scale choice, and check whether the difference between the learned and finite-difference scores shrinks as the noise scale is reduced; if it does not shrink, the peakedness assumption fails.","supporting_citations":[],"review_version":1}