REVIEW 3 major objections 5 minor 30 references
Fisher Score Matching for Simulation-Based Forecasting and Inference
T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read The paper proposes Fisher score matching, a training objective that lets a neural network recover the gradient of the log-likelihood from simulator samples alone, and shows the learned score drives Fisher forecasts and Hamiltonian Monte…
desk verdict Clean and correct core trick, modestly novel; the headline claim about non-differentiable simulators is not actually tested. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing identity is $\nabla_\theta \log P(x|\theta) = \mathbb{E}_{P(\theta^*|x,\theta)}[\nabla_\theta \log P(\theta^*|\theta)]$, derived from the Markov chain $\theta \to \theta^* \to x$; it converts the Fisher score into a Bayes least squares regression target, which is why the simple mean squared error loss of Equation (3) suffices. The second mechanism is the auxiliary-noise construction $\theta = \tilde\theta + w$ for indecomposable simulators, which replaces the original model by an extended one whose score is a convolution of the true score with $P(\theta|\tilde\theta,x)$; the method then relies on that kernel being sharply peaked so the smoothed score approximates the true one.
What would settle it
Take a non-differentiable simulator with a low-dimensional parameter space, estimate the true log-likelihood gradient by finite differences from a large number of simulations at fiducial points, train the Fisher score matching network with the paper's noise scale choice, and check whether the difference between the learned and finite-difference scores shrinks as the noise scale is reduced; if it does not shrink, the peakedness assumption fails.
Extended reading notes
Core claim
Under a latent-variable model with Markov chain $\theta \to \theta^* \to x$, the Fisher score satisfies $\nabla_\theta \log P(x|\theta) = \mathbb{E}_{P(\theta^*|x,\theta)}[\nabla_\theta \log P(\theta^*|\theta)]$, so the intractable likelihood gradient becomes the posterior mean of a tractable latent score. Since the minimizer of mean squared error over simulated triplets $(x,\theta,\theta^*)$ is exactly that posterior mean, training $s(x,\theta)$ to predict $\nabla_\theta \log P(\theta^*|\theta)$ makes $s$ converge to the true Fisher score, with no derivative of the simulator required. For models without a natural latent, the paper adds auxiliary parameter noise $\theta = \tilde\theta + w$ and learns the score of the extended model $\nabla_{\tilde\theta} \log P(x|\tilde\theta)$, which equals the true score convolved with the sharp kernel $P(\theta|\tilde\theta,x)$ and is accurate when that kernel is concentrated. The experiments show the learned score field recovers the analytic score in the linear Gaussian case and the autodifferentiated score in the weak lensing case, and that the score can be used to estimate Fisher matrices and to run Hamiltonian Monte Carlo posterior sampling.
Load-bearing premise
The method's accuracy for a non-decomposable simulator rests on the added parameter noise being so small that the artificially blurred model still has essentially the same score as the original simulator, and when the simulator is not differentiable there is no direct way to verify that equivalence.
Editorial extensions
If this is right
- A trained score estimator yields the Fisher information matrix at any parameter value as the covariance of the score over data samples, enabling forecasts without an analytic likelihood.
- The learned score provides a likelihood-free gradient for Hamiltonian Monte Carlo, so posterior sampling can use gradient-based exploration on simulators that are not differentiable.
- Maximum-likelihood-style point estimates become available by finding where the learned score vanishes, with no need to specify a prior during score training.
- The training objective is prior-independent: the same score network can be reused across different analysis choices of prior, since only the simulator and latent model enter the loss.
- For decomposable models, any valid latent decomposition trains the network to the same true Fisher score, so the choice of decomposition does not bias the result.
Reading between the lines
- A practical certification recipe suggested by the method: for a new non-differentiable simulator, train with a few noise scales and compare the resulting Fisher matrices against finite-difference estimates on a low-dimensional parameter subspace to detect when the peakedness assumption breaks.
- The same regression framework could be extended to learn the score as a function of the noise scale, letting one check convergence by seeing whether the learned score stabilizes as $\sigma \to 0$.
- Because the method only needs forward samples, it could be combined with active learning that concentrates simulation budget around the fiducial parameters, improving score accuracy exactly in the region forecasts and inference use.
- If the score can be learned at field level for high-dimensional summaries, it would connect naturally to field-level Fisher analyses and simulation-based inference pipelines.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This manuscript proposes Fisher Score Matching (FSM), a simulation-based method to train a neural network to approximate the Fisher score s(x, θ) = ∇θ log p(x | θ). The central identity (Eq. 2) represents the score as a posterior expectation of a latent score ∇θ log p(θ* | θ), which can be learned by regression (Eq. 3). For models without a natural latent variable, the authors introduce an auxiliary variable θ~ with a noise model p(θ | θ~), and show that the learned estimator approximates the score of the extended model, which is a smoothed version of the true score (Eq. 5). The method is validated on a two-dimensional linear Gaussian model and on a weak lensing angular power spectrum example using jax-cosmo, with comparisons to analytical and autodiff ground truth for HMC posterior sampling and Fisher matrix estimation.
Significance. The paper addresses a useful problem: obtaining Fisher scores and gradient-based Bayesian inference from simulators that are not differentiable. The derivation in Eq. (2) is correct, the least-squares regression argument is standard, and the authors provide code for reproducible experiments. The visual agreement with analytical and autodiff ground truth in both the toy and weak lensing examples is encouraging. However, the practical significance rests on controlling the bias introduced by auxiliary noise in non-decomposable models; the current evidence does not yet establish the headline claim for genuinely non-differentiable simulators.
major comments (3)
- [Section 2.2, Appendix B, Eq. (5)] The auxiliary-noise procedure trains the network to estimate ∇θ~ log p(x | θ~), the score of the extended model, not the original score ∇θ log p(x | θ). As Appendix B shows, this is a convolution of the true score with P(θ | θ~, x), and the approximation is accurate only when that kernel is sharply peaked. The paper states this but provides no quantitative bound or diagnostic in terms of σ and the likelihood curvature. The weak lensing validation in §3.2 uses jax-cosmo, a differentiable simulator, with σ = 1e-3 fixed, and all comparisons are made against autodiff ground truth; no experiment uses a genuinely non-differentiable simulator. For a Gaussian location model, the extended Fisher information is (Σ + σ²I)⁻¹, so forecasts are generically biased low. To support the abstract's claim that the method extends to non-differentiable simulators, the authors should either demonstrate the method on a non-differentiable forward model or provide an explicit, testable bound or diagnostic for the smoothing bias.
- [Section 3.2, Figures 2, 3, and 4] The validation is qualitative: the learned score fields, HMC posteriors, and Fisher matrix contours are compared visually, with no error bars, no quantitative score errors, and no numerical comparison of posterior or Fisher matrix elements. The paper itself reports in §3.2 that score magnitudes are underestimated by roughly 40% for implausible parameters, yet the downstream claims that the learned scores are 'suitable' for forecasting and inference rely on visual contour agreement. The authors should report metrics such as relative L2 error of the learned score in the high-likelihood region, parameter bias and coverage of the HMC posterior relative to the true-score posterior, and relative errors in Fisher matrix elements or eigenvalue ratios.
- [Section 4 and Section 3.2] The claim that the Fisher score approximation is independent of the chosen training proposal is only true for the exact minimizer of the loss in Eq. (3). In finite-sample practice, the paper acknowledges that the training set contains few examples for parameters that are implausible given the data, leading to factor-of-two errors in score magnitude. Because HMC trajectories traverse regions of moderate posterior probability, the statement that this 'will not affect' inference needs quantitative support, not just the observation that the final posterior contours look similar. A quantitative comparison of the posterior moments or credible intervals would strengthen this load-bearing point.
minor comments (5)
- [Footnote 1] There is a typo: 'workshop' should be 'workshop'.
- [Algorithm 2] The output line writes s(x, θ~) ≈ ∇θ log p(x | θ~); the subscript on the gradient should be θ~ rather than θ.
- [Section 2.2, Eq. (4)] The condition 'both covariance matrices are positive definite' should be stated as Σθ* and Σn − Σθ* being positive definite; the latter is an additional constraint on the decomposition.
- [Figure 1] The axis labels in the figure are partially garbled in the manuscript rendering; please check the final PDF.
- [References] The JAX-COSMO reference entry formats the arXiv identifier as a journal field; please standardize the bibliography style.
Circularity Check
No circularity: the estimator's minimizer is derived from an exact identity and validated against an independent autodiff ground truth; the auxiliary-noise caveat is an approximation gap, not a circular step.
full rationale
The derivation chain is self-contained and no fitted value is renamed as a prediction. Equation (2) proves the exact identity ∇θ log P(x|θ) = E_{P(θ*|x,θ)}[∇θ log P(θ*|θ)], and Appendix A shows that minimizing the MSE loss of Eq. (3) yields exactly that conditional expectation; hence the network target is the desired Fisher score by a theorem, not by an equivalent definition imported from the authors' prior work. The toy-model validation compares the learned field against the analytic score of the fixed full Gaussian likelihood, which is independent of the chosen latent decomposition. The weak-lensing validation compares the learned extended-model score against an autodiff ground truth from the external differentiable library jax-cosmo (Eq. 7), so the benchmark is not constructed from the network's fitted values. Appendix B explicitly states the one substantive caveat: for indecomposable models Eq. (5) shows the method estimates the smoothed score ∇θ~ log P(x|θ~), which equals the original true score only when P(θ|θ~,x) is sharply peaked. That is a stated approximation about the target model and an unvalidated claim for genuinely non-differentiable simulators, but it is a correctness/completeness concern rather than circularity. Self-citations such as Alsing & Wandelt (2018) are contextual and not load-bearing for the method's internal validity.
Assumptions & free parameters
free parameters (1)
- Auxiliary noise scale sigma =
1e-3
assumptions (3)
- domain assumption Markov chain theta -> theta* -> x holds
- domain assumption Auxiliary noise model theta = theta~ + w makes the learned score approximate the true score when w is small
- domain assumption Gaussian likelihood with fixed covariance Sigma_fid for the weak lensing example
invented entities (1)
-
Auxiliary latent parameter theta~
Cite this review
Pith. "Pith review of Fisher Score Matching for Simulation-Based Forecasting and Inference." pith.science (2026). https://pith.science/paper/PBP6CVNK
@misc{pith2026250707833,
author = {Pith},
title = {Pith review of: Fisher Score Matching for Simulation-Based Forecasting and Inference},
year = {2026},
howpublished = {\url{https://pith.science/paper/PBP6CVNK}},
note = {Machine review of arXiv:2507.07833}
}
read the original abstract
We propose a method for estimating the Fisher score--the gradient of the log-likelihood with respect to model parameters--using score matching. By introducing a latent parameter model, we show that the Fisher score can be learned by training a neural network to predict latent scores via a mean squared error loss. We validate our approach on a toy linear Gaussian model and a cosmological example using a differentiable simulator. In both cases, the learned scores closely match ground truth for plausible data-parameter pairs. This method extends the ability to perform Fisher forecasts, and gradient-based Bayesian inference to simulation models, even when they are not differentiable; it therefore has broad potential for advancing cosmological analyses.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
-
[2]
Alsing , J. and Wandelt , B. Generalized massive optimal data compression . , 476 0 (1): 0 L60--L64, May 2018. doi:10.1093/mnrasl/sly029
-
[3]
Fast likelihood-free cosmology with neural density estimators and active learning
Alsing , J., Charnock , T., Feeney , S., and Wandelt , B. Fast likelihood-free cosmology with neural density estimators and active learning . , 488 0 (3): 0 4440--4458, September 2019. doi:10.1093/mnras/stz1960
-
[4]
Beyond-2pt Collaboration , : , Krause , E., Kobayashi , Y., Salcedo , A. N., Ivanov , M. M., Abel , T., Akitsu , K., Angulo , R. E., Cabass , G., Contarini , S., Cuesta-Lazaro , C., Hahn , C., Hamaus , N., Jeong , D., Modi , C., Nguyen , N.-M., Nishimichi , T., Paillas , E., Pellejero Iba \ n ez , M., Philcox , O. H. E., Pisani , A., Schmidt , F., Tanaka ...
-
[5]
K., Herold, M., Ziaeemehr, A., Pals, M., Gruner, T., Bischoff, S., Krouglova, N., Gao, R., Lappalainen, J
Boelts, J., Deistler, M., Gloeckler, M., Álvaro Tejero-Cantero, Lueckmann, J.-M., Moss, G., Steinbach, P., Moreau, T., Muratore, F., Linhart, J., Durkan, C., Vetter, J., Miller, B. K., Herold, M., Ziaeemehr, A., Pals, M., Gruner, T., Bischoff, S., Krouglova, N., Gao, R., Lappalainen, J. K., Mucsányi, B., Pei, F., Schulz, A., Stefanidi, Z., Rodrigues, P., ...
2025
-
[6]
J., Leary, C., Maclaurin, D., Necula, G., Paszke, A., Vander P las, J., Wanderman- M ilne, S., and Zhang, Q
Bradbury, J., Frostig, R., Hawkins, P., Johnson, M. J., Leary, C., Maclaurin, D., Necula, G., Paszke, A., Vander P las, J., Wanderman- M ilne, S., and Zhang, Q. JAX : composable transformations of P ython+ N um P y programs, 2018. URL http://github.com/jax-ml/jax
2018
-
[7]
Mining gold from implicit models to improve likelihood-free inference
Brehmer, J., Louppe, G., Pavez, J., and Cranmer, K. Mining gold from implicit models to improve likelihood-free inference. Proceedings of the National Academy of Sciences, 117 0 (10): 0 5242–5249, February 2020. ISSN 1091-6490. doi:10.1073/pnas.1915980117. URL http://dx.doi.org/10.1073/pnas.1915980117
-
[8]
Brinckmann, T. and Lesgourgues, J. MontePython 3: boosted MCMC sampler and other features . 2018
work page 2018
Show all 30 references
-
[9]
JAX-COSMO: An End-to-End Differentiable and GPU Accelerated Cosmology Library
Campagne , J.-E., Lanusse , F., Zuntz , J., Boucaud , A., Casas , S., Karamanis , M., Kirkby , D., Lanzieri , D., Peel , A., and Li , Y. JAX-COSMO: An End-to-End Differentiable and GPU Accelerated Cosmology Library . The Open Journal of Astrophysics, 6: 0 15, April 2023. doi:1...
2023 arXiv
-
[10]
Charnock , T., Lavaux , G., and Wandelt , B. D. Automatic physical inference with information maximizing neural networks . , 97 0 (8): 0 083004, April 2018. doi:10.1103/PhysRevD.97.083004
2018 doi
-
[11]
A new approach to observational cosmology using the scattering transform
Cheng , S., Ting , Y.-S., M \'e nard , B., and Bruna , J. A new approach to observational cosmology using the scattering transform . , 499 0 (4): 0 5902--5914, December 2020. doi:10.1093/mnras/staa3165
2020 doi
-
[12]
The frontier of simulation-based inference
Cranmer , K., Brehmer , J., and Louppe , G. The frontier of simulation-based inference . Proceedings of the National Academy of Science, 117 0 (48): 0 30055--30062, December 2020. doi:10.1073/pnas.1912789117
2020 doi
-
[13]
and Mishra-Sharma , S
Cuesta-Lazaro , C. and Mishra-Sharma , S. Point cloud approach to generative modeling for galaxy surveys at the field level . , 109 0 (12): 0 123531, June 2024. doi:10.1103/PhysRevD.109.123531
2024 doi
- [14]
-
[15]
a nen , E., Kermiche , S., Kiessling , A., Kilbinger , M., Kubik , B., Kuijken , K., K \
Euclid Collaboration , Congedo , G., Miller , L., Taylor , A. N., Cross , N., Duncan , C. A. J., Kitching , T., Martinet , N., Matthew , S., Schrabback , T., Tewes , M., Welikala , N., Aghanim , N., Amara , A., Andreon , S., Auricchio , N., Baldi , M., Bardelli , S., Bender , ...
2024
-
[16]
F., Sellentin , E., de Mijolla , D., and Vianello , A
Heavens , A. F., Sellentin , E., de Mijolla , D., and Vianello , A. Massive data compression for parameter-dependent covariance matrices . , 472 0 (4): 0 4244--4250, December 2017. doi:10.1093/mnras/stx2326
2017 doi
-
[17]
J., Chartier , N., Cuesta-Lazaro , C., Ding , S., Lapel , A., Lemos , P., Lovell , C
Ho , M., Bartlett , D. J., Chartier , N., Cuesta-Lazaro , C., Ding , S., Lapel , A., Lemos , P., Lovell , C. C., Makinen , T. L., Modi , C., Pandya , V., Pandey , S., Perez , L. A., Wandelt , B., and Bryan , G. L. LtU-ILI: An All-in-One Framework for Implicit Inference in Astr...
2024 doi
-
[18]
and Seljak , U
Jain , B. and Seljak , U. Cosmological Model Predictions for Weak Lensing: Linear and Nonlinear Regimes . , 484 0 (2): 0 560--573, July 1997. doi:10.1086/304372
1997 doi
-
[19]
H., Hahn , C., Ho , S., Eickenberg , M., Hou , J., Massara , E., Modi , C., Moradinezhad Dizgah , A., R \'e galdo-Saint Blancard , B., and Spergel , D
Lemos , P., Parker , L. H., Hahn , C., Ho , S., Eickenberg , M., Hou , J., Massara , E., Modi , C., Moradinezhad Dizgah , A., R \'e galdo-Saint Blancard , B., and Spergel , D. SimBIG: Field-level Simulation-based Inference of Large-scale Structure . In Machine Learning for Ast...
-
[20]
pmwd: A Differentiable Cosmological Particle-Mesh N -body Library
Li , Y., Lu , L., Modi , C., Jamieson , D., Zhang , Y., Feng , Y., Zhou , W., Pok Kwan , N., Lanusse , F., and Greengard , L. pmwd: A Differentiable Cosmological Particle-Mesh N -body Library . arXiv e-prints, art. arXiv:2211.09958, November 2022. doi:10.48550/arXiv.2211.09958
-
[21]
L., Sui , C., Wandelt , B
Makinen , T. L., Sui , C., Wandelt , B. D., Porqueres , N., and Heavens , A. Hybrid Summary Statistics . arXiv e-prints, art. arXiv:2410.07548, October 2024. doi:10.48550/arXiv.2410.07548
2024 doi
-
[22]
Mudur , N., Cuesta-Lazaro , C., and Finkbeiner , D. P. Diffusion-HMC: Parameter Inference with Diffusion-model-driven Hamiltonian Monte Carlo . , 978 0 (1): 0 64, January 2025. doi:10.3847/1538-4357/ad8bc3
2025 doi
-
[23]
J., Barreiro , R
Planck Collaboration , Aghanim , N., Akrami , Y., Ashdown , M., Aumont , J., Baccigalupi , C., Ballardini , M., Banday , A. J., Barreiro , R. B., Bartolo , N., Basak , S., Battye , R., Benabed , K., Bernard , J. P., Bersanelli , M., Bielewicz , P., Bock , J. J., Bond , J. R., ...
2018
-
[24]
Sequential Neural Score Estimation: Likelihood-Free Inference with Conditional Score Based Diffusion Models
Sharrock , L., Simons , J., Liu , S., and Beaumont , M. Sequential Neural Score Estimation: Likelihood-Free Inference with Conditional Score Based Diffusion Models . arXiv e-prints, art. arXiv:2210.04872, October 2022. doi:10.48550/arXiv.2210.04872
- [25]
-
[26]
P., Kumar , A., Ermon , S., and Poole , B
Song , Y., Sohl-Dickstein , J., Kingma , D. P., Kumar , A., Ermon , S., and Poole , B. Score-Based Generative Modeling through Stochastic Differential Equations . arXiv e-prints, art. arXiv:2011.13456, November 2020. doi:10.48550/arXiv.2011.13456
-
[27]
J., Battye , R
Square Kilometre Array Cosmology Science Working Group , Bacon , D. J., Battye , R. A., Bull , P., Camera , S., Ferreira , P. G., Harrison , I., Parkinson , D., Pourtsidou , A., Santos , M. G., Wolz , L., Abdalla , F., Akrami , Y., Alonso , D., Andrianomena , S., Ballardini , ...
2018
-
[28]
N., and Heavens , A
Tegmark , M., Taylor , A. N., and Heavens , A. F. Karhunen-Lo \`e ve Eigenvalue Problems in Cosmology: How Should We Tackle Large Data Sets? , 480 0 (1): 0 22--35, May 1997. doi:10.1086/303939
1997 doi
-
[29]
and Lewis , A
Torrado , J. and Lewis , A. Cobaya: code for Bayesian analysis of hierarchical physical models . , 2021 0 (5): 0 057, May 2021. doi:10.1088/1475-7516/2021/05/057
2021 doi
-
[30]
Zhao , X., Mao , Y., Cheng , C., and Wandelt , B. D. Simulation-based Inference of Reionization Parameters from 3D Tomographic 21 cm Light-cone Images . , 926 0 (2): 0 151, February 2022. doi:10.3847/1538-4357/ac457d
2022 doi
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.