Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

Learned Bayesian Cram\'er-Rao Bound for Unknown Measurement Models Using Score Neural Networks

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that the Bayesian Cramér-Rao bound can be learned entirely from measurement-parameter pairs by learning score functions, and proves that, under realizability assumptions, the learned bound converges almost surely to the…

desk verdict Useful new method for learned Bayesian CRBs, with a real gap between the theory and the implemented objective. read the letter →

arxiv 2502.00724 v2 pith:76BA67K5 submitted 2025-02-02 eess.SP cs.AIcs.LGstat.ML

Hai Victor Habi , Hagit Messer , Yoram Bresler This is my paper · ORCID
classification eess.SPcs.AIcs.LGstat.ML
keywords BayesianCramér-RaoboundscorematchingFisherphysics-encodedneuralnetworkparameterestimationinformationlearnedboundsquantizedmeasurements
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The Bayesian Cramér-Rao bound (BCRB) says how accurately any estimator can perform, but it requires full knowledge of the prior and measurement distributions, which many real problems do not provide. This paper proposes a Learned BCRB (LBCRB) computed from a dataset of parameter-measurement pairs by learning score functions (gradients of log-densities) instead of the densities themselves. Two routes are given: the Posterior Approach learns the score of the posterior, while the Measurement-Prior Approach learns the prior score and the Fisher score separately and combines them through the standard decomposition of the Bayesian Fisher information. The paper introduces Fisher Score Matching, a variant of score matching for gradients with respect to the conditioning variable, and a Physics-encoded score neural network that builds known physical structure into the learned score. The result is a data-only bound with finite-sample error bounds, almost-sure convergence to the true BCRB, and demonstrated use on problems where the BCRB was previously unavailable.

What carries the argument

The engine is Fisher Score Matching (FSM), a new score-matching objective (equation 26) that targets the Fisher score, the gradient of the log-likelihood with respect to the conditioning parameter rather than the data. Classical and conditional score matching cannot do this; FSM adds a cross term involving the true prior score that cancels the intractable part and makes the objective depend only on samples, once the prior score has been learned. The Measurement-Prior Approach then uses the exact decomposition of the Bayesian Fisher information into the number of i.i.d. measurements times the single-sample measurement Fisher information plus the prior Fisher information. When the measurement distribution is known to depend on the parameter through a physical model, the Physics-encoded score neural network restricts the Fisher score model to the chain-rule form involving that known model, which reduces network complexity and sample complexity.

What would settle it

In the linear-Gaussian example of Section VI-A with known matrices A and Σ, train the Posterior and Measurement-Prior LBCRBs with increasing dataset size ND and compare the learned Bayesian Fisher information matrix with the analytic expression; if the relative error does not approach zero or does not follow the predicted square-root decay as ND grows, the consistency and finite-sample claims are contradicted.

Watch

Extended reading notes

Core claim

The central claim is that the Bayesian Fisher information matrix can be estimated consistently from i.i.d. parameter-measurement pairs by plugging learned scores into either the posterior identity or the decomposition into measurement and prior components, and that the relative error of the resulting bound is controlled by the score-matching objective. Formally, Corollaries V.12 and V.13 state that under the paper's assumptions the learned Fisher information matrix converges almost surely to the true one, so the LBCRB converges almost surely to the true BCRB. Theorems V.1–V.4 give non-asymptotic bounds in which the approximation error is bounded by a constant times the square root of the intrinsic dimension times the relative score error, and the empirical-mean error decays like a constant divided by the square root of the number of training samples, with a sample-size threshold that grows with the intrinsic dimension and the condition number of the bound.

Load-bearing premise

The almost-sure convergence and the non-asymptotic bounds rest on the assumptions that the true prior, posterior, and Fisher score functions lie exactly in the chosen neural network family and that training finds a global minimizer, and the practical objective also uses a learned prior score while the theory uses the true prior score, with the error from that substitution left unquantified.

Editorial extensions

If this is right

  • For any problem with a training set of parameter-measurement pairs, the LBCRB can be computed even when the prior and measurement distributions are completely unknown, including quantized and correlated-noise settings where density-based generative models fail.
  • The Measurement-Prior Approach can evaluate the bound for any desired number of i.i.d. measurements after a single training run, because it learns a single-sample Fisher score and scales it linearly.
  • With sufficient network capacity and a global minimizer, the learned bound converges almost surely to the true BCRB, so in the infinite-data limit the approximation error vanishes.
  • Finite-sample guarantees show the relative error of the learned bound scales with the square root of the intrinsic dimension divided by the number of training samples, and with the condition number of the true bound, so harder problems need more training samples.
  • Domain knowledge encoded through a physics-encoded score neural network reduces the network complexity and the sample complexity needed to reach a given accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The consistency proof assumes the true prior, posterior, and Fisher score functions lie exactly in the chosen neural network family; in practice this realizability is only approximate, so the convergence should be read as an idealization rather than a guarantee for any finite architecture.
  • The implemented Fisher score objective substitutes the learned prior score for the true prior score, while the theoretical analysis is carried out with the true prior score, leaving an unquantified bias in the practical algorithm that the current theorems do not cover.
  • The LBCRB is an approximation to the BCRB, not a guaranteed lower bound; a natural extension is to add error-correction or model-selection mechanisms to make it a valid bound, which the paper lists as future work.
  • The Fisher score learned by FSM is independent of the bound and could be reused for non-Bayesian Cramér-Rao bounds or for building learned maximum-likelihood estimators, directions the paper leaves open.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a learned Bayesian Cramér-Rao bound (LBCRB) that estimates the Bayesian FIM from i.i.d. parameter-measurement pairs when the prior and measurement distributions are unknown. Two approaches are presented: the Posterior Approach, which learns the posterior score via conditional score matching, and the Measurement-Prior Approach, which learns the prior score and a new Fisher score via a proposed Fisher Score Matching (FSM) objective, optionally using a physics-encoded score neural network. The authors derive finite-sample relative-error bounds for the learned FIM and LBCRB, prove strong consistency of the learned scores and of the LBCRB under realizability and global-minimization assumptions, and validate the method on linear Gaussian, quantized, and underwater-noise frequency-estimation examples. The central theoretical results are plausible and the appendices contain detailed proof sketches, but the practical FSM objective differs from the objective analyzed in the theory in a way that currently leaves a gap in the finite-sample guarantees.

Significance. If the identified gap is closed, this would be a substantial contribution: it provides the first fully learned Bayesian CRB that does not require knowledge of the Fisher score, introduces Fisher score matching as a new tool with potential independent value, and demonstrates strong empirical performance on problems where the BCRB is otherwise inaccessible. The paper ships reproducible code and validates against known bounds in linear Gaussian and quantized cases, which is a clear strength. The theoretical program is ambitious, with explicit non-asymptotic constants and a.s. consistency claims; however, those claims currently rest on an assumption set (realizability, global minimization) that is clearly stated but strong, and on an objective mismatch that must be resolved before the finite-sample results apply to the implemented method.

major comments (3)
  1. [Section IV-B.1, Eq. (14) vs Theorem IV.1, Eq. (26)] The implemented FSM objective (14) replaces the true prior score ∇_θ log f_Θ(θ) with the learned score s_P(θ; Ω_P^*), whereas Theorem IV.1 and all subsequent finite-sample results (Theorems V.2 and V.4, Corollaries V.6 and V.7) analyze the objective containing the true prior score. The equivalence L0_F = L_F + C used throughout the proofs no longer holds for the implemented objective; the additional cross term 2 E_{X,Θ}[s_F(X|Θ;Ω)^T(s_P(Θ;Ω_P^*) − ∇_Θ log f_Θ(Θ))] is neither bounded nor shown to vanish uniformly in Ω. Consequently, Theorem V.2 and Corollary V.7 do not directly bound the error of the actually trained model, and the proof of Theorem V.11 (Appendix H.1) is incomplete as written because it invokes the equivalence L0_F = L_F + C. Prior-score consistency (L0_P → 0 a.s.) may salvage the asymptotic consistency claim, but the finite-sample guarantees require either modifying the objective to use the true prior score or adding a quantified bound on this cross term.
  2. [Section II-B, Eq. (5)] Equation (5) defines the prior FIM as F_P = E_Θ[∇_Θ log f_Θ(Θ) ∇_Θ log f_{X|Θ}(X|θ)^T], which is the product of the prior score with the Fisher score. This object is not symmetric positive semidefinite in general and is not the prior FIM; it contradicts the decomposition F_B = n_iid F_M + F_P given in Eq. (3) and the later evaluation in Eq. (16), which correctly uses s_P s_P^T. The second factor should be ∇_Θ log f_Θ(Θ)^T. Although the intended definition is clear from context, as written this is a foundational error that should be corrected.
  3. [Section V-B, paragraph following Proposition V.5] The claim that with high probability c_B ≥ m_iid c_M + c_P, and hence that the Measurement-Prior Approach enjoys a √m_iid advantage in the empirical-mean error bound, is not rigorously established. Proposition V.5 only compares expectations of squared score norms; the passage from expectations to maxima over the dataset D is asserted ("suggests also a similar large gap between the maxima") without a concentration argument or additional assumptions. This is a load-bearing comparison for the claimed advantage of the Measurement-Prior Approach, so it should either be proved or explicitly labeled as a heuristic supported by experiments.
minor comments (5)
  1. [Theorem V.2] The statement defines d_P = intdim(F_M); it should read d_P = intdim(F_P), since the second term of the bound concerns the prior FIM.
  2. [Appendix G, proof of Corollary V.7] In the positive-definiteness condition, the text says "b_FB ≻ 0" but the argument concerns the Measurement-Prior learned FIM; it should refer to b_F_MP. Relatedly, the sentence "By definition we have b_V_MP ⪰ 0 and F_B ≻ 0" appears to contain a typo (b_V_MP is the bound, not the FIM).
  3. [Section VII-C] The phrase "standard divination" should be "standard deviation" in two places.
  4. [Abstract and Introduction] There are minor grammatical errors, e.g., "within a Bayesian frameworks" in the abstract; a careful proofreading pass is needed.
  5. [Section V-D and Conclusion] The consistency theorems rely on Assumption V.2 (NN-realizable score functions) and on global minimization of nonconvex objectives. These assumptions are stated explicitly, but it would help readers if the paper added a short limitations paragraph noting that the a.s. guarantees are conditional on realizability and global optimization, which are strong requirements in practice.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the learned BCRB is a plug-in estimator trained on score-matching objectives, with external validation against known BCRBs; the only notable issue is a non-circular gap between the implemented FSM objective and the analyzed true-prior objective.

full rationale

The claimed derivation chain is not circular. The LBCRB is a plug-in estimator: scores are trained by score-matching objectives that measure distance to the true score functions (or their integration-by-parts equivalents), and the learned FIMs are empirical averages of the learned scores; no parameter is fitted to the BCRB itself. The consistency arguments in Theorems V.10-V.11 and Corollaries V.12-V.13 are standard (realizability + global minimization + uniform SLLN), and the finite-sample bounds in Theorems V.1-V.4 and Corollaries V.6-V.7 propagate score-matching error through Lemma .2; they do not assume the conclusion. Validation is external: in the linear-Gaussian and 1-bit quantized examples the LBCRB is compared with analytically or numerically computed true BCRBs. The self-citations ([17]-[19], [21], and Lemma XIII.1 from [17] used in the proof of Lemma .2) are background or parameter-free technical inequalities, so they do not raise the circularity score. A genuine gap exists between the implemented FSM objective (14), which substitutes the learned prior score, and Theorem IV.1's equivalence L0_F = L_F + C, which requires the true prior score; the unquantified cross term means Theorem V.2 and Corollary V.7 do not directly bound the actually trained model, and the proof of Theorem V.11 is incomplete as written. This is a rigor gap in the theory-practice transfer, not a circular reduction: the learned prior score is not fitted to the BCRB, and the final bound is not an input to any training loss.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The method itself introduces no scalar free parameters fitted to the target bound; the neural network weights are learned from data and do not constitute a fitted constant. The central claims rest on standard BCRB regularity, score-matching boundary conditions, and strong but typical assumability conditions (realizability, Lipschitz continuity, global optimization).

assumptions (6)
  • domain assumption BCRB regularity conditions (Assumption II.1): smoothness, integrability, boundary and support conditions on the joint density.
    Standard for the Bayesian Cramér-Rao bound to hold. Invoked in Problem IV and throughout the theoretical sections.
  • domain assumption Score matching regularity and boundary conditions (Assumptions II.2, IV.1, IV.2, and boundary conditions (22), (25), (8)).
    Needed for the integration-by-parts steps that make score matching objectives equivalent to the L2 score mismatch. The boundary conditions require fΘ or fΘ|X to vanish at the parameter domain boundary, or the scores to decay suitably.
  • ad hoc to paper NN-realizable score functions (Assumption V.2): the true prior, posterior, and Fisher scores lie in the function classes of the chosen neural network architectures.
    This is the load-bearing premise for the strong consistency theorems; it is assumed rather than verified in the experiments, and is generally only approximately true in practice.
  • standard math Lipschitz continuity of the score networks and their derivatives (Assumption V.3).
    Technical regularity for the strong uniform law of large numbers used in the consistency proofs.
  • ad hoc to paper Global minimization of the nonconvex score matching objectives (Theorems V.10 and V.11, 'the minimization algorithm succeeds in finding a global minimizer').
    Neural network training is not guaranteed to find a global minimum; the theory assumes this.
  • domain assumption Measurement model factorizes through a known deterministic map M(θ) (equation (27)), with only the noise distribution unknown.
    This is the basis for the Physics-encoded Score Neural Network in the Measurement-Prior Approach.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learned Bayesian Cram\'er-Rao Bound for Unknown Measurement Models Using Score Neural Networks." pith.science (2026). https://pith.science/paper/76BA67K5

@misc{pith2026250200724,
  author       = {Pith},
  title        = {Pith review of: Learned Bayesian Cram\'er-Rao Bound for Unknown Measurement Models Using Score Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/76BA67K5}},
  note         = {Machine review of arXiv:2502.00724}
}
read the original abstract

The Bayesian Cram\'er-Rao bound (BCRB) is a crucial tool in signal processing for assessing the fundamental limitations of any estimation problem as well as benchmarking within a Bayesian frameworks. However, the BCRB cannot be computed without full knowledge of the prior and the measurement distributions. In this work, we propose a fully learned Bayesian Cram\'er-Rao bound (LBCRB) that learns both the prior and the measurement distributions. Specifically, we suggest two approaches to obtain the LBCRB: the Posterior Approach and the Measurement-Prior Approach. The Posterior Approach provides a simple method to obtain the LBCRB, whereas the Measurement-Prior Approach enables us to incorporate domain knowledge to improve the sample complexity and {interpretability}. To achieve this, we introduce a Physics-encoded score neural network which enables us to easily incorporate such domain knowledge into a neural network. We {study the learning} errors of the two suggested approaches theoretically, and validate them numerically. We demonstrate the two approaches on several signal processing examples, including a linear measurement problem with unknown mixing and Gaussian noise covariance matrices, frequency estimation, and quantized measurement. In addition, we test our approach on a nonlinear signal processing problem of frequency estimation with real-world underwater ambient noise.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Weighted Bayesian Cram$\acute{\text{e}}$r-Rao Bound for Mixed-Resolution Parameter Estimation

    eess.SP 2025-08 conditional novelty 5.0 of 10

    The paper derives a weighted Bayesian Cramér-Rao bound for mixed-resolution (analog plus 1-bit) estimation and an SNR-partitioned MSE approximation that tracks the MMSE better than the classical BCRB.

  2. Scoring ISAC: Benchmarking Integrated Sensing and Communications via Score-Based Generative Modeling

    eess.SP 2025-08 unverdicted novelty 4.0 of 10

    Score-based generative models can estimate ISAC performance metrics such as MI, MMSE, and BCRB from data, matching analytical ground truth in proof-of-concept tests.

Reference graph

Works this paper leans on

68 extracted references · 55 canonical work pages · cited by 2 Pith papers

  1. [1]

    H. L. Van Trees, Detection, estimation, and modulation theory, part I: detection, estimation, and linear modulation theory . John Wiley & Sons, 2004

    2004

  2. [2]

    Range-direction tracking and Bayesian Cramer–Rao bound analysis in mmwave systems equipped with im- perfect transceivers,

    D. A. Tubail and S. Ikki, “Range-direction tracking and Bayesian Cramer–Rao bound analysis in mmwave systems equipped with im- perfect transceivers,” IEEE Wireless Communications Letters , vol. 12, no. 10, pp. 1806–1810, 2023

    work page 2023

  3. [3]

    Bayesian Cram ´er-Rao lower bounds for magnetic field-based train localization,

    B. Siebler, S. Sand, and U. D. Hanebeck, “Bayesian Cram ´er-Rao lower bounds for magnetic field-based train localization,” in 2023 IEEE/ION Position, Location and Navigation Symposium (PLANS), 2023, pp. 814– 820

    work page 2023

  4. [4]

    Phase noise in mimo systems: Bayesian Cram´er–Rao bounds and soft-input estimation,

    A. A. Nasir, H. Mehrpouyan, R. Schober, and Y . Hua, “Phase noise in mimo systems: Bayesian Cram´er–Rao bounds and soft-input estimation,” IEEE transactions on signal processing, vol. 61, no. 10, pp. 2675–2692, 2013

    work page 2013

  5. [5]

    Bayesian bounds for matched-field parameter estimation,

    W. Xu, A. B. Baggeroer, and C. D. Richmond, “Bayesian bounds for matched-field parameter estimation,” IEEE Transactions on Signal Processing, vol. 52, no. 12, pp. 3293–3305, 2004

    work page 2004

  6. [6]

    Asymptotically tight misspecified bayesian cram ´er-rao bound,

    N. E. Rosentha and J. Tabrikian, “Asymptotically tight misspecified bayesian cram ´er-rao bound,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2024, pp. 9916–9920

    work page 2024

  7. [7]

    On the limitations of the bayesian cram ´er-rao bound for mixed-resolution data,

    Y . Mazor, I. E. Berman, and T. Routtenberg, “On the limitations of the bayesian cram ´er-rao bound for mixed-resolution data,” IEEE Signal Processing Letters, 2024

    work page 2024

  8. [8]

    Optimal adaptive waveform design for cognitive mimo radar,

    W. Huleihel, J. Tabrikian, and R. Shavit, “Optimal adaptive waveform design for cognitive mimo radar,” IEEE Transactions on Signal Process- ing, vol. 61, no. 20, pp. 5075–5089, 2013

    work page 2013

Show all 68 references
  1. [9]

    Bayesian sequential parameter estimation by cognitive radar with multiantenna arrays,

    A. Turlapaty and Y . Jin, “Bayesian sequential parameter estimation by cognitive radar with multiantenna arrays,” IEEE Transactions on Signal Processing, vol. 63, no. 4, pp. 974–987, 2014

    2014

  2. [10]

    Conditional posterior Cram ´er– Rao lower bounds for nonlinear sequential bayesian estimation,

    L. Zuo, R. Niu, and P. K. Varshney, “Conditional posterior Cram ´er– Rao lower bounds for nonlinear sequential bayesian estimation,” IEEE Transactions on Signal Processing , vol. 59, no. 1, pp. 1–14, 2010

    2010

  3. [11]

    Optimal ratio between co- herent and orthogonal signals in sparse mimo radar,

    H. Sun, J. Tabrikian, H. Messer, and H. Gao, “Optimal ratio between co- herent and orthogonal signals in sparse mimo radar,” in 2024 IEEE 13rd Sensor Array and Multichannel Signal Processing Workshop (SAM) . IEEE, 2024, pp. 1–5

    2024

  4. [12]

    Sepit: Approaching a single channel speech separation bound,

    S. Lutati, E. Nachmani, and L. Wolf, “Sepit: Approaching a single channel speech separation bound,” in Interspeech 2022, 2022, pp. 5323– 5327

    2022

  5. [13]

    Fisher information neural estimation,

    T. T. Duy, L. V . Nguyen, V .-D. Nguyen, N. L. Trung, and K. Abed- Meraim, “Fisher information neural estimation,” in 2022 30th European Signal Processing Conference (EUSIPCO) . IEEE, 2022, pp. 2111– 2115

    2022

  6. [14]

    Empirical non-parametric estimation of the fisher information,

    V . Berisha and A. O. Hero, “Empirical non-parametric estimation of the fisher information,” IEEE Signal Processing Letters , vol. 22, no. 7, pp. 988–992, 2015

    2015

  7. [15]

    Generative modeling by estimating gradients of the data distribution,

    Y . Song and S. Ermon, “Generative modeling by estimating gradients of the data distribution,” Advances in neural information processing systems, vol. 32, 2019

    2019

  8. [16]

    Normalizing flows: An introduction and review of current methods,

    I. Kobyzev, S. Prince, and M. Brubaker, “Normalizing flows: An introduction and review of current methods,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2020

    2020

  9. [17]

    Learning to bound: A generative Cram´er-Rao bound,

    H. V . Habi, H. Messer, and Y . Bresler, “Learning to bound: A generative Cram´er-Rao bound,” IEEE Transactions on Signal Processing , 2023

    2023

  10. [18]

    Learned generative misspecified lower bound,

    H. V . Habi, H. Messer, and Y . Bresler, “Learned generative misspecified lower bound,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2023, pp. 1–5

    2023

  11. [19]

    Learning the Barankin lower bound on doa estimation error,

    H. V . Habi, H. Messer, and Y . Bresler, “Learning the Barankin lower bound on doa estimation error,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 9906–9910. HABI, MESSER, AND BRESLER: LEARNED BA YESIAN ...

    2024

  12. [20]

    Normalizing flows for probabilistic modeling and inference,

    G. Papamakarios, E. Nalisnick, D. J. Rezende, S. Mohamed, and B. Lakshminarayanan, “Normalizing flows for probabilistic modeling and inference,” Journal of Machine Learning Research , vol. 22, no. 57, pp. 1–64, 2021

    2021

  13. [21]

    A generative Cram´er-Rao bound on frequency estimation with learned measurement distribution,

    H. V . Habi, H. Messer, and Y . Bresler, “A generative Cram´er-Rao bound on frequency estimation with learned measurement distribution,” in 2022 IEEE 12th Sensor Array and Multichannel Signal Processing Workshop (SAM). IEEE, 2022, pp. 176–180

    2022

  14. [22]

    Bayesian Cram ´er-Rao bound estimation with score-based models,

    E. S. Crafts, X. Zhang, and B. Zhao, “Bayesian Cram ´er-Rao bound estimation with score-based models,” IEEE Transactions on Information Theory, pp. 1–1, 2024

    2024

  15. [23]

    Estimation of non-normalized statistical models by score matching

    A. Hyv ¨arinen and P. Dayan, “Estimation of non-normalized statistical models by score matching.” Journal of Machine Learning Research , vol. 6, no. 4, 2005

    2005

  16. [24]

    Noise flow: Noise modeling with conditional normalizing flows,

    A. Abdelhamed, M. A. Brubaker, and M. S. Brown, “Noise flow: Noise modeling with conditional normalizing flows,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2019, pp. 3165–3173

    2019

  17. [25]

    Towards robust data- driven underwater acoustic localization: A deep cnn solution with per- formance guarantees for model mismatch,

    A. Weiss, A. C. Singer, and G. W. Wornell, “Towards robust data- driven underwater acoustic localization: A deep cnn solution with per- formance guarantees for model mismatch,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP...

    2023

  18. [26]

    Vtuad: Vessel type underwater acoustic data,

    L. Domingos, P. Skelton, and P. Santos, “Vtuad: Vessel type underwater acoustic data,” 2022. [Online]. Available: https://dx.doi.org/10.21227/ msg0-ag12

    2022

  19. [27]

    Model-based deep learning: On the intersection of deep learning and optimization,

    N. Shlezinger, Y . C. Eldar, and S. P. Boyd, “Model-based deep learning: On the intersection of deep learning and optimization,” IEEE Access , vol. 10, pp. 115 384–115 398, 2022

    2022

  20. [28]

    Model-based deep learning,

    N. Shlezinger, J. Whang, Y . C. Eldar, and A. G. Dimakis, “Model-based deep learning,” Proceedings of the IEEE , vol. 111, no. 5, pp. 465–499, 2023

    2023

  21. [29]

    Physics-informed computer vision: A review and perspectives,

    C. Banerjee, K. Nguyen, C. Fookes, and K. George, “Physics-informed computer vision: A review and perspectives,” ACM Computing Surveys, vol. 57, no. 1, pp. 1–38, 2024

    2024

  22. [30]

    Physics-guided, physics-informed, and physics-encoded neural networks and operators in scientific comput- ing: Fluid and solid mechanics,

    S. A. Faroughi, N. M. Pawar, C. Fernandes, M. Raissi, S. Das, N. K. Kalantari, and S. Kourosh Mahjour, “Physics-guided, physics-informed, and physics-encoded neural networks and operators in scientific comput- ing: Fluid and solid mechanics,” Journal of Computing and Informati...

    2024

  23. [31]

    Application of physics encoded neural networks to improve predictability of properties of complex multi-scale systems,

    M. B. Meinders, J. Yang, and E. v. d. Linden, “Application of physics encoded neural networks to improve predictability of properties of complex multi-scale systems,” Scientific Reports , vol. 14, no. 1, p. 15015, 2024

    2024

  24. [32]

    Estimating density models with truncation boundaries using score matching,

    S. Liu, T. Kanamori, and D. J. Williams, “Estimating density models with truncation boundaries using score matching,” Journal of Machine Learning Research, vol. 23, no. 186, pp. 1–38, 2022

    2022

  25. [33]

    Generalized score matching for non- negative data,

    S. Yu, M. Drton, and A. Shojaie, “Generalized score matching for non- negative data,” The Journal of Machine Learning Research , vol. 20, no. 1, pp. 2779–2848, 2019

    2019

  26. [34]

    Generalized score matching for general domains,

    S. Yu, M. Drton, and A. Shojaie, “Generalized score matching for general domains,” Information and Inference: A Journal of the IMA , vol. 11, no. 2, pp. 739–780, 2022

    2022

  27. [35]

    Conditional generative adversarial nets,

    M. Mirza and S. Osindero, “Conditional generative adversarial nets,” arXiv preprint arXiv:1411.1784 , 2014

    2014 arXiv

  28. [36]

    Conditional adversarial generative flow for controllable image synthesis,

    R. Liu, Y . Liu, X. Gong, X. Wang, and H. Li, “Conditional adversarial generative flow for controllable image synthesis,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 7992–8001

    2019

  29. [37]

    Classifier-free diffusion guidance,

    J. Ho and T. Salimans, “Classifier-free diffusion guidance,” in NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applica- tions, 2021

    2021

  30. [38]

    The effects of ship noise on marine mammals—a review,

    C. Erbe, S. A. Marley, R. P. Schoeman, J. N. Smith, L. E. Trigg, and C. B. Embling, “The effects of ship noise on marine mammals—a review,” Frontiers in Marine Science , vol. 6, p. 606, 2019

    2019

  31. [39]

    Learned Bayesian e Cram ´er Rao bound,

    H. V . Habi, “Learned Bayesian e Cram ´er Rao bound,” https://github. com/haihabi/Learned-BCRB, 2024

    2024

  32. [40]

    Bayesian bounds for parameter estimation and nonlinear filtering/tracking,

    H. L. Van Trees and K. L. Bell, “Bayesian bounds for parameter estimation and nonlinear filtering/tracking,” AMC, vol. 10, p. 12, 2007

    2007

  33. [41]

    A general class of lower bounds in parameter estimation,

    E. Weinstein and A. J. Weiss, “A general class of lower bounds in parameter estimation,” IEEE Transactions on Information Theory , vol. 34, no. 2, pp. 338–342, 1988

    1988

  34. [42]

    Bayesian parameter estimation using single-bit dithered quantization,

    G. Zeitler, G. Kramer, and A. C. Singer, “Bayesian parameter estimation using single-bit dithered quantization,” IEEE Transactions on Signal Processing, vol. 60, no. 6, pp. 2713–2726, 2012

    2012

  35. [43]

    Some extensions of score matching,

    A. Hyv ¨arinen, “Some extensions of score matching,” Computational statistics & data analysis , vol. 51, no. 5, pp. 2499–2512, 2007

    2007

  36. [44]

    Score-based generative modeling through stochastic differ- ential equations,

    Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differ- ential equations,” arXiv preprint arXiv:2011.13456 , 2020

    2011 arXiv

  37. [45]

    Shalev-Shwartz and S

    S. Shalev-Shwartz and S. Ben-David, Understanding machine learning: From theory to algorithms . Cambridge university press, 2014

    2014

  38. [46]

    An introduction to matrix concentration inequalities,

    J. A. Tropp et al., “An introduction to matrix concentration inequalities,” Foundations and Trends® in Machine Learning , vol. 8, no. 1-2, pp. 1– 230, 2015

    2015

  39. [47]

    Stable rank and intrinsic dimension of real and complex matrices,

    I. C. Ipsen and A. K. Saibaba, “Stable rank and intrinsic dimension of real and complex matrices,” arXiv preprint arXiv:2407.21594 , 2024

    2024 arXiv

  40. [48]

    Genz and F

    A. Genz and F. Bretz, Computation of multivariate normal and t probabilities. Springer Science & Business Media, 2009, vol. 195

    2009

  41. [49]

    Searching for activation functions,

    P. Ramachandran, B. Zoph, and Q. V . Le, “Searching for activation functions,” arXiv preprint arXiv:1710.05941 , 2017

    2017 arXiv

  42. [50]

    The Cram ´er–Rao bound for signal parameter estimation from quantized data [lecture notes],

    P. Stoica, X. Shang, and Y . Cheng, “The Cram ´er–Rao bound for signal parameter estimation from quantized data [lecture notes],” IEEE Signal Processing Magazine, vol. 39, no. 1, pp. 118–125, 2022

    2022

  43. [51]

    The gaussian data assumption leads to the largest Cram´er-Rao bound [lecture notes],

    P. Stoica and P. Babu, “The gaussian data assumption leads to the largest Cram´er-Rao bound [lecture notes],” IEEE Signal Processing Magazine , vol. 28, no. 3, pp. 132–133, 2011

    2011

  44. [52]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in International Conference on Learning Representations , 2019. [Online]. Available: https://openreview.net/forum?id=Bkg6RiCqY7

    2019

  45. [53]

    Improved techniques for training score-based generative models,

    Y . Song and S. Ermon, “Improved techniques for training score-based generative models,” Advances in neural information processing systems , vol. 33, pp. 12 438–12 448, 2020

    2020

  46. [54]

    On the perturbation of pseudo-inverses, projections and linear least squares problems,

    G. W. Stewart, “On the perturbation of pseudo-inverses, projections and linear least squares problems,” SIAM review, vol. 19, no. 4, pp. 634–662, 1977

    1977

  47. [55]

    Generic uniform convergence,

    D. W. Andrews, “Generic uniform convergence,” Econometric theory , vol. 8, no. 2, pp. 241–257, 1992. 20 FEBRUARY 2025 zℓ c θ Parameter Injection FC(ni, nh) Condition Injection NL zℓ+1 Fig. 11: Basic NN Block T ABLE I: Neural Network Configuration and Parameters for Each Exampl...

    1992

  48. [56]

    The parameters of the NN structures used for the various examples are listed in Table I

    Neural Network Structure: basic blocks illustrated in Figure 11, each performing the following sequence of oper- ations: (i) parameter injection, where the input is scaled and shifted based on the parameter vector θ; (ii) a fully connected operation; (iii) condition injection ...

  49. [57]

    Hyper-parameters and T raining: In all experiments, we use the following hyperparameters. We train the score neural network for 200 epochs using the AdamW [52] optimizer with learning rate 4e-4 and weight decay 1e − 4, and a data set of 60k samples for each S- condition.If a s...

  50. [58]

    Each element in A is generated by [A]ij ∼ N(0, 1)

    Measurement Models Setup: The linear measurement model (44) or the quantized linear measurement model (50) are set up as follows. Each element in A is generated by [A]ij ∼ N(0, 1). For the covariance matrix, we first generate U ∈ Rdx×dx as [U]ij ∼ N(0, 1) and then Σ = UUT Tr(U...

  51. [59]

    V .1: Proof

    Posterior FIM Approximation Error Thm. V .1: Proof. We set X = s∗ B θ|X and Y = ∇θ log fθ|X θ|X , then we apply Lemma .2, establishing (30)

  52. [60]

    V .2: Proof

    Measurement-Prior FIM Approximation error Thm. V .2: Proof. We apply Lemma .2 twice. First, for the measurement FIM we set X = s∗ F x|θ and Y = ∇θ log fx|θ X|θ in Lemma .2, which results in FM − FM 2 ∥FM ∥2 ≤ η(a) M ≜ 2.4 s intdim (FM ) · L0 F Tr (FM ) . (68) Second, for the p...

    2025

  53. [61]

    V .3: Proof

    Posterior Approach Empirical Mean Error Thm. V .3: Proof. Denote sk = s∗ B θk|xk and Xk = 1 ND sksT k − FB . First we validate the assumptions of Proposition .3. ED [Xk] = 1 ND ED h sksT k i − FB = 0, and ∥Xk∥ ≤ 1 ND sksT k + FB ≤ 1 ND cB + FB 2 ≜ L. (75) Next, we calculate V:...

  54. [62]

    V .4: Proof

    Measurement-Prior Empirical Mean Error Thm. V .4: Proof. Denote sk,i = s∗ F ˜xk,i|θk as the Fisher score vector and pk = s∗ P (θk) as the prior score of the kth sample in D. Now, let Xk = 1 ND   niid miid · miidX i=1 sk,isT k,i + pkpT k − FM P   = 1 ND Rk. First we validat...

    2025

  55. [63]

    V .6: Proof

    Relative Error Posterior Approach Corr. V .6: Proof. If bFB ≻ 0 it is invertible and REB = bVB − V 2 ∥V∥ 2 = bF−1 B FB − bFB F−1 B 2 ∥V∥ 2 ≤ bVB 2 FB − bFB 2 ≤ bVB 2 FB − bFB 2 + bVB 2 FB − FB 2 . (85) By combining (30) and (32) with (85) we have: REB ≤ bVB 2 FB 2 η(e) B +∥FB∥...

  56. [64]

    V .7: Proof

    Relative Error Measurement-Prior Approach Corr. V .7: Proof. If bFM P≻ 0 then it is invertible, and REM P≜ bVM P− V 2 ∥V∥ 2 = bVM P FB − bFM P V 2 ∥V∥ 2 ≤ bVM P 2 FB − bFM P 2 ≤ bVM P 2 FM P− bFM P 2 + bVM P 2 FM P− FB 2 (88) By combining (31) and (34) with (88) we have: REM P...

  57. [65]

    ∂sB Θ|X ; Ω ∂Θ # i,i   = − Z X ,Θ h sB Θ|X ; Ω i i

    Posterior Score Consistency Proof (Thm V .10): Lemma .8 (Posterior Score Loss is Lipschitz continuous) . Define the per-sample loss of the posterior score matching ℓB θ, eX ; Ω ≜ sB θ| eX ; Ω 2 2 + 2Tr ∂sB θ| eX ;Ω ∂θ ! , and ∆Ω ≜ ∥Ω1 − Ω2∥2. Assume that: • The posterior score...

    2025

  58. [66]

    Lemma .9 (Fisher Score is Lipschitz continuous)

    Prior and Fisher Score Consistency Proof (Thm V .11): First we establish that the Lipschit continuouity requirements are met. Lemma .9 (Fisher Score is Lipschitz continuous) . Define the per-sample loss of the Fisher score matching ℓF (x, θ; Ω) ≜ sF x|θ; ΩF 2 2 + 2 sF x|θ; Ω T...

  59. [67]

    (ακ − 1)2 (κ − al)2 # = (ακ − 1)2 B (ακ − 2, βκ) (ah − al)2 B (ακ, βκ) (114) Next, using the definition of Beta function we have: Eκ

    Prior FIM of Beta Distribution: Let κ ∼ Beta (ακ, βκ, al, ah) be a random variable that is distribution according to the four-parameter beta distribution. Then, the PDF of κ is given by: fκ (κ) = (κ − al)ακ−1 (au − κ)βκ−1 (ah − al)ακ+βκ−1 B (ακ, βκ) (112) where al, au are the ...

    2025

  60. [68]

    Prior FIM of Gaussian Distribution: Let κ ∼ N(µ, Σ) be a random vector of size k that distributed according to Gaussian distribution with mean µ and covariance Σ. fκ (κ) = 1q (2π)k det Σ exp − 1 2 (κ − µ)T Σ−1 (κ − µ) (120) Then the score vector of κ is given by: ∇κ log fκ (κ)...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.