Pith. sign in

REVIEW 3 major objections 5 minor 77 references

Sparse Bayesian Generative Modeling for Compressive Sensing

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper introduces sparse Bayesian generative models, a learnable prior for compressive sensing that is sparsity-inducing, trains on compressed noisy data alone, and reconstructs without optimization at inference.

desk verdict A genuinely new model class with reproducible empirical gains, but the headline sparsity-evidence theorem is vacuous because the improper prior makes the target evidence infinite; the method deserves a serious referee with the theory reframed. read the letter →

arxiv 2411.09483 v1 pith:CXJVM2MQ submitted 2024-11-14 stat.ML cs.LGeess.IV

Benedikt Böck , Sadaf Syed , Wolfgang Utschick This is my paper · ORCID
classification stat.MLcs.LGeess.IV
keywords compressivesensingsparseBayesianlearninggenerativepriorconditionalGaussianityvariationalautoencoderGaussianmixturemodeluncertaintyquantificationlinearinverseproblem
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Compressive sensing recovers an $N$-dimensional signal from $M \ll N$ linear measurements, and classical methods add sparsity in a known dictionary. This paper proposes a learnable prior, the sparse Bayesian generative model, in which the sparse coefficients $s$ are conditionally Gaussian with zero mean and diagonal covariance given a latent variable $z$, i.e., $s|z \sim \mathcal{N}(0,\mathrm{diag}(\gamma_\theta(z)))$. The authors prove that the model's log-evidence is a variational lower bound of a sparsity-inducing log-evidence, so training by variational inference maximizes a sparsity-inducing objective. Because the conditional posterior $p_\theta(s|z,y)$ is closed form, the VAE-style and GMM-style implementations can be learned from a few hundred compressed and noisy observations, with no ground-truth data, and reconstruct a new signal by a feed-forward evaluation. Experiments on MNIST, piecewise-smooth functions, CelebA, and FashionMNIST show better distortion than Lasso, SBL, CSGAN, and compressive K-SVD, with much lower inference time.

What carries the argument

The load-bearing object is the conditional-Gaussian decomposition in (7): $y|s \sim \mathcal{N}(ADs,\sigma^2 I)$, $s|z \sim \mathcal{N}(0,\mathrm{diag}(\gamma_\theta(z)))$, $z \sim p_\delta(z)$. Its two properties do all the work: zero-mean diagonal covariance makes $s|z$ a conjugate prior for $y|s$, giving the closed-form posterior $p_\theta(s|z,y)$ with mean and covariance in (8); and the same zero-mean diagonal structure lets Theorem 3.1 dominate the marginal $p_{\theta,\delta}(s)$ by the improper prior $t(s)=C\prod_i |s_i|^{-1}$. The theorem's bound is independent of $\gamma_\theta$ and $p_\delta$, so any parameterization is allowed without losing the sparsity guarantee. Inference then reduces to reading $\gamma_\theta(z)$ from the decoder, computing the closed-form conditional mean, and averaging over $z$ either by Monte Carlo (CSV AE) or in closed form (CSGMM).

What would settle it

Take a signal class whose true sparse coefficients have clearly nonzero means or heavy tails, train the proposed CSV AE and CSGMM on compressed noisy observations, and compare the resulting posterior-mean estimates with the oracle Bayesian posterior mean and with SBL; if the error remains substantially above the oracle as the training set grows, the zero-mean diagonal-Gaussian restriction is the binding limitation.

Watch

Extended reading notes

Core claim

The central claim is that conditioning the sparse Bayesian learning prior on a latent variable preserves its sparsity-inducing character while adding the adaptability of a generative model. Concretely, for $p_\theta(s|z)=\mathcal{N}(s;0,\mathrm{diag}(\gamma_\theta(z)))$ with any $p_\delta(z)$ and any positive variances, Theorem 3.1 shows $p_{\theta,\delta}(s)=\int p_\delta(z)p_\theta(s|z)\,dz \le t(s)=C\prod_{i=1}^{N}|s_i|^{-1}$, the same improper prior that drives sparse Bayesian learning. Hence the model's log-evidence is a variational lower bound of a sparsity-inducing log-evidence, and maximizing it is evidence maximization over a variational family that is necessarily sparse. The paper instantiates the family as a VAE whose decoder outputs the variances $\gamma_\theta(z)$ (CSV AE) and as a Gaussian mixture with zero-mean, diagonal-covariance components (CSGMM), and uses the closed-form Gaussian posterior in (8) to approximate the conditional mean estimator without iterative optimization.

Load-bearing premise

The method assumes that, after conditioning on a latent variable, the signal's sparse coefficients in a fixed known dictionary are exactly independent zero-mean Gaussian variables; real sparse coefficients are only approximately so, and the paper concedes this restriction introduces a bias that can prevent perfectly learning the true coefficient distribution.

Editorial extensions

If this is right

  • A compressive-sensing prior can be trained from compressed, noisy observations alone: the paper shows that several hundred samples suffice on MNIST and on piecewise-smooth functions, so ground-truth acquisition can be avoided in applications such as electron microscopy, wearable ECG, and wireless channel estimation.
  • Reconstruction at test time is a feed-forward evaluation: no iterative solver, no EM loop, and no random restarts are needed after training, with measured per-image inference times around 2 ms for MNIST.
  • The same architecture can be trained on ground-truth data when it is available, and the resulting estimators improve further in distortion, so the method interpolates between pure compressed-only and fully supervised regimes.
  • Uncertainty is available for free from the learned latent posterior: the differential entropy $h(z|y)$ of the encoder separates in-distribution from out-of-distribution compressed samples, enabling a built-in detector for data the model was not trained on.
  • The method applies to any signal compressible in a known dictionary, and the experiments show it is stable across dictionaries such as Haar, db3, and db5 as long as the domain is genuinely sparse.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: Theorem 3.1 really defines a family of sparsity-inducing priors, so a natural extension is to replace the fixed dictionary $D$ by a learned dictionary, or to allow nonzero conditional means, provided the domination $p_{\theta,\delta}(s)\le t(s)$ is maintained; the paper lists learnable dictionaries as future work.
  • Editorial inference: the entropy signal $h(z|y)$ could be developed into a practical out-of-distribution detector for compressed inputs, which is attractive in wireless channel estimation and medical imaging where ground-truth samples are scarce.
  • Editorial inference: because reconstruction already works with one Monte Carlo sample ($N_s=1$), the method has a direct compute-versus-quality trade-off suitable for real-time or embedded deployment; the paper reports this for MNIST but does not explore it as a deployment knob.
  • Editorial inference: the zero-mean diagonal restriction is a bias that may be doing useful regularization work; comparing it to a version with learned full covariance would reveal whether the sparsity guarantee or the restricted family is what drives the empirical gains.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces a class of hierarchical Bayesian priors for compressive sensing, in which the signal is modeled as x = Ds with s|z ~ N(0, diag(gamma_theta(z))) and z ~ p_delta(z), and the unknown covariance parameters are learned from compressed and noisy observations. Two instantiations are proposed: a VAE-type model (CSVAE) trained with an adapted ELBO and a GMM-type model (CSGMM) trained with EM. The authors claim that the training objective is a variational lower bound of a sparsity-inducing log-evidence, based on a pointwise bound relating the conditional Gaussian prior to the improper prior t(s) = C * prod_i 1/|s_i|. The paper also contributes fast, optimization-free inference and an uncertainty-quantification application, and validates the methods on MNIST, FashionMNIST, piecewise smooth functions, and CelebA.

Significance. If the theoretical claim were valid, the paper would make a useful contribution: learning a generative prior from compressed/noisy data without ground truth is practically relevant, the closed-form EM updates and low-dimensional reformulations in Appendix I are technically useful, and the reported reconstruction times are substantially lower than the baselines. The authors also release code, which supports reproducibility. However, the central theoretical contribution (Theorem 3.1 and Eq. (11), the claim that training maximizes a variational lower bound of a sparsity-inducing log-evidence) is vacuous as stated, and the empirical sections do not directly test the sparsity-inducing property. The practical estimator may still be valuable, but the paper's headline theoretical justification is not established.

major comments (3)
  1. [Section 3.1, Eq. (11) and Appendix A] The 'sparsity-inducing log-evidence' is infinite for every observation, so the central lower-bound claim is vacuous. Since p(y|s) is Gaussian and hence strictly positive at s = 0, the integral pi_s(y) = integral p(y|s) t(s) ds = C * integral p(y|s) * prod_i |s_i|^{-1} ds diverges to +infinity in a neighborhood of s = 0 for every y. Therefore log pi_s(y) = infinity and Eq. (11) reduces to 'infinity >= log p_{theta,delta}(y)', which imposes no constraint on theta and delta and does not support the statement that maximizing log p_{theta,delta}(y) is evidence maximization over variational approximations to a sparse-prior model. The pointwise bound (10) is mathematically correct, but it is not a sparsity property: any bounded density satisfies p(s) <= C * prod_i |s_i|^{-1} near the coordinate hyperplanes because the right-hand side diverges there. This concern is independent of the zero-mean Gaussian/diagonal modeling restriction in Section 2.4.
  2. [Section 3.1 and Appendix A] The proof strategy cannot be repaired by replacing t(s) with a fixed proper prior. Because gamma_theta(z) is allowed to approach 0, p_{theta,delta}(s) can approximate a point mass at the origin, so the density values at s = 0 are unbounded over the model class; no fixed proper prior with a finite density at 0 can dominate all densities in this class pointwise. A meaningful universal lower-bound statement therefore requires either a restriction such as gamma_theta(z) >= gamma_min > 0, which is absent and would itself change the sparsity behavior, or a fundamentally different theoretical framework. As it stands, the paper's contribution 2 (a variational lower bound of a sparsity-inducing log-evidence) is not merely underproven but false in its stated interpretation.
  3. [Section 4.2] The paper does not provide direct evidence that the learned priors actually induce sparse representations. The empirical evaluation reports reconstruction metrics (nMSE, SSIM) only; there is no analysis of the learned gamma values, the posterior coefficient distributions, or the effective sparsity of the estimates. Given that the theoretical sparsity argument is vacuous, the title claim 'sparsity-inducing' needs empirical support, for example by comparing the decay or sparsity of the recovered coefficients under CSV AE/CSGMM with those from SBL or Lasso, or by reporting the learned gamma distributions.
minor comments (5)
  1. [Appendix E, Eq. (31)] The formula for the Gaussian KL divergence is misprinted: it is missing the squares on mu_{j,phi}(y_i) and sigma_{j,phi}(y_i), and there is an extra closing parenthesis. In addition, the citation [46] is the Adam optimizer paper, which is not the source of this KL formula.
  2. [Section 3.2, Eq. (14)] The reconstruction term is written as E_{q_phi(z|y_i)}[ E_{p_theta(s|z,y_i)}[ log p_theta(y_i|s) ] ]; the inner likelihood should be log p(y_i|s), since the observation model in (7) does not depend on theta. This typo makes the derivation harder to follow.
  3. [Section 3.4] The zero-mean restriction is acknowledged as a bias that prevents perfectly learning p(s), but the paper should also state explicitly that this restriction is essential to the closed-form posterior (8) and to the (claimed) bound (10), and that the learned prior is therefore not guaranteed to approximate the true p(s) even with infinite data.
  4. [Figure 2 and Appendix K] The Lasso baseline on MNIST uses a fixed shrinkage parameter lambda = 0.1 without cross-validation, while the piecewise-smooth experiments tune lambda on a validation set. This inconsistency should be justified or corrected, since the main comparison claims 'overall the best' performance.
  5. [Section 4.2, Fig. 3(f)] The uncertainty quantification experiment reports a histogram of h(z|y) for one trained CSV AE, but there is no calibration analysis, no error bars, and no comparison with a baseline. At minimum, report the average entropies and standard deviations over multiple seeds.

Circularity Check

1 steps flagged · score 6.0 of 10

Theorem 3.1's sparsity-inducing log-evidence lower bound is infinite by construction, so the central sparsity guarantee is vacuous.

  1. other [Section 3.1, Eq. (10)-(11); proof in Appendix A]
    "pθ,δ(s) = ∫ pδ(z)N (s; 0, diag(γθ(z)))dz ≤ t(s) = C · ∏N i=1 1/|si|. (10) ... Based on Theorem 3.1, it holds that log π(s)(y) ≥ log pθ,δ(y) for all θ, δ (11) with log π(s)(y) being the log-evidence of y|s ∼ p(y|s) and the improper but sparsity promoting prior t(s)."

    The claimed lower bound is vacuous by construction. The prior t(s)=C∏1/|s_i| has a non-integrable singularity at s=0, and the Gaussian likelihood p(y|s) is strictly positive at s=0, so π_s(y)=∫p(y|s)t(s)ds=∞ for every y. Thus (11) reduces to the tautology ∞ ≥ log p_{θ,δ}(y), which imposes no constraint on θ or δ and does not establish that the model is sparsity-inducing. The 'sparsity-inducing log-evidence' is defined through the improper t(s), and the theorem's interpretation collapses: maximizing log p_{θ,δ}(y) trivially 'maximizes a lower bound' of an infinite quantity, true of any model and carrying no sparsity content.

full rationale

The empirical pipeline is not circular: CSV AE and CSGMM parameters are trained by maximizing an ELBO or EM objective on compressed observations and are then evaluated on held-out observations, with no fitted quantity disguised as a prediction. The CME approximation uses the standard law of total expectation and relies on the external result [22] plus prior work [32]; those self-citations are not themselves load-bearing for the method's internal consistency. The circular/vacuous step is the theoretical claim in Section 3.1. Equation (10) is a true pointwise bound, but the conclusion (11) fails to be a meaningful variational lower bound because the improper prior t(s) makes the evidence π_s(y) infinite for every observation. Since the Gaussian likelihood is bounded below by a positive constant near s=0, the integral defining π_s(y) diverges at the origin. Consequently, the advertised theoretical underpinning of contribution 2 reduces to the trivial statement ∞ ≥ log p_{θ,δ}(y), while the reconstruction experiments remain an independent empirical claim.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central derivation depends on the learned variances gamma_theta(z), the GMM parameters, the hand-set noise variance, and the chosen dictionary. No new physical entities are introduced; the model is a statistical prior.

free parameters (4)
  • gamma_theta(z) (CSV AE decoder outputs) = learned on Y
    Variances of the conditional Gaussian prior in (7); learned by maximizing ELBO (15) on compressed samples.
  • gamma_k, rho_k (CSGMM parameters) = learned via EM on Y
    Component variances and weights in the discrete-latent case; updated by Lemma 3.2 (Eq 18).
  • Noise variance sigma^2 = set to 40 dB SNR equivalent even in noiseless experiments
    Chosen by hand in Appendix K to enable the low-dimensional reformulations; affects all dictionary-based estimators.
  • Dictionary D = db4 overcomplete (block-diagonal for color)
    Chosen by hand, not learned; central to the compressibility assumption.
assumptions (5)
  • domain assumption x = D s with s approximately sparse and i.i.d. training samples
    Section 2.1: all xi are compressible in known D.
  • domain assumption s|z is zero-mean Gaussian with diagonal covariance, z ~ p_delta(z)
    Eq (7) in Section 2.4; enables the closed-form posterior and Theorem 3.1.
  • domain assumption Known noise model n ~ N(0, sigma^2 I) with known sigma^2
    Eq (1) and used throughout; in practice sigma^2 is set by hand.
  • standard math Variational inference and EM convergence for latent variable models
    Used to derive the ELBO (15) and m-step (18).
  • standard math Wipf-Rao bound (3): for every gamma there is C with N(s;0,diag(gamma)) <= C prod 1/|s_i|
    Imported from [18], restated in Appendix A; basis for Theorem 3.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sparse Bayesian Generative Modeling for Compressive Sensing." pith.science (2026). https://pith.science/paper/CXJVM2MQ

@misc{pith2026241109483,
  author       = {Pith},
  title        = {Pith review of: Sparse Bayesian Generative Modeling for Compressive Sensing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CXJVM2MQ}},
  note         = {Machine review of arXiv:2411.09483}
}
read the original abstract

This work addresses the fundamental linear inverse problem in compressive sensing (CS) by introducing a new type of regularizing generative prior. Our proposed method utilizes ideas from classical dictionary-based CS and, in particular, sparse Bayesian learning (SBL), to integrate a strong regularization towards sparse solutions. At the same time, by leveraging the notion of conditional Gaussianity, it also incorporates the adaptability from generative models to training data. However, unlike most state-of-the-art generative models, it is able to learn from a few compressed and noisy data samples and requires no optimization algorithm for solving the inverse problem. Additionally, similar to Dirichlet prior networks, our model parameterizes a conjugate prior enabling its application for uncertainty quantification. We support our approach theoretically through the concept of variational inference and validate it empirically using different types of compressible signals.

Figures

Figures reproduced from arXiv: 2411.09483 by the authors.

Figure 1
Figure 1. A schematic of the sparsity inducing CSVAE. [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. a) and b) nMSE and SSIM over M (Nt = 20000, MNIST), c) and d) nMSE and SSIM over Nt (M = 160, MNIST), e) exemplary reconstructed MNIST images (M = 200, Nt = 20000), f) nMSE over M (SNRdB = 10dB, Nt = 10000, piece-wise smooth fct.), g) nMSE over Nt (SNRdB = 10dB, M = 100, piece-wise smooth fct.), h) exemplary reconstructed piece-wise smooth fct. (M = 100, Nt = 1000), i) nMSE comparison of dictionaries (MNIST, M = 160… view at source ↗
Figure 3
Figure 3. a) and b) nMSE and SSIM over M (Nt = 5000), c) and d) nMSE and SSIM over Nt (M = 1800), e) exemplary reconstructed celebA images (M = 2700, Nt = 5000), f) histogram of h(z|y) for compressed test MNIST images of digits 0,1 and 7, where the CSVAE is trained on compressed zeros, g) training and reconstruction time for MNIST (M = 200, Nt = 20000). Uncertainty quantification. One possibility to determine the model’s unce… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Exemplary signals within the 1D dataset of piece-wise smooth functions. [PITH_FULL_IMAGE:figures/full_fig_p019_4.png]
Figure 6
Figure 6. Figure 6: a) and b) nMSE and SSIM comparison over number Ns of estimations per observation (MNIST, M = 200,Nt = 20000), c)-f) nMSE and SSIM performance of models trained on compressed data (solid curves) as well as models trained on ground-truth data (dashed curves) W and X over…
Figure 7
Figure 7. Figure 7: Exemplary reconstructed MNIST images for [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]
Figure 8
Figure 8. Figure 8: Exemplary reconstructed FashionMNIST images ( [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]
Figure 9
Figure 9. Figure 9: Exemplary reconstructed piece-wise smooth functions ( [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]
Figure 10
Figure 10. Figure 10: Exemplary reconstructed MNIST images (M = 160, Nt = 20000, [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]
Figure 11
Figure 11. Figure 11: Exemplary reconstructed celebA images (M = 2700, Nt = 5000, [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

77 extracted references · 73 canonical work pages

  1. [1]

    Eldar and G

    Y . Eldar and G. Kutyniok, Compressed Sensing: Theory and Applications . Cambridge University Press, 11 2012

    work page 2012

  2. [2]

    Learning fast approximations of sparse coding,

    K. Gregor and Y . LeCun, “Learning fast approximations of sparse coding,” in Proceedings of the 27th International Conference on International Conference on Machine Learning, ser. ICML’10. Madison, WI, USA: Omnipress, 2010, p. 399–406

    work page 2010

  3. [3]

    ALISTA: Analytic weights are as good as learned weights in LISTA,

    J. Liu, X. Chen, Z. Wang, and W. Yin, “ALISTA: Analytic weights are as good as learned weights in LISTA,” inInternational Conference on Learning Representations, 2019

    work page 2019

  4. [4]

    Compressed sensing using generative models,

    A. Bora, A. Jalal, E. Price, and A. G. Dimakis, “Compressed sensing using generative models,” in Proceedings of the 34th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 70. PMLR, Aug 2017, pp. 537–546

    work page 2017

  5. [5]

    Deep image prior,

    D. Ulyanov, A. Vedaldi, and V . Lempitsky, “Deep image prior,” International Journal of Computer Vision, vol. 128, no. 7, p. 1867–1888, Mar. 2020

    work page 2020

  6. [6]

    Deep decoder: Concise image representations from untrained non- convolutional networks,

    R. Heckel and P. Hand, “Deep decoder: Concise image representations from untrained non- convolutional networks,” in7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019

    work page 2019

  7. [7]

    Cryo-care: Content-aware image restoration for cryo-transmission electron microscopy data,

    T.-O. Buchholz, M. Jordan, G. Pigino, and F. Jug, “Cryo-care: Content-aware image restoration for cryo-transmission electron microscopy data,” in 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), 2019, pp. 502–506

    work page 2019

  8. [8]

    Hkf: Hierarchical Kalman filtering with online learned evolution priors for adaptive ECG denoising,

    G. Revach, T. Locher, N. Shlezinger, R. J. G. van Sloun, and R. Vullings, “Hkf: Hierarchical Kalman filtering with online learned evolution priors for adaptive ECG denoising,” IEEE Transactions on Signal Processing, vol. 72, pp. 3990–4006, 2024

    work page 2024

Show all 77 references
  1. [9]

    Towards deep learning-aided wireless channel estimation and channel state information feedback for 6G,

    W. Kim, Y . Ahn, J. Kim, and B. Shim, “Towards deep learning-aided wireless channel estimation and channel state information feedback for 6G,” Journal of Communications and Networks, vol. 25, no. 1, pp. 61–75, 2023

    2023

  2. [10]

    Robust compressed sensing using generative models,

    A. Jalal, L. Liu, A. G. Dimakis, and C. Caramanis, “Robust compressed sensing using generative models,” in Neural Information Processing Systems, 2020

    2020

  3. [11]

    Regression shrinkage and selection via the Lasso,

    R. Tibshirani, “Regression shrinkage and selection via the Lasso,” Journal of the Royal Statisti- cal Society (Series B), vol. 58, pp. 267–288, 1996

    1996

  4. [12]

    Optimal tree approximation with wavelets,

    R. Baraniuk, “Optimal tree approximation with wavelets,” Proceedings of SPIE - The Interna- tional Society for Optical Engineering, vol. 3813, pp. 196–207, Dec. 1999

    1999

  5. [13]

    Exploiting burst-sparsity in massive MIMO with partial channel support information,

    A. Liu, V . K. N. Lau, and W. Dai, “Exploiting burst-sparsity in massive MIMO with partial channel support information,” IEEE Transactions on Wireless Communications, vol. 15, no. 11, pp. 7820–7830, 2016

    2016

  6. [14]

    Tree-structured compressive sensing with variational Bayesian analysis,

    L. He, H. Chen, and L. Carin, “Tree-structured compressive sensing with variational Bayesian analysis,” IEEE Signal Processing Letters, vol. 17, no. 3, pp. 233–236, 2010

    2010

  7. [15]

    Exploiting structure in wavelet-based Bayesian compressive sensing,

    L. He and L. Carin, “Exploiting structure in wavelet-based Bayesian compressive sensing,” IEEE Transactions on Signal Processing, vol. 57, no. 9, pp. 3488–3497, 2009

    2009

  8. [16]

    Model-based compressive sensing,

    R. G. Baraniuk, V . Cevher, M. F. Duarte, and C. Hegde, “Model-based compressive sensing,” IEEE Transactions on Information Theory, vol. 56, no. 4, pp. 1982–2001, 2010. 11

    1982

  9. [17]

    Sparse Bayesian learning and the relevance vector machine,

    M. E. Tipping, “Sparse Bayesian learning and the relevance vector machine,”J. Mach. Learn. Res., vol. 1, p. 211–244, sep 2001

    2001

  10. [18]

    Sparse Bayesian learning for basis selection,

    D. Wipf and B. Rao, “Sparse Bayesian learning for basis selection,” IEEE Transactions on Signal Processing, vol. 52, no. 8, pp. 2153–2164, 2004

    2004

  11. [19]

    Statistical compressed sensing of Gaussian mixture models,

    G. Yu and G. Sapiro, “Statistical compressed sensing of Gaussian mixture models,” IEEE Transactions on Signal Processing, vol. 59, no. 12, pp. 5842–5858, 2011

    2011

  12. [20]

    Reconstruction of signals drawn from a Gaussian mixture via noisy compressive measurements,

    F. Renna, R. Calderbank, L. Carin, and M. R. D. Rodrigues, “Reconstruction of signals drawn from a Gaussian mixture via noisy compressive measurements,” IEEE Transactions on Signal Processing, vol. 62, no. 9, pp. 2265–2277, 2014

    2014

  13. [21]

    Solving inverse problems with piecewise linear estimators: From Gaussian mixture models to structured sparsity,

    G. Yu, G. Sapiro, and S. Mallat, “Solving inverse problems with piecewise linear estimators: From Gaussian mixture models to structured sparsity,”IEEE Transactions on Image Processing, vol. 21, no. 5, pp. 2481–2499, 2012

    2012

  14. [22]

    Compressive sensing by learning a Gaussian mixture model from measurements,

    J. Yang, X. Liao, X. Yuan, P. Llull, D. J. Brady, G. Sapiro, and L. Carin, “Compressive sensing by learning a Gaussian mixture model from measurements,” IEEE Transactions on Image Processing, vol. 24, no. 1, pp. 106–119, 2015

    2015

  15. [23]

    Compressive K-SVD,

    F. Pourkamali Anaraki and S. M. Hughes, “Compressive K-SVD,” in2013 IEEE International Conference on Acoustics, Speech and Signal Processing, 2013, pp. 5469–5473

    2013

  16. [24]

    Memory and computation efficient PCA via very sparse random projections,

    F. P. Anaraki and S. Hughes, “Memory and computation efficient PCA via very sparse random projections,” in Proceedings of the 31st International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 32, no. 2. Bejing, China: PMLR, 22–24 Jun 2014, ...

    2014

  17. [25]

    Efficient dictionary learning via very sparse random projections,

    F. Pourkamali-Anaraki, S. Becker, and S. M. Hughes, “Efficient dictionary learning via very sparse random projections,” in 2015 International Conference on Sampling Theory and Applica- tions (SampTA), 2015, pp. 478–482

    2015

  18. [26]

    Randnet: Deep learning with compressed measurements of images,

    T. Chang, B. Tolooshams, and D. Ba, “Randnet: Deep learning with compressed measurements of images,” in 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP), 2019, pp. 1–6

    2019

  19. [27]

    K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation,

    M. Aharon, M. Elad, and A. Bruckstein, “K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation,” IEEE Transactions on Signal Processing, vol. 54, no. 11, pp. 4311–4322, 2006

    2006

  20. [28]

    AmbientGAN: Generative models from lossy measure- ments,

    A. Bora, E. Price, and A. G. Dimakis, “AmbientGAN: Generative models from lossy measure- ments,” in 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018

    2018

  21. [29]

    Task-aware compressed sensing with generative adversarial networks,

    M. Kabkab, P. Samangouei, and R. Chellappa, “Task-aware compressed sensing with generative adversarial networks,” in AAAI Conference on Artificial Intelligence, 2018

    2018

  22. [30]

    On the asymptotic mean square error optimality of diffusion models,

    B. Fesl, B. Böck, F. Strasser, M. Baur, M. Joham, and W. Utschick, “On the asymptotic mean square error optimality of diffusion models,” 2024, arXiv:2403.02957

    2024 arXiv

  23. [31]

    An asymptotically MSE-optimal estimator based on Gaussian mixture models,

    M. Koller, B. Fesl, N. Turan, and W. Utschick, “An asymptotically MSE-optimal estimator based on Gaussian mixture models,” IEEE Transactions on Signal Processing , vol. 70, pp. 4109–4123, 2022

    2022

  24. [32]

    Leveraging variational autoencoders for parameterized MMSE estimation,

    M. Baur, B. Fesl, and W. Utschick, “Leveraging variational autoencoders for parameterized MMSE estimation,” IEEE Transactions on Signal Processing, vol. 72, pp. 3731–3744, 2024

    2024

  25. [33]

    J. O. Berger, Statistical decision theory and Bayesian analysis. New York: Springer-Verlag, 1985

    1985

  26. [34]

    G. J. McLachlan and D. Peel, Finite mixture models. Wiley Series in Probability and Statistics, 2000. 12

    2000

  27. [35]

    C. M. Bishop, Pattern Recognition and Machine Learning (Information Science and Statistics), 1st ed. Springer, 2007

    2007

  28. [36]

    An introduction to variational autoencoders,

    D. P. Kingma and M. Welling, “An introduction to variational autoencoders,”Foundations and Trends® in Machine Learning, vol. 12, no. 4, p. 307–392, 2019

    2019

  29. [37]

    The perception-distortion tradeoff,

    Y . Blau and T. Michaeli, “The perception-distortion tradeoff,” in2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Jun. 2018

    2018

  30. [38]

    Prior and posterior networks: A survey on evidential deep learning methods for uncertainty estimation,

    D. T. Ulmer, C. Hardmeier, and J. Frellsen, “Prior and posterior networks: A survey on evidential deep learning methods for uncertainty estimation,”Transactions on Machine Learning Research, 2023

    2023

  31. [39]

    Predictive uncertainty estimation via prior networks,

    A. Malinin and M. Gales, “Predictive uncertainty estimation via prior networks,” in Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds., vol. 31. Curran Associates, Inc., 2018

    2018

  32. [40]

    Deep evidential regression,

    A. Amini, W. Schwarting, A. Soleimany, and D. Rus, “Deep evidential regression,”Advances in Neural Information Processing Systems, vol. 33, 2020

    2020

  33. [41]

    Gradient-based learning applied to document recognition,

    Y . Lecun, L. Bottou, Y . Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998

    1998

  34. [42]

    Adapting to unknown smoothness via wavelet shrinkage,

    D. L. Donoho and I. M. Johnstone, “Adapting to unknown smoothness via wavelet shrinkage,” Journal of the American Statistical Association, vol. 90, no. 432, pp. 1200–1224, 1995

    1995

  35. [43]

    Deep learning face attributes in the wild,

    Z. Liu, P. Luo, X. Wang, and X. Tang, “Deep learning face attributes in the wild,” inProceedings of International Conference on Computer Vision (ICCV), December 2015

    2015

  36. [44]

    Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms,

    H. Xiao, K. Rasul, and R. V ollgraf, “Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms,” 2017, arXiv:1708.07747

    2017 arXiv

  37. [45]

    Image quality assessment: from error visibility to structural similarity,

    Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, “Image quality assessment: from error visibility to structural similarity,”IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004

    2004

  38. [46]

    Adam: A method for stochastic optimization,

    D. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International Confer- ence on Learning Representations (ICLR), San Diega, CA, USA, 2015

    2015

  39. [47]

    Pywavelets: A python package for wavelet analysis,

    G. R. Lee, R. Gommers, F. Waselewski, K. Wohlfahrt, and A. O’Leary, “Pywavelets: A python package for wavelet analysis,” Journal of Open Source Software, vol. 4, no. 36, p. 1237, 2019

    2019

  40. [48]

    R. E. Blahut, Principles and practice of information theory. USA: Addison-Wesley Longman Publishing Co., Inc., 1987

    1987

  41. [49]

    Bickel and K

    P. Bickel and K. Doksum,Mathematical Statistics: Basic Ideas and Selected Topics. Chapman and Hall/CRC, 2015

    2015

  42. [50]

    Scikit-learn: Machine learning in Python,

    F. Pedregosa, G. Varoquaux, A. Gramfort, V . Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V . Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, “Scikit-learn: Machine learning in Python,” Journal of Machine Lear...

    2011

  43. [51]

    µϕ(t) (yi), σϕ(t) (yi) Encoder ← − − − −yi

  44. [52]

    draw ˜zi ∼ qϕ(t) (z|yi) =N (z; µϕ(t) (yi), σϕ(t) (yi)) (via reparameterization trick [36])

  45. [53]

    γθ(t) ( ˜zi) Decoder ← − − − −˜zi 4) C y| ˜zi θ(t) ( ˜zi), µs|yi, ˜zi θ(t) ( ˜zi) (52),(50) ← − − − − γθ(t) ( ˜zi), A, D, σ2, yi

  46. [54]

    Epθ(t) (s| ˜zi,yi)[log p(yi|s)] Appendix D,I ← − − − − − − µs|yi, ˜zi θ(t) ( ˜zi), A, D, σ2, yi

  47. [55]

    DKL(qϕ(t) (z|yi)||p(z)) Appendix E ← − − − − − µϕ(t) (yi), σϕ(t) (yi)

  48. [56]

    (42)) Input: observation y, encoder (µϕ(·), σϕ(·)), decoder γθ(·), meas

    DKL(pθ(t) (s| ˜zi, yi)||pθ(t) (s| ˜zi)) Appendix E,I ← − − − − − − C y| ˜zi θ(t) ( ˜zi), µs|yi, ˜zi θ(t) ( ˜zi), γθ(t) ( ˜zi) end for L(CSV AE) (θ(t),ϕ(t)) (15) ← − − {5), 6), 7)}|Ybatch| i=1 θ(t+1), ϕ(t+1) ← Adamt(L(CSV AE) (θ(t),ϕ(t)), λ,θ(t), ϕ(t)) Algorithm 2 CME Approxima...

  49. [57]

    µϕ(y), σϕ(y) Encoder ← − − − −y for i = 1to |Z| do

  50. [58]

    draw ˜zi ∼ qϕ(z|y) =N (z; µϕ(y), σϕ(y))

  51. [59]

    γθ( ˜zi) Decoder ← − − − −˜zi

  52. [60]

    µs|y, ˜zi θ ( ˜zi) (50) ← − − γθ( ˜zi), A, D, σ2, y end for

  53. [61]

    (43)) Input: observation y, encoder (µϕ(·), σϕ(·)), decoder γθ(·), meas

    ˆx∗ CME = D/|Z| P|Z| i=1 µs|y, ˜zi θ ( ˜zi) 23 Algorithm 3 MAP-based Estimator with the CSV AE in the Inference Phase (cf. (43)) Input: observation y, encoder (µϕ(·), σϕ(·)), decoder γθ(·), meas. matrix A, dict. D, noise σ2 Output: MAP-based estimator ˆx∗ MAP

  54. [62]

    µϕ(y) Encoder ← − − − −y

  55. [63]

    γθ(µϕ(y)) Decoder ← − − − −µϕ(y)

  56. [64]

    µ s|y,µϕ(y) θ (µϕ(y)) (50) ← − − γθ(µϕ(y)), A, D, σ2, y

  57. [65]

    matrix A, dict

    ˆx∗ MAP = Dµ s|y,µϕ(y) θ (µϕ(y)) Algorithm 4 One EM Step in the Training Phase of the CSGMM with one fixed A Input: parameters in the tth iteration {γ(t) k , ρ(t) k }K k=1, training set Y, meas. matrix A, dict. D, noise σ2 Output: parameters in the (t + 1)th iteration {γ(t+1) ...

  58. [66]

    C y|k k,t (17) ← − − γ(t) k , D, A, σ2

  59. [67]

    diag C s|y,k k,t (53) ← − − γ(t) k , D, A, σ2 for i = 1to |Y| do

  60. [68]

    pt(k|yi) (Bayes) ← − − − − C y|k k,t , ρ(t) k

  61. [69]

    (44)) Input: observation y, GMM {ρk, γk}K k=1, meas

    µs|yi,k k,t (50) ← − − C y|k k,t , D, A, σ2, yi end for 5) γ(t+1) k , ρ(t+1) k Lemma (3.2) ← − − − − − − n pt(k|yi), µs|yi,k k,t o|Y| i=1 , diag C s|y,k k,t end for Algorithm 5 CME Approximation with the CSGMM in the Inference Phase (cf. (44)) Input: observation y, GMM {ρk, γk...

  62. [71]

    p(k|y) (Bayes) ← − − − C y|k k , ρk

  63. [72]

    µs|y,k k (50) ← − − C y|k k , D, A, σ2, y end for

  64. [73]

    (45)) Input: observation y, GMM {ρk, γk}K k=1, meas

    ˆx∗ CME = D PK k=1 p(k|y)µs|y,k k Algorithm 6 MAP-based Estimator with the CSGMM in the Inference Phase (cf. (45)) Input: observation y, GMM {ρk, γk}K k=1, meas. matrix A, dict. D, noise σ2 Output: MAP-based estimation ˆx∗ MAP for k = 1to K do

  65. [74]

    C y|k k (17) ← − − γk, D, A, σ2

  66. [75]

    p(k|y) (Bayes) ← − − − − C y|k k , ρk end for

  67. [76]

    ˆkMAP = argmax p(k|y)

  68. [77]

    µs|y,ˆkMAP ˆkMAP (50) ← − − C y|ˆkMAP k , D, A, y, σ2

  69. [78]

    ˆx∗ MAP = Dµs|y,ˆkMAP ˆkMAP 24

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.