Sparse Bayesian Learning Using Approximate Message Passing with Unitary Transformation

Man Luo; Qinghua Guo

REVIEW 2 major objections 5 minor 12 references

Sparse Bayesian Learning Using Approximate Message Passing with Unitary Transformation

T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper establishes UTAMP-SBL, a sparse Bayesian learning algorithm built on approximate message passing with unitary transformation, that recovers sparse vectors from difficult measurement matrices with robustness and speed near the…

desk verdict Useful algorithmic paper with a real hole: the Gamma shape update in Eq. (21) is asserted without derivation, and the stress-test note is right that it is not a standard estimator, so the automatic-tuning claim is unsupported as written. read the letter →

arxiv 1908.06296 v1 pith:XIOLBQS4 submitted 2019-08-17 cs.IT math.IT

Man Luo , Qinghua Guo This is my paper · ORCID

classification cs.ITmath.IT

keywords sparseBayesianlearningapproximatemessagepassingunitarytransformationGammahyperpriorcompressedsensingsupport-oracleboundrobustrecoveryexpectationmaximization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes UTAMP-SBL, a sparse Bayesian learning algorithm that runs approximate message passing on a unitarily transformed measurement model. The claim is that the transform removes the usual fragility of AMP-based SBL on difficult matrices, such as ill-conditioned, correlated, non-zero-mean, or low-rank ones, while keeping per-iteration cost low. A Gamma hyperprior over each precision controls sparsity, and its shape parameter is updated by a direct closed-form rule rather than an inner iterative loop. The paper reports that UTAMP-SBL is faster and more robust than the GGAMP-SBL baseline and, in many difficult cases, recovers signals almost as well as the support-oracle MMSE bound. If true, this gives practitioners a single low-complexity SBL algorithm that does not need damping tuning.

What carries the argument

The load-bearing object is the unitary transform $A=U\Lambda V$, which turns the measurement model into $r=U^H y=\Lambda Vx+\omega$ with IID Gaussian noise $\omega$; the transform preserves all information while making the matrix-vector operations in AMP involve the diagonal matrix $\Lambda$, and the paper replaces vector variance updates by scalar averages. The second load-bearing element is the closed-form hyperprior shape update, $\hat{\epsilon}=\tfrac12\sqrt{\log(\tfrac1N\sum_n \hat{\gamma}_n)-\tfrac1N\sum_n\log\hat{\gamma}_n}$ (Eq. (21)), which is asserted in place of the iterative update (20) and is what lets the algorithm tune its sparsity automatically each iteration.

What would settle it

Independent re-implementation using the iterative update (20) from [12] in place of Eq. (21): if the recovery performance and speed advantages disappear, then the closed-form update is the actual source of gains; if they persist, then the closed-form update is not the key and the robustness claim needs re-examination.

Watch

Extended reading notes

Core claim

The central claim is that replacing the standard AMP-SBL message-passing on $y=Ax+w$ with UTAMP on the unitarily transformed model $r=U^H y=\Lambda Vx+\omega$ produces an SBL algorithm that converges where AMP-SBL diverges and does so quickly. The paper derives UTAMP-SBL by splitting the factor graph into three subgraphs, using UTAMP for the middle dense part and mean-field updates for the noise precision $\lambda$ and the per-coordinate precisions $\gamma_n$. The shape parameter $\epsilon$ of the Gamma hyperprior is updated automatically via the closed-form expression in Eq. (21), avoiding the iterative conjugate-prior update of Eq. (20). Numerical experiments over ill-conditioned, correlated, non-zero-mean, and low-rank matrices show UTAMP-SBL approaching the support-oracle bound in regimes where GGAMP-SBL degrades; runtime is also much lower because no damping or inner EM loop is needed.

Load-bearing premise

The algorithm's automatic tuning of the sparsity-strength parameter relies on a closed-form update that the paper asserts without derivation; if that update does not estimate what it should, the claimed gains lose their basis.

Editorial extensions

If this is right

UTAMP-SBL can be applied directly to difficult measurement matrices without damping-factor tuning, removing a free parameter that slows AMP-based SBL.
On ill-conditioned, correlated, non-zero-mean, and low-rank matrices, UTAMP-SBL is reported to achieve substantially lower NMSE than GGAMP-SBL, often within a few dB of the support-oracle bound.
Because the shape parameter is updated in closed form and there are no inner iterations, the per-iteration and total runtime advantages scale to larger problem sizes.
The same factor-graph derivation can be extended to proper complex signals by removing the factor of 2 in the precision update, as noted in the paper.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If Eq. (21) is eventually derived from a variational or moment-matching argument, UTAMP-SBL may acquire a convergence guarantee analogous to state evolution; the paper itself does not provide one.
The scalar-variance averaging in UTAMP probably acts as a form of denoising or regularization that damps the error feedback that destabilizes AMP; an ablation restoring the full vector variance would test whether that averaging is the source of robustness.
Because the unitary transform is computed once via SVD, the method's cost for very large N depends on the availability of fast unitary transforms; for structured matrices a fast transform would make UTAMP-SBL scale to imaging or massive-MIMO problems.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

I read Luo and Guo's UTAMP-SBL paper. It's a decent algorithmic paper with a real hole in the middle.

What's new: they combine UTAMP with SBL and add a closed-form update for the Gamma hyperprior shape. The combination isn't in the cited literature, and the simulations under ill-conditioned, correlated, non-zero mean, and low-rank matrices show UTAMP-SBL consistently beating the GGAMP-SBL baseline, often getting close to the support-oracle bound. That's a credible empirical win, and the runtime gain is meaningful.

What's good: the derivation of the message passing is standard and readable; the algorithm is simple; the comparisons are fair enough, though I'd like to see error bars and code.

Now the soft spot: Eq. (21), the shape update, is asserted as 'simple but more effective' with no derivation or analysis. The stress-test note is on target: this isn't the MLE for a Gamma shape, nor the MF fixed point from Eq. (18). If the gamma samples were Gamma, the MLE would solve log(eps)-psi(eps)=d, with eps approximately 1/(2d) for large eps. Eq. (21) gives eps=0.5*sqrt(d), which is off by a wide margin for typical eps. So the paper's claim that the shape is 'tuned automatically' and that this contributes to robustness is unsupported as written. The algorithm might still work; the empirical results suggest it does. But the paper should either derive (21), or show by ablation that this update is what's driving the gains over a fixed shape or the iterative update (20). As it stands, the central novel component is an unexplained heuristic.

Also, no error bars or code, so I can't fully trust the magnitude of the NMSE differences. The GGAMP-SBL baseline uses 3x true noise variance in one variant, which helps, but that was suggested by [10]; still, the comparison is a bit favorable.

Bottom line: this paper is worth a serious referee. The core idea—putting SBL on top of UTAMP—is plausible and the simulations support it. The shape update needs either a derivation or an ablation. If the authors can provide that, it would be a solid contribution to the AMP-SBL literature. Otherwise it's a heuristic that works but doesn't provide much insight.

Referee Report

2 major / 5 minor

Summary. The paper proposes UTAMP-SBL, a sparse Bayesian learning algorithm that uses the unitary-transformed approximate message passing (UTAMP) algorithm to implement the E-step, with a Gamma hyperprior on the precisions whose shape parameter is tuned automatically. The algorithm is derived via mean-field message passing on a factor graph and consists of a single loop with no inner iterations. Numerical experiments on ill-conditioned, correlated, non-zero-mean, and low-rank measurement matrices compare UTAMP-SBL with GGAMP-SBL and a support-oracle bound, reporting better NMSE and runtime.

Significance. If the empirical claims hold, UTAMP-SBL would be a practically attractive SBL method for difficult measurement matrices, combining the robustness of UTAMP with automatic hyperparameter tuning. The derivation of the message-passing updates is mostly standard, and the idea of using a unitary transform to stabilize AMP for SBL is well motivated. However, the contribution's key new ingredient — the closed-form shape update in Eq. (21) — is asserted without derivation or analysis, and the numerical evidence lacks basic statistical reporting. The significance is therefore conditional on filling these gaps. The paper does provide a self-contained algorithm specification, but no code or data are made available.

major comments (2)

[III-B, Eq. (21)] The update \hat\epsilon = (1/2) sqrt( log( (1/N) \sum_n \hat\gamma_n ) - (1/N) \sum_n \log \hat\gamma_n ) is introduced as 'simple but more effective' without any derivation, relation to a statistical estimator, or comparison with the iterative mode update in Eq. (20). Since \hat\epsilon enters Line 12 of Algorithm 3 multiplicatively and controls the shrinkage of every \hat\gamma_n, the paper's central claim that the shape parameter is 'tuned automatically' and that this tuning improves robustness and speed is unsupported. If \hat\gamma_n were i.i.d. Ga(\epsilon,\eta), the MLE would solve log \epsilon - \psi(\epsilon) = mean(log \hat\gamma) - log(mean \hat\gamma), which is not equivalent to Eq. (21) for finite or asymptotic \epsilon. The authors should either derive Eq. (21) from a well-defined criterion (e.g., moment matching, variational bound, or an approximation to the mode condition) or provide an ablation study demonstrating the performance contribution of Eq. (21) relative to Eq. (20).
[IV, Figs. 2-6] The numerical evaluation does not report the number T of Monte Carlo trials, does not show error bars or confidence intervals, and the runtime comparison in Fig. 6 lacks any measure of variability. Since the paper's claims of 'remarkably better performance' and 'much more robust' rest entirely on these simulations, the evidence is incomplete as presented. Please specify T, report error bars (e.g., standard deviation over trials), state whether the reported runtimes include the SVD/computation of the unitary transform, and give the exact parameter settings for all algorithms so that the experiments are reproducible.

minor comments (5)

[III, factor graph definition] In the description of the factor graph, the hyperprior is defined as f_{\gamma_n}(\gamma_n) = Ga(\lambda|\epsilon, \eta); the argument should be \gamma_n, not \lambda.
[III-B, Eq. (19)] Eq. (19) writes m_{f_{\gamma_n} \to \gamma_n}(\gamma_n) \propto N(\gamma_n|\hat\epsilon, \eta), but the message from a Gamma prior should be a Gamma density, not a Gaussian; this typo obscures the derivation of the belief in Eq. (17).
[III-B, Eq. (20)] The expression 'log( \epsilon N \sum_n \hat\gamma_n )' in Eq. (20) is ambiguous and likely missing a division slash; it should read something like log( \epsilon N / \sum_n \hat\gamma_n ).
[Algorithm 3] The stopping criterion uses the threshold \delta_x, but \delta_x is never defined in the text; please specify its default value.
[IV, complexity] The paper claims low complexity but does not analyze the complexity of the initial SVD step required by UTAMP; the one-time cost of the unitary transform should be stated and discussed, especially for large-scale problems.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; the empirical UTAMP-SBL claims are tested against external benchmarks, while the unsupported Eq. (21) shape update is a support gap, not a circular step.

full rationale

The paper is largely self-contained: Section II derives the UTAMP recursions from the vector-step AMP by explicit variance-averaging approximations, and the UTAMP-SBL message updates in Section III follow from standard MF/EM calculations. The claimed performance gains are supported by external Monte Carlo benchmarks against GGAMP-SBL and the support-oracle bound, so the central result is not equivalent to a fitted parameter by construction. The only self-citation is the convergence/robustness statement for the base UTAMP algorithm ('it converges for any matrix A in the case of Gaussian priors [6]'), which is the same corresponding author's prior arXiv preprint; while this is a premise for choosing UTAMP, the present paper's empirical robustness claims are independently demonstrated, so the self-citation is not load-bearing enough to make the derivation circular. The main weakness is Eq. (21), introduced as 'simple but more effective' with no derivation and no check that it estimates the Gamma shape parameter; this is a missing-support/correctness concern rather than a circular reduction, since the update does not encode the benchmark performance. No equation in the paper reduces to its own input by construction, and no fitted quantity is renamed as a prediction.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The algorithm rests on standard MF message passing, the author's prior UTAMP convergence result, and an unproven closed-form update for the Gamma shape parameter. No new physical entities are introduced.

free parameters (1)

Initial hyperprior shape epsilon_0 = 0.001
Hand-chosen in Algorithm 3 initialization; not justified by prior. Likely low impact as it is adapted.

assumptions (4)

domain assumption UTAMP converges for any matrix A with Gaussian priors
Used as the basis for robustness claim; cited to [6] (author's own prior work), no proof in this paper.
domain assumption The mean field (MF) approximation is valid for the factor graph in Figure 1
Used throughout Section III to compute beliefs; standard but unverified here.
ad hoc to paper The heuristic epsilon update in Eq. (21) provides a good estimate of the shape parameter
The paper states 'we find the following simple but more effective equation' without derivation; it replaces the iterative update (20) from [12].
ad hoc to paper Replacing Diag(tau_x) with a scaled identity matrix (averaging) preserves performance
Used in UTAMP derivation (Section II) to simplify computation; this approximation is what makes UTAMP robust, but no error bound is given.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Sparse Bayesian Learning Using Approximate Message Passing with Unitary Transformation." pith.science (2026). https://pith.science/paper/XIOLBQS4

@misc{pith2026190806296,
  author       = {Pith},
  title        = {Pith review of: Sparse Bayesian Learning Using Approximate Message Passing with Unitary Transformation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XIOLBQS4}},
  note         = {Machine review of arXiv:1908.06296}
}

read the original abstract

Sparse Bayesian learning (SBL) can be implemented with low complexity based on the approximate message passing (AMP) algorithm. However, it is vulnerable to `difficult' measurement matrices as AMP can easily diverge. Damped AMP has been used to alleviate the problem at the cost of slowing the convergence speed. In this work, we propose an SBL algorithm based on the AMP with unitary transformation (UTAMP), where the shape parameter of the hyperprior is tuned automatically. It is shown that, compared to the state-of-the-art AMP based SBL algorithm, the proposed UTAMP-SBL is much more robust and much faster, leading to remarkably better performance. It is shown that in many cases, UTAMP-SBL can approach the support-oracle bound closely.

Figures

Figures reproduced from arXiv: 1908.06296 by the authors.

**Figure 1.** Factor graph for deriving UTAMP-SBL. A. Message Computations in Subgraph 1 1) Backward Message Passing: According to the derivation of AMP using loopy belief propagation, UTAMP provides the message mhm→frm (hm) = N (hm|pm, τpm ) where the mean pm and the variance τpm are given in Lines 1 and 2 of the UTAMP algorithm (Algorithm 2), which are also Lines 1 and 2 in the UTAMP-SBL algorithm (Algorithm 3). The mean field … view at source ↗

**Figure 2.** Performance under ill-conditioned matrices. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 6.** Runtime under ill-conditioned matrices ( [PITH_FULL_IMAGE:figures/full_fig_p005_6.png] view at source ↗

Figures from the paper (1 more)

**Figure 5.** Figure 5: Performance under low rank matrices (ρ = 0.1). the performance of the algorithms with the following different types of measurement matrices. 1) Ill-Conditioned Matrix: Matrix A is constructed based on the SVD A = UΛV where Λ is a singular value matrix with Λi,i/Λi+1,i+…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 11 canonical work pages

[10]

A gamp-bas ed low complexity sparse bayesian learning algorithm,

M. Al-Shoukairi, P . Schniter, and B. D. Rao, “A gamp-bas ed low complexity sparse bayesian learning algorithm,” IEEE Transactions on Signal Processing, vol. 66, no. 2, pp. 294–308, 2018

work page 2018
[1]

Message-pass ing algo- rithms for compressed sensing,

D. L. Donoho, A. Maleki, and A. Montanari, “Message-pass ing algo- rithms for compressed sensing,” Proceedings of the National Academy of Sciences , vol. 106, no. 45, pp. 18 914–18 919, 2009

work page 2009
[2]

State evolution for gene ral approxi- mate message passing algorithms, with applications to spat ial coupling,

A. Javanmard and A. Montanari, “State evolution for gene ral approxi- mate message passing algorithms, with applications to spat ial coupling,” Information and Inference: A Journal of the IMA , vol. 2, no. 2, pp. 115– 144, 2013

work page 2013
[3]

On the convergence of approximate message passing with arbitrary matrices,

S. Rangana, P . Schniterb, A. K. Fletcherc, and S. Sarkar, “On the convergence of approximate message passing with arbitrary matrices,” IEEE Transactions on Information Theory , 2019

work page 2019
[4]

Sparse estimation with the swept approximated message-passing al gorithm,

A. Manoel, F. Krzakala, E. W. Tramel, and L. Zdeborov´ a, “ Sparse estimation with the swept approximated message-passing al gorithm,” arXiv preprint arXiv:1406.4311 , 2014

arXiv 2014
[5]

Adaptive damping and mean removal for the generalized approximate me ssage passing algorithm,

J. Vila, P . Schniter, S. Rangan, F. Krzakala, and L. Zdebo rov´ a, “Adaptive damping and mean removal for the generalized approximate me ssage passing algorithm,” in 2015 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP) . IEEE, 2015, pp. 2021– 2025

work page 2015
[6]

Approximate Message Passing with Unitary Transformation

Q. Guo and J. Xi, “Approximate message passing with unita ry transfor- mation,” arXiv preprint arXiv:1504.04799 , 2015

work page Pith review arXiv 2015
[7]

V ector appro ximate message passing,

S. Rangan, P . Schniter, and A. K. Fletcher, “V ector appro ximate message passing,” in 2017 IEEE International Symposium on Information Theory (ISIT). IEEE, 2017, pp. 1588–1592

work page 2017

Show all 12 references

[8]

Orthogonal amp,

J. Ma and L. Ping, “Orthogonal amp,” IEEE Access , vol. 5, pp. 2020– 2033, 2017

2020
[9]

Sparse bayesian learning us ing approxi- mate message passing,

M. Al-Shoukairi and B. Rao, “Sparse bayesian learning us ing approxi- mate message passing,” in 2014 48th Asilomar Conference on Signals, Systems and Computers . IEEE, 2014, pp. 1957–1961

2014
[11]

Iterat ive frequency domain equalization with generalized approxima te message passing,

Q. Guo, D. D. Huang, S. Nordholm, J. Xi, and Y . Y u, “Iterat ive frequency domain equalization with generalized approxima te message passing,” IEEE Signal Processing Letters , vol. 20, no. 6, pp. 559–562, 2013

2013
[12]

A compendium of conjugate priors,

D. Fink, “A compendium of conjugate priors,” See http://www. people. cornell. edu/pages/df36/CONJINTRnew% 20TEX. pdf , vol. 46, 1997

1997

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [10]

A gamp-bas ed low complexity sparse bayesian learning algorithm,

M. Al-Shoukairi, P . Schniter, and B. D. Rao, “A gamp-bas ed low complexity sparse bayesian learning algorithm,” IEEE Transactions on Signal Processing, vol. 66, no. 2, pp. 294–308, 2018

work page 2018

[2] [1]

Message-pass ing algo- rithms for compressed sensing,

D. L. Donoho, A. Maleki, and A. Montanari, “Message-pass ing algo- rithms for compressed sensing,” Proceedings of the National Academy of Sciences , vol. 106, no. 45, pp. 18 914–18 919, 2009

work page 2009

[3] [2]

State evolution for gene ral approxi- mate message passing algorithms, with applications to spat ial coupling,

A. Javanmard and A. Montanari, “State evolution for gene ral approxi- mate message passing algorithms, with applications to spat ial coupling,” Information and Inference: A Journal of the IMA , vol. 2, no. 2, pp. 115– 144, 2013

work page 2013

[4] [3]

On the convergence of approximate message passing with arbitrary matrices,

S. Rangana, P . Schniterb, A. K. Fletcherc, and S. Sarkar, “On the convergence of approximate message passing with arbitrary matrices,” IEEE Transactions on Information Theory , 2019

work page 2019

[5] [4]

Sparse estimation with the swept approximated message-passing al gorithm,

A. Manoel, F. Krzakala, E. W. Tramel, and L. Zdeborov´ a, “ Sparse estimation with the swept approximated message-passing al gorithm,” arXiv preprint arXiv:1406.4311 , 2014

arXiv 2014

[6] [5]

Adaptive damping and mean removal for the generalized approximate me ssage passing algorithm,

J. Vila, P . Schniter, S. Rangan, F. Krzakala, and L. Zdebo rov´ a, “Adaptive damping and mean removal for the generalized approximate me ssage passing algorithm,” in 2015 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP) . IEEE, 2015, pp. 2021– 2025

work page 2015

[7] [6]

Approximate Message Passing with Unitary Transformation

Q. Guo and J. Xi, “Approximate message passing with unita ry transfor- mation,” arXiv preprint arXiv:1504.04799 , 2015

work page Pith review arXiv 2015

[8] [7]

V ector appro ximate message passing,

S. Rangan, P . Schniter, and A. K. Fletcher, “V ector appro ximate message passing,” in 2017 IEEE International Symposium on Information Theory (ISIT). IEEE, 2017, pp. 1588–1592

work page 2017