REVIEW 3 major objections 4 minor 21 references
Diffusion probabilistic LMS algorithm
T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read The paper claims that a distributed estimation algorithm whose step size is derived from a Gaussian posterior approximation is more robust to input-signal statistics and impulsive interference than three established diffusion LMS variants.
desk verdict A plausible PLMS-plus-diffusion recipe, but the published algorithm has a sign error in its variance recursion that makes it undefined, and the simulations are not reproducible. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing device is the time-varying step size $\alpha_n(i) = \frac{\sigma_n^2(i-1) + \sigma_\rho^2}{[\sigma_n^2(i-1) - \sigma_\rho^2]\|\mathbf{X}_n(i)\|^2 + \sigma_{\varepsilon,n}^2}$ together with the variance recursion $\sigma_n^2(i) = \left[1 - \frac{\alpha_n(i)\|\mathbf{X}_n(i)\|^2}{L}\right][\sigma_n^2(i-1) - \sigma_\rho^2]$. This $\alpha_n(i)$ is the PLMS step size obtained by approximating the posterior $p(\mathbf{W}_o(i)|\mathcal{Z}_i)$ as an isotropic Gaussian and taking the MAP estimate; it automatically down-weights large-input-power updates and nodes with high noise variance. Combining this update through the adapt-then-combine diffusion step is what produces the claimed robustness.
What would settle it
Simulate a two-node network with strongly correlated inputs or a node whose step size is driven by large past errors, compute the Monte Carlo average of $E[\mathbf{S}(i)\mathbf{R}(i)\hat{\mathbf{W}}(i-1)]$ and compare it with $E[\mathbf{S}(i)\mathbf{R}(i)]E[\hat{\mathbf{W}}(i-1)]$; if the difference is large and the mean weight error grows even when $\mu$ satisfies Eq. (22), the independence assumption is what failed.
Extended reading notes
Core claim
The central claim is that replacing the fixed step size that appears in existing diffusion LMS variants with the PLMS variable step size—defined from the variance of a Gaussian posterior—yields a distributed estimator that converges faster and reaches lower steady-state misalignment under both correlated input signals and impulsive interference. The paper argues this by constructing the DPLMS recursion, proving a sufficient mean-stability condition, and showing in Monte Carlo simulations that DPLMS outperforms DSE-LMS, DRVSSLMS, and DLLAD across several combinations of impulse probability and intensity, with equal or lower computational cost.
Load-bearing premise
The mean-stability proof assumes the matrix product $\mathbf{S}(i)\mathbf{R}(i)$ is independent of the current weight-error vector $\hat{\mathbf{W}}(i-1)$, even though the step-size factors in $\mathbf{S}(i)$ depend on input power and past estimation errors.
Editorial extensions
If this is right
- If DPLMS is correct, distributed estimators can be made resistant to impulsive noise without sign operations or nonlinear clipping; the adaptation is achieved entirely by the Bayesian step-size schedule.
- The mean-stability condition in Eq. (22) gives users an explicit step-size bound $\mu < 2/\rho_{\max}(\sum_l \alpha_l(i) c_{l,l} \mathbf{R}_{xx,l}(i))$, so deployment can start from a principled range rather than trial-and-error.
- Because its per-iteration complexity matches DLLAD and is lower than DSE-LMS and DRVSSLMS, the claimed robustness comes without added computational burden.
- The construction suggests that posterior-approximation techniques from Bayesian filtering can be transplanted into diffusion networks one agent at a time.
Reading between the lines
- The same probabilistic construction likely extends to other likelihood models: replacing the Gaussian likelihood with a heavier-tailed one (e.g., Student-t or Laplace) could give even stronger resistance to extreme impulses, since the Gaussian posterior is the current source of the quadratic error term.
- A direct consequence not tested here is that DPLMS should track time-varying systems better than fixed-step-size algorithms when the unknown vector drifts, because the step size already incorporates the process-noise variance $\sigma_\rho^2$ as an explicit parameter.
- The load-bearing independence assumption in the mean analysis suggests a testable refinement: measure the correlation between $\mathbf{S}(i)\mathbf{R}(i)$ and $\hat{\mathbf{W}}(i-1)$ in simulation; if non-negligible, the stability bound may need to be corrected for correlated inputs.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a distributed adaptive filtering algorithm, diffusion probabilistic LMS (DPLMS), which combines the adapt-then-combine diffusion strategy with the probabilistic least-mean-squares (PLMS) algorithm. The authors derive the local update from a Gaussian posterior approximation, introduce a constant parameter tau, and present a network-level two-step adaptation-combination scheme. The paper claims that DPLMS is more robust to input signal characteristics and impulsive interference than DSE-LMS, DRVSSLMS, and DLLAD, and it provides a mean-stability analysis and a computational-complexity comparison. The empirical section reports network MSD curves for several impulsive-noise scenarios with 60 Monte Carlo runs.
Significance. If the proposed algorithm worked as claimed, it would be a useful addition to distributed estimation under impulsive noise, with a step-size mechanism derived from a probabilistic model and with complexity equal to that of DLLAD and lower than that of DRVSSLMS. The paper's combination of a probabilistic step-size with diffusion is an interesting idea, and the complexity table is a useful summary. However, the current manuscript contains a fundamental error in the variance recursion that defines the algorithm, and the mean-stability analysis relies on an unjustified independence assumption. As a result, the algorithm as published is not well-defined for the reported simulations, and the main theoretical guarantee is not established. The empirical robustness claim is also not supported by the reported evidence because essential parameter values and statistical error information are missing. The paper would require substantial correction and re-analysis before its central claims can be accepted.
major comments (3)
- [Section 2.1, Eq. (6), Eq. (9), and Table 1] The variance recursion contains a sign error that makes the published algorithm undefined during the reported simulations. Under the state model in Eq. (2), the predictive variance of W_o(i) given Z^{i-1} is sigma^2(i-1) + sigma_rho^2, not sigma^2(i-1) - sigma_rho^2. This minus sign is then used in Eq. (9) and in Table 1's alpha_n(i) and sigma_n^2(i) update. The iteration sigma_n^2(i) = c_n(i) [sigma_n^2(i-1) - sigma_rho,n^2] with 0 < c_n(i) < 1 has only the negative fixed point -c_n(i) sigma_rho,n^2 / (1 - c_n(i)), so sigma_n^2(i) becomes negative in finite time from any positive initial value. Once sigma_n^2(i) is negative, alpha_n(i) is no longer a valid step size and the update in Eq. (16) is undefined or diverging. The paper never states sigma_rho,n^2, sigma_epsilon,n^2, the initial sigma_n^2(0), or any clipping floor, so the algorithm that produced the MSD curves in Section 4 cannot be reproduced. This error must be corrected and the simulations re-run.
- [Section 3.1, Eqs. (21) and (22)] The mean-stability derivation is not valid as written. The step E{A^T [I - S(i)R(i)] W~(i-1)} = A^T [I - E{S(i)R(i)}] E{W~(i-1)} in Eq. (21) requires S(i)R(i) to be independent of W~(i-1). But alpha_n(i) depends on sigma_n^2(i-1), which is a deterministic function of past regressors through the variance recursion in Table 1, and W~(i-1) is correlated with those past regressors. Therefore the independence assumption is unjustified, and the stability condition in Eq. (22) does not follow. Additionally, Eq. (22) still contains the stochastic alpha_l(i) inside the bound, so it is not a checkable step-size condition without further expectation or concentration arguments. The paper's only theoretical guarantee for DPLMS is therefore unsupported.
- [Section 4] The central robustness claim is not supported by the reported evidence. The simulations do not list the values of sigma_rho,n^2, sigma_epsilon,n^2, the initial sigma_n^2(0), or the constant tau, all of which are needed to run DPLMS. The MSD curves are averaged over 60 runs, but no error bars, confidence intervals, or run-to-run spread are shown, so the claim that DPLMS is more robust than DSE-LMS, DRVSSLMS, and DLLAD rests on visual inspection of single curves. The text in Experiments 1 and 2 also refers to 'DNLMS' where DPLMS is presumably intended, further complicating interpretation. These omissions are load-bearing because the empirical comparison is the main evidence for the paper's headline claim.
minor comments (4)
- [Section 2.2, Eq. (15)] The 'global cost function' in Eq. (15) is not well defined: it places argmax_{W_o(i)} p(W_o(i)|Z_i) inside a sum of local costs and is never used to derive the adaptation and combination updates in Eqs. (16) and (17). Please rewrite or remove this part to avoid implying that DPLMS minimizes a well-defined global objective.
- [Section 2.1, Eq. (7)] Equation (7) omits the Bayes normalization and confuses the predictive and posterior distributions. A cleaner derivation would state the exact Gaussian posterior before the isotropic approximation and then apply the trace-based variance update.
- [Section 2.1 and Section 3.1] The notation is inconsistent: the noise variance is called sigma_p^2 in Eq. (2) and sigma_rho^2 afterward, and the matrix definitions in Section 3.1 contain garbled expressions (e.g., 'M C C I' and the definition of O(i)). These should be corrected for readability.
- [Table 2] The complexity comparison would be clearer if the entries for DRVSSLMS were explained: the '>' markers and the roles of the sgn and absolute-value columns are not defined, and the total counts should be consistent with the equations cited.
Circularity Check
No circularity: the DPLMS derivation is a generative model plus a standard theorem application; the main problems are sign/reproducibility errors, not circular reductions.
full rationale
The paper proposes DPLMS by combining the diffusion adaptation/combination strategy with the PLMS variable step-size. The PLMS step-size is generated from a stated Gaussian posterior approximation (Eqs. (2)-(10)); it is not fitted to the MSD curves or to the robustness claim, so the robustness result is not an input renamed as a prediction. The mean-stability analysis in Section 3.1 follows the standard diffusion-LMS pattern: after writing the error recursion, Eq. (22) is the usual stability bound with the local variable step-size absorbed into S(i). Restating a known stability condition for a special step-size choice is a theorem application, not a circular derivation. The only self-citations, Ref. [15], are used for a standard zero-mean Gaussian regressor assumption and for a Bernoulli-Gaussian impulsive-noise construction; neither presupposes DPLMS's performance, so they are not load-bearing. There are genuine correctness concerns: the predictive variance in Eq. (6) should be sigma^2(i-1)+sigma_rho^2 rather than the printed minus sign; the resulting Table 1 variance recursion can become negative; the simulation leaves sigma_rho^2 and initial sigma^2 unspecified; and Eq. (21) assumes independence between S(i)R(i) and W-hat(i-1). These are technical or reproducibility defects, not cases in which an output reduces by construction to an input, so they do not raise the circularity score.
Assumptions & free parameters
free parameters (5)
- Step size μ =
0.6 (Experiment 1), 0.4 (Experiments 2 and 3)
- Constant τ =
unspecified
- Measurement noise variance σ_ε² =
unspecified
- Random walk variance σ_ρ² =
unspecified
- Initial variance σ²(0) =
unspecified
assumptions (4)
- domain assumption Unknown coefficient vector follows a random-walk model with zero-mean white Gaussian increments (Eq. 1-2).
- ad hoc to paper The posterior distribution is approximated as an isotropic Gaussian (Eq. 5).
- domain assumption Measurement noise and regressors are independent (Assumption 2) and regressors are zero-mean Gaussian, temporally white, spatially independent (Assumption 1).
- domain assumption The factorization E[S(i)R(i)Ŵ(i-1)] = E[S(i)R(i)] E[Ŵ(i-1)] in Eq. (21)
Cite this review
Pith. "Pith review of Diffusion probabilistic LMS algorithm." pith.science (2026). https://pith.science/paper/24UJKLDH
@misc{pith2026190809730,
author = {Pith},
title = {Pith review of: Diffusion probabilistic LMS algorithm},
year = {2026},
howpublished = {\url{https://pith.science/paper/24UJKLDH}},
note = {Machine review of arXiv:1908.09730}
}
read the original abstract
In this paper, a novel diffusion estimation algorithm is proposed from a probabilistic perspective by combining diffusion strategy and the probabilistic least-mean-squares (PLMS) at all agents. The proposed method diffusion probabilistic LMS (DPLMS) is more robust to input signal and impulsive interference than the DSE-LMS, DRVSSLMS and DLLAD algorithms. Instead of minimizing the estimate error, the DPLMS algorithm is derived from approximating the posterior distribution with an isotropic Gaussian distribution. The stability of mean performance and computational complexity are analyzed theoretically. Results from the simulation indicate that the DPLMS algorithm is more robust to input signal and impulsive interference than the DSE-LMS, DRVSSLMS and DLLAD algorithms. These results suggest that the DPLMS algorithm can perform better in identifying the unknown coefficients under the complex and changeable impulsive interference environments.
Reference graph
Works this paper leans on
-
[1]
C.G. Lopes, A.H. Sayed, Diffusion least -mean squares over adaptive networks: formulation and performance analysis [J]. IEEE Trans actions Signal Processing, 2008, 56(7):3122-3136
work page 2008
-
[2]
F.S. Cattivelli, A.H. Sayed, Diffusion LMS strategies for distributed estimation [J]. IEEE Transactions Signal Processing, 2010, 58(3):1035-1048
work page 2010
-
[3]
Tu S Y , Sayed A H. Diffusion strategies outperform consensus strategies for distributed estimation over adaptive networks [J]. IEEE Transactions on Signal Processing, 2012, 60(12):6217-6234
work page 2012
-
[4]
Sparse distributed learning based on diffusion adaptation [J]
Lorenzo P D, Sayed A H. Sparse distributed learning based on diffusion adaptation [J]. IEEE Transactions on Signal Processing, 2013, 61(6):1419-1433
work page 2013
-
[5]
A. H. Sayed, Adaptive networks [J], Proceedings of the IEEE , 2014, 102(4):460-497
work page 2014
-
[6]
Diffusion LMS over multitask networks [J]
Jie C, Richard C, Sayed A H. Diffusion LMS over multitask networks [J]. IEEE Transactions on Signal Processing, 2015, 63(11):2733-2748
work page 2015
-
[7]
z 2-proportionate diffusion LMS algorithm with mean square performance analysis [J]
Lee H S, Yim S H, Song W J. z 2-proportionate diffusion LMS algorithm with mean square performance analysis [J]. Signal Processing, 2017, 131:154-160
work page 2017
-
[8]
Zhang S, So H C, Mi W, et al. A family of adaptive decorrelation NLMS algorithms and its diffusion version over adaptive networks [J]. IEEE Transactions on Circuits & Systems I Regular Papers, 2017, PP (99):1-12
work page 2017
Show all 21 references
-
[9]
Cattivelli, C.G
F.S. Cattivelli, C.G. Lopes, A.H. Sayed, Diffusion recursive least -squares for distributed estimation over adaptive networks [J]. IEEE Trans actions Signal Processing, 2008, 56(5):1865-1877
2008
-
[10]
Diffusion sparse least-mean squares over networks [J]
Liu Y , Li C, Zhang Z. Diffusion sparse least-mean squares over networks [J]. IEEE Transactions Signal Processing, 2012, 60(8):4480-4485
2012
-
[11]
Diffusion least -mean p-power algorithms for distributed estimation in alpha-stable noise environments [J]
Wen F. Diffusion least -mean p-power algorithms for distributed estimation in alpha-stable noise environments [J]. Electron ics Letters. 2013, 49(21): 1355-1356
2013
-
[12]
Diffusion sign subband adaptive filtering algorithm for distributed estimation [J]
Ni J. Diffusion sign subband adaptive filtering algorithm for distributed estimation [J]. IEEE Signal Processing Letters, 2015, 22(11):2029-2033
2015
-
[13]
Diffusion sign -error LMS algorithm: Formulation and stochastic behavior analysis [J]
Ni J, Chen J, Chen X. Diffusion sign -error LMS algorithm: Formulation and stochastic behavior analysis [J]. Signal Processing, 2016, 128:142-149
2016
-
[14]
Variable step-size diffusion normalized sign-error algorithm [J]
Wen P, Zhang J. Variable step-size diffusion normalized sign-error algorithm [J]. Circuits Systems and Signal Processing, 2018, 37(20):4993-5004
2018
-
[15]
Diffusion normalized Huber adaptive filtering algorithm [J]
Zhi L, Guan S. Diffusion normalized Huber adaptive filtering algorithm [J]. Journal of the Franklin Institute, 2018, 355(8):3812-3825
2018
-
[16]
Diffusion robust variable step -size LMS algorithm over distributed networks [J]
Huang W, Li L, Li Q, et al. Diffusion robust variable step -size LMS algorithm over distributed networks [J]. IEEE Access, 2018, 6:47511-47520
2018
-
[17]
Diffusion least logarithmic absolute difference algorithm for distributed estimation [J]
Feng Chen, Tao Shi, et al. Diffusion least logarithmic absolute difference algorithm for distributed estimation [J]. Signal Processing, 2018, 142:423-430
2018
-
[18]
Robust diffusion LMS over adaptive networks [J]
Soheila Ashkezari-Toussi, Hadi Sadoghi-Yazdi. Robust diffusion LMS over adaptive networks [J]. Signal Processing, 2019, 158:201-209
2019
-
[19]
Steady -state and stability analyses of diffusion sign-error LMS algorithm [J]
Gao Y , Ni J, Chen J , et al. Steady -state and stability analyses of diffusion sign-error LMS algorithm [J]. Signal Processing, 2018, 149:62-67
2018
-
[20]
Fernandez -Bes, V
J. Fernandez -Bes, V . Elvira, et al. A probabilistic least -mean-squares filter [C]. IEEE International Conference on Acoustics, Speech and Signal Processing , Brisbane, QLD, Australia, Aug. 2015, pp.2199-2203
2015
-
[21]
Mean-square-deviation analysis of probabilistic LMS algorithm [J]
Fuyi Huang, Jiashu Zhang, Sheng Zhang. Mean-square-deviation analysis of probabilistic LMS algorithm [J]. Digital Signal Processing, 2019, 92:26-35
2019
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.