Pith. sign in

REVIEW 3 major objections 4 minor 21 references

Diffusion probabilistic LMS algorithm

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read The paper claims that a distributed estimation algorithm whose step size is derived from a Gaussian posterior approximation is more robust to input-signal statistics and impulsive interference than three established diffusion LMS variants.

desk verdict A plausible PLMS-plus-diffusion recipe, but the published algorithm has a sign error in its variance recursion that makes it undefined, and the simulations are not reproducible. read the letter →

arxiv 1908.09730 v1 pith:24UJKLDH submitted 2019-08-22 eess.SP

classification eess.SP
keywords distributedestimationdiffusionstrategyprobabilisticLMSimpulsiveinterferencevariablestepsizeadaptivefilteringsystemidentificationmeanstability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes diffusion probabilistic LMS (DPLMS), a distributed adaptive filtering algorithm in which each node estimates a common unknown vector by combining an adapt-then-combine diffusion strategy with a probabilistic variable step-size LMS update. The step size at every node is not fixed but is recalculated from an isotropic Gaussian approximation of the posterior distribution of the unknown coefficients, so it shrinks when the input power is large and adapts to the measurement-noise variance. The paper's central claim is that this makes DPLMS more robust to input signal statistics and impulsive interference than three established robust diffusion algorithms: DSE-LMS, DRVSSLMS, and DLLAD. Simulation results on a 20-node system identification task are presented for several input-signal and Bernoulli-Gaussian impulse-noise settings. A mean-stability condition and a computational-complexity comparison are also derived.

What carries the argument

The load-bearing device is the time-varying step size $\alpha_n(i) = \frac{\sigma_n^2(i-1) + \sigma_\rho^2}{[\sigma_n^2(i-1) - \sigma_\rho^2]\|\mathbf{X}_n(i)\|^2 + \sigma_{\varepsilon,n}^2}$ together with the variance recursion $\sigma_n^2(i) = \left[1 - \frac{\alpha_n(i)\|\mathbf{X}_n(i)\|^2}{L}\right][\sigma_n^2(i-1) - \sigma_\rho^2]$. This $\alpha_n(i)$ is the PLMS step size obtained by approximating the posterior $p(\mathbf{W}_o(i)|\mathcal{Z}_i)$ as an isotropic Gaussian and taking the MAP estimate; it automatically down-weights large-input-power updates and nodes with high noise variance. Combining this update through the adapt-then-combine diffusion step is what produces the claimed robustness.

What would settle it

Simulate a two-node network with strongly correlated inputs or a node whose step size is driven by large past errors, compute the Monte Carlo average of $E[\mathbf{S}(i)\mathbf{R}(i)\hat{\mathbf{W}}(i-1)]$ and compare it with $E[\mathbf{S}(i)\mathbf{R}(i)]E[\hat{\mathbf{W}}(i-1)]$; if the difference is large and the mean weight error grows even when $\mu$ satisfies Eq. (22), the independence assumption is what failed.

Watch

Extended reading notes

Core claim

The central claim is that replacing the fixed step size that appears in existing diffusion LMS variants with the PLMS variable step size—defined from the variance of a Gaussian posterior—yields a distributed estimator that converges faster and reaches lower steady-state misalignment under both correlated input signals and impulsive interference. The paper argues this by constructing the DPLMS recursion, proving a sufficient mean-stability condition, and showing in Monte Carlo simulations that DPLMS outperforms DSE-LMS, DRVSSLMS, and DLLAD across several combinations of impulse probability and intensity, with equal or lower computational cost.

Load-bearing premise

The mean-stability proof assumes the matrix product $\mathbf{S}(i)\mathbf{R}(i)$ is independent of the current weight-error vector $\hat{\mathbf{W}}(i-1)$, even though the step-size factors in $\mathbf{S}(i)$ depend on input power and past estimation errors.

Editorial extensions

If this is right

  • If DPLMS is correct, distributed estimators can be made resistant to impulsive noise without sign operations or nonlinear clipping; the adaptation is achieved entirely by the Bayesian step-size schedule.
  • The mean-stability condition in Eq. (22) gives users an explicit step-size bound $\mu < 2/\rho_{\max}(\sum_l \alpha_l(i) c_{l,l} \mathbf{R}_{xx,l}(i))$, so deployment can start from a principled range rather than trial-and-error.
  • Because its per-iteration complexity matches DLLAD and is lower than DSE-LMS and DRVSSLMS, the claimed robustness comes without added computational burden.
  • The construction suggests that posterior-approximation techniques from Bayesian filtering can be transplanted into diffusion networks one agent at a time.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same probabilistic construction likely extends to other likelihood models: replacing the Gaussian likelihood with a heavier-tailed one (e.g., Student-t or Laplace) could give even stronger resistance to extreme impulses, since the Gaussian posterior is the current source of the quadratic error term.
  • A direct consequence not tested here is that DPLMS should track time-varying systems better than fixed-step-size algorithms when the unknown vector drifts, because the step size already incorporates the process-noise variance $\sigma_\rho^2$ as an explicit parameter.
  • The load-bearing independence assumption in the mean analysis suggests a testable refinement: measure the correlation between $\mathbf{S}(i)\mathbf{R}(i)$ and $\hat{\mathbf{W}}(i-1)$ in simulation; if non-negligible, the stability bound may need to be corrected for correlated inputs.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a distributed adaptive filtering algorithm, diffusion probabilistic LMS (DPLMS), which combines the adapt-then-combine diffusion strategy with the probabilistic least-mean-squares (PLMS) algorithm. The authors derive the local update from a Gaussian posterior approximation, introduce a constant parameter tau, and present a network-level two-step adaptation-combination scheme. The paper claims that DPLMS is more robust to input signal characteristics and impulsive interference than DSE-LMS, DRVSSLMS, and DLLAD, and it provides a mean-stability analysis and a computational-complexity comparison. The empirical section reports network MSD curves for several impulsive-noise scenarios with 60 Monte Carlo runs.

Significance. If the proposed algorithm worked as claimed, it would be a useful addition to distributed estimation under impulsive noise, with a step-size mechanism derived from a probabilistic model and with complexity equal to that of DLLAD and lower than that of DRVSSLMS. The paper's combination of a probabilistic step-size with diffusion is an interesting idea, and the complexity table is a useful summary. However, the current manuscript contains a fundamental error in the variance recursion that defines the algorithm, and the mean-stability analysis relies on an unjustified independence assumption. As a result, the algorithm as published is not well-defined for the reported simulations, and the main theoretical guarantee is not established. The empirical robustness claim is also not supported by the reported evidence because essential parameter values and statistical error information are missing. The paper would require substantial correction and re-analysis before its central claims can be accepted.

major comments (3)
  1. [Section 2.1, Eq. (6), Eq. (9), and Table 1] The variance recursion contains a sign error that makes the published algorithm undefined during the reported simulations. Under the state model in Eq. (2), the predictive variance of W_o(i) given Z^{i-1} is sigma^2(i-1) + sigma_rho^2, not sigma^2(i-1) - sigma_rho^2. This minus sign is then used in Eq. (9) and in Table 1's alpha_n(i) and sigma_n^2(i) update. The iteration sigma_n^2(i) = c_n(i) [sigma_n^2(i-1) - sigma_rho,n^2] with 0 < c_n(i) < 1 has only the negative fixed point -c_n(i) sigma_rho,n^2 / (1 - c_n(i)), so sigma_n^2(i) becomes negative in finite time from any positive initial value. Once sigma_n^2(i) is negative, alpha_n(i) is no longer a valid step size and the update in Eq. (16) is undefined or diverging. The paper never states sigma_rho,n^2, sigma_epsilon,n^2, the initial sigma_n^2(0), or any clipping floor, so the algorithm that produced the MSD curves in Section 4 cannot be reproduced. This error must be corrected and the simulations re-run.
  2. [Section 3.1, Eqs. (21) and (22)] The mean-stability derivation is not valid as written. The step E{A^T [I - S(i)R(i)] W~(i-1)} = A^T [I - E{S(i)R(i)}] E{W~(i-1)} in Eq. (21) requires S(i)R(i) to be independent of W~(i-1). But alpha_n(i) depends on sigma_n^2(i-1), which is a deterministic function of past regressors through the variance recursion in Table 1, and W~(i-1) is correlated with those past regressors. Therefore the independence assumption is unjustified, and the stability condition in Eq. (22) does not follow. Additionally, Eq. (22) still contains the stochastic alpha_l(i) inside the bound, so it is not a checkable step-size condition without further expectation or concentration arguments. The paper's only theoretical guarantee for DPLMS is therefore unsupported.
  3. [Section 4] The central robustness claim is not supported by the reported evidence. The simulations do not list the values of sigma_rho,n^2, sigma_epsilon,n^2, the initial sigma_n^2(0), or the constant tau, all of which are needed to run DPLMS. The MSD curves are averaged over 60 runs, but no error bars, confidence intervals, or run-to-run spread are shown, so the claim that DPLMS is more robust than DSE-LMS, DRVSSLMS, and DLLAD rests on visual inspection of single curves. The text in Experiments 1 and 2 also refers to 'DNLMS' where DPLMS is presumably intended, further complicating interpretation. These omissions are load-bearing because the empirical comparison is the main evidence for the paper's headline claim.
minor comments (4)
  1. [Section 2.2, Eq. (15)] The 'global cost function' in Eq. (15) is not well defined: it places argmax_{W_o(i)} p(W_o(i)|Z_i) inside a sum of local costs and is never used to derive the adaptation and combination updates in Eqs. (16) and (17). Please rewrite or remove this part to avoid implying that DPLMS minimizes a well-defined global objective.
  2. [Section 2.1, Eq. (7)] Equation (7) omits the Bayes normalization and confuses the predictive and posterior distributions. A cleaner derivation would state the exact Gaussian posterior before the isotropic approximation and then apply the trace-based variance update.
  3. [Section 2.1 and Section 3.1] The notation is inconsistent: the noise variance is called sigma_p^2 in Eq. (2) and sigma_rho^2 afterward, and the matrix definitions in Section 3.1 contain garbled expressions (e.g., 'M C C I' and the definition of O(i)). These should be corrected for readability.
  4. [Table 2] The complexity comparison would be clearer if the entries for DRVSSLMS were explained: the '>' markers and the roles of the sgn and absolute-value columns are not defined, and the total counts should be consistent with the equations cited.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the DPLMS derivation is a generative model plus a standard theorem application; the main problems are sign/reproducibility errors, not circular reductions.

full rationale

The paper proposes DPLMS by combining the diffusion adaptation/combination strategy with the PLMS variable step-size. The PLMS step-size is generated from a stated Gaussian posterior approximation (Eqs. (2)-(10)); it is not fitted to the MSD curves or to the robustness claim, so the robustness result is not an input renamed as a prediction. The mean-stability analysis in Section 3.1 follows the standard diffusion-LMS pattern: after writing the error recursion, Eq. (22) is the usual stability bound with the local variable step-size absorbed into S(i). Restating a known stability condition for a special step-size choice is a theorem application, not a circular derivation. The only self-citations, Ref. [15], are used for a standard zero-mean Gaussian regressor assumption and for a Bernoulli-Gaussian impulsive-noise construction; neither presupposes DPLMS's performance, so they are not load-bearing. There are genuine correctness concerns: the predictive variance in Eq. (6) should be sigma^2(i-1)+sigma_rho^2 rather than the printed minus sign; the resulting Table 1 variance recursion can become negative; the simulation leaves sigma_rho^2 and initial sigma^2 unspecified; and Eq. (21) assumes independence between S(i)R(i) and W-hat(i-1). These are technical or reproducibility defects, not cases in which an output reduces by construction to an input, so they do not raise the circularity score.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The algorithm depends on several manually chosen parameters (μ, τ, noise variances, initial variance) that are not fully specified. The theoretical analysis relies on standard adaptive-filtering independence assumptions and an ad-hoc isotropic Gaussian posterior approximation. No new physical entities are introduced.

free parameters (5)
  • Step size μ = 0.6 (Experiment 1), 0.4 (Experiments 2 and 3)
    Learning rate in adaptation step Eq. (16); chosen manually per experiment.
  • Constant τ = unspecified
    Scaling factor introduced in Eqs. (12)-(13) and inherited in Eq. (16); no value given, user must choose.
  • Measurement noise variance σ_ε² = unspecified
    Required to compute α_n(i) in Table 1; not stated how it is set in simulations.
  • Random walk variance σ_ρ² = unspecified
    Required in the variance update and α_n(i); not specified.
  • Initial variance σ²(0) = unspecified
    Seeds the recursion for σ²_n(i); not given.
assumptions (4)
  • domain assumption Unknown coefficient vector follows a random-walk model with zero-mean white Gaussian increments (Eq. 1-2).
    This model is assumed throughout the PLMS derivation; real systems may not follow it.
  • ad hoc to paper The posterior distribution is approximated as an isotropic Gaussian (Eq. 5).
    This approximation is introduced without a bound on its error; the algorithm is derived from the approximate posterior.
  • domain assumption Measurement noise and regressors are independent (Assumption 2) and regressors are zero-mean Gaussian, temporally white, spatially independent (Assumption 1).
    Standard assumptions in adaptive filtering; may not hold for correlated or impulsive inputs, though impulsive noise is tested.
  • domain assumption The factorization E[S(i)R(i)Ŵ(i-1)] = E[S(i)R(i)] E[Ŵ(i-1)] in Eq. (21)
    Unstated independence assumption between the step-size/regressor product and the weight error; not justified given α_n(i) depends on the data.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Diffusion probabilistic LMS algorithm." pith.science (2026). https://pith.science/paper/24UJKLDH

@misc{pith2026190809730,
  author       = {Pith},
  title        = {Pith review of: Diffusion probabilistic LMS algorithm},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/24UJKLDH}},
  note         = {Machine review of arXiv:1908.09730}
}
read the original abstract

In this paper, a novel diffusion estimation algorithm is proposed from a probabilistic perspective by combining diffusion strategy and the probabilistic least-mean-squares (PLMS) at all agents. The proposed method diffusion probabilistic LMS (DPLMS) is more robust to input signal and impulsive interference than the DSE-LMS, DRVSSLMS and DLLAD algorithms. Instead of minimizing the estimate error, the DPLMS algorithm is derived from approximating the posterior distribution with an isotropic Gaussian distribution. The stability of mean performance and computational complexity are analyzed theoretically. Results from the simulation indicate that the DPLMS algorithm is more robust to input signal and impulsive interference than the DSE-LMS, DRVSSLMS and DLLAD algorithms. These results suggest that the DPLMS algorithm can perform better in identifying the unknown coefficients under the complex and changeable impulsive interference environments.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 21 canonical work pages

  1. [1]

    Lopes, A.H

    C.G. Lopes, A.H. Sayed, Diffusion least -mean squares over adaptive networks: formulation and performance analysis [J]. IEEE Trans actions Signal Processing, 2008, 56(7):3122-3136

  2. [2]

    Cattivelli, A.H

    F.S. Cattivelli, A.H. Sayed, Diffusion LMS strategies for distributed estimation [J]. IEEE Transactions Signal Processing, 2010, 58(3):1035-1048

  3. [3]

    Diffusion strategies outperform consensus strategies for distributed estimation over adaptive networks [J]

    Tu S Y , Sayed A H. Diffusion strategies outperform consensus strategies for distributed estimation over adaptive networks [J]. IEEE Transactions on Signal Processing, 2012, 60(12):6217-6234

  4. [4]

    Sparse distributed learning based on diffusion adaptation [J]

    Lorenzo P D, Sayed A H. Sparse distributed learning based on diffusion adaptation [J]. IEEE Transactions on Signal Processing, 2013, 61(6):1419-1433

  5. [5]

    A. H. Sayed, Adaptive networks [J], Proceedings of the IEEE , 2014, 102(4):460-497

  6. [6]

    Diffusion LMS over multitask networks [J]

    Jie C, Richard C, Sayed A H. Diffusion LMS over multitask networks [J]. IEEE Transactions on Signal Processing, 2015, 63(11):2733-2748

  7. [7]

    z 2-proportionate diffusion LMS algorithm with mean square performance analysis [J]

    Lee H S, Yim S H, Song W J. z 2-proportionate diffusion LMS algorithm with mean square performance analysis [J]. Signal Processing, 2017, 131:154-160

  8. [8]

    A family of adaptive decorrelation NLMS algorithms and its diffusion version over adaptive networks [J]

    Zhang S, So H C, Mi W, et al. A family of adaptive decorrelation NLMS algorithms and its diffusion version over adaptive networks [J]. IEEE Transactions on Circuits & Systems I Regular Papers, 2017, PP (99):1-12

Show all 21 references
  1. [9]

    Cattivelli, C.G

    F.S. Cattivelli, C.G. Lopes, A.H. Sayed, Diffusion recursive least -squares for distributed estimation over adaptive networks [J]. IEEE Trans actions Signal Processing, 2008, 56(5):1865-1877

  2. [10]

    Diffusion sparse least-mean squares over networks [J]

    Liu Y , Li C, Zhang Z. Diffusion sparse least-mean squares over networks [J]. IEEE Transactions Signal Processing, 2012, 60(8):4480-4485

  3. [11]

    Diffusion least -mean p-power algorithms for distributed estimation in alpha-stable noise environments [J]

    Wen F. Diffusion least -mean p-power algorithms for distributed estimation in alpha-stable noise environments [J]. Electron ics Letters. 2013, 49(21): 1355-1356

  4. [12]

    Diffusion sign subband adaptive filtering algorithm for distributed estimation [J]

    Ni J. Diffusion sign subband adaptive filtering algorithm for distributed estimation [J]. IEEE Signal Processing Letters, 2015, 22(11):2029-2033

  5. [13]

    Diffusion sign -error LMS algorithm: Formulation and stochastic behavior analysis [J]

    Ni J, Chen J, Chen X. Diffusion sign -error LMS algorithm: Formulation and stochastic behavior analysis [J]. Signal Processing, 2016, 128:142-149

  6. [14]

    Variable step-size diffusion normalized sign-error algorithm [J]

    Wen P, Zhang J. Variable step-size diffusion normalized sign-error algorithm [J]. Circuits Systems and Signal Processing, 2018, 37(20):4993-5004

  7. [15]

    Diffusion normalized Huber adaptive filtering algorithm [J]

    Zhi L, Guan S. Diffusion normalized Huber adaptive filtering algorithm [J]. Journal of the Franklin Institute, 2018, 355(8):3812-3825

  8. [16]

    Diffusion robust variable step -size LMS algorithm over distributed networks [J]

    Huang W, Li L, Li Q, et al. Diffusion robust variable step -size LMS algorithm over distributed networks [J]. IEEE Access, 2018, 6:47511-47520

  9. [17]

    Diffusion least logarithmic absolute difference algorithm for distributed estimation [J]

    Feng Chen, Tao Shi, et al. Diffusion least logarithmic absolute difference algorithm for distributed estimation [J]. Signal Processing, 2018, 142:423-430

  10. [18]

    Robust diffusion LMS over adaptive networks [J]

    Soheila Ashkezari-Toussi, Hadi Sadoghi-Yazdi. Robust diffusion LMS over adaptive networks [J]. Signal Processing, 2019, 158:201-209

  11. [19]

    Steady -state and stability analyses of diffusion sign-error LMS algorithm [J]

    Gao Y , Ni J, Chen J , et al. Steady -state and stability analyses of diffusion sign-error LMS algorithm [J]. Signal Processing, 2018, 149:62-67

  12. [20]

    Fernandez -Bes, V

    J. Fernandez -Bes, V . Elvira, et al. A probabilistic least -mean-squares filter [C]. IEEE International Conference on Acoustics, Speech and Signal Processing , Brisbane, QLD, Australia, Aug. 2015, pp.2199-2203

  13. [21]

    Mean-square-deviation analysis of probabilistic LMS algorithm [J]

    Fuyi Huang, Jiashu Zhang, Sheng Zhang. Mean-square-deviation analysis of probabilistic LMS algorithm [J]. Digital Signal Processing, 2019, 92:26-35

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.