Pith. sign in

REVIEW 3 major objections 5 minor 20 references

Online Bayesian system identification in multivariate autoregressive models via message passing

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Autoregressive identification gets exact online Bayesian updates.

desk verdict Solid, standard conjugate updating packaged as message passing; the one-step predictive is correct, but the multi-step predictive claim overreaches and Lemma 2 has a fixable singularity bug. read the letter →

arxiv 2506.02710 v1 pith:QGL6ZETG submitted 2025-06-03 eess.SP cs.LGstat.ML

T. N. Nisslbeck , Wouter M. Kouw This is my paper
classification eess.SPcs.LGstat.ML
keywords BayesiansystemidentificationmultivariateautoregressivemodelsmatrixnormalWishartdistributionmessagepassingfactorgraphsrecursiveestimationposteriorpredictiveexogenousinputs
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that Bayesian identification of multivariate autoregressive models with exogenous inputs can be made exact and recursive: a matrix normal Wishart prior over the coefficient matrix and noise precision is conjugate to the Gaussian likelihood, so the posterior at each time step is again matrix normal Wishart. The update uses only the current regressor and output plus the previous posterior parameters, which makes online system identification feasible at recursive least squares cost while returning full posterior distributions. The paper further claims that the predictive distribution for future outputs is a multivariate location-scale T-distribution, meaning parameter uncertainty is carried into prediction intervals and into online model evidence calculations. If these claims hold, online control and monitoring applications get calibrated uncertainty estimates without sampling or variational approximation.

What carries the argument

The central object is the matrix normal Wishart distribution, a joint prior in which the coefficient matrix A is matrix-normal (given precision W) and the precision W is Wishart; it is the conjugate family for the Gaussian MARX likelihood. The argument runs on three algebraic facts: the likelihood can be re-expressed as a matrix normal Wishart message (Lemma 2), the product of two matrix normal Wishart densities is again matrix normal Wishart with parameters combined by a fixed rule (Lemma 3), and marginalizing a Gaussian over a matrix normal and then over a Wishart yields a location-scale T-distribution (Theorem 2). These facts carry the recursion and make the factor graph's equality node an exact posterior computation.

What would settle it

Take one scalar observation (D_x=1, D_y=1) with nonzero regressor x_t; compute the Lemma 2 message parameters and check numerically whether the claimed matrix normal Wishart density, integrated over A and W, equals the true likelihood. The singular x_t x_t^T and scale Ω̄_t = 0 make the message improper or ambiguous unless a pseudoinverse is specified, so the equality in (22) either fails or requires an unstated convention.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes a closed-form recursion for the posterior p(A,W | D_t) in a MARX model, with coefficient matrix A and noise precision W updated by the rules ν_t = ν_{t-1}+1, Λ_t = Λ_{t-1}+x_t x_t^T, and the corresponding updates for the matrix mean M_t and scale Ω_t (equations (15)-(18)). Theorem 1 shows this same posterior arises in a Forney-style factor graph as the product of the previous posterior message with a per-observation likelihood message; Lemma 3 supplies the matrix normal Wishart product rule that makes the equality node exact. Theorem 2 derives the posterior predictive distribution for a future output y_τ as a location-scale T-distribution with location M_t^T x_τ, scale η_τ $Ω_t^{{-1}}$ λ_τ, and degrees of freedom η_τ = ν_t - D_y + 1, where λ_τ = (1 + x_τ^T $Λ_t^{{-1}}$ x_τ)^{-1}. The result is an exact recursive Bayesian estimator whose predictions are intervals rather than point forecasts.

Load-bearing premise

Everything rests on treating a single observation's likelihood as a proper matrix normal Wishart message, which requires dividing by x_t x_t^T even though that matrix is singular for a vector observation; the paper does not state how the resulting improper message is normalized.

Editorial extensions

If this is right

  • Each new observation updates the full posterior in closed form, so online identification needs no MCMC or variational approximation at run time.
  • Prediction intervals automatically expand when parameter estimates are uncertain, because the predictive is a T-distribution rather than a Gaussian with a plug-in estimate.
  • The same recursion supports online model evidence computation, enabling model comparison or forgetting-factor selection while data streams in.
  • Because the computation is organized as message passing on a factor graph, the update can be modularized and distributed across devices while remaining mathematically the same posterior.
  • The posterior mean of the coefficients behaves like a recursive least squares estimate, so RLS point estimates are recovered as a special case of the full Bayesian solution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit a time-varying extension: inserting state-transition factor nodes between the equality nodes would turn the recursion into a filtering recursion for drifting coefficients, a direction its own discussion gestures at.
  • One could check exactness against the batch posterior: processing the whole training set in one pass and in normal recursion should yield the same posterior, and if not, the single-observation degeneracy of Lemma 2 is the place to look.
  • The T-predictive suggests a calibration test: quantiles of normalized prediction errors should match the T degrees of freedom as training length varies, which a practitioner could check without modifying the algorithm.
  • An immediate practical extension is adaptive forgetting, replacing ν_t = ν_{t-1}+1 with a discount factor to track nonstationary systems, though the paper does not develop this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a recursive Bayesian estimator for multivariate autoregressive models with exogenous inputs (MARX). It uses a matrix normal Wishart prior over the coefficient matrix and noise precision, shows that the Gaussian MARX likelihood updates this prior within the same family, and presents the recursion as message passing on a Forney-style factor graph. It also derives a predictive distribution for future outputs, claimed to be a multivariate location-scale T-distribution, and reports experiments on a synthetic autoregressive system and a double mass-spring-damper system, comparing against recursive least squares.

Significance. If the claims are properly scoped, the paper provides a useful exact online Bayesian identification procedure for MARX models. The direct posterior update in Lemma 1 and Appendix I is correct and matches standard conjugate Bayesian updating for a matrix normal Wishart prior with a Gaussian likelihood. The one-step-ahead predictive derivation in Theorem 2 and Appendix IV is also correct and is a valuable closed-form result. The paper is self-contained in its main derivation, provides code, and gives explicit experiments. However, the advertised multi-step predictive distribution is actually only a one-step-ahead conditional predictive, and the likelihood-to-MNW message in Lemma 2 is stated with a nonexistent inverse; both points need to be fixed for the stated contributions to hold. The online model evidence claim is also unsupported by any derivation or experiment.

major comments (3)
  1. [Section IV-B, Eq. (37)-(38), Theorem 2] For tau > t+1, x_tau contains the lagged outputs y_{tau-1}, ..., y_{tau-Ny}, which are not in the conditioning set D_t. The integral in Eq. (38) integrates only over A and W, treating x_tau as fixed, so it does not marginalize over the intermediate unobserved outputs. The derived T-distribution is therefore a conditional predictive given hypothetical future lagged outputs, not the posterior predictive distribution for future outputs as claimed. The valid interpretation is the one-step-ahead predictive for tau = t+1. The paper should either restrict the claim to tau = t+1 or supply an additional recursion that marginalizes over y_{t+1}, ..., y_{tau-1}.
  2. [Appendix II, Lemma 2, Eqs. (24) and (63)-(64)] The proof of Lemma 2 uses the inverse of x_t x_t^T, but for a single observation x_t is a vector, so x_t x_t^T is rank-one and singular. Equation (24) defines \bar{M}_t = (x_t x_t^T)^{-1} x_t y_t^T, which does not exist in general, and Eq. (63) cancels y_t y_t^T using this inverse. The cancellation can be performed with the Moore-Penrose pseudoinverse, but the resulting message is a degenerate/improper matrix normal Wishart kernel with \bar{\Lambda}_t singular and \bar{\Omega}_t = 0, not a proper matrix normal Wishart distribution. Since Theorem 1 uses Lemma 2 to derive the recursive update, the exactness of the message-passing derivation needs an explicit treatment of this singular case.
  3. [Abstract and Section IV-A, Eq. (13)] The abstract claims the method supports online model evidence calculations, but the paper never derives, evaluates, or demonstrates an evidence update. Equation (13) merely defines the marginal likelihood p(y_t | u_t, D_{t-1}); no normalization constant for the posterior, no evidence recursion, and no model-comparison experiment is provided. If model evidence support is intended as a contribution, the normalizing constants of the messages in Lemmas 2 and 3 and an evidence recursion must be stated explicitly.
minor comments (5)
  1. [Appendix II, just before Eq. (65)] The line defining \bar{\nu}_t uses \bar{D} = D_x + D_y + 1, which is inconsistent with the definition \bar{D} = D_x - D_y - 1 used in Appendix I and Appendix III, and also inconsistent with Lemma 2's \bar{\nu}_t = 2 - D_x + D_y.
  2. [Section V-B, first paragraph] The text says "At Ttrain = 2 6"; this should read "At Ttrain = 26".
  3. [Eq. (8)] The normalization of the matrix normal Wishart density in Eq. (8) is hard to parse because the square root and powers are placed ambiguously; consider rewriting with explicit exponents for all factors.
  4. [Eq. (42)] The multivariate T-distribution in Eq. (42) requires \eta_\tau = \nu_t - D_y + 1 > 0, but the paper does not state this condition. The experimental prior satisfies it, but the general update rule should mention the restriction.
  5. [Eq. (18) and Appendix I] The update for \Omega_t is clearer if written as \Omega_t = \Omega_{t-1} + y_t y_t^T + M_{t-1}^T \Lambda_{t-1} M_{t-1} - M_t^T \Lambda_t M_t; this form makes the rank-one nature of the update and the positive-semidefiniteness argument more transparent.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the recursive Bayesian update and predictive distribution are derived from the stated likelihood and prior, and the self-citations are not load-bearing.

full rationale

The derivation chain is self-contained. Lemma 1 and Appendix I multiply the stated Gaussian likelihood (5) by the matrix-normal-Wishart prior (7), complete the square, and obtain another matrix-normal-Wishart distribution with the update rules (15)-(18); this is ordinary conjugate Bayesian updating, not a fit disguised as a prediction. Lemma 3 and Appendix III prove closure under multiplication algebraically. Theorem 2 and Appendix IV marginalize over A and W analytically, using the standard result from the Book of Statistical Proofs for the Wishart-to-T marginalization. No parameter is fitted to the data whose prediction is then reported: the experimental posteriors are estimated from training data and evaluated on separate test sequences. The only self-citations ([4] and [13]) are contextual references to earlier active-inference and time-varying autoregressive work and are not load-bearing; no uniqueness theorem or ansatz is imported from those papers. Potential mathematical concerns, such as the singular matrix x_t x_t^T in Lemma 2 and the fact that Eq. (38) is only the one-step-ahead predictive because x_tau contains unobserved lagged outputs for tau > t+1, are correctness risks rather than circular reductions: the claimed predictive distribution is not equal to its input by construction. Therefore, the paper exhibits no significant circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central derivation relies on standard conjugate Bayesian calculations; no entities are invented. The prior hyperparameters are user choices, not fitted to the target data, but they do affect experimental outcomes.

free parameters (4)
  • Prior scale Λ0 = 1e-4 * I (uninformative) or 1e-1 * I (weakly informative)
    Hand-chosen prior precision for the coefficient matrix; not fitted to data. It affects convergence speed in experiments but not the derivation of the algorithm.
  • Prior scale Ω0 = 1e-5 * I (uninformative) or 1e-1 * I (weakly informative)
    Hand-chosen scale for the Wishart prior on precision W; not fitted to data. The text erroneously writes I_{Dx} once, but it should be I_{Dy}.
  • Prior degrees of freedom ν0 = Dy + 2
    Chosen to keep the prior proper and mildly informative; not fitted to data.
  • Prior mean M0 = 0_{Dx×Dy}
    Chosen to be zero, a standard neutral choice; not fitted to data.
assumptions (5)
  • standard math Conjugacy of the matrix normal Wishart family with the multivariate Gaussian likelihood
    The paper's Lemma 1 and Theorem 1 rely on the closure of the MNW family under multiplication with a Gaussian likelihood. This is a classical result, proved in Appendix I.
  • standard math Marginalizing a normal with Wishart precision yields a multivariate T-distribution
    Used in Theorem 2 to derive the posterior predictive; cited to the Book of Statistical Proofs [16, ID: D148].
  • standard math Matrix determinant lemma and Sherman-Morrison formula
    Used in Appendix IV to simplify the predictive covariance expressions.
  • domain assumption The system is a linear MARX process with known order (Ny, Nu) and additive Gaussian noise
    The entire likelihood (5) assumes this structure. The paper does not address model order selection or non-Gaussian noise.
  • domain assumption The regressor vector x_t is known and deterministic at each time step
    Equation (4) defines x_t from past outputs and inputs; the method treats these as fixed, not as random variables with uncertainty.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Online Bayesian system identification in multivariate autoregressive models via message passing." pith.science (2026). https://pith.science/paper/QGL6ZETG

@misc{pith2026250602710,
  author       = {Pith},
  title        = {Pith review of: Online Bayesian system identification in multivariate autoregressive models via message passing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QGL6ZETG}},
  note         = {Machine review of arXiv:2506.02710}
}
read the original abstract

We propose a recursive Bayesian estimation procedure for multivariate autoregressive models with exogenous inputs based on message passing in a factor graph. Unlike recursive least-squares, our method produces full posterior distributions for both the autoregressive coefficients and noise precision. The uncertainties regarding these estimates propagate into the uncertainties on predictions for future system outputs, and support online model evidence calculations. We demonstrate convergence empirically on a synthetic autoregressive system and competitive performance on a double mass-spring-damper system.

Figures

Figures reproduced from arXiv: 2506.02710 by the authors.

Figure 1
Figure 1. Forney-style factor graph of the MARX model in recursive form. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. (Top) Log-scale Frobenius-norm differences (A˜ vs. A); (Second) W˜ vs W. Time series of selected elements of A (third) and W (bottom). Shaded areas denote standard errors; horizontal lines indicate true values. C. Validation The validation system is a double mass-spring-damper system. Mass m1 is attached to a fixed base via spring and damper coefficients k1 and c1, and mass m2 is linked to m1 via k2 and c2. The disc… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

20 extracted references · 20 canonical work pages

  1. [1]

    On the Bayesian estimation of multi- variate regression,

    G. C. Tiao and A. Zellner, “On the Bayesian estimation of multi- variate regression,” Journal of the Royal Statistical Society: Series B (Methodological), vol. 26, no. 2, pp. 277–285, 1964

    work page 1964

  2. [2]

    Recursive estimation of autoregressions,

    E. J. Hannan, A. McDougall, and D. S. Poskitt, “Recursive estimation of autoregressions,” Journal of the Royal Statistical Society: Series B (Methodological), vol. 51, no. 2, pp. 217–233, 1989

    work page 1989

  3. [3]

    Forecasting with Bayesian vector autoregression,

    S. Karlsson, “Forecasting with Bayesian vector autoregression,” Hand- book of Economic Forecasting , vol. 2, pp. 791–897, 2013

    work page 2013

  4. [4]

    Coupled autoregressive active inference agents for control of multi-joint dynamical systems,

    T. N. Nisslbeck and W. M. Kouw, “Coupled autoregressive active inference agents for control of multi-joint dynamical systems,” in International Workshop on Active Inference , Springer, 2024

    work page 2024

  5. [5]

    Multivariate autoregressive models,

    W. Penny and L. Harrison, “Multivariate autoregressive models,” Statistical Parametric Mapping: The Analysis of Functional Brain Images, pp. 534–540, 2007

    work page 2007

  6. [6]

    Bayesian identification of multivariate autoregressive processes,

    S. M. Shaarawy and S. S. Ali, “Bayesian identification of multivariate autoregressive processes,” Communications in Statistics—Theory and Methods, vol. 37, no. 5, pp. 791–802, 2008

    work page 2008

  7. [7]

    A multi-camera multi-target tracker based on factor graphs,

    F. Castaldo and F. A. Palmieri, “A multi-camera multi-target tracker based on factor graphs,” in IEEE International Symposium on Inno- vations in Intelligent Systems and Applications , pp. 131–137, IEEE, 2014

    work page 2014

  8. [8]

    Bayesian experimental design: A review,

    K. Chaloner and I. Verdinelli, “Bayesian experimental design: A review,” Statistical Science, pp. 273–304, 1995

    work page 1995

Show all 20 references
  1. [9]

    A factor graph approach to estimation and model predictive control on unmanned aerial vehicles,

    D.-N. Ta, M. Kobilarov, and F. Dellaert, “A factor graph approach to estimation and model predictive control on unmanned aerial vehicles,” in International Conference on Unmanned Aircraft Systems , pp. 181– 188, IEEE, 2014

    2014

  2. [10]

    Linear optimal control on fac- tor graphs—a message passing perspective—,

    C. Hoffmann and P. Rostalski, “Linear optimal control on fac- tor graphs—a message passing perspective—,” IFAC-PapersOnLine, vol. 50, no. 1, pp. 6314–6319, 2017

    2017

  3. [11]

    A unifying view of estimation and control using belief propagation with application to path planning,

    F. A. Palmieri, K. R. Pattipati, G. Di Gennaro, G. Fioretti, F. Verolla, and A. Buonanno, “A unifying view of estimation and control using belief propagation with application to path planning,” IEEE Access , vol. 10, pp. 15193–15216, 2022

    2022

  4. [12]

    The factor graph approach to model-based signal processing,

    H.-A. Loeliger, J. Dauwels, J. Hu, S. Korl, L. Ping, and F. R. Kschis- chang, “The factor graph approach to model-based signal processing,” Proceedings of the IEEE , vol. 95, no. 6, pp. 1295–1322, 2007

    2007

  5. [13]

    Message passing-based inference for time-varying autoregressive models,

    A. Podusenko, W. M. Kouw, and B. de Vries, “Message passing-based inference for time-varying autoregressive models,” Entropy, vol. 23, no. 6, p. 683, 2021

    2021

  6. [14]

    Reactive message passing for scal- able bayesian inference,

    D. Bagaev and B. de Vries, “Reactive message passing for scal- able bayesian inference,” Scientific Programming , vol. 2023, no. 1, p. 6601690, 2023

    2023

  7. [15]

    On the role of knowledge graphs in explainable AI,

    F. Lecue, “On the role of knowledge graphs in explainable AI,” Semantic Web, vol. 11, no. 1, pp. 41–51, 2020

    2020

  8. [16]

    The Book of Statistical Proofs (Version 2023),

    J. Soch, C. Allefeld, T. J. Faulkenberry, M. Pavlovic, K. Petrykowski, K. Sarıtas ¸, S. Balkus, A. Kipnis, H. Atze, and O. A. Martin, “The Book of Statistical Proofs (Version 2023),” Jan. 2024

    2023

  9. [17]

    Study of variational inference for flexible distributed probabilistic robotics,

    M. R. Damgaard, R. Pedersen, and T. Bak, “Study of variational inference for flexible distributed probabilistic robotics,” Robotics, vol. 11, no. 2, p. 38, 2022

    2022

  10. [18]

    Message passing neural network versus message passing algorithm for cooperative positioning,

    B. C. Tedeschini, M. Brambilla, and M. Nicoli, “Message passing neural network versus message passing algorithm for cooperative positioning,” IEEE Transactions on Cognitive Communications and Networking, vol. 9, no. 6, pp. 1666–1676, 2023

    2023

  11. [19]

    Message passing based wireless federated learning via analog message aggregation,

    Y . Zhang, W. Xu, A. Liu, and V . Lau, “Message passing based wireless federated learning via analog message aggregation,” in 2024 IEEE/CIC International Conference on Communications in China (ICCC), pp. 2161–2166, IEEE, 2024

    2024

  12. [20]

    Multi-agent trajectory planning with NUV priors,

    B. van Erp, D. Bagaev, A. Podusenko, ˙I. S ¸en¨oz, and B. de Vries, “Multi-agent trajectory planning with NUV priors,” in American Control Conference, pp. 2766–2771, IEEE, 2024

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.