Sequential Inference for Gaussian Processes: A Signal Processing Perspective

Daniel Waxman; Fernando Llorente; Petar M. Djuri\'c

arxiv: 2604.28163 · v1 · submitted 2026-04-30 · 📡 eess.SP · cs.LG· stat.CO· stat.ML

Sequential Inference for Gaussian Processes: A Signal Processing Perspective

Daniel Waxman , Fernando Llorente , Petar M. Djuri\'c This is my paper

Pith reviewed 2026-05-07 06:02 UTC · model grok-4.3

classification 📡 eess.SP cs.LGstat.COstat.ML

keywords Gaussian processessequential inferencesignal processingstreaming dataincremental learningstate-space modelstime series analysisBayesian inference

0 comments

The pith

Gaussian processes support sequential inference for signal processing tasks through incremental and streaming methods.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper provides a self-contained tutorial on Gaussian processes with emphasis on sequential inference techniques suited for streaming data. This matters for signal processing because modern systems often handle time-series data where new observations arrive continuously, requiring models that update efficiently without retraining from scratch. By bridging ML advances to SP applications, the overview aims to give practitioners a roadmap for using GPs in areas like forecasting, anomaly detection, and adaptive sensing. The central focus is on making these methods accessible and directly usable in real-world SP systems.

Core claim

The paper claims that recent advances in sequential, incremental, or streaming inference for Gaussian processes can be organized from a signal-processing perspective to support applications in state-space modeling, sequential regression and forecasting, anomaly detection in time series, sequential Bayesian optimization, adaptive and active sensing, and sequential detection and decision-making. This tutorial-style overview bridges ML developments to SP needs, providing practical tools for deploying sequential GP models.

What carries the argument

The Gaussian process model with sequential inference updates, which allow efficient incorporation of new data points while maintaining the probabilistic framework for function approximation.

Load-bearing premise

The advances in sequential GP inference are sufficiently mature and can be transferred to signal processing applications without requiring substantial additional development or adaptation.

What would settle it

A practical test would be to apply one of the surveyed sequential GP methods to a real-world signal processing problem, such as time-series anomaly detection, and measure whether it achieves expected performance gains over standard batch GP approaches without extensive custom modifications.

read the original abstract

The proliferation of capable and efficient machine learning (ML) models marks one of the strongest methodological shifts in signal processing (SP) in its nearly 100-year history. ML models support the development of SP systems that represent complex, nonlinear relationships with high predictive accuracy. Adapting these models often requires sequential inference, which differs both theoretically and methodologically from the usual paradigm of ML, where data are often assumed independent and identically distributed. Gaussian processes (GPs) are a flexible yet principled framework for modeling random functions, and they have become increasingly relevant to SP as statistical and ML methods assume a more prominent role. We provide a self-contained, tutorial-style overview of GPs, with a particular focus on recent methodological advances in sequential, incremental, or streaming inference. We introduce these techniques from a signal-processing perspective while bridging them to recent advances in ML. Many of the developments we survey have direct applications to state-space modeling, sequential regression and forecasting, anomaly detection in time series, sequential Bayesian optimization, adaptive and active sensing, and sequential detection and decision-making. By organizing these advances from a signal-processing perspective, we intend to equip practitioners with practical tools and a coherent roadmap for deploying sequential GP models in real-world systems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

Organizing tutorial on sequential GPs for SP that collects existing methods without new contributions.

read the letter

This paper is a tutorial-style survey on sequential inference for Gaussian processes, presented from a signal processing perspective. It reviews existing methods for handling streaming or incremental data with GPs and connects them to ML developments, without claiming any new derivations or experiments. It does a good job of making the material accessible. The overview starts with standard GP concepts and then moves to sequential variants like state-space models and online updates. This framing helps SP practitioners see how these tools apply to problems such as time-series forecasting, anomaly detection, and adaptive sensing. The listed applications give a clear sense of where the methods fit in real systems, and the bridge to ML advances is useful for people coming from either side. The soft spots are mostly about scope and depth. As a survey, it stands or falls on whether it covers the key papers accurately and points out practical limitations, like scalability in high dimensions or sensitivity to kernel choice in non-stationary signals. The abstract suggests the advances are mature for SP use, but in practice many sequential GP techniques still need engineering tweaks for embedded systems or specific noise models, and the paper may not emphasize those gaps enough. No major inconsistencies show up in the stated goals. This work is aimed at signal processing researchers and engineers who want to integrate modern GP inference into sequential applications, as well as students or newcomers to the area. A reader looking for a starting point or a way to navigate the literature would get value from it. It deserves a serious referee because even without novelty, a clear and balanced tutorial can help the field by reducing the effort needed to adopt these methods. I would recommend sending it for peer review. The organizational contribution is real if the coverage is thorough.

Referee Report

2 major / 2 minor

Summary. The paper claims to deliver a self-contained, tutorial-style overview of Gaussian processes (GPs), with a particular focus on recent methodological advances in sequential, incremental, or streaming inference. It introduces these techniques from a signal-processing perspective while bridging them to recent advances in ML, and highlights applications to state-space modeling, sequential regression and forecasting, anomaly detection in time series, sequential Bayesian optimization, adaptive and active sensing, and sequential detection and decision-making.

Significance. If the surveyed content is accurate and comprehensive, this tutorial could be significant for bridging ML advances in GP inference to SP applications. The self-contained format and SP-centric organization provide a coherent roadmap that could equip practitioners with practical tools for real-world sequential problems. Credit is given for the synthesis of existing work without unsubstantiated new claims and for explicitly linking theoretical developments to listed application areas.

major comments (2)

[Abstract] Abstract: The assertion that 'many of the developments we survey have direct applications' to the listed SP areas (state-space modeling, anomaly detection, adaptive sensing, etc.) is load-bearing for the goal of equipping practitioners with a roadmap. The manuscript should include at least one concrete case study or detailed reference demonstrating successful transfer of a sequential GP method to an SP problem, as the current high-level listing risks overstating readiness without evidence of adaptation.
[The section on applications and bridging to SP] The section on applications and bridging to SP: The central claim assumes that recent advances in sequential GP inference are mature enough and directly transferable to SP without significant additional adaptation. The manuscript should explicitly discuss domain-specific challenges (e.g., real-time constraints, non-stationary signals, or hardware limitations) and any required modifications to make the roadmap robust and actionable for SP practitioners.

minor comments (2)

[Abstract] Abstract: The phrase 'nearly 100-year history' of signal processing is roughly correct but would benefit from a specific citation to a foundational reference for precision and to aid readers unfamiliar with the field's timeline.
[Throughout] Throughout: Key GP notation (e.g., kernel functions, mean functions, and hyperparameters) should be defined consistently and introduced in an early section to improve accessibility for the SP audience targeted by the tutorial.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and constructive major comments. The feedback identifies opportunities to strengthen the concrete linkage between the surveyed methods and signal-processing practice. We address each point below and will incorporate the suggested revisions in the next version of the manuscript.

read point-by-point responses

Referee: [Abstract] Abstract: The assertion that 'many of the developments we survey have direct applications' to the listed SP areas (state-space modeling, anomaly detection, adaptive sensing, etc.) is load-bearing for the goal of equipping practitioners with a roadmap. The manuscript should include at least one concrete case study or detailed reference demonstrating successful transfer of a sequential GP method to an SP problem, as the current high-level listing risks overstating readiness without evidence of adaptation.

Authors: We agree that a single concrete illustration would make the abstract claim more robust. The manuscript is a survey and therefore does not contain new empirical results; however, it already cites multiple works that report end-to-end deployments. In the revised version we will (i) replace the generic list in the abstract with a brief, specific reference (e.g., “as demonstrated by the real-time implementation of streaming variational GP regression for anomaly detection in sensor networks [cite the relevant surveyed paper]”) and (ii) add a short paragraph in the introduction that summarizes the key adaptation steps taken in that reference. This change stays within the scope of the existing citations and does not require new experiments. revision: yes
Referee: [The section on applications and bridging to SP] The section on applications and bridging to SP: The central claim assumes that recent advances in sequential GP inference are mature enough and directly transferable to SP without significant additional adaptation. The manuscript should explicitly discuss domain-specific challenges (e.g., real-time constraints, non-stationary signals, or hardware limitations) and any required modifications to make the roadmap robust and actionable for SP practitioners.

Authors: We accept that the applications section would be more useful to SP readers if it explicitly treated the practical frictions that arise when moving from the ML literature to deployed systems. We will add a new subsection (approximately one page) titled “Domain-Specific Challenges and Required Adaptations.” It will cover: (a) real-time latency and memory bounds and the consequent need for constant-time or logarithmic-complexity updates (e.g., via fixed-budget inducing-point schemes); (b) non-stationarity of physical signals and the corresponding use of online kernel learning or exponential forgetting; and (c) hardware constraints such as fixed-point arithmetic and power limits, together with the modifications (quantized inference, distributed or edge implementations) that have been proposed in the surveyed literature. All points will be supported by citations already present in the manuscript plus a small number of additional SP-specific references. revision: yes

Circularity Check

0 steps flagged

No significant circularity: survey paper with no original derivations

full rationale

This is a tutorial-style survey paper whose central claim is to deliver a self-contained overview of Gaussian processes and recent advances in sequential/incremental/streaming inference, organized from a signal-processing perspective. The abstract and stated goals explicitly frame the contribution as descriptive organization and bridging of existing literature rather than any new theorem, derivation, prediction, or empirical result. No load-bearing steps involve self-definition of quantities, renaming of fitted parameters as predictions, or chains of self-citations that substitute for independent justification. The paper cites prior work externally and does not reduce any claimed result to its own inputs by construction. This matches the default expectation for non-circular survey contributions; the derivation chain is absent, so no circularity can be exhibited.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

No new free parameters, axioms, or invented entities are introduced as this is a survey paper reviewing existing literature on Gaussian processes and sequential inference.

pith-pipeline@v0.9.0 · 5525 in / 1024 out tokens · 35553 ms · 2026-05-07T06:02:20.401631+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

3 extracted references · 3 canonical work pages

[1]

Adam, V.,Chang, P.,Khan, M. E. E.andSolin, A.(2021). Dual pa- rameterization of sparse variational Gaussian processes.Advances in Neu- ral Information Processing Systems3411474–11486. Al-Shedivat, M.,Wilson, A. G.,Saatchi, Y.,Hu, Z.andXing, E. P. (2017). Learning scalable deep kernels with recurrent structure.Journal of Machine Learning Research181–37. Ar...

work page arXiv 2021
[2]

Sch¨urch, M.,Azzimonti, D.,Benavoli, A.andZaffalon, M.(2020)

Cambridge University Press. Sch¨urch, M.,Azzimonti, D.,Benavoli, A.andZaffalon, M.(2020). Recursive estimation for sparse Gaussian process regression.Automatica 120109127. Solin, A.andS ¨arkk¨a, S.(2020). Hilbert space methods for reduced-rank Gaussian process regression.Statistics and Computing30419–446. Titsias, M.(2009). Variational learning of inducin...

work page 2020
[3]

N.,Kaiser, L.andPolosukhin, I.(2017)

Vaswani, A.,Shazeer, N.,Parmar, N.,Uszkoreit, J.,Jones, L., Gomez, A. N.,Kaiser, L.andPolosukhin, I.(2017). Attention Is All You Need. InAdvances in Neural Information Processing Systems. Verma, P.,Adam, V.andSolin, A.(2024). Variational Gaussian process diffusion processes. InInternational Conference on Artificial Intelligence and Statistics1909–1917. PM...

work page 2017

[1] [1]

Adam, V.,Chang, P.,Khan, M. E. E.andSolin, A.(2021). Dual pa- rameterization of sparse variational Gaussian processes.Advances in Neu- ral Information Processing Systems3411474–11486. Al-Shedivat, M.,Wilson, A. G.,Saatchi, Y.,Hu, Z.andXing, E. P. (2017). Learning scalable deep kernels with recurrent structure.Journal of Machine Learning Research181–37. Ar...

work page arXiv 2021

[2] [2]

Sch¨urch, M.,Azzimonti, D.,Benavoli, A.andZaffalon, M.(2020)

Cambridge University Press. Sch¨urch, M.,Azzimonti, D.,Benavoli, A.andZaffalon, M.(2020). Recursive estimation for sparse Gaussian process regression.Automatica 120109127. Solin, A.andS ¨arkk¨a, S.(2020). Hilbert space methods for reduced-rank Gaussian process regression.Statistics and Computing30419–446. Titsias, M.(2009). Variational learning of inducin...

work page 2020

[3] [3]

N.,Kaiser, L.andPolosukhin, I.(2017)

Vaswani, A.,Shazeer, N.,Parmar, N.,Uszkoreit, J.,Jones, L., Gomez, A. N.,Kaiser, L.andPolosukhin, I.(2017). Attention Is All You Need. InAdvances in Neural Information Processing Systems. Verma, P.,Adam, V.andSolin, A.(2024). Variational Gaussian process diffusion processes. InInternational Conference on Artificial Intelligence and Statistics1909–1917. PM...

work page 2017