pith. sign in

arxiv: 1907.00650 · v1 · pith:7YEWSFBRnew · submitted 2019-07-01 · 💻 cs.LG · q-bio.NC· stat.ML

Neural Dynamics Discovery via Gaussian Process Recurrent Neural Networks

Qi She , Anqi Wu This is my paper

Pith reviewed 2026-05-25 12:35 UTC · model grok-4.3

classification 💻 cs.LG q-bio.NCstat.ML
keywords latent dynamicsGaussian processrecurrent neural networkneural datadimensionality reductionnonlinear dynamicsinference networkbi-LSTM
0
0 comments X

The pith

A recurrent neural network paired with Gaussian processes recovers nonlinear latent dynamics from noisy high-dimensional neural data more accurately than prior methods, especially with few samples.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces a latent dynamic model that combines recurrent neural networks to capture nonlinear, non-Markovian, long short-term time-dependent dynamics with non-parametric Gaussian processes to handle complex nonlinear embeddings. This setup addresses limitations in earlier approaches that relied on simple state transitions, linear embeddings, or inflexible inference networks. A bi-directional LSTM inference network approximates the posterior distributions to make the otherwise intractable model usable. Experiments on simulated and experimental neural datasets show the model reconstructs insightful latent trajectories more effectively than state-of-the-art alternatives under both Gaussian and Poisson observations, with particular gains when sample sizes are small.

Core claim

The GP-RNN model captures nonlinear, non-Markovian, long short-term time-dependent dynamics via recurrent neural networks and tackles complex nonlinear embedding via non-parametric Gaussian process, using a bi-directional long short-term memory inference network to encode past and future information into posterior distributions, and outperforms other state-of-the-art methods in reconstructing insightful latent dynamics from both simulated and experimental neural datasets with either Gaussian or Poisson observations, especially in the low-sample scenario.

What carries the argument

The GP-RNN architecture, which uses recurrent neural networks for state transitions and Gaussian processes for emission mappings, with a bi-LSTM inference network to approximate posteriors.

If this is right

  • More accurate recovery of latent trajectories becomes possible from high-dimensional noisy neural recordings.
  • Nonlinear and non-Markovian dynamics can be modeled without restrictive linear or Markov assumptions.
  • Performance gains appear especially when training data are limited.
  • The same model applies to both continuous Gaussian and discrete Poisson observation models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The approach may transfer to other high-dimensional time-series problems outside neuroscience where dynamics are nonlinear and data are sparse.
  • Uncertainty estimates from the Gaussian process component could support downstream tasks such as active learning for neural experiments.
  • Combining this structure with additional regularization might further improve robustness when observation noise characteristics are unknown.

Load-bearing premise

The RNN-GP architecture with bi-LSTM inference network can accurately recover true latent dynamics from high-dimensional noisy data without being limited by model complexity or inference intractability.

What would settle it

On a simulated dataset with known ground-truth latent trajectories, if the model fails to recover those trajectories more accurately than simpler baselines even in low-sample regimes, the performance advantage would not hold.

Figures

Figures reproduced from arXiv: 1907.00650 by Anqi Wu, Qi She.

Figure 1
Figure 1. Figure 1: The proposed GP-RNN models the dynamics of hidden states νt (yellow circle) with an RNN struc￾ture, and generates latent dynamics zt (blue circle) given hidden states. Both hidden states νt and latent dynamics zt contribute to νt+1. The latent states zt are mapped to observations xt (green circle) via a Gaussian process mapping function f. time steps to obtain fi ∈ R T . According to the defini￾tion of Gau… view at source ↗
Figure 2
Figure 2. Figure 2: Inference network for l-LSTM, r-LSTM and bi [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: A) True firing rates for 2 example neurons for orientation 0 ◦ averaged across 30 trials. We can tell there exists clear periodicity in the firing rate time series given the sinusoidal grating stimulus. B) 2-dimensional latent trajectories of 10 out of 30 trials using PfLDS, P-GPLVM and GP-RNN. Color denotes the phase of the grating stimulus implied in (A). Each circle corresponds to a period of latent dyn… view at source ↗
read the original abstract

Latent dynamics discovery is challenging in extracting complex dynamics from high-dimensional noisy neural data. Many dimensionality reduction methods have been widely adopted to extract low-dimensional, smooth and time-evolving latent trajectories. However, simple state transition structures, linear embedding assumptions, or inflexible inference networks impede the accurate recovery of dynamic portraits. In this paper, we propose a novel latent dynamic model that is capable of capturing nonlinear, non-Markovian, long short-term time-dependent dynamics via recurrent neural networks and tackling complex nonlinear embedding via non-parametric Gaussian process. Due to the complexity and intractability of the model and its inference, we also provide a powerful inference network with bi-directional long short-term memory networks that encode both past and future information into posterior distributions. In the experiment, we show that our model outperforms other state-of-the-art methods in reconstructing insightful latent dynamics from both simulated and experimental neural datasets with either Gaussian or Poisson observations, especially in the low-sample scenario. Our codes and additional materials are available at https://github.com/sheqi/GP-RNN_UAI2019.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes GP-RNN, a latent variable model for neural dynamics discovery that uses recurrent neural networks to capture nonlinear non-Markovian long short-term dynamics and Gaussian processes for flexible nonlinear embeddings from high-dimensional observations (Gaussian or Poisson). A bi-directional LSTM variational inference network is introduced to handle the resulting intractable posterior. Experiments on simulated and experimental neural datasets claim that the model outperforms prior state-of-the-art methods, especially in low-sample regimes.

Significance. If the empirical comparisons hold under rigorous controls, the architecture offers a principled way to relax linear/Markovian assumptions common in prior neural latent models while retaining nonparametric flexibility in the observation model. The public code release supports reproducibility and is a positive contribution.

major comments (2)
  1. [§4] §4 (Experiments): the central outperformance claim is stated in the abstract and §1 but the provided manuscript text supplies no quantitative metrics, error bars, dataset sizes, or ablation tables; without these the empirical contribution cannot be evaluated and the low-sample advantage remains unverified.
  2. [§3.2] §3.2 (Inference network): the bi-LSTM encoder is motivated by the need to encode past and future information, yet no derivation or bound is given showing that the resulting variational family is sufficiently expressive relative to the RNN-GP generative model; this directly affects whether the reported latent trajectories are reliable recoveries or artifacts of the inference approximation.
minor comments (2)
  1. [§2] Notation for the GP kernel and RNN transition function is introduced without an explicit equation reference; a single consolidated model diagram or equation block would improve readability.
  2. [Abstract] The GitHub link is given but no commit hash or exact reproduction script is cited; this should be added for long-term reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address each major comment below and indicate planned revisions to the manuscript.

read point-by-point responses

  1. Referee: [§4] §4 (Experiments): the central outperformance claim is stated in the abstract and §1 but the provided manuscript text supplies no quantitative metrics, error bars, dataset sizes, or ablation tables; without these the empirical contribution cannot be evaluated and the low-sample advantage remains unverified.

    Authors: We agree that the current text of §4 lacks explicit quantitative tables, error bars, and ablation results. In the revision we will add performance tables with means and standard deviations across multiple runs, explicit dataset sizes, and ablation studies focused on the low-sample regime to allow rigorous evaluation of the claims. revision: yes

  2. Referee: [§3.2] §3.2 (Inference network): the bi-LSTM encoder is motivated by the need to encode past and future information, yet no derivation or bound is given showing that the resulting variational family is sufficiently expressive relative to the RNN-GP generative model; this directly affects whether the reported latent trajectories are reliable recoveries or artifacts of the inference approximation.

    Authors: The bi-directional LSTM is selected to encode bidirectional temporal context into the variational posterior, which aligns with the non-Markovian generative process. While the manuscript does not derive a formal expressiveness bound, we will revise §3.2 to provide additional justification for the architecture and discuss its empirical adequacy relative to the generative model. revision: partial

Circularity Check

0 steps flagged

No significant circularity in derivation chain

full rationale

The paper introduces an RNN-GP latent dynamics model with bi-LSTM variational inference, motivated by limitations of prior methods in handling nonlinear non-Markovian dynamics and complex embeddings. Central results are empirical outperformance on simulated and experimental neural datasets (Gaussian/Poisson observations), with code released for reproducibility. No load-bearing step reduces by construction to a fitted parameter, self-definition, or self-citation chain; the architecture is explicitly constructed to address stated intractability issues rather than assuming the target result. This is a standard empirical modeling contribution with independent content.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides no explicit free parameters, axioms, or invented entities; model components are described at a high level without derivation details.

pith-pipeline@v0.9.0 · 5712 in / 959 out tokens · 29813 ms · 2026-05-25T12:35:35.540427+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

29 extracted references · 29 canonical work pages · 4 internal anchors

  1. [1]

    Gaussian-process factor analysis for low-dimensional single-trial analysis of neural pop- ulation activity

    M Yu Byron, John P Cunningham, Gopal San- thanam, Stephen I Ryu, Krishna V Shenoy, and Ma- neesh Sahani. Gaussian-process factor analysis for low-dimensional single-trial analysis of neural pop- ulation activity. In Advances in Neural Information Processing Systems (NeurIPS), pages 1881–1888, 2009

    work page 2009

  2. [2]

    Empirical models of spiking in neural popula- tions

    Jakob H Macke, Lars Buesing, John P Cunningham, M Yu Byron, Krishna V Shenoy, and Maneesh Sa- hani. Empirical models of spiking in neural popula- tions. In Advances in Neural Information Processing Systems (NeurIPS), pages 1350–1358, 2011

    work page 2011

  3. [3]

    High-dimensional neural spike train analysis with generalized count linear dynam- ical systems

    Yuanjun Gao, Lars Busing, Krishna V Shenoy, and John P Cunningham. High-dimensional neural spike train analysis with generalized count linear dynam- ical systems. In Advances in Neural Information Processing Systems (NeurIPS), pages 2044–2052, 2015

    work page 2044

  4. [4]

    Linear dynamical neural pop- ulation models through nonlinear embeddings

    Yuanjun Gao, Evan W Archer, Liam Paninski, and John P Cunningham. Linear dynamical neural pop- ulation models through nonlinear embeddings. In Advances in Neural Information Processing Systems (NeurIPS), pages 163–171, 2016

    work page 2016

  5. [5]

    Gaussian process based nonlin- ear latent structure discovery in multivariate spike train data

    Anqi Wu, Nicholas G Roy, Stephen Keeley, and Jonathan W Pillow. Gaussian process based nonlin- ear latent structure discovery in multivariate spike train data. In Advances in Neural Information Pro- cessing Systems (NeurIPS), pages 3499–3508, 2017

    work page 2017

  6. [6]

    Inferring single-trial neural population dynamics using sequential auto-encoders

    Chethan Pandarinath, Daniel J O’Shea, Jasmine Collins, Rafal Jozefowicz, Sergey D Stavisky, Jonathan C Kao, Eric M Trautmann, Matthew T Kaufman, Stephen I Ryu, Leigh R Hochberg, et al. Inferring single-trial neural population dynamics using sequential auto-encoders. Nature methods, page 1, 2018

    work page 2018

  7. [7]

    Reduced-rank linear dynamical systems

    Qi She, Yuan Gao, Kai Xu, and Rosa HM Chan. Reduced-rank linear dynamical systems. In Thirty- Second AAAI Conference on Artificial Intelligence (AAAI), 2018

    work page 2018

  8. [8]

    Structured inference networks for nonlinear state space models

    Rahul G Krishnan, Uri Shalit, and David Sontag. Structured inference networks for nonlinear state space models. In The Thirty-first AAAI Conference on Artificial Intelligence (AAAI), pages 2101–2109, 2017

    work page 2017

  9. [9]

    Stochastic dynami- cal systems based latent structure discovery in high- dimensional time series

    Qi She and Rosa HM Chan. Stochastic dynami- cal systems based latent structure discovery in high- dimensional time series. In 2018 IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), pages 886–890. IEEE, 2018

    work page 2018

  10. [10]

    Gaussian process latent variable models for visualisation of high dimensional data

    Neil D Lawrence. Gaussian process latent variable models for visualisation of high dimensional data. In Advances in Neural Information Processing Systems (NeurIPS), pages 329–336, 2004

    work page 2004

  11. [11]

    Poisson extension of gaussian pro- cess factor analysis for modeling spiking neural pop- ulations

    Hooram Nam. Poisson extension of gaussian pro- cess factor analysis for modeling spiking neural pop- ulations. Master’s thesis, Department of Neural Computation and Behaviour, Max Planck Institute for Biological Cybernetics, Tübingen, 2015

    work page 2015

  12. [12]

    Variance stabilizing transformations of poisson, binomial and negative binomial distribu- tions

    Guan Yu. Variance stabilizing transformations of poisson, binomial and negative binomial distribu- tions. Statistics & Probability Letters, 79(14):1621– 1629, 2009

    work page 2009

  13. [13]

    Stochastic variational inference

    Matthew D Hoffman, David M Blei, Chong Wang, and John Paisley. Stochastic variational inference. The Journal of Machine Learning Research (JMLR), 14(1):1303–1347, 2013

    work page 2013

  14. [14]

    Pattern recognition and machine learning

    M Bishop Christopher. Pattern recognition and machine learning. Springer-Verlag New York, 2016

    work page 2016

  15. [15]

    A recurrent latent variable model for sequential data

    Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C Courville, and Yoshua Ben- gio. A recurrent latent variable model for sequential data. In Advances in Neural Information Processing Systems (NeurIPS), pages 2980–2988, 2015

    work page 2015

  16. [16]

    DRAW: A Recurrent Neural Network For Image Generation

    Karol Gregor, Ivo Danihelka, Alex Graves, Danilo Jimenez Rezende, and Daan Wierstra. Draw: A recurrent neural network for image generation. arXiv preprint arXiv:1502.04623, 2015

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  17. [17]

    Variational gaussian process state-space models

    Roger Frigola, Yutian Chen, and Carl Edward Ras- mussen. Variational gaussian process state-space models. In Advances in Neural Information Process- ing Systems (NeurIPS), pages 3680–3688, 2014

    work page 2014

  18. [18]

    Collabo- rative multi-output gaussian processes

    Trung V Nguyen, Edwin V Bonilla, et al. Collabo- rative multi-output gaussian processes. In Annual Conference on Uncertainty in Artificial Intelligence (UAI), pages 643–652, 2014

    work page 2014

  19. [19]

    Recurrent Gaussian Processes

    César Lincoln C Mattos, Zhenwen Dai, Andreas Damianou, Jeremy Forth, Guilherme A Barreto, and Neil D Lawrence. Recurrent gaussian processes. arXiv preprint arXiv:1511.06644, 2015

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  20. [20]

    Computationally efficient bayesian learning of gaussian process state space models

    Andreas Svensson, Arno Solin, Simo Särkkä, and Thomas Schön. Computationally efficient bayesian learning of gaussian process state space models. In Artificial Intelligence and Statistics (AISTATS), 2016 International Conference on, pages 213–221, 2016

    work page 2016

  21. [21]

    Identification of gaussian process state space models

    Stefanos Eleftheriadis, Tom Nicholson, Marc Deisenroth, and James Hensman. Identification of gaussian process state space models. In Advances in Neural Information Processing Systems (NeurIPS), pages 5309–5319, 2017

    work page 2017

  22. [22]

    Black box variational inference

    Rajesh Ranganath, Sean Gerrish, and David Blei. Black box variational inference. In Artificial Intelli- gence and Statistics (AISTATS), 2014 International Conference on, pages 814–822, 2014

    work page 2014

  23. [23]

    Black box variational inference for state space models

    Evan Archer, Il Memming Park, Lars Buesing, John Cunningham, and Liam Paninski. Black box vari- ational inference for state space models. arXiv preprint arXiv:1511.07367, 2015

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  24. [24]

    Auto-Encoding Variational Bayes

    Diederik P Kingma and Max Welling. Auto- encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

    work page internal anchor Pith review Pith/arXiv arXiv 2013

  25. [25]

    Variational inference for latent variables and uncertain inputs in gaussian processes

    Andreas C Damianou, Michalis K Titsias, and Neil D Lawrence. Variational inference for latent variables and uncertain inputs in gaussian processes. The Journal of Machine Learning Research (JMLR), 17(1):1425–1486, 2016

    work page 2016

  26. [26]

    Hoffman, Rif A

    Dustin Tran, Matthew D. Hoffman, Rif A. Saurous, Eugene Brevdo, Kevin Murphy, and David M. Blei. Deep probabilistic programming. In International Conference on Learning Representations (ICLR) , 2017

    work page 2017

  27. [27]

    Variational latent gaussian process for recovering single-trial dynam- ics from population spike trains

    Yuan Zhao and Il Memming Park. Variational latent gaussian process for recovering single-trial dynam- ics from population spike trains. Neural Computa- tion, 29(5):1293–1316, 2017

    work page 2017

  28. [28]

    Bayesian learning and inference in recurrent switch- ing linear dynamical systems

    Scott Linderman, Matthew Johnson, Andrew Miller, Ryan Adams, David Blei, and Liam Paninski. Bayesian learning and inference in recurrent switch- ing linear dynamical systems. In Artificial Intelli- gence and Statistics (AISTATS), 2017 International Conference on, pages 914–922, 2017

    work page 2017

  29. [29]

    Decoding the activity of neuronal populations in macaque primary visual cortex

    Arnulf BA Graf, Adam Kohn, Mehrdad Jazayeri, and J Anthony Movshon. Decoding the activity of neuronal populations in macaque primary visual cortex. Nature neuroscience, 14(2):239, 2011

    work page 2011