pith. sign in

arxiv: 2410.13469 · v2 · submitted 2024-10-17 · 💻 cs.LG

Interpreting Temporal Graph Neural Networks with Koopman Theory

Michele Guerra , Simone Scardapane , Filippo Maria Bianchi This is my paper

Pith reviewed 2026-05-23 18:38 UTC · model grok-4.3

classification 💻 cs.LG
keywords spatiotemporal graph neural networksexplainabilityKoopman theorydynamic mode decompositionSINDyinterpretabilitytemporal graphsdynamical systems
0
0 comments X

The pith

Dynamic mode decomposition and sparse identification of nonlinear dynamics can interpret spatiotemporal graph neural networks by recovering their learned spatial and temporal patterns.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper applies two Koopman-inspired techniques to open up the decision process inside spatiotemporal graph neural networks. Dynamic mode decomposition reduces the learned dynamics to a set of modes that highlight dominant patterns, while sparse identification of nonlinear dynamics searches for governing equations that explain how those patterns evolve over the graph. On controlled dissemination data the extracted modes and equations match the exact times and locations of simulated infections, and on motion data they point to the joints that matter most for classifying actions. This supplies a post-hoc way to read out relevant input features without changing how the original network was trained.

Core claim

DMD and SINDy, when applied to the internal representations or outputs of an STGNN, identify the most relevant spatial and temporal patterns in the input; on semi-synthetic dissemination datasets they correctly recover the times at which infections occur and the infected nodes, while on a real human motion dataset the resulting explanations highlight the body parts most relevant for action recognition.

What carries the argument

Dynamic mode decomposition (DMD) and sparse identification of nonlinear dynamics (SINDy) applied to the learned representations of spatiotemporal graph neural networks to extract dominant spatial-temporal modes and governing equations.

If this is right

  • The methods recover the exact times and nodes of infection in dissemination datasets.
  • On motion data the explanations correctly isolate the body parts driving action recognition.
  • Both techniques operate directly on an already-trained STGNN without requiring retraining or architectural changes.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same pipeline could be run on traffic or epidemic forecasting STGNNs to surface which sensors or locations dominate predictions at specific future steps.
  • If the recovered equations prove stable across retrainings, they could serve as a lightweight surrogate model for quick what-if simulations.
  • Applying SINDy to graph sequences might allow discovery of explicit dynamical rules that could later be inserted back into hybrid neural-symbolic architectures.

Load-bearing premise

Applying DMD and SINDy to the neural network's internal states or outputs will recover the true underlying spatial-temporal drivers without major distortion or spurious modes introduced by the network's learned approximation.

What would settle it

On a semi-synthetic dissemination dataset with known ground-truth infection times and nodes, check whether the modes and equations extracted by DMD and SINDy match those known values.

Figures

Figures reproduced from arXiv: 2410.13469 by Filippo Maria Bianchi, Michele Guerra, Simone Scardapane.

Figure 1
Figure 1. Figure 1: The left-hand side, in green, depicts an example of TG, with node features [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparison between the distribution fgt and the distribution fr for the Facebook dataset. 6 [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Examples of time explanations for the Facebook dataset. The red line represents the smoothed [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Spatial explanations from Facebook dataset. The colour scale on either nodes or edges represents [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Top: The colour scale represents the explanation w (i) G (t, n) for each time step (x axis) and each node (y axis). The red squares for some values of t and n show where mst(t, n) = 1. Middle: They show the TG G at three times, t = 50, 80, 100. Nodes in the ground truth are highlighted in red. Bottom: It shows the value of the Brier score BS(t). 13 [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗
read the original abstract

Spatiotemporal graph neural networks (STGNNs) have shown promising results in many domains, from forecasting to epidemiology. However, understanding the dynamics learned by these models and explaining their behaviour is significantly more difficult than for models that deal with static data. Inspired by Koopman theory, which allows a simple description of intricate, nonlinear dynamical systems, we introduce new explainability approaches for temporal graphs. Specifically, we present two methods to interpret the STGNN's decision process and identify the most relevant spatial and temporal patterns in the input for the task at hand. The first relies on dynamic mode decomposition (DMD), a Koopman-inspired dimensionality reduction method. The second relies on sparse identification of nonlinear dynamics (SINDy), a popular method for discovering governing equations of dynamical systems, which we use for the first time as a general tool for explainability. On semi-synthetic dissemination datasets, our methods correctly identify interpretable features such as the times at which infections occur and the infected nodes. We also validate the methods qualitatively on a real-world human motion dataset, where the explanations highlight the body parts most relevant for action recognition.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper proposes two post-hoc explainability methods for spatiotemporal graph neural networks (STGNNs) inspired by Koopman theory: dynamic mode decomposition (DMD) and sparse identification of nonlinear dynamics (SINDy, applied here for the first time as a general explainability tool). The methods aim to identify the most relevant spatial and temporal patterns in the input. On semi-synthetic dissemination datasets the authors claim the methods correctly recover interpretable features such as infection times and infected nodes; qualitative validation is also presented on a real-world human motion dataset for action recognition.

Significance. If the central claim holds under quantitative scrutiny, the work would provide a novel, principled route to interpreting STGNNs by leveraging established dynamical-systems tools, which is valuable for domains such as epidemiology and motion analysis. The first-time use of SINDy for general explainability is a clear strength. However, the current evaluation is entirely qualitative and supplies no metrics, baselines, or controls, so the immediate significance remains limited.

major comments (2)
  1. [Abstract, §4 (experiments)] Abstract and experimental section: the claim that the methods 'correctly identify' infection times and nodes on semi-synthetic data is supported only by qualitative description. No quantitative metrics (precision/recall on recovered nodes/times, error bars, ablation studies, or comparison to other explainability baselines) are reported, leaving the central assertion unverifiable.
  2. [§3] Methods (§3): DMD and SINDy are applied to STGNN internal representations or outputs under the assumption that they recover the true underlying spatial-temporal drivers without significant distortion from the neural network's learned nonlinear mapping. No analysis, theorem, or experiment addresses the possibility that the STGNN induces a different Koopman operator, producing spurious modes instead of ground-truth features.
minor comments (1)
  1. [§3] Notation for the Koopman operator and the precise mapping from STGNN hidden states to DMD/SINDy inputs could be clarified with an explicit diagram or pseudocode.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive report. We agree that the current evaluation would benefit from quantitative metrics and additional analysis of methodological assumptions. We address each major comment below and indicate the planned revisions.

read point-by-point responses

  1. Referee: [Abstract, §4 (experiments)] Abstract and experimental section: the claim that the methods 'correctly identify' infection times and nodes on semi-synthetic data is supported only by qualitative description. No quantitative metrics (precision/recall on recovered nodes/times, error bars, ablation studies, or comparison to other explainability baselines) are reported, leaving the central assertion unverifiable.

    Authors: We acknowledge that the evaluation presented is qualitative, relying on visual inspection of recovered features against known ground truth in the semi-synthetic dissemination datasets. While this design permits direct verification of whether infection times and nodes are identified, we agree that quantitative support is needed to make the central claims verifiable. In the revised manuscript we will add precision and recall metrics for recovered nodes and times, include error bars from multiple runs, conduct ablation studies on key parameters, and provide comparisons against existing explainability baselines for graph models. revision: yes

  2. Referee: [§3] Methods (§3): DMD and SINDy are applied to STGNN internal representations or outputs under the assumption that they recover the true underlying spatial-temporal drivers without significant distortion from the neural network's learned nonlinear mapping. No analysis, theorem, or experiment addresses the possibility that the STGNN induces a different Koopman operator, producing spurious modes instead of ground-truth features.

    Authors: The concern is valid: the manuscript applies DMD and SINDy post-hoc without a formal theorem or dedicated experiment ruling out distortion from the STGNN's nonlinear mapping. The empirical recovery of known features on semi-synthetic data offers supporting evidence, but does not constitute a general guarantee. We will revise §3 to include an explicit discussion of this assumption, the conditions under which spurious modes could arise, and additional experiments testing robustness across different STGNN architectures and noise levels. revision: partial

Circularity Check

0 steps flagged

No circularity: post-hoc DMD/SINDy applied to trained models with external ground-truth validation

full rationale

The paper applies DMD and SINDy post-hoc to internal representations or outputs of already-trained STGNNs. Claims of 'correctly identify[ing]' infection times/nodes rest on comparison to known ground truth in semi-synthetic dissemination datasets, which is independent of the STGNN training and not a fitted quantity renamed as prediction. No self-definitional steps, fitted-input-called-prediction, load-bearing self-citations, uniqueness theorems, or ansatz smuggling appear in the abstract or described methodology. The derivation chain for the explainability methods is self-contained against external benchmarks and does not reduce to its inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the applicability of Koopman operator theory and its data-driven approximations (DMD, SINDy) to the latent dynamics of a trained STGNN without additional assumptions about linearity or sparsity being violated by the neural network. No free parameters or invented entities are mentioned in the abstract.

axioms (1)
  • domain assumption Koopman theory provides a linear representation of nonlinear dynamical systems that can be recovered from data via DMD or SINDy.
    Invoked when the paper states it is 'Inspired by Koopman theory, which allows a simple description of intricate, nonlinear dynamical systems' and applies the methods to STGNN decision processes.

pith-pipeline@v0.9.0 · 5726 in / 1470 out tokens · 21467 ms · 2026-05-23T18:38:45.944540+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 39 canonical work pages

  1. [1]

    Altieri, M

    M. Altieri, M. Ceci, and R. Corizzo. Explain- able spatio-temporal graph modeling. In In- ternational Conference on Discovery Science , pages 174–188. Springer, 2023

    work page 2023

  2. [2]

    Arbabi and I

    H. Arbabi and I. Mezi´ c. Ergodic theory, dy- namic mode decomposition, and computation of spectral properties of the koopman operator. SIAM Journal on Applied Dynamical Systems , 16(4):2096–2126, 2017

    work page 2096

  3. [3]

    S. L. Brunton, M. Budiˇ si´ c, E. Kaiser, and J. N. Kutz. Modern koopman theory for dy- namical systems. SIAM Review, 64(2):229–340, 2022

    work page 2022

  4. [4]

    S. L. Brunton and J. N. Kutz. Data-Driven Sci- ence and Engineering: Machine Learning, Dy- namical Systems, and Control. Cambridge Uni- versity Press, 2 edition, 2022

    work page 2022

  5. [5]

    S. L. Brunton, J. L. Proctor, and J. N. Kutz. Discovering governing equations from data by sparse identification of nonlinear dynamical sys- tems. Proceedings of the national academy of sciences, 113(15):3932–3937, 2016

    work page 2016

  6. [6]

    L. Chen, Y. Yang, A. Yu, S. Guo, K. Ren, Q. Liu, and C. Qiao. An explainable spatio- temporal graph convolutional network for the biomarkers identification of adhd. Biomedical Signal Processing and Control, 99:106913, 2025

    work page 2025

  7. [7]

    A. Cini, D. Mandic, and C. Alippi. Graph-based Time Series Clustering for End-to-End Hierar- chical Forecasting. International Conference on Machine Learning, 2024

    work page 2024

  8. [8]

    A. Cini, I. Marisca, F. M. Bianchi, and C. Alippi. Scalable spatiotemporal graph neural networks. In Proceedings of the AAAI confer- ence on artificial intelligence , volume 37, pages 7218–7226, 2023

    work page 2023

  9. [9]

    A. Cini, I. Marisca, D. Zambon, and C. Alippi. Graph deep learning for time series forecasting. arXiv preprint arXiv:2310.15978 , 2023

    work page arXiv 2023

  10. [10]

    A. Cini, D. Zambon, and C. Alippi. Sparse graph learning from spatiotemporal time se- ries. Journal of Machine Learning Research , 24(242):1–36, 2023

    work page 2023

  11. [11]

    N. Demo, M. Tezzele, and G. Rozza. Pydmd: Python dynamic mode decomposition. Journal of Open Source Software, 3(22):530, 2018

    work page 2018

  12. [12]

    S. Deng, H. Rangwala, and Y. Ning. Learn- ing dynamic context graphs for predicting so- cial events. In Proceedings of the 25th ACM SIGKDD International Conference on Knowl- edge Discovery & Data Mining , pages 1007– 1016, 2019

    work page 2019

  13. [13]

    Fritz, E

    C. Fritz, E. Dorigatti, and D. R¨ ugamer. Combining graph neural networks and spatio- temporal disease models to improve the predic- tion of weekly covid-19 cases in germany. Sci- entific Reports, 12(1):3930, 2022. 8 0.0 0.2 0.4 0.6 0.8 1.0 w(i) s (n) (a) Node mask with AUC node = 1. 0.0 0.2 0.4 0.6 0.8 1.0 we(n, m) (b) Edges mask with AUC (3) edge = 1. F...

    work page 2022

  14. [14]

    Hassija, V

    V. Hassija, V. Chamola, A. Mahapatra, A. Sin- gal, D. Goel, K. Huang, S. Scardapane, I. Spinelli, M. Mahmud, and A. Hussain. In- terpreting black-box models: a review on ex- plainable artificial intelligence. Cognitive Com- putation, 16(1):45–74, 2024

    work page 2024

  15. [15]

    S. M. Ichinaga, F. Andreuzzi, N. Demo, M. Tezzele, K. Lapo, G. Rozza, S. L. Brunton, and J. N. Kutz. Pydmd: A python package for robust dynamic mode decomposition. arXiv preprint arXiv:2402.07463, 2024

    work page arXiv 2024

  16. [16]

    Isella, J

    L. Isella, J. Stehl´ e, A. Barrat, C. Cattuto, J.- F. Pinton, and W. Van den Broeck. What’s in a crowd? analysis of face-to-face behav- ioral networks. Journal of theoretical biology , 271(1):166–180, 2011

    work page 2011

  17. [17]

    A. Jain, A. R. Zamir, S. Savarese, and A. Sax- ena. Structural-rnn: Deep learning on spatio- temporal graphs. In Proceedings of the ieee con- ference on computer vision and pattern recogni- tion, pages 5308–5317, 2016

    work page 2016

  18. [18]

    T. N. Kipf and M. Welling. Semi-supervised classification with graph convolutional net- works. In International Conference on Learning Representations, 2017

    work page 2017

  19. [19]

    B. O. Koopman. Hamiltonian systems and transformation in hilbert space. Proceedings of the National Academy of Sciences , 17(5):315– 318, 1931

    work page 1931

  20. [20]

    J. N. Kutz, S. L. Brunton, B. W. Brunton, and J. L. Proctor. Dynamic mode decomposi- tion: data-driven modeling of complex systems . SIAM, 2016

    work page 2016

  21. [21]

    Leskovec, L

    J. Leskovec, L. Backstrom, and J. Kleinberg. Meme-tracking and the dynamics of the news cycle. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discov- ery and data mining , pages 497–506, 2009

    work page 2009

  22. [22]

    Q. Li, F. Dietrich, E. M. Bollt, and I. G. Kevrekidis. Extended dynamic mode decompo- sition with dictionary learning: A data-driven adaptive spectral decomposition of the koop- man operator. Chaos: An Interdisciplinary Journal of Nonlinear Science , 27(10), 2017

    work page 2017

  23. [23]

    Longa, V

    A. Longa, V. Lachi, G. Santin, M. Bianchini, B. Lepri, P. Lio, F. Scarselli and A. Passerini Graph Neural Networks for Temporal Graphs: State of the Art, Open Challenges, and Oppor- tunities. Transactions on Machine Learning Re- search, 2835-8856, 2023

    work page 2023

  24. [24]

    Lusch, J

    B. Lusch, J. N. Kutz, and S. L. Brunton. Deep learning for universal linear embeddings of nonlinear dynamics. Nature communications, 9(1):4950, 2018

    work page 2018

  25. [25]

    Manojlovi´ c, M

    I. Manojlovi´ c, M. Fonoberova, R. Mohr, A. An- drejˇ cuk, Z. Drmaˇ c, Y. Kevrekidis, and I. Mezi´ c. Applications of koopman mode analysis to neu- ral networks. arXiv preprint arXiv:2006.11765, 2020. 9

    work page arXiv 2006

  26. [26]

    Marisca, C

    I. Marisca, C. Alippi, and F. M. Bianchi. Graph-based forecasting with missing data through spatiotemporal downsampling. In Pro- ceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research , pages 34846– 34865. PMLR, 2024

    work page 2024

  27. [27]

    Melnyk, S

    K. Melnyk, S. Klus, G. Montavon, and T. O. Conrad. Graphkke: graph kernel koopman em- bedding for human microbiome analysis. Ap- plied Network Science, 5:1–22, 2020

    work page 2020

  28. [28]

    Melnyk, K

    K. Melnyk, K. Weimann, and T. O. Conrad. Understanding microbiome dynamics via inter- pretable graph representation learning. Scien- tific Reports, 13(1):2058, 2023

    work page 2058

  29. [29]

    I. Mezi´ c. Koopman operator, geome- try, and learning of dynamical systems. Not. Am. Math. Soc. , 68(7):1087–1105, 2021

    work page 2021

  30. [30]

    Micheli and D

    A. Micheli and D. Tortorella. Discrete-time dy- namic graph echo state networks. Neurocom- puting, 496:85–95, 2022

    work page 2022

  31. [31]

    Naiman and O

    I. Naiman and O. Azencot. An operator the- oretic approach for analyzing sequence neural networks. In Proceedings of the AAAI confer- ence on artificial intelligence , volume 37, pages 9268–9276, 2023

    work page 2023

  32. [32]

    Oettershagen, N

    L. Oettershagen, N. M. Kriege, C. Morris, and P. Mutzel. Temporal Graph Kernels for Classi- fying Dissemination Processes , pages 496–504. 2020

    work page 2020

  33. [33]

    P. J. Schmid. Dynamic mode decomposition of numerical and experimental data. Journal of fluid mechanics, 656:5–28, 2010

    work page 2010

  34. [34]

    P. J. Schmid. Dynamic mode decomposition and its variants. Annual Review of Fluid Me- chanics, 54(1):225–254, 2022

    work page 2022

  35. [35]

    Y. Seo, M. Defferrard, P. Vandergheynst, and X. Bresson. Structured sequence modeling with graph convolutional recurrent networks. In Neural Information Processing: 25th Interna- tional Conference, ICONIP 2018, Siem Reap, Cambodia, December 13-16, 2018, Proceedings, Part I 25 , pages 362–373. Springer, 2018

    work page 2018

  36. [36]

    J. Tang, L. Xia, and C. Huang. Explainable spatio-temporal graph neural networks. In Pro- ceedings of the 32nd ACM International Con- ference on Information and Knowledge Man- agement, pages 2432–2441, 2023

    work page 2023

  37. [37]

    Verdone, S

    A. Verdone, S. Scardapane, and M. Panella. Explainable spatio-temporal graph neural net- works for multi-site photovoltaic energy produc- tion. Applied Energy, 353:122151, 2024

    work page 2024

  38. [38]

    Viswanath, A

    B. Viswanath, A. Mislove, M. Cha, and K. P. Gummadi. On the evolution of user interaction in facebook. In Proceedings of the 2nd ACM workshop on Online social networks , pages 37– 42, 2009

    work page 2009

  39. [39]

    Zhang, J

    Q. Zhang, J. Chang, G. Meng, S. Xiang, and C. Pan. Spatio-temporal graph structure learn- ing for traffic forecasting. In Proceedings of the AAAI conference on artificial intelligence , vol- ume 34, pages 1177–1185, 2020. 10 Supplementary Materials A Extension to non-autonomous systems The dynamical system implied by the GCRN ht+1 = GCRN(ht, xt, At) (27) ...

    work page 2020