Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

T-Graphormer: Using Transformers for Spatiotemporal Forecasting

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read T-Graphormer claims a single Transformer, with flattened graph tokens plus temporal and structural encodings, learns space and time together and beats separate spatial-temporal models on traffic benchmarks.

desk verdict A simple, well-described Graphormer extension with plausible gains, but the headline SOTA margins rest on baseline comparability that is asserted, not demonstrated. read the letter →

arxiv 2501.13274 v3 pith:WZSJMQH5 submitted 2025-01-22 cs.LG

Hao Yuan Bai , Xue Liu This is my paper · ORCID
classification cs.LG
keywords spatiotemporalforecastingtrafficpredictionTransformersGraphormergraphstructureencodingattentionmechanismmultivariatetimeseriesPEMS-BAY
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

T-Graphormer is an attempt to show that spatiotemporal forecasting does not need separate spatial and temporal learning modules. The paper flattens a graph-structured traffic sequence into one token stream and lets every node attend to every other node at every time step, adding temporal and structural encodings on top. On the PEMS-BAY and METR-LA benchmarks it reports large gains, cutting RMSE by 24% and 12% respectively against state-of-the-art baselines. If the comparison holds up, the result matters because it suggests a single unified Transformer with minimal inductive bias can learn time and space dependencies directly from the data.

What carries the argument

The central object is the flattened graph sequence as a Transformer token stream, with three Graphormer-style encodings: per-token learnable positional encoding, node-centrality embeddings indexed by in/out degree, and a learnable attention bias $b_{\phi(i,j)}$ indexed by shortest path distance between nodes. A special cls token is prepended as a supernode. The attention score between tokens $(t_1,i)$ and $(t_2,j)$ is the standard query-key similarity plus $b_{\phi(i,j)}$, so structure enters only through additive biases while the receptive field stays global across both time and space.

What would settle it

Re-run every baseline in Table 1 under the exact DCRNN preprocessing, same 70/20/10 chronological split, same horizon definitions, and same Z-score normalization; if the reported RMSE reductions (24% on PEMS-BAY and 12% on METR-LA) do not reproduce, the state-of-the-art claim fails.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that extending Graphormer's structural encodings into the temporal dimension produces a competitive spatiotemporal forecaster. After flattening the observed graph sequence, T-Graphormer adds centrality encoding and learnable positional encoding to each token, and injects a shortest-path-distance attention bias between node pairs. It reports the best MAE, RMSE, and MAPE among ten baselines at the 30-minute and 1-hour horizons on both traffic datasets, with the largest gains at horizon 12. Attention heatmaps and ablations are offered as evidence that the model learns spatial and temporal structure rather than receiving it as a hard architectural constraint.

Load-bearing premise

The state-of-the-art result assumes the ten baseline models were evaluated under the same preprocessing and data split as T-Graphormer; the paper cites DCRNN preprocessing but does not report re-running each baseline under it.

Editorial extensions

If this is right

  • On the two traffic benchmarks, a single unified attention module beats models that alternate spatial and temporal convolutions or attentions.
  • Using one model trained with horizon-12 loss and evaluated by trimming tokens gives strong results at horizons 3 and 6 as well.
  • Removing any of the three encodings degrades performance, so structural and temporal biases still matter even in a global-attention design.
  • The learned attention heatmaps show selective node-node and time-time patterns, indicating spatiotemporal structure is discoverable without explicit spatial or temporal modules.
  • The method's memory cost grows quadratically with the flattened sequence length, limiting it to small networks and short contexts without sparse attention.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The recipe is graph-agnostic: any multivariate series whose components sit on a fixed graph, such as power grids, environmental sensors, or supply chains, could use the same flatten-and-encode treatment; the paper only evaluates traffic.
  • A direct ablation of the flattening order itself would test whether the gains come from the unified attention or simply from the encodings; the paper does not run that comparison.
  • The attention analysis could be turned into a diagnostic tool for traffic management, pointing at the sensors and lagged time steps that most influence a forecast; the paper stops at visualization.
  • On larger networks the quadratic context cost would force sparse attention; whether the learned spatiotemporal patterns survive sparsification is an open question the paper does not answer.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes T-Graphormer, an extension of the Graphormer architecture for spatiotemporal forecasting. The input is flattened into a token sequence of length T' × N, and Graphormer-style structural encodings---centrality encoding, spatial encoding based on shortest-path distances, and learnable positional encodings---are added to the node features and attention scores. The architecture has no separate spatial and temporal learning modules, which is the paper's central architectural claim. The authors evaluate T-Graphormer on PEMS-BAY and METR-LA, reporting state-of-the-art results across all horizons in Table 1, and on three additional PEMS datasets in Table 6. They also provide ablation studies on the encoding components and special tokens, an attention-score analysis, and a discussion of model-size variants. The paper states that code, datasets, and pretrained weights will be released.

Significance. If the empirical claims are correct, the paper makes a useful point: flattening spatiotemporal data and applying a single Transformer with graph-structural and temporal encodings can be competitive with, or beat, specialized architectures that alternate spatial and temporal modules. This would be a meaningful contribution to the traffic-forecasting literature and to the broader question of how much inductive bias is needed for structured-sequence Transformers. The paper also ships several strengths: a clear architecture description, ablation experiments isolating each encoding component, an analysis of learned attention patterns, and a reproducibility statement with a promised code release. However, the novelty over Graphormer is incremental---the main new ingredient is the joint temporal positional encoding and the application domain---and the headline SOTA claim rests on a comparability assumption about baselines that is not demonstrated. The central architecture claim is plausible, but the quantitative SOTA claim is not yet established to the standard required for publication.

major comments (4)
  1. [§5.1, Table 1] The central SOTA claim is stated in §5.3 as reducing RMSE by 24% and 12% on PEMS-BAY and METR-LA, but the comparability of baseline numbers in Table 1 is not established. Section 5.1 says the authors follow the DCRNN preprocessing pipeline for their own model, but it never states that any baseline was rerun under that same pipeline. The cited values for VAR, FC-LSTM, DCRNN, STGCN, Graph WaveNet, ASTGCN, GMAN, PDFormer, STAEformer, and STEP are likely taken from different original papers that used different data splits (e.g., DCRNN's 7:2:1 vs. STSGCN's 6:2:2), different inclusion of time-of-day features, different normalization, and possibly different evaluation details. Without a statement that each baseline was evaluated with identical preprocessing, splits, and evaluation code, the 24%/12% RMSE margins are not reliable. Please state explicitly which numbers were rerun and which were cited, and ideally release the evaluation code so that the comparison is reproducible.
  2. [Table 1, §5.2] Table 1 reports only point estimates with no error bars or multiple seeds. Given that model configurations are selected on validation performance (§5.2) and the reported margins over second-best baselines are sometimes small (e.g., METR-LA horizon 3, where T-Graphormer's MAE is 2.62 vs. STEP's 2.61), single runs are insufficient to establish that the differences are statistically meaningful. Please report means and standard deviations over at least three independent runs for T-Graphormer and for the baselines that are rerun, and state the number of seeds used.
  3. [§5.2, horizons 3/6] The evaluation protocol for different horizons is asymmetric with respect to many baselines. Section 5.2 states that T-Graphormer is trained once with T=12 and that horizons 3 and 6 are evaluated by 'simply remov[ing] extra token values.' Many traffic-forecasting baselines are trained separately for each horizon. This protocol difference can change results nontrivially and should be disclosed in the table caption and controlled for in the comparison. At minimum, state which baselines were trained per horizon and which were trained for T=12 only, and discuss the impact on the reported numbers.
  4. [§5.3, Table 2] The paragraph citing Kaplan et al.'s scaling law D ∝ N^0.74 and claiming that the experimental datasets are 'only optimal for training the mini models' is not supported by the evidence presented. Table 2 contains three model sizes on two datasets, with no compute-budget sweeps, no loss-vs-compute curves, and no controlled variation of dataset size; applying an LLM scaling law to traffic forecasting in this way is an unjustified extrapolation. The claim that the result is 'confirmed' by Table 2 should be removed or replaced with a more modest statement that larger models overfit on these datasets, which is already visible in Figure 10.
minor comments (6)
  1. [Abstract vs. §5.3] The abstract states that T-Graphormer reduces RMSE and MAPE by 'up to 20% and 10%,' while Section 5.3 reports reductions of 24%/12% for RMSE and 13%/10% for MAPE. These numbers should be aligned.
  2. [§5.2] The baseline list in Section 5.2 omits DCRNN, STGCN, Graph WaveNet, and ASTGCN even though these methods appear in Table 1. Please make the list consistent with the reported comparisons.
  3. [Table 4, §5.3] Table 4 lists PEMS-BAY with std 19.49 and METR-LA with std 9.44, but Section 5.3 states that METR-LA has std 19.49 and PEMS-BAY has std 9.44. One of these is wrong; please correct the inconsistency.
  4. [Equation (7)] In Equation (7), the second key term is written as (h_{t2,i} W_K)^T; it should be (h_{t2,j} W_K)^T to match the stated token indices (t2, j).
  5. [Appendix A.1.1] The sentence 'This is not selectivity is not present in the first layer' contains a grammatical error and should be rewritten, e.g., 'This selectivity is not present in the first layer.'
  6. [Appendix tables] The appendix contains typos such as 'weigh decay' in Tables 3 and 7; please correct to 'weight decay.'

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: T-Graphormer's central claims are empirical benchmark results evaluated on held-out test data, not derivations from fitted inputs.

full rationale

The paper's central claims are empirical: T-Graphormer is an architecture built by adding temporal encodings to Graphormer, and the claimed SOTA margins in Table 1 are test-set numbers compared with cited baselines. No analytic quantity is derived from fitted parameters: the model is trained on the training split, selected by validation MAE, and reported on the held-out test split; this is standard model selection, not a fitted-input-called-prediction loop. Structural encodings are adopted transparently from Graphormer by citation, but they are architectural ingredients, not a conclusion that is then used to prove the architecture's effectiveness; the effectiveness claim rests on the benchmark comparison. The ablation study removes components and measures error changes, and the scaling-law remark uses an external formula (Kaplan et al., 2020) to rationalize the observed model-size ordering; even if that interpretation is debatable, confirming an external prediction on one's own runs is not circular. Concerns that baseline numbers may stem from different preprocessing pipelines are a measurement/comparability risk, not circularity: the paper states it follows DCRNN preprocessing, but it does not claim to have rerun all baselines. Internal inconsistencies (e.g., abstract 20%/10% vs. Section 5.3 24%/12%; Table 4 std values) affect reliability but do not make any derivation equivalent to its inputs. There are no load-bearing self-citations, uniqueness imports, or ansatz-smuggling steps.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on learned encodings (positional, centrality, spatial bias) whose architectures and hyperparameters are chosen on validation, plus a fixed graph construction inherited from DCRNN and cited baseline numbers. There are no invented physical entities. The main untested premise is that the fixed graph and the cited baselines make the benchmark comparison fair.

free parameters (5)
  • Architecture size and training hyperparameters = d in {64,128,192}; k in {6,6,8}; heads in {2,4,6}; learning rates 1e-3 to 3e-3; batch size 128; dropout 0.1…
    Selected by lowest validation MAE across three model variants; no derivation is given.
  • Learnable positional embedding P = Learned matrix R^{(T'N) x d}
    Added to all tokens; ablation shows MAE increases by 19% when removed.
  • Centrality embedding Z = Learned R^{deg x d}
    Indexed by node degree; ablation shows MAE increases by 9% when removed.
  • Spatial attention bias B = Learned scalar per shortest-path-distance value and attention head
    Computed from the fixed graph via phi(i,j)=SPD; ablation shows MAE increases by 5% when removed.
  • Huber loss delta = 1.5
    Set by hand for all experiments; not derived from data or theory.
assumptions (4)
  • domain assumption Thresholded Gaussian kernel adjacency W faithfully represents spatial relations among sensors.
    Appendix A.2 defines W=exp(-dist squared / sigma squared) for dist <= kappa, following Li et al. (2018); this fixed graph defines the spatial encoding in Equation 7.
  • ad hoc to paper Flattening T' x N tokens and using full attention preserves the structure needed for forecasting once encodings are added.
    Central architectural premise; supported only indirectly by ablations and benchmark wins, with no formal guarantee.
  • domain assumption Published baseline numbers are comparable under the same preprocessing.
    Baselines are cited from prior papers; the paper does not report rerunning all baselines under its exact split, normalization, and horizon evaluation.
  • domain assumption The traffic sensor graph is static over the forecast window.
    The model assumes fixed G=(V,E,W); the conclusion notes that dynamic graphs are out of scope.

how reviews work

0 comments
Cite this review

Pith. "Pith review of T-Graphormer: Using Transformers for Spatiotemporal Forecasting." pith.science (2026). https://pith.science/paper/WZSJMQH5

@misc{pith2026250113274,
  author       = {Pith},
  title        = {Pith review of: T-Graphormer: Using Transformers for Spatiotemporal Forecasting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WZSJMQH5}},
  note         = {Machine review of arXiv:2501.13274}
}
read the original abstract

Spatiotemporal data is ubiquitous, and forecasting it has important applications in many domains. However, its complex cross-component dependencies and non-linear temporal dynamics can be challenging for traditional techniques. Existing methods address this by learning the two dimensions separately. Here, we introduce Temporal Graphormer (T-Graphormer), a Transformer-based approach capable of modelling spatiotemporal correlations simultaneously. By adding temporal encodings in the Graphormer architecture, each node attends to all other tokens within the graph sequence, enabling the model to learn rich spacetime patterns with minimal predefined inductive biases. We show the effectiveness of T-Graphormer on real-world traffic prediction benchmark datasets. Compared to state-of-the-art methods, T-Graphormer reduces root mean squared error (RMSE) and mean absolute percentage error (MAPE) by up to 20% and 10%.

Figures

Figures reproduced from arXiv: 2501.13274 by the authors.

Figure 1
Figure 1. Visualization of a multivariate time se [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Information flow differences between existing methods and T-Graphormer. In part (a), the [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. T-Graphormer model architecture. Centrality encoding and positional encoding are added [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Attention heatmap averaged across time and space shows T-Graphormer learns spatial and [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Ablation results on the METR-LA dataset with a forecasting horizon of 12. Ablation [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Ablation results on the METR-LA dataset with a forecasting horizon of 12. Left: Struc [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Attention heatmap reveals T-Graphormer learns spatial and temporal relationships simul [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Validation metrics on two traffic prediction datasets. The best-performing model is dis [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Training and validation loss of the best-performing models on two traffic prediction [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: Validation and training loss differences of the best-performing models on the respective [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Prediction visualizations of T-Graphormer on PEMS-BAY. In the selected sensors shown [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Prediction visualizations of T-Graphormer on METR-LA. [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Fourier Geometric Wind Power Forecasting with Numerical Weather Prediction

    cs.AI 2026-07 conditional novelty 6.0 of 10

    A multimodal model coupling SCADA measurements with numerical weather prediction via rotation-invariant geometric features and Fourier neural operator fusion outperforms more than 20 baselines on three UK wind farms.

Reference graph

Works this paper leans on

56 extracted references · 27 canonical work pages · cited by 1 Pith paper

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Diffusion-convolutional neural networks

    James Atwood and Don Towsley. Diffusion-convolutional neural networks. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016. URL https://proceedings.neurips.cc/paper_files/paper/2016/file/390e982518a50e280d8e2b535462ec1f-Paper.pdf

    work page 2016

  3. [3]

    Layer normalization

    Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450, 2016

    arXiv 2016

  4. [4]

    Longformer: The long-document transformer

    Iz Beltagy, Matthew E Peters, and Arman Cohan. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150, 2020

    arXiv 2004

  5. [5]

    Probabilistic demand forecasting at scale

    Joos-Hendrik B \"o se, Valentin Flunkert, Jan Gasthaus, Tim Januschowski, Dustin Lange, David Salinas, Sebastian Schelter, Matthias Seeger, and Yuyang Wang. Probabilistic demand forecasting at scale. Proceedings of the VLDB Endowment, 10 0 (12): 0 1694--1705, 2017

    2017

  6. [6]

    Time series analysis: forecasting and control

    George EP Box, Gwilym M Jenkins, Gregory C Reinsel, and Greta M Ljung. Time series analysis: forecasting and control. John Wiley & Sons, 2015

    2015

  7. [7]

    Introduction to Time Series and Forecasting

    Peter J Brockwell and Richard A Davis. Introduction to Time Series and Forecasting. Springer, 2002

    work page 2002

  8. [8]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33: 0 1877--1901, 2020

    1901

Show all 56 references
  1. [9]

    Freeway performance measurement system: mining loop detector data

    Chao Chen, Karl Petty, Alexander Skabardonis, Pravin Varaiya, and Zhanfeng Jia. Freeway performance measurement system: mining loop detector data. Transportation research record, 1748 0 (1): 0 96--102, 2001

    2001

  2. [10]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  3. [11]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Con...

    2020

  4. [12]

    A generalization of transformer networks to graphs

    Vijay Prakash Dwivedi and Xavier Bresson. A generalization of transformer networks to graphs. AAAI Workshop on Deep Learning on Graphs: Methods and Applications, 2020

    2020

  5. [13]

    Masked autoencoders as spatiotemporal learners

    Christoph Feichtenhofer, Yanghao Li, Kaiming He, et al. Masked autoencoders as spatiotemporal learners. Advances in Neural Information Processing Systems, 35: 0 35946--35958, 2022

    2022

  6. [14]

    Long-range transformers for dynamic spatiotemporal forecasting

    Jake Grigsby, Zhe Wang, Nam Nguyen, and Yanjun Qi. Long-range transformers for dynamic spatiotemporal forecasting. arXiv preprint arXiv:2109.12218, 2021

    2021 arXiv

  7. [15]

    Attention based spatial-temporal graph convolutional networks for traffic flow forecasting

    Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp.\ 922--929, 2019

    2019

  8. [16]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 770--778, 2016. doi:10.1109/CVPR.2016.90

    2016 doi

  9. [17]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll \'a r, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 16000--16009, 2022

    2022

  10. [18]

    Long short-term memory

    Sepp Hochreiter and J \"u rgen Schmidhuber. Long short-term memory. Neural computation, 9 0 (8): 0 1735--1780, 1997

    1997

  11. [19]

    Robust estimation of a location parameter

    Peter J Huber. Robust estimation of a location parameter. In Breakthroughs in Statistics: Methodology and Distribution, pp.\ 492--518. Springer, 1992

    1992

  12. [20]

    Big data and its technical challenges

    Hosagrahar V Jagadish, Johannes Gehrke, Alexandros Labrinidis, Yannis Papakonstantinou, Jignesh M Patel, Raghu Ramakrishnan, and Cyrus Shahabi. Big data and its technical challenges. Communications of the ACM, 57 0 (7): 0 86--94, 2014

    2014

  13. [21]

    Pdformer: Propagation delay-aware dynamic long-range transformer for traffic flow prediction

    Jiawei Jiang, Chengkai Han, Wayne Xin Zhao, and Jingyuan Wang. Pdformer: Propagation delay-aware dynamic long-range transformer for traffic flow prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pp.\ 4365--4373, 2023

    2023

  14. [22]

    Scaling laws for neural language models

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020

    2001 arXiv

  15. [23]

    Rethinking graph transformers with spectral attention

    Devin Kreuzer, Dominique Beaini, Will Hamilton, Vincent L \'e tourneau, and Prudencio Tossou. Rethinking graph transformers with spectral attention. Advances in Neural Information Processing Systems, 34: 0 21618--21629, 2021

    2021

  16. [24]

    Imagenet classification with deep convolutional neural networks

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In F. Pereira, C.J. Burges, L. Bottou, and K.Q. Weinberger (eds.), Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc., 20...

    2012

  17. [25]

    Diffusion convolutional recurrent neural network: Data-driven traffic forecasting

    Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. In International Conference on Learning Representations, 2018. URL https://openreview.net/forum?id=SJiHXGWAZ

    2018

  18. [26]

    Short-term traffic flow forecasting: An experimental comparison of time-series analysis and supervised learning

    Marco Lippi, Matteo Bertini, and Paolo Frasconi. Short-term traffic flow forecasting: An experimental comparison of time-series analysis and supervised learning. IEEE Transactions on Intelligent Transportation Systems, 14 0 (2): 0 871--882, 2013

    2013

  19. [27]

    Spatio-temporal adaptive embedding makes vanilla transformer sota for traffic forecasting

    Hangchen Liu, Zheng Dong, Renhe Jiang, Jiewen Deng, Jinliang Deng, Quanjun Chen, and Xuan Song. Spatio-temporal adaptive embedding makes vanilla transformer sota for traffic forecasting. In Proceedings of the 32nd ACM international conference on information and knowledge manag...

    2023

  20. [28]

    Largest: A benchmark dataset for large-scale traffic forecasting

    Xu Liu, Yutong Xia, Yuxuan Liang, Junfeng Hu, Yiwei Wang, Lei Bai, Chao Huang, Zhenguang Liu, Bryan Hooi, and Roger Zimmermann. Largest: A benchmark dataset for large-scale traffic forecasting. Advances in Neural Information Processing Systems, 36, 2024

    2024

  21. [29]

    SGDR : Stochastic gradient descent with warm restarts

    Ilya Loshchilov and Frank Hutter. SGDR : Stochastic gradient descent with warm restarts. In International Conference on Learning Representations, 2017. URL https://openreview.net/forum?id=Skq89Scxx

    2017

  22. [30]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=Bkg6RiCqY7

    2019

  23. [31]

    Dynamic prediction of traffic volume through kalman filtering theory

    Iwao Okutani and Yorgos J Stephanedes. Dynamic prediction of traffic volume through kalman filtering theory. Transportation Research Part B: Methodological, 18 0 (1): 0 1--11, 1984

    1984

  24. [32]

    Wavenet: A generative model for raw audio

    Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499, 2016

    2016 arXiv

  25. [33]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 0 2...

    2022

  26. [34]

    On the difficulty of training recurrent neural networks

    Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. On the difficulty of training recurrent neural networks. In International Conference on Learning Representations, pp.\ 1310--1318. Pmlr, 2013

    2013

  27. [35]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, 32, 2019

    2019

  28. [36]

    Improving language understanding by generative pre-training

    Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training. OpenAI blog, 2018

    2018

  29. [37]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21 0 (140): 0 1--67, 2020

    2020

  30. [38]

    Learning internal representations by error propagation, parallel distributed processing, explorations in the microstructure of cognition, ed

    David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning internal representations by error propagation, parallel distributed processing, explorations in the microstructure of cognition, ed. de rumelhart and j. mcclelland. vol. 1. 1986. Biometrika, 71: 0 599--607, 1986

    1986

  31. [39]

    Discrete graph structure learning for forecasting multiple time series

    Chao Shang, Jie Chen, and Jinbo Bi. Discrete graph structure learning for forecasting multiple time series. In International Conference on Learning Representations, 2021

    2021

  32. [40]

    Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting

    Zezhi Shao, Zhao Zhang, Fei Wang, and Yongjun Xu. Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting. In Proceedings of the 28th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp.\ 1567--1577, 2022

    2022

  33. [41]

    A tutorial on support vector regression

    Alex J Smola and Bernhard Sch \"o lkopf. A tutorial on support vector regression. Statistics and Computing, 14: 0 199--222, 2004

    2004

  34. [42]

    Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting

    Chao Song, Youfang Lin, Shengnan Guo, and Huaiyu Wan. Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp.\ 914--921, 2020

    2020

  35. [43]

    Dropout: a simple way to prevent neural networks from overfitting

    Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15 0 (1): 0 1929--1958, 2014

    1929

  36. [44]

    Sequence to sequence learning with neural networks

    Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems, 27, 2014

    2014

  37. [45]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  38. [46]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 30, 2017

    2017

  39. [47]

    Graph wavenet for deep spatial-temporal graph modeling

    Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. Graph wavenet for deep spatial-temporal graph modeling. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp.\ 1907--1913, 2019

    1907

  40. [48]

    Connecting the dots: Multivariate time series forecasting with graph neural networks

    Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. Connecting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD '2...

    2020

  41. [49]

    Do transformers really perform badly for graph representation? Advances in Neural Information Processing Systems, 34: 0 28877--28888, 2021

    Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu. Do transformers really perform badly for graph representation? Advances in Neural Information Processing Systems, 34: 0 28877--28888, 2021

    2021

  42. [50]

    Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting

    Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI), 2018

    2018

  43. [51]

    Multi-scale context aggregation by dilated convolutions

    Fisher Yu and Vladlen Koltun. Multi-scale context aggregation by dilated convolutions. In International Conference on Learning Representations, 2016. URL https://arxiv.org/abs/1511.07122

    2016 arXiv

  44. [52]

    Gman: A graph multi-attention network for traffic prediction

    Chuanpan Zheng, Xiaoliang Fan, Cheng Wang, and Jianzhong Qi. Gman: A graph multi-attention network for traffic prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp.\ 1234--1241, 2020

    2020

  45. [53]

    Vector autoregressive models for multivariate time series

    Eric Zivot and Jiahui Wang. Vector autoregressive models for multivariate time series. Modeling Financial Time Series with S-PLUS , pp.\ 385--429, 2006

    2006

  46. [54]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  47. [55]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  48. [56]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.