REVIEW 3 major objections 43 references
A bike-demand model uses a few agent tokens so global station and time interactions cost only linear work, and it beats strong baselines on NYC and Chicago systems.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-11 01:40 UTC pith:BSQPLWRZ
load-bearing objection Solid applied packaging of agent attention for bike demand; gains look real under the authors’ protocol, but single-run tables and no agent-count sensitivity leave the “linear yet fully global” claim only provisionally supported. the 3 major comments →
STAGformer: A Spatio-temporal Agent Graph Transformer for Micro Mobility Demand Forecasting
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
STAGformer shows that two-step Softmax agent attention, mediated by a small fixed number of spatial and temporal agent tokens, can replace full self-attention for station-level bike demand forecasting, cutting complexity from O((NT)2) to O(NT) while still producing lower RMSE and MAE than strong baselines on NYC and Chicago data.
What carries the argument
Spatio-temporal agent attention: learnable spatial agents (ns ≪ N) and temporal agents (nt ≪ T) first aggregate global keys/values, then broadcast the summary back to every node and time step; a depthwise-convolution residual keeps feature diversity.
Load-bearing premise
That a small fixed number of agent tokens plus a simple residual convolution is enough to keep the full global modeling power of Softmax attention for real bike-demand patterns.
What would settle it
Re-run the same NYC and Chicago experiments with the agent-attention module replaced by full separated Softmax attention (or with agent counts driven to N and T) and check whether RMSE/MAE improve enough to erase the claimed accuracy advantage of the linear version.
If this is right
- City-scale bike rebalancing systems can run multi-step station forecasts that still see long-range station interactions without quadratic attention cost.
- The same linear agent pattern can be stacked with ordinary graph propagation and temporal convolution, so local and global cues remain complementary rather than mutually exclusive.
- External context (weather, time, POIs) can be fused once at the encoder and still benefit from the global agent stage.
- Ablation evidence implies that removing the agent stage hurts accuracy more than removing local graph or convolution modules, so global modeling is not optional for these systems.
Where Pith is reading between the lines
- If agent counts stay small while city networks keep growing, the same linear recipe may transfer to e-scooter or dockless fleets that share the same spatio-temporal structure.
- Sensitivity of accuracy to the exact agent count (ns, nt) is left open; an adaptive or learned agent budget would be a natural next test.
- The same two-step aggregate-then-broadcast idea could be tried on other large spatio-temporal graphs (traffic sensors, EV chargers) where full attention is already known to be too costly.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. STAGformer is a spatio-temporal forecasting architecture for station-level bike-sharing demand that fuses a multi-source encoder (node features + weather/time/POIs), graph propagation (fixed or adaptive adjacency with K-step fusion), temporal 1D convolution, and a two-step agent attention module. Separate learnable spatial and temporal agent tokens (ns, nt ≪ N, T) first aggregate then broadcast Softmax attention, reducing joint self-attention cost from O((NT)²) to O(NT) while a depthwise-convolution residual is claimed to preserve feature diversity. The model is evaluated on NYC Citi-Bike and Chicago Divvy-Bike (Sept/Oct 2025 windows) against nine baselines; Table 2 reports lower RMSE/MAE for STAGformer on all four city-month splits, and ablations attribute the largest drop to removal of agent attention.
Significance. If the linear agent construction truly retains the long-range modeling power of full Softmax attention, the work supplies a practical, scalable architecture for city-scale micro-mobility forecasting that can be deployed without the quadratic memory wall of standard Transformers. The hybrid design (local graph + local conv + global agents + external fusion) is a coherent engineering contribution, and the two-city empirical protocol with module ablations is stronger than many pure architectural papers. The paper does not ship code, multi-seed statistics, or agent-count sensitivity, so the significance remains provisional on those missing checks.
major comments (3)
- Table 2 and §5.4 claim that STAGformer “consistently outperforms” nine baselines with “significant improvements” in RMSE/MAE. All numbers appear to be single-run point estimates; no multi-seed means, standard deviations, or statistical tests are reported. Given the free parameters listed in §5.2 (learning rate, batch size, epochs, early stopping) and the small absolute margins (e.g., NYC Sept RMSE 0.4874 vs STAEformer 0.5106), the superiority claim is not yet secured against run-to-run variance.
- §4.3.2–4.3.5 and the “w/o Attn” ablation assert that a fixed small agent set (example ns = nt = 32) plus DWC residual fully preserves the global Softmax receptive field that the paper itself treats as critical. No sensitivity sweep over ns/nt, no comparison against full Softmax or separated attention under matched compute, and no analysis of information loss appear. Without that evidence the central efficiency-plus-expressiveness claim remains an untested modeling assumption rather than a demonstrated result.
- §5.1–5.2 describe data preparation (hourly alignment, 150 m POI radius, external adjacency) but omit the concrete train/validation/test temporal split, the exact values of T and H used for the multi-horizon claim, and whether the same hyper-parameters were used for every baseline. Reproducibility of Table 2 therefore cannot be verified from the manuscript alone.
Circularity Check
No circularity: standard supervised spatio-temporal forecasting model with free parameters trained and evaluated on held-out data; agent attention imported from external citation.
full rationale
STAGformer is an empirical deep-learning architecture paper. Historical node features X, global factors G and adjacency A_ext are mapped by a trainable encoder + graph propagation + temporal convolution + two-step agent attention (learnable agent tokens P_s, P_t of fixed small size n_s, n_t together with free projection matrices and DWC residual) to multi-horizon demand forecasts Ŷ. All parameters are optimized by supervised loss on training windows and evaluated on held-out months (Table 2, ablations §5.5). No equation equates a claimed prediction to a fitted constant by construction; complexity reduction O(NT) follows directly from the two Softmax steps with constant agents (Eqs. 12–15) and is independent of the numerical accuracy claims. The sole external reference for the agent mechanism is Han et al. [31] (ECCV 2024), whose authors do not overlap with the present single-author paper; no uniqueness theorem, ansatz or load-bearing result is imported from self-citation. Consequently the derivation chain contains no self-definitional, fitted-as-prediction, or self-citation circularity.
Axiom & Free-Parameter Ledger
free parameters (5)
- number of spatial agent tokens ns =
example 32
- number of temporal agent tokens nt =
example 32
- graph propagation steps K and fusion weights αk
- adaptive graph temperature τ and node embeddings E
- hidden dimension d / C, heads H, conv kernel kt, learning rate, batch size, epochs =
lr=1e-3, batch=8, 300 epochs
axioms (4)
- domain assumption Bike-sharing demand can be usefully modeled as a graph G=(V,E) with node features and an external or adaptive adjacency matrix.
- ad hoc to paper A two-step Softmax agent attention with a small agent set preserves global long-range dependency modeling while reducing complexity to O(NT).
- domain assumption External covariates (weather, time, POIs within 150 m) are informative for station demand and can be fused by linear projection and concatenation.
- domain assumption Standard supervised sliding-window forecasting with RMSE/MAE on held-out months is a valid measure of operational forecasting quality.
invented entities (1)
-
Spatial and temporal agent tokens (Ps, Pt) inside STAGformer
no independent evidence
read the original abstract
Accurate station-level demand forecasting is essential for the efficient operation of bike-sharing systems, yet it remains challenging due to complex spatio-temporal dependencies and the large scale of urban networks. This paper presents STAGformer, a Spatio-Temporal Agent Graph Transformer that achieves efficient global modeling with linear computational complexity. The model introduces a two-step agent attention mechanism, where a small set of learnable spatial and temporal agent tokens first aggregate global information and then broadcast it back to individual stations and time steps, effectively capturing long-range interactions while reducing the quadratic cost of standard self-attention to O(NT). STAGformer integrates four core modules: a spatio-temporal encoder that fuses dynamic node features with external contextual factors (weather, time, points of interest), a graph propagation module for spatial neighbor aggregation, a temporal convolution module for local pattern extraction, and the agent attention module for global dependency modeling. Extensive experiments on two real-world datasets -- NYC Citi-Bike and Chicago Divvy-Bike -- demonstrate that STAGformer consistently outperforms state-of-the-art baselines across multiple prediction horizons, achieving significant improvements in both RMSE and MAE. Ablation studies validate the contribution of each component, with the agent attention mechanism proving critical for modeling global spatio-temporal dependencies.
Figures
Reference graph
Works this paper leans on
-
[1]
J. Kim, H. Kim, H. Kim, D. Lee, and S. Yoon, “A comprehensive survey of deep learning for time series forecasting: architectural diversity and open challenges,”Artif. Intell. Rev., vol. 58, no. 7, p. 216, 2025
work page 2025
-
[2]
C. Chatfield and H. Xing,The Analysis of Time Series: An Introduction with R, 7th ed. Boca Raton, FL, USA: Chapman and Hall/CRC, 2019
work page 2019
-
[3]
Advances in neural information processing systems 22,
D. D. Lee, P. Pham, Y. Largman, and A. Ng, “Advances in neural information processing systems 22,”Neural Inf. Process. Syst., vol. 1, no. 1, pp. 1–11, 2009
work page 2009
- [4]
-
[5]
Greedy function approximation: a gradient boosting machine,
J. H. Friedman, “Greedy function approximation: a gradient boosting machine,”Ann. Statist., pp. 1189–1232, 2001
work page 2001
-
[6]
S. Hochreiter and J. Schmidhuber, “Long short-term memory,”Neural Comput., vol. 9, no. 8, pp. 1735–1780, 1997
work page 1997
-
[7]
Learning phrase representations using RNN encoder–decoder for statistical machine translation,
K. Cho et al., “Learning phrase representations using RNN encoder–decoder for statistical machine translation,” inProc. Conf. Empir. Methods Nat. Lang. Process. (EMNLP), 2014, pp. 1724–1734
work page 2014
-
[8]
Deep spatio-temporal residual networks for citywide crowd flows prediction,
J. Zhang, Y. Zheng, and D. Qi, “Deep spatio-temporal residual networks for citywide crowd flows prediction,” inProc. AAAI Conf. Artif. Intell., vol. 31, no. 1, 2017. 17 STAGformer: Spatio-temporal Agent Graph Transformer
work page 2017
-
[9]
Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting
Y. Li, R. Yu, C. Shahabi, and Y. Liu, “Diffusion convolutional recurrent neural network: Data-driven traffic forecasting,”arXiv preprint arXiv:1707.01926, 2017
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[10]
Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting
B. Yu, H. Yin, and Z. Zhu, “Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting,”arXiv preprint arXiv:1709.04875, 2017
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[11]
Attention based spatial-temporal graph con- volutional networks for traffic flow forecasting,
S. Guo, Y. Lin, N. Feng, C. Song, and H. Wan, “Attention based spatial-temporal graph con- volutional networks for traffic flow forecasting,” inProc. AAAI Conf. Artif. Intell., vol. 33, no. 1, 2019, pp. 922–929
work page 2019
-
[12]
GMAN: A graph multi-attention network for traffic prediction,
C. Zheng, X. Fan, C. Wang, and J. Qi, “GMAN: A graph multi-attention network for traffic prediction,” inProc. AAAI Conf. Artif. Intell., vol. 34, no. 1, 2020, pp. 1234–1241
work page 2020
-
[13]
X. Chen, J. Wang, and K. Xie, “TrafficStream: A streaming traffic flow forecast- ing framework based on graph neural networks and continual learning,”arXiv preprint arXiv:2106.06273, 2021
work page internal anchor Pith review Pith/arXiv arXiv 2021
-
[14]
Graph WaveNet for Deep Spatial-Temporal Graph Modeling
Z. Wu, S. Pan, G. Long, J. Jiang, and C. Zhang, “Graph WaveNet for deep spatial-temporal graph modeling,”arXiv preprint arXiv:1906.00121, 2019
work page internal anchor Pith review Pith/arXiv arXiv 1906
-
[15]
Adaptive graph convolutional recurrent network for traffic forecasting,
L. Bai, L. Yao, C. Li, X. Wang, and C. Wang, “Adaptive graph convolutional recurrent network for traffic forecasting,” inAdv. Neural Inf. Process. Syst., vol. 33, 2020, pp. 17804– 17815
work page 2020
-
[16]
A. Vaswani et al., “Attention is all you need,” inAdv. Neural Inf. Process. Syst., vol. 30, 2017
work page 2017
-
[17]
Spatial-Temporal Transformer Networks for Traffic Flow Forecasting
M. Xu et al., “Spatial-temporal transformer networks for traffic flow forecasting,”arXiv preprint arXiv:2001.02908, 2020
work page internal anchor Pith review Pith/arXiv arXiv 2001
-
[18]
Generating Long Sequences with Sparse Transformers
R. Child, S. Gray, A. Radford, and I. Sutskever, “Generating long sequences with sparse transformers,”arXiv preprint arXiv:1904.10509, 2019
work page internal anchor Pith review Pith/arXiv arXiv 1904
-
[19]
Transformers are RNNs: Fast au- toregressive transformers with linear attention,
A. Katharopoulos, A. Vyas, N. Pappas, and F. Fleuret, “Transformers are RNNs: Fast au- toregressive transformers with linear attention,” inProc. Int. Conf. Mach. Learn. (ICML), 2020, pp. 5156–5165
work page 2020
-
[20]
Predicting bike usage for New York City’s bike sharing system,
D. Singhvi, S. Singhvi, P. I. Frazier, S. G. Henderson, E. O’Mahony, D. B. Shmoys, and D. B. Woodard, “Predicting bike usage for New York City’s bike sharing system,” inProc. AAAI Workshop on Computational Sustainability, Austin, TX, USA, 2015
work page 2015
-
[21]
Rethinking Attention with Performers
K. Choromanski et al., “Rethinking attention with performers,”arXiv preprint arXiv:2009.14794, 2020
work page internal anchor Pith review Pith/arXiv arXiv 2009
-
[22]
P.Veličković, G.Cucurull, A.Casanova, A.Romero, P.Liò, andY.Bengio, “Graphattention networks,”arXiv preprint arXiv:1710.10903, 2017
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[23]
How Attentive are Graph Attention Networks?
S. Brody, U. Alon, and E. Yahav, “How attentive are graph attention networks?”arXiv preprint arXiv:2105.14491, 2021
work page internal anchor Pith review Pith/arXiv arXiv 2021
-
[24]
GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs
J.Zhang, X.Shi, J.Xie, H.Ma, I.King, andD.-Y.Yeung, “GaAN:Gatedattentionnetworks for learning on large and spatiotemporal graphs,”arXiv preprint arXiv:1803.07294, 2018
work page internal anchor Pith review Pith/arXiv arXiv 2018
-
[25]
Connecting the dots: Mul- tivariate time series forecasting with graph neural networks,
Z. Wu, S. Pan, G. Long, J. Jiang, X. Chang, and C. Zhang, “Connecting the dots: Mul- tivariate time series forecasting with graph neural networks,” inProc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., 2020, pp. 753–763. 18 STAGformer: Spatio-temporal Agent Graph Transformer
work page 2020
-
[26]
Informer: Beyond efficient transformer for long sequence time-series fore- casting,
H. Zhou et al., “Informer: Beyond efficient transformer for long sequence time-series fore- casting,” inProc. AAAI Conf. Artif. Intell., vol. 35, no. 12, 2021, pp. 11106–11115
work page 2021
-
[27]
Set transformer: A frame- work for attention-based permutation-invariant neural networks,
J. Lee, Y. Lee, J. Kim, A. Kosiorek, S. Choi, and Y. W. Teh, “Set transformer: A frame- work for attention-based permutation-invariant neural networks,” inProc. Int. Conf. Mach. Learn. (ICML), 2019, pp. 3744–3753
work page 2019
-
[28]
Predicting travel demand of a bike sharing system using graph convolutional neural networks,
A. Behroozi and A. Edrisi, “Predicting travel demand of a bike sharing system using graph convolutional neural networks,”Public Transp., vol. 17, no. 1, pp. 281–317, 2025
work page 2025
-
[29]
R. Rochas, A. Furno, and N.-E. El Faouzi, “Contextual data integration for bike-sharing demand prediction with graph neural networks in degraded weather conditions,” inProc. IEEE Int. Conf. Intell. Transp. Syst. (ITSC), 2023, pp. 5436–5441
work page 2023
-
[30]
J. Wang, T. Miwa, and T. Morikawa, “A demand truncation and migration poisson model for real demand inference in free-floating bike-sharing system,”IEEE Trans. Intell. Transp. Syst., vol. 24, no. 10, pp. 10525–10536, Oct. 2023
work page 2023
-
[31]
Agent attention: On the integration of softmax and linear attention,
D. Han et al., “Agent attention: On the integration of softmax and linear attention,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2024, pp. 124–140
work page 2024
-
[32]
An adaptive spatial-temporal method capturing for short-term bike- sharing prediction,
J. Feng and H. Liu, “An adaptive spatial-temporal method capturing for short-term bike- sharing prediction,”IEEE Trans. Intell. Transp. Syst., vol. 25, no. 11, pp. 16761–16774, Nov. 2024
work page 2024
-
[33]
BGM: Demand prediction for expanding bike- sharing systems with dynamic graph modeling,
Y. Zhao, H. Wen, X. Zhang, and M. Luo, “BGM: Demand prediction for expanding bike- sharing systems with dynamic graph modeling,” inProc. Int. Joint Conf. Artif. Intell. (IJCAI), 2025, pp. 10008–10016
work page 2025
-
[34]
Z. Xiang, F. Zeng, L. Liu, J. Wu, S. Mumtaz, and V. C. M. Leung, “Bike-sharing demand prediction based on dynamic time warping and spatio-temporal graph attention network,” IEEE Trans. Intell. Transp. Syst., 2025, to be published
work page 2025
-
[35]
S. Jiang, Z. Strout, B. He, D. Peng, P. B. Shull, and B. P. L. Lo, “Dual stream meta learning for road surface classification and riding event detection on shared bikes,”IEEE Trans. Syst. Man Cybern. Syst., vol. 53, no. 11, pp. 7188–7200, Nov. 2023
work page 2023
-
[36]
GeOT: a spatially explicit framework for evaluating spatio-temporal predictions,
N. Wiedemann, T. Uscidda, and M. Raubal, “GeOT: a spatially explicit framework for evaluating spatio-temporal predictions,”Int. J. Geogr. Inf. Sci., vol. 39, no. 10, pp. 2236– 2266, 2025
work page 2025
-
[37]
A. Faghih-Imani and N. Eluru, “Incorporating the impact of spatio-temporal interactions on bicycle sharing system demand: A case study of New York CitiBike system,”J. Transp. Geogr., vol. 54, pp. 218–227, 2016
work page 2016
-
[38]
X. Yang, J. Wang, S. Han, and S. He, “Micromobility flow prediction: A bike shar- ing station-level study via multi-level spatial-temporal attention neural network,”arXiv preprint arXiv:2507.16020, 2025
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[39]
MoDE-Boost: Boosting shared mobility demand with edge-ready prediction models,
A. Tziorvas, G. S. Theodoropoulos, and Y. Theodoridis, “MoDE-Boost: Boosting shared mobility demand with edge-ready prediction models,”arXiv preprint arXiv:2602.16573, 2026
-
[40]
Spatio-temporal adaptive embedding makes vanilla transformer sota for traffic forecasting,
H. Liu et al., “Spatio-temporal adaptive embedding makes vanilla transformer sota for traffic forecasting,” inProc. ACM Int. Conf. Inf. Knowl. Manag. (CIKM), 2023, pp. 4125–4129. 19 STAGformer: Spatio-temporal Agent Graph Transformer
work page 2023
-
[41]
STGformer: Efficient Spatiotemporal Graph Transformer for Traffic Forecasting
H. Wang, J. Chen, T. Pan, Z. Dong, L. Zhang, R. Jiang, and X. Song, “STGformer: Efficient spatiotemporal graph transformer for traffic forecasting,”arXiv preprint arXiv:2410.00385, 2024
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[42]
STGFormer: Spatio-temporal GraphFormer for 3D human pose estimation in video,
Y. Liu and Z. Zhang, “STGFormer: Spatio-temporal GraphFormer for 3D human pose estimation in video,”Pattern Recognit., p. 112239, 2025
work page 2025
-
[43]
J. Cheng, G. H. de Almeida Correia, O. Cats, and S. S. Azadeh, “T-STAR: A context-aware transformerframeworkforshort-termprobabilisticdemandforecastingindock-basedshared micro-mobility,”arXiv preprint arXiv:2602.06866, 2026. 20
work page internal anchor Pith review Pith/arXiv arXiv 2026
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.