Pith. sign in

REVIEW 4 major objections 5 minor 50 references

Decoupling Spatio-Temporal Prediction: When Lightweight Large Models Meet Adaptive Hypergraphs

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that splitting temporal modeling (a lightweight LLM) from spatial modeling (an adaptive hypergraph) gives the best accuracy on five forecasting benchmarks while cutting compute.

desk verdict The decoupled architecture is a sensible extension, but the SOTA claim is compromised by test-set selection of the hypergraph order and several reproducibility slips. read the letter →

arxiv 2505.19620 v1 pith:K2IGNKA5 submitted 2025-05-26 cs.LG cs.AI

Jiawen Chen , Qi Shao , Duxin Chen , Wenwu Yu This is my paper
classification cs.LGcs.AI
keywords spatio-temporalpredictionlargelanguagemodelsadaptivehypergraphneuralnetworkstrafficforecastingtemporal-spatialdecouplinglow-ranktemporaldynamicsgatedfusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that accurate spatio-temporal forecasting does not require one large joint model that couples time and space in the same architecture. Its proposal, STH-SepNet, splits the task: a lightweight pre-trained LLM (BERT, GPT-2, or larger backbones) processes a node-averaged global trend plus per-node patched series to capture temporal dynamics, while an adaptive hypergraph, with hyperedges built per batch from k-nearest feature neighbors, handles spatial dependencies and higher-order interactions, and a learned gate fuses the two streams. Across five benchmarks (BIKE-Inflow, BIKE-Outflow, PEMS03, BJ500, METR-LA), the paper reports the best MAE and RMSE among all compared models, including 28.8% lower RMSE than STAEformer on PEMS03 and 23.8% lower MAE than TIMELLM on METR-LA. The authors also claim that the 110M-parameter BERT version stays competitive with, and in places beats, 7B and 8B backbones, so parameter scale is not the main driver once time and space are separated. If correct, the practical upshot is that smaller, cheaper models can replace monolithic spatio-temporal transformers.

What carries the argument

The load-bearing mechanism is the two-stream decoupled architecture centered on an adaptive hypergraph. The temporal stream applies average pooling to produce a global trend token, embeds time-series patches, prepends a prompt, and passes the result through a partially frozen pre-trained LLM fine-tuned with LoRA. The spatial stream runs MixProp graph convolutions on the static adjacency, an adaptive asymmetric adjacency, and the hypergraph incidence matrix built per batch by KNN, with hyperedge order $k=3$ reported as best. Theorem 1 justifies the construction by equating the $(k-1)$-hop neighborhood of a node with the nodes covered by its $k$-order hyperedges under local connectivity, hyperedge coverage, and uniqueness conditions. A learned sigmoid gate fuses the temporal and spatial outputs so that neither stream dominates.

What would settle it

Construct a dataset with known coupled dynamics, such as contagion spread where a node's next value is a product of its own past value and its neighbors' past values, and train STH-SepNet against a jointly parameterized baseline with the same compute budget; if the joint model consistently wins and the advantage of per-batch hyperedges disappears, the decoupling premise is falsified. A simpler check on existing data is to replace the per-batch KNN hyperedges with static hyperedges on a benchmark with shifting spatial relationships and observe whether the MAE gap narrows sharply.

Watch

Extended reading notes

Core claim

The central claim is that decoupled modeling, not parameter count, carries the performance. Temporal dynamics are treated as low-rank: average pooling condenses all nodes into one regional trend token (Eq. 7), time-series patches become tokens, and a prompt prefix supplies dataset description, task instruction, and statistics, while the LLM's feed-forward layers are fine-tuned with LoRA and the rest stays frozen. Spatial drift is modeled by a hypergraph rebuilt each batch: KNN groups each node with its k nearest feature-neighbors into hyperedges, and node-to-hyperedge then hyperedge-to-node aggregation captures multi-node interactions that pairwise graphs miss. The two streams are combined by a sigmoid gating mechanism (Eqs. 26-27). On BIKE-Inflow, BIKE-Outflow, PEMS03, BJ500, and METR-LA, STH-SepNet reports the best MAE and RMSE among all compared models, including PEMS03 RMSE 34.17 versus 48.03 for STAEformer and METR-LA MAE 9.42 versus 9.98 for MTGNN, while running on a single A6000 GPU.

Load-bearing premise

The load-bearing assumption is that temporal and spatial dependencies can be cleanly separated, so a node-averaged trend plus per-node series carries all temporal information and the adaptive hypergraph carries all spatial information, which would break for processes with tightly coupled spatio-temporal causality.

Editorial extensions

If this is right

  • If decoupling is correct, a 110M-parameter BERT backbone can match or beat LLMs in the 1B-8B parameter range once time and space are modeled separately, so pushing model scale is not a reliable route to better accuracy.
  • The adaptive hypergraph is the main source of spatial gain: on PEMS03 the full model reports RMSE 34.17 versus 48.03 for STAEformer and 34.52 for the static-graph variant, suggesting static or pairwise graph assumptions are the bottleneck on dynamic road networks.
  • Node-wise decoupling turns the problem into parallelizable univariate temporal tasks, which the paper credits for GPU memory of 24.6G and faster training than TIMELLM on the same hardware.
  • The learned gated fusion beats attention-based and LSTM-based fusion, with reported MAE reductions of 17.7% to 45.6% on the more complex traffic benchmarks.
  • The framework is architecture-agnostic: consistent gains appear across BERT, GPT-2, GPT-3, LLAMA1B, LLAMA7B, LLAMA8B, and DeepSeek backbones, with performance fluctuation under 3% on BJ500.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension of the paper's reasoning is that the same decoupled recipe should transfer to other spatio-temporal domains with low-rank temporal cores and higher-order spatial interactions, such as energy load or climate fields, but the paper only demonstrates traffic and bike-sharing benchmarks.
  • Because hyperedges are recomputed per batch from current node features, the efficiency figures assume full-batch recomputation; latency-critical deployment would need cached or scheduled hyperedge updates, a variant the paper does not test.
  • The appendix's example prompt for the PEMS03 traffic experiment describes a solar power dataset, so an immediate check is whether prompt text, not architecture, contributes part of the gain; swapping the description sentence while holding the model fixed would isolate that contribution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes STH-SepNet, a spatio-temporal forecasting framework that decouples temporal and spatial modeling: a lightweight LLM (BERT, GPT-2, LLaMA, DeepSeek) processes average-pooled global trends and per-node sequences, while an adaptive hypergraph neural network constructs hyperedges dynamically to model higher-order spatial dependencies. A gated fusion mechanism combines the two branches. Experiments on BIKE-Inflow, BIKE-Outflow, PEMS03, BJ500, and METR-LA report the best MAE/RMSE among the compared baselines for a fixed 48-step horizon, along with GPU memory and training-speed comparisons. The paper also contributes a theorem on hyperedge/neighborhood equivalence and ablation studies on the LLM backbone, graph construction, gating, and hypergraph order.

Significance. If the empirical findings are reproducible, STH-SepNet is a practically attractive lightweight alternative to monolithic spatio-temporal transformers: it uses a small LLM backbone and an adaptive hypergraph to achieve reported state-of-the-art accuracy on five benchmarks while reducing computational cost. The availability of code is a strength, as is the explicit acknowledgment of the decoupling assumption as a limitation. The empirical comparison is against external benchmarks, so the main performance claim is not circular. However, the central claim currently rests on a test-set-based selection of the hypergraph order, and the theoretical statement is vacuous; these issues must be resolved before the claims can be considered valid.

major comments (4)
  1. [Section 5.3.3, Table C.2, Figure 4] The effective hypergraph order k is selected by observing test-set performance. Section 5.1 states a 7:1:2 train/validation/test split, but no validation-based selection protocol is described for k; Table C.2 reports RMSE for k in {2,3,4,5} on BIKE-Outflow and PEMS03, and k=3, the value with the lowest RMSE, is exactly the configuration reported in Table 1. Thus the headline improvements on these datasets (e.g., PEMS03 RMSE 34.17 vs. STAEformer 48.03) are minima over a hyperparameter grid evaluated on the same test sets used for the final numbers, not out-of-sample evaluations of a fixed model. Please either select k on a validation split, report all k values as part of the main results, or justify a priori fixing of k; otherwise the state-of-the-art claim is not supported for at least two of the five datasets.
  2. [Section 3.4, Eq. (5), Appendix A.1] Theorem 1 is vacuous. The claimed equivalence 'w is in the (k-1)-hops neighborhood of v iff there exists a k-order hyperedge containing v and w with at most k-2 intermediate nodes' is a direct restatement of the definitions: a node w is reachable from v by at most k-1 hyperedges exactly when it can be placed in a k-order hyperedge with v and at most k-2 intermediate nodes. The proof in Appendix A.1 assumes the conclusion in the sufficiency direction ('we can construct a k-order hyperedge' from the path), and the 'conditions' are formulated to make the equivalence true by construction. Since the theorem is not used to derive any independent property (Algorithm 1 merely refers to it to 'calculate spatial interaction'), it overstates the theoretical contribution. I recommend removing it or demoting it to a definition/construction remark.
  3. [Appendix B.2] The PEMS03 prompt shown in Appendix B.2 describes a solar power plant dataset (PV, 69 nodes, normalized power output, zero at night) instead of PEMS03 traffic speed data (358 stations, 5-minute intervals). This is a reproducibility-relevant error because the prompt template is part of the method: if this prompt was used for the PEMS03 experiments, the input statistics and dataset description do not match the data; if it is a placeholder, the appendix should be corrected to the actual prompt. Please provide the correct prompt for PEMS03 and clarify which prompt corresponds to each dataset.
  4. [Tables 1 and 2] The baseline numbers are inconsistent across tables. For example, TIMELLM (BERT) on PEMS03 is reported as MAE/RMSE 32.62/49.77 in Table 1 but 32.68/50.39 in Table 2; on BIKE-Outflow, TIMELLM is 6.93/16.30 in Table 1 but 6.74/16.13 in Table 2. Additionally, Table 1 does not specify which LLM backbone is used for TIMELLM, while Table 2 reports backbone-specific results. Please reconcile the numbers and state the exact configuration used for each baseline in each table.
minor comments (5)
  1. [Abstract and Section 1] The abstract contains a typo ('avaliable' for 'available') and a grammatical issue ('aiming to reduce computational demands and while enhancing predictive performance').
  2. [Section 5.4] The training speed reported as '392 Epoch/s' should be expressed as seconds per epoch (or a similarly correct unit); the current phrasing is physically implausible.
  3. [Section 5.3.2] The sentence 'BERT-w/i ... operates with less than 10% fewer parameters' is numerically incorrect: BERT (110M) has about 98% fewer parameters than LLaMA7B (6740M), not less than 10% fewer.
  4. [Appendix C.4] The model name is misspelled as 'SHT-SepNet' in the opening sentence of Appendix C.4; it should be 'STH-SepNet'.
  5. [Section 3.4] Definition 1 describes a hyperedge as an 'unordered set' but uses tuple notation e_j=(v^(j)_1,...,v^(j)_k); please clarify the intended semantics. Also, the notation H^v_k in Theorem 1 is not defined before its first use.

Circularity Check

2 steps flagged · score 6.0 of 10

The Table 1 SOTA numbers for BIKE-Outflow and PEMS03 are best-of-sweep test-set minima (k=3 chosen on the same test partitions), and Theorem 1 restates its own conclusion by construction; no self-citation chain is load-bearing.

  1. fitted input called prediction [Section 5.3.3, Figure 4, and Table C.2 (Appendix C.3), alongside Table 1 headline RMSEs.]
    "On the BIKE-Outflow and PEMS03 datasets, STH-SepNet models based on BERT, GPT-2, LLAMA1B, and DeepSeek1.5B demonstrate that when k = 2, the high-order structure of STH-SepNet degenerates into a pairwise relationship. The empirical results indicate that as the order k increases, the model error initially decreases and then increases. ... That is, k = 3 effectively characterizes evolving spatial dependencies."

    The hypergraph order k is a hyperparameter of the KNN hyperedge construction (Section 3.4: e_i = {v_i} ∪ N(v_i)). Appendix C.3 (Table C.2) reports test-set RMSE for k in {2,3,4,5} on the same BIKE-Outflow and PEMS03 test partitions that produce the Table 1 headline numbers, and the k=3 values (14.23 and 34.17) are exactly those headline RMSEs. Section 5.3.3 then presents 'k = 3 is effective' as a validated finding rather than as the minimum of the same sweep; no validation-based selection protocol is described (Section 5.1 only states a 7:1:2 split).

  2. self definitional [Section 3.4 (Theorem 1) and the proof in Appendix A.1.]
    "For any k ≥ 2, the (k − 1)-hops neighborhood of a node v, denoted as N_{k−1}(v), corresponds to all nodes involved in the k-order hyperedges in H^v_k, if and only if the following conditions are satisfied: ... (2) Hyperedge Coverage Condition: there exists a k-order hyperedge e ∈ H^v_k such that w ∈ e and e contains v, w, and at most k − 2 intermediate nodes. ... Formally: w ∈ N_{k−1}(v) ⇐⇒ {v, F1, F2, ..., uk, w} ∈ H^v_k"

    The theorem's formal equivalence is identical to its condition (2): the RHS 'there exists a k-order hyperedge containing v and w' is the definition of the correspondence being asserted. Section 3.4 fixes the construction so that the equivalence holds by fiat: 'each node is interconnected with all nodes within its k-hops distance, forming one or more hyperedges' and 'For each node vi, its nearest k neighbors N(vi) form a hyperedge ei = {vi} ∪ N(vi)'. The Appendix proof's sufficiency direction says 'we can construct a k-order hyperedge e that includes v and w along with at most k−2 intermediate nodes', i.e., it assumes the very hyperedge whose existence is to be shown, while the necessity direction counts that hyperedge itself as a one-hyperedge path, which is the definition of N_{k−1}(v).

full rationale

Score 6 reflects partial circularity of the central empirical claim, not a fully forced derivation. The strongest claim, state-of-the-art MAE/RMSE on five benchmarks, is tested against external baselines with fixed reported numbers, so it is not circular for BIKE-Inflow, BJ500, and METR-LA; however, for BIKE-Outflow and PEMS03 the headline RMSEs in Table 1 are identical to the k=3 entries of the same test-set sweep in Table C.2, and Section 5.3.3 converts the sweep's argmin into a claimed 'effective order' finding. That is a test-set-fit renamed as a prediction, which inflates the SOTA comparison for two of the five datasets. Theorem 1 is self-definitional (its condition (2) is its conclusion, and its proof constructs the hyperedge whose existence is at issue), but it is ornamental rather than load-bearing. There is no load-bearing self-citation chain: the reference list contains no prior work by the present authors, and external supports (e.g., low-rank dynamics [12,13]) are not used to justify the headline results. The decoupling assumption is explicitly acknowledged as a limitation in Section 6, so it is a scope caveat rather than circularity. Two additional non-circular defects are flagged per the review rules: Appendix B.2 gives a PEMS03 prompt describing a solar power plant dataset (69 nodes, North Carolina, 1-hour sampling), which is a reproducibility bug for the prompt-based LLM pipeline, and Figure 4/Section 5.3.3's 'initially decreases and then increases' claim is contradicted by Table C.2 for the BERT row on BIKE-Outflow, where RMSE increases from 13.36 (k=2) to 14.23 (k=3). These strengthen the impression that the k=3 'finding' is a post-hoc summary of the very test numbers reported as the model's performance, but they are correctness concerns, not additional circular steps.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the decoupling assumption, the unverified low-rank temporal assumption, and the validity of the hyperedge construction. The only numerical parameter fitted to data that affects the headline results is the hypergraph order k=3; several other scale and fusion parameters are not reported.

free parameters (3)
  • hypergraph order k = 3
    Selected as the best-performing order on BIKE-Outflow and PEMS03 (Figure 4, Table C.2); main results report error at k=3, so the reported performance is conditional on this choice.
  • fusion weight gamma (Eq. 22) = not reported
    Tunable parameter balancing GCN and HGCN outputs; the paper does not state its value or whether it is learned, yet it controls the spatial representation.
  • activation scaling factor alpha (Eqs. 2-4, 6) = not reported
    Modulates saturation of tanh for adaptive adjacency and hyperedge features; no value or tuning procedure is given.
assumptions (3)
  • domain assumption Temporal dynamics of spatio-temporal systems are low-rank and can be captured by a lightweight LLM from pooled and patched input.
    Invoked in Section 1 and Section 4.1; no evidence or derivation is provided for the low-rank claim.
  • domain assumption Temporal and spatial dependencies can be cleanly decoupled without loss.
    Stated explicitly as a limitation in Section 6; the whole architecture depends on this separation, and the paper admits it may fail under coupled spatio-temporal causality.
  • ad hoc to paper The equivalence in Theorem 1 holds.
    The proof in Appendix A assumes the k-order hyperedge it is supposed to construct and restates the conditions; the theorem is not needed for the KNN hyperedge construction yet is used to justify the adaptive hypergraph.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Decoupling Spatio-Temporal Prediction: When Lightweight Large Models Meet Adaptive Hypergraphs." pith.science (2026). https://pith.science/paper/K2IGNKA5

@misc{pith2026250519620,
  author       = {Pith},
  title        = {Pith review of: Decoupling Spatio-Temporal Prediction: When Lightweight Large Models Meet Adaptive Hypergraphs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K2IGNKA5}},
  note         = {Machine review of arXiv:2505.19620}
}
read the original abstract

Spatio-temporal prediction is a pivotal task with broad applications in traffic management, climate monitoring, energy scheduling, etc. However, existing methodologies often struggle to balance model expressiveness and computational efficiency, especially when scaling to large real-world datasets. To tackle these challenges, we propose STH-SepNet (Spatio-Temporal Hypergraph Separation Networks), a novel framework that decouples temporal and spatial modeling to enhance both efficiency and precision. Therein, the temporal dimension is modeled using lightweight large language models, which effectively capture low-rank temporal dynamics. Concurrently, the spatial dimension is addressed through an adaptive hypergraph neural network, which dynamically constructs hyperedges to model intricate, higher-order interactions. A carefully designed gating mechanism is integrated to seamlessly fuse temporal and spatial representations. By leveraging the fundamental principles of low-rank temporal dynamics and spatial interactions, STH-SepNet offers a pragmatic and scalable solution for spatio-temporal prediction in real-world applications. Extensive experiments on large-scale real-world datasets across multiple benchmarks demonstrate the effectiveness of STH-SepNet in boosting predictive performance while maintaining computational efficiency. This work may provide a promising lightweight framework for spatio-temporal prediction, aiming to reduce computational demands and while enhancing predictive performance. Our code is avaliable at https://github.com/SEU-WENJIA/ST-SepNet-Lightweight-LLMs-Meet-Adaptive-Hypergraphs.

Figures

Figures reproduced from arXiv: 2505.19620 by the authors.

Figure 1
Figure 1. (a) Spatio-temporal data exhibit spatial distribution [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The framework of STH-SepNet. Given a traffic network [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 4
Figure 4. Analysis of effective order on adaptive hypergraph. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Comparison of GPU and time complexity. hypergraph significantly enhances model performance. On the BIKE-Outflow and PEMS03 datasets, STH-SepNet models based on BERT, GPT-2, LLAMA1B, and DeepSeek1.5B demonstrate that when 𝑘 = 2, the high-order structure of STH-SepNet de…
Figure 3
Figure 3. Figure 3: Performance comparison of MAE between STH [PITH_FULL_IMAGE:figures/full_fig_p008_3.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

50 extracted references · 31 canonical work pages

  1. [1]

    Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. 2020. Connecting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 753–763

    work page 2020

  2. [2]

    Jeongwhan Choi, Hwangyong Choi, Jeehyun Hwang, and Noseong Park

    work page

  3. [3]

    Tung Nguyen, Johannes Brandstetter, Ashish Kapoor, Jayesh K Gupta, and Aditya Grover. 2023. ClimaX: A foundation model for weather and climate. arXiv preprint arXiv:2301.10343 (2023)

    arXiv 2023

  4. [4]

    Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. 2023. Accurate medium-range global weather forecasting with 3D neural networks. Nature 619, 7970 (2023), 533–538

    2023

  5. [5]

    Xiaodong Yan, Tengwei Song, Yifeng Jiao, Jianshan He, Jiaotuan Wang, Ruopeng Li, and Wei Chu. 2023. Spatio-temporal hypergraph learning for next POI rec- ommendation. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. 403–412

    work page 2023

  6. [6]

    Zhonghang Li, Long Xia, Lei Shi, Yong Xu, Dawei Yin, and Chao Huang. 2024. OpenCity: Open Spatio-Temporal Foundation Models for Traffic Prediction.CoRR abs/2408.10269 (2024)

    arXiv 2024

  7. [7]

    Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)

    arXiv 2016

  8. [8]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)

    arXiv 2023

Show all 50 references
  1. [9]

    Yuan Yuan, Jingtao Ding, Jie Feng, Depeng Jin, and Yong Li. 2024. Unist: A prompt- empowered universal model for urban spatio-temporal prediction. InProceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 4095–4106

    2024

  2. [10]

    Zhonghang Li, Lianghao Xia, Yong Xu, and Chao Huang. 2024. GPT-ST: genera- tive pre-training of spatio-temporal graph neural networks, Vol. 36

    2024

  3. [11]

    Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, et al . 2023. Time-llm: Time series forecasting by reprogramming large language models.arXiv preprint arXiv:2310.01728 (2023)

    2023 arXiv

  4. [12]

    Mohammad Taha Bahadori, Qi Rose Yu, and Yan Liu. 2014. Fast multivariate spatio-temporal analysis via low rank tensor learning. Proceedings of Advances in Neural Information Processing Systems 27 (2014)

    2014

  5. [13]

    Tong Nie, Guoyang Qin, Wei Ma, Yuewen Mei, and Jian Sun. 2024. ImputeFormer: Low rankness-induced transformers for generalizable spatiotemporal imputation. In Proceedings of the Knowledge Discovery and Data Mining. 2260–2271

    2024

  6. [14]

    Chao Song, Youfang Lin, Shengnan Guo, and Huaiyu Wan. 2020. Spatial-temporal synchronous graph convolutional networks: A new framework for spatial- temporal network data forecasting. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 914–921

    2020

  7. [15]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human...

    2019

  8. [16]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9

    2019

  9. [17]

    Adam Goodge, Wee Siong Ng, Bryan Hooi, and See Kiong Ng. 2025. Spatio- Temporal Foundation Models: Vision, Challenges, and Opportunities. arXiv preprint arXiv:2501.09045 (2025)

    2025 arXiv

  10. [18]

    Hao Xue and Flora D Salim. 2023. Promptcast: A new prompt-based learning paradigm for time series forecasting. IEEE Transactions on Knowledge and Data Engineering (2023)

    2023

  11. [19]

    Nate Gruver, Marc Finzi, Shikai Qiu, and Andrew G Wilson. 2024. Large language models are zero-shot time series forecasters. Proceedings of Advances in Neural Information Processing Systems 36 (2024)

    2024

  12. [20]

    Ching Chang, Wen-Chih Peng, and Tien-Fu Chen. 2023. Llm4ts: Two-stage fine-tuning for time-series forecasting with pre-trained llms. CoRR (2023)

    2023

  13. [21]

    Tian Zhou, Peisong Niu, Liang Sun, Rong Jin, et al . 2023. One fits all: Power general time series analysis by pretrained lm. In Proceedings of Advances in Neural Information Processing Systems, Vol. 36. 43322–43355

    2023

  14. [22]

    Jiawei Li, Jingshu Peng, Haoyang Li, and Lei Chen. 2024. UniCL: A Universal Contrastive Learning Framework for Large Time Series Models. arXiv preprint arXiv:2405.10597 (2024)

    2024 arXiv

  15. [23]

    Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. 2023. Are transformers effective for time series forecasting?. In Proceedings of the AAAI conference on Artificial Intelligence, Vol. 37. 11121–11128

    2023

  16. [24]

    Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. 2022. Timesnet: Temporal 2d-variation modeling for general time series analysis. arXiv preprint arXiv:2210.02186 (2022)

    2022 arXiv

  17. [25]

    Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. 2022. A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14730 (2022)

    2022 arXiv

  18. [26]

    Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. 2023. iTransformer: Inverted Transformers Are Effective for Time Series Forecasting. arXiv preprint arXiv:2310.06625 (2023)

    2023 arXiv

  19. [27]

    Hangchen Liu, Zheng Dong, Renhe Jiang, Jiewen Deng, Q Chen, and X Song

  20. [28]

    Zezhi Shao, Zhao Zhang, Fei Wang, Wei Wei, and Yongjun Xu. 2022. Spatial- temporal identity: A simple yet effective baseline for multivariate time series forecasting. InProceedings of the ACM International Conference on Information & Knowledge Management. 4454–4458

    2022

  21. [29]

    Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. 2022. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference on Machine Learning. PMLR, 27268– 27286

    2022

  22. [30]

    Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. 2021. Autoformer: De- composition transformers with auto-correlation for long-term series forecasting. Proceedings of Advances in Neural Information Processing Systems 34 (2021), 22419–22430

    2021

  23. [31]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)

    2021 arXiv

  24. [32]

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. 2024. Qwen2.5-Math Technical Report: Toward Mathematical Expert Model ...

    2024 arXiv

  25. [33]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al . 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 (2025)

    2025 arXiv

  26. [34]

    Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. 2021. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on Artificial Intelligence, Vol. 35. 11106–11115

    2021

  27. [35]

    Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long

  28. [36]

    Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. 2023. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers. In Proceedings of International Conference on Learning Representations

    2023

  29. [37]

    Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Zhang, Xiaoming Shi, Pin- Yu Chen, Yuxuan Liang, Yuan-fang Li, Shirui Pan, et al. 2024. Time-LLM: Time Series Forecasting by Reprogramming Large Language Models. In Proceedings of International Conference on Learning Representations

    2024

  30. [38]

    In Proceedings of International Conference on Learning Representations

    TimesNet: Temporal 2D-Variation Modeling for General Time Series Anal- ysis. In Proceedings of International Conference on Learning Representations

  31. [39]

    Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. 2020. Adaptive graph convolutional recurrent network for traffic forecasting.Proceedings of Advances in Neural Information Processing Systems 33 (2020), 17804–17815

    2020

  32. [40]

    Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. 2019. Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI conference on Artificial Intelligence, Vol. 33. 922–929

    2019

  33. [41]

    Zongjiang Shang, Ling Chen, Binqing Wu, and Dongliang Cui. 2024. Ada- MSHyper: Adaptive Multi-Scale Hypergraph Transformer for Time Series Fore- casting. In Proceedings of Neural Information Processing Systems

    2024

  34. [42]

    Zheng Fang, Qingqing Long, Guojie Song, and Kunqing Xie. 2021. Spatial- temporal graph ode networks for traffic flow forecasting. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 364–373

    2021

  35. [43]

    Bing Yu, Haoteng Yin, and Zhanxing Zhu. 2018. Spatio-Temporal Graph Con- volutional Networks: A Deep Learning Framework for Traffic Forecasting. In Proceedings of International Joint Conference on Artificial Intelligence. Interna- tional Joint Conferences on Artificial Intelli...

    2018

  36. [44]

    Ziyu Jia, Youfang Lin, Jing Wang, Xiaojun Ning, Yuanlai He, Ronghao Zhou, Yuhan Zhou, and H Lehman Li-wei. 2021. Multi-view spatial-temporal graph convolutional networks with domain generalization for sleep stage classification. IEEE Transactions on Neural Systems and Rehabili...

    2021

  37. [45]

    Haotian Gao, Renhe Jiang, Zheng Dong, Jinliang Deng, Yuxin Ma, and Xuan Song. 2024. Spatial-temporal-decoupled masked pre-training for spatiotemporal forecasting. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence. 3998–4006

    2024

  38. [46]

    Fuxian Li, Jie Feng, Huan Yan, Guangyin Jin, Fan Yang, Funing Sun, Depeng Jin, and Yong Li. 2023. Dynamic graph convolutional recurrent network for traffic prediction: Benchmark and solution. ACM Transactions on Knowledge Discovery from Data 17, 1 (2023), 1–21

    2023

  39. [47]

    Chuanpan Zheng, Xiaoliang Fan, Cheng Wang, and Jianzhong Qi. 2020. Gman: A graph multi-attention network for traffic prediction. In Proceedings of the AAAI conference on Artificial Intelligence, Vol. 34. 1234–1241

    2020

  40. [50]

    Jing Huang and Jie Yang. 2021. UniGNN: a Unified Framework for Graph and Hy- pergraph Neural Networks. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. International Joint Conferences on Artifi- cial Intelligence Organization. A HYPERG...

    2021

  41. [2022]

    In Proceedings of the AAAI conference on Artificial Intelligence, Vol

    Graph neural controlled differential equations for traffic forecasting. In Proceedings of the AAAI conference on Artificial Intelligence, Vol. 36. 6367– 6374

  42. [2023]

    In Proceedings of the ACM International Conference on Information and Knowledge Management

    STAEformer: Spatio-Temporal Adaptive Embedding Makes Vanilla Trans- formers SOTA for Traffic Forecasting. In Proceedings of the ACM International Conference on Information and Knowledge Management. 21–25. KDD‘25, August 03-07, 2025, Toronto, Canada Jiawen Chen, Qi Shao, Duxin ...

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.