ST-ReP: Learning Predictive Representations Efficiently for Spatial-Temporal Forecasting

Qi Zheng; Yaying Zhang; Zihao Yao

REVIEW 4 major objections 5 minor 39 references

ST-ReP: Learning Predictive Representations Efficiently for Spatial-Temporal Forecasting

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ST-ReP learns predictive spatial-temporal representations by reconstructing the present and predicting the future in one pretraining task, and the paper reports this compact representation beats twelve baselines on six datasets and scales…

desk verdict ST-ReP is a genuinely useful lightweight self-supervised pretraining recipe, but its headline accuracy claims are weakened by a dimension-mismatched linear probe evaluation. read the letter →

arxiv 2412.14537 v1 pith:AO7N22E6 submitted 2024-12-19 cs.LG

Qi Zheng , Zihao Yao , Yaying Zhang This is my paper · ORCID

classification cs.LG

keywords spatial-temporalforecastingself-supervisedrepresentationlearningreconstructionandpredictionpretraininglinear-complexitytransformermulti-scaletemporallosstrafficflowwindpowerscalable

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ST-ReP is a self-supervised pretraining method for spatial-temporal forecasting that trains one lightweight encoder to do two generative tasks at once: reconstruct masked current observations and predict unseen future values. The paper argues that coupling these tasks through a multi-scale loss over the concatenated reconstruction and prediction forces the encoder to keep both spatial relations and predictive temporal structure. The payoff, the authors report, is a compact representation that a simple linear ridge regression can turn into forecasts matching or beating twelve baselines on six traffic, energy, and climate datasets. The method also runs on an 8,600-variable dataset where all self-supervised baselines run out of memory, which is the scalability claim in action.

What carries the argument

The load-bearing component is the Compression-Extraction-Decompression (C-E-D) ST encoder. A temporal compressor maps the embedded series $E\in\mathbb{R}^{N\times T\times d}$ into a low-dimensional latent space $\mathbb{R}^{N\times p\times d}$ with $p$ small; a spatial extractor then applies two multi-head attention layers to the $N$ variable tokens, using a learned proxy tensor $P_w\in\mathbb{R}^{m\times d}$ as queries in the first attention and as keys and values in the second, so attention cost stays linear in $N$; and a temporal decompressor restores the original length $T$. The final representation is the last layer's output added to the input embedding. Around this encoder sit a reconstruction decoder, a prediction decoder, and the multi-scale loss with average-pooling kernels $\Omega$; the downstream probe is ridge regression on the last time step's representation.

What would settle it

Re-run the downstream probe with the full set of representation samples instead of the sampled 0.93% to 5.5%, keeping the same pretrained encoders and ridge regression; if any baseline overtakes ST-ReP or the margin over the second-best method disappears, the central empirical claim would be undercut. A second check is to replace the linear ridge head with a small nonlinear head and see whether the ranking among self-supervised methods changes.

Watch

Extended reading notes

Core claim

The central claim is that a spatial-temporal representation is most useful for forecasting when it is trained to reconstruct the present and predict the future in one pretext task rather than to separate positive and negative samples. ST-ReP masks part of the current series, encodes the masked input with an ST encoder, then decodes the representation twice: one decoder reconstructs the current values and the other projects to future values. A multi-scale loss compares the concatenated reconstructed-and-predicted sequence with ground truth at several pooling scales, and the total loss is a weighted sum of reconstruction, prediction, and multi-scale terms. The paper reports that this yields representations of dimension 64 that outperform all self-supervised baselines on most of the six datasets, achieve the best average ranking among nine self-supervised methods across horizons, and, because the encoder has linear complexity in the number of variables, scale to 8,600 nodes where the baselines fail.

Load-bearing premise

The paper's evaluation treats a linear ridge-regression probe trained on a small fraction (0.93% to 5.5%) of the representation samples as the measure of representation quality; if that probe does not reflect how the representations will be used in practice, the claimed semantic superiority may not transfer to real downstream forecasting models.

Editorial extensions

If this is right

If the reported accuracy holds, reconstruction-plus-prediction pretraining gives better representations than contrastive pretraining on homogeneous sensor data, where negative pairs are often false negatives.
The linear-complexity encoder means pretraining and representation extraction can be done on networks with thousands of variables on a single 32GB GPU, a regime where existing self-supervised ST baselines run out of memory.
Because a linear probe on the last time step suffices for strong forecasts, downstream deployment can be cheap: the probe uses only 0.93% to 5.5% of representation samples and needs no GPU or backpropagation.
The multi-scale loss adds temporal granularity to the representation without increasing encoder cost, which the ablations support by showing degraded accuracy when it is removed.
On the longest horizons (48 and 96), a baseline that pretrains on a two-week input window beats ST-ReP, indicating that pretraining input length interacts with horizon length.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's evidence for 'semantic density' is indirect: a low-dimensional representation with a strong linear probe. A direct test would be to feed ST-ReP and baseline representations into a shared nonlinear downstream model and compare transfer performance; the current evaluation cannot distinguish representation quality from linear separability.
The reported long-horizon weakness against the two-week-window baseline suggests a testable extension: vary the pretraining input length $T$ while holding the reconstruction-plus-prediction objective fixed, and check whether long-horizon accuracy tracks input length rather than the loss design.
Because ST-ReP avoids negative pairs entirely, it could be combined with explicit graph or physical constraints, such as road-network adjacency or wind-farm topology, that are currently only implicit in the learned spatial attention; whether explicit structure helps is an open question.
The scalability claim is demonstrated on one 8,600-node dataset; a stronger test would be to run the same recipe on datasets with tens of thousands of nodes and compare memory scaling curves against the linear-complexity prediction.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Zee, here's the short version. ST-ReP is a self-supervised pretraining method for spatial-temporal forecasting that combines current-value reconstruction with future-value prediction in one pretext task, uses a lightweight C-E-D encoder (compress temporally, extract spatial correlations via a learned proxy tensor, decompress), and adds a multi-scale loss over the concatenated reconstruction+prediction output. That combination is new as far as I know, and it ships with honest experiments: six datasets, three domains, and they report the settings where baselines beat them (STEP at horizons 48/96 on PEMS04/08, T-Rep on the two climate datasets). The efficiency story is real—Table 3 shows smaller memory footprint and shorter training time on PEMS04 and Humidity, and it runs on the 8600-node CA dataset where all baselines OOM. That alone makes it useful for practitioners with large sensor networks.

The soft spot is the evaluation probe. The downstream test trains a ridge regression on only a small fraction of the representation samples (0.93%–5.5%), and the four contrastive baselines produce 320-dimensional representations while ST-ReP produces 64-dimensional ones. With a few hundred training points, a 320-dim ridge is in an underdetermined regime; a 64-dim ridge is not. So the reported margin on PEMS04/08 (0.044 vs 0.058 MSE) and even the tiny SDWPF edge (0.167 vs 0.168) may be an artifact of the probe, not of representation quality. This is a genuine confound and it deserves a careful response. Relatedly, Table 2 has no error bars despite averaging ten repetitions, so we can't tell if the close margins are significant. The headline claim that ST-ReP 'outperforms the best baseline methods' is too broad — on average rank among the self-supervised group it's first, but on climate it's second and at long horizons STEP beats it.

That said, the method itself seems sound. The ablations show each component contributes, the code is linked, and the authors openly acknowledge the long-horizon and climate caveats in the appendix. The probe problem is fixable — use the full training set for all models, or match dimensionality, and report error bars. If those checks come out clean, this is a solid engineering contribution.

My take: send it to peer review. It deserves a serious referee who will push on the evaluation protocol. I wouldn't cite the current version, but I'd cite a revised one. Yes to reading group — the probe issue is a good discussion topic.

Referee Report

4 major / 5 minor

Summary. ST-ReP is a self-supervised representation-learning method for spatial-temporal forecasting. It pre-trains a lightweight encoder with three objectives: reconstructing masked current values, predicting unseen future values, and a multi-scale temporal consistency loss. The encoder uses a Compression-Extraction-Decompression (C-E-D) structure with a learned proxy tensor to model spatial correlations in linear complexity. The authors evaluate the learned representations with a ridge-regression downstream probe on six datasets (traffic, climate, energy), report accuracy gains at horizon 12, and show lower GPU memory and training time than several baselines, including on a large traffic dataset (CA) where baselines run out of memory.

Significance. If the reported results hold, the paper makes a useful practical contribution: it offers a lightweight, non-contrastive self-supervised objective for spatial-temporal data that avoids negative-pair construction, produces compact 64-dimensional representations, and scales to thousands of nodes. The authors provide a public code repository, use standard public datasets, and report averages over ten repetitions, which are positive reproducibility features. The efficiency gains on the CA dataset are a concrete and interesting scalability result. However, the accuracy claims are currently supported mainly by a specific downstream probe whose sampling protocol confounds representation quality with probe conditioning, and the full horizon results weaken the headline conclusion. The contribution is incremental but, with the evaluation issues addressed, would be of interest to the spatial-temporal forecasting community.

major comments (4)

[Evaluation Setup / Appendix A] The downstream probe confounds representation quality with the number of ridge regression training points. The paper samples only 0.93% to 5.5% of the extracted representation samples for the downstream ridge probe, while four MTS baselines use d=320 and ST-ReP uses d=64. On PEMS04, 5.5% of the training portion is roughly 560 samples, so the 320-dimensional baseline probes are fitted in an underdetermined regime while the 64-dimensional ST-ReP probe is not. The reported gains (e.g., 0.044 vs 0.058 MSE on PEMS04) could therefore reflect the interaction between representation dimension and probe sample size rather than representation quality. Please re-run the downstream evaluation with matched effective sample counts per method, or with enough samples to keep all probes well-posed, and report both settings; this is necessary to support the claim that ST-ReP learns semantically denser representations.
[Appendix B, Table 5] The conclusion that ST-ReP 'outperforms the best baseline methods' is not supported by the full horizon results. In Table 5, STEP beats ST-ReP at horizons 48 and 96 on PEMS04 and PEMS08 (e.g., PEMS04 MSE 0.109 vs 0.115 at horizon 48, and 0.178 vs 0.230 at horizon 96), and T-Rep is better on Temperature and Humidity at multiple horizons. The main text acknowledges this only after the fact. The accuracy claim should be restricted to the setting where it actually holds, or a statistical comparison across all horizons should be presented; as written, the abstract and conclusion overstate the evidence.
[Tables 2 and 5 / Appendix A] The paper reports 'average metrics over ten repetitions' but does not report standard deviations, confidence intervals, or seed ranges. Without variability measures, near-tied margins such as the SDWPF horizon-12 result (0.167 vs 0.168 MSE) and the Temperature result (0.005 vs 0.005 MSE) cannot be assessed for significance. Please add standard deviations or confidence intervals for all main results and for the ablation study, and state how many seeds contribute to the reported averages.
[Table 3 / Efficiency Analysis] The efficiency comparison excludes STEP, with the explanation that STEP requires much longer historical inputs. However, STEP is one of the strongest accuracy competitors at longer horizons, so the scalability claim is presented without its most relevant comparison. Please report STEP's memory and training time under its actual input configuration, or clearly state that the efficiency comparison holds only among methods with the same short input window.

minor comments (5)

[Appendix A] There is a typo in the climate dataset description: 'Huimidity' should be 'Humidity'.
[Evaluation Setup (main text and Appendix A)] The evaluation setup paragraphs are repeated nearly verbatim in the main text and the appendix; one version should be condensed to avoid redundancy.
[Figure 5] The ablation figure caption does not identify the dataset or prediction horizon shown, and no error bars are displayed; please specify these details so the reader can judge the magnitude of the ablation differences.
[Spatial Extraction, Eq. (6)] The notation in Eq. (6) is slightly ambiguous: the text says the subscript for time is omitted, but the surrounding variables are later described as hidden features at a single time step. Please define the indexing convention explicitly before the equation.
[Related Work / Methodology] The citations to the 'router mechanism in (Zhang and Yan 2023)' and the 'bottleneck setup in (Guo et al. 2023)' are not explained; a one-sentence description of how these mechanisms relate to the proxy tensor would help the reader place the contribution.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the reported accuracy gains are empirical and the lone self-citation is a non-load-bearing preprocessing detail.

full rationale

ST-ReP's derivation is empirical rather than definitional. The pre-training objective (Equations 9-11) minimizes reconstruction, prediction, and multi-scale losses on the training set, including future values X_tgt as labels; the downstream evaluation trains a ridge regression on held-out representations from the same training distribution and reports MSE/MAE on the test set, so the reported performance is not the training objective itself. The comparison is not forced by construction: baseline representations are probed with the same ridge pipeline and the paper reports cases where ST-ReP does not win (e.g., T-ReP on climate datasets and STEP at horizons 48/96). The only self-citation is the cross-time normalization trick from Zheng and Zhang 2023, used in the ST-Embedding module; it is introduced as a design choice, not as a theorem or as the basis for excluding alternatives, so it is not load-bearing for the central accuracy and scalability claims. The Appendix's note that only 0.93%-5.5% of representation samples are used for ridge training, together with the dimension mismatch between ST-ReP (d=64) and four baselines (d=320), is a legitimate evaluation-validity concern about probe conditioning, but it does not make the claimed superiority equivalent to a fitted parameter or to the paper's own equations; the probe is applied uniformly and the outcome was not predetermined.

Assumptions & free parameters 7 free parameters · 4 assumptions · 1 invented entities

The central claim rests on an empirical evaluation paradigm rather than a derivation. The model introduces hand-set hyperparameters (loss weights, p, m, d) and one internal proxy-tensor mechanism; no new physical entities are postulated. The most fragile axiom is that a ridge-regression probe reflects downstream quality.

free parameters (7)

Reconstruction loss weight alpha = searched over {0.1, 0.2, 0.3, 0.4, 0.5}, chosen on validation set
Appears in the total loss L_total = alpha * L_recon + beta * L_pred + gamma * L_MS; tuned per dataset.
Prediction loss weight beta = searched over {0.1, 0.2, 0.3, 0.4, 0.5}, chosen on validation set
Controls the future prediction term in the pretext loss; tuned on validation.
Temporal compression size p = 3
Hand-set; sensitivity analysis shows error first decreases then increases with p, and this constant is central to the efficiency claim.
Proxy tensor size m = 8
Hand-set; ablation shows no clear trend in accuracy, so a small constant is chosen to limit cost.
Representation dimension d = 64
Hand-set; the compactness claim hinges on d=64 versus 320 for several MTS baselines.
Number of ST-Encoder layers L = 3
Stacked C-E-D layers; network capacity choice.
Multi-scale pooling kernels Omega = {2, 4, 8, 16}
Defines the scales in the multi-scale loss L_MS.

assumptions (4)

domain assumption A linear ridge regression probe on a small sample of representations is a valid measure of representation quality for downstream forecasting.
Used in Evaluation Setup: representations are fed to ridge regression, and only 0.93% to 5.5% of representation samples train the downstream model.
domain assumption Spatial correlations among variables can be captured by linear attention over a compressed temporal latent space.
C-E-D structure, Equations 3-6: the compressor reduces T to p=3 states and spatial extraction uses multi-head attention with a proxy tensor. Whether p=3 preserves enough temporal information is assumed, not derived.
domain assumption Z-score normalization applied collectively across all variables is an appropriate standardization.
Appendix A states 'The normalization is applied to all variables collectively, rather than individually'; this choice affects absolute MSE and MAE scales and comparability across datasets.
ad hoc to paper Combining reconstruction, prediction, and multi-scale losses improves representations for forecasting.
Total loss Equation 11 is a heuristic linear combination; no theoretical guarantee that this combination yields superior representations, though ablations support it empirically.

invented entities (1)

Learned proxy tensor P_w
purpose: Serves as a query and key summary token in MHA1 and MHA2 to reduce attention token count from N nodes to a small constant m.
It is an internal trainable parameter with no external falsifiable prediction; the scalability claim depends on this compression mechanism.

how reviews work

0 comments

Cite this review

Pith. "Pith review of ST-ReP: Learning Predictive Representations Efficiently for Spatial-Temporal Forecasting." pith.science (2026). https://pith.science/paper/AO7N22E6

@misc{pith2026241214537,
  author       = {Pith},
  title        = {Pith review of: ST-ReP: Learning Predictive Representations Efficiently for Spatial-Temporal Forecasting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AO7N22E6}},
  note         = {Machine review of arXiv:2412.14537}
}

read the original abstract

Spatial-temporal forecasting is crucial and widely applicable in various domains such as traffic, energy, and climate. Benefiting from the abundance of unlabeled spatial-temporal data, self-supervised methods are increasingly adapted to learn spatial-temporal representations. However, it encounters three key challenges: 1) the difficulty in selecting reliable negative pairs due to the homogeneity of variables, hindering contrastive learning methods; 2) overlooking spatial correlations across variables over time; 3) limitations of efficiency and scalability in existing self-supervised learning methods. To tackle these, we propose a lightweight representation-learning model ST-ReP, integrating current value reconstruction and future value prediction into the pre-training framework for spatial-temporal forecasting. And we design a new spatial-temporal encoder to model fine-grained relationships. Moreover, multi-time scale analysis is incorporated into the self-supervised loss to enhance predictive capability. Experimental results across diverse domains demonstrate that the proposed model surpasses pre-training-based baselines, showcasing its ability to learn compact and semantically enriched representations while exhibiting superior scalability.

Figures

Figures reproduced from arXiv: 2412.14537 by the authors.

**Figure 2.** The overall workflow of ST-ReP. model individual time series without considering spatial correlations between variables. STEP (Shao et al. 2022) and GPT-ST (Li et al. 2023) are two recent studies that introduce self-supervised learning into STS forecasting. However, their focus is primarily on using self-supervised learning as an auxiliary tool to enhance the performance of complex downstream ST models. They do… view at source ↗

**Figure 3.** The overall pre-training framework of our proposed ST-ReP. The ST-Encoder utilizes a Compression-Extraction [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: An example of the ST-Encoder. MLPs are used [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

**Figure 5.** Figure 5: The results of ablation study. and Decoderrecon, along with their respective loss constraints, are removed separately to assess the roles of each decoder. (3) w/o Multi-scale Loss: The loss LMS for multiple time scales supervision is removed to evaluate the benefits…

**Figure 6.** Figure 6: The results of varying size of temporal compres [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 27 canonical work pages

[1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Brown, C. E. 1998. Coefficient of Variation, 155--157. Berlin, Heidelberg: Springer Berlin Heidelberg. ISBN 978-3-642-80328-4

work page 1998
[4]

G.; Oreshkin, B

Challu, C.; Olivares, K. G.; Oreshkin, B. N.; Ramirez, F. G.; Canseco, M. M.; and Dubrawski, A. 2023. NHITS : Neural Hierarchical Interpolation for Time Series Forecasting . Proceedings of the AAAI Conference on Artificial Intelligence, 37(6): 6989--6997. Number: 6

work page 2023
[5]

Chen, L.; Chen, D.; Shang, Z.; Wu, B.; Zheng, C.; Wen, B.; and Zhang, W. 2023. Multi- Scale Adaptive Graph Neural Network for Multivariate Time Series Forecasting . IEEE Transactions on Knowledge and Data Engineering, 35(10): 10748--10761. Conference Name: IEEE Transactions on Knowledge and Data Engineering

work page 2023
[6]

M.; Likhosherstov, V.; Dohan, D.; Song, X.; Gane, A.; Sarlos, T.; Hawkins, P.; Davis, J

Choromanski, K. M.; Likhosherstov, V.; Dohan, D.; Song, X.; Gane, A.; Sarlos, T.; Hawkins, P.; Davis, J. Q.; Mohiuddin, A.; Kaiser, L.; Belanger, D. B.; Colwell, L. J.; and Weller, A. 2021. Rethinking Attention with Performers. In International Conference on Learning Representations

2021
[7]

Dong, J.; Wu, H.; Zhang, H.; Zhang, L.; Wang, J.; and Long, M. 2023. SimMTM : A Simple Pre - Training Framework for Masked Time - Series Modeling . Advances in Neural Information Processing Systems, 36: 29996--30025

work page 2023
[8]

F.; Bennetot, A.; and Allassonniere, S

Fraikin, A. F.; Bennetot, A.; and Allassonniere, S. 2024. T-Rep: Representation Learning for Time Series using Time-Embeddings. In The Twelfth International Conference on Learning Representations

work page 2024

Show all 39 references

[9]

Guo, S.; Lin, Y.; Feng, N.; Song, C.; and Wan, H. 2019. Attention Based Spatial-Temporal Graph Convolutional Networks for Traffic Flow Forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01): 922--929

2019
[10]

Guo, S.; Lin, Y.; Gong, L.; Wang, C.; Zhou, Z.; Shen, Z.; Huang, Y.; and Wan, H. 2023. Self- Supervised Spatial - Temporal Bottleneck Attentive Network for Efficient Long -term Traffic Forecasting . In 2023 IEEE 39th International Conference on Data Engineering ( ICDE ) , 1585...

2023
[11]

Hendrycks, D.; and Gimpel, K. 2016. Gaussian error linear units (gelus). arXiv, abs/1606.08415: 1--10

2016 arXiv
[12]

Huber, P. J. 1992. Robust Estimation of a Location Parameter. In Breakthroughs in Statistics: Methodology and Distribution, 492--518. New York, NY: Springer New York. ISBN 978-1-4612-4380-9

1992
[13]

Ji, J.; Wang, J.; Huang, C.; Wu, J.; Xu, B.; Wu, Z.; Zhang, J.; and Zheng, Y. 2023. Spatio-temporal self-supervised learning for traffic flow prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 37, 4356--4364

2023
[14]

Li, Z.; Xia, L.; Xu, Y.; and Huang, C. 2023. Generative Pre-Training of Spatio-Temporal Graph Neural Networks. In Thirty-seventh Conference on Neural Information Processing Systems

2023
[15]

Liu, J.; and Chen, S. 2024. TimesURL : Self - Supervised Contrastive Learning for Universal Time Series Representation Learning . Proceedings of the AAAI Conference on Artificial Intelligence, 38(12): 13918--13926. Number: 12

2024
[16]

X.; and Dustdar, S

Liu, S.; Yu, H.; Liao, C.; Li, J.; Lin, W.; Liu, A. X.; and Dustdar, S. 2022. Pyraformer: Low-Complexity Pyramidal Attention for Long-Range Time Series Modeling and Forecasting. In International Conference on Learning Representations

2022
[17]

Liu, X.; Xia, Y.; Liang, Y.; Hu, J.; Wang, Y.; BAI, L.; Huang, C.; Liu, Z.; Hooi, B.; and Zimmermann, R. 2023. Large ST : A Benchmark Dataset for Large-Scale Traffic Forecasting. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track

2023
[18]

Liu, Y.; Hu, T.; Zhang, H.; Wu, H.; Wang, S.; Ma, L.; and Long, M. 2024. iTransformer: Inverted Transformers Are Effective for Time Series Forecasting. In The Twelfth International Conference on Learning Representations

2024
[19]

Loshchilov, I.; and Hutter, F. 2017. Fixing Weight Decay Regularization in Adam. CoRR, abs/1711.05101

2017 arXiv
[20]

McDonald, G. C. 2009. Ridge regression. Wiley Interdisciplinary Reviews: Computational Statistics, 1(1): 93--100

2009
[21]

H.; Sinthong, P.; and Kalagnanam, J

Nie, Y.; Nguyen, N. H.; Sinthong, P.; and Kalagnanam, J. 2023. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers. In The Eleventh International Conference on Learning Representations

2023
[22]

D.; Scher, S.; Weyn, J

Rasp, S.; Dueben, P. D.; Scher, S.; Weyn, J. A.; Mouatadid, S.; and Thuerey, N. 2020. WeatherBench : A benchmark dataset for data-driven weather forecasting. Journal of Advances in Modeling Earth Systems, 12(11): e2020MS002203. ArXiv:2002.00469 [physics, stat]

2020 arXiv
[23]

A.; Abdi, A

Shabani, M. A.; Abdi, A. H.; Meng, L.; and Sylvain, T. 2023. Scaleformer: Iterative Multi-scale Refining Transformers for Time Series Forecasting. In The Eleventh International Conference on Learning Representations

2023
[24]

Shao, Z.; Zhang, Z.; Wang, F.; and Xu, Y. 2022. Pre-training Enhanced Spatial -temporal Graph Neural Network for Multivariate Time Series Forecasting . In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , KDD '22, 1567--1577. New York, NY, ...

2022
[25]

Song, C.; Lin, Y.; Guo, S.; and Wan, H. 2020 a . Spatial-Temporal Synchronous Graph Convolutional Networks: A New Framework for Spatial-Temporal Network Data Forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, 914--921

2020
[26]

Song, C.; Lin, Y.; Guo, S.; and Wan, H. 2020 b . Spatial-Temporal Synchronous Graph Convolutional Networks: A New Framework for Spatial-Temporal Network Data Forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, 914--921

2020
[27]

Y.; and ZHOU, J

Wang, S.; Wu, H.; Shi, X.; Hu, T.; Luo, H.; Ma, L.; Zhang, J. Y.; and ZHOU, J. 2024. TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting. In The Twelfth International Conference on Learning Representations

2024
[28]

Woo, G.; Liu, C.; Sahoo, D.; Kumar, A.; and Hoi, S. 2022. Co ST : Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting. In International Conference on Learning Representations

2022
[29]

Wu, H.; Hu, T.; Liu, Y.; Zhou, H.; Wang, J.; and Long, M. 2023 a . TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis. In The Eleventh International Conference on Learning Representations

2023
[30]

Wu, Q.; Yang, C.; Zhao, W.; He, Y.; Wipf, D.; and Yan, J. 2023 b . DIFF ormer: Scalable (Graph) Transformers Induced by Energy Constrained Diffusion. In The Eleventh International Conference on Learning Representations

2023
[31]

Yue, Z.; Wang, Y.; Duan, J.; Yang, T.; Huang, C.; Tong, Y.; and Xu, B. 2022. TS2Vec : Towards Universal Representation of Time Series . Proceedings of the AAAI Conference on Artificial Intelligence, 36(8): 8980--8987. Number: 8

2022
[32]

Zeng, A.; Chen, M.; Zhang, L.; and Xu, Q. 2023. Are transformers effective for time series forecasting? In Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirtee...

2023
[33]

Zerveas, G.; Jayaraman, S.; Patel, D.; Bhamidipaty, A.; and Eickhoff, C. 2021. A Transformer -based Framework for Multivariate Time Series Representation Learning . In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining , KDD '21, 2114--2124. New...

2021
[34]

Zhang, J.; Wen, X.; Zhang, Z.; Zheng, S.; Li, J.; and Bian, J. 2024 a . Prob TS : Benchmarking Point and Distributional Forecasting across Diverse Prediction Horizons. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track

2024
[35]

Y.; Liang, Y.; Pang, G.; Song, D.; and Pan, S

Zhang, K.; Wen, Q.; Zhang, C.; Cai, R.; Jin, M.; Liu, Y.; Zhang, J. Y.; Liang, Y.; Pang, G.; Song, D.; and Pan, S. 2024 b . Self- Supervised Learning for Time Series Analysis : Taxonomy , Progress , and Prospects . IEEE Transactions on Pattern Analysis and Machine Intelligence...

2024
[36]

Zhang, Y.; and Yan, J. 2023. Crossformer: Transformer Utilizing Cross-Dimension Dependency for Multivariate Time Series Forecasting. In The Eleventh International Conference on Learning Representations

2023
[37]

Zheng, Q.; and Zhang, Y. 2023. TAGnn: Time Adjoint Graph Neural Network for Traffic Forecasting. In Database Systems for Advanced Applications: 28th International Conference, DASFAA 2023, Tianjin, China, April 17–20, 2023, Proceedings, Part I, 369--379. Springer-Verlag. ISBN 9...

2023
[38]

Zhou, H.; Zhang, S.; Peng, J.; Zhang, S.; Li, J.; Xiong, H.; and Zhang, W. 2021. Informer: Beyond Efficient Transformer for Long Sequence Time - Series Forecasting . Proceedings of the AAAI Conference on Artificial Intelligence, 35(12): 11106--11115. Number: 12

2021
[39]

Zhou, J.; Lu, X.; Xiao, Y.; Su, J.; Lyu, J.; Ma, Y.; and Dou, D. 2022. SDWPF: A Dataset for Spatial Dynamic Wind Power Forecasting Challenge at KDD Cup 2022. arXiv:2208.04360

2022 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Brown, C. E. 1998. Coefficient of Variation, 155--157. Berlin, Heidelberg: Springer Berlin Heidelberg. ISBN 978-3-642-80328-4

work page 1998

[4] [4]

G.; Oreshkin, B

Challu, C.; Olivares, K. G.; Oreshkin, B. N.; Ramirez, F. G.; Canseco, M. M.; and Dubrawski, A. 2023. NHITS : Neural Hierarchical Interpolation for Time Series Forecasting . Proceedings of the AAAI Conference on Artificial Intelligence, 37(6): 6989--6997. Number: 6

work page 2023

[5] [5]

Chen, L.; Chen, D.; Shang, Z.; Wu, B.; Zheng, C.; Wen, B.; and Zhang, W. 2023. Multi- Scale Adaptive Graph Neural Network for Multivariate Time Series Forecasting . IEEE Transactions on Knowledge and Data Engineering, 35(10): 10748--10761. Conference Name: IEEE Transactions on Knowledge and Data Engineering

work page 2023

[6] [6]

M.; Likhosherstov, V.; Dohan, D.; Song, X.; Gane, A.; Sarlos, T.; Hawkins, P.; Davis, J

Choromanski, K. M.; Likhosherstov, V.; Dohan, D.; Song, X.; Gane, A.; Sarlos, T.; Hawkins, P.; Davis, J. Q.; Mohiuddin, A.; Kaiser, L.; Belanger, D. B.; Colwell, L. J.; and Weller, A. 2021. Rethinking Attention with Performers. In International Conference on Learning Representations

2021

[7] [7]

Dong, J.; Wu, H.; Zhang, H.; Zhang, L.; Wang, J.; and Long, M. 2023. SimMTM : A Simple Pre - Training Framework for Masked Time - Series Modeling . Advances in Neural Information Processing Systems, 36: 29996--30025

work page 2023

[8] [8]

F.; Bennetot, A.; and Allassonniere, S

Fraikin, A. F.; Bennetot, A.; and Allassonniere, S. 2024. T-Rep: Representation Learning for Time Series using Time-Embeddings. In The Twelfth International Conference on Learning Representations

work page 2024