REVIEW 3 cited by
Scaling-laws for Large Time-series Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Scaling laws for large language models (LLMs) have provided useful guidance in training ever larger models for predictable performance gains. Time series forecasting shares a similar sequential structure to language, and is amenable to large-scale transformer architectures. Here we show that foundational decoder-only time series transformer models exhibit analogous scaling-behavior to LLMs, with architectural details (aspect ratio and number of heads) having a minimal effect over broad ranges. We assemble a large corpus of heterogenous time series data on which to train, and establish for the first time power-law scaling with parameter count, dataset size, and training compute, spanning five orders of magnitude.
Forward citations
Cited by 3 Pith papers
-
When Do Foundation Models Pay Off? A Break-Even Analysis of Pretrained Time Series Forecasters
Time-series foundation models are unconditionally better than classical methods on 15/30 datasets, lose early on 6, and a n_train<700 + seasonality rule resolves 10 deployment decisions without training.
-
Learning Spatio-Temporal Foundation Models from Pure Synthetic Data
A spatio-temporal foundation model pre-trained exclusively on synthetic stochastic graph dynamics outperforms real-data-pretrained STFMs in zero-shot traffic forecasting, according to the paper's benchmarks.
-
Investigating Compositional Reasoning in Time Series Foundation Models
On a benchmark where models train on Fourier components and test on their sums, patch-based Transformers and residual MLP architectures show the strongest compositional generalization, while most standard transformers...
Discussion (0). Continue with ORCID to comment.