Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

Investigating Compositional Reasoning in Time Series Foundation Models

T0 review · 4 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Time series models can be tested for compositional reasoning, and patch-based Transformers plus small residual MLP networks pass the test while most other architectures fail.

desk verdict Clean spectral-extrapolation benchmark and a robust architecture ranking, but the compositional-reasoning label outruns the evidence. read the letter →

arxiv 2502.06037 v2 pith:IEK2YNBC submitted 2025-02-09 cs.LG

Willa Potosnak , Cristian Challu , Mononito Goswami , Kin G. Olivares , Micha{l} Wiliński , Nina Żukowska , Artur Dubrawski This is my paper · ORCID
classification cs.LG
keywords compositionalreasoningtimeseriesfoundationmodelszero-shotforecastingspectraldecompositionpatch-basedtransformersout-of-distributiongeneralizationrepresentationsimilaritymodelefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether time series forecasting models succeed by memorizing training patterns or by reasoning about them, and it proposes a concrete test for one form of reasoning. It defines compositional reasoning as the ability to forecast a composite out-of-distribution signal after training only on that signal's Fourier components, then evaluates 16 deep learning models and two statistical baselines on synthetic and real-world data. The central finding is that patch-based Transformers and residual MLP architectures, notably the T5-based model, N-HiTS, N-BEATS, and a plain MLP, pass the test while most RNN, CNN, linear, and other Transformer variants do not. The efficient winners matter: N-HiTS and N-BEATS reach near-best accuracy with 97% fewer floating-point operations (FLOPs) and 86% fewer parameters than the best Transformer configuration. Component ablations show that input patching is the decisive design choice for Transformers, while larger model size does not improve reasoning.

What carries the argument

The load-bearing object is the spectral composition benchmark itself: the training set holds the top-$k$ discrete Fourier components of each target series (Eqs. 5–7), while the test set holds the summed signal, so the training and test distributions differ by construction and success requires composing what was learned. The Top-$k$ Basis Win metric (Eq. 9) supplies the evidence threshold — a model must beat a reconstruction from the top-$k$ Fourier components of the target to count as a reasoner. Auxiliary evidence comes from Centered Kernel Alignment on model embeddings, which tests whether composition appears in the latent space, and from critical-difference statistical tests that rank models and design choices across datasets. Controlled ablation over a modular T5-encoder Transformer isolates which components matter, with fixed-length input patching emerging as the one that unlocks reasoning.

What would settle it

Train a strictly linear forecaster (for example ridge regression on the context window) on the same top-$k$ Fourier component training set used for the deep models. If it attains Top-$k$ Basis Wins comparable to N-HiTS, N-BEATS, or the T5 model on the out-of-distribution test sets, the benchmark does not isolate compositional reasoning from linear additivity; a multiplication-based composition version of the benchmark would settle the question directly.

Watch

Extended reading notes

Core claim

The authors claim that compositional reasoning is measurable in forecasting with a spectral benchmark: train a model on the top-$k$ Fourier basis functions of a time series and evaluate it zero-shot on the full summed signal. A model counts as a reasoner when, by their proposed Top-$k$ Basis Win metric, it matches or beats reconstruction from the top-$k$ basis at threshold $k \geq 2$ — outperforming that simple benchmark rules out merely recalling single learned frequencies. On this test, the T5-based patch Transformer, N-HiTS, N-BEATS, and the plain MLP demonstrate compositional reasoning across stationary real-world datasets and on nonstationary trend-plus-sinusoid compositions, whereas iTransformer, Autoformer, TSMixer, LSTM, and most others do not. Centered Kernel Alignment of latent embeddings shows that a model trained jointly on trend and frequency components aligns more closely with a model trained on the composed ground-truth signal, which the authors take as evidence that composition occurs inside the representations. They further claim that reasoning does not scale with model size: small T5 variants rank best out-of-distribution, and among the tested components the tokenization method has the largest significant effect on Transformer reasoning performance.

Load-bearing premise

The ranking rests on the premise that forecasting a summed signal after training on its parts measures reasoning: because the composition is addition, an approximately linear model could pass by superposition, and a nonlinear model could fail for numerical rather than conceptual reasons.

Editorial extensions

If this is right

  • Reasoning is not a property of whole model classes: among Transformers, patching-based models exhibit compositional reasoning while several other Transformer variants do not.
  • Efficient architectures suffice for reasoning: N-HiTS and N-BEATS match the best patch Transformer while using 97% fewer FLOPs and 86% fewer trainable parameters.
  • Zero-shot out-of-distribution forecasts from the reasoning-capable models can beat moving-average ARIMA on all six datasets and AutoETS on two, even though the statistical baselines were fitted on in-distribution data.
  • Model size and reasoning diverge: smaller T5 variants rank best on out-of-distribution tasks while larger variants win in-distribution tasks, so scaling alone is not a route to better reasoning.
  • Among design choices, tokenization has the clearest and largest effect — patching helps and binning hurts — while model size and projection layer show smaller significant differences and many other components (attention type, positional encoding, loss, scaler, context length, decomposition) do not change reasoning performance significantly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A ridge-regression or other strictly linear control trained on the same Fourier components would sharpen the central claim: the paper's account predicts it fails the Top-$k$ threshold while the residual MLP models pass, but that exact control is not run.
  • Composing signals by a non-additive operation (multiplication, gating, thresholding) would separate genuine composition from linear additivity and is the most direct next benchmark.
  • The divergence between in-distribution and out-of-distribution rankings implies that model selection should evaluate OOD compositional tasks explicitly rather than trusting ID validation alone.
  • Since patching — preserving local temporal structure in the input tokens — is the decisive component, testing other local structure-preserving tokenizers such as wavelet packets is a natural follow-up.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper defines a benchmark for 'compositional reasoning' in time series forecasting: models are trained on the top-k Fourier components of a target signal (Eqs. 5–6) and evaluated zero-shot on the composed sum signal (Eq. 7). A new metric, Top-k Basis Win (Eq. 9), is proposed, with a threshold of k=2 used to identify models that show 'sparks' of compositional reasoning. The paper evaluates 16 deep learning and statistical forecasting models on one synthetic and five real-world datasets, plus two nonstationary synthetic benchmarks in the appendix, and reports that patch-based Transformers (PatchTST, T5) and residualized MLP-based models (MLP, NHITS, NBEATS) perform best, with NHITS/NBEATS being far more parameter- and FLOP-efficient. It also ablates design choices of a modular T5-based Transformer, finding patching to be the most beneficial tokenization method. The main claims are that these models exhibit compositional reasoning and that such reasoning does not simply follow from in-distribution forecasting performance.

Significance. If the operationalization is accepted, the paper would provide a novel and reusable benchmark for a largely unexplored capability of time series models, together with a broad, reproducible comparison and controlled ablations. Strengths include public code and datasets, evaluation across many architectures, statistical significance testing, and the explicit nonstationary extensions in the appendix. However, the significance is conditional: the central 'reasoning' interpretation is not supported by the current experimental design, and the headline ranking relies on test-selected patch lengths. The paper is best viewed at present as an empirical study of additive spectral composition, with the reasoning claim requiring additional controls.

major comments (4)
  1. [§3.1, Eqs. (5)–(7); §3.4, Eq. (9)] The benchmark composes concepts by addition, so an approximately linear input–output map trained on individual basis functions can pass the test by superposition without any internal composition of concepts. The Top-k Basis Win threshold with k=2 only excludes forecasts that are worse than the dominant single-frequency reconstruction; it does not rule out linear additivity. The linear baselines DLinear and NLinear are not adequate controls because their moving-average decomposition and input normalization break simple additivity, and NHITS and NBEATS are additive basis-expansion architectures by design, so their success is expected from their inductive bias. The paper should either add explicit controls (for example, a linear model with Fourier features trained on the basis functions and tested on sums, or a nonlinear composition operation such as a product) or reframe the claims as being about additive compositional generalization rather than compositional reasoning.
  2. [Table 2 and §4] The entries 'PatchTST (Best PL)' and 'T5 Model (Best PL)' appear to select the patch length per dataset from the test-set MAE tables. For example, in Table 9 the T5 Model OOD ECL value 0.239 corresponds to PL=96 while the Subseasonal value 1.714 corresponds to PL=128; in Table 8, PatchTST OOD Synthetic Sinusoid 10.696 is PL=128 while ETTm2 0.250 is PL=96. Selecting hyperparameters on the test set inflates the reported performance and undermines the central ranking claim. The paper should either report results for a single configuration chosen without test data, or report all patch lengths and base the ranking on a validation-based selection.
  3. [§3.2, Preprocessing] The real-world benchmark is restricted to segments that pass the Augmented Dickey-Fuller test at α=0.001 and then to the 100 segments with the highest mean ACF. This selects the most stationary, most periodic subseries and excludes nonstationary behavior, which the paper acknowledges is left for future work. However, the abstract and Section 4 make broad claims about zero-shot OOD performance on 'all 6 datasets' and about outperforming statistical baselines on real-world data. The authors should report how this selection affects the conclusions, for example by including results on the full datasets or a sensitivity analysis over the ACF selection threshold.
  4. [§7.1, Table 1] The CKA analysis is correlational and is reported for a single architecture (T5 Model with patch length 128) on a single synthetic setting. It shows that embeddings of a model trained on both frequency and trend components are more similar to the ID ground-truth model than embeddings of models trained on one component, but it does not establish that the winning models' forecasts arise from composing separable component representations. This evidence should be presented as suggestive and clearly distinguished from the benchmark-based claims, which are already subject to the linear-superposition concern in my first major comment.
minor comments (6)
  1. [Section 2] There is a duplicated word in 'through through the addition of concepts' in the last paragraph of Section 2.
  2. [Table 4 and Figure 8] 'Casual' should be 'Causal' in the attention ablation entries.
  3. [§3.4, Eq. (9)] The Top-k Basis Win definition should specify that L is MAE and clarify how the maximum in 'Top-kmax Basis Win' is computed and averaged across datasets; for noisy real signals, the maximum over k up to n is not a well-defined summary without this clarification.
  4. [Table 8, T5 Model row] There is a typographical error in the standard deviation formatting: '(0.039) 0.039)' should be a single properly parenthesized value.
  5. [Abstract and §3.3] The abstract and Section 3.3 state that the paper examines design choices in '7 popular open-source TSFMs', but the controlled studies actually train a modular T5-based architecture with components inspired by those TSFMs; the seven TSFMs themselves are not trained. This should be rephrased to avoid overstating the scope.
  6. [§3.4 and Figure 3] The Friedman test is run at α=0.2, which is an unusually liberal significance level; the authors should justify this choice or also report results at conventional levels such as α=0.05. The figure captions should also state explicitly that the reported 'Difference' is ID minus OOD MAE.

Circularity Check

2 steps flagged · score 4.0 of 10

Partial circularity: 'compositional reasoning' is defined by the additive spectral task itself, and the headline T5/PatchTST results use per-dataset best patch lengths chosen from the test tables; the central ranking is not wholly vacuous.

  1. self definitional [Section 3.1, Eqs. (5)-(8)]
    "If y(t) = Σ_{w=0}^n c_w e^{i 2πw/n t}, then D(train) = { (e^{i 2πw/n [t−l:t]}, e^{i 2πw/n [t:t+h]}) } with t≤T and |c_w| ∈ top-k(...); D(test) = {(y_{t−l:t}, y_{t:t+h})} with t > T. ... As models are evaluated in a zero-shot forecasting setting, generalization can be achieved only through the addition of time series basis functions seen during training, or composition-type reasoning."

    The test target y(t) is, by Eq. (5), the sum of the Fourier components that are exactly the training inputs in Eq. (6). The paper then asserts that success 'can be achieved only through the addition of time series basis functions seen during training, or composition-type reasoning.' This equates the linear addition that defines the target with reasoning; any approximately additive input-output map can pass by superposition without composing internal concepts. The later conclusion that T5/NHITS/NBEATS 'show sparks of compositional reasoning' therefore restates the benchmark's construction rather than providing independent evidence. Eq. (9)'s Top-k baseline is also built from the same training components, so simple recall of those components can yield a win by the <= threshold.

  2. fitted input called prediction [Table 2 caption; used in Section 4 Results]
    "PatchTST and the T5 Model with the best patch length (PL) from Tables 8 and 9 are included."

    Table 2's reported OOD MAE for T5 and PatchTST is the per-dataset minimum over patch lengths taken from the same OOD evaluation tables: PatchTST combines PL=128 for Synthetic/Solar, PL=64 for ECL/ETTm2, and PL=96 for Subseasonal/Loop Seattle; T5 similarly mixes PL=64/96/128 across datasets. Selecting the hyperparameter that minimizes the target metric on the test sets, then presenting that minimum as the model's 'reasoning performance' and ranking it against fixed-default models, is a test-set fit renamed as a prediction. The headline claim that 'patch-based Transformers have the best reasoning performance' is thus partly constructed by the selection rule rather than by a single fixed model.

full rationale

This is an empirical benchmark study, not a derivation, and most of the comparisons are self-contained: 16 models trained with public code are evaluated on external datasets against statistical baselines. Self-citations (NeuralForecast, N-HiTS, MOMENT, etc.) are toolchain or implementation references, not load-bearing theoretical premises; no uniqueness theorem is imported. The main circularity is at the construct level: Eqs. (5)-(7) define the test target as the sum of the training components, and the paper explicitly equates succeeding at that sum with 'composition-type reasoning,' so the 'reasoning' label is partly definitional and cannot rule out linear superposition. A second partial circularity is the oracle patch-length selection: the headline T5/PatchTST results are per-dataset minima over patch lengths chosen from the same OOD tables, so the 'best reasoning performance' ranking is partly fitted on the target metric. These issues undermine the interpretation of the results, but they do not make the entire model ranking vacuous: the strong empirical performance of NHITS/NBEATS/MLP and the failure of many transformer variants are findings with independent content. The paper's Limitations section does not flag either issue, identifying only dataset-count and ablation-scope constraints. Score 4 reflects partial circularity in the headline claim, not total reducibility.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central benchmark depends on definitional choices: Fourier components as concepts, additive composition as the reasoning operation, a Top-k=2 win threshold, and a stationary high-ACF segment filter. These are not derived or externally validated. The reported results also depend on test-set-selected patch lengths and an unreported k for the top-k training set.

free parameters (7)
  • top_k_components = unreported
    Eq. 6 defines D(train) as the top-k spectral components, but the exact k is never given for synthetic or real datasets; the benchmark cannot be exactly reproduced without it.
  • top_k_basis_win_threshold = 2
    Section 3.4 declares Top-k=2 wins as the threshold for evidence of compositional reasoning; no sensitivity analysis is provided for this chosen cutoff.
  • best_patch_length = varies by dataset, e.g., 96 or 128
    Headline results use Best PL selected from Tables 8 and 9 on test MAE, so some performance is fitted to the evaluation set.
  • stationarity_filter_alpha = 0.001
    ADF alpha is used to exclude nonstationary segments in real-world preprocessing, shaping which series enter the evaluation.
  • top_acf_segment_count = 100
    Only 100 highest-ACF stationary segments per dataset are kept, so the real-world results depend on this hand-chosen quantity.
  • synthetic_parameter_ranges = a in [1,32], b in [3,32], m in [-32,32]
    Amplitude, frequency, and trend slope ranges are hand-chosen; the difficulty of the composition benchmark depends on them.
  • context_length_and_horizon = 256/512 context, 48 horizon
    Context and horizon are fixed by the authors rather than derived, and they affect how much of each basis function the model can observe.
assumptions (5)
  • domain assumption Fourier basis functions are the primitive concepts of time series reasoning
    Section 3.1 assumes that training on DFT components and testing on their sum isolates compositional reasoning; this is a stipulative definition with no external validation.
  • domain assumption Beating a 2-component Fourier reconstruction indicates reasoning
    Section 3.4 sets the Top-k=2 Basis Win threshold as evidence of compositional reasoning; no mechanistic or behavioral validation is provided.
  • domain assumption High-ACF stationary segments represent real-world forecasting
    Section 3.2 filters with ADF and selects top-100 ACF segments, so real-world conclusions are restricted to strongly periodic stationary subseries.
  • standard math Standard DFT and ADF statistics are valid on these finite segments
    The framework relies on the Discrete Fourier Transform and the Augmented Dickey-Fuller stationarity test as standard tools.
  • domain assumption The modular T5 architecture represents the design space of published TSFMs
    Table 3 maps published TSFM components onto the T5 backbone, but the actual pretrained models are not run; conclusions about TSFM design rest on this mapping.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Investigating Compositional Reasoning in Time Series Foundation Models." pith.science (2026). https://pith.science/paper/IEK2YNBC

@misc{pith2026250206037,
  author       = {Pith},
  title        = {Pith review of: Investigating Compositional Reasoning in Time Series Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IEK2YNBC}},
  note         = {Machine review of arXiv:2502.06037}
}
read the original abstract

Large pre-trained time series foundation models (TSFMs) have demonstrated promising zero-shot performance across a wide range of domains. However, a question remains: Do TSFMs succeed by memorizing patterns in training data, or do they possess the ability to reason about such patterns? While reasoning is a topic of great interest in the study of Large Language Models (LLMs), it is undefined and largely unexplored in the context of TSFMs. In this work, inspired by language modeling literature, we formally define compositional reasoning in forecasting and distinguish it from in-distribution generalization. We evaluate the reasoning and generalization capabilities of 16 popular deep learning forecasting models on multiple synthetic and real-world datasets. Additionally, through controlled studies, we systematically examine which design choices in 7 popular open-source TSFMs contribute to improved reasoning capabilities. Our study yields key insights into the impact of TSFM architecture design on compositional reasoning and generalization. We find that patch-based Transformers have the best reasoning performance, closely followed by residualized MLP-based architectures, which are 97\% less computationally complex in terms of FLOPs and 86\% smaller in terms of the number of trainable parameters. Interestingly, in some zero-shot out-of-distribution scenarios, these models can outperform moving average and exponential smoothing statistical baselines trained on in-distribution data. Only a few design choices, such as the tokenization method, had a significant (negative) impact on Transformer model performance.

Figures

Figures reproduced from arXiv: 2502.06037 by the authors.

Figure 1
Figure 1. (a) Traditional forecasting paradigm: models are trained directly on the training set of forecast target signals, with forecasts for the temporally subsequent test set generated using the preceding context window. (b) Compositional reasoning forecasting paradigm: models are trained on basis function series that compose the ground truth signal, with forecasts for the temporally subsequent test set generated using the… view at source ↗
Figure 2
Figure 2. Transformer architecture based on the open-source T5 encoder backbone used in our controlled studies to evaluate various architecture components used across TSFMs. Components for which no ablation studies are conducted are shown in gray. Baselines. We utilized ARIMA [23, 24] and AutoETS [6] as our statistical baseline models. Deep Learning Models. We trained models using 16 algorithms that cover a range of archi￾tec… view at source ↗
Figure 3
Figure 3. Generalization performance of models on composition reasoning tasks across ID and [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: (a, c) Forecasts on the Subseasonal (top) and synthetic trend-seasonality (bottom) datasets using models trained with the traditional forecasting paradigm. (b, d) Forecasts from models trained with the compositional reasoning forecasting paradigm. The T5 Model and resi…
Figure 5
Figure 5. Figure 5: Comparison of average rank across datasets and random seeds versus model effi￾ciency, measured in FLOPs. The marker size reflects the number of trainable parameters. Through our experiments, we find that: (1) Transformer-based models adapted from LLMs, such as T5, and …
Figure 6
Figure 6. Figure 6: (a, b) Performance ranks on compositional reasoning tasks across datasets for T5 Model design choices. Lower ranks are better; thick lines denote non-significant differences. Limitations and Future Work. This study has two main limitations related to data and model abl…
Figure 7
Figure 7. Figure 7: Critical Difference (CD) diagrams illustrate model ranks and pairwise statistical com [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: Critical Difference (CD) diagrams illustrate model ranks and pairwise statistical com [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: (a, c, e) In-distribution forecasts for Sinusoid, ECL, and ETTm2 datasets. (b, d, f) Corresponding out-of-distribution forecasts using the compositional reasoning paradigm. Top￾performing models (Patch-based Transformers, MLPs) generalize well OOD, unlike other variant…
Figure 10
Figure 10. Figure 10: (a, c, e) In-distribution forecasts for Solar, Subseasonal, and Loop Seattle datasets. (b, d, f) Corresponding out-of-distribution forecasts using the compositional reasoning paradigm. Top￾performing models (Patch-based Transformers, MLPs) generalize well OOD, unlike …
Figure 11
Figure 11. Figure 11: Comparison of average rank across datasets and random seeds versus model computational [PITH_FULL_IMAGE:figures/full_fig_p023_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Survey of Reasoning and Agentic Systems in Time Series with Large Language Models

    cs.AI 2025-09 conditional novelty 5.0 of 10

    The authors organize LLM-based time series reasoning into three exclusive topologies (direct, chain, branch) crossed with four objectives, and use them to label 125 papers, benchmarks, and resources.

  2. Cross-device Zero-shot Label Transfer via Alignment of Time Series Foundation Model Embeddings

    eess.SP 2025-08 reject novelty 4.0 of 10

    A framework using adversarial alignment of frozen time-series foundation model embeddings transfers labels to a simulated target domain, but the target is synthetic and no real consumer device data is tested.

Reference graph

Works this paper leans on

64 extracted references · 43 canonical work pages · cited by 2 Pith papers

  1. [1]

    Gift-eval: A benchmark for general time series forecasting model evaluation,

    Taha Aksu, Gerald Woo, Juncheng Liu, Xu Liu, Chenghao Liu, Silvio Savarese, Caiming Xiong, and Doyen Sahoo. Gift-eval: A benchmark for general time series forecasting model evaluation,

  2. [2]

    Physics of language models: Part 3.2, knowledge manipula- tion, 2023

    Zeyuan Allen-Zhu and Yuanzhi Li. Physics of language models: Part 3.2, knowledge manipula- tion, 2023

    work page 2023

  3. [3]

    Physics of language models: Part 3.1, knowledge storage and extraction

    Zeyuan Allen-Zhu and Yuanzhi Li. Physics of language models: Part 3.1, knowledge storage and extraction. InProceedings of the 41st International Conference on Machine Learning, page 235, 2024

    work page 2024

  4. [4]

    Maddix, Hao Wang, Michael W

    Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, Jasper Zschiegner, Danielle C. Maddix, Hao Wang, Michael W. Mahoney, Kari Torkkola, Andrew Gordon Wilson, Michael Bohlke-Schneider, and Yuyang Wang. Chronos: Learning the language ...

    2024

  5. [5]

    Zico Kolter, and Vladlen Koltun

    Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolu- tional and recurrent networks for sequence modeling, 2018

    work page 2018

  6. [6]

    Robert G. Brown. Exponential smoothing for predicting demand.Philip Morris Records, 1956

    work page 1956

  7. [7]

    TimeSeriesExam: A time series understanding exam.NeurIPS 2024 Workshop on Time Series in the Age of Large Model, 2024

    Yifu Cai, Arjun Choudhry, Mononito Goswami, and Artur Dubrawski. TimeSeriesExam: A time series understanding exam.NeurIPS 2024 Workshop on Time Series in the Age of Large Model, 2024

    work page 2024

  8. [8]

    Olivares, Boris N

    Cristian Challu, Kin G. Olivares, Boris N. Oreshkin, Federico Garza, Max Mergenthaler- Canseco, and Artur Dubrawski. N-HiTS: Neural hierarchical interpolation for time series forecasting. InAAAI-23, 2022

    work page 2022

Show all 64 references
  1. [9]

    Yoder, Sercan ¨O

    Si-An Chen, Chun-Liang Li, Nathanael C. Yoder, Sercan ¨O. Arık, and Tomas Pfister. TSMixer: An all-MLP architecture for time series forecasting. InPublished in Transactions on Machine Learning Research, 2023

    2023

  2. [10]

    Hallgrímsson, Maxwell A

    Winnie Chow, Lauren Gardiner, Haraldur T. Hallgrímsson, Maxwell A. Xu, and Shirley You Ren. Towards time-series reasoning with llms.NeurIPS 2024 Workshop on Time Series in the Age of Large Model, 2024

    2024

  3. [11]

    Deep bidirectional and unidirectional lstm recurrent neural network for network-wide traffic speed prediction.arXiv preprint arXiv:1801.02143, 2018

    Zhiyong Cui, Ruimin Ke, and Yinhai Wang. Deep bidirectional and unidirectional lstm recurrent neural network for network-wide traffic speed prediction.arXiv preprint arXiv:1801.02143, 2018

    2018 arXiv

  4. [12]

    Zhiyong Cui, Kristian Henrickson, Ruimin Ke, and Yinhai Wang. Traffic graph convolutional recurrent neural network: A deep learning framework for network-scale traffic learning and forecasting.IEEE Transactions on Intelligent Transportation Systems, 2019

    2019

  5. [13]

    A decoder-only foundation model for time-series forecasting, 2024

    Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. A decoder-only foundation model for time-series forecasting, 2024

    2024

  6. [14]

    Statistical comparisons of classifiers over multiple data sets

    Janez Demšsar. Statistical comparisons of classifiers over multiple data sets. InJournal of Machine Learning Research, pages 1–30, 2006

    2006

  7. [15]

    Scaling-laws for large time-series models.arXiv preprint arXiv:2405.13867, 2024

    Thomas DP Edwards, James Alvey, Justin Alsing, Nam H Nguyen, and Benjamin D Wandelt. Scaling-laws for large time-series models.arXiv preprint arXiv:2405.13867, 2024

    2024 arXiv

  8. [16]

    TTMs: Fast Multi-level Tiny Time Mixers for Improved Zero- shot and Few-shot Forecasting of Multivariate Time Series.arXiv preprint arXiv:2401.03955, 2024

    Vijay Ekambaram, Arindam Jati, Nam H Nguyen, Pankaj Dayama, Chandra Reddy, Wesley M Gifford, and Jayant Kalagnanam. TTMs: Fast Multi-level Tiny Time Mixers for Improved Zero- shot and Few-shot Forecasting of Multivariate Time Series.arXiv preprint arXiv:2401.03955, 2024. 10

    2024 arXiv

  9. [17]

    Cognitron: A self-organizing multilayered neural network.Biol

    Kunihiko Fukushima. Cognitron: A self-organizing multilayered neural network.Biol. Cyber- netics, 20:121––136, 1975

    1975

  10. [18]

    Units: A unified multi-task time series model

    Shanghua Gao, Teddy Koker, Owen Queen, Thomas Hartvigsen, Theodoros Tsiligkaridis, and Marinka Zitnik. Units: A unified multi-task time series model. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  11. [19]

    TimeGPT-1, 2023

    Azul Garza and Max Mergenthaler-Canseco. TimeGPT-1, 2023

    2023

  12. [20]

    Aqua: A benchmarking tool for label quality assessment.Advances in Neural Information Processing Systems, 36, 2024

    Mononito Goswami, Vedant Sanil, Arjun Choudhry, Arvind Srinivasan, Chalisa Udompanyawit, and Artur Dubrawski. Aqua: A benchmarking tool for label quality assessment.Advances in Neural Information Processing Systems, 36, 2024

    2024

  13. [21]

    MOMENT: A family of open time-series foundation models

    Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, and Artur Dubrawski. MOMENT: A family of open time-series foundation models. In41st International Conference on Machine Learning, 2024

    2024

  14. [22]

    How does gpt-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model

    Michael Hanna, Ollie Liu, and Alexandre Variengien. How does gpt-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model. In37th Conference on Neural Information Processing Systems, 2023

    2023

  15. [23]

    R. J. Hyndman and Y . Khandakar. Automatic time series forecasting: The forecast package for r.Journal of Statistical Software, 27(3):1–22, 2008

    2008

  16. [24]

    Olivares.Forecasting: Principles and Practice, the Pythonic Way

    Rob J Hyndman, George Athanasopoulos, Azul Garza, Cristian Challu, Max Mergenthaler, and Kin G. Olivares.Forecasting: Principles and Practice, the Pythonic Way. OTexts, Melbourne, Australia, 2024. available at https://otexts.com/fpppy/

    2024

  17. [25]

    Deep learning for time series classification: a review.Data Mining and Knowledge Discovery, 33(4):917–963, 2019

    Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, and Pierre- Alain Muller. Deep learning for time series classification: a review.Data Mining and Knowledge Discovery, 33(4):917–963, 2019

    2019

  18. [26]

    Libcity: A unified library towards efficient and comprehensive urban spatial-temporal prediction, 2024

    Jiawei Jiang, Chengkai Han, Wenjun Jiang, Wayne Xin Zhao, and Jingyuan Wang. Libcity: A unified library towards efficient and comprehensive urban spatial-temporal prediction, 2024. URLhttps://arxiv.org/abs/2304.14343

    2024 arXiv

  19. [27]

    Lake and Marco Baroni

    Brenden M. Lake and Marco Baroni. Generalization without systematicity: On the com- positional skills of sequence-to-sequence recurrent networks. InProceedings of the 35th International Conference on Machine Learning, 2018

    2018

  20. [28]

    Arık, Nicolas Loeff, and Tomas Pfister

    Bryan Lim, Sercan ¨O. Arık, Nicolas Loeff, and Tomas Pfister. Temporal fusion transformers for interpretable multi-horizon time series forecasting.International Journal of Forecasting, 37(4): 1748–1764, 2021

    2021

  21. [29]

    Moirai-moe: Empowering time series foundation models with sparse mixture of experts.arXiv preprint arXiv:2410.10469, 2024

    Xu Liu, Juncheng Liu, Gerald Woo, Taha Aksu, Yuxuan Liang, Roger Zimmermann, Chenghao Liu, Silvio Savarese, Caiming Xiong, and Doyen Sahoo. Moirai-moe: Empowering time series foundation models with sparse mixture of experts.arXiv preprint arXiv:2410.10469, 2024

    arXiv

  22. [30]

    iTransformer: Inverted transformers are effective for time series forecasting, 2024

    Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. iTransformer: Inverted transformers are effective for time series forecasting, 2024

    2024

  23. [31]

    Timer-xl: Long- context transformers for unified time series forecasting.arXiv preprint arXiv:2410.04803, 2024

    Yong Liu, Guo Qin, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Timer-xl: Long- context transformers for unified time series forecasting.arXiv preprint arXiv:2410.04803, 2024

    2024 arXiv

  24. [32]

    Timer: Generative Pre-trained Transformers Are Large Time Series Models

    Yong Liu, Haoran Zhang, Chenyu Li, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Timer: Generative Pre-trained Transformers Are Large Time Series Models. InForty-first International Conference on Machine Learning, 2024

    2024

  25. [33]

    Merrill, Mingtian Tan, Vinayak Gupta, Tom Hartvigsen, and Tim Althoff

    Mike A. Merrill, Mingtian Tan, Vinayak Gupta, Tom Hartvigsen, and Tim Althoff. Language models still struggle to zero-shot reason about time series, 2024

    2024

  26. [34]

    Subseasonalclimateusa: A dataset for subseasonal forecasting and benchmarking

    Soukayna Mouatadid, Paulo Orenstein, Genevieve Flaspohler, Miruna Oprescu, Judah Cohen, Franklyn Wang, Sean Knight, Maria Geogdzhayeva, Sam Levang, Ernest Fraenkel, and Lester Mackey. Subseasonalclimateusa: A dataset for subseasonal forecasting and benchmarking. In 37th Confer...

    2023

  27. [35]

    Vinod Nair and Jeoffrey E. Hinton. Rectified linear units improve restricted boltzmann machines. InICML-23, 2010. 11

    2010

  28. [36]

    Basisformer: Attention-based time series forecasting with learnable and interpretable basis

    Zelin Ni, Hang Yu, Shizhan Liu, Jianguo Li, and Weiyao Lin. Basisformer: Attention-based time series forecasting with learnable and interpretable basis. In37th Conference on Neural Information Processing Systems, 2024

    2024

  29. [37]

    Nguyen, and Phanwadee Sinthong an Jayant Kalagnanam2

    Yuqi Nie, Nam H. Nguyen, and Phanwadee Sinthong an Jayant Kalagnanam2. A time series is worth 64 words: Long-term forecasting with transformers. InProceedings of the 11th International Conference on Learning Representations, 2023

    2023

  30. [38]

    Olivares, Cristian Challu, Grzegorz Marcjasz, Rafal Weron, and Artur Dubrawski

    Kin G. Olivares, Cristian Challu, Grzegorz Marcjasz, Rafal Weron, and Artur Dubrawski. Neural basis expansion analysis with exogenous variables: Forecasting electricity prices with nbeatsx.International Journal of Forecasting, 39(2):884–900, 2022

    2022

  31. [39]

    Olivares, Cristian Challú, Federico Garza, Max Mergenthaler Canseco, and Artur Dubrawski

    Kin G. Olivares, Cristian Challú, Federico Garza, Max Mergenthaler Canseco, and Artur Dubrawski. NeuralForecast: User friendly state-of-the-art neural forecasting models. PyCon Salt Lake City, Utah, US 2022, 2022. URLhttps://github.com/Nixtla/neuralforecast

    2022

  32. [40]

    Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio

    Boris N. Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-BEATS: neural basis expansion analysis for interpretable time series forecasting. In8th International Con- ference on Learning Representations, ICLR 2020, 2020. URL https://openreview.net/ forum?id=r1ecqn4YwB

    2020

  33. [41]

    Lag-Llama: Towards foundation models for probabilistic time series forecasting, 2024

    Kashif Rasul, Arjun Ashok, Andrew Robert Williams, Hena Ghonia, Rishika Bhagwatkar, Arian Khorasani, Mohammad Javad Darvishi Bayazi, George Adamopoulos, Roland Riachi, Nadhir Hassen, Marin Biloš, Sahil Garg, Anderson Schneider, Nicolas Chapados, Alexandre Drouin, Valentina Zan...

    2024

  34. [42]

    The perceptron: A probabilistic model for information storage and organiza- tion in the brain.Psychological Review, 65(6):386—-408, 1958

    Frank Rosenblatt. The perceptron: A probabilistic model for information storage and organiza- tion in the brain.Psychological Review, 65(6):386—-408, 1958

    1958

  35. [43]

    Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition, 2014

    Ha¸ sim Sak, Andrew Senior, and Françoise Beaufays. Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition, 2014

    2014

  36. [44]

    Time-moe: Billion-scale time series foundation models with mixture of experts.arXiv preprint arXiv:2409.16040, 2024

    Xiaoming Shi, Shiyu Wang, Yuqi Nie, Dianqi Li, Zhou Ye, Qingsong Wen, and Ming Jin. Time-moe: Billion-scale time series foundation models with mixture of experts.arXiv preprint arXiv:2409.16040, 2024

    2024 arXiv

  37. [45]

    Merrill, Vinayak Gupta, Tim Althoff, and Thomas Hartvigsen

    Mingtian Tan, Mike A. Merrill, Vinayak Gupta, Tim Althoff, and Thomas Hartvigsen. Are language models actually useful for time series forecasting? InProceedings of the 38th Conference on Neural Information Processing Systems, 2024

    2024

  38. [46]

    WaveNet: A generative model for raw audio, 2016

    Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. WaveNet: A generative model for raw audio, 2016

    2016

  39. [47]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, and Aidan N. Gomez. Attention is all you need, 2017

    2017

  40. [48]

    Grokked transformers are implicit reasoners: A mechanistic journey to the edge of generalization

    Boshi Wang, Xiang Yue, Yu Su, and Huan Sun. Grokked transformers are implicit reasoners: A mechanistic journey to the edge of generalization. In38th Conference on Neural Information Processing Systems, 2024

    2024

  41. [49]

    Exploring representations and interventions in time series foundation models

    Michał Wili´nski, Mononito Goswami, Willa Potosnak, Nina ˙Zukowska, and Artur Dubrawski. Exploring representations and interventions in time series foundation models. InProceedings of the Forty-Second International Conference on Machine Learning, 2025

    2025

  42. [50]

    Context is key: A benchmark for forecasting with essential textual information, 2025

    Andrew Robert Williams, Arjun Ashok, Étienne Marcotte, Valentina Zantedeschi, Jithendaraa Subramanian, Roland Riachi, and et al. Context is key: A benchmark for forecasting with essential textual information, 2025

    2025

  43. [51]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020

  44. [52]

    Olivares, Boris Oreshkin, Sunny Ruan, Sitan Yang, Abhinav Katoch, Shankar Ramasubramanian, Youxin Zhang, Michael W

    Malcolm Wolff, Kin G. Olivares, Boris Oreshkin, Sunny Ruan, Sitan Yang, Abhinav Katoch, Shankar Ramasubramanian, Youxin Zhang, Michael W. Mahoney, Dmitry Efimov, and Vincent Quenneville-Bélair. ♠ SPADE ♠: Split peak attention decomposition. InThirty-Eighth Annual Conference on...

    2024 arXiv

  45. [53]

    Unified training of universal time series forecasting transformers

    Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Unified training of universal time series forecasting transformers. InProceedings of the 41st International Conference on Machine Learning, 2024

    2024

  46. [54]

    Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting, 2021

    Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting, 2021

    2021

  47. [55]

    TimesNet: Temporal 2d-variation modeling for general time series analysis

    Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. TimesNet: Temporal 2d-variation modeling for general time series analysis. InProceedings of the 34th International Conference on Learning Representations, 2023

    2023

  48. [56]

    Rethinking fourier transform from a basis functions perspective for long-term time series forecasting

    Runze Yang, Longbing Cao, Jianxun Li, and Jie Yang. Rethinking fourier transform from a basis functions perspective for long-term time series forecasting. InProceedings of the 38th Conference on Neural Information Processing Systems, 2024

    2024

  49. [57]

    Do large language models latently perform multi-hop reasoning?arXiv preprint arXiv:2402.16837, 2024

    Sohee Yang, Elena Gribovskaya, Nora Kassner, Mor Geva, and Sebastian Riedel. Do large language models latently perform multi-hop reasoning?arXiv preprint arXiv:2402.16837, 2024

    2024 arXiv

  50. [58]

    Towards neural scaling laws for time series foundation models.arXiv preprint arXiv:2410.12360, 2024

    Qingren Yao, Chao-Han Huck Yang, Renhe Jiang, Yuxuan Liang, Ming Jin, and Shirui Pan. Towards neural scaling laws for time series foundation models.arXiv preprint arXiv:2410.12360, 2024

    2024 arXiv

  51. [59]

    Are transformers effective for time series forecasting? InProceedings of the AAAI Conference on Artificial Intelligence, 2023

    Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? InProceedings of the AAAI Conference on Artificial Intelligence, 2023

    2023

  52. [60]

    MQuAKE: Assessing knowledge editing in language models via multi-hop questions.arXiv preprint arXiv:2305.14795, 2023

    Zexuan Zhong, Zhengxuan Wu, Christopher D Manning, Christopher Potts, and Danqi Chen. MQuAKE: Assessing knowledge editing in language models via multi-hop questions.arXiv preprint arXiv:2305.14795, 2023

    2023 arXiv

  53. [61]

    Informer: Beyond efficient transformer for long sequence time-series forecasting, 2021

    Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting, 2021

    2021

  54. [62]

    Pre-trained large language models use fourier features to compute addition, 2024

    Tianyi Zhou, Deqing Fu, Vatsal Sharan, and Robin Jia. Pre-trained large language models use fourier features to compute addition, 2024

    2024

  55. [63]

    Pre" column denotes the dataset properties prior to preprocessing, while “Post

    Nina ˙Zukowska, Mononito Goswami, Michał Wili´nski, Willa Potosnak, and Artur Dubrawski. Towards long-context time series foundation models.arXiv preprint arXiv:2409.13530, 2024. 13 6 Supplemental Information 6.1 Reproducibility Statement The models were implemented using the ...

    2024 arXiv

  56. [2024]

    URLhttps://arxiv.org/abs/2410.10393

    arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.