REVIEW 17 cited by
Are Transformers Effective for Time Series Forecasting?
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Are Transformers Effective for Time Series Forecasting?
read the original abstract
Recently, there has been a surge of Transformer-based solutions for the long-term time series forecasting (LTSF) task. Despite the growing performance over the past few years, we question the validity of this line of research in this work. Specifically, Transformers is arguably the most successful solution to extract the semantic correlations among the elements in a long sequence. However, in time series modeling, we are to extract the temporal relations in an ordered set of continuous points. While employing positional encoding and using tokens to embed sub-series in Transformers facilitate preserving some ordering information, the nature of the \emph{permutation-invariant} self-attention mechanism inevitably results in temporal information loss. To validate our claim, we introduce a set of embarrassingly simple one-layer linear models named LTSF-Linear for comparison. Experimental results on nine real-life datasets show that LTSF-Linear surprisingly outperforms existing sophisticated Transformer-based LTSF models in all cases, and often by a large margin. Moreover, we conduct comprehensive empirical studies to explore the impacts of various design elements of LTSF models on their temporal relation extraction capability. We hope this surprising finding opens up new research directions for the LTSF task. We also advocate revisiting the validity of Transformer-based solutions for other time series analysis tasks (e.g., anomaly detection) in the future. Code is available at: \url{https://github.com/cure-lab/LTSF-Linear}.
Forward citations
Cited by 17 Pith papers
-
ConTex: Reformulating Counterfactual Generation For Time Series Forecasting
ConTex learns a global intervention strategy via a decomposed temporal-conditional encoder architecture to generate consistent, sparse counterfactuals for time series models in a single forward pass.
-
CloudCons: A Comprehensive End-to-End Benchmark for Cloud Resource Consolidation
CloudCons benchmark shows foundation models' superior zero-shot forecasting does not automatically yield better resource consolidation decisions, with predictive quantile choice acting as a key lever for efficiency-re...
-
A Time Series is Worth 64 Words: Long-term Forecasting with Transformers
PatchTST uses subseries patching and channel-independent Transformers to deliver significantly better long-term multivariate time series forecasting and strong self-supervised transfer performance.
-
Co4ICF: Co-evolving Physics-Informed Surrogate and RL-based Pulse Optimizer for Inertial Confinement Fusion
A co-evolving physics-informed surrogate and PPO optimizer reaches 146% (1D) and 247% (2D) normalized ICF yield versus baseline without 2D training data.
-
Flow-PIN: A Two-Stage Power-Flow-Guided Method for System-Wide Multivariate Profile Inpainting in Distribution Networks
A two-stage framework combining conditional flow matching with a topology-aware power-flow-guided refiner improves the accuracy and physical consistency of missing power grid data imputation.
-
GITCO: Gated Inference-Time Context Optimization in TSFMs
GITCO delivers +1.95% average MASE reduction on TimesFM 2.5 across 53 datasets by gated inference-time suppression of anomalous patches, capturing 89.9% of the improvement upper bound.
-
Beyond Similarity: Temporal Operator Attention for Time Series Analysis
TOA augments attention with learnable sequence-space operators and stochastic regularization to enable signed temporal mixing, yielding gains on forecasting and related benchmarks when added to PatchTST and iTransformer.
-
Temporal Patch Shuffle (TPS): Leveraging Patch-Level Shuffling to Boost Generalization and Robustness in Time Series Forecasting
TPS is a patch-level shuffling augmentation for time series forecasting that increases training diversity while preserving local temporal structure, leading to consistent performance gains across multiple models and datasets.
-
Neural CDEs as Correctors for Learned Time Series Models
Neural CDEs serve as correctors that reduce error accumulation in multi-step forecasts from learned time-series models across synthetic, physics, and real-world data.
-
Kairos: Toward Adaptive and Parameter-Efficient Time Series Foundation Models
Kairos is a parameter-efficient time series foundation model using dynamic patching tokenizer, mixture-of-size encoding, and spectral-conditioned positional embeddings to improve zero-shot forecasting on heterogeneous data.
-
RhyMix: A Lightweight Adaptive Multi-Rhythm Network for Long-Term Time Series Forecasting
RhyMix reaches state-of-the-art long-term multivariate forecasting on 10 of 12 public benchmarks with a ~40K-parameter dual-path adaptive architecture of linear complexity.
-
Continuity and Ordinality Matter: Constraining Time Series Tokens for Effective Time Series Analysis with Large Language Models
COM integrates geometric constraints into token initialization and training to preserve continuity and ordinality in time series tokens, improving token-based TS-LLM performance on benchmarks.
-
Beyond Similarity: Temporal Operator Attention for Time Series Analysis
Temporal Operator Attention augments softmax attention with learnable sequence-space operators for signed temporal mixing and uses stochastic regularization to enable practical training, yielding consistent gains on t...
-
MSTN: A Lightweight and Fast Model for General TimeSeries Analysis
MSTN introduces a lightweight multi-scale temporal network using convolutional encoding, recurrent or attention-based modeling, and gated fusion to achieve claimed state-of-the-art results on 21 of 27 time series benc...
-
Revisiting Long-term Time Series Forecasting: An Investigation on Linear Mapping
Affine mapping dominates LTSF benchmarks by learning similar input-to-output transition matrices, captures periodic signals well but struggles with non-periodic or cross-channel varying periods; reversible normalizati...
-
MSTN: A Lightweight and Fast Model for General TimeSeries Analysis
MSTN is a lightweight hybrid model that reports new state-of-the-art results on 33 of 40 time series benchmarks for imputation, forecasting, and classification while using under one million parameters and sub-second i...
-
Time Series Forecasting Through the Lens of Dynamics
Proposes dynamics-based analysis of time series models showing partial dynamics learning and end-positioning as key to performance, plus a plug-and-play improvement method.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.