Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
read the original abstract
Many real-world applications require the prediction of long sequence time-series, such as electricity consumption planning. Long sequence time-series forecasting (LSTF) demands a high prediction capacity of the model, which is the ability to capture precise long-range dependency coupling between output and input efficiently. Recent studies have shown the potential of Transformer to increase the prediction capacity. However, there are several severe issues with Transformer that prevent it from being directly applicable to LSTF, including quadratic time complexity, high memory usage, and inherent limitation of the encoder-decoder architecture. To address these issues, we design an efficient transformer-based model for LSTF, named Informer, with three distinctive characteristics: (i) a $ProbSparse$ self-attention mechanism, which achieves $O(L \log L)$ in time complexity and memory usage, and has comparable performance on sequences' dependency alignment. (ii) the self-attention distilling highlights dominating attention by halving cascading layer input, and efficiently handles extreme long input sequences. (iii) the generative style decoder, while conceptually simple, predicts the long time-series sequences at one forward operation rather than a step-by-step way, which drastically improves the inference speed of long-sequence predictions. Extensive experiments on four large-scale datasets demonstrate that Informer significantly outperforms existing methods and provides a new solution to the LSTF problem.
This paper has not been read by Pith yet.
Forward citations
Cited by 13 Pith papers
-
ISOMORPH: A Supply Chain Digital Twin for Simulation, Dataset Generation, and Forecasting Benchmarks
ISOMORPH is a modular digital twin simulator for supply chain networks that releases datasets exhibiting variance amplification and regime shifts for benchmarking forecasting models and performing forward uncertainty ...
-
ISOMORPH: A Supply Chain Digital Twin for Simulation, Dataset Generation, and Forecasting Benchmarks
ISOMORPH supplies the first public configurable digital-twin simulator for multi-echelon logistics networks together with generated datasets and zero-shot foundation-model benchmarks that exceed GIFT-Eval MASE at shor...
-
TelecomTS: A Multi-Modal Observability Dataset for Time Series and Language Analysis
TelecomTS is a new observability dataset from 5G networks that preserves absolute scale and supports multi-modal tasks, showing that current time series and language models struggle with abrupt noisy dynamics.
-
Spectral Transformer Neural Processes
STNPs extend TNPs with a spectral aggregator that estimates context spectra, forms spectral mixtures, and injects task-adaptive frequency features to better handle periodicity.
-
Dynamic Linear Coregionalization for Realistic Synthetic Multivariate Time Series
DynLMC creates synthetic multivariate time series with dynamic inter-channel correlations that improve zero-shot forecasting performance when used to fine-tune foundation models across nine benchmarks.
-
Dynamic Linear Coregionalization for Realistic Synthetic Multivariate Time Series
DynLMC creates synthetic time series data with dynamic inter-channel correlations that improve zero-shot forecasting in foundation models across multiple benchmarks.
-
Neural CDEs as Correctors for Learned Time Series Models
Neural CDEs serve as correctors that reduce error accumulation in multi-step forecasts from learned time-series models across synthetic, physics, and real-world data.
-
iTransformer: Inverted Transformers Are Effective for Time Series Forecasting
By applying attention and feed-forward networks to inverted variate tokens instead of temporal tokens, iTransformer achieves state-of-the-art performance on real-world time series forecasting datasets.
-
Dirichlet-Guided Group Forecasting for Alleviating Over-smoothing in Time Series Forecasting
DGF explicitly models multiple mode-conditioned predictive distributions via Dirichlet-guided sampling and reward optimization to preserve dynamical features in time series forecasts.
-
Spatiotemporal decoupled physics-informed Stone-Weierstrass neural operator for long-time prediction of time-dependent parametric PDEs
A spatiotemporally decoupled physics-informed Stone-Weierstrass neural operator for stable long-time prediction of time-dependent parametric PDEs.
-
Learning Low-Dimensional Representation for O-RAN Testing via Transformer-ESN
Transformer-ESN learns 8D embeddings from O-RAN KPIs that reduce MSE for RSRQ by up to 41.9% and spectral efficiency by 29.9% versus full high-dimensional data in limited-sample regimes.
-
MR-ImagenTime: Multi-Resolution Time Series Generation through Dual Image Representations
MR-CDM uses hierarchical multi-resolution decomposition and multi-scale conditional diffusion to generate forecasts that reduce MAE and RMSE by 6-10% versus baselines like CSDI and Informer on four datasets.
-
Signed Dual Attention: Capturing Signed Dependencies in Time Series Forecasting
Signed Dual Attention is a parameter-free attention module that models signed dependencies in time series via dual message passing to achieve two-head expressiveness in one block.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.