RECORD
BasisFormer: Attention-based Time Series Forecasting with Learnable and Interpretable Basis
Machine Learning · Submitted 31 Oct 2023
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Through extensive experiments on six datasets, we demonstrate that BasisFormer outperforms previous state-of-the-art methods by 11.04\% and 15.78\% respectively for univariate and multivariate forecasting tasks.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Bases have become an integral part of modern deep learning-based models for time series forecasting due to their ability to act as feature extractors or future references. To be effective, a basis must be tailored to the specific set of time series data and exhibit distinct correlation with each time series within the set. However, current state-of-the-art methods are limited in their ability to satisfy both of these requirements simultaneously. To address this challenge, we propose BasisFormer, an end-to-end time series forecasting architecture that leverages learnable and interpretable bases. This architecture comprises three components: First, we acquire bases through adaptive self-supervised learning, which treats the historical and future sections of the time series as two distinct views and employs contrastive learning. Next, we design a Coef module that calculates the similarity coefficients between the time series and bases in the historical view via bidirectional cross-attention. Finally, we present a Forecast module that selects and consolidates the bases in the future view based on the similarity coefficients, resulting in accurate future predictions. Through extensive experiments on six datasets, we demonstrate that BasisFormer outperforms previous state-of-the-art methods by 11.04\% and 15.78\% respectively for univariate and multivariate forecasting tasks. Code is available at: \url{https://github.com/nzl5116190/Basisformer}
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Bases have become an integral part of modern deep learning-based models for time series forecasting due to their ability to act as feature extractors or future references. | background 1.00 | 0.05 | 0.02 | asserted 1.00 |
| s1 | To be effective, a basis must be tailored to the specific set of time series data and exhibit distinct correlation with each time series within the set. | background 0.98 | 0.05 | 0.02 | asserted 0.99 |
| s2 | However, current state-of-the-art methods are limited in their ability to satisfy both of these requirements simultaneously. | gap 0.98 | 0.06 | 0.02 | asserted 1.00 |
| s3 | To address this challenge, we propose BasisFormer, an end-to-end time series forecasting architecture that leverages learnable and interpretable bases. | contribution 1.00 | 0.84 | 0.08 | asserted 0.99 |
| s4 | This architecture comprises three components: First, we acquire bases through adaptive self-supervised learning, which treats the historical and future sections of the time series as two distinct views and employs contrastive learning. | method 0.97 | 0.92 | 0.03 | asserted 0.98 |
| s5 | Next, we design a Coef module that calculates the similarity coefficients between the time series and bases in the historical view via bidirectional cross-attention. | method 0.98 | 0.92 | 0.02 | asserted 0.99 |
| s6 | Finally, we present a Forecast module that selects and consolidates the bases in the future view based on the similarity coefficients, resulting in accurate future predictions. | method 0.99 | 0.92 | 0.07 | asserted 1.00 |
| s7 | Through extensive experiments on six datasets, we demonstrate that BasisFormer outperforms previous state-of-the-art methods by 11.04\% and 15.78\% respectively for univariate and multivariate forecasting tasks. | result 1.00 | 0.97 | 0.80 | asserted 1.00 |
| s8 | Code is available at: \url{https://github.com/nzl5116190/Basisformer} | contribution 0.67 | 0.31 | 0.02 | asserted 0.85 |
What was examined, and what was not
- arXiv listing abstract v2
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Bases have become an integral part of modern deep learning-based models for time series forecasting due to their ability to act as feature extractors or future references. To be effective, a basis must be tailored to the specific set of time series data and exhibit distinct correlation with each time series within the set. However, current state-of-the-art methods are limited in their ability to satisfy both of these requirements simultaneously. To address this challenge, we propose BasisFormer, an end-to-end time series forecasting architecture that leverages learnable and interpretable bases. This architecture comprises three components: First, we acquire bases through adaptive self-supervised learning, which treats the historical and future sections of the time series as two distinct views and employs contrastive learning. Next, we design a Coef module that calculates the similarity coefficients between the time series and bases in the historical view via bidirectional cross-attention. Finally, we present a Forecast module that selects and consolidates the bases in the future view based on the similarity coefficients, resulting in accurate future predictions. Through extensive experiments on six datasets, we demonstrate that BasisFormer outperforms previous state-of-the-art methods by 11.04\% and 15.78\% respectively for univariate and multivariate forecasting tasks. Code is available at: \url{https://github.com/nzl5116190/Basisformer}
Human review
How the literature uses it
1 arXiv paper cite this in 1 sentence
Papers citing it
Cites it as background 1 paper
-
For instance, [ref] proposes a spatio-temporal attention model, others leverage trend-aware decomposition in the definition of the attention layer [ref] .
Teger: Spatiotemporal Covariance for Probabilistic Traffic Forecasting · Related Work · 0.92
What it cites · 21 sentences without a confident label
Compares with 4 papers
-
Models for comparison: In this study, we compare our proposed model against the following state-of-the-art models: four transformer-based models, namely FEDformer [ref] , Autoformer [ref] , Pyraformer [ref] ; one MLP-based model, i.e., Dlinear [ref] ; and one CNN-based model, i.e., TCN [ref] .
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling · Experiments · 1.00
-
Models for comparison: In this study, we compare our proposed model against the following state-of-the-art models: four transformer-based models, namely FEDformer [ref] , Autoformer [ref] , Pyraformer [ref] ; one MLP-based model, i.e., Dlinear [ref] ; and one CNN-based model, i.e., TCN [ref] .
FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting · Experiments · 1.00
-
Models for comparison: In this study, we compare our proposed model against the following state-of-the-art models: four transformer-based models, namely FEDformer [ref] , Autoformer [ref] , Pyraformer [ref] ; one MLP-based model, i.e., Dlinear [ref] ; and one CNN-based model, i.e., TCN [ref] .
Are Transformers Effective for Time Series Forecasting? · Experiments · 1.00
Cites as background 8 papers
-
The first group involves N-BEATS [ref] , N-HiTS [ref] , and FiLM [ref] .
N-HiTS: Neural Hierarchical Interpolation for Time Series Forecasting · Related works · 0.91
-
The first group involves N-BEATS [ref] , N-HiTS [ref] , and FiLM [ref] .
N-BEATS: Neural basis expansion analysis for interpretable time series forecasting · Related works · 0.91
-
On the other hand, the models based on learnable bases, such as Dlinear [ref] , TCN [ref] , Deepar [ref] , LogTrans [ref] , Informer [ref] , AutoFormer [ref] , FedFormer [ref] , etc, use a learnable linear or convolution layer, or covariate embeddings as the basis.
Are Transformers Effective for Time Series Forecasting? · Related works · 0.84
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.