Pith. sign in

REVIEW 3 major objections 4 minor 59 references

FreEformer: Frequency Enhanced Transformer for Multivariate Time Series Forecasting

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A frequency-domain transformer that adds a learnable matrix to softmax attention reports state-of-the-art results on 18 real-world forecasting benchmarks.

desk verdict A simple, plausible attention plug-in with broad experiments, but the 'consistently outperforms' claim is contradicted by its own tables and the rank theory doesn't prove what it says. read the letter →

arxiv 2501.13989 v1 pith:AIPKRP4H submitted 2025-01-23 cs.LG cs.AI

Wenzhen Yue , Yong Liu , Xianghua Ying , Bowei Xing , Ruohao Guo , Ji Shi This is my paper · ORCID
classification cs.LGcs.AI
keywords frequency-domaintransformermultivariatetimeseriesforecastingenhancedattentionrankdiscreteFouriertransformcross-variatedependencysoftmaxgradientflow
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FreEformer sets out to show that multivariate forecasting improves when a Transformer operates on the discrete Fourier spectra of the series rather than on raw time steps, and that the standard softmax attention matrix should be augmented with a learnable positive matrix before row normalization. The paper's diagnosis is that frequency-domain attention is abnormally low-rank because spectra are sparse and softmax concentrates on large values, and that this limits representation diversity. Its remedy is the enhanced attention map $\mathrm{Norm}(\mathrm{Softmax}(QK^T/\sqrt{D})+\mathrm{Softplus}(B))V$, which adds an input-independent learnable term intended to raise rank and improve gradient flow. The paper reports consistent state-of-the-art MSE on eighteen real-world benchmarks spanning electricity, traffic, weather, healthcare, and finance, and shows the same attention change improves four existing transformer forecasters when nothing else is modified. A sympathetic reader would care because both ingredients are simple to reuse: a frequency tokenization scheme and a one-line attention change.

What carries the argument

The load-bearing object is the enhanced attention map $$\mathrm{EnhAttn}(Q,K,V)=\mathrm{Norm}\left(\mathrm{Softmax}(QK^T/\sqrt{D})+\mathrm{Softplus}(B)\right)V,$$ with row-wise L1 normalization; the Softplus ensures positive entries and prevents division by zero. Around this sits a frequency-token construction: apply the DFT along time, keep only the first $\lceil(T+1)/2\rceil$ bins by Hermitian symmetry, process the real and imaginary parts as two independent $N\times D$ variate-token matrices through the same transformer blocks, and recombine with the IDFT. Theorem 1—the DFT-pair identity $WF+b\leftrightarrow \sum_i \Omega_i \circledast M_i(x)+\mathrm{IDFT}(b)$—shows that linear mixing in the frequency domain is a sum of circular convolutions in time, giving the frequency representation its theoretical justification. The rank argument rests on Theorem 2's bounds $|\mathrm{rank}(A)-\mathrm{rank}(B)|\le \mathrm{rank}(A+B)\le \mathrm{rank}(A)+\mathrm{rank}(B)$, paired with the Jacobian calculation $\partial c/\partial a=(1/\|\tilde b\|_1)(\mathrm{Diag}(\tilde a)-\tilde a\tilde a^T)$, which shows the added branch rescales gradients by a learnable factor and gives the model an extra route for gradient flow.

What would settle it

Train FreEformer on ECL and Weather with several seeds and compute the numerical rank of $\mathrm{Softplus}(B)$ and of $\mathrm{Softmax}(QK^T/\sqrt{D})+\mathrm{Softplus}(B)$ per layer. If the additive matrix is as low-rank as the softmax matrix, or the final attention rank is essentially unchanged, the proposed rank-raising mechanism is not what drives the reported gains. A simpler behavioral probe: replace the learned matrix with a fixed random positive full-rank matrix; if accuracy is unchanged, the learned content of $B$ is irrelevant.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that the low-rank attention matrix observed in frequency-domain transformers is a real bottleneck and that adding an input-independent learnable matrix $\mathrm{Softplus}(B)$ to the softmax attention matrix—followed by row-wise L1 normalization—removes it. The authors establish Theorem 1, a DFT-pair identity showing a frequency-domain linear projection equals a sum of circular convolutions with modulated copies of the input, which justifies learning cross-variate dependencies in the spectrum. They report that FreEformer outperforms ten baselines on eighteen real-world datasets, and that enhanced attention as a drop-in replacement yields average MSE improvements of 5.9% for iTransformer, 9.9% for PatchTST, 1.4% for Leddam, and 3.8% for FreEformer itself. They also derive Jacobian expressions showing the modified attention keeps softmax's gradient structure while adding a learnable scaling factor, which they interpret as greater flexibility for gradient flow.

Load-bearing premise

The load-bearing premise is that the learned additive matrix $\mathrm{Softplus}(B)$ is nearly full-rank after training, so adding it to the low-rank softmax attention matrix raises the rank; the rank-sum inequality alone does not guarantee that increase.

Editorial extensions

If this is right

  • The enhanced attention is a drop-in module: replacing only the attention in iTransformer, PatchTST, Leddam, and Fredformer improves their average MSE, so the mechanism transfers across structurally different transformer forecasters.
  • Frequency-domain variate tokens beat time-domain tokens under identical attention settings, so the DFT representation itself, not the transformer alone, contributes to the gains.
  • The rank diagnosis predicts the largest gains where softmax attention is most low-rank; the paper documents the transition from low-rank vanilla attention to higher-rank final attention on Weather and ECL.
  • Longer lookback windows improve frequency resolution and lower error on ECL and Traffic, consistent with the global-spectrum assumption at the core of the method.
  • FreEformer trained from scratch matches or exceeds a fine-tuned pre-trained transformer on several datasets, indicating that the frequency-domain design is data-efficient.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An extension the authors leave implicit is that the same enhanced attention could be tested in other domains where softmax attention is known to collapse in rank, such as long-context language or vision transformers; the prediction is that adding an input-independent positive matrix would improve them as well.
  • Because the additive term is input-independent, FreEformer can be read as blending a static, dataset-specific cross-variate mixing prior with the dynamic softmax term; one testable design is to initialize B to a hand-chosen full-rank mixing matrix and see whether the rank mechanism alone explains the gain.
  • A diagnostic that would help future work is to track the effective rank (singular-value spectrum) of the attention matrix per layer during training, since the paper's explanation is about rank and gradient flow rather than about any particular learned pattern in B.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. FreEformer is a multivariate time-series forecasting model that applies a Transformer to frequency-domain variate tokens. The input is mapped to the DFT domain, the real and imaginary parts are processed in independent branches, and the usual softmax attention is replaced by EnhAttn(Q,K,V)=Norm(Softmax(QK^T/sqrt(D))+Softplus(B))V with row-wise L1 normalization. The paper claims that this enhanced attention increases the rank of the attention matrix and improves gradient flow, that FreEformer 'consistently outperforms state-of-the-art models on eighteen real-world benchmarks,' and that the enhanced attention mechanism 'consistently improves' existing Transformer-based forecasters as a plug-in. The experimental section covers long- and short-term forecasting, ablation studies, attention variants, efficiency, robustness, and a comparison with pre-trained models.

Significance. If the empirical and theoretical claims were fully supported, the attention modification would be a simple and broadly useful plug-in, and FreEformer would be a strong benchmark. The paper has notable strengths: a wide benchmark coverage with full tables in the appendix, standard baselines, multi-seed robustness results for a subset of datasets, extensive ablations of the proposed components, a formal DFT-convolution equivalence theorem, and a public code repository. However, the central 'consistent SOTA' claim is contradicted by the paper's own tables, and the theoretical rank argument is not a theorem-level derivation. The manuscript is therefore a useful empirical study of an interesting attention variant, but the abstract and conclusions currently overstate what is demonstrated.

major comments (3)
  1. [Abstract, §5.1, Tables 2–4] The claim that FreEformer 'consistently outperforms state-of-the-art models on eighteen real-world benchmarks' is not supported by the results reported in the paper itself. In Table 2, FreEformer loses on Exchange Avg MSE (0.354 vs 0.333 for Fredformer), on Traffic Avg MSE (0.435 vs 0.428 for iTransformer), on ETTh2 Avg MSE (0.372 vs 0.364 for TimeMixer), and on Solar Avg MSE (0.217 vs 0.216 for TimeMixer). In Table 3, FreEformer loses on Wiki S2 MSE (6.259 vs 5.931 for Fredformer) and on METR-LA S2 MSE (0.840 vs 0.819 for DLinear and 0.804 for FreTS). In Table 4, FreEformer loses on Traffic Avg MSE (0.435 vs 0.421 for FreDF). These are not external disputes but contradictions within the paper. The authors should either replace 'consistently outperforms' with a precise win/loss statement (e.g., best on M of N settings, average rank, or median MSE) and discuss the datasets/settings where baselines win, or substantially weaken the abstract and contribution statements.
  2. [§4.2, Theorem 2, Figure 4] The theoretical argument for rank increase is not logically derived. Theorem 2 only states the bounds |rank(A)-rank(B)| <= rank(A+B) <= rank(A)+rank(B); neither inequality implies rank(A+B) > rank(A). The text says that 'According to Theorem 2, the combined matrix A+B generally achieves a higher rank,' but the theorem is silent on this unless additional assumptions are provided. The paper relies on the empirical observation that Softplus(B) is 'nearly full-rank,' but this is only illustrated for two attention matrices in Figure 4. Even if B were full rank, the lower bound N - rank(A) would only guarantee an increase when rank(A) < N/2, a condition that is neither stated nor checked across layers and datasets. Please provide evidence on the rank or singular-value spectrum of Softplus(B) over training, layers, and datasets, or present the rank increase as an empirical motivation rather than a theorem-driven guarantee.
  3. [§5.3, Table 9] The plug-in claim that enhanced attention 'consistently improves' existing Transformer-based forecasters is also contradicted by the paper's own Table 9. For example, Fredformer on ETTm1 gives MSE 0.385 with enhanced attention versus 0.384 with vanilla attention; PatchTST on METR-LA is unchanged at 0.335; and Leddam on Weather is unchanged at 0.242. The reported average improvements (5.9% for iTransformer, 9.9% for PatchTST, 1.4% for Leddam, 3.8% for FreEformer) mask per-setting regressions. The claim should be restricted to average improvement over the chosen six datasets, or the authors should report the full win/loss counts and discuss settings where the plug-in fails. In addition, the statement that FreEformer 'consistently outperforms these improved forecasters' is not supported by Table 9: the improved iTransformer achieves 0.329 on METR-LA while FreEformer reports 0.336 in Table 3.
minor comments (4)
  1. [Abstract] The abstract contains the typo 'aFrequency Enhanced Transformer'; it should read 'a Frequency Enhanced Transformer.'
  2. [Table 2 caption] The caption states 'These settings are used throughout the following tables,' but Table 3 uses a different short-term forecasting protocol; the sentence is ambiguous and should be made precise.
  3. [Table 9] Table 9 should include the per-dataset counts of wins/losses/ties for the plug-in experiments; the text currently reports only average percentage improvements, which is insufficient for a 'consistent improvement' claim.
  4. [Appendix G.2] The full plug-in results in Table 16 repeat the same numbers for several PatchTST rows (e.g., the Weather row appears identical to the average row in the main Table 9); please verify that the table has not been mis-transcribed.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; the central derivation is self-contained and the only self-citations are non-load-bearing.

full rationale

FreEformer's central derivation is self-contained: the frequency-domain variate-token architecture and the enhanced attention of Eq. (9) are defined from first principles, and the rank/gradient discussion is post-hoc motivation supported by standard matrix inequalities, not a fitted prediction. The only self-citations (e.g., [Yue et al., 2024] for LSoftmax in Section 2.3, and [Liu et al., 2024a] for the iTransformer baseline) appear in related-work or baseline contexts and do not carry the argument; no uniqueness theorem or prior result by the authors is invoked to force the design. Claims that the learned B is near-full-rank are verified by visualizations of trained models, and the claim that rank(A+B)>rank(A) is not derived from Theorem 2 alone; this is a weakness in evidence but not circularity. The paper's own tables (Exchange, Traffic, Wiki, METR-LA in Tables 2-4; ETTm1 and METR-LA in Table 9) contradict the abstract's 'consistently outperforms', but such internal inconsistencies are correctness/fairness concerns, not a reduction of the output to the input. Overall, no significant circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

No new physical entities; the enhanced attention residual B is a model parameter. Key free parameters are standard hyperparameters.

free parameters (5)
  • Hidden dimension D = 128, 256, or 512 selected per dataset
    Model width chosen by validation; affects capacity and performance.
  • Embedding dimension d = 16
    Fixed at 16; ablation shows robust.
  • Loss weight alpha = 0.5
    Weighted L1 loss exponent chosen by hyperparameter search.
  • Batch size = 4, 8, 16, or 32
    Selected based on dataset size.
  • Initial learning rate = 1e-4 or 5e-4
    Selected from a two-value set.
assumptions (5)
  • standard math DFT properties: Hermitian symmetry, convolution theorem, time shift theorem
    Used in Theorem 1 and frequency-domain modeling.
  • domain assumption Frequency spectra provide a global perspective and are consistent across temporal spans
    Motivates frequency-domain forecasting; Figure 1.
  • domain assumption Cross-variate correlations in frequency domain are beneficial for representation learning
    Rationale for variate tokens.
  • ad hoc to paper The learnable matrix Softplus(B) is nearly full-rank at convergence
    Required for rank increase; only shown empirically in Figure 4.
  • domain assumption Softmax produces near-zero entries causing low-rank attention
    Claimed cause of low-rank issue.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FreEformer: Frequency Enhanced Transformer for Multivariate Time Series Forecasting." pith.science (2026). https://pith.science/paper/AIPKRP4H

@misc{pith2026250113989,
  author       = {Pith},
  title        = {Pith review of: FreEformer: Frequency Enhanced Transformer for Multivariate Time Series Forecasting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AIPKRP4H}},
  note         = {Machine review of arXiv:2501.13989}
}
read the original abstract

This paper presents \textbf{FreEformer}, a simple yet effective model that leverages a \textbf{Fre}quency \textbf{E}nhanced Trans\textbf{former} for multivariate time series forecasting. Our work is based on the assumption that the frequency spectrum provides a global perspective on the composition of series across various frequencies and is highly suitable for robust representation learning. Specifically, we first convert time series into the complex frequency domain using the Discrete Fourier Transform (DFT). The Transformer architecture is then applied to the frequency spectra to capture cross-variate dependencies, with the real and imaginary parts processed independently. However, we observe that the vanilla attention matrix exhibits a low-rank characteristic, thus limiting representation diversity. This could be attributed to the inherent sparsity of the frequency domain and the strong-value-focused nature of Softmax in vanilla attention. To address this, we enhance the vanilla attention mechanism by introducing an additional learnable matrix to the original attention matrix, followed by row-wise L1 normalization. Theoretical analysis~demonstrates that this enhanced attention mechanism improves both feature diversity and gradient flow. Extensive experiments demonstrate that FreEformer consistently outperforms state-of-the-art models on eighteen real-world benchmarks covering electricity, traffic, weather, healthcare and finance. Notably, the enhanced attention mechanism also consistently improves the performance of state-of-the-art Transformer-based forecasters.

Figures

Figures reproduced from arXiv: 2501.13989 by the authors.

Figure 1
Figure 1. Time series and their corresponding frequency spectra. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overall structure of the FreEformer. We leverage the frequency spectrum to capture temporal patterns and employ an enhanced [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Attention matrices from state-of-the-art forecasters on the [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Attention matrices from vanilla and enhanced attention. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Illustration of the Jacobian matrices of [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Visualization of the forecasting results under the ‘Input [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Illustration of enhanced attention variants. By ensuring [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Attention matrices of iTransformer from vanilla and enhanced attention. The left column shows the low-rank attention matrix from the vanilla self-attention mechanism (Weather: 8, ECL: 260), with most entries near zero. The right three columns show the original attentio…
Figure 9
Figure 9. Figure 9: Attention matrices of PatchTST from vanilla and enhanced attention. The left column shows the low-rank attention matrix from the vanilla self-attention mechanism (Weather: 3, ECL: 10), with most entries near zero. The right three columns show the original attention mat…
Figure 10
Figure 10. Figure 10: Attention matrices of Leddam from vanilla and enhanced attention. The left column shows the low-rank attention matrix from the vanilla self-attention mechanism (Weather: 19, ECL: 178), with most entries near zero. The right three columns show the original attention ma…
Figure 11
Figure 11. Figure 11: Attention matrices of Fredformer from vanilla and enhanced attention. The left column shows the low-rank attention matrix from the vanilla self-attention mechanism (Weather: 16, ECL: 196), with most entries near zero. The right three columns show the original attentio…
Figure 12
Figure 12. Figure 12: Performance comparison under various lookback lengths, [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Performance comparison under various training ratios [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: Comparison of model efficiency. Lookback and prediction lengths are both 96, with a fixed batch size of 16. The Y-axis is inverted [PITH_FULL_IMAGE:figures/full_fig_p020_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 42 canonical work pages

  1. [1]

    Chronos: Learning the language of time series

    Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, et al. Chronos: Learning the language of time series. arXiv preprint arXiv:2403.07815 , 2024

    arXiv 2024

  2. [2]

    An empirical evaluation of generic convolutional and recurrent networks for sequence modeling

    Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 , 2018

    arXiv 2018

  3. [3]

    Tamp-s2gcnets: coupling time-aware multipersistence knowledge representation with spatio-supra graph convolutional networks for time-series forecasting

    Yuzhou Chen, Ignacio Segovia-Dominguez, Baris Coskunuzer, and Yulia Gel. Tamp-s2gcnets: coupling time-aware multipersistence knowledge representation with spatio-supra graph convolutional networks for time-series forecasting. In International Conference on Learning Representations , 2022

    work page 2022

  4. [4]

    Long sequence time-series forecasting with deep learning: A survey

    Zonglei Chen, Minbo Ma, Tianrui Li, Hongjun Wang, and Chongshou Li. Long sequence time-series forecasting with deep learning: A survey. Information Fusion , 97:101819, 2023

    work page 2023

  5. [5]

    Flashattention: Fast and memory-efficient exact attention with io-awareness

    Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher R \'e . Flashattention: Fast and memory-efficient exact attention with io-awareness. NeurIPS , 35:16344--16359, 2022

    work page 2022

  6. [6]

    A decoder-only foundation model for time-series forecasting

    Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. A decoder-only foundation model for time-series forecasting. arXiv preprint arXiv:2310.10688 , 2023

    arXiv 2023

  7. [7]

    Depts: Deep expansion learning for periodic time series forecasting

    Wei Fan, Shun Zheng, Xiaohan Yi, Wei Cao, Yanjie Fu, Jiang Bian, and Tie-Yan Liu. Depts: Deep expansion learning for periodic time series forecasting. arXiv preprint arXiv:2203.07681 , 2022

    arXiv 2022

  8. [8]

    Deep frequency derivative learning for non-stationary time series forecasting

    Wei Fan, Kun Yi, Hangting Ye, Zhiyuan Ning, Qi Zhang, and Ning An. Deep frequency derivative learning for non-stationary time series forecasting. arXiv preprint arXiv:2407.00502 , 2024

    arXiv 2024

Show all 59 references
  1. [9]

    Moment: A family of open time-series foundation models

    Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, and Artur Dubrawski. Moment: A family of open time-series foundation models. In ICML , 2024

    2024

  2. [10]

    Mamba: Linear-time sequence modeling with selective state spaces

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752 , 2023

    2023 arXiv

  3. [11]

    Flatten transformer: Vision transformer using focused linear attention

    Dongchen Han, Xuran Pan, Yizeng Han, Shiji Song, and Gao Huang. Flatten transformer: Vision transformer using focused linear attention. In ICCV , pages 5961--5971, 2023

    2023

  4. [12]

    The capacity and robustness trade-off: Revisiting the channel independent strategy for multivariate time series forecasting

    Lu Han, Han-Jia Ye, and De-Chuan Zhan. The capacity and robustness trade-off: Revisiting the channel independent strategy for multivariate time series forecasting. IEEE Transactions on Knowledge and Data Engineering , 2024

    2024

  5. [13]

    Catn: Cross attentive tree-aware network for multivariate time series forecasting

    Hui He, Qi Zhang, Simeng Bai, Kun Yi, and Zhendong Niu. Catn: Cross attentive tree-aware network for multivariate time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 4030--4038, 2022

    2022

  6. [14]

    Matrix analysis

    Roger A Horn and Charles R Johnson. Matrix analysis . Cambridge university press, 2012

    2012

  7. [15]

    Time-llm: Time series forecasting by reprogramming large language models

    Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, et al. Time-llm: Time series forecasting by reprogramming large language models. In ICLR , 2021

    2021

  8. [16]

    Reversible instance normalization for accurate time-series forecasting against distribution shift

    Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, and Jaegul Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. In ICLR , 2021

    2021

  9. [17]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR , 2015

    2015

  10. [18]

    Reformer: The efficient transformer

    Nikita Kitaev, ukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. arXiv preprint arXiv:2001.04451 , 2020

    2001 arXiv

  11. [19]

    Predict, refine, synthesize: Self-guiding diffusion models for probabilistic time series forecasting

    Marcel Kollovieh, Abdul Fatir Ansari, Michael Bohlke-Schneider, Jasper Zschiegner, Hao Wang, and Yuyang Bernie Wang. Predict, refine, synthesize: Self-guiding diffusion models for probabilistic time series forecasting. In NeurIPS , volume 36, 2024

    2024

  12. [20]

    Modeling long-and short-term temporal patterns with deep neural networks

    Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In SIGIR , pages 95--104, 2018

    2018

  13. [21]

    Revisiting long-term time series forecasting: An investigation on linear mapping

    Zhe Li, Shiyi Qi, Yiduo Li, and Zenglin Xu. Revisiting long-term time series forecasting: An investigation on linear mapping. arXiv preprint arXiv:2305.10721 , 2023

    2023 arXiv

  14. [22]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Ching-Feng Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV , pages 10012--10022, 2021

    2021

  15. [23]

    Scinet: Time series modeling and forecasting with sample convolution and interaction

    Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, and Qiang Xu. Scinet: Time series modeling and forecasting with sample convolution and interaction. NeurIPS , 35:5816--5828, 2022

    2022

  16. [24]

    Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting

    Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In ICLR , 2022

    2022

  17. [25]

    Non-stationary transformers: Exploring the stationarity in time series forecasting

    Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting. NeurIPS , 35:9881--9893, 2022

    2022

  18. [26]

    itransformer: Inverted transformers are effective for time series forecasting

    Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. In ICLR , 2024

    2024

  19. [27]

    Timer: Transformers for time series analysis at scale

    Yong Liu, Haoran Zhang, Chenyu Li, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Timer: Transformers for time series analysis at scale. In ICML , 2024

    2024

  20. [28]

    Algorithms for discrete Fourier transform and convolution

    RTMAC Lu. Algorithms for discrete Fourier transform and convolution . Springer, 1989

    1989

  21. [29]

    Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam

    Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. ICLR , 2023

    2023

  22. [30]

    Signals and systems

    Sankaran Palani. Signals and systems . Springer, 2022

    2022

  23. [31]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems , 32, 2019

    2019

  24. [32]

    Fredformer: Frequency debiased transformer for time series forecasting

    Xihao Piao, Zheng Chen, Taichi Murayama, Yasuko Matsubara, and Yasushi Sakurai. Fredformer: Frequency debiased transformer for time series forecasting. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages 2400--2410, 2024

    2024

  25. [33]

    Deepar: Probabilistic forecasting with autoregressive recurrent networks

    David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Deepar: Probabilistic forecasting with autoregressive recurrent networks. International journal of forecasting , 36(3):1181--1191, 2020

    2020

  26. [34]

    Laser: Attention with exponential transformation

    Sai Surya Duvvuri and Inderjit S Dhillon. Laser: Attention with exponential transformation. arXiv e-prints , pages arXiv--2411, 2024

    2024

  27. [35]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. NIPS , 30, 2017

    2017

  28. [36]

    Fredf: Learning to forecast in frequency domain

    Hao Wang, Licheng Pan, Zhichao Chen, Degui Yang, Sen Zhang, Yifei Yang, Xinggao Liu, Haoxuan Li, and Dacheng Tao. Fredf: Learning to forecast in frequency domain. arXiv preprint arXiv:2402.02399 , 2024

    2024 arXiv

  29. [37]

    Zhang, and Jun Zhou

    Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y. Zhang, and Jun Zhou. Timemixer: Decomposable multiscale mixing for time series forecasting. In ICLR , 2024

    2024

  30. [38]

    Card: Channel aligned robust blend transformer for time series forecasting

    Xue Wang, Tian Zhou, Qingsong Wen, Jinyang Gao, Bolin Ding, and Rong Jin. Card: Channel aligned robust blend transformer for time series forecasting. In ICLR , 2024

    2024

  31. [39]

    Unified training of universal time series forecasting transformers

    Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Unified training of universal time series forecasting transformers. In ICML , 2024

    2024

  32. [40]

    Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting

    Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. NeurIPS , 34:22419--22430, 2021

    2021

  33. [41]

    Flowformer: Linearizing transformers with conservation flows

    Haixu Wu, Jialong Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Flowformer: Linearizing transformers with conservation flows. arXiv preprint arXiv:2202.06258 , 2022

    2022 arXiv

  34. [42]

    Timesnet: Temporal 2d-variation modeling for general time series analysis

    Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In ICLR , 2023

    2023

  35. [43]

    Interpretable weather forecasting for worldwide stations with a unified deep model

    Haixu Wu, Hang Zhou, Mingsheng Long, and Jianmin Wang. Interpretable weather forecasting for worldwide stations with a unified deep model. Nature Machine Intelligence , 5(6):602--611, 2023

    2023

  36. [44]

    o mformer: A nystr \

    Yunyang Xiong, Zhanpeng Zeng, Rudrasis Chakraborty, Mingxing Tan, Glenn Fung, Yin Li, and Vikas Singh. Nystr \"o mformer: A nystr \"o m-based algorithm for approximating self-attention. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 14138--...

    2021

  37. [45]

    Fits: Modeling time series with 10 k parameters

    Zhijian Xu, Ailing Zeng, and Qiang Xu. Fits: Modeling time series with 10 k parameters. arXiv preprint arXiv:2307.03756 , 2023

    2023 arXiv

  38. [46]

    Frequency adaptive normalization for non-stationary time series forecasting

    Weiwei Ye, Songgaojun Deng, Qiaosha Zou, and Ning Gui. Frequency adaptive normalization for non-stationary time series forecasting. arXiv preprint arXiv:2409.20371 , 2024

    2024 arXiv

  39. [47]

    A survey on deep learning based time series analysis with frequency transformation

    Kun Yi, Qi Zhang, Longbing Cao, Shoujin Wang, Guodong Long, Liang Hu, Hui He, Zhendong Niu, Wei Fan, and Hui Xiong. A survey on deep learning based time series analysis with frequency transformation. arXiv preprint arXiv:2302.02173 , 2023

    2023 arXiv

  40. [48]

    Filternet: Harnessing frequency filters for time series forecasting

    Kun Yi, Jingru Fei, Qi Zhang, Hui He, Shufeng Hao, Defu Lian, and Wei Fan. Filternet: Harnessing frequency filters for time series forecasting. arXiv preprint arXiv:2411.01623 , 2024

    2024 arXiv

  41. [49]

    Fouriergnn: Rethinking multivariate time series forecasting from a pure graph perspective

    Kun Yi, Qi Zhang, Wei Fan, Hui He, Liang Hu, Pengyang Wang, Ning An, Longbing Cao, and Zhendong Niu. Fouriergnn: Rethinking multivariate time series forecasting from a pure graph perspective. Advances in Neural Information Processing Systems , 36, 2024

    2024

  42. [50]

    Frequency-domain mlps are more effective learners in time series forecasting

    Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang, Hui He, Ning An, Defu Lian, Longbing Cao, and Zhendong Niu. Frequency-domain mlps are more effective learners in time series forecasting. Advances in Neural Information Processing Systems , 36, 2024

    2024

  43. [51]

    Revitalizing multivariate time series forecasting: Learnable decomposition with inter-series dependencies and intra-series variations modeling

    Guoqi Yu, Jing Zou, Xiaowei Hu, Angelica I Aviles-Rivero, Jing Qin, and Shujun Wang. Revitalizing multivariate time series forecasting: Learnable decomposition with inter-series dependencies and intra-series variations modeling. In ICML , 2024

    2024

  44. [52]

    Sub-adjacent transformer: Improving time series anomaly detection with reconstruction error from sub-adjacent neighborhoods

    Wenzhen Yue, Xianghua Ying, Ruohao Guo, Dongdong Chen, Yuqing Zhu, Ji Shi, Bowei Xing, and Taiyan Chen. Sub-adjacent transformer: Improving time series anomaly detection with reconstruction error from sub-adjacent neighborhoods. In IJCAI , 2024

    2024

  45. [53]

    Are transformers effective for time series forecasting? In AAAI , volume 37, pages 11121--11128, 2023

    Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In AAAI , volume 37, pages 11121--11128, 2023

    2023

  46. [54]

    Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting

    Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In ICLR , 2023

    2023

  47. [55]

    Informer: Beyond efficient transformer for long sequence time-series forecasting

    Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In AAAI , volume 35, pages 11106--11115, 2021

    2021

  48. [56]

    Film: Frequency improved legendre memory model for long-term time series forecasting

    Tian Zhou, Ziqing Ma, Qingsong Wen, Liang Sun, Tao Yao, Wotao Yin, Rong Jin, et al. Film: Frequency improved legendre memory model for long-term time series forecasting. Advances in neural information processing systems , 35:12677--12690, 2022

    2022

  49. [57]

    Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting

    Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In ICML , pages 27268--27286. PMLR, 2022

    2022

  50. [58]

    One fits all: Power general time series analysis by pretrained lm

    Tian Zhou, Peisong Niu, Xue Wang, Liang Sun, and Rong Jin. One fits all: Power general time series analysis by pretrained lm. NeurIPS , 36:43322--43355, 2023

    2023

  51. [59]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.