Pith. sign in

REVIEW 5 major objections 7 minor 46 references

FaCTR: Factorized Channel-Temporal Representation Transformers for Efficient Time Series Forecasting

T0 review · 5 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A compact Transformer that splits temporal attention from low-rank Factorization-Machine channel mixing reaches state-of-the-art forecasting accuracy on eleven benchmarks with roughly 400K parameters, about 50x fewer than spatiotemporal…

desk verdict FaCTR's FM-based channel interaction is a genuinely useful idea, but the paper overclaims state-of-the-art in its own tables and has internal inconsistencies that need a serious revision. read the letter →

arxiv 2506.05597 v1 pith:HAIMMSN6 submitted 2025-06-05 cs.LG

Yash Vijay , Harini Subramanyan This is my paper
classification cs.LG
keywords timeseriesforecastingmultivariateFactorizationMachinelow-rankapproximationcross-channelinteractionspatch-basedattentionself-supervisedpretrainingparameterefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that Transformer forecasting models are overparameterized because they conflate two dependency types that come from different generative processes: temporal autocorrelation and cross-channel coupling. To fix this, FaCTR processes each channel's history with patchwise temporal self-attention, then models inter-channel relationships with a low-rank Factorization Machine applied to covariate-enriched patch embeddings, and fuses the two pathways with a learnable gate. The paper reports that this compact architecture reaches state-of-the-art accuracy on eleven multivariate forecasting benchmarks spanning long and short horizons, with its largest variant around 400K parameters, on average 50x smaller than competing spatiotemporal Transformer baselines. If correct, the central message is that structural inductive biases can replace scale in time series forecasting, and that cross-channel influence can be read out directly from the learned Factorization Machine scores.

What carries the argument

The central mechanism is the Factorization Machine (FM) used as a low-rank cross-channel attention layer: for each patch, channel pairs receive a symmetric interaction score $\langle h_i, h_j \rangle$ computed from covariate-enriched embeddings, and the full $C\times C$ similarity matrix is approximated by a rank-$r$ factor $S_{\mathrm{FM}} = VV^{\top}$, giving $O(Cr)$ parameters instead of $O(C^2)$. A two-layer low-rank projection aligns the temporal attention output to the FM's value space, and a learned gate $G = \sigma(W_g Z_{\mathrm{temp}})$ produces a per-location convex blend of temporal and spatial features, biased toward recency. The temporal pathway is patchwise single-head self-attention per channel, and reversible instance normalization plus patched static/dynamic covariate embeddings feed the FM. Total complexity is $O(CN^2 + C^2N + NT)$, sub-quadratic in the original sequence length, and the paper invokes the Eckart-Young-Mirsky theorem to justify the rank-$r$ geometry.

What would settle it

Compute the singular-value spectrum of the empirical cross-channel similarity or mutual-information matrix for a benchmark such as Traffic, and compare the Frobenius norm captured by the top $r=8$ singular values; if it is far below the norm, the rank-8 bottleneck is too aggressive. A direct behavioral test is to train FaCTR and a full-rank channel-attention variant on a synthetic multivariate process with a dense coupling matrix (for example, a high-dimensional VAR with non-negligible off-diagonal entries): if the full-rank model's test error is substantially lower, the sparsity/low-rank premise on which FaCTR's FM relies is falsified for that data class.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that a deliberately structured Transformer can outperform, or match, substantially larger spatiotemporal Transformer models on multivariate time series forecasting. The design splits the problem into three pathways: single-head temporal self-attention over non-overlapping patches per channel, a low-rank Factorization Machine that computes per-patch pairwise channel similarity as $S_{\mathrm{FM}} = VV^{\top}$ with rank $r$, and an embedding-wise MLP that mixes the fused representation. Static and dynamic covariates are embedded into the patch representations so that cross-channel interactions are conditioned on exogenous context. The paper reports state-of-the-art or near-state-of-the-art MSE/MAE results across eleven benchmarks (ETT, Weather, Electricity, Traffic, PEMS) with under 400K parameters, and presents FM scores as interpretable cross-channel influence scores, using a synthetic retail case study to argue the model recovers promotion lags and cannibalization.

Load-bearing premise

The load-bearing premise is that the important relationships between channels are sparse and can be captured by a low-rank approximation; if real multivariate series have dense, strongly non-low-rank cross-channel couplings, the Factorization Machine will miss them and the efficiency and interpretability advantages shrink.

Editorial extensions

If this is right

  • Parameter count is not a proxy for forecasting performance: a model with under 400K parameters can be competitive with or superior to models one to two orders of magnitude larger on standard benchmarks.
  • Cross-channel dependence can be modeled with symmetric low-rank interactions, which keeps complexity sub-quadratic and opens the door to forecasting settings with hundreds of thousands of channels where dense spatiotemporal attention is infeasible.
  • FM scores yield per-patch pairwise channel influence, giving a natural readout for explainability; the synthetic retail experiment indicates these scores can track lagged promotional effects and cannibalization.
  • Self-supervised masked-patch pretraining on one dataset transfers to a related dataset under linear probing and fine-tuning, suggesting the compact architecture can serve as a reusable backbone.
  • Separating temporal and cross-channel pathways means the two inductive biases can be diagnosed and improved independently, e.g., addressing abrupt-dynamics failures on particular channels without changing the spatial module.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the FM interaction is symmetric and low-rank, the same module could be dropped into high-cardinality entity-level forecasting (retail product-store combinations), where the paper's complexity analysis suggests it would run in near $O(C^2N)$ even for very large $C$; the benchmark experiments do not directly exercise this regime.
  • The gating mechanism's dependence on temporally contextualized features biases the model toward recency; a testable consequence is that cross-channel information should matter more after regime shifts or at longer horizons, which the paper does not measure.
  • The spatial embeddings are tied to channel count and identity, so the pretraining story is limited to targets with compatible channels; a channel-agnostic spatial conditioning would be the natural next step to make the representation-learning claim more general.
  • A cheap diagnostic for when FaCTR-style models should be used is the effective rank of the empirical channel similarity matrix: if the top few singular values already explain most of its energy, low-rank channel interaction is well-matched.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes FaCTR, a compact spatiotemporal Transformer for multivariate time series forecasting. The architecture combines per-channel patch-based temporal self-attention, a Factorization Machine intended to capture low-rank cross-channel interactions, a learned gate for fusing temporal and spatial representations, covariate embeddings (static and dynamic), and optional masked-patch self-supervised pretraining. The authors evaluate on seven long-term benchmarks (ETT, Weather, Electricity, Traffic) and four short-term PEMS benchmarks, reporting MSE/MAE, parameter counts, training time/memory, ablations, a synthetic retail interpretability case study, and a transfer-learning study. The paper claims state-of-the-art performance with roughly 50× parameter reduction relative to comparable spatiotemporal baselines.

Significance. If verified, the main contributions are valuable: a parameter-efficient architecture that models cross-channel dependencies with an interpretable, structured mechanism, and a broad benchmark comparison. The paper includes a complexity analysis, ablations, a diagnostic case study, and self-supervised pretraining exploration. However, the state-of-the-art claim is not supported by the paper's own tables, the low-rank FM description is internally inconsistent, and the pretraining results are reported in a way that contradicts the corresponding table. The parameter-efficiency results and the breadth of the empirical study are the paper's strongest assets; the claims need to be recalibrated and the architecture description corrected.

major comments (5)
  1. [Abstract; Section 4.2, Table 2] The abstract claims FaCTR 'achieves state-of-the-art performance on eleven public forecasting benchmarks,' but Table 2 shows FaCTR is best in only 11 of 28 long-term MSE rows and second in 13, with PatchTST winning all four Traffic horizons and PatchTST or ModernTCN ahead on several Electricity and ETTh2 rows. Section 4.2 states this accurately, so the abstract overstates the results. The central claim should be revised to 'competitive' or 'state-of-the-art in a majority of settings,' or the comparison should be strengthened.
  2. [Section 3.2] The low-rank FM module is described inconsistently. The text first defines S_FM via inner products of channel embeddings h_i,n, then says it is 'approximate[d] with a rank-r decomposition S_FM = V V^T' with V ∈ R^{C×r}, but no V is defined or learned anywhere in the architecture. Moreover, Table 6 lists the FM similarity computation as O(BC^2ND), i.e., the explicit full C×C matrix is computed, which contradicts the claimed O(Cr) parameter saving. The authors should either specify how the low-rank factorization is actually computed in the forward pass or remove the low-rank approximation claim.
  3. [Section 4.2, Table 2; Appendix C.3.2] The main long-term comparison omits iTransformer entirely and includes TimesNet only in the appendix (Table 12). iTransformer is a standard long-term baseline and appears in the paper only in the short-term PEMS table (Table 3). Without these baselines in the main long-term comparison, the 'state-of-the-art' label is not established against the relevant literature.
  4. [Section 4.4, Table 4] The text states that 'self-supervised pretraining improves performance across all horizons' and that 'full fine-tuning further improves accuracy at shorter horizons,' but Table 4 shows the from-scratch model has lower MSE than fine-tuning at horizons 96, 192, and 336, and lower MAE at 96, 192, and 336; fine-tuning is not consistently better at any horizon. The statement should be corrected to say that pretraining is competitive but not consistently superior, or the experimental setup should be revisited.
  5. [Section 4.2] All reported results are three-run averages with no standard deviations or significance tests; differences as small as 0.001 (e.g., Electricity 96 MSE 0.130 vs 0.129) are used to determine rankings. Without variance information, the claim of being 'state-of-the-art' or 'second-best' is not statistically grounded, and the paper should report error bars or at least best-of/median across seeds.
minor comments (7)
  1. [Table 2] Column headers are mangled (e.g., 'MomentMSE MAE' and 'FedFormer' run together), and model names like 'FED FORMER' and 'CSF ORMER' appear with spaces; fix formatting.
  2. [Appendix C.1.1; Appendix C.2.4] Appendix C.1.1 cites RevIN as [38], but the correct reference is [35]; C.2.4 also refers to 'preprocessing protocol introduced in [23]' which is Autoformer, not the normalization source.
  3. [Appendix A.2] Appendix A.2 attributes an exponential-decay bound on mutual information to 'Theorem 3, [40]' (Cover & Thomas); no such theorem is stated there, and the bound is an assumption rather than a proven general property, so the citation should be removed or replaced with empirical evidence.
  4. [Section 3.2] The Eckart–Young–Mirsky argument does not apply as stated because the learned V is not the truncated SVD of the observed similarity matrix; the optimality claim should be rephrased.
  5. [Section 4.2; Appendix C.2.3] Section 4.2 says FaCTR delivers results 'with minimal tuning,' but Appendix C.2.3 reports per-dataset and per-horizon tuning of the SAM neighborhood size ρ (Table 9); please reconcile these statements.
  6. [Section 4.2] The text says FaCTR is 'second in 13 more for MSE (and 8 for MAE),' but Table 2's bottom row shows the second-place count for MAE is 9; fix the text or the table.
  7. [Reproducibility] The paper does not provide a code repository or link, which limits reproducibility of the empirical claims.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: FaCTR's benchmark claims rest on held-out test evaluation against external baselines, not on fitted inputs renamed as predictions or on author-specific self-citations.

full rationale

I examined the paper's derivation chain: the architectural claims (low-rank FM cross-channel interaction, gated fusion, patching, RevIN, projection head), the complexity derivation, the interpretability case study, and the empirical evaluation. The low-rank FM module is an architectural inductive prior; the paper explicitly states it is 'imposing a low-rank inductive bias aligned with sparse cross-channel dependencies' (Section 3.2), and no benchmark forecast is computed from the FM scores themselves. The Eckart-Young-Mirsky reference is a standard external theorem, not a self-citation, and it is used to motivate the rank-r bottleneck rather than to derive a test-set prediction. No fitted parameter is renamed as a prediction: all model parameters are learned on training/validation splits and evaluated on held-out test splits, with the SAM hyperparameter rho tuned per dataset on validation data. The reference list contains no author self-citations, and no uniqueness theorem from the authors' prior work is invoked. The interpretability case study on synthetic retail demand is a qualitative check that the learned FM scores align with the data-generating causal structure; even if the alignment is unsurprising given the construction, it is not an equation-level reduction of a claimed prediction to its fitted input. Several passages do raise correctness concerns but not circularity: Table 2 shows FaCTR is first in only 11/28 long-term MSE rows and second in 13, which undercuts the abstract's 'state-of-the-art' wording; iTransformer is absent from the main long-term table; Section 4.4's claim that 'self-supervised pretraining improves performance across all horizons' is contradicted by Table 4, where the from-scratch model has lower MSE at three of four horizons; and Appendix D.2 honestly notes that channel-tied spatial embeddings limit cross-domain transfer. These are claims-strength and evaluation-fairness issues, not reductions by construction. Therefore the circularity score is 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The architecture is built on stated but unmeasured structural priors about time series (timescale separation, structural invariance, mutual information sparsity). Hyperparameters including the SAM radius are tuned per dataset and horizon, which is standard practice but means the reported numbers are selected, not a single fixed configuration.

free parameters (5)
  • FM rank r and spatial rank = 8
    Chosen by ablations over 8/16/32/64; the central low-rank interaction capacity depends on these.
  • Hidden dimension D = 32
    Chosen by ablations; drives model size and capacity.
  • Patch length and stride = 32
    Non-overlapping patches; selected via hyperparameter search; affects temporal resolution and sequence length N.
  • SAM neighborhood size rho = 0.05 to 0.95, per dataset and horizon (Tables 9 and 10)
    Tuned separately for every dataset and prediction horizon; the reported results depend on these chosen values.
  • Masking ratio for pretraining = 45%
    Used in self-supervised pretraining; not central to the forecasting claim but part of the representation learning results.
assumptions (4)
  • domain assumption Timescale separation: cross-channel dependency decays faster than temporal autocorrelation
    Invoked in Appendix A.2 to justify modeling temporal and channel dependencies separately with different capacity.
  • domain assumption Structural invariance: inter-variable relationships are stable over time and can be captured by low-rank factors
    Invoked in Appendix A.2 to justify low-rank FM and time-invariant cross-channel representations.
  • domain assumption Mutual information sparsity: most channel pairs have negligible mutual information
    Appendix A.3 asserts this structural prior, which motivates replacing dense channel attention with FM.
  • domain assumption RevIN normalization removes enough non-stationarity that the model can treat normalized series as stationary
    Used in Section 3.1 and C.1.1; accuracy depends on this preprocessing being adequate.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FaCTR: Factorized Channel-Temporal Representation Transformers for Efficient Time Series Forecasting." pith.science (2026). https://pith.science/paper/HAIMMSN6

@misc{pith2026250605597,
  author       = {Pith},
  title        = {Pith review of: FaCTR: Factorized Channel-Temporal Representation Transformers for Efficient Time Series Forecasting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HAIMMSN6}},
  note         = {Machine review of arXiv:2506.05597}
}
read the original abstract

While Transformers excel in language and vision-where inputs are semantically rich and exhibit univariate dependency structures-their architectural complexity leads to diminishing returns in time series forecasting. Time series data is characterized by low per-timestep information density and complex dependencies across channels and covariates, requiring conditioning on structured variable interactions. To address this mismatch and overparameterization, we propose FaCTR, a lightweight spatiotemporal Transformer with an explicitly structural design. FaCTR injects dynamic, symmetric cross-channel interactions-modeled via a low-rank Factorization Machine into temporally contextualized patch embeddings through a learnable gating mechanism. It further encodes static and dynamic covariates for multivariate conditioning. Despite its compact design, FaCTR achieves state-of-the-art performance on eleven public forecasting benchmarks spanning both short-term and long-term horizons, with its largest variant using close to only 400K parameters-on average 50x smaller than competitive spatiotemporal transformer baselines. In addition, its structured design enables interpretability through cross-channel influence scores-an essential requirement for real-world decision-making. Finally, FaCTR supports self-supervised pretraining, positioning it as a compact yet versatile foundation for downstream time series tasks.

Figures

Figures reproduced from arXiv: 2506.05597 by the authors.

Figure 1
Figure 1. FaCTR architecture: A disentangled spatiotemporal model where temporal attention, low [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Forecasts of sequence 240 [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 4
Figure 4. Forecast vs. actual plot for channel 0 in ETTh1 for prediction horizon 96 [PITH_FULL_IMAGE:figures/full_fig_p020_4.png] view at source ↗
Figures from the paper (13 more)
Figure 5
Figure 5. Figure 5: Forecast vs. actual plot for channel 6 in ETTh2 for prediction horizon 96 [PITH_FULL_IMAGE:figures/full_fig_p021_5.png]
Figure 6
Figure 6. Figure 6: Forecast vs. actual plot for channel 4 in ETTh1 for prediction horizon 336 [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 7
Figure 7. Figure 7: Forecast vs. actual plot for channel 16 in Weather for prediction horizon 192 [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]
Figure 8
Figure 8. Figure 8: Forecast vs. actual plot for channel 2 in Weather for prediction horizon 336 [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]
Figure 9
Figure 9. Figure 9: Forecast vs. actual plot for channel 28 in Electricity for prediction horizon 96 [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]
Figure 10
Figure 10. Figure 10: Forecast vs. actual plot for channel 310 in Electricity for prediction horizon 96 [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]
Figure 11
Figure 11. Figure 11: Forecast vs. actual plot for channel 3 in Traffic for prediction horizon 192 [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]
Figure 12
Figure 12. Figure 12: Forecast vs. actual plot for channel 86 in Traffic for prediction horizon 192 [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: Examples from channels with distinct training-time patterns influencing forecast behavior. [PITH_FULL_IMAGE:figures/full_fig_p023_13.png]
Figure 14
Figure 14. Figure 14: Examples from channels with different patterns from training-time. The model misses [PITH_FULL_IMAGE:figures/full_fig_p024_14.png]
Figure 15
Figure 15. Figure 15: Examples from channels with distinct training-time patterns influencing forecast behavior. [PITH_FULL_IMAGE:figures/full_fig_p024_15.png]
Figure 16
Figure 16. Figure 16: Simulated time series patterns for each channel. Each channel exhibits a distinct signal [PITH_FULL_IMAGE:figures/full_fig_p027_16.png]
Figure 17
Figure 17. Figure 17: Actuals vs Forecasts across channels The model demonstrates a strong ability to capture a range of temporal and cross-channel behaviors across the eight simulated demand signals. For C1 and C2, which follow stable sinusoidal patterns, the model generalizes across phas…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

46 extracted references · 43 canonical work pages

  1. [1]

    Tao Hong, Pierre Pinson, Shu Fan, Hamidreza Zareipour, Alberto Troccoli, Rob J. Hyndman. Probabilistic energy forecasting: Global energy forecasting competition 2014 and beyond. International Journal of Forecasting, 37(4):1572–1587, 2016

    work page 2014

  2. [2]

    Recurrent neural networks for multivariate time series with missing values

    Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, Yan Liu. Recurrent neural networks for multivariate time series with missing values. Scientific Reports, 8(1):6085, 2016

    work page 2016

  3. [3]

    The M4 Competition: 100,000 time series and 61 methods

    Spyros Makridakis, Evangelos Spiliotis, Vassilios Assimakopoulos. The M4 Competition: 100,000 time series and 61 methods. International Journal of Forecasting, 36(1):54–74, 2020

    work page 2020

  4. [4]

    George E. P. Box, Gwilym M. Jenkins, Gregory C. Reinsel, Greta M. Ljung. Time Series Analysis: Forecasting and Control Wiley., 1976

    work page 1976

  5. [5]

    Efficient Estimation of Word Representations in Vector Space

    Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean. Efficient Estimation of Word Representations in Vector Space. International Conference on Learning Representations, 2013

    work page 2013

  6. [6]

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.International Conference on Learning Representations (ICLR), 2021

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.International Conference on Learning Representations (ICLR), 2021

    work page 2021

  7. [7]

    Are transformers effective for time series forecasting? AAAI Conference on Artificial Intelligence, 2023

    Ailing Zeng, Muxi Chen, Lei Zhang, Qiang Xu. Are transformers effective for time series forecasting? AAAI Conference on Artificial Intelligence, 2023

    work page 2023

  8. [8]

    SAMformer: Unlocking the potential of transformers in time series forecasting with sharpness-aware minimization and channel-wise attention

    Romain Ilbert, Ambroise Odonnat, Vasilii Feofanov, Aladin Virmaux, Giuseppe Paolo, Themis Palpanas, Ievgen Redko. SAMformer: Unlocking the potential of transformers in time series forecasting with sharpness-aware minimization and channel-wise attention. International Conference on Machine Learning (ICML), 2024

    work page 2024

Show all 46 references
  1. [9]

    ST-ReP: Learning predictive representations efficiently for spatial- temporal forecasting

    Qi Zheng, Zihao Yao, Yaying Zhang. ST-ReP: Learning predictive representations efficiently for spatial- temporal forecasting. Association for the Advancement of Artificial Intelligence, 2024

    2024

  2. [10]

    Long-term forecasting with TiDE: Time-series dense encoder.Transactions on Machine Learning Research, 2023

    Abhimanyu Das, Weihao Kong, Andrew Leach, Shaan Mathur, Rajat Sen, Rose Yu. Long-term forecasting with TiDE: Time-series dense encoder.Transactions on Machine Learning Research, 2023

    2023

  3. [11]

    Nguyen, Phanwadee Sinthong, Jayant Kalagnanam

    Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. International Conference on Learning Representations (ICLR), 2023

    2023

  4. [12]

    Long-Range Transformers for Dynamic Spatiotemporal Forecasting

    Jake Grigsby, Zhe Wang, Nam Nguyen, Yanjun Qi. Long-Range Transformers for Dynamic Spatiotemporal Forecasting. arXiv preprint arXiv:2106.13705, 2021

    2021 arXiv

  5. [13]

    ModernTCN: A Modern Pure Convolution Structure for General Time Series Analysis

    Luo donghao, wang xue. ModernTCN: A Modern Pure Convolution Structure for General Time Series Analysis. International Conference on Learning Representations (ICLR), 2024

    2024

  6. [14]

    Arik, Tomas Pfister

    Si-An Chen, Chun-Liang Li, Nate Yoder, Sercan O. Arik, Tomas Pfister. TSMixer: An All-MLP Architec- ture for Time Series Forecasting. Transactions on Machine Learning Research, 2023

    2023

  7. [15]

    Informer: Beyond efficient transformer for long sequence time-series forecasting

    Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. AAAI Conference on Artificial Intelligence, pages 11106–11115, 2021

    2021

  8. [16]

    FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting

    Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, Rong Jin. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. International Conference on Machine Learning, 2022

    2022

  9. [17]

    Crossformer: Transformer Utilizing Cross-Dimension Dependency for Multi- variate Time Series Forecasting

    Yunhao Zhang, Junchi Yan. Crossformer: Transformer Utilizing Cross-Dimension Dependency for Multi- variate Time Series Forecasting. International Conference on Learning Representations, 2023

    2023

  10. [18]

    STAEformer: Spatio-temporal adaptive embedding makes vanilla transformer SOTA for traffic forecasting

    Hangchen Liu, Zheng Dong, Renhe Jiang, Jiewen Deng, Jinliang Deng, Quanjun Chen, Xuan Song. STAEformer: Spatio-temporal adaptive embedding makes vanilla transformer SOTA for traffic forecasting. Proceedings of the 32nd ACM International Conference on Information and Knowledge ...

    2023

  11. [19]

    Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks

    Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, Chengqi Zhang. Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks. Proceedings of the 26th ACM SIGKDD, pages 717–726., 2020

    2020

  12. [20]

    CrossGNN: Confronting Noisy Multivariate Time Series Via Cross Interaction Refinement37th Conference on Neural Information Processing Systems, 2021

    Qihe Huang, Lei Shen, Ruixin Zhang, Shouhong Ding, Binwu Wang, Zhengyang Zhou, Yang Wang. CrossGNN: Confronting Noisy Multivariate Time Series Via Cross Interaction Refinement37th Conference on Neural Information Processing Systems, 2021

    2021

  13. [21]

    Adversarial sparse trans- former for time series forecasting

    Sifan Wu, Xi Xiao, Qianggang Ding, Peilin Zhao, Ying WEI, Junzhou Huang. Adversarial sparse trans- former for time series forecasting. Advances in Neural Information Processing Systems, 33:1714–1724., 2020

    2020

  14. [22]

    Factorization machines

    Rendle, S. Factorization machines. 2010 IEEE International Conference on Data Mining

    2010

  15. [23]

    Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting

    Haixu Wu, Jiehui Xu, Jianmin Wang, Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. 35th Conference on Neural Information Processing Systems, 2021

    2021

  16. [24]

    Liu, Schahram Dustdar

    Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X. Liu, Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling. International Conference on Learning Representations (ICLR), 2022

    2022

  17. [25]

    TimesNet: Temporal 2D- variation modeling for general time series analysis

    Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, Mingsheng Long. TimesNet: Temporal 2D- variation modeling for general time series analysis. International Conference on Learning Representations (ICLR), 2023

    2023

  18. [26]

    DeformTime: Capturing variable dependencies with deformable attention for time series forecasting

    Yuxuan Shu, Vasileios Lampos. DeformTime: Capturing variable dependencies with deformable attention for time series forecasting. Transactions on Machine Learning Research, 2025

    2025

  19. [27]

    Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structures,arXiv preprint arXiv:2207.01186, 2022

    Tianping Zhang, Yizhuo Zhang, Wei Cao, Jiang Bian, Xiaohan Yi, Shun Zheng, Jian Li. Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structures,arXiv preprint arXiv:2207.01186, 2022

    2022 arXiv

  20. [28]

    Li, Madian Khabsa, Han Fang, Hao Ma

    Sinong Wang, Belinda Z. Li, Madian Khabsa, Han Fang, Hao Ma. Linformer: Self-attention with linear complexity. arXiv preprint arXiv:2006.04768, 2020

    2006 arXiv

  21. [29]

    Rethinking attention with performers

    Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, et al. Rethinking attention with performers. International Conference on Learning Representations, 2021

    2021

  22. [30]

    Flowformer: Linearizing transformers with conservation flows

    Haixu Wu, Jialong Wu, Jiehui Xu, Jianmin Wang, Mingsheng Long. Flowformer: Linearizing transformers with conservation flows. International Conference on Machine Learning (ICML), 2022

    2022

  23. [31]

    DeepFM: A factorization-machine based neural network for CTR prediction

    Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, Xiuqiang He. DeepFM: A factorization-machine based neural network for CTR prediction. Proceedings of the 26th IJCAI, 2017

    2017

  24. [32]

    xDeepFM: Combining explicit and implicit feature interactions for recommender systems

    Jianxun Lian, Xiaohuan Zhou, Fuzheng Zhang, Zhongxia Chen, Xing Xie, Guangzhong Sun. xDeepFM: Combining explicit and implicit feature interactions for recommender systems. Proceedings of KDD, 2018

    2018

  25. [33]

    Attentional factorization machines: Learning the weight of feature interactions via attention networks

    Jun Xiao, Hao Ye, Xiangnan He, Hanwang Zhang, Fei Wu, Tat-Seng Chua. Attentional factorization machines: Learning the weight of feature interactions via attention networks. Proceedings of IJCAI, 2017

    2017

  26. [34]

    Deep & cross network for ad click predictions

    Ruoxi Wang, Bin Fu, Gang Fu, Mingliang Wang. Deep & cross network for ad click predictions. Proceed- ings of the ADKDD Workshop, 2017

    2017

  27. [35]

    Reversible Instance Normalization for Accurate Time-Series Forecasting

    Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, Jaegul Choo. Reversible Instance Normalization for Accurate Time-Series Forecasting. International Conference on Learning Representa- tions, 2022

    2022

  28. [36]

    Gomez, Lukasz Kaiser, Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin. Attention is All You Need.Advances in Neural Information Processing Systems (NeurIPS), pages 5998–6008, 2017

    2017

  29. [37]

    CSformer: Combining Channel Independence and Mixing for Robust Multivariate Time Series Forecasting

    Haoxin Wang, Yipeng Mo, Kunlan Xiang, Nan Yin, Honghe Dai, Bixiong Li, Songhai Fan, Site Mo. CSformer: Combining Channel Independence and Mixing for Robust Multivariate Time Series Forecasting. Association for the Advancement of Artificial Intelligence, 2025

    2025

  30. [38]

    Moment: A family of open time-series foundation models

    Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, Artur Dubrawski. Moment: A family of open time-series foundation models. Proceedings of the 41st International Conference on Machine Learning (ICML), 2024. 11

    2024

  31. [39]

    Le, Ruslan Salakhutdinov

    Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V . Le, Ruslan Salakhutdinov. Transformer- XL: Attentive language models beyond a fixed-length context. arXiv preprint arXiv:1901.02860, 2019

    1901 arXiv

  32. [40]

    Thomas and Thomas M

    Joy A. Thomas and Thomas M. Cover. Elements of Information Theory. Wiley, 2nd edition, 1981

    1981

  33. [41]

    Forecasting: Principles and Practice

    Rob J Hyndman and George Athanasopoulos. Forecasting: Principles and Practice. OTexts, 3rd edition, 2021

    2021

  34. [42]

    C.W.J. Granger. Some recent development in a concept of causality. Journal of Econometrics , 39(1–2):199–211, 1988

    1988

  35. [43]

    Hanssens , Leonard J

    Dominique M. Hanssens , Leonard J. Parsons , Randall L. Schultz. Market Response Models: Econometric and Time Series Analysis. Springer, 2nd edition, 2015

    2015

  36. [44]

    iTransformer: Inverted Transformers Are Effective for Time Series Forecasting,International Conference on Learning Representations (ICLR), 2024

    Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, Mingsheng Long. iTransformer: Inverted Transformers Are Effective for Time Series Forecasting,International Conference on Learning Representations (ICLR), 2024

    2024

  37. [45]

    SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interaction, 36th Conference on Neural Information Processing Systems, 2022

    Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, Qiang Xu. SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interaction, 36th Conference on Neural Information Processing Systems, 2022. 12 A Proofs A.1 Breakdown of Univariate Depend...

    2022

  38. [2016]

    The PEMS dataset consists of traffic data in California that was introduced in [45]

    Weather includes 21 meteorological variables recorded throughout 2020. The PEMS dataset consists of traffic data in California that was introduced in [45]. For long-term forecasting datasets, the forecasting horizons are set to 96, 192, 336, and 720 time steps. For short-term ...

    2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.