Pith. sign in

REVIEW 5 major objections 6 minor 56 references

Towards Lightweight Time Series Forecasting: a Patch-wise Transformer with Weak Data Enriching

T0 review · 5 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LiPFormer claims that a patch-wise Transformer can beat state-of-the-art forecasters after removing Layer Normalization and Feed Forward Networks, with contrastively trained weak future-context labels adding accuracy at low cost.

desk verdict Genuinely novel lightweight design with a plug-in covariate module, but the accuracy claim is oversold and the inference-time numbers contradict the abstract. read the letter →

arxiv 2501.10448 v1 pith:4YXKZH4Q submitted 2025-01-14 cs.LG cs.AI

Meng Wang , Jintao Yang , Bin Yang , Hui Li , Tongxin Gong , Bo Yang , Jiangtao Cui This is my paper · ORCID
classification cs.LGcs.AI
keywords timeseriesforecastingpatch-wiseTransformerlightweightarchitectureweakdataenrichingcontrastivelearningfuturecovariatesedgedeploymentlong-term
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that accurate long-horizon time series forecasting does not need a heavyweight Transformer. It proposes LiPFormer, a patch-wise Transformer that removes Layer Normalization and Feed Forward Networks, drops positional encoding, and replaces them with a cross-patch attention mechanism plus two single-layer MLPs. A separate weak-data-enriching module pre-trains a pair of encoders with contrastive learning, aligning cheap future context (weather forecasts, holidays, time of day) with the future sequence to be predicted. On nine benchmark datasets the paper reports higher accuracy than current methods while using far fewer parameters, less training time, and less GPU memory, and on a CPU-only edge device it reports about one-third of the inference time of a classic Transformer. The underlying claim is that a lean architecture plus weak future supervision can outperform heavier models, which would make on-device forecasting practical.

What carries the argument

Cross-patch attention is the core mechanism: for a patch length $pl$, it builds $pl$ trend sequences by taking the value at a fixed position from each patch and ordering those values chronologically, then runs self-attention over these sequences together with inter-patch attention over patch embeddings. Because trend sequences span the whole input, they carry global order and trend information, which is what allows the model to drop positional encoding and Layer Normalization. The second load-bearing mechanism is the weak-data-enriching module, a dual-encoder contrastive framework in which one encoder embeds future covariates (numerical and textual, either explicit like weather forecasts or implicit like hour of day) and the other embeds the ground-truth target sequence; a symmetric cross-entropy loss over the $b \times b$ pairwise similarity matrix aligns them. At prediction time the frozen covariate representation is mapped by a learnable linear layer and added to the base predictor output, so the forecast is a weighted combination of signal learned from history and signal learned from cheap future context.

What would settle it

Run the same add-back ablation on the seven datasets where it was not shown, especially the high-channel Electricity and Traffic sets: if adding Layer Normalization, Feed Forward Networks, or both does not degrade accuracy there, the claim that these components are ineffective for time series would not hold.

Watch

Extended reading notes

Core claim

The central claim is that two heavy components of the Transformer, Layer Normalization and Feed Forward Networks, are not effective for numerical time series and can be removed without sacrificing accuracy. LiPFormer keeps a patch-wise multi-head self-attention backbone, but introduces cross-patch attention, which picks out the data point at a fixed position in every patch and arranges those points into global trend sequences; this captures order and trend information that previously required positional encoding. It also uses inter-patch attention over patch embeddings, and replaces FFNs with two linear single-layer MLPs, reducing parameter scale from $\mathcal{O}(8 \times hd^2)$ to $\mathcal{O}(hd \times pl)$. The weak-data-enriching module then pre-trains a covariate encoder and a target encoder with a symmetric contrastive loss over a $b \times b$ similarity matrix, so that explicit or implicit future covariates become aligned with the target sequence; at inference the frozen covariate embedding is linearly mapped and added to the base prediction. The paper reports that this architecture reaches the best or second-best result in 64 of 72 multivariate settings and 26 of 32 univariate settings, while cutting training and inference time by roughly half relative to a leading patch-wise baseline and running within CPU-memory limits where a classic Transformer fails.

Load-bearing premise

The design depends on the claim that Layer Normalization and Feed Forward Networks are not effective for numerical time series, yet the ablation that supports this claim was run on only two datasets (ETTh1 and ETTm2), leaving the other seven benchmarks untested.

Editorial extensions

If this is right

  • On a CPU-only edge device, LiPFormer's inference time is reported at about one-third that of a classic Transformer, and it stays within memory where the standard model runs out at longer input lengths.
  • Removing Layer Normalization and Feed Forward Networks can improve accuracy on numerical series: adding them back, individually or together, degrades reported MSE and MAE in the ablations.
  • Weak future context can be synthesized: when no explicit covariates exist, encoding time-of-day, weekday, and month via the pretrained dual encoders improves forecasts on the ETT datasets.
  • The weak-data-enriching module is transportable: attaching it to other Transformer-based forecasters improved their MSE and MAE on the dataset with explicit future covariates.
  • Longer input histories continue to help, indicating that the cross-patch and inter-patch attention design captures long-range dependencies rather than relying on a fixed window.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • This reader would want to test the same add-back ablation on the high-channel datasets (Electricity and Traffic), since the paper itself notes that LiPFormer's accuracy margin narrows there; if LayerNorm or FFNs help on those data, the architecture's core premise is domain-dependent rather than universal.
  • A natural extension is to treat weak-data enriching as a general pretraining strategy for other sequence models, not just Transformers; the dual encoder only needs a covariate embedding and a target embedding of matching dimension, so RNN- or MLP-based forecasters could benefit too.
  • The edge-deployment numbers compare LiPFormer with a classic Transformer on CPU; a fairer test against other lightweight patch-wise forecasters on the same device would clarify how much of the speed gain comes from removing LN/FFN versus from the patch design itself.
  • One could isolate the value of weak labels by corrupting future covariates (for example, shuffling weather forecasts across days) and measuring how much accuracy drops, which would separate genuine apriori context from the pretraining objective alone.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. LiPFormer is a patch-wise Transformer for multivariate long-term time series forecasting that removes Layer Normalization, Feed-Forward Networks, and positional encoding from the backbone, replacing them with two lightweight attention mechanisms (cross-patch and inter-patch) and single-layer MLPs. The paper also proposes a weak-data-enriching module in which future covariates are contrastively aligned with target sequences during pretraining, and the frozen covariate encoder is then used to guide predictions. Experiments on nine datasets report accuracy and efficiency comparisons against iTransformer, TimeMixer, FGNN, PatchTST, DLinear, and TiDE, including CPU-only edge-device inference measurements. The paper claims that LiPFormer significantly outperforms state-of-the-art methods in accuracy while substantially reducing parameter count, training time, and GPU memory.

Significance. If the accuracy and efficiency claims hold, LiPFormer would be a useful contribution to lightweight on-device time series forecasting, and the weak-data-enriching idea could transfer to other Transformer backbones. The paper has several strengths: it evaluates on nine datasets, reports training/inference time, MACs, and parameter counts, includes a real CPU-only deployment experiment, and demonstrates that the covariate encoder can be plugged into other models (Table XII). However, the load-bearing evidence is currently incomplete: the core architectural deletions are supported only by a two-dataset, non-retuned ablation; several reported numbers are internally inconsistent; and no error bars or multi-seed statistics are provided for the accuracy claims.

major comments (5)
  1. [IV-E4, Table X] The claim that removing Layer Normalization and Feed-Forward Networks is beneficial for time series forecasting is not established by the presented evidence. The ablation is run on only ETTh1 and ETTm2, and within those two datasets the result is not consistent: on ETTm2 at L=96 adding LN improves MSE (0.158 vs. 0.160), and at L=336 adding LN also improves MSE (0.266 vs. 0.273). The variants are also not retuned: hyperparameters such as dropout=0.5 and depth were selected for the LN/FFN-free model, so the added modules are at an optimization disadvantage. Since the lightweight architecture is the central contribution, this ablation needs to be extended to more datasets with retuned variants and repeated runs.
  2. [Section I vs. Table VII and Abstract] The inference-time claim is internally inconsistent. Section I states that deployment on a CPU-only edge device shows LiPFormer inference time is 'less than 1%' of Transformer models, while the abstract says 1/3. Table VII shows that on ETTh1 with input length 96, LiPFormer takes 0.55s versus 1.47s for Transformer, i.e., about 37%, not less than 1%; on Weather at input length 96 the ratio is about 47%. The later text in Section IV-E1 also says 'nearly a tenfold increase' for ETTh1 input length 336 (Table VII gives 0.62/5.82, about 10.7%), which is different from both previous statements. The efficiency claims need to be restated consistently with the actual table.
  3. [IV-E2, Table VIII vs. Table III] Tables III and VIII contain conflicting results for the default configuration. Table VIII reports that pl=48 on ETTm1 at forecast length 720 gives MSE 0.526, while Table III reports LiPFormer with the same default setting on ETTm1 at L=720 gives MSE 0.408. Moreover, in Table VIII pl=48 is not the best patch length on ETTm1 at L=720 (pl=6 gives 0.417, pl=12 gives 0.414, pl=24 gives 0.507), which contradicts the statement in Section IV-E2 that fixed patch length does not lead to performance loss and that pl=48 is generally suitable. These numbers must be reconciled, and the recommendation of pl=48 needs to be justified on all reported datasets.
  4. [IV-A2 and Table III] No error bars or multi-seed statistics are provided. Section IV-A2 says every experiment uses the same random seed and reports mean scores, but with one seed there is no meaningful mean or variance. Many accuracy differences against the strongest baselines are small (e.g., on Electricity at L=96, LiPFormer MSE 0.131 vs. PatchTST 0.129; on ETTm1 at L=96, 0.296 vs. 0.290). Given the abstract's claim of 'significantly outperforms,' the paper should report results over multiple seeds with standard deviations and, where appropriate, a significance test, especially for the differences that drive the headline conclusion.
  5. [III-B and Section IV-C] The pretraining procedure needs a clearer statement about data boundaries. The contrastive pretraining in Section III-B uses target sequences, i.e., future ground-truth values, to train the Target Encoder, and at inference only the frozen Covariate Encoder is used. The paper should state explicitly that the pretraining and the selection of the frozen covariate encoder use only the training (and possibly validation) splits, and never the test-period future values, to rule out leakage into the reported test metrics. Currently the manuscript does not specify which temporal split is used for the contrastive pretraining.
minor comments (6)
  1. [IV-A2] The sentence 'Every experiment is executed with the same random seed and the mean scores are reported' is self-contradictory; please clarify what the mean is taken over.
  2. [Abstract] The phrase 'only takes only 1/3 inference time' contains a duplicated 'only'; please also align the fraction with Table VII.
  3. [IV-E2] The sentence 'Fixed patch length does not lead to its performance loss on different datasets' is not supported by Table VIII, especially for ETTm1 at L=720; please rephrase after reconciling the numbers.
  4. [IV-E6, Figure 6] The text says that removing the covariate encoder makes MSE 'decrease by 34%' and MAE 'decrease by 17%'; since removal harms performance, the direction should be 'increase' or 'degrade by,' and the numbers should be checked against the figure.
  5. [IV-E4] There is a typo: 'the we constructed four model model variants' should read 'we constructed four model variants.'
  6. [II, References [33] and [47]] Both TSMixer and MLP-Mixer appear to be cited with the same reference number [33], and TSMixer is also cited as [47]; please disambiguate these references.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the architecture's components are justified by external benchmarks and the paper's own ablation studies, and the contrastive pretraining does not leak target values into inference inputs.

full rationale

Walking the claimed derivation chain, the paper contains no equation or fitted parameter that reduces to its own inputs by construction. The prediction is defined as Yhat = Yhat_base + MLP(F^cf_PreTrain) (Eq. 8), where the covariate encoder at inference consumes only future covariates, not future ground-truth values; the target encoder is used only during pretraining to align representations, which is standard supervised pretraining rather than circularity. The elimination of Layer Normalization, Feed Forward Networks, and Positional Encoding is justified by external citations (e.g., DLinear, PatchTST) and by the paper's own ablation studies in Section IV-E4; even if that ablation evidence is limited to two datasets and variants are not retuned, this is a weakness in empirical support, not a circular derivation. The paper cites prior works by its own authors, but only as background references; none of those citations is load-bearing for the central claim. The efficiency and accuracy claims are assessed against external baselines on benchmark datasets. Therefore no significant circularity is present.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests mainly on hand-chosen hyperparameters and domain assumptions about time series structure. No new physical entities are introduced; the model consists of standard neural network layers. The most fragile axiom is the claimed ineffectiveness of LN and FFN, which is supported by only two datasets.

free parameters (5)
  • patch length pl = 48
    Chosen by hand; Table VIII shows sensitivity across values 6, 12, 24, and 48. The authors recommend 48, indicating the result depends on this choice.
  • hidden dimension hd = 512
    Set by default in Section IV.A.2; affects capacity and efficiency.
  • dropout = 0.5
    Set by default; used to avoid overfitting, especially on ETT datasets.
  • temperature et = not stated
    Appears in logits = (VT x VC) . et in Section III.B; scaling factor for contrastive loss, value not reported.
  • smooth L1 beta = not stated
    Hyperparameter in L1Smooth loss (Section III.B) controlling error threshold; not reported.
assumptions (4)
  • domain assumption Future value changes are highly correlated with future apriori contexts such as weather forecasts, date, and time of day.
    Stated in Section I (Challenge 2); this motivates weak data enriching. If false, the covariate module cannot help.
  • domain assumption Patching preserves local order information and cross-patch attention captures global order without positional encoding.
    Argued in Section III.C.1; the model removes Positional Encoding based on this assumption.
  • ad hoc to paper Layer Normalization and Feed Forward Networks are unnecessary or harmful for time series forecasting.
    Justified by ablations on only ETTh1 and ETTm2 (Section IV.E.4); used to justify the lightweight backbone.
  • domain assumption Channel independence (each variate processed separately) is beneficial.
    Borrowed from PatchTST and DLinear (Section III.C.1); used without new evidence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Lightweight Time Series Forecasting: a Patch-wise Transformer with Weak Data Enriching." pith.science (2026). https://pith.science/paper/4YXKZH4Q

@misc{pith2026250110448,
  author       = {Pith},
  title        = {Pith review of: Towards Lightweight Time Series Forecasting: a Patch-wise Transformer with Weak Data Enriching},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4YXKZH4Q}},
  note         = {Machine review of arXiv:2501.10448}
}
read the original abstract

Patch-wise Transformer based time series forecasting achieves superior accuracy. However, this superiority relies heavily on intricate model design with massive parameters, rendering both training and inference expensive, thus preventing their deployments on edge devices with limited resources and low latency requirements. In addition, existing methods often work in an autoregressive manner, which take into account only historical values, but ignore valuable, easy-to-obtain context information, such as weather forecasts, date and time of day. To contend with the two limitations, we propose LiPFormer, a novel Lightweight Patch-wise Transformer with weak data enriching. First, to simplify the Transformer backbone, LiPFormer employs a novel lightweight cross-patch attention and a linear transformation-based attention to eliminate Layer Normalization and Feed Forward Network, two heavy components in existing Transformers. Second, we propose a lightweight, weak data enriching module to provide additional, valuable weak supervision to the training. It enhances forecasting accuracy without significantly increasing model complexity as it does not involve expensive, human-labeling but using easily accessible context information. This facilitates the weak data enriching to plug-and-play on existing models. Extensive experiments on nine benchmark time series datasets demonstrate that LiPFormer outperforms state-of-the-art methods in accuracy, while significantly reducing parameter scale, training duration, and GPU memory usage. Deployment on an edge device reveals that LiPFormer takes only 1/3 inference time compared to classic Transformers. In addition, we demonstrate that the weak data enriching can integrate seamlessly into various Transformer based models to enhance their accuracy, suggesting its generality.

Figures

Figures reproduced from arXiv: 2501.10448 by the authors.

Figure 2
Figure 2. The construction of trend sequences and Cross [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 5
Figure 5. The structure of Covariate Encoder, which uses Res-attention and linear layers to model numerical and textual weak label super￾vision. the cosine similarity of the b diagonal positive pairs while minimizing the embedding of the remaining b 2 − b negative pairs. We use symmetric cross-entropy loss Lsce to optimize the cosine similarity score as follows: Lsce = 1 2 (Lce(logits, labels) (0) + Lce(logits, labels))(1)), … view at source ↗
Figure 6
Figure 6. The impact of incorporating or excluding the future Covariate Encoder [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figures from the paper (1 more)
Figure 7
Figure 7. Figure 7: Visualization of the logits matrices for Weakly Supervised Architec [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 48 canonical work pages

  1. [1]

    Multi-horizon in- flation forecasts using disaggregated data,

    C. Capistr ´an, C. Constandse, and M. Ramos-Francia, “Multi-horizon in- flation forecasts using disaggregated data,” Economic Modelling, vol. 27, no. 3, pp. 666–677, 2010

    work page 2010

  2. [2]

    Multivariate time series dataset for space weather data analytics,

    R. A. Angryk, P. C. Martens, B. Aydin, D. Kempton, S. S. Mahajan, S. Basodi, A. Ahmadzadeh, X. Cai, S. Filali Boubrahimi, S. M. Hamdi et al., “Multivariate time series dataset for space weather data analytics,” Scientific data, vol. 7, no. 1, p. 227, 2020

    work page 2020

  3. [3]

    Electric load forecasting using an artificial neural network,

    D. C. Park, M. El-Sharkawi, R. Marks, L. Atlas, and M. Damborg, “Electric load forecasting using an artificial neural network,” IEEE transactions on Power Systems , vol. 6, no. 2, pp. 442–449, 1991

    work page 1991

  4. [4]

    Context-aware path ranking in road networks,

    S. B. Yang, C. Guo, and B. Yang, “Context-aware path ranking in road networks,” IEEE Trans. Knowl. Data Eng. , vol. 34, no. 7, pp. 3153– 3168, 2022

    work page 2022

  5. [5]

    Ultraman: A unified platform for big trajectory data management and analytics,

    X. Ding, L. Chen, Y . Gao, C. S. Jensen, and H. Bao, “Ultraman: A unified platform for big trajectory data management and analytics,” Proceedings of the VLDB Endowment, vol. 11, no. 7, pp. 787–799, 2018

    work page 2018

  6. [6]

    Real-time distributed co-movement pattern detection on streaming trajectories,

    L. Chen, Y . Gao, Z. Fang, X. Miao, C. S. Jensen, and C. Guo, “Real-time distributed co-movement pattern detection on streaming trajectories,” Proceedings of the VLDB Endowment , vol. 12, no. 10, pp. 1208–1220, 2019

    work page 2019

  7. [7]

    Freeway performance measurement system: mining loop detector data,

    C. Chen, K. Petty, A. Skabardonis, P. Varaiya, and Z. Jia, “Freeway performance measurement system: mining loop detector data,” Trans- portation research record, vol. 1748, no. 1, pp. 96–102, 2001

    work page 2001

  8. [8]

    Mdtp: A multi-source deep traffic prediction framework over spatio-temporal trajectory data,

    Z. Fang, L. Pan, L. Chen, Y . Du, and Y . Gao, “Mdtp: A multi-source deep traffic prediction framework over spatio-temporal trajectory data,” VLDB, vol. 14, no. 8, pp. 1289–1297, 2021

    work page 2021

Show all 56 references
  1. [9]

    Unsupervised path representation learning with curriculum negative sampling,

    S. B. Yang, C. Guo, J. Hu, J. Tang, and B. Yang, “Unsupervised path representation learning with curriculum negative sampling,” in IJCAI, 2021, pp. 3286–3292

    2021

  2. [10]

    Team: Topological evolution-aware framework for traffic forecasting,

    D. Kieu, T. Kieu, P. Han, B. Yang, C. S. Jensen, and B. Le, “Team: Topological evolution-aware framework for traffic forecasting,” Proc. VLDB Endow., vol. 18, 2024

    2024

  3. [11]

    Multivariate time- series modeling for forecasting sintering temperature in rotary kilns using dcgnet,

    X. Zhang, Y . Lei, H. Chen, L. Zhang, and Y . Zhou, “Multivariate time- series modeling for forecasting sintering temperature in rotary kilns using dcgnet,” IEEE Trans. Ind. Informatics , vol. 17, no. 7, pp. 4635– 4645, 2021

    2021

  4. [12]

    An edge-ai based forecasting approach for improving smart microgrid efficiency,

    L. Lv, Z. Wu, L. Zhang, B. B. Gupta, and Z. Tian, “An edge-ai based forecasting approach for improving smart microgrid efficiency,” IEEE Trans. Ind. Informatics , vol. 18, no. 11, pp. 7946–7954, 2022

    2022

  5. [13]

    Weakly guided adaptation for robust time series forecasting,

    Y . Cheng, P. Chen, C. Guo, K. Zhao, Q. Wen, B. Yang, and C. S. Jensen, “Weakly guided adaptation for robust time series forecasting,” Proc. VLDB Endow., vol. 17, no. 4, pp. 766–779, 2023

    2023

  6. [14]

    AutoCTS+: Joint neural architecture and hyperparameter search for correlated time series forecasting,

    X. Wu, D. Zhang, M. Zhang, C. Guo, B. Yang, and C. S. Jensen, “AutoCTS+: Joint neural architecture and hyperparameter search for correlated time series forecasting,” Proc. ACM Manag. Data , vol. 1, no. 1, pp. 97:1–97:26, 2023

    2023

  7. [15]

    AutoCTS++: zero-shot joint neural architecture and hyperparameter search for correlated time series forecasting,

    X. Wu, X. Wu, B. Yang, L. Zhou, C. Guo, X. Qiu, J. Hu, Z. Sheng, and C. S. Jensen, “AutoCTS++: zero-shot joint neural architecture and hyperparameter search for correlated time series forecasting,” VLDB J., vol. 33, no. 5, pp. 1743–1770, 2024

    2024

  8. [16]

    TFB: towards comprehensive and fair benchmarking of time series forecasting methods,

    X. Qiu, J. Hu, L. Zhou, X. Wu, J. Du, B. Zhang, C. Guo, A. Zhou, C. S. Jensen, Z. Sheng, and B. Yang, “TFB: towards comprehensive and fair benchmarking of time series forecasting methods,” Proc. VLDB Endow., vol. 17, no. 9, pp. 2363–2377, 2024

    2024

  9. [17]

    Qcore: Data-efficient, on-device continual calibration for quantized models,

    D. Campos, B. Yang, T. Kieu, M. Zhang, C. Guo, and C. S. Jensen, “Qcore: Data-efficient, on-device continual calibration for quantized models,” Proc. VLDB Endow., vol. 17, no. 11, pp. 2708–2721, 2024

    2024

  10. [18]

    Less is more: Efficient time series dataset condensation via two-fold modal matching,

    H. Miao, Z. Liu, Y . Zhao, C. Guo, B. Yang, K. Zheng, and C. S. Jensen, “Less is more: Efficient time series dataset condensation via two-fold modal matching,” Proc. VLDB Endow., vol. 18, 2024

    2024

  11. [19]

    Multiple time series forecasting with dynamic graph modeling,

    K. Zhao, C. Guo, Y . Cheng, P. Han, M. Zhang, and B. Yang, “Multiple time series forecasting with dynamic graph modeling,” Proc. VLDB Endow., vol. 17, no. 4, pp. 753–765, 2023

    2023

  12. [20]

    Finding structure in time,

    J. L. Elman, “Finding structure in time,” Cogn. Sci., vol. 14, no. 2, pp. 179–211, 1990

    1990

  13. [21]

    Long short-term memory,

    S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput., vol. 9, no. 8, pp. 1735–1780, 1997

    1997

  14. [22]

    Deepar: Probabilistic forecast- ing with autoregressive recurrent networks,

    V . Flunkert, D. Salinas, and J. Gasthaus, “Deepar: Probabilistic forecast- ing with autoregressive recurrent networks,” CoRR, vol. abs/1704.04110, 2017

    2017 arXiv

  15. [23]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems, NeurIPS, 2017, pp. 5998–6008

    2017

  16. [24]

    Pathformer: Multi-scale transformers with adaptive pathways for time series forecasting,

    P. Chen, Y . Zhang, Y . Cheng, Y . Shu, Y . Wang, Q. Wen, B. Yang, and C. Guo, “Pathformer: Multi-scale transformers with adaptive pathways for time series forecasting,” in ICLR, 2024

    2024

  17. [25]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., “Language models are unsupervised multitask learners,” OpenAI blog , vol. 1, no. 8, p. 9, 2019

    2019

  18. [26]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” inInternational Conference on Computer Vision, ICCV, 2021, pp. 9992–10 002

    2021

  19. [27]

    Informer: Beyond efficient transformer for long sequence time-series forecasting,

    H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, “Informer: Beyond efficient transformer for long sequence time-series forecasting,” in Conference on Artificial Intelligence, AAAI , 2021, pp. 11 106–11 115

    2021

  20. [28]

    Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting,

    H. Wu, J. Xu, J. Wang, and M. Long, “Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting,” in Advances in Neural Information Processing Systems, NeurIPS, 2021, pp. 22 419–22 430

    2021

  21. [29]

    Fedformer: Frequency enhanced decomposed transformer for long-term series fore- casting,

    T. Zhou, Z. Ma, Q. Wen, X. Wang, L. Sun, and R. Jin, “Fedformer: Frequency enhanced decomposed transformer for long-term series fore- casting,” in International Conference on Machine Learning, ICML , vol. 162, 2022, pp. 27 268–27 286

    2022

  22. [30]

    Are transformers effective for time series forecasting?

    A. Zeng, M. Chen, L. Zhang, and Q. Xu, “Are transformers effective for time series forecasting?” in Conference on Artificial Intelligence, AAAI , 2023, pp. 11 121–11 128

    2023

  23. [31]

    A time series is worth 64 words: Long-term forecasting with transformers,

    Y . Nie, N. H. Nguyen, P. Sinthong, and J. Kalagnanam, “A time series is worth 64 words: Long-term forecasting with transformers,” in International Conference on Learning Representations, ICLR , 2023

    2023

  24. [32]

    Crossformer: Transformer utilizing cross- dimension dependency for multivariate time series forecasting,

    Y . Zhang and J. Yan, “Crossformer: Transformer utilizing cross- dimension dependency for multivariate time series forecasting,” in International Conference on Learning Representations, ICLR , 2023

    2023

  25. [33]

    Tsmixer: Lightweight mlp-mixer model for multivariate time series forecasting,

    V . Ekambaram, A. Jati, N. Nguyen, P. Sinthong, and J. Kalagnanam, “Tsmixer: Lightweight mlp-mixer model for multivariate time series forecasting,” in ACM SIGKDD, 2023, pp. 459–469

    2023

  26. [34]

    Designing and training of lightweight neural networks on edge devices using early halting in knowledge distillation,

    R. Mishra and H. P. Gupta, “Designing and training of lightweight neural networks on edge devices using early halting in knowledge distillation,” IEEE Transactions on Mobile Computing , pp. 1–12, 2023

    2023

  27. [35]

    Adapyramid: Adaptive pyramid for accelerating high-resolution object detection on edge devices,

    X. Shi, S. Zhang, J. Wu, N. Chen, K. Cheng, Y . Liang, and S. Lu, “Adapyramid: Adaptive pyramid for accelerating high-resolution object detection on edge devices,” IEEE Transactions on Mobile Computing , pp. 1–16, 2023

    2023

  28. [36]

    A novel approach for seamless probabilistic photovoltaic power forecasting covering multi- ple time frames,

    T. Carriere, C. Vernay, S. Pitaval, and G. Kariniotakis, “A novel approach for seamless probabilistic photovoltaic power forecasting covering multi- ple time frames,” IEEE Trans. Smart Grid, vol. 11, no. 3, pp. 2281–2292, 2020

    2020

  29. [37]

    CGF: A category guidance based pm$ {2.5}$ sequence forecasting training framework,

    H. Yu, J. Hu, X. Zhou, C. Guo, B. Yang, and Q. Li, “CGF: A category guidance based pm$ {2.5}$ sequence forecasting training framework,” IEEE Trans. Knowl. Data Eng., vol. 35, no. 10, pp. 10 125–10 139, 2023

    2023

  30. [38]

    A transformer-based framework for multivariate time series representation learning,

    G. Zerveas, S. Jayaraman, D. Patel, A. Bhamidipaty, and C. Eickhoff, “A transformer-based framework for multivariate time series representation learning,” in ACM SIGKDD, 2021, pp. 2114–2124

    2021

  31. [39]

    Empirical evaluation of gated recurrent neural networks on sequence modeling,

    J. Chung, C ¸ . G¨ulc ¸ehre, K. Cho, and Y . Bengio, “Empirical evaluation of gated recurrent neural networks on sequence modeling,” CoRR, vol. abs/1412.3555, 2014

    2014 arXiv

  32. [40]

    A multi- horizon quantile recurrent forecaster,

    R. Wen, K. Torkkola, B. Narayanaswamy, and D. Madeka, “A multi- horizon quantile recurrent forecaster,” arXiv:1711.11053, 2017

    2017 arXiv

  33. [41]

    A dual-stage attention-based recurrent neural network for time series prediction,

    Y . Qin, D. Song, H. Chen, W. Cheng, G. Jiang, and G. W. Cottrell, “A dual-stage attention-based recurrent neural network for time series prediction,” in International Joint Conference on Artificial Intelligence, IJCAI, C. Sierra, Ed., 2017, pp. 2627–2633

    2017

  34. [42]

    Think globally, act locally: A deep neural network approach to high-dimensional time series forecasting,

    R. Sen, H. Yu, and I. S. Dhillon, “Think globally, act locally: A deep neural network approach to high-dimensional time series forecasting,” in Advances in Neural Information Processing Systems, NeurIPS , 2019, pp. 4838–4847

    2019

  35. [43]

    Reformer: The efficient transformer,

    N. Kitaev, L. Kaiser, and A. Levskaya, “Reformer: The efficient transformer,” in International Conference on Learning Representations, ICLR, 2020

    2020

  36. [44]

    Towards long-term time-series forecasting: Feature, pattern, and distribution,

    Y . Li, X. Lu, H. Xiong, J. Tang, J. Su, B. Jin, and D. Dou, “Towards long-term time-series forecasting: Feature, pattern, and distribution,” in International Conference on Data Engineering (ICDE), 2023, pp. 1611– 1624

    2023

  37. [45]

    Triformer: Triangular, variable-specific attentions for long sequence multivariate time series forecasting,

    R. Cirstea, C. Guo, B. Yang, T. Kieu, X. Dong, and S. Pan, “Triformer: Triangular, variable-specific attentions for long sequence multivariate time series forecasting,” in International Joint Conference on Artificial Intelligence, IJCAI, 2022, pp. 1994–2001

    2022

  38. [46]

    itrans- former: Inverted transformers are effective for time series forecasting,

    Y . Liu, T. Hu, H. Zhang, H. Wu, S. Wang, L. Ma, and M. Long, “itrans- former: Inverted transformers are effective for time series forecasting,” in International Conference on Learning Representations, ICLR , 2024

    2024

  39. [47]

    Tsmixer: An all- mlp architecture for time series forecasting,

    S. Chen, C. Li, N. Yoder, S. ¨O. Arik, and T. Pfister, “Tsmixer: An all- mlp architecture for time series forecasting,”CoRR, vol. abs/2303.06053, 2023

    2023 arXiv

  40. [48]

    Long- term forecasting with tide: Time-series dense encoder,

    A. Das, W. Kong, A. Leach, S. Mathur, R. Sen, and R. Yu, “Long- term forecasting with tide: Time-series dense encoder,” CoRR, vol. abs/2304.08424, 2023

    2023 arXiv

  41. [49]

    Lightts: Lightweight time series classification with adaptive ensemble distillation,

    D. Campos, M. Zhang, B. Yang, T. Kieu, C. Guo, and C. S. Jensen, “Lightts: Lightweight time series classification with adaptive ensemble distillation,” Proc. ACM Manag. Data , vol. 1, no. 2, pp. 171:1–171:27, 2023

    2023

  42. [50]

    Multi-resolution expansion of analysis in time-frequency domain for time series forecasting,

    K. Yan, C. Long, H. Wu, and Z. Wen, “Multi-resolution expansion of analysis in time-frequency domain for time series forecasting,” IEEE Trans. Knowl. Data Eng. , pp. 1–14, 2024

    2024

  43. [51]

    Timemixer: Decomposable multiscale mixing for time series forecasting,

    S. Wang, H. Wu, X. Shi, T. Hu, H. Luo, L. Ma, J. Y . Zhang, and J. Zhou, “Timemixer: Decomposable multiscale mixing for time series forecasting,” 2024

    2024

  44. [52]

    FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective,

    K. Yi, Q. Zhang, W. Fan, H. He, L. Hu, P. Wang, N. An, L. Cao, and Z. Niu, “FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective,” no. arXiv:2311.06190, 2023

    2023 arXiv

  45. [53]

    Disentangling structured components: Towards adaptive, interpretable and scalable time series forecasting,

    J. Deng, X. Chen, R. Jiang, D. Yin, Y . Yang, X. Song, and I. W. Tsang, “Disentangling structured components: Towards adaptive, interpretable and scalable time series forecasting,” IEEE Trans. Knowl. Data Eng. , vol. 36, no. 8, pp. 3783–3800, 2024

    2024

  46. [54]

    Learning transferable visual models from natural language supervi- sion,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervi- sion,” in International Conference on Machine Learning, ICML , vol. 1...

    2021

  47. [55]

    Petformer: Long-term time series forecasting via placeholder-enhanced transformer,

    S. Lin, W. Lin, W. Wu, S. Wang, and Y . Wang, “Petformer: Long-term time series forecasting via placeholder-enhanced transformer,” arXiv preprint arXiv:2308.04791, 2023

    2023 arXiv

  48. [56]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in International Conference on Learning Representations, ICLR , 2019

    2019

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.