Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

TSRM: A Lightweight Temporal Feature Encoding Architecture for Time Series Forecasting and Imputation

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read TSRM, a stackable encoder of multi-scale convolutions and self-attention, beats state-of-the-art forecasting on all seven benchmark datasets while using about 0.9 million parameters.

desk verdict A lightweight time-series encoder with real parameter-efficiency wins, but the 'beats all SOTA on all datasets' claim only holds when lookback is fixed at 96 — and the baselines are borrowed, not re-run. read the letter →

arxiv 2504.18878 v1 pith:DZINU4Z6 submitted 2025-04-26 cs.LG cs.AI

Robert Leppich , Michael Stenger , Daniel Grillmeyer , Vanessa Borst , Samuel Kounev This is my paper · ORCID
classification cs.LGcs.AI
keywords timeseriesforecastingimputationtemporalrepresentationlearningmulti-scaleconvolutionself-attentionchannelindependenceparameterefficiencylightweightarchitecture
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TSRM is a compact architecture for multivariate time series forecasting and imputation. The paper argues that interleaving multi-scale 1D convolutions with self-attention in stackable encoding layers lets a model capture temporal patterns at several abstraction levels without paying for a large transformer stack. On seven established benchmarks, the authors report that TSRM outperforms state-of-the-art baselines in forecasting on every dataset, and is best or near-best in imputation on most of them, with an average of 0.904 million trainable parameters against a 6.903 million median for the comparison models. If these results hold under matched training conditions, the practical payoff is that a lightweight, inspectable model can substitute for much larger forecasting networks in resource-constrained settings.

What carries the argument

The central object is the stackable encoding layer (EL). Its representation layer uses K independent 1D convolutions whose kernel sizes and dilations range from small local kernels to large dilated kernels, capturing details and broad trends at different scales by concatenating their outputs along the sequence dimension. The merge layer inverts this reshaping with matching transposed convolutions and a feed-forward projection, restoring the original sequence length and embedding dimension at every level. Between these two, pre-activation residual blocks built on LayerNorm, GeLU, multi-head self-attention, and dropout extract and amplify features. Because each EL preserves the input dimensions, the number of encoding layers becomes an independent hyperparameter, which is what allows hierarchical multi-scale representation with a small parameter budget.

What would settle it

Run every baseline through the same per-dataset random hyperparameter search, learning-rate range test, learning-rate scheduler, and early-stopping rule used for TSRM, with the lookback window both fixed at 96 and tuned per model; if any baseline then matches or beats TSRM's average MSE across the seven datasets, the paper's central claim fails.

Watch

Extended reading notes

Core claim

TSRM is built from encoding layers (ELs), each combining a representation layer, two transformer-style blocks, and a merge layer. The representation layer runs K parallel 1D CNN branches with different kernel sizes and dilations over the input and concatenates their outputs; the middle blocks apply LayerNorm, GeLU, multi-head self-attention, and dropout with residual connections; the merge layer uses matching transposed convolutions plus a feed-forward projection to restore the original sequence dimensions, so any number of ELs can be stacked. Processing is channel-independent by default, and a variant TSRM_IFC widens the linear layer to span all features to learn inter-feature correlations. With this design, the paper reports forecasting results averaged over horizons 96, 192, 336, and 720 that beat iTransformer, RLinear, PatchTST, TimesNet, and FEDformer on all seven datasets under the fixed lookback T=96 protocol. For imputation, TSRM is best on ECL and competitive on Weather, while on the four ETT subsets the plain channel-independent variant trails TimesNet and the inter-feature variant narrows the gap on ETTh1 and ETTh2 but not on the ETTm pair. The paper presents the combined evidence as showing that a low-parameter architecture can deliver state-of-the-art-level forecasting and competitive imputation across diverse data distributions.

Load-bearing premise

The headline comparison assumes that baseline numbers copied from earlier papers were produced under conditions equivalent to TSRM's own per-dataset random-search tuning and early stopping, but the paper used a different training protocol and, when the lookback window is tuned per model, PatchTST beats TSRM on ECL and ETTh2 in the paper's own Appendix G.

Editorial extensions

If this is right

  • State-of-the-art forecasting accuracy does not require deep or wide transformer stacks; a hierarchical multi-scale representation with self-attention can suffice.
  • A single backbone with under a million parameters can handle both forecasting and imputation, making the architecture practical for resource-constrained deployments.
  • Since attention weights can be mapped back onto the input series at each encoding layer, practitioners can inspect which time steps the model prioritizes, aiding diagnosis and model selection.
  • The stackable EL design decouples model depth from input length and feature count, so capacity can be scaled independently of the data dimensions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's forecasting numbers come from running TSRM with per-dataset random hyperparameter search and early stopping, while baseline numbers were copied from earlier papers; a matched-protocol rerun could shrink or even reverse the reported margins, and that experiment is not in the paper.
  • The channel-independent versus inter-feature split suggests a cheap diagnostic: datasets with strong cross-feature coupling, such as Weather and Exchange, should prefer TSRM_IFC, so a practitioner could preselect the variant by measuring feature correlations.
  • The N=0 ablation in the paper shows a configuration without encoding layers performs like DLinear, implying the convolutional representation layer carries much of the forecasting improvement; a follow-up could isolate the attention blocks' contribution by holding the CNN configuration fixed.
  • If the EL structure transfers to classification or anomaly detection, which the paper lists as future work, it would indicate the multi-scale representation is a general time-series feature extractor rather than a task-specific construction.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces TSRM, a lightweight architecture for multivariate time series forecasting and imputation. The model stacks encoding layers, each containing a representation layer of multiple 1D CNN branches with different kernel sizes and dilations, followed by a Transformer-style attention block and a transposed-convolution merge layer; a channel-independent variant (TSRM) and an inter-feature-correlation variant (TSRM_IFC, denoted TSRM_M in tables) are presented. Experiments on seven benchmark datasets under the fixed-lookback (T=96) protocol from the iTransformer paper report best average MSE for one of the two TSRM variants on all seven forecasting datasets, and competitive imputation results on six datasets, with a parameter count around 0.9M parameters on average. The paper includes ablations, a sensitivity study, attention visualization, and a public code repository.

Significance. If the reported results are taken at face value, TSRM is a useful addition to the lightweight end of the time-series modeling spectrum: it demonstrates that a multi-scale CNN representation layer combined with a compact attention encoder can match or beat much larger transformer models on standard benchmarks at a fraction of the parameter budget. The paper ships a public codebase, reports per-dataset hyperparameter ranges, includes multi-run statistics for two datasets, and explicitly concedes where imputation results fall short of SOTA, which are all strengths. The architecture itself is not conceptually radical—it combines existing CNN, attention, and transposed-convolution building blocks—but the empirical claim of strong performance at low parameter count is falsifiable and the ablations help isolate the contribution of the representation layer. The main significance is therefore practical rather than theoretical: the paper identifies a parameter-efficient configuration that is competitive under a specific protocol, provided the claims are properly qualified.

major comments (3)
  1. [Section 1, contribution bullet; Appendix G, Table 10] The unqualified claim that 'TSRM outperforms all SOTA models in forecasting for all datasets' is contradicted by the manuscript's own Table 10. Under the variable lookback protocol T∈{96,192,336,720}, PatchTST achieves a lower average MSE than TSRM on ECL (0.159 vs 0.162) and on ETTh2 (0.331 vs 0.348), and TSRM_IFC is also worse on both. The claim must be restricted to the fixed T=96 protocol, and even then Table 1 shows that the plain TSRM variant is not best on every dataset (e.g., ECL: TSRM 0.192 vs iTransformer 0.178), so the contribution bullet should explicitly credit the best of the two variants. As written, the central contribution statement is not supported by the evidence presented in the same paper.
  2. [Section 3.1, Tables 1/5, Table 6, Appendix G] The SOTA baseline numbers are copied from [19] and [21] while TSRM is trained with per-dataset random hyperparameter search, an automated LR range test, ReduceLROnPlateau, and early stopping. This protocol mismatch means the comparisons are not controlled: the baselines were not retrained under the same early-stopping or tuning conditions. Moreover, the only multi-run evidence, Table 6, reports standard deviations that are comparable to or larger than several win margins in Table 5 (e.g., ETTh2 at H=96: TSRM 0.289±0.028 vs RLinear 0.288; Weather at H=96: TSRM 0.162±0.013 vs TSRM_IFC-class results), and it covers only H=96 rather than the horizon-averaged metrics in Tables 1 and 5. The paper should either re-run baselines under a shared protocol or clearly state this limitation, and it should provide significance estimates for the horizon-averaged comparisons or temper the 'outperforms all SOTA' wording.
  3. [Section 3.3, Table 2; Section 1 contribution bullet] The imputation contribution claim of 'comparable or superior performance on the majority of datasets' is not precisely supported. The text admits that on the ETT datasets the authors 'were not able to match current SOTA results,' but Table 2 shows TSRM_IFC beats TimesNet on ETTh1 (0.059 vs 0.078) and comes close on ETTh2 (0.051 vs 0.050) and ETTm2 (0.024 vs 0.022). A clear count of wins/losses/ties against each baseline, and a statement of which variant is being referenced, is needed to make the majority claim verifiable. As written, the claim is too vague to be checked.
minor comments (6)
  1. [Section 2.1, Eq. (1)] The formula for D_j contains 'jT', which appears to be a typo for T; using the layer index j in the numerator makes the dimension formula ambiguous and should be corrected.
  2. [Throughout (Section 2.1, Tables 1, 2, 5, 7, 10)] The inter-feature-correlation variant is called TSRM_IFC in the text but TSRM_M in the tables; this inconsistent naming should be unified.
  3. [Table 2 and Table 7] The heading 'LigthTS' is a typo for 'LightTS'.
  4. [Section 3.4, Table 3] The expression 'median 6.903M±27.648M' mixes a median with a standard deviation; report mean±std or median with an interquartile range, and note that TSRM_IFC parameter counts are not included.
  5. [Appendix D, Table 6] Table 6 should state the forecast horizon (apparently H=96) and the exact configuration (TSRM vs TSRM_IFC) used for the five runs, since the numbers match the H=96 rows of Table 5 rather than the horizon-averaged values.
  6. [Appendix E] The hyperparameter study describes the search ranges but does not list the selected configurations per dataset; providing a table of final hyperparameters (or a pointer to config files in the repository) would strengthen reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: TSRM's empirical claims rest on held-out test evaluations against externally published baselines; the protocol-dependent 'all datasets' wording is a correctness/robustness issue, not a circular derivation.

full rationale

Walking the paper's derivation chain: the TSRM architecture is defined independently of the benchmarks (Section 2), the forecasting and imputation losses are standard MSE/MAE objectives (Eqs. 3-4), and the reported metrics are computed on held-out test splits after validation-based early stopping and per-dataset random hyperparameter search (Section 3.1). The SOTA baseline numbers are explicitly taken from external works ([19] for forecasting, [21] for imputation), not from the authors' own prior results, and no self-citation is load-bearing. There is no uniqueness theorem, no fitted parameter renamed as a prediction, and no equation in which a claimed output is defined in terms of the same output. The paper's own Appendix G (Table 10) shows that when the lookback window is treated as a hyperparameter, PatchTST achieves lower average MSE than TSRM on ECL (0.159 vs 0.162) and ETTh2 (0.331 vs 0.348), which contradicts the unqualified contribution bullet in Section 1 ("TSRM outperforms all SOTA models in forecasting for all datasets"). That is an internal consistency and protocol-fairness problem, but it is not circularity: the claim is empirical, externally falsifiable, and does not reduce by construction to its inputs. The multi-seed spread in Table 6 also overlaps with some win margins, which is a statistical-significance concern rather than a circular-reasoning concern. The ablations (Section 4, Appendix G) demonstrate that components contribute to performance, but they are not used to manufacture the headline result. Therefore the paper is self-contained against external benchmarks for the purpose of this circularity analysis, and no specific circular step can be quoted.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claims rest entirely on empirical evaluation, so the ledger's main entries are the tuned hyperparameters and the comparability assumptions behind the benchmark comparison. No physical entities are invented; the encoding, representation, and merge layers are software components. The main audit finding is that the headline 'outperforms all SOTA' is conditioned on a favorable protocol: the fixed-T=96 setup, per-dataset hyperparameter search, and baseline numbers lifted from [19] and [21] rather than rerun under identical conditions. The variable-lookback results in Appendix G show PatchTST ahead on ECL and ETTh2, which is not disclosed in the abstract or conclusion.

free parameters (6)
  • N (number of stacked encoding layers) = Not disclosed; random search over 0-12
    Chosen per dataset by validation-guided random search (Appendix E); ablation shows N=0 substantially worsens forecasting and imputation.
  • d (embedding dimension) = Not disclosed; random search over {8,16,32,64,128}
    Width of the per-timestep embedding and of attention; part of the per-dataset configuration.
  • h (attention heads) = Not disclosed; random search over {2,4,8,16,32}
    Multi-head self-attention configuration.
  • K and kernel/dilation configuration of representation-layer CNNs = Not disclosed; 1-4 CNN layers, kernels from ~3 values to 50-80% of input
    Defines the multi-scale abstraction levels; the paper states a hyperparameter study determines the constellation per dataset.
  • attention function = Not disclosed; vanilla vs sparse entmax15
    Choice between standard and sparse self-attention.
  • merge-layer gradient activation = Not disclosed; boolean
    Whether the merge layer has trainable gradients; examined in the hyperparameter study.
assumptions (4)
  • domain assumption Published SOTA numbers from [19] and [21] are directly comparable to TSRM runs under the authors' protocol
    Section 3.2 and 3.3 state all baseline results are taken from the iTransformer and TimesNet papers without re-running them under TSRM's early-stopping and scheduler setup.
  • domain assumption Fixed lookback T=96 is the correct evaluation protocol and the default claim-setting
    Section 3.2 fixes T=96 following [19]; Appendix G shows rankings change with tuned lookback, so the all-datasets claim is protocol-dependent.
  • domain assumption Channel independence is an adequate inductive bias for the base TSRM
    Section 2.1 splits the multivariate input into F univariate sequences sharing one backbone, per PatchTST; the IFC variant is needed when inter-feature correlation matters.
  • domain assumption Early stopping on 1% validation MSE improvement with patience 3 selects good models
    Section 3.1; this training rule differs from the baselines' published protocols and is part of why baseline re-runs would be needed for a clean comparison.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TSRM: A Lightweight Temporal Feature Encoding Architecture for Time Series Forecasting and Imputation." pith.science (2026). https://pith.science/paper/DZINU4Z6

@misc{pith2026250418878,
  author       = {Pith},
  title        = {Pith review of: TSRM: A Lightweight Temporal Feature Encoding Architecture for Time Series Forecasting and Imputation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DZINU4Z6}},
  note         = {Machine review of arXiv:2504.18878}
}
read the original abstract

We introduce a temporal feature encoding architecture called Time Series Representation Model (TSRM) for multivariate time series forecasting and imputation. The architecture is structured around CNN-based representation layers, each dedicated to an independent representation learning task and designed to capture diverse temporal patterns, followed by an attention-based feature extraction layer and a merge layer, designed to aggregate extracted features. The architecture is fundamentally based on a configuration that is inspired by a Transformer encoder, with self-attention mechanisms at its core. The TSRM architecture outperforms state-of-the-art approaches on most of the seven established benchmark datasets considered in our empirical evaluation for both forecasting and imputation tasks. At the same time, it significantly reduces complexity in the form of learnable parameters. The source code is available at https://github.com/RobertLeppich/TSRM.

Figures

Figures reproduced from arXiv: 2504.18878 by the authors.

Figure 1
Figure 1. Illustration of the proposed Time Series Representation Model (TSRM) framework, primarily composed of [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Ablation study results for the architecture variations (bottom) and sensitivity study for the hyperparameter [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Highlighted attention weights during an ETTh1 forecasting task for all 3 [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Decomposing the Time Series Forecasting Pipeline: A Modular Approach for Time Series Representation, Information Extraction, and Projection

    cs.AI 2025-07 conditional novelty 6.0 of 10

    REP-Net, a modular pipeline of representation, memory, and projection modules, achieves competitive forecasting accuracy on seven multivariate benchmarks with lower computational cost.

Reference graph

Works this paper leans on

51 extracted references · 8 canonical work pages · cited by 1 Pith paper

  1. [19]

    itransformer: Inverted transformers are effective for time series forecasting

    Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. arXiv preprint arXiv:2310.06625, 2024

    arXiv 2024

  2. [21]

    Timesnet: Temporal 2d-variation modeling for general time series analysis

    Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. arXiv preprint arXiv:2210.02186, 2022

    arXiv 2022

  3. [1]

    Optimal multi-scale patterns in time series streams

    Spiros Papadimitriou and Philip Yu. Optimal multi-scale patterns in time series streams. In Proceedings of the 2006 ACM SIGMOD international conference on Management of data, pages 647–658, 2006

    2006

  4. [2]

    Statstream: Statistical monitoring of thousands of data streams in real time

    Yunyue Zhu and Dennis Shasha. Statstream: Statistical monitoring of thousands of data streams in real time. In VLDB’02: Proceedings of the 28th International Conference on Very Large Databases, pages 358–369. Elsevier, 2002

    2002

  5. [3]

    Transformer-based models to deal with heterogeneous environments in human activity recognition

    Sannara Ek, François Portet, and Philippe Lalanda. Transformer-based models to deal with heterogeneous environments in human activity recognition. Personal and Ubiquitous Computing, 27(6):2267–2280, 2023

    2023

  6. [4]

    Gradient flow in recurrent nets: the difficulty of learning long-term dependencies, 2001

    Sepp Hochreiter, Yoshua Bengio, Paolo Frasconi, Jürgen Schmidhuber, et al. Gradient flow in recurrent nets: the difficulty of learning long-term dependencies, 2001

    2001

  7. [5]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008, 2017

    2017

  8. [6]

    Deep transformer models for time series forecasting: The influenza prevalence case

    Neo Wu, Bradley Green, Xue Ben, and Shawn O’Banion. Deep transformer models for time series forecasting: The influenza prevalence case. arXiv preprint arXiv:2001.08317, 2020

    arXiv 2001

Show all 51 references
  1. [7]

    An improved relative self-attention mechanism for transformer with application to music generation

    Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer, Curtis Hawthorne, Andrew M Dai, Matthew D Hoffman, and Douglas Eck. An improved relative self-attention mechanism for transformer with application to music generation. arXiv preprint arXiv:1809.04281, 2, 2018

    2018 arXiv

  2. [8]

    A time-restricted self-attention layer for asr

    Daniel Povey, Hossein Hadian, Pegah Ghahremani, Ke Li, and Sanjeev Khudanpur. A time-restricted self-attention layer for asr. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 5874–5878. IEEE, 2018

    2018

  3. [9]

    Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting

    Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Advances in neural information processing systems, 32, 2019

    2019

  4. [10]

    Informer: Beyond efficient transformer for long sequence time-series forecasting

    Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106–11115, 2021

    2021

  5. [11]

    Autoformer: Decomposition transformers with auto- correlation for long-term series forecasting

    Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto- correlation for long-term series forecasting. Advances in Neural Information Processing Systems, 34:22419–22430, 2021. 9 Time Series Representation Models PREPRINT

    2021

  6. [12]

    Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting

    Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference on Machine Learning, pages 27268–27286. PMLR, 2022

    2022

  7. [13]

    Brits: Bidirectional recurrent imputation for time series

    Wei Cao, Dong Wang, Jian Li, Hao Zhou, Lei Li, and Yitan Li. Brits: Bidirectional recurrent imputation for time series. In Advances in Neural Information Processing Systems, pages 6775–6785, 2018

    2018

  8. [14]

    Gp-vae: Deep probabilistic time series imputation

    Vincent Fortuin, Dmitry Baranchuk, Gunnar Rätsch, and Stephan Mandt. Gp-vae: Deep probabilistic time series imputation. In International conference on artificial intelligence and statistics, pages 1651–1661. PMLR, 2020

    2020

  9. [15]

    Multivariate time series imputation with generative adversarial networks

    Yonghong Luo, Xiangrui Cai, Ying Zhang, Jun Xu, et al. Multivariate time series imputation with generative adversarial networks. Advances in neural information processing systems, 31, 2018

    2018

  10. [16]

    Saits: Self-attention-based imputation for time series

    Wenjie Du, David Côté, and Yan Liu. Saits: Self-attention-based imputation for time series. Expert Systems with Applications, 219:119619, 2023

    2023

  11. [17]

    Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121–11128, 2023

    Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121–11128, 2023

    2023

  12. [18]

    A time series is worth 64 words: Long-term forecasting with transformers

    Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14730, 2022

    2022 arXiv

  13. [20]

    Pathformer: Multi-scale transformers with adaptive pathways for time series forecasting

    Peng Chen, Yingying Zhang, Yunyao Cheng, Yang Shu, Yihang Wang, Qingsong Wen, Bin Yang, and Chenjuan Guo. Pathformer: Multi-scale transformers with adaptive pathways for time series forecasting. In International Conference on Learning Representations, 2024

    2024

  14. [22]

    Reversible instance normalization for accurate time-series forecasting against distribution shift

    Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, and Jaegul Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations, 2021

    2021

  15. [23]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  16. [24]

    Long-term forecasting with tide: Time-series dense encoder

    Abhimanyu Das, Weihao Kong, Andrew Leach, Shaan Mathur, Rajat Sen, and Rose Yu. Long-term forecasting with tide: Time-series dense encoder. arXiv preprint arXiv:2304.08424, 2023

    2023 arXiv

  17. [25]

    A review on word embedding techniques for text classification

    S Selva Birunda and R Kanniga Devi. A review on word embedding techniques for text classification. Innovative Data Communication Technologies and Application: Proceedings of ICIDCA 2020, pages 267–281, 2021

    2020

  18. [26]

    Identity mappings in deep residual networks

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In ECCV, October 2016

    2016

  19. [27]

    Adversarial sparse transformer for time series forecasting

    Sifan Wu, Xi Xiao, Qianggang Ding, Peilin Zhao, Ying Wei, and Junzhou Huang. Adversarial sparse transformer for time series forecasting. Advances in Neural Information Processing Systems, 33, 2020

    2020

  20. [28]

    Uci machine learning repository

    Dheeru Dua, Casey Graff, et al. Uci machine learning repository. 2017

    2017

  21. [29]

    Revisiting long-term time series forecasting: An investigation on linear mapping

    Zhe Li, Shiyi Qi, Yiduo Li, and Zenglin Xu. Revisiting long-term time series forecasting: An investigation on linear mapping. arXiv preprint arXiv:2305.10721, 2023

    2023 arXiv

  22. [30]

    Less is more: Fast multivariate time series forecasting with light sampling-oriented mlp structures

    Tianping Zhang, Yizhuo Zhang, Wei Cao, Jiang Bian, Xiaohan Yi, Shun Zheng, and Jian Li. Less is more: Fast multivariate time series forecasting with light sampling-oriented mlp structures. arXiv preprint arXiv:2207.01186, 2022

    2022 arXiv

  23. [31]

    Are transformers effective for time series forecasting? arXiv preprint arXiv:2205.13504, 2022

    Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? arXiv preprint arXiv:2205.13504, 2022

    2022 arXiv

  24. [32]

    Non-stationary transformers: Exploring the stationarity in time series forecasting

    Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting. Advances in Neural Information Processing Systems, 35:9881–9893, 2022

    2022

  25. [33]

    Deep time series models: A comprehensive survey and benchmark

    Yuxuan Wang, Haixu Wu, Jiaxiang Dong, Yong Liu, Mingsheng Long, and Jianmin Wang. Deep time series models: A comprehensive survey and benchmark. 2024

    2024

  26. [34]

    Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting

    Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In The eleventh international conference on learning representations, 2023

    2023

  27. [35]

    Himtm: Hierarchical multi-scale masked time series modeling for long-term forecasting

    Shubao Zhao, Ming Jin, Zhaoxiang Hou, Chengyi Yang, Zengxiang Li, Qingsong Wen, and Yi Wang. Himtm: Hierarchical multi-scale masked time series modeling for long-term forecasting. arXiv preprint arXiv:2401.05012, 2024. 10 Time Series Representation Models PREPRINT

    2024 arXiv

  28. [36]

    A decoder-only foundation model for time-series forecasting

    Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. A decoder-only foundation model for time-series forecasting. arXiv preprint arXiv:2310.10688, 2024

    2024 arXiv

  29. [37]

    Transferability in deep learning: A survey

    Junguang Jiang, Yang Shu, Jianmin Wang, and Mingsheng Long. Transferability in deep learning: A survey. arXiv preprint arXiv:2201.05867, 2022

    2022 arXiv

  30. [38]

    Tsmixer: Lightweight mlp-mixer model for multivariate time series forecasting

    Vijay Ekambaram, Arindam Jati, Nam Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. Tsmixer: Lightweight mlp-mixer model for multivariate time series forecasting. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, page 459–469, New York,...

    2023

  31. [39]

    Simmtm: A simple pre- training framework for masked time-series modeling

    Jiaxiang Dong, Haixu Wu, Haoran Zhang, Li Zhang, Jianmin Wang, and Mingsheng Long. Simmtm: A simple pre- training framework for masked time-series modeling. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processi...

    2023

  32. [40]

    CoST: Contrastive learning of disentangled seasonal-trend representations for time series forecasting

    Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. CoST: Contrastive learning of disentangled seasonal-trend representations for time series forecasting. In International Conference on Learning Representations, 2022

    2022

  33. [41]

    Learning to embed time series patches independently

    Seunghan Lee, Taeyoung Park, and Kibok Lee. Learning to embed time series patches independently. In The International Conference on Learning Representations (ICLR), 2024

    2024

  34. [42]

    On the opportunities and risks of foundation models

    Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021

    2021 arXiv

  35. [43]

    Self-supervised contrastive pre-training for time series via time-frequency consistency.Advances in Neural Information Processing Systems, 35:3988–4003, 2022

    Xiang Zhang, Ziyuan Zhao, Theodoros Tsiligkaridis, and Marinka Zitnik. Self-supervised contrastive pre-training for time series via time-frequency consistency.Advances in Neural Information Processing Systems, 35:3988–4003, 2022

    2022

  36. [44]

    One fits all: Power general time series analysis by pretrained lm

    Tian Zhou, Peisong Niu, Liang Sun, Rong Jin, et al. One fits all: Power general time series analysis by pretrained lm. Advances in neural information processing systems, 36:43322–43355, 2023

    2023

  37. [45]

    Lag-llama: Towards foundation models for time series forecasting

    Kashif Rasul, Arjun Ashok, Andrew Robert Williams, Arian Khorasani, George Adamopoulos, Rishika Bhag- watkar, Marin Biloš, Hena Ghonia, Nadhir Vincent Hassen, Anderson Schneider, et al. Lag-llama: Towards foundation models for time series forecasting. arXiv preprint arXiv:2310...

    arXiv

  38. [46]

    MOMENT: A family of open time-series foundation models

    Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, and Artur Dubrawski. MOMENT: A family of open time-series foundation models. In Forty-first International Conference on Machine Learning, 2024

    2024

  39. [47]

    Unified training of universal time series forecasting transformers

    Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Unified training of universal time series forecasting transformers. In Forty-first International Conference on Machine Learning, 2024

    2024

  40. [48]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  41. [49]

    Scinet: Time series modeling and forecasting with sample convolution and interaction

    Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, and Qiang Xu. Scinet: Time series modeling and forecasting with sample convolution and interaction. Advances in Neural Information Processing Systems, 35:5816–5828, 2022

    2022

  42. [50]

    Etsformer: Exponential smoothing transformers for time-series forecasting

    Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. Etsformer: Exponential smoothing transformers for time-series forecasting. arXiv preprint arXiv:2202.01381, 2022

    2022 arXiv

  43. [51]

    Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting

    Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In International conference on learning representations, 2021. 11 Time Series Represen...

    2021

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.