Pith. sign in

REVIEW 5 major objections 5 minor 44 references

Enhancing LLMs for Time Series Forecasting via Structure-Guided Cross-Modal Alignment

T0 review · 5 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SGCMA transfers a language-derived state-transition graph into time series patches, then aligns patch embeddings to language tokens, allowing a frozen GPT-2 to achieve state-of-the-art forecasting accuracy.

desk verdict A novel alignment pipeline with credible initial results, but the paper's core claim—that language structure transfers—is unverified because the transition matrix is free to drift away from its text initialization. read the letter →

arxiv 2505.13175 v1 pith:WNSRSJK3 submitted 2025-05-19 cs.AI

Siming Sun , Kai Zhang , Xuejun Jiang , Wenchao Meng , Qinmin Yang This is my paper · ORCID
classification cs.AI
keywords timeseriesforecastinglargelanguagemodelscross-modalalignmenthiddenMarkovmodelmaximumentropystructuretransferfrozenLLMstate-transitiongraph
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that prior attempts to align time series with large language models fail because they match token-level features, whereas the real power of LLMs lies in modeling sequence-level structure. It proposes Structure-Guided Cross-Modal Alignment (SGCMA), which learns a hidden state-transition graph from natural language with a Hidden Markov Model, transfers that graph to time series patches through a Maximum Entropy Markov Model, and aligns patch embeddings to language tokens by cross-attention. The resulting structure-aware, semantically aligned embeddings are fed into a frozen GPT-2, and experiments report state-of-the-art forecasting accuracy on long-term, short-term, few-shot, and zero-shot benchmarks. The load-bearing idea is that giving time series a language-like state-transition structure is what unlocks the frozen LLM's generalization.

What carries the argument

The central mechanism is the state-transition graph shared across modalities. The HMM provides the structure: an initial-state distribution $\pi_{\text{text}}$, a transition matrix $A_{\text{text}}$, and an emission matrix $B_{\text{text}}$; the transition matrix is transferred to the temporal domain as a learnable hot-start prior. A shallow transformer-based Maximum Entropy Markov Model (MEMM) then decodes a state-probability sequence over patches, combining the language-derived transition dynamics with each patch's own embedding. Semantic alignment uses top-k tokens per state from the emission matrix as prototypes in a cross-attention layer, and the final patch embedding is the state-probability-weighted expectation of the attended embeddings.

What would settle it

Train SGCMA with the language HMM's transition matrix replaced by a random or identity transition matrix of the same size, keeping all other components and the training budget identical. If forecasting performance on ETTh1/ETTm1 does not degrade substantially, the claim that the language-derived structural prior is the source of the gains is falsified; the gains would then come from the added trainable transformer, prototypes, and cross-attention instead.

Watch

Extended reading notes

Core claim

SGCMA's central claim is that sequence-level structural consistency, not token-level mapping, is the right way to couple time series with pretrained language models. A Hidden Markov Model trained on text yields an initial-state distribution, a transition matrix, and an emission matrix; the transition matrix is then hot-started as a learnable prior in a shallow transformer-based MEMM that annotates each temporal patch with a state probability distribution. For semantic alignment, each patch attends to the top-k tokens of each hidden state as prototypes, and the final embedding is the state-probability-weighted average of those attended embeddings. Fed into a frozen GPT-2 followed by a linear head, this pipeline reports average MSE/MAE reductions of 3.7% over iTransformer in long-term forecasting and 3.6% over N-HiTS in short-term forecasting, with further gains in few-shot and zero-shot settings.

Load-bearing premise

The load-bearing assumption is that the latent state-transition structure estimated from natural language text is a meaningful structural prior for arbitrary numerical time series, so that transferring that graph helps rather than distorts the forecasting task.

Editorial extensions

If this is right

  • Frozen LLMs can serve as effective time series forecasters if the input embeddings carry language-like sequential structure, without any fine-tuning of the LLM weights.
  • The reusable structural prior extracted from text can be applied to time series from arbitrary domains, supporting few-shot and zero-shot transfer.
  • The ablation results imply that both structure alignment and semantic alignment are necessary: removing either degrades forecasting accuracy.
  • Because the LLM is frozen, the method's gains come from the input representation quality, so better structural priors should translate directly into better forecasts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not ablate the origin of the structural prior; replacing the language HMM transition matrix with a random or identity matrix, or with an HMM trained on other time series, would test whether the language-derived structure itself, rather than the added trainable components, produces the reported gains.
  • The same structure-transfer recipe could transfer to other sequential modalities with Markovian latent dynamics, such as audio, DNA, or sensor streams, wherever a state-transition metaphor is appropriate.
  • If the structural prior is truly doing the work, the latent states should map to interpretable temporal regimes (rising, falling, stationary); checking whether the assigned states align with obvious trend patterns in the data would provide a direct interpretability check.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. This paper proposes SGCMA, a framework for time series forecasting with frozen LLMs. It trains an HMM on text to obtain a state-transition graph, uses it to initialize learnable transition parameters in a MEMM-like module that assigns state probabilities to temporal patches, aligns patch embeddings with top-k language tokens per state via cross-attention, and feeds the resulting embeddings into a frozen GPT-2 with a linear head. Experiments cover long-term, short-term (M4), few-shot, and zero-shot settings, with ablations and a sensitivity analysis over the number of latent states.

Significance. If the mechanism claimed were established, SGCMA would be a novel and potentially influential contribution: it is one of the few methods that align time series and language at the sequence-structure level rather than at the token-feature level, and it avoids LLM fine-tuning. The empirical scope is broad, and the idea of transferring an HMM-derived transition graph is interesting. However, the paper's central assertion—that language-derived structural priors are what drive the gains—is not actually tested, because the language parameters only initialize free learnable parameters. The paper also contains internal inconsistencies in the reported SOTA results and lacks error bars and reproducibility details. The contribution is promising but requires substantial additional evidence.

major comments (5)
  1. [§3.2, Eq. (8)] The transfer of the language structure is implemented only by initializing π_time and A_time from π_text and A_text; these are then trained without any regularization toward the text values, and no permutation-invariant correspondence between text and time states is defined. Therefore, the experiments do not establish that a language-derived transition structure is used at inference. To support the central claim, the authors should compare against (i) random initialization of A_time/π_time, (ii) a uniform or identity transition matrix, and (iii) an HMM trained on an unrelated time series or on shuffled text. If all variants perform equally, the paper's thesis is falsified by its own design.
  2. [§4.4, Figure 4] The L1 distance of 0.046 between A_time and A_text is computed after training on raw state indices, which are arbitrary labels; the same matrices can have large L1 distance under a permutation of state indices. Without Hungarian/permutation matching and without reporting the distance at initialization or a random-matrix baseline, this number cannot be interpreted as evidence of structural transfer.
  3. [Tables 1 and 4] The abstract and §4.1/§4.3 claim state-of-the-art performance, but the tables show clear counterexamples: on Electricity and Traffic in Table 1, iTransformer achieves lower MSE than SGCMA (0.175 vs 0.189 and 0.422 vs 0.462); in Table 4 for h2→h1, SGCMA (0.574) is worse than TimeLLM (0.550) and GPT4TS (0.560). The aggregate improvement percentages should be recomputed honestly, or the claims should be qualified to 'most datasets' with explicit mention of the exceptions.
  4. [§4 (experimental setup)] All results are reported as averages over three runs with no standard deviations, confidence intervals, or significance tests. Many conclusions rest on small margins (e.g., Table 5: 0.434 vs 0.440 on ETTh1; Table 4: 0.574 vs 0.550), so without variance information the reported improvements cannot be distinguished from noise. Please provide error bars or per-run results for the main tables and the ablation.
  5. [§3.1] The text corpus used for HMM training is never specified (no name, size, tokenizer, or preprocessing), and no code is released. This is a reproducibility gap for the core structural prior. At minimum, the corpus and HMM training details (number of EM iterations, initialization, vocabulary size) must be stated, and ideally the code should be made available.
minor comments (5)
  1. [Eq. (7)] In Eq. (7), the feature vector is written as x_p although the patch embedding was defined as z_p in Eq. (5), and the numerator uses μ_n while the denominator uses μ_c; please unify the notation and the summation indices.
  2. [Eq. (11)] The summation in Eq. (11) is written as 'K' but the number of hidden states is N; K is already used for the top-k tokens, so the index should be N.
  3. [Eq. (10) and surrounding text] The sentence before Eq. (10) says the temperature τ is set to d_h = d_llm/H, while Eq. (10) defines τ = sqrt(d_h); one of these is a typo and should be corrected.
  4. [Figure 4 caption] The caption of Figure 4 should state which dataset and prediction setting are used, since the main text does not specify.
  5. [§3.2, Channel-Independent Patching] The paper does not specify whether the channel-independent patching shares parameters across channels; please clarify in the setup.

Circularity Check

1 steps flagged · score 2.0 of 10

Forecast results are external and non-circular; only the structural-transfer evidence in Figure 4 is partly self-referential because A_time is hot-started from A_text.

  1. self definitional [Section 3.2, Eq. (8); Section 4.4, Figure 4]
    "To incorporate structural priors, we transfer the language HMM’s initial latent state πtext and transition matrix Atext into temporal domain equivalents πtime, Atime as learnable parameters. ... the L1 distance between the two transition matrices is computed as 0.046, also demonstrating this effective structural alignment."

    A_time is initialized to A_text, so the post-training L1 distance of 0.046 between them is partially inherited from the initialization. The paper presents this distance as evidence that temporal dynamics 'maintains a high degree of similarity with the text,' but without a control (random or identity initialization, or an HMM trained on unrelated time series) the similarity could simply be persistence of the hot start. Since state indices are arbitrary labels, the comparison is not an independent confirmation of language-to-time-series transfer. The forecasting numbers themselves are external and not circular; only the mechanistic evidence in Figure 4 is partly self-referential.

full rationale

The central forecasting claims are evaluated against external benchmarks (ETT, Weather, Electricity, Traffic, M4), and the prediction targets are not used to define the HMM prior. The HMM transition matrix A_text is trained on text corpora in Section 3.1 and then transferred as an initialization, not as a fitted parameter of the forecasting objective. There is no self-citation chain and no uniqueness theorem imported from the authors' prior work. The only identifiable circularity is in the structural-transfer evidence: A_time is hot-started from A_text in Eq. (8), so the reported L1 distance of 0.046 in Figure 4 measures proximity to the initialization rather than independently demonstrating that language-derived structure was discovered in time series. This is a mild, localized circularity in the evidence for the mechanism, not in the benchmark results. The absence of a random-initialization ablation is a correctness risk for the transferability claim, but it is not itself a circularity argument. Overall, the paper is largely self-contained against external benchmarks, warranting a low score of 2.

Assumptions & free parameters 5 free parameters · 4 assumptions · 1 invented entities

The central claim rests on a text-derived structural prior whose transfer to arbitrary numerical channels is asserted, not demonstrated. The number of hidden states and top-k are hand-chosen, HMM parameters are fitted on an unspecified corpus, patch geometry is omitted, and the only internal validation of transfer is an L1 distance against the very matrix used for initialization.

free parameters (5)
  • number of latent states N = 100
    Set by hand as a key hyperparameter; Section 4.4 shows accuracy on Traffic still improving as N increases, so N=100 is a tuned choice rather than a derived quantity.
  • top-k tokens per state k in semantic alignment = not reported
    The top-k token selection in Section 3.3 is central to semantic alignment, but k is never stated or ablated.
  • temporal transition and initial parameters pi_time, A_time = learned
    Initialized from the text HMM but trained on the forecasting objective, so the final structure prior is partly fitted to the target data.
  • HMM parameters pi_text, A_text, B_text = not reported
    Estimated on an unnamed text corpus; the entire structural prior is determined by these fitted values, and the corpus and tokenizer details are omitted.
  • patch length lp and stride sp = not reported
    Section 3.2 invokes patch length and stride but never reports their values, which are required to reproduce the patch sequences.
assumptions (4)
  • domain assumption Time series and language share latent Markovian state dynamics that can be transferred across modalities
    Section 1 asserts this via the 'I like dogs' and 'rising/stable/falling' examples; Section 3.2 converts it into the design of Structure Alignment without independent evidence.
  • domain assumption A finite-state HMM with N=100 hidden states over text tokens captures structure useful for arbitrary numerical series
    Adopted in Section 3.1 and used to define prototypes in Section 3.2; no measure of HMM fit or state coherence is reported.
  • domain assumption Pretrained GPT-2 token embeddings form a semantic space in which cross-attention to the top-k emitted tokens improves temporal representations
    Section 3.3 assumes semantic alignment through token embeddings, but no analysis shows that the selected tokens' embeddings are semantically appropriate for numerical patches.
  • standard math The HMM forward algorithm (Eqs. 2 to 3) and MEMM recurrence (Eq. 8) are standard probabilistic machinery used correctly
    Used without formal proof, but these are textbook results; the main risk is not the math itself but the transferability assumption.
invented entities (1)
  • Shared latent state-transition graph between text and time series
    purpose: Provides the structural prior that makes frozen LLMs understand temporal patches
    The graph is never validated against an external structural benchmark; its only evidence is internal L1 proximity to an initialization (Section 4.4).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing LLMs for Time Series Forecasting via Structure-Guided Cross-Modal Alignment." pith.science (2026). https://pith.science/paper/WNSRSJK3

@misc{pith2026250513175,
  author       = {Pith},
  title        = {Pith review of: Enhancing LLMs for Time Series Forecasting via Structure-Guided Cross-Modal Alignment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WNSRSJK3}},
  note         = {Machine review of arXiv:2505.13175}
}
read the original abstract

The emerging paradigm of leveraging pretrained large language models (LLMs) for time series forecasting has predominantly employed linguistic-temporal modality alignment strategies through token-level or layer-wise feature mapping. However, these approaches fundamentally neglect a critical insight: the core competency of LLMs resides not merely in processing localized token features but in their inherent capacity to model holistic sequence structures. This paper posits that effective cross-modal alignment necessitates structural consistency at the sequence level. We propose the Structure-Guided Cross-Modal Alignment (SGCMA), a framework that fully exploits and aligns the state-transition graph structures shared by time-series and linguistic data as sequential modalities, thereby endowing time series with language-like properties and delivering stronger generalization after modality alignment. SGCMA consists of two key components, namely Structure Alignment and Semantic Alignment. In Structure Alignment, a state transition matrix is learned from text data through Hidden Markov Models (HMMs), and a shallow transformer-based Maximum Entropy Markov Model (MEMM) receives the hot-start transition matrix and annotates each temporal patch into state probability, ensuring that the temporal representation sequence inherits language-like sequential dynamics. In Semantic Alignment, cross-attention is applied between temporal patches and the top-k tokens within each state, and the ultimate temporal embeddings are derived by the expected value of these embeddings using a weighted average based on state probabilities. Experiments on multiple benchmarks demonstrate that SGCMA achieves state-of-the-art performance, offering a novel approach to cross-modal alignment in time series forecasting.

Figures

Figures reproduced from arXiv: 2505.13175 by the authors.

Figure 1
Figure 1. Our method compared to existing methods. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. An overview of SGCMA. The pipeline begins with [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Structure Alignment Workflow 3.3 Semantic Alignment To bridge temporal representations with the LLM’s semantic space, we introduce a semantic alignment module. Based on the structure-aligned state distributions Γ˜time, this module integrates semantic priors from language tokens via a cross-attention mechanism. Specifically, for each latent state sn, we extract its top-k tokens from the language-domain emission matri… view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: State Probability Before/After Com￾bining Patch Embedding Validity of Semantic Alignment To evaluate the effectiveness of semantic alignment, we remove the semantic alignment module and directly input the patch embedding sequences obtained from structure alignment into…
Figure 6
Figure 6. Figure 6: Performance with different numbers of [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 11 canonical work pages

  1. [1]

    Frozen in time: A joint video and image encoder for end-to-end retrieval

    Max Bain, Arsha Nagrani, Gül Varol, and Andrew Zisserman. Frozen in time: A joint video and image encoder for end-to-end retrieval. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1728–1738, 2021

    2021

  2. [2]

    Multimodal machine learn- ing: A survey and taxonomy

    Tadas Baltrušaitis, Chaitanya Ahuja, and Louis-Philippe Morency. Multimodal machine learn- ing: A survey and taxonomy. IEEE transactions on pattern analysis and machine intelligence, 41(2):423–443, 2018

    work page 2018

  3. [3]

    On the opportunities and risks of foundation models

    Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021

    arXiv 2021

  4. [4]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020

    1901

  5. [5]

    Nhits: Neural hierarchical interpolation for time series fore- casting

    Cristian Challu, Kin G Olivares, Boris N Oreshkin, Federico Garza Ramirez, Max Mergenthaler Canseco, and Artur Dubrawski. Nhits: Neural hierarchical interpolation for time series fore- casting. In Proceedings of the AAAI conference on artificial intelligence , volume 37, pages 6989–6997, 2023

    work page 2023

  6. [6]

    Llm4ts: Two-stage fine-tuning for time-series forecasting with pre-trained llms

    Ching Chang, Wen-Chih Peng, and Tien-Fu Chen. Llm4ts: Two-stage fine-tuning for time-series forecasting with pre-trained llms. CoRR, 2023

    work page 2023

  7. [7]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171–4186, 2019

    2019

  8. [8]

    Audioclip: Extending clip to image, text and audio

    Andrey Guzhov, Federico Raue, Jörn Hees, and Andreas Dengel. Audioclip: Extending clip to image, text and audio. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 976–980. IEEE, 2022

    2022

Show all 44 references
  1. [9]

    Context-alignment: Activating and enhancing llm capabilities in time series

    Yuxiao Hu, Qian Li, Dongxiao Zhang, Jinyue Yan, and Yuntian Chen. Context-alignment: Activating and enhancing llm capabilities in time series. arXiv preprint arXiv:2501.03747, 2025

    2025

  2. [10]

    Dnabert: pre-trained bidirectional encoder representations from transformers model for dna-language in genome

    Yanrong Ji, Zhihan Zhou, Han Liu, and Ramana V Davuluri. Dnabert: pre-trained bidirectional encoder representations from transformers model for dna-language in genome. Bioinformatics, 37(15):2112–2120, 2021

    2021

  3. [11]

    Scaling up visual and vision-language representation learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International conference on machine learning, pages 4904–4916....

    2021

  4. [12]

    Empowering time series analysis with large language models: A survey

    Yushan Jiang, Zijie Pan, Xikun Zhang, Sahil Garg, Anderson Schneider, Yuriy Nevmyvaka, and Dongjin Song. Empowering time series analysis with large language models: A survey. arXiv preprint arXiv:2402.03182, 2024

    2024 arXiv

  5. [13]

    Time-llm: Time series forecasting by reprogramming large language models

    Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, et al. Time-llm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.01728, 2023

    arXiv

  6. [14]

    Large language models are zero-shot reasoners

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199–22213, 2022

    2022

  7. [15]

    Blip: Bootstrapping language- image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language- image pre-training for unified vision-language understanding and generation. In International conference on machine learning, pages 12888–12900. PMLR, 2022

    2022

  8. [16]

    Temporal fusion transformers for interpretable multi-horizon time series forecasting

    Bryan Lim, Sercan Ö Arık, Nicolas Loeff, and Tomas Pfister. Temporal fusion transformers for interpretable multi-horizon time series forecasting. International Journal of Forecasting, 37(4):1748–1764, 2021. 10

    2021

  9. [17]

    Timecma: Towards llm-empowered multivariate time series forecasting via cross-modality alignment

    Chenxi Liu, Qianxiong Xu, Hao Miao, Sun Yang, Lingzheng Zhang, Cheng Long, Ziyue Li, and Rui Zhao. Timecma: Towards llm-empowered multivariate time series forecasting via cross-modality alignment. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pag...

    2025

  10. [18]

    Calf: Aligning llms for time series forecasting via cross-modal fine-tuning

    Peiyuan Liu, Hang Guo, Tao Dai, Naiqi Li, Jigang Bao, Xudong Ren, Yong Jiang, and Shu-Tao Xia. Calf: Aligning llms for time series forecasting via cross-modal fine-tuning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 18915–18923, 2025

    2025

  11. [19]

    itransformer: Inverted transformers are effective for time series forecasting

    Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. arXiv preprint arXiv:2310.06625, 2023

    2023 arXiv

  12. [20]

    Non-stationary transformers: Exploring the stationarity in time series forecasting

    Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting. Advances in neural information processing systems, 35:9881–9893, 2022

    2022

  13. [21]

    The m4 competition: Results, findings, conclusion and way forward

    Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. The m4 competition: Results, findings, conclusion and way forward. International Journal of forecasting, 34(4):802– 808, 2018

    2018

  14. [22]

    Maximum entropy markov models for information extraction and segmentation

    Andrew McCallum, Dayne Freitag, Fernando CN Pereira, et al. Maximum entropy markov models for information extraction and segmentation. In Icml, volume 17, pages 591–598, 2000

    2000

  15. [23]

    A time series is worth 64 words: Long-term forecasting with transformers

    Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14730, 2022

    2022 arXiv

  16. [24]

    N-beats: Neural basis expansion analysis for interpretable time series forecasting

    Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-beats: Neural basis expansion analysis for interpretable time series forecasting. arXiv preprint arXiv:1905.10437, 2019

    1905 arXiv

  17. [25]

    S2ip-llm: Semantic space informed prompt learning with llm for time series forecasting

    Zijie Pan, Yushan Jiang, Sahil Garg, Anderson Schneider, Yuriy Nevmyvaka, and Dongjin Song. S2ip-llm: Semantic space informed prompt learning with llm for time series forecasting. In Forty-first International Conference on Machine Learning, 2024

    2024

  18. [26]

    Pytorch: An imperative style, high-performance deep learning library

    A Paszke. Pytorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01703, 2019

    1912 arXiv

  19. [27]

    A tutorial on hidden markov models and selected applications in speech recognition

    Lawrence R Rabiner. A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257–286, 1989

    1989

  20. [28]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pa...

    2021

  21. [29]

    Improving language understanding by generative pre-training

    Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training. 2018

    2018

  22. [30]

    Language models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019

    2019

  23. [31]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020

    2020

  24. [32]

    Cross-modal fine-tuning: Align then refine

    Junhong Shen, Liam Li, Lucio M Dery, Corey Staten, Mikhail Khodak, Graham Neubig, and Ameet Talwalkar. Cross-modal fine-tuning: Align then refine. In International Conference on Machine Learning, pages 31030–31056. PMLR, 2023

    2023

  25. [33]

    Test: Text prototype aligned embedding to activate llm’s ability for time series

    Chenxi Sun, Hongyan Li, Yaliang Li, and Shenda Hong. Test: Text prototype aligned embedding to activate llm’s ability for time series. arXiv preprint arXiv:2308.08241, 2023

    2023 arXiv

  26. [34]

    Semantic reconstruction of continuous language from non-invasive brain recordings

    Jerry Tang, Amanda LeBel, Shailee Jain, and Alexander G Huth. Semantic reconstruction of continuous language from non-invasive brain recordings. Nature Neuroscience, 26(5):858–866, 2023

    2023

  27. [35]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 11

    2023 arXiv

  28. [36]

    Multimodal transformer for unaligned multimodal language sequences

    Yao-Hung Hubert Tsai, Shaojie Bai, Paul Pu Liang, J Zico Kolter, Louis-Philippe Morency, and Ruslan Salakhutdinov. Multimodal transformer for unaligned multimodal language sequences. In Proceedings of the conference. Association for computational linguistics. Meeting, volume 2...

    2019

  29. [37]

    Timemixer: Decomposable multiscale mixing for time series forecasting

    Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y Zhang, and Jun Zhou. Timemixer: Decomposable multiscale mixing for time series forecasting. arXiv preprint arXiv:2405.14616, 2024

    2024 arXiv

  30. [38]

    Times- net: Temporal 2d-variation modeling for general time series analysis

    Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Times- net: Temporal 2d-variation modeling for general time series analysis. arXiv preprint arXiv:2210.02186, 2022

    2022 arXiv

  31. [39]

    Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting

    Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting. Advances in neural information processing systems, 34:22419–22430, 2021

    2021

  32. [40]

    Coca: Contrastive captioners are image-text foundation models

    Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. arXiv preprint arXiv:2205.01917, 2022

    2022 arXiv

  33. [41]

    Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121–11128, 2023

    Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121–11128, 2023

    2023

  34. [42]

    A transformer-based framework for multivariate time series representation learning

    George Zerveas, Srideepika Jayaraman, Dhaval Patel, Anuradha Bhamidipaty, and Carsten Eickhoff. A transformer-based framework for multivariate time series representation learning. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pages 2114...

    2021

  35. [43]

    Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting

    Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International conference on machine learning, pages 27268–27286. PMLR, 2022

    2022

  36. [44]

    One fits all: Power general time series analysis by pretrained lm

    Tian Zhou, Peisong Niu, Liang Sun, Rong Jin, et al. One fits all: Power general time series analysis by pretrained lm. Advances in neural information processing systems, 36:43322–43355, 2023. 12

    2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.