{"work":{"id":"d6d0a3ac-d695-4de0-ba2d-4e1d31ac8359","openalex_id":null,"doi":null,"arxiv_id":"2211.14730","raw_key":null,"title":"A Time Series is Worth 64 Words: Long-term Forecasting with Transformers","authors":null,"authors_text":"Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, Jayant Kalagnanam","year":2022,"venue":"cs.LG","abstract":"We propose an efficient design of Transformer-based models for multivariate time series forecasting and self-supervised representation learning. It is based on two key components: (i) segmentation of time series into subseries-level patches which are served as input tokens to Transformer; (ii) channel-independence where each channel contains a single univariate time series that shares the same embedding and Transformer weights across all the series. Patching design naturally has three-fold benefit: local semantic information is retained in the embedding; computation and memory usage of the attention maps are quadratically reduced given the same look-back window; and the model can attend longer history. Our channel-independent patch time series Transformer (PatchTST) can improve the long-term forecasting accuracy significantly when compared with that of SOTA Transformer-based models. We also apply our model to self-supervised pre-training tasks and attain excellent fine-tuning performance, which outperforms supervised training on large datasets. Transferring of masked pre-trained representation on one dataset to others also produces SOTA forecasting accuracy. Code is available at: https://github.com/yuqinie98/PatchTST.","external_url":"https://arxiv.org/abs/2211.14730","cited_by_count":null,"metadata_source":"pith","metadata_fetched_at":"2026-05-25T05:10:21.958066+00:00","pith_arxiv_id":"2211.14730","created_at":"2026-05-10T07:26:59.330571+00:00","updated_at":"2026-05-25T05:10:21.958066+00:00","title_quality_ok":true,"display_title":"A Time Series is Worth 64 Words: Long-term Forecasting with Transformers","render_title":"A Time Series is Worth 64 Words: Long-term Forecasting with Transformers"},"hub":{"state":{"work_id":"d6d0a3ac-d695-4de0-ba2d-4e1d31ac8359","tier":"hub","tier_reason":"10+ Pith inbound or 1,000+ external citations","pith_inbound_count":60,"external_cited_by_count":null,"distinct_field_count":7,"first_pith_cited_at":"2024-08-01T14:35:24+00:00","last_pith_cited_at":"2026-05-22T09:13:29+00:00","author_build_status":"not_needed","summary_status":"needed","contexts_status":"needed","graph_status":"needed","ask_index_status":"not_needed","reader_status":"not_needed","recognition_status":"not_needed","updated_at":"2026-05-30T11:01:04.162231+00:00","tier_text":"hub"},"tier":"hub","role_counts":[{"context_role":"background","n":8},{"context_role":"baseline","n":2},{"context_role":"method","n":1}],"polarity_counts":[{"context_polarity":"background","n":7},{"context_polarity":"baseline","n":2},{"context_polarity":"unclear","n":1},{"context_polarity":"use_method","n":1}],"runs":{},"summary":{},"graph":{},"authors":[]}}