REVIEW 13 cited by
One Fits All:Power General Time Series Analysis by Pretrained LM
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
One Fits All:Power General Time Series Analysis by Pretrained LM
read the original abstract
Although we have witnessed great success of pre-trained models in natural language processing (NLP) and computer vision (CV), limited progress has been made for general time series analysis. Unlike NLP and CV where a unified model can be used to perform different tasks, specially designed approach still dominates in each time series analysis task such as classification, anomaly detection, forecasting, and few-shot learning. The main challenge that blocks the development of pre-trained model for time series analysis is the lack of a large amount of data for training. In this work, we address this challenge by leveraging language or CV models, pre-trained from billions of tokens, for time series analysis. Specifically, we refrain from altering the self-attention and feedforward layers of the residual blocks in the pre-trained language or image model. This model, known as the Frozen Pretrained Transformer (FPT), is evaluated through fine-tuning on all major types of tasks involving time series. Our results demonstrate that pre-trained models on natural language or images can lead to a comparable or state-of-the-art performance in all main time series analysis tasks, as illustrated in Figure 1. We also found both theoretically and empirically that the self-attention module behaviors similarly to principle component analysis (PCA), an observation that helps explains how transformer bridges the domain gap and a crucial step towards understanding the universality of a pre-trained transformer.The code is publicly available at https://github.com/DAMO-DI-ML/One_Fits_All.
Forward citations
Cited by 13 Pith papers
-
TSVer: A Benchmark for Fact Verification Against Time-Series Evidence
TSVer is a new benchmark dataset for fact verification against time-series evidence, with 304 annotated real-world claims, 400 time series, verdicts, and justifications, plus baseline results showing current models struggle.
-
A decoder-only foundation model for time-series forecasting
A pretrained decoder-only patched transformer achieves near state-of-the-art zero-shot forecasting performance across diverse time series datasets and settings.
-
LLM-Guided Task-Semantic Field Factorization for Industrial Process Forecasting
Offline LLM-built variable-semantic directions, activated by each numerical window via a dual-path adapter, cut industrial forecasting MAE up to 25.5% with ~2–3k extra parameters.
-
Probabilistic Low-Voltage Peak Load Forecasting with Time Series Foundation Models Evaluated on Application-Oriented Metrics
Compares foundation models for probabilistic low-voltage load forecasting on 200 real feeders and introduces a grid-planning metric that scores peak prediction by its effect on asset cost-risk decisions.
-
ADAPTive Input Training for Many-to-One Pre-Training on Time-Series Classification
ADAPT is a new pre-training paradigm that aligns physical properties of time-series data to allow simultaneous training on 162 diverse classification datasets, achieving new state-of-the-art performance.
-
Surprised by Attention: Predictable Query Dynamics for Time Series Anomaly Detection
Predicting multi-head attention queries from history and scoring cosine mismatch against an EMA target, combined with reconstruction error, improves unsupervised multivariate anomaly ranking and localization.
-
Modular Foundation Models for Time-Series Perception in Digital Twins
A gated bank of frozen self-supervised time-series encoders, aligned and aggregated by a Transformer, supports competitive multi-task perception for digital twins and hydro-generator virtual sensing.
-
Zeus: Towards Tuning-Free Foundation Model for Time Series Analysis
Zeus proposes a multi-scale Transformer with point-wise tokenization and Multi-Objective Temporal Masking to enable tuning-free performance on forecasting, interpolation, and other time series tasks.
-
MSTN: A Lightweight and Fast Model for General TimeSeries Analysis
MSTN introduces a lightweight multi-scale temporal network using convolutional encoding, recurrent or attention-based modeling, and gated fusion to achieve claimed state-of-the-art results on 21 of 27 time series benc...
-
LLM4Delay: Flight Delay Prediction via Cross-Modality Adaptation of Large Language Models and Aircraft Trajectory Representation
LLM4Delay improves flight delay prediction accuracy by using instance-level projection to adapt LLMs for integrating textual aeronautical information with multiple aircraft trajectories.
-
MSTN: A Lightweight and Fast Model for General TimeSeries Analysis
MSTN is a lightweight hybrid model that reports new state-of-the-art results on 33 of 40 time series benchmarks for imputation, forecasting, and classification while using under one million parameters and sub-second i...
-
From Index to Equity: Pre-Training Transformers for Stock Return Prediction
Pre-training a transformer on the TSX index reduces binary cross-entropy loss on individual stocks from 0.69 to 0.64 and yields lower MSE than LSTM or XGBoost in regression, though ensembles achieve higher average dai...
-
Universal Time-Series Representation Learning: A Survey
A survey that proposes a taxonomy for universal time-series representation learning and reviews existing deep learning studies along with experimental setups.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.