AOSNET is a Hilbert-guided framework that performs adaptive oscillatory-state alignment to a learnable prior for improved long-term forecasting under non-stationary conditions.
hub
Transformers in time series: A survey
22 Pith papers cite this work, alongside 67 external citations. Polarity classification is still indexing.
22
Pith papers citing it
67
external citations · Pith
abstract
Transformers have achieved superior performances in many tasks in natural language processing and computer vision, which also triggered great interest in the time series community. Among multiple advantages of Transformers, the ability to capture long-range dependencies and interactions is especially attractive for time series modeling, leading to exciting progress in various time series applications. In this paper, we systematically review Transformer schemes for time series modeling by highlighting their strengths as well as limitations. In particular, we examine the development of time series Transformers in two perspectives. From the perspective of network structure, we summarize the adaptations and modifications that have been made to Transformers in order to accommodate the challenges in time series analysis. From the perspective of applications, we categorize time series Transformers based on common tasks including forecasting, anomaly detection, and classification. Empirically, we perform robust analysis, model size analysis, and seasonal-trend decomposition analysis to study how Transformers perform in time series. Finally, we discuss and suggest future directions to provide useful research guidance. To the best of our knowledge, this paper is the first work to comprehensively and systematically summarize the recent advances of Transformers for modeling time series data. We hope this survey will ignite further research interests in time series Transformers.
hub tools
citation-role summary
background 2
baseline 1
citation-polarity summary
representative citing papers
HapticLDM is the first latent diffusion model that generates vibrotactile signals directly from text, using dynamic text curation and global denoising to improve realism and semantic alignment over autoregressive baselines.
FactoryBench reveals that frontier LLMs achieve under 50% on structured causal questions and under 18% on decision-making in industrial robotic telemetry.
This survey and benchmark of deep time series models using the released TSLib library finds that models with specific structures perform well only on distinct analysis tasks.
PatchTST uses subseries patching and channel-independent Transformers to deliver significantly better long-term multivariate time series forecasting and strong self-supervised transfer performance.
A topological evaluation framework for transformer time series forecasts that derives fidelity scores from persistence diagrams of delay embeddings, including a phase-aware localized metric.
VESTA introduces dynamic tool creation for VLMs that outperforms static-tool and no-tool baselines on distribution fitting, time series, and astronomy tasks in the new DAWN benchmark.
Timer-S1 is a released 8.3B-parameter MoE time series model that achieves state-of-the-art MASE and CRPS scores on GIFT-Eval using serial scaling and Serial-Token Prediction.
ASTEROID is a spatiotemporal Transformer that predicts multi-step MD atomic coordinates with claimed higher accuracy and lower cost than iterative simulation on quantum-derived datasets.
Overlapping inference windows improve reconstruction-based time series anomaly detection by up to 28% relative gain across models on TSB-AD and UCR benchmarks and can alter rankings.
A momentum-corrected online stacking ensemble forecasts the new Kalimati Vegetable Price Index with RMSE 1.771, MAPE 0.68 percent, and R-squared 0.845 at the 90-day horizon.
The survey groups attention-based GNNs into three stages—graph recurrent attention networks, graph attention networks, and graph transformers—while reviewing architectures and future directions.
A temporal memory-aware Transformer emulator for the Emanuel convective parameterization shows lower offline errors and 10-year stability in single-column model tests compared to memory-less MLP and LSTM baselines.
ASTRAFier is a Transformer-BiLSTM-CNN model that classifies stellar variability from light curves, reporting 94.26% accuracy on Kepler data and 88.22% on TESS, then applied to 2.8 million TESS curves to release a catalog.
AR-KAN combines a pre-trained AR module with KAN to reduce redundancy while preserving temporal features, delivering lower probabilistic approximation error and stronger forecasting results on synthetic almost-periodic signals and real datasets.
FMMVCC combines Mamba-based encoders with multi-view contrastive learning and fuzzy clustering to achieve state-of-the-art univariate time series clustering with linear computational complexity.
PhysicsFormer applies a lightweight Transformer PINN with pseudo-sequential representations to convection, Burgers, lid-driven cavity, and inverse Navier-Stokes problems, reporting near-zero error in parameter identification and flow reconstruction from sparse noisy data.
Fourier-KAN-Mamba combines Fourier features, KAN nonlinearities, and Mamba state-space modeling with a gating mechanism and reports better anomaly detection performance than prior methods on the MSL, SMAP, and SWaT benchmarks.
Transformer-guided DRL cut training time steps to 25% of vanilla DRL while reaching 97.2% of optimal energy consumption for eVTOL takeoff versus 96.1%.
Proposes Adaptive Financial Transformer with regime-gated attention and a composite loss to predict stock returns while claiming to fix backtesting issues and reduce complexity by 15.2%.
A survey of positional encoding methods in transformer-based time series models that evaluates fixed, learnable, relative, and hybrid approaches on classification tasks and links effectiveness to data characteristics.
A federated learning framework lets distributed weather sensors train shared deep learning models for forecasting and anomaly detection while keeping raw data private.
citing papers explorer
-
Adaptive Oscillatory-State Alignment for Time Series Forecasting
AOSNET is a Hilbert-guided framework that performs adaptive oscillatory-state alignment to a learnable prior for improved long-term forecasting under non-stationary conditions.
-
HapticLDM: A Diffusion Model for Text-to-Vibrotactile Generation
HapticLDM is the first latent diffusion model that generates vibrotactile signals directly from text, using dynamic text curation and global denoising to improve realism and semantic alignment over autoregressive baselines.
-
FactoryBench: Evaluating Industrial Machine Understanding
FactoryBench reveals that frontier LLMs achieve under 50% on structured causal questions and under 18% on decision-making in industrial robotic telemetry.
-
Deep Time Series Models: A Comprehensive Survey and Benchmark
This survey and benchmark of deep time series models using the released TSLib library finds that models with specific structures perform well only on distinct analysis tasks.
-
A Time Series is Worth 64 Words: Long-term Forecasting with Transformers
PatchTST uses subseries patching and channel-independent Transformers to deliver significantly better long-term multivariate time series forecasting and strong self-supervised transfer performance.
-
TopoCast: A Topological Fidelity Framework for Evaluating Transformer-Based Time Series Forecasting
A topological evaluation framework for transformer time series forecasts that derives fidelity scores from persistence diagrams of delay embeddings, including a phase-aware localized metric.
-
VESTA: Visual Exploration with Statistical Tool Agents
VESTA introduces dynamic tool creation for VLMs that outperforms static-tool and no-tool baselines on distribution fitting, time series, and astronomy tasks in the new DAWN benchmark.
-
Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scaling
Timer-S1 is a released 8.3B-parameter MoE time series model that achieves state-of-the-art MASE and CRPS scores on GIFT-Eval using serial scaling and Serial-Token Prediction.
-
ASTEROID: A Spatiotemporal Information Transformer for Forecasting Multi-Step Time Series of Molecular Dynamics
ASTEROID is a spatiotemporal Transformer that predicts multi-step MD atomic coordinates with claimed higher accuracy and lower cost than iterative simulation on quantum-derived datasets.
-
Disjoint or Overlapping? Inference Windowing for Reconstruction-Based Time Series Anomaly Detection
Overlapping inference windows improve reconstruction-based time series anomaly detection by up to 28% relative gain across models on TSB-AD and UCR benchmarks and can alter rankings.
-
Kalimati Vegetable Price Index Forecasting with a Momentum Corrected Online Stacking Ensemble
A momentum-corrected online stacking ensemble forecasts the new Kalimati Vegetable Price Index with RMSE 1.771, MAPE 0.68 percent, and R-squared 0.845 at the 90-day horizon.
-
Attention-based graph neural networks: a survey
The survey groups attention-based GNNs into three stages—graph recurrent attention networks, graph attention networks, and graph transformers—while reviewing architectures and future directions.
-
climt-paraformer: Stable Emulation of Convective Parameterization using a Temporal Memory-aware Transformer
A temporal memory-aware Transformer emulator for the Emanuel convective parameterization shows lower offline errors and 10-year stability in single-column model tests compared to memory-less MLP and LSTM baselines.
-
ASTRAFier: A Novel and Scalable Transformer-based Stellar Variability Classifier
ASTRAFier is a Transformer-BiLSTM-CNN model that classifies stellar variability from light curves, reporting 94.26% accuracy on Kepler data and 88.22% on TESS, then applied to 2.8 million TESS curves to release a catalog.
-
AR-KAN: Autoregressive-Weight-Enhanced Kolmogorov-Arnold Network for Time Series Forecasting
AR-KAN combines a pre-trained AR module with KAN to reduce redundancy while preserving temporal features, delivering lower probabilistic approximation error and stronger forecasting results on synthetic almost-periodic signals and real datasets.
-
FMMVCC: Fuzzy Mamba-based Multi-View Contrastive Clustering for Univariate Time Series
FMMVCC combines Mamba-based encoders with multi-view contrastive learning and fuzzy clustering to achieve state-of-the-art univariate time series clustering with linear computational complexity.
-
A Simple but Efficient Transformer-Based Physics-Informed Neural Network for Incompressible Navier--Stokes Equations
PhysicsFormer applies a lightweight Transformer PINN with pseudo-sequential representations to convection, Burgers, lid-driven cavity, and inverse Navier-Stokes problems, reporting near-zero error in parameter identification and flow reconstruction from sparse noisy data.
-
Fourier-KAN-Mamba: A Novel State-Space Equation Approach for Time-Series Anomaly Detection
Fourier-KAN-Mamba combines Fourier features, KAN nonlinearities, and Mamba state-space modeling with a gating mechanism and reports better anomaly detection performance than prior methods on the MSL, SMAP, and SWaT benchmarks.
-
Transformer-Guided Deep Reinforcement Learning for Optimal Takeoff Trajectory Design of an eVTOL Drone
Transformer-guided DRL cut training time steps to 25% of vanilla DRL while reaching 97.2% of optimal energy consumption for eVTOL takeoff versus 96.1%.
-
Adaptive Financial Transformer with Regime-Gated Attention for Stock Return Prediction
Proposes Adaptive Financial Transformer with regime-gated attention and a composite loss to predict stock returns while claiming to fix backtesting issues and reduce complexity by 15.2%.
-
Positional Encoding in Transformer-Based Time Series Models: A Survey
A survey of positional encoding methods in transformer-based time series models that evaluates fixed, learnable, relative, and hybrid approaches on classification tasks and links effectiveness to data characteristics.
-
Federated Weather Modeling on Sensor Data
A federated learning framework lets distributed weather sensors train shared deep learning models for forecasting and anomaly detection while keeping raw data private.