Introduces the TSCognition benchmark for cognitive time series reasoning tasks and the TSAlign alignment framework, reporting outperformance over LLM, VLM, and time-series baselines on TSCognition and TimerBed with lower compute.
hub Canonical reference
iTransformer: Inverted Transformers Are Effective for Time Series Forecasting
Canonical reference. 80% of citing Pith papers cite this work as background.
69
Pith papers citing it
366
external citations · Pith
Background
80% of classified citations
abstract
The recent boom of linear forecasting models questions the ongoing passion for architectural modifications of Transformer-based forecasters. These forecasters leverage Transformers to model the global dependencies over temporal tokens of time series, with each token formed by multiple variates of the same timestamp. However, Transformers are challenged in forecasting series with larger lookback windows due to performance degradation and computation explosion. Besides, the embedding for each temporal token fuses multiple variates that represent potential delayed events and distinct physical measurements, which may fail in learning variate-centric representations and result in meaningless attention maps. In this work, we reflect on the competent duties of Transformer components and repurpose the Transformer architecture without any modification to the basic components. We propose iTransformer that simply applies the attention and feed-forward network on the inverted dimensions. Specifically, the time points of individual series are embedded into variate tokens which are utilized by the attention mechanism to capture multivariate correlations; meanwhile, the feed-forward network is applied for each variate token to learn nonlinear representations. The iTransformer model achieves state-of-the-art on challenging real-world datasets, which further empowers the Transformer family with promoted performance, generalization ability across different variates, and better utilization of arbitrary lookback windows, making it a nice alternative as the fundamental backbone of time series forecasting. Code is available at this repository: https://github.com/thuml/iTransformer.
hub tools
citation-role summary
background 4
baseline 1
citation-polarity summary
representative citing papers
FinStressTS is a parametric synthetic benchmark with 30 environments across six mechanism families for evaluating point and probabilistic forecasting models on financial time series.
Olivia harmonizes time series datasets via normalized power spectral density using a Harmonizer module and resonator-based HarmonicAttention, achieving state-of-the-art zero-shot, few-shot, and full-shot forecasting on TSLib, GIFT-Eval, and GluonTS benchmarks.
SeesawNet dynamically balances common and instance-specific dependencies via ASNA in temporal and channel dimensions, outperforming prior methods on non-stationary forecasting benchmarks.
Introduces the task of counterfactual time series forecasting with textual conditions plus a text-attribution mechanism that improves accuracy by distinguishing mutable from immutable factors.
Synthetic data augmentation helps channel-mixing time series models but degrades channel-independent ones, with reliable gains only from seasonal-trend generators and gradual schedules in low-resource settings.
A new evaluation framework shows that blood glucose forecasting models with high overall accuracy often fail at timely hypoglycemia detection in high-risk periods and at predicting effects of changed insulin doses.
LatentTSF improves time series forecasting accuracy and representation quality by shifting prediction from observation space to a learned latent state space via autoencoding.
Super-Linear introduces a pretrained MoE architecture using frequency-specialized linear experts and spectral gating for efficient general time series forecasting.
Sundial uses TimeFlow Loss for native pre-training of Transformers on continuous time series from TimeBench, achieving SOTA point and probabilistic forecasting with millisecond inference.
This survey and benchmark of deep time series models using the released TSLib library finds that models with specific structures perform well only on distinct analysis tasks.
NEST improves long-term multivariate forecasting under dataset-level distribution shifts by clustering regimes in moment-entropy space and recomposing specialized variate-attention experts via a content-plus-geometry router.
Optimized Ridge regression with series-specific preprocessing beats prior linear forecasters and exceeds Transformer, MLP, and CNN baselines on six of eight time-series benchmarks.
MetaPS trains models via simulation rollouts to select from programmatic strategy libraries for market agents, yielding better performance than fixed or direct LLM baselines across model sizes.
LakeFM pre-trains on large ecological datasets to forecast irregular lake time series and reports competitive or superior performance with physically plausible outputs.
MF-Net learns a shared field state and mechanical transition rule from trajectories to deliver competitive forecasting and recoverable relation matrices on Lorenz-96 and real systems.
SPDM is a geometry-aware state-space model that projects covariance matrices onto the SPD manifold tangent space and uses geometric gating to modulate SSM parameters for improved multivariate time series forecasting.
AME-TS is a structure-guided sparse MoE foundation model for time series that aligns expert routing with series-level temporal descriptors to achieve strong accuracy-efficiency tradeoffs on GIFT-Eval while improving specialization stability.
DeRegiME uses a sparse variational GP with nonstationary regime-mixing kernel to decompose forecasts into mean, residual regimes, and noise for improved probabilistic forecasting under distribution shift.
DAD4TS trains a diffusion-based generator jointly with a forecaster under RL control and geometric projections to produce augmentation samples that boost accuracy on small-scale time-series data, with validation reported on five of six real-world datasets.
Empirical scaling study of ECG models finds SSL scales robustly while ResNets show 1.3-2.5x better parameter efficiency and SSL up to 16x better data efficiency than supervised baselines on out-of-distribution tasks.
XDecomposer uses set prediction and phase-query decomposition to jointly identify phases and reconstruct multiphase PXRD patterns without priors.
ST-PT turns transformers into explicit factor graphs for time series, enabling structural injection of symbolic priors, per-sample conditional generation, and principled latent autoregressive forecasting via MFVI iterations.
GeoCert uses hyperbolic geometry to unify forecasting with physical reasoning and built-in formal certification, claiming major gains in accuracy and efficiency.
citing papers explorer
-
From Recognition to Understanding: Unlocking Cognitive Time Series Reasoning with LLMs
Introduces the TSCognition benchmark for cognitive time series reasoning tasks and the TSAlign alignment framework, reporting outperformance over LLM, VLM, and time-series baselines on TSCognition and TimerBed with lower compute.
-
FinStressTS: A Parametric Synthetic Benchmark for Time-Series Forecasting in Finance
FinStressTS is a parametric synthetic benchmark with 30 environments across six mechanism families for evaluating point and probabilistic forecasting models on financial time series.
-
Olivia: Harmonizing Time Series Foundation Models with Power Spectral Density
Olivia harmonizes time series datasets via normalized power spectral density using a Harmonizer module and resonator-based HarmonicAttention, achieving state-of-the-art zero-shot, few-shot, and full-shot forecasting on TSLib, GIFT-Eval, and GluonTS benchmarks.
-
SeesawNet: Towards Non-stationary Time Series Forecasting with Balanced Modeling of Common and Specific Dependencies
SeesawNet dynamically balances common and instance-specific dependencies via ASNA in temporal and channel dimensions, outperforming prior methods on non-stationary forecasting benchmarks.
-
What if Tomorrow is the World Cup Final? Counterfactual Time Series Forecasting with Textual Conditions
Introduces the task of counterfactual time series forecasting with textual conditions plus a text-attribution mechanism that improves accuracy by distinguishing mutable from immutable factors.
-
Does Synthetic Data Help? Empirical Evidence from Deep Learning Time Series Forecasters
Synthetic data augmentation helps channel-mixing time series models but degrades channel-independent ones, with reliable gains only from seasonal-trend generators and gradual schedules in low-resource settings.
-
From Prediction to Practice: A Task-Aware Evaluation Framework for Blood Glucose Forecasting
A new evaluation framework shows that blood glucose forecasting models with high overall accuracy often fail at timely hypoglycemia detection in high-risk periods and at predicting effects of changed insulin doses.
-
From Observations to States: Latent Time Series Forecasting
LatentTSF improves time series forecasting accuracy and representation quality by shifting prediction from observation space to a learned latent state space via autoencoding.
-
Super-Linear: A Lightweight Pretrained Mixture of Linear Experts for Time Series Forecasting
Super-Linear introduces a pretrained MoE architecture using frequency-specialized linear experts and spectral gating for efficient general time series forecasting.
-
Sundial: A Family of Highly Capable Time Series Foundation Models
Sundial uses TimeFlow Loss for native pre-training of Transformers on continuous time series from TimeBench, achieving SOTA point and probabilistic forecasting with millisecond inference.
-
Deep Time Series Models: A Comprehensive Survey and Benchmark
This survey and benchmark of deep time series models using the released TSLib library finds that models with specific structures perform well only on distinct analysis tasks.
-
NEST: Tackling Dataset-Level Distribution Shifts via Regime-Oriented Mixture-of-Experts
NEST improves long-term multivariate forecasting under dataset-level distribution shifts by clustering regimes in moment-entropy space and recomposing specialized variate-attention experts via a content-plus-geometry router.
-
How Good Can Linear Models Be for Time-Series Forecasting?
Optimized Ridge regression with series-specific preprocessing beats prior linear forecasters and exceeds Transformer, MLP, and CNN baselines on six of eight time-series benchmarks.
-
MetaPS: Adaptive Programmatic Strategy Selection for Market Agents
MetaPS trains models via simulation rollouts to select from programmatic strategy libraries for market agents, yielding better performance than fixed or direct LLM baselines across model sizes.
-
LakeFM: Toward a Foundation Model for Aquatic Ecosystems Using Irregular Multivariate Multi-depth Time Series Data
LakeFM pre-trains on large ecological datasets to forecast irregular lake time series and reports competitive or superior performance with physically plausible outputs.
-
Mechanical Field Networks: Structured Neural Dynamics for Multivariate Systems
MF-Net learns a shared field state and mechanical transition rule from trajectories to deliver competitive forecasting and recoverable relation matrices on Lorenz-96 and real systems.
-
SPDM: Geometry-Modulated State Space Modeling with Manifold Constraints for Time Series Forecasting
SPDM is a geometry-aware state-space model that projects covariance matrices onto the SPD manifold tangent space and uses geometric gating to modulate SSM parameters for improved multivariate time series forecasting.
-
AME-TS: Anchored Mixture-of-Experts for Time Series Forecasting
AME-TS is a structure-guided sparse MoE foundation model for time series that aligns expert routing with series-level temporal descriptors to achieve strong accuracy-efficiency tradeoffs on GIFT-Eval while improving specialization stability.
-
DeRegiME: Deep Regime Mixtures for Probabilistic Forecasting under Distribution Shift
DeRegiME uses a sparse variational GP with nonstationary regime-mixing kernel to decompose forecasts into mean, residual regimes, and noise for improved probabilistic forecasting under distribution shift.
-
DAD4TS: Data-Augmentation-Oriented Diffusion Model for Time-Series Forecasting with Small-Scale Data
DAD4TS trains a diffusion-based generator jointly with a forecaster under RL control and geometric projections to produce augmentation samples that boost accuracy on small-scale time-series data, with validation reported on five of six real-world datasets.
-
How Do Electrocardiogram Models Scale?
Empirical scaling study of ECG models finds SSL scales robustly while ResNets show 1.3-2.5x better parameter efficiency and SSL up to 16x better data efficiency than supervised baselines on out-of-distribution tasks.
-
XDecomposer: Learning Prior-Free Set Decomposition for Multiphase X-ray Diffraction
XDecomposer uses set prediction and phase-query decomposition to jointly identify phases and reconstruct multiphase PXRD patterns without priors.
-
Exploring the Potential of Probabilistic Transformer for Time Series Modeling: A Report on the ST-PT Framework
ST-PT turns transformers into explicit factor graphs for time series, enabling structural injection of symbolic priors, per-sample conditional generation, and principled latent autoregressive forecasting via MFVI iterations.
-
GeoCert: Certified Geometric AI for Reliable Forecasting
GeoCert uses hyperbolic geometry to unify forecasting with physical reasoning and built-in formal certification, claiming major gains in accuracy and efficiency.
-
Earth System Foundation Model (ESFM): A unified framework for heterogeneous data integration and forecasting
ESFM is a single open foundation model that unifies heterogeneous Earth data sources and forecasts missing regions while preserving inter-variable physical relationships.
-
CSRA: Controlled Spectral Residual Augmentation for Robust Sepsis Prediction
CSRA applies input-adaptive spectral residual perturbations to multi-system ICU time series, trained end-to-end with the predictor and consistency losses, yielding 10.2% MSE and 3.7% MAE reductions on MIMIC-IV sepsis prediction.
-
M3R: Localized Rainfall Nowcasting with Meteorology-Informed MultiModal Attention
M3R improves localized rainfall nowcasting by using weather station time series as queries in multimodal attention to selectively extract precipitation patterns from radar imagery.
-
Frequency-aware Decomposition Learning for Sensorless Wrench Forecasting on a Vibration-rich Hydraulic Manipulator
FDN uses spectral decomposition, asymmetric heads for deterministic and probabilistic wrench components, and frequency-aware filtering to forecast high-frequency wrench from proprioception, outperforming baselines on hydraulic manipulator grinding data after pretraining and transfer.
-
MAP4TS: A Multi-Aspect Prompting Framework for Time-Series Forecasting with Large Language Models
MAP4TS combines global, local, statistical, and temporal prompts derived from classical time-series analysis with raw embeddings via cross-modality alignment to improve LLM forecasting performance across eight datasets.
-
ReNF: Rethinking the Design of Neural Long-Term Time Series Forecasters
ReNF proposes Boosted Direct Output (BDO) and parameter smoothing so a basic temporal MLP outperforms complex state-of-the-art models on long-term time series forecasting benchmarks by implicitly combining forecasts to reduce uncertainty.
-
Logo-LLM: Local and Global Modeling with Large Language Models for Time Series Forecasting
Logo-LLM improves time series forecasting by pulling local dynamics from shallow LLM layers and global trends from deeper layers, then aligning them via new Local-Mixer and Global-Mixer modules.
-
Hypergraph Multi-Modal Learning for EEG-based Emotion Recognition in Conversation
Hyper-MML integrates EEG, audio, and video using an Adaptive Brain Encoder with Mutual-cross Attention (ABEMA) and Adaptive Hypergraph Fusion Module (AHFM) to outperform prior methods on EAV and AFFEC datasets for conversational emotion recognition.
-
OrderFusion: Encoding Orderbook for End-to-End Probabilistic Intraday Electricity Price Forecasting
OrderFusion encodes orderbook buy-sell interactions in an end-to-end probabilistic model for intraday electricity price forecasting with non-crossing quantiles and reports consistent gains over baselines on European CID indices.
-
Neural equilibria for long-term prediction of nonlinear conservation laws
NeurDE learns the equilibrium closure within a kinetic solver to outperform larger neural models on long-term predictions of nonlinear conservation laws including shocks.
-
Titans: Learning to Memorize at Test Time
Titans combine attention for current context with a learnable neural memory for long-term history, achieving better performance and scaling to over 2M-token contexts on language, reasoning, genomics, and time-series tasks.
-
AutoPV: Automatically Design Your Photovoltaic Power Forecasting Model
AutoPV applies neural architecture search with a custom search space drawn from time series forecasting and photovoltaic models to automatically produce architectures that outperform predefined state-of-the-art models on a Chinese solar station dataset.
-
RhyMix: A Lightweight Adaptive Multi-Rhythm Network for Long-Term Time Series Forecasting
RhyMix reaches state-of-the-art long-term multivariate forecasting on 10 of 12 public benchmarks with a ~40K-parameter dual-path adaptive architecture of linear complexity.
-
CalM: A Self-Supervised Foundation Model for Population Dynamics in Calcium Imaging Data
A dual-axis autoregressive transformer pretrained on tokenized calcium traces transfers competitively to population forecasting and better than supervised models to behavior decoding.
-
Dualformer: Efficient Feature Extractor for Complex-valued Blind Communication Signal Analysis
Dualformer applies a parameter-sharing DualNN via Transformer patches to complex signals, claiming better results on AMR, SSR, and SSP tasks than baselines.
-
Pretrained Time-Series Foundation Models for Financial Return Forecasting
Pretrained TSFMs achieve top ranks on equity return tasks but show sparse, minimal improvements over random walk, serving as practical priors without reliable alpha generation.
-
VegSim: A Geospatial World Model for Scenario-Conditioned Vegetation Simulation
VegSim uses recurrent latent dynamics to enable both standard NDVI forecasting and user-controlled scenario simulation of vegetation from sparse satellite and weather inputs.
-
Step-adaptive multimodal fusion network with multi-scale cloud feature learning for ultra-short-term solar irradiance forecasting
A new multimodal fusion model combines InceptionNeXt cloud image features, a step-adaptive low-frequency compensation unit, and TempAttnLSTM to outperform prior methods on solar irradiance prediction tasks.
-
Online Irregular Multivariate Time Series Forecasting via Uncertainty-Driven Dual-Expert Calibration
Under-Cali is an uncertainty-driven dual-expert calibration framework for online adaptation in irregular multivariate time series forecasting that freezes the base model.
-
GenHAR: Generalizing Cross-domain Human Activity Recognition for Last-mile Delivery
GenHAR generalizes cross-domain human activity recognition by 9.97% accuracy and 6.4x lower FLOPs via tokenized sensor data, frequency channel correlations, selective masking, and efficient attention, with deployment detecting 2.15 billion activities.
-
Reasoning through Verifiable Forecast Actions: Consistency-Grounded RL for Financial LLMs
StockR1 unifies LLM-based financial reasoning and time-series forecasting by emitting verifiable forecast actions that condition a decoder, optimized via consistency-grounded RL to improve accuracy on QA and prediction tasks.
-
Discovering Entity-Conditioned Lag Heterogeneity: A Lag-Gated Neural Audit Framework for Panel Time Series
AC-GATE is a lag-gated neural encoder that conditions lag-weight distributions on entity proxies to recover heterogeneous lags as structural model outputs in panel time series.
-
Deep Learning Surrogates for Emulating Stochastic Climate Tipping Dynamics
A dynamics-informed Temporal Fusion Transformer surrogate emulates stochastic tipping events in global ocean transport simulations with 465x speedup and high-fidelity timing predictions.
-
Risk-Aware Safe Throughput Forecasting for Starlink Networks
BG-CFQS provides risk-aware quantile-based forecasting for Starlink throughput that meets overestimation budgets and reduces positive errors compared to other feasible methods.
-
A Market-Rule-Informed Neural Network for Efficient Imbalance Electricity Price Forecasting
A market-rule-informed neural network for imbalance electricity price forecasting matches generic deep learning accuracy while using substantially fewer parameters and less training time.
-
CombinationTS: A Modular Framework for Understanding Time-Series Forecasting Models
CombinationTS decomposes time-series models into modules and finds that good embeddings let simple identity encoders match complex ones, while input structural priors give better performance-stability trade-offs than complex encoders.