REVIEW 6 cited by
Ti-MAE: Self-Supervised Masked Time Series Autoencoders
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Multivariate Time Series forecasting has been an increasingly popular topic in various applications and scenarios. Recently, contrastive learning and Transformer-based models have achieved good performance in many long-term series forecasting tasks. However, there are still several issues in existing methods. First, the training paradigm of contrastive learning and downstream prediction tasks are inconsistent, leading to inaccurate prediction results. Second, existing Transformer-based models which resort to similar patterns in historical time series data for predicting future values generally induce severe distribution shift problems, and do not fully leverage the sequence information compared to self-supervised methods. To address these issues, we propose a novel framework named Ti-MAE, in which the input time series are assumed to follow an integrate distribution. In detail, Ti-MAE randomly masks out embedded time series data and learns an autoencoder to reconstruct them at the point-level. Ti-MAE adopts mask modeling (rather than contrastive learning) as the auxiliary task and bridges the connection between existing representation learning and generative Transformer-based methods, reducing the difference between upstream and downstream forecasting tasks while maintaining the utilization of original time series data. Experiments on several public real-world datasets demonstrate that our framework of masked autoencoding could learn strong representations directly from the raw data, yielding better performance in time series forecasting and classification tasks.
Forward citations
Cited by 6 Pith papers
-
Self-Distillation of Hidden Layers for Self-Supervised Representation Learning
Predicting the outputs of several hidden layers of an EMA teacher, rather than only the final layer or pixels, substantially improves self-supervised ViT representations on ImageNet and downstream tasks.
-
Masked Autoencoders for Ultrasound Signals: Robust Representation Learning for Downstream Applications
MAE pre-training on synthetic ultrasound signals transfers to real measured signals and beats from-scratch and CNN baselines on time-of-flight classification, with the biggest gains in low-label regimes.
-
Merlin: Multi-View Representation Learning for Robust Multivariate Time Series Forecasting with Unfixed Missing Rates
A knowledge-distillation plus multi-view contrastive training scheme makes multivariate time-series forecasters robust to unfixed missing rates using a single model.
-
ST-MTM: Masked Time Series Modeling with Seasonal-Trend Decomposition for Time Series Forecasting
ST-MTM shows that masking seasonal and trend components separately, with a contrastive alignment loss, improves self-supervised time series forecasting.
-
LSM-2: Learning from Incomplete Wearable Sensor Data
LSM-2 with Adaptive and Inherited Masking learns usable representations directly from incomplete day-long wearable data, outperforming imputation-based baselines on most tasks.
-
Farm-Level, In-Season Crop Identification for India
A Google DeepMind team built a transformer-based system that maps 12 crops across India at farm level, in-season, with state-level area agreement of 94% (winter) and 75% (monsoon) against the 2023-24 census.
Discussion (0). Continue with ORCID to comment.