REVIEW 19 cited by
Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts
read the original abstract
Time series foundation models have demonstrated impressive performance as zero-shot forecasters. However, achieving effectively unified training on time series remains an open challenge. Existing approaches introduce some level of model specialization to account for the highly heterogeneous nature of time series data. For instance, Moirai pursues unified training by employing multiple input/output projection layers, each tailored to handle time series at a specific frequency. Similarly, TimesFM maintains a frequency embedding dictionary for this purpose. We identify two major drawbacks to this human-imposed frequency-level model specialization: (1) Frequency is not a reliable indicator of the underlying patterns in time series. For example, time series with different frequencies can display similar patterns, while those with the same frequency may exhibit varied patterns. (2) Non-stationarity is an inherent property of real-world time series, leading to varied distributions even within a short context window of a single time series. Frequency-level specialization is too coarse-grained to capture this level of diversity. To address these limitations, this paper introduces Moirai-MoE, using a single input/output projection layer while delegating the modeling of diverse time series patterns to the sparse mixture of experts (MoE) within Transformers. With these designs, Moirai-MoE reduces reliance on human-defined heuristics and enables automatic token-level specialization. Extensive experiments on 39 datasets demonstrate the superiority of Moirai-MoE over existing foundation models in both in-distribution and zero-shot scenarios. Furthermore, this study conducts comprehensive model analyses to explore the inner workings of time series MoE foundation models and provides valuable insights for future research.
Forward citations
Cited by 19 Pith papers
-
Empowering VLMs for Few-Shot Multimodal Time Series Classification via Tailored Agentic Reasoning
MarsTSC is a VLM-based agentic reasoning framework with a self-evolving knowledge bank and Generator-Reflector-Modifier roles that achieves better few-shot multimodal time series classification than baselines on 12 be...
-
TempusBench: An Evaluation Framework for Time-Series Forecasting
TempusBench is a new evaluation framework for time-series forecasting models that supplies fresh non-overlapping datasets, tasks beyond horizon and domain, consistent tuning across models, and visualization tools.
-
Discrete Prototypical Memories for Federated Time Series Foundation Models
FeDPM learns and aligns local discrete prototypical memories across domains to create a unified discrete latent space for LLM-based time series foundation models in a federated setting.
-
TS-Reasoner: Domain-Oriented Time Series Inference Agents for Reasoning and Automated Analysis
TS-Reasoner is a domain-oriented agent using LLMs, computational tools, and error feedback for multi-step time series inference, showing better performance than general LLMs on understanding and reasoning benchmarks.
-
Probabilistic Low-Voltage Peak Load Forecasting with Time Series Foundation Models Evaluated on Application-Oriented Metrics
Compares foundation models for probabilistic low-voltage load forecasting on 200 real feeders and introduces a grid-planning metric that scores peak prediction by its effect on asset cost-risk decisions.
-
Learning Spatio-Temporal Foundation Models from Pure Synthetic Data
A spatio-temporal foundation model pre-trained exclusively on synthetic stochastic graph dynamics outperforms real-data-pretrained STFMs in zero-shot traffic forecasting, according to the paper's benchmarks.
-
CausalMoE: A Billion-Scale Multimodal Foundation Model for Granger Causal Discovery with Pattern-Routed Heterogeneous Experts
CausalMoE is a multimodal foundation model with pattern-routed heterogeneous experts and LLM/VLM integration that claims new SOTA performance on supervised and few-shot Granger causal discovery benchmarks.
-
TimeRouter: Efficient and Adaptive Routing of Time-Series Foundation Models
TimeRouter routes among time-series foundation models via discriminative routing, selective gating and ensemble fallback, reporting SOTA LB MASE 0.6765 on GIFT-EVAL.
-
AME-TS: Anchored Mixture-of-Experts for Time Series Forecasting
AME-TS is a structure-guided sparse MoE foundation model for time series that aligns expert routing with series-level temporal descriptors to achieve strong accuracy-efficiency tradeoffs on GIFT-Eval while improving s...
-
CTF4Nuclear: Common Task Framework for Nuclear Fission and Fusion Models
CTF4Nuclear proposes a common task framework for benchmarking ML methods on nuclear engineering datasets using 12 metrics and a new sparse-measurement system monitoring paradigm.
-
Fast Training of Mixture-of-Experts for Time Series Forecasting via Expert Loss Integration
A new MoE training method integrates expert-level losses and partial online updates to improve forecasting accuracy and efficiency over standard statistical and neural models.
-
Empowering VLMs for Few-Shot Multimodal Time Series Classification via Tailored Agentic Reasoning
MarsTSC is a VLM agentic system with generator, reflector, and modifier roles that iteratively refines a knowledge bank to improve few-shot multimodal time series classification and produce human-readable explanations.
-
Sonata: A Hybrid World Model for Inertial Kinematics under Clinical Data Scarcity
Sonata is a small hybrid world model pre-trained to predict future IMU states that outperforms autoregressive baselines on clinical discrimination, fall-risk prediction, and cross-cohort transfer while fitting on-devi...
-
Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scaling
Timer-S1 is a released 8.3B-parameter MoE time series model that achieves state-of-the-art MASE and CRPS scores on GIFT-Eval using serial scaling and Serial-Token Prediction.
-
Auditable Context-Aware HFMD Forecasting with Structured LLM Agents
An LLM-based two-agent system can forecast hand-foot-mouth disease cases with accuracy comparable to top numerical models while generating human-readable risk explanations.
-
MoveFM-R: Advancing Mobility Foundation Models via Language-driven Semantic Reasoning
MoveFM-R is a framework that bridges mobility foundation models and LLMs using semantically enhanced location encoding, progressive curriculum alignment, and interactive self-reflection to generate plausible trajector...
-
Hopformer: Homogeneity-Pursuit Transformer for Time Series Forecasting
A two-stage forecaster (SPA trend extraction + LoRA-fine-tuned residual Transformer) that the paper claims beats prior models by 6.56% MASE, though the claim is not robust to its own extended baseline tables.
-
Towards Intrusion Detection Systems for RPL-based IoT Networks using Foundation Models
Fine-tuning the MOMENT foundation model on simulated RPL statistics enables multi-class attack identification with performance comparable to state-of-the-art methods.
-
Empirical Assessment of Time-Series Foundation Models For Power System Forecasting Applications
The paper benchmarks foundation models like TimesFM and Chronos against baselines on eight forecasting capabilities for power system time series.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.