Pith. sign in

REVIEW 17 cited by

Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.10469 v1 pith:7HTTNDGS submitted 2024-10-14 cs.LG stat.ML

Xu Liu , Juncheng Liu , Gerald Woo , Taha Aksu , Yuxuan Liang , Roger Zimmermann , Chenghao Liu , Silvio Savarese
show 2 more authors
Caiming Xiong Doyen Sahoo
This is my paper · ORCID
classification cs.LGstat.ML
keywords seriestimefoundationfrequencymodelsmoirai-moepatternsspecialization
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Time series foundation models have demonstrated impressive performance as zero-shot forecasters. However, achieving effectively unified training on time series remains an open challenge. Existing approaches introduce some level of model specialization to account for the highly heterogeneous nature of time series data. For instance, Moirai pursues unified training by employing multiple input/output projection layers, each tailored to handle time series at a specific frequency. Similarly, TimesFM maintains a frequency embedding dictionary for this purpose. We identify two major drawbacks to this human-imposed frequency-level model specialization: (1) Frequency is not a reliable indicator of the underlying patterns in time series. For example, time series with different frequencies can display similar patterns, while those with the same frequency may exhibit varied patterns. (2) Non-stationarity is an inherent property of real-world time series, leading to varied distributions even within a short context window of a single time series. Frequency-level specialization is too coarse-grained to capture this level of diversity. To address these limitations, this paper introduces Moirai-MoE, using a single input/output projection layer while delegating the modeling of diverse time series patterns to the sparse mixture of experts (MoE) within Transformers. With these designs, Moirai-MoE reduces reliance on human-defined heuristics and enables automatic token-level specialization. Extensive experiments on 39 datasets demonstrate the superiority of Moirai-MoE over existing foundation models in both in-distribution and zero-shot scenarios. Furthermore, this study conducts comprehensive model analyses to explore the inner workings of time series MoE foundation models and provides valuable insights for future research.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 17 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Byte Pair Encoding for Efficient Time Series Forecasting

    cs.LG 2025-05 conditional novelty 7.0 of 10

    A byte-pair-encoding tokenizer that converts repeated temporal motifs into single tokens improves zero-shot forecasting accuracy and speed over sample-wise and patch-based methods.

  2. LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers

    cs.CL 2026-08 conditional novelty 6.0 of 10

    A new open-source library and benchmark, xRouteBench, evaluates LLM routers on a shared cost-aware protocol across text, memory, vision, time-series, and personalized tasks.

  3. Learning Spatio-Temporal Foundation Models from Pure Synthetic Data

    cs.LG 2026-06 conditional novelty 6.0 of 10

    A spatio-temporal foundation model pre-trained exclusively on synthetic stochastic graph dynamics outperforms real-data-pretrained STFMs in zero-shot traffic forecasting, according to the paper's benchmarks.

  4. Auditable Context-Aware HFMD Forecasting with Structured LLM Agents

    cs.LG 2025-11 conditional novelty 6.0 of 10

    An LLM-based two-agent system can forecast hand-foot-mouth disease cases with accuracy comparable to top numerical models while generating human-readable risk explanations.

  5. DIVER-0 : A Fully Channel Equivariant EEG Foundation Model

    eess.SP 2025-06 conditional novelty 6.0 of 10

    A channel-permutation-equivariant EEG transformer with full spatio-temporal attention achieves competitive BCI performance with only 10% of pretraining data.

  6. Time Series Representations for Classification Lie Hidden in Pretrained Vision Transformers

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Frozen vision transformers, applied to image representations of time series, produce classification features that outperform or match time series foundation models on UCR and UEA benchmarks.

  7. Mixture-of-Experts for Personalized and Semantic-Aware Next Location Prediction

    cs.AI 2025-05 conditional novelty 6.0 of 10

    NextLocMoE combines a location-semantics expert mixer and a persona-based expert mixer inside a trimmed LLaMA backbone to set new accuracy marks on next-location prediction benchmarks.

  8. BLAST: Balanced Sampling Time Series Corpus for Universal Forecasting Models

    cs.LG 2025-05 conditional novelty 6.0 of 10

    A balanced sampling strategy over statistically characterized time series patterns lets universal forecasting models train on 78 billion tokens instead of 419 billion, with equal or better zero-shot accuracy.

  9. Towards a Foundation Model for Communication Systems

    cs.AI 2025-05 conditional novelty 6.0 of 10

    A single pre-trained transformer can forecast and interpolate multiple wireless channel features (rank, precoder, Doppler, delay) on simulated 5G NR data.

  10. Learning Soft Sparse Shapes for Efficient Time-Series Classification

    cs.LG 2025-05 conditional novelty 6.0 of 10

    A new soft-sparsifying shapelet model with expert-based temporal learning reports top mean accuracy on the 128-dataset UCR time series benchmark.

  11. Attention Mechanism, Max-Affine Partition, and Universal Approximation

    cs.LG 2025-04 conditional novelty 6.0 of 10

    A single-head attention layer with a linear preprocessing layer is a universal approximator for continuous functions on compact domains, in both L∞ and Lp norms, for both self- and cross-attention.

  12. Investigating Compositional Reasoning in Time Series Foundation Models

    cs.LG 2025-02 conditional novelty 6.0 of 10

    On a benchmark where models train on Fourier components and test on their sums, patch-based Transformers and residual MLP architectures show the strongest compositional generalization, while most standard transformers...

  13. Ground-Truth Neighborhood Regularization for Reinforcement Learning Post-Training of Time Series Foundation Models

    cs.LG 2026-08 conditional novelty 5.0 of 10

    Reward-based fine-tuning of time series foundation models can collapse predictions away from the true future; steering probability mass into a ground-truth neighborhood reduces that collapse and improves forecasts.

  14. Hopformer: Homogeneity-Pursuit Transformer for Time Series Forecasting

    stat.ML 2026-07 reject novelty 5.0 of 10

    A two-stage forecaster (SPA trend extraction + LoRA-fine-tuned residual Transformer) that the paper claims beats prior models by 6.56% MASE, though the claim is not robust to its own extended baseline tables.

  15. AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting

    stat.ML 2025-02 conditional novelty 5.0 of 10

    Adapters that map multivariate time series into a latent space let a frozen univariate foundation model produce probabilistic multivariate forecasts, improving MSE on most tested tasks.

  16. N-BEATS-MOE: N-BEATS with a Mixture-of-Experts Layer for Heterogeneous Time Series Forecasting

    cs.LG 2025-08 conditional novelty 4.0 of 10

    Adding a gating network on top of N-BEATS block outputs gives modest SMAPE improvements on some heterogeneous benchmark series, but the gains are small and not statistically validated.

  17. Comparative Analysis of Time Series Foundation Models for Demographic Forecasting: Enhancing Predictive Accuracy in US Population Dynamics

    cs.LG 2025-08 reject novelty 3.0 of 10

    TimesFM achieved the lowest MSE on 13 of 15 US state-race population forecast tasks, but the paper fine-tuned it for 50 epochs per state despite claiming no task-specific fine-tuning.

Pith tools