pith. sign in

arxiv: 2508.12247 · v3 · pith:7RAUPXFDnew · submitted 2025-08-17 · 💻 cs.LG · cs.AI

STM3: Mixture of Multiscale Mamba for Long-Term Spatio-Temporal Time-Series Prediction

Haolong Chen , Liang Zhang , Zhengyuan Xin , Guangxu Zhu This is my paper

Pith reviewed 2026-05-21 22:12 UTC · model grok-4.3

classification 💻 cs.LG cs.AI
keywords spatio-temporal predictionmixture of expertsMambamultiscale modelingtime series forecastinggraph causal networkdisentangled learninglong-term dependencies
0
0 comments X

The pith

STM3 integrates multiscale Mamba inside a disentangled mixture-of-experts framework to model long-term spatio-temporal time series dependencies more efficiently.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

STM3 addresses the difficulty of efficiently extracting multiscale information from long temporal sequences and modeling their correlations across nodes in spatio-temporal time series. The model places a Multiscale Mamba architecture inside a Disentangled Mixture-of-Experts framework and pairs it with an adaptive graph causal network for spatial relations. Stable routing and causal contrastive learning ensure each expert learns distinct patterns, which the authors prove leads to smoother routing and better disentanglement, resulting in state-of-the-art accuracy on multiple prediction benchmarks.

Core claim

STM3 integrates a Multiscale Mamba architecture within a novel Disentangled Mixture-of-Experts (DMoE) framework to capture diverse multiscale information efficiently, while utilizing an adaptive graph causal network to model complex spatial dependencies. To ensure robust representation learning, a stable routing strategy and a causal contrastive learning strategy work with hierarchical information aggregation to guarantee scale distinguishability. The authors theoretically prove that STM3 achieves superior routing smoothness and guarantees pattern disentanglement for each expert, delivering state-of-the-art results on 10 real-world benchmarks including a 7.1% MAE, 8.5% RMSE, and 15.9% MAPE提升

What carries the argument

Disentangled Mixture-of-Experts (DMoE) framework with embedded Multiscale Mamba architecture and adaptive graph causal network, which disentangles multiscale temporal patterns through expert specialization and hierarchical aggregation.

If this is right

  • Efficient extraction of multiscale temporal information from long sequences without quadratic scaling costs.
  • Effective modeling of highly correlated multiscale information across different spatial nodes via the graph causal network.
  • Guaranteed scale distinguishability and expert specialization through the combination of stable routing and causal contrastive learning.
  • State-of-the-art empirical results across 10 diverse real-world spatio-temporal benchmarks.
  • Theoretical guarantees on routing smoothness that support reliable expert assignment during inference.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The disentanglement approach could transfer to other mixture-of-experts architectures in sequential domains such as video or sensor forecasting.
  • The causal contrastive component may improve interpretability of expert specialization in long-horizon prediction tasks.
  • Hybridizing the multiscale Mamba backbone with additional graph layers could extend applicability to even denser spatial graphs.
  • The efficiency gains from Mamba may allow deployment on resource-constrained edge devices for real-time spatio-temporal monitoring.

Load-bearing premise

The stable routing strategy together with causal contrastive learning is assumed to guarantee both routing smoothness and pattern disentanglement for each expert.

What would settle it

Ablation experiments on PEMSD8 showing no measurable gain in MAE, RMSE, or MAPE when the stable routing or causal contrastive learning modules are removed would falsify the central performance and disentanglement claims.

Figures

Figures reproduced from arXiv: 2508.12247 by Guangxu Zhu, Haolong Chen, Liang Zhang, Zhengyuan Xin.

Figure 1
Figure 1. Figure 1: Main structure of STM3. where ℎ (𝑞) ms ∈ R 𝑇 ×𝑑inner and ℎ (𝑞) ∈ R 𝑇 ×𝑑inner denote the input and output feature sequences at scale 𝑞. We then stack the out￾puts to obtain ℎ ∈ R 𝑇 ×𝑑inner×𝑄 , with symbols consistent with Sec￾tion 4.2. Through scale amplification, the maximum scale expands to 𝑠 (𝑄) 0 [𝑠 (𝑄) ] 𝐿 , where 𝐿 denotes the layer index of the backbone where the multiscale Mamba module is deployed, … view at source ↗
Figure 2
Figure 2. Figure 2: The comparison between two routing strategies. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Ablation study of STM3. for optimal spatio-temporal time-series prediction. More ablation study results are detailed in Appendix D.1 5.4 Hyperparameter Study (RQ3) As shown in [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 6
Figure 6. Figure 6: STM3’s multiscale feature extraction. (a) Expert assignment. (b) Loss [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Comparison of routing strategies. 5.5 In-Depth Analysis (RQ4 & RQ5) Expert-Wise Effectiveness. To validate MMM’s expert-wise ef￾fectiveness to model complex spatio-temporal patterns, we visual￾ized STM3’s first-layer features using t-SNE [40]. Figure 5a shows distinct feature clusters for each expert, confirming effective pat￾tern disentanglement. Figure 5b further illustrates the gating net￾work’s discrim… view at source ↗
Figure 5
Figure 5. Figure 5: MMM’s feature extraction across experts. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 8
Figure 8. Figure 8: Ablation study of STM3 [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Hyperparameter analysis of STM3 [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗
read the original abstract

Recently, spatio-temporal time-series prediction has developed rapidly, yet existing deep learning methods struggle with learning complex long-term spatio-temporal dependencies efficiently. The long-term spatio-temporal dependency learning brings two new challenges: 1) The long-term temporal sequence naturally includes multiscale information, which is hard to extract efficiently; 2) The multiscale temporal information from different nodes is highly correlated and hard to model. To address these challenges, we propose Spatio-Temporal Mixture of Multiscale Mamba (STM3). STM3 integrates a Multiscale Mamba architecture within a novel Disentangled Mixture-of-Experts (DMoE) framework to capture diverse multiscale information efficiently, while utilizing an adaptive graph causal network to model complex spatial dependencies. To ensure robust representation learning, we introduce a stable routing strategy and a causal contrastive learning strategy, which work in tandem with hierarchical information aggregation to guarantee scale distinguishability. We theoretically prove that STM3 achieves superior routing smoothness and guarantees pattern disentanglement for each expert. Extensive experiments on 10 real-world benchmarks across domains demonstrate STM3's superior performance, achieving state-of-the-art results in long-term spatio-temporal time-series prediction. Notably, on the PEMSD8 dataset, it achieves significant improvements, surpassing the second-best model by 7.1% in MAE, 8.5% in RMSE, and 15.9% in MAPE. Code is available at https://github.com/IfReasonable/STM3_KDD26.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The manuscript proposes STM3, which integrates a Multiscale Mamba architecture inside a Disentangled Mixture-of-Experts (DMoE) framework together with an adaptive graph causal network, a stable routing strategy, and causal contrastive learning. The central claims are that these components efficiently capture multiscale long-term spatio-temporal dependencies, that the authors provide theoretical proofs of superior routing smoothness and pattern disentanglement for each expert, and that the model achieves state-of-the-art results on ten real-world benchmarks, including a 7.1% MAE, 8.5% RMSE, and 15.9% MAPE improvement over the second-best model on PEMSD8.

Significance. If the theoretical guarantees on routing smoothness and expert disentanglement can be verified and the reported gains are shown to be robust via ablations and statistical reporting, the work would offer a scalable Mamba-based approach for long-horizon spatio-temporal forecasting. The public release of code at the cited GitHub repository strengthens reproducibility.

major comments (3)
  1. [Theoretical Analysis] Theoretical Analysis section: the proof that stable routing plus causal contrastive learning guarantees both routing smoothness and pattern disentanglement for each expert is presented as load-bearing for attributing the observed gains to the DMoE mechanisms rather than increased capacity or standard Mamba scaling; however, the derivation relies on unverified assumptions about how these components interact with multiscale inputs under realistic spatio-temporal correlations, and no empirical validation of those assumptions is provided.
  2. [Experiments] Experiments section, results tables (e.g., PEMSD8 row): the reported improvements (7.1% MAE, 8.5% RMSE, 15.9% MAPE) are given without error bars, standard deviations from multiple random seeds, or statistical significance tests, which is required to establish that the gains are reliable rather than artifacts of a single run.
  3. [Ablation studies] Ablation studies subsection: the manuscript lacks detailed ablations that isolate the contribution of the stable routing strategy and causal contrastive loss from the base Multiscale Mamba and DMoE components; without these, the central claim that the proposed mechanisms are responsible for the performance edge cannot be substantiated.
minor comments (2)
  1. [Methodology] Figure captions for the model architecture diagram could more explicitly label the hierarchical information aggregation and the flow of the causal contrastive loss.
  2. [Experiments] Ensure that all baseline methods in the experimental tables include their original publication references and hyper-parameter settings used for fair comparison.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the detailed and constructive feedback. We address each major comment below and outline the revisions we will make to strengthen the manuscript, particularly around theoretical validation, statistical reporting, and ablation depth.

read point-by-point responses

  1. Referee: [Theoretical Analysis] Theoretical Analysis section: the proof that stable routing plus causal contrastive learning guarantees both routing smoothness and pattern disentanglement for each expert is presented as load-bearing for attributing the observed gains to the DMoE mechanisms rather than increased capacity or standard Mamba scaling; however, the derivation relies on unverified assumptions about how these components interact with multiscale inputs under realistic spatio-temporal correlations, and no empirical validation of those assumptions is provided.

    Authors: We agree that linking the theoretical guarantees more explicitly to empirical behavior strengthens the attribution of gains to the proposed mechanisms. The proofs rely on standard MoE assumptions about input separability and correlation structure, which align with the multiscale spatio-temporal setting in our model design. In the revision we will add a new subsection with empirical validation: correlation heatmaps across scales on the benchmark datasets and a sensitivity study showing how routing smoothness and expert specialization respond to controlled changes in multiscale correlation strength. This will make the assumptions verifiable without altering the core proofs. revision: yes

  2. Referee: [Experiments] Experiments section, results tables (e.g., PEMSD8 row): the reported improvements (7.1% MAE, 8.5% RMSE, 15.9% MAPE) are given without error bars, standard deviations from multiple random seeds, or statistical significance tests, which is required to establish that the gains are reliable rather than artifacts of a single run.

    Authors: We concur that single-run results limit confidence in the reported margins. We will re-execute all experiments using five independent random seeds, report mean ± standard deviation for every metric and dataset, and add paired t-test p-values (with Bonferroni correction) comparing STM3 against the second-best baseline on the primary benchmarks including PEMSD8. These additions will appear in the updated tables and a new statistical analysis paragraph. revision: yes

  3. Referee: [Ablation studies] Ablation studies subsection: the manuscript lacks detailed ablations that isolate the contribution of the stable routing strategy and causal contrastive loss from the base Multiscale Mamba and DMoE components; without these, the central claim that the proposed mechanisms are responsible for the performance edge cannot be substantiated.

    Authors: We accept that finer-grained isolation is needed to substantiate the contribution of each new component. We will expand the ablation section with three additional controlled variants on all ten benchmarks: (i) Multiscale Mamba + DMoE without stable routing, (ii) Multiscale Mamba + DMoE with stable routing but without causal contrastive loss, and (iii) the full STM3 model. Performance deltas and routing statistics will be reported to quantify the incremental benefit of each element while holding model capacity fixed. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected in derivation chain

full rationale

The paper introduces STM3 with a Multiscale Mamba inside a Disentangled Mixture-of-Experts (DMoE) framework, plus stable routing and causal contrastive learning. It claims to theoretically prove superior routing smoothness and pattern disentanglement, but these proofs are presented as internal derivations rather than reductions to fitted parameters or prior self-citations. Performance improvements are reported as empirical results on 10 benchmarks (e.g., PEMSD8 gains), not as predictions forced by construction from inputs. No equation or claim reduces the SOTA attribution directly to a hyper-parameter fit or renames a known result via new coordinates. The central mechanisms are motivated by stated challenges in long-term spatio-temporal dependencies and are not shown to be equivalent to their inputs by the paper's own text. This is a self-contained architectural proposal with independent empirical validation.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 1 invented entities

The model rests on the prior Mamba sequence model, standard graph neural network assumptions, and several newly introduced mechanisms whose effectiveness is asserted rather than derived from first principles.

free parameters (1)
  • expert routing temperature and contrastive loss weight
    Hyper-parameters that control the stable routing and disentanglement objectives are chosen during training.
axioms (1)
  • domain assumption Mamba blocks can capture long-range temporal dependencies at multiple scales when stacked appropriately
    Invoked when the multiscale Mamba architecture is introduced.
invented entities (1)
  • Disentangled Mixture-of-Experts (DMoE) with stable routing no independent evidence
    purpose: To separate multiscale temporal patterns across experts
    New component introduced to address the correlation challenge stated in the abstract.

pith-pipeline@v0.9.0 · 5808 in / 1310 out tokens · 58636 ms · 2026-05-21T22:12:19.236360+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. PIMSM: Physics-Informed Multi-Scale Mamba for Stable Neural Representations under Distribution Shift

    cs.LG 2026-05 unverdicted novelty 6.0

    PIMSM is a Mamba-based architecture that maps knee frequencies from spectra to multi-scale discretization parameters to reduce representation drift under distribution shifts in fMRI and weather forecasting.