REVIEW 5 major objections 5 minor 37 references
FTMoMamba: Motion Generation with Frequency and Text State Space Models
T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read The paper claims that a Mamba-based diffusion denoiser reaches the lowest text-to-motion FID on HumanML3D, 0.181, by putting wavelet frequency features in the A matrix and sentence-level text features in the C matrix of the state space…
desk verdict A strong FID number with a plausible but underspecified mechanism; the core frequency-in-A / text-in-C equations don't type-check as written and there's no code, so the result is not yet reproducible. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is FreqSSM plus TextSSM inside a Mamba-based diffusion denoiser. Mamba is a linear-time state space sequence model whose discrete update is $h_t = \bar{A} h_{t-1} + \bar{B} x_t$ and whose output is $y_t = C h_t$; the paper's central move is to treat the learnable matrices A and C as conditioning channels rather than fixed parameters. FreqSSM decomposes the motion feature with a discrete wavelet transform, enhances the low and high bands with convolution, then adds them to A with learnable weights so the transition dynamics carry frequency-specific control. TextSSM sums the sentence-level CLIP embedding with C, so the observation mapping carries text-motion alignment. These two modified SSMs are embedded in FreqMamba and TextMamba blocks, which form the denoiser of a latent diffusion model trained to predict noise in the VAE latent space.
What would settle it
If a run of the released checkpoints on HumanML3D does not reproduce a FID near 0.181, or if replacing the A-matrix addition in Equation (9) with an ordinary residual connection to the hidden state leaves FID statistically unchanged, then the frequency-in-the-A-matrix mechanism is not what carries the reported gain.
Extended reading notes
Core claim
The central discovery is that the A and C matrices of a Mamba state space model are the right places to inject two kinds of conditioning. Equation (9) constructs a frequency-aware transition matrix $A_n = A + \alpha f_{\text{low}} + \beta f_{\text{high}}$, where $f_{\text{low}}$ and $f_{\text{high}}$ come from applying a discrete wavelet transform to the motion feature, so the low-frequency band guides static poses and the high-frequency band guides fine-grained motion. Equation (15) constructs a sentence-level output matrix $C_s = f_t + C$, where $f_t$ is the frozen CLIP sentence embedding, so textual semantics are aligned with the motion state at every output step. The paper claims that this dual injection yields the lowest FID among compared methods on HumanML3D at 0.181, with the ablations showing that DWT beats FFT for the frequency split, that both frequency bands contribute, and that the two SSM modules are complementary.
Load-bearing premise
The load-bearing premise is that low- and high-frequency wavelet features can be added directly into the state transition matrix A and the sentence text embedding directly into the output matrix C of the Mamba layer, with dimensions that align so the state equations still hold; the paper does not specify the projection.
Editorial extensions
If this is right
- If the reported FID holds, Mamba-based denoisers can outperform Transformer-based motion diffusion baselines without extra cross-attention branches, because conditioning is folded into the SSM matrices themselves.
- The low-frequency/high-frequency split gives an interpretable control handle: low-frequency bands steer static poses and high-frequency bands steer transitions, which could make motion editing more targeted.
- Injecting sentence-level text into the C matrix aligns text and motion at every state update, which should reduce the mismatch that plain concatenation of text and motion features leaves behind.
- The ablation showing DWT beats FFT supports the claim that separating low and high frequency bands matters more than having a global frequency spectrum.
- Using both frequency bands together lowers FID compared with either band alone, indicating that static-pose and fine-motion information are complementary rather than redundant.
Reading between the lines
- A testable extension outside this paper is to apply the same A-matrix injection to other Mamba-based generative models, such as video or audio diffusion, where static background versus transient events mirrors static pose versus fine motion.
- One could ablate the design by replacing the sentence-level text vector with token-level CLIP features; the paper's mechanism depends on a single embedding $f_t$, so this would clarify whether sentence-level aggregation is essential.
- The dimensional alignment between the wavelet features and the square matrix A, and between the text embedding and the output matrix C, is not specified in the paper; a clean reproduction would need a projection, and checking the released code would settle whether Equations (9) and (15) run as written.
- A controlled study that keeps the denoiser capacity fixed and toggles only the matrix injections on and off would isolate whether the mechanism itself, rather than the overall architecture, carries the FID gain.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FTMoMamba, a latent diffusion framework for text-to-motion generation that augments a Mamba-based denoiser with two modules: FreqSSM, which injects low- and high-frequency features (obtained via DWT) into the SSM state-transition matrix A, and TextSSM, which injects a CLIP sentence embedding into the SSM output matrix C. The authors report the lowest FID (0.181) on HumanML3D among the compared methods, alongside ablations of the two modules and of frequency-extraction choices. The central claim is that frequency information in A improves static-pose and fine-grained motion generation, and that sentence-level text in C improves text-motion alignment.
Significance. If the reported FID result is reproducible and the FreqSSM/TextSSM mechanisms operate as stated, the work would be of interest to the motion-generation community as a lightweight Mamba-based alternative to transformer denoisers. The paper includes standard benchmark comparisons, ablations, and qualitative visualizations, and it explicitly reports confidence intervals for all metrics. However, the load-bearing equations for the two core modules are dimensionally ambiguous, the ablation narrative contradicts the reported table values, and several numerical claims in the text do not match the tables. The manuscript's contribution, as written, is therefore not fully verifiable and will require substantial revision before its claims can be assessed.
major comments (5)
- [§3.2, Eqs. (8)-(9) and (15)] The additions in Eq. (9) and Eq. (15) are dimensionally undefined. In a standard Mamba layer A is a parameter of shape (D,N) or (N,), while DWT applied along the sequence length of a latent feature f_m^mo yields f_low and f_high of length L/2 with channel dimension equal to the hidden width; these cannot be added to A without a projection or reshaping rule. Similarly, in Eq. (15) the text embedding f_t ∈ R^{1×256} is added to C, which is either a parameter or a (B,L,N) tensor; no broadcasting or projection is specified. The paper provides no code or implementation details to resolve the discrepancy. Since Eqs. (8)-(9) and (15) are the exact sites of the claimed 'frequency into A, text into C' mechanism, the mechanism as stated is not implementable, and the FID gain cannot be attributed to FreqSSM/TextSSM as described.
- [§3.2, Eqs. (7), (11), (14), (17)-(18)] The notation for the output features is inconsistent. f_m^no is defined in Eq. (7) as the gated output of the FreqMamba branch, but Eq. (11) also defines f_m^no as the FreqSSM output before the linear/gating stage; likewise f_m^vo is defined in both Eq. (14) and Eq. (17), and Eq. (18) then uses both symbols. This makes the forward pass ambiguous: it is unclear whether the final feature z_t' includes the linear/gated outputs, the raw SSM outputs, or both. The equations need to be aligned or renamed before the architecture can be reproduced.
- [§4.5, Table 2] The ablation discussion is inconsistent with Table 2. The text states that 'TextMamba outperforms FreqMamba in terms of lower FID and greater diversity,' but the row with FID 0.209 (which presumably corresponds to FreqMamba-only, given the sentence order in the paragraph) has both lower FID and higher diversity than the row with FID 0.227 (presumably TextMamba-only). The table also does not show which module is enabled in each row because the checkmark symbols are missing from the FreqMamba/TextMamba columns. The comparison and its interpretation need to be corrected or explicitly labeled.
- [§4.4 and §4.5] Several percentage claims do not match the tables. In §4.4, the FID is said to be 'reduced by 5.1%' compared to MotionGPT, but Table 1 gives (0.232−0.181)/0.232 ≈ 22%. In §4.5, the combined low/high-frequency configuration is said to decrease FID 'by 2.8% relative to the baseline,' but Table 4 (and Table 2, if row 1 is the baseline) gives a reduction from 0.209/0.263 to 0.181, i.e., roughly 13%–31%. The authors should report correct percentages or clarify what baseline is being used.
- [§1, §3.2] The central assumption that 'low frequencies correlate with static poses, and high frequencies align with fine-grained motions' is asserted without direct evidence, and the related claim that the SSM matrix A 'is related to the frequency domain poles' is only supported by informal citations. No spectral analysis, frequency-band ablation against motion categories, or theoretical argument is provided. This weakens the design rationale for Eq. (9); a concrete test, such as visualizing the frequency content of static versus transitional motion segments or ablating with inverted band assignments, would help substantiate the claim.
minor comments (5)
- [§4.1] The section heading 'Expreiments' should be 'Experiments'.
- [§4.3] In the implementation details, '50 during interfering' should presumably read '50 during inference.'
- [§4.5] The sentence 'When FreqMamba is , use BiMamba instead; When TextMamba is , apply Linear Attention instead' has missing symbols (likely checkmark/cross marks), making the ablation design unclear. These should be filled in or described in prose.
- [Table 2] The table header interleaves 'R-Precision ↑' with the module indicator columns, and the column labels for FID and R-Precision are not cleanly separated; this makes the table hard to read. The module-enabled indicators should be explicitly shown.
- [§4.4] The claim that FTMoMamba 'achieves significant improvements over the baseline method (i.e., MLD) in terms of R-Precision, FID, MM-Dist, Diversity, and MModality' is not fully supported by Table 1: MModality degrades from 2.600 (MLD) to 2.277 (FTMoMamba). The sentence should be revised to acknowledge this metric.
Circularity Check
No significant circularity: the central FID result is an external benchmark measurement, not a consequence of the paper's definitions; only minor non-load-bearing self-citations appear.
full rationale
FTMoMamba is an empirical architecture paper: the headline claim is a measured FID of 0.181 on the HumanML3D test set, obtained by training a diffusion denoiser and evaluating against ground-truth motions. This result is not derived from Equations (8)-(17) in a way that makes the conclusion equivalent to the premises. FreqSSM and TextSSM are design choices that inject frequency features into the A matrix and text features into the C matrix; the paper then tests whether those choices improve FID, R-Precision, and related metrics. That is a normal hypothesis-and-evaluation loop, not a self-fulfilling definition. The learnable parameters alpha and beta in Eq. (9) are trained and their effect is measured on held-out test data, so they are not fitted values renamed as predictions. The only self-citations are [19] (CDWConv component by the first author) and [6] (related work on egocentric action recognition); neither is load-bearing for the central claim, and no uniqueness theorem or ansatz is imported from those citations. The dimensional ambiguity in Eqs. (9) and (15) is a verifiability/correctness concern about whether the implementation matches the equations, not a circularity. Therefore, no significant circularity is present.
Assumptions & free parameters
free parameters (2)
- alpha =
learned during training
- beta =
learned during training
assumptions (4)
- domain assumption Low-frequency components of latent motion correspond to static poses and high-frequency components to fine-grained motion.
- ad hoc to paper The SSM state transition matrix A is related to frequency-domain poles, so adding frequency features to A controls frequency behavior.
- domain assumption Discrete wavelet transform decomposition of latent motion sequences is meaningful in the MLD latent space.
- ad hoc to paper Adding the CLIP sentence embedding to the output matrix C aligns text and motion semantics.
Cite this review
Pith. "Pith review of FTMoMamba: Motion Generation with Frequency and Text State Space Models." pith.science (2026). https://pith.science/paper/AATYMNZO
@misc{pith2026241117532,
author = {Pith},
title = {Pith review of: FTMoMamba: Motion Generation with Frequency and Text State Space Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/AATYMNZO}},
note = {Machine review of arXiv:2411.17532}
}
read the original abstract
Diffusion models achieve impressive performance in human motion generation. However, current approaches typically ignore the significance of frequency-domain information in capturing fine-grained motions within the latent space (e.g., low frequencies correlate with static poses, and high frequencies align with fine-grained motions). Additionally, there is a semantic discrepancy between text and motion, leading to inconsistency between the generated motions and the text descriptions. In this work, we propose a novel diffusion-based FTMoMamba framework equipped with a Frequency State Space Model (FreqSSM) and a Text State Space Model (TextSSM). Specifically, to learn fine-grained representation, FreqSSM decomposes sequences into low-frequency and high-frequency components, guiding the generation of static pose (e.g., sits, lay) and fine-grained motions (e.g., transition, stumble), respectively. To ensure the consistency between text and motion, TextSSM encodes text features at the sentence level, aligning textual semantics with sequential features. Extensive experiments show that FTMoMamba achieves superior performance on the text-to-motion generation task, especially gaining the lowest FID of 0.181 (rather lower than 0.421 of MLD) on the HumanML3D dataset.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Text2action: Generative adversarial synthesis from language to action
Hyemin Ahn, Timothy Ha, Yunho Choi, Hwiyeon Yoo, and Songhwai Oh. Text2action: Generative adversarial synthesis from language to action. In ICRA, pages 5915–5920, 2018. 1
work page 2018
-
[2]
Hp-gan: Probabilistic 3d human motion prediction via gan
Emad Barsoum, John Kender, and Zicheng Liu. Hp-gan: Probabilistic 3d human motion prediction via gan. In CVPRW, pages 1418–1427, 2018. 1
work page 2018
-
[3]
Frequency domain analysis of human mo- tions in surveillance video
Hsuan T Chang, Chang-Sian Chen, Chun-Wen Hung, and Day-Fann Shen. Frequency domain analysis of human mo- tions in surveillance video. In BWCCA, pages 522–526,
-
[4]
Fre- quency domain state-space system identification
Chung-Wen Chen, Jer-Nan Juang, and Gordon Lee. Fre- quency domain state-space system identification. In ACC, pages 3057–3061, 1993. 2
work page 1993
-
[5]
Executing your commands via mo- tion diffusion in latent space
Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu. Executing your commands via mo- tion diffusion in latent space. In CVPR, pages 18000–18010,
-
[6]
Slowfast diversity-aware prototype learning for egocentric action recognition
Guangzhao Dai, Xiangbo Shu, Rui Yan, Peng Huang, and Jinhui Tang. Slowfast diversity-aware prototype learning for egocentric action recognition. In ACM MM, pages 7549– 7558, 2023. 1
work page 2023
-
[7]
Fusion-mamba for cross-modality object detection
Wenhao Dong, Haodong Zhu, Shaohui Lin, Xiaoyan Luo, Yunhang Shen, Xuhui Liu, Juan Zhang, Guodong Guo, and Baochang Zhang. Fusion-mamba for cross-modality object detection. arXiv preprint arXiv:2404.09146, 2024. 3
arXiv 2024
-
[8]
Synthesis of compositional animations from textual descriptions
Anindita Ghosh, Noshaba Cheema, Cennet Oguz, Christian Theobalt, and Philipp Slusallek. Synthesis of compositional animations from textual descriptions. In ICCV, pages 1396– 1406, 2021. 6
work page 2021
Show all 37 references
-
[9]
Tm2d: Bimodality driven 3d dance generation via music-text integration
Kehong Gong, Dongze Lian, Heng Chang, Chuan Guo, Zi- hang Jiang, Xinxin Zuo, Michael Bi Mi, and Xinchao Wang. Tm2d: Bimodality driven 3d dance generation via music-text integration. In ICCV, pages 9942–9952, 2023. 1, 2
2023
-
[10]
Mamba: Linear-time sequence modeling with selective state spaces
Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023. 2
2023 arXiv
-
[11]
Ac- tion2motion: Conditioned generation of 3d human motions
Chuan Guo, Xinxin Zuo, Sen Wang, Shihao Zou, Qingyao Sun, Annan Deng, Minglun Gong, and Li Cheng. Ac- tion2motion: Conditioned generation of 3d human motions. In ACM MM, pages 2021–2029, 2020. 6
2021
-
[12]
Generating diverse and natural 3d human motions from text
Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In CVPR, pages 5152–5161,
-
[13]
Tm2t: Stochastic and tokenized modeling for the reciprocal gener- ation of 3d human motions and texts
Chuan Guo, Xinxin Zuo, Sen Wang, and Li Cheng. Tm2t: Stochastic and tokenized modeling for the reciprocal gener- ation of 3d human motions and texts. In ECCV, pages 580– 597, 2022. 6
2022
-
[14]
Denoising dif- fusion probabilistic models
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. NeurIPS, 33:6840–6851, 2020. 4
2020
-
[15]
Irsrmamba: Infrared image super- resolution via mamba-based wavelet transform feature mod- ulation model
Yongsong Huang, Tomo Miyazaki, Xiaofeng Liu, and Shinichiro Omachi. Irsrmamba: Infrared image super- resolution via mamba-based wavelet transform feature mod- ulation model. arXiv preprint arXiv:2405.09873, 2024. 3
2024 arXiv
-
[16]
Stablemofusion: Towards robust and efficient diffusion-based motion generation framework
Yiheng Huang, Hui Yang, Chuanchen Luo, Yuxi Wang, Shibiao Xu, Zhaoxiang Zhang, Man Zhang, and Jun- ran Peng. Stablemofusion: Towards robust and efficient diffusion-based motion generation framework. In ACM MM, pages 224–232, 2024. 2, 4
2024
-
[17]
Motiongpt: Human motion as a foreign language
Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen. Motiongpt: Human motion as a foreign language. NeurIPS, 36:20067–20079, 2023. 1, 2, 6
2023
-
[18]
Differentiable frequency-based disentanglement for aerial video action recognition
Divya Kothandaraman, Ming Lin, and Dinesh Manocha. Differentiable frequency-based disentanglement for aerial video action recognition. arXiv preprint arXiv:2209.09194,
-
[19]
Tfformer: A time frequency information fusion based cnn-transformer model for osa de- tection with single-lead ecg
Chengjian Li, Zhenghao Shi, Liang Zhou, Zhijun Zhang, Chenwei Wu, Xiaoyong Ren, Xinhong Hei, Minghua Zhao, Yitong Zhang, Haiqin Liu, et al. Tfformer: A time frequency information fusion based cnn-transformer model for osa de- tection with single-lead ecg. IEEE TIM, 2023. 4
2023
-
[20]
Fouriermamba: Fourier learning integration with state space models for image deraining
Dong Li, Yidi Liu, Xueyang Fu, Senyan Xu, and Zheng- Jun Zha. Fouriermamba: Fourier learning integration with state space models for image deraining. arXiv preprint arXiv:2405.19450, 2024. 3
2024 arXiv
-
[21]
Robomamba: Mul- timodal state space model for efficient robot reasoning and manipulation
Jiaming Liu, Mengzhen Liu, Zhenyu Wang, Lily Lee, Kaichen Zhou, Pengju An, Senqiao Yang, Renrui Zhang, Yandong Guo, and Shanghang Zhang. Robomamba: Mul- timodal state space model for efficient robot reasoning and manipulation. arXiv preprint arXiv:2406.04339, 2024. 3
2024 arXiv
-
[22]
Amass: Archive of mo- tion capture as surface shapes
Naureen Mahmood, Nima Ghorbani, Nikolaus F Troje, Ger- ard Pons-Moll, and Michael J Black. Amass: Archive of mo- tion capture as surface shapes. In ICCV, pages 5442–5451,
-
[23]
Temos: Generating diverse human motions from textual descriptions
Mathis Petrovich, Michael J Black, and G ¨ul Varol. Temos: Generating diverse human motions from textual descriptions. In ECCV, pages 480–497, 2022. 2, 6
2022
-
[24]
Learn- ing transferable visual models from natural language super- vision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In ICML, pages 8748–8763, 2021. 6
2021
-
[25]
Human motion diffusion as a generative prior
Yonatan Shafir, Guy Tevet, Roy Kapon, and Amit H Bermano. Human motion diffusion as a generative prior. arXiv preprint arXiv:2303.01418, 2023. 2
2023 arXiv
-
[26]
Finestyle: Semantic-aware fine- grained motion style transfer with dual interactive-flow fu- sion
Wenfeng Song, Xingliang Jin, Shuai Li, Chenglizhao Chen, Aimin Hao, and Xia Hou. Finestyle: Semantic-aware fine- grained motion style transfer with dual interactive-flow fu- sion. IEEE TVCG, 2023. 2
2023
-
[27]
Motionclip: Exposing human motion generation to clip space
Guy Tevet, Brian Gordon, Amir Hertz, Amit H Bermano, and Daniel Cohen-Or. Motionclip: Exposing human motion generation to clip space. In ECCV, pages 358–374, 2022. 1, 2
2022
-
[28]
Human motion diffu- sion model
Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-or, and Amit Haim Bermano. Human motion diffu- sion model. In ICLR, 2023. 2, 4, 6
2023
-
[29]
Attention is all you need
A Vaswani. Attention is all you need. NeurIPS, 2017. 2
2017
-
[30]
Sigma: Siamese mamba network for multi-modal semantic segmentation
Z Wan, Y Wang, S Yong, P Zhang, S Stepputtis, K Sycara, and Y Xie. Sigma: Siamese mamba network for multi-modal semantic segmentation. arXiv preprint arXiv:2404.04256 ,
-
[31]
Fg-t2m: Fine-grained text-driven human motion generation via diffusion model
Yin Wang, Zhiying Leng, Frederick WB Li, Shun-Cheng Wu, and Xiaohui Liang. Fg-t2m: Fine-grained text-driven human motion generation via diffusion model. In ICCV, pages 22035–22044, 2023. 5, 6
2023
-
[32]
Demmamba: Alignment-free raw video demoireing with frequency-assisted spatio-temporal mamba
Shuning Xu, Xina Liu, Binbin Song, Xiangyu Chen, Qiubo Chen, and Jiantao Zhou. Demmamba: Alignment-free raw video demoireing with frequency-assisted spatio-temporal mamba. arXiv preprint arXiv:2408.10679, 2024. 3
2024 arXiv
-
[33]
Physdiff: Physics-guided human motion diffusion model
Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, and Jan Kautz. Physdiff: Physics-guided human motion diffusion model. In ICCV, pages 16010–16021, 2023. 2
2023
-
[34]
Gener- ative motion stylization of cross-structure characters within canonical motion space
Jiaxu Zhang, Xin Chen, Gang Yu, and Zhigang Tu. Gener- ative motion stylization of cross-structure characters within canonical motion space. In ACM MM, pages 7018–7026,
-
[35]
Motiondif- fuse: Text-driven human motion generation with diffusion model
Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondif- fuse: Text-driven human motion generation with diffusion model. IEEE TPAMI, 2024. 2, 5, 6, 7
2024
-
[36]
Motion mamba: Efficient and long sequence motion generation
Zeyu Zhang, Akide Liu, Ian Reid, Richard Hartley, Bo- han Zhuang, and Hao Tang. Motion mamba: Efficient and long sequence motion generation. In ECCV, pages 265–282,
-
[37]
Attt2m: Text-driven human motion generation with multi- perspective attention mechanism
Chongyang Zhong, Lei Hu, Zihao Zhang, and Shihong Xia. Attt2m: Text-driven human motion generation with multi- perspective attention mechanism. In ICCV, pages 509–519,
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.