REVIEW 4 major objections 5 minor 29 references
Efficient Long Speech Sequence Modelling for Time-Domain Depression Level Estimation
T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read A dual-path bidirectional-Mamba network fed with up to 50 seconds of raw speech estimates depression severity more accurately than spectrogram-based and shorter-segment methods, achieving RMSE 9.05 on AVEC2013 and 9.14 on AVEC2014.
desk verdict A useful engineering contribution with honest same-pipeline comparisons, but the SOTA claim rests on borrowed baselines and an unresolved table inconsistency. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the dual-path long-sequence modelling module built on Bi-Mamba. The raw waveform is reshaped into a 3D tensor of chunks; the intra-chunk block applies Bi-Mamba to each chunk, and the inter-chunk block applies it across chunks, and the two are fused and projected back to the original waveform shape. Bi-Mamba is a bidirectional variant of Mamba, a selective state space model whose parameters $\Delta$, $\hat{A}$, $\hat{B}$, and $C$ are updated per input step, computed here with two parallel convolutions for forward and backward SSMs that are gated by the input and averaged. Around this core sits the temporal external attention module, which replaces the keys and values of self-attention with learned external matrices $M_k$ and $M_v$, capturing inter-instance correlations, and the prediction module that produces the final depression score.
What would settle it
Re-run Transformer, DPRNN, and Mamba on AVEC2013 and AVEC2014 using exactly the same 8 kHz sampling, 15s/30s/50s segmentation, batch size 1, Adam with learning rate 0.002, and 100 epochs; if any baseline reaches or beats RMSE 9.05 on AVEC2013 or 9.14 on AVEC2014, the claimed superiority over the state of the art fails.
Extended reading notes
Core claim
The central claim is that depression severity can be estimated from long raw speech waveforms in the time domain, without time-frequency transforms, and that the long-range structure of speech matters for accuracy. To show this, the paper constructs a network with three parts: a long-sequence modelling module that reshapes the waveform into chunks and applies bidirectional Mamba along both intra-chunk and inter-chunk axes, reconstructing the signal through a dual-path scheme borrowed from DPRNN; a temporal external attention module that uses learned external memory matrices in place of the keys and values of self-attention to emphasise depression-related cues; and a prediction module that maps the reconstructed sequence to a BDI-II score. The reported results are RMSE 9.05 on AVEC2013 and 9.14 on AVEC2014 with 50-second inputs, with the method outperforming vanilla Transformer, DPRNN, and Mamba at every tested length (15s, 30s, 50s) and outperforming published baselines in the 30s comparison.
Load-bearing premise
The reported advantage over state-of-the-art methods relies on comparing against published baseline numbers that the authors did not re-run under the same audio lengths and training conditions; if those numbers came from a different setup, the improvement could be an artifact rather than real.
Editorial extensions
If this is right
- Longer input segments improve accuracy: the proposed model's RMSE on AVEC2013 drops from 9.50 at 15 seconds to 9.05 at 50 seconds, while the Transformer and DPRNN baselines stay flat or worsen.
- Time-domain processing can replace time-frequency representations for depression estimation, avoiding information loss from Fourier and Mel transforms.
- The gains come from both architectural components: the ablation shows that adding temporal external attention (L+TEA) outperforms adding self-attention (L+SA), and both beat the long-sequence module alone (L).
- The method points toward a practical protocol in which a clinician's full interview segment, rather than a brief clip, is used for automated depression scoring.
Reading between the lines
- If the long-context advantage holds, the same dual-path Bi-Mamba backbone could transfer to other paralinguistic assessments that suffer from short-window analysis, such as dementia or ADHD screening from speech.
- The published baseline numbers were simply adopted rather than re-run under identical conditions; until those baselines are reproduced with the same 8kHz, 50-second segmentation and training recipe, part of the reported gap could reflect protocol differences.
- A direct stress test would apply the model to an interview dataset such as DAIC-WOZ with 16kHz audio and PHQ-8 scores, checking whether the 8kHz, 50s design generalizes beyond AVEC.
- The external attention module's learned memory size is a free parameter; varying it could reveal a bias-variance tradeoff in how much external context the depression estimator needs.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a time-domain, long-sequence speech model for estimating depression severity from raw waveforms. The architecture combines a dual-path structure in which the Bi-RNN of DPRNN is replaced by a bidirectional Mamba (Bi-Mamba), together with a temporal external attention module and a prediction module. Experiments on AVEC2013 and AVEC2014 use speech segments of 15s, 30s, and 50s, with each segment inheriting the full-recording BDI-II label. The paper reports RMSE/MAE improvements over Transformer, DPRNN, and Mamba baselines re-implemented under the same pipeline (Tables II and III), and claims state-of-the-art performance over prior depression-estimation systems (Table I), although the latter comparison relies on published baseline numbers adopted without re-running. The central claim is therefore an empirical benchmark comparison, with no formal derivation required for its correctness.
Significance. If the reported gains hold under a matched protocol, the paper would provide a useful data point for raw-waveform, long-context depression estimation, and the dual-path Bi-Mamba architecture is a plausible and timely combination of recent sequence-modelling ideas. The same-pipeline comparisons in Tables II and III against Transformer, DPRNN, and Mamba are a genuine strength, as is the explicit focus on long speech segments, which is relevant to clinical interaction settings. However, the headline state-of-the-art claim is not yet supported because the comparison in Section III-D/Table I uses externally reported baseline numbers under uncontrolled conditions, and the segment-level evaluation with duplicated full-recording labels complicates the interpretation of all RMSE/MAE numbers. The paper does not provide code, error bars, or significance tests, so the magnitude and reliability of the reported improvements cannot be assessed from the manuscript alone.
major comments (4)
- [Section III-D, Table I] The state-of-the-art comparison is not a controlled comparison. The text states, 'Note that we simply adopt the original results for two dataset baseline methods,' meaning STA-EEP [29] and TDCA-Net [14] were not re-run under this paper's 30-second, 8 kHz, raw-waveform, segment-level duplicated-label protocol. Since the proposed method's advantage over these baselines (e.g., RMSE 9.24 vs. 10.54 for TDCA-Net on AVEC2013) may be wholly or partly an artifact of different segment lengths, features, or evaluation units, the 'outstanding performance over the state-of-the-art' claim in the Abstract and Section I is not established. The authors should either re-run prior methods under the exact proposed protocol or restrict the SOTA claim to the matched same-pipeline baselines in Tables II and III.
- [Tables I and II] There is an internal inconsistency for the same proposed configuration at 30 seconds. Table I reports AVEC2013 RMSE/MAE of 9.24/8.26 for 'L + TEA', while Table II reports 9.14/8.35 for 'Proposed' at 30s; the AVEC2014 numbers also differ (9.20/8.39 vs. 9.20/8.39 in Table II? Actually Table II gives 9.20/8.39, matching, but the AVEC2013 row does not match). Since both tables should describe the identical model and protocol, this discrepancy needs to be explained, and the affected numbers corrected.
- [Section III-C, evaluation protocol] Segment-level evaluation with duplicated full-recording BDI-II labels invalidates the assumption of independent test samples: multiple segments from the same recording are correlated and share the same label, so the effective sample size is much smaller than the number of segments. This affects all reported RMSE/MAE values and prevents direct comparison with prior per-recording or per-utterance protocols. The authors should either report results at the recording level (e.g., averaging segment predictions) or provide clustered error bars/significance tests (e.g., bootstrapping by subject or recording).
- [Section III-C and III-E] The paper does not report the model size, number of parameters, FLOPs, or runtime, despite the word 'Efficient' in the title and the claim of efficient long-sequence modelling. The choice of segment durations (15s, 30s, 50s) is also presented without a principled selection criterion or a held-out justification. Adding parameter counts and a complexity analysis would substantiate the efficiency claim, and a discussion of how segment lengths were chosen would strengthen the long-sequence argument.
minor comments (5)
- [Abstract and Section III-D] The spacing in 'A VEC2013' and 'A VEC2014' is inconsistent; the standard dataset names are 'AVEC2013' and 'AVEC2014'.
- [Section II-B, Eqs. (7) and (8)] The dimensions of the external attention maps are unclear. If Y ∈ R^{N×L}, then Mk and Mv are written as R^{N×L}, but then the operation Y × Mk in Eq. (8) is dimensionally ambiguous; please define the intended matrix multiplication and softmax axes explicitly.
- [Section III-A] The description of AVEC2014 as 'derived from AVEC2013, includes the Northwind and FreeForm tasks' is vague; clarify how the 100-video merged set relates to the original 150 clips and to the standard train/dev/test splits.
- [Figure 3] Figure 3 is referenced in Section III-D but contains no error bars or confidence intervals, making the claimed 'consistent upward trend' difficult to evaluate; consider adding per-seed variability or at least stating the number of runs.
- [Section III-D, Table I] The table caption says 'AVEC 2014 DATASETS' but the table includes AVEC2013 columns; the caption should be corrected.
Circularity Check
No circularity found: the proposed system is evaluated against external datasets and re-implemented baselines, and none of the claimed predictions reduce to fitted inputs or self-citations.
full rationale
The paper makes an empirical claim, not a derivation-from-first-principles claim: a Bi-Mamba dual-path time-domain network with external attention is trained and tested on AVEC2013 and AVEC2014. The central results are RMSE/MAE numbers on held-out test partitions, compared with re-implemented Transformer, DPRNN, and Mamba baselines under the same segmentation protocol in Tables II and III. There is no equation in the paper whose output is mathematically forced by its input labels or by a fitted parameter renamed as a prediction. The main correctness concern raised by a careful reader is that the state-of-the-art comparison in Table I adopts published baseline numbers without re-running STA-EEP and TDCA-Net under the same 15/30/50-second segment lengths, 8 kHz resampling, and label-duplication protocol; this threatens comparability and the strength of the 'outstanding performance over the state-of-the-art' claim. However, that is an experimental-control and validity issue, not circularity: borrowed benchmark numbers are not derived from the proposed model, and the claim is not made true by definition. The paper also contains an unexplained numerical inconsistency, with the 30-second AVEC2013 result reported as RMSE/MAE 9.24/8.26 in Table I and 9.14/8.35 in Table II; again, this is an internal consistency defect rather than a circular step. No self-citation is load-bearing: references to the authors' own prior work appear only as general background on ADHD and speech processing, not as justification of the central architecture or results. The uniqueness of Mamba or external attention is not imported from the authors' own prior theorems. The method is benchmarked against external, machine-measured targets, and the re-implemented baselines provide an independent same-pipeline comparison. Therefore the honest finding is no significant circularity, and the score is 0.
Assumptions & free parameters
free parameters (4)
- Segment duration =
15s, 30s, 50s (also 12-30s in Fig. 3)
- Chunk length K and chunk count S
- Hidden dimension H and feature dimension N
- Learning rate and training epochs =
0.002, 100
assumptions (4)
- domain assumption Speech signals contain cues about depression severity.
- domain assumption BDI-II scores assigned to full recordings remain valid for every 15-50 second segment cut from those recordings.
- domain assumption A dual-path Mamba architecture can effectively model long-range temporal dependencies in raw waveforms.
- domain assumption The AVEC2013/2014 segmentation and subject split used here matches the official challenge protocol.
Cite this review
Pith. "Pith review of Efficient Long Speech Sequence Modelling for Time-Domain Depression Level Estimation." pith.science (2026). https://pith.science/paper/GJKND7DF
@misc{pith2026250102512,
author = {Pith},
title = {Pith review of: Efficient Long Speech Sequence Modelling for Time-Domain Depression Level Estimation},
year = {2026},
howpublished = {\url{https://pith.science/paper/GJKND7DF}},
note = {Machine review of arXiv:2501.02512}
}
read the original abstract
Depression significantly affects emotions, thoughts, and daily activities. Recent research indicates that speech signals contain vital cues about depression, sparking interest in audio-based deep-learning methods for estimating its severity. However, most methods rely on time-frequency representations of speech which have recently been criticized for their limitations due to the loss of information when performing time-frequency projections, e.g. Fourier transform, and Mel-scale transformation. Furthermore, segmenting real-world speech into brief intervals risks losing critical interconnections between recordings. Additionally, such an approach may not adequately reflect real-world scenarios, as individuals with depression often pause and slow down in their conversations and interactions. Building on these observations, we present an efficient method for depression level estimation using long speech signals in the time domain. The proposed method leverages a state space model coupled with the dual-path structure-based long sequence modelling module and temporal external attention module to reconstruct and enhance the detection of depression-related cues hidden in the raw audio waveforms. Experimental results on the AVEC2013 and AVEC2014 datasets show promising results in capturing consequential long-sequence depression cues and demonstrate outstanding performance over the state-of-the-art.
Figures
Reference graph
Works this paper leans on
-
[29]
Multimodal spatiotem- poral representation for automatic depression level detection,
M. Niu, J. Tao, B. Liu, J. Huang, and Z. Lian, “Multimodal spatiotem- poral representation for automatic depression level detection,” IEEE Transactions on Affective Computing , 2020
work page 2020
-
[14]
Tdca-net: Time-domain channel attention network for depression detection.,
C. Cai, M. Niu, B. Liu, J. Tao, and X. Liu, “Tdca-net: Time-domain channel attention network for depression detection.,” in Interspeech, pp. 2511–2515, 2021
work page 2021
-
[1]
C. Otte, S. M. Gold, B. W. Penninx, C. M. Pariante, A. Etkin, M. Fava, D. C. Mohr, and A. F. Schatzberg, “Major depressive disorder,” Nature Reviews Disease primers , vol. 2, no. 1, pp. 1–20, 2016
work page 2016
-
[2]
Insights into detecting adult ADHD symptoms through advanced dual-stream machine learning,
C. Nash, R. Nair, and S. M. Naqvi, “Insights into detecting adult ADHD symptoms through advanced dual-stream machine learning,” IEEE Transactions on Neural Systems and Rehabilitation Engineering , 2024
work page 2024
-
[3]
Machine learning in ADHD and depression mental health diagnosis: A survey,
C. Nash, R. Nair, and S. M. Naqvi, “Machine learning in ADHD and depression mental health diagnosis: A survey,” IEEE Access , 2023
work page 2023
-
[4]
Harnessing video intelligence: Intelligent system for adhd detection,
Y . Li, R. Nair, and S. M. Naqvi, “Harnessing video intelligence: Intelligent system for adhd detection,” in 2024 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), pp. 274–278, IEEE, 2024
work page 2024
-
[5]
S. Li, R. Nair, and M. Naqvi, “Acoustic and text features analysis for adult ADHD screening: A data-driven approach utilizing diva interview,” IEEE Journal of Translational Engineering in Health and Medicine , 2024
work page 2024
-
[6]
Deep learning for depression recognition with audiovisual cues: A review,
L. He, M. Niu, P. Tiwari, P. Marttinen, R. Su, J. Jiang, C. Guo, H. Wang, S. Ding, Z. Wang, et al., “Deep learning for depression recognition with audiovisual cues: A review,” Information Fusion , vol. 80, pp. 56–86, 2022
work page 2022
Show all 29 references
-
[7]
Acoustical properties of speech as indicators of depression and suicidal risk,
D. J. France, R. G. Shiavi, S. Silverman, M. Silverman, and M. Wilkes, “Acoustical properties of speech as indicators of depression and suicidal risk,” IEEE Transactions on Biomedical Engineering , vol. 47, no. 7, pp. 829–837, 2000
2000
-
[8]
Automatic depression level detection via lp-norm pooling,
M. Niu, J. Tao, B. Liu, and C. Fan, “Automatic depression level detection via lp-norm pooling,” in Proc. Interspeech, (Graz, Austria), pp. 4559– 4563, 2019
2019
-
[9]
A hierarchical depression detection model based on vocal and emotional cues,
Y . Dong and X. Yang, “A hierarchical depression detection model based on vocal and emotional cues,” Neurocomputing, vol. 441, pp. 279–290, 2021
2021
-
[10]
Position and orientation-aware one-shot learning for medical action recognition from signal data,
L. Xie, Y . Yang, Z. Fu, and S. M. Naqvi, “Position and orientation-aware one-shot learning for medical action recognition from signal data,” IEEE Transactions on Multimedia , 2024
2024
-
[11]
Speechformer++: A hierarchical efficient framework for paralinguistic speech processing,
W. Chen, X. Xing, X. Xu, J. Pang, and L. Du, “Speechformer++: A hierarchical efficient framework for paralinguistic speech processing,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 31, pp. 775–788, 2023
2023
-
[12]
Wavdepressionnet: Automatic depression level prediction via raw speech signals,
M. Niu, J. Tao, Y . Li, Y . Qin, and Y . Li, “Wavdepressionnet: Automatic depression level prediction via raw speech signals,” IEEE Transactions on Affective Computing , vol. 15, no. 1, pp. 285–296, 2023
2023
-
[13]
Rabiner and B.-H
L. Rabiner and B.-H. Juang, Fundamentals of speech recognition . Prentice-Hall, Inc., 1993
1993
-
[15]
Csenet: Complex squeeze-and- excitation network for speech depression level prediction,
C. Fan, Z. Lv, S. Pei, and M. Niu, “Csenet: Complex squeeze-and- excitation network for speech depression level prediction,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pp. 546–550, IEEE, 2022
2022
-
[16]
Dual-path rnn: efficient long sequence modeling for time-domain single-channel speech separation,
Y . Luo, Z. Chen, and T. Yoshioka, “Dual-path rnn: efficient long sequence modeling for time-domain single-channel speech separation,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pp. 46–50, IEEE, 2020
2020
-
[17]
Attention is all you need in speech separation,
C. Subakan, M. Ravanelli, S. Cornell, M. Bronzi, and J. Zhong, “Attention is all you need in speech separation,” in ICASSP 2021- 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 21–25, IEEE, 2021
2021
-
[18]
Dual-path transformer network: Direct context-aware modeling for end-to-end monaural speech separation,
J. Chen, Q. Mao, and D. Liu, “Dual-path transformer network: Direct context-aware modeling for end-to-end monaural speech separation,” in Proc. Interspeech 2020 , pp. 2642–2646, 2020
2020
-
[19]
Combining recurrent, convolutional, and continuous-time models with linear state space layers,
A. Gu, I. Johnson, K. Goel, K. Saab, T. Dao, A. Rudra, and C. R ´e, “Combining recurrent, convolutional, and continuous-time models with linear state space layers,” in Advances in Neural Information Processing Systems, vol. 34, pp. 572–585, 2021
2021
-
[20]
Efficiently modeling long sequences with structured state spaces,
A. Gu, K. Goel, and C. R ´e, “Efficiently modeling long sequences with structured state spaces,” in The International Conference on Learning Representations (ICLR), 2022
2022
-
[21]
Mamba: Linear-time sequence modeling with selective state spaces,
A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752 , 2023
2023 arXiv
-
[22]
Mamba in speech: Towards an alternative to self-attention,
X. Zhang, Q. Zhang, H. Liu, T. Xiao, X. Qian, B. Ahmed, E. Ambikaira- jah, H. Li, and J. Epps, “Mamba in speech: Towards an alternative to self-attention,” arXiv preprint arXiv:2405.12609 , 2024
2024 arXiv
-
[23]
Beyond self-attention: External attention using two linear layers for visual tasks,
M.-H. Guo, Z.-N. Liu, T.-J. Mu, and S.-M. Hu, “Beyond self-attention: External attention using two linear layers for visual tasks,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 45, no. 5, pp. 5436–5447, 2022
2022
-
[24]
A VEC 2013: The Continu- ous Audio/Visual Emotion and Depression Recognition Challenge,
M. Valstar, B. Schuller, K. Smith, F. Eyben, B. Jiang, S. Bilakhia, S. Schnieder, R. Cowie, and M. Pantic, “A VEC 2013: The Continu- ous Audio/Visual Emotion and Depression Recognition Challenge,” in Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Cha...
2013
-
[25]
A VEC 2014: 3D Dimensional Affect and De- pression Recognition Challenge,
M. Valstar, B. Schuller, K. Smith, T. Almaev, F. Eyben, J. Krajewski, R. Cowie, and M. Pantic, “A VEC 2014: 3D Dimensional Affect and De- pression Recognition Challenge,” in Proceedings of the 4th International Workshop on Audio/Visual Emotion Challenge , pp. 3–10, 2014
2014
-
[26]
A narrative review of the beck depres- sion inventory (bdi) and implications for its use in an alcohol-dependent population,
A. Mcpherson and C. R. Martin, “A narrative review of the beck depres- sion inventory (bdi) and implications for its use in an alcohol-dependent population,” Journal of Psychiatric and Mental Health Nursing , vol. 17, no. 1, pp. 19–30, 2010
2010
-
[27]
Adam: A method for stochastic optimization,
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International Conference on Learning Representations (Y . Bengio and Y . LeCun, eds.), 2015
2015
-
[28]
Attention is all you need,
A. Vaswani, “Attention is all you need,” Advances in Neural Information Processing Systems, 2017
2017
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.