Pith. sign in

REVIEW 3 major objections 8 minor 59 references

Investigating Acoustic-Textual Emotional Inconsistency Information for Automatic Depression Detection

T0 review · 3 major / 8 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that emotional inconsistency between speech and text is a learnable signal for depression severity, and reports that adding it lifts subject-level accuracy from 75.74% to 81.25%.

desk verdict New application of emotional inconsistency to depression detection, but the +5.5% gain is confounded by an extra trainable branch and unreleased data. read the letter →

arxiv 2412.18614 v1 pith:MMTRFJIM submitted 2024-12-09 eess.AS cs.AIcs.CL

Rongfeng Su , Changqing Xu , Xinyi Wu , Feng Xu , Xie Chen , Lan Wangt , Nan Yan This is my paper · ORCID
classification eess.AScs.AIcs.CL
keywords depressiondetectionacoustic-textualemotionalinconsistencymultimodalcross-attentionsentimentlabeldisagreementexpressioncounselingconversationsTransformerseverityclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the mismatch between how a person sounds and what their words say carries diagnostic information for depression, and that this mismatch can be extracted automatically from counseling conversations. The authors define acoustic-textual emotional inconsistency (ATEI) as disagreement between the sentiment label of an audio segment, obtained by majority vote of five annotators, and the sentiment label of its transcript, obtained from a HowNet lexicon. A multimodal cross-attention network is trained to predict this consistency label, and its intermediate embeddings are fused with acoustic and textual features in a Transformer depression classifier. On the authors' counseling conversational dataset, adding a scaled ATEI embedding raises subject-level three-class depression severity accuracy from 75.74% to 81.25%.

What carries the argument

The central object is the Acoustic-Textual Emotional Inconsistency (ATEI) embedding, produced by a multimodal cross-attention network. The network is supervised by a consistent or inconsistent label: a segment pair is consistent if the majority-vote acoustic sentiment and the HowNet text sentiment agree, and inconsistent otherwise. Cross-attention computes $\mathbf{X}^{(AT)} = \mathrm{softmax}(\mathbf{Q}^{(A)}\mathbf{K}^{(T)\top}/\sqrt{d_k})\mathbf{V}^{(T)}$ and the reverse $\mathbf{X}^{(TA)}$, so each modality's emotional content is re-expressed through the other; the pooled concatenation of the original and cross-attended sequences is passed through fully connected layers, and the FC2 output serves as the ATEI representation. The depression classifier then concatenates acoustic, textual, and ATEI features, scales the ATEI part by a learnable $\alpha$ with $\sum_i \alpha_i = 1$, and is trained jointly with the consistency classification loss.

What would settle it

Feed the two sentiment labels directly into the depression classifier without the cross-attention embedding; if that baseline matches or beats 81.25% subject-level accuracy, the reported gain is carried by the sentiment information rather than by the ATEI mismatch representation.

Watch

Extended reading notes

Core claim

The paper's central claim is that emotional expression inconsistency across acoustic and textual modalities is a usable, depression-relevant signal, and that it can be captured without manual feature engineering by a cross-attention network supervised by a binary consistency label. The network reads frame-level self-supervised speech features and word-level text features, computes cross-domain attention in both directions, pools and concatenates the outputs, and uses a middle fully connected layer as the ATEI embedding. When that embedding is concatenated with acoustic and textual depression features and reweighted by a learnable scaling factor whose components sum to one, the resulting system reaches 81.25% subject-level accuracy for healthy, mild, and moderate classification, an absolute gain of 5.51 points over the acoustic-textual baseline. The paper also claims that this is the first depression detection system to use emotional expression inconsistency across modalities.

Load-bearing premise

The load-bearing premise is that binary disagreement between the majority-vote acoustic sentiment label and the HowNet text sentiment label faithfully captures the emotional inconsistency that distinguishes depressed from healthy speakers.

Editorial extensions

If this is right

  • The ATEI embedding is data-driven and does not require hand-designed features, so the same labeled segments can be reused to obtain an extra diagnostic signal.
  • ATEI information is complementary but not sufficient on its own: the paper reports roughly 40% accuracy when ATEI embeddings are used alone, versus the full gain when they are fused with acoustic and textual features.
  • Among the fusion strategies tested, concatenation outperforms addition and multiplication, and applying the learnable scaling factor to the ATEI embedding adds a further absolute gain of 1.84 points in the acoustic-textual-ATEI configuration.
  • The middle fully connected layer (FC2) of the cross-attention network yields the best ATEI representation, indicating that intermediate hidden states carry more depression-relevant inconsistency information than the binary consistency label or its output probability.
  • Because evaluation uses speaker-disjoint folds and majority voting across segments, the reported 81.25% subject-level accuracy is a claim about unseen speakers rather than about memorized segments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: Because the consistency label is defined as sentiment disagreement, replacing the ATEI embedding with the two sentiment labels themselves in the same fusion architecture would reveal how much of the gain is specifically the mismatch signal rather than extra sentiment information.
  • Editorial extension: The learnable scaling factor with $\sum_i \alpha_i = 1$ can be inspected post hoc; if high-weight dimensions correspond to known acoustic-textual disagreement patterns, the model doubles as a hypothesis generator for which inconsistency types track depression severity.
  • Editorial extension: The method should transfer to public interview corpora such as DAIC-WOZ; the paper expects less affective variability there, so a smaller gain in that setting would actually be consistent with the mechanism rather than a contradiction.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. The paper proposes a Transformer-based framework for automatic depression detection that incorporates Acoustic-Textual Emotional Inconsistency (ATEI) as an auxiliary signal. ATEI is defined by label disagreement between acoustic sentiment (majority vote of five annotators) and textual sentiment (HowNet lexicon score). A multimodal cross-attention network extracts ATEI embeddings, which are fused with acoustic and textual features via addition, multiplication, or concatenation, optionally modulated by a learnable scaling factor. On a private counseling dataset of 272 subjects, the best configuration (A+T+E with concatenation and scaling) achieves 81.25% subject-level accuracy, a 5.51% absolute improvement over the authors' A+T baseline. The paper also claims to be the first to use emotional expression inconsistency for depression detection.

Significance. If validated, the idea of using cross-modal emotional inconsistency as a depression biomarker is novel and clinically motivated, drawing on Emotion Context-Insensitivity theory. The manuscript is systematic in its ablations: it compares four SSL acoustic extractors, two text encoders, three fusion strategies, three embedding layers, and a scaling factor, which is a strength. The authors are also explicit that ATEI alone is not sufficient (Section 4.3.2), which is a useful negative result. However, the current empirical evidence does not yet establish that the ATEI construct, rather than the additional trainable module or the depression-label gradient flowing into it, drives the reported gains. The evaluation rests on a private dataset with no error bars or significance tests, and the baseline is the authors' own reimplementation, so the external validity of the numbers is unclear. These issues are addressable with additional control experiments and statistical reporting, but they are load-bearing for the central claim.

major comments (3)
  1. [Section 4.3.2, Eq. (16), Tables 3–5] The central comparison A+T+E vs. A+T does not isolate the ATEI construct. The ATEI branch in Fig. 3 is a large trainable module (Transformer stacks over X(A) and X(T), cross-attention, FC layers) that is jointly optimized with the depression loss L_Total = L_Depression + L_ATEI (Eq. 16). The A+T baseline has no analogous branch, so the +5.51% accuracy gain (Table 5) could be caused by the extra model capacity or by the gradient of L_Depression flowing into the ATEI branch, rather than by the inconsistency information itself. The paper's own report that an ATEI-only system reaches only ~40% accuracy (Section 4.3.2) supports the concern that the raw ATEI embedding is weak and that the gain comes from joint task-specific optimization. To support the claim, the authors should include a control in which the same architecture is trained with (a) shuffled consistency labels, (b) a non-inconsistency auxiliary label (e.g., acoustic sentiment only), or (c) a frozen/random ATEI branch. The comparison with [19]* in Table 3 is not this control because [19]* uses a different architecture.
  2. [Section 4.1 and Section 4.3.2] The operational definition of emotional inconsistency is binary label disagreement: the acoustic sentiment is the majority vote of five annotators and the text sentiment is a HowNet lexicon score, and a segment is 'consistent' if the two labels match. The paper provides no inter-annotator agreement statistic and no validation that this label-disagreement proxy corresponds to the ECI-based construct of emotional blunting cited in the introduction. Moreover, because the audio annotators listen to speech that contains the words, the 'acoustic' sentiment is not independent of lexical content, so the consistency label may reflect general sentiment polarity or annotator noise rather than a calm–negative mismatch. Please report annotation reliability (e.g., Fleiss' kappa) and, ideally, a comparison of the learned ATEI features against the pilot-study findings in [20].
  3. [Section 5.2, Tables 3–5] All reported results are single estimates from five-fold cross-validation, with no standard deviations, confidence intervals, or significance tests. Differences as large as 1.84% (Table 5) could plausibly fall within cross-fold variance for a dataset of 272 subjects. The authors should report per-fold results or at least the variance across folds, and apply a paired significance test (e.g., paired t-test or Wilcoxon test) for the key comparisons between the best ATEI system and the A+T baseline and between the scaled and unscaled variants.
minor comments (8)
  1. [Abstract and Section 1] The phrase 'To best of our knowledge' should be 'To the best of our knowledge' in both the abstract and the introduction.
  2. [Section 4.3.2 and Fig. 3] The notation FC1, FC2, FC3 is ambiguous: Fig. 3 shows FC1, FC2, FC3 followed by an output layer, while the text says 'three fully connected layers (FC1, FC2, and FC3)' and Table 3 uses FC3 as an embedding layer. Please clarify which layer is the output layer and which layers are used for the Embedding representation.
  3. [Section 4.4, Eq. (15)] The constraint sum(α_i)=1 is stated but no parameterization or training details are given for the learnable scaling factor. Also, since e(E) is used in element-wise addition and multiplication with e(A) and e(T), the scaling may interact with those fusion operations; please clarify how α is applied in each fusion case.
  4. [Table 1] The SDS/SAS score range for healthy controls is 25–45; if these are raw scores, the range overlaps with elevated depression/anxiety levels. Please specify the instrument versions (e.g., SDS raw vs. index score) and the clinical cutoffs used.
  5. [Section 5.1] The hyperparameters (learning rate, batch size, epochs) are given, but the model selection procedure is not described; specifically, whether a validation set was used for early stopping or hyperparameter tuning, and how the best epoch was chosen for each fold.
  6. [Figure 4] The t-SNE plots are qualitative; please provide a quantitative measure of cluster separability (e.g., silhouette score) to support the claim that ATEI features improve class separation.
  7. [Table 2 and Section 5.2.1] The term 'SOTA baseline' is used for what is essentially the authors' own architecture. Since no comparison with published state-of-the-art results on public benchmarks (e.g., DAIC-WOZ) is given, the 'SOTA' label is not justified.
  8. [Introduction] The pilot study [20] is cited as evidence for the ATEI phenomenon, but its methods and quantitative findings are not summarized. A brief description would help readers assess the validity of the operationalization used here.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the ATEI feature is supervised by independently defined sentiment-consistency labels, and the depression-detection gain is tested against baselines without the feature.

full rationale

The paper's central claim is that adding ATEI information improves three-class depression severity classification. The ATEI consistency label is defined in Section 4.3.2 as agreement between acoustic sentiment labels (five-annotator majority, Section 4.1) and textual HowNet sentiment labels. This supervision is not derived from the depression labels, so the depression prediction is not a restatement of the consistency label by construction. The ATEI embedding is an intermediate representation trained to predict consistency, and its utility for depression is evaluated empirically against an A+T baseline and against a reimplemented [19]* emotional-feature baseline. The self-citations [19] and [20] are used for motivation and as a comparison system, not as the proof of the main result; the ECI theory [22] is an external psychological reference. A genuine experimental concern exists: because Eq. (16) trains the ATEI branch jointly with L_Depression and the branch contains substantial extra parameters, the +5.51% gain could partly reflect added capacity or sentiment content rather than the inconsistency construct. However, this is a validity/ablation concern, not a circular reduction; no equation in the paper makes the predicted severity equivalent to the fitted consistency label or to a fitted parameter. No circular step can be quoted and reduced, so the circularity score is 0.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central empirical claim rests on a domain assumption that label disagreement is a valid depression marker, on unvalidated sentiment annotations, and on a fitted scaling factor. No new physical entities are introduced.

free parameters (2)
  • learnable scaling factor alpha = not reported per-dimension after training
    Introduced in Eq. (15) to modulate ATEI embedding strength during fusion; it is jointly optimized with the depression classifier, so the claim that ATEI features correlate with severity is partly a fitted result.
  • Transformer hyperparameters (number of blocks, heads, d_k, hidden dimension) = 12 blocks, 8 heads, d_k=128, D=1024
    Chosen by hand or by standard practice; they affect capacity and are part of the modeling choices but are not the paper's central claim.
assumptions (5)
  • standard math Softmax attention and Transformer blocks behave as specified in Eqs. (2) to (5)
    Standard deep learning building blocks; no proof needed in this context.
  • domain assumption Emotional inconsistency between acoustic and textual modalities is a meaningful marker of depression severity
    Motivated by ECI theory and the authors' pilot study [20], but treated as a premise; the paper does not independently validate that label disagreement corresponds to clinically meaningful emotional blunting.
  • domain assumption Acoustic sentiment labels from five-annotator majority vote and text sentiment labels from HowNet lexicon are reliable
    Section 4.1 defines labels this way; no inter-annotator agreement or validation against clinician ratings is reported.
  • ad hoc to paper Equality of acoustic and textual sentiment labels is a sufficient operationalization of emotional consistency
    Section 4.3.2 sets the supervised label to 1 if labels match and 0 otherwise; this binary proxy is introduced by the authors for this paper.
  • domain assumption SSL features from the 12th block of Wav2Vec, Whisper, WavLM, HuBERT and RoBERTa embeddings contain emotional expression information relevant to depression
    Section 4.2 relies on prior findings [55-57] and standard transfer learning; the SSL models are frozen, not fine-tuned.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Investigating Acoustic-Textual Emotional Inconsistency Information for Automatic Depression Detection." pith.science (2026). https://pith.science/paper/MMTRFJIM

@misc{pith2026241218614,
  author       = {Pith},
  title        = {Pith review of: Investigating Acoustic-Textual Emotional Inconsistency Information for Automatic Depression Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MMTRFJIM}},
  note         = {Machine review of arXiv:2412.18614}
}
read the original abstract

Previous studies have demonstrated that emotional features from a single acoustic sentiment label can enhance depression diagnosis accuracy. Additionally, according to the Emotion Context-Insensitivity theory and our pilot study, individuals with depression might convey negative emotional content in an unexpectedly calm manner, showing a high degree of inconsistency in emotional expressions during natural conversations. So far, few studies have recognized and leveraged the emotional expression inconsistency for depression detection. In this paper, a multimodal cross-attention method is presented to capture the Acoustic-Textual Emotional Inconsistency (ATEI) information. This is achieved by analyzing the intricate local and long-term dependencies of emotional expressions across acoustic and textual domains, as well as the mismatch between the emotional content within both domains. A Transformer-based model is then proposed to integrate this ATEI information with various fusion strategies for detecting depression. Furthermore, a scaling technique is employed to adjust the ATEI feature degree during the fusion process, thereby enhancing the model's ability to discern patients with depression across varying levels of severity. To best of our knowledge, this work is the first to incorporate emotional expression inconsistency information into depression detection. Experimental results on a counseling conversational dataset illustrate the effectiveness of our method.

Figures

Figures reproduced from arXiv: 2412.18614 by the authors.

Figure 1
Figure 1. Proposed Transformer-based framework using additional Acoustic-Textual Emotional Inconsistency (ATEI) information for automatic [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Transformer-based feature aggregation for extracting the acoustic [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Multimodal cross-attention method for extracting ATEI informa [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The t-SNE projection was performed on the outputs from the final [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 50 canonical work pages

  1. [19]

    A transformer-based depression detec- tion network leveraging speech emotional expression cues,

    C. Xu, X. Wu, N. Li, X. Wang, X. Feng, R. Su, N. Yan, and L. Wang, “A transformer-based depression detec- tion network leveraging speech emotional expression cues,” in Proc. Int. Conf. Social Rob. , 2024

    work page 2024

  2. [20]

    Depression enhances internal inconsistency between spoken and semantic emotion: Evidence from the anal- ysis of emotion expression in conversation,

    X. Wu, C. Xu, N. Li, R. Su, L. Wang, and N. Yan, “Depression enhances internal inconsistency between spoken and semantic emotion: Evidence from the anal- ysis of emotion expression in conversation,” in Proc. 12 Conf. Int. Speech Commun. Assoc. , 2024, pp. 4219–4223

    work page 2024

  3. [1]

    Au- tomatic assessment of depression based on visual cues: A systematic review,

    A. Pampouchidou, P . G. Simos, K. Marias, F. Meri- audeau, F. Yang, M. Pediaditis, and M. Tsiknakis, “Au- tomatic assessment of depression based on visual cues: A systematic review,” IEEE T ransactions on Affective Computing, vol. 10, no. 4, pp. 445–470, 2017

    work page 2017

  4. [2]

    Self-mutilation and suicidal behavior in borderline personality disorder,

    P . H. Soloff, J. A. Lis, T. Kelly, J. Cornelius, and R. Ulrich, “Self-mutilation and suicidal behavior in borderline personality disorder,” Journal of Personality Disorders, vol. 8, no. 4, pp. 257–267, 1994

    work page 1994

  5. [3]

    Detect- ing depression with audio/text sequence modeling of interviews,

    T. Al Hanai, M. M. Ghassemi, and J. R. Glass, “Detect- ing depression with audio/text sequence modeling of interviews,” in Proc. Conf. Int. Speech Commun. Assoc. , 2018, pp. 1716–1720

    work page 2018

  6. [4]

    It’s just a matter of time: Detecting depression with time- enriched multimodal transformers,

    A.-M. Bucur, A. Cosma, P . Rosso, and L. P . Dinu, “It’s just a matter of time: Detecting depression with time- enriched multimodal transformers,” in Proc. Eur. Conf. on Info. Retrieval , 2023, pp. 200–215

    work page 2023

  7. [5]

    Transformer-based multimodal feature en- hancement networks for multimodal depression detec- tion integrating video, audio and remote photoplethys- mograph signals,

    H. Fan, X. Zhang, Y. Xu, J. Fang, S. Zhang, X. Zhao, and J. Yu, “Transformer-based multimodal feature en- hancement networks for multimodal depression detec- tion integrating video, audio and remote photoplethys- mograph signals,” Information Fusion , vol. 104, 2024, doi:10.1016/j.inffus.2023.102161

    arXiv 2024

  8. [6]

    A multimodal fusion model with multi-level atten- tion mechanism for depression detection,

    M. Fang, S. Peng, Y. Liang, C.-C. Hung, and S. Liu, “A multimodal fusion model with multi-level atten- tion mechanism for depression detection,” Biomed- ical Signal Processing and Control , vol. 82, 2023, doi:10.1016/j.bspc.2022.104561

    arXiv 2023

Show all 59 references
  1. [7]

    A hierarchical attention network- based approach for depression detection from tran- scribed clinical interviews,

    A. Mallol-Ragolta, Z. Zhao, L. Stappen, N. Cummins, and B. Schuller, “A hierarchical attention network- based approach for depression detection from tran- scribed clinical interviews,” in Proc. Conf. Int. Speech Commun. Assoc., 2019, pp. 221–225

    2019

  2. [8]

    Multi- modal spatiotemporal representation for automatic de- pression level detection,

    M. Niu, J. Tao, B. Liu, J. Huang, and Z. Lian, “Multi- modal spatiotemporal representation for automatic de- pression level detection,” IEEE transactions on affective computing, vol. 14, no. 1, pp. 294–307, 2023

    2023

  3. [9]

    Multimodal fusion of bert-cnn and gated cnn representations for depression detection,

    M. Rodrigues Makiuchi, T. Warnita, K. Uto, and K. Shinoda, “Multimodal fusion of bert-cnn and gated cnn representations for depression detection,” in Proc. ACM Int. Workshop Audio/Visual Emotion Challenge , 2019, pp. 55–63

    2019

  4. [10]

    Fusing multi-level features from audio and contextual sentence embedding from text for interview- based depression detection,

    J. Xue, R. Qin, X. Zhou, H. Liu, M. Zhang, and Z. Zhang, “Fusing multi-level features from audio and contextual sentence embedding from text for interview- based depression detection,” in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. , 2024, pp. 6790–6794

    2024

  5. [11]

    A novel eeg-based graph convolution net- work for depression detection: incorporating sec- ondary subject partitioning and attention mecha- nism,

    Z. Zhang, Q. Meng, L. Jin, H. Wang, and H. Hou, “A novel eeg-based graph convolution net- work for depression detection: incorporating sec- ondary subject partitioning and attention mecha- nism,” Expert Systems with Applications , vol. 239, 2024, doi:10.1016/j.eswa.2023.122356

    2024

  6. [12]

    Context-aware deep learning for multi-modal depression detection,

    G. Lam, H. Dongyan, and W. Lin, “Context-aware deep learning for multi-modal depression detection,” in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. , 2019, pp. 3946–3950

    2019

  7. [13]

    A deep learn- ing method on audio and text sequences for automatic depression detection,

    J. Xiao, Y. Huang, G. Zhang, and W. Liu, “A deep learn- ing method on audio and text sequences for automatic depression detection,” in Proc. Int. Conf. Appl. Mach. Learn., 2021, pp. 388–392

    2021

  8. [14]

    Multi-modal depression detection based on emotional audio and evaluation text,

    J. Ye, Y. Yu, Q. Wang, W. Li, H. Liang, Y. Zheng, and G. Fu, “Multi-modal depression detection based on emotional audio and evaluation text,” Journal of Affective Disorders, vol. 295, pp. 904–913, 2021

    2021

  9. [15]

    Emotion regula- tion in depression: The role of biased cognition and re- duced cognitive control

    J. Joormann and W. M. Vanderlind, “Emotion regula- tion in depression: The role of biased cognition and re- duced cognitive control.” Clinical Psychological Science , vol. 2, no. 4, pp. 402–421, 2014

    2014

  10. [16]

    Improving depression level estimation by concur- rently learning emotion intensity,

    S. A. Qureshi, G. Dias, M. Hasanuzzaman, and S. Saha, “Improving depression level estimation by concur- rently learning emotion intensity,” IEEE Computational Intelligence Magazine, vol. 15, no. 3, pp. 47–59, 2020

    2020

  11. [17]

    Multi-modal and multi-task depression de- tection with sentiment assistance,

    S. Teng, S. Chai, J. Liu, T. Tateyama, L. Lin, and Y.- W. Chen, “Multi-modal and multi-task depression de- tection with sentiment assistance,” in Proc. IEEE Int. Consum. Electron., 2024, pp. 1–5

    2024

  12. [18]

    Climate and weather: Inspecting depression detection via emotion recogni- tion,

    W. Wu, M. Wu, and K. Yu, “Climate and weather: Inspecting depression detection via emotion recogni- tion,” in Proc. Conf. Int. Speech Commun. Assoc. , 2022, pp. 6262–6266

    2022

  13. [21]

    Capturing mismatch between textual and acoustic emotion expressions for mood identification in bipolar disorder,

    M. Niu, A. Romana, M. Jaiswal, M. McInnis, and E. M. Provost, “Capturing mismatch between textual and acoustic emotion expressions for mood identification in bipolar disorder,” in Proc. Conf. Int. Speech Commun. Assoc., 2023, pp. 1718–1722

    2023

  14. [22]

    Mood and emotion in major depres- sion,

    J. Rottenberg, “Mood and emotion in major depres- sion,” Current Directions in Psychological Science , vol. 14, no. 3, pp. 167–170, 2005

    2005

  15. [23]

    The distress analysis interview cor- pus of human and computer interviews,

    J. Gratch, R. Artstein, G. M. Lucas, G. Stratou, S. Scherer, A. Nazarian, R. Wood, J. Boberg, D. DeVault, S. Marsella et al. , “The distress analysis interview cor- pus of human and computer interviews,” in Proc. Int. Conf. Lang. Resour. Eval. , 2014, pp. 3123–3128

    2014

  16. [24]

    Automatic depression detection: An emotional audio-textual corpus and a gru/bilstm-based model,

    Y. Shen, H. Yang, and L. Lin, “Automatic depression detection: An emotional audio-textual corpus and a gru/bilstm-based model,” in Proc. Conf. Int. Speech Commun. Assoc., 2022, pp. 6247–6251

    2022

  17. [25]

    Analysis, synthesis, and perception of voice quality variations among female and male talkers,

    D. H. Klatt and L. C. Klatt, “Analysis, synthesis, and perception of voice quality variations among female and male talkers,” the Journal of the Acoustical Society of America, vol. 87, no. 2, pp. 820–857, 1990

    1990

  18. [26]

    Acoustical properties of speech as indicators of depression and suicidal risk,

    D. J. France, R. G. Shiavi, S. Silverman, M. Silverman, and M. Wilkes, “Acoustical properties of speech as indicators of depression and suicidal risk,” IEEE trans- actions on Biomedical Engineering , vol. 47, no. 7, pp. 829– 837, 2000

    2000

  19. [27]

    Reflections of depression in acoustic measures of the patient’s speech,

    M. Alpert, E. R. Pouget, and R. R. Silva, “Reflections of depression in acoustic measures of the patient’s speech,” Journal of affective disorders , vol. 66, no. 1, pp. 59–69, 2001

    2001

  20. [28]

    Automated depression analysis using convolutional neural networks from speech,

    L. He and C. Cao, “Automated depression analysis using convolutional neural networks from speech,” Journal of biomedical informatics , vol. 83, pp. 103–111, 2018

    2018

  21. [29]

    Depression recognition using a proposed speech chain model fusing speech production and per- ception features,

    M. Du, S. Liu, T. Wang, W. Zhang, Y. Ke, L. Chen, and D. Ming, “Depression recognition using a proposed speech chain model fusing speech production and per- ception features,” Journal of Affective Disorders , vol. 323, pp. 299–308, 2023

    2023

  22. [30]

    Pre- diction of depression severity based on transformer encoder and cnn model,

    J. Lu, B. Liu, Z. Lian, C. Cai, J. Tao, and Z. Zhao, “Pre- diction of depression severity based on transformer encoder and cnn model,” in Proc. Int. Symp. Chin. Spok. Lang. Process., 2022, pp. 339–343

    2022

  23. [31]

    A hybrid model for depression detection using deep learning,

    N. Marriwala, D. Chaudhary et al. , “A hybrid model for depression detection using deep learning,” Measurement: Sensors , vol. 25, 2023, doi:10.1016/j.measen.2022.100587

    2023

  24. [32]

    Automatic depres- sion level detection via lp-norm pooling,

    M. Niu, J. Tao, B. Liu, and C. Fan, “Automatic depres- sion level detection via lp-norm pooling,” Proc. Conf. Int. Speech Commun. Assoc. , pp. 4559–4563, 2019

    2019

  25. [33]

    Hybrid network feature extraction for depression assessment from speech,

    Z. Zhao, Q. Li, N. Cummins, B. Liu, H. Wang, J. Tao, and B. Schuller, “Hybrid network feature extraction for depression assessment from speech,” in Proc. Conf. Int. Speech Commun. Assoc. , 2020, pp. 4956–4960

    2020

  26. [34]

    A prompt-based topic-modeling method for depression detection on low-resource data,

    Y. Guo, J. Liu, L. Wang, W. Qin, S. Hao, and R. Hong, “A prompt-based topic-modeling method for depression detection on low-resource data,” IEEE T ransactions on Computational Social Systems , vol. 11, no. 1, pp. 1430– 1439, 2023

    2023

  27. [35]

    On the opportunities and risks of foundation models,

    R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosse- lut, E. Brunskill et al. , “On the opportunities and risks of foundation models,” arXiv preprint arXiv:2108.07258, 2021

    2021 arXiv

  28. [36]

    wav2vec 2.0: A framework for self-supervised learn- ing of speech representations,

    A. Baevski, Y. Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learn- ing of speech representations,” Advances in neural in- formation processing systems , vol. 33, pp. 12 449–12 460, 2020

    2020

  29. [37]

    Hubert: Self- supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y.-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self- supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM transactions on audio, speech, and language processing , vol. 29, pp. 3451– 3460, 2021

    2021

  30. [38]

    Robust speech recog- nition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recog- nition via large-scale weak supervision,” in Proc. Int. Conf. Mach. Learn. , 2023, pp. 28 492–28 518

    2023

  31. [39]

    Wavlm: Large-scale self-supervised pre-training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y. Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao et al. , “Wavlm: Large-scale self-supervised pre-training for full stack speech processing,” IEEE Journal of Selected T opics in Signal Processing, vol. 16, no. 6, pp. 1505–1518, 2022

    2022

  32. [40]

    A research on depression recognition based on voice pre-training model,

    X. Huang, Y. Liao, W. Zhang, and L. Zhang, “A research on depression recognition based on voice pre-training model,” Journal of Biomedical Engineering , vol. 41, no. 1, pp. 9–16, 2024

    2024

  33. [41]

    Self-supervised representations in speech-based depression detection,

    W. Wu, C. Zhang, and P . C. Woodland, “Self-supervised representations in speech-based depression detection,” in Proc. Conf. Int. Speech Commun. Assoc. , 2023, pp. 1–5

    2023

  34. [42]

    Depression detection in speech using transformer and parallel convolutional neural networks,

    F. Yin, J. Du, X. Xu, and L. Zhao, “Depression detection in speech using transformer and parallel convolutional neural networks,” Electronics, vol. 12, no. 2, p. 328, 2023

    2023

  35. [43]

    Detecting de- pression with word-level multimodal fusion,

    M. Rohanian, J. Hough, M. Purver et al., “Detecting de- pression with word-level multimodal fusion,” in Proc. Conf. Int. Speech Commun. Assoc. , 2019, pp. 1443–1447

    2019

  36. [44]

    End-to- end multimodal clinical depression recognition using deep neural networks: A comparative analysis,

    M. Muzammel, H. Salam, and A. Othmani, “End-to- end multimodal clinical depression recognition using deep neural networks: A comparative analysis,” Com- puter Methods and Programs in Biomedicine , vol. 211, 2021, doi:10.1016/j.cmpb.2021.106433

    2021

  37. [45]

    Distributed representations of sentences and documents,

    Q. Le and T. Mikolov, “Distributed representations of sentences and documents,” in Proc. Int. Conf. Mach. Learn., 2014, pp. 1188–1196

    2014

  38. [46]

    Xlnet: Generalized autoregressive pretraining for language understanding,

    Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. R. Salakhut- dinov, and Q. V . Le, “Xlnet: Generalized autoregressive pretraining for language understanding,” Advances in neural information processing systems , vol. 32, 2019

    2019

  39. [47]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proc. NAACL-HLT , 2019, pp. 4171–4186

    2019

  40. [48]

    A novel text mining approach for mental health prediction using bi-lstm and bert model,

    K. Zeberga, M. Attique, B. Shah, F. Ali, Y. Z. Jembre, and T.-S. Chung, “A novel text mining approach for mental health prediction using bi-lstm and bert model,” Com- putational Intelligence and Neuroscience , vol. 2022, 2022, doi:10.1155/2022/7893775

    2022 doi

  41. [49]

    Depressnet: A multimodal hierarchical atten- tion mechanism approach for depression detection,

    G. S. Saggu, K. Gupta, K. Arya, and C. R. Ro- driguez, “Depressnet: A multimodal hierarchical atten- tion mechanism approach for depression detection,” 13 International Journal of Engineering Sciences , vol. 15, no. 1, pp. 24–32, 2022

    2022

  42. [50]

    Design and implementation of attention depression detection model based on multi- modal analysis,

    J. Park and N. Moon, “Design and implementation of attention depression detection model based on multi- modal analysis,” Sustainability, vol. 14, no. 6, p. 3569, 2022

    2022

  43. [51]

    A self-rating depression scale,

    W. W. Zung, “A self-rating depression scale,” Archives of General Psychiatry , vol. 12, no. 1, pp. 63–70, 1965

    1965

  44. [52]

    A rating instrument for anxiety disorders,

    “A rating instrument for anxiety disorders,” Psychoso- matics, vol. 12, no. 6, pp. 371–379, 1971

    1971

  45. [53]

    A. J. Clark, Empathy in counseling and psychotherapy: Perspectives and practices . Routledge, 2014

    2014

  46. [54]

    Hownet-a hybrid language and knowledge resource,

    Z. Dong and Q. Dong, “Hownet-a hybrid language and knowledge resource,” in Proc. Int. Conf. Nat. Lang. Process. Knowl. Eng. , 2003, pp. 820–824

    2003

  47. [55]

    Exploring wav2vec 2.0 fine tuning for improved speech emotion recognition,

    L.-W. Chen and A. Rudnicky, “Exploring wav2vec 2.0 fine tuning for improved speech emotion recognition,” in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. , 2023, pp. 1–5

    2023

  48. [56]

    Speaker normalization for self-supervised speech emotion recognition,

    I. Gat, H. Aronowitz, W. Zhu, E. Morais, and R. Hoory, “Speaker normalization for self-supervised speech emotion recognition,” in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. , 2022, pp. 7342–7346

    2022

  49. [57]

    Speech emotion recognition using self- supervised features,

    E. Morais, R. Hoory, W. Zhu, I. Gat, M. Damasceno, and H. Aronowitz, “Speech emotion recognition using self- supervised features,” in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. , 2022, pp. 6922–6926

    2022

  50. [58]

    Compara- tive analyses of bert, roberta, distilbert, and xlnet for text-based emotion recognition,

    A. F. Adoma, N.-M. Henry, and W. Chen, “Compara- tive analyses of bert, roberta, distilbert, and xlnet for text-based emotion recognition,” in Proc. ICCWAMTIP, 2020, pp. 117–121

    2020

  51. [59]

    Roberta: A robustly optimized bert pretraining ap- proach,

    Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V . Stoyanov, “Roberta: A robustly optimized bert pretraining ap- proach,” arXiv preprint arXiv:1907.11692 , 2019

    1907 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.