Pith. sign in

REVIEW 4 major objections 7 minor 38 references

Metadata-Enhanced Speech Emotion Recognition: Augmented Residual Integration and Co-Attention in Two-Stage Fine-Tuning

T0 review · 4 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a two-stage fine-tuning scheme that feeds a speech emotion model all available metadata—speaker, gender, speech style, and transcript—reaches new state-of-the-art accuracy on IEMOCAP: 76.64–77.74 unweighted accuracy…

desk verdict A plausible incremental SER method whose own Table IV contradicts a headline claim, and whose SOTA comparison is uncontrolled. read the letter →

arxiv 2412.20707 v1 pith:MU4WEWXM submitted 2024-12-30 eess.AS cs.SD

Zixiang Wan , Ziyue Qiu , Yiyang Liu , Wei-Qiang Zhang This is my paper · ORCID
classification eess.AScs.SD
keywords speechemotionrecognitionmetadatamulti-tasklearningself-supervisedfine-tuningIEMOCAPco-attentionaugmentedresidualintegration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that speech emotion recognition improves when the model is trained to use every piece of metadata the dataset already carries—who is speaking, gender, whether the speech is scripted or improvised, and the words being said—as auxiliary tasks instead of discarding them. On the IEMOCAP benchmark, with a speaker-independent 5-fold protocol, the proposed two-stage fine-tuning with an Augmented Residual Integration module and a Co-attention module reaches 76.64–77.74 unweighted accuracy and 75.84–76.33 weighted accuracy across three self-supervised speech encoders, surpassing the published state-of-the-art numbers by 0.54–1.64 points in unweighted accuracy. The practical significance is that these gains come from restructuring fine-tuning around metadata that is usually already in the dataset, not from larger models or new data. The paper's contribution is the pairing of two mechanisms: one preserves acoustic features across transformer layers for the auxiliary tasks, and the other makes the emotion classifier attend to what those auxiliary tasks learned.

What carries the argument

The load-bearing mechanism is the Augmented Residual Integration (ARI) module, which computes $F_{\mathrm{ARI}} = [F_{12};\, W^{\top}(F_1, \dots, F_{11})]$: a learned weighted combination of the first eleven transformer layers is concatenated with the twelfth layer's output, so low- and mid-level acoustic features survive alongside high-level semantics instead of being diluted in a simple weighted sum. The Co-attention module then takes the ARI output together with the hidden representations of the four auxiliary tasks—gender, speaker, speech style, and ASR—and produces task-weighted emotion features for the final classifier. The two-stage schedule first fine-tunes all twelve transformer layers on the auxiliary-task losses while the CNN layers are frozen, then fine-tunes only the top eight transformer layers on the emotion loss while the first four layers stay frozen, preserving stage-one knowledge.

What would settle it

Re-run the strongest listed 5-fold prior system [4] under this paper's exact protocol and encoders; if the claimed 0.54–1.64 point unweighted-accuracy margin disappears or reverses, the state-of-the-art claim fails. A faster internal check: with WavLM-base, removing the ASR auxiliary task should lower UA from 77.74 to 76.60 according to Table IV, so a replication that finds no such drop would weaken the causal role the paper assigns to metadata.

Watch

Extended reading notes

Core claim

The central discovery is that combining multi-level feature preservation with task-aware attention is enough to push speech emotion recognition past previously published results on IEMOCAP. Across three self-supervised encoders—Wav2Vec-2.0-base, HuBERT-base, and WavLM-base—the full method obtains unweighted accuracy of 76.64, 76.97, and 77.74, respectively, with weighted accuracy of 75.84, 75.88, and 76.33, under a speaker-independent 5-fold protocol. The ablation study shows that each metadata auxiliary task contributes incrementally, that even speaker identification with zero test accuracy helps when combined with other tasks, and that the ARI and Co-attention modules together outperform either module alone, gaining 1.31 unweighted accuracy and 1.29 weighted accuracy over the no-module two-stage baseline. The paper also reports that the ARI module improves every auxiliary task relative to weighted-sum fusion, with the largest gains in the ASR task: a relative average improvement of 24.40% in CER and 29.75% in WER across the three encoders.

Load-bearing premise

The central comparison assumes that every prior result it beats was measured the same way—same classes merged, same speaker-independent 5-fold split, same pretrained speech encoders—but no prior system is re-run in this paper and one listed comparator used a different fold split.

Editorial extensions

If this is right

  • On IEMOCAP, the full configuration with WavLM-base reaches 77.74 unweighted accuracy and 76.33 weighted accuracy, the highest published numbers among the systems compared under the stated 5-fold speaker-independent protocol.
  • Each additional metadata auxiliary task improves the final emotion accuracy; even speaker identification, which scores zero test accuracy because train and test speakers are disjoint, helps when combined with other tasks, indicating the Co-attention module exploits relationships between tasks rather than relying on task outputs alone.
  • Replacing weighted-sum fusion with the ARI module improves every auxiliary task, most dramatically ASR: on average across the three encoders, CER falls by a relative 24.40% and WER by 29.75%.
  • The gains replicate on Wav2Vec-2.0-base, HuBERT-base, and WavLM-base, all twelve-layer transformer self-supervised encoders pretrained on LibriSpeech, supporting the claim that the approach generalizes across transformer-based self-supervised encoders.
  • The ablation study reports that the two modules together outperform either module alone, adding 1.31 unweighted accuracy and 1.29 weighted accuracy over the two-stage baseline with neither module.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An untested but direct consequence is that the approach should transfer to emotion corpora that lack native transcripts: since the ASR auxiliary task produced the largest auxiliary-task gains, feeding transcripts from an off-the-shelf recognizer could yield similar benefits on such datasets.
  • The paper's mechanism predicts that metadata diversity matters more than metadata accuracy: speaker ID has zero test accuracy yet still improves results when combined with other tasks, so the Co-attention module may act largely as a relational regularizer, which would make weakly relevant metadata such as age or dialect useful even when its own classifier is poor.
  • For practitioners, the result points to a comparatively cheap route to SER gains—freeze the encoder's CNN and lower transformer layers, fine-tune the upper layers on side tasks first, then fine-tune for emotion—without changing model size or acquiring new data; the paper only validates this on twelve-layer base encoders, so scaling behavior to larger self-supervised models is open.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes a two-stage fine-tuning framework for speech emotion recognition (SER) on self-supervised speech encoders (Wav2Vec-2.0, HuBERT, WavLM). The method uses all available metadata from IEMOCAP—gender, speaker ID, speech style, and ASR transcripts—as auxiliary tasks in a multi-task learning setup. Two architectural components are introduced: an Augmented Residual Integration (ARI) module that combines outputs from the first eleven transformer layers with the final layer output, and a Co-attention module that integrates the ARI output with auxiliary-task hidden states before the SER classifier. Evaluations on IEMOCAP under 5-fold speaker-independent cross-validation report unweighted accuracy (UA) from 76.64 to 77.74 and weighted accuracy (WA) from 75.84 to 76.33 across the three encoders, with internal ablations (Tables II–IV) showing gains over the authors' baselines. The abstract claims consistent state-of-the-art (SOTA) results, but the external comparison in Table I mixes evaluation protocols and lacks statistical validation, while certain internal claims in Section IV-D are contradicted by the reported ablation numbers.

Significance. If the results withstand a controlled comparison, this would be a practical empirical contribution: the idea of exploiting all available metadata as auxiliary tasks in a two-stage MTL schedule is reasonable, and the ARI module's use of intermediate transformer-layer features is well-motivated by prior observations in WavLM. The paper reports consistent internal improvements across three SSL encoders, which strengthens the plausibility of the approach, and the authors provide implementation details (PyTorch, SpeechBrain) that aid reproducibility. However, the central SOTA claim is not yet rigorously established due to the protocol mismatch in Table I and the absence of error bars or significance tests. In addition, several internal claims—most notably the monotonic benefit of adding auxiliary tasks in Section IV-D—are directly contradicted by the paper's own Table IV. The significance of the paper is therefore contingent on fixing these load-bearing issues.

major comments (4)
  1. [Section IV-A, Table I] The claim that the proposed method 'consistently outperforms' state-of-the-art is not rigorously supported because the comparison mixes evaluation protocols. Ye et al. [31] is reported under 10-fold speaker-independent cross-validation, whereas all other rows, including the proposed results, use 5-fold. Prior systems are not re-run under the paper's exact preprocessing (TDSA), fold splits, and hyperparameters, so the reported margins—as small as 0.54% UA over Gao et al. [4]—may reflect fold or seed variation rather than a systematic improvement. The paper should report error bars or fold-wise results, apply significance testing, or re-run competitors under the same protocol; failing that, the SOTA claim should be softened to 'competitive with' or 'favorable comparison to' published results.
  2. [Section IV-D, Table IV] The sentence 'each additional modality, regardless of its relevance to the primary task, improves performance' is directly contradicted by the reported numbers. Adding gender to SER-only decreases UA for all three encoders (Wav2Vec-2.0 72.57 to 67.91, HuBert 73.88 to 72.00, WavLM 74.65 to 74.11), and adding speaker to SER-only decreases UA for Wav2Vec-2.0 (71.55 vs. 72.57) and HuBert (70.81 vs. 73.88). Furthermore, 'with enough modalities, the MTL framework consistently outperforms SER-only model' is not supported for HuBert under four auxiliary tasks (73.54 vs. 73.88 for SER+gender+speaker+style). The claims of monotonic benefit should be revised to describe the actual pattern, including cases of negative transfer.
  3. [Section IV-B, Table II] The text attributes an average UA improvement of 0.44% and WA of 0.61% to the ARI module and 0.64% UA and 0.82% WA to the Co-attention module, but these numbers are actually the effects of adding Co-attention and ARI, respectively, to the module-free two-stage baseline. Specifically, 'Ours w/o ARI&Co' to 'Ours w/o ARI' (adding Co-attention) yields UA gains of 0.85, 0.14, and 0.33 (mean 0.44), while 'Ours w/o ARI&Co' to 'Ours w/o Co' (adding ARI) yields UA gains of 1.21, 0.43, and 0.29 (mean 0.64). The module names are reversed in the interpretation, which misstates the individual contribution of each component and should be corrected.
  4. [Section II-A-5] The Co-attention module is described only verbally ('integrates the output from the ARI module with the hidden layers of the auxiliary tasks') without equations, tensor shapes, or a clear definition of how queries, keys, and values are derived from the ARI output and the auxiliary-task hidden states. Given that Co-attention is one of the two key contributions, the description is insufficient for reproduction or for verifying that the module differs from previously published co-attention mechanisms. Please provide the formal definition and an explanation of how the weighted emotion features are combined with auxiliary hidden layers before the SER classifier.
minor comments (7)
  1. [Section III-B] The learning rates are written as '10e-4' and '10e-5'; these should be expressed as 1e-4 and 1e-5 (or 10^{-4} and 10^{-5}) to avoid ambiguity about the intended order of magnitude.
  2. [Section II-A-1] The TDSA description says 'resampling the waveform at varying rates (80, 100, and 120)'; please specify that these are percentage speed factors (e.g., 0.8, 1.0, 1.2) for clarity.
  3. [Section IV-C] The phrase 'relative average improvement of 24.40% in CER' is ambiguous; clarify that this is the mean across encoders of per-encoder relative reductions, rather than a single relative reduction computed on pooled data.
  4. [Table III footnote] The footnote explains that SpkR accuracy is not shown because train and test speakers are mutually exclusive, but it leaves unclear why a task with zero test accuracy is still used as an auxiliary task; please clarify the role of speaker ID in the multi-task framework.
  5. [Equation (3)] The notation W ∈ R^{1×11} followed by W^T [F1;...;F11] is dimensionally unclear; use a column vector of weights and define the weighted combination explicitly, e.g., F_ARI = [F12; sum_i w_i F_i] with w ∈ R^{11}.
  6. [References] Reference [29] is a URL access string; cite the original IEMOCAP publication (Busso et al., 2008, IEEE Transactions on Affective Computing) in addition to or instead of the URL.
  7. [Section IV-D] The sentence 'This can be mitigated by freezing the ARI module in Stage 1' is unsupported by any experiment; no results are shown for freezing the ARI module specifically, and the freeze schedule for the ARI module is not defined in Section II. Either provide evidence or remove this sentence.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the performance claims rest on held-out IEMOCAP evaluation, not on a derivation that re-imports its own conclusions.

full rationale

The load-bearing claims in this paper are empirical rather than derivational. The ARI module (Eq. 3) is a learned weighted combination of transformer-layer outputs concatenated with the last layer; it is an architectural choice evaluated on held-out folds, not a definition that presupposes the emotion-recognition outcome. The two-stage fine-tuning procedure trains auxiliary tasks (gender, speaker, style, ASR) with their own loss functions and then evaluates SER on speaker-independent test folds, so the reported UA/WA numbers are measured results rather than fitted constants renamed as predictions. The ablation studies compare configurations against the paper's own baselines, and the headline comparison is against externally published systems such as Gao et al. [4] and Shen et al. [30], giving the evaluation independent content. Self-citations occur (e.g., [37] for selecting SSL encoders, [25] for metadata-as-modality), but none is load-bearing: no uniqueness theorem or prior result from the same authors is invoked to forbid alternative architectures or to justify the central claim. The main weakness is that the SOTA comparison is not fully controlled because Ye et al. [31] uses 10-fold CV and prior systems are not re-run under the paper's exact splits and preprocessing; that is a benchmark-comparability and correctness-risk concern, not circularity. No step in the paper's argument reduces, by its own equations or by self-citation, to its own inputs.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the choice of three pretrained SSL encoders, the assumption that different transformer layers carry task-specific information (borrowed from WavLM), the correctness of IEMOCAP metadata, and the fairness of the SOTA comparison. Free parameters include the learned ARI weights and per-task loss weights that are not fully reported. No new physical or conceptual entities are postulated; the modules are built from standard neural network components.

free parameters (3)
  • ARI weight matrix W = R^(1x11), learned during training
    Equation 3 defines W as learned weights that combine layers 1 through 11. This is a free parameter fitted to the data and central to the ARI module.
  • Auxiliary task loss weights alpha, beta, gamma = not reported per task or encoder
    Section III-B states each auxiliary task has an independent hyperparameter that changes with the encoder, but the actual values are not listed. The reported results therefore depend on unstated choices.
  • Training hyperparameters: learning rates, batch size, epochs = lr 1e-4 downstream, 1e-5 SSL, batch 4, 100 epochs
    These are standard training choices reported in Section III-B. They are fixed in advance, but the central performance numbers still depend on them.
assumptions (5)
  • domain assumption Pretrained SSL encoders (Wav2Vec-2.0, HuBERT, WavLM) provide useful speech representations for SER after fine-tuning.
    Invoked in Section II-A2 and used as the base encoders. The paper does not independently verify this assumption.
  • domain assumption Different transformer layers of the SSL encoder carry task-specific information, e.g., speaker in layers 1 and 4, emotion in layers 8 through 10, text in layer 11.
    Cited to WavLM [27] in Section II-A4. The design of the ARI module rests on this claim, which is taken from prior literature.
  • domain assumption IEMOCAP metadata (gender, speaker, style, transcripts) is accurate and aligned with the audio.
    Section III-A describes the data preparation. If the metadata labels are noisy or misaligned, the auxiliary task losses in stage 1 are misspecified.
  • standard math CTC loss is an appropriate loss for the ASR auxiliary task.
    Section II-A3 uses Connectionist Temporal Classification for speech recognition, a standard sequence-to-sequence loss.
  • domain assumption Splitting multi-task learning into two stages avoids gradient conflicts.
    Section II-A2 justifies the two-stage schedule by citing [4], but the paper does not itself demonstrate that gradient conflicts are absent.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Metadata-Enhanced Speech Emotion Recognition: Augmented Residual Integration and Co-Attention in Two-Stage Fine-Tuning." pith.science (2026). https://pith.science/paper/MU4WEWXM

@misc{pith2026241220707,
  author       = {Pith},
  title        = {Pith review of: Metadata-Enhanced Speech Emotion Recognition: Augmented Residual Integration and Co-Attention in Two-Stage Fine-Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MU4WEWXM}},
  note         = {Machine review of arXiv:2412.20707}
}
read the original abstract

Speech Emotion Recognition (SER) involves analyzing vocal expressions to determine the emotional state of speakers, where the comprehensive and thorough utilization of audio information is paramount. Therefore, we propose a novel approach on self-supervised learning (SSL) models that employs all available auxiliary information -- specifically metadata -- to enhance performance. Through a two-stage fine-tuning method in multi-task learning, we introduce the Augmented Residual Integration (ARI) module, which enhances transformer layers in encoder of SSL models. The module efficiently preserves acoustic features across all different levels, thereby significantly improving the performance of metadata-related auxiliary tasks that require various levels of features. Moreover, the Co-attention module is incorporated due to its complementary nature with ARI, enabling the model to effectively utilize multidimensional information and contextual relationships from metadata-related auxiliary tasks. Under pre-trained base models and speaker-independent setup, our approach consistently surpasses state-of-the-art (SOTA) models on multiple SSL encoders for the IEMOCAP dataset.

Figures

Figures reproduced from arXiv: 2412.20707 by the authors.

Figure 1
Figure 1. The proposed model is trained in two stages, the first stage trains the auxiliary tasks and the second stage trains the SER with the auxiliary task [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 35 canonical work pages

  1. [31]

    Temporal modeling matters: A novel temporal emotional modeling approach for speech emotion recognition,

    J. Ye, X.-C. Wen, Y . Wei, Y . Xu, K. Liu, and H. Shan, “Temporal modeling matters: A novel temporal emotional modeling approach for speech emotion recognition,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2023

    work page 2023

  2. [4]

    Two-stage finetuning of wav2vec 2.0 for speech emotion recognition with ASR and gender pretraining,

    Y . Gao, C. Chu, and T. Kawahara, “Two-stage finetuning of wav2vec 2.0 for speech emotion recognition with ASR and gender pretraining,” in Proc. Interspeech, 2023, pp. 3637–3641

    work page 2023

  3. [1]

    A multilingual framework based on pre-training model for speech emotion recognition,

    Z. Zhang, X. Zhang, M. Guo, W.-Q. Zhang, K. Li, and Y . Huang, “A multilingual framework based on pre-training model for speech emotion recognition,” in Proc. Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) , 2021, pp. 750–755

    work page 2021

  4. [2]

    Speech emotion recognition using self-supervised features,

    E. Morais, R. Hoory, W. Zhu, I. Gat, M. Damasceno, and H. Aronowitz, “Speech emotion recognition using self-supervised features,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), 2022, pp. 6922–6926

    work page 2022

  5. [3]

    Multimodal Emotion Recognition using Transfer Learning from Speaker Recognition and BERT-based models

    S. Padi, S. O. Sadjadi, D. Manocha, and R. D. Sriram, “Multimodal emotion recognition using transfer learning from speaker recognition and bert-based models,” arXiv preprint arXiv:2202.08974 , 2022

    work page Pith review arXiv 2022

  6. [5]

    A critical review of state-of-the- art chatbot designs and applications,

    B. Luo, R. Y . Lau, C. Li, and Y .-W. Si, “A critical review of state-of-the- art chatbot designs and applications,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery , vol. 12, no. 1, 2022, art. No. e1434

    work page 2022

  7. [6]

    5IDER: Unified query rewriting for steering, intent carryover, disfluencies, entity carryover and repair,

    J. Lu, B.-H. Tseng, J. R. A. Moniz, S. Li, X. Zhu, H. Yu, and M. Ak- bacak, “5IDER: Unified query rewriting for steering, intent carryover, disfluencies, entity carryover and repair,” in Proc. Interspeech, 2023, pp. 760–764

    work page 2023

  8. [7]

    Cross- lingual/cross-channel intent detection in contact-center conversations,

    S. Agrawal, A. Sachdeva, S. Jain, C. George, and J. Vepa, “Cross- lingual/cross-channel intent detection in contact-center conversations,” in Proc. Interspeech, 2023, pp. 5269–5270

    work page 2023

Show all 38 references
  1. [8]

    Automated neural nursing assistant (ANNA): An over-the-phone system for cogni- tive monitoring,

    J. Solinsky, R. Finzel, M. Michalowski, and S. Pakhomov, “Automated neural nursing assistant (ANNA): An over-the-phone system for cogni- tive monitoring,” in Proc. Interspeech, 2023, pp. 684–685

    2023

  2. [9]

    Whisper-based transfer learning for alzheimer disease classification: Leveraging speech segments with full transcripts as prompts,

    J. Li and W.-Q. Zhang, “Whisper-based transfer learning for alzheimer disease classification: Leveraging speech segments with full transcripts as prompts,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2024, pp. 11 211–11 215

    2024

  3. [10]

    Cross-lingual alzheimer’s disease detection based on paralinguistic and pre-trained features,

    X. Chen, Y . Pu, J. Li, and W.-Q. Zhang, “Cross-lingual alzheimer’s disease detection based on paralinguistic and pre-trained features,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023

    2023

  4. [11]

    Exploiting emotion information in speaker embeddings for expressive text-to-speech,

    Z. Shaheen, T. Sadekova, Y . Matveeva, A. Shirshova, and M. Kudinov, “Exploiting emotion information in speaker embeddings for expressive text-to-speech,” in Proc. Interspeech, 2023, pp. 2038–2042

    2023

  5. [12]

    Fusing ASR outputs in joint training for speech emotion recognition,

    Y . Li, P. Bell, and C. Lai, “Fusing ASR outputs in joint training for speech emotion recognition,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2022, pp. 7362– 7366

    2022

  6. [13]

    Speaker-aware training of speech emotion classifier with speaker recognition,

    L. Savchenko and A. V Savchenko, “Speaker-aware training of speech emotion classifier with speaker recognition,” in Proc. International Conference on Speech and Computer (SPECOM) , 2021, pp. 614–625

    2021

  7. [14]

    Speech emotion recognition combining acoustic features and linguistic information in a hybrid support vector machine-belief network architecture,

    B. Schuller, G. Rigoll, and M. Lang, “Speech emotion recognition combining acoustic features and linguistic information in a hybrid support vector machine-belief network architecture,” in Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP),...

    2004

  8. [15]

    Mmer: Multimodal multi-task learning for speech emotion recognition,

    S. Ghosh, U. Tyagi, S. Ramaneswaran, H. Srivastava, and D. Manocha, “Mmer: Multimodal multi-task learning for speech emotion recognition,” in Proc. Interspeech, 2023, pp. 1209–1213

    2023

  9. [16]

    Speech emotion recognition using decomposed speech via multi-task learning,

    J.-H. Hsu, C.-H. Wu, and Y .-H. Wei, “Speech emotion recognition using decomposed speech via multi-task learning,” in Proc. Interspeech, 2023, pp. 4553–4557

    2023

  10. [17]

    Multi-task learning based end-to-end speaker recognition,

    Y . Pan and W.-Q. Zhang, “Multi-task learning based end-to-end speaker recognition,” in Proc. International Conference on Signal Processing and Machine Learning (SPML) , 2019, p. 56–61

    2019

  11. [18]

    Mutitask learning based muti-examples keywords spotting in low resource condition,

    J. Yang, J. Kang, W.-Q. Zhang, and J. Liu, “Mutitask learning based muti-examples keywords spotting in low resource condition,” in Proc. IEEE International Conference on Signal Processing (ICSP) , 2018, pp. 581–585

    2018

  12. [19]

    MMER: Multimodal multi-task learning for speech emotion recogni- tion,

    S. Ghosh, U. Tyagi, S. Ramaneswaran, H. Srivastava, and D. Manocha, “MMER: Multimodal multi-task learning for speech emotion recogni- tion,” in Proc. Interspeech, 2023, pp. 1209–1213

    2023

  13. [20]

    A review of speech emotion recognition: Datasets, features, and machine learning algorithms,

    B. Schuller, A. Batliner, and S. Steidl, “A review of speech emotion recognition: Datasets, features, and machine learning algorithms,” IEEE Transactions on Affective Computing , vol. 11, pp. 589–605, 2020

    2020

  14. [21]

    Gra- dient surgery for multi-task learning,

    T. Yu, S. Kumar, A. Gupta, S. Levine, K. Hausman, and C. Finn, “Gra- dient surgery for multi-task learning,” Advances in Neural Information Processing Systems, vol. 33, pp. 5824–5836, 2020

    2020

  15. [22]

    Deep auxiliary learning for visual localiza- tion and odometry,

    A. Valada and W. Burgard, “Deep auxiliary learning for visual localiza- tion and odometry,” https://ar5iv.labs.arxiv.org/html/2007.01126, 2018

    2007 arXiv

  16. [23]

    Exploring large scale pre-trained models for robust machine anomalous sound detection,

    B. Han, Z. Lv, A. Jiang, W. Huang, Z. Chen, Y . Deng, J. Ding, C. Lu, W.- Q. Zhang, P. Fan, J. Liu, and Y . Qian, “Exploring large scale pre-trained models for robust machine anomalous sound detection,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Pro...

    2024

  17. [24]

    Ef- ficient multi-task auxiliary learning: Selecting auxiliary data by feature similarity,

    P.-N. Kung, S.-S. Yin, Y .-C. Chen, T.-H. Yang, and Y .-N. Chen, “Ef- ficient multi-task auxiliary learning: Selecting auxiliary data by feature similarity,” in Proc. 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP) , 2021, pp. 416–428

    2021

  18. [25]

    Towards discriminative representations and unbiased predictions: Class-specific angular softmax for speech emotion recognition

    Z. Li, L. He, J. Li, L. Wang, and W.-Q. Zhang, “Towards discriminative representations and unbiased predictions: Class-specific angular softmax for speech emotion recognition.” in Proc. Interspeech, 2019, pp. 1696– 1700

    2019

  19. [26]

    HuBERT: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “HuBERT: Self-supervised speech representation learning by masked prediction of hidden units,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021, ...

    2021

  20. [27]

    WavLM: Large-scale self-supervised pre- training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao, J. Wu, L. Zhou, S. Ren, Y . Qian, Y . Qian, J. Wu, M. Zeng, X. Yu, and F. Wei, “WavLM: Large-scale self-supervised pre- training for full stack speech processing,” IEEE Journal of Select...

    2022

  21. [28]

    Connectionist temporal classification: labelling unsegmented sequence data with re- current neural networks,

    A. Graves, S. Fern ´andez, F. Gomez, and J. Schmidhuber, “Connectionist temporal classification: labelling unsegmented sequence data with re- current neural networks,” in Proc. International Conference on Machine Learning (ICML), 2006, pp. 369–376

    2006

  22. [29]

    IEMOCAP: Interactive emotional dyadic motion capture database,

    C. Busso, M. Bulut, C.-C. Lee, A. Kazemzadeh, E. Mower, S. Kim, J. N. Chang, S. Lee, and S. S. Narayanan, “IEMOCAP: Interactive emotional dyadic motion capture database,” https://sail.usc.edu/iemocap/, 2008, accessed: 2024-08-24

    2008

  23. [30]

    Mingling or misalignment? temporal shift for speech emotion recognition with pre-trained representations,

    S. Shen, F. Liu, and A. Zhou, “Mingling or misalignment? temporal shift for speech emotion recognition with pre-trained representations,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023

    2023

  24. [32]

    DST: Deformable speech transformer for emotion recognition,

    W. Chen, X. Xing, X. Xu, J. Pang, and L. Du, “DST: Deformable speech transformer for emotion recognition,” inProc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023

    2023

  25. [33]

    DWFormer: Dynamic window transformer for speech emotion recognition,

    S. Chen, X. Xing, W. Zhang, W. Chen, and X. Xu, “DWFormer: Dynamic window transformer for speech emotion recognition,” in Proc. 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023

    2023

  26. [34]

    Multiple acoustic features speech emotion recognition using cross-attention transformer,

    Y . He, N. Minematsu, and D. Saito, “Multiple acoustic features speech emotion recognition using cross-attention transformer,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023

    2023

  27. [35]

    PyTorch: An imperative style, high-performance deep learning library,

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al. , “PyTorch: An imperative style, high-performance deep learning library,” Advances in Neural Information Processing systems , vol. 32, 2019

    2019

  28. [36]

    SpeechBrain: A general-purpose speech toolkit,

    M. Ravanelli, T. Parcollet, P. Plantinga, A. Rouhe, S. Cornell, L. Lu- gosch, C. Subakan, N. Dawalatabad, A. Heba, J. Zhong et al. , “SpeechBrain: A general-purpose speech toolkit,” arXiv preprint arXiv:2106.04624, 2021

    2021 arXiv

  29. [37]

    Improving automatic speech recognition performance for low-resource languages with self-supervised models,

    J. Zhao and W.-Q. Zhang, “Improving automatic speech recognition performance for low-resource languages with self-supervised models,” IEEE Journal of Selected Topics in Signal Processing , vol. 16, no. 6, pp. 1227–1241, 2022

    2022

  30. [38]

    Librispeech: an ASR corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: an ASR corpus based on public domain audio books,” in Proc. IEEE international conference on acoustics, speech and signal processing (ICASSP), 2015, pp. 5206–5210

    2015

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.