Pith. sign in

REVIEW 3 major objections 6 minor 35 references

PARROT: Synergizing Mamba and Attention-based SSL Pre-Trained Models via Parallel Branch Hadamard Optimal Transport for Speech Emotion Recognition

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that PARROT, a parallel Hadamard-product and optimal-transport fusion of a Mamba-based and an attention-based self-supervised speech model, outperforms individual models, homogeneous attention-based fusion, and a…

desk verdict The fusion idea is new and the A(B)+H results are promising, but the paper's own Table 1 vs Table 2 contradicts its SOTA claim on MESD, and the missing error bars make the small gains unverifiable. read the letter →

arxiv 2506.01138 v1 pith:PJ4WB3ES submitted 2025-06-01 eess.AS cs.SD

Orchid Chetia Phukan , Mohd Mujtaba Akhtar , Girish , Swarup Ranjan Behera , Jaya Sai Kiran Patibandla , Arun Balaji Buduru , Rajesh Sharma This is my paper
classification eess.AScs.SD
keywords SpeechEmotionRecognitionSelf-SupervisedPre-TrainedModelsMambaState-SpaceAttention-BasedOptimalTransportHadamardProductHeterogeneousFusionSinkhornAlgorithm
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to establish that Mamba-based and attention-based self-supervised speech models are complementary for speech emotion recognition, and that fusing the two families can beat fusing two attention-based models. It introduces PARROT, which combines a Hadamard-product branch for local element-wise interaction with an optimal-transport branch for global distribution alignment. Across CREMA-D, Emo-DB, and MESD, the best PARROT pairings improve on every individual pre-trained model and on a concatenation baseline, with top accuracies of 73.68, 92.24, and 69.05 respectively. If that holds, heterogeneous fusion is a practical way to push SER accuracy beyond what any single self-supervised model delivers.

What carries the argument

PARROT projects pooled last-hidden representations from two frozen PTMs through 1D convolutional blocks into a 120-dimensional latent space. The Hadamard branch computes $R_p \odot R_q$ element-wise, preserving fine-grained local structure. The optimal-transport branch builds a cost matrix $C = \|R_p - R_q\|_2 / \max(\|R_p - R_q\|_2)$, applies Sinkhorn to get a transport plan $\Gamma = \mathrm{Sinkhorn}(C)$, and transports each representation into the other's space; transported and original representations are then concatenated. The two branch outputs are concatenated and passed to an FCN classifier, so local interaction and global distribution alignment are combined in one network.

What would settle it

Run the same five-fold protocol with identical seeds and multiple restarts for PARROT and its concatenation-only ablation on CREMA-D, Emo-DB, and MESD, and report confidence intervals for accuracy and macro-F1; if concatenation matches or beats PARROT on any dataset, or if PARROT no longer beats the best individual PTM, the central claim fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the core discovery is that fusing a Mamba-based SSL pre-trained model with an attention-based SSL pre-trained model through PARROT yields speech emotion recognition that outperforms every individual PTM tested, every homogeneous attention-based fusion, and every concatenation baseline. The strongest reported results are Audio-MAMBA(base) plus HuBERT at 73.68 accuracy / 72.90 macro-F1 on CREMA-D and 92.24 / 91.53 on Emo-DB, and Audio-MAMBA(base) plus MMS at 69.05 / 68.72 on MESD. The paper reads these results as evidence that the two architecture families capture complementary strengths, with attention models handling global dependencies and Mamba models handling efficient long-range processing, and that PARROT's two-branch design is what lets that complementarity emerge.

Load-bearing premise

The load-bearing premise is that the concatenation ablation is an honest control for PARROT; the paper gives no tuning details, error bars, or significance tests for that baseline, so the claimed superiority over baseline fusion could in principle be random variation.

Editorial extensions

If this is right

  • SER systems can combine Mamba-based and attention-based SSL models instead of choosing one family, since PARROT improves over every individual PTM on all three datasets.
  • Reported reference points for future comparisons: Audio-MAMBA(base)+HuBERT gives 73.68 accuracy on CREMA-D and 92.24 on Emo-DB, while Audio-MAMBA(base)+MMS gives 69.05 on MESD.
  • Dataset-dependent pairing matters: MMS, the multilingual model, produces the best fusion only on the Spanish dataset, implying that PTM language coverage is part of the fusion decision.
  • Because PARROT beats homogeneous attention-based fusion, the paper's hypothesis that heterogeneous fusion is richer than homogeneous fusion is supported on these benchmarks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the mechanism is general, the same two-branch recipe could be applied to other tasks where heterogeneous SSL PTM ensembles are used, such as speech recognition or audio deepfake detection; the paper only demonstrates it for SER.
  • The paper reports margins over concatenation of roughly 1 to 3 accuracy points with no variance or significance testing, so one natural next experiment is a multi-seed comparison with confidence intervals before treating the gains as stable.
  • An ablation the paper leaves implicit, running the Hadamard branch alone, the OT branch alone, and varying projection dimension or Sinkhorn regularization, would show which branch carries the gain; that is an editorial suggestion, not a paper claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes PARROT, a fusion framework for speech emotion recognition (SER) that combines one Mamba-based self-supervised pre-trained model (Audio-MAMBA) with one attention-based PTM. The framework applies parallel branches: a Hadamard product branch for element-wise local feature interaction and an optimal transport branch (Sinkhorn) for global distribution alignment, followed by concatenation and an FCN classifier. The authors evaluate on CREMA-D, Emo-DB, and MESD using five-fold cross-validation, compare against individual PTMs with SVM, FCN, and CNN downstreams, and against a concatenation baseline, and claim state-of-the-art results for heterogeneous fusion.

Significance. If the central claims held, the paper would be a useful empirical contribution on heterogeneous Mamba/attention fusion for SER, with a relatively lightweight fusion head (3.2M-13M parameters) over frozen PTMs and a public code repository. The methodology is not circular: the PTM features come from fixed external checkpoints, and the fusion head is trained on the target datasets. However, the reported evidence does not support the headline claim. The paper's own tables contain a direct counterexample on MESD, and the lack of statistical reliability measures makes it impossible to separate the reported gains from random variation.

major comments (3)
  1. [Section 4.2, Table 2 vs Table 1] The abstract claims SOTA results against individual PTMs, but on MESD the highlighted PARROT fusion A(B)+M reaches 69.05 accuracy, while the individual MMS CNN model in Table 1 reaches 83.24 on the same protocol, a 14.19-point deficit. Several other PARROT rows also underperform their best constituent (e.g., A(B)+U on MESD: 38.15 vs A(B) at 78.03; W2+M on MESD: 71.10 vs M at 83.24). This is not a missing baseline or missing error bar; it is an internal inconsistency between the headline SOTA claim and the reported numbers, and it directly contradicts the Section 4.2 statement that 'fusion of different PTMs through PARROT achieves better performance than the individual PTMs across all the datasets.'
  2. [Section 4.2, Table 2] No confidence intervals, standard deviations, or significance tests are reported for the five-fold cross-validation. Many PARROT gains over the concatenation baseline are small (e.g., A(T)+H on CREMA-D: 66.82 vs 65.54; A(S)+W2 on CREMA-D: 62.34 vs 61.94), and PARROT is worse than concatenation on several entries (e.g., W2+M on MESD: 71.10 vs 72.14). Without error bars or paired tests, the claimed superiority of PARROT over baseline fusion techniques cannot be separated from random variation.
  3. [Section 4.2, concatenation baseline] The concatenation baseline is described only as PARROT with the optimal transport and Hadamard product branches removed, with no details on how the remaining projections or classifier were tuned (e.g., learning rate, dropout, early stopping criteria), and it is not stated whether the same 120-dimensional projection is used. Since the central comparison is PARROT versus this baseline, the lack of an independently tuned standard fusion baseline weakens the empirical claim.
minor comments (6)
  1. [Introduction] The text contains typos and awkward phrases: 'By the end of end of last decade' should be 'By the end of the last decade', 'conventational features' should be 'conventional features', and 'topmost most performance' should be 'topmost performance'.
  2. [Section 2] The sentence 'We representations are of dimensions...' should read 'The representations are of dimensions...'.
  3. [Section 4.2] The phrases 'This brings out limelight the effect' and 'In contrast, the some of the attention-based PTMs' are ungrammatical and should be revised.
  4. [Section 3.1] The Sinkhorn regularization parameter is not reported, although it is a free parameter that affects the transport plan and the results; please provide the value and any sensitivity analysis.
  5. [Figure 1] The caption is dense and the labels U11, U22, U12, U21 are not defined in the main text; please define them explicitly in the caption or in Section 3.1.
  6. [Section 4.1] It is not stated whether the five-fold cross-validation is speaker-independent; given that speaker overlap can inflate SER results, this should be clarified.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: PARROT is an empirically trained fusion head on fixed external SSL features; the MESD SOTA contradiction is a correctness issue, not circularity.

full rationale

The paper's contribution is an empirical fusion method rather than a derived first-principles result. The inputs are frozen SSL representations from external checkpoints (Audio-MAMBA and attention-based PTMs), and the PARROT head is trained and evaluated with five-fold cross-validation on CREMA-D, Emo-DB, and MESD. No equation defines a reported accuracy as the fitted value of a parameter, no transported feature is constructed from the target labels, and no evaluation result is forced by the way PARROT is defined. The Hadamard and OT branches are new architectural choices applied to the fixed representations; Eq. (1) and the Sinkhorn step are standard operations, not a self-citation. The two self-citations in the paper ([10], [19]) are motivational and do not carry the load of the experimental claim, which rests on the tables comparing PARROT with individual PTMs, homogeneous fusions, and concatenation baselines. The paper's most serious problem is an internal empirical inconsistency, not circularity: the abstract claims SOTA against individual PTMs, but in Table 1 the individual MMS model reaches 83.24 accuracy on MESD, while the highlighted PARROT fusion A(B)+M in Table 2 reaches only 69.05. This and the lack of significance tests are correctness and statistical concerns, not a reduction of outputs to inputs. Therefore no circular step is identified and the circularity score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

No new physical entities or theoretical constructs are introduced. The only methodological choices are architectural hyperparameters plus the unstated Sinkhorn regularization. The main load-bearing external input is the set of frozen SSL checkpoints, which are standard and publicly available.

free parameters (3)
  • latent projection dimension = 120
    Chosen by hand, presumably by trial and error, with no ablation or sensitivity analysis in the paper.
  • CNN filter counts and kernel size = 64/128 filters, kernel 3
    Fixed by hand for the downstream and fusion heads; no ablation or justification provided.
  • Sinkhorn regularization parameter = not reported
    The Sinkhorn algorithm requires a regularization strength epsilon and a number of iterations; the paper specifies neither, so these are hidden free parameters.
assumptions (3)
  • domain assumption Frozen SSL features from Audio-MAMBA, WavLM, HuBERT, wav2vec2, UniSpeech-SAT and MMS are informative for SER in the languages tested.
    The whole pipeline pools last-layer representations from frozen checkpoints and assumes they carry useful emotion information; this is standard practice and supported by prior work, but it is an input assumption.
  • domain assumption Five-fold cross-validation with four training folds and one test fold gives a reliable estimate of generalization.
    The datasets are small (535 utterances for Emo-DB, 864 for MESD), so fold-level variance is likely high; without per-fold results or error bars, the averaged numbers are fragile.
  • ad hoc to paper The concatenation baseline is a faithful control that differs from PARROT only by the absence of the OT and Hadamard branches.
    The paper states this, but gives no details on the baseline's projection dimensions, classifier, or tuning, and the inconsistent Fp definition makes the exact baseline composition unclear.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PARROT: Synergizing Mamba and Attention-based SSL Pre-Trained Models via Parallel Branch Hadamard Optimal Transport for Speech Emotion Recognition." pith.science (2026). https://pith.science/paper/PJ4WB3ES

@misc{pith2026250601138,
  author       = {Pith},
  title        = {Pith review of: PARROT: Synergizing Mamba and Attention-based SSL Pre-Trained Models via Parallel Branch Hadamard Optimal Transport for Speech Emotion Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PJ4WB3ES}},
  note         = {Machine review of arXiv:2506.01138}
}
read the original abstract

The emergence of Mamba as an alternative to attention-based architectures has led to the development of Mamba-based self-supervised learning (SSL) pre-trained models (PTMs) for speech and audio processing. Recent studies suggest that these models achieve comparable or superior performance to state-of-the-art (SOTA) attention-based PTMs for speech emotion recognition (SER). Motivated by prior work demonstrating the benefits of PTM fusion across different speech processing tasks, we hypothesize that leveraging the complementary strengths of Mamba-based and attention-based PTMs will enhance SER performance beyond the fusion of homogenous attention-based PTMs. To this end, we introduce a novel framework, PARROT that integrates parallel branch fusion with Optimal Transport and Hadamard Product. Our approach achieves SOTA results against individual PTMs, homogeneous PTMs fusion, and baseline fusion techniques, thus, highlighting the potential of heterogeneous PTM fusion for SER.

Figures

Figures reproduced from arXiv: 2506.01138 by the authors.

Figure 1
Figure 1. Proposed Framework: PARROT; OTFB, HPFB stands for Optimal Transport Fusion Block and Hadamard Product Fusion Block respectively; U11, U22 represents the representational space of PTM 1 and PTM2; U12, U21 represents the transported space of PTM 2 to PTM 1 and vice versa Training Details: All models are trained using the Adam opti￾mizer with cross-entropy loss. The learning rate is 1e-3, batch size 32, and training ru… view at source ↗
Figure 2
Figure 2. t-SNE plots for CREMA-D:(a) PARROT with Audio￾MAMBA(base) and HuBERT (b) PARROT with Audio-MAMBA (base) and Wav2vec2 (a) (b) [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Confusion matrices for PARROT with Audio￾MAMBA(base) and HuBERT : (a) CREMA-D (b) EMO-DB; x-axis and y-axis represents predicted and true, respectively ther, fusion of different PTMs through PARROT achieves better performance than the individual PTMs across all the datasets. In contrast, fusion of PTMs using concatenation-based fusion overall shows comparable or less performance than individual PTMs except a few spe… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 18 canonical work pages

  1. [1]

    PARROT: Synergizing Mamba and Attention-based SSL Pre-Trained Models via Parallel Branch Hadamard Optimal Transport for Speech Emotion Recognition

    Introduction Speech Emotion Recognition (SER) bridges human-computer interaction, finds applications in mental health monitoring as well as in empathetic AI systems [ 1, 2]. It enables machines to understand and respond to human emotions, fostering more natural and intuitive interactions. Traditional SER research often employs handcrafted features such as...

    work page Pith review arXiv 2025

  2. [2]

    Pre-Trained Models In this section, we discuss the PTMs considered in our study. Audio-MAMBA [13]1: Audio Mamba is a selective state space model that is trained in a self-supervised fusion to learn general- purpose representations from randomly masked spectrogram patches. Trained on the AudioSet dataset, it outperforms its attention-based counterparts bas...

    work page 1920

  3. [3]

    We make use of SVM, Fully Con- nected Network (FCN), and CNN as the downstreams modeling with individual PTMs

    Modeling Pipeline In this section, we discuss the downstream modeling networks to be employed with individual PTMs and the proposed framework for aligning PTMs, PARROT. We make use of SVM, Fully Con- nected Network (FCN), and CNN as the downstreams modeling with individual PTMs. For SVM, we kept the default hyperpa- rameters. For CNN, we make use of two 1...

    work page

  4. [4]

    Experiments 4.1. Benchmark Datasets Crowd-Sourced Emotional Multimodal Actors Dataset (CREMA-D) [ 25] serves as a widely recognized benchmark for SER and comprising 7,442 utterances from 48 male and 43 female actors, it spans a diverse range of speaker ages and eth- nicities. This dataset includes six distinct emotional categories: anger, happiness, sadne...

    work page

  5. [5]

    To this end, we propose, PARROT, a novel framework that synergizes PTMs via parallel branch fusion of Optimal Transport and Hadamard Product

    Conclusion In this study, we explore the heterogeneous fusion of mamba and attention-based SSL PTMs for SER. To this end, we propose, PARROT, a novel framework that synergizes PTMs via parallel branch fusion of Optimal Transport and Hadamard Product. With PARROT, through the fusion of mamba and attention-based PTMs, we report SOTA performance in compariso...

    work page

  6. [6]

    Speech emotion recognition using deep 1d & 2d cnn lstm networks,

    J. Zhao, X. Mao, and L. Chen, “Speech emotion recognition using deep 1d & 2d cnn lstm networks,” Biomedical signal processing and control, vol. 47, pp. 312–323, 2019

    work page 2019

  7. [7]

    Towards a small set of robust acoustic features for emotion recognition: Challenges,

    M. Tahon and L. Devillers, “Towards a small set of robust acoustic features for emotion recognition: Challenges,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 24, pp. 16–28, 2016. [Online]. Available: https://api.semanticscholar.org/CorpusID:8157773

    2016

  8. [8]

    Is everything fine, grandma? acoustic and linguistic modeling for robust elderly speech emotion recognition,

    G. Souganciouglu, O. Verkholyak, H. Kaya, D. Fedotov, T. Cadee, A. A. Salah, and A. Karpov, “Is everything fine, grandma? acoustic and linguistic modeling for robust elderly speech emotion recognition,” in Interspeech, 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:221535008

    2020

Show all 35 references
  1. [9]

    Speaker dependent speech emotion recognition using mfcc and support vector machine,

    P. P. Dahake, K. Shaw, and P. Malathi, “Speaker dependent speech emotion recognition using mfcc and support vector machine,” in 2016 International Conference on Automatic Control and Dynamic Optimization Techniques (ICACDOT). IEEE, 2016, pp. 1080– 1084

    2016

  2. [10]

    V ocal- based emotion recognition using random forests and decision tree,

    F. Noroozi, T. Sapi´nski, D. Kami´nska, and G. Anbarjafari, “V ocal- based emotion recognition using random forests and decision tree,” International Journal of Speech Technology , vol. 20, no. 2, pp. 239–246, 2017

    2017

  3. [11]

    Deep learning techniques for speech emotion recognition, from databases to models,

    B. J. Abbaschian, D. Sierra-Sosa, and A. Elmaghraby, “Deep learning techniques for speech emotion recognition, from databases to models,” Sensors, vol. 21, no. 4, p. 1249, 2021

    2021

  4. [12]

    Speech emotion recognition using self-supervised features,

    E. Morais, R. Hoory, W. Zhu, I. Gat, M. Damasceno, and H. Aronowitz, “Speech emotion recognition using self-supervised features,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2022, pp. 6922–6926

    2022

  5. [13]

    Ensemble deep learning with hubert for speech emo- tion recognition,

    J. Yang, “Ensemble deep learning with hubert for speech emo- tion recognition,” in 2023 IEEE 17th International Conference on Semantic Computing (ICSC), 2023, pp. 153–154

    2023

  6. [14]

    Evaluating self-supervised speech representations for speech emotion recognition,

    B. T. Atmaja and A. Sasou, “Evaluating self-supervised speech representations for speech emotion recognition,” IEEE Access, vol. 10, pp. 124 396–124 407, 2022

    2022

  7. [15]

    Ser evals: In-domain and out-of-domain benchmarking for speech emotion recognition,

    M. Osman, D. Z. Kaplan, and T. Nadeem, “Ser evals: In-domain and out-of-domain benchmarking for speech emotion recognition,” in Interspeech 2024, 2024, pp. 1395–1399

    2024

  8. [16]

    Are paralinguistic representations all that is needed for speech emotion recognition?

    O. C. Phukan, G. S. Kashyap, A. B. Buduru, and R. Sharma, “Are paralinguistic representations all that is needed for speech emotion recognition?” in Interspeech 2024, 2024, pp. 4698–4702

    2024

  9. [17]

    Emotion recognition from speech using wav2vec 2.0 embeddings,

    L. Pepino, P. Riera, and L. Ferrer, “Emotion recognition from speech using wav2vec 2.0 embeddings,” inInterspeech 2021, 2021, pp. 3400–3404

    2021

  10. [18]

    Investi- gation of ensemble features of self-supervised pretrained models for automatic speech recognition,

    A. Arunkumar, V . Nileshkumar Sukhadia, and S. Umesh, “Investi- gation of ensemble features of self-supervised pretrained models for automatic speech recognition,” in Interspeech 2022, 2022, pp. 5145–5149

    2022

  11. [19]

    Audio mamba: Selective state spaces for self-supervised audio representations,

    S. Yadav and Z.-H. Tan, “Audio mamba: Selective state spaces for self-supervised audio representations,” in Interspeech 2024, 2024, pp. 552–556

    2024

  12. [20]

    Audio mamba: Bidirectional state space model for audio representation learning,

    M. H. Erol, A. Senocak, J. Feng, and J. S. Chung, “Audio mamba: Bidirectional state space model for audio representation learning,” IEEE Signal Processing Letters, vol. 31, pp. 2975–2979, 2024

    2024

  13. [21]

    Ssamba: Self-supervised audio representation learning with mamba state space model,

    S. Shams, S. S. Dindar, X. Jiang, and N. Mesgarani, “Ssamba: Self-supervised audio representation learning with mamba state space model,” arXiv preprint arXiv:2405.11831, 2024

    2024 arXiv

  14. [22]

    Mamba: Linear-time sequence modeling with selective state spaces,

    A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752, 2023

    2023 arXiv

  15. [23]

    Investigation of ensemble of self-supervised models for speech emotion recognition,

    Y . Wu, P. Yue, C. Cheng, and T. Li, “Investigation of ensemble of self-supervised models for speech emotion recognition,” in 2023 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). IEEE, 2023, pp. 988–995

    2023

  16. [24]

    Scaling speech technology to 1,000+ languages,

    V . Pratap, A. Tjandra, B. Shi, P. Tomasello, A. Babu, S. Kundu, A. Elkahky, Z. Ni, A. Vyas, M. Fazel-Zarandi et al., “Scaling speech technology to 1,000+ languages,” Journal of Machine Learning Research, vol. 25, no. 97, pp. 1–52, 2024

    2024

  17. [25]

    Heterogeneity over homogeneity: Investigating multilingual speech pre-trained models for detecting audio deepfake,

    O. Chetia Phukan, G. Kashyap, A. B. Buduru, and R. Sharma, “Heterogeneity over homogeneity: Investigating multilingual speech pre-trained models for detecting audio deepfake,” in Findings of the Association for Computational Linguistics: NAACL 2024, K. Duh, H. Gomez, and S. Be...

    2024

  18. [26]

    Wavlm: Large-scale self-supervised pre-training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiaoet al., “Wavlm: Large-scale self-supervised pre-training for full stack speech processing,” IEEE Journal of Selected Topics in Signal Processing , vol. 16, no. 6, pp. 1505– 1518, 2022

    2022

  19. [27]

    is Mexican-spanish database containing 864 utterances rep- resenting six emotional states: anger, disgust, fear, happiness, neutral, and sadness. O T Concat CONCAT Concat U11 U22 U21U12 HPFB Y1 Y2 Y3 - - Yn X1Y1 X2Y2 X3Y3 - - XnYn X1 X2 X3 - - Xn OTFB Flatten Flatten Input Max...

  20. [28]

    Unispeech-sat: Universal speech repre- sentation learning with speaker aware pre-training,

    S. Chen, Y . Wu, C. Wang, Z. Chen, Z. Chen, S. Liu, J. Wu, Y . Qian, F. Wei, J. Li, and X. Yu, “Unispeech-sat: Universal speech repre- sentation learning with speaker aware pre-training,” ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processi...

    2022

  21. [29]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Advances in neural information processing systems , vol. 33, pp. 12 449–12 460, 2020

    2020

  22. [30]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM trans- actions on audio, speech, and language processing , vol. 29, pp. 3451–3460, 2021

    2021

  23. [31]

    Crema-d: Crowd-sourced emotional multimodal actors dataset,

    H. Cao, D. G. Cooper, M. K. Keutmann, R. C. Gur, A. Nenkova, and R. Verma, “Crema-d: Crowd-sourced emotional multimodal actors dataset,” IEEE transactions on affective computing, vol. 5, no. 4, pp. 377–390, 2014

    2014

  24. [32]

    A database of german emotional speech

    F. Burkhardt, A. Paeschke, M. Rolfes, W. F. Sendlmeier, B. Weiss et al., “A database of german emotional speech.” in Interspeech, vol. 5, 2005, pp. 1517–1520

    2005

  25. [33]

    The mexican emotional speech database (mesd): elaboration and assessment based on machine learning,

    M. M. Duville, L. M. Alonso-Valerdi, and D. I. Ibarra-Zarate, “The mexican emotional speech database (mesd): elaboration and assessment based on machine learning,” in 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, ...

    2021

  26. [34]

    Speech self-supervised representation benchmarking: Are we do- ing it right?

    S. Zaiem, Y . Kemiche, T. Parcollet, S. Essid, and M. Ravanelli, “Speech self-supervised representation benchmarking: Are we do- ing it right?” in Interspeech 2023, 2023, pp. 2873–2877

    2023

  27. [35]

    Superb: Speech pro- cessing universal performance benchmark,

    S. wen Yang, P.-H. Chi, Y .-S. Chuang, C.-I. J. Lai, K. Lakhotia, Y . Y . Lin, A. T. Liu, J. Shi, X. Chang, G.-T. Lin, T.-H. Huang, W.-C. Tseng, K. tik Lee, D.-R. Liu, Z. Huang, S. Dong, S.-W. Li, S. Watanabe, A. Mohamed, and H. yi Lee, “Superb: Speech pro- cessing universal p...

    2021

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.