Pith. sign in

REVIEW 4 major objections 4 minor 37 references

MVP: Multi-source Voice Pathology detection

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Combining sustained-vowel and sentence recordings through intermediate transformer fusion lifts voice-pathology AUC by 10–13 points across three languages.

desk verdict A practical multi-source fusion study that likely overstates its headroom due to hyperparameter selection on the test folds. read the letter →

arxiv 2505.20050 v1 pith:WJ5EHSGU submitted 2025-05-26 eess.AS cs.CL

Alkis Koudounas , Moreno La Quatra , Gabriele Ciravegna , Marco Fantini , Erika Crosetti , Giovanni Succo , Tania Cerquitelli , Sabato Marco Siniscalchi
show 1 more author
Elena Baralis
This is my paper · ORCID
classification eess.AScs.CL
keywords voicepathologydetectionmulti-sourcefusionintermediatefeaturetransformerHuBERTsustainedvowelsentencereadingAUC
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MVP claims that automated voice-pathology screening is best done by analyzing two recording types together rather than either alone. The paper combines a sustained-vowel recording and a sentence reading through per-source transformer backbones and fuses their intermediate features with a small transformer encoder. Across German, Portuguese, and Italian datasets, this intermediate feature fusion reaches AUCs of 95.8%, 96.3%, and 93.6%, improving on the best single-source baseline by 10–13 AUC points. The result supports a concrete clinical workflow: collect both a vowel and a sentence, then let a model merge them before deciding.

What carries the argument

The central mechanism is Intermediate Feature Fusion (IFF) implemented with a Transformer Encoder (TE). Frame-level representations from two specialized HuBERT backbones—one pre-trained on LibriSpeech for the sentence, one pre-trained on AudioSet for the sustained vowel—are concatenated along the time axis into one sequence, and a two-layer transformer encoder allows every time step of either source to attend to every time step of both sources. Attention pooling then collapses the fused sequence to a single vector for a linear classifier, so the model learns cross-source temporal correlations rather than relying on pooled averages or late score averaging.

What would settle it

Re-run the same 10-fold protocol with full-length recordings, or draw several different sentence/vowel selections per subject, and check whether the intermediate-fusion AUC still beats the best single-source baseline by the same margin; as a leakage check, hold out a different recording session of the same speaker and see whether the model still separates healthy from pathological voices.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that sustained vowels and sentence readings carry complementary diagnostic information that is best exploited by fusing them before classification, not by training separate models or concatenating raw waveforms. Using HuBERT pre-trained on speech for sentences and HuBERT pre-trained on acoustic events for vowels, extracting frame-level features from the fifth layer, concatenating the two sequences along time, and passing them through a two-layer transformer encoder yields the highest AUC on all three datasets. The method beats waveform concatenation, decision-level averaging, and every single-source baseline, including a model trained on a mixture of both source types. The authors interpret this as evidence that fine-grained temporal interaction between the two sources is the mechanism that carries the improvement.

Load-bearing premise

The load-bearing premise is that cutting every recording to 5.0 seconds and randomly selecting one sentence and one vowel per subject preserves the diagnostically relevant information, and that the speaker-independent folds keep all recordings of a subject on one side of the split; if either fails, the reported fusion gains could be sampling artifacts rather than true complementarity.

Editorial extensions

If this is right

  • Screening protocols that collect both a sustained vowel and a sentence reading should expect better discrimination than single-task protocols, by roughly 10–13 AUC points.
  • Intermediate fusion should be preferred over both waveform concatenation and decision-level ensembling, because it is the strategy that lets the two sources interact at the feature level.
  • Specialized pretraining matters: a speech-pretrained backbone for sentences and an acoustic-event-pretrained backbone for vowels outperform other pairings.
  • Even with frozen backbones, intermediate fusion beats most single-source baselines with far fewer trainable parameters, making the approach usable where full fine-tuning is too expensive.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the 5.0-second truncation preserves diagnostic content in routine clinical settings, the same pipeline could be deployed on short prompted recordings from a phone or a portable recorder.
  • The two-source fusion pattern may transfer to other speech-disorder screenings, such as dysarthria, Parkinson's disease, or dementia, where connected speech and sustained phonation are both routinely collected.
  • The paper's random selection of one sentence and one vowel per subject leaves within-subject variability untested; a prospective study with repeated recordings would show whether the gains persist across different utterances of the same speaker.
  • Since simple concatenation of pooled features trails the transformer fusion by up to 4.7 AUC points, the value of the method may lie in modeling temporal alignment between vowel and sentence, a hypothesis that could be tested by shifting one source relative to the other in time.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The manuscript proposes MVP, a multi-source voice pathology detection framework that fuses sustained-vowel and sentence-reading recordings. Using HuBERT backbones pre-trained on LibriSpeech and AudioSet, the authors compare three fusion levels (waveform concatenation, intermediate feature fusion, and decision-level combination) and five intermediate fusion strategies (concatenation, attention pooling, gating, FiLM, and transformer encoder). Experiments across three datasets (German SVD, Portuguese AVFAD, Italian IPV) with speaker-independent 10-fold cross-validation report that IFF with a transformer encoder and fine-tuned backbones achieves AUCs of 95.8%, 96.3%, and 93.6%, respectively, a relative improvement of up to about 13% over the best single-source baseline. The paper includes ablations on fusion strategy, backbone pre-training, and feature-extraction layer.

Significance. If the reported results are unbiased, the contribution is substantial: the paper provides the first systematic comparison of fusion levels for multi-source voice pathology detection, with consistent gains across three languages and a reproducible experimental pipeline (code link provided). The use of speaker-independent folds, multiple baselines, and ablations is a genuine strength. However, the quantitative headline—especially the choice of TE over FiLM and of layer 5 over layer 4—rests on model selection performed on the same test folds, and this needs to be corrected before the specific AUC gains can be accepted. The qualitative finding that multi-source fusion helps is more robust, since WC and DLC also beat single-source baselines, but the paper's central quantitative claims require nested validation or a separate hold-out set.

major comments (4)
  1. [3.2, 4.2, Tables 2 and 4] Hyperparameter selection is performed on the same folds used to report final results. The text in Section 3.2 describes the augmentation probabilities as 'empirically-determined', sets L=2 TE layers as an 'effective balance', and states that features are extracted from the 5th layer ('see Section 4.2 for detailed ablation studies'). Section 4.2 then chooses the fusion strategy (Table 2) and the feature layer (Table 4) by AUC on the same 10-fold cross-validation. Since no held-out validation set or nested cross-validation is described, the reported IFF-TE AUCs (95.8, 96.3, 93.6) are selected, not evaluated, on the test folds. The bias is material because the gaps that drive the conclusions are small: TE vs FiLM is 0.958 vs 0.951 on SVD, 0.963 vs 0.961 on AVFAD, and 0.936 vs 0.934 on IPV, and layer 5 vs layer 4 is 0.958 vs 0.944 on SVD. The comparison with single-source baselines is also unfair because those baselines do not receive the same per-strategy selection. Please provide nested cross-validation or a separate hold-out set for all hyperparameter choices, or explicitly present the IFF-TE numbers as a selected configuration.
  2. [3.2] The choices of augmentation probabilities (25% for sentences, 10% for vowels) and the number of fusion transformer layers (L=2) are presented without any supporting ablation or validation protocol. If these values were fixed after inspecting test-fold AUCs they inherit the selection bias described in the previous comment; if they were chosen a priori, the paper should say so. In particular, L controls the capacity of the TE fusion, and no evidence is given that L=2 is optimal or that the TE advantage over FiLM and AP is stable across L.
  3. [Tables 1-4] The paper reports mean±standard deviation across 10 folds but provides no paired significance tests or confidence intervals, so the 'best' designations are not statistically supported. For example, on IPV the TE and FiLM AUCs differ by 0.002 (0.936±0.053 vs 0.934±0.053), and on SVD TE vs Gating differ by 0.011 (0.958±0.062 vs 0.947±0.068) with overlapping standard deviations. With only 10 folds, the authors should add a paired test across folds (e.g., Wilcoxon signed-rank) or report effect sizes before claiming that TE is the best fusion strategy and that the 5th layer is optimal.
  4. [Section 4, Table 1] The claim that the frozen IFF variant 'still outperforms four of five single-source baselines' is not supported by the table. Comparing AUCs, the frozen model outperforms 4 of 5 baselines on SVD (0.832 vs 0.850, 0.732, 0.810, 0.760, 0.765), but on AVFAD it outperforms only 3 of 5 (0.834 is below LS→Sent 0.877 and AS→Sent 0.855), and on IPV only 2 of 5 (0.809 is below LS→Sent 0.847, AS→Sent 0.815, and LS→Mix 0.831). This overstatement should be corrected, or the claim should be restricted to the SVD dataset.
minor comments (4)
  1. [Abstract and Section 4] There are typographical issues such as 'V oice' with a stray space in the abstract and 'This is likely because' with an upper-case T after a comma in Section 4; these should be fixed.
  2. [Eqs. (4)-(6)] The dimensions of the learnable matrices W, W_gamma, and W_beta in the Gating and FiLM equations are not specified; please define them explicitly for reproducibility.
  3. [3.1] For AVFAD and IPV, the random selection of one sentence and one vowel per subject is performed with no reported seed; please report the seed or average over multiple random selections to show that the results are not an artifact of a particular draw.
  4. [3.2] The 5.0-second truncation may discard the ends of longer sentence readings; please report how many recordings are truncated and whether the results are sensitive to the truncation length.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: MVP's central claim is an empirical comparison on external benchmarks, not a derivation from its own inputs.

full rationale

MVP is an empirical systems paper with no formal derivation, fitted theoretical model, or uniqueness theorem whose output could equal its input by construction. The central claim—that intermediate feature fusion (IFF) with a transformer encoder and fine-tuned HuBERT backbones best combines sustained vowels and sentence readings—is supported by 10-fold, speaker-independent cross-validation on three external datasets (SVD, AVFAD, IPV), and all fusion strategies (WC, IFF, DLC) and single-source baselines are trained under a shared protocol. The paper's self-citations, especially [6], are contextual: [6] motivates transformer-based voice pathology detection and the DLC baseline, but the design choice of LibriSpeech pre-training for sentences and AudioSet pre-training for vowels is independently validated by the ablation in Table 3 rather than accepted solely from [6]. Choices such as L=2 transformer layers, augmentation probabilities, and 5th-layer feature extraction are selected using the same cross-validation folds, which is a methodological selection/leakage concern, not a circularity: no reported AUC is defined in terms of another reported AUC, and no equation reduces a predicted quantity to a fitted input. No self-definitional, fitted-input-as-prediction, self-citation-load-bearing, uniqueness-imported, or ansatz-smuggled pattern is present, so the honest finding is no significant circularity.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The paper is an empirical study; its central claim rests on the quality of the datasets and the assumption that the evaluation protocol reflects real-world performance. The only explicit free parameters are training hyperparameters and the layer/fusion choices, all set by hand or by small ablations. No new theoretical entities are introduced.

free parameters (5)
  • Augmentation probability for sentence recordings = 25%
    Chosen by hand in Section 3.2 to balance data diversity and signal integrity; not derived from data.
  • Augmentation probability for sustained vowels = 10%
    Chosen by hand in Section 3.2; authors state it is empirically determined.
  • Feature extraction layer = 5th layer
    Selected after ablation in Section 4.2; the optimal layer may depend on the datasets.
  • Number of transformer encoder fusion layers L = 2
    Set in Section 3.2 to balance complexity and representational power.
  • Learning rate / weight decay / batch size = 5e-5 / 0.01 / 64
    Fixed optimizer hyperparameters in Section 3.2; not swept.
assumptions (3)
  • domain assumption HuBERT representations contain enough information about voice pathology to support detection.
    The whole method relies on self-supervised speech representations being informative for pathological voice classification (Section 2.1).
  • domain assumption The three datasets have correct pathological/healthy labels and are representative of the target population.
    Section 3.1 treats SVD, AVFAD, and IPV ground truth as reliable without an independent audit.
  • domain assumption Speaker-independent splits prevent information leakage between training and test subjects.
    Section 3.2 asserts speaker-independent splits but does not detail identity-based folding or seed control.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MVP: Multi-source Voice Pathology detection." pith.science (2026). https://pith.science/paper/WJ5EHSGU

@misc{pith2026250520050,
  author       = {Pith},
  title        = {Pith review of: MVP: Multi-source Voice Pathology detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WJ5EHSGU}},
  note         = {Machine review of arXiv:2505.20050}
}
read the original abstract

Voice disorders significantly impact patient quality of life, yet non-invasive automated diagnosis remains under-explored due to both the scarcity of pathological voice data, and the variability in recording sources. This work introduces MVP (Multi-source Voice Pathology detection), a novel approach that leverages transformers operating directly on raw voice signals. We explore three fusion strategies to combine sentence reading and sustained vowel recordings: waveform concatenation, intermediate feature fusion, and decision-level combination. Empirical validation across the German, Portuguese, and Italian languages shows that intermediate feature fusion using transformers best captures the complementary characteristics of both recording types. Our approach achieves up to +13% AUC improvement over single-source methods.

Figures

Figures reproduced from arXiv: 2505.20050 by the authors.

Figure 1
Figure 1. Proposed MVP framework: waveform concatena￾tion (top panel), intermediate feature fusion (mid panel), and decision-level combination (bottom panel). decision-level combination, each input is processed by special￾ized backbones. For sentence recordings, we use HuBERT pre￾trained on LibriSpeech [24], which captures linguistic content and prosodic variations. For sustained vowel recordings, we use HuBERT pre-trained on… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 33 canonical work pages

  1. [1]

    These conditions impact daily communication, work performance, and overall quality of life

    Introduction V oice disorders affect approximately 30% of the general pop- ulation during their lifetime [1–3]. These conditions impact daily communication, work performance, and overall quality of life. Disorders range from functional issues such as mus- cle tension dysphonia to organic pathologies such as vocal cord nodules [4]. Early detection is cruci...

    work page

  2. [2]

    MVP Framework Our MVP framework addresses voice pathology detection by combining multiple recording sources. It adopts a three-stage architecture: (i) source-specific backbone models for feature extraction, (ii) fusion strategies to combine information from different sources, and (iii) a decision-making module for pathol- ogy detection. Fig. 1 provides an...

    work page Pith review arXiv 2025

  3. [3]

    Experimental Setup To ensure reproducibility and fair comparison, in the following, we detail our experimental setup across datasets, training, data processing, and evaluation procedures1. 3.1. Datasets We evaluate our approach on three datasets that contain both sentence readings and sustained vowel emissions recorded un- der controlled conditions. SVD. ...

    work page

  4. [4]

    The IFF-TE method with fine-tuned backbones achieves the highest AUC scores: 95.8% (SVD), 96.3% (A VFAD), and 93.6% (IPV)

    Experimental Analysis Table 1 demonstrates that our multi-source approach signifi- cantly outperforms single-source baselines across all datasets. The IFF-TE method with fine-tuned backbones achieves the highest AUC scores: 95.8% (SVD), 96.3% (A VFAD), and 93.6% (IPV). This represents a 10-13% improvement over the best single-source baseline, showing the ...

    work page

  5. [5]

    Conclusions This paper presented MVP, a novel multi-source approach for voice pathology detection that effectively combines sustained vowels and sentence readings. Our experimental results across three languages demonstrate that intermediate feature fusion with transformers consistently outperforms single-source meth- ods, achieving up to 13% AUC improvement

    work page

  6. [6]

    1555 11/10/2022, PE00000013), is partially supported by the ”D.A.R.E

    Acknowledgements This work is partially supported by the FAIR - Future Artifi- cial Intelligence Research and received funding from the Eu- ropean Union Next-GenerationEU (PIANO NAZIONALE DI RIPRESA E RESILIENZA (PNRR) – MISSIONE 4 COM- PONENTE 2, INVESTIMENTO 1.3 – D.D. 1555 11/10/2022, PE00000013), is partially supported by the ”D.A.R.E. – Dig- ital Lif...

    work page 2022

  7. [7]

    V oice disorders identification using hybrid approach: Wavelet analysis and multilayer neural networks,

    L. Salhi, M. Talbi, and A. Cherif, “V oice disorders identification using hybrid approach: Wavelet analysis and multilayer neural networks,” International Journal of Electrical and Computer En- gineering, vol. 2, no. 9, pp. 3003–3012, 2008

    work page 2008

  8. [8]

    Prevalence of voice disorders in teachers and the general population,

    N. Roy, R. M. Merrill, S. Thibeault, R. A. Parsa, S. D. Gray, and E. M. Smith, “Prevalence of voice disorders in teachers and the general population,” Journal of Speech, Language, and Hearing Research, 2004

    work page 2004

Show all 37 references
  1. [9]

    V oice disor- ders in the general population: prevalence, risk factors, and occu- pational impact,

    N. Roy, R. M. Merrill, S. D. Gray, and E. M. Smith, “V oice disor- ders in the general population: prevalence, risk factors, and occu- pational impact,” The Laryngoscope, 2005

    2005

  2. [10]

    The rapidly evolving scenario of acoustic voice analysis in otolaryngology,

    M. Fantini, G. Ciravegna, A. Koudounas, T. Cerquitelli, E. Bar- alis, G. Succo, and E. Crosetti, “The rapidly evolving scenario of acoustic voice analysis in otolaryngology,” Cureus, vol. 16, no. 11, 2024

    2024

  3. [11]

    Basics of voice disorders,

    S. Akbulut, A. am Zehnhoff-Dinnesen, F. de Jong, M. Echter- nach, U. Eysholdt, M. Fuchs, T. Hacki, K. Izdebski, A. Keil- mann, P. Kummer et al., “Basics of voice disorders,” in Phoni- atrics I: Fundamentals–Voice Disorders–Disorders of Language and Hearing Development. Springe...

    2019

  4. [12]

    Delayed otolaryn- gology referral for voice disorders increases health care costs,

    S. M. Cohen, J. Kim, N. Roy, and M. Courey, “Delayed otolaryn- gology referral for voice disorders increases health care costs,” The American Journal of Medicine, 2015

    2015

  5. [13]

    V oice disorder analysis: a transformer-based approach,

    A. Koudounas, G. Ciravegna, M. Fantini, E. Crosetti, G. Succo, T. Cerquitelli, and E. Baralis, “V oice disorder analysis: a transformer-based approach,” in Interspeech 2024 , 2024, pp. 3040–3044

    2024

  6. [14]

    Dysarthric speech trans- former: A sequence-to-sequence dysarthric speech recognition system,

    S. R. Shahamiri, V . Lal, and D. Shah, “Dysarthric speech trans- former: A sequence-to-sequence dysarthric speech recognition system,” IEEE Transactions on Neural Systems and Rehabilita- tion Engineering, 2023

    2023

  7. [15]

    Byovoz automatic voice condition analysis system for the 2018 femh challenge,

    J. D. Arias-Londo ˜no, J. A. G ´omez-Garc´ıa, L. Moro-Vel ´azquez, and J. I. Godino-Llorente, “Byovoz automatic voice condition analysis system for the 2018 femh challenge,” in 2018 IEEE In- ternational Conference on Big Data (Big Data). IEEE, 2018

    2018

  8. [16]

    V oice disorder classi- fication using convolutional neural network based on deep transfer learning,

    X. Peng, H. Xu, J. Liu, J. Wang, and C. He, “V oice disorder classi- fication using convolutional neural network based on deep transfer learning,” Scientific Reports, vol. 13, no. 1, p. 7264, 2023

    2023

  9. [17]

    A voice disease detec- tion method based on mfccs and shallow cnn,

    X. Xie, H. Cai, C. Li, Y . Wu, and F. Ding, “A voice disease detec- tion method based on mfccs and shallow cnn,” Journal of Voice, 2023

    2023

  10. [18]

    Hybrid cnn-lstm model with efficient hyper- parameter tuning for prediction of parkinson’s disease,

    U. K. Lilhore et al., “Hybrid cnn-lstm model with efficient hyper- parameter tuning for prediction of parkinson’s disease,” Scientific Reports, 2023

    2023

  11. [19]

    V oice pathology detection using convolutional neural networks with electroglotto- graphic (egg) and speech signals,

    R. Islam, E. Abdel-Raheem, and M. Tarique, “V oice pathology detection using convolutional neural networks with electroglotto- graphic (egg) and speech signals,” Computer Methods and Pro- grams in Biomedicine Update, 2022

    2022

  12. [20]

    Automatic voice disorder detection using self- supervised representations,

    D. Ribas, M. A. Pastor, A. Miguel, D. Mart ´ınez, A. Ortega, and E. Lleida, “Automatic voice disorder detection using self- supervised representations,” Ieee Access, 2023

    2023

  13. [21]

    Hubert: Self-supervised speech represen- tation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdi- nov, and A. Mohamed, “Hubert: Self-supervised speech represen- tation learning by masked prediction of hidden units,”IEEE/ACM transactions on audio, speech, and language processing, 2021

    2021

  14. [22]

    E2e-dasr: End-to-end deep learning-based dysarthric automatic speech recognition,

    A. Almadhor, R. Irfan, J. Gao, N. Saleem, H. T. Rauf, and S. Kadry, “E2e-dasr: End-to-end deep learning-based dysarthric automatic speech recognition,” Expert Systems with Applications, vol. 222, p. 119797, 2023

    2023

  15. [23]

    Detecting dementia from speech and transcripts using transformers,

    L. Ilias, D. Askounis, and J. Psarras, “Detecting dementia from speech and transcripts using transformers,” Computer Speech & Language, vol. 79, p. 101485, 2023

    2023

  16. [24]

    Exploiting foundation models and speech enhancement for parkinson’s disease detection from speech in real-world operative conditions,

    M. L. Quatra, M. F. Turco, T. Svendsen, G. Salvi, J. R. Orozco- Arroyave, and S. M. Siniscalchi, “Exploiting foundation models and speech enhancement for parkinson’s disease detection from speech in real-world operative conditions,” Interspeech, 2024

    2024

  17. [25]

    Bilingual dual-head deep model for parkinson’s disease detec- tion from speech,

    M. La Quatra, J. R. Orozco-Arroyave, and M. S. Siniscalchi, “Bilingual dual-head deep model for parkinson’s disease detec- tion from speech,” inICASSP 2025-2025 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2025, pp. 1–5

    2025

  18. [26]

    V ocal fold nodules: A disorder of phonation organs or auditory feedback?

    S.-H. Lee, J.-F. Yu, T.-J. Fang, and G.-S. Lee, “V ocal fold nodules: A disorder of phonation organs or auditory feedback?” Clinical Otolaryngology, 2019

    2019

  19. [27]

    A pattern recognition approach to spasmodic dysphonia and mus- cle tension dysphonia automatic classification,

    G. Schlotthauer, M. E. Torres, and M. C. Jackson-Menaldi, “A pattern recognition approach to spasmodic dysphonia and mus- cle tension dysphonia automatic classification,” Journal of voice, 2010

    2010

  20. [28]

    Film: Visual reasoning with a general conditioning layer,

    E. Perez, F. Strub, H. De Vries, V . Dumoulin, and A. Courville, “Film: Visual reasoning with a general conditioning layer,” in AAAI, 2018

    2018

  21. [29]

    Benchmarking representations for speech, music, and acoustic events,

    M. La Quatra, A. Koudounas, L. Vaiani, E. Baralis, L. Cagliero, P. Garza, and S. M. Siniscalchi, “Benchmarking representations for speech, music, and acoustic events,” in 2024 IEEE Interna- tional Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), 2024...

    2024

  22. [30]

    voc2vec: A foundation model for non-verbal vocalization,

    A. Koudounas, M. La Quatra, S. M. Siniscalchi, and E. Baralis, “voc2vec: A foundation model for non-verbal vocalization,” in ICASSP 2025 - 2025 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP), 2025, pp. 1–5

    2025

  23. [31]

    Lib- rispeech: an asr corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Lib- rispeech: an asr corpus based on public domain audio books,” in ICASSP. IEEE, 2015

    2015

  24. [32]

    Audio set: An ontology and human-labeled dataset for audio events,

    J. F. Gemmeke, D. P. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter, “Audio set: An ontology and human-labeled dataset for audio events,” in ICASSP. IEEE, 2017

    2017

  25. [33]

    Vision-language multimodal fusion in dermato- logical disease classification,

    M. La Quatra, N. D. Cilia, V . Conti, S. Sorce, G. Garraffa, and V . M. Salerno, “Vision-language multimodal fusion in dermato- logical disease classification,” in Pattern Recognition, Computer Vision, and Image Processing. ICPR 2024 International Work- shops and Challenges . ...

    2024

  26. [34]

    Gated multimodal units for information fusion,

    J. Arevalo, T. Solorio, M. Montes-y G ´omez, and F. A. Gonz´alez, “Gated multimodal units for information fusion,” arXiv preprint arXiv:1702.01992, 2017

    2017 arXiv

  27. [36]

    wav2vec 2.0: A framework for self-supervised learning of speech representa- tions,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representa- tions,” NeurIPS, 2020

    2020

  28. [37]

    Exploring asr-based wav2vec2 for automated speech disorder as- sessment: Insights and analysis,

    T. Nguyen, C. Fredouille, A. Ghio, M. Balaguer, and V . Woisard, “Exploring asr-based wav2vec2 for automated speech disorder as- sessment: Insights and analysis,” in SLT. IEEE, 2024

    2024

  29. [64]

    For the TE fusion strategy, we set L = 2 transformer encoder layers, which provides an effective balance between model complexity and representational power

    Training was performed on a single NVIDIA A100 80GB GPU. For the TE fusion strategy, we set L = 2 transformer encoder layers, which provides an effective balance between model complexity and representational power. Data Augmentation. We implement separate augmentation strategi...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.