Pith. sign in

REVIEW 3 major objections 5 minor 56 references

The Role of Prosody in Spoken Question Answering

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Prosody alone can guide spoken question answering, but models lean on words when they can

desk verdict Solid lexical-dominance result; the prosody-alone claim is confounded by residual lexical info that the authors themselves report. read the letter →

arxiv 2502.05389 v1 pith:SSPTGNWU submitted 2025-02-08 cs.CL

Jie Chi , Maureen de Seyssel , Natalie Schluter This is my paper · ORCID
classification cs.CL
keywords prosodyspokenquestionansweringself-supervisedspeechrepresentationsdiscreteunitsdelexicalizationlow-passfilteringlexicalinformationSLUE-SQA-5
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether prosodic information—pitch, stress, rhythm—can support spoken question answering (SQA) on its own, and whether models actually use it when lexical content is also available. Using the natural-speech SLUE-SQA-5 dataset, the authors create a prosodic-only condition by low-pass filtering audio at 300 Hz and a lexical-only condition by flattening pitch and intensity. They find that a model trained on prosody alone reaches an FF1 of 18.49 versus a 6.03 chance baseline, showing prosody carries real signal. But when lexical information is present, models overwhelmingly rely on it, even when only 10% of training data contains lexical cues. The paper concludes that prosody is a useful supplement that current integration methods fail to exploit alongside lexical features.

What carries the argument

The experimental apparatus is the DUAL framework: WavLM self-supervised representations are converted via k-means clustering into discrete spoken units, deduplicated, and fed to a Longformer language model that predicts answer spans as temporal segments. Prosodic and lexical information are isolated by signal processing: the prosodic condition applies a 300 Hz low-pass filter intended to preserve F0 and rhythm while removing most vowel and consonant energy, and the lexical condition flattens fundamental frequency and intensity to remove intonation and stress while leaving word content intact. This pair of controlled manipulations lets the authors compare models trained and tested within and across conditions.

What would settle it

Train a model on the residual high-frequency band (the part above 300 Hz that is removed in the prosodic condition) and test it on the SQA task; if it reaches an FF1 comparable to the 18.49 of the prosodic condition, the low-passed signal still carries usable lexical information and the prosody-only interpretation collapses.

Watch

Extended reading notes

Core claim

On the SLUE-SQA-5 spoken question answering benchmark, models trained only on low-pass-filtered audio (preserving prosody while reducing lexical content) score FF1 18.49 on the test set, far above the 6.03 white-noise chance baseline, though well below the 32–36 range of natural and lexical conditions. When a model trained on the prosodic condition is tested on natural or lexical audio, performance collapses, and the reverse holds as well, indicating the model does not transfer prosodic knowledge to word-bearing conditions. When questions are randomly paired with contexts, prosodic-only performance drops to 9.77 FF1 but stays above chance, implying prosody directs attention to salient passage regions without connecting them to the question. In mixture training, evaluation loss shows that even 5–10% lexical data in the training set makes the model learn lexical cues first, demonstrating a strong lexical bias when both channels are available.

Load-bearing premise

The 300 Hz low-pass filter is assumed to strip away enough lexical content that above-chance performance on the prosodic condition can be credited to prosody, yet a recognizer still transcribes 57.5% of the filtered test audio correctly.

Editorial extensions

If this is right

  • Prosody-only models can meaningfully locate answer-relevant regions in a passage, suggesting prosodic cues are usable in the absence of intelligible words.
  • When building SQA systems, lexical information will dominate unless integration mechanisms are deliberately designed to weight prosodic cues.
  • Training on mixtures of prosodic and lexical data with only a small lexical fraction still shifts the model to lexical reliance, so data-balancing alone will not encourage prosody use.
  • The random-pairing result implies prosodic features can serve as an attention prior over the passage, a property that could be exploited in weakly supervised or retrieval settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If prosody's value lies in highlighting salient segments, then tasks with longer, noisier, or partially intelligible passages—where lexical cues are degraded—may show a larger prosody benefit than the extractive span-prediction setup used here.
  • A testable extension is to keep the 300 Hz cutoff but additionally flatten the F0 contour: if FF1 stays near 18.5, the prosodic score is not carried by pitch; if it collapses, F0 is the key carrier.
  • Residual lexical information in the filtered audio (WER 57.5% on the filtered test set) means the true prosody-only ceiling is lower than 18.49; better delexicalization methods, such as vocoder-based prosody transfer, could sharpen the estimate.
  • The layer choice for WavLM representations may matter: earlier layers are thought to encode prosody more strongly, so using a layer-wise search could reveal a larger prosody signal than reported.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper studies the role of prosody in spoken question answering using the natural-speech SLUE-SQA-5 dataset. The authors create a 'prosodic condition' by low-pass filtering audio at 300 Hz and a 'lexical condition' by flattening F0 and intensity to per-utterance means, then train Discrete Spoken Unit Adaptive Learning (DUAL) models separately on natural, lexical, and prosodic audio, evaluating with Frame-level F1 and Audio Overlap Score. The main results are that a model trained and tested on the prosodic condition reaches FF1 18.49 on the test set and 18.29 on the verified set, well above a 6.03 white-noise chance baseline; cross-condition experiments show strong transfer between natural and lexical conditions and poor transfer to or from the prosodic condition; mixture training shows that adding a small fraction of lexical data quickly shifts the model toward lexical cues; and random question-context pairing reduces prosodic-condition performance to 9.77 FF1. The authors conclude that prosody alone provides some useful cues but lexical information dominates when both are available, while acknowledging in Limitations that the prosodic condition may retain residual lexical information.

Significance. If cleanly established, the paper would be a valuable contribution: it moves beyond synthetic-speech SQA to natural speech, uses a controlled modification design with multiple sanity checks (white-noise chance baseline, cutoff-frequency sweep, WER diagnostics, random question-context pairing, and three-seed variance reporting), and it provides enough configuration details to reproduce the main experiments. The claim that lexical information dominates over prosody in SQA models is plausible and consistent with the cross-condition and mixture results. However, the evidence for the first central claim—that models trained on prosodic information alone can perform reasonably well by using prosody—is weakened by the fact that the 'prosodic' condition still carries substantial word-level information, as the paper's own WER table shows. The authors are honest about this in the Limitations section, but the abstract and conclusion do not carry the required qualification. The contribution is therefore potentially important but currently not fully supported.

major comments (3)
  1. [§3.1, §4.1, Table 3] At the 300 Hz cutoff used for the prosodic condition, the authors' own WavLM-CTC evaluation reports 57.5% WER on the test set and 44.5% on the verified test set. This is far from negligible lexical information, and the verified set is precisely where the prosodic-condition model reaches its best FF1 (18.29, Table 2). The cutoff sweep in Figure 4 does not resolve the confound: SQA FF1 is flat between 200 and 400 Hz, while WER improves from 80.3 at 200 Hz to 49.2 at 400 Hz (Table 3), so a constant residual-lexical contribution across that range is consistent with the data. The paper should provide a control that estimates how much SQA performance can be obtained from the residual lexical content alone, for example by training a lexical-only system on transcripts of the 300 Hz filtered audio, before attributing the above-chance FF1 to prosody.
  2. [§4.1 random question-context pairing] The random question-context pairing experiment is meant to show that prosody alone cannot connect the question to the context, but it does not rule out residual lexical leakage. Word-level cues in the passage—such as dates, numbers, and names that SQA answers target—can be correlated with answer spans independently of the question. The drop from 17–18 FF1 on correctly paired inputs to 9.77 FF1 on randomly paired inputs is therefore also compatible with the model using surviving lexical content to locate salient spans, with the question contributing little even in the prosodic condition. The paper should either measure whether recognized words from the 300 Hz filter predict answer spans or re-run the analysis controlling for lexical content.
  3. [Abstract, §6 Limitations] The Abstract states without qualification that "models trained on prosodic information alone can perform reasonably well by utilizing prosodic cues," but Section 6 concedes that "there remains the possibility of some residual lexical cues contributing to the model's performance." These two statements cannot both stand as written. The empirical core of the paper is a controlled approximation, and the central claim should be reworded to something like "models trained on heavily filtered speech, which retains prosodic cues but also some residual lexical information, perform above chance" until the residual-lexical confound is quantified or removed.
minor comments (5)
  1. [§4.2, Conclusion] Section 4.2 says the mixture experiment uses 0%, 5%, and 100% of the other training sets, but later in the same section and in the Conclusion the text says "10%" (e.g., "even when only 10% of the training data contains lexical information"); the inconsistency should be fixed to match Figure 5.
  2. [Figure 4] Figure 4 shows no error bars or significance intervals; given that all main results are reported as means over three seeds, the cutoff-sweep plot should include the same uncertainty information before claiming "no significant performance drop."
  3. [Table 3 caption] The caption uses "960h Librispeech"; this should be expanded to "960-hour LibriSpeech" and made consistent with the rest of the text.
  4. [§6 Limitations] The Limitations section refers to "SLUE-PHASE2" but the dataset is called "SLUE-SQA-5" throughout the paper; rename for consistency.
  5. [Throughout] Minor grammatical and typographical issues include: "the model predominantly rely on lexical cues" (Conclusion) should be "relies"; "Verified-Test" in Table 2 is hyphenated inconsistently; and the sentence "between the 200Hz and 400Hz" in Section 4.1 needs rewording.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: the prosody claim rests on controlled comparisons against a white-noise chance baseline, and the residual-lexical confound is an acknowledged limitation, not a fitted input.

full rationale

The paper's central claims are supported by direct experimental comparisons rather than by a derivation loop. Table 2 contrasts natural, lexical-flattened, and low-pass-filtered conditions against a white-noise chance baseline (FF1 6.03), and Figure 4 and Figure 5 provide cutoff sweeps and mixture-training curves. The prosodic condition is an operational approximation, not a definition of 'prosody' in terms of the conclusion: the paper states in Section 6 that 'in the prosodic condition, while we tried to minimize lexical information, there remains the possibility of some residual lexical cues contributing to the model's performance,' and it measures that residual with the WavLM-CTC WER results in Table 3 (57.5% test, 44.5% verified at 300 Hz). That residual-lexical issue is a construct-validity concern, not circularity, because the paper does not fit a parameter and then rename it a prediction. The only notable self-citation, de Seyssel et al. (2022), is used to support the general point that SSL discrete units retain prosodic information; this is not load-bearing, since DUAL, WavLM, and SLUE-SQA-5 are external resources and the main findings are the paper's own controlled measurements. No equation reduces to its own input by construction, so I find no circular step; score 1 reflects only a minor non-load-bearing self-citation and the acknowledged operational limitation.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper relies on operational choices and prior-model assumptions rather than invented entities. The main risk is the ad hoc 300 Hz cutoff and the assumption that discrete WavLM units preserve prosody, both partially validated but not fully.

free parameters (3)
  • Low-pass cutoff frequency for prosodic condition = 300 Hz
    Hand-chosen; justified by speech energy ranges and explored over 50 to 3000 Hz in Figure 4, but all main prosodic-condition results use this value.
  • Number of k-means clusters for WavLM discretization = 1000
    From prior work (de Seyssel et al. 2022; Sicherman and Adi 2023); cluster count controls how much prosodic and lexical detail survives in the input units.
  • Learning rate = 1e-5
    Chosen by search over [5e-6, 1e-4] on the natural condition, then fixed for all conditions; standard hyperparameter, not a scientific free parameter.
assumptions (4)
  • domain assumption WavLM discrete units with k=1000, layer 23 preserve prosodic cues needed for SQA.
    Section 3.2; the entire prosodic-condition pipeline depends on prosody surviving quantization; supported by cited prior work but not checked on this dataset.
  • ad hoc to paper A 300 Hz low-pass filter removes most lexical information while retaining prosodic cues.
    Defines the prosodic condition; Table 3 WER at 300 Hz (57.5% test, 44.5% verified) shows residual word information, so this is only approximately true.
  • ad hoc to paper Flattening F0 and intensity to per-utterance means removes most prosodic variation without removing lexical content.
    Section 3.1; rhythm is intentionally left unchanged and flattening introduces breath and silence artifacts, so the lexical condition still contains non-lexical cues.
  • domain assumption FF1 and AOS are valid metrics for SQA answer spans.
    Section 3.3; these metrics measure temporal overlap, not semantic correctness, and are standard in prior SQA work.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Role of Prosody in Spoken Question Answering." pith.science (2026). https://pith.science/paper/SSPTGNWU

@misc{pith2026250205389,
  author       = {Pith},
  title        = {Pith review of: The Role of Prosody in Spoken Question Answering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SSPTGNWU}},
  note         = {Machine review of arXiv:2502.05389}
}
read the original abstract

Spoken language understanding research to date has generally carried a heavy text perspective. Most datasets are derived from text, which is then subsequently synthesized into speech, and most models typically rely on automatic transcriptions of speech. This is to the detriment of prosody--additional information carried by the speech signal beyond the phonetics of the words themselves and difficult to recover from text alone. In this work, we investigate the role of prosody in Spoken Question Answering. By isolating prosodic and lexical information on the SLUE-SQA-5 dataset, which consists of natural speech, we demonstrate that models trained on prosodic information alone can perform reasonably well by utilizing prosodic cues. However, we find that when lexical information is available, models tend to predominantly rely on it. Our findings suggest that while prosodic cues provide valuable supplementary information, more effective integration methods are required to ensure prosody contributes more significantly alongside lexical features.

Figures

Figures reproduced from arXiv: 2502.05389 by the authors.

Figure 1
Figure 1. Illustration of the SQA format leveraged in speech-related tasks computationally. One of the primary tasks in this area is Spoken Language Understanding (SLU), which focuses on extracting meaningful information from spoken language input. Unlike Natural Language Under￾standing (NLU), which primarily deals with text￾based information, SLU incorporates the added complexity of processing signal made of prosodic feature… view at source ↗
Figure 2
Figure 2. Spectrogram of the example speech under different conditions. In each sub-figure, the top plot is the waveform, the second plot is the spectrogram, the third plot is the intensity, and the bottom plot is the F0. In the first setting, we remove the variations in both pitch and intensity, which we refer to as the lexical condition. This modification ensures that primarily lexical information remains, while that two of… view at source ↗
Figure 3
Figure 3. Illustration of ground truth span, predicted [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Performance on the test set with different cut [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Evaluation loss across different conditions. From left to right, the model is trained on (1) the full prosodic [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 43 canonical work pages

  1. [1]

    Sankaranarayanan Ananthakrishnan and Shrikanth Narayanan. 2007. https://doi.org/10.1109/ICASSP.2007.367209 Improved speech recognition using acoustic and lexical correlates of pitch accent in a n-best rescoring framework . In IEEE International Conference on Acoustics, Speech and Signal Processing, volume 4, pages IV--873--IV--876

    work page arXiv 2007

  2. [2]

    Sankaranarayanan Ananthakrishnan and Shrikanth Narayanan. 2009. https://doi.org/10.1109/TASL.2008.2005347 Unsupervised adaptation of categorical prosody models for prosody labeling and speech recognition . IEEE Transactions on Audio, Speech, and Language Processing, 17(1):138--149

    work page arXiv 2009

  3. [3]

    Nicolas Audibert, Francesca Carbone, Maud Champagne-Lavau, Aurélien Said Housseini, and Caterina Petrone. 2023. https://doi.org/10.21437/Interspeech.2023-1903 Evaluation of delexicalization methods for research on emotional speech . In Interspeech 2023, pages 2618--2622

    work page doi:10.21437/interspeech.2023-1903 2023

  4. [4]

    Alexei Baevski, Henry Zhou, Abdel rahman Mohamed, and Michael Auli. 2020. https://proceedings.neurips.cc/paper_files/paper/2020/file/92d1e1eb1cd6f9fba3227870bb6d7f07-Paper.pdf wav2vec 2.0: A framework for self-supervised learning of speech representations . In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ...

    work page 2020

  5. [5]

    Peters, and Arman Cohan

    Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. https://arxiv.org/abs/2004.05150 Longformer: The long-document transformer

    arXiv 2020

  6. [6]

    Vivek Bhardwaj, Tanya Gera, Deepak Thakur, and Amitoj Singh. 2024. https://doi.org/10.1007/s42979-024-03111-w Enhancing automatic speech recognition for punjabi dialects: An experimental analysis of incorporating prosodic features and acoustic variability mitigation . SN Comput. Sci., 5(6)

    work page doi:10.1007/s42979-024-03111-w 2024

  7. [7]

    Paul Boersma and David Weenink. 2021. P raat: doing phonetics by computer [ C omputer program]. Version 6.1.38, retrieved 2 January 2021 http://www.praat.org/

    work page 2021

  8. [8]

    Gary Buck. 2001. https://doi.org/10.1017/CBO9780511732959 Assessing Listening . Cambridge Language Assessment. Cambridge University Press

    work page doi:10.1017/cbo9780511732959 2001

Show all 56 references
  1. [9]

    Cedric Chan and Jianjing Kuang. 2024. https://doi.org/10.21437/SpeechProsody.2024-6 Exploring the accuracy of prosodic encodings in state-of-the-art text-to-speech models . In Speech Prosody 2024, pages 27--31

    2024 doi

  2. [10]

    K. Chen, M. Hasegawa-Johnson, A. Cohen, S. Borys, Sung-Suk Kim, J. Cole, and Jeung-Yoon Choi. 2006. https://doi.org/10.1109/TSA.2005.853208 Prosody dependent speech recognition on radio news corpus of american english . IEEE Transactions on Audio, Speech, and Language Processi...

    2006

  3. [11]

    Ken Chen, Sarah Borys, Mark Hasegawa-Johnson, and Jennifer Cole. 2003. https://doi.org/10.21437/Eurospeech.2003-153 Prosody dependent speech recognition with explicit duration modelling at intonational phrase boundaries . In 8th European Conference on Speech Communication and ...

    2003 doi

  4. [12]

    Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, et al. 2022. https://doi.org/10.1109/JSTSP.2022.3188113 Wavlm: Large-scale self-supervised pre-training for full stack speech processing . IEEE Journ...

    2022

  5. [13]

    Yung-Sung Chuang, Chi-Liang Liu, Hung yi Lee, and Lin shan Lee. 2020. https://doi.org/10.21437/Interspeech.2020-1570 SpeechBERT: An Audio-and-Text Jointly Learned Language Model for End-to-End Spoken Question Answering . In Interspeech 2020, pages 4168--4172

    2020 doi

  6. [14]

    Rob Clark, Hanna Silen, Tom Kenter, and Ralph Leith. 2019. https://doi.org/10.21437/SSW.2019-18 Evaluating long-form text-to-speech: Comparing the ratings of sentences and paragraphs . In 10th ISCA Workshop on Speech Synthesis, pages 99--104

    2019 doi

  7. [15]

    Adriana Colatosti, Ignacio Gil, Antonio Morant-Ventura, Emilia Monteagudo, Lucía Aranda, and Jaime Marco. 2024. https://doi.org/10.1016/j.otoeng.2024.05.005 Normal hearing and verbal discrimination in real sounds environments . Acta otorrinolaringologica espanola

    2024 doi

  8. [16]

    Maureen de Seyssel , Marvin Lavechin, Yossi Adi, Emmanuel Dupoux, and Guillaume Wisniewski. 2022. https://doi.org/10.21437/Interspeech.2022-373 Probing phoneme, language and speaker information in unsupervised speech representations . In Interspeech 2022, pages 1402--1406

    2022 doi

  9. [17]

    Erik Ekstedt and Gabriel Skantze. 2022. https://aclanthology.org/2022.sigdial-1.51 How much does prosody help turn-taking? investigations using voice activity projection models . In Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue,...

    2022

  10. [18]

    John Field. 2005. http://www.jstor.org/stable/3588487 Intelligibility and the listener: The role of lexical stress . TESOL Quarterly, 39(3):399--423

    2005

  11. [19]

    Jean-Philippe Goldman, Tea Pršir, George Christodoulides, Anne-Catherine Simon, and Antoine Auchlin. 2014. http://hdl.handle.net/2078.1/144534 Phonogenre identification: A perceptual experiment with 8 delexicalised speaking styles . Nouveaux cahiers de linguistique française, ...

    2014

  12. [20]

    Camille Guinaudeau and Julia Hirschberg. 2011. https://doi.org/10.21437/Interspeech.2011-459 Accounting for prosodic information to improve asr-based topic tracking for tv broadcast news . In Interspeech 2011, pages 1401--1404

    2011 doi

  13. [21]

    Mark Hasegawa-Johnson, Ken Chen, Jennifer Cole, Sarah Borys, Sung-Suk Kim, Aaron Cohen, Tong Zhang, Jeung-Yoon Choi, Heejin Kim, and Tae-Jin Yoon. 2005. https://doi.org/10.1016/j.specom.2005.01.009 Simultaneous recognition of words and prosody in the boston university radio sp...

    2005 doi

  14. [22]

    Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 2021. https://doi.org/10.1109/TASLP.2021.3122291 Hubert: Self-supervised speech representation learning by masked prediction of hidden units . IEEE/ACM Trans. Au...

    2021

  15. [23]

    Jui-Ting Huang, Po-Sen Huang, Yoonsook Mo, Mark Hasegawa-Johnson, and Jennifer Cole. 2010. https://doi.org/10.21437/SpeechProsody.2010-101 Prosody-dependent acoustic modeling using variable-parameter hidden markov models . In Speech Prosody 2010, page paper 623

    2010 doi

  16. [24]

    Yannick Jadoul, Bill Thompson, and Bart de Boer. 2018. https://doi.org/10.1016/j.wocn.2018.07.001 Introducing P arselmouth: A P ython interface to P raat . Journal of Phonetics, 71:1--15

    2018 doi

  17. [25]

    H Kagan Keskin, G \"o khan Ari, and Muhammet Bastug. 2019. https://doi.org/10.7575/aiac.ijels.v.7n.1p.59 Role of prosodic reading in listening comprehension. International Journal of Education and Literacy Studies, 7(1):59--65

    2019 doi

  18. [26]

    Arne K \"o hn, Florian Stegen, and Timo Baumann. 2016. https://aclanthology.org/L16-1735/ Mining the spoken W ikipedia for speech data and beyond . In Proceedings of the Tenth International Conference on Language Resources and Evaluation, pages 4644--4647, Portoro z , Slovenia...

    2016

  19. [27]

    Chia-Hsuan Lee, Szu-Lin Wu, Chi-Liang Liu, and Hung yi Lee. 2018. https://doi.org/10.21437/Interspeech.2018-1714 Spoken squad: A study of mitigating the impact of speech recognition errors on listening comprehension . In Interspeech 2018, pages 3459--3463

    2018 doi

  20. [28]

    Guan-Ting Lin, Yung-Sung Chuang, Ho-Lam Chung, Shu wen Yang, Hsuan-Jui Chen, Shuyan Annie Dong, Shang-Wen Li, Abdelrahman Mohamed, Hung yi Lee, and Lin shan Lee. 2022. https://doi.org/10.21437/Interspeech.2022-612 DUAL: Discrete Spoken Unit Adaptive Learning for Textless Spoke...

    2022 doi

  21. [29]

    Guan-Ting Lin, Chi-Luen Feng, Wei-Ping Huang, Yuan Tseng, Tzu-Han Lin, Chen-An Li, Hung-yi Lee, and Nigel G. Ward. 2023. https://doi.org/10.1109/SLT54892.2023.10023234 On the utility of self-supervised models for prosody-related tasks . In IEEE Spoken Language Technology Works...

    2023

  22. [30]

    Oli Danyi Liu, Hao Tang, and Sharon Goldwater. 2023. https://doi.org/10.21437/Interspeech.2023-871 Self-supervised predictive coding models encode speaker and phonetic information in orthogonal subspaces . In Interspeech 2023, pages 2968--2972. International Speech Communicati...

    2023 doi

  23. [31]

    Iker Luengo, Eva Navas, Inma Hern \'a ez, and Jon S \'a nchez. 2005. https://api.semanticscholar.org/CorpusID:12749548 Automatic emotion recognition using prosodic parameters . In Interspeech

    2005

  24. [32]

    Jacques Mehler, Peter Jusczyk, Ghislaine Lambertz, Nilofar Halsted, Josiane Bertoncini, and Claudine Amiel-Tison. 1988. A precursor of language acquisition in young infants. Cognition, 29(2):143--178

    1988

  25. [33]

    Mukhtar Mohamed, Oli Danyi Liu, Hao Tang, and Sharon Goldwater. 2024. https://interspeech2024.org/ Orthogonality and isotropy of speaker and phonetic information in self-supervised speech representations . In Interspeech 2024. ISCA. The 25th Interspeech Conference, Interspeech...

    2024

  26. [34]

    Navid Naderi and Babak Nasersharif. 2023. https://doi.org/10.1016/j.knosys.2023.110814 Cross corpus speech emotion recognition using transfer learning and attention-based fusion of wav2vec2 and prosody features . Knowledge-Based Systems, 277:110814

    2023

  27. [35]

    Oliver Niebuhr, Alexander Brem, Jan Michalsky, and Jana Neitsch. 2020. https://doi.org/10.25189/2675-4916.2020.V1.N1.ID272 What makes business speakers sound charismatic? a contrastive acoustic-melodic analysis of steve jobs and mark zuckerberg . Cadernos de Linguistica e Teor...

    2020

  28. [36]

    Ankita Pasad, Ju-Chieh Chou, and Karen Livescu. 2021. https://doi.org/10.1109/ASRU51503.2021.9688093 Layer-wise analysis of a self-supervised speech representation model . In 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 914--921

    2021

  29. [37]

    Waris Quamer and Ricardo Gutierrez-Osuna. 2024. https://arxiv.org/abs/2408.10997 Disentangling segmental and prosodic factors to non-native speech comprehensibility

    2024 arXiv

  30. [38]

    Shangeth Rajaa. 2023. https://doi.org/10.21437/Interspeech.2023-1760 Improving end-to-end slu performance with prosodic attention and distillation . In Interspeech 2023, pages 1114--1118

    2023 doi

  31. [39]

    Mirco Ravanelli, Titouan Parcollet, Peter Plantinga, Aku Rouhe, Samuele Cornell, Loren Lugosch, Cem Subakan, Nauman Dawalatabad, Abdelwahab Heba, Jianyuan Zhong, Ju-Chieh Chou, Sung-Lin Yeh, Szu-Wei Fu, Chien-Feng Liao, Elena Rastorgueva, François Grondin, William Aris, Hwidon...

    2021 arXiv

  32. [40]

    Suwon Shon, Siddhant Arora, Chyi-Jiunn Lin, Ankita Pasad, Felix Wu, Roshan Sharma, Wei-Lun Wu, Hung-yi Lee, Karen Livescu, and Shinji Watanabe. 2023. https://doi.org/10.18653/v1/2023.acl-long.496 SLUE phase-2: A benchmark suite of diverse spoken language understanding tasks . ...

    2023 doi

  33. [41]

    Suwon Shon, Kwangyoun Kim, Yi-Te Hsu, Prashant Sridhar, Shinji Watanabe, and Karen Livescu. 2024. https://arxiv.org/abs/2406.09345 Discreteslu: A large language model with self-supervised discrete speech units for spoken language understanding . Preprint, arXiv:2406.09345

    2024 arXiv

  34. [42]

    Elizabeth Shriberg and Andreas Stolcke. 2004. https://doi.org/10.1007/978-1-4419-9017-4_5 Prosody modeling for automatic speech recognition and understanding . In Mathematical Foundations of Speech and Language Processing, pages 105--114, New York, NY. Springer New York

    2004 doi

  35. [43]

    Amitay Sicherman and Yossi Adi. 2023. https://doi.org/10.1109/icassp49357.2023.10097097 Analysing discrete self supervised speech representation for spoken language modeling . In IEEE International Conference on Acoustics, Speech and Signal Processing, volume 9, page 1–5. IEEE

    2023

  36. [44]

    RJ Skerry-Ryan, Eric Battenberg, Ying Xiao, Yuxuan Wang, Daisy Stanton, Joel Shor, Ron Weiss, Rob Clark, and Rif A Saurous. 2018. http://arxiv.org/abs/1803.09047 Towards end-to-end prosody transfer for expressive speech synthesis with tacotron . In international conference on ...

    2018 arXiv

  37. [45]

    Aarne Talman, Antti Suni, Hande Celikkanat, Sofoklis Kakouros, J \"o rg Tiedemann, and Martti Vainio. 2019. https://aclanthology.org/W19-6129 Predicting prosodic prominence from text with pre-trained contextualized word representations . In Proceedings of the 22nd Nordic Confe...

    2019

  38. [46]

    Trang Tran, Shubham Toshniwal, Mohit Bansal, Kevin Gimpel, Karen Livescu, and Mari Ostendorf. 2018. https://doi.org/10.18653/v1/N18-1007 Parsing speech: a neural approach to integrating lexical and acoustic-prosodic information . In Proceedings of the 2018 Conference of the No...

    2018 doi

  39. [47]

    Trang Tran, Jiahong Yuan, Yang Liu, and Mari Ostendorf. 2019. https://api.semanticscholar.org/CorpusID:202723730 On the role of style in parsing speech with neural models . In Interspeech 2019

    2019

  40. [48]

    U nl \"u Menev s e, Yusufcan Manav, Ebru Arisoy, and Arzucan \

    Merve \"U nl \"u Menev s e, Yusufcan Manav, Ebru Arisoy, and Arzucan \"O zg \"u r. 2022. https://doi.org/10.18653/v1/2022.findings-emnlp.342 A framework for automatic generation of spoken question-answering data . In Findings of the Association for Computational Linguistics: E...

    2022 doi

  41. [49]

    Klára Vicsi and György Szaszák. 2010. https://doi.org/10.1016/j.specom.2010.01.003 Using prosody to improve automatic speech recognition . Speech Communication, 52(5):413--426

    2010 doi

  42. [50]

    Strimel, Nathan Susanj, Athanasios Mouchtaris, and Maurizio Omologo

    Kai Wei, Dillon Knox, Martin Radfar, Thanh Tran, Markus Muller, Grant P. Strimel, Nathan Susanj, Athanasios Mouchtaris, and Maurizio Omologo. 2022. https://arxiv.org/abs/2205.05590 A neural prosody encoder for end-ro-end dialogue act classification . Preprint, arXiv:2205.05590

    2022 arXiv

  43. [51]

    Mirjam Wester, Oliver Watts, and Gustav Eje Henter. 2016. https://doi.org/10.21437/SpeechProsody.2016-157 Evaluating comprehension of natural and synthetic conversational speech . In Speech Prosody 2016, pages 766--770

    2016 doi

  44. [52]

    Stephen J Winters and David B Pisoni. 2004. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=8e10a4c4d279e9540cd5af5aae692fe9907409ff Perception and comprehension of synthetic speech . Research on spoken language processing report, 26:95--138

    2004

  45. [53]

    Lukas Wolf, Tiago Pimentel, Evelina Fedorenko, Ryan Cotterell, Alex Warstadt, Ethan Wilcox, and Tamar Regev. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.606 Quantifying the redundancy between prosody and text . In Proceedings of the 2023 Conference on Empirical Methods i...

    2023 doi

  46. [54]

    Yijing Wu, SaiKrishna Rallabandi, Ravisutha Srinivasamurthy, Parag Pravin Dakle, Alolika Gon, and Preethi Raghavan. 2024. https://arxiv.org/abs/2304.13689 Heysquad: A spoken question answering dataset . Preprint, arXiv:2304.13689

    2024 arXiv

  47. [55]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  48. [56]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.