{"id":"e53c4be3-8a52-4b07-8bfe-9b2c7cbed207","arxiv_id":"2502.05389","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"On natural-speech spoken QA, prosodic-only models beat chance but trail lexical models, and lexical cues dominate whenever both are available.","lead":"This paper isolates prosody and word content in a spoken question answering dataset to see which cues models actually use. It finds that models can use prosodic cues alone at above-chance accuracy, but when both are present they lean almost entirely on lexical content.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'prosody-alone' result is not cleanly identified: at the 300 Hz cutoff used for the prosodic condition, Table 3 reports 57.5% WER (44.5% on verified), so residual lexical content is a plausible alternative explanation for FF1 18.49; the paper's own Limitations section concedes this.","rationale":"The reader's weakest_assumption identifies exactly the load-bearing concern: the 300 Hz prosodic condition is assumed to contain negligible lexical information, yet Table 3 demonstrates substantial residual lexical intelligibility (57.5% WER on test, 44.5% on verified). This matters because the prosodic-condition FF1 of 18.49 versus a 6.03 chance baseline is the central evidence for the 'prosody alone' claim, and the paper itself acknowledges the residual-lexical possibility in Section 6. I considered whether the lexical condition's preservation of duration/rhythm is a more serious confound, but that would only weaken the purity of the 'lexical-only' condition, not the lexical-dominance conclusion, which is independently supported by the mixture experiments and strong cross-condition generalization. I also considered the lack of released code, but that affects reproducibility rather than the validity of the core claim. Since the reader already assigned CONDITIONAL with this same concern, my stress-test does not change the verdict. The proposed concrete test would settle whether residual lexical information alone can reproduce the observed prosodic-condition performance.","tokens_in":13040,"tokens_out":3920,"duration_ms":45047,"concrete_test":"Build a text-only leakage simulator: take the verified set, run WavLM-CTC on the 300 Hz filtered audio, keep the recognized word lattice with confidences, and feed those words into a span-prediction model trained on gold text transcripts with unrecognized words masked. If this text-only system reaches FF1 comparable to the prosodic model's 18.29, residual lexical information is sufficient to explain the 'prosody alone' result. A stronger complementary check: re-run the prosodic condition using a vocoder that synthesizes buzz/noise excitation preserving F0, intensity, and duration but removing segmental spectral detail; if FF1 drops to the ~6.03 chance baseline, the original result was driven by lexical leakage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To support the abstract's claim that 'models trained on prosodic information alone can perform reasonably well,' the prosodic condition must contain negligible lexical information. The paper operationalizes prosody as a 300 Hz low-pass filter, but the authors' own WavLM-CTC evaluation (Table 3) shows 57.5% WER on test and 44.5% WER on the verified set at that cutoff — i.e., substantial word-level information survives, especially on the exact hand-picked set where the prosodic model reaches FF1 18.29. Low-pass filtering below 300 Hz retains voicing, nasals, approximants, and the amplitude/F0 envelope; these cues support partial word recognition, particularly for content words and numbers that determine SQA answer spans. The cutoff sweep in Figure 4, showing a performance plateau between 200 Hz and 400 Hz, does not resolve this concern: a small but sufficient amount of surviving lexical information could be roughly constant across that plateau. The random question-context pairing experiment also does not rule out lexical leakage, because recognized words in the passage can correlate with answer-span positions independently of the question. Section 6 explicitly concedes that 'in the prosodic condition, while we tried to minimize lexical information, there remains the possibility of some residual lexical cues contributing to the model's performance.' Because the above-chance prosodic result is the direct evidence for the first claim, this residual-lexical confound is load-bearing. The lexical-dominance finding (Table 2, Figure 5) is more robust, since it does not depend on the purity of the prosodic condition.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies the role of prosody in spoken question answering using the natural-speech SLUE-SQA-5 dataset. The authors create a 'prosodic condition' by low-pass filtering audio at 300 Hz and a 'lexical condition' by flattening F0 and intensity to per-utterance means, then train Discrete Spoken Unit Adaptive Learning (DUAL) models separately on natural, lexical, and prosodic audio, evaluating with Frame-level F1 and Audio Overlap Score. The main results are that a model trained and tested on the prosodic condition reaches FF1 18.49 on the test set and 18.29 on the verified set, well above a 6.03 white-noise chance baseline; cross-condition experiments show strong transfer between natural and lexical conditions and poor transfer to or from the prosodic condition; mixture training shows that adding a small fraction of lexical data quickly shifts the model toward lexical cues; and random question-context pairing reduces prosodic-condition performance to 9.77 FF1. The authors conclude that prosody alone provides some useful cues but lexical information dominates when both are available, while acknowledging in Limitations that the prosodic condition may retain residual lexical information.","tokens_in":13386,"tokens_out":6733,"duration_ms":63754,"significance":"If cleanly established, the paper would be a valuable contribution: it moves beyond synthetic-speech SQA to natural speech, uses a controlled modification design with multiple sanity checks (white-noise chance baseline, cutoff-frequency sweep, WER diagnostics, random question-context pairing, and three-seed variance reporting), and it provides enough configuration details to reproduce the main experiments. The claim that lexical information dominates over prosody in SQA models is plausible and consistent with the cross-condition and mixture results. However, the evidence for the first central claim—that models trained on prosodic information alone can perform reasonably well by using prosody—is weakened by the fact that the 'prosodic' condition still carries substantial word-level information, as the paper's own WER table shows. The authors are honest about this in the Limitations section, but the abstract and conclusion do not carry the required qualification. The contribution is therefore potentially important but currently not fully supported.","major_comments":[{"comment":"At the 300 Hz cutoff used for the prosodic condition, the authors' own WavLM-CTC evaluation reports 57.5% WER on the test set and 44.5% on the verified test set. This is far from negligible lexical information, and the verified set is precisely where the prosodic-condition model reaches its best FF1 (18.29, Table 2). The cutoff sweep in Figure 4 does not resolve the confound: SQA FF1 is flat between 200 and 400 Hz, while WER improves from 80.3 at 200 Hz to 49.2 at 400 Hz (Table 3), so a constant residual-lexical contribution across that range is consistent with the data. The paper should provide a control that estimates how much SQA performance can be obtained from the residual lexical content alone, for example by training a lexical-only system on transcripts of the 300 Hz filtered audio, before attributing the above-chance FF1 to prosody.","section":"§3.1, §4.1, Table 3"},{"comment":"The random question-context pairing experiment is meant to show that prosody alone cannot connect the question to the context, but it does not rule out residual lexical leakage. Word-level cues in the passage—such as dates, numbers, and names that SQA answers target—can be correlated with answer spans independently of the question. The drop from 17–18 FF1 on correctly paired inputs to 9.77 FF1 on randomly paired inputs is therefore also compatible with the model using surviving lexical content to locate salient spans, with the question contributing little even in the prosodic condition. The paper should either measure whether recognized words from the 300 Hz filter predict answer spans or re-run the analysis controlling for lexical content.","section":"§4.1 random question-context pairing"},{"comment":"The Abstract states without qualification that \"models trained on prosodic information alone can perform reasonably well by utilizing prosodic cues,\" but Section 6 concedes that \"there remains the possibility of some residual lexical cues contributing to the model's performance.\" These two statements cannot both stand as written. The empirical core of the paper is a controlled approximation, and the central claim should be reworded to something like \"models trained on heavily filtered speech, which retains prosodic cues but also some residual lexical information, perform above chance\" until the residual-lexical confound is quantified or removed.","section":"Abstract, §6 Limitations"}],"minor_comments":[{"comment":"Section 4.2 says the mixture experiment uses 0%, 5%, and 100% of the other training sets, but later in the same section and in the Conclusion the text says \"10%\" (e.g., \"even when only 10% of the training data contains lexical information\"); the inconsistency should be fixed to match Figure 5.","section":"§4.2, Conclusion"},{"comment":"Figure 4 shows no error bars or significance intervals; given that all main results are reported as means over three seeds, the cutoff-sweep plot should include the same uncertainty information before claiming \"no significant performance drop.\"","section":"Figure 4"},{"comment":"The caption uses \"960h Librispeech\"; this should be expanded to \"960-hour LibriSpeech\" and made consistent with the rest of the text.","section":"Table 3 caption"},{"comment":"The Limitations section refers to \"SLUE-PHASE2\" but the dataset is called \"SLUE-SQA-5\" throughout the paper; rename for consistency.","section":"§6 Limitations"},{"comment":"Minor grammatical and typographical issues include: \"the model predominantly rely on lexical cues\" (Conclusion) should be \"relies\"; \"Verified-Test\" in Table 2 is hyphenated inconsistently; and the sentence \"between the 200Hz and 400Hz\" in Section 4.1 needs rewording.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"To the editor: The paper is honest about its limitations and the experimental design is more careful than much of the prior work, but the first central claim is currently not cleanly identified. I recommend major revision rather than rejection because the residual-lexical confound can be addressed with additional control experiments or by appropriately qualifying the claim. The paper is a good fit for the journal's readership and will be stronger after revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The lexical-dominance finding is real and worth citing: when both channels are present, models default to lexical cues even with only 10% lexical training data. The prosody-alone claim, however, is not cleanly identified: the 300 Hz low-pass condition still has WER 57.5% on test and 44.5% on the verified set, so a chunk of the FF1 18.49 over chance 6.03 could be residual word information. The authors concede this in Section 6 but keep the abstract claim.\n\nWhat's new: a controlled separation of prosodic and lexical channels on a natural-speech SQA benchmark, with SSL discrete units through DUAL. The design has good checks—chance baseline, cutoff sweep, WER, random question-context pairing, and seed variance. The mixed-training curves in Figure 5 are a concrete new measurement of lexical dominance. Using natural speech instead of TTS matters for prosody research.\n\nSoft spots: the residual-lexical confound is load-bearing for the first research question. A low-pass filter below 300 Hz still preserves voicing, nasals, and the amplitude envelope, and the WavLM-CTC result shows substantial word-level information survives, especially on the verified set where the prosodic model scores best. The cutoff plateau in Figure 4 does not rule out a small but constant lexical signal, and the random-pairing experiment could still reflect passage-internal lexical cues. The lexical-condition flattening also leaves rhythm intact and introduces artifacts, but that matters less because the dominance result is robust. Code is not released, which hurts reproducibility.\n\nOverall: this is a careful, honest empirical study with a solid secondary finding and an over-stated primary finding. Worth a serious referee, yes—with a request to either tighten the prosodic condition (show residual lexical content is uncorrelated with answer spans, or lower the cutoff and re-check WER) or soften the abstract. If I worked on SLU, I'd cite the lexical-dominance result and treat the prosody-alone number with caution.","headline":"Solid lexical-dominance result; the prosody-alone claim is confounded by residual lexical info that the authors themselves report.","tokens_in":13913,"tokens_out":2166,"would_cite":true,"duration_ms":22010,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Prosody alone can guide spoken question answering, but models lean on words when they can","keywords":["prosody","spoken question answering","self-supervised speech representations","discrete speech units","delexicalization","low-pass filtering","lexical information","SLUE-SQA-5"],"falsifier":"Train a model on the residual high-frequency band (the part above 300 Hz that is removed in the prosodic condition) and test it on the SQA task; if it reaches an FF1 comparable to the 18.49 of the prosodic condition, the low-passed signal still carries usable lexical information and the prosody-only interpretation collapses.","tokens_in":12823,"feed_emoji":"🎙️","tokens_out":2293,"duration_ms":23830,"temperature":0.7,"pith_summary":"This paper asks whether prosodic information—pitch, stress, rhythm—can support spoken question answering (SQA) on its own, and whether models actually use it when lexical content is also available. Using the natural-speech SLUE-SQA-5 dataset, the authors create a prosodic-only condition by low-pass filtering audio at 300 Hz and a lexical-only condition by flattening pitch and intensity. They find that a model trained on prosody alone reaches an FF1 of 18.49 versus a 6.03 chance baseline, showing prosody carries real signal. But when lexical information is present, models overwhelmingly rely on it, even when only 10% of training data contains lexical cues. The paper concludes that prosody is a useful supplement that current integration methods fail to exploit alongside lexical features.","feed_headline":"Prosody alone scores 18.5 in spoken QA, but words dominate when present","feed_subtitle":"Low-pass-filtered speech still guides answer spans, yet models switch to lexical cues with just 10% word data.","key_machinery":"The experimental apparatus is the DUAL framework: WavLM self-supervised representations are converted via k-means clustering into discrete spoken units, deduplicated, and fed to a Longformer language model that predicts answer spans as temporal segments. Prosodic and lexical information are isolated by signal processing: the prosodic condition applies a 300 Hz low-pass filter intended to preserve F0 and rhythm while removing most vowel and consonant energy, and the lexical condition flattens fundamental frequency and intensity to remove intonation and stress while leaving word content intact. This pair of controlled manipulations lets the authors compare models trained and tested within and across conditions.","core_discovery":"On the SLUE-SQA-5 spoken question answering benchmark, models trained only on low-pass-filtered audio (preserving prosody while reducing lexical content) score FF1 18.49 on the test set, far above the 6.03 white-noise chance baseline, though well below the 32–36 range of natural and lexical conditions. When a model trained on the prosodic condition is tested on natural or lexical audio, performance collapses, and the reverse holds as well, indicating the model does not transfer prosodic knowledge to word-bearing conditions. When questions are randomly paired with contexts, prosodic-only performance drops to 9.77 FF1 but stays above chance, implying prosody directs attention to salient passage regions without connecting them to the question. In mixture training, evaluation loss shows that even 5–10% lexical data in the training set makes the model learn lexical cues first, demonstrating a strong lexical bias when both channels are available.","pith_inferences":["If prosody's value lies in highlighting salient segments, then tasks with longer, noisier, or partially intelligible passages—where lexical cues are degraded—may show a larger prosody benefit than the extractive span-prediction setup used here.","A testable extension is to keep the 300 Hz cutoff but additionally flatten the F0 contour: if FF1 stays near 18.5, the prosodic score is not carried by pitch; if it collapses, F0 is the key carrier.","Residual lexical information in the filtered audio (WER 57.5% on the filtered test set) means the true prosody-only ceiling is lower than 18.49; better delexicalization methods, such as vocoder-based prosody transfer, could sharpen the estimate.","The layer choice for WavLM representations may matter: earlier layers are thought to encode prosody more strongly, so using a layer-wise search could reveal a larger prosody signal than reported."],"forward_implications":["Prosody-only models can meaningfully locate answer-relevant regions in a passage, suggesting prosodic cues are usable in the absence of intelligible words.","When building SQA systems, lexical information will dominate unless integration mechanisms are deliberately designed to weight prosodic cues.","Training on mixtures of prosodic and lexical data with only a small lexical fraction still shifts the model to lexical reliance, so data-balancing alone will not encourage prosody use.","The random-pairing result implies prosodic features can serve as an attention prior over the passage, a property that could be exploited in weakly supervised or retrieval settings."],"supporting_citations":[{"why":"Supplies the SLUE-SQA-5 dataset of natural spoken questions and passages, the benchmark on which all experiments are run.","marker":"(Shon et al., 2023)"},{"why":"Contributes the DUAL framework that converts WavLM representations into discrete units for textless spoken question answering.","marker":"(Lin et al., 2022)"},{"why":"Provides WavLM, the self-supervised model whose representations are quantized into the discrete units used by DUAL.","marker":"(Chen et al., 2022)"},{"why":"Supplies Longformer, the long-document transformer used as the pre-trained language model that predicts answer spans.","marker":"(Beltagy et al., 2020)"},{"why":"Defines the Frame-level F1 metric used to evaluate temporal answer-span predictions.","marker":"(Chuang et al., 2020)"},{"why":"Introduces the Audio Overlapping Score metric and the spoken question answering format with timestamp answers.","marker":"(Lee et al., 2018)"}],"fun_headline_variants":["Prosody alone hits 18.5 in spoken QA, far above chance baseline","Spoken QA: prosody alone tops chance, but words dominate when present","Prosody alone scores 18.5 FF1 in QA; lexical cues still win","Spoken QA: prosody alone beats chance, but adds little when words appear"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The 300 Hz low-pass filter is assumed to strip away enough lexical content that above-chance performance on the prosodic condition can be credited to prosody, yet a recognizer still transcribes 57.5% of the filtered test audio correctly.","fun_headline_variants_meta":{"raw":{"variants":["Prosody alone hits 18.5 in spoken QA, far above chance baseline","Spoken QA: prosody alone tops chance, but words dominate when present","Prosody alone scores 18.5 FF1 in QA; lexical cues still win","Spoken QA: prosody alone beats chance, but adds little when words appear"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000544,"raw_usage":{"total_tokens":2578,"prompt_tokens":892,"completion_tokens":1686,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":508,"completion_tokens_details":{"reasoning_tokens":1599}},"tokens_in":508,"tokens_out":1686,"duration_ms":11604,"temperature":1.0,"reasoning_tokens":1599,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T19:32:07.591570+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a model on the residual high-frequency band (the part above 300 Hz that is removed in the prosodic condition) and test it on the SQA task; if it reaches an FF1 comparable to the 18.49 of the prosodic condition, the low-passed signal still carries usable lexical information and the prosody-only interpretation collapses.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the SLUE-SQA-5 dataset of natural spoken questions and passages, the benchmark on which all experiments are run."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Contributes the DUAL framework that converts WavLM representations into discrete units for textless spoken question answering."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Frame-level F1 metric used to evaluate temporal answer-span predictions."}],"review_version":1}