{"id":"8e7ed904-c5cf-41b0-bf34-f14a37bc87ed","arxiv_id":"2505.20050","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Combining sustained vowel and sentence reading recordings through intermediate feature fusion with transformers improves voice pathology detection AUC by up to 13% over single-source methods.","lead":"This paper reports a machine learning method that combines two kinds of voice recordings, sustained vowels and sentence readings, to detect voice pathologies automatically. The authors show that fusing the two recording types at an intermediate feature level improves detection accuracy by up to 13% over single-source baselines across three languages.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Hyperparameter selection on the same 10-fold test set likely inflates the reported IFF-TE advantage; the fusion method, layer, and augmentation probabilities need nested validation before the +13% AUC claim is accepted.","rationale":"The reader's verdict is CONDITIONAL, and my stress-test supports keeping it conditional rather than moving to accept or reject. The reader's weakest_assumption focused on truncation/random selection and speaker independence; my load-bearing concern is closely related but different in mechanism: hyperparameters (fusion strategy, layer, L, augmentation probabilities) appear to be chosen using the same folds that are subsequently reported as test AUC. This is an evaluation-protocol issue rather than an internal inconsistency, and it directly affects the headline claim of up to +13% AUC and the 'TE best' conclusion. I do not regard this as grounds for rejection: the paper presents a plausible architecture, consistent improvements over single-source baselines across three datasets, and a frozen-backbone variant that also outperforms most baselines. However, the absence of nested validation or a fixed pre-registered protocol means the exact magnitude and the optimal-configuration claim are not yet secure. The concrete test I propose would settle the question by re-evaluating with unbiased model selection. I partially agree with the reader: they identified selection of fusion layer and augmentation probabilities in their rationale, but their stated weakest assumption emphasized data truncation and random selection instead. My independent reading converges on the same overall conditional verdict.","tokens_in":8750,"tokens_out":4466,"duration_ms":46536,"concrete_test":"Run nested cross-validation: for each outer fold, use only the inner training/validation split to select the fusion method, feature extraction layer, number of transformer layers, and augmentation probabilities; then evaluate the selected configuration on the untouched outer test fold. Compare the resulting AUC for IFF-TE against the best single-source baseline and against the paper's reported values. If the selected configuration is not TE/layer 5, or if the test AUC gap drops below the reported 10-13%, the central claim's magnitude is inflated by selection bias. A simpler first check: fix all hyperparameters a priori (TE, layer 5, L=2, augmentation 10%/25%) and rerun Table 1; if any dataset's AUC shifts by more than 1-2 points, selection on the test folds was present.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that IFF with a transformer encoder (TE) and fine-tuned backbones yields 95.8/96.3/93.6 AUC, a 10-13% improvement over single-source baselines. However, several key choices appear to have been made by looking at the same folds that are later reported as test performance. Section 3.2 states that augmentation probabilities are 'empirically-determined' (25% for sentences, 10% for vowels), that L=2 transformer layers are used because they give an 'effective balance', and that features are extracted from the 5th layer 'see Section 4.2 for detailed ablation studies'. Section 4.2 then selects the fusion strategy (TE) and the layer (5th) based on AUC on the same 10-fold cross-validation. If these hyperparameters were fixed after inspecting the test-fold results, the reported IFF-TE numbers are selected, not evaluated, on the test folds. This biases not only the comparison among fusion strategies (Table 2) but also the headline gap to single-source baselines, because the baselines do not receive the same per-strategy selection. The differences driving the 'TE best' and 'layer 5th best' conclusions are modest (TE .958 vs FiLM .951 on SVD; TE .963 vs FiLM .961 on AVFAD; layer 5th .958 vs 4th .944 on SVD), so selection-on-test can plausibly account for a meaningful part of the reported advantage. The paper does not describe a held-out validation set or nested cross-validation. The general claim that multi-source fusion helps is supported by WC and DLC also beating single-source, so the concern is not that fusion is useless; rather, the exact magnitude of the improvement and the specific claim that TE at layer 5 is optimal are not securely established under the current evaluation protocol.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes MVP, a multi-source voice pathology detection framework that fuses sustained-vowel and sentence-reading recordings. Using HuBERT backbones pre-trained on LibriSpeech and AudioSet, the authors compare three fusion levels (waveform concatenation, intermediate feature fusion, and decision-level combination) and five intermediate fusion strategies (concatenation, attention pooling, gating, FiLM, and transformer encoder). Experiments across three datasets (German SVD, Portuguese AVFAD, Italian IPV) with speaker-independent 10-fold cross-validation report that IFF with a transformer encoder and fine-tuned backbones achieves AUCs of 95.8%, 96.3%, and 93.6%, respectively, a relative improvement of up to about 13% over the best single-source baseline. The paper includes ablations on fusion strategy, backbone pre-training, and feature-extraction layer.","tokens_in":9090,"tokens_out":10013,"duration_ms":105983,"significance":"If the reported results are unbiased, the contribution is substantial: the paper provides the first systematic comparison of fusion levels for multi-source voice pathology detection, with consistent gains across three languages and a reproducible experimental pipeline (code link provided). The use of speaker-independent folds, multiple baselines, and ablations is a genuine strength. However, the quantitative headline—especially the choice of TE over FiLM and of layer 5 over layer 4—rests on model selection performed on the same test folds, and this needs to be corrected before the specific AUC gains can be accepted. The qualitative finding that multi-source fusion helps is more robust, since WC and DLC also beat single-source baselines, but the paper's central quantitative claims require nested validation or a separate hold-out set.","major_comments":[{"comment":"Hyperparameter selection is performed on the same folds used to report final results. The text in Section 3.2 describes the augmentation probabilities as 'empirically-determined', sets L=2 TE layers as an 'effective balance', and states that features are extracted from the 5th layer ('see Section 4.2 for detailed ablation studies'). Section 4.2 then chooses the fusion strategy (Table 2) and the feature layer (Table 4) by AUC on the same 10-fold cross-validation. Since no held-out validation set or nested cross-validation is described, the reported IFF-TE AUCs (95.8, 96.3, 93.6) are selected, not evaluated, on the test folds. The bias is material because the gaps that drive the conclusions are small: TE vs FiLM is 0.958 vs 0.951 on SVD, 0.963 vs 0.961 on AVFAD, and 0.936 vs 0.934 on IPV, and layer 5 vs layer 4 is 0.958 vs 0.944 on SVD. The comparison with single-source baselines is also unfair because those baselines do not receive the same per-strategy selection. Please provide nested cross-validation or a separate hold-out set for all hyperparameter choices, or explicitly present the IFF-TE numbers as a selected configuration.","section":"3.2, 4.2, Tables 2 and 4"},{"comment":"The choices of augmentation probabilities (25% for sentences, 10% for vowels) and the number of fusion transformer layers (L=2) are presented without any supporting ablation or validation protocol. If these values were fixed after inspecting test-fold AUCs they inherit the selection bias described in the previous comment; if they were chosen a priori, the paper should say so. In particular, L controls the capacity of the TE fusion, and no evidence is given that L=2 is optimal or that the TE advantage over FiLM and AP is stable across L.","section":"3.2"},{"comment":"The paper reports mean±standard deviation across 10 folds but provides no paired significance tests or confidence intervals, so the 'best' designations are not statistically supported. For example, on IPV the TE and FiLM AUCs differ by 0.002 (0.936±0.053 vs 0.934±0.053), and on SVD TE vs Gating differ by 0.011 (0.958±0.062 vs 0.947±0.068) with overlapping standard deviations. With only 10 folds, the authors should add a paired test across folds (e.g., Wilcoxon signed-rank) or report effect sizes before claiming that TE is the best fusion strategy and that the 5th layer is optimal.","section":"Tables 1-4"},{"comment":"The claim that the frozen IFF variant 'still outperforms four of five single-source baselines' is not supported by the table. Comparing AUCs, the frozen model outperforms 4 of 5 baselines on SVD (0.832 vs 0.850, 0.732, 0.810, 0.760, 0.765), but on AVFAD it outperforms only 3 of 5 (0.834 is below LS→Sent 0.877 and AS→Sent 0.855), and on IPV only 2 of 5 (0.809 is below LS→Sent 0.847, AS→Sent 0.815, and LS→Mix 0.831). This overstatement should be corrected, or the claim should be restricted to the SVD dataset.","section":"Section 4, Table 1"}],"minor_comments":[{"comment":"There are typographical issues such as 'V oice' with a stray space in the abstract and 'This is likely because' with an upper-case T after a comma in Section 4; these should be fixed.","section":"Abstract and Section 4"},{"comment":"The dimensions of the learnable matrices W, W_gamma, and W_beta in the Gating and FiLM equations are not specified; please define them explicitly for reproducibility.","section":"Eqs. (4)-(6)"},{"comment":"For AVFAD and IPV, the random selection of one sentence and one vowel per subject is performed with no reported seed; please report the seed or average over multiple random selections to show that the results are not an artifact of a particular draw.","section":"3.1"},{"comment":"The 5.0-second truncation may discard the ends of longer sentence readings; please report how many recordings are truncated and whether the results are sensitive to the truncation length.","section":"3.2"}],"recommendation":"major_revision","confidential_remarks":"The methodological gap is genuine but fixable; I would not reject. The paper's qualitative conclusion that multi-source fusion helps is supported by the WC and DLC results, but the quantitative IFF-TE advantage and the layer/fusion choices need nested validation or a separate hold-out set. The manuscript reads like a strong conference paper; for a journal, the statistical validation needs to be strengthened."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a useful empirical study. Combining sustained vowel and sentence recordings for voice pathology detection is not a new idea, but the explicit comparison of three fusion levels — waveform concatenation, intermediate feature fusion, and decision-level combination — across three languages and three datasets is a well-defined contribution. The consistent gains from fusion over single-source baselines, including the simpler WC and DLC methods, suggest the core finding is real: multi-source analysis helps. The 10-fold speaker-independent splits, multiple baselines, and ablations on backbone pre-training and layer depth are more care than typical for this subfield.\n\nThe soft spots are real too. The concern about hyperparameter selection is accurate. Augmentation probabilities, the number of transformer layers, and the feature extraction layer appear to have been chosen after inspecting the same 10-fold CV results that are later reported as test performance. Section 3.2 calls the augmentation probabilities 'empirically-determined,' and Section 4.2 selects the 5th layer based on AUC on the same folds. The differences between TE and FiLM, and between layer 5 and layer 4, are small in places, so selection-on-test could plausibly account for part of the reported advantage. The fix is straightforward: nested validation or a held-out validation set. Also missing is any statistical significance testing; fold standard deviations overlap for some of the key comparisons. I also could not verify the GitHub link contains working code and data — if it is a placeholder, that hurts reproducibility.\n\nOn balance, the central claim that multi-source fusion helps is supported. The exact magnitude of the improvement and the specific claim that TE at layer 5 is optimal are not secure under the current protocol. This deserves a serious referee — it is a solid empirical contribution that needs a methodology fix, not a desk reject. I would not quote the +13% figure without caveats, but I would cite it as a baseline for multi-source fusion in pathology detection.","headline":"A practical multi-source fusion study that likely overstates its headroom due to hyperparameter selection on the test folds.","tokens_in":9666,"tokens_out":2598,"would_cite":true,"duration_ms":25924,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Combining sustained-vowel and sentence recordings through intermediate transformer fusion lifts voice-pathology AUC by 10–13 points across three languages.","keywords":["voice pathology detection","multi-source fusion","intermediate feature fusion","transformer","HuBERT","sustained vowel","sentence reading","AUC"],"falsifier":"Re-run the same 10-fold protocol with full-length recordings, or draw several different sentence/vowel selections per subject, and check whether the intermediate-fusion AUC still beats the best single-source baseline by the same margin; as a leakage check, hold out a different recording session of the same speaker and see whether the model still separates healthy from pathological voices.","tokens_in":8568,"feed_emoji":"🗣️","tokens_out":4809,"duration_ms":49300,"temperature":0.7,"pith_summary":"MVP claims that automated voice-pathology screening is best done by analyzing two recording types together rather than either alone. The paper combines a sustained-vowel recording and a sentence reading through per-source transformer backbones and fuses their intermediate features with a small transformer encoder. Across German, Portuguese, and Italian datasets, this intermediate feature fusion reaches AUCs of 95.8%, 96.3%, and 93.6%, improving on the best single-source baseline by 10–13 AUC points. The result supports a concrete clinical workflow: collect both a vowel and a sentence, then let a model merge them before deciding.","feed_headline":"Fusing vowels and sentences lifts voice-pathology AUC to 96%","feed_subtitle":"Transformer fusion of sustained vowels and read sentences beats either source alone on German, Portuguese, and Italian data.","key_machinery":"The central mechanism is Intermediate Feature Fusion (IFF) implemented with a Transformer Encoder (TE). Frame-level representations from two specialized HuBERT backbones—one pre-trained on LibriSpeech for the sentence, one pre-trained on AudioSet for the sustained vowel—are concatenated along the time axis into one sequence, and a two-layer transformer encoder allows every time step of either source to attend to every time step of both sources. Attention pooling then collapses the fused sequence to a single vector for a linear classifier, so the model learns cross-source temporal correlations rather than relying on pooled averages or late score averaging.","core_discovery":"On its own terms, the paper establishes that sustained vowels and sentence readings carry complementary diagnostic information that is best exploited by fusing them before classification, not by training separate models or concatenating raw waveforms. Using HuBERT pre-trained on speech for sentences and HuBERT pre-trained on acoustic events for vowels, extracting frame-level features from the fifth layer, concatenating the two sequences along time, and passing them through a two-layer transformer encoder yields the highest AUC on all three datasets. The method beats waveform concatenation, decision-level averaging, and every single-source baseline, including a model trained on a mixture of both source types. The authors interpret this as evidence that fine-grained temporal interaction between the two sources is the mechanism that carries the improvement.","pith_inferences":["If the 5.0-second truncation preserves diagnostic content in routine clinical settings, the same pipeline could be deployed on short prompted recordings from a phone or a portable recorder.","The two-source fusion pattern may transfer to other speech-disorder screenings, such as dysarthria, Parkinson's disease, or dementia, where connected speech and sustained phonation are both routinely collected.","The paper's random selection of one sentence and one vowel per subject leaves within-subject variability untested; a prospective study with repeated recordings would show whether the gains persist across different utterances of the same speaker.","Since simple concatenation of pooled features trails the transformer fusion by up to 4.7 AUC points, the value of the method may lie in modeling temporal alignment between vowel and sentence, a hypothesis that could be tested by shifting one source relative to the other in time."],"forward_implications":["Screening protocols that collect both a sustained vowel and a sentence reading should expect better discrimination than single-task protocols, by roughly 10–13 AUC points.","Intermediate fusion should be preferred over both waveform concatenation and decision-level ensembling, because it is the strategy that lets the two sources interact at the feature level.","Specialized pretraining matters: a speech-pretrained backbone for sentences and an acoustic-event-pretrained backbone for vowels outperform other pairings.","Even with frozen backbones, intermediate fusion beats most single-source baselines with far fewer trainable parameters, making the approach usable where full fine-tuning is too expensive."],"supporting_citations":[{"why":"supplies the prior transformer-based voice-pathology pipeline that the decision-level combination mimics and that the paper extends to multi-source fusion.","marker":"[6]"},{"why":"defines HuBERT, the self-supervised backbone used for both sentence and vowel feature extraction.","marker":"[21]"},{"why":"provides the LibriSpeech pretraining corpus for the sentence backbone.","marker":"[24]"},{"why":"provides the AudioSet pretraining data for the sustained-vowel backbone.","marker":"[25]"},{"why":"introduces voc2vec, the non-verbal vocalization model used as an alternative vowel backbone in the ablation.","marker":"[23]"},{"why":"supplies the FiLM conditioning mechanism used as one of the intermediate fusion baselines.","marker":"[28]"},{"why":"provides the wav2vec 2.0 architecture that voc2vec mimics, grounding the ablation on alternative representations.","marker":"[29]"}],"fun_headline_variants":["Fusing vowels and sentences lifts pathology detection","Intermediate fusion wins for voice disorder diagnosis","Transformer fusion boosts voice pathology AUC","MVP: vowel-sentence fusion beats single sources","Two speech types, one transformer: better pathology detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that cutting every recording to 5.0 seconds and randomly selecting one sentence and one vowel per subject preserves the diagnostically relevant information, and that the speaker-independent folds keep all recordings of a subject on one side of the split; if either fails, the reported fusion gains could be sampling artifacts rather than true complementarity.","fun_headline_variants_meta":{"raw":{"variants":["Fusing vowels and sentences lifts pathology detection","Intermediate fusion wins for voice disorder diagnosis","Transformer fusion boosts voice pathology AUC","MVP: vowel-sentence fusion beats single sources","Two speech types, one transformer: better pathology detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000143,"raw_usage":{"total_tokens":1091,"prompt_tokens":781,"completion_tokens":310,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":397,"completion_tokens_details":{"reasoning_tokens":244}},"tokens_in":397,"tokens_out":310,"duration_ms":4229,"temperature":1.0,"reasoning_tokens":244,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:59:23.772254+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same 10-fold protocol with full-length recordings, or draw several different sentence/vowel selections per subject, and check whether the intermediate-fusion AUC still beats the best single-source baseline by the same margin; as a leakage check, hold out a different recording session of the same speaker and see whether the model still separates healthy from pathological voices.","supporting_citations":[{"cited_title":"1555 11/10/2022, PE00000013), is partially supported by the ”D.A.R.E","cited_arxiv_id":null,"evidence_quote":"supplies the prior transformer-based voice-pathology pipeline that the decision-level combination mimics and that the paper extends to multi-source fusion."},{"cited_title":"Exploiting foundation models and speech enhancement for parkinson’s disease detection from speech in real-world operative conditions,","cited_arxiv_id":null,"evidence_quote":"provides the LibriSpeech pretraining corpus for the sentence backbone."},{"cited_title":"Bilingual dual-head deep model for parkinson’s disease detec- tion from speech,","cited_arxiv_id":null,"evidence_quote":"provides the AudioSet pretraining data for the sustained-vowel backbone."},{"cited_title":"Film: Visual reasoning with a general conditioning layer,","cited_arxiv_id":null,"evidence_quote":"supplies the FiLM conditioning mechanism used as one of the intermediate fusion baselines."},{"cited_title":"Benchmarking representations for speech, music, and acoustic events,","cited_arxiv_id":null,"evidence_quote":"provides the wav2vec 2.0 architecture that voc2vec mimics, grounding the ablation on alternative representations."}],"review_version":1}