{"id":"d87615a0-cf93-4eda-86cd-987e4e289f12","arxiv_id":"2507.14898","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Parameter-efficient fine-tuning (LoRA/DoRA) of Whisper improves cleft lip and palate severity classification over frozen embeddings and handcrafted features on two child speech corpora.","lead":"This paper tests whether lightly fine-tuning a large speech model, Whisper, with small trainable adapters improves detection and severity grading of cleft lip and palate speech. On two child speech datasets, the adapted models beat frozen-embedding baselines and traditional acoustic features, though the gains are modest and rest on splits without stated speaker separation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Random utterance-level partition risks speaker leakage: with severity labeled per speaker and only 65/60 speakers, overlapping speakers across train/eval can inflate all reported F1 gains and invalidate the PEFT comparison.","rationale":"The reader's weakest assumption is exactly the one I would flag. The paper never states 'speaker-independent' or 'speaker-disjoint'; it says only 'randomly divided.' Since severity labels are assigned to speakers, this is not a stylistic omission but a potential shortcut that can invalidate every comparison in Table I. I do not see an internally contradictory argument; the headline numbers are arithmetically consistent with the table, and macro-F1 is an appropriate metric for the imbalanced severity classes. Secondary weaknesses are real but less load-bearing: there are no confidence intervals or significance tests, the model-selection step on the evaluation set is ambiguous, and the DoRA equation in Sec II-B as written algebraically reduces to LoRA. These do not change the primary concern. The appropriate disposition remains conditional: the claim is plausible and directionally supported, but the speaker-split evidence must be supplied before the reported F1 improvements can be accepted as evidence of generalization to new speakers. I therefore do not move the reader's verdict.","tokens_in":9200,"tokens_out":7851,"duration_ms":89760,"concrete_test":"Check whether the released or reconstructed train/dev/eval partitions are speaker-disjoint. If they are not, rerun the severity-classification protocol (Sec. III-IV) with leave-one-speaker-out cross-validation, using identical hyperparameters for Whisper+LoRA/DoRA, Whisper-SVM, and eGeMAPS-SVM. The decisive comparison is macro-F1 on folds whose speakers were completely unseen during training. If the LoRA/DoRA gains over the baselines shrink substantially or disappear, the headline improvements are speaker-leakage artifacts; if the gains persist, the concern is refuted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III-A reports only that NMCPC (1,463 utterances) and AIISH (2,726 utterances) were 'randomly divided' into training, development, and evaluation sets, with no statement that the split is speaker-independent. This matters because severity is a speaker-level attribute: the clinicians classified each CLP speaker as mild/moderate/severe, so every utterance from one speaker carries the same label. With 65 and 60 speakers respectively, a random utterance-level split will almost certainly place utterances from the same speaker in both training and evaluation. A Whisper adapter (or an SVM on pooled embeddings) can then exploit the speaker-identity shortcut: identify the talker and copy that talker's severity label, rather than learning a transferable acoustic marker of CLP severity. The central claim — that LoRA/DoRA fine-tuning improves severity macro-F1 over frozen Whisper-SVM and eGeMAPS-SVM baselines — rests on the evaluation measuring generalization to unseen speakers. If the partitions overlap in speakers, the reported relative gains in Table I (26.4%/63.4% on NMCPC, 6.1%/52.9% on AIISH) are inflated to an unknown degree and the headline comparison is not a valid test of the method.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper evaluates CLP detection and four-class severity classification from speech using two datasets (NMCPC, English; AIISH, Kannada). It compares traditional handcrafted features (eGeMAPS, ComParE, merged) with an SVM classifier against frozen-embedding representations from Whisper, Wav2Vec2, and WavLM, also paired with SVMs. The best frozen model, Whisper, is then fine-tuned with LoRA and DoRA adapters, adding a classification head and freezing the rest of the encoder. The central claim is that this PEFT approach yields relative macro-F1 improvements over the best foundation-model baseline and the best handcrafted-feature baseline of 26.4% and 63.4% on NMCPC and 6.1% and 52.9% on AIISH for severity classification, with smaller improvements for binary CLP detection (Table I). The paper positions the contribution as the first comprehensive comparison of such foundation-model representations for CLP speech, plus a PEFT adaptation strategy.","tokens_in":9410,"tokens_out":3674,"duration_ms":43290,"significance":"If the reported results are statistically secure, the paper would provide useful evidence that lightweight adapters on a weakly supervised speech model can outperform both conventional acoustic features and frozen large-model embeddings for a low-resource clinical speech task. The choice of two typologically different languages (English and Kannada), the systematic comparison across three foundation models, and the use of macro-F1 as the primary metric are strengths. The adaptive component is modest in scale (rank 8 adapters), which makes the approach practically attractive for clinical deployments. However, the current manuscript does not demonstrate that the central comparison survives a speaker-independent evaluation, and it provides no uncertainty quantification. The claimed gains, especially the 6.1% improvement on AIISH severity, could easily be within run-to-run or partitioning noise. The paper would be strengthened substantially by speaker-disjoint splits, repeated runs with confidence intervals, and significance tests.","major_comments":[{"comment":"The paper states only that each dataset was 'randomly divided' into training, development, and evaluation sets, without specifying a speaker-independent split. Severity labels are speaker-level attributes: all utterances from one speaker carry the same severity label, and the datasets have only 65 (NMCPC) and 60 (AIISH) speakers. A random utterance-level partition will almost certainly place utterances from the same speaker in both training and evaluation, allowing models to exploit speaker identity rather than generalize to unseen talkers. This is load-bearing because the headline relative improvements in Table I are meant to demonstrate that PEFT adapters improve severity classification over frozen-embedding and handcrafted baselines. The authors must either confirm that the splits are speaker-disjoint or rerun the evaluation with speaker-independent partitioning (e.g., leave-speakers-out cross-validation).","section":"Sec. III-A, Table I"},{"comment":"All results are reported as single point estimates with no confidence intervals, standard deviations, or significance tests. Several key comparisons are very close, for example Whisper LoRA vs. Whisper on AIISH severity (0.52 vs. 0.49), Whisper DoRA vs. Whisper LoRA on AIISH detection (0.93 vs. 0.94), and Whisper DoRA vs. LoRA on NMCPC severity (0.67 vs. 0.65). Without repeated runs or a paired test such as McNemar's test or a bootstrap over utterances/speakers, the stated relative improvements are not statistically secured. The authors should report mean and variance over multiple random seeds and, ideally, over multiple speaker-disjoint resamplings, and they should assess which pairwise differences are significant.","section":"Sec. IV, Table I"},{"comment":"The description of model selection and evaluation is ambiguous. The text says, 'For all experiments, we pool the training and development sets and apply 5-fold cross-validation. We select the model that achieves the highest macro-averaged F1 score for final evaluation on the evaluation set.' It is not explained how the development set is used for the PEFT models, how many training epochs are run, whether early stopping is applied, or how the 5-fold CV interacts with the fixed train/dev/eval partition. This makes the experimental protocol irreproducible and makes it unclear whether the evaluation-set numbers are selected with any form of peeking. The authors should provide a precise protocol: what is tuned, what is selected, and how the final evaluation set is used.","section":"Sec. III, experimental protocol"}],"minor_comments":[{"comment":"In the DoRA description, the definition 'M = ||W0 + BA||' is ambiguous because it is not stated whether M is a scalar, a per-column vector, or a matrix. The original DoRA formulation uses a magnitude vector per output dimension; please clarify the notation used here.","section":"Sec. II-B, Eq. (2)"},{"comment":"The optimizer is spelled 'AdaM'; it should be 'Adam'. Also, the number of training epochs and any early-stopping criterion are not reported.","section":"Sec. III-D"},{"comment":"Please report the number of utterances per speaker or at least the range, since the risk of speaker leakage depends on how many utterances each speaker contributes. Reporting the random seed or stating that the split is speaker-disjoint would also aid reproducibility.","section":"Sec. III-A"},{"comment":"The sentence 'These results clearly show that eGeMAPS provides better discrimination between normal and CLP speech than ComParE and its combination with eGeMAPS' is confusing because the merged set is a superset of eGeMAPS; the intended claim is presumably that the reduced, curated eGeMAPS set outperforms the larger merged set.","section":"Sec. IV-A"},{"comment":"Reference [8] (Dhananjaya and Yegnanarayana, 'Speaker change detection in casual conversations using excitation source features') appears unrelated to the claim about excitation characteristics in CLP speech; please replace it with an appropriate source on excitation source features in disordered speech.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for APSIPA ASC and addresses a clinically motivated application. The core concern is methodological: the absence of a speaker-independent split, combined with no statistical confidence measures, prevents the current experiments from supporting the headline claim. I would be willing to reconsider after the authors either confirm speaker-disjoint splits or rerun the evaluation accordingly and add uncertainty estimates. The fixed rank of 8 and the small gains on detection suggest the PEFT advantage may be marginal, so the revised version should be explicit about effect sizes and significance rather than relying on raw F1 deltas."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read on arXiv:2507.14898. The useful new bit: it is the first direct comparison of Whisper, Wav2Vec2, and WavLM embeddings on CLP detection and severity grading, and the first application of LoRA/DoRA adapters to Whisper for that task. On two child speech datasets (English NMCPC, Kannada AIISH) it shows Whisper embeddings beat the self-supervised models, and fine-tuning Whisper with LoRA or DoRA nudges macro-F1 up further — from 0.53 to 0.65/0.67 on NMCPC severity and 0.49 to 0.51/0.52 on AIISH. The eGeMAPS-vs-ComParE comparison is also clean and useful. The paper is straightforward and mostly honest in its claims.\n\nThe soft spot is the one you'd expect and it is load-bearing. Section III-A says only that utterances were 'randomly divided' into train/dev/eval. Severity is a speaker-level label — each of the 65 NMCPC and 60 AIISH speakers gets one severity rating, and with 22 and 45 utterances per speaker on average, a random utterance-level split will almost certainly put the same speaker in both training and evaluation. A Whisper adapter can then learn speaker identity and copy the label, and the reported F1 gains (26.4% relative over Whisper-SVM on NMCPC, 6.1% on AIISH) would be inflated. The paper never states that the split is speaker-independent. This has to be fixed before the central comparison is interpretable. It may be that the original datasets come with standard speaker-independent partitions; if so, the authors need to say so and re-run.\n\nTwo smaller issues: there are no confidence intervals or repeated runs — one partition, one model selection via 5-fold CV, then a single evaluation — so the 0.02 F1 differences between LoRA and DoRA on AIISH are within noise. And the DoRA description in Sec. II-B is garbled (talk of pruning single-rank components is not how DoRA works), though the update equation is roughly right. The trainable classification head on top of PEFT vs. frozen SVM baselines is a mild confound, not fatal.\n\nBottom line: the paper is a legitimate new application of off-the-shelf PEFT to a small clinical speech task. The empirical claim is plausible but not statistically secured, and the speaker-leakage risk is real. A serious referee should see it, because the fix is straightforward and the recipe could be useful for low-resource speech pathology triage. I'd recommend conditional acceptance after the authors confirm speaker-independent evaluation or re-do the split and report variance.","headline":"Useful first benchmark of PEFT adapters for CLP severity, but the evaluation split may leak speaker identity; needs speaker-independent confirmation before the headline gains are credible.","tokens_in":10002,"tokens_out":3499,"would_cite":false,"duration_ms":35719,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fine-tuning Whisper with LoRA or DoRA adapters improves cleft lip and palate speech severity classification over frozen-embedding and handcrafted-feature baselines on two corpora.","keywords":["cleft lip and palate speech","hypernasality severity classification","parameter-efficient fine-tuning","LoRA","DoRA","Whisper","self-supervised speech models","clinical speech diagnostics"],"falsifier":"Re-run the English and Kannada comparisons with speaker-disjoint splits, placing all utterances of a given speaker in only one partition, and check whether LoRA and DoRA macro-F1 advantages over frozen Whisper and eGeMAPS survive; if the gains shrink toward zero, the reported improvements are largely speaker-identity effects.","tokens_in":8992,"feed_emoji":"🗣️","tokens_out":9984,"duration_ms":103981,"temperature":0.7,"pith_summary":"This paper argues that fine-tuning a speech foundation model with small low-rank adapters improves detection and severity grading of cleft lip and palate (CLP) speech, a congenital condition whose nasalization increases with severity. The authors compare frozen embeddings from Wav2Vec2, WavLM, and Whisper, plus handcrafted eGeMAPS and ComParE features with SVM classifiers, and then fine-tune Whisper with LoRA and DoRA adapters. On the English corpus, DoRA raises severity macro-F1 from 0.53 with frozen Whisper and 0.41 with eGeMAPS to 0.67; on the Kannada corpus, LoRA raises it from 0.49 and 0.34 to 0.52. The reported relative severity-classification gains are 26.4% and 63.4% on the English corpus and 6.1% and 52.9% on the Kannada corpus over the best foundation-model and handcrafted-feature baselines. The implication is that parameter-efficient fine-tuning offers a low-resource route to adapting large speech models to disordered child speech.","feed_headline":"LoRA/DoRA adapters lift cleft-palate speech grading","feed_subtitle":"Fine-tuning Whisper with rank-8 adapters beats frozen embeddings and handcrafted features on two cleft-palate speech corpora.","key_machinery":"The carrying mechanism is low-rank adaptation. LoRA freezes the pre-trained weight matrix $W_0$ and learns a low-rank update $\\Delta W = BA$ with rank $r=8$, applied to the query, key, and value projections of Whisper's transformer; DoRA refines this by normalizing the direction of the update and learning a separate magnitude component, giving $W = M \\cdot D$. A mean-pooled utterance representation from the last encoder layer feeds a fully connected classifier, and cross-entropy loss trains only the adapters and the classification head while the rest of the encoder stays frozen. This lets a 768-dimensional Whisper encoder adapt to child CLP speech with very little trainable capacity.","core_discovery":"The central claim is that low-rank adapters applied to Whisper's attention projections recover more CLP-relevant speech information than frozen embeddings or traditional acoustic features, and do so with only a small number of trainable parameters. Concretely, Whisper with DoRA reaches macro-F1 0.67 for four-class severity classification on the English corpus, and Whisper with LoRA reaches 0.52 on the Kannada corpus, versus 0.53 and 0.49 for frozen Whisper embeddings with an SVM, and 0.41 and 0.34 for eGeMAPS features with an SVM. The authors conclude that parameter-efficient fine-tuning improves both CLP detection and severity grading on both datasets, with DoRA giving the best results on English and LoRA giving the best results on Kannada.","pith_inferences":["A speaker-disjoint evaluation is the natural next test: because the paper only says the data were randomly divided, the reported gains could partly reflect the model memorizing speakers rather than CLP severity.","The paper attributes the DoRA/LoRA reversal to language differences, but the two corpora also differ in size and class balance, so the reversal may instead reflect those factors; the paper does not disentangle them.","The comparison covers frozen embeddings and handcrafted features but not full fine-tuning of Whisper, so the efficiency-accuracy trade-off of adapters relative to updating all Whisper parameters remains open."],"forward_implications":["CLP severity grading can be improved while training only adapter weights and a classification head, so the large Whisper encoder itself does not need to be stored or retrained per task.","Frozen embeddings from foundation models leave CLP-relevant signal unused; the reported gains show that this residual information is recoverable with modest trainable capacity.","The best adapter differs by corpus, with DoRA ahead on English and LoRA ahead on Kannada, so practical systems should select adapters per language or per corpus rather than assuming one method wins everywhere.","The same recipe could transfer to other speech-disorder grading tasks with small child-speech corpora, since it does not require full-model fine-tuning."],"supporting_citations":[{"why":"Supplies the Whisper encoder whose attention projections are adapted by LoRA and DoRA.","marker":"[26]"},{"why":"Defines the low-rank adaptation mechanism used to update the frozen weight matrix.","marker":"[14]"},{"why":"Defines the weight-decomposed low-rank adaptation that separates magnitude and direction updates.","marker":"[19]"},{"why":"Provides the English CLP speech corpus with clinician-labeled severity levels used as the evaluation set.","marker":"[15]"},{"why":"Provides the Kannada CLP speech corpus used as the second evaluation set.","marker":"[24]"},{"why":"Supplies the eGeMAPS handcrafted feature set used as the strongest acoustic baseline.","marker":"[11]"},{"why":"Supplies the ComParE feature set and the merged baseline features.","marker":"[28]"},{"why":"Provides the Wav2Vec2 frozen-embedding baseline compared against Whisper.","marker":"[2]"},{"why":"Provides the WavLM frozen-embedding baseline compared against Whisper.","marker":"[5]"},{"why":"Provides the SVM classifier used with handcrafted features and frozen embeddings.","marker":"[7]"}],"fun_headline_variants":["Adapters boost cleft-palate speech grading","LoRA/DoRA fine-tune Whisper for better CLP detection","Parameter-efficient adapters sharpen cleft-palate severity scoring","Small adapters, big gains: CLP speech classification via PEFT","Whisper plus adapters outperforms on cleft-palate datasets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that randomly dividing 65 or 60 speakers' utterances into training, development, and evaluation sets never places the same speaker in more than one partition, so the adapters learn CLP severity rather than speaker identity.","fun_headline_variants_meta":{"raw":{"variants":["Adapters boost cleft-palate speech grading","LoRA/DoRA fine-tune Whisper for better CLP detection","Parameter-efficient adapters sharpen cleft-palate severity scoring","Small adapters, big gains: CLP speech classification via PEFT","Whisper plus adapters outperforms on cleft-palate datasets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000225,"raw_usage":{"total_tokens":1480,"prompt_tokens":973,"completion_tokens":507,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":589,"completion_tokens_details":{"reasoning_tokens":421}},"tokens_in":589,"tokens_out":507,"duration_ms":5132,"temperature":1.0,"reasoning_tokens":421,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:44:07.973154+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the English and Kannada comparisons with speaker-disjoint splits, placing all utterances of a given speaker in only one partition, and check whether LoRA and DoRA macro-F1 advantages over frozen Whisper and eGeMAPS survive; if the gains shrink toward zero, the reported improvements are largely speaker-identity effects.","supporting_citations":[{"cited_title":"Robust speech recognition via large-scale weak supervision","cited_arxiv_id":null,"evidence_quote":"Supplies the Whisper encoder whose attention projections are adapted by LoRA and DoRA."},{"cited_title":"Single frequency filter bank based long-term average spectra for hypernasality detection and assessment in cleft lip and palate speech","cited_arxiv_id":null,"evidence_quote":"Provides the English CLP speech corpus with clinician-labeled severity levels used as the evaluation set."},{"cited_title":"Hypernasality severity analysis in cleft lip and palate speech using vowel space area","cited_arxiv_id":null,"evidence_quote":"Provides the Kannada CLP speech corpus used as the second evaluation set."},{"cited_title":"Scherer, Bj ¨orn Schuller, Johan Sundberg, Elisabeth Andr ´e, Carlos Busso, Laurence Devillers, Julien Epps, Petri Laukka, Shrikanth S","cited_arxiv_id":null,"evidence_quote":"Supplies the eGeMAPS handcrafted feature set used as the strongest acoustic baseline."},{"cited_title":"Scherer, Fabien Ringeval, Mohamed Chetouani, Felix Weninger, Florian Eyben, Erik Marchi, Marcello Mortillaro, Hugues Salamin, Anna Polychroniou, Fabio Valente, and Samuel Kim","cited_arxiv_id":null,"evidence_quote":"Supplies the ComParE feature set and the merged baseline features."},{"cited_title":"Wav2vec 2.0: A framework for self-supervised learning of speech representations","cited_arxiv_id":null,"evidence_quote":"Provides the Wav2Vec2 frozen-embedding baseline compared against Whisper."},{"cited_title":"Wavlm: Large-scale self-supervised pre-training for full stack speech processing","cited_arxiv_id":null,"evidence_quote":"Provides the WavLM frozen-embedding baseline compared against Whisper."}],"review_version":1}