{"id":"ed6728b7-e663-4a02-98cb-3603d3af8f6c","arxiv_id":"2501.00039","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Reinforcement learning with word-error and meaning-preservation rewards adapts an LLM-based speech recognizer to disordered speech better than supervised fine-tuning in this study.","lead":"This paper turns a 2-billion-parameter language model into a speech recognizer by mapping audio chunks to spare text tokens, then tunes it with reinforcement learning to transcribe disordered speech better than continued standard fine-tuning. The tuned model still trails dedicated speech recognition systems, but the approach offers a new adaptation strategy.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The RL-vs-SFT comparison rests on a single under-specified continued-SFT baseline, and the claimed meaning-preservation advantage may be inflated by SFT's reported WER regression; a matched-budget retraining check is needed.","rationale":"The paper's central claim is well-framed and partially independently supported: the RLHF rows show consistent improvements across gamma values on both dev and test sets, the human SLP evaluation of 220 examples favors the gamma=1.0 model on meaning preservation, and the abstract is appropriately modest about not beating existing ASR systems. The token-repurposing trick is a small but clearly described variant, and the audio clustering details are specified. However, the headline scientific claim is comparative: 'tuning with reinforcement learning ... leads to substantially better performance than supervised fine-tuning.' The only evidence for that comparison in Table III is the single continued-SFT row. The gamma ablation compares RLHF variants against each other, the severity breakdown compares RLHF to the starting SFT model, and the human evaluation compares the two RLHF models—none of these directly test RLHF versus continued SFT. Thus the continued-SFT row carries the entire burden of the abstract's claim. The reader identified the under-specified SFT baseline as the weakest assumption, and I agree. My concern is slightly sharper: the reported SFT baseline did not merely fail to improve; its WER regressed from 50.4 to 57.1 relative to the starting point. Such a regression is atypical for a well-tuned continued fine-tuning run and is itself enough to suppress MP, so the measured MP gap may reflect poor baseline convergence rather than a semantic-reward benefit. This is a testable concern, not a reason to reject: a matched-budget SFT retraining could confirm that no SFT configuration reaches the RLHF MP level at comparable WER. Given that the paper's internal evidence is otherwise consistent and the human evaluation supports the within-RLHF comparison, keeping the reader's CONDITIONAL verdict is appropriate; the concern strengthens the case for requiring the baseline details and artifacts, but does not by itself overturn the verdict.","tokens_in":8446,"tokens_out":3374,"duration_ms":28923,"concrete_test":"Retrain continued SFT from the 30:70 SFT checkpoint on Euphonia training data with the same step budget, batch size, and validation-based early stopping as the RLHF runs; sweep the same number of hyperparameter configurations (learning rate, input dropout, data mixture) and report dev/test WER and MP for (a) the lowest-WER checkpoint, (b) the highest-MP checkpoint, and (c) the checkpoint selected by the same criterion used for RLHF. If no SFT configuration reaches MP within noise of 55.7 on Euphonia test at WER comparable to 42.6, the RL-superiority claim survives; if one does, the central claim is an artifact of baseline effort.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that RL tuning beats continued supervised fine-tuning for adapting an LLM-based ASR system to disordered speech. The entire RL-vs-SFT comparison in Table III rests on one continued-SFT row (WER 57.1, MP 42.8 on Euphonia test), which is worse than the starting SFT checkpoint (WER 50.4, MP 48.2). Section V-B states only that 'even with significant hyper parameter tuning' continued SFT does not help, and it does not report the search space, number of trials, learning-rate range, data mixture, or checkpoint-selection criterion for this baseline. This is load-bearing because: (1) a continued-SFT run that fails to match the starting model's WER is not a convincing 'tuned' baseline; at minimum it suggests poor early stopping, since the same training distribution that produced the starting checkpoint should remain learnable; (2) the human evaluation (Table IV) compares the two RLHF models (gamma=0.0 vs gamma=1.0) and does not test RLHF against continued SFT, so it cannot rescue the comparison; (3) the MP advantage of RLHF(gamma=1.0) over continued SFT (55.7 vs 42.8) may be substantially a WER artifact, since WER and MP are correlated and the SFT baseline's WER regressed by 6.7 points. If the SFT baseline was undertuned, the abstract's headline conclusion is an artifact of uneven effort rather than a property of the method.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a three-stage recipe for adapting an LLM to disordered-speech ASR. The authors replace the 1,024 least-frequent text tokens of Gemma 2B with audio codebook tokens obtained by clustering USM embeddings (clusters learned on LibriSpeech), fine-tune the resulting model on mixtures of LibriSpeech and the Euphonia disordered-speech corpus, and then apply PPO with a reward that combines an LLM-based Meaning Preservation (MP) score and log-transformed WER, R = γ·MP + ln(1 − WER). The main empirical claim is that RLHF adaptation from the 30:70 SFT checkpoint substantially improves over continued supervised fine-tuning on Euphonia: Table III reports WER 41.0–42.6 and MP 50.4–55.7 for RLHF at different γ values, versus WER 57.1 and MP 42.8 for continued SFT, and γ = 1.0 improves MP by 7.5 points over the starting SFT model with a statistically insignificant WER change (p = 0.54). A 220-example human evaluation by a speech-language pathologist supports the γ = 1.0 preference over γ = 0.0, and severity-stratified results show the largest gains on moderate and severe speech. The authors are explicit that the resulting model does not outperform existing ASR systems.","tokens_in":8771,"tokens_out":12892,"duration_ms":100389,"significance":"If confirmed, the result is significant for accessibility-oriented ASR: it offers an LLM-native way to adapt speech recognition to a high-variance, low-resource domain, and it makes the WER-versus-meaning-preservation trade-off explicit, which matters for disordered speech where WER alone understates comprehension. Strengths worth naming: the system is built on an open model (Gemma 2B) and public corpora; the paper honestly states that the model does not beat existing ASR systems; the automated MP preference between the two RLHF variants is validated with human speech-language-pathologist ratings (Spearman 0.684/0.639); the severity-stratified analysis (Fig. 3a) locates the gains precisely; and the inclusion of WER in the reward to mitigate reward hacking is a sensible design choice. The principal risks are that MP is both the reward and the primary evaluation metric and comes from the authors' own reward model, and that the central RL-versus-SFT comparison rests on a single under-specified continued-SFT baseline. These issues are fixable within the paper's scope, but they currently make the headline claim stronger than the presented evidence.","major_comments":[{"comment":"The headline claim that RLHF is substantially better than continued supervised fine-tuning rests entirely on one continued-SFT row in Table III (WER 57.1, MP 42.8) that is worse than the starting 30:70 SFT checkpoint it is adapted from (WER 50.4, MP 48.2). Because the continued-SFT run trains on the same distribution that produced the starting checkpoint, a properly tuned and early-stopped run should at least match it; the regression suggests an early-stopping or recipe failure rather than a property of supervised fine-tuning. Section V-B says only that 'even with significant hyper parameter tuning' continued SFT does not help, and it does not report the search space, number of trials, learning-rate range, data mixture, or checkpoint-selection criterion. Please provide a fully specified, budget-matched continued-SFT baseline (ideally one whose WER matches or improves on the starting model) and the same level of hyperparameter detail that the RLHF runs receive; without this, the central comparison may reflect uneven effort.","section":"V-B / Table III"},{"comment":"The MP score is used simultaneously as the RL reward term γ·MP in Eq. (1) and as the primary evaluation metric, and the Gemma-2B MP reward model is trained by the same group on the same task family (2,840 pairs; AUC 0.87 vs. 0.89 for the 62B judge of [16]). Part of the reported MP gain may therefore reflect overoptimization toward this particular judge rather than meaning preservation itself. In addition, the MP advantage of RLHF(γ=1.0) over Continued SFT (55.7 vs. 42.8) is confounded with the 14.5-point WER gap (42.6 vs. 57.1); since WER and MP are correlated, the advantage cannot be cleanly attributed to the reward choice. The human evaluation in Table IV compares only γ=0.0 vs. γ=1.0 and thus does not independently validate the central RL-vs-SFT claim. I ask that the main comparison be re-scored with an external judge (e.g., the 62B model of [16]) or with human labels on a sample that includes the continued-SFT outputs.","section":"IV (Eq. 1) and V-A/V-B"},{"comment":"The significance statements that carry the paper's claims are reported without the information needed to assess them: the text reports p < 1×10−8 for RLHF-vs-SFT, and p < 0.0003 / p = 0.54 for γ=1.0-vs-γ=0.0, but gives no sample sizes, no confidence intervals, and no statement of whether the two-sided t-tests are paired and computed per utterance, per speaker, or per phrase. Without this, the reader cannot judge whether the deltas are consistent across speakers or driven by a few outliers. Please report per-condition means with confidence intervals (e.g., bootstrap over speakers) and state the test unit explicitly.","section":"V-B"},{"comment":"The PPO configuration is not reproducible as described: the paper names only PPO with a clipped objective and KL regularization and says that checkpoints were selected on validation WER/MP, but it omits the learning rate, the KL coefficient or target, the number of PPO steps, the rollout and minibatch sizes, and how many reward-model evaluations were used. These details are central to a methods claim about RLHF for ASR adaptation; please report them or cite a specific public recipe that was followed.","section":"IV (Reinforcement Learning)"}],"minor_comments":[{"comment":"Typo: 'demonstarted' should be 'demonstrated'.","section":"IV"},{"comment":"Typo: 'disoredred' should be 'disordered'.","section":"Fig. 1 caption"},{"comment":"The sentence 'the SLP assessed ... on a 3-point 1scale' is garbled, and the footnote defining the scale ('Meaning Completely Preserved, Mild Error, Severe Error') has no marker in the text; please fix the wording and marker.","section":"V-E"},{"comment":"The asterisk note says '* is significant w.r.t WER (γ = 0.00) strategy', but the asterisks are placed on MP values; clarify that the significance is relative to the RLHF-WER (γ = 0.00) row.","section":"Table III caption"},{"comment":"Several free parameters (1024 clusters, 16th USM layer, input dropout 5×10−2, the 30:70 mixture ratio) are stated as outcomes of experiments ('pilot experiments ... showed less difference', 'we found 5 × 10−2 to be the best value') without the supporting results; a small ablation table or figure would strengthen the paper.","section":"III (Tuning details)"},{"comment":"The large WER improvement of RLHF-WER over the starting SFT checkpoint (50.4 → 41.0) and over Continued SFT (57.1 → 41.0) is left unexplained; since the γ=0.0 reward is just ln(1−WER), a sentence on why policy-gradient optimization extracts more WER gain from the same data than continued SFT would help.","section":"V-B"},{"comment":"Please specify how the 220 human-evaluated examples were sampled and from which split (test or dev); Table IV reports only aggregate percentages.","section":"V-E"},{"comment":"The claim that 'Training only on the Librispeech data or only the Euphonia datasets leads to extremely poor generalization' is not fully supported by Table I, which shows the Librispeech-only row but no Euphonia-only row; please add the Euphonia-only numbers or qualify the claim.","section":"V-A"}],"recommendation":"major_revision","confidential_remarks":"This is a solid systems paper with an honest framing, and it fits an applied speech/ML venue well; my concerns are about experimental rigor rather than novelty. One point the editor may want to weigh: the MP judge of [16] and the reward model trained here come from the same research group (with overlapping authors), which is legitimate but makes the evaluation less independent than the text implies; the authors should be encouraged to add an external scoring pass. Also, if the authors cannot produce a matched continued-SFT baseline in revision, the abstract's 'substantially better' wording should be softened to reflect the actual evidence. The paper cites relevant work (including its own prior line) appropriately."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I'll get to the point. The genuinely new piece is the RLHF adaptation recipe for LLM-based ASR on disordered speech, with a meaning-preservation reward alongside WER. That is not in the cited prior work, and the paper makes a real empirical case that adding the MP reward improves comprehensibility without a significant WER penalty, backed by a 220-example SLP evaluation. The audio-token trick (repurposing low-frequency text tokens instead of expanding the vocabulary) is a modest variant of SpeechGPT, but it works and lets them train a 2B Gemma as an ASR model with no architecture changes.\n\nThe main soft spot is the RL-vs-SFT comparison, exactly as your stress test says. The entire 'RL beats SFT' headline rests on one continued-SFT row that gets worse WER than the starting checkpoint (57.1 vs 50.4). That is not a convincing tuned baseline; at minimum it suggests poor early stopping or a bad recipe. The paper says 'even with significant hyper parameter tuning' but gives no search space, no trial count, no learning-rate range, no checkpoint criterion. If the baseline was undertuned, the headline is an artifact of uneven effort. The human evaluation does not rescue this, because it compares the two RLHF models, not RLHF against continued SFT.\n\nThe second issue is the reward/eval circularity. The MP metric is produced by a Gemma-2B reward model trained by the same group on the same style of data, and the same metric is the RL reward. Part of the MP gain could be optimizing the judge rather than meaning itself. The SLP evaluation on 220 examples supports the gamma=1.0 vs gamma=0.0 preference, so the circularity is not total, but it does mean the headline numbers are less clean than they look.\n\nMinor: no confidence intervals on the main WER/MP tables (there are t-tests for some comparisons), no code/data, and the LibriSpeech dev numbers drift a bit under RL, which is not discussed.\n\nAll that said, the core idea is sound and the application matters. The paper deserves a serious referee, but the referee should ask for a matched-budget continued-SFT baseline with full hyperparameter details, and ideally an MP metric that isn't the same model used for reward. I'd bring it to the reading group to discuss the RLHF-vs-finetuning comparison design, and I'd cite it for the RLHF recipe, though the comparison needs work.","headline":"Worth refereeing, but the headline RL-vs-SFT comparison rests on a single under-specified baseline that regresses WER, so the paper needs a matched-budget retraining check.","tokens_in":9319,"tokens_out":4513,"would_cite":true,"duration_ms":37486,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A small LLM-based speech recognizer adapts to disordered speech better through meaning-aware reinforcement learning than through continued supervised fine-tuning.","keywords":["automatic speech recognition","large language models","reinforcement learning","disordered speech","meaning preservation","domain adaptation","speech tokenization"],"falsifier":"Re-run continued supervised fine-tuning on the same Euphonia training split with a documented grid over learning rate, number of steps, input dropout, and checkpoint selection by validation WER and MP; if an adequately tuned SFT model reaches or beats the RLHF model's MP of 55.7 at a WER of 42.6 or lower on the Euphonia test set, the paper's claim that RL is superior to fine-tuning would be contradicted.","tokens_in":8259,"feed_emoji":"🎙️","tokens_out":8272,"duration_ms":68720,"temperature":0.7,"pith_summary":"This paper claims that a small, off-the-shelf large language model can be turned into a speech recognizer, and that adapting it to disordered speech works better with reinforcement learning that rewards meaning preservation than with continued supervised fine-tuning. The authors take Gemma 2B, swap its least-used text tokens for audio tokens produced by a frozen speech encoder, and supervise it on a mixture of standard and disordered speech. They then tune it with Proximal Policy Optimization using a reward that combines word error rate with a meaning-preservation score produced by a second, smaller LLM judge. On the Euphonia disordered-speech test set, the best RL-tuned model raises meaning preservation from 48.2 to 55.7 without a statistically significant word-error penalty, while continued supervised fine-tuning makes the starting model worse. A speech-language pathologist's ratings on 220 transcripts agree with the automated metric, which matters because standard word error rates understate comprehension failures in disordered speech.","feed_headline":"RL that rewards meaning beats fine-tuning for disordered speech","feed_subtitle":"A meaning-preservation reward lifts disordered-speech accuracy from 48.2 to 55.7 with no word-error penalty.","key_machinery":"The central mechanism is the reward function $R(x,y;y^*) = \\gamma \\cdot \\text{MP}(y,y^*) + \\log(1 - \\text{WER}(y,y^*))$, where MP is a meaning-preservation score from a Gemma-2B judge trained on 2,840 human-labeled transcript pairs, and WER is the word error rate against the ground-truth transcript. The logarithmic transform puts WER on a scale comparable to MP, and $\\gamma$ trades off the two signals. Around that reward, the other load-bearing pieces are the audio-token vocabulary swap (1,024 cluster ids from a frozen USM encoder replace the least frequent text tokens, so the decoder-only LLM consumes audio with no architecture change) and PPO with KL regularization for optimization.","core_discovery":"On the paper's own terms, the discovery is that the tuning strategy—not the ASR architecture—is what enables an LLM to adapt to disordered speech. The authors first make the LLM audio-capable by mapping 1,024 cluster ids of speech-encoder embeddings onto the last 1,024 text tokens of the Gemma 2B vocabulary and supervised-tuning on a 30:70 Euphonia-to-LibriSpeech mixture. From that starting point, continued supervised fine-tuning on disordered speech alone degrades the model (WER 57.1, MP 42.8 on Euphonia test), while reinforcement learning on the same data with the reward $R(x,y;y^*) = \\gamma \\cdot \\text{MP}(y,y^*) + \\log(1 - \\text{WER}(y,y^*))$ improves it across all tested values of $\\gamma$, with $\\gamma = 1.0$ giving the best trade-off (WER 42.6, MP 55.7). The authors report significance against the WER-only RL run ($p<0.0003$ for MP, $p=0.54$ for WER) and corroborate the automated MP metric with a human evaluation by a speech-language pathologist on 220 examples.","pith_inferences":["A testable extension the authors leave implicit: run the same RL-with-MP-reward recipe on a non-disordered low-resource domain to see whether the advantage over continued SFT is a general property of the tuning method or specific to disordered speech.","The comparison is against a continued-SFT baseline whose tuning effort is not documented; a controlled comparison with an equally well-tuned SFT baseline would clarify whether the gain comes from RL itself or from the meaning-preservation objective.","The MP judge is a single 2B model trained on 2,840 pairs; using a larger judge or training it on the target domain could change the reward landscape and the reported $\\gamma=1.0$ numbers, so the numerical gains are partly tied to this particular judge.","By replacing the least frequent text tokens, the method sacrifices rare-token text coverage in exchange for audio capability; multilingual or Unicode-heavy text generation could degrade even though the paper reports little loss on LibriSpeech dev-clean."],"forward_implications":["The same vocabulary-swap-plus-RL recipe could in principle adapt LLM-based ASR to other low-resource domains, such as accented speech or child speech, where word-level errors understate comprehension.","Because the RL step uses only target-domain audio and transcripts plus an LLM judge, it offers a path for adaptation without retraining the speech encoder or changing the LLM architecture.","The result implies that when the goal is meaning preservation rather than exact word match, a learned semantic reward is a usable objective for speech-recognition tuning.","The paper's 220-example human evaluation indicates the automated meaning-preservation score tracks a speech-language pathologist's judgment, supporting its use as a reward signal.","The authors do not claim state-of-the-art recognition; the claim is specifically that RL tuning outperforms continued supervised fine-tuning for this domain shift."],"supporting_citations":[{"why":"Supplies the RL-with-human-feedback recipe (PPO with KL regularization) that the paper adapts to ASR.","marker":"[27]"},{"why":"Defines the meaning-preservation judge approach and the 0.89 AUC target the paper's smaller judge is compared against.","marker":"[16]"},{"why":"Provides the binary meaning-preservation task and 2,840 labeled transcript pairs used to train the paper's Gemma-2B reward model.","marker":"[28]"},{"why":"Supplies the Proximal Policy Optimization algorithm used for the RLHF step.","marker":"[33]"},{"why":"Supplies the USM speech encoder whose embeddings are clustered into the 1,024 audio tokens.","marker":"[20]"},{"why":"Provides the Gemma 2B open LLM that serves as the base model and the fine-tuning recipe.","marker":"[21]"},{"why":"Supplies the LibriSpeech corpus used as the standard-speech training and validation data.","marker":"[24]"},{"why":"Supplies the Euphonia disordered-speech dataset used for training and testing.","marker":"[25]"},{"why":"Defines the speaker-independent (SI) splits of Euphonia used to prevent speaker overlap.","marker":"[26]"}],"fun_headline_variants":["RL with meaning reward beats supervised tuning on disordered speech","Meaning-aware RL reward trumps fine-tuning for disordered speech","Custom RL reward improves disordered speech more than fine-tuning","RL tuning beats fine-tuning for disordered speech adaptation","For disordered speech, RL tuning outdoes supervised fine-tuning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's central comparison assumes the continued supervised fine-tuning baseline was tuned at least as carefully as the RLHF runs; the paper does not document the search space or checkpoint selection for that baseline, saying only that 'even with significant hyperparameter tuning' it did not help.","fun_headline_variants_meta":{"raw":{"variants":["RL with meaning reward beats supervised tuning on disordered speech","Meaning-aware RL reward trumps fine-tuning for disordered speech","Custom RL reward improves disordered speech more than fine-tuning","RL tuning beats fine-tuning for disordered speech adaptation","For disordered speech, RL tuning outdoes supervised fine-tuning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000823,"raw_usage":{"total_tokens":3599,"prompt_tokens":943,"completion_tokens":2656,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":559,"completion_tokens_details":{"reasoning_tokens":2579}},"tokens_in":559,"tokens_out":2656,"duration_ms":21659,"temperature":1.0,"reasoning_tokens":2579,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:32:18.419317+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run continued supervised fine-tuning on the same Euphonia training split with a documented grid over learning rate, number of steps, input dropout, and checkpoint selection by validation WER and MP; if an adequately tuned SFT model reaches or beats the RLHF model's MP of 55.7 at a WER of 42.6 or lower on the Euphonia test set, the paper's claim that RL is superior to fine-tuning would be contradicted.","supporting_citations":[{"cited_title":"Large language models as a proxy for human evaluation in assessing the comprehensibility of disordered speech transcription,","cited_arxiv_id":null,"evidence_quote":"Defines the meaning-preservation judge approach and the 0.89 AUC target the paper's smaller judge is compared against."},{"cited_title":"Automatic speech recog- nition of conversational speech in individuals with disordered speech,","cited_arxiv_id":null,"evidence_quote":"Provides the binary meaning-preservation task and 2,840 labeled transcript pairs used to train the paper's Gemma-2B reward model."},{"cited_title":"Disordered speech data collection: Lessons learned at 1 million utterances from project euphonia","cited_arxiv_id":null,"evidence_quote":"Supplies the Euphonia disordered-speech dataset used for training and testing."},{"cited_title":"Au- tomatic speech recognition of disordered speech: Personalized models outperforming human listeners on short phrases","cited_arxiv_id":null,"evidence_quote":"Defines the speaker-independent (SI) splits of Euphonia used to prevent speaker overlap."}],"review_version":1}