{"id":"77c12c36-b551-468a-978e-0290c762b47c","arxiv_id":"2509.01939","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Applying GRPO with rule-based rewards to LLM-based ASR improves WER by up to 18.4% relative and reduces hallucination errors on unseen acoustic conditions.","lead":"Researchers at Amazon applied GRPO, a reinforcement learning method, to LLM-based speech recognition, using word error rate as the reward. Models improved up to 18.4% relative WER, with fewer hallucinations on out-of-domain audio and better domain adaptation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported GRPO gains may reflect additional training steps rather than the RL objective; a continued-SFT control is missing.","rationale":"The reader's weakest assumption focused on statistical significance, which is a valid secondary concern. However, I judge the most load-bearing issue to be the missing continued-SFT control. Even if all WER differences were statistically significant, they would not establish that GRPO—rather than additional training—causes the improvements. This is a standard experimental requirement in RL fine-tuning papers, and the paper does not report it. The continued-SFT control is concrete, simple to run, and would directly validate or refute the attribution of gains to GRPO. The verdict remains CONDITIONAL because the paper presents a plausible method and striking AMI results, but the core causal claim needs this control before full acceptance. Statistical significance testing (e.g., bootstrapped CIs) would be a useful complement but would not resolve the training-steps confound.","tokens_in":12057,"tokens_out":11752,"duration_ms":125318,"concrete_test":"Take the same SFT checkpoint used as the GRPO reference and continue standard supervised fine-tuning for 5000 steps on the same training data with the same learning rate (1e-6), global batch size (64), and tokenization/decoding settings, but using ordinary cross-entropy loss (no RL, no reward, no sampling-for-training). Evaluate on the same test sets, including AMI-IHM, AMI-SDM, TEDLIUM, MLS, and Voxpopuli. If this continued-SFT model achieves WER gains similar to the GRPO model (especially on AMI), the central claim that GRPO itself improves ASR is not supported; if it does not, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that GRPO, as an RL fine-tuning stage, improves WER, reduces hallucinations, and aids domain adaptation. However, the GRPO stage consists of up to 5000 additional training steps on the same speech data, with a small learning rate, and the paper provides no control where standard SFT (cross-entropy next-token prediction) is continued for the same number of steps on the same data. Without this control, the observed improvements—especially the dramatic AMI insertion-rate reductions—could be attributed to additional gradient updates/compute rather than the GRPO objective or reward structure. The paper's internal comparisons (GRPO variants, β=0, DAPO, Dr. GRPO) all retain the RL loss, so they do not isolate this confound. If continued SFT alone produced comparable WER gains, the claim that GRPO specifically benefits ASR would be substantially weakened. This concern is load-bearing because it affects the interpretation of every reported result, including the large out-of-domain gains that are otherwise unlikely to be noise.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes applying Group Relative Policy Optimization (GRPO) as an additional RL fine-tuning stage for decoder-only LLM-based automatic speech recognition. Starting from an SFT checkpoint, the policy is optimized by sampling G transcript hypotheses and using group-relative advantages with simple rule-based rewards: negative WER, exact match, and negative total edit distance, with a KL penalty to the reference. Experiments use 2B and 8B Llama3 models with a frozen conformer speech encoder, trained on roughly 88k hours of multilingual speech. The authors report WER improvements up to 18.4% relative, large reductions on out-of-domain AMI (e.g., 2B SFT AMI-SDM WER dropping from 87.88% to 37.59%), and favorable domain adaptation compared with continued SFT.","tokens_in":12307,"tokens_out":5785,"duration_ms":61494,"significance":"If substantiated, the result would be a useful first demonstration that RLHF-style policy optimization can improve LLM-based ASR, with practical benefits in insertion/hallucination reduction and out-of-domain robustness. The method is simple; the GRPO objective, advantage estimator, and reward definitions are standard and clearly presented. The rule-based rewards directly target the evaluation metrics, and the large AMI gains are unlikely to be pure noise. However, the manuscript does not yet rule out the alternative explanation that additional gradient updates, rather than the GRPO objective, account for the gains, and it lacks statistical validation for the smaller in-domain improvements. These issues are load-bearing for the central claims.","major_comments":[{"comment":"The GRPO stage adds up to 5000 additional training steps on the same data used for SFT (fixed learning rate 1e-6, global batch size 64). No control continues SFT for the same number of steps, so the reported gains—especially the dramatic insertion-rate reductions on AMI—may reflect additional optimization rather than the GRPO objective. All ablations (β=0, DAPO, Dr. GRPO) retain an RL-style loss, so they do not isolate this confound. Please add a matched continued-SFT baseline (same data, learning rate, number of steps, checkpoint selection) for at least the 2B model on the Table II and Table III evaluations.","section":"§III-A, Tables II–III"},{"comment":"All results appear to be single runs without confidence intervals or significance tests. Many differences are within a few tenths of a percent WER (e.g., 2B Voxpopuli En 7.92→7.82; MLS En 4.83→4.85 after GRPO(WER); 8B MLS overall 4.87→4.95). Such small differences can be evaluation noise. Report multiple decoding runs/seeds and significance tests, or at least WER confidence intervals, before claiming 'significant improvements' for the in-domain results. The large AMI gains are more robust but still need statistical quantification.","section":"§IV, Tables II–III"},{"comment":"The domain-adaptation comparison does not specify whether the SFT-adapted and GRPO-adapted models receive matched training budgets, learning rates, data, and checkpoint selection. If the SFT adaptation was run for fewer steps or with a different schedule, the comparison is not fair. Please state the exact setup for both adaptation conditions and, ideally, vary SFT steps to show the comparison is not an artifact of compute allocation.","section":"§IV-B, Table IV"},{"comment":"The hallucination-reduction claim is supported only by insertion-rate changes. This is a reasonable proxy, but the manuscript should either define a direct hallucination metric or explicitly frame the claim as 'reduced insertion errors' rather than 'reduced hallucinations.' Also, the 8B SFT AMI WER values exceed 100% (100.26 and 227.98); this should be explained in the text so readers do not misread it as an error, and the corresponding insertion/deletion/substitution percentages should be interpreted carefully.","section":"Table III"}],"minor_comments":[{"comment":"The notation in the min expression is ambiguous: clip is applied to the importance ratio πϑ, but the advantage A-hat is written outside the clip. Please write the standard form min(πϑ A-hat, clip(πϑ, 1−ε, 1+ε) A-hat) explicitly.","section":"Eq. (3)"},{"comment":"The KL expression as written is a pointwise Monte Carlo estimator of D_KL[πθ||πref], not the divergence itself. Please clarify that the expectation is over πθ, or write the divergence in closed form.","section":"Eq. (5)"},{"comment":"The claim that this is 'the first attempt at application of RLHF to LLMs to improve speech recognition' should be softened or qualified, given that Refs. [24]–[32] apply DPO/GRPO to speech understanding, audio QA, and TTS. If the claim is specifically 'first for ASR,' say so explicitly and distinguish from those prior works.","section":"Introduction, §1"},{"comment":"The statement that 'both DAPO and Dr. GRPO outperform traditional GRPO in most cases' is not clearly supported by Table II (e.g., MLS En and Voxpopuli Es for DAPO are worse than GRPO(WER); several rows are mixed). Please quantify the number of languages/configurations that improve rather than making a blanket statement.","section":"§IV, Role of RL Algorithms"}],"recommendation":"major_revision","confidential_remarks":"The missing continued-SFT control is the single most important technical issue and should be addressed before publication. The AMI out-of-domain results are the strongest evidence and are likely to survive the control, but the in-domain improvements are small enough that without statistical validation the headline claims are overstated. The paper is within scope and the underlying idea is sound; a careful revision with the added control and uncertainty quantification would make it acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a solid, practical contribution. It applies GRPO to LLM-based ASR with three simple rule-based rewards (WER, exact match, edit distance), and the headline outcome is real: on the AMI meeting corpus, which is out-of-domain, GRPO cuts the 2B model's WER from 87.9% to 37.6%, and the 8B model from over 200% to about 40%, driven by a collapse in insertion-rate hallucinations. Those numbers are too large to be noise, and the pattern holds across model sizes and reward variants. The domain-adaptation experiment is also a nice addition: GRPO adaptation on AMI beat direct SFT adaptation and caused less regression on other corpora.\n\nWhat's new is the specific application of GRPO to ASR, rather than PPO or DPO, and the use of reference transcripts as the reward source. The authors' 'first' claim is narrow but likely defensible for ASR proper, since prior GRPO work on speech targets audio QA or omni-modal reasoning.\n\nThe soft spots are mostly about experimental hygiene. The main tables report single runs without confidence intervals; several in-domain gains are a few tenths of a WER point, and a couple are degradations, so the word 'significant' overreaches there. The stress-test concern about a continued-SFT control is legitimate for the main experiments—there's no comparison where you continue plain SFT for the same number of steps at the same learning rate. That said, the AMI domain-adaptation section is a direct SFT-vs-GRPO comparison on the same data, and GRPO wins, which is the right kind of control and partially mitigates the concern. I'd still want the authors to add the missing control to the main setup before I'd take the attribution at face value.\n\nThe paper doesn't release code or models, so reproducibility is limited. And equation (5) looks like the KL term is inverted relative to the standard GRPO expression; probably a typo but worth flagging.\n\nWho is this for? Anyone building LLM-based ASR systems, especially people fighting hallucinations or doing domain adaptation. It's a simple, transferable recipe. The paper deserves a serious referee; the core result is interesting enough and the missing control can be fixed in revision.\n\nRecommendation: send it to peer review, but expect the authors to need to add a continued-SFT control and some measure of variance.","headline":"A genuinely useful result on using GRPO to reduce hallucinations in LLM-based ASR, with a load-bearing but partly addressable missing-control issue.","tokens_in":12793,"tokens_out":4872,"would_cite":true,"duration_ms":45379,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Reinforcement learning from text rewards cuts speech-recognition errors and hallucinations.","keywords":["speech recognition","GRPO","reinforcement learning from human feedback","LLM-based ASR","hallucination reduction","domain adaptation","word error rate reward","rule-based rewards"],"falsifier":"Run the same GRPO training recipe with multiple random seeds and compute confidence intervals on the held-out WER for Voxpopuli, MLS, and People's Speech. If the 2B model's small gains (e.g., MLS En 4.83 to 4.85 or the 0.1-point Voxpopuli changes) flip sign across seeds, the claim of consistent significant improvement would fail, while the large AMI effect would stand or fall on whether insertion reductions replicate across seeds.","tokens_in":11995,"feed_emoji":"🎙️","tokens_out":1928,"duration_ms":25339,"temperature":0.7,"pith_summary":"This paper proposes adding a reinforcement-learning stage, Group Relative Policy Optimization (GRPO), after supervised fine-tuning of LLM-based speech recognition systems. Instead of training a separate reward model, the system scores its own generated transcripts with simple rule-based rewards such as word error rate, exact match, and total edit distance. The authors report relative WER reductions up to 18.4% on in-domain benchmarks, and much larger reductions on out-of-domain meeting speech where the baseline model hallucinates heavily. They also show that GRPO adaptation to a new domain beats conventional supervised fine-tuning on that domain while degrading less on other languages.","feed_headline":"Reward-based training cuts speech errors up to 18%","feed_subtitle":"A critic-free RL stage makes LLM transcribers robust on unseen, noisy audio and eases domain adaptation.","key_machinery":"Group Relative Policy Optimization (GRPO), a variant of PPO that omits the critic network and computes advantages from the mean and standard deviation of rewards across a group of sampled outputs. The policy objective is clipped like PPO's but uses group-relative advantages, plus an optional KL-divergence penalty to keep the policy near the reference model. In this paper, rewards are rule-based functions of the transcript: negated WER, negated total edit distance, or exact match against the reference text. The mechanism's work is to push the model toward outputs with lower edit distance to human transcripts, which the authors show translates into fewer insertions and substitutions on difficu","core_discovery":"The paper claims that GRPO, a critic-free policy-gradient method originally developed for LLM reasoning, works as an effective post-training stage for LLM-based ASR. The key move is to define rewards directly from ground-truth transcripts, so no reward model is needed. Testing on 2B and 8B Llama-3-style models, the authors find that GRPO with a word-error-rate reward reduces insertions drastically on out-of-domain data, converting a model that produced 87.88% WER on AMI far-field speech into one at 37.59% WER while preserving or improving most in-domain results. They also find that GRPO with a small KL penalty performs comparably to unregularized variants, and that DAPO and Dr. GRPO modifica","pith_inferences":["A natural extension would be semantic or task-level rewards, such as slot error rate or intent accuracy, which the paper itself mentions as future work; evidence here suggests edit-distance rewards may not align perfectly with downstream semantic goals.","The pronounced effect on AMI suggests GRPO is acting partly as a calibration step that suppresses the model's prior over common text patterns; one could test this hypothesis by measuring the per-token probability of insertions before and after GRPO.","The relative gains appear largest where baseline WER is high, implying that the benefit of RLHF may scale with domain mismatch; a testable prediction is that GRPO will show diminishing returns on already-clean, well-matched test sets.","Single-run evaluations without confidence intervals leave open the possibility that small reported gains are noise, so an independent multi-seed replication on the smaller benchmarks would clarify which of the improvements are real."],"forward_implications":["If the central claim holds, GRPO with rule-based rewards offers a simple, reward-model-free way to reduce ASR hallucinations, especially in far-field and meeting audio where insertion errors dominate.","The method provides a domain-adaptation recipe that improves target-domain WER while causing less regression on other languages than standard supervised fine-tuning.","The result suggests that reward design, not just architecture or data scale, is a practical lever for controlling LLM-based ASR behavior, since exact-match rewards rival WER rewards on clean data while WER rewards are safer on noisy data.","The reported reduction of 8B-model WER on AMI from above 100% to roughly 40% implies that RLHF can rescue large models that overfit in-domain patterns and hallucinate on unseen acoustics.","Because the loss is computed from sampled transcripts, the approach is compatible with any LLM-based ASR interface, so the same recipe could be layered onto other speech-LLM systems."],"supporting_citations":[{"why":"Introduces GRPO, the algorithm this paper applies to ASR; supplies the group-relative advantage computation and policy objective.","marker":"[20]"},{"why":"Introduces PPO, the actor-critic baseline that GRPO simplifies; provides the clipping mechanism used in the loss.","marker":"[19]"},{"why":"Characterizes hallucinations in LLM-based ASR under noise and distribution shift, motivating the robustness objective.","marker":"[12]"},{"why":"Shows how annotation noise and model priors produce hallucinated ASR outputs, supporting the need for an RL stage.","marker":"[13]"},{"why":"DAPO modifications (no KL, upper clip, token-level loss) that the paper evaluates alongside plain GRPO.","marker":"[22]"},{"why":"Dr. GRPO adjustments (no standard-deviation normalization, length normalization) that the paper evaluates as alternatives.","marker":"[23]"},{"why":"AMI meeting corpus is the out-of-domain benchmark where the largest hallucination reductions are measured.","marker":"[46]"},{"why":"TEDLIUM serves as the second out-of-domain evaluation set for robustness checks.","marker":"[47]"}],"fun_headline_variants":["Critic-free RL trims ASR word errors by 18%","GRPO post-training slashes speech hallucination","Rule-based rewards boost LLM speech robustness","RL stage cuts ASR errors on unseen audio","Policy gradient tames LLM transcribers on noise"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The reported improvements are computed from single evaluation runs without confidence intervals or significance tests, so the smaller gains could be evaluation noise rather than genuine model improvement.","fun_headline_variants_meta":{"raw":{"variants":["Critic-free RL trims ASR word errors by 18%","GRPO post-training slashes speech hallucination","Rule-based rewards boost LLM speech robustness","RL stage cuts ASR errors on unseen audio","Policy gradient tames LLM transcribers on noise"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000333,"raw_usage":{"total_tokens":1656,"prompt_tokens":683,"completion_tokens":973,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":427,"completion_tokens_details":{"reasoning_tokens":910}},"tokens_in":427,"tokens_out":973,"duration_ms":7546,"temperature":1.0,"reasoning_tokens":910,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T12:02:01.617942+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same GRPO training recipe with multiple random seeds and compute confidence intervals on the held-out WER for Voxpopuli, MLS, and People's Speech. If the 2B model's small gains (e.g., MLS En 4.83 to 4.85 or the 0.1-point Voxpopuli changes) flip sign across seeds, the claim of consistent significant improvement would fail, while the large AMI effect would stand or fall on whether insertion reductions replicate across seeds.","supporting_citations":[{"cited_title":"The AMI meeting corpus: A pre-announcement,","cited_arxiv_id":null,"evidence_quote":"AMI meeting corpus is the out-of-domain benchmark where the largest hallucination reductions are measured."},{"cited_title":"TED-LIUM 3: Twice as much data and corpus repartition for experiments on speaker adaptation,","cited_arxiv_id":null,"evidence_quote":"TEDLIUM serves as the second out-of-domain evaluation set for robustness checks."}],"review_version":1}