{"id":"bcccfb93-77c3-4acc-be41-dd5a38bff619","arxiv_id":"2607.08117","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":5,"one_line_summary":"COALA ranks biasing entities via an SLM discriminative scorer with multi-positive and point-wise losses, enabling large-list contextual ASR without context-window collapse.","lead":"COALA scores how well candidate rare words match a spoken utterance so speech-language models can pick a short biasing list instead of stuffing thousands of names into a limited context window. The new multi-positive and point-wise losses stop training from collapsing when several rare words appear in one sentence, cutting biased word error on LibriSpeech.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"No significant objection identified beyond the reader's already-noted top-K extrapolation limit.","rationale":"The strongest claim is an empirical performance statement on LibriSpeech under a concrete protocol (DPD-Loss scorer, top-K=10, N up to 5000). That protocol is fully specified, the numbers are given, and the multi-positive training collapse is addressed by the new losses. The only material limitation—the theoretical inability of top-K=10 to recover every target when an utterance contains >11 rare words—is already disclosed by the authors and does not falsify the reported gains on the bulk of the data. Because the concern is already correctly identified and does not invalidate the core result, no verdict adjustment is warranted.","tokens_in":9564,"tokens_out":477,"duration_ms":5078,"concrete_test":"Re-run the N=5000 test-other evaluation while (a) raising top-K from 10 to 20 and (b) reporting B-WER stratified by number of rare words per utterance (bins 1, 2–5, 6–10, 11+). If B-WER for the 11+ bin remains lower than CTC-Filter/K-Prompt and overall B-WER does not rise >15 % relative, the extrapolation concern is non-load-bearing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The reader's weakest assumption (scorer trained at M=120 + fixed top-K=10 + <unbiased> threshold continues to surface true targets at N=5000 and for utterances with >10 rare words) is the genuine soft spot, but it is already stated by the paper itself (§3.4, Figure 3 discussion) and does not undermine the central claim as written. Tables 1–2 and Figures 3–4 supply direct empirical support for the reported Recall#20 and B-WER numbers under the exact protocol used; the multi-entity losses (MPD/DPD) demonstrably stabilize training where the prior Bias-Loss required an auxiliary log term. No internal inconsistency, hidden assumption, or unacknowledged failure mode appears that would reverse the ranking or B-WER gains. The weaker unbiased baseline and missing error bars are secondary caveats already flagged by the reader.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes COALA, a two-stage framework that equips speech-augmented language models (SLMs) with a discriminative projector mapping SLM latents to entity-matching scores (Eq. 1–2). This enables biasing-target identification (BTI) that selects a short top-K subset from large candidate lists, circumventing SLM context-window limits. To stabilize training on multi-target utterances, the authors replace the global softmax discriminative loss of prior work with Multi-Positive Discriminative Loss (MPD-Loss, Eq. 4) and Decoupled Point-wise Discriminative Loss (DPD-Loss, Eqs. 5–7). On LibriSpeech rare-word biasing lists of size N = 500/1000/5000, DPD-Loss yields Recall#20 of 99.09 % / 96.59 % (clean/other) and reduces B-WER to 3.25/9.13 (N=500), 3.86/10.59 (N=1000) and 6.96/15.17 (N=5000) after BTI, outperforming CTC-Filter, K-Prompt and the unfiltered full-list baseline (which OOMs at N=5000).","tokens_in":9874,"tokens_out":994,"duration_ms":22383,"significance":"If the reported gains hold under broader conditions, COALA supplies a practical, lightweight (777 M) recipe for injecting large entity catalogs into SLMs without OOM or severe distractor interference. The explicit diagnosis of inter-positive gradient conflict and the two new losses that remove the need for an auxiliary log term are useful contributions; Figure 4 and Table 1 give direct empirical support. Public code and a clear two-stage training protocol further raise the work’s utility for the community. The main caveats are a comparatively weak unbiased baseline and the fixed top-K protocol, yet the relative improvements over strong filter baselines remain informative for contextual ASR.","major_comments":[{"comment":"Table 2 shows that COALA’s unbiased B-WER (23.39 / 39.49) is substantially worse than the unbiased numbers of the cited baselines (Bias Qwen 8.40/18.40, RNN-T+IB 12.96/28.09, etc.). While relative gains after BTI are large, absolute B-WER comparisons across systems are therefore difficult to interpret; the paper should either match base-model strength or explicitly qualify the absolute numbers.","section":"Table 2"},{"comment":"§3.4 trains the scorer with only M=120 candidates per utterance and then applies a fixed top-K=10 (+ <unbiased> threshold) at inference for lists up to N=5000. Figure 3 and the accompanying text acknowledge that utterances with >11 rare words cannot be fully recovered. An ablation on training-list size M and an adaptive-K or recall-oriented selection policy would strengthen the claim of robustness for the multi-entity regime shown in Figure 1.","section":"§3.4, Figure 3"}],"minor_comments":[{"comment":"Title and abstract advertise a “Contrastive Regularizer,” yet the body never uses that term; MPD-Loss and DPD-Loss are presented simply as refined discriminative objectives. Align terminology.","section":"Title / Abstract"},{"comment":"Figure 2 caption and panel (b) contain typos (“scroing”, “Entity sample and scroing”).","section":"Figure 2"},{"comment":"Acronym expansion of COALA differs slightly between title (“COntextualizedASRLeveraging biAsing scoring”) and abstract; standardize.","section":"Title / Abstract"},{"comment":"No error bars or statistical significance tests accompany the WER/recall figures; even a simple bootstrap over utterances would help.","section":"Tables 1–2"},{"comment":"The special token is written both as <unbiased> and “unbiased”; keep notation consistent.","section":"§2.1"}],"recommendation":"minor_revision","confidential_remarks":"The technical delta relative to Huang & Wang (ASRU 2025) is real but incremental; the main novelty is the multi-positive / point-wise reformulation plus the SLM-specific BTI pipeline. Fit for a solid conference or journal short paper is good once the baseline-strength discussion is clarified."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful part of this paper is straightforward. Prior discriminative scorers (especially Huang & Wang’s Bias-Loss) fall over when an utterance has several rare words at once because the softmax pits positives against each other. COALA replaces that with two simple alternatives—MPD (local softmax per positive vs. all negatives) and DPD (independent sigmoid point-wise terms)—and shows they train stably as stand-alone objectives. On LibriSpeech with N=5000 they get Recall#20 of 99.09/96.59 and cut B-WER to 3.25/9.13 (N=500), 3.86/10.59 (N=1000), 6.96/15.17 (N=5000) once they keep only the top-10 scored entities. Unfiltered lists either barely help or OOM. That is a real, usable advance for anyone who needs personalization lists larger than the SLM context window.\n\nWhat they do well: the diagnosis of gradient conflict is clear, Figure 4 shows the score distributions actually separate under DPD, and they ship code. The two-stage recipe (ASR first, then freeze and train only the projector + LoRA scorer) is practical. Baselines CTC-Filter and K-Prompt are beaten cleanly on the ranking metrics that matter.\n\nSoft spots are real but secondary. Their unbiased base model is weaker than the systems they compare against, so absolute B-WER numbers look better than a pure head-to-head would. They train the scorer with only M=120 candidates and hard-code top-K=10; the paper itself notes the theoretical failure mode once an utterance has >11 rare words. No error bars. None of that erases the ranking and multi-entity gains under the protocol they actually ran.\n\nThis is for people building contextual ASR with SLMs who hit context-window or multi-target training walls. It is not a new theory of speech or language. I would send it to peer review; the losses and the BTI numbers are concrete enough to be worth a referee’s time. Worth a look if you work in this niche; cite it if you need a stable multi-positive scorer.","headline":"Solid engineering fix for multi-entity contextual biasing in SLMs: two clean losses that stop training collapse and make large lists usable without OOM.","tokens_in":10469,"tokens_out":548,"would_cite":true,"duration_ms":5621,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"COALA scores audio against candidate rare words so speech language models can bias ASR without overflowing their context windows.","keywords":["contextual biasing","speech-augmented language models","ASR","discriminative scoring","multi-positive loss","LibriSpeech","entity filtering"],"falsifier":"Measure Recall#20 and B-WER on LibriSpeech (or a matched multi-entity set) when the inference list size exceeds 5 000 and when every utterance is forced to contain 12–24 rare words; if either metric collapses relative to the reported top-10 results, the filtering claim fails.","tokens_in":10477,"feed_emoji":"🎙️","tokens_out":581,"duration_ms":5946,"temperature":0.7,"pith_summary":"Speech-augmented language models are good at general transcription but still miss rare, domain-specific words. Feeding every candidate name into the prompt fails once the list grows large: the context window fills, memory blows up, and distractors hurt accuracy. COALA trains a small scorer on top of the frozen model that maps speech and entity tokens into a shared score space, then keeps only the top matches for the final recognition prompt. The paper also replaces the usual multi-class ranking loss with two new objectives that stop positive rare words from fighting each other during training, so multi-entity utterances no longer collapse. On LibriSpeech the filtered approach recovers nearly all targets even from 5 000-item lists and sharply cuts biased word error while the unfiltered baseline runs out of memory.","feed_headline":"Scorer trims rare-word lists so speech LMs stay accurate","feed_subtitle":"Top-K acoustic matches from 5 000 candidates cut biased word error without OOM on LibriSpeech","key_machinery":"Biasing-score estimation (length-normalized MLP scores from SLM last-token states) optimized by Multi-Positive Discriminative Loss or Decoupled Point-wise Discriminative Loss, followed by top-K selection gated by the <unbiased> token.","core_discovery":"A lightweight discriminative projector on SLM hidden states can rank rare-word candidates by acoustic match, and the new multi-positive and point-wise losses let that scorer train stably when several rare words appear in one utterance, turning large biasing lists into short, accurate prompts that improve ASR.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["COALA ranks acoustic matches to trim rare-word lists for SLMs","Multi-positive losses stabilize multi-entity rare-word scoring","SLM latent projector filters large bias lists for accurate ASR","Contrastive scorer maps audio to entities without training collapse","Pointwise losses enable short prompts from 5000 rare-word candidates"],"cache_read_input_tokens":128,"weakest_assumption_plain":"A scorer trained on only 120 candidates per utterance and a hard top-10 cutoff will still surface the true rare words when the real list grows to thousands of items or an utterance contains more than ten rare words.","fun_headline_variants_meta":{"raw":{"variants":["COALA ranks acoustic matches to trim rare-word lists for SLMs","Multi-positive losses stabilize multi-entity rare-word scoring","SLM latent projector filters large bias lists for accurate ASR","Contrastive scorer maps audio to entities without training collapse","Pointwise losses enable short prompts from 5000 rare-word candidates"]},"model":"grok-4.5","effort":"low","cost_usd":0.006708,"raw_usage":{"total_tokens":1642,"prompt_tokens":690,"num_sources_used":0,"completion_tokens":90,"cost_in_usd_ticks":67080000,"prompt_tokens_details":{"text_tokens":690,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":862,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":690,"tokens_out":90,"duration_ms":8107,"temperature":1.0,"reasoning_tokens":862,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-10T12:47:22.852594+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Measure Recall#20 and B-WER on LibriSpeech (or a matched multi-entity set) when the inference list size exceeds 5 000 and when every utterance is forced to contain 12–24 rare words; if either metric collapses relative to the reported top-10 results, the filtering claim fails.","supporting_citations":[],"review_version":1}