{"id":"63f910e0-2a88-4b12-a267-598aa24d47a1","arxiv_id":"2501.07102","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A bias-list-conditioned tagger and decoder improves Vietnamese code-switching ASR text normalization, cutting word error rate by up to 56.2% and 36.8% on the authors' new synthetic test sets.","lead":"AdaCS is a text-normalization model that fixes code-switched words in Vietnamese ASR output, using a list of expected words to guide both detection and correction. It reports large error reductions over prior systems on new Vietnamese test sets, though those test sets are synthetic and the expected words are provided in advance.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline WER gains are measured on hand-constructed Vietnamese-pronunciation text, not on actual ASR hypotheses, so the reported reductions may not transfer to real code-switched ASR.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern I would raise: the evaluation is conducted entirely on hand-constructed text pairs that idealize ASR output. The paper is internally consistent—the reported WER numbers and relative reductions match the table, and the architecture description is coherent—but the central claim is about ASR normalization, not text normalization in isolation. A model that works well on clean phonetized text may fail on actual ASR hypotheses because the BAM's argmax retrieval (Eq. 2) is sensitive to token-level noise. The paper provides no real speech experiments, no error analysis on ASR output, and no ablation isolating the contribution of BAM in the tagger versus the decoder, so the mechanism attributed to the improvement is also under-supported. These are addressable concerns rather than fatal flaws; a conditional verdict with a request for real-ASR evaluation or realistic perturbation analysis is appropriate. Since the reader already returned CONDITIONAL and my concern matches theirs, I recommend no change to the verdict.","tokens_in":8453,"tokens_out":3226,"duration_ms":36574,"concrete_test":"Take a set of real Vietnamese code-switched utterances with reference transcripts, run them through a production Vietnamese ASR model (or Whisper/VietASR), and feed the ASR hypotheses through AdaCS and AdapITN using the same bias-list protocol as Section IV-B. Compute WER and CS-WER on the normalized outputs. If AdaCS no longer beats AdapITN by a comparable margin, or if the absolute WERs are much higher than Table II, the synthetic evaluation overstates the real-world benefit. Alternatively, if audio is unavailable, perturb the synthetic Input sentences with measured ASR error patterns (random diacritic stripping, phonetic substitutions, insertions/deletions) and rerun the comparison; the concern is settled if the relative WER reduction collapses under realistic noise.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that AdaCS outperforms prior state-of-the-art on Vietnamese CS ASR normalization, with relative WER reductions of 56.2% and 36.8%. However, the evaluation in Section IV-A does not use any acoustic-model output. The 'Input' sentences are constructed by taking written English sentences and replacing English CS phrases with manually labeled Vietnamese pronunciations (Table I). Training and test pairs are therefore synthetic text pairs that assume the ASR has already produced a clean, perfectly segmented, phonetically transcribed Vietnamese rendering of every English phrase. Real ASR hypotheses for code-switched Vietnamese will contain partial transcriptions, missing or incorrect diacritics, insertions, deletions, and word-boundary errors, none of which are modeled in the dataset. The mechanism claimed to drive the improvement—BAM selecting a bias entry by argmax over similarity scores (Eq. 2) and then attending to that entry (Eq. 3)—is exactly the component most sensitive to noisy input tokens: if the token representation does not resemble any bias entry because of ASR distortion, the retrieved bias entry will be wrong and the normalization will fail. The comparison with AdapITN is fair under the synthetic protocol because both models see the same inputs, but fairness of the comparison does not establish external validity. The paper also uses bias lists drawn from English words and phrases appearing in the same test sentences (Section IV-B), an oracle-like condition that further inflates the apparent adaptability. The abstract's claim is therefore conditional on an unverified assumption: that the synthetic 'spoken text' distribution matches the true output distribution of a Vietnamese ASR system.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AdaCS, a sequence-to-sequence normalization model that augments both the tagger and the decoder with a bias attention module (BAM). BAM selects a bias entry via inner-product similarity and attends to it, allowing the model to adapt to a provided bias list. The authors construct a synthetic Vietnamese code-switching dataset by manually replacing English phrases with Vietnamese pronunciations, and compare AdaCS against Transformer, GPT-4o, and AdapITN on general and medical test sets. On the synthetic protocol with bias lists, AdaCS achieves 2.8% and 4.8% WER on the two test sets, outperforming AdapITN's best results, leading to the abstract's relative reductions of 56.2% and 36.8%.","tokens_in":8742,"tokens_out":7585,"duration_ms":66812,"significance":"If the reported results are externally valid, the idea of applying the same bias-attention mechanism to both phrase tagging and decoding is a clean and useful contribution for low-resource code-switching normalization. The paper also contributes a 50k-pair dataset and makes code and checkpoints available, and the controlled comparison to AdapITN is fair under the same protocol. However, external validity is not established because the evaluation uses hand-constructed text pairs rather than ASR hypotheses, and the inference bias lists are oracle-like (drawn from the test set's own English words). The relative gains are therefore best interpreted as a controlled comparison under a favorable protocol, with the generalization claims not yet supported.","major_comments":[{"comment":"The evaluation protocol uses synthetic text pairs rather than actual ASR hypotheses. The Input sentences are created by manually replacing English CS phrases with Vietnamese pronunciations (Table I), so they are perfectly segmented and diacritic-free; real ASR output for Vietnamese code-switched speech will contain partial transcriptions, missing or incorrect diacritics, insertions, deletions, and boundary errors. Since BAM's retrieval in Eq. (2) is based on token-level similarity to bias entries, distorted tokens are likely to yield incorrect bias_index values. Please test on real ASR hypotheses or inject realistic ASR noise, and report how the WER reductions change.","section":"Section IV-A"},{"comment":"The bias lists are oracle-like: Experiment 2 uses 'a random bias list ... drawn from the list of English words in the entire corresponding test set, combining English words from current sentences', and the phrase-bias experiment likewise includes the target phrases. This means the model is told the set of possible outputs for each test sentence. The claim of 'the ability to handle unseen CS phrases' (Section IV-D) is therefore only demonstrated for phrases explicitly provided in the bias list. To support the generalization claim, add experiments where the bias list excludes the target phrases, contains unrelated distractors, or includes near-miss spellings, and report the effect on WER.","section":"Section IV-B"},{"comment":"Training bias lists are generated as 'the English words present in the sentences within the batch, as well as random English words', i.e., word-level entries only, but the test experiments include phrase-level bias lists (Table I). The ranking losses Lenc_rank and Ldec_rank use labels that are indices of bias entries; the paper does not describe how phrase entries are encoded or labeled during training. Without phrase-level training, it is unexplained why phrase-level bias helps on test-general (2.8% WER) but hurts on test-medical (7.8% vs 4.8% for word bias). Please clarify the phrase representation in BAM and whether phrase-level training examples are used, and provide an ablation for word vs phrase biases.","section":"Section III-D / IV-C"},{"comment":"The headline reductions of 56.2% and 36.8% compare AdaCS's best bias type per test set (phrases on test-general, words on test-medical) with AdapITN's best bias type (words on both sets). Under a fixed bias type, the relative improvements are 48.4% (test-general, words: 3.3 vs 6.4), 61.6% (test-general, phrases: 2.8 vs 7.3), 36.8% (test-medical, words: 4.8 vs 7.6), and 12.4% (test-medical, phrases: 7.8 vs 8.9). The abstract should report per-condition comparisons or use a single consistent bias type, since the current phrasing is misleading.","section":"Abstract / Table II"},{"comment":"No error bars, multiple-seed runs, or ablations are reported. Table II and Figure 3 present single-run results on test sets of only 2,000 sentences each, so the significance of the WER differences is unknown. An ablation removing BAM from either the tagger or decoder would isolate the contribution of each module, and confidence intervals would support the robustness claims. Please provide these.","section":"Section IV"}],"minor_comments":[{"comment":"The notation 'score = sP^T' is ambiguous because P is described as a matrix of shape (L+1)*dmodel; clarify the intended row/column orientation and include the dummy entry dimension.","section":"Section III-A, Eq. (1)"},{"comment":"The sentence 'relative WER reductions of 46.9% to 80.9%' is inaccurate; the maximum relative reduction from the Transformer baseline in Table II is 90.2% (AdaCS, phrases, test-general). Please recalculate or specify the comparison set.","section":"Section IV-D"},{"comment":"The metrics N-WER and CS-WER are not formally defined; state how tagged regions are mapped to word error computations.","section":"Table II"},{"comment":"The base pretrained model 'EnViBERT' is cited as [27], [28], but neither reference describes this model; provide a proper citation or description.","section":"Section IV-C"},{"comment":"The reference for XLS-R is incorrect; [14] is XLA VS-R, not XLS-R. Please cite the actual XLS-R work.","section":"Section II"},{"comment":"Figure 1 has small labels and multiple overlapping annotation styles; please enlarge or split the figure for legibility.","section":"Figure 1"}],"recommendation":"major_revision","confidential_remarks":"The synthetic evaluation and oracle bias lists are the main risks; if the authors can demonstrate robustness to realistic ASR noise and non-oracle bias lists, the contribution would be solid. I would also check the reproducibility of the dataset split and training details, as the paper does not report optimizer, learning rate, or number of training steps."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick read of the AdaCS paper. The short version: this is a credible incremental extension of AdapITN, with one genuinely useful new resource and one significant evaluation gap. The new configuration—bias attention module in both the tagger and the decoder—is a reasonable idea, and the paper's own tables show it beating AdapITN consistently on its self-constructed Vietnamese test sets. The scaling experiment against bias-list size is also a nice practical touch. If the GitHub repo ships the dataset and checkpoints, it could serve as a reference benchmark for Vietnamese code-switching normalization.\n\nThe soft spot is the evaluation. The 'Input' sentences are not actual ASR hypotheses; they are hand-written with manually labeled Vietnamese pronunciations. That means the test measures normalization on clean, correctly segmented spoken text, not on the messy output of an acoustic model. The bias lists are also oracle-like—English words and phrases from the test set itself. The combination is fine for a controlled comparison between AdaCS and AdapITN on identical inputs, but it does not support the abstract's implication that these WER reductions transfer to production. There are also no error bars or ablations, and the headline numbers compare each model's best bias setting, which is a favorable way to slice it. One more minor point: on test-medical, phrase bias actually makes AdaCS worse than word bias (7.8 vs 4.8 CS-WER); the authors acknowledge this, but it tempers the 'adaptive' story.\n\nThe central argument—that adding BAM to both stages helps with code-switched normalization—holds up under this synthetic protocol. The question is external validity, not internal consistency. I'd want a revised version to either feed real ASR output through the system or simulate realistic ASR errors (diacritic loss, insertions, partial words) before believing the headline reductions.\n\nAll that said, this deserves a serious referee. The architecture is a legitimate extension, the benchmark is reusable, and the flaws are addressable. Send it to review, but flag the synthetic-data concern.","headline":"Credible incremental extension of AdapITN with a reusable Vietnamese CS benchmark, but the headline WER gains are measured on synthetic text pairs and will need real-ASR validation before they mean much in production.","tokens_in":9314,"tokens_out":2494,"would_cite":false,"duration_ms":23529,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adaptive bias attention cuts code-switching speech errors by 56 percent","keywords":["code-switching speech recognition","adaptive normalization","contextual biasing","bias attention module","Vietnamese ASR","inverse text normalization","low-resource language","encoder-decoder normalization"],"falsifier":"Run a production Vietnamese ASR system on real code-switched speech, feed the raw transcripts to AdaCS and AdapITN with the same bias lists, and compare word error rates; if AdaCS's advantage over AdapITN shrinks or reverses on that real output, the reported 56.2% and 36.8% reductions are artifacts of the synthetic test format.","tokens_in":8255,"feed_emoji":"🎙️","tokens_out":9082,"duration_ms":77319,"temperature":0.7,"pith_summary":"AdaCS is a post-ASR normalization model for Vietnamese code-switched text, where English names and technical terms appear inside Vietnamese utterances. The paper's central claim is that inserting the same bias-attention module into both the phrase-tagging encoder and the normalizing decoder lets a model use a supplied list of expected words or phrases to locate and rewrite code-switched spans, and that this is enough to beat the previous state-of-the-art normalization system. On the paper's two new test sets, AdaCS reports relative word-error-rate reductions of 56.2% on the general set and 36.8% on the medical set over AdapITN's best results when a bias list is available. This matters because it offers a cheap, plug-in route to fixing code-switching errors in low-resource languages without retraining the acoustic model or relying on a large language model.","feed_headline":"Adaptive bias attention cuts code-switching speech errors by 56 percent","feed_subtitle":"A bias-attention module in both tagger and decoder lets a small model beat GPT-4o at fixing Vietnamese-English mixes.","key_machinery":"The load-bearing object is the Bias Attention Module (BAM), which injects the bias list into both stages. For a token's hidden representation, BAM scores it against pooled vectors of every bias-list entry by inner product, selects the highest-scoring entry by argmax, runs multi-head attention with the token as query and the chosen entry's token embeddings as keys and values, and adds the attention output back to the token. Applied after the encoder, this steers the tagger's start/inside/outside classification; applied in the decoder, it steers generation of the normalized phrase. Training combines four cross-entropy losses—tagger, encoder bias-ranking, decoder bias-ranking, and next-token generation—so the model is explicitly trained to choose the right bias entry at both stages.","core_discovery":"The paper claims that adaptive normalization, rather than model scale, is what resolves intra-sentential code-switching. AdaCS inserts the Bias Attention Module (BAM) twice: after the encoder, so the tagger marks tokens that belong to a code-switched phrase using the bias list, and inside the decoder, so each generated normalized token is conditioned on the matching bias entry. The result is dynamic tagging: given the same input sentence, AdaCS changes its start/inside/outside tags when the bias list switches from words to phrases, while AdapITN keeps the same tags; the paper presents this as the reason AdaCS normalizes more accurately. With no bias list, GPT-4o has the lowest word error rate (15.4% general, 15.0% medical), but with biases AdaCS reaches 2.8% WER on test-general using phrase bias and 4.8% on test-medical using word bias, against AdapITN's best 6.4% and 7.6%. The paper also reports that phrase bias helps on the general set but slightly hurts on the medical set, where distinct phrase entries share overlapping words.","pith_inferences":["Implicit but untested: if real ASR hypotheses contain partial words, missing diacritics, insertions, or deletions rather than clean Vietnamese pronunciations, the reported WER gains may not transfer; a test on raw acoustic-model output would settle this.","The same BAM recipe should transfer to other low-resource code-switching pairs, such as Mandarin-English or Hindi-English, as long as a pretrained bilingual text encoder exists, since nothing in the mechanism depends on Vietnamese.","The medical-domain result, where phrase bias hurts relative to word bias because distinct phrases share overlapping tokens, pinpoints argmax rank-and-select as the fragile step; deduplicating or shortening overlapping list entries could be a cheap fix worth testing.","The equal-weight four-term loss invites an ablation: removing each ranking loss separately would reveal whether encoder biasing, decoder biasing, or their combination carries the improvement over AdapITN."],"forward_implications":["With a bias list, AdaCS beats both the previous state-of-the-art normalization system and GPT-4o on the two proposed test sets, showing that contextual biasing can substitute for model scale in this task.","Dynamic tagging is the mechanism: AdaCS changes its start/inside/outside tags when the bias list changes from words to phrases, so performance remains high when the list format changes.","Scaling experiments up to 2,000 bias entries show that CS-word error remains far below the no-bias baseline, and AdaCS stays below AdapITN at every list size on both test sets.","Because AdaCS and AdapITN share the same decoder design and training base, the reported gains isolate the contribution of placing the bias module in both the tagger and the decoder."],"supporting_citations":[{"why":"AdapITN is the prior state-of-the-art baseline AdaCS is compared against, and its decoder structure is reused; the headline WER reductions are measured relative to its best results.","marker":"[9]"},{"why":"Contextual spelling correction whose decoding latency grows with bias-list size; AdaCS positions itself as the adaptive, faster alternative.","marker":"[11]"},{"why":"Instant one-shot word-learning model for context-specific normalization; a prior adaptive-biasing approach that degrades with long bias lists.","marker":"[8]"},{"why":"Adaptive tagger used only for replacement in code-switching ASR; the contrast motivating BAM's joint use in tagger and decoder.","marker":"[24]"},{"why":"Transformer encoder-decoder architecture is the shared backbone and the no-bias baseline in the experiments.","marker":"[25]"},{"why":"GPT-4o is the large-model baseline that wins without a bias list, establishing the bar that AdaCS clears once biases are supplied.","marker":"[26]"},{"why":"BERT pretraining underlies the EnViBERT encoder used to initialize all trained models.","marker":"[27]"},{"why":"Vietnamese-specific pretrained model (EnViBERT) used as the base for AdaCS, AdapITN, and the Transformer baseline.","marker":"[28]"}],"fun_headline_variants":["Adaptive bias attention cuts code-switch WER by 56%","AdaCS adaptive normalization halves code-switch ASR errors","Bias-aware adaptive normalization beats GPT-4o on code-switching","Adaptive bias module achieves 56% WER drop in code-switching","Dynamic bias list makes code-switch ASR 56% more accurate"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes the paper's hand-constructed 'spoken' sentences—English phrases replaced with manually labeled Vietnamese pronunciations—faithfully represent what a real ASR system would output, because no actual acoustic-model transcript is ever passed through AdaCS.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive bias attention cuts code-switch WER by 56%","AdaCS adaptive normalization halves code-switch ASR errors","Bias-aware adaptive normalization beats GPT-4o on code-switching","Adaptive bias module achieves 56% WER drop in code-switching","Dynamic bias list makes code-switch ASR 56% more accurate"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000618,"raw_usage":{"total_tokens":2901,"prompt_tokens":1010,"completion_tokens":1891,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":626,"completion_tokens_details":{"reasoning_tokens":1797}},"tokens_in":626,"tokens_out":1891,"duration_ms":13194,"temperature":1.0,"reasoning_tokens":1797,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:49:34.001890+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a production Vietnamese ASR system on real code-switched speech, feed the raw transcripts to AdaCS and AdapITN with the same bias lists, and compare word error rates; if AdaCS's advantage over AdapITN shrinks or reverses on that real output, the reported 56.2% and 36.8% reductions are artifacts of the synthetic test format.","supporting_citations":[{"cited_title":"Instant One-Shot Word-Learning for Context-Specific Neural Sequence-to-Sequence Speech Recognition,","cited_arxiv_id":null,"evidence_quote":"Instant one-shot word-learning model for context-specific normalization; a prior adaptive-biasing approach that degrades with long bias lists."},{"cited_title":"Towards Code-switching ASR for End-to-end CTC Models,","cited_arxiv_id":null,"evidence_quote":"Adaptive tagger used only for replacement in code-switching ASR; the contrast motivating BAM's joint use in tagger and decoder."},{"cited_title":"Attention is All you Need,","cited_arxiv_id":null,"evidence_quote":"Transformer encoder-decoder architecture is the shared backbone and the no-bias baseline in the experiments."},{"cited_title":"Improving Vietnamese Named Entity Recognition from Speech Using Word Capitalization and Punctuation Recovery Models,","cited_arxiv_id":null,"evidence_quote":"Vietnamese-specific pretrained model (EnViBERT) used as the base for AdaCS, AdapITN, and the Transformer baseline."}],"review_version":1}