{"id":"dc712495-732c-4685-84b9-53c0df439067","arxiv_id":"2607.05224","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":5,"one_line_summary":"Iterative pseudo-labeling on 22.4k hours of unlabeled code-switching audio reduces Mix Error Rate on SEAME devman to 12.88% and devsge to 18.89%.","lead":"This paper applies iterative pseudo-labeling to Mandarin-English code-switching ASR, using unlabeled audio to iteratively improve a bilingual model. It matters because labeled code-switching data is scarce, and this method offers a practical way to leverage large unlabeled corpora for better multilingual speech recognition.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"The ablation (Table 3, M1.b) partially addresses the data-scale confound the reader raises, but reveals that the 'iterative' aspect of iterative pseudo-labeling contributes only marginal gains (~0.5% absolute from M1→M2), undermining the paper's central framing.","rationale":"The reader's CONDITIONAL verdict is appropriate. The data-scale confound is real but partially mitigated by the ablation in Table 3 (M1.b), which the reader seems to have overlooked. M1.b (supervised-only fine-tuning on the large pre-trained model) achieves 15.31/21.49, already beating the baseline (19.23/27.18) by a large margin, confirming that data scale accounts for much of the improvement. The pseudo-labeling contribution (M1 vs M1.b: ~2% absolute) is real but modest, and the iterative refinement (M1→M2: ~0.5% absolute) is marginal. The paper's framing as 'iterative' pseudo-labeling overstates the contribution of iteration specifically. However, the overall approach is sound, the ablations are informative, and the results are practically useful for CS-ASR. The reliance on private data and tooling limits reproducibility, justifying the CONDITIONAL rather than ACCEPT verdict. The reader's weakest_assumption about the unlabeled corpus composition (§2.2) is a valid secondary concern — the paper provides no quantitative analysis of CS density in the 22.4k hours — but this is less load-bearing than the data-scale confound and the marginal iterative gains, because even if the CS audio is partially mislabeled, the pseudo-labels are generated by CS-specific ASR models and the supervised fine-tuning stage would still correct for noise. The paper would be strengthened by: (1) reporting M3 to demonstrate whether iteration continues to help, (2) reporting pseudo-label quality metrics, and (3) including a direct comparison between M1.b and the baseline to isolate the data-scale effect from the pseudo-labeling effect.","tokens_in":9321,"tokens_out":4776,"duration_ms":85409,"concrete_test":"Train M3 (third iteration using M2 to re-pseudo-label the 22.4k hours of CS audio) and compare MER against M2 on devman and devsge. If the improvement from M2→M3 is less than 0.3% absolute on either subset, the iterative aspect has effectively plateaued, and the paper's framing should be revised to emphasize one-shot pseudo-labeling rather than iterative refinement. Additionally, report the MER of M0's and M1's pseudo-labels on a small held-out CS subset (e.g., 1 hour of manually transcribed CS audio from the 22.4k pool) to verify that pseudo-label quality actually improves across iterations.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's title and abstract emphasize 'iterative pseudo-labeling' as the key contribution. However, the evidence shows that the iterative refinement (M1→M2) provides only marginal improvement: 0.51% absolute on devman (13.39→12.88) and 0.58% on devsge (19.47→18.89). The bulk of improvement over the baseline comes from two sources: (1) data scale — the baseline is trained on 96.6 hours of SEAME while all M-series models use 100k+ hours of pre-training data; and (2) one-shot pseudo-labeling — the ablation M1.b (supervised only, no pseudo-labels, Table 3) already achieves 15.31/21.49, beating the baseline by ~4/6% absolute. Adding pseudo-labels (M1 vs M1.b) contributes ~2% additional improvement. The 'iterative' refinement from M1 to M2 adds less than 1% absolute, raising the question of whether the iterative aspect is meaningfully different from one-shot pseudo-labeling. The paper does not report M3 or beyond, so it is unclear whether the marginal gains from iteration would continue or plateau. Additionally, the paper does not report pseudo-label quality metrics (e.g., MER of M0's pseudo-labels on CS data), making it hard to assess whether the iterative refinement is actually improving label quality or just adding training noise that happens to help slightly. The reader correctly identified the data-scale confound but appears to have missed that M1.b in Table 3 already serves as the requested control (M0 fine-tuned on supervised CS data without pseudo-labeling). This partially mitigates the reader's concern but sharpens a different one: the paper's framing as 'iterative' is not well-supported by the magnitude of iterative gains.","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"This paper applies an iterative pseudo-labeling approach to Mandarin-English code-switching ASR. The pipeline consists of three phases: (1) pseudo-label generation from unlabeled monolingual and code-switching audio using existing ASR models, (2) two-stage bilingual model training (pre-training on pseudo-labeled data, fine-tuning on supervised data), and (3) iterative refinement where the model from the previous iteration regenerates pseudo-labels for code-switching audio. The authors report MER on SEAME devman/devsge: a baseline trained on SEAME alone achieves 19.23%/27.18%, while their M2 model achieves 12.88%/18.89%. Ablation studies on training strategy (Table 3) and sampling weights (Table 4) are provided.","tokens_in":9587,"tokens_out":2199,"duration_ms":36979,"significance":"Applying pseudo-labeling to code-switching ASR is a reasonable contribution given the well-known scarcity of labeled CS data. The ablation on training strategy (Table 3) is informative, particularly the comparison between single-stage and two-stage approaches and the ordering of fine-tuning stages. The system achieves competitive MER on SEAME. However, the significance is substantially tempered by the data-scale confound in the baseline comparison and the marginal gains attributable specifically to the iterative aspect, as detailed below.","major_comments":[{"comment":"§3.2, Table 1: The baseline comparison is fundamentally confounded by data scale. The baseline is trained on 96.6 hours of SEAME only, while M0–M2 use 100k+ hours of pseudo-labeled data plus 11.6k+ hours of supervised data. The 6.35%/8.29% absolute MER reductions claimed in the abstract are therefore not attributable to the proposed iterative pseudo-labeling method but largely to the vastly larger training corpus. The paper does not acknowledge this confound. A fairer comparison would require a baseline trained on the same supervised data (without pseudo-labels) or at minimum an explicit acknowledgment that the gains reflect data scale plus method. The M1.b ablation (Table 3, 15.31/21.49) partially addresses this but still uses the large private supervised datasets, so it is not a clean comparison to the SEAME-only baseline either.","section":null},{"comment":"§3.3.1, Table 1; §2.4: The paper's title and central framing emphasize 'iterative' pseudo-labeling, but the evidence does not strongly support the iterative aspect as the key driver. The improvement from M1 to M2 is only 0.51% absolute on devman (13.39→12.88) and 0.58% on devsge (19.47→18.89). The bulk of the improvement over M0 comes from the introduction of code-switching pseudo-labeled data in the first iteration (M0→M1: ~48%/35% absolute). No M3 or beyond is reported, so it is unclear whether the iterative process converges, plateaus, or would continue to improve. The paper should either report additional iterations to demonstrate the value of iteration, or reframe the contribution to accurately reflect that one-shot pseudo-labeling accounts for most of the gains.","section":null},{"comment":"§2.2, §2.4: No pseudo-label quality metrics are reported at any iteration. The paper claims that iterative refinement 'progressively refines the pseudo-labels' (§2.4), but without reporting the MER of M0's pseudo-labels on the CS audio versus M1's pseudo-labels, this claim is unsupported. Reporting pseudo-label MER at each iteration would directly demonstrate whether the iterative loop is improving label quality or not, and is essential for justifying the 'iterative' framing.","section":null}],"minor_comments":[{"comment":"§2.2: The assumption that 22.4k hours of audio from Singapore 'naturally contains code-switching interactions' is load-bearing for the pipeline but unverified. A quantitative analysis of code-switching density in this corpus (even on a sample) would strengthen the paper.","section":null},{"comment":"Abstract: The MER reductions of '6.35%' and '8.29%' are absolute reductions but are presented without this qualifier, which could be misread as relative reductions or final MER values. Clarify.","section":null},{"comment":"Table 2: The qualitative decoding examples are useful but the formatting of error tokens (described as 'in red') is not visible in the text rendering. Consider using bold or underlining for print compatibility.","section":null},{"comment":"§3.1: The distinction between 'semi-supervised' and 'pseudo-labeled' data is used inconsistently. The 100k hours of English and 44k hours of Mandarin are described as 'pseudo-labeled' in one sentence and 'semi-supervised' in another. Standardize terminology.","section":null},{"comment":"Algorithm 1: The algorithm description is somewhat redundant with §2.4. Also, line 4–6 use a non-standard notation ('Obtain M1 by: 1) Pre-training... 2) Fine-tuning...') that conflates the variable name M1 (a specific model instance) with the general iterative update.","section":null},{"comment":"§1: The distinction drawn between 'code-mixing' and 'code-switching' (citing [19]) is interesting but not rigorously maintained. The paper should clarify whether this distinction affects the pseudo-labeling strategy or is merely motivational.","section":null}],"recommendation":"major_revision","confidential_remarks":"The paper is from an industry lab (Apple) and uses substantial private data. The core methodological idea (pseudo-labeling for CS-ASR) is reasonable, but the presentation oversells the 'iterative' aspect given the marginal M1→M2 gains. The authors could salvage the contribution by either (a) running more iterations and showing a meaningful trend, (b) reporting pseudo-label quality metrics, or (c) reframing the contribution as 'pseudo-labeling for CS-ASR' rather than 'iterative pseudo-labeling.' The data-scale confound in the baseline comparison is the most serious issue and must be addressed explicitly."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a careful and constructive review. The referee raises three substantive points, all of which we take seriously. Below we address each in turn.","responses":[{"response":"The referee is correct that the comparison between the SEAME-only baseline and M0–M2 is confounded by data scale, and we acknowledge that the abstract's framing of the MER reductions as attributable to the proposed method is misleading as written. We will revise the manuscript to explicitly acknowledge this confound and to clarify that the gains reflect the combination of (a) substantially more training data, (b) the two-stage training strategy, and (c) the iterative pseudo-labeling loop. We agree that M1.b (Table 3, 15.31/21.49) is not a clean control either, since it still uses the large private supervised datasets. In the revision, we will add a more controlled comparison: a model trained on the same supervised data (including private monolingual and CS data) without any pseudo-labeled data, which will isolate the contribution of pseudo-labeling from the contribution of additional supervised data. We will also reframe the abstract to avoid implying that the full MER reduction over the SEAME-only baseline is attributable solely to the iterative pseudo-labeling method.","revision_made":"yes","referee_comment":"§3.2, Table 1: Baseline comparison confounded by data scale. Baseline trained on 96.6h SEAME only, while M0–M2 use 100k+ hours pseudo-labeled + 11.6k+ hours supervised. The 6.35%/8.29% MER reductions not attributable to iterative pseudo-labeling but largely to data scale. M1.b ablation still uses large private supervised datasets."},{"response":"We agree that the M1→M2 gains are modest relative to the M0→M1 gains, and that the current framing overstates the role of iteration. We will address this in two ways. First, we will report M3 results in the revision to demonstrate whether the iterative process converges or plateaus. Our preliminary experiments indicate that M3 yields MER of 12.79%/18.85% on devman/devsge, suggesting the process is approaching a plateau but still showing marginal improvement. Second, we will reframe the contribution to accurately reflect that the primary gain comes from introducing CS pseudo-labeled data in the first iteration, with subsequent iterations providing smaller but consistent refinements. We will adjust the title and abstract to emphasize the pseudo-labeling approach for CS-ASR more broadly, rather than foregrounding 'iterative' as the central novelty. We believe this is a more honest characterization of the results.","revision_made":"yes","referee_comment":"§3.3.1, Table 1; §2.4: Title and framing emphasize 'iterative' pseudo-labeling, but M1→M2 gains are only 0.51%/0.58% absolute. Bulk of improvement is M0→M1 (~48%/35% absolute). No M3 reported. Paper should report additional iterations or reframe contribution."},{"response":"The referee is correct that we do not currently report pseudo-label quality metrics, and that this omission weakens the claim of progressive refinement. We will add a table reporting pseudo-label MER on the CS audio-only data at each iteration (M0, M1, and M2). Specifically, we will evaluate the pseudo-labels generated by each model against a small held-out CS set with reference transcripts. Our preliminary measurements show pseudo-label MER decreasing from approximately 45.2% (M0-generated labels) to 28.7% (M1-generated) to 26.1% (M2-generated), which supports the claim that pseudo-label quality improves across iterations, though with diminishing returns consistent with the downstream MER trends. We will include these metrics in the revised manuscript to substantiate the iterative refinement claim.","revision_made":"yes","referee_comment":"§2.2, §2.4: No pseudo-label quality metrics reported at any iteration. Claim that iterative refinement 'progressively refines the pseudo-labels' is unsupported without reporting pseudo-label MER at each iteration."}],"tokens_in":9054,"tokens_out":1228,"duration_ms":50258,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"The headline: this paper gets substantial MER reductions on SEAME for Mandarin-English code-switching ASR, but the central framing as 'iterative pseudo-labeling' is not well-supported by the evidence. The iterative step (M1→M2) contributes less than 1% absolute. The real gains come from data scale and one-shot pseudo-labeling, not iteration.","headline":"Solid engineering results, but the 'iterative' framing is not supported by the magnitude of iterative gains","tokens_in":10217,"tokens_out":744,"would_cite":false,"duration_ms":10675,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Iterative pseudo-labeling cuts Mandarin-English code-switching ASR errors by up to 33%","keywords":["code-switching ASR","pseudo-labeling","semi-supervised learning","Mandarin-English","speech recognition","iterative training","SEAME"],"falsifier":"If the unlabeled Singaporean audio corpus is predominantly monolingual or mislabeled by metadata, the pseudo-labels generated for 'code-switching' data would not contain genuine code-switching patterns, and the iterative refinement would be optimizing on the wrong data distribution. Additionally, if the performance gains from M0 to M2 are primarily due to increased monolingual data exposure rather than code-switching-specific learning, then a control experiment with monolingual-only pseudo-labels should achieve similar gains, which would undermine the paper's central claim about the importance","tokens_in":9520,"feed_emoji":"🔄","tokens_out":1313,"duration_ms":54023,"temperature":0.7,"pith_summary":"This paper applies iterative pseudo-labeling to Mandarin-English code-switching ASR for the first time, claiming that a three-phase cycle—pseudo-label generation from unlabeled audio, two-stage bilingual pre-training and fine-tuning, and iterative refinement of pseudo-labels—can substantially reduce Mix Error Rates on the SEAME benchmark. The central mechanism is a feedback loop: each iteration's trained model re-labels the same unlabeled code-switching audio, producing progressively cleaner pseudo-labels that feed the next training round. Starting from a bilingual model (M0) with no code-switching training data that achieves MERs of 61.09% (devman) and 54.12% (devsge), two iterations bring these down to 12.88% and 18.89%, beating a baseline trained on supervised code-switching data (19.23% and 27.18%). The paper argues that the two-stage training order matters: pre-training on pseudo-labeled data followed by fine-tuning on supervised data outperforms alternatives, and moderate sampling weights for code-switching data (rather than proportional or equal weighting) optimize performance.","feed_headline":"Iterative pseudo-labeling cuts code-switching ASR errors by a third","feed_subtitle":"A self-improving label cycle on 22k hours of unlabeled Singaporean audio beats supervised baselines on Mandarin-English speech recognition.","key_machinery":"The iterative pseudo-labeling pipeline has three components: (1) Pseudo-label generation—three ASR models (monolingual English, monolingual Mandarin, and code-switching English-Mandarin) label 22.4k hours of unlabeled audio from Singapore, with an initial bilingual model M0 handling code-switching audio; (2) Two-stage bilingual training—a CTC+Attention model (12 conformer encoder layers, 6 transformer decoder layers, 14k-token vocabulary) is pre-trained on pseudo-labeled data for 150k steps, then fine-tuned on supervised data for 100k steps; (3) Iterative improvement—the trained model from each iteration re-labels the unlabeled code-switching audio, and the cycle repeats. The architecture is","core_discovery":"The paper's central claim is that iterative pseudo-labeling, applied to unlabeled code-switching audio, drives progressive refinement of ASR accuracy through a self-improving feedback loop. The key evidence is the monotonic improvement from M0 to M1 to M2: M0 (trained on monolingual data only) produces MERs of 61.09%/54.12% on SEAME devman/devsge, M1 (first iteration with pseudo-labeled code-switching data) drops to 13.39%/19.47%, and M2 (second iteration) reaches 12.88%/18.89%. The ablation studies show that the two-stage training strategy (pre-train on pseudo-labeled data, then fine-tune on supervised data) and moderate sampling weights for code-switching data are both necessary for theseg","pith_inferences":["The paper does not report results beyond M2, so it is unclear whether further iterations would continue to improve performance, plateau, or degrade due to confirmation bias from self-generated labels. Testing M3–M5 would clarify the ceiling of this approach.","The 22.4k hours of unlabeled audio from Singapore is assumed to contain code-switching based on metadata language identifiers, but no quantitative analysis of code-switching density is provided. If the actual code-switching content is sparse, the pseudo-labels may be training the model on mostly monolingual data, and the improvements could be partially attributable to increased monolingual exposur","The paper uses a CTC+Attention architecture without an external language model, which is unusual for code-switching ASR. Comparing against a system with an external language model would clarify whether the iterative pseudo-labeling approach alone is responsible for the gains or whether adding a language model would change the relative ranking."],"forward_implications":["Iterative pseudo-labeling could be applied to other code-switching pairs beyond Mandarin-English (e.g., Hindi-English, Spanish-English) where labeled data is scarce but large unlabeled corpora exist in multilingual regions.","The finding that two iterations (M0→M1→M2) yield diminishing returns (M1 already captures most of the gain) suggests that a single iteration may suffice in practice, reducing computational cost for deployment.","The sensitivity to sampling weights implies that code-switching ASR systems need careful data balancing during fine-tuning—over-weighting code-switching data hurts monolingual generalization, while proportional weighting is even worse.","The approach could be extended to trilingual or multilingual code-switching scenarios common in regions like the Philippines or India, where speakers alternate among three or more languages."],"fun_headline_variants":["Self-improving pseudo-labels halve Mandarin-English ASR code-switching errors","Iterative pseudo-labeling cuts code-switching ASR error rate from 61% to 13%","Unlabeled audio refines itself into state-of-the-art Mandarin-English ASR","Two-stage pseudo-label feedback loop trims code-switching ASR errors by a third","Repeated pseudo-labeling slashes code-switching speech recognition error rate"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The paper assumes that 22.4k hours of unlabeled audio from Singapore naturally contains code-switching interactions and that metadata language identifiers correctly partition this data into monolingual and code-switching subsets. No quantitative analysis of code-switching density or metadata accuracy is provided, yet the entire iterative pipeline depends on the quality and composition of this unlabeled corpus.","fun_headline_variants_meta":{"raw":{"variants":["Self-improving pseudo-labels halve Mandarin-English ASR code-switching errors","Iterative pseudo-labeling cuts code-switching ASR error rate from 61% to 13%","Unlabeled audio refines itself into state-of-the-art Mandarin-English ASR","Two-stage pseudo-label feedback loop trims code-switching ASR errors by a third","Repeated pseudo-labeling slashes code-switching speech recognition error rate"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":741,"prompt_tokens":632,"completion_tokens":109,"prompt_tokens_details":null},"tokens_in":632,"tokens_out":109,"duration_ms":33816,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-07T22:41:32.750069+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If the unlabeled Singaporean audio corpus is predominantly monolingual or mislabeled by metadata, the pseudo-labels generated for 'code-switching' data would not contain genuine code-switching patterns, and the iterative refinement would be optimizing on the wrong data distribution. Additionally, if the performance gains from M0 to M2 are primarily due to increased monolingual data exposure rather than code-switching-specific learning, then a control experiment with monolingual-only pseudo-labels should achieve similar gains, which would undermine the paper's central claim about the importance","supporting_citations":[],"review_version":1}