{"id":"af0be591-7a93-418b-9d0f-e9a2d2aaeff5","arxiv_id":"2603.06310","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Adapting speech foundation models to linguistically distant Pacific languages induces severe representational drift and a plasticity-stability dilemma in which LoRA forgets earlier languages during sequential learning.","lead":"Adapting Whisper-style speech models to Pacific Indigenous languages forces large internal representation changes and erases prior knowledge. This exposes a hard plasticity-stability trade-off that current fine-tuning methods cannot resolve for underrepresented languages.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.5","headline":"Drift metric may not causally link to forgetting; sequential results reverse the usual LoRA/FullFT stability pattern without controls.","rationale":"The reader’s weakest_assumption correctly identifies that the cosine + min-max drift measure (§3.2) is unvalidated as a causal proxy for the reorganisation that produces forgetting. That concern is real and load-bearing for the “drift induces dilemma” wording in the abstract and §4.2–4.4. I partially agree: the metric issue is necessary but not the sharpest single point. The more decisive soft spot is the unexplained reversal in Table 5 (Full FT stable on prior Pacific, LoRA catastrophic) relative to Table 3 and to standard PEFT literature, combined with the absence of any high-resource or same-language sequential control that would isolate linguistic distance from extreme data scarcity and update magnitude. The paper’s empirical demonstration of poor continual performance on three real Pacific languages remains useful and publishable; the causal narrative and the generality of the “strict dilemma” are not yet secured. Hence the verdict stays CONDITIONAL (reproducibility + the control experiments above), not REJECT. No evidence of internal inconsistency or fabrication; the gap is missing controls and causal linkage, ordinary for an empirical ASR study of this scope.","tokens_in":9998,"tokens_out":688,"duration_ms":6181,"concrete_test":"Re-run the Nafsan→Lelepa sequence (same Whisper-Small, same hours/splits) with (a) a high-resource control sequence (e.g., French→Spanish or English→French) and (b) an early-encoder freeze + LoRA-on-later-layers ablation; report English + prior-Pacific WER and layer-wise cosine drift. If Full FT still retains prior Pacific better than LoRA only on Pacific pairs, and if freezing early layers reduces both Lelepa drift and forgetting, the causal claim holds; otherwise the dilemma is confounded by data scarcity or update scale.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim (severe drift from linguistic distance forces a plasticity-stability dilemma, with LoRA plastic but forgetful) rests on two linked pieces: (1) cosine-distance drift after per-language min-max normalisation on a fixed multi-language set (§3.2, Figs. 1–2) as evidence of structural reorganisation, and (2) the Nafsan→Lelepa sequential results (Table 5) where Full FT retains Nafsan WER ~45–47 while LoRA/DoRA/O-LoRA jump to ~84–87. The metric is only correlational: it is never shown that higher early-encoder drift (Lelepa) predicts larger English or prior-language degradation after controlling for update magnitude or target CER. Moreover, Table 5’s pattern is the reverse of the usual PEFT stability advantage and of Table 3 (where Full FT forgets English more). Without a matched high-resource sequential control, a same-language sequential control, or an ablation that freezes early encoder layers while measuring both drift and forgetting, the causal story that “linguistic distance → drift → dilemma” remains under-supported. The reader correctly flags the metric; the sequential reversal is the sharper load-bearing gap.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"The paper presents an empirical study of adapting Whisper-Small to three low-resource Pacific Indigenous languages (Bislama, Nafsan, Lelepa) drawn from PARADISEC. It measures cross-lingual transfer under varying data budgets (0.5–10 h) with full fine-tuning versus LoRA, quantifies layer-wise representational drift via cosine distance of hidden states (min-max normalised per language), and evaluates catastrophic forgetting both on high-resource languages (English/Chinese/French) and in a sequential Nafsan→Lelepa continual-learning setting that also includes DoRA and O-LoRA. The central claim is that linguistic distance induces severe internal drift, producing a strict plasticity–stability dilemma in which LoRA is initially plastic yet forgets prior languages, while full fine-tuning is more stable but less plastic.","tokens_in":10266,"tokens_out":1097,"duration_ms":13238,"significance":"If the claimed drift–forgetting link holds, the work supplies a concrete stress test of speech foundation models on typologically distant, genuinely low-resource Pacific data and documents a reversal of the usual PEFT stability advantage. The curated multi-language corpus, systematic data-volume sweeps, layer-wise drift profiles (Figs. 1–2), and side-by-side comparison of Full FT / LoRA / DoRA / O-LoRA (Tables 2–5) are valuable empirical contributions for the low-resource ASR and continual-learning communities. The paper correctly flags that current adaptation recipes leave the dilemma unresolved and motivates architecture-aware or dynamic methods.","major_comments":[{"comment":"§3.2 and Figs. 1–2: The cosine-distance drift (after per-language min-max normalisation on a fixed multi-language set) is presented as evidence of the structural reorganisation that causes forgetting. No experiment shows that higher early-encoder drift (Lelepa) predicts larger English or prior-language degradation after controlling for update magnitude or target CER. Without such a control, ablation (e.g., freezing early encoder layers), or matched high-resource sequential baseline, the causal chain “linguistic distance → drift → plasticity–stability dilemma” remains correlational.","section":null},{"comment":"Table 5 (Nafsan→Lelepa): Full FT retains Nafsan WER ≈45–47 while LoRA/DoRA/O-LoRA jump to ≈84–87, reversing both the usual PEFT stability advantage and the pattern in Table 3 (where Full FT forgets English more severely). The manuscript offers no explanation, matched same-language sequential control, or high-resource sequential control that would isolate linguistic distance from other factors (data volume imbalance, optimisation trajectory, vocabulary extension). This reversal is load-bearing for the claimed dilemma and needs either mechanistic analysis or additional controls.","section":null},{"comment":"§3.4 states that three random seeds are run and mean/std of CER/WER are reported, yet Tables 2–5 contain only point estimates with no standard deviations or confidence intervals. Without variance, it is impossible to judge whether the Full-FT vs LoRA differences (especially the Lelepa 2 h crossover and the sequential forgetting gaps) are reliable.","section":null},{"comment":"Table 3 caption claims evaluation on “English LibriSpeech” yet the table columns report Chinese CER, English WER and French WER; the experimental protocol for the non-English languages is never described. Clarify the evaluation sets and ensure the forgetting numbers are comparable across languages.","section":null}],"minor_comments":[{"comment":"§2 opens with “four underrepresented Pacific Indigenous languages” but only three (Bislama, Nafsan, Lelepa) are listed and tabulated; correct the count.","section":null},{"comment":"Table 1 header uses “locales”; the body and rest of the paper use language names—align terminology.","section":null},{"comment":"LoRA rank, alpha and target modules are never stated; add them for reproducibility.","section":null},{"comment":"Figures 1–2 lack error bars or indication of which checkpoint (Full FT vs LoRA, which data budget) is plotted; the caption should specify.","section":null},{"comment":"Abstract and §1 claim “three distinct Pacific Indigenous languages” while §2 mentions four; keep consistent.","section":null},{"comment":"Minor typographical issues: “plasticity and stability dilemma” vs “plasticity–stability”; “Avg. WER” column in Table 5 appears misaligned with the two-stage numbers.","section":null}],"recommendation":"major_revision","confidential_remarks":"The empirical scope and real Pacific data are a genuine strength and the paper is a reasonable fit for a speech/ASR venue. The main risk is over-claiming a causal drift story from purely correlational measurements; once the controls and variance reporting are added (or the claims are appropriately softened), the contribution should be solid. No concerns about citation manipulation or scope mismatch beyond the usual conference-to-journal expansion."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The one thing worth knowing is that this paper actually measures what happens when you push Whisper-Small onto three genuine low-resource Pacific languages (Bislama, Nafsan, Lelepa) with real PARADISEC data, and the numbers show a clear bind: full fine-tuning holds prior knowledge better but learns the new language poorly, while LoRA (and DoRA/O-LoRA) learns the target faster yet forgets earlier languages hard. That sequential Nafsan-to-Lelepa result in Table 5 is the new piece; the layer-wise cosine drift plots simply document that Lelepa forces early-encoder restructuring that the others do not.\n\nWhat they do well is straightforward and useful. They vary data volume from 0.5 h to 10 h, compare full FT against encoder-decoder LoRA and the two regularized variants, and report both target CER/WER and English forgetting on the same checkpoints. The corpus stats and the component-ablation in Table 4 are clean enough that you can see the encoder is where the damage happens. Citations are ordinary and the self-cites are to their own prior low-resource work, not circular. No invented math; everything is measured against held-out sets.\n\nSoft spots are ordinary for this kind of empirical ASR note and do not sink the claim. Standard deviations are promised for three seeds but never appear in the tables. The drift metric is only correlational; they never ablate early-encoder freezes or match update magnitude to show it causes the forgetting. The sequential pattern also reverses the usual PEFT stability advantage without a high-resource control sequence, so the “linguistic distance forces the dilemma” story is under-supported even though the raw numbers are consistent. Only one base model, no public code. None of that makes the measurements fake.\n\nThis is for anyone working on low-resource or Indigenous ASR who needs a concrete warning that the standard LoRA recipe does not transfer cleanly. It deserves a serious referee; the gaps are fixable reporting and control issues, not conceptual collapse. I would engage with it and expect it to appear after revision.","headline":"Clean empirical demonstration that Whisper adaptation to real Pacific languages forces a hard plasticity-stability bind, even if the drift-to-forgetting causal link stays correlational.","tokens_in":10926,"tokens_out":534,"would_cite":true,"duration_ms":11535,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Adapting speech foundation models to distant Pacific Indigenous languages induces severe representational drift and creates a hard plasticity-stability dilemma.","keywords":["speech recognition","Pacific languages","continual learning","representational drift","catastrophic forgetting","LoRA","low-resource ASR","Indigenous languages"],"falsifier":"If high measured cosine drift occurred without any rise in error rates on previously learned languages, or if low-drift adaptations still produced severe forgetting on English and earlier Pacific languages, the claimed causal link between drift and the plasticity-stability dilemma would be disproved.","tokens_in":10866,"feed_emoji":"🗣️","tokens_out":824,"duration_ms":16447,"temperature":0.7,"pith_summary":"Speech foundation models fail to transfer cleanly to real low-resource Pacific languages because those languages sit far from the high-resource data used in pretraining. Fine-tuning therefore forces large internal reorganisation of the model's acoustic and linguistic features rather than simple refinement. The paper measures this reorganisation as layer-wise representational drift and shows that it produces a strict trade-off: methods that quickly acquire a new Pacific language (especially LoRA) erase earlier languages, while full fine-tuning retains more prior knowledge but underperforms on the new target under extreme data scarcity. Sequential experiments on Nafsan then Lelepa confirm that neither standard approach solves the dilemma. The result matters because it demonstrates that today's adaptation toolkit is structurally inadequate for underrepresented Indigenous languages and that speech technology for Pacific communities will require purpose-built continual-learning methods.","feed_headline":"Pacific languages force speech models to rewrite and forget","feed_subtitle":"LoRA learns new tongues fast but erases earlier ones; full fine-tuning does the reverse.","key_machinery":"Layer-wise representational drift quantified by cosine distance between pre- and post-adaptation hidden states of every encoder and decoder layer, evaluated on a fixed multi-language set and linked to sequential continual-learning error rates that expose catastrophic forgetting.","core_discovery":"Adapting speech foundation models to linguistically distant Pacific Indigenous languages induces severe internal representational drift. As a direct consequence the models confront a strict plasticity-stability dilemma: LoRA and related parameter-efficient updates achieve good initial accuracy on the new target yet suffer catastrophic forgetting of prior languages during sequential learning, while full fine-tuning better preserves earlier knowledge at the cost of weaker acquisition of the new language.","pith_inferences":["Comparable drift-and-forgetting patterns are likely for other typologically distant low-resource families outside the Pacific.","Regularisation techniques successful in text continual learning may require acoustic-specific redesign before they protect speech foundation models.","Modular, language-isolated adapters could reduce interference without sacrificing the plasticity needed for extreme low-resource targets.","Multilingual ASR benchmarks should report representational drift and sequential forgetting alongside final error rates."],"forward_implications":["Neither full fine-tuning nor LoRA (nor DoRA/O-LoRA) can simultaneously acquire a new distant Pacific language and retain earlier ones.","Adaptation success scales with data volume and linguistic proximity; English-related creoles adapt far more readily than isolated Austronesian languages.","Updating only the encoder or only the decoder fails to balance target accuracy against source-language retention.","Speech systems intended for Pacific communities will need continual-learning methods that explicitly protect historical representations.","Future architectures must handle both extreme data imbalance and deep acoustic restructuring."],"fun_headline_variants":["Pacific languages force severe drift then forgetting in speech models","Speech models rewrite internals for Pacific tongues then lose prior ones","LoRA grabs new Pacific speech yet erases earlier languages in sequence","Adapting to distant Pacific languages locks models in plasticity trap","Continual Pacific Indigenous adaptation exposes speech model stability clash"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The paper treats cosine distance between pre- and post-adaptation hidden states, after per-language min-max normalisation, as a faithful measure of the structural reorganisation that causes forgetting.","fun_headline_variants_meta":{"raw":{"variants":["Pacific languages force severe drift then forgetting in speech models","Speech models rewrite internals for Pacific tongues then lose prior ones","LoRA grabs new Pacific speech yet erases earlier languages in sequence","Adapting to distant Pacific languages locks models in plasticity trap","Continual Pacific Indigenous adaptation exposes speech model stability clash"]},"model":"grok-4.5","effort":"low","cost_usd":0.004138,"raw_usage":{"total_tokens":1169,"prompt_tokens":671,"num_sources_used":0,"completion_tokens":84,"cost_in_usd_ticks":41380000,"prompt_tokens_details":{"text_tokens":671,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":414,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":671,"tokens_out":84,"duration_ms":4059,"temperature":1.0,"reasoning_tokens":414,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-15T13:53:13.058232+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"If high measured cosine drift occurred without any rise in error rates on previously learned languages, or if low-drift adaptations still produced severe forgetting on English and earlier Pacific languages, the claimed causal link between drift and the plasticity-stability dilemma would be disproved.","supporting_citations":[],"review_version":1}