{"id":"e5f54c54-10ed-4d91-94e1-496e995f9186","arxiv_id":"2508.06280","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Regularization-based continual learning methods (EWC, MAS, LwF) reduce catastrophic forgetting in sequential multilingual Indic ASR compared to naive fine-tuning.","lead":"This paper tests three continual learning methods for teaching a speech recognition model new Indian languages without erasing earlier knowledge. It compares them on a nine-language benchmark, showing they reduce forgetting compared to naive fine-tuning.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unstated regularization-strength hyperparameters could confound the CL-vs-fine-tuning comparison; code check needed.","rationale":"The reader's weakest assumption points to overall experimental protocol consistency, which is broader than my concern. I partially agree because the most vulnerable and specific aspect of that protocol is the hyperparameter calibration for regularization-based CL methods. The abstract alone provides insufficient information to rule out confounding; the public code makes a targeted test feasible. My concern does not change the verdict: the paper remains UNVERDICTED until the experimental details (especially hyperparameters and tuning procedure) are verified. I am not alleging misconduct; the issue is standard empirical validity. If the concrete test passes, the central claim is likely to hold; if it fails, the contribution would need significant revision. No other more fundamental concern was identifiable from the abstract: the architecture choice (Conformer RNN-T/CTC) is sound, the benchmark selection is reasonable, and no internal inconsistency is apparent.","tokens_in":768,"tokens_out":3155,"duration_ms":38520,"concrete_test":"Inspect the public repository (https://github.com/FrozenWolf-Cyber/Indic-CL-ASR) and extract the exact λ values for EWC and MAS, the distillation weight/temperature for LwF, and the per-epoch training schedule. Then rerun the primary BWT/WER comparison for a representative subset (e.g., the first three languages after Hindi) with λ swept across at least three orders of magnitude (e.g., 1e-2, 1e-1, 1, 1e1, 1e2) for EWC/MAS and a correspondingly wide range for LwF's distillation weight. If the reported ordering—CL methods dominating naive fine-tuning on BWT without WER regression—holds across all tested hyperparameters, the concern is resolved. If the ordering flips or disappears for most settings, the central claim is not robust.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that EWC, MAS, and LwF mitigate forgetting relative to naive fine-tuning—rests on a controlled comparison. Regularization-based CL methods are governed by at least one critical hyperparameter: λ for EWC/MAS and the distillation weight/temperature for LwF. If these are chosen or tuned differently per method, the observed differences in backward transfer (BWT) and WER may reflect hyperparameter selection rather than algorithmic superiority. The abstract reports no details about these values, the tuning protocol, or whether compute was matched. Given that the model is pretrained on Hindi and then trained on eight languages, the training order and per-task epochs also directly affect BWT; the abstract only mentions the number of epochs (1,2,5,10) but not how these were applied per method. Without evidence of fair hyperparameter calibration, the reported 'effectiveness in mitigating forgetting' is not established. This is the standard validity threat for empirical CL papers, and it is especially load-bearing here because the paper's contribution is entirely empirical.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents an empirical study of three regularization/distillation-based continual learning (CL) methods—EWC, MAS, and LwF—applied to a Conformer-based hybrid RNN-T/CTC ASR model. The model is pretrained on Hindi and then incrementally trained on eight additional Indian languages from a subset of the IndicSUPERB benchmark, totaling nine sequential tasks. The authors report WER (clean and noisy) for both decoding paths, backward transfer (BWT), and comparisons against naive fine-tuning. They also vary the number of training epochs per task (1, 2, 5, 10), concluding that CL mitigates catastrophic forgetting. The abstract includes a code repository link. The manuscript is currently available only as an abstract in this review context, so all assessment is based on the abstract's claims.","tokens_in":998,"tokens_out":1758,"duration_ms":19900,"significance":"If the reported results hold, this would be a practically relevant contribution: it extends CL evaluation to a low-resource, multilingual ASR setting with Indic languages, using a realistic no-replay, privacy-conscious scenario. The use of standard metrics (WER, BWT) and comparison to an external baseline (naive fine-tuning) is appropriate. The release of code is a positive element for reproducibility. However, the significance cannot be fully assessed without experimental details that the abstract omits; the central comparative claim is plausible but not yet established from the available information.","major_comments":[{"comment":"The central claim that EWC, MAS, and LwF mitigate forgetting relative to naive fine-tuning depends on the regularization strengths being fairly calibrated. The abstract reports no values for the EWC/MAS penalty weight (λ), the LwF distillation weight or temperature, or the tuning protocol. If λ=0 or is effectively negligible, the methods reduce to fine-tuning; if λ is very large, they may underfit new tasks. Without reporting these values and the calibration strategy, the observed BWT differences may reflect hyperparameter selection rather than algorithmic merit. This is load-bearing because the paper's contribution is entirely empirical.","section":"Abstract (CL-vs-fine-tuning comparison)"},{"comment":"The abstract mentions varying epochs per task (1, 2, 5, 10) but does not state how these epochs were allocated across the nine languages or whether the schedule was matched across methods. For continual learning, language ordering and per-task epoch counts directly affect backward transfer and final WER. If, for example, the ordering was not held fixed or early tasks received fewer epochs than later ones, the comparison against naive fine-tuning would be confounded. The abstract gives no information on this point.","section":"Abstract (training schedule)"},{"comment":"No error bars, confidence intervals, significance tests, or number of independent runs are reported. Since ASR evaluations on small subsets can exhibit high variance, and since BWT is computed as a difference of WERs, the absence of any variability measure means the reported 'effectiveness' could be within noise. At minimum, the authors should report mean±std over multiple seeds or a significance test for the key comparative claims.","section":"Abstract (statistical reliability)"},{"comment":"The baseline is 'naive fine-tuning,' but the abstract does not clarify whether this baseline uses the same model architecture, task ordering, and computational budget as the CL methods. Specifically, it is unclear whether naive fine-tuning is simply sequential SGD without any regularization or an equivalent-capacity training schedule. This matters because LwF can be viewed as a modified fine-tuning objective; a fair baseline must isolate the effect of the CL term. The authors should state the baseline configuration explicitly.","section":"Abstract (baseline specification)"}],"minor_comments":[{"comment":"Typographical issue: 'CLs effectiveness' should be 'CL's effectiveness' or 'CL effectiveness'.","section":"Abstract"},{"comment":"The abstract says the model is 'initially pretrained on Hindi' and then trained on a total sequence of nine languages. If Hindi is one of the nine, the wording is slightly ambiguous; please clarify whether Hindi is the first task or a separate pretraining step, as this affects the continual learning setup.","section":"General"},{"comment":"The subset of IndicSUPERB is not specified. Since the full benchmark may contain many languages, the choice of eight additional languages and their similarity to Hindi could influence forgetting. Mentioning the selected languages (or the selection criterion) would help readers judge generalizability.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"This referee report is based solely on the abstract, as the full text was not available. The abstract's comparative claims are not verifiable without detailed experimental protocols—especially hyperparameter settings and language ordering. The issues are fixable in principle by expanding the manuscript with the missing experimental details, so major revision seems appropriate rather than outright rejection. If the full text already contains these details, the authors should make the abstract reflect them or ensure the relevant sections are prominent. I recommend that the editor, if possible, obtain the full manuscript before making a final decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear Colleague,\n\nQuick read on arXiv:2508.06280. It's an abstract-only review, so this is a triage, not a verdict. The paper applies three well-known regularization-based continual learning methods (EWC, MAS, and LwF) to a Conformer RNN-T/CTC ASR model trained sequentially on nine Indian languages from IndicSUPERB, with Hindi as the first task. The good part is that the setup matches a real deployment constraint: languages arrive sequentially, and you can't replay data. No-replay CL is the right tool for that, and the comparison against naive fine-tuning is the right baseline. The code is linked, which helps reproducibility. If the numbers are solid, this is a useful data point for practitioners who need privacy-preserving ASR pipelines in low-resource language contexts.\n\nThe soft spots are exactly where you'd expect them. The abstract reports WER and backward transfer but says nothing about how the regularization strength (λ for EWC/MAS, distillation weight/temperature for LwF) was chosen. With three methods plus a fine-tuning baseline, small tuning differences can flip the ranking. The training order of the eight languages after Hindi is also unspecified, and order effects on backward transfer are known to be large in CL. The paper mentions epochs of 1,2,5,10 but not whether those were applied identically across methods or whether compute/epoch budgets were matched. None of this is disqualifying—most empirical CL papers have the same holes—but it does mean the headline claim about mitigating forgetting should be read as preliminary until the full protocol is in view.\n\nI don't see a conceptual problem. The contribution is deliberately modest: apply established methods to a new language/benchmark combination and report measurements. That's a legitimate contribution to an applied area, not a mystery. The stress-test note about hyperparameters is on the mark, but it's a standard validity threat rather than evidence of a flaw. Based on the abstract alone, I'd send this to a competent reviewer who knows ASR and CL, with the instruction to check the hyperparameter protocol and the code. If the tuning is done honestly, it's a publishable empirical study.\n\nWould I cite it? Maybe, if the full version verifies the results. I'd bring it to a reading group as an example of how CL transfers to a practically motivated domain.\n\nRecommendation: send to peer review. Not my field's flagship, but a serious referee can judge the protocol more carefully than the abstract allows.\n\nBest,\n[You]","headline":"Plausible applied CL study with a real privacy-sensitive use case; the comparison is standard and the main risk—untracked regularization hyperparameters—can't be assessed from the abstract alone.","tokens_in":1421,"tokens_out":3010,"would_cite":false,"duration_ms":29091,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that regularization-based continual learning (EWC, MAS, LwF) reduces catastrophic forgetting when a single ASR model is trained sequentially on nine Indian languages, outperforming naive fine-tuning on knowledge retention.","keywords":["continual learning","automatic speech recognition","catastrophic forgetting","Indian languages","EWC","MAS","learning without forgetting","RNN-T/CTC"],"falsifier":"Take the same Hindi-pretrained model and run the identical nine-language sequence with naive fine-tuning and with each CL method at 1, 2, 5, and 10 epochs; after the final language, measure WER on the original Hindi test set. If naive fine-tuning retains Hindi accuracy as well as or better than EWC, MAS, and LwF across those settings, the claim that CL mitigates forgetting in this setting is refuted.","tokens_in":705,"feed_emoji":"🗣️","tokens_out":3882,"duration_ms":39981,"temperature":0.7,"pith_summary":"The paper argues that sequential multilingual ASR for Indian languages is practical if the model is trained with continual learning rather than naively fine-tuned. It evaluates three no-replay regularization and distillation methods—EWC, MAS, and LwF—on a Conformer-based hybrid RNN-T/CTC model pretrained on Hindi and then trained on eight further Indic languages one at a time. The central comparison is against naive fine-tuning: the CL methods are claimed to retain previously learned languages better, as measured by backward transfer, while keeping word error rates competitive on both clean and noisy speech. A reader should care because this points to a way of scaling ASR across many languages without storing all language data at once, which matters for privacy and for languages whose data arrives over time.","feed_headline":"Regularization cuts forgetting when ASR learns nine Indian languages","feed_subtitle":"EWC, MAS, and LwF retain prior languages better than naive fine-tuning in sequential training.","key_machinery":"The central machinery is a set of parameter-regularization and distillation constraints applied while the model trains sequentially on each new language. EWC adds a quadratic penalty on weights important to previously learned tasks; MAS penalizes changes to parameters that most affect the model's outputs; LwF forces new-task training to preserve old-task predictions. Together they keep gradient updates for a new language from overwriting the decision regions learned for earlier languages.","core_discovery":"The paper's central claim is that EWC, MAS, and LwF each mitigate catastrophic forgetting in sequential multilingual ASR for Indian languages relative to naive fine-tuning, while staying competitive on word error rate for both RNN-T and CTC outputs on clean and noisy speech. The results are established on a subset of the IndicSUPERB benchmark: a Conformer-based hybrid model pretrained on Hindi is incrementally trained on eight additional languages, and retention is measured by backward transfer across training epochs of 1, 2, 5, and 10. The contribution is an application of established continual-learning strategies to a realistic no-replay, privacy-conscious scenario in a linguistically dive","pith_inferences":["Language order likely interacts with retention: training similar languages adjacently may improve backward transfer, a testable hypothesis the paper does not pursue.","Because the paper deliberately excludes replay for privacy, combining these regularizers with a small stored exemplar set might further improve retention—an extension beyond the paper's no-replay design.","Backward transfer measures only preservation of past languages; a fuller picture would also measure forward transfer, i.e., whether earlier languages make new languages easier to learn.","The same regularization constraints could plausibly transfer to other Conformer-based ASR pipelines for additional low-resource languages, but that scaling claim is an extrapolation, not tested here."],"forward_implications":["If the paper's claim holds, ASR systems can be updated language-by-language without access to older audio, supporting privacy-preserving deployment.","One pretrained model can grow to cover many Indian languages, reducing the need for a separate model per language.","Regularization-based CL offers a viable baseline for any sequential ASR task where replay of old data is impossible or undesirable.","The retention benefit appears in both RNN-T and CTC output paths, so the approach applies to hybrid architectures rather than a single decoding path.","The effect is reported on clean and noisy speech, suggesting that noise robustness does not cancel the forgetting-mitigation benefit."],"supporting_citations":[],"fun_headline_variants":["EWC, MAS, LwF curb forgetting in 9-language Indic ASR","Regularization beats fine-tuning for sequential Indic ASR","No-replay continual ASR: three methods that remember","Fewer language drops: CL strategies for Indian ASR"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The comparison depends on the language order, data subset, and hyperparameters being fair and representative; if those choices favor the CL methods, the observed retention advantage may not be inherent to the methods.","fun_headline_variants_meta":{"raw":{"variants":["EWC, MAS, LwF curb forgetting in 9-language Indic ASR","Regularization beats fine-tuning for sequential Indic ASR","No-replay continual ASR: three methods that remember","Fewer language drops: CL strategies for Indian ASR"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000246,"raw_usage":{"total_tokens":1403,"prompt_tokens":798,"completion_tokens":605,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":542,"completion_tokens_details":{"reasoning_tokens":533}},"tokens_in":542,"tokens_out":605,"duration_ms":6150,"temperature":1.0,"reasoning_tokens":533,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T22:47:39.624313+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the same Hindi-pretrained model and run the identical nine-language sequence with naive fine-tuning and with each CL method at 1, 2, 5, and 10 epochs; after the final language, measure WER on the original Hindi test set. If naive fine-tuning retains Hindi accuracy as well as or better than EWC, MAS, and LwF across those settings, the claim that CL mitigates forgetting in this setting is refuted.","supporting_citations":[],"review_version":1}