RECORD
Mitigating Catastrophic Forgetting in Multi-domain Chinese Spelling Correction by Multi-stage Knowledge Transfer Framework
Computation and Language · Submitted 18 Feb 2024
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Experiments prove the effectiveness of our proposed method, and further analyses demonstrate the importance of overcoming catastrophic forgetting for improving the model performance.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Chinese Spelling Correction (CSC) aims to detect and correct spelling errors in given sentences. Recently, multi-domain CSC has gradually attracted the attention of researchers because it is more practicable. In this paper, we focus on the key flaw of the CSC model when adapting to multi-domain scenarios: the tendency to forget previously acquired knowledge upon learning new domain-specific knowledge (i.e., catastrophic forgetting). To address this, we propose a novel model-agnostic Multi-stage Knowledge Transfer (MKT) framework, which utilizes a continuously evolving teacher model for knowledge transfer in each domain, rather than focusing solely on new domain knowledge. It deserves to be mentioned that we are the first to apply continual learning methods to the multi-domain CSC task. Experiments prove the effectiveness of our proposed method, and further analyses demonstrate the importance of overcoming catastrophic forgetting for improving the model performance.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Chinese Spelling Correction (CSC) aims to detect and correct spelling errors in given sentences. | background 1.00 | 0.04 | 0.02 | asserted 1.00 |
| s1 | Recently, multi-domain CSC has gradually attracted the attention of researchers because it is more practicable. | background 1.00 | 0.04 | 0.02 | asserted 0.98 |
| s2 | In this paper, we focus on the key flaw of the CSC model when adapting to multi-domain scenarios: the tendency to forget previously acquired knowledge upon learning new domain-specific knowledge (i.e., catastrophic forgetting). | gap 0.61 | 0.18 | 0.06 | asserted 0.98 |
| s3 | To address this, we propose a novel model-agnostic Multi-stage Knowledge Transfer (MKT) framework, which utilizes a continuously evolving teacher model for knowledge transfer in each domain, rather than focusing solely on new domain knowledge. | method 0.75 | 0.87 | 0.13 | asserted 1.00 |
| s4 | It deserves to be mentioned that we are the first to apply continual learning methods to the multi-domain CSC task. | contribution 0.96 | 0.86 | 0.05 | asserted 1.00 |
| s5 | Experiments prove the effectiveness of our proposed method, and further analyses demonstrate the importance of overcoming catastrophic forgetting for improving the model performance. | result 0.92 | 0.96 | 0.82 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Chinese Spelling Correction (CSC) aims to detect and correct spelling errors in given sentences. Recently, multi-domain CSC has gradually attracted the attention of researchers because it is more practicable. In this paper, we focus on the key flaw of the CSC model when adapting to multi-domain scenarios: the tendency to forget previously acquired knowledge upon learning new domain-specific knowledge (i.e., catastrophic forgetting). To address this, we propose a novel model-agnostic Multi-stage Knowledge Transfer (MKT) framework, which utilizes a continuously evolving teacher model for knowledge transfer in each domain, rather than focusing solely on new domain knowledge. It deserves to be mentioned that we are the first to apply continual learning methods to the multi-domain CSC task. Experiments prove the effectiveness of our proposed method, and further analyses demonstrate the importance of overcoming catastrophic forgetting for improving the model performance.
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites
Cites as background 2 papers
-
The core challenge of the continual learning setting is to minimize catastrophic forgetting of previously acquired knowledge while learning in new domains [ref] .
A Comprehensive Survey of Continual Learning: Theory, Method and Application · Introduction · 0.91
-
Previous works place greater emphasis on a model's ability to generalize to unseen domains, known as zero-shot performance, leveraging shared knowledge across different domains for generalization [ref] .
Chinese Spelling Correction as Rephrasing Language Model · Introduction · 0.86
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.