{"id":"fd8ef4dc-0947-46ab-a111-aa814455dc6c","arxiv_id":"2411.09884","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A plug-in BERT extension module trained on domain terminology plus a dual-prediction heuristic improves domain-specific Chinese spelling correction on medical, legal, and official-document tests.","lead":"This paper adds a plug-in module to a BERT-style Chinese spelling corrector so it can handle medical, legal, and official-document terms without retraining the base model. Tests on two domain datasets show large F1 gains, but the paper lacks baseline comparisons and general-domain checks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Table 2 gains cannot be attributed to the plugin extension module: the '+Plugin' condition appears to include the dual-prediction algorithm (§3.2), and no ablation isolates the plugin's contribution.","rationale":"The reader's weakest assumption (synthetic data transfer) is real, but Table 2 already evaluates on real test sets, providing some evidence of transfer. The more fundamental gap is attribution: the reported gains could come from the dual-prediction post-processing rather than the learned plugin. The paper never specifies the composition of the '+Plugin' condition, and no ablation is provided. This directly threatens the central claim that the plugin causes the improvement. The reader noted the missing dual-prediction ablation but did not frame it as the primary weakness; hence partial agreement. The concrete test (2×2 ablation) would settle this. Because the reader's verdict was already CONDITIONAL, and this concern reinforces the need for major revision, the verdict remains UNCHANGED.","tokens_in":7657,"tokens_out":8509,"duration_ms":85951,"concrete_test":"Run a 2×2 ablation on the four domain test sets: (1) MM-BERT baseline, (2) MM-BERT + dual-prediction only, (3) MM-BERT + plugin only, (4) MM-BERT + plugin + dual-prediction. Report detection/correction F1 for each. If condition (2) accounts for most of the gap between (1) and (4), or condition (3) shows little improvement, the plugin's contribution is not supported. Add the same four conditions on a general-domain benchmark such as SIGHAN 2015 to test the 'without compromising general performance' claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.2 attributes the F1 increases of 2–3× in Table 2 to the plugin extension module, but the experimental comparison does not isolate this component. The method section describes two changes to the base model: the plugin module (§3.1) and the dual-prediction change analysis algorithm (§3.2). The '+Plugin' rows in Table 2 are never defined as excluding dual-prediction; since §3.2 is part of the proposed method, the comparison is between the base model and the base model plus both modifications. If dual-prediction is applied in the plugin condition, part or all of the gain may come from the post-processing heuristic, which is not ablated. If it is not applied, the dual-prediction algorithm has no experimental support at all. The paper also omits any description of the MM-BERT baseline and any measurement of general-domain performance, despite claiming it is preserved. Because the central claim is specifically that the extension module improves performance, the missing ablation leaves that causal attribution unverified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a plugin-based extension module for domain-specific Chinese spelling correction. The base model's parameters are frozen, and an additional Transformer module is trained on domain terminology (with confusion-set corruption and random padding) to inject domain knowledge. A dual-prediction change analysis algorithm is also proposed to reduce over-correction. Experiments on medical, legal, and official-document datasets (MCSC-Test, EC-LAW, EC-Med, EC-ODW) report large F1 gains over a baseline called MM-BERT after adding the medical/legal/official-document plugin. The abstract and conclusion claim that general-domain spelling correction performance is preserved because the original model parameters are frozen.","tokens_in":7832,"tokens_out":3465,"duration_ms":35849,"significance":"If the central claim were fully supported, the approach would be a practical and lightweight way to adapt existing Chinese spelling correction models to new domains without retraining the base model, and the use of public term dictionaries to generate training data would lower the annotation barrier. The paper deserves credit for a clean design idea: freezing the base model and training only an extension module is a sensible way to address catastrophic forgetting, and the reported gains on three domains are substantial. However, the current evidence does not isolate the plugin's contribution from the dual-prediction post-processing heuristic, the general-domain preservation claim is never directly measured, and the baseline and training data construction are insufficiently documented. The significance of the result therefore depends on additional experiments that the manuscript does not yet provide.","major_comments":[{"comment":"The central claim that the plugin extension module causes the 2–3× F1 gains is not supported because the '+Plugin' condition is never defined relative to the dual-prediction change analysis algorithm of §3.2. The table compares 'MM-BERT' with 'MM-BERT + Plugin' but does not state whether the dual-prediction post-processing (Algorithm 1) was applied in either condition, so the reported gains cannot be attributed to the extension module alone. An ablation with four conditions—base, base+plugin, base+dual-prediction, and base+plugin+dual-prediction—is required to establish the contribution of each component.","section":"§4.2, Table 2"},{"comment":"The claim that general-domain performance is 'not compromised' is asserted but never measured. Although freezing the base parameters prevents direct modification of those weights, the forward pass changes from Y = O(BERT(E(X))) to Y' = O(BERT(E(X)) + P(E(X))), so the extension module can alter outputs on general-domain text as well. The paper should report detection and correction F1 on a general-domain benchmark such as SIGHAN or Wang271K, with and without each plugin, to substantiate the abstract's claim that the extension module does not compromise general spelling correction performance.","section":"§3.1, §4.2"},{"comment":"For the legal and official-document domains, the extension module is trained only on pseudo datasets constructed by corrupting bare domain terms with confusion sets and padding them with random characters, and no evidence is provided that this synthetic signal transfers to real sentence-level errors. EC-LAW, EC-MED, and EC-ODW are test-only sets, so there is no validation set for early stopping or hyperparameter selection in those domains. The paper should report how these hyperparameters were chosen, and should compare against fine-tuning the base model or using adapter methods on the same training signal to justify the synthetic-data approach.","section":"§4.1, Table 1"},{"comment":"The baseline 'MM-BERT' is not described, referenced, or defined anywhere in the manuscript, and no comparison is made with existing domain-adaptive Chinese spelling correction systems such as ECSpell on the EC datasets, MedSpell on MCSCSet, or standard adapter/PEFT baselines. Without a documented baseline and comparable systems, the absolute F1 numbers in Table 2 are difficult to interpret, and the claim that the proposed method outperforms existing approaches is not yet established.","section":"§4.2, Table 2"},{"comment":"No error bars, significance tests, or multiple-run statistics are reported. Given that the EC-LAW and EC-ODW test sets contain only about 2,000–2,500 sentences and the gains differ substantially across domains, the paper should report variance across random seeds or bootstrap confidence intervals so that the reader can assess the stability of the observed improvements.","section":"§4.2, Table 2"}],"minor_comments":[{"comment":"The text says the network architecture is shown in 'Figure 3-4', but only Figure 1 is provided for the architecture; the reference should be corrected to Figure 1.","section":"§3"},{"comment":"The reference list contains duplicate entries: [Jiang et al. 2022a] and [Jiang et al. 2022b] are the same paper, and [Lv et al. 2023a] and [Lv et al. 2023b] are the same paper; these should be merged.","section":"References"},{"comment":"The experimental setup does not report the number of extension module layers used in the experiments, the batch size, the number of training epochs, or the size of the pseudo datasets; these details are needed for reproducibility.","section":"§4.1"},{"comment":"The definitions of the detection and correction metrics are not given; the paper should state whether the reported precision, recall, and F1 are computed at the character level, the word level, or the sentence level, and how the 'correct' prediction is counted.","section":"§4.2"},{"comment":"In the S4 condition, the check 'if Si = 4 and Si−1 != 3 and Si+1 != 3' uses undefined neighbor states S_−1 and S_{n+1} for boundary characters; the algorithm should specify boundary handling.","section":"Algorithm 1"},{"comment":"The description of the five change scenarios in the text and the algorithm's classification do not explicitly map the scenario 'A→B→C' to case S1; clarifying this mapping would make the method easier to follow.","section":"§3.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is an early-stage preprint with a potentially useful modular adaptation idea, but the current experimental design does not isolate the plugin's effect from the dual-prediction heuristic, and the general-domain preservation claim is not directly tested. The missing ablations and baseline documentation are fixable within the scope of the manuscript, so I did not recommend rejection, but the revision needs substantial additional experiments: a full ablation of the two proposed components, a general-domain evaluation, a documented baseline, and ideally a comparison with existing domain-adaptive methods. I would also ask the authors to clarify the relationship between this submission and the cited ECSpell and MedSpell work, since the EC and MCSC datasets are borrowed from those systems without an explicit comparison."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nShort version: this paper has a clean, plausible idea—freeze a BERT-based speller and bolt on a small adapter-like extension module trained on domain lexicons with confusion-set noise—and it reports 2–3x F1 gains on medical, legal, and official-document test sets. That is worth a look. But the experiments as reported don't actually show that the plugin module is what causes the gains, because the \"+Plugin\" condition appears to bundle in the dual-prediction post-processing heuristic from §3.2, and there is no ablation separating them. The stress-test note is right: the central claim is attributed to the plugin, yet the table doesn't isolate it.\n\nWhat's genuinely new: the specific combination is not in the prior work I know. Using a frozen backbone with a side network trained on bare terms plus random padding is a reasonable recipe for domain adaptation when you have no in-domain sentences, and the idea that the extension should learn to leave unfamiliar characters alone is sensible. The dual-prediction change analysis is a modest heuristic, but at least it's explicit.\n\nWhere it falls short, in rough order of importance. First, the component attribution problem above—needs an ablation that runs plugin-without-dual-prediction and dual-prediction-without-plugin. Second, the MM-BERT baseline is never described; we don't know what it is, how it was trained, or whether it's a strong baseline. Third, the paper claims general-domain performance is preserved, but never measures it on any open-domain benchmark. That's a key part of the pitch and it's untested. Fourth, for legal and official-document domains the training signal is entirely synthetic, and there is no check against real error corpora or against fine-tuned or adapter-based baselines. Fifth, no error bars or significance tests; some of those F1 differences are large, but with small test sets like EC-LAW (2,460 sentences) a stability check would help. Sixth, no code or data. The writing also has placeholder text, duplicate references, and inconsistent figure references—needs a cleanup pass.\n\nThis is not a fatal flaw. The idea is worth engaging, and the results, if they hold up under proper isolation and baselines, would be useful to practitioners who need to adapt general spellers to specialized domains without retraining. It's a solid workshop-level paper that could become a solid conference paper after revision.\n\nMy recommendation: send it to peer review. A serious referee can push for the ablations, general-domain numbers, and baseline documentation. I wouldn't cite it until the evaluation is tightened and ideally code/data are released.","headline":"A pragmatic domain-adaptation idea for Chinese spelling correction with impressive reported gains, but the evaluation doesn't isolate the plugin's contribution, so the main claim is not yet backed up.","tokens_in":8381,"tokens_out":1770,"would_cite":false,"duration_ms":19441,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A plug-in module added to a frozen Chinese spelling model raises domain-specific F1 scores by a factor of 2 to 3.","keywords":["Chinese spelling correction","domain-specific adaptation","plugin extension module","frozen parameters","dual-prediction change analysis","medical legal official documents","synthetic training data","over-correction"],"falsifier":"Collect real misspellings of legal or official-document terminology from a held-out corpus, run the corresponding plugin on them, and compare F1 with the frozen base model; if the synthetic-trained plugin does not beat the base model on these real errors, the core assumption fails.","tokens_in":7420,"feed_emoji":"🔌","tokens_out":6703,"duration_ms":61295,"temperature":0.7,"pith_summary":"This paper tries to show that a Chinese spelling-correction model trained on general text can be extended to specialized domains without retraining the base model. The proposed fix is a plug-in Transformer module that learns domain terminology from word lists and synthetic corrupted samples, while the original model's parameters are frozen. On medical, legal, and official-document test sets, adding the matching plugin raises F1 by about two to three times over the frozen base model (for example, medical detection F1 on MCSC-Test rises from 24.9 to 71.6). The paper also adds a dual-prediction change analysis that compares two model runs to undo over-corrections. If this works, adapting a speller to a new domain becomes a matter of supplying a terminology list and training a small add-on module.","feed_headline":"Plug-in lifts domain spelling F1 up to 3x","feed_subtitle":"Add a small trainable module to a frozen Chinese speller and it fixes medical, legal, and official-document terms.","key_machinery":"The plugin extension module is a 12-layer Transformer encoder, the same architecture as BERT, inserted in parallel with the base model's feature extractor; its output is added elementwise to the base encoder's hidden states before the correction head. Because the base weights stay frozen, all domain-specific learning is forced into this add-on. The second mechanism is dual-prediction change analysis, a post-processing rule that classifies each character's two-run prediction pattern into over-correction, no correction, confident correction, or conditional correction, and reverts characters whose predictions oscillate.","core_discovery":"The central claim is that a domain-specific plugin extension module can supply the missing vocabulary knowledge to a frozen Chinese spelling-correction model. The plugin is a 12-layer Transformer encoder; its hidden features are added to the base BERT encoder's output before the correction head, so the original weights never change. Trained only on domain terms corrupted with confusion-set noise and padded with random characters, the plugin raises detection F1 from 24.9 to 71.6 on MCSC-Test, from 29.0 to 57.8 on EC-Law, from 24.9 to 65.1 on EC-Med, and from 38.2 to 54.7 on EC-Odw. The paper reports that the same recipe works for medical, legal, and official-document domains, and that an unsuitable plugin leaves the input unchanged, which it attributes to the random-padding training trick.","pith_inferences":["A test the paper does not run: whether the legal and official-document plugins, trained only on synthetic padded word pairs, correct real human errors beyond the EC test sets; this is the main open empirical question.","Because fusion is additive, an implicit prediction is that multiple domain plugins could be composed additively, and that out-of-domain terms will be left unchanged thanks to the random-padding training; both are worth checking explicitly.","The dual-prediction rule is a heuristic; a single-pass confidence threshold could achieve similar over-correction suppression at half the inference cost, and a comparison would show whether the second pass pays for itself."],"forward_implications":["Adding a domain plugin roughly doubles to triples detection and correction F1 on the medical, legal, and official-document test sets compared with the frozen base model alone.","General-domain ability is preserved in principle, since the base model's parameters are never updated, so users can attach and detach domain modules at will.","For legal and official documents, the plugin is trained entirely on pseudo-samples built from domain word lists, so no sentence-level error corpus is needed for a new domain.","The dual-prediction algorithm trades a second forward pass for fewer over-corrections, reverting characters whose two predictions disagree or whose conditional correction lacks confident neighbors.","The recipe is presented as compatible with any neural correction model that has an embedding layer, a feature extractor, and a correction head."],"supporting_citations":[{"why":"Supplies the MCSCSet medical corpus (MCSC-Train/Dev/Test) used to train and evaluate the medical plugin.","marker":"[Jiang et al.(2022a)]"},{"why":"Supplies the EC-Law, EC-Med, and EC-Odw test sets used to evaluate the legal, medical, and official-document plugins.","marker":"[Lv et al.(2023a)]"},{"why":"Provides Wang271K, the open-domain Chinese spelling-correction corpus cited as the kind of training data that gives general models their bias against domain terms.","marker":"[Wang et al.(2018)]"}],"fun_headline_variants":["Plugin module triples Chinese spelling F1 on domain texts","Frozen speller gets domain boost from trainable plugin","Domain plugin lifts Chinese spelling F1 from 25 to 72","Add a plugin to a frozen Chinese speller for domain fixes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole reported gain rests on the assumption that synthetic training samples—domain terms corrupted with confusion-set noise and padded with random characters—teach the plugin to correct real misspellings inside full sentences.","fun_headline_variants_meta":{"raw":{"variants":["Plugin module triples Chinese spelling F1 on domain texts","Frozen speller gets domain boost from trainable plugin","Domain plugin lifts Chinese spelling F1 from 25 to 72","Add a plugin to a frozen Chinese speller for domain fixes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000797,"raw_usage":{"total_tokens":3456,"prompt_tokens":846,"completion_tokens":2610,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":462,"completion_tokens_details":{"reasoning_tokens":2540}},"tokens_in":462,"tokens_out":2610,"duration_ms":21656,"temperature":1.0,"reasoning_tokens":2540,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:10:51.414169+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect real misspellings of legal or official-document terminology from a held-out corpus, run the corresponding plugin on them, and compare F1 with the frozen base model; if the synthetic-trained plugin does not beat the base model on these real errors, the core assumption fails.","supporting_citations":[{"cited_title":"A hybrid approach to automatic corpus generation for C hinese spelling check","cited_arxiv_id":null,"evidence_quote":"Provides Wang271K, the open-domain Chinese spelling-correction corpus cited as the kind of training data that gives general models their bias against domain terms."}],"review_version":1}