CSRP: Chain-of-Thought Reasoning for Chinese Text Correction via Reinforcement Learning with Efficiency-Aware Rewards

Man Lan; Wei Tian; Yuhao Zhou

CSRP: Chain-of-Thought Reasoning for Chinese Text Correction via Reinforcement Learning with Efficiency-Aware Rewards

T0 review · 3 major / 0 minor · reviewed 2026-07-12 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A three-stage training pipeline teaches a 4B Chinese correction model to edit only when the gain is worth the change.

desk verdict Solid three-stage Chinese GEC pipeline with a clean CPT/CoT/RL ablation and real precision lift; the hand-crafted EAR is the soft spot but does not sink the result. read the letter →

arxiv 2606.00020 v1 pith:TXN65WMT submitted 2026-04-14 cs.CL cs.AI

Wei Tian , Yuhao Zhou , Man Lan This is my paper

classification cs.CLcs.AI

keywords Chinesegrammaticalerrorcorrectioncontinualpre-trainingchain-of-thoughtGroupRelativePolicyOptimizationefficiency-awarerewardover-correctionNACGECCSCD

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Chinese grammatical error correction with large language models tends to over-correct: maximum-likelihood training pushes fluent rewrites even when the input is already fine, which hurts precision-focused scores. This paper claims the fix is a staged curriculum. First the model absorbs Chinese linguistic priors from 5.9 million balanced samples; then it learns to diagnose errors with explicit chain-of-thought rationales; finally a reinforcement stage optimizes a hand-designed Efficiency-Aware Reward that scores relative improvement against edit distance and directly penalizes unnecessary changes. On the NACGEC native-speaker benchmark the resulting 4B model reaches 50.99 F0.5 and 57.17 precision, beating larger and prior specialized systems while lifting precision roughly eight points with almost no recall loss. The same pipeline also raises Chinese spelling correction to 59.61 F1, five points above GPT-4. Ablations show the reinforcement gain is largely independent of the pre-training stage, so both knowledge and edit discipline are required.

What carries the argument

Efficiency-Aware Reward (EAR): a piecewise scalar built from relative improvement RI = (d(S,G) − d(P,G)) / (d(S,G) + ε) and edit-efficiency η = (d(S,G) − d(P,G)) / (d(S,P) + ε), with fixed thresholds (near-perfect RI → +10, positive RI scaled by η, non-improvement → −3, empty output → −2, and a binary +2/−2 rule for already-correct inputs) that is optimized inside Group Relative Policy Optimization.

What would settle it

Retrain or re-evaluate with the same GRPO stage but replace EAR by a pure accuracy or F0.5 reward (or alter the fixed thresholds); if precision no longer rises ~8 points while recall stays flat, the claim that edit-efficiency optimization is essential collapses.

Watch

Extended reading notes

Core claim

The paper establishes that Chinese grammatical error correction can escape the over-correction plateau of maximum-likelihood fine-tuning by adding Group Relative Policy Optimization guided by an Efficiency-Aware Reward that rewards relative distance reduction to the gold target while penalizing excess edits; this stage alone yields an approximately 8 percent relative F0.5 gain that is orthogonal to large-scale continual pre-training, producing state-of-the-art 50.99 F0.5 and 57.17 precision on NACGEC with a 4B model.

Load-bearing premise

The hand-tuned reward thresholds and Levenshtein ratios correctly capture the linguistic idea of minimal necessary editing and will keep working outside the 67 thousand reinforcement prompts without creating new systematic biases.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The useful takeaway is simple: a 4B model trained CPT → CoT-SFT → GRPO with a Levenshtein-based Efficiency-Aware Reward hits 50.99 F0.5 / 57.17 P on NACGEC and 59.61 F1 on CSCD, beating the previous 4B SOTA and a 14B system, with the RL stage adding roughly +5.5 F0.5 and +8.4 precision at almost no recall cost. The controlled “SFT+GRPO without CPT” run shows that gain is largely additive to the CPT lift, which is the cleanest part of the paper.

What is actually new is not the three-stage recipe itself (CPT + distilled CoT + GRPO are all known) but the specific EAR (relative improvement and edit-efficiency ratio with the identity-preserving special case for already-correct inputs) plus the explicit orthogonality ablation. Tables 4–5 and the progressive configurations do the work they claim: data quantity is controlled, CoT helps, and RL mainly moves precision. The 1 000-rationale human check (95.2 % faithful, κ=0.81) and the public code link are real positives. Limitations section is honest about teacher dependence and GRPO cost.

Soft spots, in proportion. The stress-test note is right that EAR is hand-designed and never ablated on its thresholds or functional form; because ChERRANT is also edit-distance based, some of the precision surge could be metric-aligned rather than pure “minimal intervention.” They also never report human false-positive rates on clean text. That is a genuine gap in causal attribution, not a fatal flaw—the numerical gains and the additive pattern still stand. Free parameters (mixture ratio, reward cut-offs, N, β) are listed but not swept. Citation pattern is normal for the subfield; no circularity on the held-out benchmarks.

This is for people building Chinese GEC/CSC systems or anyone who cares about precision-oriented RL for editing. It deserves a serious referee. I would cite the numbers and the ablation design; I would not treat the EAR as a settled general principle without the missing sensitivity checks. Send it out.

Referee Report

3 major / 0 minor

Summary. The paper proposes CSRP, a three-stage pipeline for Chinese grammatical error correction and spelling check: (i) balanced continual pre-training on 5.9M samples (8:2 general-to-correction mix), (ii) rationale-augmented SFT with teacher-distilled CoT of the form Localization→Classification→Rationale, and (iii) GRPO policy alignment under a hand-designed Efficiency-Aware Reward (EAR) built from Levenshtein relative improvement (RI) and edit-efficiency ratio η (Eqs. 3–5), with special-case rewards for already-correct inputs. On NACGEC the 4B model reports 50.99 F0.5 and 57.17 precision (SOTA among listed baselines); on CSCD it reports 59.61 F1, +5.20 over GPT-4. Progressive ablations (Tables 4–5) attribute an ~8% relative F0.5 gain and +8.44 precision (near-zero recall cost) to the RL stage and argue that this gain is orthogonal to CPT. Code is released.

Significance. Over-correction under MLE is a genuine, practical bottleneck for LLM-based GEC; a curriculum that jointly injects domain priors, diagnostic CoT, and an explicit minimal-edit objective is a useful systems contribution. Strengths include: public code; a controlled SFT baseline that re-uses the RL data split (Table 4, “CPT + SFT (w/ RL data)”); isolation of CPT vs. RL via the “SFT + GRPO (w/o CPT)” row; a 1,000-sample double-blind CoT faithfulness study (95.2%, κ=0.81); and consistent precision–recall dynamics on two benchmarks. If the EAR truly operationalizes minimal intervention rather than gaming ChERRANT, the claimed orthogonality of knowledge internalization and edit-policy calibration would be a clear takeaway for the field.

major comments (3)

§3.3, Eqs. (3)–(5): The central causal claim—“explicit optimization for edit efficiency is essential”—rests on EAR, a piecewise function of Levenshtein RI/η with hard thresholds (RI>0.99→+10; RI>0→2+5·RI·η; RI≤0→−3) plus the binary +2/−2 rule when d(S,G)=0. Because the primary scorer (ChERRANT F0.5) is itself character-level edit-distance based, the +8.44 precision jump can be produced by metric-aligned gaming rather than a general reduction in over-correction. Tables 4–5 show that some reward of this family yields additive gains, but they never ablate functional form, numerical coefficients, or the special-case rule, nor compare against simpler rewards (e.g., pure F0.5 proxy, binary identity penalty only, or length-normalized edit cost). Without such controls the attribution to “edit efficiency” remains under-supported.
§4.2–4.3 / Table 5: Precision gains are reported only via automatic ChERRANT/CSC metrics. There is no human false-positive audit on already-correct native text (or on a held-out “no-error” subset) that would confirm the model truly preserves well-formed prose rather than suppressing edits that ChERRANT happens to penalize. A modest human FP/FN study on, e.g., 200–500 correct and incorrect sentences before vs. after GRPO would substantially strengthen the over-correction narrative and is load-bearing for the “minimal intervention” interpretation.
§4.2, Tables 2–5: All headline numbers are single-run point estimates with no seeds, confidence intervals, or significance tests. Given that the claimed SOTA margin over CEC3 is +2.25 F0.5 and the RL-only lift is +3.78 over the data-matched SFT baseline, variance estimates are needed to decide whether the ranking and the orthogonality claim are stable. At minimum, report 2–3 seeds for the SFT→GRPO transition on NACGEC.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: staged CPT/CoT-SFT/GRPO pipeline yields empirical held-out gains; EAR is a designed Levenshtein proxy, not a tautological rewrite of ChERRANT F0.5.

full rationale

The paper's central claims (NACGEC 50.99 F0.5 / 57.17 P; ~8% relative RL gain orthogonal to CPT; +8.44 precision with near-zero recall drop) are measured by running the final model on public held-out test sets (NACGEC 5.8k, CSCD 5k) using the independent ChERRANT scorer. Ablations (Table 4) explicitly control for data quantity by training an SFT baseline on the identical RL split and by isolating GRPO without CPT; the numerical improvements therefore cannot be forced by construction from the training inputs. The Efficiency-Aware Reward (Eqs. 3–5) is a hand-designed piecewise function of Levenshtein ratios RI and η relative to gold G on the RL prompts; it is optimized via GRPO and then evaluated on unseen data. While both reward and metric involve edit distance, this is ordinary reward-model alignment, not a self-definitional loop or fitted-parameter-as-prediction. No uniqueness theorem, self-citation chain, or ansatz is load-bearing for the reported numbers; prior self-citations (e.g., CEC3) appear only as baselines that are surpassed. The derivation chain is therefore self-contained and non-circular.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The central empirical claim rests on a small set of hand-chosen hyperparameters (mixture ratio, reward thresholds, GRPO group size) and on domain assumptions that Levenshtein distance and F0.5 correctly capture ‘minimal editing’ for Chinese. No new physical entities are postulated; the invented objects are the reward function and the three-stage curriculum itself.

free parameters (4)

CPT general-to-correction mixture ratio = 8:2
Fixed at 8:2 by design (inspired by prior domain-adaptation work) rather than swept; directly controls knowledge internalization vs. forgetting.
EAR reward thresholds and coefficients = 10 / 2+5RI·η / −3 / −2
Constants 10.0 (near-perfect RI), 2.0+5·RI·η, −3.0, −2.0 and the binary +2/−2 rule for already-correct inputs are chosen by hand to shape the policy.
GRPO group size N and KL coefficient β = N=8, β=0.01
N=8 candidates and β=0.01 are set without extensive sensitivity analysis (though N=4 is briefly noted to drop only 0.38 F0.5).
Levenshtein ε = 1e-6
Numerical stabilizer set to 10^{-6}.

assumptions (4)

domain assumption Levenshtein distance is an adequate proxy for edit cost and relative improvement in Chinese GEC/CSC.
Used to define both RI and η (Eq. 3–4); Chinese character-level morphology and multi-span edits may not be fully captured by pure string distance.
domain assumption F0.5 (precision-weighted) is the correct primary objective for native-speaker correction.
Stated throughout evaluation; under-correction is treated as less harmful than over-correction.
domain assumption Teacher-generated CoT rationales (after format filtering) are sufficiently faithful to supervise the student.
Supported by a 1 000-sample human study (Appendix G) but still an external dependency.
domain assumption The 20 % hold-out split used for GRPO is distributionally comparable to the SFT data and to the test sets.
Required for the claim that RL gains are not merely from extra supervised data.

invented entities (2)

Efficiency-Aware Reward (EAR)
purpose: Scalar reward that jointly scores relative Levenshtein improvement and edit efficiency, with special cases for near-perfect and already-correct inputs.
Defined in Eq. 5; no independent external validation beyond the paper’s own ablations.
CSRP three-stage curriculum
purpose: Ordered pipeline CPT → CoT-SFT → GRPO that is claimed to produce orthogonal gains.
The specific ordering and data splits are introduced by the paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of CSRP: Chain-of-Thought Reasoning for Chinese Text Correction via Reinforcement Learning with Efficiency-Aware Rewards." pith.science (2026). https://pith.science/paper/TXN65WMT

@misc{pith2026260600020,
  author       = {Pith},
  title        = {Pith review of: CSRP: Chain-of-Thought Reasoning for Chinese Text Correction via Reinforcement Learning with Efficiency-Aware Rewards},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TXN65WMT}},
  note         = {Machine review of arXiv:2606.00020}
}

abstract

Large Language Model (LLM) based Chinese Grammatical Error Correction (CGEC) systems face two critical challenges: general-purpose models lack specialized linguistic priors for subtle grammatical distinctions, and Supervised Fine-Tuning (SFT) with Maximum Likelihood Estimation fails to optimize for precision-focused metrics, leading to systematic over-correction. We propose CSRP, a three-stage framework that progressively builds correction capability through Continual Pre-training (CPT) on 5.9M balanced samples to internalize domain knowledge, Chain-of-Thought SFT with explicit error reasoning for diagnostic transparency, and Group Relative Policy Optimization with a novel Efficiency-Aware Reward that explicitly penalizes unnecessary edits. On the NACGEC benchmark, CSRP achieves state-of-the-art performance with 50.99 $F_{0.5}$ and 57.17 precision, substantially outperforming previous best results while effectively mitigating the over-correction bias inherent in MLE-trained models. Our method also advances CSCD spelling correction to 59.61 F1, surpassing GPT-4 by 5.20 points. Comprehensive ablation studies demonstrate that the RL alignment stage contributes a 8\% relative gain over the SFT baseline, and that this gain is orthogonal to the contribution of large-scale CPT, validating that explicit optimization for edit efficiency is essential for high-quality grammatical error correction. Our code is available at https://github.com/TW-NLP/ChineseErrorCorrector.

Figures

Figures reproduced from arXiv: 2606.00020 by the authors.

**Figure 2.** Overview of the proposed CSRP (CPT-SFT-RL) framework. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** CPT Data Processing Process. Despite these advancements, aligning LLMs with high-precision metrics like F0.5 remains challenging due to the non-differentiable nature of GEC objectives. Traditional SFT based on Maximum Likelihood Estimation (MLE) often fails to calibrate the “edit-or-not” decision boundary. Our framework addresses this by leveraging Group Relative Policy Optimization (GRPO) to explicitly optimize … view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Cot Processing Process. training insights in Wen et al. (2023), we employ an 8:2 ratio between general and correction-specific data. This translates to approximately 4.72M general samples (from wiki-zh-25 and cci2) and 1.18M correction samples (from lang8+HSK). The ob…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

7 extracted references · 2 linked inside Pith

[1]

Explanation based in-context demonstrations retrieval for multilingual grammatical error correc- tion. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 4881– 4897. Wei Li and Houfeng Wang. 2024. Detection-correction struc...

arXiv 2025
[2]

In2020 IEEE 6th Interna- tional Conference on Computer and Communications (ICCC), pages 1146–1150

Spelling error correction with bert based on character-phonetic. In2020 IEEE 6th Interna- tional Conference on Computer and Communications (ICCC), pages 1146–1150. IEEE. Wei Tian and 1 others. 2025. Chineseerrorcorrector3- 4b: State-of-the-art chinese spelling and grammar corrector.arXiv preprint arXiv:2511.17562. Yuen-Hsien Tseng, Lung-Hao Lee, Li-Ping C...

arXiv 2025
[3]

I today ate one ping-guo

Qwen3 technical report.arXiv preprint arXiv:2505.09388. Baolin Zhang. 2009. Features and functions of the hsk dynamic composition corpus.International Chinese Language Education, 4:71–79. Shaohua Zhang, Haoran Huang, Jicong Liu, and Hang Li. 2020. Spelling error correction with soft-masked bert. InProceedings of the 58th annual meeting of the association ...

arXiv 2009
[4]

Sampling:Extract the source sentence S from our correction corpus
[5]

Rationale Generation:Query Qwen-Plus with the instruction template and source sen- tence, obtaining a structured response R con- taining error analysis and correction
[6]

Validation:Verify that: (a) the generated ra- tionale follows the prescribed format, (b) the corrected output matches or is semantically equivalent to the gold reference G, and (c) the explanation is linguistically sound and non- trivial
[7]

The original sentence has errors, the corrected sentence fixes them

Integration:Integrate valid rationales into the training data, forming triplets (S, R, G) where R encodes the diagnostic reasoning path. E.5 Quality Control To ensure high-quality rationales, we implement three filtering mechanisms: • Format Compliance:Discard samples where the output does not adhere to the <think>...<\think> structure or lacks clear erro...

Pith tools

Reviewed July 12, 2026 · model on record in the stance chip above.

[1] [1]

Explanation based in-context demonstrations retrieval for multilingual grammatical error correc- tion. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 4881– 4897. Wei Li and Houfeng Wang. 2024. Detection-correction struc...

arXiv 2025

[2] [2]

In2020 IEEE 6th Interna- tional Conference on Computer and Communications (ICCC), pages 1146–1150

Spelling error correction with bert based on character-phonetic. In2020 IEEE 6th Interna- tional Conference on Computer and Communications (ICCC), pages 1146–1150. IEEE. Wei Tian and 1 others. 2025. Chineseerrorcorrector3- 4b: State-of-the-art chinese spelling and grammar corrector.arXiv preprint arXiv:2511.17562. Yuen-Hsien Tseng, Lung-Hao Lee, Li-Ping C...

arXiv 2025

[3] [3]

I today ate one ping-guo

Qwen3 technical report.arXiv preprint arXiv:2505.09388. Baolin Zhang. 2009. Features and functions of the hsk dynamic composition corpus.International Chinese Language Education, 4:71–79. Shaohua Zhang, Haoran Huang, Jicong Liu, and Hang Li. 2020. Spelling error correction with soft-masked bert. InProceedings of the 58th annual meeting of the association ...

arXiv 2009

[4] [4]

Sampling:Extract the source sentence S from our correction corpus

[5] [5]

Rationale Generation:Query Qwen-Plus with the instruction template and source sen- tence, obtaining a structured response R con- taining error analysis and correction

[6] [6]

Validation:Verify that: (a) the generated ra- tionale follows the prescribed format, (b) the corrected output matches or is semantically equivalent to the gold reference G, and (c) the explanation is linguistically sound and non- trivial

[7] [7]

The original sentence has errors, the corrected sentence fixes them

Integration:Integrate valid rationales into the training data, forming triplets (S, R, G) where R encodes the diagnostic reasoning path. E.5 Quality Control To ensure high-quality rationales, we implement three filtering mechanisms: • Format Compliance:Discard samples where the output does not adhere to the <think>...<\think> structure or lacks clear erro...