CEC-Zero: Chinese Error Correction Solution Based on LLM

Sophie Zhang; Zhiming Lin

REVIEW 5 major objections 6 minor 78 references

CEC-Zero: Chinese Error Correction Solution Based on LLM

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Reinforcement learning with an embedding-clustering reward lifts Chinese spelling correction accuracy of Qwen3 LLMs by about 13 F1 points on average.

desk verdict A plausible RL recipe for Chinese spelling correction with large claimed gains, but the paper omits the controls and details needed to trust them. read the letter →

arxiv 2505.09082 v1 pith:JME4MSDP submitted 2025-05-14 cs.CL cs.AI

Sophie Zhang , Zhiming Lin This is my paper · ORCID

classification cs.CLcs.AI

keywords Chinesespellingcorrectionreinforcementlearninglargelanguagemodelsembeddingclusteringrewardself-generatedtrainingdatacross-domaingeneralizationQwen3texterror

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes CEC-Zero, a reinforcement-learning framework that trains large language models to correct Chinese spelling errors without any human-annotated error data. Instead of collecting erroneous sentences, it generates them by automatically perturbing correct sentences using homophone, shape-similar, and structural edits, then rewards the LLM when its correction is semantically close to the original. The reward is computed from sentence embeddings: one term measures similarity to the known correct sentence, and a second term measures similarity to the center of a cluster of the model's own sampled outputs, which serves as a pseudo-label. On CSCD-NS, the seven-domain LEMON benchmark, and a customer-service test set, the authors report that RL training lifts the average F1 of Qwen3-14B from 52.16 to 65.14 and of Qwen3-32B from 55.47 to 68.15, surpassing the tested closed-source LLMs. The central claim is that convergence among model-generated answers is a usable reward signal in text correction, where multiple valid corrections exist for a single error sentence.

What carries the argument

The load-bearing mechanism is the embedding-clustering reward function. The model generates $l$ sampled corrections for a source sentence; their sentence embeddings are clustered by Euclidean distance, and the center of the largest tight cluster (containing more than one third of the batch) is treated as a pseudo-label. The final reward is $$\text{RLscore} = \$\alpha$ \cdot \max\left(0, \frac{\cos_1 - \$\theta$}{1 - \$\theta$}\right) + \gamma \cdot \max\left(0, \frac{\cos_2 - \$\beta$}{1 - \$\beta$}\right),$$ with defaults $\alpha = \gamma = 0.5$, $\theta = 0.8$, and $\beta = 0.85$. This reward substitutes for both ground-truth labels (RLscore1) and majority-vote labels (RLscore2), letting RL optimize for semantic correctness instead of exact symbol matching, which matters because a Chinese error sentence can have several acceptable corrections.

What would settle it

Train CEC-Zero with only homophone-based perturbations and evaluate on a test set whose errors are mainly visually similar characters; if the RL model's F1 on those errors stays near the base model's, the clustering reward is not generalizing beyond the perturbation distribution.

Watch

Extended reading notes

Core claim

The paper's central claim is that an LLM can be taught Chinese spelling correction by reinforcement learning alone, with no labeled error-correction pairs and no auxiliary validation model, provided the reward is built from semantic embeddings rather than exact string matches. The training data pairs are constructed by corrupting correct sentences with a diverse set of perturbation tools—homophone replacement, visually similar character replacement, merging and splitting of Chinese characters, and symbol insertion—so that the model never sees a real hand-annotated error during training. The reward has two components: RLscore1, the cosine similarity between the embedding of the model's prediction and the embedding of the known correct sentence, and RLscore2, the cosine similarity between the prediction and a pseudo-label formed by clustering the model's own sampled outputs and taking the center of the densest cluster; both are thresholded and linearly combined. The authors report that after RL, Qwen3-14B and Qwen3-32B reach average sentence-level F1 scores of 65.14 and 68.15 on their test collection, improving by roughly 13 points over the base models and exceeding all closed-source baselines tested, including GPT-4, Claude 3.7, and Doubao.

Load-bearing premise

The synthetic corruption of correct sentences produces training inputs whose error distribution matches the real-world errors seen in the test sets.

Editorial extensions

If this is right

Chinese spelling correction can be improved on open LLMs without annotated error corpora; only clean Chinese sentences are needed, since perturbation tools generate the errors.
Cross-domain generalization improves: the RL-trained models outperform all tested baselines on LEMON's seven domains in the zero-shot evaluation reported, including the customer-service set.
The same RL recipe should transfer to other text-generation tasks with multiple valid answers, because the reward is based on semantic similarity and answer clustering rather than task-specific rules.
Reliability of LLM output improves with no extra annotation cost and no auxiliary model, since the reward is computed from embeddings and the model's own sampled outputs.
The clustering-based pseudo-label acts as a self-consistency signal: outputs that agree with the model's own majority cluster are treated as correct, providing a practical substitute for verifiable rewards in non-math, non-code tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The authors leave implicit that the diversity of the perturbation tools is the main determinant of what the model learns; if real-world error streams contain paradigms absent from the perturbation set, the RL reward will not teach the model to fix them, regardless of the clustering signal.
A testable extension is to apply CEC-Zero to test-time adaptation: because the reward needs no labels, the same RL loop could be run on unlabeled text arriving at deployment time, something the paper motivates but does not experiment with.
If the base model's sampled outputs are confidently wrong, the pseudo-label center will be confidently wrong too, so the clustering term could reward self-consistency rather than correctness; the reported gains suggest this did not dominate on these benchmarks, but the failure mode is worth probing.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

CEC-Zero is an RL-based Chinese spelling correction method that reports large, consistent gains over base Qwen3 models (e.g., average F1 65.14 vs 52.16 for 14B). What's actually new is the specific combination: synthetic perturbation of correct sentences to build (X,Y) pairs, then a reinforcement learning reward that blends cosine similarity to the ground truth with a TTRL-style clustering score. The clustering score is a reasonable answer to the multiple-valid-corrections problem; the idea that convergent model outputs are likely correct is plausible, and the paper gamely attributes it to TTRL.

The empirical results are striking, but the paper doesn't support the attribution of those gains to RL. The most serious omission is the lack of an SFT-only baseline. Training on perturbed sentences with the same synthetic data, without RL, would tell us how much of the improvement is just supervised exposure to the error patterns. Absent that control, 'RL improves cross-domain generalization' is unverified. Second, the paper never specifies the RL algorithm or its hyperparameters, training data sizes, or evaluation variance. I can't verify a single number, and large gains without error bars are hard to assess. Third, the web training data contamination concern is real: the paper says it scraped web data from various domains, and LEMON is web-derived; without a holdout check for near-duplicates, the cross-domain numbers could reflect memorization.

The citation pattern is fine; TTRL and relevant CSC work are cited. The writing is rough, with duplicated sentences and vague phrasing, but that's a matter of revision, not a fatal flaw.

Who is this for? People working on Chinese spelling correction or RL reward design for text generation might find the formulation worth reading, but only after the experiments are made reproducible and the controls are added. As it stands, the central claim is unsupported. That said, the idea is not unserious, and the reported gains are large enough that I would not desk reject. I'd send it to reviewers and ask for major revision: add an SFT baseline, report the RL setup, run contamination checks, and release code/data.

Referee Report

5 major / 6 minor

Summary. The paper proposes CEC-Zero, a reinforcement learning (RL) framework for Chinese spelling correction that constructs training pairs by perturbing clean sentences and trains Qwen3-14B/32B with a reward combining cosine similarity to the ground-truth sentence and a clustering-based pseudo-label. Experiments on CSCD-NS, the seven-domain LEMON benchmark, and a customer-service test set report large average F1 gains (from 52.16 to 65.14 for the 14B model and from 55.47 to 68.15 for the 32B model). The central claims are that RL improves Chinese spelling correction and cross-domain generalization without external supervision.

Significance. If the reported results were robust, the embedding-clustering reward would be a useful contribution to Chinese text correction, and the idea of rewarding semantic agreement with a pseudo-label is worth exploring. The paper covers an important task and evaluates on a recent multi-domain benchmark, which is a strength. However, the evidentiary basis is currently insufficient: the reward uses the ground-truth Y, so the unsupervised framing is inaccurate; no SFT control isolates the effect of RL; and the additional web training data may overlap with the test domains. Machine-checked proofs, code, or parameter-free derivations are not provided, and the RL setup is too underspecified to reproduce. With additional controls and complete experimental detail, the contribution could be salvageable, but as submitted the central attribution claims are not supported.

major comments (5)

[§3.2, §3.3, Abstract] The reward RLscore1 is defined as the cosine similarity between the model output and the ground-truth Y, and every training pair (X,Y) is constructed by perturbing the correct sentence Y (Section 3.2, pseudocode in Section 3.3). The method therefore uses direct reference supervision, contradicting the abstract's claim that CEC-Zero works 'without external supervision' and 'eliminates dependency on annotated data.' The paper must clarify the sense in which the method is unsupervised, or revise the claim; as written, this is a mischaracterization of the central contribution.
[§4.3, Table 1] No supervised fine-tuning (SFT) control is reported. Because the training pipeline produces (X,Y) pairs, a standard SFT run on the same synthetic data would directly test whether the gains in Table 1 come from the RL objective or merely from exposure to correction pairs. Without this ablation, the claim that 'RL is able to improve the accuracy and cross-domain generalization' (Section 4.3) is not established. Please add an SFT-only baseline on the identical data, and also report the base model trained with the same number of steps for fair comparison.
[§4.1, Table 1] Section 4.1 states that 'we collected web data from various domains on the web and cleaned it as additional training data,' while LEMON is a web-derived multi-domain benchmark. No overlap or contamination analysis is provided. If near-duplicates of LEMON (or the customer-service) test sentences appear in the scraped web corpus, the cross-domain improvements in Table 1 (e.g., LEMON domains) could be due to memorization rather than to the proposed reward or perturbation pipeline. The authors should quantify n-gram overlap between training data and all test sets and, ideally, retrain after removing near-duplicates.
[§3.2, §4.3] The reinforcement learning setup is not reproducible as described. The paper never identifies the RL algorithm (PPO, GRPO, DPO, or other), nor does it report learning rate, batch size, number of training steps, KL penalty, rollout count, or generation temperature. For the clustering reward, the embedding model, distance metric, batch size l, and the criterion for the 'closest' subset are unspecified, and the equation for RLscore2 is not written precisely (the text says 'below the threshold beta' but then refers to 'theta'). These details are essential for evaluating the method.
[§2.1, §4.3] The paper states in Section 2.1 that 'we also conducted in-depth manual evaluations,' but no manual evaluation results are reported anywhere, and the conclusion's claim of 'consistent improvements' rests entirely on Table 1, which shows a single F1 number per model/domain without variance or significance tests. Either report the manual evaluation (with agreement statistics) and confidence intervals, or temper the corresponding claims.

minor comments (6)

[Section numbering] Section 3.2 appears twice; renumber the second occurrence (currently '3.2 The idea of clustered scoring' after '3.2 Theoretical approach').
[§3.2] In the RLscore2 paragraph, the default threshold is called 'theta' although the variable is 'beta' (default 0.85); use consistent notation.
[§3.3, §4.3] The manuscript contains placeholder text instead of figures, e.g., '(A case of error correction...)' and 'Training pipline flowchart!'; the referenced figures are missing.
[Pseudocode, Table 1] Typographical errors: 'datadata' and 'funtions' in the pseudocode, 'Gmini 2.5' for Gemini, and 'heteromorphic' (likely 'heterographic').
[References] Reference [10] is missing from the numbered list, and some entries (e.g., [4], [49]) are bare URLs; please complete the bibliography.
[§4.1, Table 1] Table 1 does not state whether the reported F1 is detection-level or correction-level, sentence-level or character-level, despite Section 4.1 saying both are computed; clarify the metric in the caption and text.

Circularity Check

1 steps flagged · score 5.0 of 10

Answer-clustering reward is self-referential: pseudo-label is the model's own output average, so half the final reward reduces to self-consistency.

self definitional [Section III, 3.2 Theoretical approach (RLscore2 / final reward)]
"we introduce the same mechanism to generate a batch of answers, and obtain a pseudo-answer based on the proximity of the spatial distance of the vectors within the answer to the embedding representation. ... We write e_center = avg(ex1,ex2,.... ,exi) to denote the center point of the set, i.e., the vector space representation of the pseudo-label. ... we use the formula RLscore2 = max(0,(cos2-beta)/(1-beta)) ... For the final reward function, we use RLscore = alpha*RLscore1 + gamma*RLscore2"

The pseudo-label e_center is computed as the average embedding of a subset of the model's own sampled outputs for the same input X, and RLscore2 then rewards each output for being close to that self-derived center. By construction, maximizing RLscore2 only makes the model's outputs more similar to their own average; it does not compare to the true sentence Y or to any externally verified target. Because the paper calls this answer-clustering reward a key component and includes it with default weight gamma=0.5 in the final reward, half of the learning signal reduces to a self-consistency loop rather than independent evidence of correction accuracy.

full rationale

The synthetic-data pipeline is not circular: correct sentences Y are perturbed to create X, so (X,Y) pairs carry an external target, and RLscore1 compares the model output to the known Y. The claimed total absence of external supervision is an overstatement, but it is not a definitional circularity. The load-bearing circularity is in the second reward term: RLscore2 defines its pseudo-label as the average of the model's own output embeddings and then rewards outputs for matching that average, making that component self-referential. The web-data collection and LEMON evaluation could raise a contamination concern, but the paper provides no concrete evidence of overlap, so I do not count it as demonstrated circularity. Overall, because one of the two equally weighted reward components is self-defined while the other is external, the circularity is partial rather than total.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method rests on hand-chosen reward thresholds and on assumptions about synthetic data realism and embedding similarity as a reward proxy. No formal derivation grounds these choices.

free parameters (5)

theta = 0.8 (default)
Threshold for cosine similarity reward RLscore1; chosen by hand.
beta = 0.85 (default)
Threshold for clustering-based reward RLscore2; chosen by hand.
alpha = 0.5 (default)
Weight for RLscore1 in final reward.
gamma = 0.5 (default)
Weight for RLscore2 in final reward.
clustering density threshold = 1/3 of batch size
Minimum number of nearby vectors to form the pseudo-label cluster.

assumptions (4)

domain assumption Synthetic perturbations of correct sentences produce error patterns representative of real-world Chinese spelling errors.
Section 3.1 and 3.2 rely on this to train on generated (X,Y) pairs; if the perturbation distribution is unrealistic, the reward signal is misaligned with actual corrections.
domain assumption Cosine similarity between sentence embeddings is a valid measure of correction quality for reward.
The reward functions RLscore1 and RLscore2 are defined on embedding cosine similarity (Section 3.2).
domain assumption Convergent model outputs are more likely to be correct, so the clustering pseudo-label approximates the true ground truth.
Section 3.2 'clustered scoring' states this premise.
domain assumption Training web data collected from various domains does not overlap with LEMON test domains, preserving zero-shot claims.
The paper does not report domain composition of the additional web training data; if it includes LEMON-like texts, the cross-domain result is confounded.

how reviews work

0 comments

Cite this review

Pith. "Pith review of CEC-Zero: Chinese Error Correction Solution Based on LLM." pith.science (2026). https://pith.science/paper/JME4MSDP

@misc{pith2026250509082,
  author       = {Pith},
  title        = {Pith review of: CEC-Zero: Chinese Error Correction Solution Based on LLM},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JME4MSDP}},
  note         = {Machine review of arXiv:2505.09082}
}

read the original abstract

Recent advancements in large language models (LLMs) demonstrate exceptional Chinese text processing capabilities, particularly in Chinese Spelling Correction (CSC). While LLMs outperform traditional BERT-based models in accuracy and robustness, challenges persist in reliability and generalization. This paper proposes CEC-Zero, a novel reinforcement learning (RL) framework enabling LLMs to self-correct through autonomous error strategy learning without external supervision. By integrating RL with LLMs' generative power, the method eliminates dependency on annotated data or auxiliary models. Experiments reveal RL-enhanced LLMs achieve industry-viable accuracy and superior cross-domain generalization, offering a scalable solution for reliability optimization in Chinese NLP applications. This breakthrough facilitates LLM deployment in practical Chinese text correction scenarios while establishing a new paradigm for self-improving language models.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

78 extracted references · 34 canonical work pages

[1]

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al
[2]

Theoretically, using manual annotation is a perfect solution, but it is too idealized and costly

The evaluation of utterance error correction is actually a very complex process, and there needs to be a balance between efficiency and accuracy to find a stable and efficient evaluation method. Theoretically, using manual annotation is a perfect solution, but it is too idealized and costly. And while simple word-by-word comparisons possess good engineeri...

work page
[3]

Existing datasets tend to be more fragile and the training data is cleaned too much, which is a gap with our original intention - to design a modeling service that has the value of being applied in industry, we need to address the dataset issues, which include diversity, complexity, data volume, etc. 6.2. The premise that RL can be widely used is that it ...

work page
[4]

https://github.com/ByteDance-Seed/Seed-Thinking-v1.5/blob/main/seed-thinking-v1.5.pdf

work page
[5]

Aghajanyan, A.; Shrivastava, A.; Gupta, A.; Goyal, N.; Zettlemoyer, L.; and Gupta, S

work page
[6]

Scaling llm test-time compute optimally can be more effective than scaling model parameters

Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314,2024

arXiv 2024
[7]

Understanding r1-zero-like training: A critical perspective

Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783, 2025b

arXiv
[8]

Welcome to the era of experience

David Silver and Richard S Sutton. Welcome to the era of experience. Google AI, 2025

2025

Show all 78 references

[9]

Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre, and John Jumper. 2023a. Accelerating large language model decoding with speculative sampling. arXiv preprint arXiv:2302.01318

arXiv
[11]

Cheng, X.; Xu, W.; Chen, K.; Jiang, S.; Wang, F.; Wang, T.; Chu, W.; and Qi, Y. 2020. SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check. In Jurafsky, D.; Chai, J.; Schluter, N.; and Tetreault, J. R.,eds., Proceedings o...

2020
[12]

Guo, Z.; Ni, Y.; Wang, K.; Zhu, W.; and Xie, G. 2021. Global Attention Decoder for Chinese Spelling Error Correction. In Zong, C.; Xia, F.; Li, W.; and Navigli, R., eds., Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, Online Event, August 1- 6, 202...

2021
[13]

Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

2025 arXiv
[14]

Process reinforcement through implicit rewards

Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, et al. Process reinforcement through implicit rewards. arXiv preprint arXiv:2502.01456, 2025

2025 arXiv
[15]

He, P.; Gao, J.; and Chen, W. 2023. DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with GradientDisentangled Embedding Sharing. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net

2023
[16]

Reward-augmented decoding: Efficient controlled text generation with a unidirectional reward model

Haikang Deng and Colin Raffel. Reward-augmented decoding: Efficient controlled text generation with a unidirectional reward model. arXiv preprint arXiv:2310.09520, 2023

2023 arXiv
[17]

Haoran Wu, Wenxuan Wang, Yuxuan Wan, Wenxiang Jiao, and Michael R. Lyu. 2023a. Chatgpt or grammarly? evaluating chatgpt on grammatical error correction benchmark. CoRR, abs/2303.13648

arXiv
[18]

J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W

Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W
[19]

Heng-Da Xu, Zhongli Li, Qingyu Zhou, Chao Li, Zizhen Wang, Yunbo Cao, Heyan Huang, and XianLing Mao. 2021. Read, listen, and see: Leveraging multimodal information helps Chinese spell checking.In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages...

2021
[20]

Hongqiu Wu, Shaohua Zhang, Yuchen Zhang, and Hai Zhao. 2023b. Rethinking masked language modeling for chinese spelling correction. arXiv preprint arXiv:2305.17721

arXiv
[21]

iahao Li, Quan Wang, Zhendong Mao, Junbo Guo, Yanyan Yang, and Yongdong Zhang. 2022. Improving chinese spelling check by character pronunciation prediction: the effects of adaptivity and granularity. arXiv preprint arXiv:2210.10996

2022 arXiv
[22]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre- training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Huma...

2019
[23]

Huang, L.; Li, J.; Jiang, W.; Zhang, Z.; Chen, M.; Wang, S.; and Xiao, J. 2021. PHMOSpell: Phonological and Morphological Knowledge Guided Chinese Spelling Check. In Zong, C.; Xia, F.; Li, W.; and Navigli, R., eds., Proceedings of the 59th Annual Meeting of the Association for...

2021
[24]

Let’s verify step by step

Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In The Twelfth International Conference on Learning Representations, 2023

2023
[25]

Open rlhf: An easy-to-use, scalable and high-performance rlhf framework

Jian Hu, Xibin Wu, Zilin Zhu, Xianyu, Weixun Wang, Dehao Zhang, and Yu Cao. Open rlhf: An easy-to-use, scalable and high-performance rlhf framework. arXiv preprint arXiv:2405.11143, 2024

2024 arXiv
[26]

Genprm: Scaling test-time compute of process reward models via generative reasoning

Jian Zhao, Runze Liu, Kaiyan Zhang, Zhimu Zhou, Junqi Gao, Dong Li, Jiafei Lyu, Zhouyi Qian, Biqing Qi, Xiu Li, et al. Genprm: Scaling test-time compute of process reward models via generative reasoning. arXiv preprint arXiv:2504.00891, 2025

2025 arXiv
[27]

Scaling laws for neural language models

Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020

2001 arXiv
[28]

Jiaan Wang, Yunlong Liang, Fandong Meng, Haoxiang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu, and Jie Zhou. 2023. Is chatgpt a good nlg evaluator? a preliminary study. arXiv preprint arXiv:2303.04048

2023 arXiv
[29]

Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609

2023 arXiv
[30]

Proximal policy optimization algorithms

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

2017 arXiv
[31]

Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model

Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model. arXiv preprint arXiv:2503.24290, 2025a

arXiv
[32]

Reinforcement learning with perturbed rewards

Jingkang Wang, Yang Liu, and Bo Li. Reinforcement learning with perturbed rewards. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 6202–6209, 2020

2020
[33]

Li, C.; Zhang, C.; Zheng, X.; and Huang, X. 2021. Exploration and Exploitation: Two Ways to Improve Chinese Spelling Correction Models. In Zong, C.; Xia, F.; Li, W.; and Navigli, R., eds., Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics ...

2021
[34]

Liu, S.; Yang, T.; Yue, T.; Zhang, F.; and Wang, D. 2021a. PLOME: Pre-training with Misspelled Knowledge for Chinese Spelling Correction. In Zong, C.; Xia, F.; Li, W.; and Navigli, R., eds., Proceedings of the 59th Annual Meeting of the Association for Computational Linguistic...

2021
[35]

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al
[36]

Args: Alignment as reward-guided search

Maxim Khanov, Jirayu Burapacheep, and Yixuan Li. Args: Alignment as reward-guided search. arXiv preprint arXiv:2402.01694, 2024

2024 arXiv
[37]

Lester, B.; Al-Rfou, R.; and Constant, N. 2021. The Power of Scale for Parameter- Efficient Prompt Tuning. In Moens, M.; Huang, X.; Specia, L.; and Yih, S. W., eds., Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Eve...

2021
[38]

What makes a reward model a good teacher? an optimization perspective

Noam Razin, Zixuan Wang, Hubert Strauss, Stanley Wei, Jason D Lee, and Sanjeev Arora. What makes a reward model a good teacher? an optimization perspective. arXiv preprint arXiv:2503.15477, 2025

2025
[39]

OpenAI. 2023. GPT-4 Technical Report. CoRR, abs/2303.08774. Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.; Lacroix, T.; Roziere, B.; Goyal, N.; Hambro, E.; Azhar, F.; Rodriguez, A.; Joulin, A.; Grave, E.; and Lample, G. 2023. LLaMA: Open and Efficient Foundat...

2023 arXiv
[40]

Liu, X.; Zheng, Y.; Du, Z.; Ding, M.; Qian, Y.; Yang, Z.; and Tang, J. 2021b. GPT Understands, Too. CoRR, abs/2103.10385. Liu, Y.; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; Levy, O.; Lewis, M.; Zettlemoyer, L.; and Stoyanov, V. 2019. RoBERTa:A Robustly Optimized BERT Pr...

2019 arXiv
[41]

Piji Li and Shuming Shi. 2021. Tail-to-tail non autoregressive sequence prediction for chinese grammatical error correction. arXiv preprint arXiv:2106.01609

2021 arXiv
[42]

Mutian He and Philip N Garner. 2023. Can chatgpt detect intent? evaluating large language models for spoken language understanding. arXiv preprint arXiv:2305.13512

2023 arXiv
[43]

What, how, where, and how well? a survey on test-time scaling in large language models

Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Zhihan Guo, Yufei Wang, Irwin King, Xue Liu, and Chen Ma. What, how, where, and how well? a survey on test-time scaling in large language models. arXiv preprint arXiv:2503.24235, 2025b

arXiv
[44]

Reinforcement learning: An introduction, volume 1.MIT press Cambridge, 1998

Richard S Sutton, Andrew G Barto, et al. Reinforcement learning: An introduction, volume 1.MIT press Cambridge, 1998

1998
[45]

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Kü ttler, Mike Lewis, Wen-tau Yih, Tim Rocktä schel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing...

2020
[46]

Shaohua Zhang, Haoran Huang, Jicong Liu, and Hang Li. 2020. Spelling error correction with soft-masked BERT. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 882–890, Online. Association for Computational Linguistics

2020
[47]

Dapo: An open-source llm reinforcement learning system at scale

Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476, 2025

2025 arXiv
[48]

Wong, Jinpeng Hu, Lidia S

Tao Fang, Shu Yang, Kaixin Lan, Derek F. Wong, Jinpeng Hu, Lidia S. Chao, and Yue Zhang. 2023. Is chatgpt a highly fluent grammatical error correction system? A comprehensive evaluation. CoRR,abs/2304.01746

2023 arXiv
[49]

https://github.com/QwenLM/Qwen3/blob/main/Qwen3_Technical_Report.pdf
[50]

Ruiqing Zhang, Chao Pang, Chuanqiang Zhang, Shuohuan Wang, Zhongjun He, Yu Sun, Hua Wu, and Haifeng Wang. 2021. Correcting chinese spelling errors with phonetic pre-training. In Findings of the Association for Computational Linguistics: ACLIJCNLP 2021, pages 2250–2261

2021
[51]

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901

2020
[52]

Shulin Liu, Tao Yang, Tianchi Yue, Feng Zhang, and Di Wang. 2021. PLOME: Pre- training with misspelled knowledge for Chinese spelling correction.In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conferen...

2021
[53]

Inference-time scaling for complex tasks: Where we stand and what lies ahead

Vidhisha Balachandran, Jingya Chen, Lingjiao Chen, Shivam Garg, Neel Joshi, Yash Lara, John Langford, Besmira Nushi, Vibhav Vineet, Yue Wu, et al. Inference-time scaling for complex tasks: Where we stand and what lies ahead. arXiv preprint arXiv:2504.00294,2025

2025 arXiv
[54]

Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223

2023 arXiv
[55]

Sft memorizes, rl generalizes: A comparative study of foundation model post-training

Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. Sft memorizes, rl generalizes: A comparative study of foundation model post-training. arXiv preprint arXiv:2501.17161, 2025

2025 arXiv
[56]

One-minute video generation with test-time training

Karan Dalal, Daniel Koceja, Gashon Hussein, Jiarui Xu, Yue Zhao, Youjin Song, Shihao Han, Ka Chun Cheung, Jan Kautz, Carlos Guestrin, et al. One-minute video generation with test-time training. arXiv preprint arXiv:2504.05298, 2025

2025 arXiv
[57]

Tseng, Y.; Lee, L.; Chang, L.; and Chen, H. 2015. Introduction to SIGHAN 2015 Bake- off for Chinese Spelling Check. In Yu, L.; Sui, Z.; Zhang, Y.; and Ng, V., eds., Proceedings of the Eighth SIGHAN Workshop on Chinese Language Processing, SIGHAN@IJCNLP 2015, Beijing, China, Ju...

2015
[58]

Universal self-consistency for large language model generation

Xinyun Chen, Renat Aksitov, Uri Alon, Jie Ren, Kefan Xiao, Pengcheng Yin, Sushant Prakash, Charles Sutton, Xuezhi Wang, and Denny Zhou. Universal self-consistency for large language model generation. arXiv preprint arXiv:2311.17311, 2023

2023 arXiv
[59]

Limr: Less is more for rl scaling

Xuefeng Li, Haoyang Zou, and Pengfei Liu. Limr: Less is more for rl scaling. arXiv preprint arXiv:2502.11886, 2025

2025 arXiv
[60]

TTRL: Test-Time Reinforcement Learning

Yuxin Zuo, Kaiyan Zhang, Shang Qu, Li Sheng, Xuekai Zhu, Biqing Qi, Youbang Sun, Ganqu Cui, Ning Ding, Bowen Zhou. TTRL: Test-Time Reinforcement Learning. arXiv preprint arXiv:2504.16084, 2025

2025 arXiv
[61]

Xunjian Yin and Xiaojun Wan. 2023. A comprehensive evaluation and analysis study for chinese spelling check. arXiv preprint arXiv:2307.13655

2023 arXiv
[62]

Wu, H.; Zhang, S.; Zhang, Y.; and Zhao, H. 2023c. Rethinking Masked Language Modeling for Chinese Spelling Correction. In Rogers, A.; Boyd-Graber, J. L.; and Okazaki, N., eds., Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: L...

2023
[63]

Yinghui Li, Haojing Huang, Shirong Ma, Yong Jiang, Yangning Li, Feng Zhou, Hai-Tao Zheng, and Qingyu Zhou. 2023a. On the (in) effectiveness of large language models for chinese text correction. arXiv preprint arXiv:2307.09007

arXiv
[64]

Yong Hu, Fandong Meng, and Jie Zhou. 2022. Cscdime: correcting spelling errors generated by pinyin ime. arXiv preprint arXiv:2211.08788

2022 arXiv
[65]

Self-consistency improves chain of thought reasoning in language models

Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171, 2022

2022 arXiv
[66]

Monte Carlo tree search boosts reasoning via iterative preference learning

Yuxi Xie, Anirudh Goyal, Wenyue Zheng, Min-Yen Kan, Timothy P Lillicrap, Kenji Kawaguchi, and Michael Shieh. Monte Carlo tree search boosts reasoning via iterative preference learning. arXiv preprint arXiv:2405.00451, 2024

2024 arXiv
[67]

Testtime training for out-of-distribution generalization

Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei A Efros, and Moritz Hardt. Testtime training for out-of-distribution generalization. Arxiv, 2019

2019
[68]

Zhang, R.; Pang, C.; Zhang, C.; Wang, S.; He, Z.; Sun, Y.; Wu, H.; and Wang, H. 2021. Correcting Chinese Spelling Errors with Phonetic Pre-training. In Zong, C.; Xia, F.; Li, W.; and Navigli, R., eds., Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021,...

2021
[69]

Zhang, S.; Huang, H.; Liu, J.; and Li, H. 2020. Spelling Error Correction with Soft- Masked BERT. In Jurafsky, D.; Chai, J.; Schluter,N.; and Tetreault, J. R., eds., Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July...

2020
[70]

Yue Zhang, Zhenghua Li, Zuyi Bao, Jiacheng Li, Bo Zhang, Chen Li, Fei Huang, and Min Zhang. 2022. MuCGEC: a multi-reference multi-source evaluation dataset for Chinese grammatical error correction. In Proceedings of the 2022 Conference of the North American Chapter of the Asso...

2022
[71]

Zhu, C.; Ying, Z.; Zhang, B.; and Mao, F. 2022. MDCSpell: A Multi-task Detector- Corrector Framework for Chinese Spelling Correction. In Muresan, S.; Nakov, P.; and Villavicencio, A., eds., Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland, M...

2022
[72]

Swe-rl: Advancing llm reasoning via reinforcement learning on open software evolution

Yuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux, Lingming Zhang, Daniel Fried, Gabriel Synnaeve, Rishabh Singh, and Sida I Wang. Swe-rl: Advancing llm reasoning via reinforcement learning on open software evolution. arXiv preprint arXiv:2502.18449, 2025b

arXiv
[73]

Competitive programming with large reasoning models

Ahmed El-Kishky, Alexander Wei, Andre Saraiva, Borys Minaev, Daniel Selsam, David Dohan, Francis Song, Hunter Lightman, Ignasi Clavera, Jakub Pachocki, et al. Competitive programming with large reasoning models. arXiv preprint arXiv:2502.06807, 2025

2025 arXiv
[76]

Zhao Guo, Yuan Ni, Keqiang Wang, Wei Zhu, and Guotong Xie. 2021. Global attention decoder for chinese spelling error correction. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1419–1428

2021
[78]

Zihong Liang, Xiaojun Quan, and Qifan Wang. 2023. Disentangled phonetic representation for chinese spelling correction. arXiv preprint arXiv:2305.14783

2023 arXiv
[2019]

OpenAI blog, 1(8):9

Language models are unsupervised multitask learners. OpenAI blog, 1(8):9
[2021]

In the 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021

Better Fine-Tuning by Reducing Representational Collapse. In the 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net

2021
[2022]

In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022

LoRA: Low-Rank Adaptation of Large Language Models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net

2022
[2261]

Association for Computational Linguistics

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al

[2] [2]

Theoretically, using manual annotation is a perfect solution, but it is too idealized and costly

The evaluation of utterance error correction is actually a very complex process, and there needs to be a balance between efficiency and accuracy to find a stable and efficient evaluation method. Theoretically, using manual annotation is a perfect solution, but it is too idealized and costly. And while simple word-by-word comparisons possess good engineeri...

work page

[3] [3]

Existing datasets tend to be more fragile and the training data is cleaned too much, which is a gap with our original intention - to design a modeling service that has the value of being applied in industry, we need to address the dataset issues, which include diversity, complexity, data volume, etc. 6.2. The premise that RL can be widely used is that it ...

work page

[4] [4]

https://github.com/ByteDance-Seed/Seed-Thinking-v1.5/blob/main/seed-thinking-v1.5.pdf

work page

[5] [5]

Aghajanyan, A.; Shrivastava, A.; Gupta, A.; Goyal, N.; Zettlemoyer, L.; and Gupta, S

work page

[6] [6]

Scaling llm test-time compute optimally can be more effective than scaling model parameters

Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314,2024

arXiv 2024

[7] [7]

Understanding r1-zero-like training: A critical perspective

Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783, 2025b

arXiv

[8] [8]

Welcome to the era of experience

David Silver and Richard S Sutton. Welcome to the era of experience. Google AI, 2025

2025