{"id":"84770ea0-02f3-44c9-a3f3-7292ebd9bec5","arxiv_id":"2505.09082","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"The authors claim that reinforcement learning with an embedding-clustering reward improves Chinese spelling correction and cross-domain generalization, but the evidence is missing key baselines and reproducibility artifacts.","lead":"A Chinese spelling correction system trains Qwen language models with reinforcement learning on automatically perturbed sentences, reporting large F1 gains over zero-shot baselines. The paper lacks the control experiments, code, and details needed to verify that reinforcement learning, rather than simply training on the target domain, drives the improvement.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported cross-domain gains may stem from web-data contamination rather than the RL reward; no holdout check or ablation is provided to rule this out.","rationale":"The reader's weakest assumption focuses on the synthetic corruption distribution, which is a reasonable concern but not the most decisive one: even a perfectly realistic corruption distribution would not validate the central claim if the evaluation data overlaps the undisclosed web pretraining corpus, because the reported cross-domain F1 gains could then reflect memorization. The paper itself discloses the web data collection in Section 4.1 and never reports a contamination check, so the concern is grounded in the manuscript text, not in speculation. This is correctness risk, not a stylistic objection or a mere disagreement with consensus. The no-SFT-baseline issue is also serious and independently weakens the attribution to RL, so the REJECT verdict remains appropriate. I did not find a formal or internal-inconsistency problem that would make the paper's construction invalid as far as it goes; the problem is that the central claim is not verifiable as stated. Hence the recommendation is UNCHANGED relative to the reader's REJECT verdict.","tokens_in":13662,"tokens_out":1089,"duration_ms":12614,"concrete_test":"Run near-duplicate detection (e.g., char-level n-gram overlap or MinHash) between the LEMON test sets, the CS test set, and the authors' disclosed web training corpus; if any LEMON/CS test sentence has a near-duplicate in the web corpus, recompute Table 1 after removing those overlapping items. Separately, train Qwen3-14B and Qwen3-32B on the same synthetic (X, Y) pairs with ordinary SFT (or continued pretraining) for the same number of steps, without the clustering reward; if the SFT baseline matches or exceeds the reported RL numbers, the RL attribution in the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.1 states that, in addition to CSCD-NS and customer-service data, the authors 'collected web data from various domains on the web and cleaned it as additional training data.' LEMON is itself a web-derived, multi-domain benchmark spanning games, encyclopedias, contracts, medical, automotive, novels, and news, with test sentences that could plausibly appear in the scraped web corpus. If near-duplicates of LEMON or CS test sentences occur in that web data, the reported improvements in Table 1 (e.g., 65.14 vs. 52.16 average F1) could reflect memorization of corrected patterns rather than the embedding-clustering reward or the synthetic corruption pipeline. This is the most load-bearing concern because the central claim is that RL improves cross-domain generalization, and the cross-domain results are exactly the numbers most vulnerable to this confound. A second load-bearing gap is the absence of an SFT-only ablation: since the method adds both large synthetic supervised pairs and an RL objective, the attribution of gains to RL per se is not established even if contamination is ruled out.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CEC-Zero, a reinforcement learning (RL) framework for Chinese spelling correction that constructs training pairs by perturbing clean sentences and trains Qwen3-14B/32B with a reward combining cosine similarity to the ground-truth sentence and a clustering-based pseudo-label. Experiments on CSCD-NS, the seven-domain LEMON benchmark, and a customer-service test set report large average F1 gains (from 52.16 to 65.14 for the 14B model and from 55.47 to 68.15 for the 32B model). The central claims are that RL improves Chinese spelling correction and cross-domain generalization without external supervision.","tokens_in":13986,"tokens_out":6377,"duration_ms":57871,"significance":"If the reported results were robust, the embedding-clustering reward would be a useful contribution to Chinese text correction, and the idea of rewarding semantic agreement with a pseudo-label is worth exploring. The paper covers an important task and evaluates on a recent multi-domain benchmark, which is a strength. However, the evidentiary basis is currently insufficient: the reward uses the ground-truth Y, so the unsupervised framing is inaccurate; no SFT control isolates the effect of RL; and the additional web training data may overlap with the test domains. Machine-checked proofs, code, or parameter-free derivations are not provided, and the RL setup is too underspecified to reproduce. With additional controls and complete experimental detail, the contribution could be salvageable, but as submitted the central attribution claims are not supported.","major_comments":[{"comment":"The reward RLscore1 is defined as the cosine similarity between the model output and the ground-truth Y, and every training pair (X,Y) is constructed by perturbing the correct sentence Y (Section 3.2, pseudocode in Section 3.3). The method therefore uses direct reference supervision, contradicting the abstract's claim that CEC-Zero works 'without external supervision' and 'eliminates dependency on annotated data.' The paper must clarify the sense in which the method is unsupervised, or revise the claim; as written, this is a mischaracterization of the central contribution.","section":"§3.2, §3.3, Abstract"},{"comment":"No supervised fine-tuning (SFT) control is reported. Because the training pipeline produces (X,Y) pairs, a standard SFT run on the same synthetic data would directly test whether the gains in Table 1 come from the RL objective or merely from exposure to correction pairs. Without this ablation, the claim that 'RL is able to improve the accuracy and cross-domain generalization' (Section 4.3) is not established. Please add an SFT-only baseline on the identical data, and also report the base model trained with the same number of steps for fair comparison.","section":"§4.3, Table 1"},{"comment":"Section 4.1 states that 'we collected web data from various domains on the web and cleaned it as additional training data,' while LEMON is a web-derived multi-domain benchmark. No overlap or contamination analysis is provided. If near-duplicates of LEMON (or the customer-service) test sentences appear in the scraped web corpus, the cross-domain improvements in Table 1 (e.g., LEMON domains) could be due to memorization rather than to the proposed reward or perturbation pipeline. The authors should quantify n-gram overlap between training data and all test sets and, ideally, retrain after removing near-duplicates.","section":"§4.1, Table 1"},{"comment":"The reinforcement learning setup is not reproducible as described. The paper never identifies the RL algorithm (PPO, GRPO, DPO, or other), nor does it report learning rate, batch size, number of training steps, KL penalty, rollout count, or generation temperature. For the clustering reward, the embedding model, distance metric, batch size l, and the criterion for the 'closest' subset are unspecified, and the equation for RLscore2 is not written precisely (the text says 'below the threshold beta' but then refers to 'theta'). These details are essential for evaluating the method.","section":"§3.2, §4.3"},{"comment":"The paper states in Section 2.1 that 'we also conducted in-depth manual evaluations,' but no manual evaluation results are reported anywhere, and the conclusion's claim of 'consistent improvements' rests entirely on Table 1, which shows a single F1 number per model/domain without variance or significance tests. Either report the manual evaluation (with agreement statistics) and confidence intervals, or temper the corresponding claims.","section":"§2.1, §4.3"}],"minor_comments":[{"comment":"Section 3.2 appears twice; renumber the second occurrence (currently '3.2 The idea of clustered scoring' after '3.2 Theoretical approach').","section":"Section numbering"},{"comment":"In the RLscore2 paragraph, the default threshold is called 'theta' although the variable is 'beta' (default 0.85); use consistent notation.","section":"§3.2"},{"comment":"The manuscript contains placeholder text instead of figures, e.g., '(A case of error correction...)' and 'Training pipline flowchart!'; the referenced figures are missing.","section":"§3.3, §4.3"},{"comment":"Typographical errors: 'datadata' and 'funtions' in the pseudocode, 'Gmini 2.5' for Gemini, and 'heteromorphic' (likely 'heterographic').","section":"Pseudocode, Table 1"},{"comment":"Reference [10] is missing from the numbered list, and some entries (e.g., [4], [49]) are bare URLs; please complete the bibliography.","section":"References"},{"comment":"Table 1 does not state whether the reported F1 is detection-level or correction-level, sentence-level or character-level, despite Section 4.1 saying both are computed; clarify the metric in the caption and text.","section":"§4.1, Table 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is an early-stage draft with extensive exposition issues, and the missing experimental controls are substantial. I would not rule out the method's potential, but the revision requires new experiments (SFT baseline, contamination analysis, RL details) rather than copy-editing. The claim of 'without external supervision' must be corrected. Given the scope of the revision, the editor may wish to treat the submission as a new paper if the authors cannot provide the controls."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nCEC-Zero is an RL-based Chinese spelling correction method that reports large, consistent gains over base Qwen3 models (e.g., average F1 65.14 vs 52.16 for 14B). What's actually new is the specific combination: synthetic perturbation of correct sentences to build (X,Y) pairs, then a reinforcement learning reward that blends cosine similarity to the ground truth with a TTRL-style clustering score. The clustering score is a reasonable answer to the multiple-valid-corrections problem; the idea that convergent model outputs are likely correct is plausible, and the paper gamely attributes it to TTRL.\n\nThe empirical results are striking, but the paper doesn't support the attribution of those gains to RL. The most serious omission is the lack of an SFT-only baseline. Training on perturbed sentences with the same synthetic data, without RL, would tell us how much of the improvement is just supervised exposure to the error patterns. Absent that control, 'RL improves cross-domain generalization' is unverified. Second, the paper never specifies the RL algorithm or its hyperparameters, training data sizes, or evaluation variance. I can't verify a single number, and large gains without error bars are hard to assess. Third, the web training data contamination concern is real: the paper says it scraped web data from various domains, and LEMON is web-derived; without a holdout check for near-duplicates, the cross-domain numbers could reflect memorization.\n\nThe citation pattern is fine; TTRL and relevant CSC work are cited. The writing is rough, with duplicated sentences and vague phrasing, but that's a matter of revision, not a fatal flaw.\n\nWho is this for? People working on Chinese spelling correction or RL reward design for text generation might find the formulation worth reading, but only after the experiments are made reproducible and the controls are added. As it stands, the central claim is unsupported. That said, the idea is not unserious, and the reported gains are large enough that I would not desk reject. I'd send it to reviewers and ask for major revision: add an SFT baseline, report the RL setup, run contamination checks, and release code/data.","headline":"A plausible RL recipe for Chinese spelling correction with large claimed gains, but the paper omits the controls and details needed to trust them.","tokens_in":14369,"tokens_out":1948,"would_cite":false,"duration_ms":20309,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Reinforcement learning with an embedding-clustering reward lifts Chinese spelling correction accuracy of Qwen3 LLMs by about 13 F1 points on average.","keywords":["Chinese spelling correction","reinforcement learning","large language models","embedding clustering reward","self-generated training data","cross-domain generalization","Qwen3","text error correction"],"falsifier":"Train CEC-Zero with only homophone-based perturbations and evaluate on a test set whose errors are mainly visually similar characters; if the RL model's F1 on those errors stays near the base model's, the clustering reward is not generalizing beyond the perturbation distribution.","tokens_in":13462,"feed_emoji":"✏️","tokens_out":7720,"duration_ms":67613,"temperature":0.7,"pith_summary":"This paper proposes CEC-Zero, a reinforcement-learning framework that trains large language models to correct Chinese spelling errors without any human-annotated error data. Instead of collecting erroneous sentences, it generates them by automatically perturbing correct sentences using homophone, shape-similar, and structural edits, then rewards the LLM when its correction is semantically close to the original. The reward is computed from sentence embeddings: one term measures similarity to the known correct sentence, and a second term measures similarity to the center of a cluster of the model's own sampled outputs, which serves as a pseudo-label. On CSCD-NS, the seven-domain LEMON benchmark, and a customer-service test set, the authors report that RL training lifts the average F1 of Qwen3-14B from 52.16 to 65.14 and of Qwen3-32B from 55.47 to 68.15, surpassing the tested closed-source LLMs. The central claim is that convergence among model-generated answers is a usable reward signal in text correction, where multiple valid corrections exist for a single error sentence.","feed_headline":"Embedding-cluster reward lifts Chinese spelling F1 by 13","feed_subtitle":"Self-generated errors and no labels: RL trains Qwen3 to beat GPT-4 and Doubao on cross-domain tests.","key_machinery":"The load-bearing mechanism is the embedding-clustering reward function. The model generates $l$ sampled corrections for a source sentence; their sentence embeddings are clustered by Euclidean distance, and the center of the largest tight cluster (containing more than one third of the batch) is treated as a pseudo-label. The final reward is $$\\text{RLscore} = \\$\\alpha$ \\cdot \\max\\left(0, \\frac{\\cos_1 - \\$\\theta$}{1 - \\$\\theta$}\\right) + \\gamma \\cdot \\max\\left(0, \\frac{\\cos_2 - \\$\\beta$}{1 - \\$\\beta$}\\right),$$ with defaults $\\alpha = \\gamma = 0.5$, $\\theta = 0.8$, and $\\beta = 0.85$. This reward substitutes for both ground-truth labels (RLscore1) and majority-vote labels (RLscore2), letting RL optimize for semantic correctness instead of exact symbol matching, which matters because a Chinese error sentence can have several acceptable corrections.","core_discovery":"The paper's central claim is that an LLM can be taught Chinese spelling correction by reinforcement learning alone, with no labeled error-correction pairs and no auxiliary validation model, provided the reward is built from semantic embeddings rather than exact string matches. The training data pairs are constructed by corrupting correct sentences with a diverse set of perturbation tools—homophone replacement, visually similar character replacement, merging and splitting of Chinese characters, and symbol insertion—so that the model never sees a real hand-annotated error during training. The reward has two components: RLscore1, the cosine similarity between the embedding of the model's prediction and the embedding of the known correct sentence, and RLscore2, the cosine similarity between the prediction and a pseudo-label formed by clustering the model's own sampled outputs and taking the center of the densest cluster; both are thresholded and linearly combined. The authors report that after RL, Qwen3-14B and Qwen3-32B reach average sentence-level F1 scores of 65.14 and 68.15 on their test collection, improving by roughly 13 points over the base models and exceeding all closed-source baselines tested, including GPT-4, Claude 3.7, and Doubao.","pith_inferences":["The authors leave implicit that the diversity of the perturbation tools is the main determinant of what the model learns; if real-world error streams contain paradigms absent from the perturbation set, the RL reward will not teach the model to fix them, regardless of the clustering signal.","A testable extension is to apply CEC-Zero to test-time adaptation: because the reward needs no labels, the same RL loop could be run on unlabeled text arriving at deployment time, something the paper motivates but does not experiment with.","If the base model's sampled outputs are confidently wrong, the pseudo-label center will be confidently wrong too, so the clustering term could reward self-consistency rather than correctness; the reported gains suggest this did not dominate on these benchmarks, but the failure mode is worth probing."],"forward_implications":["Chinese spelling correction can be improved on open LLMs without annotated error corpora; only clean Chinese sentences are needed, since perturbation tools generate the errors.","Cross-domain generalization improves: the RL-trained models outperform all tested baselines on LEMON's seven domains in the zero-shot evaluation reported, including the customer-service set.","The same RL recipe should transfer to other text-generation tasks with multiple valid answers, because the reward is based on semantic similarity and answer clustering rather than task-specific rules.","Reliability of LLM output improves with no extra annotation cost and no auxiliary model, since the reward is computed from embeddings and the model's own sampled outputs.","The clustering-based pseudo-label acts as a self-consistency signal: outputs that agree with the model's own majority cluster are treated as correct, providing a practical substitute for verifiable rewards in non-math, non-code tasks."],"supporting_citations":[{"why":"Supplies the repeated-sampling pseudo-reward mechanism that CEC-Zero adapts into the RLscore2 clustering signal.","marker":"[55]"},{"why":"Establishes rule-based reinforcement learning rewards for LLM reasoning, the paradigm CEC-Zero extends to text correction.","marker":"[9]"},{"why":"Provides the CSCD-NS benchmark and training material whose Pinyin-IME homophone errors motivate the perturbation design.","marker":"[64]"},{"why":"Introduces LEMON, the multi-domain Chinese spelling correction benchmark used for zero-shot cross-domain evaluation.","marker":"[17]"},{"why":"Documents the Qwen3 base models that the RL training is applied to.","marker":"[49]"},{"why":"Supplies the Soft-Masked BERT baseline that CEC-Zero must outperform in the comparisons.","marker":"[69]"},{"why":"Supplies the GPT-4 baseline, one of the closed-source LLMs the RL-trained models are measured against.","marker":"[31]"}],"fun_headline_variants":["No labels, no tutors: RL alone lifts Chinese spelling F1 by 13","Self-generated errors + RL: Chinese spelling beats GPT-4","RL embedding-cluster reward: Chinese spelling +13 F1, no labels","RL self-correction: Chinese spelling F1 up 13, beats GPT-4","No annotated data: RL teaches Qwen3 to fix Chinese spelling"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The synthetic corruption of correct sentences produces training inputs whose error distribution matches the real-world errors seen in the test sets.","fun_headline_variants_meta":{"raw":{"variants":["No labels, no tutors: RL alone lifts Chinese spelling F1 by 13","Self-generated errors + RL: Chinese spelling beats GPT-4","RL embedding-cluster reward: Chinese spelling +13 F1, no labels","RL self-correction: Chinese spelling F1 up 13, beats GPT-4","No annotated data: RL teaches Qwen3 to fix Chinese spelling"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001217,"raw_usage":{"total_tokens":4994,"prompt_tokens":920,"completion_tokens":4074,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":536,"completion_tokens_details":{"reasoning_tokens":3975}},"tokens_in":536,"tokens_out":4074,"duration_ms":26890,"temperature":1.0,"reasoning_tokens":3975,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:40:01.233552+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train CEC-Zero with only homophone-based perturbations and evaluate on a test set whose errors are mainly visually similar characters; if the RL model's F1 on those errors stays near the base model's, the clustering reward is not generalizing beyond the perturbation distribution.","supporting_citations":[{"cited_title":"CSCD-NS: a Chinese Spelling Check Dataset for Native Speakers","cited_arxiv_id":"2211.08788","evidence_quote":"Provides the CSCD-NS benchmark and training material whose Pinyin-IME homophone errors motivate the perturbation design."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents the Qwen3 base models that the RL training is applied to."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Soft-Masked BERT baseline that CEC-Zero must outperform in the comparisons."}],"review_version":1}