{"id":"acd557fe-7078-47d0-b713-e3e5675bb287","arxiv_id":"2505.24341","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A new taxonomy and dataset of 8 types of perturbed toxic Chinese show nine top LLMs often miss these obfuscated insults, and small-sample ICL or fine-tuning causes overcorrection.","lead":"Researchers built a new Chinese toxic-language benchmark that tests how well AI models detect insults and hate speech when words are obfuscated through homophones, emoji, character splitting, and similar tricks. They found even top AI models miss many of these perturbed insults, and that fine-tuning on just a few examples makes models wrongly flag normal text as toxic.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The benchmark's positive labels are not validated for toxicity preservation, so both the detection-rate gap and the overcorrection finding could be artifacts of non-toxic perturbed examples.","rationale":"The reader's weakest_assumption identifies exactly the load-bearing concern: Section 4.4 validates readability but not toxicity preservation. This is not a manufactured worry. The paper's detection-rate metric counts every perturbed example as a positive, and the overcorrection claim compares detection rates on those positives against error rates on non-toxic examples. If the positive set contains non-toxic items, the central quantitative comparisons are biased in a specific direction: LLMs are penalized for correctly rejecting non-toxic perturbed text, and fine-tuning on such labels can teach the model to over-predict toxicity, making the overcorrection finding look larger than it is. The paper deserves credit for other strengths: the taxonomy is systematic, the human readability validation is a real step, the prompt ablation is useful, and the CA-CoT exploration is a constructive addition. But those strengths do not close the label-validity gap. The proposed test is concrete and feasible: a stratified human toxicity check on the perturbed examples, followed by a restricted re-analysis if needed. If the test shows a low non-toxic rate, the current findings likely stand; if it shows a high rate, the benchmark and both headline findings need revision. Since the reader already recommended conditional acceptance, my concern reinforces that recommendation rather than moving it; hence UNCHANGED.","tokens_in":16864,"tokens_out":3511,"duration_ms":48293,"concrete_test":"Draw a stratified random sample of 100 examples per perturbation type (800 total) from CNTP. Have four native Chinese speakers, blind to the original sentences, provide a binary toxicity label for each perturbed example, plus a confidence score. Compute the rate at which perturbed examples are judged non-toxic and report inter-annotator agreement. If this rate exceeds a small threshold (e.g., 5%), re-run the Table 3, Table 5, and Table 6 evaluations restricted to the toxicity-preserving subset; if the detection-rate gaps or the ER jump shrink materially, the headline findings require qualification.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that SOTA LLMs fail to detect perturbed toxic Chinese and that small-sample ICL/SFT causes overcorrection—treats every CNTP example as a positive toxic sample. Section 4.4 validates only readability (1–5 score, discard below 3) and extraction accuracy; it never verifies that the perturbation preserves toxicity. Readability is not semantic invariance: a sentence can be readable but non-toxic, e.g., a homophone that resolves to a neutral phrase, a shuffled character order that produces a different word, or a split character that no longer spells the insult. The paper itself notes in Section 3.2 that full Pinyin can yield neutral homophones ('da ren' can mean 'adult' rather than 'hit person') and in Section 3.3 that shuffling can change meaning entirely ('海上' to '上海'). If even a modest fraction of the 20,087 positives are no longer toxic, all detection-rate numbers are contaminated: a model saying 'non-toxic' is counted as a miss, and a fine-tuned model trained on such labels learns to call benign text toxic, inflating the apparent overcorrection. Table 2 reports average readability 3.94 and perturb ratio 0.28, but no toxicity-preservation check, so the benchmark's positive denominator is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a taxonomy of 3 perturbation strategies and 8 methods for obfuscating toxic Chinese text (glyph, phonetic, and semantic), constructs a large benchmark dataset (CNTP) of 20,087 perturbed toxic texts, and evaluates 9 state-of-the-art LLMs from the US and China on their ability to detect these perturbed examples. It further studies cost-effective enhancement via in-context learning (ICL) and supervised fine-tuning (SFT), reporting that these small-sample methods improve detection rates but cause severe overcorrection, i.e., misclassifying many non-toxic Chinese sentences as toxic. The central claims are that SOTA LLMs are less capable of detecting perturbed multimodal Chinese toxic contents, and that ICL/SFT with a small number of perturbed examples leads to overcorrection.","tokens_in":17188,"tokens_out":4019,"duration_ms":49180,"significance":"If the findings hold, the paper makes a useful contribution: the taxonomy is systematic, the dataset is a potentially valuable resource for the Chinese toxicity detection community, and the overcorrection finding is practically important for deploying LLM-based moderation systems. The authors provide a public repository, include an ethics statement, and their prompt-ablation study shows that results are sensitive to prompt wording, which is a useful robustness analysis. However, the central claims depend on an unvalidated assumption that every perturbed example remains toxic, and the Misinterpretation Rate used to support the overcorrection conclusion is not formally defined. These issues need to be addressed before the empirical conclusions can be trusted.","major_comments":[{"comment":"The human validation protocol covers extraction accuracy and readability, but it never verifies that a perturbed sentence remains toxic. The paper's own Section 3.2 notes that full Pinyin can produce neutral homophones ('da ren' can mean 'adult' rather than 'hit person'), and Section 3.3 notes that shuffling can change meaning entirely ('海上' to '上海'). Because every one of the 20,087 CNTP examples is treated as a positive toxic sample in Tables 3, 5, and 6, any perturbed example that is actually non-toxic or incoherent is counted as a detection miss, and a fine-tuned model trained on such labels would learn to call benign text toxic, inflating the apparent overcorrection. The authors should add a human toxicity-preservation check (e.g., annotate a sample of perturbed sentences for whether the intended toxic meaning is retained) and report the agreement rate; without this, the detection-rate numbers and the overcorrection finding are not established.","section":"§4.4"},{"comment":"The Misinterpretation Rate (MR) is never defined. The text says it 'evaluates whether the LLM truly understands and identifies perturbed contexts' and that the authors 'select one perturbation from them,' but no formula, annotation protocol, or denominator is given. Tables 5, 6, and 9 report MR values, and the paper uses high MR to conclude that ICL/SFT improvements come from overcorrection rather than genuine understanding. Without an operational definition, these results cannot be interpreted or reproduced.","section":"§6.1"},{"comment":"The benchmark reports single-run detection rates with no error bars, confidence intervals, or significance tests. The prompt-ablation tables in Appendix C show that detection rates can swing by dozens of points with prompt wording (e.g., Qwen-turbo on VSim: 85.86% with CN vs. 46.85% with CN_Concise), so the reported cross-model and cross-perturbation differences in Table 3 should be accompanied by variance estimates or repeated trials. This is particularly important for the first claim that SOTA LLMs are less capable on certain perturbation types, since the gaps for some models are comparatively small.","section":"§5.1 / Table 3"}],"minor_comments":[{"comment":"The heading 'Abalation' should be corrected to 'Ablation'.","section":"Appendix C"},{"comment":"Several cells contain merged numeric strings without separators (e.g., the GLM-4-Air row: '92.4657.7482.6051.8980.8477.08'); these should be separated into distinct values.","section":"Table 3"},{"comment":"The text cites 'RoCBert, ToxiCloakCN, and Adversarial GLUE' without full citations; the corresponding references should be added.","section":"§4.3"},{"comment":"For API-based models, the access dates or model version identifiers should be specified to improve reproducibility.","section":"§5.1"},{"comment":"The footnote about OpenAI fine-tuning requiring at least 10 samples is placed after the mention of 10 samples, but the paper also uses 20 and 40; the relationship between the minimum sample size and the chosen sizes should be clarified.","section":"§6.1"},{"comment":"The limitation about small sample sizes in the mitigation experiments is mentioned only at the end; moving this caveat into the experimental setup in Section 6 would better frame the ICL/SFT results.","section":"Limitations"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid empirical scope and a useful resource, but the missing toxicity-preservation validation and the undefined MR are load-bearing issues. I believe these can be fixed with an additional human annotation study and a clear definition of MR, so I am not recommending reject. No concerns about integrity or citation behavior beyond the points in the report."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a read if you work on Chinese content moderation. The paper builds a systematic 8-way taxonomy of perturbations (glyph, phonetic, semantic) and releases CNTP, a 20k-example dataset. The LLM benchmark across 9 models is extensive, and the overcorrection finding is the real news: fine-tuning GPT-4o-mini on 10–40 perturbed examples pushes detection above 98% while the false-positive rate on clean text climbs from 2.7% to over 30%. That is a practical warning for anyone tuning a moderation pipeline, and it is not in ToxiCloakCN.\n\nThe soft spots are serious but not disqualifying. The biggest one, flagged in the stress test, is that the perturbed examples are never checked to still be toxic. The authors validate readability and entity-extraction accuracy but not semantic invariance. The paper itself gives examples where homophones become neutral ('da ren' = adult) and shuffling changes meaning entirely ('hai shang' to 'shang hai'), so this is not a hypothetical worry. If a non-trivial fraction of the 20k positives are no longer toxic, every detection-rate number is contaminated, and the fine-tuning result could partly be an artifact of training on mislabeled positives. The fix is straightforward: have the human annotators also judge toxicity on a sample of the perturbed set, and report agreement.\n\nThe other weaknesses are milder. There are no error bars, significance tests, or repeated trials; the numbers are single deterministic runs, so the model rankings could be within noise. The Misinterpretation Rate is under-defined—I still don't know exactly how it was computed. The fine-tuning experiment uses a tiny sample with no randomization, and the prompt-ablation shows results swing by 20–40 points depending on wording, which means the headline numbers are prompt-dependent. None of these kill the paper, but they all need to be addressed.\n\nOn the positive side, the paper is honest about its limitations, the taxonomy is a real organizational contribution, and the comparison across US and Chinese LLMs is useful. The citation pattern looks fine; ToxiCloakCN is properly credited and this clearly goes beyond it. The circularity of using GPT-4o-mini to extract entities that later evaluate GPT-4o-mini is largely mitigated by the 98.6% extraction accuracy with human validation.\n\nRecommendation: send it to peer review. The topic is important, the empirical scope is substantial, and the overcorrection finding is worth publishing even if the label-validation issue requires a major revision. A good referee can push the authors to validate toxicity preservation and report uncertainty. I would not desk-reject this.","headline":"A useful benchmark and a genuinely cautionary overcorrection result, but the positive labels in the dataset are not validated for toxicity preservation, which puts the central numbers at risk.","tokens_in":708,"tokens_out":1253,"would_cite":true,"duration_ms":55838,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LLMs cannot reliably detect perturbed toxic Chinese, and small-sample adaptation makes them overcorrect.","keywords":["toxic content detection","Chinese language","multimodal perturbation","LLM robustness","overcorrection","in-context learning","supervised fine-tuning","CNTP benchmark"],"falsifier":"Conduct a human rating study in which native Chinese speakers see each perturbed sentence without the original and independently label whether it is toxic; then recompute detection rates only on sentences the humans agree are toxic and readable. If detection rates on that confirmed subset are high, the paper's central failure claim would be weakened, while if the fine-tuned model still over-flags confirmed-benign sentences, the overcorrection finding would be strengthened.","tokens_in":16715,"feed_emoji":"🛡️","tokens_out":6202,"duration_ms":74459,"temperature":0.7,"pith_summary":"This paper argues that the multimodal structure of Chinese—glyph, phonetic, and semantic variants of the same characters—forms a practical blind spot for state-of-the-art LLMs doing toxicity detection. To show this, it proposes a taxonomy of three perturbation strategies and eight concrete rewriting methods, builds a validated dataset of about 20,000 perturbed toxic Chinese sentences, and benchmarks nine LLMs from US and Chinese developers. The benchmark shows large detection drops for homophone, pinyin-initial, and character-splitting perturbations, with Chinese-developed models holding no clear edge. The second central finding is that lightweight adaptation backfires: fine-tuning GPT-4o-mini on ten perturbed examples raises detection above 98 percent but pushes the false-positive rate on ordinary Chinese from about 2.7 percent to over 30 percent. A careful reader would care because the result separates \"the model detects toxicity\" from \"the model has learned a brittle heuristic,\" with direct stakes for content moderation.","feed_headline":"Ten examples can make an LLM call 30% of normal Chinese toxic","feed_subtitle":"A new benchmark shows glyph, pinyin and emoji rewrites evade top LLMs; quick fixes raise false positives.","key_machinery":"The load-bearing object is the perturbation taxonomy plus the generation-validation pipeline that turns it into a benchmark. The taxonomy defines three strategies—glyph, phonetic, and semantic—and eight named perturbations: visual similarity, character splitting, traditional Chinese substitution, pinyin initials, full pinyin, homophone replacement, shuffling, and emoji replacement. The pipeline samples toxic and non-toxic sentences from Toxi_CN, uses GPT-4o-mini to extract toxic entities, applies each perturbation with a controlled perturbation rate below 30 percent, and keeps only sentences rated readable by four native-speaker annotators, yielding the CNTP dataset of 20,087 perturbed toxic texts. The paper then measures LLMs with three indicators—detection rate on toxic content, error rate on non-toxic content, and misinterpretation rate—so that \"correct\" classifications achieved by over-triggering can be distinguished from genuine understanding. This combination is what lets the paper attribute drops in performance to the Chinese multimodal character system rather than to mere prompt sensitivity.","core_discovery":"On the paper's own terms, the discovery is that current LLMs do not understand perturbed toxic Chinese; they pattern-match. The authors construct a taxonomy of Chinese-specific perturbations grounded in the language's logographic nature: glyph-based (visually similar characters, character splitting, traditional-script substitution), phonetic-based (pinyin initials, full pinyin, homophones), and semantic-based (character shuffling and emoji replacement). Across nine LLMs, homophone and pinyin-initial perturbations consistently push detection rates below 60 percent, and even the strongest models lose more than 20 points on average. When the authors try to patch the weakness with in-context learning or fine-tuning on 10–40 examples, detection rates rise but the misinterpretation rate stays high, and fine-tuned GPT-4o-mini flags more than 30 percent of benign Chinese sentences as toxic; human checks confirm the model is not recovering the intended meaning. The paper's claim is that this overcorrection is not a prompt artifact but a sign that the model acquired a shallow trigger heuristic rather than semantic understanding.","pith_inferences":["A likely extension beyond the paper: the overcorrection failure mode is a precision problem, not a recall problem, so any deployment that fine-tunes on small adversarial sets should gate on non-toxic error rate to avoid censoring benign speech.","The taxonomy logic may transfer to other logographic and script-mixing languages, such as Japanese kanji variants or Arabic script manipulations, so the benchmark design could serve as a template for robustness tests beyond Chinese.","The high misinterpretation rate suggests a testable route the paper does not fully pursue: instead of fine-tuning on labeled perturbed examples, models could be trained or prompted to reconstruct the original string before judging; the paper's CA-CoT result is preliminary evidence for this direction.","The 30-percent false-positive jump under tiny fine-tuning implies that benchmark evaluations reporting only detection rate on adversarial sets can badly overstate real-world safety; error rates on clean text should be reported alongside every detection gain."],"forward_implications":["If the benchmark reflects real-world Chinese social media, current LLM-based Chinese toxicity detectors are evadable by simple, human-readable rewrites that require no model knowledge.","Detection gains from ICL or SFT with small numbers of perturbed examples do not imply understanding; the same adaptation increases false positives on ordinary Chinese, so deployment needs precision monitoring.","Language alignment matters: Chinese prompts consistently outperform English prompts for the same model, so detection quality depends on prompt language as much as model capability.","Chinese-developed LLMs do not automatically outperform US-developed models on perturbed Chinese content, so claims of native-language advantage need robustness testing.","A prompt that explicitly asks the model to recover perturbations (CA-CoT) improves detection while keeping error rate low, suggesting that decoding the perturbation is a separable skill from judging toxicity."],"supporting_citations":[{"why":"Establishes the cloaking phenomenon this paper broadens from homophones and emojis into a full taxonomy.","marker":"Xiao et al., 2024"},{"why":"Supplies Toxi_CN, the base corpus of labeled toxic and non-toxic Chinese sentences that CNTP samples from.","marker":"Lu et al., 2023"},{"why":"Contributes the adversarial-readability method and the under-30 percent perturbation rate used in dataset construction.","marker":"Zhang et al., 2021"},{"why":"Provides COLD, the prior Chinese offensive-language benchmark that this work contrasts as explicit-only.","marker":"Deng et al., 2022"},{"why":"Documents that perturbed language bypasses content detectors, motivating the threat model.","marker":"Gevers et al., 2022"},{"why":"Shows real-world adversarial text perturbations evade detectors, supporting the need for robustness evaluation.","marker":"Li et al., 2019"}],"fun_headline_variants":["Chinese glyphs and homophones fool top LLMs, new benchmark finds","LLMs overcorrect on Chinese toxicity after tiny fine-tune","Perturbed Chinese toxic text evades nine major LLMs","Homophones and emojis slip past LLM toxic detectors","Overcorrection: 10 examples turn LLM into toxic false-flag machine"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the automatically generated perturbations preserve the original sentence's toxicity and are readable to native speakers; if a perturbation changes the meaning or becomes incoherent, the detection rates measure something other than toxicity detection.","fun_headline_variants_meta":{"raw":{"variants":["Chinese glyphs and homophones fool top LLMs, new benchmark finds","LLMs overcorrect on Chinese toxicity after tiny fine-tune","Perturbed Chinese toxic text evades nine major LLMs","Homophones and emojis slip past LLM toxic detectors","Overcorrection: 10 examples turn LLM into toxic false-flag machine"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000556,"raw_usage":{"total_tokens":2657,"prompt_tokens":966,"completion_tokens":1691,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":582,"completion_tokens_details":{"reasoning_tokens":1599}},"tokens_in":582,"tokens_out":1691,"duration_ms":16301,"temperature":1.0,"reasoning_tokens":1599,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:24:37.710644+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Conduct a human rating study in which native Chinese speakers see each perturbed sentence without the original and independently label whether it is toxic; then recompute detection rates only on sentences the humans agree are toxic and readable. If detection rates on that confirmed subset are high, the paper's central failure claim would be weakened, while if the fine-tuned model still over-flags confirmed-benign sentences, the overcorrection finding would be strengthened.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the cloaking phenomenon this paper broadens from homophones and emojis into a full taxonomy."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Contributes the adversarial-readability method and the under-30 percent perturbation rate used in dataset construction."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents that perturbed language bypasses content detectors, motivating the threat model."}],"review_version":1}