Pith. sign in

REVIEW 4 major objections 6 minor 70 references

Benchmarking the Detection of LLMs-Generated Modern Chinese Poetry

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that current AI-text detectors cannot reliably identify modern Chinese poems written by large language models, and that the hardest cases are poems imitating a human poet's style.

desk verdict Useful first benchmark with a real style-imitation finding, but the headline is overbroad and the evaluation needs a leakage fix before the out-of-domain numbers can be trusted. read the letter →

arxiv 2509.01620 v1 pith:BEY2547M submitted 2025-09-01 cs.CL cs.AI

Shanshan Wang , Junchao Wu , Fengying Ye , Jingming Yao , Lidia S. Chao , Derek F. Wong This is my paper
classification cs.CLcs.AI
keywords AI-generatedtextdetectionmodernChinesepoetryLLMbenchmarkdatasetstyleimitationdetectorevaluationtemperaturesensitivity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that existing detectors for AI-generated text cannot be trusted when the text is modern Chinese poetry, a genre that deliberately breaks grammar and convention. To test this, the authors built a dataset pairing 800 poems from six professional poets with 41,600 poems generated by four large language models under 13 different prompts that isolate style, thought, theme, stanza and line structure, and emotion. Running six detectors—five statistical and one fine-tuned RoBERTa classifier—they find the hardest cases are poems that imitate a human poet's style, where average F1 drops below 60 percent, while poems that literally name emotions like fear are the easiest to catch. If this is right, the practical consequence is that style-matching prompts, already common in real-world AI poetry generation, are an effective way to evade current detection tools.

What carries the argument

The central object is the AIGenPoetry benchmark: a paired dataset of 800 human-written modern Chinese poems and 41,600 LLM-generated poems, produced by four LLMs under 13 prompts that isolate specific poetic dimensions. Prompts P2–P5 target intrinsic qualities (style imitation, thought and sentiment, theme); P6–P8 target external structure (stanza count, line count, both); P9–P13 target emotions (none, happiness, sadness, anger, fear). Each prompt yields a matched data pair (human vs. one LLM), so detector performance can be attributed to the specific feature being controlled. Six detectors—Fast-DetectGPT, LRR, Log-Likelihood, Log-Rank, Binoculars, and a fine-tuned RoBERTa classifier—are eva

What would settle it

Give a detector only style-matched poem pairs (D2, D3) that are also matched for length and stripped of explicit emotion words, and require an F1 above 90 percent. If any current detector, or a simple style-based feature, reaches that level, the paper's conclusion that intrinsic qualities—especially style—are the hardest poetic features to detect would be contradicted.

Watch

Extended reading notes

Core claim

The paper's central claim is that current detectors cannot be used as reliable tools to detect modern Chinese poems generated by large language models. The evidence comes from a controlled benchmark in which four LLMs (GPT-4.1, DeepSeek-V3, DeepSeek-R1, GLM-4) each generated 800 poems per prompt across 13 prompt types, paired one-to-one with 800 human poems. In-domain results show average F1 around 72 percent on the baseline prompt and as low as 59.68 percent when poems are generated to imitate the style of a human poem (D2). GPT-4.1 style-matched poems are the hardest of all, with detectors averaging 48.73 percent F1. Poems that explicitly express a single emotion, especially fear, are the

Load-bearing premise

The benchmark assumes that the 800 poems from six professional poets, and the 41,600 poems generated from them at temperature 1.5 with top_p 0.95, represent the real distribution of human-written and LLM-generated modern Chinese poetry; if the poet set, the prompt distribution, or the sampling settings changed, the measured detection difficulty could shift substantially.

Editorial extensions

If this is right

  • Style-matched generation is an effective evasion strategy: every detector tested loses at least 7 percentage points of F1 when the poem imitates a human poet's style, and four of the six lose more than 15 points.
  • Surface emotion words are a giveaway: poems that literally express emotions, especially fear, are the easiest to detect, which reflects a real asymmetry since human poets usually convey emotion implicitly.
  • Temperature matters: poems generated at temperature 1.5 are the hardest to detect, and detectors trained on 1.5 data generalize to lower-temperature data, so high-temperature generation is a reasonable worst-case benchmark setting.
  • Fine-tuned RoBERTa outperforms statistical detectors on this task, but its advantage shrinks on style-matched poems, meaning no current detector is reliable for real-world style-imitation use cases.
  • Detectors trained on intrinsic-quality prompts (style, thought, theme) generalize better to other feature types than detectors trained on structure or emotion prompts, pinpointing intrinsic qualities as the core unsolved difficulty.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The pattern suggests detectors are latching onto surface statistics—explicit emotion vocabulary, poem length, and stanza or line counts—rather than deeper stylistic or semantic signals; a detector built around imagery density, syntactic deviation, or cross-stanza coherence might close the style gap.
  • The one-to-one prompt pooling design could be stress-tested further by combining style imitation with other constraints (style plus structure, style plus emotion) to find which combinations are hardest, and by adding more human poets to reduce the risk that six poets' styles are unusually easy or hard to imitate.
  • The temperature results imply that a detector trained only on high-temperature, style-matched data would face the toughest possible distribution; this suggests a practical recipe for red-teaming future poetry detectors before deployment.
  • The benchmark's human poems come from six professional poets, so the finding is about matching those specific styles; how well detectors perform against a wider, more diverse human corpus remains an open question the paper does not resolve.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces AIGenPoetry, a benchmark for detecting whether modern Chinese poems were written by humans or generated by LLMs. The dataset contains 800 human poems by six professional poets and 41,600 poems generated by GPT-4.1, DeepSeek-V3, DeepSeek-R1, and GLM-4 under thirteen prompts covering intrinsic qualities (style, thought, theme), external structures (stanzas, lines), and emotions. The authors evaluate five zero-shot detectors plus a fine-tuned RoBERTa classifier in in-domain and generalization settings, and they report additional experiments across generation temperatures. The central claim is that current detectors cannot reliably detect LLM-generated modern Chinese poetry, with style-matched poems being the hardest and emotion-expressing poems—especially fear—the easiest to detect.

Significance. If the empirical findings hold, this is a useful first benchmark for an understudied domain. The dataset construction covers a broad prompt space, the human poems come from vetted professional poets, and the authors provide data/code links and a systematic comparison of six detectors. The observation that style-matched generation defeats most detectors, including near-chance performance for several zero-shot methods, is a concrete and falsifiable result. The temperature experiments also provide a practical insight for benchmark design. However, the strength of these contributions is currently limited by unresolved experimental-control issues, so the paper's sweeping conclusions outrun its evidence.

major comments (4)
  1. [Section 4, Table 4] The generalization experiments are confounded by data leakage. All Di conditions reuse the same 800 human poems (Section 4: 'Human-written poems and LLMs-generated poems through Pi constitute a pair of data (HDi, LDi), denoted as Di'). In Table 4, training on D2+4+5 or D6−8 means the RoBERTa classifier has already seen the identical human poems that appear in the test conditions (D1, D9, D10−13, etc.). The human side of the test set can therefore be solved by memorization rather than by detecting LLM-generated text, inflating F1 and corrupting both the generalization conclusions and the relative difficulty ordering of poetry features. Please use a disjoint-poet split (e.g., train on a subset of poets, test on held-out poets) and report exact train/test splits for all experiments, including the in-domain set-ups.
  2. [Tables 2–6] No variance or significance information is reported. Every F1/AUROC is a single number with no seeds, standard deviations, or statistical tests. Some conclusions rest on small differences (e.g., Table 2 zero-shot F1 scores range 67.51–69.10), and RoBERTa fine-tuning is stochastic. Without multiple runs and error bars, the claims that one detector 'significantly outperforms' another, or that current detectors 'cannot be used as reliable tools,' are not supported. Please report means and standard deviations over at least five seeds, and use a paired test where appropriate.
  3. [Section 5, Tables 2–3] For the zero-shot detectors, F1 depends on the decision threshold, but thresholds are not reported. AUROC is threshold-independent, yet the paper's central claims are framed primarily through F1. If thresholds were chosen on the test set, the F1 values are optimistic; if defaults were used, this should be stated and justified. Also, Table 2 shows Fast-DetectGPT with AUROC 43.24 on GPT-4.1, i.e., below chance, which suggests inverted scoring behavior; the paper should address why this occurs and how such detectors are being thresholded.
  4. [Abstract, Section 6.2, Conclusion] The central claim is overbroad. The experiments cover four LLMs, one generation configuration (temperature 1.5, top_p 0.95), six poets, and a specific set of prompts. The conclusion that 'current detectors cannot be used as reliable tools to detect modern Chinese poems generated by LLMs' should be qualified to the tested conditions. Additionally, the style-difficulty conclusion is partly confounded: prompts P2 and P3 explicitly supply the human reference poem in the input, so the style condition differs from the baseline and structural conditions not only in 'style' but also in the presence of the reference poem. Please add a control condition or temper the causal language.
minor comments (6)
  1. [Section 4] Typo: 'top_p to0.95' should read 'top_p to 0.95.'
  2. [Table 6] The abbreviation 'Fast-Det.' is used in the table header but only defined in the caption as 'Fast-DetectGPT' in the first row; please define it in the header or caption consistently.
  3. [Table 8] The column headers 'P > Avg.' and 'P < Avg.' are unclear; define 'P' (presumably proportion of poems) in the caption.
  4. [Section 6.1] The text says 'the remaining five detectors have F1-scores clustered between 67.51% and 68.74%,' but Table 2 lists Fast-DetectGPT at 69.10 and Log-Likelihood at 68.00. Please correct the range or explain the discrepancy.
  5. [Limitations] The Limitations section addresses only classical/rhyming poetry. It should also mention the small number of poets, the single generation temperature, and the specific LLM set as limits on external validity.
  6. [References] The ZHIPU reference is incomplete ('Zhipu ai devday glm-4' with no full title or venue); also the author name 'Fengying Y e' in the author block has an unintended space.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity: the detection results are empirical and self-contained; self-citations and a self-referential validation statement do not reduce to the paper's own inputs.

full rationale

The paper's central claims—that current detectors cannot reliably detect LLM-generated modern Chinese poetry and that style-matched poems are the hardest to detect—are supported by measured F1 scores on a newly constructed dataset. The human poems and LLM-generated poems are distinct collections, and the detector evaluations in Tables 2–4 are empirical outcomes, not quantities defined into existence by the benchmark. The 'style is hardest' conclusion follows from observed performance drops on prompt conditions D2/D3, not from the prompt wording itself. Several self-citations appear (e.g., Wu et al. 2025 for dataset-construction principles, Wu et al. 2024b for RoBERTa's prior performance, Wang et al. 2024 for poetry-related background), but none of these supplies a result that the paper then merely renames as a prediction; the detectors are evaluated fresh on the new benchmark. The statement that 'the detection results verify the effectiveness and necessity of our proposed benchmark' is self-referential rhetoric, but it is not an equation-level reduction: the benchmark's difficulty is observed, not derived from its own definition. No fitted parameter is relabeled as a prediction, no uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via citation. The shared-800-poems leakage concern is a real validity risk for generalization claims, but it is an experimental-design issue, not a circularity of the kind this analysis targets. Overall, the central derivation is self-contained against external benchmarks, so the circularity score is low.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The benchmark rests on domain assumptions about what modern Chinese poetry is and about the representativeness of two small corpora. There are no new theoretical entities. The main hand-chosen numbers are the generation temperature and the RoBERTa fine-tuning settings.

free parameters (2)
  • Generation temperature and top_p = 1.5 and 0.95
    Chosen following DeepSeek's recommendations for poetry tasks (Section 4); the temperature experiments show difficulty varies with this setting, so it is a hand-chosen parameter that shapes all benchmark results.
  • Fine-tuning hyperparameters for RoBERTa = 3 epochs, learning rate 1e-6, batch size 16
    Reported in Section 5 with no search or sensitivity analysis; these choices likely influence the reported F1.
assumptions (4)
  • domain assumption Modern Chinese poetry is free-form and deliberately violates grammatical conventions, making grammar-based detectors inapplicable.
    Invoked in Section 1 to justify the task and to dismiss GECSCORE and coherence-based methods.
  • domain assumption The 800 poems from six professional poets provide a representative sample of human-written modern Chinese poetry.
    Stated in Section 4 as the human side of the dataset; the paper generalizes conclusions to 'modern Chinese poems' from this small, expert-only corpus.
  • domain assumption Binary classification into human-written versus LLM-generated is a valid and complete formulation of the detection problem.
    Defined in Section 3; this excludes provenance among multiple models, partial AI assistance, and collaborative human-AI writing.
  • domain assumption Poems generated at temperature 1.5 and top_p 0.95 are representative of real-world LLM-generated modern Chinese poetry.
    This generation setting is used for the main benchmark (Section 4); the temperature experiments cover only two alternative values, not the full practical range.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Benchmarking the Detection of LLMs-Generated Modern Chinese Poetry." pith.science (2026). https://pith.science/paper/BEY2547M

@misc{pith2026250901620,
  author       = {Pith},
  title        = {Pith review of: Benchmarking the Detection of LLMs-Generated Modern Chinese Poetry},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BEY2547M}},
  note         = {Machine review of arXiv:2509.01620}
}
read the original abstract

The rapid development of advanced large language models (LLMs) has made AI-generated text indistinguishable from human-written text. Previous work on detecting AI-generated text has made effective progress, but has not involved modern Chinese poetry. Due to the distinctive characteristics of modern Chinese poetry, it is difficult to identify whether a poem originated from humans or AI. The proliferation of AI-generated modern Chinese poetry has significantly disrupted the poetry ecosystem. Based on the urgency of identifying AI-generated poetry in the real Chinese world, this paper proposes a novel benchmark for detecting LLMs-generated modern Chinese poetry. We first construct a high-quality dataset, which includes both 800 poems written by six professional poets and 41,600 poems generated by four mainstream LLMs. Subsequently, we conduct systematic performance assessments of six detectors on this dataset. Experimental results demonstrate that current detectors cannot be used as reliable tools to detect modern Chinese poems generated by LLMs. The most difficult poetic features to detect are intrinsic qualities, especially style. The detection results verify the effectiveness and necessity of our proposed benchmark. Our work lays a foundation for future detection of AI-generated poetry.

Figures

Figures reproduced from arXiv: 2509.01620 by the authors.

Figure 1
Figure 1. The framework of our proposed benchmark. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Stanza distribution of human-written poems. [PITH_FULL_IMAGE:figures/full_fig_p016_2.png] view at source ↗
Figure 3
Figure 3. Stanza distribution of DeepSeek-R1-generated poems. [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Stanza distribution of DeepSeek-V3-generated poems. [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]
Figure 5
Figure 5. Figure 5: Stanza distribution of GLM-4-generated poems. [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Stanza distribution of GPT-4.1-generated poems. [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Line distribution of human-written poems. [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Line distribution of DeepSeek-R1-generated poems. [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Line distribution of DeepSeek-V3-generated poems. [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Line distribution of GLM-4-generated poems. [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: Line distribution of GPT-4.1-generated poems. [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]
Figure 12
Figure 12. Figure 12: The number of lines per stanza and the corresponding count of stanzas in human-written poems. [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: The number of lines per stanza and the corresponding count of stanzas in DeepSeek-R1-generated poems. [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: The number of lines per stanza and the corresponding count of stanzas in DeepSeek-V3-generated [PITH_FULL_IMAGE:figures/full_fig_p019_14.png]
Figure 15
Figure 15. Figure 15: The number of lines per stanza and the corresponding count of stanzas in GLM-4-generated poems. [PITH_FULL_IMAGE:figures/full_fig_p020_15.png]
Figure 16
Figure 16. Figure 16: The number of lines per stanza and the corresponding count of stanzas in GPT-4.1-generated poems. [PITH_FULL_IMAGE:figures/full_fig_p020_16.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

70 extracted references · 38 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. https://arxiv.org/abs/2303.08774 Gpt-4 technical report . arXiv preprint arXiv:2303.08774

    arXiv 2023

  4. [4]

    Sina Alemohammad, Josue Casco-Rodriguez, Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaei, Daniel LeJeune, Ali Siahkoohi, and Richard G Baraniuk. 2023. https://arxiv.org/abs/2307.01850 Self-consuming generative models go mad . arXiv preprint arXiv:2307.01850, 4:14

    arXiv 2023

  5. [5]

    D. Antar. 2023. https://www.naturalspublishing.com/files/published/4i2xq2xki6j949.pdf The effectiveness of using chatgpt4 in creative writing in arabic: Poetry and short story as a model . In Information Sciences Letters

    work page 2023

  6. [6]

    Abdul Ghafoor Awan and Yasmin Khalida. 2015. https://core.ac.uk/download/pdf/234693069.pdf New trends in modern poetry . Journal of Literature, Languages and Linguistics, 13:63--72

    arXiv 2015

  7. [7]

    Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. 2023. https://arxiv.org/abs/2310.05130 Fast-detectgpt: Efficient zero-shot detection of machine-generated text via conditional probability curvature . arXiv preprint arXiv:2310.05130

    arXiv 2023

  8. [8]

    Brendan Bena and Jugal Kalita. 2020. https://arxiv.org/abs/2002.02511 Introducing aspects of creativity in automatic poetry generation . arXiv preprint arXiv:2002.02511

    work page Pith review arXiv 2020

Show all 70 references
  1. [9]

    Stefan Blohm, Valentin Wagner, Matthias Schlesewsky, and Winfried Menninghaus. 2018. https://www.sciencedirect.com/science/article/abs/pii/S0304422X17300864?casa_token=fZLWD530VLEAAAAA:6hL3soz_GX_rtn_KIC2gg3YIGOcagdgzEMJFMWje77zdeWHbsvoRzsf8I0UlPIWWAYQS_lqgtLaY Sentence judgme...

    2018

  2. [10]

    Ben Buchanan, Andrew Lohn, Micah Musser, and Katerina Sedova. 2021. https://cset.georgetown.edu/wp-content/uploads/CSET-Truth-Lies-and-Automation.pdf Truth, lies, and automation . Center for Security and Emerging technology, 1(1):2

    2021

  3. [11]

    Christopher JC Burges. 1998. https://link.springer.com/article/10.1023/A:1009715923555 A tutorial on support vector machines for pattern recognition . Data mining and knowledge discovery, 2(2):121--167

    1998 doi

  4. [12]

    Jo \ a o Phillipe Cardenuto, Jing Yang, Rafael Padilha, Renjie Wan, Daniel Moreira, Haoliang Li, Shiqi Wang, Fernanda Andal \'o , S \'e bastien Marcel, Anderson Rocha, and 1 others. 2023. https://arxiv.org/abs/2306.11503 The age of synthetic realities: Challenges and opportuni...

    2023 arXiv

  5. [13]

    Soma Chaudhuri, Maura Dooley, Dan Johnson, Roger Beaty, and Joydeep Bhattacharya. 2024. https://psycnet.apa.org/record/2024-51669-001 Evaluation of poetic creativity: Predictors and the role of expertise—a multilevel approach . Psychology of Aesthetics, Creativity, and the Arts

    2024

  6. [14]

    Xin Chen, Junchao Wu, Shu Yang, Runzhe Zhan, Zeyu Wu, Ziyang Luo, Di Wang, Min Yang, Lidia S Chao, and Derek F Wong. 2025. https://arxiv.org/abs/2508.13152 Repreguard: Detecting llm-generated text by revealing hidden representation patterns . arXiv preprint arXiv:2508.13152

    2025 arXiv

  7. [15]

    Zhongyi Chen. 2012 a . https://xueshu.baidu.com/usercenter/paper/show?paperid=93da768aff0a2eb7551cf8f26dc8a96f&site=xueshu_se Branch cross-industry, the trivial skills of external form? - research on modern poetic rhetoric v . Journal of Nanjing University of Science and Techn...

    2012

  8. [16]

    twist the neck of grammar

    Zhongyi Chen. 2012 b . https://xueshu.baidu.com/usercenter/paper/show?paperid=c974e7f2cdc678422498efa431cdf15e&site=xueshu_se Why "twist the neck of grammar" - a study of modern poetic rhetoric . Journal of Central China Normal University: Humanities and Social Sciences Editio...

    2012

  9. [17]

    Nicole A Cooke. 2018. https://books.google.com/books?hl=zh-CN&lr=&id=yh99DwAAQBAJ&oi=fnd&pg=PP1&dq=Fake+news+and+alternative+facts:+Information+literacy+in+a+post-truth+era&ots=YiSv4CVN8N&sig=TLJWgxfPETOIgygz8vjmIDcKSK0#v=onepage&q=Fake American Library Association

    2018

  10. [18]

    Kohinoor Darda, Marion Carre, and Emily Cross. 2023. https://royalsocietypublishing.org/doi/10.1098/rsos.220915 Value attributed to text-based archives generated by artificial intelligence . Royal Society Open Science, 10(2):220915

    2023 doi

  11. [19]

    Cheng Deng. 2007. https://xueshu.baidu.com/usercenter/paper/show?paperid=9e2401da68a441c1d5e3c691c144a586&site=xueshu_se Dilemma and solution: Reflections on current new poetry . Literary Review, (3):4

    2007

  12. [20]

    Wong, Jinpeng Hu, Lidia S

    Tao Fang, Shu Yang, Kaixin Lan, Derek F. Wong, Jinpeng Hu, Lidia S. Chao, and Yue Zhang. 2023. https://arxiv.org/abs/2304.01746 Is chatgpt a highly fluent grammatical error correction system? a comprehensive evaluation . arXiv preprint arXiv:2304.01746

    2023 arXiv

  13. [21]

    Wong, Keyan Jin, Lusheng Zhang, Qiang Zhang, Tianjiao Li, Jinlong Hou, and Lidia S

    Tao Fang, Tianyu Zhang, Derek F. Wong, Keyan Jin, Lusheng Zhang, Qiang Zhang, Tianjiao Li, Jinlong Hou, and Lidia S. Chao. 2025. https://doi.org/10.1016/j.eswa.2025.127397 Llmcl-gec: Advancing grammatical error correction with llm-driven curriculum learning . Expert Systems wi...

    2025

  14. [22]

    Yuan Gao, Ruili Wang, and Feng Hou. 2023. https://arxiv.org/pdf/2304.02182.pdf How to design translation prompts for chatgpt: An empirical study . arXiv e-prints, pages arXiv--2304

    2023 arXiv

  15. [23]

    Sebastian Gehrmann, Hendrik Strobelt, and Alexander Rush. 2019. https://aclanthology.org/P19-3019/ GLTR : Statistical detection and visualization of generated text . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstratio...

    2019

  16. [24]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. https://arxiv.org/abs/2501.12948 Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning . arXiv preprint arXiv...

    2025 arXiv

  17. [25]

    Moruo Guo. 1957. Talking about literary translation work. In Complete Works of Guo Moruo

    1957

  18. [26]

    Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. 2024. https://arxiv.org/abs/2401.12070 Spotting llms with binoculars: Zero-shot detection of machine-generated text . arXiv preprint arXiv:2...

    2024 arXiv

  19. [27]

    Kadhim Hayawi, Sakib Shahriar, and Sujith Samuel Mathew. 2024. https://arxiv.org/abs/2307.12166 The imitation game: Detecting human and ai-generated texts in the era of chatgpt and bard . Journal of Information Science, page 01655515241227531

    2024 arXiv

  20. [28]

    Sepp Hochreiter and J \"u rgen Schmidhuber. 1997. https://ieeexplore.ieee.org/abstract/document/6795963 Long short-term memory . Neural computation, 9(8):1735--1780

    1997

  21. [29]

    quasi-text

    Junming Huo. 2020. https://xueshu.baidu.com/usercenter/paper/show?paperid=1q0y0rk03s1v0e00vc2v0840f8747493&site=xueshu_se Clone li bai and 100 trillion poems - the "quasi-text" production and possible prospects of ai poetry . Southern Literature, (4):5

    2020

  22. [30]

    Maurice Jakesch, Jeffrey T Hancock, and Mor Naaman. 2023. https://www.pnas.org/doi/10.1073/pnas.2208839120 Human heuristics for ai-generated language are flawed . Proceedings of the National Academy of Sciences, 120(11):e2208839120

    2023 doi

  23. [31]

    David G Kleinbaum, K Dietz, M Gail, Mitchel Klein, and Mitchell Klein. 2002. https://link.springer.com/book/10.1007/978-1-4419-1742-3 Logistic regression . Springer

    2002 doi

  24. [32]

    Nils K \"o bis and Luca D Mossink. 2021. https://arxiv.org/abs/2005.09980 Artificial intelligence versus maya angelou: Experimental evidence that people cannot differentiate ai-generated from human-written poetry . Computers in human behavior, 114:106553

    2021 arXiv

  25. [33]

    Kaixin Lan, Tao Fang, Derek Wong, Yabo Xu, Lidia Chao, and Cecilia Zhao. 2024. https://doi.org/10.18653/v1/2024.findings-acl.858 FOCUS : Forging originality through contrastive use in self-plagiarism for language models . In Findings of the Association for Computational Lingui...

    2024 doi

  26. [34]

    Tian Lan, Xiangdong Su, Xu Liu, Ruirui Wang, Ke Chang, Jiang Li, and Guanglai Gao. 2025. https://aclanthology.org/2025.findings-acl.313/ M c BE : A multi-task C hinese bias evaluation benchmark for large language models . In Findings of the Association for Computational Lingui...

    2025

  27. [35]

    Andy Liaw, Matthew Wiener, and 1 others. 2002. https://journal.r-project.org/articles/RN-2002-022/RN-2002-022.pdf Classification and regression by randomforest . R news, 2(3):18--22

    2002

  28. [36]

    Christina Linardaki. 2022. https://journals.oregondigital.org/hsda/article/view/5759 Poetry at the first steps of artificial intelligence . Humanist Studies & the Digital Age, 7(1)

    2022

  29. [37]

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. https://arxiv.org/abs/2412.19437 Deepseek-v3 technical report . arXiv preprint arXiv:2412.19437

    2024 arXiv

  30. [38]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. https://arxiv.org/abs/1907.11692 Roberta: A robustly optimized BERT pretraining approach . CoRR, abs/1907.11692

    2019 arXiv

  31. [39]

    anti-traditional

    Zhenya Luo. 2000. https://xueshu.baidu.com/usercenter/paper/show?paperid=176k0c30af2q0ck0a86k0gy0pj318121&site=xueshu_se "anti-traditional" singing - the new artistic quality of bian zhilin's poetry . Literary Review, 000(002):84--91

    2000

  32. [40]

    Hanjia Lyu, Jiebo Luo, Jian Kang, and Allison Koenecke. 2025. https://doi.org/10.1145/3715275.3732182 Characterizing bias: Benchmarking large language models in simplified versus traditional chinese . In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and T...

    2025

  33. [41]

    Qingwei Mo. 2009. https://xueshu.baidu.com/usercenter/paper/show?paperid=e2f3d9dc30df34d15a9dda7993815b24&site=xueshu_se&hitarticle=1 Emotional classification of new poetry and its cultural roots . Management Observation, (13):1

    2009

  34. [42]

    Ayat Najjar, Huthaifa I Ashqar, Omar Darwish, and Eman Hammad. 2025. https://arxiv.org/abs/2501.03212 Leveraging explainable ai for llm text attribution: Differentiating human-written and multiple llms-generated text . arXiv preprint arXiv:2501.03212

    2025 arXiv

  35. [43]

    Brian Porter and Edouard Machery. 2024. https://www.nature.com/articles/s41598-024-76900-1 Ai-generated poetry is indistinguishable from human-written poetry and is rated more favorably . Scientific Reports, 14(1):26133

    2024

  36. [44]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, and 1 others. 2019. https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf Language models are unsupervised multitask learners . OpenAI blog, 1(8):9

    2019

  37. [45]

    Qi Shen. 1999. https://xueshu.baidu.com/usercenter/paper/show?paperid=d081ff4ee5b4318c8191c898f9f75e1f&site=xueshu_se Poetic, poetic form and non-poetic . Contemporary Writers Review, (6):3

    1999

  38. [46]

    Brian Phillips Skerratt. 2013. https://dash.harvard.edu/bitstream/handle/1/11181112/Skerratt_gsas.harvard_0084L_11116.pdf?sequence=3&isAllowed=y Form and Transformation in Modern Chinese Poetry and Poetics . Ph.D. thesis, Harvard University

    2013

  39. [47]

    Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, and 1 others. 2019. https://arxiv.org/abs/1908.09203 Release strategies and the social impacts of language models . arXiv preprin...

    2019 arXiv

  40. [48]

    Wai Lei Song, Haoyun Xu, Derek F Wong, Runzhe Zhan, Lidia S Chao, and Shanshan Wang. 2023. https://aclanthology.org/2023.mtsummit-research.27/ Towards zero-shot multilingual poetry translation . In Proceedings of Machine Translation Summit XIX, Vol. 1: Research Track, pages 324--335

    2023

  41. [49]

    Jinyan Su, Terry Yue Zhuo, Di Wang, and Preslav Nakov. 2023. https://arxiv.org/abs/2306.05540 Detectllm: Leveraging log rank information for zero-shot detection of machine-generated text . arXiv preprint arXiv:2306.05540

    2023 arXiv

  42. [50]

    Liyao Sun. 2011. https://xueshu.baidu.com/usercenter/paper/show?paperid=82fd55894f91be855b98ee1318be5869&site=xueshu_se The art of "travel": A new exploration of modern poetry form . Academic Monthly, (1):14

    2011

  43. [51]

    Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, and 1 others. 2022. https://arxiv.org/abs/2201.08239 Lamda: Language models for dialog applications . arXiv preprint arXiv:2201.08239

    2022 arXiv

  44. [52]

    Adaku Uchendu, Jooyoung Lee, Hua Shen, Thai Le, Dongwon Lee, and 1 others. 2023. https://arxiv.org/abs/2304.01002 Does human collaboration enhance the accuracy of identifying llm-generated deepfake texts? In Proceedings of the AAAI Conference on Human Computation and Crowdsour...

    2023 arXiv

  45. [53]

    Jian Wang. 2007. https://xueshu.baidu.com/usercenter/paper/show?paperid=44fb21334a91242bc81db3a2d0a38f82&site=xueshu_se The rationale and function of the construction of line divisions in modern poetry . Journal of Mianyang Normal University, 26(7):6

    2007

  46. [54]

    Mi Wang. 2019. https://xueshu.baidu.com/usercenter/paper/show?paperid=147n0pc0yu7m0030622r04t0xx370763&site=xueshu_se&hitarticle=1 Study on the relationship between external form and internal emotion in the rhythm of new poetry (1917-1949) . Ph.D. thesis, Sichuan Academy of So...

    2019

  47. [55]

    Shanshan Wang, Derek Wong, Jingming Yao, and Lidia Chao. 2024. https://aclanthology.org/2024.acl-long.756/ What is the best way for C hat GPT to translate poetry? In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)...

    2024

  48. [56]

    Shuting Wang. 2006 a . https://xueshu.baidu.com/usercenter/paper/show?paperid=1a5b02v0s6520mr0as4d0pn0u2589500&site=xueshu_se&hitarticle=1 Giving shape to emotion: The theory and practice of rhythm and imagery in new poetry (1917-1937) . World Literature Review

    2006

  49. [57]

    Yuchun Wang. 2006 b . https://xueshu.baidu.com/usercenter/paper/show?paperid=a157d4f289f3f209d5f03f94d90315c8 Moruo guo's view on literary translation . Moruo Guo Journal, page 5

    2006

  50. [58]

    Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, and 1 others. 2022. https://dl.acm.org/doi/10.1145/3531146.3533088 Taxonomy of risks posed by language models . In Proceedings ...

    2022

  51. [59]

    Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Lidia Sam Chao, and Derek Fai Wong. 2025. https://aclanthology.org/2025.cl-1.8/ A survey on llm-generated text detection: Necessity, methods, and future directions . Computational Linguistics, pages 1--66

    2025

  52. [60]

    Junchao Wu, Runzhe Zhan, Derek F Wong, Shu Yang, Xuebo Liu, Lidia S Chao, and Min Zhang. 2024 a . https://aclanthology.org/2025.coling-main.684.pdf Who wrote this? the key to zero-shot llm-generated text detection is gecscore . arXiv preprint arXiv:2405.04286

    2024 arXiv

  53. [61]

    Wong, Shu Yang, Xinyi Yang, Yulin Yuan, and Lidia S

    Junchao Wu, Runzhe Zhan, Derek F. Wong, Shu Yang, Xinyi Yang, Yulin Yuan, and Lidia S. Chao. 2024 b . http://papers.nips.cc/paper\_files/paper/2024/hash/b61bdf7e9f64c04ec75a26e781e2ad51-Abstract-Datasets\_and\_Benchmarks\_Track.html Detectrl: Benchmarking llm-generated text de...

    2024

  54. [62]

    Yunshu Xi. 2019. https://xueshu.baidu.com/usercenter/paper/show?paperid=1s640tu0es6r0xh0q03k08t007679271&site=xueshu_se&hitarticle=1 Two characteristics of new poetry language—talking about the creation of new poetry part three . Masterpiece Appreciation: Appreciation Edition ...

    2019

  55. [63]

    modern poetry

    Jiang Xu. 2015. https://xueshu.baidu.com/usercenter/paper/show?paperid=891c2dd584b3f174b5fd967370fef4a0&site=xueshu_se "modern poetry" and "new poetry" . Xingxing Monthly

    2015

  56. [64]

    leaving blanks

    Shichang Xue. 2010. https://xueshu.baidu.com/usercenter/paper/show?paperid=442713087618622a2fade61ba84c79a4&site=xueshu_se Modern poetry formal processing methods - talk about "leaving blanks", "raising lines" and "sectioning" . Reading and Writing, (4):3

    2010

  57. [65]

    Shichang Xue. 2016. https://xueshu.baidu.com/usercenter/paper/show?paperid=bd6148e1231d2c26338ae105eba26974&site=xueshu_se Something deeper than poetry itself -- on the artistic intention of the line arrangement of poetry . Write, page 6

    2016

  58. [66]

    Jianhao Yan, Yun Luo, and Yue Zhang. 2024. https://aclanthology.org/2024.findings-acl.818/ R efute B ench: Evaluating refuting instruction-following for large language models . In Findings of the Association for Computational Linguistics: ACL 2024, pages 13775--13791

    2024

  59. [67]

    Kuanghan Yang and Fuchun Liu. 1985. https://xueshu.baidu.com/usercenter/paper/show?paperid=2aa9302dcfa6448c66f8610193c624d7&site=xueshu_se Discussion on Chinese Modern Poetry, Part 1 . Discussion on Chinese Modern Poetry, Part 1

    1985

  60. [68]

    Kejia Yuan. 1991. https://xueshu.baidu.com/usercenter/paper/show?paperid=0245779c0eeba78bd0ae7696e7426faa&site=xueshu_se A moving quartet - the trajectory of feng zhi's poetic style . Literary Review, (4):7

    1991

  61. [69]

    ZHIPU. 2024. https://zhipuai.cn/en/devday Zhipu ai devday glm-4 . Accessed: 2024-05-01

    2024

  62. [70]

    Ling Zhong. 2003. https://xueshu.baidu.com/usercenter/paper/show?paperid=e7b50df60f6a225027c8b8978b696b54&site=xueshu_se American Poetry and Chinese Dream: Chinese Cultural Model in Modern American Poetry . American Poetry and Chinese Dream: Chinese Cultural Model in Modern Am...

    2003

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.