The Arabic AI Fingerprint: Stylometric Analysis and Detection of Large Language Models Text

Maged S. Al-shaibani; Moataz Ahmed

REVIEW 4 major objections 4 minor 1 cited by

The Arabic AI Fingerprint: Stylometric Analysis and Detection of Large Language Models Text

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Arabic text generated by large language models carries measurable stylometric fingerprints that allow near-perfect detection in academic writing, but not in social media.

desk verdict A useful public dataset and a competent detection study, but the academic human reference is unvalidated and the abstract overclaims cross-domain evidence. read the letter →

arxiv 2505.23276 v2 pith:YLGXNTLO submitted 2025-05-29 cs.CL cs.AI

Maged S. Al-Shaibani , Moataz Ahmed This is my paper

classification cs.CLcs.AI

keywords ArabicNLPmachine-generatedtextdetectionstylometricanalysiscross-domaingeneralizationXLM-RoBERTaLLMacademicabstractssocialmedia

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that large language models produce measurable stylistic fingerprints in Arabic text, and that these fingerprints are strong enough to support automated detection in formal writing. Using a newly built corpus of Arabic academic abstracts and social media reviews, the authors generate text from four LLMs with three prompting strategies, then compare word-frequency distributions, vocabulary diversity, and semantic similarity against human writing. They find that machine text overuses the most frequent words, drops off more sharply in the low-frequency tail, and is substantially shorter in social media. A fine-tuned XLM-RoBERTa detector separates machine from human academic abstracts with F1 scores up to 99.9 percent, while social media detection is weaker and cross-model transfer is often poor. The upshot is that Arabic AI text is detectable in academic settings, but informal, dialectal, or style-preserving contexts remain a frontier.

What carries the argument

The load-bearing object is the stylometric signature, operationalized as the top-100 word-frequency distribution and its Zipf-like slope, together with a fine-tuned XLM-RoBERTa classifier that learns to separate human from machine distributions. The frequency analysis is the discovery side: humans show a smooth power-law decay while LLM outputs concentrate probability mass on the highest-ranked words and fall off faster in the tail. The detector is the application side: it reproduces these contrasts in supervised form. Reference-based metrics (BLEU, METEOR, ROUGE-L, BERTScore) serve as a secondary instrument for measuring how close generated text stays to human wording.

What would settle it

Take a set of Arabic abstracts whose human authorship is individually verified (for example, hand-written scans from before the LLM era), match them to generated abstracts by topic and length, and rerun the frequency-distribution contrast and the detector: if the near-perfect academic F1 collapses or the long-tail gap vanishes, the claimed signature is an artifact of corpus differences rather than a stable property of AI text.

Watch

Extended reading notes

Core claim

The central claim is that Arabic text produced by LLMs carries consistent, domain-dependent stylometric signatures, despite being human-like in surface quality. The evidence is a systematic comparison showing that all four models overuse high-frequency words, underuse mid-frequency vocabulary, and show a steeper decay in the long tail of the top-100 word-frequency distribution compared to human academic abstracts; in social media, generated posts are 25 to 72 percent shorter than human posts yet preserve semantic meaning. On the basis of these patterns, the authors fine-tune a multilingual BERT-style classifier (XLM-RoBERTa) and report near-perfect detection in academic abstracts, up to 99.9 percent F1 for GPT-4 and above 99 percent for most cross-model pairs trained on Jais or Llama, while social media cross-model F1 can fall to roughly 29 percent. They interpret this as confirming that detection works where the linguistic signature is strong, and that cross-domain and cross-model generalization remain unresolved.

Load-bearing premise

The entire comparison assumes the 2010 to 2022 ASJP abstracts are truly human-written and that the custom segmentation scripts for splitting Arabic, English, and French abstracts made no systematic errors; no contamination check is reported.

Editorial extensions

If this is right

Academic Arabic abstracts can be screened automatically for machine authorship with near-perfect accuracy, so institutions can deploy this detection in plagiarism and integrity pipelines.
Detectors lose much of their power in social media: cross-model F1 can drop to 29 percent, so reliable informal-text detection likely needs per-model ensembles rather than a single classifier.
Stylometric signatures differ across models and generation strategies, meaning detection systems must be retrained as new Arabic LLMs appear.
Text refinement (polishing) is the hardest-to-detect generation method, since it stays closest to human wording; content-aware generation sits in between.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the frequency-distribution signature is genuine, the same analytical lens could be applied to other low-resource languages to test whether the pattern is language-specific or a general property of LLM sampling.
The unverified human-reference corpus means the headline accuracy should be re-measured on a manually confirmed gold standard before being treated as a deployable guarantee.
A direct extension the authors do not pursue is watermarking or decoding-phase constraints: the brittleness they find in informal registers suggests that detector-only defenses will not scale, and generation-side interventions may be needed.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

What you should know: this is a genuinely useful dataset paper for Arabic machine-generated text detection, but the headline numbers on academic abstracts rest on a human-reference corpus whose quality is not established. The abstract also claims a cross-domain analysis that I could not find in the body.

What is new: they built the first Arabic benchmark that combines four LLMs (ALLaM, Jais, Llama, GPT-4), three generation strategies, and two domains, and they released the datasets publicly. The stylometric analysis is careful and the Zipf plots showing LLM overuse of high-frequency words and steeper long-tail drop-off are informative. The detection experiments are competently run: fine-tuning XLM-RoBERTa, reporting precision/recall, and the cross-model generalization results clearly show the expected gaps.

Soft spots, in order of seriousness. First, the ASJP human abstracts. Section 3.1 admits the site dumps Arabic/English/French abstracts in one block and that they used custom segmentation scripts, but no error rate, manual audit, or AI-contamination check is reported beyond the 2010-2022 publication window. If the segmentation or contamination is bad, the 99.9% F1 could be separating formatting artifacts rather than human Arabic prose. That is a real validity threat, not a nitpick. Second, the abstract says "cross-domain analysis" but the paper does not train on one domain and test on the other; it treats each domain separately. That is an overclaim. Third, in the social media experiments, human posts average 867 words while generated posts average 225-627 words, so a detector could be exploiting length rather than style. No length-controlled experiment is reported. Fourth, all detection results come from a single train/validation/test split with no error bars or repeated runs, so we have no sense of variance.

That said, the social media half uses established human corpora (BRAD/HARD), and the qualitative finding that informal text is harder to detect is probably robust. The paper is not a takedown case; it is a resource paper that needs validation work.

Who this is for: people working on Arabic NLP, text detection, or benchmarking. The dataset alone is worth engaging with. I would send this to peer review, but with a requirement that the authors validate the ASJP segmentation, add length-controlled detection, and correct the abstract. As it stands, it is a strong resource with an unverified foundation.

Referee Report

4 major / 4 minor

Summary. The paper investigates whether Arabic text generated by large language models carries detectable stylometric signatures. The authors build two corpora: 3,000 human academic abstracts from the Algerian Scientific Journals platform (ASJP), with corresponding LLM-generated abstracts produced from three strategies (title-only, title-plus-content, and polishing) using ALLaM, Jais, Llama 3.1, and GPT-4; and 3,500 human social-media reviews from BRAD and HARD, with LLM-generated 'polished' versions. Stylometric analyses examine word-length statistics, top frequent words, Zipf-like frequency distributions, and BLEU/METEOR/ROUGE-L/BERTScore similarities. Detection experiments fine-tune XLM-RoBERTa for binary cross-model generalization and multi-class model identification. The paper reports near-perfect detection in academic abstracts (up to 99.9% F1) and degraded, highly variable cross-model performance in social media content. The authors claim this is the most comprehensive Arabic machine-generated text detection study to date and release datasets and code publicly.

Significance. If the core results are reliable, the paper is a useful contribution to Arabic NLP and AI-text detection. Its strengths include the breadth of generation strategies and model architectures, the combination of stylometric analysis with detector evaluation, and the public release of datasets and generation notebooks, which supports reproducibility. The finding that formal Arabic academic text is highly detectable while informal social media text resists cross-model generalization is a concrete, falsifiable claim of practical value. However, the significance is conditional on the validity of the human-reference corpus and on the stability of the reported near-perfect scores, both of which need strengthening before the headline claims can be accepted.

major comments (4)

[Section 3.1] The human academic corpus is not adequately validated. The ASJP platform stores Arabic, English, and French abstracts in a single text block, and the authors state that they wrote 'custom segmentation scripts' using statistical and rule-based methods, but they report no error rate, no manual audit, and no inter-annotator agreement for these scripts. The only AI-contamination control is the 2010-2022 publication window; no check for machine-translated or machine-assisted abstracts is described. Because the near-perfect F1 scores in Tables 9 and 10 and the stylometric contrasts in Section 5 are built directly on this human gold standard, corrupted or mis-segmented labels could produce the observed separation without reflecting genuine human-Arabic-writing characteristics. Please add a measured segmentation accuracy on a manually labeled sample, an AI-contamination check (for example, an independent detector or perplexity filter), and rerun the key experiments on the validated subset.
[Abstract and Section 6] The abstract claims that 'Our cross-domain analysis confirms generalization challenges previously reported in the literature,' but no cross-domain experiment appears in Section 6. The academic (Section 6.1) and social media (Section 6.2) experiments are each trained and tested within their own domain; there is no training-on-academic/testing-on-social-media or the reverse. Either add a direct cross-domain evaluation or revise the abstract and the relevant sentence to state that generalization challenges are observed across models within each domain, not across domains.
[Section 6] All detection results are reported from a single 75/15/15 split with no error bars, no multiple seeds, and no significance testing. The near-perfect scores in Table 9 (for example, ALLaM-trained detector achieving 99.96% F1 on its own test set) could be split-specific or reflect a particular random initialization. Please report results over at least three to five random splits or seeds with means and standard deviations, and specify whether the reported precision, recall, and F1 refer to the machine-generated class or to a macro average.
[Section 6.1] The academic detection dataset is imbalanced at roughly 1:3 human-to-machine ratio, yet the binary detection tables report only accuracy, precision, recall, and F1 without a class-wise breakdown or balanced metrics. Accuracy can be misleading under this imbalance, and the reader cannot tell how well the detector recognizes human text. Please report per-class precision/recall/F1, or macro-F1 and balanced accuracy, for the binary experiments.

minor comments (4)

[Tables 4 and 7] The Arabic word forms in Tables 4 and 7 appear garbled or reversed (for example, 'ةسﺍﺭدلﺍ'), likely a PDF-extraction artifact; please ensure the tables render correct Arabic script in the final version.
[References] Reference [2] and reference [31] contain placeholder '????' fields; these should be completed before publication.
[Section 4.1.1] In the description of the title-plus-content generation method, the phrase 'We happened to the prompt' should read 'We appended to the prompt.'
[Section 5] The stylometric analysis is descriptive and relies on visual inspection of plots and tables; adding statistical tests or effect sizes for the frequency and length differences would strengthen the claims.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the detection and stylometric results rest on held-out splits and externally sourced human corpora, with no fitted parameter renamed as a prediction.

full rationale

The paper's derivation chain is self-contained as an empirical supervised-learning study. Human academic abstracts come from an external corpus (ASJP), LLM outputs are generated from those abstracts or titles via fixed prompts, and the XLM-RoBERTa detectors are fine-tuned and evaluated on disjoint 75/15/15 splits, with cross-model tests using held-out outputs from models not seen in training. No parameter is fit to the reported 99.9% F1 score, and no target quantity is redefined in terms of the model's own output. The only potentially overlapping citations (ALLaM, Jais) are used to identify the generator models, not to justify the fingerprint claim, so their provenance does not make the core result circular. The validity threats noted in the manuscript and in the surrounding discussion — unmeasured segmentation error for the ASJP human class, absence of an explicit AI-contamination check for that class, and shared prompt/source distributions between training and test generation runs — concern construct validity and generalizability rather than circularity, because the human-vs-machine labels still follow from an external distinction and the test sets are disjoint from training. Accordingly, no circular step can be exhibited by quotation and reduction, and the circularity score is 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No theoretical constants are fitted. The entries below are hand-chosen data-construction and evaluation settings that shape the datasets and therefore the measured detection scores. The axioms capture the data provenance and measurement assumptions the results depend on.

free parameters (4)

Minimum word-count filter for generated abstracts = 30 words
Chosen by authors in Section 4.1.1 to remove invalid generations; reduces the common set from 3,000 and changes the class balance.
Minimum word-count filter for generated social posts = 50 words
Chosen in Section 4.1.2; drops short or invalid outputs and affects the final 3,318-sample social dataset.
Title+Content context window = 400 words
Set in Section 4.1.1 to fit Jais and ALLaM context limits; restricts how much paper content the models see and therefore influences generated abstracts.
Train/validation/test split ratio = 75/15/15
Fixed in Section 6.1; all detection metrics are reported for a single split, so the split choice directly determines the reported F1 values.

assumptions (5)

domain assumption The ASJP, BRAD, and HARD source texts are genuinely human-written and contain no LLM-generated content.
The paper filters ASJP by publication year (2010-2022) but does not apply or report an AI-contamination check; the BRAD/HARD reviews are assumed human because they predate LLM proliferation, yet no verification is described (Section 3).
domain assumption The custom segmentation of ASJP abstracts correctly separates Arabic from English/French segments.
Section 3.1 describes rule-based scripts and validation rules, but no accuracy estimate or human audit is reported.
domain assumption All generation models were run under settings that make cross-model stylometric comparisons meaningful.
The paper specifies prompts but not decoding temperature, top-p, or repetition penalties (Sections 4.1, Table 2); if settings differed, style differences may reflect configuration rather than model identity.
domain assumption XLM-RoBERTa fine-tuning with early stopping and the reported splits yields generalizable detectors rather than split-specific artifacts.
Section 4.2 specifies batch size 64 and early stopping patience 3, but no seeds, number of runs, or confidence intervals are given.
domain assumption BLEU, METEOR, ROUGE-L, and BERTScore provide meaningful similarity measurements for Arabic.
Section 5 uses these metrics with no Arabic-specific validation, though BERTScore is multilingual.

how reviews work

0 comments

Cite this review

Pith. "Pith review of The Arabic AI Fingerprint: Stylometric Analysis and Detection of Large Language Models Text." pith.science (2026). https://pith.science/paper/YLGXNTLO

@misc{pith2026250523276,
  author       = {Pith},
  title        = {Pith review of: The Arabic AI Fingerprint: Stylometric Analysis and Detection of Large Language Models Text},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YLGXNTLO}},
  note         = {Machine review of arXiv:2505.23276}
}

read the original abstract

Large Language Models (LLMs) have achieved unprecedented capabilities in generating human-like text, posing subtle yet significant challenges for information integrity across critical domains, including education, social media, and academia, enabling sophisticated misinformation campaigns, compromising healthcare guidance, and facilitating targeted propaganda. This challenge becomes severe, particularly in under-explored and low-resource languages like Arabic. This paper presents a comprehensive investigation of Arabic machine-generated text, examining multiple generation strategies (generation from the title only, content-aware generation, and text refinement) across diverse model architectures (ALLaM, Jais, Llama, and GPT-4) in academic, and social media domains. Our stylometric analysis reveals distinctive linguistic patterns differentiating human-written from machine-generated Arabic text across these varied contexts. Despite their human-like qualities, we demonstrate that LLMs produce detectable signatures in their Arabic outputs, with domain-specific characteristics that vary significantly between different contexts. Based on these insights, we developed BERT-based detection models that achieved exceptional performance in formal contexts (up to 99.9\% F1-score) with strong precision across model architectures. Our cross-domain analysis confirms generalization challenges previously reported in the literature. To the best of our knowledge, this work represents the most comprehensive investigation of Arabic machine-generated text to date, uniquely combining multiple prompt generation methods, diverse model architectures, and in-depth stylometric analysis across varied textual domains, establishing a foundation for developing robust, linguistically-informed detection systems essential for preserving information integrity in Arabic-language contexts.

Figures

Figures reproduced from arXiv: 2505.23276 by the authors.

**Figure 1.** General overview of our research pipeline. The study integrates multiple data sources with diverse text generation strategies across four different LLMs. The generated texts undergo stylometric analysis. Detection models are then developed and evaluated through cross-model generalization and multi-class identification. PDF, models were explicitly instructed in the prompt to disregard any existing abstracts in the e… view at source ↗

**Figure 2.** Arabic abstracts top 100 human vs LLMs words frequency. stracts (black solid line) demonstrated a relatively smooth power-law decay, particularly evident in the range of ranks 1-50. Additionally, all models consistently overused high-frequency words (ranks 1-5) compared to human writing, with their markers appearing notably above the human reference line in this region. In the mid-frequency range (ranks 5-20), model… view at source ↗

**Figure 3.** Arabic social media top 100 human vs LLMs words frequency. syntactic-level metrics (BLEU: 45.10, METEOR: 54.91, ROUGE-L: 42.26), preserving exact phrasing and structural elements, consistent with its humanlike length and word distribution patterns, and the second in BERTScore. Conversely, LLaMA achieves the highest BERTScore (95.05) despite producing the shortest text and lower surface-level scores, indicating rem… view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Confusion Matrix for Multi-class LLM Detection for Arabic Abstracts Dataset. 6.2. Social Media Content Detection The detection experimentation on Arabic social media content presents different challenges compared to academic abstracts due to the informal na22 [PITH_F…

**Figure 5.** Figure 5: Confusion Matrix for Multi-class LLM Detection for Arabic Social Media Dataset. 7. Conclusion This research investigated machine-generated Arabic text across academic and social media domains, revealing distinctive linguistic signatures that enable effective detection…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Stylometry recognizes human and LLM-generated texts in short samples
cs.CL 2025-07 conditional novelty 5.0 of 10

Stylometric features and tree-based classifiers separate human-written Wikipedia summaries from LLM-generated texts with high cross-validated accuracy on a new seven-class benchmark, though performance drops on other ...

Reference graph

Works this paper leans on

41 extracted references · 18 canonical work pages · cited by 1 Pith paper

[1]

T. B. Brown, Language models are few-shot learners, arXiv preprint arXiv:2005.14165 (2020)

arXiv 2020
[2]

Anthropic, The claude 3 model family: Opus, sonnet, haiku, ????

work page
[3]

Touvron, T

H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al., Llama: Open and efficient foundation language models, arXiv preprint arXiv:2302.13971 (2023)

arXiv 2023
[4]

Black, S

S. Black, S. Biderman, E. Hallahan, Q. Anthony, L. Gao, L. Golding, H. He, C. Leahy, K. McDonell, J. Phang, et al., Gpt-neox-20b: An open- source autoregressive language model, arXiv preprint arXiv:2204.06745 (2022)

arXiv 2022
[5]

Gunasekar, Y

S. Gunasekar, Y. Zhang, J. Aneja, C. C. T. Mendes, A. Del Giorno, S. Gopi, M. Javaheripi, P. Kauffmann, G. de Rosa, O. Saarikivi, et al., Textbooks are all you need, arXiv preprint arXiv:2306.11644 (2023)

arXiv 2023
[6]

URL: https://huggingface.co/CohereForAI/c4ai-command-r-v01

Cohere For AI, c4ai-command-r-v01 (revision 8089a08), 2024. URL: https://huggingface.co/CohereForAI/c4ai-command-r-v01 . doi:10.57967/hf/3139. 26

work page doi:10.57967/hf/3139 2024
[7]

Mitchell, Y

E. Mitchell, Y. Lee, A. Khazatsky, C. D. Manning, C. Finn, Detectgpt: Zero-shot machine-generated text detection using probability curvature, in: International Conference on Machine Learning, PMLR, 2023, pp. 24950–24962

work page 2023
[8]

G. Bao, Y. Zhao, Z. Teng, L. Yang, Y. Zhang, Fast-detectgpt: Efficient zero-shot detection of machine-generated text via conditional probability curvature, arXiv preprint arXiv:2310.05130 (2023)

arXiv 2023

Show all 41 references

[9]

X. Yang, L. Pan, X. Zhao, H. Chen, L. Petzold, W. Y. Wang, W. Cheng, A survey on detection of llms-generated content, arXiv preprint arXiv:2310.15654 (2023)

2023 arXiv
[10]

P. Yu, J. Chen, X. Feng, Z. Xia, Cheat: A large-scale dataset for detecting chatgpt-written abstracts, arXiv preprint arXiv:2304.12008 (2023)

2023 arXiv
[11]

Liang, Y

W. Liang, Y. Zhang, Z. Wu, H. Lepp, W. Ji, X. Zhao, H. Cao, S. Liu, S. He, Z. Huang, et al., Mapping the increasing use of llms in scientific papers, arXiv preprint arXiv:2404.01268 (2024)

2024 arXiv
[12]

Muñoz-Ortiz, C

A. Muñoz-Ortiz, C. Gómez-Rodríguez, D. Vilares, Contrasting linguistic patterns in human and llm-generated news text, Artificial Intelligence Review 57 (2024) 265

2024
[13]

Sengupta, S

N. Sengupta, S. K. Sahu, B. Jia, S. Katipomu, H. Li, F. Koto, W. Mar- shall, G. Gosal, C. Liu, Z. Chen, et al., Jais and jais-chat: Arabic-centric foundation and instruction-tuned open generative large language mod- els, arXiv preprint arXiv:2308.16149 (2023)

2023 arXiv
[14]

Huang, F

H. Huang, F. Yu, J. Zhu, X. Sun, H. Cheng, D. Song, Z. Chen, A. Al- harthi, B. An, J. He, et al., Acegpt, localizing large language models in arabic, arXiv preprint arXiv:2309.12053 (2023)

2023 arXiv
[15]

M. S. Bari, Y. Alnumay, N. A. Alzahrani, N. M. Alotaibi, H. A. Alyahya, S. AlRashed, F. A. Mirza, S. Z. Alsubaie, H. A. Alahmed, G. Alabdul- jabbar, et al., ALLaM: Large language models for arabic and english, arXiv preprint arXiv:2407.15390 (2024). 27

2024 arXiv
[16]

Alshammari, A

H. Alshammari, A. El-Sayed, Airabic: Arabic dataset for performance evaluation of ai detectors, in: 2023 International Conference on Machine Learning and Applications (ICMLA), IEEE, 2023, pp. 864–870

2023
[17]

Alshammari, A

H. Alshammari, A. El-Sayed, K. Elleithy, Ai-generated text detector for arabic language using encoder-based transformer architecture, Big Data and Cognitive Computing 8 (2024) 32

2024
[18]

J. Wu, S. Yang, R. Zhan, Y. Yuan, D. F. Wong, L. S. Chao, A survey on llm-gernerated text detection: Necessity, methods, and future directions, arXiv preprint arXiv:2310.14724 (2023)

2023 arXiv
[19]

Koike, M

R. Koike, M. Kaneko, N. Okazaki, Outfox: Llm-generated essay detec- tion through in-context learning with adversarially generated examples, in: Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 38, 2024, pp. 21258–21266

2024
[20]

Z. Yang, Z. Feng, R. Huo, H. Lin, H. Zheng, R. Nie, H. Chen, The imitation game revisited: A comprehensive survey on recent advances in ai-generated text detection, Expert Systems with Applications 272 (2025) 126694

2025
[21]

X. Hu, W. Ou, S. Acharya, S. H. Ding, R. D’Gama, H. Yu, Tdrlm: Sty- lometric learning for authorship verification by topic-debiasing, Expert Systems with Applications 233 (2023) 120745

2023
[22]

Ramezani, A language-independent authorship attribution approach for author identification of text documents, Expert Systems with Ap- plications 180 (2021) 115139

R. Ramezani, A language-independent authorship attribution approach for author identification of text documents, Expert Systems with Ap- plications 180 (2021) 115139

2021
[23]

Opara, P

C. Opara, P. Modesti, L. Golightly, Evaluating spam filters and stylo- metric detection of ai-generated phishing emails, Expert Systems with Applications 276 (2025) 127044

2025
[24]

Herbold, A

S. Herbold, A. Hautli-Janisz, U. Heuer, Z. Kikteva, A. Trautsch, A large-scale comparison of human-written versus chatgpt-generated es- says, Scientific Reports 13 (2023) 1–11

2023
[25]

Kumarage, J

T. Kumarage, J. Garland, A. Bhattacharjee, K. Trapeznikov, S. Ruston, H. Liu, Stylometric detection of ai-generated text in twitter timelines, arXiv preprint arXiv:2303.03697 (2023). 28

2023 arXiv
[26]

S. Feng, R. Banerjee, Y. Choi, Syntactic stylometry for deception de- tection, in: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Association for Computational Linguistics, 2012, pp. 171–175

2012
[27]

Takahashi, K

S. Takahashi, K. Tanaka-Ishii, Evaluating computational language mod- els with scaling properties of natural language, Computational Linguis- tics 45 (2019) 481–513

2019
[28]

B. Guo, X. Zhang, Z. Wang, M. Jiang, J. Nie, Y. Ding, J. Yue, Y. Wu, How close is chatgpt to human experts? comparison corpus, evaluation, and detection, arXiv preprint arxiv:2301.07597 (2023)

2023 arXiv
[29]

Z. Su, X. Wu, W. Zhou, G. Ma, S. Hu, Hc3 plus: A semantic-invariant human chatgpt comparison corpus, arXiv preprint arXiv:2309.02731 (2023)

2023 arXiv
[30]

Y. Li, Q. Li, L. Cui, W. Bi, Z. Wang, L. Wang, L. Yang, S. Shi, Y. Zhang, Mage: Machine-generated text detection in the wild, arXiv preprint arXiv:2305.13242 (2023)

2023 arXiv
[31]

Ai detector - the original ai checker for chatgpt & more, https: //gptzero.me/, ???? (Accessed on 12/04/2024)

2024
[32]

Antoun, F

W. Antoun, F. Baly, H. Hajj, Araelectra: Pre-training text discrimina- tors for arabic language understanding, arXiv preprint arXiv:2012.15516 (2020)

2020 arXiv
[33]

Conneau, Unsupervised cross-lingual representation learning at scale, arXiv preprint arXiv:1911.02116 (2019)

A. Conneau, Unsupervised cross-lingual representation learning at scale, arXiv preprint arXiv:1911.02116 (2019)

2019 arXiv
[34]

Elnagar, O

A. Elnagar, O. Einea, Brad 1.0: Book reviews in arabic dataset, in: 2016 IEEE/ACS 13th International Conference of Computer Systems and Applications (AICCSA), IEEE, 2016, pp. 1–8

2016
[35]

Elnagar, Y

A. Elnagar, Y. S. Khalifa, A. Einea, Hotel arabic-reviews dataset con- struction for sentiment analysis applications, Intelligent natural lan- guage processing: Trends and applications (2018) 35–52

2018
[36]

Grattafiori, A

A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, et al., The llama 3 herd of models, arXiv preprint arXiv:2407.21783 (2024). 29

2024 arXiv
[37]

Achiam, S

J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al., Gpt-4 technical report, arXiv preprint arXiv:2303.08774 (2023)

2023 arXiv
[38]

Papineni, S

K. Papineni, S. Roukos, T. Ward, W.-J. Zhu, Bleu: a method for automatic evaluation of machine translation, in: Proceedings of the 40th annual meeting of the Association for Computational Linguistics, 2002, pp. 311–318

2002
[39]

Banerjee, A

S. Banerjee, A. Lavie, Meteor: An automatic metric for mt evaluation with improved correlation with human judgments, in: Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, 2005, pp. 65–72

2005
[40]

Lin, Rouge: A package for automatic evaluation of summaries, in: Text summarization branches out, 2004, pp

C.-Y. Lin, Rouge: A package for automatic evaluation of summaries, in: Text summarization branches out, 2004, pp. 74–81

2004
[41]

Zhang, V

T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, Y. Artzi, Bertscore: Evaluating text generation with bert, arXiv preprint arXiv:1904.09675 (2019). 30

2019 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

T. B. Brown, Language models are few-shot learners, arXiv preprint arXiv:2005.14165 (2020)

arXiv 2020

[2] [2]

Anthropic, The claude 3 model family: Opus, sonnet, haiku, ????

work page

[3] [3]

Touvron, T

H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al., Llama: Open and efficient foundation language models, arXiv preprint arXiv:2302.13971 (2023)

arXiv 2023

[4] [4]

Black, S

S. Black, S. Biderman, E. Hallahan, Q. Anthony, L. Gao, L. Golding, H. He, C. Leahy, K. McDonell, J. Phang, et al., Gpt-neox-20b: An open- source autoregressive language model, arXiv preprint arXiv:2204.06745 (2022)

arXiv 2022

[5] [5]

Gunasekar, Y

S. Gunasekar, Y. Zhang, J. Aneja, C. C. T. Mendes, A. Del Giorno, S. Gopi, M. Javaheripi, P. Kauffmann, G. de Rosa, O. Saarikivi, et al., Textbooks are all you need, arXiv preprint arXiv:2306.11644 (2023)

arXiv 2023

[6] [6]

URL: https://huggingface.co/CohereForAI/c4ai-command-r-v01

Cohere For AI, c4ai-command-r-v01 (revision 8089a08), 2024. URL: https://huggingface.co/CohereForAI/c4ai-command-r-v01 . doi:10.57967/hf/3139. 26

work page doi:10.57967/hf/3139 2024

[7] [7]

Mitchell, Y

E. Mitchell, Y. Lee, A. Khazatsky, C. D. Manning, C. Finn, Detectgpt: Zero-shot machine-generated text detection using probability curvature, in: International Conference on Machine Learning, PMLR, 2023, pp. 24950–24962

work page 2023

[8] [8]

G. Bao, Y. Zhao, Z. Teng, L. Yang, Y. Zhang, Fast-detectgpt: Efficient zero-shot detection of machine-generated text via conditional probability curvature, arXiv preprint arXiv:2310.05130 (2023)

arXiv 2023