Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

Comparing LLM-generated and human-authored news text using formal syntactic theory

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that human-authored and LLM-generated news text differ systematically in their distributions of fine-grained HPSG grammar types, and that any single human author's text is grammatically closer to an LLM's output than to…

desk verdict A genuinely new formal-grammar lens on LLM vs. human text, with a strong pairwise result, but the missing per-corpus parse coverage leaves the central claim one important robustness check short. read the letter →

arxiv 2506.01407 v1 pith:B46LWJCM submitted 2025-06-02 cs.CL

Olga Zamaraeva , Dan Flickinger , Francis Bond , Carlos Gómez-Rodríguez This is my paper
classification cs.CL
keywords largelanguagemodelshuman-authoredtextHead-drivenPhraseStructureGrammarEnglishResourcesyntactictypedistributionsnewsdiversitycosinesimilarity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that, at the level of detailed syntactic structure, LLM-generated news text is measurably different from human-written news text, and that the difference has a shape: LLM output looks like an 'averaged' version of human grammar. Using the English Resource Grammar, a broad-coverage implementation of Head-driven Phrase Structure Grammar, the authors parse New York Times lead paragraphs written by humans and by six language models and compare the frequency distributions of HPSG construction types, lexical types, and lexical rules. They find that the six LLMs cluster tightly together, that human texts sit apart from all of them, and that a given human author is more similar to an LLM than to another human author. The paper also reports that human writers use more low-frequency, stylistically special constructions while LLMs rely more on the most general clause-building constructions. If true, this gives a grammar-based axis for characterizing synthetic text and for studying how LLMs flatten stylistic variation.

What carries the argument

The load-bearing object is the HPSG type hierarchy as implemented in the English Resource Grammar: a precision grammar whose lexicon and grammar rules assign every parsed sentence a derivation made of named construction, lexical-type, and lexical-rule types. The paper counts how often each type occurs in each corpus, turns these counts into normalized frequency vectors, and compares the vectors with cosine similarity and with Shannon entropy and Gini-Simpson diversity indices. The type hierarchy is what makes the analysis possible: raw words are grouped into types that encode syntactic behavior, so differences that are invisible in token or part-of-speech sequences become visible as different types being selected.

What would settle it

Compute ERG parse-success rates separately for the human NYT, each LLM corpus, WSJ, and Wikipedia, and re-compare only sentences that parse. If LLM sentences fail more often, or fail on exactly the low-frequency constructions that humans use, the cosine and diversity gaps could shrink or vanish once parse success is controlled.

Watch

Extended reading notes

Core claim

The paper's central discovery is a distributional fact about syntax rather than vocabulary: over the ERG's hierarchy of 298 syntactic constructions, 1,398 lexical types, and 44,366 lexical entries, human and LLM authors leave different footprints. When the relative frequencies of HPSG types are compared with cosine similarity, all LLM-generated corpora are nearly identical to each other, while human NYT text sits at the edge of the LLM cluster. In the author-by-author comparison, the difference between two human writers exceeds the difference between a human writer and an LLM, which the authors interpret as LLM output being 'averaged' with respect to grammatical features. Human texts show higher Shannon diversity of syntactic constructions, lower diversity of lexical types, and greater use of low-frequency constructions such as participial modifiers, appositions, and coordinated modifiers; LLMs use more head-complement and subject-head constructions, relative clauses, quantity phrases, and questions. Morphological-rule distributions, by contrast, are similar between humans and LLMs except for plural nouns, which humans use more.

Load-bearing premise

The argument depends on the English Resource Grammar parsing human and LLM text with roughly equal success and with the same kind of structural analysis; the paper reports overall 94% coverage for well-edited English but gives no per-corpus parse rates.

Editorial extensions

If this is right

  • LLM-generated NYT-style text can be characterized by its HPSG type distribution, not just by lexical or rhetorical tells.
  • Because any human author is grammatically closer to an LLM than to another human author, detectors and stylometric models should treat 'averageness' rather than oddity as the signature of synthetic text.
  • The differences concentrate in a small set of general constructions such as head-complement and subject-head phrases, giving a focused set of grammar types to monitor.
  • Human reliance on low-frequency, stylistically special constructions explains why diversity metrics separate humans from LLMs for syntax but not for lexical types.
  • The similarity of morphological-rule distributions suggests that in this genre morphology tracks style and genre more than authorship.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors do not test this, but the same type-frequency vectors could be fed directly into authorship-attribution classifiers, where the tightly clustered LLM outputs would act as a single synthetic 'author' class.
  • If LLM output is grammatically averaged, then prompting for a specific human style may shift lexical choices while leaving the underlying construction distribution close to the LLM centroid; this is testable with style-transfer prompts.
  • Because the per-type p-values do not survive FDR correction, a natural extension is to compare more human authors and more models at fixed sample sizes to see whether the head-complement and long-tail gaps are stable.
  • The approach could extend to other genres and languages only where a broad-coverage precision grammar exists, so the immediate generalization path is to build comparable grammars rather than to rely on surface statistics.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper compares the syntactic type distributions of human-authored New York Times lead paragraphs with texts generated by six LLMs (LLaMA variants, Falcon, Mistral) using the English Resource Grammar (ERG), a broad-coverage HPSG grammar. The authors compute frequencies of HPSG construction types, lexical types, and lexical rules from parsed sentences, then compare them via cosine similarity and diversity indices (Shannon entropy, Gini-Simpson). The main findings are that LLMs cluster together and differ from human text in syntactic and lexical type distributions; that human authors vary more among themselves than they do from LLMs, making LLM text look 'average'; and that human text is more diverse in syntactic constructions, while LLM text can be more diverse in lexical types. The paper also lists specific constructions and lexical entries that appear to differentiate the two kinds of text. The authors argue this provides a grammar-based, theory-driven axis for characterizing synthetic text.

Significance. The methodological contribution is substantial: applying a hand-built, independently developed precision grammar (ERG) as a fixed measurement instrument avoids the circularity of training-based detectors, and the authors release the parsed data and analysis code. The aggregate diversity gap for syntactic constructions is supported by a 10,000-resample permutation test (p < 0.01), and the cosine-similarity separation is visually clear. The honest disclosure in Appendix D that none of the construction-level Mann-Whitney p-values survive FDR correction is commendable, although it undermines some of the paper's own claims. If the parse-coverage confound is addressed, the 'average human' hypothesis and the diversity reversal for lexical types would be interesting and testable contributions to the study of LLM text.

major comments (3)
  1. [§3.2 / §4] Section 3.2 (and the data description in §4): The analysis is performed exclusively on sentences for which the ERG returns a parse, but the paper reports no parse-success rates for the individual corpora. The only coverage figure given is the general claim that the ERG parses 94% of 'reasonably well-edited English text' (2025 release documentation). If LLM-generated text fails to parse at a different rate than human text, or fails on systematically different constructions (e.g., unusual punctuation, novel compounds, non-standard coordination), then the type-frequency vectors in Figures 2–9 are computed over different subsets of the two text populations. The observed distributional gaps, including the diversity differences and the 'average human' result, could then be an artifact of differential parse failure rather than a property of the authors. The Limitations section does not mention this. I request that the authors report, for each dataset (each LLM, original NYT, WSJ, Wikipedia), the number and proportion of sentences parsed, and ideally analyze the characteristics of unparsed sentences. A conservative robustness check would be to recompute the main diversity and cosine-similarity results on comparable subsets of parsed sentences (e.g., matched by sentence length or parse state).
  2. [§5.1.1 / Appendix D] Section 5.1.1 and Appendix D: The paper identifies 'salient' constructional differences (e.g., head-complement, adjunct-head double modification, punctuation constructions) using Mann-Whitney U-tests with p < 0.05, but Appendix D explicitly states that after FDR correction none of the p-values remain significant. With hundreds of HPSG types tested, an uncorrected 5% threshold is expected to produce a large number of false positives; for instance, the ~80 lexical types listed in Table 12 are roughly compatible with chance among the ~1,100 types. The main text nevertheless states that constructions were 'selected ... based on the statistical significance of the comparison between relative frequencies' (§6.1) and presents Table 4 as evidence of systematic differences. This is misleading. The authors should either (a) apply a multiple-comparison correction (e.g., Benjamini-Hochberg) and report which (if any) types survive, or (b) explicitly label all construction-level examples as exploratory, hypothesis-generating observations, and adjust the wording in §5.1.1 and §6.1 accordingly. The aggregate cosine-similarity and diversity results are not affected by this comment, but the specific construction claims as written are not statistically supported.
  3. [§5.1.5] Section 5.1.5 and Figures 6–7: The central claim that 'a human-authored text is more similar to an LLM-generated text than to another human-authored text (by a different author)' is presented as a demonstrated finding, but no significance test is reported for the pairwise cosine similarities. With only 12 human authors (each with more than 100 sentences) and the inherent noise of frequency estimates from small samples, the difference in the distributions could be within sampling variation. The paper should report a permutation or bootstrap test on the pairwise similarities (human-human vs. human-LLM), with effect sizes and confidence intervals, and state the direction and magnitude of the difference. The current presentation is descriptive only, which is insufficient for a claim introduced with 'we show for the first time.'
minor comments (5)
  1. [Figure 5] Figure 5 caption: the phrase 'the blue bar representing LLM' appears twice and is presumably a typo for 'representing human writers' or 'representing the human average'; please clarify what the dots and the blue bar denote.
  2. [§5.1.1, footnote 13] The main text notes that p-values < 0.05 are listed in Appendix D but does not mention until the appendix that none survive FDR correction; readers should be alerted to this in the main text, not only in the appendix.
  3. [§5.1.4, Table 3] The sampling procedure for the 'sample 25K' in Table 3 is not described (e.g., random sampling without replacement, seed, whether it is per model or pooled); please add a precise description so the comparison is reproducible.
  4. [§5.1.4] The 'collective LLM author' construct, formed by pooling all six LLM outputs, is not a well-defined author, so the comparison of its lexical diversity to human authors should be explicitly framed as a methodological experiment rather than a property of any real author.
  5. [§4, Table 1] Table 1, reproduced from Muñoz-Ortiz et al., has rows for LLaMa with sentence counts that vary slightly (37,825; 37,800; 37,568; 38,107) but no column heading explaining these values; please add a note or column to make the table self-contained.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the ERG-based HPSG measurements are a fixed, independently developed analytic instrument, not fitted to the human-versus-LLM comparison.

full rationale

The paper's derivation chain is self-contained with respect to its measurement claims. It parses fixed corpora (NYT, WSJ, Wikipedia, and the LLM outputs from Muñoz-Ortiz et al. 2024) with the English Resource Grammar, a broad-coverage HPSG grammar developed over decades for linguistic purposes independent of this comparison, and then computes type-frequency distributions, cosine similarities, and diversity indices directly from the resulting parses. The core quantities are therefore determined by the grammar plus the texts, not by any parameter fitted to the human/LLM distinction. The only notable self-citation is Dan Flickinger's authorship of the ERG (Flickinger 2000, 2011), but that citation is not load-bearing in a reductive way: the ERG is an external, publicly released resource with documented coverage and independent applications, and the paper does not invoke an unverified uniqueness theorem or smuggle in an ansatz via self-citation. No equation defines a reported result in terms of the comparison itself, and no fitted input is later relabeled as a prediction. The unmeasured possibility of differential parse coverage between human and LLM corpora is a validity concern about the empirical precondition, not a circularity of the derivation. Under the stated hard rules, I find no step that reduces by construction or by self-citation to its own inputs, so the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The analysis introduces no free parameters and no new theoretical entities. Its claims depend on the fixed measurement instrument (ERG/HPSG), on dataset comparability, and on the assumption that the chosen distributional statistics preserve meaningful syntactic differences.

assumptions (4)
  • domain assumption The ERG parses human-authored and LLM-generated text with comparable coverage and no systematic structural bias.
    Coverage is stated as 94% of reasonably well-edited English (Section 3.2), but no per-corpus parse success rates are reported for the LLM versus human datasets; differential parse failure would distort type distributions.
  • domain assumption Texts generated by prompting a headline plus the first three words of a lead are comparable to full human-authored lead paragraphs for the purpose of authorship comparison.
    The comparison inherits the Muñoz-Ortiz et al. protocol (Section 4); human leads are self-contained summaries, while LLM continuations are a different communicative task, so authorship differences may be confounded with task.
  • domain assumption Relative frequencies of HPSG construction types and lexical types constitute a meaningful feature space for detecting authorship differences.
    The paper does not validate that these features outperform or complement existing baselines (e.g., UD or POS templates), though prior work gives reason to expect some signal.
  • domain assumption Cosine similarities near 0.99, visualized via PCA, reflect interpretable distinctions rather than numerical noise.
    No uncertainty intervals are given for the cosine values; PCA can exaggerate small absolute differences, so the 'clear' separation in Figures 2-4 should be treated cautiously.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Comparing LLM-generated and human-authored news text using formal syntactic theory." pith.science (2026). https://pith.science/paper/B46LWJCM

@misc{pith2026250601407,
  author       = {Pith},
  title        = {Pith review of: Comparing LLM-generated and human-authored news text using formal syntactic theory},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B46LWJCM}},
  note         = {Machine review of arXiv:2506.01407}
}
read the original abstract

This study provides the first comprehensive comparison of New York Times-style text generated by six large language models against real, human-authored NYT writing. The comparison is based on a formal syntactic theory. We use Head-driven Phrase Structure Grammar (HPSG) to analyze the grammatical structure of the texts. We then investigate and illustrate the differences in the distributions of HPSG grammar types, revealing systematic distinctions between human and LLM-generated writing. These findings contribute to a deeper understanding of the syntactic behavior of LLMs as well as humans, within the NYT genre.

Figures

Figures reproduced from arXiv: 2506.01407 by the authors.

Figure 1
Figure 1. Part of the HPSG type hierarchy (simplified; [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Cosine similarity: syntactic types [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Cosine similarity: lexical rules Construction type ERG Data syntactic 298 289 lexical type 1,398 1,105 lexical entry 44,366 27,311 morphological rule 100 99 [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: LLM use of syntactic constructions compared to human writers. Cases of particular interest are where the [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Humans vary more from one another than they do from an LLM, and LLMs vary little from each other [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 8
Figure 8. Figure 8: Construction Diversity (Shannon Index) [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 10
Figure 10. Figure 10: Diversity (Shannon and Simpson Indices) [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Word Overuse and Alignment in Large Language Models: The Influence of Learning from Human Feedback

    cs.CL 2025-08 conditional novelty 6.0 of 10

    People prefer text containing the words that an instruction-tuned model uses far more than its base version, linking human feedback training to LLM word overuse.

  2. Model Misalignment and Language Change: Traces of AI-Associated Language in Unscripted Spoken English

    cs.CL 2025-08 conditional novelty 6.0 of 10

    After ChatGPT's release, science and tech podcast speakers used AI-associated words like 'surpass' and 'align' more often, while control synonyms showed no average shift.

Reference graph

Works this paper leans on

40 extracted references · 27 canonical work pages · cited by 2 Pith papers

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, M \'e rouane Debbah, \'E tienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, et al. 2023. The F alcon series of open language models. arXiv preprint arXiv:2311.16867

    arXiv 2023

  4. [4]

    AJ Alvero, Jinsook Lee, Alejandra Regla-Vargas, Ren \'e F Kizilcec, Thorsten Joachims, and Anthony Lising Antonio. 2024. Large language models, social demography, and hegemony: comparing authorship in human and synthetic text. Journal of Big Data, 11(1):138

    work page 2024

  5. [5]

    Emily Bender and Jeff Good. 2005. Implementation for discovery: A bipartite lexicon to support morphological and syntactic analysis. In Proceedings from the annual meeting of the Chicago Linguistic Society, volume 41, pages 1--16. Chicago Linguistic Society

    work page 2005

  6. [6]

    Bender, Joshua Crowgey, Michael Wayne Goodman, Kristen Howell, Haley Lepp, Fei Xia, and Olga Zamaraeva

    Emily M. Bender, Joshua Crowgey, Michael Wayne Goodman, Kristen Howell, Haley Lepp, Fei Xia, and Olga Zamaraeva. 2020. Aggregation: Building computational resources automatically from igt. Invited poster at Reflections on the Impact of DEL-funded Research Over Fifteen Years, LSA 2020, New Orleans, LA

    work page 2020

  7. [7]

    Bender, Scott Drellishak, Antske Fokkens, Laurie Poulson, and Safiyyah Saleem

    Emily M. Bender, Scott Drellishak, Antske Fokkens, Laurie Poulson, and Safiyyah Saleem. 2010. https://doi.org/10.1007/s11168-010-9070-1 Grammar customization . Research on Language and Computation, 8:1--50

    doi:10.1007/s11168-010-9070-1 2010

  8. [8]

    Bender, Dan Flickinger, and Stephan Oepen

    Emily M. Bender, Dan Flickinger, and Stephan Oepen. 2002. The G rammar M atrix: A n open-source starter-kit for the rapid development of cross-linguistically consistent broad-coverage precision grammars. In Proceedings of the W orkshop on Grammar Engineering and Evaluation at the 19th I nternational C onference on C omputational L inguistics , pages 8--14, Taipei

    work page 2002

Show all 40 references
  1. [9]

    Douglas Biber. 1991. Variation across speech and writing. Cambridge University Press

    1991

  2. [10]

    Douglas Biber. 1995. Dimensions of register variation: A cross-linguistic comparison. Cambridge University Press

    1995

  3. [11]

    Douglas Biber and Susan Conrad. 2019. Register, genre, and style. Cambridge University Press

    2019

  4. [12]

    Jan Buys and Phil Blunsom. 2017. Robust incremental neural semantic graph parsing. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1215--1226

    2017

  5. [13]

    Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, et al. 2018. Universal sentence encoder for E nglish. In Proceedings of the 2018 conference on empirical methods in natural language...

    2018

  6. [14]

    Yufei Chen, Weiwei Sun, and Xiaojun Wan. 2018. https://doi.org/10.18653/v1/P18-1038 Accurate SHRG -based semantic parsing . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 408--418, Melbourne, Australia....

    2018 doi

  7. [15]

    Ann Copestake. 2002. Definitions of typed feature structures. In Stephan Oepen, Dan Flickinger, Jun-ichi Tsujii, and Hans Uszkoreit, editors, Collaborative Language Engineering, pages 227--230. CSLI Publications, Stanford, CA

    2002

  8. [16]

    Berthold Crysmann and Woodley Packard. 2012. Towards efficient HPSG generation for G erman, a non-configurational language. In COLING, pages 695--710

    2012

  9. [17]

    Dan Flickinger. 2000. On building a more efficient grammar by exploiting types. Natural Language Engineering, 6(01):15--28

    2000

  10. [18]

    Dan Flickinger. 2011. Accuracy v.\ robustness in grammar engineering. In Emily M.\ Bender and Jennifer E. Arnold, editors, Language from a Cognitive Perspective: Grammar, Usage and Processing, pages 31--50. CSLI Publications, Stanford, CA

    2011

  11. [19]

    Valerie Hajdik, Jan Buys, Michael W Goodman, and Emily M Bender. 2019. Neural text generation from rich semantic representations. In Proceedings of NAACL-HLT , pages 2259--2266

    2019

  12. [20]

    Charles R Harris, K Jarrod Millman, St \'e fan J Van Der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J Smith, et al. 2020. Array programming with numpy. Nature, 585(7825):357--362

    2020

  13. [21]

    Kristen Howell and Emily M Bender. 2022. Building analyses from syntactic inference in local languages: An HPSG grammar inference system. Northern European Journal of Language Technology, 8(1)

    2022

  14. [22]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7B . arXiv preprint arXiv:2310.06825

    2023 arXiv

  15. [23]

    Tom S Juzek and Zina B Ward. 2025. Why does ChatGPT “delve” so much? E xploring the sources of lexical overrepresentation in large language models. In Proceedings of the 31st International Conference on Computational Linguistics, pages 6397--6411

    2025

  16. [24]

    Zi Lin, Jeremiah Zhe Liu, and Jingbo Shang. 2022. Towards collaborative neural-symbolic graph semantic parsing via uncertainty. Findings of the Association for Computational Linguistics: ACL 2022

    2022

  17. [25]

    Mitchell Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of E nglish: The P enn T reebank. University of Pennsylvania Department of Computer and Information Science Technical Report No. MS-CIS-93-87

    1993

  18. [26]

    Wes Mc K inney. 2010. https://doi.org/10.25080/Majora-92bf1922-00a D ata S tructures for S tatistical C omputing in P ython . In P roceedings of the 9th P ython in S cience C onference , pages 56 -- 61

    2010 doi

  19. [27]

    Alberto Muñoz-Ortiz, Carlos Gómez-Rodríguez, and David Vilares. 2024. Contrasting linguistic patterns in human and LLM -generated news text. Artificial Intelligence Review, 57(10):265

    2024

  20. [28]

    Arun Balajiee Lekshmi Narayanan, Priti Oli, Jeevan Chapagain, Mohammad Hassany, Rabin Banjade, Peter Brusilovsky, and Vasile Rus. 2024. Explaining code examples in introductory programming courses: LLM vs humans. In AI for Education: Bridging Innovation and Responsibility at t...

    2024

  21. [29]

    Joakim Nivre, Marie-Catherine De Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, et al. 2016. Universal dependencies v1: A multilingual treebank collection. In Proceedings of the Tenth Intern...

    2016

  22. [30]

    Stephan Oepen, Dan Flickinger, Kristina Toutanova, and Christopher D Manning. 2004. Lin GO R edwoods. Research on Language and Computation, 2(4):575--596

    2004

  23. [31]

    Mose Park, Yunjin Choi, and Jong-June Jeon. 2025. Does a large language model really speak in human-like language? arXiv preprint arXiv:2501.01273

    2025 arXiv

  24. [32]

    Pedregosa, G

    F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in P ython. Journal of Machine L...

    2011

  25. [33]

    Carl Pollard and Ivan A. Sag. 1994. H ead- D riven P hrase S tructure G rammar . Studies in Contemporary Linguistics. The University of Chicago Press and CSLI Publications, Chicago, IL and Stanford, CA

    1994

  26. [34]

    Alex Reinhart, David West Brown, Ben Markey, Michael Laudenbach, Kachatad Pantusen, Ronald Yurko, and Gordon Weinberg. 2024. Do LLM s write like humans? V ariation in grammatical and rhetorical styles. arXiv preprint arXiv:2410.16107

    2024 arXiv

  27. [35]

    Morgan Sandler, Hyesun Choung, Arun Ross, and Prabu David. 2024. A linguistic comparison between human and ChatGPT -generated conversations. arXiv preprint arXiv:2401.16587

    2024 arXiv

  28. [36]

    Tony Berber Sardinha. 2024. AI -generated vs human-authored texts: A multidimensional comparison. Applied Corpus Linguistics, 4(1):100083

    2024

  29. [37]

    Chantal Shaib, Yanai Elazar, Junyi Jessy Li, and Byron C Wallace. 2024. Detection and measurement of syntactic templates in generated text. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 6416--6431

    2024

  30. [38]

    Efstathios Stamatatos. 2009. https://doi.org/10.1002/asi.21001 A survey of modern authorship attribution methods . Journal of the American Society for Information Science and Technology, 60(3):538--556

    2009 doi

  31. [39]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  32. [40]

    Olga Zamaraeva, Chris Curtis, Guy Emerson, Antske Fokkens, Michael Wayne Goodman, Kristen Howell, TJ Trimble, and Emily M Bender. 2022. 20 years of the G rammar M atrix: C ross-linguistic hypothesis testing of increasingly complex interactions. Journal of Language Modelling, 1...

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.