REVIEW 8 cited by
Delving into LLM-assisted writing in biomedical publications through excess vocabulary
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) like ChatGPT can generate and revise text with human-level performance. These models come with clear limitations: they can produce inaccurate information, reinforce existing biases, and be easily misused. Yet, many scientists use them for their scholarly writing. But how wide-spread is such LLM usage in the academic literature? To answer this question for the field of biomedical research, we present an unbiased, large-scale approach: we study vocabulary changes in over 15 million biomedical abstracts from 2010--2024 indexed by PubMed, and show how the appearance of LLMs led to an abrupt increase in the frequency of certain style words. This excess word analysis suggests that at least 13.5% of 2024 abstracts were processed with LLMs. This lower bound differed across disciplines, countries, and journals, reaching 40% for some subcorpora. We show that LLMs have had an unprecedented impact on scientific writing in biomedical research, surpassing the effect of major world events such as the Covid pandemic.
Forward citations
Cited by 8 Pith papers
-
Using Large Language Models for Idea Generation in Innovation
GPT-4-generated product ideas had higher average purchase intent than student ideas and made up 35 of the top 40 ideas, while being rated less novel and more similar to each other.
-
Artificial Epanorthosis: Why large language models overuse a classical rhetorical figure, and how to mitigate it
LLMs overuse the 'not X, but Y' self-correction pattern in persuasive registers and underuse it in informal Q&A; a prompt or a detachable LoRA dial adjusts it to human levels.
-
Word Overuse and Alignment in Large Language Models: The Influence of Learning from Human Feedback
People prefer text containing the words that an instruction-tuned model uses far more than its base version, linking human feedback training to LLM word overuse.
-
Model Misalignment and Language Change: Traces of AI-Associated Language in Unscripted Spoken English
After ChatGPT's release, science and tech podcast speakers used AI-associated words like 'surpass' and 'align' more often, while control synonyms showed no average shift.
-
Exploring the change in scientific readability following the release of ChatGPT
arXiv abstracts became less readable between 2010 and 2024, with a statistically significant jump in complexity in 2023 and 2024 following the release of ChatGPT.
-
Exploring the Structure of AI-Induced Language Change in Scientific English
In PubMed abstracts, AI-associated 'spiking' words rise together with their synonyms rather than replacing them, and declining words show less systematic, more organic patterns.
-
GPT Editors, Not Authors: The Stylistic Footprint of LLMs in Academic Preprints
Across 2,408 arXiv preprints, LLM-typical word usage does not cluster in any section, indicating that AI assistance, when used, is uniform rather than limited to specific parts of a paper.
-
What Shapes Writers' Decisions to Disclose AI Use?
A literature synthesis identifies 12 procedural, social, and personal factors that may shape writers' decisions to disclose AI use.
Discussion (0). Sign in to comment.