LLMs as Research Tools: A Large Scale Survey of Researchers' Usage and Perceptions
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:E5TLL7ELrecord.jsonopen to challenge →
read the original abstract
The rise of large language models (LLMs) has led many researchers to consider their usage for scientific work. Some have found benefits using LLMs to augment or automate aspects of their research pipeline, while others have urged caution due to risks and ethical concerns. Yet little work has sought to quantify and characterize how researchers use LLMs and why. We present the first large-scale survey of 816 verified research article authors to understand how the research community leverages and perceives LLMs as research tools. We examine participants' self-reported LLM usage, finding that 81% of researchers have already incorporated LLMs into different aspects of their research workflow. We also find that traditionally disadvantaged groups in academia (non-White, junior, and non-native English speaking researchers) report higher LLM usage and perceived benefits, suggesting potential for improved research equity. However, women, non-binary, and senior researchers have greater ethical concerns, potentially hindering adoption.
This paper has not been read by Pith yet.
Forward citations
Cited by 5 Pith papers
-
Making a Name for Myself: On Academic Naming Policies and their Impact
Empirical mixed-methods study finds CS venues with accessible name change policies show fewer citation errors (899 vs 996 per 1,000 papers) and 92% drop in deadnaming of transgender researchers from 2019-2024.
-
Thinking Like a Scientist? A Structural Study of LLM-Generated Research Methods
LLMs given only research questions from 1000 arXiv CS papers recommend a narrower set of methods than the original papers, with effective model-entity diversity dropping from 1232 to 59-96 and stronger agreement among...
-
How Researchers Navigate Accountability, Transparency, and Trust When Using AI Tools in Early-Stage Research: A Think-Aloud Study
A think-aloud study reveals that AI tools in early research misrepresent uncertainty, obscure provenance, and create fragile trust, leading researchers to develop compensatory strategies to preserve scholarly judgment.
-
Do Large Language Models know Which Published Articles have been Retracted?
LLMs fail to recognize most retracted articles from titles and abstracts alone, with over 80% error rate, but have low false positive rates on non-retracted articles.
-
Read This Paper to Get $50 Million:* An Analysis of Mobile Messaging Scams Using Reddit Data
Reddit data analysis shows reply-based mobile scams growing nearly twice as fast as click-based ones while evading commercial and open-source detectors.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.