Reviewer slates that are topically diverse, seniority-diverse, or from distinct publication networks produce broader or less redundant reviews, while geographic and organizational diversity show little or no effect.
Can we automate scientific reviewing? Journal of Artificial Intelligence Research, 75:171–212
8 Pith papers cite this work, alongside 73 external citations. Polarity classification is still indexing.
8
Pith papers citing it
73
external citations · Crossref
citation-role summary
background 2
citation-polarity summary
roles
background 2polarities
background 2representative citing papers
Through 27 interviews and a design probe, the study shows area chairs differ substantially in engagement style and are cautiously open to AI assistance that adapts to their workflows.
A simple title-and-abstract model predicts citation counts better than review scores and outperforms LLM reviewers in matching human review scores, but remains below human consistency.
Peer review reports in AI conferences have grown longer and more standardized after LLMs, with increased emphasis on surface-level clarity and summaries at the expense of deeper critiques on originality and replicability.
A systematic review of LLM-based systems for hypothesis discovery, experiment planning, scientific writing, and peer review, including benchmarks, evaluation methods, and open challenges.
The paper proposes a four-role framework for LLMs in scientific innovation and reviews methods, benchmarks, and limitations across Assistant, Collaborator, Scientist, and Evaluator roles.
A persistent, structured prompt loaded into an LLM chat session can guide reasoning models through critical analysis of experimental chemistry papers, but the evidence is a single qualitative case study.
A survey of LLM-based automated scholarly paper review, cataloging models, datasets, methods, and publisher policies as of 2023-2024.
citing papers explorer
-
Causal Effect of Group Diversity on Redundancy and Coverage in Peer-Reviewing
Reviewer slates that are topically diverse, seniority-diverse, or from distinct publication networks produce broader or less redundant reviews, while geographic and organizational diversity show little or no effect.
-
Hands-Off or Hands-On? Variation in Area Chair Practices and Implications for AI Support
Through 27 interviews and a design probe, the study shows area chairs differ substantially in engagement style and are cautiously open to AI assistance that adapts to their workflows.
-
Automatic Evaluation Metrics for Artificially Generated Scientific Research
A simple title-and-abstract model predicts citation counts better than review scores and outperforms LLM reviewers in matching human review scores, but remains below human consistency.
-
Impact of large language models on peer review opinions from a fine-grained perspective: Evidence from top conference proceedings in AI
Peer review reports in AI conferences have grown longer and more standardized after LLMs, with increased emphasis on surface-level clarity and summaries at the expense of deeper critiques on originality and replicability.
-
LLM4SR: A Survey on Large Language Models for Scientific Research
A systematic review of LLM-based systems for hypothesis discovery, experiment planning, scientific writing, and peer review, including benchmarks, evaluation methods, and open challenges.
-
Evolving Roles of LLMs in Scientific Innovation: Assistant, Collaborator, Scientist, and Evaluator
The paper proposes a four-role framework for LLMs in scientific innovation and reviews methods, benchmarks, and limitations across Assistant, Collaborator, Scientist, and Evaluator roles.
-
AI-Driven Scholarly Peer Review via Persistent Workflow Prompting, Meta-Prompting, and Meta-Reasoning
A persistent, structured prompt loaded into an LLM chat session can guide reasoning models through critical analysis of experimental chemistry papers, but the evidence is a single qualitative case study.
-
Large language models for automated scholarly paper review: A survey
A survey of LLM-based automated scholarly paper review, cataloging models, datasets, methods, and publisher policies as of 2023-2024.