Pith. sign in

REVIEW 1 cited by

Shallow Synthesis of Knowledge in GPT-Generated Texts: A Case Study in Automatic Related Work Composition

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2402.12255 v1 pith:4Z7SJ4M3 submitted 2024-02-19 cs.CL

Anna Martin-Boyle , Aahan Tyagi , Marti A. Hearst , Dongyeop Kang This is my paper
classification cs.CL
keywords textshumanrelatedai-assistedanalysiscitationgpt-generatedscholarly
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Numerous AI-assisted scholarly applications have been developed to aid different stages of the research process. We present an analysis of AI-assisted scholarly writing generated with ScholaCite, a tool we built that is designed for organizing literature and composing Related Work sections for academic papers. Our evaluation method focuses on the analysis of citation graphs to assess the structural complexity and inter-connectedness of citations in texts and involves a three-way comparison between (1) original human-written texts, (2) purely GPT-generated texts, and (3) human-AI collaborative texts. We find that GPT-4 can generate reasonable coarse-grained citation groupings to support human users in brainstorming, but fails to perform detailed synthesis of related works without human intervention. We suggest that future writing assistant tools should not be used to draft text independently of the human author.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. GRASP: Graph-Reasoning Aided Survey Planning for High-Fidelity Related Work Generation

    cs.CL 2026-07 conditional novelty 6.0 of 10

    GRASP generates related-work sections that match human discourse roles, citation intents, and groupings by combining a Graph of Thoughts, an Argument-Counterargument Planning Network, and Steiner-tree pruning.

Pith tools