Pith. sign in

REVIEW 4 major objections 6 minor 3 cited by

Large Language Models for Automated Literature Review: An Evaluation of Reference Generation, Abstract Writing, and Review Composition

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Even the best LLM fabricates nearly half its generated references

desk verdict Useful benchmark for LLM literature-review evaluation, but the headline hallucination rates are only as good as Semantic Scholar's coverage; the qualitative result is solid, the exact numbers are not. read the letter →

arxiv 2412.13612 v5 pith:CHC2D6PD submitted 2024-12-18 cs.CL cs.AI

Xuemei Tang , Xufeng Duan , Zhenguang G. Cai This is my paper · ORCID
classification cs.CLcs.AI
keywords LLMhallucinationreferencegenerationliteraturereviewautomaticevaluationfactualconsistencysemanticcoveragecitationverificationAnnualReviewsdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether large language models can be trusted to automate the writing of literature reviews, and it finds that they cannot yet be trusted to produce reliable bibliographies. The authors build an automatic benchmark from 1,105 human-written review articles across five disciplines and test five LLMs on three tasks: generating references, writing an abstract, and composing a full review with citations. The central finding is that hallucinated references remain common even for the strongest model: Claude-3.5-Sonnet, the best performer, achieves only 51.59% precision on standalone reference generation, meaning nearly half of its citations do not match verifiable publications under the paper's matching rule. Accuracy improves markedly when models write a review and cite sources within the same output, but even then the best precision reaches about 59%. These results matter because researchers routinely use vanilla LLM outputs to compile bibliographies, and the paper shows that unverified reference lists from these models are not reliable.

What carries the argument

The load-bearing mechanism is an automatic evaluation pipeline anchored on a gold-standard corpus: 1,105 human-written literature reviews crawled from 51 Annual Reviews journals, each providing title, keywords, abstract, context, and reference list. For reference accuracy, every generated citation is checked against Semantic Scholar via Eq. 1's two-part rule, requiring either a title match of at least 80% plus one matching metadata field, or at least three matching non-title fields; this yields precision, recall, F1, and a title-search rate. For text quality, the pipeline uses NLI models (TRUE and GPT-4o) to score factual consistency, embedding cosine similarity and ROUGE for semantic coverage, and Key Point Recall, which extracts key points from human reviews with GPT-4 and scores entailment of each point in the model-generated review. The mechanism makes the evaluation fully automatic and reproducible.

What would settle it

Manually verify a random sample of the references the pipeline labels as hallucinated using independent bibliographic databases and publisher records; if more than a small fraction turn out to be real publications under slightly different titles or metadata, the reported hallucination rates are inflated.

Watch

Extended reading notes

Core claim

The paper's central claim is that state-of-the-art large language models still hallucinate a substantial share of references when asked to produce literature reviews, and that this failure is measurable, systematic, and discipline-dependent. Using human-written Annual Reviews articles as the gold standard, the framework verifies each generated reference against Semantic Scholar through a two-condition matching rule: a title matched at 80% plus at least one other metadata field, or at least three non-title fields matching. Across five models, the best standalone reference precision is 51.59% (Claude-3.5-Sonnet), with only 64.82% of generated titles retrievable from an external search engine. When models write the review text and the citations together, precision rises across models (Claude-3.5-Sonnet to 59.06%), which the authors attribute to the mutual constraint between cited text and reference metadata. Factual consistency and semantic coverage against human-written abstracts also vary by model and discipline, with mathematics yielding the most accurate references and chemistry the least in the reference-generation task.

Load-bearing premise

The headline hallucination rate assumes the verification database and the paper's matching rule catch every real reference; if a real reference is missing from the index or fails the matching rule, the model is falsely counted as hallucinating.

Editorial extensions

If this is right

  • Standalone reference generation from a vanilla LLM should be treated as unverified: even the best tested model produces a verifiable citation only about half the time.
  • Generating references inside a written review improves precision for most models, with Claude-3.5-Sonnet rising from 51.59% to 59.06%, suggesting citation-grounded writing constrains hallucination.
  • Errors are not evenly distributed: reference accuracy differs significantly across disciplines, with mathematics highest and chemistry lowest, and models show different disciplinary strengths.
  • Automatic hallucination detection aligns with human judgment: on 100 references, the automated method agreed with majority human annotation at 86% accuracy with a kappa of 0.71.
  • Even in the grounded review-composition setting, the best precision stays below 60%, so LLM-produced bibliographies will need external verification before academic use.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the paper's verification database is incomplete, the measured hallucination rates are upper bounds; re-running the verification against independent bibliographic databases would separate database misses from model fabrication.
  • The positive correlation between citation counts and reference precision suggests models are most accurate for famous, heavily indexed papers, so hallucination rates may be higher for niche topics and non-English literatures.
  • The same benchmark could be applied to retrieval-augmented generation pipelines to quantify how much retrieval grounding closes the gap, though this paper deliberately evaluates only vanilla LLMs.
  • Because the dataset is fixed to 2023 reviews and the authors note possible training-data overlap, extending the benchmark to recent papers would measure whether newer model generations improve on hallucination.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces an automatic evaluation framework for LLM literature review writing, covering three tasks: reference generation, abstract writing, and review composition. Using 1,105 literature reviews from 51 Annual Reviews journals (2023) and five LLMs, it measures reference hallucination via Semantic Scholar verification (precision, recall, F1, title search rate) and evaluates text quality via similarity metrics, ROUGE, NLI-based factual consistency (TRUE and GPT-4o as judges), and Key Point Recall (KPR) with GPT-4-extracted key points. The central finding is that even the best model, Claude-3.5-Sonnet, achieves only 51.59% precision on reference generation (Table 1), and that performance varies significantly across disciplines.

Significance. If the results hold, the paper provides a practically important, actionable conclusion: vanilla LLMs should not be trusted to produce bibliographies without external verification. The framework is a useful contribution with public dataset/code, clear task decomposition, and multidimensional metrics. Strengths include the use of an external database (Semantic Scholar) to ground the hallucination evaluation, a (small) human validation study, and statistical tests of cross-discipline differences. However, the exact precision numbers and the discipline ranking are vulnerable to Semantic Scholar coverage gaps, and the NLI-based metrics in Eqs. (6) and (8) contain argument-order issues that call their current results into question.

major comments (4)
  1. [§3.3, Eq. (1), Table 1, and Limitations] The reference-verification oracle is Semantic Scholar alone, and Eq. (1) classifies every reference that does not match as hallucinated. The Limitations paragraph concedes that 'this may have resulted in incomplete reference retrieval.' Table 1 shows St = 64.82% for Claude-3.5-Sonnet, meaning over a third of generated titles are not found at all; some of these are likely real but unindexed references. This systematically deflates Precision and St. The cross-discipline ranking (Figure 4a, Table 4) is also confounded: the positive correlation between precision and average citation counts (r = 0.40 and 0.51) is precisely the pattern expected if database coverage drives match rates. To support the headline 51.59% figure and the discipline ordering, the authors should cross-validate a stratified sample (especially all not-found references) against OpenAlex or Crossref and report the rate of real references that are missed by Semantic Scholar.
  2. [§3.3, Eq. (6), Table 2] The NLI direction in Eq. (6) is inverted relative to standard factual-consistency evaluation. As written, Entail_pi = 1 if the LLM-generated abstract entails the original human abstract, whereas factual consistency of a generated summary is normally assessed as the source (human abstract) entailing the generated claim. This reversal, combined with the use of GPT-4o (an evaluated model) as the NLI judge, likely explains the surprisingly high GPT-4o-based Entail scores (92–97% in Table 2). The authors should either justify the direction with a human pilot or reverse the arguments to θ_NLI(a_i, a_g_i), and report agreement between the NLI judge and human annotations for the abstract task.
  3. [§3.3, Eq. (8), Table 3] The KPR formula in Eq. (8) does not measure what the text claims. It computes θ_NLI(x, c_g_i), i.e., whether each key point x entails the generated review c_g_i, but coverage should be measured as whether the generated review entails or contains the key point, i.e., θ_NLI(c_g_i, x). As written, KPR scores in Table 3 and Figure 4d cannot be interpreted as semantic coverage. Additionally, the key-point extractor (GPT-4) and the NLI judge (GPT-4o) include an evaluated model, introducing circularity. Please correct the argument order and re-run the analysis, and consider an independent judge for robustness.
  4. [§4.5] The human validation of the hallucination metric is too thin to rule out the Semantic Scholar coverage bias. Only 100 LLM-generated references were annotated by three raters (86% agreement, kappa = 0.71), and no breakdown is reported for references whose titles were not found by Semantic Scholar. A 14% disagreement rate can shift the headline precision by several points, and the sample does not allow estimation of the false-hallucination rate among St-negative cases. The authors should enlarge the sample, stratify it by title-search outcome, and report precision after excluding or reclassifying non-verifiable references.
minor comments (6)
  1. [§1 and §3.3] The abstract and contributions state that the evaluation framework requires 'no human involvement,' but the 80% title-match threshold in Eq. (1) is said to be 'determined through human evaluation' and §4.5 reports a human validation study; please qualify the claim.
  2. [§4.4] The text says 'we categorize 51 journals into five disciplines' but then lists six categories (Biology, Chemistry, Mathematics, Physics, Social Science, and Technology); also the reported article counts (460+90+50+113+299+94 = 1106) do not match the stated total of 1,105.
  3. [Table 3 caption] The caption states 'four LLMs,' but the table lists five models; the same wording appears in Appendix D, which says 'We select four LLMs for task evaluation.'
  4. [§3.3, Eq. (5)] Eq. (5) defines s^(n)_pi with a sum over r_g^n ∈ R_g^i, which is notationally inconsistent because s^(n) is already indexed by n; please clarify the index structure.
  5. [§4.4] One-way ANOVA is used with highly unequal group sizes (e.g., Mathematics n=50, Biology n=460) and no mention of assumption checks; report effect sizes or non-parametric tests for robustness.
  6. [References] Several reference entries are incomplete or contain typos (e.g., Chen et al. 2024 lacks venue; 'Kukliansy' should likely be 'Kukliansky'); please proofread the bibliography.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: reference hallucination is checked against an external database, so the central claim is independently grounded; minor evaluator overlap and a threshold-calibration ambiguity warrant a low score.

full rationale

The paper's central quantitative claim, that even the best model achieves only 51.59% reference precision in the Reference Generation task (Table 1), is computed by comparing LLM-generated references against Semantic Scholar candidate sets using Eq. 1, with precision defined in Eqs. 2 and 3. Because the ground truth is an external academic database rather than the evaluated models' own outputs, the headline result does not reduce to the paper's inputs by construction. The Limitations section explicitly acknowledges a coverage risk: 'this may have resulted in incomplete reference retrieval,' noting that Semantic Scholar was used as the primary auxiliary tool. This is a validity threat that could bias the absolute precision numbers or the cross-discipline ranking, but it is not a circularity. For the abstract and review composition evaluations, GPT-4o is both one of the evaluated models and the NLI judge for factual consistency (Eq. 6) and key-point recall (Eq. 8), creating evaluator overlap and possible self-preference bias. However, the metrics are defined as NLI entailment judgments against human-written texts, not as functions of the target model's own performance, so this is a bias risk rather than a definitional circularity. One further ambiguity is that the 80% title-match threshold in Eq. 1 is described as 'a threshold determined through human evaluation,' while Section 4.5 reports 86% agreement between the automatic method and human assessment on 100 references. If the 100-reference validation set overlaps with the data used to set that threshold, the reported agreement would be partly a fit; the paper does not state whether the sets are disjoint. This is a reporting gap and potential overfitting caveat, but the paper does not exhibit the specific reduction needed to establish circularity. No load-bearing self-citations or imported uniqueness theorems were found; the related-work citations motivate the methodology but do not substitute for the paper's own external verification. Overall, the derivation chain is self-contained against an external oracle, with only minor non-circular validity concerns, supporting a score of 2.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central hallucination claim rests on the completeness of Semantic Scholar and the reliability rule in Eq. 1. The factual consistency and coverage claims rest on NLI directionality and GPT-4o key-point extraction. No free parameters are fitted in a model, but several evaluation thresholds are hand-set.

free parameters (3)
  • Number of generated references N = 10
    Set by the experiment design in Section 3.2; all precision, recall, and F1 scores in Tables 1 and 3 are computed over N=10 references per article.
  • Title match correctness threshold = 80%
    A generated title counts as correct if it matches at least 80% of the ground-truth title; this threshold was 'determined through human evaluation' (Section 3.3) and directly controls the hallucination rate.
  • Number of key points q in KPR = 10
    q=10 key points extracted by GPT-4o from each human-written review (Section 4.1); KPR scores in Eq. 8 depend on this choice.
assumptions (5)
  • domain assumption Annual Reviews 2023 articles are a representative gold standard for high-quality literature reviews
    Used as ground truth for all abstract and review comparisons; Section 3.1.
  • domain assumption Semantic Scholar search results are complete enough that a reference not finding a match is hallucinated
    Eq. 1 reliability rule depends entirely on candidate set Z returned by Semantic Scholar; Limitations admit retrieval may be incomplete.
  • domain assumption NLI entailment from LLM text to human text (Eq. 6) is a valid measure of factual consistency
    Factual consistency scores in Table 2 rest on this directional entailment; the paper does not validate this direction.
  • domain assumption GPT-4o extraction of q=10 key points from human reviews is faithful
    KPR scores in Eq. 8 depend on these extracted key points.
  • ad hoc to paper If training-data leakage exists, it inflates rather than reverses the observed patterns
    Limitations section argues contamination would only lower scores, so the hallucination conclusion survives; this is an argument, not a test.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Large Language Models for Automated Literature Review: An Evaluation of Reference Generation, Abstract Writing, and Review Composition." pith.science (2026). https://pith.science/paper/CHC2D6PD

@misc{pith2026241213612,
  author       = {Pith},
  title        = {Pith review of: Large Language Models for Automated Literature Review: An Evaluation of Reference Generation, Abstract Writing, and Review Composition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CHC2D6PD}},
  note         = {Machine review of arXiv:2412.13612}
}
read the original abstract

Large language models (LLMs) have emerged as a potential solution to automate the complex processes involved in writing literature reviews, such as literature collection, organization, and summarization. However, it is yet unclear how good LLMs are at automating comprehensive and reliable literature reviews. This study introduces a framework to automatically evaluate the performance of LLMs in three key tasks of literature writing: reference generation, literature summary, and literature review composition. We introduce multidimensional evaluation metrics that assess the hallucination rates in generated references and measure the semantic coverage and factual consistency of the literature summaries and compositions against human-written counterparts. The experimental results reveal that even the most advanced models still generate hallucinated references, despite recent progress. Moreover, we observe that the performance of different models varies across disciplines when it comes to writing literature reviews. These findings highlight the need for further research and development to improve the reliability of LLMs in automating academic literature reviews.

Figures

Figures reproduced from arXiv: 2412.13612 by the authors.

Figure 1
Figure 1. Illustration of the evaluation framework. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 3
Figure 3. Radar chart of the accuracy of LLM-generated [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Three tasks evaluation scores across different disciplines. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Statistics of dataset [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Radar chart of the accuracy of LLM-generated [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LLMs in Wikipedia: Investigating How LLMs Impact Participation in Knowledge Communities

    cs.HC 2025-09 conditional novelty 6.0 of 10

    Wikipedia editors who used LLMs reported that experienced editors expanded their contributions, while newcomers were pushed into editorial judgment they lacked skills for and saw their LLM-assisted edits rejected.

  2. AI for Auto-Research: Roadmap & User Guide

    cs.AI 2026-05 unverdicted novelty 4.0 of 10

    The paper delivers a stage-by-stage roadmap for AI in research, showing reliable assistance in retrieval and tool tasks but fragility in novelty and judgment, advocating human-governed collaboration.

  3. Adapting University Policies for Generative AI: Opportunities, Challenges, and Policy Solutions in Higher Education

    cs.HC 2025-06 unverdicted novelty 2.0 of 10

    A policy review urging universities to prioritize AI-resilient assessment design, training, and layered enforcement over generic acceptable-use guidelines.

Reference graph

Works this paper leans on

16 extracted references · 3 canonical work pages · cited by 3 Pith papers

  1. [1]

    Ayush Agrawal, Mirac Suzgun, Lester Mackey, and Adam Tauman Kalai. 2024. Do language models know when they’re hallucinating references?

    work page 2024

  2. [2]

    Muhammad Bello Aliyu, Rahat Iqbal, and Anne James. 2018. https://doi.org/10.1109/SNAMS.2018.8554896 The canonical model of structure for data extraction in systematic reviews of scientific research articles . In 2018 Fifth International Conference on Social Networks Analysis, Management and Security (SNAMS), page 264–271

    work page arXiv 2018

  3. [3]

    Mesallam, Mohamed Farahat, and Khalid H

    Fadi Aljamaan, Mohamad-Hani Temsah, Ibraheem Altamimi, Ayman Al-Eyadhy, Amr Jamal, Khalid Alhasan, Tamer A. Mesallam, Mohamed Farahat, and Khalid H. Malki. 2024. https://doi.org/10.2196/54345 Reference hallucination score for medical artificial intelligence chatbots: Development and usability study . JMIR Medical Informatics, 12(1):e54345. Company: JMIR M...

    doi:10.2196/54345 2024

  4. [4]

    Sai Anirudh Athaluri, Sandeep Varma Manthena, V S R Krishna Manoj Kesapragada, Vineel Yarlagadda, Tirth Dave, and Rama Tulasi Siri Duddumpudi. 2023. https://doi.org/10.7759/cureus.37432 Exploring the boundaries of reality: Investigating the phenomenon of artificial intelligence hallucination in scientific writing through chatgpt references . Cureus, 15(4):e37432

    doi:10.7759/cureus.37432 2023

  5. [5]

    Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. 2023. https://doi.org/10.48550/arXiv.2303.12712 Sparks of artificial general intelligence: Early experiments with gpt-4 . (arXiv:2303.12712). ArXiv...

    arXiv doi:10.48550/arxiv.2303.12712 2023

  6. [6]

    Guiming Hardy Chen, Shunian Chen, Ziche Liu, Feng Jiang, and Benyou Wang. 2024. https://doi.org/10.48550/arXiv.2402.10669 Humans or llms as the judge? a study on judgement biases . (arXiv:2402.10669). ArXiv:2402.10669 [cs]

    arXiv doi:10.48550/arxiv.2402.10669 2024

  7. [7]

    Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. 2023. https://doi.org/10.48550/arXiv.2305.14627 Enabling large language models to generate text with citations . (arXiv:2305.14627). ArXiv:2305.14627 [cs]

    arXiv doi:10.48550/arxiv.2305.14627 2023

  8. [8]

    Or Honovich, Roee Aharoni, Jonathan Herzig, Hagai Taitelbaum, Doron Kukliansy, Vered Cohen, Thomas Scialom, Idan Szpektor, Avinatan Hassidim, and Yossi Matias. 2022. https://doi.org/10.18653/v1/2022.naacl-main.287 TRUE : Re-evaluating factual consistency evaluation . In Proceedings of the 2022 Conference of the North American Chapter of the Association fo...

    doi:10.18653/v1/2022.naacl-main.287 2022

Show all 16 references
  1. [9]

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. https://doi.org/10.1145/3571730 Survey of hallucination in natural language generation . ACM Comput. Surv., 55(12):248:1--248:38

    2023 doi

  2. [10]

    Georgios Kontonatsios, Sally Spencer, Peter Matthew, and Ioannis Korkontzelos. 2020. https://doi.org/10.1016/j.eswax.2020.100030 Using a neural network-based feature extraction method to facilitate citation screening for systematic reviews . Expert Systems with Applications: X...

    2020

  3. [11]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff ...

    2023 arXiv

  4. [12]

    Xing Shangyu, Zhao Fei, Wu Zhen, An Tuo, Chen Weihao, Li Chunhui, Zhang Jianbing, and Dai Xinyu. 2024. https://aclanthology.org/2024.emnlp-main.67 Efuf: Efficient fine-grained unlearning framework for mitigating hallucinations in multimodal large language models . page 1167–1181

    2024

  5. [13]

    Yidong Wang, Qi Guo, Wenjin Yao, Hongbo Zhang, Xin Zhang, Zhen Wu, Meishan Zhang, Xinyu Dai, Min Zhang, Qingsong Wen, Wei Ye, Shikun Zhang, and Yue Zhang. 2024. https://doi.org/10.48550/arXiv.2406.10252 Autosurvey: Large language models can automatically write surveys . (arXiv...

    2024 doi arXiv

  6. [14]

    Xing, Hao Zhang, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. https://doi.org/10.48550/arXiv.2306.05685 Judging llm-as-a-judge with mt-bench and chatbot ...

    2023 doi arXiv

  7. [15]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  8. [16]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.