DAHL: Domain-specific Automated Hallucination Evaluation of Long-Form Text through a Benchmark Dataset in Biomedicine

Dongjun Jang; Hyopil Shin; Jean Seo; Jongwon Lim

REVIEW 4 major objections 5 minor 15 references

DAHL: Domain-specific Automated Hallucination Evaluation of Long-Form Text through a Benchmark Dataset in Biomedicine

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that DAHL, an automated pipeline that splits long biomedical answers into atomic claims and checks each against search-grounded sources, produces a DAHL Score that tracks human hallucination judgments (Pearson r = 0.5508…

desk verdict A genuinely useful biomedical hallucination benchmark dataset and pipeline, but the validation is thin and the unknown-label exclusion could bias every model ranking. read the letter →

arxiv 2411.09255 v1 pith:Y5UTAHKQ submitted 2024-11-14 cs.CL

Jean Seo , Jongwon Lim , Dongjun Jang , Hyopil Shin This is my paper · ORCID

classification cs.CL

keywords hallucinationevaluationbiomedicalbenchmarklong-formtextgenerationatomicfactualityDAHLScorelargelanguagemodelsfactualprecision

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to make hallucination in long-form biomedical answers measurable without expensive human annotation. It introduces a benchmark of 8,573 questions built from biomedical research papers and an automated pipeline that splits each model response into atomic factual claims, checks each claim against online-sourced information, and averages the truth rate into a DAHL Score. The paper argues that this score tracks human judgments: on 99 responses the automated and human factuality scores correlate at 0.5508, with a p-value of 3.49 × $10^{{-9}}$. If valid, the score gives a cheap, repeatable way to rank models by factual reliability and to generate preference labels for alignment tuning in a high-stakes domain.

What carries the argument

The central object is the atomic unit: a sentence-sized piece of a model response that asserts one fact whose truth can be judged true or false. The pipeline works in three stages: preprocessing removes the prompt, duplicate sentences, and truncated text; a splitter model (the paper uses gpt-4o) breaks the response into atomic units; and a search-grounded API checker labels each unit true, false, or unknown. The DAHL Score for a response is the ratio of true atomic units to total atomic units, and the overall DAHL Score is the average of these ratios across responses. The atomic-unit representation is what lets the method credit a response for the claims it gets right even when other claims in the same answer are wrong.

What would settle it

A direct falsifier: on a random sample of responses, ask the Checker to label every atomic unit with a forced true or false choice instead of allowing 'unknown', then recompute the DAHL Score and its correlation with human judges; if the correlation drops substantially or model rankings change, the reported validity is an artifact of excluding ambiguous units.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that fact-conflicting hallucination in long-form biomedical text can be decomposed into atomic units and automatically scored by a splitter-and-checker pipeline, and that the resulting DAHL Score is statistically correlated with human factuality ratings. The authors report a Pearson correlation of 0.5508 between automated and human scores across 99 responses, and they further observe that larger models tend to hallucinate less, with diminishing returns beyond roughly 7 to 8 billion parameters. They position the DAHL Score as a potential efficient alternative to human-annotated preference labels, and they claim the automated dataset construction pipeline is scalable to other specialized domains.

Load-bearing premise

The central assumption is that the automated checker's true and false labels on atomic claims are accurate enough to stand in for human fact-checking, which the paper never directly validates against a gold standard.

Editorial extensions

If this is right

The DAHL Score can be computed automatically on long-form biomedical answers, replacing costly human annotation for ranking models by factual reliability.
The same pipeline can produce continuous preference labels for alignment tuning, since each response receives a factuality score rather than a binary judgment.
Factual accuracy in biomedicine improves with model size up to roughly 7 to 8 billion parameters but shows diminishing returns beyond that, at least for the tested open models.
A temperature range of roughly 0.2 to 0.3 gives moderate hallucination performance across models, with no single universal optimal temperature.
The automated dataset construction from a public biomedical full-text archive can be extended to other specialized domains and updated as new papers appear.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The scaling-plateau claim is a statement about these eight models and this benchmark, not a general law; controlling for training-data overlap and question difficulty would be needed before generalizing it.
The score's usefulness as a preference label depends on checker agreement with human judgments on individual atomic units, not just on the response-level correlation, so unit-level agreement statistics would be a natural next measurement.
Because the checker consults live online sources, DAHL scores are not fully reproducible over time; fixing a dated document snapshot would make the benchmark a stable evaluation target.
If the checker's 'unknown' exclusions remove systematically harder or more ambiguous responses, the reported scores may be optimistic; recomputing scores with forced true/false labels would reveal the size of that bias.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The useful part of this paper is the artifact: 8,573 biomedical questions from PMC across 29 categories, plus an automated FactScore-style pipeline with public code and data. That is a real contribution for anyone working on medical LLM evaluation. The paper is honest about its lineage: the atomic-unit decomposition is FactScore applied to a new domain, and the dataset construction and filtering are described in enough detail to reproduce. Credit where due: the dataset is sizable, the pipeline is fully automated, and the authors release both code and data. That alone makes this worth a look.

The soft spots are real but not fatal. The main validation is a Pearson correlation of 0.55 on 99 responses against human scores. That is moderate, and the paper overreads it as "moderate to strong." No inter-annotator agreement is reported, so we do not know how reliable the human side is either. The paper also does not validate the pplx-API checker against any gold standard at the atomic-unit level; the correlation with human scores is the only evidence, and it is at the response level.

The more specific concern is the handling of "unknown" labels. The paper states in Section 5.3 that if any atomic unit is labeled unknown, the whole response is dropped from the DAHL Score calculation. That is a selection rule that can bias model comparisons if unknown labels occur unevenly across models, and the paper never reports how often this happens. This is a genuine soft spot, but it is fixable: report exclusion rates per model, or treat unknown as a third category instead of dropping the response.

The scaling claim (plateau past 7-8B) is based on visual inspection of Table 2 with no significance tests. That is a minor overreach; the pattern is plausible given Qwen-2 and Llama-3.1, but it is not established.

Circularity is not a real issue here: the checker is separate from the evaluated models, and the labels come from a web-augmented API. Using gpt-4o to split gpt-4o's own output is a minor bias, not a load-bearing one.

I would send this to peer review. The dataset and pipeline are useful, the limitations are addressable, and the paper's central idea holds up as a reasonable domain adaptation. The authors need to tighten the validation and report exclusion rates, but this is a solid starting point for the community. A serious referee can push on those points productively.

Referee Report

4 major / 5 minor

Summary. The paper introduces DAHL, a benchmark dataset and automated evaluation system for measuring fact-conflicting hallucination in long-form biomedical text generation. The dataset consists of 8,573 questions derived from PubMed Central papers across 29 categories. The evaluation pipeline splits model responses into atomic units using gpt-4o, checks each unit with a Perplexity API-based checker, and computes the DAHL Score as the average proportion of factual units. The authors report scores for gpt-4o and seven open-source models, analyze scaling behavior within model families, examine temperature effects, and validate the automated scores against human annotation on 99 responses, obtaining a Pearson correlation of 0.5508. The central claim is that DAHL provides a valid automated alternative to human-annotated preference labels for long-form hallucination evaluation.

Significance. If the validity of the automated score were established, the paper would provide a practically useful benchmark and pipeline for biomedical hallucination evaluation, with genuine strengths: the dataset and code are publicly released, the atomic-unit decomposition is motivated by a real limitation of binary response-level judgments, and the framework is designed to be extendable to other domains. The scaling observations and temperature analysis are potentially informative empirical contributions. However, the significance is conditional on validation of the proprietary checker and on ruling out selection bias from excluded responses; as reported, the evidence for the central validity claim is moderate at best.

major comments (4)

[§3.3 Step 2 and §5.3] The factuality checker (pplx-API) is the sole source of truth for all atomic-unit labels, yet the paper never validates the checker's labels against a gold standard. The only evidence offered in §5.3 is a response-level Pearson correlation of 0.5508 on 99 responses; no atomic-level agreement, precision, recall, or confusion matrix is reported. The authors should report per-unit agreement between the checker and human labels, ideally with Cohen's kappa, and characterize the error modes (e.g., false versus unknown).
[§5.3] The exclusion of responses containing atomic units labeled 'unknown' is a load-bearing selection step, but the paper reports no exclusion counts or rates. If unknown labels occur more frequently for some models or response styles, every DAHL Score in Table 2 is computed over a different, non-random subset of responses, and the model rankings could be driven by the selection rather than by factuality. The authors should report the number and fraction of excluded responses per model, and provide a sensitivity analysis (for example, treating unknown as false, or as its own category, or imputing scores from the remaining units).
[§5.1, Table 2] The claim that 'beyond a model size of 7 to 8 billion parameters, further scaling does not significantly improve factual accuracy' is not supported by any statistical test. The differences among Llama-3.1 8B vs 70B (0.8627 vs 0.8733) and Qwen-2 7B vs 72B (0.8870 vs 0.8997) are small relative to the 0.55 correlation observed in human validation, so they may be within noise. The authors should provide per-question paired tests, bootstrapped confidence intervals, or an explicit statistical comparison of score differences across model sizes.
[§5.3] The human validation uses only 99 responses and two annotators, but no inter-annotator agreement is reported. Without a measure such as Cohen's kappa or the correlation between the two annotators' response-level scores, it is unclear how much of the 0.5508 correlation is limited by checker error, splitter disagreement, or human annotation noise. This is essential for interpreting the validity claim.

minor comments (5)

[§3.2.3] There is a typo: 'primarliy' should be 'primarily'.
[Table 2] The table lists the gpt-4o parameter size as '?', which is acceptable for a proprietary model, but the inconsistent capitalization of 'Mpt' versus 'MPT' in the same table and text should be cleaned up.
[Figure 5] The caption refers to 'Qwen-2-8b', but Table 2 lists Qwen-2-7B; the discrepancy should be corrected.
[§3.1] The paper says that noncommittal responses and redundant sentences are removed 'with regular expression', but the pipeline description in §3.3 Step 0 only mentions removing the prompt, deduplicating sentences, and removing incomplete sentences. The implementation of the noncommittal-response removal should be specified.
[References] Several references have formatting errors (e.g., 'V ogeti' for the Llama 3 author list); the reference list should be proofread.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the DAHL Score is computed from an external pplx-API checker and validated against independent human annotations.

full rationale

The paper's central claim is that the DAHL Score, an average of atomic-unit factuality labels, is a valid automated measure of fact-conflicting hallucination. The derivation chain is not circular. The factuality labels come from a separate checker, pplx-API (Section 3.3, Step 2), not from the models being evaluated and not from the human validation scores. The human correlation in Section 5.3 (Pearson r = 0.5508, p = 3.49e-9) is an independent external check, not a fitted parameter renamed as a prediction. There is no self-citation used as load-bearing evidence: the paper cites FactScore and other prior work, but none of the cited authors overlap with the present authors, and no uniqueness theorem is imported. The only mildly self-referential element is using gpt-4o as the Splitter while also evaluating gpt-4o, but splitting alone does not determine the DAHL Score; the factuality judgement is made by pplx-API. The paper's own admission that responses containing an atomic unit labeled 'unknown' are excluded from the final score (Section 5.3) is a legitimate validity concern about potential non-uniform selection across models, and it could make the benchmark unreliable, but it is not circularity: the score is not defined in terms of the human labels or the models' own outputs. Overall, the paper's evaluation pipeline is self-contained relative to its target claim, and no circular step is exhibited.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities; it adapts FactScore's atomic-unit framework and defines the DAHL Score as an aggregation. The load-bearing assumptions are the reliability of the pplx checker and the unbiasedness of excluding unknown-labeled units.

free parameters (1)

scaling plateau threshold = 7-8 billion parameters
Hand-chosen after inspecting DAHL Scores for 8 models; no statistical estimate of the plateau is provided (Section 5.1).

assumptions (5)

domain assumption Atomic decomposition into single-information units preserves the factuality content of a response.
Adopted from FactScore (Min et al., 2023); the paper validates only the count of units via a T-test, not the semantic equivalence of units across annotators.
domain assumption The pplx-API checker gives accurate binary factuality labels for atomic units.
Section 3.3 Step2; no independent evaluation of the checker's accuracy is provided.
domain assumption Responses containing atomic units labeled 'unknown' can be excluded from scoring without biasing the DAHL Score.
Section 5.3 states such responses are excluded; this can systematically remove models that hedge or make unverifiable claims.
domain assumption The 8,573 gpt-4-generated questions filtered by regex are answerable without additional context and keep the factual ground truth from the source papers.
Section 3.2.2 describes automated regex filtering with manual checks; no error analysis of the remaining questions is reported.
domain assumption Human factuality scores on 99 responses are a valid gold standard.
Section 5.3 uses two human annotators, but no inter-annotator agreement statistic is reported.

how reviews work

0 comments

Cite this review

Pith. "Pith review of DAHL: Domain-specific Automated Hallucination Evaluation of Long-Form Text through a Benchmark Dataset in Biomedicine." pith.science (2026). https://pith.science/paper/Y5UTAHKQ

@misc{pith2026241109255,
  author       = {Pith},
  title        = {Pith review of: DAHL: Domain-specific Automated Hallucination Evaluation of Long-Form Text through a Benchmark Dataset in Biomedicine},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y5UTAHKQ}},
  note         = {Machine review of arXiv:2411.09255}
}

read the original abstract

We introduce DAHL, a benchmark dataset and automated evaluation system designed to assess hallucination in long-form text generation, specifically within the biomedical domain. Our benchmark dataset, meticulously curated from biomedical research papers, consists of 8,573 questions across 29 categories. DAHL evaluates fact-conflicting hallucinations in Large Language Models (LLMs) by deconstructing responses into atomic units, each representing a single piece of information. The accuracy of these responses is averaged to produce the DAHL Score, offering a more in-depth evaluation of hallucinations compared to previous methods that rely on multiple-choice tasks. We conduct experiments with 8 different models, finding that larger models tend to hallucinate less; however, beyond a model size of 7 to 8 billion parameters, further scaling does not significantly improve factual accuracy. The DAHL Score holds potential as an efficient alternative to human-annotated preference labels, being able to be expanded to other specialized domains. We release the dataset and code in public.

Figures

Figures reproduced from arXiv: 2411.09255 by the authors.

**Figure 1.** The DAHL benchmark dataset construction process. generation task is challenging. As common metrics like accuracy or f1 score is not directly applicable, replacing human annotation remains problematic. TruthfulQA (Lin et al., 2022) tries to address this issue by training a separate model to judge whether the generated response is correct or incorrect. However, assessing hallucination in generation tasks demands a mo… view at source ↗

**Figure 2.** The categorical distribution of the DAHL [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** The automated hallucination evaluation pipeline. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (7 more)

**Figure 4.** Figure 4: An example of generated response and its [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 5.** Figure 5: The DAHL Score for Llama-3.1-8b, Gemma-2-9b, Qwen-2-8b, and Mistral-Nemo-Base-2407 (12 billion parameters) evaluated across temperatures ranging from 0.1 to 1.0. The optimal temperature for each model falls within the range of 0.1 to 0.3, with a slight linear decrease …

**Figure 6.** Figure 6: Categorical DAHL Score of each model [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

**Figure 7.** Figure 7: Prompt used for dataset construction [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

**Figure 8.** Figure 8: Prompt used for response generation [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]

**Figure 9.** Figure 9: Prompt used for splitting [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]

**Figure 10.** Figure 10: Prompt used for factuality checking [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 15 canonical work pages

[1]

Zouying Cao, Yifei Yang, and Hai Zhao. 2023. Auto- hall: Automated hallucination dataset generation for large language models. Shiqi Chen, Yiran Zhao, Jinghan Zhang, I-Chun Chern, Siyang Gao, Pengfei Liu, and Junxian He. 2023. Felm: Benchmarking factuality evaluation of large language models. Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, S...

work page 2023
[2]

Rait, Zachary DeVito, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, and Zhiwei Zhao. 2024. The llama 3 herd of models. Esin Durmus, He He, and Mona Diab. 2020. Feqa: A question answering evaluation framework for faith- fulness assessment in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics ....

work page 2024
[3]

Zhang, Yu Wang, Shizhu Liu, Pingchuan Tian, Yup- ing Wang, and Yuxuan Wang. 2023. Halo: Estima- tion and reduction of hallucinations in open-source weak large language models. Ben Goodrich, Vinay Rao, Peter J. Liu, and Mohammad Saleh. 2019. Assessing the factual accuracy of gener- ated text. In Proceedings of the 25th ACM SIGKDD International Conference o...

work page 2023
[4]

Wang, Shusen Wang, Weiguo Zheng, Hongwei Feng, and Yanghua Xiao. 2023. Xiezhi: An ever-updating benchmark for holistic domain knowledge evalua- tion. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan

work page 2023
[5]

Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation in natural language generation. ACM Comput- ing Surveys, 55(12). Saurav Kadavath, Tom Conerly, Amanda Askell, Tom

work page 2023
[6]

Amodei, Tom Brown, Jack Clark, Nicholas Joseph, Ben Mann, Sam McCandlish, Chris Olah, and Jared Kaplan. 2022. Language models (mostly) know what they know. Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023. Large language models struggle to learn long-tail knowledge. Himabindu Lakkaraju, Dylan Slack, Yuxin Chen, Chen- hao Ta...

work page 2022
[7]

Atanasova, Ramy Baly, Mitra Mohtarami, and Preslav Nakov. 2019. Semeval-2019 task 8: Fact checking in community question answering forums. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike

work page 2019
[8]

Lewis, Wen tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. Factscore: Fine-grained atomic evaluation of factual precision in long form text generation. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettle- moyer. 2022. Rethinking the role of demonstrations: What makes in-c...

work page 2023

Show all 15 references

[9]

Corrado, Yossi Matias, Katherine Chou, Juraj Gottweis, Nenad Tomasev, Yun Liu, Alvin Ra- jkomar, Joelle Barral, Christopher Semturs, Alan Karthikesalingam, and Vivek Natarajan

Kelly, Nathaneal Scharli, Aakanksha Chowdhery, Philip Mansfield, Blaise Aguera y Arcas, Dale Web- ster, Greg S. Corrado, Yossi Matias, Katherine Chou, Juraj Gottweis, Nenad Tomasev, Yun Liu, Alvin Ra- jkomar, Joelle Barral, Christopher Semturs, Alan Karthikesalingam, and Vivek...

2022
[10]

Kavukcuoglu, Clement Farabet, Elena Buchatskaya, Sebastian Borgeaud, Noah Fiedel, Armand Joulin, Kathleen Kenealy, Robert Dadashi, and Alek An- dreev. 2024. Gemma 2: Improving open language models at a practical size. Albert Webson and Ellie Pavlick. 2022. Do prompt- based mod...

2024
[11]

Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. Chain-of-thought prompting elic- its reasoning in large language models. Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli. 2024. Hallucination is inevitable: An innate limitation of large language models. An Yang...

2023
[12]

Liu, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, Zhifang Guo, and Zhihao Fan. 2024. Qwen2 techni- cal report. He Yang, Fei Wang, Matthew Greenblatt, Sharon Huang, and Yi Zhang. 2023. Ai chatbots in clinical labora- tory medicine:...

2024
[13]

Bi, Freda Shi, and Shuming Shi. 2023. Siren’s song in the ai ocean: A survey on hallucination in large language models. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen

2023
[14]

Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023. A survey of large language models. Hongjian Zhou, Fenglin Liu, Boyang Gu, Xinyu Zou, Jinfa Huang, Jinge Wu, Yiru Li, Sam S. Chen, Peilin

2023
[15]

Zhou, Junling Liu, Yining Hua, Chengfeng Mao, Chenyu You, Xian Wu, Yefeng Zheng, Lei Clifton, Zheng Li, Jiebo Luo, and David A. Clifton. 2024. A survey of large language models in medicine: Princi- ples, applications, and challenges. Appendix A Automatically Filtered Questions...

2024

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Zouying Cao, Yifei Yang, and Hai Zhao. 2023. Auto- hall: Automated hallucination dataset generation for large language models. Shiqi Chen, Yiran Zhao, Jinghan Zhang, I-Chun Chern, Siyang Gao, Pengfei Liu, and Junxian He. 2023. Felm: Benchmarking factuality evaluation of large language models. Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, S...

work page 2023

[2] [2]

Rait, Zachary DeVito, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, and Zhiwei Zhao. 2024. The llama 3 herd of models. Esin Durmus, He He, and Mona Diab. 2020. Feqa: A question answering evaluation framework for faith- fulness assessment in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics ....

work page 2024

[3] [3]

Zhang, Yu Wang, Shizhu Liu, Pingchuan Tian, Yup- ing Wang, and Yuxuan Wang. 2023. Halo: Estima- tion and reduction of hallucinations in open-source weak large language models. Ben Goodrich, Vinay Rao, Peter J. Liu, and Mohammad Saleh. 2019. Assessing the factual accuracy of gener- ated text. In Proceedings of the 25th ACM SIGKDD International Conference o...

work page 2023

[4] [4]

Wang, Shusen Wang, Weiguo Zheng, Hongwei Feng, and Yanghua Xiao. 2023. Xiezhi: An ever-updating benchmark for holistic domain knowledge evalua- tion. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan

work page 2023

[5] [5]

Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation in natural language generation. ACM Comput- ing Surveys, 55(12). Saurav Kadavath, Tom Conerly, Amanda Askell, Tom

work page 2023

[6] [6]

Amodei, Tom Brown, Jack Clark, Nicholas Joseph, Ben Mann, Sam McCandlish, Chris Olah, and Jared Kaplan. 2022. Language models (mostly) know what they know. Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023. Large language models struggle to learn long-tail knowledge. Himabindu Lakkaraju, Dylan Slack, Yuxin Chen, Chen- hao Ta...

work page 2022

[7] [7]

Atanasova, Ramy Baly, Mitra Mohtarami, and Preslav Nakov. 2019. Semeval-2019 task 8: Fact checking in community question answering forums. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike

work page 2019

[8] [8]

Lewis, Wen tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. Factscore: Fine-grained atomic evaluation of factual precision in long form text generation. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettle- moyer. 2022. Rethinking the role of demonstrations: What makes in-c...

work page 2023