Pith. sign in

REVIEW 4 major objections 5 minor 67 references

How does Misinformation Affect Large Language Model Behaviors and Preferences?

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces MISBENCH, a benchmark of 10.3 million generated passages showing that LLM susceptibility to misinformation depends systematically on conflict type and writing style, and proposes a comparison-based reconstruction…

desk verdict MisBench is a well-constructed synthetic benchmark that fills a real gap, but the paper's empirical claims are weakened by missing baselines and a circular RtD evaluation. read the letter →

arxiv 2505.21608 v1 pith:ZT6BC4PF submitted 2025-05-27 cs.CL cs.AI

Miao Peng , Nuo Chen , Jianheng Tang , Jia Li This is my paper
classification cs.CLcs.AI
keywords misinformationbenchmarkknowledgeconflictstylisticvariationLLMbehaviorpreferencereconstructtodiscriminateWikidatamulti-hopreasoning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish a precise, fine-grained account of when large language models are misled by misinformation, rather than the general observation that they sometimes are. It builds MISBENCH, a benchmark of 431,113 question-answer pairs and 10,346,712 generated passages, crossing three knowledge-conflict types (factual, temporal, and ambiguous-semantic) with six writing styles (Wikipedia entry, news report, science reference, blog, technical, and confident). Across twelve model sizes and families, the measurements show LLMs can flag falsehoods via contextual inconsistency even without prior knowledge of the fact, yet remain vulnerable to knowledge conflicts—most of all ambiguous-semantic ones—and their vulnerability shifts with style and reasoning depth. Building on this, the paper proposes Reconstruct to Discriminate (RtD), which retrieves an entity description, generates supporting evidence, and has the model compare it with the suspect passage, raising misinformation-detection success by up to 20.6 percent on Gemma2-9B. If the results hold, MISBENCH gives builders a reusable, style- and type-stratified testbed for detector reliability.

What carries the argument

MISBENCH is the load-bearing instrument. Claims are extracted as one-hop and multi-hop (subject, relation, object) triples from Wikidata; LLaMA-3-70B writes roughly 500-word passages that are either correct, factually contradicted, temporally contradicted (future time-stamped), or semantically contradicted (polysemous entity descriptions), and each passage is restyled into six genres. Quality control filters by refusal phrases, NLI entailment, and embedding similarity, with human annotation on sampled instances. RtD is the proposed countermeasure: identify the key entity, fetch its Wikipedia description, prompt the LLM to reconstruct supporting evidence, and ask the model to compare the original passage with that reconstruction, turning an isolated judgment into a comparative one.

What would settle it

Curate a held-out set of human-written misinformation matched to the same Wikidata facts, conflict types, and six styles (for example, passages authored by adversarial human writers or drawn from fact-checking archives), and run the same two-choice detection and Memorization Ratio protocol. If the type-by-style ordering of susceptibility—semantic above factual above temporal, and narrative above formal only in multi-hop settings—does not reproduce, or if RtD's gains collapse on human-written passages, the paper's central generalization is falsified.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that LLM susceptibility to misinformation is structured rather than uniform: every tested model can detect falsehoods above chance from contextual inconsistencies alone, but semantic conflicts—the same name carrying a shifted meaning—erode that ability most, and the most convincing style flips with task depth. Formal and objective language poses the greater risk in single-hop fact-matching tasks, while narrative and subjective language is more misleading in multi-hop reasoning tasks. The paper further claims that models decide better when comparing two conflicting passages than when judging a single passage in isolation, and that exploiting this comparison tendency—by retrieving neutral facts, reconstructing evidence, and comparing—materially improves detection across model families and knowledge-conflict types.

Load-bearing premise

The benchmark is generated by LLaMA-3-70B from Wikidata claims, and the paper assumes these synthetic passages are a faithful stand-in for the real-world misinformation an LLM would encounter; if the generated texts differ from human misinformation in style, refusal artifacts, or factual errors, the measured vulnerabilities and RtD gains describe the synthetic corpus rather than real-world risk.

Editorial extensions

If this is right

  • Validation should not stop at overall accuracy: a detector can look robust on one-hop formal misinformation while failing on multi-hop narrative misinformation, so benchmarks must stratify by conflict type and style.
  • Comparative judgment is a cheap reliability lever: presenting two conflicting passages and asking the model to compare outperforms single-passage judgment, which RtD exploits without retraining.
  • Semantic ambiguity is the largest failure mode, so entity disambiguation or semantic normalization before judgment addresses the vulnerability that most degrades LLM performance.
  • RtD's retrieval-reconstruction-compare loop raises detection success across LLaMA-3, Qwen2.5, and Gemma2 families, with the largest reported gain of 20.6 percent on Gemma2-9B.
  • MISBENCH is released for reuse, providing a style- and type-stratified testbed for future LLM-based misinformation detectors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • (Editorial inference) Because the passages are synthetic, RtD's dependence on Wikipedia may not transfer to high-value domains absent from Wikipedia, such as specialized medical or financial knowledge; a targeted test on such a corpus would delimit the method's scope.
  • (Editorial inference) The six-style axis doubles as an attack taxonomy: an adversarial user could pick news-report styling for multi-hop questions to maximize the chance a model accepts falsehood, a framing the paper does not explicitly develop.
  • (Editorial inference) The finding that comparison beats isolated judgment suggests a testable design for retrieval-augmented pipelines—always hand the model multiple candidate passages and let it reconcile them—which the paper does not evaluate in a retrieval setting.
  • (Editorial inference) All measurements use temperature-zero, multiple-choice prompts; whether the style- and type-dependent vulnerabilities persist under sampling, conversational pressure, or long-context settings is left open and could be tested by re-running Memorization Ratio under self-consistency sampling.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper introduces MisBench, a synthetic benchmark for analyzing LLM susceptibility to misinformation. The benchmark is constructed from Wikidata one-hop and multi-hop claims, with LLaMA-3-70B generating correct evidence and three types of conflicting misinformation (factual, temporal, semantic), then restyling each text into six formats (Wikipedia entry, news report, science reference, blog, technical language, confident language). Quality control includes completeness filtering, NLI entailment checking, semantic matching with threshold alpha=0.3, and human evaluation on 500 examples. Experiments on nine open and three closed LLMs yield three findings: LLMs can discern misinformation even without prior knowledge; they are vulnerable to knowledge conflicts, especially factual and semantic misinformation; and stylistic features affect convincingness. The paper also proposes RtD, which retrieves entity descriptions from Wikipedia and reconstructs evidence before comparing with the input text, reporting Success Rate improvements up to 20.6% (Gemma2-9B). The stated contributions are the benchmark (431,113 QA pairs, 10,346,712 evidence texts), the behavioral findings, and the RtD method.

Significance. If the benchmark faithfully represents real-world misinformation, this is a substantial resource: it is larger than existing conflict-based benchmarks, spans multiple conflict causes and styles, and the authors provide detailed pipeline documentation, resource-consumption tables, human-annotation agreement, and public release commitments. The RtD idea is simple and reproducible, and the improvement over the +Desc baseline is nontrivial. The main caveat is that all results are measured on synthetic texts generated by one LLM from the same knowledge source that RtD later retrieves from; this limits external validity until representativeness is tested. The paper would be stronger if the authors treated the benchmark as a controlled synthetic stress test rather than claiming direct generalization to real-world misinformation. Strengths to credit: the four-stage quality-control pipeline with human evaluation on 500 items (Sections 2.4 and Appendix C), explicit prompts and SPARQL queries in the appendices, and clear metrics (Success Rate, MR, TendCM).

major comments (4)
  1. [§2.2–2.4, Fig. 2; Limitations] The benchmark's central assumption—that LLaMA-3-70B-generated texts are a faithful proxy for human misinformation—is asserted rather than tested. The quality-control procedures in §2.4 verify that each generated text entails its fabricated claim and contradicts the correct evidence, but they do not check stylistic or content resemblance to human-authored misinformation, and no human-written comparison set is included. Since every passage in MisBench is produced by the same generator, the type/style ordering in §3 and the RtD gains in §4 could partly reflect detection of LLaMA-3-70B's stylistic fingerprint (e.g., the uniform ~500-word 'Studies and Research / Notable Alumni' scaffolding exemplified in Tables 20–27) rather than misinformation content. The Limitations paragraph concedes 'may not fully encompass all possible variations' but still concludes the types/styles are 'sufficiently representative'; this is exactly the load-bearing step that needs evidence. I request a human-written misinformation control set (e.g., sampled from existing fake-news corpora) evaluated under the same protocol, or an analysis showing that the findings persist when generated-text artifacts are controlled for.
  2. [Table 3; §3.2] No random-chance baseline or error bars are reported for Success Rate, although the task is framed as a two-choice judgment. Several model-type combinations score far below 50% (e.g., Qwen2.5-7B 14.22% on one-hop factual Memory, 16.13% on one-hop semantic Memory, and 18.28% on multi-hop semantic Memory), which is a strong indication that the models are systematically choosing the wrong option rather than exhibiting 'comparable abilities in discerning misinformation.' Because results are averaged over three runs but variance is not reported, and the 'Memory' and 'Unknown' subsets differ per model (Appendix F.1), none of the cross-model comparisons in Finding 1 are statistically supported. Please add a chance baseline, confidence intervals or significance tests, and report the size of the per-model subset for each reported rate.
  3. [§4; Table 4] The RtD evaluation is confounded with the benchmark construction source. MisBench is built from Wikidata claims and entity descriptions (§2.1), and RtD retrieves from Wikipedia for the key subject entity; the '+Desc' baseline also uses the same entity description. The success of RtD may therefore reflect that it resurfaces the original correct Wikidata answer rather than a general ability to 'reconstruct to discriminate.' An ablation with an external knowledge source not used in construction (or with retrieved content withheld) is needed to separate reconstruction ability from direct answer retrieval. Also report how often the reconstruction step explicitly outputs the correct answer before the comparison stage.
  4. [§3.1, F.2] Closed-source models are evaluated on a subset (20,000 one-hop and 10,000 multi-hop questions) while open-source models are evaluated on the full benchmark, and the Success Rate scenarios use per-model different totals; these differences are not accounted for in the comparison table. Please either evaluate all models on the same filtered subset or provide per-model confidence intervals and normalize by the scenario sizes. This is necessary for the headline claim that LLMs 'demonstrate comparable abilities in discerning misinformation.'
minor comments (5)
  1. [Throughout] Typographical errors: 'boarder' appears twice in §3.2 where 'broader' is intended; 'effecient' in F.2; 'Fugure 12' in F.5; 'extremeley confident' in Table 15; and Figure 10's caption labels the last panel 'Confidential Language' instead of 'Confident Language'.
  2. [References] The reference list opens with a dangling fragment 'The claude 3 model family: Opus, sonnet, haiku.' with no authors or year, and Zhou and Zafarani (2021a) and (2021b) are identical entries; please fix both.
  3. [Table 2 vs Table 8] Table 2 reports 82 one-hop relations, but Table 8 states that only 71 relations are shown 'for readability'; please clarify whether the remaining 11 relation templates are available in the released data or in an appendix, since they are part of the benchmark construction.
  4. [Figures 7, 16, 17] The y-axis in these figures is labeled 'Log Probability' but the tick values (0.2–1.0) look like probabilities, not log probabilities; please clarify the metric and correct the label.
  5. [Table 5] The 'Agreement Rate' columns appear to report per-annotator agreement with a gold label rather than inter-annotator agreement; please report pairwise or Fleiss' kappa for the three annotators.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: all findings are empirical measurements on a constructed benchmark, and RtD is an independently evaluated retrieval-augmented method.

full rationale

The paper's claims are empirical measurements on a self-constructed benchmark (MISBENCH), not derivations from a fitted model or from a self-citation chain. The benchmark is built from Wikidata claims and LLaMA-3-70B-generated texts with quality-control filters (Sections 2.1-2.4), but no parameter is fitted to the reported results; the alpha=0.3 semantic-matching threshold is a dataset filter, not a predictor of the Success Rate, MR, or Tend_CM metrics. The findings in Section 3 are descriptive statistics over the generated corpus, and the RtD method in Section 4 is evaluated on the same benchmark with an explicit retrieval-plus-comparison procedure; its gains are not equivalent by construction to the labels, because the retrieved Wikipedia descriptions are not the same strings as the correct evidence and the comparison is performed by the LLM. Cited prior work, including the authors' own papers, appears only for methodology or related work and is not load-bearing for the central empirical claim. The limitation that the corpus may not fully represent real-world misinformation is a validity caveat, not circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on Wikidata as ground truth, on one generator's outputs as representative misinformation, and on automatized quality filters validated on only 500 human-labeled examples. The only hand-set numeric parameter is the semantic matching threshold alpha=0.3. No invented entities such as new forces, particles, or mediators are proposed.

free parameters (1)
  • Semantic matching threshold alpha = 0.3
    Chosen manually in Section 2.4 and Appendix F.2 to filter out question-misinformation pairs with low embedding similarity. This threshold affects which claims enter the benchmark and therefore all downstream measurements.
assumptions (4)
  • domain assumption Wikidata triplets and entity descriptions are accurate ground truth for the origin claims.
    Section 2.1 treats Wikidata as an extensive repository of structured real-world facts. If a triplet is wrong or stale, the corresponding misinformation is invalid.
  • ad hoc to paper LLaMA-3-70B-generated misinformation is representative of real-world misinformation.
    Sections 2.2 and 2.3 use one generator for all claims and styles. The limitation section admits the dataset may not cover all real-world variations, yet representativeness is assumed for the findings.
  • domain assumption The NLI model and SentenceTransformer correctly detect entailment, contradiction, and relevance for quality control.
    Section 2.4 uses deberta-small-long-nli and all-mpnet-base-v2 to filter data, with human validation on only 500 examples.
  • domain assumption The 2WikiMultihopQA-derived multi-hop facts are correct and the retained inference and compositional questions are answerable from Wikidata.
    Section 2.1 uses 2WikiMultihopQA as the multi-hop source without reporting validation of the constructed conflict pairs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of How does Misinformation Affect Large Language Model Behaviors and Preferences?." pith.science (2026). https://pith.science/paper/ZT6BC4PF

@misc{pith2026250521608,
  author       = {Pith},
  title        = {Pith review of: How does Misinformation Affect Large Language Model Behaviors and Preferences?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZT6BC4PF}},
  note         = {Machine review of arXiv:2505.21608}
}
read the original abstract

Large Language Models (LLMs) have shown remarkable capabilities in knowledge-intensive tasks, while they remain vulnerable when encountering misinformation. Existing studies have explored the role of LLMs in combating misinformation, but there is still a lack of fine-grained analysis on the specific aspects and extent to which LLMs are influenced by misinformation. To bridge this gap, we present MisBench, the current largest and most comprehensive benchmark for evaluating LLMs' behavior and knowledge preference toward misinformation. MisBench consists of 10,346,712 pieces of misinformation, which uniquely considers both knowledge-based conflicts and stylistic variations in misinformation. Empirical results reveal that while LLMs demonstrate comparable abilities in discerning misinformation, they still remain susceptible to knowledge conflicts and stylistic variations. Based on these findings, we further propose a novel approach called Reconstruct to Discriminate (RtD) to strengthen LLMs' ability to detect misinformation. Our study provides valuable insights into LLMs' interactions with misinformation, and we believe MisBench can serve as an effective benchmark for evaluating LLM-based detectors and enhancing their reliability in real-world applications. Codes and data are available at https://github.com/GKNL/MisBench.

Figures

Figures reproduced from arXiv: 2505.21608 by the authors.

Figure 1
Figure 1. An overview of domains in MISBENCH. et al., 2021), or fictional knowledge (Goldstein et al., 2023). This vulnerability to misinformation significantly impacts their real-world performance, undermining their reliability and trustworthiness in practical applications. Following the emergence of LLMs, researchers have established various benchmarks to investi￾gate how misinformation affects these models, in￾cluding LLMF… view at source ↗
Figure 2
Figure 2. Overall illustration of data generation pipeline of [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Examples of stylized factual misinformation. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: Memorization Ratio MR of various LLMs under three types of one-hop based misinformation. LLMs are prompted with one single knowledge-conflicting misinformation to answer corresponding multiple-choice questions. Higher MR indicates LLMs more stick to their parametric co…
Figure 5
Figure 5. Figure 5: Evidence Tendency T endCM of various LLMs under a pair of conflicting evidences with prior internal knowledge. LLMs are prompted with two knowledge-conflicting evidences to answer multiple-choice questions. Higher T endCM (ranges from [−1, 1]) indicates LLMs more tend …
Figure 6
Figure 6. Figure 6: Memorization Ratio MR of LLMs under multi-hop based misinformation with different textual styles. Regularization is applied to the results to facili￾tate the observation of differences across six styles. The convincingness of misinformation to LLMs correlates with text…
Figure 7
Figure 7. Figure 7: Log probability distribution of correct options [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Relation Distribution Statistics of one-hop claims (a) and multi-hop claims (b) in [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Word Cloud Distribution of factual misinformation(a), temporal misinformation(b) and semantic misinfor [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Context-question Similarity Distribution of one-hop misinformation stylized in Wikipedia Entry(a), Sci [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: Memorization Ratio MR of various LLMs under one-hop based misinformation with different tex￾tual styles in MISBENCH. Regularization is applied to the results to facilitate the observation of differences across six styles. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Memorization Ratio MR of various LLMs under three types of multi-hop based misinformation. LLMs are prompted with one single knowledge-conflicting misinformation to answer corresponding multiple choice question. Higher MR indicates LLMs more stick to their parametric …
Figure 13
Figure 13. Figure 13: Evidence Tendency T endCM of various LLMs under a pair of conflicting evidences with prior internal knowledge. LLMs are prompted with two knowledge-conflicting evidences (correct evidence and one-hop based misinformation) to answer corresponding multiple choice questi…
Figure 14
Figure 14. Figure 14: Evidence Tendency T endCM of various LLMs under a pair of conflicting evidences with prior internal knowledge. LLMs are prompted with two knowledge-conflicting evidences (correct evidence and multi-hop based misinformation) to answer corresponding multiple choice ques…
Figure 15
Figure 15. Figure 15: Evidence Tendency T endCM of various LLMs under a pair of conflicting evidences without prior internal knowledge. LLMs are prompted with two knowledge-conflicting evidences (correct evidence and multi-hop based misinformation) to answer corresponding multiple choice q…
Figure 16
Figure 16. Figure 16: Log probability distribution of correct options when LLMs correctly answer to questions under [PITH_FULL_IMAGE:figures/full_fig_p020_16.png]
Figure 17
Figure 17. Figure 17: Log probability distribution of correct options when LLMs correctly answer to questions under [PITH_FULL_IMAGE:figures/full_fig_p021_17.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

67 extracted references · 46 canonical work pages

  1. [1]

    https://api.semanticscholar.org/CorpusID:268232499 The claude 3 model family: Opus, sonnet, haiku

  2. [2]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  3. [3]

    AI@Meta. 2024. https://doi.org/10.48550/ARXIV.2407.21783 The llama 3 herd of models . CoRR, abs/2407.21783

    arXiv doi:10.48550/arxiv.2407.21783 2024

  4. [4]

    Henning Bartsch, Ole Jorgensen, Domenic Rosati, Jason Hoelscher - Obermaier, and Jacob Pfau. 2023. Self-consistency of large language models under ambiguity. In BlackboxNLP@EMNLP, pages 89--105. Association for Computational Linguistics

    work page 2023

  5. [5]

    Nicola De Cao, Wilker Aziz, and Ivan Titov. 2021. Editing factual knowledge in language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021 , pages 6491--6506. Association for Computational Linguistics

    work page 2021

  6. [6]

    Canyu Chen and Kai Shu. 2024 a . Can llm-generated misinformation be detected? In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net

    work page 2024

  7. [7]

    Canyu Chen and Kai Shu. 2024 b . Combating misinformation in the age of llms: Opportunities and challenges. AI Mag. , 45(3):354--368

    work page 2024

  8. [8]

    Jiangjie Chen, Wei Shi, Ziquan Fu, Sijie Cheng, Lei Li, and Yanghua Xiao. 2023 a . Say what you mean! large language models speak too positively about negative commonsense knowledge. In ACL (1) , pages 9890--9908. Association for Computational Linguistics

    work page 2023

Show all 67 references
  1. [9]

    Nuo Chen, Yuhan Li, Jianheng Tang, and Jia Li. 2024. Graphwiz: An instruction-following language model for graph computational problems. In KDD , pages 353--364. ACM

    2024

  2. [10]

    Nuo Chen, Yan Wang, Haiyun Jiang, Deng Cai, Yuhan Li, Ziyang Chen, Longyue Wang, and Jia Li. 2023 b . Large language models meet harry potter: A dataset for aligning dialogue agents with characters. In EMNLP (Findings) , pages 8506--8520. Association for Computational Linguistics

    2023

  3. [11]

    Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, and Mor Geva. 2024. Evaluating the ripple effects of knowledge editing in language models. Trans. Assoc. Comput. Linguistics, 12:283--298

    2024

  4. [12]

    DeepSeek-AI. 2024. https://arxiv.org/abs/2405.04434 Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model . Preprint, arXiv:2405.04434

    2024 arXiv

  5. [13]

    Yuan Gao, Xiang Wang, Xiangnan He, Huamin Feng, and Yong - Dong Zhang. 2023. Rumor detection with self-supervised learning on texts and social graph. Frontiers Comput. Sci., 17(4):174611

    2023

  6. [14]

    Soumya Suvra Ghosal, Souradip Chakraborty, Jonas Geiping, Furong Huang, Dinesh Manocha, and Amrit Singh Bedi. 2023. Towards possibilities & impossibilities of ai-generated text detection: A survey. CoRR, abs/2310.15264

    2023 arXiv

  7. [15]

    Goldstein, Girish Sastry, Micah Musser, Renee DiResta, Matthew Gentzel, and Katerina Sedova

    Josh A. Goldstein, Girish Sastry, Micah Musser, Renee DiResta, Matthew Gentzel, and Katerina Sedova. 2023. Generative language models and automated influence operations: Emerging threats and potential mitigations. CoRR, abs/2301.04246

    2023 arXiv

  8. [16]

    Zhijiang Guo, Michael Sejr Schlichtkrull, and Andreas Vlachos. 2022. A survey on automated fact-checking. Trans. Assoc. Comput. Linguistics, 10:178--206

    2022

  9. [17]

    Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenR...

    2023

  10. [18]

    Xanh Ho, Anh - Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing A multi-hop QA dataset for comprehensive evaluation of reasoning steps. In Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain (Online...

    2020

  11. [19]

    Giwon Hong, Jeonghwan Kim, Junmo Kang, Sung - Hyon Myaeng, and Joyce Jiyoung Whang. 2024. Why so gullible? enhancing the robustness of retrieval-augmented models against counterfactual noise. In NAACL-HLT (Findings) , pages 2474--2495. Association for Computational Linguistics

    2024

  12. [20]

    Cheng Hsu, Cheng - Te Li, Diego S \' a ez - Trumper, and Yi - Zhan Hsu. 2021. Wikicontradiction: Detecting self-contradiction articles on wikipedia. In IEEE BigData , pages 427--436. IEEE

    2021

  13. [21]

    Beizhe Hu, Qiang Sheng, Juan Cao, Yuhui Shi, Yang Li, Danding Wang, and Peng Qi. 2024. Bad actor, good advisor: Exploring the role of large language models in fake news detection. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference o...

    2024

  14. [22]

    Xuming Hu, Zhijiang Guo, Junzhe Chen, Lijie Wen, and Philip S. Yu. 2023. MR2: A benchmark for multimodal retrieval-augmented rumor detection in social media. In SIGIR , pages 2901--2912. ACM

    2023

  15. [23]

    Zhuoran Jin, Pengfei Cao, Yubo Chen, Kang Liu, Xiaojian Jiang, Jiexin Xu, Qiuxia Li, and Jun Zhao. 2024. Tug-of-war between knowledge: Exploring and resolving knowledge conflicts in retrieval-augmented language models. In LREC/COLING , pages 16867--16878. ELRA and ICCL

    2024

  16. [24]

    Smith, Yejin Choi, and Kentaro Inui

    Jungo Kasai, Keisuke Sakaguchi, Yoichi Takahashi, Ronan Le Bras, Akari Asai, Xinyan Yu, Dragomir Radev, Noah A. Smith, Yejin Choi, and Kentaro Inui. 2023. Realtime QA: what's the answer right now? In NeurIPS

    2023

  17. [25]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating S...

    2023

  18. [26]

    Angeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d'Autume, Tom \' a s Kocisk \' y , Sebastian Ruder, Dani Yogatama, Kris Cao, Susannah Young, and Phil Blunsom. 2021. Mind the gap: Assessing tempor...

    2021

  19. [27]

    Haoran Li, Dadi Guo, Wei Fan, Mingshi Xu, Jie Huang, Fanpu Meng, and Yangqiu Song. 2023. Multi-step jailbreaking privacy attacks on chatgpt. In EMNLP (Findings) , pages 4138--4153. Association for Computational Linguistics

    2023

  20. [28]

    Jierui Li, Vipul Raheja, and Dhruv Kumar. 2024 a . Contradoc: Understanding self-contradictions in documents with large language models. In NAACL-HLT , pages 6509--6523. Association for Computational Linguistics

    2024

  21. [29]

    Yafu Li, Qintong Li, Leyang Cui, Wei Bi, Zhilin Wang, Longyue Wang, Linyi Yang, Shuming Shi, and Yue Zhang. 2024 b . MAGE: machine-generated text detection in the wild. In ACL (1) , pages 36--53. Association for Computational Linguistics

    2024

  22. [30]

    Yuhan Li, Zhixun Li, Peisong Wang, Jia Li, Xiangguo Sun, Hong Cheng, and Jeffrey Xu Yu. 2024 c . A survey of graph meets large language model: progress and future directions. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pages 8123--8131

    2024

  23. [31]

    Yuhan Li, Xinni Zhang, Linhao Luo, Heng Chang, Yuxiang Ren, Irwin King, and Jia Li. 2025. G-refer: Graph retrieval-augmented large language model for explainable recommendation. In Proceedings of the ACM on Web Conference 2025, pages 240--251

    2025

  24. [32]

    Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. 2021. Entity-based knowledge conflicts in question answering. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / P...

    2021

  25. [33]

    Abdurahman Maarouf, Dominik B \" a r, Dominique Geissler, and Stefan Feuerriegel. 2024. HQP: A human-annotated dataset for detecting online propaganda. In ACL (Findings) , pages 6064--6089. Association for Computational Linguistics

    2024

  26. [34]

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proceedings of the 61st Annual Meeting of the Association for Compu...

    2023

  27. [35]

    Giovanni Da San Martino, Stefano Cresci, Alberto Barr \' o n - Cede \ n o, Seunghak Yu, Roberto Di Pietro, and Preslav Nakov. 2020. A survey on computational propaganda detection. In IJCAI , pages 4826--4832. ijcai.org

    2020

  28. [36]

    Yikang Pan, Liangming Pan, Wenhu Chen, Preslav Nakov, Min - Yen Kan, and William Yang Wang. 2023. On the risk of misinformation pollution with large language models. In EMNLP (Findings) , pages 1389--1403. Association for Computational Linguistics

    2023

  29. [37]

    Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, and Jianfeng Gao. 2023. Check your facts and try again: Improving large language models with external knowledge and automated feedback. CoRR, abs/2302.12813

    2023 arXiv

  30. [38]

    Miao Peng, Ben Liu, Qianqian Xie, Wenjie Xu, Hua Wang, and Min Peng. 2022. Smile: Schema-augmented multi-level contrastive learning for knowledge graph link prediction. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 4165--4177

    2022

  31. [39]

    Miao Peng, Ben Liu, Wenjie Xu, Zihao Jiang, Jiahui Zhu, and Min Peng. 2024. Deja vu: Contrastive historical modeling with prefix-tuning for temporal knowledge graph reasoning. In NAACL-HLT (Findings) , pages 1178--1191. Association for Computational Linguistics

    2024

  32. [40]

    merge conflicts!

    Cheng Qian, Xinran Zhao, and Sherry Tongshuang Wu. 2023. "merge conflicts!" exploring the impacts of external distractors to parametric knowledge graphs. CoRR, abs/2309.08594

    2023 arXiv

  33. [41]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9

    2019

  34. [42]

    Harsh Raj, Vipul Gupta, Domenic Rosati, and Subhabrata Majumdar. 2023. Semantic consistency for assuring reliability of large language models. CoRR, abs/2308.09138

    2023 arXiv

  35. [43]

    Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton - Brown, and Yoav Shoham. 2023. https://doi.org/10.1162/TACL\_A\_00605 In-context retrieval-augmented language models . Trans. Assoc. Comput. Linguistics, 11:1316--1331

    2023 doi

  36. [44]

    Nils Reimers and Iryna Gurevych. 2019. https://arxiv.org/abs/1908.10084 Sentence-bert: Sentence embeddings using siamese bert-networks . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics

    2019 arXiv

  37. [45]

    Arkhipov, Alexander Panchenko, and Chris Biemann

    \" O zge Sevgili, Artem Shelmanov, Mikhail Y. Arkhipov, Alexander Panchenko, and Chris Biemann. 2022. Neural entity linking: A survey of models based on deep learning. Semantic Web, 13(3):527--570

    2022

  38. [46]

    Qiang Sheng, Juan Cao, Xueyao Zhang, Rundong Li, Danding Wang, and Yongchun Zhu. 2022. Zoom out and observe: News environment perception for fake news detection. In ACL (1) , pages 4543--4556. Association for Computational Linguistics

    2022

  39. [47]

    Zhaochen Su, Jun Zhang, Xiaoye Qu, Tong Zhu, Yanshu Li, Jiashuo Sun, Juntao Li, Min Zhang, and Yu Cheng. 2024. Conflictbank: A benchmark for evaluating the influence of knowledge conflicts in llm. arXiv preprint arXiv:2408.12076

    2024 arXiv

  40. [48]

    Hexiang Tan, Fei Sun, Wanli Yang, Yuanzhuo Wang, Qi Cao, and Xueqi Cheng. 2024. Blinded by generated contexts: How language models merge generated and retrieved contexts when knowledge conflicts? In ACL (1) , pages 6207--6227. Association for Computational Linguistics

    2024

  41. [49]

    Jianheng Tang, Qifan Zhang, Yuhan Li, Nuo Chen, and Jia Li. 2025. Grapharena: Evaluating and exploring large language models on graph computation. In The Thirteenth International Conference on Learning Representations

    2025

  42. [50]

    Gemma Team. 2024 a . https://doi.org/10.34740/KAGGLE/M/3301 Gemma

    2024 doi

  43. [51]

    Qwen Team. 2024 b . https://qwenlm.github.io/blog/qwen2.5/ Qwen2.5: A party of foundation models

    2024

  44. [52]

    Juraj Vladika and Florian Matthes. 2023. Scientific fact-checking: A survey of resources and approaches. In ACL (Findings) , pages 6215--6230. Association for Computational Linguistics

    2023

  45. [53]

    Denny Vrande c i \'c and Markus Kr \"o tzsch. 2014. Wikidata: a free collaborative knowledgebase. Communications of the ACM, 57(10):78--85

    2014

  46. [54]

    Alexander Wan, Eric Wallace, and Dan Klein. 2024 a . What evidence do language models find convincing? In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024 , pages 74...

    2024

  47. [55]

    Herun Wan, Shangbin Feng, Zhaoxuan Tan, Heng Wang, Yulia Tsvetkov, and Minnan Luo. 2024 b . DELL: generating reactions and explanations for llm-based misinformation detection. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual...

    2024

  48. [56]

    Jiaying Wu, Jiafeng Guo, and Bryan Hooi. 2024 a . Fake news in sheep's clothing: Robust fake news detection against llm-empowered style attacks. In KDD , pages 3367--3378. ACM

    2024

  49. [57]

    Wong, and Lidia S

    Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Derek F. Wong, and Lidia S. Chao. 2023. A survey on llm-generated text detection: Necessity, methods, and future directions. CoRR, abs/2310.14724

    2023 arXiv

  50. [58]

    Wong, Shu Yang, Xinyi Yang, Yulin Yuan, and Lidia S

    Junchao Wu, Runzhe Zhan, Derek F. Wong, Shu Yang, Xinyi Yang, Yulin Yuan, and Lidia S. Chao. 2024 b . Detectrl: Benchmarking llm-generated text detection in real-world scenarios. CoRR, abs/2410.23746

    2024 arXiv

  51. [59]

    Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, and Yu Su. 2024. Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-...

    2024

  52. [60]

    Lin, Shujian Yang, Tianqi Zhang, Weiyan Shi, Tianwei Zhang, Zhixuan Fang, Wei Xu, and Han Qiu

    Rongwu Xu, Brian S. Lin, Shujian Yang, Tianqi Zhang, Weiyan Shi, Tianwei Zhang, Zhixuan Fang, Wei Xu, and Han Qiu. 2024. The earth is flat because...: Investigating llms' belief towards misinformation via persuasive conversation. In ACL (1) , pages 16259--16303. Association fo...

    2024

  53. [61]

    Wenjie Xu, Ben Liu, Miao Peng, Xu Jia, and Min Peng. 2023. Pre-trained language model with prompts for temporal knowledge graph completion. In ACL (Findings) , pages 7790--7803. Association for Computational Linguistics

    2023

  54. [62]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. In NeurIPS

    2023

  55. [63]

    Ghorbani

    Xichen Zhang and Ali A. Ghorbani. 2020. An overview of online fake news: Characterization, detection, and discussion. Inf. Process. Manag., 57(2):102025

    2020

  56. [64]

    Xinyi Zhou and Reza Zafarani. 2021 a . A survey of fake news: Fundamental theories, detection methods, and opportunities. ACM Comput. Surv. , 53(5):109:1--109:40

    2021

  57. [65]

    Xinyi Zhou and Reza Zafarani. 2021 b . A survey of fake news: Fundamental theories, detection methods, and opportunities. ACM Comput. Surv. , 53(5):109:1--109:40

    2021

  58. [66]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  59. [67]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.