REVIEW 5 cited by
Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
By providing external information to large language models (LLMs), tool augmentation (including retrieval augmentation) has emerged as a promising solution for addressing the limitations of LLMs' static parametric memory. However, how receptive are LLMs to such external evidence, especially when the evidence conflicts with their parametric memory? We present the first comprehensive and controlled investigation into the behavior of LLMs when encountering knowledge conflicts. We propose a systematic framework to elicit high-quality parametric memory from LLMs and construct the corresponding counter-memory, which enables us to conduct a series of controlled experiments. Our investigation reveals seemingly contradicting behaviors of LLMs. On the one hand, different from prior wisdom, we find that LLMs can be highly receptive to external evidence even when that conflicts with their parametric memory, given that the external evidence is coherent and convincing. On the other hand, LLMs also demonstrate a strong confirmation bias when the external evidence contains some information that is consistent with their parametric memory, despite being presented with conflicting evidence at the same time. These results pose important implications that are worth careful consideration for the further development and deployment of tool- and retrieval-augmented LLMs. Resources are available at https://github.com/OSU-NLP-Group/LLM-Knowledge-Conflict.
Forward citations
Cited by 5 Pith papers
-
Deceptive Grounding: Entity Attribution Failure in Clinical Retrieval-Augmented Generation
Clinical RAG can attribute real evidence about drug Y to queried drug X at high rates under adversarial retrieval, a failure invisible to faithfulness and citation metrics but detectable by entity-attribution verification.
-
Lethe: Purifying Backdoored Large Language Models with Knowledge Dilution
LETHE uses parameter-level model merging plus prompt-level word definitions to dilute backdoor behavior in LLMs, cutting attack success to below 7% in most tested settings.
-
Exploring the Impact of Instruction-Tuning on LLM's Susceptibility to Misinformation
Instruction tuning shifts LLM susceptibility to misinformation from assistant-provided text to user-provided text, making user-role misinformation more likely to override the model's correct parametric knowledge.
-
CausalAbstain: Enhancing Multilingual LLMs with Causal Reasoning for Trustworthy Abstention
CausalAbstain filters multilingual self-feedback by comparing how much it changes the model's abstention decision, improving abstention accuracy over baselines on two benchmarks.
-
Small Encoders Can Rival Large Decoders in Detecting Groundedness
Task-specific encoders (e.g., RoBERTa-large) rival large decoders such as Llama-3-8B and GPT-4o on binary groundedness detection, within 5 to 10 accuracy points while requiring one to three orders of magnitude fewer FLOPs.
Discussion (0). Sign in to comment.