REVIEW 8 cited by
ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidence
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Retrieval augmented generation (RAG) is frequently used to mitigate hallucinations and provide up-to-date knowledge for large language models (LLMs). However, given that document retrieval is an imprecise task and sometimes results in erroneous or even harmful content being presented in context, this raises the question of how LLMs handle retrieved information: If the provided content is incorrect, does the model know to ignore it, or does it recapitulate the error? Conversely, when the model's initial response is incorrect, does it always know to use the retrieved information to correct itself, or does it insist on its wrong prior response? To answer this, we curate a dataset of over 1200 questions across six domains (e.g., drug dosages, Olympic records, locations) along with content relevant to answering each question. We further apply precise perturbations to the answers in the content that range from subtle to blatant errors. We benchmark six top-performing LLMs, including GPT-4o, on this dataset and find that LLMs are susceptible to adopting incorrect retrieved content, overriding their own correct prior knowledge over 60% of the time. However, the more unrealistic the retrieved content is (i.e. more deviated from truth), the less likely the model is to adopt it. Also, the less confident a model is in its initial response (via measuring token probabilities), the more likely it is to adopt the information in the retrieved content. We exploit this finding and demonstrate simple methods for improving model accuracy where there is conflicting retrieved content. Our results highlight a difficult task and benchmark for LLMs -- namely, their ability to correctly discern when it is wrong in light of correct retrieved content and to reject cases when the provided content is incorrect.
Forward citations
Cited by 8 Pith papers
-
Can a Language Model Learn Facts Continually in Its Weights?
Weight writes create question-keyed knowledge that later sequential writes redirect, so context—not weights—is the reliable channel for facts that must survive or be composed.
-
Information Discernment in Large Language Models
LLMs update their stated numeric beliefs almost regardless of source reliability or whether a claim moves them closer to the truth, performing near chance on both dimensions.
-
When Derived Measurements Mislead: Quantifying and Mitigating LLM Over-Trust with Privileged-Modality Reliability Evidence
DFOT is formalized with five frozen estimands and a matched–shuffled evidence test; on locked PPG–ECG data a privileged student modestly improves repair and specificity over a PPG-only baseline.
-
Proof-Carrying Numbers (PCN): A Protocol for Trustworthy Numeric Answers from LLMs via Claim Verification
Proof-Carrying Numbers binds each displayed LLM number to a structured claim and only marks it verified after a policy-based mechanical check, leaving all other numbers unverified.
-
DeepTRACE: Auditing Deep Research AI Systems for Tracking Reliability Across Citations and Evidence
An audit framework and empirical study showing that generative search engines and deep research agents frequently produce one-sided answers and weakly supported citations, with citation accuracy between 40 and 80%.
-
ECoRAG: Evidentiality-guided Compression for Long Context RAG
Evidentiality-guided, adaptive context compression improves open-domain QA accuracy and token efficiency over standard RAG and prior compressors.
-
MissingBench-Verified: Probing Vision-Language Models' Inability to Detect Missing Object Parts
Ten leading VLMs mostly fail to report removed essential object parts as missing, and simulated detector evidence, image tools, longer reasoning, and an easier fine-tune barely improve accuracy.
-
Building Task Bots with Self-learning for Enhanced Adaptability, Extensibility, and Factuality
A thesis that combines self-learning from dialog logs, schema-guided prompting, and self-aligned factuality to build task bots with minimal human intervention.
Discussion (0). Continue with ORCID to comment.