REVIEW 6 cited by
Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness?
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness?
read the original abstract
Neural language models (LMs) can be used to evaluate the truth of factual statements in two ways: they can be either queried for statement probabilities, or probed for internal representations of truthfulness. Past work has found that these two procedures sometimes disagree, and that probes tend to be more accurate than LM outputs. This has led some researchers to conclude that LMs "lie" or otherwise encode non-cooperative communicative intents. Is this an accurate description of today's LMs, or can query-probe disagreement arise in other ways? We identify three different classes of disagreement, which we term confabulation, deception, and heterogeneity. In many cases, the superiority of probes is simply attributable to better calibration on uncertain answers rather than a greater fraction of correct, high-confidence answers. In some cases, queries and probes perform better on different subsets of inputs, and accuracy can further be improved by ensembling the two. Code is available at github.com/lingo-mit/lm-truthfulness.
Forward citations
Cited by 6 Pith papers
-
Exposure is not manifestation: measurement target and output resolution jointly determine which behavioural-faithfulness evaluator wins
Small hyperbolic models (146M–3B) report 100% creative-seed preference, 90.7% compliance-gap detection, and a selective-gating skeleton–wallpaper memory pilot as a companion-AI stack.
-
Exposure is not manifestation: measurement target and output resolution jointly determine which behavioural-faithfulness evaluator wins
On the same 720 replies, scoring exposure versus manifestation shifts the auditor-judge gap by ~0.2 AUROC and can reverse their ranking, so single detection AUROCs are under-specified.
-
TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Robotics
The log-probability a VLM assigns to 'True' for 'does this video prefix complete the task?' is used as a zero-shot dense progress reward that outperforms GVL on open-source models.
-
Entropy Sentinel: Probing Entropy Traces for LLM Monitoring
Top-k decoding-entropy profiles can estimate and rank held-out domain accuracy for most tested LLMs, with difficulty-diverse training data the main success factor.
-
Entropy Sentinel: Probing Entropy Traces for LLM Monitoring
Entropy traces from top-k logprobs, run through a simple classifier, can rank and estimate LLM accuracy on unseen STEM benchmarks—but reliability varies by model.
-
ISACL: Internal State Analyzer for Copyrighted Training Data Leakage
An MLP trained on LLM internal states predicts Rouge-L-defined literal copying leakage with high accuracy, but not paraphrase-level leakage.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.