REVIEW 7 cited by
Inside-Out: Hidden Factual Knowledge in LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Inside-Out: Hidden Factual Knowledge in LLMs
read the original abstract
This work presents a framework for assessing whether large language models (LLMs) encode more factual knowledge in their parameters than what they express in their outputs. While a few studies hint at this possibility, none has clearly defined or demonstrated this phenomenon. We first propose a formal definition of knowledge, quantifying it for a given question as the fraction of correct-incorrect answer pairs where the correct one is ranked higher. This gives rise to external and internal knowledge, depending on the information used to score individual answer candidates: either the model's observable token-level probabilities or its intermediate computations. Hidden knowledge arises when internal knowledge exceeds external knowledge. We then present a case study, applying this framework to three popular open-weights LLMs in a closed-book QA setup. Our results indicate that: (1) LLMs consistently encode more factual knowledge internally than what they express externally, with an average relative gap of 40%. (2) Surprisingly, some knowledge is so deeply hidden that a model can internally know an answer perfectly, yet fail to generate it even once, despite large-scale repeated sampling of 1,000 answers. This reveals fundamental limitations in the generation capabilities of LLMs, which (3) put a practical constraint on scaling test-time compute via repeated answer sampling in closed-book QA: significant performance improvements remain inaccessible because some answers are practically never sampled, yet if they were, we would be guaranteed to rank them first.
Forward citations
Cited by 7 Pith papers
-
Improving LLMs via Validator-to-Generator Alignment
Frequency-corrected rank alignment of an LLM generator to its own validator improves generator AUROC and G-V Pearson correlation by up to 27 points while preserving validator quality.
-
Cross-Lingual Exploration for Parametric Knowledge
Cross-lingual prompt exploration improves factual recall and consistency in LLMs across 17 languages more efficiently than native-language scaling.
-
DECOR: Auditing LLM Deception via Information Manipulation Theory
DECOR introduces a theory-grounded multi-agent system that decomposes contexts into atomic units, scores four manipulation dimensions per unit, and aggregates profiles into a global deception index, reporting SOTA res...
-
Hallucinations Undermine Trust; Metacognition is a Way Forward
LLMs need metacognition to align expressed uncertainty with their actual knowledge boundaries, moving beyond knowledge expansion to reduce confident errors.
-
Input-Time Scaling: Adding Noise and Irrelevance into Less-Is-More Drastically Improves Reasoning Performance and Efficiency
Adding controlled noise and irrelevant persona contexts across training and testing stages for strong LLMs yields better reasoning and efficiency than high-quality data alone, reaching 76.7% on AIME24/25 with Qwen2.5-32B.
-
Improving Factuality in LLMs via Inference-Time Knowledge Graph Construction
A framework for inference-time knowledge graph construction and expansion improves factual accuracy in LLMs on three QA benchmarks by combining internal LLM knowledge with selective external retrieval.
-
Trustworthy Agents for Electronic Health Records through Confidence Estimation
TrustEHRAgent, an EHR assistant with step-by-step confidence checks, scores 44% and 25% accuracy on MIMIC-III and eICU when only answers it is 70% confident in are counted, while baseline methods score 0%.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.