REVIEW 3 cited by
Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queries
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Pretrained language models have been suggested as a possible alternative or complement to structured knowledge bases. However, this emerging LM-as-KB paradigm has so far only been considered in a very limited setting, which only allows handling 21k entities whose single-token name is found in common LM vocabularies. Furthermore, the main benefit of this paradigm, namely querying the KB using a variety of natural language paraphrases, is underexplored so far. Here, we formulate two basic requirements for treating LMs as KBs: (i) the ability to store a large number facts involving a large number of entities and (ii) the ability to query stored facts. We explore three entity representations that allow LMs to represent millions of entities and present a detailed case study on paraphrased querying of world knowledge in LMs, thereby providing a proof-of-concept that language models can indeed serve as knowledge bases.
Forward citations
Cited by 3 Pith papers
-
How much do language models memorize?
A compression-based measurement puts GPT-style model memorization capacity at roughly 3.6 bits per parameter, with membership inference success following a sigmoid in the dataset-to-capacity ratio.
-
RealFactBench: A Benchmark for Evaluating Large Language Models in Real-World Fact-Checking
A new 6K-claim benchmark evaluates LLMs and multimodal LLMs on real-world fact-checking with an explicit 'unknown' option and shows web search and multimodal input improve performance.
-
A Graph Perspective to Probe Structural Patterns of Knowledge in Large Language Models
LLM knowledge, measured by self-reported true/false checks on knowledge-graph triplets, shows homophily and degree correlations that a graph neural network exploits to select more effective fine-tuning data.
Discussion (0). Continue with ORCID to comment.