REVIEW 3 cited by
Are Pretrained Language Models Symbolic Reasoners Over Knowledge?
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
How can pretrained language models (PLMs) learn factual knowledge from the training set? We investigate the two most important mechanisms: reasoning and memorization. Prior work has attempted to quantify the number of facts PLMs learn, but we present, using synthetic data, the first study that investigates the causal relation between facts present in training and facts learned by the PLM. For reasoning, we show that PLMs seem to learn to apply some symbolic reasoning rules correctly but struggle with others, including two-hop reasoning. Further analysis suggests that even the application of learned reasoning rules is flawed. For memorization, we identify schema conformity (facts systematically supported by other facts) and frequency as key factors for its success.
Forward citations
Cited by 3 Pith papers
-
MALAMUTE: A Multilingual, Highly-granular, Template-free, Education-based Probing Dataset
MALAMUTE is a 116k-prompt cloze-style dataset derived from 71 university textbooks in three languages, used to probe language models' fine-grained subject knowledge.
-
The Power of Power Law: Asymmetry Enables Compositional Reasoning
Power-law data sampling creates beneficial asymmetry in the loss landscape that lets models acquire high-frequency skill compositions first, enabling more efficient learning of rare long-tail skills than uniform distr...
-
Automatic High-quality Verilog Assertion Generation through Subtask-Focused Fine-Tuned LLMs and Iterative Prompting
AssertCraft generates SystemVerilog assertions from specification documents using subtask decomposition, fine-tuned GPT-3.5, and iterative compiler-guided repair, reporting 7.3x more correct assertions than a plain pr...
Discussion (0). Continue with ORCID to comment.