CDS-trained BabyLMs show earlier and more appropriate production in a new frame-completion task while FineWeb-edu models lead on comprehension benchmarks, indicating current tests underestimate CDS benefits.
How do large language models acquire factual knowledge during pretraining? arXiv preprint arXiv:2406.11813
2 Pith papers cite this work, alongside 3 external citations. Polarity classification is still indexing.
2
Pith papers citing it
3
external citations · external index
citation-role summary
background 1
citation-polarity summary
fields
cs.CL 2verdicts
UNVERDICTED 2roles
background 1polarities
unclear 1representative citing papers
MEMOed framework attributes LLM generations about cultures to pretraining memorization and finds frequency-based biases across 110 cultures for food and clothing.
citing papers explorer
-
Child-directed speech facilitates production, not comprehension, in BabyLMs
CDS-trained BabyLMs show earlier and more appropriate production in a new frame-completion task while FineWeb-edu models lead on comprehension benchmarks, indicating current tests underestimate CDS benefits.
-
Attributing Culture-Conditioned Generations to Pretraining Corpora
MEMOed framework attributes LLM generations about cultures to pretraining memorization and finds frequency-based biases across 110 cultures for food and clothing.