Pith. sign in

REVIEW 3 major objections 6 minor 64 references

A Distributional Perspective on Word Learning in Neural Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Text-only language models acquire words in an order that is largely unrelated to children's word-learning order.

desk verdict New metric family and a useful negative result, but the OOD evaluation set for CHILDES/Unified models leaves the core claim only partly settled. read the letter →

arxiv 2502.05892 v1 pith:U4VMREOZ submitted 2025-02-09 cs.CL cs.AI

Filippo Ficarra , Ryan Cotterell , Alex Warstadt This is my paper · ORCID
classification cs.CLcs.AI
keywords wordlearninglanguagemodelsdistributionalsignaturesageofacquisitiontrajectorieschildsurprisallexicaldevelopment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether neural language models trained on text learn words in the same order that children do. It defines nine distributional signatures of lexical knowledge, each measuring how well a model knows where a target word belongs, where it does not belong, and how strongly it prefers the word in context, and it tracks these signatures across training for small transformer language models trained from scratch on three corpora. Across all nine signatures and all training corpora, the model's age-of-acquisition order correlates only weakly with children's ages of acquisition, with the largest absolute Pearson correlation around 0.31 and several signatures near zero or negative. The paper also finds that the signatures capture largely complementary information, so evaluating word learning with a single metric is misleading. If these results hold, text-only pretraining, even on child-directed data, does not currently produce human-like word-learning trajectories.

What carries the argument

The central object is the family of nine distributional signatures, each an expectation of the model's negative log-probability (or divergence from a reference model) for a target word $w$ in contexts drawn from one of three distributions: contexts where $w$ occurs ($p_\kappa(c \mid w)$), contexts where it does not ($p_\kappa(c \mid \neg w)$), and all contexts ($p_\kappa(c)$). The three context types are crossed with three ways of weighting the expectation: against the true context distribution estimated by Monte Carlo sampling, against the model's own intrinsic context distribution, and against a larger reference model's distribution. The argument is carried by comparing these signatures' learning trajectories and by extracting an age of acquisition using a Cauchy-sequence convergence criterion, which handles trajectories that are non-monotonic.

What would settle it

Re-estimate the nine signatures using contexts sampled from child-directed speech corpora or from each model's own training distribution instead of the held-out test corpus; if any signature's correlation with children's age of acquisition rises above roughly 0.5, the paper's conclusion that LM word-learning order is poorly aligned with children's would be refuted for that signature.

Watch

Extended reading notes

Core claim

The central claim is that distributional signatures of lexical knowledge form a design space, and that when this space is explored systematically, language models trained from scratch do not acquire words in a human-like order. The paper formalizes the earlier positive-context surprisal measure as an estimator of an expected log-loss under the true context distribution, then extends it to negative contexts, all contexts, intrinsic expectations taken under the model's own distribution, and reference comparisons against a larger pretrained model. Tracking these nine signatures through training shows they are largely mutually complementary: positive-context signatures correlate with each other and with word frequency, while negative and intrinsic signatures behave differently and some fail to converge. When ages of acquisition are extracted with a Cauchy-convergence criterion, none of the nine signature orders shows a strong correlation with children's Wordbank ages; the strongest absolute Pearson correlation is about 0.31, and several are near zero or negative. The paper concludes that current text-only LMs are poor models of human word-learning trajectories and that the signature family should be used in future evaluations of more human-like training methods.

Load-bearing premise

The load-bearing premise is that the 100 positive and 100 negative contexts sampled from the held-out test corpus fairly represent the true distributions of contexts in which each word does and does not appear; if those samples are unrepresentative of the input children receive, the estimated trajectories and ages of acquisition, and therefore the poor correlation with child data, could be an artifact of the evaluation contexts rather than a property of the models.

Editorial extensions

If this is right

  • Word-learning evaluations that rely only on positive-context surprisal miss substantial distributional information, so future evaluations should report a battery of complementary signatures.
  • Text-only language models trained from scratch, including on child-directed data, do not reproduce the human word-acquisition order under any of the nine signatures.
  • The claimed advantage of multimodal or interaction-trained models becomes directly testable with this signature family, since the paper shows a single metric can be misleading.
  • Threshold-based age-of-acquisition extraction is unreliable for non-monotonic signatures, and convergence-based extraction is needed for fair comparisons.
  • Larger training corpora improve the convergence and predictability of model ages of acquisition, so dataset size must be controlled when comparing learning trajectories across models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The near-zero correlations may partly reflect the choice to sample evaluation contexts from the held-out test corpus rather than from child-directed input; testing with child-directed contexts is a direct experiment the authors did not run.
  • Because the true context distributions are approximated with only 100 sampled contexts per word, Monte Carlo error could be large for less frequent words, and estimators that use a large pretrained model to approximate the context distribution might yield different trajectories.
  • The results indirectly favor grounded or interactive training as the likely route to human-like word learning, but this is only an inference; it would need to be confirmed by showing such models raise the correlations on these signatures.
  • The Cauchy-convergence age-of-acquisition definition is a general trajectory statistic that could be applied to other developmental comparisons, such as syntax or morphology learning curves.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper defines a family of nine distributional signatures of a word's lexical knowledge in a language model, organized along three axes: positive vs. negative vs. all contexts; expectation under the true context distribution vs. under the LM's own distribution; and absolute surprisal vs. divergence from a reference LM. It trains GPT-2 models from scratch on CHILDES, BabyLM, and Unified corpora, extracts word ages of acquisition from trajectories via a Cauchy-convergence criterion, and compares the resulting acquisition orders with Wordbank child AoA and with interpretable lexical predictors. The main empirical findings are that the signatures are largely complementary; that no signature yields an absolute Pearson correlation above about 0.31 with child AoA; and that log frequency dominates LM AoA prediction in a way that is not mirrored in children.

Significance. If the results hold, the paper makes a substantial methodological contribution: it formalizes Chang and Bergen's surprisal measure as a Monte Carlo estimator, proposes a principled typology of distributional signatures, and provides evidence that single-metric evaluations of word learning in LMs are inadequate. The negative result—that current text-only LMs do not reproduce children's word-acquisition order under any of the nine signatures—is important for the cognitive-modeling research program. The paper also ships reproducible code, uses multiple random seeds, and includes robustness checks over the convergence threshold epsilon. The main risk to the empirical conclusion is the use of a single evaluation context distribution across models trained on different corpora, which is the focus of my major comments.

major comments (3)
  1. [§5.1, Fig. 2] For all models, including those trained on CHILDES and Unified, the 100 positive and 100 negative contexts used to estimate every signature are sampled from the BabyLM test set, and the paper states that 'we use the same test contexts for all models, regardless of training data.' Because CHILDES- and Unified-trained models were never trained on BabyLM text, these contexts are largely out-of-distribution for them. The measured signatures can then reflect the model's behavior on unfamiliar context types rather than its acquired lexical knowledge, and the resulting AoA rankings may be compressed or reordered. The pattern in Fig. 2—the BabyLM model, whose training distribution matches the evaluation contexts, shows the strongest positive correlations, while the CHILDES model, the most mismatched, tends to show the weakest—is consistent with this artifact. This is load-bearing because it is the basis for the abstract's claim that LM learning trajectories fail to correlate with children's across all metrics. The authors should evaluate each model on held-out contexts from its own training distribution (e.g., a CHILDES test set for the CHILDES model) and report whether the weak correlations persist; without this, the central negative conclusion is not settled for the CHILDES and Unified conditions.
  2. [App. A.2, §5.1] The practical estimator for the intrinsic signatures is not a standard Monte Carlo estimator. Equation (6) plugs the empirical measure of Eq. (5) into Eq. (1), but the 100 contexts are drawn from the BabyLM test set, not from the model's own context distribution q_kappa(·|w); consistency is only claimed in the limit where C includes all of Σ*. For finite M, the estimator can be substantially biased, and this bias is likely to vary across words and training conditions, which can distort both the intrinsic-signature trajectories and the claim in §6.3 that the signatures are complementary. The paper should report sensitivity of bσI+, bσI−, and bσI± to the number of contexts and to the source of contexts, for example by sampling contexts from q itself or by increasing M substantially.
  3. [§7.1, App. G] Figure 2 does not report the number of words entering each correlation, while the appendix shows that convergence rates vary strongly across datasets and signatures; for instance, the CHILDES bσI− row in Table A8 has only 77 words after convergence and outlier removal. If non-convergence is not random, the correlations in Fig. 2 may be based on unrepresentative subsets, and the comparison of weak vs. strong correlations across signatures could change. The authors should report n and confidence intervals in Fig. 2, or show that the pattern of correlations is robust when words with non-converged trajectories are handled differently.
minor comments (6)
  1. [App. B, Eq. (7)] The definitions x_c = log p_kappa(c|w) and y_c = log r(w|c) are inconsistent with Table 1: the reference signatures measure |log q(w|c) − log r(w|c)|, so x_c should be log q(w|c), not log p_kappa(c|w). The distance-metric proof as written is for a different quantity.
  2. [Table A4] The row labels appear to duplicate 'Childes'; the third row should presumably be 'Unified' based on the training-duration description in §5.1.
  3. [Listing 1] The simplified R code has unbalanced parentheses in the cat() calls and a garbled multicollinearity message; these should be corrected so the listing is runnable.
  4. [Throughout] The corpus name is written inconsistently as 'CHILDES' and 'Childes' (e.g., Table A4, App. J), and the reference 'V ong' in the bibliography should be 'Vong'.
  5. [Fig. 1 caption] The list of sample words in the caption is hard to parse; consider rendering the eight words as a separate legend or table.
  6. [§5.1] The sentence 'we sample 100 positive and 100 negative contexts from the BabyLM test set' should specify whether these are unique context types or samples with replacement, since the estimator in Eq. (5) is written as a sum over a bag of contexts.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central empirical claims are measured against external Wordbank data and a pretrained reference LM, with no fitted input renamed as a prediction.

full rationale

The paper's main claims are empirical outcomes rather than consequences of its definitions. The distributional signatures are defined from the LM's own distribution (q), a true underlying distribution (p), and a fixed reference LM (Llama-3.1-8B); none of these definitions presuppose the target result, which is the correlation between LM-derived ages of acquisition and external child AoA from Wordbank. The AoA extraction uses a Cauchy-convergence criterion with a tolerance epsilon, and the paper explicitly checks that rankings are stable across epsilon values (Appendix I), so epsilon is not fitted to the child data. The reference signatures are distance metrics by construction, but the paper does not claim these are predictions of child behavior; they are comparisons between the trained LM and a reference distribution. The only place where the same evaluation contexts are reused across all models is a validity concern, not a circularity: it may bias the estimated trajectories for the CHILDES- and Unified-trained models, but it does not make any 'prediction' equal to an input by construction. Self-citations are present—e.g., the Unified dataset from Constantinescu et al. (2024) and BabyLM from Choshen et al. (2024), both involving the present authors—but they are used as data resources, not as load-bearing evidence for the central negative result; neither is a uniqueness theorem invoked to rule out alternatives. The paper is therefore self-contained against an external benchmark (Wordbank) for its central claim, and no equation reduces a derived result to a fitted value or to the paper's own prior conclusions.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new theoretical entities; the nine signatures are measurement definitions over existing LM distributions. The hand-chosen parameters are the Cauchy tolerance epsilon (0.07), the context sample size M (100 positive and 100 negative), and an unreported moving-average window. The load-bearing axioms are the finite-expected-length assumption on p, the representativeness of BabyLM test contexts for the true context distributions, and the status of Llama-3.1-8B as a meaningful reference target.

free parameters (3)
  • epsilon (Cauchy convergence tolerance) = 0.07
    Tolerance used to define AoA as the earliest step after which the signature trajectory stays within a band of width epsilon. Chosen as an intermediate value after exploring a range; results are shown to be mostly robust to this choice, but AoA values and convergence rates depend on it.
  • M (number of sampled contexts per word) = 100 positive, 100 negative
    Monte Carlo sample size for estimating each signature. Chosen by hand; no sensitivity analysis is reported, and the same contexts are reused across all models, so sampling variance is not quantified.
  • moving average window size = unspecified
    Trajectories are smoothed with a moving average (Section 4), but the window size is not reported, so the smoothing parameter is an unstated free parameter in the AoA extraction pipeline.
assumptions (3)
  • standard math The data-generating distribution p has finite expected length, so the normalizing sums in Eqs. (3) and (4) are finite.
    Invoked in Section 2 to justify the existence of the context distribution p_kappa(c|w); relies on Opedal et al. (2024, Section 2.1).
  • domain assumption Contexts sampled from the BabyLM test set are draws from the true context distributions p_kappa(·|w), p_kappa(·|not w), and p_kappa(·).
    All 'true' signatures are estimated by Monte Carlo with 100 contexts per word from the BabyLM test set (Section 5.1). If these contexts are not representative of the relevant distribution (e.g., child-directed speech), the estimated trajectories are biased.
  • domain assumption The reference LM Llama-3.1-8B provides a meaningful distributional target r for the reference signatures.
    The reference signatures measure the distance between q and r (Section 3, Table 1). The interpretation that this reflects progress toward a ground truth assumes r is a better approximation of the target distribution than q.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Distributional Perspective on Word Learning in Neural Language Models." pith.science (2026). https://pith.science/paper/U4VMREOZ

@misc{pith2026250205892,
  author       = {Pith},
  title        = {Pith review of: A Distributional Perspective on Word Learning in Neural Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U4VMREOZ}},
  note         = {Machine review of arXiv:2502.05892}
}
read the original abstract

Language models (LMs) are increasingly being studied as models of human language learners. Due to the nascency of the field, it is not well-established whether LMs exhibit similar learning dynamics to humans, and there are few direct comparisons between learning trajectories in humans and models. Word learning trajectories for children are relatively well-documented, and recent work has tried to extend these investigations to language models. However, there are no widely agreed-upon metrics for word learning in language models. We take a distributional approach to this problem, defining lexical knowledge in terms of properties of the learned distribution for a target word. We argue that distributional signatures studied in prior work fail to capture key distributional information. Thus, we propose an array of signatures that improve on earlier approaches by capturing knowledge of both where the target word can and cannot occur as well as gradient preferences about the word's appropriateness. We obtain learning trajectories for a selection of small language models we train from scratch, study the relationship between different distributional signatures, compare how well they align with human word learning trajectories and interpretable lexical features, and address basic methodological questions about estimating these distributional signatures. Our metrics largely capture complementary information, suggesting that it is important not to rely on a single metric. However, across all metrics, language models' learning trajectories fail to correlate with those of children.

Figures

Figures reproduced from arXiv: 2502.05892 by the authors.

Figure 1
Figure 1. Trajectories for a sample of 8 words for LMs trained on the Unified dataset. We sample one high [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Pearson’s correlation coefficients between [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

64 extracted references · 40 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Uri Berger, Gabriel Stanovsky, Omri Abend, and Lea Frermann. 2022. https://doi.org/10.18653/v1/2022.naacl-main.280 A computational acquisition model for multimodal word categorization . In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3819--3835

    work page doi:10.18653/v1/2022.naacl-main.280 2022

  4. [4]

    Lois Bloom. 1970. https://mitpress.mit.edu/9780262020565/language-development/ Language Development : Form and Function in Emerging Grammars . MIT Press

    arXiv 1970

  5. [5]

    Marchman, and Michael C

    Mika Braginsky, Daniel Yurovsky, Virginia A. Marchman, and Michael C. Frank. 2019. https://doi.org/10.1162/opmi_a_00026 Consistency and variability in children's word learning across languages . Open Mind: Discoveries in Cognitive Science, 3:52--67

    work page doi:10.1162/opmi_a_00026 2019

  6. [6]

    Marchman, and Mike C

    Mika Braginsky, Daniel Yurovsky, Virginia A. Marchman, and Mike C. Frank. 2016. https://api.semanticscholar.org/CorpusID:8314306 From uh-oh to tomorrow: Predicting age of acquisition for early words across languages . Proceedings of the Annual Meeting of the Cognitive Science Society, 38:1691--1696

    work page 2016

  7. [7]

    Elia Bruni, Gemma Boleda, Marco Baroni, and Nam-Khanh Tran. 2012. https://aclanthology.org/P12-1015/ Distributional semantics in technicolor . In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 136--145, Jeju Island, Korea. Association for Computational Linguistics

    work page 2012

  8. [8]

    Marc Brysbaert, Amy Beth Warriner, and Victor Kuperman. 2014. https://doi.org/10.3758/s13428-013-0403-5 Concreteness ratings for 40 thousand generally known English word lemmas . Behavior Research Methods, 46(3):904--911

    doi:10.3758/s13428-013-0403-5 2014

Show all 64 references
  1. [9]

    Buchanan, K

    Erin M. Buchanan, K. D. Valentine, and Nicholas P. Maxwell. 2019. https://doi.org/10.3758/s13428-019-01243-z English semantic feature production norms: An extended database of 4436 concepts . Behavior Research Methods, 51(4):1849--1863

    2019 doi

  2. [10]

    Chang and Benjamin K

    Tyler A. Chang and Benjamin K. Bergen. 2022. https://doi.org/10.1162/tacl_a_00444 Word acquisition in neural language models . Transactions of the Association for Computational Linguistics, 10:1--16

    2022 doi

  3. [11]

    Chang, Zhuowen Tu, and Benjamin K

    Tyler A. Chang, Zhuowen Tu, and Benjamin K. Bergen. 2024. https://aclanthology.org/2024.tacl-1.74 Characterizing learning curves during language model pre-training: Learning, forgetting, and stability . Transactions of the Association for Computational Linguistics, 12:1346--1362

    2024

  4. [12]

    Hu, Tal Linzen, Aaron Mueller, Candace Ross, Alex Warstadt, Ethan Wilcox, Adina Williams, and Chengxu Zhuang

    Leshem Choshen, Ryan Cotterell, Michael Y. Hu, Tal Linzen, Aaron Mueller, Candace Ross, Alex Warstadt, Ethan Wilcox, Adina Williams, and Chengxu Zhuang. 2024. https://arxiv.org/abs/2404.06214 [ C all for papers] the 2nd babylm challenge: Sample-efficient pretraining on a devel...

    2024 arXiv

  5. [13]

    Leshem Choshen, Guy Hacohen, Daphna Weinshall, and Omri Abend. 2022. https://doi.org/10.18653/v1/2022.acl-long.568 The grammar-learning trajectories of neural language models . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, volume 1...

    2022 doi

  6. [14]

    Eve V. Clark. 2018. https://doi.org/10.1080/15475441.2017.1340843 Conversation and Language Acquisition : A Pragmatic Approach . Language Learning and Development, 14(3):170--185

    2018

  7. [15]

    Ionut Constantinescu, Tiago Pimentel, Ryan Cotterell, and Alex Warstadt. 2024. https://arxiv.org/abs/2407.19325 Investigating critical period effects in language acquisition through neural language models . Transactions of the Association for Computational Linguistics

    2024 arXiv

  8. [16]

    Arghya Datta, Subhrangshu Nandi, Jingcheng Xu, Greg Ver Steeg, He Xie, Anoop Kumar, and Aram Galstyan. 2023. https://doi.org/10.18653/v1/2023.findings-acl.176 Measuring and mitigating local instability in deep neural networks . In Findings of the Association for Computational ...

    2023 doi

  9. [17]

    Dumais, George W

    Scott Deerwester, Susan T. Dumais, George W. Furnas, Thomas K. Landauer, and Richard Harshman. 1990. https://asistdl.onlinelibrary.wiley.com/doi/abs/10.1002/ Journal of the American Society for Information Science, 41(6):391--407

    1990

  10. [18]

    Emmanuel Dupoux. 2018. https://doi.org/10.1016/j.cognition.2017.11.008 Cognitive Science in the era of Artificial Intelligence : A roadmap for reverse-engineering the infant language-learner . Cognition, 173:43--59

    2018 doi

  11. [19]

    Linnea Evanson, Yair Lakretz, and Jean R \'e mi King. 2023. https://doi.org/10.18653/v1/2023.findings-acl.773 Language acquisition: do children and language models follow similar learning stages? In Findings of the Association for Computational Linguistics, pages 12205--12218

    2023 doi

  12. [20]

    Larry Fenson, Virginia Marchman , Donna Thal , Phillip Dale , Steven Reznick , and Elizabeth Bates . 2013. https://link.springer.com/referenceworkentry/10.1007/978-1-4419-1698-3_769#citeas MacArthur - Bates C ommunicative D evelopment I nventories . Springer

    2013 doi

  13. [21]

    Lev Finkelstein, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. 2001. https://theory.stanford.edu/ matias/papers/context_search.pdf Placing Search in Context : The Concept Revisited . In Proceedings of the 10th international confere...

    2001

  14. [22]

    J. R. Firth. 1957. https://cir.nii.ac.jp/crid/1570854175539816192 A synopsis of linguistic theory, 1930-1955 . Studies in Linguistic Analysis

    1957

  15. [23]

    Frank, Mika Braginsky, Daniel Yurovsky, and Virginia A

    Michael C. Frank, Mika Braginsky, Daniel Yurovsky, and Virginia A. Marchman. 2021. https://doi.org/10.7551/mitpress/11577.001.0001 Variability and consistency in early language learning: The Wordbank project. Variability and consistency in early language learning: The Wordbank...

    2021 doi

  16. [24]

    Frank, Mika Braginsky, Daniel Yurovsky, and Virginia A

    Micheal C. Frank, Mika Braginsky, Daniel Yurovsky, and Virginia A. Marchman. 2017. https://doi.org/10.1017/S0305000916000209 Wordbank: an open repository for developmental vocabulary data . Journal of Child Language, 44(3):677--694

    2017 doi

  17. [25]

    Martin Gerlach and Francesc Font-Clos. 2020. https://doi.org/10.3390/e22010126 A standardized project G utenberg corpus for statistical analysis of natural language and quantitative linguistics . Entropy, 22(1)

    2020 doi

  18. [26]

    Daniela Gerz, Ivan Vuli \'c , Felix Hill, Roi Reichart, and Anna Korhonen. 2016. https://doi.org/10.18653/v1/D16-1235 S im V erb-3500: A large-scale evaluation set of verb similarity . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, p...

    2016 doi

  19. [27]

    Richards, Steven F

    Jill Gilkerson, Jeffrey A. Richards, Steven F. Warren, Judith K. Montgomery, Charles R. Greenwood, D. Kimbrough Oller, John H. L. Hansen, and Terrance D. Paul. 2017. https://doi.org/10.1044/2016_AJSLP-15-0169 Mapping the early language environment using all-day recordings and ...

    2017 doi

  20. [28]

    Goodman, Philip S

    Judith C. Goodman, Philip S. Dale, and Ping Li. 2008. https://doi.org/10.1017/S0305000907008641 Does frequency count? parental input and the acquisition of vocabulary . Journal of Child Language, 35(3):515–531

    2008 doi

  21. [29]

    John Hale. 2001. https://aclanthology.org/N01-1021 A probabilistic E arley parser as a psycholinguistic model . In Second Meeting of the North A merican Chapter of the Association for Computational Linguistics

    2001

  22. [30]

    Zellig Harris. 1954. https://www.tandfonline.com/doi/abs/10.1080/00437956.1954.11659520 Distributional structure . Word, 10(2-3):146--162

    1954

  23. [31]

    Trevor Hastie and Robert Tibshirani. 1986. https://doi.org/10.1214/ss/1177013604 Generalized Additive Models . Statistical Science, 1(3):297 -- 310

    1986

  24. [32]

    Hermans, Gerasimos Spanakis, and Rico Möckel

    Joeri R. Hermans, Gerasimos Spanakis, and Rico Möckel. 2017. https://proceedings.mlr.press/v77/hermans17a.html Accumulated gradient normalization . In Proceedings of the Ninth Asian Conference on Machine Learning, volume 77 of Proceedings of Machine Learning Research, pages 43...

    2017

  25. [33]

    Felix Hill, Roi Reichart, and Anna Korhonen. 2015. https://doi.org/10.1162/COLI_a_00237 S im L ex-999: Evaluating semantic models with (genuine) similarity estimation . Computational Linguistics, 41(4):665--695

    2015 doi

  26. [34]

    Thomas Hofmann. 1999. https://arxiv.org/pdf/1301.6705 Probabilistic latent semantic analysis. In UAI, volume 99, pages 289--296

    1999 arXiv

  27. [35]

    Landauer and Susan T

    Thomas K. Landauer and Susan T. Dumais. 1997. https://psycnet.apa.org/record/1997-03612-001 A solution to P lato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological R eview , 104(2):211

    1997

  28. [36]

    Marvin Lavechin, Maureen De Seyssel, Lucas Gautheron, Emmanuel Dupoux, and Alejandrina Cristia. 2022. https://doi.org/10.1146/annurev-linguistics-031120-122120 Reverse engineering language acquisition with child-centered long-form recordings . Annual Review of Linguistics, 8(1...

    2022 doi

  29. [37]

    Angeliki Lazaridou, Anna Potapenko, and Olivier Tieleman. 2020. https://doi.org/10.18653/v1/2020.acl-main.685 Multi-agent Communication meets Natural Language : Synergies between Functional and Structural Language Learning . In Proceedings of the 58th Annual Meeting of the Ass...

    2020 doi

  30. [38]

    Alessandro Lenci. 2008. https://www.italian-journal-linguistics.com/app/uploads/2021/05/1_Lenci.pdf Distributional semantics in linguistic and cognitive research . Italian J ournal of L inguistics , 20(1):1--31

    2008

  31. [39]

    Roger Levy. 2008. https://doi.org/10.1016/j.cognition.2007.05.006 Expectation-based syntactic comprehension . Cognition, 106(3):1126--1177

    2008 doi

  32. [40]

    Tal Linzen. 2019. https://muse.jhu.edu/pub/24/article/719237 What can linguistics and deep learning contribute to each other? Language, 95(1):e99--e108

    2019

  33. [41]

    Pierre Lison and J \"o rg Tiedemann. 2016. https://aclanthology.org/L16-1147 O pen S ubtitles2016: E xtracting large parallel corpora from movie and TV subtitles . In Proceedings of the Tenth International Conference on Language Resources and Evaluation, pages 923--929

    2016

  34. [42]

    Ziqiao Ma, Zekun Wang, and Joyce Chai. 2024. http://arxiv.org/abs/2405.13828 Babysit A Language Model From Scratch : Interactive Language Learning by Trials and Demonstrations . arXiv preprint. ArXiv:2405.13828 [cs]

    2024 arXiv

  35. [43]

    Brian MacWhinney. 2000. https://psycnet.apa.org/record/2000-03631-000 The CHILDES Project: Tools for analyzing talk: The database, Vol . 2, 3rd ed. Lawrence Erlbaum Associates Publishers

    2000

  36. [44]

    Alec Marantz. 2001. Words. WCCFL XX Handout, USC

    2001

  37. [45]

    Rebecca Marvin and Tal Linzen. 2018. Targeted Syntactic Evaluation of Language Models . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pages 1192--1202

    2018

  38. [46]

    AI @ Meta. 2024. https://arxiv.org/abs/2407.21783 The Llama 3 herd of models . Preprint, arXiv:2407.21783

    2024 arXiv

  39. [47]

    Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeff Dean. 2013. https://proceedings.neurips.cc/paper_files/paper/2013/file/9aa42b31882ec039965f3c4923ce901b-Paper.pdf Distributed representations of words and phrases and their compositionality . In Advances in Neural...

    2013

  40. [48]

    Mitja Nikolaus and Abdellah Fourtassi. 2021 a . https://doi.org/10.18653/v1/2021.cmcl-1.24 Evaluating the acquisition of semantic knowledge from cross-situational learning in artificial neural networks . In Proceedings of the Workshop on Cognitive Modeling and Computational Li...

    2021 doi

  41. [49]

    Mitja Nikolaus and Abdellah Fourtassi. 2021 b . https://doi.org/10.18653/v1/2021.conll-1.31 Modeling the interaction between perception-based and production-based learning in children's early acquisition of semantic knowledge . In Proceedings of the 25th conference on computat...

    2021 doi

  42. [50]

    Mitja Nikolaus and Abdellah Fourtassi. 2023. https://doi.org/10.1016/j.newideapsych.2022.100985 Communicative Feedback in language acquisition . New Ideas in Psychology, 68:100985

    2023

  43. [51]

    Andreas Opedal, Eleanor Chodroff, Ryan Cotterell, and Ethan Wilcox. 2024. https://aclanthology.org/2024.emnlp-main.179 On the role of context in reading time prediction . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 3042--3058

    2024

  44. [52]

    OpenAI. 2022. https://openai.com/blog/chatgpt/ ChatGPT : Optimizing language models for dialogue

    2022

  45. [53]

    Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. https://doi.org/10.3115/v1/D14-1162 G lo V e: G lobal vectors for word representation . In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pages 1532--1543

    2014 doi

  46. [54]

    Frank, and Gary Lupyan

    Eva Portelance, Yuguang Duan, Michael C. Frank, and Gary Lupyan. 2023. https://doi.org/10.1111/cogs.13334 Predicting age of acquisition for children's E arly vocabulary in five languages using language model surprisal . Cognitive Science, 47(9)

    2023 doi

  47. [55]

    Frank, and Dan Jurafsky

    Eva Portelance, Michael C. Frank, and Dan Jurafsky. 2024. https://doi.org/10.1111/cogs.13448 Learning the meanings of function words from grounded language using a visual question answering model . Cognitive Science, 48(5)

    2024 doi

  48. [56]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. https://insightcivic.s3.us-east-1.amazonaws.com/language-models.pdf Language models are unsupervised multitask learners . OpenAI blog, 1(8):9

    2019

  49. [57]

    Enrico Santus, Anna Gladkova, Stefan Evert, and Alessandro Lenci. 2016. https://aclanthology.org/W16-5309/ The C og AL ex- V shared task on the corpus-based identification of semantic relations . In Proceedings of the 5th Workshop on Cognitive Aspects of the Lexicon ( C og AL ...

    2016

  50. [58]

    Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. 2000. https://doi.org/10.1162/089120100561737 Dialogue act modeling for automatic tagging and recognition of conve...

    doi

  51. [59]

    Emin Orhan, and Brenden M

    Wai Keen Vong, Wentao Wang, A. Emin Orhan, and Brenden M. Lake. 2024. https://doi.org/10.1126/science.adi1374 Grounded language acquisition through the eyes and ears of a single child . Science, 383(6682):504--511

    2024 doi

  52. [60]

    Alex Warstadt and Samuel R Bowman. 2022. https://arxiv.org/pdf/2208.07998 What artificial neural networks can tell us about human language acquisition . In Algebraic Structures in Natural Language , pages 17--60. CRC Press. Publisher: CRC Press

    2022 arXiv

  53. [61]

    Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman. 2020. https://doi.org/10.1162/tacl_a_00321 BL i MP : The benchmark of linguistic minimal pairs for E nglish . Transactions of the Association for Computational Linguistic...

    2020 doi

  54. [62]

    Ludwig Wittgenstein. 1953. https://static1.squarespace.com/static/54889e73e4b0a2c1f9891289/t/564b61a4e4b04eca59c4d232/1447780772744/Ludwig.Wittgenstein.-.Philosophical.Investigations.pdf Philosophical investigations . Wiley-Blackwell

    1953

  55. [63]

    Chengxu Zhuang, Evelina Fedorenko, and Jacob Andreas. 2024 a . https://doi.org/10.18653/v1/2024.findings-acl.15 Lexicon-level contrastive visual-grounding improves language modeling . In Findings of the Association for Computational Linguistics ACL 2024, pages 231--247

    2024 doi

  56. [64]

    Chengxu Zhuang, Evelina Fedorenko, and Jacob Andreas. 2024 b . https://doi.org/10.18653/v1/2024.naacl-long.71 Visual grounding helps learn word meanings in low-data regimes . In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computation...

    2024 doi

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.