Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Developmentally-plausible Working Memory Shapes a Critical Period for Language Acquisition

T0 review · 4 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Training a small GPT-2 with a working-memory budget that grows over time improves its grammar learning, outperforming both unconstrained and fixed-constraint models.

desk verdict A simple epoch-dependent ALiBi decay helps small LMs learn grammar, but the working-memory interpretation is stipulated rather than tested. read the letter →

arxiv 2502.04795 v3 pith:2RIJV7PK submitted 2025-02-07 cs.CL

Masato Mita , Ryo Yoshida , Yohei Oseki This is my paper · ORCID
classification cs.CL
keywords criticalperiodhypothesisworkingmemoryLess-is-MoreALiBilanguageacquisitionchild-directedspeechtargetedsyntacticevaluationsmallmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the human critical period for language acquisition can be reproduced in a small neural language model by letting its working-memory constraint grow during training. The authors propose DYNAMIC LIMIT-EXP, which starts a GPT-2 model with a strong recency bias (an ALiBi attention slope near 1) and exponentially relaxes that bias as epochs progress. On the Zorro grammaticality benchmark, this schedule outperforms both an unconstrained model and a model with a fixed memory limit, on both child-directed and adult text. The authors read this as indirect evidence for the Less-is-More hypothesis: limited early working memory helps extract basic syntactic rules before capacity expands.

What carries the argument

ALiBi (Attention with Linear Biases) recency bias is repurposed as a working-memory parameter. In this paper the slope $m$ is not fixed per head as in the original method; instead it is decayed exponentially with epoch, $m_t = m_0 r^t$, so the model starts attending almost only to nearby tokens and gradually widens its context window. The paper treats $w_t = 1 - m_t$ as the model's working-memory capacity, so the schedule simulates childhood memory growth and the critical period.

What would settle it

Train DYNAMIC LIMIT-EXP with the slope sequence permuted across epochs so the same values appear in a different order; if overall accuracy stays as high as in the ordered schedule, the developmental ordering is not the cause. Alternatively, compare against a schedule that decreases the context window instead of the ALiBi slope; if behavior is identical, the recency-bias interpretation is not needed.

Watch

Extended reading notes

Core claim

The central claim is that a developmentally plausible training schedule, in which working memory is initially constrained and then exponentially expanded, improves grammatical generalization in small language models relative to static or absent constraints. The paper defines working memory capacity as $w_t = 1 - m_t$, with $m_t$ the ALiBi attention slope decaying as $m_t = m_0 r^t$. Trained on AO-CHILDES or Wikipedia and evaluated on Zorro, the dynamic-exponential model reaches the highest overall accuracy, with gains concentrated in items requiring non-local dependencies such as case, filler-gap, and argument structure; reversing the schedule to shrink memory removes the benefit. The authors conclude that the gradual growth of working memory is the operative inductive bias, not the mere presence of a constraint.

Load-bearing premise

The load-bearing premise is that the ALiBi recency-bias slope is a valid operationalization of human working-memory capacity; if that mapping is wrong, the cognitive conclusions do not follow, even though the training-schedule improvement might remain as a generic curriculum effect.

Editorial extensions

If this is right

  • A simple epoch-dependent schedule can serve as a data-efficiency inductive bias for pretraining small language models.
  • The effect holds on both child-directed and adult corpora, implying the benefit comes from the learning algorithm rather than the input type.
  • Dynamic growth outperforms a static constraint, indicating that the gradual introduction of the constraint is crucial.
  • Reversing the constraint direction reduces accuracy, supporting the Less-is-More interpretation of the schedule.
  • The model's embedding representations remain more separated and continue shifting across epochs under the dynamic limit, suggesting ongoing structural refinement rather than early stagnation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The training-schedule improvement may be a generic curriculum effect that would also appear with other gradual task-difficulty schedules; the cognitive interpretation depends on validating $w_t = 1 - m_t$ as a measure of human working memory.
  • The method's gains on intermediate and long sentences suggest it could interact with sequence-length curricula, and combining both might yield further data-efficiency gains.
  • If future work ties ALiBi slopes to human reading-time patterns, the same mechanism could provide a single parameter linking memory development, attention, and language learnability.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes DYNAMIC LIMIT-EXP, a training schedule for small GPT-2-style language models in which the ALiBi attention-bias slope m_t decays exponentially over epochs (m_t = m_0 r^t) and the model's "working memory capacity" is defined as w_t = 1 - m_t. The authors train on AO-CHILDES and Wikipedia, evaluate on the Zorro minimal-pair benchmark, and compare against no limit, static ALiBi, linear decay, and a reversed exponential schedule. They report consistent overall accuracy gains for the exponential schedule, argue that the effect supports the Less-is-More hypothesis, and interpret the results as computational evidence for the role of working-memory development in the critical period for language acquisition.

Significance. If the empirical result holds, the paper makes a useful contribution to sample-efficient LM pretraining and to computational modeling of developmental constraints. The reversed-schedule control is a sensible test of direction; the two-corpus design shows the effect is not specific to child-directed speech; and the method is simple, reproducible (code is released), and yields a concrete, falsifiable prediction about annealing a recency bias. The main limitation is interpretive: the working-memory identification is stipulated rather than validated, so the current experiments support a generic recency-bias annealing effect more strongly than they support a specifically working-memory mechanism of the critical period. The empirical claims are still interesting, but the cognitive conclusion in the abstract and conclusion overreaches relative to the evidence presented.

major comments (4)
  1. [§3.1–§3.2, Eq. (3), §5.1] The central cognitive identification w_t := 1 − m_t is stipulated rather than validated. ALiBi's slope m is a distance-based additive penalty on attention scores, not a capacity limit over a fixed number of memory items, and no independent behavioral or neural evidence is offered that 1 − m corresponds to any measured working-memory construct. Likewise, the exponential growth curve in §3.1 is asserted, and the experiments fix m0 = 1 and r = 0.6 by hand rather than fitting them to the cited child working-memory norms. Consequently, the reported gains are equally consistent with any monotone curriculum that gradually broadens the effective context, and the reversed-schedule control in Table 3 only excludes the opposite direction. To make the CPH claim load-bearing, I would ask for a control such as context-length or attention-window annealing that keeps ALiBi fixed and monotone, a sensitivity analysis over r, and, ideally, a fit of r to the child developmental data; without these, the paper supports a generic recency-bias annealing effect but not specifically a working-memory mechanism.
  2. [§4.2, Tables 1–3] All headline comparisons rest on pooled z-tests on minimal-pair items, but the text reports only means over three seeds and significance stars. Because the model is trained with three seeds, the appropriate test of stability is a paired or mixed-effects comparison across seeds; pooled item-level z-tests also treat the 2,000 template-derived pairs in each Zorro category as independent, which overstates precision. Please report per-seed category scores, mean ± SD, and a seed-level significance test, or otherwise justify the pooled test. This is needed to support the 'consistently outperforms' claim in §4.3.
  3. [§4.2, Appendix A, Figure 2] There is a reproducibility inconsistency: the main text sets 'the number of training epochs to 10' and Figure 2 plots 10 epochs, while Appendix A, Table 6 reports 'Total Epochs 20' with early stopping on perplexity (tolerance 1 epoch). Please state which epoch count produced the final models, whether early stopping was applied before the scheduled ALiBi trajectory completed, and how Figure 3's Epochs 1, 5, and 10 relate to that schedule. In addition, Eq. (2) defines a single scalar m_t, but ALiBi assigns per-head geometric slopes; clarify whether all heads are multiplied by the same decay factor and which heads the StaticLimit head=0..3 curves in Figure 2 refer to.
  4. [Tables 1–2, §5.1] The overall gains are driven by a subset of categories, with significant losses on others (e.g., ELLIPSIS in both tables; BINDING and LOCAL.ATR in Table 1; CASE and ELLIPSIS in Table 2, depending on the comparison). The paper discusses ELLIPSIS but does not quantify the trade-off. I would like an aggregate analysis that separates categories with long-range dependency requirements from those without, and a statement of whether the method should be expected to help or hurt each category given the initial short-range bias. Without this, the method appears to be a category-specific inductive bias rather than a general grammar-learning improvement.
minor comments (6)
  1. [§3.1] The model is written y = b − ax; if a is the exponential base, this should be y = b − a^x, and the relation between this curve and m_t = m0 r^t in Eq. (2) should be stated explicitly.
  2. [Figure 1] Figure 1 is presented as the 'developmental trajectory of human working memory' but contains no data points or citation; add the empirical values underlying the schematic or label it as an idealized curve.
  3. [§5.4, Table 5] It is not described how the four length-filtered Wikipedia subsets are used; specify whether each model is trained from scratch on each subset and with which hyperparameters.
  4. [Eq. (1)] The bias matrix B is written as a single row depending on i; make the indexing explicit (e.g., B_{ij} = −(i − j)) to avoid confusion between the query index and the row definition.
  5. [Tables 2, 3, and 5] Several tables have spacing or formatting typos, e.g., 'DYNAMIC LIMIT-E XP' in Tables 2 and 3, 'N OLIMIT' in Table 5, and the overlapping header row in Table 3; these should be corrected.
  6. [Appendix D, Tables 4 and 7] Entropy and mean distance are computed on t-SNE coordinates; t-SNE is stochastic and does not preserve global distances, so the numbers in Tables 4 and 7 should be recomputed in the raw embedding space or with a distance-preserving projection, with a note on variance across t-SNE runs.

Circularity Check

1 steps flagged · score 4.0 of 10

Zorro gains are empirical, but the 'working memory' explanation is a stipulated relabeling of the ALiBi slope schedule.

  1. self definitional [Section 3.2, Eq. (3) and surrounding text]
    "In this study, the model’s working memory capacity wt is formulated as follows: wt := 1− mt. (3) This definition links the dynamically decaying slope mt to the model’s working memory capacity wt: as mt decreases exponentially, wt grows, enabling broader contextual retention over time."

    Working-memory capacity is not independently measured; it is defined as one minus the ALiBi slope. The central cognitive premise that DYNAMIC LIMIT-EXP 'initially constrains working memory and gradually relaxes this constraint' is therefore equivalent by construction to 'the ALiBi slope starts high and decays exponentially.' The grammar-learning gains on Zorro are empirical, but attributing them to developmental working-memory growth rather than to a generic context-broadening curriculum rests entirely on the stipulated equation; no behavioral or neural validation of the 1 - m mapping is supplied. The cognitive conclusion reduces to the authors' own operationalization.

full rationale

The engineering claim is self-contained and externally benchmarked: DYNAMIC LIMIT-EXP is compared against NOLIMIT, STATICLIMIT, and a linear schedule on the held-out Zorro benchmark, and the reversed-trajectory control (Table 3) excludes the opposite direction. No parameter is fitted to Zorro and then reported as a prediction, and the self-citations (Haga et al. 2024; Someya and Oseki 2023) are methodological or bibliographic, not load-bearing. The remaining issue is the construct mapping in Eq. (3): once 'working memory capacity' is defined as 1 - m_t, the exponential growth of working memory is true by definition and the cognitive interpretation of the experimental gains is a relabeling of an ALiBi annealing effect. This makes the cognitive conclusion partially circular, even though the underlying scheduling result stands independently. The Limitations section acknowledges scale and language coverage but does not address this foundational mapping.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The ledger captures the hand-set schedule parameters and the unvalidated mapping from ALiBi slope to working memory. The core method is empirical, so most of the burden is on the cognitive interpretation rather than on derivation.

free parameters (4)
  • Initial ALiBi slope m0 = 1.0
    Set at the start of training to impose a strong recency constraint; Eq. (2). No sensitivity analysis is reported.
  • Exponential decay rate r = 0.6
    Chosen by hand to approximate the human working-memory growth curve over 10 epochs (Sections 3.2 and 4.2). The main result depends on this value; no sweep is reported.
  • Reversed-schedule parameters for DynamicLimit-Exp(down) = m0=0.01, r=1.668
    Chosen to make the decreasing-memory curve nearly symmetric to the increasing one (Section 5.1), not from cognitive data. The resulting m exceeds 1.0 at later epochs.
  • DynamicLimit-Linear endpoints = m=1.0 to 0.0 over 10 epochs
    Matched to DynamicLimit-Exp for fair comparison (Section 4.2), but linearity itself is an arbitrary control schedule.
assumptions (4)
  • domain assumption Human working memory grows along an exponential trajectory y = b - a^x during development.
    Section 3.1 motivates the schedule, citing qualitative developmental stages. No quantitative fit to developmental data is provided.
  • ad hoc to paper The ALiBi attention-bias slope m can stand for working memory capacity, with capacity w_t = 1 - m_t.
    Eq. (3) defines this mapping without independent validation; the cognitive interpretation of the experiments rests on it.
  • domain assumption Accuracy on the Zorro benchmark by a 4-layer GPT-2 is a meaningful proxy for human L1 grammatical acquisition.
    The evaluation section adopts Zorro for this purpose; the cognitive conclusion depends on this proxy.
  • ad hoc to paper Training epochs correspond to developmental time during the critical period.
    The schedule runs over 10 epochs only; no mapping from epochs to ages (2-7, 8-14, 15+) is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Developmentally-plausible Working Memory Shapes a Critical Period for Language Acquisition." pith.science (2026). https://pith.science/paper/2RIJV7PK

@misc{pith2026250204795,
  author       = {Pith},
  title        = {Pith review of: Developmentally-plausible Working Memory Shapes a Critical Period for Language Acquisition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2RIJV7PK}},
  note         = {Machine review of arXiv:2502.04795}
}
read the original abstract

Large language models possess general linguistic abilities but acquire language less efficiently than humans. This study proposes a method for integrating the developmental characteristics of working memory during the critical period, a stage when human language acquisition is particularly efficient, into the training process of language models. The proposed method introduces a mechanism that initially constrains working memory during the early stages of training and gradually relaxes this constraint in an exponential manner as learning progresses. Targeted syntactic evaluation shows that the proposed method outperforms conventional methods without memory constraints or with static memory constraints. These findings not only provide new directions for designing data-efficient language models but also offer indirect evidence supporting the role of the developmental characteristics of working memory as the underlying mechanism of the critical period in language acquisition.

Figures

Figures reproduced from arXiv: 2502.04795 by the authors.

Figure 1
Figure 1. Developmental trajectory of human working [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Trajectory of working memory capacity for [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Accuracy trajectories over training epochs (1, 5, 10) for six grammatical categories that showed significant [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Embedded space at each learning stage for N [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Word distribution of the AO-CHILDES and Wikipedia datasets used in the experiment [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Embedded space at each learning stage for N [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Do Self-Supervised Speech Models Exhibit the Critical Period Effects in Language Acquisition?

    cs.CL 2025-08 conditional novelty 6.0 of 10

    Self-supervised speech models trained on child-directed speech do not reproduce human critical-period effects in phonological acquisition; later L2 exposure even improves L2 phone discrimination.

Reference graph

Works this paper leans on

56 extracted references · 18 canonical work pages · cited by 1 Pith paper

  1. [1]

    Anderson and Robert Milson

    John R. Anderson and Robert Milson. 1989. https://doi.org/10.1037/0033-295x.96.4.703 Human memory: An adaptive perspective . Psychological Review, 96(4):703--719

    work page doi:10.1037/0033-295x.96.4.703 1989

  2. [2]

    Noam Chomsky. 1965. http://www.amazon.com/Aspects-Theory-Syntax-Noam-Chomsky/dp/0262530074 Aspects of the Theory of Syntax . The MIT Press, Cambridge

    arXiv 1965

  3. [3]

    Christiansen and Nick Chater

    Morten H. Christiansen and Nick Chater. 2016. https://doi.org/10.1017/S0140525X1500031X The Now-or-Never bottleneck: A fundamental constraint on language . Behavioral and Brain Sciences, 39:e62

    doi:10.1017/s0140525x1500031x 2016

  4. [4]

    Alexander Clark and Shalom Lappin. 2011. Linguistic Nativism and the Poverty of the Stimulus. Wiley-Blackwell

    work page 2011

  5. [5]

    Christian Clark, Byung-Doh Oh, and William Schuler. 2025. https://aclanthology.org/2025.coling-main.517/ Linear recency bias during training improves transformers' fit to reading times . In Proceedings of the 31st International Conference on Computational Linguistics, pages 7735--7747, Abu Dhabi, UAE. Association for Computational Linguistics

    2025

  6. [6]

    Ionut Constantinescu, Tiago Pimentel, Ryan Cotterell, and Alex Warstadt. 2025. https://doi.org/10.1162/tacl_a_00725 Investigating critical period effects in language acquisition through neural language models . Transactions of the Association for Computational Linguistics, 13:96--120

    doi:10.1162/tacl_a_00725 2025

  7. [7]

    Elliott, Igor Ponomarev, and John Scott Saults

    Nelson Cowan, Lara Nugent, Emily M. Elliott, Igor Ponomarev, and John Scott Saults. 1999. https://api.semanticscholar.org/CorpusID:18553089 The role of attention in the development of short-term memory: age differences in the verbal span of apprehension. Child development, 70 5:1082--97

    work page 1999

  8. [8]

    wild Child

    S. Curtiss. 1977. https://books.google.co.jp/books?id=PDRsAAAAIAAJ Genie: A Psycholinguistic Study of a Modern-day "wild Child" . Mathematics in Science and Engineering. Academic Press

    work page 1977

Show all 56 references
  1. [9]

    Richard Diehl Martinez, Z \'e bulon Goriely, Andrew Caines, Paula Buttery, and Lisa Beinborn. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.344 Mitigating frequency bias and anisotropy in language model pre-training with syntactic smoothing . In Proceedings of the 2024 Con...

    2024 doi

  2. [10]

    Emmanuel Dupoux. 2018. https://doi.org/10.1016/j.cognition.2017.11.008 Cognitive science in the era of artificial intelligence: A roadmap for reverse-engineering the infant language-learner . Cognition, 173:43--59

    2018 doi

  3. [11]

    Ellis and Matthew A

    Andrew W. Ellis and Matthew A. Lambon Ralph. 2000. https://pubmed.ncbi.nlm.nih.gov/11009247/ A ge of acquisition effects in adult lexical processing reflect loss of plasticity in maturing systems: Insights from connectionist networks. Journal of Experimental Psychology: Learni...

    2000

  4. [12]

    Elman, Elizabeth A

    Jeffrey L. Elman, Elizabeth A. Bates, Mark H. Johnson, Annette Karmiloff-Smith, Domenico Parisi, and Kim Plunkett. 1996. Rethinking Innateness: A Connectionist Perspective on Development. MIT Press

    1996

  5. [13]

    Piantadosi, and Edward A

    Evelina Fedorenko, Steven T. Piantadosi, and Edward A. F. Gibson. 2024. https://doi.org/10.1038/s41586-024-07522-w Language is primarily a tool for communication rather than thought . Nature, 630:575--586

    2024 doi

  6. [14]

    Feng, Noah Goodman, and Michael Frank

    Steven Y. Feng, Noah Goodman, and Michael Frank. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.1231 Is child-directed speech effective training data for language models? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 22055-...

    2024 doi

  7. [15]

    critical period

    Victoria Fromkin, Stephen Krashen, Susan Curtiss, David Rigler, and Marilyn Rigler. 1974. https://doi.org/10.1016/0093-934X(74)90027-3 The development of language in genie: a case of language acquisition beyond the “critical period” . Brain and Language, 1(1):81--107

    1974 doi

  8. [16]

    Richard Futrell and Kyle Mahowald. 2025. https://arxiv.org/abs/2501.17047 How linguistics learned to stop worrying and love the language models . Preprint, arXiv:2501.17047

    2025

  9. [17]

    S. E. Gathercole, S. J. Pickering, B. Ambridge, and H. Wearing. 2004. https://doi.org/10.1037/0012-1649.40.2.177 The structure of working memory from 4 to 15 years of age . Developmental psychology, 40(2):177--190. Gathercole, Susan E Pickering, Susan J Ambridge, Benjamin Wear...

    2004 doi

  10. [18]

    Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen, and Marco Baroni. 2018. https://doi.org/10.18653/v1/N18-1108 Colorless green recurrent networks dream hierarchically . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for C...

    2018 doi

  11. [19]

    Akari Haga, Saku Sugawara, Akiyo Fukatsu, Miyu Oba, Hiroki Ouchi, Taro Watanabe, and Yohei Oseki. 2024. https://doi.org/10.18653/v1/2024.findings-acl.865 Modeling overregularization in children with small language models . In Findings of the Association for Computational Lingu...

    2024 doi

  12. [20]

    Hartshorne, Joshua B

    Joshua K. Hartshorne, Joshua B. Tenenbaum, and Steven Pinker. 2018. https://www.sciencedirect.com/science/article/pii/S0010027718300994 A critical period for second language acquisition: Evidence from 2/3 million English speakers . Cognition, 177:263--277

    2018

  13. [21]

    Huebner, Elior Sulem, Fisher Cynthia, and Dan Roth

    Philip A. Huebner, Elior Sulem, Fisher Cynthia, and Dan Roth. 2021. https://doi.org/10.18653/v1/2021.conll-1.49 B aby BERT a: Learning more grammar with small-scale child-directed language . In Proceedings of the 25th Conference on Computational Natural Language Learning, page...

    2021 doi

  14. [22]

    Huebner and Jon A

    Philip A. Huebner and Jon A. Willits. 2021. https://doi.org/10.1016/bs.plm.2021.08.002 Using lexical context to discover the noun category: Younger children have it easier , pages 279--331. Psychology of Learning and Motivation - Advances in Research and Theory. Academic Press Inc

    2021 doi

  15. [23]

    Florian Jaeger and Harry Tily

    T. Florian Jaeger and Harry Tily. 2011. https://doi.org/10.1002/wcs.126 On language ‘utility’: processing complexity and communicative efficiency . WIREs Cognitive Science, 2(3):323--335

    2011 doi

  16. [24]

    Jacqueline S Johnson and Elissa L Newport. 1989. https://doi.org/10.1016/0010-0285(89)90003-0 Critical period effects in second language learning: The influence of maturational state on the acquisition of english as a second language . Cognitive Psychology, 21(1):60--99

    1989 doi

  17. [25]

    Charles Kemp, Yang Xu, and Terry Regier. 2018. https://doi.org/10.1146/annurev-linguistics-011817-045406 Semantic typology and efficient communication . Annual Review of Linguistics, 4(1):109--128

    2018 doi

  18. [26]

    Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. 2017. https://doi.org/10.1073/pnas.1...

    2017 doi

  19. [27]

    Lenneberg

    E.H. Lenneberg. 1967. https://books.google.co.jp/books?id=7UZiAAAAMAAJ Biological Foundations of Language . Wiley

    1967

  20. [28]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. https://arxiv.org/abs/1907.11692 Roberta: A robustly optimized bert pretraining approach . Preprint, arXiv:1907.11692

    2019 arXiv

  21. [29]

    Garver, Trinity A

    Beatriz Luna, Krista E. Garver, Trinity A. Urban, Nicole A. Lazar, and John A. Sweeney. 2004. https://doi.org/10.1111/j.1467-8624.2004.00745.x Maturation of cognitive processes from late childhood to adulthood . Child Development, 75(5):1357--1372

    2004

  22. [30]

    Brian Macwhinney. 2000. https://doi.org/10.1177/026565909200800211 The childes project: tools for analyzing talk . Child Language Teaching and Therapy, 8

    2000 doi

  23. [31]

    David Marr. 1982. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. Henry Holt and Co., Inc., USA

    1982

  24. [32]

    Mayberry and Susan D

    Rachel I. Mayberry and Susan D. Fischer. 1989. https://doi.org/10.3758/BF03202635 L ooking through phonological shape to lexical meaning: The bottleneck of non-native sign language processing . Memory & Cognition, 17(6):740--754

    1989 doi

  25. [33]

    Thomas McCoy, Robert Frank, and Tal Linzen

    R. Thomas McCoy, Robert Frank, and Tal Linzen. 2020. https://doi.org/10.1162/tacl_a_00304 Does syntax need to grow on trees? sources of hierarchical inductive bias in sequence-to-sequence networks . Transactions of the Association for Computational Linguistics, 8:125--140

    2020 doi

  26. [34]

    Elissa L. Newport. 1990. https://doi.org/10.1016/0364-0213(90)90024-Q Maturational constraints on language learning . Cognitive Science, 14(1)

    1990 doi

  27. [35]

    OpenAI. 2023. https://arxiv.org/abs/2303.08774 Gpt-4 technical report . Preprint, arXiv:2303.08774

    2023 arXiv

  28. [36]

    Francesca Padovani, Jaap Jumelet, Yevgen Matusevych, and Arianna Bisazza. 2025. https://arxiv.org/abs/2505.23689 Child-directed language does not consistently boost syntax learning in language models . Preprint, arXiv:2505.23689

    2025

  29. [37]

    Patkowski

    Mark S. Patkowski. 1980. https://doi.org/10.1111/j.1467-1770.1980.tb00328.x The sensitive period for the acquisition of syntax in a second language . Language Learning, 30(2):449--468

    1980

  30. [38]

    Wilder Penfield. 1965. https://doi.org/10.1093/brain/88.4.787 C onditioning the uncommitted cortex for language learning . Brain, 88(4):787--798

    1965 doi

  31. [39]

    Steven Pinker. 1994. The Language Instinct: How the Mind Creates Language. William Morrow and Company

    1994

  32. [40]

    Ofir Press, Noah Smith, and Mike Lewis. 2022. https://openreview.net/forum?id=R8sQPpGCv0 Train short, test long: Attention with linear biases enables input length extrapolation . In International Conference on Learning Representations

    2022

  33. [41]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf Language models are unsupervised multitask learners . OpenAI. Accessed: 2024-11-15

    2019

  34. [42]

    Seidenberg and Jason D

    Mark S. Seidenberg and Jason D. Zevin. 2006. https://academic.oup.com/book/54488/chapter/422571818 C onnectionist models in developmental cognitive neuroscience: Critical periods and the paradox of success . In Yuko Munakata and Mark H Johnson, editors, Processes of Change in ...

    2006

  35. [43]

    Taiga Someya and Yohei Oseki. 2023. https://doi.org/10.18653/v1/2023.findings-eacl.117 JBL i MP : J apanese benchmark of linguistic minimal pairs . In Findings of the Association for Computational Linguistics: EACL 2023, pages 1581--1594, Dubrovnik, Croatia. Association for Co...

    2023 doi

  36. [44]

    Sowell, Doris A

    Elizabeth R. Sowell, Doris A. Trauner, Anthony Collins Gamst, and Terry L. Jernigan. 2002. https://api.semanticscholar.org/CorpusID:19241957 Development of cortical and subcortical brain structures in childhood and adolescence: a structural mri study . Developmental Medicine &...

    2002

  37. [45]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023 a . https://arxiv.org/abs/2302.13971 Lla...

    2023 arXiv

  38. [46]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 arXiv

  39. [47]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf Attention is all you need . In Advances in Ne...

    2017

  40. [48]

    Alex Warstadt and Samuel R. Bowman. 2022. What artificial neural networks can tell us about human language acquisition. In Algebraic Structures in Natural Language, pages 17--60. CRC Press

    2022

  41. [49]

    Alex Warstadt, Aaron Mueller, Leshem Choshen, Ethan Wilcox, Chengxu Zhuang, Juan Ciro, Rafael Mosquera, Bhargavi Paranjabe, Adina Williams, Tal Linzen, and Ryan Cotterell. 2023. https://doi.org/10.18653/v1/2023.conll-babylm.1 Findings of the B aby LM challenge: Sample-efficien...

    2023 doi

  42. [50]

    Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman. 2020. https://doi.org/10.1162/tacl_a_00321 BL i MP : The benchmark of linguistic minimal pairs for E nglish . Transactions of the Association for Computational Linguistic...

    2020 doi

  43. [51]

    Ethan Gotlieb Wilcox, Richard Futrell, and Roger Levy. 2024. https://doi.org/10.1162/ling_a_00491 Using computational models to test syntactic learnability . Linguistic Inquiry, 55(4):805--848

    2024 doi

  44. [52]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020

  45. [53]

    Beilei Xiang, Changbing Yang, Yu Li, Alex Warstadt, and Katharina Kann. 2021. https://doi.org/10.18653/v1/2021.eacl-main.242 CL i MP : A benchmark for C hinese language model evaluation . In Proceedings of the 16th Conference of the European Chapter of the Association for Comp...

    2021 doi

  46. [54]

    George K. Zipf. 1949. Human behavior and the principle of least effort. Addison-Wesley

    1949

  47. [55]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  48. [56]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.