Pith. sign in

REVIEW 2 major objections 4 minor 35 references

Quantity doesn't buy quality syntax with neural language models

T0 review · 2 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that scaling LSTM language models in size and data yields diminishing returns for syntactic agreement, and that reaching human-level syntax may require impractically large corpora unless models get structural supervision…

desk verdict Systematic LSTM scaling study with a credible saturation result and an honest extrapolation; the Transformer comparison is borrowed and under-matched, so the paper's reach slightly exceeds its grasp. read the letter →

arxiv 1909.00111 v1 pith:4GVW4XBL submitted 2019-08-31 cs.CL

Marten van Schijndel , Aaron Mueller , Tal Linzen This is my paper · ORCID
classification cs.CL
keywords neurallanguagemodelssyntacticagreementLSTMscalingdataefficiencytransformertargetedevaluationBayesfactors
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether simply making neural language models bigger and training them on more text can fix their well-documented failures on grammatical agreement. It trains 125 LSTM models spanning five hidden-layer sizes and five corpus sizes, tests them on a targeted agreement challenge set, and finds performance saturates beyond 400 hidden units and improves inconsistently as data grows from 2 million to 80 million words. Extrapolating the best observed improvement rate, every construction still below human level would require more than 10 billion tokens, and most would need trillions for near-perfect accuracy. GPT and BERT, Transformer models trained on billions of words, do not consistently beat these LSTMs. The conclusion is that reliable, data-efficient syntax learning will likely require external supervision or a stronger inductive bias than current architectures provide.

What carries the argument

The machinery is a controlled scaling grid: 125 two-layer LSTM language models, crossing hidden sizes of 100 to 1600 units with training corpora of 2 million to 80 million tokens drawn from WikiText-103, all evaluated on the same targeted syntactic agreement challenge set. Bayes factors quantify the evidence that a given increase in size or data improves accuracy, and the data-requirement estimates extrapolate the 20-million-to-40-million-token error-reduction rate under a constant-error-reduction assumption. This design lets the authors separate the effects of capacity and data on a specific structural ability rather than on average perplexity.

What would settle it

Score the LSTM models on the verb-rank metric used for GPT and BERT on the identical high-frequency-verb stimuli; if the LSTM advantage vanishes, the paper's cross-architecture conclusion is not supported.

Watch

Extended reading notes

Core claim

The paper's central claim is that quantity—in parameters and tokens—does not buy quality in syntactic generalization. Across 125 LSTM language models, agreement accuracy plateaus once hidden layers reach 400 units, and each doubling of training data beyond that point gives uneven, often statistically undetectable gains. Under a conservative assumption that doubling data keeps reducing errors at the best observed rate, human-level accuracy on the hardest remaining constructions would need more than 10 billion tokens, and 99.99 percent accuracy would need trillions—an unrealistic amount for simple agreement phenomena. The paper further reports that GPT and BERT, trained on one to three billion words, perform more poorly than the trained LSTMs on several of the ten agreement conditions, supporting the conclusion that the bottleneck is not scale but architecture or supervision.

Load-bearing premise

The paper's claim that Transformer models fare worse than LSTMs depends on an external evaluation that scored GPT and BERT by the rank of the two target verbs given the prefix rather than full-sentence log-probability and did not restrict stimuli to high-frequency verbs; if that procedural mismatch changes which model wins, the cross-architecture conclusion collapses.

Editorial extensions

If this is right

  • Agreement accuracy will not improve meaningfully by training LSTM language models beyond roughly 400 hidden units.
  • Doubling training data beyond tens of millions of tokens yields inconsistent gains; several constructions show no measurable improvement.
  • Matching human-level agreement in the remaining hard constructions would require corpora on the order of 10 billion tokens or more, and near-perfect accuracy would require trillions.
  • Transformer models trained on orders of magnitude more data fail to consistently outperform LSTMs, so scale alone is not a fix.
  • Remaining deficits call for external syntactic supervision or architectures with a stronger inductive bias.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the saturation pattern holds beyond this grid, then scaling curves for syntactic structure may be governed by an irreducible error floor, so scaling laws fitted to aggregate perplexity could mask construction-specific ceilings.
  • A testable extension would be to train on the same grid but with a syntax-augmented objective, such as auxiliary agreement prediction; the paper's account predicts such supervision should shift the saturation point upward at far lower data cost.
  • The asymmetry between short and long coordinated verb phrases suggests challenge sets should systematically vary constituent length, since models that generalize to longer structures need not generalize to shorter ones.
  • The cross-architecture comparison could be made exact by re-scoring LSTM outputs under the verb-rank metric used for GPT and BERT; the paper flags the procedural mismatch but does not itself perform the matched comparison.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper trains 125 two-layer LSTM language models, varying hidden size (100, 200, 400, 800, 1600 units) and training corpus size (2M, 10M, 20M, 40M, 80M tokens from WikiText-103), and evaluates them on the Marvin-Linzen agreement challenge set with high-frequency verb pairs. It reports that agreement accuracy improves with model size up to about 400 hidden units and then saturates; that increasing training data yields inconsistent and diminishing improvements; and that extrapolating from observed error reductions suggests that reaching human-level agreement accuracy would require on the order of 10 billion tokens or more. Using results from Wolf (2019), it also compares GPT and BERT, finding that they perform worse than the LSTM models in seven of ten subject-verb agreement conditions, and concludes that reliable and data-efficient syntactic learning likely requires external supervision or a stronger inductive bias than that provided by RNNs and Transformers.

Significance. If the LSTM scaling results hold, the paper is a valuable negative result: it provides a systematic 125-model grid with five random seeds, Bayes-factor analyses, and evaluation on an independent, externally published challenge set, and the saturation pattern and inconsistent data improvements are credible. The data-requirement extrapolation is transparently stated and the qualitative conclusion that the required data would be unrealistic is robust across a wide range of assumed improvement rates. However, the Transformer comparison is not established by the paper's own experiments because it relies on a different scoring protocol and stimulus set, and this weakens the title-, abstract-, and conclusion-level claims about Transformers. The LSTM-specific findings remain a solid contribution.

major comments (2)
  1. [Section 2, footnote 4; Section 4, 'Transformers'] The headline comparative claim that GPT and BERT 'perform even more poorly than our LSTMs in some constructions' (Abstract) is not established by the experiments in this paper. The GPT/BERT numbers are taken from Wolf (2019), who scored models by the rank of the two target verbs given the prefix, whereas the LSTM models here are scored by total sentence log-probability, and Wolf's stimuli were not restricted to the high-frequency verbs used in Section 3. These protocol differences can change accuracy: the correct verb can rank higher conditionally while the total sentence probability favors the ungrammatical continuation, and verb frequency interacts with training-data size. The seven-of-ten underperformance reported in Figures 3-5 therefore rests on an unmatched comparison. The authors should either run GPT and BERT under the LSTM scoring protocol on the same high-frequency-verb stimuli, or explicitly restrict the abstract and conclusion to LSTM models and reframe the title accordingly.
  2. [Section 5, Tables 3-4] The extrapolated data requirements are presented with point estimates such as 'over 10 billion tokens,' but Tables 3 and 4 show that the estimates vary by many orders of magnitude depending on which data-doubling rate is assumed, and Table 2 reports Bayes factors below 1 for several individual doublings, indicating no measurable improvement. The authors should state explicitly that these are idealized lower-bound projections under a constant-error-reduction assumption, not predictions, and should note that exact token counts (for example, the 317-token estimate in Table 4) are not meaningful. The qualitative conclusion that the data requirements would be unrealistic may survive this caveat, but the current wording overstates the precision of the estimates.
minor comments (4)
  1. [Section 2, footnote 4] The methodological limitation that the Transformer comparison uses a different scoring protocol and different stimuli should be promoted from a footnote to the main text, since it qualifies a result advertised in the abstract.
  2. [Section 5] Please clarify that using the largest observed error reduction (20M to 40M tokens) makes the resulting data requirements a lower bound rather than a central estimate; the word 'conservative' is currently ambiguous.
  3. [Figure 2] The caption of Figure 2 should explain more explicitly that the lines are projections and the bars are actual training-token counts; the logarithmic scale with values up to 10^96 makes it difficult to read individual estimates.
  4. [Section 2] The paper does not mention whether the trained models or evaluation code will be released; given the 125-model grid and the importance of reproducing the scaling curves, a reproducibility statement would be helpful.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the LSTM scaling results are self-contained experiments against an external benchmark, and the Transformer comparison, while methodologically imperfect, is not circular.

full rationale

The paper's central scaling results (saturation of agreement accuracy beyond 400 hidden units and inconsistent, diminishing improvements with corpus size) come from training 125 LSTM language models under a systematic grid of layer sizes and corpus sizes and evaluating them on the externally published Marvin and Linzen (2018) challenge set. Nothing in that evaluation is fitted to the paper's conclusions: the benchmark does not encode the scaling claims, the models are not selected or tuned to produce saturation, and the human performance comparisons are taken from a prior publication. The Section 5 extrapolation of required training data is explicitly an assumption (constant error reduction per doubling) applied to an external human-accuracy target; it is an estimate, not a restatement of the fitted results. The only notable weakness is the comparison to GPT and BERT via Wolf (2019), which used a different scoring metric (rank of the two target verbs versus total sentence log-probability) and did not restrict stimuli to high-frequency verbs, as acknowledged in footnote 4. That is a validity and comparability concern, not circularity: the paper does not define its LSTM accuracies in terms of Wolf's numbers, nor does it fit anything to reproduce them. The self-citations present (Marvin and Linzen; Gulordava et al.) supply pre-existing, externally available evaluation materials and models, and they do not load-bear by asserting the paper's scaling conclusions. No step in the derivation equates an output to an input by construction.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities and fits no parameters to the target result. The main load-bearing assumptions are the validity of the agreement challenge set as a proxy for syntax, the comparability of Wolf (2019)'s Transformer scores, the log-linear extrapolation of error reduction, and the choice of human accuracy as the target. The single fitted quantity, the error reduction per data doubling, drives the data requirement estimates.

free parameters (1)
  • Error reduction factor per data doubling (estimated from 20M to 40M tokens) = Varies by construction; used as the basis for all data requirement projections
    Section 5 assumes that each doubling of training data yields the same percentage error reduction as the 20M to 40M doubling, and uses this observed rate to extrapolate how many tokens would be needed for human-like or 99.99% accuracy. The projections are therefore conditional on this fitted rate.
assumptions (4)
  • domain assumption Accuracy on the Marvin and Linzen (2018) agreement challenge set is a valid measure of the quality of syntactic representations.
    The paper operationalizes 'syntax' as agreement accuracy on this challenge set and does not use an independent measure of syntactic competence.
  • domain assumption Results from Wolf (2019) for GPT and BERT are comparable to the LSTM results, despite different scoring and stimuli.
    The Transformer comparison in Section 2 and Figures 1, 3, 4, and 5 relies on this comparability; footnote 4 acknowledges that the evaluation differs.
  • domain assumption Error rate decreases log-linearly with each doubling of training data, and the 20M to 40M doubling is representative of future doublings.
    Section 5 extrapolates data requirements under a constant error-reduction factor; the paper does not validate this trend outside the observed corpus size range.
  • domain assumption Human agreement performance reported by Marvin and Linzen (2018) is an appropriate target for model evaluation.
    Human performance is used as the reference line in figures and for extrapolation; the paper acknowledges human errors but still treats perfect accuracy as the applied goal.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quantity doesn't buy quality syntax with neural language models." pith.science (2026). https://pith.science/paper/4GVW4XBL

@misc{pith2026190900111,
  author       = {Pith},
  title        = {Pith review of: Quantity doesn't buy quality syntax with neural language models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4GVW4XBL}},
  note         = {Machine review of arXiv:1909.00111}
}
read the original abstract

Recurrent neural networks can learn to predict upcoming words remarkably well on average; in syntactically complex contexts, however, they often assign unexpectedly high probabilities to ungrammatical words. We investigate to what extent these shortcomings can be mitigated by increasing the size of the network and the corpus on which it is trained. We find that gains from increasing network size are minimal beyond a certain point. Likewise, expanding the training corpus yields diminishing returns; we estimate that the training corpus would need to be unrealistically large for the models to match human performance. A comparison to GPT and BERT, Transformer-based models trained on billions of words, reveals that these models perform even more poorly than our LSTMs in some constructions. Our results make the case for more data efficient architectures.

Figures

Figures reproduced from arXiv: 1909.00111 by the authors.

Figure 1
Figure 1. LSTM agreement performance in several syntactic constructions. The solid horizontal line indicates [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Lines depict number of training tokens needed for LSTMs to achieve human-like (left) or 99.99% accuracy (right) in each syntactic agreement condition, according to our estimates. Bars depict the amount of data on which each model was trained. tions in which our models do not already perform at a human level? As a conservative estimate, we measured the error reduction achieved by doubling the data from 20M to 40M tok… view at source ↗
Figure 3
Figure 3. Language model agreement performance when the verb is adjacent to its subject ( [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Language model agreement performance when the target verb is [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Language model agreement performance between reflexive pronouns and their antecedent in simple [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 23 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Geoff Bacon and Terry Regier. 2019. https://arxiv.org/pdf/1908.09892.pdf Does BERT agree? E valuating knowledge of structure dependence through agreement relations . Technical report, UC Berkeley

    work page Pith review arXiv 2019

  4. [4]

    Alexei Baevski, Sergei Edunov, Yinhan Liu, Luke Zettlemoyer, and Michael Auli. 2019. https://arxiv.org/abs/1903.07785 Cloze-driven pretraining of self-attention networks . Technical report, Facebook AI Research

    arXiv 2019

  5. [5]

    Kathryn Bock and Carol A. Miller. 1991. http://pubman.mpdl.mpg.de/pubman/item/escidoc:2287067/component/escidoc:2287066/Bock_Miller_1991_Broken.pdf Broken agreement . Cognitive Psychology, 23(1):45--93

    work page 1991

  6. [6]

    Le, and Ruslan Salakhutdinov

    Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V. Le, and Ruslan Salakhutdinov. 2019. https://arxiv.org/pdf/1901.02860.pdf Transformer- XL : Attentive language models beyond a fixed-length context . Technical report, Carnegie Mellon University

    arXiv 2019

  7. [7]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://arxiv.org/pdf/1810.04805.pdf BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics

    arXiv 2019

  8. [8]

    \'E mile Enguehard, Yoav Goldberg, and Tal Linzen. 2017. https://doi.org/10.18653/v1/K17-1003 Exploring the syntactic abilities of RNNs with multi-task learning . In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 3--14. Association for Computational Linguistics

    work page doi:10.18653/v1/k17-1003 2017

Show all 35 references
  1. [9]

    Yoav Goldberg. 2019. https://arxiv.org/pdf/1901.05287 Assessing BERT 's syntactic abilities . Technical report, Bar Ilan University

    2019 arXiv

  2. [10]

    Srivastava, Jan Koutn\'ik, Bas R

    Klaus Greff, Rupesh K. Srivastava, Jan Koutn\'ik, Bas R. Steunebrink, and J\"urgen Schmidhuber. 2017. https://arxiv.org/pdf/1503.04069.pdf LSTM : A search space odyssey . IEEE Transactions on Neural Networks and Learning Systems , 28(10):2222--2232

    2017 arXiv

  3. [11]

    Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen, and Marco Baroni. 2018. https://www.aclweb.org/anthology/N18-1108 Colorless green recurrent networks dream hierarchically . In Proceedings of the 2018 Annual Conference of the North American Chapter of the Associ...

    2018

  4. [12]

    Alon Halevy, Peter Norvig, and Fernando Pereira. 2009. The unreasonable effectiveness of data. IEEE Intelligent Systems , 24(2):8--12

    2009

  5. [13]

    Sepp Hochreiter and J\" u rgen Schmidhuber . 1997. Long short-term memory. Neural Computation, 9(8):1735--1780

    1997

  6. [14]

    Hakan Inan, Khashayar Khosravi, and Richard Socher. 2017. https://openreview.net/pdf?id=r1aPbsFle Tying word vectors and word classifiers: A loss framework for language modeling . In Yoshua Bengio and Yann LeCun , editors, Proceedings of the Fifth International Conference on L...

    2017

  7. [15]

    Harold Jeffreys. 1961. Theory of Probability, 3rd edition. Oxford University Press, Oxford

    1961

  8. [16]

    Rafal Jozefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, and Yonghui Wu. 2016. Exploring the limits of language modeling. arXiv preprint arXiv:1602.02410

    2016 arXiv

  9. [17]

    Adhiguna Kuncoro, Chris Dyer, John Hale, Dani Yogatama, Stephen Clark, and Phil Blunsom. 2018. https://aclweb.org/anthology/P18-1132 LSTM s can learn syntax-sensitive dependencies well, but modeling structure makes them better . In Proceedings of the 2018 Annual Meeting of the...

    2018

  10. [18]

    Adhiguna Kuncoro, Chris Dyer, Laura Rimell, Stephen Clark, and Phil Blunsom. 2019. https://www.aclweb.org/anthology/P19-1337 Scalable syntax-aware language models using knowledge distillation . In Proceedings of the 57th Annual Meeting of the Association for Computational Ling...

    2019

  11. [19]

    Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016. https://www.aclweb.org/anthology/Q16-1037 Assessing the ability of LSTMs to learn syntax-sensitive dependencies . Transactions of the Association for Computational Linguistics, 4:521--535

    2016

  12. [20]

    Tal Linzen and Brian Leonard. 2018. https://mindmodeling.org/cogsci2018/papers/0147/0147.pdf Distinct patterns of syntactic agreement errors in recurrent networks and humans . In Proceedings of the 2018 Annual Meeting of the Cognitive Science Society, pages 690--695. Cognitive...

    2018

  13. [21]

    Rebecca Marvin and Tal Linzen. 2018. https://www.aclweb.org/anthology/D18-1151 Targeted syntactic evaluation of language models . In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun'ichi Tsujii, editors, Proceedings of the 2018 Conference on Empirical Methods in Natural ...

    2018

  14. [22]

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. Wikitext-103. Technical report, Salesforce

    2016

  15. [23]

    Tomas Mikolov, Martin Karafi \'a t, Lukas Burget, Jan Cernock \`y , and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In Proceedings of the 11th Annual Conference of the International Speech Communication Association (INTERSPEECH 2010) , pages 1045--1...

    2010

  16. [24]

    Morey and Jeffrey N

    Richard D. Morey and Jeffrey N. Rouder. 2018. https://CRAN.R-project.org/package=BayesFactor BayesFactor: Computation of Bayes Factors for Common Designs . R package version 0.9.12-4.2

    2018

  17. [25]

    Jeffrey Pennington , Richard Socher , and Christopher D. Manning . 2014. https://www.aclweb.org/anthology/D14-1162 GloVe : Global vectors for word representation . In Proceedings of EMNLP

    2014

  18. [26]

    Ofir Press and Lior Wolf. 2017. https://arxiv.org/pdf/1608.05859.pdf Using the output embedding to improve language models . In Proceedings of the 2017 Annual Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics

    2017 arXiv

  19. [27]

    Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf Improving language understanding by generative pre-training . Technical report, OpenAI

    2018

  20. [28]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf Language models are unsupervised multitask learners . Technical report, OpenAI

    2019

  21. [29]

    Nils Reimers and Iryna Gurevych. 2017. https://www.aclweb.org/anthology/D17-1035 Reporting score distributions makes a difference: Performance study of LSTM -networks for sequence tagging . In Martha Palmer, Rebecca Hwa, and Sebastian Riedel, editors, Proceedings of the 2017 C...

    2017

  22. [30]

    Rouder, Paul L

    Jeffrey N. Rouder, Paul L. Speckman, Dongchu Sun, Richard D. Morey, and Geoffrey Iverson. 2009. Bayesian t-tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16(2):225--237

    2009

  23. [31]

    Andrew Trask, Felix Hill, Scott E Reed, Jack Rae, Chris Dyer, and Phil Blunsom. 2018. http://papers.nips.cc/paper/8027-neural-arithmetic-logic-units.pdf Neural arithmetic logic units . In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editor...

    2018

  24. [32]

    Trinh, Andrew M

    Trieu H. Trinh, Andrew M. Dai, Minh-Thang Luong, and Quoc V. Le. 2018. https://openreview.net/pdf?id=r1aPbsFle Learning longer-term dependencies in RNNs with auxiliary losses . In Proceedings of the 35th International Conference on Machine Learning, pages 4965--4974. PMLR 80

    2018

  25. [33]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. http://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf Attention is all you need . In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fe...

    2017

  26. [34]

    Ethan Wilcox, Peng Qian, Richard Futrell, Miguel Ballesteros, and Roger Levy. 2019. https://arxiv.org/pdf/1903.00943.pdf Structural supervision improves learning of non-local grammatical dependencies . In Proceedings of the 2019 Annual Conference of the North American Chapter ...

    2019 arXiv

  27. [35]

    Thomas Wolf. 2019. https://huggingface.co/bert-syntax/extending-bert-syntax.pdf Some additional experiments extending the tech report ``assessing BERT 's syntactic abilities'' by Yoav Goldberg . Technical report, Huggingface Inc

    2019

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.