BERT's apparent knowledge of English NPI licensing varies across five evaluation methods, from near-perfect on gradient minimal pairs to inconsistent on absolute judgments and scope probing.
Do Language Models Understand Anything? On the Ability of LSTMs to Understand Negative Polarity Items
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
In this paper, we attempt to link the inner workings of a neural language model to linguistic theory, focusing on a complex phenomenon well discussed in formal linguis- tics: (negative) polarity items. We briefly discuss the leading hypotheses about the licensing contexts that allow negative polarity items and evaluate to what extent a neural language model has the ability to correctly process a subset of such constructions. We show that the model finds a relation between the licensing context and the negative polarity item and appears to be aware of the scope of this context, which we extract from a parse tree of the sentence. With this research, we hope to pave the way for other studies linking formal linguistics to deep learning.
fields
cs.CL 1years
2019 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Investigating BERT's Knowledge of Language: Five Analysis Methods with NPIs
BERT's apparent knowledge of English NPI licensing varies across five evaluation methods, from near-perfect on gradient minimal pairs to inconsistent on absolute judgments and scope probing.