Annotator identity functions as a shortcut in crowdsourced NLU benchmarks, and models often fail to generalize to examples from annotators absent from training.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2019 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets
Annotator identity functions as a shortcut in crowdsourced NLU benchmarks, and models often fail to generalize to examples from annotators absent from training.