A masked-language-model pointwise mutual information score, with no fine-tuning on commonsense databases, matches or beats supervised baselines when mining novel Wikipedia triples, though the evaluation is weakened by test-set hyperparameter tuning.
The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Task
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
A writer's style depends not just on personal traits but also on her intent and mental state. In this paper, we show how variants of the same writing task can lead to measurable differences in writing style. We present a case study based on the story cloze task (Mostafazadeh et al., 2016a), where annotators were assigned similar writing tasks with different constraints: (1) writing an entire story, (2) adding a story ending for a given story context, and (3) adding an incoherent ending to a story. We show that a simple linear classifier informed by stylistic features is able to successfully distinguish among the three cases, without even looking at the story context. In addition, combining our stylistic features with language model predictions reaches state of the art performance on the story cloze challenge. Our results demonstrate that different task framings can dramatically affect the way people write.
fields
cs.CL 1years
2019 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Commonsense Knowledge Mining from Pretrained Models
A masked-language-model pointwise mutual information score, with no fine-tuning on commonsense databases, matches or beats supervised baselines when mining novel Wikipedia triples, though the evaluation is weakened by test-set hyperparameter tuning.