GiLT augments Transformers with semantic dependency graphs by modulating attention to improve syntactic generalization while keeping perplexity competitive and enabling better finetuning on downstream tasks.
Dependency Recurrent Neural Language Models for Sentence Completion
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
fields
cs.CL 2verdicts
UNVERDICTED 2representative citing papers
Semantic constituency graphs outperform syntactic constituency and dependency structures from seven formalisms when added to a Transformer for language modeling.
citing papers explorer
-
GiLT: Augmenting Transformer Language Models with Dependency Graphs
GiLT augments Transformers with semantic dependency graphs by modulating attention to improve syntactic generalization while keeping perplexity competitive and enabling better finetuning on downstream tasks.
-
Linguistic Frameworks Go Toe-to-Toe at Neuro-Symbolic Language Modeling
Semantic constituency graphs outperform syntactic constituency and dependency structures from seven formalisms when added to a Transformer for language modeling.