On a new task-switching benchmark, a cisformer with expressive attention reaches about 95% accuracy, while standard transformers, LSTMs, and MLPs all stay near or below 60%.
In: The Eleventh International Conference on Learning Representations (2022) Elhage,N.,Hume,T.,Olsson,C.,Schiefer,N.,Henighan,T.,Kravec,S.,Hatfield-
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
REJECT 1representative citing papers
citing papers explorer
-
Small transformer architectures for task switching
On a new task-switching benchmark, a cisformer with expressive attention reaches about 95% accuracy, while standard transformers, LSTMs, and MLPs all stay near or below 60%.