A small transformer trained on counter languages encodes the current stack depth in its final-layer activations, recoverable by simple probing classifiers.
A Survey of Neural Networks and Formal Languages
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
This report is a survey of the relationships between various state-of-the-art neural network architectures and formal languages as, for example, structured by the Chomsky Language Hierarchy. Of particular interest are the abilities of a neural architecture to represent, recognize and generate words from a specific language by learning from samples of the language.
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Emergent Stack Representations in Modeling Counter Languages Using Transformers
A small transformer trained on counter languages encodes the current stack depth in its final-layer activations, recoverable by simple probing classifiers.