A deterministic Fourier expansion of token IDs plus a lightweight shared MLP matches learned embeddings on STS-B at a fraction of the parameter count in small transformers.
A large an- notated corpus for learning natural language inference
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Parameter-Efficient Transformer Embeddings
A deterministic Fourier expansion of token IDs plus a lightweight shared MLP matches learned embeddings on STS-B at a fraction of the parameter count in small transformers.