On structured SNOMED-derived memorization tasks, small transformers memorize most when embedding size is large and depth is kept low.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Capacity Matters: a Proof-of-Concept for Transformer Memorization on Real-World Data
On structured SNOMED-derived memorization tasks, small transformers memorize most when embedding size is large and depth is kept low.