Minimal single-token substitutions in BERT show that common tokens induce small embedding shifts, rare tokens induce larger shifts, and perturbation effects grow with layer depth, which the authors interpret as support for early-layer explanations.
Word lengths are optimized for efficient communication.Proceedings of the National Academy of Sciences, 108(9):3526–3529,
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Probing the Embedding Space of Transformers via Minimal Token Perturbations
Minimal single-token substitutions in BERT show that common tokens induce small embedding shifts, rare tokens induce larger shifts, and perturbation effects grow with layer depth, which the authors interpret as support for early-layer explanations.