Masked-token prediction errors in BERT reveal clusters of interchangeable, semantically related tokens, and the average per-token accuracy increases through the transformer layers and correlates with fine-tuning accuracy.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Learning Mechanism Underlying NLP Pre-Training and Fine-Tuning
Masked-token prediction errors in BERT reveal clusters of interchangeable, semantically related tokens, and the average per-token accuracy increases through the transformer layers and correlates with fine-tuning accuracy.