Bucket Masking improves protein fitness prediction by up to 14% over random masking by preferentially masking structurally coupled residue groups on four downstream tasks.
Weld, Luke Zettlemoyer, and Omer Levy
4 Pith papers cite this work. Polarity classification is still indexing.
4
Pith papers citing it
representative citing papers
Autoregressive language models trained on data with middle spans relocated to the end learn infilling without degrading left-to-right perplexity or sampling quality.
A self-supervised prosody encoder with speaker disentanglement strategies outperforms raw prosody and HuBERT baselines on pitch reconstruction and prosodic event detection while achieving strong speaker separation.
Three BERT models are further pre-trained on Norwegian clinical notes and discharge summaries, then shown to outperform their base models on synthetic clinical benchmarks and real-world tasks.
citing papers explorer
-
Efficient Training of Language Models to Fill in the Middle
Autoregressive language models trained on data with middle spans relocated to the end learn infilling without degrading left-to-right perplexity or sampling quality.
-
Privacy-preserving Prosody Representation Learning
A self-supervised prosody encoder with speaker disentanglement strategies outperforms raw prosody and HuBERT baselines on pitch reconstruction and prosodic event detection while achieving strong speaker separation.
-
KliniskVestBERT: BERT Model Specialised to Norwegian Clinical Texts
Three BERT models are further pre-trained on Norwegian clinical notes and discharge summaries, then shown to outperform their base models on synthetic clinical benchmarks and real-world tasks.