Masked-position MLM plus JEPA latent prediction outperforms MLM-only pretraining on 10-11 of 16 downstream tasks for 35M-150M protein models while JEPA alone fails.
ProteinBERT: a universal deep-learning model of protein sequence and function.Bioinformatics, 38(8): 2102–2110, February 2022
4 Pith papers cite this work. Polarity classification is still indexing.
4
Pith papers citing it
citation-role summary
baseline 1
citation-polarity summary
roles
baseline 1polarities
baseline 1representative citing papers
Bucket Masking improves protein fitness prediction by up to 14% over random masking by preferentially masking structurally coupled residue groups on four downstream tasks.
EvoFlows uses evolutionary edit flows to generate controllable mutations on protein templates, producing variants consistent with natural families but farther from the starting sequence than prior methods.
PLASMA applies regularized optimal transport with Sinkhorn iterations to produce fast, interpretable residue-level alignments and similarity scores between protein structures.
citing papers explorer
-
ProteinJEPA: Latent prediction complements protein language models
Masked-position MLM plus JEPA latent prediction outperforms MLM-only pretraining on 10-11 of 16 downstream tasks for 35M-150M protein models while JEPA alone fails.
-
Structure-Aware Masking for Protein Representation Learning
Bucket Masking improves protein fitness prediction by up to 14% over random masking by preferentially masking structurally coupled residue groups on four downstream tasks.
-
EvoFlows: Evolutionary Edit-Based Flow-Matching for Protein Engineering
EvoFlows uses evolutionary edit flows to generate controllable mutations on protein templates, producing variants consistent with natural families but farther from the starting sequence than prior methods.
-
Fast and Interpretable Protein Substructure Alignment via Optimal Transport
PLASMA applies regularized optimal transport with Sinkhorn iterations to produce fast, interpretable residue-level alignments and similarity scores between protein structures.