REVIEW 8 cited by
Protein Representation Learning by Geometric Structure Pretraining
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Learning effective protein representations is critical in a variety of tasks in biology such as predicting protein function or structure. Existing approaches usually pretrain protein language models on a large number of unlabeled amino acid sequences and then finetune the models with some labeled data in downstream tasks. Despite the effectiveness of sequence-based approaches, the power of pretraining on known protein structures, which are available in smaller numbers only, has not been explored for protein property prediction, though protein structures are known to be determinants of protein function. In this paper, we propose to pretrain protein representations according to their 3D structures. We first present a simple yet effective encoder to learn the geometric features of a protein. We pretrain the protein graph encoder by leveraging multiview contrastive learning and different self-prediction tasks. Experimental results on both function prediction and fold classification tasks show that our proposed pretraining methods outperform or are on par with the state-of-the-art sequence-based methods, while using much less pretraining data. Our implementation is available at https://github.com/DeepGraphLearning/GearNet.
Forward citations
Cited by 8 Pith papers
-
Exploring the Alignment of Generation and Understanding in Protein Structure Modeling
Aligning a protein diffusion generator's internal representations to a pretrained structure encoder (ProteinMPNN) raises the MotifBench motif-scaffolding score from 39.2 to 47.1 (~20% relative) over the Protpardelle-1...
-
Tokenizing Loops of Antibodies
Igloo is a multimodal antibody loop tokenizer that, when plugged into protein language models, modestly improves loop retrieval, affinity prediction, and structure-consistent loop generation.
-
Relation-Aware Equivariant Graph Networks for Epitope-Unknown Antibody Design and Specificity Optimization
RAAD is a relation-aware equivariant graph network that co-generates antibody CDR sequences and structures in one shot and adds a contrastive specificity loss.
-
GLProtein: Global-and-Local Structure Aware Protein Representation Learning
GLProtein pre-trains a BERT-style protein model with triplet contrastive learning on structural similarity, 3D distance encoding, and substructure-based amino acid embeddings, improving contact and PPI prediction.
-
Protein FID: Improved Evaluation of Protein Structure Generative Models
A protein-specific FID metric computed from ESM3 embeddings reveals that current generative models still substantially under-sample the structural diversity of natural PDB proteins.
-
EvoLlama: Enhancing LLMs' Understanding of Proteins via Multimodal Structure and Sequence Representations
EvoLlama aligns ESM-2 sequence embeddings and ProteinMPNN structure embeddings with Llama-3, improving protein understanding over text-only LLMs on Mol-Instructions and PEER benchmarks.
-
Computational Protein Science in the Era of Large Language Models (LLMs)
A survey that categorizes protein language models by the knowledge they learn and reviews their applications, with no new experimental results.
-
A Comprehensive Review of Protein Language Models
A survey paper that catalogs protein language models, their architectures, training data, benchmarks, and tools, but lacks a systematic methodology and contains several factual errors.
Discussion (0). Continue with ORCID to comment.