REVIEW 20 cited by
Diffusion Language Models Are Versatile Protein Learners
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Diffusion Language Models Are Versatile Protein Learners
read the original abstract
This paper introduces diffusion protein language model (DPLM), a versatile protein language model that demonstrates strong generative and predictive capabilities for protein sequences. We first pre-train scalable DPLMs from evolutionary-scale protein sequences within a generative self-supervised discrete diffusion probabilistic framework, which generalizes language modeling for proteins in a principled way. After pre-training, DPLM exhibits the ability to generate structurally plausible, novel, and diverse protein sequences for unconditional generation. We further demonstrate the proposed diffusion generative pre-training makes DPLM possess a better understanding of proteins, making it a superior representation learner, which can be fine-tuned for various predictive tasks, comparing favorably to ESM2 (Lin et al., 2022). Moreover, DPLM can be tailored for various needs, which showcases its prowess of conditional generation in several ways: (1) conditioning on partial peptide sequences, e.g., generating scaffolds for functional motifs with high success rate; (2) incorporating other modalities as conditioner, e.g., structure-conditioned generation for inverse folding; and (3) steering sequence generation towards desired properties, e.g., satisfying specified secondary structures, through a plug-and-play classifier guidance. Code is released at \url{https://github.com/bytedance/dplm}.
Forward citations
Cited by 20 Pith papers
-
A-CODE: Fully Atomic Protein Co-Design with Unified Multimodal Diffusion
A-CODE presents a fully atomic one-stage multimodal diffusion model for protein co-design that claims superior unconditional generation performance over prior one- and two-stage models plus a tenfold success-rate gain...
-
Large Language Diffusion Models
LLaDA is a scalable diffusion-based language model that matches autoregressive LLMs like LLaMA3 8B on tasks and surpasses GPT-4o on reversal poem completion.
-
Dual Triangle Attention: Effective Bidirectional Attention Without Positional Embeddings
Dual Triangle Attention achieves effective bidirectional attention with built-in positional inductive bias via dual triangular masks, outperforming standard bidirectional attention on position-sensitive tasks and show...
-
Variable-Length Generative Protein Design via Generalized Poisson Flow
Generalized Poisson Flow learns variable protein length via an inhomogeneous Poisson rate plus within-length flow matching, with KL bounds and gains on structure, sequence, motif, and peptide tasks.
-
Flexible Flows for Biological Sequence Design
Enhances Discrete Flow Matching with domain-specific couplings, latent edit-based rates, latent classifier-free guidance, and temperature scaling to reach SOTA on DNA and peptide sequence tasks.
-
AMix-2: Establishing Protein as a Native Modality in Large Language Models
AMix-2 unifies protein sequences and text in one LLM via shared tokens and block-wise diffusion modeling, introduces the ProteinArena benchmark, and reports competitive performance against task-specific protein models...
-
SurfDesign: Effective Protein Design on Molecular Surfaces
SurfDesign introduces surface-conditioned protein design via manifold modeling and equivariant message passing on surfaces integrated with pretrained language models, outperforming prior methods on binder and enzyme d...
-
Yeti: A compact protein structure tokenizer for reconstruction and multi-modal generation
Yeti is a compact tokenizer for protein structures that delivers strong codebook use, token diversity, and reconstruction while enabling from-scratch multimodal generation of plausible sequences and structures with 10...
-
Primal-Dual Guided Decoding for Constrained Discrete Diffusion
Primal-dual guided decoding casts constrained discrete diffusion as a KL-regularized optimization solved online with adaptive Lagrangian multipliers to satisfy constraints while staying close to the unconstrained mode...
-
Coupling Models for One-Step Discrete Generation
Coupling Models enable single-step discrete sequence generation via learned couplings to Gaussian latents and outperform prior one-step baselines on text perplexity, biological FBD, and image FID metrics.
-
MP2D: Constrained Monte Carlo Tree-Guided Diffusion for Multi-Objective Protein Sequence Design
MP2D is a framework that guides discrete diffusion denoising with constrained MCTS and Pareto rewards to optimize protein sequences for four to five simultaneous objectives, outperforming baselines on antimicrobial pe...
-
Towards A Generative Protein Evolution Machine with DPLM-Evo
DPLM-Evo adds explicit edit operations and a latent alignment space to discrete diffusion protein models, achieving SOTA single-sequence mutation effect prediction on ProteinGym while supporting variable-length generation.
-
Towards A Generative Protein Evolution Machine with DPLM-Evo
DPLM-Evo is an evolutionary discrete diffusion framework that models protein sequences via explicit substitution, insertion, and deletion operations, achieving state-of-the-art single-sequence mutation effect predicti...
-
Towards A Generative Protein Evolution Machine with DPLM-Evo
DPLM-Evo introduces an evolutionary discrete diffusion framework with explicit edit prediction and contextual noising that claims SOTA single-sequence mutation effect prediction on ProteinGym while supporting variable...
-
MIMIC: A Generative Multimodal Foundation Model for Biomolecules
MIMIC is a split-track encoder-decoder foundation model that unifies sequence reconstruction, prediction, and constrained design across nucleic acids, proteins, and regulatory context using partially observed multimod...
-
STRIDE: Post-Training LLMs to Reason and Refine Bio-Sequences via Edit Trajectories
Training LLMs to emit executable edit trajectories (INSERT/DELETE/REPLACE) from Levenshtein alignments plus policy optimization improves oracle-scored bio-sequence optimization success and novelty.
-
A Unification of Discrete, Gaussian, and Simplicial Diffusion
Discrete, Gaussian, and simplicial diffusion models for sequences are unified as parameterizations of the Wright-Fisher population genetics model, allowing multi-domain training and stable simplicial diffusion.
-
HD-Prot: A Protein Language Model for Joint Sequence-Structure Modeling with Continuous Structure Tokens
HD-Prot shows that a protein language model can jointly generate sequences and structures using continuous structure tokens instead of quantized tokens, reaching competitive performance on four protein design tasks.
-
On the Design Space of Discrete Diffusion Online Adaptation for Molecular Optimization
Online fine-tuning of discrete diffusion models with complementary acquisition, CVaR shaping, density-entropy debiasing, replay, and validity control finds better molecules under fixed oracle budgets than offline fine...
-
Co-Generative De Novo Functional Protein Design
CodeFP jointly generates protein sequences and structures using functional local structures and auxiliary supervision, yielding 6.1% better functional consistency and 3.2% better foldability than prior baselines.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.