Dirichlet Flow Matching with Applications to DNA Sequence Design
read the original abstract
Discrete diffusion or flow models could enable faster and more controllable sequence generation than autoregressive models. We show that na\"ive linear flow matching on the simplex is insufficient toward this goal since it suffers from discontinuities in the training target and further pathologies. To overcome this, we develop Dirichlet flow matching on the simplex based on mixtures of Dirichlet distributions as probability paths. In this framework, we derive a connection between the mixtures' scores and the flow's vector field that allows for classifier and classifier-free guidance. Further, we provide distilled Dirichlet flow matching, which enables one-step sequence generation with minimal performance hits, resulting in $O(L)$ speedups compared to autoregressive models. On complex DNA sequence generation tasks, we demonstrate superior performance compared to all baselines in distributional metrics and in achieving desired design targets for generated sequences. Finally, we show that our classifier-free guidance approach improves unconditional generation and is effective for generating DNA that satisfies design targets. Code is available at https://github.com/HannesStark/dirichlet-flow-matching.
This paper has not been read by Pith yet.
Forward citations
Cited by 8 Pith papers
-
Spherical Flows for Sampling Categorical Data
Spherical vMF flows reduce the continuity equation on the sphere to a scalar ODE in cosine similarity, enabling posterior-weighted sampling of categorical sequences via cross-entropy trained posteriors.
-
Spherical Flows for Sampling Categorical Data
Spherical vMF flows for categorical sequences reduce the continuity equation to a scalar ODE in cosine similarity and yield posterior-weighted tangent velocities for improved ODE and PC sampling.
-
Generative Modeling of Discrete Data Using Geometric Latent Subspaces
A geometric latent-subspace model on Riemannian manifolds of categorical distributions enables low-dimensional generative modeling of discrete data via isometries and geometric PCA for flow matching.
-
Flexible Flows for Biological Sequence Design
Enhances Discrete Flow Matching with domain-specific couplings, latent edit-based rates, latent classifier-free guidance, and temperature scaling to reach SOTA on DNA and peptide sequence tasks.
-
Back on Track: Aligning Rewards and States for Reasoning in Diffusion Large Language Models
PAPO improves reasoning performance in diffusion LLMs by converting sparse terminal rewards into dense step-wise credit and replaying real high-uncertainty trajectories, reporting gains up to 42.2% on Countdown.
-
Coupling Models for One-Step Discrete Generation
Coupling Models enable single-step discrete sequence generation via learned couplings to Gaussian latents and outperform prior one-step baselines on text perplexity, biological FBD, and image FID metrics.
-
Spherical Flows for Sampling Categorical Data
Spherical flows on S^{d-1} with vMF noise reduce the continuity equation to a scalar ODE in cosine similarity, yielding posterior-weighted marginal velocity and score that enable ODE and predictor-corrector sampling f...
-
Interpolating Discrete Diffusion Models with Controllable Resampling
IDDM interpolates diffusion transitions with a resampling mechanism to lessen dependence on intermediate latents and improve sample quality over masked and uniform discrete diffusion models.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.