Pith. sign in

REVIEW 12 cited by

Structured Denoising Diffusion Models in Discrete State-Spaces

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2107.03006 v3 pith:PJTKKPO5 submitted 2021-07-07 cs.LG cs.AIcs.CLcs.CV

Jacob Austin , Daniel D. Johnson , Jonathan Ho , Daniel Tarlow , Rianne van den Berg This is my paper
classification cs.LGcs.AIcs.CLcs.CV
keywords modelsdiffusiondenoisingdiscreteimageintroducematricesmodel
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Denoising diffusion probabilistic models (DDPMs) (Ho et al. 2020) have shown impressive results on image and waveform generation in continuous state spaces. Here, we introduce Discrete Denoising Diffusion Probabilistic Models (D3PMs), diffusion-like generative models for discrete data that generalize the multinomial diffusion model of Hoogeboom et al. 2021, by going beyond corruption processes with uniform transition probabilities. This includes corruption with transition matrices that mimic Gaussian kernels in continuous space, matrices based on nearest neighbors in embedding space, and matrices that introduce absorbing states. The third allows us to draw a connection between diffusion models and autoregressive and mask-based generative models. We show that the choice of transition matrix is an important design decision that leads to improved results in image and text domains. We also introduce a new loss function that combines the variational lower bound with an auxiliary cross entropy loss. For text, this model class achieves strong results on character-level text generation while scaling to large vocabularies on LM1B. On the image dataset CIFAR-10, our models approach the sample quality and exceed the log-likelihood of the continuous-space DDPM model.

Discussion (0). Sign in to comment.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 82 citations worldwide. Full citation record

  1. FRIGID: Scaling Diffusion-Based Molecular Generation from Mass Spectra at Training and Inference Time

    cs.LG 2026-04 unverdicted novelty 7.0 of 10

    FRIGID scales a diffusion-based model for de novo molecular structure generation from mass spectra, reaching over 18% top-1 accuracy on MassSpecGym and tripling prior bests on NPLIB1 via large unlabeled training and i...

  2. A collaborative constrained graph diffusion model for the generation of realistic synthetic molecules

    cs.LG 2025-05 conditional novelty 7.0 of 10

    A valence-preserving double edge-swap diffusion model with a learned time estimator generates chemically valid molecules with property distributions closer to real molecules than JTVAE and DiGress on the GuacaMol benchmark.

  3. FairDiffuseVQVAE: Sampling-Time Fairness in Tabular Diffusion via Conditional Refinement of Vector-Quantized Latents

    cs.LG 2026-07 conditional novelty 6.0 of 10

    FairDiffuseVQVAE reaches state-of-the-art fairness on the standard tabular benchmark (DPR 0.702, EOR 0.686) by uniform protected-attribute sampling at inference, paying ~15 AUC points of utility.

  4. D$e^+e^-$ffusion: Capturing the Beam-Beam Physics of $e^+e^-$ Collisions with Diffusion Models

    hep-ph 2026-07 conditional novelty 6.0 of 10

    A diffusion model trained on GuineaPig++ reproduces FCC-ee beam-induced pair-production distributions at particle and detector level, about 10^4 times faster.

  5. Just on Time: Token-Level Early Stopping for Diffusion Language Models

    cs.LG 2026-02 conditional novelty 6.0 of 10

    Jot, a token-level early stopping rule using a top-2 confidence ratio and spatial context, speeds up diffusion language model decoding by up to 19.6x with minor quality loss.

  6. MiAD: Mirage Atom Diffusion for De Novo Crystal Generation

    cs.LG 2025-11 conditional novelty 6.0 of 10

    Mirage infusion lets crystal diffusion models vary atom counts during generation and raises the S.U.N. rate on MP-20 to 8.2%.

  7. CANDI: Hybrid Discrete-Continuous Diffusion Models

    cs.LG 2025-10 conditional novelty 6.0 of 10

    CANDI combines masked and Gaussian corruption in one noising process, letting discrete diffusion models use continuous gradients for joint updates and guidance.

  8. Learning Distributions over Permutations and Rankings with Factorized Representations

    cs.LG 2025-05 conditional novelty 6.0 of 10

    Factorized codes for permutations let standard transformers learn arbitrary permutation distributions with guaranteed validity, and the paper adds a fast insertion-vector decoding theorem plus two new benchmarks.

  9. DiffMS: Diffusion Generation of Molecules Conditioned on Mass Spectra

    cs.LG 2025-02 conditional novelty 6.0 of 10

    DiffMS, a formula-constrained discrete graph diffusion model conditioned on mass spectra, achieves state-of-the-art de novo molecule generation on NPLIB1 and MassSpecGym.

  10. TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding

    cs.CL 2026-07 conditional novelty 5.5 of 10

    Trajectory-aware commit gating (TILG + History Gate + capped extra promotion) improves or preserves DLLM accuracy while reducing steps and raising tokens-per-forward without retraining.

  11. DLM-One: Diffusion Language Models for One-Step Sequence Generation

    cs.CL 2025-05 conditional novelty 5.0 of 10

    DLM-One distills a continuous diffusion language model into a one-step student, achieving roughly 500x inference speedup while staying within a few percent of the teacher on BLEU, ROUGE, and BERTScore, with substantia...

  12. Masked Diffusion Language Models with Frequency-Informed Training

    cs.CL 2025-09 conditional novelty 4.0 of 10

    Masked diffusion language models trained on 100M words match a hybrid GPT-BERT baseline on BabyLM tests, with a rare-word-focused masking variant.

Pith tools