Pith. sign in

REVIEW 2 major objections 3 minor 212 references

This survey claims that every discrete diffusion model—transition-matrix, masking, or score/ratio—instantiates the same four-component structure and that tokenization is the primary design axis.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · deepseek-v4-flash

2026-08-02 05:11 UTC pith:TTRGROUM

load-bearing objection Useful map of discrete diffusion, but the 'every model fits four components' claim is narrower than its own scope allows. the 2 major comments →

arxiv 2607.13431 v1 pith:TTRGROUM submitted 2026-07-15 cs.LG cs.AIcs.CL

Discrete Diffusion Models: A Unified Framework from Tokenization to Generation

classification cs.LG cs.AIcs.CL
keywords discrete diffusiontokenizationstate space designmasked diffusiontransition matrixscore/ratio parameterizationdiffusion language modelssurvey
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that discrete diffusion models, despite their surface variety, all share one four-component structure: corruption operator, denoiser parameterization, training objective, and sampler. It argues that how the discrete state space is built—tokenization—is the primary axis that shapes those four components, and that transition-matrix, masking, and score/ratio formulations are different instantiations of a common design space. If the paper is right, choosing a model becomes a checklist of four separable decisions, and findings from one domain (text, images, proteins, graphs) transfer to others component by component. A sympathetic reader would care because the framework promises to convert a fragmented literature into a reusable design guide rather than another taxonomy.

Core claim

The paper's central claim is that the apparent diversity of discrete diffusion models conceals a common design space. It decomposes every model into four components: (1) a corruption operator (a transition matrix or rate matrix that specifies how clean tokens become noisy), (2) a denoiser parameterization (predicting the clean token, logits, or probability ratios), (3) a training objective (variational, simplified denoising, or score/ratio matching), and (4) a sampler (ancestral, confidence-based remasking, blockwise, guided, or accelerated). Major formulation families—general transition-matrix models, absorbing/masking diffusion, continuous-time Markov chains, and score/ratio-based methods—

What carries the argument

The load-bearing object is the four-component decomposition: (1) corruption operator (transition matrix or rate matrix defining the forward noising), (2) denoiser parameterization (what the network predicts: clean-token distribution, logits, or probability ratios), (3) training objective (ELBO, simplified denoising, or score/ratio matching), and (4) sampler (ancestral, remasking, blockwise, guided, accelerated). The paper pairs this with a tokenization lens: the discrete state space—subword vocabularies, quantized codebooks, or natural scientific alphabets—determines the corruption topology and denoising difficulty. The decomposition does the work of making different formulations interconver

Load-bearing premise

The load-bearing premise is that the sample of papers surveyed, gathered by following citations from a handful of foundational discrete-diffusion formulations, fairly represents the whole field; under-representing one branch, such as embed-then-diffuse or non-Markovian corruptions, could make the shared structure look more universal than it is.

What would settle it

Find any model that is widely accepted as discrete diffusion but cannot be placed in the four components without forcing—for example, a forward process that is not a per-position categorical Markov chain, or an objective that is neither variational, denoising, nor score/ratio matching. A more empirical probe: train the paper's predicted-valid 'off-diagonal' pairings (e.g., absorbing corruption with a score-entropy objective) and show they fail to produce competitive likelihoods or samples at scale, which would falsify the claim that components are independently selectable.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Model families that look different (masking, substitution, score/ratio) differ mainly in corruption and parameterization; their training objectives and samplers can often be swapped.
  • Masking vs. substitution vs. structured corruption is orthogonal to predict-x0 vs. predict-ratio parameterization, so combinations not yet tried are legitimate candidates for new models.
  • Tokenization is a primary design axis: vocabulary granularity, codebook topology, and natural alphabets shape denoising difficulty, controllability, and cost, so tokenizer choice should be reported alongside model choice.
  • Masked diffusion's ELBO collapses to a reweighted masked-language-modeling objective, which explains why large-scale diffusion language models train like BERT-style models and why time conditioning is partly redundant.
  • Likelihood-based training and simplified denoising objectives occupy different points on a compute-vs-bound trade-off; the field has not settled on one as universally best.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If tokenization is the primary axis, then pushing tokenizer development (e.g., semantic alignment of codebooks) may yield larger quality gains per unit effort than pushing denoiser architecture—a hypothesis that could be tested by ablating tokenizer changes with a fixed denoiser.
  • The off-diagonal cells the paper highlights suggest a concrete research program: systematically crossing absorbing corruption with ratio/score parameterization, or structured substitution with predict-x0, and measuring which pairings scale; the survey does not run these experiments.
  • The 'common design space' is built from a particular citation graph; if proteins, graphs, or media tokenization had been represented differently, the centrality of tokenization as an axis might be less pronounced—so the framework's strongest test is whether it survives application to a model family outside the sampled graph.
  • Framing inference as a policy layer implies that speedups are often sampler-side gains, but the survey's own caveats suggest reported 'fewer steps' results only matter if they translate into wall-clock latency; a fair comparison should standardize hardware and batch.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. This paper is a narrative survey of discrete denoising diffusion models (DDMs), organized around the thesis that the construction of the discrete state space—'tokenization'—is the primary design axis. It proposes a four-component decomposition (corruption operator, denoiser parameterization, training objective, sampler) and argues that transition-matrix, masking/absorbing, score/ratio, and flow-matching formulations are instantiations of this common structure. The paper covers tokenization across text, code, quantized media, and scientific alphabets; systematizes training and inference; and discusses scaling, systems, applications, and evaluation. It also introduces several proposed tokenizer diagnostics and releases a companion repository.

Significance. If the unification claim is accepted, the paper provides a useful shared vocabulary and design checklist for a rapidly growing field. Its strengths include unusually explicit honesty about evidence quality: it flags that LLaDA's scale competitiveness is 'self-reported... not independently audited' (§8.2), that Mercury's throughput is 'reported by the originating work rather than independently reproduced' (§8.4), and that the proposed diagnostics in §4.5 are 'synthesized / not standardized' and not validated with new experiments. The cross-domain coverage—text, code, media, proteins, genomics, molecules, planning—is broad, and Table 5 is a practical mapping. The companion repository is a valuable resource. The main weakness is that the paper's most general claim ('every discrete diffusion model...') exceeds the evidence base it explicitly describes.

major comments (2)
  1. [§5.6; §2] The paper's central claim, stated in §5.6 as 'every discrete diffusion model instantiates the same four-component structure,' is stronger than the evidence assembled in §2. The 'Scope and selection' paragraph describes a narrative snowball sample around five roots, notes 'we do not claim exhaustive coverage,' and uses inclusion criterion (i) already expressed in the corruption/parameterization/objective/sampler vocabulary. The corpus therefore cannot support a universal quantifier. Boundary cases sharpen the problem: embed-then-diffuse methods are explicitly excluded in §1, and non-Markov forward processes (e.g., Discrete Non-Markov Diffusion Models in §7.1) are not shown to be representable as a Q_t/R_t corruption operator. I recommend narrowing the claim to 'all surveyed formulations' throughout, or adding a systematic independent census and a treatment of non-Markov processes.
  2. [§6.1, Eq. (16)] The sign convention in Eq. (16) is internally inconsistent as written. The text states that the weight α'_t/(1−α_t) is negative and that 'L∞ itself [is] negative (it is an expected log-likelihood).' But since α'_t<0 and the expectation of δ_{xt,m} x0^⊤ log μθ(xt,t) is ≤0, the integrand is nonnegative, making L∞ nonnegative. The subsequent claim that 'the ELBO reduces to −L∞' adds to the confusion about which quantity is minimized/maximized. Please reconcile Eq. (16) with the convention in Shi et al. (2024b) and state explicitly whether L∞ is the negative ELBO or the ELBO itself.
minor comments (3)
  1. [§4.5] The tokenizer diagnostics (reconstruction–generation gap, denoising loss curve, schedule sensitivity) are listed in the contributions as introduced diagnostics, but the section honestly labels them 'synthesized / not standardized' and 'we have not validated the proposed diagnostics with new experiments.' This caveat should also appear in the abstract or contributions to avoid overclaiming.
  2. [§6.6; §8.4] There are several missing spaces in citations, e.g., 'SeveriRissanen(2023)' and 'MercuryKhanna et al.' Please run a formatting pass over inline citations.
  3. [§5.2] The statement that the cosine schedule is 'Fisher-Rao optimal for masked discrete diffusion' is strong and relies on a single reference. A one-sentence explanation of the optimality criterion and its assumptions would help readers assess the claim.

Circularity Check

0 steps flagged

No significant circularity: the survey’s four-component taxonomy is an organizing framework with independent analytic content; the main caveat is a scope/representativeness limitation in §2, not a circular derivation.

full rationale

This is a narrative survey rather than a derivation chain. The four-component decomposition (corruption operator, denoiser parameterization, training objective, sampler) is presented as an organizing taxonomy, and the mathematical equivalences it relies on are derived from stated assumptions rather than from the paper’s conclusions: §5.5 shows predict-x0 and ratio parameterizations are interconvertible via Eq. 7 and Eq. 11, and §6.1 derives the discrete ELBO from the forward Markov chain. There are no fitted parameters and no quantities called predictions that reduce to fitted values. The self-citations (e.g., Ye et al. 2025a,b,c,d; 2024a) are used as literature references for diffusion’s planning/refinement motivation and for Dream 7B scaling evidence; they are not load-bearing for the framework’s internal structure. The main caveat is §2’s “Scope and selection”: the survey was assembled by snowballing around five foundational formulations and explicitly “do[es] not claim exhaustive coverage,” and inclusion criterion (i) is already phrased in the four-component vocabulary (“distinct corruption/parameterization/objective/sampler choice”). This weakens the §5.6 universal quantifier (“every discrete diffusion model instantiates the same four-component structure”) as an empirical generalization, but it is a representativeness/scope limitation rather than a circular derivation: the framework’s analytic content, such as the interconvertibility argument, does not depend on the sample. The paper also flags its own limitations honestly (§4.5 diagnostics “not validated”; §8.2/§8.4 self-reported numbers “not independently audited”), which further supports the conclusion that no load-bearing step reduces to its own inputs. Accordingly, no specific circular step is established; the score reflects only the minor, non-load-bearing self-citation pattern.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 1 invented entities

The framework rests on interpretive premises rather than fitted quantities: no free parameters (nothing is fitted), four axioms about the exhaustiveness and representativeness of the design space, and one set of invented-but-unvalidated diagnostics. The most fragile entries are axioms 1 and 2: if the decomposition is incomplete or the citation graph is unrepresentative, the “unifying” claim weakens.

axioms (4)
  • domain assumption The four-component decomposition (corruption operator, denoiser parameterization, training objective, sampler) is a complete description of any discrete diffusion model.
    Stated in §5.6 (“every discrete diffusion model instantiates the same four-component structure”) and instantiated in Table 5, without a proof of exhaustiveness.
  • domain assumption The citation-graph selection around D3PM, multinomial diffusion, MDLM/MD4, SEDD, and discrete flow matching is representative of the field through early 2026.
    §2 “Scope and selection” says the literature was assembled by tracking the citation graph around these foundations and notes “we do not claim exhaustive coverage.”
  • ad hoc to paper Tokenization is the primary causal axis shaping corruption, denoising difficulty, controllability, and cost of discrete diffusion.
    Central thesis of §4.1; argued qualitatively through four named effects rather than derived or measured.
  • ad hoc to paper The Section 4.5 diagnostics (reconstruction–generation gap, denoising loss curve, schedule sensitivity, neighborhood consistency) are informative for tokenizer–diffusion co-design.
    The paper itself states these are “largely our own proposals” and “we have not validated the proposed diagnostics with new experiments” (§4.5).
invented entities (1)
  • Diffusion-facing tokenizer diagnostics (reconstruction–generation gap, denoising loss curve, schedule sensitivity, neighborhood consistency) no independent evidence
    purpose: Proposed metrics to judge whether a tokenizer supports diffusion learning without training a full generative model
    Introduced in §4.5 as “proposed or synthesized... not validated” by the authors’ own admission; no data show they predict generation quality.

pith-pipeline@v1.3.0-alltime-deepseek · 43889 in / 19330 out tokens · 178433 ms · 2026-08-02T05:11:43.932525+00:00 · methodology

0 comments
read the original abstract

Discrete denoising diffusion models (DDMs) have recently emerged as a compelling alternative to autoregressive (AR) modeling for discrete data, offering parallel generation and iterative global refinement capabilities. Unlike continuous diffusion, where the state space is fixed, DDMs are fundamentally shaped by how the discrete state space is constructed: the tokenization scheme, the vocabulary topology, and domain-specific structural alphabets. This work introduces a unified conceptual framework that views discrete diffusion models through the construction of the underlying discrete state space. Within this framework, existing formulations, including transition-matrix, masking/absorbing-state, and score/ratio-based approaches, emerge as different instantiations of a common design space. The framework further exposes common design trade-offs across training objectives, inference algorithms, scaling behavior, systems optimization, and evaluation protocols, suggesting several promising directions for future research.

Figures

Figures reproduced from arXiv: 2607.13431 by Bowei He, Changjiang Han, Fengran Mo, Haochen Liu, Haolun Wu, Jiaxin Huang, Jikun Kang, Linfeng Du, Philip S. Yu, Rui Song, Weien Li, Weixu Zhang, Xiangyu Kong, Xiuyuan Hu, Xue Liu, Yankai Chen, Ye Yuan, Yonghan Yang, Zeyu Li, Zichen Zhao, Zipeng Sun, Zixuan Dong.

Figure 1
Figure 1. Figure 1: Paper overview. edge attributes, and combinatorial objects such as graph layouts and scheduling solutions (Gruver et al., 2023; Avdeyev et al., 2023; Vignac et al., 2023; Sun & Yang, 2023). We additionally cover an emerging direction, involving agents, planning, and tool use, where discrete diffusion serves as a planner or structured￾output generator for decision-making pipelines, including vision-language… view at source ↗
Figure 2
Figure 2. Figure 2: Discrete forward corruption and learned reverse denoising. Absorbing-state masking removes token [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Token families in discrete diffusion. (a) Semantic text and code tokens are frequency-defined [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Inference as a policy layer over a fixed denoiser. Step schedules, token selection, block structure, [PITH_FULL_IMAGE:figures/full_fig_p032_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

212 extracted references · 61 linked inside Pith

  1. [1]

    Sudhanshu Agrawal, Risheek Garrepalli, Raghavv Goel, Mingu Lee, Christopher Lott, and F. Porikli. Spiffy: Multiplying diffusion llm acceleration via lossless speculative decoding.ArXiv preprint, abs/2509.18085,

  2. [3]

    Multi- conditioned graph diffusion for neural architecture search.Trans

    Rohan Asthana, Joschua Conrad, Youssef Dawoud, Maurits Ortmanns, and Vasileios Belagiannis. Multi- conditioned graph diffusion for neural architecture search.Trans. Mach. Learn. Res., 2024,

  3. [6]

    Forget bit, it is all about token: Towards semantic information theory for llms.ArXiv preprint, abs/2511.01202,

    Bo Bai. Forget bit, it is all about token: Towards semantic information theory for llms.ArXiv preprint, abs/2511.01202,

  4. [7]

    Enabling approximate joint sampling in diffusion lms.ArXiv preprint, abs/2509.22738,

    Parikshit Bansal and Sujay Sanghavi. Enabling approximate joint sampling in diffusion lms.ArXiv preprint, abs/2509.22738,

  5. [8]

    Learning to parallel: Accelerating diffusion large language models via learnable parallel decoding, 2025a

    Wenrui Bao, Zhiben Chen, Dan Xu, and Yuzhang Shang. Learning to parallel: Accelerating diffusion large language models via learnable parallel decoding, 2025a. Wenrui Bao, Zhiben Chen, Dan Xu, and Yuzhang Shang. Learning to parallel: Accelerating diffusion large language models via adaptive parallel decoding.ArXiv preprint, abs/2509.25188, 2025b. Ethan Bar...

  6. [10]

    Simple and critical iterative denoising: A recasting of discrete diffusion in graph generatio

    64 Yoann Boget. Simple and critical iterative denoising: A recasting of discrete diffusion in graph generatio. In Forty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025, volume 267 ofProceedings of Machine Learning Research,

  7. [11]

    Discrete graph auto-encoder.ArXiv preprint, abs/2306.07735,

    Yoann Boget, Magda Gregorová, and Alexandros Kalousis. Discrete graph auto-encoder.ArXiv preprint, abs/2306.07735,

  8. [12]

    Montgomery Bohde, Mrunali Manjrekar, Runzhong Wang, Shuiwang Ji, and Connor W. Coley. Diffms: Diffusion generation of molecules conditioned on mass spectra. InForty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025, volume 267 ofProceedings of Machine Learning Research,

  9. [13]

    ScanDL: A diffusion model for generating synthetic scanpaths on texts

    Lena Bolliger, David Reich, Patrick Haller, Deborah Jakobi, Paul Prasse, and Lena Jäger. ScanDL: A diffusion model for generating synthetic scanpaths on texts. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 15513–15538, Singapore,

  10. [14]

    Breckon, and Chris G

    Sam Bond-Taylor, Peter Hessey, Hiroshi Sasaki, Toby P. Breckon, and Chris G. Willcocks. Unleashing transformers: Parallel token prediction with discrete absorbing diffusion for fast high-resolution image generation from vector-quantized codes. InComputer Vision – ECCV 2022, pp. 170–188,

  11. [15]

    Preference-based alignment of discrete diffusion models.ArXiv preprint, abs/2503.08295,

    Umberto Borso, Davide Paglieri, Jude Wells, and Tim Rocktäschel. Preference-based alignment of discrete diffusion models.ArXiv preprint, abs/2503.08295,

  12. [17]

    Jaakkola

    Andrew Campbell, Jason Yim, Regina Barzilay, Tom Rainforth, and Tommi S. Jaakkola. Generative flows on discrete state-spaces: Enabling multimodal flows with applications to protein co-design. InForty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024,

  13. [18]

    Self-speculative masked diffusions

    Andrew Campbell, Valentin De Bortoli, Jiaxin Shi, and Arnaud Doucet. Self-speculative masked diffusions. ArXiv preprint, abs/2510.03929,

  14. [19]

    Generating directed graphs with dual attention and asymmetric encoding.ArXiv preprint, abs/2506.16404,

    Alba Carballo-Castro, Manuel Madeira, Yiming Qin, Dorina Thanou, and Pascal Frossard. Generating directed graphs with dual attention and asymmetric encoding.ArXiv preprint, abs/2506.16404,

  15. [20]

    Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T. Freeman. Maskgit: Masked generative image transformer. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pp. 11305–11315,

  16. [21]

    Freeman, Michael Rubinstein, Yuanzhen Li, and Dilip Krishnan

    Huiwen Chang, Han Zhang, Jarred Barber, Aaron Maschinot, José Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Patrick Murphy, William T. Freeman, Michael Rubinstein, Yuanzhen Li, and Dilip Krishnan. Muse: Text-to-image generation via masked generative transformers. InInternational Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA,...

  17. [22]

    Dtom: Decider-guided dynamic token merging for accelerating diffusion mllms.ArXiv preprint, abs/2511.12280,

    Shuochen Chang, Xiaofeng Zhang, Qingyang Liu, and Li Niu. Dtom: Decider-guided dynamic token merging for accelerating diffusion mllms.ArXiv preprint, abs/2511.12280,

  18. [23]

    Krishnan

    65 Chen-Hao Chao, Wei-Fang Sun, Hanwen Liang, Chun-Yi Lee, and Rahul G. Krishnan. Beyond masked and unmasked: Discrete diffusion models via partial masking.ArXiv preprint, abs/2505.18495,

  19. [24]

    Aligning visual foundation encoders to tokenizers for diffusion models.ArXiv preprint, abs/2509.25162, 2025a

    Bowei Chen, Sai Bi, Hao Tan, He Zhang, Tianyuan Zhang, Zhengqi Li, Yuanjun Xiong, Jianming Zhang, and Kai Zhang. Aligning visual foundation encoders to tokenizers for diffusion models.ArXiv preprint, abs/2509.25162, 2025a. Chuangtao Chen, Qinglin Zhao, MengChu Zhou, Dusit Niyato, Zhimin He, and Haozhen Situ. Overcoming dimensional factorization limits in ...

  20. [26]

    Controllable conversation generation with conversation structures via diffusion models

    Jiaao Chen and Diyi Yang. Controllable conversation generation with conversation structures via diffusion models. InFindings of the Association for Computational Linguistics: ACL 2023, pp. 7238–7251, Toronto, Canada,

  21. [27]

    A cheaper and better diffusion language model with soft-masked noise

    Jiaao Chen, Aston Zhang, Mu Li, Alex Smola, and Diyi Yang. A cheaper and better diffusion language model with soft-masked noise. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 4765–4775, Singapore, 2023a. Jialin Chen, Shirley Wu, Abhijit Gupta, and Rex Ying. D4explainer: In-distribution explanations of graph ...

  22. [29]

    Sdar-vl: Stable and efficient block-wise diffusion for vision-language understanding.ArXiv preprint, abs/2512.14068, 2025a

    Shuang Cheng, Yuhua Jiang, Zineng Zhou, Dawei Liu, Wang Tao, Linfeng Zhang, Biqing Qi, and Bowen Zhou. Sdar-vl: Stable and efficient block-wise diffusion for vision-language understanding.ArXiv preprint, abs/2512.14068, 2025a. Zicong Cheng, Guo-Wei Yang, Jia Li, Zhijie Deng, Meng-Hao Guo, and Shi-Min Hu. Draft with diffusion, verify with autoregressive mo...

  23. [30]

    M2d2m: Multi-motion generation from text with discrete diffusion models

    Seunggeun Chi, Hyung-gun Chi, Hengbo Ma, Nakul Agarwal, Faizan Siddiqui, Karthik Ramani, and Kwon- joon Lee. M2d2m: Multi-motion generation from text with discrete diffusion models. InComputer Vision – ECCV 2024, pp. 18–36,

  24. [31]

    Speculative diffusion decoding: Accelerating language generation through diffusion

    67 JacobKChristopher, BrianR.Bartoldson, TalBen-Nun, MichaelCardei, BhavyaKailkhura, andFerdinando Fioretto. Speculative diffusion decoding: Accelerating language generation through diffusion. InProceed- ings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume ...

  25. [32]

    Split gibbs discrete diffusion posterior sampling.ArXiv preprint, abs/2503.01161,

    Wenda Chu, Yang Song, and Yisong Yue. Split gibbs discrete diffusion posterior sampling.ArXiv preprint, abs/2503.01161,

  26. [33]

    Gaus, Toby P

    Abril Corona-Figueroa, Sam Bond-Taylor, Neelanjan Bhowmik, Yona Falinie A. Gaus, Toby P. Breckon, Hubert P. H. Shum, and Chris G. Willcocks. Unaligned 2d to 3d translation with conditional vector- quantized code diffusion using transformers. InIEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023, pp. 14539–14548,

  27. [34]

    Vilad: A large vision language diffusion framework for end-to-end autonomous driving.ArXiv preprint, abs/2508.12603,

    Can Cui, Yupeng Zhou, Juntong Peng, Sung-Yeon Park, Zichong Yang, Prashanth Sankaranarayanan, Jiaru Zhang, Ruqi Zhang, and Ziran Wang. Vilad: A large vision language diffusion framework for end-to-end autonomous driving.ArXiv preprint, abs/2508.12603,

  28. [35]

    Differ: Cate- gorical diffusion for chemical retrosynthesis.ArXiv preprint, abs/2505.23721,

    Sean Current, Ziqi Chen, Daniel Adu-Ampratwum, Xia Ning, and Srinivasan Parthasarathy. Differ: Cate- gorical diffusion for chemical retrosynthesis.ArXiv preprint, abs/2505.23721,

  29. [36]

    Periodic materials generation using text-guided joint diffusion model

    Kishalay Das, Subhojyoti Khastagir, Pawan Goyal, Seung-Cheol Lee, Satadeep Bhattacharjee, and Niloy Ganguly. Periodic materials generation using text-guided joint diffusion model. InThe Thirteenth Inter- national Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025,

  30. [37]

    Discrete diffusion language model for efficient text summarization

    Do Huu Dat, Duc Anh Do, Anh Tuan Luu, and Wray Buntine. Discrete diffusion language model for efficient text summarization. InFindings of the Association for Computational Linguistics: NAACL 2025, pp. 6278–6290, Albuquerque, New Mexico,

  31. [38]

    Alexandre Défossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi

    doi: 10.1126/science.add2187. Alexandre Défossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. High Fidelity Neural Audio Compression, October

  32. [39]

    Uniform discrete diffusion with metric path for video generation.ArXiv preprint, abs/2510.24717,

    Haoge Deng, Ting Pan, Fan Zhang, Yang Liu, Zhuoyan Luo, Yufeng Cui, Wenxuan Wang, Chunhua Shen, Shiguang Shan, Zhaoxiang Zhang, and Xinlong Wang. Uniform discrete diffusion with metric path for video generation.ArXiv preprint, abs/2510.24717,

  33. [40]

    Llada-medv: Exploring large language diffusion models for biomedical image understanding.ArXiv preprint, abs/2508.01617, 2025a

    Xuanzhao Dong, Wenhui Zhu, Xiwen Chen, Zhipeng Wang, Peijie Qiu, Shao Tang, Xin Li, and Yalin Wang. Llada-medv: Exploring large language diffusion models for biomedical image understanding.ArXiv preprint, abs/2508.01617, 2025a. Yihong Dong, Zhaoyu Ma, Xue Jiang, Zhiyuan Fan, Jiaru Qian, Yongming Li, Jian Xiao, Zhi Jin, Rongyu Cao, Binhua Li, Fei Huang, Yo...

  34. [41]

    Planning with diffusion models for target-oriented dialogue systems

    Hanwen Du, Bo Peng, and Xia Ning. Planning with diffusion models for target-oriented dialogue systems. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 20307–20329, Vienna, Austria, 2025a. 68 Yongkun Du, Miaomiao Zhao, S. Fan, Zhineng Chen, Caiyan Jia, and Yu-Gang Jiang. Mdiff4str: Mask...

  35. [42]

    Vector quantized diffusion model based speech bandwidth extension

    Yuan Fang, Jinglin Bai, Jiajie Wang, and Xueliang Zhang. Vector quantized diffusion model based speech bandwidth extension. InICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5,

  36. [43]

    Alessandro Favero, Antonio Sclocchi, and M. Wyart. Bigger isn’t always memorizing: Early stopping over- parameterized diffusion models.ArXiv preprint, abs/2505.16959,

  37. [44]

    Masked diffusion captioning for visual feature learning

    Chao Feng, Zihao Wei, and Andrew Owens. Masked diffusion captioning for visual feature learning. In Findings of the Association for Computational Linguistics: EMNLP 2025, pp. 25247–25263,

  38. [45]

    Learnable sampler distillation for discrete diffusion models

    Feiyang Fu, Tongxian Guo, and Zhaoqiang Liu. Learnable sampler distillation for discrete diffusion models. ArXiv preprint, abs/2509.19962, 2025a. Hengyu Fu, Baihe Huang, Virginia Adams, Charles Wang, Venkat Srinivasan, and Jiantao Jiao. From bits to rounds: Parallel decoding with exploration for diffusion language models.ArXiv preprint, abs/2511.21103, 20...

  39. [46]

    Diffu- COMET: Contextual commonsense knowledge diffusion

    Silin Gao, Mete Ismayilzada, Mengjie Zhao, Hiromi Wakaki, Yuki Mitsufuji, and Antoine Bosselut. Diffu- COMET: Contextual commonsense knowledge diffusion. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 4809–4831, Bangkok, Thailand, 2024a. Zhangyang Gao, Cheng Tan, and Stan Z. Li. Diffs...

  40. [47]

    Zhangyang Gao, Cheng Tan, Jue Wang, Yufei Huang, Lirong Wu, and Stan Z. Li. Foldtoken: Learning protein language via vector quantization and beyond.arXiv preprint arXiv:2403.09673, 2024b. H. F. García, Prem Seetharaman, Rithesh Kumar, and Bryan Pardo. Vampnet: Music generation via masked acoustic token modeling.ArXiv preprint, abs/2307.04686,

  41. [48]

    Itai Gat, Tal Remez, Neta Shaul, Felix Kreuk, Ricky T. Q. Chen, Gabriel Synnaeve, Yossi Adi, and Yaron Lipman. Discrete flow matching. InAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024,

  42. [49]

    Mask-predict: Parallel decoding of conditional masked language models

    Marjan Ghazvininejad, Omer Levy, Yinhan Liu, and Luke Zettlemoyer. Mask-predict: Parallel decoding of conditional masked language models. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 6112–6121, Hong Kong, China,

  43. [50]

    Thibaud Gloaguen, Robin Staab, Nikola Jovanovic, and Martin T. Vechev. Watermarking diffusion language models.ArXiv preprint, abs/2509.24368,

  44. [51]

    Text-guidedmoleculegenerationwithdiffusionlanguage model

    HaisongGong, QiangLiu, ShuWu, andLiangWang. Text-guidedmoleculegenerationwithdiffusionlanguage model. InThirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 2...

  45. [52]

    Diffuseq: Sequence to sequence text generation with diffusion models

    Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and Lingpeng Kong. Diffuseq: Sequence to sequence text generation with diffusion models. InThe Eleventh International Conference on Learning Represen- tations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, 2023a. Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and Lingpeng Kong. DiffuSeq-v2: Bridging discr...

  46. [53]

    Frey, Tim G

    Nate Gruver, Samuel Stanton, Nathan C. Frey, Tim G. J. Rudner, Isidro Hötzel, Julien Lafrance-Vanasse, Arvind Rajpal, Kyunghyun Cho, and Andrew Gordon Wilson. Protein design with guided discrete diffu- sion. InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, L...

  47. [54]

    Vector quantized diffusion model for text-to-image synthesis

    Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, and Baining Guo. Vector quantized diffusion model for text-to-image synthesis. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pp. 10686– 10696,

  48. [55]

    Powers, Weili Nie, Tomas Geffner, Karsten Kreis, Jure Leskovec, Arash Vahdat, and Stefano Ermon

    Siyi Gu, Minkai Xu, Alexander S. Powers, Weili Nie, Tomas Geffner, Karsten Kreis, Jure Leskovec, Arash Vahdat, and Stefano Ermon. Aligning target-aware molecule diffusion models with exact energy optimiza- tion. InAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver,...

  49. [56]

    Hashimoto

    70 Ishaan Gulrajani and Tatsunori B. Hashimoto. Likelihood-based diffusion language models. InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023,

  50. [57]

    Revivingany-subsetautoregressivemodelswithprincipledparallelsampling and speculative decoding.ArXiv preprint, abs/2504.20456,

    GabrielGuoandStefanoErmon. Revivingany-subsetautoregressivemodelswithprincipledparallelsampling and speculative decoding.ArXiv preprint, abs/2504.20456,

  51. [58]

    Training-free guidance beyond differentiability: Scalable path steering with tree search in diffusion and flow models.ArXiv preprint, abs/2502.11420,

    Yingqing Guo, Yukang Yang, Hui Yuan, and Mengdi Wang. Training-free guidance beyond differentiability: Scalable path steering with tree search in diffusion and flow models.ArXiv preprint, abs/2502.11420,

  52. [59]

    Reward-weighted sampling: Enhancing non-autoregressive characteristics in masked diffusion llms

    Daehoon Gwak, Minseo Jung, Junwoo Park, Minho Park, ChaeHun Park, Junha Hyung, and Jaegul Choo. Reward-weighted sampling: Enhancing non-autoregressive characteristics in masked diffusion llms. InPro- ceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 34562–34582,

  53. [60]

    Wattenhofer

    Kilian Haefeli, Karolis Martinkus, Nathanael Perraudin, and R. Wattenhofer. Diffusion models for graphs benefit from discrete state spaces.ArXiv preprint, abs/2210.01549,

  54. [61]

    Discrete diffusion trajectory alignment via stepwise decomposition.ArXiv preprint, abs/2507.04832, 2025a

    Jiaqi Han, Austin Wang, Minkai Xu, Wenda Chu, Meihua Dang, Yisong Yue, and Stefano Ermon. Discrete diffusion trajectory alignment via stepwise decomposition.ArXiv preprint, abs/2507.04832, 2025a. Jun Han, Zixiang Chen, Yongqian Li, Yiwen Kou, Eran Halperin, Robert E. Tillman, and Quanquan Gu. Guided discrete diffusion for electronic health record generati...

  55. [62]

    Discovering mathemat- ical equations with diffusion language model.ArXiv preprint, abs/2509.13136, 2025c

    Xiaoxu Han, Chengzhen Ning, Jinghui Zhong, Fubiao Yang, Yu Wang, and Xin Mu. Discovering mathemat- ical equations with diffusion language model.ArXiv preprint, abs/2509.13136, 2025c. Zhitao Han and Wenhui Huang. Prostate mri super-resolution using discrete residual diffusion model. In 2023 IEEE International Conference on Bioinformatics and Biomedicine (B...

  56. [63]

    Efficient perplexity bound and ratio matching in discrete diffusion language models

    Etrit Haxholli, Yeti Ziya Gurbuz, Ogul Can, and Eli Waxman. Efficient perplexity bound and ratio matching in discrete diffusion language models. InThe Thirteenth International Conference on Learning Represen- tations, ICLR 2025, Singapore, April 24-28, 2025,

  57. [64]

    Distillation of discrete diffusion through dimensional correlations

    Satoshi Hayakawa, Yuhta Takida, Masaaki Imaizumi, Hiromi Wakaki, and Yuki Mitsufuji. Distillation of discrete diffusion through dimensional correlations. InForty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025, volume 267 ofProceedings of Machine Learning Research,

  58. [65]

    Ultrallada: Scaling the context length to 128k for diffusion large language models.ArXiv preprint, abs/2510.10481, 2025a

    Guangxin He, Shen Nie, Fengqi Zhu, Yuankang Zhao, Tianyi Bai, Ran Yan, Jie Fu, Chongxuan Li, and Binhang Yuan. Ultrallada: Scaling the context length to 128k for diffusion large language models.ArXiv preprint, abs/2510.10481, 2025a. Guojun He, Shengyu Zhang, and Tao Jiang. Residual vector quantized diffusion model-based joint source- channel coding for ta...

  59. [66]

    What exactly does guidance do in masked discrete diffusion models

    Ye He, Kevin Rojas, and Molei Tao. What exactly does guidance do in masked discrete diffusion models. ArXiv preprint, abs/2506.10971, 2025d. Yi He, Ailun Wang, Zhi Wang, Yu Liu, Xingyuan Xu, and Wen Yan. Generative molecule evolution using 3d pharmacophore for efficient structure-based drug design.ArXiv preprint, abs/2507.20130, 2025e. Zhengfu He, Tianxia...

  60. [67]

    Shao, Chaofan Gan, Shijie Li, Zuxuan Wu, and Weiyao Lin

    Zhihao He, Tieyuan Chen, Kangyu Wang, Ziran Qin, Y. Shao, Chaofan Gan, Shijie Li, Zuxuan Wu, and Weiyao Lin. Vidlada: Bidirectional diffusion large language models for efficient video understanding. ArXiv preprint, abs/2601.17868, 2026b. Michael Hersche, Samuel Moor-Smith, Thomas Hofmann, and Abbas Rahimi. Soft-masked diffusion language models.ArXiv prepr...

  61. [69]

    Unsu- pervised training of diffusion models for feasible solution generation in neural combinatorial optimization, 2024a

    Seong-Hyun Hong, Hyun sung Kim, Zian Jang, Deunsol Yoon, Hyungseok Song, and Byung-Jun Lee. Unsu- pervised training of diffusion models for feasible solution generation in neural combinatorial optimization, 2024a. Shibo Hong, Xuhong Zhang, Tianyu Du, Sheng Cheng, Xun Wang, and Jianwei Yin. Cons2plan: Vector floorplan generation from various conditions via...

  62. [70]

    Gritsenko, Jasmijn Bastings, Ben Poole, Rianne van den Berg, and Tim Salimans

    Emiel Hoogeboom, Alexey A. Gritsenko, Jasmijn Bastings, Ben Poole, Rianne van den Berg, and Tim Salimans. Autoregressive diffusion models. InThe Tenth International Conference on Learning Represen- tations, ICLR 2022, Virtual Event, April 25-29, 2022,

  63. [71]

    Bad: Bidirectional auto-regressive diffusion for text-to-motion generation

    72 Seyed Rohollah Hosseyni, Ali Ahmad Rahmani, Seyed Jamal Seyedmohammadi, Sanaz Seyedin, and Arash Mohammadi. Bad: Bidirectional auto-regressive diffusion for text-to-motion generation. InICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5,

  64. [72]

    Suganthan

    Minghui Hu, Yujie Wang, Tat-Jen Cham, Jianfei Yang, and Ponnuthurai N. Suganthan. Global context with discrete diffusion in vector quantised modelling for image generation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pp. 11492–11501,

  65. [73]

    Suganthan

    Minghui Hu, Chuanxia Zheng, Zuopeng Yang, Tat-Jen Cham, Heliang Zheng, Chaoyue Wang, Dacheng Tao, and Ponnuthurai N. Suganthan. Unified discrete diffusion for simultaneous vision-language generation. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023,

  66. [74]

    Mixed diffusion for 3d indoor scene synthesis.ArXiv preprint, abs/2405.21066,

    Siyi Hu, Diego Martín Arroyo, Stephanie Debats, Fabian Manhardt, Luca Carlone, and Federico Tombari. Mixed diffusion for 3d indoor scene synthesis.ArXiv preprint, abs/2405.21066,

  67. [75]

    Abdelfattah, Jae sun Seo, Zhiru Zhang, and Udit Gupta

    Zhanqiu Hu, Jian Meng, Yash Akhauri, Mohamed S. Abdelfattah, Jae sun Seo, Zhiru Zhang, and Udit Gupta. Flashdlm: Accelerating diffusion language model inference via efficient kv caching and guided diffusion.ArXiv preprint, abs/2505.21467,

  68. [76]

    Tuning the implicit regularizer of masked diffusion language models: Enhancing generalization via insights from -parity.ArXiv preprint, abs/2601.22450,

    Jianhao Huang and Baharan Mirzasoleiman. Tuning the implicit regularizer of masked diffusion language models: Enhancing generalization via insights from -parity.ArXiv preprint, abs/2601.22450,

  69. [77]

    Jiaxing Huang, Kaiwen Cui, Dayan Guan, Aoran Xiao, Fangneng Zhan, Shijian Lu, Shengcai Liao, and Eric P. Xing. Masked generative adversarial networks are data-efficient generation learners. InAdvances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 2...

  70. [78]

    Diffged: Computing graph edit distance via diffusion-based graph matching.ArXiv preprint, abs/2503.18245, 2025a

    Wei Huang, Hanchen Wang, Dong Wen, Wenjie Zhang, Ying Zhang, and Xuemin Lin. Diffged: Computing graph edit distance via diffusion-based graph matching.ArXiv preprint, abs/2503.18245, 2025a. Yiming Huang and Tolga Birdal. Hog-diff: Higher-order guided diffusion for graph generation.ArXiv preprint, abs/2502.04308,

  71. [79]

    Reinforcing the diffusion chain of lateral thought with diffusion language models.ArXiv preprint, abs/2505.10446, 2025b

    Zemin Huang, Zhiyang Chen, Zijun Wang, Tiancheng Li, and Guo-Jun Qi. Reinforcing the diffusion chain of lateral thought with diffusion language models.ArXiv preprint, abs/2505.10446, 2025b. Zemin Huang, Yuhang Wang, Zhiyang Chen, and Guo-Jun Qi. Don’t settle too early: Self-reflective remask- ing for diffusion language models.ArXiv preprint, abs/2509.2365...

  72. [80]

    Layoutdm: Discrete diffusion model for controllable layout generation

    Naoto Inoue, Kotaro Kikuchi, Edgar Simo-Serra, Mayu Otani, and Kota Yamaguchi. Layoutdm: Discrete diffusion model for controllable layout generation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pp. 10167–10176,

  73. [81]

    Cheng, Guy Van den Broeck, Aditya Grover, Suvinay Subramanian, and Michael Carbin

    Daniel Israel, Tian Jin, Ellie Y. Cheng, Guy Van den Broeck, Aditya Grover, Suvinay Subramanian, and Michael Carbin. Planned diffusion.ArXiv preprint, abs/2510.18087,

  74. [82]

    Layout-corrector: Alleviating layout sticking phenomenon in discrete diffusion model

    Shoma Iwai, Atsuki Osanai, Shunsuke Kitada, and Shinichiro Omachi. Layout-corrector: Alleviating layout sticking phenomenon in discrete diffusion model. InComputer Vision – ECCV 2024, pp. 92–110,

  75. [83]

    Olausson, Louis Béthune, Pierre Ablin, Michael Kirchhof, João Monteiro, Victor Turrisi, Jason Ramapuram, and Marco Cuturi

    Metod Jazbec, Theo X. Olausson, Louis Béthune, Pierre Ablin, Michael Kirchhof, João Monteiro, Victor Turrisi, Jason Ramapuram, and Marco Cuturi. Learning unmasking policies for diffusion language models. ArXiv preprint, abs/2512.09106,

  76. [84]

    A2d: Any-order, any-step safety alignment for diffusion language models.ArXiv preprint, abs/2509.23286,

    Wonje Jeung, Sangyeon Yoon, Yoonjun Cho, Dongjae Jeon, Sangwoo Shin, Hyesoo Hong, and Albert No. A2d: Any-order, any-step safety alignment for diffusion language models.ArXiv preprint, abs/2509.23286,

  77. [85]

    From denoising to refining: A corrective framework for vision-language diffusion model.ArXiv preprint, abs/2510.19871,

    Yatai Ji, Teng Wang, Yuying Ge, Zhiheng Liu, Sidi Yang, Ying Shan, and Ping Luo. From denoising to refining: A corrective framework for vision-language diffusion model.ArXiv preprint, abs/2510.19871,

  78. [86]

    Bringing stability to diffusion: Decomposing and reducing variance of training masked diffusion models.ArXiv preprint, abs/2511.18159,

    Mengni Jia, Mengyu Zhou, Yihao Liu, Xiaoxi Jiang, and Guanjun Jiang. Bringing stability to diffusion: Decomposing and reducing variance of training masked diffusion models.ArXiv preprint, abs/2511.18159,

  79. [87]

    Diffusion language models are provably optimal parallel samplers.ArXiv preprint, abs/2512.25014, 2025a

    Haozhe Jiang, Nika Haghtalab, and Lijie Chen. Diffusion language models are provably optimal parallel samplers.ArXiv preprint, abs/2512.25014, 2025a. Yuchu Jiang, Yue Cai, Xiangzhong Luo, Jiale Fu, Jiarui Wang, Chonghan Liu, and Xu Yang. dcache: Accelerating diffusion-based llms via dual adaptive caching.ArXiv preprint, abs/2509.23094, 2025b. Jaehyeong Jo...

  80. [88]

    Loopholing discrete diffu- sion: Deterministic bypass of the sampling wall.ArXiv preprint, abs/2510.19304,

    Mingyu Jo, Jaesik Yoon, Justin Deschenaux, Caglar Gulcehre, and Sungjin Ahn. Loopholing discrete diffu- sion: Deterministic bypass of the sampling wall.ArXiv preprint, abs/2510.19304,

Showing first 80 references.