REVIEW 2 major objections 3 minor 212 references
This survey claims that every discrete diffusion model—transition-matrix, masking, or score/ratio—instantiates the same four-component structure and that tokenization is the primary design axis.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-02 05:11 UTC pith:TTRGROUM
load-bearing objection Useful map of discrete diffusion, but the 'every model fits four components' claim is narrower than its own scope allows. the 2 major comments →
Discrete Diffusion Models: A Unified Framework from Tokenization to Generation
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper's central claim is that the apparent diversity of discrete diffusion models conceals a common design space. It decomposes every model into four components: (1) a corruption operator (a transition matrix or rate matrix that specifies how clean tokens become noisy), (2) a denoiser parameterization (predicting the clean token, logits, or probability ratios), (3) a training objective (variational, simplified denoising, or score/ratio matching), and (4) a sampler (ancestral, confidence-based remasking, blockwise, guided, or accelerated). Major formulation families—general transition-matrix models, absorbing/masking diffusion, continuous-time Markov chains, and score/ratio-based methods—
What carries the argument
The load-bearing object is the four-component decomposition: (1) corruption operator (transition matrix or rate matrix defining the forward noising), (2) denoiser parameterization (what the network predicts: clean-token distribution, logits, or probability ratios), (3) training objective (ELBO, simplified denoising, or score/ratio matching), and (4) sampler (ancestral, remasking, blockwise, guided, accelerated). The paper pairs this with a tokenization lens: the discrete state space—subword vocabularies, quantized codebooks, or natural scientific alphabets—determines the corruption topology and denoising difficulty. The decomposition does the work of making different formulations interconver
Load-bearing premise
The load-bearing premise is that the sample of papers surveyed, gathered by following citations from a handful of foundational discrete-diffusion formulations, fairly represents the whole field; under-representing one branch, such as embed-then-diffuse or non-Markovian corruptions, could make the shared structure look more universal than it is.
What would settle it
Find any model that is widely accepted as discrete diffusion but cannot be placed in the four components without forcing—for example, a forward process that is not a per-position categorical Markov chain, or an objective that is neither variational, denoising, nor score/ratio matching. A more empirical probe: train the paper's predicted-valid 'off-diagonal' pairings (e.g., absorbing corruption with a score-entropy objective) and show they fail to produce competitive likelihoods or samples at scale, which would falsify the claim that components are independently selectable.
If this is right
- Model families that look different (masking, substitution, score/ratio) differ mainly in corruption and parameterization; their training objectives and samplers can often be swapped.
- Masking vs. substitution vs. structured corruption is orthogonal to predict-x0 vs. predict-ratio parameterization, so combinations not yet tried are legitimate candidates for new models.
- Tokenization is a primary design axis: vocabulary granularity, codebook topology, and natural alphabets shape denoising difficulty, controllability, and cost, so tokenizer choice should be reported alongside model choice.
- Masked diffusion's ELBO collapses to a reweighted masked-language-modeling objective, which explains why large-scale diffusion language models train like BERT-style models and why time conditioning is partly redundant.
- Likelihood-based training and simplified denoising objectives occupy different points on a compute-vs-bound trade-off; the field has not settled on one as universally best.
Where Pith is reading between the lines
- If tokenization is the primary axis, then pushing tokenizer development (e.g., semantic alignment of codebooks) may yield larger quality gains per unit effort than pushing denoiser architecture—a hypothesis that could be tested by ablating tokenizer changes with a fixed denoiser.
- The off-diagonal cells the paper highlights suggest a concrete research program: systematically crossing absorbing corruption with ratio/score parameterization, or structured substitution with predict-x0, and measuring which pairings scale; the survey does not run these experiments.
- The 'common design space' is built from a particular citation graph; if proteins, graphs, or media tokenization had been represented differently, the centrality of tokenization as an axis might be less pronounced—so the framework's strongest test is whether it survives application to a model family outside the sampled graph.
- Framing inference as a policy layer implies that speedups are often sampler-side gains, but the survey's own caveats suggest reported 'fewer steps' results only matter if they translate into wall-clock latency; a fair comparison should standardize hardware and batch.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper is a narrative survey of discrete denoising diffusion models (DDMs), organized around the thesis that the construction of the discrete state space—'tokenization'—is the primary design axis. It proposes a four-component decomposition (corruption operator, denoiser parameterization, training objective, sampler) and argues that transition-matrix, masking/absorbing, score/ratio, and flow-matching formulations are instantiations of this common structure. The paper covers tokenization across text, code, quantized media, and scientific alphabets; systematizes training and inference; and discusses scaling, systems, applications, and evaluation. It also introduces several proposed tokenizer diagnostics and releases a companion repository.
Significance. If the unification claim is accepted, the paper provides a useful shared vocabulary and design checklist for a rapidly growing field. Its strengths include unusually explicit honesty about evidence quality: it flags that LLaDA's scale competitiveness is 'self-reported... not independently audited' (§8.2), that Mercury's throughput is 'reported by the originating work rather than independently reproduced' (§8.4), and that the proposed diagnostics in §4.5 are 'synthesized / not standardized' and not validated with new experiments. The cross-domain coverage—text, code, media, proteins, genomics, molecules, planning—is broad, and Table 5 is a practical mapping. The companion repository is a valuable resource. The main weakness is that the paper's most general claim ('every discrete diffusion model...') exceeds the evidence base it explicitly describes.
major comments (2)
- [§5.6; §2] The paper's central claim, stated in §5.6 as 'every discrete diffusion model instantiates the same four-component structure,' is stronger than the evidence assembled in §2. The 'Scope and selection' paragraph describes a narrative snowball sample around five roots, notes 'we do not claim exhaustive coverage,' and uses inclusion criterion (i) already expressed in the corruption/parameterization/objective/sampler vocabulary. The corpus therefore cannot support a universal quantifier. Boundary cases sharpen the problem: embed-then-diffuse methods are explicitly excluded in §1, and non-Markov forward processes (e.g., Discrete Non-Markov Diffusion Models in §7.1) are not shown to be representable as a Q_t/R_t corruption operator. I recommend narrowing the claim to 'all surveyed formulations' throughout, or adding a systematic independent census and a treatment of non-Markov processes.
- [§6.1, Eq. (16)] The sign convention in Eq. (16) is internally inconsistent as written. The text states that the weight α'_t/(1−α_t) is negative and that 'L∞ itself [is] negative (it is an expected log-likelihood).' But since α'_t<0 and the expectation of δ_{xt,m} x0^⊤ log μθ(xt,t) is ≤0, the integrand is nonnegative, making L∞ nonnegative. The subsequent claim that 'the ELBO reduces to −L∞' adds to the confusion about which quantity is minimized/maximized. Please reconcile Eq. (16) with the convention in Shi et al. (2024b) and state explicitly whether L∞ is the negative ELBO or the ELBO itself.
minor comments (3)
- [§4.5] The tokenizer diagnostics (reconstruction–generation gap, denoising loss curve, schedule sensitivity) are listed in the contributions as introduced diagnostics, but the section honestly labels them 'synthesized / not standardized' and 'we have not validated the proposed diagnostics with new experiments.' This caveat should also appear in the abstract or contributions to avoid overclaiming.
- [§6.6; §8.4] There are several missing spaces in citations, e.g., 'SeveriRissanen(2023)' and 'MercuryKhanna et al.' Please run a formatting pass over inline citations.
- [§5.2] The statement that the cosine schedule is 'Fisher-Rao optimal for masked discrete diffusion' is strong and relies on a single reference. A one-sentence explanation of the optimality criterion and its assumptions would help readers assess the claim.
Circularity Check
No significant circularity: the survey’s four-component taxonomy is an organizing framework with independent analytic content; the main caveat is a scope/representativeness limitation in §2, not a circular derivation.
full rationale
This is a narrative survey rather than a derivation chain. The four-component decomposition (corruption operator, denoiser parameterization, training objective, sampler) is presented as an organizing taxonomy, and the mathematical equivalences it relies on are derived from stated assumptions rather than from the paper’s conclusions: §5.5 shows predict-x0 and ratio parameterizations are interconvertible via Eq. 7 and Eq. 11, and §6.1 derives the discrete ELBO from the forward Markov chain. There are no fitted parameters and no quantities called predictions that reduce to fitted values. The self-citations (e.g., Ye et al. 2025a,b,c,d; 2024a) are used as literature references for diffusion’s planning/refinement motivation and for Dream 7B scaling evidence; they are not load-bearing for the framework’s internal structure. The main caveat is §2’s “Scope and selection”: the survey was assembled by snowballing around five foundational formulations and explicitly “do[es] not claim exhaustive coverage,” and inclusion criterion (i) is already phrased in the four-component vocabulary (“distinct corruption/parameterization/objective/sampler choice”). This weakens the §5.6 universal quantifier (“every discrete diffusion model instantiates the same four-component structure”) as an empirical generalization, but it is a representativeness/scope limitation rather than a circular derivation: the framework’s analytic content, such as the interconvertibility argument, does not depend on the sample. The paper also flags its own limitations honestly (§4.5 diagnostics “not validated”; §8.2/§8.4 self-reported numbers “not independently audited”), which further supports the conclusion that no load-bearing step reduces to its own inputs. Accordingly, no specific circular step is established; the score reflects only the minor, non-load-bearing self-citation pattern.
Axiom & Free-Parameter Ledger
axioms (4)
- domain assumption The four-component decomposition (corruption operator, denoiser parameterization, training objective, sampler) is a complete description of any discrete diffusion model.
- domain assumption The citation-graph selection around D3PM, multinomial diffusion, MDLM/MD4, SEDD, and discrete flow matching is representative of the field through early 2026.
- ad hoc to paper Tokenization is the primary causal axis shaping corruption, denoising difficulty, controllability, and cost of discrete diffusion.
- ad hoc to paper The Section 4.5 diagnostics (reconstruction–generation gap, denoising loss curve, schedule sensitivity, neighborhood consistency) are informative for tokenizer–diffusion co-design.
invented entities (1)
-
Diffusion-facing tokenizer diagnostics (reconstruction–generation gap, denoising loss curve, schedule sensitivity, neighborhood consistency)
no independent evidence
read the original abstract
Discrete denoising diffusion models (DDMs) have recently emerged as a compelling alternative to autoregressive (AR) modeling for discrete data, offering parallel generation and iterative global refinement capabilities. Unlike continuous diffusion, where the state space is fixed, DDMs are fundamentally shaped by how the discrete state space is constructed: the tokenization scheme, the vocabulary topology, and domain-specific structural alphabets. This work introduces a unified conceptual framework that views discrete diffusion models through the construction of the underlying discrete state space. Within this framework, existing formulations, including transition-matrix, masking/absorbing-state, and score/ratio-based approaches, emerge as different instantiations of a common design space. The framework further exposes common design trade-offs across training objectives, inference algorithms, scaling behavior, systems optimization, and evaluation protocols, suggesting several promising directions for future research.
Figures
Reference graph
Works this paper leans on
-
[1]
Sudhanshu Agrawal, Risheek Garrepalli, Raghavv Goel, Mingu Lee, Christopher Lott, and F. Porikli. Spiffy: Multiplying diffusion llm acceleration via lossless speculative decoding.ArXiv preprint, abs/2509.18085,
-
[3]
Multi- conditioned graph diffusion for neural architecture search.Trans
Rohan Asthana, Joschua Conrad, Youssef Dawoud, Maurits Ortmanns, and Vasileios Belagiannis. Multi- conditioned graph diffusion for neural architecture search.Trans. Mach. Learn. Res., 2024,
2024
-
[6]
Bo Bai. Forget bit, it is all about token: Towards semantic information theory for llms.ArXiv preprint, abs/2511.01202,
-
[7]
Enabling approximate joint sampling in diffusion lms.ArXiv preprint, abs/2509.22738,
Parikshit Bansal and Sujay Sanghavi. Enabling approximate joint sampling in diffusion lms.ArXiv preprint, abs/2509.22738,
-
[8]
Wenrui Bao, Zhiben Chen, Dan Xu, and Yuzhang Shang. Learning to parallel: Accelerating diffusion large language models via learnable parallel decoding, 2025a. Wenrui Bao, Zhiben Chen, Dan Xu, and Yuzhang Shang. Learning to parallel: Accelerating diffusion large language models via adaptive parallel decoding.ArXiv preprint, abs/2509.25188, 2025b. Ethan Bar...
-
[10]
Simple and critical iterative denoising: A recasting of discrete diffusion in graph generatio
64 Yoann Boget. Simple and critical iterative denoising: A recasting of discrete diffusion in graph generatio. In Forty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025, volume 267 ofProceedings of Machine Learning Research,
2025
-
[11]
Discrete graph auto-encoder.ArXiv preprint, abs/2306.07735,
Yoann Boget, Magda Gregorová, and Alexandros Kalousis. Discrete graph auto-encoder.ArXiv preprint, abs/2306.07735,
-
[12]
Montgomery Bohde, Mrunali Manjrekar, Runzhong Wang, Shuiwang Ji, and Connor W. Coley. Diffms: Diffusion generation of molecules conditioned on mass spectra. InForty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025, volume 267 ofProceedings of Machine Learning Research,
2025
-
[13]
ScanDL: A diffusion model for generating synthetic scanpaths on texts
Lena Bolliger, David Reich, Patrick Haller, Deborah Jakobi, Paul Prasse, and Lena Jäger. ScanDL: A diffusion model for generating synthetic scanpaths on texts. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 15513–15538, Singapore,
2023
-
[14]
Breckon, and Chris G
Sam Bond-Taylor, Peter Hessey, Hiroshi Sasaki, Toby P. Breckon, and Chris G. Willcocks. Unleashing transformers: Parallel token prediction with discrete absorbing diffusion for fast high-resolution image generation from vector-quantized codes. InComputer Vision – ECCV 2022, pp. 170–188,
2022
-
[15]
Preference-based alignment of discrete diffusion models.ArXiv preprint, abs/2503.08295,
Umberto Borso, Davide Paglieri, Jude Wells, and Tim Rocktäschel. Preference-based alignment of discrete diffusion models.ArXiv preprint, abs/2503.08295,
-
[17]
Jaakkola
Andrew Campbell, Jason Yim, Regina Barzilay, Tom Rainforth, and Tommi S. Jaakkola. Generative flows on discrete state-spaces: Enabling multimodal flows with applications to protein co-design. InForty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024,
2024
-
[18]
Self-speculative masked diffusions
Andrew Campbell, Valentin De Bortoli, Jiaxin Shi, and Arnaud Doucet. Self-speculative masked diffusions. ArXiv preprint, abs/2510.03929,
-
[19]
Alba Carballo-Castro, Manuel Madeira, Yiming Qin, Dorina Thanou, and Pascal Frossard. Generating directed graphs with dual attention and asymmetric encoding.ArXiv preprint, abs/2506.16404,
-
[20]
Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T. Freeman. Maskgit: Masked generative image transformer. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pp. 11305–11315,
2022
-
[21]
Freeman, Michael Rubinstein, Yuanzhen Li, and Dilip Krishnan
Huiwen Chang, Han Zhang, Jarred Barber, Aaron Maschinot, José Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Patrick Murphy, William T. Freeman, Michael Rubinstein, Yuanzhen Li, and Dilip Krishnan. Muse: Text-to-image generation via masked generative transformers. InInternational Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA,...
2023
-
[22]
Shuochen Chang, Xiaofeng Zhang, Qingyang Liu, and Li Niu. Dtom: Decider-guided dynamic token merging for accelerating diffusion mllms.ArXiv preprint, abs/2511.12280,
- [23]
-
[24]
Bowei Chen, Sai Bi, Hao Tan, He Zhang, Tianyuan Zhang, Zhengqi Li, Yuanjun Xiong, Jianming Zhang, and Kai Zhang. Aligning visual foundation encoders to tokenizers for diffusion models.ArXiv preprint, abs/2509.25162, 2025a. Chuangtao Chen, Qinglin Zhao, MengChu Zhou, Dusit Niyato, Zhimin He, and Haozhen Situ. Overcoming dimensional factorization limits in ...
-
[26]
Controllable conversation generation with conversation structures via diffusion models
Jiaao Chen and Diyi Yang. Controllable conversation generation with conversation structures via diffusion models. InFindings of the Association for Computational Linguistics: ACL 2023, pp. 7238–7251, Toronto, Canada,
2023
-
[27]
A cheaper and better diffusion language model with soft-masked noise
Jiaao Chen, Aston Zhang, Mu Li, Alex Smola, and Diyi Yang. A cheaper and better diffusion language model with soft-masked noise. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 4765–4775, Singapore, 2023a. Jialin Chen, Shirley Wu, Abhijit Gupta, and Rex Ying. D4explainer: In-distribution explanations of graph ...
arXiv 2023
-
[29]
Shuang Cheng, Yuhua Jiang, Zineng Zhou, Dawei Liu, Wang Tao, Linfeng Zhang, Biqing Qi, and Bowen Zhou. Sdar-vl: Stable and efficient block-wise diffusion for vision-language understanding.ArXiv preprint, abs/2512.14068, 2025a. Zicong Cheng, Guo-Wei Yang, Jia Li, Zhijie Deng, Meng-Hao Guo, and Shi-Min Hu. Draft with diffusion, verify with autoregressive mo...
arXiv 2025
-
[30]
M2d2m: Multi-motion generation from text with discrete diffusion models
Seunggeun Chi, Hyung-gun Chi, Hengbo Ma, Nakul Agarwal, Faizan Siddiqui, Karthik Ramani, and Kwon- joon Lee. M2d2m: Multi-motion generation from text with discrete diffusion models. InComputer Vision – ECCV 2024, pp. 18–36,
2024
-
[31]
Speculative diffusion decoding: Accelerating language generation through diffusion
67 JacobKChristopher, BrianR.Bartoldson, TalBen-Nun, MichaelCardei, BhavyaKailkhura, andFerdinando Fioretto. Speculative diffusion decoding: Accelerating language generation through diffusion. InProceed- ings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume ...
2025
-
[32]
Split gibbs discrete diffusion posterior sampling.ArXiv preprint, abs/2503.01161,
Wenda Chu, Yang Song, and Yisong Yue. Split gibbs discrete diffusion posterior sampling.ArXiv preprint, abs/2503.01161,
-
[33]
Gaus, Toby P
Abril Corona-Figueroa, Sam Bond-Taylor, Neelanjan Bhowmik, Yona Falinie A. Gaus, Toby P. Breckon, Hubert P. H. Shum, and Chris G. Willcocks. Unaligned 2d to 3d translation with conditional vector- quantized code diffusion using transformers. InIEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023, pp. 14539–14548,
2023
-
[34]
Can Cui, Yupeng Zhou, Juntong Peng, Sung-Yeon Park, Zichong Yang, Prashanth Sankaranarayanan, Jiaru Zhang, Ruqi Zhang, and Ziran Wang. Vilad: A large vision language diffusion framework for end-to-end autonomous driving.ArXiv preprint, abs/2508.12603,
-
[35]
Differ: Cate- gorical diffusion for chemical retrosynthesis.ArXiv preprint, abs/2505.23721,
Sean Current, Ziqi Chen, Daniel Adu-Ampratwum, Xia Ning, and Srinivasan Parthasarathy. Differ: Cate- gorical diffusion for chemical retrosynthesis.ArXiv preprint, abs/2505.23721,
-
[36]
Periodic materials generation using text-guided joint diffusion model
Kishalay Das, Subhojyoti Khastagir, Pawan Goyal, Seung-Cheol Lee, Satadeep Bhattacharjee, and Niloy Ganguly. Periodic materials generation using text-guided joint diffusion model. InThe Thirteenth Inter- national Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025,
2025
-
[37]
Discrete diffusion language model for efficient text summarization
Do Huu Dat, Duc Anh Do, Anh Tuan Luu, and Wray Buntine. Discrete diffusion language model for efficient text summarization. InFindings of the Association for Computational Linguistics: NAACL 2025, pp. 6278–6290, Albuquerque, New Mexico,
2025
-
[38]
Alexandre Défossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi
doi: 10.1126/science.add2187. Alexandre Défossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. High Fidelity Neural Audio Compression, October
-
[39]
Uniform discrete diffusion with metric path for video generation.ArXiv preprint, abs/2510.24717,
Haoge Deng, Ting Pan, Fan Zhang, Yang Liu, Zhuoyan Luo, Yufeng Cui, Wenxuan Wang, Chunhua Shen, Shiguang Shan, Zhaoxiang Zhang, and Xinlong Wang. Uniform discrete diffusion with metric path for video generation.ArXiv preprint, abs/2510.24717,
-
[40]
Xuanzhao Dong, Wenhui Zhu, Xiwen Chen, Zhipeng Wang, Peijie Qiu, Shao Tang, Xin Li, and Yalin Wang. Llada-medv: Exploring large language diffusion models for biomedical image understanding.ArXiv preprint, abs/2508.01617, 2025a. Yihong Dong, Zhaoyu Ma, Xue Jiang, Zhiyuan Fan, Jiaru Qian, Yongming Li, Jian Xiao, Zhi Jin, Rongyu Cao, Binhua Li, Fei Huang, Yo...
-
[41]
Planning with diffusion models for target-oriented dialogue systems
Hanwen Du, Bo Peng, and Xia Ning. Planning with diffusion models for target-oriented dialogue systems. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 20307–20329, Vienna, Austria, 2025a. 68 Yongkun Du, Miaomiao Zhao, S. Fan, Zhineng Chen, Caiyan Jia, and Yu-Gang Jiang. Mdiff4str: Mask...
-
[42]
Vector quantized diffusion model based speech bandwidth extension
Yuan Fang, Jinglin Bai, Jiajie Wang, and Xueliang Zhang. Vector quantized diffusion model based speech bandwidth extension. InICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5,
2025
-
[43]
Alessandro Favero, Antonio Sclocchi, and M. Wyart. Bigger isn’t always memorizing: Early stopping over- parameterized diffusion models.ArXiv preprint, abs/2505.16959,
-
[44]
Masked diffusion captioning for visual feature learning
Chao Feng, Zihao Wei, and Andrew Owens. Masked diffusion captioning for visual feature learning. In Findings of the Association for Computational Linguistics: EMNLP 2025, pp. 25247–25263,
2025
-
[45]
Learnable sampler distillation for discrete diffusion models
Feiyang Fu, Tongxian Guo, and Zhaoqiang Liu. Learnable sampler distillation for discrete diffusion models. ArXiv preprint, abs/2509.19962, 2025a. Hengyu Fu, Baihe Huang, Virginia Adams, Charles Wang, Venkat Srinivasan, and Jiantao Jiao. From bits to rounds: Parallel decoding with exploration for diffusion language models.ArXiv preprint, abs/2511.21103, 20...
-
[46]
Diffu- COMET: Contextual commonsense knowledge diffusion
Silin Gao, Mete Ismayilzada, Mengjie Zhao, Hiromi Wakaki, Yuki Mitsufuji, and Antoine Bosselut. Diffu- COMET: Contextual commonsense knowledge diffusion. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 4809–4831, Bangkok, Thailand, 2024a. Zhangyang Gao, Cheng Tan, and Stan Z. Li. Diffs...
-
[47]
Zhangyang Gao, Cheng Tan, Jue Wang, Yufei Huang, Lirong Wu, and Stan Z. Li. Foldtoken: Learning protein language via vector quantization and beyond.arXiv preprint arXiv:2403.09673, 2024b. H. F. García, Prem Seetharaman, Rithesh Kumar, and Bryan Pardo. Vampnet: Music generation via masked acoustic token modeling.ArXiv preprint, abs/2307.04686,
-
[48]
Itai Gat, Tal Remez, Neta Shaul, Felix Kreuk, Ricky T. Q. Chen, Gabriel Synnaeve, Yossi Adi, and Yaron Lipman. Discrete flow matching. InAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024,
2024
-
[49]
Mask-predict: Parallel decoding of conditional masked language models
Marjan Ghazvininejad, Omer Levy, Yinhan Liu, and Luke Zettlemoyer. Mask-predict: Parallel decoding of conditional masked language models. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 6112–6121, Hong Kong, China,
2019
-
[50]
Thibaud Gloaguen, Robin Staab, Nikola Jovanovic, and Martin T. Vechev. Watermarking diffusion language models.ArXiv preprint, abs/2509.24368,
-
[51]
Text-guidedmoleculegenerationwithdiffusionlanguage model
HaisongGong, QiangLiu, ShuWu, andLiangWang. Text-guidedmoleculegenerationwithdiffusionlanguage model. InThirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 2...
2024
-
[52]
Diffuseq: Sequence to sequence text generation with diffusion models
Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and Lingpeng Kong. Diffuseq: Sequence to sequence text generation with diffusion models. InThe Eleventh International Conference on Learning Represen- tations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, 2023a. Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and Lingpeng Kong. DiffuSeq-v2: Bridging discr...
Pith/arXiv arXiv 2023
-
[53]
Frey, Tim G
Nate Gruver, Samuel Stanton, Nathan C. Frey, Tim G. J. Rudner, Isidro Hötzel, Julien Lafrance-Vanasse, Arvind Rajpal, Kyunghyun Cho, and Andrew Gordon Wilson. Protein design with guided discrete diffu- sion. InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, L...
2023
-
[54]
Vector quantized diffusion model for text-to-image synthesis
Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, and Baining Guo. Vector quantized diffusion model for text-to-image synthesis. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pp. 10686– 10696,
2022
-
[55]
Powers, Weili Nie, Tomas Geffner, Karsten Kreis, Jure Leskovec, Arash Vahdat, and Stefano Ermon
Siyi Gu, Minkai Xu, Alexander S. Powers, Weili Nie, Tomas Geffner, Karsten Kreis, Jure Leskovec, Arash Vahdat, and Stefano Ermon. Aligning target-aware molecule diffusion models with exact energy optimiza- tion. InAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver,...
2024
-
[56]
Hashimoto
70 Ishaan Gulrajani and Tatsunori B. Hashimoto. Likelihood-based diffusion language models. InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023,
2023
-
[57]
GabrielGuoandStefanoErmon. Revivingany-subsetautoregressivemodelswithprincipledparallelsampling and speculative decoding.ArXiv preprint, abs/2504.20456,
-
[58]
Yingqing Guo, Yukang Yang, Hui Yuan, and Mengdi Wang. Training-free guidance beyond differentiability: Scalable path steering with tree search in diffusion and flow models.ArXiv preprint, abs/2502.11420,
-
[59]
Reward-weighted sampling: Enhancing non-autoregressive characteristics in masked diffusion llms
Daehoon Gwak, Minseo Jung, Junwoo Park, Minho Park, ChaeHun Park, Junha Hyung, and Jaegul Choo. Reward-weighted sampling: Enhancing non-autoregressive characteristics in masked diffusion llms. InPro- ceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 34562–34582,
2025
-
[60]
Kilian Haefeli, Karolis Martinkus, Nathanael Perraudin, and R. Wattenhofer. Diffusion models for graphs benefit from discrete state spaces.ArXiv preprint, abs/2210.01549,
-
[61]
Jiaqi Han, Austin Wang, Minkai Xu, Wenda Chu, Meihua Dang, Yisong Yue, and Stefano Ermon. Discrete diffusion trajectory alignment via stepwise decomposition.ArXiv preprint, abs/2507.04832, 2025a. Jun Han, Zixiang Chen, Yongqian Li, Yiwen Kou, Eran Halperin, Robert E. Tillman, and Quanquan Gu. Guided discrete diffusion for electronic health record generati...
arXiv 2025
-
[62]
Xiaoxu Han, Chengzhen Ning, Jinghui Zhong, Fubiao Yang, Yu Wang, and Xin Mu. Discovering mathemat- ical equations with diffusion language model.ArXiv preprint, abs/2509.13136, 2025c. Zhitao Han and Wenhui Huang. Prostate mri super-resolution using discrete residual diffusion model. In 2023 IEEE International Conference on Bioinformatics and Biomedicine (B...
arXiv 2023
-
[63]
Efficient perplexity bound and ratio matching in discrete diffusion language models
Etrit Haxholli, Yeti Ziya Gurbuz, Ogul Can, and Eli Waxman. Efficient perplexity bound and ratio matching in discrete diffusion language models. InThe Thirteenth International Conference on Learning Represen- tations, ICLR 2025, Singapore, April 24-28, 2025,
2025
-
[64]
Distillation of discrete diffusion through dimensional correlations
Satoshi Hayakawa, Yuhta Takida, Masaaki Imaizumi, Hiromi Wakaki, and Yuki Mitsufuji. Distillation of discrete diffusion through dimensional correlations. InForty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025, volume 267 ofProceedings of Machine Learning Research,
2025
-
[65]
Guangxin He, Shen Nie, Fengqi Zhu, Yuankang Zhao, Tianyi Bai, Ran Yan, Jie Fu, Chongxuan Li, and Binhang Yuan. Ultrallada: Scaling the context length to 128k for diffusion large language models.ArXiv preprint, abs/2510.10481, 2025a. Guojun He, Shengyu Zhang, and Tao Jiang. Residual vector quantized diffusion model-based joint source- channel coding for ta...
-
[66]
What exactly does guidance do in masked discrete diffusion models
Ye He, Kevin Rojas, and Molei Tao. What exactly does guidance do in masked discrete diffusion models. ArXiv preprint, abs/2506.10971, 2025d. Yi He, Ailun Wang, Zhi Wang, Yu Liu, Xingyuan Xu, and Wen Yan. Generative molecule evolution using 3d pharmacophore for efficient structure-based drug design.ArXiv preprint, abs/2507.20130, 2025e. Zhengfu He, Tianxia...
-
[67]
Shao, Chaofan Gan, Shijie Li, Zuxuan Wu, and Weiyao Lin
Zhihao He, Tieyuan Chen, Kangyu Wang, Ziran Qin, Y. Shao, Chaofan Gan, Shijie Li, Zuxuan Wu, and Weiyao Lin. Vidlada: Bidirectional diffusion large language models for efficient video understanding. ArXiv preprint, abs/2601.17868, 2026b. Michael Hersche, Samuel Moor-Smith, Thomas Hofmann, and Abbas Rahimi. Soft-masked diffusion language models.ArXiv prepr...
-
[69]
Unsu- pervised training of diffusion models for feasible solution generation in neural combinatorial optimization, 2024a
Seong-Hyun Hong, Hyun sung Kim, Zian Jang, Deunsol Yoon, Hyungseok Song, and Byung-Jun Lee. Unsu- pervised training of diffusion models for feasible solution generation in neural combinatorial optimization, 2024a. Shibo Hong, Xuhong Zhang, Tianyu Du, Sheng Cheng, Xun Wang, and Jianwei Yin. Cons2plan: Vector floorplan generation from various conditions via...
2021
-
[70]
Gritsenko, Jasmijn Bastings, Ben Poole, Rianne van den Berg, and Tim Salimans
Emiel Hoogeboom, Alexey A. Gritsenko, Jasmijn Bastings, Ben Poole, Rianne van den Berg, and Tim Salimans. Autoregressive diffusion models. InThe Tenth International Conference on Learning Represen- tations, ICLR 2022, Virtual Event, April 25-29, 2022,
2022
-
[71]
Bad: Bidirectional auto-regressive diffusion for text-to-motion generation
72 Seyed Rohollah Hosseyni, Ali Ahmad Rahmani, Seyed Jamal Seyedmohammadi, Sanaz Seyedin, and Arash Mohammadi. Bad: Bidirectional auto-regressive diffusion for text-to-motion generation. InICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5,
2025
-
[72]
Suganthan
Minghui Hu, Yujie Wang, Tat-Jen Cham, Jianfei Yang, and Ponnuthurai N. Suganthan. Global context with discrete diffusion in vector quantised modelling for image generation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pp. 11492–11501,
2022
-
[73]
Suganthan
Minghui Hu, Chuanxia Zheng, Zuopeng Yang, Tat-Jen Cham, Heliang Zheng, Chaoyue Wang, Dacheng Tao, and Ponnuthurai N. Suganthan. Unified discrete diffusion for simultaneous vision-language generation. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023,
2023
-
[74]
Mixed diffusion for 3d indoor scene synthesis.ArXiv preprint, abs/2405.21066,
Siyi Hu, Diego Martín Arroyo, Stephanie Debats, Fabian Manhardt, Luca Carlone, and Federico Tombari. Mixed diffusion for 3d indoor scene synthesis.ArXiv preprint, abs/2405.21066,
-
[75]
Abdelfattah, Jae sun Seo, Zhiru Zhang, and Udit Gupta
Zhanqiu Hu, Jian Meng, Yash Akhauri, Mohamed S. Abdelfattah, Jae sun Seo, Zhiru Zhang, and Udit Gupta. Flashdlm: Accelerating diffusion language model inference via efficient kv caching and guided diffusion.ArXiv preprint, abs/2505.21467,
-
[76]
Jianhao Huang and Baharan Mirzasoleiman. Tuning the implicit regularizer of masked diffusion language models: Enhancing generalization via insights from -parity.ArXiv preprint, abs/2601.22450,
-
[77]
Jiaxing Huang, Kaiwen Cui, Dayan Guan, Aoran Xiao, Fangneng Zhan, Shijian Lu, Shengcai Liao, and Eric P. Xing. Masked generative adversarial networks are data-efficient generation learners. InAdvances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 2...
2022
-
[78]
Wei Huang, Hanchen Wang, Dong Wen, Wenjie Zhang, Ying Zhang, and Xuemin Lin. Diffged: Computing graph edit distance via diffusion-based graph matching.ArXiv preprint, abs/2503.18245, 2025a. Yiming Huang and Tolga Birdal. Hog-diff: Higher-order guided diffusion for graph generation.ArXiv preprint, abs/2502.04308,
-
[79]
Zemin Huang, Zhiyang Chen, Zijun Wang, Tiancheng Li, and Guo-Jun Qi. Reinforcing the diffusion chain of lateral thought with diffusion language models.ArXiv preprint, abs/2505.10446, 2025b. Zemin Huang, Yuhang Wang, Zhiyang Chen, and Guo-Jun Qi. Don’t settle too early: Self-reflective remask- ing for diffusion language models.ArXiv preprint, abs/2509.2365...
arXiv 2024
-
[80]
Layoutdm: Discrete diffusion model for controllable layout generation
Naoto Inoue, Kotaro Kikuchi, Edgar Simo-Serra, Mayu Otani, and Kota Yamaguchi. Layoutdm: Discrete diffusion model for controllable layout generation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pp. 10167–10176,
2023
-
[81]
Cheng, Guy Van den Broeck, Aditya Grover, Suvinay Subramanian, and Michael Carbin
Daniel Israel, Tian Jin, Ellie Y. Cheng, Guy Van den Broeck, Aditya Grover, Suvinay Subramanian, and Michael Carbin. Planned diffusion.ArXiv preprint, abs/2510.18087,
-
[82]
Layout-corrector: Alleviating layout sticking phenomenon in discrete diffusion model
Shoma Iwai, Atsuki Osanai, Shunsuke Kitada, and Shinichiro Omachi. Layout-corrector: Alleviating layout sticking phenomenon in discrete diffusion model. InComputer Vision – ECCV 2024, pp. 92–110,
2024
-
[83]
Metod Jazbec, Theo X. Olausson, Louis Béthune, Pierre Ablin, Michael Kirchhof, João Monteiro, Victor Turrisi, Jason Ramapuram, and Marco Cuturi. Learning unmasking policies for diffusion language models. ArXiv preprint, abs/2512.09106,
-
[84]
Wonje Jeung, Sangyeon Yoon, Yoonjun Cho, Dongjae Jeon, Sangwoo Shin, Hyesoo Hong, and Albert No. A2d: Any-order, any-step safety alignment for diffusion language models.ArXiv preprint, abs/2509.23286,
-
[85]
Yatai Ji, Teng Wang, Yuying Ge, Zhiheng Liu, Sidi Yang, Ying Shan, and Ping Luo. From denoising to refining: A corrective framework for vision-language diffusion model.ArXiv preprint, abs/2510.19871,
-
[86]
Mengni Jia, Mengyu Zhou, Yihao Liu, Xiaoxi Jiang, and Guanjun Jiang. Bringing stability to diffusion: Decomposing and reducing variance of training masked diffusion models.ArXiv preprint, abs/2511.18159,
-
[87]
Haozhe Jiang, Nika Haghtalab, and Lijie Chen. Diffusion language models are provably optimal parallel samplers.ArXiv preprint, abs/2512.25014, 2025a. Yuchu Jiang, Yue Cai, Xiangzhong Luo, Jiale Fu, Jiarui Wang, Chonghan Liu, and Xu Yang. dcache: Accelerating diffusion-based llms via dual adaptive caching.ArXiv preprint, abs/2509.23094, 2025b. Jaehyeong Jo...
arXiv 2024
-
[88]
Mingyu Jo, Jaesik Yoon, Justin Deschenaux, Caglar Gulcehre, and Sungjin Ahn. Loopholing discrete diffu- sion: Deterministic bypass of the sampling wall.ArXiv preprint, abs/2510.19304,
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.