Pith. sign in

REVIEW 3 major objections 5 minor 36 references

Diffusion LLMs score higher when selected tokens coordinate before writing, not when each is chosen alone from its own marginal.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 18:55 UTC pith:3OIR46EB

load-bearing objection Clean diagnosis of within-step factorization error in dLLMs, plus a lightweight marker-gated last-n re-pass that improves all six matched benchmarks—but the joint-mode story still needs a control that isolates the marker from extra compute and LoRA CPT. the 3 major comments →

arxiv 2607.04469 v1 pith:3OIR46EB submitted 2026-07-05 cs.CL

Don't Commit Alone: Joint Token Commitment in Diffusion Large Language Models

Lin Yao This is my paper
classification cs.CL
keywords diffusion language modelsparallel decodingjoint token commitmentconditional total correlationmarker-gated coordinationCoCommitdLLMfactorized commitment
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Diffusion large language models gain speed by committing several tokens in one denoising step, but those tokens are chosen independently from the same context. When the positions depend on one another, the product of their marginals is not a coherent joint completion; the paper names that gap conditional total correlation and notes that confidence-based selection cannot see it. CoCommit briefly defers the write: after the usual confidence rule picks the commit set, a learned marker announces those positions and the backbone’s last layers are re-run so the marked tokens can attend to one another before greedy argmax. With only LoRA adapters, marker vectors, and one extra partial forward pass, the method improves accuracy or exact match on every benchmark evaluated, with the largest lifts on reasoning and short exact-answer tasks. A sympathetic reader cares because the speed of parallel decoding need not be paid for with inconsistent multi-token writes if commitment itself is allowed a short joint-mode pass.

Core claim

Factorized commitment replaces the joint posterior over a commit set with a product of marginals; the resulting error is conditional total correlation, which per-position confidence cannot bound. CoCommit intercepts the error at write time by announcing the selected set with a learned marker and re-applying the backbone’s last-n layers so marked positions coordinate, approximating joint-mode decoding under greedy argmax. On LLaDA2.1-mini trained only with LoRA and markers on generic prose, joint commitment improves all six benchmarks under matched greedy inference, most on DROP, CMATH, and AIME 2025.

What carries the argument

CoCommit: a marker-gated coordination pass that adds a learned marker to the hidden states of the confidence-selected commit set and re-applies the same last-n backbone layers so those positions exchange information before tokens are written—one extra partial forward, no auxiliary model.

Load-bearing premise

That LoRA adapters and marker vectors trained on ordinary English prose with random marked bundles are enough for the last layers to actually coordinate a confidence-selected bundle into one joint mode at inference.

What would settle it

If removing the marker (or re-running Stage B without it) while keeping the same LoRA weights and extra compute erases the benchmark gains, the coordination claim fails; the same holds if gains disappear when the base model is fully fine-tuned without any marker mechanism.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Within-step factorization error can be reduced without an external dependency model or full re-pretraining of the backbone.
  • Confidence-selected bundles need not be discarded; they can be coordinated after selection and still written greedily.
  • Marking already-visible tokens would turn commitment and revision into one joint re-negotiation operation.
  • Iterating the coordination pass (K greater than 1) adds a test-time compute axis that trades depth for stronger jointness inside the bundle.
  • Within-step coordination and after-the-fact repair methods target orthogonal error axes and can be combined.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If mode-averaging is the real failure mode, similar marker or attention-routing tricks may help any parallel decoder that still uses a factorized readout head, not only mask diffusion models.
  • Gains on math and reasoning after training only on generic prose suggest the benefit is architectural coordination rather than injected task knowledge.
  • A direct head-to-head against methods that restore the joint with a separate autoregressive model would quantify the price of internalizing coordination in the backbone’s own last layers.
  • With noise and deeper K, the same machinery sits on a continuum toward a cheap conditional joint model over the commit bundle without leaving the existing dLLM pipeline.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper argues that parallel commitment in diffusion LLMs incurs a within-step factorization error equal to the conditional total correlation of the commit bundle, and that confidence-based selection cannot bound this error because marginals average over joint modes. It proposes CoCommit: after standard confidence selection of a commit set S, a learned marker is added to the hidden states of positions in S and the backbone’s last-n layers are re-applied so marked positions can attend to one another before greedy argmax writes tokens. Training freezes LLaDA2.1-mini, adds LoRA adapters and marker vectors, and uses a two-branch objective (unmarked selection branch plus marked random-bundle assignment branch) on generic FineWeb-Edu prose. Under matched greedy inference, the method reports positive accuracy/EM gains on all six benchmarks, largest on DROP, CMATH, and AIME 2025.

Significance. If the mechanism truly approximates joint-mode commitment rather than merely adding compute or generic adaptation, the contribution is practically and conceptually useful: it targets a named distributional error at the instant of commitment with one extra partial forward, no auxiliary model, and no change to the rest of the dLLM pipeline. The conditional-total-correlation framing and the explicit separation of within-step factorization from across-step repair are clear and transferable. Strengths include the matched greedy protocol, the deliberately minimal LoRA stress-test design (honestly scoped in §4), and the pattern of larger gains on multi-token/exact-answer tasks. The work would matter for the growing dLLM decoding literature if the joint-mode claim is isolated from simpler confounds.

major comments (3)
  1. Table 1 is the sole empirical support for the central claim that marker-gated Stage B approximates joint-mode decoding and reduces within-step factorization error. The comparison is released base (factorized) vs. LoRA+marker CPT with Stage B. There is no control that isolates (i) the marker announcement and attention among marked positions from (ii) the extra partial forward itself or (iii) generic LoRA adaptation of the same last-n layers. Without at least one of: LoRA CPT without marker, Stage B with a null/random marker, or an equal-compute factorized double pass, the positive deltas (especially DROP +4.38, AIME +3.33) do not uniquely support the joint-mode story over simpler explanations. This is load-bearing given the paper’s own framing of the setup as a mechanism stress test (§4, “Compute-constrained validation”).
  2. §1 and §4 claim that under greedy argmax, one deterministic coordination pass yields a tuple from a single joint mode rather than a mode mixture, and that branch-2 CE induces coordination only through attention among marked positions. The manuscript reports no diagnostic of this: no estimate of conditional total correlation before/after Stage B, no within-bundle agreement or mode-consistency metric, and no comparison of joint vs. factorized log-probability of committed tuples. Ordinary CE on random marked bundles can improve predictions without the claimed coordination. A small diagnostic suite on held-out prose or a subset of DROP/CMATH would make the mechanism claim falsifiable rather than inferred from end-task accuracy alone.
  3. §5.1 fixes n=3, K=1, s∼Uniform{1..16}, and checkpoint step 6250 with no sensitivity or ablation. The free parameters listed in the method (K, n, marker design, training bundle distribution) are part of the claimed mechanism. At minimum, K∈{0,1} (K=0 recovers factorized decoding with the same adapters) and a marker-ablated Stage B should be reported; without them the reader cannot tell whether the coordination pass is doing the work the theory assigns to it.
minor comments (5)
  1. Figure 1 caption example (“the keys … the desk is mine”) is helpful but the figure itself is only described in text in the arXiv source; ensure the published version makes the Stage A vs Stage B contrast visually unambiguous.
  2. §4.1 cites Yao (34) for self-generated x→z corruption; a one-sentence restatement of that procedure would make the training setup self-contained for readers who do not follow the companion arXiv notes.
  3. AIME 2025 +3.33 is one extra problem on a 30-problem set; the text notes this, but Table 1 should flag the small-N nature (e.g., absolute counts or a note) so the gain is not over-read as a large effect size.
  4. Notation: TC(x_S | ctx) is introduced clearly in §1 but never appears again in experiments; either use it in a diagnostic or soften the abstract’s implication that the method is shown to reduce TC rather than to improve task metrics consistent with that reduction.
  5. Limitations correctly scope greedy commitment and single-model evaluation; consider also stating that T2T edits still read pre-update logits and are not jointly coordinated with M2T writes in the same step, so residual within-step inconsistency remains possible.

Circularity Check

0 steps flagged

No significant circularity: empirical gains on held-out benchmarks are not forced by training objective, definition, or load-bearing self-citation.

full rationale

The paper’s chain is conceptual framing (within-step error as conditional total correlation of the commit bundle; confidence selection reads only marginals) → a concrete architectural intervention (marker-gated re-application of last-n layers) → ordinary cross-entropy training of LoRA adapters and marker vectors on generic FineWeb-Edu prose with random marked bundles → evaluation under matched greedy inference on six external benchmarks (Table 1). The accuracy/EM deltas are not identities of the training loss, not fitted parameters renamed as predictions, and not uniqueness theorems imported from the author’s prior work. Self-citations (Yao remask / self-generated-error papers) only specify how corrupted inputs z are built and place the method among repair literature; they do not underwrite the Table 1 claims. The TC story is diagnostic framing, not a fitted quantity that forces the reported scores. Absence of ablations isolating marker coordination from extra partial compute or generic LoRA CPT is a causal-attribution / experimental-design concern, not circularity by construction. Score 0 with empty steps is the correct finding.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 2 invented entities

The central claim rests on a domain diagnosis (factorized commit error = conditional TC; marginals hide modes), a modeling bet (marker + last-n re-pass ≈ joint-mode under greedy argmax), and several hand-chosen training/inference knobs (n, K, LoRA, thresholds, random bundle sizes). No new physical entity is postulated; the invented pieces are the marker vectors and the split-forward protocol. Free parameters are method hyperparameters, not fits that define the reported accuracy by construction.

free parameters (7)
  • coordination depth K
    Main experiments fix K=1; deeper iteration is proposed but not trained/evaluated. Choice directly defines the method’s compute–coordination trade-off.
  • last-n layers n
    n=3 of 20 layers is chosen for Stage B re-application; no sweep reported. Controls how much of the backbone participates in coordination.
  • LoRA rank r and alpha
    r=16, α=32 on Q/K/V/O of all layers; adapters are the only trained weights besides markers. Capacity of the learned coordination path.
  • M2T confidence threshold τ_m2t
    τ_m2t=0.7 (with fallback to top-1) selects the commit bundle S at inference; selection is unchanged from base but still a free operating point.
  • training bundle size distribution s~Uniform{1..16}
    Random marked set size during branch-2 training; chosen by hand to cover inference bundles without using confidence at train time.
  • loss weights (λ_m2t, λ_edit, λ_clean)
    Set to (1.0, 0.3, 0.2); balances M2T vs T2T terms that supervise both branches.
  • checkpoint step 6250 / one-epoch CPT schedule
    Single reported checkpoint after short continued pretraining; evaluation point is a free choice without multi-seed reporting.
axioms (5)
  • domain assumption Within-step parallel error equals conditional total correlation TC(x_S|ctx) between the joint posterior and the product of marginals.
    Introduction frames factorization gap as TC; used as the governing quantity of the quality–speed trade-off without empirical TC measurement on model posteriors.
  • domain assumption Per-position marginal confidences average over joint modes and therefore cannot bound commitment error.
    Core motivation for not fixing selection alone; illustrated with New York / San Francisco example in §1.
  • ad hoc to paper Under greedy argmax, one deterministic marker-gated last-n pass yields a tuple from a single joint mode rather than a mode mixture.
    Stated as the design target of Stage B (§1, §4.2); not proved, and explicitly scoped away from temperature sampling.
  • ad hoc to paper Ordinary CE on marked random bundles induces coordination only through attention among marked positions (edit/clean hold unmarked predictions fixed).
    Training design claim in §4.1; no control that isolates attention-among-marked as the causal pathway.
  • domain assumption LLaDA2.1 block-diffusion training/inference (M2T/T2T, block-causal attention) is the correct base system for evaluating joint commitment.
    Preliminaries §3; all experiments use this family only.
invented entities (2)
  • Learned marker vectors {m_i} announcing the commit set no independent evidence
    purpose: Gate Stage B so last-n layers re-process only (or especially) positions about to be written, creating a communication channel at decision time.
    New parameters (zero-init) beyond LoRA; no independent measurement outside this training setup that markers encode a specific linguistic signal.
  • CoCommit split-forward protocol (Stage A select, Stage B coordinate, then write) no independent evidence
    purpose: Replace factorized M2T/T2T commitment with approximate joint-mode commitment without a new head or auxiliary model.
    The paper’s main algorithmic object; evidence is the Table 1 deltas under this protocol, not external replication.

pith-pipeline@v1.1.0-grok45 · 14574 in / 3984 out tokens · 41041 ms · 2026-07-11T18:55:24.593181+00:00 · methodology

0 comments
read the original abstract

Diffusion large language models (dLLMs) commit multiple tokens per denoising step by decoding each selected position independently from the shared context; when those positions are dependent, the resulting factorization error is captured by conditional total correlation, which confidence-based selection cannot observe from marginals alone. We propose CoCommit, a marker-gated coordination pass that briefly defers commitment: after the usual bundle selection, a learned marker announces the commit set and the backbone's last-$n$ layers are re-applied so marked positions coordinate -- approximating joint-mode decoding -- before greedy argmax writes tokens. The method reuses existing weights with one extra partial forward pass and no auxiliary model. On LLaDA2.1-mini with LoRA adapters and matched greedy inference, joint commitment improves accuracy on all six benchmarks we evaluate, with the largest gains on reasoning and exact-answer tasks.

Figures

Figures reproduced from arXiv: 2607.04469 by Lin Yao.

Figure 1
Figure 1. Figure 1: Factorized versus joint commitment (one denoising step). [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Split forward pass through the L-layer backbone. (a) Training: shared Prefix, then two branches in parallel—factorized selection (no marker) and marked coordination on a random bundle. (b) Inference: the same split, but serial—Stage A selects the commit bundle; Stage B coordinates it before M2T/T2T writes. iff j ∈ D. (ii) For i = 0, . . . , K−1, h (0) = h, h(i+1) = LastN h (i) + 1D ⊙ mi  , ℓchosen = Head… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references · 14 linked inside Pith

  1. [1]

    Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sekhar Sahoo, and V olodymyr Kuleshov

    Marianne Arriola, Aaron Gokaslan, Justin T. Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sekhar Sahoo, and V olodymyr Kuleshov. Block discrete denoising diffusion language models. InInternational Conference on Learning Representations, 2025

    2025

  2. [2]

    Structured denoising diffusion models in discrete state-spaces

    Jacob Austin, Daniel Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Structured denoising diffusion models in discrete state-spaces. InAdvances in Neural Information Processing Systems, 2021

    2021

  3. [3]

    Deep equilibrium models

    Shaojie Bai, J Zico Kolter, and Vladlen Koltun. Deep equilibrium models. InAdvances in Neural Information Processing Systems, 2019

    2019

  4. [4]

    On the statistical analysis of dirty pictures.Journal of the Royal Statistical Society: Series B (Methodological), 48(3):259–279, 1986

    Julian Besag. On the statistical analysis of dirty pictures.Journal of the Royal Statistical Society: Series B (Methodological), 48(3):259–279, 1986. 8

    1986

  5. [5]

    LLaDA2.1: Speeding up text diffusion via token editing.arXiv preprint arXiv:2602.08676, 2026

    Tiwei Bie, Maosong Cao, Xiang Cao, Bingsen Chen, Fuyuan Chen, Kun Chen, Lun Du, Daozhuo Feng, Haibo Feng, Mingliang Gong, Zhuocheng Gong, Yanmei Gu, Jian Guan, Kaiyuan Guan, Hongliang He, Zenan Huang, Juyong Jiang, Zhonghui Jiang, Zhenzhong Lan, Chengxi Li, Jianguo Li, Zehuan Li, Huabin Liu, Lin Liu, Guoshan Lu, Yuan Lu, Yuxin Ma, Xingyu Mou, Zhenxuan Pan...

    arXiv 2026

  6. [6]

    Universal transformers

    Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Łukasz Kaiser. Universal transformers. InInternational Conference on Learning Representations, 2019

    2019

  7. [7]

    DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs.arXiv preprint arXiv:1903.00161, 2019

    Dheeru Dua et al. DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs.arXiv preprint arXiv:1903.00161, 2019

    Pith/arXiv arXiv 1903

  8. [8]

    Scaling up test-time compute with latent reasoning: A recurrent depth approach.arXiv preprint arXiv:2502.05171, 2025

    Jonas Geiping, Sean McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, and Tom Goldstein. Scaling up test-time compute with latent reasoning: A recurrent depth approach.arXiv preprint arXiv:2502.05171, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    Mask-predict: Parallel decoding of conditional masked language models

    Marjan Ghazvininejad, Omer Levy, Yinhan Liu, and Luke Zettlemoyer. Mask-predict: Parallel decoding of conditional masked language models. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, 2019

    2019

  10. [10]

    Scaling diffusion language models via adaptation from autoregressive models.International Conference on Learning Representations, 2025

    Shansan Gong, Shivam Agarwal, Yizhe Zhang, Jiacheng Ye, Lin Zheng, Mukai Li, Chenxin An, Peilin Zhao, Wei Bi, Jiawei Han, Hao Peng, and Lingpeng Kong. Scaling diffusion language models via adaptation from autoregressive models.International Conference on Learning Representations, 2025

    2025

  11. [11]

    Non-autoregressive neural machine translation

    Jiatao Gu, James Bradbury, Caiming Xiong, Victor OK Li, and Richard Socher. Non-autoregressive neural machine translation. InInternational Conference on Learning Representations, 2018

    2018

  12. [12]

    Continuous latent diffusion language model.arXiv preprint arXiv:2605.06548, 2026

    Hongcan Guo, Qinyu Zhao, Yian Zhao, Shen Nie, Rui Zhu, Qiushan Guo, Feng Wang, Tao Yang, Hengshuang Zhao, Guoqiang Wei, and Yan Zeng. Continuous latent diffusion language model.arXiv preprint arXiv:2605.06548, 2026

    Pith/arXiv arXiv 2026

  13. [13]

    MDPO: Overcoming the training-inference divide of masked diffusion language models.arXiv preprint arXiv:2508.13148, 2025

    Haoyu He, Katrin Renz, Yong Cao, and Andreas Geiger. MDPO: Overcoming the training-inference divide of masked diffusion language models.arXiv preprint arXiv:2508.13148, 2025

    arXiv 2025

  14. [14]

    ELF: Embedded language flows.arXiv preprint arXiv:2605.10938, 2026

    Keya Hu, Linlu Qiu, Yiyang Lu, Hanhong Zhao, Tianhong Li, Yoon Kim, Jacob Andreas, and Kaiming He. ELF: Embedded language flows.arXiv preprint arXiv:2605.10938, 2026

    Pith/arXiv arXiv 2026

  15. [15]

    Don’t settle too early: Self-reflective remasking for diffusion language models.arXiv preprint arXiv:2509.23653, 2025

    Zemin Huang, Yuhang Wang, Zhiyang Chen, and Guo-Jun Qi. Don’t settle too early: Self-reflective remasking for diffusion language models.arXiv preprint arXiv:2509.23653, 2025

    arXiv 2025

  16. [16]

    TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension.arXiv preprint arXiv:1705.03551, 2017

    Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension.arXiv preprint arXiv:1705.03551, 2017

    Pith/arXiv arXiv 2017

  17. [17]

    ParallelBench: Understanding the trade-offs of parallel decoding in diffusion LLMs.arXiv preprint arXiv:2510.04767, 2025

    Wonjun Kang, Kevin Galim, Seunghyuk Oh, Minjae Lee, Yuchen Zeng, Shuibai Zhang, Coleman Hooper, Yuezhou Hu, Hyung Il Koo, Nam Ik Cho, and Kangwook Lee. ParallelBench: Understanding the trade-offs of parallel decoding in diffusion LLMs.arXiv preprint arXiv:2510.04767, 2025

    Pith/arXiv arXiv 2025

  18. [18]

    Fine-tuning masked diffusion for provable self-correction.arXiv preprint arXiv:2510.01384, 2025

    Jaeyeon Kim, Seunggeun Kim, Taekyun Lee, David Z Pan, Hyeji Kim, Sham Kakade, and Sitan Chen. Fine-tuning masked diffusion for provable self-correction.arXiv preprint arXiv:2510.01384, 2025

    Pith/arXiv arXiv 2025

  19. [19]

    GSM-Plus: A comprehensive benchmark for evaluating the robustness of LLMs as mathematical problem solvers.arXiv preprint arXiv:2402.19255, 2024

    Qintong Li, Leyang Cui, Xueliang Zhao, Lingpeng Kong, and Wei Bi. GSM-Plus: A comprehensive benchmark for evaluating the robustness of LLMs as mathematical problem solvers.arXiv preprint arXiv:2402.19255, 2024. 9

    Pith/arXiv arXiv 2024

  20. [20]

    Autoregressive image generation without vector quantization

    Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. InAdvances in Neural Information Processing Systems, 2024

    2024

  21. [21]

    Discrete copula diffusion

    Anji Liu, Oliver Broadrick, Mathias Niepert, and Guy Van den Broeck. Discrete copula diffusion. In International Conference on Learning Representations, 2025

    2025

  22. [22]

    Discrete diffusion modeling by estimating the ratios of the data distribution

    Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution. InInternational Conference on Machine Learning, 2024

    2024

  23. [23]

    American invitational mathematics examination 2025, 2025

    Mathematical Association of America. American invitational mathematics examination 2025, 2025

    2025

  24. [24]

    LLaDA: Large language diffusion with masking

    Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Zhou, Ji-Rong Wen, and Chongxuan Li. LLaDA: Large language diffusion with masking. InInternational Conference on Machine Learning, 2025

    2025

  25. [25]

    Improved large language diffusion models.arXiv preprint arXiv:2606.25331, 2026

    Shen Nie, Qiyang Min, Shaoxuan Xu, Zihao Huang, Yuxuan Song, Yong Shan, Yankai Lin, Wayne Xin Zhao, Chongxuan Li, and Ji-Rong Wen. Improved large language diffusion models.arXiv preprint arXiv:2606.25331, 2026

    Pith/arXiv arXiv 2026

  26. [26]

    Your absorbing discrete diffusion secretly models the conditional distributions of clean data

    Jingyang Ou, Shen Nie, Kaiwen Xue, Fengqi Zhu, Jiacheng Sun, Zhenguo Li, and Chongxuan Li. Your absorbing discrete diffusion secretly models the conditional distributions of clean data. InInternational Conference on Learning Representations, 2025

    2025

  27. [27]

    Chiu, Alexander M

    Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T. Chiu, Alexander M. Rush, and V olodymyr Kuleshov. Simple and effective masked diffusion language models. InAdvances in Neural Information Processing Systems, 2024

    2024

  28. [28]

    Learn from your mistakes: Self-correcting masked diffusion models.arXiv preprint arXiv:2602.11590, 2026

    Yair Schiff, Omer Belhasin, Roy Uziel, Guanghan Wang, Marianne Arriola, Gilad Turok, Michael Elad, and V olodymyr Kuleshov. Learn from your mistakes: Self-correcting masked diffusion models.arXiv preprint arXiv:2602.11590, 2026

    Pith/arXiv arXiv 2026

  29. [29]

    Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, and Michalis K. Titsias. Simplified and generalized masked diffusion for discrete data. InAdvances in Neural Information Processing Systems, 2024

    2024

  30. [30]

    Remasking discrete diffusion models with inference-time scaling

    Guanghan Wang, Yair Schiff, Subham Sekhar Sahoo, and V olodymyr Kuleshov. Remasking discrete diffusion models with inference-time scaling. InAdvances in Neural Information Processing Systems, 2025

    2025

  31. [31]

    MMLU-Pro: A more robust and challenging multi-task language understanding benchmark.arXiv preprint arXiv:2406.01574, 2024

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. MMLU-Pro: A more robust and challenging multi-task language understanding benchmark.arXiv preprint arXiv:2406.01574, 2024

    Pith/arXiv arXiv 2024

  32. [32]

    CMATH: Can your language model pass chinese elementary school math test?arXiv preprint arXiv:2306.16636, 2023

    Tianwen Wei, Jian Luan, Wei Liu, Shuang Dong, and Bin Wang. CMATH: Can your language model pass chinese elementary school math test?arXiv preprint arXiv:2306.16636, 2023

    Pith/arXiv arXiv 2023

  33. [33]

    Remask, don’t replace: Token-to-mask refinement in diffusion large language models.arXiv preprint, 2026

    Lin Yao. Remask, don’t replace: Token-to-mask refinement in diffusion large language models.arXiv preprint, 2026

    2026

  34. [34]

    Self-generated error training for token editing in diffusion language models.arXiv preprint arXiv:2606.17175, 2026

    Lin Yao. Self-generated error training for token editing in diffusion language models.arXiv preprint arXiv:2606.17175, 2026

    Pith/arXiv arXiv 2026

  35. [35]

    Dream 7B: Diffusion large language models.arXiv preprint arXiv:2508.15487, 2025

    Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream 7B: Diffusion large language models.arXiv preprint arXiv:2508.15487, 2025

    Pith/arXiv arXiv 2025

  36. [36]

    CORE: Context-robust remasking for diffusion language models.arXiv preprint arXiv:2602.04096, 2026

    Kevin Zhai, Sabbir Mollah, Zhenyi Wang, and Mubarak Shah. CORE: Context-robust remasking for diffusion language models.arXiv preprint arXiv:2602.04096, 2026. 10

    arXiv 2026