Pith. sign in

REVIEW 3 major objections 5 minor 112 references

Masked diffusion models default to left-to-right, not any-order; the paper pinpoints positional uncertainty and proposes two fixes.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 14:18 UTC pith:4T4UVW36

load-bearing objection Serious paper: the similarity analysis and LatentMDM are real contributions, but the positional-uncertainty mechanism is correlational rather than causal; worth a referee, with a request for a direct intervention. the 3 major comments →

arxiv 2607.26504 v1 pith:4T4UVW36 submitted 2026-07-29 cs.LG

From Interface to Inference: Eliciting Any-Order Inference from Any-Order Models

Seunggeun Kim , Jaeyeon Kim , Taekyun Lee , Yuyuan Chen , Yilun Du , Sham Kakade , Sitan Chen This is my paper
classification cs.LG
keywords masked diffusion modelsany-order inferencepositional uncertaintycode generationinsertion-based diffusionlatent-space diffusioncausal collapseconfidence-based decoding
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Masked diffusion models are trained to predict any masked position, which looks like a native any-order interface. The paper shows that in practice, confidence-based decoding makes them generate code that is structurally almost identical to strict left-to-right decoding. The culprit is positional uncertainty: the model often knows which semantic component should appear next but spreads its probability across several plausible locations, so confidence ranking favors locally certain left-to-right continuations. To escape this bottleneck, the paper offers two routes: insertion-based masked diffusion, which lets later insertions shift token positions, and latent-space masked diffusion, where the model chooses which semantic segment to decode next. A sympathetic reader would care because the paper recasts any-order inference as a property of the decoding interface, not just the training objective, and demonstrates concrete downstream gains from both fixes.

Core claim

The paper's central claim is that the any-order interface of masked diffusion models does not automatically yield any-order inference: under confidence-based decoding, MDMs collapse toward strict left-to-right generation because of positional uncertainty, defined via aggregate mass and localization of each token's predicted probability across masked positions. It then claims that this bottleneck can be circumvented by relaxing either the fixed canvas (FlexMDM, which inserts new mask tokens during inference and thus does not irrevocably commit tokens to positions) or the token-level granularity (LatentMDM, which performs masked diffusion over segment-level latent vectors and selects which sem

What carries the argument

Positional uncertainty, quantified by aggregate mass m(v;x_t) equal to the sum over masked positions of f_i(v|x_t) and localization LOC(v;x_t) equal to the maximum over masked positions of f_i(v|x_t) divided by m(v;x_t), is the central mechanism. The paper claims that high-mass, low-localization tokens represent the knows-what-but-not-where cases that confidence-based decoding skips, forcing left-to-right order. FlexMDM's insertion operation and LatentMDM's segment-level latent masking are the two mechanisms that bypass this bottleneck.

Load-bearing premise

The load-bearing premise is that the measured pattern of high aggregate mass with low localization is the cause of left-to-right collapse; if that pattern is a byproduct of training-data bias or of the fixed-canvas setup rather than the mechanism, both proposed fixes may be addressing a symptom.

What would settle it

Run a standard token-level MDM with a decoding policy that deliberately reveals, at each step, the masked position whose token has the highest aggregate mass even when localization is low, then measure AST similarity to strict left-to-right decoding and the generation order itself. If the outputs remain structurally left-to-right, positional uncertainty is not the bottleneck; if they become genuinely any-order, the causal claim is supported.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Confidence-based decoding on token-level, fixed-canvas MDMs cannot deliver genuine any-order inference; even a learned unmasking planner such as dUltra stays structurally close to left-to-right decoding.
  • Insertion-based masked diffusion produces non-contiguous, tree-level any-order generation, as measured by Coverage Before Commitment, Return to Unfinished Blocks, and Open-Block Width, and it beats the base model on HumanEval/HumanEval+ at Pass@16.
  • Latent-space masked diffusion enables segment-level order search: it outperforms its own left-to-right and random segment-selection variants by over six points and beats autoregressive, MDM, and flow baselines at matched wall-clock time.
  • An insertion temperature knob controls the exploration-commitment trade-off independently of token temperature, improving Pass@1 on MBPP when sharpened.
  • The interface-inference gap is not fixed by post-training the unmasking policy; the fixed-canvas, token-level interface itself must change.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the causal story would be to decode by targeting high-mass, low-localization tokens first; the paper reports the pattern but does not run that intervention, so if the test fails to produce non-left-to-right order, positional uncertainty may be a symptom rather than the bottleneck.
  • The general design principle of decoupling what-to-generate from where-to-place likely transfers beyond code to other structured discrete domains such as documents, formal proofs, or data layouts where order is not causally fixed.
  • Combining insertion-based refinement with latent-space order search may compound the two forms of any-order behavior, a direction the paper explicitly leaves open.
  • The AST-based any-order metrics (CBC, RUB, OBW) could be reused as training objectives rather than only evaluation metrics, for example as reinforcement-learning rewards that encourage non-contiguous drafting.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies whether masked diffusion models (MDMs), despite their any-order training objective, actually deliver genuine any-order inference. It first reports a negative result: under confidence-based decoding, MDMs generate Python code that is structurally close to strict left-to-right decoding (Section 3.1). It then proposes a mechanistic explanation, 'positional uncertainty': the model knows what token to generate but not where to place it, formalized through aggregate mass m(v;x_t) and localization LOC(v;x_t) (Section 3.2). Based on this diagnosis, the paper proposes two remedies: FlexMDM, an insertion-based masked diffusion model that relaxes the fixed-canvas assumption, and LatentMDM, which moves masked diffusion to segment-level latent variables. Empirically, FlexMDM fine-tuned from Dream-Coder improves Pass@16 on HumanEval/HumanEval+, while LatentMDM outperforms small autoregressive and MDM baselines on GSM8K at matched wall-clock time. The codebase is released.

Significance. If the causal story holds, the paper provides a useful mechanistic account of causal collapse in MDMs and a design principle for discrete diffusion models: any-order inference requires relaxing either token-level or fixed-canvas commitments. The similarity analysis in Section 3.1 is carefully controlled (same backbone, varied decoding policy, multiple tree metrics, best-match aggregation), and the FlexMDM/LatentMDM ablations show that distinct generation orders can be induced. The paper also ships an open-source implementation, which strengthens reproducibility. The central limitation is that the causal claim is not tested by an intervention on a fixed MDM; the proposed solutions change the model class and training objective, so the evidence for 'positional uncertainty as the key bottleneck' remains correlational. The contributions are nonetheless substantial and likely to influence follow-up work.

major comments (3)
  1. [Section 3.2, Figs. 3, 6, 7] The diagnosis that positional uncertainty is the 'key bottleneck' behind causal collapse is correlational. The paper shows that high-mass tokens often have low localization and that committed tokens are more localized, but it never intervenes on the decoding rule of a fixed MDM to test whether targeting high-mass/low-LOC tokens changes the generation order or downstream quality. The alternative training-data-bias explanation (Gong et al. 2025; Li et al. 2026) is dismissed because Python lacks a unique causal order, but that does not rule out dataset-level order preferences. A concrete test is needed: on a fixed Dream/DiffuCoder model, decode by selecting positions whose predicted token has maximum aggregate mass m(v;x_t) or minimum LOC(v;x_t), and compare ASTD-to-L2R and Pass@k against Top-K and random decoding. Without such an intervention, the design choices in Section 4 are justified
  2. [Section 4.1, Tables 4 and 5] The FlexMDM experiments do not isolate insertion as the cause of any-order behavior. FlexMDM is fine-tuned from Dream-Coder on 2.6M additional examples with a different training loss, different schedules, and a per-benchmark insertion temperature. The downstream comparison is also mixed: Table 5 shows FlexMDM trails Dream-Coder at Pass@1 on HumanEval (50.65 vs 58.65) and at k≥2 on MBPP/MBPP+. The tree metrics in Table 4 compare a model trained with insertion to a standard MDM, so the gains could reflect fine-tuning data, compute, or inference temperature rather than the insertion mechanism. A controlled comparison—fine-tuning Dream-Coder with the standard MDM objective on the same data and compute, or ablating insertion while keeping the training data fixed—is necessary to support the claim that FlexMDM circumvents positional uncertainty.
  3. [Section 4.2, Fig. 5 and Table 7] The LatentMDM ablation (LatentMDM vs LatentMDM-L2R/Random) is the only evidence that segment-level selection matters, and it supports a form of any-order inference over segments. However, the broader claim that LatentMDM 'mitigates positional uncertainty' is indirect: the mass/localization measurement is performed in token space after decoding, which is not the decision space of the latent segment model. The wall-clock comparison against AR/MDM baselines also mixes architectural and training differences. Please add a token-level MDM trained on TinyGSM with the same parameter count and compute, or at least show that the segment-selection score correlates with reduced token-level positional uncertainty. Without this, the link between the LatentMDM design and the Section 3.2 mechanism remains asserted rather than demonstrated.
minor comments (5)
  1. [Section 3.1 / Appendix B] The metric is called 'Abstract Syntax Tree Distance' but is a normalized similarity (Sim = 1 − TED/max size). The name is misleading; consider 'AST similarity' or clarify in the definition.
  2. [Tables 1 and 2] Several 'Entropy vs L2R' rows report similarity values of 0.999 or 1.000. The paper notes that confidence-based policies produce little diversity at temperature 0.2, but the tables would benefit from an explicit statement of how many distinct samples each policy produced and whether these near-1 values reflect identical or near-identical outputs.
  3. [Section 3.2 / Figure 3] The mass-localization plots aggregate points across prompts but no confidence intervals, variance, or counts are reported. Since the figure is the main visual support for positional uncertainty, adding error bars or per-prompt summaries would strengthen the claim.
  4. [Appendix B.2] The dUltra planner comparison uses a planner temperature of 0 for the learned policy and 0.2 for backbone-only policies. This small inconsistency should be acknowledged, as it may affect the similarity comparison.
  5. [Section 4.1 / Appendix D.1] The schedules (a,b)=(1.7,1.7) and inference-time â=2.9 are introduced with little sensitivity analysis. Reporting the effect of these choices on the any-order metrics would help readers understand the robustness of the FlexMDM results.

Circularity Check

0 steps flagged

No circularity found: the interface-inference gap and proposed remedies are empirically validated and not reducible to the paper's own definitions.

full rationale

The paper's derivation chain is not circular. The negative result in Section 3.1 is an empirical comparison of decoding policies on fixed pretrained MDMs: the L2R reference, confidence-based policies, and random decoding all use the same token posterior, and the ASTD/TSED/Coarse metrics are evaluated against external benchmark code. The positional-uncertainty diagnosis in Section 3.2 defines m(v;x_t) and LOC(v;x_t) from the model's own posterior and then plots the observed mass/localization pattern; this is a descriptive measurement, not a fitted parameter that is later renamed as a prediction. The proposed remedies are validated on external benchmarks (HumanEval, HumanEval+, MBPP, MBPP+, GSM8K) with ablations that isolate the claimed mechanism (LatentMDM-L2R, LatentMDM-random, segment-scoring ablations, insertion-temperature sweeps), so the central claims do not reduce by construction to the definitions. The use of Kim et al. 2025a for FlexMDM is a building block, not a load-bearing self-citation: the paper extends the model theoretically, fine-tunes Dream-Coder 7B into a general-purpose FlexMDM, and evaluates downstream performance. No uniqueness theorem or ansatz is imported from the authors' prior work as an external fact. The absence of a direct intervention on a fixed MDM's decoding policy is a genuine causal-inference limitation, but it is a correctness/evidence concern, not a circularity: the claimed causal role of positional uncertainty could in principle be false or incomplete without changing any of the paper's definitions. The paper is self-contained against external benchmarks, so the honest finding is no significant circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 2 invented entities

The central claim depends on a small number of hand-chosen hyperparameters (FlexMDM schedules, insertion temperature) and on domain assumptions about how to measure structural similarity and semantic segmentation. Positional uncertainty is introduced as a conceptual construct and is not independently evidenced outside the paper's own metrics.

free parameters (4)
  • FlexMDM training schedules (a, b) = a=b=1.7
    Power-law schedules in Eq. (2) of Appendix D.1, chosen by hand to balance insertion and unmasking; the paper does not report a sweep justifying these values.
  • Inference insertion schedule exponent a~ = 2.9
    Appendix D.1: 'we use a more aggressive insertion schedule at inference time with a~=2.9' to front-load scaffolding; chosen to improve any-order behavior.
  • Insertion temperature Tins = 0.6 on MBPP/MBPP+, 1.0 on HumanEval/HE+
    Appendix D.5/Table 6: swept per benchmark and selected to maximize pass@k; this is a post-hoc fit to the evaluation sets.
  • Sampling temperatures for models/policies = 0.2 for MDMs, 1.0 for Dream-Coder baseline, 0.1 for FlexMDM
    Different temperatures for different models/policies; some choices (e.g., 1.0 for baseline) are justified for diversity but are not standardized.
axioms (4)
  • domain assumption Tree-edit similarity (ASTD/TSED/Coarse) is a meaningful measure of structural similarity between programs
    Section 3.1 and Appendix B define the metric; the claim that confidence-based decoding is close to L2R rests on this operationalization.
  • domain assumption Confidence-based decoding (top-K, margin, entropy) is the dominant inference strategy for MDMs
    Section 3.1 states this and uses it to motivate the analysis; if the field moved to other policies, the negative result would be less general.
  • standard math The unmasking posterior f_theta accurately approximates the true posterior p(x_i|z)
    Section 2 relies on this to define confidence scores and the FlexMDM/LatentMDM losses.
  • domain assumption Segmenting solutions by newline (lines of code) yields semantically meaningful units for latent prediction
    LatentMDM in Section 4.2 treats each line as a segment; the choice of segmentation is not ablated (only line-based).
invented entities (2)
  • positional uncertainty no independent evidence
    purpose: Explains why fixed-canvas token-level MDMs collapse to left-to-right decoding: the model knows what but not where, so per-position confidence is low for semantically important tokens.
    The paper provides plots of mass vs. localization, but no independent falsifiable prediction outside the model's own posteriors; it is a post-hoc construct.
  • latent segments (lines of code as latent units) no independent evidence
    purpose: Coarse-grained units for masked diffusion over code; enables segment-level order search in LatentMDM.
    The paper chooses lines as segments; this is a design choice, not an empirically validated entity.

pith-pipeline@v1.3.0-daily-deepseek · 38372 in / 12912 out tokens · 114589 ms · 2026-08-01T14:18:13.545538+00:00 · methodology

0 comments
read the original abstract

Many discrete reasoning tasks, such as code generation, are inherently non-causal: programmers move between high-level structure and local details, a process we call any-order inference. For autoregressive language models, which lack a native any-order interface, non-causal abilities such as infilling and next-edit prediction require hand-designed mechanisms. Can we instead design models that natively support any-order inference? Masked diffusion models have recently emerged as compelling candidates, as their any-order training objective naturally offers an any-order prediction interface. This interface, however, does not automatically yield any-order inference. We demonstrate that this interface-inference gap stems from positional uncertainty: fixed-canvas, token-level models may know what semantic component should appear without knowing where to place it. In light of this, we propose two complementary approaches: (1) Insertion-based masked diffusion, building on FlexMDM (Kim et al, 2025), relaxes fixed-position commitments via insertions, enabling generation across non-contiguous regions. (2) Latent-space masked diffusion shifts prediction to coarser semantic segments, enabling search over latent generation orders. Empirically, we train a 7B FlexMDM for Python coding and a 125M LatentMDM for GSM8K and show that both approaches induce distinct any-order inference behaviors and improve downstream performance. We release our codebase at https://github.com/SeunggeunKimkr/genuine-any-order.

Figures

Figures reproduced from arXiv: 2607.26504 by Jaeyeon Kim, Seunggeun Kim, Sham Kakade, Sitan Chen, Taekyun Lee, Yilun Du, Yuyuan Chen.

Figure 1
Figure 1. Figure 1: Illustrative overview of our work. Masked diffusion models cannot perform genuine any-order inference as they operate on a fixed canvas at the token level. We demonstrate that insertion-based and latent-space masked diffusion modeling overcome this limitation, enabling genuine any-order inference. Preprint. arXiv:2607.26504v1 [cs.LG] 29 Jul 2026 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: (Left) Confidence-based decoding yields code structurally close to left-to-right decoding. (Right) A simplified AST of the python function that returns -x if x < 0 and x otherwise. Code similarity metrics. To compare program structure, we use a standard AST-based tree-edit comparison. We parse each generated program with Python’s built-in ast module, yielding an abstract syntax tree (see [PITH_FULL_IMAGE:… view at source ↗
Figure 3
Figure 3. Figure 3: Positional Uncertainty. Positional uncertainty. We now formalize the intuition for po￾sitional uncertainty stated above. For a token v, we define its aggregate mass as m(v; xt) = P i: x i t=m f i θ (v | xt). This mea￾sures the total probability assigned to token v across all currently masked positions. To measure whether this mass is concentrated at a single location or dispersed across many plausible loca… view at source ↗
Figure 4
Figure 4. Figure 4: (Left) We quantify any-order inference by tracking how generation moves across the tree. (Right) FlexMDM outperforms Dream-Coder 7B on HumanEval and HumanEval+, at Pass@16. Experimental setup. Rather than training FlexMDM from scratch, we initialize from Dream-Coder 7B [Xie et al., 2025], while attaching an auxiliary head and an AdaLN time embedding [Peebles and 7 [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: (Left) LatentMDM outperforms autoregressive and other baselines at matched wall-clock time. (Right) It mitigates positional uncertainty, decoding tokens with high aggregate mass. Experimental setup. Since LatentMDM introduces fundamentally different modeling from MDMs, we had to train it from scratch. In particular, we target a setup where (1) models pretrained from scratch can achieve reasonable performan… view at source ↗
Figure 6
Figure 6. Figure 6: Additional positional-uncertainty visualizations across model families and benchmarks. [PITH_FULL_IMAGE:figures/full_fig_p025_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Step-wise positional-uncertainty visualization for DiffuCoder on MBPP. Each panel plots [PITH_FULL_IMAGE:figures/full_fig_p026_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: (Left) graphs of the insertion and unmasking schedules in (2). (Center) the number of inserted, mask, and clean tokens as a function of time during training, normalized by sequence length. (Right) the number of inserted, mask, and clean tokens as a function of time during inference, normalized by sequence length. An important practical observation beyond the original FlexMDM formulation in Kim et al. [2025… view at source ↗
Figure 9
Figure 9. Figure 9: LatentMDM training pipeline. E.2 Architecture We use a 125M-parameter architecture for LatentMDM and all token-space baselines, following the setup of [Kim et al., 2026]. All Transformer blocks follow the Qwen2-style design [Yang et al., 2025]. We modify the attention mask according to each module: the segment encoder, LatentMDM, and token-level MDM baseline use bidirectional attention, while the autoregre… view at source ↗
Figure 10
Figure 10. Figure 10: Generation trace. At each iteration, the model tentatively decodes candidate segments for all masked positions, scores them using the segment-selection criterion, and commits the highest￾scoring segment. The highlighted row indicates the segment committed at the current iteration, while previously revealed segments are shown in gray. The row color reflects the segment score, with stronger colors correspon… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

112 extracted references · 44 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2502.21075 , year=

    Spatial reasoning with denoising models , author=. arXiv preprint arXiv:2502.21075 , year=

    Pith/arXiv arXiv

  2. [2]

    International Conference on Learning Representations , year =

    Decoupled Weight Decay Regularization , author =. International Conference on Learning Representations , year =

  3. [3]

    arXiv preprint arXiv:2506.20639 , year=

    DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation , author=. arXiv preprint arXiv:2506.20639 , year=

    Pith/arXiv arXiv

  4. [4]

    arXiv preprint arXiv:2606.24773 , year=

    Posterior Refinement: Fast Language Generation via Any-Order Flow Maps , author=. arXiv preprint arXiv:2606.24773 , year=

    Pith/arXiv arXiv

  5. [5]

    Advances in neural information processing systems , volume=

    Simplified and generalized masked diffusion for discrete data , author=. Advances in neural information processing systems , volume=

  6. [6]

    2025 , eprint=

    Path Planning for Masked Diffusion Model Sampling , author=. 2025 , eprint=

    2025

  7. [7]

    arXiv preprint arXiv:2403.07974 , year=

    Livecodebench: Holistic and contamination free evaluation of large language models for code , author=. arXiv preprint arXiv:2403.07974 , year=

    Pith/arXiv arXiv

  8. [8]

    Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina , booktitle=

  9. [9]

    arXiv preprint arXiv:2406.03736 , year=

    Your absorbing discrete diffusion secretly models the conditional distributions of clean data , author=. arXiv preprint arXiv:2406.03736 , year=

    Pith/arXiv arXiv

  10. [10]

    arXiv preprint arXiv:2409.02908 , year=

    Masked diffusion models are secretly time-agnostic masked models and exploit inaccurate categorical sampling , author=. arXiv preprint arXiv:2409.02908 , year=

    Pith/arXiv arXiv

  11. [11]

    Xie, Zhihui and Ye, Jiacheng and Zheng, Lin and Gao, Jiahui and Dong, Jingwei and Wu, Zirui and Zhao, Xueliang and Gong, Shansan and Jiang, Xin and Li, Zhenguo and others , journal=

  12. [12]

    arXiv preprint arXiv:2502.09992 , year=

    Large language diffusion models , author=. arXiv preprint arXiv:2502.09992 , year=

    Pith/arXiv arXiv

  13. [13]

    arXiv preprint arXiv:2410.14157 , year=

    Beyond autoregression: Discrete diffusion for complex reasoning and planning , author=. arXiv preprint arXiv:2410.14157 , year=

    Pith/arXiv arXiv

  14. [14]

    arXiv preprint arXiv:2509.01025 , year=

    Any-Order Flexible Length Masked Diffusion , author=. arXiv preprint arXiv:2509.01025 , year=

    Pith/arXiv arXiv

  15. [15]

    arXiv preprint arXiv:2502.06768 , year=

    Train for the worst, plan for the best: Understanding token ordering in masked diffusions , author=. arXiv preprint arXiv:2502.06768 , year=

    Pith/arXiv arXiv

  16. [16]

    Advances in Neural Information Processing Systems , volume=

    Simple and effective masked diffusion language models , author=. Advances in Neural Information Processing Systems , volume=

  17. [17]

    Advances in Neural Information Processing Systems , volume=

    Discrete flow matching , author=. Advances in Neural Information Processing Systems , volume=

  18. [18]

    arXiv preprint arXiv:2502.06764 , year=

    History-guided video diffusion , author=. arXiv preprint arXiv:2502.06764 , year=

    Pith/arXiv arXiv

  19. [19]

    Advances in Neural Information Processing Systems , volume=

    Diffusion forcing: Next-token prediction meets full-sequence diffusion , author=. Advances in Neural Information Processing Systems , volume=

  20. [20]

    arXiv preprint arXiv:2512.09106 , year=

    Learning Unmasking Policies for Diffusion Language Models , author=. arXiv preprint arXiv:2512.09106 , year=

    Pith/arXiv arXiv

  21. [21]

    arXiv preprint arXiv:2505.24857 , year=

    Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking , author=. arXiv preprint arXiv:2505.24857 , year=

    Pith/arXiv arXiv

  22. [22]

    arXiv preprint arXiv:2110.14168 , year=

    Training verifiers to solve math word problems , author=. arXiv preprint arXiv:2110.14168 , year=

    Pith/arXiv arXiv

  23. [23]

    2021 , eprint=

    Evaluating Large Language Models Trained on Code , author=. 2021 , eprint=

    2021

  24. [24]

    arXiv preprint arXiv:2108.07732 , year=

    Program Synthesis with Large Language Models , author=. arXiv preprint arXiv:2108.07732 , year=

    Pith/arXiv arXiv

  25. [25]

    arXiv preprint arXiv:2508.13070 , year=

    Reinforced context order recovery for adaptive reasoning and planning , author=. arXiv preprint arXiv:2508.13070 , year=

    Pith/arXiv arXiv

  26. [26]

    2512.15745 , archivePrefix=

    Tiwei Bie and Maosong Cao and Kun Chen and Lun Du and Mingliang Gong and Zhuochen Gong and Yanmei Gu and Jiaqi Hu and Zenan Huang and Zhenzhong Lan and Chengxi Li and Chongxuan Li and Jianguo Li and Zehuan Li and Huabin Liu and Ling Liu and Guoshan Lu and Xiaocheng Lu and Yuxin Ma and Jianfeng Tan and Lanning Wei and Ji-Rong Wen and Yipeng Xing and Xiaolu...

    Pith/arXiv arXiv

  27. [27]

    arXiv preprint arXiv:2510.05725 , year=

    Improving Discrete Diffusion Unmasking Policies Beyond Explicit Reference Policies , author=. arXiv preprint arXiv:2510.05725 , year=

    arXiv

  28. [28]

    Chen, Shirui and Jiao, Jiantao and Ratliff, Lillian J and Zhu, Banghua , journal=

  29. [29]

    Wu, Chengyue and Zhang, Hao and Xue, Shuchen and Liu, Zhijian and Diao, Shizhe and Zhu, Ligeng and Luo, Ping and Han, Song and Xie, Enze , journal=. Fast-d

  30. [30]

    arXiv preprint arXiv:2511.05563 , year=

    Lookahead Unmasking Elicits Accurate Decoding in Diffusion Language Models , author=. arXiv preprint arXiv:2511.05563 , year=

    arXiv

  31. [31]

    arXiv preprint arXiv:2510.04525 , year=

    Demystifying MaskGIT Sampler and Beyond: Adaptive Order Selection in Masked Diffusion , author=. arXiv preprint arXiv:2510.04525 , year=

    Pith/arXiv arXiv

  32. [32]

    arXiv preprint arXiv:2512.04135 , year=

    Decoding Large Language Diffusion Models with Foreseeing Movement , author=. arXiv preprint arXiv:2512.04135 , year=

    arXiv

  33. [33]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages=

    Revisiting code similarity evaluation with abstract syntax tree edit distance , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages=

  34. [34]

    arXiv preprint arXiv:2504.12216 , year=

    d1: Scaling reasoning in diffusion large language models via reinforcement learning , author=. arXiv preprint arXiv:2504.12216 , year=

    Pith/arXiv arXiv

  35. [35]

    arXiv preprint arXiv:2509.21474 , year=

    d2: Improved techniques for training reasoning diffusion language models , author=. arXiv preprint arXiv:2509.21474 , year=

    Pith/arXiv arXiv

  36. [36]

    arXiv preprint arXiv:2507.08838 , year=

    wd1: Weighted policy optimization for reasoning in diffusion language models , author=. arXiv preprint arXiv:2507.08838 , year=

    arXiv

  37. [37]

    Wang, Chenyu and Rashidinejad, Paria and Su, DiJia and Jiang, Song and Wang, Sid and Zhao, Siyan and Zhou, Cai and Shen, Shannon Zejiang and Chen, Feiyu and Jaakkola, Tommi and others , journal=

  38. [38]

    arXiv preprint arXiv:2509.06949 , year=

    Revolutionizing reinforcement learning framework for diffusion large language models , author=. arXiv preprint arXiv:2509.06949 , year=

    Pith/arXiv arXiv

  39. [39]

    arXiv preprint arXiv:2505.10446 , year=

    Reinforcing the diffusion chain of lateral thought with diffusion language models , author=. arXiv preprint arXiv:2505.10446 , year=

    arXiv

  40. [40]

    arXiv preprint arXiv:2310.16834 , year=

    Discrete diffusion modeling by estimating the ratios of the data distribution , author=. arXiv preprint arXiv:2310.16834 , year=

    Pith/arXiv arXiv

  41. [41]

    arXiv preprint arXiv:2602.23225 , year=

    Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding? , author=. arXiv preprint arXiv:2602.23225 , year=

    arXiv

  42. [42]

    arXiv preprint arXiv:2604.09921 , year=

    A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models , author=. arXiv preprint arXiv:2604.09921 , year=

    Pith/arXiv arXiv

  43. [43]

    arXiv preprint arXiv:2601.15165 , year=

    The Flexibility Trap: Why Arbitrary Order Limits Reasoning Potential in Diffusion Language Models , author=. arXiv preprint arXiv:2601.15165 , year=

    Pith/arXiv arXiv

  44. [44]

    arXiv preprint arXiv:2603.04893 , year=

    Free Lunch for Pass@ k ? Low Cost Diverse Sampling for Diffusion Language Models , author=. arXiv preprint arXiv:2603.04893 , year=

    arXiv

  45. [45]

    arXiv preprint arXiv:2511.21103 , year=

    From bits to rounds: Parallel decoding with exploration for diffusion language models , author=. arXiv preprint arXiv:2511.21103 , year=

    arXiv

  46. [46]

    arXiv preprint arXiv:2604.00375 , year=

    Locally Confident, Globally Stuck: The Quality-Exploration Dilemma in Diffusion Language Models , author=. arXiv preprint arXiv:2604.00375 , year=

    arXiv

  47. [47]

    arXiv preprint arXiv:2602.17688 , year=

    AnCoder: Anchored Code Generation via Discrete Diffusion Models , author=. arXiv preprint arXiv:2602.17688 , year=

    arXiv

  48. [48]

    arXiv preprint arXiv:2505.18456 , year=

    Anchored diffusion language model , author=. arXiv preprint arXiv:2505.18456 , year=

    Pith/arXiv arXiv

  49. [49]

    arXiv preprint arXiv:2601.20339 , year=

    Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space , author=. arXiv preprint arXiv:2601.20339 , year=

    arXiv

  50. [50]

    arXiv preprint arXiv:2508.02193 , year=

    Seed diffusion: A large-scale diffusion language model with high-speed inference , author=. arXiv preprint arXiv:2508.02193 , year=

    Pith/arXiv arXiv

  51. [51]

    2025 , url =

    Gemini Diffusion , author =. 2025 , url =

    2025

  52. [52]

    arXiv preprint arXiv:2506.17298 , year=

    Mercury: Ultra-Fast Language Models Based on Diffusion , author=. arXiv preprint arXiv:2506.17298 , year=

    Pith/arXiv arXiv

  53. [53]

    Bie, Tiwei and Cao, Maosong and Cao, Xiang and Chen, Bingsen and Chen, Fuyuan and Chen, Kun and Du, Lun and Feng, Daozhuo and Feng, Haibo and Gong, Mingliang and others , journal=

  54. [54]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Denoising, fast and slow: Difficulty-aware adaptive sampling for image generation , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  55. [55]

    arXiv preprint arXiv:2602.11590 , year=

    Learn from your mistakes: Self-correcting masked diffusion models , author=. arXiv preprint arXiv:2602.11590 , year=

    Pith/arXiv arXiv

  56. [56]

    arXiv preprint arXiv:2510.01384 , year=

    Fine-tuning masked diffusion for provable self-correction , author=. arXiv preprint arXiv:2510.01384 , year=

    Pith/arXiv arXiv

  57. [57]

    arXiv preprint arXiv:2509.23653 , year=

    Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Models , author=. arXiv preprint arXiv:2509.23653 , year=

    arXiv

  58. [58]

    Advances in Neural Information Processing Systems , volume=

    Remasking discrete diffusion models with inference-time scaling , author=. Advances in Neural Information Processing Systems , volume=

  59. [59]

    arXiv preprint arXiv:2605.11125 , year=

    Language modeling with hyperspherical flows , author=. arXiv preprint arXiv:2605.11125 , year=

    Pith/arXiv arXiv

  60. [60]

    Ye, Jiacheng and Xie, Zhihui and Zheng, Lin and Gao, Jiahui and Wu, Zirui and Jiang, Xin and Li, Zhenguo and Kong, Lingpeng , journal=. Dream

  61. [61]

    Wu, Chengyue and Zhang, Hao and Xue, Shuchen and Diao, Shizhe and Fu, Yonggan and Liu, Zhijian and Molchanov, Pavlo and Luo, Ping and Han, Song and Xie, Enze , journal=. Fast-d

  62. [62]

    arXiv preprint arXiv:2602.19980 , year=

    Discrete Diffusion Models Exploit Asymmetry to Solve Lookahead Planning Tasks , author=. arXiv preprint arXiv:2602.19980 , year=

    arXiv

  63. [63]

    arXiv preprint arXiv:2511.03276 , year=

    Diffusion language models are super data learners , author=. arXiv preprint arXiv:2511.03276 , year=

    arXiv

  64. [64]

    Liu, Bingbin and Bubeck, Sebastien and Eldan, Ronen and Kulkarni, Janardhan and Li, Yuanzhi and Nguyen, Anh and Ward, Rachel and Zhang, Yi , journal=

  65. [65]

    Reward-weighted sampling: Enhancing non-autoregressive characteristics in masked diffusion

    Gwak, Daehoon and Jung, Minseo and Park, Junwoo and Park, Minho and Park, ChaeHun and Hyung, Junha and Choo, Jaegul , booktitle=. Reward-weighted sampling: Enhancing non-autoregressive characteristics in masked diffusion

  66. [66]

    Mask-predict: Parallel decoding of conditional masked language models , author=. Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) , pages=

    2019

  67. [67]

    Proceedings of the workshop on methods for optimizing and evaluating neural language generation , pages=

    BERT has a mouth, and it must speak: BERT as a Markov random field language model , author=. Proceedings of the workshop on methods for optimizing and evaluating neural language generation , pages=

  68. [68]

    Advances in neural information processing systems , volume=

    Levenshtein transformer , author=. Advances in neural information processing systems , volume=

  69. [69]

    arXiv preprint arXiv:2602.10314 , year=

    Stop Training for the Worst: Progressive Unmasking Accelerates Masked Diffusion Training , author=. arXiv preprint arXiv:2602.10314 , year=

    Pith/arXiv arXiv

  70. [70]

    Proceedings of the IEEE/CVF international conference on computer vision , pages=

    Scalable diffusion models with transformers , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=

  71. [71]

    Ahmad, Wasi Uddin and Ficek, Aleksander and Samadi, Mehrzad and Huang, Jocelyn and Noroozi, Vahid and Majumdar, Somshubra and Ginsburg, Boris , journal=

  72. [72]

    Liu, Yifei and Zhang, Li Lyna and Zhu, Yi and Dong, Bingcheng and Zhou, Xudong and Shang, Ning and Yang, Fan and Yang, Mao , journal=

  73. [73]

    Xu, Zhangchen and Liu, Yang and Yin, Yueqin and Zhou, Mingyuan and Poovendran, Radha , booktitle=

  74. [74]

    Huang, Siming and Cheng, Tianhao and Liu, Jason Klein and Xu, Weidi and Hao, Jiaran and Song, Liuyihan and Xu, Yang and Yang, Jian and Liu, Jiaheng and Zhang, Chenchen and others , booktitle=

  75. [75]

    Advances in Neural Information Processing Systems , volume=

    Diffusion of thought: Chain-of-thought reasoning in diffusion language models , author=. Advances in Neural Information Processing Systems , volume=

  76. [76]

    arXiv preprint arXiv:2604.09784 , year=

    Discrete Flow Maps , author=. arXiv preprint arXiv:2604.09784 , year=

    Pith/arXiv arXiv

  77. [77]

    arXiv preprint arXiv:2602.16813 , year=

    Flow Map Language Models: One-step Language Modeling via Continuous Denoising , author=. arXiv preprint arXiv:2602.16813 , year=

    Pith/arXiv arXiv

  78. [78]

    arXiv preprint arXiv:2602.12233 , year=

    Categorical flow maps , author=. arXiv preprint arXiv:2602.12233 , year=

    arXiv

  79. [79]

    arXiv preprint arXiv:2512.10858 , year=

    Scaling behavior of discrete diffusion language models , author=. arXiv preprint arXiv:2512.10858 , year=

    arXiv

  80. [80]

    arXiv preprint arXiv:2506.10892 , year=

    The diffusion duality , author=. arXiv preprint arXiv:2506.10892 , year=

    arXiv

Showing first 80 references.