Pith.

Computation and Language · arXiv:2611.10369 v1 · submitted 7 Nov 2026 · 57 references

FEROM: Frontier Endogenous Reveal-Order Marginal Policy Optimization for Masked Diffusion LMs

Zian Su, Ziyang Huang, Kaiyuan Zhang, Xiangyu Zhang

The paper says

“Masked diffusion language models generate text by iteratively unmasking positions, with practical samplers often selecting reveal positions from the model's own logits.”

more of the abstract

The reveal order is therefore an endogenous latent variable of the rollout process with an enormous discrete latent space. Policy optimization objectives relying on pre-defined schedules or sampled trajectories introduce either bias or high variance. In this paper, we propose FEROM, Frontier Endogenous Reveal-Order Marginal Policy Optimization, which approximates policy gradients for the rollout-induced marginal response policy of masked diffusion LMs. FEROM derives a Rao-Blackwellized policy-gradient identity over latent reveal orders and expresses the resulting estimator as posterior edge occupancy on a reveal-state DAG. To make marginalization practical, we introduce Frontier Reveal Marginalization, a budgeted estimator that approximates this marginal edge structure by combining CDF-based computation of local probabilities of the endogenous reveal process with frontier expansion of high-mass partial states. Integrated into a GRPO-style objective, FEROM replaces single-path log-scores with a detached token-score surrogate over posterior-relevant reveal edges. Experiments on math and coding tasks show comparable or improved results over existing methods under matched scoring-state budgets. An offline proxy study further shows improved marginal-gradient approximation with increased scoring budgets.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 10 Nov 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions.

5 major objections · 7 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 57 references say.

The paper says, at §3.3, Eq. (6), Appendix D

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.68):

“This is an error decomposition, not a bound: exact local probabilities alone do not guarantee a small total error.”

Pith says: Objection 1 of 5 · major

The local posterior approximation ploc,θ(i|S,c,x) in Eq. (6) conditions only on the next token matching x and omits the future completion mass Z(S∪{i};c,x). The paper acknowledges this in Appendix D (lines 399–406) as an error decomposition but provides no theoretical bound or empirical measurement of how this omission degrades the marginal gradient estimate.

more of objection 1

The offline study in §5.1 compares FRM against the full conditional marginal-gradient reference, but this comparison aggregates all approximation errors (finite frontier, macro reveals, token-score surrogate, etc.) and does not isolate the error from the local posterior approximation. Without understanding this error, it is unclear whether FRM's branching decisions and occupancy estimates are reliable, especially when the future completion mass varies significantly across positions. This is a load-bearing issue because the entire FRM algorithm depends on this local approximation for guiding frontier expansion and computing contribution weights.

The paper says, at §5.1, Figure 3

The review did not quote this passage.

Pith says: Objection 2 of 5 · major

The offline study is well-designed but the results are not as clean as the narrative suggests. At small budgets (B=4), the 'One path, evenly spaced steps' baseline often matches or exceeds FEROM in cosine similarity, especially in the token-only comparison.

more of objection 2

The advantage of FEROM only becomes clear at larger budgets (B=24, 32). This is consistent with the paper's claim that frontier expansion helps with more budget, but it also shows that a simple single-path baseline is competitive in the low-budget regime that the main-table experiments use (the paper states 'two evaluated parent states per block' for math, which is a very small budget). The paper should discuss this tension: the main empirical results are obtained in a budget regime where the offline study suggests FRM may not provide a significant advantage over a single path.

The paper says, at §4, Table 1

The review did not quote this passage.

Pith says: Objection 3 of 5 · major

The main results are mixed. FEROM leads on some benchmarks (MATH-500 at 256, HumanEval at 256) but trails on others (MBPP at all lengths, HumanEval at 512). The paper hypothesizes that 'LLaDA-style confidence-based decoding limits the diversity of high-reward rollouts on this task' (line 143).

more of objection 3

This is a plausible post-hoc explanation, but it is not tested. The paper should either provide evidence for this claim (e.g., by measuring rollout diversity) or temper the claim. As it stands, the empirical support for FEROM is not uniform, and the paper's central claim of 'comparable or improved results' is accurate but weak.

The paper says, at §3.4, Eq. (10)

The review did not quote this passage.

Pith says: Objection 4 of 5 · major

The FEROM-GRPO objective uses a detached token-score surrogate (Eq. 9) that omits the reveal-order score. The paper shows in §5.2 (Figure 4) that including the order score ('Rollout trajectory + order score') leads to training collapse. This is an important negative result, but the paper does not analyze why this collapse occurs.

more of objection 4

Is it due to high variance in the order-score gradient? Numerical instability in the CDF-based probabilities? The paper should provide some analysis or at least a hypothesis. Without this, the decision to drop the order score feels ad-hoc, and the reader cannot assess whether a more stable estimator of the order score could recover the full marginal gradient.

The paper says, at §H, Table 3

The review did not quote this passage.

Pith says: Objection 5 of 5 · major

The comparison with JustGRPO is informative but incomplete. FEROM uses half the scoring budget (B/T=0.5) but still underperforms JustGRPO on all four benchmarks. The paper frames this as 'close to JustGRPO' but the gap on MBPP is 9 percentage points. The paper should discuss whether this gap is statistically significant and whether it can be closed by increasing the budget to match JustGRPO's B/T=1.0. The claim that FEROM 'achieves performance close to JustGRPO' is overstated.

7 minor objections

The paper says, at §3.2, Eq. (4)

Pith says: minor objection 1 of 7

The notation ∇θ log[rxθ(i|c,˜xS)pθ(xi|c,˜xS)] is ambiguous: it is the gradient of the log of the product, not the sum of the gradients. This is clear from context but could be written as ∇θ log ψθ(S,i;c,x) for consistency with Appendix D.

The paper says, at §5.1, Figure 3

Pith says: minor objection 2 of 7

The figure caption and legend are dense. The distinction between 'Token + order scores' and 'Token-only scores' is crucial but the caption could be clearer about what each panel shows. Consider adding a brief note in the caption explaining that the top row includes derivatives of the reveal-order probability while the bottom row does not.

show 5 more

The paper says, at Appendix D, Eq. (D.5)

Pith says: minor objection 3 of 7

The derivation of the response-conditioned posterior ηθ(i|S,c,x) is correct but the notation is heavy. A brief intuitive explanation of why the future completion mass Z(S∪{i};c,x) appears would help readability.

The paper says, at §3.3, Algorithm 1

Pith says: minor objection 4 of 7

The algorithm description is high-level and several steps are underspecified (e.g., 'Choose macro size k using remaining positions and the pre-split budget'). The details are in Appendix F, but the main text should at least give a pointer to the appendix at the start of the algorithm description.

The paper says, at §G.1

Pith says: minor objection 5 of 7

The KL coefficient is set to 0.0 (Table 2). This is a significant departure from standard GRPO and should be justified. If the reference-policy regularizer is not used, why include it in Eq. (10)?

The paper says, at §5.2, Figure 4

Pith says: minor objection 6 of 7

The y-axis label says 'Test-pass reward (%)' but the caption says 'Training reward (left) and validation reward (right)'. The left panel should be labeled 'Training reward' or the caption should be corrected.

The paper says, at General

Pith says: minor objection 7 of 7

The paper uses 'rollout-induced marginal response policy' throughout, but the term 'marginal response policy' is sometimes abbreviated to 'marginal policy'. This is fine, but the first use of 'marginal policy' in §3.1 could be clarified as 'reveal-order-marginal response policy' to avoid confusion with other types of marginalization.

What would settle it

Recompute the offline gradient-alignment study (Figure 3) with an additional baseline that replaces FRM's local posterior ploc,θ(i|S,c,x) with the true response-conditioned posterior ηθ(i|S,c,x) computed via dynamic programming over the remaining positions (as done for the marginal reference). Compare the token-only gradient cosine and MSE of this 'oracle-local' estimator against FRM and the single-path baseline. If the oracle-local estimator significantly outperforms FRM at small budgets, then the local approximation is a major error source; if FRM matches oracle-local, then the local approximation is not the bottleneck.

from the skeptic's reading

How the review went

3 readings by deepseek-v4-flash · 10 Nov 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic partly agrees 9 s · 37,295 tokens read, 920 written
  3. Referee major revision: 5 major objections, 7 minor 19 s · 37,812 tokens read, 2,014 written

Not run on this paper: the editor, the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The strongest objection

The central algorithmic contribution of FEROM is Frontier Reveal Marginalization (FRM), which approximates the posterior edge occupancies µ*_{θ,e}(c,x) needed for the Rao–Blackwellized policy gradient. The key approximation is the local next-reveal posterior ploc,θ(i|S,c,x) in Eq. 6, which conditions only on the next token matching x and uses the one-step selection probability rxθ(i|c,˜x_S) and token likelihood pθ(x_i|c,˜x_S). However, the true response-conditioned posterior ηθ(i|S,c,x) derived in Appendix D (Eq. D.5) also involves the future completion mass Z(S∪{i};c,x), which captures the probability of completing the rest of the sequence from the new state. FRM entirely omits this future term, replacing it with a local normalization over eligible positions. The paper acknowledges this in Appendix D (lines 399–406) as an error decomposition, but provides no theoretical bound or empirical measurement of how this local approximation degrades the marginal gradient estimate. The offline study in §5.1 compares FRM against the full conditional marginal-gradient reference (which includes future completion mass), but the comparison is aggregated over all approximation errors (finite frontier, macro reveals, token-score surrogate, etc.) and does not isolate the error from the local posterior approximation. Without understanding this error, it is unclear whether FRM's branching decisions and occupancy estimates are reliable, especially when the future completion mass varies significantly across candidate next positions. This is load-bearing because if the local approximation is poor, the entire FRM estimator could be biased in a way that degrades policy optimization, and the paper's empirical results (which are mixed, with FEROM trailing baselines on MBPP) do not rule this out.

The reader's weakest_assumption correctly identifies that FRM's approximation quality depends on budgeted frontier expansion and CDF-based local probabilities, and that the paper lacks theoretical guarantees. My concern drills deeper into a specific, unaddressed error source: the local posterior approximation in Eq. 6 omits the future completion mass, which is a known component of the true posterior (Appendix D). The paper acknowledges this only in an error decomposition (Appendix D lines 399–406) and does not bound or empirically isolate this error. The offline study aggregates all errors, so it cannot distinguish whether the local approximation or finite frontier coverage is the main limitation. This is a concrete, testable concern that, if confirmed, would weaken the claim that FRM provides a reliable marginal gradient approximation. I recommend CONDITIONAL (unchanged from reader) because the concern is significant but testable, and the paper's own limitations section (§7) acknowledges that 'finite frontier coverage, approximate continuation weights, macro reveals, and the token-score surrogate still limit marginal-gradient accuracy' – but does not specifically mention the omission of future completion mass, which is a distinct and potentially more fundamental issue. The agreement is 'partial' because the reader's weakest_assumption focuses on budgeted frontier expansion and CDF computation, while I identify the local posterior approximation (omission of future completion mass) as a more specific and load-bearing concern that is not explicitly listed in the paper's limitations.

The referee's report

The paper addresses a genuine and non-trivial problem: the mismatch between the sequence-level reward and the latent reveal-order trajectory in MDM RL. The theoretical framing (Rao-Blackwellization over reveal orders, DAG representation) is principled and clearly presented. The practical challenge is computational, and the FRM estimator is a reasonable engineering response. The offline study (Figure 3) is a strength, as it attempts to isolate the approximation quality of the gradient estimator. The paper is honest about its limitations (finite budget, local posterior approximation, token-score surrogate). If the claims hold, the work provides a useful perspective and a practical algorithm for a class of models that are gaining attention. However, the empirical results are mixed: FEROM leads on some benchmarks but trails on others (MBPP), and the comparison to JustGRPO (Table 3) shows FEROM underperforming at half the budget. The central weakness is that the paper does not provide a theoretical bound on the approximation error of the FRM estimator, and the offline study, while informative, does not fully isolate the error from the critical local posterior approximation (Eq.

Pith's reading of the paper (a machine's summary, not the authors')

The paper argues that when using reinforcement learning to fine-tune masked diffusion language models, the standard approach of treating the full generation trajectory (including the order in which tokens are revealed) as the policy introduces unnecessary variance. This is because the reward only depends on the final completed sequence, not on the internal reveal order. The authors propose FEROM, which targets the rollout-induced marginal response policy—the distribution over completed responses after integrating out the latent reveal order. They derive a Rao-Blackwellized policy-gradient identity that averages over reveal orders, reducing variance, and introduce a practical budgeted algorithm (Frontier Reveal Marginalization) to approximate the required posterior edge occupancies on a reveal-state DAG. Experiments on math and coding benchmarks show that FEROM achieves comparable or improved results under matched scoring-state budgets, and an offline study confirms that larger budgets improve marginal-gradient approximation.

What carries the argument. The central object is the reveal-state directed acyclic graph (DAG), where nodes are partially revealed sets and edges represent revealing a single masked position. The key identity is the decomposition of the marginal policy gradient as a sum over edges weighted by their posterior occupancy, i.e., the probability that a reveal order consistent with the completed sequence traverses that edge. This allows the gradient to be expressed as an expectation over local edge factors. To compute this in practice, Frontier Reveal Marginalization (FRM) uses a budgeted frontier expansion: it maintains a set of high-weight partial states, computes exact local reveal probabilities from CDFs of confidence scores, and uses macro reveals (scoring several tokens from one parent prediction) and selective branching to cover alternative orders under a fixed state-evaluation budget.

Core claim. FEROM establishes that the proper object for policy optimization in masked diffusion language models is the marginal response policy, which averages over the latent reveal orders generated by the model's own logits during rollout. It derives a Rao-Blackwellized policy-gradient identity showing that the marginal gradient is the conditional expectation of the trajectory gradient given the completed sequence, thereby reducing variance. To make this computationally tractable, the paper introduces Frontier Reveal Marginalization (FRM), a budgeted estimator that approximates the posterior edge occupancies on a reveal-state DAG by combining exact local reveal probabilities (computed via CDFs of confidence scores) with frontier expansion of high-mass partial states. The resulting FEROM-GRPO objective replaces the log-score of a single sampled reveal path with a detached token-score surrogate over posterior-relevant reveal edges.

The premise it rests on. The budgeted Frontier Reveal Marginalization (FRM) estimator can accurately approximate the posterior edge occupancies on the reveal-state DAG using a limited number of state evaluations, macro reveals, and selective branching, without theoretical guarantees on the approximation error.

If the paper is right (Pith's inferences, not the authors')

FEROM provides a principled way to reduce variance in policy gradients for masked diffusion LMs, potentially leading to more stable and sample-efficient RL fine-tuning.

The framework decouples the reveal-order policy from the token-prediction policy, allowing future work to co-design decoding strategies and policy optimization for better exploration and diversity.

The budgeted frontier approximation offers a practical trade-off between computational cost and gradient accuracy, which could enable RL training on longer sequences and larger models.

The Rao-Blackwellized gradient identity is general and could be applied to other latent-variable generation processes beyond masked diffusion, such as iterative refinement or insertion-based models.

The offline proxy study demonstrates that increasing the scoring budget improves the approximation of the marginal gradient, providing a clear path for scaling with compute.

The paper implicitly suggests that the common practice of using trajectory-level policy gradients for diffusion LMs is suboptimal because it attributes reward to the stochastic reveal order, which is irrelevant to the final outcome. This insight could extend to any generative model where the generation path is latent and the reward is sequence-level.

The CDF-based computation of local reveal probabilities is a clever technical contribution that makes the marginalization tractable without additional model forwards. This technique could be reused in other contexts where one needs to compute the probability of a specific token being selected under a confidence-based sampling rule.

The paper's focus on the marginal response policy raises the question of whether the reveal order itself should be optimized (e.g., for faster decoding or better exploration) or whether it is truly a nuisance variable. The authors leave this co-design to future work, but the framework provides the tools to explore that question.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Masked diffusion language models generate text by iteratively unmasking positions, with practical samplers often selecting reveal positions from the model's own logits. The reveal order is therefore an endogenous latent variable of the rollout process with an enormous discrete latent space. Policy optimization objectives relying on pre-defined schedules or sampled trajectories introduce either bias or high variance. In this paper, we propose FEROM, Frontier Endogenous Reveal-Order Marginal Policy Optimization, which approximates policy gradients for the rollout-induced marginal response policy of masked diffusion LMs. FEROM derives a Rao-Blackwellized policy-gradient identity over latent reveal orders and expresses the resulting estimator as posterior edge occupancy on a reveal-state DAG. To make marginalization practical, we introduce Frontier Reveal Marginalization, a budgeted estimator that approximates this marginal edge structure by combining CDF-based computation of local probabilities of the endogenous reveal process with frontier expansion of high-mass partial states. Integrated into a GRPO-style objective, FEROM replaces single-path log-scores with a detached token-score surrogate over posterior-relevant reveal edges. Experiments on math and coding tasks show comparable or improved results over existing methods under matched scoring-state budgets. An offline proxy study further shows improved marginal-gradient approximation with increased scoring budgets.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-10

The authors' main claim, as the classifier picked it out 0.96

“Experiments on math and coding tasks show comparable or improved results over existing methods under matched scoring-state budgets.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.27 · human participants 0.03 · controlled comparison 0.74 · randomized assignment 0.02 · states quantitative result 0.06 · compares with prior results 0.96 · mentions code or data availability 0.02

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Masked diffusion language models generate text by iteratively unmasking positions, with practical samplers often selecting reveal positions from the model's own logits. background 1.00 0.07 0.02 asserted 0.94
The reveal order is therefore an endogenous latent variable of the rollout process with an enormous discrete latent space. background 0.99 0.14 0.03 asserted 1.00
Policy optimization objectives relying on pre-defined schedules or sampled trajectories introduce either bias or high variance. gap 0.68 0.07 0.03 asserted 0.99
In this paper, we propose FEROM, Frontier Endogenous Reveal-Order Marginal Policy Optimization, which approximates policy gradients for the rollout-induced marginal response policy of masked diffusion LMs. contribution 0.98 0.93 0.19 asserted 1.00
FEROM derives a Rao-Blackwellized policy-gradient identity over latent reveal orders and expresses the resulting estimator as posterior edge occupancy on a reveal-state DAG. method 0.81 0.95 0.10 asserted 1.00
To make marginalization practical, we introduce Frontier Reveal Marginalization, a budgeted estimator that approximates this marginal edge structure by combining CDF-based computation of local probabilities of the endogenous reveal process with frontier expansion of high-mass partial states. method 0.86 0.94 0.06 asserted 1.00
Integrated into a GRPO-style objective, FEROM replaces single-path log-scores with a detached token-score surrogate over posterior-relevant reveal edges. method 0.98 0.90 0.07 asserted 1.00
Experiments on math and coding tasks show comparable or improved results over existing methods under matched scoring-state budgets. result 1.00 0.95 0.58 asserted 1.00
An offline proxy study further shows improved marginal-gradient approximation with increased scoring budgets. result 1.00 0.95 0.13 asserted 0.99

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.91

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-10: 1 request, 6,915 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (6)
Figure 1 of the paper
Figure 1. Reveal-order modeling on an example 4-position reveal-state DAG. Nodes show the still￾masked sets [n] \ S, where S is the revealed set. Each edge reveals one position. Node color denotes state occupancy, and edge color/width denotes reveal-edge occupancy. (a) Order-agnostic denoising uses training states from an exogenous masking process rather than the adaptive rollout distribution. (b) Single-trajectory modeling follows one sampled reveal path, assigning credit only to visited edges and retaining latent reveal-order variance. (c) FEROM aggregates reveal-order contributions using…
Figure 2 of the paper
Figure 2. Macro reveals and branching in FRM. FRM progressively expands the reveal-state DAG for a given sequence: (a) a macro reveal scores several target tokens from one parent prediction; (b) branching creates two continuations with different leading reveals; (c) further macro reveals advance both continuations to the same completed sequence. Filled cells mark revealed positions; colored arrows show newly added transitions. Posterior edge occupancy. The reveal orders consistent with x are weighted by their posterior Pθ(σ | c, x). Their combined probability of traversing an edge e defines…
Figure 3 of the paper
Figure 3. Budgeted marginal-gradient approximation on AceCoder. Token-plus-order (top) and token-only (bottom) estimates share a conditional marginal-gradient reference over 16 remaining positions. FEROM is tuned per budget on the displayed cohort, using one configuration across rows and metrics. Lines and bands show means and one standard deviation over 16 planner draws. around a dominant reveal path: a single path captures much of the marginal contribution, while branching uses additional budget to cover alternative orders. 5.2 Low-Budget Training Dynamics We use only two unique prompts per…
Figure 4 of the paper
Figure 4. Low-budget AceCoder training. Training reward (left) and validation reward (right) for the frontier branching and reveal-order scoring ablations. 6 Related Work Train-test mismatch of unmasking process in MDMs. Concurrent work observes that MDM generation depends on the unmasking process, so random-mask training or ELBO evaluation can mismatch the states visited by practical samplers. DUEL [45] formalizes deterministic position￾selection rules and computes exact likelihood along the induced unmasking path, while PUMA [23] reduces supervised train–test mismatch through progressive,…
Figure 5 of the paper
Figure 5. Reveal-state DAG for a 12-position block from an AceCoder completion.
Figure 6 of the paper
Figure 6. Computational cost of CDF-based FEROM. Matched block-random, memory-efficient replay (separate forwards), and shared-forward profiles. Stacked bars separate model work from CDF evaluation, macro-prefix sampling, and frontier bookkeeping. Error bars show sample standard deviations over three measured updates; the shared-forward timing difference is smaller than this variability. H.2 Continual Training from JustGRPO-Code Starting from the released JustGRPO-Code checkpoint, we compare paired continuations with native JustGRPO and FEROM using either confidence-based or left-to-right…

Figures are the authors', reproduced from the paper.

References (57)
  1. [1] Marianne Arriola, Aaron Gokaslan, Justin T Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sekhar Sahoo, and V olodymyr Kuleshov. Block diffusion: Interpolating between autoregressive and diffusion language models.arXiv preprint arXiv:2503.09573, 2025 on PithPith's reviewarXiv
  2. [2] Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces.Advances in Neural Information Processing Systems, 34:17981–17993, 2021 on Pith
  3. [3] Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models.arXiv preprint arXiv:2108.07732, 2021 on PithPith's reviewarXiv
  4. [4] Tiwei Bie, Maosong Cao, Kun Chen, Lun Du, Mingliang Gong, Zhuochen Gong, Yanmei Gu, Jiaqi Hu, Zenan Huang, Zhenzhong Lan, Chengxi Li, Chongxuan Li, Jianguo Li, Zehuan Li, Huabin Liu, Lin Liu, Guoshan Lu, Xiaocheng Lu, Yuxin Ma, Jianfeng Tan, Lanning Wei, Ji-Rong Wen, Yipeng Xi... on PithPith's reviewarXiv
  5. [5] Andrew Campbell, Jason Yim, Regina Barzilay, Tom Rainforth, and Tommi Jaakkola. Gener- ative flows on discrete state-spaces: Enabling multimodal flows with applications to protein co-design. InInternational Conference on Machine Learning, pages 5453–5512. PMLR, 2024 on Pith
  6. [6] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott G... on PithPith's reviewarXiv
  7. [7] Shirui Chen, Jiantao Jiao, Lillian J Ratliff, and Banghua Zhu. dUltra: Ultra-fast diffusion language models via reinforcement learning.arXiv preprint arXiv:2512.21446, 2025 on PitharXiv
  8. [8] Ziyu Chen, Xinbei Jiang, Peng Sun, and Tao Lin. Optimizing decoding paths in masked diffusion models by quantifying uncertainty.arXiv preprint arXiv:2512.21336, 2025 on PitharXiv
  9. [9] Shuang Cheng, Yihan Bian, Dawei Liu, Linfeng Zhang, Qian Yao, Zhongbo Tian, Wenhai Wang, Qipeng Guo, Kai Chen, Biqing Qi, et al. SDAR: A synergistic diffusion-autoregression paradigm for scalable sequence generation.arXiv preprint arXiv:2510.06303, 2025 on PitharXiv
  10. [10] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021 on PithPith's reviewarXiv
  11. [11] Nima Fathi, Torsten Scholak, and Pierre-André Noël. Unifying autoregressive and diffusion- based sequence generation.arXiv preprint arXiv:2504.06416, 2025. 11 on PitharXiv
  12. [12] Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and LingPeng Kong. DiffuSeq: Sequence to sequence text generation with diffusion models.arXiv preprint arXiv:2210.08933, 2022 on PithPith's reviewarXiv
  13. [13] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning.Nature, 645(8081):633–638, 2025 on Pith
  14. [14] Xiaochuang Han, Sachin Kumar, and Yulia Tsvetkov. SSD-LM: Semi-autoregressive simplex- based diffusion language model for text generation and modular control. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pag... on Pith
  15. [15] Marton Havasi, Brian Karrer, Itai Gat, and Ricky TQ Chen. Edit flows: Flow matching with edit operations.arXiv preprint arXiv:2506.09018, 2025 on PitharXiv
  16. [16] Haoyu He, Katrin Renz, Yong Cao, and Andreas Geiger. MDPO: Overcoming the training- inference divide of masked diffusion language models.arXiv preprint arXiv:2508.13148, 2025 on PitharXiv
  17. [17] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in Neural Information Processing Systems, 33:6840–6851, 2020 on Pith
  18. [18] Emiel Hoogeboom, Alexey A Gritsenko, Jasmijn Bastings, Ben Poole, Rianne van den Berg, and Tim Salimans. Autoregressive diffusion models.arXiv preprint arXiv:2110.02037, 2021 on PithPith's reviewarXiv
  19. [19] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022 on Pith
  20. [20] Zheng Huang, Kiran Ramnath, Yueyan Chen, Aosong Feng, Sangmin Woo, Balasubramaniam Srinivasan, Zhichao Xu, Kang Zhou, Shuai Wang, Haibo Ding, and Lin Lee Cheong. Diffusion language model inference with Monte Carlo tree search. InFindings of the Association for Computational Li... on Pith
show 37 more
  1. [21] Metod Jazbec, Theo X Olausson, Louis Béthune, Pierre Ablin, Michael Kirchhof, João Monteiro, Victor Turrisi, Jason Ramapuram, and Marco Cuturi. Learning unmasking policies for diffusion language models.arXiv preprint arXiv:2512.09106, 2025 on PithPith's reviewarXiv
  2. [22] Jaeyeon Kim, Lee Cheuk-Kit, Carles Domingo-Enrich, Yilun Du, Sham Kakade, Timothy Ngotiaoco, Sitan Chen, and Michael Albergo. Any-order flexible length masked diffusion. arXiv preprint arXiv:2509.01025, 2025 on PithPith's reviewarXiv
  3. [23] Jaeyeon Kim, Jonathan Geuter, David Alvarez-Melis, Sham Kakade, and Sitan Chen. Stop training for the worst: Progressive unmasking accelerates masked diffusion training.arXiv preprint arXiv:2602.10314, 2026 on PithPith's reviewarXiv
  4. [24] Jaeyeon Kim, Kulin Shah, Vasilis Kontonis, Sham M Kakade, and Sitan Chen. Train for the worst, plan for the best: Understanding token ordering in masked diffusions. InForty-second International Conference on Machine Learning, 2025 on Pith
  5. [25] Fanheng Kong, Jingyuan Zhang, Yahui Liu, Zirui Wu, Yu Tian, Victoria W., and Guorui Zhou. Accelerating diffusion LLM inference via local determinism propagation.arXiv preprint arXiv:2510.07081, 2025 on PitharXiv
  6. [26] Sanghyun Lee, Seungryong Kim, Jongho Park, and Dongmin Park. Lookahead unmasking elicits accurate decoding in diffusion language models.arXiv preprint arXiv:2511.05563, 2025 on PitharXiv
  7. [27] Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto. Diffusion-LM improves controllable text generation.Advances in Neural Information Process- ing Systems, 35:4328–4343, 2022 on Pith
  8. [28] Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. InThe Twelfth International Conference on Learning Representations, 2024. 12 on Pith
  9. [29] Yangzhou Liu, Yue Cao, Hao Li, Gen Luo, Zhe Chen, Weiyun Wang, Xiaobo Liang, Biqing Qi, Lijun Wu, Changyao Tian, et al. Sequential diffusion language models.arXiv preprint arXiv:2509.24007, 2025 on PitharXiv
  10. [30] Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution. InInternational Conference on Machine Learning, pages 32819–32848. PMLR, 2024 on Pith
  11. [31] Zanlin Ni, Shenzhi Wang, Yang Yue, Tianyu Yu, Weilin Zhao, Yeguo Hua, Tianyi Chen, Jun Song, Cheng Yu, Bo Zheng, and Gao Huang. The flexibility trap: Rethinking the value of arbitrary order in diffusion language models.arXiv preprint arXiv:2601.15165, 2026 on PithPith's reviewarXiv
  12. [32] Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025 on Pith
  13. [33] Jingyang Ou, Jiaqi Han, Minkai Xu, Shaoxuan Xu, Jianwen Xie, Stefano Ermon, Yi Wu, and Chongxuan Li. Principled RL for diffusion LLMs emerges from a sequence-level perspective. arXiv preprint arXiv:2512.03759, 2025 on PitharXiv
  14. [34] Jingyang Ou, Shen Nie, Kaiwen Xue, Fengqi Zhu, Jiacheng Sun, Zhenguo Li, and Chongxuan Li. Your absorbing discrete diffusion secretly models the conditional distributions of clean data. arXiv preprint arXiv:2406.03736, 2024 on PithPith's reviewarXiv
  15. [35] Leyi Pan, Shuchang Tao, Yunpeng Zhai, Zheyu Fu, Liancheng Fang, Minghua He, Lingzhe Zhang, Zhaoyang Liu, Bolin Ding, Aiwei Liu, et al. d-TreeRPO: Towards more reliable policy optimization for diffusion language models.arXiv preprint arXiv:2512.09675, 2025 on PithPith's reviewarXiv
  16. [36] Fred Zhangzhi Peng, Zachary Bezemek, Sawan Patel, Jarrid Rector-Brooks, Sherwood Yao, Avishek Joey Bose, Alexander Tong, and Pranam Chatterjee. Path planning for masked diffusion model sampling.arXiv preprint arXiv:2502.03540, 2025 on PitharXiv
  17. [37] Patrick Pynadath, Jiaxin Shi, and Ruqi Zhang. CANDI: Hybrid discrete-continuous diffusion models.arXiv preprint arXiv:2510.22510, 2025 on PithPith's reviewarXiv
  18. [38] Kevin Rojas, Jiahe Lin, Kashif Rasul, Anderson Schneider, Yuriy Nevmyvaka, Molei Tao, and Wei Deng. Improving reasoning for diffusion language models via group diffusion policy optimization.arXiv preprint arXiv:2510.08554, 2025 on PitharXiv
  19. [39] Subham Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin Chiu, Alexander Rush, and V olodymyr Kuleshov. Simple and effective masked diffusion language models.Advances in Neural Information Processing Systems, 37:130136–130184, 2024 on Pith
  20. [40] Subham Sekhar Sahoo, Justin Deschenaux, Aaron Gokaslan, Guanghan Wang, Justin Chiu, and V olodymyr Kuleshov. The diffusion duality.arXiv preprint arXiv:2506.10892, 2025 on PitharXiv
  21. [41] Yangyi Shen, Tianjian Feng, Jiaqi Han, Wen Wang, Tianlang Chen, Chunhua Shen, Jure Leskovec, and Stefano Ermon. Improving diffusion language model decoding through joint search in generation order and token space.arXiv preprint arXiv:2601.20339, 2026 on PitharXiv
  22. [42] Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, and Michalis Titsias. Simplified and generalized masked diffusion for discrete data.Advances in Neural Information Processing Systems, 37:103131–103167, 2024 on Pith
  23. [43] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456, 2020 on PithPith's reviewarXiv
  24. [44] Xiaohang Tang, Rares Dolga, Sangwoong Yoon, and Ilija Bogunovic. wd1: Weighted policy optimization for reasoning in diffusion language models.arXiv preprint arXiv:2507.08838, 2025 on PitharXiv
  25. [45] Gilad Turok, Yair Schiff, Chris De Sa, and V olodymyr Kuleshov. DUEL: Exact likelihood reshapes masked diffusion scaling laws and beyond.arXiv preprint arXiv:2603.01367, 2026. 13 on PithPith's reviewarXiv
  26. [46] Benigno Uria, Iain Murray, and Hugo Larochelle. A deep and tractable density estimator. In International Conference on Machine Learning, pages 467–475. PMLR, 2014 on Pith
  27. [47] Dimitri von Rütte, Janis Fluri, Yuhui Ding, Antonio Orvieto, Bernhard Schölkopf, and Thomas Hofmann. Generalized interpolating discrete diffusion.arXiv preprint arXiv:2503.04482, 2025 on PithPith's reviewarXiv
  28. [48] Chenyu Wang, Paria Rashidinejad, DiJia Su, Song Jiang, Sid Wang, Siyan Zhao, Cai Zhou, Shannon Zejiang Shen, Feiyu Chen, Tommi Jaakkola, et al. SPG: Sandwiched policy gradient for masked diffusion language models.arXiv preprint arXiv:2510.09541, 2025 on PithPith's reviewarXiv
  29. [49] Yinjie Wang, Ling Yang, Bowen Li, Ye Tian, Ke Shen, and Mengdi Wang. Revolutioniz- ing reinforcement learning framework for diffusion large language models.arXiv preprint arXiv:2509.06949, 2025 on PithPith's reviewarXiv
  30. [50] Shutong Wu and Jiawei Zhang. Free draft-and-verification: Toward lossless parallel decoding for diffusion large language models.arXiv preprint arXiv:2510.00294, 2025 on PithPith's reviewarXiv
  31. [51] Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream 7B: Diffusion large language models.arXiv preprint arXiv:2508.15487, 2025 on PithPith's reviewarXiv
  32. [52] Huaye Zeng, Dongfu Jiang, Haozhe Wang, Ping Nie, Xiaotong Chen, and Wenhu Chen. ACECODER: Acing coder RL via automated test-case synthesis. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors,Proceedings of the 63rd Annual Meeting of the Ass... on Pith
  33. [53] Siyan Zhao, Devaansh Gupta, Qinqing Zheng, and Aditya Grover. d1: Scaling reasoning in diffusion large language models via reinforcement learning. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025 on Pith
  34. [54] Kaiwen Zheng, Yongxin Chen, Hanzi Mao, Ming-Yu Liu, Jun Zhu, and Qinsheng Zhang. Masked diffusion models are secretly time-agnostic masked models and exploit inaccurate categorical sampling.arXiv preprint arXiv:2409.02908, 2024 on PithPith's reviewarXiv
  35. [55] Lin Zheng, Jianbo Yuan, Lei Yu, and Lingpeng Kong. A reparameterized discrete diffusion model for text generation.arXiv preprint arXiv:2302.05737, 2023 on PithPith's reviewarXiv
  36. [56] Fengqi Zhu, Rongzhen Wang, Shen Nie, Xiaolu Zhang, Chunwei Wu, Jun Hu, Jun Zhou, Jianfei Chen, Yankai Lin, Ji-Rong Wen, et al. LLaDA 1.5: Variance-reduced preference optimization for large language diffusion models.arXiv preprint arXiv:2505.19223, 2025. 14 A Additional Related... on PithPith's reviewarXiv
  37. [57] Institutional review board (IRB) approvals or equivalent for research with human subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals... on Pith

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment