Computation and Language · arXiv:2611.10369 v1 · submitted 7 Nov 2026 · 57 references
FEROM: Frontier Endogenous Reveal-Order Marginal Policy Optimization for Masked Diffusion LMs
The paper says
“Masked diffusion language models generate text by iteratively unmasking positions, with practical samplers often selecting reveal positions from the model's own logits.”
more of the abstract
The reveal order is therefore an endogenous latent variable of the rollout process with an enormous discrete latent space. Policy optimization objectives relying on pre-defined schedules or sampled trajectories introduce either bias or high variance. In this paper, we propose FEROM, Frontier Endogenous Reveal-Order Marginal Policy Optimization, which approximates policy gradients for the rollout-induced marginal response policy of masked diffusion LMs. FEROM derives a Rao-Blackwellized policy-gradient identity over latent reveal orders and expresses the resulting estimator as posterior edge occupancy on a reveal-state DAG. To make marginalization practical, we introduce Frontier Reveal Marginalization, a budgeted estimator that approximates this marginal edge structure by combining CDF-based computation of local probabilities of the endogenous reveal process with frontier expansion of high-mass partial states. Integrated into a GRPO-style objective, FEROM replaces single-path log-scores with a detached token-score surrogate over posterior-relevant reveal edges. Experiments on math and coding tasks show comparable or improved results over existing methods under matched scoring-state budgets. An offline proxy study further shows improved marginal-gradient approximation with increased scoring budgets.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 10 Nov 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions.
5 major objections · 7 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 57 references say.
The paper says, at §3.3, Eq. (6), Appendix D
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.68):
“This is an error decomposition, not a bound: exact local probabilities alone do not guarantee a small total error.”
Pith says: Objection 1 of 5 · major
The local posterior approximation ploc,θ(i|S,c,x) in Eq. (6) conditions only on the next token matching x and omits the future completion mass Z(S∪{i};c,x). The paper acknowledges this in Appendix D (lines 399–406) as an error decomposition but provides no theoretical bound or empirical measurement of how this omission degrades the marginal gradient estimate.
more of objection 1
The offline study in §5.1 compares FRM against the full conditional marginal-gradient reference, but this comparison aggregates all approximation errors (finite frontier, macro reveals, token-score surrogate, etc.) and does not isolate the error from the local posterior approximation. Without understanding this error, it is unclear whether FRM's branching decisions and occupancy estimates are reliable, especially when the future completion mass varies significantly across positions. This is a load-bearing issue because the entire FRM algorithm depends on this local approximation for guiding frontier expansion and computing contribution weights.
The paper says, at §5.1, Figure 3
The review did not quote this passage.
Pith says: Objection 2 of 5 · major
The offline study is well-designed but the results are not as clean as the narrative suggests. At small budgets (B=4), the 'One path, evenly spaced steps' baseline often matches or exceeds FEROM in cosine similarity, especially in the token-only comparison.
more of objection 2
The advantage of FEROM only becomes clear at larger budgets (B=24, 32). This is consistent with the paper's claim that frontier expansion helps with more budget, but it also shows that a simple single-path baseline is competitive in the low-budget regime that the main-table experiments use (the paper states 'two evaluated parent states per block' for math, which is a very small budget). The paper should discuss this tension: the main empirical results are obtained in a budget regime where the offline study suggests FRM may not provide a significant advantage over a single path.
The paper says, at §4, Table 1
The review did not quote this passage.
Pith says: Objection 3 of 5 · major
The main results are mixed. FEROM leads on some benchmarks (MATH-500 at 256, HumanEval at 256) but trails on others (MBPP at all lengths, HumanEval at 512). The paper hypothesizes that 'LLaDA-style confidence-based decoding limits the diversity of high-reward rollouts on this task' (line 143).
more of objection 3
This is a plausible post-hoc explanation, but it is not tested. The paper should either provide evidence for this claim (e.g., by measuring rollout diversity) or temper the claim. As it stands, the empirical support for FEROM is not uniform, and the paper's central claim of 'comparable or improved results' is accurate but weak.
The paper says, at §3.4, Eq. (10)
The review did not quote this passage.
Pith says: Objection 4 of 5 · major
The FEROM-GRPO objective uses a detached token-score surrogate (Eq. 9) that omits the reveal-order score. The paper shows in §5.2 (Figure 4) that including the order score ('Rollout trajectory + order score') leads to training collapse. This is an important negative result, but the paper does not analyze why this collapse occurs.
more of objection 4
Is it due to high variance in the order-score gradient? Numerical instability in the CDF-based probabilities? The paper should provide some analysis or at least a hypothesis. Without this, the decision to drop the order score feels ad-hoc, and the reader cannot assess whether a more stable estimator of the order score could recover the full marginal gradient.
The paper says, at §H, Table 3
The review did not quote this passage.
Pith says: Objection 5 of 5 · major
The comparison with JustGRPO is informative but incomplete. FEROM uses half the scoring budget (B/T=0.5) but still underperforms JustGRPO on all four benchmarks. The paper frames this as 'close to JustGRPO' but the gap on MBPP is 9 percentage points. The paper should discuss whether this gap is statistically significant and whether it can be closed by increasing the budget to match JustGRPO's B/T=1.0. The claim that FEROM 'achieves performance close to JustGRPO' is overstated.
7 minor objections
The paper says, at §3.2, Eq. (4)
Pith says: minor objection 1 of 7
The notation ∇θ log[rxθ(i|c,˜xS)pθ(xi|c,˜xS)] is ambiguous: it is the gradient of the log of the product, not the sum of the gradients. This is clear from context but could be written as ∇θ log ψθ(S,i;c,x) for consistency with Appendix D.
The paper says, at §5.1, Figure 3
Pith says: minor objection 2 of 7
The figure caption and legend are dense. The distinction between 'Token + order scores' and 'Token-only scores' is crucial but the caption could be clearer about what each panel shows. Consider adding a brief note in the caption explaining that the top row includes derivatives of the reveal-order probability while the bottom row does not.
show 5 more
The paper says, at Appendix D, Eq. (D.5)
Pith says: minor objection 3 of 7
The derivation of the response-conditioned posterior ηθ(i|S,c,x) is correct but the notation is heavy. A brief intuitive explanation of why the future completion mass Z(S∪{i};c,x) appears would help readability.
The paper says, at §3.3, Algorithm 1
Pith says: minor objection 4 of 7
The algorithm description is high-level and several steps are underspecified (e.g., 'Choose macro size k using remaining positions and the pre-split budget'). The details are in Appendix F, but the main text should at least give a pointer to the appendix at the start of the algorithm description.
The paper says, at §G.1
Pith says: minor objection 5 of 7
The KL coefficient is set to 0.0 (Table 2). This is a significant departure from standard GRPO and should be justified. If the reference-policy regularizer is not used, why include it in Eq. (10)?
The paper says, at §5.2, Figure 4
Pith says: minor objection 6 of 7
The y-axis label says 'Test-pass reward (%)' but the caption says 'Training reward (left) and validation reward (right)'. The left panel should be labeled 'Training reward' or the caption should be corrected.
The paper says, at General
Pith says: minor objection 7 of 7
The paper uses 'rollout-induced marginal response policy' throughout, but the term 'marginal response policy' is sometimes abbreviated to 'marginal policy'. This is fine, but the first use of 'marginal policy' in §3.1 could be clarified as 'reveal-order-marginal response policy' to avoid confusion with other types of marginalization.
What would settle it
Recompute the offline gradient-alignment study (Figure 3) with an additional baseline that replaces FRM's local posterior ploc,θ(i|S,c,x) with the true response-conditioned posterior ηθ(i|S,c,x) computed via dynamic programming over the remaining positions (as done for the marginal reference). Compare the token-only gradient cosine and MSE of this 'oracle-local' estimator against FRM and the single-path baseline. If the oracle-local estimator significantly outperforms FRM at small budgets, then the local approximation is a major error source; if FRM matches oracle-local, then the local approximation is not the bottleneck.
from the skeptic's reading
How the review went
3 readings by deepseek-v4-flash · 10 Nov 2026
- Reader holds up, with conditions
- Skeptic partly agrees
- Referee major revision: 5 major objections, 7 minor
Not run on this paper: the editor, the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The strongest objection
The central algorithmic contribution of FEROM is Frontier Reveal Marginalization (FRM), which approximates the posterior edge occupancies µ*_{θ,e}(c,x) needed for the Rao–Blackwellized policy gradient. The key approximation is the local next-reveal posterior ploc,θ(i|S,c,x) in Eq. 6, which conditions only on the next token matching x and uses the one-step selection probability rxθ(i|c,˜x_S) and token likelihood pθ(x_i|c,˜x_S). However, the true response-conditioned posterior ηθ(i|S,c,x) derived in Appendix D (Eq. D.5) also involves the future completion mass Z(S∪{i};c,x), which captures the probability of completing the rest of the sequence from the new state. FRM entirely omits this future term, replacing it with a local normalization over eligible positions. The paper acknowledges this in Appendix D (lines 399–406) as an error decomposition, but provides no theoretical bound or empirical measurement of how this local approximation degrades the marginal gradient estimate. The offline study in §5.1 compares FRM against the full conditional marginal-gradient reference (which includes future completion mass), but the comparison is aggregated over all approximation errors (finite frontier, macro reveals, token-score surrogate, etc.) and does not isolate the error from the local posterior approximation. Without understanding this error, it is unclear whether FRM's branching decisions and occupancy estimates are reliable, especially when the future completion mass varies significantly across candidate next positions. This is load-bearing because if the local approximation is poor, the entire FRM estimator could be biased in a way that degrades policy optimization, and the paper's empirical results (which are mixed, with FEROM trailing baselines on MBPP) do not rule this out.
The reader's weakest_assumption correctly identifies that FRM's approximation quality depends on budgeted frontier expansion and CDF-based local probabilities, and that the paper lacks theoretical guarantees. My concern drills deeper into a specific, unaddressed error source: the local posterior approximation in Eq. 6 omits the future completion mass, which is a known component of the true posterior (Appendix D). The paper acknowledges this only in an error decomposition (Appendix D lines 399–406) and does not bound or empirically isolate this error. The offline study aggregates all errors, so it cannot distinguish whether the local approximation or finite frontier coverage is the main limitation. This is a concrete, testable concern that, if confirmed, would weaken the claim that FRM provides a reliable marginal gradient approximation. I recommend CONDITIONAL (unchanged from reader) because the concern is significant but testable, and the paper's own limitations section (§7) acknowledges that 'finite frontier coverage, approximate continuation weights, macro reveals, and the token-score surrogate still limit marginal-gradient accuracy' – but does not specifically mention the omission of future completion mass, which is a distinct and potentially more fundamental issue. The agreement is 'partial' because the reader's weakest_assumption focuses on budgeted frontier expansion and CDF computation, while I identify the local posterior approximation (omission of future completion mass) as a more specific and load-bearing concern that is not explicitly listed in the paper's limitations.
The referee's report
The paper addresses a genuine and non-trivial problem: the mismatch between the sequence-level reward and the latent reveal-order trajectory in MDM RL. The theoretical framing (Rao-Blackwellization over reveal orders, DAG representation) is principled and clearly presented. The practical challenge is computational, and the FRM estimator is a reasonable engineering response. The offline study (Figure 3) is a strength, as it attempts to isolate the approximation quality of the gradient estimator. The paper is honest about its limitations (finite budget, local posterior approximation, token-score surrogate). If the claims hold, the work provides a useful perspective and a practical algorithm for a class of models that are gaining attention. However, the empirical results are mixed: FEROM leads on some benchmarks but trails on others (MBPP), and the comparison to JustGRPO (Table 3) shows FEROM underperforming at half the budget. The central weakness is that the paper does not provide a theoretical bound on the approximation error of the FRM estimator, and the offline study, while informative, does not fully isolate the error from the critical local posterior approximation (Eq.
Pith's reading of the paper (a machine's summary, not the authors')
The paper argues that when using reinforcement learning to fine-tune masked diffusion language models, the standard approach of treating the full generation trajectory (including the order in which tokens are revealed) as the policy introduces unnecessary variance. This is because the reward only depends on the final completed sequence, not on the internal reveal order. The authors propose FEROM, which targets the rollout-induced marginal response policy—the distribution over completed responses after integrating out the latent reveal order. They derive a Rao-Blackwellized policy-gradient identity that averages over reveal orders, reducing variance, and introduce a practical budgeted algorithm (Frontier Reveal Marginalization) to approximate the required posterior edge occupancies on a reveal-state DAG. Experiments on math and coding benchmarks show that FEROM achieves comparable or improved results under matched scoring-state budgets, and an offline study confirms that larger budgets improve marginal-gradient approximation.
What carries the argument. The central object is the reveal-state directed acyclic graph (DAG), where nodes are partially revealed sets and edges represent revealing a single masked position. The key identity is the decomposition of the marginal policy gradient as a sum over edges weighted by their posterior occupancy, i.e., the probability that a reveal order consistent with the completed sequence traverses that edge. This allows the gradient to be expressed as an expectation over local edge factors. To compute this in practice, Frontier Reveal Marginalization (FRM) uses a budgeted frontier expansion: it maintains a set of high-weight partial states, computes exact local reveal probabilities from CDFs of confidence scores, and uses macro reveals (scoring several tokens from one parent prediction) and selective branching to cover alternative orders under a fixed state-evaluation budget.
Core claim. FEROM establishes that the proper object for policy optimization in masked diffusion language models is the marginal response policy, which averages over the latent reveal orders generated by the model's own logits during rollout. It derives a Rao-Blackwellized policy-gradient identity showing that the marginal gradient is the conditional expectation of the trajectory gradient given the completed sequence, thereby reducing variance. To make this computationally tractable, the paper introduces Frontier Reveal Marginalization (FRM), a budgeted estimator that approximates the posterior edge occupancies on a reveal-state DAG by combining exact local reveal probabilities (computed via CDFs of confidence scores) with frontier expansion of high-mass partial states. The resulting FEROM-GRPO objective replaces the log-score of a single sampled reveal path with a detached token-score surrogate over posterior-relevant reveal edges.
The premise it rests on. The budgeted Frontier Reveal Marginalization (FRM) estimator can accurately approximate the posterior edge occupancies on the reveal-state DAG using a limited number of state evaluations, macro reveals, and selective branching, without theoretical guarantees on the approximation error.
If the paper is right (Pith's inferences, not the authors')
FEROM provides a principled way to reduce variance in policy gradients for masked diffusion LMs, potentially leading to more stable and sample-efficient RL fine-tuning.
The framework decouples the reveal-order policy from the token-prediction policy, allowing future work to co-design decoding strategies and policy optimization for better exploration and diversity.
The budgeted frontier approximation offers a practical trade-off between computational cost and gradient accuracy, which could enable RL training on longer sequences and larger models.
The Rao-Blackwellized gradient identity is general and could be applied to other latent-variable generation processes beyond masked diffusion, such as iterative refinement or insertion-based models.
The offline proxy study demonstrates that increasing the scoring budget improves the approximation of the marginal gradient, providing a clear path for scaling with compute.
The paper implicitly suggests that the common practice of using trajectory-level policy gradients for diffusion LMs is suboptimal because it attributes reward to the stochastic reveal order, which is irrelevant to the final outcome. This insight could extend to any generative model where the generation path is latent and the reward is sequence-level.
The CDF-based computation of local reveal probabilities is a clever technical contribution that makes the marginalization tractable without additional model forwards. This technique could be reused in other contexts where one needs to compute the probability of a specific token being selected under a confidence-based sampling rule.
The paper's focus on the marginal response policy raises the question of whether the reveal order itself should be optimized (e.g., for faster decoding or better exploration) or whether it is truly a nuisance variable. The authors leave this co-design to future work, but the framework provides the tools to explore that question.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Masked diffusion language models generate text by iteratively unmasking positions, with practical samplers often selecting reveal positions from the model's own logits. The reveal order is therefore an endogenous latent variable of the rollout process with an enormous discrete latent space. Policy optimization objectives relying on pre-defined schedules or sampled trajectories introduce either bias or high variance. In this paper, we propose FEROM, Frontier Endogenous Reveal-Order Marginal Policy Optimization, which approximates policy gradients for the rollout-induced marginal response policy of masked diffusion LMs. FEROM derives a Rao-Blackwellized policy-gradient identity over latent reveal orders and expresses the resulting estimator as posterior edge occupancy on a reveal-state DAG. To make marginalization practical, we introduce Frontier Reveal Marginalization, a budgeted estimator that approximates this marginal edge structure by combining CDF-based computation of local probabilities of the endogenous reveal process with frontier expansion of high-mass partial states. Integrated into a GRPO-style objective, FEROM replaces single-path log-scores with a detached token-score surrogate over posterior-relevant reveal edges. Experiments on math and coding tasks show comparable or improved results over existing methods under matched scoring-state budgets. An offline proxy study further shows improved marginal-gradient approximation with increased scoring budgets.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-11-10
The authors' main claim, as the classifier picked it out 0.96
“Experiments on math and coding tasks show comparable or improved results over existing methods under matched scoring-state budgets.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.27 · human participants 0.03 · controlled comparison 0.74 · randomized assignment 0.02 · states quantitative result 0.06 · compares with prior results 0.96 · mentions code or data availability 0.02
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Masked diffusion language models generate text by iteratively unmasking positions, with practical samplers often selecting reveal positions from the model's own logits. | background 1.00 | 0.07 | 0.02 | asserted 0.94 |
| The reveal order is therefore an endogenous latent variable of the rollout process with an enormous discrete latent space. | background 0.99 | 0.14 | 0.03 | asserted 1.00 |
| Policy optimization objectives relying on pre-defined schedules or sampled trajectories introduce either bias or high variance. | gap 0.68 | 0.07 | 0.03 | asserted 0.99 |
| In this paper, we propose FEROM, Frontier Endogenous Reveal-Order Marginal Policy Optimization, which approximates policy gradients for the rollout-induced marginal response policy of masked diffusion LMs. | contribution 0.98 | 0.93 | 0.19 | asserted 1.00 |
| FEROM derives a Rao-Blackwellized policy-gradient identity over latent reveal orders and expresses the resulting estimator as posterior edge occupancy on a reveal-state DAG. | method 0.81 | 0.95 | 0.10 | asserted 1.00 |
| To make marginalization practical, we introduce Frontier Reveal Marginalization, a budgeted estimator that approximates this marginal edge structure by combining CDF-based computation of local probabilities of the endogenous reveal process with frontier expansion of high-mass partial states. | method 0.86 | 0.94 | 0.06 | asserted 1.00 |
| Integrated into a GRPO-style objective, FEROM replaces single-path log-scores with a detached token-score surrogate over posterior-relevant reveal edges. | method 0.98 | 0.90 | 0.07 | asserted 1.00 |
| Experiments on math and coding tasks show comparable or improved results over existing methods under matched scoring-state budgets. | result 1.00 | 0.95 | 0.58 | asserted 1.00 |
| An offline proxy study further shows improved marginal-gradient approximation with increased scoring budgets. | result 1.00 | 0.95 | 0.13 | asserted 0.99 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.91
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-10: 1 request, 6,915 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (6)
Figures are the authors', reproduced from the paper.
References (57)
- [1] Marianne Arriola, Aaron Gokaslan, Justin T Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sekhar Sahoo, and V olodymyr Kuleshov. Block diffusion: Interpolating between autoregressive and diffusion language models.arXiv preprint arXiv:2503.09573, 2025 on PithPith's reviewarXiv
- [2] Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces.Advances in Neural Information Processing Systems, 34:17981–17993, 2021 on Pith
- [3] Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models.arXiv preprint arXiv:2108.07732, 2021 on PithPith's reviewarXiv
- [4] Tiwei Bie, Maosong Cao, Kun Chen, Lun Du, Mingliang Gong, Zhuochen Gong, Yanmei Gu, Jiaqi Hu, Zenan Huang, Zhenzhong Lan, Chengxi Li, Chongxuan Li, Jianguo Li, Zehuan Li, Huabin Liu, Lin Liu, Guoshan Lu, Xiaocheng Lu, Yuxin Ma, Jianfeng Tan, Lanning Wei, Ji-Rong Wen, Yipeng Xi... on PithPith's reviewarXiv
- [5] Andrew Campbell, Jason Yim, Regina Barzilay, Tom Rainforth, and Tommi Jaakkola. Gener- ative flows on discrete state-spaces: Enabling multimodal flows with applications to protein co-design. InInternational Conference on Machine Learning, pages 5453–5512. PMLR, 2024 on Pith
- [6] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott G... on PithPith's reviewarXiv
- [7] Shirui Chen, Jiantao Jiao, Lillian J Ratliff, and Banghua Zhu. dUltra: Ultra-fast diffusion language models via reinforcement learning.arXiv preprint arXiv:2512.21446, 2025 on PitharXiv
- [8] Ziyu Chen, Xinbei Jiang, Peng Sun, and Tao Lin. Optimizing decoding paths in masked diffusion models by quantifying uncertainty.arXiv preprint arXiv:2512.21336, 2025 on PitharXiv
- [9] Shuang Cheng, Yihan Bian, Dawei Liu, Linfeng Zhang, Qian Yao, Zhongbo Tian, Wenhai Wang, Qipeng Guo, Kai Chen, Biqing Qi, et al. SDAR: A synergistic diffusion-autoregression paradigm for scalable sequence generation.arXiv preprint arXiv:2510.06303, 2025 on PitharXiv
- [10] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021 on PithPith's reviewarXiv
- [11] Nima Fathi, Torsten Scholak, and Pierre-André Noël. Unifying autoregressive and diffusion- based sequence generation.arXiv preprint arXiv:2504.06416, 2025. 11 on PitharXiv
- [12] Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and LingPeng Kong. DiffuSeq: Sequence to sequence text generation with diffusion models.arXiv preprint arXiv:2210.08933, 2022 on PithPith's reviewarXiv
- [13] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning.Nature, 645(8081):633–638, 2025 on Pith
- [14] Xiaochuang Han, Sachin Kumar, and Yulia Tsvetkov. SSD-LM: Semi-autoregressive simplex- based diffusion language model for text generation and modular control. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pag... on Pith
- [15] Marton Havasi, Brian Karrer, Itai Gat, and Ricky TQ Chen. Edit flows: Flow matching with edit operations.arXiv preprint arXiv:2506.09018, 2025 on PitharXiv
- [16] Haoyu He, Katrin Renz, Yong Cao, and Andreas Geiger. MDPO: Overcoming the training- inference divide of masked diffusion language models.arXiv preprint arXiv:2508.13148, 2025 on PitharXiv
- [17] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in Neural Information Processing Systems, 33:6840–6851, 2020 on Pith
- [18] Emiel Hoogeboom, Alexey A Gritsenko, Jasmijn Bastings, Ben Poole, Rianne van den Berg, and Tim Salimans. Autoregressive diffusion models.arXiv preprint arXiv:2110.02037, 2021 on PithPith's reviewarXiv
- [19] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022 on Pith
- [20] Zheng Huang, Kiran Ramnath, Yueyan Chen, Aosong Feng, Sangmin Woo, Balasubramaniam Srinivasan, Zhichao Xu, Kang Zhou, Shuai Wang, Haibo Ding, and Lin Lee Cheong. Diffusion language model inference with Monte Carlo tree search. InFindings of the Association for Computational Li... on Pith
show 37 more
- [21] Metod Jazbec, Theo X Olausson, Louis Béthune, Pierre Ablin, Michael Kirchhof, João Monteiro, Victor Turrisi, Jason Ramapuram, and Marco Cuturi. Learning unmasking policies for diffusion language models.arXiv preprint arXiv:2512.09106, 2025 on PithPith's reviewarXiv
- [22] Jaeyeon Kim, Lee Cheuk-Kit, Carles Domingo-Enrich, Yilun Du, Sham Kakade, Timothy Ngotiaoco, Sitan Chen, and Michael Albergo. Any-order flexible length masked diffusion. arXiv preprint arXiv:2509.01025, 2025 on PithPith's reviewarXiv
- [23] Jaeyeon Kim, Jonathan Geuter, David Alvarez-Melis, Sham Kakade, and Sitan Chen. Stop training for the worst: Progressive unmasking accelerates masked diffusion training.arXiv preprint arXiv:2602.10314, 2026 on PithPith's reviewarXiv
- [24] Jaeyeon Kim, Kulin Shah, Vasilis Kontonis, Sham M Kakade, and Sitan Chen. Train for the worst, plan for the best: Understanding token ordering in masked diffusions. InForty-second International Conference on Machine Learning, 2025 on Pith
- [25] Fanheng Kong, Jingyuan Zhang, Yahui Liu, Zirui Wu, Yu Tian, Victoria W., and Guorui Zhou. Accelerating diffusion LLM inference via local determinism propagation.arXiv preprint arXiv:2510.07081, 2025 on PitharXiv
- [26] Sanghyun Lee, Seungryong Kim, Jongho Park, and Dongmin Park. Lookahead unmasking elicits accurate decoding in diffusion language models.arXiv preprint arXiv:2511.05563, 2025 on PitharXiv
- [27] Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto. Diffusion-LM improves controllable text generation.Advances in Neural Information Process- ing Systems, 35:4328–4343, 2022 on Pith
- [28] Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. InThe Twelfth International Conference on Learning Representations, 2024. 12 on Pith
- [29] Yangzhou Liu, Yue Cao, Hao Li, Gen Luo, Zhe Chen, Weiyun Wang, Xiaobo Liang, Biqing Qi, Lijun Wu, Changyao Tian, et al. Sequential diffusion language models.arXiv preprint arXiv:2509.24007, 2025 on PitharXiv
- [30] Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution. InInternational Conference on Machine Learning, pages 32819–32848. PMLR, 2024 on Pith
- [31] Zanlin Ni, Shenzhi Wang, Yang Yue, Tianyu Yu, Weilin Zhao, Yeguo Hua, Tianyi Chen, Jun Song, Cheng Yu, Bo Zheng, and Gao Huang. The flexibility trap: Rethinking the value of arbitrary order in diffusion language models.arXiv preprint arXiv:2601.15165, 2026 on PithPith's reviewarXiv
- [32] Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025 on Pith
- [33] Jingyang Ou, Jiaqi Han, Minkai Xu, Shaoxuan Xu, Jianwen Xie, Stefano Ermon, Yi Wu, and Chongxuan Li. Principled RL for diffusion LLMs emerges from a sequence-level perspective. arXiv preprint arXiv:2512.03759, 2025 on PitharXiv
- [34] Jingyang Ou, Shen Nie, Kaiwen Xue, Fengqi Zhu, Jiacheng Sun, Zhenguo Li, and Chongxuan Li. Your absorbing discrete diffusion secretly models the conditional distributions of clean data. arXiv preprint arXiv:2406.03736, 2024 on PithPith's reviewarXiv
- [35] Leyi Pan, Shuchang Tao, Yunpeng Zhai, Zheyu Fu, Liancheng Fang, Minghua He, Lingzhe Zhang, Zhaoyang Liu, Bolin Ding, Aiwei Liu, et al. d-TreeRPO: Towards more reliable policy optimization for diffusion language models.arXiv preprint arXiv:2512.09675, 2025 on PithPith's reviewarXiv
- [36] Fred Zhangzhi Peng, Zachary Bezemek, Sawan Patel, Jarrid Rector-Brooks, Sherwood Yao, Avishek Joey Bose, Alexander Tong, and Pranam Chatterjee. Path planning for masked diffusion model sampling.arXiv preprint arXiv:2502.03540, 2025 on PitharXiv
- [37] Patrick Pynadath, Jiaxin Shi, and Ruqi Zhang. CANDI: Hybrid discrete-continuous diffusion models.arXiv preprint arXiv:2510.22510, 2025 on PithPith's reviewarXiv
- [38] Kevin Rojas, Jiahe Lin, Kashif Rasul, Anderson Schneider, Yuriy Nevmyvaka, Molei Tao, and Wei Deng. Improving reasoning for diffusion language models via group diffusion policy optimization.arXiv preprint arXiv:2510.08554, 2025 on PitharXiv
- [39] Subham Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin Chiu, Alexander Rush, and V olodymyr Kuleshov. Simple and effective masked diffusion language models.Advances in Neural Information Processing Systems, 37:130136–130184, 2024 on Pith
- [40] Subham Sekhar Sahoo, Justin Deschenaux, Aaron Gokaslan, Guanghan Wang, Justin Chiu, and V olodymyr Kuleshov. The diffusion duality.arXiv preprint arXiv:2506.10892, 2025 on PitharXiv
- [41] Yangyi Shen, Tianjian Feng, Jiaqi Han, Wen Wang, Tianlang Chen, Chunhua Shen, Jure Leskovec, and Stefano Ermon. Improving diffusion language model decoding through joint search in generation order and token space.arXiv preprint arXiv:2601.20339, 2026 on PitharXiv
- [42] Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, and Michalis Titsias. Simplified and generalized masked diffusion for discrete data.Advances in Neural Information Processing Systems, 37:103131–103167, 2024 on Pith
- [43] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456, 2020 on PithPith's reviewarXiv
- [44] Xiaohang Tang, Rares Dolga, Sangwoong Yoon, and Ilija Bogunovic. wd1: Weighted policy optimization for reasoning in diffusion language models.arXiv preprint arXiv:2507.08838, 2025 on PitharXiv
- [45] Gilad Turok, Yair Schiff, Chris De Sa, and V olodymyr Kuleshov. DUEL: Exact likelihood reshapes masked diffusion scaling laws and beyond.arXiv preprint arXiv:2603.01367, 2026. 13 on PithPith's reviewarXiv
- [46] Benigno Uria, Iain Murray, and Hugo Larochelle. A deep and tractable density estimator. In International Conference on Machine Learning, pages 467–475. PMLR, 2014 on Pith
- [47] Dimitri von Rütte, Janis Fluri, Yuhui Ding, Antonio Orvieto, Bernhard Schölkopf, and Thomas Hofmann. Generalized interpolating discrete diffusion.arXiv preprint arXiv:2503.04482, 2025 on PithPith's reviewarXiv
- [48] Chenyu Wang, Paria Rashidinejad, DiJia Su, Song Jiang, Sid Wang, Siyan Zhao, Cai Zhou, Shannon Zejiang Shen, Feiyu Chen, Tommi Jaakkola, et al. SPG: Sandwiched policy gradient for masked diffusion language models.arXiv preprint arXiv:2510.09541, 2025 on PithPith's reviewarXiv
- [49] Yinjie Wang, Ling Yang, Bowen Li, Ye Tian, Ke Shen, and Mengdi Wang. Revolutioniz- ing reinforcement learning framework for diffusion large language models.arXiv preprint arXiv:2509.06949, 2025 on PithPith's reviewarXiv
- [50] Shutong Wu and Jiawei Zhang. Free draft-and-verification: Toward lossless parallel decoding for diffusion large language models.arXiv preprint arXiv:2510.00294, 2025 on PithPith's reviewarXiv
- [51] Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream 7B: Diffusion large language models.arXiv preprint arXiv:2508.15487, 2025 on PithPith's reviewarXiv
- [52] Huaye Zeng, Dongfu Jiang, Haozhe Wang, Ping Nie, Xiaotong Chen, and Wenhu Chen. ACECODER: Acing coder RL via automated test-case synthesis. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors,Proceedings of the 63rd Annual Meeting of the Ass... on Pith
- [53] Siyan Zhao, Devaansh Gupta, Qinqing Zheng, and Aditya Grover. d1: Scaling reasoning in diffusion large language models via reinforcement learning. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025 on Pith
- [54] Kaiwen Zheng, Yongxin Chen, Hanzi Mao, Ming-Yu Liu, Jun Zhu, and Qinsheng Zhang. Masked diffusion models are secretly time-agnostic masked models and exploit inaccurate categorical sampling.arXiv preprint arXiv:2409.02908, 2024 on PithPith's reviewarXiv
- [55] Lin Zheng, Jianbo Yuan, Lei Yu, and Lingpeng Kong. A reparameterized discrete diffusion model for text generation.arXiv preprint arXiv:2302.05737, 2023 on PithPith's reviewarXiv
- [56] Fengqi Zhu, Rongzhen Wang, Shen Nie, Xiaolu Zhang, Chunwei Wu, Jun Hu, Jun Zhou, Jianfei Chen, Yankai Lin, Ji-Rong Wen, et al. LLaDA 1.5: Variance-reduced preference optimization for large language diffusion models.arXiv preprint arXiv:2505.19223, 2025. 14 A Additional Related... on PithPith's reviewarXiv
- [57] Institutional review board (IRB) approvals or equivalent for research with human subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals... on Pith
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment