REVIEW 13 cited by
Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Autoregressive language models, despite their impressive capabilities, struggle with complex reasoning and long-term planning tasks. We introduce discrete diffusion models as a novel solution to these challenges. Through the lens of subgoal imbalance, we demonstrate how diffusion models effectively learn difficult subgoals that elude autoregressive approaches. We propose Multi-Granularity Diffusion Modeling (MGDM), which prioritizes subgoals based on difficulty during learning. On complex tasks like Countdown, Sudoku, and Boolean Satisfiability Problems, MGDM significantly outperforms autoregressive models without using search techniques. For instance, MGDM achieves 91.5\% and 100\% accuracy on Countdown and Sudoku, respectively, compared to 45.8\% and 20.7\% for autoregressive models. Our work highlights the potential of diffusion-based approaches in advancing AI capabilities for sophisticated language understanding and problem-solving tasks. All associated codes are available at \href{https://github.com/HKUNLP/diffusion-vs-ar}{https://github.com/HKUNLP/diffusion-vs-ar}.
Forward citations
Cited by 13 Pith papers
-
The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models
Arbitrary-order generation lowers the reasoning Pass@k of diffusion LLMs; constraining RL to autoregressive order (JustGRPO) improves accuracy and preserves parallel decoding.
-
Escaping Confidence Trap: Evolutionary Decoding for Mathematical Reasoning in Diffusion LLMs
Evolutionary Decoding, a selection-and-mutation test-time strategy, improves LLaDA 2.0 mathematical reasoning on AIME and AMC benchmarks.
-
From Interface to Inference: Eliciting Any-Order Inference from Any-Order Models
Masked diffusion language models fail to exploit their any-order interface because of positional uncertainty; insertion-based (FlexMDM) and latent-segment (LatentMDM) variants recover distinct any-order inference beha...
-
Concurrent Image Understanding and Generation: Self-Correcting Coupled Markov Jump Processes
CO2Jump couples text and image denoising through cross-modal attention and remasking, achieving best joint accuracy on three concurrent-generation tasks.
-
BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving
BlockServe raises offline diffusion-LLM batch throughput 1.9–10.6× over Fast-dLLM by block-boundary eviction, mixed-state gather-scatter execution, and token-budget admission.
-
Any-Order Flexible Length Masked Diffusion
FlexMDM is a discrete diffusion model that provably supports any-order generation over variable-length sequences by learning an insertion expectation alongside the unmasking posterior, validated by length-fidelity, ma...
-
DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation
A 7B masked-diffusion code model plus complementary-mask GRPO (coupled-GRPO) improves benchmark scores and shifts decoding away from strict left-to-right order.
-
LLM-First Search: Self-Guided Exploration of the Solution Space
LLM-First Search, where the model itself decides whether to continue or backtrack during reasoning, outperforms MCTS, BestFS, and ToT-BFS on harder Countdown and Sudoku tasks while using fewer tokens.
-
Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking
EB-Sampler dynamically unmasks multiple low-entropy tokens per function evaluation, accelerating masked diffusion model sampling by 2-3x with negligible accuracy loss.
-
Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Masking
Adaptive Classifier-Free Guidance (A-CFG) re-masks low-confidence tokens in the unconditional input at each generation step, improving reasoning and planning accuracy for masked diffusion language models.
-
FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocities
A 1.5B unified multimodal model trained with discrete flow matching and metric-induced probability paths matches autoregressive baselines of similar size on generation and understanding benchmarks.
-
A Survey on Latent Reasoning
A survey that organizes latent reasoning methods into vertical recurrence, horizontal recurrence, and infinite-depth diffusion, arguing that silent reasoning can beat explicit chain-of-thought.
-
A Survey on Diffusion Language Models
A comprehensive survey of diffusion language models covering taxonomy, training and inference techniques, and comparisons with autoregressive models.
Discussion (0). Sign in to comment.