Pith. sign in

REVIEW 6 cited by

Generalized Interpolating Discrete Diffusion

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2503.04482 v2 pith:KDOWQNOX submitted 2025-03-06 cs.CL cs.AIcs.LG

Dimitri von Rütte , Janis Fluri , Yuhui Ding , Antonio Orvieto , Bernhard Schölkopf , Thomas Hofmann This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords diffusiondiscretegiddachieveflexibilityinabilityinterpolatinglanguage
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

While state-of-the-art language models achieve impressive results through next-token prediction, they have inherent limitations such as the inability to revise already generated tokens. This has prompted exploration of alternative approaches such as discrete diffusion. However, masked diffusion, which has emerged as a popular choice due to its simplicity and effectiveness, reintroduces this inability to revise words. To overcome this, we generalize masked diffusion, deriving a new family of general interpolating discrete diffusion (GIDD) which offers greater flexibility in the design of the noising processes. Leveraging a novel diffusion ELBO, we achieve compute-matched state-of-the-art performance in diffusion language modeling. Exploiting GIDD's flexibility, we explore a hybrid approach combining masking and uniform noise, leading to improved sample quality and unlocking the ability for the model to correct its own mistakes, an area where autoregressive models notoriously have struggled. Code: https://github.com/dvruette/gidd/

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Scalable Non-Equivariant 3D Molecule Generation via Rotational Alignment

    cs.LG 2025-06 conditional novelty 7.0 of 10

    A rotationally aligned latent space lets non-equivariant diffusion models match equivariant model quality on 3D molecule generation.

  2. Lost in Interpolation: Why Predictive Feedback Fails in Diffusion Language Models

    cs.CL 2026-08 conditional novelty 6.0 of 10

    Spherical soft-masking (Fréchet mean + SLERP with mask-norm restoration) improves MAUVE and generative perplexity over LERP feedback in a 169M-parameter masked diffusion language model.

  3. WAM-Diff2: Hierarchical AR-to-Diffusion Distillation for Highly Efficient Autonomous Driving VLA

    cs.RO 2026-08 conditional novelty 6.0 of 10

    A three-stage distillation converts an autoregressive driving VLA into a block-causal masked diffusion model, preserving planning accuracy while decoding 2.8x faster (15.1x with optimized kernels).

  4. Hierarchical Domain Generalization

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Over infinite domains, hierarchy-uniform domain generalization is impossible for every nontrivial hypothesis class; a length-generalization bound is a property of the length hierarchy, not a hierarchy-free guarantee.

  5. Fine-Tuning Masked Diffusion for Provable Self-Correction

    cs.LG 2025-10 conditional novelty 6.0 of 10

    PRISM fine-tunes any masked diffusion model with a binary-cross-entropy loss so its new head provably estimates per-token quality p(x_i=y_i|y⊕m_i) and can remask low-quality tokens at inference.

  6. Anchored Diffusion Language Model

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Adding an anchor network that predicts important tokens before denoising improves diffusion language model perplexity and generation quality, narrowing the gap with autoregressive models.

Pith tools