A 130M-parameter continuous bitstream diffusion model with entropy-gated Langevin sampling achieves GenPPL 59.76 on LM1B and 27.06 on OWT, closing the gap to autoregressive models at matched entropy with 256 NFEs.
Advances in Neural Information Processing Systems , year=
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
representative citing papers
Absorbing discrete diffusion models the conditional distributions of clean data; reparameterizing yields a time-independent RADD that unifies with AO-ARMs and reaches SOTA perplexity among diffusion models on zero-shot language benchmarks.
citing papers explorer
-
Towards Closing the Autoregressive Gap in Language Modeling via Entropy-Gated Continuous Bitstream Diffusion
A 130M-parameter continuous bitstream diffusion model with entropy-gated Langevin sampling achieves GenPPL 59.76 on LM1B and 27.06 on OWT, closing the gap to autoregressive models at matched entropy with 256 NFEs.
-
Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data
Absorbing discrete diffusion models the conditional distributions of clean data; reparameterizing yields a time-independent RADD that unifies with AO-ARMs and reaches SOTA perplexity among diffusion models on zero-shot language benchmarks.