pith. sign in

arxiv: 2510.18165 · v3 · submitted 2025-10-20 · 💻 cs.AI · cs.CL· cs.LG· cs.SE

Saber: An Efficient Sampling with Adaptive Acceleration and Backtracking Enhanced Remasking for Diffusion Language Model

Yihong Dong , Zhaoyu Ma , Xue Jiang , Zhiyuan Fan , Jiaru Qian , Yongmin Li , Jianha Xiao , Zhi Jin
show 5 more authors
Rongyu Cao Binhua Li Fei Huang Yongbin Li Ge Li
This is my paper

Pith reviewed 2026-05-18 05:27 UTC · model grok-4.3

classification 💻 cs.AI cs.CLcs.LGcs.SE
keywords diffusion language modelscode generationsampling algorithmadaptive accelerationbacktracking remaskinginference speedupparallel generation
0
0 comments X

The pith

Saber uses per-token confidence to adaptively unmask tokens and backtrack on dropped-confidence errors, delivering both faster inference and higher accuracy than prior diffusion language model sampling.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Diffusion language models generate text in parallel but hit a sharp speed-quality trade-off on code because generation difficulty varies across positions and early high-confidence tokens can become errors once later context appears. Saber counters both problems with a training-free procedure that raises or lowers the number of tokens revealed each step according to current confidence and that re-masks any token whose confidence falls when new tokens arrive. Experiments on standard code-generation benchmarks show the method raises Pass@1 by 1.9 percent on average while cutting inference time by a factor of roughly 3.5. The paper supplies a theoretical argument that the adaptive and backtracking rules reduce expected error accumulation under the diffusion process. If the approach generalizes, diffusion models could close more of the remaining gap with autoregressive systems on structured tasks without requiring additional training.

Core claim

Saber is a sampling procedure that first measures the model's per-step token-wise confidence, then chooses a variable number of tokens to unmask according to that distribution and, when later tokens lower an earlier token's confidence, re-masks the low-confidence token and re-samples it; the combination yields both higher Pass@1 accuracy and substantially fewer total sampling steps on code-generation tasks.

What carries the argument

Adaptive unmasking rate plus backtracking-enhanced remasking driven by evolving per-token confidence scores.

If this is right

  • Code-generation latency drops enough to make diffusion models practical for interactive use.
  • The same sampling rule can be applied to any other diffusion language model without retraining.
  • Structured-sequence tasks that penalize early irreversible mistakes become more suitable for diffusion-style generation.
  • Theoretical analysis indicates that the expected number of error corrections decreases as context length grows.
  • Overall wall-clock time for generating complete programs is reduced while final program correctness rises.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same confidence-driven backtracking idea could be tested on non-code structured outputs such as mathematical derivations or formal proofs.
  • If confidence signals remain stable across model scales, the method may reduce the need for very large numbers of sampling steps even on long documents.
  • The approach suggests a general principle for any iterative generative process: allow reversible corrections whenever later information revises earlier beliefs.
  • Because the algorithm is training-free, it can be dropped into existing diffusion checkpoints immediately.

Load-bearing premise

The model's raw per-token confidence values give a cheap and reliable enough signal both to choose how many tokens to reveal next and to decide which earlier tokens should be rolled back.

What would settle it

Running the same diffusion language model on the same code benchmarks with the backtracking and adaptive-rate modules turned off and measuring whether both the accuracy gain and the speedup disappear.

Figures

Figures reproduced from arXiv: 2510.18165 by Binhua Li, Fei Huang, Ge Li, Jianha Xiao, Jiaru Qian, Rongyu Cao, Xue Jiang, Yihong Dong, Yongbin Li, Yongmin Li, Zhaoyu Ma, Zhi Jin, Zhiyuan Fan.

Figure 1
Figure 1. Figure 1: Left: Illustration of DLM Sampling. Right: The trade-off of DLM Sampling between [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Motivation Example. Left: (a) Average confidence per step of DLM sampling. RightL (b) [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: An Overview of Saber in DLM sampling, which consists of two key components, i.e., [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Case Study. B Detailed Experimental Setup B.1 Datasets We conduct experiments on five code generation datasets to demonstrate the effectiveness of Saber, including HumanEval (Chen et al., 2021b), MBPP (Austin et al., 2021b), HumanEval-ET and MBPP-ET (Dong et al., 2023a), and LiveCodeBench (Jain et al., 2024). For all datasets, tasks are presented in a zero-shot format. • HumanEval is a widely used benchmar… view at source ↗
read the original abstract

Diffusion language models (DLMs) are emerging as a compelling alternative to the dominant autoregressive paradigm, offering inherent advantages in parallel generation and bidirectional context modeling. However, for the tasks with strict structural constraints such as code generation, DLMs face a critical trade-off between inference speed and output quality, where accelerating generation by reducing sampling steps often leads to catastrophic performance collapse. We find that the fundamental reasons are: 1) the generation difficulty is non-uniform in the structured sequence decoding steps, making DLM's static acceleration strategy suboptimal; 2) the context of tokens generated by DLM evolves continuously, causing early high-confidence predictions to turn into irreversible errors. In this paper, we introduce efficient Sampling with Adaptive acceleration and Backtracking Enhanced Remasking (i.e., Saber), a novel training-free sampling algorithm for DLMs that first achieves both better inference speed and output quality in code generation. Saber dynamically adjusts the number of tokens unmasked per step based on the model's evolving confidence, and utilizes a backtracking mechanism to revert tokens whose confidence drops as new context emerges, with its effectiveness supported by theoretical analysis. Extensive experiments on multiple mainstream code generation benchmarks show that Saber boosts Pass@1 accuracy by an average of 1.9\% over mainstream DLM sampling methods, while achieving an average 251.4\% inference speedup. By leveraging the inherent advantages of DLMs, our work significantly narrows the performance gap with autoregressive models in code generation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper introduces Saber, a training-free sampling algorithm for diffusion language models (DLMs) in code generation. It dynamically adjusts the number of tokens unmasked per step according to the model's evolving per-token confidence scores and incorporates a backtracking mechanism to revert tokens whose confidence drops as new context emerges. The central empirical claims are an average 1.9% boost in Pass@1 accuracy and 251.4% inference speedup over mainstream DLM sampling methods across code generation benchmarks, with effectiveness supported by theoretical analysis.

Significance. If the empirical results and theoretical support hold under rigorous validation, this would be a meaningful contribution by demonstrating that adaptive, training-free modifications can simultaneously improve speed and quality in DLMs for structurally constrained tasks, helping close the gap with autoregressive models while preserving the parallel generation advantages of the diffusion paradigm.

major comments (3)
  1. [Abstract and §3] Abstract and §3 (Method): The central claim that adaptive unmasking and backtracking together deliver both higher Pass@1 and 251.4% speedup rests on the unverified premise that per-token confidence scores are a reliable, low-overhead signal for rate adjustment and error correction; no quantitative breakdown of backtracking frequency, correlation with final token correctness, or added step overhead is supplied, which directly affects whether the dual improvement can be sustained.
  2. [§5] §5 (Experiments): The reported average gains of 1.9% Pass@1 and 251.4% speedup are presented without error bars, variance across runs, ablation studies isolating adaptive acceleration from backtracking, or a complete experimental protocol (seeds, number of trials, exact benchmarks); these omissions make it impossible to assess robustness of the load-bearing performance claims.
  3. [§3.3] §3.3 (Theoretical Analysis): The abstract invokes 'theoretical analysis' to justify why the method avoids catastrophic collapse under acceleration, yet the manuscript provides no explicit derivation, bound, or formal argument linking the adaptive rule to non-uniform generation difficulty; this weakens the foundation for the claimed improvements.
minor comments (2)
  1. Ensure all tables reporting speedups include standard deviations or confidence intervals and clearly label the baseline methods being compared.
  2. [§3] Clarify the exact definition and normalization of the per-token confidence score used for both adaptive unmasking and backtracking triggers.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for their constructive feedback on our manuscript. We have carefully considered each major comment and provide point-by-point responses below. Where appropriate, we will revise the manuscript to address the concerns raised.

read point-by-point responses

  1. Referee: [Abstract and §3] Abstract and §3 (Method): The central claim that adaptive unmasking and backtracking together deliver both higher Pass@1 and 251.4% speedup rests on the unverified premise that per-token confidence scores are a reliable, low-overhead signal for rate adjustment and error correction; no quantitative breakdown of backtracking frequency, correlation with final token correctness, or added step overhead is supplied, which directly affects whether the dual improvement can be sustained.

    Authors: We agree that providing quantitative evidence for the reliability of per-token confidence scores would better support our claims. In the revised manuscript, we will add an analysis in Section 3 detailing the frequency of backtracking events, the correlation between confidence scores and token correctness, and the computational overhead of the backtracking mechanism. This will help demonstrate that the dual improvements in speed and accuracy are sustainable. revision: yes

  2. Referee: [§5] §5 (Experiments): The reported average gains of 1.9% Pass@1 and 251.4% speedup are presented without error bars, variance across runs, ablation studies isolating adaptive acceleration from backtracking, or a complete experimental protocol (seeds, number of trials, exact benchmarks); these omissions make it impossible to assess robustness of the load-bearing performance claims.

    Authors: We acknowledge the importance of statistical rigor in reporting results. We will update Section 5 to include error bars and variance measures from multiple runs with specified random seeds, conduct ablation studies to isolate the effects of adaptive acceleration and backtracking, and provide a complete experimental protocol detailing the number of trials, seeds, and exact benchmark setups used. revision: yes

  3. Referee: [§3.3] §3.3 (Theoretical Analysis): The abstract invokes 'theoretical analysis' to justify why the method avoids catastrophic collapse under acceleration, yet the manuscript provides no explicit derivation, bound, or formal argument linking the adaptive rule to non-uniform generation difficulty; this weakens the foundation for the claimed improvements.

    Authors: The current §3.3 offers a conceptual explanation based on non-uniform generation difficulty and context evolution. To address this, we will revise the section to include a more formal argument or bound that links the adaptive unmasking rule to preventing performance collapse, thereby strengthening the theoretical foundation. revision: partial

Circularity Check

0 steps flagged

No significant circularity in the algorithmic derivation.

full rationale

The paper defines Saber as a training-free procedural algorithm whose adaptive unmasking rate and backtracking trigger are specified directly from per-token confidence scores in a non-self-referential manner. The central performance claims are empirical results from benchmark experiments rather than any derived prediction or first-principles quantity that reduces to the method's own inputs by construction. No equations, fitted parameters, or self-citations are shown to bear the load of the speedup or accuracy improvements, and the invoked theoretical analysis is presented only as supporting justification without visible reduction to the algorithm definition itself. The derivation chain is therefore self-contained.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that model confidence is a monotonic proxy for prediction correctness that can be used for both step-size control and error correction. No free parameters or invented entities are explicitly introduced in the abstract.

axioms (1)
  • domain assumption Model per-token confidence scores correlate sufficiently with actual correctness to guide adaptive unmasking and backtracking decisions.
    Invoked when describing dynamic adjustment of tokens unmasked per step and the backtracking mechanism.

pith-pipeline@v0.9.0 · 5843 in / 1368 out tokens · 40834 ms · 2026-05-18T05:27:53.609156+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Evaluating the Formal Reasoning Capabilities of Large Language Models through Chomsky Hierarchy

    cs.CL 2026-04 unverdicted novelty 7.0

    LLMs display clear performance stratification on formal language tasks aligned with Chomsky hierarchy complexity levels, limited by severe efficiency barriers rather than absolute capability.

Reference graph

Works this paper leans on

29 extracted references · 29 canonical work pages · cited by 1 Pith paper · 15 internal anchors

  1. [1]

    Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg

    Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Struc- tured denoising diffusion models in discrete state-spaces.ArXiv, abs/2107.03006, 2021a. URL https://api.semanticscholar.org/CorpusID:235755106. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Mich...

    work page arXiv

  2. [2]

    URLhttps://api.semanticscholar.org/CorpusID:279070422. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert-V oss, Gretchen Krueger, T. J. Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff Wu, Clemens Winter...

    work page internal anchor Pith review Pith/arXiv arXiv 2005

  3. [3]

    URLhttps://api.semanticscholar.org/ CorpusID:218971783. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pond ´e de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mi...

    work page internal anchor Pith review Pith/arXiv arXiv

  4. [4]

    Yihong Dong, Jiazheng Ding, Xue Jiang, Ge Li, Zhuo Li, and Zhi Jin

    URLhttps://deepmind.google/models/ gemini-diffusion/. Yihong Dong, Jiazheng Ding, Xue Jiang, Ge Li, Zhuo Li, and Zhi Jin. Codescore: Evaluating code generation by learning code execution.CoRR, abs/2301.09043, 2023a. Yihong Dong, Ge Li, and Zhi Jin. CODEP: grammatical seq2seq model for general-purpose code generation. InISSTA, pp. 188–198. ACM, 2023b. 10 Y...

    work page arXiv

  5. [5]

    Scaling Diffusion Language Models via Adaptation from Autoregressive Models

    URLhttps://api.semanticscholar.org/CorpusID:271571434. Shansan Gong, Shivam Agarwal, Yizhe Zhang, Jiacheng Ye, Lin Zheng, Mukai Li, Chenxin An, Peilin Zhao, Wei Bi, Jiawei Han, Hao Peng, and Lingpeng Kong. Scaling diffusion language models via adaptation from autoregressive models.ArXiv, abs/2410.17891,

    work page internal anchor Pith review arXiv

  6. [6]

    Diffu- coder: Understanding and improving masked diffusion mod- els for code generation.arXiv preprint arXiv:2506.20639,

    URLhttps: //api.semanticscholar.org/CorpusID:273532521. Shansan Gong, Ruixiang Zhang, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong, and Yizhe Zhang. Diffucoder: Understanding and improving masked diffusion models for code gen- eration.ArXiv, abs/2506.20639,

    work page arXiv

  7. [7]

    Mamba: Linear-Time Sequence Modeling with Selective State Spaces

    URLhttps://api.semanticscholar.org/ CorpusID:280012040. Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces.arXiv preprint arXiv:2312.00752,

    work page internal anchor Pith review Pith/arXiv arXiv

  8. [8]

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y . Wu, Y . K. Li, Fuli Luo, Yingfei Xiong, and Wenfeng Liang. Deepseek-coder: When the large language model meets programming - the rise of code intelligence.CoRR, abs/2401.14196,

    work page internal anchor Pith review Pith/arXiv arXiv

  9. [9]

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

    work page internal anchor Pith review Pith/arXiv arXiv

  10. [10]

    Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMs

    URLhttps://api.semanticscholar. org/CorpusID:254044147. Feng Hong, Geng Yu, Yushi Ye, Haicheng Huang, Huangjie Zheng, Ya Zhang, Yanfeng Wang, and Jiangchao Yao. Wide-in, narrow-out: Revokable decoding for efficient and effective dllms.arXiv preprint arXiv:2507.18578,

    work page arXiv

  11. [11]

    Animesh Jain, Shunting Zhang, Edward Yang, et al

    Pengcheng Huang, Shuhao Liu, Zhenghao Liu, Yukun Yan, Shuo Wang, Zulong Chen, and Tong Xiao. Pc-sampler: Position-aware calibration of decoding bias in masked diffusion models.arXiv preprint arXiv:2508.13021,

    work page arXiv

  12. [12]

    LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code

    Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code.arXiv preprint arXiv:2403.07974,

    work page internal anchor Pith review Pith/arXiv arXiv

  13. [13]

    SEED: customize large language models with sample- efficient adaptation for code generation.CoRR, abs/2403.00046, 2024a

    Xue Jiang, Yihong Dong, Zhi Jin, and Ge Li. SEED: customize large language models with sample- efficient adaptation for code generation.CoRR, abs/2403.00046, 2024a. Xue Jiang, Yihong Dong, Lecheng Wang, Zheng Fang, Qiwei Shang, Ge Li, Zhi Jin, and Wen- pin Jiao. Self-planning code generation with large language models.ACM Trans. Softw. Eng. Methodol., 33(...

    work page arXiv

  14. [14]

    Mercury: Ultra-Fast Language Models Based on Diffusion

    Samar Khanna, Siddhant Kharbanda, Shufan Li, Harshit Varma, Eric Wang, Sawyer Birnbaum, Ziyang Luo, Yanis Miraoui, Akash Palrecha, Stefano Ermon, Aditya Grover, and V olodymyr Kuleshov. Mercury: Ultra-fast language models based on diffusion.ArXiv, abs/2506.17298,

    work page internal anchor Pith review Pith/arXiv arXiv

  15. [15]

    URLhttps://api.semanticscholar.org/CorpusID:280000358. Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, Qian Liu, Evgenii Zheltonozh- skii, Terry Yue Zhuo, Thomas Wang, Olivier Dehaene, Mishig Davaadorj, Joel Lamy-Poirier, Jo˜ao Monteiro, Oleh Shliazhko, Nicola...

    work page internal anchor Pith review Pith/arXiv arXiv

  16. [16]

    Lavida: A large diffusion model for vision-language understanding.Advances in neural information process- ing systems, 2025b

    Tianyi Li, Mingda Chen, Bowei Guo, and Zhiqiang Shen. A survey on diffusion language models.ArXiv, abs/2508.10875,

    work page arXiv

  17. [17]

    Diffusion- lm improves controllable text generation

    URLhttps://api.semanticscholar.org/ CorpusID:280650266. Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang, and Tatsunori Hashimoto. Diffusion- lm improves controllable text generation.ArXiv, abs/2205.14217, 2022a. URLhttps://api. semanticscholar.org/CorpusID:249192356. Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, R ...

    work page arXiv

  18. [18]

    Plan for Speed: Dilated Scheduling for Masked Diffusion Language Models

    12 Omer Luxembourg, Haim H. Permuter, and Eliya Nachmani. Plan for speed - dilated scheduling for masked diffusion language models.ArXiv, abs/2506.19037,

    work page internal anchor Pith review Pith/arXiv arXiv

  19. [19]

    Large Language Diffusion Models

    URLhttps://api. semanticscholar.org/CorpusID:280046263. Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji- Rong Wen, and Chongxuan Li. Large language diffusion models.ArXiv, abs/2502.09992,

    work page internal anchor Pith review Pith/arXiv arXiv

  20. [20]

    Code Llama: Open Foundation Models for Code

    Baptiste Rozi`ere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, J ´er´emy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton-Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre D ´efossez, Jade Copet, Faisal Azhar, Hugo Touvron, Louis Martin, Nicolas Usunier,...

    work page internal anchor Pith review Pith/arXiv arXiv

  21. [21]

    LLaMA: Open and Efficient Foundation Language Models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth ´ee Lacroix, Baptiste Rozi `ere, Naman Goyal, Eric Hambro, Faisal Azhar, Aur’elien Rodriguez, Ar- mand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models.ArXiv, abs/2302.13971,

    work page internal anchor Pith review Pith/arXiv arXiv

  22. [22]

    S., and Kuleshov, V

    Guanghan Wang, Yair Schiff, Subham Sekhar Sahoo, and V olodymyr Kuleshov. Remasking discrete diffusion models with inference-time scaling.ArXiv, abs/2503.00307,

    work page arXiv

  23. [23]

    Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding

    Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao, Ligeng Zhu, Ping Luo, Song Han, and Enze Xie. Fast-dllm: Training-free acceleration of diffusion llm by en- abling kv cache and parallel decoding.ArXiv, abs/2505.22618,

    work page internal anchor Pith review Pith/arXiv arXiv

  24. [24]

    Dream 7B: Diffusion Large Language Models

    URLhttps://api.semanticscholar.org/CorpusID: 281080906. Jiacheng Ye, Jiahui Gao, Shansan Gong, Lin Zheng, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Beyond autoregression: Discrete diffusion for complex reasoning and planning. InThe Thirteenth International Conference on Learning Representations, 2025a. URLhttps://openreview. net/forum?id=NRYgUzSPZz. 13 Ji...

    work page internal anchor Pith review Pith/arXiv arXiv

  25. [25]

    A survey on parallel text generation: From parallel decoding to diffusion language models.arXiv preprint arXiv:2508.08712, 2025

    URLhttps://api. semanticscholar.org/CorpusID:278789456. Lingzhe Zhang, Liancheng Fang, Chiming Duan, Minghua He, Leyi Pan, Pei Xiao, Shiyu Huang, Yunpeng Zhai, Xuming Hu, Philip S. Yu, and Aiwei Liu. A survey on parallel text genera- tion: From parallel decoding to diffusion language models.ArXiv, abs/2508.08712,

    work page arXiv

  26. [26]

    Good" array. An array is considered

    URL https://api.semanticscholar.org/CorpusID:280634995. 14 A Case Study Case Study from typing import Listclass Solution:def countCompleteSubarrays(self, nums: List[int]) ->int:# Calculate the number of distinct elements# in the whole arraydistinct_count = len(set(nums))# Initialize the count of complete subarrayscount = 0# Iterate over all possible subar...

    work page 2024

  27. [27]

    There is still room for further adjustment of hyperparameters. 16 D More Related Works D.1 Code Generation Since the advent of artificial intelligence in the 1950s, code generation has been considered the Holy Grail of computer science research (Gulwani et al., 2017). With the rapid expansion of codebases and the increasing capacity of deep learning model...

    work page 2017

  28. [28]

    With the continual increase in model parameters, researchers have discovered emergent phenomena in LLMs, leading to new breakthroughs

    pre-train models for code generation tasks. With the continual increase in model parameters, researchers have discovered emergent phenomena in LLMs, leading to new breakthroughs . Against this backdrop, LLMs such as AlphaCode (Li et al., 2022b), Codex (Chen et al., 2021a), Starcoder (Li et al., 2023), CodeLlama (Rozi `ere et al., 2023), and DeepSeek Coder...

    work page 2023

  29. [29]

    have emerged. D.2 Promising Architecture for Language Modeling While the Transformer has been the foundational architecture for modern language models (Vaswani et al., 2017), the field is experiencing a significant shift with the rise of new paradigms (Dong et al., 2024c; 2025b). Mamba (Gu & Dao, 2023), leveraging a selective State Space Model, presents a...

    work page 2017