Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Block-external attention is stable across diffusion steps; FlashBlock caches it, yielding up to 1.44x throughput and 1.6x lower attention time with negligible quality loss.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 04:16 UTC pith:SA2KG4OM

load-bearing objection FlashBlock caches block-external attention outputs and log-normalizers in block diffusion, recomputing only intra-block attention — a legitimate efficiency idea with plausible speedups, but the quality-neutrality claim is tied to math-domain distillation and the video tables have arithmetic inconsistencies. the 4 major comments →

arxiv 2602.05305 v3 pith:SA2KG4OM submitted 2026-02-05 cs.CV cs.AIcs.CL

FlashBlock: Attention Caching for Efficient Long-Context Block Diffusion

Zhuokun Chen , Jianfei Cai , Bohan Zhuang This is my paper
classification cs.CV cs.AIcs.CL
keywords block diffusionattention cachingcross-step redundancyKV cachelong-context inferencediffusion language modelsvideo generationsparse attention
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

FlashBlock claims that block diffusion models spend much of their compute re-attending to previously generated tokens whose attention contributions barely change as denoising proceeds. By caching the block-external attention output (the weighted sum of values and its log-normalizer) and recomputing only attention inside the current block, it reduces both attention computation and KV cache access. On diffusion language models this yields up to 1.44x token throughput and up to 1.6x lower attention time on video generation, with accuracy differences mostly under two points. The same cached residual can be layered on top of sparse attention, recovering part of the quality lost at aggressive sparsity. A short reuse-aware distillation step is needed for discrete diffusion language models; without it, quality drops sharply.

Core claim

The paper's central discovery is that in block diffusion, the attention contributed by tokens outside the current block is strongly correlated across adjacent diffusion steps, while attention within the block varies as tokens are refined or unmasked. FlashBlock therefore treats block-external attention as quasi-static state: at the first step of a block it caches the output A_out and log-normalizer L_out; in later steps it recomputes only the block-internal attention and combines the two parts in log space with a numerically stable max-subtraction formula. This preserves the diffusion process except for the reuse decision, and it makes KV access per step O(B^2) rather than O(BN). The authors

What carries the argument

Block-external attention caching with log-space composition: the cached pair (A_out, L_out) — the attention-weighted sum of values from outside the block and the log of the sum of exponentiated scores — is reused across diffusion steps while block-internal attention is recomputed, then merged by exponentiating both log-normalizers under a common max. This decomposition into stable external vs volatile internal attention is what turns the empirical observation into an algorithm, with a threshold τ controlling when reuse is allowed and a head-wise threshold for video models.

Load-bearing premise

The load-bearing premise is that block-external attention outputs stay similar enough between adjacent diffusion steps — after a short distillation on math data — that reusing them instead of recomputing them leaves quality within the reported negligible margin; if that stability fails in another domain, or at larger reuse thresholds, both the quality claims and the speedup claims weaken.

What would settle it

Compute adjacent-step cosine similarity of block-external attention for a diffusion language model on a non-math corpus, or for a video model with a temporal window wider than the fixed 12 frames used here. If similarity on a substantial fraction of heads falls below the reuse threshold while end-task quality drops by more than the roughly one-to-two-point margin reported, the stability assumption fails in that regime. A simpler check: run the inference procedure without distillation on such a corpus; the paper's own ablations predict sharp accuracy drops when reuse is not justified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • On diffusion language models, token throughput rises from 312 to 451 tokens/s at block size 4 and from 532 to 674 tokens/s at block size 8 (up to 1.44x), with accuracy differences within about 1.6 points on math and code benchmarks.
  • In long contexts, latency grows roughly half as fast as the baseline as context scales from 100k to 800k tokens, implying a theoretical speedup ceiling near 2x for this mechanism alone.
  • Attention time in video diffusion drops by about 1.6x (23.02s to 14.43s per step) on a 1.3B video model, while video benchmark scores stay comparable; end-to-end gains are limited by other pipeline components.
  • Combined with sparse attention, the cached residual improves accuracy at aggressive sparsity (e.g., +7.96 on GSM8K, +7.40 on MATH500, +9.76 pass@1 on HumanEval at 20% density) and shrinks the L1 gap between sparse and full attention outputs.
  • Reuse-aware distillation is what keeps quality intact: without it at the default threshold, AIME drops by 6.7 points and MATH500 by 5.6 points, while distilled models match the dense baseline.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The stability finding suggests the denoising trajectory of block diffusion lives in a low-dimensional attention subspace: the external context acts as a slowly varying boundary condition. A testable extension is to cache block-external attention at even lower frequency, or to adapt the reuse threshold per layer and head, which could push beyond the 2x ceiling the paper identifies.
  • The 1.44x system speedup is measured before a fused sparse+cache kernel exists; since sparse attention and external-context caching target different redundancies, a fused implementation (which the paper lists as future work) would likely yield a combined speedup beyond either mechanism alone.
  • Because distillation is trained on a math-reasoning corpus only, the 'negligible impact' guarantee is demonstrated for math-heavy and code benchmarks; transfer to non-mathematical domains is an open question readers can test by applying the published inference without retraining.
  • The per-step KV cost drops from O(BN) to O(B^2), so the gains grow with context length N relative to block size B; at fixed block size the benefit saturates as N grows, consistent with the paper's stated 2x upper bound.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. FlashBlock proposes to reduce attention cost in block diffusion models by caching the attention contribution from tokens outside the current block and recomputing only the block-internal attention. The central observation is that block-external attention outputs are stable across adjacent diffusion steps, while block-internal attention varies. The paper formalizes this as an attention decomposition (Eq. 5) and a log-space composition rule (Eq. 9), then introduces a reuse-aware distillation stage to mitigate quality loss in diffusion language models. Experiments on Trado-8B and LongLive-1.3B report throughput gains up to 1.44x, attention-time reduction up to 1.6x, and comparable benchmark scores when distillation is applied; the method is also combined with sparse attention to recover accuracy under aggressive sparsification.

Significance. The core idea is timely and the decomposition is mathematically correct: for a fixed query, Eq. (9) exactly reconstructs the full softmax output from cached external and freshly computed internal components. Implementing the caching inside a FlashAttention-style kernel is a credible engineering contribution, and the orthogonality to sparse attention opens a useful direction. However, the practical value of the method hinges on the reuse-aware distillation of Section 4.3, and the evidence for 'negligible quality impact' is not yet convincing: without distillation, large drops appear on reasoning benchmarks; with distillation, small degradations remain on out-of-distribution coding tasks. The video results contain numerical inconsistencies that further undermine confidence. If the stability property is robust and the distillation transfer is established, this could be a solid contribution; at present, the claims are broader than the evidence.

major comments (4)
  1. [Table 4 and Table 1] The headline claim of 'negligible impact on generation quality' is achieved only after reuse-aware distillation. At the default tau=2, removing distillation drops AIME from 33.33 to 26.67 (-6.66) and MATH500 from 86.00 to 80.40 (-5.60). The distillation is LoRA fine-tuning on DAPO-Math-17K (math/reasoning), and no evidence shows it transfers to other domains. In fact, Table 1 at block size 4 shows consistent small degradations on coding benchmarks (LiveCodeBench -1.15, LiveBench -0.78), which are outside the distillation distribution. The abstract's statement 'without modifying the diffusion process' is contradicted by this training step. Please either present the method as a distillation-dependent technique and bound its failure modes, or provide training-free results that meet the 'negligible impact' criterion on a diverse benchmark suite.
  2. [Table 4 / Table 1] The default threshold tau=2 is selected after observing the quality-efficiency trade-off on the same benchmarks that appear in the main evaluation (AIME, MATH500, HumanEval, MBPP in Table 4 are also reported in Table 1). This circular tuning means the 'negligible impact' claim is not a predictive result. The paper should report a separate validation set for hyperparameter selection, or provide a sensitivity analysis showing that the conclusion is robust across a range of tau values, ideally including tau chosen without access to the test benchmarks.
  3. [Table 5 / Table 6] There are arithmetic inconsistencies in the video evaluation. In Table 6(e), the dense Human Fidelity average is reported as 0.8188, but the average of the listed components (Anatomy 0.8413, Identity 0.7663, Clothes 0.9406) is 0.8494. For 'Ours', the reported average 0.7861 is exactly the Clothes component, while the mean of the three components is 0.7942. The Controllability averages in Table 6(b) also do not match the arithmetic mean of the listed submetrics. Since Table 5's 'comparable quality' claim for video rests on these numbers, please provide a clear explanation of the weighting scheme or correct the tables. If the reported averages are wrong, the video quality conclusions may change.
  4. [Section 3 / Section 4.2] The reuse of cached block-external attention A_out is an approximation when the query token has been updated between steps, because the cached statistics were computed with the old query vector. Eq. (9) is exact only for an unchanged query. The paper's stability analysis in Figure 1 is qualitative (one layer for the dLLM), and the appendix (Figure 5) does not provide aggregate statistics. This makes it hard to verify whether the approximation is safe under the default tau=2. Please quantify the approximation error (e.g., cosine similarity or L1 difference between reused and recomputed A_out for updated vs. non-updated queries) and show that tau=2 selects settings where this error is small. Without this, the correctness of the approximation for updated tokens is not established.
minor comments (5)
  1. [Abstract and Section 4.3] The phrase 'without modifying the diffusion process' is misleading given that Section 4.3 introduces LoRA-based distillation that changes model parameters. Suggest rewording to 'without modifying the diffusion schedule' or similar.
  2. [Figure 1] No colorbar or quantitative scale is provided; the visual claim of 'consistently higher similarity' would be better supported with a numerical summary across layers and heads.
  3. [Table 2] The delta columns are ambiguous: the deltas appear on the sparse-only row but indicate the change from adding 'Ours'. Please restructure the table so that the comparison is explicit.
  4. [Section 2 / Section 5] The Trado model is referenced but not fully introduced; give a one-sentence description or cite the relevant section of the appendix.
  5. [Table 5] The 'Ours' density of 55% should be defined: does it correspond to the effective fraction of attention computed after caching? Clarify how this number is computed.

Circularity Check

0 steps flagged

No significant circularity: the core attention decomposition is exact and the speedups are measured; tuning/distillation issues are correctness concerns, not circular derivation.

full rationale

The paper's derivation chain is not circular. The central decomposition (Section 4.2, Eqs. 3-5) is an exact identity: a_i = (U_i,out + U_i,in) / (Z_i,out + Z_i,in). Caching (A_out, L_out) and recomputing only block-internal attention (Eqs. 6-9) is a stated approximation whose validity is tested empirically (Figure 1, Tables 1-5), not assumed by construction. The reported efficiency gains (451/312 and 674/532 tokens/s; 23.02s -> 14.43s attention time) are measured latencies/throughputs, not fitted parameters renamed as predictions. The reuse-aware distillation (Section 4.3) is a training procedure with a KL objective; the reported accuracies are outcomes of optimization on held-out benchmarks, not equal to the training objective by construction. The choice of τ=2 is a hyperparameter selected from the Table 4 quality-efficiency trade-off, which weakens the independence of the 'negligible impact' claim but is not a circular reduction of the central result. The only overlapping-author citation (BlockVid, Zhang et al. 2025b) is contextual related work and is not load-bearing. The paper itself acknowledges that directly introducing caching without distillation causes distributional mismatch (Section 4.3) and that sparse-attention combinations are only evaluated for accuracy, not kernel-level latency (Limitations). Table 6 contains apparent arithmetic inconsistencies in VBench averages and the video attention-speedup figure includes sparse attention; these are correctness/measurement concerns, not circularity. Honest non-finding.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The paper introduces no new physical or model entities; the cached (A_out, L_out) pair is algorithmic state, not an entity. The load-bearing free parameters are tau, gamma, and the distillation hyperparameters, all of which are fitted to data or chosen by observed results. The key axioms are the stability assumption and the transferability of the distillation.

free parameters (4)
  • tau (updated-token threshold) = 2 (default)
    Controls when cached block-external attention is reused vs recomputed; chosen from the quality-efficiency tradeoff observed in Table 4 on the same benchmarks.
  • gamma (head-wise similarity threshold for video) = 0.9
    Only heads whose estimated cross-step similarity exceeds gamma reuse cached attention; gamma is set on a small pre-inference sample.
  • lambda (distillation regularization weight) = 1
    Weight of the dense-teacher KL regularizer in Eq. (12); no sensitivity analysis is reported.
  • LoRA rank / training steps / distillation dataset = rank 32, 5,000 iterations, DAPO-Math-17K
    Hyperparameters of the reuse-aware distillation that is essential to restore quality in Table 4; no sweeps shown.
axioms (5)
  • standard math Softmax attention decomposition into internal/external normalizers and weighted sums (Eq. 5) is algebraically exact.
    Used throughout Section 4.2 to justify log-space composition; this is a standard algebraic identity.
  • domain assumption Block-external attention outputs are sufficiently stable across adjacent diffusion steps that cached A_out and L_out can be reused without meaningful quality loss.
    This is the central empirical premise, supported by Figure 1 and Appendix A, but no quantitative thresholds or statistical tests are given in the main text.
  • domain assumption If fewer than tau tokens are updated at a step, the query representations change little enough to make reuse safe.
    Invoked by the selective reuse rule in Section 4.2; Table 4 shows that even tau=2 without distillation causes measurable degradation, so this assumption is only valid after adaptation.
  • domain assumption LoRA distillation on DAPO-Math-17K transfers to GSM8K, MATH500, AIME, and coding benchmarks.
    The reported quality preservation in Table 1 relies on this transfer; the paper provides no cross-domain distillation study.
  • ad hoc to paper Head-wise gamma estimated on a small pre-inference sample generalizes to the video test distribution.
    Introduced in Section 4.2 for video models without specifying the sample size or selection procedure.

pith-pipeline@v1.3.0-alltime-deepseek · 15239 in / 15909 out tokens · 167637 ms · 2026-08-03T04:16:49.776059+00:00 · methodology

0 comments
read the original abstract

Generating long-form content, such as minute-long videos and extended texts, is increasingly important for modern generative models. Block diffusion improves inference efficiency via KV caching and block-wise causal inference and has been widely adopted in diffusion language models and video generation. However, in long-context settings, block diffusion still incurs substantial overhead from repeatedly computing attention over a growing KV cache. We identify an underexplored property of block diffusion: cross-step redundancy of attention within a block. Our analysis shows that attention outputs from tokens outside the current block remain largely stable across diffusion steps, while block-internal attention varies significantly. Based on this observation, we propose FlashBlock, a cached block-external attention mechanism that reuses stable attention output, reducing attention computation and KV cache access without modifying the diffusion process. Moreover, FlashBlock is orthogonal to sparse attention and can be combined as a complementary residual reuse strategy, substantially improving model accuracy under aggressive sparsification. Experiments on diffusion language models and video generation demonstrate up to 1.44$\times$ higher token throughput and up to 1.6$\times$ reduction in attention time, with negligible impact on generation quality. Project page: https://caesarhhh.github.io/FlashBlock/.

Figures

Figures reproduced from arXiv: 2602.05305 by Bohan Zhuang, Jianfei Cai, Zhuokun Chen.

Figure 1
Figure 1. Figure 1: Cross-step stability of block-external vs. block-internal attention across diffusion steps. Visualization of attention similarity across diffusion steps for the same block at layer 3 of Trado-8B-Thinking. We compute the similarity of attention outputs between each diffusion step and its subsequent step within a denoising block. In each heatmap, the x-axis corresponds to token indices within the current blo… view at source ↗
Figure 2
Figure 2. Figure 2: Block-external attention caching for block diffusion. At each diffusion step, block diffusion updates a contiguous block of tokens. Our method caches attention contributions from block-external tokens and reuses them across steps, recomputing attention only within the current block. Block-internal and block-external attention are combined via log-space aggregation, reducing computation and memory I/O in lo… view at source ↗
Figure 3
Figure 3. Figure 3: Per-step inference latency under increasing context length. We report results on Trado with batch size 128 using two A100 GPUs. Each column corresponds to a different updated-token threshold τ ∈ {2, 3, 4}. Our method (orange) consistently reduces per-step inference latency compared to the Trado baseline (blue), with the gap widening as context length increases. Larger τ values enable more aggressive reuse … view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparison on video generation with LongLive-1.3B. We visualize video examples from VBench, each shown by six uniformly sampled frames. For each example, the top row shows results from the baseline, the mid row shows results from the SpargeAttention, and the bottom row shows results from SpargeAttention combined with our block-external attention caching at a fixed sparsity ratio. Our method pre… view at source ↗
Figure 5
Figure 5. Figure 5: Attention similarity across diffusion steps in video diffusion models. We visualize the cosine similarity of attention outputs between adjacent diffusion steps for block-internal (orange) and block-external (blue) attention components across all layers and attention heads. Each subplot corresponds to one transformer layer, with the horizontal axis indexing attention heads. 11 [PITH_FULL_IMAGE:figures/full… view at source ↗
Figure 6
Figure 6. Figure 6: Combining block-external attention caching with sparse attention. At diffusion step s (left), sparse attention selects a subset of keys (selected K/V) for explicit attention computation, while attention contributions from the remaining unselected keys are aggregated as block-external attention and cached. At the subsequent diffusion step s+1 (right), sparse attention is recomputed only over the selected ke… view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative examples on Motion Rationality. We visualize three representative video examples selected from the Motion Rationality dimension of VBench using the LongLive-1.3B model. For each example, we show uniformly sampled frames. Rows are organized in pairs, where the upper row corresponds to the dense baseline and the lower row corresponds to our accelerated method. Baseline Ours Baseline Ours Baseline… view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative examples on Material. We visualize three representative video examples selected from the Material dimension of VBench using the LongLive-1.3B model. For each example, we show uniformly sampled frames. Rows are organized in pairs, where the upper row corresponds to the dense baseline and the lower row corresponds to our accelerated method. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative examples on Dynamic Attribute. We visualize three representative video examples selected from the Dynamic Attribute dimension of VBench using the LongLive-1.3B model. For each example, we show uniformly sampled frames. Rows are organized in pairs, where the upper row corresponds to the dense baseline and the lower row corresponds to our accelerated method. Baseline Ours Baseline Ours Baseline O… view at source ↗
Figure 10
Figure 10. Figure 10: Qualitative examples on Complex Landscape. We visualize three representative video examples selected from the Complex Landscape dimension of VBench using the LongLive-1.3B model. For each example, we show uniformly sampled frames. Rows are organized in pairs, where the upper row corresponds to the dense baseline and the lower row corresponds to our accelerated method. 15 [PITH_FULL_IMAGE:figures/full_fig… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Elastic-dLLM: Position Preserving Context Compression and Augmentation of Diffusion LLMs

    cs.LG 2026-05 unverdicted novelty 6.0

    Position-preserving MASK token compression reduces redundancy in diffusion LLMs to accelerate parallel decoding and enable context folding for longer sequences.

Reference graph

Works this paper leans on

13 extracted references · 10 linked inside Pith · cited by 1 Pith paper

  1. [4]

    S., Seo, J.-s., Zhang, Z., and Gupta, U

    Hu, Z., Meng, J., Akhauri, Y ., Abdelfattah, M. S., Seo, J.-s., Zhang, Z., and Gupta, U. Accelerating diffusion language model inference via efficient kv caching and guided diffusion.arXiv preprint arXiv:2505.21467,

    arXiv

  2. [5]

    (a)Commonsense Method Motion Rat. Avg. Dense 0.4195 0.4195 Sparse (30%) 0.4770 0.4770 Sparse (40%) 0.4080 0.4080 Ours 0.4310 0.4310 (b)Controllability Method Dyn. Attr. Dyn. Spatial Motion Ord. Interaction Plot Landscape Camera Avg. Dense 0.3583 0.1941 0.3450 0.2222 0.6767 0.1254 0.2089 0.2796 Sparse (30%) 0.3417 0.2234 0.3382 0.1717 0.7000 0.1012 0.1356 ...

    1941

  3. [7]

    Revolutionizing reinforcement learning framework for diffusion large language models.arXiv preprint arXiv:2509.06949, 2025a

    Wang, Y ., Yang, L., Li, B., Tian, Y ., Shen, K., and Wang, M. Revolutionizing reinforcement learning framework for diffusion large language models.arXiv preprint arXiv:2509.06949, 2025a. Wang, Z., Fang, G., Ma, X., Yang, X., and Wang, X. Sparsed: Sparse attention for diffusion language mod- els.arXiv preprint arXiv:2509.24014, 2025b. White, C., Dooley, S...

    Pith/arXiv arXiv

  4. [8]

    Fast-dllm: Training-free acceler- ation of diffusion llm by enabling kv cache and parallel decoding.arXiv preprint arXiv:2505.22618,

    Wu, C., Zhang, H., Xue, S., Liu, Z., Diao, S., Zhu, L., Luo, P., Han, S., and Xie, E. Fast-dllm: Training-free acceler- ation of diffusion llm by enabling kv cache and parallel decoding.arXiv preprint arXiv:2505.22618,

    Pith/arXiv arXiv

  5. [9]

    Longlive: Real- time interactive long video generation.arXiv preprint arXiv:2509.22622,

    Yang, S., Huang, W., Chu, R., Xiao, Y ., Zhao, Y ., Wang, X., Li, M., Xie, E., Chen, Y ., Lu, Y ., et al. Longlive: Real- time interactive long video generation.arXiv preprint arXiv:2509.22622,

    Pith/arXiv arXiv

  6. [10]

    Dream 7b: Diffusion large language models.arXiv preprint arXiv:2508.15487,

    Ye, J., Xie, Z., Zheng, L., Gao, J., Wu, Z., Jiang, X., Li, Z., and Kong, L. Dream 7b: Diffusion large language models.arXiv preprint arXiv:2508.15487,

    Pith/arXiv arXiv

  7. [11]

    Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476,

    Yu, Q., Zhang, Z., Zhu, R., Yuan, Y ., Zuo, X., Yue, Y ., Dai, W., Fan, T., Liu, G., Liu, L., et al. Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476,

    Pith/arXiv arXiv

  8. [12]

    Blockvid: Block diffusion for high-quality and consistent minute-long video generation.arXiv preprint arXiv:2511.22973, 2025b

    Zhang, Z., Chang, S., He, Y ., Han, Y ., Tang, J., Wang, F., and Zhuang, B. Blockvid: Block diffusion for high-quality and consistent minute-long video generation.arXiv preprint arXiv:2511.22973, 2025b. Zheng, D., Huang, Z., Liu, H., Zou, K., He, Y ., Zhang, F., Zhang, Y ., He, J., Zheng, W.-S., Qiao, Y ., and Liu, Z. VBench-2.0: Advancing video generatio...

    Pith/arXiv arXiv

  9. [2021]

    E., and Cohan, A

    Beltagy, I., Peters, M. E., and Cohan, A. Long- former: The long-document transformer.arXiv preprint arXiv:2004.05150,

    Pith/arXiv arXiv 2004

  10. [2022]

    dkv-cache: The cache for diffusion language models.arXiv preprint arXiv:2505.15781,

    Ma, X., Yu, R., Fang, G., and Wang, X. dkv-cache: The cache for diffusion language models.arXiv preprint arXiv:2505.15781,

    Pith/arXiv arXiv

  11. [2023]

    Training verifiers to solve math word problems

    Cobbe, K., Kosaraju, V ., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168,

    Pith/arXiv arXiv

  12. [2024]

    T., Yang, Z., Qi, Z., Han, J., Sahoo, S

    Arriola, M., Gokaslan, A., Chiu, J. T., Yang, Z., Qi, Z., Han, J., Sahoo, S. S., and Kuleshov, V . Block diffusion: Inter- polating between autoregressive and diffusion language models. InICLR, 2025a. Arriola, M., Schiff, Y ., Phung, H., Gokaslan, A., and Kuleshov, V . Encoder-decoder diffusion language mod- els for efficient training and inference.arXiv ...

    arXiv

  13. [2025]

    Large language diffusion models.arXiv preprint arXiv:2502.09992,

    Nie, S., Zhu, F., You, Z., Zhang, X., Ou, J., Hu, J., Zhou, J., Lin, Y ., Wen, J.-R., and Li, C. Large language diffusion models.arXiv preprint arXiv:2502.09992,

    Pith/arXiv arXiv