Pith.

Machine Learning · arXiv:2611.16345 v1 · submitted 12 Nov 2026 · 55 references

DRAMA: Dissecting Attention Redundancy for Accelerating Multimodal Diffusion Large Language Models

Yi-Hsin Hung, Fangfu Liu, Shao-Yuan Lo, Chu-Song Chen

The paper says

“Multimodal diffusion large language models (dLLMs) generate text through iterative masked denoising with bidirectional attention, offering a compelling alternative to autoregressive vision-language modeling.”

more of the abstract

However, each denoising step requires a full forward pass over long multimodal sequences in which visual tokens dominate the sequence length, and a fixed step budget further compounds this quadratic per-step cost. Existing inference acceleration approaches for multimodal dLLMs characterize attention redundancy implicitly, motivating designs from isolated attention visualizations rather than from measurement. We close this gap with DRAMA, a training-free acceleration framework grounded in a systematic dissection of attention redundancy in multimodal dLLMs. By probing attention tensors along spatial and temporal axes, we reveal two persistent redundancy structures: (i) attention mass consistently concentrates on a sparse subset of visual tokens, and (ii) attention distributions rapidly stabilize across denoising steps after an initial formation phase. These findings motivate two complementary mechanisms: Spatial-Redundancy-Aware Visual Token Pruning (SVTP) and Sufficiency-Guided Adaptive Stopping (SGAS). Together, DRAMA achieves an average 18.67 times inference speedup across six vision-language benchmarks without training, architectural modification, or attention caching, while maintaining competitive accuracy. Our results establish a strong, measurement-grounded acceleration baseline for multimodal dLLM inference.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 13 Nov 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. Measured speedup is real and repeatable; the quality-preservation claim is weaker than the abstract suggests, but the paper is still a solid systems contribution. This line is wrong

5 major objections · 7 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 55 references say.

The paper says, at Section 3.3.2, Table 10

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.72):

“As stated in Appendix A.1, DRAMA adds no mechanism to recover quality lost to a reduced budget, and on open-ended text-dense inputs that cost is simply larger than on the six main benchmarks. 19 D Derivation of the Output-Stationarity Bound This appendix supplies the derivation of the Pinsker-type output-stationarity bound invoked in Section 3.3.2, showing that the per-step layer-output drift vanishes as a direct corollary of the JSD collapse identified in Section 3.2.”

Pith says: Objection 1 of 5 · major

The SGAS stop predicate (Eq. 5) uses lexical coherence and task-schema parseability as a sufficiency surrogate for output correctness. The paper's own held-out evaluation on TextVQA (Table 10) shows this assumption fails dramatically: accuracy drops from 64.60 to 52.50 under FG Uni, a 12.1 pp loss.

more of objection 1

The authors acknowledge this (b312: 'DRAMA adds no mechanism to recover quality lost to a reduced budget, and on open-ended text-dense inputs that cost is simply larger'). This is not a corner case; TextVQA is a standard benchmark for text-rich visual reasoning. The Appendix D derivation (Eqs. 6-13) bounds output drift between consecutive steps, not distance to ground truth. A fluent, parseable but wrong answer passes the gate. The central claim of 'competitive accuracy' is therefore conditioned on the surrogate holding, and the authors' own evidence shows it does not hold on a held-out benchmark. This is a load-bearing issue for the quality-preservation claim.

The paper says, at Table 1, Table 10

The review did not quote this passage.

Pith says: Objection 2 of 5 · major

All accuracy results in Table 1 are reported as single-run point estimates with no error bars, confidence intervals, or significance tests. Given the variability inherent in LLM evaluation, it is impossible to assess whether the reported differences (e.g., -1.9 pp, -3.4 pp) are statistically significant or within noise.

more of objection 2

This is particularly concerning for claims like 'DRAMA variants substantially outperform Baseline 16' and 'on MME, every DRAMA variant surpasses the unaccelerated baseline (78.9% vs. 78.5%)'. The paper should report standard errors or confidence intervals, or at minimum state whether multiple runs were performed.

The paper says, at Section 3.3.1, Eq. (4)

The review did not quote this passage.

Pith says: Objection 3 of 5 · major

SVTP selects patches based on cosine similarity between mean-pooled patch embeddings and mean-pooled question embeddings. This is a heuristic that is not validated against the actual attention distributions that SVTP is meant to approximate. The paper shows that attention concentrates on a few patches, but does not show that the cosine similarity score is a good proxy for attention salience.

more of objection 3

A direct comparison (e.g., overlap between top-k patches by similarity vs. top-k by attention) is missing. Without this, the claim that SVTP 'grounds' pruning in the measured redundancy is weakened.

The paper says, at Section 4.2, Table 1

The review did not quote this passage.

Pith says: Objection 4 of 5 · major

The speedup numbers are impressive but conflate two orthogonal mechanisms. The average 18.67x speedup is achieved by FG Uni, which is the most aggressive variant. However, FG Uni also has the largest accuracy drop (-3.4 pp average). The paper should more clearly separate the contributions: SVTP alone gives ~2.78x (from Table 1, w/o SGAS row), SGAS alone gives up to 7.39x (w/o SVTP FG Uni).

more of objection 4

The product of these is ~20.5x, but the actual FG Uni achieves 18.67x, suggesting some interaction or overhead. The paper should discuss this discrepancy.

The paper says, at Section 4.1, Appendix B.5

The review did not quote this passage.

Pith says: Objection 5 of 5 · major

The SGAS thresholds (Table 5) were 'set from the output statistics of 100 LLaDA-V samples on MMMU' (Appendix B.5). This means the thresholds are tuned on one benchmark and then applied to others. While this is a reasonable practice, it means the method is not entirely parameter-free; the thresholds are a free parameter.

more of objection 5

The paper should discuss sensitivity to these thresholds. More importantly, the held-out TextVQA result (Table 10) shows that these thresholds, tuned on MMMU, do not generalize well to text-dense inputs. This reinforces the concern about the surrogate's validity.

7 minor objections

The paper says, at Abstract, line 6

Pith says: minor objection 1 of 7

The abstract states 'an average 18.67 times inference speedup across six vision-language benchmarks without training, architectural modification, or attention caching, while maintaining competitive accuracy.' The phrase 'competitive accuracy' is vague. Consider quantifying the accuracy retention (e.g., 'within 3.4 pp of the baseline').

The paper says, at Table 1

Pith says: minor objection 2 of 7

The table uses 'Best 2nd best' notation but the coloring is not visible in the text. Consider using bold or other markers. Also, the 'w/o SGAS' row uses 32 denoising steps, but its speedup is normalized by ÷2 to discount the inherent gain from halving the step count. This normalization is unusual and should be explained more clearly in the caption.

show 5 more

The paper says, at Figure 1

Pith says: minor objection 3 of 7

The Pareto frontier figure is informative but the 'band' for the frontier is not clearly defined. Is it the convex hull? The caption should explain how the frontier is computed.

The paper says, at Section 3.3.2, Eq. (5)

Pith says: minor objection 4 of 7

The notation '⊮' for indicator function is non-standard. Consider using '𝟙' or 'I'. Also, the definition of 'E' as 'granularity-controlled evaluation set' is unclear; it is later defined as the set of evaluation steps. Clarify.

The paper says, at Appendix D

Pith says: minor objection 5 of 7

The derivation of the output-stationarity bound is appreciated, but it relies on the assumption that 'the converged attention assigns negligible aggregate weight' to newly unmasked tokens. This assumption should be stated explicitly and justified, as it is critical for the bound to hold.

The paper says, at Table 10

Pith says: minor objection 6 of 7

The held-out evaluation on POPE and TextVQA is valuable. However, the POPE results show that SGAS alone is lossless (88.30 F1) but adding SVTP costs 2.95 F1. This suggests that SVTP, not SGAS, is the main source of accuracy loss on object-existence tasks. The paper should discuss this trade-off more explicitly.

The paper says, at General

Pith says: minor objection 7 of 7

The paper uses the term 'competitive accuracy' without a formal definition. Given that the accuracy drops are non-negligible on some benchmarks (e.g., MM-Star -4.3 pp, MMBench -3.8 pp), the authors should state explicitly what they consider 'competitive' (e.g., within 5% of baseline).

What would settle it

Run DRAMA FG Uni on the full TextVQA validation set (or a held-out OCR-heavy benchmark like OCR-VQA) and report accuracy alongside the 64-step baseline. If the accuracy gap exceeds 10 pp (as the 100-sample subset suggests), the SGAS sufficiency surrogate is not reliable for text-dense inputs, and the 'competitive accuracy' claim must be scoped to benchmarks without dense text. Additionally, compute the correlation between SGAS stopping step and per-sample accuracy: if early-stopped samples are disproportionately wrong, the predicate is selecting for fluency, not correctness.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 13 Nov 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic agrees 7 s · 31,561 tokens read, 753 written
  3. Editor wrote the verdict above 7 s · 32,538 tokens read, 584 written
  4. Referee major revision: 5 major objections, 7 minor 15 s · 31,989 tokens read, 1,978 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

The headline result—18.67× wall-clock speedup on LLaDA-V with code released—is the real thing. The authors systematically measure attention redundancy along two orthogonal axes (spatial concentration on a few visual patches, temporal stabilization after a formation phase) and build two complementary mechanisms that target each axis independently. The spatial pruning (SVTP) is simple and effective: score patches by question similarity once, keep one patch plus the base crop. The temporal early stopping (SGAS) uses lexical coherence and parseability as a stop predicate. The speedup numbers are measured, not fitted, and the ablation study cleanly separates the contributions of each component. The paper also evaluates on a second backbone (LaViDa) and compares against concurrent methods (RedVTP, D3ToM), which is good practice.

The soft spots are real but bounded. First, the SGAS stop predicate treats fluency as a sufficiency surrogate for correctness. The authors' own TextVQA result (64.6 → 52.5) shows this breaks on text-dense inputs, and they acknowledge it in Appendix A.1. This doesn't invalidate the speedup claim, but it means the "competitive accuracy" claim is conditioned on the predicate holding, and the paper should be clearer about where it doesn't. Second, the MMBench FG Uni accuracy (69.0) is a 12.9 pp outlier relative to the −3.4 pp average drop. The paper doesn't explain this. Third, the SGAS thresholds were fit on 100 MMMU samples, so the temporal "prediction" is partly calibration. The authors disclose this in Appendix B.5, which is honest, but it qualifies the "measurement-grounded" framing. Fourth, all accuracies are single-run point estimates with no error bars. This is common in this subfield but still worth noting.

Who is this for? Anyone working on efficient inference for diffusion LLMs, especially multimodal ones. The redundancy measurement methodology is reusable, and the speedup is practically meaningful (2.6 → 51.9 tokens/s). The paper deserves a serious referee. The main issues are addressable: clarify the scope of the quality-preservation claim, explain the MMBench outlier, and add error bars or significance tests.

Recommendation: Accept with moderate revision. The core speedup claim is solid, but the accuracy claims need tighter qualification.

The strongest objection

The central quality-preservation claim of DRAMA depends on the SGAS stop predicate (Eq. 5, Section 3.3.2) treating decoded-text lexical coherence and task-schema parseability as a sufficient surrogate for output correctness. The paper's own held-out TextVQA evaluation (Table 10, b312) shows this assumption fails: accuracy drops from 64.60 to 52.50 under FG Uni, a 12.1 pp loss that the authors acknowledge (b312: 'DRAMA adds no mechanism to recover quality lost to a reduced budget, and on open-ended text-dense inputs that cost is simply larger'). This is not a corner case—TextVQA is a standard benchmark for text-rich visual reasoning. The Appendix D derivation (Eqs. 6–13) shows that when inter-step JSD collapses, layer outputs stabilize, but it does not prove that a stable output is a correct output. The bound on output drift (Eq. 13) bounds the difference between consecutive-step outputs, not the distance to the ground-truth answer. A fluent, parseable but wrong answer satisfies the SGAS predicate and passes the gate. The paper's own data confirms this failure mode on a held-out benchmark. The speedup numbers are measured and repeatable, but the 'competitive accuracy' claim is conditioned on the SGAS surrogate holding, and the authors' own evidence shows it does not hold on text-dense inputs.

The reader's weakest_assumption correctly identifies the SGAS sufficiency surrogate as load-bearing. The paper's own TextVQA result (Table 10) provides direct evidence that this assumption breaks on text-dense inputs, with accuracy falling from 64.60 to 52.50 under FG Uni. The Appendix D derivation bounds output drift between consecutive steps but does not connect stationarity to correctness. The speedup measurements are robust and repeatable, but the quality-preservation claim is conditional on the SGAS predicate being a reliable proxy for correctness. The authors acknowledge the limitation (b312), but the TextVQA result shows it is not a minor edge case—it is a systematic failure on a standard benchmark. The MMBench FG Uni outlier (69.0% vs. 81.9% baseline, a 12.9 pp drop) further suggests the surrogate can fail on non-text-dense benchmarks as well. The verdict should remain CONDITIONAL because the speedup claim is solid, but the accuracy claim needs tighter scoping. I agree with the reader's assessment.

The referee's report

If the claims hold, DRAMA provides a substantial practical speedup for an emerging class of models (multimodal dLLMs) that are currently computationally expensive. The systematic measurement of redundancy is a methodological contribution. The speedup numbers are large and measured, not asymptotic. The paper also provides code and detailed ablations. However, the significance is tempered by the fact that the quality-preservation claim depends on a surrogate (lexical coherence + parseability) that the authors' own held-out evaluation shows can fail significantly (TextVQA). The framework is also evaluated on a single backbone (LLaDA-V) with limited generalization to LaViDa (much smaller speedup). The core insight about spatial and temporal redundancy is valuable, but the practical impact depends on how often the SGAS surrogate holds.

Pith's reading of the paper (a machine's summary, not the authors')

The paper argues that multimodal diffusion large language models (dLLMs) compute massive amounts of redundant attention during inference, and that this redundancy can be systematically measured and exploited without any training or architectural changes. The authors identify two persistent redundancy structures: attention mass concentrates on a tiny fraction of visual tokens (spatial redundancy), and attention distributions stabilize rapidly after a few initial denoising steps (temporal redundancy). They design DRAMA, a framework with two complementary mechanisms that target these redundancies: Spatial-Redundancy-Aware Visual Token Pruning (SVTP) selects the most relevant image patches once before generation begins, and Sufficiency-Guided Adaptive Stopping (SGAS) terminates denoising early once the decoded text is coherent and answer-shaped. Together, these achieve an average 18.67× inference speedup across six standard vision-language benchmarks on the LLaDA-V model while maintaining competitive accuracy (within 3.4 percentage points of the full 64-step baseline). If this is correct, it establishes that the dominant computational cost of multimodal dLLM inference can be removed with a measurement-grounded, training-free approach, offering a strong acceleration baseline for this emerging model class.

What carries the argument. The key machinery is the systematic dissection of attention redundancy along two axes: spatial (how attention mass distributes over visual tokens, measured by cumulative coverage curves and the κ(0.9) metric indicating the minimum visual tokens needed for 90% attention coverage) and temporal (how attention distributions evolve across denoising steps, measured by inter-step Jensen-Shannon divergence and the redundancy complement R_l(t) = 1 - JSD/ln2). This analysis motivates two mechanisms: Spatial-Redundancy-Aware Visual Token Pruning (SVTP), which scores image patches by cross-modal similarity to the question text and retains only the most relevant patches once before denoising, reducing the key-value sequence length from N to |P|·M+|T|; and Sufficiency-Guided Adaptive Stopping (SGAS), which uses decoded-text lexical coherence statistics (diversity, peak frequency, consecutive repetition) plus task-schema parseability as a sufficiency surrogate to terminate denoising early once the output has stabilized. The paper also derives a Pinsker-type bound linking JSD collapse to output stationarity, justifying the use of decoded text as a proxy for attention convergence.

Core claim. The paper's central discovery is that attention in multimodal dLLMs exhibits two forms of redundancy that are both persistent and exploitable: (1) spatial redundancy, where the attention mass over visual tokens is concentrated on only a handful of image patches (the top 1% of visual key tokens capture ~47% of attention mass), and crucially this concentration is step-invariant, meaning a single selection made before denoising remains valid across all T steps; and (2) temporal redundancy, where inter-step attention divergence (measured by Jensen-Shannon divergence) rapidly drops near zero after an initial structure-formation phase (5-10 steps) and remains in a stable plateau where redundancy scores exceed 0.99 across all transformer layers, meaning subsequent denoising steps recompute near-identical attention at full O(T·L·N²) cost for diminishing informational return. The paper further shows that these two redundancy structures are orthogonal, enabling multiplicative acceleration gains when targeted simultaneously.

The premise it rests on. The SGAS stop predicate treats decoded-text lexical coherence (diversity, peak frequency, consecutive repetition) plus task-schema parseability as a sufficient stand-in for output correctness, assuming that a fluent, answer-shaped sequence is necessarily a correct one.

If the paper is right (Pith's inferences, not the authors')

Multimodal dLLM inference can be accelerated by over an order of magnitude with no training or architectural changes, establishing a new baseline that uniform step reduction cannot match.

The spatial and temporal redundancy structures are orthogonal, so their gains multiply; this suggests future acceleration methods should target both axes independently.

The measurement-grounded approach (dissecting attention before designing mechanisms) can replace the current ad-hoc, visualization-inspired design of acceleration methods for this model class.

The SGAS stopping predicate, based on lexical coherence and task parseability, can serve as a general-purpose early-exit criterion for diffusion language models, not just multimodal ones.

The 18.67× speedup brings multimodal dLLM throughput (51.9 tokens/s on average) into a regime competitive with autoregressive models of similar scale, potentially making dLLMs practical for interactive applications.

The paper's load-bearing premise is that lexical coherence and task-schema parseability are sufficient surrogates for output correctness. This premise likely breaks on tasks requiring precise numerical reasoning or multi-step verification, where a fluent wrong answer passes the gate. Held-out TextVQA results (accuracy dropping from 64.60 to 52.50 under FG Uni) already show this failure mode on tex

The spatial pruning strategy (SVTP) relies on the question text being available before denoising. This limits applicability to settings where the conditioning text is known upfront; in interactive or multi-turn scenarios where the question is refined during generation, the pruning would need to be re-done.

The temporal redundancy analysis (three-phase JSD structure) may generalize beyond multimodal dLLMs to text-only dLLMs and even to diffusion models in other modalities (e.g., image, video), suggesting that early stopping based on output stabilization could be a universal property of iterative denoising.

The 18.67× speedup is measured on a single backbone (LLaDA-V) with a specific generation regime (128 tokens, deterministic decoding). The actual speedup on other backbones (e.g., LaViDa, which already pools visual tokens) is much smaller (1.73-1.84×), indicating that the reported headline number is highly architecture-dependent and should not be generalized naively.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Multimodal diffusion large language models (dLLMs) generate text through iterative masked denoising with bidirectional attention, offering a compelling alternative to autoregressive vision-language modeling. However, each denoising step requires a full forward pass over long multimodal sequences in which visual tokens dominate the sequence length, and a fixed step budget further compounds this quadratic per-step cost. Existing inference acceleration approaches for multimodal dLLMs characterize attention redundancy implicitly, motivating designs from isolated attention visualizations rather than from measurement. We close this gap with DRAMA, a training-free acceleration framework grounded in a systematic dissection of attention redundancy in multimodal dLLMs. By probing attention tensors along spatial and temporal axes, we reveal two persistent redundancy structures: (i) attention mass consistently concentrates on a sparse subset of visual tokens, and (ii) attention distributions rapidly stabilize across denoising steps after an initial formation phase. These findings motivate two complementary mechanisms: Spatial-Redundancy-Aware Visual Token Pruning (SVTP) and Sufficiency-Guided Adaptive Stopping (SGAS). Together, DRAMA achieves an average 18.67 times inference speedup across six vision-language benchmarks without training, architectural modification, or attention caching, while maintaining competitive accuracy. Our results establish a strong, measurement-grounded acceleration baseline for multimodal dLLM inference.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-13

The authors' main claim, as the classifier picked it out 0.59

“Our results establish a strong, measurement-grounded acceleration baseline for multimodal dLLM inference.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.11 · states limitation 0.12 · human participants 0.03 · controlled comparison 0.36 · randomized assignment 0.01 · states quantitative result 0.99 · compares with prior results 0.46 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Multimodal diffusion large language models (dLLMs) generate text through iterative masked denoising with bidirectional attention, offering a compelling alternative to autoregressive vision-language modeling. background 1.00 0.05 0.02 asserted 0.97
However, each denoising step requires a full forward pass over long multimodal sequences in which visual tokens dominate the sequence length, and a fixed step budget further compounds this quadratic per-step cost. background 0.96 0.09 0.02 asserted 1.00
Existing inference acceleration approaches for multimodal dLLMs characterize attention redundancy implicitly, motivating designs from isolated attention visualizations rather than from measurement. gap 0.98 0.05 0.03 asserted 1.00
We close this gap with DRAMA, a training-free acceleration framework grounded in a systematic dissection of attention redundancy in multimodal dLLMs. contribution 1.00 0.80 0.08 asserted 1.00
By probing attention tensors along spatial and temporal axes, we reveal two persistent redundancy structures: (i) attention mass consistently concentrates on a sparse subset of visual tokens, and (ii) attention distributions rapidly stabilize across denoising steps after an initial formation phase. result 0.98 0.96 0.16 asserted 1.00
These findings motivate two complementary mechanisms: Spatial-Redundancy-Aware Visual Token Pruning (SVTP) and Sufficiency-Guided Adaptive Stopping (SGAS). method 0.67 0.73 0.08 asserted 0.81
Together, DRAMA achieves an average 18.67 times inference speedup across six vision-language benchmarks without training, architectural modification, or attention caching, while maintaining competitive accuracy. result 1.00 0.96 0.60 asserted 1.00
Our results establish a strong, measurement-grounded acceleration baseline for multimodal dLLM inference. interpretation 0.95 0.88 0.55 asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.98

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-13: 1 request, 6,402 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (19)
Figure 1 of the paper
Figure 1. Accuracy-latency Pareto frontier. DRAMA variants (diamonds) dominate the Pareto front, achieving higher accuracy at lower latency than all fixed-budget baselines (triangles). A concurrent line of work has begun extending acceleration techniques to multimodal dLLMs [50, 7, 48]. Their empirical speedups, together with attention visualizations, suggest the presence of exploitable redundancy in this regime. However, these studies characterize such attention redundancy either implicitly through downstream performance gains or isolated examples. Consequently, their designs are rarely…
Figure 2 of the paper
Figure 2. Spatial redundancy analysis. (a) Per-sample attention share of each image patch ranked by dominance. (b) Per-sample attention share distribution of the most-attended patch across the dataset. (c) Cumulative attention coverage by top-k% visual key tokens, stratified by query-key modality pair. (d) Minimum visual tokens needed for 90% attention coverage (κ(0.9)) across denoising steps ∼44% of aggregate attention, which is 1.7× the uniform baseline, and the top two patches suffice for ∼68% coverage, while the remaining patches contribute marginally (
Figure 3 of the paper
Figure 3. Temporal redundancy analysis across the denoising trajectory (averaged over six benchmarks). (a) Per-layer inter-step JSD (J¯ l) between consecutive denoising steps; peaks highlight steps of greatest attention change. (b) Redundancy heatmap Rl(t) across layers and steps; bright regions indicate high redundancy (Rl→1). (c) Distribution of actual stopping steps of our proposed DRAMA, whose concentration aligns with the plateau onset in (a). complement: J¯ l(t) = 1 H · N X H h=1 X N i=1 JSD a (t) l,h,i
Figure 4 of the paper
Figure 4. Overview of DRAMA. DRAMA applies Spatial-Redundancy-Aware Visual Token Pruning (SVTP) to select a compact patch subset once before denoising. Sufficiency-Guided Adaptive Stopping (SGAS) terminates the denoising trajectory early. Specifically, the frozen vision encoder and the dLLM’s token embedding layer project visual patches and question tokens into a shared representation space, providing a natural relevance signal without any additional computation. We score each patch i by the similarity between its mean-pooled embedding p¯i and the mean-pooled question representation q¯ in…
Figure 5 of the paper
Figure 5. Distribution of actual denoising steps under SGAS across six benchmarks. Each subplot shows the histogram and KDE for one SGAS variant. FG Uni concentrates most samples at very early steps (µ≈8–12), while CK variants cluster around the first eligible checkpoint (step 16). Progressive scheduling shifts the distribution rightward, reflecting its stricter early-step requirements. a second backbone (LaViDa); and (iv) head-to-head comparisons with two concurrent multimodal dLLM acceleration methods to Appendix C. Effect of SVTP. At a fixed budget of T = 32 steps, SVTP alone yields 2.78×…
Figure 6 of the paper
Figure 6. Spatial redundancy analysis – AI2D. The diagram-centric nature of AI2D leads to highly concentrated attention on a single informative patch. Most attended 2nd 3rd 0 20 40 60 Attention fraction (%, mean ± std) 63% 32% 21% (a) Output image attention by crop rank (b) Spatial attention heatmap (output image, example) 0 10 20 30 40 50 Top-k% of modality keys (per query) 0 20 40 60 80 100 Attention coverage (%) 90% (c) Token-level concentration by modality t2v (output image) v2v (image image) t2t (output text) v2t (image text) 1 4 8 16 24 32 Denoising step 0 20 40 60 80 100 Visual tokens…
Figure 7 of the paper
Figure 7. Spatial redundancy analysis – MMBench. Despite diverse image content, the top 2–3 patches consistently capture the majority of output→image attention. The temporal dissection in Section 3.2 establishes that J¯ k(t)→0 simultaneously across all k∈[L] once past the structure-formation phase (the bright plateau region of
Figure 8 of the paper
Figure 8. Spatial redundancy analysis – MME. Perceptual tasks in MME yield concentrated attention patterns, with κ(0.9) remaining stable across all denoising steps. Most attended 2nd 3rd 4th 0 10 20 30 40 50 60 Attention fraction (%, mean ± std) 51% 27% 22% 16% (a) Output image attention by crop rank (b) Spatial attention heatmap (output image, example) 0 10 20 30 40 50 Top-k% of modality keys (per query) 0 20 40 60 80 100 Attention coverage (%) 90% (c) Token-level concentration by modality t2v (output image) v2v (image image) t2t (output text) v2t (image text) 1 4 8 16 24 32 Denoising step 0…
Figure 9 of the paper
Figure 9. Spatial redundancy analysis – MM-Star. Knowledge-intensive queries in MM-Star show similar spatial concentration to MMMU. E.2 Dissecting Temporal Redundancy: Analysis per Benchmark We present the temporal analysis averaged over all benchmarks in the main paper. Here we provide per￾benchmark breakdowns, each showing (a) the per-layer inter-step JSD J¯ l(t) across all 64 denoising steps, and (b) the layer×step redundancy heatmap Rl(t). The three-phase temporal structure identified in the main paper—structure formation, stable plateau, and terminal refinement—is clearly visible in…
Figure 10 of the paper
Figure 10. Spatial redundancy analysis – SEED-Bench. SEED-Bench exhibits slightly broader spatial distributions due to its diverse scene understanding tasks, yet the dominant patch still captures ≥40% of attention. 10 20 30 40 50 60 Denoising Steps 0.000 0.005 0.010 0.015 0.020 0.025 0.030 0.035 Mean JSD (nats) (a) Inter-step JSD (MMMU) 1 8 15 22 29 36 43 50 57 64 Denoising Steps L1 L3 L5 L7 L9 L11 L13 L15 L17 L19 L21 L23 L25 L27 L29 L32 Layer (b) Redundancy Heatmap (MMMU) 10 20 30 40 50 60 Denoising Steps 0.000 0.005 0.010 0.015 0.020 0.025 0.030 Mean JSD (nats) (c) Inter-step JSD (AI2D) 1 8…
Figure 11 of the paper
Figure 11. Temporal redundancy analysis for all benchmarks. (a) Per-layer inter-step JSD across denoising steps. (b) Layer×step redundancy heatmap. saving relative to the 64-step baseline, and sample count (N). Several patterns are consistent across benchmarks: • DRAMA (bottom rows) shows similar stopping patterns to the w/o SVTP ablation (top rows), confirming that SVTP and SGAS operate on orthogonal axes (spatial vs. temporal) and compose independently. • w/o SVTP FG Uni concentrates exits at the earliest steps (typically around 7−−11 steps, with a later mean of 15.6 on MM-Star), reflecting…
Figure 12 of the paper
Figure 12. DRAMA achieves an average 18.67× inference speedup over fixed-budget baselines across six benchmarks (MMMU [54], AI2D [22], MME [13], MM-Star [8], SEED-Bench [26], and MMBench [33]) while maintaining competitive accuracy. first identifying structured attention patterns, then designing targeted strategies to exploit them. In autoregressive VLMs, methods such as FlexPrefill [25] and MInference [20, 30] observe that attention is sparse and concentrated on a few key tokens, motivating attention sparsification and token pruning. In generative DiTs, DiTFastAttn [53, 55] identifies…
Figure 13 of the paper
Figure 13. Averaged accuracy–latency Pareto frontier across all benchmarks. Each point represents a method’s mean accuracy and mean latency. DRAMA variants occupy the Pareto front, achieving superior accuracy at substantially lower latency than fixed-budget baselines. G.2 Multimodal Diffusion Language Models. Multimodal dLLMs [52, 28, 51] extend this framework by conditioning generation on a visual input. A frozen vision encoder maps an input image to Nv visual tokens v = (v 1 , . . . , vNv ); together with Nc instruction tokens c = (c 1 , . . . , cNc ), these form a fixed conditioning prefix
Figure 14 of the paper
Figure 14. Step distribution – AI2D. FG Uni achieves 85%/79% step saving (w/o SVTP / DRAMA) owing to the concentrated spatial attention on diagram elements. 0 20 40 60 0 100 200 300 400 Percentage (%) =16.0 med=16 =0.8 saving=75% N=4329 w/o SVTP (Ablation) CK Prog 0 20 40 60 =15.0 med=15 =0.3 saving=77% N=4329 FG Prog 0 20 40 60 =7.1 med=7 =1.4 saving=89% N=4329 FG Uni 0 20 40 60 =16.0 med=16 =0.5 saving=75% N=4329 CK Uni 0 20 40 60 Actual Steps 0 100 200 300 400 Percentage (%) =16.1 med=16 =1.0 saving=75% N=4329 DRAMA 0 20 40 60 Actual Steps =15.0 med=15 =0.1 saving=77% N=4329 0 20 40 60…
Figure 15 of the paper
Figure 15. Step distribution – MMBench. The diverse visual content of MMBench leads to a slightly broader stopping distribution compared to simpler benchmarks. 0 20 40 60 0 25 50 75 100 125 Percentage (%) =18.9 med=16 =7.0 saving=71% N=2374 w/o SVTP (Ablation) CK Prog 0 20 40 60 =16.1 med=15 =4.2 saving=75% N=2374 FG Prog 0 20 40 60 =10.7 med=8 =6.4 saving=83% N=2374 FG Uni 0 20 40 60 =16.1 med=16 =2.1 saving=75% N=2374 CK Uni 0 20 40 60 Actual Steps 0 50 100 150 Percentage (%) =18.2 med=16 =6.4 saving=72% N=2374 DRAMA 0 20 40 60 Actual Steps =16.1 med=16 =1.4 saving=75% N=2374 0 20 40 60…
Figure 16 of the paper
Figure 16. Step distribution – MME. Perceptual yes/no tasks converge rapidly, with most samples stopping within the first 15–20 steps. 26
Figure 17 of the paper
Figure 17. Step distribution – MMMU. Multi-discipline knowledge questions exhibit moderate step savings, with progressive scheduling providing a useful accuracy safeguard. 0 20 40 60 0 20 40 60 80 100 Percentage (%) =19.1 med=16 =9.3 saving=70% N=1488 w/o SVTP (Ablation) CK Prog 0 20 40 60 =16.4 med=16 =3.4 saving=74% N=1488 FG Prog 0 20 40 60 =15.6 med=15 =5.1 saving=76% N=1488 FG Uni 0 20 40 60 =16.6 med=16 =5.3 saving=74% N=1488 CK Uni 0 20 40 60 Actual Steps 0 20 40 60 80 100 Percentage (%) =18.9 med=16 =8.6 saving=70% N=1500 DRAMA 0 20 40 60 Actual Steps =16.4 med=16 =3.2 saving=74%…
Figure 18 of the paper
Figure 18. Step distribution – MM-Star. Knowledge-intensive queries show stopping patterns similar to MMMU, confirming that task difficulty correlates with stopping behaviour. 0 20 40 60 0 20 40 60 80 Percentage (%) =17.3 med=16 =6.0 saving=73% N=2000 w/o SVTP (Ablation) CK Prog 0 20 40 60 =15.8 med=15 =5.2 saving=75% N=2000 FG Prog 0 20 40 60 =8.3 med=5 =7.2 saving=87% N=2000 FG Uni 0 20 40 60 =17.3 med=16 =6.0 saving=73% N=2000 CK Uni 0 20 40 60 Actual Steps 0 20 40 60 80 Percentage (%) =17.5 med=16 =6.8 saving=73% N=2000 DRAMA 0 20 40 60 Actual Steps =16.0 med=15 =6.0 saving=75% N=2000 0…
Figure 19 of the paper
Figure 19. Step distribution – SEED-Bench. The diverse scene understanding tasks in SEED-Bench produce a broad stopping distribution, yet SGAS still achieves substantial step savings. 27

Figures are the authors', reproduced from the paper.

References (55)
  1. [1] Sudhanshu Agrawal, Risheek Garrepalli, Raghavv Goel, Mingu Lee, Christopher Lott, and Fatih Porikli. Spiffy: Multiplying diffusion llm acceleration via lossless speculative decoding.arXiv preprint arXiv:2509.18085, 2025 on PithPith's reviewarXiv
  2. [2] Marianne Arriola, Aaron Gokaslan, Justin T Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sekhar Sahoo, and V olodymyr Kuleshov. Block diffusion: Interpolating between autoregressive and diffusion language models.arXiv preprint arXiv:2503.09573, 2025 on PithPith's reviewarXiv
  3. [3] Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces.Advances in neural information processing systems, 34:17981–17993, 2021 on Pith
  4. [4] Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang, and Mike Zheng Shou. Hallucination of multimodal large language models: A survey.arXiv preprint arXiv:2404.18930, 2024 on PithPith's reviewarXiv
  5. [5] Parikshit Bansal and Sujay Sanghavi. Enabling approximate joint sampling in diffusion lms.arXiv preprint arXiv:2509.22738, 2025 on PithPith's reviewarXiv
  6. [6] Wenrui Bao, Zhiben Chen, Dan Xu, and Yuzhang Shang. Learning to parallel: Accelerating diffusion large language models via adaptive parallel decoding. InThe Fourteenth International Conference on Learning Representations, 2025 on Pith
  7. [7] Shuochen Chang, Xiaofeng Zhang, Qingyang Liu, and Li Niu. D3tom: Decider-guided dynamic token merging for accelerating diffusion mllms. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 19961–19969, 2026 on Pith
  8. [8] Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision-language models?Advances in Neural Information Processing Systems, 37:27056–27087, 2024 on Pith
  9. [9] Xinhua Chen, Sitao Huang, Cong Guo, Chiyue Wei, Yintao He, Jianyi Zhang, Hai Li, Yiran Chen, et al. Dpad: Efficient diffusion language models with suffix dropout.arXiv preprint arXiv:2508.14148, 2025 on PithPith's reviewarXiv
  10. [10] Zigeng Chen, Gongfan Fang, Xinyin Ma, Ruonan Yu, and Xinchao Wang. dparallel: Learnable parallel decoding for dllms.arXiv preprint arXiv:2509.26488, 2025 on PitharXiv
  11. [11] Google DeepMind. Gemini diffusion.https://blog.google/technology/google-deepmind/gemini-diffusion/. https://blog.google/technology/google-deepmind/gemini-diffusion/, 2025. Accessed: 2024-07-24 on Pith
  12. [12] Yihong Dong, Zhaoyu Ma, Xue Jiang, Zhiyuan Fan, Jiaru Qian, Yongmin Li, Jianha Xiao, Zhi Jin, Rongyu Cao, Binhua Li, et al. Saber: An efficient sampling with adaptive acceleration and backtracking enhanced remasking for diffusion language model.arXiv preprint arXiv:2510.18165, 2025 on PithPith's reviewarXiv
  13. [13] Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, et al. Mme: A comprehensive evaluation benchmark for multimodal large language models.arXiv preprint arXiv:2306.13394, 2023 on PithPith's reviewarXiv
  14. [14] Feiyang Fu, Tongxian Guo, and Zhaoqiang Liu. Learnable sampler distillation for discrete diffusion models.arXiv preprint arXiv:2509.19962, 2025. 10 on PitharXiv
  15. [15] Yifeng Gao, Ziang Ji, Yuxuan Wang, Biqing Qi, Hanlin Xu, and Linfeng Zhang. Self speculative decoding for diffusion large language models.arXiv preprint arXiv:2510.04147, 2025 on PitharXiv
  16. [16] Feng Hong, Geng Yu, Yushi Ye, Haicheng Huang, Huangjie Zheng, Ya Zhang, Yanfeng Wang, and Jiangchao Yao. Wide-in, narrow-out: Revokable decoding for efficient and effective dllms.arXiv preprint arXiv:2507.18578, 2025 on PitharXiv
  17. [17] Zhanqiu Hu, Jian Meng, Yash Akhauri, Mohamed S Abdelfattah, Jae-sun Seo, Zhiru Zhang, and Udit Gupta. Flashdlm: Accelerating diffusion language model inference via efficient kv caching and guided diffusion.arXiv preprint arXiv:2505.21467, 2025 on PitharXiv
  18. [18] Pengcheng Huang, Shuhao Liu, Zhenghao Liu, Yukun Yan, Shuo Wang, Zulong Chen, and Tong Xiao. Pc-sampler: Position-aware calibration of decoding bias in masked diffusion models.arXiv preprint arXiv:2508.13021, 2025 on PithPith's reviewarXiv
  19. [19] Daniel Israel, Tian Jin, Ellie Cheng, Guy Van den Broeck, Aditya Grover, Suvinay Subramanian, and Michael Carbin. Planned diffusion.arXiv preprint arXiv:2510.18087, 2025 on PitharXiv
  20. [20] Huiqiang Jiang, Yucheng Li, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Zhenhua Han, Amir H Abdi, Dongsheng Li, Chin-Yew Lin, et al. Minference 1.0: Accelerating pre-filling for long-context llms via dynamic sparse attention.Advances in Neural Information Processing... on Pith
show 35 more
  1. [21] Yuchu Jiang, Yue Cai, Xiangzhong Luo, Jiale Fu, Jiarui Wang, Chonghan Liu, and Xu Yang. d2 cache: Accelerating diffusion-based llms via dual adaptive caching.arXiv preprint arXiv:2509.23094, 2025 on PitharXiv
  2. [22] Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images. InEuropean conference on computer vision, pages 235–251. Springer, 2016 on Pith
  3. [23] Fanheng Kong, Jingyuan Zhang, Yahui Liu, Zirui Wu, Yu Tian, Guorui Zhou, et al. Accelerating diffusion llm inference via local determinism propagation.arXiv preprint arXiv:2510.07081, 2025 on PitharXiv
  4. [24] Inception Labs, Samar Khanna, Siddhant Kharbanda, Shufan Li, Harshit Varma, Eric Wang, Sawyer Birnbaum, Ziyang Luo, Yanis Miraoui, Akash Palrecha, et al. Mercury: Ultra-fast language models based on diffusion.arXiv preprint arXiv:2506.17298, 2025 on PithPith's reviewarXiv
  5. [25] Xunhao Lai, Jianqiao Lu, Yao Luo, Yiyuan Ma, and Xun Zhou. Flexprefill: A context-aware sparse attention mechanism for efficient long-sequence inference.arXiv preprint arXiv:2502.20766, 2025 on PithPith's reviewarXiv
  6. [26] Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. Seed-bench: Benchmarking multimodal llms with generative comprehension.arXiv preprint arXiv:2307.16125, 2023 on PithPith's reviewarXiv
  7. [27] Pengxiang Li, Yefan Zhou, Dilxat Muhtar, Lu Yin, Shilin Yan, Li Shen, Soroush V osoughi, and Shiwei Liu. Diffusion language models know the answer before decoding.arXiv preprint arXiv:2508.19982, 2025 on PithPith's reviewarXiv
  8. [28] Shufan Li, Konstantinos Kallidromitis, Hritik Bansal, Akash Gokul, Yusuke Kato, Kazuki Kozuka, Jason Kuen, Zhe Lin, Kai-Wei Chang, and Aditya Grover. Lavida: A large diffusion language model for multimodal understanding.arXiv preprint arXiv:2505.16839, 2025 on PithPith's reviewarXiv
  9. [29] Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. InProceedings of the 2023 conference on empirical methods in natural language processing, pages 292–305, 2023 on Pith
  10. [30] Yucheng Li, Huiqiang Jiang, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Amir H Abdi, Dongsheng Li, Jianfeng Gao, Yuqing Yang, et al. Mminference: Accelerating pre-filling for long-context vlms via modality-aware permutation sparse attention.arXiv preprint arXiv:2504... on PithPith's reviewarXiv
  11. [31] Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei Peng. A survey on hallucination in large vision-language models.arXiv preprint arXiv:2402.00253, 2024 on PithPith's reviewarXiv
  12. [32] Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024. URL https://llava-vl.github.io/ blog/2024-01-30-llava-next/ on Pith
  13. [33] Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pages 216–233. Springer, 2024. 11 on Pith
  14. [34] Zhiyuan Liu, Yicun Yang, Yaojie Zhang, Junjie Chen, Chang Zou, Qingyuan Wei, Shaobo Wang, and Linfeng Zhang. dllm-cache: Accelerating diffusion large language models with adaptive caching.arXiv preprint arXiv:2506.06295, 2025 on PithPith's reviewarXiv
  15. [35] Xinyin Ma, Runpeng Yu, Gongfan Fang, and Xinchao Wang. dkv-cache: The cache for diffusion language models.arXiv preprint arXiv:2505.15781, 2025 on PithPith's reviewarXiv
  16. [36] Yuxin Ma, Lun Du, Lanning Wei, Kun Chen, Qian Xu, Kangyu Wang, Guofeng Feng, Guoshan Lu, Lin Liu, Xiaojing Qi, et al. dinfer: An efficient inference framework for diffusion language models.arXiv preprint arXiv:2510.08666, 2025 on PitharXiv
  17. [37] Quan Nguyen-Tri, Mukul Ranjan, and Zhiqiang Shen. Attention is all you need for kv cache in diffusion llms.arXiv preprint arXiv:2510.14973, 2025 on PitharXiv
  18. [38] Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models.arXiv preprint arXiv:2502.09992, 2025 on PithPith's reviewarXiv
  19. [39] Yifan Pu, Zhuofan Xia, Jiayi Guo, Dongchen Han, Qixiu Li, Duo Li, Yuhui Yuan, Ji Li, Yizeng Han, Shiji Song, et al. Efficient diffusion transformer with step-wise dynamic attention mediators. InEuropean Conference on Computer Vision, pages 424–441. Springer, 2024 on Pith
  20. [40] Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. In2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8309–8318. IEEE, 2019 on Pith
  21. [41] Yuerong Song, Xiaoran Liu, Ruixiao Li, Zhigeng Liu, Zengfeng Huang, Qipeng Guo, Ziwei He, and Xipeng Qiu. Sparse-dllm: Accelerating diffusion llms with dynamic cache eviction.arXiv preprint arXiv:2508.02558, 2025 on PitharXiv
  22. [42] Yuxuan Song, Zheng Zhang, Cheng Luo, Pengyang Gao, Fan Xia, Hao Luo, Zheng Li, Yuehang Yang, Hongli Yu, Xingwei Qu, et al. Seed diffusion: A large-scale diffusion language model with high-speed inference.arXiv preprint arXiv:2508.02193, 2025 on PithPith's reviewarXiv
  23. [43] Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, et al. Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and ... on PithPith's reviewarXiv
  24. [44] Xu Wang, Chenkai Xu, Yijie Jin, Jiachun Jin, Hao Zhang, and Zhijie Deng. Diffusion llms can do faster-than-ar inference via discrete diffusion forcing.arXiv preprint arXiv:2508.09192, 2025 on PithPith's reviewarXiv
  25. [45] Qingyan Wei, Yaojie Zhang, Zhiyuan Liu, Dongrui Liu, and Linfeng Zhang. Accelerating diffusion large language models with slowfast sampling: The three golden principles.arXiv preprint arXiv:2506.10848, 2025 on PitharXiv
  26. [46] Chengyue Wu, Hao Zhang, Shuchen Xue, Shizhe Diao, Yonggan Fu, Zhijian Liu, Pavlo Molchanov, Ping Luo, Song Han, and Enze Xie. Fast-dllm v2: Efficient block-diffusion llm.arXiv preprint arXiv:2509.26328, 2025 on PitharXiv
  27. [47] Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao, Ligeng Zhu, Ping Luo, Song Han, and Enze Xie. Fast-dllm: Training-free acceleration of diffusion llm by enabling kv cache and parallel decoding.arXiv preprint arXiv:2505.22618, 2025 on PithPith's reviewarXiv
  28. [48] Chengyue Wu, Shiyi Lan, Yonggan Fu, Sensen Gao, Jin Wang, Jincheng Yu, Jose M Alvarez, Pavlo Molchanov, Ping Luo, Song Han, et al. Fast-dvlm: Efficient block-diffusion vlm via direct conversion from autoregressive vlm.arXiv preprint arXiv:2604.06832, 2026 on PithPith's reviewarXiv
  29. [49] Shutong Wu and Jiawei Zhang. Free draft-and-verification: Toward lossless parallel decoding for diffusion large language models.arXiv preprint arXiv:2510.00294, 2025 on PithPith's reviewarXiv
  30. [50] Jingqi Xu, Jingxi Lu, Chenghao Li, Sreetama Sarkar, Souvik Kundu, and Peter A Beerel. Redvtp: Training- free acceleration of diffusion vision-language models inference via masked token-guided visual token pruning.arXiv preprint arXiv:2511.12428, 2025 on PitharXiv
  31. [51] Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, and Mengdi Wang. Mmada: Multimodal large diffusion language models.arXiv preprint arXiv:2505.15809, 2025 on PithPith's reviewarXiv
  32. [52] Zebin You, Shen Nie, Xiaolu Zhang, Jun Hu, Jun Zhou, Zhiwu Lu, Ji-Rong Wen, and Chongxuan Li. Llada-v: Large language diffusion models with visual instruction tuning.arXiv preprint arXiv:2505.16933, 2025. 12 on PithPith's reviewarXiv
  33. [53] Zhihang Yuan, Hanling Zhang, Lu Pu, Xuefei Ning, Linfeng Zhang, Tianchen Zhao, Shengen Yan, Guohao Dai, and Yu Wang. Ditfastattn: Attention compression for diffusion transformer models.Advances in Neural Information Processing Systems, 37:1196–1219, 2024 on Pith
  34. [54] Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. InProceedings of the IEEE/CVF conference on co... on Pith
  35. [55] Hanling Zhang, Rundong Su, Zhihang Yuan, Pengtao Chen, Mingzhu Shen, Yibo Fan, Shengen Yan, Guohao Dai, and Yu Wang. Ditfastattnv2: Head-wise attention compression for multi-modality diffusion transformers. InProceedings of the IEEE/CVF International Conference on Computer Vis... on Pith

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment