Computation and Language · arXiv:2608.22367 v1 · submitted 23 Aug 2026 · 74 references
Context-Aware Cluster Decoding: Semantic Anchor-Driven Coherence in dMLLMs
The paper says
“Diffusion multimodal large language models (dMLLMs) frequently produce long-form outputs marred by semantic drift and repetition, with quality generally degrading as output length increases.”
more of the abstract
We identify two structural deficiencies in existing decoding methods as primary drivers of these failures: confidence-based scoring ignores decoded-neighbor support, and block partitioning prevents access to high-readiness semantic anchors, together causing tokens to be committed before their local context is sufficiently established. We propose \ours (\textbfContext-\textbfAware \textbfCluster \textbfDecoding), a training-free decoding method that scores each masked position by a multiplicative composite of softmax confidence and neighbor proximity, promoting contextually ready tokens above isolated candidates while suppressing low-confidence positional noise, operating block-free to keep high-readiness anchors globally accessible. \ours further applies architecture-aware calibration to handle confidence heterogeneity induced by diverse visual integration strategies. Experiments on three dMLLMs across four benchmarks demonstrate consistent quality gains and hallucination reduction over Original, with larger gains in several longer generation settings, highlighting the importance of neighbor support and visual integration strategy for future dMLLM decoding method design. Our code is openly available at https://github.com/zhaoyk-sysu/CACD-dMLLM.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 27 Aug 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. A genuinely new training-free decoding heuristic for diffusion MLLMs with large reported gains, but the causal story for the neighbor-support term rests on a confounded correlation and the headline numbers lack error bars. This line is wrong
5 major objections · 6 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 74 references say.
The paper says, at Section 5.4, Table 3
The review did not quote this passage.
Pith says: Objection 1 of 5 · major
The text states that "the quality gap widens monotonically" with sequence length, but the differences computed from Table 3 are 6.3, 18.6, 25.3, 18.8, 15.0, 13.7, and 14.8 points at lengths 16, 32, 64, 128, 256, 512, and 1024, which are not monotonic.
more of objection 1
Moreover, Original itself does not degrade monotonically (32.3 at 256 and 35.3 at 512). The abstract and Section 5.4 use this monotonicity to support the void-compounding narrative, so the claim should be corrected or replaced with a properly tested trend.
The paper says, at Section 5.4, Table 4
The review did not quote this passage.
Pith says: Objection 2 of 5 · major
The length confound is not ruled out. On LaViDa, CACD produces 461.2 characters versus 101.5 for Original, a 4.5x increase. LLaVABench-COCO Relative Score and CHAIR are sensitive to output length; the argument that Wavefront is longer but lower-scoring on MMaDA controls only one model and does not show that the gains on LaViDa and LLaDA-V survive length matching. Please provide a length-controlled comparison (for example, matching or truncating outputs, or a per-length-bucket analysis) and report variance.
The paper says, at Appendix D, Table 7, Section 5.1
The review did not quote this passage.
Pith says: Objection 3 of 5 · major
The headline numbers are selected after tuning on the evaluation benchmarks. The per-model threshold tau is chosen based on MathVista accuracy on 100 samples per model (Figure 8 caption), and beta, delta, and rho are swept on MMaDA MathVista and LLaVABench-COCO (Figure 9 and Tables 7-9).
more of objection 3
Reporting the best configuration on the same benchmarks used for the main results makes the reported gains non-independent. Please add a held-out validation protocol, or show that a single fixed default configuration yields comparable gains across models.
The paper says, at Section 3.1, Figure 3a
The review did not quote this passage.
Pith says: Objection 4 of 5 · major
The 5.3x entropy gap between positions with both neighbors decoded and isolated positions is observational and is not stratified by decoding round, number of committed tokens, or confidence. Later-decoded positions have more global context and are more likely to be intrinsically easy tokens, so the gap may reflect progress or intrinsic difficulty rather than a causal effect of neighbor support.
more of objection 4
The void-compounding mechanism in Section 3.1 is inferred from this gap and from commit-round gaps; please stratify the analysis and consider an intervention (such as forced deferral of high-context tokens) to test the causal claim.
The paper says, at Tables 1 and 2
The review did not quote this passage.
Pith says: Objection 5 of 5 · major
The empirical results are reported as single runs with no error bars, so we cannot assess whether small differences such as the 33.50 versus 33.50 tie on MathVista for LLaDA-V or ScienceQA differences of less than one point are significant. Please report multiple seeds with standard deviations and significance tests, and specify the evaluation protocol for LLaVABench-COCO Relative Score.
6 minor objections
The paper says, at Figure 2 and Section 1
Pith says: minor objection 1 of 6
Section 1's Finding 2 refers to "(Figure 2b)" for the block-boundary exclusion rate, but the figure caption places this panel as (c).
The paper says, at Algorithm 1, line 1
Pith says: minor objection 2 of 6
The notation p<--N^t_dec,non-EOS/L is used before N^t_dec is formally defined; please define it or state clearly that it is the number of decoded non-EOS tokens at round t.
show 4 more
The paper says, at Eq. 4
Pith says: minor objection 3 of 6
When no neighbor exists, d_l and d_r are set to infinity and the corresponding term vanishes; please state explicitly that ctx(i) is computed only over masked positions, since committed tokens are excluded from the score.
The paper says, at Title and Section 4.1
Pith says: minor objection 4 of 6
The title and abstract describe "semantic anchors," but Eq. 4 uses only positional distance to any decoded token, not semantic relatedness; please clarify the terminology or justify that positional proximity approximates semantic anchoring.
The paper says, at References
Pith says: minor objection 5 of 6
Zhao et al. 2026a and Zhao et al. 2026b appear to be the same paper with the same title in two venues; please unify the citation.
The paper says, at Table 4
Pith says: minor objection 6 of 6
The "Len" column lacks units; the values are presumably characters, but this should be stated explicitly.
What would settle it
Recompute Figure 3a using the same 295,629 masked positions, but stratify the analysis by decoding round (or by the number of already-decoded tokens at the time of measurement) and by confidence decile. Within each stratum, compare the average softmax entropy of positions with both neighbors decoded against isolated positions; if the entropy gap shrinks substantially or reverses after matching round and confidence, the 5.3× claim is a confound and the context score in Eq. 4 loses its causal justification. As a complementary check, run CACD on MMaDA/LaViDa with ctx(i) replaced by a position-independent constant while keeping β=1.0 and the same thresholds; if the LLaVABench-COCO scores stay near 49.0/88.7, neighbor support is not the operative ingredient and the gains are attributable to other components.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 27 Aug 2026
- Reader holds up, with conditions
- Skeptic agrees
- Editor wrote the verdict above
- Referee major revision: 5 major objections, 6 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Quick take: this is a real contribution to the dMLLM decoding subfield, not a breakthrough. The composite score in Eq. 5 (confidence times a harmonic-decay neighbor term) plus block-free cluster commitment is genuinely new relative to Wavefront and AHD, and the evaluation is broad: three models, four benchmarks, ablations, efficiency tables, text-only extension. The gains on LLaVABench-COCO are large (+37.6 on LaViDa, +23.6 on LLaDA-V, +13.7 on MMaDA). The method is clearly described and the code is promised. If the numbers hold, this is a useful plug-and-play tool.
Credit where due: the paper ships a plausible mechanism, a clean algorithm, and unusually complete sensitivity analysis (β, τ, δ, ρ all swept, with the interaction grid in Figure 9). The ablation shows each component contributes. The text-only extension is a nice sanity check. This is a solid empirical paper, not a toy.
Soft spots, in proportion. The central mechanistic claim — that neighbor support, via ctx(i), identifies contextually ready tokens — rests on Figure 3a: a 5.3× entropy gap between fully-neighbored and isolated positions. The stress-test note is right that this is confounded by decoding round and intrinsic token difficulty. Positions with both neighbors decoded are later in the trajectory and more likely easy tokens. The paper does not stratify by round or confidence. That said, this is a motivation figure, not the load-bearing result. Even if the causal story is overstated, the method could still work for mundane reasons: global confidence selection, EOS suppression, and length effects. Section 5.4's claim of monotonic widening is contradicted by Table 3 (the gap narrows from 512 to 1024), and the paper doesn't acknowledge it. The LaViDa length confound is real but partially addressed by the Wavefront counterexample in Table 4. The absence of error bars is the bigger issue; with per-model τ tuned on 100 MathVista samples, I'd want to see variance across seeds or subsets.
The circularity burden is minor. The method is an empirical heuristic, not a derived prediction. Sweeping hyperparameters on the target benchmarks is normal in this literature, though it does cap how much independent confirmation the headline numbers carry.
Who this is for: anyone working on diffusion LLM decoding, especially dMLLMs. It deserves a serious referee. I'd accept it for review, with a request for error bars, a stratification of Figure 3a, and a fix to the monotonicity wording.
Recommendation: engage. This is the kind of paper worth the referee time, and the authors will likely be responsive to the concerns.
The strongest objection
The central claim is that neighbor proximity, as encoded by the harmonic-decay context score ctx(i) in Eq. 4 and multiplied into the composite score in Eq. 5, identifies contextually ready positions and that exploiting this readiness produces the reported gains. The only direct evidence for this mechanism is Figure 3a, which shows a 5.3× lower average softmax entropy for positions with both neighbors decoded, plus Figure 3c showing commit-round gaps. This evidence is observational and confounded. Positions with both neighbors decoded occur later in the decoding trajectory, when the model has access to more global context; they are also more likely to be intrinsically easy tokens that were committed early under confidence-based decoding. The paper does not stratify by decoding round, by number of already-decoded tokens, or by confidence interval, so the entropy gap may reflect progress or intrinsic difficulty rather than a causal effect of bilateral neighbor support. The mechanism is further strained by the first round: before any neighbor is decoded, ctx(i)=0 for every masked position, so CACD's initial anchors are selected by pure confidence, not by context. Eq. 4 also uses only positional distance to any decoded token, not semantic relatedness, so the semantic-anchor interpretation is not directly tested. If the entropy gap is confounded, the theoretical narrative of void compounding and hard-zone promotion loses support, and the observed LLaVABench-COCO gains could instead come from block-free global confidence selection, EOS suppression, or output-length effects. The headline empirical result could survive, but the design rationale for Eqs. 4-5 would not be established.
I read CACD as claiming a training-free decoding rule whose improvement is caused by rewarding positions near decoded neighbors, with the entropy analysis as the key evidence. That is the load-bearing premise: without it, Eq. 4 is an arbitrary positional heuristic. The reader's weakest assumption identified correlation versus causation, and I agree; I sharpen the concern by noting that 'both neighbors decoded' is a late-round property that is not controlled for in Figure 3a, and that the first decoding round cannot use context at all because ctx(i)=0 everywhere. The purely positional nature of Eq. 4 also weakens the semantic-anchor claim. However, the paper has independent checks that keep this concern from being fatal: the component ablation in Table 5 shows the context term contributes 2.6 points on MMaDA (49.0 vs. 46.4), Table 4 directly addresses length confounding, and the code is released. These are real pieces of evidence, so I would not move the reader's verdict. The recommended verdict remains CONDITIONAL: a stratified re-analysis of Figure 3a (or an equivalent causal test), error bars across seeds, and a pre-specified threshold-selection protocol should be required before the mechanism and the magnitude of the claimed gains are accepted.
The referee's report
If the results hold, CACD would be a useful plug-and-play improvement for dMLLM decoding, and the paper's identification of neighbor support as a scoring signal is a plausible design insight. Strengths include the openly available code, the training-free and architecture-agnostic formulation, and the component ablation. However, the evidence as presented does not yet establish the headline claims: the length-scaling claim is internally contradicted by the paper's own Table 3, the length confound is not resolved, and the hyperparameters are tuned on the evaluation benchmarks without a held-out protocol. The causal narrative also rests on an uncontrolled observational correlation. These issues require substantial revision before the reported gains can be certified.
Pith's reading of the paper (a machine's summary, not the authors')
This paper seeks to show that a large share of long-form quality loss in diffusion multimodal large language models comes from the order in which tokens are committed during decoding, not from the model weights themselves. It claims that existing decoders rank masked positions by raw softmax confidence and partition the sequence into blocks, so they systematically commit isolated, low-support tokens while leaving contextually ready tokens stranded. The proposed Context-Aware Cluster Decoding (CACD) scores every masked position by confidence multiplied by a harmonic-decay proximity to already-decoded neighbours, commits all positions above a threshold in a single global cluster, and calibrates the threshold to each architecture's confidence profile. On three models across four benchmarks, the method raises long-form LLaVABench-COCO scores by 13.7 to 37.6 points and cuts hallucination on MMaDA, with gains growing as outputs lengthen. If the account is right, commitment order is a tractable training-free lever for coherence in parallel diffusion generators.
What carries the argument. The machine that carries the argument is the composite commitment score, equation (5): score(i)=c_i*(1+beta*ctx(i)), where c_i is the top softmax probability at masked position i and ctx(i) is a harmonic-decay context score, 1/(1+d_l(i))+1/(1+d_r(i)), aggregating inverse distances to the nearest decoded tokens on both sides. The context term amplifies confidence when a position is already sandwiched by decided neighbours, which the paper says are 5.3 times more predictable in entropy terms, and collapses to zero for isolated positions. Around this score, CACD builds a block-free cluster commitment: each round takes one full-sequence forward pass and commits every position whose score clears a dynamically scheduled threshold, so a strong anchor anywhere in the sequence can be chosen early. Three calibrations keep it stable across architectures: a two-phase threshold schedule that relaxes from strict precision to completion, soft suppression of premature end-of-sequence tokens, and per-model threshold selection to absorb confidence heterogeneity.
Core claim. The central claim is that contextual readiness — measured as the harmonic-decay proximity of a masked position to its nearest decoded left and right neighbours — should gate commitment, not confidence alone. The paper defines ctx(i)=1/(1+d_l(i))+1/(1+d_r(i)), in [0,2], and scores each masked position as score(i)=c_i*(1+beta*ctx(i)), committing all positions above a scheduled threshold in the same round on the full sequence. It argues that block-free cluster commitment keeps the globally highest-readiness anchor reachable at every step (claimed to be outside the current block in 75% of first-round steps under block decoding), and that architecture-aware calibration is needed because the three evaluated models have radically different confidence distributions (means 0.63, 0.47, and 0.12). The empirical claim is that this recipe raises LLaVABench-COCO from 35.3 to 49.0 on MMaDA, from 51.1 to 88.7 on LaViDa, and from 50.3 to 73.9 on LLaDA-V, with monotonic gains as generation length grows, and reduces object hallucination on two of the three models.
The premise it rests on. The load-bearing premise is that a masked position's low entropy when both neighbours are already decoded is caused by that neighbour support, so the context score names positions that are genuinely ready; if the entropy gap merely reflects that easy tokens were committed earlier, the composite score's gains would not hold up outside the tested models.
If the paper is right (Pith's inferences, not the authors')
If CACD's account is correct, long-form quality degradation in dMLLMs is largely a commitment-order problem and can be improved without any training or weight modification.
The quality gap over block decoding widens with output length: on MMaDA, the relative LLaVABench-COCO improvement is 44.8% at 1024 tokens, so the benefit is largest exactly where baseline drift is worst.
The same commitment rule transfers to text-only diffusion LLMs, improving GSM8K accuracy from 61.0 to 63.0 and IFEval instruction-strict accuracy from 64.74 to 68.59, with fewer forward evaluations on all tested benchmarks.
The method cuts decoding cost as a side effect: on MMaDA LLaVABench-COCO, the number of function evaluations drops from 512 to 45.9, a measured 11.2x speedup under the reported settings.
Hallucination metrics improve with neighbor-aware commitment: CHAIR_s on MMaDA drops from 24.58 to 19.76, and LLaDA-V reaches the lowest CHAIR_i and highest recall among the compared methods.
A natural extension the authors leave implicit is to make the context score a learned or calibrated function of local entropy rather than a fixed harmonic decay; the entropy gap in their Figure 3a suggests the decay rate could be tuned per model.
Because the composite score amplifies positions already adjacent to committed tokens, CACD effectively performs greedy agglomerative clustering over the token sequence; this invites a formal comparison with minimum-weight spanning-tree or shortest-path commitment orders.
The correlation between both-neighbours-decoded and low entropy may partly reflect that easy tokens get committed first; a held-out study matching positions by lexical frequency before comparing entropy would separate these causes.
The architecture-aware calibration finding suggests that a universal decoder may need to estimate the model's confidence distribution online; a testable extension is to set the initial threshold from a running quantile of masked-position confidences, removing the per-model hand-tuning the paper acknowledges as a limitation.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Diffusion multimodal large language models (dMLLMs) frequently produce long-form outputs marred by semantic drift and repetition, with quality generally degrading as output length increases. We identify two structural deficiencies in existing decoding methods as primary drivers of these failures: confidence-based scoring ignores decoded-neighbor support, and block partitioning prevents access to high-readiness semantic anchors, together causing tokens to be committed before their local context is sufficiently established. We propose \ours (\textbfContext-\textbfAware \textbfCluster \textbfDecoding), a training-free decoding method that scores each masked position by a multiplicative composite of softmax confidence and neighbor proximity, promoting contextually ready tokens above isolated candidates while suppressing low-confidence positional noise, operating block-free to keep high-readiness anchors globally accessible. \ours further applies architecture-aware calibration to handle confidence heterogeneity induced by diverse visual integration strategies. Experiments on three dMLLMs across four benchmarks demonstrate consistent quality gains and hallucination reduction over Original, with larger gains in several longer generation settings, highlighting the importance of neighbor support and visual integration strategy for future dMLLM decoding method design. Our code is openly available at https://github.com/zhaoyk-sysu/CACD-dMLLM.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-09-23
The authors' main claim, as the classifier picked it out 1.00
“Experiments on three dMLLMs across four benchmarks demonstrate consistent quality gains and hallucination reduction over Original, with larger gains in several longer generation settings, highlighting the importance of neighbor support and visual integration strategy for future dMLLM decoding method design.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.10 · human participants 0.03 · controlled comparison 0.93 · randomized assignment 0.02 · states quantitative result 0.13 · compares with prior results 0.96 · mentions code or data availability 0.99
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Diffusion multimodal large language models (dMLLMs) frequently produce long-form outputs marred by semantic drift and repetition, with quality generally degrading as output length increases. | background 1.00 | 0.14 | 0.03 | asserted 0.99 |
| We identify two structural deficiencies in existing decoding methods as primary drivers of these failures: confidence-based scoring ignores decoded-neighbor support, and block partitioning prevents access to high-readiness semantic anchors, together causing tokens to be committed before their local context is sufficiently established. | gap 0.88 | 0.77 | 0.10 | asserted 1.00 |
| We propose \ours{} (\textbf{C}ontext-\textbf{A}ware \textbf{C}luster \textbf{D}ecoding), a training-free decoding method that scores each masked position by a multiplicative composite of softmax confidence and neighbor proximity, promoting contextually ready tokens above isolated candidates while suppressing low-confidence positional noise, operating block-free to keep high-readiness anchors globally accessible. | method 0.77 | 0.86 | 0.19 | asserted 1.00 |
| \ours{} further applies architecture-aware calibration to handle confidence heterogeneity induced by diverse visual integration strategies. | method 0.90 | 0.84 | 0.04 | asserted 1.00 |
| Experiments on three dMLLMs across four benchmarks demonstrate consistent quality gains and hallucination reduction over Original, with larger gains in several longer generation settings, highlighting the importance of neighbor support and visual integration strategy for future dMLLM decoding method design. | result 0.95 | 0.96 | 0.75 | asserted 1.00 |
| Our code is openly available at https://github.com/zhaoyk-sysu/CACD-dMLLM. | contribution 0.81 | 0.41 | 0.02 | asserted 0.95 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 1.00
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 5,194 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (12)
Figures are the authors', reproduced from the paper.
References (74)
- [1] Large Language Diffusion Models , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year= on Pith
- [2] Llada2. 0: Scaling up diffusion language models to 100b , author=. arXiv preprint arXiv:2512.15745 , year= on PithPith's reviewarXiv
- [3] Dimple: Discrete diffusion multimodal large language model with parallel decoding , author=. arXiv preprint arXiv:2505.16990 , year= on PithPith's reviewarXiv
- [4] Unified multimodal discrete diffusion , author=. arXiv preprint arXiv:2503.20853 , year= on PithPith's reviewarXiv
- [5] Ling Yang and Ye Tian and Bowen Li and Xinchen Zhang and Ke Shen and Yunhai Tong and Mengdi Wang , booktitle=. 2025 , url= on Pith
- [6] LaViDa: A Large Diffusion Model for Vision-Language Understanding , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year= on Pith
- [7] Zebin You and Shen Nie and Xiaolu Zhang and Jun Hu and Jun Zhou and Zhiwu Lu and Ji-Rong Wen and Chongxuan Li , title=. 2025 , journal= on Pith
- [8] Marianne Arriola and Aaron Gokaslan and Justin T. Chiu and Zhihan Yang and Zhixuan Qi and Jiaqi Han and Subham Sekhar Sahoo and Volodymyr Kuleshov , title=. 2025 , journal= on Pith
- [9] Guanxi Lu and Hao Mark Chen and Yuto Karashima and Zhican Wang and Daichi Fujiki and Hongxiang Fan , booktitle=. AdaBlock-d. 2026 , url= on Pith
- [10] WavefrontDiffusion: Dynamic Decoding Schedule for Improved Reasoning , author=. The Fourteenth International Conference on Learning Representations , year= on Pith
- [11] Yingte Shu and Yuchuan Tian and Chao Xu and Yunhe Wang and Hanting Chen , title=. 2026 , note= on Pith
- [12] Chengyue Wu and Hao Zhang and Shuchen Xue and Zhijian Liu and Shizhe Diao and Ligeng Zhu and Ping Luo and Song Han and Enze Xie , booktitle=. Fast-d. 2026 , url= on Pith
- [13] Daniel Mingyi Israel and Guy Van den Broeck and Aditya Grover , booktitle=. Accelerating Diffusion. 2025 , url= on Pith
- [14] Kecheng Chen and Ziru Liu and Xijia Tao and Hui Liu and Xinyu Fu and Suiyun Zhang and Dandan Tu and Lingpeng Kong and Rui Liu and Haoliang Li , title=. 2025 , note= on Pith
- [15] Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions , author=. Forty-second International Conference on Machine Learning , year= on Pith
- [16] Diffusion Language Models Know the Answer Before Decoding , author=. NeurIPS 2025 Workshop on Efficient Reasoning , year= on Pith
- [17] Next Semantic Scale Prediction via Hierarchical Diffusion Language Models , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year= on Pith
- [18] Qiyan Zhao and Xiaofeng Zhang and Shuochen Chang and Qianyu Chen and Xiaosong Yuan and Xuhang Chen and Luoqi Liu and Jiajun Zhang and Xu-Yao Zhang and Da-Han Wang , title=. CoRR , volume=. 2026 , month= on Pith
- [19] Mitigating Mask Prior Drift and Positional Attention Collapse in Large Diffusion Vision-Language Models , author =. 2026 , eprint = on Pith
- [20] Zhiyuan Liu and Yicun Yang and Yaojie Zhang and Junjie Chen and Chang Zou and Qingyan Wei and Shaobo Wang and Yichen Zhu and Linfeng Zhang , year=. d on Pith
show 54 more
- [21] Empirical Analysis of Decoding Biases in Masked Diffusion Models , author=. 2026 , eprint= on Pith
- [22] Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Models , author=. The Fourteenth International Conference on Learning Representations , year= on Pith
- [23] Wonjun Kang and Kevin Galim and Seunghyuk Oh and Minjae Lee and Yuchen Zeng and Shuibai Zhang and Coleman Richard Charles Hooper and Yuezhou Hu and Hyung Il Koo and Nam Ik Cho and Kangwook Lee , booktitle=. 2026 , url= on Pith
- [24] Shun Zou and Yong Wang and Zehui Chen and Lin Chen and Chongyang Tao and Feng Zhao and Xiangxiang Chu , title=. CoRR , volume=. 2026 , month= on Pith
- [25] Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principles , author=. The Fourteenth International Conference on Learning Representations , year= on Pith
- [26] Visual instruction tuning , author=. Advances in Neural Information Processing Systems , volume= on Pith
- [27] Instructblip: Towards general-purpose vision-language models with instruction tuning , author=. Advances in Neural Information Processing Systems , volume= on Pith
- [28] Minigpt-4: Enhancing vision-language understanding with advanced large language models , author=. arXiv preprint arXiv:2304.10592 , year= on PithPith's reviewarXiv
- [29] Object hallucination in image captioning , author=. arXiv preprint arXiv:1809.02156 , year= on PithPith's reviewarXiv
- [30] MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts , author=. The Twelfth International Conference on Learning Representations , year= on Pith
- [31] Pan Lu and Swaroop Mishra and Tony Xia and Liang Qiu and Kai-Wei Chang and Song-Chun Zhu and Oyvind Tafjord and Peter Clark and Ashwin Kalyan , title=. CoRR , volume=. 2022 , cdate= on Pith
- [32] Discrete Diffusion Language Modeling by Estimating the Ratios of the Data Distribution , author=. 2024 , url= on Pith
- [33] Simple and Effective Masked Diffusion Language Models , author=. NeurIPS 2024 Workshop on Mathematics of Modern Machine Learning , year= on Pith
- [34] Simplified and Generalized Masked Diffusion for Discrete Data , author=. The Thirty-eighth Annual Conference on Neural Information Processing Systems , year= on Pith
- [35] Jingyang Ou and Shen Nie and Kaiwen Xue and Fengqi Zhu and Jiacheng Sun and Zhenguo Li and Chongxuan Li , title=. CoRR , volume=. 2024 , cdate= on Pith
- [36] Shen Nie and Fengqi Zhu and Chao Du and Tianyu Pang and Qian Liu and Guangtao Zeng and Min Lin and Chongxuan Li , title=. 2024 , journal= on Pith
- [37] Dream 7B: Diffusion Large Language Models , author=. 2025 , eprint= on Pith
- [38] DiffusionVL: Translating Any Autoregressive Models into Diffusion Vision Language Models , author=. 2026 , eprint= on Pith
- [39] Jiacheng Ye and Shansan Gong and Jiahui Gao and Junming Fan and Shuang Wu and Wei Bi and Haoli Bai and Lifeng Shang and Lingpeng Kong , title=. CoRR , volume=. 2025 , month= on Pith
- [40] LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Model , author=. 2026 , eprint= on Pith
- [41] Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference , author=. 2025 , eprint= on Pith
- [42] Yu Zhang and Xinchen Li and Jialei Zhou and Hongnan Ma and Zhongwei Wan and Yiwei Shi and Duoqian Miao and Qi Zhang and Longbing Cao , title=. 2026 , journal= on Pith
- [43] Review, Remask, Refine: Process-Guided Block Diffusion for Text Generation , author=. ICML 2025 Workshop on Methods and Opportunities at Small Scale , year= on Pith
- [44] Xu Wang and Chenkai Xu and Yijie Jin and Jiachun Jin and Hao Zhang and Zhijie Deng , title=. 2025 , journal= on Pith
- [45] Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Models , author=. 2025 , url= on Pith
- [46] Parallel Sampling from Masked Diffusion Models via Conditional Independence Testing , author=. The Fourteenth International Conference on Learning Representations , year= on Pith
- [47] Stop the Flip-Flop: Context-Preserving Verification for Fast Revocable Diffusion Decoding , author=. 2026 , eprint= on Pith
- [48] Speculative Diffusion Decoding: Accelerating Language Generation through Diffusion , author=. Proceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , year= on Pith
- [49] Sudhanshu Agrawal and Risheek Garrepalli and Raghavv Goel and Mingu Lee and Christopher Lott and Fatih Porikli , year=. Spiffy: Multiplying Diffusion. 2509.18085 , archivePrefix= on PithPith's reviewarXiv
- [50] Chengyue Wu and Hao Zhang and Shuchen Xue and Shizhe Diao and Yonggan Fu and Zhijian Liu and Pavlo Molchanov and Ping Luo and Song Han and Enze Xie , booktitle=. Fast-d. 2026 , url= on Pith
- [51] Yuchu Jiang and Yue Cai and Xiangzhong Luo and Jiale Fu and Jiarui Wang and Chonghan Liu and Xu Yang , booktitle=. d. 2026 , url= on Pith
- [52] Lingkun Long and Yushi Huang and Shihao Bai and Ruihao Gong and Jun Zhang and Ao Zhou and Jianlei Yang , title=. 2026 , journal= on Pith
- [53] Mask Tokens as Prophet: Fine-Grained Cache Eviction for Efficient dLLM Inference , author=. 2025 , eprint= on Pith
- [54] Mosaic: Unlocking Long-Context Inference for Diffusion LLMs via Global Memory Planning and Dynamic Peak Taming , author=. 2026 , eprint= on Pith
- [55] WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference , author=. 2025 , eprint= on Pith
- [56] Justin Deschenaux and Caglar Gulcehre , title=. 2024 , journal= on Pith
- [57] Minseo Kim and Chenfeng Xu and Coleman Hooper and Harman Singh and Ben Athiwaratkun and Ce Zhang and Kurt Keutzer and Amir Gholami , year=. 2511.19269 , archivePrefix= on PitharXiv
- [58] Runpeng Yu and Qi Li and Xinchao Wang , title=. 2025 , journal= on Pith
- [59] Top 10 Open Challenges Steering the Future of Diffusion Language Model and Its Variants , author=. 2026 , eprint= on Pith
- [60] Confidence-Based Decoding is Provably Efficient for Diffusion Language Models , author=. 2026 , eprint= on Pith
- [61] Adaptation to Intrinsic Dependence in Diffusion Language Models , author=. 2026 , eprint= on Pith
- [62] Why mask diffusion does not work , author=. 2025 , eprint= on Pith
- [63] Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Models , author=. The Fourteenth International Conference on Learning Representations , year= on Pith
- [64] Ziqi Jin and Bin Wang and Xiang Lin and Lidong Bing and Aixin Sun , year=. On the Role of Discreteness in Diffusion. 2512.22630 , archivePrefix= on PitharXiv
- [65] Guhao Feng and Yihan Geng and Jian Guan and Wei Wu and Liwei Wang and Di He , title=. 2025 , journal= on Pith
- [66] From Redundancy to Relevance: Enhancing Explainability in Multimodal Large Language Models , author=. NAACL , year= on Pith
- [67] Xiaofeng Zhang and Yihao Quan and Chen Shen and Chaochen Gu and Xiaosong Yuan and Shaotian Yan and Jiawei Cao and Hao Cheng and Kaijie Wu and Jieping Ye , booktitle=. Shallow Focus, Deep Fixes: Enhancing Shallow Layers Vision Attention Sinks to Alleviate Hallucination in. 2025 , url= on Pith
- [68] Simignore: Exploring and enhancing multimodal large model complex reasoning via similarity computation , author=. Neural Networks , pages= on Pith
- [69] Enhancing Multimodal Large Language Models Complex Reason via Similarity Computation , author=. Proceedings of the AAAI Conference on Artificial Intelligence , year= on Pith
- [70] What drives attention sinks? A study of massive activations and rotational positional encoding in large vision--language models , author=. Information Processing & Management , volume=. 2026 , publisher= on Pith
- [71] Context Tokens are Anchors: Understanding the Repetition Curse in dMLLMs from an Information Flow Perspective , author=. The Fourteenth International Conference on Learning Representations , year= on Pith
- [72] Hallucination Begins Where Saliency Drops , author=. The Fourteenth International Conference on Learning Representations , year= on Pith
- [73] D3ToM: Decider-Guided Dynamic Token Merging for Accelerating Diffusion MLLMs , author=. arXiv preprint arXiv:2511.12280 , year= on PitharXiv
- [74] Diffusion-CAM: Faithful Visual Explanations for dMLLMs , author=. arXiv preprint arXiv:2604.11005 , year= on PithPith's reviewarXiv
Cited by 1 Pith paper
Reviewed papers on Pith that cite this one. The full citation record
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 10 sentences without a confident label
Compares with 1 paper
-
Hong et al. [ref] approach the same symptoms from the attention level, proposing training-free interventions to address repetition and visual grounding degradation; by contrast, operates at the commitment-order level without modifying attention mechanisms.
Mitigating Mask Prior Drift and Positional Attention Collapse in Large Diffusion Vision-Language Models · Related Work · 0.98
Cites as background 3 papers
-
Diffusion multimodal large language models (dMLLMs [ref] ) have emerged as a compelling alternative to autoregressive generation, offering parallel decoding under bidirectional attention as a principled departure from left-to-right token commitment.
Unified Multimodal Discrete Diffusion · Introduction · 1.00
-
Diffusion multimodal large language models (dMLLMs [ref] ) have emerged as a compelling alternative to autoregressive generation, offering parallel decoding under bidirectional attention as a principled departure from left-to-right token commitment.
Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding · Introduction · 1.00
-
Diffusion multimodal large language models (dMLLMs [ref] ) have emerged as a compelling alternative to autoregressive generation, offering parallel decoding under bidirectional attention as a principled departure from left-to-right token commitment.
LLaDA2.0: Scaling Up Diffusion Language Models to 100B · Introduction · 1.00
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment