Pith.

Artificial Intelligence · arXiv:2610.23290 v1 · submitted 16 Oct 2026 · 88 references

CoTA++: Understanding the Repeat Curse in dMLLMs from an Information Flow Perspective

Qiyan Zhao, Xiaofeng Zhang, Xu-Yao Zhang, Shijie Han, Zhepeng Li, Shunzhi Zhu, Da-Han Wang

The paper says

“Diffusion-based multimodal large language models (dMLLMs) decode by iterative denoising with bidirectional attention, which precludes the standard autoregressive key–value cache. Practical deployments therefore rely on approximate caching that reuses intermediate states across denoising steps.”

more of the abstract

Such caching, however, often makes a response repeat one token at adjacent positions in a run that breaks the sentence, a failure we term the Repeat Curse. Tracing the context-token information flow reveals three disruptions. Within a step, caching disperses the attention that anchors a position on nearby context tokens. Across steps, similarity-based refresh is not staleness-aware, so some positions are decoded against stale key–value states. Across layers, the deep-layer entropy of the context fails to converge where repetition arises. We accordingly propose CoTA++, a training-free, plug-and-play framework that counteracts each disruption where it originates: Context-Token Attention Re-anchoring re-forms the attention routing against the current context, Decode-Aware Refresh reserves part of the cache's refresh budget for the positions about to be committed, and Context-Token Entropy-Guided Voting discounts the confidence of a position by the deep-layer entropy of its context, which reorders the unmasking. The three components act at non-overlapping stages of a decoding step and compose freely with existing dMLLMs and caches. Across three dMLLMs and two caching backends, CoTA++ reduces the sample repetition rate of long responses from up to 99.6% to at most 17.0%, recovers most of the accuracy lost to caching on nine benchmarks, and keeps the speedup of caching at under 15% overhead, making approximate caching reliable for dMLLMs.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 19 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up. A well-executed diagnostic-and-fix paper that makes caching reliable for diffusion MLLMs — the empirical work is thorough and the claims are supported. This line is wrong

1 major objection · 6 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 88 references say.

The paper says, at §VI-A, Table VII

The review did not quote this passage.

Pith says: Objection 1 of 1 · major

The 'Judge score' for response quality relies on a proprietary model (gpt-5.6-luna). While the paper provides the protocol reference to supplementary material, the judge is not a publicly available, fixed checkpoint. This raises a limited reproducibility concern for the quality metric.

more of objection 1

The central claims about repetition reduction (SRR, ARR) do not depend on the judge, so this does not undermine the main thesis, but for the journal version the authors should either (a) confirm that the judge model is documented with a specific version identifier that allows exact replication, or (b) supplement with a human evaluation on a subset or with a widely-used open model (e.g., GPT-4o) to increase confidence.

6 minor objections

The paper says, at Abstract and §I

Pith says: minor objection 1 of 6

The phrase 'at most 17.0%' is precise and supported by Table VI (LaViDa under SlowFast+CoTA++ at L=512 gives 17.0%). However, the claim 'makes approximate caching reliable for dMLLMs' is broad; consider softening to 'makes approximate caching substantially more reliable for the evaluated dMLLMs' to match the scope acknowledged in §VIII.

The paper says, at §IV-A, Eq. (5–6)

Pith says: minor objection 2 of 6

The definition of ARR over content tokens (truncated at first EOS) is clear. For completeness, specify how responses without any content token (e.g., empty) are handled; presumably they are excluded from ARR but counted in SRR only if ARR>0? Clarify in the supplementary material.

This objection may describe the copy Pith gave the reviewer, not the paper.

show 4 more

The paper says, at §V-A, Eq. (9–10) and Algorithm 1

Pith says: minor objection 3 of 6

CTAR recomputes attention for positions selected by the monitor. The monitor counts 'context tokens within ±w committed since its last re-anchoring'. It may be helpful to explicitly state that the monitor resets ni to zero for re-anchored positions (as Algorithm 1 line 3 states). In the main text, consider adding a one-sentence note that the monitor is cheap and independent of the cache policy.

The paper says, at §VII-B, Table XII

Pith says: minor objection 4 of 6

The design alternatives table is informative. The row 'Dispersed anchoring (F1)' for DAR slots seems to test filling reserved slots by the F1 signal rather than confidence. The label could be more explicit in the caption: 'DAR slots filled by the positions of most dispersed anchoring (F1)' to avoid confusion.

The paper says, at Fig. 5 and Fig. 6

Pith says: minor objection 5 of 6

The figures are dense but generally legible. Consider enlarging the axis labels in Fig. 5(b–d) and Fig. 6(a) for the print version. The color scheme (red vs. blue) is consistent and accessible.

The paper says, at §IV-E, Joint Diagnosis

Pith says: minor objection 6 of 6

The sentence 'the three findings describe one failure along the three axes of the information flow' is clear. However, the compound-repetition statistic (55% vs. 2.4%) is relegated to supplementary material. Including it in the main text would strengthen the case for the joint diagnosis; consider moving it or a summary to §IV-E.

What would settle it

Verify that the mechanistic patterns reported in Fig. 5 (attention dispersion), Fig. 6 (KV staleness at decode moment), and Fig. 7 (entropy consolidation) hold under the SlowFast caching backend with its default settings, using the same LLaDA-V model at L=128 on the 94 MS-COCO images that repeat under SlowFast. If the same three signatures are found at repeat positions under SlowFast, the claim that the disruptions are generic across caching backends is confirmed; if not, the diagnostic generality is weakened.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 19 Oct 2026

  1. Reader holds up confidence high
  2. Skeptic partly agrees 38 s · 33,413 tokens read, 3,333 written
  3. Editor wrote the verdict above 14 s · 34,263 tokens read, 1,150 written
  4. Referee minor revision: 1 major objection, 6 minor 24 s · 33,708 tokens read, 2,352 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

This paper does something rare: it identifies a concrete failure mode in deployed systems, traces it to specific mechanistic disruptions, and designs a training-free fix that works across models and backends. The core claim — that the Repeat Curse in cached dMLLMs is the convergent symptom of three distinct information-flow disruptions (dispersed anchoring, stale KV states, failed entropy consolidation) — is supported by controlled experiments, ablation studies, and statistical tests. The three components (CTAR, DAR, CTEV) are cleanly motivated, each targeting one disruption, and the ablation grid (Table X) shows that each contributes and that the full combination is the most consistent. The evaluation is unusually broad for this literature: three dMLLMs, two caching backends, three response lengths, nine benchmarks. The results are clear: from up to 99.6% sample repetition rate down to at most 17.0%, while preserving quality and adding under 15% latency overhead. The code is released. The paper is a genuine advance over the conference version [1], both in analysis (the cross-step F2 axis and DAR are new) and in evaluation scale.

The soft spots are proportionate. The independence of the three disruptions is the weakest assumption; the authors acknowledge it and partially test it via ablations, but a full factorial interaction analysis would strengthen the diagnosis. The hyperparameters (κ, w, r, λ) are fixed and ablated, but one could ask whether they generalize across caches without re-tuning. The evaluation covers three models and two caches; the authors honestly note that open-weight dMLLMs and caching methods are still few, so the scope is reasonable but not exhaustive. These are not fatal flaws — they are the usual limits of an empirical paper that chooses depth over breadth. The central argument holds.

Who is this for? Anyone working on dMLLM deployment, inference acceleration, or text degeneration in generation. The paper shows that approximate caching can be made reliable with targeted, lightweight interventions.

Verdict: ACCEPT. I would send this to review without hesitation. It is a solid, honest, reproducible piece of work.

The strongest objection

The paper provides a thorough, multi-faceted argument for both the diagnostic claim (three information-flow disruptions cause the Repeat Curse) and the practical claim (CoTA++ counteracts them). Mechanistic evidence (attention maps, staleness histograms, entropy profiles) is presented for the default cache configuration, and ablation studies (Table X) empirically test the independence and composition of the components. Across three dMLLMs, two caching backends, three response lengths, and nine benchmarks, the central claim holds: CoTA++ reduces the sample repetition rate from up to 99.6% to at most 17.0% while preserving quality and speed. The weakest assumption—that the three disruptions are independent and additive—is partially addressed by component ablations, and the full method's strength across varied settings further validates the design. No single load-bearing concern undermines the central argument; the limitations stated by the authors (few models, backends, and compute) are transparent and do not invalidate the findings within the reported scope.

The reader identified the independence of disruptions/components as the weakest assumption. While the paper does rely on additive decomposition, the ablation study (Table X) and the compound analysis (Section IV-E) provide strong empirical support that the components work in concert without destructive interference. Moreover, the central claim does not hinge on strict independence; even if interactions exist, the full method robustly solves the curse. The paper acknowledges its limitations (few models, backends, compute) and does not overclaim generality beyond the tested configurations. Given the controlled experiments, statistical tests, and reproducible code, the verdict of ACCEPT remains well-supported. I find no load-bearing concern that would warrant a change.

The referee's report

The paper makes a significant contribution to the practical deployment of dMLLMs by providing a mechanistic understanding of a critical failure mode (the Repeat Curse) and offering a lightweight, training-free mitigation. The three-axis information-flow analysis is novel and actionable. The method is validated extensively across multiple models, backends, and benchmarks, and the experimental design includes controlled ablations, statistical tests, and component-wise efficiency analyses. The results are practically impactful: they enable approximate caching to be used without catastrophic repetition, preserving the inference speedup that is essential for dMLLM deployment.

Pith's reading of the paper (a machine's summary, not the authors')

Diffusion-based multimodal large language models (dMLLMs) decode by iterative denoising with bidirectional attention, precluding standard autoregressive caching and forcing reliance on approximate caching that reuses states across steps. This caching often makes a response repeat one token at adjacent positions in a run that breaks the sentence, a failure termed the Repeat Curse. Tracing the information flow of context tokens reveals three disruptions: within a step caching disperses the attention anchoring on nearby context tokens (F1), across steps similarity-based refresh leaves some positions decoded against stale key–value states (F2), and across layers the deep-layer entropy of the context fails to consolidate where repetition arises (F3). The paper proposes CoTA++, a training-free, plug-and-play framework whose three components—Context-Token Attention Re-anchoring, Decode-Aware Refresh, and Context-Token Entropy-Guided Voting—each counteract one disruption at its origin. Across three dMLLMs and two caching backends, CoTA++ reduces the sample repetition rate from up to 99.6% to at most 17.0%, recovers most of the accuracy lost to caching on nine benchmarks, and keeps the speedup of caching at under 15% overhead.

What carries the argument. The central machinery is the three-component framework CoTA++ that operates on the objects corrupted by each disruption. CTAR (Context-Token Attention Re-anchoring) recomputes the attention distribution for positions in deep layers (25–32) when a context token within ±5 has been committed since the last re-anchoring, restoring local anchoring. DAR (Decode-Aware Refresh) reserves r slots in the cache's refresh set for the r masked positions with highest confidence, ensuring they are decoded against fresh KV states. CTEV (Context-Token Entropy-Guided Voting) computes the average deep-layer entropy (layers 26–30) of the committed context tokens within ±5 and subtracts a scaled version from the confidence, producing a voting score that ranks candidates; this causes tokens in consolidated contexts to be committed before those in unconsolidated ones. The three components act on non-overlapping stages of a decoding step (attention output, refresh set, voting score) and compose freely with existing dMLLMs and caching backends.

Core claim. The central claim is that the Repeat Curse is the convergent symptom of three distinct disruptions that approximate caching causes along the three axes of dMLLM inference: routing within a step (dispersed attention anchoring), timeliness across steps (stale KV states due to similarity-based refresh), and consolidation across layers (failed deep-layer entropy convergence). Each disruption has its own signature, object, and mechanism, and the paper demonstrates that they are not redundant by showing that staleness accounts for most of the anchoring loss in the shallow band but almost none in the deep band, and that failed consolidation persists even at equal staleness. The paper further claims that addressing each disruption independently with a targeted intervention suffices to mitigate the curse: CTAR re-forms the attention routing against current keys, DAR reserves cache refresh slots for positions about to be committed, and CTEV discounts the confidence by the context's deep-layer entropy to reorder commitment. Empirical validation across three dMLLMs, two caching backends, and three response lengths shows that the combined framework brings repetition rates close to the uncached baseline while preserving the latency gains of caching.

The premise it rests on. The three disruptions (dispersed anchoring, stale KV states, and failed entropy consolidation) are each independent and additive, and the three proposed components (CTAR, DAR, CTEV) each address exactly one disruption without interfering with the others.

If the paper is right (Pith's inferences, not the authors')

Approximate caching for dMLLMs becomes reliable for long-form generation once the Repeat Curse is removed, preserving both speed and output quality.

The training-free nature of CoTA++ means it can be deployed on existing systems without retraining or fine-tuning, requiring only access to the cache states and confidence ordering.

The framework's component design allows individual parts to be enabled or tuned per deployment, e.g., using only DAR and CTAR for maximum speed while still eliminating long runs.

Because the disruptions are diagnosed along orthogonal axes, the information-flow analysis method may transfer to other artifacts in diffusion inference, such as position-specific quality degradation.

The paper's results imply that future caching designs for dMLLMs should incorporate decode-aware refresh policies and preserve local attention anchoring, not just average state freshness.

The assumption of independent and additive disruptions is challenged by the ablation results: pairs of components sometimes leave residual repetition that the full method removes, and DAR with CTEV at L=128 leaves a run of 25 tokens, suggesting interaction effects that future work could disentangle.

The fixed hyperparameters (κ=1, w=5, r=4, λ=0.25, deep window 26–30) were validated on LLaDA-V and may need adjustment for other dMLLMs or languages; the paper does not provide a sensitivity analysis across model families.

The Repeat Curse is shown to be caching-induced, but recent work (cited as [76]) notes that cache-free decoding can also repeat under aggressive step reduction, hinting at a possible shared root cause (mask-prior drift) that the paper does not address; further isolation experiments could clarify whether CoTA++ also handles that scenario.

The context-entropy penalty (CTEV) could be extended to a continuous risk score that adaptively allocates computational budgets across components, as the paper mentions but leaves to future work.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Diffusion-based multimodal large language models (dMLLMs) decode by iterative denoising with bidirectional attention, which precludes the standard autoregressive key–value cache. Practical deployments therefore rely on approximate caching that reuses intermediate states across denoising steps. Such caching, however, often makes a response repeat one token at adjacent positions in a run that breaks the sentence, a failure we term the Repeat Curse. Tracing the context-token information flow reveals three disruptions. Within a step, caching disperses the attention that anchors a position on nearby context tokens. Across steps, similarity-based refresh is not staleness-aware, so some positions are decoded against stale key–value states. Across layers, the deep-layer entropy of the context fails to converge where repetition arises. We accordingly propose CoTA++, a training-free, plug-and-play framework that counteracts each disruption where it originates: Context-Token Attention Re-anchoring re-forms the attention routing against the current context, Decode-Aware Refresh reserves part of the cache's refresh budget for the positions about to be committed, and Context-Token Entropy-Guided Voting discounts the confidence of a position by the deep-layer entropy of its context, which reorders the unmasking. The three components act at non-overlapping stages of a decoding step and compose freely with existing dMLLMs and caches. Across three dMLLMs and two caching backends, CoTA++ reduces the sample repetition rate of long responses from up to 99.6% to at most 17.0%, recovers most of the accuracy lost to caching on nine benchmarks, and keeps the speedup of caching at under 15% overhead, making approximate caching reliable for dMLLMs.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-20

The authors' main claim, as the classifier picked it out 1.00

“Across three dMLLMs and two caching backends, CoTA++ reduces the sample repetition rate of long responses from up to 99.6\% to at most 17.0\%, recovers most of the accuracy lost to caching on nine benchmarks, and keeps the speedup of caching at under 15\% overhead, making approximate caching reliable for dMLLMs.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.26 · human participants 0.03 · controlled comparison 0.77 · randomized assignment 0.01 · states quantitative result 0.99 · compares with prior results 0.78 · mentions code or data availability 0.04

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Diffusion-based multimodal large language models (dMLLMs) decode by iterative denoising with bidirectional attention, which precludes the standard autoregressive key--value cache. background 1.00 0.09 0.02 asserted 1.00
Practical deployments therefore rely on approximate caching that reuses intermediate states across denoising steps. background 1.00 0.06 0.02 asserted 1.00
Such caching, however, often makes a response repeat one token at adjacent positions in a run that breaks the sentence, a failure we term the \emph{Repeat Curse}. background 0.87 0.34 0.09 asserted 0.84
Tracing the context-token information flow reveals three disruptions. contribution 0.48 0.87 0.05 asserted 0.99
Within a step, caching disperses the attention that anchors a position on nearby context tokens. interpretation 0.59 0.76 0.03 asserted 1.00
Across steps, similarity-based refresh is not staleness-aware, so some positions are decoded against stale key--value states. result 0.47 0.59 0.03 asserted 0.97
Across layers, the deep-layer entropy of the context fails to converge where repetition arises. result 0.75 0.86 0.04 asserted 1.00
We accordingly propose CoTA++, a training-free, plug-and-play framework that counteracts each disruption where it originates: Context-Token Attention Re-anchoring re-forms the attention routing against the current context, Decode-Aware Refresh reserves part of the cache's refresh budget for the positions about to be committed, and Context-Token Entropy-Guided Voting discounts the confidence of a position by the deep-layer entropy of its context, which reorders the unmasking. method 0.67 0.92 0.21 asserted 1.00
The three components act at non-overlapping stages of a decoding step and compose freely with existing dMLLMs and caches. method 0.72 0.65 0.06 asserted 1.00
Across three dMLLMs and two caching backends, CoTA++ reduces the sample repetition rate of long responses from up to 99.6\% to at most 17.0\%, recovers most of the accuracy lost to caching on nine benchmarks, and keeps the speedup of caching at under 15\% overhead, making approximate caching reliable for dMLLMs. result 1.00 0.95 0.88 asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.99

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-20: 1 request, 7,729 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (13)
Figure 1 of the paper
Figure 1. Motivation. LLaDA-V [7] responses to the same query: vanilla decoding is normal (top); dLLM-Cache [18] induces the Repeat Curse (repetitions in red, middle); CoTA++ removes them while retaining the acceleration (bottom). parallel and commits the most confident ones. This paral￾lelism promises efficient decoding, but in practice each step recomputes the whole sequence. Bidirectional attention lets every update perturb the key–value (KV) states of all tokens, so the exact KV cache of autoregressive decoding does not apply [16], [17] and inference remains slow. Recent work instead…
Figure 2 of the paper
Figure 2. The Repeat Curse and its three information-flow signatures. (a) Sample repetition rate on 500 MS-COCO captions, L=512. (b)–(d) LLaDA-V, L=128, vanilla against dLLM-Cache. (b) Within-step attention of suffix queries: caching disperses local anchoring (F1). (c) KV staleness at the decode moment along one response: most repeats (red) decode stale (F2). (d) Cross-layer entropy after generation at that response’s repeat positions and at normal positions: repeats fail to consolidate (F3). three axes. Along each axis, we trace the flow of the context tokens, the tokens around a position…
Figure 3 of the paper
Figure 3. Overview. Caching disrupts the information flow of context tokens on the three axes of dMLLM inference, and CoTA++ counteracts each disruption where it originates. Context-Token Entropy-Guided Voting (CTEV) discounts the confidence of a candidate by the deep-layer entropy of its context, so that the resulting voting score favors candidates in consolidated contexts (against F3). They act on different objects at different moments of a step, the attention output, the refresh set and the voting score, and therefore compose freely with each other and with existing dMLLMs and caches. Exte
Figure 4 of the paper
Figure 4. Radar charts of the Repeat Curse. Near zero without caching, every metric surges with it (labels: vanilla→cached). Severity across models, backends and settings
Figure 5 of the paper
Figure 5. Context tokens are anchors, and caching disperses them (Finding F1). (a) Attention of vanilla LLaDA-V (brighter is higher). (b)–(d) Change in local anchoring under dLLM-Cache on 94 MS-COCO images, ∆w5 (cached − vanilla), where w5 is the share of a position’s suffix attention within ±5 positions. Repeat regions: repeat positions ±5. Distant regions: all other positions. Numbers in (b): ratio of the two losses. Error bars: 95% intervals over images. suffix positions go unrefreshed for several consecutive steps and are decoded against outdated KV states. The analysis below formalizes…
Figure 6 of the paper
Figure 6. Cache staleness concentrates on repeat positions (Finding F2). (a) Staleness at the decode moment on 94 images. (b)–(d) Reference sample, on shared axes: (b) staleness of each position up to its decode (faded afterwards), (c) and (d) how each KV state is read at layer 22 (attention received, averaged over heads and queries). Triangles: repeat positions. 20 24 28 32 Layer 0 5 10 15 Entropy (bits) deep window half of plateau Vanilla Cached, normal Cached, repeat 1 2 3 4 5 6 7 8 9 10 Decile of context entropy 0 10 20 30 40 Repetition rate (%) overall 7.9% 4.6× the rest (a) End of…
Figure 7 of the paper
Figure 7. Consolidation fails at repeat positions (Finding F3). (a) Entropy per layer (shaded: deep window). (b) Repetition rate by decile of the deep-window entropy of the committed neighbors within ±5. Intervals: 95%, over 94 images. recomputed at the next step, whereas a still-masked position’s input is unchanged. The selection thus favors committed positions (40% against 10% per step) and, in the second half of decoding, reaches a still-masked one in under 0.1% of the steps (Table III(b)). The KV states of the positions about to be committed thus grow stale until the next full refresh (
Figure 8 of the paper
Figure 8. The CoTA++ framework. Top: a cached dMLLM reuses the feed-forward, attention and KV states of Step K at Step K−1 and unmasks the response in order of confidence. Badges mark where each component acts. Bottom: each component acts on its own object. CTAR re-anchors the attention of the positions its monitor selects, DAR refreshes the KV states of the positions about to commit, and CTEV discounts the confidence by the deep-layer entropy of the context. at or below the overall rate (7.9%) in the first eight deciles and rises steeply in the last two. The fifth of positions with the…
Figure 9 of the paper
Figure 9. Case study, LLaDA-V under dLLM-Cache: the uncached model, the cached model, and the cached model with CoTA++. (a) The typical case, L=128. (b) A residual failure, L=512: a doubled period at a sentence boundary. TABLE IX: General multimodal benchmarks under (a) dLLM-Cache and (b) SlowFast: DocVQA, ChartQA, MMStar, MME (perception, cognition), SEED, MMBench, LLaVA-Bench (in-the-wild), MathVista and MathVerse. MMaDA uses the MMaDA￾8B-MixCoT checkpoint, the only one with a released evaluation protocol. (a) dLLM-Cache Model Doc Chart MMS MMEP MMEC SEED MMB LLaVAW MVista MVerse LLaDA-V…
Figure 10 of the paper
Figure 10. Attention at the decode moment, with and with￾out CTAR, layers 25–32; triangles: repeat positions. (a)–(b) LLaDA-V and (c)–(d) LaViDa under dLLM-Cache, L=128. 0 32 64 96 127 Decoding step t 0 2 4 6 Staleness Δ (steps) dLLM-Cache: mean 1.89 +DAR: mean 0.003, zero at 99%
Figure 11 of the paper
Figure 11. Staleness of the KV state at the decode moment, with and without DAR. Mean over 100 images at each decoding step, dLLM-Cache, L=128. cache it follows the refresh interval, rising to six steps between refreshes; with DAR it is zero at 99% of the decode moments. 5) CTEV: Consolidation of the Context: F3 located the Repeat Curse in the consolidation of the cross-layer flow: under the cache, repetitions arise in unconsolidated contexts
Figure 13 of the paper
Figure 13. Tokens that open a repeated run under the cache, sized by run count; a leading hyphen marks a word piece. as it was; by the least consolidated (F3), more than half of the responses still repeat; by confidence, 3% do. CTEV: whose entropy, and how strongly. Penalizing a candidate by its own entropy leaves the curse in place, and a veto on the context entropy reproduces the cache (Table XII). The graded penalty mitigates the curse, and λ trades the two statistics: 0.5 lowers the share of responses with a repeat from 61% to 36% but raises the phrase-level statistic above the uncached le

Showing 12 of the paper's 13 figures. Figures are the authors', reproduced from the paper.

References (88)
  1. [1] Q. Zhao, X. Zhang, S. Chang, Q. Chen, X. Yuan, X. Chen, L. Liu, J. Zhang, X.-Y . Zhang, and D.-H. Wang, “Context tokens are anchors: Understanding the repeat curse in dMLLMs from an information flow perspective,” inThe Fourteenth International Conference on Learning Representa... on Pith
  2. [2] S. Nie, F. Zhu, Z. You, X. Zhang, J. Ou, J. Hu, J. ZHOU, Y . Lin, J.-R. Wen, and C. Li, “Large language diffusion models,” inThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025 on Pith
  3. [3] F. Zhu, R. Wang, S. Nie, X. Zhang, C. Wu, J. Hu, J. Zhou, J. Chen, Y . Lin, J.-R. Wen, and C. Li, “Llada 1.5: Variance-reduced preference optimization for large language diffusion models,” 2025 on Pith
  4. [4] J. Ye, Z. Xie, L. Zheng, J. Gao, Z. Wu, X. Jiang, Z. Li, and L. Kong, “Dream 7b,” 2025. [Online]. Available: https: //hkunlp.github.io/blog/2025/dream on Pith
  5. [5] T. Bie, M. Cao, K. Chen, L. Du, M. Gong, Z. Gong, Y . Gu, J. Hu, Z. Huang, Z. Lanet al., “Llada2. 0: Scaling up diffusion language models to 100b,”arXiv preprint arXiv:2512.15745, 2025 on PithPith's reviewarXiv
  6. [6] H. Liu, C. Li, Y . Li, and Y . J. Lee, “Improved baselines with visual instruction tuning,”2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 26 286–26 296, 2023 on Pith
  7. [7] Z. You, S. Nie, X. Zhang, J. Hu, J. Zhou, Z. Lu, J.-R. Wen, and C. Li, “Llada-v: Large language diffusion models with visual instruction tuning,”arXiv preprint arXiv:2505.16933, 2025 on PithPith's reviewarXiv
  8. [8] L. Yang, Y . Tian, B. Li, X. Zhang, K. Shen, Y . Tong, and M. Wang, “MMaDA: Multimodal large diffusion language models,” inThe Thirty- ninth Annual Conference on Neural Information Processing Systems, 2025 on Pith
  9. [9] S. Li, K. Kallidromitis, H. Bansal, A. Gokul, Y . Kato, K. Kozuka, J. Kuen, Z. Lin, K.-W. Chang, and A. Grover, “Lavida: A large diffusion model for vision-language understanding,” inThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025 on Pith
  10. [10] S. Li, J. Gu, K. Liu, Z. Lin, Z. Wei, A. Grover, and J. Kuen, “Lavida-o: Elastic large masked diffusion models for unified multimodal understanding and generation,” inInternational Conference on Learning Representations, vol. 2026, 2026, pp. 50 571–50 603 on Pith
  11. [11] L. Zeng, J. Yao, B. Liao, H. Tao, W. Liu, and X. Wang, “Diffusionvl: Translating any autoregressive models into diffusion vision language models,”arXiv preprint arXiv:2512.15713, 2025 on PitharXiv
  12. [12] I. AI, T. Bie, H. Chen, T. Chen, Z. Cheng, L. Cui, K. Gan, Z. Huang, Z. Lan, H. Liet al., “Llada2. 0-uni: Unifying multimodal understanding and generation with diffusion large language model,”arXiv preprint arXiv:2604.20796, 2026 on PithPith's reviewarXiv
  13. [13] Y . Xin, Q. Qin, S. Luo, K. Zhu, J. Yan, Y . Tai, J. Lei, Y . Cao, K. Wang, Y . Wanget al., “Lumina-dimoo: An omni diffusion large language model for multi-modal generation and understanding,”arXiv preprint arXiv:2510.06308, 2025 on PitharXiv
  14. [14] A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughanet al., “The llama 3 herd of models,”arXiv preprint arXiv:2407.21783, 2024 on PithPith's reviewarXiv
  15. [15] A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Liet al., “Qwen2.5 technical report,”CoRR, vol. abs/2412.15115, 2024 on PithPith's reviewarXiv
  16. [16] R. Xu, J. Xu, Q. Zhao, and X. Zhang, “Fixing semantic blind spots in anchor tokens of dmllms,” inFindings of the Association for Computa- tional Linguistics: ACL 2026, 2026, pp. 19 359–19 374 on Pith
  17. [17] S. Chang, X. Zhang, Q. Liu, and L. Niu, “D3tom: Decider-guided dy- namic token merging for accelerating diffusion mllms,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 40, no. 24, 2026, pp. 19 961–19 969 on Pith
  18. [18] Z. Liu, Y . Yang, Y . Zhang, J. Chen, C. Zou, Q. Wei, S. Wang, Y . Zhu, and L. Zhang, “dLLM-cache: Accelerating diffusion large language models with adaptive caching,” inForty-third International Conference on Machine Learning, 2026 on Pith
  19. [19] Q. Wei, Y . Zhang, Z. Liu, P. Zeng, Y . Wang, B. Qi, D. Liu, and L. Zhang, “Accelerating diffusion large language models with slowfast sampling: The three golden principles,” inInternational Conference on Learning Representations, vol. 2026, 2026, pp. 4340–4354 on Pith
  20. [20] C. Wu, H. Zhang, S. Xue, Z. Liu, S. Diao, L. Zhu, P. Luo, S. Han, and E. Xie, “Fast-dllm: Training-free acceleration of diffusion llm by enabling kv cache and parallel decoding,” inInternational Conference on Learning Representations, vol. 2026, 2026, pp. 57 027–57 051 on Pith
show 68 more
  1. [21] T.-Y . Lin, M. Maire, S. J. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll ´ar, and C. L. Zitnick, “Microsoft coco: Common objects in context,” inEuropean Conference on Computer Vision, 2014 on Pith
  2. [22] Z. Yu, Z. Wang, Y . Fu, H. Shi, K. Shaikh, and Y . C. Lin, “Unveiling and harnessing hidden attention sinks: Enhancing large language models without training through attention calibration,” inForty-first Interna- tional Conference on Machine Learning, 2024 on Pith
  3. [23] L. Wang, L. Li, D. Dai, D. Chen, H. Zhou, F. Meng, J. Zhou, and X. Sun, “Label words are anchors: An information flow perspective for understanding in-context learning,” inProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, pp. 9840–9855 on Pith
  4. [24] L. Chen, H. Zhao, T. Liu, S. Bai, J. Lin, C. Zhou, and B. Chang, “An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models,” inEuropean Conference on Computer Vision, 2024 on Pith
  5. [25] I. Yona, I. Shumailov, J. Hayes, and Y . Gandelsman, “Interpreting the repeated token phenomenon in large language models,” inForty-second International Conference on Machine Learning, 2025 on Pith
  6. [26] X. Zhang, Y . Quan, C. Shen, C. Gu, X. Yuan, S. Yan, J. Cao, H. Cheng, K. Wu, and J. Ye, “Shallow focus, deep fixes: Enhancing shallow layers vision attention sinks to alleviate hallucination in lvlms,” inProceedings of the 2025 Conference on Empirical Methods in Natural Langu... on Pith
  7. [27] D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhorn, J. M ¨uller, J. Penna, and R. Rombach, “Sdxl: Improving latent diffusion models for high-resolution image synthesis,” inInternational Conference on Learning Representations, vol. 2024, 2024, pp. 1862–1874 on Pith
  8. [28] P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. M ¨uller, H. Saini, Y . Levi, D. Lorenz, A. Sauer, F. Boeselet al., “Scaling rectified flow transformers for high-resolution image synthesis,” inForty-first international conference on machine learning, 2024 on Pith
  9. [29] R. ZHANG, S. Zhai, Y . Zhang, J. Thornton, Z. Ou, J. M. Susskind, and N. Jaitly, “Target concrete score matching: A holistic framework for discrete diffusion,” inForty-second International Conference on Machine Learning, 2025 on Pith
  10. [30] S. Gong, M. Li, J. Feng, Z. Wu, and L. Kong, “Diffuseq: Sequence to sequence text generation with diffusion models,” inThe Eleventh International Conference on Learning Representations, 2023 on Pith
  11. [31] K. Zheng, Y . Chen, H. Mao, M.-Y . Liu, J. Zhu, and Q. Zhang, “Masked diffusion models are secretly time-agnostic masked models and exploit inaccurate categorical sampling,” inInternational Conference on Learning Representations, vol. 2025, 2025, pp. 63 186–63 227 on Pith
  12. [32] J. Ou, S. Nie, K. Xue, F. Zhu, J. Sun, Z. Li, and C. Li, “Your absorbing discrete diffusion secretly models the conditional distributions of clean IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. XX, NO. X, 202X 16 data,” inInternational Conference on Learn... on Pith
  13. [33] S. Nie, F. Zhu, C. Du, T. Pang, Q. Liu, G. Zeng, M. Lin, and C. Li, “Scaling up masked diffusion models on text,” inInternational Conference on Learning Representations, vol. 2025, 2025, pp. 82 974– 82 997 on Pith
  14. [34] Z. Xie, J. Ye, L. Zheng, J. Gao, J. Dong, Z. Wu, X. Zhao, S. Gong, X. Jiang, Z. Liet al., “Dream-coder 7b: An open diffusion language model for code,”arXiv preprint arXiv:2509.01142, 2025 on PithPith's reviewarXiv
  15. [35] F. Zhu, Z. You, Y . Xing, Z. Huang, L. Liu, Y . Zhuang, G. Lu, K. Wang, X. Wang, L. Weiet al., “Llada-moe: A sparse moe diffusion language model,”arXiv preprint arXiv:2509.24389, 2025 on PitharXiv
  16. [36] M. Tschannen, A. Gritsenko, X. Wang, M. F. Naeem, I. Alabdul- mohsin, N. Parthasarathy, T. Evans, L. Beyer, Y . Xia, B. Mustafa et al., “Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features,”arXiv preprint arXiv... on PithPith's reviewarXiv
  17. [37] S. Bai, K. Chen, X. Liu, J. Wanget al., “Qwen2.5-vl technical report,” 2025 on Pith
  18. [38] X. Zhang, Y . Zhu, C. Gu, X. Yuan, Q. Zhao, J. Cao, F. Tang, S. Fan, Y . Shen, C. Shenet al., “Hallucination begins where saliency drops,” arXiv preprint arXiv:2601.20279, 2026 on PitharXiv
  19. [39] Q. Zhao, X. Zhang, B. Liu, M. Chen, W. Xiong, J. Chen, G. Ye, W. Yu, X. Yuan, S. Hanet al., “Coalition-aware skill reliability for self-evolving agents,”arXiv preprint arXiv:2608.22610, 2026 on PithPith's reviewarXiv
  20. [40] Y . Zhao, Q. Zhao, J. Zhang, X. Zhang, X. Yuan, and P. Cheng, “Context- aware cluster decoding: Semantic anchor-driven coherence in dmllms,” arXiv preprint arXiv:2608.22367, 2026 on PithPith's reviewarXiv
  21. [41] X. Zhang, Y . Zhu, Q. Zhao, X. Yuan, J. Cao, and X. Chen, “Token- Penalty: Alleviating attention sinks and positional decay in LVLMs,” in Findings of the Association for Computational Linguistics: ACL 2026. San Diego, California, United States: Association for Computational Li... on Pith
  22. [42] R. Pope, S. Douglas, A. Chowdhery, J. Devlin, J. Bradbury, J. Heek, K. Xiao, S. Agrawal, and J. Dean, “Efficiently scaling transformer inference,” inProceedings of Machine Learning and Systems, vol. 5, 2023, pp. 606–624 on Pith
  23. [43] J. Huang, Y . Zhang, Y . Yang, B. Huang, and L. Zhang, “Mask tokens as prophet: Fine-grained cache eviction for efficient dllm inference,” in Findings of the Association for Computational Linguistics: ACL 2026, 2026, pp. 3456–3479 on Pith
  24. [44] Z. Hu, J. Meng, Y . Akhauri, M. Abdelfattah, J.-s. Seo, Z. Zhang, and U. Gupta, “Flashdlm: Accelerating diffusion language model inference via efficient kv caching and guided diffusion,” inInternational Con- ference on Learning Representations, vol. 2026, 2026, pp. 153 529– 153 544 on Pith
  25. [45] Y . Song, Z. Zhang, C. Luo, P. Gao, F. Xia, H. Luo, Z. Li, Y . Yang, H. Yu, X. Quet al., “Seed diffusion: A large-scale diffusion language model with high-speed inference,”arXiv preprint arXiv:2508.02193, 2025 on PithPith's reviewarXiv
  26. [46] W. Wang, B. Fang, C. Jing, Y . Shen, Y . Shen, Q. Wang, H. Ouyang, H. Chen, and C. Shen, “Time is a feature: Exploiting temporal dynamics in diffusion language models,” inInternational Conference on Learning Representations, vol. 2026, 2026, pp. 72 909–72 935 on Pith
  27. [47] X. Ma, R. Yu, G. Fang, and X. Wang, “dkv-cache: The cache for diffusion language models,”Advances in Neural Information Processing Systems, vol. 38, pp. 149 009–149 033, 2026 on Pith
  28. [48] S. Abnar and W. Zuidema, “Quantifying attention flow in transformers,” inProceedings of the 58th annual meeting of the association for computational linguistics, 2020, pp. 4190–4197 on Pith
  29. [49] J. Ferrando and E. V oita, “Information flow routes: Automatically interpreting language models at scale,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, pp. 17 432–17 445 on Pith
  30. [50] Z. Zhang, S. Yadav, F. Han, and E. Shutova, “Cross-modal information flow in multimodal large language models,” in2025 IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025, pp. 19 781–19 791 on Pith
  31. [51] Z. Qiu, Z. Wang, B. Zheng, Z. Huang, K. Wen, S. Yang, R. Men, L. Yu, F. Huang, S. Huanget al., “Gated attention for large language models: Non-linearity, sparsity, and attention-sink-free,”Advances in Neural Information Processing Systems, vol. 38, pp. 100 092–100 118, 2026 on Pith
  32. [52] Y . Song, X. Liu, R. Li, Z. Liu, Z. Huang, Q. Guo, Z. He, and X. Qiu, “Sparse-dllm: Accelerating diffusion llms with dynamic cache eviction,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 40, no. 39, 2026, pp. 33 038–33 046 on Pith
  33. [53] X. Chen, S. Huang, C. Guo, C. Wei, Y . He, J. Zhang, H. Li, and Y . Chen, “Dpad: Efficient diffusion language models with suffix dropout,” in International Conference on Learning Representations, vol. 2026, 2026, pp. 57 286–57 316 on Pith
  34. [54] J. Yao, S. Yang, J. Xu, L. Hu, M. Li, and D. Wang, “Understanding the repeat curse in large language models from a feature perspective,” in Findings of the Association for Computational Linguistics: ACL 2025, 2025, pp. 7787–7815 on Pith
  35. [55] D. Nguyen, A. Payani, and B. Mirzasoleiman, “Beyond semantic en- tropy: Boosting llm uncertainty quantification with pairwise semantic similarity,” inFindings of the Association for Computational Linguistics: ACL 2025, 2025, pp. 4530–4540 on Pith
  36. [56] Y . Zhang, C. Wang, Y . Chen, C. Fan, J. Jia, and S. Liu, “The fragile truth of saliency: Improving llm input attribution via attention bias optimization,”Advances in Neural Information Processing Systems, vol. 38, pp. 154 395–154 421, 2026 on Pith
  37. [57] X. Zhang, Y . Quan, C. Shen, X. Yuan, S. Yan, L. Xie, W. Wang, C. Gu, H. Tang, and J. Ye, “From redundancy to relevance: Information flow in lvlms across reasoning tasks,” inProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Comput... on Pith
  38. [58] Y . Li, H. Wang, X. Ding, H. Wang, and X. Li, “Token activation map to visually explain multimodal llms,” in2025 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2025, pp. 48–58 on Pith
  39. [59] C. Gan, Y . Tu, X. Chen, T. Chen, Y . Li, M. Harandi, and W. Lin, “Unleashing diffusion transformers for visual correspondence by modu- lating massive activations,”Advances in Neural Information Processing Systems, vol. 38, pp. 114 432–114 462, 2026 on Pith
  40. [60] C. Gan, Z. Zhao, Y . Tu, X. Chen, Z. Qin, T. Chen, M. Harandi, and W. Lin, “Massive activations are the key to local detail synthesis in diffusion transformers,” inInternational Conference on Learning Representations, vol. 2026, 2026, pp. 104 855–104 883 on Pith
  41. [61] Q. Huang, X. wen Dong, P. Zhang, B. Wang, C. He, J. Wang, D. Lin, W. Zhang, and N. H. Yu, “Opera: Alleviating hallucination in multi- modal large language models via over-trust penalty and retrospection- allocation,”2024 IEEE/CVF Conference on Computer Vision and Pat- tern Rec... on Pith
  42. [62] Q. Zhao, X. Zhang, Y . Li, Y . Xing, X. Yuan, F. Tang, S. Fan, X. Chen, D.-H. Wang, and X.-Y . Zhang, “Mca-llava: Manhattan causal attention for reducing hallucination in large vision-language models,” in Proceedings of the 33rd ACM International Conference on Multimedia, 2025... on Pith
  43. [63] C. Olsson, N. Elhage, N. Nanda, N. Joseph, N. DasSarma, T. Henighan, B. Mann, A. Askell, Y . Bai, A. Chenet al., “In-context learning and induction heads,”arXiv preprint arXiv:2209.11895, 2022 on PithPith's reviewarXiv
  44. [64] M. E. Rulli, S. Petruzzi, E. Michielon, F. Silvestri, S. Scardapane, and A. Devoto, “Attention sinks in diffusion language models,” inFindings of the Association for Computational Linguistics: ACL 2026, 2026, pp. 43 244–43 256 on Pith
  45. [65] Z. Zhang, Z. Xie, L. Zhong, H. Liu, and S. Cao, “One token is enough: Improving diffusion language models with a sink token,” inFindings of the Association for Computational Linguistics: ACL 2026, 2026, pp. 6479–6490 on Pith
  46. [66] H. Zuo, Y . Li, L. Yang, and X. Zhang, “Diffusion-cam: Faithful visual explanations for dmllms,” inProceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2026, pp. 12 085–12 101 on Pith
  47. [67] W. Zhu, H. Hao, and R. Wang, “Penalty decoding: Well suppress the self-reinforcement effect in open-ended text generation,” inProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, pp. 1218–1228 on Pith
  48. [68] Y . Su, T. Lan, Y . Wang, D. Yogatama, L. Kong, and N. Collier, “A contrastive framework for neural text generation,”Advances in neural information processing systems, vol. 35, pp. 21 548–21 561, 2022 on Pith
  49. [69] Z. Fu, W. Lam, A. M.-C. So, and B. Shi, “A theoretical analysis of the repetition problem in text generation,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 14, 2021, pp. 12 848– 12 856 on Pith
  50. [70] E. Lagutin, D. Gavrilov, and P. Kalaidin, “Implicit unlikelihood train- ing: Improving neural text generation with reinforcement learning,” in Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, 2021, pp. 1432–1441 on Pith
  51. [71] J. Xu, X. Liu, J. Yan, D. Cai, H. Li, and J. Li, “Learning to break the loop: Analyzing and mitigating repetitions for neural text generation,” IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. XX, NO. X, 202X 17 Advances in Neural Information Processing Sys... on Pith
  52. [72] H. Li, T. Lan, Z. Fu, D. Cai, L. Liu, N. Collier, T. Watanabe, and Y . Su, “Repetition in repetition out: Towards understanding neural text degeneration from the data perspective,”Advances in Neural Information Processing Systems, vol. 36, pp. 72 888–72 903, 2023 on Pith
  53. [73] H. Yang, D. Cai, H. Li, W. Lam, S. Shiet al., “A frustratingly simple decoding method for neural text generation,” inProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), 2024, pp. 536–557 on Pith
  54. [74] A. A. Ginart, N. Kodali, J. Lee, C. Xiong, S. Savarese, and J. R. Emmons, “Lz penalty: An information-theoretic repetition penalty for autoregressive language models,”arXiv preprint arXiv:2504.20131, 2025 on PithPith's reviewarXiv
  55. [75] W. Wang, Z. Li, D. Lian, C. Ma, L. Song, and Y . Wei, “Mitigating the language mismatch and repetition issues in llm-based machine translation via model editing,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, pp. 15 681– 15 700 on Pith
  56. [76] S. Hong, C. Yoon, and S. J. Hwang, “Mitigating mask prior drift and positional attention collapse in large diffusion vision-language models,” inForty-third International Conference on Machine Learning, 2026 on Pith
  57. [77] S. Welleck, I. Kulikov, S. Roller, E. Dinan, K. Cho, and J. Weston, “Neural text generation with unlikelihood training,”arXiv preprint arXiv:1908.04319, 2019 on PithPith's reviewarXiv
  58. [78] G. Xiao, Y . Tian, B. Chen, S. Han, and M. Lewis, “Efficient streaming language models with attention sinks,” inInternational Conference on Learning Representations, vol. 2024, 2024, pp. 21 875–21 895 on Pith
  59. [79] R. Ali, F. Caso, C. Irwin, and P. Li `o, “Entropy-lens: Uncovering decision strategies in llms,”arXiv preprint arXiv:2502.16570, 2025 on PitharXiv
  60. [80] M. Mathew, D. Karatzas, and C. Jawahar, “Docvqa: A dataset for vqa on document images,” in2021 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2021, pp. 2199–2208 on Pith
  61. [81] A. Masry, J. Q. Tan, S. Joty, E. Hoqueet al., “Chartqa: A benchmark for question answering about charts with visual and logical reasoning,” inFindings of the association for computational linguistics: ACL 2022, 2022, pp. 2263–2279 on Pith
  62. [82] L. Chen, J. Li, X. Dong, P. Zhang, Y . Zang, Z. Chen, H. Duan, J. Wang, Y . Qiao, D. Linet al., “Are we on the right way for evaluating large vision-language models?”Advances in Neural Information Processing Systems, vol. 37, pp. 27 056–27 087, 2024 on Pith
  63. [83] C. Fu, P. Chen, Y . Shen, Y . Qin, M. Zhang, X. Lin, J. Yang, X. Zheng, K. Li, X. Sunet al., “Mme: A comprehensive evaluation benchmark for multimodal large language models,”Advances in Neural Information Processing Systems, vol. 38, 2026 on Pith
  64. [84] B. Li, R. Wang, G. Wang, Y . Ge, Y . Ge, and Y . Shan, “Seed-bench: Benchmarking multimodal llms with generative comprehension,”arXiv preprint arXiv:2307.16125, 2023 on PithPith's reviewarXiv
  65. [85] Y . Liu, H. Duan, Y . Zhang, B. Li, S. Zhang, W. Zhao, Y . Yuan, J. Wang, C. He, Z. Liuet al., “Mmbench: Is your multi-modal model an all-around player?” inEuropean conference on computer vision. Springer, 2024, pp. 216–233 on Pith
  66. [86] P. Lu, H. Bansal, T. Xia, J. Liu, C. Li, H. Hajishirzi, H. Cheng, K.-W. Chang, M. Galley, and J. Gao, “Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts,” inInternational Conference on Learning Representations, vol. 2024, 2024, pp. 23 439– 23 554 on Pith
  67. [87] R. Zhang, D. Jiang, Y . Zhang, H. Lin, Z. Guo, P. Qiu, A. Zhou, P. Lu, K.- W. Chang, Y . Qiaoet al., “Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems?” inEuropean Conference on Computer Vision. Springer, 2024, pp. 169–186 on Pith
  68. [88] R. Paulus, C. Xiong, and R. Socher, “A deep reinforced model for abstractive summarization,”arXiv preprint arXiv:1705.04304, 2017 on PithPith's reviewarXiv

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment