{"id":"864fba95-b78d-42c5-98bf-a63467057854","arxiv_id":"2506.12035","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"MARche accelerates masked autoregressive image generation by caching stable token projections and refreshing only attention-selected tokens, reaching up to 1.72x speedup with some loss in FID.","lead":"This paper introduces MARche, a training-free inference framework that speeds up masked autoregressive image generation by reusing cached key/value projections for stable tokens and refreshing only contextually important ones. On ImageNet 256x256 it reports up to 1.72x faster generation than MAR, with FID scores that increase by 0.2 to 0.4 depending on model size.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central refresh-selection premise is never directly tested: no evidence that attention-selected tokens are the ones whose K/V actually drift, so the quality-preservation claim rests on an unmeasured correlation.","rationale":"The paper makes a genuine contribution: a training-free, architecture-agnostic inference optimization for MAR with a clean kernel-level online-softmax partition of active and cached attention, and self-reported speedups. The ablations in Tables 2 and 3 and Figure 6 are the right empirical shape. However, the correctness of the approximate algorithm is not established by these ablations. Appendix A proves equivalence only for the exact attention kernel with fresh K/V; the actual algorithm reuses stale K/V and skips FFN for cached tokens, so its behavior is entirely determined by the unquantified stability assumption. The paper's own Figure 1 is a single-image illustration, and Appendix F reports similarity heatmaps but never measures the overlap between high-drift tokens and the attention-selected refresh set. Without that measurement, the top-K heuristic could be selecting tokens that matter for attention but are not the tokens whose projections actually change, or the reverse. The FID comparisons in Table 1 (e.g., MAR-H 1.62 to MARché-H 2.02) already suggest the quality impact is not literally negligible; a per-seed/per-class analysis would reveal whether degradation is systematic or concentrated in failure cases. This concern matches the reader's weakest_assumption, so no verdict change is needed; it remains CONDITIONAL pending a direct test of the refresh-selection mechanism and code release.","tokens_in":15000,"tokens_out":6463,"duration_ms":55064,"concrete_test":"Run MAR-H (using released or reimplemented code) on 1,000 ImageNet validation images with fixed seeds and the default 64-step schedule. At each decoding step, compute per-token cosine drift of K and V between consecutive steps for every layer, and record the Layer-2 attention-based top-K refresh set. Report the mean Jaccard overlap between the refresh set and the top-K highest-drift tokens, plus the per-seed FID distribution of MARché versus MAR. If the overlap is consistently below roughly 50%, or if any seed or class shows FID degradation well beyond the Table 1 average (e.g., > 0.5), the refresh-selection premise is unsupported and the 'negligible impact' claim does not generalize.","verdict_should_be":"UNCHANGED","load_bearing_attack":"MARché's 1.72x speedup and quality-preservation claim rest on two empirical premises: (i) most K/V projections are stable across decoding steps, and (ii) the top-K tokens selected by Layer-2 attention scores from generating tokens are exactly the tokens whose recomputation matters. Figure 1 documents premise (i) for one image, one layer, and one pair of steps; Appendix F broadens this to several layers and steps but never connects stability to the refresh choice. Section 4.2 defines the refresh set by attention scores, but no experiment reports overlap between that set and the set of tokens with the largest K/V drift. Algorithm 1 leaves all non-active tokens stale, skipping both their attention and FFN updates, so active-token outputs carry an unbounded approximation error between the periodic full refreshes (every 3 steps). Appendix A's equivalence proof assumes fresh K/V for all tokens and therefore does not cover the actual algorithm. Tables 2 and 3 show the selection heuristic beats random and low-attention baselines, but that is indirect: a heuristic can beat those baselines while still failing to refresh the truly drifting tokens, with quality loss hidden by FID averaging over seeds and classes. Because the method is explicitly training-free and architecture-preserving, the entire contribution hinges on this unmeasured correlation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MARché, a training-free inference-time framework to accelerate masked autoregressive (MAR) image generation by caching key/value projections and recomputing them only for a small active set of tokens. The active set is formed from newly generated tokens and a fixed budget of tokens selected by attention scores in an early decoder layer, with a periodic full refresh every three steps. The authors report speedups up to 1.72x on ImageNet 256x256 with FID and IS close to the original MAR models, and they provide a mathematical equivalence argument for a cache-aware attention kernel based on online softmax. The central claim is that most token representations are stable across decoding steps and that attention-based selection identifies exactly the tokens whose recomputation matters for quality.","tokens_in":15233,"tokens_out":2563,"duration_ms":20909,"significance":"If the speedup and quality-preservation claims are established, MARché would be a practically useful, architecture-agnostic inference optimization for a prominent class of generative transformers, since it requires no retraining or architectural changes. The motivation is well grounded in prior KV-cache work for LLMs, and the kernel-level latency measurements in Appendix B provide concrete evidence that the partitioned attention formulation can be implemented efficiently. The online-softmax equivalence is correct under the assumption of fresh cache values. However, the paper's quality claim rests on an empirical link between attention scores and token drift that is motivated only indirectly, and the reported FID/IS degradations are larger than the phrase 'negligible impact' suggests. The lack of error bars further weakens the quantitative claims.","major_comments":[{"comment":"The abstract and Section 5.2 characterize the image-quality impact as 'negligible,' but Table 1 shows FID increases of 0.21 (MAR-B), 0.32 (MAR-L), and 0.40 (MAR-H), while IS decreases by 10.8, 17.7, and 17.2 points respectively. These are not obviously negligible, and no error bars or multiple-seed results are reported for FID, IS, or latency. The authors should either temper the claim or provide statistical evidence (e.g., confidence intervals over seeds) that the degradation is within noise.","section":"§5.2, Table 1"},{"comment":"The central premise that attention-score-selected tokens are the ones whose K/V representations actually drift is never directly tested. Figure 1 and Appendix F show that most tokens are stable, but they do not identify which tokens drift or whether the drift set overlaps with the attention-selected refresh set. Tables 2 and 3 compare MARché against random and low-attention baselines, which is indirect evidence; a heuristic can beat those baselines while still failing to refresh the truly drifting tokens. The authors should add a direct measurement, e.g., the overlap between the top-K attention tokens and the top-K tokens by K/V change, along with a failure-case analysis for images where FID degrades most.","section":"§4.2, Algorithm 1, Appendix F"},{"comment":"The equivalence proof in Appendix A assumes that cached K/V values equal the values that would be produced by a fresh forward pass. In the actual algorithm, non-active tokens' K/V values are stale for up to three steps, and active tokens' outputs are computed using these stale contexts. Therefore the proof does not cover the approximation error of the deployed method. The paper provides no bound or empirical characterization of the cumulative drift introduced by stale caches, which is load-bearing for the quality-preservation claim.","section":"§4.3, Algorithm 1, Appendix A"},{"comment":"The latency numbers are inconsistent across the paper. Table 1 reports MARché-B at 0.064 s/image, but Figure 5 reports the Layer-2 default configuration at about 0.159 s/image and Figure 6 reports around 0.12–0.18 s/image for similar settings. The figure captions do not specify which model scale is used, and the same issue affects Table 4, where the text quotes 0.158 s/im while the table lists 0.155 s/im. The authors should unify the latency reporting and clearly state the model scale and hardware configuration for every ablation.","section":"Table 1 vs. §5.3/Figures 5–6"},{"comment":"The two random-selection baselines differ by a factor of nearly 200 in FID (564.61 in Table 2 vs. 3.01 in Table 3), which is confusing because the captions are similar. If these are different experiments (random construction of the entire active set vs. random selection of refreshing tokens only), the distinction should be explicit in the text and captions; otherwise the results appear mutually inconsistent and undermine confidence in the ablations.","section":"§5.3, Tables 2 and 3"}],"minor_comments":[{"comment":"The sentence 'We then use the attention scores of the first decode layer to select the top-K tokens' appears to contradict Section 4.2, which says Layer 2 is used for selection. Please clarify which layer is used.","section":"§4.3"},{"comment":"The text says 'Full at layer 1 only yields the fastest inference (0.158 s/im)', but Table 4 lists 0.155 s/im for that entry. Please align the numbers.","section":"Appendix D, Table 4"},{"comment":"The caption says 'Deeper layers improves FID' (subject-verb agreement) and the text contains 'but but' in the same paragraph. Please proofread.","section":"Section 5.3, Figure 5 caption"},{"comment":"The name is typeset as 'MARch\\'e' in the abstract and introduction, but the rest of the paper uses 'MARché' or 'MARche'. Please use a consistent notation throughout, including in the title.","section":"Abstract and Section 1"},{"comment":"The claim that outputs are 'virtually indistinguishable' is based on visual inspection of a single figure; please either show quantitative perceptual metrics or soften the claim.","section":"Appendix E"}],"recommendation":"major_revision","confidential_remarks":"The paper's idea is timely and the kernel-level speedup evidence is useful, but the quality-preservation claim is not yet supported by the reported numbers or by a direct test of the refresh-selection mechanism. The missing overlap analysis between attention-selected tokens and high-drift tokens is the key experiment that would resolve the central concern; without it, the contribution is an engineering heuristic whose failure modes are unknown. I would encourage the editors to request that experiment in the revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is real and useful: MAR recomputes KV projections for all tokens at every step, and MARch\\'e shows you can cache the stable ones and only recompute a small active set, with attention scores guiding which tokens to refresh. That is a genuine extension of LLM KV eviction and of LazyMAR, which caches externally at the feature level rather than inside the attention mechanism. The ablations are also thoughtful: excluding generating tokens collapses FID, attention-based selection beats random and low-attention baselines, and the refresh frequency/layer placement trade-offs are honestly shown as latency-vs-FID curves. The online-softmax equivalence proof in Appendix A is correct as far as it goes.\n\nThe soft spots are real but fixable. First, Table 1 shows FID increasing by 0.21 to 0.40 and IS dropping at every scale; calling that \"negligible\" is an overstatement for state-of-the-art generation. Second, and more important, the paper never directly tests the central premise that the attention-selected tokens are the ones whose KV projections actually drift. Figures 1, 10, and 11 only show that most projections are stable; they do not connect that stability to the refresh choice. The ablation against random/low-attention selection is indirect evidence, not a verification of the correlation the method relies on. Third, the latency numbers are inconsistent: Table 1 reports 0.064 s/image for MARch\\'e-B, while Section 5.3 and Figure 5 report around 0.155-0.160 s/image for what appears to be the same model, and Table 4 adds another conflicting number. That needs reconciliation. Fourth, there is no comparison against LazyMAR, which is the closest prior work. Missing code is also a practical barrier.\n\nNone of this sinks the paper. The central mechanism is plausible, the speedups are meaningful at the reported scale, and the trade-off analysis is more transparent than most efficiency papers. But the current version overclaims quality preservation and under-delivers on evidence for the refresh heuristic. I would send this to peer review rather than desk reject it, with the expectation of major revision: add a direct overlap analysis between attention-selected tokens and highest-drift tokens, fix the latency reporting, temper the \"negligible\" language, and compare to LazyMAR.\n\nWho is this for? Practitioners working on efficient inference for MAR-style models, and researchers extending KV-reuse techniques from causal to bidirectional masked generation. It is not a paradigm shift, but it is a step forward in a practical direction.","headline":"A solid, incremental training-free speedup for MAR image generation, but the quality-preservation claim is overstated and the refresh-selection premise needs direct validation before this is publishable.","tokens_in":15807,"tokens_out":1865,"would_cite":true,"duration_ms":24472,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MARché caches stable key/value projections and refreshes only attention-relevant tokens, claiming up to 1.72x faster masked autoregressive image generation with small FID cost.","keywords":["masked autoregressive image generation","KV cache reuse","cache-aware attention","selective KV refresh","inference acceleration","transformer decoding","ImageNet 256x256"],"falsifier":"A direct test is to compare MARché against full MAR across many images and several random generation-order permutations: if the FID gap varies strongly with the permutation, or if some image shows large FID degradation while its top-K attention-selected tokens all have near-0.95 cosine KV stability, then the temporal-locality and attention-selection assumptions fail.","tokens_in":14746,"feed_emoji":"⚡","tokens_out":6851,"duration_ms":49511,"temperature":0.7,"pith_summary":"Masked autoregressive (MAR) image transformers recompute attention and feed-forward outputs for every token at every decoding step, even though only a few tokens are newly generated. This paper claims that most key/value projections change very little across steps, so most of that recomputation is redundant. MARché is a training-free inference strategy that caches stable KV projections, recomputes only active tokens, and refreshes the top-K contextually relevant tokens chosen by attention scores from newly generated tokens. On ImageNet 256x256 it reports 1.57x, 1.68x, and 1.72x speedups over MAR-B/L/H while keeping FID increases to roughly 0.2-0.4. If right, MAR models can be served markedly faster without retraining or architecture changes.","feed_headline":"MAR image generation gets a 1.72x KV-cache speedup","feed_subtitle":"A training-free cache refreshes only attended tokens, holding FID penalties near 0.2-0.4.","key_machinery":"The key mechanism is cache-aware attention, which partitions tokens into active and cached sets at each decoding step and runs two separate computation paths. Active tokens compute fresh queries, keys, and values and attend over both sets, while cached tokens contribute only their stored key/value projections; the two attention sums are merged with the safe online softmax formulation, making the split exactly equivalent to standard full attention whenever cache values are current. Selective KV refresh identifies the active set's refreshing tokens by ranking attention scores from newly generated tokens at decoder layer 2, and a periodic full refresh every three steps prevents stale cached values from accumulating error.","core_discovery":"The central claim is that MAR models exhibit strong temporal locality: between adjacent decoding steps, most token key projections stay above roughly 0.95 cosine similarity, and value projections are only slightly more dynamic. Exploiting this, cache-aware attention divides tokens into an active set and a cached set, computing fresh key/value projections and feed-forward outputs only for active tokens while reusing stored KV projections for the rest. Selective KV refresh then picks a fixed budget of refreshing tokens by aggregating attention scores from generating tokens across heads in an early decoder layer, plus periodic full refresh every three steps to control drift. The paper reports up to 1.72x lower latency with FID penalties around 0.2-0.4 across model scales, with no change to the underlying architecture or training.","pith_inferences":["An adaptive refresh budget that grows when attention scores are diffuse could improve the quality-latency trade-off beyond the fixed 64-token budget tested here.","The same KV-stability argument could transfer to other bidirectional iterative generators, such as multi-modal masked models, but the paper's evidence is limited to class-conditional ImageNet at 256 resolution.","A drift-triggered full refresh, recomputing all KV values when measured similarity drops below a threshold, might beat the fixed every-three-steps schedule at the cost of an extra similarity computation."],"forward_implications":["Masked autoregressive models can be served 1.57-1.72x faster on existing hardware without retraining or changing the transformer architecture.","The reported speedup grows with model size, so larger MAR models stand to gain more from cache-aware decoding.","Because the active/cached split is exact when cache values are fresh, the only quality loss comes from stale KV entries, not from the attention formulation.","Attention scores from an early decoder layer suffice to decide which tokens to refresh, avoiding the cost of running all layers to make that decision."],"supporting_citations":[{"why":"Defines the MAR architecture, decoding schedule, and baseline FID/latency numbers that MARché must beat.","marker":"[22]"},{"why":"Masked generative transformer baseline and predecessor that motivates parallel token generation with bidirectional attention.","marker":"[19]"},{"why":"Prior work using attention scores to manage KV caches, whose token-relevance idea is repurposed for selecting which tokens to refresh.","marker":"[27]"},{"why":"Supplies the safe online softmax formulation used to merge active and cached attention outputs without concatenation.","marker":"[49]"},{"why":"A related feature-caching acceleration for MAR, serving as the contrast that shows MARché works at the attention level and training-free.","marker":"[46]"},{"why":"Diffusion transformer baseline used in the latency-quality comparison table.","marker":"[54]"},{"why":"Fréchet Inception Distance, the metric used to verify that speedups do not degrade image quality.","marker":"[52]"}],"fun_headline_variants":["MARch\\'e: 1.7x faster masked autoregressive image generation","Cache-aware attention speeds up MAR image gen by 1.7x","Selective KV refresh cuts MAR latency, preserves image quality","Training-free cache reuse gives MAR image generation 1.7x speedup","Masked autoregressive image models get 1.7x speedup via token caching"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is empirical: most key/value projections in a MAR decoder stay nearly unchanged across decoding steps, so stale cached values can be reused without error accumulating, and the top-K tokens chosen by attention scores from newly generated tokens are the ones that must be recomputed; the paper provides no bound or failure-case analysis for this premise.","fun_headline_variants_meta":{"raw":{"variants":["MARch\\'e: 1.7x faster masked autoregressive image generation","Cache-aware attention speeds up MAR image gen by 1.7x","Selective KV refresh cuts MAR latency, preserves image quality","Training-free cache reuse gives MAR image generation 1.7x speedup","Masked autoregressive image models get 1.7x speedup via token caching"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000315,"raw_usage":{"total_tokens":1780,"prompt_tokens":933,"completion_tokens":847,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":549,"completion_tokens_details":{"reasoning_tokens":747}},"tokens_in":549,"tokens_out":847,"duration_ms":5593,"temperature":1.0,"reasoning_tokens":747,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:48:21.109530+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test is to compare MARché against full MAR across many images and several random generation-order permutations: if the FID gap varies strongly with the permutation, or if some image shows large FID degradation while its top-K attention-selected tokens all have near-0.95 cosine KV stability, then the temporal-locality and attention-selection assumptions fail.","supporting_citations":[{"cited_title":"Scalable diffusion models with transformers","cited_arxiv_id":null,"evidence_quote":"Diffusion transformer baseline used in the latency-quality comparison table."},{"cited_title":"Gans trained by a two time-scale update rule converge to a local nash equilibrium","cited_arxiv_id":null,"evidence_quote":"Fréchet Inception Distance, the metric used to verify that speedups do not degrade image quality."}],"review_version":1}