{"id":"ffa299d0-1ac2-4c83-aa95-caca96da80c6","arxiv_id":"2607.08056","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":6,"one_line_summary":"A learnable control block trained with GRPO to select token generation order in multimodal masked diffusion models improves text-to-image alignment and multimodal understanding over logit-based heuristics.","lead":"This paper trains a small neural module to decide which tokens a multimodal diffusion model should generate first, improving image generation and visual question answering. It matters because diffusion models normally generate tokens in random order, and a smarter order yields better spatial reasoning and image quality.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"No variance estimates or multiple seeds reported; 4-5% relative gains may not be statistically distinguishable from noise.","rationale":"The reader's weakest_assumption focuses on reward model fidelity (PickScore as a proxy for compositional correctness). This is a valid concern but is partially addressed by the paper's own cross-metric experiment (Table 1: training with PickScore, evaluating on GenEval yields 0.50 vs 0.51 with GenEval rewards — a small gap suggesting reward fidelity matters but is not the dominant issue). The more fundamental concern is statistical: without variance estimates, we cannot determine whether *any* of the reported differences are real. The reader does flag this as issue (1) in their rationale, but does not elevate it to the primary load-bearing concern. I elevate it because: (a) it is the simplest explanation for the mixed results pattern (gains on some benchmarks, losses on others, tiny absolute differences); (b) it is directly testable and would either confirm or refute the central claim; (c) the stochastic nature of both the Plackett-Luce unmasking policy and GRPO trajectory sampling makes variance a first-order concern, not a secondary detail. The reader's reward-fidelity concern is real but secondary — even with a perfect reward model, if the gains are within noise, the method does not work. The architectural ablation concern (full-trajectory vs single-step gradients, reader's issue 5) is also valid but is about understanding *why* the method works, not *whether* it works. The unexplained Counting degradation (0.34→0.25, reader's issue 3) is concerning and may relate to reward misalignment, but again is secondary to the question of whether any observed difference is statistically meaningful. The verdict remains CONDITIONAL — the method is reasonable, the experiments are standard, but the evidence is insufficiently robust for the claims made.","tokens_in":12365,"tokens_out":815,"duration_ms":182257,"concrete_test":"Re-run the GenEval and VLMEvalKit evaluations with at least 3 independent seeds (varying both GRPO training initialization and inference sampling). Report mean ± standard deviation for each metric in Tables 1 and 2. If the 95% confidence intervals for the overall scores overlap between the baseline and the proposed method, the 'broad effectiveness' claim is not supported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim of 'broad effectiveness' rests on relative improvements of 4.08% (GenEval) and 4.85% (VLMEvalKit). However, Tables 1 and 2 report single-run point estimates with no standard deviations, confidence intervals, or multiple-seed runs. For GenEval overall, the improvement is 0.49→0.51 (PickScore reward) or 0.49→0.51 (GenEval reward) — a 0.02 absolute gain. On VLMEvalKit, several sub-benchmarks show the proposed method performing *worse* than the baseline (GQA: 48.5→47.6; MMMU: 80.2→10.3 appears to be a typo but MMB: 70.2→10.3 is also suspicious; SEED: 0.52→0.24). The aggregate '4.85% relative improvement' appears to be computed across benchmarks where some show degradation. Without variance estimates, it is impossible to determine whether these small gains reflect a real effect of the learned generation order policy or simply run-to-run sampling variance in diffusion models (which use stochastic unmasking via Plackett-Luce). The GRPO training itself involves stochastic sampling of G=4 trajectories per prompt, adding another source of variance. This is the most load-bearing concern because if the gains are within noise, the entire contribution collapses.","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"This paper investigates the optimization of token generation order in multimodal masked diffusion models (MDMs). The authors first establish that existing logit-based heuristics (Top-K, Top-K margin), which are effective for structured language tasks like Sudoku, do not transfer to multimodal settings (text-to-image generation and multimodal understanding). To address this, they propose a learnable control block — built on the Unmask Policy Module (UPM) from Huang et al. (2025) — that assigns ranking scores to masked tokens at each denoising step, with selection governed by a Plackett-Luce policy. The control block is trained via Group Relative Policy Optimization (GRPO), accumulating policy gradients across all denoising steps rather than using a single-step density ratio as in prior diffu-GRPO work. Experiments are conducted on the GenEval benchmark for text-to-image generation and VLMEvalKit for multimodal understanding, using MMaDA-8B-MixCoT as the base model. The authors report 4.08% and 4.85% relative improvements on these benchmarks, respectively.","tokens_in":13218,"tokens_out":1627,"duration_ms":173148,"significance":"The paper addresses a timely and well-motivated question: whether generation order matters in multimodal MDMs and, if so, how to optimize it. The negative result — that logit-based heuristics fail in multimodal settings — is a useful empirical finding. The proposed solution (a GRPO-trained control block with full-trajectory gradient accumulation) is technically reasonable and builds cleanly on prior work (MMaDA, diffu-GRPO, UPM) with appropriate citations. The experimental setup includes proper deduplication of GenEval prompts to prevent data leakage and evaluation on standard benchmarks. However, the significance of the results is tempered by the small magnitude of improvements (0.02 absolute on GenEval overall), the absence of variance estimates, and apparent data presentation issues in Table 2 that make the multimodal understanding results difficult to interpret.","major_comments":[{"comment":"Tables 1 and 2 report only single-run point estimates with no standard deviations, confidence intervals, or multiple-seed runs. The GenEval overall improvement is 0.49 to 0.51 (absolute gain of 0.02). Given that the generation process involves stochastic unmasking via Plackett-Luce sampling (Section 3.1) and GRPO training samples G=4 stochastic trajectories per prompt (Section 3.2), there are multiple sources of variance. Without variance estimates, it is impossible to determine whether these small gains reflect a real effect of the learned generation order policy or run-to-run noise. This is load-bearing for the central claim of 'substantial improvements.'","section":null},{"comment":"Table 2 appears to contain data presentation errors that undermine the multimodal understanding results. For MMMU, the baseline MMaDA-COT reports 80.2 while the proposed method reports 10.3 — this looks like a decimal point or formatting error (likely 80.2 vs 10.3 should be 80.2 vs 10.3 is implausible; perhaps 80.2 vs 10.3 should be 80.2 vs 10.3 — regardless, a drop from 80.2 to 10.3 would be catastrophic, not an improvement). Similarly, MMB shows 70.2 for the baseline and 10.3 for the proposed method, and SEED shows 0.52 for the baseline and 0.24 for the proposed method. These values are inconsistent with the claimed 4.85% relative improvement and must be corrected before the results can be properly assessed.","section":null},{"comment":"The '4.85% relative improvement in multimodal understanding' claimed in the abstract is not clearly supported by Table 2. Several sub-benchmarks show the proposed method performing worse than the baseline (GQA: 48.5 to 47.6; SEED: 0.52 to 0.24). The paper does not specify how the aggregate improvement is computed — whether it is an average across benchmarks, a weighted average, or something else. The authors should clarify the aggregation method and report per-benchmark improvements transparently, including which benchmarks show degradation.","section":null},{"comment":"Section 3.2, Eq. (1): The paper states that gradients are accumulated across all T denoising steps, in contrast to the single-step density ratio estimation in Zhao et al. (2025) and Yang et al. (2025). However, the paper does not provide an ablation isolating the contribution of full-trajectory gradient accumulation versus single-step estimation. Since this is presented as a key methodological difference from prior work, an ablation or at least a discussion of why full-trajectory accumulation is necessary would strengthen the contribution. Additionally, the relationship between the per-step policy ratio in Eq. (1) and the sequence-level likelihood approximation used in diffu-GRPO (Section 2.2) should be clarified — is the product of per-step ratios equivalent to the mean-field sequence likelihood used in diffu-GRPO?","section":null}],"minor_comments":[{"comment":"Only one base model (MMaDA-8B-MixCoT) is tested. While this is understandable given computational constraints, the generality of the approach across different multimodal MDM architectures is unverified. A brief discussion of whether the control block is expected to transfer to other architectures would strengthen the generality claim.","section":null},{"comment":"Table 1: The two variants of the proposed method (PickScore reward vs GenEval reward) show very similar overall scores (0.50 vs 0.51). The paper notes that training with PickScore and evaluating on GenEval demonstrates generalization, but the difference is within what could be noise given no variance estimates.","section":null},{"comment":"Section 4.1: The prompt sampling ratio (Position:Counting:Attribute Binding:Colors:Two Objects:Single Object = 7:5:3:1:1:0) is stated without justification. How sensitive are the results to this ratio? Was it tuned on a validation set?","section":null},{"comment":"The reward weight R = 2*R_correctness + 0.4*R_format (Section 4.2) is presented without justification for the specific coefficients. A brief ablation or sensitivity analysis on this weighting would be helpful.","section":null},{"comment":"Figure references (Figures 2, 3, 4) are mentioned in the text but the figures themselves were not fully legible in the review materials. The authors should ensure trajectory visualizations and qualitative comparisons are clearly readable and properly labeled.","section":null},{"comment":"The notation switches between x_T for clean data (Section 2.1) and x_0 for the initial masked state, which may confuse readers familiar with the standard convention where x_0 is clean data. The footnote explains this choice but it adds cognitive overhead.","section":null},{"comment":"The KL penalty coefficient beta is set to 0 (Appendix A). This means the KL regularization term in the GRPO objective is effectively disabled. The authors should discuss whether this affects training stability or policy collapse, and whether experiments with non-zero beta were attempted.","section":null}],"recommendation":"major_revision","confidential_remarks":"The Table 2 data presentation issues are the most pressing concern — the values for MMMU, MMB, and SEED appear to contain formatting or transcription errors that make the multimodal understanding results unverifiable as presented. If these are simply formatting errors (e.g., decimal points in wrong positions), the corrected results may still support the claims, but this must be verified. The absence of variance estimates is also a significant concern given the small effect sizes. I would recommend the authors be asked to provide corrected tables with multiple-seed runs before the paper can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"glm-5.2","letter":"Quick read on 2607.08056. The core finding is that logit-based order heuristics (Top-K, Top-K margin) which work for Sudoku-style language tasks don't help for multimodal diffusion — and a learned control block trained with GRPO does. That's a genuine observation worth making, and the method is a reasonable combination of existing pieces (UPM from Huang et al., diffu-GRPO from Zhao et al., Plackett-Luce ordering) applied to a new setting. The paper is honest about the limitations of confidence-based strategies and provides a clean negative result before proposing the fix. Credit for that. The trajectory visualizations in Figures 2-3 are actually informative — showing the control block producing more detailed reasoning chains in multimodal understanding is a nice qualitative result. The prompt deduplication on GenEval to prevent leakage is good practice. Now the problems. The gains are small: GenEval overall goes 0.49 to 0.51, VLMEvalKit shows mixed results with some benchmarks degrading (GQA drops, SEED drops). The paper reports no error bars, no multiple seeds, no variance estimates at all. For a stochastic method using Plackett-Luce sampling with G=4 trajectories, that's a real gap — a 0.02 absolute improvement on GenEval could easily be within noise. The stress-test concern about this lands squarely. The Counting sub-task degrading from 0.34 to 0.25 when switching from PickScore to GenEval rewards is unexplained and concerning — if the control block is learning a better order policy, why does one reward make counting worse? The paper also tests only one base model (MMaDA-8B), so generalization is unverified. The full-trajectory vs single-step gradient accumulation is highlighted as a key design choice but never ablated. No code or checkpoints released. The reader's take is about right. The weakest assumption — that PickScore is a sufficient reward proxy — is partially validated by the paper's own cross-metric results showing weaker transfer. This is a solid idea with insufficiently robust evidence. Worth a serious referee who can demand variance estimates and the missing ablation.","headline":"Learned generation order for multimodal diffusion models: real idea, modest gains, no variance estimates","tokens_in":13363,"tokens_out":509,"would_cite":false,"duration_ms":164255,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Learning what to draw first: RL-trained order control for multimodal diffusion","keywords":["masked diffusion models","generation order","multimodal generation","reinforcement learning","GRPO","text-to-image generation","multimodal understanding","token ordering policy"],"falsifier":"If the learned control block's improvements vanish when the reward model is swapped for a different one of comparable quality, or if the gains disappear under a different base model architecture, the contribution would reduce to 'GRPO with a good reward improves outputs' rather than 'generation order is the lever.' The strongest falsification would be showing that a randomly initialized control block trained with the same GRPO pipeline yields comparable gains — meaning the order policy is irrelevant and the improvement comes entirely from the RL fine-tuning of the base model.","tokens_in":12532,"feed_emoji":"🎨","tokens_out":1289,"duration_ms":115066,"temperature":0.7,"pith_summary":"Masked diffusion models can generate tokens in any order, but which order they choose matters. Prior work showed that for structured language tasks like Sudoku, you can pick the next token to reveal based on how confident the model is at each position — high-confidence tokens go first. This paper shows that the same trick fails for image generation and multimodal understanding: a token's predicted probability says nothing about whether generating it now will help the overall image composition or reasoning chain. Instead, the authors train a separate neural module — a control block — that learns to rank which tokens to unmask at each denoising step. They train it with Group Relative Policy Optimization, a reinforcement learning method that compares groups of sampled generation trajectories and rewards those that produce better images or more correct answers. The key technical move is accumulating policy gradients across all denoising steps of the full trajectory, rather than estimating quality from a single step as prior diffusion-RL work did. On text-to-image generation, this improves spatial relationship handling and multi-object composition. On multimodal understanding, it lets the model produce longer, more structured reasoning chains instead of collapsing to single-token answers.","feed_headline":"Learning what to draw first: RL-trained order control for multimodal diffusion","feed_subtitle":"Confidence-based token ordering works for Sudoku but fails for images — a learned policy trained with group RL fixes it, improving both text","key_machinery":"A control block built on the Unmask Policy Module (UPM) that takes hidden states from the final transformer layer plus a step index and mask indicator, and outputs ranking scores for each masked token. These scores define a Plackett-Luce policy that samples which K tokens to unmask at each denoising step without replacement. The policy is optimized via GRPO, where G trajectories are sampled per prompt, rewards are computed (PickScore for images, rule-based correctness for understanding), advantages are normalized within the group, and gradients are accumulated across all T denoising steps rather than estimated from a single transition.","core_discovery":"The central finding is that the generation order problem in multimodal masked diffusion is not solvable by logit-based heuristics — confidence at a position does not predict its importance to the global output — but is solvable by a learned policy optimized via reinforcement learning over full generation trajectories. The control block, trained with GRPO using trajectory-level reward signals, produces orderings that improve both image composition (particularly spatial relations and multi-object scenes) and multimodal reasoning depth. This separates the 'what to generate next' question from the 'what value to assign' question, and shows that the former requires learned, task-specific policies","pith_inferences":["The reward model's quality bounds the control block's learned policy: the paper shows PickScore-trained evaluation on GenEval yields 0.50 vs 0.51 for GenEval-trained, suggesting the ceiling is the reward signal's fidelity to the true objective. Better reward models would likely unlock larger gains than architectural improvements to the control block itself.","The observation that the baseline model collapses to single-token answers despite being allowed 256 tokens suggests the generation order policy implicitly controls response length and reasoning depth — making order optimization a form of learned chain-of-thought scaffolding.","If the control block is lightweight relative to the base model, the marginal inference cost of learned ordering may be small enough that it becomes a default add-on rather than a research luxury — but the paper does not report the control block's parameter count or latency overhead, leaving this open.","The 4–5% relative improvements are modest in absolute terms, but they appear concentrated in the hardest subtasks (spatial relations, multi-object composition, open-ended reasoning), which are exactly where current multimodal models struggle most — suggesting the method addresses a genuine bottleneck rather than uniformly smoothing performance."],"forward_implications":["If learned order policies generalize beyond the tested tasks, any masked diffusion model — for audio, video, or 3D generation — could benefit from a similar control block, since the core issue (positional confidence ≠ global importance) is not specific to images.","The separation of order policy from token prediction suggests a modular training paradigm: freeze the base diffusion model and train only the lightweight order controller, reducing the cost of adapting a single backbone to many downstream tasks.","The trajectory-level gradient accumulation method could apply to other multi-step generative processes (autoregressive, flow matching) where per-step credit assignment is difficult.","The failure of logit-based heuristics in multimodal settings implies that confidence calibration research focused on language may not transfer to visual token spaces, redirecting where calibration work is needed."],"fun_headline_variants":["Logits can't decide generation order in multimodal diffusion — RL can","RL-trained ordering improves multimodal diffusion over logit-based heuristics","Learned generation order beats confidence heuristics in masked diffusion models","GRPO-trained control block fixes generation ordering in multimodal diffusion","Why logit-based ordering fails for multimodal diffusion and what fixes it"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The method assumes that the reward signal used during training — PickScore for image generation and rule-based correctness for understanding — is a sufficient proxy for the true quality of a generation order policy. PickScore is a learned human-preference model whose alignment with compositional correctness is imperfect: the paper's own results show that training with PickScore but evaluating on GenEval yields weaker gains than training directly with GenEval rewards (0.50 vs ","fun_headline_variants_meta":{"raw":{"variants":["Logits can't decide generation order in multimodal diffusion — RL can","RL-trained ordering improves multimodal diffusion over logit-based heuristics","Learned generation order beats confidence heuristics in masked diffusion models","GRPO-trained control block fixes generation ordering in multimodal diffusion","Why logit-based ordering fails for multimodal diffusion and what fixes it"]},"model":"glm-5.2","effort":"low","cost_usd":0.0,"raw_usage":{"total_tokens":644,"prompt_tokens":568,"completion_tokens":76,"prompt_tokens_details":null},"tokens_in":568,"tokens_out":76,"duration_ms":32797,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-10T00:52:52.433789+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If the learned control block's improvements vanish when the reward model is swapped for a different one of comparable quality, or if the gains disappear under a different base model architecture, the contribution would reduce to 'GRPO with a good reward improves outputs' rather than 'generation order is the lever.' The strongest falsification would be showing that a randomly initialized control block trained with the same GRPO pipeline yields comparable gains — meaning the order policy is irrelevant and the improvement comes entirely from the RL fine-tuning of the base model.","supporting_citations":[],"review_version":1}