{"id":"b42c3cc5-a3d7-4943-9194-de6d6781a2bc","arxiv_id":"2507.22058","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"GRPO reinforcement learning applied to a discrete autoregressive image generator with a diffusion decoder improves instruction following, image quality, and long-text rendering in a unified multimodal model.","lead":"X-Omni generates images by predicting discrete tokens one at a time, like writing a sentence, and then turns those tokens into pixels with a diffusion model. The paper shows that reinforcement learning fixes the usual quality problems of this approach, especially for complex instructions and long in-image text.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"OCR/VLM reward-evaluation overlap makes the headline text-rendering gains unproven: the model is optimized with OCR/alignment rewards (§3.2.2) and then scored by OCR-family evaluators (§4.3.1, App. A), with no human evaluation to rule out reward overoptimization.","rationale":"The paper's central empirical claim is that GRPO-style RL, driven by a composite reward, converts a discrete AR image generator from SFT-level quality to SOTA quality. For this to be true, the composite reward must be a faithful proxy for the qualities the paper claims (aesthetics, instruction following, text rendering), and the benchmarks used to demonstrate the gain must not be optimized by the same proxy. Section 3.2.2 defines rewards from HPSv2, Unified Reward, Qwen2.5-VL-32B, and OCR models GOT-OCR2.0/PaddleOCR, but gives no aggregation weights or anti-hacking controls. The headline results in §4.3.1 are then measured by OCR-based metrics: OneIG-Bench's text accuracy and the new LongText-Bench, whose appendix specifies Qwen2.5-VL-7B as the OCR parser. This creates a pipeline where the policy is optimized to satisfy OCR/alignment models and then evaluated by an OCR model of the same family, with no human transcription or held-out OCR check. GRPO with 200 steps and 16 rollouts on 180K prompts is a high-overoptimization regime; the training curve in Figure 2(b) uses the same composite reward and therefore cannot distinguish genuine improvement from reward hacking. I considered instead making the missing SFT-vs-RL final-benchmark ablation the primary concern; it is real, but the reward-validity/evaluation-overlap issue is more fundamental because it threatens the meaning of both the training curve and the benchmark tables. The proposed human-transcription study would settle both: it compares SFT and RL under a reward-independent metric, and it directly tests whether OCR-scored gains transfer to humans. This agrees with the reader's weakest assumption; the verdict stays CONDITIONAL.","tokens_in":17331,"tokens_out":7732,"duration_ms":88038,"concrete_test":"Release or re-run the SFT and RL checkpoints, then run a controlled human transcription study on 200 images sampled from LongText-Bench and OneIG-Bench prompts (100 per checkpoint). Have human annotators transcribe the rendered text and compute word accuracy; also score the same images with Qwen2.5-VL-7B, GOT-OCR2.0, and PaddleOCR. If the RL checkpoint does not beat SFT on human word accuracy, or if the OCR-human agreement diverges substantially after RL, then the claimed 'largely enhance' is at least partly reward overoptimization rather than generalizable image-generation quality.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that RL 'largely enhances' the quality of discrete AR image generation rests heavily on benchmark results, especially long-text rendering, where X-Omni is claimed to be state of the art. The reward system in §3.2.2 trains the policy on HPSv2, Unified Reward, Qwen2.5-VL-32B text-image alignment, and OCR accuracy via GOT-OCR2.0 and PaddleOCR. The evaluation that supports the headline is OCR-based: OneIG-Bench's text rendering metric and, more importantly, the new LongText-Bench (Appendix A) use Qwen2.5-VL-7B to transcribe rendered text and compute Text Accuracy. Thus the model is optimized with OCR/alignment-family signals and then scored with an OCR-family signal. GRPO (§3.2.1) with G=16 rollouts over 200 steps on 180K prompts is exactly the regime where a fixed proxy reward can be overoptimized: the policy can exploit the fixed diffusion decoder to produce token sequences that the OCR/VLM reward models read as correct even when human readability does not improve. No reward aggregation weights, no reward-hacking analysis, no human preference evaluation, and no held-out OCR evaluator are reported. Without these, the SOTA text-rendering numbers and the 'RL makes discrete AR great again' conclusion do not yet establish generalizable quality improvement; they may establish optimization of the specific reward/OCR stack. This is a correctness risk, not a stylistic objection.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes X-Omni, a unified autoregressive model for image generation and understanding. Images are tokenized with a SigLIP-VQ semantic tokenizer, modeled with a Qwen2.5-7B autoregressive backbone, and decoded through a frozen FLUX diffusion decoder. After large-scale pretraining and supervised fine-tuning, the model is optimized with GRPO using a weighted combination of reward models: HPSv2, Unified Reward, Qwen2.5-VL-32B text-image alignment, and GOT-OCR2.0/PaddleOCR for text rendering. The authors report state-of-the-art results on text-rendering benchmarks including a new LongText-Bench, competitive DPG-Bench and GenEval numbers, comparable image-understanding performance, and a finding that the model does not require classifier-free guidance. The central claim is that reinforcement learning effectively mitigates artifacts of discrete autoregressive generation and makes such models competitive again.","tokens_in":17686,"tokens_out":6038,"duration_ms":71181,"significance":"If validated, the result is significant: it would revive discrete autoregressive image generation, which has largely been replaced by diffusion hybrids, and would provide a clean recipe—semantic tokens plus a fixed diffusion decoder plus RL—for unifying image generation and understanding in one transformer. The paper is also timely in applying GRPO, which is well developed for language models, to image generation. The work includes a large-scale training pipeline and introduces a new long-text benchmark, which is useful if properly validated. The main weakness is that the headline claims rest on reward-model-based evaluation that substantially overlaps with the training reward stack, and the paper does not provide the ablations, reward weights, or human evaluation needed to rule out reward overoptimization. The core idea is promising and the evidence is suggestive, but the central claim is not yet established at the standard the paper states.","major_comments":[{"comment":"The training rewards in §3.2.2 include OCR accuracy rewards (GOT-OCR2.0, PaddleOCR) and a Qwen2.5-VL-32B text-image alignment reward, while the headline text-rendering results are evaluated with OCR-family metrics: the LongText-Bench metric is explicitly Qwen2.5-VL-7B OCR accuracy (Appendix A), and the OneIG-Bench text-rendering score in Table 1 is composed of OCR-derived Edit Distance, Completion Rate, and Word Accuracy. The paper reports no human evaluation, no held-out OCR evaluator, no reward aggregation weights, and no reward-hacking analysis. With G=16 rollouts over 200 steps on 180K prompts, GRPO is exactly the regime in which a fixed proxy reward can be overoptimized, so the policy may produce token sequences that the OCR/VLM reward models read as correct even when human readability does not improve correspondingly. Please provide a human study on LongText-Bench, an independent OCR engine not used in training, and an analysis of reward versus true benchmark performance; without these, the SOTA long-text claim is not yet supported.","section":"§3.2.2 and §4.3.1 / Appendix A"},{"comment":"The central attribution of gains to RL is supported mainly by the training curve in Figure 2(b), which claims that the RL reward surpasses the SFT best-of-N baseline. However, no final-checkpoint benchmark numbers are reported for the SFT or SFT+BoN baselines on any of the evaluation tables (Tables 1–4). Figure 2(b) also lacks axis labels and quantitative endpoints, making it impossible to assess the magnitude of the improvement. Please add a direct comparison of the final RL model, the SFT model, and the SFT+BoN baseline under identical evaluation protocols on DPG-Bench, GenEval, and the text-rendering benchmarks; this is necessary to substantiate the statement that RL 'outperforms SFT with best-of-N sampling'.","section":"§4.4 and Figure 2(b)"},{"comment":"All benchmark tables report single runs without confidence intervals or significance tests. This is particularly concerning for the new LongText-Bench, which has only 160 prompts with 4 generated images each (Appendix A), and for several comparisons in Tables 2 and 3 where the differences are small (e.g., Table 2: X-Omni 87.65 vs Show-o2 86.14; Table 3: X-Omni 0.83 vs Mogao 0.89). Given the stochasticity of sampling and the small size of the new benchmark, the reported margins may not be meaningful. Please provide error bars, significance tests, or at least per-prompt variance for the key claims, especially for the new benchmark.","section":"Tables 1–4 and Appendix A"},{"comment":"The reward system is a central component of the method, but the paper does not specify the aggregation weights for HPSv2, Unified Reward, text-image alignment, and OCR rewards, nor the GRPO clipping parameter epsilon in Eq. (1). Section 4.2 reports learning rate, batch size, number of rollouts, and KL weight, but the missing reward weights and clipping value prevent reproduction and make it impossible to analyze reward overoptimization. Please report these values and, ideally, an ablation showing the contribution of each reward component; the current description is too underspecified for a method whose entire claim is that carefully designed rewards drive the improvement.","section":"§3.2.2 and Eq. (1)"},{"comment":"The paper states that 'The Chinese text rendering model is derived by incorporating training on Chinese data at an intermediate checkpoint during the reinforcement learning stage.' This means the Chinese and English X-Omni results in Table 1 may come from different checkpoints or training recipes. Please clarify whether Table 1 reports a single unified model or separate English and Chinese variants, what the intermediate checkpoint is, how much additional Chinese data is used, and how this affects the comparison with other models. Without this clarification, the claim of a single unified model and the SOTA Chinese text-rendering result are ambiguous.","section":"§4.2"}],"minor_comments":[{"comment":"The abstract claims 'state-of-the-art performance in image generation tasks using a 7B language model,' but on GenEval X-Omni (0.83) is below Mogao (0.89) and BAGEL (0.88). Please qualify the SOTA claim to refer to the specific benchmarks or unified-model category where it holds.","section":"Table 3"},{"comment":"The finding that X-Omni does not rely on classifier-free guidance is supported only by qualitative examples in Figure 6. Please add a quantitative comparison of CFG vs no-CFG generation on at least one benchmark, since this is presented as a key result.","section":"§4.4 and Figure 6"},{"comment":"The training curve in Figure 2(b) would benefit from labeled axes, units, and explicit endpoints for the SFT BoN baseline and the final RL reward; currently the reader cannot assess the magnitude of the claimed improvement.","section":"Figure 2(b)"},{"comment":"The LongText-Bench evaluation metric 'Text Accuracy' is not formally defined. Please specify whether it is character-level, word-level, normalized, and how partial matches are scored, so that the numbers in Table 1 are reproducible.","section":"Appendix A"},{"comment":"The training status of the diffusion decoder is ambiguous: the text says a linear layer is added to FLUX.1-dev and the decoder is 'trained with the objective of image reconstruction,' while later 'Both components are maintained frozen' appears to refer only to the visual encoder and vector quantizer. Please clarify whether FLUX.1-dev is fine-tuned or kept frozen during pretraining, SFT, and RL, and whether the linear adapter is trained in all stages.","section":"§3.1"},{"comment":"There are minor typesetting issues in Eq. (1), including the rendering of the summation and the subscript of the reference policy. Please fix these for readability.","section":"Eq. (1)"}],"recommendation":"major_revision","confidential_remarks":"The main risk is that the evaluation stack is too close to the training reward stack, and the missing SFT+BoN final-benchmark ablation leaves the central RL attribution unsupported. If the authors can add a human study, an independent OCR evaluator, and a head-to-head SFT/RL benchmark comparison, the paper could be a strong contribution. The paper is also light on reward-design transparency; the missing aggregation weights are a reproducibility blocker. I would not recommend rejection because the architecture and idea are promising and the concerns are addressable with additional experiments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know about X-Omni is that it does something genuinely new: it runs GRPO on a discrete-autoregressive image generator that decodes through a fixed diffusion decoder, and the qualitative results look like they work. That combination is not in LaViT, Janus-Pro, Emu3, or Show-o2, and it is a plausible reason the field might look again at unified next-token models. The paper also gives a clear mechanism for why RL helps: it aligns the AR model's token distribution with what the frozen diffusion decoder expects, mitigating the cumulative-error and discretization problems that have historically sunk discrete AR image generation.\n\nThe ingredients are all published pieces—SigLIP-VQ, FLUX decoder, GRPO, off-the-shelf reward models—and the paper is honest about that. The new part is the combination and the claim that RL is the key driver. The qualitative improvements during training (Figure 2) are striking, and the CFG-free result is a nice bonus that supports the alignment story.\n\nBut the evidence for the central claim is thinner than the abstract suggests. The only direct support that RL beats SFT is the training curve in Figure 2(b); there is no final-benchmark comparison of the SFT+best-of-N baseline against the RL model. The benchmark tables have no error bars or significance tests, and the reward weights for the four reward components are not given.\n\nThe more serious issue is the OCR/VLM feedback loop. The policy is trained with OCR rewards (GOT-OCR2.0, PaddleOCR) and alignment rewards (Qwen2.5-VL-32B), then evaluated on text rendering using Qwen2.5-VL-7B transcription and on the new LongText-Bench whose evaluator is the same family. That is exactly the setup where reward overoptimization can manufacture benchmark gains that do not transfer to human readability. Without a human preference study or a held-out OCR evaluator, the state-of-the-art text-rendering numbers should be read as provisional. LongText-Bench itself is a reasonable addition, but with 160 prompts and a single evaluation model it is not a strong basis for the strongest claims.\n\nNone of this kills the paper. The core finding—that RL can align AR token generation with a fixed diffusion decoder and remove the need for CFG—is interesting and deserves scrutiny. What it needs is a revision that releases code/weights or at least a demo, reports the SFT+BoN final numbers, discloses reward weights and checks for reward hacking, and adds an independent human evaluation of long-text rendering.\n\nThis paper is for researchers working on unified multimodal generation, text-to-image, and RL for generative models. I would send it to peer review, but with the expectation of major revision. The idea is important enough and the empirical package substantial enough to merit referee time.","headline":"A genuinely new combination—GRPO on discrete AR image tokens with a fixed diffusion decoder—that looks like it works, but the headline text-rendering claims are weakened by reward-evaluation overlap and missing ablations.","tokens_in":18264,"tokens_out":2033,"would_cite":true,"duration_ms":23102,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that reinforcement learning, applied to a discrete autoregressive image model built from a semantic tokenizer and a frozen diffusion decoder, removes the artifacts that have pushed the field toward diffusion hybrids, and…","keywords":["reinforcement learning","discrete autoregressive image generation","unified multimodal model","semantic image tokenizer","GRPO","text-to-image generation","long text rendering","classifier-free guidance"],"falsifier":"Run a blind human-preference study comparing X-Omni's RL outputs against its own SFT checkpoint with best-of-N sampling on prompts drawn from benchmarks outside the RL training and reward distribution, such as T2I-CompBench or DreamBench; if humans do not prefer the RL outputs, the central claim fails. As a mechanistic check, inject spelling errors into ground-truth text and verify whether the OCR-based reward actually tracks human readability, or whether it saturates once the OCR model can guess the text.","tokens_in":17114,"feed_emoji":"🎨","tokens_out":7574,"duration_ms":78834,"temperature":0.7,"pith_summary":"This paper claims that reinforcement learning removes the long-standing weaknesses of discrete autoregressive image generation—low fidelity, distorted details, and failure to follow complex instructions—so that a single next-token-prediction model can handle both text and images and beat diffusion-centered unified systems. To show this, the authors build X-Omni, in which a semantic tokenizer (SigLIP-VQ) turns images into discrete tokens, a 7B autoregressive language model predicts those tokens, and a frozen diffusion decoder renders them into pixels. The autoregressive model is then trained with GRPO against a composite reward that scores aesthetics, text-image alignment, and OCR-verified text rendering. If the claim holds, unified open-source image models no longer need diffusion training objectives, and a 7B model can render long in-image text in English and Chinese competitively with specialized commercial systems.","feed_headline":"Rewards make discrete autoregressive image models great again","feed_subtitle":"A 7B unified model trained with GRPO beats diffusion hybrids on text rendering and instruction following.","key_machinery":"The load-bearing mechanism is the closed loop: an autoregressive policy samples discrete semantic tokens, a frozen diffusion decoder converts them into pixels, external reward models score the pixels, and GRPO converts group-normalized reward differences into policy-gradient updates. The tokenizer is SigLIP-VQ, which discretizes a frozen SigLIP2-g semantic encoder's outputs through a 16,384-entry vector-quantization codebook into the same kind of token stream as language, and the decoder is a frozen FLUX.1-dev model fed through a linear adapter that injects the semantic token embeddings into its intermediate features. GRPO, or Group Relative Policy Optimization, is the optimizer that avoids a separate critic by normalizing rewards within a group of 16 rollouts per prompt, with a KL penalty toward the reference policy. The composite reward—HPSv2, Unified Reward, Qwen2.5-VL-32B alignment, and GOT-OCR2.0/PaddleOCR accuracy—is what the paper argues carries the aesthetic, alignment, and text-rendering quality gains.","core_discovery":"The central discovery is that the quality gap between discrete autoregressive image generation and diffusion-based generation is not inherent to discrete tokens; it is a trainable distribution mismatch. The autoregressive model samples semantic tokens in a region of token space that the pixel decoder was not trained to accept, and reinforcement learning pulls the sampler into the decoder's expected distribution while suppressing errors that accumulate over long token sequences. Concretely, the paper reports that after three pre-training stages, supervised fine-tuning, and 200 steps of GRPO, a 7B model reaches state-of-the-art overall on DPG-Bench (87.65), the best OneIG-Bench English text-rendering score among unified and open models (0.901), a Chinese score of 0.895, and the best LongText-Bench Chinese long-text score (0.814), while matching or exceeding unified baselines on image understanding including OCRBench (704). A second reported finding is that the RL-trained model generates high-quality images without classifier-free guidance, unlike earlier autoregressive image models, and that RL exceeds the best-of-N sampling of its own SFT checkpoint, a contrast with typical language-modeling experience. The paper also introduces LongText-Bench, 160 prompts across eight text-rich scenarios, as an evaluation for long-text rendering.","pith_inferences":["If the operative mechanism is distribution alignment rather than the specific model choices, the same GRPO-plus-frozen-decoder recipe should transfer to other discrete autoregressive image systems; the clean test is applying it to an Emu3- or Janus-Pro-style model without architectural changes.","The reported result that RL beats SFT best-of-N—atypical for language—is plausibly because one image yields many reward-bearing tokens, giving group-normalized advantages dense spatial credit; a testable corollary is that the advantage shrinks as tokens per image are compressed.","The missing reward weights and absent reward-hacking analysis mark the true risk surface: on dimensions the reward models do not see, such as counting and spatial relations, the method could regress while scores rise, and the relatively weaker GenEval result (0.83) is consistent with such blind spots.","The CFG-free finding suggests the reward signal absorbed the role of sampling-time guidance; whether that robustness persists on out-of-distribution or adversarial prompts is an open, testable question."],"forward_implications":["Unified discrete next-token prediction becomes a viable base for frontier image generation, removing the need for diffusion objectives inside the main model.","Autoregressive image sampling can run without classifier-free guidance, lowering inference cost and indicating that visual and language token generation share one consistent distribution.","A 7B open model can render long in-image text in English and Chinese, beating specialized commercial systems on Chinese long-text rendering.","Because generation and understanding use the same token stream, multi-turn joint understanding and generation no longer requires re-encoding generated images with a separate encoder.","The recipe of GRPO plus a frozen diffusion decoder plus external reward models is the component that closes the token-distribution gap, and it outperforms best-of-N sampling from the SFT model."],"supporting_citations":[{"why":"Supplies the GRPO policy-gradient algorithm whose group-normalized advantages drive the RL updates.","marker":"[61]"},{"why":"Provides the frozen SigLIP2-g semantic encoder that the SigLIP-VQ tokenizer discretizes.","marker":"[62]"},{"why":"Is the frozen FLUX.1-dev diffusion decoder that converts predicted semantic tokens into pixels.","marker":"[64]"},{"why":"HPSv2 gives the human-preference and aesthetic reward signal.","marker":"[66]"},{"why":"Unified Reward supplies the high-resolution human-alignment reward complementing HPSv2.","marker":"[67]"},{"why":"Qwen2.5-VL computes the text-image alignment reward and is also used for benchmark OCR evaluation.","marker":"[68]"},{"why":"GOT-OCR2.0 is one of the two OCR models that score text-rendering accuracy during RL.","marker":"[69]"},{"why":"PaddleOCR is the second OCR model used to verify rendered text accuracy.","marker":"[70]"},{"why":"Emu3 is the discrete autoregressive baseline this work aims to surpass and whose CFG dependence is contrasted.","marker":"[12]"},{"why":"Janus-Pro is the unified autoregressive model compared on text rendering, DPG-Bench, and CFG dependence.","marker":"[21]"}],"fun_headline_variants":["RL beats best-of-N and needs no CFG for autoregressive image generation","SOTA discrete AR image generation with RL training at 7B scale","Reinforcement learning aligns AR token sampling with the pixel decoder","RL training closes the gap for discrete autoregressive image models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire quality gain is driven by external scoring models that judge aesthetics, text-image match, and OCR-readable text, with no published aggregation weights and no analysis of reward hacking, so if those scorers are miscalibrated the claimed improvement may not reflect real human preference or transfer to benchmarks outside their coverage.","fun_headline_variants_meta":{"raw":{"variants":["RL beats best-of-N and needs no CFG for autoregressive image generation","SOTA discrete AR image generation with RL training at 7B scale","Reinforcement learning aligns AR token sampling with the pixel decoder","RL training closes the gap for discrete autoregressive image models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000574,"raw_usage":{"total_tokens":2745,"prompt_tokens":1015,"completion_tokens":1730,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":631,"completion_tokens_details":{"reasoning_tokens":1655}},"tokens_in":631,"tokens_out":1730,"duration_ms":15426,"temperature":1.0,"reasoning_tokens":1655,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T12:03:35.338677+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a blind human-preference study comparing X-Omni's RL outputs against its own SFT checkpoint with best-of-N sampling on prompts drawn from benchmarks outside the RL training and reward distribution, such as T2I-CompBench or DreamBench; if humans do not prefer the RL outputs, the central claim fails. As a mechanistic check, inject spelling errors into ground-truth text and verify whether the OCR-based reward actually tracks human readability, or whether it saturates once the OCR model can guess the text.","supporting_citations":[{"cited_title":"Paddleocr, awesome multilingual ocr toolkits based on paddlepaddle","cited_arxiv_id":null,"evidence_quote":"PaddleOCR is the second OCR model used to verify rendered text accuracy."}],"review_version":1}