pith. sign in

arxiv: 2606.01911 · v1 · pith:TLHUHC7Tnew · submitted 2026-06-01 · 💻 cs.CV

Residual Decoder Adapter: ID-Preserving Tokenizer Adaption for Autoregressive Text Rendering

Dongxing Mao , Jinpeng Wang , Jiahao Tang , Kevin Qinghong Lin , Linjie Li , Zhengyuan Yang , Lijuan Wang , Min Li
show 1 more author
Jingru Tan
This is my paper

Pith reviewed 2026-06-28 15:36 UTC · model grok-4.3

classification 💻 cs.CV
keywords autoregressive image generationvisual tokenizertext renderingresidual decoder adapterpost-hoc adaptationOCR accuracyimage synthesis
0
0 comments X

The pith

A residual decoder adapter upgrades existing visual tokenizers to sharpen text rendering in autoregressive image models without retraining or altering tokens.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper seeks to establish that blurry text in visual autoregressive models arises from the tokenizer's inability to reconstruct fine details, and that this can be fixed after the fact. It introduces the Residual Decoder Adapter, which adds a paired codebook matching the original token distribution plus a parallel branch that learns pixel-space residuals between reconstructed and ground-truth images. This setup refines decoder output while leaving the token space and the downstream autoregressive model untouched. A sympathetic reader would care because the change is non-invasive and promises large gains in text quality at modest extra cost. Reported results include OCR accuracy on TextVisionBlend rising from 24.52 percent to 58.26 percent for a finetuned Janus-Pro model.

Core claim

The Residual Decoder Adapter upgrades an existing visual tokenizer post-hoc without changing its token space by introducing a paired codebook that shares the original token distribution and a parallel branch that learns the tiny pixel-space differences between the reconstructed image and ground-truth images. This residual design enhances fine-grained reconstruction for text while preserving full compatibility with prior autoregressive models.

What carries the argument

The Residual Decoder Adapter, built from a paired codebook sharing the original token distribution and a parallel residual branch that corrects decoder output in pixel space.

If this is right

  • Finetuned Janus-Pro OCR accuracy on TextVisionBlend rises from 24.52 percent to 58.26 percent.
  • OCR accuracy on StyledTextSynth rises from 12.75 percent to 36.81 percent on the TextAtlas benchmark.
  • The original token space remains unchanged so prior AR models continue to work without modification.
  • Text rendering improves through better reconstruction of fine strokes and letter shapes.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same residual correction could be tested on non-text fine details such as small object textures or facial features.
  • Adoption of such adapters would reduce the need to retrain full AR pipelines when adapting to new visual domains.
  • The approach might generalize to other autoregressive architectures beyond the ones evaluated here.

Load-bearing premise

A paired codebook sharing the original token distribution plus a parallel residual branch can be trained to correct fine-grained reconstruction errors while leaving the token space and downstream AR model completely unchanged.

What would settle it

Train the RDA on a visual tokenizer, attach it to a finetuned Janus-Pro model, and measure OCR accuracy on the TextVisionBlend subset of TextAtlas; if accuracy stays below 40 percent the improvement claim is falsified.

Figures

Figures reproduced from arXiv: 2606.01911 by Dongxing Mao, Jiahao Tang, Jingru Tan, Jinpeng Wang, Kevin Qinghong Lin, Lijuan Wang, Linjie Li, Min Li, Zhengyuan Yang.

Figure 1
Figure 1. Figure 1: (a) Comparison between AR model (Janus-Pro) and Diffusion model (FLUX 1.0-dev). They exhibit similar perfor￾mance on the general text-to-image benchmark GenEval, with ac￾curacy of 0.80 and 0.82, respectively. (b) Comparison between VQ-VAE (from Janus-Pro) and VAE (from FLUX) at 512 resolu￾tion. Their reconstruction metrics on ImageNet with rFID scores of 9.63 and 7.92, respectively. lenge that has recently… view at source ↗
Figure 2
Figure 2. Figure 2: The intuition behind our method. (a) The modern text-to-image generation ecosystem consists of an AR model and a visual tokenizer. (b) Existing methods develop an improved to￾kenizer so inevitably re-train AR models. (c) Our approach en￾hances the tokenizer with an adopter while preserving the compat￾ibility with original AR model(AR-Training Free). model to obtain a stronger tokenizer [29, 35, 62]. Howeve… view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the proposed Residual Decoder Adapter (RDA). RDA enhances an existing image tokenizer without modifying its token space or retraining. RDA predict an ∆Iˆ, which is added into Iˆto obtain the refined output Iˆ+∆Iˆwith more accurate text details. for better perceptual alignment, and [29, 57] designs a modality-aware tokenizer that unifies visual and textual spaces. [17] further enhances image tok… view at source ↗
Figure 4
Figure 4. Figure 4: Inference pipeline of the AR model with equipped RDA. RDA uses the same IDs to refine the generated image. Standard losses. We respectively apply MAE and MSE re￾construction losses to supervise the residual output and the final reconstruction, and denote their combination as Lrec. Following VQ-VAE [47], we also apply perceptual loss on the final reconstructed image by using ϕ(·) to extract multi-layer VGG … view at source ↗
Figure 5
Figure 5. Figure 5: Reconstruction performance of image tokenizer equipped with RDA. We use LlamaGen-VQ as the VQVAE tokenizer. During training, we freeze the base tokenizer and opti￾mize only the RDA module. 4.4. Plug-and-Play Inference in AR model During inference time, an AR model predicts a sequence of discrete indices {in|in ∈ {1, 2, . . . , K}}N n=1. We can use those indices to reconstruct the initial coarse image ˆI wi… view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative generation results of AR models. General AR uses TAR-7B at 512 resolution, while text-specific AR uses Janus￾Pro∗ 1B 1024 (left) and Lumina-mGPT∗ 7B 1024 (right). Please zoom in for better visualization of text details. More visual results are provided in the Appendix [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Comparison of RDA with direct decoder fine-tuning. RDA preserves fine-grained details and sharper textures, while di￾rect decoder fine-tuning leads to over-smoothing. 5.6. Qualitative Results We present qualitative comparisons to illustrate the effec￾tiveness of RDA in both generation and reconstruction set￾tings [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Visualization of Sobel mask and frequency mask [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: Failure Case of Tokenizer. 1. Token prediction: Weak text token prediction from the AR model. 2. Reconstruction: Limited reconstruction fidelity from the tokenizer. Text-specific fine-tuning addresses the first bottleneck, making the tokenizer decoder the dominant limitation. RDA directly targets this by enhancing reconstruction with￾out modifying the AR model, enabling large improvements on text-tuned mo… view at source ↗
Figure 12
Figure 12. Figure 12: presents training curves for RDA. The optimization is stable throughout training. Critically, without residual perceptual loss L res perc, the residual branch fails to converge and produces only blurry gray regions. This underscores the importance of perceptual supervision for learning meaning￾ful high-frequency details. 12. Design Justifications 12.1. Why Shared-ID Preserves Compatibility The Shared-ID m… view at source ↗
Figure 13
Figure 13. Figure 13: Where RDA is applied in TAR. TAR modifies the tokenization and embedding pipeline before decoding, while RDA attaches after the decoder and refines pixel-level outputs without altering token IDs. 15. Additional Information 15.1. Recaption Prompt We use the following prompt to generate recaptions via Qwen-2.5-VL: Recaption Prompt Carefully describe the image by precisely combin￾ing visual elements with all… view at source ↗
Figure 14
Figure 14. Figure 14: Qualitative Results of LlamaGenVQ Applying RDA. Left: low-resolution setting. Right: high-resolution setting. 5 [PITH_FULL_IMAGE:figures/full_fig_p016_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Qualitative Results of ChameleonVQ Applying RDA. Left: low-resolution setting. Right: high-resolution setting. 6 [PITH_FULL_IMAGE:figures/full_fig_p017_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Qualitative Results of Janus Pro Applying RDA 7 [PITH_FULL_IMAGE:figures/full_fig_p018_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Qualitative Results of Tar 1B Applying RDA 8 [PITH_FULL_IMAGE:figures/full_fig_p019_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Qualitative Results of Tar 7B Applying RDA 9 [PITH_FULL_IMAGE:figures/full_fig_p020_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Qualitative Results of Finetuned Janus Pro Applying RDA 10 [PITH_FULL_IMAGE:figures/full_fig_p021_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Qualitative Results of Finetuned Luminamgpt 512 Applying RDA 11 [PITH_FULL_IMAGE:figures/full_fig_p022_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: Qualitative Results of Finetuned Luminamgpt 1024 Applying RDA 12 [PITH_FULL_IMAGE:figures/full_fig_p023_21.png] view at source ↗
read the original abstract

Visual Autoregressive (AR) models generate images by predicting discrete tokens that are decoded by a visual tokenizer. Despite demonstrating strong overall image generation ability, they still underperform on text rendering with blur strokes and disrupt letter shapes. In this work, we trace this limitation to the visual tokenizer, which struggles to reconstruct fine-grained detail. Improving the tokenizer is straightforward but expensive, as it necessitates retraining both the tokenizer and the AR model. Can we improve text rendering performance of AR models without retraining the existing tokenizer and AR model? To achieve this, we propose the Residual Decoder Adapter(RDA) that upgrades an existing tokenizer post-hoc without changing its token space. Specifically, it refines the decoder output of the visual tokenizer by introducing two novel components: (i) a paired codebook that shares the token distribution with the original one; (ii) a parallel branch to learn the tiny differences (residual) between the reconstructed image and the ground-truth images in the pixel space. This residual design allows us to enhance the tokenizer non-invasively while preserving compatibility with prior AR models. RDA substantially improves text rendering significantly by a large margin. For instance, we boost finetuned Janus-Pro OCR accuracy rises from 24.52% to 58.26% (TextVisionBlend), from 12.75% to 36.81% (StyledTextSynth) on competitive TextAtlas benchmark. The code is available at https://github.com/CSU-JPG/RDA

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The manuscript proposes the Residual Decoder Adapter (RDA) as a post-hoc module to improve text rendering quality in visual autoregressive (AR) models. RDA attaches to an existing visual tokenizer via a paired codebook (sharing the original token distribution) and a parallel residual branch that learns pixel-space differences between reconstructed and ground-truth images. The design is presented as non-invasive, preserving the original discrete token space and AR model compatibility while delivering large empirical gains, such as raising OCR accuracy on the TextAtlas benchmark from 24.52% to 58.26% (TextVisionBlend) and 12.75% to 36.81% (StyledTextSynth) for finetuned Janus-Pro.

Significance. If the compatibility premise holds and the reported gains are reproducible under the stated protocol, RDA would provide a low-cost route to upgrading deployed AR tokenizers for fine-grained tasks without retraining either the tokenizer or the downstream AR model. The public code release is a clear strength for verification.

major comments (3)
  1. [§3] §3 (RDA architecture): The paired codebook is described as sharing the token distribution, yet the text does not specify whether the encoder still produces exactly the same discrete indices at inference or whether the residual branch can alter index selection; this mechanism is load-bearing for the central claim of unchanged token space and AR compatibility.
  2. [§4] §4 (experiments): The OCR accuracy jumps are presented without training details for the adapter (loss terms, optimizer, epochs), number of runs, standard deviations, or controls that isolate the contribution of the paired codebook versus the residual branch; these omissions prevent verification that the gains support the non-invasive upgrade claim.
  3. [§3.1] §3.1 (paired codebook definition): No equation or pseudocode shows how the paired codebook is queried or combined with the original decoder output while guaranteeing identical token indices; without this, the assertion that the AR model requires “no modification” cannot be evaluated.
minor comments (2)
  1. [Abstract] Abstract: the sentence “we boost finetuned Janus-Pro OCR accuracy rises from” is grammatically incorrect and should be rephrased for clarity.
  2. Figure captions and tables lack explicit statements of whether reported OCR numbers use the exact evaluation protocol described in the text or any post-processing.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive feedback on our manuscript. The comments highlight important areas for clarification on the RDA mechanism and experimental reporting. We address each point below and will revise the manuscript accordingly to strengthen the presentation of the non-invasive upgrade claim.

read point-by-point responses

  1. Referee: [§3] §3 (RDA architecture): The paired codebook is described as sharing the token distribution, yet the text does not specify whether the encoder still produces exactly the same discrete indices at inference or whether the residual branch can alter index selection; this mechanism is load-bearing for the central claim of unchanged token space and AR compatibility.

    Authors: The original encoder remains frozen and unchanged; it produces the discrete indices exactly as before, and these indices are passed directly to both the original decoder and the adapter. The residual branch operates strictly in pixel space on the decoded output and has no access to or influence over the discrete token selection. The paired codebook shares the same distribution but is queried only within the adapter for refinement. We will add explicit text and a flowchart in the revised §3 to make this separation unambiguous. revision: yes

  2. Referee: [§4] §4 (experiments): The OCR accuracy jumps are presented without training details for the adapter (loss terms, optimizer, epochs), number of runs, standard deviations, or controls that isolate the contribution of the paired codebook versus the residual branch; these omissions prevent verification that the gains support the non-invasive upgrade claim.

    Authors: We agree that additional experimental details are necessary for reproducibility. The revised manuscript will include the complete training protocol for the adapter (loss formulation, optimizer, learning rate schedule, and epochs), results averaged over multiple random seeds with standard deviations, and ablation experiments that separately disable the paired codebook and the residual branch to quantify their individual contributions. revision: yes

  3. Referee: [§3.1] §3.1 (paired codebook definition): No equation or pseudocode shows how the paired codebook is queried or combined with the original decoder output while guaranteeing identical token indices; without this, the assertion that the AR model requires “no modification” cannot be evaluated.

    Authors: We will insert a precise mathematical formulation and pseudocode in §3.1 that defines the index generation step (unchanged from the original tokenizer) and shows how the paired codebook output is added only at the pixel level after decoding. This will explicitly demonstrate that the AR model continues to operate on the original discrete token sequence without any architectural or input changes. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical adapter proposal validated on external benchmarks

full rationale

The paper proposes an architectural adapter (paired codebook + residual branch) and reports measured OCR accuracy gains on TextAtlas benchmarks. No equations, parameter-fitting derivations, or load-bearing self-citations appear in the provided text. The compatibility claim (unchanged token space) is an engineering premise tested by downstream AR model reuse rather than a quantity derived from the adapter itself. Improvements are presented as experimental outcomes, not quantities forced by construction from fitted inputs or prior author results. This is a standard self-contained empirical contribution.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 1 invented entities

The abstract introduces two new components (paired codebook and residual branch) without stating any fitted numerical parameters, mathematical axioms, or new physical entities; the central claim therefore rests on the empirical effectiveness of these architectural additions.

invented entities (1)
  • Residual Decoder Adapter (RDA) no independent evidence
    purpose: Post-hoc refinement of visual tokenizer decoder output to improve text rendering while preserving original token space
    New module introduced in the abstract; no independent evidence provided beyond the reported benchmark gains

pith-pipeline@v0.9.1-grok · 5829 in / 1303 out tokens · 26602 ms · 2026-06-28T15:36:51.953722+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

83 extracted references · 5 canonical work pages

  1. [1]

    Efficient-vqgan: To- wards high-resolution image generation with efficient vision transformers

    Shiyue Cao, Yueqin Yin, Lianghua Huang, Yu Liu, Xin Zhao, Deli Zhao, and Kaigi Huang. Efficient-vqgan: To- wards high-resolution image generation with efficient vision transformers. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 7368–7377, 2023. 2

    2023

  2. [2]

    Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T. Freeman. Maskgit: Masked generative image transformer, 2022. 2

    2022

  3. [3]

    Softvq-vae: Efficient 1-dimensional contin- uous tokenizer

    Hao Chen, Ze Wang, Xiang Li, Ximeng Sun, Fangyi Chen, Jiang Liu, Jindong Wang, Bhiksha Raj, Zicheng Liu, and Emad Barsoum. Softvq-vae: Efficient 1-dimensional contin- uous tokenizer. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 28358–28370, 2025. 2

    2025

  4. [4]

    Textdiffuser: Diffusion models as text painters, 2023

    Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei. Textdiffuser: Diffusion models as text painters, 2023. 2, 5

    2023

  5. [5]

    Blip3- o: A family of fully open unified multimodal models- architecture, training and dataset, 2025

    Jiuhai Chen, Zhiyang Xu, Xichen Pan, Yushi Hu, Can Qin, Tom Goldstein, Lifu Huang, Tianyi Zhou, Saining Xie, Sil- vio Savarese, Le Xue, Caiming Xiong, and Ran Xu. Blip3- o: A family of fully open unified multimodal models- architecture, training and dataset, 2025. 2

    2025

  6. [6]

    Janus- pro: Unified multimodal understanding and generation with data and model scaling, 2025

    Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus- pro: Unified multimodal understanding and generation with data and model scaling, 2025. 1, 2

    2025

  7. [7]

    Paddleocr-vl: Boosting multilingual document parsing via a 0.9b ultra-compact vision-language model, 2025

    Cheng Cui, Ting Sun, Suyin Liang, Tingquan Gao, Zelun Zhang, Jiaxuan Liu, Xueqing Wang, Changda Zhou, Hongen Liu, Manhui Lin, Yue Zhang, Yubo Zhang, Handong Zheng, Jing Zhang, Jun Zhang, Yi Liu, Dianhai Yu, and Yanjun Ma. Paddleocr-vl: Boosting multilingual document parsing via a 0.9b ultra-compact vision-language model, 2025. 5

    2025

  8. [8]

    Paddleocr 3.0 technical report, 2025

    Cheng Cui, Ting Sun, Manhui Lin, Tingquan Gao, Yubo Zhang, Jiaxuan Liu, Xueqing Wang, Zelun Zhang, Changda Zhou, Hongen Liu, Yue Zhang, Wenyu Lv, Kui Huang, Yichao Zhang, Jing Zhang, Jun Zhang, Yi Liu, Dianhai Yu, and Yanjun Ma. Paddleocr 3.0 technical report, 2025. 5

    2025

  9. [9]

    Emerging properties in unified multimodal pretraining, 2025

    Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, Guang Shi, and Haoqi Fan. Emerging properties in unified multimodal pretraining, 2025. 1, 2

    2025

  10. [10]

    Cogview: Mastering text-to- image generation via transformers, 2021

    Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, and Jie Tang. Cogview: Mastering text-to- image generation via transformers, 2021. 2

    2021

  11. [11]

    Textcrafter: Accurately rendering multiple texts in complex visual scenes, 2025

    Nikai Du, Zhennan Chen, Shan Gao, Zhizhou Chen, Xi Chen, Zhengkai Jiang, Jian Yang, and Ying Tai. Textcrafter: Accurately rendering multiple texts in complex visual scenes, 2025. 2, 5

    2025

  12. [12]

    Taming transformers for high-resolution image synthesis, 2021

    Patrick Esser, Robin Rombach, and Bj ¨orn Ommer. Taming transformers for high-resolution image synthesis, 2021. 1, 2

    2021

  13. [13]

    D-ar: Diffusion via au- toregressive models, 2025

    Ziteng Gao and Mike Zheng Shou. D-ar: Diffusion via au- toregressive models, 2025. 2

    2025

  14. [14]

    X-omni: Reinforcement learning makes discrete autoregressive image generative models great again, 2025

    Zigang Geng, Yibing Wang, Yeyao Ma, Chen Li, Yongming Rao, Shuyang Gu, Zhao Zhong, Qinglin Lu, Han Hu, Xi- aosong Zhang, Linus, Di Wang, and Jie Jiang. X-omni: Reinforcement learning makes discrete autoregressive image generative models great again, 2025. 2, 5

    2025

  15. [15]

    arXiv preprint arXiv:2506.18898 (2025) 11, 12

    Jiaming Han, Hao Chen, Yang Zhao, Hanyu Wang, Qi Zhao, Ziyan Yang, Hao He, Xiangyu Yue, and Lu Jiang. Vi- sion as a dialect: Unifying visual understanding and gen- eration via text-aligned representations.arXiv preprint arXiv:2506.18898, 2025. 2, 5

    work page arXiv 2025

  16. [16]

    Flowtok: Flowing seamlessly across text and image tokens

    Ju He, Qihang Yu, Qihao Liu, and Liang-Chieh Chen. Flow- tok: Flowing seamlessly across text and image tokens.arXiv preprint arXiv:2503.10772, 2025. 2

    work page arXiv 2025

  17. [17]

    Rear: Rethinking visual autoregressive models via generator-tokenizer consistency regularization, 2025

    Qiyuan He, Yicong Li, Haotian Ye, Jinghao Wang, Xinyao Liao, Pheng-Ann Heng, Stefano Ermon, James Zou, and An- gela Yao. Rear: Rethinking visual autoregressive models via generator-tokenizer consistency regularization, 2025. 3

    2025

  18. [18]

    Plangen: Towards unified layout planning and image generation in auto-regressive vision language models,

    Runze He, Bo Cheng, Yuhang Ma, Qingxiang Jia, Shanyuan Liu, Ao Ma, Xiaoyu Wu, Liebucha Wu, Dawei Leng, and Yuhui Yin. Plangen: Towards unified layout planning and image generation in auto-regressive vision language models,

  19. [19]

    Denoising diffu- sion probabilistic models, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models, 2020. 2

    2020

  20. [20]

    Flux.https://github.com/ black-forest-labs/flux, 2024

    Black Forest Labs. Flux.https://github.com/ black-forest-labs/flux, 2024. 1

    2024

  21. [21]

    Unleashing in-context learning of autoregressive models for few-shot image manipulation

    Bolin Lai, Felix Juefei-Xu, Miao Liu, Xiaoliang Dai, Nikhil Mehta, Chenguang Zhu, Zeyi Huang, James M Rehg, Sang- min Lee, Ning Zhang, et al. Unleashing in-context learning of autoregressive models for few-shot image manipulation. InProceedings of the Computer Vision and Pattern Recogni- tion Conference, pages 18346–18357, 2025. 2

    2025

  22. [22]

    Photo-realistic single image super-resolution using a generative adversarial network, 2017

    Christian Ledig, Lucas Theis, Ferenc Huszar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, and Wenzhe Shi. Photo-realistic single image super-resolution using a generative adversarial network, 2017. 4

    2017

  23. [23]

    Draft-and-revise: Effective image gen- eration with contextual rq-transformer.Advances in Neural Information Processing Systems, 35:30127–30138, 2022

    Doyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho, and WOOK SHIN HAN. Draft-and-revise: Effective image gen- eration with contextual rq-transformer.Advances in Neural Information Processing Systems, 35:30127–30138, 2022. 2

    2022

  24. [24]

    Autoregressive image generation using residual quantization, 2022

    Doyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho, and Wook-Shin Han. Autoregressive image generation using residual quantization, 2022. 2

    2022

  25. [25]

    Vt- bench: Evaluating visual tokenizers for autoregressive image generation, 2025

    Huawei Lin, Tong Geng, Zhaozhuo Xu, and Weijie Zhao. Vt- bench: Evaluating visual tokenizers for autoregressive image generation, 2025. 1

    2025

  26. [26]

    Lumina-mgpt: Illuminate flexible photorealistic text- to-image generation with multimodal generative pretraining,

    Dongyang Liu, Shitian Zhao, Le Zhuo, Weifeng Lin, Yi Xin, Xinyue Li, Qi Qin, Yu Qiao, Hongsheng Li, and Peng Gao. Lumina-mgpt: Illuminate flexible photorealistic text- to-image generation with multimodal generative pretraining,

  27. [27]

    Decoupled weight decay regularization, 2019

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019. 5

    2019

  28. [28]

    Open- magvit2: An open-source project toward democratizing auto-regressive visual generation

    Zhuoyan Luo, Fengyuan Shi, Yixiao Ge, Yujiu Yang, Limin Wang, and Ying Shan. Open-magvit2: An open-source project toward democratizing auto-regressive visual gener- ation.arXiv preprint arXiv:2409.04410, 2024. 2

    work page arXiv 2024

  29. [29]

    Unitok: A uni- 9 fied tokenizer for visual generation and understanding, 2025

    Chuofan Ma, Yi Jiang, Junfeng Wu, Jihan Yang, Xin Yu, Ze- huan Yuan, Bingyue Peng, and Xiaojuan Qi. Unitok: A uni- 9 fied tokenizer for visual generation and understanding, 2025. 2, 3, 5, 6

    2025

  30. [30]

    Textground4m: A prompt-aligned dataset for layout-aware text rendering.Proceedings of the AAAI Conference on Artificial Intelligence, 40(10): 7918–7926, 2026

    Dongxing Mao, Yilin Wang, Linjie Li, Zhengyuan Yang, and Alex Jinpeng Wang. Textground4m: A prompt-aligned dataset for layout-aware text rendering.Proceedings of the AAAI Conference on Artificial Intelligence, 40(10): 7918–7926, 2026. 2

    2026

  31. [31]

    Hello gpt-4o, 2024

    OpenAI. Hello gpt-4o, 2024. Accessed: 2024-09-09. 1

    2024

  32. [32]

    Janus-pro-r1: Ad- vancing collaborative visual comprehension and generation via reinforcement learning, 2025

    Kaihang Pan, Yang Wu, Wendong Bu, Kai Shen, Juncheng Li, Yingting Wang, Yunfei Li, Siliang Tang, Jun Xiao, Fei Wu, Hang Zhao, and Yueting Zhuang. Janus-pro-r1: Ad- vancing collaborative visual comprehension and generation via reinforcement learning, 2025. 2

    2025

  33. [33]

    Im- age transformer

    Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Noam Shazeer, Alexander Ku, and Dustin Tran. Im- age transformer. InInternational conference on machine learning, pages 4055–4064. PMLR, 2018. 2

    2018

  34. [34]

    Scalable diffusion models with transformers, 2023

    William Peebles and Saining Xie. Scalable diffusion models with transformers, 2023. 2

    2023

  35. [35]

    Du, Zehuan Yuan, and Xin- glong Wu

    Liao Qu, Huichao Zhang, Yiheng Liu, Xu Wang, Yi Jiang, Yiming Gao, Hu Ye, Daniel K. Du, Zehuan Yuan, and Xin- glong Wu. Tokenflow: Unified image tokenizer for multi- modal understanding and generation, 2025. 2

    2025

  36. [36]

    Zero-shot text-to-image generation, 2021

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation, 2021. 2

    2021

  37. [37]

    Gener- ating diverse high-fidelity images with vq-vae-2, 2019

    Ali Razavi, Aaron van den Oord, and Oriol Vinyals. Gener- ating diverse high-fidelity images with vq-vae-2, 2019. 2

    2019

  38. [38]

    High-resolution image syn- thesis with latent diffusion models, 2022

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models, 2022. 1, 2

    2022

  39. [39]

    Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding, 2022. 2

    2022

  40. [40]

    Very deep convo- lutional networks for large-scale image recognition, 2015

    Karen Simonyan and Andrew Zisserman. Very deep convo- lutional networks for large-scale image recognition, 2015. 4

    2015

  41. [41]

    Denois- ing diffusion implicit models, 2022

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models, 2022. 2

    2022

  42. [42]

    Autoregressive model beats diffusion: Llama for scalable image generation, 2024

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation, 2024. 2, 5, 1

    2024

  43. [43]

    Chameleon: Mixed-modal early-fusion foundation models, 2025

    Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models, 2025. 1, 2, 5, 3

    2025

  44. [44]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Li- wei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. InAdvances in Neural Information Processing Systems, pages 84839–84865. Cur- ran Associates, Inc., 2024. 1

    2024

  45. [45]

    Metamorph: Multimodal un- derstanding and generation via instruction tuning, 2024

    Shengbang Tong, David Fan, Jiachen Zhu, Yunyang Xiong, Xinlei Chen, Koustuv Sinha, Michael Rabbat, Yann LeCun, Saining Xie, and Zhuang Liu. Metamorph: Multimodal un- derstanding and generation via instruction tuning, 2024. 2

    2024

  46. [46]

    Anytext: Multilingual visual text gener- ation and editing, 2024

    Yuxiang Tuo, Wangmeng Xiang, Jun-Yan He, Yifeng Geng, and Xuansong Xie. Anytext: Multilingual visual text gener- ation and editing, 2024. 2, 5

    2024

  47. [47]

    Neural discrete representation learning,

    Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learning,

  48. [48]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023. 2

    2023

  49. [49]

    Beyond words: Advancing long-text im- age generation via multimodal autoregressive models, 2025

    Alex Jinpeng Wang, Linjie Li, Zhengyuan Yang, Lijuan Wang, and Min Li. Beyond words: Advancing long-text im- age generation via multimodal autoregressive models, 2025. 1

    2025

  50. [50]

    Textat- las5m: A large-scale dataset for dense text image generation,

    Alex Jinpeng Wang, Dongxing Mao, Jiawei Zhang, Weiming Han, Zhuobai Dong, Linjie Li, Yiqi Lin, Zhengyuan Yang, Libo Qin, Fuwei Zhang, Lijuan Wang, and Min Li. Textat- las5m: A large-scale dataset for dense text image generation,

  51. [51]

    Emu3: Next-token prediction is all you need, 2024

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, Yingli Zhao, Yulong Ao, Xuebin Min, Tao Li, Boya Wu, Bo Zhao, Bowen Zhang, Liangdong Wang, Guang Liu, Zheqi He, Xi Yang, Jingjing Liu, Yonghua Lin, Tiejun Huang, and Zhongyuan Wang. Emu3: Next-token prediction is all you need, 2024. 1, 2

    2024

  52. [52]

    Edge-enhanced feature distillation network for efficient super-resolution, 2022

    Yan Wang. Edge-enhanced feature distillation network for efficient super-resolution, 2022. 4

    2022

  53. [53]

    Autoregressive visual tracking

    Xing Wei, Yifan Bai, Yongchao Zheng, Dahu Shi, and Yi- hong Gong. Autoregressive visual tracking. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 9697–9706, 2023. 2

    2023

  54. [54]

    Janus: Decoupling visual encoding for unified multimodal understanding and genera- tion, 2024

    Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, and Ping Luo. Janus: Decoupling visual encoding for unified multimodal understanding and genera- tion, 2024. 2

    2024

  55. [56]

    Qwen-image technical report,

    Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, Yuxiang Chen, Zecheng Tang, Zekai Zhang, Zhengyi Wang, An Yang, Bowen Yu, Chen Cheng, Dayiheng Liu, De- qing Li, Hang Zhang, Hao Meng, Hu Wei, Jingyuan Ni, Kai Chen, Kuan Cao, Liang Peng, Lin Qu, Minggang Wu, Peng Wang, Shuting Yu, Tingk...

  56. [57]

    Vila-u: a unified founda- tion model integrating visual understanding and generation,

    Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu 10 Yin, Li Yi, Song Han, and Yao Lu. Vila-u: a unified founda- tion model integrating visual understanding and generation,

  57. [58]

    Stylear: Customizing multimodal autoregressive model for style-aligned text-to-image genera- tion.arXiv preprint arXiv:2505.19874, 2025

    Yi Wu, Lingting Zhu, Shengju Qian, Lei Liu, Wandi Qiao, Lequan Yu, and Bin Li. Stylear: Customizing multimodal autoregressive model for style-aligned text-to-image genera- tion.arXiv preprint arXiv:2505.19874, 2025. 2

    work page arXiv 2025

  58. [59]

    Sana: Efficient high-resolution im- age synthesis with linear diffusion transformers, 2024

    Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, and Song Han. Sana: Efficient high-resolution im- age synthesis with linear diffusion transformers, 2024. 2

    2024

  59. [60]

    Sana 1.5: Efficient scaling of training-time and inference-time compute in linear diffusion transformer,

    Enze Xie, Junsong Chen, Yuyang Zhao, Jincheng Yu, Ligeng Zhu, Chengyue Wu, Yujun Lin, Zhekai Zhang, Muyang Li, Junyu Chen, Han Cai, Bingchen Liu, Daquan Zhou, and Song Han. Sana 1.5: Efficient scaling of training-time and inference-time compute in linear diffusion transformer,

  60. [61]

    Lumina-mgpt 2.0: Stand-alone autoregressive image model- ing, 2025

    Yi Xin, Juncheng Yan, Qi Qin, Zhen Li, Dongyang Liu, Shicheng Li, Victor Shea-Jay Huang, Yupeng Zhou, Ren- rui Zhang, Le Zhuo, Tiancheng Han, Xiaoqing Sun, Siqi Luo, Mengmeng Wang, Bin Fu, Yuewen Cao, Hongsheng Li, Guangtao Zhai, Xiaohong Liu, Yu Qiao, and Peng Gao. Lumina-mgpt 2.0: Stand-alone autoregressive image model- ing, 2025. 2

    2025

  61. [62]

    Gigatok: Scaling visual tokenizers to 3 billion parameters for autoregressive image generation, 2025

    Tianwei Xiong, Jun Hao Liew, Zilong Huang, Jiashi Feng, and Xihui Liu. Gigatok: Scaling visual tokenizers to 3 billion parameters for autoregressive image generation, 2025. 2

    2025

  62. [63]

    Vector-quantized image modeling with improved vqgan, 2022

    Jiahui Yu, Xin Li, Jing Yu Koh, Han Zhang, Ruoming Pang, James Qin, Alexander Ku, Yuanzhong Xu, Jason Baldridge, and Yonghui Wu. Vector-quantized image modeling with improved vqgan, 2022. 2

    2022

  63. [64]

    Randomized autoregressive visual generation

    Qihang Yu, Ju He, Xueqing Deng, Xiaohui Shen, and Liang- Chieh Chen. Randomized autoregressive visual generation. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 18431–18441, 2025. 2

    2025

  64. [65]

    Regularized vector quantization for tokenized im- age synthesis

    Jiahui Zhang, Fangneng Zhan, Christian Theobalt, and Shi- jian Lu. Regularized vector quantization for tokenized im- age synthesis. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18467– 18476, 2023. 2

    2023

  65. [66]

    Distill- ing inter-class distance for semantic segmentation.arXiv preprint arXiv:2205.03650, 2022

    Zhengbo Zhang, Chunluan Zhou, and Zhigang Tu. Distill- ing inter-class distance for semantic segmentation.arXiv preprint arXiv:2205.03650, 2022. 3

    work page arXiv 2022

  66. [67]

    Diff-tracker: text-to-image diffusion models are un- supervised trackers

    Zhengbo Zhang, Li Xu, Duo Peng, Hossein Rahmani, and Jun Liu. Diff-tracker: text-to-image diffusion models are un- supervised trackers. InEuropean Conference on Computer Vision, pages 319–337. Springer, 2024

    2024

  67. [68]

    Performing defocus deblurring by model- ing its formation process

    Zhengbo Zhang, Lin Geng Foo, Hossein Rahmani, Jun Liu, and De Wen Soh. Performing defocus deblurring by model- ing its formation process. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 5791– 5801, 2025

    2025

  68. [69]

    Visual prompting for one-shot controllable video editing without inversion

    Zhengbo Zhang, Yuxi Zhou, Duo Peng, Joo-Hwee Lim, Zhi- gang Tu, De Wen Soh, and Lin Geng Foo. Visual prompting for one-shot controllable video editing without inversion. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 7784–7794, 2025. 3

    2025

  69. [70]

    Movq: Modulating quantized vectors for high- fidelity image generation.Advances in Neural Information Processing Systems, 35:23412–23425, 2022

    Chuanxia Zheng, Tung-Long Vuong, Jianfei Cai, and Dinh Phung. Movq: Modulating quantized vectors for high- fidelity image generation.Advances in Neural Information Processing Systems, 35:23412–23425, 2022. 2 11 Residual Decoder Adapter: ID-Preserving Tokenizer Adaption for Autoregressive Text Rendering Supplementary Material Contents

    2022

  70. [71]

    Shared-ID Hint Codebook

    Residual Decoder Adapter 3 4.1. Shared-ID Hint Codebook . . . . . . . . . . 3 4.2. Residual Decoder . . . . . . . . . . . . . . . 4 4.3. Training Objective and Optimization . . . . 4 4.4. Plug-and-Play Inference in AR model . . . . 5

  71. [72]

    Implementation details

    Experiments 5 5.1. Implementation details . . . . . . . . . . . . 5 5.2. Evaluation Protocol . . . . . . . . . . . . . . 5 5.3. Main Results. . . . . . . . . . . . . . . . . . 5 5.4. Ablation Study . . . . . . . . . . . . . . . . 6 5.5. Discussion . . . . . . . . . . . . . . . . . . 7 5.6. Qualitative Results . . . . . . . . . . . . . . 8

  72. [73]

    Model Architecture

    Implementation Details 1 7.1. Model Architecture . . . . . . . . . . . . . . 1 7.2. Training Configuration . . . . . . . . . . . . 1

  73. [74]

    Parameters and Latency

    Computational Cost Analysis 2 8.1. Parameters and Latency . . . . . . . . . . . 2

  74. [75]

    Tokenizer Reconstruction Limit

    Understanding the Bottleneck 2 9.1. Tokenizer Reconstruction Limit . . . . . . . 2 9.2. Dual Bottleneck in Text Rendering . . . . . 2 10 . Failure Case Analysis 2 10.1 . Tokenizer-Level Failures . . . . . . . . . . . 2 10.2 . AR Model-Level Failures . . . . . . . . . . 3 11 . Training Analysis 3 11.1 . Training Stability and Convergence . . . . . 3 12 . D...

  75. [76]

    Model Architecture Hint CodebookWe instantiate a paired codebook that mir- rors the size and index space of the original tokenizer code- book

    Implementation Details 7.1. Model Architecture Hint CodebookWe instantiate a paired codebook that mir- rors the size and index space of the original tokenizer code- book. The embedding vectors are learned from scratch, but their indices remain aligned with the base codebook. Projector DesignWe use a lightweight1×1Conv2d to map hint-codebook embeddings (d ...

  76. [77]

    Parameters and Latency Tab

    Computational Cost Analysis 8.1. Parameters and Latency Tab. 9 quantifies the computational overhead introduced by RDA on different AR models and tokenizers. All measure- ments are conducted in inference mode on a single V100 GPU. RDA introduces negligible overhead (<2%latency), while achieving substantial improvements in text rendering quality

  77. [78]

    Tokenizer Reconstruction Limit We verify that the base tokenizer exhibits inherent recon- struction limitations even when provided perfect ground- truth input

    Understanding the Bottleneck 9.1. Tokenizer Reconstruction Limit We verify that the base tokenizer exhibits inherent recon- struction limitations even when provided perfect ground- truth input. Fig. 9 shows that the quantization-decoding process inherently loses fine-grained details, blurring text strokes and distorting glyph edges. This confirms that the...

  78. [79]

    Tokenizer-Level Failures When characters are extremely small or visually ambigu- ous, the tokenizer may assign incorrect visual tokens, lead- ing to unrecoverable errors

    Failure Case Analysis We categorize failures into two types based on their source: 10.1. Tokenizer-Level Failures When characters are extremely small or visually ambigu- ous, the tokenizer may assign incorrect visual tokens, lead- ing to unrecoverable errors. Fig. 10 illustrates such cases. 2 Wrong textMass textText too small Figure 11.Failure Case of AR ...

  79. [80]

    Training Stability and Convergence Fig

    Training Analysis 11.1. Training Stability and Convergence Fig. 12 presents training curves for RDA. The optimization is stable throughout training. Critically, without residual perceptual lossL res perc, the residual branch fails to converge and produces only blurry gray regions. This underscores the importance of perceptual supervision for learning mean...

  80. [81]

    Why Shared-ID Preserves Compatibility The Shared-ID mechanism ensures that the token ID distri- bution remains identical to the base tokenizer

    Design Justifications 12.1. Why Shared-ID Preserves Compatibility The Shared-ID mechanism ensures that the token ID distri- bution remains identical to the base tokenizer. Since AR models learn a distribution over token IDs (not codebook embeddings), they can directly benefit from im- proved reconstruction without retraining. 12.2. Why TAR Can Use RDA (Ll...

Showing first 80 references.