Pith. sign in

REVIEW 3 major objections 6 minor 96 references

One MLLM-conditioned next-scale auto-regressive decoder can handle over 15 image-out tasks with up to 19× lower latency than diffusion UVG systems.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 22:20 UTC pith:2AAM6DI2

load-bearing objection Solid MLLM+next-scale-VAR UVG system with real multi-task evidence; the “19× latency” headline is oversold FLOPs, not measured wall-clock. the 3 major comments →

arxiv 2607.24157 v1 pith:2AAM6DI2 submitted 2026-07-27 cs.CV

UniGen-AR: Unifying Visual Generation with Auto-Regressive Modeling

Zhipeng Bao , Zhen Zhu , Nupur Kumari , Anurag Bagchi , Yu-Xiong Wang , Pavel Tokmakov , Martial Hebert This is my paper
classification cs.CV
keywords unified visual generationvisual auto-regressive modelingnext-scale predictionmultimodal language modelVQ-VAE tokenizerimage editingimage restorationinference latency
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Computer vision still runs separate models for text-to-image, editing, restoration, and perception. This paper argues those jobs can share one image-out interface, called Unified Visual Generation. Diffusion systems already do this well but pay a heavy price in iterative sampling speed. UniGen-AR keeps a multimodal language model for free-form instructions and swaps the usual diffusion backbone for a next-scale visual auto-regressive decoder. The language model turns text and reference images into one conditioning sequence; the decoder then predicts multi-scale discrete tokens that become the final image or dense map. Trained jointly on public data for more than fifteen tasks, the single weight set matches or beats prior unified systems on quality while cutting inference cost by roughly an order of magnitude. The work also shows that the discrete visual tokenizer—especially codebook size and hierarchy—is the main bottleneck for scaling this style of model.

Core claim

A single MLLM-conditioned next-scale visual auto-regressive backbone can serve as a unified image-out engine for text-to-image synthesis, editing, restoration, and classical perception, delivering competitive or better quality than diffusion-based unified models at up to 19× lower inference latency.

What carries the argument

Next-scale visual auto-regressive (VAR) decoding conditioned by MLLM text embeddings, with finest-scale reference tokens prepended as a non-predictive context prefix under a block-wise causal mask; tokens are produced by a multi-scale bitwise VQ-VAE and decoded to the final image-valued output.

Load-bearing premise

A discrete multi-scale codebook trained mainly for natural images is expressive enough to represent depth, normals, edges, poses, masks, and restored images without any task-specific heads.

What would settle it

Train the same architecture with a weaker tokenizer (higher reconstruction FID) or drop the finest-scale reference tokens and measure whether restoration and perception metrics collapse while text-to-image stays intact; if quality holds, the shared-latent claim is wrong.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Unified visual systems can move from multi-step diffusion sampling to a handful of coarse-to-fine auto-regressive stages without sacrificing task coverage.
  • One weight set and one prompting interface can replace separate pipelines for generation, editing, restoration, and dense prediction.
  • Tokenizer design (codebook size and hierarchy) becomes a first-order lever for scaling multi-task image-out models.
  • Joint multimodal-understanding fine-tuning of the language front-end can further lift generation metrics, especially multi-object reasoning.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the shared discrete latent continues to improve, the same backbone could absorb video or 3D image-valued tasks without new decoder heads.
  • The latency gap suggests interactive editing and on-device restoration become practical once tokenizer fidelity matches continuous latents.
  • Catastrophic forgetting on pure text-to-image after multi-task training points to a need for continual or mixture-of-experts schedules in future UVG systems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes UniGen-AR, a unified visual generation (UVG) framework that couples a Qwen2.5-VL multimodal encoder with an Infinity-style next-scale visual autoregressive (VAR) decoder. Reference images are injected as finest-scale (non-quantized) VQ-VAE tokens prepended as a non-predictive context prefix, while target outputs follow the standard coarse-to-fine VAR schedule; a two-stage training recipe first aligns the MLLM to the frozen VAR decoder, then jointly finetunes on an 8M-example, 15+-task mixture spanning T2I, editing, restoration, perception, and referring segmentation. The authors report competitive or improved quality over prior AR-based UVG systems and comparable-to-better results versus diffusion UVG baselines, together with efficiency claims: ~5x measured latency reduction over an SD3 decoder under matched finetuning (Table 4), and larger FLOPs ratios in Table 3, which the abstract summarizes as "up to 19x lower inference latency." Ablations cover tokenizer choice, two-stage training, data mixing, reference-token design, and a multimodal-understanding auxiliary objective.

Significance. If the results hold, the paper demonstrates that next-scale VAR backbones are viable general-purpose decoders for unified visual generation, an empirical question that was open — prior AR-based UVG systems (e.g., X-Prompt) lagged noticeably, and UniGen-AR closes or reverses that gap on several benchmarks. Strengths worth naming: the controlled Infinity (UVG) baseline isolates the contribution of MLLM conditioning; Table 4 provides a matched-finetune diffusion-vs-VAR comparison; the tokenizer ablation (Fig. 4) and reference-token ablation (Table E) give actionable design guidance; the model matches LISA-level RefCOCO performance without SAM or extra segmentation data; and training uses only public data with a promised release of the synthetic degradation pipeline. The efficiency claim, while currently overstated in the abstract, is directionally supported by a real ~5x latency measurement and a ~7-8x FLOPs gap at the supported resolution, which is still practically meaningful.

major comments (3)
  1. [Abstract; §1 contributions; Table 3 vs Table 4] The abstract states 'up to 19x lower inference latency', but the only wall-clock measurement in the paper (Table 4) shows 5.23s vs 1.05s, i.e. ~5x, and the Introduction itself says 'reducing inference latency by up to ~5x'. The 19x figure traces to Table 3's FLOPs ratio at 1024x1024 (79T vs 4.1T) — FLOPs, not latency. FLOPs ratios for next-scale VAR do not translate 1:1 to wall-clock (sequential scale decoding, KV-cache overhead), which the authors partially acknowledge in §4.3. Since 'up to 19x lower inference latency' is listed as a contribution, this internal contradiction must be resolved: either report latency at the configuration that yields the claimed number, or restate the claim as the measured ~5x latency plus a FLOPs comparison.
  2. [Table 3; §4.1; §4.5] Table 3 reports FLOPs and quality metrics for 'UniGen-AR (1024)' and 'Qwen + SD3 (1024)', yet §4.1 and §4.5 state that all outputs are generated at a fixed 512x512 resolution and name this a key limitation. It is unclear what model produced the 1024x1024 row — was UniGen-AR run out-of-distribution at 1024, or is this an extrapolated FLOPs calculation? Note also that the 1024 row shows *better* metrics (CLIP-I 0.862 vs 0.849) than the 512 row, which is surprising if the model was only trained at 512. The provenance and meaning of the 1024 rows must be clarified; if the 1024 numbers are analytic FLOPs estimates rather than measured runs, they should be labeled as such and removed from the latency narrative.
  3. [§4.3; Table 3; Supp. C] The diffusion comparator underlying both the latency and FLOPs claims is SD3 sampled at the diffusers default of 28 steps (Supp. C) with no step reduction, distillation, or scheduler tuning. This is a reasonable reference point but an unoptimized one; against the efficient-diffusion SANA baseline the FLOPs advantage is already only ~7x (14T vs 1.9T), and no latency is reported for SANA at all. Given that the efficiency claim is one of the paper's headline contributions, the comparison should either (a) include latency for the efficient-diffusion baseline, or (b) soften the framing to acknowledge that the gap narrows substantially against modern efficient diffusion backbones. Additionally, the text in §4.3 claims '>10x' FLOPs reduction, but at the actually supported 512 resolution the ratio is 16T/1.9T ≈ 8.4x; the >10x figure only holds at the 1024 resolution the model does not support.
minor comments (6)
  1. [Table 1] Table 1: the VAREdit row reports CLIP-OUT but leaves CLIP-I and DINO blank; please clarify whether these were unavailable from the source paper or not evaluated.
  2. [Figures 1–2] Figure 1 contains OCR/typographic artifacts in the task labels ('Inpain6ng', 'Coloriza6on', 'AFribute-level Edi6ng', 'Deblur' vs 'Deblur'); Figure 2 shows 'A'n' for attention. Please clean up the figure text.
  3. [Supp. D.3, Table D] Table D's rows are run together in the current formatting (the 0.1 and 0.5 rows lack line breaks), making the T2I-ratio ablation hard to parse.
  4. [§3.2] §3.2: the citation of EditVAR [49] as 'concurrent work' supporting the finest-scale-only reference design deserves a sentence on what is independently validated here versus inherited; Table E helps but is only in the supplement — consider referencing it explicitly from §3.2.
  5. [§4.2; Table 6] §4.2 attributes the GenEval drop relative to Infinity to 'large-scale proprietary training data'; Table 6's stage-I checkpoint analysis supports this, but the claim would be strengthened by stating the stage-I data volume (4.1M public images) alongside Infinity's training scale for context.
  6. [Supp. A] Supp. A states the synthetic degradation pipeline will be released; given that restoration results are among the paper's strongest, a concrete release commitment (code/seed/parameters) would materially improve reproducibility.

Circularity Check

0 steps flagged

Empirical systems paper with external benchmarks; no derivation reduces a claimed result to its own inputs by construction.

full rationale

UniGen-AR is an engineering/empirical UVG paper: it pairs a frozen-then-finetuned MLLM conditioner with a next-scale VAR decoder, trains on public multi-task data, and reports quality on external suites (GenEval, Emu-Edit, NYUv2, LOL, GoPro, Rain100L, RefCOCO) plus wall-clock/FLOPs comparisons to diffusion baselines. None of the load-bearing claims is a fitted constant renamed as a prediction, a quantity defined in terms of itself, or a uniqueness/ansatz result imported only via overlapping-author citation. Infinity (UVG) is an explicit controlled ablation under the same data and schedule, not a self-proving premise. Tokenizer rFID–GenEval correlation (Fig. 4), two-stage training gains (Table 5), and multi-task synergy (Table 6) are empirical observations against held-out metrics. Disputes about the Abstract’s “19× latency” figure (FLOPs at 1024 vs measured ~5× wall-clock at 512) are overclaim/correctness issues, not circularity. No circular steps identified.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 1 invented entities

Load-bearing content is engineering assumptions and pretrained components, not new physical entities. The claim rests on the VAR next-scale factorization, Infinity’s bitwise VQ-VAE, Qwen text-embedding conditioning, finest-scale reference tokens, two-stage alignment, and a hand-chosen multi-task data mixture. Free parameters are training knobs (learning rates, T2I batch ratio, stages) that affect reported scores but are not hidden inside a closed-form “law.”

free parameters (4)
  • T2I batch sampling probability = 0.25
    Set to 0.25 in Stage II; ablation Table D shows GenEval vs restoration trade-off when varied (0.1/0.25/0.5).
  • Stage I / Stage II learning rates and schedule = 1e-4 then 5e-6
    1e-4 for 2 epochs (Stage I) then 5e-6 for 100k steps (Stage II); two-stage design is required for reported scores (Table 5).
  • Output resolution = 512×512
    Fixed 512×512 training/eval; limitation section notes this choice blocks arbitrary-size inputs and affects latency comparisons.
  • CFG / dropout rates in Stage II = 0.1 / 0.1
    Dropout 0.1 and CFG rate 0.1 listed in supplement Table B; standard generative knobs that move sample quality.
axioms (4)
  • domain assumption Images (and dense maps) admit a multi-scale discrete VQ factorization where next-scale AR prediction is a valid generative model (Eqs. 1–3).
    Imported from VAR/Infinity; UniGen-AR does not re-derive why scale-wise tokens capture UVG outputs.
  • ad hoc to paper Finest-scale reference tokens alone suffice as visual context for image-to-image UVG; coarser reference scales can be dropped.
    §3.2 design choice, justified by EditVAR concurrence and Table E ablation, not a general theorem.
  • domain assumption Pure text embeddings from the last MLLM layer (without visual embeddings) are adequate conditioning for cross-attention into the VAR decoder.
    Follows Qwen-Image-style practice cited in §3.2; central to the interface.
  • domain assumption Public paired datasets plus synthetic degradations are representative enough to evaluate unified restoration/perception quality.
    Training data §4.1 and supplement A; synthetic rain/low-light/blur pipelines define several labels.
invented entities (1)
  • UniGen-AR unified token sequence (r_ref, [SoS], multi-scale target tokens) with block-wise causal mask no independent evidence
    purpose: Glue MLLM conditioning and reference images into Infinity-style next-scale decoding without task-specific heads.
    Architectural construct, not a new physical object; independent evidence is only the paper’s empirical metrics.

pith-pipeline@v1.2.0-grok45-kimik3 · 26513 in / 3429 out tokens · 73220 ms · 2026-07-31T22:20:35.081306+00:00 · methodology

0 comments
read the original abstract

Modern computer vision pipelines remain fragmented, with tasks such as text-to-image generation, editing, restoration, and classical perception handled by separate models. We study Unified Visual Generation (UVG), where a single model produces diverse image-valued outputs through a unified multimodal interface. While diffusion-based systems dominate UVG due to strong quality and controllability, their iterative sampling incurs substantial inference latency, limiting practical deployment. To address these limitations, we propose UniGen-AR, a framework that pairs a general-purpose multi-modal language model (MLLM) with an efficient next-scale visual auto-regressive (VAR) decoder. This design retains the flexibility of MLLM-based conditioning while leveraging the sampling efficiency and latent unification properties of VAR models. In our framework, the MLLM encodes free-form instructions and control signals into a unified sequence, which guides the VAR decoder to generate image-valued outputs for over 15 tasks spanning four families. Empirically, UniGen-AR achieves up to $19 \times$ lower inference latency than diffusion-based baselines while maintaining or improving output quality. Our ablations further reveal that VQ-VAE tokenizer design, particularly codebook size and hierarchy, is a critical factor for VAR scalability in UVG. These results establish visual auto-regressive modeling as a compelling and efficient backbone for unified visual generation. Our project page is at https://zpbao.github.io/projects/unigenar.

Figures

Figures reproduced from arXiv: 2607.24157 by Anurag Bagchi, Martial Hebert, Nupur Kumari, Pavel Tokmakov, Yu-Xiong Wang, Zhen Zhu, Zhipeng Bao.

Figure 1
Figure 1. Figure 1: UniGen-AR: A single model for Unified Visual Generation (UVG). Our framework jointly handles over 15 diverse tasks, spanning text-to-image synthesis, restoration, perception, and image editing. All outputs are generated by a single, MLLM-conditioned auto-regressive backbone, using a unified prompting interface and a single set of model weights without any task-specific heads. Abstract Modern computer visio… view at source ↗
Figure 2
Figure 2. Figure 2: Architecture of UniGen-AR. Left: We extend an Infinity-style VAR backbone with a multimodal language model encoder to support Unified Visual Generation with reference images. The MLLM encodes the instruction and reference image, whose text embeddings seed a learnable [SoS] token and provide keys/values for cross-attention, while finest-scale reference tokens are prepended as a non￾predictive context prefix… view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative comparisons across UVG tasks. UniGen￾AR consistently produces higher-quality results than the UVG baseline OmniGen and achieves visually comparable or supe￾rior outputs to specialized models, notably outperforming Instruc￾tIR [11] on deraining. tion are also included in the supplementary material. 4.2. Main results We report results on T2I generation and image editing in Ta￾ble 1, and results o… view at source ↗
Figure 4
Figure 4. Figure 4: Impact of visual tokenizers. Better recon￾struction fidelity leads to improved generation quality. Model Two Obj. (↑) Position (↑) Color Attr. (↑) Overall (↑) LOL-Lowlight (↑) GoPro-Deblur (↑) Rain100L-Derain (↑) UniGen-AR (Single) 0.68 0.31 0.40 0.59 20.51 22.18 32.26 UniGen-AR (TwoStage) 0.76 0.41 0.45 0.68 21.03 22.99 33.71 [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

96 extracted references · 41 linked inside Pith

  1. [1]

    Estimating and exploiting the aleatoric uncertainty in surface normal estimation

    Gwangbin Bae, Ignas Budvytis, and Roberto Cipolla. Estimating and exploiting the aleatoric uncertainty in surface normal estimation. InICCV, 2021. 6

    2021

  2. [2]

    Refereverything: Towards segmenting everything we can speak of in videos

    Anurag Bagchi, Zhipeng Bao, Yu-Xiong Wang, Pavel Tokmakov, and Martial Hebert. Refereverything: Towards segmenting everything we can speak of in videos. InICCV, 2025. 2

    2025

  3. [3]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025. 2, 4, 5, 17

    Pith/arXiv arXiv 2025

  4. [4]

    Sequential modeling enables scalable learning for large vision models

    Yutong Bai, Xinyang Geng, Karttikeya Mangalam, Amir Bar, Alan L Yuille, Trevor Darrell, Jitendra Ma- lik, and Alexei A Efros. Sequential modeling enables scalable learning for large vision models. InCVPR,

  5. [5]

    Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, Sumith Kulal, et al. Flux. 1 kontext: Flow matching for in-context image generation and editing in latent space.arXiv e-prints, 2025. 2, 3, 9

    2025

  6. [6]

    Improving image generation with better captions.Computer Science

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions.Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2023. 6

    2023

  7. [7]

    Instructpix2pix: Learning to follow image edit- ing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image edit- ing instructions. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 18392–18402, 2023. 3, 6

    2023

  8. [8]

    Visual generation without guidance.arXiv preprint arXiv:2501.15420, 2025

    Huayu Chen, Kai Jiang, Kaiwen Zheng, Jianfei Chen, Hang Su, and Jun Zhu. Visual generation without guidance.arXiv preprint arXiv:2501.15420, 2025. 3

    Pith/arXiv arXiv 2025

  9. [9]

    Scaling instruction-finetuned language models.JMLR, 2024

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models.JMLR, 2024. 4, 5

    2024

  10. [10]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    Gheorghe Comanici, Eric Bieber, Mike Schaek- ermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025. 2

    Pith/arXiv arXiv 2025

  11. [11]

    Instructir: High-quality image restoration following human instructions

    Marcos V Conde, Gregor Geigle, and Radu Timofte. Instructir: High-quality image restoration following human instructions. InECCV, 2024. 6, 7

    2024

  12. [12]

    Emerging proper- ties in unified multimodal pretraining.arXiv preprint arXiv:2505.14683, 2025

    Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. Emerging proper- ties in unified multimodal pretraining.arXiv preprint arXiv:2505.14683, 2025. 3

    Pith/arXiv arXiv 2025

  13. [13]

    Imagenet: A large-scale hierarchi- cal image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchi- cal image database. InCVPR, 2009. 8

    2009

  14. [14]

    Vision-language transformer and query genera- tion for referring segmentation

    Henghui Ding, Chang Liu, Suchen Wang, and Xudong Jiang. Vision-language transformer and query genera- tion for referring segmentation. InICCV, 2021. 17

    2021

  15. [15]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. InForty-first international confer- ence on machine learning, 2024. 6, 7, 9, 17

    2024

  16. [16]

    Univg: A generalist diffusion model for unified image generation and editing.arXiv preprint arXiv:2503.12652, 2025

    Tsu-Jui Fu, Yusu Qian, Chen Chen, Wenze Hu, Zhe Gan, and Yinfei Yang. Univg: A generalist diffusion model for unified image generation and editing.arXiv preprint arXiv:2503.12652, 2025. 3

    Pith/arXiv arXiv 2025

  17. [17]

    Removing rain from single images via a deep detail network

    Xueyang Fu, Jiabin Huang, Delu Zeng, Yue Huang, Xinghao Ding, and John Paisley. Removing rain from single images via a deep detail network. InCVPR,

  18. [18]

    Instructcv: Instruction-tuned text-to-image diffusion models as vision generalists.arXiv preprint arXiv:2310.00390,

    Yulu Gan, Sungwoo Park, Alexander Schubert, An- thony Philippakis, and Ahmed M Alaa. Instructcv: Instruction-tuned text-to-image diffusion models as vision generalists.arXiv preprint arXiv:2310.00390,

    Pith/arXiv arXiv

  19. [19]

    Seed-x: Multimodal models with unified multi- granularity comprehension and generation.arXiv preprint arXiv:2404.14396, 2024

    Yuying Ge, Sijie Zhao, Jinguo Zhu, Yixiao Ge, Kun Yi, Lin Song, Chen Li, Xiaohan Ding, and Ying Shan. Seed-x: Multimodal models with unified multi- granularity comprehension and generation.arXiv preprint arXiv:2404.14396, 2024. 3

    Pith/arXiv arXiv 2024

  20. [20]

    Geneval: An object-focused framework for evaluating text-to-image alignment.NeurIPS, 2023

    Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment.NeurIPS, 2023. 6, 8

    2023

  21. [21]

    Vector quantized diffusion model for text-to- image synthesis

    Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, and Baining Guo. Vector quantized diffusion model for text-to- image synthesis. InCVPR, 2022. 2

    2022

  22. [22]

    Infin- ity: Scaling bitwise autoregressive modeling for high- resolution image synthesis

    Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaobing Liu. Infin- ity: Scaling bitwise autoregressive modeling for high- resolution image synthesis. InCVPR, 2025. 2, 3, 4, 5, 6, 8, 9, 15

    2025

  23. [23]

    Ace: All-round creator and editor following instructions via diffusion transformer.arXiv preprint arXiv:2410.00086, 2024

    Zhen Han, Zeyinzi Jiang, Yulin Pan, Jingfeng Zhang, Chaojie Mao, Chenwei Xie, Yu Liu, and Jingren Zhou. Ace: All-round creator and editor following instructions via diffusion transformer.arXiv preprint arXiv:2410.00086, 2024. 3

    Pith/arXiv arXiv 2024

  24. [24]

    Stylebooth: Image style editing with multimodal instruction

    Zhen Han, Chaojie Mao, Zeyinzi Jiang, Yulin Pan, and Jingfeng Zhang. Stylebooth: Image style editing with multimodal instruction. InICCV, 2025. 5, 16

    2025

  25. [25]

    Gans trained by a two time-scale up- date rule converge to a nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Un- terthiner, Bernhard Nessler, G ¨unter Klambauer, and Sepp Hochreiter. Gans trained by a two time-scale up- date rule converge to a nash equilibrium. InNeurIPS,

  26. [26]

    Denoising diffusion probabilistic models.NeurIPS, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.NeurIPS, 2020. 2

    2020

  27. [27]

    We- gen: A unified model for interactive multimodal gen- eration as we chat

    Zhipeng Huang, Shaobin Zhuang, Canmiao Fu, Binxin Yang, Ying Zhang, Chong Sun, Zhizheng Zhang, Yali Wang, Chen Li, and Zheng-Jun Zha. We- gen: A unified model for interactive multimodal gen- eration as we chat. InCVPR, 2025. 3

    2025

  28. [28]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Rad- ford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024. 2

    Pith/arXiv arXiv 2024

  29. [29]

    Unified language-vision pretrain- ing in llm with dynamic discrete visual tokenization

    Yang Jin, Kun Xu, Liwei Chen, Chao Liao, Jianchao Tan, Quzhe Huang, Bin Chen, Chenyi Lei, An Liu, Chengru Song, et al. Unified language-vision pretrain- ing in llm with dynamic discrete visual tokenization. arXiv preprint arXiv:2309.04669, 2023. 3

    arXiv 2023

  30. [30]

    Video-lavit: Unified video- language pre-training with decoupled visual-motional tokenization.arXiv preprint arXiv:2402.03161, 2024

    Yang Jin, Zhicheng Sun, Kun Xu, Liwei Chen, Hao Jiang, Quzhe Huang, Chengru Song, Yuliang Liu, Di Zhang, Yang Song, et al. Video-lavit: Unified video- language pre-training with decoupled visual-motional tokenization.arXiv preprint arXiv:2402.03161, 2024. 3

    arXiv 2024

  31. [31]

    Repurposing diffusion-based image generators for monocular depth estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, Rodrigo Caye Daudt, and Konrad Schindler. Repurposing diffusion-based image generators for monocular depth estimation. InCVPR, 2024. 6

    2024

  32. [32]

    Auto-encoding variational bayes.arXiv preprint arXiv:1312.6114,

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes.arXiv preprint arXiv:1312.6114,

    Pith/arXiv arXiv

  33. [33]

    Generating images with multimodal language models

    Jing Yu Koh, Daniel Fried, and Russ R Salakhutdinov. Generating images with multimodal language models. NeurIPS, 2023. 3

    2023

  34. [34]

    Unleashing in- context learning of autoregressive models for few-shot image manipulation

    Bolin Lai, Felix Juefei-Xu, Miao Liu, Xiaoliang Dai, Nikhil Mehta, Chenguang Zhu, Zeyi Huang, James M Rehg, Sangmin Lee, Ning Zhang, et al. Unleashing in- context learning of autoregressive models for few-shot image manipulation. InCVPR, 2025. 3

    2025

  35. [35]

    Lisa: Reason- ing segmentation via large language model

    Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Reason- ing segmentation via large language model. InCVPR,

  36. [36]

    All-in-one image restora- tion for unknown corruption

    Boyun Li, Xiao Liu, Peng Hu, Zhongqin Wu, Jiancheng Lv, and Xi Peng. All-in-one image restora- tion for unknown corruption. InCVPR, 2022. 6, 7

    2022

  37. [37]

    Imagefolder: Autore- gressive image generation with folded tokens.arXiv preprint arXiv:2410.01756, 2024

    Xiang Li, Kai Qiu, Hao Chen, Jason Kuen, Jiuxiang Gu, Bhiksha Raj, and Zhe Lin. Imagefolder: Autore- gressive image generation with folded tokens.arXiv preprint arXiv:2410.01756, 2024. 3

    Pith/arXiv arXiv 2024

  38. [38]

    Brushedit: All-in-one image inpainting and edit- ing.arXiv preprint arXiv:2412.10316, 2024

    Yaowei Li, Yuxuan Bian, Xuan Ju, Zhaoyang Zhang, Junhao Zhuang, Ying Shan, Yuexian Zou, and Qiang Xu. Brushedit: All-in-one image inpainting and edit- ing.arXiv preprint arXiv:2412.10316, 2024. 3

    Pith/arXiv arXiv 2024

  39. [39]

    Visualcloze: A universal image generation framework via visual in-context learning

    Zhong-Yu Li, Ruoyi Du, Juncheng Yan, Le Zhuo, Zhen Li, Peng Gao, Zhanyu Ma, and Ming-Ming Cheng. Visualcloze: A universal image generation framework via visual in-context learning. InICCV,

  40. [40]

    Realgeneral: Unifying visual genera- tion via temporal in-context learning with video mod- els.arXiv preprint arXiv:2503.10406, 2025

    Yijing Lin, Mengqi Huang, Shuhan Zhuang, and Zhendong Mao. Realgeneral: Unifying visual genera- tion via temporal in-context learning with video mod- els.arXiv preprint arXiv:2503.10406, 2025. 3

    Pith/arXiv arXiv 2025

  41. [41]

    Visual instruction tuning.NeurIPS,

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.NeurIPS,

  42. [42]

    Improved baselines with visual instruction tun- ing

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tun- ing. InCVPR, 2024. 9

    2024

  43. [43]

    Step1x-edit: A practical framework for general image editing.arXiv preprint arXiv:2504.17761, 2025

    Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang, Wei Cheng, Jiaqi Liao, Yingming Wang, Hong- hao Fu, Chunrui Han, et al. Step1x-edit: A practical framework for general image editing.arXiv preprint arXiv:2504.17761, 2025. 2, 3, 6, 7 11

    Pith/arXiv arXiv 2025

  44. [44]

    Decou- pled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Ilya Loshchilov and Frank Hutter. Decou- pled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017. 6

    Pith/arXiv arXiv 2017

  45. [45]

    Unified-io: A unified model for vision, language, and multi-modal tasks.arXiv preprint arXiv:2206.08916,

    Jiasen Lu, Christopher Clark, Rowan Zellers, Roozbeh Mottaghi, and Aniruddha Kembhavi. Unified-io: A unified model for vision, language, and multi-modal tasks.arXiv preprint arXiv:2206.08916,

    Pith/arXiv arXiv

  46. [46]

    Unified-io 2: Scaling au- toregressive multimodal models with vision language audio and action

    Jiasen Lu, Christopher Clark, Sangho Lee, Zichen Zhang, Savya Khosla, Ryan Marten, Derek Hoiem, and Aniruddha Kembhavi. Unified-io 2: Scaling au- toregressive multimodal models with vision language audio and action. InCVPR, 2024. 3

    2024

  47. [47]

    Multi- task collaborative network for joint referring expres- sion comprehension and segmentation

    Gen Luo, Yiyi Zhou, Xiaoshuai Sun, Liujuan Cao, Chenglin Wu, Cheng Deng, and Rongrong Ji. Multi- task collaborative network for joint referring expres- sion comprehension and segmentation. InCVPR,

  48. [48]

    Open- magvit2: An open-source project toward democratiz- ing auto-regressive visual generation.arXiv preprint arXiv:2409.04410, 2024

    Zhuoyan Luo, Fengyuan Shi, Yixiao Ge, Yu- jiu Yang, Limin Wang, and Ying Shan. Open- magvit2: An open-source project toward democratiz- ing auto-regressive visual generation.arXiv preprint arXiv:2409.04410, 2024. 3

    Pith/arXiv arXiv 2024

  49. [49]

    Visual autoregressive modeling for instruction-guided image editing.arXiv preprint arXiv:2508.15772,

    Qingyang Mao, Qi Cai, Yehao Li, Yingwei Pan, Mingyue Cheng, Ting Yao, Qi Liu, and Tao Mei. Visual autoregressive modeling for instruction-guided image editing.arXiv preprint arXiv:2508.15772,

    arXiv

  50. [50]

    Deep multi-scale convolutional neural network for dy- namic scene deblurring

    Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee. Deep multi-scale convolutional neural network for dy- namic scene deblurring. InCVPR, 2017. 6

    2017

  51. [51]

    Kosmos-g: Generat- ing images in context with multimodal large language models.arXiv preprint arXiv:2310.02992, 2023

    Xichen Pan, Li Dong, Shaohan Huang, Zhiliang Peng, Wenhu Chen, and Furu Wei. Kosmos-g: Generat- ing images in context with multimodal large language models.arXiv preprint arXiv:2310.02992, 2023. 3

    Pith/arXiv arXiv 2023

  52. [52]

    Transfer between modalities with metaque- ries.arXiv preprint arXiv:2504.06256, 2025

    Xichen Pan, Satya Narayan Shukla, Aashu Singh, Zhuokai Zhao, Shlok Kumar Mishra, Jialiang Wang, Zhiyang Xu, Jiuhai Chen, Kunpeng Li, Felix Juefei- Xu, et al. Transfer between modalities with metaque- ries.arXiv preprint arXiv:2504.06256, 2025. 3

    Pith/arXiv arXiv 2025

  53. [53]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023. 6

    Pith/arXiv arXiv 2023

  54. [54]

    Visual autoregressive modeling for image super-resolution.arXiv preprint arXiv:2501.18993, 2025

    Yunpeng Qu, Kun Yuan, Jinhua Hao, Kai Zhao, Qizhi Xie, Ming Sun, and Chao Zhou. Visual autoregressive modeling for image super-resolution.arXiv preprint arXiv:2501.18993, 2025. 3

    Pith/arXiv arXiv 2025

  55. [55]

    Hierarchical text- conditional image generation with clip latents.arXiv preprint arXiv:2204.06125, 2022

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents.arXiv preprint arXiv:2204.06125, 2022. 6, 7

    Pith/arXiv arXiv 2022

  56. [56]

    High- resolution image synthesis with latent diffusion mod- els

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High- resolution image synthesis with latent diffusion mod- els. InCVPR, 2022. 2, 3, 6, 9

    2022

  57. [57]

    Upweighting easy sam- ples in fine-tuning mitigates forgetting.arXiv preprint arXiv:2502.02797, 2025

    Sunny Sanyal, Hayden Prairie, Rudrajit Das, Ali Kavis, and Sujay Sanghavi. Upweighting easy sam- ples in fine-tuning mitigates forgetting.arXiv preprint arXiv:2502.02797, 2025. 9

    Pith/arXiv arXiv 2025

  58. [58]

    Laion-400m: Open dataset of clip- filtered 400 million image-text pairs.arXiv preprint arXiv:2111.02114, 2021

    Christoph Schuhmann, Richard Vencu, Romain Beau- mont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Ko- matsuzaki. Laion-400m: Open dataset of clip- filtered 400 million image-text pairs.arXiv preprint arXiv:2111.02114, 2021. 5, 16

    Pith/arXiv arXiv 2021

  59. [59]

    Contin- uous visual autoregressive generation via score maxi- mization.arXiv preprint arXiv:2505.07812, 2025

    Chenze Shao, Fandong Meng, and Jie Zhou. Contin- uous visual autoregressive generation via score maxi- mization.arXiv preprint arXiv:2505.07812, 2025. 3

    Pith/arXiv arXiv 2025

  60. [60]

    Emu edit: Precise image editing via recognition and generation tasks

    Shelly Sheynin, Adam Polyak, Uriel Singer, Yuval Kirstain, Amit Zohar, Oron Ashual, Devi Parikh, and Yaniv Taigman. Emu edit: Precise image editing via recognition and generation tasks. InCVPR, 2024. 6

    2024

  61. [61]

    Indoor segmentation and support in- ference from rgbd images

    Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support in- ference from rgbd images. InECCV, 2012. 6

    2012

  62. [62]

    Autore- gressive model beats diffusion: Llama for scalable image generation.arXiv preprint arXiv:2406.06525,

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autore- gressive model beats diffusion: Llama for scalable image generation.arXiv preprint arXiv:2406.06525,

    Pith/arXiv arXiv

  63. [63]

    Emu: Generative pretraining in multimodality.arXiv preprint arXiv:2307.05222, 2023

    Quan Sun, Qiying Yu, Yufeng Cui, Fan Zhang, Xiaosong Zhang, Yueze Wang, Hongcheng Gao, Jingjing Liu, Tiejun Huang, and Xinlong Wang. Emu: Generative pretraining in multimodality.arXiv preprint arXiv:2307.05222, 2023. 3

    Pith/arXiv arXiv 2023

  64. [64]

    Generative multi- modal models are in-context learners

    Quan Sun, Yufeng Cui, Xiaosong Zhang, Fan Zhang, Qiying Yu, Yueze Wang, Yongming Rao, Jingjing Liu, Tiejun Huang, and Xinlong Wang. Generative multi- modal models are in-context learners. InCVPR, 2024

    2024

  65. [65]

    X-prompt: Towards universal in-context image generation in auto-regressive vision language foundation models.arXiv preprint arXiv:2412.01824,

    Zeyi Sun, Ziyang Chu, Pan Zhang, Tong Wu, Xiaoyi Dong, Yuhang Zang, Yuanjun Xiong, Dahua Lin, and Jiaqi Wang. X-prompt: Towards universal in-context image generation in auto-regressive vision language foundation models.arXiv preprint arXiv:2412.01824,

    Pith/arXiv arXiv

  66. [66]

    Ugen: Unified autoregressive multimodal model with progressive vocabulary learning.arXiv preprint arXiv:2503.21193, 2025

    Hongxuan Tang, Hao Liu, and Xinyan Xiao. Ugen: Unified autoregressive multimodal model with progressive vocabulary learning.arXiv preprint arXiv:2503.21193, 2025. 3

    Pith/arXiv arXiv 2025

  67. [67]

    Codi-2: In- context interleaved and interactive any-to-any genera- tion

    Zineng Tang, Ziyi Yang, Mahmoud Khademi, Yang Liu, Chenguang Zhu, and Mohit Bansal. Codi-2: In- context interleaved and interactive any-to-any genera- tion. InCVPR, 2024. 3 12

    2024

  68. [68]

    Chameleon: Mixed-modal early-fusion foundation models.arXiv preprint arXiv:2405.09818, 2024

    Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models.arXiv preprint arXiv:2405.09818, 2024. 3, 6

    Pith/arXiv arXiv 2024

  69. [69]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. NeurIPS, 2024. 2, 3, 4, 8

    2024

  70. [70]

    Neural discrete representation learning.NeurIPS, 2017

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning.NeurIPS, 2017. 4

    2017

  71. [71]

    Switti: Designing scale-wise trans- formers for text-to-image synthesis.arXiv preprint arXiv:2412.01819, 2024

    Anton V oronov, Denis Kuznedelev, Mikhail Khoroshikh, Valentin Khrulkov, and Dmitry Baranchuk. Switti: Designing scale-wise trans- formers for text-to-image synthesis.arXiv preprint arXiv:2412.01819, 2024. 3, 4, 8

    Pith/arXiv arXiv 2024

  72. [72]

    Editinfinity: Image editing with binary-quantized generative models.arXiv preprint arXiv:2510.20217, 2025

    Jiahuan Wang, Yuxin Chen, Jun Yu, Guangming Lu, and Wenjie Pei. Editinfinity: Image editing with binary-quantized generative models.arXiv preprint arXiv:2510.20217, 2025. 3

    arXiv 2025

  73. [73]

    Scalable autoregressive monocular depth estima- tion

    Jinhong Wang, Jian Liu, Dongqi Tang, Weiqiang Wang, Wentong Li, Danny Chen, Jintai Chen, and Jian Wu. Scalable autoregressive monocular depth estima- tion. InCVPR, 2025. 3

    2025

  74. [74]

    Images speak in images: A gener- alist painter for in-context visual learning

    Xinlong Wang, Wen Wang, Yue Cao, Chunhua Shen, and Tiejun Huang. Images speak in images: A gener- alist painter for in-context visual learning. InCVPR,

  75. [75]

    Emu3: Next-token prediction is all you need.arXiv preprint arXiv:2409.18869, 2024

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need.arXiv preprint arXiv:2409.18869, 2024. 3, 6

    Pith/arXiv arXiv 2024

  76. [76]

    Training- free text-guided image editing with visual autoregres- sive model.arXiv preprint arXiv:2503.23897, 2025

    Yufei Wang, Lanqing Guo, Zhihao Li, Jiaxing Huang, Pichao Wang, Bihan Wen, and Jian Wang. Training- free text-guided image editing with visual autoregres- sive model.arXiv preprint arXiv:2503.23897, 2025. 3

    Pith/arXiv arXiv 2025

  77. [77]

    Deep retinex decomposition for low-light en- hancement.arXiv preprint arXiv:1808.04560, 2018

    Chen Wei, Wenjing Wang, Wenhan Yang, and Jiay- ing Liu. Deep retinex decomposition for low-light en- hancement.arXiv preprint arXiv:1808.04560, 2018. 6

    Pith/arXiv arXiv 2018

  78. [78]

    Omniedit: Building im- age editing generalist models through specialist super- vision

    Cong Wei, Zheyang Xiong, Weiming Ren, Xeron Du, Ge Zhang, and Wenhu Chen. Omniedit: Building im- age editing generalist models through specialist super- vision. InICLR, 2024. 5, 15, 16

    2024

  79. [79]

    Janus: Decou- pling visual encoding for unified multimodal under- standing and generation

    Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, et al. Janus: Decou- pling visual encoding for unified multimodal under- standing and generation. InCVPR, 2025. 3

    2025

  80. [80]

    Qwen-image technical re- port.arXiv preprint arXiv:2508.02324, 2025

    Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng-ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, et al. Qwen-image technical re- port.arXiv preprint arXiv:2508.02324, 2025. 2, 3, 4, 5, 16

    Pith/arXiv arXiv 2025

Showing first 80 references.