Pith. sign in

REVIEW 17 cited by

Unified Autoregressive Visual Generation and Understanding with Continuous Tokens

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2503.13436 v1 pith:HT52AJRV submitted 2025-03-17 cs.CV cs.LG

Lijie Fan , Luming Tang , Siyang Qin , Tianhong Li , Xuan Yang , Siyuan Qiao , Andreas Steiner , Chen Sun
show 6 more authors
Yuanzhen Li Tao Zhu Michael Rubinstein Michalis Raptis Deqing Sun Radu Soricut
This is my paper
classification cs.CVcs.LG
keywords generationimageunderstandingunifiedtokensvisualautoregressivecontinuous
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We present UniFluid, a unified autoregressive framework for joint visual generation and understanding leveraging continuous visual tokens. Our unified autoregressive architecture processes multimodal image and text inputs, generating discrete tokens for text and continuous tokens for image. We find though there is an inherent trade-off between the image generation and understanding task, a carefully tuned training recipe enables them to improve each other. By selecting an appropriate loss balance weight, the unified model achieves results comparable to or exceeding those of single-task baselines on both tasks. Furthermore, we demonstrate that employing stronger pre-trained LLMs and random-order generation during training is important to achieve high-fidelity image generation within this unified framework. Built upon the Gemma model series, UniFluid exhibits competitive performance across both image generation and understanding, demonstrating strong transferability to various downstream tasks, including image editing for generation, as well as visual captioning and question answering for understanding.

Discussion (0). Sign in to comment.

Forward citations

Cited by 17 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. FeynmanBench: Benchmarking Multimodal LLMs on Diagrammatic Physics Reasoning

    cs.AI 2026-04 unverdicted novelty 8.0 of 10

    FeynmanBench is the first benchmark for evaluating multimodal LLMs on diagrammatic reasoning with Feynman diagrams, revealing systematic failures in enforcing physical constraints and global topology.

  2. Pseudo-Unification: Entropy Probing Reveals Divergent Information Patterns in Unified Multimodal Models

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    Unified multimodal models exhibit pseudo-unification due to modality-asymmetric entropy encoding and pattern-split responses between text and image generation.

  3. FeynmanBench: Benchmarking Multimodal LLMs on Diagrammatic Physics Reasoning

    cs.AI 2026-04 conditional novelty 7.0 of 10

    Across 19 multimodal LLMs, local Feynman-diagram recognition stays high while topological reconstruction and full amplitude derivation collapse.

  4. HYDRA-X: Native Unified Multimodal Models with Holistic Visual Tokenizers

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    HYDRA-X presents the first unified multimodal model using a single ViT for holistic image-video tokenization, with ablations on attention and compression plus a latent-level editing improvement.

  5. Lance: Unified Multimodal Modeling by Multi-Task Synergy

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    Lance presents a dual-stream mixture-of-experts model with modality-aware positional encoding and staged multi-task training that outperforms prior open-source unified models on image and video generation while keepin...

  6. Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generation

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    Tuna-2 shows that direct pixel embeddings can replace vision encoders in unified multimodal models, achieving competitive generation and stronger understanding at scale.

  7. Generation Enhances Understanding in Unified Multimodal Models via Multi-Representation Generation

    cs.CV 2026-01 conditional novelty 6.0 of 10

    Adding depth- and segmentation-generation objectives to UMM post-training improved spatial understanding and reduced hallucinations on Harmon and OpenUni while preserving generation quality.

  8. Image Diffusion Preview with Consistency Solver

    cs.LG 2025-12 unverdicted novelty 6.0 of 10

    ConsistencySolver enables high-quality low-step diffusion previews by adapting general linear multistep methods into a lightweight RL-optimized solver, matching multistep DPM-Solver FID with 47% fewer steps and cuttin...

  9. Reconstruction Alignment Improves Unified Multimodal Models

    cs.CV 2025-09 conditional novelty 6.0 of 10

    RECA, a self-supervised post-training objective that conditions unified multimodal models on their own visual understanding embeddings to reconstruct input images, improves text-to-image and editing benchmarks across ...

  10. Generative Distribution Distillation

    cs.LG 2025-07 conditional novelty 6.0 of 10

    Knowledge distillation is reformulated as conditional diffusion over teacher feature tokens, with class-center contraction replacing the classification loss, yielding state-of-the-art ImageNet distillation numbers.

  11. Argus-Unified: Towards A Compact and Economical Unified Model for Image Understanding and Generation

    cs.CV 2026-07 conditional novelty 5.0 of 10

    A compact unified model that reuses a frozen VLM encoder and hybrid continuous/discrete tokens reaches competitive image understanding and generation with 15.6M training images and about $2,000 in compute.

  12. Lance: Unified Multimodal Modeling by Multi-Task Synergy

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    Lance introduces a dual-stream MoE model with modality-aware rotary positional encoding and staged multi-task training that outperforms open-source unified models on image and video generation while retaining understa...

  13. Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generation

    cs.CV 2026-04 unverdicted novelty 5.0 of 10

    Tuna-2 shows pixel embeddings can replace vision encoders in unified multimodal models, achieving competitive or superior results on understanding and generation benchmarks.

  14. Skywork UniPic: Unified Autoregressive Modeling for Visual Understanding and Generation

    cs.CV 2025-08 conditional novelty 5.0 of 10

    A 1.5B unified autoregressive model with separate encoders for generation and understanding reports strong text-to-image and editing scores while running on commodity hardware.

  15. Instruction-based Image Editing: A Survey on Data, Models, Evaluation, and Applications

    cs.CV 2026-07 conditional novelty 4.0 of 10

    A survey of instruction-based image editing plus a new 21-task benchmark, CDD-IIE, on which ten open models are scored by human experts.

  16. Show-o2: Improved Native Unified Multimodal Models

    cs.CV 2025-06 unverdicted novelty 4.0 of 10

    Show-o2 unifies text, image, and video understanding and generation in a single autoregressive-plus-flow-matching model built on 3D causal VAE representations.

  17. Step1X-Edit: A Practical Framework for General Image Editing

    cs.CV 2025-04 unverdicted novelty 4.0 of 10

    Step1X-Edit integrates a multimodal LLM with a diffusion decoder, trained on a custom high-quality dataset, to deliver image editing performance that surpasses open-source baselines and approaches proprietary models o...

Pith tools