Pith. sign in

REVIEW 5 cited by

Vision as a Dialect: Unifying Visual Understanding and Generation via Text-Aligned Representations

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2506.18898 v1 pith:YDY4JWRX submitted 2025-06-23 cs.CV cs.AIcs.CLcs.MM

Jiaming Han , Hao Chen , Yang Zhao , Hanyu Wang , Qi Zhao , Ziyan Yang , Hao He , Xiangyu Yue
show 1 more author
Lu Jiang
This is my paper
classification cs.CVcs.AIcs.CLcs.MM
keywords visualgenerationmodelmultimodaltext-alignedunderstandingdecodingdiscrete
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

This paper presents a multimodal framework that attempts to unify visual understanding and generation within a shared discrete semantic representation. At its core is the Text-Aligned Tokenizer (TA-Tok), which converts images into discrete tokens using a text-aligned codebook projected from a large language model's (LLM) vocabulary. By integrating vision and text into a unified space with an expanded vocabulary, our multimodal LLM, Tar, enables cross-modal input and output through a shared interface, without the need for modality-specific designs. Additionally, we propose scale-adaptive encoding and decoding to balance efficiency and visual detail, along with a generative de-tokenizer to produce high-fidelity visual outputs. To address diverse decoding needs, we utilize two complementary de-tokenizers: a fast autoregressive model and a diffusion-based model. To enhance modality fusion, we investigate advanced pre-training tasks, demonstrating improvements in both visual understanding and generation. Experiments across benchmarks show that Tar matches or surpasses existing multimodal LLM methods, achieving faster convergence and greater training efficiency. Code, models, and data are available at https://tar.csuhan.com

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Generative Refinement Networks for Visual Synthesis

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    Hierarchical Binary Quantization plus global refinement AR yields 0.56 rFID reconstruction and 1.81 gFID class-conditional generation on ImageNet, with competitive T2I/T2V at 2B scale.

  2. dRAE: Representation Autoencoder with Hyper-Spherical Codes

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Switching codebook assignment and update to cosine similarity while keeping a magnitude-preserving commitment loss avoids codebook collapse and scales visual tokenizers to 131,072 codes with high utilization.

  3. Argus-Unified: Towards A Compact and Economical Unified Model for Image Understanding and Generation

    cs.CV 2026-07 conditional novelty 5.0 of 10

    A compact unified model that reuses a frozen VLM encoder and hybrid continuous/discrete tokens reaches competitive image understanding and generation with 15.6M training images and about $2,000 in compute.

  4. Bridging Information Asymmetry: A Hierarchical Framework for Deterministic Blind Face Restoration

    cs.CV 2026-01 conditional novelty 5.0 of 10

    Pref-Restore combines AR semantic tokens, a diffusion generator, and DiffusionNFT-style RL to make blind face restoration more consistent, but its deterministic-identity claim is weakened by self-referential rewards a...

  5. TBAC-UniImage: Unified Understanding and Generation by Ladder-Side Diffusion Tuning

    cs.CV 2025-08 conditional novelty 5.0 of 10

    A 3B model that injects intermediate MLLM query states into matching diffusion transformer layers reports competitive text-to-image and image-editing benchmark scores.

Pith tools