Pith. sign in

REVIEW 10 cited by

Nexus-Gen: Unified Image Understanding, Generation, and Editing via Prefilled Autoregression in Shared Embedding Space

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2504.21356 v3 pith:6E2K2UBH submitted 2025-04-30 cs.CV cs.AI

Hong Zhang , Zhongjie Duan , Xingjun Wang , Yuze Zhao , Weiyi Lu , Zhipeng Di , Yixuan Xu , Yingda Chen
show 1 more author
Yu Zhang
This is my paper
classification cs.CVcs.AI
keywords imageeditinggenerationmodelsnexus-genunderstandingautoregressiveembedding
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Unified multimodal generative models aim to integrate image understanding and generation abilities, offering significant advantages in harnessing multimodal corpora, particularly interleaved text-image data. However, existing unified models exhibit limitations in image synthesis quality, autoregressive error accumulation, and image editing capability. In this work, we propose Nexus-Gen, a novel architecture that unifies image understanding, generation, and editing tasks in a shared image embedding space. This shared space serves as a bridge for the autoregressive and diffusion models, which seamlessly integrates their complementary strengths in cross-modal modeling. To mitigate the severe error accumulation during autoregressive embedding prediction, we propose a novel prefilled autoregression strategy that aligns training-inference dynamics by prefilling input sequences with learnable embeddings. After multi-stage and multi-task training on our constructed large-scale dataset with 26.3 million samples, Nexus-Gen achieves state-of-the-art performance on the evaluation benchmarks spanning image understanding, generation and editing tasks. All models, datasets, and source codes are released in https://github.com/modelscope/Nexus-Gen to facilitate further advancements across the field.

Discussion (0). Sign in to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. UniGen-AR: Unifying Visual Generation with Auto-Regressive Modeling

    cs.CV 2026-07 conditional novelty 6.0 of 10

    An MLLM-conditioned next-scale VAR decoder handles 15+ unified visual generation tasks with competitive quality and substantially lower latency than diffusion baselines.

  2. Twins: Learn to Predict Unified Representations with Focal Loss

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Channel-wise concatenation of SigLIP2 and Flux VAE features into one token, trained with a focal-style flow-matching loss, yields a unified representation with 1.59 gFID on ImageNet 256 and VAE-level reconstruction.

  3. UniICL: Systematizing Unified Multimodal In-context Learning through a Capability-Oriented Taxonomy

    cs.CV 2026-03 conditional novelty 6.0 of 10

    A six-level capability taxonomy plus UniICL-760K and a lightweight CAPM module improve unified multimodal few-shot learning and beat larger MLLMs on most understanding ICL tasks.

  4. Reconstruction Alignment Improves Unified Multimodal Models

    cs.CV 2025-09 conditional novelty 6.0 of 10

    RECA, a self-supervised post-training objective that conditions unified multimodal models on their own visual understanding embeddings to reconstruct input images, improves text-to-image and editing benchmarks across ...

  5. ShareGPT-4o-Image: Aligning Multimodal Models with GPT-4o-Level Image Generation

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A 91K GPT-4o-generated image and editing dataset, and a fine-tuned open model Janus-4o, report improved text-to-image scores and new editing ability.

  6. Image Editing As Programs with Diffusion Models

    cs.CV 2025-06 conditional novelty 6.0 of 10

    IEAP decomposes complex editing instructions into atomic operations executed sequentially on a diffusion transformer, and reports state-of-the-art results on MagicBrush and AnyEdit.

  7. Physical Plausibility Reasoning via HCM-GRPO: Empowering Compact Model for Superior Performance

    cs.CV 2025-11 conditional novelty 5.0 of 10

    A 2B multimodal model trained with HCM-GRPO, a GRPO variant with partial-credit rewards and hard-case oversampling, outperforms larger models on the authors' private physical-plausibility test set.

  8. FramePrompt: In-context Controllable Animation with Zero Structural Changes

    cs.GR 2025-06 conditional novelty 5.0 of 10

    FramePrompt turns character animation into a video-continuation task by concatenating reference image, skeleton frames, and target frames into one sequence, then training the pretrained Wan-I2V model to generate only ...

  9. Draw ALL Your Imagine: A Holistic Benchmark and Agent Framework for Complex Instruction-based Image Generation

    cs.CV 2025-05 conditional novelty 5.0 of 10

    A new benchmark and agent framework for complex text-to-image generation, with an unvalidated AI-judge evaluation and claims that the agent outperforms GPT-4o on the authors' own benchmark.

  10. Instruction-based Image Editing: A Survey on Data, Models, Evaluation, and Applications

    cs.CV 2026-07 conditional novelty 4.0 of 10

    A survey of instruction-based image editing plus a new 21-task benchmark, CDD-IIE, on which ten open models are scored by human experts.

Pith tools