Pith. sign in

REVIEW 9 cited by

MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizer

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2401.10208 v2 pith:F2GY6RTN submitted 2024-01-18 cs.CV cs.CL

Changyao Tian , Xizhou Zhu , Yuwen Xiong , Weiyun Wang , Zhe Chen , Wenhai Wang , Yuntao Chen , Lewei Lu
show 5 more authors
Tong Lu Jie Zhou Hongsheng Li Yu Qiao Jifeng Dai
This is my paper
classification cs.CVcs.CL
keywords interleavedmm-interleavedimage-textgenerativemodelsmulti-modalvisualdata
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Developing generative models for interleaved image-text data has both research and practical value. It requires models to understand the interleaved sequences and subsequently generate images and text. However, existing attempts are limited by the issue that the fixed number of visual tokens cannot efficiently capture image details, which is particularly problematic in the multi-image scenarios. To address this, this paper presents MM-Interleaved, an end-to-end generative model for interleaved image-text data. It introduces a multi-scale and multi-image feature synchronizer module, allowing direct access to fine-grained image features in the previous context during the generation process. MM-Interleaved is end-to-end pre-trained on both paired and interleaved image-text corpora. It is further enhanced through a supervised fine-tuning phase, wherein the model improves its ability to follow complex multi-modal instructions. Experiments demonstrate the versatility of MM-Interleaved in recognizing visual details following multi-modal instructions and generating consistent images following both textual and visual conditions. Code and models are available at \url{https://github.com/OpenGVLab/MM-Interleaved}.

Discussion (0). Sign in to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. VIG-RL: Learning to Search and Insert for Verified Image Grounding

    cs.IR 2026-07 conditional novelty 6.0 of 10

    An RL-trained ReAct agent learns joint search–selection–insertion of retrieved authentic images, setting SOTA on MRAMG-Bench Verified Image Grounding.

  2. DynaVieW: Schema-Guided World Modeling for Understanding Hierarchical Visual Dynamics

    cs.LG 2026-07 accept novelty 6.0 of 10

    Schema-guided interleaved state-transition pretraining with selective attention and reweighted loss improves hierarchical visual dynamics modeling for narrative generation and world simulation.

  3. M2IO-R1: An Efficient RL-Enhanced Reasoning Framework for Multimodal Retrieval Augmented Multimodal Generation

    cs.IR 2025-08 conditional novelty 6.0 of 10

    M2IO-R1 uses GRPO reinforcement learning to train a 3B model that selects and places retrieved images into generated text answers, improving MRAMG quality on several benchmarks.

  4. GM-PRM: A Generative Multimodal Process Reward Model for Multimodal Mathematical Reasoning

    cs.CL 2025-08 conditional novelty 6.0 of 10

    A generative multimodal process reward model that produces step-level critiques and corrections improves average math accuracy for six multimodal LLMs by 2.9 to 5.9 points under a refinement-based Best-of-N strategy.

  5. Are Unified Vision-Language Models Necessary: Generalization Across Understanding and Generation

    cs.CV 2025-05 conditional novelty 6.0 of 10

    Unified vision-language models outperform task-specific models on understanding and generation, with benefits that grow with data and depend on alignment between vision input and output spaces.

  6. A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data

    cs.AI 2026-01 conditional novelty 5.0 of 10

    A metric-oriented survey that classifies intrinsic quality and trustworthiness metrics for LLM-generated data across six modalities and documents systematic evaluation gaps in the current literature.

  7. Docopilot: Improving Multimodal Models for Document-Level Understanding

    cs.CV 2025-07 conditional novelty 5.0 of 10

    A new academic-paper dataset and a retrieval-free fine-tuned InternVL2 model improve multi-page document QA accuracy and latency on several benchmarks.

  8. POVQA: Preference-Optimized Video Question Answering with Rationales for Data Efficiency

    cs.CV 2025-10 reject novelty 4.0 of 10

    POVQA reports large F1 gains on a new 239-example video QA dataset after rationale-based fine-tuning, but its own keyframe-only ablation matches the full pooling pipeline, and fine-tuning hurts zero-shot TVQA accuracy.

  9. Pisces: An Auto-regressive Foundation Model for Image Understanding and Generation

    cs.CV 2025-06 conditional novelty 4.0 of 10

    A unified image understanding and generation model with decoupled visual encoders achieves competitive benchmark scores on both tasks.

Pith tools