Pith. sign in

REVIEW 13 cited by

A Survey of Multimodal Large Language Model from A Data-centric Perspective

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2405.16640 v2 pith:3D6QE4XH submitted 2024-05-26 cs.AI cs.CLcs.CVcs.MM

Tianyi Bai , Hao Liang , Binwang Wan , Yanran Xu , Xi Li , Shiyu Li , Ling Yang , Bozhou Li
show 7 more authors
Yifan Wang Bin Cui Ping Huang Jiulong Shan Conghui He Binhang Yuan Wentao Zhang
This is my paper
classification cs.AIcs.CLcs.CVcs.MM
keywords mllmsdatalanguagelargemodelsmultimodalsurveydata-centric
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Multimodal large language models (MLLMs) enhance the capabilities of standard large language models by integrating and processing data from multiple modalities, including text, vision, audio, video, and 3D environments. Data plays a pivotal role in the development and refinement of these models. In this survey, we comprehensively review the literature on MLLMs from a data-centric perspective. Specifically, we explore methods for preparing multimodal data during the pretraining and adaptation phases of MLLMs. Additionally, we analyze the evaluation methods for the datasets and review the benchmarks for evaluating MLLMs. Our survey also outlines potential future research directions. This work aims to provide researchers with a detailed understanding of the data-driven aspects of MLLMs, fostering further exploration and innovation in this field.

Discussion (0). Sign in to comment.

Forward citations

Cited by 13 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. TraceAV-Bench: Benchmarking Multi-Hop Trajectory Reasoning over Long Audio-Visual Videos

    cs.CV 2026-05 unverdicted novelty 8.0 of 10

    TraceAV-Bench is the first benchmark for multi-hop trajectory reasoning over long audio-visual videos, showing top models reach only 51-68% accuracy with substantial room for improvement.

  2. Towards Multimodal Active Learning: Efficient Learning with Limited Paired Data

    cs.LG 2025-09 unverdicted novelty 7.0 of 10

    Introduces the first active learning framework for unaligned multimodal data that selects alignments using uncertainty and diversity to cut annotation costs by up to 40% on benchmarks while preserving accuracy.

  3. Agent Security Needs Redefinition through a Holistic Framework

    cs.CR 2026-07 conditional novelty 6.0 of 10

    Agent security should be redefined around four contextual authorization properties instead of the content of the action performed.

  4. GraspLLM: Towards Zero-Shot Generalization on Text-Attributed Graphs with LLMs

    cs.CL 2026-06 unverdicted novelty 6.0 of 10

    GraspLLM extracts dataset-agnostic structural patterns via motif contrastive learning and aligns contextual subgraphs to LLM tokens, outperforming prior LLM-based methods on TAGs especially in zero-shot settings.

  5. All in One: A Unified Synthetic Data Pipeline for Multimodal Video Understanding

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    A unified synthetic data generation pipeline produces unlimited annotated multimodal video data across multiple tasks, enabling models trained mostly on synthetic data to generalize effectively to real-world video und...

  6. Failure-Informed Image Self-Augmentation for Multimodal Large Language Model Self-Improvement

    cs.AI 2026-08 conditional novelty 5.0 of 10

    FISA improves MLLM visual question answering by generating and filtering augmented images based on the model's own failure cases.

  7. DecoupleMix: Decoupled Ratio Search and Convex Allocation for Scalable VLM Data Recipes

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Decoupling inter-class ratio search from intra-class convex allocation yields VLM data recipes that beat stacking and transfer from small proxies to larger scales.

  8. DataPrep-Bench: Benchmarking LLMs as Training Data Preparators

    cs.LG 2026-05 conditional novelty 5.0 of 10

    DataPrep-Bench jointly benchmarks data construction and data-quality evaluation for LLMs across six domains with downstream fine-tuning performance as ground truth.

  9. OpenWorldLib: A Unified Codebase and Definition of Advanced World Models

    cs.CV 2026-04 conditional novelty 4.0 of 10

    OpenWorldLib defines world models as perception-centered systems with interaction and long-term memory, and provides a modular inference codebase unifying interactive video, 3D, reasoning, and VLA tasks.

  10. OpenWorldLib: A Unified Codebase and Definition of Advanced World Models

    cs.CV 2026-04 unverdicted novelty 4.0 of 10

    OpenWorldLib offers a standardized codebase and definition for world models that combine perception, interaction, and memory to understand and predict the world.

  11. Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges

    cs.AI 2025-10 unverdicted novelty 4.0 of 10

    A survey that taxonomizes threats to agentic AI, reviews benchmarks and evaluation methods, discusses technical and governance defenses, and identifies open challenges.

  12. A Survey of Context Engineering for Large Language Models

    cs.CL 2025-07 accept novelty 4.0 of 10

    The survey organizes Context Engineering into retrieval, processing, management, and integrated systems like RAG and multi-agent setups while identifying an asymmetry where LLMs handle complex inputs well but struggle...

  13. Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning

    cs.CL 2025-02 unverdicted novelty 2.0 of 10

    Position paper claims multimodal LLMs can significantly advance scientific reasoning and proposes a four-stage roadmap plus challenges and suggestions.

Pith tools