Pith. sign in

REVIEW 27 cited by

A Survey of Multimodal Retrieval-Augmented Generation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2504.08748 v1 pith:ULEQCEAK submitted 2025-03-26 cs.IR cs.AIcs.CLcs.ETcs.LG

Lang Mei , Siyu Mo , Zhihan Yang , Chong Chen This is my paper
classification cs.IRcs.AIcs.CLcs.ETcs.LG
keywords generationmultimodalmragretrievalretrieval-augmenteddataenhancesinformation
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Multimodal Retrieval-Augmented Generation (MRAG) enhances large language models (LLMs) by integrating multimodal data (text, images, videos) into retrieval and generation processes, overcoming the limitations of text-only Retrieval-Augmented Generation (RAG). While RAG improves response accuracy by incorporating external textual knowledge, MRAG extends this framework to include multimodal retrieval and generation, leveraging contextual information from diverse data types. This approach reduces hallucinations and enhances question-answering systems by grounding responses in factual, multimodal knowledge. Recent studies show MRAG outperforms traditional RAG, especially in scenarios requiring both visual and textual understanding. This survey reviews MRAG's essential components, datasets, evaluation methods, and limitations, providing insights into its construction and improvement. It also identifies challenges and future research directions, highlighting MRAG's potential to revolutionize multimodal information retrieval and generation. By offering a comprehensive perspective, this work encourages further exploration into this promising paradigm.

Discussion (0). Sign in to comment.

Forward citations

Cited by 27 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Knowledge Poisoning Attacks on Medical Multi-Modal Retrieval-Augmented Generation

    cs.CR 2026-05 unverdicted novelty 8.0 of 10

    M³Att poisons medical multimodal RAG by pairing covert textual misinformation with query-agnostic visual perturbations that increase retrieval of the bad content, causing LLMs to generate clinically plausible but inco...

  2. MKG-RAG-Bench: Benchmarking Retrieval in Multimodal Knowledge Graph-Augmented Generation

    cs.AI 2026-06 unverdicted novelty 7.0 of 10

    MKG-RAG-Bench is a cross-domain benchmark for retrieval in multimodal knowledge graph-augmented generation, constructed via LLM curation from two MKGs with aligned QA datasets.

  3. Utility-Oriented Visual Evidence Selection for Multimodal Retrieval-Augmented Generation

    cs.CL 2026-05 unverdicted novelty 7.0 of 10

    Evidence utility is defined as information gain on the model's output distribution, with ranking by gain on a latent helpfulness variable shown equivalent to answer-space utility under mild assumptions, enabling a tra...

  4. Very Efficient Listwise Multimodal Reranking for Long Documents

    cs.IR 2026-05 unverdicted novelty 7.0 of 10

    ZipRerank delivers state-of-the-art multimodal listwise reranking accuracy for long documents at up to 10x lower latency via early interaction and single-pass scoring.

  5. Beyond Bag-of-Patches: Learning Global Layout via Textual Supervision for Late-Interaction Visual Document Retrieval

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    A text-supervised global layout embedding augments local patch representations in late-interaction VDR, yielding +2.4 nDCG@5 and +2.3 MAP@5 gains over ColPali/ColQwen baselines on ViDoRe-v2.

  6. Purifying Multimodal Retrieval: Fragment-Level Evidence Selection for RAG

    cs.IR 2026-04 unverdicted novelty 7.0 of 10

    FES-RAG reframes multimodal RAG as fragment-level selection using Fragment Information Gain to outperform document-level methods with up to 27% relative CIDEr gains on M2RAG while shortening context.

  7. Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Framework

    cs.CL 2026-02 unverdicted novelty 7.0 of 10

    Prune-then-Merge combines adaptive pruning of low-signal patches with hierarchical merging to achieve higher compression rates and better performance than prior single-stage methods in visual document retrieval.

  8. DualG-MRAG: Decoupling Macro-Reasoning and Micro-Matching for Multimodal Retrieval-Augmented Generation

    cs.AI 2026-07 conditional novelty 6.0 of 10

    Decoupling macro topological routing from micro visual matching, plus query-driven GNN path decoding, improves multimodal multi-hop retrieval and QA over strong MM-RAG baselines.

  9. M2Note: Continual Evolution of Vision Language Models via Mistake Notebook Learning

    cs.MA 2026-07 unverdicted novelty 6.0 of 10

    M2Note stores failed VLM trajectories as subject-guidance notes in an external notebook and retrieves them via multimodal RAG to avoid past errors during inference.

  10. Invoice Haystack: Benchmarking Document Retrieval and Visual Question Answering Under Strong Visual Homogeneity

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    New Invoice Haystack benchmark exposes embedding collapse under high visual homogeneity (0.73 cosine similarity) and VL-RAG hybrid retrieval reaches 60% Recall@1, beating prior methods by 13.5 points on the new set an...

  11. Invoice Haystack: Benchmarking Document Retrieval and Visual Question Answering Under Strong Visual Homogeneity

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    Presents Invoice Haystack benchmark for homogeneous document retrieval and VL-RAG hybrid framework achieving 60% Recall@1 and up to 13.5 point gains over prior methods.

  12. MM-BizRAG: Rethinking Multimodal Retrieval-Augmented Generation for General Purpose Enterprise Q&A

    cs.CL 2026-06 unverdicted novelty 6.0 of 10

    MM-BizRAG applies layout-aware document splitting and decoupled multimodal assembly to raise generative recall on enterprise Q&A tasks by up to 32 points over vision-centric baselines while adding FastRAGEval as a che...

  13. Navigating the Emotion Tree: Hierarchical Hyperbolic RAG for Multimodal Emotion Recognition

    cs.LG 2026-05 unverdicted novelty 6.0 of 10

    HyperEmo-RAG uses hierarchical hyperbolic embeddings and graph-based evidence injection to outperform prior methods in multimodal emotion recognition.

  14. Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    PVM adds a parallel branch to LVLMs that directly supplies visual embeddings to prevent attention decay over long generated sequences, yielding accuracy gains on reasoning tasks with minimal overhead.

  15. MEG-RAG: Quantifying Multi-modal Evidence Grounding for Evidence Selection in RAG

    cs.CL 2026-04 unverdicted novelty 6.0 of 10

    MEG-RAG defines a new MEG metric based on Semantic Certainty Anchoring and trains a multimodal reranker to select evidence aligned with ground-truth semantic anchors, yielding higher accuracy and consistency on the M²...

  16. Towards Long-horizon Agentic Multimodal Search

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    LMM-Searcher uses file-based visual UIDs and a fetch tool plus 12K synthesized trajectories to fine-tune a multimodal agent that scales to 100-turn horizons and reaches SOTA among open-source models on MM-BrowseComp a...

  17. M2IO-R1: An Efficient RL-Enhanced Reasoning Framework for Multimodal Retrieval Augmented Multimodal Generation

    cs.IR 2025-08 conditional novelty 6.0 of 10

    M2IO-R1 uses GRPO reinforcement learning to train a 3B model that selects and places retrieved images into generated text answers, improving MRAMG quality on several benchmarks.

  18. Mixture-of-Retrieval Experts for Reasoning-Guided Multimodal Knowledge Exploitation

    cs.CL 2025-05 unverdicted novelty 6.0 of 10

    MoRE enables MLLMs to dynamically coordinate heterogeneous retrieval experts via Step-GRPO training, yielding over 7% average gains on open-domain QA benchmarks.

  19. Evidence-Grounded Multimodal Knowledge Graph Construction for Multi-Lecture Educational Reasoning

    cs.AI 2026-08 conditional novelty 5.0 of 10

    A pipeline that transcribes lecture videos, reads slides and diagrams, and builds an evidence-linked knowledge graph, tested on three neural-network lectures with a three-question sanity check.

  20. SearchArt: Training Long-Horizon Search Agent with Scalable Synthetic and Verified Task

    cs.IR 2026-07 conditional novelty 5.0 of 10

    SearchArt post-trains Qwen3.5-27B on verification-filtered synthetic search trajectories, scoring 74.39 on BrowseComp-ZH, 70.06 on BrowseComp, and 52.55 on DeepResearch-Bench, competitive with several 200B-700B agents.

  21. Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    PVM adds a parallel learnable branch to LVLMs that supplies visual embeddings on demand to structurally prevent attention decay and visual signal dilution during deep autoregressive generation.

  22. AffectAgent: Collaborative Multi-Agent Reasoning for Retrieval-Augmented Multimodal Emotion Recognition

    cs.CV 2026-04 unverdicted novelty 5.0 of 10

    AffectAgent deploys a query planner, evidence filter, and emotion generator as collaborative agents trained via MAPPO with shared reward, plus MB-MoE and RAAF modules, to achieve superior multimodal emotion recognitio...

  23. UniRank: End-to-End Domain-Specific Reranking of Hybrid Text-Image Candidates

    cs.IR 2026-02 conditional novelty 5.0 of 10

    UniRank natively scores hybrid text-image candidates with a VLM, then SFT plus hard-negative RLHF adapts it to a domain; it outperforms baselines on scientific literature and design patent retrieval.

  24. MMGraphRAG: Bridging Vision and Language with Interpretable Multimodal Knowledge Graphs

    cs.AI 2025-07 conditional novelty 5.0 of 10

    MMGraphRAG links scene-graph entities from images to text knowledge graph entities via SpecLink, and reports accuracy gains over naive RAG and GraphRAG on multimodal document QA.

  25. Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Ranking

    cs.CL 2026-01 unverdicted novelty 4.0 of 10

    Qwen3-VL-Embedding-8B achieves state-of-the-art performance with a 77.8 overall score on the MMEB-V2 multimodal embedding benchmark.

  26. Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding

    cs.CL 2025-10 accept novelty 4.0 of 10

    A systematic survey of Multimodal RAG for document understanding proposing a taxonomy based on domain, retrieval modality, and granularity while reviewing graph structures, agentic frameworks, datasets, benchmarks, ap...

  27. Automotive Engineering-Centric Agentic AI Workflow Framework

    cs.AI 2026-04 unverdicted novelty 3.0 of 10

    The paper presents the Agentic Engineering Intelligence (AEI) framework for modeling automotive engineering workflows as sequential decision processes with AI agent support.

Pith tools