SUPERGLASSES is the first VQA benchmark built from actual smart glasses data, and SUPERLENS is an agent using automatic object detection, query decoupling, and multimodal search that outperforms GPT-4o by 2.19% on it.
mKG-RAG: Leveraging Multimodal Knowledge Graphs in Retrieval-Augmented Generation for Knowledge-intensive VQA
6 Pith papers cite this work. Polarity classification is still indexing.
6
Pith papers citing it
abstract
Retrieval-Augmented Generation (RAG) has emerged as an effective paradigm for expanding the knowledge capacity of Multimodal Large Language Models (MLLMs) by incorporating external knowledge sources into the generation process, and has been widely adopted for knowledge-based Visual Question Answering (VQA). Despite impressive advancements, vanilla RAG-based VQA methods that rely on unstructured documents and overlook the structural relations among knowledge elements frequently introduce irrelevant or misleading content, degrading answer accuracy and reliability. To overcome these challenges, a promising solution is to integrate multimodal knowledge graphs (KGs) into RAG-based VQA frameworks, thereby enhancing generation through structured multimodal knowledge. To this end, this paper proposes mKG-RAG, a novel retrieval-augmented generation framework built upon multimodal KGs for knowledge-intensive VQA tasks. Specifically, mKG-RAG leverages MLLM-driven graph extraction and vision-text matching to distill semantically consistent, modality-complementary entities and relations from multimodal documents, constructing high-quality multimodal KGs as structured knowledge representations. Furthermore, a dual-stage retrieval strategy equipped with a query-aware multimodal retriever is introduced to improve retrieval efficiency while progressively refining precision. Comprehensive experiments demonstrate that our approach significantly outperforms existing approaches and sets new state-of-the-art results for knowledge-based VQA. The code is available at https://github.com/xandery-geek/mKG-RAG.
citation-role summary
baseline 1
citation-polarity summary
years
2026 6roles
baseline 1polarities
baseline 1representative citing papers
An agentic mRAG framework uses GRPO-trained visual reranking and active rejection to verify retrieved candidate entities, achieving state-of-the-art on three KB-VQA benchmarks.
EviProp improves evidence-page retrieval in long multimodal documents by combining dense visual priors with sparse chunk seeds and running Personalized PageRank on a multimodal Chunk-Page graph.
A lightweight hierarchical multimodal graph RAG that fuses entity-grounded visual objects with text nodes and propagates relevance via multi-granularity PPR, delivering SOTA multimodal task performance at far lower construction cost.
QKVQA proposes a question-focused filtering method with QFF and CDA modules that boosts accuracy by 3.2 points on Encyclopedic-VQA and 2.2 points on InfoSeek over prior state-of-the-art.
AIR framework achieves ~400x faster LLM-based cross-domain recommendation via offline intent construction and online retrieval, with SOTA results on public data and +3.446% GMV lift in live A/B tests.
citing papers explorer
-
SUPERGLASSES: Benchmarking Vision Language Models as Intelligent Agents for AI Smart Glasses
SUPERGLASSES is the first VQA benchmark built from actual smart glasses data, and SUPERLENS is an agent using automatic object detection, query decoupling, and multimodal search that outperforms GPT-4o by 2.19% on it.
-
MMAgent-R$^2$: Learning to Rerank and Reject for Agentic mRAG
An agentic mRAG framework uses GRPO-trained visual reranking and active rejection to verify retrieved candidate entities, achieving state-of-the-art on three KB-VQA benchmarks.
-
EviProp: Seeded Relevance Diffusion on Chunk-Page Graphs for Long Multimodal Document Retrieval
EviProp improves evidence-page retrieval in long multimodal documents by combining dense visual priors with sparse chunk seeds and running Personalized PageRank on a multimodal Chunk-Page graph.
-
MG$^2$-RAG: Multi-Granularity Graph for Multimodal Retrieval-Augmented Generation
A lightweight hierarchical multimodal graph RAG that fuses entity-grounded visual objects with text nodes and propagates relevance via multi-granularity PPR, delivering SOTA multimodal task performance at far lower construction cost.
-
QKVQA: Question-Focused Filtering for Knowledge-based VQA
QKVQA proposes a question-focused filtering method with QFF and CDA modules that boosts accuracy by 3.2 points on Encyclopedic-VQA and 2.2 points on InfoSeek over prior state-of-the-art.
-
Atomic Intent Reasoning: Bringing LLM Semantics to Industrial Cross-Domain Recommendations
AIR framework achieves ~400x faster LLM-based cross-domain recommendation via offline intent construction and online retrieval, with SOTA results on public data and +3.446% GMV lift in live A/B tests.