Pith. sign in

REVIEW 3 major objections 5 minor 48 references

Decoupling global graph routing from local visual matching lets multimodal RAG recover multi-hop evidence without drowning in noise.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 03:10 UTC pith:MJH6IQT3

load-bearing objection Solid systems paper on the multimodal graph noise/granularity tradeoff; gains look real, but the causal credit for dual-tier+GNN vs a stronger shared extractor is not fully locked down. the 3 major comments →

arxiv 2607.28580 v1 pith:MJH6IQT3 submitted 2026-07-30 cs.AI

DualG-MRAG: Decoupling Macro-Reasoning and Micro-Matching for Multimodal Retrieval-Augmented Generation

Jiacheng Tao , Qingyun Sun , Haonan Yuan , Ziwei Zhang , Jianxin Li This is my paper
classification cs.AI
keywords Multimodal RAGGraph Neural NetworkMulti-hop reasoningKnowledge graphPath decodingMacro-micro decouplingRetrieval-augmented generation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Multimodal retrieval-augmented generation still fails on questions that need several hops across text, tables, and images. Vector matching treats each item in isolation, and ordinary graph methods either explode when they add fine visual detail or lose that detail when they stay coarse. DualG-MRAG splits the problem into two graphs: a Macro Graph that only routes among global entities and relations, and a Micro Graph that keeps fine local facts inside documents and images. A query first activates the Micro Graph, seeds a Graph Neural Network on the Macro Graph, and lets relevance flow only from those seeds. The same forward pass is then decoded by dynamic programming into an explicit chain of entities and relations that is handed to the generative model instead of a flat list of chunks. On multi-hop multimodal benchmarks the design raises both evidence recall and final answer accuracy over strong vector and graph baselines.

Core claim

Isolating global topological routing (Macro Graph) from fine-grained local verification (Micro Graph), then running query-driven GNN message passing and endogenous path decoding on that dual structure, recovers multi-hop multimodal evidence more accurately than either flat vector matching or unified multimodal graphs that mix scales.

What carries the argument

The dual-tier Multimodal Knowledge Graph plus query-driven GNN retriever: Micro Graph matching supplies the seed mask and visual verification scores; Macro Graph message passing (NBFNet-style) propagates relevance; layer-wise dynamic programming reads the local flow probabilities to extract explicit reasoning paths for the generator.

Load-bearing premise

The offline extraction of entities, relations, and image captions, plus simple similarity linking, produces a dual graph faithful enough that message passing on it recovers the true multi-hop evidence chain.

What would settle it

On a held-out multi-hop multimodal set, measure whether the decoded Macro paths still contain the gold supporting documents and images when the OpenIE/caption quality is deliberately degraded or the entity-linking threshold is swept; if recall and QA accuracy collapse while flat vector baselines stay flat, the construction assumption fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Multimodal RAG systems can keep fine visual detail without paying the full cost of a giant unified graph.
  • Downstream generators receive explicit cross-document reasoning chains rather than unordered chunks, lowering their implicit multi-hop burden.
  • Query-conditioned message passing replaces static graph walk rules, so the same topology can serve many different intents.
  • Evidence recall and end-to-end QA accuracy both rise on multi-hop multimodal benchmarks relative to strong vector and graph baselines.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same macro/micro split could be applied to other heterogeneous corpora (code + docs, sensor streams + logs) where fine local features and global topology conflict.
  • Because paths are recovered from the GNN forward pass itself, the method offers a natural audit trail for why a particular multimodal answer was produced.
  • If construction quality is the bottleneck, improving OpenIE or captioning alone may yield larger gains than further GNN engineering.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes DualG-MRAG, a multimodal RAG system that decouples a Macro Reasoning Graph (global entities/relations from OpenIE plus VLM image captions, with equivalence edges) from Micro Matching Graphs (fine-grained intra-document 4-tuples with document pointers). Retrieval is cast as query-driven message passing with an NBFNet-style GNN initialized from Micro-matched and NER seeds, with visual scores modulated by micro-matching; a layer-wise DP decoder then extracts reasoning paths from local flow probabilities to serialize structured evidence for the MLLM. Experiments on MMQA, WebQA (1k validation subsets), and ScienceQA report gains over vector RAG and graph-enhanced baselines in EM/F1/ROUGE/BERTScore, Top-K recall, and efficiency, with ablations of Macro, Micro, and path injection.

Significance. If the gains are truly driven by the dual-tier decoupling and query-conditioned GNN/path stack—not by a stronger shared extraction front-end—the work offers a concrete architectural answer to a real multimodal GraphRAG tension (fine visual detail vs. graph blow-up and noise) and a practical path to sub-second structured retrieval with explicit chains for the generator. The endogenous DP path extraction from GNN flows, the micro-modulated cross-modal score fusion, and the efficiency contrast to prior graph RAG (~0.44s vs. tens of seconds) are useful contributions for MM multi-hop QA. The multi-benchmark tables, recall curves, latency scatter, and component ablations give a solid empirical skeleton once causal attribution and evaluation variance are tightened.

major comments (3)
  1. [§4.1, §5.2–5.6, Table 3, C.1] Central attribution is under-controlled. §4.1 and C.1 build Macro/Micro graphs with Qwen3-VL-8B OpenIE and fine-grained visual parsing on every document; Table 1/Fig. 4 credit large lifts (e.g., MMQA EM 44.2 vs. 37.2; R@5 61.9 vs. 42.1) to dual-tier + GNN + path decoding. Ablations in Table 3 only strip Macro/Micro/Path inside DualG-MRAG. There is no control that freezes the same triples/captions/pointers and swaps only the dual-tier+GNN+path stack against a flat dense index or a single unified graph over that identical extraction. Without that, gains may partly reflect denser/cleaner upstream extraction rather than the architectural thesis. Please add same-extraction baselines (or document that MMGraphRAG/HM-RAG used the identical extractor and triple density) and report those numbers.
  2. [§5.1, Table 1, Figure 4] MMQA and WebQA results rest on a single random 1,000-query draw from each validation set (§5.1) with no seeds, confidence intervals, or significance tests, while claiming clear superiority in Table 1 and Fig. 4. For load-bearing multi-hop claims, report mean±std over multiple splits/seeds (or the full official splits where feasible) and a simple paired test vs. the strongest baseline under the same generator prompt. ScienceQA’s train-as-KB setup should also be stress-checked for leakage relative to zero-shot Qwen3-VL-8B.
  3. [§4.4, §5.6, Table 3] Path injection’s necessity is only partially supported. Table 3: w/o Path leaves retrieval unchanged (as expected) but on Qwen3-VL-8B raises MMQA EM slightly (46.0→46.5) while hurting WebQA R-L/BERTScore; the 4B backbone drops. §5.6’s “over-constrain” note is plausible but leaves Challenge III and the DP decoder (§4.4, Eqs. 10–11) as optional formatting rather than a robust structural gain. Add path-quality metrics (e.g., hop fidelity vs. gold multi-hop chains, faithfulness of serialized paths) and/or a condition under which path injection is applied, with results stratified by hop count and model size.
minor comments (5)
  1. [Appendix C.1, §4.2–4.4] Many free parameters (τ=0.8, L=6, α/β, unverified decay 0.25, subgraph Top-K, τ_flow, kv(q)) appear in C.1 with little sensitivity analysis. A short sweep or stability plot for τ, L, and α/β would strengthen reproducibility.
  2. [§4.2, Appendix A.1] §4.2 acknowledges approximate Top-K + branch-and-bound micro matching but never reports match precision/recall vs. exact isomorphism on a small labeled subset; even a small diagnostic would ground the ‘sufficient surrogate’ claim in A.1.
  3. [§3, §4.1] Notation: T_M / T_m are called graphs but defined via triple sets; E_M vs V in Preliminaries is slightly inconsistent. Unify graph vs. triple-set notation early.
  4. [Figure 2, Figure 5] Figure 2 panel labels (A Micro matching vs. B Macro GNN) are clear; Figure 5 log-time axis would benefit from explicit numeric latency labels next to each method for readability in print.
  5. [Appendix C.2, §5.1] C.2 notes special-case prompting (complexity-gated path bypass on WebQA; dual candidate with/without retrieval on ScienceQA). State these clearly in the main experimental protocol so baseline comparisons remain interpretable.

Circularity Check

0 steps flagged

No derivation circularity: empirical architecture paper evaluated on external QA/retrieval benchmarks; results are not forced by definition or self-citation.

full rationale

DualG-MRAG is a systems/methods paper. Its load-bearing claims are empirical (higher EM/F1/R@K vs VisRAG, VLM2Vec-V2.0, CoRe-MMRAG, ViDoRAG, HM-RAG, MMGraphRAG on MMQA/WebQA/ScienceQA) and architectural (Macro/Micro decoupling, query-driven NBFNet-style message passing, DP path decoding from GNN flows). None of these reduce by construction to fitted identities or to author-only uniqueness theorems. Graph construction (OpenIE + VLM captions, τ=0.8 equivalence, ColBERT linking) and GNN scoring are design choices evaluated by ablations and held-out metrics, not predictions that equal their inputs. Self-citations (e.g., related GNN-RAG work by overlapping authors) appear in Related Work and are not used to forbid alternatives or force the reported lifts. Experimental confounds about shared extractors vs baselines are fairness/correctness concerns, not circularity of a derivation chain. Score 0; steps empty.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 3 invented entities

Load-bearing content is architectural and empirical, not theorematic. The claim rests on construction choices (how graphs are built and linked), hand-chosen fusion/retrieval hyperparameters, and the assumption that GNN flows on the constructed Macro graph correspond to human multi-hop evidence paths. No new physical entities; invented objects are system modules.

free parameters (5)
  • Entity resolution threshold τ = 0.8
    Cosine similarity cutoff for adding Macro equivalence edges; directly shapes connectivity used by the GNN.
  • GNN depth L and hidden size = L=6, hidden=512
    Controls hop horizon and capacity of query-driven message passing; chosen, not derived.
  • Visual fusion weights α, β and unverified decay = α=0.1, β=0.9, decay=0.25
    Hand weights that modulate visual document scores by micro-match evidence; strongly affect final ranking.
  • Subgraph search Top-K and visual budget kv(q) = Top-K=3; kv query-dependent
    Limits micro matching candidates and how many raw images reach the MLLM; procedural knobs.
  • Flow temperature τ_flow = 1.0
    Softmax temperature in local flow probabilities for path DP decoding.
axioms (5)
  • domain assumption OpenIE triples plus VLM image captions adequately represent cross-document multimodal relations for multi-hop QA routing.
    Macro graph construction (§4.1.1) treats extracted triples and captions as the structural backbone without independent completeness guarantees.
  • domain assumption Fine-grained visual structure can be stored as textual micro-triples (u,r,v,d) without losing evidence needed at generation time, because raw images are reattached via pointer d.
    Stated in §4.1.2; generation still sees raw pixels, but retrieval decisions depend on the textualized micro graph.
  • ad hoc to paper Approximate Top-K + branch-and-bound subgraph matching is a sufficient surrogate for exact micro-graph isomorphism at QA latency.
    §4.2.2 explicitly replaces NP-hard exact matching with heuristics that can miss valid mappings.
  • domain assumption Query-initialized NBFNet-style message passing with DistMult messages yields document relevance that correlates with multi-hop answerability.
    §4.3 adopts this GNN family as the retriever; scoring MLP and max fusion are design choices validated only empirically.
  • standard math Standard dense retrieval and MLLM generation formalisms (shared encoders, autoregressive P(A|q,D_ret)).
    Preliminaries §3; background not re-proved.
invented entities (3)
  • Dual-tier MMKG (Macro Reasoning Graph T_M + Micro Matching Graph T_m) no independent evidence
    purpose: Isolate global topological routing from local visual/table verification to limit noise vs information loss.
    Core architectural object introduced in §4.1; existence is definitional to the method, not an external natural kind.
  • Query-driven GNN retriever with micro-modulated multimodal score fusion no independent evidence
    purpose: Propagate relevance from query seeds on Macro graph and rank documents with visual noise control.
    §4.3; builds on NBFNet but specialized fusion rules are paper-specific.
  • Endogenous DP path decoder over GNN local flows no independent evidence
    purpose: Serialize multi-hop reasoning paths from the GNN forward pass for MLLM prompting.
    §4.4; alternative to flat top-k chunks; validated only by ablation on generation metrics.

pith-pipeline@v1.2.0-daily-grok45 · 22563 in / 3808 out tokens · 71997 ms · 2026-07-31T03:10:14.467944+00:00 · methodology

0 comments
read the original abstract

While Multimodal Retrieval-Augmented Generation (MM-RAG) has shown promising results, it still struggles with complex multi-hop reasoning tasks. Existing methods primarily focus on independent instance-level matching, which often fails to capture explicit relationships across modalities and documents. Although Graph-enhanced methods introduce structural modeling, they face a fundamental challenge in multimodal scenarios: incorporating fine-grained visual features leads to rapid graph expansion and retrieval noise, whereas coarse-grained representations cause the discarding of critical local evidence. To address this dilemma, we propose DualG-MRAG, a Dual-tier framework that introduces a decoupled architecture comprising Macro-reasoning and Micro-matching Graphs for Multimodal RAG. Specifically, to suppress retrieval noise by isolating global structural reasoning from fine-grained evidence matching, we construct a Macro Graph for global topological routing and a Micro Graph for precise local verification. Subsequently, to enable dynamic relevance propagation across heterogeneous evidence sources, we formulate retrieval as a query-driven message passing process via a GNN Retriever. Furthermore, to provide the generative model with coherent structural guidance, we introduce a dynamic programming decoding mechanism that extracts explicit reasoning paths directly from the GNN's forward pass, replacing the standard input of isolated document chunks. Extensive experiments demonstrate that DualG-MRAG outperforms baselines in both evidence recall and complex QA accuracy.

Figures

Figures reproduced from arXiv: 2607.28580 by Haonan Yuan, Jiacheng Tao, Jianxin Li, Qingyun Sun, Ziwei Zhang.

Figure 1
Figure 1. Figure 1: Comparison of existing graph-enhanced MRAG and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The overall architecture of DualG-MRAG. The framework operates in three query-driven phases. (A) Dual-Tier [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: ScienceQA fine-grained performance. Effectiveness of Macro-Level Reasoning. Traditional multi￾modal RAG methods are highly dependent on isolated instance-level feature matching, which often limits their capability to integrate cross-document evidence when handling complex multi-hop rea￾soning tasks. In contrast, DualG-MRAG achieves superior perfor￾mance on the MMQA dataset, achieving an EM score of 44.20% … view at source ↗
Figure 5
Figure 5. Figure 5: Efficiency vs. Effectiveness [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

48 extracted references · 21 linked inside Pith

  1. [1]

    Mohammad Mahdi Abootorabi, Amirhosein Zobeiri, Mahdi Dehghani, Moham- madali Mohammadkhani, Bardia Mohammadi, Omid Ghahroodi, Mahdieh Soley- mani Baghshah, and Ehsaneddin Asgari. 2025. Ask in any modality: A compre- hensive survey on multimodal retrieval-augmented generation.Findings of ACL (2025), 16776–16809

    2025

  2. [2]

    Gabor Angeli, Melvin Jose Johnson Premkumar, and Christopher D Manning

  3. [3]

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al . 2025. Qwen3-vl technical report.arXiv preprint arXiv:2511.21631(2025)

    Pith/arXiv arXiv 2025

  4. [4]

    Yuzheng Cai, Zhenyue Guo, Yiwen Pei, Wanrui Bian, and Weiguo Zheng. 2025. SimGRAG: Leveraging similar subgraphs for knowledge graphs driven retrieval- augmented generation. InFindings of ACL. 3139–3158

    2025

  5. [5]

    Yingshan Chang, Mridu Narang, Hisami Suzuki, Guihong Cao, Jianfeng Gao, and Yonatan Bisk. 2022. Webqa: Multihop and multimodal qa. InCVPR. 16495–16504

    2022

  6. [6]

    Boyu Chen, Zirui Guo, Zidan Yang, Yuluo Chen, Junze Chen, Zhenghao Liu, Chuan Shi, and Cheng Yang. 2026. Pathrag: Pruning graph-based retrieval augmented generation with relational paths. InAAAI, Vol. 40. 30183–30191

    2026

  7. [7]

    Wenhu Chen, Hexiang Hu, Xi Chen, Pat Verga, and William Cohen. 2022. Murag: Multimodal retrieval-augmented generator for open question answering over images and text. InEMNLP. 5558–5570

    2022

  8. [8]

    Zhanpeng Chen, Chengjin Xu, Yiyan Qi, and Jian Guo. 2024. Mllm is a strong reranker: Advancing multimodal retrieval-augmented generation via knowledge- enhanced reranking and noise-injected training.arXiv preprint arXiv:2407.21439 (2024)

    Pith/arXiv arXiv 2024

  9. [9]

    Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. 2024. From local to global: A graph rag approach to query-focused summarization.arXiv preprint arXiv:2404.16130(2024)

    Pith/arXiv arXiv 2024

  10. [10]

    Manuel Faysse, Hugues Sibille, Tony Wu, Bilel Omrani, Gautier Viaud, Céline Hudelot, and Pierre Colombo. 2024. Colpali: Efficient document retrieval with vision language models.arXiv preprint arXiv:2407.01449(2024)

    Pith/arXiv arXiv 2024

  11. [11]

    Mikhail Galkin, Xinyu Yuan, Hesham Mostafa, Jian Tang, and Zhaocheng Zhu

  12. [12]

    Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. 2017. Neural message passing for quantum chemistry. InICML. Pmlr, 1263–1272

    2017

  13. [13]

    Zirui Guo, Xubin Ren, Lingrui Xu, Jiahao Zhang, and Chao Huang. 2025. Rag- anything: All-in-one rag framework.arXiv preprint arXiv:2510.12323(2025)

    arXiv 2025

  14. [14]

    Zirui Guo, Lianghao Xia, Yanhua Yu, Tian Ao, and Chao Huang. 2024. Lightrag: Simple and fast retrieval-augmented generation.arXiv preprint arXiv:2410.05779 2, 3 (2024)

    Pith/arXiv arXiv 2024

  15. [15]

    Bernal J Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su. 2024. Hipporag: Neurobiologically inspired long-term memory for large language models.Advances in neural information processing systems37 (2024), 59532– 59569

    2024

  16. [16]

    Bernal Jiménez Gutiérrez, Yiheng Shu, Weijian Qi, Sizhe Zhou, and Yu Su. 2025. From rag to memory: Non-parametric continual learning for large language models.arXiv preprint arXiv:2502.14802(2025)

    Pith/arXiv arXiv 2025

  17. [17]

    Xiaoxin He, Yijun Tian, Yifei Sun, Nitesh V Chawla, Thomas Laurent, Yann LeCun, Xavier Bresson, and Bryan Hooi. 2024. G-retriever: Retrieval-augmented generation for textual graph understanding and question answering.NeurIPS37 (2024), 132876–132907

    2024

  18. [18]

    Ziyan Jiang, Rui Meng, Xinyi Yang, Semih Yavuz, Yingbo Zhou, and Wenhu Chen. 2024. Vlm2vec: Training vision-language models for massive multimodal embedding tasks.arXiv preprint arXiv:2410.05160(2024)

    Pith/arXiv arXiv 2024

  19. [19]

    Imed Keraghel, Stanislas Morbieu, and Mohamed Nadif. 2024. Recent advances in named entity recognition: A comprehensive survey and comparative study. arXiv preprint arXiv:2401.10825(2024)

    Pith/arXiv arXiv 2024

  20. [20]

    Omar Khattab and Matei Zaharia. 2020. Colbert: Efficient and effective passage search via contextualized late interaction over bert. InSIGIR. 39–48

    2020

  21. [21]

    Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks.arXiv preprint arXiv:1609.02907(2016)

    Pith/arXiv arXiv 2016

  22. [22]

    Xuan Li, Yining Wang, Haocai Luo, Shengping Liu, Jerry Liang, Ying Fu, Jun Yu, Junnan Zhu, et al. 2026. BayesRAG: Probabilistic Mutual Evidence Corroboration for Multimodal Retrieval-Augmented Generation.arXiv preprint arXiv:2601.07329 (2026)

    arXiv 2026

  23. [23]

    Zihan Ling, Zhiyao Guo, Yixuan Huang, Yi An, Shuai Xiao, Jinsong Lan, Xiaoyong Zhu, and Bo Zheng. 2025. Mmkb-rag: A multi-modal knowledge-based retrieval- augmented generation framework.arXiv preprint arXiv:2504.10074(2025)

    Pith/arXiv arXiv 2025

  24. [24]

    Junming Liu, Siyuan Meng, Yanting Gao, Song Mao, Pinlong Cai, Guohang Yan, Yirong Chen, Zilin Bian, Ding Wang, and Botian Shi. 2025. Aligning vision to language: Annotation-free multimodal knowledge graph construction for enhanced llms reasoning. InCVPR. 981–992

    2025

  25. [25]

    Pei Liu, Xin Liu, Ruoyu Yao, Junming Liu, Siyuan Meng, Ding Wang, and Jun Ma. 2025. Hm-rag: Hierarchical multi-agent multimodal retrieval augmented generation. InMM. 2781–2790

    2025

  26. [26]

    Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. 2022. Learn to explain: Multimodal reasoning via thought chains for science question answering.NeurIPS 35 (2022), 2507–2521

    2022

  27. [27]

    Linhao Luo, Zicheng Zhao, Gholamreza Haffari, Dinh Phung, Chen Gong, and Shirui Pan. 2025. GFM-RAG: graph foundation model for retrieval augmented generation.arXiv preprint arXiv:2502.01113(2025)

    arXiv 2025

  28. [28]

    Linhao Luo, Zicheng Zhao, Junnan Liu, Zhangchi Qiu, Junnan Dong, Serge Panev, Chen Gong, Thuy-Trang Vu, Gholamreza Haffari, Dinh Phung, et al. 2025. G-reasoner: Foundation Models for Unified Reasoning over Graph-structured Knowledge.arXiv preprint arXiv:2509.24276(2025)

    Pith/arXiv arXiv 2025

  29. [29]

    Costas Mavromatis and George Karypis. 2024. Gnn-rag: Graph neural retrieval for large language model reasoning.arXiv preprint arXiv:2405.20139(2024)

    Pith/arXiv arXiv 2024

  30. [30]

    Lang Mei, Siyu Mo, Zhihan Yang, and Chong Chen. 2025. A survey of multimodal retrieval-augmented generation.arXiv preprint arXiv:2504.08748(2025)

    Pith/arXiv arXiv 2025

  31. [31]

    Rui Meng, Ziyan Jiang, Ye Liu, Mingyi Su, Xinyi Yang, Yuepeng Fu, Can Qin, Zeyuan Chen, Ran Xu, Caiming Xiong, et al . 2025. Vlm2vec-v2: Advancing multimodal embedding for videos, images, and visual documents.arXiv preprint arXiv:2507.04590(2025)

    Pith/arXiv arXiv 2025

  32. [32]

    Liu Pai, Wenyang Gao, Wenjie Dong, Lin Ai, Ziwei Gong, Songfang Huang, Li Zongsheng, Ehsan Hoque, Julia Hirschberg, and Yue Zhang. 2024. A survey on open information extraction from rule-based model to large language model. Findings of EMNLP(2024), 9586–9608

    2024

  33. [33]

    Hyeongcheol Park, Jiyoung Seo, Jaewon Mun, Hogun Park, Wonmin Byeon, Sung June Kim, Hyeonsoo Im, JeungSub Lee, and Sangpil Kim. 2025. M 3 KG- RAG: Multi-hop Multimodal Knowledge Graph-enhanced Retrieval-Augmented Generation.arXiv preprint arXiv:2512.20136(2025)

    Pith/arXiv arXiv 2025

  34. [34]

    Boci Peng, Yun Zhu, Yongchao Liu, Xiaohe Bo, Haizhou Shi, Chuntao Hong, Yan Zhang, and Siliang Tang. 2025. Graph retrieval-augmented generation: A survey. ACM Transactions on Information Systems44, 2 (2025), 1–52

    2025

  35. [35]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. InICML. 8748–8763

    2021

  36. [36]

    Alon Talmor, Ori Yoran, Amnon Catav, Dan Lahav, Yizhong Wang, Akari Asai, Gabriel Ilharco, Hannaneh Hajishirzi, and Jonathan Berant. 2021. Multimodalqa: Complex question answering over text, tables and images.arXiv preprint arXiv:2104.06039(2021)

    Pith/arXiv arXiv 2021

  37. [37]

    Yang Tian, Fan Liu, Jingyuan Zhang, Yupeng Hu, Liqiang Nie, et al. 2025. CoRe- MMRAG: Cross-Source Knowledge Reconciliation for Multimodal RAG. InACL. 32967–32982

    2025

  38. [38]

    Julian R Ullmann. 1976. An algorithm for subgraph isomorphism.J. ACM23, 1 (1976), 31–42

    1976

  39. [39]

    Xueyao Wan and Hang Yu. 2025. Mmgraphrag: Bridging vision and language with interpretable multimodal knowledge graphs.arXiv preprint arXiv:2507.20804 (2025)

    Pith/arXiv arXiv 2025

  40. [40]

    Qiuchen Wang, Ruixue Ding, Zehui Chen, Weiqi Wu, Shihang Wang, Pengjun Xie, and Feng Zhao. 2025. Vidorag: Visual document retrieval-augmented generation via dynamic iterative reasoning agents. InEMNLP. 9124–9145

    2025

  41. [41]

    Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2014. Em- bedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575(2014)

    Pith/arXiv arXiv 2014

  42. [42]

    Wei Yang, Jingjing Fu, Rui Wang, Jinyu Wang, Lei Song, and Jiang Bian. 2025. Omgm: Orchestrate multiple granularities and modalities for efficient multi- modal retrieval. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 24545–24563

    2025

  43. [43]

    Shi Yu, Chaoyue Tang, Bokai Xu, Junbo Cui, Junhao Ran, Yukun Yan, Zheng- hao Liu, Shuo Wang, Xu Han, Zhiyuan Liu, et al . 2024. Visrag: Vision-based retrieval-augmented generation on multi-modality documents.arXiv preprint arXiv:2410.10594(2024)

    Pith/arXiv arXiv 2024

  44. [44]

    Haonan Yuan, Qingyun Sun, Junhua Shi, Mingjun Liu, Jiaqi Yuan, Ziwei Zhang, Xingcheng Fu, and Jianxin Li. 2026. Retrieving Minimal and Sufficient Reasoning Subgraphs with Graph Foundation Models for Path-aware GraphRAG.arXiv preprint arXiv:2603.07179(2026)

    arXiv 2026

  45. [45]

    Shaowen Zhou, Bowen Yu, Aixin Sun, Cheng Long, Jingyang Li, Haiyang Yu, Jian Sun, and Yongbin Li. 2022. A survey on neural open information extraction: Current status and future directions.arXiv preprint arXiv:2205.11725(2022)

    Pith/arXiv arXiv 2022

  46. [46]

    yellow painted section

    Zhaocheng Zhu, Zuobai Zhang, Louis-Pascal Xhonneux, and Jian Tang. 2021. Neural bellman-ford networks: A general graph neural network framework for link prediction.NeurIPS34 (2021), 29476–29490. 9 MM ’26, November 10–14, 2026, Rio de Janeiro, Brazil Jiacheng Tao, Qingyun Sun, Haonan Yuan, Ziwei Zhang, and Jianxin Li A Algorithm and Complexity Analysis Con...

    2021

  47. [2015]

    Leveraging linguistic structure for open domain information extraction. In ACL. 344–354

  48. [2023]

    Towards foundation models for knowledge graph reasoning.arXiv preprint arXiv:2310.04562(2023)

    Pith/arXiv arXiv 2023