REVIEW 11 cited by
MDocAgent: A Multi-Modal Multi-Agent Framework for Document Understanding
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Document Question Answering (DocQA) is a very common task. Existing methods using Large Language Models (LLMs) or Large Vision Language Models (LVLMs) and Retrieval Augmented Generation (RAG) often prioritize information from a single modal, failing to effectively integrate textual and visual cues. These approaches struggle with complex multi-modal reasoning, limiting their performance on real-world documents. We present MDocAgent (A Multi-Modal Multi-Agent Framework for Document Understanding), a novel RAG and multi-agent framework that leverages both text and image. Our system employs five specialized agents: a general agent, a critical agent, a text agent, an image agent and a summarizing agent. These agents engage in multi-modal context retrieval, combining their individual insights to achieve a more comprehensive understanding of the document's content. This collaborative approach enables the system to synthesize information from both textual and visual components, leading to improved accuracy in question answering. Preliminary experiments on five benchmarks like MMLongBench, LongDocURL demonstrate the effectiveness of our MDocAgent, achieve an average improvement of 12.1% compared to current state-of-the-art method. This work contributes to the development of more robust and comprehensive DocQA systems capable of handling the complexities of real-world documents containing rich textual and visual information. Our data and code are available at https://github.com/aiming-lab/MDocAgent.
Forward citations
Cited by 11 Pith papers
-
Locating Failure in Multi-Page Visually Rich Document Understanding: An Empirical Attribution
A controlled attribution study of multi-page document understanding finds that missing evidence and cross-page integration failures dominate over distractor noise and text extraction quality.
-
HierDoc: Hierarchical Page-to-Region Evidence Routing for Long-Document Visual Question Answering
HierDoc trains separate page- and region-selection policies with GRPO, then answers from full selected pages plus selected region crops and text.
-
XL-DocBench: Benchmarking Evidence-Grounded Extra-Long Document Understanding
XL-DocBench is a 1,519-question, fully human-verified extra-long document QA benchmark across six professional domains; state-of-the-art systems still score at most 44% on it.
-
FinSAgent: Corpus-Aligned Multi-Agent RAG Framework for Evidence-Grounded SEC Filing Question Answering
FinSAgent improves financial filing QA by conditioning sub-queries on a summary of the local corpus and gating semantic reranking with a learned validity signal, beating baseline systems on five benchmarks.
-
Dual Latent Memory for Visual Multi-agent System
L2-VMAS replaces text-based inter-agent communication in visual multi-agent systems with decoupled latent perception and thinking memories, improving accuracy by 2.7–5.4% and cutting token use by 21.3–44.8%.
-
Structured Attention Matters to Multimodal LLMs in Document Understanding
Structured LaTeX encoding of OCR text, combined with document images, improves DocQA accuracy across four MLLMs and four benchmarks without any training.
-
From EduVisBench to EduVisAgent: A Benchmark and Multi-Agent Framework for Reasoning-Driven Pedagogical Visualization
EduVisAgent, a five-agent framework, outperforms all baseline AI models at generating pedagogically effective interactive visualizations for STEM problems, according to the new EduVisBench benchmark and its GPT-4o-bas...
-
MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization
MMedPO weights preference-optimization training samples by clinical relevance scores, combining hallucinated text answers and locally noised lesion images, and reports improved medical VQA and report generation metrics.
-
Enhancing Large Multimodal Models in Key Information Extraction via Scene-Aware Document Synthesis
Scene-aware multi-agent document synthesis plus error-driven hard-example expansion improves compact Qwen3-VL models on constrained and open-category KIE, topping reported on-device baselines.
-
MoLoRAG: Bootstrapping Document Understanding via Multi-modal Logic-aware Retrieval
MoLoRAG boosts multi-page document QA by traversing a page graph and scoring pages with both semantic and VLM-judged logical relevance, improving retrieval and answer accuracy on four benchmarks.
-
A Multimodal Multi-Agent Framework for Radiology Report Generation
A five-agent retrieval-augmented pipeline for radiology report generation outperforms a single LLaVA-Med baseline on IU X-ray, but the comparison is limited to one weak baseline with no ablations and no statistical tests.
Discussion (0). Continue with ORCID to comment.