Pith. sign in

REVIEW 5 cited by

SV-RAG: LoRA-Contextualizing Adaptation of MLLMs for Long Document Understanding

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2411.01106 v2 pith:MG54F3ZQ submitted 2024-11-02 cs.CV

SV-RAG: LoRA-Contextualizing Adaptation of MLLMs for Long Document Understanding

Jian Chen , Ruiyi Zhang , Yufan Zhou , Tong Yu , Franck Dernoncourt , Jiuxiang Gu , Ryan A. Rossi , Changyou Chen
show 1 more author
Tong Sun
This is my paper
classification cs.CV
keywords mllmssv-ragpagesunderstandingdocumentmllmmultimodalperformance
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Multimodal large language models (MLLMs) have recently shown great progress in text-rich image understanding, yet they still struggle with complex, multi-page visually-rich documents. Traditional methods using document parsers for retrieval-augmented generation suffer from performance and efficiency limitations, while directly presenting all pages to MLLMs leads to inefficiencies, especially with lengthy ones. In this work, we present a novel framework named **S**elf-**V**isual **R**etrieval-**A**ugmented **G**eneration (SV-RAG), which can broaden horizons of any MLLM to support long-document understanding. We demonstrate that **MLLMs themselves can be an effective multimodal retriever** to fetch relevant pages and then answer user questions based on these pages. SV-RAG is implemented with two specific MLLM adapters, one for evidence page retrieval and the other for question answering. Empirical results show state-of-the-art performance on public benchmarks, demonstrating the effectiveness of SV-RAG.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. DocTrace: Towards Traceable Long Document VQA via Hierarchical Evidence Graph Reasoning

    cs.AI 2026-08 conditional novelty 6.0

    DocTrace reaches 52.9/56.4/85.1 on MMLongBench-Doc, LongDocURL, and SlideVQA by generating explicit evidence graphs, surpassing open-source baselines on all three and large closed models on two.

  2. MoDora: Tree-Based Semi-Structured Document Analysis System

    cs.IR 2026-02 unverdicted novelty 6.0

    MoDora introduces local-alignment aggregation, a Component-Correlation Tree, and question-type-aware retrieval to improve accuracy on semi-structured document QA by 5.97-61.07% over baselines.

  3. VisR-Bench: An Empirical Study on Visual Retrieval-Augmented Generation for Multilingual Long Document Understanding

    cs.CV 2025-08 conditional novelty 6.0

    A multilingual, multi-page document retrieval benchmark with 35K+ QA pairs shows MLLM retrievers lead but still fail on tables and low-resource languages.

  4. DocSeeker: Structured Visual Reasoning with Evidence Grounding for Long Document Understanding

    cs.AI 2026-04 unverdicted novelty 5.0

    DocSeeker uses supervised fine-tuning on distilled data followed by evidence-aware group relative policy optimization to improve long-document understanding and evidence grounding in MLLMs.

  5. DocSeeker: Structured Visual Reasoning with Evidence Grounding for Long Document Understanding

    cs.AI 2026-04 unverdicted novelty 5.0

    DocSeeker improves long-document understanding in MLLMs via a two-stage training process that combines supervised fine-tuning from distilled data with evidence-aware group relative policy optimization and memory-effic...