Pith. sign in

REVIEW 3 major objections 4 minor 47 references

DocMemo: Dynamic Evidence Discovery via Probabilistic Memory-Guided Retrieval for Multi-Modal Document Understanding

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Tracking page relevance as a Bayesian belief updated after each reasoning round — rather than fixing a top-k page set upfront — lifts long-document visual question answering to state-of-the-art accuracy on three benchmarks.

desk verdict A promising memory-guided retrieval framework with a solid controlled experiment, but the appendix reports cumulative accuracy (69.0) that contradicts the headline 71.3; that discrepancy undercuts the SOTA and ablation claims until resolved. read the letter →

arxiv 2608.07067 v1 pith:ZSJHZZKS submitted 2026-08-07 cs.AI cs.CLcs.IRcs.MM

Hanshu Yao , Janfeng Zhong , Niu Lian , Jinpeng Wang This is my paper · ORCID
classification cs.AIcs.CLcs.IRcs.MM
keywords AgentMemoryLongDocumentUnderstandingThompsonSamplingBayesianBeliefUpdatingMultimodalVisualQuestionAnsweringRetrieval-AugmentedGenerationIterativeRetrievalPage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Long-document question answering routinely misses scattered evidence because retrieval is committed too early: typical systems fix a top-k page set before reasoning, and iterative systems pile up context without passing useful state between rounds. DocMemo instead maintains a live probabilistic map of page relevance — one Beta distribution per page — that is updated after every reasoning round from the model's own judgments of which pages were useful and which were not, and then used to steer the next round's search. On three long-document benchmarks, MMLongBench-Doc, LongDocURL, and PaperTab, the paper reports state-of-the-art accuracy of 71.3, 81.1, and 80.4, and its ablations trace the gains to the belief-update loop and the tri-level memory rather than to extra retrieval rounds. If correct, the result is evidence that treating long-document evidence search as a dynamic, stateful exploration problem — not a one-shot lookup — is a practical route to better multimodal understanding.

What carries the argument

The load-bearing object is the Page Belief Memory: for each page $p_i$ the system maintains a Beta distribution $\text{Beta}(\alpha_i^{(t)}, \beta_i^{(t)})$ whose posterior mean is the accumulated relevance confidence. After each reasoning round the distribution is updated with the Beta–Bernoulli conjugacy rule $\alpha_i \leftarrow \alpha_i + \mathbf{1}[i \in U_t]$ and $\beta_i \leftarrow \beta_i + \mathbf{1}[i \in V_t]$, positive feedback is propagated to neighboring pages within a fixed radius, and Thompson samples drawn from the posteriors balance exploiting high-confidence pages against exploring uncertain ones. This object is what converts reasoning feedback into retrieval control, and its removal produces the largest single accuracy drop in the ablations.

What would settle it

Take a sample of retrieval–reasoning rounds from any of the three benchmarks, have human annotators mark which retrieved pages actually contain evidence for the question, and compare those ground-truth labels with the model's self-reported useful_pages output. If the precision of the model's positive labels is no better than the base rate of relevant pages among retrieved pages, the Bayesian update cannot be concentrating search on truly relevant pages, and the accuracy gap between full DocMemo and the no-belief-update ablation must instead be explained by components such as the LLM reranker or the extra retrieval rounds.

Watch

Extended reading notes

Core claim

DocMemo frames long-document evidence search as a retrieval–reasoning–memory loop. A Document Schema Memory stores offline structural priors (document type, section page ranges, global summary); a Page Belief Memory holds a Beta distribution for every page, encoding the running confidence that the page is relevant to the current question, initialized from visual retrieval scores and updated after each reasoning round with the reasoner's useful-page and irrelevant-page labels; and a Question Episodic Memory records what has been found, what is still missing, and the rewritten query. Page selection draws Thompson samples from the beliefs, combines them with current query–page visual similarity, propagates positive belief to neighboring pages, and feeds up to nine pages to the reasoning model, which may request fine-grained table or figure crops. The paper reports that this system reaches 71.3 on MMLongBench-Doc, 81.1 on LongDocURL, and 80.4 on PaperTab, and that removing the Bayesian belief update drops the MMLongBench-Doc score to 68.8 while removing all memory drops it to 68.5.

Load-bearing premise

The belief-update loop is driven entirely by the model's own self-reported 'useful pages' and 'irrelevant pages' judgments, and the paper does not measure whether those self-labels are correct, so a mislabel is written back into memory and can steer later retrieval in the wrong direction.

Editorial extensions

If this is right

  • The belief-update loop recovers evidence the first round misses: on MMLongBench-Doc, evidence recall climbs from 28.3% after round one to 69.6% after round three, and the all-hit rate rises from 12.9% to 58.1%.
  • The largest single-category gain is on unanswerable questions (78.8%), which the paper attributes to accumulated cross-round states making it clearer when evidence is genuinely absent from the document.
  • Table-heavy questions benefit doubly: page beliefs steer retrieval toward the right pages and fine-grained page-region crops (up to five per query) supply the high-resolution table values that full-page downscaling loses.
  • Efficiency improves alongside accuracy: DocMemo uses on average 1.24 iterations per question versus a fixed three for the main iterative baseline, an estimated 2.4x efficiency gain at higher accuracy.
  • The gains survive a matched comparison: holding the backbone model, retriever, page budget, and evaluation protocol fixed, DocMemo still outscores the strongest iterative baseline (71.3 vs 69.3 on MMLongBench-Doc).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper never measures the quality of the model's self-reported page labels, so a testable extension would be a label-corruption sweep: perturb a controlled fraction of useful/irrelevant labels and observe whether the accuracy gap from belief updating shrinks proportionally, which would reveal how much of the gain rests on the reasoner being a good judge of its own evidence.
  • The belief mechanism is not visual-document-specific: it only needs a per-candidate relevance score and an adjacency notion, so the same Bayesian-update loop could transfer to long audio, video, or plain-text corpora by replacing 'neighboring pages' with neighboring time windows or sections.
  • The hand-set hyperparameters (prior strength $S=5$, propagation radius $r=2$, fusion schedule $\lambda=[0.0, 0.3, 0.6, 0.6]$) hint at a natural next step: learning these schedules per document type or per query difficulty instead of tuning them once.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. DocMemo proposes a memory-guided iterative retrieval framework for long-document multimodal DocVQA. It maintains three memory levels (Document Schema Memory, Page Belief Memory, and Question Episodic Memory), selects pages through Thompson sampling over Beta posteriors, propagates positive page feedback spatially, and uses adaptive-granularity evidence access. The paper reports state-of-the-art results on MMLongBench-Doc (71.3), LongDocURL (81.1), and PaperTab (80.4), with ablations attributing the gains to the tri-level memory and Bayesian page belief updating.

Significance. If the reported numbers are correct, DocMemo is a useful step beyond static and simple iterative retrieval: the controlled comparison in Table 2, the module ablations in Tables 4 and 12, the Thompson-vs-greedy comparison in Table 14, and the human-agreement analysis in Appendix C.6 are all appropriate and somewhat strengthen the empirical case. The paper also provides a code repository. However, the central numerical inconsistency in Appendix C.1 currently prevents accepting the headline claims at face value, and the self-supervised nature of the page-feedback labels deserves a direct validation. The contribution is significant in scope and potentially publishable after these points are resolved.

major comments (3)
  1. [Appendix C.1, Table 8] Table 8 reports cumulative accuracy on MMLongBench-Doc reaching 69.0% after the third retrieval–reasoning iteration, while Tables 1, 3, and 4 report the full system at 71.3% on the same benchmark. The round accuracies (72.0, 59.1, 46.2) and cumulative gains (60.6 to 65.5 to 69.0) imply, under the standard interpretation in which every question is counted once and unanswered questions are counted as incorrect, that essentially all questions have already received an answer by the third round; the three round denominators sum to approximately 100.07% of the test set when round accuracy is interpreted as the fraction of answers issued in that round that are correct. This leaves no remaining questions with which the extra 2.3 points could be obtained. The paper describes no fallback, subsetting, or rescoring procedure that would reconcile 69.0% with 71.3%. Because the state-of-the-art claim and the central Bayesian-updating ablation (71.3 vs. 68.8) both depend on the 71.3 figure, this inconsistency must be fixed or fully explained before the headline results can be accepted.
  2. [Section 3.2 and Listing 3] The positive and negative feedback used in the Beta updates comes entirely from the reasoner's own useful_pages and irrelevant_pages judgments, and the paper provides no measurement of the quality of these self-reported labels. Incorrect labels are absorbed into the posterior and propagated to neighboring pages, so they can reinforce rather than correct early retrieval errors; this is a real risk because the belief-update loop is the main mechanism of the method. I recommend reporting label precision/recall against the benchmarks' gold evidence annotations, or a robustness experiment with flipped or noisy labels, to show that the update remains effective under imperfect self-supervision.
  3. [Appendix A, Tables 6 and 7] The sensitivity analyses for the prior strength S and the fusion schedule lambda are evaluated on the full MMLongBench test set, and the main results use the configuration that performs best on that set. The spread across S (69.37–71.28) and across lambda (69.04–71.28) is comparable to the 2.5-point contribution attributed to Bayesian updating in Table 4 (68.80 vs. 71.28). This makes it difficult to determine whether the core gains reflect the mechanism itself or tuning on the evaluation benchmark. Please provide validation-set-based selection or a held-out analysis to establish robustness of the reported margins.
minor comments (4)
  1. [Abstract and Section 1] There are missing spaces in the abstract ('whichrespectively', 'dynamicrelevance') and in Section 1 ('whichrespectively'); please fix these typographical errors throughout.
  2. [Table 3 and Table 2] The SimpleDoc results differ between Table 2 (69.3 with Qwen3.5-9B) and Table 3 (60.6 without an explicit backbone note); please clarify the configuration associated with each reported baseline so that the two tables are not open to misinterpretation.
  3. [Figure 1] Figure 1 contains rendering artifacts in the submitted text version (e.g., unreadable block characters); please ensure the final figure is clean and legible.
  4. [Appendix C.6] The human-verification section states that 300 questions were sampled from MMLongBench but does not report the number of human judges or the instruction they received; a sentence with this detail would make the high kappa easier to assess.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DocMemo's central claims rest on external benchmarks and an independently implemented Bayesian update loop; the appendix consistency issues are not derivation circularity.

full rationale

I walked the claimed derivation chain from the tri-level memory definitions (Sec. 3.1) through Bayesian page belief updating (Sec. 3.2), Thompson-sampling-based selection, and the final evaluation (Sec. 4). None of the headline results is defined in terms of the method's own outputs: accuracy is measured against external benchmarks (MMLongBench-Doc, LongDocURL, PaperTab) with GPT-4.1 judging, and the ablations are empirical removals of components rather than algebraic consequences. The Beta-Bernoulli update is a standard conjugate model, and the belief prior is initialized from ColQwen2.5 retrieval scores; the later retrieval score is a convex combination of a fresh similarity signal and the sampled belief, so the loop is not a tautology. The closest concern is that hyperparameters S, lambda, r, and gamma are selected with sensitivity analysis performed on the full MMLongBench test set (Appendix A, Tables 6-7), which is a benchmark-tuning caveat, not a constructional circularity; the reported sensitivity is modest. The paper also cites its own authors' prior work at [19] in related work, but that citation is not load-bearing for the method's correctness. Finally, Appendix C.1 reports a cumulative MMLongBench-Doc accuracy of 69.0% while Tables 1-4 report 71.3%, and Table 8's per-iteration numbers are not arithmetically reconciled with the higher headline number; this is an internal-consistency problem that the authors should fix, but it is not an instance of a prediction reducing to its inputs, so it does not raise the circularity score.

Assumptions & free parameters 4 free parameters · 6 assumptions · 3 invented entities

The framework rests on standard probabilistic machinery (Beta-Bernoulli conjugacy, Thompson sampling) and on domain assumptions about document locality and self-reported page feedback. The main free parameters are the belief-update hyperparameters S, lambda, r, and gamma, which are tuned on the evaluation benchmarks; sensitivity analyses show modest variation around the chosen values.

free parameters (4)
  • S (Beta prior strength) = 5.0
    Controls the influence of the initial visual retrieval score on the Beta prior initialization; tuned on MMLongBench-Doc.
  • lambda_t (cross-round fusion schedule) = [0.0, 0.3, 0.6, 0.6]
    Weights the interpolation between current visual similarity and sampled page belief across retrieval rounds; tuned on MMLongBench-Doc.
  • r (spatial propagation radius) = 2
    Number of neighboring pages receiving propagated positive feedback; chosen to cover locally relevant pages while avoiding cross-section noise.
  • gamma (spatial decay factor) = 0.5
    Distance decay for spatial proximity propagation of positive page feedback.
assumptions (6)
  • standard math Beta-Bernoulli conjugacy gives a closed-form page relevance posterior.
    Used in Section 3.2 to update page beliefs with useful/irrelevant page feedback.
  • standard math Thompson sampling is an appropriate exploration-exploitation strategy for page selection.
    Used in Section 3.3 to sample relevance estimates from page posteriors; cites Thompson (1933).
  • domain assumption Evidence in long documents tends to cluster in local page ranges.
    Motivates the spatial proximity propagation mechanism in Section 3.2; attributed to cognitive load theory and empirical observations.
  • domain assumption The reasoner's self-reported useful_pages and irrelevant_pages labels are reliable enough to guide belief updates.
    The belief update loop in Section 3.2 relies on these labels; the paper provides no direct measurement of their accuracy.
  • domain assumption Page summaries and visual embeddings faithfully represent page content for schema construction and belief initialization.
    Document Schema Memory and Page Belief Memory initialization in Sections 3.1 and 3.2 assume these representations capture the necessary evidence.
  • domain assumption GPT-4.1 judge decisions align with human judgments of answer correctness.
    Automatic evaluation uses GPT-4.1; Appendix C.6 reports 96.7% agreement and kappa 0.92 on a 300-question sample, partially validating this assumption.
invented entities (3)
  • Document Schema Memory
    purpose: Stores query-independent structural priors: document type, section layout, and global summary.
    An internal software component whose value is measured jointly through ablations; no external falsifiable handle outside this paper.
  • Page Belief Memory
    purpose: Maintains Beta distributions over page-level relevance confidence that evolve with reasoning feedback.
    An internal architectural component; its effectiveness is shown through ablation experiments, not through an independent external signal.
  • Question Episodic Memory
    purpose: Records query-specific reasoning trajectories, including evidence summaries, missing information, and refined queries.
    An internal component that shapes retrieval behavior; no independent evidence outside the framework.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DocMemo: Dynamic Evidence Discovery via Probabilistic Memory-Guided Retrieval for Multi-Modal Document Understanding." pith.science (2026). https://pith.science/paper/ZSJHZZKS

@misc{pith2026260807067,
  author       = {Pith},
  title        = {Pith review of: DocMemo: Dynamic Evidence Discovery via Probabilistic Memory-Guided Retrieval for Multi-Modal Document Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZSJHZZKS}},
  note         = {Machine review of arXiv:2608.07067}
}
abstract

Long-document understanding requires locating sparse and heterogeneous evidence across hundreds of pages, yet existing systems remain limited by static retrieval and fragile cross-round memory. Mainstream single-round methods commit to a fixed top-$k$ page set at the outset and struggle to recover from early retrieval errors; recent iterative approaches allow multi-round evidence acquisition, but they do not investigate the propagation mechanism of cross-round states, making it difficult to track the dynamic changes in page relevance. To address these limitations, we propose DocMemo, a memory-guided framework that formulates long-document reasoning as dynamic evidence exploration. DocMemo maintains a tri-level retrieval state consisting of Document Schema Memory, Page Belief Memory, and Question Episodic Memory, which respectively capture structural priors, dynamic relevance estimation, and query-specific reasoning trajectories. During reasoning, DocMemo continuously refines cross-round page selection through Bayesian page belief updating with Thompson sampling, spatial proximity propagation, and structure-aware adaptive-granularity evidence access, while supplementing page-level evidence with fine-grained visual regions. Experiments on 3 benchmarks show that DocMemo achieves state-of-the-art performance and validate the efficacy of structured memory and dynamic page belief updating. Code is available at https://github.com/Harrygof/DocMemo.

Figures

Figures reproduced from arXiv: 2608.07067 by the authors.

Figure 1
Figure 1. Overview and motivation of DocMemo. (a) Single￾turn static retrieval commits to a fixed candidate page pool be￾fore reasoning; once key evidence is missed, the model cannot revise its search trajectory. (b) Iterative retrieval allows multiple search rounds, yet it lacks a cross-round state propagation mech￾anism. (c) DocMemo introduces a tri-level memory structure. Reasoning feedback is written back through Bayesian… view at source ↗
Figure 2
Figure 2. Overview of DocMemo: (left) tri-level document memory maintains document schema, page belief, and question episodic traces; (right) memory-guided dynamic retrieval uses Thompson sampling and Bayesian belief updating to iteratively select evidence pages, incorporate reasoning feedback, and access fine-grained local regions for answer generation. query and each page using the late-interaction mechanism of ColQwen2.5. … view at source ↗
Figure 3
Figure 3. Iterative evidence discovery and relative efficiency comparison. (a) Evidence recall and all-hit rate progressively im￾prove with iterative retrieval-reasoning. (b) Relative efficiency comparison. further reports category-wise results on MMLongBench￾Doc. DocMemo achieves an overall accuracy of 71.3% and attains the best performance. Its gains are especially pronounced on TAB and UNA, reaching 73.3% and 78.8%, respec… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: A qualitative example of DocMemo’s memory-guided iterative retrieval. resulting in a 2.40× overall efficiency improvement. This indicates that memory-guided dynamic retrieval can re￾duce unnecessary searches and improve evidence acquisi￾tion efficiency. In addition, th…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 29 canonical work pages

  1. [1]

    System card: Claude opus 4 & claude sonnet 4.Claude-4 Model Card, 2025

    AI Anthropic. System card: Claude opus 4 & claude sonnet 4.Claude-4 Model Card, 2025

    work page 2025

  2. [2]

    Self-rag: Learning to retrieve, generate, and critique through self-reflection

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avi Sil, and Hannaneh Hajishirzi. Self-rag: Learning to retrieve, generate, and critique through self-reflection. In International conference on learning representations, volume 2024, pages 9112–9141, 2024

    work page 2024

  3. [3]

    Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025

    arXiv 2025

  4. [4]

    M3docrag: Multi-modal retrieval is what you need for multi-page multi-document understanding.arXiv preprint arXiv:2411.04952, 2024

    JaeminCho,DebanjanMahata,OzanIrsoy,YujieHe, and Mohit Bansal. M3docrag: Multi-modal retrieval is what you need for multi-page multi-document understanding.arXiv preprint arXiv:2411.04952, 2024

    arXiv 2024

  5. [5]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long con- text, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    Gheorghe Comanici, Eric Bieber, Mike Schaek- ermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long con- text, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    arXiv 2025

  6. [6]

    Longdocurl: a comprehensive mul- timodal long document benchmark integrating un- derstanding, reasoning, and locating

    Chao Deng, Jiale Yuan, Pi Bu, Peijie Wang, Zhong- Zhi Li, Jian Xu, Xiao-Hui Li, Yuan Gao, Jun Song, Bo Zheng, et al. Longdocurl: a comprehensive mul- timodal long document benchmark integrating un- derstanding, reasoning, and locating. InProceed- ings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), page...

    work page 2025

  7. [7]

    Towards large language models with human-like episodic memory.Trends in Cognitive Sciences, 2025

    Cody V Dong, Qihong Lu, Kenneth A Norman, and Sebastian Michelmann. Towards large language models with human-like episodic memory.Trends in Cognitive Sciences, 2025

    2025

  8. [8]

    Colpali: Efficient document retrieval with vision language models

    Manuel Faysse, Hugues Sibille, Tony Wu, Bilel Om- rani, Gautier Viaud, Céline Hudelot, and Pierre Colombo. Colpali: Efficient document retrieval with vision language models. InInternational Conference on Learning Representations, volume 2025, pages 61424–61449, 2025

    work page 2025

Show all 47 references
  1. [9]

    Retrieval-augmented generation for large language models: A survey

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yixin Dai, Jiawei Sun, Haofen Wang, Haofen Wang, et al. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997, 2(1):32, 2023

    2023 arXiv

  2. [10]

    Ma-lmm: Memory-augmented large multimodal model for long-term video under- standing

    Bo He, Hengduo Li, Young Kyun Jang, Menglin Jia, Xuefei Cao, Ashish Shah, Abhinav Shrivastava, and Ser-Nam Lim. Ma-lmm: Memory-augmented large multimodal model for long-term video under- standing. InProceedings of the IEEE/CVF conference on computer vision and pattern recognit...

    2024

  3. [11]

    Licomemory: Lightweight and cognitive agentic memory for efficient long-term reasoning.arXiv preprint arXiv:2511.01448, 2025

    Zhengjun Huang, Zhoujin Tian, Qintian Guo, Fangyuan Zhang, Yingli Zhou, Di Jiang, Zeying Xie, and Xiaofang Zhou. Licomemory: Lightweight and cognitive agentic memory for efficient long-term reasoning.arXiv preprint arXiv:2511.01448, 2025

    2025

  4. [12]

    Uda: A benchmarksuiteforretrievalaugmentedgeneration in real-world document analysis.Advances in Neural Information Processing Systems, 37:67200–67217, 2024

    Yulong Hui, Yao Lu, and Huanchen Zhang. Uda: A benchmarksuiteforretrievalaugmentedgeneration in real-world document analysis.Advances in Neural Information Processing Systems, 37:67200–67217, 2024

    2024

  5. [13]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Rad- ford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  6. [14]

    Simpledoc: Multi-modal docu- ment understanding with dual-cue page retrieval and iterative refinement

    Chelsi Jain, Yiran Wu, Yifan Zeng, Jiale Liu, Shengyu Dai, Zhenwen Shao, Qingyun Wu, and Huazheng Wang. Simpledoc: Multi-modal docu- ment understanding with dual-cue page retrieval and iterative refinement. InProceedings of the 2025 Conference on Empirical Methods in Natural L...

    2025

  7. [15]

    Active retrieval aug- mented generation

    Zhengbao Jiang, Frank F Xu, Luyu Gao, Zhiqing Sun,QianLiu,JaneDwivedi-Yu,YimingYang,Jamie Callan, and Graham Neubig. Active retrieval aug- mented generation. InProceedings of the 2023 con- ference on empirical methods in natural language processing, pages 7969–7992, 2023

    2023

  8. [16]

    Dense passage retrieval for open-domain question answering

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. InProceedings of the 2020 conference on empirical methods in natu- ral language processing (EMNLP), pag...

    2020

  9. [17]

    Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in neu- ral information processing systems, 33:9459–9474, 2020

    PatrickLewis, EthanPerez, AleksandraPiktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Hein- rich Küttler, Mike Lewis, Wen-tau Yih, Tim Rock- täschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in neu- ral information processing syste...

    2020

  10. [18]

    Regionrag: Region- level retrieval-augmented generation for visual doc- ument understanding

    Yinglu Li, Zhiying Lu, Zhihang Liu, Yiwei Sun, Chuanbin Liu, and Hongtao Xie. Regionrag: Region- level retrieval-augmented generation for visual doc- ument understanding. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 6662–6670, 2026

    2026

  11. [19]

    From verbatim to gist: Distilling pyramidal multimodal memory via semantic information bot- tleneckforlong-horizonvideoagents.arXivpreprint arXiv:2603.01455, 2026

    Niu Lian, Yuting Wang, Hanshu Yao, Jinpeng Wang, Bin Chen, Yaowei Wang, Min Zhang, and Shu-Tao Xia. From verbatim to gist: Distilling pyramidal multimodal memory via semantic information bot- tleneckforlong-horizonvideoagents.arXivpreprint arXiv:2603.01455, 2026

    2026 arXiv

  12. [20]

    Ra-isf: Learning to answer and understand from retrieval augmentation via iterative self-feedback

    Yanming Liu, Xinyue Peng, Xuhong Zhang, Wei- hao Liu, Jianwei Yin, Jiannan Cao, and Tianyu Du. Ra-isf: Learning to answer and understand from retrieval augmentation via iterative self-feedback. InFindings of the Association for Computational Lin- guistics: ACL 2024, pages 4730...

    2024

  13. [21]

    Seeing, listening, remembering, and reasoning: A multi- modal agent with long-term memory.arXiv preprint arXiv:2508.09736, 2025

    Lin Long, Yichen He, Wentao Ye, Yiyuan Pan, Yuan Lin, Hang Li, Junbo Zhao, and Wei Li. Seeing, listening, remembering, and reasoning: A multi- modal agent with long-term memory.arXiv preprint arXiv:2508.09736, 2025

    2025

  14. [22]

    Mmlongbench-doc: Benchmark- inglong-contextdocumentunderstandingwithvisu- alizations.AdvancesinNeuralInformationProcessing Systems, 37:95963–96010, 2024

    Yubo Ma, Yuhang Zang, Liangyu Chen, Meiqi Chen, Yizhu Jiao, Xinze Li, Xinyuan Lu, Ziyu Liu, Yan Ma, Xiaoyi Dong, et al. Mmlongbench-doc: Benchmark- inglong-contextdocumentunderstandingwithvisu- alizations.AdvancesinNeuralInformationProcessing Systems, 37:95963–96010, 2024

    2024

  15. [23]

    JamesLMcClelland, BruceLMcNaughton, andRan- dall C O’Reilly. Why there are complementary learn- ing systems in the hippocampus and neocortex: in- sights from the successes and failures of connection- ist models of learning and memory.Psychological review, 102(3):419, 1995

    1995

  16. [24]

    Memgpt: towards llms as operating systems

    Charles Packer, Vivian Fang, Shishir_G Patil, Kevin Lin, Sarah Wooders, and Joseph_E Gonzalez. Memgpt: towards llms as operating systems. 2023

    2023

  17. [25]

    Cognitive load during problem solv- ing: Effects on learning.Cognitive science, 12(2): 257–285, 1988

    John Sweller. Cognitive load during problem solv- ing: Effects on learning.Cognitive science, 12(2): 257–285, 1988

    1988

  18. [26]

    Vdocrag: Retrieval-augmented generation over visually-rich documents

    Ryota Tanaka, Taichi Iki, Taku Hasegawa, Kyosuke Nishida, Kuniko Saito, and Jun Suzuki. Vdocrag: Retrieval-augmented generation over visually-rich documents. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 24827– 24837, 2025

    2025

  19. [27]

    Qwen Team. Qwen3. 5-omni technical report.arXiv preprint arXiv:2604.15804, 2026

    2026 arXiv

  20. [28]

    On the likelihood that one unknown probability exceeds another in view of the evidence of two samples.Biometrika, 25(3/4): 285–294, 1933

    William R Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples.Biometrika, 25(3/4): 285–294, 1933

    1933

  21. [29]

    Interleaving retrieval with chain-of-thought reasoning for knowledge- intensive multi-step questions

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Interleaving retrieval with chain-of-thought reasoning for knowledge- intensive multi-step questions. InProceedings of the 61st annual meeting of the association for com- putational linguistics (volume...

    2023

  22. [30]

    Mineru: An open-source solution for precise document content extraction

    Bin Wang, Chao Xu, Xiaomeng Zhao, Linke Ouyang, Fan Wu, Zhiyuan Zhao, Rui Xu, Kaiwen Liu, Yuan Qu, Fukai Shang, et al. Mineru: An open-source solution for precise document content extraction. arXiv preprint arXiv:2409.18839, 2024

    2024 arXiv

  23. [31]

    Agenticocr: Parsing only what you need for effi- cientretrieval-augmentedgeneration.arXivpreprint arXiv:2602.24134, 2026

    Zhengren Wang, Dongsheng Ma, Huaping Zhong, Jiayu Li, Wentao Zhang, Bin Wang, and Conghui He. Agenticocr: Parsing only what you need for effi- cientretrieval-augmentedgeneration.arXivpreprint arXiv:2602.24134, 2026

    2026

  24. [32]

    Molorag: Bootstrapping docu- ment understanding via multi-modal logic-aware retrieval

    Xixi Wu, Yanchao Tan, Nan Hou, Ruiyang Zhang, and Hong Cheng. Molorag: Bootstrapping docu- ment understanding via multi-modal logic-aware retrieval. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 14035–14056, 2025

    2025

  25. [33]

    Large multimodal agents: A survey

    Junlin Xie, Zhihong Chen, Ruifei Zhang, Xiang Wan, and Guanbin Li. Large multimodal agents: A survey. arXiv preprint arXiv:2402.15116, 2024

    2024 arXiv

  26. [34]

    A-mem: Agentic mem- ory for llm agents.Advances in Neural Information Processing Systems, 38:17577–17604, 2026

    WujiangXu, ZujieLiang, KaiMei, HangGao, Juntao Tan, and Yongfeng Zhang. A-mem: Agentic mem- ory for llm agents.Advances in Neural Information Processing Systems, 38:17577–17604, 2026

    2026

  27. [35]

    Memory-r1: Enhancing large language model agents to manage and utilize memories via reinforcement learning

    Sikuan Yan, Xiufeng Yang, Zuchao Huang, Ercong Nie, Zifeng Ding, Zonggen Li, Xiaowen Ma, Jinhe Bi, Kristian Kersting, Jeff Z Pan, et al. Memory-r1: Enhancing large language model agents to manage and utilize memories via reinforcement learning. arXiv preprint arXiv:2508.19828, 2025

    2025 arXiv

  28. [36]

    Visrag: Vision-based retrieval-augmented generation on multi-modality documents

    Shi Yu, Chaoyue Tang, Bokai Xu, Junbo Cui, Jun- hao Ran, Yukun Yan, Zhenghao Liu, Shuo Wang, Xu Han, Zhiyuan Liu, et al. Visrag: Vision-based retrieval-augmented generation on multi-modality documents. InInternational Conference on Learning Representations, volume 2025, pages ...

    2025

  29. [37]

    Qintong Zhang, Bin Wang, Victor Shea-Jay Huang, Junyuan Zhang, Zhengren Wang, Hao Liang, Con- ghui He, and Wentao Zhang. Document parsing 10 DocMemo: Dynamic Evidence Discovery via Probabilistic Memory-Guided Retrieval for Multi-Modal Document Understanding unveiled: Technique...

    arXiv

  30. [38]

    Mldocrag: Multi- modal long-context document retrieval augmented generation.arXiv preprint arXiv:2602.10271, 2026

    Yongyue Zhang and Yaxiong Wu. Mldocrag: Multi- modal long-context document retrieval augmented generation.arXiv preprint arXiv:2602.10271, 2026

    2026

  31. [39]

    Doclens: A tool-augmented multi-agent framework for long visual document understanding.arXiv preprint arXiv:2511.11552, 2025

    Dawei Zhu, Rui Meng, Jiefeng Chen, Sujian Li, Tomas Pfister, and Jinsung Yoon. Doclens: A tool-augmented multi-agent framework for long visual document understanding.arXiv preprint arXiv:2511.11552, 2025

    2025

  32. [40]

    Internvl3: Ex- ploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internvl3: Ex- ploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025. 11 DocMemo: Dynam...

    2025 arXiv

  33. [41]

    Document Type ( doc_type ) Provide a specific , free - form d e s c r i p t i o n of the document's genre and subject matter

  34. [42]

    Document St ru ct ur e ( d o c _ s t r u c t u r e ) Identify the major thematic sections of the document with their a p p r o x i m a t e page ranges

  35. [43]

    Document Summary ( d o c _ s u m m a r y ) Write 4 -6 s en te nc es that describe what the document covers overall . Output strictly in the fo ll ow in g format : < doc_type > [ specific d e s c r i p t i o n of document genre and topic ] </ doc_type > < doc_structure > Pages ...

  36. [44]

    Identify distinct thematic t r a n s i t i o n s within this segment

  37. [45]

    Group c o n s e c u t i v e pages that cover the same theme into a single section entry

  38. [46]

    Use concise , d e s c r i p t i v e topic labels

  39. [47]

    Output : < local_structure > Pages X - Y : [ section topic ] Pages Y - Z : [ section topic ]

    Do not sp ec ul at e about content outside this page range . Output : < local_structure > Pages X - Y : [ section topic ] Pages Y - Z : [ section topic ] ... </ local_structure > Listing 2:Prompt excerpt for local structure extraction. The merge prompt then synthesizes multipl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.