pith. sign in

arxiv: 2604.08952 · v2 · submitted 2026-04-10 · 💻 cs.CL · cs.IR

MAB-DQA: Addressing Query Aspect Importance in Document Question Answering with Multi-Armed Bandits

Yixin Xiang , Yunshan Ma , Xiaoyu Du , Yibing Chen , Yanxin Zhang , Jinhui Tang This is my paper

Pith reviewed 2026-05-10 17:51 UTC · model grok-4.3

classification 💻 cs.CL cs.IR
keywords Document Question AnsweringMulti-Armed BanditsMultimodal RAGQuery DecompositionAspect ImportanceRetrieval Budget AllocationDocument Understanding
0
0 comments X

The pith

MAB-DQA uses multi-armed bandits to reallocate retrieval budgets toward high-utility aspects in document queries.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces MAB-DQA to fix a retrieval bias in multimodal document question answering. Current systems often keep only the most visually prominent pages and miss informative content that supports less salient aspects of a query. The method splits a query into aspect-specific subqueries, treats them as bandit arms, and draws reward signals from quick reasoning on a few sample pages to decide which aspects deserve more retrieval slots. An exploration-exploitation policy then shifts the budget dynamically. Experiments on four benchmarks report gains of 5 to 18 percent over prior approaches.

Core claim

MAB-DQA decomposes a query into aspect-aware subqueries, retrieves separate candidate sets for each, treats the subqueries as arms in a multi-armed bandit, and uses preliminary reasoning results on representative pages as reward signals to estimate aspect utility. Guided by an exploration-exploitation policy, the framework reallocates the retrieval budget toward high-value aspects, assembles the most informative pages along with their correlations, and generates the final answer.

What carries the argument

Multi-armed bandit where each arm corresponds to an aspect-aware subquery and rewards are derived from preliminary reasoning on a small set of representative pages to drive dynamic budget reallocation.

If this is right

  • Less visually salient pages that support important query aspects are retrieved and used in answer generation.
  • Budget reallocation reduces the chance of overlooking content tied to secondary but relevant query facets.
  • Correlations among selected pages from different aspects become available for final answer synthesis.
  • The same performance pattern holds across multiple document QA benchmarks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The bandit formulation could be adapted to other retrieval settings where the number of pages or passages that can be processed is strictly limited.
  • Aspect utilities learned during one query might be cached and reused for similar follow-up questions to reduce repeated reasoning cost.
  • Scaling the number of aspects or the size of the preliminary sample set offers a direct way to test whether reward signal quality remains stable.

Load-bearing premise

Preliminary reasoning results from a small number of representative pages provide reliable reward signals that accurately estimate aspect utility without bias or omission of key aspects.

What would settle it

Removing the preliminary reasoning step or replacing the bandit policy with uniform random allocation of the retrieval budget produces no improvement or a drop in answer accuracy on the same four benchmarks.

Figures

Figures reproduced from arXiv: 2604.08952 by Jinhui Tang, Xiaoyu Du, Yanxin Zhang, Yibing Chen, Yixin Xiang, Yunshan Ma.

Figure 1
Figure 1. Figure 1: Aspect Retrieval Degradation in DQA. (a) [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The overview of our proposed framework MAB-DQA. (a) Decompose a query into aspect-aware [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of Average Inference Time Across Different Models on 4 NVIDIA V100 GPUs, with retrieval time (Top-10) in blue and QA time (Top￾4) in orange. ing all other parameters fixed. The results indicate that: (1) Parameter α has a significant positive ef￾fect on performance (Fig. 3a), with higher values better leveraging the visual-language model (VLM) to extract effective semantics; (2) Parameter β has … view at source ↗
Figure 3
Figure 3. Figure 3: Sensitivity Analysis of Key Hyperparameters [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative Analysis of the MAB-DQA Framework. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative Analysis of the MAB-DQA Framework. We selected a representative query that requires [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative Analysis of the MAB-DQA Framework. We selected a representative query that requires [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative Analysis of the MAB-DQA Framework. We selected a representative query that requires [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Sensitivity Analysis of Key Hyperparameters [PITH_FULL_IMAGE:figures/full_fig_p019_9.png] view at source ↗
read the original abstract

Document Question Answering (DQA) involves generating answers from a document based on a user's query, representing a key task in document understanding. This task requires interpreting visual layouts, which has prompted recent studies to adopt multimodal Retrieval-Augmented Generation (RAG) that processes page images for answer generation. However, in multimodal RAG, visual DQA struggles to utilize a large number of images effectively, as the retrieval stage often retains only a few candidate pages (e.g., Top-4), causing informative but less visually salient content to be overlooked in favor of common yet low-information pages. To address this issue, we propose a Multi-Armed Bandit-based DQA framework (MAB-DQA) to explicitly model the varying importance of multiple implicit aspects in a query. Specifically, MAB-DQA decomposes a query into aspect-aware subqueries and retrieves an aspect-specific candidate set for each. It treats each subquery as an arm and uses preliminary reasoning results from a small number of representative pages as reward signals to estimate aspect utility. Guided by an exploration-exploitation policy, MAB-DQA dynamically reallocates retrieval budgets toward high-value aspects. With the most informative pages and their correlations, MAB-DQA generates the expected results. On four benchmarks, MAB-DQA shows an average improvement of 5%-18% over the state-of-the-art method, consistently enhancing document understanding. Codes are available at https://github.com/ElephantOH/MAB-DQA.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The manuscript proposes MAB-DQA, a multi-armed bandit framework for multimodal document question answering. It decomposes user queries into implicit aspects, treats subqueries as arms, uses preliminary reasoning on a small number of representative pages to compute reward signals for aspect utility estimation, and employs an exploration-exploitation policy to dynamically reallocate retrieval budgets toward high-value aspects. The authors claim average performance improvements of 5-18% over state-of-the-art methods across four benchmarks.

Significance. If the reported gains hold and stem from the proposed mechanism rather than confounding factors like increased retrieval volume or base model improvements, the work could contribute to more effective handling of complex queries in visual document understanding. The open-sourcing of code supports potential for further validation and extension.

major comments (3)
  1. [§3.2] §3.2 (Reward Signal Computation): The method for selecting representative pages and computing rewards from preliminary reasoning outputs is not described in sufficient detail. It is unclear how the reward signal avoids bias toward visually salient pages or ensures coverage of low-salience but answer-critical content, which is central to validating the 5-18% gains.
  2. [§4] §4 (Experiments): The results section reports average improvements but does not provide per-benchmark breakdowns, statistical significance tests, or ablation studies isolating the contribution of the MAB component versus the aspect decomposition alone. This makes it difficult to confirm that the bandit policy is the source of the improvement rather than other design choices.
  3. [§3.3] §3.3 (Exploration-Exploitation Policy): The specific exploration-exploitation parameter and how it interacts with potentially noisy LLM-based reasoning rewards are not specified, raising concerns about the stability of the dynamic budget reallocation.
minor comments (2)
  1. [Abstract] Abstract: The abstract mentions 'Codes are available' but the link should be verified for accessibility and completeness of the repository.
  2. [Methods] Notation: Define the reward function R(a) explicitly in the methods section for clarity.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the thoughtful and constructive comments, which have helped us identify areas for improvement. We address each major comment below and will incorporate the suggested clarifications and additional analyses into the revised manuscript.

read point-by-point responses

  1. Referee: [§3.2] §3.2 (Reward Signal Computation): The method for selecting representative pages and computing rewards from preliminary reasoning outputs is not described in sufficient detail. It is unclear how the reward signal avoids bias toward visually salient pages or ensures coverage of low-salience but answer-critical content, which is central to validating the 5-18% gains.

    Authors: We agree that §3.2 would benefit from greater detail. In the revised manuscript we will expand the description as follows: representative pages are selected by first performing an initial lightweight retrieval (top-3 per aspect using CLIP embeddings) followed by k-means clustering on a combination of visual layout features and text embeddings to ensure diversity across salience levels; the reward for each aspect is then computed as the average of normalized LLM reasoning scores (relevance + answerability) over these pages, with an explicit penalty term for visual saliency (measured via edge density and text density) to down-weight overly prominent but low-information pages. This design explicitly promotes coverage of low-salience content. We will also add a short illustrative example and a diagram of the reward pipeline. revision: yes

  2. Referee: [§4] §4 (Experiments): The results section reports average improvements but does not provide per-benchmark breakdowns, statistical significance tests, or ablation studies isolating the contribution of the MAB component versus the aspect decomposition alone. This makes it difficult to confirm that the bandit policy is the source of the improvement rather than other design choices.

    Authors: We acknowledge the need for more granular reporting. The revised §4 will include: (1) a full per-benchmark table with exact scores for all baselines and MAB-DQA variants; (2) statistical significance results using paired t-tests across 5 random seeds with p-values reported; and (3) ablation studies that isolate the MAB policy by comparing (a) aspect decomposition with fixed equal budget allocation versus (b) the full dynamic MAB reallocation. These ablations will directly quantify the incremental gain attributable to the bandit mechanism. revision: yes

  3. Referee: [§3.3] §3.3 (Exploration-Exploitation Policy): The specific exploration-exploitation parameter and how it interacts with potentially noisy LLM-based reasoning rewards are not specified, raising concerns about the stability of the dynamic budget reallocation.

    Authors: We will clarify the policy in the revised §3.3. The implementation uses an epsilon-greedy strategy with ε = 0.15 (chosen via grid search on a held-out validation split) and a decaying learning rate for reward updates. To mitigate noise from LLM reasoning, we maintain an exponentially weighted moving average of rewards (α = 0.7) and enforce a minimum exploration floor of 10% of the total retrieval budget per aspect. We will add the exact hyper-parameter values, pseudocode for the update rule, and a brief sensitivity analysis showing performance remains stable for ε in [0.1, 0.2]. revision: yes

Circularity Check

0 steps flagged

No significant circularity; MAB-DQA is an applied framework with external rewards

full rationale

The paper describes an algorithmic framework that decomposes queries into subqueries treated as bandit arms, then uses independent preliminary reasoning outputs on representative pages as reward signals to drive budget reallocation. No equations, fitted parameters, or self-citations are presented that reduce the claimed performance gains or aspect-utility estimates to the inputs by construction. The approach is self-contained against external benchmarks and does not invoke uniqueness theorems or ansatzes from prior author work.

Axiom & Free-Parameter Ledger

1 free parameters · 2 axioms · 0 invented entities

The framework rests on assumptions about query decomposition and reward estimation from limited pages; no free parameters or invented entities are explicitly quantified in the abstract.

free parameters (1)
  • exploration-exploitation parameter
    Controls the bandit policy for reallocating retrieval budgets; specific value or tuning method not stated in abstract.
axioms (2)
  • domain assumption A query can be meaningfully decomposed into aspect-aware subqueries that capture varying importance
    Invoked as the starting point for treating subqueries as independent arms.
  • domain assumption Preliminary reasoning on a small number of representative pages yields reliable reward signals for aspect utility
    Used to estimate utility and guide dynamic allocation without full retrieval.

pith-pipeline@v0.9.0 · 5590 in / 1308 out tokens · 29524 ms · 2026-05-10T17:51:37.760628+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

15 extracted references · 15 canonical work pages

  1. [1]

    Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson

    From uncertainty to decision: Enhancing goal- oriented dialogue planning under hesitation.IEEE Transactions on Audio, Speech and Language Pro- cessing. Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. 2025. From Local to Global: A Graph RAG Approach...

    work page 2025

  2. [2]

    InThe Thirteenth Interna- tional Conference on Learning Representations

    ColPali: Efficient Document Retrieval with Vision Language Models. InThe Thirteenth Interna- tional Conference on Learning Representations. ShunLiang Fu, Yanxin Zhang, Yixin Xiang, Xiaoyu Du, and Jinhui Tang. 2026. Dmap: Human-aligned structural document map for multimodal document understanding.arXiv preprint arXiv:2601.18203. Siwei Han, Peng Xia, Ruiyi ...

    work page arXiv 2026

  3. [3]

    correct and complete

    Micro-act: Mitigate knowledge conflict in question answering via actionable self-reasoning. In Proceedings of the 63rd Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 18550–18574, Vienna, Austria. Association for Computational Linguistics. Songtao Jiang, Chenyi Zhou, Yan Zhang, Yeying Jin, and Zuozhu Liu. 2...

    work page 2025

  4. [4]

    If the question is clear and can be answered us- ing ONLY information in the screenshots, keep it essentially the same

    work page

  5. [5]

    If the question is ambiguous or vague, clar- ify it based on what information appears to be available in the screenshots

    work page

  6. [6]

    If the question cannot be answered with the screenshots, note this, but still try to rephrase for clarity

    work page

  7. [7]

    The rewritten question should be specific, direct, and answerable using visible document content

    work page

  8. [8]

    Keep the core intent of the original question

    work page

  9. [9]

    If screenshots show specific entities (names, dates, numbers, terms), use them in the rewritten question

    work page

  10. [10]

    { question }

    Output only the rewritten question, nothing else Rewritten question: C.6 Prompt For Answer Reflection This prompt template facilitates the multi-stage verification process by assessing whether answers adequately address the original query requirements: You will be given a question and a correspond- ing answer. Your task is to determine whether the answer ...

    work page

  11. [11]

    First, analyze what the question is REALLY asking for

    work page

  12. [12]

    Compare the initial answer with the available information

    work page

  13. [13]

    Identify gaps or inaccuracies in the initial answer

    work page

  14. [14]

    Synthesize information from the summary to fill these gaps

    work page

  15. [15]

    Not answerable

    Formulate a coherent response that directly addresses the question DO NOT simply copy phrases from the sum- mary. Instead, use the information to construct a thoughtful answer. If the summary indicates no relevant informa- tion, respond: "Not answerable" Reasoning process: - [Analyze the question requirements] - [Compare initial answer with evidence] - [I...

    work page