Pith. sign in

REVIEW 4 major objections 6 minor 37 references

POQD: Performance-Oriented Query Decomposer for Multi-vector retrieval

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Query decomposition for multi-vector retrieval can be optimized end-to-end by searching an LLM prompt with an LLM-based optimizer, alternating with brief fine-tuning of the downstream RAG generator.

desk verdict Useful empirical recipe, but Theorem 4.4's proof is not valid as written—the α-improvement is applied where it was never established, so the theory needs repair before the paper's central claim can stand. read the letter →

arxiv 2505.19189 v2 pith:HICDTAHD submitted 2025-05-25 cs.IR cs.DB

Yaoyang Liu , Junlin Li , Yinjun Wu , Zhen Chen This is my paper · ORCID
classification cs.IRcs.DB
keywords multi-vectorretrievalquerydecompositionretrieval-augmentedgenerationpromptoptimizationLLM-basedoptimizerRAGquestionansweringlateinteractionweakly-supervised
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that how a query is broken into sub-queries for multi-vector retrieval is a performance lever that can be optimized, not just chosen by tokenization or hand-written prompts. It proposes treating the decomposer's prompt as a searchable parameter: an LLM-based optimizer proposes new prompts, each candidate is scored by the downstream RAG training loss, and the generator is retrained only briefly between prompt updates. This matters because query decomposition is not end-to-end differentiable, so the paper offers a gradient-free, weakly supervised route that applies to arbitrary retrieval-augmented systems without retraining the retriever. The central theoretical claim is that with appropriate hyperparameters the alternating algorithm lowers the converged RAG loss by a positive margin, and the reported experiments on four QA benchmarks show consistent gains over dense retrieval, token-level ColBERT, and supervised, unsupervised, in-context, and feedback-based decomposition strategies.

What carries the argument

The load-bearing machinery is the alternating optimization loop of Algorithm 2 combined with Algorithm 1's prompt search, and the formal identity that carries the proof is the linear-convergence inequality for gradient descent under the PL* and smoothness assumptions. Algorithm 1 uses an LLM-based optimizer (built on the approach of Yang et al., 2024) that reads solution-score pairs of prompt and training loss $L(\Theta; p)$ and proposes new prompt prefixes; the query decomposer LLM turns the full prompt into sub-queries; those sub-queries drive MaxSim multi-vector retrieval, which feeds the RAG objective. The key identity is $L(\Theta_k; p) - L(\Theta^{*}(p); p) \leq (1 - \mu/(2L))^k (L(\Theta_0; p) - L(\Theta^{*}(p); p))$, which lets a short burst of $\tau$ gradient steps transfer the $\alpha$ improvement in immediate loss into a positive gap between converged losses, provided $\alpha > (1 - \mu/(2L))^{\tau} M$. The PL* condition is a strong-convexity-like bound—gradient norm squared at least $\mu$ times the loss—which together with $L$-smoothness gives linear convergence, and the paper imports it from prior LLM fine-tuning theory rather than verifying it for the retrieval-weighted objective in Equation (2).

What would settle it

Track how often Algorithm 1's line 7 condition actually fires on a fresh set of training queries: if the prompt optimizer frequently fails to find $p_{\text{new}}$ with $L(\Theta; p_{\text{new}}) \leq L(\Theta; p_{\text{old}}) - \alpha$, the premise of Theorem 4.4 is violated and the claimed guarantee does not apply. A complementary numerical check measures gradient norms and loss values from random restarts for the objective in Equation (2) to test whether the $\mu$-PL* and smoothness assumptions hold.

Watch

Extended reading notes

Core claim

The paper asserts that the query-decomposition step in multi-vector retrieval—how a question is split into phrases or tokens—can be optimized without gradients. The proposed mechanism treats the prefix of a prompt given to a decomposer LLM as the optimization variable: an LLM-based optimizer iteratively proposes new prompt prefixes, the decomposer produces sub-queries, and the training loss of a downstream RAG generator scores each candidate. Because retrieval is non-differentiable, the score is obtained by actually running retrieval and evaluating the RAG objective $L(\Theta; p)$. Algorithm 2 alternates between searching the prompt until the loss drops by at least $\alpha$ and training the generator for $\tau$ gradient steps with the new sub-queries. The central guarantee is Theorem 4.4: under a $\mu$-PL* and $L$-smoothness assumption on $L(\Theta; p)$, the converged losses satisfy $L(\Theta^{*}(p_{\text{old}}); p_{\text{old}}) - L(\Theta^{*}(p_{\text{new}}); p_{\text{new}}) \geq \alpha - (1 - \mu/(2L))^{\tau} M$, so with $\tau$ large enough the new prompt is provably better. The paper reports that on WebQA, MultiModalQA, ManyModalQA, and StrategyQA, the resulting sub-queries beat dense retrieval, ColBERT, and existing decomposition baselines in both retrieval accuracy and end-to-end QA accuracy.

Load-bearing premise

The whole improvement guarantee rests on the assumption that each round of prompt optimization actually produces a new decomposition prompt that lowers the training loss by at least a small fixed amount, and that the RAG loss behaves like a smooth, well-conditioned function while the generator is fine-tuned; if either part fails, the promised bound in Theorem 4.4 does not follow.

Editorial extensions

If this is right

  • Because POQD keeps the retrieval model frozen and only fine-tunes the generator, the approach can be dropped into an existing RAG pipeline without re-embedding the corpus or rebuilding indexes.
  • The bound in Theorem 4.4 makes the training-time/quality trade-off explicit: with larger $\tau$ the correction term $(1 - \mu/(2L))^{\tau} M$ shrinks, so the converged loss under the new prompt is lower by a larger margin; the experiments with $\tau = 3$ confirm the trend.
  • The same optimized decomposition improves multi-hop QA where sub-queries are generated dynamically during reasoning, so the benefit is not limited to single-hop retrieval.
  • On the reported benchmarks, the approach improves retrieval accuracy by up to 5.28% and end-to-end QA exact-match accuracy by up to 12.61% over the strongest baselines compared.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the result generalizes, prompt-level optimization of non-differentiable retrieval decisions could be applied to other RAG design choices, such as chunking granularity, reranking depth, or fusion weights, wherever a training loss can be scored cheaply.
  • Because the proof depends on a single successful prompt update per outer iteration, variance in the LLM optimizer is a natural next bottleneck; maintaining an ensemble of candidate prompts and selecting by validation loss would be a direct testable extension.
  • A natural extension is to ablate the final optimized prompt with and without the alternating generator updates, separating the contribution of prompt search from the contribution of generator re-tuning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces POQD, a query-decomposition framework for multi-vector retrieval (MVR) in retrieval-augmented generation (RAG) systems. POQD prompts one LLM to produce sub-queries and uses a second LLM-based optimizer to search for a prompt that minimizes the downstream RAG loss; it then alternates this prompt search with a small number of gradient-descent steps on the generator. The authors claim an end-to-end training algorithm whose effectiveness is supported by a theoretical bound (Theorem 4.4), and they report experiments on WebQA, MultiModalQA, ManyModalQA, and StrategyQA showing improved Hit@1/Hit@2 retrieval accuracy and end-to-end QA accuracy relative to several query-decomposition baselines. The paper also reports ablations over the prompt-acceptance threshold alpha, the number tau of generator update steps, the choice of LLM, generator model, embedding model, and retrieval count.

Significance. If the theoretical guarantee and empirical conclusions were fully supported, the paper would make a useful contribution: it targets an underexplored component of MVR, namely query decomposition beyond token-level splitting, and it evaluates on held-out retrieval metrics that are not the training objective, which is a genuine strength. The release of code and the breadth of ablations are also positive. However, the formal support is currently not valid as presented: the proof of Theorem 4.4 applies the alpha-improvement at a parameter where it was never established, and Algorithm 1's acceptance condition has a sign inconsistency. The empirical contribution is promising and largely consistent, but the absence of repeated runs or significance tests leaves some of the claimed margins unquantified. The central ideas are defensible and the empirical finding may survive a repaired theory or a weakened theoretical claim, so the paper is worth revising rather than rejecting.

major comments (4)
  1. [Appendix A, proof of Theorem 4.4] The proof applies the alpha-improvement at a parameter where it was not established. Algorithm 1 certifies only L(Theta_0; p_new) <= L(Theta_0; p_old) - alpha at the parameter Theta_0 at which the prompt search is run; the proof then asserts "L(Theta^old_tau; p_old) - L(Theta^old_tau; p_new) >= alpha" and equates L(Theta^old_tau; p_new) with L(Theta^new_0; p_new). Neither statement follows from the algorithm or from Assumptions 4.1-4.3, because the alpha-gap need not persist after tau gradient steps and Theta^old_tau and Theta^new_0 are different parameter points. Without an additional assumption linking prompt improvements across parameter movement, the lower bound Gold - Gnew >= alpha - A^tau M in Theorem 4.4 is not proven; this is load-bearing because the abstract's "as our theoretical analysis suggests" rests on this theorem.
  2. [Algorithm 1, Line 7] The acceptance condition is inconsistent with the proof. Line 7 breaks when L(Theta; p) - L(Theta; p_old) <= alpha, which accepts a prompt that is up to alpha worse than p_old; however the text in Section 4.2 and the proof in Appendix A require a reduction by alpha, i.e., L(Theta; p_old) - L(Theta; p) >= alpha. This sign issue means that Algorithm 1 as written can terminate with a p_new that does not reduce the loss, so the premise of Theorem 4.4 is not guaranteed by the algorithm.
  3. [Assumption 4.1 and Eq. (2)] The mu-PL* and L-smoothness assumptions are imported from fine-tuning theory for plain LLM losses, but the objective in Eq. (2) is a retrieval-weighted RAG loss whose gradient flows through the Top-K selection and MaxSim scoring in Eq. (1). The paper gives no argument that this objective satisfies the PL* condition or L-smoothness with the required constants for the models and corpora used, so the quantitative bound of Theorem 4.4 is conditional on unverified assumptions about the very objective being optimized.
  4. [Tables 1-3] All quantitative results are single runs without error bars, confidence intervals, or significance tests. Some reported advantages are small (e.g., Table 2, WebQA image QA: POQD 82.83 versus U-QD 82.26; Table 3, DeepSeek Top-2 retrieval: POQD 55.40 equals ICL-QD 55.40), so the claim that POQD "outperforms ... by a large margin" is not fully supported at the reported precision.
minor comments (6)
  1. [Section 5.1, Table 3 caption] The caption says "by using the Roberta model as the embedding model for retrieval," but the table compares GPT-4 and DeepSeek-V3; this caption appears to belong to Table 9 and is misleading.
  2. [Section 5.4, Table 4 caption] The caption repeats the Roberta-model text even though the table lists decomposed sub-queries; the caption should be corrected to describe the qualitative comparison.
  3. [Section 4.2, text after Algorithm 1] The prose says Algorithm 1 terminates if the updated-prompt loss is "at least smaller" by alpha, which contradicts Line 7's condition with the <= sign; the pseudocode and the surrounding text should be aligned, beyond the sign issue noted in the major comments.
  4. [Section 6] The sentence "based on a history of a history of past instructions and performance scores" contains a duplicated phrase and should be edited.
  5. [Section 2.2, Problem definition] The notation says the query is decomposed into n sub-queries {q_i}^K_{i=1}, mixing n and K; one symbol should be used consistently.
  6. [Assumption 4.1] The citation "(Liu et al.)" has no year and no specific theorem; the paper should point to the exact result that establishes the mu-PL* condition for the cited fine-tuning setting.

Circularity Check

1 steps flagged · score 6.0 of 10

Theorem 4.4's proof assumes the alpha-improvement at a parameter where it was never established, so the formal prediction of improved converged loss reduces to Algorithm 1's own acceptance threshold.

  1. other [Appendix A, proof of Theorem 4.4 (after Equation (6)); premise stated in Section 4.3 after Algorithm 2]
    "Recall that when pold is updated to pnew, the training loss is reduced by α. Therefore, the following formula holds: L(Θold τ ; pold) − L(Θold τ ; pnew) ≥ α, which can be integrated into Formula (6), resulting in: L(Θold τ ; pnew) + α − Gold = L(Θnew 0 ; pnew) + α − Gold ≤ Aτ (L(Θold 0 ; pold) − Gold)."

    Algorithm 1 only certifies the α-reduction at the current parameter Θ0 where the prompt search is run; no argument extends it to Θold τ, a hypothetical parameter after τ gradient steps with pold that Algorithm 2 never performs. The proof then asserts the equality L(Θold τ; pnew) = L(Θnew 0; pnew), which is generally false when Θold τ differs from Θnew 0. The theorem's bound Gold − Gnew ≥ α − Aτ M is obtained solely from this transplanted α-gap, so the formal 'prediction' of improved converged loss is not derived from the stated assumptions; it reuses Algorithm 1's acceptance condition as its own conclusion at a shifted parameter. The central theoretical claim in the abstract therefore reduces to the algorithm's stopping threshold rather than to independent theory.

full rationale

The empirical part is not circular: Hit@1/Hit@2 retrieval metrics and end-to-end QA accuracy are held-out measurements that are not the training objective optimized by prompt search, so the reported gains are genuine empirical comparisons rather than refitted predictions. The circularity is confined to the formal support. Section 4.3 explicitly acknowledges that the α-reduction from Algorithm 1 'may not necessarily guarantee decreased training loss at convergence,' and Appendix A's proof closes that gap only by reasserting the same α-reduction at Θold τ, where it has not been established. The subsequently asserted equality between L(Θold τ; pnew) and L(Θnew 0; pnew) is also not implied by Assumptions 4.1–4.3. Thus the theorem's conclusion is effectively an input condition of the algorithm restated at a different point, giving partial circularity in the theoretical prediction while the empirical claims remain independent. A separate sign inconsistency in Algorithm 1 Line 7 (the condition accepts L(Θ;p) − L(Θ;pold) ≤ α, while the text and proof require a reduction by α) is a correctness bug rather than a circularity, so it is not scored here.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central empirical comparison does not depend on heavy axioms, but the theoretical guarantee does. The free parameters alpha, tau and kappa are tuned on the same WebQA benchmark used in the main tables, introducing a mild selection effect. The PL*, smoothness, bounded-loss and gradient-descent assumptions are imported from LLM optimization theory and not verified for the specific RAG loss. The most fragile implicit assumption is that the LLM optimizer reliably produces a loss-reducing prompt; this is the event that makes Algorithm 2 progress and is needed by Theorem 4.4.

free parameters (3)
  • alpha (prompt acceptance threshold) = 0.02
    Tuned on WebQA text QA in Section 5.3; controls how much training-loss reduction is required before a new prompt is accepted in Algorithm 1.
  • tau (generator training steps per prompt update) = 3
    Tuned on WebQA text QA in Section 5.3; balances training cost and performance in Algorithm 2.
  • kappa (max prompt-search iterations) = 5
    Default termination bound in Algorithm 1; listed in Section 5.1 without a dedicated ablation.
assumptions (6)
  • domain assumption L(Theta; p) satisfies the mu-PL* condition for any fixed p
    Assumption 4.1; justified by citations to Liu et al. for LLM fine-tuning with GaLore, but not verified for the retrieval-weighted RAG objective in Equation (2).
  • domain assumption L(Theta; p) is L-smooth for any fixed p
    Assumption 4.1; needed for linear convergence in Lemma A.1 and Theorem 4.4.
  • domain assumption L(Theta; p) is bounded above by M for all Theta and p
    Assumption 4.2; used to bound the term (1 - mu/(2L))^tau M in Theorem 4.4.
  • standard math Model updates follow gradient descent with fixed learning rate eta = 1/L
    Assumption 4.3 and the proof of Lemma A.1; the real generator training uses optimizers such as GaLore, not exact gradient descent.
  • ad hoc to paper The LLM-based prompt optimizer can find p_new with L(Theta; p_new) <= L(Theta; p_old) - alpha
    Implicit in Algorithm 1's line 7 and used as a premise in the proof of Theorem 4.4; not proven, only observed in the reported empirical runs.
  • domain assumption Only the generator model Theta is trained while the retrieval model beta is frozen
    Section 3.2 states retrieval models are kept fixed to avoid index rebuilding; the method's improvement is therefore limited to query-side decomposition and generator updates.

how reviews work

0 comments
Cite this review

Pith. "Pith review of POQD: Performance-Oriented Query Decomposer for Multi-vector retrieval." pith.science (2026). https://pith.science/paper/HICDTAHD

@misc{pith2026250519189,
  author       = {Pith},
  title        = {Pith review of: POQD: Performance-Oriented Query Decomposer for Multi-vector retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HICDTAHD}},
  note         = {Machine review of arXiv:2505.19189}
}
read the original abstract

Although Multi-Vector Retrieval (MVR) has achieved the state of the art on many information retrieval (IR) tasks, its performance highly depends on how to decompose queries into smaller pieces, say phrases or tokens. However, optimizing query decomposition for MVR performance is not end-to-end differentiable. Even worse, jointly solving this problem and training the downstream retrieval-based systems, say RAG systems could be highly inefficient. To overcome these challenges, we propose Performance-Oriented Query Decomposer (POQD), a novel query decomposition framework for MVR. POQD leverages one LLM for query decomposition and searches the optimal prompt with an LLM-based optimizer. We further propose an end-to-end training algorithm to alternatively optimize the prompt for query decomposition and the downstream models. This algorithm can achieve superior MVR performance at a reasonable training cost as our theoretical analysis suggests. POQD can be integrated seamlessly into arbitrary retrieval-based systems such as Retrieval-Augmented Generation (RAG) systems. Extensive empirical studies on representative RAG-based QA tasks show that POQD outperforms existing query decomposition strategies in both retrieval performance and end-to-end QA accuracy. POQD is available at https://github.com/PKU-SDS-lab/POQD-ICML25.

Figures

Figures reproduced from arXiv: 2505.19189 by the authors.

Figure 1
Figure 1. Motivating example from ManyModalQA dataset (Hannan et al., 2020). We aim to answer the question “Victoria Hong Kong has many what type of buildings?” using retrieval-augmented generation (RAG). To enhance the retrieval accuracy and thus ensure the answer correctness, we employ Multi-Vector Retrieval (MVR), which decomposes the query into sub-queries and embeds them. MaxSim operations (as defined in (Khattab & Zahar… view at source ↗
Figure 2
Figure 2. Further analysis on the motivating example: the token “kong” is relevant to the photo of a black gorilla-like monster, which is mostly black. Coincidentally, in the photo of Lee Kuan Yew, the patch identified as the most relevant to the token ’kong’ is also mostly black. the fine-grained image patches, i.e., determine the similarity score between one token and its most similar image patch. Such similarity scores are… view at source ↗
Figure 3
Figure 3. The pipeline of generating decomposed sub-queries. This pipeline primarily consists of two LLMs, one serving as the Prompt Optimizer while the other one serving as the Query Decomposer. The prompt optimizer first takes two pieces of meta-prompts as well as solution-score pairs collected during the execution of Algorithm 1 as input and produces the optimized solution, which is an essential prompt prefix for the Query… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Overall training time of Algorithm 2 same default embedding model as others. For the generator models, we leverage Llama3.1-8B (Dubey et al., 2024) and Llava-v1.5-7B (Liu et al., 2024) as genera￾tors for single-hop text QA and image QA, respectively. In the experiments…
Figure 5
Figure 5. Figure 5: Overall inference time of POQD issue. In contrast, if α is too small (say α=0.01), POQD con￾verges much slower than our default with α=0.02. Hence, the default configuration of α, i.e., 0.02, can balance the con￾vergence speed and the final performance. In addition, wi…
Figure 7
Figure 7. Figure 7: Training time of POQD with varied values of τ for query decomposition. Specifically, this experiment is conducted by leveraging the GPT-4 model (Achiam et al., 2023) and DeepSeek-V3 (DeepSeek Team, 2024) as the query decomposer, which leads to the results in [PITH_FUL…
Figure 8
Figure 8. Figure 8: The prompt used for query decomposition in ICL-QD D.2. Comparison against the query rewrite strategy There is an emerging trend in the literature for rewriting user queries to maximize the RAG performance. Considering that both POQD and this line of work aim to manipul…
Figure 9
Figure 9. Figure 9: The prompt used for query decomposition in ICLF-QD text QA, which produces results in [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: Prompts produced in the first four iterations by Algorithm 1 19 [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: Retrieved images by using decomposed sub-queries produced by baseline methods 20 [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 26 canonical work pages

  1. [5]

    deepseek.com

    URL https://www. deepseek.com. Devlin, J. Bert: Pre-training of deep bidirectional trans- formers for language understanding. arXiv preprint arXiv:1810.04805,

    arXiv

  2. [6]

    We therefore further evaluate how this filtering step influences the performance of POQD as well as baseline methods

    End-to-end QA accuracy on the WebQA dataset in text QA by varying the number of retrieved relevant documents in the retrieval process Effect of the filtering step Considering that LLMs may produce irrelevant tokens during the query decomposition process, POQD filters out these irrelevant tokens in the generated sub-queries. We therefore further evaluate h...

    work page 2025

  3. [8]

    Coil: Revisit exact lex- ical match in information retrieval with contextualized inverted list

    Gao, L., Dai, Z., and Callan, J. Coil: Revisit exact lex- ical match in information retrieval with contextualized inverted list. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies, pp. 3030–3042,

    work page 2021

  4. [9]

    Dr3: Ask Large Language Models Not to Give Off-Topic Answers in Open Domain Multi-Hop Question Answering

    Gao, Y ., Zhu, Y ., Cao, Y ., Zhou, Y ., Wu, Z., Chen, Y ., Wu, S., Hu, H., and Dai, X. Dr3: Ask large language models not to give off-topic answers in open domain multi-hop question answering. arXiv preprint arXiv:2403.12393,

    work page Pith review arXiv

  5. [11]

    Dense passage retrieval for open-domain question answering

    Karpukhin, V ., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W.-t. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 6769–6781,

    work page 2020

  6. [12]

    Decomposed prompting: A modular approach for solving complex tasks

    Khot, T., Trivedi, H., Finlayson, M., Fu, Y ., Richardson, K., Clark, P., and Sabharwal, A. Decomposed prompting: A modular approach for solving complex tasks. In The Eleventh International Conference on Learning Repre- sentations. Khot, T., Khashabi, D., Richardson, K., Clark, P., and Sab- harwal, A. Text modular networks: Learning to decom- pose tasks i...

    work page 2021

  7. [14]

    Llm-based optimization of compound ai systems: A survey

    Lin, M., Sheng, J., Zhao, A., Wang, S., Yue, Y ., Wu, Y ., Liu, H., Liu, J., Huang, G., and Liu, Y .-J. Llm-based optimization of compound ai systems: A survey. arXiv preprint arXiv:2410.16392,

    arXiv

  8. [15]

    On the optimiza- tion landscape of low rank adaptation methods for large language models

    Liu, X.-H., Du, Y ., Wang, J., and Yu, Y . On the optimiza- tion landscape of low rank adaptation methods for large language models. In The Thirteenth International Con- ference on Learning Representations. Liu, Y ., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V . Roberta: A robustly optimized bert ...

    arXiv 1907

Show all 37 references
  1. [17]

    Unsu- pervised question decomposition for question answering

    11 POQD: Performance-Oriented Query Decomposer for Multi-vector retrieval Perez, E., Lewis, P., Yih, W.-t., Cho, K., and Kiela, D. Unsu- pervised question decomposition for question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Proce...

    2020

  2. [18]

    A., and Lewis, M

    Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N. A., and Lewis, M. Measuring and narrowing the composi- tionality gap in language models. In Findings of the As- sociation for Computational Linguistics: EMNLP 2023, pp. 5687–5711,

    2023

  3. [19]

    T., Zhu, C., and Zeng, M

    Pryzant, R., Iter, D., Li, J., Lee, Y . T., Zhu, C., and Zeng, M. Automatic prompt optimization with” gradient descent” and beam search. In The 2023 Conference on Empirical Methods in Natural Language Processing. Qi, J., Xu, Z., Shen, Y ., Liu, M., Jin, D., Wang, Q., and Huang...

    2023

  4. [20]

    Sentence-bert: Sentence embeddings using siamese bert-networks

    Reimers, N. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084,

    1908 arXiv

  5. [21]

    Plaid: an efficient engine for late interaction retrieval

    Santhanam, K., Khattab, O., Potts, C., and Zaharia, M. Plaid: an efficient engine for late interaction retrieval. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pp. 1747–1756, 2022a. Santhanam, K., Khattab, O., Saad-Falcon, J., P...

    2022

  6. [22]

    Multi- modalqa: complex question answering over text, tables and images

    Talmor, A., Yoran, O., Catav, A., Lahav, D., Wang, Y ., Asai, A., Ilharco, G., Hajishirzi, H., and Berant, J. Multi- modalqa: complex question answering over text, tables and images. In International Conference on Learning Representations. Wang, X., Li, C., Wang, Z., Bai, F., ...

    arXiv

  7. [24]

    and Zhu, X

    12 POQD: Performance-Oriented Query Decomposer for Multi-vector retrieval Yang, X. and Zhu, X. Exploring decomposition for table- based fact verification. In Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 1045– 1052,

    2021

  8. [25]

    Yang, Z., Qi, P., Zhang, S., Bengio, Y ., Cohen, W., Salakhut- dinov, R., and Manning, C. D. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Meth- ods in Natural Language Processing , pp. 2369–2380,

    2018

  9. [26]

    Offline training of language model agents with functions as learnable weights

    Zhang, S., Zhang, J., Liu, J., Song, L., Wang, C., Krishna, R., and Wu, Q. Offline training of language model agents with functions as learnable weights. In Forty-first Inter- national Conference on Machine Learning. Zhang, X., Wang, M., Yang, X., Wang, D., Feng, S., and Zhang...

    arXiv

  10. [27]

    Empowering large language model agents through action learning

    Zhao, H., Ma, C., Wang, G., Su, J., Kong, L., Xu, J., Deng, Z.-H., and Yang, H. Empowering large language model agents through action learning. arXiv preprint arXiv:2402.15809, 2024a. Zhao, J., Zhang, Z., Chen, B., Wang, Z., Anandkumar, A., and Tian, Y . Galore: Memory-efficie...

    2022 arXiv

  11. [28]

    Chain-of-questions training with latent answers for robust multistep question answering

    Zhu, W., Thomason, J., and Jia, R. Chain-of-questions training with latent answers for robust multistep question answering. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 8845–8860,

    2023

  12. [29]

    By leveraging the fact that L(Θold 0 ; pold) ≤ M and Gold > 0, then the above formula is further bounded by: Gold − Gnew ≥ α − Aτ M

    This thus can be leveraged to lower bound the left side of the above formula, leading to: Gnew − Gold + α ≤ Aτ (L(Θold 0 ; pold) − Gold). By leveraging the fact that L(Θold 0 ; pold) ≤ M and Gold > 0, then the above formula is further bounded by: Gold − Gnew ≥ α − Aτ M. B. Add...

    2013

  13. [30]

    Among these datasets, WebQA, MultiModalQA and ManyModalQA contain questions that may require retrieving data from different modalities

    and Text QA dataset (Berant et al., 2013). Among these datasets, WebQA, MultiModalQA and ManyModalQA contain questions that may require retrieving data from different modalities. Hence, for image QA, in particular, for WebQA, MultiModalQA and ManyModalQA dataset, we only selec...

    2013

  14. [31]

    Since ColBERT decomposes queries at the token level, which may not be suitable for decomposed sub-queries, we thus employ the Sentence-Bert model (Reimers,

    on the MSMARCO dataset (Nguyen et al., 2016). Since ColBERT decomposes queries at the token level, which may not be suitable for decomposed sub-queries, we thus employ the Sentence-Bert model (Reimers,

    2016

  15. [32]

    to embed individual tokens in documents, which is even impossible for images in the context of image QA. Hence, for text QA, except ColBERT, we embed individual sentences for other baseline methods and POQD while for image QA, we segment each image into superpixels with existi...

    2010

  16. [33]

    or employ heuristics (Jiang et al., 2022; Gandhi et al., 2022; Yang et al.,

    2022

  17. [34]

    (say syntax rules (Yang et al., 2023)), or perform in-context learning with LLMs (Li et al., 2024; Pereira et al., 2023; Niu et al., 2023; Ye et al., 2023; Xue et al.; Wu et al., 2024b; Chen et al., 2024; Bhattacharya et al., 2023; Liao et al., 2024; Khot et al.; Press et al.,...

    2023

  18. [35]

    for query decomposition. To further enhance the decomposition performance, one can optionally collect feedback on the decomposed sub-queries and incorporate it into the above in-context learning-based methods. Typical feedback includes confidence scores (Qi et al.), quality sc...

    2024

  19. [36]

    Comparison against the query rewrite strategy There is an emerging trend in the literature for rewriting user queries to maximize the RAG performance

    The prompt used for query decomposition in ICL-QD D.2. Comparison against the query rewrite strategy There is an emerging trend in the literature for rewriting user queries to maximize the RAG performance. Considering that both POQD and this line of work aim to manipulate the ...

    2023

  20. [2010]

    L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774,

    arXiv

  21. [2013]

    In-context ability transfer for question decomposition in complex qa

    Bhattacharya, S., Anand, A., et al. In-context ability transfer for question decomposition in complex qa. arXiv preprint arXiv:2310.18371,

    arXiv

  22. [2018]

    The llama 3 herd of models

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783,

    arXiv

  23. [2019]

    Query rewriting in retrieval-augmented large language mod- els

    Ma, X., Gong, Y ., He, P., Zhao, H., and Duan, N. Query rewriting in retrieval-augmented large language mod- els. In Proceedings of the 2023 Conference on Empiri- cal Methods in Natural Language Processing, pp. 5303– 5315,

    2023

  24. [2020]

    Latent reasoning for low-resource question generation

    Huang, X., Qi, J., Sun, Y ., and Zhang, R. Latent reasoning for low-resource question generation. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 3008–3022,

    2021

  25. [2021]

    Chen, J., Kim, G., Sriram, A., Durrett, G., and Choi, E

    URL https://arxiv.org/abs/2109.00590. Chen, J., Kim, G., Sriram, A., Durrett, G., and Choi, E. Complex claim verification with evidence retrieved in the wild. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: ...

    2024 arXiv

  26. [2022]

    and Callan, J

    Gao, L. and Callan, J. Condenser: a pre-training architecture for dense retrieval. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 981–993,

    2021

  27. [2023]

    A framework of knowledge graph-enhanced large language model based on question decomposition and atomic retrieval

    Li, Y ., Song, D., Zhou, C., Tian, Y ., Wang, H., Yang, Z., and Zhang, S. A framework of knowledge graph-enhanced large language model based on question decomposition and atomic retrieval. In Findings of the Association for Computational Linguistics: EMNLP 2024 , pp. 11472– 11485,

    2024

  28. [2024]

    Semantic parsing on freebase from question-answer pairs

    Berant, J., Chou, A., Frostig, R., and Liang, P. Semantic parsing on freebase from question-answer pairs. In Pro- ceedings of the 2013 conference on empirical methods in natural language processing, pp. 1533–1544,

    2013

  29. [2025]

    Search-in-the-chain: Interactively enhancing large lan- guage models with search for knowledge-intensive tasks

    Xu, S., Pang, L., Shen, H., Cheng, X., and Chua, T.-S. Search-in-the-chain: Interactively enhancing large lan- guage models with search for knowledge-intensive tasks. In Proceedings of the ACM on Web Conference 2024, pp. 1362–1373,

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.